Ralph Kraft1, Maxim Markevitch2, Caroline Kilbourne2, Joseph S. Adams2, Hiroki
Akamatsu3, Mohammadreza Ayromlou4, Simon R. Bandler2, Marco Barbera51,52,
Douglas A. Bennett5, Anil Bhardwaj6, Veronica Biffi7, Dennis Bodewits8, Ákos
Bogdán1, Massimiliano Bonamente9, Stefano Borgani7, Graziella Branduardi-
Raymont10, Joel N. Bregman11, Joseph N. Burchett12, Jenna Cann2, Jenny
Carter13, Priyanka Chakraborty1, Eugene Churazov14,15, Robert A. Crain16,
Renata Cumbee2, Romeel Davé17, Michael DiPirro2, Klaus Dolag18, W. Bertrand
Doriese5, Jeremy Drake1, William Dunn10, Megan Eckart19, Dominique Eckert20,
Stefano Ettori32, William Forman1, Massimiliano Galeazzi21, Amy Gall1, Efrain
Gatuzz22, Natalie Hell19, Edmund Hodges-Kluck2, Caitriona Jackman23, Amir
Jahromi2, Fred Jennings17, Christine Jones1, Philip Kaaret24, Patrick J.
Kavanagh23, Richard L. Kelley2, Ildar Khabibullin18,14, Chang-Goo Kim25,
Dimitra Koutroumpa26, Orsolya Kovács27, K. D. Kuntz28, Erwin Lau1, Shiu-Hang
Lee30, Maurice Leutenegger2, Sheng-Chieh Lin29, Carey Lisse31, Ugo Lo
Cicero52, Lorenzo Lovisari32,1, Dan McCammon33, Seán McEntee23, François
Mernier2,36, Eric D. Miller34, Daisuke Nagai35, Michela Negro2,37, Dylan
Nelson4, Jan-Uwe Ness38, Paul Nulsen1, Anna Ogorzałek2,36, Benjamin D.
Oppenheimer39, Lidia Oskinova40, Daniel Patnaude1, Ryan W. Pfeifle2, Annalisa
Pillepich41, Paul Plucinsky1, David Pooley42, Frederick S. Porter2, Scott
Randall1, Elena Rasia7, John Raymond1, Mateusz Ruszkowski11,14, Kazuhiro
Sakai2, Arnab Sarkar34, Manami Sasaki43, Kosuke Sato44, Gerrit
Schellenberger1, Joop Schaye45, Aurora Simionescu3,45,46, Stephen J. Smith2,
James F. Steiner1, Jonathan Stern47, Yuanyuan Su29, Ming Sun9, Grant
Tremblay1, Nhut Truong41, James Tutt48, Eugenio Ursino53, Sylvain Veilleux36,
Alexey Vikhlinin1, Stephan Vladutescu-Zopp18, Mark Vogelsberger34, Stephen A.
Walker9, Kimberly Weaver2, Dale M. Weigt23, Jessica Werk49, Norbert Werner27,
Scott J. Wolk1, Congyao Zhang50, William W. Zhang2, Irina Zhuravleva50, John
ZuHone1

Draft mission concept, March 2023

1 Center for Astrophysics $|$ Harvard & Smithsonian
2 NASA Goddard Space Flight Center
3 SRON, the Netherlands
4 Universität Heidelberg, Germany
5 NIST, Boulder, Colorado
6 Physical Research Laboratory, Ahmedabad, India
7 INAF, Trieste, Italy
8 Auburn University, Auburn, Alabama
9 University of Alabama, Huntsville
10 University College London, U.K.
11 University of Michigan
12 New Mexico State University
13 Leicester University, UK
14 MPA, Garching, Germany
15 IKI, Moscow, Russia
16 Liverpool John Moores University, U.K.
17 University of Edinburgh, U.K.
18 Ludwig-Maximilians-Univ. München, Germany
19 LLNL
20 University of Geneva, Switzerland
21 University of Miami, Florida
22 MPE, Garching, Germany
23 DIAS, Dublin, Ireland
24 University of Iowa
25 Princeton University
26 LATMOS, CNRS, France
27 Masaryk University, Brno, Czech Republic
28 Johns Hopkins University
29 University of Kentucky
30 Kyoto University, Japan
31 APL
32 INAF, Bologna, Italy
33 University of Wisconsin-Madison
34 MIT
35 Yale University
36 University of Maryland, College Park
37 University of Maryland, Baltimore County
38 ESA
39 University of Colorado, Boulder
40 Universität Potsdam, Germany
41 MPIA, Heidelberg, Germany
42 Trinity University, Texas
43 Friedrich-Alexander Univ. Erlangen-Nürnberg, Germany
44 Saitama University, Japan
45 Leiden University, the Netherlands
46 University of Tokyo, Japan
47 Tel Aviv University, Israel
48 Pennsylvania State University
49 University of Washington
50 University of Chicago
51 Università degli Studi di Palermo, Italy
52 INAF, Palermo, Italy
53 Purdue University, Fort Wayne

${{}^{5}2}$ lem-observatory.org

##  1 SUMMARY

The Line Emission Mapper (LEM) is an X-ray Probe for the 2030s that will
answer the outstanding questions of the Universe’s structure formation. It
will also provide transformative new observing capabilities for every area of
astrophysics, and to heliophysics and planetary physics as well. LEM’s main
goal is a comprehensive look at the physics of galaxy formation, including
stellar and black-hole feedback and flows of baryonic matter into and out of
galaxies. These processes are best studied in X-rays; as emphasized by the
2020 Decadal Survey, emission-line mapping is the pressing need in this area.
LEM will use a large microcalorimeter array/IFU (that builds on Athena XIFU
technology developments), covering a $30\times 30$′ field with 10′′ angular
resolution, to map the soft X-ray line emission from objects that constitute
galactic ecosystems. These include supernova remnants, star-forming regions,
superbubbles, galactic outflows (such as the Fermi/eROSITA bubbles in the
Milky Way and their analogs in other galaxies), the Circumgalactic Medium in
the Milky Way and other galaxies, and the Intergalactic Medium at the
outskirts and beyond the confines of galaxies and clusters. LEM’s 1–2 eV
spectral resolution in the 0.2–2 keV band will make it possible to disentangle
the faintest emission lines in those objects from the bright Milky Way
foreground, providing groundbreaking measurements of the physics of these
plasmas, from temperatures, densities, chemical composition to gas dynamics.
While the mission is optimized to provide critical observations that will push
our understanding of galaxy formation, LEM will provide transformative
capability for all classes of astrophysical objects, from the Earth’s
magnetosphere, planets and comets to the interstellar medium and X-ray
binaries in nearby galaxies, AGN, and cooling gas in galaxy clusters. In
addition to pointed observations, LEM will perform a shallow all-sky survey
that will dramatically expand the discovery space.

##  2 MISSION CONCEPT

The Line Emission Mapper (LEM) is an X-ray Probe concept aimed at one of the
fundamental unsolved problems of modern astrophysics — galaxy formation.
Galaxies are governed by the competition of gravity and the energetic
“feedback” from the forming stars and black holes. Theoretical studies predict
that key information on these processes is encoded in the tenuous
Circumgalactic Medium (CGM) and even more rarefied Intergalactic Medium (IGM),
which together contain most of the baryonic matter in the Universe3. Most of
that matter has so far eluded detection, much less detailed study. The
Astro2020 Decadal Survey has identified mapping CGM and IGM in emission as the
key missing observing capability and a major “discovery area.” LEM is designed
to provide this new capability in the soft X-ray band, where the dominant CGM
and IGM ion species emit (Fig. 1). While the importance of studying these
hidden baryons has been recognized for decades, only now has the technology
arrived that makes it possible.

Figure 1: Why soft X-rays? Predicted fraction of Oxygen ions in a CGM halo of
a Milky-Way mass galaxy (simulations using EAGLE model16; dotted line is the
virial radius; vertical scale is linear). Probing this circumgalactic gas is
critical to understanding the physics of galaxies. The temperature of the halo
is such that the dominant species over most of the halo is Ovii, whose
emission (along with that of Oviii) lies in the soft X-ray band — the energy
of the Oxygen He$\,\alpha$ resonant line is 574 eV. The lower ionization
species are observable in the UV, but modeling and extrapolation are required
to deduce the properties of the CGM. An X-ray instrument will probe the bulk
of the CGM directly, complementing the UV data on the cooler gas phases.

Figure 2: LEM mission design (under development in collaboration with Lockheed
Martin Corp.; a snapshot as of November 2022 is shown).

Figure 3: LEM effective area and grasp — the product of the effective area and
the field of view solid angle — compared to XRISM Resolve and Athena XIFU
(prior to 2022 reformulation). Grasp is the quantity relevant for mapping
faint extended objects with sizes comparable to or greater than the field of
view (such as nearby galaxies, clusters, the IGM, or the Milky Way structures)
— for such targets, the number of photons collected in a given exposure is
directly proportional to grasp.

LEM is a single telescope that consists of an X-ray mirror with a large
collecting area and moderate angular resolution, and a cryogenic
microcalorimeter array (Fig. 2). Both are optimized for the 0.2–2 keV energy
band, where the most abundant CGM/IGM ion species have multiple bright
emission lines. Because the CGM and IGM are diffuse sources, mapping them
requires a nondispersive imaging spectrometer, or an integral field unit (IFU)
as they are known in other wavebands. (Chandra HETG and LETG, XMM-Newton RGS
and the ARCUS concept are dispersive spectrometers whose full spectral
resolution is available only for point sources.) Furthermore, as will be seen
below, our own Milky Way places a bright screen in front of the very faint IGM
and CGM around other galaxies. Removing this foreground requires eV-class
spectral resolution. These two conditions require a microcalorimeter. Such
imaging X-ray calorimeters have not been flown before, with the exception of
the short-lived Hitomi, to be replaced in 2023 by XRISM. LEM will provide a
leap beyond XRISM, and surpass in relevant capabilities other missions planned
for the more distant future. Table 1 compares LEM to future X-ray
microcalorimeters — namely, XRISM Resolve, Athena XIFU (to be launched in late
2030s) and HUBS (under study). The effective area and “grasp” as functions of
energy for LEM, XRISM and Athena are compared in Fig. 3. We note that Athena
is currently undergoing redefinition and as of this writing its final
instrument parameters are unknown. The above comparisons are done with the
pre-2022 Athena concept.

The LEM concept is being prepared for the NASA 2023 Astrophysics Probes call
for proposals. If selected, LEM will be launched in 2032.

|  LEM |  XRISM Resolve |  Athena XIFU∗ |  HUBS
---|---|---|---|---
Energy band, keV |  0.2–2 |  0.4–12 |  0.2–12 |  0.2–2
Effective area, cm2 0.5 keV |  |  |  |
6 keV |  1600 |  |  |
0 |  50 |  |  |
300 |  6000 |  |  |
2000 |  500 |  |  |
0 |  |  |  |
Field of view |  30′ |  3′ |  5′ |  60′
Grasp, $10^{4}$ cm2 arcmin2 0.5 keV |  140 |  0.05 |  12 |  180
Angular resolution |  15′′ |  75′′ |  5′′ |  60′′
Spectral resolution |  1 eV (central 7′), |  |  |
2 eV (rest of FOV) |  7 eV |  2.5 eV |  2 eV |
Detector size, pixels (equiv. square) |  118$\times$118 |  6$\times$6 |  50$\times$50 |  60$\times$60
Table 1: LEM compared with the requirements for the future X-ray nondispersive
spectrometers onboard XRISM (JAXA/NASA), Athena (ESA/NASA) and HUBS (Chinese
Space Agency). For XRISM, based on Hitomi, the actual spectral resolution is
expected to be 5 eV. The main advantages of LEM are its large grasp and high
spectral resolution. ∗For Athena, pre-reformulation requirements are listed.
Figure 4: Point Spread Function measurement for a subset of the mirror
consisting of 5 pairs of reflective shells. The image is elongated because it
comes from an incomplete mirror. A 2.8′′ HPD is achieved for this subset; the
requirement is 10′′ HPD (or, equivalently, for 90% of the photons to fall in
the 15′′ detector pixel) for the whole LEM mirror that will combine
approximately 3500 shell pairs. Figure 5: Spectral resolution for a prototype
LEM detector array24, measured for the single-absorber pixels (the central
array) and the 4-pixel hydras (the rest of the detector). The measurements
were performed at $E=1.5$ keV. The blue dotted line is the intrinsic Al line
profile (arbitrary normalization) and the black line is the model that
includes detector broadening. The performance already nearly meets the LEM
requirements.

###  2.1 X-ray mirror I

The grazing-incidence mirror will consist of many pairs (for two reflections)
of thin monocrystalline silicon shells, coated with either Ir or Pt to provide
X-ray reflectivity. The mirror’s 1.5m diameter and 4m focal length are
selected to provide a large effective area at the energies of interest ($E<2$
keV) and a broad angular field of view, covering a Milky Way like galaxy at
$z=0.01$ (40 Mpc), to fit within the detector of a feasible linear size. This
choice of geometry means that the effective area at higher energies is low.
The mirror is being developed at NASA GSFC. The angular resolution
requirements imposed by the main science drivers is modest, 10′′ half-power
diameter (HPD) or 15′′ square pixel enclosing 90% energy. A resolution of
2.8′′ HPD has already been demonstrated in the lab for a small subset of the
mirror (Fig. 4). The mirror is designed to have a constant PSF and
insignificant vignetting across the 30′ field of view.

###  2.2 Detector

The LEM detector is a cryocooled array of Transition-Edge Sensor (TES)
microcalorimeters. It will be built at NASA GSFC using the technology
developed for Athena XIFU and the Lynx concept by the same team9, 11. The
array will consist of 13,806 absorber pixels (equivalent of $118\times 118$
square) with a 290 $\mu$m pitch, in hexagonal arrangement. For a 4m focal
length, it will cover a solid angle equal to a 29.4′ square.

The inner 1062 absorber pixels (a $7^{\prime}\times 7^{\prime}$ square array)
will each be connected to a single TES and provide an energy resolution of 1
eV (FWHM). The rest of the array will consist of “hydras,” where $2\times 2$
absorber pixels are connected to a single TES, which will determine which
absorber was hit with a photon based on the different time constants for the
different absorbers23. These pixels will provide a resolution of 2 eV (FWHM)
for $E<2$ keV. The absorber thickness will be optimized for the low X-ray
energies and the sensor will take advantage of the narrow energy band of the
mirror. Figure 5 shows the measured line response for a prototype LEM array24,
which already nearly meets our requirements both for the single pixels and the
hydras. The position discrimination within the hydra has also been
demonstrated at least for $E>300$ eV.24

LEM will use the IR and optical blocking filters developed for Athena XIFU,
scaled up in size for a wider viewing cone of the LEM mirror.

The calorimeter array is cooled to the $35-50$ mK operating temperature by the
Adiabatic Demagnetization Refrigerators built at NASA GSFC and the cryocooler
built by Lockheed Martin.

A modulated X-ray source will operate in orbit and illuminate the array with a
1.5 keV Al K$\alpha$ near-monochromatic line for continuous calibration of the
energy response. The resulting accuracy on the pixel energy gain, 0.25 eV at
1.5 keV (and proportionally lower at lower energies), corresponds to a 50 km/s
accuracy (1$\sigma$) on the relative velocity measurements using the Doppler
shift.

###  2.3 Spacecraft and orbit

The spacecraft will be built by Lockheed Martin Corp. The main mode of
operations will be long pointed observations. LEM will also be able to slew to
perform efficient scans of large areas of the sky — such as the whole sky
(§3.2). While LEM is not a time-domain mission and not required to have a fast
response capability, it will be able to respond to the highest-profile events
on a several-day timescale from first notice to pointing — similar to Chandra.
The area of the sky accessible at any particular moment will be approximately
half the sky; this fraction is currently under study.

The orbit for LEM is currently under study. The Sun-Earth Lagrange points L1
and L2 would provide a stable thermal environment for the microcalorimeter and
the high observing efficiency to take full advantage of the 5 yr mission
minimum lifetime. A possibility to extend the field of regard to include
observations of the Earth’s magnetosphere from the L1 orbit is being
considered.

##  3 SCIENCE DRIVERS

Below is a brief description of the science investigations that will be
enabled by LEM. They will be presented in more detail in several forthcoming
papers.

###  3.1 Physics of galaxy formation

Galaxies form from the collapse of primordial matter density fluctuations
under the effect of gravity, which is counteracted by feedback from stars and
the central supermassive black holes that pushes the gas out of the galaxy.
The result is a circulation of gas and metals (synthesized by stars and
supernovae in galaxies) into and out of galaxies — a galactic ecosystem.
Understanding this ecosystem is key to understanding galaxy formation. Quoting
Astro2020, “the key missing link in unveiling the physics driving galaxy
growth is to measure the properties of the diffuse gas within, surrounding,
and between galaxies,” and these three locations constitute the main LEM
science drivers.

Figure 6: Predicted brightness profiles of the Ovii emission for Milky Way
mass galaxies from three cosmological simulations (TNG13, EAGLE22, 5, Simba6).
Ovii is abundant at $T=(0.3-2)\times 10^{6}$ K that is characteristic of CGM.
Each curve is a median profile for 50 simulated galaxies. The expected LEM
counts are shown for a 2 eV wide interval centered on the forbidden line of
the Ovii triplet at $z=0.01$. $R_{200}\simeq 1.5R_{500}$. These numerical
models include different physical prescriptions for feedback, and the
resulting predicted X-ray brightness at large radii differs by over an order
of magnitude. The CGM extent and spatial structure are sensitive diagnostics
of the feedbak physics; detecting and mapping this emission will strongly
constrain the physical processes shaping the galaxies.

Figure 7: Simulated LEM spectrum of the CGM of a galaxy twice as massive as
the Milky Way (from TNG simulations13), at $z=0.01$ or 40 Mpc, for a 1 Ms
exposure and the 2 eV resolution of the full detector. Blue line shows the
spectrum of the foreground, red line includes the CGM signal. The spectrum is
extracted from a broad annulus between $0.25-0.75\,R_{500}$. Point sources
detected with the LEM angular resolution are excluded. Because the Milky Way
emits in the same spectral lines but is over an order of magnitude brighter,
it is impossible to disentangle the two with the present CCD instruments such
as Chandra and XMM-Newton (green line). A nondispersive spectrometer with 2 eV
resolution allows the study of the faint CGM for galaxies at $z\gtrsim 0.01$,
where they can also be well-resolved spatially.

Figure 8: Simulated Milky Way-like galaxy at $z=0.01$ (TNG5018, 19, 14 of the
IllustrisTNG simulation suite). Panels are 30′ with 15′′ pixels (LEM field of
view and angular resolution). The ellipse shows the size of the optical disk,
seen edge-on. Left: Soft X-ray image with a CCD-like (70 eV) spectral
resolution centered at the Oviii line, including the Milky Way foreground and
CXB (the brightest sources accounting for half of CXB flux have been spatially
removed). The bright foreground makes it impossible to detect extended CGM
emission. Middle, Right: LEM 1 Ms images of the same galaxy, but in 2 eV wide
bins (the LEM spectral resolution) centered on the Oviii and Fexvii CGM
emission lines. The MW foreground is almost completely resolved away, taking
advantage of the galaxy’s known redshift; the use of the narrow bin also
removes most of the residual power-law CXB flux. LEM unveils the rich spatial
structure of the line-emitting CGM halo — including the analog of the
Fermi/eROSITA bubbles produced by the central SMBH. Mapping different emission
lines provides a measurement of the plasma temperature.

Figure 9: Simulated LEM spectrum of the Intergalactic Medium (IGM) in the
Cosmic Web. Owing to its high spectral resolution and large grasp, LEM will be
able to detect emission from metals in the denser regions of the Cosmic Web
filaments, outside the confines of galaxies (and their CGM halos) and galaxy
clusters. It is a diagnostic of galaxy formation processes at early epochs.
Filaments will be selected from galaxy surveys and the redshift of the IGM
will be known, i.e., it is not a blind search for emission lines. The image
shows Oviii emission in cosmological simulations (TNG10012). The sample
spectrum corresponds to a typical bright region of an IGM filament near a
massive cluster (at a distance $r\sim 2R_{200}$ from the cluster), excluding
halos of all galaxies and clusters. The eV spectral resolution is essential
for separating the faint IGM lines (highlighted by red) from the much brighter
Milky Way foreground (cf. the green line that shows the same spectrum at CCD
resolution).

CGM. At the mass scale of the Milky Way, the observable galaxy properties
exhibit a dramatic change — from star-forming disks thought to be fed by cold
gas streams, to passive ellipticals surrounded by hot hydrostatic gas.
Apparently, this reflects the shift in the balance of the various galaxy-
shaping physical processes. Galaxies around this mass should thus offer a
particularly sensitive probe of those processes. Indeed, cosmological
simulations of the MW-mass galaxies that include different physics make
strikingly different predictions for the X-ray properties of the outer regions
of the circumgalactic halos (Fig. 6). At present, these predictions are
entirely unconstrained by observations, and detecting and measuring this X-ray
signal can tell us which physical processes shape the CGM.

LEM will be able to map the CGM (defined here as the gaseous halo within the
galaxy virial radius $R_{200}$) in the emission lines of the dominant ion
species, Ovii, Oviii and Fexvii. The high spectral resolution will allow us to
disentangle the faint signal from the CGM of external galaxies from the
overpowering Milky Way foreground using the galaxy’s redshift. Because the
Milky Way itself has a CGM halo that emits in the same atomic transitions —
and we are immersed in its bright central region and see that emission across
the sky — the ability to separate the redshifted extragalactic signal by
energy is a critical advantage of LEM, as illustrated in Fig. 7. The current
X-ray CCD detectors lack spectral resolution for this approach.

For the Milky Way mass halos, LEM will be able to map the CGM emission for
individual galaxies (as opposed to stacking the radial profiles of many
galaxies) in the brightest emission lines out to $R_{500}$ and beyond, and
derive the distribution of gas temperature, density, pressure and elemental
abundances out to a large fraction of $R_{500}$. The maps are expected to
reveal a wealth of spatial structure, possibly including the extragalactic
analogs of our own Milky Way’s Fermi bubbles (Fig. 8) — if black hole / AGN
feedback is as important as theory predicts. Such measurements are only
possible because the Milky Way foreground can be separated using the redshift
of the signal. The minimum redshift for such separation for the 2 eV detector
resolution is $\sim 0.01$ (Fig. 7); at this redshift, the $R_{500}$ for a
Milky Way mass galaxy fills the LEM FOV and the whole galaxy can be mapped in
one deep pointing. The LEM angular resolution will allow efficient spatial
masking of the background point sources and resolving the interesting linear
scales in the halo (e.g., shocks around the Fermi bubbles). A typical CGM halo
is optically thin for the emission lines of interest over most of its volume
except for the very central regions, where the optical depth for the Ovii
resonant line becomes significant. LEM will easily resolve the components of
the Ovii He$\alpha$ triplet (where the resonant line is affected by scattering
while other components are not), which will enable interesting physical
diagnostics of the gas.

Using the high-resolution (1 eV) central array, LEM will be able to
simultaneously map the velocities in the brighter central regions of the
galactic halos. It will detect gas bulk velocities and velocity dispersion
predicted for the particularly energetic modes of feedback.

Figure 10: Left: Sample spectrum from one pixel in the LEM calorimeter from a
50 ks exposure of the Puppis A supernova remnant. Right: LEM image of Puppis
A. The white square shows one LEM FOV and the circle shows the Athena XIFU
FOV. LEM will map such large SNRs very efficiently, which is especially
valuable for old, large Galactic SNRs. Each LEM pixel will provide an
extremely line-rich spectrum, resolving multiple lines for each ion, allowing
us to map chemical composition, diagnose nonequilibrium conditions in the
plasma and measure velocities of ejecta. A CCD spectrum (Chandra ACIS) is
shown for comparison; all spectral features are blended. A dispersive
spectrometer (gratings) provides very limited spectral resolution for such
extended sources because of spatial blending.

IGM. An even more challenging observing target is the IGM, which we define as
the diffuse matter outside $R_{200}$ of any galactic or cluster halos. The
warm-hot phase of it ($>10^{5}$ K) is known as WHIM. It is the ultimate
repository of everything expelled from the galaxies over their lifetime, and
its metal content thus encodes the history of the feedback processes in the
Universe. The IGM regions immediately surrounding massive hot galaxy clusters,
where the gas temperature is still above $10^{7}$ are accessible for sensitive
low-background CCD instruments, which could use the emission at $E>1$ keV.
However, most of the IGM has lower temperatures, and its emission will be
flooded by the bright Milky Way. LEM will be able to detect the brightest
emission lines from the dense regions of the Cosmic Web filaments that connect
galaxy clusters, at distances $2R_{200}$ from the clusters and beyond.

We will use optical galaxy redshift surveys to select such filaments at an
optimal redshift ($z=0.07-0.08$) to be able to see the most promising IGM
emission lines through the Milky Way line forest (Fig. 9) and survey these
regions with multiple deep pointings. This is where the large grasp is
critically important, as the number of photons collected from a very extended
source is directly proportional to it — and the IGM is extremely faint. While
we will not be able to detect the continuum emission from the IGM (it is too
far below the background), we will detect the faint emission lines of Ovii,
Oviii and Fexvii (if, that is, the IGM is indeed enriched, as the current
theory predicts), and compare the brightness in these lines as a function of
the galaxy number density with cosmological simulations, in order to determine
when the enrichment of the IGM by galactic ejecta has occurred and what
physical processes were responsible.

LEM’s large grasp and good angular resolution will allow us to detect the IGM
in absorption in the cumulative spectra of the CXB sources in the same fields
where we will search for the IGM emission. A comparison of emission and
absorption for the same filament will allow us to derive the physical
properties of the filament gas.

The line emission from the low-density IGM can be significantly enhanced (in
the resonant transitions only), compared to intrinsic thermal emission, by
resonant scattering of the CXB photons, if the CXB point sources are excised
from the image8. This improves the detectability of the IGM emission. This
also applies to the outer regions of the CGM, with a significant additional
enhancement from the resonant scattering of the photons from the galaxy’s own
bright central region. This will extend the radius at which the CGM could be
detected by LEM. Of course, interpretation of the detected signal will require
modeling of the scattered contribution; LEM will be well equipped for such
spectral diagnostics because it will resolve the resonant and forbidden
transitions for the relevant ions.

Abundances in the ICM. LEM will address the cumulative effect of the galactic
feedback from another angle. The ratios of the various elements in the ICM in
the outskirts of galaxy clusters, as well as the total metal content of the
ICM and the spatial distribution of metals in the ICM, depend on the prevalent
star populations and types of supernovae that generated those metals, when it
happened, and how the metals mixed into the intergalactic gas10. LEM will map
the abundances and abundance ratios for O and Fe, but also for a number of
less-abundant elements that can be used as sensitive diagnostics for the
sources of chemical enrichment. LEM’s large grasp and good angular resolution
will allow us to efficiently and completely map the abundances of all the
interesting elements in nearby groups and clusters in the interesting radial
range $0.5R_{500}-R_{200}$.

Velocities of the ICM. LEM will efficiently and completely map the gas
velocities and velocity dispersions in nearby clusters and groups, relaxed and
dynamic, out to $R_{200}$ (using mostly the Fe-L lines). It will determine
where the clusters are hydrostatic and where the contribution of turbulence
and gas bulk motions into the total energy budget becomes important. This
information is key for our understanding of the physics of the large scale
structure and cluster formation.

Figure 11: LEM shallow all-sky survey. The all-sky map20 in Galactic
coordinates overlays soft X-ray emission from SRG/eROSITA (cyan) and
$\gamma$-ray emission from Fermi (magenta). Top: Example simulated spectra for
$10^{\circ}\times 10^{\circ}$ regions (marked by squares) and 100s depth are
shown for an average high-latitude region outside of any bright features
(“extragalactic field”), and another for the North Polar Spur; the CXB is
included. Purely thermal CIE plasma models are assumed. Red curve shows the
NPS spectrum at CCD resolution (e.g., eROSITA). LEM will resolve the forest of
lines and gain access to line diagnostics for temperature, nonequilibrium and
charge exchange processes. The survey will allow us to map the temperature
structure and velocities of the Milky Way inner CGM on few-degree scales —
including the expansion of the eROSITA/Fermi bubbles, believed to be produced
by the central SMBH or star-forming regions. Numerous other investigations,
some of which are labeled in the figure, will become possible.

Stellar feedback. LEM will study the gas inside the Milky Way that is part of
the galactic ecosystem. It will map the chemical composition and velocities of
ejecta in well-resolved supernova remnants Fig. 10). It will also map the
physical conditions in the Milky Way and Local Group superbubbles — regions
where stellar winds from many massive stars and the supernovae combine and
form large cavities in the ISM filled with hot plasma. If superbubbles break,
the hot plasma can escape from the galactic disk, forming galactic outflows or
fountains and flowing into the CGM. This is the mechanism by which the stellar
feedback in galaxies operates. Mapping the X-ray line emission will offer
diagnostics of gas out of collisional ionization equilibrium (rapidly
heating/cooling) and separating thermal emission from the charge exchange
emission that arises from the mixing of cold and hot gas phases. With its
large grasp, LEM will be able to efficiently map very extended structures as
Orion-Eridanus and other nearby superbubbles.

###  3.2 All-Sky Survey I

As described above, the IGM and the CGM of external galaxies are observed
through the forest of bright Milky Way emission lines. It will be helpful to
find areas in the sky where this foreground is less bright, at least in
certain problematic lines such as Fexvii and Fexviii. (This is similar to
Hubble peering into the Lockman Hole to observe high-$z$ galaxies.) The
eROSITA and HaloSat sky surveys will be helpful, but lack spectral resolution
to detect faint emission lines. LEM has sufficient grasp to perform its own
shallow all-sky survey in a relatively modest total exposure — e.g., an all-
sky coverage with a useful 10s depth will require 1.6 Ms. We envision
performing such shallow pathfinder survey very early in the mission to
identify areas in the sky best suited for deep CGM/IGM exposures and for other
interesting studies, and add up to 10 times more depth over the mission
lifetime.

In addition to the above technical purpose, the first-ever all-sky survey with
an imaging calorimeter will open enormous discovery space. With LEM’s large
effective area, even a shallow depth will be sufficient for groundbreaking
studies. The statistical quality and richness of the spectra for typical low-
brightness and high-brightness regions of the sky for the eventual full depth
are illustrated in Fig. 11. The figure also lists examples of investigations
using all-sky or partial-sky data that we can envision at present.

Some important aspects of the feedback processes in Milky-Way mass galaxies
would be best viewed from inside the Milky Way. LEM will map the velocities of
the inner regions of the Milky Way CGM, and in particular, the expansion of
the Fermi/eROSITA bubbles, believed to be evidence of feedback from either the
SMBH or star-forming regions in the Galactic Center. It will map the
temperature structure of the inner CGM across the sky and along the line of
sight using lines of the various ion species — something only a calorimeter
can do, in the presence of multiple temperature components and solar wind
charge exchange on each line of sight. The MW observations will complement the
studies of CGM in other galaxies (§3.1), where LEM will map the outer halos
but have limited insight into the interface between the disk and the halo,
where the exchange of mass and metals with the outer CGM takes place.

The emission of the lower MW halo (thought to be formed from the exhaust of
Galactic chimneys) can be separated from the emission of the Galactic disk ISM
by observing a dense absorbing extraplanar cloud, and determining the spectra
of the foreground and background to the cloud. LEM’s large FOV and high
spectral resolution will allow the first definitive separation of the disk and
halo spectra for individual plasma diagnostics.

LEM’s sensitivity and spectral resolution at low energies will let us
unambiguously separate the the heliospheric solar wind charge exchange (SWCX)
line emission from thermal emission of the Local Hot Bubble (LHB) and the
Galactic halo. This will let us explore the physics of the LHB. The North
Polar Spur is another enigmatic structure in the Milky Way sky, possibly
projected onto the northern Fermi bubble (Fig. 11); its spectrum is known (at
CCD resolution) to be very different from the rest of the sky above the
Galactic plane. A survey with LEM will provide the plasma temperature and
velocity map for this feature, as well as any possible contribution from the
charge exchange mechanism, and may explain its nature.

The Fermi bubbles (as distinct from the eROSITA bubbles that surround them,
Fig. 11) contain cosmic ray electrons that generate the synchrotron “microwave
haze” observed by WMAP and Planck and the inverse Compton $\gamma$-ray
emission discovered by Fermi1. The synchrotron radiation generated by the
highest-energy end of this population may be detectable in the soft X-ray band
covered by LEM. LEM is a very sensitive instrument for detecting faint
continuum emission at $E<1$ keV, because it can resolve away the emission
lines that dominate the soft X-ray sky. LEM will have the requisite grasp to
map the entire Fermi bubbles as part of the all-sky survey and yield
constraints on the X-ray synchrotron emission from the southern Fermi bubble.
LEM upper limits — or a positive detection — will provide insights into the
energetics of the Fermi bubbles.

Figure 12: LEM will be able to spatially separate and perform spectroscopy of
the ISM and the X-ray binaries in nearby spiral galaxies, such as M33. A
simulated 20 ks LEM image (based on the existing XMM-Newton observation) and a
sample ISM spectrum are shown. LEM will detect many binaries and probe their
variability. It will also produce detailed maps of the line-dominated ISM
emission. For such nearby galaxies, the Milky Way foreground cannot be
separated using the redshift, but its contamination for most ISM lines is
under 10%. In each observation, LEM will provide the same amount of imaging
information as XMM-Newton, but 20–30 times more spectral information.

###  3.3 Observatory science I

In addition to the main science drivers described above (which will be
addressed during the 30% of the mission observing time allocated to the
science team), LEM will transform all areas of X-ray astrophysics. It is
useful to think of LEM as an XMM-Newton with a calorimeter — its imaging
capabilities (FOV, angular resolution, effective area) are similar in the
$E<2$ keV band to EPIC’s, while its spectral resolution is 50 times better.
Many studies will be serendipitous to the long CGM and IGM exposures, while
others will be proposed by general observers (GO) for the remaining 70% of the
observing time. Many of them will need only modest exposures. An incomplete
sample of possible novel short investigations is given below.

Cooling, AGN feedback, gas mixing in cool cores of galaxy clusters. Where is
the coolest gas in the Perseus cluster core and how cool does it get? LEM will
use its 1 eV resolution in the central array to derive the temperature
distribution for the cooling gas. It will also map the charge exchange
emission that may come from the interface between the hot ICM and the
molecular gas nebula. If such emission is detected and spatially correlated
with the cold nebula (for which at least a LEM-like angular resolution is
required), it would provide invaluable insight into the process of gas mixing
— a critical piece of the puzzle of the energy balance in the cluster cool
cores.

Power spectrum of turbulence in cool galaxy clusters and groups using the
velocity structure function. Because of its large grasp, LEM will be able to
cover the entire cluster in a short exposure. The full spatial coverage
greatly increases the accuracy of the structure function and the power
spectrum measurement.26 A power spectrum of turbulence will provide the
measure of the energy flux from the cluster mergers down the turbulent cascade
to the dissipation scale.

Protoclusters at high z. Deep exposures toward IGM and CGM fields will return
significant numbers of serendipitous distant galaxy clusters, for which LEM
will be able to determine the redshift using its own X-ray spectra. Dedicated
surveys can be also devised. LEM will derive chemical abundances for high-$z$
clusters and constrain the cosmological enrichment history.

High-z AGN. Every LEM deep field will have multiple AGN at $z>2.5$ with a
detectable Fe K$\alpha$ line. LEM will measure the profiles of these lines to
study broad-line region clouds and accretion disk reflection. It will study
heavily obscured AGN and expand significantly upon the AGN science derived
from current Chandra and XMM-Newton deep fields.

AGN outflows and feedback. With its superior energy resolution, LEM will be
able to precisely map fluorescent and photoionization lines and decouple AGN
signatures in emission and absorption from ionization by stellar sources to
understand the role that AGN outflows and feedback play in galaxy evolution.

Figure 13: LEM will map the process of galaxy stripping and mixing of the
stripped gas in detail. A 100 ks LEM exposure of interacting galaxies at
$z=0.01$ (TNG100 simulations25) — an $M_{\rm tot}=2\times 10^{11}$ $M_{\odot}$
galaxy (at the panel center, with contours showing optical light) after the
passage through a group-mass halo (at the lower edge of the panels). The CXB
is included, except for its brightest point sources, which are removed. Right:
LEM image in a narrow 3 eV interval at the redshifted Ovii line (the forbidden
component of the triplet, which at this redshift can be resolved from the
corresponding Milky Way foreground line as seen in Fig. 7) reveals a
“jellyfish” — a trail of cool gas ram-pressure stripped from the infalling
galaxy. The hotter group halo does not emit in this line. Such gas tails are a
sensitive diagnostic of the plasma physics in galaxy groups and clusters.

Figure 14: LEM will revolutionize solar system studies. (a) XMM-Newton EPIC-pn
spectrum of Jupiter’s Northern X-ray aurora (130 ks). There are two different
models that fit the data well, but each offers fundamentally different
explanations for the processes that produce Jupiter’s aurora. The first model
is a solar wind charge exchange model (SWCX), implying entrance of solar wind
into the Jovian system. The second model is an Iogenic model, which uses
charge exchange on the S and O ions from Io’s volcanos. (b,c) LEM 1 eV
resolution spectra for each scenario. Counts per resolution element for a 20
ks exposure is shown; background is not included for clarity (it is negligible
because Jupiter is under 1′ in size). The most prominent emission lines are
marked. The instrument may offer significant sensitivity at $E<0.1$ keV with
the same high spectral resolution, providing access to never-before-seen
charge exchange lines at low energies (an option currently under study; gray
area in the plots). LEM will easily distinguish between these models and
address key questions on if and when Jupiter’s magnetosphere is open to the
solar wind — a point of longstanding debate in the heliophysics and planetary
science communities.

Hot ISM and X-ray binaries in nearby galaxies. In face-on spirals, LEM will
map the ISM dominated by emission lines and isolate the X-ray binaries (XRB)
spatially and using their time variability. In nearby spirals such as M33
(Fig. 12), LEM will be able to study XRB populations and star-forming regions
that are generally not detected in the Galaxy due to obscuration. We will be
able to apply plasma diagnostics to determine whether the gas in and around
star-forming regions is in CIE, or whether it has lost a significant amount of
energy in expansion, while remaining in a strongly overionized state. This
determination will be pivotal for understanding the formation of ISM
superbubbles.

In accreting stellar-mass black hole systems, relativistically-broadened O-K
and Fe-L lines are fluoresced off the surface of the accretion disk. These
features can be used to constrain the accretion disk inner radius and/or black
hole spin.

Jellyfish galaxies. LEM will map the process of ram-pressure stripping of
galaxies traveling through the gas in groups, clusters and the Cosmic Web
filaments, and of galaxies undergoing collisions with other galaxies. This is
an important mechanism by which the ICM and IGM are enriched by metals. LEM
will be particularly well-equipped to detect the stripped gas and study its
mixing with the ICM — a process that strongly depends on plasma physics and
such properties as plasma viscosity, thermal conduction and the structure of
its magnetic fields.

Figure 13 illustrates how LEM will use its energy resolution to separate the
cool stripped gas, whose emission is dominated by certain lines, from the
hotter ICM, revealing the morphology of the stripped tail, which strongly
depends on plasma properties21. Furthermore, as stripping can bring neutral
gas from the galaxy in direct contact with the hot ICM, one can in principle
expect the charge exchange emission from their interface, with its tell-tale
line signatures in the LEM spectra. This is a completely unexplored area that
holds great promise for probing the ICM plasma physics. LEM will also study
the effect of galaxy collisions and ram pressure stripping on ISM and star
formation.

Shocks in outskirts of massive galaxies. As noted above (§3.1), the emission
from the outer regions of the CGM of massive galaxies may be enhanced by
resonant scattering of the CXB photons8 and of the photons from the galaxy’s
own bright central emission. This “illumination” in the resonant Ovii line may
help LEM detect and map the outer reaches of the galactic halos, where Ovii
should be abundant but the intrinsic thermal emission is too faint. It is at
those virial regions that we expect to find shock fronts from galaxy/galaxy
mergers and matter infall. Recently, a radio phenomenon was discovered that
may be caused by those shocks (“odd radio circles,” or ORCs, around massive
galaxies at several hundred kpc from their centers15). LEM is well-equipped
for detecting such shock features via their associated density jumps in
thermal plasma, highlighted by the resonantly scattered light. This will
uncover an additional piece of the galaxy formation puzzle. Such measurements
will have to be done at $z>0.035$ in order to move the Ovii resonant line from
behind the MW Ovii triplet.

Light echo on giant molecular clouds in the Galactic Center. The SMBH in the
center of our Galaxy is quiescent at present, but may have been more active in
past several hundred years. The light echo from past explosions will be seen
with a time delay in the fluorescent line emission from the GC molecular
clouds.4 While the bright Fe fluorescent line at 6.4 keV is inaccessible for
LEM, we will be able to detect such light echo in the Si fluorescence. LEM
will require only 4–5 pointings to map the entire region of the giant
molecular clouds and map the most recent history of activity of our own SMBH.

Charge exchange emission in the Jovian magnetosphere, planets and comets. LEM
will offer revolutionary insights for heliophysics and planetary science. One
example is the Jovian system. Jupiter produces the most powerful aurora in the
solar system and these are key to understanding which processes govern
rapidly-rotating magnetospheres. It is unknown to what extent the X-ray aurora
are produced by the precipitation of material that originates in Io’s volcanos
or whether they are indicative of the direct entry of solar wind particles
into the system7.

Figure 14 shows how LEM’s exquisite energy resolution will finally solve this
problem. The figure shows the unique new access LEM provides to the low energy
line emissions from planets. This will open an entirely new paradigm, enabling
the bright emissions in this spectral range to be utilized to study previously
inaccessible objects or to offer a step change in capability to advance
previous X-ray studies of Mars, Saturn, comets, the Ice Giants and the
heliosphere2.

A possibility to extend the LEM calibrated energy band down to $E=0.05$ keV is
currently under study; while the effective area there would be small, the CX
signal at those energies is very luminous. This will also allow diagnostics
for species more closely linked to the standard UV lines of Civ, Siiv, Nv, and
Ovi. The sub-eV resolution of LEM’s central array would be critical in this
very crowded spectral region.

Earth magnetosphere. A possibility to point LEM toward Earth from an L1 orbit
is currently under study. If it is possible, LEM will be able to observe the
SWCX emission from Earth’s magnetopause and magnetosheath, study their
dynamics and their response to the varying solar wind pressure — a matter of
longstanding interest for the heliophysics community.

In addition, the LEM observatory will greatly advance the studies of variable
black hole winds, flares in exoplanet host stars, protostar accretion rates,
interstellar dust in emission and absorption, and numerous other fields. Given
the big leap in observing capability that LEM will bring, observers will most
certainly come up with experiments that we cannot even foresee at present.

##  4 CONCLUSIONS

The Line Emission Mapper Probe will fundamentally transform our understanding
of one of the most important questions in modern astrophysics — the formation
of galaxies. The unique ability of this observatory to detect the low surface
brightness X-ray emission from the warm and hot gas in galaxy halos and in the
filamentary structures connecting larger mass structures, and to measure the
temperature, entropy, and elemental abundances will allow an understanding of
the state of this gas that is, at present, only a matter of theoretical
speculation. With its combination of wide field of view X-ray optic and
energy-dispersive microcalorimeter array, LEM offers the opportunity for
complete dynamical and thermodynamic characterization of this gas out to a
large fraction of the virial radius in Milky Way-sized galaxies and beyond the
virial radius for galaxy clusters. LEM will detect metals expelled by galaxies
over their lifetime and accumulated in the Cosmic Web filaments, and directly
measure the imprint of AGN feedback, stellar feedback, and merging over cosmic
time in galaxies halos and cluster filaments.

LEM is the X-ray mission for the 2030s, and it should be the first of the
exciting new line of astrophysics Probes. The primary science of LEM is of
great interest to the entire astrophysical community, not just the X-ray
community. Most of the LEM observing time will be dedicated to the guest
observer program. The combination of wide field of view and 1–2 eV spectral
resolution of the calorimeter offers a unique host of guest observer
investigations covering every area of X-ray astrophysics, including studies of
AGN, stellar astrophysics, exoplanets, Solar system physics, and time domain
and multi-messenger astrophysics. LEM also has significant synergies with
virtually every ground-based and space-based observatory planned to be in
operation in the 2030s, including SKA, Roman Space Telescope, CMB-S4, and many
others. Finally, the technologies required to implement this mission — the
lightweight high-resolution X-ray optics and the large microcalorimeter arrays
— are rapidly maturing. With small, well-defined, ongoing investments in
technology development, LEM is already on the path to meet its key mission
milestones. LEM offers the astrophysics community paradigm-changing science in
a Probe package.

## References

  * 1 Ackermann, M., et al. 2014, ApJ, 793, 64
  * 2 Bhardwaj, A., et al. 2007, Planetary & Space Sci., 55, 1135
  * 3 Cen, R., & Ostriker, J. P. 1999, ApJ, 514, 1
  * 4 Churazov, E., et al. 2017, MNRAS, 465, 45
  * 5 Crain, R. A., et al. 2015, MNRAS, 450, 1937
  * 6 Davé, R., et al. 2019, MNRAS, 486, 2827
  * 7 Dunn, W., 2022, arXiv:2208.13455
  * 8 Khabibullin, I. & Churazov, E. 2019, MNRAS, 482, 4972
  * 9 Lee, S. J., et al. 2015, Appl. Phys. Lett., 107, 223503
  * 10 Mernier, F., & Biffi, V. 2022, arxiv:2202.07097
  * 11 Miniussi, A. R., et al. 2018, J. Low Temp. Phys., 193, 337
  * 12 Nelson, D., et al. 2018, MNRAS, 477, 450
  * 13 Nelson, D., et al. 2019a, Comp. Astrophys. & Cosmology, 6, 2
  * 14 Nelson, D., et al. 2019b, MNRAS, 490, 3234
  * 15 Norris, R. P., et al. 2021, PASA, 38, 3
  * 16 Oppenheimer, B. D., et al. 2016, MNRAS, 460, 2157
  * 17 Pillepich, A., et al. 2018, MNRAS, 473, 4077
  * 18 Pillepich, A., et al. 2019, MNRAS, 490, 3196
  * 19 Pillepich, A., et al. 2021, MNRAS, 508, 4667
  * 20 Predehl, P., et al. 2020, Nature, 588, 227
  * 21 Roediger, E., et al. 2015, ApJ, 806, 104
  * 22 Schaye, J., et al. 2015, MNRAS, 446, 521
  * 23 Smith, S. J., et al. 2020, J. Low Temperature Physics, 199, 330
  * 24 Smith, S. J., et al. 2023, J. Astron. Telescopes, Instruments, and Systems, to be submitted
  * 25 Yun, K., et al. 2019, MNRAS, 483, 1042
  * 26 ZuHone, J. A., et al. 2016, ApJ, 817, 110


# AMPERSAND: Argument Mining for PERSuAsive oNline Discussions

Tuhin Chakrabarty1, Christopher Hidey1, Smaranda Muresan1,2
, Kathleen Mckeown1 and Alyssa Hwang1
1Department of Computer Science, Columbia University
2Data Science Institute, Columbia University
{tuhin.chakrabarty, smara<EMAIL_ADDRESS>
{chidey<EMAIL_ADDRESS>
###### Abstract

Argumentation is a type of discourse where speakers try to persuade their
audience about the reasonableness of a claim by presenting supportive
arguments. Most work in argument mining has focused on modeling arguments in
monologues. We propose a computational model for argument mining in online
persuasive discussion forums that brings together the micro-level (argument as
product) and macro-level (argument as process) models of argumentation.
Fundamentally, this approach relies on identifying relations between
components of arguments in a discussion thread. Our approach for relation
prediction uses contextual information in terms of fine-tuning a pre-trained
language model and leveraging discourse relations based on Rhetorical
Structure Theory. We additionally propose a candidate selection method to
automatically predict what parts of one’s argument will be targeted by other
participants in the discussion. Our models obtain significant improvements
compared to recent state-of-the-art approaches using pointer networks and a
pre-trained language model.

##  1 Introduction

Argument mining is a field of corpus-based discourse analysis that involves
the automatic identification of argumentative structures in text. Most current
studies have focused on monologues or micro-level models of argument that aim
to identify the structure of a single argument by identifying the argument
components (classes such as “claim” and “premise”) and relations between them
(“support” or “attack”) (Somasundaran et al., 2007; Stab and Gurevych, 2014;
Swanson et al., 2015; Feng and Hirst, 2011; Habernal and Gurevych, 2017;
Peldszus and Stede, 2015). Macro-level models (or dialogical models) and
rhetorical models which focus on the _process_ of argument in a dialogue
(Bentahar et al., 2010) have received less attention.

Figure 1:  Annotated Discussion Thread (C: Claim, P: Premise, MC: Main Claim)

We propose a novel approach to automatically identify the argument structure
in persuasive dialogues that brings together the micro-level and the macro-
level models of argumentation. Our approach identifies argument components in
a full discussion thread and two kinds of argument relations: _inter-turn
relations_ (argumentative relations to support or attack the other person’s
argument) and _intra-turn_ relations (to support one’s claim). Figure 1 shows
a thread structure consisting of multiple posts with argumentative components
(main claims, claims or premises) and both intra- and inter-turn relations. We
focus here on the relation identification task (i.e, predicting the existence
of a relation between two argument components).

To learn to predict relations, we annotate argumentative relations in threads
from the Change My View (CMV) subreddit. In addition to the CMV dataset we
also introduce two novel distantly-labeled data-sets that incorporate the
macro- and micro-level context (further described in Section 3). We also
propose a new transfer learning approach to fine-tune a pre-trained language
model (Devlin et al., 2019) on the distant-labeled data (Section 4) and
demonstrate improvement on both argument component classification and relation
prediction (Section 5). We further show that using discourse relations based
on Rhetorical Structure Theory (Mann and Thompson, 1988) improves the results
for relation identification both for inter-turn and intra-turn relations.
Overall, our approach for argument mining obtains statistically significant
improvement over a state-of-the-art model based on pointer networks (Morio and
Fujita, 2018) and a strong baseline using a pre-trained language model (Devlin
et al., 2019). We make our data,111https://github.com/chridey/change-my-view-
modes code, and trained models publicly
available.222https://github.com/tuhinjubcse/AMPERSAND-EMNLP2019

##  2 Related Work

#### Argument mining on monologues

Most prior work in argument mining (AM) has focused on monologues or essays.
Stab and Gurevych (2017) and Persing and Ng (2016) used pipeline approaches
for AM, combining parts of the pipeline using integer linear programming (ILP)
to classify argumentative components. Stab and Gurevych (2017) represented
relations of argument components in essays as tree structures. Both Stab and
Gurevych (2017) and Persing and Ng (2016) propose joint inference models using
ILP to detect argumentative relations. They however rely on handcrafted
(structural, lexical, indicator, and discourse) features. We on the other hand
use a transfer learning approach for argument mining in discussion forums.

Recent work has examined neural models for argument mining. Potash et al.
(2017) use pointer networks (Vinyals et al., 2015) to predict both
argumentative components and relations in a joint model. For our work, we
focus primarily on relation prediction but conduct experiments with predicting
argumentative components in a pipeline. A full end-to-end neural sequence
tagging model was developed by Eger et al. (2017) that predicts argumentative
discourse units (ADUs), components, and relations at the token level. In
contrast, we assume we have ADUs and predict components and relations at that
level.

#### Argument mining on discussion forums (online discussion)

Most computational work related to argumentation as a process has focused on
the detection of agreement and disagreement in online interactions (Abbott et
al., 2011; Sridhar et al., 2015; Rosenthal and McKeown, 2015; Walker et al.,
2012). However, these approaches do not identify the argument components that
the (dis)agreement has scope over (i.e., what has been targeted by a
disagreement or agreement move). Also in these approaches, researchers predict
the type of relation (e.g. agreement) given that a relationship exists. On the
contrary, we predict both argument components as well as the existence of a
relation between them. Boltužić and Šnajder (2014) and Murakami and Raymond
(2010) address relations between complete arguments but without the micro-
structure of arguments as in Stab and Gurevych (2017). Ghosh et al. (2014)
introduce a scheme to annotate inter-turn relations between two posts as
“target-callout”, and intra-turn relations as “stance-rationale”. However,
their empirical study is reduced to predicting the type of inter-turn
relations as agree/disagree/other. Our computational model on the other hand
handles both macro- and micro- level structures of arguments (argument
components and relations).

Budzynska et al. (2014) focused on building foundations for extracting
argument structure from dialogical exchanges (radio-debates) in which that
structure may be implicit in the dynamics of the dialogue itself. By combining
recent advances in theoretical understanding of inferential anchors in
dialogue with grammatical techniques for automatic recognition of pragmatic
features, they produced results for illocutionary structure parsing which are
comparable with existing techniques acting at a similar level such as
rhetorical structure parsing. Furthermore, Visser et al. (2019) presented
US2016, the largest publicly available set of corpora of annotated dialogical
argumentation. Their annotation covers argumentative relations, dialogue acts
and pragmatic features.

Although Niculae et al. (2017) tried relation prediction between arguments in
user comments on web discussion forums using structured SVM and RNN the work
closest to our task is that of Morio and Fujita (2018). They propose a pointer
network model that takes into consideration the constraints of thread
structure. Their model discriminates between types of arguments (e.g., claims
or premises) and both intra-turn and inter-turn relations, simultaneously.
Their dataset, which is not publically available, is three times larger than
ours, so instead we focus on transfer learning approaches that take advantage
of discourse and dialogue context and use their model as our baseline.

##  3 Data

###  3.1 Labeled Persuasive Forum Data

To learn to predict relations, we use a corpus of 78 threads from the CMV
subreddit annotated with argumentative components (Hidey et al., 2017). The
authors annotate _claims_ (“propositions that express the speaker’s stance on
a certain matter”), and _premises_ (“propositions that express a justification
provided by the speaker in support of a claim”). In this data, the _main
claim_ , or the central position of the original poster, is always the title
of the original post.

We expand this corpus by annotating the argumentative relations among these
propositions (both inter-turn and intra-turn) and extend the corpus by
annotating additional argument components (using the guidelines of the
authors) for a total of 112 threads. It is to be noted that the claims,
premises, and relations have been annotated jointly. Our annotation results on
the argumentative components for the additional threads were similar to Hidey
et al. (2017) ( IAA using Kripendorff alpha is $0.63$ for claims and $0.65$
for premises). The annotation guidelines for relations are available in the
aforementioned Github repository.

#### Intra-turn Argumentative Relations

As in previous work (Morio and Fujita, 2018), we restrict intra-turn relations
to be between a premise and another claim or premise, where the premise either
supports or attacks the claim or other premise. Evidence in the form of a
premise is either support or attack. Consider the following example:

> [Living and studying overseas is an irreplaceable experience.]0:CLAIM [One
> will struggle with loneliness]1:PREMISE:ATTACK:0 [but those difficulties
> will turn into valuable experiences.]2:PREMISE:ATTACK:1 [Moreover, one will
> learn to live without depending on anyone.]3:PREMISE:SUPPORT:0

The example illustrates that the argumentative component at index 0 is a
claim, and is followed by attacking one’s own claim, and in turn attacking
that premise (an example of the rhetorical move known as prolepsis or
prebuttal). They conclude the argument by providing supporting evidence for
their initial claim.

#### Inter-turn Argumentative Relations

Inter-turn relations connect the arguments of two participants in the
discussion (agreement or disagreement). The argumentative components involved
in inter-turn relations are claims, as the nature of dialogic argumentation is
a difference in stance. An _agreement_ relation expresses an agreement or
positive evaluation of another user’s claim whereas a _disagreement/attack_
relation expresses disagreement. We further divide disagreement/attack
relations into _rebuttal_ and _undercutter_ types to distinguish when a claim
directly challenges the truth of the other claim or challenges the reasoning
connecting the other claim to the premise that supports it, respectively. We
also allowed annotators to label claims as _partial_ agreement or
disagreement/attack if the response concedes part of the claim, depending on
the order of the concession.

In the following example, User A makes a claim and supports it with a premise.
User B agrees with the claim but User C disputes the idea that there even is
global stability. Meanwhile, User D disagrees with the reasoning that the past
is always a good predictor of current events.

> A: [I think the biggest threat to global stability comes from the political
> fringes.]0:CLAIM [It has been like that in the past.]1:PREMISE
>
> B: [Good arguments.]2:AGREEMENT:0
>
> C: [The only constant is change.]3:REBUTTAL:0
>
> D: [What happened in the past has nothing to do with the
> present.]4:UNDERCUTTER:1

We obtain moderate agreement for relation annotations, similar to other
argumentative tasks (Morio and Fujita, 2018). The Inter-Annotator Agreement
(IAA) with Kripendorff’s $\alpha$ is 0.61 for relation presence and 0.63 for
relation types.

|  CMV: A rise in female representation in elected government isn’t a good or
bad thing.
---
According to this new story, a record number of women are seeking office in
this year’s US midterm elections.
While some observers hail this phenomenon as a step in the right direction, I
don’t think it’s good thing one way or the
other: a politician’s sex has zero bearing on their ability to govern or craft
effective legislation. As such…
| “I don’t think it’s good thing one way or the other: a politician’s sex has
zero bearing on their ability to govern or craft
---
effective legislation” Nobody is saying that women are better politicians than
men, and thus, more female representation
is inherently better for our political system. Rather, the argument is that…
Table 1:  Two posts from the CMV sub-reddit where a claim is targeted by the
response

In total, the dataset contains 380 turns of dialogue for 2756 sentences. There
were 2741 argumentative propositions out of which 1205 are claims and 1536 are
premises, with an additional 799 non-argumentative propositions. 66% of
relations were in support, 26% attacking, and 8% partial. As we found that
most intra-turn relations were in support and inter-turn relations were
attacking, due to the dialogic nature of the data, for our experiments we only
predicted whether a relation was present and not the relation type.

Overall, there are 7.06 sentences per post for our dataset, compared to 4.19
for Morio and Fujita (2018). This results in a large number of possible
relations, as all pairs of argumentative components are candidates. The
resulting dataset is very unbalanced (only 4.6% of 27254 possible pairs have a
relation in the intra-turn case with only 3.2% of 26695 for inter-turn),
adding an extra challenge to modeling this task.

###  3.2 Distant-Labeled Data

As the size of this dataset is small for modern deep learning approaches, we
leverage distant-labeled data from Reddit and use transfer learning techniques
to fine-tune a model on the appropriate context — micro-level for intra-turn
relations and macro-level (dialogue) for inter-turn relations.

#### Micro-level Context Data

In order to leverage transfer learning methods, we need a large dataset with
distant-labeled relation pairs. In previous work, Chakrabarty et al. (2019)
collected a distant-labeled corpus of opinionated claims using sentences
containing the internet acronyms IMO (in my opinion) or IMHO (in my humble
opinion) from Reddit. We leverage their data to model relation pairs by
considering the following sentence (i.e., a premise) as well (when it was
present), resulting in a total of 4.6 million comments. We denote this dataset
as IMHO+context. The following example shows an opinionated claim (in bold)
backed by a supporting premise (in italics).

> IMHO, Calorie-counting is a crock what you have to look at is how wholesome
> are the foods you are eating. _Refined sugar is worse than just empty
> calories - I believe your body uses a lot of nutrients up just processing
> and digesting it._

We assume in this data that a relation exists between the sentence containing
IMHO and the following one. While a premise does not always directly follow a
claim, it does so frequently enough that this noisy data should be helpful.

#### Macro-level Context Data

While the IMHO data is useful for modeling context from consecutive sentences
from the same author, inter-turn relations are of a dialogic nature and would
benefit from models that consider that macro-level context. We take advantage
of a feature of Reddit: when responding to a post, users can easily quote
another user’s response, which shows up in the metadata. Particularly in CMV,
this feature is used to highlight exactly what part of someone’s argument a
particular user is targeting. In the example in Table 1, the response contains
an exact quote of a claim in the original post. We collect 95,406 threads from
the full CMV subreddit between 2013-02-16 and 2018-09-05 and find pairs of
posts where the quoted text in the response is an exact match for the original
post (removing threads that overlap with the labeled data). This phenomenon
occurs a minority of the time, but we obtain 19,413 threads. When the quote
feature is used, posters often respond to multiple points in the original
text, so for the 19,413 threads we obtain 97,636 pairs. As most language model
fine-tuning is performed at the sentence level, we take the quoted text and
the following sentence as our distant-labeled inter-post pairs. We refer to
this dataset as QR, for quote-response pairs.

Figure 2:  Schematic of our pipeline involving various components. The
Candidate Selection Component is only used for Inter-turn relation
identification

##  4 Methods

Identifying argumentative components is a necessary precursor to predicting an
argumentative relation. In our data, we require the “source” of an intra-turn
relation to be a premise that supports or attacks a “target” (a premise or
another claim). For inter-turn relations, the source is always a claim that
agrees or disagrees with a target. Thus, we model this process as a pipeline:
perform three-way classification on claims, premises, and non-arguments and
then predict if an outgoing relation exists from the source premise/claim to a
target premise/claim. In predicting these relations, we consider all possible
source-target pairs of premises and argumentative components within a single
post (for intra-turn) and claims from one post and argumentative components
from another post (for inter-turn).

As our data set is relatively small, we leverage recent advances in transfer
learning for natural language processing. BERT (Devlin et al., 2019) has shown
excellent performance on many semantic tasks, both for single sentences and
pairs of sentences, making this model an ideal fit for both argument component
classification and relation prediction. The BERT model is initially trained
with a multi-task objective (masked language modeling and next-sentence
prediction) over a 3.3 billion word English corpus. In the standard use, given
a pre-trained BERT model, the model can be used for transfer learning by fine-
tuning on a domain-specific corpus using a supervised learning objective.

We take an additional step of fine-tuning on the relevant distant-labeled data
from Section 3.2 before again fine-tuning on our data from Section 3.1. We
hypothesize that this novel use of BERT will help because the data is
structured such that the sentence and next sentence pair (for intra-turn) or
quote and response pair (for inter-turn) will encourage the model to learn
features that improve performance on detecting argumentative components and
relations. For argumentative components, we use this fine-tuned BERT model to
classify a single proposition (as BERT can be used for both single inputs and
input pairs). For relations (both intra-turn and inter-turn), we predict the
relation between a pair of propositions. In the relation case, we postulate
that additional components besides BERT can help: adding features derived from
RST structure and pruning the space of possible pairs by selecting candidate
target components. The pipeline is illustrated in Figure 2.

#### Argument Component Classification

We fine-tune a BERT model on the IMHO+context dataset described in Section 3.2
using both the masked language modeling and next sentence prediction
objectives. We then again fine-tune BERT to predict the argument components
for both sources and targets, as indicated in Figure 2. Given the predicted
argumentative components, we consider all possible valid pairs either within a
post (for intra-turn) or across (inter-turn) and make a binary prediction of
whether a relation is present.

#### Context Fine-Tuning

For intra-turn relation prediction we use the same fine-tuned BERT model on
IMHO+context that we used for argument component classification, as premises
often immediately follow claims so this task is a noisy analogue to the task
of interest. We then fine-tune on the relation prediction task on all possible
pairs, using the labeled relations in the CMV data. For inter-turn relation
prediction, we fine-tune first on the QR dataset, where the dialogue context
more closely represents our labeled inter-post relations. Then, we fine-tune
on inter-turn relation prediction using all possible pairs as training. This
process is indicated in Figure 2, where we use the appropriate context fine-
tuning to obtain a relation classifier.

#### Discourse Relations

Rhetorical Structure Theory was originally developed to offer an explanation
of the coherence of texts. Musi et al. (2018) and, more recently Hewett et al.
(2019), showed that discourse relations from RST often correlate with
argumentative relations. We thus derive features from RST trees and train a
classifier using these features to predict an argumentative relation. To
extract features from a pair of argumentative components, we first concatenate
the two components so that they form a single text input. We then use a state-
of-the-art RST discourse parser (Ji and Eisenstein, 2014)333We use Wang et al.
(2018) for segmentation of text into elementary discourse units. to create
parse trees and take the predicted discourse relation at the root of the parse
tree as a categorical feature in a classifier. There are 28 unique discourse
relations predicted in the data, including Circumstance, Purpose, and
Antithesis. We use a one-hot encoding of these relations as features and train
an XGBoost Classifier Chen and Guestrin (2016) to predict whether an argument
relation exists. This classifier with discourse relations, as indicated in
Figure 2, is then ensembled with our predictions from the BERT classifier by
predicting a relation if either one of the classifiers predicts a relation.

#### Candidate Target Selection

For inter-turn relations, we take additional steps to reduce the number of
invalid relation pairs. Predicting an argumentative relation is made more
difficult by the fact that we need to consider all possible relation pairs.
However, some argumentative components may contain linguistic properties that
allow us to predict when they are targets even without the full relation pair.
Thus, if we can predict the targets with high recall, we are likely to
increase precision as we can reduce the number of false positives. Our
candidate selection component, which identifies potential targets (as shown in
Figure 2), consists of two sub-components: an extractive summarizer and a
source-target constraint.

First, we use the QR data to train a model to identify candidate targets using
techniques from extractive summarization, with the idea that targets may be
salient sentences or propositions. We treat the quoted sentences as gold
labels, resulting in 19,413 pairs of document (post) and summary (quoted
sentences). For the example provided in Table 1, this would result in one
sentence included in the summary. Thus, for a candidate argument pair A
$\rightarrow$ B, where B is the quoted sentence in Table 1, if B is not
extracted by the summarization model we predict that there is no relation
between A and B. An example of target selection via extractive summarization
is shown in Figure 3.

We use a state-of-the-art extractive summarization approach (Liu, 2019) for
extracting the targets. The authors obtain sentence representations from BERT
(Devlin et al., 2019), and build several summarization specific layers stacked
on top of the BERT outputs, to capture document-level features for extracting
summaries. We refer the reader to Liu (2019) for further details. We select
the best summarization model on a held out subset using recall at the top $K$
sentences.

Figure 3: Schematic of our Target Extraction Approach

Second, in addition to summarization, we take advantage of a dataset-specific
constraint: a target cannot also be a source unless it is related to the main
claim. In other words, if B is a predicted target in A $\rightarrow$ B, we
predict that there is no relation for B $\rightarrow$ A except when A is a
main claim. In the CMV data, the main claim is always the title of the
original Reddit post, so it is trivial to identify.

Method | C | P | NA
---|---|---|---
Stab and Gurevych (2017) \+ EWE | 56.0 | 65.9 | 69.6
Morio and Fujita (2018) | 54.2 | 68.5 | 73.2
Chakrabarty et al. (2019) | 57.8 | 70.8 | 70.5
BERT Devlin et al. (2019) | 62.0 | 72.2 | 71.3
IMHO Context Fine-Tuned BERT | 67.1 | 72.5 | 75.7
Table 2: F-scores for 3-way Classification: Claim (C), Premise (P), Non-Argument (NA) Method | Precision | Recall | F-Score
---|---|---|---
Gold | Pred | Gold | Pred | Gold | Pred
All relations | 5.0 | - | 100.0 | - | 9.0 | -
Menini et al. (2018) | 7.0 | 5.9 | 82.0 | 80.0 | 13.0 | 11.0
Menini et al. (2018) \+ RST Features | 7.4 | 6.1 | 83.0 | 81.0 | 13.7 | 11.4
RST Features | 6.3 | 5.7 | 79.5 | 77.0 | 11.8 | 10.6
Morio and Fujita (2018) | 10.0 | - | 48.8 | - | 16.6 | -
BERT Devlin et al. (2019) | 12.0 | 11.0 | 67.0 | 60.0 | 20.3 | 18.5
IMHO Context Fine-Tuned BERT | 14.3 | 13.2 | 69.0 | 65.0 | 23.7 | 21.8
\+ RST Ensemble | 16.7 | 15.5 | 73.0 | 70.2 | 27.2 | 25.4
Table 3: Results for Intra-turn Relation Prediction with Gold and Predicted
Premises

##  5 Experiments and Results

We use the CMV data from Section 3.1 for training and testing, setting aside
10% of the data for test. Hyper-parameters are discussed in Appendix A.

###  5.1 Argumentative Component Classification

For baseline experiments on argumentative component classification we rely on
the handcrafted features used by Stab and Gurevych (2017): lexical (unigrams),
structural (token statistics and position), indicator (I, me, my), syntactic
(POS, modal verbs), discourse relation (PDTB), and word embedding features.
Hidey et al. (2017) show that emotional appeal or pathos is strongly
correlated with persuasion and appears in premises. This motivated us to
augment the work of Stab and Gurevych (2017) with emotion embeddings (Agrawal
et al., 2018) which capture emotion-enriched word representations and show
improved performance over generic embeddings (denoted in the table as EWE).

We also compare our results to several neural models - a joint model using
pointer networks (Morio and Fujita, 2018), a model that leverages context
fine-tuning (Chakrabarty et al., 2019), and a BERT baseline (Devlin et al.,
2019) using only the pre-trained model without our additional fine-tuning
step.

Table 2 shows that our best model gains statistically significant improvement
over all the other models ($p<0.001$ with a Chi-squared test). To compare
directly to the work of Chakrabarty et al. (2019), we also test our model on
the binary claim detection task and obtain a Claim F-Score of 70.0 with fine-
tuned BERT, which is a 5-point improvement in F-score over pre-trained BERT
and a 12-point improvement over Chakrabarty et al. (2019). These results show
that fine-tuning on the appropriate context is key.

Method | Precision | Recall | F-Score
---|---|---|---
Gold | Pred | Gold | Pred | Gold | Pred
All relations | 5.0 | - | 100.0 | - | 9.0 | -
Menini et al. (2018) | 5.9 | 4.8 | 82.0 | 80.0 | 11.0 | 9.0
Menini et al. (2018) \+ RST Features | 6.4 | 4.9 | 83.0 | 80.0 | 11.8 | 9.3
RST Features | 5.1 | 3.8 | 80.0 | 77.0 | 9.6 | 7.2
Morio and Fujita (2018) | 7.6 | - | 40.0 | - | 12.7 | -
BERT Devlin et al. (2019) | 8.8 | 7.9 | 76.0 | 70.0 | 15.8 | 14.1
QR Context Fine-Tuned BERT | 11.0 | 10.0 | 75.3 | 72.5 | 19.1 | 17.6
\+ RST Features | 11.0 | 12.2 | 79.0 | 75.5 | 21.2 | 19.1
\+ Extractive Summarizer | 16.0 | 14.5 | 79.4 | 75.6 | 26.8 | 24.3
\+ Source $\neq$ Target Constraint | 18.9 | 17.5 | 79.0 | 74.0 | 30.5 | 28.3
Table 4: Results for Inter-Turn Relation Prediction with Gold and Predicted
Claims

###  5.2 Relation Prediction

For baseline experiments on relation prediction, we consider prior work in
macro-level argument mining. Menini et al. (2018) predict argumentative
relations between entire political speeches from different speakers, which is
similar to our dialogues. We re-implement their model using their features
(lexical overlap, negation, argument entailment, and argument sentiment, among
others). As with component classification, we also compare to neural models
for relation prediction - the joint pointer network architecture (Morio and
Fujita, 2018) and the pre-trained BERT (Devlin et al., 2019) baseline.

As the majority of component pairs contain no relation, we could obtain high
accuracy by predicting that all pairs have no relation. Instead, we want to
measure our performance on relations, so we also include an “all-relation”
baseline, where we always predict that there is a relation between two
components, to indicate the difficulty of modeling such an imbalanced data
set. In the test data, for intra-turn relations there are 2264 relation pairs,
of which only 174 have a relation, and for inter-turn relations there are 120
relation pairs, compared to 2381 pairs with no relation.

As described in Section 3.1, for intra-turn relations, the source is
constrained to be a premise whereas for intra-turn, it is constrained to be a
claim. We thus provide experiments using both gold claims/premises and
predicted ones.

#### Intra-turn Relations

We report the results of our binary classification task in Table 3 in terms of
Precision, Recall and F-score for the “true” class, i.e., when a relation is
present. We report results given both gold premises and predicted premises
(using our best model from 5.1). Our best results are obtained from ensembling
the RST classifier with BERT fine-tuned on IMHO+context, for statistically
significant ($p<0.001$) improvement over all other models. We obtain
comparable performance to previous work on relation prediction in other
argumentative datasets (Niculae et al., 2017; Morio and Fujita, 2018).

#### Inter-turn Relations

As with intra-turn relations, we report F-score on the “true” class in Table 4
for both gold and predicted claims. Our best results are obtained by fine-
tuning the BERT model on the appropriate context (in this case the QR data)
and ensembling the predictions with the RST classifier. We again obtain
statistically significant ($p<0.001$) improvement over all baselines.

Our methods for candidate target selection obtain further improvement. Using
our extractive summarizer, we found that we obtained the best target recall of
62.7 at $K=5$ (the number of targets to select). This component improves
performance by approximately 5 points in F-score by reducing the search space
of relation pairs. By further constraining targets to only be a source when
targeting a main claim, we obtain another 4 point gain.

#### Window Clipping

We also conduct experiments showing the performance for intra-turn relation
prediction when constraining the relations to be within a certain distance in
terms of the number of sentences apart. Often, in persuasive essays or within
a single post the authors use premises to back/justify claims they immediately
made. As shown in Figure 4, this behavior is also reflected in our dataset
where the distance between the two arguments in the majority of the relations
is +1.

We thus limit the model’s prediction of a relation to be within a certain
window and predict “no relation” for any pairs outside of that window. Table 5
shows that this window clipping on top of our best model improves F-score by
only limiting the context where we make predictions. As our models are largely
trained on discourse context and the next sentence usually has a discourse
relation, we obtain improved performance as we narrow the window size. While
we see a drop in recall the precision improves compared to our previous
results in Table 3. It is also important to note that window clipping is only
beneficial once we have a high recall, low precision scenario because when we
predict everything at a distance of +1 as a relation we obtain low F-scores.

Figure 4: Distances $d$ between Intra-Turn Relations Method | Window | Precision | Recall | F-Score
---|---|---|---|---
Gold | Pred | Gold | Pred | Gold | Pred
All relations | 0 TO +1 | 5.0 | 4.0 | 31.0 | 25.0 | 8.7 | 6.9
Best Model | 0 TO +5 | 19.5 | 17.1 | 70.0 | 67.0 | 30.5 | 27.2
0 TO +4 | 21.4 | 19.5 | 67.0 | 65.0 | 32.2 | 30.0
0 TO +3 | 25.2 | 23.3 | 61.1 | 58.0 | 35.6 | 33.2
0 TO +2 | 32.5 | 29.8 | 50.0 | 48.0 | 39.3 | 36.8
0 TO +1 | 41.5 | 39.1 | 47.0 | 42.0 | 44.1 | 40.3
Table 5: Intra-Turn Relation Prediction with Varying Window Settings

##  6 Qualitative Analysis

#### Role of Context

We retrieve examples from the IMHO+context and QR data using TF-IDF similarity
to pairs of argument components from our data that were predicted incorrectly
by pre-trained BERT but correctly by the respective fine-tuned model. The
first two rows in Table 6 show a relation between a claim and premise in the
IMHO+Context and the CMV data respectively while the last four rows show a
relation between a claim and premise in the QR data and the CMV data. The
model learns discriminative discourse relations from the IMHO+context data and
correctly identifies this pair. The last four rows show rebuttal from the QR
and the CMV data respectively, where the model learns discriminative dialogic
phrases (highlighted in bold).

Context |  Pair
---|---
IMHO |  IMHO, you should not quantify it as good or bad. Tons of people have monogamous relationships without issue.
CMV |  [how would you even quantify that] [there are many people who want close relationships without romance]
QR |  [It might be that egalitarians,anti-feminists, MRAs & redpillers, groups that I associate with opposing feminism - might be in fact very distinct & different groups, but I don’t know that] [I do see all four of these as distinct groups].
CMV |  [I may have a different stance on seeing no difference between companion animals and farm animals.] [I do see distinction between a pet and livestock]
QR |  [Of course you intend to kill the person if you draw your weapon, if you can reasonably assume that they have a weapon] [I don’t think some of them would start killing].
CMV |  [So i thought, why would a police officer even use firearms if he/she doesn’t intend to kill?] [I don’t think, police are allowed to start killing someone with their gun if they don’t intend to . ]
Table 6: CMV and Context Examples

#### Role of Discourse

Discourse | Argument1 | Argument2
---|---|---
| Evaluation
---
| The only way your
---
life lacks meaning
is if you give it none
to begin with
| Life is ultimately
---
meaningless and
pointless.
| Antithesis
---
| Joseph was just a
---
regular Jew without
the same kind of
holiness as the
other two
| Aren’t Mary and
---
Joseph, two holy
people especially
perfect virgin Mary,
both Jews? Wasn’t
Jesus a Jew?
Table 7: Predicted Discourse Relations in CMV

We also provide examples that are predicted incorrectly by BERT but correctly
by our classifier trained with RST features. For the first example in Table 7
the RST parser predicts an Evaluation relation, which is an indicator of an
argumentative relation according to our model. For the second example the RST
parser predicts Antithesis, which is correlated with attack relations (Musi et
al., 2018), and is predicted correctly by our model.

##  7 Conclusion

We show how fine-tuning on data-sets similar to the task of interest is often
beneficial. As our data set is small we demonstrated how to use transfer
learning by leveraging discourse and dialogue context. We show how the
structure of the fine-tuning corpus is essential for improved performance on
pre-trained language models. We also showed that predictions that take
advantage of RST discourse cues are complementary to BERT predictions.
Finally, we demonstrated methods to reduce the search space and improve
precision.

In the future, we plan to experiment further with language model fine-tuning
on other sources of data. We also plan to investigate additional RST features.
As the RST parser is not perfect, we want to incorporate other features from
these trees that allow us to better recover from errors.

##  8 Acknowledgements

The authors thank Fei-Tzin Lee and Alexander Fabbri for their helpful comments
on the initial draft of this paper and the anonymous reviewers for helpful
comments.

## References

  * Abbott et al. (2011) Rob Abbott, Marilyn Walker, Pranav Anand, Jean E Fox Tree, Robeson Bowmani, and Joseph King. 2011.  How can you say such things?!?: Recognizing disagreement in informal political argument.  In _Proceedings of the Workshop on LSM_ , pages 2–11.
  * Agrawal et al. (2018) Ameeta Agrawal, Aijun An, Papagelis Chen, and Manos. 2018.  Learning emotion-enriched word representations.  In _Proceedings of the 27th International Conference on Computational Linguistics_.
  * Bentahar et al. (2010) Jamal Bentahar, Bernard Moulin, and Micheline Bélanger. 2010.  A taxonomy of argumentation models used for knowledge representation.  _Artif. Intell. Rev._ , 33(3):211–259.
  * Boltužić and Šnajder (2014) Filip Boltužić and Jan Šnajder. 2014.  Back up your stance: Recognizing arguments in online discussions.  In _Proceedings of the First Workshop on Argumentation Mining_ , pages 49–58.
  * Budzynska et al. (2014) Katarzyna Budzynska, Mathilde Janier, Juyeon Kang, Chris Reed, Patrick Saint-Dizier, Manfred Stede, and Olena Yaskorska. 2014.  Towards argument mining from dialogue.
  * Chakrabarty et al. (2019) Tuhin Chakrabarty, Christopher Hidey, and Kathleen McKeown. 2019.  Imho fine-tuning improves claim detection.  _arXiv preprint arXiv:1905.07000_.
  * Chen and Guestrin (2016) Tianqi Chen and Carlos Guestrin. 2016.  XGBoost: A scalable tree boosting system.  In _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , KDD ’16, pages 785–794, New York, NY, USA. ACM.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Lee Kenton, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 17th Annual Meeting of the North American Association for Computational Linguistics_.
  * Eger et al. (2017) Steffen Eger, Johannes Daxenberger, and Iryna Gurevych. 2017.  Neural end-to-end learning for computational argumentation mining.  In _In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics._ , pages 11––22.
  * Feng and Hirst (2011) Vanessa Wei Feng and Graeme Hirst. 2011.  Classifying arguments by scheme.  In _ACL_ , pages 987–996.
  * Ghosh et al. (2014) Debanjan Ghosh, Smaranda Muresan, Wacholder Nina, Mark Aakhus, and Matthew. Mitsui. 2014.  Analyzing argumentative discourse units in online interactions.  In _In Proceedings of the First Workshop on Argument Mining, hosted by the 52nd Annual Meeting of the Association for Computational Linguistics, ArgMining@ACL_ , pages 39––48.
  * Habernal and Gurevych (2017) Ivan Habernal and Iryna Gurevych. 2017.  Argumentation mining in user-generated web discourse.  _Computational Linguistics_ , 43(1):125–179.
  * Hewett et al. (2019) Freya Hewett, Roshan Prakash Rane, Nina Harlacher, and Manfred Stede. 2019.  The utility of discourse parsing features for predicting argumentation structure.  In _Proceedings of the 6th Workshop on Argument Mining_ , pages 98–103, Florence, Italy. Association for Computational Linguistics.
  * Hidey et al. (2017) Christopher Hidey, Elena Musi, Alyssa Hwang, Smaranda Muresan, and Kathleen McKeown. 2017.  Analyzing the semantic types of claims and premises in an online persuasive forum.  In _In Proceedings of the 4th Workshop on Argument Mining. EMNLP_ , pages 11––21.
  * Ji and Eisenstein (2014) Yangfeng Ji and Jacob Eisenstein. 2014.  Representation learning for text-level discourse parsing.  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics,_ , pages 13–24.
  * Liu (2019) Yang Liu. 2019.  Fine-tune bert for extractive summarization.  In _https://arxiv.org/abs/1903.10318_.
  * Mann and Thompson (1988) William C Mann and Sandra A Thompson. 1988.  Rhetorical structure theory: Toward a functional theory of text organization.  _Text-Interdisciplinary Journal for the Study of Discourse_ , 8(3):243–281.
  * Menini et al. (2018) Stefano Menini, Elena Cabrio, Sara Tonelli, and Villata Serena. 2018.  Never retreat, never retract: Argumentation analysis for political speeches.  In _Association for the Advancement of Artificial Intelligence_.
  * Morio and Fujita (2018) Gaku Morio and Katsuhide Fujita. 2018.  End-to-end argument mining for discussion threads based on parallel constrained pointer architecture.  In _In Proceedings of the 5th Workshop on Argument Mining. EMNLP_ , pages 11–21.
  * Murakami and Raymond (2010) Akiko Murakami and Rudy Raymond. 2010.  Support or oppose?: Classifying positions in online debates from reply activities and opinion expressions.  In _In Proceedings of the 23rd International Conference on Computational Linguistics, ArgMining@ACL_ , pages 869––875.
  * Musi et al. (2018) Elena Musi, Tariq Alhindi, Stede Manfred, Leonard Kriese, Smaranda Muresan, and Andrea Rocci. 2018.  A multi-layer annotated corpus of argumentative text: From argument schemes to discourse relations.  In _Proceedings of Language Resources and Evaluation Conference (LREC 2018)_.
  * Niculae et al. (2017) Vlad Niculae, Joonsuk Park, and Claire Cardie. 2017.  Argument mining with structured SVMs and RNNs.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 985–995, Vancouver, Canada. Association for Computational Linguistics.
  * Peldszus and Stede (2015) Andreas Peldszus and Manfred Stede. 2015.  Joint prediction in mst-style discourse parsing for argumentation mining.  In _EMNLP_ , pages 938–948.
  * Persing and Ng (2016) Isaac Persing and Vincent Ng. 2016.  End-to-end argumentation mining in student essays.  In _In Proceedings of the 15th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies._ , pages 1384–1394.
  * Potash et al. (2017) Peter Potash, Alexey Romanov, and Anna Rumshisky. 2017.  Here’s my point: Joint pointer architecture for argument mining.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 1364–1373.
  * Rosenthal and McKeown (2015) Sara Rosenthal and Kathleen McKeown. 2015.  I couldn’t agree more: The role of conversational structure in agreement and disagreement detection in online discussions.  In _16th Annual Meeting of the SIGDIAL_.
  * Somasundaran et al. (2007) Swapna Somasundaran, Josef Ruppenhofer, and Janyce Wiebe. 2007.  Detecting arguing and sentiment in meetings.  In _Proceedings of the SIGdial Workshop on Discourse and Dialogue_ , volume 6.
  * Sridhar et al. (2015) Dhanya Sridhar, James R Foulds, Bert Huang, Lise Getoor, and Marilyn A Walker. 2015\.  Joint models of disagreement and stance in online debate.  In _ACL_ , pages 116–125.
  * Stab and Gurevych (2014) Christian Stab and Iryna Gurevych. 2014.  Identifying argumentative discourse structures in persuasive essays.  In _EMNLP_.
  * Stab and Gurevych (2017) Christian Stab and Iryna Gurevych. 2017.  Parsing argumentation structures in persuasive essays.  In _Computational Linguistics, pages in press, preprint available at arXiv:1604.07370_.
  * Swanson et al. (2015) Reid Swanson, Brian Ecker, and Marilyn Walker. 2015.  Argument mining: Extracting arguments from online dialogue.  In _Proceedings of the 16th Annual Meeting of the SIGDIAL_ , pages 217–226.
  * Vinyals et al. (2015) Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015.  Pointer networks.  In _In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28_ , pages 2692––2700.
  * Visser et al. (2019) Jacky Visser, Barbara Konat, Rory Duthie, Marcin Koszowy, Katarzyna Budzynska, and Chris Reed. 2019.  Argumentation in the 2016 us presidential elections: annotated corpora of television debates and social media reaction.  _Language Resources and Evaluation_.
  * Walker et al. (2012) Marilyn A Walker, Jean E Fox Tree, Pranav Anand, Rob Abbott, and Joseph King. 2012\.  A corpus for research on deliberation and debate.  In _LREC_ , pages 812–817.
  * Wang et al. (2018) Yizhong Wang, Sujian Li, and Jingfeng Yang. 2018.  Toward fast and accurate neural discourse segmentation.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 962–967.