# Explosive chromospheric evaporation driven by nonthermal electrons around
one footpoint of a solar flare loop

D. Li11affiliationmark:  22affiliationmark:  33affiliation: Correspondence
should be sent to<EMAIL_ADDRESS>, Z. J. Ning11affiliationmark:  , Y.
Huang11affiliationmark:  , and Q. M. Zhang11affiliationmark:
22affiliationmark:  1Key Laboratory of Dark Matter and Space Astronomy, Purple
Mountain Observatory, CAS, Nanjing 210008, China  2CAS Key Laboratory of Solar
Activity, National Astronomical Observatories, Beijing 100012, China  3Key
Laboratory of Modern Astronomy and Astrophysics (Nanjing University), Ministry
of Education, Nanjing 210023, China

###### Abstract

We explore the temporal relationship between microwave/HXR emission and
Doppler velocity during the impulsive phase of a solar flare on 2014 October
27 (SOL2014-10-27), which displays a pulse on the light curves in microwave
(34 GHz) and hard X-ray (HXR, 25$-$50 keV) bands before the flare maximum.
Imaging observation shows that this pulse mainly comes from one footpoint of a
solar flare loop. The slit of Interface Region Imaging Spectrograph (IRIS)
stays at this footpoint during this solar flare. The Doppler velocities of Fe
XXI  1354.09 Å and Si IV  1402.77 Å are extracted from the Gaussian fitting
method. We find that the hot line of Fe XXI  1354.09 Å (log$T\sim$7.05) in
corona exhibits blue shift, while the cool line of Si IV  1402.77 Å
(log$T\sim$4.8) in transition region exhibits red shift, indicating explosive
chromospheric evaporation. The evaporative upflows along the flare loop are
also observed in the AIA 131 Å image. To our knowledge, this is the first
report of chromospheric evaporation evidence from both spectral and imaging
observations in the same flare. Both microwave and HXR pulses are well
correlated with the Doppler velocities, suggesting that the chromospheric
evaporation is driven by nonthermal electrons around this footpoint of a solar
flare loop.

Sun: flares — Sun: UV radiation — Sun: radio radiation — Sun: X-rays, gamma
rays — line: profiles

##  1 Introduction

In a typical solar flare, the released energy is about 1030 erg within dozens
of minutes. Such huge energy is thought to be transferred from the magnetic
energy via reconnection, which is thought to heat the plasma and to accelerate
the bi-directional nonthermal electrons through the solar chromosphere,
transition region and corona. This is well known as the standard solar flare
model. These accelerated electrons are guided by the newly reconnected
magnetic field lines. Some of the accelerated electrons travel up to the
inter-planetary space, and the others propagate downward to the lower corona
and upper chromosphere, where they lose their energies through Coulomb
collisions with the denser plasma. Subsequently, two footpoint sources are
often produced along the flare loop legs in the hard X-ray (HXR) and microwave
bands (Brown, 1971; Asai et al., 2006), and the double ribbons are brightened
in H$\alpha$, ultraviolet (UV), and extreme-ultraviolet (EUV) wavelengths
(Czaykowska, 1999; Del Zanna et al., 2006; Milligan, 2015). Meanwhile, the
local chromospheric material is rapidly heated up to $\sim$10 MK (Antonucci et
al., 1982), resulting in a higher pressure, which drives the chromospheric
material upward into the corona along the solar flare loops. Thus the hot
plasma fills the solar flare loops in a process called ‘chromospheric
evaporation’ (Brown, 1973; Fisher et al., 1985a, b; Liu et al., 2006; Ning &
Cao, 2010; Brosius & Daw, 2015; Tian et al., 2015; Young et al., 2015; Zhang
et al., 2016; Lee et al., 2017). At the same time, the overpressure also
drives the denser plasma downward into the lower chromosphere in a process
called ‘chromospheric condensation’ (Fisher et al., 1985a, b; Teriaca et al.,
2006; Li et al., 2015).

Chromospheric evaporation is observed in multiple wavelengths, i.e., HXR,
EUV/UV, and microwave bands. Imaging observations show that the HXR emission
tends to rise with the double footpoint sources along the loop legs,
eventually merging them into a single source at the loop top (e.g., Liu et
al., 2006; Ning & Cao, 2010; Ning, 2011; Nitta et al., 2012). This process is
thought to be the HXR signature of chromospheric evaporation. On the other
hand, the heated materials rise upward to the corona and disturb the local
plasma. This process causes the microwave emission suddenly cut-off on the
dynamic spectra. Observationally, a high-frequency cut-off drifts to low-
frequency is the radio signature of chromospheric evaporation (Aschwanden &
Benz, 1995; Karlicky, 1998; Ning et al., 2009). Joint observations from
spectra and images show that the blue shift of hot lines in corona tends to
appear at the outside of flare ribbon (Czaykowska, 1999; Li & Ding, 2004; Li
et al., 2015).

Chromospheric evaporation is divided into two types, i.e., ‘explosive’ and
‘gentle’. Explosive evaporation takes place if the input energy flux exceeds a
critical value of $\sim$1010 erg cm-2 s-1 (Fisher et al., 1985a, b). In
spectroscopic observations, explosive evaporation is usually accompanied by
blue shift with high speed ($\sim$100$-$400 km s-1) in the hot lines from
corona, and by red shift with low speed ($\sim$10$-$40 km s-1) in the cool
lines from upper chromosphere and transition region (Feldman et al., 1980;
Ding et al., 1996; Del Zanna et al., 2006; Teriaca et al., 2006; Milligan &
Dennis, 2009; Veronig et al., 2010; Chen & Ding, 2010; Brosius, 2013; Doschek
et al., 2013; Tian et al., 2014; Brosius & Daw, 2015; Brosius et al., 2016;
Zhang et al., 2016). Observationally, the speed of blue shift is an order of
magnitude larger than that of red shift, which is due to the fact that the
density of the overlying corona is less than that of the underlying lower
chromosphere (Fisher et al., 1985a, b; Milligan & Dennis, 2009; Doschek et
al., 2013). Gentle evaporation occurs when the input energy flux is smaller
than the critical value, and all the emission lines display blue shift from
chromosphere through transition region to corona (Milligan et al., 2006;
Brosius, 2009; Li & Ding, 2011).

In the literatures, there are three mechanisms to drive chromospheric
evaporation. The first one is electron-driven, which emphases that the
nonthermal electrons accelerated by magnetic reconnection play an important
role in driving chromospheric evaporation (Fisher et al., 1985b; Brosius,
2003; Milligan & Dennis, 2009; Tian et al., 2014, 2015; Li et al., 2015; Zhang
et al., 2016). The second one focusses on the thermal energy (Fisher et al.,
1985a; Falewicz et al., 2009). While the third one is driven by the energy
deposition through Alfvén waves (Reep & Russell, 2016). The correlation
between Fe XXI 13540.9 Å Doppler velocity and HXR emission from the whole
flare region has already been documented in the literatures (Li et al., 2015;
Tian et al., 2015; Lee et al., 2017). In this paper, using the observations
from Interface Region Imaging Spectrograph (IRIS, De Pontieu et al., 2014),
NoRH (Hanaoka et al., 1994), Reuven Ramaty High Energy Solar Spectroscopic
Imager (RHESSI, Lin et al., 2002), Atmospheric Imaging Assembly (AIA, Lemen et
al., 2012) and Helioseismic and Magnetic Imager (HMI, Schou et al., 2012)
aboard Solar Dynamics Observatory (SDO), we study the temporal relationship
between the microwave/HXR emission and Doppler velocity of the SOL2014-10-27
event to prove the nonthermal electrons driving chromospheric evaporation
around one footpoint of a solar flare loop.

##  2 Data Analysis and Results

The studied event takes place in NOAA AR12192 on 2014 October 27. It is an
M7.1 flare in GOES light curve. It displays typical features of solar flares,
such as two footpoint sources in microwave and HXR images. The slit of IRIS
stays at one footpoint during the impulsive phase of the solar flare, which
gives us a good chance to study the temporal evolution of the Doppler velocity
in coronal and transition region lines, i.e., Fe XXI  1354.09 Å and Si IV
1402.77 Å. Figure 1 shows the GOES SXR fluxes in 1.0$-$8.0 Å (black) and
0.5$-$4.0 Å (turquoise), respectively. The event starts at around 00:06 UT,
and peaks at about 00:34 UT from GOES SXR light curves. Meanwhile, the
microwave 34 GHz (purple) and HXR 25$-$50 keV (orange) emissions display a
series of regular and periodic peaks during the impulsive phase of this solar
flare, which result in several nonthermal pulses (e.g., Li et al., 2017), as
marked by the Roman Capitals ‘I’, ‘II’. The first pulse (‘I’) is studied in
this paper.

Figure 2 shows the multi-wavelength images from several instruments during the
first pulse. They have the same field-of-view (FOV) of 90″$\times$90″. Panel
(a) displays the integral intensity image ($\sim$2.45″/pixel) in microwave 34
GHz from 00:09 UT to 00:10 UT. The double footpoint sources are marked by the
white boxes. Among them, one is bright (ft1), while the other one is faint
(ft2). This is confirmed by RHESSI CLEAN image ($\sim$2″/pixel) at 25$-$50
keV, as shown in panel (b). The contour of ft2 is from the microwave emission.
Panel (c) gives the line-of-sight (LOS) magnetogram ($\sim$0.6″/pixel)
overlaid with the AIA 1600 Å contours. The bottom panels show the EUV/UV
snapshots in SDO/AIA 131 Å and 1600 Å ($\sim$0.6″/pixel), IRIS/SJI 1330 Å
($\sim$0.33″/pixel). The AIA 1600 Å image is applied to co-align with the SJI
1330 Å image (Cheng et al., 2015; Tian et al., 2015; Li et al., 2015), because
they both contain continuum emission around the temperature minimum which is
dominant in many bright features. The flare ribbons in AIA 1600 Å and SJI 1330
Å are connected by the hot loops in AIA 131 Å, as indicated by the dashed
lines. Consistent with the standard solar flare model, panel (e) shows that
the double footpoint sources overlap on the two flare ribbons. The slit of
IRIS crosses one flare ribbon and fixes on the edge of one footpoint (ft1), as
shown by the long blue line along the 45 degree to the North-South direction.

The hot line of Fe XXI  1354.09 Å (log $T$$\sim$7.05, De Pontieu et al., 2014)
in corona and the cool line of Si IV  1402.77 Å (log $T$$\sim$4.8, De Pontieu
et al., 2014) in transition region are widely used to investigate
chromospheric evaporation (Tian et al., 2014, 2015; Li et al., 2015; Zhang et
al., 2016). Figure 3 shows the IRIS observation at spectral windows of ‘Fe XXI
’ (a, c, e) and ‘Si IV ’ (b, d, f) at three times. The Y-axis is along the
slit direction. They have been pre-processed with the routines of
‘iris_orbitval_corr_l2.pro’ (Tian et al., 2014; Cheng et al., 2015) and
‘iris_prep_despike.pro’ (De Pontieu et al., 2014) in Solar Soft Ware (SSW).
The absolute wavelength calibration is also manually performed with the
relatively strong neutral lines, i.e., ‘O I ’ 1355.5977 Å and ‘S I ’ 1401.5136
Å (e.g., Tian et al., 2015). The overplotted black profile shows the spectrum
at the position of about 38.6″, which is marked by the short purple line on
the left-hand side. This position is located at the edge of a flare ribbon in
the northwest labeled with the short blue line in Figure 2 (e). The multi-
Gaussian functions (purple profile) superimposed on a linear background (green
line) are used to fit the IRIS spectrum at ‘Fe XXI ’ window to extract the
flare line of Fe XXI  1354.09 Å. The emission lines which are blended with the
broad line of Fe XXI  1354.09 Å should be excluded, i.e., C I line at 1354.29
Å, Fe II lines at 1353.02 Å, 1354.01 Å, and 1354.75 Å, Si II lines at 1352.64
Å and 1353.72 Å, some unidentified lines at 1353.32 Å and 1353.39 Å (Polito et
al., 2015; Young et al., 2015; Tian et al., 2015). Here, we fixed these
blended line positions, constrained their widths, and tied their intensities
to the lines in other spectral windows (e.g., Li et al., 2015, 2016). Finally,
we can obtain the line profile of Fe XXI 1354.09 Å, as shown by the turquoise
line. The transition region line of Si IV  1402.77 Å is fitted with a single-
Gaussian function (purple profile) with a linear background (green line). Both
the Doppler velocities of Fe XXI  1354.09 Å and Si IV  1402.77 Å are detected
by the fitting line centers subtracting the referenced line centers (Li et
al., 2014; Tian et al., 2015).

To exactly study the relationship between the microwave/HXR emission and the
evaporation speed, Figure 4 (a) shows the normalized flux in microwave (black)
and HXR (green) bands, and the temporal evolution of Doppler velocity in Fe
XXI  1354.09 Å (blue) and Si IV  1402.77 Å (red) on the slit position of 38.6″
(short purple line in Figure 3). The hot line of Fe XXI  1354.09 Å in corona
displays blue shift, while the cool line of Si IV  1402.77 Å in transition
region exhibits red shift, which indicates the explosive chromospheric
evaporation around the footpoint of ft1. The maximum velocities of blue and
red shifts are about -67 km s-1 and 36 km s-1 during the first pulse between
00:08$-$00:11 UT. The microwave light curves are solid (black) at ft1 and
dashed (black) at ft2. They are integrated over the regions in the white boxes
in Figure 2 (a), respectively. The observations show that the pulse (between
00:08$-$00:11 UT) studied here completely arise from ft1, and the microwave
emission at ft2 is monotonously increasing. However, the HXR flux is
integrated over the whole flare region, which is the same as that in Figure 1.
These light curves are well correlated during the pulse interval, but their
different time cadences make the points impossible to correspond one-by-one.
NoRH, RHESSI and IRIS have the cadences of 1 s, 4 s and 16.2 s, respectively.
In this paper, we use all 9 points from the Doppler velocity around the pulse
(between 00:08:30 UT and 00:11:00 UT), and their nearby points of microwave
and HXR emissions. Thus, 9 points are extracted from all the light curves, as
marked by the purple symbols (‘+’ or ‘$\times$’).

Figure 4 (b) shows the Doppler velocities of Fe XXI  1354.09 Å (‘+’) and Si IV
1402.77 Å (‘$\times$’) dependence on microwave 34 GHz (black) and HXR 25$-$50
keV (purple) emissions during the pulse interval, i.e., between 00:08:30 UT
and 00:11:00 UT. As expected from the model of electron-driven chromospheric
evaporation, we find a negative correlation (-0.96/-0.87) between the
microwave/HXR emission and the Doppler velocity of hot line (Fe XXI  1354.09
Å) at ft1 (‘-’ indicates that the Doppler velocity of Fe XXI  1354.09 Å is
blue shift), and a positive correlation (+0.91/+0.82) between the
microwave/HXR emission and the Doppler velocity of cool line (Si IV  1402.77
Å, ‘+’ indicates that the Doppler velocity of Si IV  1402.77 Å is red shift).
Such high correlation coefficients suggest that the nonthermal electrons which
are accelerated by the magnetic reconnection drive the explosive chromospheric
evaporation around the footpoint of ft1.

Based on the chromospheric evaporation process, the plasma upflows are
expected to be observed along the flaring loops. Figure 2 (d) shows that the
loops (red dashed lines) in AIA 131 Å connect the double ribbons in AIA 1600
Å. These loops become brighter and brighter with the flare development.
Meanwhile, these loops expand with time. Figure 5 (a) shows the time
difference image in AIA 131 Å between the pulse maximum ($\sim$00:09:32 UT)
and the flare beginning ($\sim$00:06:08 UT). The two purple lines outline the
expansion loop which covers the hot plasma upflows due to chromospheric
evaporation. Figure 5 (b) gives its time-distance image, and the Y-axis from
bottom to top corresponds to the loop from ft2 to ft1. Thus the loop top is
near the middle position in Y-axis, i.e., $\sim$41″. It clearly shows the
evaporation upflows from double footpoints through legs to loop top, as
indicated by the purple arrows during 00:08:30$-$00:12:00 UT, which is
consistent with the microwave and HXR pulse (‘I’). The evaporation speeds are
estimated to be $\sim$112 km s-1 and $\sim$99 km s-1 without considering the
projection effect.

##  3 Conclusions and Discussions

Based on the observations from IRIS, NoRH, and RHESSI, we explore the temporal
relationship between Doppler velocity of Fe XXI  1354.09 Å and Si IV  1402.77
Å lines and microwave/HXR emission in microwave 34 GHz and HXR 25$-$50 keV
during a pulse of solar flare on 2014 October 27. The completely new
observational result is that the explosive chromospheric evaporation driven by
nonthermal electrons originates from one footpoint of a solar flare loop.
Meanwhile, the evaporation upflows are also observed from the double
footpoints to loop top in AIA 131 Å image. This is consistent well with the
standard solar flare model. Our results agree well with previous findings
about the temporal correlation between the Doppler velocity and HXR flux which
represents the deposition rate (Li et al., 2015; Tian et al., 2015; Lee et
al., 2017), and are also consistent well with the studies of the spatial
correlation between the upflows/dowflows and HXR sources in solar flares
(Brosius & Holman, 2009; Milligan & Dennis, 2009; Veronig et al., 2010;
Brosius et al., 2016; Zhang et al., 2016). It has been recently demonstrated
that the explosive evaporation can also be driven by the dissipation through
Alfvén waves, and this mechanism can produce the HXR bursts (Reep & Russell,
2016). Therefore, only the presence of an HXR burst is impossible to rule out
the contribution from the heating of Alfvén waves. On the other hand, a
microwave pulse is observed around the peak time of IRIS Doppler velocity at
one footpoint of a solar flare loop, implying that the nonthermal electrons
are injected into the chromosphere along the flare loop in the Sun. Thus, the
observations in this case support an electron-driven chromospheric
evaporation.

Figure 2 (e) and (f) shows that the slit of IRIS crosses one of flare ribbons.
The studied position of Doppler velocity is just at the edge of flare ribbon,
indicating that the explosive chromospheric evaporation appears at the outside
of flare ribbon, which is consistent well with previous findings (Czaykowska,
1999; Li & Ding, 2004; Li et al., 2015; Tian et al., 2015). On the other hand,
Figure 2 (a) exhibits that the slit of IRIS does not appear in the footpoint
center, which may be due to the lower spatial resolution of microwave image
($\sim$4.9$\arcsec$). Future work needs higher resolution observations in
microwave or HXR bands, including the time and spatial resolutions.

In this letter, only the first pulse (‘I’) in microwave and HXR emissions is
used to investigate the explosive chromospheric evaporation. We did not find
any correspondence between the microwave (or HXR) emission and Doppler
velocity during the other pulses, i.e., the second microwave pulse at about
00:13 UT (‘II’). This may be because the upflows/downfows of the other
microwave pulses are very complex. The upflows are from the new heated plasma
which moves upward along the loop, while the downflows from the last
evaporated material may eventually cool and precipitate back down along the
loop (e.g., Brosius, 2003; Milligan & Dennis, 2009). It could also be because
the location of energy deposition is not covered by the slit of IRIS. The good
correlation between Doppler velocity and microwave/HXR emission may be found
when shifting the slit position (e.g., Li et al., 2015).

The authors would appreciate the anonymous referee for his/her valuable
comments and suggestions to improve the manuscript. We thank the teams of
IRIS, GOES, NoRH, RHESSI, SDO/AIA and it SDO/HMI for their open data use
policy. The authors also thank Y. Gong for reading our manuscript. This study
is supported by NSFC under grants 11603077, 11573072, 11473071, 11333009,
KLSA201708, and Laboratory No. 2010DP173032. Dr. D. Li is also supported by
the Youth Fund of Jiangsu No. BK20161095, and Dr. Q. M. Zhang is supported by
the Surface Project of Jiangsu No. BK 20161618 and the Youth Innovation
Promotion Association CAS.

## References

  * Antonucci et al. (1982) Antonucci, E., Gabriel, A. H., Acton, L. W., et al. 1982, Sol. Phys., 78, 107
  * Aschwanden & Benz (1995) Aschwanden, M. J., & Benz, A. O. 1995, ApJ, 438, 997
  * Asai et al. (2006) Asai, A., Nakajima, H., Shimojo, M., et al. 2006, PASJ, 58, L1
  * Brosius (2003) Brosius, J. W. 2003, ApJ, 586, 1417
  * Brosius (2009) Brosius, J. W. 2009, ApJ, 701, 1209
  * Brosius & Holman (2009) Brosius, J. W., & Holman, G. D. 2009, ApJ, 692, 492
  * Brosius (2013) Brosius, J. W. 2013, ApJ, 762, 133
  * Brosius & Daw (2015) Brosius, J. W., & Daw, A. N. 2015, ApJ, 810, 45
  * Brosius et al. (2016) Brosius, J. W., Daw, A. N., & Inglis, A. R. 2016, ApJ, 830, 101
  * Brown (1971) Brown, J. C. 1971, Sol. Phys., 18, 489
  * Brown (1973) Brown, J. C. 1973, Sol. Phys., 31, 143
  * Chen & Ding (2010) Chen, F., & Ding, M. D. 2010, ApJ, 724, 640
  * Cheng et al. (2015) Cheng, X., Ding, M. D., & Fang, C. 2015, ApJ, 804, 82
  * Czaykowska (1999) Czaykowska, A., & De Pontieu, B., & Alexander, D., & Rank, G. 1999, ApJ, 521, L75
  * De Pontieu et al. (2014) De Pontieu, B., & Title, A. M., & Lemen, J. R., et al. 2014, Sol. Phys., 289, 2733
  * Del Zanna et al. (2006) Del Zanna, G., & Schmieder, B., & Mason, H., & Berlicki, A., & Bradshaw, S. 2006, Sol. Phys., 234, 95
  * Ding et al. (1996) Ding, M. D., & Watanabe, T., & Shibata, K., et al. 1996, ApJ, 458, 391
  * Doschek et al. (2013) Doschek, G. A., & Warren, H. P., & Young, P. R. 2013, ApJ, 767, 55
  * Falewicz et al. (2009) Falewicz, R., & Rudawy, P., & Siarkowski, M. 2009, A&A, 508, 971
  * Feldman et al. (1980) Feldman, U., & Doschek, G. A., & Kreplin, R. W., & Mariska, J. T. 1980, ApJ, 241, 1175
  * Fisher et al. (1985a) Fisher, G. H, & Canfield, R. C., & McClymont, A. N. 1985a, ApJ, 289, 414
  * Fisher et al. (1985b) Fisher, G. H, & Canfield, R. C., & McClymont, A. N. 1985b, ApJ, 289, 425
  * Hanaoka et al. (1994) Hanaoka, Y., Shibasaki, K., Nishio, M., et al. 1994, Proceedings of Kofu Symposium, 35
  * Karlicky (1998) Karlicky, M. 1998, A&A, 338, 1084
  * Lee et al. (2017) Lee, K.-S., Imada, S., Watanabe, K., Bamba, Y., & Brooks, D. H. 2017, ApJ, 836, 150
  * Lemen et al. (2012) Lemen, J. R., & Title, A. M., & Akin, D. J., et al. 2012, Sol. Phys., 275, 17
  * Li & Ding (2011) Li, Y., & Ding, M. D. 2011, ApJ, 727, 98
  * Li et al. (2014) Li, L. P., Peter, H., Chen, F., & Zhang, J. 2014, A&A, 570, A93
  * Li et al. (2015) Li, D., Ning, Z. J., & Zhang, Q. M. 2015, ApJ, 813, 59
  * Li et al. (2016) Li, D., Innes, D. E., & Ning, Z. J. 2016, A&A, 587, A11
  * Li et al. (2017) Li, D., Zhang, Q. M., Huang, Y., Ning, Z. J., & Su, Y. N. 2017, A&A, 597, L4
  * Li & Ding (2004) Li, J. P., & Ding, M. D. 2004, ApJ, 606, 583
  * Lin et al. (2002) Lin, R. P., & Dennis, B. R., & Hurford, G. J., et al. 2002, Sol. Phys., 210, 3
  * Liu et al. (2006) Liu, W., & Liu, S. M., & Jiang, Y. W., et al. 2006, ApJ, 649, 1124
  * Milligan et al. (2006) Milligan, R. O., & Gallagher, P. T., & Mathioudakis, M., et al. 2006, ApJ, 642, L169
  * Milligan & Dennis (2009) Milligan, R. O., & Dennis, B. R. 2009, ApJ, 699, 968
  * Milligan (2015) Milligan, R. O. 2015, Sol. Phys., 290, 3399
  * Ning et al. (2009) Ning, Z. J., & Cao, W. D., & Huang, J., et al. 2009, ApJ, 699, 15
  * Ning & Cao (2010) Ning, Z. J., & Cao, W. D. 2010, ApJ, 717, 1232
  * Ning (2011) Ning, Z. J. 2011 Sol. Phys., 273, 81
  * Nitta et al. (2012) Nitta, S., Imada, S., & Yamamoto, T. T. 2012, Sol. Phys., 276, 183
  * Polito et al. (2015) Polito, V., Reeves, K. K., Del Zanna, G., Golub, L., & Mason, H. E. 2015, ApJ, 803, 84
  * Schou et al. (2012) Schou, J., Scherrer, P. H., Bush, R. I., et al. 2012, Sol. Phys., 275, 229
  * Reep & Russell (2016) Reep, J. W., & Russell, A. J. B. 2016, ApJ, 818, L20
  * Teriaca et al. (2006) Teriaca, L., & Falchi, A., & Falciani, R., & Cauzzi, G., & Maltagliati, L. 2006, A&A, 455, 1123
  * Tian et al. (2014) Tian, H., & Li, G., & Reeves, K. E., et al. 2014, ApJ, 797, L14
  * Tian et al. (2015) Tian, H., Young, P. R., Reeves, K. K., et al. 2015, ApJ, 811, 139
  * Veronig et al. (2010) Veronig, A. M., & Rybák, J., & Gömöry, P., et al. 2010, ApJ, 719, 655
  * Young et al. (2015) Young, P. R., Tian, H., & Jaeggli, S. 2015, ApJ, 799, 218
  * Zhang et al. (2016) Zhang, Q. M., Li, D., Ning, Z. J., et al. 2016, ApJ, 827, 27

Figure 1: SXR light curves of 2014 October 27 flare from 00:06 UT to 00:40 UT
in 1.0$-$8.0 Å (black) and 0.5$-$4.0 Å (turquoise). Normalized flux in
microwave of 34 GHz (purple) and HXR 25$-$50 keV (orange) from the whole flare
region during the same time intervals.  Figure 2: The multi-wavelength images
with a FOV of 90″$\times$90″on the 2014 October 27 flare. The purple contours
are from the microwave of 34 GHz, and the white boxes indicate the sources of
two footpoints. The green contours represent the AIA 1600 Å intensities at the
scale of 200 DN s-1. The red dashed lines in AIA 131 Å outline the possible
flare loops. The long blue line represents the slit of IRIS and the short blue
line marks the region studied here.  Figure 3: IRIS spectra at ‘Fe XXI ’
(left) and ‘Si IV ’ (right) windows on 2014 October 27. The black profile is
the line spectrum across the slit position marked by a purple line on the
left-hand side of each image, the overplotted purple profile represents the
fitting results. The turquoise line is Fe XXI  1354.09 Å, and the orange line
is C I  1354.29 Å. The green line is the fitting background.  Figure 4: Panel
(a): The microwave emission in NoRP 34 GHz at ft1 (solid black) and ft2
(dashed black) during 00:06$-$00:14 UT. The HXR flux in RHESSI 25$-$50 keV
(green) is from the whole flare at the same time. The temporal evolution of
Doppler velocities at Fe XXI  1354.09 Å (blue) and Si IV  1402.77 Å (red)
lines are detected from a distinct position along the slit of IRIS. Panel (b):
Scatter plots of Doppler velocities dependence on microwave and HXR emissions
during the first pulse. The correlation coefficients (cc) are labeled.  Figure
5: Upper: The difference map in AIA 131 Å, two solid lines (purple) outline a
flare loop. Bottom: The time-distance image along flare loop in AIA 131 Å. The
start point (d=0) is from ft2. The arrows indicate the upflow directions. Two
dotted lines mark the upflow time between 00:08:30$-$00:12:00 UT.


# GraphNLI: A Graph-based Natural Language Inference Model for Polarity
Prediction in Online Debates

Vibhor Agarwal  Department of Computer Science
University of SurreyP.O. Box 1212GuildfordSurreyUnited KingdomGU2 7XH
<EMAIL_ADDRESS>,  Sagar Joglekar  Department of Informatics
King’s College LondonBush House, Strand Campus
30 Aldwych, WC2B 4BGLondonUnited Kingdom<EMAIL_ADDRESS>,  Anthony
P. Young  Department of Informatics
King’s College LondonBush House, Strand Campus
30 Aldwych, WC2B 4BGLondonUnited Kingdom<EMAIL_ADDRESS>and  Nishanth
Sastry  Department of Computer Science
University of SurreyP.O. Box 1212GuildfordSurreyUnited KingdomGU2 7XH
<EMAIL_ADDRESS>
(2022)

###### Abstract.

Online forums that allow participatory engagement between users have been
transformative for public discussion of important issues. However, debates on
such forums can sometimes escalate into full blown exchanges of hate or
misinformation. An important tool in understanding and tackling such problems
is to be able to infer the argumentative relation of whether a reply is
supporting or attacking the post it is replying to. This so called polarity
prediction task is difficult because replies may be based on external context
beyond a post and the reply whose polarity is being predicted. We propose
GraphNLI, a novel graph-based deep learning architecture that uses graph walk
techniques to capture the wider context of a discussion thread in a principled
fashion. Specifically, we propose methods to perform root-seeking graph walks
that start from a post and captures its surrounding context to generate
additional embeddings for the post. We then use these embeddings to predict
the polarity relation between a reply and the post it is replying to. We
evaluate the performance of our models on a curated debate dataset from Kialo,
an online debating platform. Our model outperforms relevant baselines,
including S-BERT, with an overall accuracy of 83%.

Online debates, argument mining, polarity prediction, Kialo

††copyright: acmcopyright††journalyear: 2022††conference: Proceedings of the
ACM Web Conference 2022; April 25–29, 2022; Virtual Event, Lyon,
France††booktitle: Proceedings of the ACM Web Conference 2022 (WWW ’22), April
25–29, 2022, Virtual Event, Lyon, France††price: 15.00††doi:
10.1145/3485447.3512144††isbn: 978-1-4503-9096-5/22/04††ccs: Computing
methodologies Natural language processing††ccs: Computing methodologies
Machine learning algorithms††ccs: Computing methodologies Model development
and analysis††ccs: Computing methodologies Information extraction††ccs:
Information systems World Wide Web

##  1\. Introduction

The Internet has enabled people to participate in sharing their views, often
as comments or posts, about many topics online. Online debates can sometimes
become large and acrimonious, with some escalating into full blown exchanges
of hate and misinformation. As many of these debates concern topics of
societal importance, it is crucial to be able to model these debates
accurately and at scale, so that we can better understand and control
phenomena such as the spread of hate (Cinelli et al., 2021; Köffer et al.,
2018), fake news (Allcott and Gentzkow, 2017; Hanselowski et al., 2018), how
best to moderate political polarisation (Bail et al., 2018), and how to break
echo chambers by linking appropriate users of opposing views (Garimella et
al., 2017).

An important task in modelling online debates is to be able to predict whether
the reply of one comment to another is attacking (disagreeing) or supporting
the post it is replying to. This is called the polarity of the reply (Cayrol
and Lagasquie-Schiex, 2005). The ability to accurately predict the polarity of
replies in an online debate can allow us to measure properties of the debate,
such as how “controversial” a discussion is, _e.g._ by counting the number of
supporting vs. attacking replies (Boschi et al., 2021). Perhaps more
importantly, if the polarity is known, we can then use techniques from
argumentation theory, a branch of artificial intelligence concerned with the
formal representation and resolution of disagreements (Rahwan and Simari,
2009), to formally compute which arguments have been attacked and rebutted,
and which ones stand unrebutted or are further justified by additional
supporting replies.

One obvious approach to predicting the polarity of replies in any debate is to
apply natural language processing (NLP) techniques. Such models typically take
as input the natural language text of a comment $b$ and the comment $a$ that
it is replying to, and output a predicted polarity for whether $b$ is
attacking or supporting $a$ (_e.g._ (Cabrio and Villata, 2013; Cocarascu and
Toni, 2017)). However, one shortcoming of such an approach is that it risks
loosing crucial information by considering the comments $a$ and $b$ in
isolation from the rest of the debate.

We call the posts $a$, $b$ as the local context for the polarity prediction
task and the whole discussion thread in which these comments are embedded as
the global context. This paper asks and answers the question: Can we improve
performance on the polarity prediction task by incorporating additional
context beyond the posts $a$, $b$?

Typically, discussion threads can be seen as a tree, starting with an original
post which forms the root of the tree and each reply such as $b$ creating a
directed edge in the tree, from $b$ to its parent node $a$ that it is replying
to. We hypothesise that nodes near $a$ and $b$, _e.g._ their children,
ancestors and siblings in the discussion tree, contain additional information
that may help understand whether $b$ is attacking $a$ or supporting it. For
example, if other siblings of $b$ (_i.e._ children of $a$ other than $b$) are
also attacking $a$, then it may be more likely that $b$ is also an attacking
reply. Our key idea is to use graph walk techniques to discover and utilise
neighbouring context in a principled fashion. Our contributions are as
follows:

  1. (1)

We compare and contrast several NLP models, including Sentence-BERT (Reimers
and Gurevych, 2019), to establish a baseline for the polarity prediction task.
We use data from Kialo, an online debating platform111See
https://www.kialo.com/, last accessed 17 October 2021.; this data is in the
form of discussion trees where the nodes are arguments submitted to the
debate222We will use the terms “node”, “post”, “comment” and “argument”
interchangeably. and the edges denote which arguments reply to which other
arguments, as will be explained in Section 4.1.

  2. (2)

We propose graph walk techniques that sample discussion trees with the aim of
capturing parts of the global context of the online debates, and input the
additional nodes sampled into deep learning model along with the local context
of the replying argument and the argument being replied to. We find that a
weighted root-seeking graph walk works the best in capturing the wider context
of the debates on Kialo.

  3. (3)

We present and evaluate GraphNLI – a novel graph-based deep learning
architecture to predict the polarity of replies. Our model outperforms
baselines including S-BERT, achieving an accuracy of $83\%$. We provide an
open source implementation of the model available for public usage333The model
code is available at https://github.com/socsys/GraphNLI and the dataset is
available at https://tinyurl.com/kialo-debates-dataset..

  4. (4)

We systematically investigate through ablation studies what features can be
helpful in capturing the wider context for the polarity prediction task and
show that upstream (or earlier) text, _i.e._ , the parent and other ancestor
nodes, help the model more than siblings and children replies. Moreover, we
find that the importance of ancestor nodes decreases as their distance from
the given node increases.

The rest of this paper is structured as follows. Sec. 2 provides an overview
of the polarity prediction task in the context of argumentation theory. In
Sec. 3, we discuss the details of Kialo dataset. We next consider different
ways of incorporating the wider context of discussions in Sec. 4, and present
the GraphNLI architecture. We then evaluate the model in Sec. 5, comparing its
performance relative to a range of baseline classifiers and conducting an
ablation study to better understand which features are important. In Sec. 6,
we summarize our results, and outline possible future work.

##  2\. Background and Related Work

Argumentation theory is a branch of AI that is concerned with the transparent
and rational resolution of disagreements (e.g. (Rahwan and Simari, 2009)).
Many formal models of argumentation have been devised and studied, starting
with abstract argumentation theory (Baroni et al., 2011; Dung, 1995; Young,
2018), where the arguments of interest are elements of a set $A$, and a not-
necessarily-symmetric binary relation $R\subseteq A\times A$ \- the attack
relation \- represents when two arguments disagree, i.e. $(b,a)\in R$ denotes
that argument $b$ disagrees with argument $a$. The directed graph (digraph)
$\left\langle{A,R}\right\rangle$ is called an abstract argumentation framework
(AF); this abstracts away from the structure of the arguments and the nature
of their disagreements. Resolving these disagreements formally amounts to
identifying subsets of arguments $S\subseteq A$ that satisfy various normative
properties. For example, the property of conflict-freeness, $(S\times S)\cap
R=\varnothing$, formalises the idea of self-consistency because winning
arguments should not attack each other.

Of course, arguments can agree as well as disagree. Bipolar argumentation
frameworks (BAFs) enrich AFs with a support relation that models when
arguments agree (Cayrol and Lagasquie-Schiex, 2005). Formally, a BAF is a
digraph $\left\langle{A,R_{att},R_{sup}}\right\rangle$ where
$R_{att},R_{sup}\subseteq A\times A$ are two disjoint binary relations
respectively denoting attack and support between arguments - every edge of a
BAF is either attacking or supporting but never both. To determine the winning
arguments, there are various ways to transform a BAF into an AF by “absorbing”
supports into attacks (e.g. (Cayrol and Lagasquie-Schiex, 2013)) depending on
how supports are interpreted. Following this literature, given an edge in a
BAF, we call the status of whether this edge is attacking or supporting its
polarity.

Both AFs and BAFs are theoretically interesting and mathematically elegant,
and offer a normative and perhaps intuitive way of identifying winning
arguments in the presence of agreement and disagreement (Cayrol and Lagasquie-
Schiex, 2005; Dung, 1995; Rahwan et al., 2010). However, in order to apply
these models to reason about online debates, we need to map the concepts
identified in these models to their real-life counterparts. Argument mining
(e.g. (Lawrence and Reed, 2020; Lippi and Torroni, 2016; Cabrio and Villata,
2018)) is the application of NLP tools to extract arguments and identify their
relationships from raw text. Example tasks involve identifying when Tweets
from Twitter are well-defined arguments instead of insults, single URLs or
pictures (e.g. (Bosc et al., 2016)), identifying the claims, their reasons and
relationships between claims from clinical trials to inform medical decision
making (e.g. (Mayer et al., 2021)), or detecting fallacies from the
transcripts of the United States Presidential Debates (Villata, 2021). Online
debates, such as those on Reddit or Twitter for example, can be converted into
the argumentation framework by first defining nodes or arguments corresponding
to each well-defined logical claim, and drawing signed edges representing
supports or attacks between them. For instance, when a comment $b$ replies to
another comment $a$, we may have two nodes $a,b$ linked by an edge
representing a support or an attack depending on the polarity relation between
them. The result, therefore, is to formally represent an online debate as a
BAF.

Many other analyses can be performed once a debate has been represented as a
BAF. For instance, based on the polarities of the edges, we may calculate
which arguments are justified and which arguments have been rebutted. This
could potentially be used to present only the justified arguments as summary
to a reader. Previous work has also looked at how the conclusions of a logical
reader can change depending on which parts of a debate they read, thus
underscoring the dangers of sampling only parts of a large online debate
(Young et al., 2018, 2021; Young, 2021). Other work has shown how the location
of justified arguments can be significantly influenced by whether the debate
is acrimonious or supporting (Boschi et al., 2021). BAFs are thus useful
representational tools for modelling online debates, allowing the application
of both argument-theoretic and graph-theoretic ideas to gain insights about
online discussions.

The polarity prediction task for online debates has been addressed in the
argument mining literature.444See (Cabrio and Villata, 2018) for reviews of
the polarity prediction task in other settings, such as persuasive essays or
political debates. An early example is (Cabrio and Villata, 2013), which
applied textual entailment (Bos and Markert, 2006; Dagan et al., 2010;
Kouylekov and Negri, 2010) to predict the polarity of replies on the now-
defunct Debatepedia dataset, with a test accuracy of $67\%$. In (Cocarascu and
Toni, 2017), long-short-term memory networks were used to classify polarity,
achieving $89\%$ accuracy.555Although this shows better results on the
polarity prediction task than what we report here, neither their data nor
their framework were available for benchmarking. A more recent overview of the
polarity prediction task (Cocarascu et al., 2020) has provided context-
independent baselines of neural network models using a range of learning
representations and architectures, and have found an averaged performance of
$51\%$ to $55\%$ of these different neural networks across such contexts;
these contexts involve online debates on controversial topics such as abortion
and gun rights, persuasive essays, and presidential debates.

In all of the above-mentioned approaches, the inputs to the model are the
texts of the replying argument and the argument being replied to, often
represented by some appropriate word embedding. Arguably, this is the least
amount of information one must input into the model to predict the polarity of
the reply. What has not yet been considered is whether it is helpful to input
more information. For example, if argument $c$ replies to argument $b$, and
$b$ replies to argument $a$, and we would like to predict the polarity
relation between $c$ and $b$, then is it useful to design a model that accepts
as input the texts of $c$, $b$ and $a$? How about if we randomly sample
additional “nearby” comments? What if we incorporate graph-theoretic features
into the input such as in-degree? To the best of our knowledge, these
questions have not yet been addressed in the argument mining literature. We
thus seek to investigate these questions by measuring whether the
incorporation of such features can improve the polarity prediction accuracy.

##  3\. Kialo Dataset

(a) An example of the arguments made in a Kialo debate. The thesis is,
“Pregnant people should have the right to choose abortion”. A supporting reply
is, “Access to legal abortion improves the health and safety of women”. An
attacking reply to this support is, “When abortion is easily available, it
incentivises irresponsible behaviour”.

(b) Visualization of this debate’s tree

Figure 1. Example of a Kialo discussion. Every debate on Kialo would start
with a thesis, which in this example is “Pregnant people should have the right
to choose abortion”. This thesis can be either supported (shown in green) or
attacked (shown in red), and the exchange can go on at multiple levels as seen
in Figure 1(b). Both figures are taken from https://www.kialo.com/when-
abortion-is-easily-available-it-incentivises-irresponsible-
behaviour-5637.1340?path=5637.0~5637.1_5637.10160-5637.1340&active=_5637.3912,
last accessed 25 Jan 2022.

Kialo is an online debating platform that helps people “engage in thoughtful
discussion, understand different points of view, and help with collaborative
decision-making”.666Quoted from https://www.kialo.com/about, last accessed 21
Oct 2021. In this study, we use data from discussions hosted on the Kialo
debating platform as used by (Boschi et al., 2021). Figure 1 illustrates an
example Kialo discussion. In a Kialo debate, users submit claims. The starting
claim of a debate is its thesis. To start a discussion in Kialo, the user
creates a thesis along with a tag that indexes the discussion by indicating
the content of the discussion. A thesis can have many tags, which increases
its visibility to the users. Then users comment on the discussions of their
choice, which as the discussion develops, takes the shape of a directed tree —
this is because each non-thesis claim replies to exactly one other claim. The
dataset contains $1,560$ discussion threads, and is the most complete snapshot
of Kialo, as of 28 January 2020.777We collected data continuously until a
change in the Kialo backend in January 2020 which made our crawler obsolete.
This was sufficient data to test GraphNLI, so no more data was collected. Each
discussion thread has data about the tree structure, votes on each argument’s
impact on the debate it has been submitted to, and the arguments’ texts.
Further, each reply between arguments is clearly labelled as attacking or
supporting. There is also supplemental metadata such as the time of posting,
the time of editing, and the author metadata. All discussions crawled from
Kialo thus have a tree structure with a root node that represents the main
thesis and each other node is a reply to its parent which either supports or
attacks the parent. On each topic, there is a reasonable amount of debate,
with a mean of $204$ and a median of $68$ arguments (standard deviation
$463$). Kialo debates are typically balanced, with the vast majority of
discussion trees having between $40\%$ to $60\%$ of its edges as supporting
edges, with the rest being attacking edges.

Due to Kialo’s strict moderation policy, each piece of text submitted to a
debate is a self-contained argument that has clear claim backed by reasoning.
Thus, each post in Kialo can be taken as a node and directed edges can be
drawn based on which post is replying to which other post. The polarity
prediction task is to decide whether these edges are attacking or supporting.

##  4\. The Polarity Prediction Model

As stated in Section 2, polarity prediction is an important task in argument
mining. It aims to identify the argumentative relations of attack and support
between natural language arguments by classifying pairs of text accordingly,
where in our case such texts are comments submitted to online debates, and one
text is replying to another text. Various deep learning models have been used
in the literature to perform the polarity prediction task. However, they
usually consider a pair of texts for the polarity prediction. In this section,
we propose a novel graph-based deep learning architecture that not only
considers as input the pair of texts, but also systematically captures the
context of nearby comments via graph walks.

###  4.1. Representing Debates as Discussion Trees

For every online debate D on Kialo, we construct a tree structure, where a
node represents a post and a directed edge from a node to its parent, the post
it is replying to. Each such edge has an associated label, support or attack,
depending upon whether the comment is respectively for or against its parent
comment. The root node of this discussion tree represents the thesis (topic)
of a debate.

###  4.2. GraphNLI Architecture

We propose a novel graph-based deep learning architecture which captures both
the local and the global context of the online debates through graph-based
walks. The GraphNLI architecture is shown in Figure 2, and will be explained
in Section 4.2.2.

Figure 2. GraphNLI Architecture

####  4.2.1. Capturing Global Context through Graph Walks

Our GraphNLI model captures the global context of online debates through
graph-based walks. A walk is defined as a sequence of nodes traversed from a
given node in a tree. We choose to ignore the direction of the edges in these
walks, to enable capturing context such as siblings of the reply node
(reachable by going up to the parent node and then back down again to the
sibling). Walk length $L\in\mathbb{N}$ is the maximum allowed length after
which the graph walk terminates. We propose the following strategies for graph
walks in discussion trees to capture the global and neighbouring context of a
given argument (node).

Weighted Root-seeking Graph Walk: In a discussion tree, a root-seeking graph
walk is a walk starting from a given node up towards the root of the
discussion tree. In a tree, there is exactly one path that goes upwards to the
root of the tree from a given node. Depending on the walk length $L$, the
graph walk terminates irrespective of whether or not it reaches the root node.
We experiment with different walk lengths and find that $L=5$ gives the best
results (see Section 5). The graph walk is weighted, which means that the
contributions of the ancestor nodes in the node’s embedding vector are
discounted by a factor of $\gamma^{k}$, where $k$ is the distance of the node
from the given node and $\gamma$ is the discount parameter. Therefore, the
highest weight is given to the node’s parent, then a discounted weight to the
parent’s parent, and so on. This means the closer the node is to the starting
node, the higher its weight will be, and the more it will contribute as an
input towards predicting the polarity.

###### Example 4.1.

Suppose we have a section of a discussion tree with arguments $a_{0}$,
$a_{1}$, $a_{2}$, $a_{3}$, $a_{4}$ and $a_{5}$, where the edges are
$(a_{k},a_{k+1})$ for $0\leq k\leq 4$, and $a_{k+1}$ is the parent of $a_{k}$.
As $L=5$, a weighted root-seeking graph walk from $a_{0}$ will sample
arguments $a_{1}$ to $a_{5}$. As for their weights, suppose we use a discount
parameter $\gamma=0.5$, we compute the weight of $a_{k}$ as $\gamma^{k}$.
Therefore, the weight of $a_{0}$ is $1$, the weight of $a_{1}$ is $0.5$, the
weight of $a_{2}$ is $0.25$, and so on.

###### Example 4.2.

Suppose we have a section of a discussion tree with arguments $a_{0}$,
$a_{1}$, $a_{2}$, where the edges are $(a_{k},a_{k+1})$ for $0\leq k\leq 2$,
and $a_{k+1}$ is the parent of $a_{k}$. Further, assume $a_{2}$ is the root of
the tree. As $L=5$, a weighted root-seeking graph walk from $a_{0}$ will
sample arguments $a_{1}$ and $a_{2}$, and no more as $a_{2}$ is the root of
the tree itself.

Biased Root-seeking Random Walk: This is a random walk starting from a given
node in the tree and is biased towards the root. From a given node, each
immediate neighbor is assigned a probability of being selected by the random
walk. Since the discussion thread is a tree, each node has one parent and zero
or more children. To bias the walk towards the root (or thesis node), we
assign a probability $p>0.5$ to the random walk selecting the parent of the
given node as the next step in the walk. Recall that we ignore the direction
of edges in these walks, allowing the walk to traverse downwards as well as
upwards in the discussion tree. Therefore, the remaining probability,
$1-p<0.5$, is divided equally among all the children of the given node. The
walk length $L\in\mathbb{N}$ determines the maximum length of this random
walk, that is, the number of nodes to be visited from the given node until the
random walk terminates. After experimentation, we find $L=4$ to be optimal
(see Section 5).

The random walk is a different way of sampling the neighbouring nodes as a
means of incorporating the surrounding global context for predicting the
polarity. Note that there is no guarantee that the parent node will be visited
in the random walk. By choosing the probability $p$, we can directly affect
the probability of visiting the parent. Empirically, we find that $p=0.75$
gives the best results. Even if the parent is not visited, there is likely to
be information in the surrounding nodes that still helps predict the polarity
of the parent-child relationship. For example, if the majority of children
nodes replying to the parent are attacking (perhaps because the parent post
makes a controversial or wrong statement), knowing the sibling context may
help predict the polarity of a reply.

###### Example 4.3.

Suppose we have a section of a discussion tree with nodes $a_{0}$, $a_{1}$,
$a_{2}$, $a_{3}$ and $a_{4}$, with edges $(a_{1},a_{0})$, $(a_{2},a_{1})$,
$(a_{3},a_{1})$ and $(a_{4},a_{1})$, such that for $(a_{i},a_{j})$, $a_{j}$ is
the parent of $a_{i}$. Let $p=0.75$, then $1-p=0.25$. A random walk starting
at $a_{1}$ will have probability $0.75$ moving to $a_{0}$ next. Similarly,
starting from $a_{1}$, there is a probability $\frac{0.25}{3}=\frac{1}{12}$ of
moving to any of $a_{2}$, $a_{3}$, or $a_{4}$ next.

Also, the same node can be visited multiple times in a random walk, especially
when the graph walk chooses one of the children nodes:

###### Example 4.4.

Suppose we have a section of a discussion tree with nodes $a_{0}$, $a_{1}$,
$a_{2}$, $a_{3}$ and $a_{4}$, with edges $(a_{1},a_{0})$, $(a_{2},a_{1})$,
$(a_{3},a_{1})$ and $(a_{4},a_{2})$, such that for $(a_{i},a_{j})$, $a_{j}$ is
the parent of $a_{i}$. Let $p=0.75$, then $1-p=0.25$. Assume that a random
walk, starting at $a_{1}$, moves to $a_{2}$ with probability
$\frac{0.25}{2}=\frac{1}{8}$. Now from $a_{2}$, the random walk can either
move to $a_{1}$ again with probability $0.75$ or to $a_{4}$ with probability
$0.25$.

At the end of a graph walk for each node, we obtain at most L arguments which
are either parents or neighbouring responses of a given node. We input these
sets of arguments into our GraphNLI model as described in the next section.

####  4.2.2. Model Overview

Our model is inspired by S-BERT (Reimers and Gurevych, 2019); its architecture
is shown in Figure 2. Firstly, each of the L arguments sampled by the graph
walk or the random walk is input into the RoBERTa model (Liu et al., 2019) to
get their corresponding embeddings and then, a mean-pooling operation, that
is, calculating the mean of all the output vectors, is applied to derive a
fixed-sized sentence embedding for each argument. The starting node in a graph
walk is a point-of-interest (PoI) node. Let $u$ denote the sentence embedding
corresponding to the PoI node. Let $v$ denote the aggregated embedding from
its neighbors (including parent) sampled by the graph walk starting from the
PoI node. These $u$ and $v$ embeddings together are used in GraphNLI model to
predict the polarity. We experiment with three aggregation strategies:
summation, average and weighted average. As stated in Section 4.2.1, nodes
sampled by a weighted root-seeking walk are weighted in the descending order
from the PoI node up to the root. The resultant sentence embeddings $u$ and
$v$ are concatenated with element-wise difference $|u-v|$ to get the final
embedding vector, which is then fed into a softmax classifier for the polarity
prediction task.

In order to fine-tune BERT, we make the GraphNLI model end-to-end trainable to
update weights during backpropagation such that the produced sentence
embeddings are semantically meaningful for the downstream prediction tasks.

##  5\. Experiments and Results

In this section, we describe our experimental setup for training the GraphNLI
model, the results obtained, and a detailed ablation study to better
understand different aspects of the model.

###  5.1. Dataset Preprocessing

As described in Section 3, we use data from online debates conducted on Kialo
in order to train and evaluate our model. All discussions crawled from Kialo
have a tree structure with a root node that represents the main thesis and
each other node is a reply to its parent which either supports or attacks the
parent.

As discussed in Section 4.1, we represent Kialo debates as undirected
discussion trees. Each edge can have a positive or negative sign, indicating
support or attack respectively. As mentioned in Section 3, Kialo debates are
typically balanced, with the majority of discussion trees having a fraction of
supportive replies between 0.4 and 0.6 with the rest being attacks; this
justifies our use of accuracy as the metric to evaluate our polarity
classifiers (see Section 5.4). We randomly sample $80\%$ of the Kialo debates
into a training set with the remainder serving as a test set. Overall, the
training set contains of $259,499$ arguments (replies) in total, while the
test set contains $64,874$ arguments in Kialo debates.

###  5.2. Training Details

After preprocessing the Kialo dataset, we use the graph walk techniques
described in Section 4.2.1 to capture the neighbourhood and parent contexts
for each of the nodes and feed them into our GraphNLI model.

In the case of the weighted root-seeking graph walk, we assign weights to the
nodes in the graph walk progressively. We define a discount parameter
$\gamma=0.75$ such that the nodes are weighted in the form of $\gamma^{k}$,
where $k$ is the distance of the node from the given node. So, the largest
weight $\gamma^{1}=0.75$ is assigned to the node’s parent, and then
$\gamma^{2}$ is assigned to the parent’s parent and so on in the graph walk
with $L$ nodes. Therefore, the influence of the ancestor nodes decreases
exponentially as we move farther away from the given node towards the root in
a graph walk.

We fine-tune GraphNLI model with a softmax classifier objective function and
cross-entropy loss for four epochs. We use a batch-size of 16, Adam optimizer
with learning rate $2\times 10^{-5}$, and a linear learning rate warm-up over
$10\%$ of the training data.

###  5.3. Baselines

We compare GraphNLI with the following relevant baselines on the
classification accuracy.

Bag-of-Words with Logistic Regression: A bag-of-words (BoW) model that uses
unigram features as input obtained from the arguments/replies in online
debates. Then the parent and the child BoW embeddings are concatenated and fed
into a Logistic Regression classifier with L2 regularization trained for 100
epochs.

Prompt embeddings (Rhetorics) with Logistic Regression: The prompt embedding
model (Zhang et al., 2017) infers a vector representation of utterances /
arguments in terms of the responses that similar utterances tend to prompt, as
well as the rhetorical intentions encapsulated by such utterances / arguments.
Once the embedding vectors for the arguments are obtained, the parent and
child embeddings are concatenated and input to a Logistic Regression
classifier for the final polarity prediction task. The classifier is trained
for 100 epochs with L2 regularizaton and cross-entropy loss.

Sentence-BERT: S-BERT (Reimers and Gurevych, 2019) is a modification of a pre-
trained BERT transformer network (Devlin et al., 2018) to derive semantically
meaningful sentence embeddings. We use the S-BERT architecture with
classification objective function and input the sentence pairs (parent and
child arguments) into the model to get their sentence embeddings. Later on,
these embeddings are concatenated and fed into a softmax classifier. The
S-BERT model is fine-tuned on Kialo training dataset for 4 epochs with a
batch-size of 16, the Adam optimizer, and cross-entropy loss function.

Non-trainable BERT embeddings with graph walks and Multi-layer Perceptron: For
each of the arguments in online debates, their embeddings are derived using a
pre-trained BERT model and using CLS-token embeddings. Using different graph
walk techniques as described in Section 4.2.1, various neighborhood siblings
and parent nodes are sampled for each node, and using their node embeddings, a
resulting aggregated embedding is formed using an average aggregation
function. These node embeddings are then fed into a multi-layer perceptron
(MLP) with two layers and with a softmax objective function for polarity
prediction. The initial BERT embeddings are non-trainable. We train the MLP
for 50 epochs or until the model converges on Kialo training dataset with
batch-size of 16, and the Adam optimizer.

###  5.4. Model Evaluation

Table 1. Accuracy scores of different models trained on Kialo dataset for
polarity prediction, discussed in Section 5.4.

Model | Accuracy (%)
---|---
Bag-of-Words + Logistic Regression | 67.00
Prompt Embeddings + Logistic Regression | 61.20
Sentence-BERT with classifier layer | 79.86
BERT Embeddings: Root-seeking Graph Walk + MLP | 70.27
GraphNLI: Root-seeking Graph Walk + Sum | 80.70
GraphNLI: Root-seeking Graph Walk + Avg. | 81.96
GraphNLI: Root-seeking Graph Walk + Weighted Avg. | 82.87
GraphNLI: Biased Root-seeking Random Walk + Sum | 79.95
GraphNLI: Biased Root-seeking Random Walk + Avg. | 80.44

We evaluate the performance of GraphNLI model on the test set of Kialo
dataset. Since the polarity prediction task is a binary classification problem
and that the datasets are roughly balanced between both classes (attacks and
supports), we use accuracy as the evaluation metric. We train models with five
different random seeds and report their average performances. Table 1 shows
the accuracy scores of different models trained on Kialo train set for the
polarity prediction task. The baseline model, Bag-of-Words embeddings with
Logistic Regression, achieves an accuracy of $67\%$, while Prompt embeddings
(Rhetorics) with Logistic Regression achieves just $61.20\%$ accuracy. The
sentence-BERT model trained on Kialo dataset achieves an accuracy of
$79.86\%$. The initial MLP model with non-trainable BERT embeddings and root-
seeking graph walk achieves an accuracy of $70.27\%$ which is even worse than
the Sentence-BERT.

Our model, GraphNLI with root-seeking graph walk and averaging node embeddings
in the graph walk to get the aggregated node embeddings achieves an overall
accuracy of $81.96\%$, whereas, the model achieves even better accuracy of
$82.87\%$ using weighted average node embeddings. Using biased root-seeking
random walk and average node embeddings, GraphNLI achieves an accuracy of
$80.44\%$. Clearly, all the variants of GraphNLI model achieve better accuracy
scores than all the baselines including sentence-BERT. GraphNLI with root-
seeking graph walk and weighted average node embeddings achieves the highest
accuracy overall. This shows that global context of the online debates or
discussions along with the local context of the argument pairs indeed helps in
improving the performance of the model.

The best performing variant, GraphNLI with root-seeking graph walk and
weighted average node embeddings, shows that having context of the upstream
arguments in the online debates helps the model in predicting argumentative
relations of support and attack. Also, weighted average aggregation gives
higher weights to the arguments near to the given argument pair in the
discussion tree whose polarity needs to be predicted, and progressively
reduces the weights when the graph walk moves towards the root.

###  5.5. Ablation Study

We have demonstrated superior performance of the GraphNLI architecture with
respect to various baselines in Table 1. In this section, we perform an
ablation study and discuss different aspects of the GraphNLI model and
intuition behind the choices in order to gain a better understanding of the
model.

First, we evaluate different kinds of graph walks by feeding the resultant
embeddings obtained with the average aggregation strategy into our model and
compare their accuracy scores. As shown in Table 1, all the graph walks with
GraphNLI architecture perform better than the S-BERT model significantly.
S-BERT just considers the argument pairs (node and its parent embeddings),
whereas through graph walks, GraphNLI considers the global context of
discussion trees by exploring parents and neighborhoods of a node. Therefore,
the global context with the local context of the ongoing discussions indeed
helps in predicting the polarities. On comparing different graph walks
themselves, we find the Root-seeking Graph Walks performing better than the
Biased Root-seeking Random Walks with $1.52\%$ higher accuracy. It shows that
upstream text in the discussions (parent and other ancestor nodes) indeed
helps the model more than the sibling and child nodes possibly obtained with
the biased root-seeking random walk. Intuitively, a person’s reply can be
influenced by the on-going discussion in the upstream text, but cannot be
influenced by future replies (child nodes) or sibling (parallel) replies.

We evaluate different aggregation strategies (summation, average and weighted
average) to aggregate the node embeddings of the neighbouring nodes using a
Root-seeking Graph Walk. As shown in Table 1, the weighted average aggregation
function performs better than the summation and average strategies. This shows
that influence of the upstream text (ancestor nodes) decreases as the graph
walk moves away from the given node towards the root. Hence, ancestor nodes
cannot be weighted equally but instead, progressively in the decreasing order
of their distance from the given node.

Table 2. Accuracy scores of GraphNLI model trained on Kialo dataset with different concatenation techniques using weighted average aggregation, discussed in Section 5.5. Concatenation | Accuracy (%)
---|---
$(u,v)$ | 76.78
$(u,v,u*v)$ | 82.05
$(u,v,|u-v|)$ | 82.87
$(u,v,|u-v|,u*v)$ | 82.38

We also evaluate different methods for concatenating a node’s embedding u with
the aggregated embedding of its neighbours v obtained using the root-seeking
graph walk. The impact of the concatenation method on the model’s performance
is significant. As depicted in Table 2, the concatenation of $(u,v,|u-v|)$
works the best. As reported by (Reimers and Gurevych, 2019), adding the
element-wise multiplication $u*v$ decreased performance. Element-wise absolute
difference $|u-v|$ which measures the distance between the two node embeddings
is an important component.

Our initial non-end-to-end trainable model as discussed in Section 5.3 in
which we keep the node embeddings obtained from the BERT model fixed, performs
even worse than the Sentence-BERT. This throws light on the importance of end-
to-end training of the model for fine-tuning on specific tasks. After end-to-
end training, the model outputs node embeddings that are rich in context
suitable for downstream tasks like polarity prediction.

##  6\. Conclusions and Future work

In this paper, we demonstrated a novel model, GraphNLI, that quantifies the
polarity of an online interaction by building a representation that captures
its content and context. GraphNLI derives inspiration from S-BERT, but it is
novel in the sense that it captures the context by sampling a discussion tree
of a discourse using different strategies based on graph walks (Section
4.2.1). Empirically, we found that a Root Seeking Graph Walk with a weighted
average aggregation of the ancestors’ contexts is the best strategy in terms
of classification accuracy (Table 1). This strategy addresses the shortcomings
of previous approaches that only capture the local context features, such as
the reply and the post it is replying to. We also showed through an ablation
study that information from parent and other ancestor nodes provide more
relevant contextual information than siblings and children of the reply node
whose polarity is being determined. Furthermore, the importance of ancestor
nodes decreases as the distance from the reply node increases.

A framework for polarity prediction, such as GraphNLI, can have a number of
applications for Computational Social Science (CSS):

Understanding conversation health: Online discussion forums provide a great
opportunity for creative and socially positive interactions, such as peer-
support for long-term medical problems (Joglekar et al., 2018; Panzarasa et
al., 2020). On the other hand, many forums have unfortunately become a medium
for rampant misinformation (Kumar et al., 2016) and hate (Gagliardone et al.,
2015). As such, identifying and promoting “healthy” conversations has been
identified as an important priority by many (_e.g._ Twitter (Inc., 2022)).
Once the polarity of posts is known, it can be used to develop conversation
health metrics such as whether a conversation is supporting or acrimonious
(_e.g._ , _cf._ (Boschi et al., 2021)).

Hate speech: Hate speech on online forums is a common (Cinelli et al., 2021)
challenge, including in nationally important conversations between citizens
and their elected representatives (Agarwal et al., 2021a). In other cases,
some members of a discussion can be unfairly targeted, for example, misogyny
is understood to be an important problem on online forums such as Reddit
(Guest et al., 2021). Inferring argument polarities at scale can help
platforms to detect such problems before they spiral out of control (for
example, highly attacking comments towards female participants can be a
possible indicator of potential misogyny).

Detecting filter bubbles: Democratic conversations on news and social media
sites can exhibit partisan tendencies (Bhatt et al., 2018; Agarwal et al.,
2021b; Karamshuk et al., 2016; Agarwal et al., 2020). This can lead to filter
bubbles: two (or more) parallel conversations about the same topic, with each
conversation consisting of posts largely agreeing with other posts in that
conversation, and yet having a large amount of disagreement with the other
conversations happening in parallel. Frameworks like GraphNLI could help
detect filter bubbles by quantifying agreeability in conversations: _e.g._ if
we find that posts reachable from each other also agree with each other (i.e.,
are supporting), and yet if an imaginary edge is induced between posts in
different parts of a conversation (or a different discussion thread), we find
that the imaginary edge would be an attack edge, this could be indicative of a
filter bubble.

Understanding multiple viewpoints in online debates: A novel way to mitigate
problems such as filter bubbles in large online debates is to present readers
with a balanced sample of all the justified arguments representing the
multiple important viewpoints. Once an argumentation framework has been
induced from a discussion and the attack/support relation between posts has
been established, it is possible to use the tools of argumentation theory to
compute the “winning” arguments from all sides of a debate (Young et al.,
2018, 2021; Young, 2021): arguments which have not been refuted and are left
standing as valid viewpoints (whether one may agree with them or not).

In future work, we hope to apply GraphNLI to some of the above.

Our work partly fixes the gap of inferring polarity from the content and
context of an online discourse. However, several questions remain to be
answered to make it relevant for wider usage. For instance, we have
demonstrated and evaluated our approach on Kialo, a tightly moderated online
debating platform. We would, however, like to test its validity on much
noisier, weakly moderated discourses, such as those found on
Reddit.888https://www.reddit.com/, last accessed 22 Jan 2022.. Other forums,
such as BBC’s Have Your Say?999See the comments of, e.g.
https://www.bbc.co.uk/news/uk-43253389#comments, last accessed 22 Jan 2022. do
not have an explicit threaded reply structure, requiring us to infer from the
text of a reply which other post it is replying to, prior to applying
GraphNLI’s graph walk techniques. In this less restrictive user interface,
posts may refer to multiple other posts, which in turns means that the reply
graph is no longer a tree. However, we note that discussions that are not
trees pose no limitations to our graph walk techniques (Section 4.2.1) because
there would only be more contexts for the graph walk to sample.

## References

  * (1)
  * Agarwal et al. (2021a) Pushkal Agarwal, Oliver Hawkins, Margarita Amaxopoulou, Noel Dempsey, Nishanth Sastry, and Edward Wood. 2021a.  Hate Speech in Political Discourse: A Case Study of UK MPs on Twitter. In _Proceedings of the 32nd ACM Conference on Hypertext and Social Media_ (Virtual Event, USA) _(HT ’21)_. Association for Computing Machinery, New York, NY, USA, 5–16.  https://doi.org/10.1145/3465336.3475113
  * Agarwal et al. (2020) Pushkal Agarwal, Sagar Joglekar, Panagiotis Papadapoulos, Nishanth Sastry, and Nicolas Kourtellis. 2020.  Stop tracking me Bro! Differential Tracking of User Demographics on Hyper-Partisan Websites. In _Proceedings of the The Web Conference (WWW 2020)_ _(WWW ’20)_. International World Wide Web Conferences Steering Committee, Taipei, Taiwan, 10 pages.
  * Agarwal et al. (2021b) Vibhor Agarwal, Yash Vekaria, Pushkal Agarwal, Sangeeta Mahapatra, Shounak Set, Sakthi Balan Muthiah, Nishanth Sastry, and Nicolas Kourtellis. 2021b.  Under the Spotlight: Web Tracking in Indian Partisan News Websites. In _Proceedings of the International AAAI Conference on Web and Social Media_ , Vol. 15. 26–37.
  * Allcott and Gentzkow (2017) Hunt Allcott and Matthew Gentzkow. 2017.  Social media and fake news in the 2016 election.  _Journal of Economic Perspectives_ 31, 2 (2017), 211–36.
  * Bail et al. (2018) Christopher A. Bail, Lisa P. Argyle, Taylor W. Brown, John P. Bumpus, Haohan Chen, M. B. Fallin Hunzaker, Jaemin Lee, Marcus Mann, Friedolin Merhout, and Alexander Volfovsky. 2018.  Exposure to opposing views on social media can increase political polarization.  _Proceedings of the National Academy of Sciences_ 115, 37 (2018), 9216–9221.
  * Baroni et al. (2011) Pietro Baroni, Martin Caminada, and Massimiliano Giacomin. 2011\.  An introduction to argumentation semantics.  _The Knowledge Engineering Review_ 26, 4 (2011), 365–410.
  * Bhatt et al. (2018) Shweta Bhatt, Sagar Joglekar, Shehar Bano, and Nishanth Sastry. 2018.  Illuminating an Ecosystem of Partisan Websites. In _Companion Proceedings of the The Web Conference 2018_ (Lyon, France) _(WWW ’18)_. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 545–554.  https://doi.org/10.1145/3184558.3188725
  * Bos and Markert (2006) Johan Bos and Katja Markert. 2006.  When logical inference helps determining textual entailment (and when it doesn’t). In _Proceedings of the Second PASCAL RTE challenge_. 26.
  * Bosc et al. (2016) Tom Bosc, Elena Cabrio, and Serena Villata. 2016.  Tweeties Squabbling: Positive and Negative Results in Applying Argument Mining on Social Media.  _6th International Conference on Computational Models of Argument_ 2016 (2016), 21–32.
  * Boschi et al. (2021) Gioia Boschi, Anthony P. Young, Sagar Joglekar, Chiara Cammarota, and Nishanth Sastry. 2021.  Who Has the Last Word? Understanding How to Sample Online Discussions.  _ACM Transactions on the Web (TWEB)_ 15, 3 (2021), 1–25.
  * Cabrio and Villata (2013) Elena Cabrio and Serena Villata. 2013.  A natural language bipolar argumentation approach to support users in online debate interactions.  _Argument & Computation_ 4, 3 (2013), 209–230.
  * Cabrio and Villata (2018) Elena Cabrio and Serena Villata. 2018.  Five Years of Argument Mining: a Data-driven Analysis.. In _IJCAI_ , Vol. 18. 5427–5433.
  * Cayrol and Lagasquie-Schiex (2005) Claudette Cayrol and Marie-Christine Lagasquie-Schiex. 2005\.  On the Acceptability of Arguments in Bipolar Argumentation Frameworks. In _European Conference on Symbolic and Quantitative Approaches to Reasoning and Uncertainty_. Springer, 378–389.
  * Cayrol and Lagasquie-Schiex (2013) Claudette Cayrol and Marie-Christine Lagasquie-Schiex. 2013\.  Bipolarity in Argumentation Graphs: Towards a Better Understanding.  _International Journal of Approximate Reasoning_ 54, 7 (2013), 876–899.
  * Cinelli et al. (2021) Matteo Cinelli, Andraž Pelicon, Igor Mozetič, Walter Quattrociocchi, Petra Kralj Novak, and Fabiana Zollo. 2021.  Online Hate: Behavioural Dynamics and Relationship with Misinformation.  _arXiv preprint arXiv:2105.14005_ (2021).
  * Cocarascu et al. (2020) Oana Cocarascu, Elena Cabrio, Serena Villata, and Francesca Toni. 2020.  A dataset independent set of baselines for relation prediction in argument mining.  _arXiv preprint arXiv:2003.04970_ (2020).
  * Cocarascu and Toni (2017) Oana Cocarascu and Francesca Toni. 2017.  Identifying attack and support argumentative relations using deep learning. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_. 1374–1379.
  * Dagan et al. (2010) Ido Dagan, Bill Dolan, Bernardo Magnini, and Dan Roth. 2010\.  Recognizing textual entailment: Rational, evaluation and approaches–erratum.  _Natural Language Engineering_ 16, 1 (2010), 105–105.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
  * Dung (1995) Phan Minh Dung. 1995\.  On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and $n$-Person Games.  _Artificial Intelligence_ 77, 2 (1995), 321–357.
  * Gagliardone et al. (2015) Iginio Gagliardone, Danit Gal, Thiago Alves, and Gabriela Martinez. 2015.  _Countering online hate speech_.  UNESCO Publishing.
  * Garimella et al. (2017) Kiran Garimella, Gianmarco De Francisci Morales, Aristides Gionis, and Michael Mathioudakis. 2017.  Reducing controversy by connecting opposing views. In _Proceedings of the Tenth ACM International Conference on Web Search and Data Mining_. 81–90.
  * Guest et al. (2021) Ella Guest, Bertie Vidgen, Alexandros Mittos, Nishanth Sastry, Gareth Tyson, and Helen Margetts. 2021.  An Expert Annotated Dataset for the Detection of Online Misogyny. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_. 1336–1350.
  * Hanselowski et al. (2018) Andreas Hanselowski, PVS Avinesh, Benjamin Schiller, Felix Caspelherr, Debanjan Chaudhuri, Christian M Meyer, and Iryna Gurevych. 2018\.  A Retrospective Analysis of the Fake News Challenge Stance-Detection Task. In _Proceedings of the 27th International Conference on Computational Linguistics_. 1859–1874.
  * Inc. (2022) Twitter Inc. 2022\.  Healthy conversations.  https://about.twitter.com/en/our-priorities/healthy-conversations
  * Joglekar et al. (2018) Sagar Joglekar, Nishanth Sastry, Neil S Coulson, Stephanie JC Taylor, Anita Patel, Robbie Duschinsky, Amrutha Anand, Matt Jameson Evans, Chris J Griffiths, Aziz Sheikh, et al. 2018\.  How online communities of people with long-term conditions function and evolve: network analysis of the structure and dynamics of the asthma UK and British lung foundation online communities.  _Journal of Medical Internet Research_ 20, 7 (2018), e238.
  * Karamshuk et al. (2016) Dmytro Karamshuk, Tetyana Lokot, Oleksandr Pryymak, and Nishanth Sastry. 2016.  Identifying Partisan Slant in News Articles and Twitter During Political Crises. In _Social Informatics_ , Emma Spiro and Yong-Yeol Ahn (Eds.). Springer International Publishing, Cham, 257–272.
  * Köffer et al. (2018) Sebastian Köffer, Dennis M Riehle, Steffen Höhenberger, and Jörg Becker. 2018.  Discussing the value of automatic hate speech detection in online debates.  _Multikonferenz Wirtschaftsinformatik (MKWI 2018): Data Driven X-Turning Data in Value, Leuphana, Germany_ (2018).
  * Kouylekov and Negri (2010) Milen Kouylekov and Matteo Negri. 2010.  An open-source package for recognizing textual entailment. In _Proceedings of the ACL 2010 System Demonstrations_. 42–47.
  * Kumar et al. (2016) Srijan Kumar, Robert West, and Jure Leskovec. 2016\.  Disinformation on the web: Impact, characteristics, and detection of wikipedia hoaxes. In _Proceedings of the 25th international conference on World Wide Web_. 591–602.
  * Lawrence and Reed (2020) John Lawrence and Chris Reed. 2020.  Argument Mining: A Survey.  _Computational Linguistics_ 45, 4 (2020), 765–818.
  * Lippi and Torroni (2016) Marco Lippi and Paolo Torroni. 2016.  Argumentation Mining: State of the Art and Emerging Trends.  _ACM Transactions on Internet Technology (TOIT)_ 16, 2 (2016), 1–25.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ (2019).
  * Mayer et al. (2021) Tobias Mayer, Santiago Marro, Elena Cabrio, and Serena Villata. 2021.  Enhancing Evidence-Based Medicine with Natural Language Argumentative Analysis of Clinical Trials.  _Artificial Intelligence in Medicine_ (2021), 102098.
  * Panzarasa et al. (2020) Pietro Panzarasa, Christopher J Griffiths, Nishanth Sastry, and Anna De Simoni. 2020.  Social medical capital: how patients and caregivers can benefit from online social interactions.  _Journal of Medical Internet Research_ 22, 7 (2020), e16337.
  * Rahwan et al. (2010) Iyad Rahwan, Mohammed I Madakkatel, Jean-François Bonnefon, Ruqiyabi N Awan, and Sherief Abdallah. 2010\.  Behavioral experiments for assessing the abstract argumentation semantics of reinstatement.  _Cognitive Science_ 34, 8 (2010), 1483–1502.
  * Rahwan and Simari (2009) Iyad Rahwan and Guillermo R. Simari. 2009.  _Argumentation in Artificial Intelligence_. Vol. 47.  Springer.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing_.
  * Villata (2021) Serena Villata. 2021\.  Towards assessing natural language argument strength: results and open challenges.  http://argstrength2021.argumentationcompetition.org/
  * Young (2018) Anthony P. Young. 2018\.  Notes on Abstract Argumentation Theory.  _arXiv preprint arXiv:1806.07709_ (2018).
  * Young (2021) Anthony P. Young. 2021\.  Likes as Argument Strength for Online Debates. In _The Third Workshop on Argument Strength_.  Available from http://argstrength2021.argumentationcompetition.org/papers/ArgStrength2021_paper_8.pdf, last accessed 22/1/2022.
  * Young et al. (2021) Anthony P. Young, Sagar Joglekar, Gioia Boschi, and Nishanth Sastry. 2021.  Ranking comment sorting policies in online debates.  _Argument & Computation_ 12, 2 (2021), 265–285.
  * Young et al. (2018) Anthony P. Young, Sagar Joglekar, Kiran Garimella, and Nishanth Sastry. 2018.  Approximations to truth in online comment networks. In _The Workshop on Argumentation and Society at the 7th International Conference on Computational Models of Argument_.  Available from https://nishrs.github.io/publication/young-2018-comma/, last accessed 22/1/2022.
  * Zhang et al. (2017) Justine Zhang, Arthur Spirling, and Cristian Danescu-Niculescu-Mizil. 2017\.  Asking too much? The rhetorical role of questions in political discourse. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_. 1558–1572.