# User Engagement and the Toxicity of Tweets

Nazanin Salehabadi  The University of Texas at Arlington500 UTA
BlvdArlingtonTexasUSA76013 ,  Anne Groggel  North Central College30 N.
Brainard StreetNapervilleIllinoisUSA60540 ,  Mohit Singhal  The University of
Texas at Arlington500 UTA BlvdArlingtonTexasUSA76013 ,  Sayak Saha Roy  The
University of Texas at Arlington500 UTA BlvdArlingtonTexasUSA76013 and  Shirin
Nilizadeh  The University of Texas at Arlington500 UTA
BlvdArlingtonTexasUSA76013

(2021)

###### Abstract.

Twitter is one of the most popular online micro-blogging and social networking
platforms. This platform allows individuals to freely express opinions and
interact with others regardless of geographic barriers. However, with the good
that online platforms offer, also comes the bad. Twitter and other social
networking platforms have created new spaces for incivility. With the growing
interest on the consequences of uncivil behavior online, understanding how a
toxic comment impacts online interactions is imperative. We analyze a random
sample of more than 85,300 Twitter _conversations_ to examine differences
between toxic and non-toxic conversations and the relationship between
toxicity and user engagement. We find that toxic conversations, those with at
least one toxic tweet, are longer but have fewer individual users contributing
to the dialogue compared to the non-toxic conversations. However, within toxic
conversations, toxicity is positively associated with more individual Twitter
users participating in conversations. This suggests that overall, more visible
conversations are more likely to include toxic replies.

Additionally, we examine the sequencing of toxic tweets and its impact on
conversations. Toxic tweets often occur as the main tweet or as the first
reply, and lead to greater overall conversation toxicity. We also find a
relationship between the toxicity of the first reply to a toxic tweet and the
toxicity of the conversation, such that whether the first reply is toxic or
non-toxic sets the stage for the overall toxicity of the conversation,
following the idea that hate can beget hate.

Twitter, Toxicity, Conversation, Direct replies, User engagement

††copyright: acmcopyright††journalyear: 2021††doi: TBA††conference: Make sure
to enter the correct conference title from your rights confirmation emai; June
03–05, 2018; Woodstock, NY††price: 15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs:
Information systems Social networking sites

##  1\. Introduction

Social media allows individuals to freely express opinions, engage in
interpersonal communications, and learn about new trends and new stories.
However, these platforms also create spaces for uncivil behavior. Mediated
communication restricts social cues, such as body language, that help to
discourage negative behavior and instead allows for greater perceived
anonymity, which can lead to more toxic behaviors online (Citron, 2014). In
particular, toxicity as explicit language, derogatory, or disrespectful
content has become endemic on online platforms (Papacharissi, 2002). This has
created a growing concern over incivility on social media platforms (Rost et
al., 2016). Twitter, as one of the leading micro-blogging social media
platforms, provides an important opportunity to examine differences in the
dynamics of _toxic_ and _non-toxic_ conversations.

In this paper, we assess a random sample of _tweets_ to examine how social
cues provided from toxic main tweets or direct replies may influence
subsequent exchanges. For instance, a toxic tweet or direct reply can set the
tone of a conversation by serving as a social cue, indicating the
permissibility of such language to other users. We highlight the differences
between _toxic_ and _non-toxic_ Twitter conversations as well the _less toxic_
and _more toxic_ conversations. This is accomplished by utilizing Google’s
Perspective API (Google Perspective API, 2020), as a state-of-the-art toxicity
detection tool, to assign toxicity scores to main tweets and its direct
replies.

Using this tool allowed us to create a binary toxic variable, in which
conversational exchanges are grouped into two categories: _toxic_
conversations are those with at least one toxic tweet, and _non-toxic_
conversations are those with no toxic tweet. In the context of this study, we
refer to conversations as the immediate _direct replies_ a tweet garners
within a 48 hour window. By classifying these conversations or exchanges by
toxicity, we compare the length of conversations and the number of unique
Twitter accounts participating posting direct replies, while controlling for a
number of factors that could affect these measures of engagement, including,
the characteristics of the Twitter account that posted the root tweet and the
averaged friends, followers, and lists of users directly replying to the root
tweet. We also construct a continuous measure for toxicity called _toxicity
score_. We calculated toxicity by taking the total number toxic tweets and
dividing it by the total number of tweets (both the main tweet and direct
replies). This continuous toxicity measure restricts the data to toxic
conversations and allows us to explore the association between conversation
length and toxicity. How online communities react to divisive behavior
reflects social norms- either spreading negative behaviors, or calling out
racist or sexist behavior, or ignoring toxic behaviors (Binns, 2012).
Therefore, we track the sequencing of toxic tweets to examine how the location
(index) of a toxic main tweet or direct reply impact user engagement overall.
We first examine the relationship between the location of the first initial
toxic tweet (either as the main tweet or a toxic direct reply) and user
engagement after this initial toxic tweet. We then investigate the
relationship between the first _direct reply_ to the first toxic tweet and
user engagement. With these aims we propose the following five hypotheses in
the next section.

##  2\. Hypotheses

Both online and offline, individuals are often concerned with their
presentation of self and seek to maintain a positive impression from others.
Online platforms may lessen this concern and may encourage incivility.
Anonymity provided through online platforms can lead to disinhibition (Suler,
2004), deindividuation, and a lack of accountability (Brown, 2018; Lee and
Kim, 2015), which can encourage negative behavior (Lowry et al., 2016; Siegel
et al., 1986; Kiesler et al., 1984; Fichman and Peters, 2019). Communicating
with others on Twitter may reduce the potential backlash one might experience
when making toxic comments in a face-to-face conversation. Even if users’
Twitter accounts are linked to their offline identities, such as their name,
it is unlikely that someone will meet the person they reply to. By
transcending physical interactions, online users can avoid the consequence of
face-to-face confrontation. Moreover, the nature of online platforms with
sending instant replies make online platforms more apt to capture unfiltered
prejudices (Brown, 2018). The expectations of behavior for conversations and
the consequences when individuals breach those norms, that would keep toxic
behavior in check offline, often do not apply online.

Hansen et al. (Hansen et al., 2011) found that, on Twitter, news are retweeted
more if they portray some negative content or opinion, while negative
sentiment was detrimental to retweeting in a generic setting. Toxic
conversations might gain visibility in a similar way with more individuals
joining on. Or, given the triggering nature of toxic tweets, they might lead
to longer conversations as more Twitter users counter that language and speak
out. Or, as in the case of social media trolls, users may opt to ignore
negative behavior (Binns, 2012; Shachaf and Hara, 2010; Herring et al., 2002)
rather than increasing the visibility of toxic content. Given these bodies of
research, we propose the following two hypotheses:

  * _H1. Toxic conversations and non-toxic conversations will have significantly different lengths, i.e., number of tweets._

  * _H2. Toxic conversations and non-toxic conversations will have significantly different levels of user participation._

Twitter users can customize their profiles through adding profile pictures,
bio descriptions, and URL links. Such changes can provide cues to users’ real
identities, as other Twitter users may opt to not make these changes to their
profiles. Since online factors such as anonymity or perceived anonymity can
foster uncivil behavior (Wulczyn et al., 2017), we would expect that Twitter
users who are less identifiable are more likely to join toxic conversations.
Thus, we hypothesize that:

  * _H3. Twitter accounts with less identifiability will be more likely to participate in Toxic conversations than accounts that provide a profile image, URL, have a verified account, or provide other potential markers of their real identity._

However, what extent do other Twitter users join in? Other work shows that
observing trolling behavior by others influences new users (Cheng et al.,
2017a). In other words, individuals may be more likely to engage in toxic
exchanges after seeing others do it, believing it to be the norm. Does Twitter
follow a similar pattern with conversations turning toxic if the initial tweet
is toxic? An initial toxic tweet or toxic replies may serve as cues,
indicating to Twitter users that toxic content is permissible. Thus, we
propose our final two hypotheses:

  * _H4. If the initial tweet is toxic then the conversation is more likely to be toxic._

  * _H5. If the reply to an initial toxic tweet is toxic then the rest of conversation is more likely to get toxic._

These hypotheses highlight the importance of understanding intergroup dynamics
or group mentality in a digital age by investigating the effects of a toxic
main tweet or toxic replies. As stated earlier, these direct replies reflect
the immediacy of online conversations given that data collection was
restricted to a 48 hour window. Examining these patterns of direct replies
outlines potential avenues to counter toxicity. If we understand the nature of
immediate direct replies, greater efforts can be made to address toxicity on
social media platforms.

##  3\. Related Work

Detection and Classification. Research on toxicity has employed machine
learning based detection algorithms to identify and classify offensive
language, hate speech, and cyberbully (Zhang et al., 2016; Davidson et al.,
2017). For example, Koratana and Hu (Koratana and Hu, [n.d.]) classified
comments into seven groups of clear, toxic, obscene, insult, identity hate,
severe toxic, and threat. The machine learning methods use a variety of
features, including lexical properties (Nobata et al., 2016; Mehdad and
Tetreault, 2016; De La Peña Sarracén, 2021), demographic and geographic
features (Waseem and Hovy, 2016; Vijayaraghavan et al., 2021), sentiment
scores (Gitari et al., 2015; Dinakar et al., 2012; Sood et al., 2012),
paragraph embeddings (Nobata et al., 2016; Djuric et al., 2015), and
linguistic psychological features (ElSherief et al., 2018a). The state-of-the-
art toxicity detection tool is available through, although the granular
details of this tool have not been published, broadly speaking, the API which
uses machine learning techniques to identify toxicity in text (Google
Perspective API, 2020). Another tool that has been used by researchers in
detecting offensive language and hate speech is HateSonar (Davidson et al.,
2017), which is a logistic regression-based classifier (ElSherief et al.,
2018a).

In this project, we will use this API to detect the toxicity of a sample of
main tweets and direct replies.

Characterization. Several approaches have been proposed to measure abusive
behavior on social media platforms. Chen et al. (Chen et al., 2012) used both
textual and structural features to predict a user’s intrinsic desire in
producing toxic content in YouTube, while others (Kayes et al., 2015) found
that users tend to flag toxic content posted on Yahoo Answers in an
exorbitance correct way. Also, some users considerably deviate from community
norms, posting a large amount of toxic content. Through careful feature
extraction, They also used machine learning approaches to predict which users
will be suspended. Chatzakou et al. (Chatzakou et al., 2017) studied the
properties of bullies and aggressors, focusing on the users of tweets with the
#GamerGate hashtag. (ElSherief et al., 2018b) compared the characteristics of
hate instigators and hate targets and showed both hate instigator and target
users are more likely to get attention on Twitter, i.e., get more followers,
retweets and listed. Others have studied the prevalence and characteristics of
hate speech on specific web communities, such as Gab (Zannettou et al., 2018),
4chan’s Politically Incorrect board (/pol/) (Hine et al., 2017a), Twitter
(ElSherief et al., 2018a; Chatzakou et al., 2017; Finkelstein et al., 2018;
Singhal et al., 2022) and Whisper (Silva et al., 2016).

Impact. A few studies have examined the impact of hate speech on hate targets
and the impact of toxicity on flow of conversations. For example, some work
have examined the association between hate speech and contributing factors,
including terrorist attacks (Olteanu et al., 2018; Williams and Burnap, 2015),
crime (Wired, 2016), and political events (Hine et al., 2017b; Harlow, 2015).
For example, (Kwak et al., 2015) shows that online games make players
particularly vulnerable to the exhibition of, and negative effects from,
cyberbullying and toxic behavior. (Zhang et al., 2018a) examined conversation
dynamics in an Oxford-style debate dataset. Their findings showed the outcome
of debate depends on aspects of conversational flow, including number of
discussion points, talking points, and discussion feedback. Maity et al.
(Maity et al., 2018) studied the factors associated with incivility on
Twitter, which caused users to leave Twitter. They found that the act of
incivility is highly correlated with the opinion differences between the
account holder (i.e., the user writing the incivil tweet) and the target
(i.e., the user for whom the incivil tweet is meant for or targeted),

Dialogues in Social Media. Dialogue modeling has been extensively studied by
researchers. (Kim and Kang, 2014) presents an approach for classifying student
discussions according to a set of discourse structures, and identifying
discussions with confusion or unanswered questions. Works such as (Kittur and
Kraut, 2008; Medelyan et al., 2009; Stvilia et al., 2008; Ferschke et al.,
2012; Panciera et al., 2009) have studied dialogue modeling in Wikipedia talk
pages. Previous works have also looked at how MOOC discussion forums can be
modeled to categorize threads based on whether or not they are related to
course content (D’Mello and Graesser, 2012; Chaturvedi et al., 2014; Brinton
et al., 2014; Wise et al., 2017; Agrawal et al., 2015; Kumar et al., 2015;
Chaturvedi et al., 2014; Rossi and Gnawali, 2014; Stump et al., 2013). Other
works have proposed some approaches for modeling dialogue acts in
conversational speech (Stolcke et al., 2000; Artstein and Poesio, 2008; Rojas
Barahona et al., 2019; Mairesse et al., 2007; Liu et al., 2006; Rosé et al.,
2008; Platonov et al., 2020), or have classified comments into a set of coarse
discourse acts to better understand the conversation (Zhang and Cranshaw,
2018; Jain et al., 2020; Chang et al., 2020; Tan et al., 2016; Cheng et al.,
2017b; Zhang et al., 2017; Kim et al., 2010; Michael, 2020). Some studies also
have focused on emotion recognition in conversations (Wang et al., 2020;
Cercas Curry and Rieser, 2019). Scholars have explored detecting toxicity of
conversations (Zhang et al., 2018b; Jurgens et al., 2019; Aroyo et al., 2019;
Chang and Danescu-Niculescu-Mizil, 2019; Zhang et al., 2018a; Hessel and Lee,
2019). Despite this scholarship, understanding the dynamics of toxic main
tweets or toxic direct replies and subsequent exchanges has been understudied.

##  4\. Data Collection

We used Twitter Standard API to collect a random sample of public tweets
during the period of April 24th to 26th 2020. In total, we obtained more than
3.6 million tweets. From this sample, we removed retweets and kept the _quote
tweets_ , which are similar to retweets except that they include a new tweet
message. We removed retweets from the data to focus on conversational content
with main tweets and direct replies. We restricted our sample of _main_ tweets
to those in English because the Perspective API can determine the toxicity of
text for a limited set of languages (Fernquist et al., 2019; Parekh and Patel,
2017; Noever, 2018).

This can increase the probability of obtaining mainly direct replies in
English. We obtained an initial sample of 441,152 tweets. Then, we employed
the _replies_ function included in the Twarc library (Twarc, 2020), a Python
library for obtaining Twitter data. This tool allowed us to obtain the direct
replies within 48 hours after the main tweet was posted in our sample. We
collected the direct replies to the tweets in our sample, which we call “main”
or “root” tweets. Twitter Standard search API returns all tweet replies for a
particular tweet up to a 7 day timeframe from the first day of the data
collection. While some tweets receive countless replies as soon as they are
posted, others have a slower rate of replies. Therefore, it is important to
note that: First, we collected the replies after a few days, so that main
tweets have enough time to get responses. Second, we provided conversations
the same amount of time to receive replies. As noted previously, our dataset
is restricted to direct replies posted in the first 48 hours after the main
tweet was posted. Tweets that did not have a reply, or conversations that were
generated by a single user were removed. Thus, our final dataset includes
85,389 conversations and 257,694 tweets, where the conversation length ranges
from 2 to 6,101 tweets.

###  4.1. Conversation Toxicity Detection

To identify toxic main tweet and toxic direct replies in our data, we ran
tweets through the Google Perspective API (Google Perspective API, 2020),
which has been used by many scholars (Zannettou et al., 2020; Gröndahl et al.,
2018; ElSherief et al., 2018a). Google perspective API applies different
machine learning models to score the toxicity of textual data, including
_Toxicity_ , and _severe-toxicity_. _Toxicity_ is defined as rude,
disrespectful, or unreasonable comment and _severe-toxicity_ is defined as a
very hateful, aggressive, disrespectful comment. We randomly selected 200
tweets and used both models for detecting their toxicity scores. Two coders
then separately classified the tweets as _toxic_ and _non-toxic_. For
inconsistent results, coders discussed how to resolve disagreements. The
inter-rater agreement was 94% with a Cohen’s kappa of 0.61. Using this manual
labeling, we computed the accuracy of _toxicity_ , and _severe-toxicity_
models, which are 94% and 96%, respectively. The inter-rater reliability
showed substantial agreement between manual coders and the Perspective API
classification for for both _toxicity_ and _severe-toxicity_ models.
Specifically, results suggest substantial agreement with a Cohen’s kappa of
0.64 for toxic tweets and 0.69 for severe toxicity. Since the accuracy and the
agreement between the _severe-toxicity_ scores of the Perspective API and the
coders was higher, we used this model to detect toxicity score for every tweet
in the conversations.

Before processing the tweets through the Google Perspective API, we pre-
processed the tweets by removing URLs and punctuation. However, we did not
remove hashtag keywords as they might contain toxic content. Additionally,
emojis were included in our sample because they may show Twitter users’
emotions, which can contribute to the toxicity of the tweets (Riordan, 2017).
We converted emoticons or emojis into text using a Python module called demoji
(Kim and Wurster, 2020). Although the main tweets in our sample were
restricted to English, since the sequencing of replies was an area of
interest, therefore direct replies were included regardless of language. This
was accomplished by using Google Translate API (Han, 2020) to translate tweets
written in other languages into English, and then the translated text was
passed through the Perspective API to get their sever toxicity scores. We
identified 40,007 tweets in other languages, i.e., about 15% of all the tweets
in our dataset.

###  4.2. Data Modeling

Although we were unable to collect nested conversations due to Twitter API
limitations, examining nested conversations might reflect a different pattern
of interactions than the ones examined in our hypotheses. For instance,
replies to a reply can be viewed as new conversations since what motivates
users to engage in those sub-conversation may differ from what prompts users
to engage in the main conversation. While it is likely that Twitter exchanges
continue and evolve under replies, that is beyond the scope of this study.
Limiting direct replies within a 48 hour window allowed us to better examine
effects of the toxicity of the main tweet on immediate replies. For instance,
individuals may be more likely to post toxic replies after seeing a toxic main
tweet or after reading toxic direct replies. If individuals post toxic tweets
because they perceive it to be acceptable within a conversation, those
perceptions may perhaps be strongest when exchanges reflect temporal
immediacy.

Since in our hypotheses, the order of the direct replies plays a role, we
define a _Twitter Conversation_ as a time series variable,
$C=(t,r_{1},r_{2},...)$, where $t$ is the main (root) tweet in the
conversation, $r_{1}$ is the first reply (or, response) to this tweet, $r_{2}$
is the second direct reply to this tweet, and so on. The length of
conversation specifies the number of tweets in the conversation and can be two
or more, $len|C|\geq 2$.

###  4.3. Dataset Characterization

(a) CCDF of convs. length

(b) Toxicity score

Figure 1. Dataset Characteristics

Our final dataset includes 85,389 conversations with 257,694 tweets, posted by
227,658 unique Twitter users. In our dataset, conversation length ranges from
2 to 6,101 tweets, while the majority of conversations (about 81%) have a
length of 2. Figure 1a shows the histogram of length of conversations. CCDF
plot for conversation length in our dataset, where it is fitted to a discrete
power law, with $x_{min}=9$ and $\alpha=2.21$. To test if the power-law
distribution is a good fit to our data, we test the power-law hypothesis using
goodness-of-fit test (Clauset et al., 2009). We use Kolmogorov-Smirnov
statistic to measure the distance between the distribution of the conversation
length and 5,000 synthetic datasets from a true power-law distribution. The
$p$-value quantifies the plausibility of the hypothesis which is defined to be
the fraction of the synthetic distances that are larger than the empirical
distance. If $p$ is close to 1, then the difference between the empirical data
and the model can be attributed to statistical fluctuations alone; if it is
small, the model is not a plausible fit to the data. We obtained $p=0.40$,
which indicates that the power-law distribution is a plausible distribution
for the length of conversations. Even though our data is well fit by a power
law, another distribution, such as a log-normal, might give a fit as good or
better. Therefore, we compared two models by employing Vuong’s test, which is
a likelihood ratio test for model selection using the Kullback-Leibler
criteria. Our null hypothesis is that both distributions are equally far from
the true distribution. Running the test, we could not reject the null
hypothesis ($p\\_two\\_sided=0.89$, $p\\_one\\_sided=0.55$), which means one
model is not closer to the true distribution than the other. Therefore we
cannot say power law distribution is a better model for the distribution of
conversation length. We also collected metadata of users who post the main
tweets and replied to the main tweets. Table 1 shows the descriptive
statistics of the Twitter users, including their number of followers, friends,
tweets, the age of the account (in years), and if they are verified accounts.
About 1.2% percent of our sample are verified Twitter accounts. Since the
number of followers, friends and tweets exhibit heavy-tailed distributions, we
report the median as well as the mean.

Table 1. Descriptive statistics of users in our conversations.

Feature | Type | Min | Max | Mean | Median
---|---|---|---|---|---
Followers | Count | 0 | 46,441,405 | 6,384 | 402
Friends | Count | 0 | 608,163 | 1,190 | 444
#tweets | Count | 1 | 5,030,869 | 21,721 | 6,300
Listed Count | Count | 0 | 206,848 | 37.56 | 2
Profile Descr. (length) | Count | 0 | 205 | 69.68 | 61
Age (Years) | Count | 0 | 14 | 4.71 | 4
URL | Boolean | 0 | 1 | 0.30 | 0
Profile Image | Boolean | 0 | 1 | 0.34 | NA
Verified | Boolean | 0 | 1 | 2,845 (0.012) | NA
N= 227,658 Unique Twitter accounts |  |  |

##  5\. Analysis

Online communities create social norms that can either prescribe civil
exchanges or negative behaviors when users believe or desire toxicity to be
the norm. To understand the construction of these norms we compare _toxic_ and
_non-toxic_ Twitter conversations. We accomplish this by using Google
Perspective’s _severe-toxicity_ model, to classify 29,458 (11%) tweets as
_toxic_ and 228,236 (89%) tweets as _non-toxic_. We find that 65,687 (about
77%) of conversations do not include any toxic tweet, and 19,702 (about 23%)
conversations include at least one toxic tweet. Next, we empirically test our
proposed hypotheses about the differences between _toxic_ and _non-toxic_
conversations.

We constructed two variables to measure toxicity.First, we use a binary
variable of toxicity. Conversations that include a toxic main tweet or a toxic
direct reply are classified as _toxic_ , and those that include no toxic
tweets are classified as _non-toxic_.

Second, we construct a continuous variable for toxicity, representing the
overall toxicity score of _toxic_ conversations. Figure 1b shows the histogram
of toxicity scores of _toxic_ conversations (percentage), where the toxicity
score is computed by dividing the number of toxic tweets in the conversation
by the length of conversation. We define the length of conversation as the
number of direct replies + 1 (for the main tweet). This figure shows that the
majority of _toxic_ conversations (more than 60%) have a toxicity score of
0.5, which means half of the tweets in these conversations are toxic. Also,
some conversations (about 12%) only include toxic tweets.

###  5.1. Toxicity and Number of Direct Replies

In this study we examine the relationship between the toxicity of
conversations and the length of conversations. Specifically, we hypothesize
that _toxic_ conversations and _non-toxic_ conversations have significantly
different lengths in terms of the number of tweets.

Table 2. Toxic vs. Non-toxic Conversations

| Toxic Conversations | Non-toxic Conversations
---|---|---
| Count | Min | Mean | Median | Max | Count | Min | Mean | Median | Max
Length | 19,702 | 2 | 5.19 | 2 | 6,101 | 65,687 | 2 | 2.37 | 2 | 96
Unique users | 95,130 | 2 | 5.02 | 2 | 6,019 | 140,187 | 2 | 2.31 | 2 | 96

Table 2 shows the length of conversations, where _toxic_ and _non-toxic_
conversations have a mean length of 5.19 and 2.37, and max length of 6,101 and
96, respectively. Since the distribution of conversation length is not normal
and it has a long tail, we ran a non-parametric Mann-Whitney U test to examine
whether there are significant differences in the length of conversation by
toxic and non-toxic conversations. The Mann-Whitney results suggest that, on
average, toxic conversations are longer than non-toxic conversations,
($\mu=5.19$, vs. $\mu=2.37$) ($W=741760000$, $p<0.001$).

Next, we use linear regression to examine the association between length and
toxicity of conversations, where the dependent variable is conversation
length, and toxicity, as a binary variable, is the independent variable. In
our regression model we incorporate the following control variables: the
averaged follower count, friend count, and listed count of users participating
in the conversation. We also control for if the root user has a verified
account, account age (years) of the root Twitter account, if the root account
has used a profile image, and the root user’s profile description length.

Table 4 shows the results of our linear regression model for length of
conversations and the conversation toxicity (as a binary variable). All
control variables were included in the model. For toxic conversations, the
predicted length of Twitter conversations is approximately 2.7 tweets longer
than non-toxic conversations ($p<0.001$). Thus, more than establishing that
toxic conversations and non-toxic conversations will have significantly
different lengths, we establish conversation toxicity as a significant
predictor for conversation length. This may be attributed to longer
conversations provided greater opportunity for toxic tweets to be included in
the conversation. Or toxic conversations, defined as those including those
tweets may draw voices of dissent into Twitter conversations as individuals
disagree with one another.

Table 3. Relationship between toxicity as a binary variable
(toxic vs. non-toxic) and conversation length

Dependent variable: Conversation length
---
| Linear Regression | std err
Toxic (binary) | 2.68∗∗∗ | (0.202)
Follower count | 0.00001∗∗∗ | (0.00000)
Friend count | 0.0001∗∗∗ | (0.00001)
Listed count |  $-$0.001∗∗∗ | (0.0001)
Avg. Followers count |  $-$0.00002∗∗∗ | (0.00000)
Avg. Listed count | 0.0002 | (0.001)
Verified account | 8.931∗∗∗ | (0.619)
Account age (years) |  $-$0.007 | (0.028)
Has profile image | 0.228 | (0.212)
Description length (Chars) | 0.004∗∗ | (0.002)
Observations | 85,389 |
Log Likelihood |  $-$395,200.000 |
Akaike Inf. Crit. | 790,422.000 |
Note: |  ∗∗p$<$0.05; ∗∗∗p$<$0.001 |

Table 4. The relationship between conversation length and
toxicity, as a continuous variable, in _toxic_ conversations

_Dependent variable: Conversation length_
---
| Linear Regression | std err
Toxicity |  $-$19.74∗∗∗ | (1.7)
Follower count | 0.00001∗∗∗ | (0.00000)
Friend count | 0.0002∗∗∗ | (0.00004)
Listed count |  $-$0.002∗∗∗ | (0.0003)
Avg. Followers count |  $-$0.0001∗∗∗ | (0.00001)
Avg. Listed count | 0.001 | (0.003)
Verified account | 28.376∗∗∗ | (2.655)
Account age (years) |  $-$0.022 | (0.120)
Has profile image | 0.609 | (0.920)
Description length (Chars) | 0.011 | (0.007)
Observations | 19,702 |
Log Likelihood |  $-$105,438.100 |
Akaike Inf. Crit. | 210,898.300 |
Note: |  ∗∗∗p$<$0.001 |

However, since conversation length exhibits a skewed distribution, examining
the whole population may not capture nuanced patterns within the data. To
address this we adopt two approaches: First, we follow existing work that used
quantile regression to analyze highly skewed variables (Yu et al., 2003). For
conversation length, we divided our dataset into quartiles and ran
multivariate regression models on each quartile. The results in the second and
forth quartiles are consistent with our results on the full data, while the
results in the other quartiles did not reach a level of statistical
significance. Second, we transform conversation length using a log function,
and then run multivariate regression models on the whole dataset. This
approach yielded the same results, where conversation toxicity was positively
associated with conversation length. Additionally, to examine whether
conversations with a length of 2 impacted our results, we dropped these
conversations and re-ran the analyses. Our significant results remain the
same.

_Toxicity in toxic conversations._ As shown in Figure 1b, some Twitter
conversations are more toxic than others. For example, 2,331 conversations
have a toxicity score of 1, while others are much less toxic, with 4,574
conversations have a toxicity score of less than 0.5. To better understand the
relationship between conversation length and toxicity, we ran additional
analyses restricting the data to only toxic conversations. In particular, we
test the association between toxicity and length in _toxic_ conversations
using linear regression, where the dependent variable is length, and the
independent variable is toxicity as a _continuous_ variable. In this model, we
also control for the characteristics of the root Twitter account, as well as
the average follower and friend counts of all users participating in the
conversation. Table 4 displays that toxicity becomes negatively associated
with conversation length and for every unit increase in toxicity, a 19 unit
decrease in conversation length is predicted, holding all other variables
constant ($p<0.001$). This suggests that across toxic conversations, the more
toxic conversations become, the fewer direct replies they receive.

We also investigate the impact of a skewed distribution of conversation length
on the analysis by running the regression models on each of the four
quartiles. Our results show toxicity is negatively associated with length of
conversation in the third and fourth quartiles. However, results in the other
quartiles were not significant. By transforming conversation length using a
log function, we again find that toxicity was negatively associated with
conversation. These results remain the same after dropping conversations with
a length of 2.

In summary, we find that toxic conversations are positively associated with
length of conversation. However, when analyses were restricted to only toxic
conversations, the level of conversation toxicity was negatively associated
with the total number of tweets within a conversation. While these results
first appear contradictory with our previous results, they make sense
intuitively. Toxicity may be a predictor of length as Twitter users feel
compelled to join in a toxic conversation, whether their tweet be one of
agreement or dissent. Or rather, longer conversations which involve more
direct replies have a greater chance of including toxic content. But the
nature of toxicity is more nuanced within conversations. The level of toxicity
across all conversations classified as toxic suggests that there is a tipping
point in conversations wherein Twitter users may be discouraged from joining
in the conversation.

###  5.2. Toxicity and User Participation

_User account characteristics._ We examine the relationship between users’
account characteristics (and potential identifiability) and conversation
toxicity. Table 5 displays the descriptive statistics of Twitter account
characteristics for toxic and non-toxic conversations. Table 6 displays our
logistic regression results, where toxicity (binary) is outcome variable, and
users’ account characteristics is the predictor, while controlling for Twitter
users’ account age, follower, friends, tweets, and listed counts. Table 6
shows that verified accounts, compared to Twitter accounts without a verified
status, have a 0.32 decrease in the log-odds of being in a toxic conversation,
holding all other variables in the model constant ($p<0.001$). We also find
that accounts that provide a profile image have a 0.26 decrease in the log-
odds of being in a toxic conversation, holding other variables constant
($p<0.001$). Twitter accounts that provide a URL have a 0.4 decrease in the
log-odds of participating in toxic conversations ($p<0.001$). Likewise, the
length of descriptions in profiles is negatively associated with participating
in a toxic conversation. Thus, we find support for _Hypothesis 3_ that:
Twitter accounts with less identifiability will be more likely to participate
in toxic conversations than accounts that provide a profile image, a URL, have
a verified account status, or provide other potential markers of their real
identity.

Table 5. Descriptive statistics of Users in Toxic vs. Non-toxic Conversations.

| With no toxic tweet | With more than $\geq$ 1 toxic tweet
---|---|---
Statistic | Count | Min | Max | Mean | Median | Count | Min | Max | Mean | Median
Followers | 140,187 | 0 | 30,502,080 | 5,840.98 | 503 | 95,130 | 0 | 46,441,405 | 7,459.34 | 292
Friends | 140,187 | 0 | 508,064 | 1,223.53 | 453 | 95,130 | 0 | 608,163 | 1,195.16 | 412
#tweets | 140,187 | 1 | 5,030,869 | 24,908.6 | 7919 | 95,130 | 1 | 5,031,063 | 18,225.80 | 4,751
Listed Count | 140,187 | 0 | 123,899 | 41.93 | 3 | 95,130 | 0 | 206,848 | 32.23 | 1
Profile Description | 140,187 | 0 | 205 | 72.00 | 64 | 95,130 | 0 | 193 | 67.02 | 57
Age (years) | 140,187 | 0 | 14 | 4.89 | 4 | 95,130 | 0 | 14 | 4.46 | 4
URL | 140,187 | 0 | 1 | 0.34 | NA | 95,130 | 0 | 1 | 0.24 | NA
Profile Image | 140,187 | 0 | 1 | 0.37 | NA | 95,130 | 0 | 1 | 0.28 | NA
Verified | 2,219 (1.58%) | NA | NA | NA | NA | 726 (0.76%) | NA | NA | NA | NA

Table 6. User account characteristics

Dependent variable: Toxic (binary)
---
| Logistic Regression | Std err
Verified account |  $-$0.32∗∗∗ | (0.04)
Has profile image |  $-$0.26∗∗∗ | (0.01)
Has a URL |  $-$0.40∗∗∗ | (0.01)
Description length (Chars) |  $-$0.001∗∗∗ | (0.0001)
Observations | 257,694 |
Log Likelihood |  $-$170,333.600 |
Akaike Inf. Crit. | 340,687.100 |
Note: |  ∗∗∗p$<$0.001

_User Participation._ We hypothesize that _toxic_ conversations and _non-
toxic_ conversations have significantly different levels of user
participation. Table 2 shows that in total 95,130 and 140,187 users have
participated in _toxic_ and _non-toxic_ conversations, respectively. Note that
some users have contributed to both toxic and non-toxic conversations, and
thus the sum of number of users in each population is more than the number of
unique users in our dataset. In order to examine the relationship between
toxicity and number of users participating in the conversation, we defined
_user participation_ as a continuous variable that shows the number of unique
users divided by the length of conversation. This variable can get a value
between 0 and 1, where larger values reflect higher levels of user
participation. Note that with this transformation, we address the issue of
skewed distributions.

Table 8 displays the results of a regression model that examines the effects
of toxicity of conversations _(toxic and non-toxic)_ on user participation. In
this model, we control for the account characteristics of the user who posted
the main tweet such as their followers, listed count, whether the user had a
profile image, the length of the description, and the account age. We find
that for toxic conversations the predicted user participation level would be
about 0.01 lower than for non-toxic conversations ($p<0.001$).

Table 7. User participation in toxic vs. non-toxic conversations.

Dependent variable: User Participation
---
| Linear Regression | Std err
Toxic (binary) |  $-$0.01∗∗∗ | (0.001)
Follower count |  $-$0.000 | (0.00)
Friend count | 0.00 | (0.00)
Listed count | 0.00 | (0.00)
Avg. Followers count | 0.00 | (0.00)
Avg. Listed count |  $-$0.00 | (0.00)
Verified account |  $-$0.009∗∗∗ | (0.002)
Account age (years) | 0.0001 | (0.0001)
Has profile image |  $-$0.001 | (0.001)
Has a URL |  $-$0.002∗∗∗ | (0.001)
Description length (Chars) |  $-$0.00003∗∗∗ | (0.00)
Observations | 85,389 |
Log Likelihood | 104,062.400 |
Akaike Inf. Crit. |  $-$208,100.700 |
Note: |  ∗∗∗p$<$0.001

Table 8. User participation in toxic conversations.

Dependent variable: User participation
---
| Linear Regression | Std err
Toxicity | 0.059∗∗∗ | (0.003)
Follower count | 0.00 | (0.00)
Friend count | 0.00 | (0.00)
Listed count | 0.00 | (0.00)
Avg. Followers count | 0.00 | (0.00)
Avg. Listed count |  $-$0.00 | (0.00)
Verified account |  $-$0.002 | (0.004)
Account age (years) | 0.0002 | (0.0002)
Has profile image |  $-$0.002 | (0.001)
Has a URL | 0.0001 | (0.001)
Description length (Chars) |  $-$0.00001 | (0.00001)
Observations | 19,702 |
Log Likelihood | 21,708.720 |
Akaike Inf. Crit. |  $-$43,393.430 |
Note: |  ∗∗∗p$<$0.001

However, the effect of toxicity changes direction when we consider user
engagement solely within toxic conversations. Table 8 shows that for every
unit increase in conversation toxicity, about a 0.06 unit increase in user
engagement is predicted, holding other variables constant ($p<0.001$). Thus,
_hypothesis 2_ that: Toxic conversations and non-toxic conversations will have
significantly different levels of user engagement is supported. Moreover, we
find that toxicity is a significant predictor for higher levels of user
engagement when considering toxic conversations. This suggests the greater
toxicity levels are associated with greater user engagement.

One explanation can be that toxic tweets may incite a reaction, good or bad.
The most toxic tweets may lead to more users participating in conversations as
users disagree or try to counter that toxicity. Another explanation is that
tweets with higher levels of toxicity can serve as a social cue indicating to
others that such language or rhetoric is permissible, and emboldens others to
join in. However, when comparing toxic conversations to non-toxic
conversations, we find that toxic conversations have lower levels of user
engagement. This suggests that while toxicity appears to incite engagement,
non-toxic conversations still involve more Twitter users.

###  5.3. Location of First Toxic Tweet in Conversations

We examine the location of first toxic tweets in conversations since toxic
comments early on may act as social cues, indicating to others that toxic
content is permissible. Figure 2a shows the histogram of index (location) of
the first toxic tweet in toxic conversations. Since about 70% of toxic
conversations have a length of two, we generated the plot after removing
conversations with length 2 (Figure 2b). The majority of initial toxic tweets
in conversations occur early on. Specifically, within toxic conversations, we
find that about 50% conversations begin with a toxic main tweet. Also, about
50% of the initial toxic tweets occurred as the first direct reply to the main
tweet. This suggests that the probability of toxic replies is highest in the
initial response to a tweet, even if main tweet is toxic or non-toxic.
Individuals may be more likely to tweet toxic replies earlier on in the
conversation because online interactions often lack social cues (Metzger et
al., 2010). However, as the conversation evolves, Twitter account users may
perceive more conversational cues, and they may be more hesitant to express an
unpopular opinion or one that differs from that of the group (Ulbig and Funk,
1999).

We hypothesize that if the main tweet is toxic then the conversation is more
likely to be _toxic_. We examine the relationship between the toxicity of the
main tweet and the overall toxicity of the conversation, only considering
toxic conversations. Table 9 shows the results of linear regression, where the
conversation toxicity is a continuous variable, and the independent variable
is the toxicity of root tweet as a binary variable. In this analysis we
include the same control variables as in our previous model. We find that
toxicity of the main tweet is positively associated with the toxicity of the
overall conversation with a coefficient of 0.18 ($p<0.001$). Thus, we find
support for Hypothesis 4.

(a) All toxic conversations

(b) Excluding length 2 convs.

Figure 2. Histogram of index of the first toxic tweet.  Table 9. Conversation
toxicity and first tweet being toxic

Dependent variable: Conversation toxicity
---
| Linear Regression | Std err
Toxicity of first tweet | 0.18∗∗∗ | (0.003)
Follower count |  $-$0.00 | (0.00)
Friend count |  $-$0.00∗∗∗ | (0.00)
Listed count |  $-$0.00 | (0.00)
Avg. Followers count |  $-$0.00 | (0.00)
Avg. Listed count | 0.00002∗∗ | (0.00001)
Verified account |  $-$0.162∗∗∗ | (0.010)
Account age (years) | 0.0001 | (0.0005)
Has profile image |  $-$0.015∗∗∗ | (0.004)
Has a URL |  $-$0.024∗∗∗ | (0.003)
Description length (Chars) |  $-$0.0001∗∗∗ | (0.00003)
Observations | 19,702 |
Log Likelihood | 4,468.375 |
Akaike Inf. Crit. |  $-$8,912.750 |
Note: |  ∗∗∗p$<$0.001

###  5.4. Immediate Response to Toxicity

Toxic behavior online impacts both the tone and level of interaction. Previous
work has classified bystander behavior in the face of cyberbullying, as a form
of toxicity, into confrontation responses and supportive behaviors (Macháčková
et al., 2013). We seek to examine responses to toxic comments or aggressive
behavior on social media platforms.

We begin by examining how Twitter users respond to a toxic tweet in
conversations. Out of the 19,702 toxic conversations, we find that 3,220 (16%)
and 9,779 (50%) conversations have an immediate _toxic_ and _non-toxic_ reply,
respectively; while 6,703 (34%) conversations did not have a reply to the
first toxic tweet. This suggests that toxic content may discourage others from
joining or contributing to Twitter conversations. However, we see that in
about half of conversations, most toxic tweets responded to non-toxic replies.
We find that only a small number of conversations involve toxic initial
replies to the first toxic tweets.

####  5.4.1. Toxicity of first reply and conversation toxicity

Toxic replies may lead to more toxicity within online conversations, with one
hateful or toxic tweet instigating another. Previous scholarship has suggested
as much, with hateful content spreading farther, wider and faster on social
media platforms, such as Gab (Gab.com) (Mathew et al., 2019). We examine the
effect of the first reply to the first toxic tweet and the toxicity of the
rest of replies in the conversation. Figure 3a shows the toxicity distribution
of rest of replies in the conversations after the first toxic reply, where the
red bars shows the toxicity distribution of conversations that have an
immediate _toxic_ reply after the first toxic tweet, and the blue bars shows
the toxicity distribution of conversations that have an immediate _non-toxic_
reply after the first toxic tweet. We find that conversations with a toxic
reply are, on average more toxic, than conversations in which the initial
reply was non-toxic (mean toxicity score of conversations with an immediate
toxic and non-toxic reply is $M=0.91$ and $M=0.03$, respectively).

(a) Toxicity distribution

(b) User participation

Figure 3. Toxicity and user participation of conversations after a toxic and
non-toxic reply to the first toxic tweet.

To examine the effect of a toxic reply to an initial toxic tweet, we construct
a binary variable that is coded 0 if the first reply to the toxic tweet is
non-toxic, and 1 if the first reply is toxic. We then include this toxic reply
variable as a predictor in a regression analysis for the overall conversation
toxicity. For this analysis, we only use the conversations that have a reply
to the toxic tweet ($n=12,999$). We also employ the control variables used in
previous models. Table 11 shows that if the first reply to a toxic tweet is
also toxic then we expect a 0.87 increase in the predicted conversation
toxicity ($p<0.001$). Thus, we find support for _Hypothesis 4_ : that if the
initial tweet is toxic then the conversation is more likely to be toxic. This
finding shows that toxicity may be contagious with one toxic tweet and toxic
reply impacting the toxicity of conversations overall.

Table 10. Relationship between the first reply to the first toxic tweet and
conversation toxicity.

Dependent variable: Toxicity of rest of conversation
---
| Linear Regression | Std err
Toxicity of first reply | 0.87∗∗∗ | (0.003)
to the first toxic tweet |  |
Follower count | 0.00 | (0.00)
Friend count | 0.00∗ | (0.00)
Listed count |  $-$0.00∗∗∗ | (0.00)
Avg. Followers count | 0.00 | (0.00)
Avg. Listed count | 0.00 | (0.00001)
Verified account |  $-$0.023∗∗∗ | (0.008)
Account age (years) | 0.0003 | (0.0004)
Has profile image |  $-$0.007∗∗ | (0.003)
Has a URL |  $-$0.001 | (0.003)
Description length (Chars) |  $-$0.00 | (0.00003)
Observations | 12,999 |
Log Likelihood | 6,678.316 |
Akaike Inf. Crit. |  $-$13,332.630 |
Note: |  ∗∗p$<$0.05; ∗∗∗p$<$0.001

Table 11. Relationship between the first reply to the first toxic tweet and
User Participation.

Dependent variable: User participation
---
| Linear Regression | Std err
Toxicity of first reply |  $-$0.82∗∗∗ | (0.005)
Follower count |  $-$0.00∗∗∗ | (0.00)
Friend count |  $-$0.00∗∗∗ | (0.00)
Listed count | 0.00∗∗ | (0.00)
Avg. Followers count | 0.00 | (0.00)
Avg. Listed count | 0.00003∗∗ | (0.00002)
Verified account |  $-$0.178∗∗∗ | (0.015)
Account age (years) | 0.0003 | (0.001)
Has profile image | 0.007 | (0.006)
Has a URL |  $-$0.012∗∗ | (0.005)
Description length (Chars) |  $-$0.0001∗∗∗ | (0.00004)
Observations | 12,999 |
Log Likelihood |  $-$551.679 |
Akaike Inf. Crit. | 1,127.358 |
Note: |  ∗∗∗p$<$0.001

####  5.4.2. Toxicity of first reply and user participation

We also investigate the effect of the first reply to the initial toxic tweet
in conversations on user participation. This allows us to examine how replies
to the first toxic tweet may influence how many users participation in the
conversation. We define user participation as the number of unique users
participating in the rest of conversation after the first toxic tweet, divided
by conversation length. Figure 3b shows user participation in conversations
after the first toxic tweet. As you can see, on average the number of unique
users participating after a _non-toxic_ reply is greater than the number of
Twitter users participating in conversations after a _toxic_ reply.

To examine the effect of a toxic direct reply to an initial toxic tweet on the
user participation, we construct a binary variable that is coded 0 if the
first direct reply to the toxic tweet is non-toxic, and 1 if the first reply
is toxic. We then include this variable as a predictor in a regression
analysis for the overall conversation user participation. For this analysis,
conversations are restricted to those that have a reply to a toxic tweet
($n=12,999$). We include the same control variables as outlined previously.
Table 11 shows that if the first direct reply to the first toxic tweet is also
toxic, then we expect a 0.82 decrease in the predicted user participation
($p<0.001$). Therefore, we find support for _Hypothesis 5_ : that if the reply
to an initial toxic tweet is toxic then the conversation is more likely be
toxic. Our results suggest that how Twitter users respond to initial toxic
tweets carries important consequences for the tone of subsequent direct
replies.

##  6\. Limitations and Future Work

We only obtained direct replies posted within a 48 hours time span after the
main tweet was posted. We could have potentially missed some replies, however,
replies to tweets are often fast paced with most replies happening within an
hour of the original post (Paul et al., 2011). Most Twitter conversations are
short with only a small minority of conversations growing to include thousands
of tweets or users (Goncalves et al., 2011). Additionally, we restricted our
discussion of conversations to direct replies to the main tweet. While replies
to replies is another critical piece of the puzzle to understand the dynamics
of toxic conversations, our goal for this study was to understand the more
immediate and firsthand dynamics of toxicity within conversational exchanges
on Twitter. Another limitation is that Twitter conversations triggered after a
current event may capture different community norms of toxicity. We were also
unable to discern the nature of toxic replies. The first reply may be a
counterargument to the main tweet or a user could be replying in agreement to
the main tweet using the same toxic language. Our continuous toxicity measure
allows us to determine the toxicity of conversations overall but not the
sentiment of tweets.

Future Work Despite these limitations, our findings lay the groundwork for
outlining strategies to decrease the toxicity online. Others can fully examine
nested conversations to study why and when toxic exchanges spike. While our
findings speak to differences in the Twitter accounts that participate in
toxic and non-toxic conversations, future inquiry can build upon our work.
Other work could investigate which users respond to toxic tweets and the
extent to which certain users stymie or incite the spread of toxicity.

##  7\. Conclusion

Social media platforms overcome geographic distance to create a space where
individuals can express opinions and engage in interpersonal communication
(Walther, 2011). But with the good that online platforms offer, also comes the
bad. To analyze the impact of toxicity on Twitter conversations, we create two
metrics for toxicity. This allows us to assess the relationship between
toxicity and conversational engagement, measured through the length of
conversations and overall user participation in Twitter conversations. One key
finding is that toxic conversations, defined as those with toxic tweets are
longer. Another key finding is that within toxic conversations, toxicity is
positively associated with user participation. Our findings also indicate that
Twitter accounts with less identifiability are more likely to post toxic
tweets. Twitter accounts that provide a profile image, URL, or have a verified
account status were significantly less likely to participate in toxic
conversations. Our work also highlights the importance of understanding the
sequencing of toxic tweets and its impact on immediate direct replies.

We find that the initial direct reply to the first toxic tweet within
conversations matters. Toxic replies lead to more toxic conversations.
However, this suggests that the inverse is also true. Thus, non-toxic replies
may potentially be the antidote to toxicity online. These findings provide key
insights for developing tools to alleviate the negative outcomes of toxic
content in online communities.

## References

  * (1)
  * Agrawal et al. (2015) Akshay Agrawal, Jagadish Venkatraman, Shane Leonard, and Andreas Paepcke. 2015.  YouEDU: addressing confusion in MOOC discussion forums by recommending instructional video clips.  (2015).
  * Aroyo et al. (2019) Lora Aroyo, Lucas Dixon, Nithum Thain, Olivia Redfield, and Rachel Rosen. 2019.  Crowdsourcing subjective tasks: the case study of understanding toxicity in online discussions. In _Companion Proceedings of The 2019 World Wide Web Conference_. 1100–1105.
  * Artstein and Poesio (2008) Ron Artstein and Massimo Poesio. 2008.  Inter-coder agreement for computational linguistics.  _Computational Linguistics_ 34, 4 (2008), 555–596.
  * Binns (2012) Amy Binns. 2012\.  DON’T FEED THE TROLLS! Managing troublemakers in magazines’ online communities.  _Journalism practice_ 6, 4 (2012), 547–562.
  * Brinton et al. (2014) Christopher G Brinton, Mung Chiang, Shaili Jain, Henry Lam, Zhenming Liu, and Felix Ming Fai Wong. 2014.  Learning about social learning in MOOCs: From statistical analysis to generative model.  _IEEE transactions on Learning Technologies_ 7, 4 (2014), 346–359.
  * Brown (2018) Alexander Brown. 2018\.  What is so special about online (as compared to offline) hate speech?  _Ethnicities_ 18, 3 (2018), 297–326.
  * Cercas Curry and Rieser (2019) Amanda Cercas Curry and Verena Rieser. 2019.  A Crowd-based Evaluation of Abuse Response Strategies in Conversational Agents. In _Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue_. Association for Computational Linguistics, Stockholm, Sweden, 361–366.  https://www.aclweb.org/anthology/W19-5942
  * Chang et al. (2020) Jonathan P. Chang, Caleb Chiam, Liye Fu, Andrew Wang, Justine Zhang, and Cristian Danescu-Niculescu-Mizil. 2020.  ConvoKit: A Toolkit for the Analysis of Conversations. In _Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue_. 57–60.  https://www.aclweb.org/anthology/2020.sigdial-1.8
  * Chang and Danescu-Niculescu-Mizil (2019) Jonathan P Chang and Cristian Danescu-Niculescu-Mizil. 2019\.  Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they Develop. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. 4745–4756.
  * Chaturvedi et al. (2014) Snigdha Chaturvedi, Dan Goldwasser, and Hal Daumé III. 2014\.  Predicting instructor’s intervention in mooc forums. In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 1501–1511.
  * Chatzakou et al. (2017) Despoina Chatzakou, Nicolas Kourtellis, Jeremy Blackburn, Emiliano De Cristofaro, Gianluca Stringhini, and Athena Vakali. 2017\.  Mean birds: Detecting aggression and bullying on twitter. In _ACM Web Science Conference_.
  * Chen et al. (2012) Ying Chen, Yilu Zhou, Sencun Zhu, and Heng Xu. 2012\.  Detecting offensive language in social media to protect adolescent online safety. In _2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing_. IEEE, 71–80.
  * Cheng et al. (2017b) Hao Cheng, Hao Fang, and Mari Ostendorf. 2017b.  A factored neural network model for characterizing online discussions in vector space. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_. 2296–2306.
  * Cheng et al. (2017a) Justin Cheng, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2017a.  Anyone can become a troll: Causes of trolling behavior in online discussions. In _Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing_. 1217–1230.
  * Citron (2014) Danielle Keats Citron. 2014\.  _Hate crimes in cyberspace_.  Harvard University Press.
  * Clauset et al. (2009) Aaron Clauset, Cosma Rohilla Shalizi, and Mark EJ Newman. 2009.  Power-law distributions in empirical data.  _SIAM review_ 51, 4 (2009), 661–703.
  * Davidson et al. (2017) Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017.  Automated hate speech detection and the problem of offensive language. In _Eleventh international aaai conference on web and social media_.
  * De La Peña Sarracén (2021) Gretel Liz De La Peña Sarracén. 2021\.  Multilingual and Multimodal Hate Speech Analysis in Twitter. In _Proceedings of the 14th ACM International Conference on Web Search and Data Mining_. 1109–1110.
  * Dinakar et al. (2012) Karthik Dinakar, Birago Jones, Catherine Havasi, Henry Lieberman, and Rosalind Picard. 2012\.  Common sense reasoning for detection, prevention, and mitigation of cyberbullying.  _ACM Transactions on Interactive Intelligent Systems (TiiS)_ 2, 3 (2012), 18.
  * Djuric et al. (2015) Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. 2015.  Hate speech detection with comment embeddings. In _Proceedings of the 24th international conference on world wide web_. ACM, 29–30.
  * D’Mello and Graesser (2012) Sidney D’Mello and Art Graesser. 2012.  Dynamics of affective states during complex learning.  _Learning and Instruction_ 22, 2 (2012), 145–157.
  * ElSherief et al. (2018a) Mai ElSherief, Vivek Kulkarni, Dana Nguyen, William Yang Wang, and Elizabeth Belding. 2018a.  Hate lingo: A target-based linguistic analysis of hate speech in social media. In _Twelfth International AAAI Conference on Web and Social Media_.
  * ElSherief et al. (2018b) Mai ElSherief, Shirin Nilizadeh, Dana Nguyen, Giovanni Vigna, and Elizabeth Belding. 2018b.  Peer to peer hate: Hate speech instigators and their targets. In _Twelfth International AAAI Conference on Web and Social Media_.
  * Fernquist et al. (2019) Johan Fernquist, Oskar Lindholm, Lisa Kaati, and Nazar Akrami. 2019.  A Study on the Feasibility to Detect Hate Speech in Swedish. In _2019 IEEE International Conference on Big Data (Big Data)_. IEEE, 4724–4729.
  * Ferschke et al. (2012) Oliver Ferschke, Iryna Gurevych, and Yevgen Chebotar. 2012\.  Behind the article: Recognizing dialog acts in wikipedia talk pages. In _Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics_. 777–786.
  * Fichman and Peters (2019) Pnina Fichman and Elizabeth Peters. 2019.  The impacts of territorial communication norms and composition on online trolling.  _International Journal of Communication_ 13 (2019), 20.
  * Finkelstein et al. (2018) Joel Finkelstein, Savvas Zannettou, Barry Bradlyn, and Jeremy Blackburn. 2018.  A Quantitative Approach to Understanding Online Antisemitism.  _arXiv preprint arXiv:1809.01644_ (2018).
  * Gitari et al. (2015) Njagi Dennis Gitari, Zhang Zuping, Hanyurwimfura Damien, and Jun Long. 2015.  A lexicon-based approach for hate speech detection.  _International Journal of Multimedia and Ubiquitous Engineering_ 10, 4 (2015), 215–230.
  * Goncalves et al. (2011) Bruno Goncalves, Nicola Perra, and Alessandro Vespignani. 2011\.  Validation of Dunbar’s number in Twitter conversations.  _arXiv preprint arXiv:1105.5170_ (2011).
  * Google Perspective API (2020) Google Perspective API. 2020\.  https://www.perspectiveapi.com/.
  * Gröndahl et al. (2018) Tommi Gröndahl, Luca Pajola, Mika Juuti, Mauro Conti, and N Asokan. 2018.  All You Need is” Love” Evading Hate Speech Detection. In _Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security_. 2–12.
  * Han (2020) SuHun Han. 2020\.  Google Translate API for Python.  https://pypi.org/project/googletrans/.
  * Hansen et al. (2011) Lars Kai Hansen, Adam Arvidsson, Finn Årup Nielsen, Elanor Colleoni, and Michael Etter. 2011\.  Good friends, bad news-affect and virality in twitter.  In _Future information technology_. Springer, 34–43.
  * Harlow (2015) Summer Harlow. 2015\.  Story-chatterers stirring up hate: Racist discourse in reader comments on US newspaper websites.  _Howard Journal of Communications_ 26, 1 (2015), 21–42.
  * Herring et al. (2002) Susan Herring, Kirk Job-Sluder, Rebecca Scheckler, and Sasha Barab. 2002.  Searching for safety online: Managing” trolling” in a feminist forum.  _The information society_ 18, 5 (2002), 371–384.
  * Hessel and Lee (2019) Jack Hessel and Lillian Lee. 2019.  Something’s Brewing! Early Prediction of Controversy-causing Posts from Discussion Features. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , Vol. 1.
  * Hine et al. (2017a) Gabriel Emile Hine, Jeremiah Onaolapo, Emiliano De Cristofaro, Nicolas Kourtellis, Ilias Leontiadis, Riginos Samaras, Gianluca Stringhini, and Jeremy Blackburn. 2017a.  Kek, Cucks, and God Emperor Trump: A Measurement Study of 4chan’s Politically Incorrect Forum and its Effects on the Web.  _ICWSM_.
  * Hine et al. (2017b) Gabriel Emile Hine, Jeremiah Onaolapo, Emiliano De Cristofaro, Nicolas Kourtellis, Ilias Leontiadis, Riginos Samaras, Gianluca Stringhini, and Jeremy Blackburn. 2017b.  Kek, cucks, and god emperor trump: A measurement study of 4chan’s politically incorrect forum and its effects on the web. In _Eleventh International AAAI Conference on Web and Social Media_.
  * Jain et al. (2020) Ayush Jain, Maria Pacheco, Steven Lancette, Mahak Goindani, and Dan Goldwasser. 2020\.  Identifying Collaborative Conversations using Latent Discourse Behaviors. In _Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue_. Association for Computational Linguistics, 1st virtual meeting, 74–78.
  * Jurgens et al. (2019) David Jurgens, Libby Hemphill, and Eshwar Chandrasekharan. 2019\.  A Just and Comprehensive Strategy for Using NLP to Address Online Abuse. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_. 3658–3666.
  * Kayes et al. (2015) Imrul Kayes, Nicolas Kourtellis, Daniele Quercia, Adriana Iamnitchi, and Francesco Bonchi. 2015\.  The social world of content abusers in community question answering. In _Proceedings of the 24th International Conference on World Wide Web_. International World Wide Web Conferences Steering Committee, 570–580.
  * Kiesler et al. (1984) Sara Kiesler, Jane Siegel, and Timothy W McGuire. 1984\.  Social psychological aspects of computer-mediated communication.  _American psychologist_ 39, 10 (1984), 1123.
  * Kim and Kang (2014) Jihie Kim and Jeon-Hyung Kang. 2014.  Towards identifying unresolved discussions in student online forums.  _Applied intelligence_ 40, 4 (2014), 601–612.
  * Kim et al. (2010) Su Nam Kim, Li Wang, and Timothy Baldwin. 2010.  Tagging and linking web forum posts. In _Proceedings of the Fourteenth Conference on Computational Natural Language Learning_. 192–202.
  * Kim and Wurster (2020) Taehoon Kim and Kevin Wurster. 2020.  Emoji for Python.  https://pypi.org/project/emoji/.
  * Kittur and Kraut (2008) Aniket Kittur and Robert E Kraut. 2008.  Harnessing the wisdom of crowds in wikipedia: quality through coordination. In _Proceedings of the 2008 ACM conference on Computer supported cooperative work_. 37–46.
  * Koratana and Hu ([n.d.]) Animesh Koratana and Kevin Hu. [n.d.].  Toxic Speech Detection.  ([n. d.]).
  * Kumar et al. (2015) Muthu Kumar, Min-Yen Kan, Bernard CY Tan, and Kiruthika Ragupathi. 2015.  Learning Instructor Intervention from MOOC Forums: Early Results and Issues.  _International Educational Data Mining Society_ (2015).
  * Kwak et al. (2015) Haewoon Kwak, Jeremy Blackburn, and Seungyeop Han. 2015\.  Exploring cyberbullying and other toxic behavior in team competition online games. In _Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems_. ACM, 3739–3748.
  * Lee and Kim (2015) So-Hyun Lee and Hee-Woong Kim. 2015.  Why people post benevolent and malicious comments online.  _Commun. ACM_ 58, 11 (2015), 74–79.
  * Liu et al. (2006) Yang Liu, Elizabeth Shriberg, Andreas Stolcke, Dustin Hillard, Mari Ostendorf, and Mary Harper. 2006.  Enriching speech recognition with automatic detection of sentence boundaries and disfluencies.  _IEEE Transactions on audio, speech, and language processing_ 14, 5 (2006), 1526–1540.
  * Lowry et al. (2016) Paul Benjamin Lowry, Jun Zhang, Chuang Wang, and Mikko Siponen. 2016.  Why do adults engage in cyberbullying on social media? An integration of online disinhibition and deindividuation effects with the social structure and social learning model.  _Information Systems Research_ 27, 4 (2016), 962–986.
  * Macháčková et al. (2013) Hana Macháčková, Lenka Dedkova, Anna Sevcikova, and Alena Cerna. 2013.  Bystanders’ support of cyberbullied schoolmates.  _Journal of community & applied social psychology_ 23, 1 (2013), 25–36.
  * Mairesse et al. (2007) François Mairesse, Marilyn A Walker, Matthias R Mehl, and Roger K Moore. 2007.  Using linguistic cues for the automatic recognition of personality in conversation and text.  _Journal of artificial intelligence research_ 30 (2007), 457–500.
  * Maity et al. (2018) Suman Kalyan Maity, Aishik Chakraborty, Pawan Goyal, and Animesh Mukherjee. 2018.  Opinion conflicts: An effective route to detect incivility in twitter.  _Proceedings of the ACM on Human-Computer Interaction_ 2, CSCW (2018), 117.
  * Mathew et al. (2019) Binny Mathew, Ritam Dutt, Pawan Goyal, and Animesh Mukherjee. 2019\.  Spread of hate speech in online social media. In _Proceedings of the 10th ACM Conference on Web Science_. 173–182.
  * Medelyan et al. (2009) Olena Medelyan, David Milne, Catherine Legg, and Ian H Witten. 2009.  Mining meaning from Wikipedia.  _International Journal of Human-Computer Studies_ 67, 9 (2009), 716–754.
  * Mehdad and Tetreault (2016) Yashar Mehdad and Joel Tetreault. 2016.  Do characters abuse more than words?. In _Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue_. 299–303.
  * Metzger et al. (2010) Miriam J Metzger, Andrew J Flanagin, and Ryan B Medders. 2010\.  Social and heuristic approaches to credibility evaluation online.  _Journal of communication_ 60, 3 (2010), 413–439.
  * Michael (2020) Thilo Michael. 2020\.  Retico: An incremental framework for spoken dialogue systems. In _Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue_. Association for Computational Linguistics, 1st virtual meeting, 49–52.  https://www.aclweb.org/anthology/2020.sigdial-1.6
  * Nobata et al. (2016) Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016.  Abusive language detection in online user content. In _Proceedings of the 25th international conference on world wide web_. International World Wide Web Conferences Steering Committee, 145–153.
  * Noever (2018) David Noever. 2018\.  Machine learning suites for online toxicity detection.  _arXiv preprint arXiv:1810.01869_ (2018).
  * Olteanu et al. (2018) Alexandra Olteanu, Carlos Castillo, Jeremy Boy, and Kush R Varshney. 2018.  The effect of extremist violence on hateful speech online. In _Twelfth International AAAI Conference on Web and Social Media_.
  * Panciera et al. (2009) Katherine Panciera, Aaron Halfaker, and Loren Terveen. 2009\.  Wikipedians are born, not made: a study of power editors on Wikipedia. In _Proceedings of the ACM 2009 international conference on Supporting group work_. 51–60.
  * Papacharissi (2002) Zizi Papacharissi. 2002\.  The virtual sphere: The internet as a public sphere.  _New media & society_ 4, 1 (2002), 9–27.
  * Parekh and Patel (2017) Pooja Parekh and Hetal Patel. 2017.  Toxic comment tools: A case study.  _International Journal of Advanced Research in Computer Science_ 8, 5 (2017).
  * Paul et al. (2011) Sharoda A Paul, Lichan Hong, and Ed H Chi. 2011.  Is twitter a good place for asking questions? a characterization study. In _Fifth International AAAI Conference on Weblogs and Social Media_.
  * Platonov et al. (2020) Georgiy Platonov, Lenhart Schubert, Benjamin Kane, and Aaron Gindi. 2020.  A Spoken Dialogue System for Spatial Question Answering in a Physical Blocks World. In _Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue_. Association for Computational Linguistics, 1st virtual meeting, 128–131.  https://www.aclweb.org/anthology/2020.sigdial-1.16
  * Riordan (2017) Monica A Riordan. 2017\.  Emojis as tools for emotion work: Communicating affect in text messages.  _Journal of Language and Social Psychology_ 36, 5 (2017), 549–567.
  * Rojas Barahona et al. (2019) Lina M. Rojas Barahona, Pascal Bellec, Benoit Besset, Martinho Dossantos, Johannes Heinecke, munshi asadullah, Olivier Leblouch, Jeanyves. Lancien, Geraldine Damnati, Emmanuel Mory, and Frederic Herledan. 2019\.  Spoken Conversational Search for General Knowledge. In _Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue_. Association for Computational Linguistics, Stockholm, Sweden, 110–113.
  * Rosé et al. (2008) Carolyn Rosé, Yi-Chia Wang, Yue Cui, Jaime Arguello, Karsten Stegmann, Armin Weinberger, and Frank Fischer. 2008.  Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning.  _International journal of computer-supported collaborative learning_ 3, 3 (2008), 237–271.
  * Rossi and Gnawali (2014) Lorenzo A Rossi and Omprakash Gnawali. 2014.  Language independent analysis and classification of discussion threads in Coursera MOOC forums. In _Proceedings of the 2014 IEEE 15th International Conference on Information Reuse and Integration (IEEE IRI 2014)_. IEEE, 654–661.
  * Rost et al. (2016) Katja Rost, Lea Stahel, and Bruno S Frey. 2016.  Digital social norm enforcement: Online firestorms in social media.  _PLoS one_ 11, 6 (2016), e0155923.
  * Shachaf and Hara (2010) Pnina Shachaf and Noriko Hara. 2010.  Beyond vandalism: Wikipedia trolls.  _Journal of Information Science_ 36, 3 (2010), 357–370.
  * Siegel et al. (1986) Jane Siegel, Vitaly Dubrovsky, Sara Kiesler, and Timothy W McGuire. 1986.  Group processes in computer-mediated communication.  _Organizational behavior and human decision processes_ 37, 2 (1986), 157–187.
  * Silva et al. (2016) Leandro Silva, Mainack Mondal, Denzil Correa, Fabrício Benevenuto, and Ingmar Weber. 2016\.  Analyzing the targets of hate in online social media. In _Tenth International AAAI Conference on Web and Social Media_.
  * Singhal et al. (2022) Mohit Singhal, Chen Ling, Nihal Kumarswamy, Gianluca Stringhini, and Shirin Nilizadeh. 2022.  SoK: Content Moderation in Social Media, from Guidelines to Enforcement, and Research to Practice.  _arXiv preprint arXiv:2206.14855_ (2022).
  * Sood et al. (2012) Sara Sood, Judd Antin, and Elizabeth Churchill. 2012\.  Profanity use in online communities. In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_. ACM, 1481–1490.
  * Stolcke et al. (2000) Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000.  Dialogue act modeling for automatic tagging and recognition of conversational speech.  _Computational linguistics_ 26, 3 (2000), 339–373.
  * Stump et al. (2013) Glenda S Stump, Jennifer DeBoer, Jonathan Whittinghill, and Lori Breslow. 2013.  Development of a framework to classify MOOC discussion forum posts: Methodology and challenges. In _NIPS Workshop on Data Driven Education_. 1–20.
  * Stvilia et al. (2008) Besiki Stvilia, Michael B Twidale, Linda C Smith, and Les Gasser. 2008.  Information quality work organization in Wikipedia.  _Journal of the American society for information science and technology_ 59, 6 (2008), 983–1001.
  * Suler (2004) John Suler. 2004\.  The online disinhibition effect.  _Cyberpsychology & behavior_ 7, 3 (2004), 321–326.
  * Tan et al. (2016) Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016.  Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In _Proceedings of the 25th international conference on world wide web_. 613–624.
  * Twarc (2020) Twarc. 2020.  Collect Twitter Data with Twarc!  https://scholarslab.github.io/learn-twarc/.
  * Ulbig and Funk (1999) Stacy G Ulbig and Carolyn L Funk. 1999.  Conflict avoidance and political participation.  _Political Behavior_ 21, 3 (1999), 265–282.
  * Vijayaraghavan et al. (2021) Prashanth Vijayaraghavan, Hugo Larochelle, and Deb Roy. 2021\.  Interpretable multi-modal hate speech detection.  _arXiv preprint arXiv:2103.01616_ (2021).
  * Walther (2011) Joseph B Walther. 2011\.  Theories of computer-mediated communication and interpersonal relations.  _The handbook of interpersonal communication_ 4 (2011), 443–479.
  * Wang et al. (2020) Yan Wang, Jiayu Zhang, Jun Ma, Shaojun Wang, and Jing Xiao. 2020.  Contextualized Emotion Recognition in Conversation as Sequence Tagging. In _Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue_. Association for Computational Linguistics, 1st virtual meeting, 186–195.  https://www.aclweb.org/anthology/2020.sigdial-1.23
  * Waseem and Hovy (2016) Zeerak Waseem and Dirk Hovy. 2016.  Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In _Proceedings of the NAACL student research workshop_. 88–93.
  * Williams and Burnap (2015) Matthew L Williams and Pete Burnap. 2015.  Cyberhate on social media in the aftermath of Woolwich: A case study in computational criminology and big data.  _British Journal of Criminology_ 56, 2 (2015), 211–238.
  * Wired (2016) Wired. 2016.  Inside Google’s Internet Justice League and its AI-Powered War on Trolls.  https://goo.gl/Nvf6ZA.
  * Wise et al. (2017) Alyssa Friend Wise, Yi Cui, Wanqi Jin, and Jovita Vytasek. 2017\.  Mining for gold: Identifying content-related MOOC discussion threads across domains through linguistic modeling.  _The Internet and Higher Education_ 32 (2017), 11–28.
  * Wulczyn et al. (2017) Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017\.  Ex machina: Personal attacks seen at scale. In _Proceedings of the 26th International Conference on World Wide Web_. 1391–1399.
  * Yu et al. (2003) Keming Yu, Zudi Lu, and Julian Stander. 2003.  Quantile regression: applications and current research areas.  _Journal of the Royal Statistical Society: Series D)_ 52, 3 (2003), 331–350.
  * Zannettou et al. (2018) Savvas Zannettou, Tristan Caulfield, Jeremy Blackburn, Emiliano De Cristofaro, Michael Sirivianos, Gianluca Stringhini, and Guillermo Suarez-Tangil. 2018.  On the Origins of Memes by Means of Fringe Web Communities. In _IMC_.
  * Zannettou et al. (2020) Savvas Zannettou, Mai ElSherief, Elizabeth Belding, Shirin Nilizadeh, and Gianluca Stringhini. 2020\.  Measuring and Characterizing Hate Speech on News Websites. In _12TH ACM WEB SCIENCE CONFERENCE_. ACM.
  * Zhang et al. (2017) Amy Zhang, Bryan Culbertson, and Praveen Paritosh. 2017\.  Characterizing online discussion using coarse discourse sequences.  (2017).
  * Zhang and Cranshaw (2018) Amy X Zhang and Justin Cranshaw. 2018.  Making sense of group chat through collaborative tagging and summarization.  _Proceedings of the ACM on Human-Computer Interaction_ 2, CSCW (2018), 1–27.
  * Zhang et al. (2018a) Justine Zhang, Jonathan P Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Nithum Thain, and Dario Taraborelli. 2018a.  Conversations gone awry: Detecting early signs of conversational failure.  _arXiv preprint arXiv:1805.05345_ (2018).
  * Zhang et al. (2018b) Justine Zhang, Cristian Danescu-Niculescu-Mizil, Christina Sauper, and Sean J Taylor. 2018b.  Characterizing online public discussions through patterns of participant interactions.  _Proceedings of the ACM on Human-Computer Interaction_ 2, CSCW (2018), 1–27.
  * Zhang et al. (2016) Justine Zhang, Ravi Kumar, Sujith Ravi, and Cristian Danescu-Niculescu-Mizil. 2016.  Conversational flow in Oxford-style debates.  _arXiv preprint arXiv:1604.03114_ (2016).