# The Unappreciated Role of Intent in Algorithmic Moderation
of Social Media Content

Xinyu Wang1, Sai Koneru1*, Pranav Narayanan Venkit1*, Brett Frischmann2, Sarah
Rajtmajer1

###### Abstract

As social media has become a predominant mode of communication globally, the
rise of abusive content threatens to undermine civil discourse. Recognizing
the critical nature of this issue, a significant body of research has been
dedicated to developing language models that can detect various types of
online abuse, e.g., hate speech, cyberbullying. However, there exists a
notable disconnect between platform policies, which often consider the
author’s _intention_ as a criterion for content moderation, and the current
capabilities of detection models, which typically lack efforts to capture
intent. This paper examines the role of intent in content moderation systems.
We review state of the art detection models and benchmark training datasets
for online abuse to assess their awareness and ability to capture intent. We
propose strategic changes to the design and development of automated detection
and moderation systems to improve alignment with ethical and policy
conceptualizations of abuse.

**footnotetext: These authors contributed equally to this work.

## Introduction

The onset of Web 2.0 has revolutionized the ways we engage with information
and with one another. Within vast digital information ecosystems, social media
stands out as the torchbearer of the new age information revolution
(Westerman, Spence, and Van Der Heide 2014; Kümpel, Karnowski, and Keyling
2015). Platforms such as Twitter and Facebook bear witness to the daily lives
of billions and serve as hubs of public discourse (Shearer 2018; Bode 2016).

Alongside exciting benefits of access and connectivity, come the perils of
digital abuse (Ciampaglia 2018; Zimdars and McLeod 2020). Various forms of
digital abuse, particularly hate speech, has been linked to considerable real-
world harm, with research indicating that online hate speech may materialize
subsequent acts of violence in offline environments (Awan and Zempi 2015;
Olteanu et al. 2018). Similarly, disinformation has been utilized
strategically to manipulate group perceptions and undermine public trust
(Hameleers, Brosius, and de Vreese 2022; Wang et al. 2023).

To this end, researchers have dedicated significant effort to define, detect,
measure and model different types of abusive content online. With respect to
definition, the term _abuse_ spans a spectrum of harmful language, including
generalized hate speech and specific offenses like sexism and racism.
Definitions for these types of abuse often intersect and lack precise
boundaries. The various definitions and taxonomies which have been proposed to
describe abusive content is a focus later in this paper. Broadly, we will use
the term to refer to any expression that is meant to denigrate or offend a
particular person or group (Mishra, Yannakoudakis, and Shutova 2019).

Common to many definitions of digital abuse is some notion of _intent_ , e.g.,
(Vidgen and Derczynski 2020; Molina et al. 2021; Hashemi 2021; French, Storey,
and Wallace 2023). Platform policies often highlight the intentional nature of
abusive content, yet intent is not a feature of content itself. Rather, it is
a subjective state of mind attributable to an actual person, typically the
speaker, poster, or sharer. In cognitive science, ethics, law, and philosophy,
intent is a contested and complicated concept. As Frischmann and Selinger
(2018, p. 364) explain:

> Intention is a mental state that is part belief, part desire, and part
> value. My intention to do something – say to write th[is] explanatory text…
> or to eat an apple – entails (1) beliefs about the action, (2) desire to
> act, and (3) some sense of value attributable to the act.

In pragmatic ethical and legal contexts, the focus often turns to evidence of
intent. For example, a written signature is taken to be an objective
manifestation that a person intends to enter into a contract. Thus, by
including intent in definitions of abusive content, the implicit challenge is
divining subjective state of mind from evidence manifested in the content
itself and the surrounding context.

Unsurprisingly, then, intent is exceedingly difficult to capture
algorithmically through short text. ToxicBert (Hanu and Unitary team 2020),
for example, can label sentences as hateful or toxic but lacks the ability to
interpret context in the input, oftentimes because insufficient context is
provided (Wang et al. 2020a; MacAvaney et al. 2019). ToxicBert flags the
sentence ‘I’m going to kill you if you leave the dishes for me again’ as toxic
and threatening, highlighting its limitations differentiating between literal
and figurative language.

In the context of abuse detection online, intricacies of human intent are
collapsed down to a class label in a training dataset, e.g., hate speech vs.
not hate speech. Moreover, algorithms trained and deployed on short text in
isolation lack information about sociocultural and situational context central
to questions of intent (Gao and Huang 2017; Wang et al. 2020a; MacAvaney et
al. 2019). In this way, there is misalignment between automated detection
models for instances of online abuse and the human-drafted and human-centered
systems they serve (Arora et al. 2023). Our work examines this mismatch,
scaffolded by the following research questions:

RQ1: What role does intent play in existing social media policies for abuse
moderation?

RQ2: What is the current state of annotating and detecting common forms of
online abuse, particularly focusing on hate speech and cyberbullying?

RQ3: How can intent be better incorporated into existing annotation,
detection, and moderation pipelines to align with content moderation policies?

In the following sections of this survey, we construct a landscape of the
online abuse moderation policies of major social media platforms, outline
existing taxonomies of online abuse, survey state of the art detection models
and benchmark, labeled datasets, and propose prospective approaches to better
align detection technologies with policies.

Surveys of social media harms and automated detection are documented in the
literature, typically categorizing detection algorithms by their underlying
technologies. Our focus here is assessment of detection algorithms along the
axis of intent. This approach highlights the implications of these
technologies in real-world applications, emphasizing their intersection with
ethics and policy.

## Taxonomies of Digital Abuse

Extensive research has investigated the diverse forms of online harm across
various digital platforms, including Facebook, Twitter, Tinder, and others
(Arora et al. 2023; Scheuerman et al. 2021; Im et al. 2022; Keipi et al.
2016). These studies have categorized a range of abusive actions, with
prominent themes including hate speech, cyberbullying, and discrimination
(Arora et al. 2023; Scheuerman et al. 2021). Each of these harms are
consequent to nuanced user interactions, which are often platform-specific,
culture-sensitive and context-dependent. Prior work has highlighted the
differing definitions of online harms in the literature (Fortuna and Nunes
2018). There is still substantial messiness around definitions, posing
challenges for the development of consistent and effective moderation
pipelines.

Various approaches to classification of online abuse emphasize different
dimensions of the phenomena. Some taxonomies highlight the target of abuse,
exploring whether the abuse is directed at individuals, groups, or concepts
(Vidgen et al. 2019; Waseem et al. 2017; Al Mazari 2013). Conversely, some
focus on characteristics of the abuse—whether it is explicit or implicit
(Mladenović, Ošmjanski, and Stanković 2021), while still others explore
subcategories of abuse or harm (Lewandowska-Tomaszczyk et al. 2023; Gashroo
and Mehrotra 2022). An overview of existing taxonomies is provided in Table 1.

Our work centers around the dimension of intent—which features prominently in
platform policies, but is often implicit in current taxonomies, e.g.,
operationalized through the notion of targeting. We emphasize two common types
of explicit online abuse: hate speech and cyberbullying (Wiegand, Ruppenhofer,
and Kleinbauer 2019). These forms of abuse have not only received significant
attention in Natural Language Processing (NLP) but are also intertwined with
the concept of intent; both reflect deliberate aim to harm or intimidate
specific individuals or groups.

Reference |  Author-defined scope |  Categories
---|---|---
Nocentini et al. (2010) |  Cyberbullying |  Written-verbal behavior, Visual behavior, Exclusion, Impersonation.
Al Mazari (2013) |  Cyberbullying |  Child cyberbullying, Cybergrooming, Adults cyberstalking, Workplace cyber-bullying.
Agrafiotis et al. (2016) |  Cyber Harm |  Physical, Psychological, Economic, Reputational, Cultural, Political.
Miró-Llinares and Rodríguez-Sala (2016) |  Hate Speech & Violent Communication |  Violent incitement, Personal offence, Discrimination incitement, Collective offence.
Waseem et al. (2017) |  Online Abuse |  Two-fold typology: Directed towards a specific individual or entity & Used towards a generalized other; Explicit & Implicit.
Salminen et al. (2018) |  Online Hate |  Targets: Financial power (Corporation, wealthy), Political issues (Terrorism, Politics, Ideology), Racism & xenophobia (Anti-white, Anti-black, Xenophobia), Religion (Anti-Islam, Anti-Semitist), Specific nation(s), Specific person, Media (Towards media company, Other), Armed forces (Police, Military), Behavior (Humanity, Other); Language: Accusations, Humiliation, Swearing, Promoting violence.
Anzovino, Fersini, and Rosso (2018) |  Misogyny |  Discredit, Stereotype and objectification, Sexual harassment and threats of Violence, Dominance, Derailing.
Vidgen et al. (2019) |  Online Abuse |  Individual-directed abuse, Identity-directed abuse, Concept-directed abuse.
Vidgen and Derczynski (2020) |  Online Abuse |  Person-directed abuse, Group-directed abuse, Flagged content, incivil content, Mixed.
Banko, MacKeen, and Ray (2020) |  Online Harm |  Hate and harassment (Doxxing, Identity attack, Identity misrepresentation, Insult, Sexual aggression), Self-Inflicted harm (Eating disorder promotion, Self-harm, Threat of violence), Ideological harm (Extremism, Terrorism & Organized crime, Misinformation), Exploitation (Adult sexual services, Scams, Child sexual abuse Material).
Vidgen et al. (2021a) |  Online Abuse |  Identity-directed abuse (Derogation, Animosity, Threatening, Glorification, Dehumanization), Affiliation-directed abuse (Derogation, Animosity, Threatening, Glorification, Dehumanization), Person-directed abuse (Abuse to them, Abuse about them)
Sajadi Ansari et al. (2021) |  Cyberbullying |  Flaming, Harassment, Sexual, Threat, Trickery.
Mladenović, Ošmjanski, and Stanković (2021) |  Objectionable Content (Cyberbullying) |  Fourfold typology: Expression (Explicit, Implicit), Targeting (Targeted, Untargeted), Orientation (Directed, Generalized), Frequency (Repeated, Unrepeated).
Alrashidi et al. (2022) |  Abusive Content |  Abusive and offensive language, Hate speech, Cyberbullying, Targeted groups (Religious and racism, Gender and misogyny).
Gashroo and Mehrotra (2022) |  Online Abuse |  Abusive language, Aggression, Cyberbullying, Insults, Personal attacks, Provocation, Racism, Sexism, Toxicity.
Lewandowska-Tomaszczyk et al. (2023) |  Offensive Language |  Taboo (Obscene, Profane), Insulting (Abusive), Hate Speech (Slur), Harassment (Cyber-bullying), Toxic.
Kogilavani et al. (2023) |  Offensive Language |  Aggression, Cyberbullying, Hate speech, Offensive language, Toxic comments.

Table 1: Summary of existing taxonomies of digital abuse on social media.

Hate Speech. Hate speech, a common form of digital abuse, has been widely
recognized as a harmful act in online spaces. However, its definition remains
a subject of ongoing debate. One prominent definition of hate speech is speech
that attacks or discriminates against a person or group on the basis of
attributes such as race, religion, ethnic origin, national origin, sex,
disability, sexual orientation, or gender identity (Lepoutre et al. 2023;
Brown 2017a).

Hate speech can take various forms, including verbal, non-verbal, and symbolic
expressions (Nielsen 2002). It often employs ambiguous, metaphorical, and
roundabout language (Giglietto and Lee 2017; Ana 1999), making identification
challenging (Paz, Montero-Díaz, and Moreno-Delgado 2020). This complexity is
compounded by the fact that hate speech can be intention-driven, and its
interpretation can vary depending on the context and cultural background (Paz,
Montero-Díaz, and Moreno-Delgado 2020; Wang, Wu, and Rajtmajer 2023). In the
annotation guidelines for their hate speech dataset, Kennedy et al. (2022)
define it as “language that intends to attack the dignity of a group of people
— through rhetorical devices and contextual references — either by inciting
violence, encouraging the incitement to violence, or inciting hatred." This
definition underscores the role of intent, as reflected through context and
presence of a specific target.

Existing literature offers various frameworks for understanding these
behaviors. One perspective views hate speech as a ‘myth’, arguing that it
lacks a coherent definition and is instead a family resemblance concept with
fuzzy boundaries (Brown 2017b). An alternative approach proposes a corpus-
driven definition, allowing hate speech to be understood as an evolving
concept that adapts to societal changes (Lepoutre et al. 2023). Brown (2017b)
also distinguishes between direct attacks on protected groups and indirect,
charged rhetoric that incites hatred or discrimination.

Cyberbullying. Another common form of online abuse is cyberbullying. One
definition of cyberbullying is the use of electronic communication
technologies like the internet, social media, and mobile phones to
intentionally harass, threaten, humiliate or target another person or group
(Campbell and Bauman 2018; Wright 2021). It can take many forms, e.g., sending
mean messages or threats, posting embarrassing pictures/videos, creating fake
profiles, or excluding someone from online groups (Wright 2019; Iqbal and Jami
2022). Prior work elaborates on how this behavior allows perpetrators to
remain anonymous and reach wider audiences more rapidly traditional bullying.
Cyberbullying can occur 24/7 and follow victims into their private spaces,
making it difficult to escape (Wright 2019). Common tactics include
harassment, denigration, outing/trickery, exclusion, cyberstalking, and
fraping (sending embarrassing content from someone’s account) (Iqbal and Jami
2022; Campbell and Bauman 2018). Across all prior work, cyberbullying is
considered an explicit and intentional act.

Recent work Arunkarthick et al. (2023); Nisha and Jebathangam (2022); Sultan
et al. (2023) has tried to use automatic methods from NLP to identify
cyberbullying. These methods use XGBoost, LSTM and CNNs for fake account
detection and logistic regression to detect cyberbullying. They learn patterns
in training data and identify similar patterns during test, largely ignoring
the role of intent.

In the upcoming sections, we survey existing algorithms for abuse detection,
with focus on hate speech and bullying. In particular, we look at the training
datasets underlying these algorithms and the role (if any) of intent during
dataset annotation. We also survey the set of features extracted from datasets
and used for algorithm development and testing. These features offer insight
into the types of context available to algorithmic content moderation
algorithms.

## Defining Intent

Intentions are a state of mind, or put another way, a component of a person’s
will. People exercise their autonomy by acting upon their intentions. It is
notoriously difficult to read another person’s mind and determine a person’s
beliefs, desires, preferences, and intentions. Most often, we rely on external
manifestations, such as what people say or do. In content moderation, like
many other contexts, intent is often inexorably intertwined with actions.
Thus, when attempting to determine the intent associated with abusive content,
one must identify the relevant actor(s) and action(s). After all, content is a
sociotechnical artifact with history and context, and multiple relevant actors
and actions may be involved with its arrival in the moderation system. In
legal contexts, proving intent often depends on evidence of either purpose,
where there is a conscious desire to perform an act or achieve a specific
outcome, or knowledge, where an individual is aware that their actions will
likely lead to a particular result and is indifferent to this outcome (Crump
2009).

For moderating abuse or offense online, this foundational definition of intent
can be built upon to address the specific challenges of digital communication.
In this context, evaluating/determining intent involves trying to understand
the states of mind of different users (content creators, posters, sharers,
influencers, etc.) regarding their actions (producing, posting, distributing,
promoting, etc.) and attendant consequences, e.g., inciting violence,
promoting hate, or bullying.

In the context of automated detection systems like language models, intent
must be inferred from text without the benefit of human insights. Compounding
this challenge, the relevant actor may be the speaker (content creator), the
poster, the sharer, or someone else involved in the series of events
(amplifier, ad booster, influencer). There may be one or more relevant and
interdependent acts. A content creator may generate hate speech with or
without intent to cause harm (whether evaluated in terms of purpose or
knowledge); the same is true with respect to the poster or sharer.

In practice, moderating content based on intent creates an additional
governance hurdle and thus narrows the range of content subject to possible
removal. Stated another way, consider intent to be one dimension by which
taxonomies of abuse and subsequent moderation systems are scaffolded.
_Intentional harm_ is then a cross-cutting category of online abuse for which
governance is more easily legitimized.

## Intent in Platform Policies and Actions

Major platforms like Twitter (now "X") and Meta (formerly Facebook) emphasize
the importance of understanding the intent behind content to determine its
appropriateness. For example, Twitter’s guidelines outline specific criteria
for identifying violent and hateful entities, focusing on deliberate actions
to promote violence or hate.

> Violent entities are those that deliberately target humans or essential
> infrastructure with physical violence and/or violent rhetoric as a means to
> further their cause. These include, but are not limited to, terrorist
> organizations, violent extremist groups, and perpetrators of violent
> attacks…
>
> Hateful entities are those that have systematically and intentionally
> promoted, supported and/or advocated for hateful conduct, which includes
> promoting violence or engaging in targeted harassment towards a protected
> category.111https://help.twitter.com/en/rules-and-policies/violent-entities

Similarly, Meta’s policies underscore the role of context and intent,
particularly in cases reported as hate speech.

> … context can indicate a person’s intent, which can come into play when
> something is reported as hate
> speech.222https://about.fb.com/news/2017/06/hard-questions-hate-speech/

Instagram also addresses the complexity of moderating hate speech by
considering the intent behind shared content. The platform allows content that
might be deemed hateful if it’s shared to challenge or raise awareness about
the issues discussed, provided the intent is clearly communicated.

> It’s never OK to encourage violence or attack anyone based on their race,
> ethnicity, national origin, sex, gender, gender identity, sexual
> orientation, religious affiliation, disabilities, or diseases. When hate
> speech is being shared to challenge it or to raise awareness, we may allow
> it. In those instances, we ask that you express your intent
> clearly.333https://help.instagram.com/477434105621119

TikTok describes hate speech as intentional as well.

> …content that intends to or does attack, threaten, incite violence against,
> or dehumanize an individual or group of individuals on the basis of
> protected attributes like race, religion, gender, gender identity, national
> origin, and more.444https://newsroom.tiktok.com/en-us/countering-hate-on-
> tiktok

From the platforms’ perspectives, detection is closely linked to _response_.
Moderation policies should be aligned with evolving social norms and user
expectations, integrated within a comprehensive governance framework that
bridges the gap between detection of abusive content and taking appropriate
actions. We suggest that this frame has important design implications for
detection algorithms: they should not be designed as independent products, but
instead should be incorporated into broader governance systems, as assistive,
enabling technologies for human-centered efforts to create safe and inclusive
digital spaces.

## Online Abuse Datasets

In this section, we review existing datasets used for training online abuse
detection models. We categorize and critique these datasets based on the
extent to which they consider the context and intent of abusive expressions
during annotation. The limitations identified here underscore the need for
enriched datasets that better mirror the complexities of real-world
interactions for detection tasks.

### Inclusion criteria

We reviewed dataset papers from 2016 to 2024 using Scopus search along with
datasets referenced in (Vidgen and Derczynski
2020)555https://hatespeechdata.com/, and citation search. We applied the
following inclusion criteria: 1) The paper presents a novel dataset for which
annotation procedures are described; 2) The dataset is intended for training
and testing algorithm(s) aimed at abuse detection; 3) The dataset is curated
from one or more widely-used social media platforms; 4) The dataset is solely
in English; 5) The dataset includes textual content. Search and filtering
processes resulted in 42 dataset papers/datasets. Specific search terms and
PRISMA diagram for screening pipeline are provided in the Appendix.

### Dataset categorization

Table 2 outlines the papers reviewed, noting whether any contextual
information was provided to annotators, whether intent was explicitly
mentioned in annotation instructions, and whether the annotation task included
identification of a target person or group.

Intent mentioned. Many papers/datasets do not ask annotators to consider
intent during labeling. We capture whether annotation guidelines mention
intent, intention, etc., anywhere in instructions or definitions.

Context provided. When provided, context takes one of two forms–conversations
surrounding the text or metadata. _Conversations._ Annotators are provided
conversation/dialogue surrounding the text being annotated. _Metadata._
Annotators are provided user-level metadata, e.g., profile content, and
geographical location, or post-level metadata, e.g., images or text extracted
from images.

Target annotation. In some cases, authors request that annotators identify the
person or group targeted by a comment during the annotation process.
Understanding of the target may impact the presence and/or category of abuse.

### Limitations of current datasets

Our review highlights several key challenges to the alignment between
platforms’ policies around abuse and the datasets used to train abuse
detection algorithms.

Ambiguity in definitions of digital abuse. Compounding the diversity of
definitions and taxonomies of abuse proposed in academic work and discussed
above, instructions to annotators are often vague. This can result in lack of
reusable training data and benchmarks. In the box below, we provide excerpts
describing annotation processes from surveyed papers.

Reference |  Source |  Author-defined |  Content type |  Intent |  Context provided |  Target
---|---|---|---|---|---|---
|  |  scope |  |  mentioned |  |  annotation
Waseem and Hovy (2016) |  Twitter |  Hate Speech |  Text |  No |  No |  No
Waseem (2016) |  Twitter |  Hate Speech |  Text |  No |  No |  No
Golbeck et al. (2017) |  Twitter |  Online Harassment |  Text |  Yes |  No |  No
Chatzakou et al. (2017) |  Twitter |  Cyberbullying |  Multimodal |  Yes |  Metadata |  No
Gao and Huang (2017) |  Fox News |  Hate Speech |  Text |  No |  Conversation |  No
Davidson et al. (2017) |  Twitter |  Hate Speech |  Text |  Yes |  No |  No
Gao, Kuppersmith, and Huang (2017) |  Twitter |  Hate Speech |  Text |  No |  No |  No
Jha and Mamidi (2017) |  Twitter |  Sexism |  Text |  No |  No |  No
Van Hee et al. (2018) |  ASKfm |  Cyberbullying |  Text |  Yes |  Conversation |  Yes
Fersini et al. (2018) |  Twitter |  Misogyny |  Text |  Yes |  No |  Yes
Ribeiro et al. (2018) |  Twitter |  Hate Speech |  Multimodal |  No |  Metadata |  No
ElSherief et al. (2018) |  Twitter |  Hate Speech |  Text |  No |  No |  Yes
Founta et al. (2018) |  Twitter |  Abusive Behavior |  Multimodal |  Yes |  No |  No
Rezvan et al. (2018) |  Twitter |  Online Harassment |  Text |  Yes |  No |  No
Salminen et al. (2018) |  YouTube, Facebook |  Online Hate |  Text |  Yes |  No |  Yes
Zampieri et al. (2019) |  Twitter |  Offensive Language |  Text |  No |  No |  Yes
Qian et al. (2019) |  Reddit, Gab |  Hate Speech |  Text |  No |  Conversation |  No
Ousidhoum et al. (2019) |  Twitter |  Hate Speech |  Text |  No |  No |  Yes
Basile et al. (2019) |  Twitter |  Hate Speech |  Text |  No |  No |  Yes
Mandl et al. (2019) |  Twitter |  Hate Speech and Offensive Content |  Text |  No |  No |  Yes
Wijesiriwardene et al. (2020) |  Twitter |  Toxicity |  Multimodal |  Yes |  Conversation |  No
Gomez et al. (2020) |  Twitter |  Hate Speech |  Multimodal |  No |  Metadata |  No
Vidgen et al. (2020) |  Twitter |  Hate Speech |  Text |  Yes |  No |  No
Caselli et al. (2020) |  Twitter |  Abusive Language |  Text |  Yes |  No |  No
Ziems, Vigfusson, and Morstatter (2020) |  Twitter |  Cyberbullying |  Text |  Yes |  Conversation |  Yes
Kennedy et al. (2020) |  Twitter, Reddit, YouTube |  Hate Speech |  Text |  Yes |  No |  Yes
Aggarwal, Maurya, and Chaudhary (2020) |  Twitter, Reddit, Formspring |  Cyberbullying |  Text |  No |  No |  No
Suryawanshi et al. (2020) |  Kaggle, Reddit, Facebook, Twitter, Instagram |  Offensive Language |  Multimodal |  Yes |  Metadata |  No
Van Bruwaene, Huang, and Inkpen (2020) |  Facebook, Instagram, Twitter, Pinterest, Tumblr, Youtube |  Cyberbullying |  Text |  Yes |  Metadata |  No
Grimminger and Klinger (2021) |  Twitter |  Hate Speech |  Text |  No |  No |  No
Qureshi and Sabih (2021) |  Twitter |  Hate Speech |  Text |  No |  No |  No
Salawu, Lumsden, and He (2021) |  Twitter |  Cyberbullying |  Text |  Yes |  No |  No
Samory et al. (2021) |  Twitter |  Sexism |  Text |  No |  No |  No
He et al. (2021) |  Twitter |  Hate Speech |  Multimodal |  Yes |  No |  No
Vidgen et al. (2021a) |  Reddit |  Abusive Language |  Text |  Yes |  Conversation |  Yes
Ashraf, Zubiaga, and Gelbukh (2021) |  Youtube |  Abusive Language |  Text |  Yes |  Conversation |  No
Mathew et al. (2021) |  Twitter, Gab |  Hate Speech |  Text |  No |  No |  Yes
Albanyan and Blanco (2022) |  Twitter |  Hate Speech |  Text |  No |  Conversation |  No
Toraman, Şahinuç, and Yilmaz (2022) |  Twitter |  Hate Speech |  Text |  No |  No |  No
Thapa et al. (2022) |  Twitter |  Hate Speech |  Multimodal |  No |  Metadata |  No
Kennedy et al. (2022) |  Gab |  Hate Speech |  Text |  Yes |  No |  Yes
Albanyan, Hassan, and Blanco (2023) |  Twitter |  Hate Speech |  Text |  No |  Conversation |  Yes

Table 2: Summary of datasets for social media abuse.

“We label tweets as containing hate speech if they target, incite violence
against, threaten, or call for physical damage for an individual or a group of
people because of some identifying trait or characteristic.” (Toraman,
Şahinuç, and Yilmaz 2022) “The text was further annotated as being hateful and
non-hateful. We did not separate if a group or a single person was targeted by
hateful language." (Grimminger and Klinger 2021) “Not Offensive (NOT): Posts
that do not contain offense or profanity; Offensive (OFF): Posts containing
any form of non-acceptable language (profanity) or a targeted offense, which
can be veiled or direct. This includes insults, threats, and posts containing
profane language or swear words."(Zampieri et al. 2019)

Inconsistent consideration of intent during annotation. Of the dataset papers
surveyed, 47.6% (20 of 42) explicitly mention intent during annotation. While
35.7% (15 of 42) provide context to annotators to help them better infer
intent. 33.3% (14 of 42) of the papers require annotators to identify the
target of abuse. Just 3 papers (Ziems, Vigfusson, and Morstatter 2020; Van Hee
et al. 2018; Vidgen et al. 2021a) provide contextual information to
annotators, explicitly acknowledge the role of intent in annotation
guidelines, and request annotators identify the target of the abuse.

Cross-platform differences. Text and annotation instructions are typically
provided to annotators outside of the platforms from which they were
collected. Labels are considered universal, rather than tailored to the
environments and operational settings of specific social media platforms. This
may undermine their applicability in real-world scenarios.

### An annotation checklist

Critical to annotation tasks is a comprehensive and well-defined rubric. For
annotations of digital abuse, we suggest that the development of such a rubric
should prioritize questions that crisply define the bounds of the phenomenon
and highlight the role of intent:

Regarding information provided to annotators.

  * What are the definitions and scope of online abuse presented to the annotator?

What underlying taxonomy is provided, and how should it be applied (e.g.,
modified, integrated) during annotation?

What contextual information is provided to annotators to assist in the
annotation process?

What is the platform’s content moderation policy, and does the annotation
rubric align with this policy?

What is the platform’s moderation policy for abusive content and does the
annotation rubric adhere to the policy?

Regarding information solicited from annotators.

  * Is abusive or offensive language present?

Is there identifiable intent behind the dissemination of the content, if there
is abusive language present?

Who are the initiators and the targets?

Regarding assessment and reporting.

  * When was the data collected and when was it labeled?

Who are the annotators (demographics, etc.)?

What is the agreement score amongst annotators?

What are the data points with low agreement? What are potential reasons for
disagreement?

Careful attention to these questions will ensure a more methodical and
_interpretable_ approach to the development of benchmark datasets. This will
in turn support the development of detection models that do what we expect
them to do, particularly in the context of real-world applications.

## Online Abuse Detection Algorithms

Focusing on the technical aspects, this section details the current state of
online abuse detection models, offering a categorization of the features
utilized by these models and identifying the gaps that exist in effectively
detecting abuse.

### Inclusion criteria

We queried SCOPUS for published papers presenting abuse detection algorithms.
We removed papers that were not accessible, not written in English, or did not
describe an algorithm for detection of online abuse. As before, we removed
models designed for multi-lingual or non-English tasks. This process resulted
in 168 detection-related papers. Specific search terms and PRISMA diagram for
screening pipeline are provided in the Appendix.

### Features considered by detection models

A limitation of current detection models is their reliance on text analysis.
While these models have made notable advances in identifying abusive content
through textual data, they often fail to consider the complex nature of social
media interactions, which include aspects like user status, social network
structures, and offline context. Here we argue that social media interactions
cannot be fully understood or assessed for intent and harm solely through
short text. Social media dynamics, group behaviors and affiliations, and
specific platform affordances play a role in the creation and manifestation of
online abuse. Effective detection models must integrate rich data to capture
this complexity, including user metadata, contextual information, network
analysis, and policy-aware modeling.

Following, we enumerate and categorize features utilized by the detection
models we surveyed to capture context beyond traditional text mining
approaches, i.e., Bag-of-Words, TF-IDF, and embeddings. A full mapping of
features to papers for the 168 papers in our survey is in the Appendix.

User metadata. Information about a user or an account, whether the speaker or
the target of a potentially abusive comment, may help to infer intentionality
or harm. For example, certain words might be acceptable among some users,
whereas the same words could be considered abusive when used by others.
Likewise, patterns of behavior can be indicative of intent, e.g., users who
repeatedly engage in abusive behavior may be more intentional. This can be
operationalized through characterization of the history of a user’s activities
(Dadvar et al. 2013). More standardized user-level metadata, such as the
geographical location of the user and the follower-following statistics of the
message sender, have been shown to correlate with the occurrence of abusive
content and are integrated as features in detection models (Bozyiğit, Utku,
and Nasibov 2021).

Post metadata. Most social media platforms attach metadata to each post, e.g.,
engagement metrics, mentions, and hashtags. These can reflect the broader
context of a message. For instance, high engagement levels (likes, shares,
comments) might indicate the popularity of (or controversy around) a post,
while particular mentions and hashtags can indicate relevance to specific
communities or ongoing discussions. Suhas et al. (2022) incorporate hashtags
and emojis as distinct features separate from the main text content. Bozyiügit
et al. (2021) integrate post-level metadata, such as the number of retweets or
mentions, to improve the performance of these models for detection of
cyberbullying.

Image and video data. Many platforms have evolved to include a variety of
media formats. Recognizing this, some researchers have extended their focus
beyond text to include images and videos (Nisha and Jebathangam 2022; Qiu et
al. 2022). The additional context that visual and audio elements can provide
may improve the detection of abusive content.

Psychological and cognitive features. Patterns of language may reflect
personality, emotional states, and psychological traits (Alonso and Romero
2017). Understanding the psychological and cognitive dimensions of users’
behavior is particularly critical for understanding intent. Balakrishnan et
al. (2020) incorporate multidimensional personality traits as features for
cyber-aggression detection models.

Conversations. The conversation thread and previous interactions can offer
useful context around potentially abusive language and provide evidence of
intent. Ziems et al. (2020) incorporated features such as timeline similarity
and mentions overlap based on shared conversations between the author and the
target.

Graph structure. The relationships and interactions within social
networks—such as who users connect with, how they interact with these
connections, and the nature of the communities they are part of—can offer
clues about users’ intent. For instance, users embedded in tight networks may
adopt similar communication patterns, which could be innocuous or abusive
depending on norms of that group. Authors have incorporated network centrality
measures for detection of cyberbullying (Singh, Huang, and Atrey 2016).

Policy or rule-aware models. Norms within various online communities can shape
what is viewed as inappropriate (Chandrasekharan et al. 2018). Policy or rule-
aware models are aim to ensure that automated systems adhere to guidelines and
standards. The approach is particularly effective in environments where
regulations may vary significantly, e.g., across cultural contexts. Kumar et
al. (2023) conducted prompt engineering to incorporate large language models
(LLMs) into content moderation by including rules within the prompts.
Calabrese et al. (2022) proposed a representation of moderation policies
tailored for machine interpretation and illustrated how techniques from intent
classification and slot filling can be applied to detect abusive content.

Sentiment. Sentiment analysis is a valuable component of many detection
models. Sentiment features provide insights into the emotional tone of
language which might not be apparent through baseline text analysis (Geetha et
al. 2021).

Topics and themes. Topic modeling techniques, such as Latent Dirichlet
Allocation (LDA) (Blei, Ng, and Jordan 2003) or theme categorization (Perera
and Fernando 2021), allow detection models to understand the subject matter of
discussions. Models can learn whether certain topics are more likely to
involve harmful language or cyberbullying. Mushed et al. (2023) employed a
clustering-based topic modeling technique to improve the accuracy of
cyberbullying detection. Perera et al. (2021) measured frequency of
themes/categories associated with cyberbullying, e.g., racist, sexual, and
physical, to improve detection.

Linguistic cues. Words and phrases that are associated with offensive or
abusive language are commonly used for abuse detection. This includes explicit
language, slurs, and aggressive or threatening terms. Common approaches
include constructing personalized dictionaries and using Linguistic Inquiry
and Word Count (LIWC) for feature extraction (Geetha et al. 2021). Since TF-
IDF and Bag-of-Words approaches are standard practices in NLP, we do not
categorize them as nuanced uses of linguistic cues.

Figure 1 shows the number of papers that use the above 10 features to capture
context and infer intent. We observe that user metadata are the most
frequently used features with 16 (9.5%) articles using them for detection.
Similarly, linguistic cues are used in 14 (8.3%) studies and post metadata in
10 (5.9%). In contrast, psychological and cognitive dimensions, and policy or
rule-aware models are less frequently explored, with just 4 (2.2%) and 2
(1.1%) papers respectively. A majority of the reviewed papers, 58.3% (98 of
168), focus on comparing model performance, enhancing predictive capabilities,
and benchmarking against various metrics.

Figure 1: Distribution of papers based on features utilized by the detection
models.

Among 168 surveyed papers, three of them present models incorporating six out
of the ten features listed above, meaningfully including context and
potentially addressing the issue of intent (López-Vizcaíno et al. 2021;
Dhingra et al. 2023; Ziems, Vigfusson, and Morstatter 2020). Building upon
(Ziems, Vigfusson, and Morstatter 2020), a subsequent paper utilized the
dataset from the earlier study and applied methodological improvements
(Dhingra et al. 2023).

### Limitations of current detection algorithms

Figure 2: Example of prompts and detection model performance to showcase the
importance of context and intent in understanding online abuse.

Notably, the relationship between intent and abuse is also recognized by Chat
GPT. We prompted GPT 3.5 to classify statements as hateful or not hateful,
with and without contextual cues (see Fig. 2). The model was able to
appreciate the ways in which context and intention inform the statement and
change the label accordingly. However, many existing models, like those based
on BERT, fall short in capturing context to infer intent. We evaluated four
different models (Mathew et al. 2020; Barbieri et al. 2020; Das, Banerjee, and
Mukherjee 2022; Vidgen et al. 2021b) using the same examples as those tested
with GPT-3.5. The results (see Fig. 2) show inconsistent outputs and a
significant reliance on word-level cues, highlighting their limitations in
contextual understanding. Based on these observations and our review of the
relevant literature, we summarize the current limitations of detection
algorithms for online abuse.

Insufficient information from text-based data. Traditional text-based models,
though robust in their linguistic analyses, fall short when it comes to
understanding the complexities of group dynamics in the spread of online
abuses (Salminen et al. 2018). For instance, an abusive narrative might emerge
and propagate not merely because of its textual content but due to the
influence and endorsement of a closely-knit group within the network (Marwick
and Lewis 2017). Such group-based manifestations often serve as catalysts,
amplifying the reach and impact of abusive content. Their omission from
detection algorithms unintentionally leaves models vulnerable to overlooking
significant sources of online abuse. Moreover, these group-based interactions
are influenced by factors such as shared ideologies, mutual affiliations, or
even orchestrated campaigns, which sometimes employ subtle linguistic cues not
easily detectable by conventional text-based models. The complexity is further
compounded when such groups employ tactics like code-switching, euphemisms, or
meme-based communication, thereby effectively circumventing text-based
detection mechanisms.

Static perspectives on abuse. Social norms are highly dynamic, influencing the
interpretation of what constitutes abusive content (Crandall, Eshleman, and
O’brien 2002). Traditional detection models often fail to account for these
longitudinal shifts. For example, a phrase may take on a new meaning in the
digital realm, and the rapid evolution of internet slang also necessitates a
more flexible approach for continuous training and updating.

Poor generalizability. Model performance can fluctuate significantly even when
utilizing the same dataset, with optimal outcomes frequently exclusive to that
specific dataset (Leo, Santoso, and Pratomo 2023). This inconsistency
highlights the need for robust testing pairing different models with different
datasets to understand the generalizability of existing methods in new
contexts.

## Prospective Strategies and Challenges

Having reviewed the current state of NLP research on hate speech and
cyberbullying annotation and detection, we have pinpointed several gaps that
suggest opportunities for further exploration and improvement. Following, we
explore potential strategies and associated challenges in four key areas:
annotation, detection, moderation, and technology design.

### Annotation: Introduce context and intent

As discussed, abuse does not have a universally agreed-upon definition. What
might be considered offensive in one culture or context might be acceptable or
even commonplace in another. Dataset curators must recognize these variations
and design annotation tasks sensitive to and mindful of regional and cultural
differences.

Moreover, dataset curators seeking to annotate intent must provide sufficient
context to annotators to permit assessment (Anuchitanukul, Ive, and Specia
2022). This can also minimize personal biases introduced by annotators.

Challenge: Trade-offs between capturing intent and achieving high annotation
agreement. A primary challenge in enhancing the annotation phase with
contextual details is the trade-off between accurately inferring an
individual’s intent reflected through content and maintaining high agreement
among annotators (Ross et al. 2017). Context can be subjective, and different
annotators might interpret the same information differently based on their
backgrounds, experiences, and biases. On the other hand, annotator
disagreements are sometimes due to lack of sufficient context (Zhang et al.
2023). Consequently, establishing standardized guidelines that incorporate
diverse perspectives is essential to mitigate this concern and improve
annotation consistency.

### Detection: Develop context-aware, policy-aware, explainable models

Models that depend solely on short text struggle to capture users’ intent,
which impacts their usability in real-world scenarios. Incorporation of
contextual features into detection algorithms can substantially improve the
accuracy of intent-based abuse detection (Markov and Daelemans 2022). Prior
work indicates that context-aware detection is challenging yet more applicable
to real-life scenarios (Menini, Aprosio, and Tonelli 2021).

Moreover, recent progress in NLP, such as the introduction of Retrieval-
Augmented large language models, incorporates retrieval components that access
relevant contextual information from knowledge bases—such as a user’s past
interactions—in real-time, offering a deeper context for content evaluation
(Shi et al. 2024). This feature is especially beneficial for platforms aiming
to implement context-aware moderation policies that adapt to dynamic social
norms.

The immediate context of the target text is not the only relevant context;
specific rules or policies of the platform used for moderation are critical as
well. Research has demonstrated that state-of-the-art large language models
(LLMs) are more effective at executing rule-based moderation (Kumar,
AbuHashem, and Durumeric 2023). Explainable AI (XAI) enhances the capability
of AI systems to accurately capture intent or context by providing
transparency into how decisions are made, which helps users and developers
understand the reasoning behind a model’s outputs (Kohli and Devi 2023; Islam,
Bataineh, and Zulkernine 2023). XAI can potentially be strategically used to
help pinpoint why a model might misunderstand particular intents or contexts
and ensure compliance with social media moderation policies that require clear
explanations of AI-driven decisions. We advocate for the implementation of
policy-aware and dynamic abuse detection, which can be facilitated by XAI
(Calabrese, Ross, and Lapata 2022) and retrieval-augmented models (Li et al.
2024). This approach allows the models to adapt to the particular policies and
enhance the relevance and fairness of moderation actions.

Challenge: Balancing performance and applicability. Incorporating contextual
features presents a significant challenge, particularly in terms of
performance, as standard detection models often prove to be overly optimistic
(Menini, Aprosio, and Tonelli 2021). Ensuring that the system achieves high
performance without compromising its applicability can be difficult, thus
advanced machine learning models and continuous algorithm training with
updated datasets are required to address this balance effectively.

### Moderation: Incorporate wisdom of the crowd and establish feedback loops

While AI and machine learning play an essential role in content moderation at
scale, they have clear limitations. Algorithms often struggle to understand
the nuance and subtlety of human communication, and meaning cannot be learned
from form alone (Bender and Koller 2020). On the other hand, humans, although
more adept at contextual and intent-based understanding, can exhibit
inconsistency due to inherent biases (Basile et al. 2022).

Moderation can be enhanced by leveraging the ’wisdom of the crowd’—utilizing
user reports and community feedback to identify and moderate abusive content.
Additionally, establishing feedback loops where moderators can provide
insights back to the detection systems helps refine and improve the accuracy
of automated systems. Throughout, platforms must design moderation systems to
be sensitive to regional and cultural differences.

Moreover, given the dynamic nature of societal norms and the evolving
definitions of inappropriate content, social media platforms must update
moderation guidelines and train their moderators frequently (Jiang et al.
2020).

Challenge: Managing scale and bias. The challenge in this phase lies in
managing the scale of data and potential biases that can arise from crowd-
sourced inputs. User reports can be influenced by personal biases or
coordinated attacks, leading to false positives or negatives. Implementing
robust filtering algorithms to verify and validate user-generated reports
before they influence the moderation process is crucial for maintaining the
integrity of the system.

### Technology design: Introduce friction to assess intent

For the most part, we have focused on the detection and evaluation of content
flowing through social media systems as if the moderation pipeline is designed
to operate independently, only triggering governance procedures upon detection
of content that violates a policy. One difficulty we have highlighted is
determining the intent of various actors based on their actions and the
content they generate and share. Of course, the actions of users depend to
some degree on the affordances of the social media platform itself.

If we relax the assumption of independence between social media platform
design and content moderation pipeline design, additional strategies for
governance emerge. Specifically, a range of friction-in-design (Frischmann and
Selinger 2018; Frischmann and Benesch 2023) measures might generate reliable
evidence of intent to better guide content moderation systems. For example,
social media platforms might introduce prompts that query users about their
intentions when users post or share (certain types of) content. Such prompts
might be triggered based on different criteria, such as sensitive content or
intended audience (friends versus strangers). The prompt could provide a
simple means for the user to express their intentions. The prompt might be
framed in terms of purpose. Not only would a response provide potentially
reliable evidence of intent that would be useful for the content moderation
system itself, but it also would provide the user with an opportunity to think
about their own intentions. Other friction-in-design measures could provide
users with knowledge about the potential consequences of their actions. Such
measures would generate another source of reliable evidence about intent.

The basic idea of this strategy is to focus on the dynamic interactions
between the social media platform and the content moderation pipeline from a
design perspective. The associated challenges here are substantively different
from the technical challenges we have outlined in prior sections. The success
of friction-in-design approaches center around platform business models and
regulatory frameworks that prioritize and reward healthy information
ecosystems.

## Conclusions

Our survey highlights the unappreciated role of intent in the annotation,
detection, and moderation of online abuse. Despite advancements in NLP, a
significant gap remains in effectively capturing users’ intent considering
rich context around user-generated content. By identifying these gaps, we hope
to steer the research community towards solutions better aligned with ethics,
law, and policy. As platforms strive to balance free expression with the
prevention of harm and abuse, understanding intent may become a key
determinant of appropriate action. Thus, we advocate for the meticulous design
of abuse detection and mitigation frameworks that incorporate: robust training
datasets annotated with context that reflect the complexities of intent;
state-of-the-art detection models that utilize contextual information as input
and provide explanations as output; and, moderation systems that effectively
integrate automated detection with wisdom of the crowd to reflect evolving
social norms. These technological innovations must be informed by and situated
within literature in cognitive science, ethics, law, and policy to offer
meaningful solutions to complex challenges in dynamic social media ecosystems.

## Limitations and Future Work

This work offers comprehensive discussion of the role of intent in automated
detection algorithms for online abuse. Consequently, our review methodology
included papers from the SCOPUS database and citation searches that met strict
inclusion criteria. To maintain focus, we limited our review to papers that
discuss English-language texts. Although non-English and multi-lingual studies
might offer valuable insights into context capture and intent inference, their
consideration is beyond our current scope and could be addressed in future
research.

Moreover, our analysis is confined to specific forms of online abuse,
primarily hate speech and cyberbullying. These terms are frequently used
interchangeably or referenced together, and do not comprehensively represent
the broad spectrum of online abuse. Exploring additional types of online abuse
is an important direction for future studies.

## Ethical Considerations Statement

In undertaking research that involves analyzing potentially abusive content
and reviewing datasets and scholarly articles on hate speech and cyberbullying
detection, we are aware of the sensitive nature of the language involved. The
example abusive content used in the paper is solely for research purposes,
thus does not aim to target any individuals or groups. Instead, the primary
objective of this work is to enhance understanding of the role of intent in
algorithmic content moderation systems and better align technical approaches
with current discourse in existing ethics, law, and policy.

## Positionality Statement

Our research team is comprised of experts in information science, ethics, and
law. The authors’ different academic and cultural backgrounds have lent
diverse perspectives to the discussions in this paper around online abuse.

## Adverse Impact Statement

The discussions in the paper highlight the complexities of identifying digital
abuse and the shortcomings on current detection algorithms. This information
may inadvertently provide malicious actors with insights to circumvent abuse
detection. We have provided very few specific examples beyond what has been
published in the referenced works, so we believe this risk is low.

## References

  * Abro et al. (2020) Abro, S.; Shaikh, S.; Khand, Z. H.; Zafar, A.; Khan, S.; and Mujtaba, G. 2020.  Automatic hate speech detection using machine learning: A comparative study.  _International Journal of Advanced Computer Science and Applications_ , 11(8).
  * Aggarwal, Maurya, and Chaudhary (2020) Aggarwal, A.; Maurya, K.; and Chaudhary, A. 2020.  Comparative study for predicting the severity of cyberbullying across multiple social media platforms.  In _2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS)_ , 871–877. IEEE.
  * Agnes, Solomon, and Tamilmaran (2023) Agnes, S. A.; Solomon, A. A.; and Tamilmaran, D. J. C. 2023.  Abusive Comment Detection in Social Media with Bidirectional LSTM Model.  In _2023 5th International Conference on Smart Systems and Inventive Technology (ICSSIT)_ , 1368–1373. IEEE.
  * Agrafiotis et al. (2016) Agrafiotis, I.; Bada, M.; Cornish, P.; Creese, S.; Goldsmith, M.; Ignatuschtschenko, E.; Roberts, T.; and Upton, D. M. 2016.  Cyber harm: concepts, taxonomy and measurement.  _Saïd Business School WP_ , 23.
  * Ahmed et al. (2022) Ahmed, T.; Ivan, S.; Kabir, M.; Mahmud, H.; and Hasan, K. 2022.  Performance analysis of transformer-based architectures and their ensembles to detect trait-based cyberbullying.  _Social Network Analysis and Mining_ , 12(1): 99.
  * Ahmed et al. (2021) Ahmed, T.; Kabir, M.; Ivan, S.; Mahmud, H.; and Hasan, K. 2021.  Am i being bullied on social media? an ensemble approach to categorize cyberbullying.  In _2021 IEEE international conference on big data (Big data)_ , 2442–2453. IEEE.
  * Aind, Ramnaney, and Sethia (2020) Aind, A. T.; Ramnaney, A.; and Sethia, D. 2020.  Q-bully: a reinforcement learning based cyberbullying detection framework.  In _2020 International conference for emerging technology (INCET)_ , 1–6. IEEE.
  * Akinyemi et al. (2023) Akinyemi, J. D.; Ibitoye, A. O.; Oyewale, C. T.; and Onifade, O. F. 2023.  Cyberbullying Detection and Classification in Social Media Texts Using Machine Learning Techniques.  In _International Conference on Computer Science, Engineering and Education Applications_ , 440–449. Springer.
  * Al-Garadi, Varathan, and Ravana (2016) Al-Garadi, M. A.; Varathan, K. D.; and Ravana, S. D. 2016.  Cybercrime detection in online communications: The experimental case of cyberbullying detection in the Twitter network.  _Computers in Human Behavior_ , 63: 433–443.
  * Al Mazari (2013) Al Mazari, A. 2013.  Cyber-bullying taxonomies: Definition, forms, consequences and mitigation strategies.  In _2013 5th International Conference on Computer Science and Information Technology_ , 126–133. IEEE.
  * Albanyan and Blanco (2022) Albanyan, A.; and Blanco, E. 2022.  Pinpointing fine-grained relationships between hateful tweets and replies.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 36, 10418–10426.
  * Albanyan, Hassan, and Blanco (2023) Albanyan, A.; Hassan, A.; and Blanco, E. 2023.  Not All Counterhate Tweets Elicit the Same Replies: A Fine-Grained Analysis.  In _Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (* SEM 2023)_ , 71–88.
  * Ali, Mohd, and Fauzi (2018) Ali, W. N. H. W.; Mohd, M.; and Fauzi, F. 2018.  Cyberbullying detection: an overview.  In _2018 Cyber Resilience Conference (CRC)_ , 1–3. IEEE.
  * ALjohani, Yafooz, and Alsaeedi (2023) ALjohani, E. J.; Yafooz, W. M.; and Alsaeedi, A. 2023.  Cyberbullying Detection Approaches: A Review.  In _2023 5th International Conference on Inventive Research in Computing Applications (ICIRCA)_ , 1310–1316. IEEE.
  * Alkomah and Ma (2022) Alkomah, F.; and Ma, X. 2022.  A literature review of textual hate speech detection methods and datasets.  _Information_ , 13(6): 273.
  * Alksasbeh et al. (2021) Alksasbeh, M. Z.; Alqaralleh, B. A.; Abukhalil, T.; Abukaraki, A.; Al Rawashdeh, T.; and Al-Jaafreh, M. 2021.  Smart detection of offensive words in social media using the soundex algorithm and permuterm index.  _International Journal of Electrical and Computer Engineering (IJECE)_ , 11(5): 4431–4438.
  * Alonso and Romero (2017) Alonso, C.; and Romero, E. 2017.  Aggressors and victims in bullying and cyberbullying: A study of personality profiles using the five-factor model.  _The Spanish journal of psychology_ , 20: E76.
  * Alonso, Saini, and Kovács (2019) Alonso, P.; Saini, R.; and Kovács, G. 2019.  TheNorth at HASOC 2019: Hate Speech Detection in Social Media Data.  In _FIRE (Working Notes)_ , 293–299.
  * Alonso, Saini, and Kovács (2020) Alonso, P.; Saini, R.; and Kovács, G. 2020.  Hate speech detection using transformer ensembles on the hasoc dataset.  In _International conference on speech and computer_ , 13–21. Springer.
  * Alotaibi, Alotaibi, and Razaque (2021) Alotaibi, M.; Alotaibi, B.; and Razaque, A. 2021.  A multichannel deep learning framework for cyberbullying detection on social media.  _Electronics_ , 10(21): 2664.
  * Alrashidi et al. (2022) Alrashidi, B.; Jamal, A.; Khan, I.; and Alkhathlan, A. 2022.  A review on abusive content automatic detection: approaches, challenges and opportunities.  _PeerJ Computer Science_ , 8: e1142.
  * Alrehili (2019) Alrehili, A. 2019.  Automatic hate speech detection on social media: A brief survey.  In _2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA)_ , 1–6. IEEE.
  * Ambareen and Meenakshi Sundaram (2023) Ambareen, K.; and Meenakshi Sundaram, S. 2023.  A Survey of Cyberbullying Detection and Performance: Its Impact in Social Media Using Artificial Intelligence.  _SN Computer Science_ , 4(6): 859.
  * Ana (1999) Ana, O. S. 1999.  Like an animal I was treated’: Anti-immigrant metaphor in US public discourse.  _Discourse & society_, 10(2): 191–224.
  * Anjum and Katarya (2022) Anjum; and Katarya, R. 2022.  Analysis of online toxicity detection using machine learning approaches.  In _International Conference on Artificial Intelligence and Sustainable Engineering: Select Proceedings of AISE 2020, Volume 1_ , 381–392. Springer.
  * Anjum and Katarya (2024) Anjum; and Katarya, R. 2024.  Hate speech, toxicity detection in online social media: a recent survey of state of the art and opportunities.  _International Journal of Information Security_ , 23(1): 577–608.
  * Antypas and Camacho-Collados (2023) Antypas, D.; and Camacho-Collados, J. 2023.  Robust hate speech detection in social media: A cross-dataset empirical evaluation.  _arXiv preprint arXiv:2307.01680_.
  * Anuchitanukul, Ive, and Specia (2022) Anuchitanukul, A.; Ive, J.; and Specia, L. 2022.  Revisiting contextual toxicity detection in conversations.  _ACM Journal of Data and Information Quality_ , 15(1): 1–22.
  * Anzovino, Fersini, and Rosso (2018) Anzovino, M.; Fersini, E.; and Rosso, P. 2018.  Automatic identification and classification of misogynistic language on twitter.  In _Natural Language Processing and Information Systems: 23rd International Conference on Applications of Natural Language to Information Systems, NLDB 2018, Paris, France, June 13-15, 2018, Proceedings 23_ , 57–64. Springer.
  * Arora et al. (2023) Arora, A.; Nakov, P.; Hardalov, M.; Sarwar, S. M.; Nayak, V.; Dinkov, Y.; Zlatkova, D.; Dent, K.; Bhatawdekar, A.; Bouchard, G.; et al. 2023.  Detecting harmful content on online platforms: what platforms need vs. where research efforts go.  _ACM Computing Surveys_ , 56(3): 1–17.
  * Arunkarthick et al. (2023) Arunkarthick, A.; Naseeba, B.; Challa, N. P.; and Ajay, K. 2023.  Using NLP Techniques for Cyberbullying Tweet Recognition.  In _2023 3rd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA)_ , 1231–1236. IEEE.
  * Ashraf, Zubiaga, and Gelbukh (2021) Ashraf, N.; Zubiaga, A.; and Gelbukh, A. 2021.  Abusive language detection in youtube comments leveraging replies as conversational context.  _PeerJ Computer Science_ , 7: e742.
  * Awal et al. (2021) Awal, M. R.; Cao, R.; Lee, R. K.-W.; and Mitrović, S. 2021.  Angrybert: Joint learning target and emotion for hate speech detection.  In _Pacific-Asia conference on knowledge discovery and data mining_ , 701–713. Springer.
  * Awan and Zempi (2015) Awan, I.; and Zempi, I. 2015.  We fear for our lives: Offline and online experiences of anti-Muslim hostility.
  * Babaeianjelodar et al. (2022) Babaeianjelodar, M.; Poorna Prudhvi, G.; Lorenz, S.; Chen, K.; Mondal, S.; Dey, S.; and Kumar, N. 2022.  Interpretable and high-performance hate and offensive speech detection.  In _International Conference on Human-Computer Interaction_ , 233–244. Springer.
  * Balakrishnan, Khan, and Arabnia (2020) Balakrishnan, V.; Khan, S.; and Arabnia, H. R. 2020.  Improving cyberbullying detection using Twitter users’ psychological features and machine learning.  _Computers & Security_, 90: 101710.
  * Banko, MacKeen, and Ray (2020) Banko, M.; MacKeen, B.; and Ray, L. 2020.  A unified taxonomy of harmful content.  In _Proceedings of the fourth workshop on online abuse and harms_ , 125–137.
  * Barbieri et al. (2020) Barbieri, F.; Camacho-Collados, J.; Neves, L.; and Espinosa-Anke, L. 2020.  TweetEval: Unified benchmark and comparative evaluation for tweet classification.  _arXiv preprint arXiv:2010.12421_.
  * Basile et al. (2019) Basile, V.; Bosco, C.; Fersini, E.; Nozza, D.; Patti, V.; Pardo, F. M. R.; Rosso, P.; and Sanguinetti, M. 2019.  Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter.  In _Proceedings of the 13th international workshop on semantic evaluation_ , 54–63.
  * Basile et al. (2022) Basile, V.; Caselli, T.; Balahur, A.; and Ku, L.-W. 2022.  Bias, subjectivity and perspectives in natural language processing.  _Frontiers in Artificial Intelligence_ , 5: 926435.
  * Baydogan and Alatas (2021) Baydogan, C.; and Alatas, B. 2021.  Metaheuristic ant lion and moth flame optimization-based novel approach for automatic detection of hate speech in online social networks.  _IEEE Access_ , 9: 110047–110062.
  * Baydogan et al. (2022) Baydogan, C.; et al. 2022.  Deep-Cov19-Hate: A textual-based novel approach for automatic detection of hate speech in online social networks throughout COVID-19 with shallow and deep learning models.  _Tehnički vjesnik_ , 29(1): 149–156.
  * Beddiar, Jahan, and Oussalah (2021) Beddiar, D.; Jahan, M.; and Oussalah, M. 2021.  Data expansion using back translation and paraphrasing for hate speech detection. Online Soc Networks Media 24.
  * Bender and Koller (2020) Bender, E. M.; and Koller, A. 2020.  Climbing towards NLU: On meaning, form, and understanding in the age of data.  In _Proceedings of the 58th annual meeting of the association for computational linguistics_ , 5185–5198.
  * Bhagya and Deepthi (2021) Bhagya, J.; and Deepthi, P. 2021.  Cyberbullying detection on social media using SVM.  In _Inventive Systems and Control: Proceedings of ICISC 2021_ , 17–27. Springer.
  * Bhatt et al. (2023) Bhatt, C.; Saini, N.; Chauhan, R.; and Sahoo, A. K. 2023.  Machine Learning Techniques for Hate Speech Detection on Social Media.  In _2023 3rd International Conference on Innovative Sustainable Computational Technologies (CISCT)_ , 1–5. IEEE.
  * Bilen (2023) Bilen, A. 2023.  A Review: Detection of Discrimination and Hate Speech Shared on Social Media Platforms Using Artificial Intelligence Methods.  _Algorithmic Discrimination and Ethical Perspective of Artificial Intelligence_ , 171–181.
  * Blei, Ng, and Jordan (2003) Blei, D. M.; Ng, A. Y.; and Jordan, M. I. 2003.  Latent dirichlet allocation.  _Journal of machine Learning research_ , 3(Jan): 993–1022.
  * Bode (2016) Bode, L. 2016.  Political news in the news feed: Learning politics from social media.  _Mass communication and society_ , 19(1): 24–48.
  * Bokolo and Liu (2023) Bokolo, B. G.; and Liu, Q. 2023.  Cyberbullying detection on social media using machine learning.  In _IEEE INFOCOM 2023-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)_ , 1–6. IEEE.
  * Bozyiğit, Utku, and Nasibov (2021) Bozyiğit, A.; Utku, S.; and Nasibov, E. 2021.  Cyberbullying detection: Utilizing social media features.  _Expert Systems with Applications_ , 179: 115001.
  * Brown (2017a) Brown, A. 2017a.  What is hate speech? Part 1: The myth of hate.  _Law and Philosophy_ , 36: 419–468.
  * Brown (2017b) Brown, A. 2017b.  What is hate speech? Part 2: Family resemblances.  _Law and Philosophy_ , 36: 561–613.
  * Buan and Ramachandra (2020) Buan, T. A.; and Ramachandra, R. 2020.  Automated cyberbullying detection in social media using an svm activated stacked convolution lstm network.  In _Proceedings of the 2020 4th International Conference on Compute and Data Analysis_ , 170–174.
  * Bunde (2021) Bunde, E. 2021.  AI-assisted and explainable hate speech detection for social media moderators–A design science approach.
  * Calabrese, Ross, and Lapata (2022) Calabrese, A.; Ross, B.; and Lapata, M. 2022.  Explainable abuse detection as intent classification and slot filling.  _Transactions of the Association for Computational Linguistics_ , 10: 1440–1454.
  * Campbell and Bauman (2018) Campbell, M.; and Bauman, S. 2018.  Cyberbullying: Definition, consequences, prevalence.  In _Reducing cyberbullying in schools_ , 3–16. Elsevier.
  * Caselli et al. (2020) Caselli, T.; Basile, V.; Mitrović, J.; Kartoziya, I.; and Granitzer, M. 2020.  I feel offended, don’t be abusive! implicit/explicit messages in offensive and abusive language.  In _Proceedings of the twelfth language resources and evaluation conference_ , 6193–6202.
  * Cécillon et al. (2021) Cécillon, N.; Labatut, V.; Dufour, R.; and Linares, G. 2021.  Graph embeddings for abusive language detection.  _SN Computer Science_ , 2: 1–15.
  * Chandrasekaran et al. (2022) Chandrasekaran, S.; Singh Pundir, A. K.; Lingaiah, T. B.; et al. 2022.  Deep learning approaches for cyberbullying detection and classification on social media.  _Computational Intelligence and Neuroscience_ , 2022.
  * Chandrasekharan et al. (2018) Chandrasekharan, E.; Samory, M.; Jhaver, S.; Charvat, H.; Bruckman, A.; Lampe, C.; Eisenstein, J.; and Gilbert, E. 2018.  The Internet’s hidden rules: An empirical study of Reddit norm violations at micro, meso, and macro scales.  _Proceedings of the ACM on Human-Computer Interaction_ , 2(CSCW): 1–25.
  * Chatzakou et al. (2017) Chatzakou, D.; Kourtellis, N.; Blackburn, J.; De Cristofaro, E.; Stringhini, G.; and Vakali, A. 2017.  Mean birds: Detecting aggression and bullying on twitter.  In _Proceedings of the 2017 ACM on web science conference_ , 13–22.
  * Chelmis and Zois (2021) Chelmis, C.; and Zois, D.-S. 2021.  Dynamic, incremental, and continuous detection of cyberbullying in online social media.  _ACM Transactions on the Web (TWEB)_ , 15(3): 1–33.
  * Chen, Mckeever, and Delany (2017) Chen, H.; Mckeever, S.; and Delany, S. J. 2017.  Harnessing the power of text mining for the detection of abusive content in social media.  In _Advances in Computational Intelligence Systems: Contributions Presented at the 16th UK Workshop on Computational Intelligence, September 7–9, 2016, Lancaster, UK_ , 187–205. Springer.
  * Chen, McKeever, and Delany (2018) Chen, H.; McKeever, S.; and Delany, S. J. 2018.  A comparison of classical versus deep learning techniques for abusive content detection on social media sites.  In _Social Informatics: 10th International Conference, SocInfo 2018, St. Petersburg, Russia, September 25-28, 2018, Proceedings, Part I 10_ , 117–133. Springer.
  * Chen and Li (2020) Chen, H.-Y.; and Li, C.-T. 2020.  HENIN: Learning heterogeneous neural interaction networks for explainable cyberbullying detection on social media.  _arXiv preprint arXiv:2010.04576_.
  * Cheng et al. (2019a) Cheng, L.; Li, J.; Silva, Y.; Hall, D.; and Liu, H. 2019a.  PI-bully: Personalized cyberbullying detection with peer influence.  In _The 28th International Joint Conference on Artificial Intelligence (IJCAI)_.
  * Cheng et al. (2019b) Cheng, L.; Li, J.; Silva, Y. N.; Hall, D. L.; and Liu, H. 2019b.  Xbully: Cyberbullying detection within a multi-modal context.  In _Proceedings of the twelfth acm international conference on web search and data mining_ , 339–347.
  * Ciampaglia (2018) Ciampaglia, G. L. 2018.  Fighting fake news: a role for computational social science in the fight against digital misinformation.  _Journal of Computational Social Science_ , 1(1): 147–153.
  * Crandall, Eshleman, and O’brien (2002) Crandall, C. S.; Eshleman, A.; and O’brien, L. 2002.  Social norms and the expression and suppression of prejudice: the struggle for internalization.  _Journal of personality and social psychology_ , 82(3): 359.
  * Crump (2009) Crump, D. 2009.  What Does Intent Mean.  _Hofstra L. Rev._ , 38: 1059.
  * Dadvar, Trieschnigg, and De Jong (2014) Dadvar, M.; Trieschnigg, D.; and De Jong, F. 2014.  Experts and machines against bullies: A hybrid approach to detect cyberbullies.  In _Advances in Artificial Intelligence: 27th Canadian Conference on Artificial Intelligence, Canadian AI 2014, Montréal, QC, Canada, May 6-9, 2014. Proceedings 27_ , 275–281. Springer.
  * Dadvar et al. (2013) Dadvar, M.; Trieschnigg, D.; Ordelman, R.; and De Jong, F. 2013.  Improving cyberbullying detection with user context.  In _Advances in Information Retrieval: 35th European Conference on IR Research, ECIR 2013, Moscow, Russia, March 24-27, 2013. Proceedings 35_ , 693–696. Springer.
  * Daniel et al. (2023) Daniel, R.; Murthy, T. S.; Kumari, C. D.; Lydia, E. L.; Ishak, M. K.; Hadjouni, M.; and Mostafa, S. M. 2023.  Ensemble Learning with Tournament Selected Glowworm Swarm Optimization Algorithm for Cyberbullying Detection on Social Media.  _IEEE Access_.
  * Das, Banerjee, and Mukherjee (2022) Das, M.; Banerjee, S.; and Mukherjee, A. 2022.  Data Bootstrapping Approaches to Improve Low Resource Abusive Language Detection for Indic Languages.  _arXiv preprint arXiv:2204.12543_.
  * Davidson et al. (2017) Davidson, T.; Warmsley, D.; Macy, M.; and Weber, I. 2017.  Automated hate speech detection and the problem of offensive language.  In _Proceedings of the international AAAI conference on web and social media_ , volume 11, 512–515.
  * De Souza and Da Costa-Abreu (2020) De Souza, G. A.; and Da Costa-Abreu, M. 2020.  Automatic offensive language detection from Twitter data using machine learning and feature selection of metadata.  In _2020 international joint conference on neural networks (IJCNN)_ , 1–6. IEEE.
  * Dhingra et al. (2023) Dhingra, N.; Chawla, S.; Saini, O.; and Kaushal, R. 2023.  An Improved Detection of Cyberbullying on Social Media Using Randomized Sampling.  _International Journal of Bullying Prevention_ , 1–13.
  * Elsafoury et al. (2021) Elsafoury, F.; Katsigiannis, S.; Pervez, Z.; and Ramzan, N. 2021.  When the timeline meets the pipeline: A survey on automated cyberbullying detection.  _IEEE access_ , 9: 103541–103563.
  * ElSherief et al. (2018) ElSherief, M.; Nilizadeh, S.; Nguyen, D.; Vigna, G.; and Belding, E. 2018.  Peer to peer hate: Hate speech instigators and their targets.  In _Proceedings of the International AAAI Conference on Web and Social Media_ , volume 12.
  * Escalante et al. (2017) Escalante, H. J.; Villatoro-Tello, E.; Garza, S. E.; López-Monroy, A. P.; Montes-y Gómez, M.; and Villaseñor-Pineda, L. 2017.  Early detection of deception and aggressiveness using profile-based representations.  _Expert Systems with Applications_ , 89: 99–111.
  * Fale, Goyal, and Shivani (2023) Fale, P. N.; Goyal, K. K.; and Shivani, S. 2023.  A hybrid deep learning approach for abusive text detection.  In _AIP Conference Proceedings_ , volume 2753. AIP Publishing.
  * Fersini et al. (2018) Fersini, E.; Rosso, P.; Anzovino, M.; et al. 2018.  Overview of the task on automatic misogyny identification at IberEval 2018.  _Ibereval@ sepln_ , 2150: 214–228.
  * Fortuna and Nunes (2018) Fortuna, P.; and Nunes, S. 2018.  A survey on automatic detection of hate speech in text.  _ACM Computing Surveys (CSUR)_ , 51(4): 1–30.
  * Founta et al. (2018) Founta, A.; Djouvas, C.; Chatzakou, D.; Leontiadis, I.; Blackburn, J.; Stringhini, G.; Vakali, A.; Sirivianos, M.; and Kourtellis, N. 2018.  Large scale crowdsourcing and characterization of twitter abusive behavior.  In _Proceedings of the international AAAI conference on web and social media_ , volume 12.
  * French, Storey, and Wallace (2023) French, A.; Storey, V. C.; and Wallace, L. 2023.  A typology of disinformation intentionality and impact.  _Information Systems Journal_.
  * Frischmann and Benesch (2023) Frischmann, B.; and Benesch, S. 2023.  Friction-In-Design Regulation as 21st Century Time, Place, and Manner Restriction.  _Yale JL & Tech._, 25: 376.
  * Frischmann and Selinger (2018) Frischmann, B.; and Selinger, E. 2018.  _Re-engineering humanity_.  Cambridge University Press.
  * Gandhi et al. (2024) Gandhi, A.; Ahir, P.; Adhvaryu, K.; Shah, P.; Lohiya, R.; Cambria, E.; Poria, S.; and Hussain, A. 2024.  Hate speech detection: A comprehensive review of recent works.  _Expert Systems_ , e13562.
  * Gangurde et al. (2022) Gangurde, A.; Mankar, P.; Chaudhari, D.; and Pawar, A. 2022.  A systematic bibliometric analysis of hate speech detection on social media sites.  _Journal of Scientometric Research_ , 11(1): 100–111.
  * Gao and Huang (2017) Gao, L.; and Huang, R. 2017.  Detecting online hate speech using context aware models.  _arXiv preprint arXiv:1710.07395_.
  * Gao, Kuppersmith, and Huang (2017) Gao, L.; Kuppersmith, A.; and Huang, R. 2017.  Recognizing explicit and implicit hate speech using a weakly supervised two-path bootstrapping approach.  _arXiv preprint arXiv:1710.07394_.
  * Gashroo and Mehrotra (2022) Gashroo, O. B.; and Mehrotra, M. 2022.  Analysis and classification of abusive textual content detection in online social media.  In _Intelligent Communication Technologies and Virtual Mobile Networks: Proceedings of ICICV 2022_ , 173–190. Springer.
  * Geetha et al. (2021) Geetha, R.; Karthika, S.; Sowmika, C. J.; and Janani, B. M. 2021.  Auto-Off ID: Automatic Detection of Offensive Language in Social Media.  In _Journal of Physics: Conference Series_ , volume 1911, 012012. IOP Publishing.
  * Giglietto and Lee (2017) Giglietto, F.; and Lee, Y. 2017.  A hashtag worth a thousand words: Discursive strategies around# JeNeSuisPasCharlie after the 2015 Charlie Hebdo shooting.  _Social Media+ Society_ , 3(1): 2056305116686992.
  * Golbeck et al. (2017) Golbeck, J.; Ashktorab, Z.; Banjo, R. O.; Berlinger, A.; Bhagwan, S.; Buntain, C.; Cheakalos, P.; Geller, A. A.; Gnanasekaran, R. K.; Gunasekaran, R. R.; et al. 2017.  A large labeled corpus for online harassment research.  In _Proceedings of the 2017 ACM on web science conference_ , 229–233.
  * Gomez et al. (2020) Gomez, R.; Gibert, J.; Gomez, L.; and Karatzas, D. 2020.  Exploring hate speech detection in multimodal publications.  In _Proceedings of the IEEE/CVF winter conference on applications of computer vision_ , 1470–1478.
  * Gongane, Munot, and Anuse (2022) Gongane, V. U.; Munot, M. V.; and Anuse, A. 2022.  Feature Representation Techniques for Hate Speech Detection on Social Media: A Comparative Study.  In _2022 International Conference on Signal and Information Processing (IConSIP)_ , 1–6. IEEE.
  * Gongane, Munot, and Anuse (2024) Gongane, V. U.; Munot, M. V.; and Anuse, A. D. 2024.  A survey of explainable AI techniques for detection of fake news and hate speech on social media platforms.  _Journal of Computational Social Science_ , 1–37.
  * Gopalan et al. (2023) Gopalan, A.; Mohanavel, V.; Geo, A. A.; Rajkumar, G. V.; Kavitha, T.; and Pooja, P. 2023.  Experimental Evaluation of Robust Cyberbullying Detection over social media using Intelligent Learning Scheme.  In _2023 International Conference on Research Methodologies in Knowledge Management, Artificial Intelligence and Telecommunication Engineering (RMKMATE)_ , 1–7. IEEE.
  * Grimminger and Klinger (2021) Grimminger, L.; and Klinger, R. 2021.  Hate towards the political opponent: A Twitter corpus study of the 2020 US elections on the basis of offensive speech and stance detection.  _arXiv preprint arXiv:2103.01664_.
  * Gudumotu et al. (2023) Gudumotu, C. E.; Nukala, S. R.; Reddy, K.; Konduri, A.; and Gireesh, C. 2023.  A Survey on Deep Learning Models to Detect Hate Speech and Bullying in Social Media.  In _Artificial Intelligence for Societal Issues_ , 27–44. Springer.
  * Haider et al. (2023) Haider, F.; Dipty, I.; Rahman, F.; Assaduzzaman, M.; and Sohel, A. 2023.  Social Media Hate Speech Detection Using Machine Learning Approach.  In _International Conference on Computational Intelligence in Data Science_ , 218–229. Springer.
  * Hamdy, Mitrović, and Granitzer (2020) Hamdy, E.; Mitrović, J.; and Granitzer, M. 2020.  nlpUP at SemEval-2020 Task 12: A blazing fast system for offensive language detection.  In _Proceedings of the Fourteenth Workshop on Semantic Evaluation_ , 2098–2104.
  * Hameleers, Brosius, and de Vreese (2022) Hameleers, M.; Brosius, A.; and de Vreese, C. H. 2022.  Whom to trust? Media exposure patterns of citizens with perceptions of misinformation and disinformation related to the news media.  _European journal of communication_ , 37(3): 237–268.
  * Hani et al. (2019) Hani, J.; Nashaat, M.; Ahmed, M.; Emad, Z.; Amer, E.; and Mohammed, A. 2019.  Social media cyberbullying detection using machine learning.  _International Journal of Advanced Computer Science and Applications_ , 10(5): 703–707.
  * Hanu and Unitary team (2020) Hanu, L.; and Unitary team. 2020.  Detoxify.  Github. https://github.com/unitaryai/detoxify.
  * Harish et al. (2023) Harish, D.; Alamelu, M.; Manimaran, M.; et al. 2023.  Automatic Detection of Cyberbullying on Social Media Using Machine Learning.  In _2023 2nd International Conference on Advancements in Electrical, Electronics, Communication, Computing and Automation (ICAECA)_ , 1–6. IEEE.
  * Hashemi (2021) Hashemi, M. 2021.  A data-driven framework for coding the intent and extent of political tweeting, disinformation, and extremism.  _Information_ , 12(4): 148.
  * He et al. (2021) He, B.; Ziems, C.; Soni, S.; Ramakrishnan, N.; Yang, D.; and Kumar, S. 2021.  Racism is a virus: Anti-Asian hate and counterspeech in social media during the COVID-19 crisis.  In _Proceedings of the 2021 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining_ , 90–94.
  * Herath et al. (2020) Herath, M.; Atapattu, T.; Dung, H. A.; Treude, C.; and Falkner, K. 2020.  AdelaideCyC at SemEval-2020 task 12: Ensemble of classifiers for offensive language detection in social media.  In _Proceedings of the Fourteenth Workshop on Semantic Evaluation_ , 1516–1523.
  * Huang, Singh, and Atrey (2014) Huang, Q.; Singh, V. K.; and Atrey, P. K. 2014.  Cyber bullying detection using social and textual analysis.  In _Proceedings of the 3rd International Workshop on Socially-aware Multimedia_ , 3–6.
  * Huang et al. (2022) Huang, Y.; Song, R.; Giunchiglia, F.; and Xu, H. 2022.  A multitask learning framework for abuse detection and emotion classification.  _Algorithms_ , 15(4): 116.
  * Hussein and Aleqabie (2023) Hussein, F. N. A.; and Aleqabie, H. J. 2023.  Cyberbullying Detection on Social Media: A Brief Survey.  In _2023 Second International Conference on Advanced Computer Applications (ACA)_ , 1–6. IEEE.
  * Ibrahim, Torki, and El-Makky (2020) Ibrahim, M.; Torki, M.; and El-Makky, N. M. 2020.  AlexU-BackTranslation-TL at SemEval-2020 task 12: Improving offensive language detection using data augmentation and transfer learning.  In _Proceedings of the fourteenth workshop on semantic evaluation_ , 1881–1890.
  * Im et al. (2022) Im, J.; Schoenebeck, S.; Iriarte, M.; Grill, G.; Wilkinson, D.; Batool, A.; Alharbi, R.; Funwie, A.; Gankhuu, T.; Gilbert, E.; et al. 2022.  Women’s Perspectives on Harm and Justice after Online Harassment.  _Proceedings of the ACM on Human-Computer Interaction_ , 6(CSCW2): 1–23.
  * Iqbal and Jami (2022) Iqbal, S.; and Jami, H. 2022.  Exploring definition of cyberbullying and its forms from the perspective of adolescents living in Pakistan.  _Psychological studies_ , 67(4): 514–523.
  * Islam, Bataineh, and Zulkernine (2023) Islam, M. R.; Bataineh, A. S.; and Zulkernine, M. 2023.  Detection of Cyberbullying in Social Media Texts Using Explainable Artificial Intelligence.  In _International Conference on Ubiquitous Security_ , 319–334. Springer.
  * Istaiteh, Al-Omoush, and Tedmori (2020) Istaiteh, O.; Al-Omoush, R.; and Tedmori, S. 2020.  Racist and sexist hate speech detection: Literature review.  In _2020 international conference on intelligent data science technologies and applications (IDSTA)_ , 95–99. IEEE.
  * Iwendi et al. (2023) Iwendi, C.; Srivastava, G.; Khan, S.; and Maddikunta, P. K. R. 2023.  Cyberbullying detection solutions based on deep learning architectures.  _Multimedia Systems_ , 29(3): 1839–1852.
  * Jahan et al. (2022) Jahan, M. S.; Beddiar, D. R.; Oussalah, M.; and Mohamed, M. 2022.  Data expansion using wordnet-based semantic expansion and word disambiguation for cyberbullying detection.  In _Proceedings of the Thirteenth Language Resources and Evaluation Conference_ , 1761–1770.
  * Jahan and Oussalah (2023) Jahan, M. S.; and Oussalah, M. 2023.  A systematic review of Hate Speech automatic detection using Natural Language Processing.  _Neurocomputing_ , 126232.
  * Jain et al. (2021) Jain, V.; Kumar, V.; Pal, V.; and Vishwakarma, D. K. 2021.  Detection of cyberbullying on social media using machine learning.  In _2021 5th international conference on computing methodologies and communication (ICCMC)_ , 1091–1096. IEEE.
  * Jha and Mamidi (2017) Jha, A.; and Mamidi, R. 2017.  When does a compliment become sexist? analysis and classification of ambivalent sexism using twitter data.  In _Proceedings of the second workshop on NLP and computational social science_ , 7–16.
  * Jiang et al. (2020) Jiang, J.; Middler, S.; Brubaker, J. R.; and Fiesler, C. 2020.  Characterizing community guidelines on social media platforms.  In _Companion Publication of the 2020 Conference on Computer Supported Cooperative Work and Social Computing_ , 287–291.
  * Karatsalos and Panagiotakis (2020) Karatsalos, C.; and Panagiotakis, Y. 2020.  Attention-based method for categorizing different types of online harassment language.  In _Machine Learning and Knowledge Discovery in Databases: International Workshops of ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part II_ , 321–330. Springer.
  * Kaur, Singh, and Kaushal (2021) Kaur, S.; Singh, S.; and Kaushal, S. 2021.  Abusive content detection in online user-generated data: a survey.  _Procedia Computer Science_ , 189: 274–281.
  * Kavatagi and Rachh (2021) Kavatagi, S.; and Rachh, R. 2021.  A context aware embedding for the detection of hate speech in social media networks.  In _2021 International Conference on Smart Generation Computing, Communication and Networking (SMART GENCON)_ , 1–4. IEEE.
  * Kavitha et al. (2023) Kavitha, S.; Anchitaalagammai, J.; Murali, S.; Deepalakshmi, R.; Himal, L.; and Suryakanth, M. 2023.  Smart Language Checker: A Machine Learning Solution for Offensive Language detection in Social Media.  In _2023 International Conference on Data Science, Agents & Artificial Intelligence (ICDSAAI)_, 1–6. IEEE.
  * Kazbekova et al. (2023) Kazbekova, G.; Ismagulova, Z.; Kemelbekova, Z.; Tileubay, S.; Baimurzayev, B.; and Bazarbayeva, A. 2023.  Offensive Language Detection on Online Social Networks using Hybrid Deep Learning Architecture.  _International Journal of Advanced Computer Science & Applications_, 14(11).
  * Keipi et al. (2016) Keipi, T.; Näsi, M.; Oksanen, A.; and Räsänen, P. 2016.  _Online hate and harmful content: Cross-national perspectives_.  Taylor & Francis.
  * Kennedy et al. (2022) Kennedy, B.; Atari, M.; Davani, A. M.; Yeh, L.; Omrani, A.; Kim, Y.; Coombs, K.; Havaldar, S.; Portillo-Wightman, G.; Gonzalez, E.; et al. 2022.  Introducing the Gab Hate Corpus: defining and applying hate-based rhetoric to social media posts at scale.  _Language Resources and Evaluation_ , 1–30.
  * Kennedy et al. (2020) Kennedy, C. J.; Bacon, G.; Sahn, A.; and von Vacano, C. 2020.  Constructing interval variables via faceted Rasch measurement and multitask deep learning: a hate speech application.  _arXiv preprint arXiv:2009.10277_.
  * Kogilavani et al. (2023) Kogilavani, S. V.; Malliga, S.; Jaiabinaya, K.; Malini, M.; and Kokila, M. M. 2023.  Characterization and mechanical properties of offensive language taxonomy and detection techniques.  _Materials Today: Proceedings_ , 81: 630–633.
  * Kohli and Devi (2023) Kohli, A.; and Devi, V. S. 2023.  Explainable Offensive Language Classifier.  In _International Conference on Neural Information Processing_ , 299–313. Springer.
  * Kovács, Alonso, and Saini (2021) Kovács, G.; Alonso, P.; and Saini, R. 2021.  Challenges of hate speech detection in social media: Data scarcity, and leveraging external resources.  _SN Computer Science_ , 2(2): 95.
  * Kovács et al. (2022) Kovács, G.; Alonso, P.; Saini, R.; and Liwicki, M. 2022.  Leveraging external resources for offensive content detection in social media.  _AI Communications_ , 35(2): 87–109.
  * Kumar and Kumar (2023) Kumar, A.; and Kumar, S. 2023.  Hate Speech Detection in Multi-social Media Using Deep Learning.  In _International Conference on Advanced Communication and Intelligent Systems_ , 59–70. Springer.
  * Kumar and Sachdeva (2022) Kumar, A.; and Sachdeva, N. 2022.  A Bi-GRU with attention and CapsNet hybrid model for cyberbullying detection on social media.  _World Wide Web_ , 25(4): 1537–1550.
  * Kumar, Tyagi, and Das (2021) Kumar, A.; Tyagi, V.; and Das, S. 2021.  Deep learning for hate speech detection in social media.  In _2021 IEEE 4th International Conference on Computing, Power and Communication Technologies (GUCON)_ , 1–4. IEEE.
  * Kumar, AbuHashem, and Durumeric (2023) Kumar, D.; AbuHashem, Y.; and Durumeric, Z. 2023.  Watch your language: large language models and content moderation.  _arXiv preprint arXiv:2309.14517_.
  * Kumar and Bhat (2022) Kumar, R.; and Bhat, A. 2022.  A study of machine learning-based models for detection, control, and mitigation of cyberbullying in online social media.  _International Journal of Information Security_ , 21(6): 1409–1431.
  * Kümpel, Karnowski, and Keyling (2015) Kümpel, A. S.; Karnowski, V.; and Keyling, T. 2015.  News sharing in social media: A review of current research on news sharing users, content, and networks.  _Social media+ society_ , 1(2): 2056305115610141.
  * Leo, Santoso, and Pratomo (2023) Leo, C. O.; Santoso, B. J.; and Pratomo, B. A. 2023.  Enhancing Hate Speech Detection for Social Media Moderation: A Comparative Analysis of Machine Learning Algorithms.  In _2023 International Conference on Advanced Mechatronics, Intelligent Manufacture and Industrial Automation (ICAMIMIA)_ , 960–964. IEEE.
  * Lepoutre et al. (2023) Lepoutre, M.; Vilar-Lluch, S.; Borg, E.; and Hansen, N. 2023.  What is hate speech? The case for a corpus approach.  _Criminal Law and Philosophy_ , 1–34.
  * Lewandowska-Tomaszczyk et al. (2023) Lewandowska-Tomaszczyk, B.; Bączkowska, A.; Liebeskind, C.; Valunaite Oleskeviciene, G.; and Žitnik, S. 2023.  An integrated explicit and implicit offensive language taxonomy.  _Lodz Papers in Pragmatics_ , 19(1): 7–48.
  * Li et al. (2024) Li, G.; Lu, W.; Zhang, W.; Lian, D.; Lu, K.; Mao, R.; Shu, K.; and Liao, H. 2024.  Re-Search for The Truth: Multi-round Retrieval-augmented Large Language Models are Strong Fake News Detectors.  _arXiv preprint arXiv:2403.09747_.
  * Li and Shimada (2022) Li, Z.; and Shimada, K. 2022.  Combining Pre-Trained Language Models and Features for Offensive Language Detection.  In _2022 13th International Congress on Advanced Applied Informatics Winter (IIAI-AAI-Winter)_ , 5–10. IEEE.
  * Liu, Li, and Zou (2019) Liu, P.; Li, W.; and Zou, L. 2019.  NULI at SemEval-2019 task 6: Transfer learning for offensive language detection using bidirectional transformers.  In _Proceedings of the 13th international workshop on semantic evaluation_ , 87–91.
  * Liu, Zavarsky, and Malik (2019) Liu, Y.; Zavarsky, P.; and Malik, Y. 2019.  Non-linguistic features for cyberbullying detection on a social media platform using machine learning.  In _Cyberspace Safety and Security: 11th International Symposium, CSS 2019, Guangzhou, China, December 1–3, 2019, Proceedings, Part I 11_ , 391–406. Springer.
  * López-Vizcaíno et al. (2023) López-Vizcaíno, M.; Nóvoa, F. J.; Artieres, T.; and Cacheda, F. 2023.  Site agnostic approach to early detection of cyberbullying on social media networks.  _Sensors_ , 23(10): 4788.
  * López-Vizcaíno et al. (2021) López-Vizcaíno, M. F.; Nóvoa, F. J.; Carneiro, V.; and Cacheda, F. 2021.  Early detection of cyberbullying on social media networks.  _Future Generation Computer Systems_ , 118: 219–229.
  * Lu et al. (2020) Lu, N.; Wu, G.; Zhang, Z.; Zheng, Y.; Ren, Y.; and Choo, K.-K. R. 2020.  Cyberbullying detection in social media text based on character-level convolutional neural network with shortcuts.  _Concurrency and Computation: Practice and Experience_ , 32(23): e5627.
  * MacAvaney et al. (2019) MacAvaney, S.; Yao, H.-R.; Yang, E.; Russell, K.; Goharian, N.; and Frieder, O. 2019.  Hate speech detection: Challenges and solutions.  _PloS one_ , 14(8): e0221152.
  * Malik, Aggrawal, and Vishwakarma (2021) Malik, P.; Aggrawal, A.; and Vishwakarma, D. K. 2021.  Toxic speech detection using traditional machine learning models and bert and fasttext embedding with deep neural networks.  In _2021 5th International Conference on Computing Methodologies and Communication (ICCMC)_ , 1254–1259. IEEE.
  * Mandl et al. (2019) Mandl, T.; Modha, S.; Majumder, P.; Patel, D.; Dave, M.; Mandlia, C.; and Patel, A. 2019.  Overview of the hasoc track at fire 2019: Hate speech and offensive content identification in indo-european languages.  In _Proceedings of the 11th annual meeting of the Forum for Information Retrieval Evaluation_ , 14–17.
  * Mansur, Omar, and Tiun (2023) Mansur, Z.; Omar, N.; and Tiun, S. 2023.  Twitter hate speech detection: a systematic review of methods, taxonomy analysis, challenges, and opportunities.  _IEEE Access_ , 11: 16226–16249.
  * Markov and Daelemans (2022) Markov, I.; and Daelemans, W. 2022.  The role of context in detecting the target of hate speech.  In _Proceedings of the Third Workshop on Threat, Aggression and Cyberbullying (TRAC 2022), October, Gyeongju, Republic of Korea_ , 37–42.
  * Marwick and Lewis (2017) Marwick, A. E.; and Lewis, R. 2017.  Media manipulation and disinformation online.
  * Mathew et al. (2020) Mathew, B.; Saha, P.; Yimam, S. M.; Biemann, C.; Goyal, P.; and Mukherjee, A. 2020.  HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection.  _arXiv preprint arXiv:2012.10289_.
  * Mathew et al. (2021) Mathew, B.; Saha, P.; Yimam, S. M.; Biemann, C.; Goyal, P.; and Mukherjee, A. 2021.  Hatexplain: A benchmark dataset for explainable hate speech detection.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 35, 14867–14875.
  * Mathur et al. (2023) Mathur, S. A.; Isarka, S.; Dharmasivam, B.; and Jaidhar, C. 2023.  Analysis of Tweets for Cyberbullying Detection.  In _2023 Third International Conference on Secure Cyber Computing and Communication (ICSCCC)_ , 269–274. IEEE.
  * Mehta and Passi (2022) Mehta, H.; and Passi, K. 2022.  Social media hate speech detection using explainable artificial intelligence (XAI).  _Algorithms_ , 15(8): 291.
  * Menini, Aprosio, and Tonelli (2021) Menini, S.; Aprosio, A. P.; and Tonelli, S. 2021.  Abuse is contextual, what about nlp? the role of context in abusive language annotation and detection.  _arXiv preprint arXiv:2103.14916_.
  * Mercan et al. (2021) Mercan, V.; Jamil, A.; Hameed, A. A.; Magsi, I. A.; Bazai, S.; and Shah, S. A. 2021.  Hate speech and offensive language detection from social media.  In _2021 International Conference on Computing, Electronic and Electrical Engineering (ICE Cube)_ , 1–5. IEEE.
  * Miran and Yahia (2023) Miran, A. Z.; and Yahia, H. S. 2023.  Hate Speech Detection in Social Media (Twitter) Using Neural Network.  _J. Mobile Multimedia_ , 19(3): 765–798.
  * Miró-Llinares and Rodríguez-Sala (2016) Miró-Llinares, F.; and Rodríguez-Sala, J. J. 2016.  Cyber hate speech on twitter: Analyzing disruptive events from social media to build a violent communication and hate speech taxonomy.  _International Journal of Design & Nature and Ecodynamics_, 11(3): 406–415.
  * Mishra, Yannakoudakis, and Shutova (2019) Mishra, P.; Yannakoudakis, H.; and Shutova, E. 2019.  Tackling online abuse: A survey of automated abuse detection methods.  _arXiv preprint arXiv:1908.06024_.
  * Mladenović, Ošmjanski, and Stanković (2021) Mladenović, M.; Ošmjanski, V.; and Stanković, S. V. 2021.  Cyber-aggression, cyberbullying, and cyber-grooming: A survey and research challenges.  _ACM Computing Surveys (CSUR)_ , 54(1): 1–42.
  * Modi (2018) Modi, S. 2018.  AHTDT-Automatic Hate Text Detection Techniques in Social Media.  In _2018 International Conference on Circuits and Systems in Digital Enterprise Technology (ICCSDET)_ , 1–3. IEEE.
  * Mohtaj and Möller (2022) Mohtaj, S.; and Möller, S. 2022.  On the importance of word embedding in automated harmful information detection.  In _International Conference on Text, Speech, and Dialogue_ , 251–262. Springer.
  * Molina et al. (2021) Molina, M. D.; Sundar, S. S.; Le, T.; and Lee, D. 2021.  “Fake news” is not simply false information: A concept explication and taxonomy of online content.  _American behavioral scientist_ , 65(2): 180–212.
  * Mozafari, Farahbakhsh, and Crespi (2020a) Mozafari, M.; Farahbakhsh, R.; and Crespi, N. 2020a.  A BERT-based transfer learning approach for hate speech detection in online social media.  In _Complex Networks and Their Applications VIII: Volume 1 Proceedings of the Eighth International Conference on Complex Networks and Their Applications COMPLEX NETWORKS 2019 8_ , 928–940. Springer.
  * Mozafari, Farahbakhsh, and Crespi (2020b) Mozafari, M.; Farahbakhsh, R.; and Crespi, N. 2020b.  Hate speech detection and racial bias mitigation in social media based on BERT model.  _PloS one_ , 15(8): e0237861.
  * Mullah and Zainon (2021) Mullah, N. S.; and Zainon, W. M. N. W. 2021.  Advances in machine learning algorithms for hate speech detection in social media: a review.  _IEEE Access_ , 9: 88364–88376.
  * Muneer et al. (2023) Muneer, A.; Alwadain, A.; Ragab, M. G.; and Alqushaibi, A. 2023.  Cyberbullying detection on social media using stacking ensemble learning and enhanced BERT.  _Information_ , 14(8): 467.
  * Murshed et al. (2022) Murshed, B. A. H.; Abawajy, J.; Mallappa, S.; Saif, M. A. N.; and Al-Ariki, H. D. E. 2022.  DEA-RNN: A hybrid deep learning approach for cyberbullying detection in Twitter social media platform.  _IEEE Access_ , 10: 25857–25871.
  * Murshed et al. (2023) Murshed, B. A. H.; Suresha; Abawajy, J.; Saif, M. A. N.; Abdulwahab, H. M.; and Ghanem, F. A. 2023.  FAEO-ECNN: cyberbullying detection in social media platforms using topic modelling and deep learning.  _Multimedia Tools and Applications_ , 82(30): 46611–46650.
  * Muzakir, Adi, and Kusumaningrum (2022) Muzakir, A.; Adi, K.; and Kusumaningrum, R. 2022.  Classification of Hate Speech Language Detection on Social Media: Preliminary Study for Improvement.  In _International Conference on Networking, Intelligent Systems and Security_ , 146–156. Springer.
  * Nagar et al. (2022) Nagar, S.; Gupta, S.; Bahushruth, C.; Barbhuiya, F. A.; and Dey, K. 2022.  Hate speech detection on social media using graph convolutional networks.  In _Complex Networks & Their Applications X: Volume 2, Proceedings of the Tenth International Conference on Complex Networks and Their Applications COMPLEX NETWORKS 2021 10_, 3–14. Springer.
  * Nascimento, Cavalcanti, and Da Costa-Abreu (2022) Nascimento, F. R.; Cavalcanti, G. D.; and Da Costa-Abreu, M. 2022.  Unintended bias evaluation: An analysis of hate speech detection and gender bias mitigation on social media using ensemble learning.  _Expert Systems with Applications_ , 201: 117032.
  * Nath et al. (2022) Nath, N.; George, J. P.; Kesan, A.; and Rodrigues, A. 2022.  An Efficient Deep Learning-Based Hybrid Architecture for Hate Speech Detection in Social Media.  In _Data Science and Security: Proceedings of IDSCS 2022_ , 347–355. Springer.
  * Nielsen (2002) Nielsen, L. B. 2002.  Subtle, pervasive, harmful: Racist and sexist remarks in public as hate speech.  _Journal of Social issues_ , 58(2): 265–280.
  * Nisha and Jebathangam (2022) Nisha, M.; and Jebathangam, J. 2022.  Detection and classification of cyberbullying in social media using text mining.  In _2022 6th International Conference on Electronics, Communication and Aerospace Technology_ , 856–861. IEEE.
  * Nitya Harshitha et al. (2024) Nitya Harshitha, T.; Prabu, M.; Suganya, E.; Sountharrajan, S.; Bavirisetti, D. P.; Gadde, N.; and Uppu, L. S. 2024.  ProTect: a hybrid deep learning model for proactive detection of cyberbullying on social media.  _Frontiers in artificial intelligence_ , 7: 1269366.
  * Nocentini et al. (2010) Nocentini, A.; Calmaestra, J.; Schultze-Krumbholz, A.; Scheithauer, H.; Ortega, R.; and Menesini, E. 2010.  Cyberbullying: Labels, behaviours and definition in three European countries.  _Australian Journal of Guidance and Counselling_ , 20(2): 129–142.
  * Olteanu et al. (2018) Olteanu, A.; Castillo, C.; Boy, J.; and Varshney, K. 2018.  The effect of extremist violence on hateful speech online.  In _Proceedings of the international AAAI conference on web and social media_ , volume 12.
  * Omran, Al Tararwah, and Al Qundus (2023) Omran, E.; Al Tararwah, E.; and Al Qundus, J. 2023.  A comparative analysis of machine learning algorithms for hate speech detection in social media.  _Online Journal of Communication and Media Technologies_ , 13(4): e202348.
  * Ousidhoum et al. (2019) Ousidhoum, N.; Lin, Z.; Zhang, H.; Song, Y.; and Yeung, D.-Y. 2019.  Multilingual and multi-aspect hate speech analysis.  _arXiv preprint arXiv:1908.11049_.
  * Pahuja, Neema, and Dubey (2023) Pahuja, V.; Neema, S.; and Dubey, R. 2023.  Securing Social Spaces: Cyberbullying Detection with ML and DL on Social Media Platforms.  In _2023 International Conference on Sustainable Communication Networks and Application (ICSCNA)_ , 1471–1476. IEEE.
  * Pariyani et al. (2021) Pariyani, B.; Shah, K.; Shah, M.; Vyas, T.; and Degadwala, S. 2021.  Hate speech detection in twitter using natural language processing.  In _2021 Third International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV)_ , 1146–1152. IEEE.
  * Pavlopoulos et al. (2019) Pavlopoulos, J.; Thain, N.; Dixon, L.; and Androutsopoulos, I. 2019.  Convai at semeval-2019 task 6: Offensive language identification and categorization with perspective and bert.  In _Proceedings of the 13th international Workshop on Semantic Evaluation_ , 571–576.
  * Paz, Montero-Díaz, and Moreno-Delgado (2020) Paz, M. A.; Montero-Díaz, J.; and Moreno-Delgado, A. 2020.  Hate speech: A systematized review.  _Sage Open_ , 10(4): 2158244020973022.
  * Perera and Fernando (2021) Perera, A.; and Fernando, P. 2021.  Accurate cyberbullying detection and prevention on social media.  _Procedia Computer Science_ , 181: 605–611.
  * Pradhan, Yatam, and Bera (2020) Pradhan, A.; Yatam, V. M.; and Bera, P. 2020.  Self-attention for cyberbullying detection.  In _2020 International Conference on Cyber Situational Awareness, Data Analytics and Assessment (CyberSA)_ , 1–6. IEEE.
  * Preetham and Anitha (2023) Preetham, J.; and Anitha, J. 2023.  Offensive Language Detection in Social Media Using Ensemble Techniques.  In _2023 International Conference on Circuit Power and Computing Technologies (ICCPCT)_ , 805–808. IEEE.
  * Qian et al. (2019) Qian, J.; Bethke, A.; Liu, Y.; Belding, E.; and Wang, W. Y. 2019.  A benchmark dataset for learning to intervene in online hate speech.  _arXiv preprint arXiv:1909.04251_.
  * Qian et al. (2023) Qian, W.; Yu, S.; Nie, Z.; Lu, X. S.; Liu, H.; and Huang, B. 2023.  Improved Hierarchical Attention Networks for Cyberbullying Detection via Social Media Data.  In _2023 IEEE International Conference on Networking, Sensing and Control (ICNSC)_ , volume 1, 1–6. IEEE.
  * Qiu et al. (2022) Qiu, J.; Hegde, N.; Moh, M.; and Moh, T.-S. 2022.  Investigating user information and social media features in cyberbullying detection.  In _2022 IEEE International Conference on Big Data (Big Data)_ , 3063–3070. IEEE.
  * Qureshi and Sabih (2021) Qureshi, K. A.; and Sabih, M. 2021.  Un-compromised credibility: Social media based multi-class hate speech classification for text.  _IEEE Access_ , 9: 109465–109477.
  * Ramiandrisoa (2022) Ramiandrisoa, F. 2022.  Multi-task Learning for Hate Speech and Aggression Detection.  In _CIRCLE_.
  * Rawat, Kumar, and Samant (2024) Rawat, A.; Kumar, S.; and Samant, S. S. 2024.  Hate speech detection in social media: Techniques, recent trends, and future challenges.  _Wiley Interdisciplinary Reviews: Computational Statistics_ , 16(2): e1648.
  * Reddy et al. (2023) Reddy, B. A. C.; Chandra, G. K.; Sisodia, D. S.; and Anuragi, A. 2023.  Balancing techniques for improving automated detection of hate speech and offensive language on social media.  In _2023 2nd International Conference for Innovation in Technology (INOCON)_ , 1–8. IEEE.
  * Rezvan et al. (2018) Rezvan, M.; Shekarpour, S.; Balasuriya, L.; Thirunarayan, K.; Shalin, V. L.; and Sheth, A. 2018.  A quality type-aware annotated corpus and lexicon for harassment research.  In _Proceedings of the 10th acm conference on web science_ , 33–36.
  * Ribeiro et al. (2018) Ribeiro, M.; Calais, P.; Santos, Y.; Almeida, V.; and Meira Jr, W. 2018.  Characterizing and detecting hateful users on twitter.  In _Proceedings of the International AAAI Conference on Web and Social Media_ , volume 12.
  * Ross et al. (2017) Ross, B.; Rist, M.; Carbonell, G.; Cabrera, B.; Kurowsky, N.; and Wojatzki, M. 2017.  Measuring the reliability of hate speech annotations: The case of the european refugee crisis.  _arXiv preprint arXiv:1701.08118_.
  * Sachdeva et al. (2021) Sachdeva, J.; Chaudhary, K. K.; Madaan, H.; and Meel, P. 2021.  Text based hate-speech analysis.  In _2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS)_ , 661–668. IEEE.
  * Saha et al. (2019) Saha, P.; Mathew, B.; Goyal, P.; and Mukherjee, A. 2019.  Hatemonitors: Language agnostic abuse detection in social media.  _arXiv preprint arXiv:1909.12642_.
  * Sahana, KM, and Darem (2020) Sahana, V.; KM, A. K.; and Darem, A. A. 2020.  A Comparative Analysis of Machine Learning Techniques for Cyberbullying Detection on FormSpring in Textual Modality.
  * Sajadi Ansari et al. (2021) Sajadi Ansari, F.; Barhamgi, M.; Khelifi, A.; and Benslimane, D. 2021.  An approach to detect cyberbullying on social media.  In _Model and Data Engineering: 10th International Conference, MEDI 2021, Tallinn, Estonia, June 21–23, 2021, Proceedings 10_ , 53–66. Springer.
  * Salawu, Lumsden, and He (2021) Salawu, S.; Lumsden, J.; and He, Y. 2021.  A large-scale English multi-label Twitter dataset for cyberbullying and online abuse detection.  In _The 5th Workshop on Online Abuse and Harms_ , 146–156. Association for Computational Linguistics.
  * Salehgohari et al. (2022) Salehgohari, A.; Mirhosseini, M.; Tabrizchi, H.; and Koczy, A. V. 2022.  Abusive Language Detection on Social Media using Bidirectional Long-Short Term Memory.  In _2022 IEEE 26th International Conference on Intelligent Engineering Systems (INES)_ , 000243–000248. IEEE.
  * Salminen et al. (2018) Salminen, J.; Almerekhi, H.; Milenković, M.; Jung, S.-g.; An, J.; Kwak, H.; and Jansen, B. 2018.  Anatomy of online hate: developing a taxonomy and machine learning models for identifying and classifying hate in online news media.  In _Proceedings of the International AAAI Conference on Web and Social Media_ , volume 12.
  * Samory et al. (2021) Samory, M.; Sen, I.; Kohne, J.; Flöck, F.; and Wagner, C. 2021.  “Call me sexist, but…”: Revisiting Sexism Detection Using Psychological Scales and Adversarial Samples.  In _Proceedings of the international AAAI conference on web and social media_ , volume 15, 573–584.
  * Sathishkumar et al. (2023) Sathishkumar, R.; Karthikeyan, T.; Shamsundar, S.; et al. 2023.  Ensemble Text Classification with TF-IDF Vectorization for Hate Speech Detection in Social Media.  In _2023 International Conference on System, Computation, Automation and Networking (ICSCAN)_ , 1–7. IEEE.
  * Scheuerman et al. (2021) Scheuerman, M. K.; Jiang, J. A.; Fiesler, C.; and Brubaker, J. R. 2021.  A framework of severity for harmful content online.  _Proceedings of the ACM on Human-Computer Interaction_ , 5(CSCW2): 1–33.
  * Shakeel and Dwivedi (2022) Shakeel, N.; and Dwivedi, R. K. 2022.  A survey on detection of cyberbullying in social media using machine learning techniques.  In _Intelligent Communication Technologies and Virtual Mobile Networks: Proceedings of ICICV 2022_ , 323–340. Springer.
  * Shankar et al. (2022) Shankar, K.; Abirami, A.; Indira, K.; Angeline, C. N.; and Shubhavya, K. 2022.  Cyberbullying Detection in Social Media Using Supervised ML and NLP Techniques.  In _Communication and Intelligent Systems: Proceedings of ICCIS 2021_ , 817–828. Springer.
  * Sharif, Hossain, and Hoque (2021) Sharif, O.; Hossain, E.; and Hoque, M. M. 2021.  Nlp-cuet@ dravidianlangtech-eacl2021: Offensive language detection from multilingual code-mixed text using transformers.  _arXiv preprint arXiv:2103.00455_.
  * Sharma and Bhalla (2022) Sharma, A.; and Bhalla, R. 2022.  Automatic and Advance Techniques for Hate Speech Detection on Social Media: A Review.  In _2022 Algorithms, Computing and Mathematics Conference (ACM)_ , 54–61. IEEE.
  * Sharma et al. (2022) Sharma, G.; Brar, G. S.; Singh, P.; Gupta, N.; Kalra, N.; and Parashar, A. 2022.  An Exploration of Machine Learning and Deep Learning Techniques for Offensive Text Detection in Social Media—A Systematic Review.  In _International Conference on Innovative Computing and Communications: Proceedings of ICICC 2022, Volume 3_ , 541–559. Springer.
  * Sharma and Tiwari (2023) Sharma, P.; and Tiwari, R. K. 2023.  Deep Learning Approach for Hate and Non Hate Speech Detection in Online Social Media.  In _2023 3rd International Conference on Technological Advancements in Computational Sciences (ICTACS)_ , 492–496. IEEE.
  * Shearer (2018) Shearer, E. 2018.  Social media outpaces print newspapers in the US as a news source.
  * Shi et al. (2024) Shi, K.; Sun, X.; Li, Q.; and Xu, G. 2024.  Compressing Long Context for Enhancing RAG with AMR-based Concept Distillation.  _arXiv preprint arXiv:2405.03085_.
  * Singh, Singh, and Chand (2022) Singh, N. K.; Singh, P.; and Chand, S. 2022.  Deep learning based methods for cyberbullying detection on social media.  In _2022 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS)_ , 521–525. IEEE.
  * Singh and Sharma (2024) Singh, N. M.; and Sharma, S. K. 2024.  An efficient automated multi-modal cyberbullying detection using decision fusion classifier on social media platforms.  _Multimedia Tools and Applications_ , 83(7): 20507–20535.
  * Singh et al. (2023) Singh, R. K.; Sanjay, H.; SA, P. J.; Rishi, H.; and Bhardwaj, S. 2023.  NLP Based Hate Speech Detection And Moderation.  In _2023 7th International Conference on Computation System and Information Technology for Sustainable Solutions (CSITSS)_ , 1–5. IEEE.
  * Singh, Ghosh, and Jose (2017) Singh, V. K.; Ghosh, S.; and Jose, C. 2017.  Toward multimodal cyberbullying detection.  In _Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems_ , 2090–2099.
  * Singh, Huang, and Atrey (2016) Singh, V. K.; Huang, Q.; and Atrey, P. K. 2016.  Cyberbullying detection using probabilistic socio-textual information fusion.  In _2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM)_ , 884–887. IEEE.
  * Sookarah and Ramwodin (2022) Sookarah, D.; and Ramwodin, L. S. 2022.  Combatting online harassment by using transformer language models for the detection of emotions, hate speech and offensive language on social media.  In _2022 4th International Conference on Emerging Trends in Electrical, Electronic and Communications Engineering (ELECOM)_ , 1–6. IEEE.
  * Suhas Bharadwaj, Kuzhalvaimozhi, and Vedavathi (2022) Suhas Bharadwaj, R.; Kuzhalvaimozhi, S.; and Vedavathi, N. 2022.  A novel multimodal hybrid classifier based cyberbullying detection for social media platform.  In _Proceedings of the Computational Methods in Systems and Software_ , 689–699. Springer.
  * Sultan et al. (2023) Sultan, D.; Mendes, M.; Kassenkhan, A.; and Akylbekov, O. 2023.  Hybrid CNN-LSTM Network for Cyberbullying Detection on Social Networks using Textual Contents.  _International Journal of Advanced Computer Science and Applications_ , 14(9).
  * Suryawanshi et al. (2020) Suryawanshi, S.; Chakravarthi, B. R.; Arcan, M.; and Buitelaar, P. 2020.  Multimodal meme dataset (MultiOFF) for identifying offensive content in image and text.  In _Proceedings of the second workshop on trolling, aggression and cyberbullying_ , 32–41.
  * Tai Phung, Khanh Ly Dang, and Tuan Pham (2020) Tai Phung, H.; Khanh Ly Dang, H.; and Tuan Pham, M. 2020.  Cyberbullying detection based on word curve representations using B-spline interpolation.  In _Proceedings of the 4th International Conference on Future Networks and Distributed Systems_ , 1–7.
  * Tanase, Cercel, and Chiru (2020) Tanase, M.-A.; Cercel, D.-C.; and Chiru, C.-G. 2020.  Upb at semeval-2020 task 12: Multilingual offensive language detection on social media by fine-tuning a variety of bert-based models.  _arXiv preprint arXiv:2010.13609_.
  * Thapa et al. (2022) Thapa, S.; Shah, A.; Jafri, F. A.; Naseem, U.; and Razzak, I. 2022.  A multi-modal dataset for hate speech detection on social media: Case-study of russia-ukraine conflict.  In _CASE 2022-5th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events from Text, Proceedings of the Workshop_. Association for Computational Linguistics.
  * Themeli, Giannakopoulos, and Pittaras (2019) Themeli, C.; Giannakopoulos, G.; and Pittaras, N. 2019.  A study of text representations for Hate Speech Detection.  In _International Conference on Computational Linguistics and Intelligent Text Processing_ , 424–437. Springer.
  * Thenmozhi et al. (2020) Thenmozhi, D.; Pr, N.; Arunima, S.; and Sengupta, A. 2020.  Ssn_nlp at SemEval 2020 Task 12: Offense Target Identification in Social Media Using Traditional and Deep Machine Learning Approaches.  In _Proceedings of the Fourteenth Workshop on Semantic Evaluation_ , 2155–2160.
  * Thenmozhi et al. (2019) Thenmozhi, D.; Sharavanan, S.; Chandrabose, A.; et al. 2019.  SSN_NLP at SemEval-2019 task 6: Offensive language identification in social media using traditional and deep machine learning approaches.  In _Proceedings of the 13th International Workshop on Semantic Evaluation_ , 739–744.
  * Toraman, Şahinuç, and Yilmaz (2022) Toraman, C.; Şahinuç, F.; and Yilmaz, E. H. 2022.  Large-scale hate speech detection with cross-domain transfer.  _arXiv preprint arXiv:2203.01111_.
  * Van Bruwaene, Huang, and Inkpen (2020) Van Bruwaene, D.; Huang, Q.; and Inkpen, D. 2020.  A multi-platform dataset for detecting cyberbullying in social media.  _Language Resources and Evaluation_ , 54(4): 851–874.
  * Van Hee et al. (2018) Van Hee, C.; Jacobs, G.; Emmery, C.; Desmet, B.; Lefever, E.; Verhoeven, B.; De Pauw, G.; Daelemans, W.; and Hoste, V. 2018.  Automatic detection of cyberbullying in social media text.  _PloS one_ , 13(10): e0203794.
  * Vidgen et al. (2020) Vidgen, B.; Botelho, A.; Broniatowski, D.; Guest, E.; Hall, M.; Margetts, H.; Tromble, R.; Waseem, Z.; and Hale, S. 2020.  Detecting East Asian prejudice on social media.  _arXiv preprint arXiv:2005.03909_.
  * Vidgen and Derczynski (2020) Vidgen, B.; and Derczynski, L. 2020.  Directions in abusive language training data, a systematic review: Garbage in, garbage out.  _Plos one_ , 15(12): e0243300.
  * Vidgen et al. (2019) Vidgen, B.; Harris, A.; Nguyen, D.; Tromble, R.; Hale, S.; and Margetts, H. 2019.  Challenges and frontiers in abusive content detection.  In _Proceedings of the third workshop on abusive language online_. Association for Computational Linguistics.
  * Vidgen et al. (2021a) Vidgen, B.; Nguyen, D.; Margetts, H.; Rossini, P.; and Tromble, R. 2021a.  Introducing CAD: the contextual abuse dataset.
  * Vidgen et al. (2021b) Vidgen, B.; Thrush, T.; Waseem, Z.; and Kiela, D. 2021b.  Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection.  In _ACL_.
  * Vora et al. (2023) Vora, D.; Mukherjee, A.; Repaka, S.; Das, S.; and Ingle, S. 2023.  Multimodal Cyberbullying Detection on Social Media: Review and Challenges.  In _2023 International Conference on Integration of Computational Intelligent System (ICICIS)_ , 1–8. IEEE.
  * Wang et al. (2020a) Wang, K.; Lu, D.; Han, S. C.; Long, S.; and Poon, J. 2020a.  Detect all abuse! toward universal abusive language detection models.  _arXiv preprint arXiv:2010.03776_.
  * Wang et al. (2020b) Wang, K.; Xiong, Q.; Wu, C.; Gao, M.; and Yu, Y. 2020b.  Multi-modal cyberbullying detection on social networks.  In _2020 International Joint Conference on Neural Networks (IJCNN)_ , 1–8. IEEE.
  * Wang et al. (2023) Wang, X.; Li, J.; Srivatsavaya, E.; and Rajtmajer, S. 2023.  Evidence of inter-state coordination amongst state-backed information operations.  _Scientific reports_ , 13(1): 7716.
  * Wang, Wu, and Rajtmajer (2023) Wang, X.; Wu, M.; and Rajtmajer, S. 2023.  From Yellow Peril to Model Minority: Asian stereotypes in social media during the COVID-19 pandemic.  In _Proceedings of the 15th ACM Web Science Conference 2023_ , 283–291.
  * Waseem (2016) Waseem, Z. 2016.  Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter.  In _Proceedings of the first workshop on NLP and computational social science_ , 138–142.
  * Waseem et al. (2017) Waseem, Z.; Davidson, T.; Warmsley, D.; and Weber, I. 2017.  Understanding abuse: A typology of abusive language detection subtasks.  _arXiv preprint arXiv:1705.09899_.
  * Waseem and Hovy (2016) Waseem, Z.; and Hovy, D. 2016.  Hateful symbols or hateful people? predictive features for hate speech detection on twitter.  In _Proceedings of the NAACL student research workshop_ , 88–93.
  * Westerman, Spence, and Van Der Heide (2014) Westerman, D.; Spence, P. R.; and Van Der Heide, B. 2014.  Social media as information source: Recency of updates and credibility of information.  _Journal of computer-mediated communication_ , 19(2): 171–183.
  * Wiegand, Ruppenhofer, and Kleinbauer (2019) Wiegand, M.; Ruppenhofer, J.; and Kleinbauer, T. 2019.  Detection of abusive language: the problem of biased datasets.  In _Proceedings of the 2019 conference of the North American Chapter of the Association for Computational Linguistics: human language technologies, volume 1 (long and short papers)_ , 602–608.
  * Wijesiriwardene et al. (2020) Wijesiriwardene, T.; Inan, H.; Kursuncu, U.; Gaur, M.; Shalin, V. L.; Thirunarayan, K.; Sheth, A.; and Arpinar, I. B. 2020.  Alone: A dataset for toxic behavior among adolescents on twitter.  In _Social Informatics: 12th International Conference, SocInfo 2020, Pisa, Italy, October 6–9, 2020, Proceedings 12_ , 427–439. Springer.
  * Wright (2019) Wright, M. F. 2019.  Cyberbullying: Definition, Description, Characteristics, and Consequences.  In _Handbook of Research on Children’s Consumption of Digital Media_ , 223–234. IGI Global.
  * Wright (2021) Wright, M. F. 2021.  Cyberbullying: Definition, behaviors, correlates, and adjustment problems.  In _Encyclopedia of Information Science and Technology, Fifth Edition_ , 356–373. IGI Global.
  * Xiang et al. (2021) Xiang, T.; MacAvaney, S.; Yang, E.; and Goharian, N. 2021.  ToxCCIn: Toxic content classification with interpretability.  _arXiv preprint arXiv:2103.01328_.
  * Xingyi and Adnan (2024) Xingyi, G.; and Adnan, H. 2024.  Potential cyberbullying detection in social media platforms based on a multi-task learning framework.  _International Journal of Data and Network Science_ , 8(1): 25–34.
  * Yao (2019) Yao, M. 2019.  Robust detection of cyberbullying in social media.  In _Companion Proceedings of The 2019 World Wide Web Conference_ , 61–66.
  * Yao, Chelmis, and Zois (2019) Yao, M.; Chelmis, C.; and Zois, D.-S. 2019.  Cyberbullying ends here: Towards robust detection of cyberbullying in social media.  In _The World Wide Web Conference_ , 3427–3433.
  * Yi and Zubiaga (2023a) Yi, P.; and Zubiaga, A. 2023a.  Learning like human annotators: Cyberbullying detection in lengthy social media sessions.  In _Proceedings of the ACM Web Conference 2023_ , 4095–4103.
  * Yi and Zubiaga (2023b) Yi, P.; and Zubiaga, A. 2023b.  Session-based cyberbullying detection in social media: A survey.  _Online Social Networks and Media_ , 36: 100250.
  * Yin and Zubiaga (2021) Yin, W.; and Zubiaga, A. 2021.  Towards generalisable hate speech detection: a review on obstacles and solutions.  _PeerJ Computer Science_ , 7: e598.
  * Yuan et al. (2023) Yuan, L.; Wang, T.; Ferraro, G.; Suominen, H.; and Rizoiu, M.-A. 2023.  Transfer learning for hate speech detection in social media.  _Journal of Computational Social Science_ , 6(2): 1081–1101.
  * Zampieri et al. (2019) Zampieri, M.; Malmasi, S.; Nakov, P.; Rosenthal, S.; Farra, N.; and Kumar, R. 2019.  Predicting the type and target of offensive posts in social media.  _arXiv preprint arXiv:1902.09666_.
  * Zampieri, Illina, and Fohr (2021) Zampieri, N.; Illina, I.; and Fohr, D. 2021.  Multiword expression features for automatic hate speech detection.  In _International Conference on Applications of Natural Language to Information Systems_ , 156–164. Springer.
  * Zhang et al. (2023) Zhang, W.; Guo, H.; Kivlichan, I. D.; Prabhakaran, V.; Yadav, D.; and Yadav, A. 2023.  A Taxonomy of Rater Disagreements: Surveying Challenges & Opportunities from the Perspective of Annotating Online Toxicity.  _arXiv preprint arXiv:2311.04345_.
  * Ziems, Vigfusson, and Morstatter (2020) Ziems, C.; Vigfusson, Y.; and Morstatter, F. 2020.  Aggressive, repetitive, intentional, visible, and imbalanced: Refining representations for cyberbullying classification.  In _Proceedings of the International AAAI Conference on Web and Social Media_ , volume 14, 808–819.
  * Zimdars and McLeod (2020) Zimdars, M.; and McLeod, K. 2020.  _Fake news: understanding media and misinformation in the digital age_.  MIT Press.

## Appendix

### Detection Paper Categorization

User Metadata. (López-Vizcaíno et al. 2021; Liu, Zavarsky, and Malik 2019;
Nagar et al. 2022; Nisha and Jebathangam 2022; Qiu et al. 2022; Sajadi Ansari
et al. 2021; Qian et al. 2023; Dhingra et al. 2023; Ziems, Vigfusson, and
Morstatter 2020; Dadvar et al. 2013; Bozyiğit, Utku, and Nasibov 2021; Dadvar,
Trieschnigg, and De Jong 2014; Al-Garadi, Varathan, and Ravana 2016; Escalante
et al. 2017; Cheng et al. 2019b; Balakrishnan, Khan, and Arabnia 2020)

Post Metadata.  (López-Vizcaíno et al. 2021; Liu, Zavarsky, and Malik 2019;
Nisha and Jebathangam 2022; Geetha et al. 2021; Babaeianjelodar et al. 2022;
Suhas Bharadwaj, Kuzhalvaimozhi, and Vedavathi 2022; Dhingra et al. 2023;
Ziems, Vigfusson, and Morstatter 2020; Bozyiğit, Utku, and Nasibov 2021;
Balakrishnan, Khan, and Arabnia 2020)

Image and video data (Nisha and Jebathangam 2022; Qiu et al. 2022; Singh and
Sharma 2024; López-Vizcaíno et al. 2021; Thapa et al. 2022; Singh, Ghosh, and
Jose 2017; Wang et al. 2020b; Cheng et al. 2019b)

Psychological and cognitive features

(Balakrishnan, Khan, and Arabnia 2020; Al-Garadi, Varathan, and Ravana 2016;
Cheng et al. 2019a)

Features from conversations. (Ashraf, Zubiaga, and Gelbukh 2021; Chen and Li
2020; Nisha and Jebathangam 2022; Qian et al. 2023; Dhingra et al. 2023;
Ziems, Vigfusson, and Morstatter 2020)

Graph structure. (Dhingra et al. 2023; Ziems, Vigfusson, and Morstatter 2020;
Liu, Zavarsky, and Malik 2019; Nagar et al. 2022; Qiu et al. 2022; Cécillon et
al. 2021; Huang, Singh, and Atrey 2014; Cheng et al. 2019a; Singh, Huang, and
Atrey 2016; Cheng et al. 2019b)

Policy or rule-aware models.

(Kumar, AbuHashem, and Durumeric 2023; Calabrese, Ross, and Lapata 2022)

Sentiment features. (Perera and Fernando 2021; López-Vizcaíno et al. 2021;
Liu, Zavarsky, and Malik 2019; Nisha and Jebathangam 2022; Geetha et al. 2021;
Babaeianjelodar et al. 2022; Ziems, Vigfusson, and Morstatter 2020; Dhingra et
al. 2023)

Topics and themes. (Perera and Fernando 2021; López-Vizcaíno et al. 2021;
Murshed et al. 2023; Van Hee et al. 2018)

Linguistic cues (Perera and Fernando 2021; López-Vizcaíno et al. 2021; Liu,
Zavarsky, and Malik 2019; Zampieri, Illina, and Fohr 2021; Geetha et al. 2021;
Babaeianjelodar et al. 2022; Sajadi Ansari et al. 2021; Li and Shimada 2022;
Dhingra et al. 2023; Ziems, Vigfusson, and Morstatter 2020; Van Hee et al.
2018; Cheng et al. 2019a; Dadvar et al. 2013; Cheng et al. 2019b)

Model Evaluation, improvement and application (Mozafari, Farahbakhsh, and
Crespi 2020a; Liu, Li, and Zou 2019; Mozafari, Farahbakhsh, and Crespi 2020b;
Hani et al. 2019; Kovács, Alonso, and Saini 2021; Abro et al. 2020; Yao,
Chelmis, and Zois 2019; Beddiar, Jahan, and Oussalah 2021; Alotaibi, Alotaibi,
and Razaque 2021; Murshed et al. 2022; Lu et al. 2020; Iwendi et al. 2023;
Qureshi and Sabih 2021; Pavlopoulos et al. 2019; Kumar and Sachdeva 2022;
Chandrasekaran et al. 2022; Chen, Mckeever, and Delany 2017; Baydogan and
Alatas 2021; Awal et al. 2021; Alonso, Saini, and Kovács 2020; Sharif,
Hossain, and Hoque 2021; Chen, McKeever, and Delany 2018; Malik, Aggrawal, and
Vishwakarma 2021; Pariyani et al. 2021; De Souza and Da Costa-Abreu 2020; Jain
et al. 2021; Pradhan, Yatam, and Bera 2020; Sachdeva et al. 2021; Aind,
Ramnaney, and Sethia 2020; Nascimento, Cavalcanti, and Da Costa-Abreu 2022;
Ibrahim, Torki, and El-Makky 2020; Baydogan et al. 2022; Ahmed et al. 2022;
Tanase, Cercel, and Chiru 2020; Aggarwal, Maurya, and Chaudhary 2020; Ahmed et
al. 2021; Xiang et al. 2021; Yao 2019; Saha et al. 2019; Jahan et al. 2022;
Bhagya and Deepthi 2021; Mercan et al. 2021; Yi and Zubiaga 2023a; Thenmozhi
et al. 2019; Singh, Singh, and Chand 2022; Huang et al. 2022; Kavatagi and
Rachh 2021; Buan and Ramachandra 2020; Alksasbeh et al. 2021; Kumar, Tyagi,
and Das 2021; Herath et al. 2020; Sookarah and Ramwodin 2022; Karatsalos and
Panagiotakis 2020; Haider et al. 2023; Mohtaj and Möller 2022; Nath et al.
2022; Mathur et al. 2023; Anjum and Katarya 2022; Chelmis and Zois 2021; Reddy
et al. 2023; Antypas and Camacho-Collados 2023; Fale, Goyal, and Shivani 2023;
Hamdy, Mitrović, and Granitzer 2020; Kovács et al. 2022; Shankar et al. 2022;
López-Vizcaíno et al. 2023; Alonso, Saini, and Kovács 2019; Kumar and Kumar
2023; Muzakir, Adi, and Kusumaningrum 2022; Muneer et al. 2023; Kazbekova et
al. 2023; Nitya Harshitha et al. 2024; Pahuja, Neema, and Dubey 2023; Yuan et
al. 2023; Preetham and Anitha 2023; Xingyi and Adnan 2024; Kavitha et al.
2023; Sharma and Tiwari 2023; Leo, Santoso, and Pratomo 2023; Omran, Al
Tararwah, and Al Qundus 2023; Daniel et al. 2023; Sahana, KM, and Darem 2020;
Sathishkumar et al. 2023; Gopalan et al. 2023; Thenmozhi et al. 2020; Akinyemi
et al. 2023; Bokolo and Liu 2023; Agnes, Solomon, and Tamilmaran 2023;
Ramiandrisoa 2022; Tai Phung, Khanh Ly Dang, and Tuan Pham 2020; Salehgohari
et al. 2022; Themeli, Giannakopoulos, and Pittaras 2019; Harish et al. 2023;
Mehta and Passi 2022; Bunde 2021; Kohli and Devi 2023; Islam, Bataineh, and
Zulkernine 2023; Singh et al. 2023)

Surveys

(Yin and Zubiaga 2021; Jahan and Oussalah 2023; Elsafoury et al. 2021; Mullah
and Zainon 2021; Ali, Mohd, and Fauzi 2018; Alrehili 2019; Sultan et al. 2023;
Kaur, Singh, and Kaushal 2021; Istaiteh, Al-Omoush, and Tedmori 2020; Mansur,
Omar, and Tiun 2023; Kumar and Bhat 2022; Yi and Zubiaga 2023b; Gangurde et
al. 2022; Alrashidi et al. 2022; Modi 2018; Gashroo and Mehrotra 2022; Shakeel
and Dwivedi 2022; Sharma et al. 2022; Anjum and Katarya 2024; Rawat, Kumar,
and Samant 2024; Bhatt et al. 2023; Vora et al. 2023; Hussein and Aleqabie
2023; Bilen 2023; Gandhi et al. 2024; Gongane, Munot, and Anuse 2024; Sharma
and Bhalla 2022; ALjohani, Yafooz, and Alsaeedi 2023; Miran and Yahia 2023;