# Adversarial Attacks and Defenses for Social Network Text Processing
Applications: Techniques, Challenges and Future Research Directions

Izzat Alsmadi<EMAIL_ADDRESS>Kashif Ahmad  Mahmoud Nazzal  Firoj Alam
Ala Al-Fuqaha  Abdallah Khreishah  Abdulelah Algosaibi  Department of
Computing and Cyber Security, Texas A&M University, San Antonio, Texas, USA
Information and Computing Technologies (ICT) Division, College of Science and
Engineering (CSE), Hamad Bin Khalifa University, Doha, Qatar.  Department of
Electrical and Computer Engineering, Newark College of Engineering, New Jersey
Institute of Technology, Newark, NJ 07102, USA  Qatar Computing Research
Institute, HBKU, Doha, Qatar  Department of Computer Science , College of
Computer Science and Information Technology, King Faisal University, Saudi
Arabia

###### Abstract

The growing use of social media has led to the development of several Machine
Learning (ML) and Natural Language Processing (NLP) tools to process the
unprecedented amount of social media content to make actionable decisions.
However, these ML and NLP algorithms have been widely shown to be vulnerable
to adversarial attacks. These vulnerabilities allow adversaries to launch a
diversified set of adversarial attacks on these algorithms in different
applications of social media text processing. In this paper, we provide a
comprehensive review of the main approaches for adversarial attacks and
defenses in the context of social media applications with a particular focus
on key challenges and future research directions. In detail, we cover
literature on six key applications, namely (i) rumors detection, (ii) satires
detection, (iii) clickbaits & spams identification, (iv) hate speech
detection, (v) misinformation detection, and (vi) sentiment analysis. We then
highlight the concurrent and anticipated future research questions and provide
recommendations and directions for future work.

###### keywords:

Adversarial Attacks, NLP, Social Media , Rumour Detection , Sentiment Analysis
, Misinformation , Hate Speech.

††journal: Journal of LaTeX Templates

##  1 Introduction

Social media has become a major communication channel in everyday life.
According to the Datareportal report published on 27th January 2021
111https://datareportal.com/reports/digital-2021-global-overview-report more
than half of the world population (4.48 billion) use social media. Among the
social media platforms, the most popular ones include Facebook, Youtube,
WhatsApp, Instagram, WeChat, and Twitter. Such platforms are widely used for
information dissemination and consumption [1]. Nowadays, social media
platforms form an important gateway for connecting people, spreading thoughts,
and linking business entities to customers through question advertisement,
review collection, and feedback [2]. The key advantage of such platforms is
the wide scope of the audience that allows individuals to directly connect and
share content. For example, expressing an opinion, sentiment, and emotion
towards products, entities, individuals, and/or society [3, 4, 5, 6].

The reach to a large audience and freedom of generating and sharing their
content also poses several challenges. For instance, these platforms have also
been used for political or financial gains (e.g., Brexit) [7], which has been
typically done by targeting focused communities. Malicious actors have also
been using them to spread rumors and mis/disinformation [8, 9]. These
platforms has also been used a ground for hate speech [10, 11], racism [12,
13], xenophobia [14], and prejudice [15].

Two evolving major factors influence the growing attacks on machine learning
algorithms and applications: (1) The rapidly growing dependence on automated
decisions, and (2) The level and nature of influence such attacks can cause or
trigger. In Natural Language Processing (NLP) in general and Online Social
Networks (OSNs) in particular, perhaps the Russian Troll Tweets (RTT) dataset
and the influence in the USA 2016 election is one major example. RTT showed a
recent trend in cyber attacks where humans, rather than their computing
devices are targets of such attacks. Social bots and trolls are examples of
text generators in Generative Adversarial Networks (GAN) models. The
discriminators are first the social network websites and their efforts to
eliminate social bots/trolls and counter the spread of mis/disinformation. The
other category of discriminators is other users of social networks who are the
targets of those bots/trolls and their media campaigns. Autocratic countries
see social network websites such as Twitter and Facebook as threats or
weapons, [16]. This explains why many of those countries have state-sponsored
agencies that are active in creating social media content for propaganda and
anti-propaganda. They create social bots or trolls to target rival media. They
also have social bots or trolls to target their own citizens. The widespread
of social bots can be observed in recent social and political movements such
as the Arab Spring, 2013-2014 protests in Ukraine, etc.

To address such issues a major research field has emerged, namely
computational social science [17, 18], a major part of it is to analyze social
media content. The focus is automatically analysis of social media content
using Machine Learning (ML) and Natural Language Processing (NLP) techniques.
A major interest came from NLP research communities, which led to organizing
workshops [19, 20, 21, 22], evaluation campaigns [23, 24, 25, 26, 27, 28, 29].
These efforts have made significant progress in the field in terms of
automatically identifying such content, debunking them, and facilitating
organizations, social media platforms, and society as a whole.

While such progresses are in place, the discovery of adversarial examples [30,
31] raised a major concern in different research communities and has been
studied for many different problems including image classification [32, 33],
security [34], malware detection [35, 36], robot-vision system [37], medical
diagnosis system [38, 39], mis/disinformation [8, 9], fake-news [40], and
rumor [41]. An adversarial attack refers to the process of crafting an
adversarial example, which is intended to fool a model through adversarial
perturbation. Adversarial attacks can be launched with different intentions
including getting influence on a classifier’s decision as well as violating
security. One recent example of such attacks is the exploitation of the
vulnerabilities of Microsoft’s Tay chatbot, which was shut down due to the
racist tweets [42].

In the context of social media content analysis, the attackers (i.e., the
individuals who want to misuse social media and spread inappropriate
information) can perturb their content to ditch the AI-based content filters
incorporated with social media platforms. For instance, an adversarial example
may fool a hate speech detection system, which can result in misclassification
of toxic and abusive content as acceptable and legitimate content.

To address the problem of adversarial attacks on ML models, there have been
efforts, reported in the literature, to overcome such challenges. Such efforts
include adversarial training with noise [43], gradient masking [44], defense
distillation [45], ensemble adversarial learning [46], and feature squeezing
[47].

Ref.-Year | Domain | Main Focus
---|---|---
[48] 2019 |  NLP |  Adversarial attack methods in NLP. However, it does not address defense approaches.
[49] 2019 |  DNNs |  Adversarial attacks and defense techniques for DNN-based solutions for English and Chinese text. The adversarial attacks are categorized on the perturbation units. The key text classifications covered in the survey include NLP inference, reading comprehension systems, questions and answer systems, and machine translation.
[50] 2020 |  DNNs |  Adversarial textual examples generation methods for DNNs. The key text analysis tasks discussed include machine translation, comprehension, text summarization, and text entailment. Also discusses multi-modal attacks on key applications of text to image, and image to text, such as scene-text recognition, image captioning, and visual question answering.
[51] 2020 |  Generic |  Adversarial attacks and defense techniques for images, graph, and text analysis.
This Survey 2021 |  NLP in Social Media Applications |  Adversarial attacks and defense systems for text-based social media applications, such as rumors, satires, clickbaits & spam, hate speech, misinformation detection and sentiment.

Table 1: A comparative summary and of existing surveys.

###  1.1 Scope of the Survey

The adversarial ML has been well explored for images [32, 33]. There are also
significant efforts on adversarial ML on text [50] and speech [52], [53],
[54]. In this paper, we mainly target adversarial attacks and the
corresponding defense methods for text based social media
applications.222Throughout the paper, we use social networks and social media
interchangeably. We particularly focus on key applications of social networks,
such as (i) rumors, (ii) satires, (iii) click baits and spam, (vi) hate
speech, (v) misinformation and (v) sentiment. Since text is a primary
communication means in social media platforms [55], the paper also provides
detailed taxonomies of text adversarial attacks and corresponding defense
techniques, which makes it self-contained in terms of relevant concepts. The
paper also advises on key challenges, limitations, and future research
directions for text-based social media applications.

###  1.2 Related Surveys

The literature on adversarial text analysis is quite rich [49, 50, 51]. Want
et al. [49] provide an overview of the literature on adversarial attacks and
corresponding defense strategies for DNN-based English and Chinese text
analysis systems. Zhang et al. [50], provide a more detailed survey of
adversarial attacks on deep learning-based models for NLP with a particular
focus on adversarial textual examples generation methods. Xu et al. [51] on
the other hand review the literature on adversarial attacks and defenses on
images, graphs, and text analysis models. In contrast to existing surveys,
this paper focuses on adversarial attacks and defense solutions for text-based
social network applications. In Table 1, we provide an overview of existing
surveys on adversarial learning, attack and defense methods.

###  1.3 Contributions

The key contributions of the survey can be summarized as follows.

  * •

We first provide a detailed taxonomy of adversarial attacks and the
corresponding defense techniques for text analysis frameworks.

  * •

We then provide a comprehensive survey of adversarial learning, attacks, and
defense systems for major social media applications including rumors, satires,
clickbaits & spam, hate speech, misinformation detection, and sentiment.

  * •

We also highlight key research challenges, open issues, and future research
directions.

The rest of the paper is organized as follows. Section 2 provides a detailed
taxonomy of adversarial attacks on the text analysis model. Attack techniques
at the character, word, sentence, and inter levels are revised in Section 3.
Section 4 covers the corresponding defense techniques. Section 5 provides an
overview of adversarial attacks and corresponding defense solutions for
different social media applications. Section 6 describes the key challenges
and future research directions in adversarial text analysis. Finally, Section
7 provides some concluding remarks.

##  2 Adversarial Attacks

Figure 1: Categorization of adversarial attack and defense techniques. Dotted
boxes and arrows present that loosely defined categories.

Current literature and several surveys categorized adversarial techniques into
attack and defense [50, 49]. These techniques can be further classified based
on how attack and defense are applied to ML models. Such a classification have
evolved based on the type of data (e.g. image, text), the knowledge level of
the attacker (e.g. white vs. black box), the goal of the attack (targeted vs.
non-targeted), and the granularity of the input content (e.g., for NLP,
character, word, sentence). Other classifications include the modality of the
input content, the type of application or task, and the architecture of the ML
model[50]. In Figure 1, we list several categorizations of the main
adversarial attack and defense techniques. In the figure, we use dotted lines
and boxes to represent the categories that are loosely defined in the overall
adversarial attack and defense techniques. For example, both model and target-
based attacks apply to any model and application. Here is a brief description
of each of these categorization criteria.

Attack: The purpose of an adversarial attack is to manipulate the inputs and
or the ML model so that it is fooled to produce faulty outcomes. This process
takes on a variety of possible objectives and approaches.

Model Knowledge: This category corresponds to classifying the attack
techniques based on the attacker’s knowledge about the target model. Broadly
speaking, this can be sub-categorized into:

  * •

White Box Attack: In a white-box attack, the adversary knows and exploits the
input and the model information to optimize its attack. The model information
includes input-output data, architecture, parameters, loss, and activation
functions. In this attack type, adversarial data can be generated in a way
that maximizes its impact on the classifier while being an imperceptible
change. Typically, an adversary adjusts adversarial modifications to be in the
direction of the model’s gradient concerning the current input. This maximizes
the increases in the loss function and thus optimizes the attack. White-box
attacks have received a lot of attention in the literature due to their
adaptive nature.

  * •

Black Box Attack: The black box attack techniques do not require or have
access to any knowledge of the model except input and output. This type of
attack uses heuristic approaches or repeated queries to build the attack.

Targeted Output: This feature categorizes attacks into targeted and
untargeted. In a targeted attack, the adversary maps the original model’s
output to a required faulty output for a given input. However, in an
untargeted attack, the adversary cares only about causing the model to produce
incorrect outputs, regardless of what they might be.

Defense: The goal of defense techniques is to design a robust model to fight
against those attacks or to deal with such types of adversarial examples. As
presented in Fig. 1, the major defense techniques include (i) adversarial
example detection and (ii) model enhancement. The goal of the former approach
is to detect an adversarial example that is distinguishable from legitimate
input. Whereas the goal of the latter approach is to train models with
additional parameters, which are commonly referred to as adversarial training.

Our focus in this study is social medial applications with a particular
emphasis on NLP. Hence, in the following sections, we provide in-depth studies
of adversarial attacks and defense techniques reported in social media
applications.

##  3 Adversarial Attacks in NLP

Today’s social media platforms use a versatile bundle of networking tools,
such as messaging, chatting, voice, and video calling. Amongst these, text is
a primary tool for communication in social media [55]. Therefore, we will
especially focus on the research body on adversarial attacks and defenses on
NLP techniques from a social media perspective.

Adversarial attack in an NLP context faces more challenges compared to the
case of computer vision models. Primarily, the discrete nature of text inputs
imposes certain limitations on how one can efficiently modify them. In this
regard, character, word, and sentence-level modifications are primarily text
replacements at the reception levels. To this end, many challenges face the
replacement process. Examples include how one characterizes token 333A token
means a character or a word. or sentence similarity, establishing the
embedding space, i.e., the space of acceptable replacements, and more
importantly, how to search for the best replacement candidate within such a
space. For NLP tasks, adversarial attacks are mainly categorized based on the
granularity of the input. The main categorization of such granularity of the
input includes the following.

  1. 1.

Character-level attacks: achieved by adding, deleting, or swapping a character
in a word.

  2. 2.

Word-level attacks: achieved by replacing an original word with an adversary
equivalent. There is a variety of methods to characterize this word
equivalency. Examples include synonyms, antonyms, and semantic equivalence.

  3. 3.

Sentence-level attacks: are carried out by adding, deleting, or paraphrasing
certain sentences.

  4. 4.

Inter-level attacks are combinations of character, word, or sentence level
attacks.

Such categorizations are inspired based on how these linguistic components are
used in NLP applications to train the machine learning model. For example,
character and word level n-grams and their different representations (e.g.,
tf-idf, word2vec, contextual representation using transformers) have been used
to train the machine. In Figure 2, we present the most common categories of
granularity-based adversarial attacks used in NLP tasks.

Figure 2: Categorization of adversarial attack for NLP tasks.

Ref. | Swap | Scramble | Delete | Insert | Replace
---|---|---|---|---|---
[56] | N | N | Y | Y | Y
[57] | Y | N | Y | Y | Y
[58] | Y | N | Y | Y | Y
[59] | Y | Y | N | N | Y

Table 2: Character modifications of the main character-level adversarial
attacks. Y: Yes, N: No.

Ref. |  | Model
---
Acc.
| Targeted
---
attack
| Targeted
---
Model
| Targeted
---
Application
[56] | W | N | CharCNN | TC
[57] | B | N | CharCNN | SA, TC
[58] | W, B | N | CharCNN | TC
[59] | B | N | CharCNN | MT
[60] | B | Y | Perspective | Toxic word detection
[61] | B | N | Perspective | Toxic word detection
[62] | B | N | DNN | SC, paraphrase detection
[63] | B | N | CharCNN | Multiple
[40] | W, B | N | DNN | Fake news detection

Table 3: Relevant work for the main character-level adversarial attacks in
terms of model knowledge, targeted attack, targeted model, and applications.
B: black-box, W: white-box, Y: yes, N: no, CharCNN: character-level CNN, DNN:
deep neural network, TC: text classification, SA: sentiment analysis, MT:
machine translation, SC: sentiment classification. Model, and, Acc.: Model
Accessibility.

###  3.1 Character-level Attacks

Character level information has been widely used in NLP for text
classification tasks. [64] presents a pioneering work on applying
convolutional neural networks (CNN) for text classification at the character
level. This work has paved the way for character-level operations. In Table 2,
we summarize several character-level perturbations. These primarily include
insertion, deletion, substitution/replacing, swapping, and/or scrambling a
character. In Table 3, we list sample works on character-level attacks with
their model accessibility, attack type, targeted model, application, or task.

As a pioneering work on the character-level attack, [56] investigates white-
box attack with character-level adversarial examples to maximize the model’s
loss at limited numbers of modifications. This is referred to as the HotFlip
algorithm. It is based on performing an atomic flip operation where the
gradient of the model is used to select between several perturbations with
respect to a one-hot vector representation. Such perturbations can be obtained
by flipping, inserting, or deleting a character. In this regard, the authors
argue that a simple beam search approach is superior to a greedy search
between the flip operations.

Along the line of black-box attacks is the work of [57], referred to as
DeepWordBug based on characters and words. Despite its black-box nature, this
method is adaptive to the input. The key idea is to adaptively choose the most
(critical) tokens. DeepWordBug operates in two main stages. First, it
identifies critical tokens, which is based on the ranking of the perturbed
tokens measured in terms of the classifier’s output. Second, it changes the
identified tokens using simple transformations. Such transformations include
four primary types; (i) swapping two consecutive characters, (ii) substituting
a character with another one by randomly selecting from the same word, (iii)
randomly deleting a character, and (iv) inserting a randomly selected
character. This study opened a new research direction in terms of what to
modify and why?. Subsequently, this question was addressed by the TextBugger
method [58] that considered the more general framework of deep learning-based
text understanding. Thus, TextBugger mainly differs in the way how the tokens
are ranked. Similar to DeepWordBug, it operates in two successive stages.
First, it identifies the most dominant tokens for modification. Second, it
modifies the identified tokens. TextBugger selects between five modifications;
(i) inserting a space between characters, (ii) randomly deleting a character,
(iii) swapping two characters selected at random, (iv) replacing a character
with another visually similar character, and (v) replacing a word with a
semantically similar one.

###  3.2 Word-level Attacks

Compared to character-level attacks, a word-level attack is naturally more
imperceptible for humans and more difficult for machine learning algorithms to
defend. In Table 4, we summarize several word-level perturbations proposed in
several research works. In Table 5, we report several word-level attack
contributions and categorize them based on model accessibility, attack type,
targeted model, application, or task. In a broader sense, summarize the main
research contributions made at word-level adversarial attack in Fig. 3. These
can be divided into four main categories; (i) optimizing adversarial attacks,
(ii) identifying vulnerability in new applications, (iii) attack analysis and
understanding, and (iv) investigating new attack aspects. Below we discuss
each of them in detail.

Ref. | Insert | Delete | Swap | Flip | Sub. | Rand. | Embed
---|---|---|---|---|---|---|---
[56] | N | N | N | N | Y | N | N
[56] | Y | Y | Y | Y | N | N | N
[65] | N | N | N | N | Y | N | N
[66] | N | N | N | N | Y | N | N
[57] | Y | Y | Y | N | Y | N | N
[67] | N | N | N | N | Y | N | N
[68] | N | N | N | N | Y | N | N
[69] | Y | Y | N | N | N | Y | Y
[70] | N | N | N | N | Y | N | N

Table 4: Word modifications of the main word-level adversarial attacks. Sub.:
Substitution, Rand.: random attack randomly samples a word to replace the
target word. Embed: it replaces the word with a word among the top-10 nearest
words in the embedding space.

Ref. |  | Model
---
Access
| Targeted
---
attack
| Targeted
---
Model
| Targeted
---
Application
[34] | W | N | DNN | IC
[56] | W | Y | CharCNN-LSTM | TC
[56] | W | N | CharCNN-LSTM | MT
[57] | B | N | WordLSTM, CharCNN | SA, TC
[65] | W, B | N | LSTM, CNN | TC
[71] | B | N | CharCNN, WordCNN | TC
[66] | B | N | CNNS, RNNs | TC, TE
[67] | B | N | DNN | TC
[72] | W | N | DNN | TC, MRC and CTG
[73] | B | N | WordCNN, WordLSTM | TC
[74] | B | N | DNN | MRC, SA, VQA
[75] | B | N | LSTM | SA, TE
[76] | B | N | DNN | Sequence tagging,WC
[77] | B | N | WordLSTM, CharCNN | SA
[68] | W | N | LSTM | SC, CC,GEC
[69] | B | N | BERT | TC
[70] | B | N | CNN, LSTM | TC
[78] | W | N | DNN | IC
[79] | N.A. | N | DNN | Object detection
[80] | W | N | CNN | TC, SA
[81] | W | Y | CNN | TC, SA
[82] | W, B | Y | CNN | TC

Table 5: Relevant work for the main word-level adversarial attacks. B: black-
box, W: white-box, Y: yes N: no, MRC: machine reading comprehension, word-
level CNN: WordCNN, word-level SLTM: WordLSTM, TC: text classification, MT:
machine translation, SA: sentiment analysis, SA: sentiment classification, TE:
text entailment, IC: image classification, GEC: grammatical error
classification, WC: word classification, CTG: conditional text generation.
Tar.: Targeted. Figure 3: A summary of the main research contributions in the
area of word-level attack.

####  3.2.1 Optimizing adversarial attacks

A major part of research on a word-level attack is focused on optimizing
adversarial attacks. In this study, we categorize the endeavors made in
optimizing word-level attacks into the following sub-categories.

##### Minimizing word replacements

The first research body under the umbrella of optimizing world-level attacks
focuses on minimizing word replacements. Generally speaking, the most common
perturbation type in word-level attack is based on word replacement.
Typically, a target word is replaced by an equivalent one chosen from a space
of possible replacements. Such a space was identified by Ebrahimi et al. [83]
and referred to as the embedding space. In [83], the authors consider neural
machine translation (NMT) and propose elementary modifications to achieve
word-level adversarial attacks. In this regard, the authors extend the HotFLip
algorithm proposed in [56] to the word level. Essentially, Hotflip is assumed
to have white-box access. Thus it uses the model’s gradient to optimize word
modification selection such that the model’s loss is maximized greedily.
Similar to other works, word modifications include adding, removing, or
replacing a word in the translation input.

Another work on a word-level adversarial attack is proposed in [81]. In this
work, the authors highlight the differences between adversarial attacks in
vision and NLP domains. In particular, they point out that one can not
arbitrarily modify text inputs as done in a computer-vision context. Thus, the
most important contribution in this work is necessitating the importance of
exploiting semantic and syntactic similarity between an original word and its
adversarial replacements. Such similarities were overlooked by the literature
before this work. The authors exploit these similarities in word replacement
search using a population-based optimization algorithm to generate the
embedding space of adversarial samples exhibiting semantic and synthetic
similarities. Besides, they use this algorithm to select the best replacement
of a given original word with respect to semantic and syntactic similarity and
maximizing the model’s loss. It is noted that they selected the word to be
modified at random. Technically, this process is conducted in the following
order.

  1. 1.

Compute the nearest neighbors for a given word within its embedding space
based on Euclidean distance.

  2. 2.

Rank the similarity of those neighbors to the original word in terms of the
difference between their label productions and that of the original word. Then
keep a few neighbors having the maximum similarity

  3. 3.

Amongst the neighbors of maximum similarities, select the one that maximizes
the loss function of the model.

##### Improving the search for word replacements

Another sub-category of works on optimizing adversarial word-level attacks is
concerned with improving the search for word replacements in an embedding
space. In the current literature, there are three main categories of text
replacement techniques: (i) gradient-based [83, 8, 80], (ii) sampling-based
[34, 75], and (iii) enumeration based [59, 84, 74], as shown in Fig. 4. These
techniques vary in the extent of the assumed model accessibility, and thus the
information used to dictate the search process.

Herein, we provide two examples of improving the search process in the
embedding space. First, the above-mentioned DeepWordBug algorithm [57] also
applies to the word level. In this setting, the same selection process is
adopted. In other words, the algorithm works in two stages; first, it selects
a set of critical words and then chooses the best amongst them. Another
example is the TextBugger algorithm of [58] which also applies to the word
level.

Figure 4: Word replacement search techniques.

##### Obtaining embedding spaces

As discussed earlier, selecting the best replacement is of crucial importance
for the success of the attack. Still, it is also important to first provide a
suitable embedding space. So, the next research body under the umbrella of
optimizing word-level attacks concerns obtaining the embedding space. Along
this line, [75] propose a black-box, gradient-free genetic optimization
algorithm for generating adversarial replacements based on natural selection
of the genetic algorithm. Specifically, it is an iterative process for
providing a generation of selections per iteration. It is noted that a fitness
function is used to quantify the quality of a candidate. Besides, candidates
of each generation are obtained by a cross-over of their preceding ancestors.

Several approaches aim at leveraging methods for defining image-based
embedding space to the text application area while keeping in mind the
differences between text and image data. Along this line, [80] proposes a
framework for utilizing ideas from image attacks to be used as text attacks.
First, the authors highlight the main differences between image and text
domains in terms of generating adversarial attacks. In this regard, the
authors report the bottleneck in this borrowing which can be cast as two main
limitations. First, is the discrete nature of text rendering small
perturbations inapplicable in a text context. Second, is the difficulty in
evaluating how good an adversarial attack is. As a remedy to the first
limitation, the authors propose searching within the set of possible word
embedding models and then choosing the best adversarial alternative to the
original word in terms of its nearest neighbor similar to the original. To
alleviate the second limitation, the authors use a Word Mover’s Distance (WMD)
measure as an adversarial replacement quality metric.

##### Improving word replacement similarities

It is well-known that an adversarial replacement should be, at least visually,
similar to the original word. Moreover, it is necessary to quantify this
similarity. Therefore, another research body under the umbrella of optimizing
word-level attacks focuses on improving the similarities used in identifying
word equivalency. Along this line, [74] introduces two approaches for
quantifying word similarity. First, is the concept of semantically equivalent
adversaries (SEAs). In this context, SEAs are used to generate semantically
equivalent adversarial replacements based on paraphrase generation techniques.
These techniques include back-translation, i.e. translating a given sentence
to a certain language, and then back-translation of the original language.
Another example is the use of paraphrasing to obtain such replacements. In
this setting, it is noted that the semantics of the text are preserved since
the meaning is not changed. The second approach proposed by the authors is
devising equivalent adversarial rules (SEARs) that govern the process of
obtaining semiannually equivalent replacements.

In line with identifying and exploiting word replacement similarity, Ren et
al. [70] consider imposing further restrictions on adversarial replacements.
Namely, lexical correctness, grammatical correctness, and semantic similarity.
Specifically, the authors propose an algorithm for adversarial word selection
ordering where the saliency of the word and its classification probability are
taken into account for determining its order in the embedding space. This
algorithm is referred to as probability-weighted word saliency (PWWS) for text
adversarial attacks. Ren et al. experimentally validate a high success rate of
their algorithm at minimal word replacements. Moreover, the authors
demonstrate that their modifications are salient at the human level through
human evaluation of the attacks. Similar to other methods, this algorithm also
possesses a high degree of transferability of its examples across different
models and datasets.

To this end, word-level adversarial replacement techniques are accused to tend
to overlook the linguistic context while replacing targeted words. This
renders them vulnerable to human discovery. Accordingly, [73] calls for a
wiser selection of the words. In particular, the authors incorporate context
consistency as a constraint for replacement in a black-box adversarial attack
algorithm. Algorithmically, this is achieved by sorting replacement
suitability values based on a BERT-masked language model.

##### Developing attack triggers

The studies surveyed so far consider primarily input-dependent adversarial
attacks. Therefore, the next research body under optimizing word-level attack
considers developing (input-agnostic) attack triggers. Along this line, [72]
investigates input-agnostic adversarial examples, referred to as triggers, at
the token level in NLP models. The authors propose a gradient-based search for
the best token to change and use minimal perturbation length. In [72], the
authors validate a high success rate of their algorithm as tested over various
tasks and networks. Besides, they demonstrate that it possesses a degree of
transferability regardless of its white-box nature. In this regard, such
attacks transfer across both examples and models for all tasks. As a future
outlook, the authors necessitate investigating the development of grammatical
triggers that can work anywhere in the input. Besides, future research may
also consider dataset- or even task-agnostic triggered. Moreover, the authors
report that adversarial attacks raise questions on whose responsibility it is
that models are vulnerable to such attacks.

##### Investigating attack design trade-offs

The next research body under the umbrella of word-level attack concerns
identifying and balancing design trade-offs. An inherent trade-off in the
design of adversarial attacks is the saliency-strength trade-off. In this
context, a strong perturbation is better to change the model output at the
cost of reduced saliency, and vice-versa. This forms the fundamental design
trade-off. However, identifying and balancing other design trade-offs was not
considered until the work of [71]. In this work, the authors devised and
investigated a probabilistic framework for generating adversarial attacks on
models with discrete inputs. [71] propose two methods for world-level
adversarial attacks and then investigate the underlying design trade-offs
encountered with these methods. .

####  3.2.2 Attack analysis and understanding

The second aspect of research contributions in word-level attacks concerns
developing the understanding of adversarial attacks. In this regard, [76]
argues that the interest in the applicability of ML models came at the cost of
not giving enough care to the interpretation and the understanding of how ML
models yield their outputs based on inputs. To bridge this gap, the authors
investigate the sensitivity of the model’s operation to input deletion, in
particular. Specifically, the authors analyze how the behavior of a given
model will change with erasure operations of different types and natures on
input data. The authors investigate the use of reinforcement learning for
detecting output-changing phrases at the input side. The result of the work
conducted by [76] is a set of important interpretations and explanations of
certain ML operational phenomena.

####  3.2.3 Identifying vulnerability in new applications

Another research body in the context of word-level attack focuses on
identifying the vulnerability to adversarial attacks exhibited by new NLP
tasks and application areas. In this regard, Ling et al. [82] consider the
vulnerability of DNN-based text classifiers to adversarial input attacks. In
particular, the authors propose three main methods for generating text
adversarial examples; insertion, removal, and modification. Experiments
validate that these methods, whether applied on the character level or the
word level, can trick DNN models and cause them to generate wrong outputs.
This is while the perturbations generated by such methods are still
imperceptible at the human level.

It is worth mentioning that Liang et al.’s work is based on identifying the
best characters to change, i.e., the hot characters. Then, it applies
adversarial changes on other words containing more than 3 hot characters and
refers to them as hot words. Similarly, hot phrases are identified as the ones
having the maximum number of hot words. Therefore, this approach seems
conceptually similar to what HotFlip does.

Another work by [65] considers the vulnerability of natural language
classification models to adversarial attacks. This work establishes the
existence of adversarial attacks in an NLP context. Specifically, it points
out that, similar to other ML application areas, natural language
classification models are vulnerable to attacks composed of small
perturbations at the input leading to changing the NLP model’s output.

As an attempt at investigating other text-related ML applications, [79]
consider ML models used in the application of object detection. The authors
show that such models are, similar to the case with other ML applications,
inherently vulnerable to adversarial attacks on the image patch level. This is
shown by proposing an efficient adversarial attack algorithm that targets a
specific class of the object detection model. Thus, the authors craft their
attack by tracking the model to be blind with respect to a certain object in
an image patch called the invisibility patch. This algorithm has the ability
to track several state-of-the-art object detection approaches. Besides, it
transfers across different models and datasets.

####  3.2.4 Investigating new aspects of the attack

Lastly, another research body of word-level attack contributions focuses on
the attack itself. The common practice in adversarial machine learning has
remained restricted to altering the input/model while leaving the input labels
intact. In contrast to this widely-used assumption, [78] proposes a bilateral
perturbation process, i.e., altering both input and the label. Thus, for
altering a given input’s label, the authors propose a closed-form mathematical
expression. However, to alter an input, the authors use a one-step
perturbation process while adopting a class label that maximizes confusion.
The idea of perturbing both input and the label has been shown effective.

###  3.3 Sentence-level attacks

In sentence-level attacks, an adversary considers the inputs as sentences, and
thus, applies adversarial modifications on them by inserting, replacing, or
deleting sentences. Compared to character-and word-level attacks, sentence-
level attacks demand a longer time in adversary text generation. The key
attributes of several sentence-level adversarial attacks are presented in
Table 6.

Ref. |  | Model
---
Acc.
| Targeted
---
attack
| Targeted
---
Model
| Targeted
---
App
[77] | W | N | WCNN, LSTM | TC
[85] | W, B | N | DNN | QA
[84] | B | N | LSTM | SA, TE
[86] | W | N | Transformer ([87]) | MT
[88] | W | N | LSTM, Transformer ([87]) | MT
[89] | B | N | DNN | Lip reading
[90] | W, B | N | DNN | IC

Table 6: Specifications of the main sentence-level adversarial attacks. B:
black-box, W: white-box, QA: question answering, TC: text classification, MT:
machine translation, SA: sentiment analysis, and TE:textual entailment.

There are several challenges to conducting adversarial attacks on models with
textual input (e.g., sentence). One is finding suitable candidate replacements
so that generated text preserves syntax and semantics. Another is developing
an efficient approach to finding good transformations. The work of Lei et al.
[77] proposes methods that address both challenges (i) sentence and word
paraphrasing that preserve syntax and semantics, (ii) gradient-guided greedy
paraphrasing approach to find suitable transformations. Along this line, Iyyer
et al. [84] propose syntactically controlled paraphrase networks to generate a
paraphrase of the sentence, which is syntactically valid. This study shows
that such generated adversarial examples i) can fool pre-trained models, and
ii) when trained as augmented data can improve the robustness of the model.
The work of Jia and Liang [85] also explores sentence-level adversarial
examples, where the authors investigated whether a distorted sentence in a
paragraph can lead to an incorrect answer.

###  3.4 Multi-level text adversarial attacks

By multi-level attack, we mean an adversarial attack that perturbs characters
and words or words and sentences, or all of these. A multi-level attack is
largely considered unexplored and open for future research. The few research
studies mentioned in this section evaluate a preliminary approach to
integrating attacks at different text levels. Another reason why multi-level
attacks are unexplored heavily in literature is that they have nothing
comparable to image-based adversarial attacks. As adversarial attacks in text
are more recent than adversarial attacks in images, many of the methods in
text attacks are inspired by early image attacks.

As multi-level attacks involve at least two of the three text-level attacks,
they are typically more complex and computationally expensive. Similar to the
earlier categories, multi-level attacks can be used in white-box attacks
(e.g., [56], [85], [72]), as well as black-box attacks (e.g., [91]).

##  4 Defense Methods in NLP

Similar to the case of adversarial attacks, adversarial defense research in
NLP is still in its early stages as compared to its counterparts in computer
vision. In the broad sense, the existing efforts in this domain can be divided
into four categories: (i) adversarial training, (ii) spell and grammatical
check, (iii) defensive distillation, and (iv) recovery of perturbations. In
the next subsections, we provide a detailed overview of the literature on each
type of adversarial defense methods.

###  4.1 Adversarial training

Adversarial training is the process of training a model on a mixture of clean
data and adversarial examples. The inclusion of adversarial examples in such a
training set has been shown to improve the model’s robustness against those
examples[31]. This defense scheme aims at suppressing the impact of an
adversarial attack, assuming it has already happened or will inevitably
happen. Intuitively, the re-trained model is expected to be more robust to the
adversarial attacks whose instants are included in the training, as compared
to the original model. In the adversarial security research area, adversarial
training has been used by almost all adversarial attack works [92]. It is
worth mentioning that, improving the model robustness by adversarial training
has been widely used as an implicit indicator of the success of the underlying
adversarial attack.

Despite being the first line of defense, there are certain limitations and
drawbacks of adversarial training, as summarized below.

  1. 1.

Some recent works are skeptical about the extent to which adversarial training
can truly improve the model’s robustness. However, to address such limitations
more robust approaches have all been proposed [68, 62]. Some works report
marginal/moderate robustness improvement for the adversarial training in
certain NLP applications [68, 62].

  2. 2.

Adversarial training has recently been shown effective exclusively with the
attack whose adversarial samples are used in its training [85]. This led to
the inception of “blind-spot” attacks [93] where the modified input resides in
a so-called blind spot. This refers to the state of being far away from
regular and potential inputs while still belonging to the data distribution.
Thus, any adversarial training attempt is very unlikely to include such an
example in its training set.

  3. 3.

The current adversarial training procedures cannot scale to datasets with a
large (intrinsic) dimension [93]. Besides, adversarial training is shown to
work well only with repetitive data patterns in a training set.

Recently, there has been an interest in extending the classical adversarial
training concept. Along this line, [68] considers an inherent gap between
adversarial training in image processing and NLP application domains. While
perturbations used in adversarial training for image processing can take any
arbitrary direction, the authors signify that this approach should not
directly apply to the NLP case. This is because such an approach ignores the
inter-portability of the added perturbations. Therefore, the authors call for
restricting the perturbation space to the dimension of possible word embedding
models. In other words, perturbations in an NLP context should only yield
meaningful tokens from a linguistic perspective. Through a set of experiments,
the authors demonstrate that their approach generates interpretable
adversarial outcomes while maintaining the attack performance.

###  4.2 Spelling and Grammatical Check

As character or word-level attacks are based on modifying characters and
words, several studies investigate the utility of spelling and grammatical
checks of the input text as means for defense [57, 58, 62, 94, 95]. There have
been several contributions in this direction. For instance, Gao et al. [57]
use an auto spell corrector to boost and improve the model’s robustness
against adversarial attacks. Similarly, Li et al. [58] employ spell check as a
tool for eliminating character-level modifications. In a more recent work,
[62] propose appending text classification by word recognition. This aims at
counteracting the effect of adversarial perturbations applied to characters.
In this regard, various perturbations and changes such as character swapping,
replacement, and keyboard typos can be counter-acted.

Word-level input correction defense considers context-guided spell and grammar
checks to guard against adversarial attacks. For instance, in [94], a two-
stage spell correction scheme to identify and correct misspelled words is
proposed to guard against adversarial attacks. As another example, Bao et al.
[95] propose a multi-task learning framework to identify adversarial sentences
where adversarial words are embedded.

###  4.3 Defensive distillation

The distillation concept originated as a means for reducing the size of a
given DNN architecture. This is possible by the virtue of training a smaller
network on the logits and the training data of a given (larger) network. Thus,
the smaller network will inherit the same functionality as the former one.
More recently, [34] has adopted this idea as a defensive means for combating
adversarial attacks. Specifically, the authors anticipate that the trained
network will only inherit the benign functionality of the original one while
being distilled from any adversarial functionality. It is noted that the
above-mentioned approach has been shown successful in combating adversarial
attacks in the image domain [96].

###  4.4 Recovery of Perturbations

The literature also witnesses some efforts for the identification and recovery
of perturbations. For instance, a defense framework proposed by Zhou et al.
[69] aims to determine whether a particular token is a perturbation or not.
The process is composed of two phases. In the first phase, a set of possible
perturbations that could have been applied to the text of interest is
identified. The second phase aims at calculating the value of a specific
attack estimator for each of the attack possibilities. After that, the token
that is identified as a perturbation is reconstructed/restored from the
embedding space based on a certain similarity measure. The authors used the
k-nearest neighbors (kNN) to guide the search. The study has also concluded
that the technique is generally successful in identifying attacks in a variety
of NLP models and applications without the need for any retraining.

Another work in [63] develops a rule-based recovery of adversarial
perturbations. The defense technique is based on replacing each non-standard
character in the input stream with its nearest standard neighbor. In other
words, this approach is based on reversing a text attack. The proposed
solution has been proven very effective in machine learning translation
applications.

##  5 Adversarial Attack and Defense on Social Media Applications

In this section, we study five major social media applications that are
vulnerable to adversarial attacks and summarize the literature that explores
attack and defense techniques for these applications.

###  5.1 Rumors

As social media has become a major communication channel where users are not
only consuming news but also propagating them by sharing and producing them.
Such a consumption and dissemination approach led to widespread rumors which
causes serious consequences to the individuals, organization, or the society
as whole [97]. To address such an issue social media platforms, government
entities, journalists, fact-checkers, research communities, and other
stakeholders are fighting to reduce the negative impact of rumors. While fact-
checking organizations such as FactCheck.org and Snopes.com are manually
checking and making them publicly available to facilitate other stakeholders
and reduce the spread. As such an effort does not scale well, hence, the
research community has been trying to develop an automated system to detect
and alert users about such rumors (see recent surveys [98, 97]).

In literature rumor has several definitions [98, 98]. (i) Rumor is a story or
statement whose truth value is yet to be verified at the time of posting [99].
(ii) The truth value of a story or statement is verified authoritative sources
and confirmed that it is false or fabricated, which is also referred to as
false rumor [100]. (iii) The third definition of a rumor is based on the
users’ subjective judgment of the veracity of the story or statement [101].
The former one is the most widely used and consistent with the definition of
different dictionaries (e.g., Oxford English Dictionary).

Rumor can be of different types: (i) long-standing rumors that have been
circulating for a long time, (ii) emerging rumors during any event (e.g.,
rumor about COVID-19). Depending on the type of rumor classification systems
have been designed accordingly.

A typical rumor classification system consists of several components, as can
be seen in Figure 5 (adopted from [97]). News or social media posts are
monitored to detect whether they are rumors or not. Once a rumor is identified
then posts related to the rumor are detected to flag them. The stance
classification component then determines how each post reflects a particular
stance on its like veracity. Then the last component determines the truth
value of the rumor.

Figure 5: A typical system architecture of rumor classification system.
Adopted from [97].

To design the machine learning models for such components different types of
methods have been used. The work by Cao et al. [98] categorizes three types of
methods. Handcrafted feature-based approach: features are extracted from
different modalities (textual, visual, and social features) depending on their
availability then classical machine learning methods (e.g., Decision Tree,
Bayesian Networks, SVM, Random Forest, and Logistic regression) are used to
train the model. Propagation-based approach: uses network-based information
such as users, messages, and events to train the model. Deep learning
approach: trains the model using a combination of different sets of features.

Adversarial work: While malicious actors are trying to spread rumors to
achieve political and financial goals, the research community and social media
platforms are trying to develop automated systems/models to debunk them. These
models are susceptible to malicious attacks too by malicious actors. Hence, it
is important to develop models that are robust enough to deal with such
adversarial attacks. Work in this direction for rumor detection are relatively
very few. Ma et al. [41] is the first study that attempted to create robust
models using GAN, which has been evaluated using two real-world datasets.
Xiaoyu et al. [102] propose graph adversarial learning to graph structure to
reduce the intentional perturbations. Song et al. [103] propose a method,
which includes weighted-edge transformer-graph network, and position-aware
adversarial response generator modules. In Table 7, we report notable work on
rumor detection, which propose different attack and defense methods.

Ref. | Dataset | Method | Attack | Defense
---|---|---|---|---
[41] |  | Twitter [104],
---
Pheme [105]
GAN |  | Generator generates
---
proximately realistic
examples
| Discriminator tries
---
to distinguish
between real and fake
[102] |  | Weibo [106],
---
Twitter [107]
| Graph
---
adv.
learning
| Dynamically add
---
intentional perturbations
on the graph structure
| Learn more distinctive
---
structure features to resist
such perturbations
[103] |  | Pheme [105],
---
Twitter 15,
Twitter16 [108]
| WETGN,
---
PRAG
White box attack |  | Generates new adver-
---
sarial responses

Table 7: Recent work of adversarial techniques for rumor detection. WETGN:
Weighted-Edge Transformer-Graph Network, PARG: Position-aware Adversarial
Response Generator.

###  5.2 Satires

Satire is defined in the Cambridge Dictionary as “a way of criticizing people
or ideas in a humorous way”.444https://dictionary.cambridge.org/ It is a
literary device that writers use to mock or ridicule a person, group, or
ideology by judging them for various issues, particularly in the context of
contemporary politics and other topical issues [109]. Such devices include
humor, irony, sarcasm, exaggerations, parody, or caricature [110]. These are
typically applied to news and social media posts and the purpose is not to
cause harm but to ridicule, or expose behavior that is shameful, corrupt, or
otherwise “bad” [111, 112, 113]. Even the intention is not to mislead,
however, the content can be mistaken by the reader as legitimate news, which
can lead to the spread of misinformation [109, 114] and can be harmful [115].
Hence, it became essential to automatically identify them at a large scale and
the typical classification task was to differentiate between real, fake, or
satire news [112, 109].

For designing the classification model, different classical and deep learning-
based algorithms have been used. These include Naive Bayes Multinomial, LSTM,
CNN [112, 116]. To train the model typical features include entity mentions
[116] coherence information [117], distributions of parts-of-speech,
sentiment, and exaggerations [118]. Other approaches include neural network
and attention mechanism to incorporate paragraph-level linguistic features
[115], hierarchical deep neural network to incorporate sentence level and at
the document level [119] and multimodal learning [109] using state-of-the-art
vision-linguistic model ViLBERT.

Adversarial work: The work on adversarial training for satire detection is
relatively new. The study in [110] used adversarial training while training
the model, however, the purpose was not to defend an adversarial attack,
rather the idea was to control the effect of publication source information in
satire detection.

###  5.3 Clickbaits and Spams

Social media platforms, such as Facebook and Twitter, have enabled people
around the globe to connect and exchange ideas with each other. In doing so,
they have also created opportunities for cyber-attackers and cyber-trolls to
disseminate misinformation, offend and cyber-bully the users, and cause
disputes through social media and public forums. [120] describe online trolls
as malicious users pretending to be sincere members of a group discussion but
subtly attempting to disrupt the discussion and cause conflict. [121] claim
that the lack of attention to social science by the designers and developers
of social media is responsible for spreading misinformation. Cyber-attackers
leverage digitalization for online discrimination, privacy breaches,
misinformation, and cyberattacks [122].

[123] provides an example of misinformation in Italy. The Five Star Movement,
founded by a national celebrity, linked with numerous social media pages and
websites promoting the Catholic faith and spreading nationalist, anti-
immigrant, and anti-Islam rhetorics. Similarly, [124] addresses the political
threats posed by misinformation towards elections, democracy, and citizens via
OSNs. These threats include releasing fake documents and ridiculing the
candidates, spreading misinformation about the voting procedure, and harassing
and intimidating minority groups. Also, the authors highlight the steps, such
as extending existing criminal and human rights laws to online space, taken by
the Canadian government to mitigate cyber-attacks.

ML models have proven effective in detecting malicious emails, fake news, and
harmful data traffic in real-time ([125]). [126] propose a method by drawing a
connection between the surface features and the document-level characteristics
of discourse for automated detection of fake news and misinformation. The
proposed model achieves a veracity accuracy of 74%. Likewise, [127] combines
text-mining, behavior analysis, and connection graphs of a discourse, which
involves users. It compares the results with a predefined knowledge base as an
automated fact-checking and fake news detection model.

Attackers can leverage adversarial machine learning (AML) to evade detection
and amplify their attacks (e.g., an attack on spam email filter [128]). [120]
introduces TrollHunter, an ML model to detect the spread of misinformation and
fake news on Twitter by leveraging linguistic analysis. TrollHunter achieves
an accuracy of 98.5% when detecting malicious tweets. The authors also present
TrollHunter-Evader, which utilizes adversarial machine learning techniques,
such as Test Time Evasion (TTE) and Ambient Tactical Deception (ATD), to evade
detection, with a success rate of 40%. [40] Malicious Comment Generation
Framework (MALCOM) generates relevant and acceptable phrases. It replaces them
with the ones in a malicious discourse to evade ML detection models with a
success rate of 94%.

Cyber-attackers utilize various psychological and technology-based techniques
to perform scams and hoaxes on their targets. Internet users face online
threats in the forms of phishing, pharming, hacking, profiling, and
physiological influence [129]. In response, [130] implements an anti-spamming
email filter based on unsupervised Artificial Neural Networks (ANNs), which
achieves ROC AUC of 0.97, with an average sensitivity $>$ 0.95. However, the
performance declines due to concept drift as emails with new topics and
structures are introduced to the model. The accuracy of ML models decreases
over time due to changes in news topics, the way they are reported, and
evasion techniques used by adversaries, known as ‘concept drift’ [131]. Thus,
the authors recommend two approaches to mitigate the effects of concept drift:
periodically training machine learning using the previous data samples and
deploying Dynamic Weighted Majority into the system.

In addition to adversaries causing misclassification by masking the input data
with noise and poisoning training samples, implementation of ML models can be
challenging due to their complexity and lack of transparency, making them
incomprehensible to humans, making their decisions untrustworthy [132]. To
mitigate the negative perception of AI, [133] recommends educating developers
on technology-based ethics and the potential effects on society, having a
regulatory authority to monitor AI-based processes and making the AI
computations and databases transparent to the consumers.

In Table 8, we summarize the related work on clickbaits and spam detection
addressing different attack and defense methods.

Ref. | Dataset | Method | Attack | Defense
---|---|---|---|---
[72] | GOSSIPCOP | RNNs |  | Universal
---
targeted
attack
| Uses word
---
recognizer
and topic
relevancy
method
[134] | Untrue.News [135] |  | RoBERTa,
---
BERTweet,
FlairEmbeddings
| TextAttack
---
recipes [136]
N/A
[137] |  | AdaBoost | Relabeling attack [138] |  | a k-NN-based
---
mechanism
to sport
outliers

Table 8: Recent work of adversarial techniques for clickbaits and spam
detection.

###  5.4 Hate Speech Detection

Hate speech, which refers to abusive or threatening speech or writing
expressing a preconceived opinion against a particular group based on
prohibitive attributes, such as color, race, religious beliefs, gender, or
sexual orientation, is considered as one of the main causes of increasing
global violence [139]. Social networks, such as Facebook, Instagram, and
Twitter, and greater accessibility to Internet have further amplified it by
allowing people to express their opinions to the global audience more freely
and effectively [10, 11].

Considering the increasing concerns over global violence, several efforts have
been made to identify the potential sources and reduce the spread of hate
speech over social networks. AI, ML, Data Science, and NLP communities are
also playing their part by proposing interesting hate speech detection
techniques [140]. For instance, [141] proposes a hate speech detection and
classification framework for Twitter text streams based on BERT (Bidirectional
Encoder Representations from Transformers). . However, similar to other NLP
applications, hate speech detection methods are also subject to adversarial
attacks, for instance, as demonstrated in [142] where hate speech recognition
models were fooled by modifying the text using state-of-the-art NLP
adversarial attacks. The goal of such adversarial attacks is to disturb the
classification capabilities of the classifiers resulting in misclassification
of abusive and toxic content. The literature provides several examples of
adversarial attacks on hate speech detection techniques. For instance,
Grondahl et al. [52] employed three different types of adversarial attacks to
fool hate speech recognition models through (i) word changes, (ii) word-
boundary changes, and (iii) appending unrelated innocuous words. Similarly, in
[53] hate speech detection models are fooled through (i) typos, (ii) removing
white spaces, (iii) inserting benign words, and (iv) appending character
boundaries. The authors also launched attacks on the models by combining all
the individual types of attacks resulting in a more effective adversarial
attack.

To guard against the attacks on hate speech detection models, several
interesting defense strategies have been introduced in the literature. The
majority of the proposed methods rely on adversarial training to cope with the
perturbations [51, 143, 144, 52]. For instance, in [143] adversarial training
is used to guard against Fast Gradient Method (FGM) attack by extending FGM
with a learnable and fine-grained noise magnitude where noise (perturbation)
is added to misleading samples. Besides adversarial training, some solutions
also rely on pre-processing to defend against adversarial attacks on hate
speech detection models [52, 54]. For instance, Moh et al. [53] propose four
different pre-processing defense techniques, namely (i) word segmentation no
redo (WSNR), (ii) word balance no redo (WBNR), (iii) good grammar no redo, and
(iv) vowel search no redo (VSNR). The defense techniques deal with white-space
removal, typos, benign word insertion, and character boundary appending
attacks, respectively.

Table 9 summarizes some key papers on adversarial attacks and defense methods
for hate speech detection.

Ref. | Dataset | Method | Attack | Defense
---|---|---|---|---
[53] |  | W, T1,
---
T2,
T3 [145]
| MLP,
---
LA,
LSTM
| Pre-processing
---
attacks
| WSNR,
---
WBNR,
GGNR,
VSNR
[143] |  | Self
---
collected
BERT |  | Fast
---
Gradient
Method (FGM)
| Adversarial
---
training
[52] |  | W, T1,
---
T2,
T3 [145]
| LSTM,
---
RNNs
| word &
---
word-boundary
changes,
appending
unrelated
words
| Transfer learning
---

Table 9: Recent work of adversarial techniques for hate speech detection. MLP:
Multilayer Perceptron, LA: Linear Regression, WSNR: word segmentation no redo,
WBNR: word balance no redo, GGNR: good grammar no redo, VSNR: vowel search no
redo. Here T1, T2, T3 represent three different datasets collected from
Twitter while W represents a dataset collected from Wikipedia.

###  5.5 Misinformation

Misinformation is perhaps the most innocent of the terms discussed here. Its
misleading information is created or shared without the intent to manipulate
people. An example would be sharing a rumor that a celebrity died, before
finding out that it is false.

Disinformation, by contrast, refers to deliberate attempts to confuse or
manipulate people with dishonest information.

To fight against such false or misleading information, several initiatives for
manual fact-checking have been launched. Some notable fact-checking
organizations include _FactCheck.org_ ,555http://www.factcheck.org _Snopes_
,666http://www.snopes.com/fact-check/ _PolitiFact_
,777http://www.politifact.com and _FullFact_.888http://fullfact.org A large
body of research focused on developing automatic systems for detecting the
factuality of such information [146, 147, 148, 149, 150].

Such detection systems are also vulnerable to adversarial attacks. [133, 151].

The study by [133] demonstrates three adversarial examples: Fact distortion,
subject-object exchange and cause confounding. As a defense mechanism against
adversarial attacks, the authors propose a crowdsourced knowledge graph to
collect timely facts about news events. [8] describes adversarial attacks on
social networks from two perspectives, attacks or manipulation of the network
versus manipulation of the content.

Several papers (e.g. [151] called for the need to build adaptive ML models
that are more immune and robust to variations and perturbations of text inputs
or features.

Several defense approaches against adversarial attacks on NLP are proposed in
the literature (e.g. adversarial training, [152], [68], [153], optimization-
based methods: e.g. [154], [155], defense against neural fake news: e.g. [156]
and [157], [40], and word/sentence embedding-based defense: e.g. [158], [159].
[160] discussed three defense methods to adversarial attacks on
misinformation: data modification, models modification and using auxiliary
tools.

Ref. | Dataset | Method | Attack | Defense
---|---|---|---|---
[152] |  | IMDB
---
DBpedia
Self
collected
LSTM |  | Random
---
perturbation
| Adversarial
---
training
[159] |  |  | RNN,
---
BERT
| Pre-processing
---
attacks
| Word/sentence
---
embedding-based
defense
[40] |  | GOSSIPCOP [161],
---
PHEME [162]
RNNs |  | Comment generator
---
in white box
setting
| Adversarial
---
training

Table 10: Recent work of adversarial techniques for misinformation detection.

###  5.6 Sentiment Analysis

Sentiment analysis, which is also known as opinion mining, is another
interesting application of NLP, and generally involves the use of modern
technologies to analyze and extract quantitative results. Generally, the
results of sentiment analysis are presented in the form of positive, negative,
and neutral sentiments. In the literature, both textual and visual contents
have been analyzed to extract opinions about an entity [5, 163, 164]. However,
text is more exploited due to a diversified set of applications and the level
and freedom of expressing sentiments

The recent increase in the popularity of social media outlets, such as Twitter
and Facebook, has further increased the importance, opportunities, and
challenges associated with sentiment analysis [165]. For instance, sentiment
analysis tools allow businesses to monitor and analyze the popularity and
users’ feedback on them and their competitors’ products/services by processing
text reviews shared by the users in online social networks. The objective
quantitative results obtained through sentiment analysis are then utilized in
making critical business decisions.

Similar to other NLP applications, due to the importance of results obtained
from sentiment analysis in the decision-making process, sentiment analysis
algorithms are also subject to several adversarial attacks. In sentiment
analysis, an attacker can launch adversarial attacks by adding small
perturbations to text to generate different perceptions than the actual
opinions. For instance, in [166], the vulnerabilities of a lexical natural
language sentiment analysis algorithm are identified and analyzed under
various types of adversarial attacks. Based on the experimental results, the
authors conclude that the classifier’s results could be significantly affected
by exploiting the identified vulnerabilities. Alzntot et al. [81] on the other
hand utilize a black-box population-based optimization algorithm for the
generation of semantically and syntactically similar adversarial examples to
launch attacks on sentiment analysis models. In [167], three different types
of character-level adversarial attacks are launched against a BERT-based
sentiment analysis framework. The strategies used for generating the
adversarial examples include mimicking human behavior and using leetspeak,
misspellings, or misplaced commas. These strategies allow maximizing
misclassification rates of sentiment analysis classifier with minimal changes.

The literature also reports some interesting solutions to guard against
adversarial attacks and extract correct perceptions on content shared in
social networks. For instance, in [168], an adversarial training-based
solution has been proposed for aspect-based sentiment analysis. To this aim,
two different BERT models, pre-trained on general-purpose and domain-specific
data, are fine-tuned in a novel framework-based adversarial training. Wang et
al. [169] on the other hand propose an adversarial defense tool namely
TextFirewall for sentiment analysis algorithms. TextFirewall mainly relies on
the inconsistency between the sentiment analysis model’s prediction and the
impact value, which is calculated by quantifying the positive and negative
impact of a word on the sentiment polarity. Hosseini et al. [60] used spell-
checking for limiting adversarial modifications at the character level. This
scheme is also used in [166] to guard a sentiment analysis classifier against
insertion and word substitution attacks. Du et al. [170], on the other hand,
employ network distillation technique along with adversarial training for
robust sentiment analysis.

Table 11 summarizes some key findings on adversarial attacks and defense
methods for sentiment analysis.

Ref. | Dataset | Method | Attack | Defense
---|---|---|---|---
[169] |  | IMDB [171],
---
Yelp [64]
DNNs |  | Deepwordbug [57],
---
GA [81],
PWWS [70]
| Considers inconsistency
---
between the model’s
output and the
impact value.
[81] | IMDB | DNNs |  | black-box
---
population-based
optimization
Adversarial Training
[166] | IMDB |  | lexical
---
natural language
classifier
| Characters insertion
---
Word substitution
N/A

Table 11: Recent work of adversarial techniques for sentiment analysis.

##  6 Research Challenges, and Lesson Learned

###  6.1 Research Challenges

In this section, we discuss current challenges and limitations in the area of
adversarial attack generation and defense in text-based social media
applications.

  * •

Discrete data perturbation: This is a fundamental challenge in generating
adversarial examples in a text as compared to images. It is widely believed
that computer vision and image processing approaches to adversarial example
generation do not directly apply to discrete inputs. In essence, such a direct
application will result in exposing the attack as characters and words are not
salient anymore. This is the case as perturbations have to be discrete.
Moreover, text perturbations are mainly replacement operations. This poses an
important research question on how to characterize good replacements?. Hence,
an active area of research considers identifying and quantifying new aspects
of text similarity. Similar to attack methods, continuous input defense
techniques cannot be easily applied. As an example, the GAN approach is based
on adding artificial noise to the inputs. Thus, it is not applicable in an NLP
context. Accordingly, an interesting research area is how to enable the off-
the-shelf adversarial generation and defense methods of continuous variables
to be used in a textual context.

  * •

Discrete data- perceivability: While small arbitrary image perturbations are
typically imperceptible to the human eye, humans can easily detect text
modifications. This poses certain limitations on what makes a text
modification imperceptible to human understanding. Recent research highlights
grammatical, semantic, and context correctness [62] for viable adversarial
text replacements. Still, imposing such strict constraints is known to
restrict the space of possible replacements, thereby degrading the strength of
the generative adversarial examples. Given this trade-off, there is an immense
need for more research on characterizing what defines an imperceptible, yet
efficient adversarial example.

  * •

Human-factor dependency: As seen in the papers surveyed in this section, there
is almost always a human effort in the design and optimization of adversarial
attack and defense methodologies. This is especially the case for measuring
the imperceptibility of adversarial replacements from a linguistic point of
view [74]. Other human factor example aspects include concerns with setting
the parameters of the attack model and optimizing such parameters. Therefore,
it seems interesting to invest more research on how to automate these
processes in an attempt towards eliminating or, at least, regulating the
extent to which the human factor is necessary for this area [82]. In fact,
this is a general research challenge that concerns ML in general, including
the context of this section.

  * •

Transferability- of attack and defense: Similar to the case in computer
vision, adversarial examples in an NLP context are known to possess different
aspects and extensions of transferability. Typically, they transfer across
different models as well as training and testing datasets. Furthermore, recent
research envisions focused on developing attacks that transfer even across
different ML tasks, as well [172]. The transferability character exempts an
attacker from the need to use the same model, architecture, data, or even a
task of the attacked ML model [72]. This resembles a growing challenge against
efficient defense techniques [40]. As intuitively expected, black-box and
untargeted adversarial attacks have higher degrees of transferability as
compared to white-box and/or targeted attacks. Therefore, there will be an
increased demand for developing advanced defense techniques that work with
this challenge. It is noted that transferability can be treated at the
character, word, and sentence levels.

  * •

Universality- of attack and defense: This property refers to the extent to
which an adversary is independent of the input, either status of nature. Thus,
in an NLP context, the universality of an adversary can be seen in how it can
attack irrespective of the input. For example, an NLP attack can be
independent of the language used. Another aspect is defense universality; the
robustness against character-, word-, and sentence-level attacks at the same
time. The incorporation of this property in adversarial attack and defense is
an interesting area of research. Recent research signifies that the success of
a defense strategy against a specific adversarial attack may not necessarily
mean the system is robust against other adversaries. Therefore, developing
universal defense mechanisms is of growing importance. This is especially the
case with the state-of-the-art deep learning models that can learn the task
irrespective of inputs or architecture. Besides, it is also worth
investigating the relationship between efficient defenses against different
attacks, and then the attacker-attacker relationship holds between their
respective defenders.

  * •

Lack of appropriate defense: An area worth investigating is whether defense
methods have the transferability property, and if so, in what sense? and, more
importantly, how to enhance and boost this transferability to have a generic
defense approach?

Improved detection of grammatical errors is worth a further investigation [72,
66]. This will have the added benefit of decreasing the chances of adversarial
attacks.

The development of more robust NLP architectures: Recent research works hint
at a promising research outlook, which is on how to design inherently
robust/defensive ML architectures that can suppress the impact of any
adversarial attack [59].

  * •

Embedding space size-perturbation selection trade-offs: Several works on
characterizing a suitable embedding space have been proposed. Besides, there
is a variety of methods to search for the best replacement amongst the
elements in the embedding space. Still, there is a need for more work on how
to define the space, and how to efficiently search between its elements
approaching the best replacement. This requires also balancing the trade-off
between the embedding space size and the search computational burden.
Moreover, there is a need for novel ideas on how to regularize and guide the
search process within the embedding space. A few works have considered the
question of identifying critical characters and words that are best to change
amongst all others. However, there is still an evident need for more research
on this idea. In this context, one may think of developing a unified framework
for answering a more common question of what, where, when, and how to change,
i.e., is it better to change a character, as (sub) word or a (sub) sentence?
This opens up the horizon for developing new optimization settings that may
prove to answer such questions in a systematic or tractable way.

  * •

Lack of standards in benchmark datasets with proper collection of adversarial
examples or instances : Given the fact that there is a variety of datasets
used in training, testing, and performance evaluation, there is a need for
standardizing benchmark datasets. Although several large-scale datasets have
been collected for common NLP tasks in recent years, there remains a need for
new datasets for more challenging social networks NLP applications. It is
noted that this is a general concern in text and NLP adversarial attack
research areas.

  * •

Lack of standards quality metrics: With the diversity of attack and defense
methods, there is a corresponding diversity of employed attack and defense
quality metrics. A classical attack quality metric is the accuracy of the
targeted ML model. Furthermore, in an NLP context, the semantic similarity
between the original input and its adversarial replacement has also been
widely used as a quality metric. Along this line, researchers have used Blue,
Self-Blue, EmbSim, Euclidean distance, cosine, and semantic similarity to
evaluate their attack methods or the quality of a generated text. On the other
hand, human evaluation seems to be an unavoidable measure of imperceptibility
[74]. To this end, there is an immense need for research on developing
descriptive, fair, and standardized performance evaluation metrics for the
quality of the attack and the defense. Moreover, other quality metrics
evaluating other aspects of the attack, such as transferability, universality,
and imperceptibility need also to be developed and evaluated. From an
adversarial replacement generation perspective, there is a gowning need for
standardizing measures and rules for semantic similarity. This is of great
importance for ensuring the imperceptibility of attacks. Moreover, there is a
need for developing and standardizing measures for the ML model’s stability
against adversarial attacks. Along the line of quality metrics, [88] recently
showed the added benefit of incorporating the attack’s meaning preservation
capability into quality metrics for attack performance evaluation. Also, a
core question is on defining the distance between the original text and its
adversarial modification, i.e., on how to measure a text change?

  * •

Computational complexity concerns: Either on the attack or defense sides,
there is an inherent trade-off between the performance and computational
complexity, just like any other application area. It can be seen that settings
with more access to system parameters are better in performance but at the
cost of more computational complexity. With The increase of the attack and
defense degrees of complexity, there should be an accompanying research effort
on how to implement such techniques at affordable computational complexity
levels, execution time, and memory requirements. Similar to several
aforementioned research challenges, computational complexity is a general
concern in ML in general, including the context of this section.

  * •

Balance between protection against attacks, censorship, and freedom of speech:
Social media outlets and feeds from these outlets can be attacked with groups
with radical views to promote their views. Thus, the social network
applications are more susceptible to attacks and mitigation techniques are
hard to integrate with social platforms as that might be seen as an attack on
“freedom of speech.” It is important to maintain the balance between
protection against attacks, censorship, and freedom of speech.

  * •

Graph Neural Networks: Graphs representation has been receiving a lot of
attention as a key framework for representing entities with relations. In the
context of a social media network, users, user information, and relationships
between users can be represented by graph nodes, node attributes, and graph
edges, respectively. Besides, graph-level operations represent operations on
the level of a social media network. As a typical example, predicting the link
between two graph nodes corresponds to predicting a friendship or following
relationship between two members on a social media platform [173]. As another
example, estimating whether a node belongs to a subgroup corresponds to
estimating whether a user belongs to a social media group.

The primary added benefit of a graph representation is rendering the inherent
links between nodes. Classical methods such as random walk can preserve these
links. Nevertheless, the success of deep neural networks in various fields has
led to the development of graph neural networks (GNNS), a special type of
neural network well-suited for embracing node links in their representation
across the network layers. GNNs have been receiving increasing attention in
recent years in the representation of data with relationships. A key usage of
GNNs is in the representation of social media networks. This elegant
representation has tweaked downstream tasks such as node and graph
classification and link prediction.

Despite their success in modeling graph data, GNNs have been shown vulnerable
to adversarial attacks [174]. In this context, imperceptible perturbations
applied at the node, edge, graph, and attribute level are shown to trick the
GCN. Therefore, reaping the promising potential of GNNs in representing social
media networks requires improving their robustness to such attacks and
preserving the privacy of the data they represent. Here is a summary of key
research challenges facing the security and privacy of GNNs in the context of
social media platforms.

    * –

Addressing the dynamic nature of social media network graphs: The majority of
existing research considers static graphs with node attributes [175]. Still,
social media networks are continuously dynamic [176]. For example, a person
may make new friendships or terminate existing ones with certain persons.
Also, a new indecent may happen and attract the interest of people all of a
sudden, such as the so-called trending events. Thus, this dynamic nature needs
to be modeled well in GCNs and taken care of in the underlying design of
adversarial attack and defense techniques.

    * –

Imperceptibility-definition and quantification: There is a need for defining
to which extent a given perturbation is imperceptible. This requires
developing novel imperceptibility metrics.

    * –

Identifying what makes a good perturbation: An interesting research direction
is to study the commonality between good attacks attained by the existing
techniques. This can hint at how to systematically design good attacks. This
will also be reflected on how to exploit this knowledge in designing efficient
defense measures.

    * –

Extension of knowledge from other domains: A possible research direction is to
consider extending the existing techniques for attack and defense in text and
NLP domains to the graph domain.

    * –

Scalability of attack and defense methods: there is a need for developing
effective attack and defense methods at a small scale. Then, they can be
easily scaled up to the level of the entire network.

    * –

Mitigating privacy leakage in GCNs: Social media networks typically contain
private data that should be kept away from the reach of the public, as well as
publically published data; that the persons and platforms share publically at
no harm. To this end, privacy leakage happens when public information can be
exploited to infer private data. Examples along this line include revealing
private links between persons based on public information [177], and even re-
identifying anonymized persons in a social media network based on public
information [178]. In fact, a series of recent works accuse GCNs to leak
private data, especially in the context of social media [179, 180]. The main
reason for this leakage is the correlation between data of different users in
a social GNN [181]. Thus, an important future research venue is to mitigate
this leakage. Examples along this line include leveraging differential privacy
and developing new GCN architectures to minimize this leakage.

Fig. 6 summarizes the outstanding challenges in the areas of attack and
defense, along with the underlying research trends; concurrent and future
ones.

Figure 6: A summary of the main outstanding challenges in text adversarial
attacks and defense research, along with relevant research trends and
recommendations. Challenges are represented with rectangles, whereas research
trends and recommendations are in oval shapes.

###  6.2 Discussion and Lessons Learned

  * •

Adversarial ML for social media NLP applications is a very important and
growing research area as the stake is very high. For example, it allows
influencing public opinions by hackers/attackers are typically from state-
sponsor agencies.

  * •

Textual content in social media is very noisy, Due to such diverse
characteristics, NLP in general and adversarial NLP, in particular, is more
challenging. Developing defense techniques is even harder.

  * •

Online social media applications are more prone to adversarial attacks than
conventional sources of text.

  * •

Social media applications are more susceptible to attacks and mitigation
techniques are hard to integrate with social platforms as that might be seen
as an attack on “freedom of speech.” So there is a delicate balance between
protection against attacks, censorship, and freedom of speech.

  * •

The widespread use of social media attracts the attackers to launch different
types of adversarial attacks on these networks to fulfill their objectives.

  * •

Social media outlets are distributed in nature, therefore, distributed attacks
are easier to manifest in social media applications.

  * •

A successful attack against the most vulnerable social media outlets might be
enough to have unintended consequences (e.g., promote extreme views, etc.).

  * •

The unstructured nature of text shared in social networks allows attackers to
launch different types of adversarial attacks on the applications.

  * •

The literature indicates that several interesting NLP applications of social
networks are subject to adversarial attacks. Some of the notable applications
include rumors, satires, parodies, clickbait, spam, hate speech, and
misinformation detection.

  * •

Recent literature necessities the need for developing efficient adversarial
defense techniques.

  * •

There is a need for reducing human dependency both in adversarial attack and
defense

  * •

The existing literature on adversarial NLP in general and in social media
applications in particular lacks in focused benchmark datasets and quality
metrics.

  * •

Graph neural networks form a promising framework with a great potential for
improving the representation of social media information. however, this
requires improving their security against adversarial attacks and mitigating
their privacy leakage.

##  7 Conclusions

We surveyed the state-of-the-art in adversarial attack and defense on five
major text-based social media applications, which include rumors, satires,
clickbait & spam, hate speech, misinformation detection, and sentiment. These
applications are susceptible to adversarial attacks. In this paper, we first
provide a general overview of adversarial attack and defense techniques
applicable to both text and image applications. As text is a primary
communication means in social media platforms, we then review state-of-the-art
techniques for attack and defense techniques specifically applicable for NLP.
After that, we discuss the attack and defense studies in the aforementioned
five major social media applications. Finally, we highlight current research
challenges in the context of security of ML for social media applications and
state key lessons.

## Acknowledgment

The authors extend their appreciation to the Deputyship for Research &
Innovation, Ministry of Education in Saudi Arabia for funding this research
work through the project number 1120.

## References

  * [1] K. Ahmad, K. Pogorelov, M. Riegler, N. Conci, P. Halvorsen, Social media and satellites, Multimedia Tools and Applications 78 (3) (2019) 2837–2875.
  * [2] S. Bhanot, Use of social media by companies to reach their customers., SIES Journal of Management 8 (1).
  * [3] B. Liu, Sentiment analysis and opinion mining, Synthesis lectures on human language technologies 5 (1) (2012) 1–167.
  * [4] J. K. Rout, K.-K. R. Choo, A. K. Dash, S. Bakshi, S. K. Jena, K. L. Williams, A model for sentiment and emotion analysis of unstructured social media text, Electronic Commerce Research 18 (1) (2018) 181–199.
  * [5] S. Z. Hassan, K. Ahmad, A. Al-Fuqaha, N. Conci, Sentiment analysis from images of natural disasters, in: International Conference on Image Analysis and Processing, Springer, 2019, pp. 104–113.
  * [6] J. Khan, S. Lee, Implicit user trust modeling based on user attributes and behavior in online social networks, IEEE Access 7 (2019) 142826–142842.
  * [7] Y. Gorodnichenko, T. Pham, O. Talavera, Social media, sentiment and public opinions: Evidence from #brexit and #uselection, European Economic Review 136 (2021) 103772.  doi:https://doi.org/10.1016/j.euroecorev.2021.103772.
URL https://www.sciencedirect.com/science/article/pii/S0014292121001252

  * [8] L. Wu, F. Morstatter, K. M. Carley, H. Liu, Misinformation in social media: definition, manipulation, and detection, ACM SIGKDD Explorations Newsletter 21 (2) (2019) 80–90.
  * [9] F. Alam, S. Shaar, F. Dalvi, H. Sajjad, A. Nikolov, H. Mubarak, G. D. S. Martino, A. Abdelali, N. Durrani, K. Darwish, A. Al-Homaid, W. Zaghouani, T. Caselli, G. Danoe, F. Stolk, B. Bruntink, P. Nakov, Fighting the COVID-19 infodemic: Modeling the perspective of journalists, fact-checkers, social media platforms, policy makers, and the society, in: Findings of EMNLP 2021, 2021.
  * [10] N. Chetty, S. Alathur, Hate speech review in the context of online social networks, Aggression and violent behavior 40 (2018) 108–118.
  * [11] A. Matamoros-Fernández, J. Farkas, Racism, hate speech, and social media: A systematic review and critique, Television & New Media 22 (2) (2021) 205–224.
  * [12] M. Hasanuzzaman, G. Dias, A. Way, Demographic word embeddings for racism detection on Twitter, in: Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Asian Federation of Natural Language Processing, Taipei, Taiwan, 2017, pp. 926–936.
URL https://aclanthology.org/I17-1093

  * [13] O. Onabola, Z. Ma, X. Yang, B. Akera, I. Abdulrahman, J. Xue, D. Liu, Y. Bengio, hBERT+ BiasCorp-Fighting racism on the web, in: Proceedings of the First Workshop on Language Technology for Equality, Diversity and Inclusion, 2021, pp. 26–33.
  * [14] F. M. P. del Arco, M. D. Molina-González, M. T. Martín-Valdivia, L. A. U. Lopez, Sinai at semeval-2019 task 5: Ensemble learning to detect hate speech against inmigrants and women in english and spanish tweets, in: Proceedings of the 13th International Workshop on Semantic Evaluation, 2019, pp. 476–479.
  * [15] B. Vidgen, S. Hale, E. Guest, H. Margetts, D. Broniatowski, Z. Waseem, A. Botelho, M. Hall, R. Tromble, Detecting east asian prejudice on social media, in: Proceedings of the Fourth Workshop on Online Abuse and Harms, 2020, pp. 162–172.
  * [16] T. Farnsworth, China and russia submit cyber proposal, Arms Control Today 41 (9) (2011) 35.
  * [17] D. Lazer, A. S. Pentland, L. Adamic, S. Aral, A. L. Barabasi, D. Brewer, N. Christakis, N. Contractor, J. Fowler, M. Gutmann, et al., Life in the network: the coming age of computational social science, Science (New York, NY) 323 (5915) (2009) 721.
  * [18] R. Conte, N. Gilbert, G. Bonelli, C. Cioffi-Revilla, G. Deffuant, J. Kertesz, V. Loreto, S. Moat, J.-P. Nadal, A. Sanchez, et al., Manifesto of computational social science, The European Physical Journal Special Topics 214 (1) (2012) 325–346.
  * [19] L.-W. Ku, C.-T. Li (Eds.), Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media, Association for Computational Linguistics, Online, 2021.
URL https://aclanthology.org/2021.socialnlp-1.0

  * [20] L.-W. Ku, C.-T. Li (Eds.), Proceedings of the Eighth International Workshop on Natural Language Processing for Social Media, Association for Computational Linguistics, Online, 2020.
URL https://aclanthology.org/2020.socialnlp-1.0

  * [21] S. T. Roberts, J. Tetreault, V. Prabhakaran, Z. Waseem (Eds.), Proceedings of the Third Workshop on Abusive Language Online, Association for Computational Linguistics, Florence, Italy, 2019.
URL https://aclanthology.org/W19-3500

  * [22] S. Akiwowo, B. Vidgen, V. Prabhakaran, Z. Waseem (Eds.), Proceedings of the Fourth Workshop on Online Abuse and Harms, Association for Computational Linguistics, Online, 2020.
URL https://aclanthology.org/2020.alw-1.0

  * [23] S. Shaar, F. Alam, G. Da San Martino, A. Nikolov, W. Zaghouani, P. Nakov, A. Feldman, Findings of the NLP4IF-2021 shared tasks on fighting the COVID-19 infodemic and censorship detection, in: Proceedings of the Fourth Workshop on NLP for Internet Freedom: Censorship, Disinformation, and Propaganda, 2021, pp. 82–92.
  * [24] A. Barrón-Cedeno, T. Elsayed, P. Nakov, G. Da San Martino, M. Hasanain, R. Suwaileh, F. Haouari, N. Babulkov, B. Hamdan, A. Nikolov, S. Shaar, Z. Sheikh Ali, Overview of checkthat! 2020: Automatic identification and verification of claims in social media, in: International Conference of the Cross-Language Evaluation Forum for European Languages, Springer, 2020, pp. 215–236.
  * [25] S. Shaar, A. Nikolov, N. Babulkov, F. Alam, A. Barrón-Cedeño, T. Elsayed, M. Hasanain, R. Suwaileh, F. Haouari, G. Da San Martino, P. Nakov, Overview of CheckThat! 2020 English: Automatic identification and verification of claims in social media, in: Working Notes of CLEF 2020—Conference and Labs of the Evaluation Forum, CEUR Workshop Proceedings, CEUR-WS.org, 2020.
  * [26] P. Nakov, G. Da San Martino, T. Elsayed, A. Barrón-Cedeño, R. Míguez, S. Shaar, F. Alam, F. Haouari, M. Hasanain, W. Mansour, B. Hamdan, Z. S. Ali, N. Babulkov, A. Nikolov, G. K. Shahi, J. M. Struß, T. Mandl, M. Kutlu, Y. S. Kartal, Overview of the CLEF-2021 CheckThat! lab on detecting check-worthy claims, previously fact-checked claims, and fake news, CLEF 2021, Springer, 2021.
  * [27] D. Dimitrov, B. Bin Ali, S. Shaar, F. Alam, F. Silvestri, H. Firooz, P. Nakov, G. Da San Martino, Task 6 at SemEval-2021: Detection of persuasion techniques in texts and images, in: Proceedings of the 15th International Workshop on Semantic Evaluation, SemEval ’21, 2021, pp. 70–98.
  * [28] M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, R. Kumar, SemEval-2019 task 6: Identifying and categorizing offensive language in social media (OffensEval), in: Proceedings of the 13th International Workshop on Semantic Evaluation, Association for Computational Linguistics, Minneapolis, Minnesota, USA, 2019, pp. 75–86.  doi:10.18653/v1/S19-2010.
URL https://aclanthology.org/S19-2010

  * [29] M. Zampieri, P. Nakov, S. Rosenthal, P. Atanasova, G. Karadzhov, H. Mubarak, L. Derczynski, Z. Pitenis, Ç. Çöltekin, SemEval-2020 task 12: Multilingual offensive language identification in social media (OffensEval 2020), in: Proceedings of the Fourteenth Workshop on Semantic Evaluation, International Committee for Computational Linguistics, Barcelona (online), 2020, pp. 1425–1447.
URL https://aclanthology.org/2020.semeval-1.188

  * [30] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, R. Fergus, Intriguing properties of neural networks, arXiv preprint arXiv:1312.6199.
  * [31] I. J. Goodfellow, J. Shlens, C. Szegedy, Explaining and harnessing adversarial examples, arXiv preprint arXiv:1412.6572.
  * [32] S.-M. Moosavi-Dezfooli, A. Fawzi, P. Frossard, Deepfool: a simple and accurate method to fool deep neural networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2574–2582.
  * [33] A. Nguyen, J. Yosinski, J. Clune, Deep neural networks are easily fooled: High confidence predictions for unrecognizable images, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 427–436.
  * [34] N. Papernot, P. McDaniel, X. Wu, S. Jha, A. Swami, Distillation as a defense to adversarial perturbations against deep neural networks, in: 2016 IEEE symposium on security and privacy (SP), IEEE, 2016, pp. 582–597.
  * [35] K. Grosse, N. Papernot, P. Manoharan, M. Backes, P. McDaniel, Adversarial examples for malware detection, in: European symposium on research in computer security, Springer, 2017, pp. 62–79.
  * [36] O. Suciu, S. E. Coull, J. Johns, Exploring adversarial examples in malware detection, in: 2019 IEEE Security and Privacy Workshops (SPW), IEEE, 2019, pp. 8–14.
  * [37] M. Melis, A. Demontis, B. Biggio, G. Brown, G. Fumera, F. Roli, Is deep learning safe for robot vision? adversarial examples against the icub humanoid, in: Proceedings of the IEEE International Conference on Computer Vision Workshops, 2017, pp. 751–759.
  * [38] S. G. Finlayson, J. D. Bowers, J. Ito, J. L. Zittrain, A. L. Beam, I. S. Kohane, Adversarial attacks on medical machine learning, Science 363 (6433) (2019) 1287–1289.
  * [39] S. G. Finlayson, H. W. Chung, I. S. Kohane, A. L. Beam, Adversarial attacks against medical deep learning systems, arXiv preprint arXiv:1804.05296.
  * [40] T. Le, S. Wang, D. Lee, Malcom: Generating malicious comments to attack neural fake news detection models, arXiv preprint arXiv:2009.01048.
  * [41] J. Ma, W. Gao, K.-F. Wong, Detect rumors on twitter by promoting information campaigns with generative adversarial learning, in: The World Wide Web Conference, WWW ’19, Association for Computing Machinery, New York, NY, USA, 2019, p. 3049–3055.  doi:10.1145/3308558.3313741.
URL https://doi.org/10.1145/3308558.3313741

  * [42] Microsoft exec apologizes for tay chatbot’s racist tweets, says users ‘exploited a vulnerability, https://tinyurl.com/4mhaywr2, accessed: 2021-10-20.
  * [43] S. Jenni, P. Favaro, On stabilizing generative adversarial training with noise, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 12145–12153.
  * [44] B. Vivek, A. Baburaj, R. V. Babu, Regularizer to mitigate gradient masking effect during single-step adversarial training., in: CVPR Workshops, 2019, pp. 66–73.
  * [45] D. Wang, C. Li, S. Wen, S. Nepal, Y. Xiang, Defending against adversarial attack towards deep neural networks via collaborative multi-task training, IEEE Transactions on Dependable and Secure Computing.
  * [46] F. Tramèr, A. Kurakin, N. Papernot, I. Goodfellow, D. Boneh, P. McDaniel, Ensemble adversarial training: Attacks and defenses, arXiv preprint arXiv:1705.07204.
  * [47] W. Xu, D. Evans, Y. Qi, Feature squeezing: Detecting adversarial examples in deep neural networks, arXiv preprint arXiv:1704.01155.
  * [48] Y. Belinkov, J. Glass, Analysis methods in neural language processing: A survey, Transactions of the Association for Computational Linguistics 7 (2019) 49–72.
  * [49] W. Wang, R. Wang, L. Wang, Z. Wang, A. Ye, Towards a robust deep neural network in texts: A survey, arXiv preprint arXiv:1902.07285.
  * [50] W. E. Zhang, Q. Z. Sheng, A. Alhazmi, C. Li, Adversarial attacks on deep-learning models in natural language processing: A survey, ACM Transactions on Intelligent Systems and Technology (TIST) 11 (3) (2020) 1–41.
  * [51] H. Xu, Y. Ma, H.-C. Liu, D. Deb, H. Liu, J.-L. Tang, A. K. Jain, Adversarial attacks and defenses in images, graphs and text: A review, International Journal of Automation and Computing 17 (2) (2020) 151–178.
  * [52] T. Gröndahl, L. Pajola, M. Juuti, M. Conti, N. Asokan, All you need is” love” evading hate speech detection, in: Proceedings of the 11th ACM workshop on artificial intelligence and security, 2018, pp. 2–12.
  * [53] M. Moh, T.-S. Moh, B. Khieu, No” love” lost: Defending hate speech detection models against adversaries, in: 2020 14th International Conference on Ubiquitous Information Management and Communication (IMCOM), IEEE, 2020, pp. 1–6.
  * [54] B. T. Khieu, Tsar: A system for defending hate speech detection models against adversaries.
  * [55] A. Farzindar, D. Inkpen, Natural language processing for social media, Synthesis Lectures on Human Language Technologies 8 (2) (2015) 1–166.
  * [56] J. Ebrahimi, A. Rao, D. Lowd, D. Dou, Hotflip: White-box adversarial examples for text classification, arXiv preprint arXiv:1712.06751.
  * [57] J. Gao, J. Lanchantin, M. L. Soffa, Y. Qi, Black-box generation of adversarial text sequences to evade deep learning classifiers, in: 2018 IEEE Security and Privacy Workshops (SPW), IEEE, 2018, pp. 50–56.
  * [58] J. Li, S. Ji, T. Du, B. Li, T. Wang, Textbugger: Generating adversarial text against real-world applications, arXiv preprint arXiv:1812.05271.
  * [59] Y. Belinkov, Y. Bisk, Synthetic and natural noise both break neural machine translation, arXiv preprint arXiv:1711.02173.
  * [60] H. Hosseini, S. Kannan, B. Zhang, R. Poovendran, Deceiving google’s perspective api built for detecting toxic comments, arXiv preprint arXiv:1702.08138.
  * [61] S. Brown, P. Milkov, S. Patel, Y. Z. Looi, Z. Dong, H. Gu, N. S. Artan, E. Jain, Acoustic and visual approaches to adversarial text generation for google perspective, in: 2019 International Conference on Computational Science and Computational Intelligence (CSCI), IEEE, 2019, pp. 355–360.
  * [62] D. Pruthi, B. Dhingra, Z. C. Lipton, Combating adversarial misspellings with robust word recognition, arXiv preprint arXiv:1905.11268.
  * [63] S. Eger, G. G. Şahin, A. Rücklé, J.-U. Lee, C. Schulz, M. Mesgar, K. Swarnkar, E. Simpson, I. Gurevych, Text processing like humans do: Visually attacking and shielding nlp systems, arXiv preprint arXiv:1903.11508.
  * [64] X. Zhang, J. Zhao, Y. LeCun, Character-level convolutional networks for text classification, Advances in neural information processing systems 28 (2015) 649–657.
  * [65] V. Kuleshov, S. Thakoor, T. Lau, S. Ermon, Adversarial examples for natural language classification problems (2018).
  * [66] D. Jin, Z. Jin, J. T. Zhou, P. Szolovits, Is bert really robust? a strong baseline for natural language attack on text classification and entailment, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, 2020, pp. 8018–8025.
  * [67] Y. Zang, F. Qi, C. Yang, Z. Liu, M. Zhang, Q. Liu, M. Sun, Word-level textual adversarial attacking as combinatorial optimization, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 6066–6080.
  * [68] M. Sato, J. Suzuki, H. Shindo, Y. Matsumoto, Interpretable adversarial perturbation in input embedding space for text, arXiv preprint arXiv:1805.02917.
  * [69] Y. Zhou, J.-Y. Jiang, K.-W. Chang, W. Wang, Learning to discriminate perturbations for blocking adversarial attacks in text classification, arXiv preprint arXiv:1909.03084.
  * [70] S. Ren, Y. Deng, K. He, W. Che, Generating natural language adversarial examples through probability weighted word saliency, in: Proceedings of the 57th annual meeting of the association for computational linguistics, 2019, pp. 1085–1097.
  * [71] P. Yang, J. Chen, C.-J. Hsieh, J.-L. Wang, M. I. Jordan, Greedy attack and gumbel attack: Generating adversarial examples for discrete data., Journal of Machine Learning Research 21 (43) (2020) 1–36.
  * [72] E. Wallace, S. Feng, N. Kandpal, M. Gardner, S. Singh, Universal adversarial triggers for attacking and analyzing nlp, arXiv preprint arXiv:1908.07125.
  * [73] S. Garg, G. Ramakrishnan, Bae: Bert-based adversarial examples for text classification, arXiv preprint arXiv:2004.01970.
  * [74] M. T. Ribeiro, S. Singh, C. Guestrin, Semantically equivalent adversarial rules for debugging nlp models, in: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018, pp. 856–865.
  * [75] M. Alzantot, Y. Sharma, S. Chakraborty, H. Zhang, C.-J. Hsieh, M. B. Srivastava, Genattack: Practical black-box attacks with gradient-free optimization, in: Proceedings of the Genetic and Evolutionary Computation Conference, 2019, pp. 1111–1119.
  * [76] J. Li, W. Monroe, D. Jurafsky, Understanding neural networks through representation erasure, arXiv preprint arXiv:1612.08220.
  * [77] Q. Lei, L. Wu, P.-Y. Chen, A. G. Dimakis, I. S. Dhillon, M. Witbrock, Discrete adversarial attacks and submodular optimization with applications to text classification, arXiv preprint arXiv:1812.00151.
  * [78] J. Wang, H. Zhang, Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks, in: Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 6629–6638.
  * [79] Y. Wang, H. Lv, X. Kuang, G. Zhao, Y.-a. Tan, Q. Zhang, J. Hu, Towards a physical-world adversarial patch for blinding object detection models, Information Sciences.
  * [80] Z. Gong, W. Wang, B. Li, D. Song, W.-S. Ku, Adversarial texts with gradient methods, arXiv preprint arXiv:1801.07175.
  * [81] M. Alzantot, Y. Sharma, A. Elgohary, B.-J. Ho, M. Srivastava, K.-W. Chang, Generating natural language adversarial examples, arXiv preprint arXiv:1804.07998.
  * [82] B. Liang, H. Li, M. Su, P. Bian, X. Li, W. Shi, Deep text classification can be fooled, arXiv preprint arXiv:1704.08006.
  * [83] J. Ebrahimi, D. Lowd, D. Dou, On adversarial examples for character-level neural machine translation, arXiv preprint arXiv:1806.09030.
  * [84] M. Iyyer, J. Wieting, K. Gimpel, L. Zettlemoyer, Adversarial example generation with syntactically controlled paraphrase networks, arXiv preprint arXiv:1804.06059.
  * [85] R. Jia, P. Liang, Adversarial examples for evaluating reading comprehension systems, arXiv preprint arXiv:1707.07328.
  * [86] Y. Cheng, L. Jiang, W. Macherey, Robust neural machine translation with doubly adversarial inputs, arXiv preprint arXiv:1906.02443.
  * [87] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, in: Advances in neural information processing systems, 2017, pp. 5998–6008.
  * [88] P. Michel, X. Li, G. Neubig, J. M. Pino, On evaluation of adversarial perturbations for sequence-to-sequence models, arXiv preprint arXiv:1903.06620.
  * [89] M. Jethanandani, D. Tang, Adversarial attacks against lipnet: End-to-end sentence level lipreading, in: 2020 IEEE Security and Privacy Workshops (SPW), IEEE, 2020, pp. 15–19.
  * [90] H. Zheng, Z. Zhang, H. Lee, A. Prakash, Understanding and diagnosing vulnerability under adversarial attacks, arXiv preprint arXiv:2007.08716.
  * [91] M. Blohm, G. Jagfeld, E. Sood, X. Yu, N. T. Vu, Comparing attention-based convolutional and recurrent neural networks: Success and limitations in machine reading comprehension, arXiv preprint arXiv:1808.08744.
  * [92] P. Benz, C. Zhang, A. Karjauv, I. S. Kweon, Universal adversarial training with class-wise perturbations, in: 2021 IEEE International Conference on Multimedia and Expo (ICME), IEEE, 2021, pp. 1–6.
  * [93] H. Zhang, H. Chen, Z. Song, D. Boning, I. S. Dhillon, C.-J. Hsieh, The limitations of adversarial training and the blind-spot attack, arXiv preprint arXiv:1901.04684.
  * [94] Z. Liu, F. Wang, Z. Lin, L. Wang, Z. Yin, De-co: A two-step spelling correction model for combating adversarial typos, in: 2020 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom), IEEE, 2020, pp. 554–561.
  * [95] R. Bao, J. Wang, H. Zhao, Defending pre-trained language models from adversarial word substitutions without performance sacrifice, arXiv preprint arXiv:2105.14553.
  * [96] H. Zhang, Z. Hu, W. Qin, M. Xu, M. Wang, Adversarial co-distillation learning for image recognition, Pattern Recognition 111 (2021) 107659.
  * [97] A. Zubiaga, A. Aker, K. Bontcheva, M. Liakata, R. Procter, Detection and resolution of rumours in social media: A survey, ACM Comput. Surv. 51 (2).  doi:10.1145/3161603.
URL https://doi.org/10.1145/3161603

  * [98] J. Cao, J. Guo, X. Li, Z. Jin, H. Guo, J. Li, Automatic rumor detection on microblogs: A survey, arXiv preprint arXiv:1807.03505.
  * [99] G. W. Allport, L. Postman, The psychology of rumor.
  * [100] K. Wu, S. Yang, K. Q. Zhu, False rumors detection on sina weibo by propagation structures, in: 2015 IEEE 31st international conference on data engineering, IEEE, 2015, pp. 651–662.
  * [101] C. Castillo, M. Mendoza, B. Poblete, Information credibility on twitter, in: Proceedings of the 20th international conference on World wide web, 2011, pp. 675–684.
  * [102] X. Yang, Y. Lyu, T. Tian, Y. Liu, Y. Liu, X. Zhang, Rumor detection on social media with graph structured adversarial learning, in: C. Bessiere (Ed.), Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, International Joint Conferences on Artificial Intelligence Organization, 2020, pp. 1417–1423, main track.  doi:10.24963/ijcai.2020/197.
URL https://doi.org/10.24963/ijcai.2020/197

  * [103] Y.-Z. Song, Y.-S. Chen, Y.-T. Chang, S.-Y. Weng, H.-H. Shuai, Adversary-aware rumor detection, in: ACL-IJCNLP: Findings, 2021.
  * [104] J. Ma, W. Gao, P. Mitra, S. Kwon, B. J. Jansen, K.-F. Wong, M. Cha, Detecting rumors from microblogs with recurrent neural networks, in: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI’16, AAAI Press, 2016, p. 3818–3824.
  * [105] A. Zubiaga, M. Liakata, R. Procter, Exploiting context for rumour detection in social media, in: G. L. Ciampaglia, A. Mashhadi, T. Yasseri (Eds.), Social Informatics, Springer International Publishing, Cham, 2017, pp. 109–123.
  * [106] C. Song, C. Yang, H. Chen, C. Tu, Z. Liu, M. Sun, Ced: credible early detection of social media rumors, IEEE Transactions on Knowledge and Data Engineering.
  * [107] A. Zubiaga, M. Liakata, R. Procter, Learning reporting dynamics during breaking news for rumour detection in social media (2016).  arXiv:1610.07363.
  * [108] J. Ma, W. Gao, K.-F. Wong, Detect rumors in microblog posts using propagation structure via kernel learning, in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Vancouver, Canada, 2017, pp. 708–717.  doi:10.18653/v1/P17-1066.
URL https://aclanthology.org/P17-1066

  * [109] L. Li, O. Levi, P. Hosseini, D. A. Broniatowski, A multi-modal method for satire detection using textual and visual cues, arXiv preprint arXiv:2010.06671.
  * [110] R. McHardy, H. Adel, R. Klinger, Adversarial training for satire detection: Controlling for confounding variables, arXiv preprint arXiv:1902.11145.
  * [111] E. C. Tandoc Jr, Z. W. Lim, R. Ling, Defining “fake news” a typology of scholarly definitions, Digital journalism 6 (2) (2018) 137–153.
  * [112] J. Golbeck, M. Mauriello, B. Auxier, K. H. Bhanushali, C. Bonk, M. A. Bouzaghrane, C. Buntain, R. Chanduka, P. Cheakalos, J. B. Everett, et al., Fake news vs satire: A dataset and analysis, in: Proceedings of the 10th ACM Conference on Web Science, 2018, pp. 17–21.
  * [113] X. Zhou, R. Zafarani, A survey of fake news: Fundamental theories, detection methods, and opportunities, ACM Comput. Surv. 53 (5).  doi:10.1145/3395046.
URL https://doi.org/10.1145/3395046

  * [114] R. Garret, R. Bond, S. Poulsen, Too many people think satirical news is real, The Conversation.
  * [115] F. Yang, A. Mukherjee, E. Dragut, Satirical news detection and analysis using attention mechanism and linguistic features, in: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Copenhagen, Denmark, 2017, pp. 1979–1989.  doi:10.18653/v1/D17-1211.
URL https://aclanthology.org/D17-1211

  * [116] C. Burfoot, T. Baldwin, Automatic satire detection: Are you having a laugh?, in: Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, Association for Computational Linguistics, Suntec, Singapore, 2009, pp. 161–164.
URL https://aclanthology.org/P09-2041

  * [117] D. Goldwasser, X. Zhang, Understanding satirical articles using common-sense, Transactions of the Association for Computational Linguistics 4 (2016) 537–549.  doi:10.1162/tacl_a_00116.
URL https://aclanthology.org/Q16-1038

  * [118] V. Rubin, N. Conroy, Y. Chen, S. Cornwell, Fake news or truth? using satirical cues to detect potentially misleading news, in: Proceedings of the Second Workshop on Computational Approaches to Deception Detection, Association for Computational Linguistics, San Diego, California, 2016, pp. 7–17.  doi:10.18653/v1/W16-0802.
URL https://aclanthology.org/W16-0802

  * [119] S. De Sarkar, F. Yang, A. Mukherjee, Attending sentences to detect satirical fake news, in: Proceedings of the 27th International Conference on Computational Linguistics, Association for Computational Linguistics, Santa Fe, New Mexico, USA, 2018, pp. 3371–3380.
URL https://aclanthology.org/C18-1285

  * [120] P. Jachim, F. Sharevski, P. Treebridge, Trollhunter [evader]: Automated detection [evasion] of twitter trolls during the covid-19 pandemic, in: New Security Paradigms Workshop 2020, 2020, pp. 59–75.
  * [121] R. Tromble, S. C. McGregor, You break it, you buy it: The naiveté of social engineering in tech–and how to fix it, Political Communication 36 (2) (2019) 324–332.
  * [122] M. McKee, M. C. van Schalkwyk, D. Stuckler, The second information revolution: digitalization brings opportunities and concerns for public health, European journal of public health 29 (Supplement_3) (2019) 3–6.
  * [123] M. Cinelli, M. Conti, L. Finos, F. Grisolia, P. K. Novak, A. Peruzzi, M. Tesconi, F. Zollo, W. Quattrociocchi, (mis) information operations, Journal of Information Warfare 18 (3) (2019) 83–98.
  * [124] C. Tenove, H. Tworek, Online disinformation and harmful speech: Dangers for democratic participation and possible policy responses, Journal of Parliamentary & Political Law 13 (2019) 215–232.
  * [125] L. Yao, How artificial intelligence transforms cybersecurity.
  * [126] A. Uppal, V. Sachdeva, S. Sharma, Fake news detection using discourse segment structure analysis, in: 2020 10th International Conference on Cloud Computing, Data Science & Engineering (Confluence), IEEE, 2020, pp. 751–756.
  * [127] G. Gadek, V. Justine, J. Everwyn, Manipulation and fake news detection on social media: a two domain survey, combining social network analysis and knowledge bases exploitation.
  * [128] B. Biggio, I. Corona, D. Maiorca, B. Nelson, N. Šrndić, P. Laskov, G. Giacinto, F. Roli, Evasion attacks against machine learning at test time, in: Joint European conference on machine learning and knowledge discovery in databases, Springer, 2013, pp. 387–402.
  * [129] R. Chiesa, S. Ducci, S. Ciappi, Profiling hackers, Auerbach Publications, 2008.
  * [130] Y. Cabrera-León, P. G. Báez, C. P. Suárez-Araujo, E-mail spam filter based on unsupervised neural architectures and thematic categories: design and analysis, in: Computational Intelligence, Springer, 2019, pp. 239–262.
  * [131] B. D. Horne, J. Nørregaard, S. Adali, Robust fake news detection over time and attack, ACM Transactions on Intelligent Systems and Technology (TIST) 11 (1) (2019) 1–23.
  * [132] L. Zhao, Event prediction in the big data era: A systematic survey, arXiv preprint arXiv:2007.09815.
  * [133] Z. Zhou, H. Guan, M. M. Bhat, J. Hsu, Fake news detection via nlp is vulnerable to adversarial attacks, arXiv preprint arXiv:1901.09657.
  * [134] C. Koenders, J. Filla, N. Schneider, V. Woloszyn, How vulnerable are automatic fake news detection methods to adversarial attacks?, arXiv preprint arXiv:2107.07970.
  * [135] V. Woloszyn, F. Schaeffer, B. Boniatti, E. Cortes, S. Mohtaj, S. Möller, Untrue. news: A new search engine for fake stories, arXiv preprint arXiv:2002.06585.
  * [136] J. X. Morris, E. Lifland, J. Y. Yoo, J. Grigsby, D. Jin, Y. Qi, Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp, arXiv preprint arXiv:2005.05909.
  * [137] P. Kantartopoulos, N. Pitropakis, A. Mylonas, N. Kylilis, Exploring adversarial attacks and defences for fake twitter account detection, Technologies 8 (4) (2020) 64.
  * [138] B. C. Boatwright, D. L. Linvill, P. L. Warren, Troll factories: The internet research agency and state-sponsored agenda building, Resource Centre on Media Freedom in Europe (2018) 29.
  * [139] Z. Laub, Hate speech on social media: Global comparisons, Council on Foreign Relations 7.
  * [140] A. Schmidt, M. Wiegand, A survey on hate speech detection using natural language processing, in: Proceedings of the fifth international workshop on natural language processing for social media, 2017, pp. 1–10.
  * [141] M. Mozafari, R. Farahbakhsh, N. Crespi, A bert-based transfer learning approach for hate speech detection in online social media, in: International Conference on Complex Networks and Their Applications, Springer, 2019, pp. 928–940.
  * [142] R. Oak, Poster: Adversarial examples for hate speech classifiers, in: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, 2019, pp. 2621–2623.
  * [143] T. Tran, Y. Hu, C. Hu, K. Yen, F. Tan, K. Lee, S. Park, Habertor: An efficient and effective deep hatespeech detector, arXiv preprint arXiv:2010.08865.
  * [144] M. Xia, A. Field, Y. Tsvetkov, Demoting racial bias in hate speech detection, in: Proceedings of the Eighth International Workshop on Natural Language Processing for Social Media, 2020, pp. 7–14.
  * [145] T. Davidson, D. Warmsley, M. Macy, I. Weber, Automated hate speech detection and the problem of offensive language, in: Proceedings of the International AAAI Conference on Web and Social Media, Vol. 11, 2017.
  * [146] Y. Li, J. Gao, C. Meng, Q. Li, L. Su, B. Zhao, W. Fan, J. Han, A survey on truth discovery, SIGKDD Explor. Newsl. 17 (2) (2016) 1–16.
  * [147] K. Shu, A. Sliva, S. Wang, J. Tang, H. Liu, Fake news detection on social media: A data mining perspective, SIGKDD Explor. Newsl. 19 (1) (2017) 22–36.
  * [148] D. M. Lazer, M. A. Baum, Y. Benkler, A. J. Berinsky, K. M. Greenhill, F. Menczer, M. J. Metzger, B. Nyhan, G. Pennycook, D. Rothschild, M. Schudson, S. A. Sloman, C. R. Sunstein, E. A. Thorson, D. J. Watts, J. L. Zittrain, The science of fake news, Science 359 (6380) (2018) 1094–1096.
  * [149] S. Vosoughi, D. Roy, S. Aral, The spread of true and false news online, Science 359 (6380) (2018) 1146–1151.
  * [150] N. Vo, K. Lee, The rise of guardians: Fact-checking url recommendation to combat fake news, in: SIGIR, 2018, pp. 275–284.
  * [151] B. Guo, Y. Ding, L. Yao, Y. Liang, Z. Yu, The future of false information detection on social media: New perspectives and trends, ACM Computing Surveys (CSUR) 53 (4) (2020) 1–36.
  * [152] T. Miyato, A. M. Dai, I. Goodfellow, Adversarial training methods for semi-supervised text classification, arXiv preprint arXiv:1605.07725.
  * [153] C. Zhu, Y. Cheng, Z. Gan, S. Sun, T. Goldstein, J. Liu, Freelb: Enhanced adversarial training for language understanding, arXiv preprint arXiv:1909.11764.
  * [154] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, A. Vladu, Towards deep learning models resistant to adversarial attacks, arXiv preprint arXiv:1706.06083.
  * [155] A. Athalye, N. Carlini, D. Wagner, Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples, in: International conference on machine learning, PMLR, 2018, pp. 274–283.
  * [156] R. Zellers, A. Holtzman, H. Rashkin, Y. Bisk, A. Farhadi, F. Roesner, Y. Choi, Defending against neural fake news, arXiv preprint arXiv:1905.12616.
  * [157] T. Schuster, R. Schuster, D. J. Shah, R. Barzilay, The limitations of stylometry for detecting machine-generated fake news, Computational Linguistics 46 (2) (2020) 499–510.
  * [158] D. Zhang, Z. Yang, Word embedding perturbation for sentence classification, arXiv preprint arXiv:1804.08166.
  * [159] S. Kula, M. Choraś, R. Kozik, Application of the bert-based architecture in fake news detection, in: Computational Intelligence in Security for Information Systems Conference, Springer, 2019, pp. 239–249.
  * [160] S. Qiu, Q. Liu, S. Zhou, C. Wu, Review of artificial intelligence adversarial attack and defense technologies, Applied Sciences 9 (5) (2019) 909.
  * [161] K. Shu, D. Mahudeswaran, S. Wang, D. Lee, H. Liu, Fakenewsnet: A data repository with news content, social context and spatialtemporal information for studying fake news on social media, arXiv preprint arXiv:1809.01286.
  * [162] E. Kochkina, M. Liakata, A. Zubiaga, All-in-one: Multi-task learning for rumour verification, arXiv preprint arXiv:1806.03713.
  * [163] S. Z. Hassan, K. Ahmad, S. Hicks, P. Halvorsen, A. Al-Fuqaha, N. Conci, M. Riegler, Visual sentiment analysis from disaster images in social media, arXiv preprint arXiv:2009.03051.
  * [164] A. Alfarrarjeh, S. Agrawal, S. H. Kim, C. Shahabi, Geo-spatial multimedia sentiment analysis in disasters, in: 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA), IEEE, 2017, pp. 193–202.
  * [165] K. Ahmed, N. El Tazi, A. H. Hossny, Sentiment analysis over social networks: an overview, in: 2015 IEEE international conference on systems, man, and cybernetics, IEEE, 2015, pp. 2174–2179.
  * [166] G. A. de Oliveira Júnior, R. T. de Sousa Jr, R. de Oliveira Albuquerque, L. J. G. Villalba, Adversarial attacks on a lexical sentiment analysis classifier, Computer Communications 174 (2021) 154–171.
  * [167] N. Hofer, P. Schöttle, A. Rietzler, S. Stabinger, Adversarial examples against a bert absa model–fooling bert with l33t, misspellign, and punctuation, in: The 16th International Conference on Availability, Reliability and Security, 2021, pp. 1–6.
  * [168] A. Karimi, L. Rossi, A. Prati, Adversarial training for aspect-based sentiment analysis with bert, in: 2020 25th International Conference on Pattern Recognition (ICPR), IEEE, 2021, pp. 8797–8803.
  * [169] W. Wang, R. Wang, J. Ke, L. Wang, Textfirewall: Omni-defending against adversarial texts in sentiment classification, IEEE Access 9 (2021) 27467–27475.
  * [170] C. Du, H. Sun, J. Wang, Q. Qi, J. Liao, Adversarial and domain-aware bert for cross-domain sentiment analysis, in: Proceedings of the 58th annual meeting of the Association for Computational Linguistics, 2020, pp. 4019–4028.
  * [171] A. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, C. Potts, Learning word vectors for sentiment analysis, in: Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, 2011, pp. 142–150.
  * [172] A. Demontis, M. Melis, M. Pintor, M. Jagielski, B. Biggio, A. Oprea, C. Nita-Rotaru, F. Roli, Why do adversarial attacks transfer? explaining transferability of evasion and poisoning attacks, in: 28th $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 19), 2019, pp. 321–338.
  * [173] N. Z. Gong, A. Talwalkar, L. Mackey, L. Huang, E. C. R. Shin, E. Stefanov, E. Shi, D. Song, Joint link prediction and attribute inference using a social-attribute network, ACM Transactions on Intelligent Systems and Technology (TIST) 5 (2) (2014) 1–20.
  * [174] X. Zhang, M. Zitnik, Gnnguard: Defending graph neural networks against adversarial attacks, arXiv preprint arXiv:2006.08149.
  * [175] W. Jin, Y. Li, H. Xu, Y. Wang, S. Ji, C. Aggarwal, J. Tang, Adversarial attacks and defenses on graphs, ACM SIGKDD Explorations Newsletter 22 (2) (2021) 19–34.
  * [176] S. M. Kazemi, R. Goel, K. Jain, I. Kobyzev, A. Sethi, P. Forsyth, P. Poupart, Representation learning for dynamic graphs: A survey., J. Mach. Learn. Res. 21 (70) (2020) 1–73.
  * [177] A. Korolova, R. Motwani, S. U. Nabar, Y. Xu, Link privacy in social networks, in: Proceedings of the 17th ACM conference on Information and knowledge management, 2008, pp. 289–298.
  * [178] A. Narayanan, V. Shmatikov, De-anonymizing social networks, in: 2009 30th IEEE symposium on security and privacy, IEEE, 2009, pp. 173–187.
  * [179] V. Duddu, A. Boutet, V. Shejwalkar, Quantifying privacy leakage in graph embedding, arXiv preprint arXiv:2010.00906.
  * [180] X. He, R. Wen, Y. Wu, M. Backes, Y. Shen, Y. Zhang, Node-level membership inference attacks against graph neural networks, arXiv preprint arXiv:2102.05429.
  * [181] D. Kifer, A. Machanavajjhala, No free lunch in data privacy, in: Proceedings of the 2011 ACM SIGMOD International Conference on Management of data, 2011, pp. 193–204.