# M-VADER: A Model for Diffusion with Multimodal Context

Samuel Weinbach1 Marco Bellagente1 11footnotemark: 1
Constantin Eichenberg1 Andrew Dai1
Robert Baldock1 Souradeep Nanda1
Björn Deiseroth1,2 Koen Oostermeijer1
Hannah Teufel1 Andres Felipe Cruz-Salinas1
1Aleph Alpha GmbH, Heidelberg, Germany
2Artificial Intelligence and Machine Learning Lab, TU Darmstadt, Germany
Correspondence: {samuel.weinbach<EMAIL_ADDRESS>
###### Abstract

We introduce M-VADER111The name was suggested by the model itself: a diffusion
model (DM) for image generation where the output can be specified using
arbitrary combinations of images and text. We show how M-VADER enables the
generation of images specified using combinations of image and text, and
combinations of multiple images. Previously, a number of successful DM image
generation algorithms have been introduced that make it possible to specify
the output image using a text prompt. Inspired by the success of those models,
and led by the notion that language was already developed to describe the
elements of visual contexts that humans find most important, we introduce an
embedding model closely related to a vision-language model. Specifically, we
introduce the embedding model S-MAGMA: a 13 billion parameter multimodal
decoder combining components from an autoregressive vision-language model
MAGMA and biases finetuned for semantic search.

##  1 Introduction

Figure 1: M-VADER: image synthesis with multimodal context. The guidance
prompt, comprised of interleaved images and text, is embedded using a
multimodal decoder, S-MAGMA. The output of S-MAGMA is used to condition the
generation process of a fine-tuned version of Stable Diffusion via cross-
attention. This allows the finetuned version of Stable Diffusion to convert
the starting (random noise) image into the output image shown.

In recent years, advances in text-to-image synthesis have enabled the creation
of artistic and photorealistic images with impressive generalization from text
prompts [1, 2, 3, 4, 5]. In particular, text guided diffusion models have been
developed, which transform either random noise, or a starting image, into a
new image, guided by a text contextual prompt. The natural question to ask is
how one can guide diffusion models using multimodal guidance contexts.

Recently, a version of Stable Diffusion [1] has been developed that uses
single images as conditional information for the diffusion process [6].
However, there is no effective method to use multimodal context guidance in
diffusion models: a context that combines one or more passages of text with
one or more images. The richness in information contained in the desired image
output creation is sometimes hard to capture with a (single modal) text prompt
alone. This is the key contribution of this paper: a method for image
generation with multimodal contextual guidance (of arbitrary prompt length).
To this end, we develop a novel embedding model for multimodal contextual
prompts, and we finetune Stable Diffusion to follow contextual guidance.

M-VADER is composed by assembling components from two 13B transformer-based
models: a MAGMA image captioning model [7], and a 13B parameter “Luminous-
Explore” symmetric semantic search model [8, 9]. We additionally show how to
control the relative importance of the components of the multimodal guidance
prompt, by reweighting the attention scores to each component. Our
contributions are therefore:

  * •

S-MAGMA, a multimodal decoder embedding model

  * •

An image generation diffusion model, which we call M-VADER, (finetuned from
Stable Diffusion), which follows multimodal guidance

  * •

A method for scaling the relative importance of individual components in the
multimodal guidance prompt, leading to qualitatively predictable changes in
the images generated by M-VADER

Using M-VADER we demonstrate the following capabilities:

  * •

Image generation guided using multimodal context

  * •

Image composition, generation guided using the combination of two images

  * •

Image variation, by transforming random noise into a final image, guided using
an image to be varied.

##  2 Related Work

Various approaches exists to generative modelling of high dimensional
distributions. Generative adversarial networks (GANs) [10, 11] are likelihood-
free models whose objective is to minimize a sample-based distance with
respect to the target distribution. Image generation with GANs scales
efficiently with the pixel space size, while also being able to capture the
bulk of distributions. However, they struggle at modelling distribution tails
and their training dynamic is often unstable.

Conversely, likelihood-based models minimize the negative log-likelihood with
respect to the target distribution either in pixel space or a compressed
space. They can be realized as amortized inference models [12, 13], bijections
[14, 15] or autoregressive models (ARMs) based on chain-rule decomposition
into conditional probabilities [16, 17, 18]. Within the family of likelihood-
based models, ARMs achieve the best results in terms of density estimation.
However, modelling images directly in pixel space with ARMs scales poorly with
the pixel size, limiting such models to low resolution images. This scaling
issue has been solved by getting rid of the pixel space representation of
images and rather compress them to a latent space which is then modelled by an
ARM [19, 13, 20].

Diffusion models have seen a renaissance in recent years. Originally
introduced in [21], they went largely ignored in favour of other generative
methods such as variational autoencoders (VAEs) [12] and GANs. In 2020,
interest picked up again after Ho et al. published a state-of-the-art
diffusion model and a simplified training objective [22]. It was found that
diffusion models outperform GANs in terms of sample quality in unconditional
generation [23].

A diffusion model is a generative model defined by a forward and backward
process. In the forward process noise is gradually added to the data until it
becomes indistinguishable from the prior distribution, typically an isotropic
Gaussian distribution. The model is trained to reverse this noise corruption
in the backward process, after which it can be used to generate new samples.
It can be thought of as a hierarchical VAE, consisting of a chain of latents,
each corresponding to an increasing level of noise. Like a VAE, it is
optimised with an evidence lower-bound (ELBO) loss objective, however, the
encoder is pre-defined. This prevents mode collapse, caused by the
simultaneous training of two networks, which have historically plagued GANs
and standard VAEs.

They are also shown to be equivalent to denoising score-based methods [22],
which use the gradient of the logarithm of the data distribution and Langevin
dynamics for estimation [24, 25, 26]. The continuous-time version is defined
as a stochastic differential equation (SDE), it can be shown that all the
different diffusion models are discretizations of this more general framework
[27].

A drawback of diffusion models is that traversing the whole Markov chain in
the backward process would require thousands of model calls, resulting in
sampling speeds that are orders of magnitude slower than other methods. This
shortcoming was alleviated by a strided method in which multiple steps are
taken at once [28]. From the SDE perspective following the backward process is
equivalent to solving the SDE. This insight has allowed for other sampling
methods to be introduced, based on SDE and ODE solvers, leading to further
improvements [27, 29].

Dhariwal and Nichol [28] introduced a method whereby the diffusion process
could be steered through the use of a classifier as a way to realize
conditional generation. To avoid having to rely on a classifier, it is also
possible to train the model on a dataset where part of the images have text
labels. The same model can then be used to do both conditional and
unconditional generation [30]. This has become a standard tool for text-to-
image diffusion models [2, 3, 4, 1]. Our work extends the conditional
generation method used in these prior works to multimodal conditioning,
allowing for a new, more-flexible way to steer image generation.

Besides diffusion models, autoregressive generative models such as Parti [5]
have yielded comparable results.

Figure 2: M-VADER architecture for training. We condition the denoising model
through cross attention with embeddings from a Decoder Language Model
augmented with multimodal components (MAGMA) and finetuned biases for semantic
search. In training we use either the image or the caption for conditioning,
while inference is performed with multimodal input. Notation \- $z$: Image
latent. $z_{T}$: Noisy latent. $z_{c}$: Conditioning latent.

##  3 Method

In this section we describe the model architecture and discuss the details of
training. Then we present our new multimodal prompting paradigm for inference
which makes use of attention manipulation.

###  3.1 Architecture

The general architecture consists of three main components: A pre-trained LLM;
finetuned prefix and bias components enabling multimodal input and semantic
embeddings; and a diffusion model to generate image output which is
conditioned through embeddings produced by the LM stack. We describe these
components in detail.

Language model. Embeddings from a pre-trained LM can be employed effectively
to condition diffusion models, and scaling the LM backbone is an effective way
to achieve better quality and alignment in generation [4]. For this work we
use the 13B parameter model from Aleph Alpha’s Luminous family trained on
$\sim$400B language tokens. Luminous largely follows the architecture of
GPT-3, the major difference being the use of rotary positional embeddings in
our model. Luminous 13B is trained on a curated multilingual corpus containing
sources in English, German, French, Italian and Spanish.

Finetuned components. It has been shown in [7, 31, 32] that the few-shot
capabilities of LMs can be extended to multimodal prompts consisting of
arbitrary combinations of images and text. Following the architecture
introduced in MAGMA [7], we extend Luminous 13B to a multimodal model by
adding a CLIP Resnet encoder [33], a prefix layer and adapters which are then
finetuned on an autoregressive captioning task. Independent of multimodal pre-
training we also finetune the biases (c.f. BitFit [34]) of Luminous 13B on a
semantic contrastive objective [9]. These biases can be combined in plug-and-
play fashion with the multimodal components and yield surprisingly good
results on image-text similarity tasks, implying that the semantic feature
extraction from the biases extends to multimodal MAGMA embeddings. We stress
that both finetuning procedures leave the weights of the LM completely frozen,
thereby keeping the world knowledge of the model intact and preventing
catastrophic forgetting. In the following we refer to the stack of Luminous
13B together with its finetuned multimodal and semantic components as Luminous
S-MAGMA.

Diffusion model. The final component is the U-Net-based Denoising Diffusion
Probabilistic Model (DDPM) from Stable
Diffusion222https://github.com/CompVis/stable-diffusion333While finalizing
this work, a new version of Stable Diffusion has been released
https://stability.ai/blog/stable-diffusion-v2-release. The starting checkpoint
as well as all results refer only to version 1.4. The diffusion model takes an
image input and a conditional input, which can both be empty, and outputs an
image444in Stable Diffusion, the diffusion process happens in a compressed
latent space of a pre-trained VAE, hence image input/output needs to be
additionally encoded/decoded. For readability, we mostly suppress this in our
notation. by applying a denoising sampling procedure resembling Langevin
dynamics. Given an image and conditional input (in the case of Stable
Diffusion, a natural language prompt embedded by CLIP’s text encoder), gradual
noise is added to the image to create a latent vector. Then a new image is
generated by applying the denoising process to the noised latent while using
the conditional input to steer the generation through classifier-free guidance
[30]. If no image input is given, the latent vector is sampled from Gaussian
noise, and if no condition is given the denoising process is applied without
any bias.

End-to-end architecture. The formulation of DDPMs allows the generation
process to be conditioned on a variety of modalities of inputs such as text
prompts or bounding boxes. In Fig. 2 we provide an illustration of the model.
In our architecture, we leverage our augmented decoder model as follows. Given
any multimodal input, i.e. a sequence of interleaved images and texts, we
perform a forward pass through Luminous S-MAGMA to obtain a sequence of hidden
states after the final transformer layer. These hidden states are then used to
condition the unet through a cross-attention layer in each of the unet’s
attention layers [1, 4]. Note that the different finetuned components of
S-MAGMA can be plugged in and out in this approach, and be finetuned or frozen
during training. This allows for interesting combinations that have a
confounding effect on the quality, but also the style of the generations, see
Section 4.

###  3.2 Training

Training objective. We follow standard practice and train the diffusion model,
denoted by $F_{\theta}(x,c,t)$, to predict the noise
$\epsilon\sim\mathcal{N}(0,1)$ from a corrupted sample $\tilde{x}(\epsilon,t)$
via MSE loss at a randomly sampled diffusion time step
$t\sim\mathcal{U}(0,T)$,

$\displaystyle\begin{split}&\mathcal{L}_{\theta}=\mathbb{E}_{t,\epsilon,x}||\epsilon-
F_{\theta}(\tilde{x}(\epsilon,t),c,t)||^{2}.\\\ \end{split}$ (1)

Here, $x$ is the image input and $c$ is the conditional input. We train on
image caption pairs $(x_{img},x_{txt})$ and denote by $H(x)$ the sequence of
hidden states after the forward pass through Luminous S-MAGMA, detailed in the
previous subsection. For each image caption pair during training we use either
$c=H(x_{img})$ or $c=H(x_{txt})$ for conditioning the diffusion model, with
probability of using the image being $0.2$. More training details can be found
in Tab. 2.

Figure 3: Examples of image generation with multimodal guidance. Output images
(right) are composed from the combination of interleaved images and text shown
on the left.

Trained components. State-of-the-art DDPM models are pre-trained for millions
of steps on large batch sizes [1, 3, 4]. Given the availability of the public
checkpoint from Stable Diffusion, an efficient strategy is to only train from
scratch the newly added parameters. In this work, these are the cross-
attention parameters needed to accommodate the multimodal hidden states. There
is also the option to unfreeze and finetune various components from S-MAGMA,
meaning the embedding preprocessor $H=H_{\theta}$ would be updated during
training as well. There are numerous possible ablations using (or not using)
the components of MAGMA or Semantic Search, from which we have trained 2
models so far, see Tab. 1.

Dataset. We train the model on LAION aesthetics V.2
5+555https://laion.ai/blog/laion-aesthetics/, i.e. the subset of LAION 5B
filtered by aesthetic score with a predicted aesthetic score $>5$ [35].
Filtering only images with a minimum resolution of $512\times 512$, this
results in roughly $40$M captioned images.

###  3.3 Multimodal prompting

In inference we synthesize images using a variety of conditioning prompts
consisting of interleaved visual and textual data

prompt: [<Text1>, <Image1>, <Text2>, <Image2>, …].

Any prompt $x$ of the above format is converted into a conditional embedding
$c=H(x)$ and we utilize classifier free guidance to generate an image from a
pure Gaussian noise vector. Throughout our experiments we use a guidance scale
of 8.0 and the denoising differential equation is solved in 50 steps with the
pseudo-numerical method described in [36]. Because of the flexible format,
multimodal prompting can be used for quite complex image synthesis tasks, see
Fig. 3 for some examples. Two specific tasks where this method excels are
style change of images and image composition, which we elaborate on in Section
4.
An important ingredient for successful multimodal prompting is attention
manipulation, which we explain below.

###  3.4 Attention manipulation

By MAGMA’s architecture, every image translates into a sequence of $144$
embeddings, whereas the caption typically is between $10$ and $20$ tokens
long. Because of this, for conditional prompts of the form
$[\mathrm{image},\mathrm{text}]$ (or the other way around) the image is
implicitly upweighted in the cross-attention layers, making multimodal
prompting difficult out of the box. To counteract this phenomenon, during
inference we modify the attention scores in every attention layer of the
transformer as follows.
Let $S=(s_{ij})$ denote the pre-softmax attention scores inside an attention
head. Let $\theta_{i}$ be a non-negative number for every token $i$. Then we
obtain the modified scores $\tilde{S}=(\tilde{s}_{ij})$ by the simple
operation

$\displaystyle\tilde{s}_{ij}=s_{ij}+\log\theta_{i},$ (2)

meaning that $\theta_{i}>1$ upweights the $i$-th token in the self-attention,
whereas $\theta_{i}<1$ downweights it, with $\theta_{i}=0$ completely
suppressing token $i$ after the softmax operation.
For a prompt $[\mathrm{image},\mathrm{text}]$ we found that it is usually
beneficial to downweight all image tokens uniformly, meaning we would set
$\theta_{i}=\rho<1$ for $i\leq 144$ and $\theta_{i}=1$ else. Attention
manipulation also helps with prompts containing two images
$[\mathrm{image}_{1},\mathrm{image}_{2}]$, where we found that sometimes one
image has a much more dominant effect on the diffusion process than the other
one, and attention manipulation can be used to balance the two effectively.
In theory, different attention factors across layers and/or attention heads
could be used, but we kept it simple with $\theta_{i}$ being the same in every
attention operation.

Figure 4: Style modification of a base image. For Stable Diffusion, we begin
the diffusion process from the noised base image and guide the diffusion
process with a text prompt. The strength parameter $s$ denotes the relative
level of input noise, where $s=0$ and $s=1$ correspond respectively to no
noise and full noise levels. We compare the outputs of this approach with
varying strengths to the M-VADER multimodal prompt which always generates from
pure noise.

##  4 Results

In this section we highlight the capabilities of M-VADER and multimodal
prompting. A number of examples are given in Fig. 3, demonstrating meaningful
composition of several concepts (captured in our multimodal embeddings) into a
generated output image that aligns with the input condition more than a purely
textual condition. We mainly focus on the image synthesis tasks of style
modification and image composition. As it is not the goal of this work to
improve the original model of [1] on pure text conditioning, we only include
such examples in Appendix B. We also remark that our model offers an alternate
approach to the task of image variation by prompting with just a single image
and include examples of this in Appendix A. We remark that these are purely
qualitative results, as a proper quantitative evaluation is currently not
feasible (see Section 5).

###  4.1 Style modification

prompt: [<Text1>, <Image1>, …].

In pure text conditioned diffusion models, style modification can be achieved
by starting from a corrupted version of an image and using a natural language
instruction as a condition to steer the generation. The level of image
corruption has to be chosen carefully, since there is a tradeoff between being
close to the original image and enforcing the desired style. On the one hand,
if only a little amount of noise is added to the image, the denoising process
is very short and the conditional guidance has not enough steps to have a
major effect. On the other hand, degrading the image too much one loses all
information about it and the result is a random image in the desired style. In
our experience, finding a sweetspot for the noise strength can be tricky for
some combination of images and styles and the outputs are inconsistent.
As a byproduct of image synthesis with multimodal context, we present a new
paradigm for style change using a multimodal prompt. We simply concatenate the
style instruction with the image to form a single multimodal conditional
prompt and then generate from pure noise. This way we can use the full
diffusion length to steer generation while maintaining information about the
original image consistently. A direct comparison of our new method to the old
text-to-image approach with Stable Diffusion is depicted in Fig. 4. In the
same way we can also directly modify single image elements as illustrated in
Fig. 5.

Figure 5: M-VADER allows image variation with the addition of an element
specified in text. In this example, the prompt is given by The painting of a
person wearing a beautiful <color> hat, <Image>, with the colour of the hat
specified below each image.

###  4.2 Image composition

While the visual part of a CLIP encoder may directly be used to get variations
of a base image [6], M-VADER can also out-of-the-box compose images with a
prompt of the form

prompt: [<Image1>, <Image2>, …].

In Fig. 6 we show an example of composition. The output image does not just
reproduce the main elements of the prompt independently of one another, but
also combines them in a complex and meaningful way. Note that this is
conceptually very different from image interpolation in diffusion models,
which works by adding a certain noise level to a set of images, then forming a
convex combination of the noised latents and denoising again, resulting in a
pixel space combination of the images rather than a combination in “semantic
space”.

Figure 6: Combination of images for composition. M-VADER can meaningfully
combine elements e.g. by having the astronaut ride the horse (left and center
outputs), or, when the angle of the picture doesn’t allow it, by turning the
space suit into a space harness (right output).

##  5 Limitations and societal impact

Data limitations. Available large datasets of captioned images such as Laion
5B [35] significantly shrink once filtered for unsafe content, minimal size
and watermarks. In particular, the final set of images used for training
stable diffusion and the model presented in this work consists of a modest
amount of roughly 40M, which may be an obstacle to its generalization
capabilities. We also found that in spite of the fact that stable diffusion
has been trained on the unfiltered Laion 5B only for a small fraction of its
full training, there’s a significant probability of generating a watermarked
image, depending on the prompt. See Appendix B for examples. Aside for better
or stricter filters, an interesting approach in this direction is to decrease
the odds of generating a watermark in inference by the use of negative
prompts. Finally, our backbone model is trained on single pairs of images and
text descriptions. Prior work has emphasised the importance of training AR
models on multimodal sequences, i.e. on data sequences containing arbitrarily
interleaved text, images and videos [37, 32]. We expect our multimodal
decoder, and the image diffusion capabilities as a consequence, to greatly
benefit from training on such data, and leave to future work to estimate its
impact.

Model limitations. As this work is an extension of Stable Diffusion, we of
course inherit limitations related to its architecture, e.g. concerning the
use of latent diffusion for high resolution. We therefore refer to the
original to [1] for a more detailed discussion. Related to the use of a
multimodal decoder, we observe that the content of generated images sometime
depends on the order in which the images are concatenated.

Ethical concerns. We identify misuse of the model and data as as the primary
elements to consider for this work. The field of generative modelling,
specially of image distributions, is haunted by the threat of misuse, and this
work is no exception. While it is clear how developments in the field are
initiating a new era for creative content creation, research, and business
applications, it is likewise apparent that countering a model’s misuse gets
harder and harder. This becomes specially true with the continuous
democratization of the field in terms of code/checkpoint availability and
hardware resources needed to finetune a model or run inference with it. We
refer the interested reader to in depth studies on ”deep fakes” [38, 39].

The final aspect to consider is the interaction between models and the data
they are trained on. While we may cautiously consider training algorithms and
to be unbiased, up to the fact that they were developed by humans, the same
can’t be said about data. ”Biases in, biases out”, i.e. it is well known [40]
that generative models can and do reproduce biases inherited from training
data. Furthermore, the degree of memorization of a model make them prone to
leak private data seen during training, limiting their application in data
sensitive applications, e.g. in synthetic data generation for healthcare
applications. We refer to [41] for a more detailed breakdown of ethical
concerns and additional sources on the topic.

Quantitative evaluations. A standard and natural practice to evaluate text-to-
image models would be to compute an FID score [42] on the validation set of
datasets like MS-COCO [43], conditioning on the provided caption.
Unfortunately it is not clear how to quantitatively evaluate more complex
tasks which M-VADER seems to perform well at. For example, for the task of
style modification it would be preferable to evaluate on a dataset which
provides triplets (style instruction, input image, output image ground
truths). Alternatively, style modification might be evaluated by using MS-COCO
captions to retrieve images from an external pool based on a similarity score.
However, doing so introduces an extra layer of complexity, as the alignment of
the output with respect to the prompt would depend on the model used for
search. Image composition presents a similar challenge, with no out-of-the box
suitable datasets available (to our knowledge). We believe that evaluating
complex image synthesis tasks is an important research topic in of itself with
the potential to advance image generation models even further. We plan to
address these challenges and properly evaluate M-VADER in future work.

##  6 Conclusion

In this work we have introduced M-VADER, a diffusion model capable of
synthesizing images based on multimodal embeddings. We achieved this by
assembling S-MAGMA, a 13B decoder-only transformer model combining components
from a captioning model and biases finetuned for semantic search. We have
displayed the capabilities of M-VADER using a variety of prompts, from generic
sequences of visual and text elements, to subsets of it, such as image
combinations and image variations. We see great potential for future
developments in research on multimodal prompting, bringing us closer to image
generation models that can better capture the intent of users for enhanced
creative expression.

##  7 Acknowledgements

We are grateful to Stability AI for open sourcing the Stable Diffusion
checkpoints and for the numerous details they provide to replicate the
training set used in their work, as well as the hyperparameters used and
details about the training procedure.

## References

  * [1] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684–10695, 2022.
  * [2] A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image generation and editing with text-guided diffusion models,” arXiv preprint arXiv:2112.10741, 2021.
  * [3] A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125, 2022.
  * [4] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. Denton, S. K. S. Ghasemipour, B. K. Ayan, S. S. Mahdavi, R. G. Lopes, et al., “Photorealistic text-to-image diffusion models with deep language understanding,” arXiv preprint arXiv:2205.11487, 2022.
  * [5] J. Yu, Y. Xu, J. Y. Koh, T. Luong, G. Baid, Z. Wang, V. Vasudevan, A. Ku, Y. Yang, B. K. Ayan, et al., “Scaling autoregressive models for content-rich text-to-image generation,” arXiv preprint arXiv:2206.10789, 2022.
  * [6] “Image variations by single-image guidance prompting in stable diffusion.” https://huggingface.co/spaces/lambdalabs/stable-diffusion-image-variations.
  * [7] C. Eichenberg, S. Black, S. Weinbach, L. Parcalabescu, and A. Frank, “Magma–multimodal augmentation of generative models through adapter-based finetuning,” arXiv preprint arXiv:2112.05253, 2021.
  * [8] “Aleph Alpha blogpost: Luminous-Explore - A model for world-class semantic representation.” https://www.aleph-alpha.com/luminous-explore-a-model-for-world-class-semantic-representation, 2022\.
  * [9] N. Muennighoff, “Sgpt: Gpt sentence embeddings for semantic search,” arXiv preprint arXiv:2202.08904, 2022.
  * [10] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020.
  * [11] K. Crowson, S. Biderman, D. Kornis, D. Stander, E. Hallahan, L. Castricato, and E. Raff, “Vqgan-clip: Open domain image generation and editing with natural language guidance,” in European Conference on Computer Vision, pp. 88–105, Springer, 2022.
  * [12] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
  * [13] A. Razavi, A. Van den Oord, and O. Vinyals, “Generating diverse high-fidelity images with vq-vae-2,” Advances in neural information processing systems, vol. 32, 2019.
  * [14] L. Dinh, D. Krueger, and Y. Bengio, “Nice: Non-linear independent components estimation,” arXiv preprint arXiv:1410.8516, 2014.
  * [15] L. Dinh, J. Sohl-Dickstein, and S. Bengio, “Density estimation using real nvp,” arXiv preprint arXiv:1605.08803, 2016.
  * [16] M. Chen, A. Radford, R. Child, J. Wu, H. Jun, D. Luan, and I. Sutskever, “Generative pretraining from pixels,” in International conference on machine learning, pp. 1691–1703, PMLR, 2020.
  * [17] D. P. Kingma, T. Salimans, R. Jozefowicz, X. Chen, I. Sutskever, and M. Welling, “Improved variational inference with inverse autoregressive flow,” Advances in neural information processing systems, vol. 29, 2016\.
  * [18] G. Papamakarios, T. Pavlakou, and I. Murray, “Masked autoregressive flow for density estimation,” Advances in neural information processing systems, vol. 30, 2017.
  * [19] P. Esser, R. Rombach, and B. Ommer, “Taming transformers for high-resolution image synthesis,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12873–12883, 2021.
  * [20] J. Yu, X. Li, J. Y. Koh, H. Zhang, R. Pang, J. Qin, A. Ku, Y. Xu, J. Baldridge, and Y. Wu, “Vector-quantized image modeling with improved vqgan,” arXiv preprint arXiv:2110.04627, 2021.
  * [21] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International Conference on Machine Learning, pp. 2256–2265, PMLR, 2015.
  * [22] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in Neural Information Processing Systems, vol. 33, pp. 6840–6851, 2020.
  * [23] P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” Advances in Neural Information Processing Systems, vol. 34, pp. 8780–8794, 2021.
  * [24] Y. Du and I. Mordatch, “Implicit generation and modeling with energy based models,” Advances in Neural Information Processing Systems, vol. 32, 2019\.
  * [25] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” arXiv preprint arXiv:2011.13456, 2020.
  * [26] Y. Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  * [27] T. Karras, M. Aittala, T. Aila, and S. Laine, “Elucidating the design space of diffusion-based generative models,” arXiv preprint arXiv:2206.00364, 2022\.
  * [28] A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion probabilistic models,” in International Conference on Machine Learning, pp. 8162–8171, PMLR, 2021.
  * [29] C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu, “Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps,” arXiv preprint arXiv:2206.00927, 2022.
  * [30] J. Ho and T. Salimans, “Classifier-free diffusion guidance,” arXiv preprint arXiv:2207.12598, 2022.
  * [31] M. Tsimpoukelli, J. L. Menick, S. Cabi, S. Eslami, O. Vinyals, and F. Hill, “Multimodal few-shot learning with frozen language models,” Advances in Neural Information Processing Systems, vol. 34, pp. 200–212, 2021.
  * [32] J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al., “Flamingo: a visual language model for few-shot learning,” arXiv preprint arXiv:2204.14198, 2022.
  * [33] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning, pp. 8748–8763, PMLR, 2021.
  * [34] E. Ben Zaken, Y. Goldberg, and S. Ravfogel, “BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language-models,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), (Dublin, Ireland), pp. 1–9, Association for Computational Linguistics, May 2022.
  * [35] C. Schuhmann, R. Vencu, R. Beaumont, R. Kaczmarczyk, C. Mullis, A. Katta, T. Coombes, J. Jitsev, and A. Komatsuzaki, “Laion-400m: Open dataset of clip-filtered 400 million image-text pairs,” arXiv preprint arXiv:2111.02114, 2021.
  * [36] L. Liu, Y. Ren, Z. Lin, and Z. Zhao, “Pseudo numerical methods for diffusion models on manifolds,” 2022.
  * [37] A. Aghajanyan, B. Huang, C. Ross, V. Karpukhin, H. Xu, N. Goyal, D. Okhonko, M. Joshi, G. Ghosh, M. Lewis, et al., “Cm3: A causal masked multimodal model of the internet,” arXiv preprint arXiv:2201.07520, 2022.
  * [38] M. A. Franks and A. E. Waldman, “Sex, lies, and videotape: Deep fakes and free speech delusions,” 2019.
  * [39] T. T. Nguyen, Q. V. H. Nguyen, D. T. Nguyen, D. T. Nguyen, T. Huynh-The, S. Nahavandi, T. T. Nguyen, Q.-V. Pham, and C. M. Nguyen, “Deep learning for deepfakes creation and detection: A survey,” Computer Vision and Image Understanding, vol. 223, p. 103525, 2022.
  * [40] P. Esser, R. Rombach, and B. Ommer, “A note on data biases in generative models,” arXiv preprint arXiv:2012.02516, 2020.
  * [41] N. Rostamzadeh, E. Denton, and L. Petrini, “Ethics and creativity in computer vision,” arXiv preprint arXiv:2112.03111, 2021.
  * [42] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” in Advances in Neural Information Processing Systems (I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, eds.), vol. 30, Curran Associates, Inc., 2017.
  * [43] T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Dollár, “Microsoft coco: Common objects in context,” 2014.

##  Appendix A Image variations

M-VADER can out-of-the-box generate variations of an image by conditioning the
diffusion model with an image embedding $H(I)$. While conceptually different,
a somewhat similar result can be obtained by denoising a partially corrupt
image with a vanilla diffusion model, with the corruption strength regulating
the deviation from to original image. We provide in Fig. 7 a qualitative
comparison of the two approaches.

Figure 7: Three variations of a single image in two flavors. S-MAGMA embeds
the input into a representation $H(I)$ which is then used to condition a
denoising process from Gaussian noise via cross-attention. With a latent text-
to-image model like Stable Diffusion, we start from a latent code of the image
$z=Enc(I)$ and corrupt it with noise up to a time step $\tilde{T}$.
$\tilde{T}$ needs to be tuned for each image, in such a way that the denoising
of $z_{\tilde{T}}$ is not too close and not too far from $z$.

##  Appendix B Watermarked outputs

Following the original work of Stable Diffusion, we filtered out images with
an estimated watermark probability $>0.5$. However, we observe that some
prompts have a significant chance of producing a watermarked output as shown
in Fig. 8.

|  |
---|---|---
|  |
Figure 8: Examples of watermarked outputs. In the first row we condition the
model over text with prompt: coconut trees near the beach. In the second row,
we instead produce variations of an image (prompt can be visualized in Fig.
11).

As per the authors description, the checkpoint used as starting point for this
work has been pre-trained for $237$k steps on the much larger and much less
filtered LAION 5B. This makes it hard to disentangle the source of watermarks
training examples. While training on higher quality data is an attractive
option, this will certainly come at the cost of slimming the training set even
more, with unclear consequences on the generalization capabilities of the
model. An alternative option is to accept instead the trade and use larger,
less filtered data, provided that negative prompting is used in inference.

##  Appendix C Input order

As the embeddings are generated autoregressively, the order of the elements in
the prompt is more important with respect to models with bi-directional
attention. This is particularly important for image composition, where no
natural language prompt is used to specify the relative relationship between
elements in the inputs. Since given a sequence of tokens information only
flows causally in the forward direction, we expect the first element in the
prompt to provide a stronger conditioning signal. We qualitatively verify this
in Fig. 9: the background as well as the relative importance of the elements
in the output is sensitive to the images order.

Figure 9: Image composition with the same pair of input figures, prompted in
different orders

##  Appendix D Experiments details

In this section we summarize the combinations of parameters finetuned during
training as well as the relevant hyperparameters used in the experiments
described in Sec. 3.2.

| UNET | Bias | MAGMA Adapters
---|---|---|---
V1 | Finetuned | Sem. Search, finetuned | Not used
V2 | Finetuned | Sem. Search, frozen | Yes, frozen

Table 1: Trained architecture variations

| training iters | lr cross attn | lr unet | batch size | embedding dropout rate
---|---|---|---|---|---
V1 | 54k | 0.0005 | 5e-5 | 2048 | 0.05
V2 | 60k | 0.0001 | 5e-5 | 2048 | 0.1

Table 2: Training details DDPM

##  Appendix E Selection of generated images

We present in Fig. 10-Fig. 14 a selection of examples generated with M-VADER,
spanning all the main use cases: multimodal prompts, images combination, image
variations and style modification.

Figure 10: Selection of compositions of image pairs. Figure 11: Selection of
variations of a base image.

Figure 12: Selection of images synthesized with a multimodal prompt Figure 13:
Selection of images synthesized with a multimodal prompt

Figure 14: Selection of images synthesized with a multimodal prompt


# DeepHate: Hate Speech Detection via
Multi-Faceted Text Representations

Rui Cao  University of Electronic Science and Technology of China, China
<EMAIL_ADDRESS>,  Roy Ka-Wei Lee  University of Saskatchewan, Canada
<EMAIL_ADDRESS>and  Tuan-Anh Hoang  Leibniz University of Hanover, Germany
<EMAIL_ADDRESS>
(2020)

###### Abstract.

Online hate speech is an important issue that breaks the cohesiveness of
online social communities and even raises public safety concerns in our
societies. Motivated by this rising issue, researchers have developed many
traditional machine learning and deep learning methods to detect hate speech
in online social platforms automatically. However, most of these methods have
only considered single type textual feature, e.g., term frequency, or using
word embeddings. Such approaches neglect the other rich textual information
that could be utilized to improve hate speech detection. In this paper, we
propose DeepHate, a novel deep learning model that combines multi-faceted text
representations such as word embeddings, sentiments, and topical information,
to detect hate speech in online social platforms. We conduct extensive
experiments and evaluate DeepHate on three large publicly available real-world
datasets. Our experiment results show that DeepHate outperforms the state-of-
the-art baselines on the hate speech detection task. We also perform case
studies to provide insights into the salient features that best aid in
detecting hate speech in online social platforms.

Online Toxic Content, Hate Speech Detection, Social Media

††journalyear: 2020††copyright: acmcopyright††conference: 12th ACM Conference
on Web Science; July 6–10, 2020; Southampton, United Kingdom††booktitle: 12th
ACM Conference on Web Science (WebSci ’20), July 6–10, 2020, Southampton,
United Kingdom††price: 15.00††doi: 10.1145/3394231.3397890††isbn:
978-1-4503-7989-2/20/07††ccs: Information systems Data mining††ccs: Computing
methodologies Information extraction††ccs: Computing methodologies Neural
networks††ccs: Applied computing Sociology

##  1\. Introduction

Motivation. The proliferation of social media has enabled users to share and
spread ideas at a prodigious rate. While the information exchanges in social
media platforms may improve an individual’s sense of connectedness with real
and virtual communities, these platforms are increasingly exploited for the
propagation of toxic content such as hate speeches (Schmidt and Wiegand, 2017;
Fortuna and Nunes, 2018), which are defined by the Cambridge dictionary as
“public speech that expresses hate or encourages violence towards a person or
group based on something such as race, religion, sex, or sexual orientation ”
(Cambridge, [n.d.]). The spread of hate speech in social media has not only
sowed discord among individuals or communities online but also resulted in
violent hate crimes (Williams, 2019; Relia et al., 2019; Mathew et al., 2019).
Therefore, it is a pressing issue to detect and curb hate speech in online
social media.

Major social media platforms such as Facebook and Twitter have made great
efforts to combat the spread of hate speech in their platforms (Times, 2019;
Bloomberg, 2019). For instance, the platforms have provided clear policies on
hateful conducts (Facebook, [n.d.]; Twitter, [n.d.]), implemented mechanisms
for users to report hate speech, and employed content moderators to detect
hate speeches actively. However, such approaches are labor-intensive, time-
consuming, and thus not scalable or sustainable in the long run (Waseem and
Hovy, 2016; Gambäck and Sikdar, 2017).

The gravity of the issues and limitations of manual approaches has motivated
the search for automatic hate speech detection methods. In recent years,
researchers from data mining, information retrieval, and Natural Language
Processing (NLP) fields have proposed several such methods (Fortuna and Nunes,
2018; Schmidt and Wiegand, 2017). These methods can be broadly grouped into
two categories: (i) methods that adopted classic machine-learning strategies
(Chen et al., 2012; Waseem and Hovy, 2016; Waseem, 2016; Nobata et al., 2016;
Chatzakou et al., 2017; Davidson et al., 2017), and more recently, (ii) deep
learning-based methods (Djuric et al., 2015; Mehdad and Tetreault, 2016;
Gambäck and Sikdar, 2017; Badjatiya et al., 2017; Park and Fung, 2017;
Gröndahl et al., 2018; Zhang et al., 2018; Arango et al., 2019; Founta et al.,
2019).

For traditional machine learning-based methods, textual features such as bag-
of-words are commonly extracted from social media posts to train a classifier
to detect hate speech in the posts. For deep learning-based methods, words in
the posts are often represented by some word embedding vectors and fed as
inputs to train a neural network for hate speech detection. Although the
existing methods, especially the deep learning ones, have shown promising
results in automatic hate speech detection in social media, there are
limitations in these models. Firstly, most of the existing methods have only
considered single type textual features, neglecting the other rich textual
information that could be utilized to improve hate speech detection. Secondly,
the current deep learning methods offered limited explainability on why a
particular post should be flagged as hate speech.

Research Objectives. In this paper, we address the limitations in existing
methods and propose DeepHate111Code:
https://gitlab.com/bottle_shop/safe/deephate, a novel deep learning
architecture that effectively combines multi-faceted textual representations
for automatic hate speech detection in social media. At a high level, DeepHate
utilizes three types of textual representations: semantic, sentiment, and
topical information. The semantic representations of social media posts are
extracted from several word embeddings pre-trained on a much larger text
corpus (Mikolov et al., 2018; Pennington et al., 2014; Wieting et al., 2015).
For sentiment representation, we proposed a two-step approach that utilizes an
existing sentiment analysis tool (Hutto and Gilbert, 2014) and neural network
to train a new word embedding that captures the sentiment information in a
social media post. Latent Dirichlet Allocation (LDA) (Blei et al., 2003) is
used to extract topical representations of posts. The three types of textual
representations are subsequently used as input to train our DeepHate model for
hate speech classification. The underlying intuition is that the multi-faceted
textual representations enrich the representation of social media posts, and
this enables the deep learning classifier to perform better hate speech
detection. To gain a better understanding of how DeepHate decides which posts
to classify as hate speech, we also conduct empirical studies to analyze the
salient features aided in hate speech detection.

Contributions. Our main contributions in this work consist of the following.

  * •

We propose a novel deep learning model called DeepHate, which utilizes multi-
faceted textual representations for automatic hate speech detection in social
media.

  * •

We conduct extensive experiments on three real-world and publicly available
datasets. Our experiment results show that DeepHate consistently outperforms
state-of-the-art methods in the hate speech detection task.

  * •

We conduct empirical analyses on the DeepHate model and provide insights into
the salient features that helped in detecting hate speech in social media. The
salient feature analysis also improves the explainability of our proposed
model.

##  2\. Related Work

Automatic detection of hate speech has received considerable attention from
the data mining, information retrieval, and natural language processing (NLP)
research communities. Interest in this field has increased with the
proliferation of social media and social platforms. In this section, we focus
on reviewing the existing works for detecting hate speech in social media text
content. Mostly, we will be focusing on hate speech detection on Twitter short
messages (i.e., tweets). These works can be broadly categorized into two
approaches: (i) works that adopt classic machine-learning strategies, and,
more recently, (ii) those that deep learning methods. Readers are encouraged
to refer to recent the surveys on the topic (Fortuna and Nunes, 2018; Schmidt
and Wiegand, 2017) for great detail of methods for both Twitter and other
social media.

Traditional machine learning methods have been applied to detect hate speech
in social media (Chen et al., 2012; Waseem and Hovy, 2016; Waseem, 2016;
Nobata et al., 2016; Chatzakou et al., 2017; Davidson et al., 2017).
Typically, these methods include an initial feature extraction phase, where
features are extracted from the raw textual content. The most commonly
extracted features include Term Frequency Inverse Document Frequency (TF-IDF)
scores, Bag-of-Words vectors, and other linguistic attributes. Xiang et al.
(Xiang et al., 2012) had also explored the latent semantic features extracted
from tweets for hate speech detection by mining topics of the tweets. Beyond
the textual content, some studies have also utilized other meta-information
from the users’ profiles and network structures (i.e., followers, mentioned,
etc.) (Chatzakou et al., 2017; Papegnies et al., 2017; Singh et al., 2017).
The extracted features are subsequently used as input for classifiers such as
Logistic Regression, SVM, Random Forest, etc., to predict if the given tweet
contains hate speeches. In this paper, we focus on utilizing only the textual
content to perform hate speech detection as other meta-information is scarce
and often incomplete. Moreover, by not using these meta-information, we would
avoid to bias our hate detectors by any personal information of the users.

Deep learning methods have achieved notable performance in many classification
tasks. Unlike traditional machine learning methods, deep learning methods are
able to automatically learn latent representations of the input data to
perform classification (Goodfellow et al., 2016). Such deep learning
approaches have also been applied to several natural language processing
tasks, including text classification (Goldberg, 2016; Yang et al., 2016). The
increasing popularity of deep learning approaches also sees a number of recent
studies adopting these methods to detect hate speech in social media (Djuric
et al., 2015; Mehdad and Tetreault, 2016; Gambäck and Sikdar, 2017; Badjatiya
et al., 2017; Park and Fung, 2017; Gröndahl et al., 2018; Zhang et al., 2018;
Arango et al., 2019; Founta et al., 2019).

Mehdad et al. (Mehdad and Tetreault, 2016) experimented applying Recurrent
Neural Network (RNN) model different input types such as word, unigram, and
bigram character embeddings for hate speech detection. Gambäck et al. (Gambäck
and Sikdar, 2017) conducted similar studies using Convolutional Neural Network
(CNN). Badjatiya et al. (Badjatiya et al., 2017) proposed an ensemble approach
that combines Long-Short Term Memory (LSTM) model and Gradient-Boost Decision
Tree to detect hate speech on Twitter. Park and Fung (Park and Fung, 2017)
introduced the HybridCNN method that trains a CNN over both word and unigram
character embeddings for hate speech detection. Zhang et al. (Zhang et al.,
2018) proposed a new neural network architecture that combines CNN with Gate
Recurrent Unit (GRU) to classify hate speech using word embeddings as input.
Founta (Founta et al., 2019) trains a combined RNN and Multilayer perceptron
(MLP) on textual content and meta-information to perform hate speech
detection. Most of these studies have applied and experimented with their
proposed methods on datasets proposed in (Waseem and Hovy, 2016; Davidson et
al., 2017). Unlike the existing deep learning methods, which mainly utilized
word or character embeddings as input, we proposed a novel deep learning
architecture to combine multi-faceted text representations for the
classification of hate speech. We will evaluate and benchmark our proposed
model against these state of the art in Section 5.

##  3\. Proposed Model

In this section, we elaborate on our proposed DeepHate model. The intuition of
our proposed model is to learn the latent representations of multi-faceted
text information, and effectively combine them to improve hate speech
detection. Figure 1 illustrates the overall architecture of the DeepHate
model. The proposed model first utilizes different types of feature embeddings
to represent a post $p$. The feature embeddings are subsequently fed into
neural network models to learn three types of textual information latent
representations, namely, semantic, sentiment, and topic. The latent
representations are then combined via a feed-forward network. Finally, a
softmax layer takes the combined representation as input to predict
probability distribution over all possible classes. The details of the
individual components in DeepHate model are described in the subsequent
subsections.

Figure 1. Overall architecture of DeepHate model

###  3.1. Semantic Representation

To learn the latent semantic representation of a post $p$, we first represent
the post as a sequence word, i.e.,
$W^{p}=(w^{p}_{1},w^{p}_{2},\cdots,w^{p}_{L})$, where $L$ refers to the length
of the post. Each word in the sequence is then represented by pre-trained word
embeddings. In order to obtain expressive word-level representations, we use
three popular pre-trained word embeddings, namely: GloVe (Pennington et al.,
2014), word2vec trained on wiki news (Mikolov et al., 2018), and Paragram
(Wieting et al., 2015). Consequently, we denote the pre-trained word
embeddings of post $p$ as follows:

  * •

Glove:
$e^{p}_{w\\_g}=(e^{p}_{w\\_g_{1}},e^{p}_{w\\_g_{2}},\cdots,e^{p}_{w\\_g_{L}})$

  * •

Word2Vec-Wiki:
$e^{p}_{w\\_v}=(e^{p}_{w\\_v_{1}},e^{p}_{w\\_v_{2}},\cdots,e^{p}_{w\\_v_{L}})$

  * •

Paragram:
$e^{p}_{w\\_r}=(e^{p}_{w\\_r_{1}},e^{p}_{w\\_r_{2}},\cdots,e^{p}_{w\\_r_{L}})$

Note that all pre-trained word embeddings are trained as 300-dimensional
vectors. Each pre-trained word embedding of a post is used as input into a
C-LSTM-Att encoder to learn a latent representation of the post. For instance,
we learn the latent post representations $x^{p}_{w\\_g}$ from $e^{p}_{w\\_g}$,
$x^{p}_{w\\_v}$ from $e^{p}_{w\\_v}$, and $x^{p}_{w\\_r}$ from
$e^{p}_{w\\_r}$. The detail implementation of the C-LSTM-Att encoder will be
presented in section 3.4.

Finally, we generate the post’s latent semantic representation, $x^{p}_{w}$,
by combining the three latent post representations learned from various types
of pre-trained word embeddings. We want to fully exploit the advantage of each
kind of representation. As a consequence, we assign an attention weight, which
is a single value, to each vector of representation. To make it easier for
training, these attention weights $a_{g},a_{v},a_{r}$ are initialized by
standard uniform distribution. Element-wise summation are performed on the
three attended latent post representation vectors.

(1) $x^{p}_{w}=a_{g}x^{p}_{w\\_g}+a_{v}x^{p}_{w\\_v}+a_{r}x^{p}_{w\\_r}$

$x^{p}_{w}$ will be subsequently combined with other textual information
latent representations for hate speech detection in the fusion process
described in Section 3.5.

###  3.2. Sentiment Representation

Tweets often contain abundant sentiment information (Giachanou and Crestani,
2016). Each tweet encodes the attitude and emotion of the writer, and such
information may be helpful in hate speech detection. From this intuition, we
aim to incorporate sentiment information in the DeepHate model. Inspired by
the method proposed in (Tang et al., 2014), our goal here is to train a word
embedding that contains sentiment information. Then, similar to the training
of latent semantic representation, this trained sentiment-specific word
embedding, denoted by $e^{p}_{s}$, will be used to represent the words in a
post, which is subsequently used as input into the C-LSTM-Att encoder to learn
the latent sentiment representation of the post.

However, the technique proposed in (Tang et al., 2014) is a supervised method
that requires labels for sentiment classification, and the existing hate
speech datasets lack such sentiment labels. To overcome this limitation, we
first explore a sentiment analysis tool, VADER (Hutto and Gilbert, 2014), to
label the sentiment for the tweets in the hate speech datasets. VADER is a
lexicon and rule-based sentiment analysis tool that is specifically tuned to
extract sentiments expressed in social media. Given a tweet, the tool
generates scores over three polarities: negative, positive, and neutral. In
our implementation, we assume the sentiment label for a tweet to be the
polarity with the highest score. With the generated sentiment labels for the
hate speech datasets, we train a sentiment-specific word embedding (Tang et
al., 2014) by performing a sentiment classification task to predict the
sentiment labels in hate speech datasets. Similar to the pre-trained word
embeddings, the sentiment-specific word embedding (a.k.a sentiment embedding)
is trained as 300-dimensional vectors.

Finally, we represent the word sequence of a post using the trained sentiment
embedding, $e^{p}_{s}=(e^{p}_{s_{1}},e^{p}_{s_{2}},\cdots,e^{p}_{s_{L}})$, and
use it as input into a C-LSTM-Att encoder to learn a latent sentiment
representation of the post, $x^{p}_{s}$. Note that we also freeze the
parameters in the sentiment embedding layers such that the sentiment embedding
is not updated; this ensures the sentiment information is preserved and not
updated by the back-propagation from our hate speech detection task.

###  3.3. Topic Representation

We employ the probabilistic topic modeling approach (Blei, 2012) to derive the
topic representation of the posts. Generally, this approach allows to
represent each text document by a multinomial distribution over topics where
each topic is mathematically defined as a multinomial distribution over words.
Often, in mining topics from Twitter posts, the documents are formed by
aggregating the posts by hashtags, published time, or authors (Hong and
Davison, 2010; Zhao et al., 2011). However, since our datasets are highly
sparse in those dimensions (e.g., there are not many common hashtags among
posts, posts are published in a long span of time, and each user has very few
posts), we consider each single post is a document and make use of the
original Latent Dirichlet Allocation model (Blei et al., 2003) (LDA) for
computing the posts’ topic distribution. Moreover, inspired by previous works
that each post should focus on one or a few topics, we exploit sparsity
regularization in the learning of the LDA model (Balasubramanyan and Cohen,
2013). Thus, posts’ topic representations, $x^{p}_{t}$, are sparse vectors
(which are highly skewed multinomial distribution over topics).

Specifically, we first remove all stopwords and non-English words. Next, we
iteratively filter away infrequent words and too short posts such that: each
word must appear in at least five remaining posts while each post contains at
least three remaining words. These minimum thresholds are to ensure that for
each post and each word, we have enough observations to learn the topics
accurately. To determine the appropriate number of topics in each dataset, we
run the LDA model (with sparsity regularization) with the number of topics
varied from 5 to 20 and measure the likelihood of the learned model in each
case. The number of topics in each dataset is then set by considering both
computational complexity and improvement in the likelihood. Lastly, for too
short posts that are previously filtered away, we assume that their topic
distributions are uniform. There is just a small proportion (often less than
10%) of such posts in each dataset.

###  3.4. C-LSTM-Att Encoder

Zhou et al. (Zhou et al., 2015) introduced the C-LSTM model, which stacked a
convolutional neural network (CNN) and long-short term memory (LSTM)
(Hochreiter and Schmidhuber, 1997), for text classification. Motivated by
C-LSTM’s good performance in the text classification task, we modify C-LSTM by
adding an attention mechanism (Yang et al., 2016) to select informative words
that contribute more towards detecting hate speech in social media posts.
Also, our proposed network can be adaptive to convolution operation with
multiple filter sizes. We name the modified model C-LSTM-Att. The C-LSTM-Att
model is used to encode the post’s pre-trained word embeddings, i.e.,
$e^{p}_{w\\_g}$, $e^{p}_{w\\_v}$, $e^{p}_{w\\_r}$, and sentiment embedding,
$e^{p}_{s}$, into its latent semantic and sentiment representations. As the
process of encoding the four feature embeddings is similar, we use $e^{p}$ as
a generic word embedding and $x^{p}$ as a generic latent semantics or
sentiment representation of a post for illustration in our subsequent
discussion in this section.

CNN Component. This component is a slight variant of the traditional
convolutional networks (Goodfellow et al., 2016). Let
$e^{p}_{i}\in\mathbb{R}^{d}$ be the $d$-dimensional word vectors for the
$i$-th word in a post $p$. Let $e^{p}\in\mathbb{R}^{L\times d}$ denote the
input post where $L$ is the length of the post $p$. Let $k$ be the length of
the filter, and the vector $\mathbf{m}\in\mathbb{R}^{k\times d}$ is a filter
for the convolution operation. For each position $j$ in the post, we have a
window vector $w_{j}$ with $k$ consecutive word vectors, denoted as:

(2) $w_{j}=[e^{p}_{j},e^{p}_{j+1},\cdots,e^{p}_{j+k-1}]$

Where the commas represent row vector concatenation. A feature map
$\mathbf{c}\in\mathbb{R}^{L-k+1}$ is generated with as the filter $\mathbf{m}$
convolves with the window vectors ($k$-grams) at each position. Each element
$c_{j}$ of the feature map for window vector $w_{j}$ is produced as follows:

(3) $c_{j}=f(\mathbf{m}w_{j}+b)$

where $b$ is a bias term and $f$ is the ReLU nonlinear transformation
function. We use $n$ filters to generate $n$ feature maps. To be more
specific, the output from convolution operation with windeow size $k$ and $n$
filters is: $C^{s1}\in R^{(L-k+1)\times n}$. We assume there are $v$ different
sizes of filters, so the output is: $C^{s1},C^{s2}\dots C^{sv}$. Each of the
output is subsequently fed into the LSTM component individually. As LSTM is
specified for sequence input while pooling operators break the sequence
organization due to the discontinuous selected features, we do not use these
operators after the convolution operation.

LSTM Component. We adopt the LSTM model (Goodfellow et al., 2016) to learn the
long-range dependencies from higher-order sequential features generated by the
CNN component of our model. Formally, the transition functions are defined as
follows:

(4) $\displaystyle i_{t}=\sigma(W_{i}\cdot[h_{t-1},x_{t}]+b_{i})$
$\displaystyle f_{t}=\sigma(W_{f}\cdot[h_{t-1},x_{t}]+b_{f})$ $\displaystyle
o_{t}=\sigma(W_{o}\cdot[h_{t-1},x_{t}]+b_{o})$ $\displaystyle c_{t}=f_{t}\odot
c_{t-1}+i_{t}\odot tanh(W_{c}\cdot[h_{t-1},x_{t}]+b_{c})$ $\displaystyle
h_{t}=o_{t}\odot tanh(c_{t})$

where $x_{t}$ denotes the current input feature representation from $C^{si}$
and $x_{t}\in\mathbb{R}^{n}$; $h_{t-1}$ is the old hidden state at previous
time $t-1$; $i_{t}$, $o_{t}$, $f_{t}$ represent the values of an input gate,
an output gate, a forget gate at time step $t$, respectively. We use $z$ as
the number of hidden states in the LSTM. These gates collectively decide how
to update the current memory cell $c_{t}$ and the current hidden state
$h_{t}$. $W_{i},W_{f},W_{o},W_{c}\in\mathbb{R}^{z\times(n+z)}$ are the
weighted matrices and $b_{i},b_{f},b_{o},b_{c}\in\mathbb{R}^{z}$ are biases,
which parameterizing the transformations. Here, $\sigma$ is the logistic
sigmoid function that has an output in [0, 1], $tanh$ denotes the hyperbolic
tangent function that has an output in [-1, 1], and $\odot$ denotes the
element-wise multiplication. Attention mechanism is subsequently applied on
the learned hidden states to capture the informative words that contribute
more towards detecting hate speech in social media post.

Attention Mechanism. Not all words in a post contribute equally to the
detection of hate speech. We adapt the word attention proposed by Yang et al.
(Yang et al., 2016) to emphasis the important word features extracted from
LSTM that aided hate speech detection and aggregate the representation of
these informative word features to form the post’s latent representations.

Specifically, we let $H\in\mathbb{R}^{z\times l}$ be a matrix consisting of
hidden vectors $[h1,\cdots,h_{l}]$ that the C-LSTM model produced, where $z$
is the size of hidden layers and $l$ is the length of the output. The length
$l$ can be represented as $L-k+1$, when the window size of filters is $k$. The
final hidden state of LSTM, $h_{l}$, contains information of the whole
sentence. To implement self attention over the sentence, we consider the
relation between each word and the meaning of the whole sentence. As a
consequence, we use the final hidden state to attend to hidden state of each
word to obtain the importance of each word for detecting hate speech. The
attention mechanism will produce an attention weight vector $\alpha$ and the
post’s latent representations $x^{p}$.

(5) $M=tanh(W_{H}H+W_{h}h_{L}+b_{h})$ (6) $\alpha=softmax(w^{T}M)$ (7)
$x^{p}=H\alpha^{T}$

where, $M\in\mathbb{R}^{a\times l}$ , $w\in\mathbb{R}^{a}$ are projection
parameters and $a$ is a middle dimension of computation.
$\alpha\in\mathbb{R}^{l}$ is a vector consisting of attention weights and
$x^{p}$ is attended output where there is only single size of filters. As
there are multiple filter sizes, we concatenate output from attention module
to form the final latent representation of the post. As such, the post’s
semantic representations $x^{p}_{w\\_g}$, $x^{p}_{w\\_v}$, and
$x^{p}_{w\\_r}$, and sentiment representation $x^{p}_{s}$ are learned in
similar fashion.

Figure 2. Sentiment distribution of datasets. Figure a, b, c corresponds to
the distribution of WZ-LS, DT and FOUNTA. Green in the pie chart denotes
tweets labeled positive, orange denotes negtive and blue denotes neural.

###  3.5. Representation Fusion

After learning the latent semantic, sentiment, and topic representations, we
perform operations to effectively combine the different post representations
in the representation fusion layer.

In this layer, we adopt the Gate Attention Network to fuse information from
multiple modalities. The Gate Attention Network proposed by Tang et al. (Tang
et al., 2019) is effective for combining two vectors. Here we extend the gate
fusion from two modalities to three modalities. The sentiment representation
and topic representation are regarded as complementary information of semantic
representation. So they are weighted by the interaction with semantic
representation at first. Eventually, element-wise sum operation is performed
to combine the weighted sentiment and topic representation with the semantic
representation:

(8) $att_{w}^{s}=\sigma(W_{s}(x_{w}^{p}+x_{s}^{p}))$ (9)
$att_{w}^{t}=\sigma(W_{t}(x_{w}^{p}+x_{t}^{p}))$ (10)
$x_{J}^{p}=x_{w}^{p}+att_{w}^{s}x_{s}^{p}+att_{w}^{t}x_{t}^{p}$

where $W_{s}$ and $W_{t}$ are weighted matrices. The resulting joint
representation $x_{J}^{p}$ is a high level representation of the post and can
be used as features for hate speech multi-class classification in the softmax
layer.

###  3.6. Implementation Details

We use the same embedding size $d=300$ for pre-trained word embeddings and
sentiment embedding. Additionally, we have added a dropout layer with $150\%$
dropout after the embedding layer for regularization and $20\%$ dropout for
fully connected layer. Each CNN layer has three kinds of filter window size
with 50 filters each. The number of hidden states in LSTM is set as 200. We
use ADAM (Kingma and Ba, 2014) as the optimizer with a learning rate of
$0.001$ to train our model.

##  4\. Dataset

We evaluate DeepHate on three publicly available datasets and one dataset
reconstructed by merging tweets of the three datasets. The first three
datasets, namely, WZ-LS (Park and Fung, 2017), DT (Davidson et al., 2017), and
FOUNTA (Founta et al., 2018), are widely used in hate speech detection
studies. The merged dataset was constructed by concatenating the three
datasets and removing the spam tweets. Table 1 shows the statistical summary
of the these datasets. The table clearly shows that these datasets are diverse
in both size and nature, thus allows us to evaluate our proposed model more
comprehensively.

Table 1. Statistic information about datasets in experiments Dataset | #tweets |  Classes (#tweets)
---|---|---
WZ-LS | 13,202 |  racism (82), sexism (3,332), both (21), neither (9,767)
DT | 24,783 |  hate (1,430), offensive (19,190), neither (4,163)
FOUNTA | 89,990 |  normal (53,011), abusive (19,232), spam (13,840), hate (3,907)
COMBINED | 114,120 |  normal (66,941), inappropriate (47,194)

WZ-LS dataset. Park et al. (Park and Fung, 2017) combined two Twitter datasets
(Waseem and Hovy, 2016; Waseem, 2016) to form the WZ-LS dataset. The dataset
breaks down the types of hate speech into four classes: racism, sexism, both,
and neither. As only the ids of the tweets are released in (Park and Fung,
2017), we retrieve the text of the tweets using Twitter’s APIs. However, some
of the tweets have been deleted by Twitter due to their inappropriate content.
Thus, our dataset is slightly smaller than the original dataset reported in
(Park and Fung, 2017).

DT dataset. Davidson et al. (Davidson et al., 2017) argued that hate speech
should be differentiated from offensive tweets; some tweets may contain
hateful words but may be just offensive and did not meet the threshold of
classifying them as hate speech. The researchers constructed the DT Twitter
dataset, which manually labeled and categorized tweets into three categories:
offensive, hate, and neither.

FOUNTA dataset. The FOUNTA dataset is recently published in (Founta et al.,
2018). It’s a human-annotated dataset that went through two rounds of
annotations. In the first round, annotators are required to classify tweets
into three categories: normal, spam, and inappropriate. Subsequently, the
annotators were asked to refine further the labels of the tweets in the
“inappropriate” category. Specifically, the final version of the dataset
consists of four classes: normal, spam, hate, and abusive. We found that there
were duplicated tweets in FOUNTA dataset as the dataset annotators have
included retweets in their dataset. For our experiments, we remove the
retweets resulting in the distribution in Table 1.

COMBINED dataset. The COMBINED dataset is an aggregation of all inappropriate
tweets including offensive and hate tweets, and normal tweets in three
datasets above. We postulate that the aggregated dataset is closer to the
real-world application as the social media platform provider is likely to be
interested in detecting and reducing both hate and offensive tweets. The
COMBINED dataset is the largest dataset with diverse types of inappropriate
tweets.

##  5\. Experiment

In this section, we will first describe the settings of experiments conducted
to evaluate our DeepHate model. Next, we discuss the experiment results and
evaluate how DeepHate fare against other state-of-the-art baselines. We
conduct more in-depth ablation studies on the various components in the
DeepHate. Empirical analyses on the sentiment, topics, and salient features
that aided in hate speech detection are also conducted. Finally, we discuss
some interesting case studies on how DeepHate’s strengths and limitations in
hate speech detection.

###  5.1. Experiment Setup

Baselines. For evaluation, we compare the DeepHate model with the following
state-of-the-art baselines that utilized textual content for hate speech
detection:

Table 2. Experiment results of DeepHate and baselines on WZ-LS dataset

Model | Micro-Prec | Micro-Rec | Micro-F1
---|---|---|---
CNN-W | 75.95 | 78.57 | 75.54
CNN-C | 54.77 | 74.01 | 62.95
CNN-B | 76.30 | 79.08 | 74.78
LSTM-W | 75.39 | 79.52 | 74.52
LSTM-C | 74.82 | 78.13 | 71.95
LSTM-B | 54.77 | 74.01 | 62.95
HybridCNN | 76.35 | 78.93 | 75.98
CNN-GRU | 75.33 | 79.27 | 74.42
DeepHate | 77.95 | 79.48 | 78.19

Table 3. Experiment results of DeepHate and baselines on DT dataset

Model | Micro-Prec | Micro-Rec | Micro-F1
---|---|---|---
CNN-W | 87.88 | 88.65 | 87.95
CNN-C | 60.53 | 77.43 | 67.60
CNN-B | 78.02 | 80.33 | 77.01
LSTM-W | 88.08 | 89.08 | 87.87
LSTM-C | 77.21 | 79.88 | 76.47
LSTM-B | 59.97 | 77.44 | 67.60
HybridCNN | 88.33 | 88.96 | 88.07
CNN-GRU | 87.60 | 88.24 | 87.23
DeepHate | 89.97 | 90.39 | 89.92

Table 4. Experiment results of DeepHate and baselines on FOUNTA dataset.

Model | Micro-Prec | Micro-Rec | Micro-F1
---|---|---|---
CNN-W | 78.26 | 79.71 | 78.27
CNN-C | 69.66 | 70.15 | 64.40
CNN-B | 52.01 | 58.41 | 50.64
LSTM-W | 78.54 | 79.87 | 78.48
LSTM-C | 70.15 | 70.89 | 66.30
LSTM-B | 55.22 | 62.71 | 54.52
HybridCNN | 78.34 | 79.24 | 77.73
CNN-GRU | 78.62 | 80.17 | 78.39
DeepHate | 78.95 | 80.43 | 79.09

Table 5. Experiment results of DeepHate and baselines on COMBINED dataset.

Model | Micro-Prec | Micro-Rec | Micro-F1
---|---|---|---
CNN-W | 91.86 | 91.77 | 91.72
CNN-C | 79.56 | 79.06 | 78.50
CNN-B | 42.26 | 58.63 | 43.67
LSTM-W | 91.54 | 91.54 | 91.52
LSTM-C | 82.46 | 80.64 | 79.70
LSTM-B | 64.93 | 65.50 | 63.85
HybridCNN | 91.77 | 91.72 | 91.67
CNN-GRU | 91.63 | 91.40 | 91.31
DeepHate | 92.48 | 92.45 | 92.43

  * •

CNN: Previous studies have utilized CNN to perform automatic hate speech
detection and achieved good results (Badjatiya et al., 2017; Gambäck and
Sikdar, 2017; Agrawal and Awekar, 2018). For baselines, we train three CNN
models with different input embeddings: word embedding (CNN-W), character
embedding (CNN-C), and character-bigram embedding (CNN-B).

  * •

LSTM: The LSTM model ,is another model that was commonly explored in previous
hate speech detection studies (Badjatiya et al., 2017; Agrawal and Awekar,
2018; Gröndahl et al., 2018). Similarly, we train three LSTM models with
different input embeddings: word embedding (LSTM-W), character embedding
(LSTM-C), and character-bigram embedding (LSTM-B).

  * •

HybridCNN: We replicate the HybridCNN model proposed by Park and Fung (Park
and Fung, 2017) for comparison. The HybridCNN model trains CNN over both word
and character embeddings for hate speech detection.

  * •

CNN-GRU: The CNN-GRU model that was proposed in a recent study by Zhang et al.
(Zhang et al., 2018) is also replicated in our study as a baseline. The CNN-
GRU model takes word embeddings as input.

Sentiment Learning. As mentioned in Section 3.2, we learn the post’s sentiment
using the VADER (Hutto and Gilbert, 2014) sentiment analysis tool. The learned
sentiment will be used as labels (i.e, negative, positive, and neutral) to
train the sentiment representations of the post. Figure 2 shows the Sentiment
distributions of the three datasets.

Topic Modeling. In learning post’s topic representation, we set the number of
topics in WZ-LS, DT, and FOUNTA datasets to 15, 10, 15, respectively.

Training and Testing Set. In our experiments, we adopt an 80-20 split, where
for each dataset, 80% of the posts are used for training with the remaining
20% used for testing.

Evaluation Metrics. Similar to most existing hate speech detection studies, we
use micro averaging precision (Micro-Prec), recall (Micro-Rec), and F1 score
(Micro-F1) as the evaluation metrics. Micro averaging is preferred in our
experiments as there are classes imbalance in the hate speech datasets. Also,
five-fold cross-validation is used in our experiments, and the average results
of the five folds are reported.

###  5.2. Experiment Results

Table 2 shows the experiment results on WZ-LS dataset while Tables 3, 4 and 5
show that on DT, FOUNTA and COMBINED datasets respectively. In the tables, the
highest figures are highlighted in bold. We observe that DeepHate outperformed
the state-of-the-art baselines. Interestingly, we observed that the CNN and
LSTM models with word embedding perform better than character-level embeddings
input, suggesting that the semantics information in words is able to yield
good performance for hate speech detection. The performance of character-level
bi-gram embedding is worse among the three types of input feature embeddings.
A possible reason may be that posts are short, and character-level bi-gram
embedding may be ambiguous sometimes. We also noted that the basic CNN and
LSTM with word embedding models are able to outperform some of the more
complicated models such as HybridCNN and CNN-GRU in some occasions.

It is worth noting that there are differences between the results of HybridCNN
and CNN-GRU models in our experiments and the results that were reported in
previous studies (Park and Fung, 2017; Zhang et al., 2018). For instance,
previous studies for HybridCNN (Park and Fung, 2017) and CNN-GRU (Zhang et
al., 2018) had conducted experiments on the WZ-LS dataset. However, we did not
cite the previous scores directly as some of the tweets in WZ-LS has been
deleted. Similarly, CNN-GRU was also previously applied to the DT dataset.
However, in the previous work (Zhang et al., 2018), the researchers have cast
the problem into binary classification by re-labeling the offensive tweets as
non-hate. In our experiment, we perform the classification based on the
original DT dataset (Davidson et al., 2017). Therefore, we replicated the
HybridCNN and CNN-GRU models and applied them to the updated WZ-LS dataset and
original DT dataset.

###  5.3. Ablation Study

Our proposed DeepHate model is made up of several components. In this
evaluation, we perform an ablation study to investigate the effects of
different latent post representations. Specifically, we compare the following
settings:

  * •

Semantic: We apply the DeepHate model using only the post’s latent semantic
representation $x^{p}_{w}$.

  * •

Topic+Semantic: We apply the DeepHate model using only the post’s latent
semantic representation $x^{p}_{w}$ and topic representation $x^{p}_{t}$.

  * •

Sentiment+Semantic: We apply the DeepHate model using only the post’s latent
semantic representation $x^{p}_{w}$ and sentiment representation $x^{p}_{s}$.

  * •

DeepHate: Our original DeepHate model which utilized post’s semantic, topic,
and sentiment representations.

Table 6. Performance of various DeepHate components on WZ-LS dataset

Model | Micro-Prec | Micro-Rec | Micro-F1
---|---|---|---
Semantic | 77.00 | 78.75 | 77.04
Topic+Semantic | 77.98 | 79.32 | 77.98
Sentiment+Semantic | 77.09 | 78.62 | 77.35
DeepHate | 77.95 | 79.48 | 78.19

Table 7. Performance of various DeepHate components on DT dataset

Model | Micro-Prec | Micro-Rec | Micro-F1
---|---|---|---
Semantic | 89.44 | 90.24 | 89.49
Topic+Semantic | 89.56 | 90.28 | 89.64
Sentiment+Semantic | 89.59 | 90.39 | 89.64
DeepHate | 89.97 | 90.39 | 89.92

Table 8. Performance of various DeepHate components on FOUNTA dataset

Model | Micro-Prec | Micro-Rec | Micro-F1
---|---|---|---
Semantic | 78.68 | 80.40 | 78.57
Topic+Semantic | 78.77 | 80.45 | 78.62
Sentiment+Semantic | 78.88 | 80.53 | 78.79
DeepHate | 78.95 | 80.43 | 79.09

Table 9. Performance of various DeepHate components on COMBINED dataset

Model | Micro-Prec | Micro-Rec | Micro-F1
---|---|---|---
Semantic | 92.26 | 92.23 | 92.20
Topic+Semantic | 92.33 | 92.30 | 92.27
Sentiment+Semantic | 92.32 | 92.28 | 92.25
DeepHate | 92.48 | 92.45 | 92.43

Table 6, 7, 8 and 9 shows the results of our ablation studies on WZ-LS, DT,
FOUNTA and COMBINED datasets respectively. Our post’s latent semantic
representations, which combined the latent representations learned from three
pre-trained word embeddings, outperforms the LSTM or CNN model with randomly
initialized word embedding. We postulate that the combination of three pre-
trained word embeddings allows more expressive semantic representations and
ultimately leads to better hate speech detection. The addition of the post’s
latent topic and sentiment representations to its semantic representation is
observed to improve performance, suggesting that topic and sentiment
information are useful in hate speech detection. Finally, our DeepHate model
that utilizes the post’s semantic, topic, and sentiment representations
constantly outperforms the other configurations in the four datasets,
suggesting that multi-faceted textual representations can improve hate speech
detection.

###  5.4. Salient Feature Analysis

To analyze how our DeepHate is able to extract important features from the
textual information to perform hate speech detection, we examine the most
salient sections of a single post’s latent semantic representations. We adopt
the saliency score defined by (Li et al., 2016) to measure the saliency of
input features. The score indicates how sensitive a model is to the changes in
the embedding input, i.e., in our case, how much a specific word, contributes
to the final classification decision. Figure 3, 4, and 5 show the visualized
computed saliency scores for WZ-LS, DT, and FOUNTA dataset respectively. Note
that the more salient that section of the post’s input feature representation,
the darker is its shade of red.

From the visualizations, we notice that the salient words highlighted in a
post are descriptive of its true-label. For example, in Figure 3, the words
“lady she’s awful” are highlighted for the post labeled as sexism. Similarly,
words such as “nigga” are salient in the racism post. More interesting, words
that are indicative of both sexism and racism are highlighted for posts
labeled to contain both racism and sexism. Similar observations are made for
other datasets. For instance, in the DT dataset, Figure 4 visualization shows
that offensive lexicons are differentiated from hateful ones. The salient
feature analysis demonstrates DeepHate’s capabilities to extracting critical
textual features for hate speech detection.

Label | Example Post
---|---
Sexism |
Racism |
Both |

Figure 3. Visualized saliency scores on posts in WZ-LS dataset.

Label | Example Post
---|---
Offensive |
Hate |

Figure 4. Visualized saliency scores on posts in DT dataset.

Label | Example Post
---|---
Abusive |
Hate |

Figure 5. Visualized saliency scores on posts in FOUNTA dataset.

The extracted critical textual features also provide some form of explanation
to DeepHate’s classification decision; it highlights the keywords which may
suggest some form of hate or abusive language in the text. Besides achieving
excellent detection performances, hate speech detection models should also be
explainable to aid the content moderator in providing provide reasons for
content removal. To the best of our knowledge, this is the first deep-
learning-based hate speech detection study that supports explainability.

###  5.5. Case Studies

To gain a better understanding of the difference between DeepHate and baseline
models, we qualitatively evaluate the models by examining some example posts
and their classification results. Table 10, 11, and 12 show example posts from
WZ-LS, DT, and FOUNTA datasets respectively. In each example post, we display
the true label and predicted labels from DeepHate and the best baseline, CNN-
GRU. The correct predictions are marked in green font, while the incorrect
predictions are represented in red font.

From the example posts, we observed that the posts’ sentiments have a profound
impact on DeepHate’s hate speech detection performance. For instance, for
posts with “normal” or “non-hate” true labels speech (e.g., post 1, 9, 15),
DeepHate is correctly classified the posts while CNN-GRU made incorrect
predictions. We postulate that sentiment information, i.e., neural sentiment,
utilized by DeepHate has helped the model to minimize false negatives, i.e.,
labeling non-hate speech as hate speeches. However, the dependent on sentiment
information also exposes a limitation of Deephate model: the inaccuracy of the
sentiment information may have an adverse effect on DeepHate’s performance.
For example, in post 5, 11, 12, 16, 17, the posts’ sentiments are labeled as
neural, and DeepHate has wrongly classified these posts. The sentiment labels
learned using VADER model are also inaccurate as examining the posts’ text
clearly suggests that these are posts with negative sentiment. Thus, for
future work, we will explore utilizing or designing better sentiment
classification techniques to boost DeepHate performance in hate speech
detection.

Table 10. Example WZ-LS dataset’s posts evaluated using DeepHate and best baseline (i.e., CNN-GRU) S/N | Post | True Label | CNN-GRU | DeepHate
---|---|---|---|---
1 |  Kats face the only thing worse than her cooking. | Neither | Sexist |  Neither (Sentiment: Neural)
2 |  I’m annoyed to have missed this live need some b*tchy. | Sexist | Neither |  Sexist (Sentiment: Negative)
3 |  Man w*f is this d*mn ass nig*a doing on that d*mn commercial. | Racist | Sexist |  Racist (Sentiment: Negative)
4 |  Kat is f*cking c*nt. | Sexist | Sexist |  Neither (Sentiment: Negative)
5 |  It’s not the only thing. She’s done lot of, I’m guessing. | Sexist | Neither |  Neither (Sentiment: Neural)
6 |  Would slap the smirk off Kat’s face. | Neither | Sexist |  Sexist (Sentiment: Neural )
Table 11. Example DT dataset’s posts evaluated using DeepHate and best baseline (i.e., CNN-GRU) S/N | Post | True Label | CNN-GRU | DeepHate
---|---|---|---|---
7 |  B*tch shut the f*ck up goddam your sl*t b*tch wh*re nig*a. | Hate | Offensive |  Hate (Sentiment: Negative)
8 |  Stop looking p*rn fag. | Offensive | Non-hate |  Offensive (Sentiment: Negative)
9 |  Okay, I’m going to say it once comb yer beards. | Non-hate | Offensive |  Non-hate (Sentiment: Neural)
10 |  Why do people even talk about white privilege when the majority of food stamp recipients are white people. | Hate | Hate |  Non-hate (Sentiment: Neural)
11 |  And f*ck you too ya, little b*tch. You look like Mexican sucking c*ck in ur profile. | Hate | Offensive |  Offensive (Sentiment: Neural)
12 |  The nig is fun to watch. Got to admit Ebola boy can speak broken Spanish too, lol. | Hate | Offensive |  Offensive (Sentiment: Neural)
Table 12. Example FOUNTA dataset’s posts evaluated using DeepHate and best baseline (i.e., CNN-GRU) S/N | Post | True Label | CNN-GRU | DeepHate
---|---|---|---|---
13 |  Hate this b*tch with all the hate on the world. | Hate | Abusive |  Hate (Sentiment: Negative)
14 |  Slow replies annoy the hell out of me. | Abusive | Normal |  Abusive (Sentiment: Negative)
15 |  We have the worst maternal mortality rate of any modern country and that will kill women. What is wrong with them? | Normal | Hate |  Normal (Sentiment: Neural)
16 |  Don’t like to hear women call their baby their little men. This is just another gender storm in cup. | Normal | Normal |  Abusive (Sentiment: Neural)
17 |  Looks like both the Chinese and the India news media have temporary amnesia. | Hate | Normal |  Abusive (Sentiment: Neural)
18 |  Netizens are so f*ckin annoying and dumb. | Hate | Abusive |  Abusive (Sentiment: Negative)

##  6\. Conclusion and Future Work

In this paper, we proposed a novel deep learning framework known as DeepHate,
which utilized multi-faceted text representations for automatic hate speech
detection. We evaluated Deephate on three publicly available real-world
datasets, and our extensive experiments have shown that Deephate outperformed
the state-of-the-art baselines. We have also empirically analyzed the Deephate
model and provided insights into the salient features that best aided in
detecting hate speech in online social platforms. Our salient feature analysis
provided some form of explanation to Deephate’s hate classification decision.
For future works, we will like to incorporate non-textual features into
Deephate model and improve the posts’ sentiment and topic representations with
more advanced techniques.

## References

  * (1)
  * Agrawal and Awekar (2018) Sweta Agrawal and Amit Awekar. 2018.  Deep learning for detecting cyberbullying across multiple social media platforms. In _European Conference on Information Retrieval_. Springer, 141–153.
  * Arango et al. (2019) Aymé Arango, Jorge Pérez, and Barbara Poblete. 2019\.  Hate Speech Detection is Not as Easy as You May Think: A Closer Look at Model Validation. In _Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval_. 45–54.
  * Badjatiya et al. (2017) Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. 2017.  Deep learning for hate speech detection in tweets. In _Proceedings of the 26th International Conference on World Wide Web Companion_. 759–760.
  * Balasubramanyan and Cohen (2013) Ramnath Balasubramanyan and William W Cohen. 2013.  Regularization of latent variable models to obtain sparsity. In _Proceedings of the 2013 SIAM International Conference on Data Mining_. SIAM, 414–422.
  * Blei (2012) David M Blei. 2012\.  Probabilistic topic models.  _Commun. ACM_ 55, 4 (2012), 77–84.
  * Blei et al. (2003) David M Blei, Andrew Y Ng, and Michael I Jordan. 2003\.  Latent dirichlet allocation.  _Journal of machine Learning research_ 3, Jan (2003), 993–1022.
  * Bloomberg (2019) Bloomberg. 2019\.  _Twitter, Facebook Join Global Pledge to Fight Hate Speech Online_.  Retrieved February 11, 2020 from https://www.bloomberg.com/news/articles/2019-05-15/macron-ardern-to-meet-twitter-facebook-google-on-hate-speech
  * Cambridge ([n.d.]) Cambridge. [n.d.].  _Hate Speech_.  Retrieved February 11, 2020 from https://dictionary.cambridge.org/dictionary/english/hate-speech
  * Chatzakou et al. (2017) Despoina Chatzakou, Nicolas Kourtellis, Jeremy Blackburn, Emiliano De Cristofaro, Gianluca Stringhini, and Athena Vakali. 2017\.  Mean birds: Detecting aggression and bullying on twitter. In _Proceedings of the 2017 ACM on web science conference_. 13–22.
  * Chen et al. (2012) Ying Chen, Yilu Zhou, Sencun Zhu, and Heng Xu. 2012\.  Detecting offensive language in social media to protect adolescent online safety. In _2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing_. IEEE, 71–80.
  * Davidson et al. (2017) Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017.  Automated hate speech detection and the problem of offensive language. In _Eleventh international aaai conference on web and social media_.
  * Djuric et al. (2015) Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. 2015.  Hate speech detection with comment embeddings. In _Proceedings of the 24th international conference on world wide web_. 29–30.
  * Facebook ([n.d.]) Facebook. [n.d.].  _Hate Speech_.  Retrieved February 11, 2020 from https://www.facebook.com/communitystandards/hate_speech
  * Fortuna and Nunes (2018) Paula Fortuna and Sérgio Nunes. 2018.  A survey on automatic detection of hate speech in text.  _ACM Computing Surveys (CSUR)_ 51, 4 (2018), 1–30.
  * Founta et al. (2019) Antigoni Maria Founta, Despoina Chatzakou, Nicolas Kourtellis, Jeremy Blackburn, Athena Vakali, and Ilias Leontiadis. 2019.  A unified deep learning architecture for abuse detection. In _Proceedings of the 10th ACM Conference on Web Science_. 105–114.
  * Founta et al. (2018) Antigoni Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018.  Large scale crowdsourcing and characterization of twitter abusive behavior. In _Twelfth International AAAI Conference on Web and Social Media_.
  * Gambäck and Sikdar (2017) Björn Gambäck and Utpal Kumar Sikdar. 2017.  Using convolutional neural networks to classify hate-speech. In _Proceedings of the first workshop on abusive language online_. 85–90.
  * Giachanou and Crestani (2016) Anastasia Giachanou and Fabio Crestani. 2016.  Like it or not: A survey of twitter sentiment analysis methods.  _ACM Computing Surveys (CSUR)_ 49, 2 (2016), 1–41.
  * Goldberg (2016) Yoav Goldberg. 2016\.  A primer on neural network models for natural language processing.  _Journal of Artificial Intelligence Research_ 57 (2016), 345–420.
  * Goodfellow et al. (2016) Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016\.  Deep feedforward networks.  _Deep learning_ (2016), 168–227.
  * Gröndahl et al. (2018) Tommi Gröndahl, Luca Pajola, Mika Juuti, Mauro Conti, and N Asokan. 2018.  All You Need is” Love” Evading Hate Speech Detection. In _Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security_. 2–12.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural computation_ 9, 8 (1997), 1735–1780.
  * Hong and Davison (2010) Liangjie Hong and Brian D Davison. 2010.  Empirical study of topic modeling in twitter. In _Proceedings of the first workshop on social media analytics_. 80–88.
  * Hutto and Gilbert (2014) Clayton J Hutto and Eric Gilbert. 2014.  Vader: A parsimonious rule-based model for sentiment analysis of social media text. In _Eighth international AAAI conference on weblogs and social media_.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ (2014).
  * Li et al. (2016) Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2016\.  Visualizing and Understanding Neural Models in NLP. In _NAACL:HLT_.
  * Mathew et al. (2019) Binny Mathew, Ritam Dutt, Pawan Goyal, and Animesh Mukherjee. 2019\.  Spread of hate speech in online social media. In _Proceedings of the 10th ACM Conference on Web Science_. 173–182.
  * Mehdad and Tetreault (2016) Yashar Mehdad and Joel Tetreault. 2016.  Do characters abuse more than words?. In _Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue_. 299–303.
  * Mikolov et al. (2018) Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018\.  Advances in Pre-Training Distributed Word Representations. In _Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018)_.
  * Nobata et al. (2016) Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016.  Abusive language detection in online user content. In _Proceedings of the 25th international conference on world wide web_. 145–153.
  * Papegnies et al. (2017) Etienne Papegnies, Vincent Labatut, Richard Dufour, and Georges Linares. 2017.  Graph-based features for automatic online abuse detection. In _International Conference on Statistical Language and Speech Processing_. Springer, 70–81.
  * Park and Fung (2017) Ji Ho Park and Pascale Fung. 2017.  One-step and Two-step Classification for Abusive Language Detection on Twitter. In _Proceedings of the First Workshop on Abusive Language Online_. 41–45.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014.  Glove: Global vectors for word representation. In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_. 1532–1543.
  * Relia et al. (2019) Kunal Relia, Zhengyi Li, Stephanie H Cook, and Rumi Chunara. 2019\.  Race, ethnicity and national origin-based discrimination in social media and hate crimes across 100 US cities. In _Proceedings of the International AAAI Conference on Web and Social Media_ , Vol. 13. 417–427.
  * Schmidt and Wiegand (2017) Anna Schmidt and Michael Wiegand. 2017.  A survey on hate speech detection using natural language processing. In _Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media_. 1–10.
  * Singh et al. (2017) Vivek K Singh, Souvick Ghosh, and Christin Jose. 2017\.  Toward multimodal cyberbullying detection. In _Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems_. 2090–2099.
  * Tang et al. (2014) Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014\.  Learning sentiment-specific word embedding for twitter sentiment classification. In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 1555–1565.
  * Tang et al. (2019) Min Tang, Jiaran Cai, and Hankz Hankui Zhuo. 2019.  Multi-Matching Network for Multiple Choice Reading Comprehension. In _The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019_. 7088–7095.
  * Times (2019) Times. 2019.  _Facebook Says It’s Removing More Hate Speech Than Ever Before. But There’s a Catch_.  Retrieved February 11, 2020 from https://time.com/5739688/facebook-hate-speech-languages/
  * Twitter ([n.d.]) Twitter. [n.d.].  _Hateful conduct policy_.  Retrieved February 11, 2020 from https://help.twitter.com/en/rules-and-policies/hateful-conduct-policy
  * Waseem (2016) Zeerak Waseem. 2016\.  Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In _Proceedings of the first workshop on NLP and computational social science_. 138–142.
  * Waseem and Hovy (2016) Zeerak Waseem and Dirk Hovy. 2016.  Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In _Proceedings of the NAACL student research workshop_. 88–93.
  * Wieting et al. (2015) John Wieting, Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2015.  From paraphrase database to compositional paraphrase model and back.  _Transactions of the Association for Computational Linguistics_ 3 (2015), 345–358.
  * Williams (2019) Matthew Williams. 2019\.  _The connection between online hate speech and real-world hate crime_.  Retrieved February 11, 2020 from https://blog.oup.com/2019/10/connection-between-online-hate-speech-real-world-hate-crime/
  * Xiang et al. (2012) Guang Xiang, Bin Fan, Ling Wang, Jason Hong, and Carolyn Rose. 2012.  Detecting offensive tweets via topical feature discovery over a large scale twitter corpus. In _Proceedings of the 21st ACM international conference on Information and knowledge management_. 1980–1984.
  * Yang et al. (2016) Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016\.  Hierarchical attention networks for document classification. In _Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies_. 1480–1489.
  * Zhang et al. (2018) Ziqi Zhang, David Robinson, and Jonathan Tepper. 2018\.  Detecting hate speech on twitter using a convolution-gru based deep neural network. In _European semantic web conference_. Springer, 745–760.
  * Zhao et al. (2011) Wayne Xin Zhao, Jing Jiang, Jianshu Weng, Jing He, Ee-Peng Lim, Hongfei Yan, and Xiaoming Li. 2011.  Comparing twitter and traditional media using topic models. In _European conference on information retrieval_. Springer, 338–349.
  * Zhou et al. (2015) Chunting Zhou, Chonglin Sun, Zhiyuan Liu, and Francis Lau. 2015\.  A C-LSTM neural network for text classification.  _arXiv preprint arXiv:1511.08630_ (2015).