# K-MHaS: A Multi-label Hate Speech Detection Dataset
in Korean Online News Comment

Jean Lee1 Taejun Lim1 Heejun Lee3 Bogeun Jo3
Yangsok Kim4 Heegeun Yoon5 Soyeon Caren Han1,2
The University of Sydney1 The University of Western Australia2
BigWave AI3 Keimyung University4 National Information Society Agency5
{jean.lee, tlim6535<EMAIL_ADDRESS>
{leej3471<EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>Corresponding author
<EMAIL_ADDRESS>
###### Abstract

Online hate speech detection has become an important issue due to the growth
of online content, but resources in languages other than English are extremely
limited. We introduce K-MHaS111The dataset is available at
https://github.com/adlnlp/K-MHaS., a new multi-label dataset for hate speech
detection that effectively handles Korean language patterns. The dataset
consists of 109k utterances from news comments and provides a multi-label
classification using 1 to 4 labels, and handles subjectivity and
intersectionality. We evaluate strong baselines on K-MHaS. KR-BERT with a sub-
character tokenizer outperforms others, recognizing decomposed characters in
each hate speech class.

##  1 Introduction

The growth of online content including social media (Zampieri et al., 2020),
news comments (Gao and Huang, 2017), Wikipedia (Wulczyn et al., 2017), and in-
game chat (Weld et al., 2021) presents challenges in detecting hate speech
using advanced Natural Language Processing. Hate speech is language that
attacks or diminishes individuals or groups based on certain characteristics
such as physical appearance, religion, gender, or other attributes, and it can
occur across different linguistic styles (Fortuna and Nunes, 2018). Hate
speech detection is intrinsically a complex task (Wang et al., 2020) due to
the fuzzy boundary with other overlapping concepts such as abusive language
(Nobata et al., 2016), toxic comments (Wulczyn et al., 2017), or offensive
language (Davidson et al., 2017).

Recently, the rise in popularity of Korean TV, movies, and music (e.g. Squid
Game, BTS) has led to many young people showing an interest in learning
Korean. This phenomenon could result in exposure to harmful content and hate
speech in Korean. However, (1) the most common language in hate speech
research is English and only limited resources are available in other
languages such as Arabic (Mubarak et al., 2017), Dutch (Caselli et al., 2021),
and Korean (Moon et al., 2020). In addition, most datasets are annotated (2)
using a single label classification of particular aspects, even though the
subjectivity of hate speech cannot be explained with a mutually exclusive
annotation scheme.

We propose K-MHaS, a Korean multi-label hate speech detection dataset that
allows overlapping labels associated with intersectionality, a concept from
sociology that identifies combined attributes (Crenshaw, 1989). Our dataset
consists of 109,692 utterances from Korean online news comments, labeled with
8 fine-grained hate speech classes. K-MHaS is compatible with previous work on
hate speech in other languages, by providing binary classification and multi-
label classification from 1(one) to 4(four) labels.

We investigate the K-MHaS dataset by analyzing label distribution, keywords,
and label pairs. In addition, we provide strong baseline pre-trained language
models using Multilingual-BERT, KoELECTRA, KoBERT, and KR-BERT, and compare
the results using six metrics for multi-label classification tasks. Overall,
the KoELECTRA model achieves the best performance for all labels, indicating
the effects of the pre-training data source. The KR-BERT with a sub-character-
level tokenizer outperforms the others on several label pairs, showing that
decomposing various Korean characters is essential for the task. Our
contribution can be summarized as follows:

Publication | Language | Source | Data size | Labels | M-label
---|---|---|---|---|---
Waseem and Hovy (2016) | English | Twitter | 16.2k | Sexism, Racism, Neither | N
Davidson et al. (2017) | English | Twitter | 24.8k | Hate Speech, Offensive, Neither | N
Wulczyn et al. (2017) | English |  | Wikipedia
---
comments
115k |  | Toxic, Severe Toxic, Obscene, Threat,
---
Insult, Identity Hate, Neutral
Y
Ibrohim and Budi (2019) | Indonesian | Twitter | 11k |  | (a) Individual, Group
---
(b) Religion, Race, Pysical, Gender, Other
(c) Weak, Moderate, Strong Hate Speech
P
Fortuna et al. (2019) | Portuguese | Twitter | 5.6k |  | (a) Hate Speech, Not Hate Speech
---
(b) Sexism, Body, Origin, Homophobia, Racism,
Ideology, Religion, Health, Other-Lifestyle
P
Ousidhoum et al. (2019) |  | English
---
French
Arabic
Twitter |  | 6k (EN)
---
4k (FR)
3k (AR)
| Labels for five different aspects
---
(a) Directness, (b) Hostility, (c) Target,
(d) Group, and (e) Annotator
P
Moon et al. (2020) | Korean |  | News
---
comments
9k |  | (a) Hate Speech, Offensive, None
---
(b) Gender, Others, None
N
Ours | Korean |  | News
---
comments
109k |  | (a) Hate Speech, Not Hate Speech
---
(b) Politics, Origin, Physical, Age, Gender
Religion, Race, Profanity, Not Hate Speech
Y
Table 1: Comparison of datasets. A "M-label" indicates a multi-label
annotation scheme that allows overlapping labels for intersectionality (P =
partially applied). The (a) - (e) indicates a layer containing a single label
from each aspect.

  * •

We propose a large size Korean multi-label hate speech detection dataset that
represents Korean language patterns effectively;

  * •

We propose a multi-label hate speech annotation scheme, which can handle the
subjectivity of hate speech and the intersectionality;

  * •

We evaluate strong baseline experiments on our dataset using Korean-BERT-based
language models with six different metrics.

##  2 Korean Multi-label Hate Speech Detection Dataset (K-MHaS)

Our dataset is based on the Korean online news comments available on Kaggle
222https://www.kaggle.com/datasets/junbumlee/kcbert-pretraining-corpus-korean-
news-comments and Github 333https://github.com/kocohub/korean-hate-speech. The
unlabeled raw data was collected between January 2018 and June 2020. In order
to curate the data, we randomly select more than 109,692 news comments. Our
data preprocessing is designed to tokenize a Korean character and to filter
the length. We remove URLs and bad characters (e.g. U+1100 to U+11FF - Hangul
Jamo) using regular expressions while keeping uppercase and lowercase letters
in English and emoji. We discard sentences with fewer than 10 characters as it
is often only one word. For the data derived from online comments, we
normalized repeated characters by truncating their number of consecutive
repetitions to two.

### Multi-label Annotation

We consider a multi-label annotation scheme in order to deliver fine-grained
hate speech categories and intersectionality from the overlapping labels. The
annotation scheme has two layers: (a) binary classification (‘Hate Speech’ or
‘Not Hate Speech’) and (b) fine-grained classification (8 labels or ‘Not Hate
Speech’). For the fine-grained classification, a ‘Hate Speech’ class from the
binary classification is broken down into 8 classes associated with the hate
speech category 444 Fine-grained labels (matching in Korean) : Politics
(정치성향차별), Origin (출신차별), Physical (외모차별), Age (연령차별), Gender (성차별), Religion
(종교차별), Race (인종차별), and Profanity (혐오욕설) . As shown in Table 1, this scheme
allows non-exclusive concepts, accounting for the overlapping shades of given
categories. We select the 8 hate speech classes in order to reflect the social
and historical context as the nature of hate speech is different in each
language (Kang et al., 2020). For example, the ‘politics’ class is chosen due
to a significant influence on the style of Korean hate speech.

### Annotation Instructions

Given the subjectivity of the task and our annotation scheme, we perform a
preliminary round to identify the topics of hate speech and develop annotation
instructions. We begin with the common categories of hate speech found in
literature and match the keywords for each category. After the preliminary
round, we investigate the results to merge or remove labels in order to
provide the most representative subtype labels of hate speech contextual to
the cultural background. Our annotation instruction includes the criteria as
follows: Politics: hate speech based on political stance; Origin: hate speech
based on place of origin or identity; Physical: hate speech based on physical
appearance (e.g. body, face) or disability; Age: hate speech based on age;
Gender: hate speech based on gender or sexual orientation (e.g. woman,
homosexual); Religion: hate speech based on religion; Race: hate speech based
on ethnicity; Profanity: hate speech in the form of swearing, cursing,
cussing, obscene words, or expletives; or an unspecified hate speech category
from above; and Not Hate Speech.

Our annotation instructions explain a two-layered annotation to (a)
distinguish hate and not hate speech, and (b) the categories of hate speech.
Annotators are requested to consider given keywords or alternatives of each
category within social, cultural, and historical circumstances. For example, a
comment using the word “women” is not hate speech, whereas, if it is critical
of “women” or uses language that attacks the group, it is classified as
‘gender’. Notably, we annotate multi-labels if a comment includes several hate
speech categories. Since hate speech can be varied, any comments in the form
of swearing or cursing are marked as ‘profanity’. For instance, a comment
containing hate speech about appearance, political stance, and gender in
profane language (e.g. "fuck you ugly communist bitch.")555 (Korean) “면상도 개
조가치 생겼네 개빨갱이년”  is labeled within ‘physical’, ‘politics’, ‘gender’ and
‘profanity’ classes.

Figure 1: Overview of Annotation Process.

### Annotation Process

Five native speakers were recruited for manual annotation in both the
preliminary and main rounds. During the preliminary round, we facilitated the
annotation instructions by conducting an annotators’ discussion and providing
some examples of keywords for each class. As shown in Figure 1, we introduced
an iterative process that enables faster annotation in the main round. We
provided an ‘uncertain’ additional field that was used for the unspecified
label in annotation guidelines or when the annotator had difficulties in
choosing labels. Any ‘uncertain’ labeled data was flagged by individual
annotators, then reviewed by five annotators. The final labels were chosen
based on the majority vote, and the annotation guidelines were updated to
handle similar cases. Additionally, the other labeled data was reviewed, in
line with the annotation guideline by two random annotators for the final
dataset. The inter-annotator agreement returns an average Cohen Kappa score of
0.892, indicating substantial agreement (Siegel and Castellan, 1988).

##  3 Dataset Analysis

K-MHaS dataset contains 109,692 comments as shown in Table 2. For binary
classification, the proportion of the ‘hate speech’ (45.7%) and ‘not hate
speech’ (54.3%) satisfies data balancing. The ‘hate speech’ label consists of
a single label (33.2%) and multi-labels (12.4%), containing from 2 to 4
labels. Other hate speech datasets reviewed have an approximate ratio of ‘hate
speech’ to ‘not hate speech’ of around 40% (Vidgen and Derczynski, 2020). Our
dataset is consistent with this figure, where the ‘hate speech’ in a single
label to ‘not hate speech’ ratio is 38%.

Label Types | Count (%)
---|---
Total Utterances | 109,692 (100%)
Multi-label (Hate Speech) | 1 label (Single) | 36,470 (33.2%)
2 labels | 12,073 (11.0%)
3 labels | 1,440 (1.3%)
4 labels | 94 (0.1%)
Not Hate Speech | 59,615 (54.3%)
Table 2: Dataset Statistics. The total is the combination of all ‘hate speech’ and ‘not hate speech’ label. Together the ‘hate speech’ label makes up 45.7% of the data. Class | Count - Single (%) | Count - Multi (%)
---|---|---
Politics | 6,931 (19.0%) | 4,961 (17.2%)
Origin | 5,739 (15.7%) | 4,458 (15.5%)
Physical | 5,443 (14.9%) | 3,364 (11.7%)
Age | 4,192 (11.5%) | 3,178 (11.0%)
Gender | 3,348 (9.2%) | 4,696 (16.3%)
Religion | 1,862 (5.1%) | 513 (1.8%)
Race | 160 (0.4%) | 163 (0.6%)
Profanity | 8,795 (24.1%) | 7,509 (26.0%)
Table 3: Fine-grained label distributions on hate speech labels. A ‘not hate
speech’ label is not included. A single means 1 label and a multi is the sum
of 2, 3, and 4 labels. A multi-labeled data counts each overlapping class.

### Label Distribution

Table 3 shows the fine-grained label distribution across our K-MHaS. For both
single (s) and multi-label (m) distribution, the ‘profanity’ class (24.1%-s,
26.0%-m) is more frequent than any other class, indicating that swear words
are critical for detecting hate speech. Also, the ‘religion’ (5.1%-s, 1.8%-m)
and ‘race’ (0.4%-s, 0.6%-m) classes are the smallest portions in both
distributions, which are significantly more common in other hate speech
datasets. This difference could be because Korea is a highly homogenous
monoculture with little variation in race and religion (Kang et al., 2020).
Interestingly, the ‘gender’ class (16.3%) occurs at almost twice the frequency
in a multi-label distribution, compared to a single label distribution (9.2%).
This indicates that gender-based hate speech is used extensively in combined
aspects.

### Keyword Analysis

To understand the lexical aspects, we list the top 5 keywords for each hate
speech category in Table 4, identifying which tokens are highly associated
with each class. In the ‘politics’ class, we find that far-right extremism is
dominant, and new tokens such as “catastrophe” [jae ang](재앙) appears related
to the former president’s given name ([jae in]) as the two words are near-
homophones. Across all classes, one-word tokens are often used in their stem
form to modify the meanings of other words. For example, a token [teul] (틀)
comes from the word “denture” [teulni] (틀니) which is used as an offensive
reference to the elderly. In addition, one-word tokens can be used as a prefix
(e.g. “dog” [gae] (개)) or a suffix (e.g. “insect” [chung] (충)), and combined
with other neutral words to create a new offensive term.

Rank | Politics | Origin | Physical | Age
---|---|---|---|---
1 | 재앙 (1427) | 짱깨 (615) | 얼굴 (962) | 틀 (1918)
2 | 문재인 (951) | 전라도 (596) | 돼지 (772) | 나이 (599)
3 | 좌파 (464) | 중국 (539) | 여자 (294) | 노인 (139)
4 | 좌빨 (402) | 쪽 (448) | 성형 (216) | 충 (112)
5 | 빨갱이 (367) | 짱 (446) | 관상 (183) | 놈 (106)
Rank | Gender | Religion | Race | Profanity
1 | 여자 (1704) | 개독 (526) | 흑인 (44) | 새끼 (1103)
2 | 남자 (990) | 신천지 (460) | 백인 (32) | 년 (1014)
3 | 페미 (172) | 사이비 (409) | 양키 (32) | 지랄 (564)
4 | 맘충 (138) | 종교 (305) | 깜둥이 (19) | 개 (459)
5 | 여성 (134) | 예수 (227) | 놈 (13) | 놈 (404)
Table 4: Top 5 keywords associated with each fine-grained label. The number in
brackets is the token count. The keyword analysis is from the total dataset
and different from some examples in annotation guidelines.

(a) Label types

(b) Class types I

(c) Class types II

Figure 2: Average utterance length. (a) label types from 1 to 4 labels. 8
class types (b) in a single label and (c) in multi-labels.

### Label Pair Analysis

Figure 2 shows the average length of utterance by label count and class type.
The total average length of an utterance is 33 tokens. An increase in the
number of labels shows an increasing trend in utterance length, indicating
that multi-labeled hate speech contains more linguistic content. The ‘gender’
class has relatively longer lengths (43 tokens) compared to other classes in a
single label, whereas all multi-labels utterances have a similar length. This
indicates that the gender class has different linguistic features.

Model | F1 (macro) | F1 (micro) | F1 (weighted) | E.M. | AUC | H.L. (↓)
---|---|---|---|---|---|---
BERT | 0.6912 | 0.8139 | 0.8119 | 0.7579 | 0.8878 | 0.0464
KoELECTRA | 0.7245 | 0.8493 | 0.8480 | 0.7994 | 0.9122 | 0.0380
KoBERT | 0.7651 | 0.8413 | 0.8424 | 0.7926 | 0.9083 | 0.0401
KR-BERT-c | 0.7444 | 0.8500 | 0.8470 | 0.7901 | 0.9028 | 0.0368
KR-BERT-s | 0.7245 | 0.8445 | 0.8437 | 0.7825 | 0.9076 | 0.0390

Table 5: Overall multi-label classification performance on K-MHaS for the five
baseline models at epoch 4 (E.M.:Exact Match, H.L.:Hamming Loss / KR-BERT-*: c
= character-level, s = sub-character-level)

Metric | # labels | BERT | KoELECTRA | KoBERT | KR-BERT-c | KR-BERT-s
---|---|---|---|---|---|---
F1 (micro) | 1 | 0.8190 | 0.8490 | 0.8320 | 0.8553 | 0.8392
2 | 0.8043 | 0.8612 | 0.8854 | 0.8405 | 0.8703
3 | 0.7517 | 0.7987 | 0.8290 | 0.7827 | 0.8329
4 | 0.7093 | 0.7044 | 0.6832 | 0.7439 | 0.7771

Table 6: A breakdown of F1 for multi-label classification from 1 to 4 labels.

##  4 Experiment Setup

### Data Preparation

We split the data into train/test in the proportions of 0.8/0.2. From the
training set, we randomly select 0.1 as a validation set (78,977/8,776/21,939
samples for train/val/test sets, preserving the class proportion). The data
passed to the models is the preprocessed sentences and binary label vectors.

Label Pairs | # pairs | F1 (macro) | F1 (micro)
---|---|---|---
Overall Performance (F1) | BERT | KoELECTRA | KoBERT | KR-BERT-c | KR-BERT-s | BERT | KoELECTRA | KoBERT | KR-BERT-c | KR-BERT-s
0.6912 | 0.7245 | 0.7651 | 0.7444 | 0.7245 | 0.8139 | 0.8493 | 0.8413 | 0.8500 | 0.8445
Profanity & Politics | 323 | 0.1959 | 0.2045 | 0.2072 | 0.2013 | 0.2034 | 0.8379 | 0.8853 | 0.9010 | 0.8687 | 0.8616
Profanity & Physical | 311 | 0.1931 | 0.2061 | 0.2115 | 0.2099 | 0.2121 | 0.8393 | 0.9096 | 0.9331 | 0.9369 | 0.9334
Profanity & Origin | 269 | 0.1887 | 0.1989 | 0.1987 | 0.1961 | 0.2050 | 0.8144 | 0.8731 | 0.8729 | 0.8661 | 0.9070
Gender & Origin | 242 | 0.2035 | 0.2017 | 0.2134 | 0.1905 | 0.2141 | 0.8920 | 0.8780 | 0.9440 | 0.8354 | 0.9494
Politics & Origin | 224 | 0.1962 | 0.1976 | 0.1991 | 0.1872 | 0.2013 | 0.8666 | 0.8714 | 0.8846 | 0.8295 | 0.8918
Age & Politics | 222 | 0.1996 | 0.2114 | 0.2104 | 0.1964 | 0.2014 | 0.8765 | 0.9329 | 0.9357 | 0.8734 | 0.8878
Gender & Profanity | 181 | 0.1895 | 0.1991 | 0.1957 | 0.1911 | 0.2054 | 0.8157 | 0.8715 | 0.8542 | 0.8450 | 0.8994
Gender & Physical | 177 | 0.1160 | 0.1833 | 0.1958 | 0.1813 | 0.1953 | 0.4562 | 0.7867 | 0.8455 | 0.8045 | 0.8585
Age & Profanity | 132 | 0.1908 | 0.2102 | 0.2139 | 0.2063 | 0.2043 | 0.8414 | 0.9240 | 0.9459 | 0.9105 | 0.9095
Gender & Age | 130 | 0.1738 | 0.1781 | 0.1903 | 0.1517 | 0.1339 | 0.7277 | 0.7452 | 0.8159 | 0.6368 | 0.5686

Table 7: F1 score for the top 10 two-label pairs on the K-MHaS dataset for the
five pre-trained language models at epoch 4 (# total label pairs = 2,439 / KR-
BERT-*: c = character-level tokenizer, s = sub-character-level tokenizer).

### Baselines

We select four baselines. 1) Multi-BERT (Wolf et al., 2020) is pre-trained on
Wikipedia in 104 different languages. We adopted the BERT-Base, uses the
WordPiece tokenizer and contains 110M parameters and 119K vocabs. 2) KoELECTRA
(Park, 2020) is pre-trained on 34GB Korean news, Korean Wikipedia, Namuwiki
(Korean-based wiki) and Modu (Korean corpus data publicly provided by the
Korean government). The KoELECTRA-Small-v3 is used with the WordPiece
tokenizer and contains 14M parameters and 35K vocabs. 3) KoBERT (SKTBrain,
2019) is pre-trained on 54M words from Korean Wikipedia, used the
SentencePiece tokenizer, 92M parameters and 8K vocabs. 4) KR-BERT (Lee et al.,
2020) is pre-trained on 2.47GB corpus with 233M words from Korean Wikipedia
and news. We applied either (1) the character-level tokenizer or (2) the sub-
character-level tokenizer666The KR-BERT tokenization variants can be found as
follows: https://github.com/snunlp/KR-BERT#tokenization.

### Evaluation Metrics

In multi-label classification, the prediction contains a set of labels, which
means the prediction can be fully correct, partially correct, or fully
incorrect. We propose to use the widely used six metrics (Godbole and
Sarawagi, 2004) for conducting our multi-label classification, including
F1-[macro, micro, weighted], Exact Match, AUC and Hamming Loss (Sorower,
2010).

##  5 Results

### Evaluation for All Labels

The overall performance for all labels is provided in Table 5. The F1(micro)
range between 0.8139 (Multi-BERT), 0.8493 (KoELECTRA) and 0.8500 (KR-BERT-c),
while the F1(macro) scores show a range from 0.6912 (Multi-BERT) to 0.7651
(KoBERT) with 4 epochs. We observe that all baselines achieve a similar
performance, whereas Multi-BERT pre-trained on 104 languages present
relatively lower performance. The KoELECTRA obtains overall the best or second
best among six metrics, although this model has a seven times smaller
parameter size (14M) than an average of other models (99M). This indicates the
effects of the pre-training data source, considering that the KoELECTRA
includes the corpus from Namuwiki and Modu that contain modern slang and
buzzwords, while other models generally use Korean Wikipedia.

### Evaluation for Multi-labels

Table 6 shows the breakdown F1(micro) for multi-label classification from 1 to
4 labels777Further details are shown in Appendix Table 8.. A single label
task, achieving 0.8553 and 0.8490 from the KR-BERT-c and KoELECTRA,
outperforms other multi-label tasks due to domain similarity. For the multi-
label classification, KR-BERT-s achieved the best performance. It uses a sub-
character tokenizer that can decompose Hangul(Korean language) syllable
characters into sub-characters. Therefore, it provides greater granularity in
detecting hate speech words, by identifying the sub-characters from different
hate speech categories.888(e.g.) 개빠ㄹ갱이년 = 개 (”dog” - profanity) + 빠ㄹ갱이
(”communist” - politics) + 년 (”bitch” - gender).

### Evaluation for Label-pairs

Table 7 shows the F1-[macro, micro] scores for curated label pairs based on
the proportion in the 2-labels classification. It illustrates that the KR-
BERT-s model outperforms in six label pairs. In particular, it is very
effective at detecting the origin and gender pairs, achieving the highest F1
micro scores of 0.9494 across all label pairs and models. This model uses the
sub-character-level tokenizer that can decompose various Korean characters
(Hangul syllables) into sub-characters or graphemes to enable handling the
bottom consonant (e.g. "gold-digger" [kko#t#baem] 꼬#ㅊ#뱀) or initial consonant
(e.g. [k] ㅋ). This approach can detect new slang even if it is only a minor
variation from other neutral words.

##  6 Conclusion

We propose K-MHaS, a new large-sized dataset for Korean hate speech detection
with a multi-label annotation scheme. We provided extensive baseline
experiment results, presenting the usability of a dataset to detect Korean
language patterns in hate speech. In future work, the automatic hate speech
moderation and counter-speech can be expanded.

## Acknowledgements

We thank you the anonymous reviewers for their useful comments. This research
is supported by an Australian Government Research Training Program (RTP)
Scholarship.

## References

  * Caselli et al. (2021) Tommaso Caselli, Arjan Schelhaas, Marieke Weultjes, Folkert Leistra, Hylke van der Veen, Gerben Timmerman, and Malvina Nissim. 2021.  Dalc: the dutch abusive language corpus.  In _Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021)_ , pages 54–66.
  * Crenshaw (1989) Kimberlé Crenshaw. 1989.  Demarginalizing the intersection of race and sex: A black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics.  _u. Chi. Legal f._ , page 139.
  * Davidson et al. (2017) Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017.  Automated hate speech detection and the problem of offensive language.  In _Proceedings of the International AAAI Conference on Web and Social Media_ , volume 11.
  * Fortuna et al. (2019) Paula Fortuna, Joao Rocha da Silva, Leo Wanner, Sérgio Nunes, et al. 2019.  A hierarchically-labeled portuguese hate speech dataset.  In _Proceedings of the Third Workshop on Abusive Language Online_ , pages 94–104.
  * Fortuna and Nunes (2018) Paula Fortuna and Sérgio Nunes. 2018.  A survey on automatic detection of hate speech in text.  _ACM Computing Surveys (CSUR)_ , 51(4):1–30.
  * Gao and Huang (2017) Lei Gao and Ruihong Huang. 2017.  Detecting online hate speech using context aware models.  In _Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP_ , pages 260–266.
  * Godbole and Sarawagi (2004) Shantanu Godbole and Sunita Sarawagi. 2004.  Discriminative methods for multi-labeled classification.  In _Pacific-Asia conference on knowledge discovery and data mining_ , pages 22–30. Springer.
  * Ibrohim and Budi (2019) Muhammad Okky Ibrohim and Indra Budi. 2019.  Multi-label hate speech and abusive language detection in indonesian twitter.  In _Proceedings of the Third Workshop on Abusive Language Online_ , pages 46–57.
  * Kang et al. (2020) Myungkoo Kang, Marie-Orange Rivé-Lasan, Wooja Kim, and Philippa Hall. 2020.  _Hate speech in Asia and Europe: Beyond hate and fear_.  Routledge.
  * Lee et al. (2020) Sangah Lee, Hansol Jang, Yunmee Baik, Suzi Park, and Hyopil Shin. 2020.  Kr-bert: A small-scale korean-specific language model.  _ArXiv_ , abs/2008.03979.
  * Moon et al. (2020) Jihyung Moon, Won Ik Cho, and Junbum Lee. 2020.  Beep! korean corpus of online news comments for toxic speech detection.  In _Proceedings of the Eighth International Workshop on Natural Language Processing for Social Media_ , pages 25–31.
  * Mubarak et al. (2017) Hamdy Mubarak, Kareem Darwish, and Walid Magdy. 2017.  Abusive language detection on arabic social media.  In _Proceedings of the first workshop on abusive language online_ , pages 52–56.
  * Nobata et al. (2016) Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016\.  Abusive language detection in online user content.  In _Proceedings of the 25th international conference on world wide web_ , pages 145–153.
  * Ousidhoum et al. (2019) Nedjma Ousidhoum, Zizheng Lin, Hongming Zhang, Yangqiu Song, and Dit-Yan Yeung. 2019\.  Multilingual and multi-aspect hate speech analysis.  In _Proceedings of the 2019 Conference on EMNLP and the 9th IJCNLP (EMNLP-IJCNLP)_ , pages 4675–4684.
  * Park (2020) Jangwon Park. 2020.  Koelectra: Pretrained electra model for korean.  https://github.com/monologg/KoELECTRA.
  * Siegel and Castellan (1988) S. Siegel and N.J. Castellan. 1988.  _Nonparametric Statistics for the Behavioral Sciences_.  McGraw-Hill.
  * SKTBrain (2019) SKTBrain. 2019.  Kobert: Korean bert pre-trained cased.  https://github.com/SKTBrain/KoBERT.
  * Sorower (2010) Mohammad S Sorower. 2010.  A literature survey on algorithms for multi-label learning.  _Oregon State University, Corvallis_ , 18:1–25.
  * Vidgen and Derczynski (2020) Bertie Vidgen and Leon Derczynski. 2020.  Directions in abusive language training data, a systematic review: Garbage in, garbage out.  _Plos one_ , 15(12):e0243300.
  * Wang et al. (2020) Kunze Wang, Dong Lu, Caren Han, Siqu Long, and Josiah Poon. 2020.  Detect all abuse! toward universal abusive language detection models.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 6366–6376.
  * Waseem and Hovy (2016) Zeerak Waseem and Dirk Hovy. 2016.  Hateful symbols or hateful people? Predictive features for hate speech detection on Twitter.  In _Proceedings of the NAACL student research workshop_ , pages 88–93.
  * Weld et al. (2021) Henry Weld, Guanghao Huang, Jean Lee, Tongshu Zhang, Kunze Wang, Xinghong Guo, Siqu Long, Josiah Poon, and Caren Han. 2021.  Conda: a contextual dual-annotated dataset for in-game toxicity understanding and detection.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 2406–2416.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on EMNLP: System Demonstrations_ , pages 38–45. Association for Computational Linguistics.
  * Wulczyn et al. (2017) Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017.  Ex machina: Personal attacks seen at scale.  In _Proceedings of the 26th international conference on world wide web_ , pages 1391–1399.
  * Zampieri et al. (2020) Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Çağrı Çöltekin. 2020.  Semeval-2020 task 12: Multilingual offensive language identification in social media (offenseval 2020).  In _Proceedings of the Fourteenth Workshop on Semantic Evaluation_ , pages 1425–1447.

##  Appendix A Appendix

### Ethics/Broader Impact Statement

The study follows the ethical policy set out in the ACL code of
Ethics999https://www.aclweb.org/portal/content/acl-code-ethics and addresses
the ethical impact of presenting a new dataset. As described in the data
section 2, our annotated dataset is based on the online news comments data
publicly available on Kaggle and Github. All annotators were recruited from a
crowdsourcing platform. They were informed about hate speech before handling
the data. Our instructions allowed them to feel free to leave if they were
uncomfortable with the content. With respect to the potential risks, we note
that the subjectivity of human annotation would impact on the quality of the
dataset.

### The Korean language

The Korean language is morphologically rich and the character structure is
different to Latin-based language. A brief components used in the paper as
follows:

  * •

Consonant (자음) : A consonant is a sound such as ‘p’, ‘f’, ‘n’, or ‘t’ which
you pronounce by stopping the air flowing freely through your mouth.
\- initial consonant (초성)
\- bottom consonant (받침)

  * •

Vowel (모음) : A vowel is a sound such as the ones represented in writing by the
letters ‘a’, ‘e’, ‘i’, ‘o’, and ‘u’, which you pronounce with your mouth open,
allowing the air to flow through it.

  * •

Syllable (음절) : A syllable is a part of a word that contains a single vowel
sound and that is pronounced as a unit. So, for example, ‘book’ has one
syllable, and ‘reading’ has two syllables.
\- Korean romanization : (e.g. [kko#t#baem])
\- Character level : (e.g. 꽃#뱀)
\- Sub-character level : (e.g. 꼬#ㅊ#뱀)

### Implementation Details

For all baselines, we set the number of epochs as 4 and use a batch size of
32. For other hyper-parameters, we follow the configuration in the official
GitHub implementation of the baselines. The source codes or pre-trained models
for the baselines are available at the following GitHub addresses:
Multilingual BERT101010https://github.com/google-research/bert,
KoELECTRA111111https://github.com/monologg/KoELECTRA,
KoBERT121212https://github.com/SKTBrain/KoBERT and KR-
BERT131313https://github.com/snunlp/KR-BERT.

### Experiments.

A brief of tables displayed in Appendix as follows:

  * •

Table 8: a breakdown of multi-label classification performance from 1 to 4
labels;

  * •

Table 9: overall binary classification performance;

  * •

Table 10: a breakdown of binary classification performance;

  * •

Table 10: F1 score for the top 10 three-label pairs in 3-labels
classification;

  * •

Table 12: F1 score for the top 5 four-label pairs in 4-labels classification.

# Labels | Model | F1 (Macro) | F1 (Micro) | F1 (Weighted) | E.M. | AUC | H.L. (↓)
---|---|---|---|---|---|---|---
1 | BERT | 0.6666 | 0.8190 | 0.8202 | 0.7919 | 0.9011 | 0.0406
KoELECTRA | 0.6953 | 0.8490 | 0.8508 | 0.8263 | 0.9213 | 0.0341
KoBERT | 0.7321 | 0.8320 | 0.8370 | 0.8142 | 0.9110 | 0.0379
KR-BERT(w. char) | 0.7336 | 0.8553 | 0.8543 | 0.8239 | 0.9145 | 0.0318
KR-BERT(w. sub) | 0.6985 | 0.8392 | 0.8419 | 0.8062 | 0.9123 | 0.0360
2 | BERT | 0.6389 | 0.8043 | 0.8174 | 0.5580 | 0.8524 | 0.0788
KoELECTRA | 0.6777 | 0.8612 | 0.8700 | 0.6511 | 0.8934 | 0.0577
KoBERT | 0.7249 | 0.8854 | 0.8911 | 0.6794 | 0.9112 | 0.0482
KR-BERT(w. char) | 0.6748 | 0.8405 | 0.8451 | 0.5912 | 0.8735 | 0.0642
KR-BERT(w. sub) | 0.6718 | 0.8703 | 0.8723 | 0.6535 | 0.9000 | 0.0542
3 | BERT | 0.5784 | 0.7517 | 0.7522 | 0.2448 | 0.8040 | 0.1402
KoELECTRA | 0.6146 | 0.7987 | 0.7953 | 0.3310 | 0.8362 | 0.1169
KoBERT | 0.6523 | 0.8290 | 0.8251 | 0.3759 | 0.8589 | 0.1019
KR-BERT(w. char) | 0.5828 | 0.7827 | 0.7732 | 0.2828 | 0.8239 | 0.1230
KR-BERT(w. sub) | 0.6164 | 0.8329 | 0.8263 | 0.3586 | 0.8615 | 0.0996
4 | BERT | 0.4776 | 0.7093 | 0.7029 | 0.1200 | 0.7610 | 0.2222
KoELECTRA | 0.4511 | 0.7044 | 0.6639 | 0.0000 | 0.7680 | 0.2089
KoBERT | 0.4177 | 0.6832 | 0.6460 | 0.0400 | 0.7510 | 0.2267
KR-BERT(w. char) | 0.4837 | 0.7439 | 0.7226 | 0.1200 | 0.7930 | 0.1867
KR-BERT(w. sub) | 0.5068 | 0.7771 | 0.7618 | 0.1200 | 0.8120 | 0.1733

Table 8: A breakdown of multi-label classification performance from 1 to 4
labels on K-MHaS for the five pre-trained language models at epoch 4
(E.M.:Exact Match, H.L.:Hamming Loss / KR-BERT (w. *): char = character-level,
sub = sub-character-level)

Model | F1 (Macro) | F1 (Micro) | F1 (Weighted) | E.M. | AUC | H.L. (↓)
---|---|---|---|---|---|---
BERT | 0.8495 | 0.8507 | 0.8505 | 0.8507 | 0.8488 | 0.1493
KoELECTRA | 0.8756 | 0.8766 | 0.8765 | 0.8766 | 0.8750 | 0.1234
KoBERT | 0.8687 | 0.8692 | 0.8693 | 0.8692 | 0.8696 | 0.1308
KR-BERT (w. char) | 0.8846 | 0.8850 | 0.8851 | 0.8850 | 0.8862 | 0.1150
KR-BERT (w. sub) | 0.8869 | 0.8879 | 0.8877 | 0.8879 | 0.8857 | 0.1121

Table 9: Overall binary classification performance on the K-MHaS dataset for
the five pre-trained language models at epoch 4 (E.M.:Exact Match,
H.L.:Hamming Loss / KR-BERT (w. *): char = character-level, sub = sub-
character-level)

Label | Model | F1 (Macro) | F1 (Micro) | F1 (Weighted) | E.M. | H.L. (↓)
---|---|---|---|---|---|---
Hate Speech | BER | 0.4518 | 0.8243 | 0.9037T | 0.8243 | 0.1757
KoELECTRA | 0.4606 | 0.8540 | 0.9212 | 0.8540 | 0.1460
KoBERT | 0.4666 | 0.8746 | 0.9331 | 0.8746 | 0.1254
KR-BERT (w. char) | 0.4611 | 0.8558 | 0.7892 | 0.8558 | 0.1442
KR-BERT (w. sub) | 0.4724 | 0.8953 | 0.8458 | 0.8953 | 0.1047
None | BERT | 0.4662 | 0.8733 | 0.9323 | 0.8733 | 0.1267
KoELECTRA | 0.4726 | 0.8960 | 0.9452 | 0.8960 | 0.1040
KoBERT | 0.4637 | 0.8645 | 0.9273 | 0.8645 | 0.1355
KR-BERT (w. char) | 0.4772 | 0.9126 | 0.8709 | 0.9126 | 0.0874
KR-BERT (w. sub) | 0.4687 | 0.8821 | 0.8268 | 0.8821 | 0.1179

Table 10: A breakdown of binary classification performance on the K-MHaS
dataset for the five pre-trained language models at epoch 4 (E.M.:Exact Match,
H.L.:Hamming Loss / KR-BERT (w. *): char = character-level, sub = sub-
character-level, bi = Bidirectional WordPiece tokenizer)

Label Triplets | # triplets | F1 (macro) | F1 (micro)
---|---|---|---
Overall Performance (F1) | BERT | KoELECTRA | KoBERT | KR-BERT-c | KR-BERT-s | BERT | KoELECTRA | KoBERT | KR-BERT-c | KR-BERT-s
0.6912 | 0.7245 | 0.7651 | 0.7444 | 0.7245 | 0.8139 | 0.8493 | 0.8413 | 0.8500 | 0.8445
Origin & Politics & Profanity | 41 | 0.2780 | 0.2935 | 0.2954 | 0.2937 | 0.3125 | 0.8224 | 0.8739 | 0.8869 | 0.8739 | 0.9316
Politics & Profanity & Age | 37 | 0.2781 | 0.3054 | 0.3174 | 0.2981 | 0.2971 | 0.8205 | 0.9126 | 0.9395 | 0.8867 | 0.8824
Physical & Politics & Profanity | 32 | 0.2483 | 0.2809 | 0.2960 | 0.2823 | 0.2939 | 0.7296 | 0.8304 | 0.8750 | 0.8421 | 0.8764
Origin & Profanity & Gender | 30 | 0.2545 | 0.2314 | 0.2527 | 0.2463 | 0.2886 | 0.7467 | 0.7397 | 0.7368 | 0.7671 | 0.8712
Physical & Profanity & Gender | 24 | 0.2151 | 0.2665 | 0.2730 | 0.2459 | 0.2811 | 0.6306 | 0.7869 | 0.8226 | 0.7521 | 0.8413
Origin & Physical & Profanity | 14 | 0.2692 | 0.2811 | 0.2873 | 0.2351 | 0.2865 | 0.7532 | 0.8378 | 0.8312 | 0.7273 | 0.8421
Politics & Age & Gender | 14 | 0.2593 | 0.2933 | 0.2830 | 0.2319 | 0.2406 | 0.7606 | 0.8684 | 0.8158 | 0.6970 | 0.7059
Profanity & Age & Gender | 13 | 0.2686 | 0.2712 | 0.2932 | 0.2593 | 0.2695 | 0.8182 | 0.8060 | 0.8732 | 0.8000 | 0.8358
Origin & Physical & Gender | 12 | 0.2692 | 0.2327 | 0.2407 | 0.2347 | 0.2703 | 0.7812 | 0.6780 | 0.7458 | 0.7143 | 0.8065
Origin & Age & Gender | 10 | 0.2736 | 0.2781 | 0.3093 | 0.2428 | 0.2411 | 0.8235 | 0.8302 | 0.9123 | 0.7347 | 0.7600

Table 11: F1 score for the top 10 three-label pairs on the K-MHaS dataset for
the five pre-trained language models at epoch 4 (# total label triplets = 290
/ KR-BERT-*: c = character-level WordPiece tokenizer, s = sub-character-level
WordPiece tokenizer)

Label Quadruplets | # quadruplets | F1 (macro) | F1 (micro)
---|---|---|---
Overall Performance (F1) | BERT | KoELECTRA | KoBERT | KR-BERT-c | KR-BERT-s | BERT | KoELECTRA | KoBERT | KR-BERT-c | KR-BERT-s
0.4776 | 0.4511 | 0.4177 | 0.4837 | 0.5068 | 0.7093 | 0.7044 | 0.6832 | 0.7439 | 0.7771
Origin & Profanity & Age & Gender | 5 | 0.3567 | 0.2950 | 0.2346 | 0.2932 | 0.3086 | 0.8000 | 0.6875 | 0.5806 | 0.7500 | 0.7879
Origin & Physical & Profanity & Gender | 4 | 0.2582 | 0.2804 | 0.2508 | 0.3302 | 0.3757 | 0.5385 | 0.7200 | 0.6400 | 0.7692 | 0.8571
Origin & Physical & Politics & Profanity | 3 | 0.4000 | 0.3333 | 0.3111 | 0.3333 | 0.3667 | 0.9091 | 0.8000 | 0.7368 | 0.8000 | 0.8571
Origin & Politics & Profanity & Age | 3 | 0.4222 | 0.3444 | 0.3667 | 0.4444 | 0.4444 | 0.8800 | 0.8000 | 0.8571 | 1.0000 | 0.9600
Origin & Politics & Profanity & Gender | 2 | 0.1852 | 0.2963 | 0.2593 | 0.2963 | 0.2963 | 0.5455 | 0.7692 | 0.6667 | 0.6667 | 0.7692

Table 12: F1 score for the top 5 four-label pairs on the K-MHaS dataset for
the five pre-trained language models at epoch 4 (# total label quadruplets =
25 / KR-BERT-*: c = character-level WordPiece tokenizer, s = sub-character-
level WordPiece tokenizer)