# Language Models are Drummers:
Drum Composition with Natural Language Pre-Training

Li Zhang, Chris Callison-Burch

###### Abstract

Automatic music generation with artificial intelligence typically requires a
large amount of data which is hard to obtain for many less common genres and
musical instruments. To tackle this issue, we present ongoing work and
preliminary findings on the possibility for deep models to transfer knowledge
from language to music, by finetuning large language models pre-trained on a
massive text corpus on only hundreds of MIDI files of drum performances. We
show that by doing so, one of the largest, state-of-the-art models (GPT3) is
capable of generating reasonable drum grooves, while models that are not pre-
trained (Transformer) shows no such ability beyond naive repetition.
Evaluating generated music is a challenging task, more so is evaluating drum
grooves with little precedence in literature. Hence, we propose a tailored
structural evaluation method and analyze drum grooves produced by GPT3
compared to those played by human professionals, exposing the strengths and
weaknesses of such generation by language-to-music transfer. Our findings
suggest that language-to-music transfer learning with large language models is
viable and promising.111Accepted to the 1st workshop on Creative AI across
Modalities in AAAI 2023.222Data and code can be found at
https://github.com/zharry29/drums-with-llm. The title is a parody of the viral
trend of titling papers as “Language Models are …” in NLP venues.

## Introduction

Music understanding and generation using artificial intelligence has a long
history (Roads 1985) and has gained steady interests in recent years
(Kaliakatsos-Papakostas, Floros, and Vrahatis 2020). One strand of work
focuses on symbolic music rather than audio, where music is represented as
sequential data such as MIDI. While the analogy between music and language has
long been studied (McMullen and Saffran 2004), the symbolic representation of
music exhibits an even clearer similarity to language in their surface form.
For example, music has notes, measures, and sections, while language has
tokens, sentences, and paragraphs. It is thus intuitive that some work has
applied natural language processing (NLP) techniques to music. Specifically,
most have attempted to learn an embedding space of music (Liang et al. 2020)
similar to that of texts.

Figure 1: The sheet music of an example of our generated drum track,
demonstrating our model’s ability to somewhat musically follow the motif and
make variations. The first two measures are provided while the rest are
generated.

Recent work has leveraged Transformers (Vaswani et al. 2017) for symbolic
music processing (Huang et al. 2018; Yu et al. 2022). Transformers, when pre-
trained on a massive text corpus, are known as large language models (LLMs),
the current state-of-the-art approach to many NLP tasks (Devlin et al. 2019;
Brown et al. 2020). Notably, Zeng et al. (2021) is one of the first and only
work to pre-train a Transformer on a large symbolic music corpus containing
more than 1 million songs, using a similar approach of pre-training on a text
corpus, achieving state-of-the-art performance in various music understanding
tasks. However, one of the most significant limitations of this and most other
work in data-driven music processing is the large amount of symbolic music
data required for training, which is extremely challenging to obtain for less-
mainstream genres, particular styles, less-prevalent instruments, or out-of-
the-ordinary specifications, severely limiting the versatility of such a
method. Hence, low-resource symbolic music processing remains highly
challenging. For texts, on the other hand, few-shot learning has been greatly
empowered by LLMs due to the extremely large size of their pre-training
textual data, magnitudes more than music data.

In this work, we are the first step to explore such text-to-music transfer
learning potential in LLMs. In other words, we pose the hypothesis that
present-day state-of-the-art LLMs, pre-trained with a massive amount of
textual data, is capable of generating symbolic music with little music data
to some nontrivial extent. We specifically focus on one instrument, the drum
set, for multiple reasons. First, the drum set is one of the most common and
important instruments in many genres of music such as jazz, funk, blues,
gospel, latin, pop, rock, metal, etc. Second, the symbolic representation of
the drum set is simpler than most pitched instruments, as each note does not
have a pitch but corresponds to a hit on one drum. As the number of drums is
usually greatly smaller than that of possible pitches, the resulting sequence
is much shorter, and thus easier to be processed by models. Third, the
performance of a drum set typically is endowed with more degree of freedom
with regard to the audience’s aesthetics than many other instruments, making
it an appropriate entry point for studying music generation with LLMs, which
is presumed to be highly challenging.

We focus on the task of drum generation or composition, which has a small body
of published work in literature. While most if not all all of the existing
work has treated drums as an accompaniment, we instead focus on drum solo
generation, with a convenient analogy to story generation at which LLMs are
known to excel.

We finetune a state-of-the-art LLM, GPT3 model (Brown et al. 2020) on the
Groove dataset (Gillick et al. 2019) of about 400 drum groove performances
recorded as MIDI. To leverage the textual pre-training of GPT3, we propose a
textual representation of a drum performance. We present the following core
findings:

  1. 1.

The largest and smallest GPT3 models both can generate nontrivial drum grooves
after being finetuned.

  2. 2.

A similar-sized model that is not pre-trained on language data, however,
cannot.

We claim that the existing automatic evaluation of music generation is
insufficient for our task. Hence, we propose an evaluation methodology
specifically for drum grooves to both qualitatively and quantitatively
evaluate the strengths and weaknesses of machine-generated drum grooves
compared to those performed by humans. Finally, we provide some preliminary
listening test results, with a plan to conduct scaled and rigorous tests in
future work.

## The Drum Set

Figure 2: The anatomy of a basic drum set, and how each drum may appear on a
sheet music.

The drum set, also known as the drum kit, or colloquially the drums, is a
compound musical instrument consisting of many sub-instruments, including
drums and cymbals, both generally referred to as drums in this paper. Here, we
assume a simple, stereotypical drum set with a hi-hat, a crash cymbal, a ride
cymbal, a bass drum, a snare drum, and a tom (see Figure 2, but without the
hi-tom and the floor-tom333https://i.redd.it/cen72phsihb81.png).

The performance on a drum set can be notated as sheet music, a human-readable
symbolic representation, or as MIDI, which records the when each drum is hit
at what velocity, a computer-readable symbolic representation.

## Dataset

| train | dev | test
---|---|---|---
Total num. MIDI | 373 | 47 | 35
\- rock | 169 | 16 | 15
\- jazz | 41 | 6 | 4
\- latin | 37 | 10 | 3
\- funk | 31 | 6 | 4
\- hiphop | 26 | 1 | 3
\- others | 69 | 8 | 6
Table 1: The number of MIDI files in each style in the filtered Groove dataset
used in this work.

Among just a few datasets of drum performances, Google’s Groove MIDI
Dataset444https://magenta.tensorflow.org/datasets/groove is the largest and
the most high-quality to date, containing 1,150 MIDI files and over 22,000
measures of drumming by 10 professional drummers. In this dataset, the drum
performances are either grooves, long sequences of rhythmic ideas, or fills,
short bursts of free-flowing expressions. As we focus on drum generation or
composition throughout a long sequence, we only consider the grooves in the
dataset. Each MIDI is marked with the style (e.g., rock, funk, gospel, etc.),
the tempo (in beats per minute, BPM), and the time signature. For simplicity,
we only consider those in the time signature of 4/4. We follow the train-
development-test splits in the dataset. The statistics of the filtered Groove
dataset are shown in Figure 1.

The Groove dataset was originally proposed to study microtiming and expressive
performance, and therefore the drum MIDI files encapsulate human imperfection.
However, we re-purpose the dataset to study drum composition, leading to the
following choice of simplification. First, we quantize all notes to a 16-th
note grid. In other words, all note events in the MIDI are re-timed to the
closest of the 16 equidistant timestamps in a measure. An implication of such
quantization is that deliberate off-grid playing such as triplets or swing
feels is lost. Second, we discard the velocity information (i.e., how hard a
drum is hit), which can usually be inferred post-hoc to a coarse-grained
extent. Third, while the drum set can be played with many expressions and
articulations (e.g., hitting different part of a drum using different part of
different tools), we reduce them to simply the basic articulation (head hit)
of the hi-hat, crash cymbal, ride cymbal, bass drum, snare drum, and floor
tom. In other words, each note only has 6 possible values. Fourth, we truncate
each MIDI file to only the first 16 measures; at 128 BPM, for example, this
equates to 30 seconds. Finally, we remove empty leading measures whose first
quarter note is a rest, and ignore grooves with less than 8 measures.

## Experiments

### Representation

Figure 3: The drumroll representation of an example drum sheet music. Each
measure corresponds to 16 lines in texts, where each line corresponds to a
16-th note. Each line contains 6 characters corresponding to 6 drums, where
‘o’ and ‘-’ denotes whether each drum is hit.

As discussed before, LLMs have demonstrated extremely strong few-shot transfer
learning ability from one textual task to other. To possibly exploit this in
music, it is necessary to come up with a textual representation of the drum
grooves. We propose a pianoroll-like representation (Brunner et al. 2018),
referred to as a drumroll, that is essentially a multi-line string where each
row corresponds to a 16-th note in the time sequence, and each character in a
line corresponds to whether a drum is hit. Specifically, each character is ‘o’
if the particular drum is hit at the particular 16-th note, and ‘-’ otherwise.
See Figure 3 for an example. To help LLMs identify the boundary between
measures, we add a newline of “SEP” between every 16 lines (a measure) and a
newline of “END” after the final line.

### Task

We focus on an instance of drum generation referred to as drum completion,
where the model is given the first 2 measures and must complete the rest of
the 14 measures of the groove. The model may terminate at any point. This is
analogous to conditioned story generation in NLP.

### Model

First, we consider two naive baselines, randomly choosing whether to play a
note and repeating the second given measure. We then finetune a state-of-the-
art LLM, OpenAI’s GPT3 Davinci with 175 billion parameters, on the training
set. In each file of a drum groove, the input (prompt) is the first 2 measures
and the output (completion) is the remaining 14 measures. The temperature is
set to 0.85 to encourage creativity. Finetuning the model on the training set
costs $38.33 and takes around 30 minutes using OpenAI’s API.

To ascertain the role model size plays in drum generation, we further consider
a smaller GPT3 Ada model with 350 million parameters, which has been pre-
trained on the same corpus and we use the identical settings as the larger
Davinci model. Finetuning the model on the training set costs $0.51 and takes
around 5 minutes using OpenAI’s API. Both GPT3 models are later found to be
able to generate nontrivial drum grooves.

The ascertain the role language pre-training plays in drum generation, we set
the control to be an un-pre-trained GPT3 model, namely a Transformer (Vaswani
et al. 2017) with the same size as GPT3. Because our computing resources
cannot accommodate a model as big as Davinci, and also because merely 373 text
files each with 256 lines in the training set are likely insufficient to
converge the training loss, we finetune a smaller, un-pre-trained Transformer
with 85 million parameters, the same magnitude as GPT3 Ada. While the training
loss does converge, the model predicts the same certain sequence regardless of
what 2 measures are provided, performing no better than the random baseline.
This suggests that language pre-training is a necessary condition for
effective drum generation.

Figure 4: The variation (y-axis) of each measure (x-axis) in 8 randomly
sampled drum grooves from each style.

## Evaluation

How good are the drum grooves generated by GPT3? We follow the convention of
music generation and consider both objective/automatic and subjective/human
evaluation. We report our findings based on the test set.

### Objective Evaluation

We deem the established methods of automatic evaluation of symbolic music
generation unsuitable for our task. To name a few: perplexity, for example,
has long been shown to exhibit low correlation with human perception in NLP
(Kuribayashi et al. 2021); structural similarity, while a reasonable metric,
is often approximated with some simple similarity measures (Yu et al. 2022)
and discourages creativity that stray from the reference, which by no means
should be treated as the ground-truth.

In contrast, we consider what constitutes a good drum groove and propose a
structural evaluation called the pattern and fill analysis. We assume that
often a good drum groove minimally satisfies the following criteria:

  1. 1.

There exists one or more consistent patterns of some rhythmic idea and
occasional change-ups known as fills.

  2. 2.

The measures in a pattern are sufficiently similar, but ideally not identical.

  3. 3.

The measures in a fill are sufficiently different from those in adjacent
patterns.

Later, we verify that the human-performed grooves mostly satisfy these
criteria.

In a drum performance represented as a drumroll, each measure is represented
by a string of 16 lines (Figure 3). To classify each measure as either a
pattern or a fill, we take a sliding window of size 3 centered at some measure
$m_{i}$ and calculate the edit distances between this measure and its two
neighbors. The minimum of these two distances is referred to as the variation
of the central measure:

$\textit{variation}(m_{i})=\min(dist(m_{i},m_{i-1}),dist(m_{i},m_{i+1}))$

Intuitively, the variation of a measure in a pattern would be small, while
that in a fill would be large. Therefore, in a good drum groove, the variation
of all measures (except the first and the last) can be expected to be clearly
separated.

| human | random | repeat | Davinci | Ada
---|---|---|---|---|---
avg. variation | 5.1 | 40.4 | 0 | 3.0 | 3.4
avg. intra- centroids | 10.1 | 5.7 | 0 | 6.9 | 7.2
avg. inter- centroids | 1.4 | 1.2 | 0 | 0.6 | 0.8
Table 2: Qualitative statistics of drum grooves produced with different means.
The distance between cluster centroids suggests how dissimilar the patterns
and the fills are. The distance between a cluster centroid and a measure
suggests the amount that the measures in one class vary by.

Next, we plot the variation of each measure sequentially for each drum groove
in the development set and observe consistent patterns. Eight randomly chosen
grooves of different styles are shown in Figure 4. Intuitively, for human
performances, the variation of a measure in a pattern should be small, while
that in a fill should be large. This is indeed the case for most examples,
where human-played patterns are consistent but with some variations
(plateaus), while fills are largely different (spikes). As expected, the
random baseline results in high variation across all measures, while the
repeat baseline no variation at all – both are undesirable.

| human | random | repeat | Davinci | Ada
---|---|---|---|---|---
Repetitive | 0 | 0 | 35 | 3 | 7
Consistent | 32 | 0 | 0 | 29 | 15
Chaotic | 3 | 35 | 0 | 3 | 13
Has fill | 30 | 0 | 0 | 13 | 10
Avg. length | 13.3 | 16 | 16 | 13.7 | 12.5
Table 3: The number of drum grooves judged to satisfy each criteria produced
by each model in the test set.

Upon qualitative examination, the two GPT3 models of different sizes can
clearly generate nontrivial drum grooves, with many plateaus and occasional
spikes, though less spikes than those performed by human. Quantitatively, we
calculate the average variation of all measures in all grooves. As shown in
Table 2, drum grooves generated by GPT3 models tend to vary less than human.
To see if the generated patterns have less variation while the fills have much
more variation, we perform K-means to separate the measures in a drum groove
into two clusters by their variation. We then calculate the average intra-
distance between the two centroids, and the average inter-distance between
each measure and the centroid it is assigned to. As shown in Table 2, the
intra-centroid distance shows that the grooves performed by human have a much
clearer-cut pattern-versus-fill separation than GPT3, than the random
baseline. The inter-centroid distance shows that the spread of variations
within the class of pattern or fill is more pronounced in human-performed
grooves than in GPT3-generated ones.

Figure 5: The sheet music of a satisfactory drum groove generated by GPT3
(above), juxtaposed with the groove with the same first 2 measures played by
human from the dataset (below). We annotate each measure as either a pattern
or a fill. For example, pattern$x$.$a$ and pattern$x$.$b$ are adjacent, have
the same accented back-beats, but are not identical.

### Subjective Evaluation

Our objective evaluation is clearly informative but insufficient. Hence, we
conduct a listening test and perform an error analysis based on the following
criteria:

  * •

Is the groove repetitive, meaning there is little or no variation among
measures?

  * •

Is the groove consistent, meaning there is some variation among measures but a
steady rhythmic idea (specifically, the back-beat placement) can be followed?

  * •

Is the groove chaotic, meaning there is either too much variation, or a lack
of a clear rhythmic idea?

  * •

Does the groove contain any reasonable drum fill?

While scaling up this analysis rigorously with carefully chosen subjects is
left for future work, our own judgements are shown in Table 2 as preliminary
findings. Concretely, all drum grooves produced via different means are
shuffled and randomly present to one of the authors who has had years of
training in drumming. Naturally, all randomly generated grooves are judged as
chaotic without any consistent motif, while all repeated ones are by nature
repetitive. For the grooves performed by human, most are judged as consistent
and most include at least one fill which is sufficiently different from the
rest of the rhythmic patterns. In comparison, grooves generated by GPT3
Davinci is only slightly less consistent with desirable variations among
measures, but significantly less of them contain any fills, rendering the
grooves less interesting and more predictable overall. For the smaller GPT3
Ada model, the observation holds to a larger extent, with more inconsistent
grooves and less fills.

Figure 6: The sheet music of two problematic drum grooves generated by GPT3
(the 1st and 3rd), juxtaposed with those performed by human (the 2nd and 4th).
In the first case, GPT3 generates an extra note at the beginning of the fourth
measure, displacing all grooves afterwards. In the second case, GPT3 generates
strictly repeated measures; the rest are omitted.

### Case Study and Takeaways

We claim that the LLMs demonstrate an impressive ability to write good drum
grooves given only hundreds of training examples without any knowledge of what
features to pay attention to. From our objective and subjective evaluation
above, we have observed that both LLMs and human are able to compose drum
grooves with a structure and some variations. Why, however, are models’ drum
composition still worse than human’s? We postulate several factors.

We observe that a common source of the lack of musicality in machine-generated
drum grooves stems from the misplacement of back-beats, which are steadily
accented beats in a measure, usually the 2nd and the 4th in 4/4 in many genres
of music. Human drummers, when playing variations, tend to respect the back-
beat placements, while models tend to disregard such concept. Another source
of the lack of musicality is the lack of drum fills. While human drummers
often inherently think about the cycle between patterns and fills to give a
captivating performance, models are yet to realize the importance of those
occasional deviations.

To qualitatively examine these claims, we showcase two sets drum grooves
composed by GPT3, one showcasing its strengths and one its weaknesses, along
with the human performed grooves with the same first 2 measures.

A positive example is shown in Figure 5. Clearly, both grooves produced by
human and GPT3 alike contain consistent patterns interspersed with fills, a
prototypical structure that is expected and desired by most audience. Closely
examining the pattern groupings, both grooves vary the placement of the bass
drum and the snare drum, known as syncopation, while keeping the back-beats at
the 2nd and 4th quarter note in tact. As a typical drum groove, patterns in
both grooves consistently place a snare drum hit in all patterns (with two
exceptions in human’s pattern 3.1 and 3.2). Such composition is highly
desirable, as the steady back-beat placement guides the listener with a solid
rhythmic foundation, while the variations keep the groove interesting and
natural. Besides the ability to conforming to human drummers’ standard
practice, this example also showcases LLMs’ ability to be creative. While the
human-performed groove closely adheres to the template of 3 measures of
patterns followed by 1 measure of fill, the GPT3-generated groove does not.
Instead, it employs multi-measure fills, and more interestingly, grouping of
an odd number of patterns followed by fills, which is uncommon in the genre.

Two negative examples are shown in Figure 6, representing two common problems
of GPT3-generated drum grooves. This first case is referred to as displaced
measures. When human drummers play a pattern, they often fit each measure with
some rhythmic idea. In other words, the boundaries between two measures are
often clear. In our drumroll representation, each measure, 16 lines of texts,
is followed by a newline of ‘SEP’ to denote such boundary. However, GPT3 Ada
sometimes fail to respect these boundaries and displaces the measures, thus
having more “chaotic” patterns as reported in Table 3. In comparison, GPT3
Davinci makes a lot less such mistakes. The second case demonstrates LLMs’
known drawback of the proclivity to repeat generations. In the given example,
GPT3 Davinci generates 14 identical measures, while a human drummer would
“sneak in” minor variations while maintaining the motif. It is worth noting
that the repetition of drum grooves is accepted in some music genres such as
pop, dance, or rock, but frowned upon in most others as they are often thought
to be mechanical. For drum grooves generated by LLMs, repetition is logically
a trait to be avoided.

As of now, we have taken a model-free approach where the LLMs are only trained
on some drum groove data without any additional information or priors. To
reinforce the strengths and alleviate the weaknesses discussed before, we
suggest future work to take a modular approach instead of tackling the task in
an end-to-end fashion. For example, the patterns and fills can be generated
from different distributions or by different models; repetition can be
explicitly discouraged by over-generating each measure and perform some voting
or selection. Furthermore, it is possible to control the variation within a
drum groove by injecting additional labels in the training data to condition
the LLM on.

### Improvisation or Recitation?

By far, we have hinted at LLMs’ ability to be creative with regard to drum
composition. Here, we pose one additional question: are the LLMs really
creating some drum grooves that they have never seen during finetuning, or are
they simply regurgitating what they have already seen?

We answer this question by calculating the frequency of each generated measure
in the test set appearing in the the training set. As shown in Figure 7, only
a small portion of generated measures are duplicates of any seen measures
during finetuning, suggesting LLMs’ ability to compose novel and unseen drum
grooves.

Figure 7: The count (y-axis) of occurrences (x-axis) of each generated measure
appearing in the training set.

## Related Work

Automatic music generation has a long history, and recent work has focused on
using artificial intelligence (Kaliakatsos-Papakostas, Floros, and Vrahatis
2020) and specifically deep neural networks. Such efforts are driven by not
only the prospect of having AI aid or replace human composers and arrangers
for a variety of music, but also the pursuit of probing the artistic
creativity of the state-of-the-art data-driven models. Most modern work in
automatic music generation leverages model architectures that have shown to be
effective in computer vision or NLP, such as LSTM (Lyu et al. 2015),
Transformers (Huang et al. 2018; Zeng et al. 2021; Yu et al. 2022), or custom
architectures to learn an embedding space (Liang et al. 2020). However, the
majority of the work on music generation with AI has happened in a supervised
setting, which greatly limits its application to lesser known genres or
specific instruments, such as drum generation. While there is yet to be a pre-
trained music generation model as versatile as its counterpart in NLP such as
GPT3, we believe the exploration of language-to-music transfer is necessary
but uncharted.

Automatic drum generation has a small body of existing work. Similar to music
generation in general, the drum generation task can happen in various
settings. In the simplest setting, only one measure of drum pattern (also
known as a “beat”) is generated that is supposed to repeat throughout a song
(Vogl and Knees 2017; Bruford, McDonald, and Sandler 2020; Tikhonov. and
Yamshchikov. 2021), while we focus on a more involved setting of non-
repetitive drum composition. Alternatively, some work simply considers the
general rhythm (Lattner and Grachten 2019), but not the orchestration of
different drums that we emphasize on. In more practical settings, a long
sequence of drum composition is generated conditioned on musical signals such
as the basslines (Makris et al. 2017, 2019). While this line of work is most
similar to ours which by far only deal with drum solo performance, all the
work above has used drum composition data with limited size and variations as
well as models (such as LSTM) that are relatively outdated in the AI
community. Nevertheless, a direct comparison would be beneficial in future
work. Less related is another line of work focuses on the microtiming and
humanization of drum performance (Gillick et al. 2019; Burloiu and Unatc 2020;
Burloiu 2020), striving to mimic human’s expressive imperfections.

A small body of work has focused on rhythm games (Donahue, Lipton, and McAuley
2017; Liang, Li, and Ikeda 2019). While rhythm games and drumming have certain
similarities such as the focus on note placement with regard to the rhythm,
their core difference lies in that the choreography of rhythm games is
optimized for difficulty and playability, while the composition of drums is
optimized for musicality. Due to this fundamental difference of motivation, we
consider this line of work to be mostly irrelevant to ours.

Large language models (LLMs) are deep neural models, such as Transformers,
pre-trained on a massive text corpus. For example, GPT3 is pre-trained on the
compilation of Common Crawl, containing most texts in the world wide net,
publicly available books, Wikipedia, and so on. From BERT (Devlin et al. 2019)
to GPT3 (Brown et al. 2020), LLMs have been dominant is most tasks and
applications in NLP. While much about how LLMs work is unknown, and thus LLMs
are notoriously known as black-boxes, there is a generally consensus that
LLMs’ power can be attribute to the large size of both the pre-training data
and the model, which give rise to LLMs’ ability to effectively adapt to low-
resource domains via transfer learning by being finetuned on a small amount of
data. Interestingly, a few recent work has found that some of these abilities
include transfer from pre-trained language to non-language tasks, such as
chess (Stöckl 2021). While non-language textual representations such as chess
moves or music charts are similar to natural language superficially, each
manifests vastly different structures, and so we claim that transfer learning
between them is well worth studying.

## Conclusion and Future Work

Our preliminary findings show that pre-trained large language models (LLMs)
finetuned with merely hundreds of symbolic music files, such as drum grooves,
can learn to generate music non-trivially. We also provide evidence that such
ability can be attributed both the model size and the presence of language
pre-training. We hope this observation inspires research efforts in not only
low-resource music generation, but also exploration of extraordinary
potentials of LLMs. We also attempt to pioneer the automatic evaluation of
drum grooves which we hope to facilitate future work in drum generation with
AI.

We also briefly discuss our plans for future work. While drum generation is
scarce in literature and challenging to reproduce, we will still strive
compare some existing specialized models. While our current drumroll
representation ignores velocity, such information can easily be encoded by
replacing the marker ‘o’ with the velocity value. However, the effects of
doing so remains to be explored. Our methodology might be ported to other
instruments such as piano, which we plan to explore, whereas it would be more
involved to tackle multi-instrument conditioned generation.

## References

  * Brown et al. (2020) Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; Agarwal, S.; Herbert-Voss, A.; Krueger, G.; Henighan, T.; Child, R.; Ramesh, A.; Ziegler, D.; Wu, J.; Winter, C.; Hesse, C.; Chen, M.; Sigler, E.; Litwin, M.; Gray, S.; Chess, B.; Clark, J.; Berner, C.; McCandlish, S.; Radford, A.; Sutskever, I.; and Amodei, D. 2020.  Language Models are Few-Shot Learners.  In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., _Advances in Neural Information Processing Systems_ , volume 33, 1877–1901. Curran Associates, Inc.
  * Bruford, McDonald, and Sandler (2020) Bruford, F.; McDonald, S.; and Sandler, M. 2020.  jaki: user-controllable generation of drum patterns using an LSTM encoder-decoder and deep reinforcement learning.  In _The 2020 Joint Conference on AI Music Creativity_.
  * Brunner et al. (2018) Brunner, G.; Wang, Y.; Wattenhofer, R.; and Zhao, S. 2018.  Symbolic music genre transfer with cyclegan.  In _2018 ieee 30th international conference on tools with artificial intelligence (ictai)_ , 786–793. IEEE.
  * Burloiu (2020) Burloiu, G. 2020.  Interactive Learning of Microtiming in an Expressive Drum Machine.  In _The 2020 Joint Conference on AI Music Creativity_.
  * Burloiu and Unatc (2020) Burloiu, G.; and Unatc, C. 2020.  Adaptive Drum Machine Microtiming with Transfer Learning and RNNs.  In _Extended Abstracts for the Late-Breaking Demo Session of the International Society for Music Information Retrieval Conference (ISMIR)_.
  * Devlin et al. (2019) Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , 4171–4186. Minneapolis, Minnesota: Association for Computational Linguistics.
  * Donahue, Lipton, and McAuley (2017) Donahue, C.; Lipton, Z. C.; and McAuley, J. 2017.  Dance dance convolution.  In _International conference on machine learning_ , 1039–1048. PMLR.
  * Gillick et al. (2019) Gillick, J.; Roberts, A.; Engel, J.; Eck, D.; and Bamman, D. 2019.  Learning to groove with inverse sequence transformations.  In _International Conference on Machine Learning_ , 2269–2279. PMLR.
  * Huang et al. (2018) Huang, C.-Z. A.; Vaswani, A.; Uszkoreit, J.; Shazeer, N.; Simon, I.; Hawthorne, C.; Dai, A. M.; Hoffman, M. D.; Dinculescu, M.; and Eck, D. 2018.  Music transformer.  _arXiv preprint arXiv:1809.04281_.
  * Kaliakatsos-Papakostas, Floros, and Vrahatis (2020) Kaliakatsos-Papakostas, M.; Floros, A.; and Vrahatis, M. N. 2020.  Artificial intelligence methods for music generation: a review and future perspectives.  _Nature-Inspired Computation and Swarm Intelligence_ , 217–245.
  * Kuribayashi et al. (2021) Kuribayashi, T.; Oseki, Y.; Ito, T.; Yoshida, R.; Asahara, M.; and Inui, K. 2021\.  Lower Perplexity is Not Always Human-Like.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , 5203–5217. Online: Association for Computational Linguistics.
  * Lattner and Grachten (2019) Lattner, S.; and Grachten, M. 2019.  High-level control of drum track generation using learned patterns of rhythmic interaction.  In _2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)_ , 35–39. IEEE.
  * Liang et al. (2020) Liang, H.; Lei, W.; Chan, P. Y.; Yang, Z.; Sun, M.; and Chua, T.-S. 2020.  Pirhdy: Learning pitch-, rhythm-, and dynamics-aware embeddings for symbolic music.  In _Proceedings of the 28th ACM International Conference on Multimedia_ , 574–582.
  * Liang, Li, and Ikeda (2019) Liang, Y.; Li, W.; and Ikeda, K. 2019.  Procedural content generation of rhythm games using deep learning methods.  In _Joint International Conference on Entertainment Computing and Serious Games_ , 134–145. Springer.
  * Lyu et al. (2015) Lyu, Q.; Wu, Z.; Zhu, J.; and Meng, H. 2015.  Modelling high-dimensional sequences with lstm-rtrbm: Application to polyphonic music generation.  In _Twenty-Fourth International Joint Conference on Artificial Intelligence_.
  * Makris et al. (2017) Makris, D.; Kaliakatsos-Papakostas, M.; Karydis, I.; and Kermanidis, K. L. 2017\.  Combining LSTM and feed forward neural networks for conditional rhythm composition.  In _International conference on engineering applications of neural networks_ , 570–582. Springer.
  * Makris et al. (2019) Makris, D.; Kaliakatsos-Papakostas, M.; Karydis, I.; and Kermanidis, K. L. 2019\.  Conditional neural sequence learners for generating drums’ rhythms.  _Neural Computing and Applications_ , 31(6): 1793–1804.
  * McMullen and Saffran (2004) McMullen, E.; and Saffran, J. R. 2004.  Music and Language: A Developmental Comparison.  _Music Perception: An Interdisciplinary Journal_ , 21(3): 289–311.
  * Roads (1985) Roads, C. 1985.  Research in music and artificial intelligence.  _ACM Computing Surveys (CSUR)_ , 17(2): 163–190.
  * Stöckl (2021) Stöckl, A. 2021.  Watching a Language Model Learning Chess.  In _Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)_ , 1369–1379. Held Online: INCOMA Ltd.
  * Tikhonov. and Yamshchikov. (2021) Tikhonov., A.; and Yamshchikov., I. 2021.  Artificial Neural Networks Jamming on the Beat.  In _Proceedings of the 6th International Conference on Complexity, Future Information Systems and Risk - COMPLEXIS,_ , 37–44. INSTICC, SciTePress.  ISBN 978-989-758-505-0.
  * Vaswani et al. (2017) Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L. u.; and Polosukhin, I. 2017.  Attention is All you Need.  In Guyon, I.; Luxburg, U. V.; Bengio, S.; Wallach, H.; Fergus, R.; Vishwanathan, S.; and Garnett, R., eds., _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc.
  * Vogl and Knees (2017) Vogl, R.; and Knees, P. 2017.  An intelligent drum machine for electronic dance music production and performance.  In _NIME_ , 251–256.
  * Yu et al. (2022) Yu, B.; Lu, P.; Wang, R.; Hu, W.; Tan, X.; Ye, W.; Zhang, S.; Qin, T.; and Liu, T.-Y. 2022.  Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation.
  * Zeng et al. (2021) Zeng, M.; Tan, X.; Wang, R.; Ju, Z.; Qin, T.; and Liu, T.-Y. 2021.  Musicbert: Symbolic music understanding with large-scale pre-training.  _arXiv preprint arXiv:2106.05630_.


# Bipol: Multi-axes Evaluation of Bias with Explainability in Benchmark
Datasets

Tosin Adewumi*+, Isabella Södergren++, Lama Alkhaled+, Sana Sabah Sabry+,
Foteini Liwicki+ and Marcus Liwicki+
+Machine Learning Group, EISLAB, ++Digital Services and Systems
Luleå University of Technology, Sweden
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

We investigate five English NLP benchmark datasets (on the superGLUE
leaderboard) and two Swedish datasets for bias, along multiple axes. The
datasets are the following: Boolean Question (Boolq), CommitmentBank (CB),
Winograd Schema Challenge (WSC), Winogender diagnostic (AXg), Recognising
Textual Entailment (RTE), Swedish CB, and SWEDN. Bias can be harmful and it is
known to be common in data, which ML models learn from. In order to mitigate
bias in data, it is crucial to be able to estimate it objectively. We use
bipol, a novel multi-axes bias metric with explainability, to estimate and
explain how much bias exists in these datasets. Multilingual, multi-axes bias
evaluation is not very common. Hence, we also contribute a new, large Swedish
bias-labeled dataset (of 2 million samples), translated from the English
version and train the SotA mT5 model on it. In addition, we contribute new
multi-axes lexica for bias detection in Swedish. We make the codes, model, and
new dataset publicly available.

Caution: This paper contains examples, from datasets, of what some may
consider as stereotypes or offensive text.

##  1 Introduction

Recent advances in artificial intelligence (AI), large language models (LLM),
and chatbots have raised concerns about their potential risks to humanity
Bender et al. (2021); Adewumi et al. (2022); Yudkowsky et al.
(2008).111bbc.com/news/world-us-canada-65452940 One major concern is the issue
of social bias, particularly with the data AI models are trained on. Bias,
which can be harmful, is the unfair prejudice in favor of or against a thing,
person or group Maddox (2004); Dhamala et al. (2021); Mehrabi et al. (2021);
Antoniak and Mimno (2021). Measuring bias in text data can be challenging
because of the axes that may be involved (e.g. religious or gender bias).

In this work, our motivation is to determine whether social bias exists in NLP
benchmark datasets and estimate it. After reviewing some potential bias
methods, as discussed in Section 2, we settled for the recent bipol Alkhaled
et al. (2023) because of its advantages. It is a metric that estimates bias
along multiple axes in text data and provides an explanation for its scores,
unlike other metrics. We investigate social bias in benchmark datasets that
are available on the English SuperGLUE leaderboard and two Swedish datasets.
The SuperGLUE was introduced by Wang et al. (2019) and provides benchmark
datasets for different NLP tasks. Benchmark datasets are datasets for
comparing the performance of algorithms for specific use-cases Dhar and Shamir
(2021); Paullada et al. (2021). Such datasets have been the foundation for
some of the significant advancements in the field Paullada et al. (2021). We
investigate the following English datasets: Boolq Clark et al. (2019), CB De
Marneffe et al. (2019), WSC Levesque et al. (2012), AXg Rudinger et al.
(2018a), and RTE Wang et al. (2019). The Swedish datasets are the Overlim CB
and SWEDN. We discuss more about the datasets in Section 3.2.

##### Our contributions

Firstly, we show quantitatively and through explainability that bias exists in
the datasets. The findings correlate with characteristics of bias, such as
heavy lopsidedness Zhao et al. (2018). This work will provide researchers with
insight into how to mitigate bias in text data and possibly add impetus to the
conversation on whether it is even ethical to remove these social biases from
data, because they represent the real world. Secondly, we create and release,
possibly, the largest labeled dataset and lexica for bias detection in Swedish
(multi-axes bias dataset (MAB)-Swedish) and train a model based on the state-
of-the-art (SotA)multilingual T5 (mT5)Xue et al. (2021). We release our codes,
dataset and artefacts publicly. 222github.com/LTU-Machine-
Learning/bipolswedish.git

The rest of this paper is structured as follows. Section 2 discusses some of
the previous related work. Section 3 describes the methodology, including
details of the characteristics of bipol and the new MAB-Swedish dataset.
Section 4 presents the results and discusses some of the qualitative results.
In Section 5, we give concluding remarks.

##  2 Related Work

There have been considerable effort in identifying and measuring the level of
bias in datasets Cryan et al. (2020); Dhamala et al. (2021); Stanley (1977);
Chandrabose et al. (2021). These are usually targeted at gender bias in a
binary form Zhao et al. (2018); Rudinger et al. (2018a). However, studies have
shown that the biases in language models for the intersection of gender and
race can be greater than those for gender and race individually and that
addressing bias along only one axis can lead to more issues Tan and Celis
(2019); Subramanian et al. (2021). To determine the level of bias in NLP
datasets along multiple axes can be a significant challenge, more so that many
of these methods admit their approaches may demonstrate the presence of bias
but not prove its absence Zhao et al. (2018); Rudinger et al. (2018a). Table 1
compares some of the methods that have been introduced.

Metric/Evaluator | Axis | Terms
---|---|---
WinoBias Zhao et al. (2018) | 1 | 40
Winogender Rudinger et al. (2018a) | 1 | 60
StereoSet Nadeem et al. (2021) | 4 | 321
Hurtlex Nozza et al. (2021) | 6 | 1,072
CrowS-Pairs Nangia et al. (2020) | 9 | 3,016
Bipol Alkhaled et al. (2023) |  $>$2, 13*$<$ |  $>$45, 466*$<$

Table 1:  Comparison of some bias evaluation methods. (*The upper bounds are
not limited by the bipol algorithm but the dataset & lexica.)

Furthermore, Bassignana et al. (2018) proposed a multi-language approach using
HurtLex to target misogyny because addressing bias in only the English
language is not sufficient for addressing the potential harm to society. In
the English language, there are common biases that associate female terms with
subjects such as liberal arts and family while associating male terms with
subjects such as science Nosek et al. (2002). There are also more words that
sexualize females more than males Stanley (1977). Other languages have their
own peculiarities Nozza et al. (2021).

In addition to the various methods identified in Table 1 for quantifying the
extent of discrimination or bias, there is also odds ratio (OR), which
compares the chance of a specific outcome happening, with a certain exposure,
to the likelihood of that outcome happening without the exposure Szumilas
(2010). Another method is the impact ratio (IR), which calculates the ratio of
positive outcomes for a protected group to the general group. In Cryan et al.
(2020), they compare lexicon method to model classification for gender bias in
English language only. Our approach combines the strengths of both approaches
and evaluates on English and Swedish data across multiple axes.

##  3 Methodology

###  3.1 Bipol

There are two stages in the implementation of bipol (see 1a) before it gives a
final score between 0.0 (zero or undetected bias) and 1.0 (extreme bias). The
first stage involves the classification of the data samples (into biased and
unbiased categories) using a trained model (see 1b). Ideally, it is the ratio
of the number of true positives (tp)to the total samples (true positives (tp),
false positives (fp), true negatives (tn), and false negatives (fn)), where fp
is preferably zero. However, since the trained models will be evaluated on
unseen data, the predicted biased samples are likely to have fp in the
numerator as expressed in the equation. The evaluations thus come with
positive error rate ($\frac{\lx@glossaries@gls@link{acronym}{fp}{\leavevmode
fp}}{\lx@glossaries@gls@link{acronym}{fp}{\leavevmode
fp}+\lx@glossaries@gls@link{acronym}{tp}{\leavevmode tp}}$) to establish the
lower bound of error for the predictions. A good classifier should minimize
the number of fp and maximize the number of tp but there’s hardly any perfect
classifier, even in other tasks such as spam detection or hate speech (Heron,
2009; Feng et al., 2018).

$\mathit{b}=\begin{cases}\mathit{b_{c}}.\mathit{b_{s}},&\text{if $b_{s}>0$}\\\
\mathit{b_{c}},&\text{otherwise}\end{cases}$ (1a)

$\mathit{b_{c}}=\frac{tp+fp}{tp+fp+tn+fn}$ (1b)

$\mathit{b_{s}}=\frac{1}{r}\sum_{t=1}^{r}{\left(\frac{1}{q}\sum_{x=1}^{q}{\left(\frac{|\sum_{s=1}^{n}a_{s}-\sum_{s=1}^{m}c_{s}|}{\sum_{s=1}^{p}d_{s}}\right)}_{x}\right)}_{t}$
(1c)

The second stage evaluates the biased samples for sensitive terms listed in
the multi-axes lexica (see 1c). It involves finding the difference between the
two maximum summed frequencies in the types (e.g. female) of an axis (e.g.
gender) ($|\sum_{s=1}^{n}a_{s}-\sum_{s=1}^{m}c_{s}|$), which is then divided
by the summed frequencies of all the terms in that axis
($\sum_{s=1}^{p}d_{s}$). The average over all the axes
($\frac{1}{q}\sum_{x=1}^{q}$) is then averaged over all the biased samples
($\frac{1}{r}\sum_{t=1}^{r}$ ). Table 2 provides the Swedish lexica sizes. The
lexica are derived from Adewumi et al. (2020a, b) and
Wikipedia333en.wikipedia.org/wiki/Swedish_profanity and may be expanded as
needed. They include terms that may be stereotypically associated with certain
groups and specific gender Cryan et al. (2020); Zhao et al. (2018). The
English lexica contain more and are also derived from public sources Alkhaled
et al. (2023).

Axis | Axis type 1 | Axis type 2
---|---|---
Gender | 17 (female) | 19 (male)
Racial | 10 (black) | 10 (white)
Table 2:  Swedish lexica sizes. These may be expanded.

The rationale for using bipol is because of the strengths of the metric. These
include 1) the relative simplicity of calculating a score, 2) it is straight-
forward to implement since it is based on existing concepts like lexica and
classifiers, 3) it captures semantic and term frequency (TF)aspects of data,
4) it has explainability built in, 5) it’s possible to determine the error
rate of predictions, and 6) it is not limited in the total number of axes that
may be used. We acknowledge, however, that it has limitations that are based
on the limitations of the tools that may be used to calculate it.

###  3.2 Datasets

####  3.2.1 The New MAB-Swedish Dataset

The dataset was machine-translated (from MAB Alkhaled et al. (2023)) using the
Helsinki-NLP model Tiedemann and Thottingal (2020), which was mostly trained
with guided alignment. The automatic translation took over 48 hours on one
GPU. It has 1,946,975 samples, as given in Table 3. Quality control (QC) for
the MAB-Swedish involved translation verification by back-translating some
random samples using Google NMT before a review by a Swedish speaker. The
English version was constructed from two datasets:
Jigsaw444medium.com/jigsaw/creating-labeled-datasets-and-exploring-the-role-
of-human-raters-56367b6db298 and the Social Bias Inference Corpus v2
(SBICv2)by Sap et al. (2020). Personal identifiable information (PII)were
removed from the dataset. More details about the annotation of the base
datasets for the MAB can be found in Alkhaled et al. (2023). Some examples in
the MAB-Swedish are given in Table 4.

##### Machine-Translation concerns

Bias is a universal concern, though there can be culture-specific biases. A
stereotype or degrading comment can be considered of universal concern if it
is relevant across cultures. For instance, the first example in Table 4 is of
universal concern because women should be respected, regardless of nationality
or culture. This also applies to the other biased examples in the table.
Hence, MAB-Swedish is relevant for bias detection in Swedish, though it was
translated. A limitation of the dataset, however, is that it may not contain
examples that are specific to the Swedish culture since the original dataset
is from English, with specific references to the USA and Canada.

Set | Biased | Unbiased | Total
---|---|---|---
Training | 533,544 | 1,209,433 | 1,742,977
Validation | 32,338 | 69,649 | 101,987
Test | 33,470 | 68,541 | 102,011
| 599,352 | 1,347,623 | 1,946,975
Table 3:  MAB-Swedish dataset split.  comment_text |  label |  old_id |  id
---|---|---|---
Den här slynan är galen som skulle läsa en bok av en kvinna. |  biased |  239612 |  1212584
(This b*tch is nuts. Who would read a book by a woman.) |  |  |
9 av 10 människor njuter av mobbning. Jag kommer alltid att vara den 10: e |  biased |  none |  1517517
(9 out of 10 people enjoy bullying. I will always be the 10th.) |  |  |
”Författaren säger att det finns ¿ många, många fler anledningar till att jag antar att en uppföljare finns i verken, då?” |  unbiased |  383811 |  110831
(”The author says that there are many, many more reasons I am assuming a sequel is in the works, then?”) |  |  |
Vad kallar du underkläder för araber? |  biased |  none |  1618146
(What do you call lingerie for Arabs? Socks.) |  |  |
Table 4:  Examples from the MAB-Swedish (The English in the original is in
italics.)

####  3.2.2 Boolq

It is a question-answering (QA)task where each example has a short passage and
a yes/no question about the passage Clark et al. (2019) . These questions were
provided anonymously by Google search users and afterwards paired with a
paragraph from a Wikipedia article that has the answer. We evaluated the
passage column of the dataset.

####  3.2.3 CB

This contains short texts in which, at least, one sentence has an embedded
clause De Marneffe et al. (2019). The resulting task is framed as three-class
textual entailment on examples that are drawn from the following datasets:
Wall Street Journal, fiction from the British National Corpus, and
Switchboard. We evaluated the premise column of the dataset.

####  3.2.4 WSC

This is a coreference resolution dataset Levesque et al. (2012). Examples
consist of a sentence with a pronoun and a list of noun phrases from the
sentence. We evaluated the text column of the dataset.

####  3.2.5 AXg

It is designed to measure gender bias in coreference resolution systems
Rudinger et al. (2018b). Each example consists of a premise sentence having a
male or female pronoun and a hypothesis giving a possible antecedent of the
pronoun. We evaluated the premise column of the dataset.

####  3.2.6 RTE

The datasets come from a series of annual competitions on textual entailment
Wang et al. (2019). Data from several sources were merged and converted to
two-class classification: entailment and not_entailment. We evaluated the
premise column of the dataset.

####  3.2.7 Swedish CB

This is part of the OverLim dataset by the National Library of Sweden. It
contains some of the GLUE and SuperGLUE tasks automatically translated to
Swedish, Danish, and Norwegian, using the OpusMT models for
MarianMT555huggingface.co/datasets/KBLab/overlim. We evaluated its training
set.

####  3.2.8 SWEDN

This is a text summarization corpus based on 1,963,576 news articles from the
Swedish newspaper Dagens Nyheter (DN) during the years 2000 to
2020.666spraakbanken.gu.se/resurser/swedn There are five categories of
articles in the dataset: domestic news, economy, sports, culture, and others
Monsen and Jönsson (2021). The training set consists of the first three
categories and we evaluate the first 1,000 samples because of the computation
cost of evaluation.

###  3.3 Experiments

The experiments were conducted on two shared Nvidia DGX-1 clusters running
Ubuntu 18.04 and 20.04 with 8 × 32GB V100 and 8 x 40GB A100 GPUs,
respectively. Average results are reported after running each experiment
twice. To evaluate the benchmark datasets, we utilize bias-detection models
Alkhaled et al. (2023) based on RoBERTa (Liu et al., 2019), Electra (Clark et
al., 2020), and DeBERTa (He et al., 2021). We train a small mT5 model with
batch size of 16, due to memory constraints, on the MAB-Swedish. Wandb Biewald
(2020), an experiment tracking tool, is run for 5 counts with bayesian
optimization to suggest the best hyper-parameter combination for the learning
rate (1e-3 - 2e-5) and epochs (6 - 10) before final training of the model. We
use the pretrained model from the HuggingFace hub Wolf et al. (2020). Average
training time was 15 hours. Average evaluation time ranges from about 30
minutes to over 24 hours.777particularly when cpulimit is used, in fairness to
other users

##  4 Results and Discussion

From Table 5 we observe that all the datasets have bias, though little, given
that they are smaller than a bipol score of 1. The dataset with the least
amount of bias is Boolq, which is confirmed by all the three models. This is
despite the dataset having the highest number of unique samples. CB has the
largest amount of bias and this is also confirmed by the three models. This is
also the case for the Swedish CB, when compared with SWEDN.

The average macro F1 score on the validation set of MAB-Swedish is 0.7623 with
standard deviation (s.d.) of 0.0075. The resulting error rate is 0.2893. This
is relatively reasonable though a bit higher than the error rate for the
English RoBERTa, Electra, and DeBERTa, which are 0.198, 0.196, and 0.2,
respectively Alkhaled et al. (2023).

|  |  bipol level $\downarrow$ (s.d.)
---|---|---
RoBERTa | samples | corpus | sentence | bipol (b)
Boolq | 7,929 | 0.0066 | 0.8027 | 0.0053 (0)
CB | 250 | 0.08 | 0.8483 | 0.0679 (0)
WSC | 279 | 0.0466 | 0.8718 | 0.0406 (0)
AXg | 178 | 0.0112 | 1 | 0.0112 (0)
RTE | 2,379 | 0.0294 | 0.8518 | 0.0251 (0)
Electra |  |  |  |
Boolq | 7,929 | 0.0073 | 0.8089 | 0.0059 (0)
CB | 250 | 0.0316 | 0.881 | 0.074 (0)
WSC | 279 | 0.0609 | 0.9559 | 0.0582 (0)
AXg | 178 | 0.0112 | 1 | 0.0112 (0)
RTE | 2,379 | 0.0269 | 0.8593 | 0.0231 (0)
DeBERTa |  |  |  |
Boolq | 7,929 | 0.0103 | 0.7212 | 0.0075 (0)
CB | 250 | 0.084 | 0.9048 | 0.076 (0)
WSC | 279 | 0.0609 | 1 | 0.0609 (0)
AXg | 178 | 0.0112 | 1 | 0.0112 (0)
RTE | 2,379 | 0.0366 | 0.8655 | 0.0316 (0)
mT5 on Swedish data
CB | 201 | 0.0796 | 0.7188 | 0.0572 (0)
SWEDN | 1,000 | 0.053 | 0.9433 | 0.05 (0)
Table 5:  Results of average bipol scores. All the datasets have bias, though
little.

###  4.1 Error Analysis

Figure 1 presents the confusion matrix for the mT5 on the MAB-Swedish. The tn,
fp, fn, tp are 61,689, 7,960, 12,781, and 19,557, respectively, which are
relevant for Eq. 1b. We observe that the model is better at predicting
unbiased samples. This is expected since the training data contains more
examples of unbiased samples. Table 6 presents some qualitative examples of
apparently correct and incorrect predictions in two of the datasets. The first
correct example in the English CB appears to have a clear stereotype that men
are naturally right and it is the role of women to follow their lead. The
second correct example, in both the English and Swedish data, may have been
perceived as biased by the two different models because of the offensive term
fool or the overgeneralization that folk will always take advantage of
weakness or both. Overgeneralization is a characteristic of bias Rudinger et
al. (2018a); Nadeem et al. (2021).

Figure 1: Confusion matrix for mT5. examples |  label |  csv id
---|---|---
Correct predictions from the English CB, based on DeBERTa
nicky approached her with the assumption that men are naturally right and it is the role of women to follow their lead. constance whose confidence was growing daily was not prepared to give in to nickys wishes merely because of his sex. if she felt he was right then she agreed with him. |  biased |  85
she rubbed them away with an angry fist. she was a fool to let anyone get round her. how long before she learned that folk ll always take advantage of weakness |  biased |  73
Incorrect prediction
he d gone. philip had to get them back. his dad would kill him if he found that he d taken them. |  biased |  107
|  |
Correct predictions from the Swedish CB, based on mT5
det är en del av deras religion en religion som jag inte hånar eftersom den innehåller många element som matchar vår egen trots att den saknar vår sanning. vid en av deras stora högtider har de ritualen att driva ut djävlarna från sina kroppar. först kommer trummisarna jag kan säga att inga kvinnor tillåts delta i denna ritual och damerna här kanske håller med mig om att de är lyckligt lottade i detta utelämnande. |  biased |  2
(it is part of their religion a religion that I do not mock because it contains many elements that match our own even though it lacks our truth. at one of their great festivals they have the ritual of casting the devils out of their bodies. first come the drummers i can say no women are allowed to participate in this ritual and the ladies here may agree with me that they are lucky in this omission.) |  |
hon gnuggade bort dem med en arg knytnäve. hon var en idiot som lät vem som helst komma runt henne. hur lång tid tar det innan hon får veta att folk alltid kommer att utnyttja svaghet |  biased |  59
(she rubbed them away with an angry fist. she was a fool to let anyone get round her. how long before she learned that folk will always take advantage of weakness) |  |
Incorrect prediction
han var borta. philip var tvungen att få tillbaka dem. hans pappa skulle döda honom om han fann att han hade tagit dem. |  biased |  86
(he had gone. philip had to get them back. his dad would kill him if he found that he had taken them.) |  |
Table 6:  Qualitative examples of apparently correct and incorrect predictions
in some of the datasets. The English translations appear in italics for the
Swedish examples.

### Explaining bias type

The type of overall bias (for the gender axis) in many of the datasets is
explained by the dictionary of lists produced by bipol (see Appendix .1) and
represented in ”top-5 frequent terms” bar graphs of Figures 2 to 13. As
expected, we observed that AXg is limited to only gender, unlike Boolq, which
also reflects religious bias, as explained in their bipol dictionaries of
lists. We observe from Figures 2, 3, and 4 that Boolq is male-biased. Figures
5, 6, and 7 show that CB is also male-biased. This is the case also for RTE,
as revealed by Figures 8, 9, and 10. On the other hand, we observe that the
case of WSC is not clear-cut because Figure 11 shows only a marginal lead for
female bias, Figure 12 shows the difference among the top-5 is zero and Figure
13 shows a slight overall male bias.

Figure 2: Top-5 gender frequent terms in Boolq by RoBERTa. Figure 3: Top-5
gender frequent terms in Boolq by DeBERTa. Figure 4: Top-5 gender frequent
terms in Boolq by Electra. Figure 5: Top-5 gender frequent terms in CB by
Roberta. Figure 6: Top-5 gender frequent terms in CB by DeBERTa. Figure 7:
Top-5 gender frequent terms in CB by Electra. Figure 8: Top-5 gender frequent
terms in RTE by RoBERTa. Figure 9: Top-5 gender frequent terms in RTE by
DeBERTa. Figure 10: Top-5 gender frequent terms in RTE by Electra. Figure 11:
Top-5 gender frequent terms in WSC by RoBERTa. Figure 12: Top-5 gender
frequent terms in WSC by DeBERTa. Figure 13: Top-5 gender frequent terms in
WSC by Electra.

##  5 Conclusion

We show that all benchmark datasets we evaluated, including the Swedish
datasets, contain bias to different degrees. This is likely the first time
these datasets are evaluated in such a way that estimates the amount of bias
and the type. We believe these evaluations will motivate research on how to
more effectively mitigate bias along multiple axes in datasets. This work may
encourage discussions on whether the biased samples from the benchmark
datasets should be disregarded entirely or if they should be utilized in a
different manner than previously done. Our public release of the new MAB-
Swedish dataset, lexica and model will also facilitate future work in
multilingual bias detection.

## Ethics Statement

The authors made the effort to obscure offensive terms in examples that were
used in this paper. We note that the models for estimating the biases in the
datasets are limited in scope, as they only cover certain number of axes (12).
Therefore, a result of 0 on any dataset does not necessarily indicate a bias-
free dataset.

## Acknowledgments

We sincerely thank the anonymous reviewers for their valuable feedback on this
work.

## References

  * Adewumi et al. (2022) Tosin Adewumi, Foteini Liwicki, and Marcus Liwicki. 2022.  State-of-the-art in open-domain conversational ai: A survey.  _Information_ , 13(6).
  * Adewumi et al. (2020a) Tosin P Adewumi, Foteini Liwicki, and Marcus Liwicki. 2020a.  Corpora compared: The case of the swedish gigaword & wikipedia corpora.  _arXiv preprint arXiv:2011.03281_.
  * Adewumi et al. (2020b) Tosin P Adewumi, Foteini Liwicki, and Marcus Liwicki. 2020b.  Exploring swedish & english fasttext embeddings for ner with the transformer.  _arXiv preprint arXiv:2007.16007_.
  * Alkhaled et al. (2023) Lama Alkhaled, Tosin Adewumi, and Sana Sabah Sabry. 2023.  Bipol: A novel multi-axes bias evaluation metric with explainability for nlp.  _Manuscript_.
  * Antoniak and Mimno (2021) Maria Antoniak and David Mimno. 2021.  Bad seeds: Evaluating lexical methods for bias measurement.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1889–1904.
  * Bassignana et al. (2018) Elisa Bassignana, Valerio Basile, and Viviana Patti. 2018.  Hurtlex: A multilingual lexicon of words to hurt.  In _5th Italian Conference on Computational Linguistics, CLiC-it 2018_ , volume 2253, pages 1–6. CEUR-WS.
  * Bender et al. (2021) Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021.  On the dangers of stochastic parrots: Can language models be too big?  In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ , pages 610–623.
  * Biewald (2020) Lukas Biewald. 2020.  Experiment tracking with weights and biases.  Software available from wandb.com.
  * Chandrabose et al. (2021) Aravindan Chandrabose, Bharathi Raja Chakravarthi, et al. 2021.  An overview of fairness in data–illuminating the bias in data pipeline.  In _Proceedings of the First Workshop on Language Technology for Equality, Diversity and Inclusion_ , pages 34–45.
  * Clark et al. (2019) Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019.  BoolQ: Exploring the surprising difficulty of natural yes/no questions.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2924–2936, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Clark et al. (2020) Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020.  Electra: Pre-training text encoders as discriminators rather than generators.  _arXiv preprint arXiv:2003.10555_.
  * Cryan et al. (2020) Jenna Cryan, Shiliang Tang, Xinyi Zhang, Miriam Metzger, Haitao Zheng, and Ben Y. Zhao. 2020.  Detecting gender stereotypes: Lexicon vs. supervised learning methods.  In _Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems_ , CHI ’20, page 1–11, New York, NY, USA. Association for Computing Machinery.
  * De Marneffe et al. (2019) Marie-Catherine De Marneffe, Mandy Simons, and Judith Tonhauser. 2019.  The commitmentbank: Investigating projection in naturally occurring discourse.  In _proceedings of Sinn und Bedeutung_ , volume 23, pages 107–124.
  * Dhamala et al. (2021) Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. 2021.  Bold: Dataset and metrics for measuring biases in open-ended language generation.  In _ACM FAccT 2021_.
  * Dhar and Shamir (2021) Sanchari Dhar and Lior Shamir. 2021.  Evaluation of the benchmark datasets for testing the efficacy of deep convolutional neural networks.  _Visual Informatics_ , 5(3):92–101.
  * Feng et al. (2018) Bo Feng, Qiang Fu, Mianxiong Dong, Dong Guo, and Qiang Li. 2018.  Multistage and elastic spam detection in mobile social networks through deep learning.  _IEEE Network_ , 32(4):15–21.
  * He et al. (2021) Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021.  Deberta: Decoding-enhanced bert with disentangled attention.  In _International Conference on Learning Representations_.
  * Heron (2009) Simon Heron. 2009.  Technologies for spam detection.  _Network Security_ , 2009(1):11–15.
  * Levesque et al. (2012) Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012.  The winograd schema challenge.  In _Thirteenth international conference on the principles of knowledge representation and reasoning_.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Maddox (2004) Keith B Maddox. 2004.  Perspectives on racial phenotypicality bias.  _Personality and Social Psychology Review_ , 8(4):383–401.
  * Mehrabi et al. (2021) Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021.  A survey on bias and fairness in machine learning.  _ACM Computing Surveys (CSUR)_ , 54(6):1–35.
  * Monsen and Jönsson (2021) Julius Monsen and Arne Jönsson. 2021.  A method for building non-english corpora for abstractive text summarization.  In _Proceedings of CLARIN Annual Conference_.
  * Nadeem et al. (2021) Moin Nadeem, Anna Bethke, and Siva Reddy. 2021.  StereoSet: Measuring stereotypical bias in pretrained language models.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 5356–5371, Online. Association for Computational Linguistics.
  * Nangia et al. (2020) Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020.  CrowS-pairs: A challenge dataset for measuring social biases in masked language models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1953–1967, Online. Association for Computational Linguistics.
  * Nosek et al. (2002) Brian A Nosek, Mahzarin R Banaji, and Anthony G Greenwald. 2002.  Harvesting implicit group attitudes and beliefs from a demonstration web site.  _Group Dynamics: Theory, Research, and Practice_ , 6(1):101.
  * Nozza et al. (2021) Debora Nozza, Federico Bianchi, and Dirk Hovy. 2021.  Honest: Measuring hurtful sentence completion in language models.  In _The 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. Association for Computational Linguistics.
  * Paullada et al. (2021) Amandalynne Paullada, Inioluwa Deborah Raji, Emily M Bender, Emily Denton, and Alex Hanna. 2021.  Data and its (dis) contents: A survey of dataset development and use in machine learning research.  _Patterns_ , 2(11):100336.
  * Rudinger et al. (2018a) Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018a.  Gender bias in coreference resolution.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 8–14, New Orleans, Louisiana. Association for Computational Linguistics.
  * Rudinger et al. (2018b) Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018b.  Gender bias in coreference resolution.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , New Orleans, Louisiana. Association for Computational Linguistics.
  * Sap et al. (2020) Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, and Yejin Choi. 2020.  Social bias frames: Reasoning about social and power implications of language.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5477–5490, Online. Association for Computational Linguistics.
  * Stanley (1977) Julia Penelope Stanley. 1977.  Paradigmatic woman: The prostitute.  _Papers in language variation_ , pages 303–321.
  * Subramanian et al. (2021) Shivashankar Subramanian, Xudong Han, Timothy Baldwin, Trevor Cohn, and Lea Frermann. 2021.  Evaluating debiasing techniques for intersectional biases.  _arXiv preprint arXiv:2109.10441_.
  * Szumilas (2010) Magdalena Szumilas. 2010.  Explaining odds ratios.  _Journal of the Canadian academy of child and adolescent psychiatry_ , 19(3):227.
  * Tan and Celis (2019) Yi Chern Tan and L Elisa Celis. 2019.  Assessing social and intersectional biases in contextualized word representations.  _Advances in Neural Information Processing Systems_ , 32.
  * Tiedemann and Thottingal (2020) Jörg Tiedemann and Santhosh Thottingal. 2020.  OPUS-MT — Building open translation services for the World.  In _Proceedings of the 22nd Annual Conferenec of the European Association for Machine Translation (EAMT)_ , Lisbon, Portugal.
  * Wang et al. (2019) Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019.  Superglue: A stickier benchmark for general-purpose language understanding systems.  In _Advances in Neural Information Processing Systems_ , volume 32. Curran Associates, Inc.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online. Association for Computational Linguistics.
  * Xue et al. (2021) Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021.  mT5: A massively multilingual pre-trained text-to-text transformer.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 483–498, Online. Association for Computational Linguistics.
  * Yudkowsky et al. (2008) Eliezer Yudkowsky et al. 2008.  Artificial intelligence as a positive and negative factor in global risk.  _Global catastrophic risks_ , 1(303):184.
  * Zhao et al. (2018) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018\.  Gender bias in coreference resolution: Evaluation and debiasing methods.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 15–20, New Orleans, Louisiana. Association for Computational Linguistics.

###  .1 Experiment

#### Dictionary of lists for RoBERTa on Boolq

{’gender’: [’ she ’: 23, ’ her ’: 17, ’ woman ’: 2, ’ lady ’: 1, ’ female ’:
6, ’ girl ’: 1, ’ skirt ’: 0, ’ madam ’: 0, ’ gentlewoman ’: 0, ’ madame ’: 0,
’ dame ’: 0, ’ gal ’: 0, ’ maiden ’: 0, ’ maid ’: 0, ’ damsel ’: 0, ’ senora
’: 0, ’ lass ’: 0, ’ beauty ’: 0, ’ ingenue ’: 0, ’ belle ’: 0, ’ doll ’: 0, ’
señora ’: 0, ’ senorita ’: 0, ’ lassie ’: 0, ’ ingénue ’: 0, ’ miss ’: 0, ’
mademoiselle ’: 0, ’ señorita ’: 0, ’ babe ’: 0, ’ girlfriend ’: 0, ’ lover ’:
0, ’ mistress ’: 0, ’ ladylove ’: 0, ’ inamorata ’: 0, ’ gill ’: 0, ’ old ’:
2, ’ beloved ’: 0, ’ dear ’: 0, ’ sweetheart ’: 0, ’ sweet ’: 0, ’ flame ’: 2,
’ love ’: 5, ’ valentine ’: 0, ’ favorite ’: 1, ’ moll ’: 0, ’ darling ’: 0, ’
honey ’: 0, ’ significant ’: 0, ’ wife ’: 3, ’ wifey ’: 0, ’ missus ’: 0, ’
helpmate ’: 0, ’ helpmeet ’: 0, ’ spouse ’: 0, ’ bride ’: 1, ’ partner ’: 0, ’
missis ’: 0, ’ widow ’: 0, ’ housewife ’: 0, ’ mrs ’: 0, ’ matron ’: 0, ’ soul
’: 3, ’ mate ’: 1, ’ housekeeper ’: 0, ’ dowager ’: 0, ’ companion ’: 0, ’
homemaker ’: 0, ’ consort ’: 0, ’ better half ’: 0, ’ hausfrau ’: 0, ’ stay-
at-home ’: 0, ’ he ’: 80, ’ him ’: 49, ’ boy ’: 3, ’ man ’: 1, ’ male ’: 10, ’
guy ’: 1, ’ masculine ’: 0, ’ virile ’: 0, ’ manly ’: 0, ’ man-sized ’: 0, ’
hypermasculine ’: 0, ’ macho ’: 0, ’ mannish ’: 0, ’ manlike ’: 0, ’ man-size
’: 0, ’ hairy-chested ’: 0, ’ butch ’: 0, ’ ultramasculine ’: 0, ’ boyish ’:
0, ’ tomboyish ’: 0, ’ hoydenish ’: 0, ’ amazonian ’: 0, ’ gentleman ’: 0, ’
dude ’: 0, ’ fellow ’: 0, ’ cat ’: 2, ’ gent ’: 0, ’ fella ’: 0, ’ lad ’: 0, ’
bloke ’: 0, ’ bastard ’: 0, ’ joe ’: 0, ’ chap ’: 0, ’ chappie ’: 0, ’ hombre
’: 0, ’ galoot ’: 0, ’ buck ’: 0, ’ joker ’: 3, ’ mister ’: 0, ’ jack ’: 8, ’
sir ’: 0, ’ master ’: 1, ’ buddy ’: 0, ’ buster ’: 0], ’racial’:… }