###  0.B.44 Neopronoun Substitution

This transformation performs grammatically correct substitution from English
to English of the gendered pronouns, he/she, in a given sentence with their
neopronoun counterparts, based on a list compiled by UNC Greensboro and LGBTA
WIKI202020https://intercultural.uncg.edu/wp-content/uploads/Neopronouns-
Explained-UNCG-Intercultural-Engagement.pdf. NLP models, such as those for
neural machine translation, often fail to recognize the neopronouns and treat
them as proper nouns. This transformation seeks to render the training data
used in NLP pipelines more neopronoun aware to reduce the risk of trans-
erasure. The reason why a simple look-up-table approach might not work is due
to the fact that the case may differ depending on the context.

🖙 She $\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ They had her
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ their friends tell her
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ them about the event.

###  0.B.45 Gender Neutral Rewrite

This transformation involves rewriting an English sentence containing a single
gendered entity with its gender-neutral variant. One application is machine
translation, when translating from a language with gender-neutral pronouns
(e.g. Turkish) to a language with gendered pronouns (e.g. English). This
transformation is based on the algorithm proposed by Sun et al. (2021).

🖙 His $\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ Their dream is to be a fireman
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ firefighter when he
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ they grows
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ grow up.

###  0.B.46 GenderSwapper

This transformation introduces gender diversity to the given data. If used as
data augmentation for training, the transformation might mitigate gender bias,
as shown in Dinan et al. (2020). It also might be used to create a gender-
balanced evaluation dataset to expose the gender bias of pre-trained models.
This transformation performs lexical substitution of the opposite gender. The
list of gender pairs (shepherd <–> shepherdess) is taken from Lu et al.
(2019). Genderwise names used from Ribeiro et al. (2020) are also randomly
swapped.

###  0.B.47 GeoNames Transformation

This transformation augments the input sentence with information based on
location entities (specifically cities and countries) available in the
GeoNames database212121http://download.geonames.org/export/dump/. E.g., if a
country name is found, the name of the country is appended with information
about the country like its capital city, its neighbouring countries, its
continent, etc. Some initial ideas of this nature were explored in Pais
(2019).

###  0.B.48 German Gender Swap

This transformation replaces the masculine nouns and pronouns with their
female counterparts for German sentences from a total of 2226 common German
names.222222https://de.wiktionary.org/wiki/Verzeichnis:Deutsch/Namen

🖙 Er $\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ Sie ist ein Arzt
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ eine Ärztin und mein Vater
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ meine Mutter .

###  0.B.49 Grapheme to Phoneme Substitution

This transformation adds noise to a sentence by randomly converting words to
their phonemes. Grapheme-to-phoneme substitution is useful in NLP systems
operating on speech. An example of grapheme to phoneme substitution is
“permit” P ER0 M IH1 T’.

###  0.B.50 Greetings and Farewells

This transformation replaces greetings (e.g. "Hi", "Howdy") and farewells
(e.g. "See you", "Good night") with their synonymous equivalents.

🖙 Hey $\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ Hi everyone. It’s nice
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ Pleased to meet you. How have
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ are you been ?

###  0.B.51 Hashtagify

This transformation modifies an input sentence by identifying named entities
and other common words and turning them into hashtags, as often used in social
media.

###  0.B.52 Insert English and French Abbreviations

This perturbation replaces in texts some well known English and French words
or expressions with (one of) their abbreviations. Many of the abbreviations
covered here are quite common on social medias platforms, even though some of
them are quite generic. This implementation is partly inspired by recent work
in Machine Translation Berard et al. (2019).

###  0.B.53 Leet Transformation

Visual perturbations are often used to disguise offensive comments on social
media (e.g., “!d10t”) or as a distinct writing style (“1337” in “leet speak”)
Eger et al. (2019a), especially common in scenarios like video gaming. Humans
are unconsciously robust to such visually similar texts. This perturbation
replaces letters with their visually similar “leet”
counterparts.232323https://simple.wikipedia.org/wiki/Leet

🖙 Ujjal Dev Dosanjh served
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ U7jal 0ev D0san74 serv3d as 33rd Premier of British Columbia from
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ Pr33i3r 0f 8ritis4 00lu36ia fr0m 2000 to
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ t0 2001

###  0.B.54 Lexical Counterfactual Generator

This transformation generates counterfactuals by simply substituting negative
words like “not”, “neither” in one sentence of a semantically similar sentence
pair. The substituted sentence is then backtranslated in an attempt to correct
for grammaticality. This transformation would be useful for tasks like
entailment and paraphrase detection.

###  0.B.55 Longer Location for NER

This transformation augments data for Named Entity Recognition (NER) tasks by
augmenting examples which have a Location Tag. Names of locations are expanded
by appending them with cardinal directions like “south”, “N”, “northwest”,
etc. The transformation ensures that the tags of the new sentence are
accordingly modified.

###  0.B.56 Longer Location Names for testing NER

This transformation augments data for Named Entity Recognition (NER) tasks by
augmenting examples that have a Location (LOC) Tag. Names of location are
expanded by inserting random prefix or postfix word(s). The transformation
also ensures that the labels of the new tags are accordingly modified.

###  0.B.57 Longer Names for NER

This transformation augments data for Named Entity Recognition (NER) tasks by
augmenting examples which have a Person Tag. Names of people are expanded by
inserting random characters as initials. The transformation also ensures that
the labels of the new tags are accordingly modified.

###  0.B.58 Lost in Translation

This transformation is a generalization of the BackTranslation transformation
to any sequence of languages supported by the Helsinki-NLP OpusMT models
(Tiedemann and Thottingal, 2020).

🖙 Andrew finally returned
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ brought Chris back the French book the French book I bought last
week I bought last week

###  0.B.59 Mixed Language Perturbation

Mixed language training has been effective for cross-lingual tasks Liu et al.
(2020), to help generate data for low-resource scenarios Liu et al. (2021) and
for multilingual translation Fan et al. (2021). Two transformations translate
randomly picked words in the text from English to other languages (e.g.,
German). It can be used to test the robustness of a model in a multilingual
setting.

🖙 Andrew finally returned the
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ die Comic book to Chris that I bought last week
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ woche

###  0.B.60 Mix transliteration

This transformation transliterates randomly picked words from the input
sentence (of given source languae script) to a target language script. It can
be used to train/test multilingual models to improve/evaluate their ability to
understand complete or partially transliterated text.

###  0.B.61 MR Value Replacement

This perturbation adds noise to a key-value meaning representation (MR) (and
its corresponding sentence) by randomly substituting values/words with their
synonyms (or related words). This transformation uses a simple strategy to
align values of a MR and tokens in the corresponding sentence inspired by how
synonyms are substituted for tasks like machine translation Fadaee et al.
(2017). This way, there could be some problems in complex sentences. Besides,
the transformation might introduce non-grammatical segments.

###  0.B.62 Multilingual Back Translation

This transformation translates a given sentence from a given language into a
pivot language and then back to the original language. This transformation is
a simple paraphraser that works on 100 different languages. Back Translation
has been quite popular now and has been a quick way to augment Li and Specia
(2019); Sugiyama and Yoshinaga (2019); Fan et al. (2020).

🖙 Being honest
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ Honesty should be one of our most important character traits
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ characteristics

###  0.B.63 Multilingual Dictionary Based Code Switch

This transformation generates multi-lingual code-switching data to fine-tune
encoders of large language models Qin et al. (2020); Tan and Joty (2021); Wang
et al. (2019b) by making use of bilingual dictionaries of MUSE Lample et al.
(2018).

###  0.B.64 Multilingual Lexicon Perturbation

This perturbation helps to creates code-mixed sentences for both high-resource
and low-resource languages by randomly translating words with a specified
probability from any supported languages (e.g., English) to other supported
languages (e.g., Chinese) by using a multilingual lexicon. Thus, it can be
used to generate code-mixed training data to improve models for multilingual
and cross-lingual settings. As of now 100 languages are supported and 3000
common English words listed on ef.com 242424https://www.ef.com/wwen/english-
resources/english-vocabulary/top-3000-words/ are supported. The lexicon
implementation is also 160x faster than its model based counterpart.

###  0.B.65 Causal Negation & Strengthening

This transformation is targeted at augmenting Causal Relations in text and
adapts the code from the paper Causal Augmentation for Causal Sentence
Classification Tan et al. (2021a). There are two operations: 1\. Causal
Negation: Negative words like "not, no, did not" are introduced into sentences
to unlink the causal relation. 2\. Causal Strengthening: Causal meaning is
strengthened by converting weaker modal words into stronger ones like "may" to
"will" to assert causal strength.

The implementation provides users with the option to amend causal meaning
automatically from the root word of the sentence, or by explicitly
highlighting the index of the word they wish to amend. Additionally, we
include WordNet Miller (1998) synonyms and tense matching to allow for more
natural augmentations.

🖙 The rs7044343 polymorphism could be
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$ was involved in regulating the production of IL-33.

###  0.B.66 Question Rephrasing transformation

This implementation rephrases questions for sentence tasks by using the T5
model used in 0.B.75 for Question Answering tasks.

###  0.B.67 English Noun Compound Paraphraser [N+N]

This transformation replaces two-word noun compounds with a paraphrase, based
on the compound paraphrase dataset from SemEval 2013 Task 4 Hendrickx et al.
(2013). It currently only works for English. Any two-word compound that
appears in a dataset of noun compound paraphrases will be replaced by a
paraphrase. If more than one two-word compound appears, then all combinations
of compound paraphrases (including no paraphrase at all) will be returned. For
example, the paraphrases of “club house” include “house for club activities”,
“house for club members”, “house in which a club meets”, etc. We start with
replacing paraphrases with the highest score (the specified frequency in the
annotated dataset), and paraphrases with the same score (ties) are sorted
randomly. This transformation currently only checks for noun compounds from
Hendrickx et al. (2013) and therefore has low coverage. To improve it, other
datasets could be added, e.g., from Ponkiya et al. (2018) or Lauer (1995). To
attain even wider-coverage (at the expense of lower precision), machine
learning approaches such as Shwartz and Dagan (2018) or Ponkiya et al. (2020)
could be considered. In addition, some of the the paraphrases in Hendrickx et
al. (2013) sound a little odd (e.g., "blood cell" -> "cell of blood") and may
not fit well in context.

###  0.B.68 Number to Word

This transformation acts like a perturbation to improve robustness on
processing numerical values. The perturbated sentence contains the same
information as the initial sentence but with a different representation of
numbers.

###  0.B.69 Numeric to Word

This transformation translates numbers in numeric form to their textual
representations. This includes general numbers, long numbers, basic math
characters, currency, date, time, phone numbers, etc.

###  0.B.70 OCR Perturbation

This transformation directly induces Optical Character Recognition (OCR)
errors into the input text. It renders the input sentence as an image and
recognizes the rendered text using the OCR engine Tesseract 4 (Smith, 2007).
It works with text in English, French, Spanish, and German. The implementation
follows previous work by Namysl et al. (2021).

###  0.B.71 Add Noun Definition

This transformation appends noun definitions onto the original nouns in a
sentence. Definitions of nouns are collected from Wikidata
252525https://www.wikidata.org/wiki/Wikidata:Main_Page.

###  0.B.72 Pig Latin Cipher

This transformation translates the original text into pig latin. Pig Latin is
a well-known deterministic transformation of English words, and can be viewed
as a cipher which can be deciphered by a human with relative ease. The
resulting sentences are completely unlike examples typically used in language
model training. As such, this augmentation change the input into inputs which
are difficult for a language model to interpret, while being relatively easy
for a human to interpret.

###  0.B.73 Pinyin Chinese Character Transcription

This transformation transcribes Chinese characters into their Mandarin
pronunciation using the Pinyin romanization scheme. The Character-to-Pinyin
converter at the core of this transformation is a neural model by Park and Lee
(2020).

###  0.B.74 SRL Argument Exchange

This perturbation adds noise to all types of English text sources (sentence,
paragraph, etc.) proportional to the number of arguments identified by SRL
BERT (Shi and Lin, 2019b). Different rules are applied to deterministically
modify the sentence in a meaning-preserving manner. Rules look as follows: if
ARGM-LOC and ARGM-TMP both present, exchange them.

Example: [ARG0: Alex] [V: left] [ARG2: for Delhi] [ARGM-COM: with his wife]
[ARGM-TMP: at 5 pm] . Alex left for Delhi at 5 pm with his wife.

The transformation relies on propbank annotations (Bonial et al., 2012;
Kingsbury and Palmer, 2002; Palmer et al., 2005; Gildea and Palmer, 2002).

###  0.B.75 ProtAugment Diverse Paraphrasing

This transformation utilizes the ProtAugment method by Dopierre et al. (2021).
The paraphrase generation model is a BART model (Lewis et al., 2020),
finetuned on the paraphrase generation task using 3 datasets: Google-PAWS
(Zhang et al., 2019b), MSR (Dolan and Brockett, 2005),
Quora262626https://quoradata.quora.com/First-Quora-Dataset-Release-Question-
Pairs.

When parpahrasing a sentence, the transformation useu Diverse Beam Search
(Vijayakumar et al., 2016) to generate diverse outputs. The diversity penalty
term is by default set to 0.5 but can be set to custom values. Additionally,
the transformation can use the following generation constraints: (1) A
fraction of the words in the input sentence are forbidden in the paraphrase
(default 0.7). (2) All bi-grams in the input sentence are forbidden in the
paraphrase. This means the paraphrase cannot contain any bi-gram that are in
the input sentence. This constraint enforces the paraphrase generation model
to change the sentence structure.

###  0.B.76 Punctuation

This transformation removes/adds punctuation from an English sentence. This
transformation was first introduced by Mille et al. (2021) and used as an
example implemention for NL-Augmenter.

###  0.B.77 Question-Question Paraphraser for QA

This transformation creates new QA pairs by generating question paraphrases
from a T5 model fine-tuned on Quora Question pairs 272727Quora Question Pairs.
Generated questions can have a very different surface form from the original
question making it a strong paraphrase generator. A T5 model Raffel et al.
(2019); Wolf et al. (2020) fine tuned
282828https://huggingface.co/ramsrigouthamg/t5_paraphraser on the Quora
Question Pairs dataset was being used to generate question paraphrases. This
transformation would benefit Question Answering, Question Generation as well
as other tasks which could indirectly benefit eg. for dialog tasks Shrivastava
et al. (2021); Dhole (2020).

###  0.B.78 Question in CAPS

This transformation upper-cases the context of a question answering example.
It also adds upper-cased versions of the original answers to the set of
acceptable model responses.

###  0.B.79 Random Word Deletion

This transformation randomly removes a word with a given probability $p$ (by
default 0.25). The transformation relies on whitespace tokenization and thus
only works for English and other languages that are segmented via whitespace.
Due to the destructive nature of the transformation, it is likely that the
meaning of a sequence may be changed as a result of the change. A similar
transformation was suggested by Wei and Zou (2019). Word dropout Goldberg
(2017) has been common to help models understand unknown words encountered
during evaluation by exposing them to this unknown-word condition during
training itself.

###  0.B.80 Random Upper-Case Transformation

This perturbation adds noise to all types of text sources (sentence,
paragraph, etc.) by randomly adding upper cased letters. With a default
probably of 0.1, each character in a sequence is upper-cased. This
transformation does not rely on a tokenizer and thus works with all languages
that have upper and lower-case letters. One limiation of this transformation
is that it will not affect a tokenizer that does lower case for all input. A
similar transformation was suggested by Wei and Zou (2019). Further
improvement of this transformation exists by potentially relying on extreme
value theory Jalalzai et al. (2020).

###  0.B.81 Double Context QA

This transformation repeats the context of a question answering example. This
should not change the result in any way.

###  0.B.82 Replace Abbreviations and Acronyms

This transformation changes abbreviations and acronyms appearing in an English
text to their expanded form and respectively, changes expanded abbreviations
and acronyms appearing in a text to their shorter form. For example, “send
this file asap to human resources” might be changed to “send this file as soon
as possible to HR”. The list of abbreviations and acronyms used in this
transformation where manually gathered focusing on common abbreviations
present in business communications. When abbreviation are context-dependent or
highly specific, the induced change may change the meaning of a text, or an
abbreviation may not be available in the lookup. The transformation was first
introduced by Regina et al. (2020).

###  0.B.83 Replace Financial Amounts

This transformation replaces financial amounts throughout a text with the same
value in a different currency. The replacement changes the amount, the writing
format as well as the currency of the financial amount. For example, the
sentence “I owe Fred € 20 and I need € 10 for the bus.” might be changed to “I
owe Fred 2 906.37 Yen and I need 1 453.19 Yen for the bus.” The transformation
was first introduced by Regina et al. (2020).

###  0.B.84 Replace Numerical Values

This transformation looks for numerical values in an English text and replaces
it with another random value of the same cardinality. For example, “6.9” may
be replaced by “4.2”, or “333” by “789”. The transformation was first
introduced by Mille et al. (2021).

###  0.B.85 Replace Spelling

This transformation adds noise to all types of English text sources (sentence,
paragraph, etc.) using corpora of common spelling errors introduced by
Deorowicz and Ciura (2005). Each word with a common misspelling is replaced by
the version with mistake with a probability $p$ which by default is set to
$0.2$.

###  0.B.86 Replace nouns with hyponyms or hypernyms

This transformation replaces common nouns with other related words that are
either hyponyms or hypernyms. Hyponyms of a word are more specific in meaning
(such as a sub-class of the word), eg: ’spoon’ is a hyponym of ’cutlery’.
Hypernyms are related words with a broader meaning (such as a generic category
/super-class of the word), eg: ’colour’ is a hypernym of ’red’. Not every word
will have a hypernym or hyponym.

###  0.B.87 Sampled Sentence Additions

This transformation adds generated sentence to all types of English text
sources (sentence, paragraph, etc.) by passing the input text to a GPT-2 model
(Radford et al., 2019). By default, GPT-XL is used, together with the prompt
“paraphrase:” appended to the original text, after which up to 75 tokens are
sampled. Since the additional text is sampled from a model, the model may
introduce harmful language or generate text that contradicts the earlier text
or changes its meaning. The idea to sample one or more additional sentences
was first introduced by Jia and Liang (2017a).

###  0.B.88 Sentence Reordering

This perturbation adds noise to all types of text sources (paragraph,
document, etc.) by randomly shuffling the order of sentences in the input text
Lewis et al. (2020). Sentences are first partially decontextualized by
resolving coreference Lee et al. (2018).

This transformation is limited to input text that has more than one sentence.
There are still cases where coreference can not be enough for
decontextualization. For example, there could be occurences of ellipsis as
demonstrated by Gangal et al. (2021) or events could be mentioned in a
narrative style which makes it difficult to perform re-ordering or shuffling
Kočiskỳ et al. (2018) while keeping the context of the discourse intact.

###  0.B.89 Emoji Addition for Sentiment Data

This transformation adds positive emojis and smileys to positive sentiment
data and negative emojis to negative sentiment data. For non-labelled data, it
adds neutral smileys.

###  0.B.90 Shuffle Within Segments

In this transformation, a token sequence, for example BIO-tagged, is split
into coherent segments. Thus, each segment corresponds to either a mention or
a sequence of out-of-mention tokens. For example, a sentence “She did not
complain of headache or any other neurological symptoms .” with tags O O O O O
B-problem O B-problem I-problem I-problem I-problem O is split into five
segments: [She did not complain of], [headache], [or], [any other neurological
symptoms], [.]. Then for each segment, a binomial distribution (p=0.5) is used
to decide whether it should be shuffled. If yes, the order of the tokens
within the segment is shuffled while the label order is kept unchanged. This
transformation is inspired by Dai and Adel (2020).

###  0.B.91 Simple Ciphers

This transformation modifies the text in ways that a human could rapidly
decipher, but which make the input sequences almost completely unlike typical
input sequences which are used during language model training. This
transformation includes the following text modifications: double the
characters, double the words, add spaces between the characters, reverse all
characters in the text, reverse the characters within each word, reverse the
order of the words in the text, substitute homoglyphs, rot13 cipher.

###  0.B.92 Slangificator

This transformation replaces some of the words (in particular, nouns,
adjectives, and adverbs) of an English text with their corresponding slang.
The replacement is done with the subset of the "Dictionary of English Slang &
Colloquialisms".292929http://www.peevish.co.uk/slang/index.htm The amount of
replacement is proportional to the corresponding probabilities of replacement
(by default, 0.5 for nouns, adjectives, and adverbs each).

###  0.B.93 Spanish Gender Swap

This transformation changes the gender of all animate entities (mostly
referring to people, and some animals) in a given Spanish sentence from
masculine to feminine. This includes masculine nouns with feminine equivalents
(e.g., doctor → doctora), nouns with a common gender (“sustantivos comunes en
cuanto al género”, e.g., el violinista → la violinista), personal pronouns,
and (optionally) given names often used with a given gender (e.g., Pedro →
Alicia). Epicene nouns are excluded. In addition, the gender of adjectives,
determiners, pronouns and participles are modified in order to maintain the
grammatical agreement.

###  0.B.94 Speech Disfluency Perturbation

This perturbation randomly inserts speech disfluencies in the form of filler
words into English texts. With a given probability (0.2 by default), a speech
disfluency is inserted between words. The default disfluencies are "um", "uh",
"erm", "ah", and "er". At least one filler word is always inserted by this
transformation.

###  0.B.95 Paraphrasing through Style Transfer

This transformation provides a range of possible styles of writing English
language. The following styles can be chosen:

  * •

Shakespeare - Trained on written works by Shakespeare.

  * •

Switchboard - Trained on a collection of conversational speech transcripts.

  * •

Tweets - Trained on 5.2M English tweets.

  * •

Bible - Trained on texts from the Bible.

  * •

Romantic poetry - Trained on romantic poetry.

  * •

Basic - A light, basic paraphraser with no specific style.

The transformation follows the models and formulations by Krishna et al.
(2020).

###  0.B.96 Subject Object Switch

This transformation switches the subject and object of English sentences to
generate new sentences with a very high surface similarity but very different
meaning. This can be used, for example, for augmenting data for models that
assess Semantic Similarity.

###  0.B.97 Sentence Summarizaiton

This transformation compresses English sentences by extracting subjects,
verbs, and objects of the sentence. It also retains any negations. For
example, “Stillwater is not a 2010 American live-action/animated dark fantasy
adventure film” turns into “Stillwater !is film”. Zhang et al. (2021) used a
similar idea to this transformation.

###  0.B.98 Suspecting Paraphraser for QA

This paraphraser transforms a yes/no question into a declarative sentence with
a question tag 303030https://www.englishclub.com/grammar/tag-questions.htm,
which helps to add more question specific informality to the dataset. Example:
”Did the American National Shipment company really break its own fleet?” ->
”The American National Shipment company really broke its own fleet, didn’t
it”.

###  0.B.99 Swap Characters Perturbation

This perturbation randomly swaps two adjacent characters in a sentence or a
paragraph with a default probability Zhang et al. (2019a).

###  0.B.100 Synonym Insertion

This perturbation adds noise to all types of text sources (sentence,
paragraph, etc.) by randomly inserting synonyms of randomly selected words
excluding punctuations and stopwords Marivate and Sefara (2020).

###  0.B.101 Synonym Substitution

This perturbation randomly substitutes some words in an English text with
their WordNet Miller (1998) synonyms.

###  0.B.102 Syntactically Diverse Paraphrasing using Sow Reap models

This transformation is capable of generating multiple syntactically diverse
paraphrases for a given sentence based on the work of Goyal and Durrett
(2020). The model paraphrases inputs using a two step framework: 1) SOW
(Source Order reWriting): This step enumerates multiple feasible syntactic
transformations of the input sentence. 2) REAP (REarrangement Aware
Paraphrasing): This step conditions on the multiple reorderings/ rearragements
produced by SOW and outputs diverse paraphrases corresponding to these
reoderings. The transformation is designed to work only on single-sentence
inputs. Multi-sentence inputs results in an empty string/no transformation.
The model are trained on the ParaNMT-50M dataset Wieting and Gimpel (2017);
Wieting et al. (2017), which can be argued to be a bit noisy.

###  0.B.103 Subsequence Substitution for Sequence Tagging

This transformation performs same-label subsequence substitution for the task
of sequence tagging, which replaces a subsequence of the input tokens with
another one that has the same sequence of tags Shi et al. (2021). This is done
as follows: (1) Draw a subsequence A from the input (tokens, tags) tuple. (2)
Draw a subsequence B within the whole dataset, with the same tag subsequence.
(3) Substitute A with B in the input example.

###  0.B.104 Change English Tense

This transformation converts English sentences from one tense to the other,
for example simple present to simple past. This transformation was introduced
by Logeswaran et al. (2018).

###  0.B.105 Token Replacement Based on Lookup Tables

This transformation replaces input tokens with their perturbed versions
sampled from a given lookup table of replacement candidates. Lookup tables
containing OCR errors and misspellings from prior work are given as examples.
Thus, by default, the transformation induces plausible OCR errors and human
typos to the input sentence.

The transformation is an adapted and improved version of the lookup table-
based noise induction method from Namysl et al. (2020). The OCR lookup table
is from Namysl et al. (2021) and the misspellings from Piktus et al. (2019).

###  0.B.106 Transformer Fill

This perturbation replaces words based on recommendations from a masked
language model. The transformation can limit replacements to certain POS tags
(all enabled by default). Many previous papers have used this technique for
data augmentation (Ribeiro et al., 2020; Li et al., 2020b, inter alia).

###  0.B.107 Underscore Trick

This perturbation adds noise to the text sources like sentence, paragraph,
etc. This transformation acts like a perturbation to test robustness. It
replaces some random spaces with underscores (or even other selected symbols).
This perturbation would benefit all tasks which have a
sentence/paragraph/document as input like text classification and text
generation, especially on tasks related to understanding/generating scripts.

###  0.B.108 Unit converter

This transformation converts length and weight measures to different units
(e.g., kilometers to miles) picking at random the new unit but converting
accurately the quantity. The transformation conserves the format of the
original quantity: "100 pounds" is converted to "1600 ounces" but "one-hundred
pounds" is converted to "one thousand, six hundred ounces". Generated
transformations display high similarity to the source sentences.

###  0.B.109 Urban Thesaurus Swap

This perturbation randomly picks nouns from the input source to convert to
related terms drawn from the Urban Dictionary
313131https://www.urbandictionary.com/ resource. It can be applied to an input
text to produce semantically-similar output texts in order to generate more
robust test sets. We first select nouns at random, then query the Urban
Thesaurus website 323232https://urbanthesaurus.org/ to obtain a list of
related terms to swap in Wilson et al. (2020).

###  0.B.110 Use Acronyms

This transformation changes groups of words for their equivalent acronyms.
It’s a simple substitution of groups of words for their acronyms. It helps to
increase the size of the dataset as well as improving the understanding of
acronyms of models trained on data augmented with this transformation. This
transformations works to increase the data for any task that has input texts.
It is specially interesting for tasks on semantic similarity, where models
should be aware of the equivalence between a set of words and their acronym.
The quality of the transformation depends on the list of acronyms. As of now,
this list was scraped from wikipedia’s List of Acronyms
333333https://en.wikipedia.org/wiki/Lists_of_acronyms and naively filtered,
which leaves space for improvement .

###  0.B.111 Visual Attack Letter

This perturbation replaces letters with visually similar, but different,
letters. Every letter was embedded into 576-dimensions. The nearest neighbors
are obtained through cosine distance. To obtain the embeddings the letter was
resized into a 24x24 image, then flattened and scaled. This follows the Image
Based Character Embedding (ICES) Eger et al. (2019a).

The top neighbors from each letter are chosen. Some were removed by judgment
(e.g. the nearest neighbors for ’v’ are many variations of the letter ’y’)
which did not qualify from the image embedding Eger et al. (2019b).

###  0.B.112 Weekday Month Abbreviation

This transformation abbreviates or expands the names of months and weekdays,
e.g. Mon. -> Monday. Generated transformations display high similarity to the
source sentences and does not alter the meaning and the semantic of the
original texts. It does not abbreviate plural names, e.g. Sundays, and does
not influence texts without names of weekdays or months.

###  0.B.113 Whitespace Perturbation

This perturbation adds noise to text by randomly removing or adding
whitespaces.

###  0.B.114 Context Noise for QA

This transformation chooses a set of words at random from the context and the
question and forms a sentence out of them. The sentence is then prepended or
appended to the context to create a new QA pair. The transformation is
inspired by the the AddAny method described in Adversarial SQUAD Jia and Liang
(2017b). However, instead of probing the model to generate adversaries, random
words from the context and the question are simply selected and joined
together into a sentence, ignoring grammaticality. The transformation attempts
to create novel QA pairs assuming that the introduction of random words to the
context is less likely to change the answer choice to an asked question.

###  0.B.115 Writing System Replacement

This transformation replaces the writing system of the input with another
writing system. We use CJK Unified
Ideographs343434https://en.wikipedia.org/wiki/CJK_Unified_Ideographs as the
source of characters for the generated writing systems. The transformation
would benefit text classification tasks, especially in the cases where the
input writing system is undeciphered.

###  0.B.116 Yes-No Question Perturbation

This transformation turns English non-compound statements into yes-no
questions. The generated questions can be answered by the statements that were
used to generate them. The text is left largely unchanged other than the
fronted/modified/added auxiliaries and be-verbs.

The transformation works by getting dependency parse and POS tags from a
machine learning model and applying human-engineered, rule-based
transformations to those parses/tags. This transformation would particularly
benefit question-answering and question-generation tasks, as well as providing
surplus legal text for language modeling and masked language modeling.

###  0.B.117 Yoda Transformation

This perturbation modifies sentences to flip the clauses such that it reads
like "Yoda Speak". For example, "Much to learn, you still have". This form of
construction is sometimes called "XSV", where "the “X” being a stand-in for
whatever chunk of the sentence goes with the verb", and appears very rarely in
English normally. The rarity of this construction in ordinary language makes
it particularly well suited for NL augmentation and serves as a relatively
easy but potentially powerful test of robustness.

##  Appendix 0.C Filters

The following is the list of all submitted filters to NL-Augmenter. Filters
are used to filter data and create subpopulations of given inputs, according
to features such as input complexity, input size, etc. Therefore, the output
of a filter is a boolean value, indicating that whether the input meet the
filter criterion. We discuss the implementations of each filter alongwith
their limitations. The title of each filter subsection is clickable and
redirects to the actual python implementation.

###  0.C.1 Code-Mixing Filter

This filter identifies whether the input text is code-mixed. It checks that
there is at least one sentence in the text where there are tokens representing
at least ‘k’ unique languages (with at least a ‘threshold‘ level of confidence
that the token is of that language). It is useful for collecting code-mixed
data to test the model’s performance on multilingual tasks. The filter relies
on ftlid353535https://pypi.org/project/ftlid/ for language detection,
therefore, this filter might be limited by the performance of the language
detection tool.

🖙 (containing code-mixing) Yo estaba con Esteban yesterday, he was telling me
about lo que su esposa vio en los Estados Unidos.
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.2 Diacritics Filter

This filter checks whether any character in the sentence has a diacritic. It
can be used to create splits of the dataet where the sentences have
diacritics. Accented characters are typically among the rarer characters and
checking the model performance on such a split might help investigate model
robustness.

🖙 (containing diacritics) She lookèd east an she lookèd west.
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.3 Encoding Filter

This filter filters examples which contain characters outside a given
encoding. It can be used to find examples containing e.g. non-ASCII Unicode
characters. Filtering out and testing examples that contain these characters
can provide feedback on how to improve the models accordingly, since most
models are trained with plain English text, which contains mostly ASCII
characters. Sometimes non-ASCII character are even explicitly stripped away.

🖙 (containing non-ASCII characters) That souvenir sure was expensive at 60£..
or was it 60€?
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.4 Englishness Filter

This filter identifies texts that contain uniquely British spellings,
vocabulary, or slang. The filter uses a vocabulary of common British
words/phrases and checks the number of occurence of British words in the given
texts. The text is selected if the number exceeds a pre-defined threshold.

🖙 (containing British spellings) Colour is an attribute of light that is
perceived by the human eye.
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.5 Gender Bias Filter

This filter filters a text corpus to measure gender fairness with respect to a
female gender representation. It supports four languages (i.e. English,
French, Polish and Russian) and can be used to define whether the female
gender is sufficiently represented in a tested subset of sentences. The filter
uses a list of lexicals, which includes filter categories such as personal
pronouns, words defining the relation, titles and names, corresponding to the
female and male genders accordingly.

🖙 (texts with unbalanced representation) "He went home", "He drives a car",
"She has returned"
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.6 Group Inequity Filter

This is a bilingual filter (for English and French languages), which helps to
discover potential group inequity issues in the text corpus. It is a topic
agnostic filter which accepts user-defined parameters, consisting of keywords
inherent to minor group (which potentially might suffer from the
discrimination), major group, minor factor and major factor. The filter first
flags the sentences as belonging to the minor, and the major groups, and then,
the sentences from each of the groups are used to define the intersection with
both factors. The filter then compares whether the percentage of major factors
exceeds that of the minor factors to determine if the sentences have group
inequity issues.

🖙 (containing group inequity issues) "He is a doctor", "She is a nurse", "She
works at the hospital"
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.7 Keyword Filter

This is a simple filter, which filters examples based on a pre-defined set of
keywords. It can be useful in creating splits for a specific domain.

🖙 (containing keyword "at") Andrew played cricket in India
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.8 Language Filter

This filter selects texts that match any of a given set of ISO 639-1 language
codes (the default language being English). Language matching is performed
using a pre-trained langid.py model instance. The model provides normalized
confidence scores. A minimum threshold score needs to be set, and all
sentences with confidence scores above this threshold are accepted by the
filter.

🖙 (is English texts) Mein Luftkissenfahrzeug ist voller Aale
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✗False

###  0.C.9 Length Filter

This filter filters data with the input text length matching a specified
threshold. It can be useful in creating data with different length
distributions.

🖙 (containing more than 3 words) Andrew played cricket in India
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.10 Named-entity-count Filter

This filter filters data where the number of Named Entities in the input match
a specified threshold (based on the supported conditions).

🖙 (containing more than 1 named entity) Novak Djokovic is the greatest tennis
player of all time.
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.11 Numeric Filter

This filter filters example which contain a numeric value. In the tasks like
textual entailment, question answering etc., a quantity (number) could
directly affect the final label/response. This filter can be used to create
splits to measure the performance separately on texts containing numeric
values.

🖙 (containing numbers in texts) John bought a car worth dollar twenty five
thousand . $\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.12 Oscillatory Hallucinations Filter

This filter is designed to operate in text generation systems’ outputs, with
the purpose of extracting oscillatory hallucinations. Oscillatory
hallucinations are one class of hallucinations characterized by repeating
bigram structure in the outputRaunak et al. (2021). Typically, these behaviors
are observed in models trained on noisy corpora. The filter counts the
frequency of bigrams in both source and target texts, and compare the
frequency difference with a pre-set threshold to determine whether the texts
includes oscillatory hallucinations.

🖙 (containing hallucinations in target texts) Source: "Community, European
Parliament common regional policy, Mediterranean region", Target:
"Arbeitsbedingungen, berufliche Bildung, berufliche Bildung, berufliche
Bildung" $\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.13 Polarity Filter

This filter filters a transformed text if it does not retain the same polarity
as an original text. This filter helps not to distort training data during
augmentation for sentiment analysis-related tasks. While generating new data
for a sentiment analysis task, it is important to make sure that generated
data is labelled correctly.

🖙 (texts retaining polarity) "Hotel is terrible", "Hotel is great"
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✗False

###  0.C.14 Quantitative Question Filter

This is a simple rule-based filter that can be used to identify quantitative
questions. It can help to analyse models’ performance on questions which
require numerical understanding. It is also useful to study possible biases in
question generation.

🖙 (being quantitative question) How long does the journey take?
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.15 Question type filter

This filter helps identify the question category of a question answering
example based on the question word or the named entity type of the answer.
Knowledge of the question type can help in the development of question
answering systems Parikh et al. (2019) as well as for assessing performance on
individual splits.

🖙 (being where question) Where is Delhi located ?
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.16 Repetitions Filter

This filter finds texts with repetitions with simple heuristic rules. It might
be helpful in finding repetitions that frequently occur in the spoken language
data.

🖙 (containing repetitions in texts) I I want to sleep
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.17 Phonetic Match Filter

This filter selects texts that contain matching entries to a list of supplied
keywords. It first transform the input sentence and the keywords into phenetic
units and then compare whether the two phenetic unit sets have overlap.

🖙 (containing homophones of keyword "trombone") I left my trombno on the train
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.18 Special Casing Filter

This filter checks if the input sentence has a special casing, i.e. the string
is either all lowercased, all uppercased or has title casing. It might be
useful for creating splits that contain texts with unusual casing, e.g.
misspellings.

🖙 (text being uppercased/lowercased/titlecased) let’s go to chipotle
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.19 Speech-Tag Filter

This filter filters an example text based on a set of speech tags and
identifies whether the count of selected POS tags meet the pre-defined
conditions (e.g. above the threshold).

🖙 (containing 1 verb and 2 numbers in texts) It all happened between November
2007 and November 2008.
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.20 Token-Amount filter

This filter filters an example text based on whether certain keywords are
present in a specified amount.

🖙 (containing 2 occurances of "in") Andrew played cricket in a soccer stadium
in India at 9pm
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True

###  0.C.21 Toxicity Filter

This filter filters an example text which has a toxicity value matching a
particular threshold. It uses a pre-trained toxicity detector, which can
provide 7 toxicity scores. All the 7 types of toxicity scores can be used as
criteria for the filtering.

🖙 (text being toxic) I disagree. It is not supposed to work that way.
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✗False

###  0.C.22 Universal Bias Filter

This filter works the same way as the Gender Bias Filter, but measures balance
of representation for more categories (religion, race, ethnicity, gender,
sexual orientation, age, appearance, disability, experience, education,
economic status). The lexical seeds representing these categories are
currently available in English only, however the pool of languages can be
extended by a simple addition of the lexical seeds in a desired language to
the lexicals.json file.

🖙 (texts being biased) "He is going to make a cake.", "She is going to
program", "Nobody likes washing dishes", "She agreed to help him"
$\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✗False

###  0.C.23 Yes/no question filter

This filter allows to select questions that can be correctly answered with
either ’yes’ or ’no’. Since it is rule-based, the limitation of this filter is
that questions that are ambiguous might not be recognized.

🖙 (text being yes/no question) Wasn’t she angry when you told her about the
accident? $\mathrel{\hbox{\rule[-0.2pt]{3.0pt}{0.4pt}}\mkern-4.0mu\hbox{\char
41\relax}}$✓True