# GATE X-E : A Challenge Set for Gender-Fair Translations from Weakly-Gendered
Languages

Spencer Rarrick Ranjita Naik Sundar Poudel Vishal Chowdhary All authors are
affiliated with Microsoft.Contact author at<EMAIL_ADDRESS>
###### Abstract

Neural Machine Translation (NMT) continues to improve in quality and adoption,
yet the inadvertent perpetuation of gender bias remains a significant concern.
Despite numerous studies on gender bias in translations into English from
weakly gendered-languages, there are no benchmarks for evaluating this
phenomenon or for assessing mitigation strategies. To address this gap, we
introduce GATE X-E, an extension to the GATE (Rarrick et al., 2023) corpus,
that consists of human translations from Turkish, Hungarian, Finnish, and
Persian into English. Each translation is accompanied by feminine, masculine,
and neutral variants. The dataset, which contains between 1250 and 1850
instances for each of the four language pairs, features natural sentences with
a wide range of sentence lengths and domains, challenging translation
rewriters on various linguistic phenomena. Additionally, we present a
translation gender rewriting solution built with GPT-4 and use GATE X-E to
evaluate it. We open source our contributions to encourage further research on
gender debiasing.

GATE X-E : A Challenge Set for Gender-Fair Translations from Weakly-Gendered
Languages

Spencer Rarrick Ranjita Naik††thanks: All authors are affiliated with
Microsoft. ††thanks: Contact author at<EMAIL_ADDRESS>Sundar Poudel
Vishal Chowdhary

##  1 Introduction

Despite dramatic improvement in general NMT quality and breadth of supported
languages over recent years (Team et al., 2022), gender bias in NMT output
remains a significant problem (Piazzolla et al., 2023). One such type of
gender bias is spurious gender-markings in NMT output when none were present
in the source. This occurs most frequently when translating from a weakly-
gendered language into a more strongly gendered one. We explore this
phenomenon in translations from Turkish, Persian, Finnish, and Hungarian into
English.

Gender can be marked in English through gendered pronouns (_he_ , _she_ ,
etc.) and possessive determiners (_his_ , _her_), or through a limited number
of intrinsically gendered nouns (_mother_ , _uncle_ , _widow_ , etc), many of
which are kinship terms.

In each of our selected set of source languages, all personal pronouns are
gender-neutral, such as _O_ in Turkish meaning _he/she/singular they_. These
languages do use some intrinsically gendered noun words, but not necessarily
for all of the same concepts that English does. Turkish differentiates
_mother_ (anne) from _father_ (baba), but does not differentiate _nephew_ from
_niece_ (both are _yeğen_).

Figure 1:  Gender Bias in Turkish-English Translation. When translating from
Turkish to English, the model tends to use the female pronoun _she_ for
gender-unspecified individuals, likely due to a perceived link between women
and child care. This bias can be mitigated by providing feminine, masculine,
and neutral rewrites.

This difference in gender on third-person singular pronouns leads to
translation scenarios such as the one seen in Figure 1, where someone with no
specified gender in the source is marked as female in the translation through
the pronoun _she_. NMT models often make gender assignments according to
stereotypes (Stanovsky et al., 2019) \- in this case a model appears to
associate child care with women. One remedy for this category of problems is
to supplement the default feminine translation with masculine and gender-
neutral alternatives, so that all possible gender interpretations are covered.
This can be accomplished by applying a gender rewriter to the original NMT
output, as shown in the bottom portion of Figure 1.

GATE (Gender-Ambiguous Translation Examples, Rarrick et al. 2023) introduced
an evaluation benchmark for gender rewrites for translations from English into
French, Spanish, and Italian. In this work, we introduce GATE X-E111X-E
indicates translation from ‘X’ language into English, an extension to GATE
that focuses on translations _into_ English from a set of more weakly-gendered
languages. It consists of natural sentences with strong diversity of sentence
lengths and domains, and challenges translation rewriters on a wide range of
linguistic phenomena. GATE X-E contains between 1250 and 1850 instances for
each of our language pairs.

We also present a translation-rewriting solution that utilizes GPT-4 (OpenAI,
2022) to provide gendered and gender-neutral alternatives. It achieves high
accuracy on the pronoun-only subset of GATE X-E, while. Finally, we also
perform human evaluation and provide a detailed error analysis of the results.

The remainder of this paper is organized as follows – In section 2, we discuss
the corpus creation process and structure of GATE X-E. In section 3, we
discuss how various properties can affect the difficulty of translation
rewriting problems. In section 4 we introduce a GPT-4-based translation-
rewriting solution and discuss how it is evaluated with GATE X-E. In section 5
we discuss results of our experiments and perform detailed error analysis.
Finally, In section 6 we cover related work.

##  2 GATE X-E Dataset

We introduce GATE X-E by describing the annotation process and labels used, as
well as providing statistics on the collected data.

###  2.1 Arbitrarily Gender-Marked Entities

Following Rarrick et al. (2023), we use Arbitrarily Gender-Marked Entity
(AGME) to refer to individuals whose gender is not marked in a source
sentence, but is in a translation, either through a gendered pronoun or an
intrinsically gendered noun. Presence of an AGME in a translation indicates
that alternate gender translations are possible.

The subject pronoun from the example translation shown in Figure 1 is an AGME.
Because there is no gender marking in the source sentence, it is valid to
translate the subject as _she_ , _he_ or _they_.

###  2.2 Dataset Creation and Annotation

All instances in GATE X-E consist of a single source sentence with one or more
translations covering possible gender interpretations. We pulled sentence
pairs for each of our language pairs from several corpora found on
OPUS222https://opus.nlpl.eu/: Europarl (Koehn, 2005), TED talks (Raine, 2020),
tatoeba333https://tatoeba.org/en/about, wikimatrix (Schwenk et al., 2021),
OpenSubtitles (Lison and Tiedemann, 2016), QED (Lamm et al., 2021) and
CCAligned (El-Kishky et al., 2020). We then apply the following filters:

  * •

The source sentence scores at least 0.7 match for the intended language when
using the python langdetect444https://pypi.org/project/langdetect/ package.

  * •

The English translation contains at least one word on a curated word list
consisting of 79 English nouns (e.g. _mother, uncle, actress, duke_) and
pronouns (_he, she, him, her, his, hers, himself, and herself_.) This list is
found in Table 13 in the appendix.

We then sampled sentences from the filtered set and provide them to
annotators. From this data, the annotators selected appropriate sentences and
annotated them for entity types, number of AGMEs, and gendered-alternative
translations if AGMEs are present. To be included, a translation must include
at least one gender-marked term in the target, which could be a
pronoun555including possessive determiners _his_ , _her_ , _their_ or
noun666including other gender-marking modifiers, such as _female_.

For each language, a second annotator then reviewed the data to correct errors
and inconsistencies. All of the annotators are native speakers of the source
language, fluent in English, and hold advanced degrees in linguistics or a
related field.

If there are one or two AGMEs in the translated pair, they will provide
translation variants so that all possible gender combinations for those AGMEs
(among female, male, and neutral) are covered. They do so by replacing all
gendered pronoun and noun mentions with corresponding words of the respective
gender. Neutral variants are omitted if there is no suitable gender neutral
term in common usage for a concept. For example, the term _nibling_ exists as
a gender neutral variant of _niece_ or _nephew_ , but is not in common usage
and so neutral variants of translations using _niece_ or _nephew_ would be
left out.

Some sentences may contain a mixture of references to AGMEs as well as to
humans who are gender-marked in the source. In these cases, gender indicated
in the source will be preserved in all translations, as in _father_ and _his
will_ in the example shown in Figure 2. In this example, _Babası_ explicitly
indicates _father_ in the source.

Src |  Babası vasiyetinde arabayı ona bıraktı.
---|---
Fem |  Her father left her the car in his will.
Masc |  His father left him the car in his will.
Neut |  Their father left them the car in his will.
Lbls |  target_only_gendered_pronoun, source+target_gendered_noun+pronoun, 1-AGME, mixed

Figure 2: GATE X-E Example Instance. This includes Turkish source; feminine,
masculine and gender-neutral English translations; and labels.

###  2.3 Labels

The labels used in GATE X-E are defined in Table 1, along with examples for
each. All instances in GATE X-E refer to at least one person who is marked for
gender in the English target. We include both positive and negative examples.
In positive examples, at least one of those individuals was not marked for
gender in the source, and is therefore an AGME, meaning that alternative
translations with different gender markings are possible. In negative
examples, all individuals who are marked for gender in the target are also
marked in the source, so no alternative translations are possible.

Description |  Example (tr $>$ en)
---|---
Negative/Non-AGME labels
source+target_gendered_noun |
A person is referred to by a gendered noun in both source and translations. |  Git ve erkek kardeşine yardım et. $\rightarrow$
Go and help your brother.
source+target_gendered_noun+pronoun |
A person referred to by a gendered noun in the source is referred to by both a gendered noun and one or more gendered pronouns in the translations. |  Annem zaten kararını verdi. $\rightarrow$
My mom has already made her decision.
source_gendered_noun_target_pronoun |
A person is referred to by a gendered noun in the source, and one or more gendered pronouns in the translations (but not by a gendered noun). |  O, gerçek bir bilim adamıdır. $\rightarrow$
He is a scholar to the core.
(_bilim adamı_ indicates a male scholar)
non-AGME-name |
A non-AGME person is referred to by name. |  Umut’un _torunu_ ünlü bir yazar değil mi? $\rightarrow$
Umut’s _granddaughter/grandson/grandchild_ is a famous writer, isn’t/aren’t
_she/he/they_?
Positive/AGME labels
target_only_gendered_noun |
A person who is not gender-marked in the source is referred to with a gendered noun in the translations. |  Yeğenim bugün geliyor. $\rightarrow$
My niece/nephew is coming today.
target_only_gendered_pronoun |
A person who is not gender-marked in the source is referred to with a gendered pronoun in the translations. |  Onun yardımı paha biçilmezdi. $\rightarrow$
Her/His/Their help has been invaluable.
target_only_gendered_noun+pronoun |
A person who is not gender-marked in the source is referred to with both a gendered noun and gendered pronoun in the translations. |  Torunun işini seviyor olmalı. $\rightarrow$
Your granddaughter/grandson/grandchild must love her/his/their job.
name |
An AGME is referred to by name. We treat personal names as non-gender-marking. |  Beyza akşam yemeğini bitiremedi. $\rightarrow$
Beyza wasn’t able to finish her/his/their dinner.
(_Beyza_ is typically considered a feminine name)
Other
mixed |
Both positive and negative examples are present |  _Baba_ sı yine uçağ _ını_ kaçırdı. $\rightarrow$
Her/His/Their _father_ missed _his_ plane again.
_N_ AGME(s) |
_N_ is a whole number representing the number of AGMEs in the instance. Negative examples are annotated as 0 AGMEs. |  0 AGME: My mother read her book.
1 AGME: She/He ate her/his lunch alone.
2 AGME: She/He annoyed _her/him_ with her/his music.
Table 1: Label Definitions and Examples. Words relevant to the label are
bolded or italicized in source and target. _Pronoun_ in these definitions
includes possessive determiners _her_ , _his_ , _their_.

###  2.4 Corpus Statistics

Table 4 in the appendix provides a comprehensive breakdown of corpus
statistics for GATE X-E, with instance counts per language for each label.

More than half of the instances for each language pair have exactly one AGME,
with around 20-30% are negative instances, having no AGMEs at all. Most AGMEs
are target_only_gendered_pronoun, meaning that they have no gender markings in
the source and the only words in the target that mark their gender are
pronouns. This is in part because there are relatively few nouns which are
gendered in English but not gendered in the source languages.

Non-AGME references will involve a gendered noun on the source, and for most
of the languages about half of these also include a pronoun reference.

Each language pair contains around 250 instances labeled mixed. These
instances contain at least one AGME and at least one individual who is marked
for gender in the source sentence. name indicates that there is an AGME who is
referred to by name. Between 15 and 25% of instances per language have this
label. textttnon-AGME-name indicates that a non-AGME person is referred to by
name. We present the distribution of sentence lengths in source and target
languages in Figure 3.

##  3 Translation Gender Rewriting

Translation gender rewriting is the process of taking a translated source-
target pair and producing alternative translations with different gender
markings. In a correctly rewritten translation, the gender markings should
remain compatible with all gender information found in the source sentence
(Habash et al., 2019). We consider this problem from the viewpoint of a user
who wishes to see a set of three gendered-alternative translations with
uniform output gender side-by-side: all-female, all-male and all-neutral.
Because the translations will be viewed as a set, the translations should only
vary from one-another in specific words that mark gender.

Here we discuss the difference in difficulty between rewrite problems where
gendered nouns are included and those where gender is only marked by pronouns.

#### Pronoun-Only Problems

For our source languages, if the only gender markers in the target sentence
are gendered pronouns, there typically cannot be gender markers in the source
sentence, since those languages do not have any gendered pronouns. We can
therefore assume that if we have no gender information from external context,
then all individuals mentioned by a gendered pronoun in the translation must
be AGMEs.

Rewriting in this scenario reduces to the relatively simple task of adjusting
surface forms of all pronouns to match the desired gender. For rewrites
involving _he_ or _she_ to gender-neutral _they_ , some verbs must
additionally be adjusted to compatible surface forms. Since our focus is on
rewrites with uniform gender assignments in the output (all-female, all-male,
or all-neutral), this removes any need to determine which pronouns refer to
which individual where more than one is mentioned.

Here we see an example of a Pronoun-only instance with two AGMEs:

Female + Female | She gave her her umbrella.
---|---
Male + Male | He gave him his umbrella.
Neutral + Neutral | They gave them their umbrella.

Instances of GATE X-E that fall into this subtype will always have the label
target_gendered_pronoun_only, and never any labels with containing
gendered_noun.

Note that there is an exceptional scenario, where external context makes it
reasonable for a gendered noun in one of our source languages to be translated
into a gendered pronoun in English. See the example for
source_gendered_noun_target_pronoun in Table 1, and refer to Appendix
References for further discussion.

#### Gendered-Noun Problems

If we expand our scope to include translations containing gendered nouns, we
encounter several new challenges that render the rewriting problem
significantly more difficult. The following pair of examples illustates some
of those new challenges.

_Kardeşine ziyarete gelip gelmeyeceğini sordu._

$\big{\Downarrow}$

_He asked his sister if she would visit._

In this translation, both the male and female individuals are AGMEs since
_Kardeşine_ simply denotes a _sibling_ without any gender specification.
Therefore, there are nine possible rewrites, including the original
translation: _He_ and _his_ can be optionally replaced with _she/her_ or
_they/their_ , and _sister/she_ can be optionally replaced with _brother/he_
or _sibling/they_.

In the next example, however, the gender of the sibling is specified in the
source as female by the addition of the word _Kız_ , even though the default
English translation is exactly the same:

_Kız kardeşine ziyarete gelip gelmeyeceğini sordu._

$\big{\Downarrow}$

_He asked his sister if she would visit._

Here _sister_ must remain fixed because of the gender marking in the source.
_She_ is also fixed because it is coreferent with the sister. Only the
individual referred to by _he_ and _his_ is an AGME, so only three valid
rewrites exist (including the original):

Female | She asked her sister if she would visit.
---|---
Male | He asked his sister if she would visit.
Neutral | They asked their sister if she would visit.

More specifically, the additional challenges inherent in this problem class
include the following.

  * •

Gendered-marked nouns may appear on the source as well, so we must examine
both source and target to determine if variants are needed. This is
demonstrated by the behavior of _sister_ in the two examples above.

  * •

Gendered pronouns in the target may refer to individuals whose gender was
marked in the source, and are therefore not appropriate to modify. When
multiple individuals are mentioned, we must differentiate which ones refer to
such non-AGME individuals to produce a correct rewrite. This is demonstrated
by the behavior of _she_ in the two examples above.

A system that is capable of solving these problems must then able to
implicitly perform coreference resolution and alignment of nouns between the
source and target, significantly increasing the complexity of a solution.

##  4 Experiments

We introduce a novel translation gender-rewriting solution that leverages
GPT-4 and evaluate it on GATE X-E.

###  4.1 Rewriting with GPT-4

Our solution uses chain-of-thought prompting (Wang et al., 2023) to elicit
GPT-4 to produce three variant translations for each input source-translation
pair – all-neutral, all-female and all-male, while leaving any gendered words
associated with non-AGMEs unmodified.

Each step in the prompt is accompanied by detailed clarifications and example
vocabulary. The prompt also includes three full examples, customized per
source language. The examples indicate that "None" should be returned in lieu
of translation variants when there are no AGMEs present. The full prompt for
Turkish-English can be found in the appendix in Figures 7 and 8.

We process extract the three variants from GPT’s output. The output in some
cases is "None", in which case we treat all outputs as identical to the
original translation.

###  4.2 Data Preparation

Each GATE X-E instance consists of a source sentence and a set of translations
in English. We pull a set of sentence-tuples from each instance for use in
evaluation as test tuples. Test tuples consists of a source sentence, an
original translation, and a reference for a specific set of gender
assignments. We group these by the gender-assignments of AGMEs in the original
translation, producing four subsets: feminine, masculine, mixed feminine and
masculine (with 2 AGME), and negative.

For 1-AGME instances, we extract four tuples:
feminine(f)${\rightarrow}$masculine(m), f${\rightarrow}$neutral(n),
m${\rightarrow}$f, and m${\rightarrow}$n. For 2-AGME instances, we
additionally extract 6 tuples covering both combinations of mixed-gender
original translations and all three sets of uniform gender assignments in the
reference ({f+m, m+f}$\rightarrow${f+f, m+m, n+n}).

Negative instances, which do not include AGMEs and should not be modified by
the rewriter, are handled separately. For each negative instance we create
tuples to check that the original target remains unmodified in masculine,
feminine and neutral outputs.

For simplicity, we exclude the handful of instances with three or more AGMEs.
We do not include any tuples that include gender-neutral forms in the original
target because because of the ambiguity in distinguishing singular neutral
pronouns them plural _they_ forms. For any instance where the neutral
reference is empty (e.g. because there is no neutral form of term), no test
tuple with neutral reference is created.

As we are interested in exploring the difference in behavior of our solution
we also mark each test tuple according to whether they contain gendered nouns
in either source or target, or only gendered pronouns in the target. The
former group we designate as _Gendered-Noun_ and the latter group _Pronoun-
Only_. Note that negative examples always contain a gendered noun in the
source, and so they do not have a distinction between the two types.

Table 5 in the appendix shows counts for each source language and category.
Note that tuples with uniform gender original translations (f/m) are always
created in pairs, so they have the same counts.

##  5 Results

Language | Subtype | Fem Orig $\uparrow$ | Masc Orig $\uparrow$ | Mixed Orig $\uparrow$ | Negative $\uparrow$
---|---|---|---|---|---
M | Neut | F | Neut | F | M | Neut | Gender | Neut
tu $\rightarrow$ en | Overall | 0.81 | 0.86 | 0.80 | 0.85 | 0.78 | 0.82 | 0.57 | 0.87 | 0.80
Pronoun-Only | 0.99 | 0.99 | 0.96 | 0.99 | 0.93 | 0.96 | 0.98 | - | -
Gendered-Noun | 0.59 | 0.63 | 0.60 | 0.63 | 0.75 | 0.80 | 0.45 | - | -
fi $\rightarrow$ en | Overall | 0.90 | 0.80 | 0.82 | 0.81 | 0.67 | 0.83 | 0.63 | 0.89 | 0.88
Pronoun-Only | 0.98 | 0.98 | 0.97 | 0.98 | 0.94 | 0.96 | 0.97 | - | -
Gendered-Noun | 0.75 | 0.53 | 0.54 | 0.57 | 0.34 | 0.68 | 0.54 | - | -
hu $\rightarrow$ en | Overall | 0.85 | 0.79 | 0.84 | 0.81 | 0.75 | 0.92 | 0.78 | 0.84 | 0.84
Pronoun-Only | 0.98 | 0.99 | 0.96 | 0.98 | 0.92 | 0.97 | 0.97 | - | -
Gendered-Noun | 0.66 | 0.61 | 0.68 | 0.76 | 0.56 | 0.86 | 0.56 | - | -
fa $\rightarrow$ en | Overall | 0.86 | 0.81 | 0.86 | 0.81 | 0.81 | 0.84 | 0.66 | 0.54 | 0.53
Pronoun-Only | 0.99 | 0.98 | 0.99 | 0.98 | 0.93 | 0.93 | 0.96 | - | -
Gendered-Noun | 0.67 | 0.54 | 0.68 | 0.55 | 0.78 | 0.81 | 0.58 | - | -
Table 2: Accuracy of our Rewriting Solution. Accuracy on test elements for
each source language, problem subtype, original target gender (top header
row), and requested output gender (second header row). Only exact matches to
reference are counted.

Table 2 shows the accuracy of our solution on test elements over each
combination of source language, and subtype level, with _Overall_ indicating
an aggregate score over _Pronoun-Only_ and _Gendered-Noun_ elements. The top
labels in the header row indicate gender of AGMEs in the original target, and
the bottom indicates the desired output gender.

Following Rarrick et al. (2023), we focus on exact match accuracy to the
reference. Frequently only one or two words will be different between an
original translation and a correct rewrite. In this context, metrics such as
BLEU (Papineni et al., 2002) and WER are not very effective at determining the
significance of single extraneous or missed word modification.

###  5.1 Pronoun-Only Subset

On the Pronoun-Only subset, the solution rarely makes mistakes for masculine
and feminine original targets, with scores ranging from 0.96 to 0.99. Test
cases where the original target has mixed gender all come from 2-AGME
instances. These skew towards longer and more complicated sentence, which thus
leads to slightly lower accuracy.

On most language pairs we see that Pronoun-Only rewrites into masculine
outperform rewrites into feminine by a few percentage points. The largest gap
is 5 points for mixed-gender original target on Hungarian. This may indicate a
slight general tendency of GPT-4 to prefer phrasing using masculine pronouns.

###  5.2 Gendered-Noun Subset

Scores on the Gendered-Noun subset are substantially lower than for _Pronoun-
Only_ , generally ranging from about 0.5 to 0.8, with Finnish
mixed$\rightarrow$feminine as an outlier at the low end at 0.34. The score
differential with _Pronoun-Only_ can be mostly attributed to the more
difficult nature of the problems. However, there are some cases where there
are multiple acceptable alternative phrases to use in a rewrite, and GPT-4
chooses a different one from the reference.

This effect is most pronounced in the Finnish _Complex_ mixed-original target
data. This subset contains a large amount of data from Europarl that includes
titles such as _Mr._ and _Mrs._ and addresses to _Mr. President_. The feminine
rewrites often choose a mismatched form, such as _Ms. Müller_ rather than
_Mrs. Müller_ , or _Mrs. President_ rather than _Madam President_.

Similarly, neutral rewrites on such sentences often produces _Honorable
President_ , _Chairperson_ , or _Honorable Speaker_ as a rewrite of _Mr.
President_ or _Madam President_ , mismatching _Honored President_ in the
reference.

###  5.3 Negative Subset

Error Type | Pronoun-Only $\uparrow$ | Gendered-Noun $\uparrow$ | Negative $\uparrow$
---|---|---|---
Gen (%) | Neut (%) | Gen (%) | Neut (%) | Gen (%) | Neut (%)
extraneous noun change | 12.5 | 10.0 | 2.8 | 3.5 | 0.0 | 34.5
extraneous pronoun change | 0.0 | 10.0 | 4.0 | 5.6 | 100 | 65.5
missing noun change | 0.0 | 0.0 | 55.5 | 51.9 | - | -
missing pronoun change | 87.5 | 80.0 | 37.5 | 38.6 | - | -
total errors | 30 | 8 | 427 | 381 | 57 | 58
Table 3: Distribution of Errors from Human Evaluation for Turkish-English.
Shows percentages of errors over Pronoun-Only, Gendered-Noun and Negative
subsets of the data, for gendered and neutral requested outputs.

_Negative $\rightarrow$Gender_ score indicates how often both the female and
male variants produced by our solution exactly matched the original
translation, while _Negative $\rightarrow$Neutral_ measures the same for the
neutral variant. An output of _None_ from GPT-4 is also possible, indicating
that all variants should be considered a copy of the original translation. On
the negative data subset, this is considered a reference match.

For Turkish, Finnish and Hungarian, we see scores for both _Gender_ and
_Neutral_ subsets in the 0.8 to 0.9 range. Farsi is an outlier with 0.54 and
0.53 at the low end.

With the exception of Turkish, we find that we almost never see matches on
negative test items aside from _None_ outputs. For Turkish, 47% of non-_None_
outputs match for Gendered and 15% of neutral outputs. For all languages,
however, we do see a large number of neutral outputs that match a version of
the original translation where all pronouns are modified to their neutral
variants777and verb agreement modified to match. This occurs in particular
when there are gendered nouns in the source which determine the gender of some
pronouns in the translation as well. For example, instead of _the man has
something under his coat_ , it would output _the man has something under their
coat_.

If were relax matching criteria to allow this variant, neutral negative
accuracy increases to 0.91 for Farsi, 0.92 for Turkish, 0.95 for Finnish and
0.95 for Hungarian.

###  5.4 Human Evaluation

For the Turkish data, one of the Turkish annotators provided annotations for
error type on all outputs that did not exactly match the reference. These are
found in Table 3.

For each test item, they mark whether there were nouns or pronouns were
changed where the reference and original translation matched (_extraneous
noun/pronoun change_ , as well as whether any nouns or pronouns should have
been changed but were not (_missing noun/pronoun change_). Distributions for
masculine and feminine outputs were similar, so we show combined _gendered_
outputs for each subtype. We also aggregate over mixed- and uniform-gender
inputs.

For positive test cases, missing noun and pronoun changes were far more common
than extraneous changes. For cases containing gendered nouns, noun changes
were missed more often than pronoun changes, while extraneous pronoun changes
were more common than extraneous noun changes.

For Negative test cases, gendered output only ever contained extraneous
pronouns changes, while neutral outputs did have a fair number of extraneous
noun changes. An example of these is changing _man_ to _person_ even when
_man_ was gender-marked in the source sentence.

Among missing pronoun errors, missing possessive determiners was by for the
most, with subject and object pronoun errors roughly equivalent. Missed and
extraneous reflexives were extremely rare. We also saw a single of subject-
verb agreement error each when changing _he_ and _she_ to _they_ , and one
case where other wording was changed in the sentence.

##  6 Related Work

Understanding and Assessing Gender Bias: It has been documented that machine
translation engines often make mistakes and show gender biases when
translating between languages with differing gender norms (Stanovsky et al.,
2019; Prates et al., 2019; Rescigno et al., 2020; Lopez-Medel, 2021; Prates et
al., 2019; Fitria, 2021).

Evaluation Benchmarks: Translating from English: Bentivogli et al. (2020) and
Savoldi et al. (2022) introduced the MuST-SHE dataset, which includes triplets
of audio, transcript, and reference translations for English to Spanish,
French, and Italian languages, classified by gender. Stanovsky et al. (2019)
developed the WinoMT challenge set, which includes English sentences with two
animate nouns, one of which is coreferent with a gendered pronoun.

Renduchintala et al. (2021) introduced the SimpleGEN dataset for English-
Spanish and English-German language pairs, which includes short sentences with
occupation nouns and clear gender indications. The Translated Wikipedia
Biographies dataset includes human translations of Wikipedia biographies for
gender disambiguation evaluation. Lastly, Currey et al. (2022) presented the
MT-GenEval dataset, which includes gender-balanced, counterfactual data in
eight language pairs, specifically focusing on translation from English into
eight widely-spoken languages.

Translating into English: Numerous studies have focused evaluating bias in
translating from from a weakly gendered language such as Turkish into English.
(Prates et al., 2019; Fitria, 2021; Ciora et al., 2021; Ghosh and Caliskan,
2023).

Strategies for Gender Bias Mitigation : To mitigate gender bias when
translating queries that are gender-neutral in the source language, Google
Translate announced a feature (Kuczmarski, 2018; Johnson, 2020) that provides
gender-specific translations. Both Sun et al. (2021) and Vanmassenhove et al.
(2021) have explored monolingual gender-neutral rewriting of English
demonstrating that a neural model can perform this task with reasonable
accuracy. Ghosh and Caliskan (2023) evaluates gender bias in GPT-3.5 Turbo
output when translating from gender-neutral languages into English. To the
best of our knowledge, our work is the first to leverage GPT-4 for mitigation.

##  7 Conclusion

We have presented GATE X-E, a diverse dataset covering a wide range of
scenarios relevant to translation gender-rewriting for English-target language
pairs, covering gendered and gender-neutral rewrites. We have discussed
intricacies of the English-target translation rewrite problems, and explained
what properties lead to easier or more difficult rewrite problems. We have
explored the ability of GPT-4 to provide rewrites for these translations,
showing that it can achieve very high accuracy on a pronoun-only rewriting
problems, but performs less well when gendered nouns are introduced.

We also hope that by making GATE X-E accessible to the broader research
community, we can encourage further research on gender debiasing in the
machine translation space.

##  8 Limitations

Our study has some limitations that could be addressed in future research.
Firstly, while we utilized GPT-4 for rewriting tasks, the potential of open-
source models remains unexplored and could be beneficial. Secondly, our
rewriter operates on few-shot chain-of-thought prompting. Future
investigations could consider exploring the zero-shot setting, which could
potentially be more cost-efficient.

## References

  * Bentivogli et al. (2020) Luisa Bentivogli, Beatrice Savoldi, Matteo Negri, Mattia A. Di Gangi, Roldano Cattoni, and Marco Turchi. 2020.  Gender in danger? evaluating speech translation technology on the MuST-SHE corpus.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6923–6933, Online. Association for Computational Linguistics.
  * Ciora et al. (2021) Chloe Ciora, Nur Iren, and Malihe Alikhani. 2021.  Examining covert gender bias: A case study in turkish and english machine translation models.
  * Currey et al. (2022) Anna Currey, Maria Nadejde, Raghavendra Reddy Pappagari, Mia Mayer, Stanislas Lauly, Xing Niu, Benjamin Hsu, and Georgiana Dinu. 2022.  MT-GenEval: A counterfactual and contextual dataset for evaluating gender accuracy in machine translation.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 4287–4299, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * El-Kishky et al. (2020) Ahmed El-Kishky, Vishrav Chaudhary, Francisco Guzmán, and Philipp Koehn. 2020.  CCAligned: A massive collection of cross-lingual web-document pairs.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 5960–5969, Online. Association for Computational Linguistics.
  * Fitria (2021) Tira Nur Fitria. 2021.  Gender bias in translation using google translate: Problems and solution.  _Language Circle: Journal of Language and Literature_ , 15(2).
  * Ghosh and Caliskan (2023) Sourojit Ghosh and Aylin Caliskan. 2023.  Chatgpt perpetuates gender bias in machine translation and ignores non-gendered pronouns: Findings across bengali and five other low-resource languages.
  * Habash et al. (2019) Nizar Habash, Houda Bouamor, and Christine Chung. 2019.  Automatic gender identification and reinflection in Arabic.  In _Proceedings of the First Workshop on Gender Bias in Natural Language Processing_ , pages 155–165, Florence, Italy. Association for Computational Linguistics.
  * Johnson (2020) Melvin Johnson. 2020.  A scalable approach to reducing gender bias in google translate.
  * Koehn (2005) Philipp Koehn. 2005.  Europarl: A parallel corpus for statistical machine translation.  In _Proceedings of Machine Translation Summit X: Papers_ , pages 79–86, Phuket, Thailand.
  * Kuczmarski (2018) James Kuczmarski. 2018.  Reducing gender bias in google translate.
  * Lamm et al. (2021) Matthew Lamm, Jennimaria Palomaki, Chris Alberti, Daniel Andor, Eunsol Choi, Livio Baldini Soares, and Michael Collins. 2021.  QED: A Framework and Dataset for Explanations in Question Answering.  _Transactions of the Association for Computational Linguistics_ , 9:790–806.
  * Lison and Tiedemann (2016) Pierre Lison and Jörg Tiedemann. 2016.  OpenSubtitles2016: Extracting large parallel corpora from movie and TV subtitles.  In _Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16)_ , pages 923–929, Portorož, Slovenia. European Language Resources Association (ELRA).
  * Lopez-Medel (2021) Maria Lopez-Medel. 2021.  Gender bias in machine translation: an analysis of google translate in english and spanish.  _Academia Letters_.
  * OpenAI (2022) OpenAI. 2022.  Introducing chatgpt.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and WeiJing Zhu. 2002.  Bleu: A method for automatic evaluation of machine translation.  In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_ , pages 311–318.
  * Piazzolla et al. (2023) Silvia Alma Piazzolla, Beatrice Savoldi, and Luisa Bentivogli. 2023.  Good, but not always fair: An evaluation of gender bias for three commercial machine translation systems.
  * Prates et al. (2019) Marcelo O. R. Prates, Pedro H. C. Avelar, and Luis Lamb. 2019.  Assessing gender bias in machine translation – a case study with google translate.
  * Raine (2020) Paul Raine. 2020.  Talk corpus: A web-based corpus of ted talks for english language teachers and learners.
  * Rarrick et al. (2023) Spencer Rarrick, Ranjita Naik, Varun Mathur, Sundar Poudel, and Vishal Chowdhary. 2023.  Gate: A challenge set for gender-ambiguous translation examples.  In _Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society_ , AIES ’23, page 845–854, New York, NY, USA. Association for Computing Machinery.
  * Renduchintala et al. (2021) Adithya Renduchintala, Denise Diaz, Kenneth Heafield, Xian Li, and Mona Diab. 2021.  Gender bias amplification during speed-quality optimization in neural machine translation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 99–109, Online. Association for Computational Linguistics.
  * Rescigno et al. (2020) Argentina Anna Rescigno, Eva Vanmassenhove, Johanna Monti, and Andy Way. 2020.  A case study of natural gender phenomena in translation a comparison of google translate, bing microsoft translator and deepl for english to italian, french and spanish.  In _Computational Linguistics CLiC-it_ , page 359.
  * Savoldi et al. (2022) Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2022.  Under the morphosyntactic lens: A multifaceted evaluation of gender bias in speech translation.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1807–1824, Dublin, Ireland. Association for Computational Linguistics.
  * Schwenk et al. (2021) Holger Schwenk, Vishrav Chaudhary, Shuo Sun, Hongyu Gong, and Francisco Guzmán. 2021.  WikiMatrix: Mining 135M parallel sentences in 1620 language pairs from Wikipedia.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 1351–1361, Online. Association for Computational Linguistics.
  * Stanovsky et al. (2019) Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019.  Evaluating gender bias in machine translation.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 1679–1684, Florence, Italy. Association for Computational Linguistics.
  * Sun et al. (2021) Tony Sun, Kellie Webster, Apu Shah, William Yang Wang, and Melvin Johnson. 2021.  They, them, theirs: Rewriting with gender-neutral english.
  * Team et al. (2022) NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. 2022.  No language left behind: Scaling human-centered machine translation.
  * Vanmassenhove et al. (2021) Eva Vanmassenhove, Chris Emmery, and Dimitar Shterionov. 2021.  NeuTral Rewriter: A rule-based and neural approach to automatic rewriting into gender neutral alternatives.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 8940–8948, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Wang et al. (2023) Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2023.  Towards understanding chain-of-thought prompting: An empirical study of what matters.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2717–2739, Toronto, Canada. Association for Computational Linguistics.

##  Appendix A Further details on GATE X-E

| tr $\rightarrow$ en | fa $\rightarrow$ en | fi $\rightarrow$ en | hu $\rightarrow$ en
---|---|---|---|---
total instance count | 1,429 | 1,259 | 1,832 | 1,308
target_only_gendered_noun | 142 | 118 | 159 | 95
target_only_gendered_pronoun | 1,074 | 906 | 1,096 | 914
target_only_gendered_noun+pronoun | 114 | 49 | 105 | 115
source+target_gendered_noun | 239 | 244 | 379 | 75
source+target_gendered_noun+pronoun | 328 | 292 | 361 | 422
source_gendered_pronoun_target_noun | 3 | 0 | 0 | 33
0 AGMEs | 300 | 264 | 502 | 264
1 AGME | 900 | 869 | 1,164 | 848
2 AGMEs | 225 | 124 | 161 | 192
3 AGMEs | 4 | 2 | 5 | 4
mixed | 271 | 263 | 237 | 262
name | 328 | 175 | 408 | 159
non-AGME-name | 32 | 5 | 136 | 16
Table 4: GATE X-E Statistics. Sentence counts per language associated with
each label.

Table 4 provides a comprehensive breakdown of corpus statistics for GATE X-E,
with instance counts per language for each label.

Figure 3 presents boxplots that demonstrate the sentence length distribution
on source and target for four language pairs: Finnish to English, Hungarian to
English, Persian to English, and Turkish to English. The left plot shows the
sentence lengths in the source languages, and the right plot displays the
sentence lengths in English, the target language. The legend indicates the
color corresponding to each language pair. Compared to the other three
language pairs, Finnish to English contains longer sentences.

Table 5 shows counts for each source language and category. Note that tuples
with uniform gender original translations (f/m) are always created in pairs,
so they have the same counts.

Category | tr | fa | fi | hu
---|---|---|---|---
Pronoun-Only (f/m) | 628 | 857 | 590 | 580
Gendered-Noun (f/m) | 500 | 473 | 454 | 415
Pronoun-Only (mix) | 54 | 180 | 198 | 44
Gendered-Noun (mix) | 392 | 142 | 186 | 200
Negative | 300 | 502 | 264 | 264
Table 5: Test tuple Counts By Category and Source Language. Counts of pronoun-
only, gendered-noun and negative test tuples per source language. f/m
signifies count for uniform gender original targets, while mix signifies a
mixture of male and female references in the original target. Figure 3:
Boxplots representing the distribution of sentence lengths in source and
target languages. The four language pairs are Finnish to English (fi > en),
Hungarian to English (hu > en), Persian to English (fa > en), and Turkish to
English (tu > en). The left plot represents the source language sentence
lengths, and the right plot represents the target language (English) sentence
lengths. The color of each boxplot corresponds to the language pair as
indicated in the legend.

##  Appendix B Monolingual Rewriting with GPT-3.5 Turbo

GPT-4 performs very well on the pronoun-only subset of examples. However, its
inference cost is high. Therefore, we evaluate the pronoun-only subset using
GPT-3.5 Turbo.

###  B.1 Gender-Neutral Rewriting

As shown in 3, pronoun-only uniform-gender do not require access to source
information, and so our GPT-3.5 Turbo solution is only given access to
original translation target. We first use GPT-3.5 Turbo to produce an all-
neutral rewrite and then use a rule based solution to convert the all-neutral
rewrite to gendered rewrites.

The trickiest aspects of the gender-neutral rewrite are disambiguating pronoun
classes for _her_ and _him_ , and adjusting verb forms when subjects change
from _she/he_ to _they_ , so these are the primary decisions that GPT-3.5
Turbo must make.

We experiment with zero-shot and few-shot approaches. The zero-shot approach
uses a single-sentence prompt as seen in Figure 5. The few-shot approach
expands on this prompt by adding five examples, and it can be seen in Figure
6.

###  B.2 Gender-Neutral to Gendered Rewriting

A useful simplifying observation is for uniform-gender pronoun only rewrites,
we can generate a correct feminine or masculine rewrite from the original
target and a correct all-neutral rewrite. Referring back to 12, we see that
all elements in the neutral column are unique, while the masculine and
feminine columns each have one surface form fitting two categories. Knowing
the correct neutral pronoun fully determines what pronouns should be used for
a given gender.

In practice the neutral rewrite may contain errors. To minimize their impact
on the gendered rewrites, we begin with the original translation, and map to
pronouns directly to the desired gender where unambiguous. When ambiguous
(i.e. for _her_ or _his_ in the original target), we rely on the chosen form
of the neutral pronoun to disambiguate.

Figure 4: Distribution of errors in GPT-3.5 Turbo’s zero-shot and few-shot settings. The majority of errors in both settings stem from unrelated modifications and the model’s ’None’ response, indicating no need for gender-neutral rewriting. Language Pair | Method | Neutral Rewriting | Gendered Rewriting
---|---|---|---
|  | Accuracy (%) $\uparrow$ | BLEU $\uparrow$ | WER $\downarrow$ | Accuracy (%) $\uparrow$ | BLEU $\uparrow$ | WER $\downarrow$
tr $\rightarrow$ en |  Sun et al. 2021 | 96.16 | 99.65 | 0.53 | - | - | -
Zero-shot | 97.24 | 99.30 | 0.80 | 99.50 | 99.90 | 0.90
Few-shot | 98.90 | 99.55 | 0.44 | 99.46 | 99.00 | 0.10
hu $\rightarrow$ en |  Sun et al. 2021 | 96.14 | 99.66 | 0.53 | - | - | -
Zero-shot | 96.58 | 99.04 | 1.27 | 99.27 | 99.95 | 0.08
Few-shot | 97.00 | 99.03 | 1.20 | 99.20 | 99.94 | 0.09
fi $\rightarrow$ en |  Sun et al. 2021 | 95.24 | 99.63 | 0.62 | - | - | -
Zero-shot | 94.80 | 98.61 | 1.75 | 98.41 | 99.85 | 0.24
Few-shot | 96.77 | 98.62 | 1.54 | 98.99 | 99.80 | 0.19
fa $\rightarrow$ en |  Sun et al. 2021 | 94.43 | 99.57 | 0.65 | - | - | -
Zero-shot | 95.59 | 99.00 | 1.11 | 98.75 | 99.91 | 1.13
Few-shot | 97.84 | 99.16 | 1.01 | 99.00 | 99.93 | 0.09
Table 6: Results of Gender Neutral and Gendered Rewriting on the Pronoun-Only
Subset of GATE X-E. We report the performance of the rule-based system
proposed by Sun et al. 2021. Additionally, we evaluate GPT-3.5 Turbo in both
zero-shot and few-shot settings. Gendered alternatives are generated using the
algorithm described in Section B

###  B.3 Experiments

####  B.3.1  Rewriters

Neutral rewriter Systems: We consider the following rewriting systems:

  1. 1.

Rule-based system proposed by Sun et al. (2021): It uses Spacy and GPT-2 to
resolve ambiguity with _his_ and _her_ , and to adjust verb forms as needed.
They also trained a neural model, but it was unfortunately not accessible.

  2. 2.

GPT-3.5 Turbo: We evaluate GPT-3.5 Turbo on zero-shot and few-shot settings,
using the prompts shown in Figures 5 and 6 in the appendix.

We investigated the neural model introduced by Vanmassenhove et al. (2021) as
well, but were unable to reproduce results on their test data.

For all GPT-based rewrites we set temperature $T=0$

####  B.3.2  Evaluation

We report the rewriter systems’ performance using BLEU (Papineni et al.,
2002), Word Error Rate (WER), and Accuracy.

In the gender-neutral rewriting task (Table 6), GPT-3.5 Turbo performs better
in the few-shot setting compared to the zero-shot setting. Although GPT-3.5
Turbo provides slightly higher accuracy compared to the rule-based system
proposed by Sun et al. (2021), the rule-based system performs better based on
BLEU and WER. This is because GPT-3.5 Turbo makes modifications unrelated to
neutral rewriting, as detailed in the error analysis section.

In the gendered-alternatives rewriting task (Table 6), the zero-shot setting
indicates that for resolving the _her_ $\rightarrow$_his/him_ and _his_
$\rightarrow$_her/hers_ ambiguity, gender-neutral rewrites from the zero-shot
prompt are used. Similarly, the few-shot setting uses the corresponding
gender-neutral outputs from the few-shot prompt. The performance of both
settings is comparable.

####  B.3.3 Error Analysis

Figure 4 illustrates the distribution of aggregated errors across four
language pairs for GPT-3.5 Turbo in both zero-shot and few-shot settings,
specifically for the task of gender-neutral rewriting. The definitions of
these errors are provided in Table 8 in the appendix, while Table 9 offers
examples for each error label.

In both settings, the majority of errors stem from modifications unrelated to
gender-neutral rewriting and from instances where the model suggests no
changes are necessary to render the input text gender-neutral. Additional
examples of errors due to unrelated modifications can be found in Table 10 in
the appendix. The few-shot setting, however, does show an improvement in
neutral rewriting errors (such as POS(part-of-speech) errors and _them_ being
rewritten as _themselves_) when compared to the zero-shot setting.

Tables 11 presents the error distribution for each of the four languages. Upon
closer examination of the Finnish data, which has the highest error rate, we
found that the errors are primarily due to the longer input length. This
increases the scope for modifications of the text that are unrelated to
gender-neutral rewriting.

##  Appendix C Prompting Templates

###  C.1 GPT-3.5 Turbo Prompts

Figures 5 and 6 show the GPT-3.5 Turbo zero-shot and few-shot prompts used in
the gender-neutral rewriting task. We use the same prompt across all the
language pairs as the task is source agnostic.

Change all gendered pronouns to use singular "they" instead. Don’t modify
anything else : {input_text} Figure 5: Zero-shot prompt template utilized in
GPT-3.5 Turbo experiments. Change all gendered pronouns to use singular "they"
instead. Don’t modify anything else.
input : His bike is better than mine.
gender neutral variant : Their bike is better than mine.

input : Jack bores me with stories about her trip.
gender neutral variant: Jack bores me with stories about their trip.

input : He kissed him goodbye and left, never to be seen again.
gender neutral variant : They kissed them goodbye and left, never to be seen
again.

input : Is she your teacher?
gender neutral variant : Are they your teacher?

input : Anime director Satoshi Kon died of pancreatic cancer on August 24,
2010, shortly before her 47th birthday.
gender neutral variant : Anime director Satoshi Kon died of pancreatic cancer
on August 24, 2010, shortly before their 47th birthday.

input : {input_text}
gender neutral variant : Figure 6: Few-shot prompt template utilized in
GPT-3.5 Turbo experiments. I need help with a linguistic annotation task for a
translation. I will give you an Turkish sentence along with its translation
into English. I would like you to help me find Arbitrarily Gender-Marked
Entities (AGMEs), where someone is mentioned without any marked gender in the
Turkish sentence, but in the translation they have gender marking. Please
follow the following steps:

1\. Identify all unique individuals mentioned in the English translation in
the third person and find all words that explicitly indicate those
individuals’ genders.
\- Group words for each individual separately, considering possessive
determiners (e.g., "his", "her") as referring to a separate individual from
the one indicated by the noun they modify. For example, in "his uncle," "his"
and "uncle" refer to two separate individuals.
\- Pay attention to gender indicated by kinship terms and other gendered
nouns, like "mother", "nephew", "actress".
\- If the gender is explicitly indicated by pronouns in the target language,
consider that gender information for the analysis. (i.e. "she", "he", "him",
"her", "his", "hers", "himself", "herself" all explicitly indicate gender)
\- Treat names as if they do not indicate a gender, even if they are often
associated with a gender. For example, "Michael" could be either male or
female, so it does not mark gender.
\- Pay attention to how forms or "to be" (particularly "is") can join two
mentions of the same individual. For example, in "She is my daughter,"
"daughter" and "she" refer to the same person.

2\. Find all words in the Turkish source sentence that refer to each of the
individuals found in step one.

3\. For each individual, do any of the corresponding words in the Turkish
source explicitly indicate a gender.
\- Remember, pay attention to gender indicated by kinship words. For example,
words like "erkek", "kız" , "amca", "anne" all explicitly indicate gender.
\- Remember that some kinship words in Turkish are gender-neutral, such as
yeğen. Do not include these as marking gender.
\- Treat names as if they do not (e.g. ’Michael’ can refer equally well to a
man or woman).

4\. Identify any instances where the gender-neutral terms in Turkish have been
translated into gender-specific terms in English (AGMEs).
\- Answer separately for each individual identified.

5\. Next create a set of variant translations with the following notes:
\- If no changes are needed, then just use the original translation exactly as
it is.
\- Remember to only change the words referring to AGMEs.
\- if any gendered words refer to non-AGMEs, leave them untouched.
\- Do not make assumptions about heterosexual relationships. Men can have
husbands and boyfriends. Women can have wives and girlfriends.

Please create these three variant translations:
a. If any individuals are AGMEs and are referred to with gendered words in
English, rewrite the English translation changing only those words to use
their gender-neutral variants where possible. Use singular "they" instead of
he, she, etc. Use "themselves" for gender neutral singular reflexives (never
"themself"). Change nothing else.
b. rewrite the English translation so that any masculine words referring to
AGMEs are replaced by their feminine variants. Don’t change any words
referring to non-AGMEs. Change nothing else.
c. rewrite the English translation so that any feminine words referring to
AGMEs are replaced by their masculine variants. Don’t change any words
referring to non-AGMEs. Change nothing else.

Figure 7: Part I of Few-shot prompt template utilized in GPT-4 experiments.
Example 1 -
Source Sentence: Amcası kendi kendine konuşuyor.
Original Translation: His uncle talks to himself.
1\. individual 1: "His" is masculine. individual 2: "uncle", "himself" are
masculine.
2\. individual 1: no explicit words in the source indiviudal 2: "Amcası",
"kendi kendine"
3\. individual 1: no words indicate gender indivudual 2: "Amcası" is
masculine.
4\. individual 1: AGME - masculine in translation ("His"), but gender neutral
in the source (no explicit words) individual 2: not an AGME - gender is
masculine in both the source ("Amcası") and translation ("uncle")
5\. a. Their uncle talks to himself.
b. Her uncle talks to himself.
c. His uncle talks to himself.

Example 2 -
Source Sentence: Annem öğle yemeğini yalnız yiyordu.
Original Translation: My mother ate her lunch alone.
1\. individual 1: "mother", "her" are feminine.
2\. individual 1: "Annem"
3\. individual 1: "Annem" is feminine.
4\. individual 1: not an AGME since gender is feminine in both the source and
translation
5\. None

Example 3 -
Source Sentence: O benim kızım
Original Translation: She is my daughter.
1\. individual 1: "she", "daughter" are feminine.
2\. individual 1: "O", "kızım"
3\. individual 1: "kızım" is feminine.
4\. individual 1: not an AGME since gender is feminine in both the source and
translation
5\. None
Source Sentence: {source_sentence}
Original Translation: {original_translation}
Figure 8: Part II of Few-shot prompt template utilized in GPT-4 experiments.

###  C.2 GPT-4 Prompts

The full prompt for Turkish-English can be found in Figures 7 and 8. Prompts
for other languages use the same structure, but examples are customized to fit
those languages.

##  Appendix D Rewriting with GPT-4

Our solution uses chain-of-thought prompting (Wang et al., 2023) to elicit
GPT-4 to produce three variant translations for each input source-translation
pair – all-neutral, all-female and all-male, while leaving any gendered words
associated with non-AGMEs unmodified. We ask it to work step-by-step through
the process of identifying AGMEs before finally rewriting the original
translation:

  * •

Identify unique individuals mentioned in the target, as well as any gendered
words that refer to them.

  * •

Identify words in the source that refer to those same individuals.

  * •

Determine which source words mark for gender.

  * •

Designate any individuals referred to by gendered words in the target, but not
in the source as AGMEs

  * •

Produce a neutral, feminine and masculine variant translation where any
gendered words referring to AGMEs are modified to match the respective gender.

Each step in the prompt is accompanied by detailed clarifications and example
vocabulary. The prompt also includes three full examples, customized per
source language. The examples indicate that "None" should be returned in lieu
of translation variants when there are no AGMEs present. The full prompt for
Turkish-English is shown found in Figures 7 and 8.

##  Appendix E Mitigation Strategies Based on Source

Rather than rewriting English-target translations into feminine, masculine,
and neutral forms, one could use the source sentence as input to create these
three variants directly. This section explains how GATE X-E can be employed to
assess such a system.

The first step is to verify that the generated feminine, masculine, and
neutral variants are the same, except for changes related to gender. This is a
crucial step as it ensures that the meaning of the translation remains
consistent, regardless of the gender. If there are differences in the
translations beyond the gender-related changes, it could imply that the
translation is not accurate or is introducing additional bias. After this, the
generated output can be compared with the feminine, masculine, and neutral
references provided in GATE X-E using contextual MT evaluation metrics.

Kuczmarski (2018) initially explored a source-based debiasing approach in
which they enhanced a Neural Machine Translation (NMT) system to produce
gender-specific translations. This was achieved by adding an additional input
token at the beginning of the sentence to specify the required gender for
translation (e.g., _< 2FEMALE> O bir doktor $\rightarrow$ She is a doctor_).
However, they encountered challenges in generating masculine and feminine
translations that were exactly equivalent, with the exception of gender-
related changes. As a result, they later switched to a target-based rewriting
approach in their subsequent work (Johnson, 2020).

Original |  Gender-Neutral |  Gendered Alternatives
---|---|---
The teacher compared my poem with one of his. |  The teacher compared my poem with one of theirs. |  The teacher compared my poem with one of hers.
|  |  The teacher compared my poem with one of his.
Table 7: Examples illustrating the generation of gendered alternatives using gender-neutral rewrites  Error Category | Error Label |  Description
---|---|---
Corrections unrelated to neutral rewriting | Comma |  Comma added or removed.
Other corrections |  Error corrections that are not related to making the text gender-neutral.
Neutral Rewriting | POS |  Wrong form of ‘they’ produced by the rewriter.
SVA |  Failure to make correct subject-verb agreement.
Them $\rightarrow$ Themselves |  Second appearance of them changed to themselves.
’None’ response |  The rewriter returns ‘None’ suggesting no changes needed to make the input gender neutral.
Irrelevant modifications | Other modifications |  Modifications unrelated to neutral rewriting.
Table 8: Description of the error labels Error Label |  Input |  Output |  Reference
---|---|---|---
Comma |  Well, you surprised me!, Afshin said as she opened the door and saw Mary standing there. |  Well, you surprised me! Afshin said as they opened the door and saw Mary standing there. |  Well, you surprised me!, Afshin said as they opened the door and saw Mary standing there.
Other corrections |  I have never heard of him before that. |  I had never heard of them before that. |  I have never heard of them before that.
POS |  The secretary noted down what her boss had said. |  The secretary noted down what they boss had said. |  The secretary noted down what their boss had said.
SVA |  Does she come here every week? |  Does they come here every week? |  Do they come here every week?
Them $\rightarrow$ Themselves |  She saw her play baseball. |  They saw themselves play baseball. |  They saw them play baseball.
’None’ response |  He has no capacity to be a teacher. |  none |  They have no capacity to be a teacher.
Other modifications |  In any case, I will tell him about the critical tone your House has adopted on this issue. |  In any case, I will tell them about the critical tone their House has adopted on this issue. |  In any case, I will tell them about the critical tone your House has adopted on this issue.
Table 9: Examples for the error labels described in Table 8 Output |  Reference
---|---
They advised them to give up smoking, but they wouldn’t listen. |  They advised them to give up smoking, but they wouldn’t listen to them.
They were able to hold back their anger and avoid a fight. |  Jim was able to hold back their anger and avoid a fight.
The news that they had gotten injured was a shock to them. |  The news that they had got injured was a shock to them.
They have done it with their colleagues and the Committee of Legal Affairs. |  They have done it with the colleagues and the Committee of Legal Affairs.
In this respect , they have been very successful. |  In this respect I believe that they have been very successful.
They cannot be older than me. |  They cannot be older than I.
They suggested going to the theater, but there weren’t any performances that night. |  They suggested to go to the theater, but there weren’t any performances that night.
Table 10: More examples of errors of type ’Other Modifications’. Differences are in red. Category | Error Label | Zero-shot | Few-shot
---|---|---|---
|  | tu | hu | fi | fa | tu | hu | fi | fa
Corrections | Comma | 0 | 0 | 2 | 2 | 0 | 0 | 0 | 0
Other Corrections | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0
Neutral rewriting | POS | 2 | 1 | 9 | 4 | 0 | 0 | 0 | 1
SVA | 5 | 5 | 3 | 0 | 0 | 9 | 0 | 2
Them $\rightarrow$ Themselves | 0 | 10 | 10 | 0 | 0 | 4 | 0 | 0
’None’ response | 4 | 6 | 20 | 11 | 4 | 6 | 22 | 10
Irrelevant Modifications | Other modifications | 16 | 16 | 33 | 8 | 4 | 10 | 32 | 10
Total |  | 27 | 38 | 77 | 27 | 4 | 19 | 54 | 23
Table 11: Error analysis of GPT-3.5 Turbo’s zero-shot and few-shot performance
in English gender-neutral rewriting task.

Category | Feminine | Masculine | Neutral
---|---|---|---
Subject | She | He | They
Object | Her | Him | Them
Possessive Determiner | Her | His | Their
Possessive Pronoun | Hers | His | Theirs
Reflexive | Herself | Himself | Themselves
Table 12: Pronoun categories he | she | him
---|---|---
her | his | himself
herself | Ms | Mrs
Ms. | Mrs. | madam
woman | women | actress
actresses | airwoman | airwomen
aunts | aunt | uncle
uncles | brother | brothers
boyfriend | boyfriends | girlfriend
girlfriends | girl | girls
bride | brides | sister
sisters | businesswoman | businesswomen
chairwoman | chairwomen | chick
chicks | mom | moms
mommy | mommies | grandmother
daughter | daughters | mother
mothers | female | females
gal | gals | lady
ladies | granddaughter | granddaughters
grandmother | grandmothers | grandson
grandsons | grandfather | grandfathers
wife | wives | queen
queens | policewoman | policewomen
princess | princesses | spokeswoman
spokeswomen | stepson | stepdaughter
stepfather | stepmother | stepgrandmother
stepgrandfather |  |
Table 13: Gendered English Nouns and Pronouns


# VIC-KD: Variance-Invariance-Covariance Knowledge Distillation to Make
Keyword Spotting More Robust Against Adversarial Attacks

###### Abstract

Keyword spotting (KWS) refers to the task of identifying a set of predefined
words in audio streams. With the advances seen recently with deep neural
networks, it has become a popular technology to activate and control small
devices, such as voice assistants. Relying on such models for edge devices,
however, can be challenging due to hardware constraints. Moreover, as
adversarial attacks have increased against voice-based technologies,
developing solutions robust to such attacks has become crucial. In this work,
we propose VIC-KD, a robust distillation recipe for model compression and
adversarial robustness. Using self-supervised speech representations, we show
that imposing geometric priors to the latent representations of both Teacher
and Student models leads to more robust target models. Experiments on the
Google Speech Commands datasets show that the proposed methodology improves
upon current state-of-the-art robust distillation methods, such as ARD and
RSLAD, by 12% and 8% in robust accuracy, respectively.

Index Terms—  Keyword Spotting, Adversarial Robustness, Knowledge
Distillation, Robust Distillation, VICReg.

##  1 Introduction

Fig. 1: Diagram of our proposed method VIC-KD for robust distillation.

Recent advances in the deep learning field have profoundly impacted the
speech-processing community. As such, it has resulted in cutting-edge systems
for tasks such as keyword spotting (KWS) [1] and speech/emotion recognition
[2, 3], to name a few. In particular, KWS systems are usually the first layer
of virtual assistants, where such models attempt to identify preset words in
an utterance to activate the device (e.g., “Hey Siri”). On-device speech
processing, such as Google Now and Alexa, is becoming ubiquitous in our daily
lives; however, this can impose challenges on energy efficiency, real-time
processing, and on user privacy.

For KWS, the use of self-supervised speech representation learning (S3RL) has
become a popular tool. S3RL models, such as _Wav2Vec 2.0_ [4] and WavLM [5],
are designed to learn combined acoustic and language models for continuous
audio inputs. Notwithstanding, when working with edge devices, it is crucial
to take into account model size due to hardware constraints. To this end,
knowledge distillation (KD) [6] has emerged as a powerful tool to transfer
knowledge from a larger (Teacher) model to a smaller (Student) one, resulting
in comparable generalization capabilities.

Moreover, it is known that edge processing can provide an extra layer of
security protection for the user, ensuring that private data is processed
locally on the device and not sent over the cloud to a third-party server.
However, recent research has shown that KWS systems based on self-supervised
representations can be vulnerable to so-called adversarial attacks during
inference time [7]. Adversarial attacks aim to design a small perturbation
$\delta$ that when added to the test signal will force the system to fail.
Over-the-air adversarial attacks, for example, have shown that imperceptible
noise can be added to the user’s voice resulting in misclassifications [8].

Adversarial training (AT) [9] has emerged as a potential defense technique
against adversarial attacks. AT is a robust optimization formulation that, in
practice, can be seen as a data augmentation technique that generates
adversarial versions of a natural sample to be used during training of Teacher
models. The TRADES (TRadeoff-inspired Adversarial DEfense via Surrogate-loss
minimization) method, for example, has generated inspiring results [10].
Efficiently designing small (Student) models, however, is a challenging task.
The Adversarial Robust Distillation (ARD) [11] method combines adversarial
training with KD, emphasizing the importance of minimizing the Kullback-
Leibler divergence between Student and Teacher logits to enhance Student
robustness. In contrast, the Robust Soft Label Adversarial Distillation
(RSLAD) [12] method uses soft labels from the Teacher model (in lieu of hard
labels) for guidance during distillation, yielding better results than ARD
empirically. Notwithstanding, neither of these methods can consistently
surpass the accuracy of a smaller model trained with AT, thus suggesting that
further innovations are needed if large self-supervised speech representations
are to be used in edge devices.

In this work, we propose VIC-KD, a novel distillation recipe that compresses
the S3RL model size while increasing its robustness against adversarial
attacks. In particular, we show that multi-view inputs and geometric
constraints on the latent space of the Student model are essential to achieve
these advantages. We consider Student models with fewer than 96K parameters
and 3 MMACs, and knowledge distillation is performed from fine-tuned versions
of _Wav2Vec 2.0_ and WavLM. Results show that our distillation recipe can
achieve better robustness than traditional defense techniques, such as TRADES,
and improve upon robust distillation methods, such as ARD and RSLAD.

##  2 Variance-Invariance-Covariance Knowledge Distillation

Table 1: Baseline results obtained directly in a supervised setting without
knowledge distillation.

|  |  | Natural Training | Robust Training
---|---|---|---|---
|  |  | GSC v12 | GSC v35 | GSC v12 | GSC v35
Model | #Params | MMACs | Clean | AutoAttack | Clean | AutoAttack | Clean | AutoAttack | Clean | AutoAttack
_Wav2Vec 2.0_ | 94.4M | 6160.9 | 99.07 | 9.40 | 97.53 | 2.68 | 97.58 | 93.80 | 95.77 | 90.63
WavLM | 94.4M | 6132.7 | 98.94 | 6.48 | 97.42 | 0.64 | 97.95 | 94.79 | 97.20 | 93.26
XVector | 80.4K | 2.9 | 96.43 | 3.09 | 94.54 | 1.36 | 94.70 | 80.33 | 93.56 | 78.27
TC-ResNet | 67.7K | 1.6 | 94.95 | 2.67 | 93.53 | 0.09 | 93.69 | 79.25 | 92.95 | 77.30

Herein, instead of using the Teacher model to guide how the Student logits
should behave, we induce some geometric properties of the latent space of the
Student via the variance-invariance-covariance regularization and the usage of
multi-view inputs to each model. Although the usage of multi-view inputs for
KD has already been explored in the literature to increase the environmental
robustness of Student models [13, 14], its effect on robust distillation still
needs to be determined. Figure 1 depicts a block diagram of the proposed VIC-
KD distillation recipe.

In the self-supervised learning literature, joint embedding architectures
(JEA) are becoming popular due to their effectiveness in learning latent
factors from data [15, 16]. VICReg [17] is one such method that is based on
preserving the information of the embeddings while avoiding representational
collapse. Here, we expand those ideas to a robust distillation method. First,
we sample an utterance from $x\sim\mathcal{D}_{\text{train}}$ and two random
transformations $\\{t,t^{\prime}\sim\mathcal{T}|t\neq t^{\prime}\\}$. As
depicted in the upper branch of Figure 1, the Teacher model $T_{\theta}$ is
responsible for extracting representations from the speech input. More
precisely, representation $Z$ is extracted, which is the weighted sum of the
intermediate representations from all Transformer layers, and aggregated over
the time dimension.

The Student branch, on the other hand, receives the other utterance view
generated by $t^{\prime}$ with an added adversary perturbation. First, for
both inputs, we extract the latent representations of the Student model,
denoted by $H^{\prime}$. This representation is then fed to a classification
head, generating the model logits $Y^{\prime}$ that attempt to predict the
spoken commands. In parallel, $H^{\prime}$ is fed to a projection head
responsible for generating $Z^{\prime}$, which has the same dimensionality as
the Teacher latent representation. Note that, at test time, only the Student
encoder $S_{\theta^{\prime}}$ and the classification head are used.

In addition to the mechanism described above, we now describe the contribution
of each term that composes the three-factor VIC-KD loss function, presented in
equation 1: (1) variance, (2) invariance, and (3) covariance terms. Following
our previous notation, we define $Z=[z_{1},z_{2},...,z_{n}]$ and
$Z^{\prime}=[z_{1}^{\prime},z_{2}^{\prime},...,z_{n}^{\prime}]$ as the
$d$-dimensional latent representations generated by the Teacher and Student
models for natural and adversarially perturbed input, respectively, for a
batch of $n$ utterances. First, the invariance term consists of the mean
squared error between the two latent representations, $Z$ and $Z^{\prime}$.
Next, the variance term induces the learned latent embedding to vary within
the other batch elements, thus avoiding collapse on the same vector. Lastly,
the covariance term prevents the Student model, which already has limited
capacity, from encoding similar features. In our experiments, we consider an
equal contribution of each term in the final VICReg loss. The interested
reader is referred to [17] for more details about the loss implementation.

$\displaystyle\mathcal{L}_{\text{VICReg}}$
$\displaystyle=\text{Var}(Z^{\prime})+\text{Inv}(Z,Z^{\prime})+\text{Cov}(Z^{\prime})$
(1)

Finally, the VIC-KD loss is computed as the convex combination between the
TRADES [10] and VICReg losses, controlled through a hyperparameter $\alpha$ as
described in Figure 1.

##  3 Experimental Setup

To train the KWS system, we use the _Google Speech Commands v0.02_ (GSC)
dataset [18]. This dataset has about 100k 1-second utterances spread across 35
commands and sampled at 16 kHz. We consider two versions of the dataset, one
with all commands and another with only 12 classes that include labels such as
{yes, no, up, down, left, right, on, off, stop, go, unknown, and silence}. The
_silence_ class is made up of background noises with no speech, while the
_unknown_ label includes utterances uniformly sampled from unused classes. We
follow the SpeechBrain [19] recipe, where the dataset is split into train,
validation, and test sets in the ratio of 80%, 10%, and 10%, respectively.

In this study, we examine two Teacher models, _Wav2Vec 2.0_ and WavLM, along
with two Student models, TC-ResNet-8 [20] and a custom XVector [21], with a
reduced number of TDNN layers and smaller kernel sizes to fit our resource
constraints. Furthermore, as summarized in Table 1, we show the results with
conventional training, as well as AT via TRADES. Moreover, we report accuracy
for clean speech files, as well as speech files corrupted by the AutoAttack
method [22]. Experiments herein rely on these reference values for comparison.

Baseline models are trained for 100 epochs, with a batch size of 32 samples,
using an Adam optimizer, and learning of $10^{-3}$ that linearly decays to
$10^{-4}$. Conversely, we fine-tune the Teacher model for the KWS task for 10
epochs, and the learning rate is scheduled from $5\times 10^{-4}$ to $5\times
10^{-5}$. The Teacher model consists of the S3RL encoder with a weighted sum
and aggregation mechanism described previously and a simple linear layer that
classifies the input speech signal into the desired number of classes. Note
that, for VIC-KD, we can discard the Teacher’s linear layer since we are
interested in the latent representation. For adversarial attacks, we rely on
AutoAttack [22], an ensemble method comprised of APGD [22], APGD-T [22], and
FAB [23] attacks. We consider $\ell_{\infty}$ bounded attacks with an
$\epsilon=1.5\times 10^{-3}$.

Three distillation methods, namely KD, ARD, and RSLAD, are used to benchmark
the proposed technique. Training is performed for 250 epochs with parameters
matching the small baseline models, as specified above. VIC-KD relies on a
10-step PGD-like attack, also with an $\epsilon=1.5\times 10^{-3}$ and a step
size of $3\times 10^{-4}$, to generate the adversarial perturbation for
training. Additionally, multi-view inputs are used for Student and Teacher
models, incorporating clean data, noise, reverberation, noise-plus-
reverberation, wave chunk dropping, and speed perturbation into the
transformations $\mathcal{T}$.

##  4 Experimental Results and Discussion

###  4.1 Classification accuracy

Table 2: Results on the GSC dataset with 12 classes for distillation methods.
Distilling from standard and robust Teachers.

Teacher / Student | Standard Teacher | Robust Teacher
---|---|---
Clean | AutoAttack | Clean | AutoAttack
KD [6]
_Wav2Vec 2.0_ / TC-ResNet | 95.80 | 5.54 | 95.61 | 4.49
_Wav2Vec 2.0_ / XVector | 96.77 | 7.59 | 96.68 | 6.84
WavLM / TC-ResNet | 96.20 | 9.29 | 95.70 | 5.11
WavLM / XVector | 96.91 | 8.00 | 96.62 | 7.19
ARD [11]
_Wav2Vec 2.0_ / TC-ResNet | 94.54 | 76.41 (-2.8%) | 94.92 | 74.16 (-5.1%)
_Wav2Vec 2.0_ / XVector | 95.37 | 79.74 (-0.6%) | 95.71 | 79.94 (-0.4%)
WavLM / TC-ResNet | 95.17 | 74.13 (-5.1%) | 95.30 | 76.87 (-2.4%)
WavLM / XVector | 95.27 | 78.33 (-2.0%) | 95.29 | 74.03 (-6.3%)
RSLAD [12]
_Wav2Vec 2.0_ / TC-ResNet | 94.65 | 78.06 (-1.2%) | 94.59 | 76.27 (-3.0%)
_Wav2Vec 2.0_ / XVector | 95.37 | 80.79 (+0.5%) | 95.45 | 81.98 (+1.7%)
WavLM / TC-ResNet | 95.11 | 77.17 (-2.1%) | 94.11 | 77.99 (-1.3%)
WavLM / XVector | 95.34 | 81.30 (+1.0%) | 95.69 | 81.90 (+1.6%)
VIC-KD (Ours)
_Wav2Vec 2.0_ / TC-ResNet | 95.31 | 83.75 (+4.5%) | 95.48 | 83.38 (+4.1%)
_Wav2Vec 2.0_ / XVector | 96.50 | 86.12 (+5.8%) | 96.29 | 85.33 (+5.0%)
WavLM / TC-ResNet | 95.39 | 83.30 (+4.1%) | 95.37 | 83.73 (+4.5%)
WavLM / XVector | 95.92 | 86.08 (+5.8%) | 96.39 | 86.31 (+6.0%)

Table 2 presents the main experimental results exploring two scenarios:
standard vs. robust Teachers as guides. We report clean and robust accuracies
for each robust distillation method, with differences relative to the
respective baseline in parentheses. In fact, given the low robust accuracy and
that KD is not a robust method, we do not compute its delta against the robust
baseline. KD with standard Teacher outperforms baselines and other robust
methods in clean accuracy. However, despite some improvement, the robust
accuracy is still below random guess. ARD, on the other hand, significantly
enhances overall robust accuracy, but still falls short of the baseline
performance under the robust condition.

Regardless, we observe an improvement in the overall scenario for RSLAD and
VIC-KD, which are built upon TRADES. For the Student model with bigger
capacity, e.g., the XVector, we observe the first results where the model can
improve upon the baseline for robust accuracy. The TC-ResNet on the RSLAD
distillation has inline results with the baseline, but improves upon the ARD
method. On the other hand, VIC-KD for both TC-ResNet and XVector can
substantially improve upon both robust distillation recipes and the baselines
for the clean and under-attack scenarios, thus showing the benefits of the
proposed methodology. In fact, VIC-KD with the _Wav2Vec 2.0_ / XVector pair
outperforms the baseline by a relative percentual difference of 7.2% on robust
accuracy. Similarly, the proposed model outperforms ARD and RSLAD on attacked
scenarios by 8.0% and 6.6%, respectively. On the downside, the VIC-KD recipe
training time is four times slower than RSLAD. Note that, at the inference
stage, the time is the same for all distillation recipes since it depends
solely on the architecture of the student model.

Finally, some conclusions can be drawn from the distillation of robust
Teachers. First, from the KD experiment, it is crucial to notice that the
Student model does not necessarily inherit robustness from the Teacher; hence,
specific techniques are needed if the goal is to design adversarially robust
models. However, a tradeoff between clean and robust accuracy needs to be
decided. For the robust distillation methods, we observe that using robust
Teachers, in general, does not improve the final performance of the Student
model; thus, for our specific use case on the GSC dataset, unless one already
has off-the-shelf robust Teacher models, it is not worth the extra computation
of transforming a standard Teacher into a robust one first and then perform
the robust distillation.

###  4.2 The effects of multi-view inputs

Next, we investigate the effect of multi-view (MV) inputs on robust
distillation, as shown in Fig.2. Here, we investigate the WavLM/TC-ResNet pair
as our Teacher and Student models, respectively. For comparison, a red dashed
line shows the baseline robust accuracy of TC-ResNet trained with TRADES
without any guidance from Teacher models. As observed in the figure, all
distillation methods benefit from multi-view. For instance, ARD and RSLAD,
which had a robust accuracy below the baseline, can surpass this landmark
after MV. Our proposed VIC-KD method can outperform the baseline with or
without MV. However, MV helps us to achieve the best overall robust accuracy.

Fig. 2: Effects of the Multi-View on several distillation Methods for the
WavLM / TC-ResNet pair. The dashed line represents the robust accuracy of TC-
ResNet via TRADES.

We hypothesize that the reason behind the performance gain by using MV is two-
fold. First, MV induces the student model to learn perturbation-invariant
features from speech signals and to better disentangle noisy factors from
speech features, thus improving generalization. Lastly, other works already
have discussed that self-supervised learning methods that employ MV are
forcing the model to maximize the mutual information of representations
between the two views [17]. We suppose a similar conclusion can be drawn for
the KD case, but more studies are still needed.

###  4.3 Expanding the distillation to more classes

Fig. 3: Scaling robust distillation to a 35-class GSC dataset and comparing it
to clean and robust accuracy baselines.

Lastly, we further stress the distillation recipes to a more significant
number of output classes by using all 35 classes from the GSC dataset. In Fig.
3, we show the accuracy results for both clean and AutoAttack-based methods
using the WavLM / XVector models as Teacher and Student respectively. The blue
and green dashed lines denote the clean and robust accuracy of the supervised
baseline, XVector, trained via TRADES. Similarly to previous findings, KD can
surpass the clean accuracy of the baseline, but the robust accuracy is not
satisfactory. Among the robust distillation methods, VIC-KD exhibits the
overall best performance by achieving the best accuracy and showing the
smallest gap between clean and robust accuracies.

##  5 Conclusions

Here, we propose VIC-KD, a methodology to improve the adversarial robustness
of distilled models. Our recipe is two-fold: (1) use multi-view inputs to
induce the Student to learn perturbation-invariant features and (2) apply the
variance-invariance-covariance regularization (VICReg) to the latent
representation of the Teacher/Student model. Experiments on the Google Speech
Command dataset, with 12 and 35 classes, show the proposed methodology
outperforming state-of-the-art robust distillation recipes on
$\ell_{\infty}$-bounded ensemble of attack (AutoAttack). Overall, VIC-KD
canbetter balance the tradeoff between clean and robust accuracy, making this
technique a strong candidate for developing and deploying trustworthy speech
applications on the edge.

## References

  * [1] G. Chen, C. Parada, and G. Heigold,  “Small-footprint keyword spotting using deep neural networks,”  in 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP), 2014, pp. 4087–4091.
  * [2] D. O’Shaughnessy,  “Trends and developments in automatic speech recognition research,”  Computer Speech & Language, vol. 83, pp. 101538, 2023.
  * [3] A. R. Avila, J. Monteiro, D. O’Shaughneussy, and T. H. Falk,  “Speech emotion recognition on mobile devices based on modulation spectral feature pooling and deep neural networks,”  in 2017 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), 2017, pp. 360–365.
  * [4] A. Baevski, Y. Zhou, A. Mohamed, et al.,  “wav2vec 2.0: A framework for self-supervised learning of speech representations,”  Advances in Neural Information Processing Systems, vol. 33, pp. 12449–12460, 2020.
  * [5] S. Chen, C. Wang, Z. Chen, et al.,  “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,”  IEEE Journal of Selected Topics in Signal Processing, 2022.
  * [6] G. Hinton, O. Vinyals, and J. Dean,  “Distilling the knowledge in a neural network,”  in NIPS Deep Learning and Representation Learning Workshop, 2015\.
  * [7] H. R. Guimarães, Y. Zhu, O. Mengara, et al.,  “Assessing the vulnerability of self-supervised speech representations for keyword spotting under white-box adversarial attacks,”  in 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2023.
  * [8] C. Zhao, Z. Li, H. Ding, et al.,  “Utio: Universal, targeted, imperceptible and over-the-air audio adversarial example,”  in 2022 IEEE 28th International Conference on Parallel and Distributed Systems, 2023, pp. 346–353.
  * [9] A. Madry, A. Makelov, L. Schmidt, et al.,  “Towards deep learning models resistant to adversarial attacks,”  in International Conference on Learning Representations, 2018.
  * [10] H. Zhang, Y. Yu, J. Jiao, E. Xing, et al.,  “Theoretically principled trade-off between robustness and accuracy,”  in International conference on machine learning. PMLR, 2019, pp. 7472–7482.
  * [11] M. Goldblum, L. Fowl, S. Feizi, et al.,  “Adversarially robust distillation,”  in Proc. of the AAAI Conference on Artificial Intelligence, 2020, vol. 34, pp. 3996–4003.
  * [12] B. Zi, S. Zhao, X. Ma, et al.,  “Revisiting adversarial robustness distillation: Robust soft labels make student better,”  in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 16443–16452.
  * [13] K.P. Huang, Y.K. Fu, T.Y. Hsu, et al.,  “Improving generalizability of distilled self-supervised speech processing models under distorted settings,”  in 2022 IEEE Spoken Language Technology Workshop, 2023, pp. 1112–1119.
  * [14] H. R. Guimarães, A. Pimentel, A. R. Avila, et al.,  “Robustdistiller: Compressing universal speech representations for enhanced environment robustness,”  in 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5.
  * [15] J.B. Grill, F. Strub, F. Altché, et al.,  “Bootstrap your own latent-a new approach to self-supervised learning,”  Advances in neural information processing systems, vol. 33, pp. 21271–21284, 2020.
  * [16] J. Zbontar, Li Jing, I. Misra, et al.,  “Barlow twins: Self-supervised learning via redundancy reduction,”  in International Conference on Machine Learning. PMLR, 2021, pp. 12310–12320.
  * [17] A. Bardes, J. Ponce, and Y. LeCun,  “VICReg: Variance-invariance-covariance regularization for self-supervised learning,”  in International Conference on Learning Representations, 2022.
  * [18] P. Warden,  “Speech commands: A dataset for limited-vocabulary speech recognition,”  arXiv preprint arXiv:1804.03209, 2018.
  * [19] M. Ravanelli, T. Parcollet, P. Plantinga, et al.,  “SpeechBrain: A general-purpose speech toolkit,” 2021,  arXiv:2106.04624.
  * [20] S. Choi, S. Seo, B. Shin, et al.,  “Temporal Convolution for Real-Time Keyword Spotting on Mobile Devices,”  in Proc. Interspeech 2019, 2019, pp. 3372–3376.
  * [21] D. Snyder, D. Garcia-Romero, G. Sell, et al.,  “X-vectors: Robust dnn embeddings for speaker recognition,”  in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 5329–5333.
  * [22] F. Croce and M. Hein,  “Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks,”  in International conference on machine learning. PMLR, 2020, pp. 2206–2216.
  * [23] F. Croce and M. Hein,  “Minimally distorted adversarial examples with a fast adaptive boundary attack,”  in International Conference on Machine Learning. PMLR, 2020, pp. 2196–2205.