# Gender Bias in Machine Translation and The Era of Large Language Models

Eva Vanmassenhove

###### Abstract

This chapter examines the role of Machine Translation in perpetuating gender
bias, highlighting the challenges posed by cross-linguistic settings and
statistical dependencies. A comprehensive overview of relevant existing work
related to gender bias in both conventional Neural Machine Translation
approaches and Generative Pretrained Transformer models employed as Machine
Translation systems is provided. Through an experiment using ChatGPT (based on
GPT-3.5) in an English-Italian translation context, we further assess
ChatGPT’s current capacity to address gender bias. The findings emphasize the
ongoing need for advancements in mitigating bias in Machine Translation
systems and underscore the importance of fostering fairness and inclusivity in
language technologies.

Keywords: Neural Machine Translation, Bias, Gender, Translation Technology,
Large Language Models

##  1 Introduction

This chapter investigates the role of Machine Translation (MT) in perpetuating
and shaping societal biases, with a specific emphasis on gender bias. Within
the realm of MT, biases, specifically gender biases, are particularly
prominent due to contrastive linguistic settings that necessitate
disambiguation and explicitness in their representation of gender. The
(over)reliance of MT systems on statistical patterns, can lead to erroneous
disambiguation and thus the generation of morphologically incorrect
alternatives (Vanmassenhove et al., 2021). Moreover disambiguation based
solely on statistical likelihood combined with the fact that current
(traditional) MT systems exhibit limited capability in providing multiple
potential translation candidates reinforces or even exacerbates (potentially
harmful) stereotypes (Cho et al., 2019; Rescigno et al., 2020; Prates et al.,
2020). We advocate for more accountability in the design, development, and
deployment of MT systems to address these biases and limitations.

To explore the potential for addressing gender bias in an English-Italian
translation setting, we investigate the extent to which ChatGPT (based on
GPT-3.5), a powerful Large Language Model (LLM), handles gender across these
languages and whether it can contribute and help with bias mitigation.
Specifically, we examine whether ChatGPT (GPT-3.5) is capable of generating
translations that effectively address gender bias when explicitly prompted to
do so. The translations generated by the model, despite specific prompts
regarding gender, seem to indicate that there are limitations to the model’s
ability to provide a systematic solution for gender bias in translation. While
the overall observations aligned with our expectations and the related work,
it was surprising that explicitly prompting for gender alternatives often led
to additional biases and inaccurate translations rather than more balanced,
fair and/or less-biased responses. These findings underscore the need for
further exploration and development in this area. In light of the limitations
and potential biases of current MT systems, we advocate for continued research
and improvements to promote fair and unbiased translations. Given the
multidisciplinary nature of the fields, we advocate for more hybridization
both in terms of the technology by, for instance, integrating linguistic
information into neural models as in the field itself through more
collaboration among (computational) linguists, computer scientists,
sociolinguists, ethicists and others.

The remainder of this chapter is organized as follows: Section 2 provides an
overview of gender bias in MT and highlights the linguistic settings and
statistical dependencies that contribute to its manifestation. Section 3
presents related work and discusses existing efforts to address gender bias in
MT systems. Section 4 outlines the experiment, including the specific prompts
and the analysis. Finally, Section 5 concludes the article by summarizing our
findings and their implications, emphasizing the need for further research and
advancements in addressing gender bias in MT systems, and advocating for
increased accountability in the development and deployment of these
technologies. By conducting a systematic analysis of gender bias in MT and
evaluating the potential of LLMs in mitigating such bias, we hope to
contribute to the ongoing efforts in creating fair, unbiased, and inclusive
language technologies.

##  2 Gender Bias in MT

The advent of Neural Machine Translation (NMT) played a pivotal role in
breaking down language barriers and facilitating knowledge exchange among
diverse (linguistic) communities. With their success and adoption, it also
became increasingly important to investigate and address the inherent
limitations and shortcomings these systems possess. First, current systems,
have a tendency to not only perpetuate but also exacerbate biases and lack
control in order to handle contrastive linguistic challenges, such as gender,
in a systematic manner (Vanmassenhove et al., 2019, 2021). Second, these
systems are unable to adhere and abide by language-specific guidelines or
policies related to gender-inclusive or gender-neutral language. As the
influence of MT systems grows, it becomes increasingly important to scrutinize
and analyse the biases embedded within MT systems and their potential impact
on societal inequalities.

One prevalent form of bias, which warrants particular attention in translation
contexts, is gender bias. Within the context of MT, we define gender bias as
an (explicit or implicit) skew across or within language(s) towards gender(s),
leading to systematic associations with gender(s) and thus creating
representational (and in some cases allocational) harms that perpetuate
unfair, inaccurate and potentially discriminatory stereotypes.

Gender biases in MT are often exacerbated by the need for explicit
representation and disambiguation of gender in contrastive linguistic
settings. For example, the English sentence ‘my husband is a kindergarten
teacher [neutral]’ contains a profession noun ‘kindergarten teacher’ that is
not explicitly marked for gender. However, within this sentences, it is clear
that it refers to my husband, a masculine nouns with which it semantically
agrees. The sentence is translated by Google Translate11endnote: 1Accessed on
June 12, 2023. as ‘mijn man is kleuterjuf [feminine]’ in Dutch, where ‘juf’ is
a gender-marked noun that can be translated into English as ‘miss’. In many
other cases, the MT systems do not produce translation errors but consistently
opt for gender stereotypical translations such as the French ‘tu es belle
[feminine]’ as a translation for the English gender-neutral ‘you are beautiful
[neutral]’ (where the English sentence is besides gender, also ambiguous in
terms of number and degree of politeness as opposed to French) and ‘tu es
intelligent [masculine]’ as a translation for the English sentence ‘you are
smart [neutral]’. Another illustration, based on a tweet by Dora Vargha
(2021)22endnote: 2Re-translated on June 12 2023 by the author., is presented
in Table 1 with Hungarian as the source language and English as the target.
Hungarian has a gender-neutral pronoun ö which can be translated into English
as either he, she or gender-neutral they. These translations were generated by
Google Translate.

Hungarian | English
---|---
Ö szép. |  She is beautiful.
Ö okos. |  He is clever.
Ö olvas. |  He reads.
Ö varr. |  She sews.
Ö tanít. |  He teaches.
Ö főz. |  She cooks.
Ö kutat. |  He is researching.
Ö gyereket nevel. |  She is raising a child.
Ö sok pénzt keres. |  He makes a lot of money.
Ö süteményt süt. |  She bakes cakes.
Table 1: Illustration Gender Bias in Hungarian-English translations provided
by Google Translate

While none of the generated translations in Table 1 are wrong from a
linguistic point of view, they do exhibit a fair amount of bias and
stereotyping when it comes to opting for the feminine or masculine pronoun in
the automatically generated English translations. In a systematic examination
of this gender bias phenomenon, Prates et al. (2020) translated sentences
containing professions from 12 different gender-neutral language (including
Hungarian) into English and examined how often female, male or gender-neutral
pronouns appeared in the translations. Their analysis not only revealed a
significant imbalance in the distribution of pronouns but also highlighted
that this imbalance exceeds the gender disparities observed in society itself
for fields such as STEM (Science, Technology, Engineering and Mathematics).
Their comparison with results from the U.S. Bureau of Labor Statistics
revealed that Google Translate underestimated the expected frequency of female
translations, demonstrating an even higher prevalence of male defaults
compared to what job occupation data alone would suggest.

###  2.1 Gender Bias Sources

Different languages handle and mark gender in different ways, resulting in
various types of potential ambiguities that are hard to resolve for MT
systems, making a one-solution-fits-all very unlikely. The requirement for
explicitation and disambiguation in combination with the fact that languages
differ in terms of the absence or presence of gender features, the number of
gender classes and whether and where gender features are explicitly marked
sets gender aside form other biases in contrastive linguistic settings. By
this, it not implied that other biases do not occur, however, they do manifest
themselves in different, more implicit ways.33endnote: 3Although one could
argue, that this implicitness makes it all the more dangerous.

When state-of-the-art NMT systems are faced with (gender) ambiguous words in
cross-linguistic settings, they will generate the most statistically probable
translation derived from the training data. This would imply that, one of the
main sources of bias in the output are the datasets used as an input, referred
to as dataset bias. We want to emphasize that reducing the problem solely to a
matter of data bias is an oversimplification since such a reduction neglects
the intricate web of factors underlying these biases, including historical
legacies and societal injustices (Birhane, 2021). However, given that an in-
depth discussion of historical and societal injustices is out of the scope of
this chapter, we will further zoom in on data bias where we distinguish
between coverage and social bias.44endnote: 4For a more comprehensive
exploration of this topic, we refer to Birhane (2021) which provides valuable
insights into the ethical dimensions of algorithmic systems and their impact
while advocating for a more critical analysis in the field of machine learning
and data science. Additionally, we will briefly discuss statistical and
algorithmic bias.

Coverage bias in the context of gender and MT, refers to the
underrepresentation and/or the misrepresentation of gender identities and
gender-related terminology in the data used for training. For instance, an
analysis of Europarl (Koehn, 2005), one of the more popular training datasets
for MT systems, revealed a 2:1 male-female speaker ratio. Such an imbalanced
dataset in terms of male/female representation, most likely implies a higher
frequency of male pronouns, male-endings (nouns, adjectives…), increasing the
likelihood of generating "male" translations when facing ambiguity. This can
be interpreted in the broad way, since there is also evidence that word
embeddings are conforming to an overall androcentric worldview (Petreski and
Hashim, 2023).

Taking the example below, a few possible translations for the English word
‘teacher’ [neutral] are provided in French: ‘enseignant’ [masculine singular],
‘enseignante’ [feminine], ‘enseignant-e’ [binary alternative] or ‘le personnel
enseignant’ [gender-neutral alternative].55endnote: 5We acknowledge that the
gender-neutral alternative ‘le personnel enseignant’ is not the most literal
translation of ‘teacher’ and furthermore implies multiple people (plural).
However, in settings where ‘teacher’ is used in a generic way, it can be
considered a valid gender-neutral alternative.

teacherenseignantenseignantele personnel enseignantenseignant-e Figure 1: Some
French translations of the English noun ‘teacher’.

An imbalanced dataset could affect the probability of these translations being
generated by an MT systems and affect the ability fo handle all possibilities.
If the dataset contains a higher proportion of male speakers or masculine
forms as opposed to feminine or gender-neutral ones, the MT system will become
biased towards producing more masculine translations in case of ambiguity. It
would thus consider ‘enseignant’ the more probable translation as opposed to
the other valid options (given that there is no additional context to
disambiguate). Additionally, a limited or no exposure to forms such as the
binary gender inclusive ‘enseignant-e’ or the gender-neutral ‘le personnel
enseignant’ in the training data can lead to erroneous or suboptimal
translations that are less fluent, accurate and lack naturalness. While
balancing training data in terms of male and female genders can be a somewhat
viable option in many cases to alleviate these (binary) biases, more and more
languages are accommodating for non-binary gender66endnote: 6We will use the
term non-binary gender to refer to all genders that fall outside the binary
female/male distinction. employing different strategies (for instance, gender-
neutral pronouns like they in English, gender-neutral endings like x in
Spanish…), a trend that will need to be reflected in MT technology as well.
Currently this is not the case, among others, due to a lack of readily
available gender-neutral/inclusive data, posing additional challenges for
these data-hungry models. Rule-based or hybrid approaches that leverage
linguistic knowledge in order to automatically generate gender-neutral
alternatives in a post-processing way could be further explored to address the
data scarcity issue, especially given the rather systematic nature of gender
agreement rules (Vanmassenhove et al., 2021).

Social bias in MT, refers to the bias that arises due to societal stereotypes
and norms surrounding gender roles, which are embedded in and (un)consciously
transmitted via the language in the training data. Such biases may influence
the system to generate translations that align with traditional gender
stereotypes or assumptions. Even more so because of the use of word embeddings
which are used to capture semantic relationships and associations between
words based on patterns present in the data. These embeddings can
inadvertently reinforce existing biases as they learn from the prevailing
language usage, including gendered language and stereotypes and embed co-
occurence patterns in the representations of the word themselves. A recent
study by Caliskan et al. (2022) investigating the extent of gender bias in
static word embeddings revealed that the most common words and concepts are
strongly associated with men more so than with women. Additionally, their
findings highlighted how various gender biases are still prevalent across
multiple dimensions as they observed differences in terms of the part-of-
speech (e.g. women are more often associated with adjectives), clusters of
concepts (e.g. women are more associated with sexual content and men with big
tech), and basic dimensions of the meaning of words (e.g. women are more
associated with positive valence and men with dominance).

While data bias is a well-acknowledged phenomenon, from a technical point of
view, also statistical and algorithmic bias introduced by the translation
models are important dimensions of bias in MT since they can introduce,
perpetuate and even exacerbate biases observed in the training data. In the
context of MT, Vanmassenhove et al. (2019) investigated the effect of
statistical biases on generated translations. A statistically biased MT system
may exhibit preferences for frequently occurring words or sub-words,
potentially overlooking less common but equally valid synonyms or
morphological variants. This bias leads to a loss of morphological variety and
can impact the system’s ability to generate diverse and grammatically correct
translations. Toral (2019) reached similar conclusions focusing on automatic
post-editing, demonstrating its simpler and more normalized nature compared to
human post-edited or translated texts. Both these observations can, in turn,
be linked directly to gender bias in translations. Looking back at Figure 1,
it is likely that some of the potential translations are way more common in
the data than others. The model would strengthen the more frequently occurring
connections and weaken those that are more rarely observed in the data. In a
follow-up paper, Vanmassenhove et al. (2021) conducted experiments where they
train a model with which they then translate the original training data. They
give an example where the most commonly occurring form of the French lemma
‘président’ in the training data, the masculine singular form, becomes even
more common after training. The least common word form of that same lemma, the
feminine plural ‘présidentes’, decreases in frequency and is for some models
even completely lost in translation. In line with the abovementioned research,
recent work by Costa-jussà et al. (2022) shows how different architectures
trained on the same data exhibit different levels of gender bias. When
comparing language-specific encoder-decoders to shared encoder-decoder models,
the former are less biased (Costa-jussà et al., 2022). If machine
translationese, i.e. the language produced by MT systems, represent
simplified, less lexically and morphologically diverse versions of the
training data, this also raises more general questions related to the
sociolinguistic implications and its potential long-term effects on language
(Vanmassenhove et al., 2021).

We ought to note, however, that while the abovementioned biases might explain
the biased outputs observed in automatically generated translations, simply
‘debiasing’ or ’balancing’ the datasets or the model will not enable systems
to provide multiple correct alternatives (e.g. the previously provided
translations from Hungarian into English) since this would require the
generation of multiple alternative translations. More balanced data might
however enable systems to more accurately translate sentences where the gender
of a referent is present (see the previously given example: ‘My husband is a
kindergarten teacher’) and improve the overall quality of translations for all
genders.

##  3 Related Work

While it is out of the scope of this chapter to discuss the entire body of
research on gender bias in Natural Language Technology and MT, we will discuss
the most pioneering, impactful and/or most closely related work. We will cover
some of the existing research on gender bias in both neural MT systems
(Section 3.1) and more recent work focusing on GPT models used as MT systems
(Section 3.2). NMT models are typically bidirectional sequence-to-sequence
(seq2seq)(Sutskever et al., 2014) architectures with encoder-decoder
frameworks while GPT (Generative Pre-trained Transformer) models are primary a
language models for language generation tasks. Both the more commonly employed
NMT systems (the type of model used by Google Translate77endnote:
7https://translate.google.com/, DeepL88endnote: 8https://www.deepl.com, and
others.) and GPT models are based on Transformers (Vaswani et al., 2017).
However, GPT models are decoder-only models as opposed to the usual encoder-
decoder architectures of the conventional MT systems. They are furthermore
primarily trained on monolingual data (as opposed to carefully curated
parallel corpora) and they require a significantly larger number of parameters
(Hendy et al., 2023). When using a GPT model for a translation task, it
involves relying on the GPT model’s decoding capabilities in order to generate
target language translations given a specific input in a unidirectional
autogressive way, generating one token at a time, conditioned on the source.

###  3.1 "Conventional" NMT systems

The related research on bias in MT can be largely divided into approaches that
aim to mitigate bias by addressing the (i) pre-processing stage (training data
and/or representations); (ii) processing stage (algorithm); or (iii) post-
processing stage (output). A large body of research on gender bias in language
technology focused on debiasing models by changing the internal
representations of separate words (often limited to animate nouns or
adjectives) as a pre-processing step (a.o. Bolukbasi et al. (2016); Zhao et
al. (2018)). Debiasing is then usually done by identifying and capturing a
gender subspace using seed words (such as ‘she’, ‘he’, ‘girl’, ‘boy’…) and
subtracting the obtained gender dimension from words such as ‘plumber’,
‘receptionist’ or ‘doctor’ that are gender-neutral and should not carry gender
information. The approach has been criticised in subsequent papers due to the
fact that there is no clear correlation between removing bias from the
internal representations, the so-called intrinsic bias, and the bias observed
in downstream tasks or the extrinsic bias (Gonen and Goldberg, 2019; Goldfarb-
Tarrant et al., 2021). Aside from the limited impact of these debiasing
techniques on the actual translations (Font and Costa-jussà, 2019), pre-
processing techniques have other shortcomings, for instance, these methods (i)
rely on lists of manually selected seed words that are ought to capture the
‘gender’ dimension; (ii) are usually applied to animate nouns (and sometimes
adjectives) while other part-of-speechs (for instance, verbs, pronouns) might
also require debiasing; (iii) offer no control over the eventual output as the
MT system will still offer only one translation option; and (iv) do not allow
to go beyond the binary gender distinction, i.e. more recent phenomena such as
gender-neutral they cannot be handled in an appropriate manner.

Another type of preprocessing techniques that have been explored is the
integration of additional features that can help resolve ambiguities from the
source into the target language (Vanmassenhove et al., 2018; Moryossef et al.,
2019). Vanmassenhove et al. (2018) built the first speaker-informed NMT system
where a tag indicating the gender of the speaker was fed to the NMT system
along with the source input. Although the gender tags (that were limited to
binary gender) improved the translation in terms of automatic evaluation
scores, the manual evaluation revealed some (potentially undesirable) side-
effects. While the gender tags were intended to improve the morphological
gender agreement with the gender of the speaker, they often resulted in
different lexical word choices. Similarly Moryossef et al. (2019) employed a
parataxis construction such as She told him:’ to help the translation system
disambiguate the gender of referents. Both approaches are limited in terms of
the referents that can be controlled (first person singular as a subject
(Vanmassenhove et al., 2018), and first person singular as a subject as well
as the third person singular as the (in)direct object (Moryossef et al.,
2019). In Stafanovičs et al. (2020), a method is presented whereby the gender
of the words in the target are projected onto the source. The grammatical
gender annotations from the target language in the source data can be
leveraged by the MT system. The idea is somewhat similar, although more fine-
grained, to the previous approaches presented given that it is assumed that
the additional information provided will be a strong enough learning signal
for the MT system to generate alternative translation. The evaluation of their
approach is limited to automatic evaluation metrics based on the WinoMT test
suite (Stanovsky et al. 2019). WinoMT is a limited test suite given that it
contains only sentences that are unambiguous in the source (i.e. the gender of
the referents can be inferred from the source).

To the best of our knowledge, only one work has attempted to control gender by
opening the NMT black-box. Bau et al. (2019) developed an unsupervised method
to identify and (de)activate the neurons responsible for specific linguistic
phenomena. From their experiments, it resulted that gender is the most
difficult phenomenon to capture and control with a 21% success rate using the
best setup. Gender is particularly hard to control because the ‘gender
neurons’ are distributed along the network making it a difficult property to
modify.

We would like to note that most of the related work focuses on gender from a
binary point of view (male/female) with exception to the work by Saunders et
al. (2020) who added gender-neutral tags to an existing dataset and replaced
gendered endings by placeholders, and the work by Vanmassenhove et al. (2021)
and Sun et al. (2021) which focuses on developing a gender-neutral rewriter
for English.

For a more in-depth overview of concepts related to bias, a summary of work on
gender bias and a discussion of mitigation strategies proposed (up until 2021)
for MT, we refer the reader to the work by Savoldi et al. (2021).

###  3.2 GPT for MT

GPT models are a type of LLMs that started gaining mainstream popularity near
the end of 2022, when ChatGPT’s user-friendly interface99endnote:
9https://chat.openai.com/ provided easy access to some of the capabilities of
GPT models, impressing the public by its human-like responses in a
conversational setup (Toews, 2023). GPT models are developed by
OpenAI’s1010endnote: 10www.openai.com research lab. The first one, GPT, was
developed back in 2018. Since then, various newer versions saw the light of
day (GPT-2 (2019), GPT-3 (2020) and GPT-4 (2023)..), which are all iterations
of the same GPT language model. The main difference between the different
versions being the (exponential) growth in parameters: from 117 million
parameters (GPT) to 175 billion for GPT-3, making it at the time the largest
LLM. Reuters1111endnote: 11https://www.reuters.com article accessible via
https://shorturl.at/dewzZ reported that according to an UBS study, ChatGPT
became the fastest growing consumer application ever recorded, reaching 100
million monthly active users in January 2023, only two months after its launch
on November 30, 2022. Given its popularity and wide-range usage (recreational
and professional), ChatGPT is considered the first notable threat to Google’s
monopoly1212endnote: 12https://www.nytimes.com article accessible via
https://shorturl.at/ruRS3(Ghosh and Caliskan, 2023).

When talking about LLMs, since 2023, it is common to distinguish between bi-
directional Transformer architecture (for instance, Bidirectional Encoder
Representations from Transformers (BERT) (Devlin et al., 2019)) and uni-
directional ones such as the GPT models. While unidirectional language models
process and generate strictly in one direction (they process input from left-
to-right and generate output left-to-right, token per token), bidirectional
ones can attend to both the left and right context when processing input and
output sequences, conditioning on preceding and succeeding tokens to generate
output. The choice between unidirectional and bidirectional language models
depends on the specific requirements of the task at hand. Unidirectional
models are well-suited for sequential text generation and are computationally
efficient, while bidirectional models tend to excel in tasks that demand a
deeper understanding of context and long-range dependencies within the text.
Both type of models (e.g. BERT vs GPT-3) have been used for the task of MT,
each with their own strengths and weaknesses. BERT itself is not typically
used as a direct MT system, it has been incorporated into MT pipelines in
various ways to improve translation performance and address certain challenges
(see previous section). As for GPT models, the interactive nature of ChatGPT
(based on GPT models) allows it to engage in back-and-forth conversations with
users which is an interesting ability to experiment with in the context of
bias since it implies we can explicitly prompt it to be mindful of biases or
to generate multiple alternative translations and thus potentially mitigate
biased output. This interactivity opens avenues for experimentation in
addressing bias in MT and tailoring translations to meet user preferences and
specific contexts. Hereafter, we will therefore focus on related work that
specifically explores the use of GPT models for MT. We ought to note that many
of these works present very recent and somtimes preliminary findings published
on arxiv1313endnote: 13https://arxiv.org/ which have been approved for posting
after moderation but have not gone through a peer review process yet.

Various studies have already investigated the more general translation
abilities of GPT models. Among those are the work by Hendy et al. (2023); Jiao
et al. (2023) and Wang et al. (2023) which evaluates GPT models for MT on a
sentence-level (Hendy et al., 2023; Jiao et al., 2023; Wang et al., 2023) and
on a document-level Hendy et al. (2023); Wang et al. (2023). Hendy et al.
(2023) conducted an extensive automatic and human evaluation. They assessed
the performance of three GPT models (ChatGPT, GPT3.5 (text-davinci-003), and
text-davinci002) on 18 language pairs across 4 domains and compared it to
state-of-the-art (research and commercial) translation systems. They used the
top-ranked systems from the WMT evaluation campaign to identify the state-of-
the-art and the respective baselines for comparison. Jiao et al. (2023)
present a more preliminary study using ChatGPT (GPT-3) for MT. Their analysis
is deemed preliminary since it contains only 50 sentences for every
set/condition and is additionally limited by the fact that they only employ
automatic evaluation metrics. Both studies, however, find that while GPT
models can perform competitively with existing MT systems in high-
resource1414endnote: 14We ought to note that this is an ill-defined term.
settings/languages, they significantly underperform when used in low-resource
setting or for distant language paris (Hendy et al., 2023; Jiao et al., 2023).
Jiao et al. (2023) re-evaluated the performance with launch of GPT-4 (March
15, 2023) observing that the performance of ChatGPT based on GPT-4 became more
comparable to commercial translation systems for more distant
languages.1515endnote: 15Similar to ’low-resource languages’ the terms
’distant languages’ is not very well-defined. In the paper itself, they refer
to ‘European’ vs ‘Asian’ languages. Wang et al. (2023) compare commercial MT
systems (Google Translate1616endnote: 16https://translate.google.com/,
DeepL1717endnote: 17https://www.deepl.com/en/translator, and Tencent
TranSmart1818endnote: 18https://transmart.qq.com/) and document-level NMT
models with GPT models and focus particularly on evaluating their discourse
awareness through automatic and human evaluations. They conclude that ChatGPT
outperforms the commercial MT systems according to human evaluation and is
thus a potentially promising avenue to explore for document-level translation.

Rather than concentrating on a direct comparison between existing MT systems
and GPT models, other research (Peng et al., 2023) has explored how to
effectively harness the capabilities and potential of GPT models in the
context of translation. These investigations seek to optimize and maximize the
benefits that GPT models can offer for improving translation quality and
addressing specific translation challenges, for instance by exploring
different prompting strategies or the possibility for personalized MT.

In Peng et al. (2023), ChatGPT is evaluated and compared to Google Translate
with an emphasis on how to maximize the potential of ChatGPT. Their findings
highlight the importance of clarifying the task information. They find that
ChatGPT performs better when you make the task at hand very clear by stating
that you are expecting it to serve as an MT system in a particular domain. The
temperature, a (hyper)parameter that can be set to control the level of
randomness/unpredictibility in the texts it generates, has a relatively small
impact when dealing with high-resource languages (i.e. languages for which we
generally have a large amount of linguistic resources, e.g. German or French).
Additionally, they conduct a ’chain of thought’ experiment where ChatGPT is
prompted to not only translate but also explain its ‘thoughts’ while doing so.
The chain-of-thought experiments, however, led to word-by-word translation,
thus degradeding the overall translation quality. It is important to highlight
here that their evaluation consisted only of automatic evaluation metrics
(Comet (Rei et al., 2020) and SacreBLEU (Post, 2018). In contrast with the
findings presented in Peng et al. (2023), chain-of-thought experiments (Wei et
al., 2022) combined with error analysis (Lu et al., 2022), an approach coined
error analysis prompting, can lead to human-like MT evaluation at both the
system and segment level (Lu et al., 2023). When prompted to handle multiple
translation in one query, they observed unstable scorings and biases. Aside
from automatic evaluation metrics (COMET, BLEU (Papineni et al., 2002),
BERTscore (Zhang et al., 2019) and BLEURT(Sellam et al., 2020)), they refer to
a ‘human evaluation’ which, in reality, consists of an evaluation based on a
high-quality dataset annotated by humans (and thus not an actual evaluation
conducted by humans). Their findings should be considered preliminary given
that they sample only 10 segments from each group they investigate (groups are
based on the number of tokens) for Chinese–English, English–German and their
reliance on automatic evaluation metrics.

Two other papers investigate the usefulness of additional linguistic
information integrated in the prompts fed to ChatGPT. Gu (2023) investigate a
two-step prompt strategy where they inject linguistic knowledge into the
prompt designed to aid MT. In Gao et al. (2023), aside from explicitly
mentioning the translation task information (e.g. domain), POS-tags are
injected. While these are potentially interesting avenues to explore from an
academic point of view in order to explore GPT models’ full potential as an MT
system, we can question to what extent it is realistic to explore all possible
prompts for various language pairs/settings and effectively expect MT users to
employ those strategies.

While the aforementioned research focused on the quality and potential of GPT
models as MT systems, they did not particularly look into its ability to
handle specific cross-linguistic phenomenon or biases. In Ghosh and Caliskan
(2023), the work most closely related to ours, they examine whether ChatGPT
(based on GPT-3) perpetuates gender bias across Bengali and five other low-
resource languages: Farsi, Malay, Tagalog, Thai and Turkish. All the
investigated languages, aside from English, use gender-neutral pronouns. This
implies that, when translating from one of those languages into English, the
gender-neutral pronoun can be translated as either he or she (or gender-
neutral they). In this translation setup, it becomes apparent that ChatGPT
perpetuates and amplifies biases related to gender roles and stereotypes, in
terms of actions (e.g. ‘cooking’ is a female activity) and occupations (e.g.
‘doctor’ is a male occupation), when translating gender-neutral pronouns into
English. Aside from that, in the context of occupations, it confers more
respect to men than to women.1919endnote: 19Bengali has two pronouns, one of
which is the more respectful form and they found it to be used mainly in
sentences featuring a male pronoun. Vice versa, when translating from English,
the gender-neutral pronoun they could easily be translated into the equivalent
gender-neutral pronoun in the other languages, however, their experiments show
that currently, ChatGPT is unable to do so.

Ghosh and Caliskan (2023) furthermore rightfully stress that, while other MT
providers have been criticized for the biased outputs produced by their MT
systems, OpenAI has claimed to have put extensive bias mitigation measures in
place. Yet, the experiments seem to indicate that the same gender biases that
have been observed and criticized in commercial systems (e.g. Google Translate
or MS Translator2020endnote: 20https://www.microsoft.com/nl-nl/translator/)
persist. Given the popularity of ChatGPT, this warrants further investigation
and the authors vouch for a more human-centered approach. While our
experiments are still preliminary, we aim to contribute to the investigation
of gender biases in ChatGPT used for MT.

##  4 Preliminary Experiment and Analysis

In this preliminary experiment, we explore how and to what extent ChatGPT can
handle gender across the English-Italian Language pair. While English is a
language with pronominal gender, where only the pronouns are
gendered2121endnote: 21A few exceptions are nouns such as ’waitress’ vs
’waiter’ or ‘actress’ vs ‘actor’., Italian is a more highly inflected language
where different categories (nouns, adjectives…) agree in gender with their
referent. Additionally, as opposed to English, Italian is a language with
grammatical gender meaning that also inanimate nouns are assigned to a gender
category. The example below is one taken from the challenge set used for our
experiment and it provides a translation for an English gender-neutral
sentence into Italian, where the past participle ‘sentito/a/[‘to feel’] agrees
with the gender of its subject you.2222endnote: 22We added a version that is
not included in gENder-IT using the ‘’ which has been recommended by Baiocco
et al. (2023) as a non-gender-specific ending for Italian.

English
‘What did you expect it to feel like?

Italian
‘Como pensavi che ti saresti sentito’
‘Como pensavi che ti saresti sentita’ ‘Como pensavi che ti saresti sentit’

###  4.1 Experimental Setup

The following sections describe the experimental setup, datasets and promps
along with the error analysis conducted. For the experiments, we
accessed2323endnote: 23We accessed ChatGPT to generated these translations
between May 30, 2023 and June 5, 2023. ChatGPT (based on GPT-3.5) through the
API. Given that the related work on using GPT models for MT highlighted the
importance of (explicit) prompting, we looked at two different prompting
scenarios.

####  4.1.1 Dataset

We used a subset of an existing challenge dataset, gENder-IT (Vanmassenhove
and Monti, 2021), designed specifically for resolving natural gender phenomena
in translation contexts. The challenge set is a word-level (human) annotated,
adapted and cleaned version of a subset of the MuST-SHE corpus (Bentivogli et
al., 2020), but unlike MuST-SHE which is annotated on the paragraph level, the
sentences have been split and annotated on the word-level with $<$F$>$
(female), $<$M$>$ (male) or $<$A$>$ (ambiguous) tags. It focuses on sentences
containing textually ambiguous words (in terms of gender) in the English
source and provides the possible translation alternatives (in terms of binary
gender)2424endnote: 24There is no crystallized approach to gender-neutral
endings in Italian as to date. on the Italian target side. Two example of
tagged sentences from the gENder-IT corpus are provided below for
illustration. The first one, Example 1, only contains <F> tags since the
gender of ‘ambiguous’ words such as ‘I’ can be determined based on sentence
context. Example 2, is one where within the sentence context, no assumption
can or should be made with respect to the gender of the referent. To indicate
that not all ambiguous words have the same referent, numbers are appended
(e.g. $<$A1$>$, $<$A2$>$…). We used a subset of gENder-IT for the manual error
analysis consisting for which we used the first 50 sentences of the gENderIT
corpus.

  1. 1

“So she turned and she looked at her dad, and she said, “Dad, I <F> know how
you <M> feel, but I <F> don’t believe in the death penalty.”’

  2. 2

‘And it was there that another nurse <A1>, not the nurse <A2> who <A2> was
looking after Mrs. Drucker <F> before, but another nurse <A1>, said three
words to me <A3> that are the three words that most emergency physicians <A4>
I <A3> know dread.’

####  4.1.2 Prompting Scenarios

GPT’s autoregressive approach allows it to produce contextually relevant
responses which allow for more engaging interactions or requests as opposed to
the conventional MT systems. This combined with the fact that previous
research highlighted the importance of explicit prompting, we conducted the
experiments with two different prompting scenarios. Furthermore, given that
the research by Peng et al. (2023) showed an improvement in performance for MT
when prepending information stating the task at hand, in both scenarios, we
entered the following prompt before generating any translations: ‘You are a
Machine translation system’. We generated the translations one by one in both
cases. In the first one, we simply prompted ChatGPT to translate sentences
containing an ambiguous pronoun or noun into Italian. In the second one, we
explicitly prompted it to provide all the possible alternatives in terms of
gender.

  1. 1

Prompt Scenario 1 ‘Can you translate the following sentence into Italian: +
[insert English sentence]’

  2. 2

Prompt Scenario 2 ‘Can you translate the following sentence into Italian
providing all the possible alternatives in terms of gender + [insert English
sentence]’

Scenario 1 is probably closer to how an MT-user would prompt ChatGPT since
they are not necessarily aware of the fact that the target language might
differ from the source in terms of gender marking. It would also give us some
insight into how gender is handled ‘by default’. Ideally, ChatGPT would
provide alternative translation even when not explicitly prompted to do so.
Scenario 2 explicitly asks for alternative translation in terms of gender. In
this scenario we hope to explore what the potential of GPT models is with
respect to handling gender phenomena across languages. We ought to note that
the tags ($<$F$>$, $<$M$>$ and $<$A$>$) with which sentences are annotated in
the gENder-IT challenge set were removed from the English sentences for the
prompt. For convenience, we added the tags to the examples provided in the
subsequent section.

###  4.2 Manual error analysis

We conducted a manual analysis/evaluation of the translations generated by
ChatGPT for both scenarios.

####  4.2.1 Prompt Scenario 1

For Scenario 1, the one where the prompt did not explicitly state that
alternatives are to be generated, only 5 out of the 50 sentences did not miss
valid alternatives in terms of gender. Out of those 5, three times ChatGPT had
provided both alternatives (see Example 1, ‘voltato/a’) in the translation and
two times there were no missing alternatives since the translation differed
from the original one in such a way that the translation did not contain any
forms marking the gender (see Example 1).

  1. 1

EN “So I <A> turned around and I <A> looked at her, and I <A> said, "Are you
<F> gonna wear this?"”
IT “Mi sono voltato/a e l’ho guardata dicendo: ""Lo indosserai?"”

  2. 2

EN “What did you <A> expect it to feel like?”
IT “Cosa ti aspettavi che si sentisse?”

For all other 45 sentences, there were either errors or valid translation
alternatives in terms of gender missing. Two sentences contained errors with
respect to gender in the generated translation. In Example 1, it is clear from
the sentential context that ‘classmate’ refers to someone with a female
gender. Nevertheless, it is translated as ‘compagno [masculine]’ in the
Italian translation generated by ChatGPT. Similarly, Example 2 shows how child
which should be translated into the feminine form according to the given
context, is translated into Italian as ‘figlio’ which means ‘son’. Note that
in both cases, the error occurred due to the fact that a masculine form was
generated while the correct form was feminine.

  1. 1

EN “A college classmate <F> of mine wrote me <A> a couple weeks ago and said
she thought I <A> was a little strident.”
IT “Un compagno di classe del college mi ha scritto un paio di settimane fa
dicendo che pensava che fossi un po’ stridente.”

  2. 2

EN “I <A1> knew that if in one hour, our child <F> didn’t come, I <A1> was
going to leave her there and go back to work and make a presentation about AI
to my  boss <A2>, Apple’s CEO <A2>.”
IT “Sapevo che se nostro figlio non fosse arrivato entro un’ora, l’avrei
lasciata lì e sarei tornato al lavoro per fare una presentazione sull’IA al
mio capo, il CEO di Apple.”

In the remaining 43 sentences there was at least one alternative translation
missing. To illustrate that, we can once more refer back to Example 2 where
‘boss’ in English could be translated into Italian as capa [feminine] or capo
[masculine], yet only the masculine alternative was provided. When only
considering the sentences with two alternative translations, in the vast
majority of the cases, the feminine alternatives were lacking with the
exception to a few sentences containing the noun ‘nurse’, ‘secretary’ and
names stereotypically associates with the female gender (e.g. ‘Vera’). In the
gENder-IT challenge set, however, they opted to not assign genders based
solely on the names because of potential cultural biases - which all names are
considered ‘ambiguous’ unless a full name is provided and the gender of the
person could be retrieved through a Wikipedia page (i.e. public/historical
figures).

####  4.2.2 Prompt Scenario 2

For Scenario 2, ChatGPT was prompted to translate the same 50 sentences, the
only difference being that this time it was explicitly stated that it should
generate all possible alternatives in terms of gender for the translation. Out
of the 50 sentences, 16 of the translations were entirely correct with no
missing alternatives. For half of the sentences, valid alternatives were
missing and in 17 sentences there were erroneous translations (in terms of
gender). In this scenario, all the alternatives missing were alternatives
containing feminine forms. For instance, in 1 only one translation is
generated while ‘pediatrician’ is ambiguous in the source context and could be
translated into the feminine or masculine form in Italian (dal pediatra
[masculine] or dalla pediatra [feminine]) yet only the masculine alternative
is provided. Note as well how the proper noun ‘Sarah’ is translated as
‘Laura’.

  1. 1

EN “And when Elliot was first born, we <A1> took her to the pediatrician <A2>
and we met our first nurse <F> — we’ll call her Sarah.”
IT “E quando Elliot è nata per la prima volta, l’abbiamo portata dal pediatra
e abbiamo incontrato la nostra prima infermiera — la chiameremo Laura.”

Arguably more problematic than the missing alternatives, were the alternatives
that were erroneous since they did not stay true to the source. In Example 1,
four alternative translations were generated given the English source
sentence. While the first alternative stays true to the source by translating
‘mother’ into ‘madre’, in the subsequent alternatives it is erroneously
translated into the Italian as ‘padre’, ‘fratello’ and ‘sorella’ which mean
‘father’, ‘brother’ and ‘sister’ in English.

  1. 1

EN “My mother had been left behind in the camp, and I <A1> had not seen her
since I <A1> left it <A2>.”
IT: Alternative 1 “Mia madre era stata lasciata indietro nel campo e non
l’avevo più vista da quando l’avevo lasciato.”
IT: Alternative 2 “Mio padre era stato lasciato indietro nel campo e non
l’avevo più visto da quando l’avevo lasciato.”
IT: Alternative 3 “Mio fratello era stato lasciato indietro nel campo e non
l’avevo più visto da quando l’avevo lasciato.”
IT: Alternative 4 “Mia sorella era stata lasciata indietro nel campo e non
l’avevo più vista da quando l’avevo lasciata.”

Similarly, in Example 1, ChatGPT manages to correctly provide two alternatives
for the gender-neutral English psychiatrist, un/una psichiatra. However, it
then proceeds by providing two additional inaccurate alternatives where
psychiatrist is translated as ginecologo (gynaecologist).

  1. 1

EN “I’m a psychiatrist <A> who <A> works with pregnant and postpartum women…”
IT: Alternative 1 “Sono uno psichiatra che lavora con donne in gravidanza e
nel periodo postpartum…”
IT: Alternative 2 “Sono una psichiatra che lavora con donne in gravidanza e
nel periodo postpartum…”
IT: Alternative 3 “Sono un ginecologo che lavora con donne in gravidanza e nel
periodo postpartum…”
IT: Alternative 4 “Sono un ginecologo che lavora con donne in gravidanza e nel
periodo postpartum…”

###  4.3 Findings & Implications

Our preliminary findings on the small challenge set are in line with the
findings of Ghosh and Caliskan (2023). Our analysis, of both prompting
scenarios, indicates a (strong) male bias which seems to become even more
prevalent when asked explicitly to generate alternatives. While ChatGPT has
some potential when it comes to generating alternatives when prompted
explicitly, it currently cannot handle gender in a systematic manner. Given
the current hype surrounding LLMs and their potential, we deem it important to
raise awareness among researchers and potential users of these technologies.
We will summarize the main findings regarding the model’s handling of gender
and its potential implications.

First of all, in both the Scenario 1 and 2, the analysis revealed serious
shortcomings in ChatGPT’s ability to generate multiple possible translations
in terms of gender. In Scenario 1, only a very small proportion of the
translated sentences contained all binary gender translation alternatives. In
Scenario 2, there were numerous instances where feminine gender alternatives
were entirely missing. Furthermore, while in Italian, there is no crystallized
approach yet to gender-neutral language, gender-neutral markers such as the
asterisk or the schwa are used by the transgender Italian community. Such
endings or markers were never provided or suggested by the model. Its
inability to account for gender alternatives when not explicitly prompted to
do so is already problematic as it demonstrates a lack of sensitivity to
gender-inclusive language. However, the absence of gender-neutral and feminine
alternatives, especially in Scenario 2, where explicit instructions were
provided, reveals an actual inability of the model to handle gender
systematically across languages. Both observations are leading to incomplete
and biased translations and emphasize the need for ongoing improvements and
vigilance. Furthermore, the absence of gender-neutral language markers in the
model’s output is a missed opportunity for fostering greater inclusivity. The
failure to provide such alternatives reflects a lack of awareness of evolving
language norms and the specific needs of different user groups.

Second, in Scenario 2, there were various erroneous translations among the
alternatives provided. The model would provide alternatives that deviated from
the source text and introduced misleading and incorrect information. These
erroneous alternatives were sometimes not (directly) related to gender
(Example 1) but often the (feminine) terms were inaccurately replaced with
masculine alternatives. These translations not only distort the intended
message and thus the accuracy of the translation but also often perpetuate
gender biases that are deeply ingrained in our society. Erroneous and biased
translations that misrepresent the source content furthermore carry the risk
of alienating or excluding individuals who do not conform to traditional
gender norms.

Overall, these findings underscore the need for continued research and
development to enhance our understanding of LLMs’ handling of gender-related
nuances in translation. Like more traditional MT models, ChatGPT struggles
with gender (biases) across languages. The ability to explicitly prompt it to
handle issues related to bias and its tendency to generate answers that at
first sight seem to address the issue appropriately furthermore risks creating
a feeling of complacency. It is however crucial to recognise that while LLMs,
like GPT3.5 have shown remarkable capabilities in various NLP tasks, they are
not immune to biases. While explicit prompting might be a useful technique to
alleviate some biases, this does not guarantee consistent bias-free or fully
accurate responses. When LLMs generate responses that seem to handle gender
bias well on the surface, users might become complacent and assume that the
issue has been adequately addressed while underlying biases persist. Vigilance
is thus required to assess the generated content critically and carefully.
Addressing these issues is not only a matter of linguistic accuracy but also a
step towards promoting more inclusive and respectful communication in the
digital age. The AI community at large, in collaboration with linguists,
translators and a diverse set of users should collaborate to further evaluate
these models to eventually minimize bias, and ensure that AI-generated content
aligns with evolving societal standards of gender-inclusivity and neutrality.

####  4.3.1 Limitations

We would like to discuss some of the limitations of our work. First of all,
there is a reproducibility problem that occurs when querying ChatGPT. The
results of the same query/prompt may vary across multiple trials and the
models underlying the app can be updated at any time. This implies that there
is a certain randomness to the evaluation and analysis of the translations.
Furthermore, currently, a novel LLM (GPT-4) has been developed and can be
accessed through a $20 monthly subscription to ChatGPT. According to Jiao et
al. (2023), GPT-4 outperforms GPT-3.5 as an MT system which warrants for a
replication using GPT-4. Nevertheless, the free version of ChatGPT is still
based on GPT-3.5, implying that a large amount of users will still be using
the currently investigated model for their daily needs.

Aside from the reproducibility limitations, the gENder-IT corpus does not
contain any gender-neutral sentences (e.g. using the gender-neutral pronoun
they in the English source). We prompted ChatGPT to generate gender
alternatives but did not ask specifically for gender-neutral alternatives.
While Ghosh and Caliskan (2023) finds that ChatGPT cannot handle the English
gender-neutral pronoun, they also did not explicitly prompt it to translate in
a gender-neutral way.

##  5 Conclusion

In conclusion, this chapter has delved into the critical role of MT in
perpetuating and shaping societal biases, particularly gender bias. Within the
realm of MT, gender biases are especially pronounced due to the need for
disambiguation and explicit representation of gender in specific contrastive
linguistic settings. The reliance on biased data and statistical patterns in
current MT systems can lead to erroneous disambiguation, resulting in the
generation of morphologically incorrect alternatives, reinforcing potentially
harmful stereotypes.

We provided an overview of existing work related to gender bias in both
conventional neural machine translation (NMT) approaches and the use of GPT
models as MT systems. We furthermore covered and explained some of the
origins/sources of gender bias we observe in language technology. Given the
rising popularity of tools like ChatGPT, it is crucial to raise awareness
about these issues and take proactive steps to address them. Our experiments
with ChatGPT, based on GPT-3.5, in an English-Italian translation setting
showed how GPT models perpetuate biases even when explicitly prompted to
provide alternative translation. Surprisingly, prompting for gender
alternatives often even led to additional biases in the model’s outputs,
highlighting the need for further research and development in this area.

The implications of these findings underscore the importance of ongoing
research and improvements to promote fair and unbiased translations. We
advocate for more hybridization in both technology and collaboration among
researchers from various disciplines, including computational linguists,
computer scientists, sociolinguists, and ethicists. Such multidisciplinary
efforts are essential to address the complex and multifaceted nature of gender
bias in MT. We acknowledge that our experiments have been conducted with a
relatively small dataset. To gain a deeper understanding of GPT’s ability to
deal with gender across languages and to expand the scope of our research, a
more detailed analysis with a larger set of sentences across multiple language
pairs should be conducted in future work to assess the model’s effectiveness
across different linguistic contexts.

Looking ahead, an important area of future work is to investigate how GPT
models can be used to generate gender-inclusive datasets. This involves
exploring ways to integrate linguistic knowledge into the model to guide the
generation of gender alternatives, including those beyond the binary. One
potential approach could be to provide tags, similar to those used in the
gENder-IT challenge, to influence the model’s output and encourage gender-
inclusive translations. As the use of GPT models and other language generation
technologies continues to grow, it becomes increasingly vital to ensure that
these systems are equipped to handle gender and promote inclusivity. Our
research aims to contribute to the ongoing efforts to make MT systems more
equitable and sensitive to issues of gender representation.

In closing, this chapter has shed light on the challenges posed by gender bias
in MT. It emphasizes the need for continued multidisciplinary research,
advancements, and collaborative endeavors to pave the way towards more
equitable and unbiased language technologies. As the field progresses, we hope
to see the transformation of MT systems into powerful tools that promote
diversity, inclusivity, and societal progress.

## Acknowledgements

I would like to thank Esther Monzó-Nebot for setting up a collaborative
initiative gathering expertise from linguistics, computer science and law in
order to discuss some of the more pressing issues regarding translation
technology. Additionally, I would like to thank Deborah Giustini, Heidi
Salaets and Maurizio Veglio for sharing their expertise and broadening my
knowledge across diverse fields.

## References

  * Baiocco et al. (2023) Baiocco, R., F. Rosati, and J. Pistella (2023).  Italian proposal for non-binary and inclusive language: The schwa as a non-gender–specific ending.  Journal of Gay & Lesbian Mental Health, 1–6.
  * Bau et al. (2019) Bau, A., Y. Belinkov, H. Sajjad, N. Durrani, F. Dalvi, and J. Glass (2019, May).  Identifying and Controlling Important Neurons in Neural Machine Translation.  In Proceedings of the Seventh International Conference on Learning Representations (ICLR), New Orleans, USA.
  * Bentivogli et al. (2020) Bentivogli, L., B. Savoldi, M. Negri, M. A. Di Gangi, R. Cattoni, and M. Turchi (2020, July).  Gender in danger? evaluating speech translation technology on the MuST-SHE corpus.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, pp. 6923–6933. Association for Computational Linguistics.
  * Birhane (2021) Birhane, A. (2021).  Algorithmic injustice: a relational ethics approach.  Patterns 2(2).
  * Bolukbasi et al. (2016) Bolukbasi, T., K.-W. Chang, J. Y. Zou, V. Saligrama, and A. T. Kalai (2016, December).  Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings.  In Proceedings of Thirtieth Conference on Neural Information Processing Systems (NIPS), Barcelona, Spain, pp. 4349–4357.
  * Caliskan et al. (2022) Caliskan, A., P. P. Ajay, T. Charlesworth, R. Wolfe, and M. R. Banaji (2022).  Gender bias in word embeddings: a comprehensive analysis of frequency, syntax, and semantics.  In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, pp. 156–170.
  * Cho et al. (2019) Cho, W. I., J. W. Kim, S. M. Kim, and N. S. Kim (2019, August).  On measuring gender bias in translation of gender-neutral pronouns.  In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, Florence, Italy, pp. 173–181. Association for Computational Linguistics.
  * Costa-jussà et al. (2022) Costa-jussà, M. R., C. Escolano, C. Basta, J. Ferrando, R. Batlle, and K. Kharitonova (2022).  Interpreting gender bias in neural machine translation: Multilingual architecture matters.  In Proceedings of the AAAI Conference on Artificial Intelligence, Volume 36, pp. 11855–11863.
  * Devlin et al. (2019) Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova (2019).  Bert: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, (NAACL-HLT 2019), Volume 1 (Long and Short Papers), Minneapolis, Minnesota, USA, pp. 4171–4186.
  * Font and Costa-jussà (2019) Font, J. E. and M. R. Costa-jussà (2019, August).  Equalizing Gender Biases in Neural Machine Translation with Word Embeddings Techniques.  In _To appear in_ Proceedings of the 1st ACL Workshop on Gender Bias for Natural Language Processing, Florence, Italy.
  * Gao et al. (2023) Gao, Y., R. Wang, and F. Hou (2023).  How to design translation prompts for chatgpt: An empirical study.  arXiv e-prints, arXiv–2304.
  * Ghosh and Caliskan (2023) Ghosh, S. and A. Caliskan (2023).  Chatgpt perpetuates gender bias in machine translation and ignores non-gendered pronouns: Findings across bengali and five other low-resource languages.  arXiv preprint arXiv:2305.10510.
  * Goldfarb-Tarrant et al. (2021) Goldfarb-Tarrant, S., R. Marchant, R. M. Sánchez, M. Pandya, and A. Lopez (2021).  Intrinsic bias metrics do not correlate with application bias.  In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1926–1940.
  * Gonen and Goldberg (2019) Gonen, H. and Y. Goldberg (2019).  Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 609–614.
  * Gu (2023) Gu, W. (2023).  Linguistically informed chatgpt prompts to enhance japanese-chinese machine translation: A case study on attributive clauses.  arXiv preprint arXiv:2303.15587.
  * Hendy et al. (2023) Hendy, A., M. Abdelrehim, A. Sharaf, V. Raunak, M. Gabr, H. Matsushita, Y. J. Kim, M. Afify, and H. H. Awadalla (2023).  How good are gpt models at machine translation? a comprehensive evaluation.  arXiv preprint arXiv:2302.09210.
  * Jiao et al. (2023) Jiao, W., W. Wang, J.-t. Huang, X. Wang, and Z. Tu (2023).  Is chatgpt a good translator? a preliminary study.  arXiv preprint arXiv:2301.08745.
  * Koehn (2005) Koehn, P. (2005, September).  Europarl: A Parallel Corpus for Statistical Machine Translation.  In Proceedings of The Tenth Machine Translation Summit (MT Summit 2005), Phuket, Thailand, pp. 79–86.
  * Lu et al. (2022) Lu, Q., L. Ding, L. Xie, K. Zhang, D. F. Wong, and D. Tao (2022).  Toward human-like evaluation for natural language generation with error analysis.  arXiv preprint arXiv:2212.10179.
  * Lu et al. (2023) Lu, Q., B. Qiu, L. Ding, L. Xie, and D. Tao (2023).  Error analysis prompting enables human-like translation evaluation in large language models: A case study on chatgpt.  arXiv preprint arXiv:2303.13809.
  * Moryossef et al. (2019) Moryossef, A., R. Aharoni, and Y. Goldberg (2019, August).  Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injection.  In _TO APPEAR IN_ 1st ACL Workshop on Gender Bias for Natural Language Processing, Florence, Italy.
  * Papineni et al. (2002) Papineni, K., S. Roukos, T. Ward, and W.-J. Zhu (2002, July).  BLEU: A Method for Automatic Evaluation of Machine Translation.  In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics (ACL 2002), Philadelphia, Pennsylvania, USA, pp. 311–318. Association for Computational Linguistics.
  * Peng et al. (2023) Peng, K., L. Ding, Q. Zhong, L. Shen, X. Liu, M. Zhang, Y. Ouyang, and D. Tao (2023).  Towards making the most of chatgpt for machine translation.  arXiv preprint arXiv:2303.13780.
  * Petreski and Hashim (2023) Petreski, D. and I. C. Hashim (2023).  Word embeddings are biased. but whose bias are they reflecting?  AI & SOCIETY 38(2), 975–982.
  * Post (2018) Post, M. (2018, October).  A call for clarity in reporting BLEU scores.  In Proceedings of the Third Conference on Machine Translation: Research Papers, Belgium, Brussels, pp. 186–191. Association for Computational Linguistics.
  * Prates et al. (2020) Prates, M. O., P. H. Avelar, and L. C. Lamb (2020).  Assessing gender bias in machine translation: a case study with google translate.  Neural Computing and Applications 32, 6363–6381.
  * Rei et al. (2020) Rei, R., C. Stewart, A. C. Farinha, and A. Lavie (2020).  Comet: A neural framework for mt evaluation.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2685–2702.
  * Rescigno et al. (2020) Rescigno, A. A., E. Vanmassenhove, J. Monti, and A. Way (2020).  A case study of natural gender phenomena in translation a comparison of google translate, bing microsoft translator and deepl for english to italian, french and spanish.  Computational Linguistics CLiC-it 2020 359.
  * Saunders et al. (2020) Saunders, D., R. Sallis, and B. Byrne (2020, December).  Neural machine translation doesn’t translate gender coreference right unless you make it.  In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, Barcelona, Spain (Online), pp. 35–43.
  * Savoldi et al. (2021) Savoldi, B., M. Gaido, L. Bentivogli, M. Negri, and M. Turchi (2021).  Gender bias in machine translation.  Transactions of the Association for Computational Linguistics 9, 845–874.
  * Sellam et al. (2020) Sellam, T., D. Das, and A. P. Parikh (2020).  Bleurt: Learning robust metrics for text generation.  In Proceedings of ACL.
  * Stafanovičs et al. (2020) Stafanovičs, A., T. Bergmanis, and M. Pinnis (2020).  Mitigating gender bias in machine translation with target gender annotations.  In Proceedings of the Fifth Conference on Machine Translation, pp. 629–638.
  * Sun et al. (2021) Sun, T., K. Webster, A. Shah, W. Y. Wang, and M. Johnson (2021).  They, them, theirs: Rewriting with gender-neutral english.  arXiv preprint arXiv:2102.06788.
  * Sutskever et al. (2014) Sutskever, I., O. Vinyals, and Q. V. Le (2014).  Sequence to sequence learning with neural networks.  In Proceedings of Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems, Montreal, Quebec, Canada, pp. 3104–3112.
  * Toews (2023) Toews, R. (2023, Sep).  The next generation of large language models.
  * Toral (2019) Toral, A. (2019).  Post-editese: an exacerbated translationese.  In Proceedings of Machine Translation Summit XVII: Research Track, pp. 273–281.
  * Vanmassenhove et al. (2021) Vanmassenhove, E., C. Emmery, and D. Shterionov (2021).  Neutral rewriter: A rule-based and neural approach to automatic rewriting into gender neutral alternatives.  In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 8940–8948.
  * Vanmassenhove et al. (2018) Vanmassenhove, E., C. Hardmeier, and A. Way (2018).  Getting gender right in neural machine translation.  In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3003–3008.
  * Vanmassenhove and Monti (2021) Vanmassenhove, E. and J. Monti (2021).  gender-it: An annotated english-italian parallel challenge set for cross-linguistic natural gender phenomena.  In Proceedings of the 3rd Workshop on Gender Bias in Natural Language Processing, pp. 1–7.
  * Vanmassenhove et al. (2021) Vanmassenhove, E., D. Shterionov, and M. Gwilliam (2021, April).  Machine translationese: Effects of algorithmic bias on linguistic complexity in machine translation.  In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, Online, pp. 2203–2213. Association for Computational Linguistics.
  * Vanmassenhove et al. (2019) Vanmassenhove, E., D. Shterionov, and A. Way (2019, August).  Lost in translation: Loss and decay of linguistic richness in machine translation.  In Proceedings of Machine Translation Summit XVII Volume 1: Research Track, Dublin, Ireland, pp. 222–232. European Association for Machine Translation.
  * Vargha (2021) Vargha, D. (2021).  Hungarian is a gender-neutral language, it has no gendered pronouns, so google translate automatically chooses the gender for you. here is how everyday sexism is..  Twitter post.  March 20, 2021.
  * Vaswani et al. (2017) Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin (2017).  Attention is all you need.  In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett (Eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 5998–6008.
  * Wang et al. (2023) Wang, L., C. Lyu, T. Ji, Z. Zhang, D. Yu, S. Shi, and Z. Tu (2023).  Document-level machine translation with large language models.  arXiv preprint arXiv:2304.02210.
  * Wei et al. (2022) Wei, J., X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. (2022).  Chain-of-thought prompting elicits reasoning in large language models.  Advances in Neural Information Processing Systems 35, 24824–24837.
  * Zhang et al. (2019) Zhang, T., V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi (2019).  Bertscore: Evaluating text generation with bert.  arXiv preprint arXiv:1904.09675.
  * Zhao et al. (2018) Zhao, J., T. Wang, M. Yatskar, V. Ordonez, and K.-W. Chang (2018, June).  Gender bias in coreference resolution: Evaluation and debiasing methods.  In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), New Orleans, Louisiana, pp. 15–20. Association for Computational Linguistics.

###### Notes

  1. 1 Accessed on June 12, 2023.
  2. 2 Re-translated on June 12 2023 by the author.
  3. 3 Although one could argue, that this implicitness makes it all the more dangerous.
  4. 4 For a more comprehensive exploration of this topic, we refer to Birhane (2021) which provides valuable insights into the ethical dimensions of algorithmic systems and their impact while advocating for a more critical analysis in the field of machine learning and data science.
  5. 5 We acknowledge that the gender-neutral alternative ‘le personnel enseignant’ is not the most literal translation of ‘teacher’ and furthermore implies multiple people (plural). However, in settings where ‘teacher’ is used in a generic way, it can be considered a valid gender-neutral alternative.
  6. 6 We will use the term non-binary gender to refer to all genders that fall outside the binary female/male distinction.
  7. 7 https://translate.google.com/
  8. 8 https://www.deepl.com
  9. 9 https://chat.openai.com/
  10. 10 www.openai.com
  11. 11 https://www.reuters.com article accessible via https://shorturl.at/dewzZ
  12. 12 https://www.nytimes.com article accessible via https://shorturl.at/ruRS3
  13. 13 https://arxiv.org/
  14. 14 We ought to note that this is an ill-defined term.
  15. 15 Similar to ’low-resource languages’ the terms ’distant languages’ is not very well-defined. In the paper itself, they refer to ‘European’ vs ‘Asian’ languages.
  16. 16 https://translate.google.com/
  17. 17 https://www.deepl.com/en/translator
  18. 18 https://transmart.qq.com/
  19. 19 Bengali has two pronouns, one of which is the more respectful form and they found it to be used mainly in sentences featuring a male pronoun.
  20. 20 https://www.microsoft.com/nl-nl/translator/
  21. 21 A few exceptions are nouns such as ’waitress’ vs ’waiter’ or ‘actress’ vs ‘actor’.
  22. 22 We added a version that is not included in gENder-IT using the ‘’ which has been recommended by Baiocco et al. (2023) as a non-gender-specific ending for Italian.
  23. 23 We accessed ChatGPT to generated these translations between May 30, 2023 and June 5, 2023.
  24. 24 There is no crystallized approach to gender-neutral endings in Italian as to date.