# Entropy production and correlation spreading in the interaction between
particle detector and thermal baths

Hao Xu<EMAIL_ADDRESS>Center for Gravitation and Cosmology, College of
Physical Science and Technology, Yangzhou University,
180 Siwangting Road, Yangzhou City, Jiangsu Province 225002, China  Shanghai
Frontier Science Center for Gravitational Wave Detection, Shanghai Jiao Tong
University, Shanghai 200240, China  Si Yu Chen  Center for Gravitation and
Cosmology, College of Physical Science and Technology, Yangzhou University,
180 Siwangting Road, Yangzhou City, Jiangsu Province 225002, China

###### Abstract

Entropy production is the key to the second law of thermodynamics, and it is
well defined by considering a joint unitary evolution of a system $S$ and a
thermal environment $E$. However, due to the diversity of the initial state
and Hamiltonian of the system and environment, it is hard to evaluate the
characterisation of entropy production. In the present work, we propose that
the evolution of $S$ and $E$ can be solved non-perturbatively in the framework
of Gaussian quantum mechanics (GQM). We study the entropy production and
correlation spreading in the interaction between Unruh-DeWitt-like particle
detector and thermal baths, where the particle detector is set to be a
harmonic oscillator and the thermal baths are made of interacting and
noninteracting Gaussian states. We can observe that the entropy production
implies quantum recurrence and shows periodicity. In the case of interacting
bath, the correlation propagates in a periodic system and leads to a revival
of the initial state. Our analysis can be extended to any other models in the
framework of GQM, and it may also shed some light on the AdS/CFT
correspondence.

_Introduction_.—Entropy production, which is non-negative and becomes zero
only in the case where the thermodynamic process is reversible, serves as the
key to the second law of thermodynamics. It predicts there is a lack of time
reversal in the irreversibility thermodynamic process coming from the time
arrow Batalhao2015 . Its research has also become one of the most important
topics in modern physics Landi:2020bsq .

However, the exact definition of entropy production was, surprisingly, not
universal. The natural candidate for the definition of entropy, namely, the
von Neumann entropy, is invariant under unitary evolution, thus it can not
reflect the irreversibility in the evolution of the system. Studies on entropy
production were often based on specific models and different assumptions,
until it was addressed in esposito2010  by considering a joint unitary
evolution of a system $S$ and an environment $E$ under specific conditions. A
general formula of entropy production was defined, and this method was also
later used to clarify the long controversial Landauer’s principle in Reeb2013
. The setup for the joint evolution was based on four assumptions: (i) both
the system $S$ and environment $E$ are described by Hilbert spaces, (ii) the
environment is initially in a thermal state
$\rho_{E}=e^{-\beta\hat{H}_{E}}/\text{Tr}(e^{-\beta\hat{H}_{E}})$, where
$\hat{H}_{E}$ is the Hamiltonian of the environment and $\beta$ is the inverse
temperature, (iii) system and environment are uncorrelated initially,
$\rho_{SE}=\rho_{S}\otimes\rho_{E}$, (iv) the process proceeds by unitary
evolution $\rho^{\prime}_{SE}=U\rho_{SE}U^{\dagger}$. If all the four
assumptions are satisfied, Landauer’s principle holds and the entropy
production $\zeta$ can be defined as

$\zeta=I(S^{\prime}:E^{\prime})+D(\rho^{\prime}_{E}||\rho_{E}),$ (1)

where the
$I(S^{\prime}:E^{\prime}):=S(\rho^{\prime}_{S})+S(\rho^{\prime}_{E})-S(\rho^{\prime}_{SE})$
is the mutual information (MI) that measures the correlation between the
system and environment, while the $S(\bullet)$ denotes the von Neumann entropy
and $\rho^{\prime}_{S/E}:=\text{Tr}_{E/S}[\rho^{\prime}_{SE}]$ is reduced
density matrix. The
$D(\rho^{\prime}_{E}||\rho_{E}):=\text{Tr}(\rho^{\prime}_{E}\ln{\rho^{\prime}_{E}})-\text{Tr}(\rho^{\prime}_{E}\ln{\rho_{E}})$
is the relative entropy measures the displacement of the environment from the
initial thermal equilibrium state. The non-negativity of entropy production
follows from the fact that both $I(S^{\prime}:E^{\prime})$ and
$D(\rho^{\prime}_{E}||\rho_{E})$ are non-negative.

On the other hand, although the setup in esposito2010 ; Reeb2013  are general,
it is also minimal thus can not provide further information about the
thermodynamical process. It is natural for us to ask how these two terms (MI
and relative entropy) contribute to entropy production. It was often held
that, compared with the MI, the relative entropy is negligible for large
thermal reservoirs li2017 ; strasberg2017 ; chen2017 ; engelhardt2018 ;
manzano2018 ; you2018 ; li2019 ; santos2019 ; bera2019 . However, recently
Ptaszynski2019  it was found that the MI is strongly bounded from above by the
Araki-Lieb inequality araki1970 . The entropy production could be time-
extensive, so the entropy production could also predominantly come from the
relative entropy that measures the displacement of the environment from
equilibrium state $\rho_{E}$. In Ptaszynski2019  a numerical analysis of
fermions with noninteracting resonant levels was also presented to support
this point.

Although in some cases the relative entropy starts to dominate after the early
stage of evolution, this does not mean we have fully understood the
characterization and assessment of entropy production. Ultimately, the details
of entropy production are determined by the initial state and the Hamiltonian
of the total system. The initial state only needs to satisfy assumptions
(i)-(iii), and there are no restrictions of the Hamiltonian. In principle,
there could be infinite kinds of choices. To make matters worse, the
Hamiltonian of $S$ may be very different from $E$, making it almost impossible
to solve the evolution of the joint system. In many cases, we can only use
perturbative method to solve the equation order by order.

However, perturbative methods naturally have disadvantages. Long time
interaction, strong coupling, or high-average-energy exchange processes will
make the perturbative expansion invalid. One way to address these issues is to
impose a few restrictions on the choice of Hamiltonian and the initial state
of the total system, so that the time evolution can be solved non-
perturbatively. This can be achieved by using the symplectic formalism in
Gaussian quantum mechanics (GQM), which maps the Schrödinger equation in the
infinite-dimensional Hilbert space onto a linear evolution of finite-
dimensional phase space Adesso2007 ; Adesso2014 . GQM provides a way for us to
solve a class of models completely. It has been widely used in quantum optics
Olivares2012 , and recently it is also applied to analogue gravity systems
Bruschi2013a , relativistic quantum field theory (QFT) Bruschi2013 ; Brown2012
; Bruschi2015 ; Bruschi2020  and quantum thermodynamics Kerstjens2018 . Using
GQM, we can solve the time evolution of the total system and gain access to
the states at any moment of time for finite-size, finite-time, and strong-
coupling regimes.

In the present work we will apply GQM to study the entropy production and
correlation spreading in the interaction between Unruh-DeWitt-like particle
detector and thermal baths. The traditional Unruh-DeWitt detector, which is
often described by an accelerating qubit coupled to the vaccum state of QFT
DeWitt1979 , is used to characterize the Unruh effect Unruh1976 ; Unruh1983 .
However, in our case we set the particle detector to be an inertial harmonic
oscillator, which represents the system $S$ in the interaction. The
environments $E$ made of interacting or noninteracting baths are described by
the thermal Gaussian states. The joint evolution of $S$ and $E$ can be solved
in the framework of GQM. Since the $S$ is a detector, the interaction can also
be viewed as a measurement process. In principle, we can obtain the
information of $E$ by studying the feedback obtained by $S$. In the present
work we concentrate on the interaction and correlation between the particle
detector and thermal states.

Our results show that the entropy production is not dominated by the relative
entropy as in Ptaszynski2019  when considering the long-time interactions, but
implies quantum recurrence and shows periodicity. The statement of entropy
production predominantly comes from relative entropy is only part of the
story, and there is still much to study about the evolution of the whole
system. In the case of interacting bath, the correlation propagates in a
periodic system and lead to a revival of the initial state. Similar examples
of this phenomenon can also be found in AdS/CFT (Anti-de Sitter/Conformal
Field Theory) correspondence Maldacena:1997re ; Witten:1998qj , where the
evolution of CFT after a quantum quench can be explained by the left and
right-moving quasiparticles Cardy:2014rqa . Henceforth we adopt the natural
unit system, setting $c=\hbar=k_{B}=1$.

_Gaussian quantum mechanics_.— In this section we give a brief review of the
GQM. The reader is referred to Adesso2007 ; Adesso2014  for a detailed
introduction to this topic. For a bosonic system with $N$ modes, we can form a
vector of operators as

$\boldsymbol{x}=(\hat{q}_{1},\hat{p}_{1},\cdots,\hat{q}_{N},\hat{p}_{N})^{{T}},$
(2)

where the canonical quadrature operators
$\hat{q}_{i}=(\hat{a}_{i}+\hat{a}_{i}^{{\dagger}})/\sqrt{2}$ and
$\hat{p}_{i}=i(\hat{a}_{i}^{{\dagger}}-\hat{a}_{i})/\sqrt{2}$ satisfy
$[\hat{q}_{i},\hat{p}_{i}]=i\delta_{ij}$. The phase space formed by the above
vector is a symplectic space. In GQM we only consider the Gaussian states that
are fully described by the first and second moments of their quadratures, and
the Gaussian states stay Gaussian under the evolution with any time-dependent
_quadratic_ Hamiltonian. For simplicity we set the first moment of the vector
to be zero and characterize the state of the system as the $2N\times 2N$
covariance matrix $\boldsymbol{\sigma}$ whose entries are given by

$\sigma_{ab}=\langle\hat{x}_{a}\hat{x}_{b}+\hat{x}_{b}\hat{x}_{a}\rangle=\text{Tr}[\rho(\hat{x}_{a}\hat{x}_{b}+\hat{x}_{b}\hat{x}_{a})].$
(3)

Any combined state of two systems $A$ and $B$ has the form

$\sigma_{AB}=\begin{pmatrix}\boldsymbol{\sigma}_{A}&\boldsymbol{\gamma}_{AB}\\\
\boldsymbol{\gamma}_{AB}^{T}&\boldsymbol{\sigma}_{B}\par\end{pmatrix},$ (4)

where $\boldsymbol{\sigma}_{A}$ and $\boldsymbol{\sigma}_{B}$ are the states
of $A$ and $B$ respectively, and the $\boldsymbol{\gamma}_{AB}$ represents the
correlation between them. For systems under unitary transformation $U$
generated by a time-dependent quadratic Hamiltonian $H(t)$, it corresponds to
a linear symplectic transformation on the phase space:
$\boldsymbol{x}\rightarrow\boldsymbol{x}^{\prime}=U^{{\dagger}}\boldsymbol{x}U=S\boldsymbol{x}$.
Similarly the covariance matrix transforms as
$\boldsymbol{\sigma}\rightarrow\boldsymbol{\sigma}^{\prime}=S\boldsymbol{\sigma}S^{T}$.

Since $H(t)$ is quadratic, we can write it as
$H(t)=\boldsymbol{x}^{T}F(t)\boldsymbol{x}$, where $F(t)$ is a $2N\times 2N$
phase space matrix. The symplectic evolution matrix $S(t)$ generated by this
Hamiltonian obeys a Schrödinger-like equation:

$\frac{dS(t)}{dt}=\Omega F_{s}(t)S(t),$ (5)

where $F_{s}=F+F^{T}$ and $\Omega:=\bigoplus^{N}_{i=1}\begin{pmatrix}0&1\\\
-1&0\par\end{pmatrix}$ is the symplectic form. Once we have the formula of the
total Hamiltonian, we can solve the above differential Schrödinger-like
equation numerically. The inital condition of the equation is $S(0)=\openone$.
We can decompose the continuous time $T$ into many steps and solve the
equation step by step. An example of the codes, both in Matlab and Python,
which are developed by the authors of Kerstjens2018 , are available in
Kerstjens2018a .

In quantum statistical physics we always need to do the tracing over $\rho$ to
obtain the expectation value of operator $\hat{O}$ as
$O=\text{Tr}[\rho\hat{O})]$. In GQM we only care about the first and second
moments, and the information of the states has been encoded in the covariance
matrix, so we can always represent the expectation values of other physical
quantities as the function of covariance matrix. For example, the average
energy reads $\langle H\rangle=\frac{1}{2}\text{Tr}[F\sigma]$. If we
diagonalize the system’s Hamiltonian as
$F=\frac{1}{2}\text{diag}(\omega_{1},\omega_{1},\omega_{2},\omega_{2},\ldots)$,
the thermal state will be

$\sigma_{T}=\bigoplus^{N}_{i=1}\begin{pmatrix}\nu^{(th)}_{i}&0\\\
0&\nu^{(th)}_{i}\par\end{pmatrix},\qquad\nu^{(th)}_{i}=\frac{e^{\omega_{i}/T}+1}{e^{\omega_{i}/T}-1},$
(6)

which corresponds to a tensor product state with a diagonal density matrix
given by

$\bigotimes_{i}\frac{2}{\nu^{(th)}_{i}+1}\sum_{n_{i}=0}^{\infty}\left(\frac{\nu^{(th)}_{i}-1}{\nu^{(th)}_{i}+1}\right)|n_{i}\rangle\langle
n_{i}|,$ (7)

where $|n_{i}\rangle$ denotes the number state in the Fock space. Each mode
with frequency $\omega_{i}$ is a Gaussian state in thermal equilibrium at a
temperature $T$, and the average photons number

$\bar{n}_{i}=\frac{\nu^{(th)}_{i}-1}{2}=\frac{1}{e^{\omega_{i}/T}-1}.$ (8)

Since we already obtain the density matrix (7), we can calculate the the von
Neumann entropy directly. Finally we have

$S(\sigma)=\sum^{N}_{i=1}f(\nu_{i}^{(th)}),$ (9)

where

$f(\nu)=\frac{\nu+1}{2}\log{\frac{\nu+1}{2}}-\frac{\nu-1}{2}\log{\frac{\nu-1}{2}}.$
(10)

Of course, not all Gaussian states are thermal, nevertheless, there always
exists a symplectic matrix to diagonalize any Gaussian state. Given any joint
state of $S$ and $E$, we can calculate the von Neumann entropy directly, thus
obtaining the MI Demarie2012 . On the other hand, from Reeb2013  we know the
entropy production can also be written as
$\zeta=\beta\left(H(\rho^{\prime}_{E})-H(\rho_{E})\right)-(S(\rho_{S})-S(\rho^{\prime}_{S}))$,
evaluating the energy change of $E$ and the entropy change of $S$, we can also
obtain the relative entropy $D(\rho^{\prime}_{E}||\rho_{E})$.

_Models_.— First we choose a harmonic oscillator with frequency $\omega$ to be
the particle detector (system $S$), and we set a one-dimensional free massless
scalar QFT in a _cavity_ to be the environment $E$ so that we can obtain
discrete modes by applying an infrared cutoff and Dirichlet boundary
condition. The Hamiltonian of $E$ reads
$\hat{H}_{E}=\sum_{j=1}^{N}\omega_{j}\hat{a}^{{\dagger}}_{j}\hat{a}_{j}$,
where $N$ is the number of modes we include, $\omega_{j}=j\pi/L$ and $L$ is
the scale of the cavity. The interaction Hamiltonian
$\hat{H}_{\text{int}}(t)=\lambda\chi(t)\mu\phi[x]$, in which $\lambda$ is
coupling constant, $\chi(t)$ is the switching function that controls the
interaction, $\mu=\hat{a}_{s}+\hat{a}_{s}^{{\dagger}}$ is the monopole moment
of the particle detector, and
$\phi[x]=\sum_{j=1}^{N}\left(\hat{a}_{j}u_{j}\left[x\right]+\hat{a}^{\dagger}_{j}u_{j}^{*}\left[x\right]\right)$
is the field operator at the position of the particle detector in the cavity,
where $u_{j}(x)=\sin(j\pi x)$ is the basis and the normalization constant is
absorbed in the coupling constant. This formula of $\hat{H}_{\text{int}}$
means the particle detector interacts with all modes of the environment.

Note that the above Hamiltonian is written in the form of creation-
annihilation operators, not position-momentum operators as in
$\boldsymbol{x}$. Nevertheless, we can always do the transformation to obtain
the right form of $F(t)$.

Since we are considering non-interacting cavity QFT, the $F$ for $\hat{H}_{E}$
is naturally diagonalized. If at $t=0$ the initial state the particle detector
is in temperature $T_{S}$ and environment is in temperature $T_{E}$, the
covariance matrix can be written as the direct sum of (6). Setting the smooth
and compactly-supported switching function as Kerstjens2018

$\chi(t)=\begin{cases}0&t<0\\\ \frac{1}{2}-\frac{1}{2}\tanh\cot\frac{\pi
t}{\delta}&0\leq t<\delta\\\ 1&\delta\leq t<\tau-\delta\\\
\frac{1}{2}+\frac{1}{2}\tanh\cot\frac{\pi(t-\tau)}{\delta}&\tau-\delta\leq
t<\tau\\\ 0&t>\tau\end{cases},$ (11)

we can solve the eq.(5) numerically and obtain $S(t)$.

Figure 1: Evolution during the interaction between $S$ and $E$ for the cavity
QFT. We set $N=30$, $L=1$, $T_{S}=1$, $T_{E}=3$, $\lambda=0.05$,
$\delta=0.1\tau$, $\tau=150$, and the frequency of the system $\omega=3\pi$.
The green solid line is the effective temperature of $S$, corresponding to the
right vertical axis, while the other curves are the MI, corresponding to the
left vertical axis: the blue solid line is the MI between $S$ and $E$, and the
black dashed line is the MI between $S$ and mode 3 of the cavity QFT. Figure
2: Evolution during the interaction between $S$ and $E$ for the cavity QFT.
The parameters are set as in the FIG.1. The black solid line is the entropy
production. The blue solid line the MI between $S$ and $E$, and the red solid
line is the relative entropy $D(\rho^{\prime}_{E}||\rho_{E})$.

In FIG.1 and FIG.2 we present some numerical results. We set $N=30$, $L=1$,
$T_{S}=1$, $T_{E}=3$, $\lambda=0.05$, $\delta=0.1\tau$, $\tau=150$. The
authors of Kerstjens2018  refer to $\delta$ as the ramp-up time of the
interaction. Note that the switching on and off could not be too fast
($\delta$ could not be too small), otherwise the transition rate of the system
$S$ would encounter divergence Sriramkumar:1994pb ; Louko:2006zv . The
frequency $\omega$ of the system $S$ is $3\pi$, which is equal to the third
modes $\omega_{3}$ (mode 3) of the cavity QFT. The numerical result shows that
the convariance matrix of the system $S$ effectively has the form of
$\begin{pmatrix}\nu&0\\\ 0&\nu\par\end{pmatrix}$, thus according to (6) we can
define the effective temperature of the mode as
$T_{\text{eff}}=\omega\ln{\frac{\nu-1}{\nu+1}}$. In FIG.1 we find the
effective temperature of $S$ (green solid line), does not reach a steady state
of thermal equilibrium as one would expect. The effective temperature of $S$,
and also the MI between $S$ and $E$ (blue solid line), are both oscillating.
This MI is almost indistinguishable from the one between the $S$ and the mode
3 (black dashed line). This is consistent with the result obtained by using
the perturbative method that indicates the system can only be affected by a
small neighbourhood of frequency $\omega$ of the cavity QFT Xu2021 . An
analogous phenomenon in classical mechanics was reported in Smith2008 .

In FIG.2 we study the entropy production $\zeta$, $I(S^{\prime}:E^{\prime})$
and $D(\rho^{\prime}_{E}||\rho_{E})$. Similar to FIG.1, these three quantities
are all oscillating. In the early stage of evolution, entropy production
mainly comes from MI. After that, MI decays and the relative entropy
increases, meaning the displacement of the environment from equilibrium begins
to become significant while the correlation between $S$ and $E$ decreases.
After reaching the peak, the relative entropy decreases rapidly, and entropy
production also begins to decline soon. Meanwhile, the effective temperature
is also decreasing. Eventually they all reach their lowest point and then a
new cycle begins. Our results show that neither MI nor relative entropy has
always provided a dominant contribution to the entropy production throughout
the evolutionary process. They are each becoming dominant terms at different
times.

From the above analysis we can conclude that although the cavity QFT contains
various modes, the interaction is largely determined by the mode with the same
frequency of $S$. The effective temperature of the system, and also the
$I(S^{\prime}:E^{\prime})$ and $D(\rho^{\prime}_{E}||\rho_{E})$, are all
oscillating and implying quantum recurrence.

Next, we consider the environment $E$ in which all modes share the same
frequency with the system $S$, but there is an interaction between adjacent
modes of the environment. This model is also used as the heat bath of quantum
Otto engine in quantum thermodynamics Kerstjens2018 . In continuous and
massless limit it also reduces to the CFT with central charge $c=1$
Chapman:2018hou ; Camargo:2020yfv . We not only want to know how the
correlation between system and environment evolves, but also how it propagates
in the environment. The Hamiltonian of the environment reads

$\hat{H}_{E}=\sum_{i=1}^{N}\frac{\omega}{2}\left(\hat{p}_{i}^{2}+\hat{q}_{i}^{2}\right)+\sum_{i=1}^{N}\alpha\hat{q}_{i}\hat{q}_{i+1},$
(12)

where $N$ is the number of modes, $\omega$ is the frequency, and $\alpha$ is
the position-position coupling constant. Applying the periodic boundary
condition $\hat{q}_{N+1}=\hat{q}_{1}$, $E$ becomes an oscillator chain.

The interaction Hamiltonian can be set as

$\hat{H}_{\text{int}}=\lambda\chi(t)\hat{q}_{s}\sum_{i\in\\{\text{int}\\}}\hat{q}_{i}$
(13)

where $\lambda$ and $\chi(t)$ are still the coupling constant and switching
function respectively, $q_{s}$ is the position operator of system $S$, and
{int} denotes the modes that $S$ interacts with. For simplicity we only
consider $S$ interacts with one mode of the $E$, and without loss of
generality, we set $i=1$.

Note that now due to the position-position coupling constant $\alpha$, the
phase space matrix $F$ corresponding to the $\hat{H}_{E}$ is no longer
diagonal. The corresponding covariance matrix of the thermal state with
temperature $T_{E}$ is also not diagonal. Nevertheless, we can symplectically
diagonalize the phase space matrix to obtain the corresponding diagonal
covariance matrix and then transform it back to the physical basis to find the
thermal state of $E$.

Figure 3: Evolution during the interaction between $S$ and $E$ for the
oscillator chain. We set $\omega=4$, $N=25$, $T_{S}=1$, $T_{E}=10$,
$\alpha=\lambda=0.15$, $\delta=0.1\tau$ and $\tau=500$. The green solid line
is the effective temperature of the system, corresponding to the right
vertical axis, while the other curves are the MI, corresponding to the left
vertical axis: the blue solid line is the MI between the $S$ and $E$, black
dashed line is the MI between $S$ and mode 1, and the red dotted line is the
MI between mode 1 and the rest modes. Figure 4: Evolution during the
interaction between $S$ and $E$ for the oscillator chain. The parameters are
set as in the FIG.3. The black, blue, red, green, and deep blue lines
correspond to the MI between System- mode1, mode 1-mode 2, mode 2-mode 3, mode
3-mode 4 and mode 4-mode 5 respectively. Figure 5: Evolution during the
interaction between $S$ and $E$ for the oscillator chain. The parameters are
set as in the FIG.3. The black solid line is the entropy production. The blue
solid line the MI between $S$ and $E$, and the red solid line is the relative
entropy $D(\rho^{\prime}_{E}||\rho_{E})$.

In FIG.3-5 we present some numerical examples of the evolution during the
interaction between $S$ and $E$. We set $\omega=4$, $N=25$, $T_{S}=1$,
$T_{E}=10$, $\alpha=\lambda=0.15$, $\delta=0.1\tau$ and $\tau=500$. In FIG. 3
we can find the effective temperature of the system (green line) keeps
increasing until it is equivalent to the environment’s temperature. The value
of the temperature stabilizes for some time, then starts to decrease, reaches
a lower point, and then increases again. If we extend the interaction time, we
will find the same pattern: the temperature of the system remains stable for a
period of time, then experiences an oscillation, after which it returns to
stability and waits for the next disturbance. On the other hand, we can also
observe that the MI between the system and the environment (blue solid line)
shows similar characteristics. The MI between the system and mode 1 also
increases at the beginning, but later decreases until the arrival of the
disturbance. The increase in MI at the beginning of evolution comes from the
fact that we turn on the interaction, while the decrease is due to the
internal interactions of the environment that make correlation propagate in
the environment. On the other hand, we can find that the MI between mode 1 and
the rest is always oscillating.

We take a closer look at the correlation of the environment in FIG.4, where we
consider the correlations of adjacent modes. In addition to the MI between
system-mode 1, we also present the evolution of the MI between mode 1-mode 2,
mode 2-mode 3, mode 3-mode 4 and mode 4-mode 5. Similar steps can be performed
for all oscillators. Due to the periodic boundary condition
$\hat{q}_{N+1}=\hat{q}_{1}$, they are equivalent to the MI of mode 1-mode $N$,
mode $N-1$-mode $N$, mode $N-2$-mode $N-1$, mode $N-3$-mode $N-2$, etc. In
FIG.4, we can observe how the correlation propagates through the oscillator
chain. Starting from mode 1, the correlation between the adjacent modes
increases and then decays one by one. After some time we can find the MI
oscillates again, because the correlation from the other side arrives. The
correlation makes a complete circle in both directions back to mode 1, causing
the oscillation of all the physical quantities, then the correlation between
the $S$ and mode 1 decays and the evolution starts a new cycle again.

In FIG.5 we study the evolution of the entropy production,
$I(S^{\prime}:E^{\prime})$ and $D(\rho^{\prime}_{E}||\rho_{E})$. The behavior
is also regular. In the early stage of evolution, entropy production mainly
comes from MI, since $S$ and $E$ have just interacted. As the interaction
continues, the effective temperature of $S$ increases until it is
approximately the same as the temperature of $E$. On the other hand, the MI
decreases, meaning the correlation between $S$ and $E$ decays, while relative
entropy increase instead, implying the displacement of the environment from
equilibrium begins to become significant. When the effective temperature is
stable, the relative entropy reaches the highest, while the MI is the lowest.
This state will continue for some time, depending on the number of the
oscillator in the chain, until the correlation comes back around a full circle
and causes a new period. If we set $N=1$, the curves would be exactly like the
FIG.2. For a larger value of $N$, the temperature and other quantities, such
as entropy production, $I(S^{\prime}:E^{\prime})$ or
$D(\rho^{\prime}_{E}||\rho_{E})$, would stay stable for a longer time, because
the correlation would take a longer time to travel through the entire
oscillator chain.

_Final remarks_.—In summary, the interaction between $S$ and $E$ creats
correlations. If the mode in $E$ that interacts with $S$ also interact with
other modes in $E$, the correlation will propagate in the $E$, depending on
the coupling strength, the number of modes, boundary condition, etc. In the
free scalar cavity QFT, the mode that shares the same frequency with the $S$
does not interact with other modes in the cavity QFT, thus the correlation
does not propagate to other modes. For the interacting oscillate chain with
periodic boundary condition, the $S$ interacts with one mode of the $E$, and
the correlation propagates in circles like a kind of “flow” so the physical
quantities oscillate regularly. Neither MI nor relative entropy has always
provided a dominant contribution to the entropy production. In our case the
entropy production is mainly from MI in the early stage of the interaction and
from relative entropy when the system is in “thermal equilibrium”. In fact, if
we add more internal interactions to the environment or include more modes of
the environment in the interaction Hamiltonian, the evolutionary process of
the whole system will become more complicated. There is still much to study
about the evolution of the whole system.

Note that the entropy production has also been studied in other bosonic
Gaussian systems, such as the quantum Brownian motion (QBM) model for the
harmonic potential Pucci2013 ; Colla:2021mbe . Similar oscillations of the
thermodynamic quantities are observed and explained as a phenomenon of
Poincaré recurrences. This brings no surprise since we are all considering
finite dynamical periodic systems. However, there are two main diferences
between our second model and (QBM) model. The first one is that in our second
model the bath $B$ (12) contains the interaction term
$\sum_{i=1}^{N}\alpha\hat{q}_{i}\hat{q}_{i+1}$ between the adjacent
oscillators, while in the QBM cases the bath is non-interacting. The second
one is we set our interaction (13) to be “ _local_ ” because the $S$ interacts
with only one oscillator, while the interaction in QBM cases is “ _global_ ”
since $S$ interacts with all the oscillators. In our case the interaction
“perturbs” the whole system locally and the correlation is created between $S$
and $\hat{q}_{1}$, then it passes through the total oscillator chain as in
FIG.4 to make circles in both directions due to the interaction term
$\sum_{i=1}^{N}\alpha\hat{q}_{i}\hat{q}_{i+1}$.

Another important motivation to study our second model is that it captures
some important features of CFT, and it may shed some light on the AdS/CFT
correspondence Maldacena:1997re ; Witten:1998qj . If we consider the
discretized Hamiltonian on a lattice with interaction between adjacent
oscillators as (12), taking $N\rightarrow\infty$ and massless limit, we can
obtain the CFT with central charge $c=1$. Thus once we discrete the CFT, we
can apply the GQM to study it. This method has been used to investigate the
complexity for thermofield double states Chapman:2018hou  and the purification
of (1+1)-dimensional CFT Camargo:2020yfv . In our work we consider the local
interaction between the system $S$ and bath $B$, which is similar to the
thermalization and revivals after a local quantum quench in CFT Cardy:2014rqa
, where J. Cardy used the quasiparticle picture to explain the quantum
recurrence. The left and right-moving quasiparticles are initially entangled,
being emitted at $t=0$ and thereafter moving semiclassically. In a periodic
system, an oppositely moving pair of particles will meet again and this should
lead to a revival of the initial state. The quasiparticle picture is very
similar with the correlation propogating in our work, and in our previous work
Xu2021  we indeed studied the (de-)excition of the QFT in this $S$ and $B$
interaction. Our model and Gaussian techniques may be able to help us to get a
better understanding of the time evolution of CFT and the nature of
holography. Other models in GQM may also bring new perspectives to the study
of AdS/CFT. In recent years quantum information theory has become inter-
disciplinary and has been used to study gravity and high energy physics
1409.1231 ; 1411.7041 ; 1503.06237 . There have also been some recent
discussions on the connection between quantum gravity and Gaussianity of
states Howl2021 . GQM may also be helpful in these fields. However, we also
need to emphasize that it appears quite a step to really gain something new
about the full AdS/CFT correspondence. There are still many problems to be
studied in this field and we hope to get some interesting results in the near
future.

Finally, as we previously suggested, the interaction between $S$ and $E$ can
also be regarded as the measurement of $S$ to $E$. Measurements are of great
importance in physics. In principle, all measurable observations are also
representative of all possible properties of the system. The study of the
evolution of joint systems may also help us to understand the nature of the
measurements themselves Xu2022  and facilitate the development of practical
models to detect or simulate different systems. We will continue to study this
direction in our future work.

###### Acknowledgements.

Hao Xu thanks Karen V. Hovhannisyan for useful discussion on the numerical
method. He also thanks the Natural Science Foundation of the Jiangsu Higher
Education Institutions of China (No.20KJD140001) for funding support.

## Data Availability Statement

The datasets generated during and/or analysed during the current study are
available from the corresponding author on reasonable request.

## References

  * (1) T. B. Batalhão, A. M. Souza, R. S. Sarthour, I. S. Oliveira, M. Paternostro, E. Lutz, R. M. Serra, “Irreversibility and the Arrow of Time in a Quenched Quantum System”, Phys. Rev. Lett. 115 (2015) 190601.
  * (2) G. T. Landi and M. Paternostro, “Irreversible entropy production, from quantum to classical”, Rev. Mod. Phys. 93, 035008 (2021).
  * (3) M. Esposito, K. Lindenberg, and C. Van den Broeck, “Entropy production as correlation between system and reservoir”, New J. Phys. 12, 013013 (2010).
  * (4) D. Reeb, M. M. Wolf, “An Improved Landauer Principle With Finite-Size Corrections”, New J. Phys. 16 (2014) 103011.
  * (5) P. Strasberg, G. Schaller, T. Brandes, and M. Esposito, “Quantum and Information Thermodynamics: A Unifying Framework Based on Repeated Interactions”, Phys. Rev. X 7, 021003 (2017).
  * (6) H.-B. Chen, G.-Y. Chen, and Y.-N. Chen, “Thermodynamic description of non-Markovian information flux of nonequilibrium open quantum systems”, Phys. Rev. A 96, 062114 (2017).
  * (7) S.-W. Li, “Production rate of the system-bath mutual information”, Phys. Rev. E 96, 012139 (2017).
  * (8) G. Engelhardt and G. Schaller, “Maxwell’s demon in the quantum-Zeno regime and beyond”, New J. Phys. 20, 023011 (2018).
  * (9) G. Manzano, J. M. Horowitz, and J. M. R. Parrondo, “Quantum Fluctuation Theorems for Arbitrary Environments: Adiabatic and Nonadiabatic Entropy Production”, Phys. Rev. X 8, 031037 (2018).
  * (10) Y.-N. You and S.-W. Li, “Entropy dynamics of a dephasing model in a squeezed thermal bath”, Phys. Rev. A 97, 012114 (2018).
  * (11) S.-W. Li, “The correlation production in thermodynamics”, Entropy 21, 111 (2019).
  * (12) J. P. Santos, L. C. Céleri, G. T. Landi, and M. Paternostro, “The role of quantum coherence in non-equilibrium entropy production”, npj Quantum Inf. 5, 23 (2019).
  * (13) M. N. Bera, A. Rieral, M. Lewenstein, Z. Baghali Khanian, and A. Winter, “Thermodynamics as a Consequence of Information Conservation”, Quantum 3, 121 (2019).
  * (14) K. Ptaszynski, M. Esposito, “Entropy Production in Open Systems: The Predominant Role of Intra-Environment Correlations”, Phys. Rev. Lett. 123 (2019) 200603.
  * (15) H. Araki and E. H. Lieb, “Entropy inequalities”, Commun. Math. Phys. 18, 160 (1970).
  * (16) G. Adesso and F. Illuminati, “Entanglement in continuous-variable systems: recent advances and current perspectives”, J. Phys. A: Math. Theor. 40 (2007) 7821.
  * (17) G. Adesso, S. Ragy, A. R. Lee, “Continuous variable quantum information: Gaussian states and beyond”, Open Syst. Inf. Dyn. 21, 1440001 (2014).
  * (18) S. Olivares, “Quantum Optics in the Phase Space”, Eur. Phys. J. Special Topics 203 (2012) 3.
  * (19) D. E. Bruschi, N. Friis, I. Fuentes and S. Weinfurtner, “On the robustness of entanglement in analogue gravity systems,” New J. Phys. 15, 113016 (2013).
  * (20) E. G. Brown, E. Martín-Martínez, N. C. Menicucci, R. B. Mann, “Detectors for probing relativistic quantum physics beyond perturbation theory”, Phys. Rev. D. 87 (2013) 084062.
  * (21) D. E. Bruschi, A. R. Lee, I. Fuentes, “Time Evolution Techniques for Detectors in Relativistic Quantum Information”, J. Phys. A: Math. Theor. 46 (2013) 165303.
  * (22) D. E. Bruschi, M. Perarnau-Llobet, N. Friis, K. V. Hovhannisyan, M. Huber, “The thermodynamics of creating correlations: Limitations and optimal protocols,” Phys. Rev. E 91, 032118 (2015).
  * (23) D. E. Bruschi, B. Morris, I. Fuentes “Thermodynamics of relativistic quantum fields confined in cavities,” Phys. Lett. A 384:25, 126601 (2020).
  * (24) A. Pozas-Kerstjens, E. G. Brown, K. V. Hovhannisyan, “A quantum Otto engine with finite heat baths: energy, correlations, and degradation”, New J. Phys. 20, 043034 (2018).
  * (25) B. S. DeWitt, in “General Relativity: An Einstein Centenary Survey”, edited by S. W. Hawking, and W. Israel Cambridge University Press, Cambridge, England, pp. 680–745, (1979).
  * (26) W. G. Unruh, “Notes on Black-Hole Evaporation”, Phys. Rev. D 14 (1976) 870.
  * (27) W. G. Unruh, R. M. Wald, “What Happens When an Accelerating Observer Detects a Rindler Particle”, Phys. Rev. D 29 (1984) 1047.
  * (28) J. M. Maldacena, “The Large N limit of superconformal field theories and supergravity,” Adv. Theor. Math. Phys. 2, 231 (1998).
  * (29) E. Witten, “Anti-de Sitter space and holography,” Adv. Theor. Math. Phys. 2, 253 (1998).
  * (30) J. Cardy, “Thermalization and Revivals after a Quantum Quench in Conformal Field Theory,” Phys. Rev. Lett. 112, 220401 (2014).
  * (31) A. Pozas-Kerstjens, E. G. Brown, K. V. Hovhannisyan, Computational appendix of “A quantum Otto engine with finite heat baths: energy, correlations, and degradation”,  Zenodo 847182 (2017)..
  * (32) T. F. Demarie, “Pedagogical introduction to the entropy of entanglement for Gaussian states”, arXiv:1209.2748.
  * (33) L. Sriramkumar and T. Padmanabhan, “Response of finite time particle detectors in noninertial frames and curved space-time,” Class. Quant. Grav. 13, 2061-2079 (1996).
  * (34) J. Louko and A. Satz, “How often does the Unruh-DeWitt detector click? Regularisation by a spatial profile,” Class. Quant. Grav. 23, 6321-6344 (2006).
  * (35) H. Xu, Y. C. Ong, M. Yung “Landauer’s Principle in Qubit-Cavity Quantum Field Theory Interaction in Vaccum and Thermal States”, arXiv:2109.08391.
  * (36) S. T. Smith, R. Onofrio,“Thermalization in Open Classical Systems With Finite Heat Baths”, Eur. Phys. J. B 61 (2008) 271.
  * (37) S. Chapman, J. Eisert, L. Hackl, M. P. Heller, R. Jefferson, H. Marrochio and R. C. Myers, “Complexity and entanglement for thermofield double states,” SciPost Phys. 6, no.3, 034 (2019).
  * (38) H. A. Camargo, L. Hackl, M. P. Heller, A. Jahn, T. Takayanagi and B. Windt, “Entanglement and complexity of purification in ( 1+1 )-dimensional free conformal field theories,” Phys. Rev. Res. 3, no.1, 013248 (2021).
  * (39) L. Pucci, M. Esposito and L. Peliti “Entropy production in quantum Brownian motion,” J. Stat. Mech. (2013) P04005.
  * (40) A. Colla and H. P. Breuer, “Entropy Production and the Role of Correlations in Quantum Brownian Motion,” Phys. Rev. A 104, 052408 (2021).
  * (41) D. Harlow, “Jerusalem Lectures on Black Holes and Quantum Information”, Rev. Mod. Phys. 88 (2016) 015002.
  * (42) A. Almheiri, X. Dong, D. Harlow, “Bulk Locality and Quantum Error Correction in AdS/CFT”, JHEP 04 (2015) 163.
  * (43) F. Pastawski, B. Yoshida, D. Harlow, J. Preskill, “Holographic Quantum Error-Correcting Codes: Toy Models for the Bulk/Boundary Correspondence”, JHEP 06 (2015) 149.
  * (44) R. Howl, V. Vedral, D. Naik, M. Christodoulou, C. Rovelli, A. Iyer, “Non-Gaussianity as a Signature of a Quantum Theory of Gravity”, PRX Quantum 2 (2021) 010325.
  * (45) H. Xu, S.Y. Chen, Y. C. Ong, to appear soon.


11institutetext:  11email<EMAIL_ADDRESS>
11email<EMAIL_ADDRESS>
# Extending Challenge Sets to Uncover Gender Bias in Machine Translation

Impact of Stereotypical Verbs and Adjectives

Jonas-Dario Troles  Ute Schmid

###### Abstract

Human gender bias is reflected in language and text production. Because state-
of-the-art machine translation (MT) systems are trained on large corpora of
text, mostly generated by humans, gender bias can also be found in MT. For
instance when occupations are translated from a language like English, which
mostly uses gender neutral words, to a language like German, which mostly uses
a feminine and a masculine version for an occupation, a decision must be made
by the MT System. Recent research showed that MT systems are biased towards
stereotypical translation of occupations. In 2019 the first, and so far only,
challenge set, explicitly designed to measure the extent of gender bias in MT
systems has been published. In this set measurement of gender bias is solely
based on the translation of occupations. In this paper we present an extension
of this challenge set, called _WiBeMT_ , with gender-biased adjectives and
adds sentences with gender-biased verbs. The resulting challenge set consists
of over $70,000$ sentences and has been translated with three commercial MT
systems: DeepL Translator, Microsoft Translator, and Google Translate. Results
show a gender bias for all three MT systems. This gender bias is to a great
extent significantly influenced by adjectives and to a lesser extent by verbs.

###### Keywords:

Gender Bias Machine Translation Challenge Set.

##  1 Introduction

The problem of unfair and biased models has been recognized as an important
problem for many applications of machine learning [6]. The source of
unfairness typically is based on a biases in the training data. In many
domains, unfairness is caused by sampling biases. In machine translation (MT),
however, the main source of unfairness is due to historical or social biases
when there is a misalignment between the world as it is and the values or
objectives to be encoded and propagated in a model [12, 1]. One source of
imbalance in natural language is the association of specific occupations with
gender. Typically, occupations in more technical domains as well as
occupations with high social status are associated with male gender [3].

In natural language processing (NLP), gender bias has been investigated for
word embeddings [1]. In analogy puzzles such as “man is to king as woman is to
$x$” generated with _word2vec_ 111https://code.google.com/archive/p/word2vec/
yields $x=queen$ while for “man is to computer programmer as woman is to $x$”,
the output is $x=homemaker$. Only few publications exist that directly address
gender bias and MT. Although gender bias is particularly relevant for
translations into gender-inflected languages, for instance from English to
German, when biased models can result in translation errors [10]. The English
sentence “The doctor told the nurse that she had been busy.”, a human
translator would resolve the co-reference of ‘she’ to the doctor, correctly
translating it to ‘die Ärztin’. However, a neural machine translation model
(NMT) trained on a biased dataset, in which most doctors are male might
incorrectly default to the masculine form, ‘der Arzt’. [5] investigated the
prediction of age and gender of a text’s author before and after translation.
They found that _Bing Translator_ , _DeepL Translator_ , and _Google
Translate_ all skew the predictions to be older and more masculine, which
shows that NMT systems and the expression of gender interact.

According to Saunders et al. [10], the first systematic analysis of gender
bias in MT is from Stanovsky et al. [11]. They introduce _WinoMT_ which is the
first challenge set explicitly designed to quantify gender bias in MT systems.
Furthermore, they introduce an automatic evaluation method for eight different
languages. Evaluating six MT systems, they found a strong preference for
masculine translations in all eight gender-inflected languages. The above
example sentence, where ’doctor’ has been errouneousely translated into its
male German form (_Arzt_) is one of the $3,888$ sentences used in _WinoMT_.

_WinoMT_ focuses solely on the translation of occupations to evaluate gender
bias. In this paper, we present the extended data set _WiBeMT_ to uncover
gender bias in neural machine translation systems. Gender stereotypical
occupations are augmented by gender stereotypical adjectives and verbs and we
investigate how congruent and incongruent combinations impact translation
accuracy of the NMT systems DeepL, Microsoft and Google Translate. In the next
section, the original _WinoMT_ data set is introduced together with our
extensions. Afterwards, results for the three NMT systems based on 70,686
sentences are presented, followed by a discussion and an outlook.

##  2 Constructing the Extended Challenge Set WiBeMT

To construct a more diverse challenge set, we extend _WinoMT_ , respectively
its core data base _WinoBias_. Therefore, we identify verbs and adjectives of
high stereotypicality with respect to gender. A gender score is determined by
the cosine similarity of these words to a list of gender specific words. To
calculate the similarity different pretrained word embeddings are used, where
each of these words is represented by a vector. With the resulting most
feminine and masculine adjectives the original sentences of _WinoBias_ are
extended. Furthermore, new sentences are created combining occupations with
gender stereotypical verbs.

###  2.1 WinoBias and its Extension

_WinoMT_ is based on two previous challenge sets, _Winogender_ [9] and
_WinoBias_ [13]. Both were introduced to quantify gender bias in co-reference
resolution systems. Since _WinoBias_ constitutes $81.5\%$ of _WinoMT_ , we use
it as basis for our extension. In total _WinoBias_ consists of $3,168$
sentences, of which $1,582$ are feminine and $1,586$ are masculine sentences.
The sentences are based on $40$ occupations. An example sentence of _WinoBias_
involving a cleaner and a developer is:

  * •

WinoBias sentence: The cleaner hates the developer because she always leaves
the room dirty.

  * •

DeepL translation: Die Reinigungskraft hasst den Entwickler, weil sie das
Zimmer immer schmutzig hinterlässt.

DeepL fails to correctly translate developer to the female inflection die
Entwicklerin, but instead favors the stereotypical male inflection der
Entwickler.

The _WinoBias_ set is constructed such that each sentence is given in a
stereotypical and an anti-stereotypical version. Stereotypical in this context
means that the gender of the pronoun matches the predominant gender in the
sentences occupation. The example sentence above is the anti-stereotypical
version for the occupational noun ’developer’, where the stereotypical version
contains a ’he’ instead of a ’she’.

We argue that a measurement of gender bias solely based on the translation of
occupational nouns does not do justice to the complexity of language.
Therefore, we want to diversify the given approach by taking gender-
stereotypical adjectives and verbs into account. This is realized in two
steps: First, _WinoBias_ sentences are extended such that each occupational
noun is preceded with an adjective which is congruent or incongruent with
respect to the gender stereotypical interpretation of the occupation. For
instance, a developer might be _eminent_ (male) or _brunette_ (female). By
adding feminine and masculine adjectives to each WinoBias sentence, we create
a new subset of extended WinoBias sentences.

Second, we create completely new sentences based on feminine and masculine
verbs. One example with a feminine verb being: “The X dances in the club”, and
one with a masculine verb: “The X boasts about the new car.”. Those base
sentences are then extended with $99$ occupations from [13, 9, 4], resulting
in a new subset of verb sentences. All $100$ occupational nouns used in
_WinoBias_ and in the new verb sentences are listed in Table 1. After
concatenating both subsets, the complete extended gender-bias challenge set
consists of $70,686$ sentences. Overall, $100$ occupations are used in the
set, $42$ gender-verbs, and $20$ gender-adjectives.

We hypothesize that gender-stereotypical verbs and adjectives influence the
gender of the translation of the occupational nouns, when translating from
English to the gender-inflected language German:

  * •

Hypothesis 1

    * Sentences with a feminine verb result in significantly more translations into the female inflection of an occupation than sentences with a masculine verb.

  * •

Hypotheses 2

    * 2a

WinoBias sentences extended with a feminine adjective result in significantly
more translations into the female inflection of an occupation than original
WinoBias sentences (without a preceded adjective).

    * 2b

WinoBias sentences extended with a masculine adjective result in significantly
more translations into the masculine inflection of an occupation than original
WinoBias sentences (without a preceded adjective).

Table 1: All $100$ occupations, used in this thesis with the corresponding
percentage of women in the occupation in the US.

Occupation | % | Occupation | % | Occupation | % | Occupation | %
---|---|---|---|---|---|---|---
electrician | 2 | analyst | 41* | tailor | 75 | dancer | –
carpenter | 3 | physician | 41 | attendant | 76* | doctor | –
firefighter | 3 | surgeon | 41 | counselor | 76 | economist | –
plumber | 3 | cook | 42 | teacher | 78* | educator | –
construction- worker | 4 | chemist | 43 | planner | 79 | engineer | –
laborer | 4* | manager | 43* | librarian | 80 | examiner | –
mechanic | 4* | supervisor | 44* | psychologist | 80 | gardener | –
driver | 6* | salesperson | 48* | assistant | 85* | geologist | –
machinist | 6 | photographer | 49 | cleaner | 89* | hygienist | –
painter | 9 | bartender | 53 | housekeeper | 89* | inspector | –
mover | 18* | dispatcher | 53 | receptionist | 89 | investigator | –
sheriff | 18 | judge | 53 | nurse | 90* | mathematician | –
developer | 20* | artist | 54 | paralegal | 90 | officer | –
programmer | 20 | designer | 54 | dietitian | 92 | pathologist | –
guard | 22* | baker | 60 | hairdresser | 92 | physicist | –
architect | 25 | pharmacist | 60 | nutritionist | 92 | practitioner | –
farmer | 25 | accountant | 62 | secretary | 93 | professor | –
chief | 28 | auditor | 62 | administrator | – | sailor | –
dentist | 34 | editor | 63 | advisor | – | scientist | –
paramedic | 34 | writer | 63* | appraiser | – | soldier | –
athlete | 35 | author | 64 | broker | – | specialist | –
lawyer | 36 | instructor | 65 | CFO | – | student | –
janitor | 37 | veterinarian | 68 | collector | – | surveyor | –
musician | 37 | cashier | 71 | conductor | – | technician | –
CEO | 39* | clerk | 72* | CTO | – | therapist | –

###  2.2 Finding Gender-Stereotypical Verbs and Adjectives

To determine gender-stereotypicality of adjectives and verbs, large
collections of these word types have been scored with respect to their
similarity to a list of gender specific words given by [1] As input we used a
list of $3.250$ verbs from
patternbasedwriting.com222www.patternbasedwriting.com offers teaching
materials for primary school children. and a combined list of $4.889$
adjectives from patternbasedwriting.com and from Garg et al. [4].

Calculation of gender score is based on word embeddings and the cosine-
similarity, following the work of [1, 4]. Their, similarity of the words to
the pronouns “she” and “he” has been used. We extended scoring using a longer
list of feminine and masculine words such as “mother”, “uncle”, “menopause” or
“semen” to enhance robustness of the gender-score. This list, containing $95$
feminine and $108$ masculine words, is taken from [1], who used it for their
debiasing methods of word embeddings.

Two families of _word embeddings_ were used: two pre-trained _fastText_
333Downloaded from: https://fasttext.cc/docs/en/english-vectors.html from [7]
and two pre-trained versions of _GloVe_ 444Downloaded from:
https://nlp.stanford.edu/projects/glove/ from [8]. All four word embeddings
have a vector size of 300 dimensions. Table 2 gives an overview of all word
embeddings and their training data.

Table 2: Summary of the origin of training corpora and their corresponding
size for each word embedding.

| Size [billion]
---|---
corpora |  fastText-small |  fastText-large |  GloVe-small |  GloVe-large
Wikipedia 2014 | — | — | 1.6 | —
Gigaword 5 | — | — | 4.3 | —
Wikipedia meta-page 2017 | 9.2 | — | — | —
Statmt.org News | 4.2 | — | — | —
UMBC News | 3.2 | — | — | —
Common Crawl | — | 630 | — | 840

_Cosine Similarity_ is a measure of similarity between two normalized and non-
zero vectors and can take values in the range between -1 and 1. Equation 1
shows the calculation of the cosine similarity between the vectors
$\mathbf{a}$ and $\mathbf{b}$ of two words:

$\mathbf{similarity}=\cos(\theta)=\frac{\mathbf{a}\cdot\mathbf{b}}{\|\mathbf{a}\|\|\mathbf{b}\|}=\frac{\sum\limits_{i=1}^{n}a_{i}b_{i}}{\sqrt{\sum\limits_{i=1}^{n}a_{i}^{2}}\sqrt{\sum\limits_{i=1}^{n}b_{i}^{2}}}\\\
$ (1)

where $a_{i}$ and $b_{i}$ are components of vector $\mathbf{a}$ and
$\mathbf{b}$ respectively.

Since word embeddings inherit to some extend the meaning of words, it is
possible to use the cosine similarity as a measure for the similarity in
meaning or, furthermore, the relationships between words. This enables
mathematical operations on the vectors representing words such that:
$cos(\overrightarrow{brunette}\cdot\overrightarrow{her})\geq
cos(\overrightarrow{brunette}\cdot\overrightarrow{him})$ becomes true for
“brunette” and other gender-biased adjectives. If the feminine-gender value
($cos(\overrightarrow{brunette}\cdot\overrightarrow{her})$) is then subtracted
from the masculine-gender value
($cos(\overrightarrow{brunette}\cdot\overrightarrow{him})$) the resulting
single float value indicates whether a word is gender-biased in the word
embedding with which the cosine-similarity was computed.

The total gender-score is the sum of eight single scores resulting from the
combination of the four different word embeddings with the cosine similarity
with “she” and “he” and with the list of feminine and masculine words. Since
different word embeddings vary in the strength of the inherited gender-bias
and all word embeddings should have equal impact on the overall score, the
interim results were normalized to fit a range between $-1$ and $1$:

$x^{\prime}=a+\frac{(x-\min(x))(b-a)}{\max(x)-\min(x)}$ (2)

To validate this procedure to determine a gender score for adjectives and
verbs, the same method has been applied to the occupations given in Table 1.
The gender-score for these occupations shows a strong correlation to the
percentage of women working in each given occupation (see Figure 1).

Figure 1: Scatter plot of the $99$ single word occupations with the percentage
of women in the profession y-axis and the gender score on the x-axis.

Gender-score has been calculated for all verbs and adjectives for which word
embeddings existed. They were sorted ascending from the most negative – and
therefore most feminine – score value, to the most positive, i.e., most
masculine, value.

Of the $3,250$ _verbs_ , $3,210$ could be ranked ($med=0.151$, $std=0.165$)

After sorting the verbs by their gender-score, they were analyzed for their
suitability to build a sentence in which person $P$ actively does action $A$.
The gender score of the $21$ selected feminine verbs ranges from $-0.772$ to
$-0.233$ and of the masculine verbs from $0.733$ to $0.445$:

  * •

Feminine verbs: crochet, sew, accessorize, bake, embroider, primp, gossip,
shriek, dance, undress, milk, giggle, marry, knit, twirl, wed, flirt, allure,
shower, seduce, kiss;

  * •

Masculine verbs: draft, tackle, swagger, trade, brawl, reckon, preach,
sanction, build, boast, gamble, succeed, regard, retire, chuck, overthrow,
rev, resign, apprehend, appoint, fool.

#### Adjectives:

Of the $5,441$ _adjectives_ , $4,762$ could be ranked ($med=0.189$m
$std=0.142$). After calculating the gender-score for the $4,762$ adjectives,
beginning with the most feminine, respectively, most masculine, each adjective
was tested for its suitability considering the extension of existing WinoBias
sentences. Words that semantically could not be combined with occupations were
discarded (examples: hormonal, satin, luminous, philosophical, topographical).
Furthermore, adjectives like “pregnant” – which only apply to persons with a
uterus – were also discarded. The gender score of the $10$ selected feminine
adjectives ranges from $-0.600$ to $-0.205$ and of the masculine adjectives
from $0.654$ to $0.480$:

  * •

Feminine adjectives: sassy, perky, brunette, blonde, lovely, vivacious, saucy,
bubbly, alluring, married;

  * •

Masculine adjectives: grizzled, affable, jovial, suave, debonair, wiry,
rascally, arrogant, shifty, eminent.

###  2.3 Translation of _WiBeMT_ and Evaluation Design

To test the extent to which machine translation systems inherit a gender-bias,
three different services were tested with _WiBeMT_ : _Google Translate_ ,
_Microsoft Translator_ , and _DeepL Translator_. All three NMT systems were
accessed via their API, and all translation processes took place in July 2020.

To test our hypotheses that adjectives and verbs significantly influence the
gender in translations of NMT systems, translations from English to German
have to be categorized as either (correctly) feminine, masculine, neutral, or
wrong. [11] use an automated method in the form of different morphological
analyzers such as _spaCy_ to determine the gender of the occupation in the
translated sentence. While this method is convenient for an automated approach
that other researchers can use with different data, it also suffers from a
certain degree of inaccuracy. To measure the accuracy of their automated
evaluation method, [11] compared the automated evaluations with a random
sample of samples evaluated by native speakers. They found an average
agreement of 87% between automated and native speaker evaluations. To evaluate
NMT systems with respect to _WiBeMT_ , we preferred to augment automated
evaluation by “manual” evaluation to gain higher accuracy.

Evaluating the gender of all translations is based on a nested list, we refer
to as _classification-list_ , and a set of rules for automated evaluation and
manual evaluation for all remaining ambiguous translations. The
classification-list contains four sublists for each occupation: a sublist of
correct feminine translations, a sublist of correct masculine translations, a
sublist of correct neutral translations, and a sublist of inconclusive or
wrong translations. The gender of the translated occupation $Occ$ is then
classified by checking in which sublist $Occ$ is listed and the gender is
labeled as the sublist’s category.

To control classifications for possible errors, $N=665$ (1%) of the WinoBias
sentences extended by adjectives were manually controlled for each translation
system (in total $N=1,995$ sentences). Not a single one was miscategorized. Of
the verb-sentences translations, even 5% were manually controlled by the
authors, and here also, not a single one of the $618$ translations was
miscategorized.

##  3 Results

After the creation of the _WiBeMT_ challenge set and the translation of all
$70,686$ sentences with each NMT system, the translations were categorized as
_feminine_ , _masculine_ , _neutral_ , or _inconclusive / wrong_. The latter
will be referred to as “wrong”. Due to these discrete categories the $Chi^{2}$
test of independence will be used for all statistical tests. As the extended
_WinoBias_ sentences include a pronoun that defines the gender, the results
considering these data can be divided into _true_ and _false_ , and _feminine_
and _masculine_ translations. On the other hand, the verb sentences do not
include any cue for a _correct_ gender in the translation. Therefore, they are
just analyzed for feminine and masculine translations. The calculated
_feminine-ratio_ ($\%TFG$) results from all feminine-translations divided by
the sum of feminine- and masculine-translations. The calculated _correct-
gender-ratio_ ($\%TCG$) results from all translations with correct gender
divided by the sum of all translations with correct and incorrect gender. All
$Chi^{2}$ were, if necessary, Bonferroni corrected. First, the results for the
verb-sentences; Second, the results for the extended WinoBias sentences; And
third, further results are presented.

###  3.1 Hypothesis 1: Verb Sentences

The statistical tests regarding Hypothesis 1 yielded mixed results, depending
on which NMT system is looked at. Of the $2.079$ sentences with a feminine
verb DeepL translated the occupations in $242$ ($11.9\%$) sentences into the
female gender ($\%TFG$) compared to Microsoft Translator with $149$ ($7.5\%$),
and Google Translate with $120$ ($6.0\%$). Of the $2.079$ sentences with a
masculine verb DeepL (DL) translated the occupations in $135$ ($6.6\%$)
sentences into the female gender compared to Microsoft Translator (MS) with
$104$ ($5.2\%$), and Google Translate (GT) with $109$ ($5.4\%$). For DeepL and
Microsoft Translate the difference becomes significant with
$\Delta\%TFG_{DL}=5.3\%$, $Chi^{2}_{DL}=33.7$ and $p_{DL}<0.001$, and
$\Delta\%TFG_{MS}=2.3\%$, $Chi^{2}_{MS}=8.4$ and $p_{MS}=0.004$. For Google
Translate the difference does not become significant. As two of three NMT
systems inherit a gender bias that is influenced as expected by verbs, meaning
that translations of sentences with feminine verbs result in a significantly
higher $\%TFG$ than translations of sentences with masculine verbs, $H1$ is
accepted. Figure 2 gives an overview of the translations to the female gender
($\%TFG$) for all three NMT systems and the two categories of verb sentences.

Figure 2: Percentage of translations to female gender ($\%TFG$) of occupations
in verb sentences, organized by the category of verbs.

###  3.2 Hypothesis 2: Extended WinoBias Sentences

Both hypotheses assume that adjectives that inherit a gender-bias in word
embeddings also influence the gender of translations from English to German.
While the results confirm the assumptions of Hypothesis H$2_{a}$, they also
yield unexpected results for Hypothesis H$2_{b}$. Of the 3,168 original
WinoBias sentences DeepL translated $1,259$ ($41.7\%$), Microsoft Translator
translated $1,041$ ($35.8\%$), and Google Translate translated $976$
($33.2\%$) to the female inflection of the occupation. These percentages are
used as the baseline to test, whether feminine and masculine adjectives skew
the outcome of translations into the expected direction.

Of the sentences extended with a feminine adjective DeepL translated $49.5\%$,
Microsoft Translator translated $42.5\%$, and Google Translate translated
$40.1\%$ to the female inflection. The difference to the percentage of
translations to the female inflection of the original WinoBias sentences
becomes significant for all three NMT systems: $\Delta\%TFG_{DL}=7.8\%$,
$Chi^{2}_{DL}=66.3$ and $p_{DL}<0.001$; $\Delta\%TFG_{MS}=6.7\%$,
$Chi^{2}_{MS}=49.0$ and $p_{MS}<0.001$; and $\Delta\%TFG_{GT}=6.9\%$,
$Chi^{2}_{GT}=53.8$ and $p_{GT}<0.001$. Therefore Hypothesis H$2_{a}$ is
accepted.

Of the sentences extended with a feminine adjective DeepL translated $44.6\%$,
Microsoft Translator translated $39.3\%$, and Google Translate translated
$37.9\%$ to the female inflection. The difference to the percentage of
translations to the female inflection of the original WinoBias sentences
becomes significant for all three NMT systems: $\Delta\%TFG_{DL}=2.9\%$,
$Chi^{2}_{DL}=9.1$ and $p_{DL}=0.008$; $\Delta\%TFG_{MS}=3.5\%$,
$Chi^{2}_{MS}=13.3$ and $p_{MS}<0.001$; and $\Delta\%TFG_{GT}=4.7\%$,
$Chi^{2}_{GT}=25.1$ and $p_{GT}<0.001$. While all differences are significant,
they contradict our assumption that preceding masculine adjectives to WinoBias
sentences results in less translations to the female inflection. Instead the
preceded masculine adjectives have the opposite effect. Therefore, Hypothesis
H$2_{b}$ is rejected. Figure 3 gives an overview of the translations to the
female gender ($\%TFG$) for all three NMT systems and the three categories of
sentences. Table 3 lists all numbers of different translation categories for a
deeper insight.

Figure 3: Percentage of translations to female gender ($\%TFG$) of occupations in all three types of EWB sentences: feminine adjective, masculine adjective, and no adjective. Table 3: Numbers of the categorizations of the gender of translations of extended WinoBias sentences sorted by the gender-class of the adjective. |  | NMT system
---|---|---
adjective gender | translation category | DeepL | Microsoft | Google
feminine |  true feminine false masculine true masculine false feminine neutral wrong |  13,341 1,553 13,833 1,751 550 652 |  11,352 3,405 13,814 1,382 440 1,287 |  10,177 4,593 13,312 1,826 472 1,300
masculine |  true feminine false masculine true masculine false feminine neutral wrong |  12,497 2,378 14,425 1,028 483 869 |  8,966 3,405 11,943 963 346 6,057 |  8,724 4,533 12,128 1,450 447 4,398
no adjective |  true feminine false masculine true masculine false feminine neutral wrong |  1,162 324 1,434 97 75 76 |  916 503 1,365 125 77 182 |  812 652 1,313 164 53 174

###  3.3 Influence of Gender-Stereotypical Verbs and Adjectives on
Translations

Our findings show that gender stereotypical verbs and adjectives influence
gender bias in translations of NMT systems. In the following, we discuss our
results in more detail.

_Hypothesis 1:_ In general, the results from verb sentences differ drastically
from the ones of the extended WinoBias sentences, with far fewer sentences
being translated to their feminine gender. The percentage of sentences
translated to their feminine gender ($\%TFG$) in the verb sentences ranges
from $5.2\%$ for masculine verb sentences translated by Microsoft Translator,
to $11.9\%$ for feminine verb sentences translated by DeepL. In comparison to
that, $\%TFG$ ranges from $33.2\%$ in original WinoBias sentences without
adjective translated by Google Translate, to $49.5\%$ in extended WinoBias
sentences with feminine adjectives translated by DeepL Translator.

Two reasons could be responsible for the low $\%TFG$. Firstly, the generic
masculine in German: As mostly the masculine gender is used to address all
genders, this must be present in the training data of all three NMT systems.
Therefore, they tend to use the male inflection whenever the gender bias for a
specific occupation does not outweigh the bias by the generic masculine.
Secondly, the verb-sentences lack a gender pronoun like “her” or “him”, which
urges the NMT system to decide which gender would be correct in the
translation. This probably leads to the strong bias towards male inflections
in translations.

To check whether the bias induced by occupations and the generic masculine
outweighed an existing bias of verbs, we analyzed the data of the verb
sentences again, looking at the different groups of occupations555With the
calculated gender score we split the list of occupations in three equally
sized categories (each $n=33$): feminine, neutral and masculine. For all three
NMT systems the $\%TFG$ for verb sentences with neutral and masculine
occupations was below 2% regardless of the stereotypical feminine verbs. The
$\%TFG$ in sentences with female occupations was 25.1% in DeepL, 19.0% in
Microsoft Translator and 17.6% in Google Translate. All differences to
sentences with neutral and masculine occupations were significant with
p-values below $0.001$ and $Chi^{2}$ values reaching from $352$ to $256$.

_Hypotheses 2a & 2b:_ As gender bias works both ways in word embeddings,
meaning that words can be stereotypical feminine or stereotypical masculine,
we assumed that, depending on their gender-score derived from word embeddings,
feminine adjectives would skew translations of NMT systems more often to their
feminine gender and vice versa that masculine adjectives would skew
translations more often to their masculine gender. This assumption was also
supported by the findings of [11], who preceded “handsome” to occupations of
sentences which would be correct, if translated to their masculine gender, and
“pretty” to occupations of sentences which would be correct if translated to
their feminine gender. With this measure, they could improve the accuracy of
NMT systems and reduce gender bias.

Contrary to the prior assumptions, adjectives preceded to occupations led to
significantly more translations with feminine gender, regardless of their
gender-score derived from word embeddings. Finding that masculine adjectives
also lead to more translations with feminine gender does not only result in
the rejection of H$2_{b}$, but also weakens the acceptance of H$2_{a}$.
Therefore, we introduce a new Hypothesis H$2_{c}$: “WinoBias sentences
extended with a feminine adjective result in significantly more translations
into the female inflection of an occupation than WinoBias sentences extended
with a masculine adjective.”.

The difference between the percentage of translations to the female inflection
of WinoBias sentences extended with feminine and WinoBias sentences extended
with masculine adjectives becomes significant for all three NMT systems:
$\Delta\%TFG_{DL}=4.9\%$, $Chi^{2}_{DL}=147.6$ and $p_{DL}<0.001$;
$\Delta\%TFG_{MS}=3.2\%$, $Chi^{2}_{MS}=59.1$ and $p_{MS}<0.001$; and
$\Delta\%TFG_{GT}=2.2\%$, $Chi^{2}_{GT}=29.2$ and $p_{GT}<0.001$. Therefore,
Hypothesis H$2_{c}$ is accepted which strengthens H$2_{a}$, as stereotypical
feminine adjectives preceded to occupational nouns lead to significantly more
translations to the female gender than stereotypical masculine adjectives
preceded to occupational nouns.

###  3.4 Influence of Adjectives on Correct Gender in Translations

The comparison of all three NMT systems was not one of the main research
questions. Nevertheless, the extent of gender bias in the NMT systems can be
of interest to users. Therefore, and because of the surprising findings
considering Hypothesis $2_{b}$ a short comparison is presented in the
following paragraphs.

Figure 4: Bar plot of the percentage of translations into the correct gender
($\%TCG$) of original WinoBias sentences and extended WinoBias sentences.

To better understand our results regarding hypotheses H$2_{a}$, H$2_{b}$ and
H$2_{c}$ we plotted percentage of translations with the correct gender
($\%TCG$) organized by NMT systems and the type of sentence: original WinoBias
sentence (no adjective), with masculine adjective extended WinoBias sentence
(M adjective), and with feminine adjective extended WinoBias sentence (F
adjective). Additionally, we split each type of sentences into sentences with
male pronouns (M pronouns) and sentences with female pronouns (F pronouns).
Figure 4 shows the resulting plot.

The results are quite astonishing: while the $\%TCG$ slightly decreases for
sentences with male pronouns and any adjective ($1\leq\Delta\%TCG\leq 5$) it
drastically improves for sentences with female pronouns and any adjective
($12\leq\Delta\%TCG\leq 13$). Considering that sentences with female and male
pronouns are equally prevalent in original WinoBias sentences and extended
WinoBias sentences, it shows that preceding an occupational noun with any
adjective likely improves the overall percentage of translations to the
correct gender inflection. This reflects the findings of [11], who preceded
“handsome” and “pretty” to occupational nouns of WinoMT sentences. With this
measure, they could improve the accuracy hence $\%TCG$ of NMT systems and
reduce gender bias. Our findings give more insight to this results: Stanovsky
et al. [11] very likely could have added either “handsome” or “pretty” to all
sentences, regardless of their pronoun and would nonetheless have been able to
record a higher accuracy.

Furthermore, Figure 4 shows that DeepL Translator performed best when it comes
to $\%TCG$. Google Translate and Microsoft Translator again show more similar
results, but Google Translate performed notably worse than Microsoft
Translator. With the least discrepancy in $\%TCG$ between sentences with a
feminine pronoun and sentences with a masculine pronoun in all three
conditions (feminine adjective, masculine adjective, no adjective) DeepL
Translator, therefore, inherits the lowest gender bias.

##  4 Conclusions and Further Work

The three neural machine translation systems evaluated with respect of their
gender bias are black boxes, in so far as the architecture is not publicly
available and – even more important – it is not transparent on what data these
systems are trained. It is most likely that the gender bias in all three
systems is inherited from the data used for training and their use of word
embeddings.

To give a closer look on gender-bias of the NMT systems DeepL, Microsoft, and
Google Translate, an extension of the _WinoMT_ challenge set – the first
challenge set designed to measure gender bias in NMT systems – has been
presented. While _WinoMT_ relies solely on the gender of the translation of
occupations, our extended set _WiBeMT_ includes gender-adjectives and gender-
verbs. Thereby, a more detailed assessment of gender bias has been possible.
The number of sentences in our challenge set is, with over $70,000$ sentences,
nearly $20$ times as large as the original WinoMT challenge set. This makes it
less prone to overfitting when used to evaluate or reduce gender bias in NMT
systems.

We could show that adjectives do significantly influence the gender of
translated occupations. Against the hypotheses, feminine as well as masculine
adjectives skew the translations of NMT systems to more translations with the
feminine gender. Nevertheless, feminine adjectives still produce significantly
more translations with the feminine gender than masculine adjectives do.

All three NMT systems prefer translations to the generic masculine when no
pronoun defines a correct gender for the translation. Despite this preference,
$5.7\%$ to $9.3\%$ of all verb sentences are translated to their feminine
gender by the MT systems. This can mostly be attributed to a gender bias in
translating occupations, as our results regarding verb sentences and
occupation categories show. The effect of gender-verbs on the gender of
translations only became significant in DeepL and Microsoft Translator and was
smaller than the effect of the occupations gender-categories. It can be
assumed that Google Translate only performed better on the verb sentences
task, as it generally has the strongest tendency to translate sentences to
their masculine gender.

Surprisingly, gender-adjectives drastically improve the overall accuracy of
all three NMT systems when it comes to the correct gender in translations.
While the accuracy slightly drops for sentences with masculine pronouns, it
drastically improves for sentences with feminine pronouns. Therefore, the
discrepancy in accuracy between masculine and feminine pronoun sentences
decreases, resulting in lower discrimination. One could even argue that
gender-adjectives reduce gender bias in the output of NMT systems, but at the
same time, it can be discriminating in itself that, as soon as you add an
adjective to describe a person, the instance of this person is more likely to
be translated to its feminine gender. Further research is certainly needed to
find reasons for this effect and to assess the potentials of discrimination.

The sentences in our _WiBeMT_ challenge set are – as the original _WinoMT_
challenge set – constructed in a systematic way. While this allows for a
controlled experiment environment, it might also introduce some artificial
biases [11]. A solution could be to collect real-world examples of sentences,
which are suitable for gender bias detection. Furthermore, the limitation on
one source language (English) and one target language (German) does not allow
for a generalization of the results.

Another limitation of our study – as most other studies – is that it does not
take into account that gender should not be seen as a binary, but rather a
continuous variable. [2], for example, outline why “trans exclusionary” co-
reference resolution systems can cause harm, which is probably also valid for
MT systems. A further extension of the challenge set could help to shine a
light on the shortcomings of the inclusion of transgender persons of NMT
systems.

## References

  * [1] Bolukbasi, T., Chang, K.W., Zou, J., Saligrama, V., Kalai, A.: Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Advances in Neural Information Processing Systems pp. 4356–4364 (2016)
  * [2] Cao, Y.T., Daumé, H.: Toward Gender-Inclusive Coreference Resolution. arXiv preprint (2019)
  * [3] Cheryan, S., Ziegler, S.A., Montoya, A.K., Jiang, L.: Why are some STEM fields more gender balanced than others? Psychological Bulletin 143(1), 1–35 (2017), http://dx.doi.org/10.1037/bul0000052
  * [4] Garg, N., Schiebinger, L., Jurafsky, D., Zou, J.: Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences of the United States of America 115(16), E3635–E3644 (2018)
  * [5] Hovy, D., Bianchi, F., Fornaciari, T.: “You Sound Just Like Your Father” Commercial Machine Translation Systems Include Stylistic Biases. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pp. 1686–1690. Association for Computational Linguistics (2020)
  * [6] Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A Survey on Bias and Fairness in Machine Learning. arXiv preprint (2019)
  * [7] Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., Joulin, A.: Advances in Pre-Training Distributed Word Representations. In: LREC 2018 - 11th International Conference on Language Resources and Evaluation. pp. 52–55 (2017)
  * [8] Pennington, J., Socher, R., Manning, C.D.: GloVe: Global Vectors for Word Representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 1532–1543 (2014)
  * [9] Rudinger, R., Naradowsky, J., Leonard, B., Van Durme, B.: Gender Bias in Coreference Resolution. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. vol. 2, pp. 8–14 (2018)
  * [10] Saunders, D., Byrne, B.: Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pp. 7724–7736 (2020)
  * [11] Stanovsky, G., Smith, N.A., Zettlemoyer, L.: Evaluating Gender Bias in Machine Translation. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pp. 1679–1684 (2019)
  * [12] Suresh, H., Guttag, J.V.: A Framework for Understanding Unintended Consequences of Machine Learning (2019)
  * [13] Zhao, J., Wang, T., Yatskar, M., Ordonez, V., Chang, K.W.: Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods pp. 15–20 (2018)