# Finding the nitrogen-vacancy singlet manifold energy level using charge
conversion pulse sequences

I. Meirzada  The Racah Institute of Physics, The Hebrew University of
Jerusalem, Jerusalem 91904, Israel  S. A. Wolf  The Racah Institute of
Physics, The Hebrew University of Jerusalem, Jerusalem 91904, Israel  The
Center for Nanoscience and Nanotechnology, The Hebrew University of Jerusalem,
Jerusalem 91904, Israel  N. Bar-Gill  The Racah Institute of Physics, The
Hebrew University of Jerusalem, Jerusalem 91904, Israel  The Center for
Nanoscience and Nanotechnology, The Hebrew University of Jerusalem, Jerusalem
91904, Israel  Dept. of Applied Physics, Rachel and Selim School of
Engineering, Hebrew University, Jerusalem 91904, Israel

###### Abstract

The vast research conducted on Nitrogen-Vacancy (NV) centers in diamond in
recent years opened the door for a wide range of NV based applications.
However, some properties of the NV center dynamics and energy levels remain
unknown. In this work, we propose a charge conversion pulsed sequence scheme
for locating the NV center singlet manifold energy level, by multi-color
excitation using a tunable laser. We present two approaches for readout -
photo induced current and NV0 population - and discuss their efficiency for
different regimes in the relevant spectrum.

##  I Introduction

NV centers in diamond [1] stand at the center of a variety of scientific and
technological research avenues, such as quantum sensing and quantum
information processing [2, 3, 4, 5, 6, 7], due to their reliability and
versatility of operation. These applications rely on the NV center’s unique
optical and spin properties, which have been studied both theoretically and
experimentally [8, 9, 10, 11, 12, 13]. However, some of these properties are
yet to be determined. Completing this picture holds the potential for both
improving the current NV based applications and discovering new opportunities
using NV centers.

The NV center is composed of a nitrogen atom and a vacancy occupying two
adjacent sites in the diamond lattice. The electronic ground state is a spin
triplet with 2.87 GHz zero field splitting between spin projections $m_{s}=0$
and $m_{s}=\pm 1$. The electronic excited state is composed of a highly
radiative spin triplet located 1.95 eV above the triplet ground state and 0.65
eV below the conduction band, and a spin singlet with weak radiative coupling.
The singlet manifold comprises two states, ${}^{1}A$ (”excited”) and ${}^{1}E$
(”ground”). Although the energy gap between the two singlet states was
measured to be 1.19 eV, their exact location relative to the triplet states
remains unknown. Fig. 1 summarizes the known energy gaps between the different
states of the NV-.

Figure 1: A simplified summary of the energy levels in the NV- and their
distance from each other. The exact energy of singlet manifold (${}^{1}A$ and
${}^{1}E$) compared to the triplet manifold (${}^{3}E$ and ${}^{3}A$) or the
conduction band is unknown.

In this work we propose a detailed scheme for identifying the singlet manifold
energies. The aforementioned energies bound the energy gap between the singlet
ground state and the conduction band between 1.84 eV, corresponding to 674 nm,
and 2.6 eV, corresponding to 477 nm. Thus, we can divide this range into three
parts, each resulting in different dynamics during the pulse sequence:

  1. 1.

637 nm $\leq\lambda_{ion}\leq$ 674 nm - $\lambda_{ion}$ is longer than the ZPL
for both NV charge states. These photons can only be absorbed by the excited
states, either to be ionized (recombined) to NV0 (NV-) or decay back to the
ground states via stimulated emission.

  2. 2.

575 nm $\leq\lambda_{ion}\leq$ 637 nm - $\lambda_{ion}$ is longer than the NV0
ZPL and shorter then the NV- ZPL. The NV- can be excited from the ground state
but not the NV0.

  3. 3.

477 nm $\leq\lambda_{ion}\leq$ 575 nm - $\lambda_{ion}$ is shorter than the
ZPL for both NV charge states, allowing for excitation from both ground states
[13, 14].

The proposed pulse sequence aims to induce substantial ionization of the NV-
ground singlet state into the neutral charge NV0, thus generating additional
NV0 population or photo-induced current [15], once the threshold wavelength
for ionization is reached. The timings and powers in the sequences utilize the
previously published spin and charge dynamics for both charge states [16, 10,
11, 17, 18, 19, 14].

The pulse sequence, described in Fig. 2, contains six steps. We start with
initialization of the NV- to the $m_{s}=0$ state, followed by a microwave (MW)
$\pi$-pulse to initialize the NV into the $m_{s}=\pm 1$ state. Then, a short
($\sim$ 30 ns) green excitation populates the triplet excited state
(”population pulse”). Preparing the NV in the spin 1 state will help populate
the singlet state, thus enhancing the sensitivity to ionization from the
singlet. A short delay allows the NV to either decay to the singlet level or
back to the triplet ground state, thus preventing unwanted ionization from the
triplet excited state while having a negligible effect on the singlet ground
state population due to its metastable nature. Then a stronger ionization
pulse (477 nm $\leq\lambda\leq$ 674 nm) is applied while photo-current is
collected from the NV. A fluorescence based measurement requires an additional
readout pulse for collecting photons. Unless specified otherwise, the
population and ionization pulses powers are 2 mW and 5 mW, respectively. The
readout pulse power is 100 $\mu$W, with a 1$\mu$s duration.

Figure 2: The seven-step sequence for identifying ionization from the singlet
ground state. 1\. After initializing the NV- into the $m_{s}=0$ spin state, a
$\pi$ pulse is applied to initialize the NV into the $m_{s}=\pm$1 spin state.
2\. A short, strong green pulse populates the triplet excited state. 3\. Delay
of $\sim$ 30 ns is applied for the excited state to decay to avoid ionization
from the excited state. 4\. A $\sim$ 0.1 $\mu s$, second pulse (ionization
pulse) is applied while collecting photo-current. 5\. (For fluorescence based
measurement) green laser excitation while collecting fluorescence.

In the results given below, each simulation presents two curves. A red curve,
in which the ionization laser energy is higher than the singlet ionization
threshold ($\lambda_{ion}\leq\lambda_{s}$) and a blue curve, in which the
ionization laser energy is lower than the singlet ionization threshold
($\lambda_{s}\leq\lambda_{ion}$). In cases for which 532 nm
$\leq\lambda_{ion}$ the simulations take into account that the pulses used for
initialization and population ionize the singlet, for both the red and blue
curves.

###  I.1 Fluorescence-based measurements

In the fluorescence-based measurements, reaching the ionization threshold
manifests in the final NV- population, which can be accurately measured [20,
21]. We start with the longer wavelengths, for which the threshold ionization
does not excite the triplet ground state. In this case the green excitation
laser (532 nm) also ionizes the NV from the singlet ground state. The
following figures draw the expected NV- population under different conditions
such as laser power, ionization duration, delay time and ionization cross
section, outlining the parameter space in which the pulse sequence results in
strong evidence that the wavelength threshold $\lambda_{s}$ for singlet
ionization is reached. All laser power parameters in this work assume
diffraction limited excitation beams.

The first set of simulations addresses the case in which the ionization
threshold wavelength does not have enough energy to excite the NV- triplet
ground state (”Red” range). Fig. 3(a) presents the NV- population as a
function of green population power. Weak powers create a small population in
the singlet, while very strong powers ionize the singlet, thus also resulting
in small populations in this state. Fig. 3(b) presents the NV- population vs.
red ionization power. The red ionization power reduces the population
significantly as power increases (red line), although mild recombination
(NV${}^{0}\rightarrow$ NV-) occurs at the same time from leftover populations
in the NV0 excited state, as illustrated by the blue line. Fig. 3(c) presents
the NV- population as a function of singlet ionization cross section
(expressed in units of MHz/mW) for a constant red ionization power of 10 mW,
assuming the same cross section for both green and red excitation. Here the
red curve decreases faster, since the green-induced ionization changes in
addition to the red-induced ionization, unlike Fig. 3(b) for which only the
red-induced ionization changes with the increase in laser power. The blue
curve decreases for the same reason. The excitation in the range 637 nm
$\leq\lambda\leq$ 674 nm can also deplete electrons from the triplet excited
state directly to the triplet ground state, reducing the population needed in
the singlet state for the ionization detection. Fig. 3(d) shows the population
dependence on the delay between the initialization and ionization pulses,
providing strong evidence that the electron is ionized from the singlet state,
since it’s timescale changes according to the singlet-state lifetime assumed
in the simulation. The small decreasing trend found for short delay times
stems from ionizing the electron from the excited state, before it decays to
the singlet and ground triplet states. Lastly, Fig. 3(e) depicts the NV-
behavior as a function of ionization duration. The timescale here is a
function of both power and the singlet state lifetime. The weaker the
ionization would be (due to red power or ionization cross section), the closer
the timescale would be to the singlet state lifetime.

Figure 3: NV- final population for various sequence parameters where 637 nm
$\leq\lambda_{ion}\leq$ 674 nm. a. NV- population vs green power, populating
the singlet level. A power $\leq$ 1 mW is preferred for significant contrast.
b. NV- population vs red excitation power. Higher power provides higher
contrast, due to the delay between the green population pulse and the
ionization pulse. c. NV- population vs singlet ionization cross section (in
MHz/mW). Ionization rate higher than 0.5 MHz/mW is expected to result in
detectable contrast. d. NV- population vs the delay between the population and
ionization pulses. For short times, the recombination from NV0 builds higher
population in the negative state. After a few tens of ns the excited states
decay, and the contrast between the two cases decreases with the singlet state
lifetime. e. NV- population vs ionization pulse duration. The contrast
increases as long as the singlet state is occupied, and doesn’t decrease
afterwards since no other ionization mechanism exists.

The next set of figures analyzes the case in which the threshold wavelength
has enough energy to excite the NV- triplet ground state, although
insufficient to excite the NV0 ground state (”Orange” range). In this case,
the green excitation laser still ionizes the singlet. We note that in this
range the steady-state population while using the ionization laser strongly
favors the neutral charge state. Fig. 4(a) depicts the effect of the green
excitation power on the final NV- population, suggesting that it will be good
to work around 2 mW. Changing the ionization laser power creates a more
complex picture, as shown in Fig. 4(b). The curves seem independent of
ionization at low powers, while higher laser powers split the curves
significantly. This is because the ionization cross section from the ground
state is much higher than that from the singlet. At high powers the blue curve
reaches a plateau, since the population in the ground state has been
completely ionized. At this regime the singlet ionization becomes stronger,
resulting in a splitting between the red and blue curves. The behavior of the
population vs. singlet ionization cross section [Fig. 4(c)] is pretty straight
forward. An ionization cross section equivalent to a rate of 1 MHz/mW should
suffice for detecting the singlet ionization. The fact that the ionization
laser excites the NV- manifests in Fig. 4(d), as the curves show different
behaviour compared to Fig. 3(d). Both curves change with the timescale of the
singlet lifetime, yet with different coefficients, due to the additional
ionization process from the singlet state. The unique nature of the orange
wavelength for the NV system further expresses itself in Fig. 4(e), as it
leaves us a very limited time interval for ionization, before most of the NV-
population diminishes due to two-step ionization from the ground state.
However, this behavior results in high contrast in the population difference
between the two scenarios, thus making it easier to distinguish whether
ionization from the singlet happens given enough signal from the NVs.
Naturally, this time window depends strongly on the excitation power. In
addition, both curves present a double exponential behavior: A standard, fast
dynamic originated in the two-step ionization of the triplet (for both
figures), followed by a distinct, slower process, expressing the effective
singlet lifetime during the ionization process. The red curve decreases faster
than the blue curve, suggesting that an additional process clears the
population in the singlet manifold, as expected.

Figure 4: NV- final population for various sequence parameters, where 637 nm
$\leq\lambda_{s}\leq$ 575 nm. a. NV- population vs green population power. A
power $\leq$ 1 mW is preferred for significant contrast. b. NV- population vs
orange ionization power. Higher power provides higher contrast, as the
population in the triplet is expected to empty before the population in the
singlet, due to the assumption of low ionization cross section of the singlet.
c. NV- population vs singlet ionization cross section (in MHz/mW). Ionization
cross section higher than 0.5 MHz/mW is expected to result in detectable
contrast. d. NV- population vs the delay between the population and ionization
pulses. The contrast decreases with a time scale of the singlet lifetime e.
NV- population vs ionization pulse duration. The ionization duration must be
similar to the singlet lifetime, otherwise no significant contrast is
expected.

Moving forward to the next wavelength range, 575 nm $\leq\lambda_{s}\leq$ 532
nm (”Green” range), in which the ionization laser excites both ground states.
The steady-state population, in this case, favors the negative charge state
even during the ionization pulse. We note that the lower bound of this range
is dictated by the initialization and population wavelength and not an
intrinsic property of the NV center. Fig. 5(a) and 5(b) draw the NV-
population as a function of population and ionization pulse powers,
respectively. Both suggest using power $\geq$ 2 mW for a substantial decrease
in the final NV- population if singlet ionization occurs. This stems from the
requirement for substantial population in the singlet manifold, and due to the
assumption that the singlet ionization cross section is much smaller than the
excited state cross section. The increase in NV- population in Fig. 5(a) comes
from the way the sequence is designed. The initialization pulse creates
substantial population in the singlet state. During the ionization pulse
($<300$ ns), this population does not play a role in the charge dynamics,
resulting in a new steady state between the charge states, favoring NV-. At
high ionization laser powers, however, the rate from the NV- excited state to
the singlet becomes negligible compared to the ionization rate and causes a
decrease in NV- population. This same mechanism also explains the behavior of
the blue curve. For low powers, the decay from the singlet back to the triplet
state is faster than the charge dynamics between NV- and NV0. Using powers
around 1 mW, the ionization rate from the triplet exceeds the decay to the
singlet state, while not being strong enough to take advantage of the
population already in the singlet state. Similarly to Fig. 4(c), an ionization
cross section higher than 0.5 MHz/mW should suffice for detecting ionization,
as depicted in Fig. 5(c). The NV- population as a function of delay time,
shown in Fig. 5(d), behaves similarly to Fig. 4(d), with a larger NV-
population. Fig. 5(e) illustrates the qualitative difference of using a
wavelength that excites both charge states. The ionization pulse duration no
longer has an upper limit, as the contrast in the populations exists as a
difference between the two steady-state populations, as opposed to Fig. 4(e)
in which the difference in populations is transient. The blue curve increases,
since the previous, population pulse, also ionized the singlet, thus the
steady-state population increases during this step. For the same reason, the
NV- population does not display a significant difference in the red curve. The
fast dynamics stem from the delay between the population and ionization
pulses.

Figure 5: NV- final population for various sequence parameters when 532 nm
$\leq\lambda_{s}\leq$ 575 nm. a. NV- population vs green population pulse
power. A power $\leq$ 1 mW is preferred for significant contrast. b. NV-
population vs the green ionization power. Higher power provides higher
contrast, as the singlet ionization cross section is expected to be smaller
that the excited state cross section. c. NV- population vs singlet ionization
cross section (in MHz/mW). Ionization rate higher than 0.5 MHz/mW is expected
to result in detectable contrast. d. NV- population vs the delay between the
population and ionization pulses. The contrast decreases with a time scale of
the singlet lifetime e. NV- population vs ionization pulse duration. The
ionization duration is not bounded from above due to the fact that the
populations differences are not transient.

Up to now, the population laser also ionized the NV- singlet state, as it was
more energetic than the threshold wavelength for ionization. The last
wavelength range, 532 nm $\leq\lambda_{s}\leq$ 477 nm (”Blue” range), presents
a different situation, in which the population laser does not ionize the NV-
singlet state while still exciting both ground states. For this reason, the
population power simulated in this set of simulations was 5 mW (instead of 2
mW used in Figs. 7-9), in order to reach higher population in the singlet.
Fig. 6(a) shows the typical behavior as in the previous figures. Fig 10(b)
behaves like a combination of Figs. 8(b) and 9(b). For lower powers the
singlet becomes less populated, causing a preference for the neutral state
compared to the higher power used in the population step. Fig 6(c) shows the
expected behavior for cases in which the population pulse does not ionize the
singlet, and Figs. 10(d) and 10(e) resemble 9(d) and 9(e) with slight changes
for the same reason.

Figure 6:  NV- final population for various sequence parameters when 477 nm
$\leq\lambda_{s}\leq$ 532 nm, showing almost identical behavior as in Fig. 5.
a. NV- population vs green population pulse power. A power $\leq$ 1 mW is
preferred for significant contrast. b. NV- population vs the green ionization
power. Higher power provides higher contrast, as the singlet ionization cross
section is expected to be smaller than the excited state cross section. c. NV-
population vs singlet ionization cross section (in MHz/mW). Ionization rate
higher than 0.5 MHz/mW is expected to result in detectable contrast. d. NV-
population vs the delay between the population and ionization pulses. The fast
dynamics (short delays) stem from ionization from the triplet states. The
contrast decreases with a time scale of the singlet lifetime. e. NV-
population vs ionization pulse duration. The ionization duration is not
bounded from above due to the fact that the populations differences are not
transient.

###  I.2 photo-current based measurement

Measuring using photo-current (PC) provides a ’direct’ look into the charge
dynamics of the system, as it isolates the signal coming directly from the
NV${}^{-}\rightarrow$ NV0 process, instead of relying on transient or steady-
state populations in one of the charge states. As opposed to fluorescence-
based measurements, in this case no-ionization results in no signal in the
system (up to background noise). On the other hand, as the energy increases,
other defects may cause a non-negligible background current which can mask the
signal from the singlet [15]. The PC signal is calculated as the (ionization
rate) x (population) in the relevant states (triplet excited state and singlet
ground state), and is given in arbitrary units (AU).

As before, we start with the longest wavelength range - 637 nm
$\leq\lambda_{ion}\leq$ 674 nm. Fig. 7 depicts the expected behavior of the PC
signal as a function of various parameters. The fact that the ionization laser
does not excite the NV- triplet ground state or ionize P1 centers results in a
rather ’clean’ signal, as almost no PC is generated when the laser energy is
lower than the ionization energy. Fig. 7.a draws the PC signal as a function
of green population power. The red curve trend changes according to the change
in the singlet state population with green power, as high powers start to
ionize the singlet state as well, resulting in lower population in this state.
Increasing the ionization power, however, shows a monotonic behavior, as only
the singlet ionization rate changes with this laser power while the blue curve
is almost not affected (see Fig. 7.b). Similarly to the fluorescence based
measurement, the signal as a function of ionization cross section, depicted in
Fig. 7.c, resembles the behavior as a function of green power, as although it
causes higher ionization from the singlet during the ionization pulse, it
reduces the population in the singlet prior to the ionization step due to
green ionization during the population step. Fig. 7.d shows the PC signal as a
function of delay time between the population and ionization steps. For delays
longer than few tens of ns, the red PC signal drops as the singlet lifetime,
while for short times the signal decreases with the timescale of the triplet
excited state lifetime for both curves. Increasing the ionization duration
increases the PC until no population remains in the singlet state. However,
other current inducing mechanisms, not included in this model, may provide an
optimal duration for this step.

Figure 7: PC signal for various sequence parameters when 637 nm
$\leq\lambda_{ion}\leq$ 674 nm. a. Even very low green population powers may
generate high contrast in the signal. b. PC signal vs red excitation power.
Higher power provides higher contrast, due to the delay between the green
population pulse and the ionization pulse. c. NV- population vs singlet
ionization cross section (in MHz/mW). Even very low cross section is expected
to suffice, as almost no signal is expected under the threshold wavelength. d.
PC signal vs the delay between the population and ionization pulses. The fast
dynamics (short delays) stem from ionization from the triplet states. When
longer delays are applied decreases with a time scale of the singlet lifetime.
e. PC signal vs ionization pulse duration. The contrast builds as long as the
singlet state is occupied, as no other ionization mechanism exists.

Moving to the second range, 575 nm $\leq\lambda_{ion}\leq$ 637 nm, we start
seeing significant signal when no ionization from the singlet occurs as well,
due to two-step ionization induced by the orange laser. Fig. 8.a shows a
preference to higher laser powers with a weak optimum. In both curves most of
the signal stems from the two-step ionization of the population in the triplet
state, and the difference is proportional to the population in the singlet. At
very high powers the singlet is less populated because of the high ionization
rate, bringing the curves closer to each other. Fig. 8.b depicts the PC signal
as a function of the orange power. While low powers produce similar signal for
both scenarios, as the orange power increases the blue curve remains constant
(since most of the triplet population has already been ionized), while the red
curve continues to increase due to ionization from the singlet, creating the
contrast between the two curves. For measurements in this regime, one can use
the blue curve’s plateau as a baseline, to increase sensitivity for the
ionization signal. As expected, higher cross section results in higher signal,
and 0.5 MHz/mW should suffice for detectable contrast (Fig. 8.c). Both curves
are affected by the cross section, as it changes the ionization during the
population step (see Fig. 8.c). Fig. 8.d shows the PC signal as a function of
delay time. Similar to Fig. 7.d, the fast decrease in short times occurs due
to the short lifetime of the triplet excited state. Later, the slow increase
happens due to the singlet lifetime, since the simulations assume slower
ionization from the singlet state than two-step ionization from the ground
state. Lastly, Fig. 8.e depicts the PC signal as a function of ionization
duration, showing a very similar behavior to Fig. 4.e, limiting our ionization
duration for detectable contrast. However, this measurement can be
substantially improved just by selecting the correct time to start the
measurement. Both curves start with a fast increase in the PC signal, which
stems almost entirely from ionization from the triplet excited state. Starting
the measurement at the end of the fast step would eliminate most of the
triplet ionization signal, thus enhance the contrast once ionization from the
singlet occurs.

Figure 8: PC signal for various sequence parameters, where 637 nm
$\leq\lambda_{s}\leq$ 575 nm. a. PC signal vs green power, populating the
singlet level. powers $\leq$ 1 mW are preferred for significant contrast. b.
PC signal vs orange excitation power. Higher power provides higher contrast,
as the population in the triplet is expected to empty before the population in
the singlet due to the assumption of low ionization cross section in the
singlet. c. NV- population vs singlet ionization cross section (in MHz/mW).
Ionization rate higher than 0.5 MHz/mW is expected to result in detectable
contrast. d. PC signal vs the delay between the population and ionization
pulses. The fast dynamics (short delays) stem from ionization from the triplet
states. When longer delays are applied decreases with a time scale of the
singlet lifetime. e. PC signal vs ionization pulse duration. The ionization
duration must be similar to the singlet lifetime, otherwise no significant
contrast is expected.

The next range, 532 nm $\leq\lambda_{ion}\leq$ 575 nm, shows a simpler
behavior compared to the previous range. Fig. 9.a resembles Fig. 8.a, as the
main source of contrast in this figure comes from the population of the
singlet state, while at some point the population laser start ionizing the NV
from the singlet. However, Fig. 9.b shows much less contrast as a function of
ionization power compared to Fig. 8.b, because in this case the wavelength is
below 575 nm, enabling two step recombination process. This causes multiple
ionization of the same NV, and due to the assumption that the two-step
ionization process from the ground state would be faster than the ionization
from the singlet, this results in a lower contrast of the PC signal. Similarly
to the previous figures, a cross section corresponding to a rate higher than
0.5 MHz/mW should provide a detectable contrast when reaching the threshold
ionization wavelength, while the delay time between the population and
ionization pulses should be below 200 ns for optimal results, as depicted in
Figs. 9.c and 9.d respectively. Lastly, Fig. 9.e shows higher signal as a
function of ionization duration, while the contrast reaches a plateau. This is
how the steady state charge-state, described in 5.e, is manifested in the PC
signal.

Figure 9: PC signal for various sequence parameters when 532 nm
$\leq\lambda_{s}\leq$ 575 nm. a. PC signal vs green population pulse power. A
power $\leq$ 1 mW is preferred for significant contrast. b. PC signal vs green
ionization power. Higher power provides higher contrast, as the singlet
ionization cross section is expected to be smaller that the excited state
cross section. c. PC signal vs singlet ionization cross section (in MHz/mW).
Ionization rate higher than 0.5 MHz/mW is expected to result in detectable
contrast. d. PC signal vs the delay between the population and ionization
pulses. The contrast decreases with a time scale of the singlet lifetime. e.
PC signal vs ionization pulse duration. The ionization duration is not bounded
from above due to the fact that the population differences are not transient.
The contrast builds up to 30%.

Like the fluorescence based measurement, the last range - 477 nm
$\leq\lambda_{ion}\leq$ 532 nm presents similar behavior as the previous range
(see Fig. 10).

Figure 10: PC signal for various sequence parameters when 477 nm
$\leq\lambda_{s}\leq$ 532 nm, showing almost identical behavior as in Fig. 9.
a. PC signal vs green population pulse power. A power $\leq$ 1 mW is preferred
for significant contrast. b. PC signal vs the green ionization power. Higher
power provides higher contrast, as the singlet ionization cross section is
expected to be smaller that the excited state cross section. c. PC signal vs
singlet ionization cross section (in MHz/mW). Ionization rate higher than 0.5
MHz/mW is expected to result in detectable contrast. d. PC signal vs the delay
between the population and ionization pulses. The fast dynamics (short delays)
stem from ionization from the triplet states. The contrast decreases with a
time scale of the singlet lifetime. e. PC signal vs ionization pulse duration.
The ionization duration is not bounded from above due to the fact that the
populations differences are not transient.

##  II discussion

Comparing the two detection methods, there is no clear advantage for one over
the other for wavelengths below 637 nm. For wavelengths longer than 637 nm the
PC approach is expected to produce significantly higher contrast since
ionization from the excited state is expected to be eliminated. In addition,
multiple cycles of population + ionization steps are likely to further enhance
the signal without adversely affecting the contrast, as long as the population
laser powers are carefully controlled. However, as the ionization energy
increases, it induces additional ionization processes of other defects in the
diamond lattice, thus masking the desired signal from the NV.

Fluorescence-based measurements can be done by either reading the NV- signal
or the NV0. However, most of the cases covered in this work have shown limited
contrast in the NV- population due to expected spin mixing. Additional spin
mixing generated by the relatively high laser powers used in the sequences
might eliminate the remaining contrast below detection threshold. Assuming
high NV density, accurate NV0 population measurements may result in higher SNR
compared to collecting NV- fluorescence, despite the weaker signal, especially
when working in the blue and green regimes. In these regimes, the NV strongly
prefers the negative charge state, thus the ’jump’ in the NV0 population once
reaching the threshold wavelength may provide substantially higher contrast,
overcoming the reduced number of photons.

The simulations assume a conservative approach regarding the lasers used
during the experiment. Specifically, the dynamics presented consider CW lasers
and a modulation component. However, using a pulsed laser (for the ionization
step) holds the potential to boost the SNR of the measurements dramatically,
if the pulse duration is short enough. As long as the laser is non-resonant
with the ${}^{3}A\rightarrow^{3}E$ transition, pulses shorter than $\sim$ 5 ps
are not expected to cause substantial ionization from the ${}^{3}E$ state into
the conduction band [22]. Thus, most of the ionization will originate from NVs
in the singlet state. In this case, the blue curves in the PC section become
negligible at all wavelength ranges (as in Fig. 7), while the red curves in
the fluorescence section drop (as in Fig. 3). This is, of course, as long as
the repetition time is $\stackrel{{\scriptstyle{\textstyle<}}}{{\sim}}$ 30 ns,
such that the ${}^{3}E$ state decays almost fully between pulses.

##  III Acknowledgements

This work has been supported by the Ministry of Science and Technology,
Israel, and the European Union’s Horizon 2020 research and innovation program
under grant agreements No. 714005 (ERC StG Q-DIM-SIM), No. 820374
(MetaboliQs), and No. 828946 (PATHOS).

## References

  * Doherty  _et al._ [2013] M. W. Doherty, N. B. Manson, P. Delaney, F. Jelezko, J. Wrachtrup, and L. C. Hollenberg, The nitrogen-vacancy colour centre in diamond, Physics Reports 528, 1 (2013).
  * Clevenson  _et al._ [2015] H. Clevenson, M. E. Trusheim, C. Teale, T. Schröder, D. Braje, and D. Englund, Broadband magnetometry and temperature sensing with a light-trapping diamond waveguide, Nature Physics 11, 393 (2015).
  * Acosta  _et al._ [2010a] V. M. Acosta, E. Bauch, A. Jarmola, L. J. Zipp, M. P. Ledbetter, and D. Budker, Broadband magnetometry by infrared-absorption detection of nitrogen-vacancy ensembles in diamond, Applied Physics Letters 97, 174104 (2010a).
  * Dolde  _et al._ [2011] F. Dolde, H. Fedder, M. W. Doherty, T. Nöbauer, F. Rempp, G. Balasubramanian, T. Wolf, F. Reinhard, L. C. L. Hollenberg, F. Jelezko, and J. Wrachtrup, Electric-field sensing using single diamond spins, Nature Physics 7, 459 (2011).
  * Taylor  _et al._ [2008] J. M. Taylor, P. Cappellaro, L. Childress, L. Jiang, D. Budker, P. R. Hemmer, A. Yacoby, R. Walsworth, and M. D. Lukin, High-sensitivity diamond magnetometer with nanoscale resolution, Nature Physics 4, 810 (2008).
  * Loretz  _et al._ [2014] M. Loretz, S. Pezzagna, J. Meijer, and C. L. Degen, Nanoscale nuclear magnetic resonance with a 1.9-nm-deep nitrogen-vacancy sensor, Applied Physics Letters 104, 033102 (2014).
  * Trusheim and Englund [2016] M. E. Trusheim and D. Englund, Wide-field strain imaging with preferentially aligned nitrogen-vacancy centers in polycrystalline diamond, New Journal of Physics 18, 123023 (2016).
  * Maze  _et al._ [2011] J. R. Maze, A. Gali, E. Togan, Y. Chu, A. Trifonov, E. Kaxiras, and M. D. Lukin, Properties of nitrogen-vacancy centers in diamond: the group theoretic approach, New Journal of Physics 13, 025025 (2011).
  * Ma  _et al._ [2010] Y. Ma, M. Rohlfing, and A. Gali, Excited states of the negatively charged nitrogen-vacancy color center in diamond, Phys. Rev. B 81, 041204 (2010).
  * Acosta  _et al._ [2010b] V. M. Acosta, A. Jarmola, E. Bauch, and D. Budker, Optical properties of the nitrogen-vacancy singlet levels in diamond, Physical Review B 82, 10.1103/PhysRevB.82.201202 (2010b).
  * Manson  _et al._ [2006] N. B. Manson, J. P. Harrison, and M. J. Sellars, Nitrogen-vacancy center in diamond: Model of the electronic structure and associated dynamics, Phys. Rev. B 74, 104303 (2006).
  * Neumann  _et al._ [2009] P. Neumann, R. Kolesov, V. Jacques, J. Beck, J. Tisler, A. Batalov, L. Rogers, N. B. Manson, G. Balasubramanian, F. Jelezko, and J. Wrachtrup, Excited-state spectroscopy of single NV defects in diamond using optically detected magnetic resonance, New Journal of Physics 11, 013017 (2009).
  * Aslam  _et al._ [2013] N. Aslam, G. Waldherr, P. Neumann, F. Jelezko, and J. Wrachtrup, Photo-induced ionization dynamics of the nitrogen vacancy defect in diamond investigated by single-shot charge state detection, New Journal of Physics 15, 013064 (2013).
  * Meirzada  _et al._ [2018] I. Meirzada, Y. Hovav, S. A. Wolf, and N. Bar-Gill, Negative charge enhancement of near-surface nitrogen vacancy centers by multicolor excitation, Phys. Rev. B 98, 245411 (2018).
  * Bourgeois  _et al._ [2017] E. Bourgeois, E. Londero, K. Buczak, J. Hruby, M. Gulka, Y. Balasubramaniam, G. Wachter, J. Stursa, K. Dobes, F. Aumayr, M. Trupke, A. Gali, and M. Nesladek, Enhanced photoelectric detection of nv magnetic resonances in diamond under dual-beam excitation, Phys. Rev. B 95, 041402 (2017).
  * Robledo  _et al._ [2011] L. Robledo, H. Bernien, T. van der Sar, and R. Hanson, Spin dynamics in the optical cycle of single nitrogen-vacancy centres in diamond, New Journal of Physics 13, 025013 (2011).
  * Tetienne  _et al._ [2012] J.-P. Tetienne, L. Rondin, P. Spinicelli, M. Chipaux, T. Debuisschert, J.-F. Roch, and V. Jacques, Magnetic-field-dependent photodynamics of single NV defects in diamond: an application to qualitative all-optical magnetic imaging, New Journal of Physics 14, 103033 (2012).
  * Chen  _et al._ [2015] X. Chen, C. Zou, Z. Gong, C. Dong, G. Guo, and F. Sun, Subdiffraction optical manipulation of the charge state of nitrogen vacancy center in diamond, Light: Science & Applications 4, e230 (2015).
  * Jeske  _et al._ [2017] J. Jeske, D. W. M. Lau, X. Vidal, L. P. McGuinness, P. Reineck, B. C. Johnson, M. W. Doherty, J. C. McCallum, S. Onoda, F. Jelezko, T. Ohshima, T. Volz, J. H. Cole, B. C. Gibson, and A. D. Greentree, Stimulated emission from nitrogen-vacancy centres in diamond, Nature Communications 8, 14000 (2017).
  * Shields  _et al._ [2015] B. J. Shields, Q. P. Unterreithmeier, N. P. de Leon, H. Park, and M. D. Lukin, Efficient readout of a single spin state in diamond via spin-to-charge conversion, Phys. Rev. Lett. 114, 136402 (2015).
  * Hopper  _et al._ [2018] D. A. Hopper, R. R. Grote, S. M. Parks, and L. C. Bassett, Amplified Sensitivity of Nitrogen-Vacancy Spins in Nanodiamonds Using All-Optical Charge Readout, ACS Nano 12, 4678 (2018).
  * Huxter  _et al._ [2013] V. M. Huxter, T. A. A. Oliver, D. Budker, and G. R. Fleming, Vibrational and electronic dynamics of nitrogen–vacancy centres in diamond revealed by two-dimensional ultrafast spectroscopy, Nature Physics 9, 744 (2013).


# Generating Gender Augmented Data for NLP

Nishtha Jain1, Maja Popovic2, Declan Groves3, Eva Vanmassenhove4

1ADAPT Centre, Trinity College Dublin,
2ADAPT Centre, Dublin City University,
3Microsoft, Dublin,
4Department of CSAI, Tilburg University
<EMAIL_ADDRESS>3
degroves<EMAIL_ADDRESS>

###### Abstract

Gender bias is a frequent occurrence in NLP-based applications, especially
pronounced in gender-inflected languages. Bias can appear through associations
of certain adjectives and animate nouns with the natural gender of referents,
but also due to unbalanced grammatical gender frequencies of inflected words.
This type of bias becomes more evident in generating conversational utterances
where gender is not specified within the sentence, because most current NLP
applications still work on a sentence-level context. As a step towards more
inclusive NLP, this paper proposes an automatic and generalisable re-writing
approach for short conversational sentences. The rewriting method can be
applied to sentences that, without extra-sentential context, have multiple
equivalent alternatives in terms of gender. The method can be applied both for
creating gender balanced outputs as well as for creating gender balanced
training data. The proposed approach is based on a neural machine translation
(NMT) system trained to ‘translate’ from one gender alternative to another.
Both the automatic and manual analysis of the approach show promising results
for automatic generation of gender alternatives for conversational sentences
in Spanish.

##  1 Introduction

Recent studies have exposed challenging systematic issues related to bias that
extend to a range of AI applications, including Natural Language Processing
(NLP) technology Costa-jussà (2019); Blodgett et al. (2020). Observed bias
problems range from copying biases already existing in data to claims that the
training process can lead to an exacerbation or amplification of observed
biases Zhou and Schiebinger (2018); Vanmassenhove et al. (2021). The
algorithms learn to maximize the overall probability of an occurrence, leading
to preferences for more frequently appearing training patterns.

With this work, we propose a method for generating (more) balanced data in
terms of one of the main types of bias frequently observed in language: gender
bias. Gender bias can occur in language due to the fact that some languages
have a way of explicitly marking (natural or grammatical) gender while others
do not Stahlberg et al. (2007). Gender bias in translation is usually
manifested when animate entities (e.g. professions) are translated from gender
neutral language (e.g. English) into a gendered language (e.g. Spanish)
because the instances seen in training data are biased. Also, conversational
utterances are prone to bias, both in machine translation as well as in other
NLP applications, because systems often do not have the ability to provide
multiple gender variants. Therefore, users are simply presented with the most
probable option which is prone to bias. In our work, we aim to enable the
generation of multiple gender variants by expanding each sentence with the
missing gender variants, thus fostering inclusion in online conversations/NLP
applications. Generating gender variants can and should also be used to create
gender balanced conversational data that can be used to train less biased NLP
models such as machine translation models, language models, chat bots, etc.

Unlike previous studies, we did not want to limit ourselves to one specific
gender phenomenon, such as gender markings on professions Zmigrod et al.
(2019)) (for which the gender can easily be swapped by using hand-crafted
lists) or first person personal pronouns Habash et al. (2019)). The objective
of this research aims to include as many cases as possible of gender
alternatives related not only to gender of persons but also to grammatical
gender of the objects referred to. In Example 2, (a) illustrates an example of
two alternatives for a sentence where there is agreement with the grammatical
gender of an object referred to in the previous sentence, while in (b) there
is agreement with the gender of the speaker/writer (i.e. a person).

Example 1.
(a) [MALE] ¿Está completo? – [FEMALE] ¿Está completa? 111English: ”Is it
complete?”
(b) [MALE] Estoy confundido. – [FEMALE] Estoy confundida.222English: ”I am
confused.”

At this stage, our approach does not discriminate between human referents and
objects. It is furthermore limited to the generation of binary gender
alternatives. We are aware of the importance and challenge of dealing with
non-binary genderAckerman (2019) which we aim to tackle in future work.

The research was carried out in collaboration with an anonymous industry
partner with a specific application in mind that deals with conversational
sentences. Our approach aims to alleviate gender bias in the said application.
We focus on one gender-rich language (Spanish), however, scalability and
generalizability were kept in mind while designing the approach. Our approach
can be summarized as follows:

  1. 1.

Identifying (appropriate) sentences/segments that should have the opposite
gender variant for some words. POS sequences were used to extract such
segments from the OpenSubtitles corpus333https://opus.nlpl.eu/.

  2. 2.

Creating gendered variants for the words in such segments by applying a rule-
based approach.

  3. 3.

Training a neural rewriter on the compiled gender-parallel Spanish data in
order to be able to automatically generate gendered variants on unseen data
sets. This additional step makes the approach more scalable as it removes the
need for any preprocessing.

The first two steps are necessary since there is a lack of readily available
open-source gender-parallel data for training. Although language knowledge and
a POS tagger are necessary for these steps, the human effort and necessity for
external linguistic tools are minimal (contrary to other approaches which
heavily rely on linguistic tools Zmigrod et al. (2019) or on manually created
gender-parallel data Habash et al. (2019).

##  2 Related Work

In the literature on gender in NLP, two main approaches for bias mitigation
can be identified: (a) approaches that attempt to mitigate bias during model
or word representation training, and/or (b) approaches that aim to augment the
data by creating more variety in the training set (pre-processing step) or in
the output (post-processing step). In the following paragraphs, we focus on
the latter as it is most closely related to our approach.

There have been attempts to artificially increase the variety in already
existing data sets by creating alternatives to sentences in order to decrease
the overall bias (in terms of gender).444Different types of bias exist,
however, the current approaches have focused on gender, possibly because many
languages have explicit gender markers. This approach has been referred to in
the literature as ‘Counterfactual Data Augmentation’(CDA) (Lu et al., 2018).
Their CDA approach consists of a simple bidirectional dictionary of gendered
words such as he:she, her:him/his, queen:king, etc. Zhao et al. (2018) does
not use the term CDA as this was introduced later, but what they describe can
be interpreted as a rudimentary approach to CDA: they augmented the existing
data set by adding additional sentences in which personal pronouns ‘he’ and
‘she’ had been swapped.

Another CDA approach is described in Zmigrod et al. (2019). Similar to Lu et
al. (2018), the approach relies on a bidirectional dictionary of animate
nouns. Unlike Lu et al. (2018), pronouns are not handled and the languages
worked on are Hebrew and Spanish, languages that have more gender markers than
English. Since solely changing the nouns into their male/female counterpart
often requires the enforcement of grammatical gender agreement of accompanying
articles and adjectives, they introduce Markov Random Fields with optional
neural parametrisation that can infer the effect of the swap on the remaining
words in the segment. Their approach is limited to mitigating gender
stereotypes related to animate nouns and relies on dependency trees, lemmata,
POS-tags and morpho-syntactic tags in order to solve issues related to the
morpho-syntactic agreement.

In the field of machine translation (MT), due to specific discrepancies
between the information encoded in the source and target data, there has been
some work on generating the appropriate gender variant for ambiguous source
sentences.555‘I am a teacher’ or ‘I am smart’ in English are not marked for
gender. However, in many other languages they would be morphologically marked
for the male or female gender (e.g. French, Spanish…). Vanmassenhove et al.
(2019) appends gender tags to the source side of the training data indicating
the gender of the speaker. As such, during testing, the desired (or multiple)
gender variant(s) can be generated by adding tags. Basta et al. (2020) also
experiment with incorporating a gender tag, and investigate adding the
previous sentence as additional context information. Both methods result in
the improvement of automatic MT scores as well as on gender accuracy for
English-to-Spanish translation. Similarly, Bentivogli et al. (2020) developed
NMT systems using gender tags and evaluated them specifically on gender
phenomena.

The work described in Habash et al. (2019) is the most similar to ours. They
proposed an approach for automatic gender reinflection (“re-gendering") for
Arabic. They propose a method which consists of two components: a gender
classifier and a NMT gender rewriter. In order to build the NMT rewriter, they
first manually created a corpus annotated with gender information.
Subsequently, each gendered sentence is re-gendered manually in order to
obtain the necessary gender-parallel data for training. This way, they are
able to provide gender alternatives for sentences with natural gender
agreement with the first person singular.

Our research, in contrast, aims to augment existing data with gender
alternatives in a broader sense: it is not limited to singular first person
phenomena, ambiguity in multilingual settings, or phenomena related solely to
gender agreement. It involves the gender of adjectives, past participles, and
several types of pronouns for which the referent is not explicitly mentioned
within the context of the sentence.

##  3 Generating gender-parallel data

As mentioned in the introduction, our main objective is to create an automatic
gender rewriter using NMT. In order to do so, we need gender-parallel training
data that consists of possible gender variants in both directions (masculine-
to-feminine and feminine-to-masculine). Such data sets are, unfortunately, not
publicly available, which is why we first leveraged linguistic knowledge and
rules to generate a sufficient amount of gender-parallel data.

Therefore, we identified the sequences of POS classes that show gender
agreement in Spanish and can thus be ‘re-gendered’: adjectives, past
participles, and several types of pronouns. A detailed description of how the
different word classes are tackled to generate gender alternatives is
described below. We would like to point out that our target data consisted of
very short sentences, where there is at most agreement with one
referent.666For example, sentences such as “I am happy and they are angry.”
are not covered by our approach as both ‘happy’ and ‘angry’ are in agreement
but with different referents, ‘I’ and ‘they’ respectively. Such sentences
would require the generation of more than two alternatives since both
referents are ambiguous. As such, our approach is limited to tackle sentences
alike and cannot handle the generation of alternatives for sentences where
more than two gender alternatives could be generated (due to grammatical
agreement of the re-genderable word with multiple entities).

###  3.1 Re-genderable word classes

##### Past participles

In principle, almost all Spanish past participles have an explicit agreement
with their referent and can thus be re-gendered. However, in certain contexts
they should not be: if they follow or precede a referent noun (“Película
aburrida”, “Acceso permitido.”) thus agreeing with the gender of the noun, or
if they follow the auxiliary verb “haber” thus representing past tense and not
a property of a person/object (“he enviado”, “has descansado”). If they appear
in isolation (“Ocupado/ocupada.”, “Aburrido/aburrida.”), or merely surrounded
by interjections or punctuation (“Ocupado/ocupada, gracias.”, “Buenos dias,
recibido/recibida, ¡gracias!”), adverbs (“muy cansado/cansada”), or a linking
verb (“Estoy registrado/registrada.”, “Parece acabado/acabada.”), they can be
re-gendered.

We also included pairs of past participles bound by conjunctions, referring to
the same person or object, since in these sentences, both instances should be
re-gendered (“aburrido/aburrida y cansado/cansada.”, “acabado/acabada y
pagado/pagada.”).

##### Adjectives

Many Spanish adjectives are gendered and have an explicit gender marker
corresponding to the gender of its referent. However, some adjectives are
gender neutral. Gendered and neutral adjectives can (largely) be identified
based on their specific suffixes (for example “-al”, “-nte”, “-ble”, so the
adjectives “genial”, “interesante”, and “probable” are neutral), while other
suffixes indicate gendered adjectives (for example “o/a”, so the adjective
“correcto/correcta” has variants).

In addition, similarly to past participles, the given context has to be taken
into account for gendered adjectives: they should not be re-gendered if they
immediately precede or follow a noun (with or without article) which
determines the gender (“Presupuestos adjuntos.”, “¡Maravillosa idea!”, “La
información correcta.”). Also, adjectives following neutral demonstrative
pronouns “eso” or “esto” should not be re-gendered (“Eso es bueno.”).
Analogous to past participles, adjectives in isolation (“Listo/Lista.”,
“perfecto/perfecta.”, “seguro/segura.”, “¡fantástico/fantástica!”), surrounded
by punctuation (“Correcto/correcta, saludos.”), preceding verb (“¿Estás
listo/lista?”) or adverb (“Es muy lindo/linda.”) can be re-gendered.

When two adjectives are present, in a conjunction, and refer to the same
referent, both should be re-gendered.

##### Clitic pronouns

Some Spanish clitic pronouns, namely “lo(s)” and “la(s)” should be re-gendered
(e.g. “Lo/la veo.”, “Lo/la adjunto.”) while “le(s)” should not be changed (“Le
veo.”, “Le digo.”). However, in some cases “lo” can represent a general
concept not referring to a particular object, such as in “lo siento” (I’m
sorry), “lo sé” (I know). If some of these are re-gendered, the precision will
decrease.

##### Clitic pronouns attached to verbs

Clitic pronouns can be attached to a verb infinitive (“Gracias por
acabarlo/acabarla.” (thanks for finishing it), “Quiero verlo/verla.” (I want
to see it)). Similar to the isolated clitic pronouns, there are certain
exceptions, such as “Es bueno saberlo” (it is good to know). If the gender
neutral clitic pronoun “le” is attached to a verb (“Quiero tenerle informado.”
(I want to keep you/him/her informed)), it should not be re-gendered. Gendered
pronouns attached to an imperative should also be re-gendered
(“Déjalo/Déjala.” (leave it), “Hazlo/Hazla.” (do it)). On the other hand,
clitic pronouns which refer to an indirect object, such as “mándame” (send
me), are neutral. Finally, if there are two attached clitic pronouns,
“Mándamelo/Mándamela.” (send it to me), only the gendered part (in this case
“lo”/“la”) should be re-gendered.

##### Demonstrative pronouns

Demonstrative pronouns “esto”, “eso” and “aquello” are neutral, while
“estos/estas”, “este/esta”, “ese/esa”, “aquello/aquella” are gendered. If the
referent is missing in the sentence and the pronoun is gendered, they should
be re-gendered.

###  3.2 Adding gender variants by rules

Whether a gender alternative translation should be generated does not solely
depend on the word classes it contains but also on the structure of the
sentence. If the referent is missing in a sentence, then an additional variant
with the opposite gender should be generated. If the referent is present in a
sentence, only one gender variant is grammatically correct, and as such, these
sentences are to be left unchanged. The presence or absence of a referent can
be determined by the sequence of POS tags in a sentence777Assuming that the
sentences are short- this approach would not generalize to longer sentences.
For example, if we want to check whether a sentence with an adjective “creo
que es correcta" (gloss: “I believe (it) is correct-feminine") needs an
additional re-gendered variant or not, its POS sequence “VERB CONJUNCTION VERB
ADJECTIVE" indicates that there is no referent noun within the given context.
Therefore, another variant of the adjective “correct" should be provided:
“creo que es correcto". In contrast, the sentence “la solución es correcta"
with POS sequence “ARTICLE NOUN VERB ADJECTIVE" contains a referent noun
“solución", and therefore it should not be re-gendered.

For each re-genderable sentence, we apply rules for changing the ending of the
corresponding word, if necessary. The POS sequences to identify re-genderable
sentences and the subsequent rules used to re-gender the corresponding words
in such sentences are given in detail in the Appendix. It is worth mentioning
we also used POS sequences to identify neutral sentences (those which should
be not re-gendered ) since we wanted the parallel corpus to contain both.

##  4 Gender-parallel data

In order to create gender-parallel data, a set of Spanish subtitles was
downloaded from the OPUS Tiedemann (2012) website.888http://opus.nlpl.eu/
After basic filtering (removing too long and non-alpha numeric segments), a
set of short sentences with up to 10 (untokenized) words was extracted. This
candidate set consisted of 22 458 968 sentences. This data set was POS tagged
using Treetagger999https://www.cis.uni-muenchen.de/ schmid/tools/TreeTagger/.
The sentences matching the POS sequences mentioned in the Appendix were
extracted from this data set. This set consisted of more than 1M sentences.
For each extracted re-genderable sentence, the alternative gender variant is
created by applying appropriate rules described in the Appendix. After
applying rules on all re-genderable structures, we joined both re-gendering
directions (masculine-to-feminine and feminine-to-masculine) in order to
create a balanced data set. As already mentioned, the corpus also contains a
number of sentences that are not to be regendered. By including these neutral
sentences in our training data, we encourage the rewriter to: (a) learn when
to generate alternatives and when not to, and (b) how to generate those
alternatives, if necessary. In this way, a corpus with about 2.2M gender-
parallel sentences was created. This corpus was then separated into train,
development ($\sim$1k sentences) and test ($\sim$3k sentences) sets. The
rewritten parts of the development and test sets were revised manually and the
errors were corrected for about 6% of sentences and 1.5% of words. The
training set, being large, was not verified manually, thus it contained some
noise.

In addition to OpenSubtitles, we also obtained data from the industry partner
consisting of around 8 000 sentences readily available with all possible
alternative versions of the sentences provided. An additional 22 000 sentences
had to be revised manually in order to produce the correct gender variant for
re-genderable sentences. This set was used as an additional test set for the
re-writer. One part of this set can be handled by the described POS sequences
and rules (“structured test 1"), while another part contains different POS
sequences and cannot be handled by these rules at all (“unstructured test 1").
The latter test set will give a good estimation of the scalability of our
approach. An overall split of data sets is described in Table 1. The
OpenSubtitles data was split in the standard way for machine translation,
namely a few thousands of segments for development and test sets and the rest
for the training set.

set | segments
---|---
training (OpenSubtitles) | 2 193 657
development (OpenSubtitles) | 1 018
test (OpenSubtitles) | 3 066
structured test1 | 5 648
unstructured test1 | 15 892
Table 1: Statistics of data used for building the NMT rewriter.

##  5 Neural Rewriter

Once we compiled a sufficient amount of gender-paralell data, we were able to
train our automatic rewriter. The automatic rewriter is a NMT system trained
on the following parallel data: original sentences as the source language, and
re-gendered sentence as the target language. For neutral sentences, the source
and the target parts are identical.

The NMT rewriter was built using the publicly available
Sockeye101010https://github.com/awslabs/sockeye implementation Hieber et al.
(2018) of the Transformer architecture Vaswani et al. (2017). The system
operates on sub-word units generated by byte-pair encoding (BPE)Sennrich et
al. (2016). We set the number of BPE merging operations to 32000. We have
experimented with the following setups:

  * •

a Standard NMT system without any additional tags

  * •

an NMT system with neutrality/re-genderability tags in the source part

The system with tags was built using the same technique as proposed in Johnson
et al. (2017) for multilingual MT systems and used for many other applications
including gender-informed MT Vanmassenhove et al. (2019). For our experiments,
we added a label ‘N’ (neutral) or ‘G’ (re-genderable) to each source sentence.
These tags are implicitly present in the gender-parallel data – if the source
and the target parts differ, it is a re-genderable sentence, if they are
identical it is neutral. Therefore, the tags are certainly available for the
training and development sets, but they might not be available for the test
sets. Therefore, this system was assessed in two ways:

  * •

“NMT-T”: neutrality/re-genderability tags are available for the test sets

  * •

“NMT-AT”: the tags are not available for the test sets (a realistic scenario)
and therefore are assigned automatically by the gender classifier described in
the next section (which is similar to the approach described in Habash et al.
(2019).)

###  5.1 Gender Classifier

In order to explore potential benefits of automatic pre-classification for
automatic rewriting, a classifier to distinguish between ‘re-genderable’
(G)111111Grammatical gender markings are not related to a referent within the
sentence, therefore these markings have to be expanded. and ‘neutral’
(N)121212No gender markers that need to be expanded. sentences was also
designed. The tags generated by this classifier were used to assess the
performance of the “NMT-AT” re-writer by appending them to the sentences.

#### Data

The classifier was built on the data set of about 8 000 sentences provided by
the industry partner. These sentences were balanced in both directions i.e.,
both masculine-to-feminine as well as feminine-to-masculine counterparts of a
given sentence were present and labelled as G. The rest of the sentences were
labeled as N.

For the sake of designing a generalised classifier, the development set
consisted of sentences from the OpenSubtitles corpus (and was the same as the
development set used for the NMT system).

The final classifier was tested on two different test sets - one consisted of
the 22 000 conversational sentences sourced from the industry partner and
another extracted from the OpenSubtitles corpus.

#### Features

Following on the work of Habash et al. (2019) for the gender identification
step, features using character $n$-grams, word $n$-grams and morphological
information were created from the training data. To begin with, TF-IDF scores
of character $n$-grams of length 4-7 with maximum features capped at 20 000
and of word $n$-grams of length 1-3 were generated. These two feature matrices
were joined together along with a morphological feature that denoted the
presence of a gendered word in the sentence. The resulting training data was a
high dimensional data frame with around 40 000 features.

Due to the limited size of the training set, neural network based classifiers
were ruled out. Instead, owing to the high dimensional nature of the data, we
used a SVM based classifier for training. All the steps described in this
section were implemented in Python 3.7 using sklearn131313https://scikit-
learn.org/stable/, pandas141414https://pandas.pydata.org/ and
StanzaNLP151515https://stanfordnlp.github.io/stanza/ libraries.

#### Precision and Recall

The SVM based classifier was tested on two sets of data as described in
Section 5.1. This was done in order to assess the generalisability of the
classifier. Given the small size of the training data, the performance of the
classifier looks promising thus far (see Table 2).

It can be observed in Table 2 that the classifier clearly performs better on
the test data set consisting of sentences sourced from the industry partner as
compared to the data extracted from OpenSubtitles. While the accuracy is
comparable on both sets ( 80%), the precision and recall of neutral sentences
is higher on the industry data than the set compiled from OpenSubtitles data.
The high recall of sentences labelled as G implies that the classifier is
almost always successful at recognising sentences that need to be re-gendered
(i.e. sentences that need an alternative variant). However, it incorrectly
predicts the labels of a substantial number of N-labelled sentences, which in
turn results in a low precision of re-genderable sentences. As we want to
avoid generating (incorrect) gender alternatives for neutral sentences, our
aim was to first attain a high precision for neutral sentences and then aim
towards a high recall for the same. The tags generated by this classifier for
the industry sourced data and OpenSubtitles data were used to test the “NMT-
AT” rewriter.

| Industry Test Set | OpenSubs
---|---|---
| Acc. | Rec. | Prec. | Acc. | Rec. | Prec.
Overall | 82% | - | - | 80% | - | -
G | - | 96% | 60% | - | 97% | 76%
N | - | 76% | 98% | - | 56% | 93%
Table 2: Gender Classifier Results

##  6 Results for generating gender variants

Our first experiment consisted of using the implementation of CDA by Zmigrod
et al. (2019) to generate gendered variants. However, this work only tackled
animate nouns, which rarely occur in the conversational sentences we
investigated in this work. Our re-implementation of their approach generated
the correct gender variant for only 1% of the sentences. Because of the very
low recall, this implementation was not directly applicable for our research.
In addition to this, since our work aims to tackle multiple gender related
word classes, we explored extending the implementation by augmenting the list
with character adjectives. On doing so, we found that this implementation
generated the correct gendered variant in only 9% of the cases. An important
point to note is that 3% of the neutral sentences (for which variants should
not have been generated) were also converted as opposed to the 1% with only
animate nouns, attributed to the presence of more words in the hand-crafted
lists. In order to cover more words and improve the performance of this
implementation on our data set, we considered augmenting the hand-crafted list
with past participles and/or clitic pronouns. However, that increased the size
of the list exponentially and made the approach prone to errors, inefficient
and not scalable to other languages.

###  6.1 Automatic evaluation of neural rewriter

The results in the form of error rates are shown in 3. Since we are not
performing typical machine translation, namely converting one language into
another one, but only converting a few words in the sentence into a sentence
in the same language, these error rates are not related to any of the typical
automatic evaluation metrics (such as TER, etc.) but to the amount of
incorrectly converted words. For each system, numbers in the left column
represent the count of incorrectly converted words normalised by the total
number of sentences, while numbers in the right column represent the count of
incorrectly converted words normalised by the total number of words in the
corpus. The numbers in the first row and first two columns can be interpreted
as follows: left: 6.4% of all sentences have incorrectly converted words in ;
right: 1.50% of all words are incorrectly converted.

set | type | NMT | NMT-T | NMT-AT | rules
---|---|---|---|---|---
test | all | 6.4 | 1.50 | 4.5 | 1.03 | 17.9 | 4.21 | 6.1 | 1.43
(structured) | neutral | 5.3 | 1.13 | 2.5 | 0.48 | 33.3 | 7.07 | 0.0 | 0.0
| re-genderable | 7.1 | 1.81 | 6.0 | 1.51 | 6.0 | 1.72 | 6.1 | 1.43
test1 | all | 2.4 | 0.54 | 1.3 | 0.27 | 4.5 | 0.99 | 3.2 | 0.7
(structured) | neutral | 4.8 | 0.95 | 2.2 | 0.43 | 8.7 | 1.73 | 0.0 | 0.0
| re-genderable | 0.8 | 0.19 | 0.6 | 0.14 | 1.6 | 0.38 | 3.2 | 0.7
test2 | all | 11.9 | 2.13 | 5.2 | 0.93 | 10.4 | 1.87 | not
(unstructured) | neutral | 3.3 | 0.58 | 0.3 | 0.04 | 6.0 | 1.07 | applicable
| re-genderable | 57.3 | 10.7 | 31.1 | 5.84 | 33.4 | 6.26 |
Table 3: Results for NMT rewriter: error rates (%): count of incorrectly
converted words normalised by the total number of sentences (left columns) and
normalised by the total number of words (right columns).

First, it can be noted that the error rates are lower for the template-based
“in-domain” test sets than for the unstructured “out-of-domain” test sets,
which is in line with our expectations. The change in error rate is mainly due
to discrepancies in the re-genderable segments. The error rates in the neutral
segments are comparable in the out-of-domain and in-domain test sets.

Adding manual tags indicating whether a sentence should get a gender
alternative or not (e.g. ‘neutral’ vs ‘regenderable’) reduces the error rates
on all test sets for both types of segments. A similar performance can not be
achieved by adding automatic tags. Automatic tags deteriorate the performance
on neutral segments, but reduce the error rates for re-genderable segments,
especially for the unstructured “out-of-domain” test set. The manually tagged
results indicate the potential of a classifier. These results tie up with the
results of the gender classifier (Section 5.1) which is good at classifying
the re-genderable sentences as denoted by a high recall of sentences labelled
‘G’, however it doesn’t do very well at labelling neutral sentences as ‘N’. It
tends to mislabel many of those sentences as ‘G’, resulting in a low recall
and, consequently, incorrect re-gendering.

For the sake of completeness, error rates are reported for the rule-based
rewriter, too. The error rates for re-genderable sentences are lower than the
NMT rewriter without tags and for neutral sentences the error rate is 0%; it
should be noted that the rules are applicable only to data sets which strictly
conform to the described template structures.

###  6.2 Qualitative manual inspection of errors

In order to better understand the nature of errors and remaining challenges, a
qualitative manual inspection was carried out on all test sets. First of all,
it is observed that in general, the NMT re-writer does not intervene on large
portions of a sentence but addresses only specific words, which is exactly
what it is expected to do. This is a positive result, as generating gender
variants implies changing specific gendered words and does not involve
changing entire segments. It also facilitates the evaluation since manual
inspection is needed only to identify the nature of incorrect words.

type | original | correct | NMT | NMT-T
---|---|---|---|---
N | esto es perfecto | esto es perfecto | esto es perfecta | esto es perfecto
G | está adjunto | está adjunta | está adjunto | está adjunto
(a) structured sentences

| type | original | correct | NMT | NMT-T
---|---|---|---|---|---
1) | N | no son lo mismo | no son lo mismo | no son la misma | no son lo mismo
2) | N | aquello fue encantador | aquello fue encantador | aquello fue encantadora | aquello fue encantador
3) | N | ¿a quién aprovecha? | ¿a quién aprovecha? | ¿a quién aprovecho? | ¿a quién aprovecha?
4) | N | indíqueme la | indíqueme la | indíqueme la | indíqueme la
|  | disponibilidad | disponibilidad | emperbilidad | evelbilidad
5) | N | indíqueme su | indíqueme su | indíqueme su | indíqueme su
|  | disponibilidad | disponibilidad | disponibilidad | escorpibilidad
6) | N | unos momentos | unos momentos | unos momentos | unos momentos
|  | extraordinarios | extraordinarios | extraordinarias arios | extraordinarios
7) | N | indíquenos cuánto | indíquenos cuánto |  indíquenas cuánto | indíquenos cuánto
8) | G | esta es la adecuada | este es el adecuado |  esta es la adecuada |  esta es lo adecuada
9) | G | esta la hemos recibido | este lo hemos recibido |  esta la hemos recibido |  esta lo hemos recibido
(b) unstructured sentences

Table 4: Examples of incorrectly generated sentence variants for (a)
structured sentences and (b) unstructured sentences.

The analysis revealed that the most frequent error for neutral sentences are
re-gendered pronouns and adjectives which should not be changed. Also, the
most frequent error in re-genderable sentences is leaving them unchanged.
These types of errors are predominant in structured sentences, and two
examples, one for neutral and one for regenderable sentence, can be seen in
Table 4(a). It can also be seen that adding tags can help in some cases.

For unstructured sentences, there are more error types especially for neutral
sentences, and examples can be seen in Table 4(b). In the first three
sentences, the same error type as for structured sentences can be seen, namely
some words are changed which should not be changed. Adding tags helped in both
cases. However, some other error types can be seen, such as converting some
(not gender-related) words into non-existing words in sentences 4) and 5). For
sentence 5), generating a non-existing word was triggered by adding tags.
Sentence 6) shows an unnecessary re-gendering as well as adding non-existing
words. This was also resolved by adding tags. In sentence 7), a word which is
not at all related to gender was converted, and this was prevented by adding
tags.

As for regenderable sentences, the vast majority of errors are again the
unchanged words which had to be changed. If there is more than one word to be
regendered, sometimes they all remain unchanged (sentence 8) and sometimes
only some of them are regendered (sentence 9). Tags can help to some extent,
but only for some words, not all.

Adding tags generated by the classifier also increases the number of correctly
re-gendered structures at the cost of a small number of additions of non-
existing words.

##  7 Conclusions and Future Work

In this paper, we describe an initial approach towards enriching short
conversational sentences with their gender variants. Unlike other related
work, our approach is not limited to tackling the first person singular
phenomena, swapping third person pronouns or merely dealing with occupational
or generally animate nouns. In addition, with our approach, the reliance on
linguistic knowledge and tools is kept to a minimum in order to facilitate
real-world deployment.

The main hurdle for this type of research is the absence of large training
sets. Although provided with some manually annotated data from the industry
partner, the data provided was far from sufficient to train a state-of-the-art
automatic gender re-writer.

Therefore, training data was extracted from OpenSubtitles using linguistic
knowledge about the targeted language, namely Spanish. Re-genderable types of
words (POS classes) were identified and then frequently occurring ‘re-
genderable’ as well as ’neutral’ POS patterns were extracted. By applying the
corresponding rules to the re-genderable sentences, a large gender-parallel
Spanish data set was compiled.

Next, an NMT rewriter was trained in order to ‘translate’ each re-genderable
sentence into its gender alternative which showed promising performance both
in terms of automatic as well as of manual evaluation.

In addition, it is shown that providing additional information regarding the
need for rewriting in the form of tags could be helpful for the NMT system, as
similar tags have shown to be useful for other applications such as
multilingual translation, controlling politeness and gender in MT, etc. While
gold standard labels show better performance than the labels generated by the
gender classifier, the classifier shows promising results given the very small
training set. Further experiments should investigate a classifier trained on
larger amount of data.

In future work, we would like to explore how a similar approach can be applied
on more sentence structures in Spanish, as well as for different languages
which exhibit distinct gendering rules. Furthermore, different NMT
architectures, e.g. character-level NMT or an NMT system with linguistically
motivated subword units could be an interesting extension to the conducted
experiments, given that gender is usually marked by specific morphemes
(usually not more than one or two specific characters). In addition to that,
the performance of the gender classifier can be improved to produce more
accurate tags by using larger annotated training sets, adding more
morphological information in features and using word embeddings instead of TF-
IDF scores.

## References

  * Ackerman (2019) Lauren Ackerman. 2019.  Syntactic and cognitive issues in investigating gendered coreference.  _Glossa: a journal of general linguistics_ , 4(1).
  * Basta et al. (2020) Christine Basta, Marta R Costa-jussà, and Noe Casas. 2020.  Extensive study on the underlying gender bias in contextualized word embeddings.  _Neural Computing and Applications_ , pages 1–14.
  * Bentivogli et al. (2020) Luisa Bentivogli, Beatrice Savoldi, Matteo Negri, Mattia Antonino Di Gangi, Roldano Cattoni, and Marco Turchi. 2020.  Gender in danger? evaluating speech translation technology on the must-she corpus.  _arXiv preprint arXiv:2006.05754_.
  * Blodgett et al. (2020) Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020.  Language (technology) is power: A critical survey of “bias” in NLP.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5454–5476, Online. Association for Computational Linguistics.
  * Costa-jussà (2019) Marta R Costa-jussà. 2019.  An analysis of gender bias studies in natural language processing.  _Nature Machine Intelligence_ , pages 1–2.
  * Habash et al. (2019) Nizar Habash, Houda Bouamor, and Christine Chung. 2019.  Automatic gender identification and reinflection in arabic.  In _Proceedings of the First Workshop on Gender Bias in Natural Language Processing_ , pages 155–165.
  * Hieber et al. (2018) Felix Hieber, Tobias Domhan, Michael Denkowski, David Vilar, Artem Sokolov, Ann Clifton, and Matt Post. 2018.  The sockeye neural machine translation toolkit at AMTA 2018.  In _Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)_ , pages 200–207, Boston, MA. Association for Machine Translation in the Americas.
  * Johnson et al. (2017) Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. 2017.  Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation.  In _Transactions of the Association of Computational Linguistics, Volume 5:1_ , pages 339–351, Vancouver, Canada.
  * Lu et al. (2018) Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. 2018\.  Gender Bias in Natural Language Processing.  In _arXiv:1807.11714_.
  * Sennrich et al. (2016) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016.  Neural machine translation of rare words with subword units.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL 2016)_ , pages 1715–1725, Berlin, Germany.
  * Stahlberg et al. (2007) Dagmar Stahlberg, Friederike Braun, Lisa Irmen, and Sabine Sczesny. 2007.  Representation of the sexes in language.  _Social communication_ , pages 163–187.
  * Tiedemann (2012) Jörg Tiedemann. 2012.  Parallel Data, Tools and Interfaces in OPUS.  In _Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12)_ , pages 2214–2218, Istanbul, Turkey.
  * Vanmassenhove et al. (2019) Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2019.  Getting gender right in neural machine translation.  _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 3003–3008.
  * Vanmassenhove et al. (2021) Eva Vanmassenhove, Dimitar Shterionov, and Matthew Gwilliam. 2021.  Machine translationese: Effects of algorithmic bias on linguistic complexity in machine translation.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 2203–2213.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017.  Attention is All you Need.  In _Proceedings of The Thirty-first Annual Conference on Neural Information Processing Systems 30 (NIPS)_ , pages 5998–6008, Long Beach, CA, USA.
  * Zhao et al. (2018) Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018.  Learning Gender-Neutral Word Embeddings.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4847–4853, Brussels, Belgium.
  * Zhou and Schiebinger (2018) J Zhou and L Schiebinger. 2018.  AI Can be Sexist and Racist – It’s Time to Make it Fair.  In _Nature 559_ , pages 324–326. https://www.nature.com/articles/d41586-018-05707-8.
  * Zmigrod et al. (2019) Ran Zmigrod, Sabrina J. Mielke, Hanna Wallach, and Ryan Cotterell. 2019.  Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019)_ , pages 1651–1661, Florence, Italy.