# Exploring the Impact of Training Data Distribution and Subword Tokenization
on Gender Bias in Machine Translation

Bar Eluz♢11footnotemark: 1 Tomasz Limisiewicz♠ Gabriel Stanovsky♢ David
Mareček♠
${}^{\diamondsuit}\;$School of Computer Science, The Hebrew University of
Jerusalem
${}^{\spadesuit}\;$Faculty of Mathematics and Physics, Charles University in
Prague
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>Equal contribution. Work partially done
while visiting the Hebrew University.

###### Abstract

We study the effect of tokenization on gender bias in machine translation, an
aspect that has been largely overlooked in previous works. Specifically, we
focus on the interactions between the frequency of gendered profession names
in training data, their representation in the subword tokenizer’s vocabulary,
and gender bias. We observe that female and non-stereotypical gender
inflections of profession names (e.g., Spanish “doctora” for “female doctor”)
tend to be split into multiple subword tokens. Our results indicate that the
imbalance of gender forms in the model’s training corpus is a major factor
contributing to gender bias and has a greater impact than subword splitting.
We show that analyzing subword splits provides good estimates of gender-form
imbalance in the training data and can be used even when the corpus is not
publicly available. We also demonstrate that fine-tuning just the token
embedding layer can decrease the gap in gender prediction accuracy between
female and male forms without impairing the translation quality. 111The code
is available at https://github.com/tomlimi/MT-Tokenizer-Bias

##  1 Introduction

Figure 1:  The schema depicts two factors affecting the accuracy of
translating profession words into correct gender-inflected forms in
morphologically rich languages. The first factor is the frequency of gender
inflections of profession names in the training corpus, and the second is the
number of subword tokens that these forms are split into. Our analysis reveals
that the frequency significantly correlates with the translation accuracy and
number of tokens per word. However, when we control for frequency, the
correlation between the number of tokens and translation accuracy is
insignificant, indicating that frequency is a confounding variable.

Machine translation has been one of the fastest-growing research directions in
NLP. However, with the intensive growth of the technology, multiple potential
harms were identified Hovy and Spruit (2016), including gender bias, where
models rely on spurious correlations (doctors tend to be male) to make their
predictions rather than more meaningful signals in their input Stanovsky et
al. (2019).

There are many reasons for the bias, such as imbalances in the training set or
architecture choice. Previous works proposed various approaches to combat
gender bias in translation models Saunders and Byrne (2020); Escudé Font and
Costa-jussà (2019).

In this paper, we focus on the role of tokenization on gender bias which has
been largely overlooked in previous approaches to the problem.

We want to study the causal relationship between tokenization and gender bias.
Specifically, we want to know: 1. How do subword tokenizers handle different
gender forms, i.e., are female and non-stereotypical gender forms split into
more tokens than male and stereotypical gender forms? 2. Whether subword
splitting has an impact on the accuracy of translation? 3. Would subword
tokenization’s effect be significant when accounting for the frequency of
gender forms in the training corpus?

To answer those questions, we analyze pre-trained machine translation models
from English to a diverse set of three languages that denote morphological
gender in nouns ( German, Spanish, and Hebrew).

First, we compare the number of tokens of different gender forms in the target
language and find out that, indeed, female and anti-stereotypical forms are
split into more tokens. Second, the causality analysis shows that the number
of subword tokens may initially appear to explain the translation accuracy of
gender forms. However, we find out that these factors are conditionally
independent when we also consider the word frequency in the training set, as
depicted in Figure 1. To support this finding, we fine-tune the model on
gender balanced dataset and update its tokenizer, showing that the dataset’s
role is more impactful on gender bias than tokenization.

To the best of our knowledge, this work is the first in-depth analysis of
interactions between training data, tokenization method, and gender bias. Our
findings confirm the previous observations Saunders and Byrne (2020); Zmigrod
et al. (2019) indicating that the distribution of gender forms in the training
data significantly influences the bias of a model. Subword tokenizers,
typically trained on the same data, can also perpetuate biases present in the
data. We show that it is feasible to analyze the representation of gender
forms in the learned vocabulary to obtain information on gender distribution
in the model’s training corpus even without having access to it.

##  2 Experimental Setup

###  2.1 Models, languages and tokenizers

The translation models we used for the analysis are OpusMT Tiedemann and
Thottingal (2020), based on Marian NMT framework Junczys-Dowmunt et al.
(2018), trained on the Opus dataset222https://opus.nlpl.eu/, and mBART50 Tang
et al. (2020), a multilingual encoder-decoder model trained on 50 languages.
Both OpusMT and mBART50 use SentencePiece based tokenizer Kudo and Richardson
(2018) with unigram language model Kudo (2018). We test models in translating
from English to German, Spanish, and Hebrew.

#### Choice of target languages.

We chose German, Spanish, and Hebrew as target languages since they are
diverse, and all assign grammatical gender to profession names, adjectives,
and nouns. Moreover, the authors of this study possess a proficiency ranging
from intermediate to native levels in the aforementioned languages. To
highlight typological differences between the languages: Hebrew is a Semitic
language from the Afroasiatic language family with Abjad script. German is a
Germanic language with the Latin alphabet. Spanish is a Romance language with
Latin script that uses a change of suffix (instead of addition) for male-to-
female infections. Both German and Spanish belong to the Indo-European
language family.

#### Choice of models.

We chose OPUS and mBART models because they are accessible through
Huggingface, they support all the languages we selected for analysis, and they
manifest strong performance in translation tasks. Both models follow state-of-
the-art design choices, specifically Transformer architecture and
SentencePiece tokenizer, which was shown to be preferred over BPE in
multilingual models (Bostrom and Durrett, 2020a).

###  2.2 Data

For the evaluation of gender bias, we use the WinoMT (Stanovsky et al., 2019).
It is a synthetic English dataset of sentences containing profession names
coreferred by gendered pronouns. The sentences are balanced in terms of the
number of male and female pronouns used. The construction of this dataset
allows checking if translation models show preference toward particular gender
forms. The methods for measuring these preferences are described in the
following subsection 2.3.

#### Gender forms in target languages.

To validate the gender translation correctness of professions, we collected
human translations from native speakers of Hebrew, German, and Spanish (three
annotators for each language) for a list of 40 professions from WinoBias
dataset Zhao et al. (2018). The list contains an equal share of professions
that are majorly performed by men and women (based on labor statistics). Each
profession is translated to a pair of masculine and feminine forms with the
same stem (e.g., “Mediziner” – “Medizinerin” and “Arzt” – “Ärztin” are two
pairs in German).333Both pairs are German translations of the English
profession “Doctor”. The annotators could propose up to 3 pairs of
translations for each profession. Subsequently, the authors selected a list of
pairs that were proposed by at least two annotators 444For detailed rules for
combining the final lists and discussion of special cases, see Appendix A. As
a result, we accepted 67 pairs for German (77% of the annotators’
propositions), 54 pairs for Hebrew (76% of the propositions), and 45 pairs for
Spanish (73% of the propositions). The lists with the translations will be
released upon the publication of this work.

#### Gender forms frequency analysis.

To estimate the frequency of gender forms in the training corpus, we analyze
the OPUS-100 dataset (Zhang et al., 2020). It is a sample from OPUS collection
on which OpusMT was trained. It comprises multiple corpora subjects like movie
subtitles, code documentation, and the Bible. We used all three training,
development, and test corpora of OPUS-100, which contains 1,004,000 sentences
for each of the analyzed languages.

#### Gender-balanced dataset for fine-tuning.

We compile a gender-balanced bilingual dataset containing a simple English
template: “He/She is the [profession]” paired with translations to Hebrew and
German. The number of examples with male and female pronouns is equal. The
templates are filled with profession names from WinoMT from English and their
translations proposed by the annotators.

###  2.3 Metrics

#### Gender Translation Accuracy (F1).

To evaluate the model’s performance, we check whether the model translates
professions in English from WinoMT dataset to correct gender forms in the
target language proposed by annotators. For instance, the profession
“Physician”, when the pronouns indicate male gender in the source sentence,
should be translated to corresponding male forms in German (“Arzt” or
“Mediziner”). Respectively, when a pronoun indicates female gender, we should
obtain female forms in the translator’s output sentence (“Ärztin” or
“Medizinerin” for German). We compute the number of correct occurrences, i.e.,
translated sentences for which profession and gender match the English source.

We define _recall_ for each profession in a specific gender as the share of
correct occurrences out of a total number of source (English) sentences where
the profession appeared.

The _precision_ for each attested (i.e., proposed by annotators) profession
translation is the share of correct occurrences out of the total number of
outputted sentences where the profession translation appeared.

In our experiments, we report _F1_ , which is the harmonic mean of precision
and recall for each attested profession translation.

#### Measures of Gender Bias ($\Delta G$, $\Delta S$, $\Delta T$).

We use metrics proposed by Stanovsky et al. (2019) to measure gender bias in
MT: $\Delta G$ measures the difference in gender translation correctness (F1)
between masculine and feminine entities; similarly, $\Delta S$ measures the
difference in F1 between pro-stereotypical and anti-stereotypical instances of
gender role assignments. 555We use previous works’ attribution of stereotype
to particular English professions (Zhao et al., 2018)

We compute $\Delta G$ for each pair of male and female translations. It is a
more fine-grained approach than in previous works:

$\Delta G=\text{F1}_{m.trans.}-\text{F1}_{f.trans.}$ (1)

Analogically we compute $\Delta S$ as the difference in F1 between pro- and
anti-stereotypical translation.

$\Delta S=\text{F1}_{pro.trans.}-\text{F1}_{anti.trans.}$ (2)

Additionally, we define new metrics that measure the differences in the number
of tokens that distinct gender forms split into for each pair of profession
name translations. $\Delta T_{G}$, analogically to $\Delta G$, quantifies the
difference in the number of tokens between the male form and the female form:

$\Delta T_{G}=\text{n. Tokens}_{m.trans.}-\text{n. Tokens}_{f.trans.}$ (3)

While, $\Delta T_{S}$ corresponds to $\Delta S$ and measures the difference in
number of tokens between the pro- and anti-stereotypical forms:

$\Delta T_{S}=\text{n. Tokens}_{pro.trans.}-\text{n. Tokens}_{anti.trans.}$
(4)

With $\Delta T$ we metricize bias already on the tokenization level and
inspect its effect on the machine translation performance. We expect the words
split into more tokens to be harder to predict and thus observe correlations
between pairs $\Delta T$ and translation bias metrics: $\Delta G$ and $\Delta
S$.

#### Examples:

In case of translation from English to German, recall for English profession
“Physician” in female form is the number of times “Physician” appeared in the
source dataset as female and was translated to “Ärztin” or “Medizinerin”,
divided by the number of times “Physician” appeared as a female in the source
dataset. Precision for translation “Ärztin” is the number of times “Physician’
appeared in the source dataset as female and was translated to “Ärztin”
divided by the number of times the word “Ärztin” appeared in the translator’s
output. F1 for translation “Ärztin” is a harmonic mean of recall for the
female “Physician” and precision for “Ärztin”.

$\Delta G$ for a pair “Arzt” – “Ärztin” is difference in F1 for translation
“Arzt” and translation “Ärztin”. Because “Physician” is a stereotypically male
profession $\Delta S$ equals to $\Delta G$.

Both $\Delta T_{G}$ and $\Delta T_{S}$ are the difference between the number
of tokens the words “Arzt” and “Ärztin” are divided into by the German
tokenizer. 666Please note that for stereotypically female professions, we have
$\Delta G=-\Delta S$ and $\Delta T_{G}=-\Delta T_{S}$.

##  3 Experiments and Results

(a) German

(b) Spanish

(c) Hebrew

Figure 2: OpusMT: Human translated profession names were grouped by the number
of tokens they were split into. On the x-axis: number of tokens per word. On
the y-axis: the count of male and female forms professions in each of the
groups. Male forms tend to be split into fewer tokens than female forms.

(a) German

(b) Spanish

(c) Hebrew

Figure 3: OpusMT: Human translated profession names were grouped by the number
of tokens they were split into. On the x-axis: number of tokens per word. On
the y-axis: the count of pro- and anti-stereotypical forms of professions in
each of the groups. Pro-stereotypical forms tend to be split into fewer tokens
than anti-stereotypical forms.

To test how tokenization affects the translation accuracy of professionals’
gender, and whether a model prefers to generate translations with fewer
tokens, we design four experiments explained in the following section.

###  3.1 Are female and anti-stereotypical forms split into more tokens?

We take human translations (obtained from the procedure described in Section
2.2) and check how many tokens they are split into by the analyzed system’s
tokenizer.

We expect that female forms will be split into more tokens than male forms,
partially due to derivational suffixes appearing only in female forms.

(a) German

(b) Spanish

(c) Hebrew

Figure 4: OpusMT: $\Delta G$ as the difference between F1-score for male and
female test instances for each of paired translations. $\Delta T_{G}$ is the
difference between the number of tokens in a male and a female form. An orange
line marks the median. Pearson’s correlation coefficients and corresponding
p-values: $\rho=-0.25$, $p=0.09$ for German, $\rho=-0.21$, $p=0.20$ for
Spanish, and $\rho=-0.11$, $p=0.50$ for Hebrew.

(a) German

(b) Spanish

(c) Hebrew

Figure 5: OpusMT: $\Delta S$ as the difference between F1-score for pro- and
anti-stereotypical test instances for each of paired translations. $\Delta
T_{G}$ is the difference between the number of tokens in a pro- and an anti-
stereotypical translation. An orange line marks the median. We observe a
significant negative correlation between the two measures. Pearson’s
correlation coefficients and corresponding p-values: $\rho=-0.37$, $p=0.010$
for German, $\rho=-0.37$, $p=0.024$ for Spanish, and $\rho=-0.41$, $p=0.008$
for Hebrew.

#### Results

Figures 2 and 3 show how many translated WinoMT professions are split into a
specific number of tokens. We observe that female forms tend to be divided
into more tokens than male ones. Only a small portion of female forms are not
split. Similarly, pro-stereotypical translations are split into fewer tokens
than anti-stereotypical ones. However, the difference is smaller than in the
case of gender. We observe that the difference in the male and female number
of tokens in Spanish is smaller because female forms in Spanish are sometimes
expressed by changing the suffix rather than addition (e.g. Consejero vs.
Consejera).

The reason why female and anti-stereotypical forms are split into more tokens
is probably that they appear in the training corpus less often.777The results
of mBART50 are similar to Opus-MT and are represented in Appendix C.

###  3.2 Does subword splitting affect the accuracy of translation?

We compute bias metrics: $\Delta G$, $\Delta S$ for pairs of translations to
compare them with the difference in the number of tokens between gender forms
($\Delta T$).

We expect that when profession names differ in the number of tokens, the model
will more likely generate the shorter form (typically the male or pro-
stereotypical). Our inspection is that the preference for shorter forms is
connected to gender bias. Therefore we expect to observe a negative
correlation between the difference in translation accuracy and the number of
subword tokens.

#### Results

In Figures 4 and 5, we observe negative correlations between $\Delta T$ and
both $\Delta G$ and $\Delta S$. The relationship is stronger in the latter
case. This finding supports our hypothesis that the difference in the number
of tokens leads to the model’s preference for a form with fewer tokens.
Additionally, for translation pairs with $\Delta T=0$, the median of the bias
measure distribution is close to zero (with the notable exception of $\Delta
G$ for Hebrew). This suggests that the model is less biased for professions
when both translation forms are divided into the same number of tokens.

(a) German

(b) Spanish

(c) Hebrew

Figure 6: Pair analysis of gender prediction performance (F1), number of
tokens, and the frequency of each profession in the OPUS-100 dataset. Each
style of dots represents professions in the male/female pro-
stereotypical/anti-stereotypical form. The diagonal plots show the density of
the feature for the specific gender and stereotype sets.

###  3.3 What is the causal relationship between tokenization, training data,
and gender prediction accuracy?

An alternative explanation of the negative trend observed in the previous
experiment is the presence of an underlying factor, in our case, the frequency
of specific gender forms in the training corpus. Previous research has shown
that the terms’ frequency affects both tokenization Kudo (2018) and gender
bias Escudé Font and Costa-jussà (2019).

In this experiment, we measure the significance of the correlation between
those three factors. Also, we check the conditional independence between the
number of tokens per target profession and gender prediction accuracy
(measured by F1 score for each target profession described in Section 2) given
the profession form’s frequency in the training corpus:

$F1\perp\text{n. Tokens}\mid\text{Frequency}$ (5)

#### Results

In Figure 6, we observe that as the number of tokens decreases, F1 increases.
Taking into account the correlation coefficient, the F1 measure is more
sensitive to the frequency of a word in the training corpus. Moreover, less
frequent words tend to be split into more tokens. From the density plots (on
the diagonal of the figure), we see that male profession words (especially
pro-stereotypical ones) appeared much more often in the training corpora.
Thus, they tended to be split into fewer tokens.

All the correlations between frequency and the two remaining factors are
statistically significant ($p<0.05$), while the correlation between the number
of tokens and F1 score is significant only for German.

We performed Jonckheere-Terpstra test Jonckheere (1954) to check conditional
independence as described in Equation 5. The test showed that conditional
independence cannot be rejected for all the target languages ($p=0.78$ for
German, $p=0.62$ for Spanish, and $p=0.39$ for Hebrew). Those results show
that the frequency of a word is a confounding factor affecting both the number
of tokens per profession word and F1 scores. Hence subword splitting of
profession words is not a significant contributor to the correctness of gender
prediction given the frequency of gender forms in the training corpus.

| Accuracy |  $\Delta$ G |  $\Delta$ S | BLEU
---|---|---|---|---
| DE | ES | HE | DE | ES | HE | DE | ES | HE | DE | ES | HE
original | 62.2 | 56.3 | 52.1 | 9.6 | 16.4 | 12.3 | 12.5 | 16.7 | 40.4 | 31.4 | 42.8 | 36.0
FT | 67.1 | 58.9 | 54.0 | -2.9 | 10.9 | 7.8 | 21.5 | 20.0 | 35.1 | 31.3 | 42.0 | 35.8
FT + VU | 67.8 | 58.0 | 52.4 | 1.1 | 14.0 | 10.4 | 21.7 | 18.2 | 40.7 | 29.1 | 38.3 | 35.1
Table 1: Embedding fine-tuning results in German and Hebrew. Original OpusMT
model results compared with the model after embedding layer fine-tuning (row
2) and the model after embedding layer fine-tuning and updating vocabulary
with all profession gender forms in target language (row 3).

###  3.4 Will intervening in the model’s training data and the tokenizer’s
vocabulary reduce gender bias?

To verify our findings about causes of gender bias, we propose two
interventions in the translation models to German, Spanish, and Hebrew: 1.
fine-tuning on a dataset with an equal number of male and female forms;888The
dataset contains the sentences in the form “He/She is the [profession]”
translated to the target language and filled with professions from WinoMT. 2\.
Adapting the tokenizer’s vocabulary by adding all translations proposed by
annotators to assert that they will not be split into subword tokens.

We monitor standard gender bias measures from Stanovsky et al. (2019): gender
accuracy, $\Delta G$, and $\Delta S$,999In this experiment, we use the
original formulation of $\Delta G$ and $\Delta S$. It means $F1$ is computed
for the whole dataset, instead of particular professions as in the previous
experiments. and also BLEU on OPUS-100 test split to check if fine-tuning
leads to deterioration of translation quality. To determine if the potential
improvement results from embeddings’ fine-tuning or adding profession words to
the vocabulary, we evaluate the baseline, where the embedding layer is fine-
tuned without updating the tokenizer’s vocabulary.

#### Results

Table 1 shows that fine-tuning embedding layers improve the accuracy of
translating to the correct gender and decrease preference of male forms
($\Delta G$), while the quality of translation stays on a similar level as in
the original model. Protecting gender forms from splitting (by adding them to
the tokenizer’s vocabulary) only slightly further converges $\Delta G$ towards
zero for German while bringing over 2 points drop in BLEU. Performing the
vocabulary update before fine-tuning models deteriorates bias measures and
BLEU for Hebrew. For Spanish, $\Delta S$ improves while $\Delta G$ and BLEU
worsen.

Interestingly, fine-tuning the embedding layer increases the stereotypical
bias ($\Delta S$) for German and Spanish and decreases it only slightly for
Hebrew, suggesting that this method could not be sufficient for mitigating
this type of bias.

The results confirm our previous observation that the training (or fine-
tuning) dataset has more impact on gender bias encoded in the model than
subword splitting.

##  4 Related Work

#### Evaluation and Mitigation of Gender Bias in NMT

Escudé Font and Costa-jussà (2019) created a set of sentences of the format
"I’ve known [him/her/Mary/John] for a long time, my friend works as a
[profession]." They translated all sentences to Spanish and checked whether
"friend" was translated as "amigo" (male) or "amiga" (female). They found out
that "Him" is predicted at almost 100% accuracy for all models, but the
accuracy drops when predicting the word "her" on all models.

Stanovsky et al. (2019) created a mechanism to evaluate gender bias in machine
translation. They showed that almost all translation systems perform
significantly better on male roles.

The past works acknowledged the effect of gender form imbalance in training
corpus on gender bias manifested in the models. Specifically, Zmigrod et al.
(2019) reduce the bias by training the NMT systems on data augmented by adding
female forms. Saunders and Byrne (2020); Costa-jussà and de Jorge (2020)
propose a debiasing algorithm based on fine-tuning the model on a dataset with
a comparable number of male and female forms. These approaches are more
sustainable because they do not require training the model from scratch. These
approaches are in line with our observation that the imbalance of gender forms
in training data is the key source of gender bias in the model.

Savoldi et al. (2021) survey the methods for evaluating and mitigating bias in
machine translation, identify risks connected to them and propose direction
for their improvement.

Guo et al. (2022) propose an automatic prompt-based method to mitigate the
biases in pretrained language models. They identify biased prompts and propose
a distribution alignment loss to mitigate it.

#### The Role of Tokenization on System’s Performance

Domingo et al. (2018) show that tokenizers play a significant role in the
neural machine translation pipeline. The tokenization of the target language
affects evaluation measures (BLEU) by up to 20 points.

Bostrom and Durrett (2020b) compare the popular subword tokenizers: Unigram
Kudo (2018) and BPE Sennrich et al. (2016). They show that the former more
often splits words on morphological boundaries and thus can improve the
model’s performance on downstream tasks.

While the recent survey Mielke et al. (2021) shows that the choice of
effective tokenization method depends on the task, and there is no specific
tokenization algorithm suiting all applications.

The role of tokenization on gender bias was not widely evaluated in past
works. Libovický et al. (2022) showed that character-based translators to
morphologically rich languages (Czech and German) obtain similar bias results
as subword-based systems, even though female forms in these languages contain
relatively more characters. It aligns with our observation that tokenization
is not a significant source of bias.

Gaido et al. (2021) explored how segmenting methods influence gender bias in
speech translation models. Such models have different features, such as vocal
characteristics used to measure gender bias over all words, while our work
focuses on gender bias in profession words. In contrast to their work, we
focus on formal gender bias definitions and analyzing the relation between
frequency, accuracy, and number of tokens.

##  5 Discussion and Future Work

Our confirms the validity of the causal schema depicted in Figure 1 explaining
the lower translation accuracy for female and anti-stereotypical profession
names. We consciously analyzed the dataset of only non-ambiguous sentences
where the gender of each profession is known. This selection was made to
enable us to evaluate the accuracy of the ground-truth gender.

Furthermore, the stereotypical occupations are based on US Department of Labor
statistics, but it cannot be guaranteed that these same stereotypes are
present in other cultures. However, it can be considered a reasonable estimate
for other languages, as evidenced by the observation that non-stereotypical
professions appear less frequently in their training corpora. Future research
may analyze gender roles in target languages to corroborate these
observations.

Another future point of interest is mapping more factors for bias by isolating
more features. The dependencies of tokens, frequency, and gender bias will be
examined on a larger scale, with more words and different types of
tokenization. We also intend to broaden the scope of the analysis to other
languages and include neutral gender forms for the already analyzed languages.

##  6 Conclusions

Our study found that profession words in German, Hebrew, and Spanish tend to
be split into more tokens in the female form than in the male form. We then
investigated whether this phenomenon amplifies the tendency of NMT models to
translate male professions more accurately than female ones. Our results
showed that the frequency of gender forms confounds the relationship between
the number of tokens and gender bias in the training set. However, the number
of subword tokens per word can be used to estimate its frequency in the
unavailable training corpus.

The findings of our analysis of translation models were supported by the
trends observed in the results of the proposed debiasing method. Specifically,
we found that fine-tuning token embeddings on a gender-balanced dataset had a
more significant impact on reducing bias than updating the tokenizer’s
vocabulary with underrepresented gender forms.

These findings suggest that future research should also focus on other aspects
of NMT models in order to mitigate gender bias effectively.

## Acknowledgments

We thank our annotators for their immense help in collecting data: Johannes
Knaute, Kathy Hämmerl, Leonie Weissweiler, Uri Berger, Daniel Rotem, Almog
Guriel, Itamar Katz, Jonathan Posternak.

We also thank anonymous reviewers for their valuable comments on the previous
versions of this article. This work was partially supported by the Israeli
Ministry of Science and Technology (grant no. 2088) and by grant 23-06912S of
the Czech Science Foundation. Tomasz Limisiewicz’s visit to the Hebrew
University has been supported by grant 338521 of the Charles University Grant
Agency and the Mobility Fund of Charles University. We have been using
language resources and tools developed, stored, and distributed by the
LINDAT/CLARIAH-CZ project of the Ministry of Education, Youth and Sports of
the Czech Republic (project LM2018101).

## Limitations

As pointed out in the last section, tokenization can reinforce gender bias in
the models but is not the sole factor responsible for gender bias. We expect
that even if male and female forms were tokenized to the same number of
tokens, bias could still persist in the models.

The important aspect is the frequency of words in the training corpus, which
also affects tokenization. We did not have exact information on the model’s
training corpus. However, the analysis of the OPUS-100 dataset, which was
sampled from the full dataset, showed that male forms are much more frequent
than female ones.

In addition, we analyzed only 40 stereotypical professions from WinoMT, which
is only a subset of the whole dataset. Extending the analysis to other
professions without typical gender-role assigned can reveal slightly different
trends.

Finally, non-zero bias measures indicate the presence of bias in the model,
but the opposite implication is not true. Even in the case where both $\Delta
G$ and $\Delta S$ are close to zero, we cannot claim that the model is
unbiased, as there may be other bias manifestations of bias involved.

A significant limitation of this work is that we compare only male and female
forms of the professions while neglecting neutral forms. We expect neutral
forms to be even harder for the model to predict than female forms because of
the added suffixes that are rarely present in the training data. For instance,
German uses forms with “*” and female prefixes (e.g., “Mechaniker*in”) while
“\” is used in Hebrew for that purpose. Our initial analysis shows that
neutral forms are sporadic in the training corpora. For instance, in German,
only 7 neutral forms are using the inclusive suffix “*in”, compared to 1351
female forms of the analyzed professions. The proportion of
neutral:female:male forms is approximately 1:193:1845.

## Ethics Statement

We introduce new methods for evaluating and mitigating gender bias in machine
translation. We believe that this solution will incentivize the development of
fairer models in the future.

We do not identify any ethical risks connected to this work.

## References

  * Bostrom and Durrett (2020a) Kaj Bostrom and Greg Durrett. 2020a.  Byte pair encoding is suboptimal for language model pretraining.  _arXiv preprint arXiv:2004.03720_.
  * Bostrom and Durrett (2020b) Kaj Bostrom and Greg Durrett. 2020b.  Byte pair encoding is suboptimal for language model pretraining.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 4617–4624, Online. Association for Computational Linguistics.
  * Costa-jussà and de Jorge (2020) Marta R. Costa-jussà and Adrià de Jorge. 2020.  Fine-tuning neural machine translation on gender-balanced datasets.  In _Proceedings of the Second Workshop on Gender Bias in Natural Language Processing_ , pages 26–34, Barcelona, Spain (Online). Association for Computational Linguistics.
  * Domingo et al. (2018) Miguel Domingo, Mercedes García-Martínez, Alexandre Helle, Francisco Casacuberta, and Manuel Herranz. 2018.  How much does tokenization affect neural machine translation?  _ArXiv_ , abs/1812.08621.
  * Escudé Font and Costa-jussà (2019) Joel Escudé Font and Marta R. Costa-jussà. 2019.  Equalizing gender bias in neural machine translation with word embeddings techniques.  In _Proceedings of the First Workshop on Gender Bias in Natural Language Processing_ , pages 147–154, Florence, Italy. Association for Computational Linguistics.
  * Gaido et al. (2021) Marco Gaido, Beatrice Savoldi, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2021.  How to split: the effect of word segmentation on gender bias in speech translation.  _arXiv preprint arXiv:2105.13782_.
  * Guo et al. (2022) Yue Guo, Yi Yang, and Ahmed Abbasi. 2022.  Auto-debias: Debiasing masked language models with automated biased prompts.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1012–1023, Dublin, Ireland. Association for Computational Linguistics.
  * Hovy and Spruit (2016) Dirk Hovy and Shannon L. Spruit. 2016.  The social impact of natural language processing.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 591–598, Berlin, Germany. Association for Computational Linguistics.
  * Jonckheere (1954) A. R. Jonckheere. 1954.  A distribution-free k-sample test against ordered alternatives.  _Biometrika_ , 41:133–145.
  * Junczys-Dowmunt et al. (2018) Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, and Alexandra Birch. 2018.  Marian: Fast neural machine translation in C++.  In _Proceedings of ACL 2018, System Demonstrations_ , pages 116–121, Melbourne, Australia. Association for Computational Linguistics.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Kudo (2018) Taku Kudo. 2018.  Subword regularization: Improving neural network translation models with multiple subword candidates.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 66–75, Melbourne, Australia. Association for Computational Linguistics.
  * Kudo and Richardson (2018) Taku Kudo and John Richardson. 2018.  SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 66–71, Brussels, Belgium. Association for Computational Linguistics.
  * Libovický et al. (2022) Jindřich Libovický, Helmut Schmid, and Alexander Fraser. 2022.  Why don’t people use character-level machine translation? In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 2470–2485, Dublin, Ireland. Association for Computational Linguistics.
  * Mielke et al. (2021) Sabrina J. Mielke, Zaid Alyafeai, Elizabeth Salesky, Colin Raffel, Manan Dey, Matthias Gallé, Arun Raja, Chenglei Si, Wilson Y. Lee, Benoît Sagot, and Samson Tan. 2021.  Between words and characters: A brief history of open-vocabulary modeling and tokenization in nlp.
  * Saunders and Byrne (2020) Danielle Saunders and Bill Byrne. 2020.  Reducing gender bias in neural machine translation as a domain adaptation problem.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7724–7736, Online. Association for Computational Linguistics.
  * Savoldi et al. (2021) Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2021.  Gender Bias in Machine Translation.  _Transactions of the Association for Computational Linguistics_ , 9:845–874.
  * Sennrich et al. (2016) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016.  Neural machine translation of rare words with subword units.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1715–1725, Berlin, Germany. Association for Computational Linguistics.
  * Stanovsky et al. (2019) Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019.  Evaluating gender bias in machine translation.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 1679–1684, Florence, Italy. Association for Computational Linguistics.
  * Tang et al. (2020) Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020.  Multilingual translation with extensible multilingual pretraining and finetuning.  _arXiv preprint arXiv:2008.00401_.
  * Tiedemann and Thottingal (2020) Jörg Tiedemann and Santhosh Thottingal. 2020.  OPUS-MT — Building open translation services for the World.  In _Proceedings of the 22nd Annual Conferenec of the European Association for Machine Translation (EAMT)_ , Lisbon, Portugal.
  * Zhang et al. (2020) Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. 2020.  Improving massively multilingual neural machine translation and zero-shot translation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1628–1639, Online. Association for Computational Linguistics.
  * Zhao et al. (2018) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018\.  Gender bias in coreference resolution: Evaluation and debiasing methods.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 15–20, New Orleans, Louisiana. Association for Computational Linguistics.
  * Zmigrod et al. (2019) Ran Zmigrod, Sabrina J. Mielke, Hanna Wallach, and Ryan Cotterell. 2019.  Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 1651–1661, Florence, Italy. Association for Computational Linguistics.

##  Appendix A Human Translation

###  A.1 Post-processing of Collected Data

The post-processing of the human translations was done according to the
following rules: First, we kept the translations suggested by at least two
translators. If there’s no translation option suggested at least two times, we
decided which translations to keep according to our knowledge of the language.
This happened only for the following translations in Hebrew: “attendant”,
“construction worker”, and “analyst”. The translation selection for those
words was made by an author who is a native Hebrew speaker. For professions
where translators propose the same word as both male and female translations,
we keep those as valid pairs.

In case spelling is inconsistent among annotators, we kept the more common
spelling while fixing pronunciation mistakes according to our knowledge.
Lastly, we keep at most five pairs of translations per profession.

###  A.2 Instructions for Annotators

The goal of the task is to get the correct translation of profession words
into your language for both Male and Female forms.

  1. 1.

Select the language you want to translate to (your native language: German,
Spanish, or Hebrew).

  2. 2.

For each word in English, write up to 3 translations both in Male and Female
form.

  3. 3.

Organize the translation into pairs that differ only in endings (e.g., words:
Berater and Beraterin are a pair but Beraterin and Ratgaber are not a pair in
German). Put these words in the neighboring cells (“Translation N Male" and
"Translation N Female").

  4. 4.

in case there is no translation in the other gender that differ only in an
ending, leave the other cell of the pair blank.

  5. 5.

If you propose less than 3 translations in each gender, leave the last cells
in a row empty.

##  Appendix B Details of Fine-tuning

In fine-tuning, we train only the translation model’s input/output embedding
layer while keeping the rest of the parameters frozen. We further trained the
model for three epochs on the gender-balanced dataset (Section 2.2) with
batches consisting of 16 examples. For optimization, we used Adam Kingma and
Ba (2015) with default parameters with a constant learning rate of $5e-5$. The
fine-tuning took about 10 minutes on Nvidia A40 GPU.

##  Appendix C mBART results

(a) German

(b) Spanish

(c) Hebrew

Figure 7: mBART: Human translated profession names grouped by the number of
tokens they were split into. On the x-axis: number of tokens per word. On the
y-axis: the count of male and female forms professions in each of the groups.
Male forms tend to be split into fewer tokens than female forms.

(a) German

(b) Spanish

(c) Hebrew

Figure 8: mBART: Human translated profession names grouped by the number of
tokens they were split into. On the x-axis: number of tokens per word. On the
y-axis: the count of pro- and anti-stereotypical forms of professions in each
of the groups. Pro-stereotypical forms tend to be split into fewer tokens than
anti-stereotypical forms.

(a) German

(b) Spanish

(c) Hebrew

Figure 9: mBART: $\Delta G$ as the difference between F1-score for male and
female test instances for each of paired translations. $\Delta T$ is the
difference between the number of tokens in a male and a female form. The
median is marked by an orange line.

(a) German

(b) Spanish

(c) Hebrew

Figure 10: mBART: $\Delta S$ as the difference between F1-score for pro- and
anti-stereotypical test instances for each of paired translations. $\Delta T$
is the difference between number of tokens in a pro- and an anti-stereotypical
translation. Median is marked by an orange line.

We repeat the analysis of the number of tokens per word described in Section
3.1 for mBART. Figures 7 and 8 show that similarly to Opus models, mBART split
female and anti-stereotypical words in the target language into more tokens.
Noticeably, almost no female words, in all the languages are represented as
one token.

Figures 9 and 10 show similar trends for $\Delta G$ and $\Delta S$ against
$\Delta T$ as observed in Section 3.2 for OPUS models.

##  Appendix D WinoMT details

The WinoMT dataset that we used for evaluation contains 3,888 sentences. Each
sentence contains a profession and a pronoun that describes the gender of the
profession. The dataset is equally balanced between male and female examples
and also between stereotypical and non-stereotypical gender-role assignments.
Specifically, there are 1826 sentences with a male pronoun assigned to the
profession and 1822 sentences with a female pronoun. The remaining 240
sentences are gender-neutral, i.e., they contain the pronoun “they”.

Note that this dataset contains only non-ambiguous sentences where the gender
of each profession is known. This selection was made to enable us to evaluate
the accuracy of the ground-truth gender.


# Star-topology Registers: NMR and Quantum Information Perspectives

T S Mahesh<EMAIL_ADDRESS>Deepak Khurana  Current Affiliation:
Section of Quantum Physics and Information Technology (QPIT), Department of
Physics, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark. Email:
<EMAIL_ADDRESS> Krithika V R<EMAIL_ADDRESS>Sreejith G J
<EMAIL_ADDRESS>C S Sudheer Kumar
<EMAIL_ADDRESS>Department of Physics and NMR Research
Center,
Indian Institute of Science Education and Research, Pune 411008, India

###### Abstract

Quantum control of large spin registers is crucial for many applications
ranging from spectroscopy to quantum information. A key factor that determines
the efficiency of a register for implementing a given information processing
task is its network topology. One particular type, called star-topology,
involves a central qubit uniformly interacting with a set of ancillary qubits.
A particular advantage of the star-topology quantum registers is in the
efficient preparation of large entangled states, called NOON states, and their
generalized variants. Thanks to the robust generation of such correlated
states, spectral simplicity, ease of polarization transfer from ancillary
qubits to the central qubit, as well as the availability of large spin-
clusters, the star-topology registers have been utilized for several
interesting applications over the last few years. Here we review some recent
progress with the star-topology registers, particularly via nuclear magnetic
resonance methods.

##  I Introduction

The long-lasting quantum memory of nuclear spins is at the heart of versatile
applications of nuclear magnetic resonance from spectroscopy and biomedical
imaging to quantum information processing. While remarkable progress has been
achieved with few-spin systems forming small quantum registers, scaling the
register size has been a daunting task. The challenges include the highly
mixed state of NMR ensembles, spectral complexity of large registers that
limit their quantum control, and decoherence causing the loss of quantum
memory. In this review, we explain how certain symmetries, particularly the
star-symmetry, can provide a way forward to realize large quantum registers,
albeit for specific applications.

Quantum registers may be categorized based on the network-topology of qubits.
Some common topologies are shown in Fig. 1. In principle, linear and cyclic
chains are sufficient to realize universal quantum gates, while all to all
topology is most efficient, since it allows a direct transport of information
from one qubit to any other. As we scale up the size of the register though,
it is hard to maintain all to all interactions, and it will become necessary
to consider a practical topology that is best suited for a particular
application. In this review, we focus on the star-topology registers (STRs),
which consist of a central qubit uniformly interacting with a set of ancillary
qubits. We shall discuss various aspects of star-topology registers, mainly
from the perspectives of nuclear magnetic resonance (NMR) and quantum
information. Although here we treat a qubit synonymous with a spin-1/2
particle, the underlying principles hold in non-NMR architectures.

For completeness, we shall first introduce the theoretical and spectroscopic
aspects of the STRs in the following section. In section III, we shall review
recent progress with utilizing the STRs for various applications. Finally, we
shall summarize in section IV.

Figure 1: Some common topologies of quantum registers: (a) linear chain, (b)
cyclic chain, (c) all-to-all, (d) central-spin or star-topology register
(STR), and (e) double-STR.

##  II Star-topology register (STR)

As illustrated in Fig. 1(d), an STR consists of a central qubit (C) directly
connected to a set of ancillary (A) or satellite qubits. Many of the STRs
realized by nuclear spin-systems via liquid-state NMR techniques have the
following properties: (i) no net effective interactions among the ancillary
qubits; (ii) the central qubit uniformly interacts with each of the ancillary
qubits; (iii) the central qubit is realized by a different nuclear isotope and
hence can be selectively addressed; (iv) the ancillary qubits are
indistinguishable from one another. The last point implies that the ancillary
qubits can only be collectively addressed, but no selective addressing of a
particular ancillary qubit is possible. While this fact limits the type of
quantum gates that can be realized, crucially, it also brings about spectral
simplicity that allows us to conveniently handle much larger registers.

We shall consider an STR with $N-1$ ancillary qubits surrounding a central
qubit. For such a system, the NMR Hamiltonian (in units of $\hbar$) can be
written as levitt2013spin

$\displaystyle H_{0}$ $\displaystyle=$ $\displaystyle H_{C}+H_{A}+H_{CA}$ (1)
$\displaystyle=$ $\displaystyle\omega_{C}I_{z}^{C}+\omega_{A}I_{z}^{A}+2\pi
J_{CA}I_{z}^{C}I_{z}^{A},$

where $\omega_{C}$ and $\omega_{A}$ are the Larmor frequencies of $C$ and $A$
qubits, $J_{CA}$ is the fixed strength of the indirect spin-spin interaction
that is characteristic of the spin-system, $I_{z}^{C}$ is the spin operator
for the central qubit and $I_{z}^{A}=\sum_{k=1}^{N-1}I_{z}^{A,k}$ is the
collective spin operator for the ancillary qubits. The Larmor frequencies are
proportional to the strength of the magnetic field $B_{0}$,

$\displaystyle\omega_{C}=-\gamma_{C}B_{0}~{}~{}\mbox{and}~{}~{}\omega_{A}=-\gamma_{A}B_{0},$
(2)

where $\gamma_{C}$ and $\gamma_{A}$ are the gyromagnetic ratios of the nuclei.

The eigenstates of STR are Zeeman product states of individual qubit-states
$|{\uparrow_{z}}\rangle\equiv|{0}\rangle$ and
$|{\downarrow_{z}}\rangle=|{1}\rangle$. Thus, we label the STR eigenstates
with $N$-bit binary strings and form a computational basis. Each binary string
is associated with a Hamming weight $\mathbbm{h}$ that counts the number of
1’s in the string. The labeling is chosen such that ascending energy levels
correspond to ascending Hamming weights and degenerate levels get the same
Hamming weight. Levels of same Hamming weight are collectively represented by
$|{N-\mathbbm{h},\mathbbm{h}}\rangle$. Similar labeling schemes had been
conveniently employed for information encoding purposes even when the
eigenstates are not product states sinha2001toward; mahesh2002ensemble;
murali2002quantum. The total spin magnetic moment quantum number for the state
vector $|{N-\mathbbm{h},\mathbbm{h}}\rangle$ is thus

$\displaystyle m_{\mathbbm{h}}=\frac{N}{2}-\mathbbm{h}.$ (3)

Similarly, if ${\tt h}$ is the Hamming weight for the ancillary part of the
state vector, i.e., $|{N-1-{\tt h},{\tt h}}\rangle_{A}$, then

$\displaystyle m_{{\tt h}}^{A}=\frac{N-1}{2}-{\tt h},$ (4)

is the total ancillary spin magnetic moment quantum number. As illustrated in
Fig. 2, the energy eigenstates form two subspaces corresponding to
$|{0}\rangle_{C}$ and $|{1}\rangle_{C}$ states of the central qubit:

Eigenstates ${\tt h}\in\\{0,1,\cdots,N-1\\}$ |  Total magnetic quantum number |  Eigenvalue
---|---|---
$|{0}\rangle_{C}|{N-1-{\tt h},{\tt h}}\rangle_{A}$ |  $m_{0,{\tt h}}=\frac{1}{2}+m_{{\tt h}}^{A}$ |  $+\omega_{C}/2+m_{{\tt h}}^{A}(\omega_{A}+\pi J_{CA})$
$|{1}\rangle_{C}|{N-1-{\tt h},{\tt h}}\rangle_{A}$ |  $m_{1,{\tt h}}=-\frac{1}{2}+m_{{\tt h}}^{A}$ |  $-\omega_{C}/2+m_{{\tt h}}^{A}(\omega_{A}-\pi J_{CA})$

The selection rule for the radio frequency (RF) excitation allows only
transitions with a unit change in the total spin magnetic moment quantum
number. Thus, all inter-subspace transitions of $C$-spin are allowed, while
$A$-spin transitions within each subspace satisfy $\Delta m_{\tt
h}^{A}=-\Delta{\tt h}=\pm 1$. As a result, C-qubit possesses $N$ distinct
transition frequencies $\omega_{C}+2\pi J_{CA}m_{\tt h}^{A}$ (gray arrows in
Fig. 2). On the other hand, A-qubits collectively have only two distinct
transition frequencies, $\omega_{A}+\pi J_{CA}$ in the $|{0}\rangle_{C}$
subspace and $\omega_{A}-\pi J_{CA}$ in the $|{1}\rangle_{C}$ subspace. Fig. 3
shows the experimental spectrum of 31P spin coupled with nine ancillary 1H
spins in trimethyl phosphite.

Figure 2: Energy levels of an STR forming two subspaces corresponding to the
$|{0}\rangle_{C}$ and $|{1}\rangle_{C}$ states of the central spin. The
transitions of $A$ and $C$ are indicated by blue and gray arrows respectively.
Figure 3: 31P spectra of trimethylphosphite corresponding to single quantum
excitation from thermal equilibrium (upper trace) and corresponding to the
NOON state (lower trace). The molecular structure of trimethylphosphite shows
the central 31P spin (in pink) that is coupled to nine 1H spins (in white) via
an indirect scalar coupling of $J_{CA}=11$ Hz.

Owing to mathematical ease, most of the NMR experiments are described in a
rotating frame, that precesses with the applied RF drive, thereby rendering it
time-independent. Since $C$ and $A$ are of different nuclear isotopes, their
Larmor frequencies differ over several tens or even hundreds of megahertz.
This allows them to be controlled by independent radio-wave channels. In a
doubly rotating frame, precessing at the respective carrier frequencies, the
Hamiltonian is of the form,

$\displaystyle H$ $\displaystyle=$
$\displaystyle-2\pi\nu_{C}I_{z}^{C}-2\pi\nu_{A}I_{z}^{A}$ (5)
$\displaystyle+2\pi
J_{CA}I_{z}^{C}I_{z}^{A}+\Omega_{C}(I_{x}^{C}\cos\phi_{C}+I_{y}^{C}\sin\phi_{C})$
$\displaystyle+\Omega_{A}(I_{x}^{A}\cos\phi_{A}+I_{y}^{A}\sin\phi_{A}).$

Here $\nu_{C},~{}\nu_{A}$ are resonance offsets, $\Omega_{C},~{}\Omega_{A}$
are RF amplitudes on the respective channels, and $\phi_{C},~{}\phi_{A}$ are
the RF phases, all of which being tunable experimental parameters.

###  II.1 Thermal state with low purity

Under normal NMR conditions, thermal energy is much higher than the Zeeman
energy gaps, so that for a given spin-1/2 nucleus,

$\displaystyle\rho=\frac{1}{2}e^{-H/kT}\approx\frac{1}{2}\mathbbm{1}+\epsilon
I_{z}$ (6)

where $\epsilon=\hbar\gamma B_{0}/(kT)\ll 1$ is known as the purity factor,
and $\mathbbm{1}$ is the $2\times 2$ identity matrix. Defining
$\epsilon_{C}=\hbar\gamma_{C}B_{0}/(kT)$ and
$\epsilon_{A}=\hbar\gamma_{A}B_{0}/(kT)$, we obtain

$\displaystyle\rho_{C}$ $\displaystyle=$
$\displaystyle\frac{1+\epsilon_{C}}{2}|{0}\rangle\langle{0}|+\frac{1-\epsilon_{C}}{2}|{1}\rangle\langle{1}|$
$\displaystyle\rho_{A}$ $\displaystyle=$
$\displaystyle\left[\frac{1+\epsilon_{A}}{2}|{0}\rangle\langle{0}|+\frac{1-\epsilon_{A}}{2}|{1}\rangle\langle{1}|\right]^{\otimes(N-1)},~{}~{}\mbox{so
that}$ $\displaystyle\rho_{CA}$ $\displaystyle=$
$\displaystyle\rho_{C}\otimes\rho_{A}$ (7) $\displaystyle\approx\sum_{{\tt
h}=0}^{N-1}p_{0,{\tt h}}\rho_{0,{\tt h}}+p_{1,{\tt h}}\rho_{1,{\tt h}}.$

Here $\rho_{0,{\tt h}}$ corresponds to an uniform mixture of degenerate levels
$|{0}\rangle_{C}|{N-1-{\tt h},{\tt h}}\rangle$ of $|{0}\rangle_{C}$ subspace,
while $\rho_{1,{\tt h}}$ corresponds to that of $|{1}\rangle_{C}|{N-1-{\tt
h},{\tt h}}\rangle$ of $|{1}\rangle_{C}$ subspace. At low purity, i.e.,
$\\{\epsilon_{A},\epsilon_{C}\\}\ll 1$, the relative populations are

$\displaystyle p_{0,{\tt h}}$ $\displaystyle\approx$
$\displaystyle{{N-1}\choose{{\tt h}}}\frac{1+\epsilon_{C}+2m_{{\tt
h}}^{A}\epsilon_{A}}{2^{N}}~{}~{}\mbox{and,}$ $\displaystyle p_{1,{\tt h}}$
$\displaystyle\approx$ $\displaystyle{{N-1}\choose{{\tt
h}}}\frac{1-\epsilon_{C}+2m_{{\tt h}}^{A}\epsilon_{A}}{2^{N}},$ (8)

where ${{N-1}\choose{{\tt h}}}=\frac{(N-1)!}{(N-1-{\tt h})!{\tt h}!}$ denotes
the binomial coefficient.

Thus the thermal magnetizations are

$\displaystyle M_{C}$ $\displaystyle=$ $\displaystyle\sum_{{\tt
h}=0}^{N-1}(p_{0,{\tt h}}-p_{1,{\tt
h}})/2=\frac{\epsilon_{C}}{2},~{}\mbox{for}~{}C~{}\mbox{spin, and}$
$\displaystyle M_{A}$ $\displaystyle=$ $\displaystyle\sum_{{\tt
h}=0}^{N-1}(p_{0,{\tt h}}+p_{1,{\tt h}})m_{{\tt
h}}^{A}=(N-1)\frac{\epsilon_{A}}{2}~{}\mbox{for}~{}A~{}\mbox{spins}.~{}~{}~{}~{}~{}$
(9)

Later, we shall see how we can exploit the large reserve magnetization present
in the ancillary qubits.

##  III Preparing many-body entanglement: NOON and MSSM states

One of the key advantages of STRs is the efficiency with which the central
qubit can get correlated with the ancillary qubits. Since the central qubit is
uniformly coupled to all the indistinguishable ancillary qubits, a single CNOT
gate can simultaneously act on all the ancillary qubits, thus offering the
most efficient way to prepare many-body entanglement. We shall consider the
following cases.

###  III.1 Pure ground state

In this case, each of qubits is initially prepared in the ground state, i.e.,
$|{\psi_{C}}\rangle=|{0}\rangle$ and $|{\psi_{A}}\rangle=|{0\cdots
0}\rangle=|{N-1,0}\rangle$. Now consider the following circuit.


††thanks: Corresponding author

# General Relativistic Fluctuation Theorems

Yifan Cai<EMAIL_ADDRESS>School of Physics, Nankai University,
Tianjin 300071, China  Tao Wang<EMAIL_ADDRESS>School of Physics,
Nankai University, Tianjin 300071, China  Liu Zhao<EMAIL_ADDRESS>School
of Physics, Nankai University, Tianjin 300071, China

(July 13, 2024)

###### Abstract

Using the recently proposed covariant framework of general relativistic
stochastic mechanics and stochastic thermodynamics, we proved the detailed and
integral fluctuation theorems in curved spacetime. The time-reversal
transformation is described as a transformation from the perspective of
future-directed observer to that of the corresponding past-directed observer,
which enables us to maintain general covariance throughout the construction.
The result presented in this work may be applied in understanding the origin
of irreversibility in macroscopic processes in the presence of relativistic
gravity, as in most astrophysical or cosmological processes.

Langevin equation, fluctuation theorem, time-reversal symmetry, general
relativity

_Introduction_. – A fundamental problem in modern statistical physics is the
emergence of macroscopic irreversibility in systems which have time-reversal
symmetry (TRS) in the underlying microscopic description. The relevant
researches can be traced back to Boltzmann, who employed a scattering model
with TRS in deriving the H-theorem [1], which states that the entropy of a
macroscopic system cannot decrease in the course of time. However, the
H-theorem has been under debates and challenges ever since its birth. The most
famous challenge is known as Loschmidt paradox [2, 3], which argues that, if
an entropy-increasing process exists for a system, the underlying TRS should
also permit a corresponding entropy-decreasing process. The fluctuation
theorems, proposed in the 1990s [4, 5, 6], offer an interpretation for
Loschmidt paradox. Basically, these theorems state that, while both entropy-
increasing and entropy-decreasing processes are permissible, the former is
more probable.

In the context of stochastic mechanics, Sekimoto [7] utilized the overdamped
Langevin equation to properly classify the energy exchange of a Brownian
particle with the heat reservoir into trajectory heat and work, and thus
establishes the first law of stochastic thermodynamics on the trajectory
level. This allows for the construction of fluctuation theorems based on
stochastic mechanics [8, 9, 10, 11, 12].

The initial studies on fluctuation theorems are mostly carried out in non-
relativistic theories. Since 2007, some attempts [13, 14, 15, 16, 17, 18] in
establishing fluctuation theorems in the special relativistic context
appeared. Such attempts are important because relativity imposes stronger
protection on the spacetime symmetry, making it harder to break the TRS. The
purpose of the present work is to broaden the scope of the fluctuation theorem
to encompass curved spacetime while maintaining general covariance. This is
also important, because, on the one hand, gravity is a universal interaction,
it is desirable to see whether gravity has any impact on the origin of
irreversibility — another universal phenomenon that appear in the scope of
macroscopic theories; on the other hand, the choice of time parameter is more
subtle in general relativistic theories than in special relativistic cases.

Conventionally, the time-reversal transformation (TRT) is merely described as
a transformation of the time parameter, $t\mapsto-t$. However, if the time
parameter is identified to be the coordinate time $x^{0}$, such a
transformation will result in a lack of covariance. Our covariant framework
[19, 20] for relativistic stochastic mechanics urges that the choice of time
parameter should be closely linked with the observer. Consequently, the TRT
needs be realized as a change from the perspective of a future-directed
observer to that of a past-directed observer. In such a realization, the
coordinate system is left intact.

_Relativistic Langevin dynamics_. – Langevin equation describes the motion of
a heavy particle, referred to as the Brownian particle, under the random
disturbance of a heat reservoir. Conventionally, Langevin equation can be
expressed in the form of Newton’s second law, incorporating elements such as
the random force, damping force, and various other external forces. For a
relativistic particle of mass $m$ and charge $q$ moving in $(d+1)$-dimensional
spacetime $\mathcal{M}$ with the metric $g_{\mu\nu}$, the Langevin equation
employing the particle’s proper time $\tau$ as evolution parameter is is
referred to as LEτ [19],

$\displaystyle{\rm d}\tilde{x}_{\tau}^{\mu}$
$\displaystyle=\frac{\tilde{p}_{\tau}^{\mu}}{m}{\rm d}\tau,$ (1)
$\displaystyle{\rm d}\tilde{p}_{\tau}^{\mu}$
$\displaystyle=\xi^{\mu}_{\tau}{\rm d}\tau+\mathcal{F}_{\text{dp}}^{\mu}{\rm
d}\tau-\frac{1}{m}\varGamma^{\mu}{}_{\alpha\beta}\tilde{p}_{\tau}^{\alpha}\tilde{p}_{\tau}^{\beta}{\rm
d}\tau+\mathcal{F}_{\text{em}}^{\mu}{\rm d}\tau,$ (2)

where $\xi^{\mu}_{\tau}$ is the random force,
$\mathcal{F}_{\text{dp}}^{\mu}:=\mathcal{K}^{\mu\nu}U_{\nu}$ is the damping
force in which the damping coefficient tensor $\mathcal{K}^{\mu\nu}$ obeys
$\mathcal{K}^{\mu}{}_{\nu}p^{\nu}=0$,
$\displaystyle\mathcal{F}_{\text{em}}^{\mu}:=\frac{q}{m}F^{\mu}{}_{\nu}\tilde{p}^{\nu}_{\tau}$
is the electromagnetic force, and $\varGamma^{\mu}{}_{\alpha\beta}$ denotes
the Christoffel connection associated with $g_{\mu\nu}$. Tilded symbols such
as $\tilde{x}_{\tau}$ and $\tilde{p}_{\tau}$ represent random variables, and
the un-tilded ones represent their realization. The random force is consisted
of a term encoding the Stratonovich coupling between the stochastic amplitudes
$\mathcal{R}^{\mu}{}_{\mathfrak{a}}$ (which transforms as a vector field for
each fixed $\mathfrak{a}$) with a set of $d$ independent Wiener process ${\rm
d}\tilde{w}_{\tau}^{\mathfrak{a}}$ obeying the probability distribution

$\displaystyle\Pr[{\rm d}\tilde{w}^{\mathfrak{a}}_{\tau}={\rm
d}w^{\mathfrak{a}}]=\frac{1}{(2\pi{\rm
d}\tau)^{d/2}}\exp\left[-\frac{\delta_{\mathfrak{a}\mathfrak{b}}{\rm
d}w^{\mathfrak{a}}{\rm d}w^{\mathfrak{b}}}{2{\rm d}\tau}\right]$ (3)

of variance ${\rm d}\tau$, together with a term incorporating an additional
stochastic force
$\displaystyle\mathcal{F}^{\mu}_{\text{add}}:=\frac{\delta^{\mathfrak{a}\mathfrak{b}}}{2}\mathcal{R}^{\mu}{}_{\mathfrak{a}}\nabla^{(h)}_{i}\mathcal{R}^{i}{}_{\mathfrak{b}}$,

$\displaystyle\xi_{\tau}^{\mu}:=\mathcal{R}^{\mu}{}_{\mathfrak{a}}\circ_{S}{\rm
d}\tilde{w}^{\mathfrak{a}}_{\tau}/{\rm d}\tau+\mathcal{F}^{\mu}_{\text{add}},$
(4)

where $\nabla^{(h)}$ is the covariant derivative on the mass shell and
$\mathcal{R}^{\mu}{}_{\mathfrak{a}}$ stands for the stochastic amplitude which
may depend on $\tilde{x}_{\tau}^{\mu}$ and $\tilde{p}_{\tau}^{\mu}$ and
transform as a vector field for each fixed $\mathfrak{a}$. The Stratonovich
coupling $\circ_{S}$ maintains the chain rule in stochastic calculus, making
the general covariance of LEτ self-evident.

The energy of a relativistic particle measured by the observer with proper
velocity $Z^{\mu}$ is defined as

$\displaystyle E(x,p):=-Z_{\mu}(x)p^{\mu}.$ (5)

As the microstate of the Brownian particle evolves in a stochastic process,
the energy of the Brownian particle also becomes a stochastic process
$\tilde{E}_{\tau}:=E(\tilde{x}_{\tau},\tilde{p}_{\tau})$. The chain rule
implies that

$\displaystyle{\rm d}\tilde{E}_{\tau}$ $\displaystyle=\frac{\partial
E}{\partial x^{\mu}}{\rm d}\tilde{x}^{\mu}_{\tau}+\frac{\partial E}{\partial
p^{\mu}}{\rm d}\tilde{p}^{\mu}_{\tau}$
$\displaystyle=-Z_{\mu}\left[\xi_{\tau}^{\mu}+\mathcal{F}_{\text{dp}}^{\mu}\right]{\rm
d}\tau-\frac{\tilde{p}_{\tau}^{\mu}\tilde{p}_{\tau}^{\nu}}{m}\nabla_{\nu}Z_{\mu}{\rm
d}\tau-Z_{\mu}\mathcal{F}_{\text{em}}^{\mu}{\rm d}\tau.$ (6)

The second and last terms in eq. (General Relativistic Fluctuation Theorems)
are recognized to be the gravitational [21] and electromagnetic works,

$\displaystyle{\rm
d}\tilde{\mathcal{P}}_{\tau}:=-\frac{\tilde{p}_{\tau}^{\mu}\tilde{p}_{\tau}^{\nu}}{m}\nabla_{\nu}Z_{\mu}{\rm
d}\tau,\quad{\rm
d}\tilde{\mathcal{W}}_{\tau}:=-Z_{\mu}\mathcal{F}^{\mu}_{\text{em}}{\rm
d}\tau,$ (7)

and the first term incorporates the energy change caused by the impact of the
heat reservoir, which is by definition the heat received by the Brownian
particle,

$\displaystyle{\rm
d}\tilde{Q}_{\tau}:=-Z_{\mu}\left[\xi_{\tau}^{\mu}+\mathcal{F}_{\text{dp}}^{\mu}\right]{\rm
d}\tau.$ (8)

Therefore, eq. (General Relativistic Fluctuation Theorems) becomes precisely
the first law of general relativistic stochastic thermodynamics,

$\displaystyle{\rm d}\tilde{E}_{\tau}={\rm d}\tilde{\mathcal{Q}}_{\tau}+{\rm
d}\tilde{\mathcal{P}}_{\tau}+{\rm d}\tilde{\mathcal{W}}_{\tau}.$ (9)

_Time-reversal symmetry_. – Given a worldline $x_{\tau}$ for a massive
particle in the spacetime $\mathcal{M}$, its tangent vector $p^{\mu}_{\tau}$
can be either aligned with the proper velocity $Z_{\mu}$ of some chosen
observer Alice, _i.e._ $p^{\mu}_{\tau}Z_{\mu}<0$, or opposite to the proper
velocity $C_{\mu}$ of some other observer, Carol, _i.e._
$p^{\mu}_{\tau}C_{\mu}>0$. We will refer to Alice as future-directed and to
Carol as past-directed. To be more specific, we fix Carol to be the TRT image
of Alice, _i.e._ $C_{\mu}=-Z_{\mu}$. If Alice perceives the worldline evolving
from $t_{I}$ to $t_{F}$, then Carol will perceive the worldline evolving from
$t_{F}$ to $t_{I}$. This leads to a change in the sign of the time
derivatives, e.g. ${\rm d}/{\rm d}\tau\rightarrow-{\rm d}/{\rm d}\tau$, and
consequently the momentum $p$ needs to reverse its sign. Notice however that
${\rm d}\tau$ resp. ${\rm d}t$ denote the size of infinitesimal temporal
steps, both remain unchanged under the TRT.

The mass shell bundle $\Gamma_{m}:=\\{(x,p)\in
T\mathcal{M}|p_{\mu}p^{\mu}=-m^{2}\\}$ can be separated into two non-path-
connected regions, _i.e._ the future mass shell bundle

$\displaystyle\Gamma^{+}_{m}:=\\{(x,p)\in\Gamma_{m}|Z_{\mu}p^{\mu}<0\\},$ (10)

and the past mass shell bundle

$\displaystyle\Gamma^{-}_{m}:=\\{(x,p)\in\Gamma_{m}|Z_{\mu}p^{\mu}>0\\},$ (11)

both are defined relative to Alice. Clearly, there is a homeomorphism between
$\Gamma^{+}_{m}$ and $\Gamma^{-}_{m}$:

$\displaystyle I:\Gamma^{+}_{m}\to\Gamma^{-}_{m},\qquad(x,p)\mapsto(x,-p).$
(12)

This homeomorphism is the mathematical realization of the TRT. The phase
trajectory $Y_{t}=(y_{t},k_{t})$ of the Brownian particle is the uplift of the
worldline into $\Gamma^{+}_{m}$, and is referred to as the forward trajectory,
while the TRT image of the phase trajectory $Y^{-}_{t}=I(Y_{t_{I}+t_{F}-t})$
is the uplift of the worldline into $\Gamma^{-}_{m}$, and is known as the
reversed trajectory.

Now consider a macroscopic system consisting of a great number of massive
charged particles which can be classified into several species. The
electromagnetic field is produced by the charges carried by the particles and
the geometry of the $(d+1)$-dimensional spacetime is determined by the masses
and charges within the system. The phase trajectories of these particles obey
the following equations,

$\displaystyle\frac{{\rm d}x^{\mu}_{s}}{{\rm d}\tau_{s}}$
$\displaystyle=\frac{p^{\mu}_{s}}{m_{s}},$ (13) $\displaystyle\frac{{\rm
d}p^{\mu}_{s}}{{\rm d}\tau_{s}}$
$\displaystyle=\frac{q_{s}}{m_{s}}F^{\mu}{}_{\nu}p^{\nu}_{s}-\frac{1}{m_{s}}\varGamma^{\mu}{}_{\alpha\beta}p^{\alpha}_{s}p^{\beta}_{s},$
(14)

where the suffices $s$ indicate different species. Assuming that each species
of particles obeys a distinct TRT invariant distribution
$\Phi_{s}(x_{s},p_{s})$, the electric current and the energy-momentum tensor
contributed by the particles can be written as

$\displaystyle J^{\mu}$
$\displaystyle=\sum_{s}\int\varpi_{s}q_{s}p^{\mu}_{s}\Phi_{s},$ (15)
$\displaystyle\qquad T^{\mu\nu}_{\text{pa}}$
$\displaystyle=\sum_{s}\int\varpi_{s}p^{\mu}_{s}p^{\nu}_{s}\Phi_{s},$ (16)

where $\varpi_{s}:=(\sqrt{g}/(p_{s})_{0}){\rm d}p^{1}_{s}\wedge...\wedge{\rm
d}p^{d}_{s}$ is the invariant volume element on the mass shell of the
$s$-species. The Maxwell equation and the energy-momentum tensor produced by
the electromagnetic field are given by

$\displaystyle\nabla_{\nu}F^{\mu\nu}=J^{\mu},\quad
T^{\mu\nu}_{\text{em}}=F^{\mu\rho}F^{\nu}{}_{\rho}-\frac{1}{4}g^{\mu\nu}F^{\rho\sigma}F_{\rho\sigma},$
(17)

and finally, the Einstein equation that determines the spacetime geometry
reads

$\displaystyle R^{\mu\nu}-\frac{1}{2}g^{\mu\nu}R=8\pi
G\left(T^{\mu\nu}_{\text{pa}}+T^{\mu\nu}_{\text{em}}\right).$ (18)

As we shift the perspective from Alice to Carol, there exist multiple
equivalent conventions [22, 23] regarding the TRT of the electromagnetic field
and the charge. We adopt Feynman’s convention [24]:

$\displaystyle q\mapsto-q,\quad F^{\mu\nu}\mapsto F^{\mu\nu}.$ (19)

It becomes straightforward to verify that all equations of motion for the
above system, _i.e._ eqs. (13)-(18), are invariant under the TRT. Therefore,
in the presence of TRS, our designation of Alice as future-directed observer
has no microscopic meaning: one cannot distinguish Alice from Carol purely
from the mechanical description.

Stochastic mechanics, in essence, serves as an effective theory for
complicated mechanical systems within specific spatial and temporal scales
[25, 26, 27]. Consider a single heavy particle in the above system, and
assuming that the remaining particles constitute a heat reservoir which have
already attained thermal equilibrium. Due to the presence of gravity, the
equilibrium state is not a state with uniform temperature, but rather a state
of a fluid with the temperature obeying [28]

$\displaystyle\nabla_{\mu}T_{\rm B}+{T_{\rm B}}U^{\nu}\nabla_{\nu}U_{\mu}=0,$
(20)

where $U^{\mu}$ denotes the proper velocity of the fluid element. The non-
uniform distribution of the temperature described by eq. (20) is known as
Tolman-Ehrenfest effect. Please be reminded that, in relativistic context, the
temperature is observer-dependent. The particular temperature $T_{\rm B}$
appeared above is the one perceived by the observer comoving with the heat
reservoir, Bob, as indicated by the suffix $\rm B$.

The electromagnetic interaction exerted on the heavy particle can be separated
into two components: the coarse-grained averaging effects at larger spatial
and temporal scales, and the stochastic approximations at smaller scales. The
latter encompasses both the random force and damping force. As a result, eqs.
(13)-(14) can be approximated by LEτ [19], rendering this heavy particle as
the Brownian particle. It will be clear shortly that the above coarse-grained
picture breaks the TRS.

_Fluctuation theorem_. – Selecting an integral curve of Alice, we can
interpret its arc length as the proper time $t$ of Alice, which can be further
extended as a scalar field on $\mathcal{M}$ [19]. The proper time of Alice and
that of the Brownian particle are connected via the following relation:

$\displaystyle{\rm d}t=\frac{p^{\mu}}{m}\partial_{\mu}t{\rm
d}\tau=\gamma(x,p){\rm d}\tau,$ (21)

where $\gamma(x,p)$ is the local Lorentz factor arising from the relative
motion between Alice and the particle. Since the microstate $(x,p)$ of the
Brownian particle is random, the infinitesimal increment of $\tau$ also
becomes random from Alice’s perspective. Consequently, for Alice, LEτ needs to
be reparameterized using the deterministic parameter $t$ in place of the
random parameter $\tau$, yielding a new form of Langevin equation known as LEt
[19],

$\displaystyle{\rm d}\tilde{y}_{t}^{\mu}$
$\displaystyle=\frac{\tilde{k}^{\mu}_{t}}{m}\gamma^{-1}{\rm d}t,$ (22)
$\displaystyle{\rm d}\tilde{k}_{t}^{\mu}$
$\displaystyle=\hat{\xi}_{t}^{\mu}\gamma^{-1}{\rm
d}t+\mathcal{F}^{\mu}_{\text{dp}}\gamma^{-1}{\rm d}t$
$\displaystyle+\mathcal{F}^{\mu}_{\text{em}}\gamma^{-1}{\rm
d}t-\frac{1}{m}\varGamma^{\mu}{}_{\alpha\beta}\tilde{k}^{\alpha}_{t}\tilde{k}^{\beta}_{t}\gamma^{-1}{\rm
d}t,$ (23)

where
$\tilde{Y}_{t}=(\tilde{y}_{t},\tilde{k}_{t})=(\tilde{x}_{\tilde{\tau}_{t}},\tilde{p}_{\tilde{\tau}_{t}})$,
and

$\displaystyle\hat{\xi}^{\mu}_{t}:=\gamma^{1/2}\mathcal{R}^{\mu}{}_{\mathfrak{a}}\circ_{S}{\rm
d}\tilde{W}_{t}^{\mathfrak{a}}/{\rm
d}t+\mathcal{F}^{\mu}_{\text{add}}-\frac{1}{2}\mathcal{D}^{\mu
i}\gamma^{1/2}\nabla^{(h)}_{i}\gamma^{-1/2},$ (24)

in which ${\rm d}\tilde{W}_{t}^{\mathfrak{a}}$ represents a Wiener process of
variance ${\rm d}t$ for each fixed $\mathfrak{a}$. It is evident that LEt is
observer-dependent, but still manifestly general covariant. The stochastic
process $\tilde{Y}_{[t]}$ reparameterized by Alice is referred to as the
_forward process_ , while the stochastic process $\tilde{Y}^{-}_{[t]}$
reparameterized by Carol is referred to as the _reversed process_. It is worth
noticing that process and trajectory are different concepts, the latter is a
concrete realization of the former. In particular, the initial state of the
reversed process and final state of the forward process are related via

$\displaystyle\tilde{Y}^{-}_{I}=I(\tilde{Y}_{F}),$ (25)

and there need not be any relationship between other states from the forward
and reversed processes.

Contrary to conventional mechanical equations, Langevin equation lacks the
capacity to determine whether a trajectory qualifies as its solution. Instead,
the best one can do is to determine the probability of a given trajectory.
Correspondingly, the TRS breaking manifests as the difference in the
probabilities of the forward and reversed trajectories,

$\displaystyle\Pr[\tilde{Y}_{[t]}=Y_{[t]}]\neq\Pr[\tilde{Y}_{[t]}^{-}=Y_{[t]}^{-}].$
(26)

In order to quantify to what extent the TRS is broken in the general
relativistic Langevin dynamics described by LEt, let us turn to the
construction of the fluctuation theorem. The stochastic process governed by
LEt is a Markov process. Consequently, the trajectory probability can be
expressed as the product of the initial and conditional probabilities:

$\displaystyle\Pr[\tilde{Y}_{[t]}=Y_{[t]}]=\Pr[\tilde{Y}_{[t]}=Y_{[t]}|\tilde{Y}_{I}=Y_{I}]\Pr[\tilde{Y}_{I}=Y_{I}],$
(27)

and the conditional probability can be further decomposed into the product of
the transition probabilities for a sequence of intermediate steps [20]. The
initial probability $\Pr[\tilde{Y}_{I}=Y_{I}]$ is related to the one particle
distribution $\varphi(Y_{I})$ of the relativistic Brownian particle via

$\displaystyle\Pr[\tilde{Y}_{I}=Y_{I}]=\gamma\lambda^{-1}\varphi(Y_{I}),\quad\lambda:=|\partial_{\mu}t|,$
(28)

where $\varphi(Y_{I})$ obeys the reduced Fokker-Planck equation [20], but is
not a probability distribution by itself. The entropy density of the Brownian
particle should be defined as [29]

$\displaystyle S:=-\ln\varphi,$ (29)

the difference of which at the final and the initial states of a trajectory is
defined to be the trajectory entropy production. Furthermore, the ratio of the
conditional probabilities of the forward and reversed trajectories is given by
[23]:

$\displaystyle\frac{\Pr[\tilde{Y}_{[t]}=Y_{[t]}|\tilde{Y}_{I}=Y_{I}]}{\Pr[\tilde{Y}_{[t]}^{-}=Y^{-}_{[t]}|\tilde{Y}^{-}_{I}=Y^{-}_{I}]}$
$\displaystyle=$
$\displaystyle\frac{(\lambda\gamma^{-1})|_{Y_{I}}}{(\lambda\gamma^{-1})|_{Y_{F}}}\exp\left[\int^{t_{F}}_{t_{I}}\gamma^{-1}{\rm
d}t\frac{1}{T_{\rm B}}(ma^{\mu}-\mathcal{F}^{\mu}_{\text{em}})U_{\mu}\right],$
(30)

where $a^{\mu}=p^{\nu}\nabla_{\nu}p^{\mu}/m^{2}$ is the proper acceleration of
the Brownian particle. From the perspective of Bob, the complete differential
of the energy is:

$\displaystyle{\rm d}E=-\frac{p^{\nu}}{m}\nabla_{\nu}(p^{\mu}U_{\mu}){\rm
d}\tau=-ma^{\mu}U_{\mu}{\rm d}\tau+{\rm d}\mathcal{P}.$ (31)

Comparing with eq. (9), it can be deduced that the integral over the exponent
in eq. (General Relativistic Fluctuation Theorems) is equivalent to the
entropy increase of the heat reservoir, thanks to the relativistic Clausius
equality at the trajectory level:

$\displaystyle\int\gamma^{-1}{\rm d}t\frac{1}{T_{\rm
B}}(ma^{\mu}-\mathcal{F}^{\mu}_{\text{em}})U_{\mu}=-\int\frac{{\rm
d}\mathcal{Q}}{T_{\rm B}}=\Delta S_{R}.$ (32)

The relativistic Clausius equality at the ensemble level has been discussed in
our previous work [29]. Although the relativistic Clausius equalities are only
valid from the perspective of Bob, the amount of the entropy increase is
observer-independent. Therefore, the ratio of the probabilities of the forward
and reversed trajectories can be deduced from eq. (29) and eq. (General
Relativistic Fluctuation Theorems) ,

$\displaystyle\frac{\Pr[\tilde{Y}_{[t]}=Y_{[t]}]}{\Pr[\tilde{Y}^{-}_{[t]}=Y^{-}_{[t]}]}=$
$\displaystyle\frac{\Pr[\tilde{Y}_{[t]}=Y_{[t]}|\tilde{Y}_{I}=Y_{I}]}{\Pr[\tilde{Y}_{[t]}^{-}=Y_{[t]}^{-}|\tilde{Y}_{I}^{-}=Y_{I}^{-}]}\frac{\Pr[\tilde{Y}_{I}=Y_{I}]}{\Pr[\tilde{Y}_{I}^{-}=Y_{I}^{-}]}$
$\displaystyle=$ $\displaystyle{\rm e}^{\Delta
S_{R}}\frac{(\lambda\gamma^{-1})|_{Y_{I}}}{(\lambda\gamma^{-1})|_{Y_{F}}}\frac{\Pr[\tilde{Y}_{I}=Y_{I}]}{\Pr[\tilde{Y}_{F}=Y_{F}]}$
$\displaystyle=$ $\displaystyle{\rm e}^{\Delta S_{R}+\Delta S}.$ (33)

Let $\Sigma_{Y_{[t]}}=\Delta S_{R}+\Delta S$ be the total entropy production
contributed by the trajectory $Y_{[t]}$, eq. (General Relativistic Fluctuation
Theorems) can be rewritten as

$\displaystyle\frac{\Pr[\tilde{Y}_{[t]}=Y_{[t]}]}{\Pr[\tilde{Y}^{-}_{[t]}=Y^{-}_{[t]}]}={\rm
e}^{\Sigma_{Y_{[t]}}}.$ (34)

This is the standard form of fluctuation theorem on the trajectory level.
Furthermore, by integration over the space of trajectories and employing
Jensen’s inequality, we arrive at the integral fluctuation theorem:

$\displaystyle{\rm e}^{-\left\langle\Sigma_{\tilde{Y}_{[t]}}\right\rangle}$
$\displaystyle\leq\left\langle{\rm
e}^{-\Sigma_{\tilde{Y}_{[t]}}}\right\rangle$
$\displaystyle=\int\mathcal{D}[Y_{[t]}]{\rm
e}^{-\Sigma_{\tilde{Y}_{[t]}}}\Pr[\tilde{Y}_{[t]}=Y_{[t]}]$
$\displaystyle=\int\mathcal{D}[Y_{[t]}]\Pr[\tilde{Y}^{-}_{[t]}=Y^{-}_{[t]}]$
$\displaystyle=\int\mathcal{D}[Y^{-}_{[t]}]\Pr[\tilde{Y}^{-}_{[t]}=Y^{-}_{[t]}]=1,$
(35)

which means the statistical expectation value of the total entropy production
must be no-negative, _i.e._
$\left\langle\Sigma_{\tilde{Y}_{[t]}}\right\rangle\geq 0$. For most
trajectories, the total entropy production $\Sigma_{Y_{[t]}}$ is nonzero,
indicating distinct probabilities for forward and reversed trajectories. The
fluctuation theorem thus provides a quantitative link between TRS breaking and
entropy production.

_Conclusion._ – Based on the covariant framework of general relativistic
stochastic mechanics, we formulated the first law of stochastic thermodynamics
in curved spacetime. By properly addressing the description for the TRS and
its breaking, the corresponding fluctuation theorem is also proved in both the
differential and integral forms. The observers and their behaviors under the
TRT play a key role in our construction. The fact that the final fluctuation
theorems (34) and (General Relativistic Fluctuation Theorems) take the same
form as in non-relativistic context [8] is expected, because both the
trajectory probability and the total entropy production are neither coordinate
dependent nor observer dependent. The version of the fluctuation theorems
proved in this work may shed some light on understanding the origin of
irreversibility in the presence of relativistic gravitational interaction, as
in most cases of astrophysical or cosmological processes.

_Acknowledhement_ – This work is supported by the National Natural Science
Foundation of China under the grant No. 12275138.

_Conflict of interests declaration:_ – The authors declare no known conflict
of interests.

## References

  * [1] L. Boltzmann, Weitere Studien über das Wärmegleichgewicht unter Gasmolekülen, volume 67. Vieweg+Teubner Verlag, 1970. ISBN: 9783322849861.
  * [2] P. Ehrenfest and T. Ehrenfest, The conceptual foundations of the statistical approach in mechanics, volume 11. Dover Publications, 2015. ISBN: 9780486662503.
  * [3] T. Y. Wu, , Boltzmann’s H theorem and the Loschmidt and the Zermelo paradoxes, Int. J. Theor. Phys., 14:289–294, 1975.
  * [4] D. J. Evans and D. J. Searles, Equilibrium microstates which generate second law violating steady states, Phys. Rev. E, 50(2):1645, 1994.
  * [5] G. Gallavotti and E. G. D. Cohen, Dynamical ensembles in nonequilibrium statistical mechanics, Phys. Rev. Lett., 74(14):2694, 1995. [arXiv:chao-dyn/9410007].
  * [6] G. E. Crooks, Entropy production fluctuation theorem and the nonequilibrium work relation for free energy differences, Phys. Rev. E, 60(3):2721, 1999. [arXiv:cond-mat/9901352].
  * [7] K. Sekimoto, Langevin equation and thermodynamics, Prog Theor Phys Supp, 130:17–27, 1998.
  * [8] U. Seifert, Entropy production along a stochastic trajectory and an integral fluctuation theorem, Phys. Rev. Lett., 95(4):040602, 2005. [arXiv:cond-mat/0503686].
  * [9] A Imparato and L Peliti, Fluctuation relations for a driven Brownian particle, Phys. Rev. E, 74(2):026106, 2006. [arXiv:cond-mat/0603506].
  * [10] V. Y. Chernyak, M. Chertkov, and C. Jarzynski, Path-integral analysis of fluctuation theorems for general Langevin processes, J. Stat. Mech. Theory Exp., 2006(08):P08001, 2006. [arXiv:cond-mat/0605471].
  * [11] T. Ohkuma and T. Ohta, Fluctuation theorems for non-linear generalized Langevin systems, J. Stat. Mech. Theory Exp., 2007(10):P10010, 2007.
  * [12] Y. F. Cai, T. Wang, and L. Zhao, Fluctuation theorem on a Riemann manifold, Phys. Rev. Research 6, 023007 (2024), [arXiv:2401.00046].
  * [13] F. Axel, Relativistic fluctuation theorems, C. R. Phys., 8:696–713, 2007.
  * [14] B. Cleuren, K. Willaert, A. Engel, and C. Van den Broeck, Fluctuation theorem for entropy production during effusion of a relativistic ideal gas, Phys. Rev. E, 77:022103, 2008. [arXiv:0709.3801].
  * [15] Z. Y. Fei, J. N. Zhang, R. Pan, T. Qiu, and H. T. Quan, Quantum work distributions associated with the dynamical Casimir effect, Phys. Rev. A, 99:052508, 2019. [arXiv:1902.00155].
  * [16] A. Teixidó-Bonfill, A. Ortega, and E. Martín-Martínez, First law of quantum field thermodynamics, Phys. Rev. A, 102:052219, 2020. [arXiv:2008.09146].
  * [17] P. S. Pal, and S. Deffner, Stochastic thermodynamics of relativistic Brownian motion, New J. Phys., 22:073054, 2020. [arXiv:2003.02136].
  * [18] J. H. Pei, J. F. Chen, and H. T. Quan, Special relativistic covariant fluctuation theorems, [arXiv:2312.17621].
  * [19] Y. F. Cai, T. Wang, and L. Zhao, Relativistic stochastic dynamics I: Langevin equation from observer’s perspective, J. Stat. Phys. 190:193, 2023. [arXiv:2306.01982].
  * [20] Y. F. Cai, T. Wang, and L. Zhao, Relativistic stochastic mechanics II: Reduced Fokker-Planck equation in curved spacetime, J Stat Phys, 190:181, 2023. [arXiv:2307.07805].
  * [21] S. F. Liu, and L. Zhao, Work and work-energy theorem in curved spacetime, [arXiv:2010.13071].
  * [22] F. Arntzenius, and H. Greaves, Time reversal in classical electromagnetism, Brit. J. Philos. Sci., 60, 2009.
  * [23] Y. F. Cai, T. Wang, and L. Zhao, Fluctuation theorems in general relativistic stochastic thermodynamics, [arXiv:2407.09914].
  * [24] R. Feynman, The development of the space-time view of quantum electrodynamics, Nobel Lectures in Physics 1963-1970, 1965.
  * [25] G. W. Ford, M. Kac, and P. Mazur, Statistical mechanics of assemblies of coupled oscillators, J. Math. Phys., 6:504–515, 1965.
  * [26] H. Mori, Transport, collective motion, and Brownian motion, Prog. Theor. Phys., 33:423–455, 1965.
  * [27] R. Zwanzig, Nonlinear generalized Langevin equations, J. Stat. Phys., 9:215–220, 1973.
  * [28] C. Eckart, The thermodynamics of irreversible processes. III. Relativistic theory of the simple fluid, Physical Review, 58(10):919, 1940.
  * [29] T. Wang, Y. F. Cai, and L. Zhao, General relativistic stochastic thermodynamics, [arXiv:2311.04025].