# KaLM at SemEval-2020 Task 4: Knowledge-aware Language Models for
Comprehension And Generation

Jiajing Wan* Xinting Huang*

Sun Yat-Sen University
Guangzhou, China
{wanjj<EMAIL_ADDRESS>

###### Abstract

00footnotetext:  *denotes equal contribution. This work is licensed under a
Creative Commons Attribution 4.0 International License. License details:
http://creativecommons.org/licenses/by/4.0/.

This paper presents our strategies in SemEval 2020 Task 4: Commonsense
Validation and Explanation. We propose a novel way to search for evidence and
choose the different large-scale pre-trained models as the backbone for three
subtasks. The results show that our evidence-searching approach improves model
performance on commonsense explanation task. Our team ranks 2nd in subtask C
according to human evaluation score.111The code is available at
https://github.com/huangxt39/KaLM

##  1 Introduction

Commonsense reasoning has been seen as one of the key ability for intelligent
machines to perform various activities [Davis and Marcus, 2015]. SemEval 2020
Task 4 [Wang et al., 2020] is a commonsense validation and explanation task
which is inspired by Wang et al. [Wang et al., 2019]. This task consists of
three subtasks. The first subtask is to choose the one which makes sense from
two natural language statements with similar wordings; The second subtask is
to decide among three options the most crucial reason why a given statement
does not make sense; The third subtask requires the machine to generate the
reasons.

To make predictions or generate reasons, background knowledge is essential. A
simple way to supplement that knowledge is utilizing plain texts from natural
language databases, e.g. Wikipedia. Intuitively, for a specific given
statement, plain texts can be provided by searching similar sentences in the
database. In other words, each evidence sentences contain the keywords of the
given statement. This method is used in some of the state-of-the-art models
[Lv et al., 2019]. However, for the purpose of explaining the reason, evidence
which has similar wording with the given statement may lack information, and
sometimes can be misleading when the given statement does not make sense.

In this work, we propose a novel way for evidence-searching using plain texts.
We obtain evidence by searching for the meaning of the keywords in the given
statement. In other words, using evidence containing the meanings of the
keywords rather than containing the keywords themselves. The reason for our
method is that these meanings may provide important information to explain why
a statement makes sense or does not make sense. For example in Figure 1, the
definition of ”aircraft carrier”—”A warship designed to carry aircraft”—is
given by the evidence obtained from the database. Such a definition well
explains why the given statement is wrong and can be used when generating the
reason. In contrast, evidence containing both ”aircraft carrier” and ”human”
will not be helpful.

We conduct experiments on subtask A, B, and C. Results show that our evidence-
searching method boosts the performance on subtask C. Our team achieves
accuracy of 95.3 (9th place) in subtask A and 93.2 (7th place) in subtask B.
In subtask C, Our approach achieves the BLEU score of 18.5 (3rd place) and
human evaluation score of 2.08 (2nd place). Moreover, the best BLEU score in
our experiments (20.4) even outperforms the score we obtain in competition.

Task | Description | Question example
---|---|---
Subtask A |  | Choose the one which makes
---
sense
| Statement1: He put a turkey into the fridge.
---
Statement2: He put an elephant into the fridge.
Subtask B |  | Select the most corresponding
---
reason why this statement is
against common sense
| Statement: He put an elephant into the fridge.
---
A: An elephant is much bigger than a fridge.
B: Elephants are usually white while fridges
are usually white.
C: An elephant cannot eat a fridge.
Subtask C |  | Generate the reason why this
---
statement is against common
sense
Statement: He put an elephant into the fridge.
Table 1: A brief summarization of Subtask A, B, and C in SemEval 2020 Task 4.

##  2 Related Work

Pre-trained Language models have been proved to be essential while dealing
with sequence-to-sequence task. We present the language model we utilized in
this section and demonstrate our consideration. We use RoBERTa [Liu et al.,
2019] for language comprehension task. The structure of RoBERTa is based on
BERT [Devlin et al., 2018]. We also use BART [Lewis et al., 2019] for
generation task. BART is a denoising autoencoder for pre-training sequence-to-
sequence models. It uses the standard sequence-to-sequence Transformer
architecture except the activation functions which is replaced by GeLUs. We
find the structure of autoregressive decoder make the model can be directly
ﬁne tuned for sequence generation tasks. BART is also enable to apply any type
of document corruption which make the additional knowledge available to the
model.

Machine common sense has long been acknowledged as a critical component for
natural language understanding. The challenge for the model has turned to
abstractive knowledge comprehension. A task that is closely related to SemEval
2020 Task 4 is CommonsenseQA [Talmor et al., 2018], in which the commonsense
knowledge is required to make the correct prediction. In CommonsenseQA task,
large-scale pre-trained models have brought significant performance gains.
These gains are obtained by developing training strategies and enlarging
training data [Liu et al., 2019], or improving parameter efficiency [Lan et
al., 2019].

While some of the improvement is achieved by developing the pre-trained model
itself, some other approaches resort to external modules, e.g. knowledge
extraction and graph-based reasoning in [Lv et al., 2019]. In our work, we are
interested in the knowledge extraction method because developing the pre-
trained model itself will be computationally expensive.

Moreover, different from recent common strategy [Lin et al., 2019, Lv et al.,
2019, Ma et al., 2019] to use external knowledge which involves structured
knowledge like ConceptNet [Speer et al., 2016], we extract external knowledge
from plain texts.

##  3 System Description

We first describe our evidence-searching approach which can be utilized in
downstream tasks. Then we describe our systems for three subtasks.

###  3.1 Evidence-Searching Approach

As shown in Figure 1, we first extract more than 1M two-element tuples (word,
gloss) from Wiktionary222Wiktionary version: enwiktionary-20200220 with the
help of Wiktextract package333https://github.com/tatuylonen/wiktextract and
adopt Elastic Search tools444https://www.elastic.co/ to index these tuples
(note that the same word can have many meanings, thus in practice the tuples
actually consist of 3 elements, adding an element which indicates the
importance of this meaning). Then for each given statement, we extract its
keywords with the help of Spacy555https://spacy.io/. For each keyword, we
search for those tuples whose ”word” field matches the keyword. The Elastic
Search engine ranks the matching score for tuples. We select top K tuples for
each keyword. Thus the number of evidence tuples for a given statement is K*M
(M denotes the number of keywords). In short, we search for the meaning of the
keywords. Finally, the input sentences are produced by concatenating the
original statement and corresponding evidence together. The detailed format is
varied according to different subtasks and will be discussed later.

Figure 1: The Evidence-Searching Procedure

###  3.2 Systems for Subtask A&B

Both of subtask A&B are to select one among several choices. Thus we implement
two of the same model for subtaskA&B. In a nutshell, our model is RoBERTa (A
Robustly Optimized BERT Pretraining Approach)[Liu et al., 2019] since it has
been found that large-scale pre-trained contextualized representation masters
a certain degree of commonsense knowledge [Zhou et al., 2019]. This is also
supported by our experimental results.

For subtask A, we adapt pre-trained $\mathsf{RoBERTa_{LARGE}}$ to subtask A
dataset. We denote the hidden size for each layer (transformer blocks) as $H$,
aggregate sequence representation as $C\in\mathbb{R}^{H}$ (final hidden state
corresponding to the special [CLS] word embedding). The task-specific
parameters we introduce is a vector $V\in\mathbb{R}^{H}$. For each example, we
simply input the two choice respectively and obtain the final aggregate
representation $C_{i}\in\mathbb{R}^{H}$ for each choice $i$ whose dot product
with the vector $V$ denotes a score for choice $i$. Thus the probability
distribution is the softmax over the two choices:

$P_{i}=\frac{e^{V\cdot C_{i}}}{\sum_{j=1}^{n}e^{V\cdot C_{j}}}$ (1)

$n$ denotes the number of choices, which is 2 in subtask A. At testing time,
the model’s prediction $\hat{i}$ is the choice with the highest probability:

$\hat{i}=\mathop{\arg\max}_{i}P_{i}$ (2)

The model is trained with back propagation, using negative log-likelihood as
loss function.

In subtask B, given a statement that does not make sense, select the key
reason from three options to explain why it does not make sense. Adapting
RoBERTa to subtask B dataset is similar to the adaptation for subtask A. For
each example, we construct three input sentences by concatenating statements
with three choices respectively. Then input these sentences respectively. We
compute the score for each of choice according to Equation 1 where n is 3 in
subtask B. Then we follow the procedure described in subtask A.

###  3.3 System for Subtask C

Subtask C is an NLG (Natural Language Generation) task, which is quite
different from subtask A&B. Meanwhile, this subtask also requires commonsense
knowledge and reasoning ability, which makes it more challenging. Our model
for subtask C is BART (Denoising Sequence-to-Sequence Pre-training for Natural
Language Generation, Translation, and Comprehension) [Lewis et al., 2019].
BART is a sequence-to-sequence model using a standard Transformer-based neural
machine translation architecture. It is pre-trained by learning a model to
reconstruct the original text from the corrupted text. It uses the same pre-
training data as RoBERTa [Liu et al., 2019], consisting of 160Gb of news,
books, stories, and web text.

Specifically, we adapt pre-trained $\mathsf{BART_{LARGE}}$ to subtask C
dataset. For each example, we follow our aforementioned evidence-searching
approach to obtain evidence for the given statement. In the subtask C dataset,
each statement has 3 referential reasons. During training, we construct 3 new
example for each example in original dataset (e.g., for one original example
$\mathsf{<statement,evidence,reason_{1},reason_{2},reason_{3}>}$ we construct
three new example $\mathsf{<statement,evidence,reason_{1}>}$,
$\mathsf{<statement,evidence,reason_{2}>}$,
$\mathsf{<statement,evidence,reason_{3}>}$). Thus the total number of training
examples is 3*N (N denotes the number of training examples originally). We
denote this method as $\mathsf{Multi-target}$ training since the same input
has multiple different targets. For each new example, the input sentence is
the concatenation of the statement and evidence. Because BART has an
autoregressive decoder, it can be directly fine-tuned for such a sequence
generation task and can generate outputs autoregressively.

##  4 Experiments

We use the officially released dataset and standard train/trial/dev split of
SemEval 2020 task 4 for experiments. We will give the performance of the best
settings on test split. Note that we compare different settings through
performance obtained by training the model on train&trial split and testing it
on dev split since testing on test split is inconvenient. We will also give
our configuration of final submissions for subtask A, B, C in section 4.1,
4.2, 4.3

###  4.1 Experiments for Subtask A

We implement RoBERTa in FAIRSEQ [Ott et al., 2019]. RoBERTa is optimized with
Adam [Kingma and Ba, 2014] with the following parameters:
$\beta_{1}=0.9,\beta_{2}=0.98,\epsilon=1e-6$ and $L_{2}$ weight decay of 0.01.
The learning rate is warmed up over the first 800 steps to a peak value of
1e-5, and then polynomially decayed. The clipping threshold of gradients is
0.1. RoBERTa is fine-tuned with a dropout of 0.1 on all layers and attention
weights. It is fine-tuned for S=8,000 updates, with mini-batches containing
B=8 sequences of maximum length T=512 tokens.

In experiments where we add evidence, for each statement we have several
tuples (word, gloss). The evidence for the statement is in such format:
”$\mathsf{<word_{1}>}$: $\mathsf{<gloss_{1}>}$ $\setminus$
$\mathsf{<word_{2}>}$: $\mathsf{<gloss_{2}>}$ $\setminus\cdots$”. We construct
two input sentences for each example in such format:
”$\mathsf{<Statement_{1}>}$ Context: $\mathsf{<Evidence_{1}>}$”,
”$\mathsf{<Statement_{2}>}$ Context: $\mathsf{<Evidence_{2}>}$”. Note that due
to the unavoidable memory limitation problem, we use memory efficient floating
point numbers option provided by FAIRSEQ when we add evidence.

We train the model on 1 $\times$ 11GB GeForce RTX 2080 GPU for around 15
minutes when we input statements without evidence and 2 $\times$ GPUs for
around 100 minutes when we add evidence.

We notice that there are some statements whose letters are all capitalized
(e.g., A GIRL WON THE RACE WITH HER FRIEND) in Subtask A dataset. We
capitalize the first letter and make other letters in lowercase. We denote
this operation as $\mathsf{Lowercase}$ in Table 2.

As shown in Table 2, we can see that the performance is slightly improved
after we make some letter lowercase, since otherwise different forms of a word
can be mapped to different embeddings while they have the same meaning. Then
we explore the effect of evidence. By adding evidence to the input statement,
we obtain a slightly better result in the development dataset while
performance degradation is found in the test dataset. We hypothesize this
discrepancy is because the amount of noise data in evidence is unstable.

Our configuration of the final submission of subtask A has the same setting
discussed in the first paragraph, with the lowercase operation, and without
additional evidence. Our approach achieves the accuracy of 95.3% on subtask A
test dataset

| Dev Acc | Test Acc
---|---|---
$\mathsf{RoBERTa_{LARGE}}$ (our implementation) | 95.4 | -
\+ Lowercase | 95.7 | 95.3
\+ Lowercase + Evidence | 96.3 | 94.1
Table 2: Model performance on the official subtask A development set and test
set. Lowercase: make all the letters in lowercase except for the first letter.
Evidence: add evidence from wiktionary

###  4.2 Experiments for Subtask B

We primarily follow the optimization hyperparameters, given in Section 4.1,
except for the batch size, number of warmup steps, and number of total updates
which are 4, 500, and 10,000 separately. In the following experiments, we also
follow the lowercase operation in Section 4.1 as part of the default setting.

In subtask A, given two similar statements, one makes sense while another one
does not. In subtask B, the one that does not make sense is given, thus the
other one—the statement that makes sense—can be used as a kind of evidence
since the different words between two statements may be the keywords for
explaining why given statement does not make sense. We denote this evidence as
$\mathsf{Reasonable\;Statement}$, the evidence from wiktionary as
$\mathsf{Wiktionary}$.

In experiments where we do not add any evidence, we construct input sentences
in such format ”The statement ’$\mathsf{<Statement>}$’ is absurd. Because
$\mathsf{<Choice_{i}>}$” in which we concatenate some additional words (”The
statement ’$\cdots$’ is absurd. Because $\cdots$”) to the sentence and denote
this technique as $\mathsf{Extra\;Words}$. Moreover, in experiments where we
add the reasonable statement, the input format for each choice is ”Reasonable
statement: $\mathsf{<Reasonable\;Statement>}$ $\setminus$ The statement
’$\mathsf{<Statement>}$’ is absurd. Because $\mathsf{<Choice_{i}>}$”. If the
evidence is added, then the format will be ”Context: $\mathsf{<Wiktionary>}$
Reasonable statement: $\mathsf{<Reasonable\;Statement>}$ $\setminus$ The
statement ’$\mathsf{<Statement>}$’ is absurd. Because $\mathsf{<Choice_{i}>}$”

In Table 3 we present the result of different settings. We see that the extra
words can bring 1.6% absolute improvement since they indicate that the given
statement does not make sense and the choice is the reason for that. We also
see that using corresponding reasonable statement and wiktionary evidence
achieve comparable performance while they involve extra computational cost.
Therefore, we only use extra words in the final submission of subtask B and
achieve accuracy of 93.2%

| Dev Acc | Test Acc
---|---|---
$\mathsf{RoBERTa_{LARGE}}$ (our implementation) | 91.6 | -
\+ Extra Words | 93.2 | 93.2
\+ Extra Words + Reasonable Statement | 93.2 | -
\+ Extra Words + Wiktionary | 92.4 | -
\+ Extra Words + Reasonable Statement + Wiktionary | 93.1 | -
Table 3: Model performance on the official subtask B development set and test
set. Extra Words: extra words indicating given statement does not make sense.
Reasonable Statement: the corresponding reasonable statement. Wiktionary: the
obtained evidence from wiktionary

###  4.3 Experiments for Subtask C

BART is also implemented in FAIRSEQ [Ott et al., 2019]. It is optimized with
Adam [Kingma and Ba, 2014] with the following parameters:
$\beta_{1}=0.9,\beta_{2}=0.999,\epsilon=1e-8$ and $L_{2}$ weight decay of
0.01. The learning rate is warmed up over the first 500 steps to a peak value
of 3e-5, and then polynomially decayed. The clipping threshold of gradients is
0.1. BART is fine-tuned with a dropout of 0.1 on all layers and attention
weights. It is fine-tuned for S=1,200 updates, with mini-batches containing
B=32 sequences of maximum length T=512 tokens. Note that we keep the same
experiment settings in the following experiments. We train the model on 4
$\times$ 11GB GeForce RTX 2080 GPU, for 15 minutes to 1 hour according to
different settings.

In the following experiments, we follow the input format described in Section
4.2, removing the ”$\mathsf{<Choice_{i}>}$” only. We conduct experiments on
different combinations of the methods described above (Multi-target: Section
3.3; Extra Words, Reasonable Statement, and Wiktionary: Section 4.2) and
explore the effect of them.

As shown in Table 4, by using Multi-target training, we can obtain a 4.51
improvement on BLEU score. Compare to the baseline where we simply use the
first referential answer of each example in training data as the target of the
model output, the Multi-target method provides a larger amount of training
data and thus helps the model get better performance. From Table 4 we can see
performance degradation appears as we add some extra material but then
performance improved as we add more material. We hypothesize the degradation
is because the complexity of the input sentence increases as we add extra
material. When Extra Words, Reasonable Statement, and Wiktionary are all
added, the benefits outweigh the disadvantages they bring. Therefore, we use
all of them in the final submission of subtask C and achieve the BLEU score of
18.5 (3rd place) and human evaluation score of 2.08 (2nd place), which obtains
a 0.14 gain over 3rd place and only 0.02 less than 1st place.

The best score (20.39) on the test set in Table 4 outperforms the score we
achieved during the competition (18.5). Note that we might achieve a better
human evaluation score accordingly. There are two reasons for that. Firstly,
we optimize our evidence-searching approach after the competition, improving
the quality of the evidence (All the experiments with evidence added in this
paper are using the improved version. Details of optimizing method are shown
in Appendices). Secondly, we observe that during the training process, the
model performs well at the beginning but turns to mess later. Thus it’s
difficult to choose the best model during the training process when we cannot
evaluate it on the test set. When the competition has ended, however, we can
evaluate our models and choose the best one during the training process.

| Dev BLEU | Test BLEU
---|---|---
$\mathsf{BART_{LARGE}}$ (our implementation) | 15.15 | -
+Multi-target | 19.66 | 19.43
+Multi-target + Extra Words | 18.10 | -
+Multi-target + Wiktionary | 18.98 | -
+Multi-target + Extra Words + Wiktionary | 19.50 | -
+Multi-target + Extra Words + Reasonable Statement | 18.98 | -
+Multi-target + Extra Words + Reasonable Statement + Wiktionary | 20.03 | 20.39
Table 4: Model performance on the official subtask C development set and test
set. Multi-target: the same input has multiple different targets. Extra Words:
extra words indicating given statement does not make sense. Reasonable
Statement: the corresponding reasonable statement. Wiktionary: the obtained
evidence from wiktionary

##  5 Conclusion

In this work, we choose the different large-scale pre-trained models as the
backbone for three subtasks and propose a novel way to search for evidence,
which aims to obtain the meaning of the keywords in the given statement. Our
experiments demonstrate the importance of additional knowledge for language
models to understand the content. The results show that our evidence-searching
approach is helpful to commonsense explanation task.

## References

  * [Davis and Marcus, 2015]  Ernest Davis and Gary Marcus.  2015\.  Commonsense reasoning and commonsense knowledge in artificial intelligence.  September.
  * [Devlin et al., 2018]  Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  2018\.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805.
  * [Kingma and Ba, 2014]  Diederik P. Kingma and Jimmy Ba.  2014\.  Adam: A method for stochastic optimization.
  * [Lan et al., 2019]  Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut.  2019\.  Albert: A lite bert for self-supervised learning of language representations.
  * [Lewis et al., 2019]  Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer.  2019\.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.
  * [Lin et al., 2019]  Bill Yuchen Lin, Xinyue Chen, Jamin Chen, and Xiang Ren.  2019\.  Kagnet: Knowledge-aware graph networks for commonsense reasoning.
  * [Liu et al., 2019]  Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  2019\.  Roberta: A robustly optimized BERT pretraining approach.  CoRR, abs/1907.11692.
  * [Lv et al., 2019]  Shangwen Lv, Daya Guo, Jingjing Xu, Duyu Tang, Nan Duan, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, and Songlin Hu.  2019\.  Graph-based reasoning over heterogeneous external knowledge for commonsense question answering.
  * [Ma et al., 2019]  Kaixin Ma, Jonathan Francis, Quanyang Lu, Eric Nyberg, and Alessandro Oltramari.  2019\.  Towards generalizable neuro-symbolic systems for commonsense question answering.
  * [Ott et al., 2019]  Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli.  2019\.  fairseq: A fast, extensible toolkit for sequence modeling.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages 48–53, Minneapolis, Minnesota, June. Association for Computational Linguistics.
  * [Speer et al., 2016]  Robyn Speer, Joshua Chin, and Catherine Havasi.  2016\.  Conceptnet 5.5: An open multilingual graph of general knowledge.  CoRR, abs/1612.03975.
  * [Talmor et al., 2018]  Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant.  2018\.  Commonsenseqa: A question answering challenge targeting commonsense knowledge.  CoRR, abs/1811.00937.
  * [Wang et al., 2019]  Cunxiang Wang, Shuailong Liang, Yue Zhang, Xiaonan Li, and Tian Gao.  2019\.  Does it make sense? and why? A pilot study for sense making and explanation.  CoRR, abs/1906.00363.
  * [Wang et al., 2020]  Cunxiang Wang, Shuailong Liang, Yili Jin, Yilong Wang, Xiaodan Zhu, and Yue Zhang.  2020\.  SemEval-2020 task 4: Commonsense validation and explanation.  In Proceedings of The 14th International Workshop on Semantic Evaluation. Association for Computational Linguistics.
  * [Zhou et al., 2019]  Xuhui Zhou, Yue Zhang, Leyang Cui, and Dandan Huang.  2019\.  Evaluating commonsense in pre-trained language models.

## Appendices

We optimize our evidence-searching approach after the competition. That is,
filtering out some meaningless word-gloss pairs while keeping the framework
unchanged. We observe two kinds of meaningless pairs in the evidence. First,
pairs contain misleading gloss, e.g., when we search the gloss for the keyword
”car”, we obtain ”CAR: initialism of ’Central African Republic’ CAR:
(uncountable chemistry) abbreviation of ’carnitine’”; when we search the gloss
for ”like”, ”like like: (slang) To fancy; to be attracted to” is obtained. To
address this problem, we remove the pairs whose gloss contains
”initialism/historical/obsolete/abbreviation/(dated)/slang/acronym/(US)/synonym/archaic/surname/(rare)”,
and remove the pairs whose word contains ”-” or capital letters. Replacing
these with other pairs. Second, we also observe many pairs only contain
prototype information. For example: ”watermelons: plural of ’watermelon’”,
”concentrated: past of ’concentrate’”, ”facebook: alternative form of
’Facebook’”. To obtain more meaningful gloss, we search the pairs for the
prototype word, regarding the prototype word as one of the keywords.
Specifically, we detect whether the gloss contain words like ”plural of/past
of/third person singular of/clipping of/alternative form of/alternative
spelling of” which indicate the gloss points to a prototype word. In that
case, we use the same search function to acquire evidence for prototype word
and incorporate the new evidence. (note that to avoid infinite loop, we do not
detect the prototype in sub-search). Besides, we also adjust the number of
evidence tuples for a keyword dynamically. For the statement contains less
keywords, we obtain more evidence tuples for each keywords. Thus the length of
the evidence for statements will be more stable.


# Fock Space and Field Theoretic Description of Nonequilibrium Work Relations

Andrew J. Baish  Kyocera SLD Laser, 111 Castilian Dr, Goleta, CA 93117
Benjamin P. Vollmayr-Lee  Department of Physics and Astronomy, Bucknell
University, Lewisburg, PA 17837

###### Abstract

We consider classical, interacting particles coupled to a thermal reservoir
and subject to a local, time-varying potential while undergoing hops on a
lattice. We impose detailed balance on the hopping rates and map the dynamics
to the Fock space Doi representation, from which we derive the Jarzynski and
Crooks relations. Here the local potential serves to drive the system far from
equilibrium and to provide the work. Next, we utilize the coherent state
representation to map the system to a Doi-Peliti field theory and take the
continuum limit. We demonstrate that time reversal in this field theory takes
the form of a gauge-like transformation which leaves the action invariant up
to a generated work term. The time-reversal symmetry leads to a fundamental
identity, from which we are able to derive the Jarzynski and Crooks relations,
as well as a far-from-equilibrium generalization of the fluctuation-
dissipation relation.

##  I Introduction

In the past decades remarkable advances have been made in the field of far
from equilibrium statistical physics. The celebrated Jarzynski equation,

$\langle e^{-W/k_{B}T}\rangle=e^{-\Delta F/k_{B}T},$ (1)

relates via equality the difference in the equilibrium free energy $\Delta F$
to a nonequilibrium average of the work $W$ for processes that start in
equilibrium at temperature $T$ but are driven arbitrarily far from
equilibrium. This holds for Hamiltonian systems [1] as well as stochastic
systems described by a master equation [2]. This universal constraint on the
possible distributions of nonequilibrium work was further developed by Crooks,
who showed that the probability $P_{F}(W)$ of doing work $W$ while following
the “forward” protocol (a specified variation of control parameters) can be
related to the probability $P_{R}(-W)$ of doing negative the work during a
reversed protocol via

$P_{F}(W)=e^{(W-\Delta F)/k_{B}T}P_{R}(-W),$ (2)

thus elucidating the central role of time reversal [3, 4]. The Jarzynski and
Crooks relations are part of an ensemble of fluctuation theorems that describe
work fluctuations and entropy production, with by now a large literature that
has been summarized in reviews [5, 6, 7] and more recently in textbooks [8,
9].

In these studies one typically examines the impact of time reversal on the
probability distribution of all possible trajectories, although this
probability distribution is generally not available in explicit form. In the
present work, we propose to recast the nonequilibrium dynamics into a field
theory. While this necessarily represents a loss of trajectory-level
information, similar to a master equation treatment, the field theory
nevertheless offers advantages. The equivalent of the probability distribution
for field, namely the exponential of the action, can be obtained explicitly.
The explicit expression for the action enables us to make progress on three
fronts: one, as with all field theories, it provides a useful framework for
exploring symmetries and near-symmetries, in this case, the behavior under
time reversal; two, it enables a variety of possible approximation schemes by
perturbation theory; and three, a hierarchy of nonequilibrium identities can
be derived by functional differentiation, such as a nonequilibrium
generalization of the fluctuation-dissipation relation.

Mallick, Moshe, and Orland examined nonequilibrium work relations via field
theory for the critical dynamics of a nonconserved scalar order parameter
[10], i.e. Model A in the Heisenberg-Haltering scheme [11]. They were able to
derive the Jarzynski and Crooks relations, as well as a nonequilibrium
generalization of the fluctuation-dissipation relation, and to elucidate
properties of time-reversal symmetry breaking with the field theory. This work
was extended by Täuber to the critical dynamics of a conserved scalar order
parameter, i.e., Model B [12].

In this work we present a field theoretic description of the Jarzynski and
Crooks relations for a system of interacting classical particles coupled to
the thermal reservoir and driven by a tun able local potential. We employ Doi-
Peliti field theory, i.e., a mapping of the master equation for the classical
particle model to a Fock space representation (the Doi representation [13])
and then a subsequent mapping to a field theory via coherent states [14, 15,
16]. One of the advantages of this approach is that we are not restricted to
near-criticality dynamics.

Our primary results are the following. After defining the model in Sec. II, we
recast the dynamics in the Doi representation in Sec. III, deriving the
general form of a time-evolution operator that is consistent with detailed
balance. With this formalism in Sec. IV we obtain a direct and simple Fock
space derivation of the Jarzynski and Crooks relations. Next, in Sec. V we
obtain the Doi-Peliti field theory for interacting particles coupled to a
thermal bath. This field theory contains boundary “initial” and “final” terms
in the action as well as a bulk action governing the time evolution from $t=0$
to $t=t_{f}$. We demonstrate that the bulk action can be transformed by a
Cole-Hopf transformation to the Dean-Kawasaki field theory [17, 18, 19, 20]
obtained by mapping Dean’s Langevin equation for interacting particles [21] to
a field theory via the Martin-Siggia-Rose-Janssen-de Dominicis procedure [22,
23, 24]. This connection was reported previously by Andreanov, et al. [17],
but our treatment differs importantly in the boundary terms, which play a role
in time reversal. We also demonstrate for noninteracting particles that our
field theory reduces to the Fokker-Planck equation for a Brownian particle in
a potential.

In Sec. VI we examine time reversal within the field theory, showing that
time-reversal takes the form of a gauge-like transformation, and that the
action is invariant under time reversal apart from the introduction of the
Jarzynski work term. We note that for interacting particles, it is necessary
to introduce a Hubbard-Stratonovich transformation and add a complex field to
fully reveal the underlying time reversal symmetry. We obtain a fundamental
nonequilibrium identity, Eq. (65), from which the Jarzynski and Crook’s
identities follow. Further, in Sec. VII we employ functional differentiation
of Eq. (65) to derive a far-from-equilibrium generalization of the
fluctuation-dissipation relation.

##  II Model

###  II.1 Energy and Thermal Equilibrium

We begin with a model of $N$ classical particles on a $d$-dimensional lattice
of size $L^{d}$; the continuum limit will be taken later. The particles
interact with the pair potential $V_{ij}$, which can be any function of the
separation ${\bf r}_{j}-{\bf r}_{i}$ between sites $i$ and $j$, such as a
Lennard-Jones potential, with the only restriction that it is symmetric under
interchange of particles, $V_{ij}=V_{ji}$. Additionally, we consider a time-
dependent site potential $U_{i}(t)$, which allows for work to be done on the
system. The occupancy of site $i$ is denoted $n_{i}$, and the energy of the
occupation configuration ${\bf n}=(n_{1},n_{2},\dots)$ is given by

$E_{\bf
n}(t)=\frac{1}{2}\sum_{i,j}(n_{i}-\delta_{ij})V_{ij}n_{j}+\sum_{i}U_{i}(t)n_{i},$
(3)

where the Kronecker delta function ensures proper counting of the same-site
pairs.

These particles are coupled to a thermal reservoir at temperature $T$. Because
of the time-dependent potential $U_{i}(t)$, the equilibrium distribution also
depends on time $t$ via

$P^{\rm eq}_{\bf n}(t)=\frac{N!}{{\bf n}!}\,\frac{1}{Z(t)}e^{-\beta E_{\bf
n}(t)}.$ (4)

where $\beta^{-1}=k_{B}T$, $Z(t)$ is the partition function associated with
$E_{\bf n}(t)$, and ${\bf n}!\equiv\prod_{i}n_{i}!$. The prefactor $N!/{\bf
n}!$ is the multinomial coefficient that accounts for the multiplicity of
microstates for configuration ${\bf n}$ under permutation of the particles.

###  II.2 Dynamics

The particles undergo stochastic hops, and the probability $P({\bf n},t)$ of
configuration ${\bf n}$ at time $t$ obeys the master equation

$\frac{d}{dt}P_{\bf n}(t)=\sum_{{\bf m}\neq{\bf n}}\Bigl{[}w_{{\bf n},{\bf
m}}(t)P_{\bf m}(t)-w_{{\bf m},{\bf n}}(t)P_{\bf n}(t)\Bigr{]}.$ (5)

The particle hopping rates $w_{{\bf n},{\bf m}}(t)$ for a transition ${\bf
m}\to{\bf n}$ are chosen to obey detailed balance for the instantaneous energy
$E_{\bf n}(t)$,

$\frac{w_{{\bf n},{\bf m}}(t)}{w_{{\bf m},{\bf n}}(t)}=\frac{P^{\rm eq}_{\bf
n}(t)}{P^{\rm eq}_{\bf m}(t)}=\frac{{\bf m}!}{{\bf n}!}e^{-[\beta E_{\bf
n}(t)-\beta E_{\bf m}(t)]}$ (6)

and are thus time-dependent.

Work is performed on the system by varying the site potentials $U_{i}(t)$. For
configuration ${\bf n}$ at time $t$, the instantaneous rate of work performed
on the system is given by

$\dot{W}=\sum_{i}n_{i}\dot{U}_{i}(t).$ (7)

This term appears in the nonequilibrium generalization of the first law:

$\frac{d}{dt}\langle E\rangle=\sum_{\bf n}\biggl{[}\frac{dP_{\bf
n}(t)}{dt}E_{\bf n}(t)+P_{\bf n}(t)\sum_{i}n_{i}\dot{U}_{i}(t)\biggr{]}.$ (8)

The first term gives the rate of change in $\langle E\rangle$ due to the
particle hopping dynamics, reflecting the heat flow between the system and
reservoir, while the second term is simply the average rate of work performed,
$\langle\dot{W}\rangle$.

For sufficiently slow driving, the system remains in thermal equilibrium and
the probabilities are given by Eq. (4). In contrast, for fast driving, the
distribution $P_{\bf n}(t)$ obtained from Eq. (5) may be driven far from
equilibrium.

##  III The Doi Representation

These particle dynamics can be mapped to a Fock space representation, i.e.,
the Doi representation [13], using a standard procedure which we summarize
briefly below. Even though these are classical particles, the Fock space
representation reflects the permutation symmetry of the particles, and thus
provides in this sense the simplest expression of the particle dynamics. We
shall see that, indeed, in the Doi representation the Jarzynski relation
emerges elegantly.

###  III.1 States and the Liouvillian

Bosonic creation and annihilation operators $\hat{a}_{i}^{\dagger}$ and
$\hat{a}_{i}$ are introduced at each lattice site and, together with the
vacuum state $|0\rangle$, used to create a state $|{\bf
n}\rangle=\prod_{i}(\hat{a}_{i}^{\dagger})^{n_{i}}|0\rangle$ corresponding to
the configuration ${\bf n}$. Note that this state is normalized as as
$\langle{\bf n}|{\bf n}\rangle={\bf n}!$. The full probability distribution of
the system at time $t$ can then be represented by the state

$|\psi(t)\rangle=\sum_{\bf n}P_{\bf n}(t)|{\bf n}\rangle,$ (9)

which enables writing the master equation as

$\frac{d}{dt}|\psi(t)\rangle=-\hat{L}(t)|\psi(t)\rangle.$ (10)

with Liouvillian operator $\hat{L}$. The utility of the Doi representation is
that whenever the particle dynamics respects the permutation symmetry of the
particles, the operator $\hat{L}$ can be expressed solely in terms of the
$\hat{a}_{i}$ and $\hat{a}_{i}^{\dagger}$ operators, with no dependence on the
configuration ${\bf n}$. That is, the Liouvillian is determined solely by the
processes involved and not dependent on the state of the system. For example,
particles undergoing unbiased hops between neighboring lattice sites leads to

$\hat{L}_{\rm diff}=\Gamma\sum_{\langle
ij\rangle}(\hat{a}_{i}^{\dagger}-\hat{a}_{j}^{\dagger})(\hat{a}_{i}-\hat{a}_{j}),$
(11)

where the sum runs over $i,j$ that are nearest neighbor sites.

###  III.2 The Projection State

In the Doi representation, averages and distributions are extracted from the
Fock states by means of a projection state, defined as $\langle{\cal
P}|=\langle 0|e^{\sum_{i}\hat{a}_{i}}$, with the property $\langle{\cal
P}|\hat{a}_{i}^{\dagger}=\langle{\cal P}|$ for all $i$. Averages are computed
via

$\langle A\rangle\equiv\sum_{\bf n}A_{\bf n}P_{\bf n}(t)=\langle{\cal
P}|\hat{A}|\psi(t)\rangle,$ (12)

where $\hat{A}\equiv\sum_{\bf n}(A_{\bf n}/{\bf n}!)|{\bf n}\rangle\langle{\bf
n}|$ is an operator diagonalized by the occupation states $|{\bf n}\rangle$
with eigenvalues $A_{\bf n}$. Normalization ensures $\langle{\cal
P}|\psi(t)\rangle=\sum_{\bf n}P_{\bf n}(t)=1$, and probability conservation
requires

$0=\frac{d}{dt}\langle 1\rangle=-\langle{\cal P}|\hat{L}(t)|\psi(t)\rangle$
(13)

for all $|\psi(t)\rangle$, thus

$\langle{\cal P}|\hat{L}(t)=0.$ (14)

This is obeyed, for example, by the diffusion Liouvillian in Eq. (11).

###  III.3 The Hamiltonian and Detailed Balance

Now we consider the class of Liouvillians that correspond to transition rates
that satisfy detailed balance, and for clarity we suppress the explicit time
dependence in $\hat{L}$ and the rates. Starting from a state ${\bf m}$, the
probability of being in a state ${\bf n}\neq{\bf m}$ a short time $\delta t$
later is, according to the master equation, $P({\bf n},t)=w_{{\bf n},{\bf
m}}\delta t$, while the probability of remaining in ${\bf m}$ is $P_{\bf
m}(\delta t)=1-\delta t\sum_{{\bf n}^{\prime}\neq{\bf m}}w_{{\bf
n}^{\prime},{\bf m}}$. In the Doi representation, the state at time $\delta t$
is $|\psi(\delta t)\rangle=(1-\hat{L}\delta t)|{\bf m}\rangle$. Using
$\langle{\bf n}|\psi(t)\rangle={\bf n}!P_{\bf n}(t)$ we obtain for ${\bf
n}\neq{\bf m}$

$\langle{\bf n}|\hat{L}|{\bf m}\rangle=-{\bf n}!\,w_{{\bf n},{\bf m}}$ (15)

while for ${\bf n}={\bf m}$

$\langle{\bf n}|\hat{L}|{\bf n}\rangle=-{\bf n}!\sum_{{\bf n}^{\prime}\neq{\bf
n}}w_{{\bf n}^{\prime},{\bf n}}.$ (16)

For rates that obey the detailed balance condition of Eq. (6), we obtain

$\langle{\bf n}|\hat{L}|{\bf m}\rangle=\langle{\bf m}|\hat{L}|{\bf n}\rangle
e^{-\beta E_{\bf n}(t)+\beta E_{\bf m}(t)}.$ (17)

with $E_{\bf n}$ given by Eq. (3). Now we introduce the hermitian Hamiltonian
operator

$\hat{H}(t)=\frac{1}{2}\sum_{i,j}V_{ij}\hat{a}_{i}^{\dagger}\hat{a}_{j}^{\dagger}\hat{a}_{j}\hat{a}_{i}+\sum_{i}U_{i}(t)\hat{a}_{i}^{\dagger}\hat{a}_{i}$
(18)

which is diagonalized by the states $|{\bf n}\rangle$ with eigenvalues $E_{\bf
n}(t)$. With this energy operator Eq. (17) becomes $\langle{\bf
n}|\hat{L}|{\bf m}\rangle=\langle{\bf
m}|e^{\beta\hat{H}}\hat{L}e^{-\beta\hat{H}}|{\bf n}\rangle$, which implies

$\hat{L}^{\dagger}=e^{\beta\hat{H}}\hat{L}e^{-\beta\hat{H}}.$ (19)

Together Eqs. (14) and (19) imply that necessary and sufficient conditions for
detailed balance are that $\hat{L}$ has the form

$\hat{L}=\hat{Q}e^{\beta\hat{H}}$ (20)

where $\hat{Q}$ is a hermitian operator that annihilates the projection state:
$\hat{Q}|{\cal P}\rangle=0$.

###  III.4 Constructing the Liouvillian

Finally, we discuss specific choices for the Liouvillian. Consider a particle
hop from site $i$ to neighboring site $j$, which takes the system from
configuration ${\bf m}$ to ${\bf n}$, where
$n_{k}=m_{k}-\delta_{ik}+\delta_{jk}$. The detailed balance condition (6) for
this hop is

$\frac{w_{{\bf n},{\bf m}}}{w_{{\bf m},{\bf
n}}}=\frac{m_{i}}{n_{j}}e^{-\beta[E_{\bf n}-E_{\bf m}]}.$ (21)

We restrict consideration to rates that depend on the energies only via the
difference $\Delta E\equiv E_{\bf n}-E_{\bf m}$. These have the general form

$w_{{\bf n},{\bf m}}=\Gamma m_{i}f_{e}(\beta\Delta E)e^{-\beta\Delta E/2},$
(22)

where $f_{e}(x)$ is an even function.

To construct the Liouvillian corresponding to (22) it is useful to introduce
the hermitian operator

$\hat{\epsilon}_{k}=U_{k}+\sum_{\ell}V_{k\ell}\hat{a}_{\ell}^{\dagger}\hat{a}_{\ell}$
(23)

with eigenvectors $|{\bf m}\rangle$ and eigenvalues $\epsilon_{k}({\bf
m})=U_{k}+\sum_{\ell}V_{k\ell}m_{\ell}$ equal to the energy required to
introduce a particle at site $k$ to configuration ${\bf m}$. It is
straightforward to show that for the $i\to j$ jump

$(\hat{\epsilon}_{j}-\hat{\epsilon}_{i})\,\hat{a}_{i}|{\bf m}\rangle=\Delta
E\,\hat{a}_{i}|{\bf m}\rangle,$ (24)

which allows us to write (22) as

$w_{{\bf n},{\bf m}}=\frac{1}{{\bf n}!}\langle{\bf
n}|\Gamma\hat{a}_{j}^{\dagger}f_{e}(\hat{\epsilon}_{i}-\hat{\epsilon}_{j})e^{\beta(\hat{\epsilon}_{i}-\hat{\epsilon}_{j})/2}\hat{a}_{i}|{\bf
m}\rangle.$ (25)

By comparison with (15), we can identify the operator above with the gain term
in $\hat{L}$.

For the loss term, we consider the same $i\to j$ hop but starting in ${\bf n}$
and going to ${\bf n}^{\prime}$, where
$n_{k}^{\prime}=n_{k}-\delta_{ik}+\delta_{jk}$, and $\Delta E=E_{{\bf
n}^{\prime}}-E_{\bf n}$. The appropriate rate is obtained from (16) and (22)
to be

$w_{{\bf n}\to{\bf n}^{\prime}}=\frac{1}{{\bf n}!}\langle{\bf
n}|\Gamma\hat{a}_{i}^{\dagger}f_{e}(\hat{\epsilon}_{i}-\hat{\epsilon}_{j})e^{\beta(\hat{\epsilon}_{i}-\hat{\epsilon}_{j})/2}\hat{a}_{i}|{\bf
n}\rangle.$ (26)

Combining (25) and (26) with the analogous terms for a $j\to i$ hop provides
the Liouvillian corresponding to (22),

$\hat{L}=\Gamma\sum_{\langle
ij\rangle}(\hat{a}_{i}^{\dagger}-\hat{a}_{j}^{\dagger})f_{e}(\hat{\epsilon}_{i}-\hat{\epsilon}_{j})\Bigl{(}e^{\beta(\hat{\epsilon}_{i}-\hat{\epsilon}_{j})/2}\hat{a}_{i}-e^{\beta(\hat{\epsilon}_{j}-\hat{\epsilon}_{i})/2}\hat{a}_{j}\Bigr{)}.$
(27)

As advertised, the Liouvillian is purely an expression of the process and has
no dependence on the state of the system, i.e., the occupation numbers. The
identity
$e^{\beta\hat{\epsilon}_{k}}\hat{a}_{k}=e^{-\beta\hat{H}}\hat{a}_{k}e^{\beta\hat{H}}$,
which follows from $[\hat{H},\hat{a}_{k}]=-\hat{\epsilon}_{k}\hat{a}_{k}$ and
the Hadamard lemma, can be used to show this Liouvillian has the necessary
form (20).

Later, when taking the continuum limit, the energy difference due to a hop
will be of the order of the lattice spacing $\Delta x$, which will allow us to
linearize (27) in $\hat{\epsilon}_{i}-\hat{\epsilon}_{j}$:

$\hat{L}\simeq\Gamma\sum_{\langle
ij\rangle}(\hat{a}_{i}^{\dagger}-\hat{a}_{j}^{\dagger})\biggl{(}\hat{a}_{i}-\hat{a}_{j}+\beta(\hat{\epsilon}_{i}-\hat{\epsilon}_{j})\frac{\hat{a}_{i}+\hat{a}_{j}}{2}\biggr{)},$
(28)

where we have set $f_{e}(0)=1$ without loss of generality. Note that this
linearized Liouvillian is normal ordered.

###  III.5 Equilibrium State

The Jarzynski and Crooks relations require starting in thermal equilibrium. In
the Doi representation the equilibrium state can be written as

$|\psi_{\text{eq}}\rangle=\sum_{\bf n}P_{\bf n}^{\text{eq}}|{\bf
n}\rangle=\frac{1}{Z}e^{-\beta\hat{H}}|\bar{n}_{0}\rangle,$ (29)

where $\bar{n}_{0}=N/L^{d}$ is the average number of particles per site, and
$|\bar{n}_{0}\rangle\equiv
e^{\sum_{i}\bar{n}_{0}\hat{a}_{i}^{\dagger}}|0\rangle$. This is obtained from
(4) by expressing the multiplicity of the occupation state ${\bf n}$ in terms
of a product of Poisson distributions: $N!/{\bf
n}!\propto\prod_{i}\bar{n}_{0}^{n_{i}}/n_{i}!$, with the implied constraint
$\sum_{i}n_{i}=N$ and the proportionality constant absorbed into $Z$.

The equilibrium state is necessarily a stationary state of the detailed
balance Liouvillian, Eq. (20). In the Doi representation this can be seen from

$\hat{L}|\psi_{\text{eq}}\rangle=Z^{-1}\hat{Q}|\bar{n}_{0}\rangle=0,$ (30)

where the last equation follows from taking $\hat{a}\to\bar{n}_{0}\hat{a}$ and
$\hat{a}^{\dagger}\to\bar{n}_{0}^{-1}\hat{a}^{\dagger}$, which leaves
$\hat{Q}$ invariant and takes $|\bar{n}_{0}\rangle\to|{\cal P}\rangle$.

Finally, we note that all of $\hat{L}$, $\hat{H}$, $E$, $Z$, and
$|\psi_{\text{eq}}\rangle$ are defined in terms of the instantaneous
$U_{i}(t)$, and thus can be time-dependent.

##  IV Work Relations in the Doi Representation

We are now equipped to derive the Jarzynski relation for general detailed
balance dynamics within the Doi representation. Perhaps unsurprisingly, our
methods bear a strong similarity to the operator framework Kurchan used for
Langevin dynamics [25]. As usual, the system must begin in thermal equilibrium
at time $t=0$; we denote this state $|\psi_{\text{eq}}(0)\rangle$ to emphasize
that the time-dependent Hamiltonian is to be evaluated at $t=0$.

The solution to the master equation (10) can be written as

$|\psi(t)\rangle=\lim_{\Delta t\to 0}\prod_{n=0}^{t/\Delta
t-1}(1-\hat{L}_{t_{n}}\Delta t)|\psi_{\text{eq}}(0)\rangle.$ (31)

where the product of the time-dependent $\hat{L}_{t}$ operators is time
ordered with earlier times on the right, and $t_{n}=n\Delta t$. Work, as
defined in (7), appears in the Doi representation as

$W=\int_{0}^{t_{f}}dt\,\sum_{i}\dot{U}_{i}(t)\hat{a}_{i}^{\dagger}\hat{a}_{i}.$
(32)

The average of the work from time $t=0$ to $t_{f}$ is given by

$\langle e^{-\beta W}\rangle=\langle{\cal
P}|\prod_{n=0}^{n_{f}}\left[e^{-\beta\dot{W}_{t_{n}}\Delta
t}(1-\hat{L}_{t_{n}}\Delta t)\right]|\psi_{\text{eq}}(0)\rangle$ (33)

where $n_{f}=t_{f}/\Delta t-1$, the product is again time ordered, and the
limit $\Delta t\to 0$ is implied. Eq. (33) is equivalent to applying a weight
$e^{-\beta w(t)}$ to each trajectory, where $w(t)$ is the work done up to time
$t$ along that trajectory. This has been demonstrated to provide the desired
average [2, 26].

Substituting $e^{-\beta\dot{W}_{t}\Delta t}=e^{-(\beta\hat{H}_{t+\Delta
t}-\beta\hat{H}_{t})}$ and regrouping terms gives

$\displaystyle\langle e^{-\beta W}\rangle=\langle{\cal P}|$ $\displaystyle
e^{-\beta\hat{H}_{t_{f}}}\prod_{n=0}^{n_{f}}\Bigl{[}e^{\beta\hat{H}_{t_{n}}}(1-\hat{L}_{t_{n}}\Delta
t)e^{-\beta\hat{H}_{t_{n}}}\Bigr{]}$ $\displaystyle\times
e^{\beta\hat{H}_{0}}|\psi_{\text{eq}}(0)\rangle.$ (34)

Utilizing the detailed balance condition (19), the square brackets become

$e^{\beta\hat{H}_{t}}(1-\hat{L}_{t}\Delta
t)e^{-\beta\hat{H}_{t}}=1-\hat{L}_{t}^{\dagger}\Delta t,$ (35)

which can be interpreted as simply the time evolution operator acting to the
left. This is the essence of how the fluctuation relations appear in the Doi
representation: a forward-time average including the work term transforms into
the reverse-time average absent the work.

It remains to analyze the initial and final states. We rescale the operators
$\hat{a}\to\bar{n}_{0}\hat{a}$ and
$\hat{a}^{\dagger}\to\bar{n}_{0}^{-1}\hat{a}^{\dagger}$, which leaves
$\hat{L}$ and $\hat{H}$ (and therefore the work term) unchanged, but modifies
the initial and final terms via

$e^{\beta\hat{H}_{0}}|\psi_{\text{eq}}(0)\rangle=Z(0)^{-1}|\bar{n}_{0}\rangle\to
Z(0)^{-1}|{\cal P}\rangle$ (36)

and

$\langle{\cal P}|e^{-\beta H_{t_{f}}}\to\langle\bar{n}_{0}|e^{-\beta
H_{t_{f}}}=Z(t_{f})^{-1}\langle\psi_{\text{eq}}(t_{f})|$ (37)

Putting this together, we have

$\langle e^{-\beta
W}\rangle=\frac{Z(t_{f})}{Z(0)}\langle\psi_{\text{eq}}(t_{f})|\prod_{n=0}^{t_{f}/\Delta
t-1}(1-\hat{L}^{\dagger}_{t_{n}}\Delta t)|{\cal P}\rangle$ (38)

Conjugating the real expectation value reverses the time ordering, giving

$\langle e^{-\beta W}\rangle=\frac{Z(t_{f})}{Z(0)}\langle{\cal
P}|(1-\hat{L}_{0}\Delta t)\dots(1-\hat{L}_{t_{f}-\Delta t}\Delta
t)|\psi_{\text{eq}}(t_{f})\rangle_{R}$ (39)

which can be interpreted as time ordered in the variable $t^{\prime}=t_{f}-t$.
The expectation value corresponds to $\langle{\cal
P}|\psi(t^{\prime}=t_{f})\rangle_{R}=1$ and we obtain the Jarzynski relation

$\langle e^{-\beta W}\rangle=\frac{Z(t_{f})}{Z(0)}=e^{-\beta[\Delta
F(t_{f})-F(0)]}.$ (40)

To derive the Crooks relation, consider the characteristic function
$\phi_{F}(q)$ of the forward work distribution function, which is simply the
Fourier transform:

$\phi_{F}(q)=\int_{-\infty}^{\infty}dW\,e^{iqW}P_{F}(W)=\langle
e^{iqW}\rangle_{F}.$ (41)

This takes the form of Eqs. (33) but with $\beta\to iq$. We now have a
repeating pattern of

$e^{-iq\hat{H}_{t}}(1-\hat{L}_{t}\Delta
t)e^{iq\hat{H}_{t}}=e^{-(iq+\beta)\hat{H}}(1-\hat{L}_{t}^{\dagger}\Delta
t)e^{(iq+\beta)\hat{H}}$ (42)

which means after time reversal

$\langle e^{iqW}\rangle_{F}=\frac{Z(t_{f})}{Z(0)}\langle e^{-iqW}e^{-\beta
W}\rangle_{R}$ (43)

or $\phi_{F}(q)=\phi_{R}(iW-q)e^{-\beta\Delta F}$. Inverse transforming then
gives

$P_{F}(W)=\int\frac{dq}{2\pi}e^{-iqW}\phi_{R}(iW-q)=e^{\beta W}P_{R}(-W),$
(44)

which is the Crooks relation.

##  V Doi-Peliti Field Theory

For taking the limit from a lattice to a spatial continuum, it is desirable to
convert the Doi representation of the dynamics to a field theory. In this
section we use the coherent state representation [15] to derive the Doi-Peliti
field theory for the particle model with detailed balance dynamics. The
general technique has been presented elsewhere [14, 27, 16], so we provide
only a brief sketch here.

###  V.1 Coherent State Representation

Coherent states are introduced at each lattice site,

$|{\bm{\phi}}\rangle=e^{-\frac{1}{2}\sum_{i}|\phi_{i}|^{2}+\sum_{i}\phi_{i}\hat{a}_{i}^{\dagger}}|0\rangle$
(45)

with ${\bm{\phi}}=(\phi_{1},\phi_{2},\dots)$ and complex $\phi_{i}$. These are
eigenstates of the annihilation operator,
$a_{i}|{\bm{\phi}}\rangle=\phi_{i}|{\bm{\phi}}\rangle$. The identity operator
can be expressed with the overcompleteness relation
$\bm{1}=\int\prod_{i}\frac{d^{2}\phi_{i}}{\pi}|{\bm{\phi}}\rangle\langle{\bm{\phi}}|$.
Time evolution is broken into discrete steps of size $\Delta t$, as shown in
(31), and the identity operator is inserted at each time slice with a distinct
set of coherent states, leading to evaluation of terms of the form

$\langle{\bm{\phi}}_{t+\Delta t}|1-\hat{L}_{t}\Delta
t|{\bm{\phi}}_{t}\rangle=\langle\bm{\phi}_{t+\Delta
t}|\bm{\phi}_{t}\rangle[1-\Delta t{\cal L}(\bm{\phi^{*}}_{t+\Delta
t},\bm{\phi}_{t})].$ (46)

Using the linearized, normal ordered Liouvillian (28) we obtain

${\cal L}(\bm{\phi^{*}}_{t+\Delta t},\bm{\phi}_{t})=\Gamma\sum_{\langle
ij\rangle}(\phi^{*}_{i,t+\Delta t}-\phi^{*}_{j,t+\Delta
t})\left(\phi_{i,t}-\phi_{j,t}+\frac{\phi_{i,t}+\phi_{j,t}}{2}\left\\{\beta\epsilon_{i}({\bm{\phi}}_{t+\Delta
t},{\bm{\phi}}_{t})-\beta\epsilon_{j}({\bm{\phi}}_{t+\Delta
t},{\bm{\phi}}_{t})\right\\}\right)$ (47)

with the effective potential

$\epsilon_{i}({\bm{\phi}}_{t+\Delta
t})=U_{i}+\sum_{k}V_{ik}\phi^{*}_{k,t+\Delta t}\phi_{k,t}.$ (48)

The coherent state overlap in (46) can be written as

$\langle\bm{\phi}_{t+\Delta
t}|\bm{\phi}_{t}\rangle=e^{\frac{1}{2}|\bm{\phi}_{t+\Delta
t}|^{2}-\frac{1}{2}|\bm{\phi}_{t}|^{2}-\bar{\bm{\phi}}_{i,t+\Delta
t}\cdot(\bm{\phi}_{t+\Delta t}-\bm{\phi}_{t})}$ (49)

where $|{\bm{\phi}}|^{2}=\sum_{i}|\phi_{i}|^{2}$ and
$\bar{\bm{\phi}}_{1}\cdot\bm{\phi}_{2}=\sum_{i}\phi^{*}_{1,i}\phi_{2,i}$. The
squared terms cancel between successive time slices.

We take the continuum limit via $\phi_{i,t}\to\phi({\bf x},t)\Delta x^{d}$ and
$\phi^{*}_{i,t}\to\bar{\phi}({\bf x},t)$, so $\phi({\bf x})$ has dimensions of
density while $\bar{\phi}({\bf x})$ is non-dimensional. The result is an
action $S$ containing a “bulk” contribution as well as initial and final
contributions at $t=0$ and $t_{f}$, which can be used to compute averages via

$\langle A\rangle=\int{\cal
D}(\bar{\phi},\phi)\,A(\bar{\phi},\phi)e^{-S[\bar{\phi},\phi]}$ (50)

###  V.2 Doi-Peliti Action for Interacting Particles

From Eqs. (46) and (49) we obtain the bulk action

$S_{B}=\int_{0}^{t_{f}}dt\int d{\bf
x}\biggl{[}\bar{\phi}(\partial_{t}-D\nabla^{2})\phi-\gamma\bar{\phi}\nabla\cdot\left(\phi\nabla
U+\phi\nabla\int d{\bf y}V({\bf x}-{\bf y})\bar{\phi}({\bf y})\phi({\bf
y})\right)\biggr{]}$ (51)

with diffusion constant $D=\Gamma\Delta x^{2}$ and mobility $\gamma=D/k_{B}T$.

Note that for noninteracting particles, with $V=0$, the action (51) is linear
in $\bar{\phi}$, which creates a delta function when integrated over
$\bar{\phi}$. The resulting field $\phi$ then obeys

$\frac{\partial\phi}{\partial t}=D\nabla^{2}\phi+\gamma\nabla\cdot(\phi\nabla
U)$ (52)

which is the Fokker-Planck equation for particles diffusing in a potential
$U({\bf x})$.

The final term, consisting of the projection state and part of the coherent
state overlap, contributes a $t_{f}$ “boundary” contribution to the action of
the form

$e^{-S_{f}}=e^{\frac{1}{2}|{\bm{\phi}}_{t_{f}}|^{2}}\langle{\cal
P}|{\bm{\phi}}\rangle=e^{\sum_{i}\phi_{i,t_{f}}}$ (53)

from which we obtain the continuum limit

$S_{f}=-\int d{\bf x}\;\phi({\bf x},t_{f}).$ (54)

Generally this term is eliminated by performing a Doi shift. Instead, we will
retain this term, since it plays an essential role in understanding the time-
reversal symmetry.

The initial term also provides a $t=0$ boundary term

$e^{-S_{i}}=Z(0)^{-1}e^{-\frac{1}{2}|\bm{\phi}_{0}|^{2}}\langle\bm{\phi}_{0}|e^{-\beta\hat{H}_{0}}|\bar{n}_{0}\rangle,$
(55)

In the case of noninteracting particles, where
$\hat{H}=\sum_{j}U_{j}\hat{a}_{j}^{\dagger}\hat{a}_{j}$, this becomes

$e^{-S_{i}}=Z(0)^{-1}e^{-|\bm{\phi}_{0}|^{2}}\exp\left(\sum_{j}e^{-\beta
U_{j,0}}\phi^{*}_{j,0}\bar{n}_{0}\right)$ (56)

where we used the coherent state identity
$\langle\phi_{2}|e^{\lambda\hat{a}^{\dagger}\hat{a}}|\phi_{1}\rangle=\exp\\{(e^{\lambda}-1)\phi_{2}^{*}\phi_{1}\\}\langle\phi_{2}|\phi_{1}\rangle$
[28]. In the continuum limit, we take $\bar{n}_{0}\to n_{0}\Delta x^{d}$ and
obtain

$S_{i}=\ln Z(0)-\int d{\bf x}\left[\bar{\phi}({\bf x},0)e^{-\beta U({\bf
x},0)}n_{0}-\bar{\phi}({\bf x},0)\phi({\bf x},0)\right]$ (57)

To derive a simple expression for the initial action with interacting
particles, it will be necessary to introduce a Hubbard-Stratonovich
transformation, which we will describe below.

Finally, we note that in the field theory the work term (32) becomes

$W[\bar{\phi},\phi]=\int_{0}^{t_{f}}dt\int d{\bf x}\,\dot{U}(t)\bar{\phi}({\bf
x},t)\phi({\bf x},t).$ (58)

###  V.3 Cole-Hopf Transformation and Dean-Kawasaki Field Theory

The bulk action (51) can be transformed via the Cole-Hopf transformation
$\phi\to e^{-\bar{\rho}}\rho$, $\bar{\phi}\to e^{\bar{\rho}}$ to

$\displaystyle S_{B}=\int_{0}^{t_{f}}dt\int d{\bf
x}\biggl{[}\bar{\rho}(\partial_{t}-D\nabla^{2})\rho-\gamma\bar{\rho}\nabla\cdot(\rho\nabla\epsilon)-\gamma
D\rho(\nabla\bar{\rho})^{2}\biggr{]}$ (59)

with the effective potential

$\epsilon({\bf x})=U({\bf x})+\int d{\bf y}V({\bf x}-{\bf y})\rho({\bf y}).$
(60)

This matches the Dean-Kawasaki field theory [20] obtained from the Langevin
equation for interacting particles derived by Dean [21], then mapped to a
field theory using the MSRJD technique [22, 23, 24]. However, there appear to
be discrepancies with the initial and final terms. In particular, Dean-
Kawasaki field theory has no analog of the projection state and final action
$S_{f}$. Further the Cole-Hopf transformation does not map our initial action
to the generally assumed initial conditions for Dean-Kawasaki field theory.

##  VI Time Reversal in Doi-Peliti Field Theory

We are now equipped to examine the role of time reversal in the field theory.
We begin with noninteracting particles, where the symmetry is more directly
manifest, and then introduce a Hubbard-Stratonovich transformation to reveal
the symmetry for interacting particles.

###  VI.1 Noninteracting Particles

When $V=0$, the bulk action (51) can be written as

$S_{B}=\int_{0}^{t_{f}}dt\int d{\bf
x}\biggl{[}\bar{\phi}\partial_{t}\phi+De^{-\beta
U}\nabla\bar{\phi}\cdot\nabla(\phi e^{\beta U})\biggr{]}$ (61)

with $S_{i}$ and $S_{f}$ given by Eqs. (57) and (54). The time-reversal
symmetry of the action is revealed by the transformation

$\phi\to n_{0}e^{-\beta U}\bar{\psi}\qquad\bar{\phi}\to n_{0}^{-1}e^{\beta
U}\psi\qquad t\to t^{\prime}=t_{f}-t$ (62)

This gauge-like transformation maintains the bilinear product
$\bar{\phi}\phi\to\bar{\psi}\psi$ and the action (61) is invariant apart from
the time-derivative term. Suppressing the spatial integral, we have

$\int_{0}^{t_{f}}dt\,\bar{\phi}\partial_{t}\phi\to(\bar{\psi}\psi)\bigg{|}_{t=0}^{t=t_{f}}+\int_{0}^{t_{f}}dt^{\prime}\biggl{(}\bar{\psi}\partial_{t^{\prime}}\psi+\beta\frac{\partial
U}{\partial t^{\prime}}\bar{\psi}\psi\biggr{)}$ (63)

where we have used integration by parts. Note that time reversal generates a
work term (58).

The time reversal transformation (62) also maps the initial action (57) into
the final action (54) and vice-versa, with the help of the boundary terms in
(63). The one exception is the partition function factor, which remains
$Z(0)^{-1}$ although the time-reversed averaging would require that we start
at $t^{\prime}=0$ with $Z(t=t_{f})^{-1}$. Adjusting for the partition function
and combining the bulk, initial, and final actions, we obtain

$S[\bar{\phi},\phi]\to S_{R}[\bar{\psi},\psi]+\beta W_{R}+\beta
F(t=t_{f})-\beta F(t=0),$ (64)

where $S_{R}$ and $W_{R}$ are the time-reversed action and work term, and
$\beta F=-\ln Z$.

As a consequence of this time reversal symmetry we have the identity

$\displaystyle\langle Ae^{-\beta W}\rangle$ $\displaystyle=\int{\cal
D}(\bar{\phi},\phi)A[\bar{\phi},\phi]e^{-\beta
W[\bar{\phi},\phi]}e^{-S[\bar{\phi},\phi]}$ $\displaystyle\to e^{-\beta\Delta
F}\int{\cal
D}(\bar{\psi},\psi)A_{R}[\bar{\psi},\psi]e^{-S_{R}[\bar{\psi},\psi]}$
$\displaystyle=e^{-\beta\Delta F}\langle A_{R}\rangle_{R}$ (65)

for any observable $A[\bar{\phi},\phi]$, with time-reversed
$A_{R}[\bar{\psi},\psi]$. Here we have used that work (58) is odd under time
reversal, $W[\bar{\phi},\phi]\to-W_{R}[\bar{\psi},\psi]$, to move the
exponential of negative work to the forward time average. We will show Eq.
(65) holds for interacting particles as well, and provides the basis for
deriving multiple nonequilibrium identities.

In particular, setting $A=1$ produces the Jarzynski relation, while taking
$A=e^{(iq+\beta)W}$ reproduces Eq. (43), which can be inverse Fourier
transformed to give the Crooks relation.

###  VI.2 Interacting Particles

For interacting particles the bulk action (51) can be written as

$S_{B}=\int_{0}^{t_{f}}dt\int d{\bf
x}\biggl{[}\bar{\phi}\partial_{t}\phi+De^{-\beta\epsilon}\nabla\bar{\phi}\cdot\nabla(\phi
e^{\beta\epsilon})\biggr{]}$ (66)

where $\epsilon({\bf x},t)$ is the effective potential, given by

$\epsilon({\bf x},t)=U({\bf x},t)+\int d{\bf y}\,V({\bf x}-{\bf
y})\bar{\phi}({\bf y},t)\phi({\bf y},t).$ (67)

To affect time reversal with a local transformation like (62), we need to
employ a field Hubbard-Stratonovich transformation:

$\displaystyle\exp$ $\displaystyle\biggl{(}\beta\int d{\bf y}V({\bf x}-{\bf
y})\bar{\phi}({\bf y})\phi({\bf y})\biggr{)}$ $\displaystyle=A\int{\cal
D}\eta\exp\biggl{\\{}{-\frac{1}{2\beta}}\int d{\bf r}_{1}d{\bf
r}_{2}\,\eta^{*}({\bf r}_{1})V^{-1}({\bf r}_{12})\eta({\bf r}_{2})$
$\displaystyle\quad+\int d{\bf r}_{1}\,\eta({\bf r}_{1})\bar{\phi}({\bf
r}_{1})\phi({\bf r}_{1})\biggr{\\}}e^{-i\eta_{2}({\bf x})}$
$\displaystyle\equiv\int{\cal D}\eta F[\eta,\bar{\phi}\phi]e^{-i\eta_{2}({\bf
x})}$ (68)

where $\eta({\bf x},t)=\eta_{1}({\bf x},t)+i\eta_{2}({\bf x}.t)$ is a complex
field, $A$ is a normalization constant, ${\bf r}_{12}={\bf r}_{1}-{\bf
r}_{2}$, and $V^{-1}$ is defined formally via $\int d{\bf y}V({\bf x}-{\bf
y})V^{-1}({\bf y}-{\bf z})=\delta^{(d)}({\bf x}-{\bf z})$. Eq. (68) applies
for all times $t$, with $\eta({\bf r},t)$ uncorrelated for different times.
With this notation, the bulk action becomes

$S_{B}=\int{\cal D}\eta\,F\int dtd{\bf
x}\biggl{[}\bar{\phi}\partial_{t}\phi+De^{-\beta\epsilon}\nabla\bar{\phi}\cdot\nabla(\phi
e^{\beta U-i\eta_{2}})\biggr{]}$ (69)

where we have suppressed the time integration limits. Now the gauge-like field
transformation

$\phi\to n_{0}e^{-\beta U+i\eta_{2}}\bar{\psi}\qquad\bar{\phi}\to
n_{0}^{-1}e^{\beta U-i\eta_{2}}\psi$ (70)

with time reversal $t\to t^{\prime}=t_{f}-t$ once again leaves the action
invariant, i.e.,

$\nabla\bar{\phi}\cdot\nabla(\phi e^{\beta U-i\eta_{2}})\to\nabla(\psi
e^{\beta U-i\eta_{2}})\cdot\nabla\bar{\psi}$ (71)

apart from the time derivative term, which again has the form given in Eq.
(63), generating the work term. Importantly, the time derivative on the $\eta$
field averages to zero because the field is uncorrelated over time.

The Hubbard-Stratonovich transformation also simplifies the initial action.
The exponential of the Doi hamiltonian (18) can be written as

$e^{-\beta\hat{H}}=A\int\prod_{i}d\eta_{i}e^{-\frac{1}{2\beta}\sum_{jk}\eta_{j}V^{-1}_{jk}\eta_{k}+\sum_{j}(-\beta
U_{j}+i\eta_{j})\hat{a}_{j}^{\dagger}\hat{a}_{j}}.$ (72)

Using this in our initial action $S_{i}$, as defined in Eq. (55), we obtain an
identical expression to (56) with $e^{-\beta U_{j,0}}\to e^{-\beta
U_{j,0}+i\eta_{j}}$ and the additional integration over the $\eta_{j}$
variables. Taking the continuum limit, with $\eta_{j}\to\eta_{2}({\bf x},0)$,
results in an initial action of the form

$\displaystyle S_{i}=\ln Z(0)-\int d\eta F\int d{\bf x}\Bigl{[}$
$\displaystyle\bar{\phi}({\bf x},0)e^{-\beta U({\bf x},0)+i\eta_{2}({\bf
x},0)}n_{0}$ $\displaystyle-\bar{\phi}({\bf x},0)\phi({\bf x},0)\Bigr{]}$ (73)

while the final action (54) remains unchanged. Thus the field transformation
(70) swaps the initial and final actions apart from the partition function,
exactly as before with the noninteracting theory. Thus the fundamental
identity (65) continues to hold for interacting particles.

##  VII Nonequilibrium Identities

Starting from the fundamental relation (65) we can derive a number of
nonequilibrium identities by appropriate choice of the operator
$A[\bar{\phi},\phi]$ and functional differentiation. As discussed in the text
following (65), choosing $A=1$ gives the Jarzynski relation, and choosing
$A=e^{(iq+\beta)W}$ leads to the Crooks relation.

We can obtain a nonequilibrium generalization of the fluctuation-dissipation
relation by differentiating (65) with respect to $U({\bf x},t)$ for some
intermediate time $0<t<t_{f}$. From the field theoretic average (50) it
follows that the derivative of the left hand side is

$\frac{\delta}{\delta U({\bf x},t)}\langle Ae^{-\beta
W}\rangle=-\biggl{\langle}\biggl{(}\beta\frac{\delta W}{\delta U}+\frac{\delta
S}{\delta U}\biggr{)}Ae^{-\beta W}\biggr{\rangle},$ (74)

with

$\displaystyle\frac{\delta W}{\delta U}$ $\displaystyle=\frac{\delta}{\delta
U({\bf x},t)}\int_{0}^{t_{f}}dt^{\prime}\int d{\bf x}^{\prime}\,\dot{U}({\bf
x}^{\prime},t^{\prime})\bar{\phi}({\bf x}^{\prime},t^{\prime})\phi({\bf
x}^{\prime},t^{\prime})$ $\displaystyle=-\frac{\partial}{\partial
t}\Bigl{(}\bar{\phi}({\bf x},t)\phi({\bf x},t)\Bigr{)}$ (75)

from (58), where we have used integration by parts, and

$\displaystyle\frac{\delta S}{\delta U}$
$\displaystyle=-\gamma\frac{\delta}{\delta U({\bf
x},t)}\int_{0}^{t_{f}}dt^{\prime}\int d{\bf
x}^{\prime}\bar{\phi}\nabla\cdot(\phi\nabla U)$
$\displaystyle=-\gamma\nabla\cdot\Bigl{(}\phi({\bf x},t)\nabla\bar{\phi}({\bf
x},t)\Bigr{)}$ (76)

from (51), where we have used integration by parts twice. In contrast, the
functional derivative of the right hand side of (65) only acts on the action,
yielding

$\frac{\delta}{\delta
U}\langle\tilde{A}\rangle_{R}=\gamma\Bigl{\langle}\nabla\cdot\Bigl{(}\phi({\bf
x},t_{f}-t)\nabla\bar{\phi}({\bf
x},t_{f}-t)\Bigr{)}\tilde{A}\Bigr{\rangle}_{R}.$ (77)

Here we have assumed that $A$ and $A_{R}$ do not depend on the site potential
$U$. Combining these gives an identity

$\displaystyle\frac{\partial}{\partial t}$
$\displaystyle\langle\bar{\phi}({\bf x},t)\phi({\bf x},t)Ae^{-\beta
W}\rangle+D\langle\nabla\cdot(\phi({\bf x},t)\nabla\bar{\phi}({\bf
x},t)Ae^{-\beta W}\rangle$ $\displaystyle=De^{-\beta\Delta
F}\langle\nabla\cdot(\phi({\bf x},t_{f}-t)\nabla\bar{\phi}({\bf
x},t_{f}-t)A_{R}\rangle_{R},$ (78)

where we have used $\gamma=\beta D$. The significant of this identity is that
it holds arbitrarily far from equilibrium, i.e., it is not a linear response
relationship. Additional relations could be derived by further differentiation
with respect to $U$.

For the choice $A=\bar{\phi}({\bf x}^{\prime},t^{\prime})\phi({\bf
x}^{\prime},t^{\prime})$ Eq. (LABEL:noneq-fd-i) becomes a nonequilibrium
fluctuation dissipation relation. Let $\rho=\bar{\phi}\phi$ and take $U({\bf
x},t)\to U({\bf x},t)+U_{1}({\bf x},t)$. Then

$\displaystyle\beta$ $\displaystyle\frac{\partial}{\partial t}\langle\rho({\bf
x},t)\rho({\bf x}^{\prime},t^{\prime})e^{-\beta W}\rangle$
$\displaystyle=\frac{\delta\langle\rho({\bf x}^{\prime},t^{\prime})e^{-\beta
W}\rangle}{\delta U_{1}({\bf x},t)}-e^{-\beta\Delta
F}\frac{\delta\langle\rho({\bf
x}^{\prime},t_{f}-t^{\prime})\rangle_{R}}{\delta U_{1}({\bf x},t_{f}-t)}$ (79)

with the functional derivatives evaluated at $U_{1}=0$, and with the
understanding that $W$ is determined by $U$ and not $U_{1}$, as defined in
(58). Thus a time derivative of a correlation function, modified to include
the Jarzynski work term, is related to a response function. The right hand
side has effectively a $\theta$ function, with the first term nonzero when
$t^{\prime}>t$ and the second term nonzero with $t^{\prime}<t$.

We note that taking functional derivatives of the Jarzynski relation to obtain
a fluctuation dissipation relation within linear response, as done in [29,
30], differs from this field theoretic result which, like that obtained in
[10], applies arbitrarily far from equilibrium.

##  VIII Summary

We have generalized the Doi representation to describe interacting particles
coupled to a thermal reservoir, undergoing hops with rates determined by
detailed balance. Such a system can be driven far from equilibrium by a
rapidly-varying local potential. We demonstrated that the Jarzynski and Crooks
relations arise straightforwardly in the Doi representation, with the initial
state and the projection state playing a crucial role.

Then, by mapping the Doi representation to a field theory via the coherent
state representation, we obtained the Doi-Peliti field theory for these
interacting Brownian particles. We demonstrated that time reversal in this
field theory has a gauge-like form, and that the time reversal operation
results in a fundamental identity (65) from which the Jarzynski and Crooks
relations can be derived, along with a nonequilibrium generalization of the
fluctuation-dissipation relation.

Future work could include generalizing the formalism to include chemical
reactions and allowing for chemical work to be done on the system, and
deriving a general framework for entropy production within Doi-Peliti field
theory. It could also be fruitful to develop a perturbative expansion in the
interaction strength.

###### Acknowledgements.

BPV-L would like to thank Chris Jarzynski and Uwe Täuber for helpful
discussions.

## References

  * Jarzynski [1997a] C. Jarzynski, Phys. Rev. Lett. 78, 2690 (1997a).
  * Jarzynski [1997b] C. Jarzynski, Phys. Rev. E 56, 5018 (1997b).
  * Crooks [1998] G. E. Crooks, J. Stat. Phys. 90, 1481 (1998).
  * Crooks [1999] G. E. Crooks, Phys. Rev. E 60, 2721 (1999).
  * Jarzynski [2008] C. Jarzynski, The European Physical Journal B 64, 331 (2008).
  * Jarzynski [2011] C. Jarzynski, Ann. Rev. of Cond. Matt. Phys. 2, 329 (2011).
  * Seifert [2012] U. Seifert, Rep. Prog. Phys. 75, 126001 (2012).
  * Peliti and Pigolotti [2021] L. Peliti and S. Pigolotti,  _Stochastic Thermodynamics: An Introduction_ (Princeton University Press, Princeton, NJ, USA, 2021).
  * Shiraishi [2023] N. Shiraishi,  _An Introduction to Stochastic Dynamics_ (Springer, Singapore, 2023).
  * Mallick  _et al._ [2011] K. Mallick, M. Moshe, and H. Orland, J. Phys. A 44, 095002 (2011).
  * Hohenberg and Halperin [1977] P. C. Hohenberg and B. I. Halperin, Rev. Mod. Phys. 49, 435 (1977).
  * Täuber [2014] U. C. Täuber,  _Critical Dynamics: A Field Theory Approach to Equilibrium and Non-Equilibrium Scaling Behavior_ (Cambridge University Press, Cambridge, UK, 2014).
  * Doi [1976] M. Doi, J. Phys. A 9, 1465 (1976).
  * Peliti [1985] L. Peliti, J. Phys. 46, 1469 (1985).
  * Schulman [1996] L. Schulman,  _Techniques and Applications of Path Integration_ (Wiley, New York, 1996).
  * Täuber  _et al._ [2005] U. C. Täuber, M. Howard, and B. P. Vollmayr-Lee, J. Phys. A 38, R79 (2005).
  * Andreanov  _et al._ [2006a] A. Andreanov, G. Biroli, J.-P. Bouchaud, and A. Lefèvre, Phys. Rev. E 74, 030101 (2006a).
  * Andreanov  _et al._ [2006b] A. Andreanov, G. Biroli, and A. Lefèvre, J. Stat. Mech. 2006, P07008 (2006b).
  * Lefèvre and Biroli [2007] A. Lefèvre and G. Biroli, J. Stat. Mech. 2007, P07024 (2007).
  * Kim  _et al._ [2014] B. Kim, K. Kawasaki, H. Jacquin, and F. van Wijland, Phys. Rev. E 89, 012150 (2014).
  * Dean [1996] D. S. Dean, J. Phys. A 29, L613 (1996).
  * Martin  _et al._ [1973] P. C. Martin, E. D. Siggia, and H. A. Rose, Phys. Rev. A 8, 423 (1973).
  * Janssen [1976] H.-K. Janssen, Z. Phys. B 23, 377 (1976).
  * de Dominicis [1976] C. de Dominicis, J. Phys. Colloques 37, C1 (1976).
  * Kurchan [2007] J. Kurchan, J. Stat. Mech. 2007, P07005 (2007).
  * Hummer and Szabo [2001] G. Hummer and A. Szabo, Proc. Ntl. Acad. Sci. 98, 3658 (2001).
  * Lee [1994] B. P. Lee, J. Phys. A 27, 2633 (1994).
  * Katriel [2000] J. Katriel, Phys. Lett. A 273, 159 (2000).
  * Chetrite and Gawedzki [2008] R. Chetrite and K. Gawedzki, Communications in Mathematical Physics 282, 469 (2008).
  * Chetrite and Gawedzki [2009] R. Chetrite and K. Gawedzki, Journal of Statistical Physics 137, 890 (2009).