# Reducing Flipping Errors in Deep Neural Networks

Xiang Deng1222Interns at JD.com., Yun Xiao2, Bo Long2, Zhongfei Zhang1

# Reducing Flipping Errors in Deep Neural Networks 111Code:
https://github.com/Xiang-Deng-DL/FER

Xiang Deng1222Interns at JD.com., Yun Xiao2, Bo Long2, Zhongfei Zhang1

###### Abstract

Deep neural networks (DNNs) have been widely applied in various domains in
artificial intelligence including computer vision and natural language
processing. A DNN is typically trained for many epochs and then a validation
dataset is used to select the DNN in an epoch (we simply call this epoch “the
last epoch”) as the final model for making predictions on unseen samples,
while it usually cannot achieve a perfect accuracy on unseen samples. An
interesting question is “how many test (unseen) samples that a DNN
misclassifies in the last epoch were ever correctly classified by the DNN
before the last epoch?”. In this paper, we empirically study this question and
find on several benchmark datasets that the vast majority of the misclassified
samples in the last epoch were ever classified correctly before the last
epoch, which means that the predictions for these samples were flipped from
“correct” to “wrong”. Motivated by this observation, we propose to restrict
the behavior changes of a DNN on the correctly-classified samples so that the
correct local boundaries can be maintained and the flipping error on unseen
samples can be largely reduced. Extensive experiments on different benchmark
datasets with different modern network architectures demonstrate that the
proposed flipping error reduction (FER) approach can substantially improve the
generalization, the robustness, and the transferability of DNNs without
introducing any additional network parameters or inference cost, only with a
negligible training overhead.

## Introduction

The superior performances of deep neural networks (DNNs) have driven their
wide deployment in varieties of applications in artificial intelligence such
as medical diagnosis (Miotto et al. 2016), automated vehicle control (Levinson
et al. 2011), and financial business (Ozbayoglu, Gudelek, and Sezer 2020).
Despite the remarkable performances, it is appealing and necessary to further
improve them, thus benefiting these applications and extending the application
horizon to more accuracy-critical or safety-critical domains. However,
furthering improving the performance of a DNN usually needs to substantially,
even exponentially, increase the number of its parameters (Simonyan and
Zisserman 2015; He et al. 2016a), which leads to a large amount of computation
and memory costs. In this work, we aim to enhance the performance of a DNN
classifier without introducing any additional network parameters, without
increasing inference cost, and almost without training overhead by exploring
the learning dynamics of DNNs.

Figure 1: RFEs of different DNNs on CIFAR-100 and Standford Dogs.

Currently, DNNs are typically trained for many epochs and then a validation
dataset is used to select the DNN in the best epoch as the final model for
making predictions on unseen samples or the validation dataset is used to set
up the total number of training epochs and then the DNN at the last epoch is
selected as the final model. Without losing generality, we simply denote the
selected epoch in both cases by the last epoch. An interesting question
regarding the generalization of a DNN classifier is “how many misclassified
test samples in the last epoch were ever correctly classified by the DNN
before the last epoch?”. To quantitatively study this question, we first
introduce several novel metrics and definitions: (1) Wrongly Flipped Samples
(WFSs) are the samples that are misclassified by the DNN in the last epoch but
were ever classified correctly by the DNN before the last epoch; (2) Relative
Flipping Error (RFE) is the ratio of WFSs to the total number of misclassified
test samples (MTSs) in the last epoch, i.e., $RFE=\frac{\\#WFSs}{\\#MTSs}$. We
report the RFEs of various modern networks on two different benchmark datasets
including a regular classification dataset, i.e., CIFAR-100 (Krizhevsky and
Hinton 2009) and a fine-grained classification dataset, i.e., Standford Dogs
(Khosla et al. 2011). As shown in Figure 1, across different modern network
architectures on both datsets, a large percentage of the wrongly-classified
test samples in the last epoch were ever correctly classified before the last
epoch, e.g., the RFEs of ResNet-18 are over 90% on both datasets. This
indicates that the DNN has ever learned the correct local boundaries for these
samples but then flipped them to the wrong directions.

Inspired by the observation, we propose a novel framework, i.e., flipping
error reduction (FER), to reduce the number of wrongly-flipped samples and
thus to enhance DNN performances. FER is based on the idea that once a DNN has
made a correct prediction on a sample, we limit the behavior change of the DNN
on this sample with the goal of maintaining the correct local classification
boundaries near this sample to avoid wrong flipping. To capture the local
boundaries, we define the behavior of a DNN on a sample as its output soft
distribution over different classes which captures the distances between a
sample and different class boundaries. Since a DNN may make correct
predictions in many epochs for a sample, we maintain a weighted average of
these correct behaviors based on their confidences and epoch numbers. A
behavior with a higher confidence or in a later epoch contributes more to the
average. The weighted average behavior is more stable and accurate than a
single behavior for restricting the DNN behavior changes on well-learned
samples.

FER is a simple and easy-to-implement yet effective approach for training
DNNs. Extensive experiments on different types of benchmark datasets
demonstrate that FER can improve the generalization, the robustness, and the
transferability of DNNs without introducing any additional network parameters,
without increasing inference cost but only with a negligible training overhead
for maintaining the average behaviors of a DNN on correctly-classified
samples.

Our contributions are summarized as follows:

  * •

We study a novel problem about ”how many wrongly-classified test samples in
the last epoch were ever correctly classified before the last epoch?” and
define novel metrics to quantitatively investigate this question and further
propose a solution to this problem. To our best knowledge, this is the first
work to address this issue.

  * •

To reduce flipping errors, we propose a novel framework, i.e., FER, that
restricts the behavior changes of a DNN on correctly-classified samples and
thus maintains the correct local boundaries near these samples.

  * •

We empirically show on different types of benchmark datasets that FER is able
to improve the generalization, the robustness, and the transferability of
modern DNNs, without introducing any network parameters or inference cost,
only with a negligible training overhead.

## Related Work

In this paper, we study the wrongly flipped predictions on unseen samples.
This phenomenon also happens to training (seen) samples (Toneva et al. 2019).
However, as current DNNs are powerful enough to memorize the whole training
dataset, these wrongly-flipped predictions on training samples can almost
always be corrected by the DNNs through memorization in the later epochs.
Memorization does not mean generalization and thus the wrong flipping issue on
unseen samples is more challenging. FER is designed to solve this problem by
restricting the soft output distribution changes of a DNN on correctly-
classified samples to maintain local correct classification boundaries. Hence,
it is technically related to label smoothing (Szegedy et al. 2016) and
knowledge distillation (KD) techniques (Hinton, Vinyals, and Dean 2015; Deng
and Zhang 2021a, b) which are also based on soft distribution.

Label Smoothing Szegedy et al. (2016) propose the label smoothing regularizer
(LSR) that linearly interpolates between a one-hot label and a uniform
distribution to generate a soft distribution as the label for training DNNs,
and empirically show that it is able to improve the performance of the
Inception architecture on image classification. Ever since then, LSR has been
used in various advanced image classification models (Zoph et al. 2018; Real
et al. 2019; Huang et al. 2019) and has also been introduced to other domains.
In speech recognition, Chorowski and Jaitly (2016) show that label smoothing
reduces the word error rate on the WSJ dataset. In machine translation,
Vaswani et al. (2017) show that label smoothing improves the BLEU score but
hurts the perplexity. Muller et al. (2019) empirically demonstrate that LSR
leads to a better calibration for DNNs. Recently, Yuan et al. (2020) propose a
new manually designed label smoothing based regularizer, i.e., TF-Reg, that
combines LSR and a temperature. These label smoothing techniques manually
assign the same small probability to the non-ground-truth classes and thus
fail to take into account sample-to-class similarities and cannot fully
utilize the advantages of soft labels.

Figure 2: Framework of FER.

Knowledge Distillation Knowledge distillation (KD) (Hinton, Vinyals, and Dean
2015) overcomes the limitations of the manually designed soft labels by taking
advantage of a pretrained teacher network to generate soft labels which
contain sample-to-class similarity (distance) information. However, it also
brings several times of training cost over the manually designed soft labels
due to the two facts: (1) KD first requires training a large DNN as the
teacher; (2) when training the student, KD needs to process each sample twice
in each iteration, once by the teacher and once by the student. To reduce the
cost of training a large teacher, many self-distillation approaches including
but not limited to (Xu and Liu 2019; Zhang et al. 2019; Yang et al. 2019b, a;
Furlanello et al. 2018; Bagherinezhad et al. 2018; Yun et al. 2020; Deng and
Zhang 2021c; Ji et al. 2021) have been proposed. Zhang et al. (2019) and Ji et
al. (2021) add additional layers or parameters to a DNN to generate soft
labels, which improves the performance but introduces large computation and
memory cost. Born-again networks (Furlanello et al. 2018; Yang et al. 2019a)
and label-refine networks (Bagherinezhad et al. 2018) train a DNN for many
generations and the network in the $(i-1)$th generation is used as the teacher
to train the network in the $i$th generation. In other words, these approaches
avoid pretraining a large teacher network by pretraining the network itself as
its own teacher, and this process can be repeated many times. Nevertheless,
they still have a large training overhead as they need to train a network for
many times. To further reduce the training cost for multiple-generation
training, SD (Yang et al. 2019b) introduces mini-generation training. It
borrows the idea from (Huang et al. 2017) to rely on a cyclic learning rate
schedule (Loshchilov and Hutter 2017) to train DNNs for many mini-generations.
However, this learning rate schedule causes that SD cannot update the soft
labels frequently and introduces wrong supervision signals from mini-
generations. To free the update frequency of soft labels, LWR (Deng and Zhang
2021c) uses the sample-to-class similarities learned in a past epoch in a
standard training procedure to generate soft labels. However, it still
introduces wrong supervision signals for misclassified samples and the soft
labels generated from the information in one epoch are unstable and less
accurate. FER addresses these issues by limiting the behavior changes of a DNN
on correctly-classified samples and taking into account the DNN behaviors in
many epochs.

Note that we aim to improve the DNN performance with comparable training and
inference costs to the standard training procedure through exploring the
learning dynamics of DNNs. Thus, we do not compare the proposed method with
those approaches (e.g., born-again networks or KD) whose training or inference
cost is several times of ours.

## Framework

In this section, we introduce FER for improving DNN performances by exploring
the learning dynamics of DNNs. To show the connection between FER and the
standard training procedure, we first review the standard procedure and then
make further derivations to present FER.

### Standard Training Procedure

We denote the training dataset by $\mathcal{D}=\\{(x_{i},y_{i})\\}_{i=1}^{n}$
of $n$ samples where $x_{i}$ denotes an input sample and $y_{i}$ represents
the corresponding one-hot label. The standard training (STD) procedure trains
a DNN $f$ by minimizing the cross-entropy loss between the DNN output and the
one-hot label:

$\mathcal{L}_{CE}(x)=\mathcal{H}(f(x),y)$ (1)

where $\mathcal{H}$ denotes cross-entropy.

At the beginning of the training process, a DNN with initial weights makes
wrong predictions on most samples. The DNN is then trained for multiple epochs
with the goal of making correct predictions on data samples. However, we find
that the behaviors of a DNN on many test samples are flipped from correct
predictions to wrong predictions as the training progresses, which is opposite
to the learning goal. Inspired by this observation, we propose to improve DNN
performances by reducing this kind of flipping errors.

### Flipping Error Reduction

To reduce flipping errors, we propose to regularize the behavior changes of a
DNN on the samples that are already classified correctly. Note that we do not
restrict the behavior of a DNN on the misclassified samples, since the local
boundaries near these samples are still wrong and cannot be used to make
correct predictions on unseen samples. The behavior of a DNN on a sample is
defined as a soft distribution:

$b(x)=\sigma(\frac{f(x)}{\tau})$ (2)

where $\sigma$ denotes the softmax function and $\tau$ is a temperature to
soften the outputs. The advantage of using a soft distribution is that soft
labels are able to capture the sample-to-class-margin distances, i.e., sample-
to-class similarities. By restricting the behavior changes of a DNN on
correctly-classified samples, the correct local boundaries near these samples
can be maintained, thus reducing wrong flipping.

Since a DNN is typically trained for many epochs, there can be multiple
correct behaviors of a DNN on a sample. These correct behaviors are generated
in different epochs and may also have different confidences. Here the
confidence means the probability for the ground-truth class, i.e.,
$c=\sigma(f(x))[\textbf{y}]$ where y is the ground-truth class index. A
natural question is “which correct behavior should be used for restricting the
DNN behavior on a sample?”. As shown in Figure 2, we maintain a weighted
average of all these correct behaviors based on the confidence and the epoch
number of each behavior for each sample. Specifically, in epoch $k$, suppose
that the DNN has made correct predictions in $j$ epochs for sample $x$ where
$0\leq j\leq k-1$. We denote the behaviors in these $j$ epochs with correct
predictions by $[b_{1}(x),b_{2}(x),...,b_{j}(x)]$ where $b_{i}(x)$ is the
behavior of the DNN on sample $x$ in the $i$th behavior-correct epoch. For the
behavior in a later epoch or with a high confidence, we assign a larger weight
as the DNN has learned more information or is more confident. The weighted
average behavior in epoch $k$ is calculated with:

$\mathbf{\hat{b}}_{k}(x)=\mathrm{e}^{-\mu
c_{j}}\mathbf{\hat{b}}_{k-1}(x)+(1-\mathrm{e}^{-\mu c_{j}})b_{j}(x)$ (3)

where $c_{j}$ is the confidence of $b_{j}(x)$ and $\mu$ is a coefficient to
scale the contribution of confidence $c_{j}$. Besides $b_{j}(x)$, all the
other behaviors with correct predictions also contribute to the current
average behavior $\mathbf{\hat{b}}_{k}(x)$ as they are involved in computing
$\mathbf{\hat{b}}_{k-1}(x)$. We then restrict the behavior changes of the DNN
on correctly-classified samples with a regularizer:

$\mathcal{L}_{cor}(x)=\alpha\mathcal{L}_{CE}(x)+\beta\mathcal{K}(\sigma(\frac{f(x)}{\tau}),\mathbf{\hat{b}}_{k}(x))$
(4)

where $\alpha$ and $\beta$ are two balancing weights; $\mathcal{K}$ denotes
KL-divergence. The gradients do not flow through $\mathbf{\hat{b}}_{k}(x)$ as
it is the record of the past behaviors of a DNN. In most cases, we simply set
$\alpha$ and $\beta$ to $1.0-0.9\frac{k}{M}$ and $0.9\frac{k}{M}$ (where $M$
is the number of total training epochs), respectively, based on the fact that
the average behavior becomes more accurate in the later epoch with the DNN
learning more information.

FER restricts the behaviors of a DNN on correctly-classified samples but gives
the totally free space for the DNN on learning misclassified samples, thus the
final objective is written as:

$\mathcal{L}_{FER}(x)=\mathbf{1}_{[T(x)]}\mathcal{L}_{cor}(x)+(1-\mathbf{1}_{[T(x)]})\mathcal{L}_{CE}(x)$
(5)

where $\mathbf{1}_{[.]}$ is the indicator function which equals to one if the
argument in the subscript [.] is true and zero otherwise; $T(x)$ denotes the
argument that sample $x$ has been ever classified correctly before the current
epoch. As we can see, the training overhead of FER is the cost for maintaining
a soft behavior average (i.e., $\mathbf{\hat{b}}_{k}(x)$) for each correctly-
classified sample, which is negligible as these soft behaviors do not need to
be stored in the GPU memory.

### Analysis of FER

##### Label Smoothing Effects

FER is highly related to LSR as it defines the DNN behavior as a soft
distribution. LSR assigns a small probability to the non-ground-truth classes
by using the weighted average of the one-hot label $y$ and a uniform
distribution as the training targets, i.e.,
$y_{LSR}=(1-\epsilon)y+\frac{\epsilon}{m}$ where $\epsilon$ is a manually set
hyper-parameter and usually set to 0.1; $m$ is the total number of classes.
The objective of LSR is thus expressed as:

$\begin{split}\mathcal{L}_{LSR}=\mathcal{K}(\sigma(f(x)),y_{LSR})\\\
=(1-\epsilon)\mathcal{L}_{CE}(x)+\epsilon\mathcal{K}(\sigma(f(x),\frac{\epsilon}{m}))\end{split}$
(6)

It is observed that (6) shares a similar form to (4) that is the loss for the
correctly-classified samples in FER. FER is thus reduced to LSR when
$\mathbf{\hat{b}}(x)$ follows a manually-set uniform distribution and the
regularizer is put on all samples by ignoring whether the predictions on them
are already correct or not. This indicates that FER is an adaptive version of
LSR and should inherit the advantage of LSR such as a better generalization.
The advantage of FER over LSR is that FER can capture the sample-to-class
distance information which is accumulated from the past behaviors of the DNN,
while LSR fails to provide this kind of information as it assigns the same
small probability to all the non-ground-truth classes. To further illustrate
this point, we provide an intuitive example in Figure 3.

##### Maintaining Correct Local Classification Boundaries

FER defines the the DNN behavior as the soft output distribution over
different classes which contains sample-to-class distance information. It
restricts the DNN behavior changes on the correctly-classified samples and
gives free space for learning wrongly-classified samples. FER thus mitigates
wrong flipping through maintaining the correct local boundaries while allowing
correct flipping (i.e., from wrong predictions to correct predictions) for
wrongly-classified samples. We empirically validate the flipping error
reduction effects of FER in the ablation studies.

## Experiments

We first conduct ablation studies and then compare FER with cost-comparable
label-smoothing approaches.

Figure 3: Due to the body type differences, the dog in the first row is more
visually similar to a cat while the the dog in the second row is more visually
similar to a wolf. Different from LSR that assigns them the same soft label,
FER uses the weighted average behavior that assigns different probabilities to
different classes.

Figure 4: Flipping errors on CIFAR-100.

Figure 5: Flipping errors on Standford Dogs.

|  | ShuffleNetV2 | ResNet-18
---|---|---|---
CIFAR -100 | FER | 74.82$\pm$0.24 | 80.29$\pm$0.12
FER w/o S1 | 74.48$\pm$0.33 | 80.00$\pm$0.18
FER w/o S2 | 74.46$\pm$0.19 | 79.90$\pm$0.07
LWR | 73.53$\pm$0.33 | 79.73$\pm$0.32
Standford Dogs | FER | 68.76$\pm$0.17 | 70.64$\pm$0.25
FER w/o S1 | 68.01$\pm$0.40 | 70.25$\pm$0.25
FER w/o S2 | 67.82$\pm$0.10 | 70.08$\pm$0.18
LWR | 67.39$\pm$0.42 | 69.84$\pm$0.45
Table 1: Ablation studies regarding the strategies in FER. |  $\mu$ = 0.5 |  $\mu$ = 1.0 |  $\mu$ = 1.5
---|---|---|---
$\tau$ = 5 | 79.95$\pm$0.14 | 80.29$\pm$0.12 | 79.91$\pm$0.16
$\tau$ = 10 | 79.40$\pm$0.33 | 79.63$\pm$0.15 | 79.64$\pm$0.09
Table 2: Effects of $\tau$ and $\mu$ with ResNet-18 on CIFAR-100.

|  | ResNet-18 | WRN-40-2 | ShuffleNetV2 | MobileNetV2
---|---|---|---|---|---
CIFAR-100 | STD | 77.31$\pm$0.33 | 76.24$\pm$0.15 | 71.58$\pm$0.19 | 64.27$\pm$0.81
LSR | 78.24$\pm$0.16 | 76.29$\pm$0.25 | 71.90$\pm$0.13 | 64.54$\pm$0.74
Max-Entropy | 77.65$\pm$0.35 | 76.34$\pm$0.15 | 71.23$\pm$0.22 | 64.34$\pm$0.46
SD | 78.31$\pm$0.24 | 76.34$\pm$0.12 | 68.70$\pm$0.51 | 65.78$\pm$0.13
CS-KD | 78.01$\pm$0.13 | 76.25$\pm$0.10 | 71.43$\pm$0.28 | 64.81$\pm$0.20
TF-Reg | 77.36$\pm$0.23 | 76.32$\pm$0.19 | 72.09$\pm$0.34 | 64.87$\pm$0.27
FER (Ours) | 80.29$\pm$0.12 ($\uparrow$ 2.98) | 77.80$\pm$0.13 ($\uparrow$ 1.56) | 74.82$\pm$0.24 ($\uparrow$ 3.24) | 66.84$\pm$0.79 ($\uparrow$ 2.57)
Tiny ImageNet | STD | 65.57$\pm$0.16 | 58.71$\pm$0.44 | 60.80$\pm$0.25 | 55.53$\pm$0.76
LSR | 64.91$\pm$0.08 | 58.97$\pm$0.10 | 61.85$\pm$0.27 | 56.04$\pm$0.41
Max-Entropy | 65.25$\pm$0.16 | 58.82$\pm$0.22 | 61.56$\pm$0.40 | 55.92$\pm$0.42
SD | 65.87$\pm$0.28 | 61.01$\pm$0.10 | 60.79$\pm$0.15 | 55.97$\pm$0.59
CS-KD | 64.29$\pm$0.25 | 59.67$\pm$0.13 | 61.66$\pm$0.34 | 57.08$\pm$0.52
TF-Reg | 64.88$\pm$0.47 | 58.62$\pm$0.49 | 61.55$\pm$0.42 | 56.03$\pm$0.23
FER (Ours) | 67.72$\pm$0.32 ($\uparrow$ 2.15) | 62.10$\pm$0.15 ($\uparrow$ 3.39) | 62.61$\pm$0.04 ($\uparrow$ 1.81) | 57.43$\pm$0.23 ($\uparrow$ 1.90)

Table 3: Test accuracies (%) on CIFAR-100 and Tiny ImageNet. $\uparrow$
denotes the absolute improvement over the standard training procedure (i.e.,
STD).

### Ablation Studies

#### Can FER Reduce Flipping Errors?

FER is developed from the STD training procedure with the goal of reducing the
number of wrongly flipped samples. In this part, we aim to empirically
validate that FER indeed reduces the flipping errors of DNNs. The flipping
error (FE) is quantitatively defined as the ratio of the number of wrongly
flipped test samples (WFSs) to the total number of test samples (TSs):

$FE=\frac{\\#WFSs}{\\#TSs}$ (7)

We conduct experiments on two different types of datasets including one
regular classification dataset and one fine-grained dataset, i.e., CIFAR-100
(Krizhevsky and Hinton 2009) and Standard Dogs (Khosla et al. 2011).

Figure 5 and Figure 5 report the FEs on CIFAR-100 and Standford Dogs,
respectively. It is observed that by simply restricting the DNN behaviors on
correctly-classified samples, FER consistently and substantially reduces the
FEs across different networks and datasets, which demonstrates the ability of
FER to mitigate wrong flipping.

#### FER v.s. LWR

There are two strategies in FER: (1) only restricting the DNN behavior changes
on correctly-classified samples while giving the DNN totally free space for
learning on misclassified samples, instead of restricting the DNN behavior on
all the samples; (2) using the weighted average of multiple correct behaviors
instead of one single behavior to restrict a DNN. In this part, we empirically
examine the effects of the two strategies. We denote FER without strategy (1)
by FER w/o S1, without strategy (2) by FER w/o S2. Without both of the two
strategies, FER is reduced to LWR (Deng and Zhang 2021c) that is a special
case of FER.

Table 1 reports the performances on CIFAR-100 and Standford Dogs. As expected,
without either of the two strategies or without both (i.e., LWR), the
performances drop significantly on both datasets, which demonstrates the
effectiveness of the two strategies and the superiority of FER over LWR.

#### Effects of $\tau$ and $\mu$

$\tau$ in FER controls the softness of the behavior while $\mu$ scales the
contribution of the confidence of each behavior to the weighted average. Their
effects on the performances of FER are reported in Table 2. It is observed
that $\tau$ = 5 consistently outperforms $\tau$ = 10. The reason is that as
seen from Eq. (2), a large $\tau$ makes the behavior too flat to contain
sample-to-class distance information. On the other hand, we observe that the
performance of FER is not sensitive to $\mu$.

|  | ResNet-10 | ResNet-18 | MobileNetV2 | ShuffleNetV2
---|---|---|---|---|---
CUB | STD | 58.96$\pm$0.12 | 61.09$\pm$0.49 | 67.20$\pm$0.21 | 61.67$\pm$0.85
LSR | 59.31$\pm$0.21 | 63.57$\pm$0.50 | 67.97$\pm$0.43 | 62.66$\pm$0.11
Max-Entropy | 59.00$\pm$0.30 | 61.23$\pm$0.37 | 66.56$\pm$0.30 | 61.10$\pm$0.15
SD | 59.28$\pm$0.34 | 64.19$\pm$0.24 | 68.15$\pm$0.32 | 63.99$\pm$0.29
CS-KD | 60.70$\pm$0.21 | 64.57$\pm$0.29 | 67.48$\pm$0.32 | 63.32$\pm$0.25
TF-Reg | 58.84$\pm$0.60 | 62.04$\pm$0.28 | 67.20$\pm$0.43 | 61.19$\pm$0.58
FER (Ours) | 64.86$\pm$0.45 ($\uparrow$ 5.90) | 67.65$\pm$0.25 ($\uparrow$ 6.56) | 69.08$\pm$0.47 ($\uparrow$ 1.88) | 65.60$\pm$0.26 ($\uparrow$ 3.93)
Stanford Dogs | STD | 63.91$\pm$0.25 | 66.56$\pm$0.28 | 68.05$\pm$0.26 | 66.08$\pm$0.32
LSR | 63.36$\pm$0.03 | 67.12$\pm$0.86 | 69.13$\pm$0.09 | 66.90$\pm$0.34
Max-Entropy | 63.94$\pm$0.30 | 66.42$\pm$0.50 | 67.97$\pm$0.30 | 66.25$\pm$0.60
SD | 64.65$\pm$0.36 | 68.79$\pm$0.06 | 70.26$\pm$0.35 | 67.30$\pm$0.26
CS-KD | 64.91$\pm$0.26 | 69.17$\pm$0.19 | 68.73$\pm$0.25 | 66.75$\pm$0.31
TF-Reg | 63.72$\pm$0.44 | 66.53$\pm$0.50 | 68.36$\pm$0.26 | 66.63$\pm$0.26
FER (Ours) | 67.12$\pm$0.29 ($\uparrow$ 3.21) | 70.64$\pm$0.25 ($\uparrow$ 4.08) | 70.78$\pm$0.31 ($\uparrow$ 2.73) | 68.76$\pm$0.17 ($\uparrow$ 2.68)
FGVC-Aircraft | STD | 73.89$\pm$0.25 | 79.58$\pm$0.25 | 83.01$\pm$0.30 | 78.00$\pm$0.45
LSR | 74.52$\pm$0.18 | 80.91$\pm$0.28 | 83.88$\pm$0.10 | 78.40$\pm$0.94
Max-Entropy | 73.39$\pm$0.09 | 79.59$\pm$0.41 | 82.81$\pm$0.45 | 78.20$\pm$0.25
SD | 74.98$\pm$0.38 | 80.55$\pm$0.80 | 83.36$\pm$0.13 | 78.09$\pm$0.34
CS-KD | 74.95$\pm$0.40 | 79.72$\pm$0.19 | 80.62$\pm$0.38 | 77.89$\pm$1.55
TF-Reg | 73.69$\pm$0.38 | 80.12$\pm$0.33 | 83.39$\pm$0.11 | 78.50$\pm$0.31
FER (Ours) | 77.02$\pm$0.11 ($\uparrow$ 3.13) | 82.38 $\pm$0.10 ($\uparrow$ 2.80) | 84.56$\pm$0.82 ($\uparrow$ 1.55) | 78.60$\pm$0.46 ($\uparrow$ 0.60)

Table 4: Test accuracies (%) on fine-grained classification datasets.
$\uparrow$ denotes the absolute improvement over the STD procedure.

| Arcene | Iris
---|---|---
STD | 83.00$\pm$2.16 | 90.00$\pm$2.72
LSR | 81.00$\pm$4.55 | 92.22$\pm$1.57
Max-Ent | 79.67$\pm$3.77 | 94.44$\pm$1.57
SD | 81.00$\pm$1.63 | 93.33$\pm$2.72
CS-KD | 83.67$\pm$0.94 | 94.44$\pm$1.57
TF-Reg | 80.00$\pm$2.16 | 90.00$\pm$2.72
FER (Ours) | 86.33$\pm$0.94 ($\uparrow$ 3.33) | 95.56$\pm$1.57 ($\uparrow$ 5.56)

Table 5: Test accuracies (%) on tabular datasets.

### Generalization on Test Data

We use the test accuracy to measure the generalization of different methods on
test data.

Datasets We adopt different kinds of datasets across different domains
including two regular classification datasets, i.e., CIFAR-100 (Krizhevsky and
Hinton 2009) and Tiny-ImageNet 333https://tiny-imagenet.herokuapp.com, three
fined-grained classification datasets, i.e., CUB-200-2011 (Wah et al. 2011),
Stanford Dogs (Khosla et al. 2011), and FGVC-Aircraft (Maji et al. 2013), two
tabular datasets, i.e., Arcene (Dua and Graff 2017), and Iris (Dua and Graff
2017).

Architectures To examine whether FER works on different modern architectures,
we adopt ResNet (He et al. 2016a, b), WRN (Zagoruyko and Komodakis 2016),
ShuffleNet (Ma et al. 2018), and MobileNet (Sandler et al. 2018).

Baselines As FER is modified from the the standard training procedure (STD),
STD is considered as one of the most important baselines. In addition, since
FER uses soft distribution, we also compare FER with label smoothing based
methods including cost-comparable self-distillation approaches, i.e., LSR
(Müller, Kornblith, and Hinton 2019; Szegedy et al. 2016), Max-Entropy
(Pereyra et al. 2017), SD (Yang et al. 2019b), CS-KD (Yun et al. 2020), and
TF-Reg (Yuan et al. 2020). The comparison and discussion with LWR (Deng and
Zhang 2021c) are given in the above ablation studies.

More specific details regarding these datasets, baselines, and hyper-
parameters are given in Appendix due to space limitation.

#### Regular Classification

The regular classification is conducted on CIFAR-100 and Tiny ImageNet. The
comparison results are reported in Table 3. It is clearly observed that by
simply restricting the DNN behavior changes on correctly-classified samples to
maintain correct local boundaries, FER consistently outperforms the standard
training procedure (i.e., STD) by a large margin with different network
architectures across different datasets, e.g., the absolute accuracy
improvements with ResNet-18 are 2.98% and 2.15% on CIFAR-100 and Tiny
ImageNet, respectively, which demonstrates the effectiveness of FER. On the
other hand, FER also beats the other competitors significantly with different
architectures on different datsets, which demonstrates the superiority of FER
over these label smoothing based approaches.

Noise | Accuracy | STD | LSR | SD | CS-KD | TF-Reg | FER(Ours)
---|---|---|---|---|---|---|---
30% | Last | 57.22$\pm$0.15 | 60.01$\pm$0.41 | 60.10$\pm$0.30 | 53.86$\pm$0.12 | 60.21$\pm$0.15 | 67.94$\pm$0.22 ($\uparrow$ 10.72)
Best | 65.99$\pm$0.10 | 67.38$\pm$0.33 | 60.57$\pm$0.14 | 54.22$\pm$0.36 | 67.72$\pm$0.09 | 71.75$\pm$0.26 ($\uparrow$ 5.76)
60% | Last | 32.22$\pm$0.32 | 32.92$\pm$0.16 | 32.43$\pm$0.40 | 28.68$\pm$0.34 | 33.19$\pm$0.40 | 61.57$\pm$0.08 ($\uparrow$ 29.35)
Best | 53.81$\pm$0.21 | 55.50$\pm$0.35 | 47.70$\pm$0.12 | 40.54$\pm$1.29 | 54.95$\pm$0.37 | 61.99$\pm$0.21 ($\uparrow$ 8.18)
80% | Last | 12.37$\pm$0.35 | 12.08$\pm$0.44 | 16.49$\pm$1.27 | 9.61$\pm$0.05 | 12.51$\pm$0.26 | 42.74$\pm$0.77 ($\uparrow$ 30.37)
Best | 37.40$\pm$0.21 | 36.78$\pm$1.36 | 29.51$\pm$0.35 | 20.31$\pm$0.53 | 38.22$\pm$0.70 | 43.36$\pm$0.63 ($\uparrow$ 5.96)
Table 6: Robustness results with ResNet-18 on CIFAR-100 with different levels of noise. Cross-dataset | STD | LSR | SD | CS-KD | TF-Reg | FER (Ours)
---|---|---|---|---|---|---
CIFAR-100 to STL-10 | 69.71$\pm$0.14 | 68.85$\pm$0.23 | 69.27$\pm$0.13 | 69.96$\pm$0.02 | 69.13$\pm$0.07 | 70.80$\pm$0.15
CIFAR-100 to Tiny ImageNet | 33.47$\pm$0.25 | 32.66$\pm$0.10 | 31.87$\pm$0.05 | 33.46$\pm$0.05 | 33.07$\pm$0.07 | 34.20$\pm$0.12
Table 7: Transferability performances.

#### Fine-Grained Classification

We further evaluate FER on fine-grained classification which is a more
challenging task due to the small intra-class differences but large inter-
class differences. Table 4 reports the comparison results on three fine-
grained classification benchmark datsets. We observe that the superiority of
FER over the baselines is more obvious on the fine-grained datasets than on
regular datasets, e.g., the absolute accuracy improvements over STD with
ResNet-18 are 6.56%, 4.08%, and 2.80% on CUB, Standford Dogs, and FGVC-
Aircraft, respectively. The reason is that the samples in different classes of
the fine-grained classification share more visual similarities, which leads to
more wrong flipping. FER mitigates this issue by maintaining local correct
boundaries and thus obtains a much better performance.

#### Classification on Non-Image Data

To investigate the applicability of FER on non-image data, we conduct several
experiments on two tabular datasets, i.e., Arcene and Iris. We follow (Zhang
et al. 2018) to adopt the DNN with two hidden, fully-connected layers of 128
units. Table 5 reports the comparison results. FER improves the accuracy over
STD by 3.33% and 5.56% on Arcene and Iris, respectively, and also outperforms
the other competitors substantially, which validates the applicability and
usefulness of FER on non-image data.

### Robustness

With DNNs being applied to more and more safety-critical domains such as
autonomous driving and medical diagnosis, the robustness of DNNs to data noise
becomes extraordinarily essential. We follow the existing study (Zhang et al.
2017) to evaluate the robustness of different approaches to different levels
of noise. We use the open-source implementation
444https://github.com/pluskid/fitting-random-labels of (Zhang et al. 2017) to
replace 30%, 60%, and 80% of the training labels in CIFAR-100 by random noise
to generate three different levels of noisy data. Note that all the test
labels are kept intact for evaluation. When the data contain a large amount of
noise, the labels are not reliable anymore and thus cannot be used to judge
whether a sample has been classified correctly. FER thus trains DNNs on noisy
data by simply using Eq. (4).

The robustness performances of different approaches are summarized in Table 6
where the test accuracy in the last epoch and the best test accuracy achieved
during the whole training process are reported. We have the following
observations. First, for most approaches, there is a large gap between the
last accuracy and the best accuracy. The reason is that as the training
progresses, the DNN overfits a large amount of noise in the later epochs,
which results in the accuracy decrease in the later epochs. The existing
studies (Arpit et al. 2017; Han et al. 2018; Deng and Zhang 2021c) have a
similar observation that DNNs first fit training data with clean labels and
then memorize those with noisy labels. Second, FER outperforms STD and the
other label smoothing baselines substantially, e.g., the last accuracy
improvement over STD under 60% noise achieves 29.35%. The reason is that STD
totally believes in the one-hot labels and fits substantial noise in the later
epochs while FER uses the weighted average behavior accumulated from the
previous epochs to guide the training instead of fitting the noise, which
enables it to have a much better robustness.

### Transferability

We investigate the transferability of the representations learned by FER, as
one goal of deep representation learning is to learn general representations
that can be transferred to different datasets. Specifically, we first train
ResNet-18 on CIFAR-100, and then freeze the feature encoder and train a linear
classifier on STL-10 (Coates, Ng, and Lee 2011) and Tiny ImageNet. All images
in STL-10 and Tiny ImageNet are resized to 32$\times$32 to match the input
size of the ResNet-18 pretrained on CIFAR-100.

The transferability performances are reported in Table 7. The representations
learned by FER have a much better cross-dataset performance than those of the
other baselines. The reason can be that FER takes into account the sample-to-
class-margin distance information when learning the representations, thus
avoiding overfitting the dataset and leading to more general representations.

## Conclusion

In this paper, we study a novel problem about “how many misclassified unseen
samples in the last epoch were ever correctly classified by the DNN?” and then
propose a simple yet effective framework FER to mitigate the wrong flipping
issue, which is to our best knowledge the first work to address this problem.
This is achieved by restricting the DNN behavior changes on correctly-
classified samples with the goal of maintaining the correct local
classification boundaries to avoid wrongly flipping on unseen samples.
Extensive experiments on several benchmark datasets with different modern
network architectures demonstrate that FER consistently improves the
generalization, the robustness, and the transferability of DNNs substantially.

## References

  * Arpit et al. (2017) Arpit, D.; Jastrzebski, S. K.; Ballas, N.; Krueger, D.; Bengio, E.; Kanwal, M. S.; Maharaj, T.; Fischer, A.; Courville, A. C.; Bengio, Y.; et al. 2017.  A Closer Look at Memorization in Deep Networks.  In _ICML_.
  * Bagherinezhad et al. (2018) Bagherinezhad, H.; Horton, M.; Rastegari, M.; and Farhadi, A. 2018.  Label refinery: Improving imagenet classification through label progression.  _arXiv preprint arXiv:1805.02641_.
  * Chorowski and Jaitly (2016) Chorowski, J.; and Jaitly, N. 2016.  Towards better decoding and language model integration in sequence to sequence models.  _arXiv preprint arXiv:1612.02695_.
  * Coates, Ng, and Lee (2011) Coates, A.; Ng, A.; and Lee, H. 2011.  An analysis of single-layer networks in unsupervised feature learning.  In _Proceedings of the fourteenth international conference on artificial intelligence and statistics_ , 215–223. JMLR Workshop and Conference Proceedings.
  * Deng and Zhang (2021a) Deng, X.; and Zhang, Z. 2021a.  Comprehensive Knowledge Distillation with Causal Intervention.  _Advances in Neural Information Processing Systems_ , 34.
  * Deng and Zhang (2021b) Deng, X.; and Zhang, Z. 2021b.  Graph-Free Knowledge Distillation for Graph Neural Networks.  In _The 30th International Joint Conference on Artificial Intelligence_.
  * Deng and Zhang (2021c) Deng, X.; and Zhang, Z. 2021c.  Learning with Retrospection.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, 7201–7209.
  * Dua and Graff (2017) Dua, D.; and Graff, C. 2017.  UCI Machine Learning Repository.
  * Furlanello et al. (2018) Furlanello, T.; Lipton, Z. C.; Tschannen, M.; Itti, L.; and Anandkumar, A. 2018\.  Born again neural networks.  _arXiv preprint arXiv:1805.04770_.
  * Han et al. (2018) Han, B.; Yao, Q.; Yu, X.; Niu, G.; Xu, M.; Hu, W.; Tsang, I.; and Sugiyama, M. 2018\.  Co-teaching: Robust training of deep neural networks with extremely noisy labels.  In _Advances in neural information processing systems_ , 8527–8537.
  * He et al. (2016a) He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016a.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 770–778.
  * He et al. (2016b) He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016b.  Identity mappings in deep residual networks.  In _European conference on computer vision_ , 630–645. Springer.
  * Hinton, Vinyals, and Dean (2015) Hinton, G.; Vinyals, O.; and Dean, J. 2015.  Distilling the knowledge in a neural network.  _arXiv preprint arXiv:1503.02531_.
  * Huang et al. (2017) Huang, G.; Li, Y.; Pleiss, G.; Liu, Z.; Hopcroft, J. E.; and Weinberger, K. Q. 2017\.  Snapshot ensembles: Train 1, get m for free.  _arXiv preprint arXiv:1704.00109_.
  * Huang et al. (2019) Huang, Y.; Cheng, Y.; Bapna, A.; Firat, O.; Chen, D.; Chen, M.; Lee, H.; Ngiam, J.; Le, Q. V.; Wu, Y.; et al. 2019.  Gpipe: Efficient training of giant neural networks using pipeline parallelism.  In _Advances in neural information processing systems_ , 103–112.
  * Ji et al. (2021) Ji, M.; Shin, S.; Hwang, S.; Park, G.; and Moon, I.-C. 2021.  Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 10664–10673.
  * Khosla et al. (2011) Khosla, A.; Jayadevaprakash, N.; Yao, B.; and Li, F.-F. 2011.  Novel dataset for fine-grained image categorization: Stanford dogs.  In _Proc. CVPR Workshop on Fine-Grained Visual Categorization (FGVC)_.
  * Krizhevsky and Hinton (2009) Krizhevsky, A.; and Hinton, G. 2009.  Learning multiple layers of features from tiny images.  Technical report, Citeseer.
  * Levinson et al. (2011) Levinson, J.; Askeland, J.; Becker, J.; Dolson, J.; Held, D.; Kammel, S.; Kolter, J. Z.; Langer, D.; Pink, O.; Pratt, V.; et al. 2011.  Towards fully autonomous driving: Systems and algorithms.  In _2011 IEEE Intelligent Vehicles Symposium (IV)_ , 163–168. IEEE.
  * Loshchilov and Hutter (2017) Loshchilov, I.; and Hutter, F. 2017.  Sgdr: Stochastic gradient descent with warm restarts.  _International Conference for Learning Representations_.
  * Ma et al. (2018) Ma, N.; Zhang, X.; Zheng, H.-T.; and Sun, J. 2018.  Shufflenet v2: Practical guidelines for efficient cnn architecture design.  In _Proceedings of the European conference on computer vision (ECCV)_ , 116–131.
  * Maji et al. (2013) Maji, S.; Rahtu, E.; Kannala, J.; Blaschko, M.; and Vedaldi, A. 2013.  Fine-grained visual classification of aircraft.  _arXiv preprint arXiv:1306.5151_.
  * Miotto et al. (2016) Miotto, R.; Li, L.; Kidd, B. A.; and Dudley, J. T. 2016.  Deep patient: an unsupervised representation to predict the future of patients from the electronic health records.  _Scientific reports_ , 6(1): 1–10.
  * Müller, Kornblith, and Hinton (2019) Müller, R.; Kornblith, S.; and Hinton, G. E. 2019.  When does label smoothing help?  In _Advances in Neural Information Processing Systems_ , 4694–4703.
  * Ozbayoglu, Gudelek, and Sezer (2020) Ozbayoglu, A. M.; Gudelek, M. U.; and Sezer, O. B. 2020.  Deep learning for financial applications: A survey.  _Applied Soft Computing_ , 93: 106384.
  * Pereyra et al. (2017) Pereyra, G.; Tucker, G.; Chorowski, J.; Kaiser, Ł.; and Hinton, G. 2017.  Regularizing neural networks by penalizing confident output distributions.  _arXiv preprint arXiv:1701.06548_.
  * Real et al. (2019) Real, E.; Aggarwal, A.; Huang, Y.; and Le, Q. V. 2019.  Regularized evolution for image classifier architecture search.  In _Proceedings of the aaai conference on artificial intelligence_ , volume 33, 4780–4789.
  * Sandler et al. (2018) Sandler, M.; Howard, A.; Zhu, M.; Zhmoginov, A.; and Chen, L.-C. 2018.  Mobilenetv2: Inverted residuals and linear bottlenecks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 4510–4520.
  * Simonyan and Zisserman (2015) Simonyan, K.; and Zisserman, A. 2015.  Very deep convolutional networks for large-scale image recognition.  In _International Conference for Learning Representations_.
  * Szegedy et al. (2016) Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; and Wojna, Z. 2016.  Rethinking the inception architecture for computer vision.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2818–2826.
  * Toneva et al. (2019) Toneva, M.; Sordoni, A.; Combes, R. T. d.; Trischler, A.; Bengio, Y.; and Gordon, G. J. 2019.  An empirical study of example forgetting during deep neural network learning.  In _International Conference for Learning Representations_.
  * Vaswani et al. (2017) Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I. 2017.  Attention is all you need.  In _Advances in neural information processing systems_ , 5998–6008.
  * Wah et al. (2011) Wah, C.; Branson, S.; Welinder, P.; Perona, P.; and Belongie, S. 2011.  The caltech-ucsd birds-200-2011 dataset.  _California Institute of Technology_.
  * Xu and Liu (2019) Xu, T.-B.; and Liu, C.-L. 2019.  Data-distortion guided self-distillation for deep neural networks.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, 5565–5572.
  * Yang et al. (2019a) Yang, C.; Xie, L.; Qiao, S.; and Yuille, A. L. 2019a.  Training deep neural networks in generations: A more tolerant teacher educates better students.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, 5628–5635.
  * Yang et al. (2019b) Yang, C.; Xie, L.; Su, C.; and Yuille, A. L. 2019b.  Snapshot distillation: Teacher-student optimization in one generation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2859–2868.
  * Yuan et al. (2020) Yuan, L.; Tay, F. E.; Li, G.; Wang, T.; and Feng, J. 2020.  Revisiting Knowledge Distillation via Label Smoothing Regularization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 3903–3911.
  * Yun et al. (2020) Yun, S.; Park, J.; Lee, K.; and Shin, J. 2020.  Regularizing class-wise predictions via self-knowledge distillation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 13876–13885.
  * Zagoruyko and Komodakis (2016) Zagoruyko, S.; and Komodakis, N. 2016.  Wide Residual Networks.  In _BMVC_.
  * Zhang et al. (2017) Zhang, C.; Bengio, S.; Hardt, M.; Recht, B.; and Vinyals, O. 2017.  Understanding deep learning requires rethinking generalization.  _International Conference for Learning Representations_.
  * Zhang et al. (2018) Zhang, H.; Cisse, M.; Dauphin, Y. N.; and Lopez-Paz, D. 2018.  mixup: Beyond empirical risk minimization.  _International Conference for Learning Representations_.
  * Zhang et al. (2019) Zhang, L.; Song, J.; Gao, A.; Chen, J.; Bao, C.; and Ma, K. 2019.  Be your own teacher: Improve the performance of convolutional neural networks via self distillation.  In _Proceedings of the IEEE International Conference on Computer Vision_ , 3713–3722.
  * Zoph et al. (2018) Zoph, B.; Vasudevan, V.; Shlens, J.; and Le, Q. V. 2018.  Learning transferable architectures for scalable image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 8697–8710.


# Thermodynamics of Reduced State of the Field

Stefano Cusumano  International Centre for Theory of Quantum Technologies,
University of Gdansk, 80-308 Gdańsk, Poland<EMAIL_ADDRESS>Łukasz
Rudnicki  International Centre for Theory of Quantum Technologies, University
of Gdansk, 80-308 Gdańsk, Poland  Center for Theoretical Physics, Polish
Academy of Sciences, 02-668 Warszawa, Poland

###### Abstract

Recent years have seen the flourishing of research devoted to quantum effects
on mesoscopic and macroscopic scales. In this context, in  _Entropy_ 2019,
_21_ , 705, a formalism aiming at describing macroscopic quantum fields,
dubbed Reduced State of the Field (RSF), was envisaged. While, in the original
work, a proper notion of entropy for macroscopic fields, together with their
dynamical equations, was derived, here, we expand thermodynamic analysis of
the RSF, discussing the notion of heat, solving dynamical equations in various
regimes of interest, and showing the thermodynamic implications of these
solutions.

##  I Introduction

In recent years, a considerable attention has been given to the study of
quantum phenomena on mesoscopic scale, as many physical systems that are
nowadays fundamental for physical applications fall into this regime datta;
imamoglu. The main characteristic of mesoscopic systems is that, while they
are still large enough not to be considered purely quantum, they are neither
small enough to ignore quantum effects.

Furthermore, while the behavior of macroscopic fields is well described by
classical wave equations with coherent sources, incorporation of thermal and
random sources into the field equations still represents an open problem
thorne. As a matter of fact, the most common description of such a situation
relies on the introduction of phenomenological terms, for example, terms
describing the damping. This solution is not fully satisfactory from a
theoretical point of view, as these extra terms do not give a correct
thermodynamic description of such systems. On this basis, and on drawing from
the fact that the ultimate description of any physical system should be given
by quantum mechanics, the Reduced State of the Field (RSF) formalism was
conceived alicki.

Since a completely quantum picture is generally too complex and, consequently,
not convenient to treat macroscopic fields, the RSF aims at describing
macroscopic waves using a coarse-grained version of the quantum formalism.
Such a description allows one to retain the most important quantum features
that would even emanate at macroscopic scale alicki, while, at the same time,
mitigating the complexity that would have no effect beyond the microscopic
realm. Interestingly, in the same spirit, one can answer the question being a
sort of opposite to the former one, namely which features of the quantum
evolution can be classified as classical linowski.

On the other hand, recent years have seen the flourishing of quantum
thermodynamics anders, namely the study of thermodynamic phenomena on the
quantum scale. This interest has been fostered by progressive miniaturization
of electronic and optical devices, at the level where quantum phenomena cannot
be ignored pekola. We, therefore, observe a huge development of the field of
quantum thermodynamics, where a wide range of topics is being covered, e.g.,
thermalization and heat transfer podolski; cusumano; xu; ma, quantum heat
engines and refrigerators linden; ono; kosloff; peterson; watanabe; yao; xiao,
and quantum batteries binder; campaioli; andolina.

The biggest advantage of the RSF formalism is that it provides both a suitable
definition of entropy for radiation fields and dynamical equations describing
the field which are in a closed form (do not depend on other degrees of
freedom). It is, thus, of interest to see how thermodynamics intersects with
the description of mesoscopic and macroscopic fields since, especially on the
mesoscopic scale, one typically does not have full control over the system,
yet quantum effects need to be taken into account in order to describe the
system appropriately rudnicki.

In this paper, we want to explore how thermodynamic phenomena, such as heat
exchange, fit the RSF formalism. Moreover, we want to analyze the behavior of
the entropy of RSF alicki, as its definition differs from the one usually
found in the classical or quantum realms. The paper is organized as follows.
In Section II, we briefly review the RSF formalism, pointing out its main
features. In Section III, starting from the evolution equations of RSF, we
consistently define the main thermodynamic quantities, such as internal
energy, heat, and work. Then, in Section IV, we solve the equations of motion
in some simple but relevant situations, highlighting the thermodynamic meaning
of the different terms present therein. Finally, in Section
LABEL:sec:conclusions, we give our conclusions and some outlooks for future
works.

##  II The RSF Formalism

This section mostly follows Reference alicki, since we summarize here the most
important background and ingredients of the RSF formalism. In particular, all
formulas appearing in this section are taken from Reference alicki.

We start with classical electromagnetic field which, in a finite volume, is
described by a set of modes $f_{k}(x;t)=e^{-i\omega_{k}t}f_{k}(x)$, where $x$
is the position, $k$ is a discrete index, and $\omega_{k}$ is the frequency at
which the mode oscillates. In the first quantization picture, these modes
represent eigenstates of the single-particle Hamiltonian of quasi-particles
associated with the field. Under a proper normalization, these modes form an
orthonormal basis of the single-particle Hilbert space, where the energy of
each mode is equal to $\hbar\omega_{k}$.

In the second quantization picture, a pair of operators
$\hat{a}_{k},\hat{a}_{k}^{\dagger}$ is associated to each mode $f_{k}$.
Standard bosonic commutation relations hold:

$\displaystyle\commutator{\hat{a}_{k}}{\hat{a}_{k^{\prime}}^{\dagger}}=\delta_{kk^{\prime}}\quad\commutator{\hat{a}_{k}}{\hat{a}_{k^{\prime}}}=\commutator{\hat{a}_{k}^{\dagger}}{\hat{a}_{k^{\prime}}^{\dagger}}=0,$
(1)

so that the action of the annihilation and creation operators, on the vectors
in the corresponding Fock space spanned by the orthonormal set
$\\{\ket{n}_{k}\\}$, is

$\displaystyle\hat{a}_{k}\ket{n_{k}}=\sqrt{n_{k}}\ket{n_{k}-1}\quad\hat{a}_{k}^{\dagger}\ket{n_{k}}=\sqrt{n_{k}+1}\ket{n_{k}+1}.$
(2)

The RSF formalism relies on a correspondence between operators acting on the
single-particle Hilbert space and additive operators acting on the Fock space.
The former can be written as:

$\displaystyle\hat{b}=\sum_{k,k^{\prime}}b_{kk^{\prime}}\outerproduct{k}{k^{\prime}},$
(3)

where $\ket{k}\equiv\ket{f_{k}}$, while the corresponding additive observable
in the Fock space is (We follow the convention introduced in Reference alicki,
according to which operators in the Fock space are denoted by capital letters
(with the density operator $\hat{\rho}$ being an exception), while operators
acting on a single-particle Hilbert space are denoted by small letters.):

$\displaystyle\hat{B}=\sum_{k,k^{\prime}}b_{kk^{\prime}}\hat{a}_{k}^{\dagger}\hat{a}_{k^{\prime}}.$
(4)

Consequently, unitary operators $\hat{u}$ acting on the single-particle
Hilbert space are in correspondence with multiplicative operators on the Fock
space via:

$\displaystyle\hat{u}=e^{i\hat{b}}\rightarrow\hat{U}=e^{i\hat{B}}.$ (5)

From now on, we also use “$\Tr$” for trace operations in the Fock space and
“$\tr$” for traces applied to the level of the RSF, i.e., on a single-particle
Hilbert space.

The RSF description of the state of a macroscopic field is based on the couple
$(\hat{r},\ket{\alpha})$, defined from the full quantum state of the field
$\hat{\rho}$ in the Fock space as:

$\displaystyle\hat{r}=\sum_{k,k^{\prime}}\Tr\left[\hat{\rho}\,\hat{a}^{\dagger}_{k^{\prime}}\hat{a}_{k}\right]\outerproduct{k}{k^{\prime}}\coloneqq\sum_{k,k^{\prime}}r_{kk^{\prime}}\outerproduct{k}{k^{\prime}},$
(6a)
$\displaystyle\ket{\alpha}=\sum_{k}\Tr\left[\hat{\rho}\,\hat{a}_{k}\right]\ket{k}\coloneqq\sum_{k}\alpha_{k}\ket{k}.$
(6b)

The matrix $\hat{r}$ is a single-particle density operator, while the vector
$\ket{\alpha}$ contains the information about the phase of the macroscopic
field.

It is important to observe that the single-particle density operator is not
normalized to unity but, rather, to the total number of particles in the
state, i.e.,

$\displaystyle\tr{\hat{r}}=N=\Tr{\hat{\rho}\hat{N}},\quad\hat{N}=\sum_{k}\hat{a}_{k}^{\dagger}\hat{a}_{k}.$
(7)

In fact, the same expectation-value identification holds for any additive
observable

$\displaystyle\tr{\hat{r}\hat{b}}=\Tr{\hat{\rho}\hat{B}}.$ (8)

Furthermore, it turned out beneficial to define an another object, the
correlation matrix

$\displaystyle\hat{r}^{\alpha}=\hat{r}-\outerproduct{\alpha}{\alpha},\qquad\mathrm{where}\qquad\outerproduct{\alpha}{\alpha}=\sum_{k,k^{\prime}}\alpha_{k}\alpha_{k^{\prime}}^{*}\outerproduct{k}{k^{\prime}},$
(9)

which is a positive semi-definite operator being zero if and only if the state
is coherent. Using this operator, it is then possible to give a suitable
definition of entropy for macroscopic fields, which is

$\displaystyle
S[\hat{r}^{\alpha}]=k_{B}\tr\left[(\hat{r}^{\alpha}+1)\ln(\hat{r}^{\alpha}+1)-\hat{r}^{\alpha}\ln\hat{r}^{\alpha}\right].$
(10)

This definition of entropy has an appealing feature of being always greater
than or equal to zero, and being zero only when the RSF is coherent. This also
highlights the fact that the coherent states are the only pure states in this
formalism.

To shortly summarize the above, the RSF formalism is particularly suited to
deal with situation where one does not have full quantum control of the system
(we just control first and second moments, so to speak), as is in the case of
macroscopic fields, but quantum effects are still visible. Having revised the
RSF formalism and its main features, we are now ready to start thermodynamic
considerations.

##  III Thermodynamics of the RSF

In a usual scenario described by thermodynamics, one deals with a system $S$,
often called the working fluid, interacting with one or more thermal baths,
i.e., much larger systems with infinite heat capacity that are typically
assumed to have a well-defined temperature. By changing the Hamiltonian, i.e.,
the energy, of the working fluid $S$ and letting it interact appropriately
with the thermal baths, it is possible to extract work from the system (i.e.,
we have a heat engine) or to use work to transfer heat from a cold to a hot
bath (i.e., we implement a refrigerator).

As in what follows, we will not be interested in a description of the thermal
baths but, rather, in their action on the working fluid $S$. Therefore, we
want to define heat and work only in terms of the state $S$, in the current
context sufficiently well described by the couple $(\hat{r},\ket{\alpha})$. In
order to study the thermodynamics of a macroscopic field described under the
RSF formalism, we first need to recall the dynamical equations describing the
behavior of the field when it interacts with an external bath. This was
already done in Reference alicki, where the system of equation for the RSF was
derived from the standard expression for a map belonging to a so-called quasi-
free dynamical semigroup lendi; alicki2, thus extending this concept to RSF
formalism. The set of equations alicki describing the dynamics of the couple
$(\hat{r},\ket{\alpha})$ can be derived from the equations describing the
temporal evolution of the full state in Fock space $\hat{\rho}$ through:

$\displaystyle\frac{d}{dt}r_{kk^{\prime}}=\Tr{\hat{a}_{k^{\prime}}^{\dagger}\hat{a}_{k}\frac{d\hat{\rho}}{dt}},\qquad\frac{d}{dt}\alpha_{k}=\Tr{\hat{a}_{k}\frac{d\hat{\rho}}{dt}}.$
(11)

Considering a generic model of dynamics for $\hat{\rho}$, given by the
evolution equation alicki

$\displaystyle\begin{split}\frac{d}{dt}\hat{\rho}=&-\frac{i}{\hbar}\big{[}\hat{H},\hat{\rho}\big{]}+\sum_{k=1}^{N}\big{[}\zeta_{k}\hat{a}_{k}^{\dagger}-\zeta_{k}^{*}\hat{a}_{k},\hat{\rho}\big{]}+\sum_{k,k^{\prime}=1}^{N}\Gamma^{k^{\prime}k}_{\downarrow}\left(\hat{a}_{k}\hat{\rho}\,\hat{a}_{k^{\prime}}^{\dagger}-\frac{1}{2}\big{\\{}\hat{a}_{k^{\prime}}^{\dagger}\hat{a}_{k},\hat{\rho}\big{\\}}\right)\\\
&+\sum_{k,k^{\prime}=1}^{N}\Gamma^{k^{\prime}k}_{\uparrow}\left(\hat{a}_{k^{\prime}}^{\dagger}\hat{\rho}\,\hat{a}_{k}-\frac{1}{2}\big{\\{}\hat{a}_{k}\hat{a}_{k^{\prime}}^{\dagger},\hat{\rho}\big{\\}}\right)+\int\mu(du)\left(\hat{U}\hat{\rho}\,\hat{U}^{\dagger}-\hat{\rho}\right),\end{split}$
(12)

which includes the presence of a coherent source, a thermal bath, and random
scattering, one can write the following equations for the couple
$(\hat{r},\ket{\alpha})$ (Note that the anticommutator terms, in comparison
with Reference alicki, have been divided by $2$. See Reference linowski for
details.):

$\displaystyle\frac{d}{dt}\hat{r}$
$\displaystyle=-\frac{i}{\hbar}[\hat{h},\hat{r}]+(\outerproduct{\zeta}{\alpha}+\outerproduct{\alpha}{\zeta})+\frac{1}{2}\left\\{(\hat{\gamma}_{\uparrow}-\hat{\gamma}_{\downarrow}),\hat{r}\right\\}+\hat{\gamma}_{\uparrow}$
$\displaystyle+\int\mu(du)(\hat{u}\hat{r}\hat{u}^{\dagger}-\hat{r}),$ (13a)
$\displaystyle\frac{d}{dt}\ket{\alpha}=-\frac{i}{\hbar}\hat{h}\ket{\alpha}+\ket{\zeta}+\frac{1}{2}(\hat{\gamma}_{\uparrow}-\hat{\gamma}_{\downarrow})\ket{\alpha}+\int\mu(du)(\hat{u}-1)\ket{\alpha}.$
(13b)

Let us start by explaining the meaning of each term in (12) viz. Equations
(13) and (13b). In the dynamical equation for $\hat{r}$, we first find the
commutator of $\hat{r}$ with the single-particle Hamiltonian
$\hat{h}=\hbar\sum_{k}\omega_{k}\outerproduct{k}{k}$ stemming from
$\hat{H}=\hbar\sum_{k}\omega_{k}\hat{a}_{k}^{\dagger}\hat{a}_{k}$, and this
term describes nothing but the standard unitary dynamics induced by the free
Hamiltonian. Next, we find the term $\outerproduct{\zeta}{\alpha}$, which
describes the effect of a coherent source, and, thus, also depends on the
phase of the system $\ket{\alpha}$. Then, we can see the anticommutator term
with the operators

$\displaystyle\hat{\gamma}_{\updownarrow}=\sum_{k,k^{\prime}}\Gamma^{kk^{\prime}}_{\updownarrow}\outerproduct{k}{k^{\prime}},$
(14)

describing stimulated absorption and emission processes, while the isolated
term $\hat{\gamma}_{\uparrow}$ describes spontaneous emission processes. The
coefficients $\Gamma_{\updownarrow}^{kk^{\prime}}$ encode the information
about the state of the thermal bath and its interaction with the system.
Finally, the integral term describes the effect of random scattering
phenomena, where the operators $\hat{u}$ are unitary. Similar considerations
apply to the dynamical equation for $\ket{\alpha}$. Note also that, although
the usual single particle approach is one where recursive systems of equations
are truncated through appropriate approximations or boundary conditions, in
the RSF approach, one deals with a closed system of equation, a feature that
greatly simplifies the study of the dynamics of a macroscopic field.

As the entropy is defined in terms of the correlation matrix
$\hat{r}^{(\alpha)}$, it is also useful to derive the dynamical equation for
this quantity. Since
$\hat{r}^{(\alpha)}=\hat{r}-\outerproduct{\alpha}{\alpha}$, we only need to
compute the time derivative of $\outerproduct{\alpha}{\alpha}$ using Equation
(13b):

$\displaystyle\frac{d}{dt}\outerproduct{\alpha}{\alpha}=$
$\displaystyle\left(\frac{d}{dt}\ket{\alpha}\right)\bra{\alpha}+\ket{\alpha}\left(\frac{d}{dt}\bra{\alpha}\right)$
$\displaystyle=$
$\displaystyle-\frac{i}{\hbar}\commutator{\hat{h}}{\outerproduct{\alpha}{\alpha}}+(\outerproduct{\alpha}{\zeta}+\outerproduct{\zeta}{\alpha})+\frac{1}{2}\anticommutator{(\hat{\gamma}_{\uparrow}-\hat{\gamma}_{\downarrow})}{\outerproduct{\alpha}{\alpha}}$
$\displaystyle+$
$\displaystyle\int\mu(du)(\hat{u}\outerproduct{\alpha}{\alpha}+\outerproduct{\alpha}{\alpha}\hat{u}^{\dagger})-2\outerproduct{\alpha}{\alpha}),$
(15)

from which we can write the dynamical evolution for the correlation matrix
$\hat{r}^{(\alpha)}$ as

$\displaystyle\frac{d}{dt}\hat{r}^{(\alpha)}=-\frac{i}{\hbar}[\hat{h},\hat{r}^{(\alpha)}]+\frac{1}{2}\left\\{(\hat{\gamma}_{\uparrow}-\hat{\gamma}_{\downarrow}),\hat{r}^{(\alpha)}\right\\}+\hat{\gamma}_{\uparrow}$
$\displaystyle\qquad\quad+\int\mu(du)\left(\hat{u}\hat{r}^{(\alpha)}\hat{u}^{\dagger}-\hat{r}^{(\alpha)}\right)+\int\mu(du)(\hat{u}-1)\outerproduct{\alpha}{\alpha}(\hat{u}^{\dagger}-1).$
(16)

From this equation, we can see that the dynamics of the correlation matrix are
not influenced by the presence of coherent sources. Consequently, the entropy
$S[\hat{r}^{(\alpha)}]$ is also invariant with respect to coherent evolution.
This feature of the theory is associated with the fact that we are dealing
with a mesoscopic or macroscopic system, where, in fact, we do not have access
to all degrees of freedom alicki. In particular, the single-particle
Hamiltonian $\hat{h}$ does not carry the whole content of the Hamiltonian in
the Fock space which also contains contributions due to the displacement. In
view of this, we define the internal energy as

$\displaystyle
U=\tr[\hat{h}\hat{r}^{(\alpha)}]\equiv\tr[\hat{h}\hat{r}]-\bra{\alpha}\hat{h}\ket{\alpha}.$
(17)

This definition is motivated by the form of the entropy in Equation (10) and
from the related discussion in Reference alicki: as the definition of entropy
relies on the effective degree of control that one has over the physical
system under examination, the same should apply to other quantities of
interest. Since, in the RSF formalism, the entropy is invariant under the
application of the Weyl displacement operator, one could expect the internal
energy to follow the same behavior. In particular, if, for instance, we were
to define the internal energy in the “intuitive” way as $\tr[\hat{h}\hat{r}]$,
then displacement would be a process implying heat absorption from the system,
with no change of entropy. In Section IV, we are going to show that this issue
is resolved by Equation (17), and that, thanks to this definition, we are able
to define properly the free energy of the system. Last but not least, let us
emphasize that the internal energy of the system is a notion which depends on
an arbitrary choice in which degrees of freedom describe the system and which
belong to its environment.

Using the notion of internal energy in Equation (17), one has a natural
decomposition

$\displaystyle
dU=\tr[\frac{d\hat{h}}{dt}\hat{r}^{(\alpha)}]dt+\tr[\hat{h}\frac{d\hat{r}^{(\alpha)}}{dt}]dt=dW+\delta
Q.$ (18)

Two observations are in place here. First of all, the single particle
Hamiltonian is time independent by construction. This is because the
frequencies, as well as the eigenmode basis, of the Hamiltonian, are not under
control and do not vary over time due to the dynamics of the sole field.
Therefore, for generic macroscopic fields, there is no work, just the heat.
Work would require an engineered variant of time evolution, i.e., one can
perform (extract) work on (from) the system only by changing the frequencies
$\omega_{k}$.

Second of all, only the scattering term couples $\hat{r}^{(\alpha)}$ with
$\ket{\alpha}$ in Equation 16. This feature in a salient way distinguishes the
scattering processes from the other processes subsumed in the dynamical
equations. Within a thermodynamic description, which is solely based here on
the correlation matrix, the scattering belongs to a different (more complex)
class of (likely non-equilibrium) processes. The latter property, however,
would strongly depend on the measure $\mu(du)$ chosen. Perhaps, for the
invariant Haar measure, the situation would simplify, still, the
aforementioned coupling will be there.

Therefore, we believe that the scattering processes deserve a separate and
detailed treatment. Consequently, here, we shall neglect random scattering
terms, with the goal of delineating the heat exchange and entropy production
due to other processes. Under this simplifying assumption, the heat exchanged
is equal to

$\displaystyle\delta
Q=\tr[\hat{h}\frac{d\hat{r}^{(\alpha)}}{dt}]dt=\frac{1}{2}\tr[\hat{r}^{(\alpha)}\anticommutator{\hat{\gamma}_{\uparrow}-\hat{\gamma}_{\downarrow}}{\hat{h}}]dt+\tr[\hat{h}\hat{\gamma}_{\uparrow}]dt$
$\displaystyle\qquad=\hbar\sum_{k,k^{\prime}}\frac{\omega_{k}+\omega_{k^{\prime}}}{2}r_{kk^{\prime}}^{(\alpha)}\left(\Gamma_{\uparrow}^{k^{\prime}k}-\Gamma_{\downarrow}^{k^{\prime}k}\right)dt+\hbar\omega_{k}\Gamma_{\uparrow}^{kk}dt,$
(19)

that is, it only depends on interactions with the thermal bath. In particular,
the second term on the right-hand side of Equation (19) is responsible for the
equilibration process towards the equilibrium populations dictated by the bath
structure, while the first term describes heat exchanges due to changes in the
modes’ occupations happening because of the interaction with the bath.

The variation of the entropy in time is also found to be

$\displaystyle\frac{d}{dt}S[\hat{r}^{(\alpha)}]=k_{B}\tr[\frac{d\hat{r}^{(\alpha)}}{dt}\ln(\frac{\hat{r}^{(\alpha)}+1}{\hat{r}^{(\alpha)}})].$
(20)

We use the notation in which the fraction of non-negative operators needs to
be understood in terms of their eigenvalues. This is possible because,
whenever some eigenvalue approaches $0$, the time derivative also vanishes,
killing the potential singularities derivative.

Note that the trace of $\hat{r}^{(\alpha)}$ does not need to be constant in
time. For a quasi-static process, in which the state $\hat{\rho}$ is always in
thermal equilibrium, the correlation matrix is always of the form

$\displaystyle\hat{r}^{(\alpha)}=\frac{1}{e^{\beta\hat{h}}-1}.$ (21)

Since, in this case,

$\displaystyle\ln(\frac{\hat{r}^{(\alpha)}+1}{\hat{r}^{(\alpha)}})=\beta\hat{h},$
(22)

we recover the equality from standard thermodynamics

$\displaystyle dS=k_{B}\beta\delta Q.$ (23)

This observation further strengthens our definition of work and heat.
Moreover, for a non-quasi-static process, one has that $\hat{r}^{(\alpha)}$ is
not of the form in Equation (22); thus, one has also entropy production.

##  IV Some Examples of RSF Thermodynamics

In the following subsections, we want to solve the dynamical Equations (13)
and (13b) under various circumstances where some of the terms are absent or
can be simplified, thus highlighting their thermodynamic meaning.

###  IV.1 Free Dynamics of the RSF

The simplest, and almost trivial, case that one can analyze is the one where
no interaction with either a coherent source or a thermal bath is present, so
that the dynamics of the RSF is fully described by the Hamiltonian term alone.
Assuming the Hamiltonian in the Fock space to be:

$\displaystyle\hat{H}=\sum_{k}\hbar\omega_{k}\hat{a}_{k}^{\dagger}\hat{a}_{k}\rightarrow\hat{h}=\sum_{k}\hbar\omega_{k}\outerproduct{k}{k},$
(24)

we can explicitly write down the equations governing the matrix elements
$r_{kk^{\prime}}(t)$ and the vector components $\alpha_{k}(t)$ as:

$\displaystyle\frac{d}{dt}r_{kk^{\prime}}(t)=-i(\omega_{k}-\omega_{k^{\prime}})r_{kk^{\prime}}(t),\quad\frac{d}{dt}\alpha_{k}(t)=-i\omega_{k}\alpha_{k}(t).$
(25)

The solutions to these equations are easily found:

$\displaystyle r_{kk}(t)=r_{kk}(0);\quad
r_{kk^{\prime}}=e^{-i(\omega_{k}-\omega_{k^{\prime}})t}r_{kk^{\prime}}(0);\quad\alpha_{k}(t)=e^{-i\omega_{k}t}\alpha_{k}(0).$
(26)

These solutions imply that, under purely free dynamics, the populations stay
constant, while the coherences among them rotate at a frequency equal to the
detuning between the modes. Finally, the components of the phase vector
$\ket{\alpha}$ rotate at the corresponding frequency. In accordance with
Equation (19), there is no heat exchange, as there is no thermal bath. An
important fact to be noted is that, as in Equation (16), the correlation
matrix depends on the Hamiltonian $\hat{h}$ only through the commutator term,
and the entropy is unchanged under purely Hamiltonian dynamics, since the
eigenvalues of $\hat{r}^{(\alpha)}$ are left unchanged.

###  IV.2 RSF Dynamics in Presence of a Coherent Source

We now want to solve Equations (13) and (13b) subject to a coherent source,
but still without a thermal bath, so that we get:

$\displaystyle\frac{d}{dt}\hat{r}=-\frac{i}{\hbar}[\hat{h},\hat{r}]+(\outerproduct{\alpha}{\zeta}+\outerproduct{\zeta}{\alpha}),$
(27)
$\displaystyle\frac{d}{dt}\ket{\alpha}=-\frac{i}{\hbar}\hat{h}\ket{\alpha}+\ket{\zeta},$
(28)

where $\ket{\zeta}=\sum_{k}\zeta_{k}\ket{k}$. We can easily get the dynamical
equations for the matrix elements:

$\displaystyle\frac{d}{dt}r_{kk^{\prime}}=-i(\omega_{k}-\omega_{k^{\prime}})r_{kk^{\prime}}+\left(\alpha_{k}\zeta_{k^{\prime}}^{*}+\alpha_{k^{\prime}}^{*}\zeta_{k}\right),$
(29) $\displaystyle\frac{d}{dt}\alpha_{k}=-i\omega_{k}\alpha_{k}+\zeta_{k}.$
(30)

Solving the second equation first, we get

$\displaystyle\alpha_{k}(t)=e^{-i\omega_{k}t}\alpha_{k}(0)-i\frac{\zeta_{k}}{\omega_{k}}(1-e^{-i\omega_{k}t}),$
(31)

so that the $\hat{r}$ matrix elements are

$\displaystyle
r_{kk^{\prime}}(t)=e^{-i(\omega_{k}-\omega_{k^{\prime}})t}\left[r_{kk^{\prime}}(0)+\int_{0}^{t}ds\,e^{i(\omega_{k}-\omega_{k^{\prime}})s}\big{(}\alpha_{k}(s)\zeta_{k^{\prime}}^{*}+\alpha_{k^{\prime}}^{*}(s)\zeta_{k}\big{)}\right].$
(32)

After we perform the integral, we get

$\displaystyle r_{kk^{\prime}}(t)$ $\displaystyle=$ $\displaystyle
e^{-i(\omega_{k}-\omega_{k^{\prime}})t}\left(r_{kk^{\prime}}(0)+\frac{\zeta_{k}\zeta_{k^{\prime}}}{\omega_{k}\omega_{k^{\prime}}}\right)+\frac{\zeta_{k}\zeta_{k^{\prime}}^{*}}{\omega_{k}\omega_{k^{\prime}}}\left(1-e^{-i\omega_{k}t}-e^{i\omega_{k^{\prime}}t}\right)$
(33) $\displaystyle+$ $\displaystyle
i\left(\frac{\alpha_{k}(0)\zeta_{k^{\prime}}^{*}}{\omega_{k^{\prime}}}e^{-i\omega_{k}t}-\frac{\alpha_{k^{\prime}}^{*}(0)\zeta_{k}}{\omega_{k}}e^{i\omega_{k^{\prime}}t}\right).$

Let us consider the case where the initial phase vector $\ket{\alpha}$ is
null, i.e., $\alpha_{k}(0)=0$, for all $k$. In this case, the solution for the
phase and the matrix elements of $\hat{r}$ reads:

$\displaystyle\alpha_{k}(t)=-i\frac{\zeta_{k}}{\omega_{k}}(1-e^{-i\omega_{k}t}),$
(34) $\displaystyle
r_{kk^{\prime}}(t)=e^{-i(\omega_{k}-\omega_{k^{\prime}})t}\left(r_{kk^{\prime}}(0)+\frac{\zeta_{k}\zeta_{k^{\prime}}^{*}}{\omega_{k}\omega_{k^{\prime}}}\right)+\frac{\zeta_{k}\zeta_{k^{\prime}}^{*}}{\omega_{k}\omega_{k^{\prime}}}\left(1-e^{-i\omega_{k}t}-e^{i\omega_{k^{\prime}}t}\right).$
(35)

The latter result, for the diagonal elements $r_{kk}(t)$, reduces to

$\displaystyle
r_{kk}(t)=r_{kk}(0)+2\frac{|\zeta_{k}|^{2}}{\omega_{k}^{2}}\left(1-\cos{\omega_{k}t}\right).$
(36)

This implies that the populations oscillate around the average values
$r_{kk}(0)+|\zeta_{k}|^{2}/\omega_{k}^{2}$. Of course, the correlation matrix
remains constant (also if initial $\ket{\alpha}$ is not null), so does the
entropy.

###  IV.3 Dynamics of the RSF in Presence of a Coherent Source and a Thermal
Bath

Let us now consider the case where also a dissipation term is present, i.e.,
we want to analyze the case where the system interacts with both a coherent
source and a heat bath. In this case, the dynamical equations for $\hat{r}$
and $\ket{\alpha}$ are:

$\displaystyle\frac{d\hat{r}}{dt}=-\frac{i}{\hbar}[\hat{h},\hat{r}]+(\outerproduct{\alpha}{\zeta}+\outerproduct{\zeta}{\alpha})+\frac{1}{2}\\{(\hat{\gamma}_{\uparrow}-\hat{\gamma}_{\downarrow}),\hat{r}\\}+\hat{\gamma}_{\uparrow},$
(37)
$\displaystyle\frac{d}{dt}\ket{\alpha}=-\frac{i}{\hbar}\hat{h}\ket{\alpha}+\ket{\zeta}+\frac{1}{2}(\hat{\gamma}_{\uparrow}-\hat{\gamma}_{\downarrow})\ket{\alpha},$
(38)

where the operators $\hat{\gamma}_{\updownarrow}$ have already been defined as

$\displaystyle\hat{\gamma}_{\updownarrow}=\sum_{k,k^{\prime}}\Gamma^{kk^{\prime}}_{\updownarrow}\outerproduct{k}{k^{\prime}}.$
(39)

Let us remind that the matrix elements $\Gamma_{\updownarrow}^{kk^{\prime}}$
are the particle creation and decay rates that can be derived using the Fermi
golden rule. Under the typical Born, Markov, and secular approximations, the
operators $\hat{\gamma}_{\updownarrow}$ become diagonal

$\displaystyle\hat{\gamma}_{\updownarrow}=\sum_{k}\Gamma_{\updownarrow}^{k}\outerproduct{k}{k},$
(40)

where the rates $\Gamma_{\updownarrow}^{k}$, due to the thermal character of
the bath, are related via

$\displaystyle\frac{\Gamma_{\uparrow}^{k}}{\Gamma_{\downarrow}^{k}}=e^{-\frac{\hbar\omega_{k}}{k_{B}T}},$
(41)

with $k_{B}$ being the Boltzmann constant, and $T$ being the temperature of
the heat bath.

In this case, the dynamical equations for the RSF become:

$\displaystyle\frac{dr_{kk^{\prime}}}{dt}=-i(\omega_{\kappa}-\omega_{k^{\prime}})r_{kk^{\prime}}-\frac{1}{2}\left(\frac{\Gamma_{\downarrow}^{k}}{Z_{k}}+\frac{\Gamma_{\downarrow}^{k^{\prime}}}{Z_{k^{\prime}}}\right)r_{kk^{\prime}}+\delta_{kk^{\prime}}\Gamma_{\uparrow}^{k}+\left(\alpha_{k}\zeta_{k^{\prime}}^{*}+\alpha_{k^{\prime}}^{*}\zeta_{k}\right),$
(42) $\displaystyle\hskip
100.0pt\frac{d\alpha_{k}}{dt}=-i\omega_{k}\alpha_{k}-\frac{\Gamma_{\downarrow}^{k}}{2Z_{k}}\alpha_{k}+\zeta_{k},$
(43)

where we have defined $Z_{k}=\left(1-e^{-\beta\hbar\omega_{k}}\right)^{-1}$.
These equations are of the same form as Equations (29) and (30). This can be
noted by defining complex frequencies
$\tilde{\omega}_{k}=\omega_{k}-i\Gamma_{\downarrow}^{k}/2Z_{k}$. In this
notation, we get:

$\displaystyle\frac{dr_{kk^{\prime}}}{dt}=-i(\tilde{\omega}_{k}-\tilde{\omega}_{k^{\prime}}^{*})r_{kk^{\prime}}+\delta_{kk^{\prime}}\Gamma_{\uparrow}^{k}+(\alpha_{k}\zeta_{k^{\prime}}^{*}+\alpha_{k^{\prime}}^{*}\zeta_{k}),$
(44) $\displaystyle\hskip
60.0pt\frac{d\alpha_{k}}{dt}=-i\tilde{\omega}_{k}\alpha_{k}+\zeta_{k},$ (45)

so that one can immediately write down the solution to the second equation as:

$\displaystyle\alpha_{k}(t)=e^{-i\tilde{\omega}_{k}t}\alpha_{k}(0)-i\frac{\zeta_{k}}{\tilde{\omega}_{k}}\left(1-e^{-i\tilde{\omega}_{k}t}\right),$
(46)

which implies that the phases $\alpha_{k}$ are driven towards their steady-
state values

$\displaystyle\alpha_{k}^{\textnormal{steady}}=-i\frac{\zeta_{k}}{\tilde{\omega}_{k}}.$
(47)

As for the matrix elements $r_{kk^{\prime}}$, one finds:

$\displaystyle
r_{kk^{\prime}}(t)=e^{-i(\tilde{\omega}_{k}-\tilde{\omega}_{k^{\prime}}^{*})t}\left[r_{kk^{\prime}}(0)+\int_{0}^{t}ds\,e^{i(\tilde{\omega}_{k}-\tilde{\omega}_{k^{\prime}}^{*})s}\left(\alpha_{k}(s)\zeta_{k^{\prime}}^{*}+\alpha_{k^{\prime}}^{*}(s)\zeta_{k}+\delta_{kk^{\prime}}\Gamma_{\uparrow}^{k}\right)\right],$

and, consequently,

$\displaystyle r_{kk^{\prime}}(t)$ $\displaystyle=$ $\displaystyle
e^{-i(\tilde{\omega}_{k}-\tilde{\omega}_{k^{\prime}}^{*})t}\left(r_{kk^{\prime}}(0)+\frac{\zeta_{k}\zeta_{k^{\prime}}}{\tilde{\omega}_{k}\tilde{\omega}_{k^{\prime}}^{*}}\right)+\frac{\zeta_{k}\zeta_{k^{\prime}}^{*}}{\tilde{\omega}_{k}\tilde{\omega}_{k^{\prime}}^{*}}\left(1-e^{-i\tilde{\omega}_{k}t}-e^{i\tilde{\omega}_{k^{\prime}}^{*}t}\right)$
(48) $\displaystyle+$ $\displaystyle
i\left(\frac{\alpha_{k}(0)\zeta_{k^{\prime}}^{*}}{\tilde{\omega}_{k^{\prime}}^{*}}e^{-i\tilde{\omega}_{k}t}-\frac{\alpha_{k^{\prime}}^{*}(0)\zeta_{k}}{\tilde{\omega}_{k}}e^{i\tilde{\omega}_{k^{\prime}}^{*}t}\right)+\delta_{kk^{\prime}}e^{-\beta\hbar\omega_{k}}Z_{k}\left(1-e^{-\frac{\Gamma_{\downarrow}^{k}}{Z_{k}}t}\right).$

It is of particular interest to see the steady values of the matrix elements
$r_{kk^{\prime}}$,

$\displaystyle
r_{kk^{\prime}}^{\textnormal{steady}}=\frac{\zeta_{k}\zeta_{k^{\prime}}^{*}}{\tilde{\omega}_{k}\tilde{\omega}_{k^{\prime}}^{*}}+\delta_{kk^{\prime}}e^{-\beta\hbar\omega_{k}}Z_{k}.$
(49)

From this steady-state solution, together with Equation (47), we can compute
the associated correlation matrix $\hat{r}^{(\alpha)}$, for which one simply
obtains:

$\displaystyle
r_{kk^{\prime}}^{(\alpha)\textnormal{steady}}=r_{kk^{\prime}}^{\textnormal{steady}}-\outerproduct{\alpha^{\textnormal{steady}}}{\alpha^{\textnormal{steady}}}_{kk^{\prime}}=\delta_{kk^{\prime}}e^{-\beta\hbar\omega_{k}}Z_{k}=\frac{1}{e^{\beta\hat{h}}-1}.$
(50)

From this result, one can see clearly what was already noted in Reference
alicki, namely that, in presence of random scattering (which is absent in this
case) or a thermal environment with temperature different from zero, it is
impossible to obtain a coherent state, and that only an initial pure state
remains pure when the above conditions are met.

Next, we express the entropy of the steady state as a function of $\beta$ (we
set $k_{B}=1$):

$\displaystyle S[\hat{r}^{(\alpha)\textnormal{steady}}](\beta)$
$\displaystyle=$
$\displaystyle\tr[(\hat{r}^{(\alpha)\textnormal{steady}}+1)\ln(\hat{r}^{(\alpha)\textnormal{steady}}+1)-\hat{r}^{(\alpha)\textnormal{steady}}\ln\hat{r}^{(\alpha)\textnormal{steady}}]$
(51) $\displaystyle=$
$\displaystyle\tr[\beta\hat{h}\hat{r}^{(\alpha)\textnormal{steady}}]+\tr[\ln(\hat{r}^{(\alpha)\textnormal{steady}}+1)]$
$\displaystyle=$ $\displaystyle\beta
U+\tr[\ln(\hat{r}^{(\alpha)\textnormal{steady}}+1)],$

as it can be found using Equation (50) and going through some algebra. One can
immediately see that the entropy depends on the temperature, both through the
partition functions and the occupation numbers of the modes. We plot in Figure
1 the entropy as a function of the temperature $\beta$ for different values of
the frequency. From the plot, it can be seen that lower frequency modes have a
greater entropy than the modes with higher frequency.

Figure 1: In this plot, the entropy as a function of temperature $\beta$ is
shown. The various lines are plotted using different frequency, so that one
can see that the low frequency modes contribute more to the entropy,
especially at low temperatures.

The Equation (51) can also be rearranged as:

$\displaystyle
U-\beta^{-1}S=-\frac{1}{\beta}\tr[\ln(\hat{r}^{(\alpha)\textnormal{steady}}+1)],$
(52)

so that, in this way, we are driven to define the equilibrium-free energy
$F_{eq}$

$\displaystyle
F_{eq}=-\frac{1}{\beta}\tr[\ln(\hat{r}^{(\alpha)\textnormal{steady}}+1)]=-\frac{1}{\beta}\sum_{k}\ln
Z_{k}.$ (53)

This is exactly the sum of the equilibrium-free energies of each mode. We can
then define the free energy as:

$\displaystyle F$ $\displaystyle=$