# Information form of the second law of thermodynamics

Miroslav Holeček<EMAIL_ADDRESS>New Technologies Research Center,
University of West Bohemia, Plzeň 301 00, Czech Republic

###### Abstract

An essential role of information in microscopic thermodynamics (e.g. Maxwell’s
demon) opens a challenging question if there exists a formulation of the
second law of thermodynamics based only on pure information ideas. Here, such
a formulation is suggested for unitary processes by introducing information as
a full-valuable physical quantity defining an (objective) microscopic
information entropy as ’information about microstate’. We show that various
forms of entropy (Boltzmann, Shannon, Clausius) are in fact only a special
cases of information entropy whose general form is found out. An observer
plays here the role of a special (information) reference frame (IRF) towards
which the entropy is defined. Some paradoxes or misunderstandings connected
with the concept of entropy or the content of the second law arise by
describing a situation without specifying a concrete IRF. Typically, the
Boltzmann statistical approach cannot be symmetrically used towards the past
as towards the future in one IRF. The information second law is full usable at
meso- or microscopic scales: the information form of the generalized second
law is found out too.

††preprint: APS/123-QED

##  I Introduction

As noticed by J.C. Maxwell in 1871, if a being (”demon”) has _information_
about individual molecules of a system and is able to manipulate with them,
the second law of thermodynamics can be violated Maxwell1871 . The patch to
this ”hole in law” has been found in the last decades: demon’s information
about molecules has to be taken as a full-valued physical component of the
description of the whole situation MarNorVed ; SagUed2012 ; ParHorSag ;
DefJar2013 ; BarSei2014b ; ShiMatSag2016 . This idea has been demonstrated and
verified in various experiments at meso- and nanoscales Toy2010 ; Mih2016 ;
Cot2017 , it has surprising applications in molecular biology BaSei2013 ;
HorSagPar2013  and nanoscale technologies Seifert2012 ; TanShe2008 . A general
physical theory of information addressing the basic problems of statistical
physics (macroscopic time-asymmetry, objective microscopic definition of
entropy) ParHorSag , however, does not exist yet.

In such a theory, information should be a basic concept expressing someone’s
knowledge about a physical system mutual . To bring this idea closer, consider
a system observed by several observers so that each of them has a different
knowledge about it. The perfect observer Laplace1812  knows its microscopic
state, $x$, others know less. The knowledge of individual observers can be
quantified by using the ingenious idea by C.E. Shannon Shannon1948 . Imagine
that the observers receive the message fully describing $x$. The _value of
information_ included in this message is different for each of them (for the
perfect observer the message is worthless). Denote as ${\cal I}_{\cal O}(x)$
this value for an observer ${\cal O}$. We can say that this value quantifies
observers’ ignorance (lack of information) as to $x$ before receiving the
message.

If ${\cal O}={\cal O}(M)$ is a typical macroscopic observer then $S={\cal
I}_{{\cal O}(M)}(x)$ is the definition of entropy by E.T. Jaynes Jaynes1957 .
Entropy is then a function of microscopic state $x$. Its evident dependence on
a ”typical macroscopic observer” is, however, incompatible with objective
physics. But notice that the observer in the definition of ${\cal I}_{\cal
O}(x)$ plays only the role of some informational _reference_ with respect to
which information in the message is assessed. The fact that the value of a
quantity is defined only with respect to another value of this quantity is not
unfamiliar in physics.

To determine the position of a body, for example, we must do it in reference
to positions of other bodies. The body has undoubtedly an objective location
in space, its concrete meaning, however, is given in a concrete reference
frame. The concept of microscopic information entropy, $S={\cal I}_{{\cal
O}(M)}(x)$, is analogical. It can be understood as an abstract objective
quantity expressing information connected with a given microscopic state $x$.
Its concrete value and meaning, however, must be specified in a concrete
_information reference frame_ (IRF) represented, for example, by a macroscopic
observer ${\cal O}(M)$.

We do not observe processes at the very microscopic level and information
about them is usualy given indirectly by observing other (usualy some coarse-
grained) quantities, which is the original idea by L. Botzmann Boltzmann1896 .
Information gained by observing a system at an arbitrary spatial scale is what
define a concrete IRF. It implies that the role of a macroscopic observer is
not exclusive. We can have arbitrary information reference frames defined with
respect to our experimental abilities or our intentions to model a concrete
system. Information entropy - an objective quantity - has different values at
these reference frames. A full microscopic description of a system (e.g. as a
pure quantum state) is also a special (microscopic) information frame.

In this contribution, we use the idea of information reference frames to
derive a novel formulation of the second law of thermodynamics which
introduces information as a fundamental physical quantity without a need to
define a particular observer and to introduce the concept of probabilities.
The law is formulated for single deterministic and time-symmetric microscopic
trajectories that play the role of adiabatic processes at arbitrary spatial
scales.

The finding of an explicit relation between information and entropy allows us
to derive a general form of the entropy dependence on the microscopic state of
the system in a concrete IRF,
$S_{\Gamma}(x)=k_{B}(\ln|\Gamma|_{\Psi}+\Psi_{\Gamma}(x))$, where $\Gamma$ is
a subset of the state space determined by a chosen IRF, $|\Gamma|_{\Psi}$ is a
special measure, and $\Psi$ is a real function whose concrete choice (the so-
called $\Psi$-representation) defines a concrete form of the entropy. A
special choice of $\Psi$ defines probabilities, $p$, and the Shannon entropy,
$S=-k_{B}\ln p$.

The information form of the second law is in fact an information
generalization of Boltzmann’s ingenious ideas Boltzmann1896 ; Lebowitz1993 ;
GLTZ2019 . The information approach, however, avoids the problematic point of
Boltzmann’s statistical program: namely the fact that his statistical method
gives peculiar results when applied towards the past. We argue that the way of
thinking towards the past is not symmetric from information point of view: it
must be done in another information reference frame. The paradox then
disappears. The use of our approach into thermodynamics defined at microscopic
or mesoscopic scales is straightforward. We can connect the change of entropy
with the dissipation work. The generalized second law is presented here in a
pure information form.

The paper is organized as follows. The concept of value of information and the
information reference frame are introduced and the information entropy is
defined in Section II. The general definition of adiabatic processes is
formulated and the information formula described the change of information
entropy in these processes is found out in Section III. In Section IV, the
concrete form of information entropy (i.e. its dependence on the microstate of
the system and the used information reference frame) is derived. It
generalizes the Boltzmann entropy. The information form of the second law of
thermodynamics is discussed in Section V. Special forms of the information
entropy (the so-called $\Psi$-representations) are found out in Section VI:
the Shannon entropy and Clausius entropy of classical equilibrium
thermodynamics. In Section VII, the concept of probability arising in a
special $\Psi$-representation of the information entropy is studied,
especially its referential meaning given by a special choice of the
information reference frame. We show in Section VIII that some concepts of
nonequilibrium statistical physics and the generalized second law of
thermodynamics are special applications of the information second law.

##  II Information reference frame and entropy

Information about a system ${\bf S}$ can be identified with a subset $\Omega$
of its microscopic state space $\Gamma_{\bf S}$. Namely the set $\Omega$ can
be interpreted as including all microscopic states that are consistent with
attainable information about the system observed (studied) by a concrete
observer. A typical example is Boltzmann’s macrostate that includes all
microstates that look the same for a macroscopic observer. A subset of the
microscopic state space $\Omega\subset\Gamma_{\bf S}$ may represent, however,
arbitrary information about the system gained at micro-, meso- or macroscopic
scales.

If we want to quantify information we need some _referential information_ that
plays the role of an information unit. Consider that an observer knows that
the actual microstate belongs into a set $\Gamma$. The set $\Gamma$ plays the
role of referential information. The value of information that the actual
microstate, $x$, belongs into a set $\Omega\subset\Gamma$ can be defined with
respect to referential information ”$x\in\Gamma$” as follows. Let the observer
receive the message that ”$x\in\Omega$”. The _information value_ of this
message (for this observer) is a real number ${\cal I}_{\Gamma}(\Omega)\geq
0$. If $\Omega=\Gamma$ this value is obviously zero (the message is
worthless), i.e. ${\cal I}_{\Gamma}(\Gamma)=0$. The larger the information
value ${\cal I}_{\Gamma}(\Omega)$ the more valuable information is gained by
the observer after receiving the message ”$x\in\Omega$”.

It is important to say that the referential set $\Gamma$ is not an arbitrarily
chosen subset of the state space. It plays the role of an ’information
content’ of a concrete observer, whereas ’observer’ may be an experimental
device, a robot with sensors, a typical macroscopic observer, etc. If an
observer connected with referential information $\Gamma$ gains additional
information about the system, say ”$x\in\Omega_{0}$”, the referential set
changes into $\Gamma^{\prime}=\Gamma\cap\Omega_{0}$.

A collection of referential sets $\Gamma(t)$ for various $t$ during a single
microscopic process is called the _information reference frame_ (IRF) in which
this process is studied. Let an observer (or observers) study a process in
between times $t_{i}$ and $t_{f}$. The IRF may be formed by all $\Gamma(t)$,
$t\in\langle t_{i},t_{f}\rangle$, if the referential set can be defined at
each time moment (e.g. when the process can be continuously observed). An
extreme case of IRF describing a process from $t_{i}$ to $t_{f}$ is the IRF
formed only by $\Gamma(t_{i})$ and $\Gamma(t_{f})$.

Figure 1:  An example of IRF. Consider an isolated system including a
horizontal, free-movable massive piston in a gravity field and a gas occupying
the space bellow the piston. The set of observable indicators consists of the
position of the piston, $X$, and its current velocity $V$ (in vertical
direction), i.e. $\alpha=(X,V)$. At $t_{i}$, the gas is in an arbitrary (non-
equilibrium) state $s_{i}$, the piston at the state
$\alpha_{i}=(X_{i},V_{i})$. The piston is falling down so that it has the
position $X_{f}$ and velocity $V_{f}$ at $t_{f}>t_{i}$ while the gas reaches
the state $s_{f}$. At $t$, the set $\overline{\Gamma}(\alpha(t))$ is formed by
_all_ possible states of the gas being consistent with the current state of
the piston, $\alpha(t)$. It implies that $\Gamma_{i}\equiv\Gamma(t_{i})$
includes many states that _cannot_ evolve the system to the state in which the
piston state is $\alpha_{f}$ at $t_{f}$, and the set
$\Gamma_{f}\equiv\Gamma(t_{f})$ includes many states that cannot be reached
from the given initial conditions at $t_{i}$. See Fig. 2.

The important situation occurs if referential sets can be identified with
values of some physical quantities $\alpha$, we call them the _observable
indicators_. The IRF is then defined by time evolution of these quantities
(see Fig. 1). The set $\Gamma=\Gamma(\alpha)$ then includes all microscopic
states that are consistent with the value of $\alpha$. Typical indicators are
various coordinates defining the thermodynamic state Jaynes1965 ; Callen1985
regardless at which length scale the observation is done (volume of a gas,
quantities at small hydrodynamic cells, distance of the ends of a
macromolecule Jar2011 ).

The observable indicators may be fully controlled by an observer (e.g. driven
by an external agent Jar1997 ; Crooks1998 ; Crooks2000 ) or may be understood
as special degrees of freedom of a fully autonomous system. In the latter
case, the microstate $x$ of the system equals $x=(s,\alpha)$, where $s$ are
_internal_ (not-observed) degrees of freedom (see Fig. 1), and

$\Gamma(\alpha)=\\{(s,\alpha),\,\,s\in\overline{\Gamma}(\alpha)\\}.$ (1)

The set $\Gamma$ may represent a ”grain” in a coarse-grained modeling
Levitt2014  and, as an extreme case, it can be even identical with the actual
microstate of the system, $\Gamma=\\{x\\}$, which means that an observer has a
complete information about the system. Information about the system can be
also formed by all quantum states $|\psi_{i}\rangle$ in a quantum projector
$\sum|\psi_{i}\rangle\langle\psi_{i}|$ describing an incomplete knowledge
about a system in a mixed state StrasWint2021 . A special case of IRF is the
IRFmic defined by $\Gamma_{mic}(t)=\\{x(t)\\}$.

The important point of the presented approach is that referential sets (and
IRF) may not be connected with any observed indicators. An important example
of such a situation arises if the microscopic evolution of the system is
deterministic. Let us denote ${\cal F}$ the evolution operator $x_{f}={\cal
F}(x_{i})$, where $x_{i}$, $x_{f}$ is the initial and final microstate,
respectively. Let $\Gamma_{i}$, $\Gamma_{f}$ be the referential sets at times
$t_{i}$, $t_{f}$, respectively. Denote $G_{i}\subset\Gamma_{i}$ the set of all
states $x\in\Gamma_{i}$ so that ${\cal F}(x)\in\Gamma_{f}$, and
$G_{f}\equiv{\cal F}(G_{i})$ (see Fig. 2).

We define the so-called _dynamic_ information reference frame IRF’ as
$\Gamma^{\prime}(t_{i})=G_{i}$, $\Gamma^{\prime}(t_{f})=G_{f}$. The
referential sets here cannot be defied by current values of some observable
indicators. If $\Gamma(t)$ are defined by current values of observable
indicators $\alpha$ the sets $G(t)$ are given by knowledge of the process
$\alpha_{i}\to\alpha_{f}$. The transformation of the reference frame into the
dynamic one, IRF $\to$ IRF’, plays important role in what follows.

Figure 2: The mapping ${\cal F}$ represents the evolution of states from a
large state space to their values after the time $\tau=t_{f}-t_{i}$. The
change $\alpha_{i}\to\alpha_{f}$ happens only if the initial state $x_{i}\in
G_{i}\subset\Gamma_{i}$. A state $z_{i}\in\Gamma_{i}$ not belonging into
$G_{i}$ evolves to a microscopic state $z_{f}$ that corresponds to another
observable indicators, $\alpha^{\prime}\neq\alpha_{f}$. Similarly, a state
$z^{\prime}\in\Gamma_{f}$ not belonging into $G_{f}$ is a result of an
observed process $\alpha_{0}\to\alpha_{f}$ that cannot begin at $\alpha_{i}$.
The sets $\Gamma(t)$ define the IRF given by knowledge of current values of
$\alpha$ at individual times, the sets $G(t)$ define the IRF’ based on the
knowledge of the process $\alpha_{i}\to\alpha_{f}$.

_Entropy_. — From an information theoretic perspective, entropy is associated
with observer’s ignorance (lack of information) as to the microstate of the
system Jaynes1957 ; Crooks1999 . In other words, the observer has some
(incomplete) information about the microstate of the system and entropy
measures the value of additional information that is necessary to ”add” to
this knowledge to determine the actual microstate precisely. The knowledge of
observer is identified here with referential information. It implies that the
_information entropy_ is defined in an IRF as

${\cal S}_{\Gamma}(x)=k_{B}{\cal I}_{\Gamma}(\\{x\\}),$ (2)

where $k_{B}$ is Boltzmann’s constant and $\Gamma$ is a referential set from
this IRF.

From thermodynamic point of view, entropy is a _state_ quantity Callen1985 ;
LiebYngvason2013  what means that ${\cal S}$ can depend _only_ on such
information that concerns the system at a concrete time. It implies that each
referential set from an IRF must be interpreted as information about the
system at a concrete time moment only. If a set of the dynamic IRF defined
above, i.e. $G(t)$, is used to define entropy it must be interpreted as
special information about a concrete time moment that is not determined by
observable indicators.

The information entropy defined by Eq. (2) may be understood as an objective,
microscopic quantity (it depends on $x$) whose meaning is simply ’information
about the actual microstate’. The well-known doubts concerning objectivity of
entropy become irrelevant here because its observer-dependence is nothing else
but a usual dependence of any physical quantity on some reference element
(unit, frame, etc.). The concrete value of entropy is the value of information
evaluated always in a concrete IRF. Notice that the entropy is, for example,
always zero in the IRFmic.

_Properties of information value_. — Consider a physical phenomenon that can
happen if and _only_ if $x\in\Omega$. A very large value of ${\cal
I}_{\Gamma}(\Omega)$ means that the realization of this phenomenon is very
rare for this observer (the value of message ”this number will win in lotto”
is very high). Hence ${\cal I}_{\Gamma}(\Omega)\to\infty$ means that the
observer can observe this phenomenon with a overwhelmingly low probability,
i.e. practically cannot observe this phenomenon at all.

Let us postulate the two important properties of ${\cal I}_{\Gamma}$:
(i) If $\Omega\subset\Gamma_{2}\subset\Gamma_{1}$ then the observer who knows
$\Gamma_{1}$ needs additional information $x\in\Gamma_{2}$ to become
equivalent to that one who knows $\Gamma_{2}$, i.e. we postulate

${\cal I}_{\Gamma_{1}}(\Omega)={\cal I}_{\Gamma_{2}}(\Omega)+{\cal
I}_{\Gamma_{1}}(\Gamma_{2}).$ (3)

(ii) The smaller is the set $\Omega$ with respect to $\Gamma$ the higher must
be the value of information detecting $\Omega$. That is why we postulate

${\cal I}_{\Gamma}(\Omega)=\Phi_{\Gamma}(M_{\Gamma}(\Omega))\geq 0,$ (4)

where $\Phi_{\Gamma}$ is a decreasing real function and $M_{\Gamma}$ is a real
measure of $\Omega$ on $\Gamma$ so that ${\cal I}_{\Gamma}(\Gamma)=0$.
Concerning the measure $M_{\Gamma}$ we suppose that $M_{\Gamma}(\Omega)\to 0$
if the size of $\Omega$ goes to zero. It implies Rudin1987  that $M_{\Gamma}$
is expressed via a real, non-negative function $f_{\Gamma}$ so that
$M_{\Gamma}(\Omega)=\int_{\Omega}f_{\Gamma}(z){\rm d}z$ in continuous, and
$M_{\Gamma}(\Omega)=\sum_{z_{i}\in\Omega}f_{\Gamma}(z_{i})$ in discrete cases.

The special case of ${\cal I}_{\Gamma}$ fulfilling the postulated properties
is ${\cal I}_{\Gamma}^{Sh}(\Omega)=-\ln p(\Omega|\Gamma)$ (Shannon’s formula),
where $p(\Omega|\Gamma)$ is the probability measure on $\Gamma$, i.e.
$M_{\Gamma}(\Omega)=p(\Omega|\Gamma)$, $\Phi_{\Gamma}=-\ln$. The information
value ${\cal I}_{\Gamma}$, however, can be defined without any connotation to
probabilities.

##  III Adiabatic processes

Any process governed only by inner dynamics of an isolated system is
_adiabatic_. The concept of adiabatic processes is, however, broader. Namely,
it is any process during which the exchange of energy with its surrounding
exists only in the form of work. Work is defined in classical macroscopic
thermodynamics, its generalization to meso- or microscopic processes is far
from being easy and straightforward.

The well-accepted, scale-independent definition of processes during which the
system-surrounding interaction is realized only with the exchange of work
concerns situations when the evolution of observable indicators is firmly
given by a prescribed protocol Jar1997 ; Crooks1998 ; Crooks2000  and no other
external intervention exists. Though there exists an exchange of energy with
the system environment, such a process is fully deterministic and reversible.
It motivates us to _define_ the adiabatic process as _any_ deterministic and
reversible process $x_{i}\to x_{f}$, where $x$ is the full (microscopic) state
of the system.

A concrete realization of the ’prescribed-protocol’ interaction means the
existence of macroscopic massive bodies or some device that are insensitive to
microscopic interactions. The work then can be defined for the system alone
(supplied work) while its transfer into the surrounding cannot be quantify
anyhow (see Fig. 3a) MaesTasaki2007 ; Peliti2008 ; Holecek2019 . The above
definition of the adiabatic process as any deterministic and reversible
(unitary) process, however, does not exclude possibility of defining the work
performed in the surrounding.

Let ${\bf S}$ be a system and we study processes beginning at various
microscopic states of this system, $x_{i}\in\Gamma_{\bf S}$. During any
process, the system interacts with its environment, ${\bf E}$, so that ${\bf
S}+{\bf E}$ is a fully isolated system (universe). Suppose that the
environment and the system-environment interaction is designed in such a way
that the initial (microscopic) state of the environment is always $e_{i}$ and
its final state (say, at a sufficiently large time) is always $e_{f}$. The
transferred work into this environment, $W_{tr}$, is a function of
$(e_{f},e_{i})$ (see Fig. 3b).

Figure 3: Two examples of an adiabatic process. a) The piston moves with an
externally controlled velocity $V$. A molecule elastically impacting this
piston changes own kinetic energy, $\Delta E_{kin}=-2mV(v_{z}-V)$, where
$v_{z}$ is its perpendicular velocity, $v_{z}>V$, and $m$ its mass. This
change of energy is interpreted as the _supplied work_. No transfer of energy
into the piston can be described here. b) The system ${\bf S}$ interacts with
its environment ${\bf E}$ so that during any process the environment is at
given states $e_{i}$ and $e_{f}$ at the beginning and the end of the process,
respectively. These states define the energy transformed from ${\bf S}$ to
${\bf E}$, $W_{tr}(e_{f},e_{i})$, e.g. the change of a weigh in gravity field.
The processes are deterministic and reversible.

The universe ${\bf S}+{\bf E}$ is an isolated system, hence the process
$(x_{i},e_{i})\to(x_{f},e_{f})$ is deterministic and reversible. Since $e_{i}$
and $e_{f}$ are fixed, the process $z_{i}\to z_{f}$ is deterministic and
reversible too (though the trajectory of the environment and its interaction
with the system is different for various initial conditions of the system).
This idea is a generalization of the concept of adiabatic accessibility
LiebYngvason1999  into microscopic scales.

_Information conservation_. — During adiabatic evolution, information about
possible microstates of the system has to be conserved. The reason is that the
evolution is deterministic and reversible which means that the configuration
at any time moment carries information about that at any other time.

Let ${\cal F}_{\tau}$, $\tau\in\bf{R}$ be a set of one-to-one mappings
expressing the time shift of the system microstate during time evolution, i.e.
$x(t+\tau)={\cal F}_{\tau}(x(t))$ (hence the inverse mapping of ${\cal
F}_{\tau}$ is ${\cal F}_{-\tau}$). Since the value of information about
microstates is always related to referential information the _law of
conservation of information_ must be expressed as follows:

$\forall\tau\,\forall\Omega\subset\Gamma:\,\,\,{\cal I}_{{\cal
F}_{\tau}(\Gamma)}({\cal F}_{\tau}(\Omega))={\cal I}_{\Gamma}(\Omega).$ (5)

The referential set, however, usually does _not_ vary according the rule
$\Gamma\to{\cal F}_{\tau}(\Gamma)$. If it is defined by values of some
observable indicators, i.e. $\Gamma(\alpha)$, the set $\Gamma(\alpha(t+\tau))$
is usually different from ${\cal F}_{\tau}(\Gamma(\alpha))$ (see Figs. 1,2).

_Adiabatic entropy change_. — The transformation into a dynamic reference
frame IRF’, IRF $\to$ IRF’, can be written by the use of Eq. (3). If
$\Omega\subset G_{i}$ we get

${\cal I}_{\Gamma_{i}}(\Omega)={\cal I}_{G_{i}}(\Omega)+{\cal
I}_{\Gamma_{i}}(G_{i}),$

and

${\cal I}_{\Gamma_{f}}({\cal F}_{\tau}(\Omega))={\cal I}_{G_{f}}({\cal
F}_{\tau}(\Omega))+{\cal I}_{\Gamma_{f}}(G_{f}).$

Since $G_{f}={\cal F}_{\tau}(G_{i})$ the law of conservation of information
cancels the first terms on the right-hand sides of the equations and we get
that ${\cal I}_{\Gamma_{i}}(\Omega)-{\cal I}_{\Gamma_{f}}({\cal
F}_{\tau}(\Omega))$ does not depend on $\Omega$, i.e.

${\cal I}_{\Gamma_{f}}({\cal F}_{\tau}(\Omega))-{\cal
I}_{\Gamma_{i}}(\Omega)={\cal I}_{\Gamma_{f}}(G_{f})-{\cal
I}_{\Gamma_{i}}(G_{i})\equiv\Delta{\cal J},$ (6)

where $\Delta{\cal J}={\cal J}^{\prime}-{\cal J}$, ${\cal J}\equiv{\cal
I}_{\Gamma_{i}}(G_{i})$, ${\cal J}^{\prime}\equiv{\cal
I}_{\Gamma_{f}}(G_{f})$.

If we chose $\Omega=\\{x\\}$, $x\in G_{i}$, we get the change of the entropy
distribution $\Delta{\cal S}(x)={\cal S}_{\Gamma_{f}}({\cal
F}_{\tau}(x))-{\cal S}_{\Gamma_{i}}(x)$ so that

$\Delta{\cal S}(x)=k_{B}\Delta{\cal J}.$ (7)

The right-hand side of Eq. (7) depends only on $\Gamma_{i}$, $\Gamma_{f}$ and
the mapping ${\cal F}_{\tau}$. It implies that the change of entropy is the
same for each $x\in G_{i}$,

$\forall x,x^{\prime}\in G_{i}:\,\,\Delta{\cal S}(x)=\Delta{\cal
S}(x^{\prime})\equiv\Delta{\cal S}.$ (8)

The change of entropy during an adiabatic process that is observed as
$\alpha_{i}\to\alpha_{f}$ must begin at $G_{i}$ hence $\Delta{\cal S}$ is
determined by the change of $\alpha$, i.e. $\Delta{\cal
S}(\alpha_{i}\to\alpha_{f})$. It is a very important result: though the
information entropy is a microscopic quantity its change is determined only by
the change of quantities defined at the scale where the system is studied.

##  IV Concrete form of the information entropy

Let $\Gamma$ be a subset of a system state space and $x\in\Gamma$. The value
of information entropy ${\cal S}_{\Gamma}(x)$ is uniquely determined by $x$
and $\Gamma$. To find out this dependence we suppose that there exists a
sufficiently small $\tau_{0}$ so that for each $0<\tau<\tau_{0}$ the element
${\cal F}_{\tau}(x)\in\Gamma$, where ${\cal F}_{\tau}$ represent an adiabatic
process. Hence we can choose the same referential sets, $\Gamma(t)=\Gamma$,
for each $t\in\langle 0,\tau_{0}\rangle$. It allows us to form the IRF for
each $\tau$ ($\tau<\tau_{0}$) so that $\Gamma(0)\equiv\Gamma_{i}=\Gamma$ and
$\Gamma(\tau)\equiv\Gamma_{f}=\Gamma$.

Since $\Gamma$ is fixed in this consideration we omit the index $\Gamma$ in
all quantities, i.e. ${\cal S}\equiv{\cal S}_{\Gamma}$ and the quantities in
Eq. (4): $M\equiv M_{\Gamma}$, $f\equiv f_{\Gamma}$, and
$\Phi\equiv\Phi_{\Gamma}$. Since ${\cal S}(x)=k_{B}{\cal
I}_{\Gamma}(\\{x\\})=k_{B}\Phi(f(x))$ we get

$f(x)=\Phi_{-1}(\psi(x)),$ (9)

where $\Phi_{-1}$ is the inverse function of $\Phi$ and $\psi(x)\equiv
k_{B}^{-1}{\cal S}(x)$.

Suppose that the state space is discrete. Then the measure
$M(\Omega)=\sum_{x\in\Omega}f(x)$ and Eq. (6) becomes

$\Phi(\sum_{x\in\Omega}\Phi_{-1}(\psi(x)+\Delta{\cal
J}))=\Phi(\sum_{x\in\Omega}\Phi_{-1}(\psi(x)))+\Delta{\cal J}$ (10)

(we use Eq. (7) to express that $\psi({\cal F}(x))=\psi(x)+\Delta{\cal J}$).

The validity of Eq. (10) for all $\tau\in(0,\tau_{0})$ determines a possible
form of ${\cal I}_{\Gamma}$ (see Appendix), namely

${\cal
I}_{\Gamma}(\Omega)=-\gamma\ln\sum_{x\in\Omega}e^{-\gamma^{-1}k_{B}^{-1}{\cal
S}_{\Gamma}(x)},$ (11)

where $\gamma$ is a positive constant. It is easy to check that this form of
${\cal I}_{\Gamma}$ fulfills Eq. (6) for any $\Gamma_{i}$, $\Gamma_{f}$ and
$\Delta{\cal J}$ if the entropy distribution fulfills Eq. (7).

To fulfill the condition ${\cal I}_{\Gamma}(\Gamma)=0$ the entropy has to have
a form

${\cal S}_{\Gamma}(x)=k_{B}\gamma(\ln|\Gamma|_{\Psi}+\Psi_{\Gamma}(x)),$ (12)

where $\Psi_{\Gamma}$ is a real function on the state space of the system that
may depend also on $\Gamma$, $\gamma$ is a constant, and $|\Gamma|_{\Psi}$ is
the special measure of the set $\Gamma$,

$|\Gamma|_{\Psi}=\sum_{x\in\Gamma}e^{-\Psi_{\Gamma}(x)}.$ (13)

The sums $\sum_{x\in\Omega}$ in Eqs. (12,13) must be replaced by integrals,
$\int_{\Omega}{\rm d}x$, in the case of the continuous state space. The
constant $\gamma$ only rescales the information value and information entropy.
In what follows we fix $\gamma=1$.

The distribution of $\Psi_{\Gamma}$ on $\Gamma$ depends on the mappings ${\cal
F}_{\tau}$. Namely putting Eq. (12) into Eq. (6) we get

$\Delta\Psi(x)\equiv\Psi_{\Gamma_{f}}(x(t+\tau))-\Psi_{\Gamma_{i}}(x(t))=\Delta{\cal
J}(\tau)+\ln\frac{|{\Gamma_{f}}|_{\Psi}}{|{\Gamma_{i}}|_{\Psi}}.$ (14)

It implies that

$\forall x\in G_{i}({\cal
F}):\,\,\,\Delta\Psi(x)=C(\Gamma_{i},\Gamma_{f},{\cal F}).$ (15)

This condition is analogical to Eq. (8) which means that the change of
function $\Psi$ is similarly defined on the observation scale.

The only entropy distribution that fulfills Eq. (7) for an arbitrary ${\cal
F}$ arises if we choose $\Psi_{\Gamma}(x)=C(\Gamma)$, i.e. it does not depend
on $x$. We get

${\cal S}_{\Gamma}(x)=k_{B}\ln|\Gamma|=S_{B},\,\,\,\,(x\in\Gamma)$ (16)

where $|\Gamma|$ is the volume of the set $\Gamma$. That is we get the
Boltzmann entropy $S_{B}$ as a special case of the information entropy (12).

_Probabilistic interpretation_. — The formula (11) can be interpreted
otherwise. If we define

$p_{\Gamma}(\Omega)\equiv e^{-\gamma^{-1}{\cal I}_{\Gamma}(\Omega)}>0.$ (17)

and write simply $p_{\Gamma}(x)$ instead of $p_{\Gamma}(\\{x\\})$, Eq. (11)
can be written as

$p_{\Gamma}(\Omega)=\sum_{x\in\Gamma}p_{\Gamma}(x),$ (18)

and $\sum_{x\in\Gamma}p_{\Gamma}(x)=1$ because of the condition ${\cal
I}_{\Gamma}(\Gamma)=0$.

The values of $p_{\Gamma}$ thus can be understood as a probability
distribution over the referential set $\Gamma$ so that the probability

$P(x\in\Omega)=e^{-{\cal I}_{\Gamma}(\Omega)}$ (19)

when putting $\gamma=1$. It implies the Shannon-like expression of entropy,

${\cal S}_{\Gamma}(x)=-k_{B}\ln p_{\Gamma}(x).$ (20)

Notice that Eq. (19) implies that ${\cal I}_{\Gamma}(\Omega)\to\infty$ if and
only if $P(x\in\Omega)\to 0$.

##  V The second law of thermodynamics

Consider an observer, say Alice, who knows data about a system ${\bf S}$
(possibly meso- or microscopic) only at a given time $t_{i}$: she performs a
measurement at $t_{i}$ and gains some data $\alpha(t_{i})=\alpha_{i}$ about
the system. Alice knows that the next evolution of ${\bf S}$ is adiabatic (its
microscopic evolution is deterministic and reversible). Alice asks which value
of $\alpha$ can be detected at a time $t_{f}>t_{i}$.

Each possible value of $\alpha(t_{f})\equiv\alpha_{\star}$ forms a subset
$G(\alpha_{\star})\subset\Gamma_{i}$ of all possible initial microstates that
realize the evolution $\alpha_{i}\to\alpha_{\star}$ (this subset may be
empty). Alice does not know, however, in which subset the actual microstate
occurs. Nevertheless, information that $x_{i}\in G(\alpha_{\star})$ has
different values for various $\alpha_{\star}$, namely $I(\alpha_{\star})={\cal
I}_{\Gamma_{i}}(G(\alpha_{\star}))$. The largest this value the smaller
probability $P$ that $x_{i}$ belongs into this subset as implied by Eq. (19),

$P(x_{i}\in G(\alpha_{\star}))=e^{-I(\alpha_{\star})}.$ (21)

Hence if $I(\alpha_{\star})\to\infty$ then the result
$\alpha(t_{i})=\alpha_{\star}$ cannot occur since its probability is zero.

In macroscopic thermodynamics, the second law can be expressed as
impossibility of adiabatic processes during which the entropy decreases, i.e.
during which $\Delta{\cal S}<0$. Let us study the situation when the entropy
decreases by the use of Eq. (7) in the macroscopic limit $k_{B}\to 0$ so that
$\Delta S\neq 0$.

If $\Delta{\cal S}/{k_{B}}\to-\infty$, Eq. (7) implies ${\cal J}-{\cal
J}^{\prime}\to\infty$, i.e. ${\cal J}\to\infty$ since ${\cal J}^{\prime}\geq
0$. ${\cal J}={\cal I}_{\Gamma_{i}}(G_{i})=I(\alpha_{f})$ is the value of
information that $x_{i}\in G_{i}$, i.e. that the process
$\alpha_{i}\to\alpha_{f}$ will happen. Eq. (21) implies that if $\Delta{\cal
S}<0$ then the probability of this process is zero. Hence only macroscopic
adiabatic processes with non-decreasing entropy are possible. Eq. (7) thus
expresses the second law of thermodynamics.

Figure 4: Consider a process $\alpha_{i}\to\alpha_{f}$ from $t_{i}$ to $t_{f}$
so that the set $\Gamma_{f}$ corresponding to $\alpha_{f}$ is overwhelmingly
larger than the set $\Gamma_{i}$ corresponding to $\alpha_{i}$. It is a
typical macroscopic situation when $\Delta S/k_{B}$ is extremely large, say
$\sim 10^{20}$. a) If an observer, Bob, has information _only_ concerning
$t_{f}$, i.e. he knows $\alpha_{f}$, he describes the process at the IRF given
by sets $\Gamma(t)$ and he must infer that the probability that the origin of
the situation at $t_{i}$ is a point from an extremely small region
$\Gamma_{i}$ is negligibly small, $P\sim 10^{-10^{20}}$. He must conclude that
$\alpha(t_{i})$ cannot be $\alpha_{i}$. b) If Bob has information (a record)
that $\alpha$ at $t_{i}$ was $\alpha_{i}$ then he has information that
corresponds to the description in IRF’ defined by sets $G(t)$.

This formulation of the second law is formally very close to Boltzmann’s
statistical derivation of this law. The main argument is probabilistic too: if
$\Delta S<0$ and $|\Delta S|\gg k_{B}$ the ”target” set $G_{f}$ is extremely
smaller than the set of all initial possibilities ($\Gamma_{i}$) and there is
an overwhelmingly small probability of ”hitting” it (to realize the process
$\alpha_{i}\to\alpha_{f}$) Lebowitz1993 ; Penrose2005 .

The information formulation, however, can avoid the principle problem of
Boltzmann, i.e. the fact that the statistical argumentation can be used in the
opposite time direction to make a paradox Penrose2005 ; Albert2000 ;
Earman2006 ; Callender2021 . To show it, imagine another observer, say Bob,
who has information about the same system ${\bf S}$ at time $t_{f}$ without
communicating with Alice. Bob detects the value $\alpha(t_{f})=\alpha_{f}$ and
asks which value of $\alpha$ was at time $t_{i}<t_{f}$. If $\Delta{\cal
S}={\cal S}(t_{f})-{\cal S}(t_{i})>0$ we get from Eq. (7) in the macroscopic
limit that ${\cal J}^{\prime}\to\infty$. The probability that $x_{i}$ was in
$\Gamma_{i}$ is thus overwhelmingly small (see Fig. 4a). Bob must conclude
that the past of the system could not be so that $\alpha(t_{i})$ was
$\alpha_{i}$. This conclusion is, however, false.

Nevertheless, the situation of Alice and Bob is not symmetric from information
point of view. Bob can receive a message from Alice about the situation at
$t_{i}$ while Alice cannot have a message from Bob about the situation at
$t_{f}$ (information cannot be send into the past). The conclusion of Alice is
thus based on _all_ possibly attainable information (at her observation scale)
about the system at $t_{i}$. The situation of Bob is different. There are two
possibility concerning his information state (see Fig. 4):
(i) Bob has a _record_ about the situation at $t_{i}$ (e.g. Alice’s message).
Then his information about the system is not only $\alpha(t_{f})$ (his
measurement) but also $\alpha(t_{i})$ (the record). Having this information
Bob cannot use $\Gamma_{f}$ as the set of _all_ possibilities corresponding to
his information about the system. If he knows that $\alpha(t_{i})=\alpha_{i}$
and $\alpha(t_{f})=\alpha_{f}$ then all possibilities are given by sets
$G_{i}$ and $G_{f}$. Bob thus uses the dynamic referential frame, IRF’, in
which $\Gamma^{\prime}(t_{i})=G_{i}$, $\Gamma^{\prime}(t_{f})=G_{f}$ (see Fig.
4b). In this referential frame $\Delta{\cal S}^{\prime}=0$ and Eqs. (7,19) in
IRF’ gives that $P(x_{i}\in G_{i})=P(x_{f}\in G_{f})$. No paradox arises.
(ii) There is no record about the situation at $t_{i}$. Bob concludes from Eq.
(7) that $\alpha$ at $t_{i}$ could not be $\alpha_{i}$. His conclusion,
however, cannot be verified anyhow: Bob cannot send a message into the past
and no record about the situation at $t_{i}$ exists. Whenever a record about
the situation at $t_{i}$ appears (e.g. in a form of an indirect physical
proof) Bob must shift his consideration into the dynamic referential frame
defined by sets $\Gamma^{\prime}=G$ and we get again the situation (i).

##  VI $\Psi$-representations of entropy

The function $\Psi_{\Gamma}$ has a unique meaning at the observable scale
since its change is determined only by the process $\alpha_{i}\to\alpha_{f}$
as implied by Eq. (15). A concrete choice of the function $\Psi_{\Gamma}$
defines the information entropy and, consequently, the information value
${\cal I}_{\Gamma}$. We call it the $\Psi$-_representation_. The simplest
$\Psi$-representations are related to several kinds of entropy:
_Boltzmann entropy_. It arises in the $C$-representation in which the function
$\Psi_{\Gamma}$ is constant (Eq. (16)).
_Shannon entropy_. The probabilistic interpretation of information entropy as
given by Eqs. (19,20) can be interpreted as a special $\Psi$-representation,
called the $p$-representation. We get it if the first term in Eq. (12) is
identically zero, i.e. $|\Gamma|_{\Psi}=1$ for an arbitrary $\Gamma$. If
denote $\pi_{\Gamma}(x)\equiv\exp(-\Psi_{\Gamma}(x))>0$, we get that the
information entropy in $p$-representation, ${\cal S}_{p}$, has now the form of
Shannon entropy, Eq. (20), $\sum_{x\in\Gamma}\pi_{\Gamma}(x)=1$ and
$\pi_{\Gamma}=p_{\Gamma}$ are probabilities.
_Clausius (equilibrium) entropy_. Another possible $\Psi$-representation we
get if $\Psi_{\Gamma}$ is a constant of motion, i.e. $\Delta\Psi=0$. If the
system energy is conserved during the studied process we can identify
$\Psi_{\Gamma}(x)=\beta{\cal H}(x)$, where ${\cal H}(x)$ is the system
hamiltonian and $\beta\equiv(k_{B}T)^{-1}$ with $T$ being an artificially
chosen constant. In this ${\cal H}$-representation we have

${\cal I}_{\Gamma}(\Omega)=-\ln\frac{\sum_{x\in\Omega}e^{-\beta{\cal
H}(x)}}{\sum_{x\in{\Gamma}}e^{-\beta{\cal H}(x)}},\,\,\,\,\,\,({\cal H}-{\rm
rep.})$ (22)

and Eq. (12) becomes

${\cal S}_{\Gamma}(x)=T^{-1}({\cal H}(x)-F_{\Gamma})=S_{C},\,\,\,\,\,\,({\cal
H}-{\rm rep.})$ (23)

where $F_{\Gamma}\equiv-k_{B}T\ln\sum_{\Gamma}\exp(-\beta{\cal H}(x))$ is the
free energy. If $x\in\Gamma_{eq}$, where $\Gamma_{eq}$ is the equilibrium
macrostate of the system, the hamiltonian can be identified with the system
internal energy, $U={\cal H}(x)$, the parameter $T=T_{eq}$ is the
thermodynamic temperature, and the information entropy becomes the classical
(Clausius) entropy $S_{C}$.

##  VII Probabilities

When assuming that the entropy in $\Psi$-representation equals the entropy in
$p$-representation, i.e. ${\cal S}_{\Gamma}=S_{p}$, we find the probabilities
in individual representations. In $C$-representation we get

$p_{\Gamma}(x)=|\Gamma|^{-1}.\,\,\,\,\,\,(C-{\rm rep.})$ (24)

Similarly in ${\cal H}$-representation we have

$p_{\Gamma}(x)=\frac{e^{-\beta{\cal H}(x)}}{Z_{\Gamma}}=p_{eq}({\cal
H}(x)),\,\,\,\,\,\,({\cal H}-{\rm rep.})$ (25)

where $Z_{\Gamma}=\sum_{\Gamma}e^{-\beta{\cal H}(z)}$. It means that the
probability of a microstate corresponds in the ${\cal H}$-representation with
the Boltzmann-Gibbs equilibrium distribution, $p_{eq}({\cal H})$. Its
interpretation is, however, different. The microscopic energy of a system in
thermal contact with a reservoir fluctuates and reaches the energy ${\cal E}$
with the probability $p_{eq}({\cal E})$. We, however, study an isolated
system, i.e. ${\cal H}(x(t))$ is constant along its trajectory. The
probability $p_{\Gamma}(x)$ expresses in ${\cal H}$-representation nothing but
observer’s knowledge as to $x$ in dependence on $\Gamma(t)$ (that is expressed
through $Z_{\Gamma}$).

During the unitary evolution the entropy change in $p$-representation,
$\Delta{\cal S}=-(\ln p_{\Gamma_{f}}({\cal F}(x))-\ln p_{\Gamma_{i}}(x))$,
must be the same for all $x\in G_{i}$, i.e.

$P_{out}=P_{in}e^{-k_{B}^{-1}\Delta{\cal S}},$ (26)

where

$P_{in}=\sum_{x\in G_{i}}p_{\Gamma_{i}}(x),\,\,\,P_{out}=\sum_{x^{\prime}\in
G_{f}}p_{\Gamma_{f}}(x^{\prime}).$

This result looks strange. Namely we expect that the probability of occurrence
of the system at a (micro)state $x$ so that the process
$\alpha_{i}\to\alpha_{f}$ is realized, i.e. $P_{in}$, must be the same as the
probability of finding the final (micro)state that corresponds to the fact
that the process $\alpha_{i}\to\alpha_{f}$ has been realized, i.e. $P_{out}$.
However, if $\Delta{\cal S}\neq 0$ then $P_{in}\neq P_{out}$.

The explanation of this (and other) seeming inconsistencies connected with Eq.
(26) consists in understanding the entropy in information context. Namely
information entropy is given only by information about the system concerning
only one time moment. A correct interpretation of Eq. (26) should be done via
the idea of two independent observers so that the first has information about
the system at $t_{i}$ only, the second does the same at $t_{f}$ (Alice and Bob
in Section V). The first one observes $\alpha_{i}$ and can (in principle)
deduce that the process $\alpha_{i}\to\alpha_{f}$ happens with the probability
$P_{in}$. The second one observes $\alpha_{f}$ and can (in principle) deduce
that the process $\alpha_{i}\to\alpha_{f}$ has happened with the probability
$P_{out}$ (she/he does not know the past of the system).

If an observer knows that the process $\alpha_{i}\to\alpha_{f}$ has happened
this observer knows more than the previous ones, i.e. she/he knows that
$x_{i}\in G_{i}$ and $x_{f}\in G_{f}$. We can describe this situation in the
dynamic information reference frame, IRF’$=\\{\Gamma^{\prime}(t)\\}$, in which
the referential sets $\Gamma^{\prime}(t_{i})=G_{i}$ and
$\Gamma^{\prime}(t_{f})=G_{f}$. In the IRF’, $\Delta{\cal S}^{\prime}=0$ and
the corresponding probabilities $P_{in}^{\prime}$ and $P_{out}^{\prime}$ are
now identical.

A direct connection of information entropy and probabilities shows the
necessity of defining also probabilities with respect to an information
reference frame, IRF. The interpretation of ’probability of an event’ thus
depends on a chosen IRF too. It is the context in which Eq. (26) must be
interpreted.

##  VIII Information form of the generalized second law

The expression of entropy in ${\cal H}$-representation, Eq. (23), can be
interpreted in standard thermodynamics concepts in the case of a macroscopic
system in thermal equilibrium. In this Section, we use the ${\cal
H}$-representation to find the thermodynamic interpretation of the information
entropy for an arbitrary (micro-, meso-, macroscopic) system in a general
nonequilibrium state.

We study a more complex structure called here the supersystem that is
perfectly isolated from surroundings so that its evolution is deterministic
and reversible and its energy is constant. The observable indicators of the
supersystem are its special degrees of freedom, $\alpha$, so that the complete
state of the supersystem $x=(s,\alpha)$. The studied thermodynamic system is
then defined by the (internal) degrees of freedom $s$. We can write

${\cal H}(x)=\epsilon(\alpha)+{\cal H}_{int}(s,\alpha),$ (27)

where $\epsilon(\alpha)$ is the energy connected with the observable degrees
of freedom and ${\cal H}_{int}$ represents the rest of energy connected with
internal degrees of freedom and their interactions with the observed ones. The
energy of the studied thermodynamic system is identified as ${\cal H}_{int}$.

A simple example is presented at Fig. 1. The supersystem is the whole
structure including the gas and the piston. The position and velocity of the
piston represent the observable indicators so that
$\epsilon(\alpha)=MgX+1/2MV^{2}$ with $M$ being the mass of the piston. The
gas is the own thermodynamic system. The change of $\epsilon$ during a process
measures the change of energy of the gas.

We study processes during which $\Delta{\cal H}=0$ (the supersystem is
isolated). The change of information entropy in ${\cal H}$-representation
during the single (microscopic) process $x_{i}\to x_{f}$ is given by Eq. (23),

$T\Delta{\cal S}=\Delta{\cal H}_{int}-\Delta F(\alpha,T),\,\,\,\,\,\,({\cal
H}-{\rm rep.})$ (28)

where $F$ corresponds to the free energy of the internal degrees of freedom,

$F(\alpha,T)=-k_{B}T\ln\sum_{s^{\prime}\in\Gamma(\alpha)}\exp(-\beta{\cal
H}_{int}(s^{\prime},\alpha)).$ (29)

During the process the energy $W=\Delta{\cal H}_{int}=-\Delta\epsilon$ (since
$\Delta{\cal H}=0$) is transferred from observable degrees of freedom into the
internal ones, i.e. into the thermodynamic system. This energy transfer can be
called the _supplied work_ regardless if the system is macroscopic or
microscopic Jar2004 . The quantity $T\Delta{\cal S}$ thus can be identified
with the _dissipated work_ , $W_{dis}=W-\Delta F$, defined for a single
trajectory KavParBro2007 ,

$T\Delta{\cal S}=W_{dis}.\,\,\,\,\,\,({\cal H}-{\rm rep.})$ (30)

The parameter $T$ is an arbitrarily chosen positive real number. Its value can
be fixed and identified with a thermodynamic temperature with using the idea
of contact temperature Muschik2021 .

Namely if an arbitrary system (micro-, meso-, macroscopic) is at the state
$x_{0}=(s_{0},\alpha_{0})$ we can put it into a thermal contact with a
sufficiently large thermal reservoir in equilibrium with the temperature
$T_{eq}$, fix the value of $\alpha$ and keep relaxing the internal degrees of
freedom into thermal equilibrium, i.e.
$(s_{0},\alpha_{0})\to(s_{eq},\alpha_{0})$. During the relaxation the energy
$Q_{rel}=Q(s_{0},\alpha_{0},T_{eq})$ is absorbed by the internal degrees of
freedom. In dependence on $T_{eq}$ this energy may be positive or negative. We
can choose $T_{eq}$ so that $Q_{rel}=0$ and identify the parameter $T$ with
this temperature, $T=T_{eq}$. The temperature $T$ thus depends on a chosen
state of the system, i.e. $T=T(s_{0},\alpha_{0})$.

_Generalized second law_. — The change of energy $\epsilon$ connected with
observable degrees of freedom $\alpha$ is the maximal energy that can be used
by the observer who detects (and principally can control) these variables,
i.e. the _usable work_ , $W_{u}=-W=\Delta\epsilon$. If $\Delta{\cal S}\geq 0$
we get from Eq. (28) the familiar inequality of classical thermodynamics,

$W_{u}\leq-\Delta F.$ (31)

We can express the relation between $W_{u}$ and $\Delta F$ more precisely in a
full general situation by using the transformation of the used IRF (that is
defined by observable indicators $\alpha$) into the dynamic IRF’ with
$\Gamma^{\prime}(t)=G(t)$. In the IRF’ we can define the free energy,
$F^{\prime}$, and the entropy ${\cal S}^{\prime}$, and we get $\Delta{\cal
S}^{\prime}=0$ and Eq. (28) gives $\Delta\epsilon=-\Delta F^{\prime}$. Using
Eq. (22) we get $\Delta F^{\prime}=\Delta F+\Delta{\cal J}$, i.e.

$W_{u}=-\Delta F-k_{B}T\Delta{\cal J}.$ (32)

Since $\Delta{\cal J}={\cal J}^{\prime}-{\cal J}$ and ${\cal J}^{\prime}\geq
0$ we get the general inequality

$W_{u}\leq-\Delta F+k_{B}T{\cal J},$ (33)

that generalizes the standard inequality Eq. (31) since ${\cal J}\geq 0$. The
additional term increases a possible value of the gained energy. This increase
has a pure informational character since ${\cal J}={\cal
I}_{\Gamma_{i}}(G_{i})$ is the value of information about such microscopic
initial conditions leading to the demanded gain of energy.

It is instructive to use the inequality (33) in a situation when an observer
repeats the experiment in which the initial observed indicator is always
$\alpha_{i}$ and the probability of occurrence of a concrete initial state of
internal degrees of freedom (microstate), $s_{i}$, corresponds to the state of
thermal equilibrium, $p_{eq}({\cal H}_{int}(s_{i},\alpha_{i}))$. If
$G_{i}\neq\Gamma_{i}$ the resulting value of $\alpha$ is not always
$\alpha_{f}$ and various results of $\Delta\epsilon=W_{u}(s_{i})$ can be
expected. The observed averaged quantity,

$\langle{W_{u}}\rangle=\sum_{s_{i}\in\Gamma}p_{eq}(s_{i},\alpha_{i})W_{u}(s_{i}),$

where $W_{u}(s_{i})=\epsilon(\alpha(s_{i}))-\epsilon(\alpha_{i})$ with
$\alpha(s_{i})$ being the final value of $\alpha$ if the initial $s$ is
$s_{i}$ (if $s_{i}\in G_{i}$ then $\alpha(s_{i})=\alpha_{f}$, see Fig. (2)).
Other averaged values are defined similarly.

The use of Eq. (22) for determining ${\cal J}={\cal I}_{\Gamma_{i}}(G_{i})$
and averaging Eq. (33) gives the averaged form of this inequality, namely

$\langle{W_{u}}\rangle\leq\langle\Delta F\rangle+k_{B}TI,$ (34)

where $I=\langle{{\cal J}}\rangle$ is the mutual information. The inequality
Eq. (34) is called the generalized second law of thermodynamics SagUed2009 ;
SagUed2010 ; SagUed2013 . It is worth stressing that this concrete form is
valid in the case when the averaging is done over the initial distribution
corresponding to the thermal equilibrium. Its information form, Eq. (33), is
valid for a single adiabatic process on an arbitrary length scale with an
arbitrary initial conditions (e.g. that representing a highly nonequilibrium
macroscopic state).

##  IX Concluding discussion

The statistical interpretation of the second law of thermodynamics is the most
natural and logical explanation of why reversible microscopic behavior can
manifest as irreversible at macroscopic scales. It is only a statistical
reason why a drop of ink put into a bottle with water will always smear over
the water, and why the dissolved ink does not return into the initial drop
formation. Namely the macrostate with a higher Boltzmann entropy is
overwhelmingly larger then that with a lower one (see Fig. 4a). Hence the
microstate wanders into this huge set in overwhelmingly many cases Penrose2005
.

I argue that the statistical way of analyzing macroscopic processes at the
microscopic level is nothing but an expression of information ideas. The set
of possible microstates $\Gamma$ (e.g. Boltzmann’s macrostate) is information
about the actual microstate $x$: it belongs into $\Gamma$. Entropy is
information about the actual microstate, ${\cal S}\sim{\cal I}(x)$. The
crucial idea is that the _value_ of information (as well as value of other
physical quantities) can be defined only with respect to referential
information. Contrary to the majority of physical quantities that can be
defined with respect to a firmly given referential element, the referential
sets $\Gamma$ are usually defined by actual values of some physical quantities
(observable indicators), $\alpha$, that vary in time. Moreover, the
referential set changes (diminishes) whenever the observer (a human, an
experimental device, robot, etc.) gains new information about the studied
system. The information reference frame (IRF) formed by referential sets
$\Gamma(t)$ thus resembles more referential frames of the general theory of
relativity firmy connected with dynamics of physical fields Rovelli2004 .

The essential element of the information form of the second law of
thermodynamics is the value of information gained at time $t$ that concerns
the coarse-grained state of the system at another time $t^{\prime}$. Let the
observer know only the actual coarse-grained state at $t$, say the current
values of some physical quantities $\alpha$. If she gets the message that this
state will be/was $\alpha^{\prime}$ at $t^{\prime}$ the value of this message,
$J$, depends on $\alpha$ and $\alpha^{\prime}$ ($t,t^{\prime}$ are given).
This message reveals the process between $t$ and $t^{\prime}$ if the situation
at $t$ is known, i.e. we can write $J=J(\alpha,\alpha\to\alpha^{\prime})\geq
0$. The second law of thermodynamics for adiabatic processes, Eq. (7), then
can be written as

$k_{B}^{-1}({\cal S}(t^{\prime})-{\cal
S}(t))=J(\alpha^{\prime},\alpha^{\prime}\to\alpha)-J(\alpha,\alpha\to\alpha^{\prime}).$
(35)

In the macroscopic limit, $|\Delta{\cal S}|/k_{B}\to\infty$, the value
$J(\alpha,\alpha\to\alpha^{\prime})\to\infty$ if $S(t^{\prime})<S(t)$, i.e.
$P(\alpha,\alpha\to\alpha^{\prime})\to 0$ due to Eq. (19). Hence the observer
who knows that the observable indicator is $\alpha$ at $t$ must conclude that
the observable indicator at $t^{\prime}$ cannot be $\alpha^{\prime}$.

It looks paradoxically since we must get either
$P(\alpha,\alpha\to\alpha^{\prime})\to 0$ or
$P(\alpha^{\prime},\alpha^{\prime}\to\alpha)\to 0$ if $\Delta{\cal S}\neq 0$
in the macroscopic limit, hence the occurrence of any couple of coarse-grained
states $\alpha$, $\alpha^{\prime}$ differing by a nonzero entropy appears as
impossible. The explanation of this evident nonsensical result consists in the
fact that the past and future are not symmetric: the observer can verify the
situation at $t^{\prime}$ if $t^{\prime}>t$. It implies that if
$P(\alpha,\alpha\to\alpha^{\prime})\to 0$ and $t^{\prime}>t$ the observer
_cannot_ detect $\alpha^{\prime}$ at $t^{\prime}$. Eq. (35) thus implies that
the adiabatic evolution of a macroscopic system cannot be connected with a
macroscopic decrease of entropy.

If $t^{\prime}<t$ the fact that $P(\alpha,\alpha\to\alpha^{\prime})\to 0$ does
not make a controversy since the observer who detects $\alpha$ at $t$ then
cannot verify the situation at $t^{\prime}$ (we cannot move into the past). If
there is, however, information that $\alpha$ _was_ $\alpha^{\prime}$ at
$t^{\prime}$ available at $t$, this information must occur in form of a record
(e.g. a change in some brain cells) that exists at $t$. The observer at $t$
thus has information about the system that forms the dynamic referential frame
IRF’ in which $\Delta{\cal S}^{\prime}=0$ and Eq. (35) implies that
$P^{\prime}(\alpha^{\prime},\alpha^{\prime}\to\alpha)=P^{\prime}(\alpha^{\prime},\alpha^{\prime}\to\alpha)$
may have an arbitrary value. The information form of the second law thus
inheres an important referential asymmetry between past and future though no
such asymmetry occurs in microscopic physics Holecek2022 .

The important question is why the change of the referential frame, IRF $\to$
IRF’, ”automatically” happens whenever a record about the past appears. The
explanation consists in the fact that the information reference frame is
primarily defined by _information_. A collection of subsets
$\Gamma(t)\subset\Gamma_{\bf S}$ is only its mathematical expression. A record
is a physical event that can be interpreted as information about a past
moment. If so, any physical consideration must take into account this
information.

An analogy with spatial reference frames is instructive here. Namely the
reference frame in space is primarily done by a collection of referential
bodies (an analogy of attainable information about a physical system). These
bodies define the mathematical structure ${\cal R}^{3}$ denoting coordinates
of individual spatial points (an analogy with the sets $\Gamma(t)$). Whenever
the original configuration of the referential bodies changes the same physical
point is described by other coordinates. Analogically, whenever information
about the system changes the same physical situation is described via
different sets $\Gamma(t)$.

The information form of the second law of thermodynamics for adiabatic
systems, $\Delta{\cal S}=k_{B}\Delta{\cal J}$, is valid at arbitrary length
scales. In probabilistic interpretation, it can be formulated by Eq. (26) that
expresses some stochastic character of the gained results whenever information
about the system is incomplete (i.e. when the sets $\Gamma(t)$ does not
include only the actual microstate $x(t)$).

Let us denote $x^{R}$ the time reversed state of $x$ (e.g. the state in which
velocities of all particles have an opposite sign) and $\alpha^{R}$ be the
time reversed state of $\alpha$ with
$\Gamma(\alpha^{R})=\\{x^{R},x\in\Gamma(\alpha)\\}$. If $P_{out}$ is
interpreted as the probability $P_{rev}$ of realizing the reversed process
$\alpha_{f}^{R}\to\alpha_{i}^{R}$ then Eq. (19) has the form

$\frac{P_{rev}}{P}=e^{-\Sigma},$ (36)

where $\Sigma=(k_{B}T)^{-1}W_{dis}$ in ${\cal H}$-representation. We thus get
the information form of fluctuation theorem Sev2008 ; SeiHafJar2021 . Its
content is, however, somewhat different from the standard fluctuation
theorems. Namely it concerns the adiabatic processes only and the
probabilities are defined with respect to the used IRF.

There are many conceptual questions concerning the presented approach. For
example, the implementation of ’information’ as a full valuable physical
quantity means a change of viewpoint concerning the meaning of some physical
quantities. Namely when some physical event (structure, configuration) is
interpreted as information about the studied system we immediately get
different values of quantities like entropy or free energy since the
information reference frame changes. This effect does not seem to play a role
in macroscopic physics where the information reference frame is usualy fixed
(given by a typical macroscopic observer). It may be important, however, in
microphysics: the role of ’observer’ (whatever it means) is nontrivial here
and transfromations between various reference frames might play an essential
role. It corresponds to recent discoveries in microscopic statistical physics
underlaying the crucial role of information, for example, in energy conversion
ParHorSag ; HorSagPar2013 ; PanYunTluPak2018 .

## Appendix

The equality Eq. (10) holds for all sufficiently small $\tau>0$. Since
$\lim_{\tau\to 0}\Delta{\cal J}=0$ we denote
$H(x_{1},x_{2},\ldots)=\Phi(\Phi_{-1}(x_{1})+\Phi_{-1}(x_{2})\ldots)$ and
study the condition $H(x_{1}+y,x_{2}+y,\ldots)=H(x_{1},x_{2},\ldots)+y$ in the
limit $y\to 0$. The derivation of this condition by $y$ gives
$\phi(\sum\Phi_{-1}(x_{i}))=\sum\phi(\Phi_{-1}(x_{i}))$ in the limit $y\to 0$,
where $\phi(Z)\equiv(\Phi^{\prime}(Z))^{-1}$. It implies that $\phi(Z)=\kappa
Z$, where $\kappa$ is a constant. After integration we get $\Phi(Z)=-\gamma\ln
Z+\kappa_{0}$, where $\gamma=-\kappa^{-1}>0$ ($\Phi$ is a decreasing function)
which implies $f(z)=e^{-\gamma^{-1}\psi(z)}e^{\gamma^{-1}\kappa_{0}}$. Hence
${\cal
I}_{\Gamma}(\Omega)=-\gamma\ln\sum_{z\in\Omega}e^{-\gamma^{-1}k_{B}^{-1}{\cal
S}_{\Gamma}(z)}$, where $\gamma$ cannot depend on $\Gamma$ to fulfill Eq. (6)
for $\Gamma_{i}\neq\Gamma_{f}$.

## Acknowledgement

The author is indebted to Ján Minár for his help and constructive discussions,
and to Philipp Strasberg for inspirational and critical comments concerning
the preparation of the manuscript. The work is supported by the New
Technologies Research Center of the West Bohemia University in Pilsen.

## References

  * (1) J. C. Maxwell _Theory of Heat_ , Appleton, London, 1871
  * (2) K. Maruyama, F. Nori, and V. Vedral, Colloquium: The physics of Maxwell’s demon and information, Rev. Mod. Phys., 81, 1 (2009)
  * (3) T. Sagawa and M. Ueda, Fluctuation Theorem with Information Exchange: Role of Correlations in Stochastic Thermodynamics, Phys. Rev. Lett. 109, 180602(1)-180602(5) (2012)
  * (4) J.M.R. Parrondo, J.M. Horiwitz, and T. Sagawa, Thermodynamics of Information, Nat. Phys. 11, 131-139 (2015)
  * (5) S. Deffner and C. Jarzynski, Information Processing and the Second Law of Thermodynamics: An Inclusive, Hamiltonian Approach, Phys. Rev. X 3, 041003 (2013)
  * (6) A. Barato and U. Seifert, Stochastic thermodynamics with information reservoirs, Phys. Rev. E 90, 042150 (2014)
  * (7) N. Shiraishi, T. Matsumoto, and T. Sagawa, Measurement-feedback Formalism meets information reservoirs, New J. Phys. 18 (2016)
  * (8) S. Toyabe, T. Sagawa, M. Ueda, E. Muneyuki, and M. Sano, Experimental demonstration of information-to-energy conversion and validation of the generalized Jarzynski equality, Nat. Phys. 6, 988-992 (2010)
  * (9) M. D. Vidrighin, O. Dahlsten, M. Barbieri, M.S. Kim, V. Vedral, and I. A. Walmsley, Photonic Maxwell’s Demon, Phys. Rev. Lett. 116, 050401 (2016)
  * (10) N. Cottet, S. Jezouin, L. Bretheau, P. Campagne-Ibarcq, Q. Ficheux, J. Anders, A. Auffeves, R. Azouit, P. Rouchon, and B. Huard, Observing a quantum Maxwell demon at work, Proc. Natl. Acad. Sci. USA 114 (29), 7561-7564 (2017)
  * (11) A. C. Barato and U. Seifert, An autonomous and reversible Maxwell’s demon, Europhys. Lett. 101, 6001 (2013)
  * (12) J.M. Horiwitz, T. Sagawa, and J.M.R. Parrondo, Imitating Chemical Motors with Optimal Information Motors, Phys. Rev. Lett. 111, 010602 (2013)
  * (13) U. Seifert, Stochastic Thermodynamics, Fluctuation Theorems and Molecular Machines, Rep. Prog. Phys. 75, 126001 (2012)
  * (14) Z. Tang, O. Sheng (Eds.) _Nanoscale Phenomena_ , Springer, New York, NY, 2008
  * (15) In the spirit of Maxwell’s gedankenexperiment Maxwell1871  (the standard expression of information via probabilities does not capture information clearly as someone’s knowledge)
  * (16) As Laplace’s demon (P.S. Laplace _Théorie analytique des probabilités_ , 2 vols., Courcier Imprimeur, Paris, 1812)
  * (17) C.E. Shannon, A Mathematical Theory of Communication, The Bell System Technical Journal, vol. XXVII, no. 3 (1948)
  * (18) E.T. Jaynes, Information theory and statistical mechanics, Phys. Rev., Vol. 106, No. 4, 15 (1957)
  * (19) L. Boltzmann, Ann. Phys. (Leipzig) 57 (1896) 773
  * (20) J.L. Lebowitz, Macroscopic laws, microscopic dynamics, time’s arrow and Boltzmann’s entropy, Physica A 194 (1993)
  * (21) S. Goldstein, J.L. Lebowitz, R. Tumulka, and N. Zanghi, Gibbs and Boltzmann Entropy in CPastawski, and assical and Quantum Mechanics, arXiv:1903.11870v2 (2019)
  * (22) H. Callen, _Thermodynamics and an Introduction to Thermostatics_ , John Willey & Sons, 1985
  * (23) E.T. Jaynes, Gibbs vs Boltzmann entropies, Am. J. Phys., Vol. 33, No. 3 (1965)
  * (24) C. Jarzynski, Single Molecule Experiments Out of Equilibrium, Nat. Phys. 7 (2011)
  * (25) C. Jarzynski, Nonequilibrium Equality for Free Energy Differences, Phys. Rev. Lett. 78. 2690 (1997)
  * (26) G.E. Crooks, Nonequilibrium Measurements of Free Energy Differences for Microscopically Reversible Markovian Systems, J. Stat. Phys. 90, 1481 (1998)
  * (27) G.E. Crooks, Path-ensemble averages in systems driven far from equilibrium, Phys. Rev. E, Vol. 61, 3 (2000)
  * (28) M. Levitt, Birth and Future of Multiscale Modeling for Macromolecular Systems (Nobel Lecture), Angewandte Chemie International Edition, 53 (2014)
  * (29) P. Strasberg and A. Winter, First and Second Law of Quantum Thermodynamics: A Consistent Derivation Based on a Microscopic Definition of Entropy, PRX Quantum 2, 030202 (2021)
  * (30) G.E. Crooks, Entropy production fluctuation theorem and the nonequilibrium work relation for free energy difference, Phys. Rev. E 60, 2721 (1999)
  * (31) E.H. Lieb and J. Yngvason, The entropy concept for non-equilibrium states, Proc. R. Soc. A 469 (2013)
  * (32) W.Rudin, _Real and Complex Analysis_ , McGraw-Hill, 1987
  * (33) C. Maes and H. Tasaki, Second law of thermodynamics for macroscopic mechanics coupled to thermodynamic degrees of freedom, Lett. Math. Phys. 79, 251 (2007)
  * (34) L. Peliti, On the work-Hamiltonian connection in manipulated systems, J. Stat. Mech. P05002 (2008)
  * (35) M. Holeček, Work as a memory record, Phys. Rev. E 99, 062130 (2019)
  * (36) E.H. Lieb and J. Yngvason, The physics and mathematics of the second law of thermodynamics, Phys. Rep. 310, 1 (1999)
  * (37) R. Penrose, _The road to reality: a complete guide to the laws of the universe_ , Alfred A. Knoff, New York (2005)
  * (38) D.Z. Albert, _Time and Chance_ , Cambridge, MA: Harvard University Press (2000)
  * (39) J. Earman, The ”Past Hypothesis”: Not even false, Studies in History and Philosophy of Modern Physics 37 (2006)
  * (40) C. Callender, Thermodynamic Asymmetry in Time, The Stanford Encyclopedia of Philosophy (2021)
  * (41) C. Jarzynski, Nonequilibrium work theorem for a system strongly coupled to a thermal environment, J. Stat. Mech.: Theor. Exp. (2004) P09005
  * (42) R. Kawai, J.M.R. Parrondo, and C.Van den Broeck, Dissipation: The Phase-Space Perspective, Phys. Rev. Lett. 98, 080602 (2007)
  * (43) W. Muschik, Discrete systems in thermal physics and engineering: a glance from non-equilibrium thermodynamics, Continuum Mech. Thermodyn. 33 (2021)
  * (44) T. Sagawa and M. Ueda, Minimal Energy Cost for Thermodynamic Information Processing: Measurement and Information Erasure, Phys. Rev. Lett. 102, 250602 (2009)
  * (45) T. Sagawa and M. Ueda, Generalized Jarzynski Equality under Nonequilibrium Feedback Control, Phys. Rev. Lett. 104, 090602 (2010)
  * (46) T. Sagawa and M. Ueda, Role of mutual information in Entropy Production under Information Exchange, New J. Phys. 15, 125012 (2013)
  * (47) C. Rovelli, _Quantum Gravity_ , Cambridge University Press, 2004
  * (48) M. Holeček, Information break of time symmetry in the macroscopic limit, arXiv 2202.11576 (2022)
  * (49) E.M. Sevick, R. Prabhakar, S.R. Williams, and D.J. Searles, Fluctuation Theorems, Annual Rev. of Phys. Chem., 59:603-633 (2008)
  * (50) A. Seif, M. Hafezi, and C. Jarzynski, Machine learning and thermodynamic arrow of time, Nat. Phys., Vol 17 (2021)
  * (51) G. Paneru, D.Y. Lee, T. Tlusty, and H.K. Pak, Lossless Brownian Information Engine, Phys. Rev. Lett. 120, 020601 (2018)