# The foundations of statistical physics: entropy, irreversibility, and
inference

Jonathan Pachter  Laufer Center for Physical and Quantitative Biology, Stony
Brook University, NY, USA  Department of Physics and Astronomy, Stony Brook
University, NY, USA  Ying-Jen Yang  Laufer Center for Physical and
Quantitative Biology, Stony Brook University, NY, USA  Ken Dill  Laufer Center
for Physical and Quantitative Biology, Stony Brook University, NY, USA
Department of Physics and Astronomy, Stony Brook University, NY, USA  e-mail:
<EMAIL_ADDRESS>
###### Abstract

Statistical physics aims to describe properties of macroscale systems in terms
of distributions of their microscale agents. Its central tool is the
maximization of entropy, a variational principle. We review the history of
this principle, first considered as a law of nature, more recently as a
procedure for inference in model-making. And while equilibria (EQ) have long
been grounded in the principle of Maximum Entropy (MaxEnt), until recently no
equally foundational generative principle has been known for non-equilibria
(NEQ). We review evidence that the variational principle for NEQ is Maximum
Caliber. It entails maximizing path entropies, not state entropies. We also
describe the role of entropy in characterizing irreversibility, and describe
the relationship between MaxCal and other prominent approaches to NEQ physics,
including Stochastic Thermodynamics (ST), Large Deviations Theory (LDT),
Macroscopic Fluctuation Theory (MFT), and non-extensive entropies.

## Introduction

Statistical physics emerged in the mid-1800’s as a way to explain and predict
macroscopic material phenomena from the physics of their constituent atoms,
molecules, or other particles. Initially, it focused primarily on states of
equilibrium (EQ) and utilized the principle of Maximum Entropy (MaxEnt), which
is equivalent to some formulations of the Second Law of Thermodynamics. Its
methods and practice are well-known[1, 2, 3, 4, 5]. But two foundational
questions have been subject to changing answers over time. First, what is the
reason for using the Boltzmann-Gibbs (BG) form of entropy in statistical
physics, and why is it maximized? Is it a law of physics or a law of
inference? Second, what general procedure can predict the behaviors of non-
equilibrium (NEQ) systems? Driving an interest in these foundational matters
is increasing applications of MaxEnt to biology[6], ecology[7], economics[8],
active matter[9], and artificial intelligence and neural networks[10]. In
contrast to previous reviews of the mechanics and applications of EQ (Maximum
Entropy) and NEQ (Maximum Caliber)[11, 12], here we review the history of
their foundations and frame the context of related sub-fields.

We first note that two disparate perspectives are used in statistical physics.
From its earliest days, it was framed in terms of Hamilton’s equations of the
positions and momenta of collisional particles, their conservation of energy
through the Liouville equation, and puzzles of irreversibility and
disorder[13, 14, 15]. The second perspective – which is our main focus here –
is the language of probabilities, which is the basis for most present-day
problem-solving. The probabilistic basis extends the reach of statistical
physics to problems and processes well beyond collisional particles as
functions of temperature and pressure. Newtonian mechanics is an unnecessarily
limiting starting point for developing the most general principles of model-
making, especially for many-body living systems whose constituents are
themselves high-dimensional[16]. Furthermore, the probability framework gives
tools that are practical and simple to apply without sacrificing rigor.

## A Tale of Two Entropies: The Foundations of Equilibrium Statistical Physics

At the heart of statistical physics is a remarkable equality between two
seemingly unrelated quantities – a macroscopic physical property of matter and
a mathematical property of probability distributions – which are both now
called entropy:

$\Delta S_{\mathrm{Clausius}}=\Delta S_{\mathrm{BG}}^{*}$ (1)

On the left side of Eq. (1) is the change in Clausius entropy, a macroscopic
physical quantity defined by Rudolph Clausius in 1855, encapsulating certain
transformations between thermodynamic equilibria. On the right side is the
change in maximized Boltzmann-Gibbs (BG) entropy, a mathematical property of a
probability distribution over microscopic degrees of freedom of a system. In
the following sections, we define these two entropies and review the history
and basis for the principle embodied in Eq. (1).

### The Clausius Entropy describes macroscopic processes.

At the foundation of thermodynamics is an inequality due to Clausius,
accompanied by a definition of reversible, quasi-equilibrium processes as a
very special limiting case of dynamics. Clausius’ theorem states that the
integral of the infinitesimal heat a system dissipates to its environment,
divided by temperature, is always non-negative along a cyclic transformation,
and is only zero for a reversible cyclic process – a result that relates to
both Kelvin’s and Carnot’s more qualitative statements of the Second Law of
Thermodynamics[1]:

$\oint_{\mathrm{general}}\frac{\delta
Q}{T}\geq\oint_{\mathrm{reversible}}\frac{\delta Q}{T}=0.$ (2)

This result can be used to define the change in Clausius (or thermodynamic)
entropy as this heat integral along any reversible trajectory, which must be
less than or equal to the same integral for a general trajectory connecting
the same two thermal equilibrium states, due to the theorem above:

$-\Delta S_{\mathrm{Clausius}}=\int_{\mathrm{reversible}}\frac{\delta
Q}{T}\leq\int_{\mathrm{general}}\frac{\delta Q}{T}.$ (3)

While this property of $\Delta S_{\mathrm{Clausius}}$ can be rearranged to
explain heat flow from hotter to colder systems, it contains no information
about time dependence, such as how long two bodies at different temperatures
would take to equilibrate with each other, and it says nothing about the
underlying microscopic constituents of matter.

### Statistical physics arose from the need for probability distributions.

The need for probability distributions in connecting macroscale to microscale
behavior was already evident in the 1850’s when James Clerk Maxwell developed
the Kinetic Theory of Gases. Whereas the mean velocity of particles in a gas
at equilibrium is zero (and therefore uninformative), the variance of velocity
was found to be related to the temperature $T$ through $\langle
v^{2}\rangle=3kT/2m$ for an ideal gas. Where there is a variance, there is
uncertainty, and thus a need for distributions over microscopic agents to
explain some macroscopic behaviors.

Throughout its history, statistical physics has been enmeshed within a broader
controversy about the nature of probabilities themselves. Probabilities are
mathematically defined as any fractional quantities, associated with events,
that can be combined into other Boolean collections of independent and
mutually exclusive events through the addition and multiplication logical
rules of probability[17]. The controversy is in whether probabilities can only
be descriptions of frequencies of replicatable events, as when a dice is
rolled multiple times, or can more broadly also describe inferences about non-
repeatable events, i.e. _frequentist_ versus _Bayesian_. While the methodology
of statistical physics has been stable since the earliest days, how that
methodology is justified on principle has changed – initially primarily as
frequentist, with the Bayesian inferential gaining more traction over the
decades.

## Gibbs’ ensembles: Probabilities interpreted as frequencies

Following preliminary work by L. E. Boltzmann, J. C. Maxwell, and H.W.
Watson[18], J. Willard Gibbs constructed a framework of “ensembles” to
encapsulate statistical physics theory, which employed frequencies, the
prevailing language for probabilities at the time. The concept of an ensemble
is simple. Consider a system with $M$ possible microstates, e.g. $M$ possible
ways of arranging its microscopic constituents. Now, imagine we have $N$
identical copies of our system, each subject to the exact same
macroscopic/thermodynamic external conditions. We can count the number of
times $n_{m}$ we see microstate $m$ within this ensemble of copies, giving us
a set of occurrence numbers for all the different $m$. The number of different
ways we could observe the same set of occurrences $\\{n_{m}\ |\
m=1,2,...,M\\}$ with $\sum_{m}n_{m}=N$ is a multinomial:

$W={N!}\bigg{/}\left(\prod\limits_{m=1}^{M}n_{m}!\right).$ (4)

Defining the frequency/proportion of each outcome as $f_{m}=n_{m}/N$, we can
calculate the original Boltzmann definition of entropy in terms of
multiplicity $S=\ln W/N$. Taking the limit of infinitely large ensemble and
using Stirling’s approximation for factorials in the limit
$N\rightarrow\infty$, we get the Boltzmann-Gibbs (BG) entropy:

$S_{\mathrm{BG}}=\lim_{N\rightarrow\infty}\frac{\ln
W}{N}=-\sum\limits_{m=1}^{M}f_{m}\ln f_{m}.$ (5)

In the standard treatment, the Boltzmann distribution for a system in a heat
bath is derived by maximizing the BG entropy. As detailed below in our
discussion on large deviation theory, the set of frequencies that maximize
$S_{\mathrm{BG}}$ corresponds to what is most likely to be observed (see
below), subject to constraints on the average energy and normalization of the
frequencies, which are imposed using Lagrange multipliers:

$S_{\mathrm{BG}}=-\sum_{m=1}^{M}f_{m}\ln
f_{m}-\beta\left(\sum_{j}f_{m}\varepsilon_{m}-\langle\varepsilon\rangle\right)+\alpha\left(\sum_{m=1}^{M}f_{m}-1\right).$
(6)

The maximized value $S_{\mathrm{BG}}^{*}$ is achieved for the Boltzmann
distribution:

$f_{m}^{*}=\frac{e^{-\beta\varepsilon_{m}}}{Z}$ (7)

where

$Z=\sum_{m=1}^{M}e^{-\beta\varepsilon_{m}}.$ (8)

### Large Deviation Theory and related problems of data inference

It is useful here to introduce Large Deviations Theory (LDT). One reason is
that while means and variances often summarize key aspects of probability
distributions, the rare events found in the tails of distributions are often
of great interest as well. While LDT has applications in statistical physics
[19], it is particularly relevant in what could be called “data-rich”
situations, in which events are replicable and Gibbs’ ensemble approach is
imitated in real life, leading to an “idealized infinite dataset” [20, 16].

Another usage of LDT is to make the connection, noted above, between maximum
multiplicity and most probable observation. Given a prior distribution
$\\{q_{m}\\}$, which in textbook treatment was chosen to be uniform according
to the _principle of equal a priori probabilities_[5] , the probability of
observing a particular set of frequencies $\\{f_{m}\\}$ in the Gibbs ensemble
with respect to the prior has the following asymptotic form[21]:

$\mathrm{Prob}(f)=\exp\left[-N\cdot D(f||q)+o(N)\right]=\exp\left[N\cdot
S_{\mathrm{BG}}(f||q)+o(N)\right].$ (9)

where $o(N)$ are terms that are much smaller than $N$ in the limit of large
$N$. When $N\rightarrow\infty$, this probability distribution is exponentially
dominated by the frequency $f$ that is closest to the prior $q$, as measured
by the KL divergence. One can perform a quadratic approximation of
$S_{\mathrm{BG}}$ near $q$ for small, linear deviations, leading to a Gaussian
probability[22], as dictated by the central limit theorem. But Eq. (9)
contains information for $f$ further away from $q$, hence the name large
deviations. When newly acquired data constrains the possible frequency $f$,
the most-probable frequency $f^{*}$ under the constraints is the one that
uniquely maximizes the entropy $S_{\mathrm{BG}}(f||q)$. In such a “data-rich”
situation, entropy maximization in Gibbs’ framework is equivalent to Bayesian
model updating[20, 16]. This LDT formulation is useful in generalizing Gibbs’
framework to infer Markov processes from correlated data [23, 24, 16].

### The equality of the two entropies.

We return to Eq. (1), the equivalence between two entropies. Substituting the
posterior $f_{m}^{*}$ of Eq. (7) into Eq. (6) for the BG entropy yields the
maximized entropy $S_{\mathrm{BG}}^{*}=\beta\langle\varepsilon\rangle+\ln Z$.
Consider a process in which we transform the system by slightly perturbing the
energy levels by $\delta\varepsilon_{j}$. We can compute the change in $\ln Z$
as

$\delta\ln Z=\sum_{m=1}^{M}\frac{1}{Z}\frac{\partial Z}{\partial\epsilon_{m}}\
\delta\varepsilon_{m}=-\beta\sum_{m=1}^{M}\frac{1}{Z}q_{m}e^{-\beta\varepsilon_{m}}\
\delta\varepsilon_{m}=-\beta\sum_{m=1}^{M}f_{m}^{*}\
\delta\varepsilon_{m}=-\beta\delta W$ (10)

where $\delta W$ is the work performed on the system in a reversible process.
Thus the change in maximized BG entropy is

$\delta S_{\mathrm{BG}}^{*}=\beta\delta\langle\varepsilon\rangle+\delta\ln
Z=\beta\delta(\langle\varepsilon\rangle-W)=-\beta\delta Q=\delta
S_{\mathrm{Clausius}}$ (11)

where $\delta Q$ is the heat dissipated in this reversible process, and we
equate the Lagrange multiplier $\beta$ with the inverse temperature
$1/(k_{B}T).$

The ensemble approach has been useful for predicting macroscopic properties of
microscopic models. However, its conceptualization of probabilities as
frequencies limits its applicability beyond replicable events. The ensemble
approach envisions a set of infinite copies of a system, but in reality we
often have only one system. The replicates could arise from different
snapshots of a system spaced out in time. This conception, however, requires
two companion assumptions which restrict its generality: first, the ergodic
hypothesis, which equates time averages with ensemble averages, and second,
the independence of replicates due to chaotic interactions or random
collisions. These limitations can be circumvented by framing statistical
physics instead as a matter of drawing inferences in probabilistic modeling.

## Entropy maximization: consistent inference in model-making

In the inference-based perspective, the Maximum Entropy principle (MaxEnt) is
the unique procedure to generate probability distributions that are logically
consistent and match known externally supplied information. In this view,
probabilities are not limited to being frequencies of repeatable events – they
can characterize uncertainties of all kinds. This is the Bayesian school of
thought, in which we can discuss the probability of a nuclear disaster, or
life on Mars, or the chance of snow at some particular time and date in the
future, even though none of these are repeatable events. Beginning with the
1957 Physical Review paper by E.T. Jaynes[25], the foundations of statistical
physics have been re-framed in terms of Bayesian inference in order to avoid
the need for replicates of physical systems.

Jaynes saw statistical physics as a procedure for inference and model-making
rather than as a law of nature. He advised asserting only the information that
you have (for example, average energy $\langle\varepsilon\rangle$ in the
textbook[1, 4] case described above), and asserting maximal ignorance
otherwise. Jaynes’ arguments removed the dependence on replicability of the
event, but some physicists objected that statistical physics should not be
tied to an observer’s state of knowledge[26].

This conceptual problem was resolved in the seminal work of Shore and Johnson
in 1980[27], in which they fully axiomatized the inference process. They
showed that, given a prior distribution $q_{m}$ representing some initial
information about the system, and given new information constraining the
possible forms for the posterior distribution $p_{m}$, there is only one
posterior distribution $p_{m}^{*}$ obeying consistency with the logical
structure of probabilities which introduces no unwarranted bias; the unique
posterior $p_{m}^{*}$ is that which maximizes the relative BG entropy of the
posterior with respect to the prior, constrained by the relevant information
(alternatively, minimizes the Kullback-Leibler (KL) divergence $D(p||q)$ of
the posterior with respect to the prior):

$S_{\mathrm{BG}}=-\sum\limits_{m}p_{m}\ln\left(\frac{p_{m}}{q_{m}}\right)=-D(p||q).$
(12)

Although $S_{\mathrm{BG}}$ can be computed for all possible posteriors with
respect to the given prior, it is only the maximum value $S_{\mathrm{BG}}^{*}$
that selects the proper posterior $p_{m}^{*}$. Up to a constant, this relative
$S_{\mathrm{BG}}$ reduces to those described above in Eqns. (5)-(6) when the
prior is uniform; the common assumption of uniform prior, known as the
principle of equal a priori probabilities, is sometimes justified by symmetry
arguments. For example, if the microstate refers to the position of a
particle, and the system has isotropic symmetry, the starting point of
knowledge is that no position should be different than any other position, so
all positions are equally likely.

Figure 1: The recipe of statistical physics: A model-maker provides the
“physics”: the relevant state space, prior distribution, and constraints on
possible posteriors; these serve as inputs for the Maximum Entropy (MaxEnt)
inferential procedure – “the statistics”, which generates the best possible
probability distribution given those inputs

In this inference-based perspective (see Fig. 1), statistical physics entails:
(i) a user-defined state space, (ii) a user-defined prior distribution, (iii)
appropriate user-defined constraints, such as known averages from external
data, and (iv) the MaxEnt procedure, predicting the posterior distribution
that faithfully represents the premises and priors without unwarranted bias.
In this view, Maximum Entropy is not a law of physics; rather, Maximum Entropy
is a method of drawing inferences. A user asserts a base model structure (i.e.
the state space, prior, and constraints – the input ingredients needed for the
MaxEnt recipe) and computes a posterior distribution. If a predicted
distribution is found to be inconsistent with experiments, the burden is on
the user to find a better model.

### “Non-extensive” statistical physics.

MaxEnt typically yields exponential distributions. However, many systems,
ubiquitous throughout nature, display non-exponential distributions, such as
power-law tails[28]. What approach would predict non-exponential
distributions? A field called “non-extensive” statistical physics [29] derives
non-exponential distributions using non-BG entropies, such as Tsallis or Renyi
entropies. However, non-BG entropies introduce unwarranted bias[30, 31, 32].
The key axiom in Shore and Johnson that prevents that is what Jizba and Korbel
called strong-system independence[33] and what Caticha summarized as the
combination of subset independence and minimal updating[34]: When two systems
are a priori assumed to be independent and new information does not reveal
whether the systems are independent or not, the independence in the prior
should be maintained. Proper inference without unwarranted bias can only be
done with the BG entropy and its maximization.

How can non-exponential distributions be explained on the basis of the BG
entropy? This readily occurs for certain types of correlations in the imposed
constraints. For example, in social and financial situations, an “economy of
scale” model gives non-exponential distributions with power-law tails [35].
Exponentials arise when the function being constrained, which in classic
statistical physics systems is the energy $\varepsilon_{m}$ is any function
$f(m)$ of only the state index $m$. However, if the energy or energy-like
function $\varepsilon_{m}=g(m,\varepsilon_{n})$ is a function both of state
index $j$, and also of the energy values of other states[35], power-law tails
emerge naturally from the maximization of BG entropy. Other methods yielding
non-exponentials from BG entropy include superstatistics (which itself is
simply a nested MaxEnt procedure) [36, 37, 38, 39], non-ideal reservoirs [40,
41, 42], and others [43, 44, 45].

## Beyond equilibria: defining irreversibility

Figure 2:  Four classes of dynamical change: “State” here in the vertical axis
means the macroscopic/thermodynamic state of the system. (a) Starting from one
equilibrium, ending at another equilibrium through slow reversible processes.
Clausius’ equality applies. (b) From one near-equilibrium to another near-
equilibrium through an arbitrary near-equilibrium path, the subject of
stochastic thermodynamics. Shaded area indicates near-equilibrium of a system
embedded in equilibrium baths. (c) From an initial equilibrium to an arbitrary
end state through an arbitrary non-equilibrium path, the subject of Clausius
inequality and Jarzynski equality. The generally non-equilibrium end state
could be related to an equilibrium state through a relaxation process where no
work is performed (indicated by the dashed-line arrow), a special case that is
the subject of Boltzmann’s H Theorem. (d) Arbitrary non-equilibrium process,
the subject of Maximum Caliber.

We now turn to non-equilibrium statistical physics. Fig 2 divides dynamical
processes into four classes of behaviors. Described in more detail below, they
are briefly: (Fig 2a) Largely as described in the Clausius entropy section
above – begin in initial equilibrium (EQ) state and end in final EQ state,
proceeding through an idealized extremely-slow, quasi-equilibrium pathway.
(Fig 2b) Begin in an arbitrary near EQ state and end in an arbitrary near EQ
state through an arbitrary near EQ path – this is the subject of Stochastic
Thermodynamics (ST) and Macroscopic fluctuation theory; see below. (Fig 2c)
Begin in initial EQ state, take an arbitrary path to an arbitrary point, then
relax to a final EQ state under fixed external conditions – this is the
purview of the Clausius inequality and Jarzynski relation[46]. The final
relaxation segment is the realm of the Boltzmann’s H Theorem. (Fig 2d) Begin
in an arbitrary state and end in arbitrary state with arbitrary connecting
pathway. This is the most general class, and it is the realm of the principle
of Maximum Caliber.

We note that “near-equilibrium” in the classification above (and in our later
discussions) means something different from what one might expect. Consider a
total super-system, consisting of a system of interest that can hypothetically
be arbitrarily far from EQ, coupled to bath(s); if the bath(s) remain
perpetually in EQ, then we refer to the super-system as “near EQ”, because a
system that is driven strongly enough will eventually perturb its environment
noticeably out of EQ[42]. Note that this is NOT the same as what “near-
equilibrium” means in stochastic thermodynamics (ST), where the term is
reserved for processes in which the state of the system is always close to the
equilibrium distribution. In ST, when the system is said to be near-
equilibrium, linear response theory applies, whereas linear response theory
breaks down when the system is far from equilibrium in ST.

A central question: Can the various forms of irreversibility in nature be
quantified by some entropy-like variational principle? We review below
successes in modeling in the four classes of processes above. We note that
local-equilibrium – _a.k.a._ local-detailed-balance – approximations, and the
assumption of infinite equilibrium heat baths, can limit the generality of
most of the methods. Maximum Caliber, however – a variational principle that
maximizes the path entropy – is free from such limitations.

### Boltzmann’s H Theorem and Irreversible Relaxation

In seeking connections between microscopic particles and macroscopic
thermodynamics, Boltzmann considered the following entropy-like functional of
the distribution density $f(x,v,t)$ for particles in a gas to have position
$x$ and velocity $v$ at time $t$:

$H=-\int\int\mathrm{d}v\mathrm{d}x\ f(x,v,t)\ln f(x,v,t)$ (13)

which reduces to the classic Boltzmann entropy when all microstates are
equally probable.

Boltzmann used this quantity to state his $H$-theorem: namely, that $H$
increases in time until the gas equilibrates, at which point $H$ attains its
maximum value equal to the equilibrium entropy; see Fig (2c), and see Huang’s
textbook[1] for more detailed information. Conceptually, this is an early
example of what mathematicians now call a _Lyapunov function_ – a quantity
whose changes in time are monotonic for a particular dynamics and thus
characterize the irreversible relaxation to steady state.

For stochastic dynamics or dissipative deterministic dynamics, Lyapunov
functions have proven useful for, among other things, characterizing the
direction of time. For Markov dynamics with a steady-state, a well-known
Lyapunov function is the relative entropy[47], i.e. the KL divergence

$D(t)=D(p(t)||\pi)=\sum_{i}p_{i}(t)\log\frac{p_{i}(t)}{\pi_{i}}$ (14)

where $p_{i}(t)$ is the probability distribution at time $t$ and $\pi_{i}$ is
the steady-state probability distribution. For Markov dynamics with an unique
steady state, $\mathrm{d}D(t)/\mathrm{d}t\leq 0$, which characterizes the
stability of the steady state[48] and also bounds the maximum extractable work
in a transition between arbitrary distributions[49]. Studies of the
geometrical origin of these results grew into the field of _information
geometry_[50], which continues producing new insights, such as an upper limit
on the relaxation speed of an observable related to its fluctuations and the
local curvature of $D(t)$[51, 52, 53].

For dissipative deterministic dynamics, a Lyapunov function known as the “
_quasi potential_ ” can be derived by considering the deterministic dynamics
as the zero-noise limit of a stochastic dynamics[54]. This Lyapunov function
is useful in analyzing limit cycles[55], Kramers’ theory of barrier-crossing
in chemical reaction kinetics[54, 56], and nonequilibrium dynamics in living
systems in general[57, 58, 59]. More recently, the quasi potential has been
given a stochastic thermodynamics interpretation[60, 61], making it
conceptually unified with the stochastic Lyapunov function described above.

### Stochastic Thermodynamics (ST) computes properties of irreversibility.

The field of Stochastic Thermodynamics (ST)[46, 62, 63] – which inherits key
concepts from chemical thermodynamics[64] and from Schnakenberg’s chemical-
reaction network physics[48] – has emerged in the last few decades,
generalizing Clausius’ theorem in order to more broadly quantify
irreversibility; see Fig (2b) (and (2c) for Jarzynski’s equality of
Hamiltonian systems[65, 46]).

Note that Eq. (3) from Clausius admits a separation for any process starting
and ending in equilibrium into two components,

$\int_{\mathrm{general}}\frac{\delta Q}{T}=-\Delta
S_{\mathrm{Clausius}}+I_{\mathrm{general}}$ (15)

where $-\Delta S_{\mathrm{Clausius}}$ is the heat dissipation integral
computed along a reversible trajectory. So the total heat dissipation integral
of an arbitrary process can be parsed into two contributions: the baseline
dissipation one would get for a reversible process, and the remaining term
$I_{\mathrm{general}}\geq 0$ measuring the extra heat dissipation due to
irreversibility, i.e. it quantifies the degree of deviation from
reversibility.

The original Clausius picture is limited in two ways: (1) The starting and
ending states must be equilibria, so it cannot apply to arbitrary dynamical
processes, and (2) a well-defined temperature must exist at all points along
the transformation, which is equivalent to assuming an idealized thermal bath.
The goal of ST is to analyze the irreversibility of general processes beyond
these limitations (although the second limitation still applies to the
prominent interpretation of many ST results).

At the heart of ST is the quantity $\delta A_{ij}$ called _affinity_ ,
associated with transitions between states $i$ and $j$. In Markov processes
described by transition rates $k_{ij}$, the _affinity_ is defined as

$\delta A_{ij}=\log\left(\frac{k_{ij}}{k_{ji}}\right).$ (16)

The necessary and sufficient condition for a Markov process to be detailed-
balanced is for the affinity to be zero for any cycles. This is the Markov
process analogue of Clausius’ theorem, and is also known as the Kolmogorov
criterion in mathematics[66, 67]. In further analogy with the Clausius
picture, the affinity can be decomposed into an equilibrium-like landscape-
descending term and an additional term that quantifies the breakdown of
detailed balance,

$\delta
A_{ij}=\log\left(\frac{k_{ij}}{k_{ji}}\right)=-\Delta(-\log\pi)+\log\left(\frac{\pi_{i}k_{ij}}{\pi_{j}k_{ji}}\right)$
(17)

where $\pi_{i}$ is the steady-state distribution. The steady-state average of
Eq. (17), denoted by angle brackets $\langle\cdot\rangle$, gives the Markov
process generalization of Eq. (15):

$\langle\delta
A\rangle=\sum_{i<j}J_{ij}\log\left(\frac{k_{ij}}{k_{ji}}\right)=-\Delta\langle-\log\pi\rangle+\underset{\geq
0}{\underbrace{\sum_{i<j}J_{ij}\log\left(\frac{\pi_{i}k_{ij}}{\pi_{j}k_{ji}}\right)}}$
(18)

where $J_{ij}=\pi_{i}k_{ij}-\pi_{j}k_{ji}$ is the steady-state probability
flux.

The affinity decomposition of Eq. (17) exemplifies the key idea of ST,
inherited from Clausius: dynamical properties can be split into a detailed-
balance component and an additional component related to breaking of detailed-
balance. This type of decomposition appears in many places, with arguably the
best-known result[68, 69] parsing the total irreversibility in nonequilibrium
Markov processes $I_{\mathrm{tot}}(t)$ into two contributions: (i) the
deviation from its steady state (non-stationarity), quantified by
$-\mathrm{d}D(t)/\mathrm{d}t\geq 0$ as defined in Eq. (14), and (ii) the
additional irreversibility due to detailed-balance-breakdown

$I_{\mathrm{DBB}}(t)=\sum_{i,j}p_{i}(t)k_{ij}\log\frac{\pi_{i}k_{ij}}{\pi_{j}k_{ij}}\geq
0$ (19)

The total irreversibility rate in ST is the KL divergence of the probability
of a two-state trajectory with respect to its time reversal:

$I_{\mathrm{tot}}(t)=\sum_{i,j}p_{i}(t)k_{ij}\log\frac{p_{i}(t)k_{ij}}{p_{j}(t)k_{ji}}=-\frac{\mathrm{d}D}{\mathrm{d}t}(t)+I_{\mathrm{DBB}}(t)$
(20)

For detailed-balanced systems, irreversibility comes solely from the
relaxation to the steady state, characterized by the monotonic behavior of the
first term $-\mathrm{d}D(t)/\mathrm{d}t$. If, however, the system has broken
detailed-balance, it has non-zero probability flux even at steady-state, which
produces a non-zero $I_{\mathrm{DBB}}$; in other words, this second term
captures the irreversibility due to an underlying non-equilibrium steady-
state.

Not only the total irreversibility, but also the two components – the non-
stationarity and the detailed-balance-breakdown – can be expressed as KL
divergences of path probabilities for forward processes with respect to the
probabilities of reversed processes[70, 68, 69, 71]. These path-probabilistic
expressions, which can be considered as the mathematical definitions of these
notions of time irreversibility, lead to a zoo of fluctuation relations and
uncertainty relations for ST of Markov processes[70, 46, 71] and help identify
the domain of validity for the various forms of fluctuation relations[71].

The same decomposition technique proves useful in analyzing other quantities.
For example, a generalized Einstein relation exists[72] in which the detailed-
balanced component of the covariance between the system’s state and its rate-
of-change is related to fluctuations. Another example is the velocity and flux
decomposition for diffusion processes [73, 74, 75, 61] and its zero-noise
limit[54, 61]. Velocity and flux in these systems are decomposed into
gradient-descending parts (the detailed-balance component) and a complementary
cyclic part embodying the breaking of detailed-balance. This is actually the
previously mentioned method for finding a Lyapunov landscape function[54, 60,
61], which exemplifies the fundamental insufficiency of describing a non-
detailed-balanced system solely by its scalar landscape function.

### The Local Detailed Balance Assumption in Stochastic Thermodynamics

A key ingredient that ST uses to provide physical meaning to mathematical
results is a condition known as Local Detailed balance (LDB); this is an
assumption that all transitions of a system occur through exchanges with
perfect thermodynamic reservoirs, which are infinite in size and always in
equilibrium (though there exists some work in ST regarding NEQ baths[76]). The
system may be in contact with multiple distinct reservoirs[77], but exchanges
with each reservoir are assumed to occur independently of each other, so that
each transition of the system can be associated with an exchange of conserved
quantity with exactly one bath. Then, for example, we can equate the affinity
$\delta A_{ij}$ for heat bath-induced transitions with the physical heat
dissipation divided by temperature $\delta Q/T$, where $T$ is the temperature
of the heat bath associated with that particular transition, and similarly for
exchanges of other quantities[78, 79].

On one hand, LDB only enters when we wish to understand affinity in terms of
heat and temperature[67] or other physical quantities. Many mathematical
results in ST, such as Eq. (17), speed limit relations[52, 53], fluctuation
relations[70, 46, 71], uncertainty relations[80, 81], and others remain valid
and useful for the analysis of Markov stochastic dynamics in general, even if
LDB does not hold. On the other hand, we must be aware of the limitations of
applicability of LDB: we[42] have previously discussed the limits of the
perfect bath assumption and how it essentially constrains the system to not be
driven too strongly away from EQ; Falasco and Esposito [82] show that LDB
holds only in the limit of weak driving and/or weak noise; Hartich and Godec
[83] show that LDB might not hold in a coarse-grained system even when there
is clear time-scale separation; and Wasnik [84] disproves the generality of
the decomposition of transition rates due to multiple reservoirs into the sum
of transition rates due to each reservoir.

Figure 3: Different types of entropy, their uses, and the connections between
them

When LDB holds, the irreversibility $I_{\mathrm{tot}}(t)$ can be written as
the change in BG entropy of the system plus the change in Clausius entropy of
the environment, which lead the ST community to call the total irreversibility
“Entropy Production”. However, this terminology can lead to confusion, since
it conflates thermodynamic and inferential entropy, which are only equivalent
for quasi-static, reversible processes, as we described in a previous section.
Furthermore, inferential entropy is not a tangible quantity that can be
produced during a process, with a measurable difference in quantity from the
beginning to the end. Since the usefulness of $I_{\mathrm{tot}}$ goes beyond
situations where LDB is applicable, we recommend exclusively using the term
“irreversibility”. See Fig. 3 for further clarification of the distinct
notions of entropy and their uses.

### Jarzynski Equality

After work by Hunter and Reinhardt[85, 86] relating to computational methods,
Jarzynski independently showed how to generalize Clausius’ inequality of work
$W$ and free energy $F$ into an equality[87]:

$\langle e^{-W/k_{B}T}\rangle=e^{-\Delta F/k_{B}T}.$ (21)

There are technically two versions of the Jarzynski equality (JE): the
original, as derived in 1997, applies to Hamiltonian dynamics for a system-
plus-bath super-system that starts in equilibrium and gets driven arbitrarily
away from EQ[46] before relaxing back to equilibrium once the external driving
is completed – this is depicted in Fig. (2c). This Hamiltonian formulation of
the JE does not need the LDB assumptions (although it does need another
external perfect heat bath described by temperature $T$ with which the total
super-system can equilibrate), and it can be generalized to strongly-coupled
systems[88, 89, 90].

For Markov dynamics as typically employed in ST, an equality of the same form
as the original JE can be derived[91]. As with many other results in ST, while
the mathematical statement is general[71], the understanding of Markov
transition probabilities in terms of free energy change and work requires the
LDB assumption. Hence, as with ST, the JE of Markov dynamics belongs in the
classification of Fig. (2b), where the typical physical interpretations are
only valid for systems embedded in idealized equilibrium baths. Similarly, for
other fluctuation relations[92], the irreversibilities associated with them
can lose their connections to heat dissipation when the bath itself is driven
out of equilibrium.

### Macroscopic Fluctuation Theory

The field of Macroscopic Fluctuation Theory (MFT)[93] uses LDT to describe the
non-trivial asymptotic behaviors and time irreversibility of stochastic
systems in the hydrodynamic limit, in which the particles are so numerous that
we can define continuous density and flow. This extends textbook hydrodynamics
to the realm of fluctuations of density and flow in physical space-time. Since
MFT also assumes a fast equilibrating bath for its physical interpretations,
it faces the same LDB limitations as ST, and therefore can be put in to the
same silo as ST in Fig. (2b). Both ST and MFT bring us powerful mathematical
tools for analyzing stochastic processes and their irreversibilities, with
results framed in terms of entropy-like quantities that provide direct
physical insight in certain circumstances. Yet, a generative principle for
building dynamical models based on prior assumptions and data – like what
entropic inference has done for equilibrium physics and beyond – seems to be
missing in these approaches to non-equilibrium physics.

## Towards a generative statistical physics of non-equilibria

Equilibrium statistical physics has stood alone on a pedestal for more than a
century. The rigorous foundations of equilibrium statistical physics, combined
with its breadth of applicability and generative power have – until recently –
been unequalled by any treatment for non-equilibrium. This was well
articulated by Touchette[19]:

> There is no general principle whereby one can calculate the distribution of
> the system’s states from the sole knowledge of the system’s invariants or
> external constraints imposed on the system … Such a general principle is
> precisely what a statistical-mechanical ensemble is, and what is missing
> from the theory of nonequilibrium systems.

Said differently, Touchette’s aspiration is for a generative principle. Yes,
NEQ physics has specific models – of random flights, Langevin and Fokker-
Planck diffusion, and others, and useful mathematical results for their
analysis, many in ST, LDT, and MFT. But, as indicated by Fig 2, NEQ modeling
has largely been limited to starting and ending points of processes that are
at or near equilibrium and they require contact with a perfect bath. Such
theories do not derive their probability distribution functions (i.e. rates
and routes for dynamics) from a model and data, as MaxEnt does for EQ. Rather
these approaches assume distributions, and in particular, they assume
equilibrium distributions. Touchette’s aspiration is for some enveloping
principle for making models that can derive NEQ dynamical distribution from
the model and data. We argue below that: (i) the problems referred to in
Touchette’s remark largely resulted from efforts that were too closely tied to
EQ principles, and (ii) the principle of Maximum Caliber appears to fill this
gap.

###  Entropy production rate is not the basis for NEQ variational principle.

Historically, there were efforts to seek a NEQ variational principle in the
time rate of change of the entropy of the system state
$\mathrm{d}S_{\mathrm{BG}}(\\{p_{i}(t)\\})/\mathrm{d}t$ or the entropy
production of the total super system – either its minimization[94, 95], or
maximization[96, 97]. While they can be useful in certain cases, there are
counter examples, e.g. the Schlögl model for bistable chemical system[98],
demonstrating that neither of them is generally valid. From the inference
perspective, to infer $\\{p_{i}(t)\\}$ in a non-stationary system from time
$0$ to $\tau$ requires applying MaxEnt to every instance $t\in[0,\tau]$. At
each instance $t$, $S_{\mathrm{BG}}$ is maximized by $p_{i}^{*}(t)$ among all
$p_{i}$ satisfying the corresponding constraints, but this says nothing about
the derivative of $S_{\mathrm{BG}}(\\{p_{i}^{*}(t)\\})$ being maximized or
minimized. Furthermore, the above procedure is not enough for a NEQ variation
principle of dynamics: there are more than one dynamics that can produce a
given $\\{p_{i}(t)\\}$. This objective is met by the principle of Maximum
Caliber, described below.

### Limitations of local detailed balance and heat baths.

Assumptions of local detailed balance and infinite baths have been important
and useful theoretical tools since the development of statistical physics,
especially for the recent development of ST[62, 63]. Yet, it can be limiting
to assume that fluctuations in a system correspond to an equilibrium
reservoir, as commonly occurs in Langevin or Fokker-Planck models. It requires
that a system’s fluctuations are neither too large nor too fast, as described
above. Rather, one requires a broader principle for generating non-equilibrium
fluctuations that is not tied down by equilibrium ideas. Furthermore, for
broadest generality, a non-equilibrium principle should apply to force-flow
relations that go beyond fluids, particles, electrical charges, and heat, to
encompass also the flows of vehicles and goods along networks, of proteins and
genes in cells, of electrical signals in brains, and of biological changes in
organisms in evolution and ecology. These requirements are fulfilled by the
principle of Maximum Caliber, as we now describe.

## Maximum Caliber: the variational principle for non-equilibria

A general and generative variational method for NEQ is the principle of
Maximum Caliber. Maximum Caliber (MaxCal) – a concept and term originating
with Jaynes[99] – does for trajectories what classic Maximum Entropy (MaxEnt)
does for states. In fact, it is identical to MaxEnt except that the
probabilities involved refer to paths through state-space, rather than single
states; it updates from a prior probability distribution to a unique posterior
by incorporating any known constraints, satisfying the axioms of entropic
inference that go back to Shore and Johnson.

Consider a prototypical example which follows almost exactly the form of the
MaxEnt example given above in Eqs. (6)-(8). We use Lagrange multipliers to
maximize the Caliber $\mathcal{C}$, which is the BG entropy of paths, subject
to constraints on normalization and some average flux quantity (similar to the
average energy in the equilibrium example):

$\mathcal{C}=-\sum_{k=\mathrm{paths}}p_{k}\log\left(\frac{p_{k}}{q_{k}}\right)-\gamma\left(\sum_{k}p_{k}J_{k}-\langle
J\rangle\right)+\alpha\left(\sum_{k}p_{k}-1\right).$ (22)

In Eq. (22), $\gamma$ is a Lagrange multiplier that tunes the average $\langle
J\rangle$ , while $\alpha$ ensures normalization. The maximizing distribution
is

$\displaystyle p_{k}^{*}=q_{k}\frac{e^{-\gamma J_{k}}}{Z_{d}}$ (23)

where

$\displaystyle Z_{d}=\sum_{k}q_{k}e^{-\gamma J_{k}},$ (24)

a weighted sum over paths, is the dynamical equivalent of a partition
function. Thermodynamic-like force-flux relations between the conjugated
variables of $\gamma$ and $\langle J\rangle$ can then be derived[11, 16].

Overviews of MaxCal have been given elsewhere [11, 100, 12, 34]; here is just
a brief summary. First, MaxCal recapitulates – as it should – many known
results of near-equilibrium NEQ physics, including the Green-Kubo fluctuation-
dissipation relations, Onsager reciprocal relations, Prigogine’s minimum
entropy-production principle in certain situations, and Kirchoff’s current
law. MaxCal can derive the phenomenological laws of force-flow relations, such
as Newtonian viscosities of simple fluids, Fourier’s Law of heat flow, Ohm’s
Law of electrical currents, and Fick’s Law of particle flows. It can also be
shown to provide a basis for Markov modelling [101, 102], and it frames
attempts to derive much of fundamental physics, such as Newtonian[103, 104]
and quantum [105, 106] mechanics, from an entropic inference perspective.
MaxCal – and more broadly MaxEnt – inference methods are employed successfully
in many cutting edge areas like computational science[107, 10, 108] and
theoretical endeavors like superstatistics[109, 34], as well as guiding
procedures for dealing with systems with strong coupling [110] and non-ideal
baths [42], beginning to bridge the gaps outlined above regarding perfect bath
assumptions, and therefore treating systems that are truly arbitrarily far
from equilibrium. MaxCal can successfully treat non-linearities, small
systems, and it can be applied to all kinds of flows. These successes plant
the seeds for ever more promising MaxCal results.

### How to choose constraints for the MaxCal procedure.

As noted above, the burdens of modeling with MaxEnt or MaxCal lie with the
user. The user defines the model. The user must also assert the appropriate
constraints relevant to the process at hand. To illustrate, first consider
simply EQ modeling. If you know only the temperature $T$ at a system’s
boundary, in a process where volume $V$ or particle number $N$ are also
controlling, your predictions will be misinformed. Similarly, modeling will
err if it treats only bulk constraints $(T,V,N)$, where interfacial area $A$
also matters as an independent variable. This is not a flaw of MaxEnt; it is
the burden of the user to know which, and how many, variables matter for the
problem at hand.

The problem is identical for non-equilibrium modeling using MaxCal. Consider a
ball falling through a viscous liquid. Suppose its average velocity is
$\langle v\rangle$. Now, measure the same ball dropping through a more viscous
liquid, but at the same velocity $\langle v\rangle$ (hence driven by a larger
force). The ball’s velocity alone is not sufficient to characterize this
system – the other relevant variable here is heat dissipation, $\langle
q\rangle$; MaxCal modeling of microscopic rate distributions in this situation
requires knowing both of these constraints[111]. Even in equilibrium modeling
for a system in contact with a heat bath, if the system undergoes large
changes, or fast changes, such that the heat bath cannot respond sufficiently,
one must specify the thermal conductivity or the heat capacity of the
bath[42], in addition to its temperature. For more discussion of constraints,
see [112].

## Conclusions

The early conceptions of statistical physics saw systems as large ensembles of
replicates, with probabilities seen as frequencies, and with assumptions of
ergodicity and chaotic collisions. A newer view holds that maximizing entropy
is simply a way of drawing consistent inferences about probabilities, where
the user is responsible for a proper model of physics, proper assumptions
about the equivalencies among states, and proper choice of constraints. The
same maximization of entropy procedure applies to non-equilibrium predictions
of forces and flows, except one must use path entropies instead of state
entropies. This opens up a broad area of dynamical modeling – applicable
beyond heat baths, beyond thermal materials, to situations far-from-
equilibrium and with non-linearities, applicable even to few-particle
distributions.

## Acknowledgements

We are grateful to the Stony Brook Laufer Center for support. We also thank
Bill Cannon, Chris Jarzynski, Charles Kocher, Steve Presse, David Sivak, and
Jin Wang for insightful comments and helpful feedback.

## Author contributions

The authors contributed equally to all aspects of the article.

## Competing interests

The authors declare no competing interests.

## References

  * [1] Huang, K.  _Statistical Mechanics_ (Wiley, New York, 1987).
  * [2] Hill, T. L.  _Statistical Mechanics: Principles and Selected Applications_ (Dover Publications, New York, 1987).
  * [3] McQuarrie, D. A.  _Statistical Mechanics_ (University Science Books, Sausalito, Calif, 2000).
  * [4] Dill, K. A. & Bromberg, S.  _Molecular Driving Forces: Statistical Thermodynamics in Biology, Chemistry, Physics, and Nanoscience, 2nd Edition_ (Garland Science, Boca Raton, Florida, 2010).
  * [5] Pathria, R. K. & Beale, P. D.  _Statistical Mechanics_ (Academic Press, London San Diego Cambridge, MA Oxford, 2021).
  * [6] Martino, A. D. & Martino, D. D.  An Introduction to the Maximum Entropy Approach and its Application to Inference Problems in Biology.  _Heliyon_ 4 (2018).
  * [7] Harte, J. & Newman, E. A.  Maximum Information Entropy: A Foundation for Ecological Theory.  _Trends in Ecology & Evolution_ 29, 384–389 (2014).
  * [8] Golan, A. & Harte, J.  Information Theory: A Foundation for Complexity Science.  _Proceedings of the National Academy of Sciences_ 119, e2119089119 (2022).
  * [9] Fodor, É., Jack, R. L. & Cates, M. E.  Irreversibility and Biased Ensembles in Active Matter: Insights from Stochastic Thermodynamics.  _Annual Review of Condensed Matter Physics_ 13 (2022).
  * [10] Tsai, S.-T., Fields, E., Xu, Y., Kuo, E.-J. & Tiwary, P.  Path Sampling of Recurrent Neural Networks by Incorporating Known Physics.  _Nat Commun_ 13, 7231 (2022).
  * [11] Pressé, S., Ghosh, K., Lee, J. & Dill, K. A.  Principles of Maximum Entropy and Maximum Caliber in Statistical Physics.  _Rev. Mod. Phys._ 85, 1115–1141 (2013).
  * [12] Ghosh, K., Dixit, P. D., Agozzino, L. & Dill, K. A.  The Maximum Caliber Variational Principle for Nonequilibria.  _Annual Review of Physical Chemistry_ 71, 213–238 (2020).
  * [13] Bagchi, B.  _Statistical Mechanics for Chemistry and Materials Science_ (CRC Press, 2018).
  * [14] Reichl, L.  _A Modern Course in Statistical Physics_ (Wiley, 2016).
  * [15] Kubo, R.  Statistical-Mechanical Theory of Irreversible Processes. I. General Theory and Simple Applications to Magnetic and Conduction Problems.  _Journal of the Physical Society of Japan_ 12, 570–586 (1957).
  * [16] Yang, Y.-J. & Qian, H.  Statistical Thermodynamics and Data ad Infinitum: Conjugate Variables as Forces, and their Statistical Variations (2023).  ArXiv:2205.09321 [cond-mat].
  * [17] Kolmogorov, A. N.  _Foundations of the Theory of Probability: Second English Edition_ (Dover Publications, Muneola, New York, 2018).
  * [18] Inaba, H.  The Development of Ensemble Theory.  _The European Physical Journal H_ 40, 489–526 (2015).
  * [19] Touchette, H.  The Large Deviation Approach to Statistical Mechanics.  _Physics Reports_ 478, 1–69 (2009).
  * [20] Lu, Z. & Qian, H.  Emergence and Breaking of Duality Symmetry in Generalized Fundamental Thermodynamic Relations.  _Phys. Rev. Lett._ 128, 150603 (2022).
  * [21] Sanov, I. N.  On the Probability of Large Deviations of Random Variables.  _Mat. Sbornik_ 42, 11–44 (1958).
  * [22] Landau, L. D. & Lifshitz, E. M.  _Statistical Physics, Third Edition, Part 1: Volume 5_ (Butterworth-Heinemann, Amsterdam u.a, 1980).
  * [23] Csiszar, I., Cover, T. & Byoung-Seon Choi.  Conditional Limit Theorems under Markov Conditioning.  _IEEE Trans. Inform. Theory_ 33, 788–801 (1987).
  * [24] Chetrite, R. & Touchette, H.  Nonequilibrium Microcanonical and Canonical Ensembles and Their Equivalence.  _Phys. Rev. Lett._ 111, 120601 (2013).
  * [25] Jaynes, E. T.  Information Theory and Statistical Mechanics.  _Phys. Rev._ 106, 620–630 (1957).
  * [26] Denbigh, K. & Denbigh, J.  _Entropy in Relation to Incomplete Knowledge_ (Cambridge University Press, 1985).
  * [27] Shore, J. & Johnson, R.  Axiomatic Derivation of the Principle of Maximum Entropy and the Principle of Minimum Cross-entropy.  _IEEE Trans. Inf. Theor._ 26, 26–37 (1980).
  * [28] Bak, P.  _How Nature Works: The Science of Self-organized Criticality_ (Copernicus, New York, NY, USA, 1996).
  * [29] Tsallis, C.  Nonextensive Statistical Mechanics: Construction and Physical Interpretation.  In _Nonextensive Entropy: Interdisciplinary Applications_ (Oxford University Press, 2004).
  * [30] Pressé, S., Ghosh, K., Lee, J. & Dill, K. A.  Nonadditive Entropies Yield Probability Distributions with Biases not Warranted by the Data.  _Phys. Rev. Lett._ 111, 180604 (2013).
  * [31] Tsallis, C.  Conceptual Inadequacy of the Shore and Johnson Axioms for Wide Classes of Complex Systems.  _Entropy_ 17, 2853–2861 (2015).
  * [32] Pressé, S., Ghosh, K., Lee, J. & Dill, K. A.  Reply to C. Tsallis’ “Conceptual Inadequacy of the Shore and Johnson Axioms for Wide Classes of Complex Systems”.  _Entropy_ 17, 5043–5046 (2015).
  * [33] Jizba, P. & Korbel, J.  Maximum Entropy Principle in Statistical Inference: Case for Non-Shannonian Entropies.  _Phys. Rev. Lett._ 122, 120601 (2019).
  * [34] Caticha, A.  Entropy, Information, and the Updating of Probabilities.  _Entropy_ 23, 895 (2021).
  * [35] Peterson, J., Dixit, P. D. & Dill, K. A.  A Maximum Entropy Framework for Nonexponential Distributions.  _Proceedings of the National Academy of Sciences_ 110, 20380–20385 (2013).
  * [36] Davis, S.  Conditional Maximum Entropy and Superstatistics.  _Journal of Physics A: Mathematical and Theoretical_ 53, 445006 (2020).
  * [37] Davis, S. _et al._ A Derivation of the Kappa Distribution in Non-equilibrium, Steady-state Plasmas (2023).  ArXiv:2304.13792 [cond-mat, physics:physics].
  * [38] Davis, S.  The $q$-canonical Ensemble as a Consequence of Bayesian Superstatistics (2022).  ArXiv:2112.11496 [cond-mat].
  * [39] Van der Straeten, E. & Beck, C.  Superstatistical Distributions from a Maximum Entropy Principle.  _Physical Review E_ 78, 051101 (2008).
  * [40] Ramshaw, J. D.  Maximum Entropy and Constraints in Composite Systems.  _Physical Review E_ 105, 024138 (2022).
  * [41] Vasconcelos, G. L., Salazar, D. S. P. & Macêdo, A. M. S.  Maximum Entropy Approach to $H$-theory: Statistical Mechanics of Hierarchical Systems.  _Physical Review E_ 97, 022104 (2018).
  * [42] Pachter, J. A. & Dill, K. A.  Nonequilibrium Statistical Physics beyond the Ideal Heat Bath Approximation.  _Phys. Rev. E_ 107, 014131 (2023).
  * [43] Umpierrez, H. & Davis, S.  Fluctuation Theorems in $q$-canonical Ensembles.  _Physica A: Statistical Mechanics and its Applications_ 563, 125337 (2021).
  * [44] Bercher, J. F.  Tsallis Distribution as a Standard Maximum Entropy Solution with ‘Tail’ Constraint.  _Physics Letters A_ 372, 5657–5659 (2008).
  * [45] Hernando, A. & Plastino, A.  Variational Principle Underlying Scale Invariant Social Systems.  _The European Physical Journal B_ 85, 293 (2012).
  * [46] Jarzynski, C.  Equalities and Inequalities: Irreversibility and the Second Law of Thermodynamics at the Nanoscale.  _Annual Review of Condensed Matter Physics_ 2, 329–351 (2011).
  * [47] Cover, T. M. & Thomas, J. A.  _Elements of Information Theory 2nd Edition_ (Wiley-Interscience, Hoboken, N.J, 2006).
  * [48] Schnakenberg, J.  Network Theory of Microscopic and Macroscopic Behavior of Master Equation Systems.  _Rev. Mod. Phys._ 48, 571–585 (1976).
  * [49] Parrondo, J. M. R., Horowitz, J. M. & Sagawa, T.  Thermodynamics of Information.  _Nature Phys_ 11, 131–139 (2015).
  * [50] Amari, S.-i.  _Information Geometry and Its Applications_.  Applied Mathematical Sciences (Springer Japan, 2016).
  * [51] Crooks, G. E.  Measuring Thermodynamic Length.  _Phys. Rev. Lett._ 99, 100602 (2007).
  * [52] Nicholson, S. B., García-Pintos, L. P., del Campo, A. & Green, J. R.  Time-information Uncertainty Relations in Thermodynamics.  _Nat Phys_ 1–5 (2020).
  * [53] Ito, S. & Dechant, A.  Stochastic Time Evolution, Information Geometry, and the Cramer-Rao Bound.  _Phys. Rev. X_ 10, 021056 (2020).
  * [54] Freidlin, M. I. & Wentzell, A. D.  _Random Perturbations of Dynamical Systems_.  Grundlehren der mathematischen Wissenschaften (Springer-Verlag, New York, 1984).
  * [55] Strogatz, S. H.  _Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering_ (CRC Press, 2018).
  * [56] Gardiner, C.  _Stochastic Methods: A Handbook for the Natural and Social Sciences_.  Springer Series in Synergetics (Springer-Verlag, Berlin Heidelberg, 2009).
  * [57] Nolting, B. C. & Abbott, K. C.  Balls, Cups, and Quasi-potentials: Quantifying Stability in Stochastic Systems.  _Ecology_ 15–1047.1 (2015).
  * [58] Fang, X., Kruse, K., Lu, T. & Wang, J.  Nonequilibrium Physics in Biology.  _Rev. Mod. Phys._ 91, 045004 (2019).
  * [59] Agozzino, L., Balázsi, G., Wang, J. & Dill, K. A.  How Do Cells Adapt? Stories Told in Landscapes.  _Annual Review of Chemical and Biomolecular Engineering_ 11, 155–182 (2020).
  * [60] Qian, H., Cheng, Y.-C. & Yang, Y.-J.  Kinematic Basis of Emergent Energetics of Complex Dynamics.  _EPL_ 131, 50002 (2020).
  * [61] Yang, Y.-J. & Cheng, Y.-C.  Potentials of Continuous Markov Processes and Random Perturbations.  _J. Phys. A: Math. Theor._ 54, 195001 (2021).
  * [62] Seifert, U.  From Stochastic Thermodynamics to Thermodynamic Inference.  _Annual Review of Condensed Matter Physics_ 10, 171–192 (2019).
  * [63] Peliti, L. & Pigolotti, S.  _Stochastic Thermodynamics: An Introduction_ (Princeton University Press, 2021).
  * [64] Prigogine, I. & Defay, R.  _Chemical Thermodynamics_ (Wiley, 1962).
  * [65] Jarzynski, C.  Hamiltonian Derivation of a Detailed Fluctuation Theorem.  _J. Stat. Phys._ 98, 77–102 (2000).
  * [66] Jiang, D.-Q., Qian, M. & Qian, M.-P.  _Mathematical Theory of Nonequilibrium Steady States: On the Frontier of Probability and Dynamical Systems_ (Springer, Berlin ; New York, 2004).
  * [67] Yang, Y.-J. & Qian, H.  Bivectorial Nonequilibrium Thermodynamics: Cycle Affinity, Vorticity Potential, and Onsager’s Principle.  _J Stat Phys_ 182, 46 (2021).
  * [68] Ge, H.  Extended Forms of the Second Law for General Time-dependent Stochastic Processes.  _Phys. Rev. E_ 80, 021137 (2009).
  * [69] Esposito, M. & Van den Broeck, C.  Three Detailed Fluctuation Theorems.  _Phys. Rev. Lett._ 104 (2010).
  * [70] Crooks, G. E.  Path-ensemble Averages in Systems Driven far from Equilibrium.  _Phys. Rev. E_ 61, 2361–2366 (2000).
  * [71] Yang, Y.-J. & Qian, H.  Unified Formalism for Entropy Production and Fluctuation Relations.  _Phys. Rev. E_ 101, 022129 (2020).
  * [72] Yang, Y.-J. & Qian, H.  Time-translational Symmetry in Statistical Dynamics Dictates Einstein Relation, Green-Kubo Formula, and their Generalizations.  _Phys. Rev. E_ 107, 024110 (2023).
  * [73] Graham, R.  Covariant Stochastic Calculus in the Sense of Itô.  _Physics Letters A_ 109, 209–212 (1985).
  * [74] Ao, P.  Potential in Stochastic Differential Equations: Novel Construction.  _J. Phys. A: Math. Gen._ 37, L25–L30 (2004).
  * [75] Wang, J., Xu, L. & Wang, E.  Potential Landscape and Flux Framework of Nonequilibrium Networks: Robustness, Dissipation, and Coherence of Biochemical Oscillations.  _PNAS_ 105, 12271–12276 (2008).
  * [76] Cockrell, C. & Ford, I. J.  Stochastic Thermodynamics in a Non-markovian Dynamical System.  _Physical Review E_ 105, 064124 (2022).
  * [77] Esposito, M.  Stochastic Thermodynamics under Coarse Graining.  _Phys. Rev. E_ 85, 041125 (2012).
  * [78] Sekimoto, K.  _Stochastic Energetics_ (Springer, Heidelberg ; New York, 2010).
  * [79] Seifert, U.  Stochastic Thermodynamics: From Principles to the Cost of Precision.  _Physica A: Statistical Mechanics and its Applications_ 504, 176–191 (2018).
  * [80] Barato, A. C. & Seifert, U.  Thermodynamic Uncertainty Relation for Biomolecular Processes.  _Phys. Rev. Lett._ 114, 158101 (2015).
  * [81] Horowitz, J. M. & Gingrich, T. R.  Thermodynamic Uncertainty Relations Constrain Non-equilibrium Fluctuations.  _Nat Phys_ 16, 15–20 (2020).
  * [82] Falasco, G. & Esposito, M.  Local Detailed Balance across Scales: From Diffusions to Jump processes and beyond.  _Physical Review E_ 103, 042114 (2021).
  * [83] Hartich, D. & Godec, A.  Violation of Local Detailed Balance Despite a Clear Time-Scale Separation (2022).  ArXiv:2111.14734 [cond-mat, physics:physics].
  * [84] Wasnik, V.  Revisiting Multiple Thermal Reservoir Stochastic Thermodynamics (2023).  ArXiv:2303.14949 [cond-mat].
  * [85] Reinhardt, W. P. & Hunter, J. E., III.  Variational Path Optimization and Upper and Lower Bounds to Free Energy Changes via Finite Time Minimization of External Work.  _The Journal of Chemical Physics_ 97, 1599–1601 (1992).
  * [86] Hunter, J. E., III, Reinhardt, W. P. & Davis, T. F.  A Finite-time Variational Method for Determining Optimal Paths and Obtaining Bounds on Free Energy Changes from Computer Simulations.  _The Journal of Chemical Physics_ 99, 6856–6864 (1993).
  * [87] Jarzynski, C.  Nonequilibrium Equality for Free Energy Differences.  _Phys. Rev. Lett._ 78, 2690–2693 (1997).
  * [88] Seifert, U.  First and Second Law of Thermodynamics at Strong Coupling.  _Phys. Rev. Lett._ 116, 020601 (2016).
  * [89] Jarzynski, C.  Stochastic and Macroscopic Thermodynamics of Strongly Coupled Systems.  _Phys. Rev. X_ 7, 011008 (2017).
  * [90] Miller, H. J. D. & Anders, J.  Entropy Production and Time Asymmetry in the Presence of Strong Interactions.  _Phys. Rev. E_ 95, 062123 (2017).
  * [91] Crooks, G. E.  Nonequilibrium Measurements of Free Energy Differences for Microscopically Reversible Markovian Systems.  _J. Stat. Phys._ 90, 1481–1487 (1998).
  * [92] Spinney, R. & Ford, I.  Fluctuation Relations: A Pedagogical Overview.  _Nonequilibrium Statistical Physics of Small Systems: Fluctuation Relations and Beyond_ 3–56 (2013).
  * [93] Bertini, L., De Sole, A., Gabrielli, D., Jona-Lasinio, G. & Landim, C.  Macroscopic Fluctuation Theory.  _Rev. Mod. Phys._ 87, 593–636 (2015).
  * [94] Klein, M. J. & Meijer, P. H. E.  Principle of Minimum Entropy Production.  _Phys. Rev._ 96, 250–255 (1954).
  * [95] de Groot, S. R. & Mazur, P.  _Non-Equilibrium Thermodynamics_ (Dover Publications, New York, 2011).
  * [96] Dewar, R. C.  Maximum Entropy Production as an Inference Algorithm that Translates Physical Assumptions into Macroscopic Predictions: Don’t Shoot the Messenger.  _Entropy_ 11, 931–944 (2009).
  * [97] Martyushev, L. M.  Maximum Entropy Production Principle: History and Current Status.  _Phys. Usp._ 64, 558–583 (2021).
  * [98] Vellela, M. & Qian, H.  Stochastic Dynamics and Non-equilibrium Thermodynamics of a Bistable Chemical System: the Schlögl Model Revisited.  _Journal of The Royal Society Interface_ 6, 925–940 (2008).
  * [99] Jaynes, E. T.  The Minimum Entropy Production Principle.  _Annual Review of Physical Chemistry_ 31, 579–601 (1980).
  * [100] Hazoglou, M. J., Walther, V., Dixit, P. D. & Dill, K. A.  Communication: Maximum Caliber is a General Variational Principle for Nonequilibrium Statistical Mechanics.  _The Journal of Chemical Physics_ 143, 051104 (2015).
  * [101] Ge, H., Pressé, S., Ghosh, K. & Dill, K. A.  Markov Processes Follow from the Principle of Maximum Caliber.  _The Journal of Chemical Physics_ 136, 064108 (2012).
  * [102] Lee, J. & Pressé, S.  A Derivation of the Master Equation from Path Entropy Maximization.  _The Journal of Chemical Physics_ 137, 074103 (2012).
  * [103] Ge, H. & Qian, H.  Analytical Mechanics in Stochastic Dynamics: Most Probable Path, Large-deviation Rate Function and Hamilton-Jacobi Equation.  _Int. J. Mod. Phys. B_ 26, 1230012 (2012).
  * [104] González, D., Davis, S. & Gutiérrez, G.  Newtonian Dynamics from the Principle of Maximum Caliber.  _Foundations of Physics_ 44, 923–931 (2014).
  * [105] General, I. J.  Principle of Maximum Caliber and Quantum Physics.  _Physical Review E_ 98, 012110 (2018).
  * [106] Chetrite, R., Muratore-Ginanneschi, P. & Schwieger, K.  E. Schrödinger’s 1931 paper “On the Reversal of the Laws of Nature” [“Über dieUmkehrung der Naturgesetze”,Sitzungsberichte der preussischen Akademie der Wissenschaften, physikalische mathematische Klasse, 8 N9 144-153].  _arXiv:2105.12617 [cond-mat, physics:physics, physics:quant-ph]_ (2021).
  * [107] Bolhuis, P. G., Brotzakis, Z. F. & Vendruscolo, M.  A Maximum Caliber Approach for Continuum Path Ensembles.  _The European Physical Journal B_ 94, 188 (2021).
  * [108] Alves, A., Dias, A. G. & da Silva, R.  Maximum Entropy Principle and the Higgs Boson Mass.  _Physica A: Statistical Mechanics and its Applications_ 420, 1–7 (2015).
  * [109] Sattin, F.  Bayesian Approach to Superstatistics.  _The European Physical Journal B - Condensed Matter and Complex Systems_ 49, 219–224 (2006).
  * [110] Dixit, P. D.  A Maximum Entropy Thermodynamics of Small Systems.  _The Journal of Chemical Physics_ 138, 184111 (2013).
  * [111] Agozzino, L. & Dill, K.  Minimal Constraints for Maximum Caliber Analysis of Dissipative Steady-state Systems.  _Phys. Rev. E_ 100, 010105 (2019).
  * [112] Caticha, A.  Entropic Inference: Some Pitfalls and Paradoxes We Can Avoid.  _AIP Conference Proceedings_ 1553, 200–211 (2013).