# Infomechanics of Independent and Identically Distributed Particles

Arnaldo Spalvieri∗
∗Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di
Milano

###### Abstract

The physical system that we consider is a quantum thermodynamical system
composed by non-interacting particles whose quantized energies are modelled as
independent and identically distributed random variables. Making use of the
information-theoretic concept of typicality and thanks to the identification
of the information-theoretic conditional entropy with the physical conditional
entropy of the microstate given the macrostate, the paper shows that the
standard assumption of equally probable microstates is non-necessary (if not
misleading) and that the celebrated Boltzmann-Planck entropy is indeed a
conditional entropy with deterministic condition. Besides these new conceptual
results, we claim several new specific technical results of major physical
relevance, among which are the probability distribution of the occupancy
numbers of the energy levels, the probability distribution of the total
quantum number in a system of monochromatic quantum harmonic oscillators at
the thermal equilibrium, an exact formula for the ideal gas in a container
that gives the entropy of the gas also at low temperature, thus giving a
complete explanation of the Gibbs paradox that, at low temperature, is still
not explained by today’s standard theory. These specific results are pieces of
a self-consistent and unified framework that encompasses the cases of low and
high temperature, of indexed and non-indexed particles, of small and large
number of particles, of microcanonical, canonical and grand canonical
ensembles.

Keywords: Boltzmann-Planck entropy, Gibbs entropy, Shannon entropy,
conditional entropy, mutual information, Holevo accessible information,
microstates, macrostates, multinomial distribution, negative binomial
distribution, ideal dilute gas, stars and bars method.

##  I Introduction

The following passage of an interview with Shannon is extracted from [1]: My
greatest concern was what to call it. I thought of calling it ’information,’
but the word was overly used, so I decided to call it ’uncertainty.’ When I
discussed it with John von Neumann, he had a better idea. Von Neumann told me,
”You should call it entropy, for two reasons. In the first place your
uncertainty function has been used in statistical mechanics under that name,
so it already has a name. In the second place, and more important, no one
really knows what entropy really is, so in a debate you will always have the
advantage.”

The commonly accepted definition of entropy in statistical mechanics and
thermal physics is

$S=k_{B}\log(W),$ (1)

where $S$ is the entropy, $\log(x)$ is the natural logarithm of $x$,

$k_{B}=1.38\cdot 10^{-23}\ \ \mbox{J}\cdot\mbox{K}^{-1}$

is Boltzmann’s constant and $W$ is the number of microstates that are
compatible with the constraints imposed on the system. The phrase of Von
Neumann that opens this introduction reflects the fact that the above entropy,
called by many authors Boltzmann-Planck entropy, has been for a long time and
is still today object of discussion. Actually, the right side of the equality
is an entropy only if the $W$ microstates are equally probable but, still
today, there is not any mathematical proof of microstates’ equiprobability.
Nonetheless, equation (1) is universally recognized to be the very foundation
of thermal physics and statistical mechanics, that, in the absence of a
mathematical proof but, at the same time, in the presence of strong physical
evidences, take it as a postulate. As far as we know, Einstein was the first,
in [2], to demand equiprobability of Boltzmann-Planck microstates.

There is wide skepticism in the research community about the possibility of
producing a mathematical proof of microstates equiprobability. This skepticism
dates back to the following quotation from Khinchin [3], that is widely
reported in the open bibliography, e.g. [4, 5]: All existing attempts to give
a general proof of this postulate must be considered as an aggregate of
logical and mathematical errors superimposed on a general confusion in the
definition of the basic quantities. About fifteen years ago, by arguments
based on typicality, the two papers [6] and [7] proved that microstates cannot
be equiprobable, but, at the same time, they showed that microstates converge
to equiprobability as the number of particle grows. These two papers have
attracted noticeable interest, see for instance [8], and can be considered a
cornerstone of modern quantum thermodynamics, see [10, 9].

A result similar to that of [6] and [7] has been recently published by this
author in [11]. The approach of [11], although relying on typicality as [6]
and [7], is much more information-theoretically oriented than [6] and [7].
Starting from the concept of entropy that statistical physics and information
theory share, many authors built in the past bridges between the two
disciplines. Making a comprehensive review of the bibliography that links
information theory and statistical physics and, more generally, of the
extremely vast bibliography that, in various manners, touches the topics that
are teated in this paper, is out of our scope. We will limit ourselves to cite
and to comment along the paper the bibliography that we find relevant to the
specific point at hand. Among the works that more contributed to the
strengthen link between the two disciplines we cite the book [12], which
explores the relations between codes based on factor graphs and ferromagnetic
and spin glasses models. The deep connection between information and
thermodynamics has been enlightened by the paper of Landauer [13], that showed
the equivalence between heat and logical information. This major result is
today proved experimentally [14] and it is widely accepted that (quantum)
thermodynamics can be treated by (quantum) information-theoretic tools, see
for instance the tutorial paper [15]. With reference to the links between
classical statistical physics and classical information and communication
theory, we limit ourselves to mention the recent survey [16], the pioneering
work of Jaynes [17] and the large attention that has been attracted in the
past by the connections between information theoretic inequalities and the
irreversibility of certain thermodynamical processes, see [18] and Chapter 4
of [19]. Recently, the development of quantum information theory has brought
the community of information theory researchers closer to quantum
thermodynamics than ever before, see for instance the recent tutorial [20].

Despite the evidence that heat and logical information are equivalent, still
today the standard textbook approach to statistical mechanics and thermal
physics is strongly physically oriented, leaving to classical and quantum
information theory only a minor role. Although many textbooks have a chapter
or a section devoted to information theory, quantities as random entropy,
conditional entropy, mutual information, Holevo accessible information, are
often overlooked. This happens because these quantities and the associated
concepts, that are of fundamental importance in information theory, actually
seem not to have yet found their role in classical and quantum statistical
mechanics. The identification of these information-theoretic quantities with
the corresponding physical quantities is the vision that guides the research
reported in this paper. In our understanding, more than proposing a theory,
this paper proposes a vision, the infomechanical vision, where the
hybridization between the two disciplines is deep as never it has been before.
Assuming that particles can be modelled as independent subsystems and that the
probability distribution of particle’s quantized energy is the same for all
the particles, and starting from the guiding idea that macrostates are random
global attributes of the system that play the role of random conditions in the
conditional probability of microstates, we demonstrate the following specific
results of physical relevance that, to our best knowledge, are new:

  * •

the probability distribution of the occupancy numbers in a system of a fixed
number of particles that can take many energy values is the multinomial
distribution;

  * •

microstates of the above system are conditionally equiprobable given the
occupancy numbers;

  * •

microstates of the above system in the canonical ensemble are conditionally
equiprobable given the system energy;

  * •

the probability distribution of the total quantum number in a canonical
ensemble of monochromatic quantum harmonic oscillators is the negative
binomial distribution;

  * •

the thermal state of a system of quantum harmonic oscillators tends to the
coherent state of the quantum harmonic oscillator in the same way as the
negative binomial distribution tends to the Poisson distribution;

  * •

all the arrangements of energy quanta of a system of monochromatic quantum
harmonic oscillators counted by the stars and bars method are equiprobable
given the total quantum number (the total number of stars);

  * •

the probability distribution of the occupancy numbers of the ideal gas in the
grand canonical ensemble is the product of Poisson distributions also at low
temperature;

  * •

an exact formula for the entropy of the ideal gas in a container, that holds
with equality also at low temperature.

These specific results are pieces of a puzzle where low and high temperature,
indexed and non-indexed particles, small and large number of particles,
microcanonical, canonical and grand canonical ensembles, coexist and find
their own place in harmony between them.

The outline of the paper is as follows. In section II we present the notation
and the main definitions. Although this material is standard in information
theory, we present it because, since it could be not part of the background of
researchers of statistical mechanics, it contributes to make the paper self-
contained. Section III presents the assumptions that we make and the system
model. The patient and interested reader that will read the entire paper,
looking backward from the conclusions to sections II and III, will recognize
that these two sections contain all the premises that, once developed in a
quite straightforward way, lead to all the results of the paper. In section
IV, systems made by non-indexed particles are considered. The main conceptual
points of this section are the information-theoretic typicality and the
associated typical set and the demonstration of conditional equiprobability of
microstates given the corresponding macrostate. When considered together,
these two concepts take the place of the traditional assumption of equally
probable microstates. Also, in section IV it is shown that the probability
distribution of the occupancy numbers with fixed number of particles is the
multinomial distribution, a result that, as claimed before, seems to be new.
Section V is devoted to the application of the findings of section IV to
systems at the thermal equilibrium and to the discussion of the role of
macrostates in the conditional equiprobability of microstates. Section VI
shows that the probability distribution of the total quantum number of a
system of monochromatic quantum harmonic oscillators at the thermal state is
the negative binomial distribution. This results is analogous to the
multinomial distribution of the occupancy numbers found before. In this
section it is also shown that, as a consequence of conditional
equiprobability, all the configurations of energy quanta counted by the stars
and bars method are equally probable given the total quantum number. Section
VII discusses systems of non-indexed particles, showing the compatibility
between our approach and both the classical and the quantum approaches and
showing the application to the cases of fixed and random number of particles.
One specific instance of non-indexed particles is the ideal dilute gas in a
container, that is studied in section VIII, where it is given an exact entropy
formula that describes the thermodynamics of the gas both at low and at high
temperature. In section IX the main points of the paper are summarized and
comments about open points that could be addressed in the future are
presented.

##  II Notation and definitions

Let the uppercase calligraphic character, e.g. ${\cal L}$, denote a random
variable and let $\\{{\cal L}\\}$ be the support set of the random variable.
Unless explicitly stated, in the following we will refer to discrete random
variables. We denote $|{\cal L}|$ the number of elements of the support set of
the discrete random variable and we indicate the probability $Pr({\cal L}=l)$
with the shorthand $p_{\cal L}(l)$, that is

$p_{\cal L}(l)\stackrel{{\scriptstyle\text{def}}}{{=}}Pr({\cal L}=l),\ \
\forall\ l\in\\{{\cal L}\\}.$

The probability distribution, or, simply, distribution, of the random ${\cal
L}$, is the set $\\{p_{\cal L}(l)\\}$, $\forall\ l\in\\{{\cal L}\\}$, that
will be often hereafter indicated with the shorthand $p_{\cal L}$. In the
following we will use the angular brackets to denote the expectation (when it
exists) of a function of random variable with respect to the probability
distribution of the random variable (the calligraphic character) that is
inside the brackets:

$\braket{\epsilon({\cal
L})}\stackrel{{\scriptstyle\text{def}}}{{=}}\sum_{l\in\\{{\cal L}\\}}p_{\cal
L}(l)\epsilon(l),$

where $\epsilon(\cdot)$ is a deterministic function of its argument. The above
notation can be extended in a straightforward way to the case of functions of
many random variables, by taking the expectation over the joint probability
distribution of the many random variables that are inside the angular
brackets. Let us consider the random variable

${\cal E}=\epsilon({\cal L}).$

When the context leaves no room to ambiguity about the deterministic function
$\epsilon(\cdot)$ that maps ${\cal L}$ onto ${\cal E}$, we will use the
familiar notation $\mu_{\cal E}$ and $\sigma^{2}_{\cal E}$ for mean value and
variance of ${\cal E}$:

$\mu_{{\cal E}}\stackrel{{\scriptstyle\text{def}}}{{=}}\braket{\epsilon({\cal
L})},$ $\sigma^{2}_{{\cal
E}}\stackrel{{\scriptstyle\text{def}}}{{=}}\braket{\epsilon^{2}({\cal
L})}-(\braket{\epsilon({\cal L})})^{2}.$

In general, when we want to shorten the notation of expectation we put the
random variable in the subscript, while when we want to indicate a
deterministic function of a random variable we put the random variable inside
the round bracket. The special case $\epsilon(\cdot)=\delta(\cdot)$, where
$\delta(\cdot)$ is the indicator function,

$\displaystyle\delta(x)=\left\\{\begin{array}[]{cc}1,&\ \ x=0,\\\ 0,&\ \ x\neq
0,\\\ \end{array}\right.$ (4)

shows two interpretations of the function $p_{\cal L}(\cdot)$. If the argument
of $p_{\cal L}(\cdot)$ is deterministic, then the deterministic $p_{\cal
L}(\cdot)$ expresses probability as expectation:

$\displaystyle p_{\cal L}(l)$ $\displaystyle=\sum_{k\in\\{{\cal L}\\}}p_{\cal
L}(k)\delta(k-l)$ (5) $\displaystyle=\braket{\delta({\cal L}-l)}.$

Instead, if the argument of $p_{\cal L}(\cdot)$ is random, then $p_{\cal
L}(\cdot)$ is the random variable that is obtained by mapping the outcome of
the random argument into its probability:

$\displaystyle p_{\cal L}({\cal L})=\sum_{l\in\\{{\cal L}\\}}p_{\cal
L}(l)\delta(l-{\cal L}).$ (6)

The reader could find the above discussion about the meaning of $p_{\cal
L}(l)$ and $p_{\cal L}({\cal L})$ redundant, if not trivial, but we want to
stress the difference between $p_{\cal L}(l)$ and $p_{\cal L}({\cal L})$
because, in this paper, this difference is crucial: in the rest of the paper,
$p_{\cal L}(l)$ means the deterministic quantity (5) while $p_{\cal L}({\cal
L})$ means the random quantity (6).

Even if we know that the reader is for sure familiar with the concept that
follows, we spend few words on the convergence of the random sample mean to
its expectation, because it plays a key role in this paper. Let $\bar{v}^{N}$
denote a vector with $N$ elements:

$\bar{v}^{N}=(v_{1},v_{2},\cdots,v_{N}).$

We will drop the superscript N when it is understood from the context.
Consider a random vector $\bar{\cal L}$ of $N$ independent and identically
distributed (i.i.d.) random variables, that is

$p_{\bar{\cal L}}(\bar{l})=\prod_{i=1}^{N}p_{{\cal L}_{i}}(l_{i}),$ $p_{{\cal
L}_{1}}=p_{{\cal L}_{2}}=\cdots=p_{{\cal L}_{N}}=p_{\cal L}.$

Let

$\mu_{{\cal E}_{\Sigma}}=N\mu_{{\cal E}},\ \ \ \sigma_{{\cal
E}_{\Sigma}}=\sqrt{N}\sigma_{{\cal E}},$

be the mean value and the standard deviation of the random sum

${\cal E}_{\Sigma}=\sum_{i=1}^{N}\epsilon({\cal L}_{i}),$

where the subscript Σ recalls that we are dealing with a sum over all the $N$
random variables. Although the randomness of ${\cal E}_{\Sigma}$, represented
in this picture by $\sigma_{{\cal E}_{\Sigma}}$, increases with the square
root of $N$, when ${\cal E}_{\Sigma}$ is compared to its mean value it tends
to become deterministic with the inverse of the square root of $N$:

$\frac{\sigma_{{\cal E}_{\Sigma}}}{\mu_{{\cal
E}_{\Sigma}}}=\frac{\sigma_{{\cal E}}}{\sqrt{N}\mu_{{\cal E}}}.$

When ${\cal E}_{\Sigma}$ is regarded as an unbiased random estimator of its
expectation, the above ratio is the relative error between the random
estimator and its expectation. However, it remains that $\sigma_{{\cal
E}_{\Sigma}}$ diverges for $N\rightarrow\infty$. Therefore, when
$N\rightarrow\infty$ and the above ratio is the object that one wants to
study, the above equality suggests to consider, in place of ${\cal
E}_{\Sigma}$, the random sample mean $N^{-1}{\cal E}_{\Sigma}$, which, for
$N\rightarrow\infty$, remains limited and tends to become deterministic:

$\lim_{N\rightarrow\infty}\frac{{\cal E}_{\Sigma}}{N}=\mu_{\cal E}.$

This limit is the law of large numbers. Convergence of the limit can be in
various senses. In this paper we will consider convergence in probability,
that is, the weak law of large numbers.

### Surprise, uncertainty, information

In classical information and communication theory, the random entropy $H({\cal
L})$ is the following deterministic function of the discrete random variable
${\cal L}$:

$\infty>H({\cal L})\stackrel{{\scriptstyle\text{def}}}{{=}}-k\log(p_{\cal
L}({\cal L}))\geq 0,$

where $k>0$ is a constant that depends on the context and the inequalities are
obvious because $0<p_{\cal L}({\cal L})\leq 1$ (we exclude from the support
set of ${\cal L}$ events with zero probability to guarantee that the random
entropy is limited). In physics $k=k_{B}$, while, in information and
communication theory,

$k=\frac{1}{\log(2)},$

or, equivalently, one takes the base-2 logarithm and $k=1$. In what follows we
take the natural logarithm with $k=1$ and put explicitly $k_{B}$ in front of
the entropy when it is required by the context. Information and communication
theory researchers often call $H({\cal L})$ surprise, because it
quantitatively expresses the surprise that one experiences when he observes
the result ${\cal L}$ of the random experiment. Being a deterministic function
of a random variable, see (6) and the relevant discussion about its meaning,
the above $H({\cal L})$ is a random variable itself. The expectation of
$H({\cal L})$ with respect to the probability distribution $p_{\cal L}$ is the
Shannon entropy of ${\cal L}$:

$H_{\cal L}\stackrel{{\scriptstyle\text{def}}}{{=}}-\sum_{l\in\\{{\cal
L}\\}}p_{\cal L}(l)\log(p_{\cal L}(l))\geq 0,$

where we use the shorthand notation

$H_{\cal L}=\braket{H({\cal L})}$

and the inequality becomes equality only when ${\cal L}$ is non-random. In the
information and communication theory community, the expectation of the
surprise is often called uncertainty. This name reflects the uncertainty that
we have about the result of the random experiment before observing the random
outcome. Actually, uncertainty can be seen as the surprise that we expect to
experience when we will see the random result.

The extension to random vectors is straightforward. The random entropy of the
random vector $\bar{\cal L}$ is

$H(\bar{\cal L})\stackrel{{\scriptstyle\text{def}}}{{=}}-\log(p_{\bar{\cal
L}}(\bar{\cal L})).$ (7)

Its expectation, that is the Shannon entropy of the random vector, is

$\displaystyle H_{\bar{\cal L}}$
$\displaystyle\stackrel{{\scriptstyle\text{def}}}{{=}}-\sum_{\bar{l}\in\\{\bar{\cal
L}\\}}p_{\bar{\cal L}}(\bar{l})\log(p_{\bar{\cal L}}(\bar{l}))$ (8)
$\displaystyle\leq\sum_{i=1}^{N}H_{{\cal L}_{i}}$ (9) $\displaystyle=NH_{{\cal
L}}$

where the inequality becomes equality when the random variables that compose
the random vector are independent, see (2.96) of [19], and the last equality
holds in case of identically distributed random variables. The expectation of
(7), that is (8), is called in thermodynamics Gibbs entropy. Gibbs entropy is
commonly accepted as the entropy of an ensemble of systems at the thermal
equilibrium (canonical ensemble), see for instance eqn. (13) of 3.3 of [21].
Inequality (9) was pointed out by Jaynes in a paper that attempts to clarify
the relation between the Boltzmann-Planck entropy and the Gibbs entropy, see
[22].111The inequality by Jaynes is between the differential entropies of
continuous random variables, but the same inequality holds also for discrete
random variables. It follows immediately from the definitions (1) and (2) of
Jaynes’ paper, without the need of any other consideration than
dependency/independency of particles between them. We observe that still today
researchers are actively trying to put in a coherent relation the Gibbs
entropy and the Boltzmann-Planck entropy, see e.g. [23]. The better succeeded
of these attempts is, to our opinion, paper [24]. We will return on [24] in
one of the next sections to discuss the relation between this paper and [24].

Given two random variables ${\cal L}$ and ${\cal X}$, the conditional random
entropy of ${\cal L}$ given ${\cal X}$ is

$H({\cal L}|{\cal X})\stackrel{{\scriptstyle\text{def}}}{{=}}-\log(p_{{\cal
L}|{\cal X}}({\cal L},{\cal X})),$

where the familiar notation is used for the conditional probability
distribution. The expectation of the conditional random entropy over the joint
probability distribution of ${\cal L}$ and ${\cal X}$ is the conditional
entropy of ${\cal L}$ given ${\cal X}$, that is

$H_{{\cal L}}\geq H_{{\cal L}|{\cal
X}}\stackrel{{\scriptstyle\text{def}}}{{=}}\sum_{x\in\\{{\cal X}\\}}p_{\cal
X}(x)H_{{\cal L}|x}\geq 0,$ (10)

where

$H_{{\cal L}|x}\stackrel{{\scriptstyle\text{def}}}{{=}}-\sum_{l\in\\{{\cal
L}\\}}p_{{\cal L}|{\cal X}}(l,x)\log(p_{{\cal L}|{\cal X}}(l,x))$

is the conditional entropy of ${\cal L}$ with deterministic condition ${\cal
X}=x$, see chapter 2 of [19] for the definition of conditional entropy and for
the inequalities in (10). The conditional entropy of ${\cal L}$ given ${\cal
X}$ is a measure of the uncertainty that we have about the unknown ${\cal L}$
given the outcome of ${\cal X}$. One of the most fundamental quantities in
classical information theory is the information $I_{{\cal L};{\cal X}}$
between ${\cal L}$ and ${\cal X}$:

$I_{{\cal L};{\cal X}}\stackrel{{\scriptstyle\text{def}}}{{=}}H_{{\cal
L}}-H_{{\cal L}|{\cal X}}\geq 0,$

where the inequality becomes equality only when ${\cal L}$ and ${\cal X}$ are
independent random variables, in which case the uncertainty that we have about
the unknown ${\cal L}$ is unchanged by the given ${\cal X}$. The above
quantity, introduced by Shannon in [25], is interpreted as the reduction of
uncertainty about ${\cal L}$ achieved thanks to the given ${\cal X}$, that is,
the information that ${\cal X}$ brings about the unknown ${\cal L}$. By
straightforward use of Bayes rule one has

$I_{{\cal L};{\cal X}}=H_{{\cal L}}-H_{{\cal L}|{\cal X}}=H_{{\cal
X}}-H_{{\cal X}|{\cal L}}=I_{{\cal X};{\cal L}}.$ (11)

For this reason, often the information is called mutual information. The
extension to random vectors is straightforward and is not pursued for brevity.

### Quantum entropy and quantum accessible information

The appropriate measures of entropy and information for quantum systems are
the Von Neumann entropy and the Holevo accessible information. With the help
of the popular representation of quantum information acted by Alice and Bob,
we recall hereafter the definitions of these two quantities, linking them to
the corresponding classical quantities. We suppose that Alice prepares the
system in a state represented by the density operator $\hat{\rho}({\bar{\cal
X}})$, where the discrete random vector ${\bar{\cal X}}$ takes its values in
the set $\\{{\bar{\cal X}}\\}$ with probability distribution $p_{\bar{\cal
X}}$. After preparation, Alice passes the quantum system to Bob, who performs
a POVM measurement on the state prepared by Alice, and, from the result
${\bar{\cal Y}}$ of the measurement, tries to guess ${\bar{\cal X}}$. The
above process is subject to the following quantum information-theoretic
inequalities:

$\displaystyle H_{{\bar{\cal X}}}$ $\displaystyle\geq
S(\hat{\rho})-\sum_{\bar{x}\in\\{\bar{\cal X}\\}}p_{\bar{\cal
X}}(\bar{x})S(\hat{\rho}({\bar{x}}))$ (12)
$\displaystyle\stackrel{{\scriptstyle\text{def}}}{{=}}\chi$ $\displaystyle\geq
I_{\bar{\cal X};\bar{\cal Y}},$ (13)

where $\hat{\rho}({\bar{x}})$ is the density operator of the state prepared by
Alice when $\bar{\cal X}=\bar{x}$,

$\hat{\rho}\stackrel{{\scriptstyle\text{def}}}{{=}}\sum_{\bar{x}\in\\{\bar{\cal
X}\\}}p_{\bar{\cal X}}(\bar{x})\hat{\rho}({\bar{x}}),$ (14)

and

$S(\hat{\rho})\stackrel{{\scriptstyle\text{def}}}{{=}}-trace(\hat{\rho}\log(\hat{\rho})),$

is the Von Neumann entropy of the density operator $\hat{\rho}$. When the
density operators $\\{\hat{\rho}({\bar{x}})\\}$ of the prepared states have
support on orthogonal, hence separable, subspaces, inequality (12) saturates
to equality, see theorem 11.10 of [26]. The Greek letter $\chi$, which is
commonly used to indicate the right hand side of (12), is called Holevo
quantity, or Holevo accessible information. According to (13), the Holevo
quantity is an upper bound on the classical mutual information $I_{\bar{\cal
X};\bar{\cal Y}}$, see chapter 12 of [26].

##  III System model and scope of the paper

We represent a spin-less quantum particle with one degree of freedom by the
pseudo-quantum model (the name pseudo-quantum is taken from 16.4.1 of [27]),
where quantum refers to energy quantization while pseudo means that energy is
the only attribute of the particle considered in the model. Particles whose
consideration needs to account for other attributes than energy, for instance
the spin of bosons and fermions in full quantum (as opposite to pseudo-
quantum) systems, are out of the scope of this paper. The pseudo-quantum model
is commonly accepted in statistical mechanics and is adopted in standard
textbooks, e.g. [27] and [21]. In this model, the state of the particle is a
stationary energy eigenstate and the energy of the particle takes its values
in the set $\\{{\cal E}\\}$ of the eigenvalues of the solutions of the time-
independent Schrödinger equation, see again the books cited before. The model
finds application, for instance, to the quantum harmonic oscillator, where

$\epsilon(l)=l\omega\hbar+\frac{1}{2}\omega\hbar,\ \ l\ \in\\{0,1,\cdots\\},$
(15)

where $l$ is the quantum number,

$\hbar=1.05\cdot 10^{-34}\ \ \mbox{J}\cdot\mbox{s}$

is the reduced Planck constant, $\omega$ is the angular frequency of the
oscillation and

$\epsilon(0)=\frac{1}{2}\omega\hbar$

is the energy of the ground state. Another example is that of a particle of
mass $m$ in an one-dimensional box of length $L$. In this case, the mapping
from energy level to energy is

$\epsilon(l)=l^{2}\frac{\pi^{2}\hbar^{2}}{2mL^{2}},\ \ l\
\in\\{1,2,\cdots\\}.$ (16)

The pseudo-quantum approach of Sackur and Tetrode, see e.g. [28], is also
based on quantization, but in that case what is quantized is the phase space,
while here we quantize energy and, in place of defining entropy as a function
of position and momentum, we define it as a function of the quantized energy.
Note that, compared to the phase-space approach, which needs approximations to
take into account quantum corrections at low temperature, see the recent paper
[29], the consideration of quantized energy natively allows to exactly import
the quantum corrections inside the model, leading to an exact description of
entropy also at low temperature. In the following, our main concerns are
entropy and information of discrete random variables. These quantities remain
unchanged when the discrete random variable is mapped onto another discrete
random variable by a deterministic one-to-one mapping, as (15) and (16).
Hence, when random entropy and entropy of (15) and (16) are considered, we
could take the one that we prefer between the random energy ${\cal E}$ and the
random level ${\cal L}$. Commonly the symbol ${\cal L}$ is an index and for
this reason it is immediately recognized as a discrete variable, therefore we
will use ${\cal L}$ and $l$ in place of ${\cal E}$ and $\epsilon$, the one-to
one mapping $\epsilon(l)$ being understood.

Unless otherwise specified (it will be the case of the grand canonical
ensemble in Section VII), in the following we consider a system made by a
fixed and known number $N$ particles, all with the same quantization rule
$\epsilon(\cdot)$. Also, we assume that particles are non interacting or
weakly interacting, an assumption that is standard in many cases studied in
statistical mechanics. With this assumption, the multi-particle Schrödinger
equation factorizes into $N$ independent single-particle equations, the energy
eigenvalue of the Hamiltonian operator $\hat{H}$ operating on the Hilbert
space of the multi-particle system is the sum of the energy eigenvalues of the
Hamiltonian operators operating on the Hilbert subspaces of the individual
particles and the energy eigenket $\ket{\bar{l}}$ of system’s Hamiltonian is
the tensor product, hereafter denoted $\otimes$, of the energy eigenkets of
the individual Hamiltonian operators. In bra-ket notation, this assumption
reads

$\hat{H}\ket{\bar{l}}=\left(\sum_{i=1}^{N}\epsilon(l_{i})\right)\ket{\bar{l}},$
$\ket{{\bar{l}}}=\ket{l_{1}}_{1}\otimes\ket{l_{2}}_{2}\otimes\cdots\otimes\ket{l_{N}}_{N},\
\ $ (17)
$\hat{H}=\sum_{i=1}^{N}\hat{I}_{1}\otimes\cdots\otimes\hat{I}_{i-1}\otimes\hat{H}_{i}\otimes\hat{I}_{i+1}\otimes\cdots\otimes\hat{I}_{N},$

where $\hat{I}_{i}$ is the identity operator that operates on the $i$-th
Hilbert subspace, the one of the $i$-th particle, $\hat{H}_{i}$ is the
Hamiltonian operator that operates on the $i$-th Hilbert subspace, and
$\ket{k}_{i}$ is the $k$-th eigenket of $\hat{H}_{i}$.

In the following, for the density operators $\\{\hat{\rho}({\bar{x}})\\}$
appearing in the right hand side of (14) we will take a linear combination of
the type

$\hat{\rho}({\bar{x}})=\sum_{\bar{l}\in\\{\bar{\cal L}\\}}p_{\bar{\cal
L}|\bar{\cal X}}(\bar{l},\bar{x})\hat{\rho}({\bar{l}}).$ (18)

with

$\hat{\rho}({\bar{l}})=\ket{{\bar{l}}}\bra{{\bar{l}}}.$ (19)

Since $\ket{{\bar{l}}}$ is a pure state,

$S(\hat{\rho}({\bar{l}}))=0,$

therefore, with $\hat{\rho}({\bar{l}})$ of (19) in the role of
$\hat{\rho}({\bar{x}})$ of (12) and $\hat{\rho}({\bar{x}})$ of (18) in the
role of $\hat{\rho}$ of (12), we write (12) as

$H_{\bar{\cal L}|\bar{x}}=-\sum_{\bar{l}\in\\{\bar{\cal L}\\}}p_{\bar{\cal
L}|\bar{\cal X}}(\bar{l},\bar{x})\log(p_{\bar{\cal L}|\bar{\cal
X}}(\bar{l},\bar{x}))\geq S(\hat{\rho}({\bar{x}})).$

Finally, by separability of the support subspaces (17), we see that the above
inequality is saturated to equality:

$H_{\bar{\cal L}|\bar{x}}=S(\hat{\rho}({\bar{x}})).$

From the above equation one promptly recognizes that

$\sum_{\bar{x}\in\\{\bar{\cal X}\\}}p_{\bar{\cal
X}}(\bar{x})S(\hat{\rho}({\bar{x}}))=H_{\bar{\cal L}|\bar{\cal X}}.$ (20)

Substituting (18) in (14), the density operator $\hat{\rho}$ of (14) results

$\displaystyle\hat{\rho}$ $\displaystyle=\sum_{\bar{x}\in\\{\bar{\cal
X}\\}}p_{\bar{\cal X}}(\bar{x})\sum_{\bar{l}\in\\{\bar{\cal L}\\}}p_{\bar{\cal
L}|\bar{\cal X}}(\bar{l},\bar{x})\hat{\rho}({\bar{l}})$
$\displaystyle=\sum_{\bar{l}\in\\{\bar{\cal L}\\}}p_{\bar{\cal
L}}(\bar{l})\hat{\rho}({\bar{l}})\sum_{\bar{x}\in\\{\bar{\cal
X}\\}}p_{\bar{\cal X}|\bar{\cal L}}(\bar{x},\bar{l})$
$\displaystyle=\sum_{\bar{l}\in\\{\bar{\cal L}\\}}p_{\bar{\cal
L}}(\bar{l})\hat{\rho}({\bar{l}}),$ (21)

leading to

$S(\hat{\rho})=H_{\bar{\cal L}}.$ (22)

Substituting (20) and (22) in (12) we write

$\displaystyle H_{\bar{\cal X}}$
$\displaystyle=S(\hat{\rho})-\sum_{\bar{x}\in\\{\bar{\cal X}\\}}p_{\bar{\cal
X}}(\bar{x})S(\hat{\rho}({\bar{x}}))$ $\displaystyle=H_{\bar{\cal
L}}-H_{\bar{\cal L}|\bar{\cal X}}$ $\displaystyle=I_{\bar{\cal L};\bar{\cal
X}}$ $\displaystyle=\chi,$

where the first equality is due to separability of the support subspaces of
the set $\\{\hat{\rho}({\bar{x}})\\}$. Note that the specific form (18) that
we take for the mixed states of the preparation forces $H_{\bar{\cal
X}|\bar{\cal L}}=0$, which means that the preparation is known given
$\bar{\cal L}$. In the end, by the assumptions (17) and (18), the Holevo
accessible information is equal to the classical information $I_{\bar{\cal
L};\bar{\cal X}}$, which, since $H_{\bar{\cal X}|\bar{\cal L}}=0$, is equal to
the classical entropy $H_{\bar{\cal X}}$. In the rest of this paper we take
the role of Bob, the experimental physician, and, invoking Landauer’s
principle, we identify system’s physical entropy with logical information,
both classical, where information is the mutual information, and quantum,
where information is the Holevo accessible information, the two being, by the
above sequence of equalities, equal between them and equal to the classical
entropy $H_{\bar{\cal X}}$ of the random vector of the preparation of the
quantum system.

Let us now consider the quantum thermodynamical framework of [6, 7]. In this
framework, the system of interest, which, in the quantum information-theoretic
framework, is the system received by Bob, is prepared by the interactions
between the particles of the system and between the particles of the system
and the environment. The particles of the system that Bob receives can be
entangled between them and can be entangled with the environment from which
the system is separated, but Bob is completely unaware of past interactions,
therefore he cannot guess the state of the $i$-the particle, represented by
the random variable ${\cal L}_{i}$, from the result ${\cal L}_{k}$ of the
measurement that he makes on the $k$-th particle of the system. Also, Bob
cannot make measurements on the environment that prepared the system. Bob
could improve his capability of guessing ${\cal L}_{i}$ given ${\cal L}_{k}$
if he would know the distribution $p_{{\cal L}_{i}|{\cal L}_{k}}$, but, in
absence of any knowledge of this distribution, he must use in place of
$p_{{\cal L}_{i}|{\cal L}_{k}}$ a tentative distribution $q_{{\cal L}_{i}}$
that is independent of ${\cal L}_{k}$. In other words, Bob’s ignorance of the
past is imported into system model by the assumption of independency between
${\cal L}_{i}$ and ${\cal L}_{k}$. To see which is the best tentative
distribution, we can consider the Kullback-Leibler divergence $D(p_{{\cal
L}_{i}|{l}_{k}}||q_{{\cal L}_{i}})$ as a measure of quality of the tentative
distribution used by Bob,

$\displaystyle D(p_{{\cal L}_{i}|{l}_{k}}||q_{{\cal
L}_{i}})=\sum_{l_{i}\in\\{{\cal L}_{i}\\}}p_{{\cal L}_{i}|{\cal
L}_{k}}(l_{i},l_{k})\log\left(\frac{p_{{\cal L}_{i}|{\cal
L}_{k}}(l_{i},l_{k})}{q_{{\cal L}_{i}}(l_{i})}\right)\geq 0,$

average it over all the possible outcomes of ${\cal L}_{k}$,

$\displaystyle D(p_{{\cal L}_{i}|{\cal L}_{k}}||q_{{\cal L}_{i}})=$
$\displaystyle\sum_{l_{k}\in\\{{\cal L}_{k}\\}}\sum_{l_{i}\in\\{{\cal
L}_{i}\\}}p_{{\cal L}_{k}}(l_{k})p_{{\cal L}_{i}|{\cal
L}_{k}}(l_{i},l_{k})\log\left(\frac{p_{{\cal L}_{i}|{\cal
L}_{k}}(l_{i},l_{k})}{q_{{\cal L}_{i}}({l_{i}})}\right),$

and take the tentative distribution that minimizes the above divergence. Using
Jensen inequality, one immediately realizes that the best distribution is

$q_{{\cal L}_{i}}=p_{{\cal L}_{i}}.$

Independency of the random variables ${\cal L}_{1},{\cal L}_{2},\cdots,{\cal
L}_{N},$ and separability of the Hilbert support space are synthesized in the
separability of the density operator $\hat{\rho}$ (21) associated to the
random vector $\bar{\cal L}$:

$\hat{\rho}=\hat{\rho}_{1}\otimes\hat{\rho}_{2}\otimes\cdots\otimes\hat{\rho}_{N},$
(23)

where the subscript i below the density operators identifies the $i$-th
Hilbert subspace, so

$\hat{\rho}_{i}=\sum_{l\in\\{{\cal
L}_{i}\\}}p_{{\cal{L}}_{i}}(l)\ket{{{l}}}_{ii}\bra{{{l}}},\ \ i=1,2,\cdots,N,$

is the density operator operating on the subspace of the $i$-th particle.
Equation (23) means that particles, besides being non interacting, are also
non entangled. The algebraic implications of (23) have been widely studied in
the past, see for instance [30], the more recent [31] and the bibliography
therein. Besides independency, we will also assume that the random variables
of the random vector $\bar{\cal L}$ are identically distributed, therefore

$\hat{\rho}_{i}=\sum_{l\in\\{{\cal
L}\\}}p_{{\cal{L}}}(l)\ket{{{l}}}_{ii}\bra{{{l}}},\ \ i=1,2,\cdots,N.$ (24)

This seems to be a physically sound assumption for certain systems at the
thermal equilibrium, for instance, a system of monochromatic harmonic
oscillators or the particles of a gas in a container.

To recap, our assumptions of separability of Hilbert system’s space into the
individual particle’s Hilbert subspaces and of i.i.d. random variables for the
discrete random vector $\bar{\cal L}$ of particles’ energy levels are mapped
into a density operator $\hat{\rho}$ of (21) that satisfies equations (23) and
(24). Moreover, the scope of the investigation is restricted to a preparation
of the type (18) and (19). In the following, different cases for the random
vector $\bar{\cal X}$ of the preparation will be considered. In the considered
cases, we will analyze the entropy $H_{\bar{\cal X}}$, which, under the
mentioned assumptions and restrictions, is equal to the classical mutual
information and to the Holevo accessible information, of quantum systems of
particles whose quantized energies are those of the quantum harmonic
oscillator (15) and those of the particle in a box (16).

##  IV Indexed particles, $\bar{\cal X}=\bar{\cal L}$

We use the term indexed in place of the more common distinguishable because we
look forward to the case of non-indexed particles, that we will introduce in
one of the coming sections, where we assign to (non-)indexed a different
meaning than to (in)distinguishable. With ”indexed” we mean that $N$ indexes
can permanently associated with a one-to-one correspondence to the $N$
particles. The permanent association is possible, for instance, when particles
occupy distinct portions of space in a solid where particles are at fixed
points, e.g. the vertexes of a regular lattice. In this case, Bob can try to
recognize the preparation of the individual indexed particles, making an
indexed list of his results. When the listed results are actually error free,
the information extracted by Bob is equal to $H_{\bar{\cal L}}$ nats,
therefore for the classical random vector ${\bar{\cal X}}$ of the preparation
we take

${\bar{\cal X}}=\bar{\cal L}$

and the density operators of the prepared states (18) are

$\hat{\rho}({\bar{x}})=\hat{\rho}({\bar{l}}).$

Vector $\bar{l}$ completely describes the system particle-by-particle, hence
at the microscopic level. For this reason, in statistical mechanics and
thermal physics, $\bar{l}$ is called the microstate of the system, see for
instance 1.1. of [21]. Distinct microstates of the indexed system are obtained
by permuting any two elements of the vector, unless the values of the two
permuted elements are the same, in which case the permutation does not lead to
a distinct microstate.

### Information-theoretic typicality

When the $N$ random variables of the random vector $\bar{\cal L}^{N}$ are
i.i.d. we have

$\displaystyle\lim_{N\rightarrow\infty}\frac{1}{N}H(\bar{\cal L}^{N})$
$\displaystyle=\lim_{N\rightarrow\infty}-\frac{1}{N}\log(p_{\bar{\cal
L}^{N}}(\bar{\cal L}^{N}))$ (25)
$\displaystyle=\lim_{N\rightarrow\infty}-\frac{1}{N}\sum_{i=1}^{N}\log(p_{{\cal
L}_{i}}({\cal L}_{i}))$ (26)
$\displaystyle=\lim_{N\rightarrow\infty}-\frac{1}{N}\sum_{i=1}^{N}\log(p_{{\cal
L}}({\cal L}_{i}))$ (27) $\displaystyle=H_{{\cal L}},$ (28)

where (26) is the assumption of independency, (27) is the assumption of
identical distribution, and (28) is the law of large numbers. The convergence
of the limit (25) to (28) is in various senses. Our main concern is the case
of large but finite $N$, hence we consider convergence in probability, that is

$\lim_{N\rightarrow\infty}Pr\left(\left|\frac{1}{N}\log(p_{\bar{\cal
L}^{N}}(\bar{\cal L}^{N}))+H_{{\cal L}}\right|>\eta\right)=0,$ (29)

for every $\eta>0$. The above limit is a pillar of classical information
theory. In chapter 3 of their book [19], Cover and Thomas call it the
Asymptotic Equipartition Property (AEP).222Here it is the probability that is
equally partitioned, not the energy, hence this AEP has nothing to do with the
classical energy equipartition property. In the context of ergodic theory,
many authors refer to (29) as to the Shannon-McMillan theorem. Almost
everywhere convergence, that strengthens convergence in probability, is known
as the Shannon-McMillan-Breiman theorem.

Starting from (29) we arrive at the concept of typical set $\\{\bar{\cal
T}_{\eta}^{N}\\}$, which is a subset of the set $\\{\bar{\cal L}^{N}\\}$,

$\\{\bar{\cal T}_{\eta}^{N}\\}\subseteq\\{\bar{\cal L}^{N}\\},$ (30)

whose definition is based on the random probability (6). Specifically, the
typical set is made by those random vectors whose random probability, for
every $\eta$ and for sufficiently large $N$, is a random number that lies in
the narrow range

$e^{-N(H_{{\cal L}}+\eta)}\leq p_{\bar{\cal L}^{N}}(\bar{\cal
T}^{N}_{\eta})\leq e^{-N(H_{{\cal L}}-\eta)}.$ (31)

The properties of the typical set are that the number of its elements is in
the narrow range

$e^{N(H_{{\cal L}}+\eta)}\geq|\bar{\cal
T}^{N}_{\eta}|\geq(1-\eta)e^{N(H_{{\cal L}}-\eta)}$ (32)

and that the probability that the outcome of the random vector $\bar{\cal
L}^{N}$ belongs to the typical set is

$Pr(\bar{\cal L}^{N}\in\\{\bar{\cal T}^{N}_{\eta}\\})>1-\eta,$ (33)

see again [19]. Using the language of statistical mechanics, we can look at
the elements of the typical set as at the accessible microstates. For $N$
large enough, inequalities (31)-(33) show that their number is nearly
$e^{NH_{{\cal L}}}$ and that their overwhelming majority is nearly
equiprobable. Compared to the standard approach, where microstates are either
accessible (with uniform probability) or non-accessible (zero probability of
access), here all the microstates that have non-zero probability are in
principle accessible, but with different probabilities.

Paper [6] claims in the abstract that …the main postulate of statistical
mechanics, the equal a priori probability postulate, should be abandoned as
misleading and unnecessary. This claim is substantiated by an argument based
on the law of large numbers, which, as such, shares many similarities with the
information-theoretic concept of typicality. We completely agree with the
above quotation and, more generally, we find that our results, including those
to be presented in the following, are compatible with those of [6] and of [7],
which presents arguments and reaches results similar to [6]. Besides this, we
must add that the system model of the two cited papers is substantially
different from ours. In [6, 7], the system of interest is entangled with the
environment and the analysis is based on the idea that the union of system and
environment, the universe of [6], instead of being, as postulated in the
standard approach, in a mixed state made by equiprobable pure states all with
the same specified macro attributes, that in statistical mechanics is the so-
called microcanonical ensemble, for instance, all with the same total energy,
is in a pure state that has the same macro attributes of the systems that form
the microcanonical ensemble. Our approach is different because we firmly keep
the position of Bob, the experimental physician, who is not concerned with the
state of the universe, but only with the state of the system that he is going
to measure. In his life, Bob has always considered and will always consider
only systems whose macro attributes are unknown to him before his measurement.
For instance, in the canonical case, Bob knows the temperature of the heath
bath that prepares the system, but nobody tells him the total energy of the
system that he is receiving. Actually, the system could be made also of only
one particle, in which case, if the total energy were known, the system would
be completely known. As such, by Landauer’s principle and consistently with
the notion of information-theoretic entropy, it would have zero physical
entropy, and, in fact, there would be no reason for Bob of making experiments
and for us of writing this paper. The example of a system made by one particle
shows that the ultimate theoretical foundation of our approach, that puts
together the AEP and Landauer’s principle, is (quantum) information theory,
where ”quantum” is inside the round brackets because, thanks to separability
of system’s Hilbert support space (17), in this paper we treat the statistical
mechanics of the system by the classical random entropy and by its
expectation. Note that Bob’s ignorance of macro attributes is inherently
imported inside the system model by the assumption of independency between
particles (23). In this perspective, the microcanonical ensemble is for Bob a
metaphysical abstraction that has no relation with the physics that he can
experience, therefore, being an experimental physician, he prefers to leave
its consideration to others.

### Macrostates

A macrostate of the system is the union of all the microstates that have the
same macroscopic attributes, where with macroscopic attribute we mean the
random sum ${\cal M}_{\Sigma}$ over all the particles of a deterministic
function $m(\cdot)$ of the random energy level:

${\cal M}_{\Sigma}=\sum_{i=1}^{N}m({\cal L}_{i}).$

Since, in our model, the energy levels are i.i.d. random variables,
macrostates, being deterministic functions of random variables, are themselves
random variables. We hereafter consider the occupancy macrostate, which is
defined as follows. Given a random vector $\bar{\cal L}$ of energy levels, the
$l$-th random occupancy number ${\cal N}_{\Sigma}(l)$ is defined as the number
of elements of $\bar{\cal L}$ whose value is $l$:

${\cal
N}_{\Sigma}(l)\stackrel{{\scriptstyle\text{def}}}{{=}}\sum_{i=1}^{N}\delta(l-{\cal
L}_{i}),\ \ \forall\ l\ \in\\{{\cal L}\\},$

where, since we consider a fixed number of particles, the constraint

$\sum_{l\in\\{{\cal L}\\}}{\cal N}_{\Sigma}(l)=N$ (34)

is imposed here and in what follows, unless explicitly stated. Let us assume
that the size $|{\cal L}|$ of the set of energy level is finite,333When the
number of energy eigenvalues is infinite, to obtain a vector of occupancy
numbers with a finite number of elements we truncate the set $\\{{\cal L}\\}$
to a subset $\\{{\cal L}^{\prime}\\}$ with finite number of elements. The
truncation is such that  $Pr({\cal L}\not\in\\{{\cal L}^{\prime}\\})$ is small
enough for our purposes. When the event ${\cal L}\not\in\\{{\cal
L}^{\prime}\\}$ happens, we force ${\cal L}$ to the closest energy level in
$\\{{\cal L}^{\prime}\\}$. This transformation of ${\cal L}$ produces a new
random vector ${\cal L}^{\prime}$ that we consider in place of ${\cal L}$. The
probability distribution of ${\cal L}^{\prime}$ and the occupancy numbers
follow in a straightforward way. let $\bar{\cal N}_{\Sigma}$ denote the
generic vector of the $|{\cal L}|$ occupancy numbers, and let the random
occupancy macrostate associated to $\bar{\cal N}_{\Sigma}$ be the set

$\displaystyle\\{\bar{l}(\bar{\cal N}_{\Sigma})\\}=\\{\bar{l}_{1}(\bar{\cal
N}_{\Sigma}),\bar{l}_{2}(\bar{\cal
N}_{\Sigma}),\cdots,\bar{l}_{W_{M}(\bar{\cal N}_{\Sigma})}(\bar{\cal
N}_{\Sigma})\\},$ (35)

where $\bar{l}_{1}(\bar{\cal N}_{\Sigma})$ is a vector whose occupancy numbers
are the elements of $\bar{\cal N}_{\Sigma}$, the subscript i indicates the
$i$-th distinct permutation of the $N$ elements of $\bar{l}_{1}(\bar{\cal
N}_{\Sigma})$,444As in the case of microstates, also here the permutation of
two particles with the same energy is not a distinct permutation. and the
number of elements $W_{M}(\bar{\cal N}_{\Sigma})$ of the set of distinct
permutations is the multinomial coefficient,

$W_{M}(\bar{\cal N}_{\Sigma})=\frac{N!}{\prod_{l\in\\{{\cal L}\\}}{\cal
N}_{\Sigma}(l)!},$ (36)

where we put the subscript M to emphasize that $W_{M}$ is the Multinomial
coefficient and the constraint (34) makes the multinomial coefficient
dependent only of the vector of the random occupancy numbers.

The random macrostate $\bar{\cal N}_{\Sigma}$ of the occupancy numbers is the
most detailed macrostate, because it originates any other random macrostate as

${\cal M}_{\Sigma}=\sum_{i=1}^{N}m({\cal L}_{i})=\sum_{l\in\\{{\cal
L}\\}}{\cal N}_{\Sigma}(l)m(l),$ (37)

where the first equality gives the random macrostate in the form of a random
sum, while the second equality can be seen as the random version of the
expectation of $m({\cal L})$, with $\bar{\cal N}_{\Sigma}$ in the role of
”random” probability distribution multiplied by the number of particles. The
probability of a macrostate is the probability that comes out a microstate
that belongs to the macrostate, hence the probability of the macrostate can be
identified with the probability of the sum:

$\displaystyle Pr(\bar{\cal L}\in\\{\bar{l}({\cal M}_{\Sigma})\\})=p_{{\cal
M}_{\Sigma}}({m}_{\Sigma}),$

where $m_{\Sigma}$ spans the support set of ${\cal M}_{\Sigma}$. The set of
vectors belonging to $\\{\bar{l}({\cal M}_{\Sigma})\\}$ can be partitioned
into non-overlapping subsets of the type (35), hence the probability
distribution of the random macrostate is

$p_{{\cal M}_{\Sigma}}(m_{\Sigma})=\sum_{\bar{n}_{\Sigma}:\sum_{l\in\\{{\cal
L}\\}}{n}_{\Sigma}(l)m(l)=m_{\Sigma}}p_{\bar{\cal
N}_{\Sigma}}(\bar{n}_{\Sigma}),$ (38)

where $\bar{n}_{\Sigma}$ spans the support set of $\bar{\cal N}_{\Sigma}$,
hence it is subject to the constraint

$\sum_{l\in\\{{\cal L}\\}}{n}_{\Sigma}(l)=N.$

The random number of microstates belonging to the random macrostate is

$W({\cal M}_{\Sigma},N)=\sum_{\bar{\cal N}_{\Sigma}:\sum_{l\in\\{{\cal
L}\\}}{\cal N}_{\Sigma}(l)m(l)={\cal M}_{\Sigma}}W_{M}(\bar{\cal
N}_{\Sigma}).$ (39)

### Multinomial distribution of the occupancy numbers for fixed number of
particles

A consequence of the i.i.d. assumption that we make for the random energy
levels is that the probability distribution of the random vector $\bar{\cal
N}_{\Sigma}$ is the multinomial distribution:

$\displaystyle p_{\bar{\cal N}_{\Sigma}}(\bar{n}_{\Sigma})$
$\displaystyle=W_{M}(\bar{n}_{\Sigma})\prod_{l\in\\{{\cal L}\\}}(p_{\cal
L}(l))^{{n}_{\Sigma}(l)}.$ (40)

To our best knowledge, it is observed here for the first time that the
distribution of the occupancy numbers is the multinomial distribution (40).
Although the multinomial coefficient is a recurring figure in statistical
mechanics since the times of Boltzmann, the term $\prod_{l\in\\{{\cal
L}\\}}(p_{\cal L}(l))^{n_{\Sigma}(l)}$ that appears in (40) is not present in
Boltzmann neither, to our best knowledge, in the subsequent literature, at
least with the meaning that it has in (40). It could seem obvious, but it is
worth remarking that, while the multinomial coefficient is a number, the
product between the multinomial coefficient and the probability distribution
$\\{\prod_{l\in\\{{\cal L}\\}}(p_{\cal L}(l))^{n_{\Sigma}(l)}\\}$ is a
probability distribution! In the open literature, the multinomial distribution
is considered only when the probability distribution $p_{\cal L}$ is uniform.
In this case, see e.g. [33, 32] and II.5 of [34], the uniform distribution of
course is not the distribution of energy levels. The multinomial distribution
appears in eqn. (8) of 6.2 of [21] with the same meaning as here, but in the
cited reference it appears only in a passage of a mere algebraic manipulation
of a partition function that allows to calculate the partition function itself
by the help of the multinomial theorem. Actually, the authors of [21] seem not
to give to the multinomial distribution any other meaning than that of a term
involved in an algebraic manipulation. In [21], eqn. 6.3.12, the authors claim
that the distribution of the occupancy numbers in the canonical ensemble is
the Poisson distribution. However, the occupancy numbers cannot follow the
Poisson distribution if the total number of particles is deterministic, as it
actually is in the canonical ensemble, hence this claim is not compatible with
the constraint on the total number of particle of eqn. 6.2.3 of the same book.
An interesting use of the multinomial distribution is made in [35], but, in
that paper, entropy is defined in the phase space and the multinomial
distribution is used to describe the occupancy of space, not of the energy
levels.

### Conditional equiprobability of microstates given the occupancy numbers

Since

$\prod_{l\in\\{{\cal L}\\}}(p_{\cal L}(l))^{{n}_{\Sigma}(l)}=p_{\bar{\cal
L}}(\bar{l}),\ \ \ \ \forall\ \bar{l}\in\\{\bar{l}(\bar{n}_{\Sigma})\\},$

equation (40), that holds with fixed number of particles, can be written as

$\displaystyle p_{\bar{\cal L}}(\bar{l})$
$\displaystyle=(W_{M}(\bar{n}_{\Sigma}))^{-1}p_{\bar{\cal
N}_{\Sigma}}(\bar{n}_{\Sigma}),\ \ \ \ \forall\
\bar{l}\in\\{\bar{l}(\bar{n}_{\Sigma})\\},$ (41)

which shows that microstates’ probability depends only on the random occupancy
numbers, or, in other words, that microstates are equiprobable given the
random occupancy numbers. We express this concept by saying that microstates
are conditionally equiprobable given the random occupancy numbers.

To obtain a formal expression of conditional equiprobability we must use
conditional probability distributions. Since $\bar{\cal N}_{\Sigma}$ is known
given $\bar{\cal L}$, the conditional probability distribution of $\bar{\cal
N}_{\Sigma}$ given $\bar{\cal L}$ is

$\displaystyle p_{\bar{\cal N}_{\Sigma}|\bar{\cal
L}}(\bar{n}_{\Sigma},\bar{l})=\left\\{\begin{array}[]{cc}1,&\bar{l}\in\\{\bar{l}(\bar{n}_{\Sigma})\\},\\\
0,&\bar{l}\not\in\\{\bar{l}(\bar{n}_{\Sigma})\\},\\\ \end{array}\right.$ (44)

hence, by Bayes formula,

$\displaystyle p_{\bar{\cal L}|\bar{\cal
N}_{\Sigma}}(\bar{l},\bar{n}_{\Sigma})=\left\\{\begin{array}[]{cc}\frac{p_{\bar{\cal
L}}(\bar{l})}{p_{\bar{\cal
N}_{\Sigma}}(\bar{n}_{\Sigma})},&\bar{l}\in\\{\bar{l}(\bar{n}_{\Sigma})\\},\\\
0,&\bar{l}\not\in\\{\bar{l}(\bar{n}_{\Sigma})\\}.\\\ \end{array}\right.$ (47)

Substituting (41) in (47) we get

$\displaystyle p_{\bar{\cal L}|\bar{\cal
N}_{\Sigma}}(\bar{l},\bar{n}_{\Sigma})=\left\\{\begin{array}[]{cc}\frac{1}{W_{M}(\bar{n}_{\Sigma})},&\bar{l}\in\\{\bar{l}(\bar{n}_{\Sigma})\\},\\\
0,&\bar{l}\not\in\\{\bar{l}(\bar{n}_{\Sigma})\\},\\\ \end{array}\right.$ (50)

which shows that microstates are conditionally equiprobable given the
occupancy numbers. Note that conditional equiprobability is here demonstrated
starting from the i.i.d. assumption of the elements of $\bar{\cal L}$.

### Poisson distribution of the occupancy numbers for random number of
particles

Let us consider the case where the system can randomly exchange particles with
the environment. This happens, for instance, when the system, e.g. a gas,
occupies a region of space that is not bounded by impenetrable walls, so that
particles can freely come in and out of the region of interest. In this case,
the total number of particles, that we call ${\cal N}_{\Sigma}$, is the
following random variable:

$\displaystyle{\cal N}_{\Sigma}=\sum_{l\in\\{{\cal L}\\}}{\cal
N}_{\Sigma}(l).$

The probability distribution of the occupancy numbers is

$\displaystyle p_{\bar{\cal N}_{\Sigma}}(\bar{n}_{\Sigma})$
$\displaystyle=p_{\bar{\cal N}_{\Sigma},{\cal
N}_{\Sigma}}(\bar{n}_{\Sigma},{n}_{\Sigma})$ $\displaystyle=p_{{\cal
N}_{\Sigma}}({n}_{\Sigma})p_{\bar{\cal N}_{\Sigma}|{\cal
N}_{\Sigma}}(\bar{n}_{\Sigma},{n}_{\Sigma})$ $\displaystyle=p_{{\cal
N}_{\Sigma}}({n}_{\Sigma}){n}_{\Sigma}!\prod_{l\in\\{{\cal
L}\\}}\frac{(p_{\cal L}(l))^{{n}_{\Sigma}(l)}}{{n}_{\Sigma}(l)!},$ (51)

where the first equality holds because ${\cal N}_{\Sigma}$ is a deterministic
function of $\bar{\cal N}_{\Sigma}$, in the third equality the multinomial
distribution is substituted to the probability distribution of the occupancy
numbers given the total number of particles and

$\displaystyle n_{\Sigma}=\sum_{l\in\\{{\cal L}\\}}n_{\Sigma}(l).$ (52)

For the conditional probability of microstates given the occupancy numbers we
have the same result as before:

$\displaystyle p_{\bar{\cal L}|\bar{\cal
N}_{\Sigma}}(\bar{l},\bar{n}_{\Sigma})$ $\displaystyle=p_{\bar{\cal L},{\cal
N}_{\Sigma}|\bar{\cal N}_{\Sigma}}(\bar{l},\bar{n}_{\Sigma})$
$\displaystyle=\frac{p_{\bar{\cal L},{\cal
N}_{\Sigma}}(\bar{l},{n}_{\Sigma})p_{\bar{\cal N}_{\Sigma}|\bar{\cal L},{\cal
N}_{\Sigma}}(\bar{l},\bar{n}_{\Sigma})}{p_{\bar{\cal
N}_{\Sigma}}(\bar{n}_{\Sigma})}$ $\displaystyle=\frac{p_{\bar{\cal L}|{\cal
N}_{\Sigma}}(\bar{l},{n}_{\Sigma})p_{{\cal
N}_{\Sigma}}({n}_{\Sigma})p_{\bar{\cal N}_{\Sigma}|\bar{\cal
L}}(\bar{l},\bar{n}_{\Sigma})}{p_{\bar{\cal N}_{\Sigma}}(\bar{n}_{\Sigma})}$
$\displaystyle=\frac{p_{\bar{\cal L}|{\cal
N}_{\Sigma}}(\bar{l},{n}_{\Sigma})p_{\bar{\cal N}_{\Sigma}|\bar{\cal
L}}(\bar{l},\bar{n}_{\Sigma})}{W_{M}(\bar{n}_{\Sigma})\prod_{l\in\\{{\cal
L}\\}}(p_{\cal L}(l))^{{n}_{\Sigma}(l)}}$ (53)
$\displaystyle=\left\\{\begin{array}[]{cc}\frac{1}{W_{M}(\bar{n}_{\Sigma})},&\bar{l}\in\\{\bar{l}(\bar{n}_{\Sigma})\\},\\\
0,&\bar{l}\not\in\\{\bar{l}(\bar{n}_{\Sigma})\\},\\\ \end{array}\right.$ (56)

where the first three equalities hold because $n_{\Sigma}$ is a deterministic
function of $\bar{n}_{\Sigma}$ and $\bar{n}_{\Sigma}$ is a deterministic
function of $\bar{l}$, in (53) we use (51), in (56) we make use of (44) and of

$\prod_{l\in\\{{\cal L}\\}}(p_{\cal L}(l))^{{n}_{\Sigma}(l)}=p_{\bar{\cal
L}|{\cal N}_{\Sigma}}(\bar{l},{n}_{\Sigma}),\ \ \ \ \forall\
\bar{l}\in\\{\bar{l}(\bar{n}_{\Sigma})\\}.$

We assume that the position where a particle can be detected is a continuous
random variable with uniform distribution over a virtually unbounded region of
space that includes the region of interest and the surroundings. This
assumption is reasonable when the open systems of interest is made by the
particles detected in the region of interest and it is at the thermal
equilibrium with the surrounding environment. In this case, the open system of
interest is one element of the grand canonical ensemble. If, besides the
uniform distribution, we also assume that two particles cannot be detected in
the same position and that the positions where particles are detected are
independent random variables, then the random number ${\cal N}_{\Sigma}$ of
particles inside the region of interest follows the Poisson distribution,
therefore

$p_{{\cal
N}_{\Sigma}}(n_{\Sigma})=\frac{\lambda^{{n}_{\Sigma}}e^{-\lambda}}{{n}_{\Sigma}!},$

where $\lambda=\braket{{\cal N}_{\Sigma}}$, the expected number of particles
inside the region of interest, plays here the same role that $N$ plays in case
of fixed number of particles. We hereafter show that the random occupancy
numbers of the random vector $\bar{\cal N}_{\Sigma}$ are independent between
them. Independency between the occupancy numbers is not that obvious. For
instance, the occupancy numbers of the multinomial distribution are not
independent between them, because their sum is forced to be equal to a
deterministic number. Substituting the Poisson distribution for $p_{{\cal
N}_{\Sigma}}(n_{\Sigma})$ in (51) and then substituting (52) one gets

$\displaystyle p_{\bar{\cal
N}_{\Sigma}}(\bar{n}_{\Sigma})=\lambda^{\sum_{l\in\\{{\cal
L}\\}}n_{\Sigma}(l)}e^{-\lambda}\prod_{l\in\\{{\cal L}\\}}\frac{(p_{\cal
L}(l))^{{n}_{\Sigma}(l)}}{{n}_{\Sigma}(l)!}.$ (57)

Substituting

$p_{\cal L}(l)=\frac{\braket{{\cal N}_{\Sigma}(l)}}{\braket{{\cal
N}_{\Sigma}}}=\frac{\lambda(l)}{\lambda},$ (58)
$e^{-\lambda}=\prod_{l\in\\{{\cal L}\\}}e^{-\lambda({l})},$

in equation (57), one finds

$\displaystyle p_{\bar{\cal N}_{\Sigma}}(\bar{n}_{\Sigma})=\prod_{l\in\\{{\cal
L}\\}}\frac{(\lambda(l))^{{n}_{\Sigma}(l)}e^{-\lambda(l)}}{{n}_{\Sigma}(l)!}.$
(59)

The probability distribution (59) is the product of Poisson distributions with
parameters $\lambda(l)=\braket{{\cal N}_{\Sigma}(l)}$, hence the individual
random variables are independent between them. The Poisson distribution has
been used in the past in very many cases. A case that is worth mentioning
because it looks very close to ours is that of London’s bombing in the second
war, where the random number of bombs (in [53] the distributions of V-1 bombs
and of V-2 rockets, the ”flying bombs,” are analyzed) falling in a sub-area of
an area subject to almost uniform bombing is modelled by the Poisson
distribution. Here, in place of the random number of bombs of the two types we
have the random number of particles of different energies. The relationship
between the multinomial and the Poisson probability distributions and between
them and the canonical and grand canonical ensembles has been investigated in
[54]. The conclusion that, when the system exchanges particles with the
environment, the probability distribution of ${\cal N}_{\Sigma}(l)$ is
Poisson, is achieved, by different arguments, also in the standard treatment
of the classical ideal gas at high temperature, see for instance 21.2.4-21.2.6
of [27] where, however, independency between the random occupancy numbers is
not discussed. Here the Poisson distribution for the random number of
particles descends directly from the uniformity of the spatial distribution of
particles in the unbounded region of space that includes the region of
interest, an assumption that seems reasonable also at low temperature.

### Conditional entropy and mutual information

For the conditional random entropies we have

$\displaystyle H(\bar{\cal N}_{\Sigma}|\bar{\cal L})=0,$ $\displaystyle
H(\bar{\cal L}|\bar{\cal N}_{\Sigma})=\log(W_{M}(\bar{\cal N}_{\Sigma})),$

leading to

$\displaystyle H_{\bar{\cal N}_{\Sigma}|\bar{\cal L}}=0,$ $\displaystyle
H_{\bar{\cal L}|\bar{n}_{\Sigma}}=\log(W_{M}(\bar{n}_{\Sigma})),$
$\displaystyle H_{\bar{\cal L}|\bar{\cal
N}_{\Sigma}}=\sum_{\bar{n}_{\Sigma}\in\\{\bar{\cal N}_{\Sigma}\\}}p_{\bar{\cal
N}_{\Sigma}}(\bar{n}_{\Sigma})\log(W_{M}(\bar{n}_{\Sigma})).$

By (11) we have the following equality

$\displaystyle I_{\bar{\cal L};\bar{\cal N}_{\Sigma}}=H_{\bar{\cal
L}}-H_{\bar{\cal L}|\bar{\cal N}_{\Sigma}}=H_{\bar{\cal N}_{\Sigma}}\geq 0,$
(60)

where the inequality becomes equality only when $\bar{\cal N}_{\Sigma}$ is
non-random. The above inequality says that, while $H(\bar{\cal L})$ is an
unbiased estimator of the Gibbs entropy, $H(\bar{\cal L}|\bar{\cal
N}_{\Sigma})$ is not. We will find again (60) when we will discuss the entropy
of non-indexed particles.

Using Stirling’s formula and

$\lim_{N\rightarrow\infty}\frac{{\cal N}_{\Sigma}(l)}{N}=p_{\cal L}(l),$

it is straightforward to manipulate the random $W_{M}(\bar{\cal N}_{\Sigma})$
to show that

$\displaystyle\lim_{N\rightarrow\infty}\frac{\log(W_{M}(\bar{\cal
N}_{\Sigma}))}{N}$ $\displaystyle=H_{{\cal L}},$
$\displaystyle\lim_{N\rightarrow\infty}\frac{H_{\bar{\cal L}^{N}}-H_{\bar{\cal
L}^{N}|\bar{\cal N}_{\Sigma}}}{N}$
$\displaystyle=\lim_{N\rightarrow\infty}\frac{H_{\bar{\cal
N}_{\Sigma}}}{N}=0.$ (61)

Hence, for $N\rightarrow\infty$, the randomness of $\bar{\cal N}_{\Sigma}$
impacts only the conditional entropy, not the conditional entropy per
particle, which is actually equal to the entropy per particle.

More generally, any generic random macrostate $\bar{\cal M}_{\Sigma}$ is known
given $\bar{\cal N}_{\Sigma}$, therefore $H_{\bar{\cal M}_{\Sigma}|\bar{\cal
N}_{\Sigma}}=0$, leading to

$I_{\bar{\cal N}_{\Sigma};\bar{\cal M}_{\Sigma}}=H_{\bar{\cal
N}_{\Sigma}}-H_{\bar{\cal N}_{\Sigma}|\bar{\cal M}_{\Sigma}}=H_{\bar{\cal
M}_{\Sigma}}-H_{\bar{\cal M}_{\Sigma}|\bar{\cal N}_{\Sigma}}=H_{\bar{\cal
M}_{\Sigma}},$ $\displaystyle I_{\bar{\cal L};\bar{\cal
M}_{\Sigma}}=H_{\bar{\cal M}_{\Sigma}}=I_{\bar{\cal N}_{\Sigma};\bar{\cal
M}_{\Sigma}}.$

Since

$H_{\bar{\cal N}_{\Sigma}}\geq H_{\bar{\cal M}_{\Sigma}},$

also the entropy per particle of any macrostate becomes vanishingly small as
$N\rightarrow\infty$.

### Boltzmann’s number of complexions and Bose’s entropy

The combinatorial approach to entropy was introduced by Boltzmann in 1877, see
[36] for the English translation of Boltzmann’s paper, and become standard in
the following years, see for instance 18.1 and 19.1 of [27], 6.1 and 6.2 of
[21]. Boltzmann’s combinatorial approach, which is based on the assumption of
large $N$, rules out the randomness of the occupancy numbers and considers a
deterministic vector $\bar{n}$. Boltzmann denotes with ${\cal P}$ the
multinomial coefficient $W_{M}(\bar{n}_{\Sigma})$ and calls it the permutation
number. It counts what Boltzmann calls the number of complexions associated to
the vector $\bar{n}_{\Sigma}$ of the occupancy numbers. To our best knowledge
Bose was the first that used Boltzmann’s permutation number inside the
logarithm to calculate the entropy of a system of quantum harmonic
oscillators, see the English translation [37] of Bose’s paper. Both Boltzmann
and Bose consider the case of uniformly quantized energies,

$\epsilon(l)=lE,\ \ l=0,1,\cdots,\ \ E>0.$

From the formal point of view there is no difference between Boltzmann’s and
Bose’s quantization. Of course, while Boltzmann comes before the birth of
quantum physics and therefore uses quantization as a tool that allows him to
treat by combinatorics a classical, non-quantized, energy, Bose is aware of
Planck’s constant and of the quantization formula $E=\hbar\omega$ for the
energy of the quantum harmonic oscillator.

The deterministic $\log(W)$ that we introduced in (1) now can step again into
the picture. Due to conditional equiprobability of microstates, the random
conditional entropy with deterministic condition $\bar{n}_{\Sigma}$ becomes
deterministic and equal to the conditional entropy with the same deterministic
condition:

$H(\bar{\cal L}|\bar{n}_{\Sigma})=\log(W_{M}(\bar{n}_{\Sigma}))=H_{\bar{\cal
L}|\bar{n}_{\Sigma}}.$

Let us take for the deterministic occupancy numbers an approximation to the
expectation of the random entries of vector $\bar{\cal N}_{\Sigma}$, for
instance

$n_{\Sigma}(l)=round(Np_{\cal L}(l))=Np_{\cal L}(l)+{\cal O}(1),\forall\
l\in\\{{\cal L}\\},$ (62)

where $round(\cdot)$ is the rounding to the nearest integer and in the last
equality we use the big-${\cal O}$ notation. Putting the above deterministic
occupancy numbers in the multinomial coefficient (36), for the (random)
conditional entropy with deterministic condition we get

$\displaystyle H(\bar{\cal L}|\bar{n}_{\Sigma})=\log(W_{M}(\bar{n}_{\Sigma}))$
(63) $\displaystyle=N\log(N)-\sum_{l\in\\{{\cal
L}\\}}n_{\Sigma}(l)\log(n_{\Sigma}(l))+{\cal O}(\log(N))$ (64)
$\displaystyle=-N\sum_{l\in\\{{\cal L}\\}}p_{\cal L}(l)\log(p_{\cal
L}(l))+{\cal O}(\log(N))$ (65) $\displaystyle=H_{\bar{\cal L}}+{\cal
O}(\log(N)),$

where (64) is Stirling’s formula and (65), which is obtained substituting
(62), holds because ${\cal O}(\log(N))$ dominates ${\cal O}(1)$. We therefore
come to the conclusion that the deterministic entropy
$\log(W_{M}(\bar{n}_{\Sigma}))$ is a random conditional entropy with
deterministic condition that, thanks to the conditional equiprobability of
microstates, becomes deterministic. For large $N$, $\log(W(\bar{n}_{\Sigma}))$
is a good approximation to the Gibbs entropy, the accuracy of the
approximation being that of Stirling’s formula, if, for the occupancy numbers
$\bar{n}_{\Sigma}$ one takes the deterministic occupancy numbers (62) and for
$W_{M}(\bar{n}_{\Sigma})$ takes the corresponding multinomial coefficient. One
more time, we remark that all descends from the i.i.d. assumption, without any
need of assuming equiprobability. We will return on the Boltzmann-Bose entropy
formula (63) in the section devoted to the entropy of a system of
monochromatic quantum harmonic oscillators.

##  V Indexed particles at the thermal equilibrium

Suppose that the system is at the thermal equilibrium with a heath bath at
known temperature. Even if, besides the temperature and the number of
particles, other constraints are imposed, e.g., in the case of a gas, the
volume occupied by the particles, the constraints can be not enough to
uniquely define the probability distribution $p_{\cal L}$. In this situation,
it is widely accepted that the probability distribution $p_{\cal L}$ is the
one that maximizes system’s entropy given the quantization rule $\epsilon(l)$
and the few macro constraints just mentioned. Boltzmann was the first to
propose to maximize the multinomial coefficient discussed in the previous
section. After Boltzmann, Bose in [37] maximizes the same multinomial
coefficient for a system of quantum harmonic oscillators. The principle of
entropy maximization become an universally accepted standard after Jaynes
[17]. Since the $N$ entries of the random vector $\bar{\cal L}^{N}$ are
assumed to be i.i.d., the entropy of the system is the sum of the entropies of
the individual particles, therefore in the maximization we can consider the
one-particle entropy. Maximization can be worked out by the Lagrange
multipliers method, the Lagrangian being

$\displaystyle\Lambda$ $\displaystyle=-\sum_{l\in\\{{\cal L}\\}}p_{\cal
L}(l)\log(p_{\cal L}(l))-\beta\left(\sum_{l\in\\{{\cal
L}\\}}\epsilon(l)p_{\cal L}(l)-\mu_{\cal E}\right)$
$\displaystyle-\alpha\left(\sum_{l\in\\{{\cal L}\\}}p_{\cal L}(l)-1\right),$

where $\alpha$ and $\beta$ are the Lagrange multipliers, and $\mu_{\cal E}$ is
the expected energy per particle, which is often called $U/N$ in statistical
mechanics textbooks, where $U$ is system’s internal energy.

The Lagrange multiplier $\beta$ brings inside the probability distribution the
temperature constraint:

$\beta=(k_{B}T)^{-1},$

where $T$ is the temperature of the heat bath in Kelvin degrees. The partial
derivatives of the Lagrangian are

$\displaystyle\frac{\partial\Lambda}{\partial p_{\cal L}(l)}$
$\displaystyle=-\log(p_{\cal L}(l))-1-\beta\epsilon(l)-\alpha,\ \ \forall\
l\in\\{{\cal L}\\},$

the solution is therefore

$p_{\cal L}(l)=e^{-\beta\epsilon(l)-\alpha-1}.$ (66)

For $\alpha$ we have

$\displaystyle\sum_{l\in\\{{\cal L}\\}}e^{-\beta\epsilon(l)-\alpha-1}=1,$
$\displaystyle e^{\alpha+1}=\sum_{l\in\\{{\cal L}\\}}e^{-\beta\epsilon(l)}.$

The above sum takes the name of (one-particle) canonical partition function
and is usually indicated with the symbol $Z$:

$\displaystyle Z=e^{\alpha+1}=\sum_{l\in\\{{\cal L}\\}}e^{-\beta\epsilon(l)}.$
(67)

Substituting (67) in (66) one gets the Boltzmann distribution in the familiar
form

$p_{\cal L}(l)=\frac{1}{Z}e^{-\beta\epsilon(l)}.$ (68)

The Boltzmann distribution (68) can now be used to find the one-particle
random entropy and its expectation:

$\displaystyle H({\cal L})$ $\displaystyle=\log(Ze^{\beta\epsilon({\cal
L})}),$ $\displaystyle H_{{\cal L}}=\sum_{l\in\\{{\cal
L}\\}}(Ze^{\beta\epsilon(l)})^{-1}\log(Ze^{\beta\epsilon(l)}).$ (69)

We can manipulate the above two formulas as follows:

$\displaystyle H({\cal L})$ $\displaystyle=\log(Z)+\beta\epsilon({\cal L}),$
$\displaystyle H_{{\cal L}}=\log(Z)+\beta\mu_{\cal E},$ (70)

where

$\displaystyle\mu_{\cal E}=\frac{1}{Z}\sum_{l\in\\{{\cal
L}\\}}\epsilon(l)e^{-\beta\epsilon(l)}.$ (71)

If desired, $\beta$ can be found as a function of $\mu_{\cal E}$ by solving
for $\beta$ the transcendental equation (71). However, when the entropy-
temperature relation is the concern, there is no need of expressing $\beta$ as
a function of the expected energy per particle, it is enough to use directly
(69), or, equivalently, to substitute (71) in (70).

From the derivatives of the partition function one gets all the classical
thermodynamics of systems at the thermal equilibrium. For instance, from

$\displaystyle\frac{\partial\log(Z)}{\partial\beta}$
$\displaystyle=\frac{\partial\log(Z)}{\partial Z}\frac{\partial
Z}{\partial\beta}$ $\displaystyle=\frac{1}{Z}\left(-\sum_{l\in\\{{\cal
L}\\}}\epsilon(l)e^{-\beta\epsilon(l)}\right)$ $\displaystyle=-\mu_{\cal E},$

one has

$\displaystyle\frac{\partial H_{{\cal L}}}{\partial\mu_{\cal E}}$
$\displaystyle=\frac{\partial\log(Z)}{\partial\mu_{\cal E}}+\beta+\mu_{\cal
E}\frac{\partial\beta}{\partial\mu_{\cal E}}$
$\displaystyle=\frac{\partial\log(Z)}{\partial\beta}\frac{\partial\beta}{\partial\mu_{\cal
E}}+\beta+\mu_{\cal E}\frac{\partial\beta}{\partial\mu_{\cal E}}$
$\displaystyle=\beta.$

Also, it is easy to see that

$\displaystyle\frac{\partial F}{\partial T}$ $\displaystyle=-k_{B}H_{\cal L},$

where

$F\stackrel{{\scriptstyle\text{def}}}{{=}}-k_{B}T\log(Z)=\mu_{\cal
E}-k_{B}TH_{\cal L}$

is the Helmholtz free energy, in this case of one particle.

The probability distribution of microstates is

$\displaystyle p_{\bar{\cal L}}(\bar{l})$
$\displaystyle=\prod_{i=1}^{N}p_{{\cal L}}(l_{i})$
$\displaystyle=\prod_{i=1}^{N}\frac{e^{-\beta\epsilon(l_{i})}}{Z}$
$\displaystyle=\frac{e^{-\beta\sum_{i=1}^{N}\epsilon(l_{i})}}{Z^{N}}.$ (72)

For the random system entropy, from (72) we have

$H(\bar{\cal L})=N\log(Z)+\beta\sum_{i=1}^{N}\epsilon({\cal L}_{i}),$

whose expectation is

$H_{\bar{\cal L}}=N(\log(Z)+\beta\mu_{\cal E}).$

The Boltzmann distribution (72) for systems at the thermal equilibrium is one
of the central results of thermal physics and statistical mechanics. A
derivation very similar to ours but based on the constrained maximization of
the multinomial coefficient can be found, for instance, in 18.1 of [27] and in
3.2 of [21]. Another derivation of (72) can be obtained by separating the
system from the heath bath and by assuming equiprobability of microstates of
the heath bath after separation, see for instance 19.1 of [27], 3.1 of [21].

The case of thermal equilibrium can be seen as a special case of the general
case treated in the previous section. Specifically, we can figure out that, in
the quantum information-theoretic representation, Bob receives a system whose
particles are independently prepared by the heath bath, which plays in this
case the role of Alice, and therefore the distribution of the random vector
${\bar{\cal L}}$ of the preparation, in place of being a generic probability
distribution $p_{\bar{\cal L}}$, is the Boltzmann distribution (72).

### Conditional equiprobability

The probability distribution (72) is independent of the specific vector
$\bar{\epsilon}$, it depends only on the sum of its elements. Therefore, with
an argument analogous the one that led to (47), we write

$\displaystyle p_{\bar{\cal L}|{\cal
E}_{\Sigma}}(\bar{l},\epsilon_{\Sigma})=\left\\{\begin{array}[]{cc}\frac{Z^{-N}e^{-\beta\epsilon_{\Sigma}}}{p_{{\cal
E}_{\Sigma}}(\epsilon_{\Sigma})},&\bar{l}\in\\{\bar{l}(\epsilon_{\Sigma})\\},\\\
0,&elsewhere,\\\ \end{array}\right.$ (75)

where

$\epsilon_{\Sigma}=\sum_{i=1}^{N}\epsilon(l_{i})$

and $\\{\bar{l}(\epsilon_{\Sigma})\\}$ is the set of vectors whose total
energy is $\epsilon_{\Sigma}$. We see therefore that the conditional
probability distribution $p_{\bar{\cal L}|{\cal E}_{\Sigma}}$ is uniform given
any condition ${\epsilon}_{\Sigma}\in\\{{\cal E}_{\Sigma}\\}$ or,
equivalently, that microstates are conditionally equiprobable given the total
system energy, provided only that it belongs to the set $\\{{\cal
E}_{\Sigma}\\}$. Again, conditional equiprobability descends from (72) without
the need of any extra assumption.

A consequence of conditional equiprobability is that the number of elements of
the conditional random support set given any condition must be the inverse of
the uniform value assumed by the conditional probability, therefore

$\displaystyle p_{\bar{\cal L}|{\cal
E}_{\Sigma}}(\bar{l},\epsilon_{\Sigma})=\left\\{\begin{array}[]{cc}\frac{1}{W(\epsilon_{\Sigma})},&\bar{l}\in\\{\bar{l}(\epsilon_{\Sigma})\\},\\\
0,&elsewhere.\\\ \end{array}\right.$ (78)

The probability distribution of the total system random energy is

$\displaystyle p_{{\cal
E}_{\Sigma}}(\epsilon_{\Sigma})=\sum_{\bar{n}_{\Sigma}:\sum_{l\in\\{{\cal
L}\\}}{n}_{\Sigma}(l)\epsilon(l)=\epsilon_{\Sigma}}p_{\bar{\cal
N}_{\Sigma}}(\bar{n}_{\Sigma})$ (79)
$\displaystyle=\sum_{\bar{n}_{\Sigma}:\sum_{l\in\\{{\cal
L}\\}}{n}_{\Sigma}(l)\epsilon(l)=\epsilon_{\Sigma}}W_{M}(\bar{n}_{\Sigma})p_{\bar{\cal
L}}(\bar{l}),\ \bar{l}\in\\{\bar{l}(\epsilon_{\Sigma})\\},$ (80)
$\displaystyle=\left(Z^{-N}e^{-\beta\epsilon_{\Sigma}}\right)\sum_{\bar{n}_{\Sigma}:\sum_{l\in\\{{\cal
L}\\}}{n}_{\Sigma}(l)\epsilon(l)=\epsilon_{\Sigma}}W_{M}(\bar{n}_{\Sigma})$
(81) $\displaystyle=W(\epsilon_{\Sigma})Z^{-N}e^{-\beta\epsilon_{\Sigma}}$
$\displaystyle=W(\epsilon_{\Sigma})p_{\bar{\cal L}}(\bar{l}),\ \ \forall\
\bar{l}\ \in\\{\bar{l}(\epsilon_{\Sigma})\\},$

where (79) is (38), (80) is obtained from (41), and (81) is obtained by
substituting the Boltzmann distribution. Working out a closed form for the
number of conditionally equiprobable microstates, that depend on the
quantization rule $\epsilon(l)$, can be difficult. The same difficulty can be
found with the probability distribution $p_{{\cal E}_{\Sigma}}$.555For
$N\rightarrow\infty$, by the central limit theorem, the probability
distribution of the random sample mean $N^{-1}{\cal E}_{\Sigma}$ tends to a
normal probability density function with mean value $\mu_{\cal E}$ and
variance $N^{-1}\sigma^{2}_{\cal E}$, therefore, for large $N$, the
distribution $p_{{\cal E}_{\Sigma}}$ can be approximated to the discretization
with step $N^{-1}$ of the normal probability density function that
approximates the probability distribution of the random sample mean:
$p_{{\cal E}_{\Sigma}}(\epsilon_{\Sigma})\approx
N^{-1}\sqrt{\frac{N}{2\pi\sigma^{2}_{{\cal
E}}}}e^{-\frac{N(N^{-1}\epsilon_{\Sigma}-\mu_{{\cal
E}})^{2}}{2\sigma^{2}_{{\cal E}}}}.$ However, this is an approximation, not an
equality. See [38] for the Gaussian approximation of the energy distribution
in quantum thermodynamical systems. A remarkable exception is the quantum
harmonic oscillator, that, thanks to the uniform quantization rule, allows for
an elegant closed form expression of $p_{{\cal E}_{\Sigma}}$ that will be
presented in the following section.

### Macrostates are random conditions

Deterministic constraints imposed on the system must be distinguished from
macrostates. A deterministic constraint is the expectation of the
corresponding macrostate divided by the number of particles. The most detailed
deterministic constraint is the entire probability distribution,

$\displaystyle p_{\cal L}(l)$ $\displaystyle=\frac{1}{N}\braket{{\cal
N}_{\Sigma}(l)},\ \forall\ l\in\\{{\cal L}\\}.$

The probability distribution allows for the computation of any other
deterministic constraint, exactly as the occupancy numbers do with any other
random macrostate through (37). Our distinction between deterministic
constraint and macrostate is compatible with the Gibbsian view of a canonical
ensemble of systems, where the systems of the ensemble are assumed to be at
the thermal equilibrium at a given temperature and the number of particles is
the same for all the systems of the ensemble, therefore the deterministic
constraint is the Boltzmann distribution with a prescribed number of particles
and a prescribed expected energy per particle that depends on the temperature,
while the random macrostate is system’s total energy which, in the random
experiment of picking the system of interest from the ensemble, comes together
with the picked system. It should be added that, with the assumption of i.i.d.
particles, we don’t need an ensemble of systems: an ensemble of particles is
all we need if we build the system of interest by randomly picking $N$
particles from the ensemble of particles. This ensemble of particles is
equivalent to the Gibbsian canonical ensemble of systems and the two are
compatible with the time evolution of an ergodic system resulting from the
random interaction of the system with the surrounding environment.

Randomness of macrostates in the canonical ensemble is universally recognized
since the times of Maxwell. The randomness that is present in the total system
energy makes it possible that system’s energy fluctuates around its
expectation. One can figure out the extreme situation where a system made by
only one particle is in contact with the heath bath. Even if, occasionally,
the energy of the one-particle system is below (above) what expected, it can
get even lower (higher) if, in the ”contact” with the heath bath, the particle
of the system kicks a slower (faster) particle of the heath bath. Quoting from
[39]: As Maxwell points out, measurable thermodynamic quantities are averages
over many molecular quantities; if the molecular quantities exhibit
fluctuations that are probabilistically independent of each other, these
fluctuations will tend to be washed out as the number of molecules considered
is increased.

However, the consequences of this randomness seem not to be fully exploited in
the standard treatment of statistical mechanics, at least in the sense pursued
in this paper. Specifically, we cannot find equation (60), that for systems of
the canonical ensemble reads

$H_{\bar{\cal L}}=H_{\bar{\cal L}|{\cal E}_{\Sigma}}+H_{{\cal E}_{\Sigma}},$
(82)

in anyone of the many textbooks that we consulted, where, in fact, also the
conditional probability distribution $p_{\bar{\cal L}|{\cal E}_{\Sigma}}$ is
overlooked. All these textbooks deeply consider the canonical entropy
$H_{\bar{\cal L}}$ and the microcanonical entropy $H_{\bar{\cal
L}|{\epsilon}_{\Sigma}}$. The canonical system imports macrostate’s randomness
through the Boltzmann distribution, an approach that does not need to
explicitly show macrostate’s probability distribution and macrostate’s
entropy. In the microcanonical system, the ”given” macrostate
${\epsilon}_{\Sigma}$ is treated as a deterministic condition that is often
taken equal to the expectation of system’s energy. In the end, neither the
canonical approach nor the microcanonical approach explicitly analyze
macrostate’s probability distribution and, with it, macrostate’s entropy. This
has made impossible in the past, with the exception of [24] that we are going
to discuss, to explain with a simple formula like (82) the relation between
the canonical entropy and the microcanonical entropy and, more generally,
between the Gibbs entropy and the Boltzmann-Planck entropy.

We argue that the incomplete consideration of macrostates’ randomness
originates from a fallacy in the argument that supports the last part of the
passage of [39] quoted above. In thermodynamics, where $N$ is large, the
random sum that characterizes the random macrostate becomes virtually equal to
its expectation within a small relative random error, which, in the case of
the random total entropy, can be seen as the random version of the
deterministic $\eta$ that appears in (29)-(33). At the same time, for
$N\rightarrow\infty$, equation (61) shows that macrostate’s entropy per
particle becomes vanishingly small. At a first glance, this argument could
seem convincing because, physically speaking, one could think that the
relative error and the entropy per particle are what really matters to all
practical purposes. However, it remains that, for $N\rightarrow\infty$, the
random sum does not converge to any deterministic value. On the contrary, if
we represent the randomness of the random sum with its standard deviation, we
see that it increases with the square root of the number of particles.
Therefore the fallacy of this argument is that it is not true that this
randomness is washed out when $N\rightarrow\infty$, what is washed out is the
relative randomness. The distinction between absolute and relative randomness
is not a mere sophism or a subtle technicality that has no impact on reality.
First, the contribution of the term $H_{{\cal E}_{\Sigma}}$ to system’s
entropy $H_{\bar{\cal L}}$ can significantly impact the entropy of systems of
few particles, till to become the only contribution to the entropy of a system
made by one particle, for which macrostate and microstate coincide. Second, in
a forthcoming section of this paper we will show that randomness of
macrostates is the only randomness that contributes to the macroscopic entropy
of systems of a large number of non-indexed particles.

Paper [24] is the only one, among the very many papers that we studied during
this research, that fully catches randomness of macrostates in the sense
discussed above. Specifically, what the authors of [24] call the probability
distribution of the macroscopic state of the system and denote $P(a)$ can be
identified with our $p_{\bar{\cal N}_{\Sigma}}(\bar{n}_{\Sigma})$ and what
they call $W(a)$ can be identified with our $W_{M}(\bar{n}_{\Sigma})$. With
this identification, our $H(\bar{\cal L}|\bar{n}_{\Sigma})$ is what they call
Boltzmann entropy in their eqn. (6) and our $H_{\bar{\cal L}|\bar{\cal
N}_{\Sigma}}$ is what they call the mean value of the Boltzmann entropy in
their eqn. (7). Then they use this mean value of Boltzmann entropy in the
entropic relation (60). In our opinion, paper [24] is particularly important
because, to our best knowledge, it is the first paper that makes use in a
physical context of the concepts of random entropy and of conditional entropy
and that recognizes that Gibbs entropy $H_{\bar{\cal L}}$ is the sum of the
term $H_{\bar{\cal N}_{\Sigma}}$, which is due to the randomness of the
macrostate, and of the term $H_{\bar{\cal L}|\bar{\cal N}_{\Sigma}}$, which is
due to the randomness of the microstates whose union form the macrostate.
However, it should be added that the authors of [24] seem not to be aware that
their Boltzmann entropy is a conditional entropy. As a matter of fact, they do
not make any explicit use of conditional probability distributions. They hide
the conditional probability distribution $p_{\bar{\cal L}|\bar{\cal
N}_{\Sigma}}$ behind the corresponding multinomial coefficient
$W_{M}(\bar{n}_{\Sigma})$ and implicitly consider understood the equality
(44). Also, they claim but don’t prove that the probability distribution of
the occupancy numbers is multinomial, neither they use the multinomial
distribution, in fact they consider the Gaussian approximation in place of the
exact multinomial distribution. Finally, the big difference with our paper is
that the authors of [24] still assume that microstates are conditionally
equiprobable.

##  VI Indexed monochromatic quantum harmonic oscillators at the thermal
equilibrium

For the quantum harmonic oscillator (15) at the thermal state, the probability
distribution of the energy levels, the entropy, the partition function and the
energy-temperature ratio can be found in closed form by calculating the two
sums in (67) and (71). To our knowledge, calculus of (67) and (71) has been
proposed for the first time in [40] to work out the expected energy of the
oscillator and applied to the entropy in Section II of [41]. The partition
function (67) turns out to be a geometric series, whose calculation gives666As
in 3.8 of [21], for completeness we consider the energy of the ground state in
the calculations. However, as observed in 16.3 of [27] and in [41], adding the
energy of the ground state produces only a shift that does not impact neither
the probability distribution nor the entropy.

$Z=e^{-\frac{\beta\omega\hbar}{2}}\sum_{l=0}^{\infty}e^{-\beta
l\omega\hbar}=\frac{e^{-\frac{\beta\omega\hbar}{2}}}{1-e^{-\beta\omega\hbar}}.$
(83)

Using (83) in (68), we see that the probability distribution of the random
quantum number ${\cal L}$ of (15) is the geometric distribution:777Besides
(83), Einstein calculates in [2] the expected energy (hence the sum (71)) as
$\displaystyle\mu_{\cal E}$
$\displaystyle=\frac{\omega\hbar}{2}+\frac{\omega\hbar e^{-\frac{\beta
l\omega\hbar}{2}}}{Z}\sum_{l=0}^{\infty}le^{-\beta l\omega\hbar}$
$\displaystyle=\frac{\omega\hbar}{2}+\frac{\omega\hbar e^{-\frac{\beta
l\omega\hbar}{2}}}{Z}\frac{e^{\beta\omega\hbar}}{(e^{\beta\omega\hbar}-1)^{2}}$
$\displaystyle=\frac{\omega\hbar}{2}+\frac{\omega\hbar}{e^{\beta\omega\hbar}-1},$
(84) which, together with  $\mu_{\cal L}=\frac{\mu_{\cal
E}}{\omega\hbar}-\frac{1}{2},$ gives (87).

$\displaystyle p_{{\cal L}}(l)$ $\displaystyle=p(1-p)^{l},\ \ \ l=0,1,\cdots,$
(85)

with

$\displaystyle
p=1-e^{-\beta\omega\hbar}=\frac{e^{-\frac{\beta\omega\hbar}{2}}}{Z},$ (86)

see 7.5.6 of [42] for the derivation of (85) in a full quantum-mechanical
approach. The mean value and the variance of the geometric distribution are
known to be

$\displaystyle\mu_{\cal L}$
$\displaystyle=\frac{1-p}{p}=\frac{e^{-\beta\omega\hbar}}{1-e^{-\beta\omega\hbar}}=\frac{1}{e^{\beta\omega\hbar}-1},$
(87) $\displaystyle\sigma^{2}_{\cal L}$
$\displaystyle=\frac{1-p}{p^{2}}=\mu_{\cal L}(1+\mu_{\cal
L})=\frac{e^{-\beta\omega\hbar}}{(1-e^{-\beta\omega\hbar})^{2}}.$

At high temperature, where

$\displaystyle e^{\pm\beta\omega\hbar}\approx 1\pm\beta\omega\hbar,$ (88)

we have

$\sigma_{\cal L}\approx\mu_{\cal L}\approx\frac{1}{\beta\omega\hbar}.$

Taking the mean value as a parameter, the geometric distribution can be
expressed in the form

$\displaystyle p_{\cal L}(l)$ $\displaystyle=\frac{1}{1+\mu_{\cal
L}}\left(\frac{\mu_{\cal L}}{1+\mu_{\cal L}}\right)^{l},\ \ \ l=0,1,\cdots,$

leading, for the random entropy and for the entropy, to

$\displaystyle H({\cal L})$ $\displaystyle=\log(1+\mu_{\cal L})+{\cal
L}\log(1+\mu_{\cal L}^{-1}),$ $\displaystyle H_{{\cal L}}$
$\displaystyle=\log(1+\mu_{\cal L})+\mu_{\cal L}\log(1+\mu_{\cal L}^{-1}).$

Substituting the last term of (87) in place of $\mu_{\cal L}$ in the above
equation, the entropy-temperature relation results

$\displaystyle H_{{\cal L}}$
$\displaystyle=\log\left(\frac{1}{1-e^{-\beta\omega\hbar}}\right)+\frac{\beta\omega\hbar}{e^{\beta\omega\hbar}-1}.$
(89)

Substituting (88) in the above equation, at high temperature we have

$\displaystyle H_{{\cal L}}$
$\displaystyle\approx\log\left(\frac{1}{1-e^{-\beta\omega\hbar}}\right)+1.$
(90)

Equivalently, since at high temperature the contribution of the term
$\omega\hbar/2$ to the expected energy expressed in (84) becomes negligible,
we can write (84) as

$\mu_{\cal E}\approx\frac{\omega\hbar}{e^{\beta\omega\hbar}-1},$

and substitute it in (89), leading to

$\displaystyle H_{{\cal L}}$
$\displaystyle\approx\log\left(\frac{1}{1-e^{-\beta\omega\hbar}}\right)+\beta\mu_{\cal
E},$

which, compared to (90), shows that

$\beta\mu_{\cal E}\approx 1.$

The probability distribution of the random microstate $\bar{\cal L}$ of i.i.d.
random energy levels is the product of the marginal probability distributions
(85):

$\displaystyle p_{\bar{\cal L}}(\bar{l})=p^{N}(1-p)^{\sum_{i=1}^{N}{l}_{i}},$

where $p$ is given in (86). The random total quantum number ${\cal
L}_{\Sigma}$,

${\cal L}_{\Sigma}=\sum_{i=1}^{N}{\cal L}_{i},$

is the sum of $N$ i.i.d. geometric random variables, therefore its
distribution is the negative binomial distribution:

$\displaystyle p_{{\cal L}_{\Sigma}}(l_{\Sigma})$
$\displaystyle=W_{NB}({l}_{\Sigma},N)p^{N}(1-p)^{{l}_{\Sigma}}$ (91)
$\displaystyle=W_{NB}(l_{\Sigma},N)p_{\bar{\cal L}}(\bar{l}),\ \ \forall\
\bar{l}\ \in\\{\bar{l}(l_{\Sigma})\\},$

where ${l}_{\Sigma}$ spans the support set of ${\cal L}_{\Sigma}$ and

$\displaystyle W_{NB}({l}_{\Sigma},N)$
$\displaystyle=\frac{({l}_{\Sigma}+N-1)!}{{l}_{\Sigma}!(N-1)!}$ (92)

is the negative binomial coefficient. To our best knowledge, it is recognized
here for the first time that the probability distribution of the random total
quantum number ${\cal L}_{\Sigma}$ of a system of $N$ i.i.d. quantum harmonic
oscillators all oscillating with the same angular frequency is negative
binomial. The reader is referred to chapter 5 of [44] for the relation between
the negative binomial distribution and the geometric distribution and to [45]
for the entropy of the negative binomial distribution.

As the multinomial coefficient, also the negative binomial coefficient is a
recurring figure in statistical mechanics and quantum mechanics. It can be
calculated by the stars and bars method, which, as reported in [46], was
introduced in 1914 in a paper by Ehrenfest and Kamerlingh-Onnes and was later
brought by Feller [34] in a context different to physics. The stars and bars
method counts the number of distinct arrangements of ${l}_{\Sigma}$ stars
(energy quanta) into $N$ cells (oscillators) separated by $N-1$ bars plus two
additional bars at the beginning and at the end of the pattern. This number is
the number of distinct vectors $\bar{l}^{N}$ whose sum is ${l}_{\Sigma}$.
Equation (39) becomes

$W_{NB}(l_{\Sigma},N)=\sum_{\bar{n}_{\Sigma}:\sum_{l\in\\{{\cal
L}\\}}l{n}_{\Sigma}(l)={l}_{\Sigma}}W_{M}(\bar{n}_{\Sigma}).$ (93)

Boltzmann already pointed out that the negative binomial coefficient, that he
calls $J$ in [36], is the sum of all possible complexions.

### Conditional equiprobability and conditional entropy

Here the random condition is ${\cal L}_{\Sigma}$ and, with arguments analogous
to those that led to (50), for the conditional probability of microstates we
get

$\displaystyle p_{\bar{\cal L}|{\cal
L}_{\Sigma}}(\bar{l},l_{\Sigma})=\left\\{\begin{array}[]{cc}\frac{1}{W_{NB}(l_{\Sigma},N)},&\bar{l}\in\\{\bar{l}(l_{\Sigma})\\},\\\
0,&elsewhere,\\\ \end{array}\right.$ (96)

This conditional equiprobability has the same interpretation as (50). What
changes between the two is only the random condition and its probability
distribution. Since

$\displaystyle H({\cal L}_{\Sigma}|\bar{\cal L})=0,$ $\displaystyle H_{{\cal
L}_{\Sigma}|\bar{\cal L}}=0,$

the entropic relation analogous to (60) is

$\displaystyle H_{\bar{\cal L}}-H_{\bar{\cal L}|{\cal L}_{\Sigma}}=H_{{\cal
L}_{\Sigma}}\geq 0,$

where the inequality is satisfied with equality only when ${\cal L}_{\Sigma}$
is actually deterministic, that is, at the ground state, where ${\cal
L}_{\Sigma}=0$. By standard manipulations based on

$\frac{({\cal L}_{\Sigma}+N-1)!}{{\cal L}_{\Sigma}!(N-1)!}\approx\frac{({\cal
L}_{\Sigma}+N)!}{{\cal L}_{\Sigma}!N!}$

and on Stirling’s formula, for the conditional random entropy per particle one
finds

$\displaystyle\lim_{N\rightarrow\infty}\frac{H(\bar{\cal L}^{N}|{\cal
L}_{\Sigma})}{N}$
$\displaystyle=\lim_{N\rightarrow\infty}\frac{\log(W_{NB}({\cal
L}_{\Sigma},N))}{N}=H_{{\cal L}}.$

Also in this case, the randomness of the condition impacts only the
conditional entropy, not the conditional entropy per oscillator in a system of
a large number of oscillators.

### Boltzmann’s total number of complexions and Planck’s entropy

Planck takes Boltzmann’s total number of complexions (92), forcing inside it a
deterministic total quantum number ${l}_{\Sigma}$ that approximates the
expectation of the total random quantum number, for instance

${l}_{\Sigma}=round(N\mu_{\cal L})=N\mu_{\cal L}+{\cal O}(1),$ (97)

see [47], see also 16.3 of [27] and 3.8 of [21]. Then Planck, citing
Boltzmann, claims that the logarithm of $W_{NB}({l}_{\Sigma},N)$ with the
above $l_{\Sigma}$ is entropy, even if Boltzmann never assigned it the role of
entropy. In fact, nobody understands why it should be entropy, but,
undoubtedly, it works.

A question naturally arises. Bose’s entropy and Planck’s entropy are the
logarithm of two different numbers, how can they be the same entity? Luckily,
as Von Neumann said to Shannon, no one really knows what entropy really is. As
a consequence, everyone can feel free to put inside the logarithm whichever of
the two numbers he likes, provided that it works. In fact, in the
thermodynamical limit the two work. For large $N$ the two logarithms are both
a good approximation to the Gibbs entropy, because in (93) the term
$W_{M}(\bar{n}_{\Sigma})$ with the occupancy numbers (62) dominates the sum,
see problem 3.4 of [21]. What changes is the accuracy of the approximation,
that, although being in both cases in the order of ${\cal O}(\log(N))$, can be
different in the two cases. Repeating the same reasoning that we already made
for the logarithm of the multinomial coefficient, from (96) we see that what
Planck calls entropy is a (random) conditional entropy with deterministic
condition that, when the deterministic condition is (97), can be manipulated
as follows:

$\displaystyle H(\bar{\cal L}|{l}_{\Sigma})=\log(W_{NB}(l_{\Sigma},N))$
$\displaystyle=l_{\Sigma}\log\left(1+\frac{N}{l_{\Sigma}}\right)+N\log\left(1+\frac{l_{\Sigma}}{N}\right)+{\cal
O}(\log(N))$ $\displaystyle=N(\mu_{\cal L}\log(1+\mu_{\cal
L}^{-1})+\log(1+\mu_{\cal L}))+{\cal O}(\log(N))$ $\displaystyle=H_{\bar{\cal
L}}+{\cal O}(\log(N)).$

This result is analogous to the one that we found before for Boltzmann’s
number of complexions and Bose’s entropy.

One more comment is in order about the conditional equiprobability of
microstates expressed by (96). We start from the expectation of ${\cal
L}_{\Sigma}$, which is

$\displaystyle\mu_{{\cal L}_{\Sigma}}$ $\displaystyle=N\mu_{\cal
L}=\frac{N}{e^{\beta\omega\hbar}-1}.$

The rightmost term of the above equation is the expected number of energy
quanta with quantum energy $\omega\hbar$, which was calculated for the first
time by Bose in [37]. For completeness, we hereafter explicitly map our
entities to the corresponding entities of Bose’s paper. The consideration of
only one frequency means that we are considering only one of Bose’s species,
that Bose indexes with the index $s$. Since we consider only one frequency,
the index $s$ of [37] does not appear at all in our paper. We observe that it
is the assumption of identical distribution that restricts our consideration
to monochromatic oscillators. If one wants to analyze a non-monochromatic
system, say, a system that includes two different frequencies, then the two
different frequencies, through (86), will generate two geometric distributions
(85) that differ between them because they are based on two different values
of the parameter $p$ appearing in (86) and (85), hence the oscillators
oscillating on one frequency will have a distribution of the random energy
different from that of the oscillators oscillating on the other frequency. The
rest of the mapping between Bose’s entities and the entities of this paper is
in the following footnote.888Bose on the left, this paper on the right of the
arrow:  • $A$ cells in phase space $\rightarrow$ $N$ indexed oscillators, •
number of cells at the $l$-th energy level, $p_{l}\rightarrow Np_{\cal L}(l)$,
• $W\rightarrow W_{M}(\bar{n}_{\Sigma})$, • $k_{B}$ times the temperature,
$\beta\rightarrow\beta^{-1},$ • energy of one quantum,
$h\nu\rightarrow\hbar\omega,$ • expected total number of energy quanta,
$N\rightarrow N\mu_{\cal L}=\mu_{{\cal L}_{\Sigma}},$ • expected total energy,
$E\rightarrow N\mu_{\cal E}$, • $B\rightarrow Np,$ this is only a service
variable. The entropy, calculated by Bose as the logarithm of the multinomial
coefficient based on the deterministic occupancy numbers, denoted $W$ by Bose,
${\cal P}$ in [36] and $W_{M}(\bar{n}_{\Sigma})$ here, appears in the second-
top equation of the last column of [37]. In passing, we point out that the
right-hand side of the second-last equation of the first column of pag. 2 must
be multiplied by the volume $V$ and that the sign of the argument of the
exponential in Bose’s second-last equation that expresses the entropy is
flipped. Finally, note that Bose puts to zero the energy of the ground state,
while here we put it to $\omega\hbar/2$. After the identification of all the
entities, it should be clear that the negative binomial probability
distribution $p_{{\cal L}_{\Sigma}}$ of the total quantum number is introduced
here for the first time. At least to our best knowledge, it has no equivalent
in the open bibliography, where only the mean value and the variance of our
${\cal L}_{\Sigma}$ are considered. This distribution plays the same role as
the multinomial distribution (40), that is the probability distribution of the
random condition or, if preferred, of the random macrostate, that here is the
random total quantum number. In analogy to the conditional equiprobability of
microstates given the random occupancy numbers, also in this case, thanks to
this distribution, we can demonstrate by equation (96) that all the
$W_{NB}({\cal L}_{\Sigma},N)$ arrangements of energy quanta are conditionally
equiprobable given the total number ${\cal L}_{\Sigma}$ of energy quanta. The
words experience compelled physics to… and …came as a shock to intuition are
used by Feller in chapter 2 of his book [34] to comment on the counter-
intuitive equiprobability of all the arrangements of a given number of energy
quanta and on the need of forcing equiprobability on the basis of the
experimental evidence. Actually, no experience compelled us to this result,
neither it causes any shock to our intuition: it is a mere consequence of
conditional equiprobability of microstates given the macrostate.

##  VII Non-indexed particles, $\bar{\cal X}=\bar{\cal N}_{\Sigma}$

We say that particles are non-indexed when it is impossible to recognize the
permutation of any two particles. To prevent misunderstanding, we use the term
non-indexed, not the term indistinguishable. In standard textbooks, the term
indistinguishable is often used to indicate quantum particles of the same
species. These indistinguishable particles have the same energy, for instance,
the energy quantum $\omega\hbar$ in a monochromatic radiation. Instead, our
non-indexed particles can have different energies between them, as it happens
when many gas particles are in the same container. In this case, two non-
indexed particles with different energies could be distinguished from their
energy.

Besides the different meaning that we attribute to non-indexed and
indistinguishable, it remains that in both cases the permutation of any two
particles does not change system’s state. In the quantum information-theoretic
representation, lack of indexing can be imported by saying that the most
detailed knowledge of the system that Bob can extract is limited to the
occupancy numbers, therefore for the classical random vector of the
preparation we take

$\bar{\cal X}=\bar{\cal N}_{\Sigma},$

hence the entropy of the system of non-indexed particles is

$H_{\bar{\cal X}}=H_{\bar{\cal N}_{\Sigma}}.$

It is pretty much unexpected that, as anticipated in a previous section, the
entropy of the vector of the occupancy numbers, which in the standard approach
to the entropy of indexed particles is overlooked, when non-indexed particles
are considered is equal to system’s entropy!999In [11] we missed the role that
the random entropy $H(\bar{\cal N}_{\Sigma})$ has in systems made by indexed
particles and, with it, we missed the role of the random conditional entropy
$H(\bar{\cal L}|\bar{\cal N}_{\Sigma})$. We recognized that $H(\bar{\cal
N}_{\Sigma})$ has a role also in the case of indexed particles only when,
after a comment of a reviewer of [11], we considered systems made by non-
indexed particles.

Imposing the conditional equiprobability of microstates given the occupancy
numbers, for the density operators of the prepared mixed states (18) we have

$\hat{\rho}({\bar{x}})=\hat{\rho}({\bar{n}_{\Sigma}})=\frac{1}{W_{M}(\bar{n}_{\Sigma})}\sum_{i=1}^{W_{M}(\bar{n}_{\Sigma})}\ket{{\bar{l}_{i}(\bar{n}_{\Sigma})}}\bra{{\bar{l}_{i}(\bar{n}_{\Sigma})}}.$
(98)

As desired, the prepared state doesn’t change when
$\bar{l}_{i}(\bar{n}_{\Sigma})$ and $\bar{l}_{k}(\bar{n}_{\Sigma})$ are
exchanged. We remark again that, while in the standard approach
equiprobability is assumed, here conditional equiprobability given the vector
of the occupancy numbers has been demonstrated.

The entropy does not change if we say that the prepared state is the pure
state

$\ket{\bar{x}}=\sqrt{\frac{1}{W_{M}(\bar{n}_{\Sigma})}}\sum_{i=1}^{W_{M}(\bar{n}_{\Sigma})}\ket{{\bar{l}_{i}(\bar{n}_{\Sigma})}}.$
(99)

We take the above pure state because the probability that a specific outcome
$\ket{{\bar{l}_{i}(\bar{n}_{\Sigma})}}$ is obtained from an energy measurement
made on $\ket{\bar{x}}$ and on the $\hat{\rho}_{\bar{x}}$ of (98) is the same.
Since the prepared state is pure, now

$S(\hat{\rho}({\bar{x}}))=0,$

therefore we have again

$\chi=S\left(\sum_{\bar{x}\in\\{\bar{\cal X}\\}}p_{\bar{\cal
X}}(\bar{x})\hat{\rho}({\bar{x}})\right)=H_{\bar{\cal X}}.$

Basically, (99) treats lack of indexing in the same way as the standard
quantum statistical mechanics approach treats bosons, see e.g. eqn. 7.7 of
[48], eqn. 3.18 of [49].

### Fixed number of non-indexed particles

Consider a fixed number $N$ of particles, as it is the case, for instance,
when a gas is bounded by impenetrable walls. In this case, the classical
random vector of the preparation is the multinomial random variable. System’s
random entropy is

$\displaystyle H(\bar{\cal N}_{\Sigma})=H(\bar{\cal
L}^{N})-\log(N!)+\sum_{l\in\\{{\cal L}\\}}\log({\cal N}_{\Sigma}(l)!),$ (100)

where, since $\bar{\cal X}=\bar{\cal N}_{\Sigma}$, we take $H(\bar{\cal
X})=H(\bar{\cal N}_{\Sigma})$. System’s entropy is

$\displaystyle H_{\bar{\cal N}_{\Sigma}}$ $\displaystyle=H_{\bar{\cal
L}^{N}}-\log(N!)+\sum_{l\in\\{{\cal L}\\}}\braket{\log({\cal
N}_{\Sigma}(l)!)}$ (101) $\displaystyle=NH_{\cal
L}-\log(N!)+\sum_{l\in\\{{\cal L}\\}}\braket{\log({\cal B}(l)!)},$

where ${\cal B}(l)$ is a binomial random variable with parameter $p_{\cal
L}(l)$ and $N$ trials, see [50], and, for a binomial random variable ${\cal
B}$ with parameter $p$ and $N$ trials,

$\displaystyle\braket{\log({\cal
B})}=\sum_{n=0}^{N}\left(\begin{array}[]{c}N\\\ n\\\
\end{array}\right)p^{n}(1-p)^{N-n}\log(n!),$ (104)

which has been recently calculated in integral form in [45].

Looking at (101), we see that the term $-\log(N!)$ is the one introduced by
Gibbs to make the phase-space entropy of non-indexed particles compatible with
his famous paradox. The genesis of the term $-\log(N!)$ has been in the past
object of debate. Many authors report that this term was introduced by Gibbs
as an ad hoc correction of the entropy of indexed particles to ensure
extensivity of entropy, but Jaynes argues in [51] that an early work of Gibbs
contains an analysis that correctly explains the presence of the term
$-\log(N!)$ in the entropy formula. Swedensen in [35] defines the entropy in
the phase space by embedding in the definition the term $-\log(N!)$. The
crucial point of Swedensen’s definition is that the probability distribution
of finding a random number of particles in sub-regions of space is multinomial
(even if Swedensen’s example is based on the binomial distribution, the
extension to the multinomial distribution is straightforward). The term
$-\log(N!)$ is then imported inside the entropy just from the multinomial
distribution, exactly as it happens in our case. Independently of Gibbs and of
his famous paradox, Tetrode followed a reasoning that brought him very close
to (101). He argued that gas states obtained by permutation of particles
should not be counted as different, see [28]. At high temperature all the
particles occupy different energy levels, hence the number of permutations
expressed by the multinomial coefficient becomes equal to $N!$, whose
logarithm is just the correction term considered by Tetrode. Basically,
equation (98) can be seen as a consequence of Tetrode’s approach, it is enough
to consider the random multinomial coefficient as it is and then to take the
expectation of its logarithm. The denominator of the multinomial coefficient,
overlooked by Tetrode, generates the third term that appears in (100) and
(101). This third term becomes important at low temperature, where many
particles occupy the same energy level, reducing significantly below $N!$ the
number of distinct permutations. If this term is overlooked, at temperature
low enough the correction represented by $-\log(N!)$ becomes too large and the
entropy falls below zero, as it actually happens in the Sackur-Tetrode entropy
formula. Einstein showed in [52] that the term $-\log(N!)$ is inherently
present in the entropy of quantum gases, without the need of any ad hoc
intervention. After Einstein, it became standard to treat the classical ideal
gas at high temperature as a quantum gas, thus incorporating the term
$-\log(N!)$ in the entropy. Actually, at high temperature the effect of
quantization is negligible therefore there is virtually no distinction between
classical and quantum gases. However, both the Sackur-Tetrode approach and the
full-quantum approach describe the ideal gas only at high temperature, while
(101) gives the entropy of the ideal gas at low and high temperature.

### Random number of non-indexed particles

Using for the occupancy numbers the independent Poisson random variables (59),
for the random entropy and for the entropy we have

$\displaystyle H({\bar{\cal N}_{\Sigma}})=\sum_{l\in\\{{\cal
L}\\}}\lambda(l)-{\cal N}_{\Sigma}(l)\log(\lambda(l))+\log({\cal
N}_{\Sigma}(l)!),$ $\displaystyle H_{\bar{\cal N}_{\Sigma}}$
$\displaystyle=\sum_{l\in\\{{\cal
L}\\}}\lambda(l)-\lambda(l)\log(\lambda(l))+\braket{\log({\cal
N}_{\Sigma}(l)!)},$ $\displaystyle=\lambda H_{{\cal
L}}-\lambda(\log(\lambda)-1)+\sum_{l\in\\{{\cal L}\\}}\braket{\log({\cal
N}_{\Sigma}(l)!)},$

where in the second equality we use (58) and

$\displaystyle\braket{\log({\cal N}_{\Sigma}(l)!)}$
$\displaystyle=e^{-\lambda(l)}\sum_{n=0}^{\infty}\frac{(\lambda(l))^{n}\log(n!)}{n!}.$

An integral form for the above expectation has been recently worked out in
[45]. Putting $\lambda=N$ we immediately appreciate the strong similarity
between the above entropy and that of the multinomial random variable (101).

##  VIII Ideal gas in a container

The pseudo-quantum model for a particle of an ideal gas in a container is that
of the particle in a box, whose energy levels are given in (16). As we will
see, through the quantization rule (16) the volume of the container will be
imported in the probability distribution of energy levels and, as a
consequence, in the entropy formula. In the following, we take as a
deterministic constraint the Boltzmann distribution for the energy of gas’
particles, even if this could be not the entropy-maximizing distribution,
because entropy maximization should take into account also the third term of
(101), which is important at low temperature. However, this maximization is
out of the scope of the present paper.

### High temperature

At high temperature, where quantization effects become negligible, sums are
approximated to integrals, discrete random variables are approximated to dense
random variables, and the gas behaves approximately as a classical gas made by
hard balls. Specifically, the sums (67) and (71) can be approximated to the
discretization with unit step of the following two integrals

$Z\approx\int_{l=0}^{\infty}e^{-\beta
l^{2}\frac{\pi^{2}\hbar^{2}}{2mL^{2}}}dl=\sqrt{\frac{mL^{2}}{2\beta\pi\hbar^{2}}},$
(105)

and

$A=\frac{\pi^{2}\hbar^{2}}{2mL^{2}}\int_{l=0}^{\infty}l^{2}e^{-\beta
l^{2}\frac{\pi^{2}\hbar^{2}}{2mL^{2}}}dl=\sqrt{\frac{mL^{2}}{8\beta^{3}\pi\hbar^{2}}},$

respectively, leading to

$\mu_{{\cal E}}\approx\frac{A}{Z}\approx\frac{1}{2\beta},$ $\displaystyle
H_{{\cal L}}$ $\displaystyle=\log(Z)+\beta\mu_{{\cal E}}$
$\displaystyle\approx\frac{1}{2}\left(\log\left(\frac{mL^{2}}{2\beta\pi\hbar^{2}}\right)+1\right),$
(106)

which, used in (101), provides us with

$\displaystyle H_{\bar{\cal N}_{\Sigma}}$ $\displaystyle\approx NH_{{\cal
L}}-\log(N!)$
$\displaystyle\approx\frac{N}{2}\left(\log\left(\frac{mL^{2}}{2\beta\pi\hbar^{2}}\right)+1\right)-\log(N!)$
(107)
$\displaystyle\approx\frac{N}{2}\left(\log\left(\frac{mL^{2}}{2\beta\pi\hbar^{2}N^{2}}\right)+3\right),$
(108)

where the first approximation neglects the third term of (101), the second
approximation is (106) and the third approximation is Stirling’s formula. As
expected, (108) is the Sackur-Tetrode entropy formula for a classical ideal
gas of $N$ particles in an one-dimensional container of size $L$ at
temperature

$T=\frac{1}{k_{B}\beta}.$

At high temperature, the discrete index of energy levels can be approximated
to a dense (continuous) random variable that we call again ${\cal L}$. Its
probability density function is obtained by looking at the quantization rule
appearing in the exponent of the Boltzmann distribution as at a continuous
function of continuous variable. The normalization of the dense distribution
is just the division by the integral of equation (105), leading to

$f_{\cal
L}(l)=\sqrt{\frac{2\beta\pi\hbar^{2}}{mL^{2}}}e^{-\frac{\beta\pi^{2}\hbar^{2}}{2mL^{2}}l^{2}},\
\ l>0,$

where $f_{\cal X}(\cdot)$ indicates the probability density function of the
continuous random variable ${\cal X}$. The probability density function of the
continuous random energy, that again we call ${\cal E}$, is found by making
the following change of random variable from the continuous energy level
${\cal L}$ to the continuous energy ${\cal E}$:

${\cal E}=\frac{\pi^{2}\hbar^{2}}{2mL^{2}}{\cal L}^{2},$

hence

${\cal L}=\sqrt{\frac{2mL^{2}{\cal E}}{\pi^{2}\hbar^{2}}}.$

Calling $\epsilon$ the continuous variable that spans the support of ${\cal
E}$, we have

$\displaystyle f_{\cal E}(\epsilon)$ $\displaystyle=f_{\cal
L}\left(\sqrt{\frac{2mL^{2}\epsilon}{\pi^{2}\hbar^{2}}}\right)\cdot\frac{d}{d\epsilon}\sqrt{\frac{2mL^{2}\epsilon}{\pi^{2}\hbar^{2}}}$
$\displaystyle=\sqrt{\frac{2\beta\pi\hbar^{2}}{mL^{2}}}e^{-\beta\epsilon}\cdot\sqrt{\frac{mL^{2}}{2\pi^{2}\hbar^{2}\epsilon}}$
$\displaystyle=\sqrt{\frac{\beta}{\pi\epsilon}}e^{-\beta\epsilon}$
$\displaystyle=\frac{1}{\Gamma(0.5)}\sqrt{\frac{\beta}{\epsilon}}e^{-\beta\epsilon},\
\ \epsilon>0,$

that is a Gamma distribution $\Gamma(k,\theta)$ with shape parameter $k=0.5$
and scale parameter $\theta=\beta^{-1}$, or, equivalently, a $\chi^{2}$
distribution with one degree of freedom and mean value $(2\beta)^{-1}$. As in
the case of the Sackur-Tetrode formula, also this is the expected result. The
probability density function of momentum of the classical one-dimensional gas
particle at the thermal equilibrium is in fact known to be normal with zero
mean value and variance $m\beta^{-1}$,101010This famous result can be obtained
in our setting as follows. If, in place of aperiodic boundary conditions,
periodic boundary conditions are imposed, one gets the probability
distribution  $p_{\cal L}(l)=Z^{-1}e^{-\beta
l^{2}\frac{\pi^{2}\hbar^{2}}{2mL^{2}}},\ \ l=\cdots,-1,0,1,\cdots.$ At high
temperature  $Z\approx\int_{l=-\infty}^{\infty}e^{-\beta
l^{2}\frac{\pi^{2}\hbar^{2}}{2mL^{2}}}dl=\sqrt{\frac{2mL^{2}}{\beta\pi\hbar^{2}}},$
therefore the probability density function of the dense level becomes
$f_{\cal L}(l)=\sqrt{\frac{\beta\pi\hbar^{2}}{2mL^{2}}}e^{-\beta
l^{2}\frac{\pi^{2}\hbar^{2}}{2mL^{2}}}.$ Calling ${\cal P}$ the classical
dense random momentum,  ${\cal P}=\pm\sqrt{\frac{{\cal E}}{2m}},$ and changing
random variable  ${\cal P}=\frac{\pi\hbar{\cal L}}{L},\ \ {\cal L}=\frac{{\cal
P}L}{\pi\hbar},$ we get  $f_{\cal P}(p)=f_{\cal
L}\left(\frac{pL}{\pi\hbar}\right)\cdot\frac{d}{dp}\frac{pL}{\pi\hbar}=\sqrt{\frac{\beta}{2\pi
m}}e^{-\frac{\beta p^{2}}{2m}},$ that is the normal distribution with zero
mean value and variance $m\beta^{-1}$ mentioned before. From the normal
distribution of the dense classical momentum and from the uniform distribution
of the dense classical position ${\cal Q}$, it follows that the differential
entropy of one classical particle in phase space is  $-\braket{\log(f_{\cal
P}({\cal P}))}-\braket{\log(f_{\cal Q}({\cal Q}))}=\frac{1}{2}\log\left(2\pi
em\beta^{-1}\right)+\log(L).$ The differential entropy can be converted into
the entropy (106) by quantizing phase space into cells of size $2\pi\hbar$,
that is, adding the the term $-\log(2\pi\hbar)$ to the differential entropy,
see 8.3 of [19]. therefore, since the energy is wholly kinetic, the
probability density function of the continuous energy must be $\chi^{2}$ with
one degree of freedom and mean value $(2\beta)^{-1}$, as it actually is.

### Low temperature

At low temperature, the sums (67) and (69), or, equivalently (71), as well as
the expectation (104), cannot be calculated in closed form, but numerical
evaluation is possible. Numerical results have been obtained for the common
three-dimensional case, where $N$ particles are inside a three-dimensional
cubic box of volume $V=L^{3}$. The partition function is calculated in one
dimension, that is, for the one-dimensional box of size $L=\sqrt[3]{V}$,
obtaining the Boltzmann distribution and the entropy of one particle in one
dimension. Calling this entropy $H_{\cal L}$, the entropy of the $N$ three-
dimensional indexed particles is $3NH_{\cal L}$ and the entropy of the system
of $N$ non-indexed three-dimensional particles is

$\displaystyle H_{\bar{\cal N}_{\Sigma}}$ $\displaystyle=3NH_{\cal
L}-\log(N!)+\sum_{l\in\\{{\cal L}\\}}\braket{\log({\cal N}_{\Sigma}(l)!)}$
(109)
$\displaystyle\approx\frac{3N}{2}\left(\log\left(\frac{mL^{2}}{2\beta\pi\hbar^{2}}\right)+1\right)-\log(N!)$
(110)
$\displaystyle=N\left(\log\left(V\left(\frac{m}{2\beta\pi\hbar^{2}}\right)^{\frac{3}{2}}\right)+\frac{3}{2}\right)-\log(N!)$
(111) $\displaystyle\approx
N\left(\log\left(\frac{V}{N}\left(\frac{m}{2\beta\pi\hbar^{2}}\right)^{\frac{3}{2}}\right)+\frac{5}{2}\right),$
(112)

where (110) is the high temperature approximation (107), in (111) we
substitute $V$ in place of $L^{3}$ and (112), which is the classical Sackur-
Tetrode entropy formula in three dimensions, is obtained by Stirling’s
approximation. The results of Figs. 1, 2, 3, 4, report the entropy (or terms
that contribute to the entropy) in $k_{B}$ units versus temperature in Kelvin
degrees. All the results are obtained for volume of the cubic box $V=10^{-27}$
cubic meters and mass of one particle $m=1.67\cdot 10^{-27}$ kilograms. With
these parameters, the energy of the one-dimensional ground state is

$\epsilon(1)=\frac{\pi^{2}\hbar^{2}}{2mL^{2}}=3.25\cdot 10^{-23}.$

The size of the set of energy level has been truncated to $|{\cal L}|=50$,
having verified that the probability of ${\cal L}>50$ is so small that it does
not impact the graphs presented in the following. We hasten to point out that
the graphs reported in the following are intended to be only a numerical
exercise based on the formulas presented in this paper, therefore they will
describe the behavior of a real physical system only inside the range of
validity of the assumptions that we made.

Fig. 1 shows the approximation (106) and the numerical evaluation of the
entropy $3H_{\cal L}$ of one particle in three dimensions. The
energy/temperature term $\beta\mu_{\cal E}$ is reported in Fig. 2. At $T=1$
Kelvin this term is approximately equal to 2.37, which corresponds to the
following expected energy per dimension

$\mu_{\cal E}|_{T=1}=1.005\cdot\epsilon(1).$

Since the mean value of the energy per degree of freedom is lower bounded by
$\epsilon(1)$ and since the lower bound is virtually reached at 1 Kelvin, at
temperature below 1 Kelvin the energy/temperature term $\beta\mu_{\cal E}$ is
forced to grow hyperbolically. In passing, we observe that the
energy/temperature term $\beta\mu_{\cal E}$ strongly resembles a hyperbole in
the entire range of temperature. The heuristic hyperbolic fit

$\beta\mu_{\cal E}\approx\frac{1}{2}+\frac{\epsilon(1)}{k_{B}T}$ (113)

is the dash-dotted line in Fig. 2. At high-temperature, the energy/temperature
term reaches the horizontal asymptote of the hyperbole, that, as predicted by
the high temperature approximation, is equal to $0.5$. However, in the
transition region, say $0.3<T<3$, the approximation error is appreciable. For
instance, for $T=1$ the numerical calculation gives $\beta\mu_{\cal E}=2.37$,
while the heuristic hyperbolic fit (113) gives $\beta\mu_{\cal E}=2.86$.

Figs. 3 and 4 report the results obtained numerically for $N=30$ three-
dimensional particles. Fig. 3 shows that the entropy of indexed particles and,
consequently, the entropy of non indexed particles, which must be between zero
and the entropy of indexed particles, drops virtually to zero at the ”cutoff”
temperature of 1 Kelvin, where the entropy of the 30 three dimensional indexed
particles is 0.62 (0.62/90 per degree of freedom), while the entropy of the
non-indexed particles is 0.53. A so low value of entropy is compatible with
the fact, already pointed out, that the expected energy per particle is close
to that of the ground state, meaning that particles are with high probability
in the ground state. Fig. 4 reports the relative quantum correction, that is

$\frac{\sum_{l\in\\{{\cal L}\\}}\braket{\log({\cal
N}_{\Sigma}(l)!)}}{\log(N!)},$

versus temperature. The graph shows that, for temperature below 1 Kelvin, the
numerator of the above relative quantum correction is virtually equal to the
denominator, hence that the third term of (109) completely compensates the
term $-\log(N!)$, preventing negative values of the entropy. As expected, the
relative quantum correction tends to zero at high temperature, but at $100$
Kelvin it is still appreciably above zero, in fact the entropy of non-indexed
particles is still appreciably above the Sackur-Tetrode entropy, see Fig. 3.
Even at $2000$ Kelvin, more three decades above the cutoff temperature, with
the parameters used in this example the quantum correction is still about
$0.1$ times its maximum value.

Figure 1:  Solid line: numerical evaluation of $3H_{{\cal L}}$. Dashed line:
approximation to $3H_{\cal L}$ given by (106).  Figure 2:  Solid line:
energy/temperature contribution $\beta\mu_{{\cal E}}$ to the entropy per
particle per dimension. Dash-dotted line: hyperbolic fit (113).  Figure 3:
Dash-dotted line: entropy of indexed particles, $3NH_{\cal L}$ in (109). Solid
line: entropy of non-indexed particles, $H_{\bar{\cal N}_{\Sigma}}$ in (109).
Dashed line: Sackur-Tetrode entropy (112). Figure 4:  Relative quantum
correction, third term of (109) divided by $\log(N!)$.

##  IX Conclusions

A step towards a full information-theoretic paradigm for the statistical
mechanics of systems made by independent particles with identical probability
distribution has been proposed in the paper. This approach, we would say, this
vision, is based on the identification of information theoretic concepts as
typicality, conditional entropy, mutual information, Holevo accessible
information, with the corresponding physically equivalent quantities. Our
paradigm rules out the postulate of equally probable microstate, which has
been object of debate since when it was originally proposed by Einstein,
putting in place of it the information-theoretic typical set and the
conditional equiprobability of microstates given the corresponding macrostate.
The renounce to the postulate not only is compatible with all the results that
we know about many-particle systems, but also leaves space to an harmonic and
unified treatment of low and high temperature, small and large number of
particles, systems of indexed and non-indexed particles.

Much work is to be done along the direction taken by this paper. We hereafter
mention few points that are not covered in this paper and that could be
considered in the future. One point that needs investigation is the role of
the assumption of identical distribution. For systems at the thermal
equilibrium, it seems that this assumption is reasonable, but, even if in
principle this assumption could be made also for a system that is not at the
thermal equilibrium, it could become questionable and could lead to misleading
conclusions when the system evolves towards equilibrium through to non-
reversible transformations. For instance, when two gases at different
temperatures are mixed, can de mix be described by one probability
distribution obtained by the weighted sum of the two and, if so, till what
extent? Another open point is the degeneration of states with the same energy.
One could consider, in place of only the energy level, a joint random variable
that includes, for instance, energy and spin, but this consideration is not
pursued in this paper. Finally, we mention that another open point is how and
if the Bose-Einstein condensed state can be inserted inside the framework
sketched in this paper.

##  X Acknowledgements

The author is indebted to Matteo Paris for pointing out an error in Section
III of a previous version of the paper.

## References

  * [1] Tribus, Myron, and Edward C. McIrvine. ”Energy and information.” Scientific American 225.3 (1971): 179-190.
  * [2] Einstein, Albert. ”Zum gegenwartigen Stand des Strahlungsproblesm.” Phys. Zeits 10 (1909): 185-193. For english translation, see Stachel, John, et al. ”The collected papers of Albert Einstein. Volume 2. The Swiss years: Writings, 1900-1909,” Princeton University Press, 1989, Doc. 56, pp. 357-375.
  * [3] A. I. Khinchin (1941): Matematiceskie osnovanija statisticeskoj mechaniki. Moscow. English translation by G. Gamow: Mathematical foundations of statistical mechanics. New York: Dover (1949)
  * [4] Campisi, Michele, and Donald H. Kobe. ”Derivation of the Boltzmann principle.” American Journal of Physics 78.6 (2010): 608-615.
  * [5] Wright, P. G. ”Entropy and disorder.” Contemporary Physics 11.6 (1970): 581-588.
  * [6] Popescu, Sandu, Anthony J. Short, and Andreas Winter. ”Entanglement and the foundations of statistical mechanics.” Nature Physics 2.11 (2006): 754-758.
  * [7] Goldstein, Sheldon, et al. ”Canonical typicality.” Physical review letters 96.5 (2006): 050403.
  * [8] Bartsch, Christian, and Jochen Gemmer. ”Dynamical typicality of quantum expectation values.” Physical review letters 102.11 (2009): 110403.
  * [9] Von Spakovsky, Michael R., and Jochen Gemmer. ”Some trends in quantum thermodynamics.” Entropy 16.6 (2014): 3434-3470.
  * [10] Gemmer, Jochen, Mathias Michel, and G nter Mahler. Quantum thermodynamics: Emergence of thermodynamic behavior within composite quantum systems. Vol. 784. Springer, 2009.
  * [11] Spalvieri, Arnaldo. ”The Shannon-McMillan theorem proves convergence to equiprobability of Boltzmann’s microstates.” Entropy 2021, 23, 899.
  * [12] Mezard, Marc, and Andrea Montanari. Information, physics, and computation. Oxford University Press, 2009.
  * [13] Landauer, Rolf. ”Irreversibility and heat generation in the computing process.” IBM journal of research and development 5.3 (1961): 183-191.
  * [14] B rut, Antoine, et al. ”Experimental verification of Landauer s principle linking information and thermodynamics.” Nature 483.7388 (2012): 187-189.
  * [15] Plenio, Martin B., and Vincenzo Vitelli. ”The physics of forgetting: Landauer’s erasure principle and information theory.” Contemporary Physics 42.1 (2001): 25-60.
  * [16] Martinelli, M. ”Photons, Bits and Entropy: From Planck to Shannon at the Roots of the Information Age.” Entropy, 2017, 19, 341.
  * [17] Jaynes, E. T. ”Information theory and statistical mechanics.” Physical review, 1957, 4, 620–630.
  * [18] Merhav, N. ”Physics of the Shannon Limits,” IEEE Trans. on Inform. Theory, 2010, 9, 4274–4285.