# A genuinely natural information measure

Andreas Winter  GIQ UAB  ICREA

(1 April 2021)

###### Abstract

The theoretical measuring of information was famously initiated by Shannon in
his mathematical theory of communication, in which he proposed a now widely
used quantity, the entropy, measured in _bits_. Yet, in the same paper,
Shannon also chose to measure the information in continuous systems in _nats_
, which differ from bits by the use of the natural rather than the binary
logarithm.

We point out that there is nothing natural about the choice of logarithm
basis, rather it is arbitrary. We remedy this problematic state of affairs by
proposing a genuinely natural measure of information, which we dub _gnats_. We
show that gnats have many advantages in information theory, and propose to
adopt the underlying methodology throughout science, arts and everyday life.

Nuts and bolts of information theory. The ongoing digital revolution is fond
of considering Claude Shannon as its godfather, his information theory
Shannon:IT  providing the theoretical underpinning for the pervasive
digitization of every kind of data, which today makes the representation and
indeed measurement of information in _bits_ seem almost inevitable, if not
entirely natural. Indeed, in the same founding paper, Shannon started the
habit of measuring the information, and in particular the capacity, of
continuous-variable systems in _nats_. This parallel use of two conflicting
conventions has persisted in information theory, and has been embedded into
the grain of its various sub-disciplines.

Today, after more than seventy years of information theory, which saw it
become one of the most successful tools of all engineering branches occupied
with the development of information processing and communication devices, we
can state that this was an unfortunate move, as it imported an annoying
ambiguity into the very foundations of this otherwise awe-inspiring
theoretical framework. The detractors who wish to deny the severity of this
well-known problem will inevitably point to the mathematical fact that bits
and nats are distinguished by the choice of binary and natural logarithm,
respectively, when defining the entropy. Indeed, for the simplest case of a
uniform distribution on $N$ objects,

$\displaystyle S$ $\displaystyle=\log N\text{ [bits]},$ (1) $\displaystyle S$
$\displaystyle=\ln N\text{ [nats]},$ (2)

are the two commonly employed formulas. They differ by a factor of $\ln 2$,
which is well-known to be impossible to apply correctly (does it divide or
multiply?).

In addition, it is often claimed that the difference between Eqs. (1) and (2)
is merely down to a choice of units (of information, presumably). But that is
evidently nonsense, as entropy is, by definition, a unitless quantity. In
other words, writing the entropy as a number, one cannot intrinsically deduce
this elusive “unit” from it, nor is there any way of attaching a unit in the
usual sense that would make it unambiguous. The usual way out is to attach the
words “bits” or “nats” to indicate the calculation by which the result was
obtained.

We are thus faced with the truly fundamental problem of information science,
which has been left unattended from the beginning, but cannot be ignored any
longer: to devise a universal measure of information, avoiding the ultimately
arbitrary choice of the basis of a logarithm. Obviously, despite all its
success in information technology, the binary logarithm (basis $2$) cannot
claim any more fundamental significance than the common one (basis $10$); as
for the logarithm to basis $e$, it may be called “natural”, but it is not any
more natural then the previous two when it comes to calculating entropies.

In the present work, we finally address this issue at the very foundations of
the field. Our approach shall be a natural one, looking for guidance in the
physical world. While the conclusion is that henceforth we ought to abandon
formulas like Eqs. (1) and (2), it is hoped that this slight discomfort
(surely overcome quickly with habit) will provide a sounder basis for
information science.

From bits to gnats. Our quest is to find a _genuinely natural information
measure_ , which directly suggests to consider _gnats_ , to obtain a
reasonable acronym.

While gnats have been studied in biology for centuries (see Fig. 1), they have
never been considered in connection with information technology. Indeed, the
only role insects have played hitherto in computer science was restricted to
_bugs_ , which is rather unspecific on the one hand, and on the other hand
seems to originally refer to a completely different order (_lepidoptera_
rather than _diptera_).

Figure 1: A gnat (after Robert Hooke’s _Micrographia_).

However, once the idea is presented, the advantages of gnats in information
technology are blindingly obvious:

  * •

gnats are everywhere, and occur in unlimited supply, thus guaranteeing global
connectivity and sustainability;

  * •

because of their omnipresence, both data storage and calibration are easily
solved problems;

  * •

gnats are discrete by design, yet entirely natural;

  * •

because of their eponymous attention span, information systems based on gnats
will to good approximation be memoryless, thus rendering obsolete much of one-
shot information theory, and restoring the wonderful world of iid.

Methodology. Unfortunately we have to have this section, but that does not
mean we have to waste any time on it.

In the few words necessary, our method of discovery is morphological, lexical,
and strictly asemantic. In these regards it follows the paradigm (a dozen
indeed) of Shannon’s approach to information Shannon:IT . It is however
influenced by certain aspects of gnomonology and numerology, but the reader
does not have to be an expert in the latter to understand the derivation or
the result. Let it be enough to point out that the number seven, and hence the
seventh letter of the Latin (English) alphabet, is of fundamental importance.

Calibration and conversion. As gnats occur naturally, their comparison with
bits (and for that matter, nats) is a question of empirical observation, see
Fig. 1. Initial rough measurements indicate that $1$ gnat equals $956828.3$
bits ($663222.8$ nats); these values are obtained from the entropy of a simple
black hole formed by an average gnat. The identification is justified by the
fact that throwing it into a black hole is the only reliable way known of
definitely getting rid of a gnat. As is well-known, the black hole entropy
depends quadratically on the mass,

$1\text{ gnat}\equiv
S_{\text{BH}}(\vbox{\hbox{\includegraphics[width=10.00002pt]{muecke.jpg}}})=\frac{4\pi
GM_{\vbox{\hbox{\includegraphics[width=4.89998pt]{muecke.jpg}}}}^{2}}{\hbar
c},$ (3)

and so high-precision measurements of the gnat mass
$M_{\vbox{\hbox{\includegraphics[width=4.89998pt]{muecke.jpg}}}}$ will be
vital for the future development of natural information theory.

For the moment, a good rule of thumb is that $1$ gnat is roughly $10^{6}$
bits, meaning that we can in practice continue to use bits in information
theory, keeping in mind that $1$ bit is roughly $1$ $\mu$gnat (micro-gnat).
Evidently, in due course bits and nats will be _defined_ in terms of gnats, as
soon as a sufficiently reliable standard for the latter has been established.

One might ask why we do not use the Planck mass $m_{P}=\sqrt{\frac{\hbar
c}{G}}$, which is only two orders of magnitude smaller than
$M_{\vbox{\hbox{\includegraphics[width=4.89998pt]{muecke.jpg}}}}$, and is
supposedly a universal unit of mass. Well, nice try; but it won’t work.

Figure 2: Holy gnu.

Discussion. Our proposal to base information theory on gnats has been shown to
unite several advantages: gnats appear naturally in discrete units, they are
fungible, being for all practical purposes infinite in number, and
omnipresent, meaning that global calibration is easy. This gives a wholly new
foundation (see Fig. 2) to information theory, which we expect to have
profound repercussions.

Figure 3: A pretty gnu goat.

Our novel approach points to a more fundamental principle, which has the
potential to revolutionise all gnatural sciences, but also medicine Pompey ,
psychology Pompey-2 , and economics Gnash , all the way to literature
GregorSamsa  and psychological self-help Gnome . The possibilities are endless
Shannon:bandwagon  (see Fig. 3).

Not all conceptual problems are solved by our proposal, and it leaves in
particular a number of open questions, not least in relation to quantum
physics and quantum information theory. First, black hole evaporation and
holography are waiting to be reformulated in terms of the new information
measure. In fact, we conjecture that the interior of black holes has a
consistent description entirely in terms of gnats; consequently, when a black
hole evaporates, it should do so by releasing all its gnats. Secondly, in
quantum information theory there are other types of information beyond the
bit, most importantly qubits and ebits, cf. quantum-leap . It remains an open
question how to quantise gnats, since “qnats” is not very convincing. Among
the proposals currently being evaluated are knats and knots.

Acknowledgments. The author thanks an anonymous gnarled gnof for sharing their
gnaphalium and a priceless gnetum gnemon during the preparation of the present
article. Gnawing at gnocchi, knocking them back knowingly, and persistent
gnashing and gnarring were essentially supported by assorted gnathostomata,
which are gratefully acknowledged.

## References

  * [1] Claude E. Shannon, “A mathematical theory of communication”, _Bell System Technical Journal_ , vol. 27, no. 3, pp. 379-423 & 623-656, 1948.
  * [2] Gnaeus Pompeius Magnus, “Old oats and gnu goats”, reprinted in: _International Review of Poxy Puns_ , vol. 1, no. 2, pp. 3-4, 1956.
  * [3] Gnaeus Julius Agricola, “Ils sont fous ces Bretons”, _private communication_ (apocryphal), ca. 23 June 77; quoted after Publius Cornelius Tacitus, _De vita et moribus Iulii Agricolae_.
  * [4] John Gnash, _Gnuts: from Gnobel to Ignobel_ , Gnu Yolk, NJ, unpublished manuscript, ca. 1982.
  * [5] Franz Kafka, _Die Verwandlung_ , Kurt Wolff Verlag, Leipzig, 1915/16.
  * [6] Terbor D. Galoshes, _Gnome Ore Mr. Gneiss Guy_ , Running Gnosis Publishing Inc., Cerro Gordo, CA, to appear, spring 2021.
  * [7] Claude E. Shannon, “The Bandwagon”, _IRE Transactions on Information Theory_ , vol. 2, no. 1, p. 3, 1956.
  * [8] Stefano Mancini and Andreas Winter, _A Quantum Leap in Information Theory_ , World Scientific Publishing Ltd., Singapore, 2020.