# A note on Pontryagin duality and continuous logic

Nicolas Chavarria
University of Notre Dame  Anand Pillay
University of Notre Dame Supported by NSF grants DMS-1665035, DMS-1760212, and
DMS-2054271

###### Abstract

We exhibit Pontryagin duality as a special case of Stone duality in a
continuous logic setting. More specifically, given an abelian topological
group $A$, and $\mathcal{F}$ the family (group) of continuous homomorphisms
from $A$ to the circle group $\mathbb{T}$, then, viewing $(A,+)$ equipped with
the collection $\mathcal{F}$ as a continuous logic structure $M$, we show that
the local type space $S_{\mathcal{F}}(M)$ is precisely the Pontryagin dual of
the group $\mathcal{F}$ where the latter is considered as a discrete group.

We conclude, using Pontryagin duality (between compact and discrete abelian
groups), that $S_{\mathcal{F}}(M)$ is the Bohr compactification of the
topological group $A$.

##  1 Introduction

We make some observations relating type spaces in continuous logic, Pontryagin
duals, and Bohr compactifications. Writing this down was motivated by some
questions of Marcus Tressl about our paper [3] asking what, if anything, is
the connection with Pontryagin duality.

The point of our paper [3] was that the theory, in continuous logic, of
structures of the form —an abelian group equipped with homomorphisms to a
compact group— is stable. However, stability is not relevant to the issues we
discuss in the present paper, which will be elementary at the technical level,
but possibly of general interest.

Given an abelian topological group $A$, we consider the family $\mathcal{F}$
of all _continuous_ homomorphisms to $\mathbb{T}$, and consider
$(A,+,-,0,f)_{f\in{\mathcal{F}}}$ as a continuous logic structure $M$. We form
the space $S_{\mathcal{F}}(M)$ of ${\mathcal{F}}$-types over $M$, a compact
space. There are two points that we make in this paper:

  1. (i)

$S_{\mathcal{F}}(M)$, with its natural group structure, is the Pontryagin dual
of ${\mathcal{F}}$ where the latter is considered as a discrete abelian group,
and

  2. (ii)

$S_{\mathcal{F}}(M)$, with the natural homomorphism from $A$, is the Bohr
compactification $\iota:A\to bA$ of $A$.

Point (i) says that the move from the family ${\mathcal{F}}$ of “formulas” to
the space of ${\mathcal{F}}$-types is precisely taking the Pontryagin dual.
Recall that the Pontryagin dual of a discrete abelian group $B$ is the group
of all homomorphisms from $B$ to $\mathbb{T}$, with the Tychonoff topology
(induced from ${\mathbb{T}}^{B}$). And the Bohr compactification of a
topological group is the universal object among continuous homomorphisms to
compact groups. Then point (ii) follows from (i) by Pontryagin duality. Both
(i) and (ii) are explained (proved) in Section 3.

Thanks to the referee for the helpful suggestions and comments.

##  2 Preliminaries

We go through the necessary background. When we talk about topological spaces
or groups, we will always assume that the topology is Hausdorff. Throughout,
$\mathbb{T}$ will denote the circle group, a compact abelian topological group
which also has a compatible metric. We will treat it as a subset of
$\mathbb{C}$ and so write it multiplicatively.

###  2.1 Bohr compactification and Pontryagin duality

We repeat the definition of Bohr compactifications. Given an arbitrary
topological group $G$, we can consider continuous homomorphisms $f$ from $G$
to compact topological groups $C$. There is a universal such $f:G\to C$,
namely such that for every continuous homomorphism $g$ from $G$ to a compact
group $D$ there is unique continuous homomorphism $h:C\to D$ such that
$g=h\circ f$. This universal object is called the Bohr compactification of
$G$, and sometimes written $bG$, or $\iota:G\to bG$ to include the continuous
homomorphism $\iota$.

######  Definition 2.1.

Let $A$ be an abelian topological group. By $\widehat{A}$, the Pontryagin dual
of $A$, we mean the group of continuous homomorphisms from $A$ to $\mathbb{T}$
equipped with the compact-open topology.

Recall that this compact-open topology has as a sub-basis of opens,
$\\{f\in\widehat{A}:f(K)\subseteq U\\}$, where $K\subseteq A$ is compact, and
$U\subseteq{\mathbb{T}}$ is open. It coincides with the topology of uniform
convergence on compact subsets of $A$ (using the metric on $\mathbb{T}$). When
$A$ is discrete the topology is precisely that induced by the Tychonoff
topology on $\mathbb{T}^{A}$.

Given an abelian topological group $A$, we have a homomorphism $ev$ (for
evaluation) from $A$ to $\widehat{\widehat{A}}$, where $ev(a)$ is the
continuous function from $\widehat{A}$ to $\mathbb{T}$ which takes $f$ to
$f(a)$. Pontryagin duality says that, for $A$ a locally compact abelian group
$A$, this homomorphism is an isomorphism of topological groups. We will only
use it in the case of compact $A$, which we state now (see [6], Chapter IV).

######  Fact 2.2.

  1. (i)

If $A$ is compact abelian, then $\widehat{A}$ is discrete abelian, and if $A$
is discrete then $\widehat{A}$ is compact.

  2. (ii)

For $A$ compact, $ev:A\to\widehat{\widehat{A}}$ is an isomorphism of
topological groups.

We can now state a (well-known) description in the above terms of the Bohr
compactification of an abelian topological group. This is often stated in the
literature only for locally compact abelian groups (and sometimes even as the
definition). See [6], VII.5. We give a proof for completeness.

######  Lemma 2.3.

Let $A$ be an abelian topological group. Let $(\widehat{A})_{d}$ be
$\widehat{A}$ but with the discrete topology. Then
$bA=\widehat{(\widehat{A})_{d}}$, namely $A\to bA$ is isomorphic to the
natural evaluation map $ev:A\to\widehat{(\widehat{A})_{d}}$

###### Proof.

Let $\iota$ be the canonical continuous homomorphism from $A$ to $bA$, which
has dense image.

Claim. $(\widehat{A})_{d}$ is (naturally) isomorphic to the group
$\widehat{bA}$ (which note is discrete by 2.2 (i)).

Proof of Claim. For each continuous homomorphism $f$ from $bA$ to
$\mathbb{T}$, $f\circ\iota$ is a continuous homomorphism from $A$ to
$\mathbb{T}$. Moreover, by the universality property of $bA$, the homomorphism
taking $f$ to $f\circ\iota$ establishes an isomorphism (of abstract groups)
between the group of homomorphisms from $A$ to $\mathbb{T}$, and the group of
homomorphisms from $bA$ to $\mathbb{T}$. But as $bA$ is compact,
$\widehat{bA}$ is discrete, hence the map taking $f$ to $f\circ\iota$ is an
isomorphism $i$ of topological groups between $(\widehat{A})_{d}$ and
$\widehat{bA}$.∎

By the claim, $\widehat{(\widehat{A})_{d}}$ is isomorphic as a topological
group to $\widehat{(\widehat{bA})}$, which by 2.2 is equal to $bA$. Checking
the appropriate maps gives the full statement of the proposition. ∎

###  2.2 Continuous logic

We discuss briefly continuous logic as in [1], where passing from formulas to
type spaces can also be interpreted as passing from a C*-algebra to its
Gelfand space (as we mention below). A different version of continuous logic
is developed in [3], from whence the question regarding Pontryagin duality
originally sprang. Modulo some care regarding the definition and
topologization of the space of types, the result and proof in Section 3 go
through essentially unchanged. We refer to the original paper for the
interested reader.

We first remind the reader of “local type spaces” in classical first order
logic. We consider an $L$-structure $M$, and a collection ${\mathcal{F}}$ of
$L$-formulas of the form $\phi(x,\overline{y})$ where the single variable $x$
is fixed and $\overline{y}$ can vary depending on $\phi$. Let $a\in M^{*}$, a
(saturated) elementary extension of $M$. The $\mathcal{F}$-type of $a$ over
$M$ is the information of which formulas of the form $\phi(x,\overline{b})$
for $\phi(x,\overline{y})$ in ${\mathcal{F}}$ and $\overline{b}$ a tuple from
$M$ are true of $a$ in $M^{*}$. We call this collection of $\mathcal{F}$-types
$S_{\mathcal{F}}(M)$. The formulas $\phi(x,\overline{b})$ for
$\phi\in{\mathcal{F}}$ and $\overline{b}$ in $M$ (modulo equivalence in $M$)
generate a Boolean algebra (under $\wedge,\vee,\neg$) and $S_{\mathcal{F}}(M)$
is precisely the set of ultrafiliters on this Boolean algebra, equipped
naturally with the Stone space topology. So the passage from a collection
${\mathcal{F}}$ of $L$-formulas with a distinguished free variable $x$ to the
collection of ${\mathcal{F}}$-types over $M$ is precisely passing from a
Boolean algebra to its Stone space.

Continuous logic is about logic where the formulas are real- or, in our case,
complex-valued and the structures may be equipped with a metric (which is
equality in the discrete case). It has been developed in a mathematical logic
or model-theoretic framework since the 1960’s, starting with Chang and
Keisler’s book [7]. Since about 15 years or so, a certain attractive framework
has been developed, the basics of which are expounded in the papers [1] and
[2], and which the reader is referred to. Some of the interest or motivation
is to do model theory or classification theory in more general environments,
as well as develop tools to apply model theory to functional analysis (and
even combinatorics). There are various informal and formal connections with
many-valued logics which other logic communities have pursued for a long time.

We are interested in abelian groups $(A,+,-,0)$ with a collection
${\mathcal{F}}$ of homomorphisms from $A$ to ${\mathbb{T}}$. From the point of
view of continuous logic, we can view these homomorphism as
$\mathbb{C}$-valued predicates. We think of $+$, $-$, $0$ as (binary, unary,
zero-ary) functions (or function symbols) on $A$. Treat $=$ as a
$\\{0,1\\}$-valued formula with value $0$ meaning equal. Atomic formulas are
of the form $f(t)$ where $f\in\mathcal{F}$ and $t$ a term. The “propositional
connectives” are continuous functions from $\mathbb{C}^{n}$ to $\mathbb{C}$.
The quantifiers are $\sup_{x}|-|$ and $\inf_{x}|-|$ (which are now
$\mathbb{R}$-valued). From these we build up the relevant formulas of
continuous logic. We denote by $M$ this continuous logic structure
$(A,+,-,0,f)_{f\in{\mathcal{F}}}$ of $(A,+,-,0)$ equipped with the
$\mathbb{C}$-valued predicates for functions in ${\mathcal{F}}$.

The compactness theorem is valid, and we have saturated elementary extensions
$M^{*}=(A^{*},+^{*},-^{*},0^{*},f^{*})_{f\in\mathcal{F}}$ of $M$. The
$\mathbb{T}$-valued formulas $f\in{\mathcal{F}}$ are without additional
parameter variables. The ${\mathcal{F}}$-type of an element $a\in A^{*}$,
$\operatorname{tp}_{\mathcal{F}}(a)$, is the information consisting of the
values $f^{*}(a)\in{\mathbb{T}}$ for $f\in{\mathcal{F}}$ (where $f^{*}$ is the
interpretation of the predicate symbol for $f$ in the model $M^{*}$). We might
want to say $\operatorname{tp}_{\mathcal{F}}(a/M)$ but there are no parameters
from $M$ to worry about. We call this collection of types $S_{\mathcal{F}}(M)$
(or $S_{\mathcal{F}}(T)$ where $T$ is the continuous logic theory of $M$). The
topology on $S_{\mathcal{F}}(M)$ is given as follows: a basic closed set is
the set of types containing the information $u(f_{1}(x),..,f_{n}(x))=r$ for
$f_{i}\in{\mathcal{F}}$, $u:{\mathbb{C}}^{n}\to{\mathbb{C}}$ continuous and
$r\in\mathbb{C}$. This is a compact Hausdorff space but not necessarily
profinite.

We can also see all this as the “usual” adaptation of Stone duality to not
necessarily profinite spaces. Namely, we can consider the C*-algebra generated
by the family ${\mathcal{F}}$ of continuous functions from $A$ to
${\mathbb{T}}\subseteq\mathbb{C}$, and then the Gelfand space of this
C*-algebra will be, as a topological space, precisely the space
$S_{\mathcal{F}}(M)$. So from this point of view our results will exhibit
Pontryagin duality as a special case of Gelfand duality.

Note that the quantifiers play no role in the type space $S_{\mathcal{F}}(M)$
(other than in defining the elementary extension $M^{*}$ of $M$ where we
realize these types.)

##  3 Theorem and Proof

Let us set up the context for the main theorem. We fix an abelian topological
group $A$, and let $\mathcal{F}$ be the collection of all continuous
homomorphisms from $A$ to $\mathbb{T}$. So $\mathcal{F}$ is a group isomorphic
to $\widehat{A}$, where the latter is considered just as an abstract group. We
let $M$ be the continuous logic structure $(A,+,-,0,f)_{f\in\mathcal{F}}$ as
in the last section. And let $M^{*}$ be a saturated elementary extension.

Consider the type space $S_{\mathcal{F}}(M)$ as defined earlier. Note that,
for $p\in S_{\mathcal{F}}(M)$, $f^{*}(p)$ makes sense defined as
$f^{*}(a)\in{\mathbb{T}}$ for some/any $a$ realizing $p$. The topology on
$S_{\mathcal{F}}(M)$ is the coarsest making all of these maps continuous. Now,
for $p,q\in S_{\mathcal{F}}(M)$, define
$p+^{*}q=\operatorname{tp}_{\mathcal{F}}(a+^{*}b)$ where $a$ realizes $p$ and
$b$ realizes $q$. Note that $+^{*}$ is well-defined, because if $a$ realizes
$p$, $b$ realizes $q$, and $f\in\mathcal{F}$ then the value of $f^{*}$ at
$a+^{*}b$ is $f^{*}(a)f^{*}(b)$, so it depends only on the value of $f^{*}$ at
$a$ and at $b$, so it depends only on $p$ and $q$.

######  Theorem 3.1.

  1. (i)

The operation $+^{*}$, together with the type space topology on
$S_{\mathcal{F}}(M)$, makes $S_{\mathcal{F}}(M)$ into a compact abelian
topological group.

  2. (ii)

This compact abelian topological group structure on $S_{\mathcal{F}}(M)$ is
precisely the Pontryagin dual of $\mathcal{F}$ (where $\mathcal{F}$ is
considered as a discrete group), under the identification of $p\in
S_{\mathcal{F}}(M)$ with the map taking $f\in\mathcal{F}$ to $f^{*}(p)$.

  3. (iii)

The map $j:A\to S_{\mathcal{F}}(M)$ taking $a\in A$ to
$\operatorname{tp}_{\mathcal{F}}(a)$, is a continuous homomorphism which is
moreover (isomorphic to) the Bohr compactification $\iota:A\to bA$ of $A$.

###### Proof.

(i) The fact that $+^{*}$ gives a group operation on $S_{\mathcal{F}}(M)$ is
obvious from the definition. To show that it is a topological group, fix some
$q_{0},q_{1}\in S_{\mathcal{F}}(M)$ and let $U=\\{p\in
S_{\mathcal{F}}(M):|f^{*}(p)-t_{0}t_{1}|<\epsilon\\}$ for some
$f\in\mathcal{F}$ and $\epsilon>0$, where $t_{0}=f^{*}(q_{0})$ and
$t_{1}=(f^{*}(q_{1}))^{-1}$. This is a subbasic open neighbourhood of
$q_{0}-^{*}q_{1}$ in $S_{\mathcal{F}}(M)$. Let $V_{0}=\\{p\in
S_{\mathcal{F}}(M):|f^{*}(p)-t_{0}|<\epsilon/2\\}$ and $V_{1}=\\{p\in
S_{\mathcal{F}}(M):|(f^{-1})^{*}(p)-t_{1}|<\epsilon/2\\}$. Take $p_{0}\in
V_{0}$, $p_{1}\in V_{1}$ and let $b_{0},b_{1}\in A^{*}$ realize $p_{0}$ and
$p_{1}$, respectively. Then

$|f^{*}(b_{0}-b_{1})-t_{0}t_{1}|\leq|f^{*}(b_{0})||(f^{-1})^{*}(b_{1})-t_{1}|+|t_{1}||f^{*}(b_{0})-t_{0}|<\epsilon.$

(Remember that $f^{*}(b_{0}),t_{1}\in\mathbb{T}$) We conclude that the map
$(p,q)\mapsto p-^{*}q$ is continuous at $(q_{0},q_{1})$ for all such pairs, as
desired.

(ii) Each $p\in S_{\mathcal{F}}(M)$ gives rise to a homomorphism from the
group $\mathcal{F}$ to $\mathbb{T}$, which we call $p^{*}$ and which is
precisely the evaluation of $f^{*}$ at a realization of $p$:
$p^{*}(f)=f^{*}(a)\in{\mathbb{T}}$, where $a$ realizes $p$ and
$f\in{\mathcal{F}}$. This clearly gives a homomorphism $h$ from
$S_{\mathcal{F}}(M)$ to the Pontryagin dual $\widehat{\mathcal{F}}$ of the
(discrete abelian) group ${\mathcal{F}}$. $h$ is also an embedding because
$p^{*}=q^{*}$ implies $f^{*}(a)=f^{*}(b)$ for all $f\in{\mathcal{F}}$ and
realizations $a$ of $p$ and $q$ of $p$.

We want $h$ to be continuous. $\widehat{\mathcal{F}}$ has the compact-open
topology, so take a basic open given by $O=\\{\alpha:\alpha(K)\subseteq U\\}$,
where $K$ is a compact so finite subset $\\{f_{1},\ldots,f_{n}\\}$ of
$\mathcal{F}$ and $U$ is an open subset of $\mathbb{T}$. Then $p^{*}\in O$ iff
$(f_{1}(x),\ldots,f_{n}(x))\in U^{n}$ is in $p$, which is an open condition on
$p$ in $S_{\mathcal{F}}(M)$.

So we have that $h$ is a continuous embedding of the compact group
$S_{\mathcal{F}}(M)$ in the compact group $\widehat{\widehat{\mathcal{F}}}$.
So $h(S_{\mathcal{F}}(M))$ is a closed subgroup of the image. However
$h(S_{\mathcal{F}}(M))$ separates points in $\widehat{\mathcal{F}}$. To see
this, suppose $f_{1}\neq f_{2}\in{\mathcal{F}}$. Suppose $a\in A$ and
$f_{1}(a)\neq f_{2}(a)$. Let $p=\operatorname{tp}_{\mathcal{F}}(a)$. Then
$p^{*}(f_{1})\neq p^{*}(f_{2})$. Hence, by Stone-Weierstrass,
$h(S_{\mathcal{F}}(M))=\widehat{\mathcal{F}}$.

(iii) $j$ is a homomorphism by definition of the group structure on
$S_{\mathcal{F}}(M)$. For continuity: Let $C$ be a closed subset of
$S_{\mathcal{F}}(M)$, so defined by a formula $(f_{1}(x),...,f_{n}(x))\in D$
for some closed $D\subseteq{\mathbb{T}}^{n}$. Then $j^{-1}(C)=\\{a\in
A:(f_{1}(a),..,f_{n}(a))\in D\\}$ which is closed in the topological group $A$
because the $f_{i}:A\to{\mathbb{T}}$ are continuous.

Let $\iota:A\to bA$ be the Bohr compactification of $A$. By Lemma 2.3 this is
isomorphic to $ev:A\to\widehat{({\hat{A}})_{d}}$. But $(\widehat{A})_{d}$ is
precisely the discrete group ${\mathcal{F}}$. So by part (ii) (and inspection
of the maps), $ev:A\to\widehat{(\widehat{A})_{d}}$ identifies with $j:A\to
S_{\mathcal{F}}(M)$. This gives the result. ∎

##  4 Comparison with the literature

In this section we comment on the extent to which Theorem 3.1 can be extracted
from or is related to results in the literature. In Subsection 4.1 we make the
connection with [8]. In Subsection 4.2 we make the connection with results
appearing in [4] including the relation between Pontryagin and Gelfand
dualities.

###  4.1 Bohr and Stone-Čech compactifications

There are a few papers dealing with the relation between the Stone-Čech
compactification $\beta G$ of a topological group $G$ and the Bohr
compactification $bG$ of $G$. See, for example, [9] for the case of discrete
abelian $G$. But we should mention to begin with that it is immediate that
$bG$ is a quotient of $\beta G$. This is because $\beta G$ is the universal
compact space on which $G$ acts by homeomorphisms with a dense orbit, and $G$
also acts by homeomorphisms on $bG$ with dense orbit.

In any case, in [8] we also describe $bG$ for $G$ a topological group as an
image of $\beta G$, working in classical first order logic/nonstandard
analysis, and this will give another account of Theorem 3.1 (iii) which we
briefly describe.

We fix a topological group $G$ and let $M$ be the structure consisting of $G$,
its group operation, and predicates for all subsets of $G$. The space of
complete $1$-types over $M$, which we write as $S_{1}(M)$ is by definition the
space of ultrafilters on the Boolean algebra of all subsets of $G$, which is
the same thing as the Stone-Čech compactification $\beta G$ of $G$. Let
$M^{*}$ be a sufficiently saturated elementary extension of $M$ (nonstandard
model), and write $M^{*}=(G^{*},\times^{*},P^{*})_{P\subseteq G}$. Then there
is a smallest subgroup of $G^{*}$ which has index at most $2^{2^{|G|}}$ and is
the intersection of some subsets of $G$ of the form $U^{*}$ where $U$ is an
open subset of $G$. We call this subgroup $(G^{*})^{00}_{top}$. It is proved
in Fact 2.4 of [8] that $(G^{*})^{00}_{top}$ is a normal subgroup of $G^{*}$
and that the quotient group $G^{*}/(G^{*})^{00}_{top}$ with the “logic
topology” and the natural homomorphism from $G$ is the Bohr compactification
$bG$ of the topological group $G$. (The statement actually first appears in
[5].)

The logic topology is as follows. First, because of the “bounded index”
condition, the quotient homomorphism $G^{*}\to G^{*}/(G^{*})^{00}_{top}$
factors through $S_{1}(M)=\beta G$, so we have a surjective map $\pi:\beta
G\to G^{*}/(G^{*})^{00}_{top}$ and the logic topology is the quotient
topology. In particular, $bG$ is a continuous image of $\beta G$.

The proof of Lemma 2.2 of [8] yields that any map $f$ from $G$ to a compact
space $C$ extends uniquely to a map $f^{*}:G^{*}\to C$ with the property that
$f^{*}$ factors through a continuous map $S_{1}(M)\to C$. Moreover, if $C$ has
a compact group structure and $f$ is a homomorphism, then $f^{*}$ is also a
homomorphism.

The only “new thing” we say here is the following:

######  Proposition 4.1.

Assume now that $G=A$ is an abelian topological group. Then
$(A^{*})^{00}_{top}=\bigcap_{f\in\widehat{A}}\ker(f^{*})$.

###### Proof.

This comes out of the proof of Fact 2.4 (ii) in [8]. Let $f\in\widehat{A}$. By
Lemma 2.2 of [8], for any $f\in\widehat{A}$, $\ker(f^{*})$ is an intersection
of sets $U^{*}$, where $U$ is an open subset of $A$, whereby
$(A^{*})^{00}_{top}\subseteq\bigcap_{f\in\widehat{A}}\ker(f^{*})$.

Hence, we have a surjective continuous homomorphism from
$A^{*}/(A^{*})^{00}_{top}$ to $A^{*}/\bigcap_{f\in\widehat{A}}\ker(f^{*})$. We
claim that this is an isomorphism (of topological groups). If not, there is
nonzero $x\in A^{*}/(A^{*})^{00}_{top}$ whose image in
$A^{*}/\bigcap_{f\in\widehat{A}}\ker(f^{*})$ is $0$. But, from the structure
of compact abelian groups, there will be a continuous homomorphism
$h:A^{*}/(A^{*})^{00}_{top}\to\mathbb{T}$ such that $h(x)\neq 0$. Considering
the composition of the canonical homomorphism from $A^{*}$ to
$A^{*}/(A^{*})^{00}_{top}$ with $h$, we obtain a (suitable) homomorphism from
$A^{*}$ to $\mathbb{T}$ which is nonzero, contradicting that the image of $x$
in $A^{*}/\bigcap_{f\in\widehat{A}}\ker(f^{*})$ is $0$. This proves the
result. ∎

Hence, for $p,q\in S_{1}(M)=\beta A$, $\pi(p)=\pi(q)$ (in
$bA=A^{*}/(A^{*})^{00}_{top}$) if $f^{*}(a)=f^{*}(b)$ for each
$f\in\widehat{A}$ and realizations $a$ of $p$ and $b$ of $q$. In other words,
if $f^{*}(p)=f^{*}(q)$ for all $f\in\widehat{A}$.

Hence $bA$ can again be seen as the space of $\mathcal{F}$-types, where
$\mathcal{F}=\widehat{A}$. But now $f\in\mathcal{F}$ is considered as a
definable over $M$ homomorphism from $G^{*}$ to $\mathbb{T}$ in the sense of
first order logic (i.e. factoring through $S_{1}(M)$). In any case, its
topology as a quotient of $\beta G$ and as a type space in continuous logic in
the sense of Section 2 coincide.

###  4.2 Gelfand and Pontryagin dualities

In the functional analysis literature, Theorem 3.1 occurs modulo Gelfand
duality, but stated just for the case where $A$ is also locally compact. For
example, Theorem 4.79 in [4], with some paraphrasing and some editing, reads:

######  Theorem.

Let $A$ be a locally compact abelian group and $f:A\to\mathbb{C}$ a bounded
and continuous function. Then the following are equivalent:

  1. 1.

$f$ factors through $\iota:A\to bA$.

  2. 2.

$f$ is the uniform limit of linear combinations of characters of $A$.

Paraphrasing once more, this theorem says that $C(bA)$ and the C*-algebra
generated by the $\mathcal{F}$-formulas coincide. Taking their spectra yields,
on the one hand, the Bohr compactification $bG$ of $G$ (see e.g. Theorem 1.16
in [4]); and on the other, the space of $\mathcal{F}$-types
$S_{\mathcal{F}}(M)$. In Section 3 we go through the proof of this from a
model-theoretic perspective, without invoking the nature of the space of types
as a Gelfand dual.

Alternatively, the literature also reveals the Pontryagin dual of a locally
compact group $G$ as the spectrum of the Banach algebra $L^{1}(G)$ of Haar-
integrable functions on $G$ modulo the Fourier transform ([4] Theorem 4.2).
Regarding the latter, we have

######  Proposition ([4] Proposition 4.13).

The Fourier transform is a norm-decreasing *-homomorphism from $L^{1}(G)$ to
$C_{0}(\widehat{G})$ with dense image.

In particular, for any locally compact abelian group $A$, we have the Fourier
transform $L^{1}((\widehat{A})_{d})\to
C_{0}(\widehat{(\widehat{A})_{d}})=C(bA)$. The C*-algebra generated by
$L^{1}((\widehat{A})_{d})$ is then the algebra of $\mathcal{F}$-formulas, as
implied by the theorem above, and their spectra again are both the Bohr
compactification $bA$ of $A$ and $S_{\mathcal{F}}(M)$.

Returning to Theorem 4.79 in [4], there is actually a third equivalent
statement mentioned there: $f$ is uniformly almost periodic. This is a strong
form of stability, possibly connected to “$1$-basedness” and we will
investigate the implications in future work.

In any case we have not managed to see our proofs in Section 3 “mirroring” the
accounts just mentioned.

## References

  * [1] Itaï Ben Yaacov, Alexander Berenstein, C. Ward Henson, and Alexander Usvyatsov.  Model theory for metric structures.  Model theory with applications to algebra and analysis, 2:315–427, 2008.
  * [2] Itaï Ben Yaacov and Alexander Usvyatsov.  Continuous first order logic and local stability.  Transactions of the American Mathematical Society, 362(10):5213–5259, 2010.
  * [3] Nicolas Chavarria and Anand Pillay.  On $pp$-elimination and stability in a continuous setting.  https://arxiv.org/abs/2107.14329, 2021.
  * [4] Gerald B. Folland.  A Course in Abstract Harmonic Analysis.  CRC Press, Inc., 1995.
  * [5] Jakub Gismatullin, Davide Penazzi, and Anand Pillay.  On compactifications and the topological dynamics of definable groups.  Annals of Pure and Applied Logic, 165(2):552–562, 2014.
  * [6] Yitzhak Katznelson.  An introduction to Harmonic Analysis.  Cambridge University Press, 2004.
  * [7] H. Jerome Keisler and Chen Chung Chang.  Continuous Model Theory.  Princeton University Press, 1966.
  * [8] Krzysztof Krupiński and Anand Pillay.  Amenability, definable groups, and automorphism groups.  Advances in Mathematics, 345:1253–1299, 2019.
  * [9] Pavol Zlatoš.  The Bohr compactification of an abelian group as a quotient of its Stone–Čech compactification.  Semigroup Forum, 101:497–506, 2020.


in terms of an $N$-leading Gaussian viz. Eq. (78). Let $\lambda_{a}>0$ denote
the $a$-th eigenvalue of the quadratic form in Eq. (130) and group its
orthonormal eigenvectors into a $k\times k$ orthogonal matrix $\mathbf{U}$.
Setting

$\displaystyle\pi_{a}=\sum_{b=1}^{k}U_{ab}x_{b}$

we diagonalize the Gaussian in Eq. (130) by the canonical fluctuations
$\boldsymbol{x}\in\mathbb{R}^{k}$,

$\displaystyle\mathcal{K}(\boldsymbol{x})=\sum_{a=1}^{k}\frac{x_{a}^{2}}{\lambda_{a}}~{},$

with unit-$\det$ Jacobian. Subsequently we eliminate the positive eigenvalues
by setting

$\displaystyle x_{a}=\sqrt{\lambda_{a}}\,y_{a}\quad\text{so
that}\quad\mathcal{K}(\boldsymbol{y})=\sum_{a=1}^{k}y_{a}^{2}~{},$

where the Jacobian $\sqrt{\prod_{a}\lambda_{a}}=\mathcal{O}\left(1\right)$ of
the total differential in Eq. (131) gets canceled by the partition function
$Z$. Eventually, switching to spherical coordinates with unit-$\det$ Jacobian
via

$\displaystyle r^{2}=\sum_{a=1}^{k}y_{a}^{2}$

such that the probability becomes with Gaussian function $f(r)\equiv
f(\boldsymbol{\pi})$

$\displaystyle\text{d}\Omega_{k}\,\text{d}r\,r^{k-1}f(r)~{},$ (132)

the spherical symmetry of the $N$-leading Gaussian term readily manifests,

$\displaystyle\mathcal{K}(r)=r^{2}~{}.$

It is trivial to execute the angular integration, where the solid angle in $k$
dimensions equals

$\displaystyle\Omega_{k}=\frac{2\pi^{k/2}}{\Gamma(k/2)}~{}.$ (133)

Finally, we express Eq. (132) in terms of the quadratic form,

$\displaystyle\Omega_{k}\,\text{d}r\,r^{k-1}\,f(r)=\Omega_{k}\,\text{d}\mathcal{K}\underbrace{\mathcal{K}^{k/2-1}\,f(\mathcal{K})}_{\rho(\mathcal{K})}$
(134)

using the squared radius in kernel space as integration variable,
$\text{d}(r^{2}/2)=\text{d}\mathcal{K}$. From Eq. (77), we know that

$\displaystyle\mathcal{K}(\boldsymbol{r})=D\hskip
0.28453pt(\mathfrak{p}\;\|\;\mathfrak{q})+\mathcal{O}\left(N^{-1/2}\right)$
(135)

so that the r.h.s. of probability Eq. (134) can be re-expressed in terms of
$D\hskip 0.28453pt(\mathfrak{p}\;\|\;\mathfrak{q})$, with the density $\rho$
taking the desired form of Eq. (81) up to $N$-suppressed corrections.

###  B.7 The probability of equivalence classes

See 4.2

#### Proof

As outlined in Eq. (76), a distribution on inner TotEmplex
$[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]\subset[\mathfrak{f};{\mathbf{\,\hat{M}}}]$
can be parameterized through probability fluctuations around the
$I$-projection of reference distribution onto the ambient TotEmplex
$[\mathfrak{f};{\mathbf{\,\hat{M}}}]$. According to Theorem 3.7, a Gaussian
density always approximates at sufficiently large-$N$ the distribution of such
probability fluctuations around
$\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}$ on the
ambient TotEmplex.

Next, we formulate

######  Lemma B.5.

Given the ambient TotEmplex $[\mathfrak{f};{\mathbf{\,\hat{M}}}]$, the
probability at large $N$ to sample from reference $\boldsymbol{\upsilon}$ any
distribution on the inner TotEmplex
$[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]$ is determined to $N$-leading
order by $D\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]}\boldsymbol{\upsilon}\;\|\;\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon})$.
Parametrizing this $I$-divergence by probability fluctuations in the normal
space complementing $\ker\mathbf{\,\hat{M}^{\prime}}$ to
$\ker\mathbf{\,\hat{M}}$, the associated probability density follows a
multivariate Gaussian distribution with $D^{\prime}-D$ degrees of freedom.

#### Proof

Using the definition of quotient space, the quadratic form of Lemma 3.8 that
governs the distribution of distributions Eq. (73) on the ambient TotEmplex
decomposes with
$\mathfrak{q}\equiv\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}$
according to

$\displaystyle\,\,2D\hskip
0.28453pt(\mathfrak{p}\;\|\;\mathfrak{q})=2\langle\boldsymbol{\pi}\mathfrak{q}^{-1}\boldsymbol{\pi}\rangle$
(136) $\displaystyle=$
$\displaystyle\,\,\sum_{a^{\prime},b^{\prime}=1}^{|\ker\mathbf{\,\hat{M}}^{\prime}|}\pi_{a^{\prime}}\pi_{b^{\prime}}\langle\,\hat{N}_{a^{\prime}}\mathfrak{q}^{-1}\,\hat{N}_{b^{\prime}}\rangle+\sum_{a,b=1}^{D^{\prime}-D}\pi_{a}\pi_{b}\langle\,\hat{N}_{a}\mathfrak{q}^{-1}\,\hat{N}_{b}\rangle+2\sum_{a,a^{\prime}}\pi_{a^{\prime}}\langle\,\hat{N}_{a^{\prime}}\mathfrak{q}^{-1}\,\hat{N}_{a}\rangle\pi_{a}$

for any $\mathfrak{p}\in[\mathfrak{f};{\mathbf{\,\hat{M}}}]$. Integrating Eq.
(73) over all fluctuations $\pi_{a^{\prime}}$ with positive-definite precision
matrix
$\mathcal{K}^{\prime}_{a^{\prime}b^{\prime}}=\langle\,\hat{N}_{a^{\prime}}\mathfrak{q}^{-1}\,\hat{N}_{b^{\prime}}\rangle$
we can calculate the probability density to sample from reference
${\boldsymbol{\upsilon}}$ any distribution ${\mathfrak{p}^{\prime}}$ on the
inner TotEmplex. In the resulting marginal, there still remains a Gaussian in
$|\ker\mathbf{\,\hat{M}}|-|\ker\mathbf{\,\hat{M}}^{\prime}|=D^{\prime}-D$
dimensions defined by quadratic form

$\displaystyle 2D\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]}\boldsymbol{\upsilon}\;\|\;\mathfrak{q})=\min\limits_{\mathfrak{p}^{\prime}\in[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]}2D\hskip
0.28453pt(\mathfrak{p}^{\prime}\;\|\;\mathfrak{q})$ (137)
$\displaystyle=\sum_{a,b=1}^{D^{\prime}-D}\pi_{a}\left(\langle\,\hat{N}_{a}\mathfrak{q}^{-1}\,\hat{N}_{b}\rangle-\sum_{a^{\prime},b^{\prime}=1}^{|\ker\mathbf{\,\hat{M}}^{\prime}|}\langle\,\hat{N}_{a^{\prime}}\mathfrak{q}^{-1}\,\hat{N}_{a}\rangle(\mathcal{K}^{\prime})^{-1}_{a^{\prime}b^{\prime}}\langle\,\hat{N}_{b^{\prime}}\mathfrak{q}^{-1}\,\hat{N}_{b}\rangle\right)\pi_{b}~{},$

see Appendix D.1 for technical details.

Descending from a positive-definite quadratic form $\mathcal{K}$ on the
ambient TotEmplex (due to Lemma 3.8), the resulting quadratic form is also
positive-definite. In total, we find that in the quotient space
$[\mathfrak{f};{\mathbf{\,\hat{M}}}]/\mathbf{\,\hat{M}^{\prime}}$ the
conjectured density Eq. (88) that governs the probability of inner TotEmplex
$[\mathfrak{p};{\mathbf{\,\hat{M}}^{\prime}}]$ is self-consistently
approximated by a Gaussian distribution in the normal fluctuations. …∎

Along the lines of Corollary 3.9, we can transform to spherical coordinates in
order to integrate-out the angular dependence. After rescaling the radius
squared to achieve $Q=2ND\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]}\boldsymbol{\upsilon}\;\|\;\mathfrak{q})$,
we can eventually express the radial probability to select from reference
${\boldsymbol{\upsilon}}$ any TotEmplex from the quotient space with
$I$-divergence in the vicinity of $Q/2N$ by a chi-squared distribution in
$D^{\prime}-D$ dimensions.

###  B.8 The efficiency of MLE

See 4.3

#### Proof

In the convenient scenario that we know the target distribution, we exemplify
the efficiency of operator selection. Starting from some reference
distribution ${\boldsymbol{\upsilon}}$ we concretely verify that information
score Eq. (87) is expected to be maximized for TotEms constructed by vector
operators that are fapp-equivalent to $\mathbf{\,\hat{T}}$. To do so we need
to take expectations over (infinitely) many datasets.

In a hypothetical setting, datasets $\mathbf{X}$ of fixed size $N$ are
generated from target distribution ${\mathfrak{t}}$. Each dataset is described
by empirical distributions ${\mathfrak{f}}\in\mathcal{P}$, as outlined in Eq.
(7). Over sampled phenomenologies
$\langle\mathcal{E}_{\star}\;|\mathfrak{f}\mathbf{\,\hat{M}}|\;\mathcal{E}_{\star}\rangle$,
we take the expectation of information score Eq. (87) assigned to TotEms
constructed by $\mathbf{\,\hat{M}}$,

$\displaystyle\sum_{N\mathfrak{f}}\text{mult}\left(N\mathfrak{f};\mathfrak{t}\right)\text{score}_{I}=-N\sum_{N\mathfrak{f}}\text{mult}\left(N\mathfrak{f};\mathfrak{t}\right)D\hskip
0.28453pt(\mathfrak{f}\;\|\;\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon})+\frac{|\ker\mathbf{\,\hat{M}}|}{2}\log
N+\mathcal{O}\left(1\right)~{},$ (138)

where the sum runs over all count vectors
$N\mathfrak{f}\in\mathbb{N}_{0}^{|\mathcal{E}|}$ that are partitions of $N$.
At this level, recall that incompatible prior knowledge which falsely declares
nullentities from $\mathcal{E}\setminus\mathcal{T}_{0}$ would eventually lead
Eq. (87) to diverge to $-\infty$, see also discussion below Definition 2.1.
Consequently, we only need to consider TotEms with nullentities from some
$\mathcal{E}_{0}\subseteq\mathcal{T}_{0}$.

Henceforth, we focus on the $N$-leading contribution to the $I$-score.
Recycling the argumentation of Section 3.1, which becomes meaningful at larger
$N$, we can write an integral representation for the expectation:

$\displaystyle Z^{-1}\int_{\mathfrak{f}\in\mathcal{P}}\exp\left\\{-ND\hskip
0.28453pt(\mathfrak{f}\;\|\;\mathfrak{t})\right\\}D\hskip
0.28453pt(\mathfrak{f}\;\|\;\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon})~{}.$
(139)

In each TotEmplex $[\mathfrak{q};{\mathbf{\,\hat{M}}}]$ from the quotient
space $\mathcal{P}/\mathbf{\,\hat{M}}$ labeled by
$\mathbf{\,\hat{M}}$-distinct knowledge $\mathfrak{q}$, we next apply Lemma
B.2 from the true perspective,

$\displaystyle D\hskip 0.28453pt(\mathfrak{f}\;\|\;\mathfrak{t})=D\hskip
0.28453pt(\mathfrak{f}\;\|\;\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t})+D\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t}\;\|\;\mathfrak{t})$
(140)

for sampled empirical $\mathfrak{f}\in[\mathfrak{q};{\mathbf{\,\hat{M}}}]$.
This observation covariantly breaks the integral Eq. (139) over the simplex
into

$\displaystyle\int_{\mathfrak{q}^{*}}\exp\left\\{-ND\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t}\;\|\;\mathfrak{t})\right\\}\int_{\mathfrak{f}\in[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\exp\left\\{-ND\hskip
0.28453pt(\mathfrak{f}\;\|\;\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t})\right\\}D\hskip
0.28453pt(\mathfrak{f}\;\|\;\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon})~{}.$
(141)

The first integral of the density in quotient space runs over all
$\mathbf{\,\hat{M}}$-distinct distributions $\mathfrak{q}$ in the sense of
equivalence relation Eq. (85), while the second integral of TotEmplex density
runs within $\ker\mathbf{\,\hat{M}}$ over distributions
${\mathfrak{f}}\in[\mathfrak{q};{\mathbf{\,\hat{M}}}]$, as in Section 3.4.
Appropriate normalization factors are understood for both integrals.

Applying localization techniques on the inner integral, the saddle point of
the exponential function is readily given by the $I$-projection
$\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t}$ of target
distribution onto the TotEmplex. Considering probability fluctuations around
this distribution gives a Gaussian integral in $\ker\mathbf{\,\hat{M}}$ (cf.
Eq. (130)), which is well-defined at large $N$. On the other hand, the
$I$-score is calculated using prior knowledge $\boldsymbol{\upsilon}$ with an
a priori different $I$-projection on the same TotEmplex. Generically, we thus
anticipate the integral representation Eq. (139) to scale as
$\mathcal{O}\left(1\right)$, so that the expected score Eq. (138) scales
linearly with negative sample size.

However, if $\mathbf{\,\hat{M}}$ acting over
$\mathcal{E}\setminus\mathcal{T}_{0}$ captures all measurements of
$\mathbf{\,\hat{T}}$, then $D\hskip
0.28453pt(\mathfrak{f}\;\|\;\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon})=D\hskip
0.28453pt(\mathfrak{f}\;\|\;\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t})$
on account of

######  Lemma B.6.

Consider a TotEm over $\mathcal{E}\setminus\mathcal{T}_{0}$ constructed by
vector operator $\mathbf{\,\hat{M}}$ describing $D$ independent measurements
which imply in the sense of Definition 3.5 the
$\boldsymbol{\upsilon}$-efficient vector operator $\mathbf{\,\hat{T}}$ that
parameterizes the target distribution,

$\displaystyle\hat{T}_{\alpha}=\sum_{\alpha^{\prime}=1}^{D}S_{\alpha\alpha^{\prime}}\hat{M}_{\alpha^{\prime}}\quad\text{for}\quad\alpha=1,\ldots,D_{min}~{}.$
(142)

The $I$-projection of reference distribution $\boldsymbol{\upsilon}$ onto the
TotEmplex $[\mathfrak{q};{\mathbf{\,\hat{M}}}]$ constructed by
$\mathbf{\,\hat{M}}$ coincides with the $I$-projection of true knowledge
$\mathfrak{t}$ onto $[\mathfrak{q};{\mathbf{\,\hat{M}}}]$.

#### Proof

For the rest of this proof, we silently imply the set $\mathcal{T}_{0}$ of
truly non-realizable entities. We start from the definition of the
$I$-projection of ${\mathfrak{t}}$ onto $[\mathfrak{q};{\mathbf{\,\hat{M}}}]$:

$\displaystyle\forall~{}\mathfrak{p}\in[\mathfrak{q};{\mathbf{\,\hat{M}}}]\,:\quad
D\hskip 0.28453pt(\mathfrak{p}\;\|\;\mathfrak{t})\geq D\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t}\;\|\;\mathfrak{t})\quad$
$\displaystyle\Leftrightarrow$ (143) $\displaystyle D\hskip
0.28453pt(\mathfrak{p}\;\|\;\boldsymbol{\upsilon})\geq D\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t}\;\|\;\boldsymbol{\upsilon})+\sum_{\alpha=1}^{D_{min}}\tau_{\alpha}\sum_{\alpha^{\prime}=1}^{D}S_{\alpha\alpha^{\prime}}\underbrace{\langle\left(\mathfrak{p}-\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}\right)\,\hat{M}_{a^{\prime}}\rangle}_{=0}~{}.$

The expectation difference of all $\,\hat{M}_{\alpha^{\prime}}$ vanishes,
since both $\mathfrak{p}$ and
$\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}$ belong to
$[\mathfrak{q};{\mathbf{\,\hat{M}}}]$. Hence,
$\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t}$ is also the
$I$-projection of $\boldsymbol{\upsilon}$ onto
$[\mathfrak{q};{\mathbf{\,\hat{M}}}]$ and since the $I$-projection of a
reference distribution onto the TotEmplex is by Theorem 3.1 unique, we
conclude that
$\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t}=\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}$.
…∎

This Lemma helps evaluate the second integral in Eq. (141) within
$\ker\mathbf{\,\hat{M}}$,

$\displaystyle\int_{\mathfrak{f}\in[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\exp\left\\{-ND\hskip
0.28453pt(\mathfrak{f}\;\|\;\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t})\right\\}ND\hskip
0.28453pt(\mathfrak{f}\;\|\;\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon})=\frac{|\ker\mathbf{\,\hat{M}}|}{2}=\mathcal{O}\left(1\right)$
(144)

using Eq. (203), whenever $\mathbf{\,\hat{M}}$ satisfies Eq. (142).

In total, we conclude for Eq. (138) that

$\displaystyle\sum_{N\mathfrak{f}}\text{mult}\left(N\mathfrak{f};\mathfrak{t}\right)\text{score}_{I}=\begin{cases}\frac{|\ker\mathbf{\,\hat{M}}|}{2}\log
N+\mathcal{O}\left(1\right)&\mathbf{\,\hat{M}}\text{ implies
}\mathbf{\,\hat{T}}\text{ over
}\mathcal{E}\setminus\mathcal{T}_{0}\\\\[4.30554pt]
-\mathcal{O}\left(N\right)&\text{otherwise}\end{cases}~{}.$ (145)

Among all the $\mathbf{\,\hat{M}}$ that imply vector operator
$\mathbf{\,\hat{T}}$ parametrizing the target knowledge, clearly the operator
that is fapp equivalent to $\mathbf{\,\hat{T}}$ exhibits the largest kernel
and hence maximizes our information score according to mle principle.

###  B.9 The expected divergence from target

To provide further evidence in favor of the minimization of $I$-divergence, we
next evaluate the expected $I$-divergence from target distribution
${\mathfrak{t}}$ of the $I$-projection of reference ${\boldsymbol{\upsilon}}$
onto the TotEmplex $[\mathfrak{f};{\mathbf{\,\hat{M}}}]$ constructed on
empirical $\mathfrak{f}$ which was sampled from $\mathfrak{t}$. Similar to Eq.
(141), the integral representation over datasets breaks under the quotient
action into integration over equivalence classes and integration within
TotEmplices. Trivially executing the inner integral within
$\ker\mathbf{\,\hat{M}}$ like in Eq. (144), we are left this time with an
integral over $\mathbf{\,\hat{M}}$-distinct distributions,

$\displaystyle N\sum_{N\mathfrak{f}}D\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}\;\|\;\mathfrak{t})=Z^{-1}\int_{\mathfrak{q}^{*}}\exp\left\\{-ND\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t}\;\|\;\mathfrak{t})\right\\}ND\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}\;\|\;\mathfrak{t})+\ldots~{}.$
(146)

Saddle-point techniques indicate that the integral localizes around
$\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t}$. As noted in
the proof of Theorem 4.3, whenever $\mathcal{S}_{0}\subset\mathcal{T}_{0}$ or
if $\mathbf{\,\hat{M}}$ is not sufficient to capture all the measurements of
$\mathbf{\,\hat{T}}$, the expectation of the $I$-divergence based on
${\boldsymbol{\upsilon}}$ remains non-vanishing at large $N$. Hence, we only
need to focus on vector operators $\mathbf{\,\hat{M}}$ acting over
$\mathcal{E}\setminus\mathcal{T}_{0}$ that satisfy Eq. (142). On such
TotEmplices, the $I$-projections of reference and target distribution coincide
due to Lemma B.6.

Introducing a displaced target operator

$\displaystyle\hat{q}=\exp\left\\{\sum_{\alpha=1}^{D-1}\theta_{\alpha}\,\hat{M}_{\alpha}\right\\}\mathfrak{t}$
(147)

via linearly independent operators $\,\hat{M}_{\alpha}$ that do not imply the
identity $\,\mathbb{I}_{\mathcal{E}}$, we can parameterise the $I$-projection
of ${\mathfrak{t}}$ onto TotEmplices from the quotient space
$\mathcal{P}(\mathcal{E}\setminus\mathcal{T}_{0})/\mathbf{\,\hat{M}}$
according to

$\displaystyle\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}=\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t}=\langle\hat{q}\rangle^{-1}\hat{q}~{}.$
(148)

Recall that expectations are taken w.r.t.
$|\mathcal{E}_{\star}\rangle=\sum_{{\boldsymbol{e}}\in{\boldsymbol{\mathcal{D}}_{\star}}}|{\boldsymbol{e}}\rangle$.
Expanding in “small” fluctuations,

$\displaystyle\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t}=\left[1+\sum_{\alpha=1}^{D-1}\left(\,\hat{M}_{\alpha}-\langle\mathfrak{t}\,\hat{M}_{\alpha}\rangle\right)\theta_{\alpha}+\mathcal{O}\left(\boldsymbol{\theta}^{2}\right)\right]\mathfrak{t}~{},$
(149)

gives rise to

$\displaystyle D\hskip
0.28453pt(\mathfrak{t}\;\|\;\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t})=\mbox{$\frac{1}{2}$}\vartheta_{\alpha}\langle(\,\hat{M}_{\alpha}-\langle\mathfrak{t}\,\hat{M}_{\alpha}\rangle)\mathfrak{t}(\,\hat{M}_{\beta}-\langle\mathfrak{t}\,\hat{M}_{\beta}\rangle)\rangle\vartheta_{\beta}+\mathcal{O}\left(\boldsymbol{\vartheta}^{2}\right)=D\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{q};{\mathbf{\,\hat{M}}}]}\mathfrak{t}\;\|\;\mathfrak{t})~{}.$
(150)

Due to vanishing linear term, the Taylor expansion up to quadratic order
symmetrizes the $I$-divergence.

The measuring operators $\,\hat{M}_{\alpha}$ are per assumption linearly
independent, while
$c\,\mathbb{I}_{\mathcal{E}}=\sum_{a=1}^{D-1}c_{\alpha}\,\hat{M}_{\alpha}$
implies $c=c_{\alpha}=0$. Hence, the shifted operators
$\,\hat{M}_{\alpha}-\langle\mathfrak{t}\,\hat{M}_{\alpha}\rangle=\,\hat{M}_{\alpha}-\langle\mathfrak{t}\,\hat{M}_{\alpha}\rangle\,\mathbb{I}_{\mathcal{E}}$
remain linearly independent and can be summarized by an irreducible vector
operator in the sense of 2.2. Applying Lemma B.3, we conclude that the
precision matrix in Eq. (150) is positive definite, following the
argumentation in the proof of Theorem 4.2. Along similar lines, the Gaussian
approximation in the small-fluctuations expansion Eq. (149) becomes a
large-$N$ expansion around the saddle point of Eq. (146), so that we can
evaluate the expectation by Eq. (203). In total, we have

$\displaystyle\sum_{N\mathfrak{f}}ND\hskip
0.28453pt(\mathfrak{t}\;\|\;\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon})$
(151) $\displaystyle\approx$ $\displaystyle\sum_{N\mathfrak{f}}ND\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}\;\|\;\mathfrak{t})=\begin{cases}\frac{D-1}{2}+\mathcal{O}\left(N^{-1/2}\right)&\mathbf{\,\hat{M}}\text{
implies }\mathbf{\,\hat{T}}\text{ over
}\mathcal{E}\setminus\mathcal{T}_{0}\\\\[4.30554pt]
\mathcal{O}\left(N\right)&\text{otherwise}\end{cases}~{}.$

For all operators that imply $\mathbf{\,\hat{T}}$ the expectation depends on
the number $D$ of measurements that construct the associated phenomenology.
For any other operator, the expectation diverges with $N$. Therefore, the
$I$-projection of ${\boldsymbol{\upsilon}}$ onto the TotEmplex constructed by
$\mathbf{\,\hat{T}}$ over $\mathcal{E}\setminus\mathcal{T}_{0}$ exhibits the
lowest expected $I$-divergence from the target distribution. In particular,
whenever ${\boldsymbol{\upsilon}}={\mathfrak{t}}$ the expected $I$-divergence
vanishes asymptotically. This extends the results of [10] beyond uniform prior
and categorical attributes.

###  B.10 Logistic regression

To relate to the formal definition of logistic regression we assume for
simplicity that both response and predictors are binary, i.e.
$\mathcal{E}=\\{\texttt{TRUE},\texttt{FALSE}\\}^{m+1}$. Since the constructing
element Eq. (91) is reducible, the maxent distribution Eq. (92)

$\displaystyle\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{R}}}]}\mathfrak{u}=\sum_{y,\mathbf{x}}\exp\\{H(y,\mathbf{x})\\}|y,\mathbf{x}\rangle\langle
y,\mathbf{x}|$ (152)

with energy function

$\displaystyle
H(y,\mathbf{x})=\theta_{0}+\theta_{\texttt{Y}}(y)+\sum_{i=1}^{m}\theta^{i}_{\texttt{Y},\texttt{X}}(y,x_{i})+\Theta(\mathbf{x})$
(153)

enjoys gauge symmetry that leaves $H$ invariant in Lagrange multipliers:

$\displaystyle\theta_{0}~{}{\rightarrow}~{}$
$\displaystyle\tilde{\theta}_{0}=\theta_{0}+\gamma_{0}\quad,\quad\theta_{\texttt{Y}}(y)~{}{\rightarrow}~{}\tilde{\theta}_{\texttt{Y}}(y)=\theta_{\texttt{Y}}(y)-\gamma_{0}+\sum_{i=1}^{m}\gamma^{i}_{\texttt{Y}}(y)~{},$
$\displaystyle\theta^{i}_{\texttt{Y},\texttt{X}}(y,x_{i})~{}{\rightarrow}~{}$
$\displaystyle\tilde{\theta}^{i}_{\texttt{Y},\texttt{X}}(y,x_{i})=\theta^{i}_{\texttt{Y},\texttt{X}}(y,x_{i})-\gamma^{i}_{\texttt{Y}}(y)-\gamma^{i}_{\texttt{X}}(x_{i})$
(154) $\displaystyle\quad\text{and}\quad\Theta(\mathbf{x})~{}{\rightarrow}~{}$
$\displaystyle\tilde{\Theta}(\mathbf{x})=\Theta(\mathbf{x})+\sum_{i=1}^{m}\gamma^{i}_{\texttt{X}}(x_{i})~{}.$

In the gauge where we take

$\displaystyle\gamma_{0}=\theta_{\texttt{Y}}(0)+\sum_{i=1}^{m}\left(\theta^{i}_{\texttt{Y},\texttt{X}}(0,0)-\gamma^{i}_{\texttt{X}}(0)\right)\quad,\quad$
$\displaystyle\gamma^{i}_{\texttt{X}}(1)=\gamma^{i}_{\texttt{X}}(0)+\theta^{i}_{\texttt{Y},\texttt{X}}(0,1)-\theta^{i}_{\texttt{Y},\texttt{X}}(0,0)$
$\displaystyle\gamma^{i}_{\texttt{Y}}(0)=-\gamma^{i}_{\texttt{X}}(0)+\theta^{i}_{\texttt{Y},\texttt{X}}(0,0)\quad,\quad$
$\displaystyle\gamma^{i}_{\texttt{Y}}(1)=-\gamma^{i}_{\texttt{X}}(0)+\theta^{i}_{\texttt{Y},\texttt{X}}(1,0)$
with
$\displaystyle\sum_{i=1}^{m}\gamma^{i}_{\texttt{X}}(0)=-\Theta(\mathbf{0})\quad\text{and}\quad\gamma^{i}_{\texttt{X}}(0)\in\mathbb{R}~{},$

most transformed Lagrange multipliers on the r.h.s. of Eq. (B.10) vanish
identically:

$\displaystyle\tilde{\theta}_{\texttt{Y}}(0)=0\quad,\quad\tilde{\theta}^{i}_{\texttt{Y},\texttt{X}}(Y,0)=0\quad,\quad\tilde{\theta}^{i}_{\texttt{Y},\texttt{X}}(0,X_{i})=0\quad\text{and}\quad\tilde{\Theta}(\mathbf{0})=0~{}.$
(155)

Subsequently, we count

$\displaystyle\text{rank}\,\mathbf{\,\hat{R}}=1+1+m+2^{m}-1=1+m+2^{m}$ (156)

remaining Lagrange multipliers encoding independent marginal constraints.

In such gauge, we can equivalently write Lagrange multipliers in terms of
regression coefficients $\beta_{0}\in\mathbb{R}$ and
$\boldsymbol{\beta}\in\mathbb{R}^{m}$ as

$\displaystyle\tilde{\theta}_{0}=-\log
Z\quad,\quad\tilde{\theta}_{\texttt{Y}}(y)=\beta_{0}s_{0}\quad\text{and}\quad\tilde{\theta}^{i}_{\texttt{Y},\texttt{X}}(y,x_{i})=\beta_{i}s_{0}s_{i}$
(157)

via encoding

$\displaystyle
s_{0}(y=\text{False})=0~{},~{}s_{0}(y=\text{True})=1\quad\text{and}\quad
s_{i}(x_{i}=\text{False})=0~{},~{}s_{i}(x_{i}=\text{True})=1~{}.$

This parametrizes the entity space by a binary vector
$|s_{0};\mathbf{s}\rangle$ where $s_{0}\in\\{0,1\\}$ and
$\mathbf{s}\in\\{0,1\\}^{m}$. Our energy function Eq. (153) becomes

$\displaystyle
H(s_{0},\mathbf{s})=\left(\beta_{0}+\boldsymbol{\beta}^{T}\mathbf{s}\right)s_{0}+\Theta(\mathbf{s})-\log
Z~{}.$ (158)

Using the phenomenological constraint on the joint predictor probabilities

$\displaystyle\langle(\mathfrak{p}-\mathfrak{f})\,\mathbb{I}_{\mathbf{s}}\rangle=0$
(159)

we can eventually bring the maxent distribution Eq. (92) to the more familiar
form

$\displaystyle\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{R}}}]}\mathfrak{u}=\sum_{s_{0},\mathbf{s}}\langle\mathfrak{f}\,\mathbb{I}_{\mathbf{s}}\rangle\frac{\exp\left\\{\left(\beta_{0}+\boldsymbol{\beta}^{T}\mathbf{s}\right)s_{0}\right\\}}{1+\exp\left\\{\beta_{0}+\boldsymbol{\beta}^{T}\mathbf{s}\right\\}}|s_{0},\mathbf{s}\rangle\langle
s_{0},\mathbf{s}|~{}.$ (160)

where multiple summation over the encoded Eq. (B.10) entity space is
abbreviated by

$\displaystyle\sum_{s_{0},\mathbf{s}}\equiv\sum_{s_{0}\in\\{0,1\\}}\prod_{i=1}^{m}\sum_{s_{i}\in\\{0,1\\}}$
(161)

In particular, the conditional probability invoked in the definition of the
odds for response $s_{0}=1$ given predictor profile $\mathbf{s}$ can be
directly read off:

$\displaystyle\frac{\langle
y,\mathbf{s}\;|\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{R}}}]}\mathfrak{u}|\;y,\mathbf{s}\rangle}{\langle\mathfrak{f}\,\mathbb{I}_{\mathbf{s}}\rangle}=\frac{\exp\left\\{\left(\beta_{0}+\boldsymbol{\beta}^{T}\mathbf{s}\right)s_{0}\right\\}}{1+\exp\left\\{\beta_{0}+\boldsymbol{\beta}^{T}\mathbf{s}\right\\}}~{}.$
(162)

The remaining non-vanishing Lagrange multipliers $\beta_{0}$ and
$\boldsymbol{\beta}$ are fixed by the empirical estimates of one- and two-site
marginals

$\displaystyle\langle(\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{R}}}]}\mathfrak{u}-\mathfrak{f})\,\mathbb{I}_{s_{0}}\rangle=0\quad\text{and}\quad\langle(\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{R}}}]}\mathfrak{u}-\mathfrak{f})\,\mathbb{I}_{s_{0},\mathbf{s}}\rangle=0~{},$
(163)

respectively. Due to redundancies in the marginal description, it suffices to
consider e.g. $s_{0}=1$ and $s_{i}=1$, so that (using the eigenvalues of
projector operators $\delta(s_{0},1)=s_{0}$ and $\delta(s_{i},1)=s_{1}$)

$\displaystyle\sum_{\mathbf{s}}\frac{\langle\mathfrak{f}\,\mathbb{I}_{\mathbf{s}}\rangle}{1+\exp\\{-\beta_{0}-\boldsymbol{\beta}^{T}\mathbf{s}\\}}\overset{!}{=}$
$\displaystyle\,\langle\mathfrak{f}\,\mathbb{I}_{s_{0}=1}\rangle\quad\text{and}$
(164)
$\displaystyle\sum_{\mathbf{s}}\frac{\langle\mathfrak{f}\,\mathbb{I}_{\mathbf{s}}\rangle
s_{i}}{1+\exp\\{-\beta_{0}-\boldsymbol{\beta}^{T}\mathbf{s}\\}}\overset{!}{=}$
$\displaystyle\,\langle\mathfrak{f}\,\mathbb{I}_{s_{0}=1,s_{i}=1}\rangle~{}.$

In the model-centric picture, the same equations arise by maximizing the
likelihood

$\displaystyle\ell(\mathfrak{f};\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{R}}}]}\mathfrak{u})=-H[\mathfrak{f}]-D\hskip
0.28453pt(\mathfrak{f}\;\|\;\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{R}}}]}\mathfrak{u})$
(165)

(equivalently minimizing the $I$-divergence of empirical from model
distribution) as a function of the regression coefficients, see also
discussion in C.

###  B.11 Coin experiments

#### Perception of one coin

According to Theorem 3.2, we anticipate

$\displaystyle\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{H}}}]}\mathfrak{u}=Z^{-1}\exp\\{\theta\,\hat{H}\\}\mathfrak{u}~{},$
(166)

for the $I$-projection of $\mathfrak{u}$ onto the TotEmplex
$[\mathfrak{f};{\mathbf{\,\hat{H}}}]$ constructed by
$\mathbf{\,\hat{H}}=\\{\,\mathbb{I}_{\mathcal{E}},\,\hat{H}\\}$. The partition
function implementing normalization (from $\,\mathbb{I}_{\mathcal{E}}$)
evaluates to a closed form,

$\displaystyle
Z=\langle\exp\\{\theta\,\hat{H}\\}\mathfrak{u}\rangle=2^{-L}\prod_{i=1}^{L}\sum_{s_{i}\in\mathcal{D}_{i}}\exp\\{\theta
l(\boldsymbol{s})/L\\}=$
$\displaystyle\,\,2^{-L}\sum_{l=0}^{L}b_{L,l}\exp\\{\theta l/L\\}$
$\displaystyle=$
$\displaystyle\,\,2^{-L}\left(1+\exp\\{\theta/L\\}\right)^{L}~{},$ (167)

Substituting into the phenomenological constraint of mean success rate $\eta$,
we can solve for the Lagrange multiplier,

$\displaystyle\langle\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{H}}}]}\mathfrak{u}\,\hat{H}\rangle=\frac{1}{1+e^{-\theta/L}}\overset{!}{=}\langle\mathfrak{f}\,\hat{H}\rangle\equiv\eta\quad\Rightarrow\quad\theta=L\log\frac{\eta}{1-\eta}~{},$
(168)

so that in total we arrive at (recall that the resolution of identity is given
by Eq. (18))

$\displaystyle\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{H}}}]}\mathfrak{u}=$
$\displaystyle\,\,\exp\left\\{L\log\eta\,\hat{H}+L\log(1-\eta)(\hat{I}-\hat{H})\right\\}\,\mathbb{I}_{\mathcal{S}}$
(169) $\displaystyle=$
$\displaystyle\,\,\sum_{\boldsymbol{s}\in\mathcal{S}}\eta^{l(\boldsymbol{s})}\left(1-\eta\right)^{L-l(\boldsymbol{s})}|\boldsymbol{s}\rangle\langle\boldsymbol{s}|~{}.$

This is the least biased distribution given knowledge of entity space and the
empirical success rate $\eta=\langle\mathfrak{f}\hat{H}\rangle$.

Whenever the frequencies of success numbers are provided, we analogously find
the maxent distribution

$\displaystyle\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{K}}}]}\mathfrak{u}=\exp\left\\{\sum_{k=0}^{L}\vartheta_{k}\,\mathbb{I}_{k}\right\\}\mathfrak{u}=\prod_{k=0}^{L}\exp\left\\{\log\frac{\phi_{k}}{b_{L,k}}\,\,\mathbb{I}_{k}\right\\}\,\mathbb{I}_{\mathcal{S}}=\sum_{\boldsymbol{s}\in\mathcal{S}}\frac{\phi_{l(\boldsymbol{s})}}{b_{L,l(\boldsymbol{s})}}|\boldsymbol{s}\rangle\langle\boldsymbol{s}|$
(170)

where Lagrange multipliers are fixed according to

$\displaystyle\vartheta_{k}=\log\frac{2^{L}\phi_{k}}{b_{L,k}}~{}.$ (171)

Ultimately, we can test the “binomial hypothesis”, i.e. whereas the mean
success rate is sufficient to describe the distribution of success rates,
using statistics Eq. (101) or in operator language:

$\displaystyle D\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{K}}}]}\mathfrak{u}\;\|\;\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{H}}}]}\mathfrak{u})=$
(172)
$\displaystyle\left\langle\exp\left\\{\sum_{k=0}^{L}\log\frac{\phi_{k}}{b_{L,k}}\,\mathbb{I}_{k}\right\\}\left[\sum_{k^{\prime}=0}^{L}\log\frac{\phi_{k^{\prime}}}{b_{L,k^{\prime}}}\,\mathbb{I}_{k^{\prime}}-L\log\eta\,\hat{H}-L\log(1-\eta)(\,\mathbb{I}_{\mathcal{E}}-\,\hat{H})\right]\right\rangle$
$\displaystyle=\sum_{k=0}^{L}\phi_{k}\log\frac{\phi_{k}}{b_{L,k}\eta^{k}\left(1-\eta\right)^{L-k}}~{}.$

To arrive at Eq. (172), we do not need to assume independence of Bernoulli
experiments. $k$-marginal distribution
$\langle\mathfrak{f}\,\mathbb{I}_{k}\rangle$ can have any form, even
evidencing extreme violation of independence between Bernoulli experiments,
which would be quickly detected by the $I$-test.

To analytically investigate the sensitivity of TotEm description when we know
the target $\mathfrak{t}$ beforehand, assume that there exists a mild
violation of the independence between $i_{0}$-th and $j_{0}$-th Bernoulli
experiments quantified via the two-point correlator

$\displaystyle\kappa\equiv\langle\mathfrak{t}\,\mathbb{I}_{s_{i_{0}}=\texttt{head}}\,\mathbb{I}_{s_{j_{0}}=\texttt{head}}\rangle-\langle\mathfrak{t}\,\mathbb{I}_{s_{i_{0}}=\texttt{head}}\rangle\langle\mathfrak{t}\,\mathbb{I}_{s_{j_{0}}=\texttt{head}}\rangle~{}.$
(173)

By encoding

$\displaystyle\sigma_{i}(s_{i}=\texttt{head})=1\quad\text{and}\quad\sigma_{i}(s_{i}=\texttt{tail})=0\quad\text{for}\quad
i=1,\ldots L$ (174)

Bernoulli configurations can be for example generated through an Ising-like
system [68] described by target distribution

$\displaystyle\mathfrak{t}=Z^{-1}\sum_{\boldsymbol{\sigma}\in\\{0,1\\}^{L}}\exp\left\\{J\sigma_{i_{0}}\sigma_{j_{0}}+h\sum_{i=1}^{L}\sigma_{i}\right\\}|\boldsymbol{\sigma}\rangle\langle\boldsymbol{\sigma}|$
(175)

with partition function

$\displaystyle Z=\left(1+e^{h}\right)^{L-2}\left(1+2e^{h}+e^{J+2h}\right)~{}.$
(176)

This implies “free” success rates

$\displaystyle\langle\mathfrak{t}\,\mathbb{I}_{s_{i}=\texttt{head}}\rangle=\frac{1}{1+e^{-h}}\quad\text{for}\quad
i\neq i_{0},j_{0}~{},$ (177)

as before ($J=0\Leftrightarrow\kappa=0$ recovers the “free” theory of
independent Bernoulli trials) and in addition “interacting” success rates

$\displaystyle\langle\mathfrak{t}\,\mathbb{I}_{s_{i_{0}}=\texttt{head}}\rangle=\langle\mathfrak{t}\,\mathbb{I}_{s_{j_{0}}=\texttt{head}}\rangle=\frac{e^{h}\left(1+e^{h+J}\right)}{1+2e^{h}+e^{J+2h}}~{}.$
(178)

In particular, the $k$-marginals estimate to

$\displaystyle\langle\mathfrak{t}\,\mathbb{I}_{k}\rangle=Z^{-1}\left[b_{L-2,k-2}e^{hk+J}+b_{L-2,k}e^{hk}+2b_{L-2,k-1}e^{hk}\right]~{}.$
(179)

The Ising parameters (coupling $J$ and external field $h$) are given by the
inverse problem, perturbatively:

$\displaystyle
h=\log\frac{\eta}{1-\eta}-\frac{2\kappa}{L\eta(1-\eta)^{2}}+\mathcal{O}\left(\kappa^{2}\right)\quad\text{and}\quad
J=\frac{\kappa}{\eta^{2}(1-\eta)^{2}}+\mathcal{O}\left(\kappa^{2}\right)~{}.$
(180)

In such scenario, the expected $I$-divergence Eq. (172), which at large $N$ is
dominated by the saddle point (cf. Eq. (139)) around the true distribution,
would scale

$\displaystyle\int_{\mathfrak{f}}\exp\\{-ND\hskip
0.28453pt(\mathfrak{f}\;\|\;\mathfrak{t})\\}D\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{K}}}]}\mathfrak{u}\;\|\;\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{H}}}]}\mathfrak{u})=$
$\displaystyle\,\,D\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{t};{\mathbf{\,\hat{K}}}]}\mathfrak{u}\;\|\;\mathbb{P}_{[\mathfrak{t};{\mathbf{\,\hat{H}}}]}\mathfrak{u})+\ldots$
$\displaystyle=$
$\displaystyle\,\,\frac{1}{L(L-1)}\frac{\kappa^{2}}{\eta^{2}(1-\eta)^{2}}+\ldots~{}.$
(181)

Consequently, the test statistics in 4.2 is expected to lead to significant
conclusions when

$\displaystyle N\gg L(L-1)\frac{\eta^{2}(1-\eta)^{2}}{2\kappa^{2}}~{}.$ (182)

#### Perception of two coins

The $I$-projection Eq. (107) of $\mathfrak{u}$ onto a TotEmplex constructed by
$\mathbf{\,\hat{H}}=\\{\,\mathbb{I}_{\mathcal{E}},\,\mathbb{I}_{\texttt{A}},\,\hat{H}\\}$
can be analytically expressed as

$\displaystyle\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{H}}}]}\mathfrak{u}=$
$\displaystyle\,\,\exp\left\\{\log\Phi_{\texttt{A}}\,{\,\mathbb{I}_{\texttt{A}}}+\log\Phi_{\texttt{B}}\,{\,\mathbb{I}_{\texttt{B}}}+L\log\eta\,\,\hat{H}+L\log(1-\eta)(\,\mathbb{I}_{\mathcal{E}}-\,\hat{H})\right\\}\text{\small
1}\text{1}_{\mathcal{E}}$ $\displaystyle=$
$\displaystyle\,\,\sum_{{\boldsymbol{e}}\in\boldsymbol{\mathcal{D}}}\Phi_{\texttt{A}}^{\delta(g,\texttt{A})}\Phi_{\texttt{B}}^{\delta(g,\texttt{B})}\eta^{l(\boldsymbol{s})}(1-\eta)^{L-l(\boldsymbol{s})}|{\boldsymbol{e}}\rangle\langle{\boldsymbol{e}}|$
$\displaystyle=$
$\displaystyle\,\,\sum_{g\in\\{\texttt{A,B}\\}}\Phi_{\texttt{g}}\sum_{\boldsymbol{s}\in\mathcal{S}}\eta^{l(s)}(1-\eta)^{L-l(\boldsymbol{s})}|g;\boldsymbol{s}\rangle\langle
g;\boldsymbol{s}|$

in terms of the global mean success rate $\eta$ and Group prevalences
$\Phi_{g}\equiv\langle\mathfrak{f}\,\mathbb{I}_{g}\rangle$ for
$g=\texttt{A,B}$.

The $I$-projection Eq. (110) of $\mathfrak{u}$ onto the TotEmplex
$[\mathfrak{f};{\mathbf{\,\hat{H}^{\prime}}}]$ constructed by
$\mathbf{\,\hat{H}^{\prime}}=\\{\,\mathbb{I}_{\texttt{A}},\,\mathbb{I}_{\texttt{B}},\,\hat{H}\,\mathbb{I}_{\texttt{A}},\,\hat{H}\,\mathbb{I}_{\texttt{B}}\\}$
(fapp equivalently by
$\\{\,\mathbb{I}_{\mathcal{E}},\,\mathbb{I}_{\texttt{A}},\,\hat{H},\,\hat{H}\,\mathbb{I}_{\texttt{A}}\\}$)
can also be analytically written as

$\displaystyle\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{H}^{\prime}}}]}\mathfrak{u}=$
$\displaystyle\,\,\prod_{g\in\\{\texttt{A,B}}\exp\left\\{\log\Phi_{g}\,\mathbb{I}_{g}+L\log\eta_{g}\,\hat{H}\,\mathbb{I}_{g}+L\log(1-\eta_{g})(\,\mathbb{I}_{\mathcal{E}}-\,\hat{H})\,\mathbb{I}_{g}\right\\}\,\mathbb{I}_{\mathcal{E}}$
$\displaystyle=$
$\displaystyle\,\,\sum_{g\in\\{\texttt{A,B}\\}}\Phi_{g}\sum_{\boldsymbol{s}\in\mathcal{S}}\eta_{g}^{l(\boldsymbol{s})}\left(1-\eta_{g}\right)^{L-l(\boldsymbol{s})}|g;\boldsymbol{s}\rangle\langle
g;\boldsymbol{s}|~{},$ (183)

in terms of Group prevalences and Group mean success rates.

##  Appendix C Relations to model-centric literature

#### The BIC score

The $I$-score of Definition 4.1 can be obtained in a model-centric approach,
as well, which is in a sense perpendicular to the data-centric derivation
previously discussed.

Conventionally, Eq. (38) is assumed to be a function of some parameters
reflecting structural assumptions – called a model $\mathcal{M}$ – about the
underlying problem. Notice that the likelihood Eq. (40) can be thought of as
the leading model-dependent term governing the large-$N$ expansion Eq. (44) of

$\displaystyle\log\text{mult}\left(N\mathfrak{f};\mathfrak{p}\right)=N\left[\ell(\mathfrak{f};\mathfrak{f})-\ell(\mathfrak{f};\mathfrak{p})\right]+\mathcal{O}\left(\log
N\right)$ (184)

A priori, lacking information about microscopic interactions, there is no
appealing reason to commit to any structural assumption. In any case, fully
working with some parameterized structure is only sensible in absence of any
nullentities, otherwise singularities would plague the parametrization
eventually leading to its break-down. Hence, we can only – without
regularizing – investigate problems with $\mathcal{E}_{0}=\emptyset$ in the
following of this paragraph.

According to the insight previously gained, the model distribution could be
expressed via a parameterized141414In a slight abuse of notation, we let
$\boldsymbol{\theta}$ denote both Lagrange multipliers in Section 3.2 that are
fixed by the given data as well as generic parameters in the present context
that can vary independently of the data. distribution

$\displaystyle\mathfrak{p}(\boldsymbol{\theta})=\exp\left\\{\sum_{a=1}^{D}\theta_{\alpha}\,\hat{M}_{\alpha}\right\\}\boldsymbol{\upsilon}=\sum_{{\boldsymbol{e}}\in\boldsymbol{\mathcal{D}}}v_{\boldsymbol{e}}\exp\left\\{\sum_{a=1}^{D}\theta_{\alpha}m_{\alpha}({\boldsymbol{e}})\right\\}|{\boldsymbol{e}}\rangle\langle{\boldsymbol{e}}|~{},$
(185)

in terms of the $D\times|\mathcal{E}|$ architecture matrix $m$ (of full row-
rank) whose components would correspond to the eigenvalues in Eq. (17) of our
measuring operators. In other words, model knowledge parametrizes
$\mathbf{\,\hat{M}}$-disinct distributions over the quotient space
$\mathcal{P}/\mathbf{\,\hat{M}}$, cf. discussion below Eq. (85). In the model-
centric approach, $D-1$ is often called the model dimension.

In a Bayesian setting under uniform prior (leading-order derivation should not
depend on any sensible prior [69]), we represent the model evidence through a
truncated expansion of Eq. (184)

$\displaystyle
P(\texttt{X}|\mathcal{M})=Z^{-1}\int\text{d}^{D}\boldsymbol{\theta}\,\delta\left(N\langle\mathfrak{p}(\boldsymbol{\theta})\rangle-N\right)\exp\left\\{-N\ell(\mathfrak{f};\mathfrak{p}(\boldsymbol{\theta}))\right\\}~{}.$
(186)

This can be seen as a large-$N$ approximation (viz. Eq. (46)) of the
cumulative probability to sample the observed counts $Nf_{\boldsymbol{e}}$
starting from all possible parametric realizations
$\boldsymbol{\theta}\in\mathbb{R}^{D}$ of the considered model structure.
Again, $Z$ is the partition function which self-consistently normalizes $P$ in
the large-$N$ expansion. Using the integral representation of Dirac-delta
function, we can rewrite the model evidence as

$\displaystyle
P(\texttt{X}|\mathcal{M})=Z^{-1}\int\frac{\text{d}\theta_{0}}{2\pi}\int\text{d}^{D}\boldsymbol{\theta}\,e^{N\mathcal{L}}$
(187)

in terms of Lagrangian (constants in $\boldsymbol{\theta}$ can be absorbed by
$Z$)

$\displaystyle\mathcal{L}=\sum_{\alpha=1}^{D}\theta_{\alpha}\sum_{{\boldsymbol{e}}\in\mathcal{E}}m_{\alpha}({\boldsymbol{e}})f_{\boldsymbol{e}}+i\theta_{0}\left(\sum_{{\boldsymbol{e}}\in\mathcal{E}}p_{\boldsymbol{e}}(\boldsymbol{\theta})-1\right)~{}.$
(188)

Due to the $\mathcal{O}\left(N\right)$ scaling of the exponent in the
integrand, Laplace approximation applies, as in the proof of Theorem 3.7. The
saddle point equations are

$\displaystyle\sum_{{\boldsymbol{e}}\in\boldsymbol{\mathcal{D}}}m_{\alpha}({\boldsymbol{e}})\left[f_{\boldsymbol{e}}-i\theta_{0}\,p_{\boldsymbol{e}}(\boldsymbol{\theta})\right]\overset{!}{=}0\quad\text{for}\quad\alpha=1,\ldots,D\quad\text{and}\quad\sum_{{\boldsymbol{e}}\in\boldsymbol{\mathcal{D}}}p_{\boldsymbol{e}}(\boldsymbol{\theta})-1\overset{!}{=}0~{}.$
(189)

Since normalization on $\mathcal{P}$ is always implied by the eigenvalues
$m_{\boldsymbol{e}}({\boldsymbol{e}})$, it follows from the first set of
equations given normalization of empirical ${\mathfrak{f}}\in\mathcal{P}$ that

$\displaystyle
i\theta_{0}\langle\mathfrak{p}(\boldsymbol{\theta})\rangle\overset{!}{=}\langle\mathfrak{f}\rangle=1~{},$
(190)

which combined with the latter equation implies $\overline{\theta}_{0}=-i$.
Hence, Eq. (189) reduces to

$\displaystyle\sum_{{\boldsymbol{e}}\in\mathcal{E}}m_{\alpha}({\boldsymbol{e}})\left[f_{\boldsymbol{e}}-p_{\boldsymbol{e}}(\overline{\boldsymbol{\theta}})\right]\overset{!}{=}0$
(191)

which reproduces the phenomenological system defining a TotEm in 2.4. Thus,
the Maximum likelihood Estimator (mle) at $\overline{\boldsymbol{\theta}}$
coincides with the $I$-projection of ${\boldsymbol{\upsilon}}$ onto the
corresponding TotEmplex,
$\mathfrak{p}(\overline{\boldsymbol{\theta}})=\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}$.

At the same time, the Hessian of the Lagrangian computes to

$\displaystyle\alpha,\beta=1,\ldots,D~{}:\quad\frac{\partial^{2}\mathcal{L}}{\partial\theta_{\alpha}\partial\theta_{\beta}}=\langle\,\hat{M}_{\alpha}\mathfrak{p}(\boldsymbol{\theta})\,\hat{M}_{\beta}\rangle\quad,\quad\frac{\partial^{2}\mathcal{L}}{\partial\theta_{0}\partial\theta_{\alpha}}=i\langle\mathfrak{p}(\boldsymbol{\theta})\,\hat{M}_{\alpha}\rangle\quad\text{and}\quad\frac{\partial^{2}\mathcal{L}}{\partial\theta_{0}^{2}}=0~{}.$
(192)

At the saddle point,

$\displaystyle\overline{\mathcal{L}}=-N\ell(\mathfrak{f};\mathfrak{q})\quad\text{with}\quad\mathfrak{q}\equiv\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}~{},$
(193)

the determinant of the Hessian breaks into block determinants

$\displaystyle\det\left(\langle\mathbf{\,\hat{M}}\mathfrak{q}\mathbf{\hat{M}}^{T}\rangle\right)\cdot\det\left[\left(i\langle\mathfrak{q}\mathbf{M}\rangle\right)^{\dagger}\left(\langle\mathbf{\,\hat{M}}\mathfrak{q}\mathbf{\hat{M}}^{T}\rangle\right)^{-1}\left(i\langle\mathfrak{q}\mathbf{M}\rangle\right)\right]~{}.$
(194)

Transposition and hermitian conjugation act on the indices of vector operator
2.2. Recall from Eq. (54) that the $I$-projection does not introduce any zero
probabilities, if not dictated by the phenomenology. Due to Lemma B.3,
$\langle\mathbf{\,\hat{M}}\mathfrak{q}\mathbf{\hat{M}}^{T}\rangle$ is positive
definite. Furthermore, the second determinant simply reduces to the quadratic
form of the inverse of
$\langle\mathbf{\,\hat{M}}\mathfrak{q}\mathbf{\hat{M}}^{T}\rangle$ (recall
that the inverse of a positive definite matrix is also positive definite). The
determinants of the blocks thus remain positive asserting the consistency of
the mle at the saddle point Eq. (189).

Executing eventually the integration in Eq. (187) we arrive at

$\displaystyle\log
P\left(\texttt{X};\mathcal{M}\right)=-\mbox{$\frac{1}{2}$}\textsc{bic}+\mathcal{O}\left(1\right)$
(195)

in terms of the Bayesian information criterion [70]

$\displaystyle\textsc{bic}\equiv\left(D-1\right)\log
N-2\overline{\mathcal{L}}~{}.$ (196)

In absence of sampling zeros when parametric models are expected to behave
smoothly, $\mathcal{E}_{\star}=\mathcal{E}$ at least for all considered
constructing elements $\mathbf{\,\hat{M}}$. In such setting, maximizing score
Eq. (87) given some data X becomes equivalent to minimizing the bic score Eq.
(196). Whether a universally accepted way exists to compute
$\mathcal{O}\left(1\right)$ corrections to the bic and whether the equivalence
of the model- and data-centric pictures continuous remains open.

#### The likelihood ratio

The $I$-divergence governing the distribution over nested TotEmplices in the
quotient space
$[\mathfrak{f};{\mathbf{\,\hat{M}}}]/\mathbf{\,\hat{M}^{\prime}}$ can be
viewed as determining a likelihood ratio of simpler $\mathcal{M}$ to more
complex $\mathcal{M}^{\prime}$ model, $D<D^{\prime}$. For that, we consider
some inner TotEmplex labeled by
$\mathfrak{p}\in[\mathfrak{f};{\mathbf{\,\hat{M}}}]$ and a test distribution
$\mathfrak{q}\in[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]\subset[\mathfrak{f};{\mathbf{\,\hat{M}}}]$.
Starting from Eq. (184) evaluated at the saddle point Eq. (189) where the mle
over parameter space fixes the model distributions to the corresponding
$I$-projection of $\boldsymbol{\upsilon}$ in ambient and inner TotEmplex,
$\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}$ and
$\mathbb{P}_{[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]}\boldsymbol{\upsilon}$
respectively, the likelihood ratio ($LR$) could be defined via

$\displaystyle\log
LR\equiv\log\frac{\text{mult}(N\mathfrak{q};\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon})}{\text{mult}(N\mathfrak{q};\mathbb{P}_{[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]}\boldsymbol{\upsilon})}=-N\left[D\hskip
0.28453pt(\mathfrak{q}\;\|\;\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon})-D\hskip
0.28453pt(\mathfrak{q}\;\|\;\mathbb{P}_{[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]}\boldsymbol{\upsilon})\right]+\mathcal{O}\left(1\right)~{}.$
(197)

In the model-centric dual picture, this $N$-leading estimate gives the ratio
of the likelihood to sample e.g. the data X summarised by
$\mathfrak{q}=\mathfrak{f}$ under $\mathcal{M}$ over the likelihood to sample
${\mathfrak{f}}$ under $\mathcal{M}^{\prime}$.

Applying Lemma B.2 twice for
${\mathfrak{q}}\in[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]$ and
${\mathfrak{q}}\in[\mathfrak{f};{\mathbf{\,\hat{M}}}]$, we see that

$\displaystyle\log LR=-N\left[D\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]}\boldsymbol{\upsilon}\;\|\;\boldsymbol{\upsilon})-D\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}\;\|\;\boldsymbol{\upsilon})\right]+\mathcal{O}\left(1\right)~{}.$
(198)

In turn, applying again Lemma B.2 for
$\mathbb{P}_{[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]}\boldsymbol{\upsilon}\in[\mathfrak{f};{\mathbf{\,\hat{M}}}]$
we arrive at

$\displaystyle\log LR=-ND\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{p};{\mathbf{\,\hat{M}^{\prime}}}]}\boldsymbol{\upsilon}\;\|\;\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon})+\mathcal{O}\left(1\right)~{}.$
(199)

This expression for the $LR$ confirms the form of the density appearing in Eq.
(88), when interpreted as a likelihood ratio [71] among the nested TotEm
constructed by $\mathbf{\,\hat{M}}^{\prime}$ that implies the ambient TotEm
constructed by $\mathbf{\,\hat{M}}$. To rigorously derive the distribution of
the $LR$ – analogously to Theorem 4.2 – in the model-centric formulation
requires an extended discussion, see e.g. [72, 73].

##  Appendix D Gaussian integrals

For any real symmetric positive-definite matrix $\mathbf{A}$ the associated
quadratic form

$\displaystyle
Q_{\mathbf{A}}(\mathbf{x})=\mathbf{x}^{T}\mathbf{A}\mathbf{x}=\sum_{i,j=1}^{n}x_{i}A_{i,j}x_{j}$
(200)

remains strictly positive for any
$\mathbf{x}\in\mathbb{R}^{n}\setminus\\{0\\}$. The core formula we shall need
is the multivariate Gaussian integral with source term
$\mathbf{j}\in\mathbb{R}^{n}$

$\displaystyle\int\text{d}^{n}\mathbf{x}\exp\left\\{-\mbox{$\frac{1}{2}$}Q_{\mathbf{A}}(\mathbf{x})-\mathbf{x}^{T}\mathbf{j}\right\\}=\sqrt{\frac{(2\pi)^{n}}{\det\mathbf{A}}}\exp\left\\{\mbox{$\frac{1}{2}$}\mathbf{j}^{T}\mathbf{A}^{-1}\mathbf{j}\right\\}~{}.$
(201)

In the context of multivariate statistics, $\mathbf{A}$ is called the
precision matrix and its inverse the covariance of the associated normal
distribution

$\displaystyle\text{norm}\left(\mathbf{x};\mathbf{A}^{-1}\mathbf{j},\mathbf{A}^{-1}\right)=\sqrt{\frac{1}{\det\left(2\pi\mathbf{A}^{-1}\right)}}\exp\left\\{-\mbox{$\frac{1}{2}$}\left(\mathbf{x}-\mathbf{A}^{-1}\mathbf{j}\right)^{T}\mathbf{A}\left(\mathbf{x}-\mathbf{A}^{-1}\mathbf{j}\right)\right\\}$
(202)

with mean $\langle\mathbf{x}\rangle=\mathbf{A}^{-1}\mathbf{j}$.

An often encountered expectation over a Gaussian function is of the quadratic
form

$\displaystyle\sqrt{\frac{1}{\det\left(2\pi\mathbf{A}^{-1}\right)}}\int\text{d}^{n}\mathbf{x}\,e^{-\mbox{$\frac{1}{2}$}Q_{\mathbf{A}}(\mathbf{x})}\,\mbox{$\frac{1}{2}$}Q_{\mathbf{A}}(\mathbf{x})=\frac{n}{2}~{}.$
(203)

###  D.1 Marginals of Gaussian distribution

To marginalize over some of the directions in $\mathbf{x}$ only, we need to
partition the ambient space into subspaces

$\displaystyle\mathbf{x}=\begin{pmatrix}\mathbf{x}_{1}\\\
\mathbf{x}_{2}\end{pmatrix}~{},$ (204)

where $\mathbf{x}_{a}\in\mathbb{R}^{d_{a}}$ for $a=1,2$. This breaks the
precision matrix into so-called conformable partitions

$\displaystyle\mathbf{A}=\begin{pmatrix}\mathbf{A}_{1}&\mathbf{B}\\\
\mathbf{B}&\mathbf{A}_{2}\end{pmatrix}~{},$ (205)

where $\mathbf{B}$ is a $d_{1}\times d_{2}$ real matrix. $\mathbf{A}_{a}$ are
taken to be symmetric positive-definite matrices and hence invertible.
Accordingly, the quadratic form Eq. (200) gives

$\displaystyle
Q_{\mathbf{A}}(\mathbf{x})=\mathbf{x}^{T}_{1}\mathbf{A}_{1}\mathbf{x}_{1}+\mathbf{x}^{T}_{2}\mathbf{A}_{2}\mathbf{x}_{2}+2\mathbf{x}^{T}_{1}\mathbf{B}\mathbf{x}_{2}$
(206)

For clarity, we set the source term to zero, $\mathbf{j}=\mathbf{0}$, which
can be achieved by a shift of the coordinate system.

Marginalizing over the Gaussian integral in say $\mathbf{x}_{1}$ gives

$\displaystyle\int\text{d}^{d_{1}}\mathbf{x}_{1}\text{norm}\left(\mathbf{x};\mathbf{0},\mathbf{A}^{-1}\right)=\sqrt{\frac{\det\left(2\pi\mathbf{A}_{1}^{-1}\right)}{\det\left(2\pi\mathbf{A}^{-1}\right)}}\exp\left\\{-\mbox{$\frac{1}{2}$}\mathbf{x}_{2}^{T}\left(\mathbf{A}_{2}-\mathbf{B}^{T}\mathbf{A}^{-1}_{1}\mathbf{B}\right)\mathbf{x}_{2}\right\\}$
(207)

which is again a $d_{2}$-dimensional Gaussian integral in $\mathbf{x}_{2}$.
That the resulting precision matrix is positive definite can be easily seen by
completing the square for $\mathbf{x}_{1}$ in Eq. (206):

$\displaystyle
Q_{\mathbf{A}}(\mathbf{x})=\left(\mathbf{x}_{1}+\mathbf{A}_{1}^{-1}\mathbf{B}\mathbf{x}_{2}\right)^{T}\mathbf{A}_{1}\left(\mathbf{x}_{1}+\mathbf{A}_{1}^{-1}\mathbf{B}\mathbf{x}_{2}\right)+\mathbf{x}_{2}^{T}\left(\mathbf{A}_{2}-\mathbf{B}^{T}\mathbf{A}_{1}^{-1}\mathbf{B}\right)\mathbf{x}_{2}~{}.$
(208)

Per assumption, the l.h.s. always stays positive for any
$\mathbf{x}\in\mathbb{R}^{n}\setminus\\{0\\}$, thus for any vectors
$\mathbf{x}_{a}\in\mathbb{R}^{d_{a}}\setminus\\{0\\}$. Choose in particular
$\mathbf{x}_{1}=-\mathbf{A}_{1}^{-1}\mathbf{B}\mathbf{x}_{2}$, then

$\displaystyle\forall~{}\mathbf{x}_{2}\in\mathbb{R}^{d_{2}}\setminus\\{0\\}\,:\quad
0<Q_{\mathbf{A}}=\mathbf{x}_{2}^{T}\left(\mathbf{A}_{2}-\mathbf{B}^{T}\mathbf{A}_{1}^{-1}\mathbf{B}\right)\mathbf{x}_{2}$
(209)

Hence,
$\mathbf{A}/\mathbf{A}_{1}\equiv\mathbf{A}_{2}-\mathbf{B}^{T}\mathbf{A}_{1}^{-1}\mathbf{B}$
has to be positive definite. Incidentally, $\mathbf{A}/\mathbf{A}_{1}$ is
called the Schur complement of block $\mathbf{A}_{1}$ for which the relation

$\displaystyle\det\mathbf{A}=\det\mathbf{A}_{1}\cdot\det\mathbf{A}/\mathbf{A}_{1}$
(210)

holds by the rules of determinants. This means that the normalization of the
marginal integral in Eq. (207) reduces to

$\displaystyle\frac{\det\left(2\pi\mathbf{A}_{1}^{-1}\right)}{\det\left(2\pi\mathbf{A}^{-1}\right)}=\left[{\det\left(2\pi\left(\mathbf{A}/\mathbf{A}_{1}\right)^{-1}\right)}\right]^{-1}$
(211)

using that $n=d_{1}+d_{2}$ for complementary spaces in $\mathbf{R}^{n}$.
Consequently,

$\displaystyle\int\text{d}^{d_{1}}\mathbf{x}_{1}\text{norm}\left(\mathbf{x};\mathbf{0},\mathbf{A}^{-1}\right)=\text{norm}\left(\mathbf{x}_{2};\mathbf{0},\left(\mathbf{A}/\mathbf{A}_{1}\right)^{-1}\right)~{},$
(212)

asserting the closure of normal distribution under marginalization.

##  Appendix E Constrained integrals over multinomial expansions

In support of continuum limit Eq. (46), we verify the consistency of large-$N$
expansion in a TotEmplex $[\mathfrak{f};{\,\hat{N}}]=\mathcal{P}$. For
simplicity, we take $\mathcal{E}_{0}=\emptyset$.

Following the strategy of Section 3.4, we first express probabilities on the
simplex via fluctuations around the $I$-projection which in absence of
phenomenological constraints coincides with
$\boldsymbol{\upsilon}\in\mathcal{P}$. Starting from expansion Eq. (44), we
assert that

$\displaystyle\sum_{N\mathfrak{p}\in\mathcal{P}}\text{mult}(N\mathfrak{p};\boldsymbol{\upsilon})=$
$\displaystyle(2\pi
N)^{-\frac{|\mathcal{E}|-1}{2}}\int\prod_{{\boldsymbol{e}}\in\mathcal{E}}\text{d}(N\mathfrak{p}_{\boldsymbol{e}})\exp\left\\{-N\sum_{{\boldsymbol{e}}\in\mathcal{E}}\mathfrak{p}_{\boldsymbol{e}}\log\frac{\mathfrak{p}_{\boldsymbol{e}}}{\boldsymbol{\upsilon}_{\boldsymbol{e}}}-\mbox{$\frac{1}{2}$}\sum_{{\boldsymbol{e}}\in\mathcal{E}}\log\mathfrak{p}_{\boldsymbol{e}}+\mathcal{O}\left(N^{-1}\right)\right\\}\delta(N\langle\mathfrak{p}\rangle-N)=$
$\displaystyle(2\pi
N)^{-\frac{|\mathcal{E}|-1}{2}}\int\prod_{{\boldsymbol{e}}\in\mathcal{E}}\text{d}(N\boldsymbol{\pi}_{\boldsymbol{e}})\exp\left\\{-N\sum_{{\boldsymbol{e}}\in\mathcal{E}}\frac{\boldsymbol{\pi}_{\boldsymbol{e}}^{2}}{2\boldsymbol{\upsilon}_{\boldsymbol{e}}}-\mbox{$\frac{1}{2}$}\sum_{{\boldsymbol{e}}\in\mathcal{E}}\log\mathfrak{p}_{\boldsymbol{e}}+\mathcal{O}\left(N^{-1/2}\right)\right\\}\int\frac{\text{d}k}{2\pi}\exp\left\\{iNk\sum_{{\boldsymbol{e}}\in\mathcal{E}}\boldsymbol{\pi}_{\boldsymbol{e}}\right\\}$
$\displaystyle=\sqrt{\frac{N}{2\pi}}\int\text{d}k\exp\left\\{-\frac{N}{2}k^{2}\right\\}=1\quad\Rightarrow\quad\checkmark$

For Dirac’s delta function in the first line, we have used integral
representation [74]. After decomposing into fluctuations in the kernel of
$\,\hat{N}$,

$\sum_{{\boldsymbol{e}}\in\mathcal{E}}\boldsymbol{\pi}_{\boldsymbol{e}}=0~{},$

Eq. (201) was used to resolve the sourced Gaussian integral. In the last line,
we have exploited that $\langle\boldsymbol{\upsilon}\rangle=1$ for a valid
reference distribution. Latter two equalities are understood up to
$1/N$-suppressed quantities. Imposing next $D-1$ phenomenological constraints,
we can analogously compute for each TotEmplex constructed by an irreducible
element $\mathbf{\,\hat{M}}$ a probability mass

$\displaystyle\mathfrak{P}=$
$\displaystyle\,\,\log\int_{\mathfrak{p}\in[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\text{mult}(N\mathfrak{p};\boldsymbol{\upsilon})$
(213) $\displaystyle=$ $\displaystyle\,\,-ND\hskip
0.28453pt(\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}\;\|\;\boldsymbol{\upsilon})-\frac{D-1}{2}\log
N-\mbox{$\frac{1}{2}$}\log\det\langle\mathbf{\,\hat{M}}\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}\mathbf{\,\hat{M}^{T}}\rangle+\mathcal{O}\left(N^{-1/2}\right)~{}.$

The $\mathcal{O}\left(1\right)$ corrections depend on the eigenvalue problem
Eq. (17).

Theorem 3.7 presents the $N$-leading estimate for the distribution over
distributions on the TotEmplex. The positive-definiteness of kernel Eq. (130)
has already been established. Our aim is then to explicitly verify that the
multinomial distribution restricted to the TotEmplex and normalized by the
total probability mass $\mathfrak{P}$ reduces to the multivariate Gaussian
distribution given in Eq. (78) to tractable order in the large-$N$ expansion.

Combining Lemma 3.8 substituted into expansion Eq. (44) together with Eq.
(213) we indeed have

$\displaystyle\frac{\text{mult}(N\mathfrak{p};\boldsymbol{\upsilon})}{\mathfrak{P}}=\sqrt{\frac{\det\left(2\pi
N\langle\mathbf{\,\hat{M}}\mathfrak{q}\mathbf{\,\hat{M}^{T}}\rangle\right)}{\det\left(2\pi
N\langle\text{\small 1}\text{1}\mathfrak{q}\text{\small
1}\text{1}\rangle\right)}}\exp\left\\{-\frac{N}{2}\mathcal{K}(\boldsymbol{\pi})\right\\}\quad\text{with}\quad\mathfrak{q}\equiv\mathbb{P}_{[\mathfrak{f};{\mathbf{\,\hat{M}}}]}\boldsymbol{\upsilon}~{}.$
(214)

Readily the factors of $(2\pi N)^{D/2-|\mathcal{E}|/2}=(2\pi
N)^{-|\ker\mathbf{\,\hat{M}}|/2}$ from the determinant ratio combined with
$N^{|\ker\mathbf{\,\hat{M}}|}$ from the integral measure (viz. Eq. (46))
produce the appropriate data-independent normalization for
$|\ker\mathbf{\,\hat{M}}|$-dimensional Gaussian kernel that scales linearly in
$N$. Hence, it only remains to show that the data-dependent ratio of
determinants properly reduces to the normalizing factor of a multivariate
Gaussian distribution with $\mathcal{O}\left(1\right)$ precision
$\langle\,\hat{N}_{a}\mathfrak{q}^{-1}\,\hat{N}_{b}\rangle$ from Eq. (130).

In fact, relation

$\displaystyle\det\left(\langle\mathbf{\,\hat{N}}\mathfrak{q}^{-1}\mathbf{\,\hat{N}^{T}}\rangle\right)=\frac{\det\left(\langle\mathbf{\,\hat{M}}\mathfrak{q}\mathbf{\,\hat{M}^{T}}\rangle\right)}{\det\langle\text{\small
1}\text{1}\mathfrak{q}\text{\small 1}\text{1}\rangle}~{},$ (215)

which holds along the lines of Eq. (211) between the determinants in the
appropriate subspaces, ensures the proper normalization of data-dependent part
in Eq. (214). Based on Eq. (28), we can arrange for

$\displaystyle\langle\,\hat{M}_{\alpha}\,\hat{M}_{\beta}\rangle=\delta_{\alpha\beta}\quad,\quad\langle\,\hat{N}_{a}\,\hat{N}_{b}\rangle=\delta_{ab}\quad\text{and}\quad\langle\,\hat{M}_{\alpha}\,\hat{N}_{a}\rangle=0~{},$
(216)

so that

$\displaystyle\mathbf{U}=\begin{pmatrix}\mathbf{\,\hat{M}}\\\
\mathbf{\,\hat{N}}\end{pmatrix}~{}.$ (217)

defines an $|\mathcal{E}|$-dimensional vector operator. In turn, this enables
us to introduce two block matrices

$\displaystyle\mathbf{A}=\langle\mathbf{U}\mathfrak{q}\mathbf{U}^{T}\rangle=\begin{pmatrix}\langle\mathbf{\,\hat{M}}\mathfrak{q}\mathbf{\,\hat{M}^{T}}\rangle&\langle\mathbf{\,\hat{M}}\mathfrak{q}\mathbf{\,\hat{N}^{T}}\rangle\\\
\langle\mathbf{\,\hat{N}}\mathfrak{q}\mathbf{\,\hat{M}^{T}}\rangle&\langle\mathbf{\,\hat{N}}\mathfrak{q}\mathbf{\,\hat{N}^{T}}\rangle\end{pmatrix}\quad\text{and}\quad\mathbf{B}=\langle\mathbf{U}\mathfrak{q}^{-1}\mathbf{U}^{T}\rangle~{}.$
(218)

Evidently, $\mathbf{B}$ is the inverse of $\mathbf{A}$ so that we can write

$\displaystyle\begin{pmatrix}\text{\small
1}\text{1}_{\text{rank}\,\mathbf{C}}&\mathbf{0}_{\text{rank}\,\mathbf{C}\times|\ker\mathbf{C}|}\\\
\mathbf{B}_{21}&\mathbf{B}_{22}\end{pmatrix}\begin{pmatrix}\mathbf{A}_{11}&\mathbf{A}_{12}\\\
\mathbf{A}_{21}&\mathbf{A}_{22}\end{pmatrix}=\begin{pmatrix}\mathbf{A}_{11}&\mathbf{A}_{12}\\\
\mathbf{0}_{|\ker\mathbf{C}|\times\text{rank}\,\mathbf{C}}&\text{\small
1}\text{1}_{|\ker\mathbf{C}|}\end{pmatrix}~{},$ (219)

where $\mathbf{A}_{\star\star}$ and $\mathbf{B}_{\star\star}$ compactly refer
to the corresponding block-matrices from Eq. (218). Taking the determinant on
both sides of latter equality,

$\displaystyle\det\mathbf{B}_{22}\det\mathbf{A}=\det\mathbf{A}_{11}~{},$ (220)

we arrive noting that $\det\mathbf{A}=\det\left(\text{\small
1}\text{1}\mathfrak{q}\text{\small 1}\text{1}\right)$ at Eq. (215).

## References

  * [1] E. Hüllermeier and W. Waegeman “Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods” Machine Learning 110 (2021) 457–506.
  * [2] E. L. Lehmann “Model specification: The views of fisher and neyman, and later developments” Statistical Science 5 (1990) 160–168.
  * [3] S. Greenland “Invited Commentary: The Need for Cognitive Science in Methodology” American Journal of Epidemiology 186 (08, 2017) 639–645.
  * [4] B. Dennis, J. M. Ponciano, M. L. Taper, and S. R. Lele “Errors in statistical inference under model misspecification: Evidence, hypothesis testing, and aic” Frontiers in Ecology and Evolution 7 (2019).
  * [5] D. Mayo Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars.  Cambridge University Press 2018.
  * [6] E. Ionides and Y. Ritov “The scientific method and p-values: response to mayo (2022)” Conservation Biology 36 (2022) no. 5, e13984.
  * [7] R. A. Fisher “On the mathematical foundations of theoretical statistics” Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character 222 (1922) 309–368.
  * [8] T. M. Cover and J. A. Thomas Elements of Information Theory 2nd Edition (Wiley Series in Telecommunications and Signal Processing).  Wiley-Interscience July, 2006.
  * [9] O. Loukas and H. R. Chung “Categorical distributions of maximum entropy under marginal constraints” arXiv preprint arXiv:2204.03406 (2022).
  * [10] O. Loukas and H. R. Chung “Entropy-based characterization of modeling constraints” arXiv preprint arXiv:2206.14105 (2022).
  * [11] Y. Kato, D. M. Tax, and M. Loog “A view on model misspecification in uncertainty quantification” arXiv preprint arXiv:2210.16938 (2022).
  * [12] H. O. A. Wold Theoretical Empiricism: A General Rationale for Scientific Model-Building.  New York: Paragon House 1987.
  * [13] N. Maxwell “Induction and scientific realism: Einstein versus van fraassen part three: Einstein, aim-oriented empiricism and the discovery of special and general relativity” The British Journal for the Philosophy of Science 44 (1993) no. 2, 275–305.
  * [14] K. Popper Conjectures and refutations: The growth of scientific knowledge.  routledge 2014.
  * [15] Y. M. Bishop, S. E. Fienberg, and P. W. Holland Discrete multivariate analysis: theory and practice.  Springer Science & Business Media 2007.
  * [16] J. Keynes A Treatise on Probability.  No. Bd. 4 in A Treatise on Probability. Macmillan 1921.
  * [17] C. E. Shannon “A mathematical theory of communication” The Bell System Technical Journal 27 (1948) no. 3, 379–423.
  * [18] S.-i. Amari and H. Nagaoka Methods of information geometry vol. 191.  American Mathematical Soc. 2000.
  * [19] S. Kullback and R. A. Leibler “On Information and Sufficiency” The Annals of Mathematical Statistics 22 (1951) no. 1, 79 – 86.
  * [20] S. Kullback Information theory and statistics.  Courier Corporation 1997.
  * [21] J. HAJEK “Limiting distributions in simple random sampling from a finite population” Publications of the Mathematical Institute of the Hungarian Academy of Sciences 5 (1960) 361–374.
  * [22] J. Hájek “Limiting distributions in simple random sampling from a finite population” Publications of the Mathematical Institute of the Hungarian Academy of Sciences 5 (1960) 361–374.
  * [23] R. Beran Bootstrap methods in statistics.  Sonderforschungsbereich 123, Stochast. Math. Modelle, Univ. Heidelberg 1983.
  * [24] D. N. Politis, J. P. Romano, and M. Wolf Subsampling.  Springer Science & Business Media 1999.
  * [25] A. Dembo and O. Zeitouni Large Deviations Techniques and Applications.  Jones and Bartlett books in mathematics. Jones and Bartlett 1993.
  * [26] J. Zinn-Justin Quantum Field Theory and Critical Phenomena.  International series of monographs on physics. Oxford University Press 2021.
  * [27] W. Feller An Introduction to Probability Theory and Its Applications, Volume 1.  An Introduction to Probability Theory and Its Applications. Wiley 1968\.
  * [28] G. Ziegler Lectures on Polytopes.  Graduate Texts in Mathematics. Springer New York 2012.
  * [29] A. Schrijver Combinatorial Optimization: Polyhedra and Efficiency.  No. Bd. 3 in Algorithms and combinatorics : study and research texts. Springer 2003.
  * [30] F. Nielsen “What is an information projection” Notices of the AMS 65 (2018) no. 3, 321–324.
  * [31] T. S. Jaakkola and M. I. Jordan “Bayesian parameter estimation via variational methods” Statistics and Computing 10 (2000) no. 1, 25–37.
  * [32] J. M. Keynes “Chapter iv: The principle of indifference” A treatise on probability 4 (1921) 41–64.
  * [33] E. T. Jaynes “Information theory and statistical mechanics” Physical review 106 (1957) no. 4, 620.
  * [34] I. Csiszár “Maxent, mathematics, and information theory” in Maximum entropy and Bayesian methods pp. 35–50.  Springer 1996.
  * [35] E. T. Jaynes Probability theory: The logic of science.  Cambridge university press 2003.
  * [36] J. N. Kapur Maximum-entropy models in science and engineering.  John Wiley & Sons 1989.
  * [37] J. Shore and R. Johnson “Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy” IEEE Transactions on information theory 26 (1980) no. 1, 26–37.
  * [38] J. B. Paris and A. Vencovská “A note on the inevitability of maximum entropy” International Journal of Approximate Reasoning 4 (1990) no. 3, 183–223.
  * [39] I. Csiszar “Why least squares and maximum entropy? an axiomatic approach to inference for linear inverse problems” The annals of statistics 19 (1991) no. 4, 2032–2066.
  * [40] O. Loukas and H.-R. Chung “Demographic parity: Mitigating biases in real-world data” 2023.
  * [41] H. C. Nguyen, R. Zecchina, and J. Berg “Inverse statistical problems: from the inverse ising problem to data science” Advances in Physics 66 (2017) no. 3, 197–261.
  * [42] O. Loukas and H.-R. Chung “Model selection in the world of maximum entropy” Physical Sciences Forum 5 (2022) no. 1,.
  * [43] S. V. Beentjes and A. Khamseh “Higher-order interactions in statistical physics and machine learning: A model-independent solution to the inverse problem at equilibrium” Phys. Rev. E 102 (Nov, 2020) 053314.
  * [44] T. Plefka “Convergence condition of the TAP equation for the infinite-ranged ising spin glass model” Journal of Physics A: Mathematical and General 15 (jun, 1982) 1971–1978.
  * [45] T. Tanaka “Mean-field theory of boltzmann machine learning” Physical Review E 58 (1998) no. 2, 2302.
  * [46] A. Montanari and T. Rizzo “How to compute loop corrections to the bethe approximation” Journal of Statistical Mechanics: Theory and Experiment 2005 (2005) no. 10, P10011.
  * [47] V. Sessak and R. Monasson “Small-correlation expansions for the inverse ising problem” Journal of Physics A: Mathematical and Theoretical 42 (2009) no. 5, 055001.
  * [48] S. Cocco and R. Monasson “Adaptive cluster expansion for the inverse ising problem: convergence, algorithm and tests” Journal of Statistical Physics 147 (2012) 252–314.
  * [49] J. S. Yedidia, W. T. Freeman, and Y. Weiss “Constructing free-energy approximations and generalized belief propagation algorithms” IEEE Transactions on information theory 51 (2005) no. 7, 2282–2312.
  * [50] R. Malouf “A comparison of algorithms for maximum entropy parameter estimation” in COLING-02: The 6th Conference on Natural Language Learning 2002 (CoNLL-2002).  2002\.
  * [51] S. Boyd and L. Vandenberghe Convex Optimization.  Cambridge University Press 2004.
  * [52] Y. Nesterov Introductory lectures on convex optimization: A basic course vol. 87.  Springer Science & Business Media 2003.
  * [53] I. Csiszár “I-divergence geometry of probability distributions and minimization problems” The annals of probability (1975) 146–158.
  * [54] C. T. Ireland and S. Kullback “Contingency tables with given marginals” Biometrika 55 (1968) no. 1, 179–188.
  * [55] J. Kruithof “Telefoonverkeersrekening” De Ingenieur 52 (1937) 15–25.
  * [56] J. N. Darroch and D. Ratcliff “Generalized iterative scaling for log-linear models” The annals of mathematical statistics (1972) 1470–1480.
  * [57] S. J. Haberman “Log-linear models for frequency tables derived by indirect observation: Maximum likelihood equations” The Annals of Statistics (1974) 911–924.
  * [58] L. Ruschendorf “Convergence of the iterative proportional fitting procedure” The Annals of Statistics 23 (1995) no. 4, 1160–1174.
  * [59] M. Zaloznik Iterative Proportional Fitting - Theoretical Synthesis and Practical Limitations.  PhD thesis 11, 2011.
  * [60] E. T. Jaynes “Prior probabilities” IEEE Transactions on systems science and cybernetics 4 (1968) no. 3, 227–241.
  * [61] E. T. Jaynes Concentration of Distributions at Entropy Maxima (1979) pp. 315–336.  Springer Netherlands Dordrecht 1989.
  * [62] A. van der Vaart Asymptotic Statistics.  Asymptotic Statistics. Cambridge University Press 2000.
  * [63] J. Friedman, T. Hastie, R. Tibshirani, et al. The elements of statistical learning vol. 1.  Springer series in statistics New York 2001.
  * [64] F. Morcos, A. Pagnani, B. Lunt, A. Bertolino, D. S. Marks, C. Sander, R. Zecchina, J. N. Onuchic, T. Hwa, and M. Weigt “Direct-coupling analysis of residue coevolution captures native contacts across many protein families” Proceedings of the National Academy of Sciences 108 (2011) no. 49, E1293–E1301.
  * [65] T. Cover and J. Thomas Elements of Information Theory.  Wiley 2012.
  * [66] J. Soch, T. B. of Statistical Proofs, T. J. Faulkenberry, K. Petrykowski, C. Allefeld, and C. D. McInerney StatProofBook/StatProofBook.github.io: StatProofBook 2021.  Zenodo Jan., 2022.
  * [67] I. Csiszár and P. C. Shields “Information theory and statistics: A tutorial”.
  * [68] E. Ising “Beitrag zur theorie des ferromagnetismus” Zeitschrift für Physik 31 (1925) no. 1, 253–258.
  * [69] A. Gelman, J. Carlin, H. Stern, D. Dunson, A. Vehtari, and D. Rubin Bayesian Data Analysis.  Chapman & Hall/CRC Texts in Statistical Science. CRC Press 2013.
  * [70] G. Schwarz “Estimating the Dimension of a Model” The Annals of Statistics 6 (1978) no. 2, 461 – 464.
  * [71] S. S. Wilks “The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses” The Annals of Mathematical Statistics 9 (1938) no. 1, 60 – 62.
  * [72] A. Buse “The likelihood ratio, wald, and lagrange multiplier tests: An expository note” The American Statistician 36 (1982) no. 3a, 153–157.
  * [73] A. W. v. d. Vaart Asymptotic Statistics.  Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press 1998.
  * [74] “NIST Digital Library of Mathematical Functions.” https://dlmf.nist.gov/, Release 1.1.10 of 2023-06-15.  F. W. J. Olver, A. B. Olde Daalhuis, D. W. Lozier, B. I. Schneider, R. F. Boisvert, C. W. Clark, B. R. Miller, B. V. Saunders, H. S. Cohl, and M. A. McClain, eds.