# Bayesian distances for quantifying tensions in cosmological inference and
the surprise statistic

Benedikt Schosser 1,⋆ 0009-0007-8905-7749 Pedro Riba Mello 2
0009-0005-0325-2400 Miguel Quartin 2,3,4 0000-0001-5853-6164 Björn Malte
Schäfer 1,♯ 0000-0002-9453-5772 1Zentrum für Astronomie der Universität
Heidelberg, Astronomisches Rechen-Institut, Philosophenweg 12, 69120
Heidelberg, Germany  2Instituto de Física, Universidade Federal do Rio de
Janeiro, 21941-972, Rio de Janeiro, RJ, Brazil  3Observatório do Valongo,
Universidade Federal do Rio de Janeiro, 20080-090, Rio de Janeiro, RJ, Brazil
4PPGCosmo, Universidade Federal do Espírito Santo, 29075-910, Vitória, ES,
Brazil

###### Abstract

Tensions between cosmological parameters derived through different channels
can be a genuine signature of new physics that $\Lambda$CDM as the standard
model is not able to reproduce, in particular in the missing consistency
between parameter estimates from measurements the early and late Universe. Or,
they could be caused by yet to be understood systematics in the measurements
as a more mundane explanation. Commonly, cosmological tensions are stated in
terms of mismatches of the posterior parameter distributions, often assuming
Gaussian statistics. More importantly, though, would be a quantification if
two data sets are consistent to each other before combining them into a joint
measurement, ideally isolating hints at individual data points that have a
strong influence in generating the tension. For this purpose, we start with
statistical divergences applied to posterior distributions following from
different data sets and develop the theory of a Fisher metric between two data
sets, in analogy to the Fisher metric for different parameter choices. As a
topical example, we consider the tension in the Hubble-Lemaître constant
$H_{0}$ from supernova and measurements of the cosmic microwave background,
derive a ranking of data points in order of their influence on the tension on
$H_{0}$. For this particular example, we compute Bayesian distance measures
and show that in the light of CMB data, supernovae are commonly too bright,
whereas the low-$\ell$ CMB spectrum is too high, in agreement with intuition
about the parameter sensitivity.

###### keywords:

Bayesian inference, tensions in cosmology, Fisher-formalism, supernova
distance-redshift relation, cosmic microwave background

sf

††thanks: ⋆<EMAIL_ADDRESS>♯
<EMAIL_ADDRESS>
##  1 introduction

The Bayes’ theorem as the central tool of statistical inference, describes the
process of updating the knowledge on the model parameters $\theta^{\mu}$ given
a data set $y^{i}$ within a preselected model (for summaries and applications
to cosmology, see Fisher, 1935; Caticha & Giffin, 2006; Trotta, 2017; Loredo,
2012),

$p(\theta|y)=\frac{\mathcal{L}(y|\theta)\pi(\theta)}{p(y)}\,,$ (1)

by setting the posterior distribution $p(\theta|y)$ into relation with the
likelihood $\mathcal{L}(y|\theta)$. The normalisation is given by the Bayes-
evidence $p(y)$

$p(y)=\int\mathrm{d}^{n}\theta\>\mathcal{L}(y|\theta)\pi(\theta)\,,$ (2)

where the prior distribution $\pi(\theta)$ encapsulates the knowledge on the
parameters $\theta^{\mu}$ before the data set $y^{i}$ has been recorded. The
likelihood $\mathcal{L}(y|\theta)$ as a conditional probability describes the
distribution of data points $y$ that can be expected under a particular
parameter choice $\theta$ and contains the knowledge of the physical model and
the error process in the measurement. As two parameter choices $\theta$ and
$\theta^{\prime}=\theta+\mathrm{d}\theta$ will differ in their prediction of
the data distribution it makes sense to quantify this difference in terms of
e.g. the Kullback-Leibler divergence $\Delta
S(\theta,\theta+\mathrm{d}\theta)$ (Kullback & Leibler, 1951; Rényi, 1960),

$S(\theta,\theta+\mathrm{d}\theta)=\int\mathrm{d}y\>\mathcal{L}(y|\theta)\ln\frac{\mathcal{L}(y|\theta)}{\mathcal{L}(y|\theta+\mathrm{d}\theta)}\simeq\frac{1}{2}F_{\mu\nu}(\theta)\mathrm{d}\theta^{\mu}\mathrm{d}\theta^{\nu}\,,$
(3)

which can be expanded with a quadratic, leading-order term, defining the
Fisher metric $F_{\mu\nu}(\theta)$ (Tegmark et al., 1997; Elsner & Wandelt,
2012; Coe, 2009; Bassett et al., 2011; Wolz et al., 2012; Schäfer & Reischke,
2016; Sellentin et al., 2014) as a positive definite distance measure on the
manifold of parameters (Amari, 2016): Effectively, it relates the question of
distinguishability of the data distributions originating from two different
parameter choices to a distance measure between two infinitesimally different
parameter choices, which in the Gaussian case corresponds exactly to the
$\Delta\chi^{2}$-functional.

Bayes’ theorem is perfectly symmetric, as it arises from decomposing the joint
distribution $p(y,\theta)$ either in terms of
$p(y,\theta)=\mathcal{L}(y|\theta)\pi(\theta)$ or equivalently, according to
$p(\theta,y)=p(\theta|y)p(y)$. Therefore, it might be sensible to ask whether
one can define a second Fisher metric for the posterior distribution
$p(\theta|y)$ in analogy to the Fisher metric for the likelihood
$\mathcal{L}(y|\theta)$, as both quantities are conditional probabilities. It
would be used to quantify if an infinitesimally different data set
$y^{i}+\mathrm{d}y^{i}$ would have given rise to a different posterior
distribution $p(\theta|y)$ according to (Amari, 2016)

$S(y,y+\mathrm{d}y)=\int\mathrm{d}\theta\>p(\theta|y)\ln\frac{p(\theta|y)}{p(\theta|y+\mathrm{d}y)}\simeq\frac{1}{2}F_{ij}(y)\mathrm{d}y^{i}\mathrm{d}y^{j},$
(4)

where $F_{ij}$ would likewise be a distance measure, but in this case in data
space. It would quantify how much a second data set would differ in its
induced posterior distribution. This point motivates our paper: We would like
to quantify the amount of dissimilarity between two data sets, as a measure of
tension. Consistency in the data sets would mean that they generate posteriors
that are compatible with each other and have a small, or vanishing in the case
of equal posteriors, Kullback-Leibler divergence $S(y,y+\mathrm{d}y)$.

The question of consistency (Raveri & Hu, 2019; Raveri et al., 2020; Charnock
et al., 2017; Lin & Ishak, 2017; Seehars et al., 2016) of parameter estimates
is commonplace in post-Planck cosmology. Different observational channels
suggest estimates for cosmological parameters that are in tension with each
other, most notably in the Hubble-Lemaître constant $H_{0}$ from the CMB and
low-redshift measurements like supernovae, or the determination of $S_{8}$
from the CMB in comparison to weak lensing (Joudaki et al., 2017). The
overarching physical questions in this context concern systematics, possibly
of astrophysical origin, and the consistency of $\Lambda$CDM as the standard
model of cosmology between the early and late Universe (Efstathiou & Lemos,
2017; Santos et al., 2017; Ange & Meyers, 2023; Douspis et al., 2019; Couchot
et al., 2017; MacCrann et al., 2015), and the emergence of new physics in
relation to e.g. dark energy or modified gravity (Scott, 2018; Babichev &
Deffayet, 2013; Bernal et al., 2016; Dossett et al., 2015; Renk et al., 2017;
Heavens et al., 2017). Given the deviation of the posterior parameter
distributions from their idealised Gaussian shape and the high dimensionality
of the parameter space, it is clear that Gaussian measures like the
Mahalanobis distance are not fully capturing all dissimilarities between the
distributions and that tools from information theory are needed (Nicola et
al., 2019; Grandis et al., 2016; Handley & Lemos, 2019; Lindley, 1956; Seehars
et al., 2014).

We review the concept of metric distances on statistical manifolds in Sect. 2
before introducing the idea of a Fisher-like metric for measuring distances
between data sets in Sect. 3. We apply this method to the well-known tension
between estimates of the Hubble-Lemaître constant $H_{0}$ from supernova and
cosmic microwave background data in Sect. 4 and summarise our main results in
Sect. 5. Throughout the paper, we work with a spatially flat $\Lambda$CDM
cosmology constrained by supernova data (Suzuki et al., 2012) and by Planck’s
measurements of the CMB (Planck Collaboration et al., 2018a), with best-fit
values from these two data sets. Notation-wise, we use Latin letters and
indices for data tuples, Greek letters with Greek indices for parameter
tuples, and impose the Einstein summation convention. Then, the parameter
covariance is given as
$\langle\theta^{\mu}\theta^{\nu}\rangle-\langle\theta^{\mu}\rangle\langle\theta^{\nu}\rangle$,
and for a Gaussian distribution equal to the inverse Fisher metric
$F^{\mu\nu}$. It fulfills $F^{\alpha\mu}F_{\alpha\nu}=\delta^{\mu}_{\nu}$ with
the Fisher metric $F_{\mu\nu}$. Derivatives
$\partial_{\mu}=\partial/\partial\theta^{\mu}$ with respect to the parameters
lead to linear forms,
$F_{\mu\nu}=\langle\partial_{\mu}\ln\mathcal{L}\>\partial_{\nu}\ln\mathcal{L}\rangle$,
such that a quantity like
$\Delta\chi^{2}=F_{\mu\nu}\Delta\theta^{\mu}\Delta\theta^{\nu}$ is an
invariant. We would like to emphasise that the $H_{0}$-tension is a topical
and relevant example for testing our methods (c.f. Verde et al., 2013), but we
do _not_ intend to provide physical mechanisms or identify a systematic to
resolve the tension.

##  2 From entropies and the Bayes’ theorem to the Fisher metric

###  2.1 Fisher metric and relative entropies

The inference process, as determined by Bayes’ theorem, suggests the
distribution $p(y)$ in data space $\mathcal{D}$ and the prior distribution
$\pi(\theta)$ in parameter space $\mathcal{M}$. These spaces, to take up the
foundational idea of information geometry, are actually manifolds. Mediating
between the two manifolds are the two conditional probabilities: The
likelihood $\mathcal{L}(y|\theta)$ maps from $\theta$ to $y$ as it provides a
distribution of data $y$ for each choice of the parameters $\theta$, and the
posterior distribution $p(\theta|y)$ as a map from $y$ to $\theta$, which
describes the uncertainty on the parameters $\theta$ if the data $y$ has been
taken into account, all illustrated by Fig. 1.

$T_{y}\mathcal{D}$$T_{\theta}\mathcal{M}$likelihood
$\mathcal{L}(y|\theta)$metric $F_{\mu\nu}$posterior $p(\theta|y)$metric
$F_{ij}$$\mathcal{D}$ with $y^{i}$, evidence $p(y)$$\mathcal{M}$ with
$\theta^{\mu}$, prior $\pi(\theta)$ Figure 1: Sketch of data manifold
$\mathcal{D}$ and the parameter manifold $\mathcal{M}$ with the corresponding
probabilities. The two conditional distributions $\mathcal{L}(y|\theta)$ and
$p(\theta|y)$ imply the existence of a metric $F_{\mu\nu}(\theta)$ on the
tangent space $T_{\theta}\mathcal{M}$ of parameters $\theta^{\mu}$ and of a
second metric $F_{ij}(y)$ on the tangent space $T_{y}\mathcal{D}$ of data
points $y^{i}$.

By definition, one can assign a metric to each conditional probability, and
conventionally this is done for the likelihood $\mathcal{L}(y|\theta)$ as
outlined by Eq. (3). In parallel, there is a second metric associated with the
posterior distribution $p(\theta|y)$ giving rise to the data space Fisher
metric $F_{ij}$, as summarised by Eq. (4). In both cases, one would proceed by
considering the difference in the distributions for an infinitesimal change in
the condition, $\mathrm{d}\theta$ or $\mathrm{d}y$, respectively. The
difference in distribution is measured by an $f$-divergence, most commonly by
the Kullback-Leibler divergence. For our particular application, this choice
becomes a necessity as only the Kullback-Leibler entropy measure is compatible
with Bayes’ law: For instance, Rényi- or Amari-divergences (Amari, 2016) would
provide a sensible construction of a metric in the spirit of Eq. (3), but they
neither allow an additive definition of conditional entropies nor an entropy-
version of Bayes’ law, $S(y|\theta)+S(\theta)=S(\theta|y)+S(y)$.

In fact, employing Bayes’ law (1) in conjunction with the relative entropy (4)
between the posteriors $p(\theta|y^{\prime})$ and $p(\theta|y)$ originating
from two data sets $y$ and $y^{\prime}$ under the assumption of identical
priors $\pi(\theta)$ leads to

$S(y,y^{\prime})=\int\mathrm{d}\theta\>p(\theta|y)\ln\left(\frac{p(\theta|y)}{p(\theta|y^{\prime})}\right)=\int\mathrm{d}\theta\>p(\theta|y)\ln\left(\frac{\mathcal{L}(y|\theta)}{\mathcal{L}(y^{\prime}|\theta)}\frac{p(y^{\prime})}{p(y)}\right)=\int\mathrm{d}\theta\>p(\theta|y)\ln
r-\ln B\,,$ (5)

because the posterior distribution is normalised,
$\int\mathrm{d}\theta\>p(\theta|y)=1$. In Eq. (5) one can identify the Bayes-
ratio $B=p(y)/p(y^{\prime})$ of evidences and the likelihood ratio
$r=\mathcal{L}(y|\theta)/\mathcal{L}(y^{\prime}|\theta)$. Typically, the
Bayes-ratio is used to compare different models, as the evidence differs for
different models, but equal data. Here, the model stays the same, but two
different data sets are considered, generating the deviation from unity in the
Bayes-ratio. The relative entropy $S(y,y^{\prime})$ can be interpreted as a
measure of compatibility of the posterior distributions derived from two
different data sets and consists of the two contributions: The likelihood
ratio $r$ averaged over posterior distribution and the Bayes-ratio $B$: The
two posterior distributions surely differ according of how probable the
observation of the respective data set was under the assumption of the
parameter choice, but the second contribution suggests the influence of the
posterior, too: It would matter to $S(y,y^{\prime})$ how probable the
observation of $y$ and $y^{\prime}$ could have been given the prior (Marshall
et al., 2006; Hobson et al., 2002; Karpenka et al., 2015; Adhikari & Huterer,
2019): In general, different data sets would generate different Bayesian
evidences, which by itself is a sensible (and sensitive) discriminant. In some
sense, Eq. (5) is remarkable as it combines the likelihood ratio $r$ as a
typical frequentist quantity with Bayes-ratio $B$ in a single formula in this
way.

It is possible to define a metric distance for every conditional probability,
as motivated in the introduction. Then, the Fisher metric $F_{ij}$ derived
from the posterior $p(\theta|y)$ for a data set $y$ and an infinitesimally
different data set $y+\mathrm{d}y$ would lead to

$S(y,y+\mathrm{d}y)=\int\mathrm{d}\theta\>p(\theta|y)\ln\left(\frac{p(\theta|y)}{p(\theta|y+\mathrm{d}y)}\right)=\frac{1}{2}F(y)_{ij}\mathrm{d}y^{i}\mathrm{d}y^{j}$
(6)

and would have the interpretation of data consistency between two
infinitesimally distant data sets with respect to implying similar posterior
distributions. For the particular choice of Kullback-Leibler divergences one
can derive the Fisher metric $F_{ij}$ from $S(y,y+\mathrm{d}y)$,

$F_{ij}(y)=\int\mathrm{d}\theta\>p(\theta|y)\>\partial_{i}\ln
p(\theta|y)\partial_{j}\ln p(\theta|y)\,,$ (7)

with derivatives $\partial_{i}=\partial/\partial y^{i}$. Then, finite
differences $\Delta y^{i}$ between two data sets $y$ and $y^{\prime}=y+\Delta
y$ defines a metric line element $s^{2}_{y}$

$S(y,y^{\prime})\simeq\frac{1}{2}F(y)_{ij}\Delta y^{i}\Delta
y^{j}=\frac{1}{2}s^{2}_{y}\,,$ (8)

under the assumption of a constant Fisher metric $F(y)_{ij}$. Typically, one
has Gaussian error processes for the data $y$ with $S(y,y+\Delta
y)=\Delta\chi^{2}/2$, such that $\Delta\chi^{2}=F_{ij}\Delta y^{i}\Delta
y^{j}=s^{2}_{y}$, agreeing with expectations. Due to the choice of natural
logarithms, both $S$ and $s_{y}^{2}$ are measured in nats.

Interestingly, the interpretation of the conventional Kullback-Leibler
divergence in parameter space between likelihoods $\mathcal{L}(y|\theta)$ and
$\mathcal{L}(y|\theta^{\prime})$ as the starting point of the derivation of
the Fisher metric $F_{\mu\nu}(\theta)$ is not as clear: Repeating the same
steps of inserting Bayes’ law into the Kullback-Leibler measure for the same
data set $y$, identical evidence $p(y)$ implies

$S(\theta,\theta^{\prime})=\int\mathrm{d}y\>\mathcal{L}(y|\theta)\ln\left(\frac{\mathcal{L}(y|\theta)}{\mathcal{L}(y|\theta^{\prime})}\right)=\int\mathrm{d}y\>\mathcal{L}(y|\theta)\ln\left(\frac{p(\theta|y)}{p(\theta^{\prime}|y)}\frac{\pi(\theta^{\prime})}{\pi(\theta)}\right)=\int\mathrm{d}y\>\mathcal{L}(y|\theta)\ln\left(\frac{p(\theta|y)}{p(\theta^{\prime}|y)}\right)+\ln\frac{\pi(\theta^{\prime})}{\pi(\theta)}\,,$
(9)

because the likelihood is normalised,
$\int\mathrm{d}y\>\mathcal{L}(y|\theta)=1$. The relation implies that the
relative entropy, which is to become a metric distance in the framework of
information geometry, has two contributions, one from the likelihood-weighted
logarithmic ratio between the posteriors at given $\theta$ and
$\theta^{\prime}$, as well as from the logarithmic prior ratio.

Much clearer, fortunately, is the interpretation of the conventional Fisher
metric $F_{\mu\nu}$ derived from the likelihood $\mathcal{L}(y|\theta)$ for a
parameter choice $\theta$ and an infinitesimally distant parameter choice
$\theta+\mathrm{d}\theta$, for fixed data $y$

$S(\theta,\theta+\mathrm{d}\theta)=\int\mathrm{d}y\>\mathcal{L}(y|\theta)\ln\left(\frac{\mathcal{L}(y|\theta)}{\mathcal{L}(y|\theta+\mathrm{d}\theta)}\right)=\frac{1}{2}F(\theta)_{\mu\nu}\mathrm{d}\theta^{\mu}\mathrm{d}\theta^{\nu}$
(10)

as the difference in goodness of fit, which becomes a line element
$\mathrm{d}\chi^{2}$ in parameter space. With derivatives
$\partial_{\mu}=\partial/\partial\theta^{\mu}$ one arrives at (Amari, 2016)

$F_{\mu\nu}(\theta)=\int\mathrm{d}y\>\mathcal{L}(y|\theta)\>\partial_{\mu}\ln\mathcal{L}(y|\theta)\partial_{\nu}\ln\mathcal{L}(y|\theta)\,,$
(11)

such that two model choices $\theta$ and $\theta^{\prime}=\theta+\Delta\theta$
with finite differences have a metric distance

$S(\theta,\theta^{\prime})\simeq\frac{1}{2}F_{\mu\nu}(\theta)\Delta\theta^{\mu}\Delta\theta^{\nu}=\frac{1}{2}s^{2}_{\theta}$
(12)

approximating the metric distance $s^{2}_{\theta}$ by assuming a constant
$F_{\mu\nu}(\theta)$: Otherwise, the computation of an arc length would be
required in a line integral along a parameterised curve
$\theta^{\mu}(\lambda)$ (Giesel et al., 2020).

###  2.2 Bayes’ theorem from the point of view of information entropy

As the Fisher-matrices $F_{ij}$ and $F_{\mu\nu}$ are derived from the
posterior distribution $p(\theta|y)$ and the likelihood
$\mathcal{L}(y|\theta)$, respectively, there should be a deeper relation
between these relative entropies and the Bayes’ theorem (Baez & Fritz, 2014;
Shore & Johnson, 1980): Starting with the mutual entropy for the joint
probability $p(\theta,y)=p(\theta|y)p(y)=\mathcal{L}(y|\theta)\pi(\theta)$,
and with $\pi(\theta)$ and $p(y)$ are the marginals of the joint distribution
$p(\theta,y)$, one defines the conditionals:
$\mathcal{L}(y|\theta)=p(y,\theta)/\pi(\theta)$ and
$p(\theta|y)=p(y,\theta)/p(y)$. Then, the mutual entropy $S(\theta,y)$ is
given by

$S(\theta,y)=\int\mathrm{d}\theta\>\int\mathrm{d}y\>p(\theta,y)\ln\frac{p(\theta,y)}{\pi(\theta)p(y)}=\int\mathrm{d}y\>p(y)\>\int\mathrm{d}\theta\>p(\theta|y)\ln\frac{p(\theta|y)}{\pi(\theta)}\,,$
(13)

and would vanish if $p(\theta,y)$ factorizes into $\pi(\theta)p(y)$, which can
only be the case if the data contains no information about the physical model.
In general, though, the evidence-weighted Kullback-Leibler divergence between
posterior and prior distributions corresponds to the surprise statistic (Kunz
et al., 2006).

Vice versa, one obtains

$S(y,\theta)=\int\mathrm{d}\theta\>\int\mathrm{d}y\>p(\theta,y)\ln\frac{p(\theta,y)}{\pi(\theta)p(y)}=\int\mathrm{d}\theta\>\pi(\theta)\>\int\mathrm{d}y\>\mathcal{L}(y|\theta)\ln\frac{\mathcal{L}(y|\theta)}{p(y)}\,,$
(14)

implying that the surprise statistic is given by the prior-averaged
information entropy of the likelihood minus the information entropy of the
posterior distribution:

$\left\langle S(\pi(\theta)\rightarrow
p(\theta|y))\right\rangle_{p(y)}=\left\langle
S(p(y)\rightarrow\mathcal{L}(y|\theta)\right\rangle_{\pi(\theta)}\,,$ (15)

pointing at an interpretation of the surprise statistic as the relative
entropy between likelihood and evidence, averaged over the prior allowed
parameter range.

##  3 Fisher metric in data space

The Fisher metric $F_{ij}(y)$ in data space quantifies the difference between
the posterior distributions derived from two infinitesimally different data
sets. In an application to a cosmological inference problem, for instance, the
tension between the values for the Hubble-Lemaître constant $H_{0}$ from
supernovae and CMB-measurements, one would proceed like this: The first data
set implies a posterior distribution from which one can construct a
hypothetical distribution of data sets of the second measurement. Then, one
can compare this hypothetical second measurement to the actual second
measurement and quantify the amount of compatibility with a Kullback-Leibler-
type measure, or in approximation, with a distance induced by the Fisher
metric $F_{ij}(y)$. It is important to realise that in this way $(i)$ one
compares the compatibility of two data sets, allowing $(ii)$ a ranking of data
points in their influence on the posterior distribution. And $(iii)$, it would
be the case as for all relative entropies (with the exception of the
Bhattacharyya-entropy (Bhattacharyya, 1943)) that the distance measure is not
symmetric: In Eq. (7) it matters which data set is used as the basis for the
posterior distribution $p(\theta|y)$.

The Fisher metric can be rewritten by using

$\langle\partial_{i}\ln p(\theta|y)\partial_{j}\ln
p(\theta|y)\rangle=-\langle\partial_{i}\partial_{j}\ln p(\theta|y)\rangle\,,$
(16)

with the expectation value $\langle\,.\,\rangle$ computed by integration over
the random variable $\theta$. Then, Eq. (7) becomes

$F_{ij}=-\int\mathrm{d}\theta\>p(\theta|y)\>\partial_{i}\partial_{j}\ln
p(\theta|y)\,.$ (17)

To calculate the partial derivatives we can use Bayes’ theorem Eq. (1) and
substitute the posterior, yielding:

$\partial_{i}\partial_{j}\ln
p(\theta|y)=\partial_{i}\partial_{j}\ln\left(\frac{\mathcal{L}(y|\theta)\pi(\theta)}{p(y)}\right)=\partial_{i}\partial_{j}\left(\ln\mathcal{L}(y|\theta)-\ln
p(y)\right)\,,$ (18)

where the derivative of the prior vanishes, as it does not depend on the data.

To continue with our calculation we need to assume the explicit shape of the
likelihood. Here, we will assume, that we have a Gaussian likelihood with
covariance matrix $\Sigma_{ij}$, data points $y^{i}$ and a model prediction
$y(\theta)$, resulting in the following shape:

$\mathcal{L}\propto\exp\left\\{-\frac{1}{2}(y-y(\theta))^{i}\Sigma_{ij}(y-y(\theta))^{j}\right\\}$
(19)

Note, that the likelihood is only Gaussian in the data and not in the
parameters, as $y(\theta)$ can be any function. Now, the derivatives of the
log-likelihood follow as

$\partial_{i}\partial_{j}\ln\mathcal{L}(y|\theta)=-\partial_{i}\partial_{j}\frac{1}{2}\sum_{k}\left((y-y(\theta))^{l}\Sigma_{lk}(y-y(\theta))^{k}\right)=-\partial_{i}\left(\Sigma_{jk}(y-y(\theta))^{k}\right)=-\Sigma_{ij}\,,$
(20)

with $\partial_{i}\Sigma_{jk}=0$, due to the small impact of varying data on
the covariance matrix (Schäfer & Reischke, 2016; Reischke et al., 2017). The
derivative of the evidence is more involved. We will start by explicitly
performing the derivative and substituting the formula for the evidence Eq.
(2):

$\displaystyle-\partial_{i}\partial_{j}\ln p(y)$
$\displaystyle=\frac{1}{p(y)^{2}}\partial_{i}p(y)\partial_{j}p(y)-\frac{1}{p(y)}\partial_{i}\partial_{j}p(y)$
(21)
$\displaystyle=\frac{1}{p(y)^{2}}\int\pi(\theta)\>\partial_{i}\mathcal{L}(y|\theta)\mathrm{d}\theta\,\int\pi(\theta)\>\partial_{j}\mathcal{L}(y|\theta)\mathrm{d}\theta-\frac{1}{p(y)}\int\pi(\theta)\>\partial_{i}\partial_{j}\mathcal{L}(y|\theta)\mathrm{d}\theta\,,$
(22)

where we used the interchangeability of integral and derivative and that the
prior does not depend on the data. Now, the first and second partial
derivatives of the likelihood are

$\partial_{i}\partial_{j}\mathcal{L}(y|\theta)=-\partial_{i}\left(\mathcal{L}(y|\theta)\Sigma_{jk}(y-y(\theta))^{k}\right)=\mathcal{L}(y|\theta)\Sigma_{il}(y-y(\theta))^{l}\Sigma_{jk}(y-y(\theta))^{k}-\mathcal{L}(y|\theta)\Sigma_{ij}\,.$
(23)

Putting it all together the derivatives of the log posterior read as

$\displaystyle\partial_{i}\partial_{j}\ln
p(\theta|y)=-\Sigma_{ij}+\int\frac{\mathcal{L}(y|\theta)\pi(\theta)}{p(y)}\Sigma_{il}(y-y(\theta))^{l}\mathrm{d}\theta\,\int\frac{\mathcal{L}(y|\theta)\pi(\theta)}{p(y)}\Sigma_{jk}(y-y(\theta))^{k}\mathrm{d}\theta$
$\displaystyle-\int\frac{\mathcal{L}(y|\theta)\pi(\theta)}{p(y)}\Sigma_{il}(y-y(\theta))^{l}\Sigma_{jk}(y-y(\theta))^{k}\mathrm{d}\theta+\int\frac{\mathcal{L}(y|\theta)\pi(\theta)}{p(y)}\Sigma_{ij}\mathrm{d}\theta\,,$
(24)

where we can identify Bayes’ theorem multiple times and simplify it to

$\displaystyle\partial_{i}\partial_{j}\ln
p(\theta|y)=\Sigma_{il}\Sigma_{jk}\left(\int
p(\theta|y)(y-y(\theta))^{l}\mathrm{d}\theta\,\int
p(\theta|y)(y-y(\theta))^{k}\mathrm{d}\theta-\int
p(\theta|y)(y-y(\theta))^{l}(y-y(\theta))^{k}\mathrm{d}\theta\right)\,.$ (25)

Here, the terms only with $\Sigma_{ij}$ cancel each other and the rest are
expectation values with respect to the posterior:

$\displaystyle\partial_{i}\partial_{j}\ln p(\theta|y)$
$\displaystyle=\Sigma_{il}\Sigma_{jk}\left(\langle(y-y(\theta))^{l}\rangle_{p(\theta|y)}\langle(y-y(\theta))^{k}\rangle_{p(\theta|y)}-\langle(y-y(\theta))^{l}(y-y(\theta))^{k}\rangle_{p(\theta|y)}\right)$
$\displaystyle=\Sigma_{il}\Sigma_{jk}\left(\langle
y^{l}(\theta)\rangle_{p(\theta|y)}\langle
y^{k}(\theta)\rangle_{p(\theta|y)}-\langle
y^{l}(\theta)y^{k}(\theta)\rangle_{p(\theta|y)}\right)\,,$ (26)

where the last simplification was obtained by collecting terms. It is
important to note, that Eq. (26) does not depend on the model parameters
$\theta$ as they are integrated out by taking the expectation value.
Therefore, the Fisher metric is given by

$F_{ij}=-\Sigma_{il}\Sigma_{jk}\left(\langle
y^{l}(\theta)\rangle_{p(\theta|y)}\langle
y^{k}(\theta)\rangle_{p(\theta|y)}-\langle
y^{l}(\theta)y^{k}(\theta)\rangle_{p(\theta|y)}\right)\,,$ (27)

which is a product of the inverse covariance matrices of the data points times
the covariance matrix of the model predictions:

$F_{ij}=\Sigma_{il}\Sigma_{jk}\mathrm{cov}_{p(\theta|y)}[y^{l}(\theta),y^{k}(\theta)]$
(28)

##  4 Application to the cosmological tension between supernova and CMB data

The tension in the Hubble-Lemaître constant $H_{0}$ arising from the two
measurements, supernovae type Ia (SNIa) and the cosmic microwave background
(CMB), is typically measured in $\sigma$. The formulation of the Fisher metric
in data space allows us to describe the distance between datasets in a natural
way. The goal of this section is to obtain a metric on the data space of the
SNIa measurements as well as the data space of the CMB spectra and obtain a
distance between SNIa data and CMB data.

###  4.1 Likelihoods

For this analysis, we need likelihoods for the supernovae data and the CMB
data, since samples from the posterior are necessary to calculate the data
space fisher metric defined in Eq. (28). This section briefly summarises the
likelihood and data for the SN1a and the CMB.

####  4.1.1 Supernova Likelihood

The SNIa dataset is from SCP Union2.1 with 518 individual measurements (Suzuki
et al., 2012). We choose this dataset in preference over newer measurements
like Pantheon+ due to the more straightforward implementation. While this
comes with the downside of fewer data points, the main objective of this work
is to show the idea and explore state-of-the-art data in future projects. Each
data point consists of a redshift $z$ and the distance modulus $\mu$. This
allows to derive the cosmological parameters $\Omega_{\text{m}},h$ in the
following way: The distance modulus is defined through the total and apparent
magnitude as

$\displaystyle\mu=m-M=5\log_{10}\frac{d_{L}}{10\text{pc}}\,,$ (29)

where $d_{L}$ is the luminosity distance. In this analysis, the unit of
$d_{L}$ is kpc, which allows to rewrite Eq. (29) as

$\displaystyle\mu=5\log_{10}d_{L}+10\,.$ (30)

The luminosity distance is defined via the differential equation

$\displaystyle\frac{\mathrm{d}d_{L}}{\mathrm{d}z}=\frac{d_{L}}{1+z}+c\frac{1+z}{H(z)}\,,$
(31)

with

$\displaystyle
H^{2}(z)=H_{0}^{2}\left(\Omega_{\text{m}}(1+z)^{3}+\Omega_{w}(1+z)^{3(1+w)}\right)\,,$
(32)

where $H_{0}=h\times 100\,\text{(km/s)/Mpc}$ and $c$ is the speed of light.
For simplicity of the analysis, we constrain ourselves to a spatially flat
$\Lambda$CDM-universe, which means setting $w=-1$. $\Omega_{w}$ can simply be
calculated with $\Omega_{w}=1-\Omega_{\text{m}}$. Therefore, given the
parameter set $\theta=\\{\Omega_{\text{m}},h\\}$ one can calculate the
distance modulus as a function of redshift $z$, denoted by $\mu(z|\theta)$.
The likelihood is a simple Gaussian one given by

$\displaystyle\mathcal{L}(\mu|\theta)\propto\exp\left\\{-\frac{1}{2}(\mu-\mu(z|\theta))^{i}\Sigma_{ij}(\mu-\mu(z|\theta))^{j}\right\\}\,,$
(33)

where $\mu^{i}$ and $\Sigma_{ij}$ are the measurements and covariance matrix
for the distance modulus and $z$ is the redshift of the respective
measurement. The likelihood has the same general form as is required in Eq.
(19).

####  4.1.2 CMB Likelihood

For the analysis of the CMB data, we use the Planck-lite high-$\ell$
likelihood, presented in (Planck Collaboration et al., 2018b). Compared to the
full Plik likelihood, all except one nuisance parameter are integrated out.
This saves us a lot of computational cost at a marginal loss in precision,
which is not relevant to this conceptual work. We consider base
$\Lambda\text{CDM}$ parametrized by the parameter set
$\\{\Omega_{m},\,h,\,\omega_{b},\,\ln
10^{10}A_{s},\,n_{s},\,\tau_{\text{reio}}\\}$. Over all parameters, except
$\Omega_{m}$ and $h$ is marginalized to obtain a comparable posterior.

The $C_{\ell}$ are averaged into 215 bins, where $\ell_{\text{min}}=30$ and
$\ell_{\text{max}}=2508$. The spectra are calculated with the Cosmic Linear
Anisotropy Solving System (CLASS) (Blas et al., 2011). The likelihood is given
by

$\mathcal{L}(C_{\ell}|\theta)\propto\exp\left\\{-\frac{1}{2}(C_{\ell}-C_{\ell}(\theta))^{i}\Sigma_{ij}(C_{\ell}-C_{\ell}(\theta))^{j}\right\\}\,,$
(34)

with the inverse covariance matrix $\Sigma_{ij}$, the data $C_{\ell}$ and the
spectra calculated from given parameters $C_{\ell}(\theta)$. Note, that we
have the same functional shape as in Eq. (33). The posterior samples are
obtained with an MCMC analysis using MontePython (Brinckmann & Lesgourgues,
2018).

Figure 2: True data from the SCP Union2.1 supernova dataset (orange) and the
Planck CMB power spectrum (blue). Translating the inferred best-fit parameters
from one experiment into hypothetical expected data from the other experiment
results in the grey-shaded region. Figure 3: Inferred 2d and 1d marginalized
posteriors from supernova and CMB data, shown in Fig. 2. The four posteriors
emerge from considering real and hypothetical data, the grey regions in Fig.
2. The dotted posteriors are from hypothetical data. Note the coinciding
maxima of the posteriors when the data has the superscript CMB or SN,
respectively, and the same width if the data comes from the same data space,
$\mu$ or $c$.

###  4.2 Translation between data spaces

The measurements obtained from Cosmic Microwave Background (CMB) and Type Ia
Supernovae (SNIa) experiments are characterized by different data spaces. In
the context of CMB, the spectral information is represented by $C_{\ell}$,
while SNIa measurements are expressed through the distance modulus $\mu$. The
necessity to compare and combine these datasets, as required by Eq. (28),
mandates the translation of data into a unified data space. Given the absence
of a preferred data space, both CMB and SNIa data spaces are examined.

To facilitate the translation between these spaces, two models are introduced:
$m^{\text{SN}}$ and $m^{\text{CMB}}$. These models enable the computation of
data for both spaces using the same underlying parameters. Explicitly, they
are both $\Lambda\text{CDM}$, but for different times of the universe.
Consequently, starting with a real dataset $c^{\text{CMB}}$, one can deduce
the best-fit parameters $\theta^{\text{CMB}}$ and generate hypothetical data
in the SN space through $m^{\text{SN}}(\theta^{\text{CMB}})=\mu^{\text{CMB}}$.
The concept of hypothetical data corresponding to different parameter values
is illustrated in Fig. 2. By using this dataset $\mu^{\text{CMB}}$ to infer
best-fit parameters, one would recover $\theta^{\text{CMB}}$.

Since the uncertainties of data points are influenced by the statistics and
systematics of the original experiment, it is assumed that $\mu^{\text{CMB}}$
and the real measurement $\mu^{\text{SN}}$ share the same covariance matrix.
This results in distinct posteriors for the two data spaces, specifically
$p(\theta|\mu^{\text{CMB}})\neq p(\theta|c^{\text{CMB}})$. The visual
representation of these four posteriors, two based on real data and two on
hypothetical data, is illustrated in Fig. 3. The tension between the real data
becomes evident, as the contours for $p(\theta|c^{\text{CMB}})$ and
$p(\theta|\mu^{\text{SN}})$ barely overlap. The posteriors derived from
hypothetical data successfully recover the means, but exhibit different widths
in their distributions, highlighting the respective experiment’s ability to
constrain the chosen parameter set.

###  4.3 Distances

The previous Sect. 4.2 showed how we can compare data from two different data
spaces. Now, we can compare SNIa and CMB data in the two possible spaces,
where we expect differences in compatibility. The measure of compatibility is
the distance between the two datasets, calculated with Eq. (6), i.e.
$s(\mu^{\text{SN}},\mu^{\text{CMB}})$ and $s(c^{\text{SN}},c^{\text{CMB}})$,
where the previously used $y$ is now $\mu$ or $c$. For the rest of this work,
$y\in\\{\mu,c\\}$.

Intrinsically, $s_{y}$ does not have an error, however, the data does. We
expect the data to be on the best-fit cosmology line, yet data is distributed
according to a Gaussian distribution. Therefore, it is sensible to resample
the hypothetical data from this distribution and recalculate $s_{y}$ to
estimate the uncertainty. We observe, that the the Fisher metric is not
influenced by the resampling. This is to be expected because the posterior is
unaffected and $F$ only depends on its shape. The effect of the resampling is
solely in $\Delta y=y^{\text{SN}}-y^{\text{CMB}}$.

The resulting distribution for $s_{y}$ is shown in Fig. 4. The values read
$s^{\text{SN}}_{\mu}=3.3\pm 1.0$ and $s^{\text{CMB}}_{c}=20.1\pm 2.9$. Note,
that we show an actual metric distance, which is related to the KL-divergence
with $S=s_{y}^{2}/2$, whose value is to be interpreted in nats. The values are
significantly different in the two data spaces, which falls back to the
constraining power of the different datasets. CMB data sets tighter limits on
the chosen parameters, therefore, a hypothetical CMB dataset corresponding to
the supernova best-fit cosmology is further away from the real CMB data than
the same scenario in the supernova data space. We can see tension in both
results, but it is more pronounced in the CMB data space.

Figure 4: Distribution of the data distance (8) between the CMB and supernova
dataset. The left panel (orange) shows the distance in SN space for many
realizations of hypothetical data, corresponding to CMB best-fit parameters.
The right panel (blue) displays the corresponding scenario in the CMB space
with realizations of hypothetical data calculated from SN best-fit parameters.

Having calculated the Fisher metric allows us to further investigate the
influence of individual data points on the tension. We can investigate which
data point has the largest impact on the distance, by calculating the
logarithmic derivative

$\displaystyle\frac{\partial s_{y}}{\partial\ln
y^{i}}=\frac{\sum_{j}F_{ij}y^{i}\Delta y^{j}}{s_{y}}\,,$ (35)

where the Einsum summation convention is not applied and we explicitely write
the sum over $j$. This derivative combines the error of the data point with
its actual value in order to quantify the leverage of $y^{i}$ for a fixed
index $i$ on the posterior distribution in a dimensionless way.

The idea of this object is to quantify the possible impact of one data point
on the distance between two datasets. Therefore, characterising which data
points are most important for a precise measurement. The uncertainty on this
object is calculated by resampling the possible realizations of the
hypothetical data. The results are shown in Fig. 5, where the error bars
correspond to $1\sigma$-intervals.

The first panel relates the derivate of the distance with the redshift of the
data point. Almost all points show a negative derivative, implying that the
distance between the two datasets reduces if the measured supernova would be
further away, which is consistent with what one would expect intuitively.
There is no clear visible trend that would point towards which redshifts are
most important. There are some outliers with the largest impact on the
distance, allowing us to label them as the most important data. The second
panel relates the derivate with the uncertainty of the measurement. Here, a
small uncertainty corresponds to a large impact on the distance and vice
versa.

The third and fourth panels show the same as the first and second, but now for
the CMB space. Low-$\ell$ multipoles have a much larger pull on the distance
than high-$\ell$, even though their uncertainty is larger. This can be
observed in the fourth panel, where we see the inverse relation, compared to
the SN scenario. The uncertainty of the low-$\ell$ region is dominated by
cosmic variance and a more precise measurement is not feasible. Therefore,
more accuracy up to $\ell\approx 500$ shows the largest potential, either in
theory or in systematics.

Figure 5: Importance of each data point quantified by the log derivative of
distance. The left side (orange) is in the SN space and the right (blue) is in
the CMB space. The first panel shows which redshifts are most important. The
third panel shows the same concept but for multipole moments in the CMB space.
The second and fourth panels show how the uncertainty of the individual
measurement influences the capability to reduce the distance in the SN and CMB
space, respectively.

Coming back to the tension between the two datasets, we can test which data
point has the largest influence on the incompatibility if it were to be
measured incorrectly. Here, we assume that all of them are independent: While
this is technically not the case due to correlations in calibration on
couplings between binned multipoles, these correlations are weak and could
nevertheless be incorporated into the analysis. The procedure is to propose a
hypothetical dataset, where the $i$th point of one dataset is altered, such
that it is equal to the $i$th point in the other set, i.e.
$y^{i,\text{SN}}=y^{i,\text{CMB}}$, where the $i$ is fixed for each
recalculation. We change the value of the true datasets and compare it to the
hypothetical ones. We want to compare this idea of quantifying the influence
of a wrong measurement with the number of sigmas between two data points
$N_{\sigma}$. Here, $N_{\sigma}$ is allowed to be negative to indicate whether
a data point is too high or too low, as shown by Fig. 6. There is a trend of
proportionality, even more pronounced in the CMB space. In the SN space,
multiple points that differ up to $2\sigma$, but do not shift the distance.
Rather, the figure suggests that all low data points contribute evenly to the
shift.

Figure 6: Relation between $N_{\sigma}$, the number of standard deviations,
one hypothetical and real data point differ, and the shift in distance induced
by changing one of the true data points to the hypothetical one. The left
panel (orange) shows the SN space and the right (blue) the CMB space.

##  5 summary and discussion

In our study we apply methods from information geometry to the issue of
cosmological tensions, in particular, the tension between estimates of the
Hubble-Lemaître constant $H_{0}$ on the basis of supernova and CMB data, which
is highly significant in Gaussian terms, and widely discussed in the
literature as a signature of new physics beyond $\Lambda$CDM.

  1. (i)

In terms of information geometry, tensions or dissimilarities between
conditional distributions are quantified by axiomatically defined statistical
divergences and suggest a metric distance in the manifold of their parameters.
We use the symmetry of Bayes’ law for establishing Kullback-Leibler
divergences for the posterior distributions in their dependence on data, which
becomes the basis of an information geometrical distance measure. In
particular, this relative information entropy seems to be composed of the
averaged likelihood ratio and the Bayesian evidence ratio as an intuitive
explanation. As entropies are measured in units of nats, the distance measure
on the data manifold would correspondingly be quantified in
$\sqrt{\mathrm{nats}}$. In contrast to intuitive measures like the Mahalanobis
distance, the information theoretical distance does not rely on Gaussianity of
the distributions.

  2. (ii)

There seems to be a relation between the surprise statistic as the Kullback-
Leibler divergence between the prior and the posterior distribution. Starting
from their mutual information and by employing Bayes’ law one can show that
the surprise statistic corresponds to relative entropy between the likelihood
and the Bayes evidence as distributions of the expected data.

  3. (iii)

We present an explicit derivation of the Fisher metric $F_{ij}$ in data for a
Gaussian error process in the measurement. The result is applied to the
tension in estimates for the Hubble-Lemaître constant $H_{0}$ from CMB and
supernova measurements. For this purpose, we ask what supernova distance
moduli one would have expected in the light of the CMB posterior, and what CMB
spectra one would have observed if the supernova posterior had been true, with
a propagation of the uncertainty contained in the prior of the first data set
onto an error band for the second data set, with a subsequent estimation of
the information theoretical distance measure. As expected from Gaussian
arguments, there is an asymmetric tension between the two data sets, now
quantified in terms of a metric distance. The origin of this tension is
generated by supernovae in the entire redshift range if the CMB data is
considered to be correct. But vice versa, it is predominantly the low-$\ell$
CMB spectrum that is responsible for the tension if the supernova posterior is
valid. We would like to emphasise, though, that statistical divergences like
Kullback-Leibler do not differentiate between the relative centroid shift of
two distributions and their differing width.

  4. (iv)

Using the data space Fisher metric as a diagnostic tool, it is possible to
disentangle the two contributions to the power of a data point to influence
the posterior distribution: Clearly, an outlying data point can influence the
fit according to its distance from the fit suggested by the other data points,
but only to the extend how strongly the model varies at that particular point.
Our study seems to suggest that in the case of supernova data there, almost
all data points contribute to the tension irrespective of their significance
and that in the case of the CMB data, the significant deviations in the
low-$\ell$ data points predominantly generate the tension.

It would be interesting to see if Eq. (15) could provide means by detecting
anomalies in data, developing the idea of this paper further: In an abstract
way, one could quantify the surprise statistic in data space for individual
data points and quantify outliers using this methodology.

## Acknowledgements

We would like to thank Alan Heavens for an insightful conversation.

##### Funding information

BS and BMS would like to acknowledge financial support by the Vector Stiftung.
We acknowledge the usage of the AI-clusters Tom and Jerry funded by the Field
of Focus 2 of Heidelberg University. MQ is supported by the Brazilian research
agencies FAPERJ, CNPq (Conselho Nacional de Desenvolvimento Científico e
Tecnológico) and CAPES. This study was financed in part by the Coordenação de
Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code
001. We acknowledge support from the CAPES-DAAD bilateral project “Data
Analysis and Model Testing in the Era of Precision Cosmology”.

##### data availability statement

No new data were generated or analysed in support of this research.

## References

  * Adhikari & Huterer (2019) Adhikari S., Huterer D., 2019, J. Cosmology Astropart. Phys, 2019, 036
  * Amari (2016) Amari S.-i., 2016, Information Geometry and Its Applications. Springer Japan
  * Ange & Meyers (2023) Ange J., Meyers J., 2023, Improving Constraints on Models Addressing the Hubble Tension with CMB Delensing (arXiv:2307.01662 [astro-ph]), doi:10.48550/arXiv.2307.01662, http://arxiv.org/abs/2307.01662
  * Babichev & Deffayet (2013) Babichev E., Deffayet C., 2013, CQG, 30, 184001
  * Baez & Fritz (2014) Baez J. C., Fritz T., 2014, Theory and Applications of Categories, 29, 421
  * Bassett et al. (2011) Bassett B. A., Fantaye Y., Hlozek R., Kotze J., 2011, International Journal of Modern Physics D, 20, 2559
  * Bernal et al. (2016) Bernal J. L., Verde L., Cuesta A. J., 2016, JCAP, 02
  * Bhattacharyya (1943) Bhattacharyya A. K., 1943, Bull. Calcutta Math. Soc., 35, 99
  * Blas et al. (2011) Blas D., Lesgourgues J., Tram T., 2011, JCAP, 07, 034
  * Brinckmann & Lesgourgues (2018) Brinckmann T., Lesgourgues J., 2018, arXiv 1804.07261
  * Caticha & Giffin (2006) Caticha A., Giffin A., 2006, in Mohammad-Djafari A., ed., American Institute of Physics Conference Series Vol. 872, Bayesian Inference and Maximum Entropy Methods In Science and Engineering. pp 31–42 (arXiv:physics/0608185), doi:10.1063/1.2423258
  * Charnock et al. (2017) Charnock T., Battye R. A., Moss A., 2017, Phys. Rev. D, 95, 123535
  * Coe (2009) Coe D., 2009, ArXiv 0906.4123
  * Couchot et al. (2017) Couchot F., Henrot-Versillé S., Perdereau O., Plaszczynski S., d’Orfeuil B. R., Spinelli M., Tristram M., 2017, A+A, 597, 126
  * Dossett et al. (2015) Dossett J. N., Ishak M., Parkinson D., Davis T. M., 2015, PRD, 92
  * Douspis et al. (2019) Douspis M., Salvati L., Aghanim N., 2019, arXiv 1901.05289
  * Efstathiou & Lemos (2017) Efstathiou G., Lemos P., 2017, MNRAS, 476, 151
  * Elsner & Wandelt (2012) Elsner F., Wandelt B. D., 2012, A+A, 540, L6
  * Fisher (1935) Fisher R. A., 1935, JRSS, 98, 39
  * Giesel et al. (2020) Giesel E., Reischke R., Schäfer B. M., Chia D., 2020, Information geometry in cosmology (arXiv:2005.01057)
  * Grandis et al. (2016) Grandis S., Seehars S., Refregier A., Amara A., Nicola A., 2016, JCAP, p. 034
  * Handley & Lemos (2019) Handley W., Lemos P., 2019, PRD, 100, 023512
  * Heavens et al. (2017) Heavens A., Fantaye Y., Sellentin E., Eggers H., Hosenie Z., Kroon S., Mootoovaloo A., 2017, Phys. Rev. Lett., 119, 101301
  * Hobson et al. (2002) Hobson M. P., Bridle S. L., Lahav O., 2002, MNRAS, 335, 377
  * Joudaki et al. (2017) Joudaki S., et al., 2017, MNRAS, 471, 1259
  * Karpenka et al. (2015) Karpenka N. V., Feroz F., Hobson M. P., 2015, MNRAS, 449, 2405
  * Kullback & Leibler (1951) Kullback S., Leibler R. A., 1951, Ann. Math. Statist., 22, 79
  * Kunz et al. (2006) Kunz M., Trotta R., Parkinson D. R., 2006, Phys. Rev. D, 74, 023503
  * Lin & Ishak (2017) Lin W., Ishak M., 2017, Phys. Rev. D, 96, 023532
  * Lindley (1956) Lindley D. V., 1956, The Annals of Mathematical Statistics, 27, 986
  * Loredo (2012) Loredo T. J., 2012, Astrostatistical Challenges for the New Astronomy, Joseph M. Hilbe, ed., Springer, New York
  * MacCrann et al. (2015) MacCrann N., Zuntz J., Bridle S., Jain B., Becker M. R., 2015, MNRAS, 451, 2877
  * Marshall et al. (2006) Marshall P., Rajguru N., Slosar A., 2006, Phys. Rev. D, 73, 067302
  * Nicola et al. (2019) Nicola A., Amara A., Refregier A., 2019, JCAP, 01, 011
  * Planck Collaboration et al. (2018a) Planck Collaboration Aghanim N., et al. 2018a, ArXiv 1807.06209, p. arXiv:1807.06209
  * Planck Collaboration et al. (2018b) Planck Collaboration Aghanim N., et al. 2018b, Astronomy & Astrophysics, 641
  * Raveri & Hu (2019) Raveri M., Hu W., 2019, PRD, 99, 043506
  * Raveri et al. (2020) Raveri M., Zacharegkas G., Hu W., 2020, PRD, 101, 103527
  * Reischke et al. (2017) Reischke R., Kiessling A., Schäfer B. M., 2017, Mon. Not. Roy. Astron. Soc., 465, 4016
  * Renk et al. (2017) Renk J., Zumalacárregui M., Montanari F., Barreira A., 2017, JCAP, 10
  * Rényi (1960) Rényi A., 1960, Proceedings of the fourth Berkeley Symposium o Mathematics, Statistics and Probability, p. 547
  * Santos et al. (2017) Santos B., Devi N. C., Alcaniz J. S., 2017, PRD, 95, 123514
  * Schäfer & Reischke (2016) Schäfer B. M., Reischke R., 2016, MNRAS, 460, 3398
  * Scott (2018) Scott D., 2018, arXiv 1804.01318
  * Seehars et al. (2014) Seehars S., Amara A., Refregier A., Paranjape A., Akeret J., 2014, Phys. Rev. D, 90, 023533
  * Seehars et al. (2016) Seehars S., Grandis S., Amara A., Refregier A., 2016, Phys. Rev. D, 93, 103507
  * Sellentin et al. (2014) Sellentin E., Quartin M., Amendola L., 2014, MNRAS, 441, 1831
  * Shore & Johnson (1980) Shore J., Johnson R., 1980, IEEE Transactions on Information Theory, 26, 26
  * Suzuki et al. (2012) Suzuki N., et al., 2012, The Astrophysical Journal, 746, 85
  * Tegmark et al. (1997) Tegmark M., Taylor A., Heavens A., 1997, ApJ, 480, 22
  * Trotta (2017) Trotta R., 2017, arXiv 1701.01467
  * Verde et al. (2013) Verde L., Protopapas P., Jimenez R., 2013, Physics of the Dark Universe, 2, 166
  * Wolz et al. (2012) Wolz L., Kilbinger M., Weller J., Giannantonio T., 2012, JCAP, 09, 9