# A geometric way to find the measures of uncertainty from statistical
divergences for discrete and finite probability distributions

Gautam Sharma<EMAIL_ADDRESS>Optics and Quantum Information Group,
Institute of Mathematical Sciences, HBNI, CIT Campus, Taramani, Chennai
600113, India  Sk Sazim<EMAIL_ADDRESS>RCQI, Institute of Physics,
Slovak Academy of Sciences, Bratislava, 845 11, Slovakia

###### Abstract

Exploiting the geometric nature of statistical divergences, we devise a way to
define associated induced uncertainty measures for discrete and finite
probability distributions. We also report new uncertainty measures and discuss
their properties. Further, we apply a similar technique to measure the
uncertainty in the preparation of a quantum state.

##  I To begin with

Modern science has been going through a transformation with the inclusion of
tools from information science. The information theoretic approach has led to
describe physical phenomenon in a more operational way. A common brigde is the
information measure of an event – the entropy. After the seminal work by
Shannon Shannon (1948), a plethora of measures of entropy have been discovered
and studied. Most of these constructions of entropies were built under the
assumption of certain axioms Faddeev (1956); Diderrich (1975); Csiszár (2008).
One of the operational application of the entropic quantities have been to
quantify the uncertainty in a measurement outcome.

Distance between two probability distributions captures the difference in
information content between them. These distances are sometime called
divergences because of their non-symmetric nature. It is well understood that
for each divergence there exists an information measure. For example, consider
Shannon entropy and Kullback-Liebler divergence Csiszár (2008). One can obtain
the Shannon entropy of a finite probability distribution
$\mathcal{P}=\\{p_{i}\\}_{1}^{n}$, in terms of the Kullback-Liebler divergence
Kullback and Leibler (1951) of $\mathcal{P}$ from the uniform distribution
$\mathcal{P}_{U}=(1/n,...,1/n)$ van Erven and Harremos (2014), i.e.

$\displaystyle S(\mathcal{P})=\log n-D_{KL}(\mathcal{P},\mathcal{P}_{U}),$

where $S(\mathcal{P})=-\sum_{i}p_{i}\log p_{i}$ is the Shannon entropy,
$D_{KL}(\mathcal{P},\mathcal{P}_{U})$ is the Kullback-Liebler divergence of
$\mathcal{P}$ from $\mathcal{P}_{U}$, and the base of the logarithm is taken
$2$ for the whole paper. However, such a derivation does not suggest why these
quantities are a valid measures of information (uncertainty). It has been
shown that all the non-negative Schur concave functions which take zero value
for a maximally certain probability distributions are valid measures of
uncertainty Friedland  _et al._ (2013). It is not always clear how to obtain
an information measure for a given divergence measure.

In this work, we show using geometric approach how to obtain an uncertainty
measure from statistical divergence measures. Our approach is universal and
intuitive in the sense that we define the uncertainty measure of a probability
distribution $\mathcal{P}$ from a maximally certain probability distribution.

The rest of the paper is organised as follows. In sec.II, we define
uncertainty as the distance from a maximally certain distribution. Then, we
give examples of obtaining several known and new uncertainty measures in
sec.III. In sec.IV, we discuss the properties of new uncertainty measures that
we found. We use a similar technique to quantify the uncertainty in the
preparation of a quantum state in sec.V and finally we conclude in sec.VI.

##  II Uncertainty as the statistical distance from the most certain
distributions

Given a discrete probability distribution $\mathcal{P}=\\{p_{i}\\}_{i=1}^{n}$,
we want to give an intuitive meaning to its uncertainty, with
$\sum_{i}p_{i}=1$. A probability distribution has zero uncertainty iff one of
the $p_{i}=1$. There can be $n$ number of such probability distributions, we
label them as $\mathcal{P}_{C}$. On the other hand, a probability distribution
is said to be maximally uncertain iff each $p_{i}=\frac{1}{n}$, labelled as
$\mathcal{P}_{U}$. It is obvious that, there can only be a unique
$\mathcal{P}_{U}$.

Intuitively, we know that the uncertainty in a probability distribution
$\mathcal{P}$ will be large if $\mathcal{P}$ is very far from
$\mathcal{P}_{C}$ or very close to $\mathcal{P}_{U}$. Thus, we can quantify
uncertainty using a statistical divergence measure
$D(\mathcal{P}||\mathcal{P}_{C})$, from a maximally certain distribution
$\mathcal{P}_{C}$. But there is an issue that there are $n$ possible
distributions $\mathcal{P}_{C}$, so for a distribution $\mathcal{P}$ there
will be $n$ possible distances $D(P||\mathcal{P}_{C})$, where as we are
looking for a unique value. To resolve this, we note that the maximally
uncertain distribution $\mathcal{P}_{U}$, is unique and it is equidistant from
all $\mathcal{P}_{C}$. Using this fact we employ the following expression to
define uncertainty.

$\displaystyle
U^{\uparrow}(\mathcal{P})=D(\mathcal{P}_{C}||\mathcal{P}_{U})-D(\mathcal{P}||\mathcal{P}_{U}).$
(1)

The above expression of $U^{\uparrow}(\mathcal{P})$ is uniquely defined and
captures the distance of $\mathcal{P}$ from the set of $\mathcal{P}_{C}$
uniquely. Using any statiscal divergence measure $D(\mathcal{P}||\mathcal{Q})$
between two probability distributions $\mathcal{P}$ and $\mathcal{Q}$, one can
get different measures of uncertainty.

Figure 1: A schematic describing how the uncertainty measures are
geometrically defined. All the points lying on a unique circle are equally
uncertain. The divergences represented by vectors pointing away from the
center give the uncertainty measure $U^{\uparrow}(\mathcal{P})$ where as the
divergences represented by vectors pointing towards the center give
$U^{\downarrow}(\mathcal{P})$ as the measure of uncertainty.

Further, a divergence measure need not be symmetric, i.e.,
$D(\mathcal{P}||\mathcal{Q})$ might not be equal to
$D(\mathcal{Q}||\mathcal{P})$. Therefore, we can have another definition of
uncertainty as

$\displaystyle
U^{\downarrow}(\mathcal{P})=D(\mathcal{P}_{U}||\mathcal{P}_{C})-D(\mathcal{P}_{U}||\mathcal{P}).$
(2)

We explain the two measures from a schematic diagram in Fig.(1). The asymmetry
of the divergence measures is depicted via vectors in opposite directions, one
directed away from the center and other directed towards the center.

To ensure that the uncertainty measures $U^{\uparrow}(\mathcal{P})$ and
$U^{\downarrow}(\mathcal{P})$ are valid uncertainty measures, they should be
non-negative Schur-concave functions, see Friedland  _et al._ (2013). In
Eq.(1) and Eq.(2) the first terms $D(\mathcal{P}_{C}||\mathcal{P}_{U})$ and
$D(\mathcal{P}_{U}||\mathcal{P}_{C})$ respectively, are constants. Therefore,
the second terms $D(\mathcal{P},\mathcal{P}_{U})$ and
$D(\mathcal{P}_{U},\mathcal{P})$ must be a Schur-convex function(as the
negative of Schur-convex function is a Schur-concave function), to ensure that
the quantities in Eq.(1) and Eq.(2) are a valid uncertainty measures.

##  III Constructing uncertainty measures using f-divergences

In this section, we show how different statistical divergence measures can
lead to various uncertainty measures. A category of divergence measures for
which the second terms in Eq.(1) and Eq.(2) are Schur-convex, are the
f-divergence measures CSISZAR (1967); Csiszár (2008); Ali and Silvey (1966).
This can be seen very easily as, for all the f-divergences
$D(\mathcal{P}||\mathcal{P}_{U})=\sum_{i=1}^{n}\frac{1}{n}f(np_{i})$, where
$f:[0,\infty)\rightarrow\mathbb{R}$ is a convex function, which gives a Schur-
convex function. Here we have used the fact that the linear sum of convex
function leads to a Schur-convex function Peajcariaac and Tong (1992). Using
the same property we also have,
$D(\mathcal{P}_{U}||\mathcal{P})=\sum_{i=1}^{n}p_{i}f(\frac{1}{np_{i}})$, a
Schur-convex function.

Next, we give a few examples of constructing the uncertainty measures by
substituting a well known f-divergences measures in Eq.(1) and Eq.(2). We will
show, how both equations can lead to different measures whenever the given
f-divergence is asymmetric.

###  III.1 Renyi Divergence

For two discrete probability distributions $\mathcal{P}=\\{p_{i}\\}_{i=1}^{n}$
and $\mathcal{Q}=\\{q_{i}\\}_{i=1}^{n}$, the Renyi Divergence is defined as
Rényi  _et al._ (1961)

$\displaystyle
D_{\alpha}(\mathcal{P}||\mathcal{Q})=\frac{1}{\alpha-1}\log\left(\sum_{i=1}^{n}\frac{p_{i}^{\alpha}}{q_{i}^{\alpha-1}}\right).$
(3)

where $\alpha\in(0,1)\cup(1,\infty)$. Renyi divergence can also be defined for
$\alpha=0,1$ and $\infty$ by taking a limit.

####  III.1.1 Measures from $U^{\uparrow}(\mathcal{P})$

First we substitute the Renyi divergence in Eq.(1), which gives the well known
Renyi EntropyBen-Bassat and Raviv (1978); Aczél and Daróczy (1975) as the
uncertainty measure.

$\displaystyle U^{\uparrow}_{\alpha}(\mathcal{P})$
$\displaystyle=D_{\alpha}(\mathcal{P}_{C}||\mathcal{P}_{U})-D_{\alpha}(\mathcal{P}||\mathcal{P}_{U})$
$\displaystyle=\frac{1}{\alpha-1}\log
n^{\alpha-1}-\frac{1}{\alpha-1}\log\left(n^{\alpha-1}\sum_{i}^{n}p_{i}^{\alpha}\right)$
$\displaystyle=\frac{1}{1-\alpha}\log\sum_{i}^{n}p_{i}^{\alpha}.$ (4)

In the following we mention the form of Renyi divergence and corresponding
uncertainty measure for a few special values of $\alpha$.

  * •

For $\alpha=0$,
$D_{0}(\mathcal{P}||\mathcal{Q})=-\log Q$,
$U^{\uparrow}_{0}(\mathcal{P})=\log Q.$

where $\mathcal{Q}$ is the cardinality of the probability space for which
$p_{i}$ is non-zero. Thus, we get the Hartley/Max Entropy measure of
uncertainty for $\alpha=0$.

  * •

$\alpha=\frac{1}{2},\\\ \hskip
36.98866ptD_{\frac{1}{2}}(\mathcal{P}||\mathcal{Q})=-2\log\sum_{i=1}^{n}\sqrt{p_{i}q_{i}}\\\
\hskip
36.98866ptU^{\uparrow}_{\frac{1}{2}}(\mathcal{P})=2\log(\sum_{i=1}^{n}\sqrt{p_{i}})$.

In this case, the Renyi divergence becomes the negative log of “Bhattacharya
Coefficient”, which is also a measure of overlap of probability distributions
Bhattacharyya (1946).

  * •

$\alpha=1,\\\ \hskip
22.76228ptD_{1}(\mathcal{P}||\mathcal{Q})=D_{KL}(\mathcal{P}||\mathcal{Q})=\sum_{i=1}^{n}p_{i}\log\frac{p_{i}}{q_{i}}\\\
\hskip 36.98866ptU^{\uparrow}_{1}(\mathcal{P})=-\sum_{i}^{n}p_{i}\log p_{i}$.

In this case, Renyi divergence takes the form of “Kullback-Leibler divergence”
donoted as $D_{KL}(\mathcal{P}||\mathcal{Q})$, which gives the well known
Shannon entropic measure of uncertainty. There also exists a symmetric form of
Kullback-Leibler divergence, known as the Jensen-Shannon divergence, which we
discuss in the next subsection.

  * •

$\alpha=\infty,\\\ \hskip
36.98866ptD_{\infty}(\mathcal{P}||\mathcal{Q})=\log\frac{p_{i}}{q_{i}}_{max}\\\
\hskip 36.98866ptU^{\uparrow}_{\infty}(\mathcal{P})=-\log{p_{i}}_{max}$.

This is known as the min-Entropy, as it is the smallest in the family of Renyi
entropies.

####  III.1.2 Measures from $U^{\downarrow}(\mathcal{P})$

Next, we substitute Renyi divergence in Eq.(2), which gives the following
measure of uncertainty.

$\displaystyle U^{\downarrow}_{\alpha}(\mathcal{P})$
$\displaystyle=D_{\alpha}(\mathcal{P}_{U}||\mathcal{P}_{C})-D_{\alpha}(\mathcal{P}_{U}||\mathcal{P})$
$\displaystyle=\frac{1}{\alpha-1}\log\left(\frac{1}{n^{\alpha}}\right)-\frac{1}{\alpha-1}\log\left(\sum_{i=1}^{n}\frac{1}{n^{\alpha}p_{i}^{\alpha-1}}\right)$
$\displaystyle=\frac{1}{1-\alpha}\log\left(\sum_{i=1}^{n}p_{i}^{1-\alpha}\right).$

The above measure is well defined only for $\alpha\in(0,1)$. We can redefine
$1-\alpha=\gamma$, so that

$\displaystyle
U^{\downarrow}_{\gamma}(\mathcal{P})=\frac{1}{\gamma}\log\left(\sum_{i=1}^{n}p_{i}^{\gamma}\right).$
(5)

It can be easily seen on comparing the above measure with the Renyi entropy
that
$U^{\downarrow}_{\alpha}(\mathcal{P})=\frac{\alpha-1}{\alpha}U^{\uparrow}_{\alpha}(\mathcal{P})$,
i.e. the above measure is the rescaled Renyi Entropy measure for
$\alpha\in(0,1)$.

###  III.2 Jensen-Shannon Divergence

The Jensen-Shannon divergence between two probability distributions,
$D_{JS}(\mathcal{P}||\mathcal{Q})$ has the following form Lin (1991); Menéndez
_et al._ (1997)

$\displaystyle D_{JS}(\mathcal{P}||\mathcal{Q})$
$\displaystyle=D_{KL}(\mathcal{P}||\frac{\mathcal{P}+\mathcal{Q}}{2})+D_{KL}(\mathcal{Q}||\frac{\mathcal{P}+\mathcal{Q}}{2}).$

As the Jensen-Shannon divergence is symmetric, it will give the same
uncertainty measure via both Eq.(1) and Eq.(2). On substituting
$D_{JS}(\mathcal{P}||\mathcal{Q})$ in Eq.(1), we get the following measure of
uncertainty which we denote as $U_{JS}(\mathcal{P})$

$\displaystyle U_{JS}(\mathcal{P})=$
$\displaystyle\log\Big{(}\frac{4n^{2}}{(n+1)^{\frac{1+n}{n}}}\Big{)}-\sum_{i=1}^{n}p_{i}\log(p_{i})$
$\displaystyle+\frac{2}{n}\sum_{i=1}^{n}\frac{np_{i}+1}{2}\log\Big{(}\frac{np_{i}+1}{2}\Big{)}.$
(6)

This entropy looks different than the usual Shannon entropy because of the
last term. In next Section, we will discuss a few of its properties in
details. Note also that one can consider more general Shannon-Jenson
divergence by replacing KL-divergence with one parameter f-divergence (see
Csiszár (2008)), which may induce a new information measure.

###  III.3 Tsallis Divergence

Again, we consider two probability distributions
$\mathcal{P}=\\{p_{i}\\}^{n}_{i=1}$ and $\mathcal{Q}=\\{q_{i}\\}^{n}_{i=1}$.
The Tsallis divergence between them can be defined as Nielsen and Nock (2011)

$\displaystyle
D_{\beta}(\mathcal{P}||\mathcal{Q})=\sum_{i=1}^{n}\frac{p_{i}^{\beta}q_{i}^{1-\beta}-1}{\beta-1},$
(7)

where $\beta\in\mathbb{R}$ and limit has to be taken for $\beta$ tending to 1.

####  III.3.1 Measures from $U^{\uparrow}(\mathcal{P})$

On substituting the Tsallis divergence in Eq.(1), we get the Tsallis entropy
as the measure of uncertaintyTsallis (1988).

$\displaystyle U^{\uparrow}_{\beta}(\mathcal{P})$
$\displaystyle=D_{\beta}(\mathcal{P}_{C}||\mathcal{P}_{U})-D_{\beta}(\mathcal{P}||\mathcal{P}_{U})$
$\displaystyle=\frac{1}{(\beta-1)n^{1-\beta}}\Big{(}1-\sum_{i=1}^{n}p_{i}^{\beta}\Big{)}.$
(8)

####  III.3.2 Measures from $U^{\downarrow}(\mathcal{P})$

If we substitute the Tsallis divergence in Eq.(2), we get

$\displaystyle U^{\downarrow}_{\beta}(\mathcal{P})$
$\displaystyle=D_{\beta}(\mathcal{P}_{U}||\mathcal{P}_{C})-D_{\beta}(\mathcal{P}_{U}||\mathcal{P})$
$\displaystyle=\frac{1}{n^{\beta}(\beta-1)}(1-n^{\beta}-\sum_{i=1}^{n}p_{i}^{1-\beta}).$

This measure is well defined only for $\beta\in(0,1)$. It can be easily
observed that above measure can be obtained from Tsallis uncertainty by adding
a constant term followed by rescaling.

###  III.4 Hellinger distance

Hellinger distance between $\mathcal{P}=\\{p_{i}\\}^{n}_{i=1}$ and
$\mathcal{Q}=\\{q_{i}\\}^{n}_{i=1}$ is defined as Le Cam (2012)

$\displaystyle
D_{H}(\mathcal{P}||\mathcal{Q})=\sum_{i}(||\sqrt{p_{i}}-\sqrt{q_{i}}||)^{2}.$

As this is a symmetric divergence measure, it will give same measure of
uncertainty via both Eqs. (1 and 2). By using Hellinger distance in Eq.(1), we
get the following measure of uncertainty

$\displaystyle U_{H}(\mathcal{P})$
$\displaystyle=D_{H}(\mathcal{P}_{C}||\mathcal{P}_{U})-D_{H}(\mathcal{P}||\mathcal{P}_{U})$
$\displaystyle=\frac{1}{n}+1-\frac{2}{\sqrt{n}}-\sum_{i}p_{i}^{2}-\frac{1}{n}+\frac{2}{\sqrt{n}}$
$\displaystyle=1-\sum_{i}p_{i}^{2}.$ (9)

We see that this reproduces the rescaled Tsallis entropic measure of
uncertainty with $\beta=2$ and $n=1$ in Eq.(III.3.1).

###  III.5 The Total variation distance

The Total variation distance between two probability distributions
$\mathcal{P}$ and $\mathcal{Q}$ on $\mathcal{E}$ is defined as

$\displaystyle
D_{TV}(\mathcal{P}||\mathcal{Q})=\operatorname{sup}_{A\subset\mathcal{E}}|\mathcal{P}(A)-\mathcal{Q}(A)|.$

Intuitively, it is the largest possible difference between two distributions
on $\mathcal{P}(\mathcal{E})$, set of probabilities on $\mathcal{E}$.

For a finite or countable $\mathcal{E}$, this reduces to $\frac{1}{2}$ times
the $l_{1}$-norm Aldous (2019)

$\displaystyle
D_{TV}(\mathcal{P}||\mathcal{Q})=\frac{1}{2}\sum_{i}|p(i)-q(i)|,$

which is symmetric with respect to the probabilities. Hence, this will give
same measure of uncertainty from Eqns.(1) and (2). Using the total variation
as the measure of statistical divergence in Eqn.(1), we get the following

$\displaystyle U_{TV}(\mathcal{P})$
$\displaystyle=D_{TV}(\mathcal{P}_{C}||\mathcal{P}_{U})-D_{TV}(\mathcal{P}||\mathcal{P}_{U})$
$\displaystyle=1-\frac{1}{n}-\frac{1}{2}\sum_{i}|\frac{1}{n}-p_{i}|.$ (10)

The above quantity is a new measure of uncertainty, which we name as “Absolute
uncertainty”. We will also discuss its properties in the next Section.

In Fig.(2), we plot Shannon, Jensen-Shannon, Absolute, and Hellinger
uncertainty measures for a two dimensional probability distribution
$\\{p,1-p\\}$ with the parameter $p$. This figure shows that all the measures
are faithful and continuous. However, Absolute uncertainty shows discontinuity
in its first derivative at the maximum uncertain point.

Figure 2: (Color online) Numerical plots of Shannon, Jensen-Shannon (JS),
Absolute, and Hellinger measures of uncertainty. The Absolute measure of
uncertainty has a discountinuity in its derivative at the maximally uncertain
point. For fair comparison, we normalize the Hellinger and Absolute
uncertainty.

##  IV Properties of new Uncertainty measures

We will discuss the properties of the Jensen-Shannon uncertainty
$U_{JS}(\mathcal{P})$ and absolute uncertainty $U_{TV}(\mathcal{P})$. A
general discussion on the properties of various entropic uncertainty measures
can be found in Csiszár (2008); Ilić and Stanković (2014, 2014); Suyari
(2004).

###  IV.1 Jensen-Shannon uncertainty

  * •

Continuity-It can be easily seen that the quantity $U_{JS}(\mathcal{P})$ in
Eq.(III.2) is a continuous function of the allowed values of $p_{i}$’s.

  * •

Maximality\- As $U_{JS}(\mathcal{P})$ is a Schur-concave non-negative
function, it attains its maximum value for the uniform distribution
$\mathcal{P}_{U}$.

  * •

Expandibility\- Expandability means that
$U(p_{1},...,p_{n})=U(p_{1},...,p_{n},0)$. Unlike Renyi and Tsallis entropies,
the Jensen-Shannon is not expandable as $U_{JS}(\mathcal{P})$ has dimension
dependent terms. One can check this for the simplest case of
$\mathcal{P}=\\{p_{1},p_{2}\\}$ as follows

$\displaystyle
U_{JS}(p_{1},p_{2})=\log\Big{(}\frac{16}{\sqrt{27}}\Big{)}+S(\mathcal{P})-S\Big{(}\frac{2\mathcal{P}+1}{2}\Big{)}$
$\displaystyle
U_{JS}(p_{1},p_{2},0)=\log\Big{(}\frac{36}{4^{\frac{4}{3}}}\Big{)}+S(\mathcal{P})-S(\frac{3\mathcal{P}^{\prime}+1}{2}),$

where $\mathcal{P}^{\prime}=\\{p_{1},p_{2},0\\}$.

We argue here that expandability property need not be a necessary condition
for an uncertainty measure. For example, consider a two dimensional
probability distribution $\mathcal{P}_{2}=\\{\frac{1}{2},\frac{1}{2}\\}$ which
is expanded to a three dimensional probability distribution
$\mathcal{P}_{3}=\\{\frac{1}{2},\frac{1}{2},0\\}$. While $\mathcal{P}_{2}$ is
itself the uniform distribution in two dimensions, $\mathcal{P}_{3}$ is far
away from the uniform distribution $\\{\frac{1}{3},\frac{1}{3},\frac{1}{3}\\}$
in three dimensions. Thus, it is not natural to expect expandability in this
scenario.

  * •

Additivity\- Additivity means that for the probability distributions,
$\mathcal{P}=\\{p_{1},p_{2},...,p_{n}\\}$,
$\mathcal{R}=\\{r_{11},r_{12},...,r_{nm}\\}$ $\forall n,m\in\mathbb{N}$ if
there exists a probability distribution
$\mathcal{Q}=\\{q_{1|k},q_{2|k},...,q_{m|k}\\}$, where $r_{ij}=q_{i|k}p_{i}$,
then we can express the uncertainty in $\mathcal{R}$ as following

$\displaystyle U(\mathcal{R})=U(\mathcal{P})+U(\mathcal{Q}|\mathcal{P}).$

Instead, for the Jensen-Shannon uncertainty, we have

$\displaystyle U_{JS}(\mathcal{R})$
$\displaystyle=\log\Big{(}\frac{4(nm)^{2}}{(n+1)^{\frac{1+nm}{nm}}}\Big{)}-\sum_{i=1}^{nm}r_{i}\log(r_{i})$
$\displaystyle+\frac{2}{nm}\sum_{i=1}^{nm}\frac{nmr_{i}+1}{2}\log\Big{(}\frac{nmr_{i}+1}{2}\Big{)}.$

Clearly, except for the second term, no other terms can be written as linear
addition.

###  IV.2 Absolute uncertainty

  * •

Continuity-Again, it is easy to check that the quantity $U_{TV}(\mathcal{P})$
in Eq.(III.5) is a continuous function of the allowed values of $p_{i}$’s.

  * •

Maximality\- As $U_{TV}(\mathcal{P})$ is a Schur-concave non-negative
function, it attains its maximum value for the uniform distribution
$\mathcal{P}_{U}$.

  * •

Expandibility\- Similar to Jensen-Shannon uncertainty, the Absolute
uncertainty is not expandable as $U_{TV}(\mathcal{P})$ has dimension dependent
terms. One can again find it via the simple example of $U_{TV}(p_{1},p_{2})$
to $U_{TV}(p_{1},p_{2},0)$ as following

$\displaystyle
U_{TV}(p_{1},p_{2})=\frac{1}{2}-\frac{1}{2}\Big{(}\big{|}\frac{1}{2}-p_{1}\big{|}+\big{|}\frac{1}{2}-p_{2}\big{|}\Big{)},$
$\displaystyle
U_{TV}(p_{1},p_{2},0)=\frac{5}{9}-\frac{1}{3}\Big{(}\big{|}\frac{1}{3}-p_{1}\big{|}+\big{|}\frac{1}{3}-p_{2}\big{|}\Big{)}.$

  * •

Additivity\- For the Absolute uncertainty we have

$\displaystyle
U_{TV}(\mathcal{R})=1-\frac{1}{nm}-\frac{1}{2}\sum_{i}|\frac{1}{nm}-r_{i}|$

Here also, we can not expand it as a linear sum of $U_{TV}(\mathcal{P})$ and
$U_{TV}(\mathcal{Q}|\mathcal{P})$ as the terms inside the modulus can not be
separated.

##  V Uncertainty in the preparation of a quantum state

Here, we will discuss the analogical extension of the above formalism to
quantum domain. Let us consider that a finite dimensional quantum system is
described by density matrix $\rho\in{\mathcal{L}}({\mathcal{H}})$. If
$\rho^{2}\neq\rho$, it is a mixed state or in other words we say that it has
preparation uncertainty. For pure state, the uncertainty is zero while it is
maximum for maximally mixed state. Now, question is: how far our $\rho$ is
from the pure states, will be its preparation uncertainty. But finding this
distance requires a minimization over all pure states. To bypass this, one can
consider the following distances,
$\mathcal{D}\left(V|\psi\rangle\langle\psi|V^{\dagger}||\frac{\mathbb{I}}{d}\right)$
and $\mathcal{D}\left(\rho||\frac{\mathbb{I}}{d}\right)$, then finally can
reach to

$\displaystyle
U(\rho)=\mathcal{D}\left(V|\psi\rangle\langle\psi|V^{\dagger}||\frac{\mathbb{I}}{d}\right)-\mathcal{D}\left(\rho||\frac{\mathbb{I}}{d}\right),$
(11)

where $\mathcal{D}(\rho||\sigma)$ is a valid distance measure between two
density matrices, $V$ is an arbitrary $d\times d$ unitary matrix and $d$ is
dimension of the Hilbert space of the states. Now we will consider some known
statistical divergences in the quantum case and see what measures of
uncertainty they will induce.

###  V.1 Bures distance and Hellinger distance

Bures and Hellinger distance for two density matrices
$\rho,\sigma\in{\mathcal{L}}({\mathcal{H}})$ are defined as Nielsen and Chuang
(2010); Luo and Zhang (2004)

$\displaystyle
D_{B}(\rho||\sigma)=2-2F(\rho,\sigma)\>\>\mbox{and}\>\>D_{H}(\rho||\sigma)=2-2A(\rho,\sigma)$

respectively, where $F(\rho,\sigma)={\rm
Tr}\sqrt{\sqrt{\rho}\sigma\sqrt{\rho}}$ is Fidelity and $A(\rho,\sigma)={\rm
Tr}\sqrt{\rho}\sqrt{\sigma}$ is Affinity between two states $\rho,\sigma$. It
can be seen that the both distances induce the same uncertainty measure, i.e.,

$\displaystyle U(\rho)=\frac{1}{\sqrt{d}}({\rm Tr}\sqrt{\rho}-1),$ (12)

by noticing that
$F(\mathbb{I}/d,V|\psi\rangle\langle\psi|V^{\dagger})=A(\mathbb{I}/d,V|\psi\rangle\langle\psi|V^{\dagger})=\frac{1}{\sqrt{d}}$
and $F(\mathbb{I}/d,\rho)=A(\mathbb{I}/d,\rho)=\frac{1}{\sqrt{d}}{\rm
Tr}\sqrt{\rho}$. The uncertainty measure in Eq.(12) is similar to linear
entropy and related to Tsallis entropy $T_{\frac{1}{2}}(\rho)$.

###  V.2 Distance induced by $l_{p}$-norm and Schatten $p$-norm

For a $d_{1}\times d_{2}$ matrix $M=\\{M_{ij}\\}$ and $p\in[1,\infty)$, the
two norms are defined as

$\displaystyle||M||_{l_{p}}=\left(\sum_{i,j}|M_{ij}|^{p}\right)^{\frac{1}{p}}\>\>\mbox{and}\>\>||M||_{p}=\left(\sum_{i}^{r}\lambda_{i}^{p}\right)^{\frac{1}{p}},$

where $\lambda_{i}$ are non-zero eigen values of $|M|=\sqrt{M^{\dagger}M}$ and
$r$ is the rank of $M$. Now the distance induced by these two norms are,
respectively, $D_{l_{p}}(\rho||\sigma)=||\rho-\sigma||_{l_{p}}$ and
$D_{p}(\rho||\sigma)=||\rho-\sigma||_{p}$. We find that these two distances
yield same information measure as

$\displaystyle
U(\rho)=\frac{[(d-1)^{p}+d-1]^{\frac{1}{p}}}{d}-\left(\sum_{i=1}^{d}|\lambda_{i}^{\rho}-\frac{1}{d}|^{p}\right)^{\frac{1}{p}},$
(13)

where $\lambda_{i}^{\rho}$ are the eigenvalues of $\rho$.

###  V.3 Hilbert-Schmidt distance

Hilbert-Schmidt distance is induced by Hilbert-Schmidt norm, and for two
density matrices $\rho,\sigma$, it is defined as

$\displaystyle\mathcal{D}_{HS}(\rho||\sigma)={\rm Tr}[(\rho-\sigma)^{2}].$

We notice that
$\mathcal{D}_{HS}(V|\psi\rangle\langle\psi|V^{\dagger}||\mathbb{I}/d)=1-1/d$
and $\mathcal{D}_{HS}(\rho||\mathbb{I}/d)={\rm Tr}\rho^{2}-1/d$. This tells us
that the induced uncertainty measure is given by

$\displaystyle U_{HS}(\rho)=1-{\rm Tr}\rho^{2},$ (14)

which we all recognise as linear entropy.

###  V.4 Generalized Rényi divergence

The generalized Rényi divergence was introduced in Müller-Lennert  _et al._
(2013) and is defined as

$\displaystyle D_{\alpha}(\rho||\sigma)=\frac{1}{\alpha-1}\log{\rm
Tr}\left(\sigma^{\frac{1-\alpha}{2\alpha}}\rho\sigma^{\frac{1-\alpha}{2\alpha}}\right)^{\alpha},$

where $\alpha\in\mathbb{R}$ and $\rho$ is not orthogonal to $\sigma$.
Whenever, $\rho$ and $\sigma$ commute, the generalized Rényi divergence
reduces to classical $\alpha$-Rényi divergence. Clearly, for our case, as
maximally mixed state commutes with both $\rho$ and
$V|\psi\rangle\langle\psi|V^{\dagger}$, the distances,
$D_{\alpha}(V|\psi\rangle\langle\psi|V^{\dagger}||\mathbb{I}/d)=\log d$ and
$D_{\alpha}(\rho||\mathbb{I}/d)=\log
d-\frac{1}{\alpha-1}\log\operatorname{Tr}\rho^{\alpha}$. Hence, we reach

$\displaystyle
U_{\alpha}(\rho)=\frac{1}{1-\alpha}\log\operatorname{Tr}\rho^{\alpha}=S_{\alpha}(\rho),$
(15)

the well known Rényi entropy.

Figure 3: Numerical plot of Bures (Hellinger), $l_{1}$-norm, Hilbert-Schmidt
(HS), and Shannon measures of uncertainty for the state,
$\rho=p|\psi\rangle\langle\psi|+(1-p)\frac{\mathbb{I}}{2}$, where
$|\psi\rangle$ is an arbitrary pure state in $d=2$. For fair comparison, we
normalize the uncertainty induced by Bures and Hellinger distances.

###  V.5 Generalized Tsallies divergence

The Tsallis divergence is defined as Rajagopal  _et al._ (2014)

$\displaystyle D_{\beta}(\rho||\sigma)=\frac{1}{1-\beta}{\rm
Tr}\left(\sigma^{\frac{1-\beta}{2}}\rho^{\frac{\beta}{}}\sigma^{\frac{1-\beta}{2}}\right).$
(16)

Then,
$D_{\beta}(V|\psi\rangle\langle\psi|V^{\dagger}||\mathbb{I}/d)=\frac{1}{1-\beta}(\frac{1}{d^{1-\beta}}-1)$
and $D_{\beta}(\rho||\mathbb{I}/d)=\frac{1}{1-\beta}(\frac{1}{d^{1-\beta}}{\rm
Tr}[\rho^{\beta}]-1)$. Thus,

$\displaystyle
U_{\beta}(\rho)=\frac{-1}{d^{1-\beta}}\left[\frac{1}{1-\beta}({\rm
Tr}[\rho^{\beta}]-1)\right]=\frac{(-1)}{d^{1-\beta}}T_{\beta}(\rho),$ (17)

is the Tsallies entropy, $T_{\beta}(\rho)$ with a factor
$\frac{-1}{d^{1-\beta}}$.

We plot Bures (Hellinger), $l_{1}$-norm, Hilbert-Schmidt, and Shannon
uncertainty measures in Fig.(3) for an arbitrary state $\rho$ in $d=2$. The
figure hints that the uncertainty captured by $l_{1}$-norm measure is lowest
for $0<p<1$ and Bures measure upper bound the others.

##  VI Conclusion

In this work we have shown how the uncertainty measures arise from the
geometry of statistical divergence measures. It captures the essence of
uncertainty as the distance of a probability distribution form a certain
probability distribution. We also report two new forms of uncertainty from the
Jensen-Shannon divergence and Total Variation distance and discuss their
properties. In particular, the two new uncertainty measures do not satisfy the
expandability axiom, which is satisfied by the more commonly used uncertainty
measures.

We also apply a similar geometric technique to obtain the uncertainty in the
preparation of a state or the mixedness. We reproduce the commonly used
measures of mixedness using various distance measures of two quantum states.

This work opens up several new directions of research. First, it would be
interesting to see which other geometric approaches can produce uncertainty
measures from divergences. Our work also sets up a standard method for finding
various uncertainty measures. Second, it would also be important to find the
uncertainty relations and various applications of the new uncertainty measures
found here.

Acknowledgement:– SS acknowledges the financial support through the Štefan
Schwarz stipend from Slovak Academy of Sciences, Bratislava. SS also
acknowledges the financial support through the project OPTIQUTE (APVV-18-0518)
and HOQIP (VEGA 2/0161/19).

## References

  * Shannon (1948) C. E. Shannon, Bell System Technical Journal 27, 379 (1948), https://onlinelibrary.wiley.com/doi/pdf/10.1002/j.1538-7305.1948.tb01338.x .
  * Faddeev (1956) D. K. Faddeev, Uspekhi Mat. Nauk 11, 227 (1956).
  * Diderrich (1975) G. T. Diderrich, Information and Control 29, 149 (1975).
  * Csiszár (2008) I. Csiszár, Entropy 10, 261 (2008).
  * Kullback and Leibler (1951) S. Kullback and R. A. Leibler, The annals of mathematical statistics 22, 79 (1951).
  * van Erven and Harremos (2014) T. van Erven and P. Harremos, IEEE Transactions on Information Theory 60, 3797 (2014).
  * Friedland  _et al._ (2013) S. Friedland, V. Gheorghiu, and G. Gour, Phys. Rev. Lett. 111, 230401 (2013).
  * CSISZAR (1967) I. CSISZAR, Studia Scientiarum Mathematicarum Hungarica 2, 229 (1967).
  * Ali and Silvey (1966) S. M. Ali and S. D. Silvey, Journal of the Royal Statistical Society. Series B (Methodological) 28, 131 (1966).
  * Peajcariaac and Tong (1992) J. E. Peajcariaac and Y. L. Tong,  _Convex functions, partial orderings, and statistical applications_ (Academic Press, 1992).
  * Rényi  _et al._ (1961) A. Rényi _et al._ , in  _Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics_ (The Regents of the University of California, 1961).
  * Ben-Bassat and Raviv (1978) M. Ben-Bassat and J. Raviv, IEEE Transactions on Information Theory 24, 324 (1978).
  * Aczél and Daróczy (1975) J. Aczél and Z. Daróczy, New York 168 (1975).
  * Bhattacharyya (1946) A. Bhattacharyya, Sankhyā: The Indian Journal of Statistics (1933-1960) 7, 401 (1946).
  * Lin (1991) J. Lin, IEEE Transactions on Information Theory 37, 145 (1991).
  * Menéndez  _et al._ (1997) M. Menéndez, J. Pardo, L. Pardo, and M. Pardo, Journal of the Franklin Institute 334, 307 (1997).
  * Nielsen and Nock (2011) F. Nielsen and R. Nock, ArXiv abs/1105.3259 (2011).
  * Tsallis (1988) C. Tsallis, Journal of statistical physics 52, 479 (1988).
  * Le Cam (2012) L. Le Cam,  _Asymptotic methods in statistical decision theory_ (Springer Science & Business Media, 2012).
  * Aldous (2019) D. Aldous, The Mathematical Intelligencer 41, 90 (2019).
  * Ilić and Stanković (2014) V. M. Ilić and M. S. Stanković, Physica A: Statistical Mechanics and its Applications 411, 138 (2014).
  * Suyari (2004) H. Suyari, IEEE Transactions on Information Theory 50, 1783 (2004).
  * Nielsen and Chuang (2010) M. A. Nielsen and I. L. Chuang,  _Quantum Computation and Quantum Information: 10th Anniversary Edition_ (Cambridge University Press, 2010).
  * Luo and Zhang (2004) S. Luo and Q. Zhang, Phys. Rev. A 69, 032106 (2004).
  * Müller-Lennert  _et al._ (2013) M. Müller-Lennert, F. Dupuis, O. Szehr, S. Fehr, and M. Tomamichel, Journal of Mathematical Physics 54, 122203 (2013).
  * Rajagopal  _et al._ (2014) A. K. Rajagopal, Sudha, A. S. Nayak, and A. R. U. Devi, Phys. Rev. A 89, 012331 (2014).