# Local magnetic moments due to loop currents in metals

Arkady Shekhter<EMAIL_ADDRESS>Los Alamos National Laboratory, Los
Alamos, New Mexico 87545, USA  Chandra M. Varma<EMAIL_ADDRESS>Physics
Department, University of California, Berkeley, California 94704, USA
Recalled Professor  Physics Department, University of California, Riverside,
California 92521, USA  Emeritus Distinguished Professor

###### Abstract

We present Hartree-Fock calculations on a simple model to obtain the
conditions of formation of local magnetic moments due to loop-currents $L_{o}$
and spin-loop currents $L_{s}$ and compare them to the conditions of formation
of local spin-moments $M$ which were given long ago in a similar approximation
by Anderson. A model with three degenerate orbitals sitting on an equilateral
triangle, with on-site and nearest-neighbor repulsions $U$ and $V$
respectively, and inter-site kinetic energy, hybridizing with conduction
electrons with a parameter $Δ$ is investigated. $L_{o}$ and $L_{s}$ are
promoted by large $V/Δ$ and their magnitude is relatively unaffected by $U/Δ$.
Spin-magnetic moments $M$ promoted by large $U/Δ$ on the other hand are
adversely affected by $V/Δ$. In this model, $L_{o}$ for $V$ multiplied by the
number of neighbors is approximately the same as the $M$ promoted by $U$ in
Anderson’s local model for $M$. $L_{o}$ and $L_{s}$ are degenerate if exchange
interactions and Hund’s rule are neglected but $L_{o}$ is favored when they
are included. Many of the qualitative results are visible in an expression for
the Hartree-Fock ground state energy derived as a function of small
$L_{o},L_{s}$ and $M$. Numerical minimization of the Hartree-Fock energy is
presented for larger values. We also briefly discuss the connection and
differences of the interaction generated orbital currents and spin-currents
discussed here and generalized to a lattice with the topological states in
metals and semi-conductors.

Today

##  I Introduction

A number of metallic compounds and some magnetic insulators have presented
evidence recently of broken time-reversal and associated changes in lattice
symmetry associated with loop-currents. These include the cuprate high
temperature superconductors Bourges2021 ; Kaminski-diARPES ; Hsieh-SHG-Srcuocl
; Hsieh2017  in their so-called pseudogap phase and some metallic kagome
compounds Kagome_STM_2021 ; Kagome_Kerr2022 . The insulating state in cuprates
has also shown evidence for such states Hsieh-SHG-Srcuocl  as well as the
closely related insulating and conducting states in Sr2(Ir-Rh)O4 HsiehSrRh ;
BourgesSr2Ir . For the cuprates, such states were predicted cmv1997 ; simon-
cmv  to occur at the boundary to the marginal Fermi-liquid CMV-MFL ; Aji-V-
qcf1  phase with its $T$-linear resistivity Varma-rmp2020  and singular Fermi-
liquid Varma_2016  anomalies in other properties. Such states have also been
predicted for bi-layer graphene Zhu-A-V2013  and a closely related state has
been suggested for twisted bi-layer graphene Zaletel2020 ; Berg2021 . The
orbital moments due to loop-currents are hard to detect experimentally. It is
quite likely that many other metals and correlated insulators also have such
moments. For example, the recently discovered Kalisky2022  spontaneous vortex
state in the superconducting state in layers of TaS2 with field applied and
then removed below a specific temperature in the normal state may be due to
such a normal state. If described by a time-reversal breaking vector order
parameter such states are expected to have mobile vortices in the vector field
generated by an applied magnetic field CMV2022_R_H .

Much insight was gained in understanding metallic spin-magnetism through first
understanding the formation of local magnetic moments Friedel_Magmom ; PWA1961
on d-orbitals with a large local on-site interaction compared to the
hybridization energy with non-interacting s-electrons. One can then study
long-range order and other phenomena by considering interactions between the
local moments. This is a strong coupling approach, as distinct from the
Stoner-type weak-coupling theory. In this paper, we investigate the formation
of magnetic moments through loop-current order in the simplest system that can
support such states: three degenerate states on a triangle of sites with on-
site and nearest neighbor interactions hybridizing with a continuum of non-
interacting states. We do this by following the Hartree-Fock method used by
Anderson in the local spin-moment problem PWA1961 . Although our calculation
is instructive, there are several other more subtle features in the
correlations in the loop-current states in a lattice. We refer the reader to
the results of a detailed variational Monte Carlo calculation Weber-Giam-V
for them.

It should be appreciated that the magnitude of the orbital moment can be
considerable. Suppose there is an electron going around in a circle of radius
$a$ and speed $v$. Its classical magnetic moment is
$M=πa^{2}\leavevmode\nobreak\ \mathcal{I}$, where $\mathcal{I}$ is the current
going around the wire. If we take that there are $n$ electrons in the wire
(i.e., in its volume $(2πa)\,(πr_{0}^{2})$, where $r_{0}$ is the radius of the
cross-section of the wire), $M=(1/2)n(v/c)\,ea$ which for $n=1,a=2\text{Å}$
and $v=10^{8}$ cm/sec, the velocity in graphene, is about $1.5×10^{-20}$
ergs/Gauss, i.e., close to $1.5\leavevmode\nobreak\ μ_{\text{B}}$. The
velocity will often be lower and the density of electrons in the loop-current
order may be much less than 1 per unit-cell. In polarized neutron scattering
experiments in cuprates Bourges2021 , a magnitude of (staggered moment) of
about $0.1μ_{\text{B}}$ per unit-cell is deduced. The orbital moments are
generated by condensing part of the spatially dependent orbital magnetic
susceptibility. The integral of the spectral weight over frequency and momenta
of the orbital magnetic fluctuations in the incoherent weight in a
noninteracting model of metals is comparable to a similar quantity for spin
fluctuations.

##  II The Model

We consider the model given by the Hamiltonian

$\displaystyle H=H_{s}+H_{d}+H_{s-d}\,.$ (1)

$H_{s}$ is the Hamiltonian for free electrons

$\displaystyle H_{s}=∑_{{\bf k}σ}(ϵ_{{\bf k}σ}-μ)c^{+}_{{\bf k}σ}c_{{\bf
k}σ}\,.$ (2)

$H_{d}$ describes three orbitals with Kramers’ degeneracy on three atoms
sitting on an equilateral triangle immersed in a metal of non-interacting
electrons. It consists of the noninteracting part $H_{d}^{\prime}$ and the
part $H_{d}^{{}^{\prime\prime}}$ with onsite and nearest-neighbor
interactions,

$\displaystyle H_{d}^{\prime}=$ $\displaystyle
ϵ_{d}∑_{i=1,2,3,σ}d_{iσ}^{+}d_{iσ}+t_{dd}∑_{i}d^{+}_{iσ}d^{+}_{i+1σ}+h.c.,$
(3) $\displaystyle H_{d}^{{}^{\prime\prime}}=$ $\displaystyle
U∑_{i}n_{i\textuparrow}n_{i\textdownarrow}+V∑_{i}n_{i}n_{i+1}\,,$ (4)

where $n_{i}=n_{i\textuparrow}+n_{i\textdownarrow}$ and
$n_{iσ}=d_{iσ}^{+}d_{iσ}$. $H_{sd}$ is the hybridization

$\displaystyle H_{sd}=∑_{i\bf kσ}t_{i{\bf k}}d_{iσ}^{+}c_{\bf kσ}+h.c.$ (5)

The problem is best discussed in the basis of the eigenstates of angular
momenta defined with respect to the center of the triangle

$\displaystyle D_{ℓσ}=\frac{1}{\sqrt{3}}∑_{n}d_{nσ}e^{iω_{ℓ}n}.$ (6)

where $ℓ=0,±1$ and $ω_{ℓ}=\frac{2π}{3}ℓ$. When an electron is in the state
$ℓ=±1$, the triangle carries a clockwise or anticlockwise loop current. In the
new basis

$\displaystyle H_{sd}=∑_{ℓ}t_{ℓ}D^{+}_{ℓσ}c_{ℓk,σ}+h.c.$ (7)

conserving both angular and spin momenta. $c_{ℓk,σ}$ are operators for hole
states with angular momenta $ℓ=0,±1$ with respect to the center of the
triangle. The Hamiltonian $H_{d}^{\prime}$ is diagonal in the angular momentum
basis

$\displaystyle H_{d}^{\prime}=∑_{ℓσ}ϵ_{dℓ}n_{ℓσ}.$ (8)

The energy levels, including the shift due to the hybridization with the
Fermi-sea, are $ϵ_{dℓ}=ϵ_{d}+2t_{d}\cos ω_{ℓ}$. Note that the relative
positions of the singlet $ℓ=0$ level and the doublet $ℓ=±1$ are given by the
sign of $t_{d}$. We will assume that the doublet is below the singlet, as is
the case for $t_{d}>0$.

Finally we write the interactions in the new basis. To demonstrate the basic
idea, we first consider spinless electrons. In this case, the interactions in
the angular momentum basis are

$\displaystyle
H_{d}{{}^{\prime\prime}}=Vn_{0}(n_{+1}+n_{-1})+Vn_{+1}n_{-1}\,.$ (9)

The first term serves to shift the energy difference between the orbital
singlet and the orbital doublet. The last term introduces the repulsion
between $ℓ=±1$ states in the Hartree approximation. If this were the only
interaction term in the model, the problem would have a one-to-one
correspondence with the Hartree-Fock repulsion between up- and down-spins due
to the on-site interactions in Anderson’s calculation PWA1961 .

We now go back to the spinful model, which is a bit more involved. We must
consider the Hartree-Fock energy $E_{ℓ,σ}$ of the states
($ℓ=0,σ=±1);(ℓ=±1,σ=±1).$ Terms in $E_{ℓ,σ}$ that are linear in $n_{ℓσ}$ give
the bare energies, and those due to interactions are quadratic in $n_{ℓ,σ}$.
$n_{ℓσ}$ are determined variationally from the Hartree-Fock ground state
energy $E_{0}({n_{ℓσ}})$ which can be written as the sum of a potential energy
$P({n_{ℓσ}})$ and a kinetic energy $K({n_{ℓσ}})$:

$\displaystyle E_{0}(n_{ℓσ})=P(n_{ℓσ})+K(n_{ℓσ})\,.$ (10)

For a constant density of states of conduction electrons $ν$ per spin, the
kinetic energy is, just as in Anderson’s calculations

$\displaystyle K({n_{ℓσ}})=-∑_{ℓσ}\frac{Δ_{ℓ}}{π}\ln\sin(πn_{ℓσ})\,.$ (11)

$Δ_{ℓ}=π|t_{ℓ}|^{2}ν$ is the width of the levels due to hybridization with the
itinerant electrons.

The (coupled) variational equations are

$\displaystyle Δ_{ℓ}\cot(πn_{ℓσ})=\frac{∂P(n_{ℓσ})}{∂n_{ℓσ}}.$ (12)

The potential energy may be written as the part for $n_{0σ}$ and the part for
$n_{1σ}$:

$\displaystyle P($ $\displaystyle n_{0σ})=∑_{σ}ϵ_{d0}n_{0σ}$ $\displaystyle+$
$\displaystyle\frac{U+2V}{3}n_{0σ}N_{-σ}+Vn_{0σ}N_{σ}+\frac{U+2V}{6}n_{0σ}n_{0-σ},$
(13) $\displaystyle P($ $\displaystyle n_{±1σ})=∑_{σ}ϵ_{d1}N_{σ}$
$\displaystyle\qquad+\frac{U+2V}{6}N_{σ}N_{-σ}+Vn_{1σ}n_{-1σ}\,,$ (14)

where $N_{σ}≡(n_{1σ}+n_{-1σ})$.

We introduce ground state expectation values

$\displaystyle n≡∑_{σ}\left\langle n_{0σ}\right\rangle\,,\qquad
m≡∑_{σ}\left\langle σn_{0σ}\right\rangle\,,$ (15)

as the occupation and total spin-moment of the state with $ℓ=0$. We also
define

$\displaystyle N≡$ $\displaystyle∑_{ℓσ}\left\langle
n_{ℓσ}\right\rangle\,,\qquad\;\;M≡∑_{ℓσ}σ\left\langle n_{ℓσ}\right\rangle,$
$\displaystyle L_{o}≡$ $\displaystyle∑_{ℓσ}ℓ\left\langle
n_{ℓσ}\right\rangle\,,\qquad L_{s}≡∑_{ℓσ}ℓσ\left\langle
n_{ℓσ}\right\rangle\,,$ (16)

as, respectively, the total occupation, the total spin, the total orbital
moment and the total spin-current.

To simplify the analysis, let us consider the case where the $ℓ=0$ level is
well above the chemical potential and therefore $n_{0σ}=0$. (Similar results
pertain when it is well below the chemical potential and is filled with both
spins.) The potential energy then is

$\displaystyle P(N,M,L_{o},L_{s})=ϵ_{d1}N$ $\displaystyle+$
$\displaystyle\frac{1}{8}\left[\frac{2U+7V}{3}N^{2}-\frac{2U+V}{3}M^{2}-V(L_{o}^{2}+L_{s}^{2})\right]\,.$
(17)

To get an idea of the different phases as a function of the parameters, let us
fix the total occupation $N$ and expand the kinetic energy as a function of
small $M,L_{o}$ and $L_{s}$. This gives

$\displaystyle\frac{K}{Δ}=c_{0}+c_{2}(M^{2}+L_{o}^{2}+L_{s}^{2})+c_{3}ML_{o}L_{s}$
$\displaystyle+$ $\displaystyle
c_{4}(M^{4}+L_{o}^{4}+L_{s}^{4}+6L_{o}^{2}M^{2}+6L_{s}^{2}M^{2}+6L_{o}^{2}L_{s}^{2}),$
(18)

where

$\displaystyle c_{0}=$ $\displaystyle-\frac{1}{π}\ln\sin\frac{πN}{4},\qquad
c_{2}=\frac{π}{8}\csc\frac{πN}{4}\,,$ $\displaystyle c_{3}=$
$\displaystyle-\frac{π^{2}}{8}\cot\frac{πN}{4}\csc^{2}\frac{πN}{4}\,,$
$\displaystyle c_{4}=$
$\displaystyle\frac{1}{24}\;\frac{π^{3}}{16}\;\;\frac{\frac{1}{2}+\cos^{2}\frac{πN}{4}}{\sin^{4}\frac{πN}{4}}\,.$
(19)

Several noteworthy features are already apparent. Obviously the potential
energy favors non-zero values of $M,L_{o}$ and $L_{s}$ while the kinetic
energy favors $0$. In the potential energy as well as the kinetic energy, one
can interchange $L_{o}$ and $L_{s}$. Potential energy allows both
$\sqrt{L_{o}^{2}+L_{s}^{2}}$ and $M$ to be non-zero. Since $c_{4}>0$, the
kinetic energy favors either $M$ or $L_{o}$ or $L_{s}$. The degeneracy between
$L_{o}$ and $L_{s}$ together with competition with non-zero $M$ from the cubic
term proportional to $c_{3}$, which has a positive sign above for $2<N<4$
introduces interesting features in numerical calculations. The degeneracy is
removed as discussed below when we introduce exchange interactions.

In the vicinity of empty, $N≈0$, and full, $N≈4$, $c_{2}$ is very large and
therefore $M,L_{o}$, and $L_{s}$ are all $0$. Near half filling, $N≈2$,
$c_{2}$ is near its minimum value $1/2$. So this is the most favorable value
for all the polarizations. Near this value of $N$, $c_{4}≈1/10$ and $c_{3}≈0$.
Comparing coefficients in Eqs. (II) and (II), one easily finds that for
$V>c_{2}Δ$, and $(2U+V)/3<c_{2}Δ$, the bi-linear terms favor non-zero loop-
currents $L_{o},L_{s}$ while $M=0$. At precisely $N=2$, where $c_{3}=0$, we
get $L_{o}^{2}=\frac{2}{5}(V/Δ-c_{2})/c_{4}$. Away from half filling, cubic
terms allow “first-order” changes in the values of $M,L_{o}$, and $L_{s}$.
Magnetic moments enter linearly in the cubic terms and lead to $M∝L^{2}$.

For small values of $c_{4}$, values of $M,L_{o}$, and $L_{s}$ quickly reach
$1$ with increasing $U/Δ,V/Δ$ and the expansion above becomes invalid. We
therefore have performed a numerical minimization of the Hartree-Fock energy
for fixed $N$ and varying $M,L_{o}$, and $L_{s}$. The results are shown in
Figs. 1 and 2.

###  II.1 Orbital moment due to loop-current $L_{o}$ versus spin-loop current
$L_{s}$

In the Hartree-Fock effective Hamiltonian of the model with energy given by
Eq. (10) and subsequent equations following from it, there is an “accidental”
symmetry, which is also manifest in the low order parameter expansion for the
energy, Eqs. (II) and (II). The accidental character of this symmetry is
obvious because the state with finite $L_{o}$ has the same energy as the state
with finite $L_{s}$ despite the fact that these states have different
symmetries — $L_{o}$ is odd in time reversal while $L_{s}$ is even. The
symmetry of the energy under the exchange of orbital and spin currents,
$L_{o}$ and $L_{s}$ is introduced due to the de-couplings of the biquadratic
operators of spin and angular momentum in the Hartree-Fock approximation which
in an exact theory would not be allowed.

This matter is resolved by including exchange interactions. In the limit of
small inter-site repulsion and large on-site repulsion, $V<t≪U$ we can include
the exchange Hamiltonian,

$\displaystyle H_{\text{exch}}=J({\bf S}_{1}⋅{\bf S}_{2}+{\bf S}_{2}⋅{\bf
S}_{3}+{\bf S}_{3}⋅{\bf S}_{1})$ (20)

where the subscripts are site indices. The value of the exchange
$J≈{2t^{2}}/({U-V})$. Neglecting the $ℓ=0$ state, the exchange Hamiltonian in
the orbital states basis is

$\displaystyle H_{\text{exch}}=-J{\bf S}_{ℓ=-1}⋅{\bf S}_{ℓ=1}$ (21)

This includes the fact that when a pair of electrons are in the same orbital,
the Pauli principle ensures a spin-singlet state and when they are in
different orbitals, Hund’s rule favors a triplet state. We lose the $SU(2)$
symmetry by the Hartree-Fock approximation on (21) and write it in terms of
only z-components of spin as done above for the rest of the interactions. This
exchange energy shifts the expectation values of the energy of the states
described in terms of occupation numbers Eq. (10), $H_{\text{exch-
HF}}=-\tilde{J}(M^{2}-L_{s}^{2})$, where $\tilde{J}=J/4$ and
$L_{s}=\left\langle S_{ℓ=1}\right\rangle-\left\langle S_{ℓ=-1}\right\rangle$,
$M=\left\langle S_{ℓ=1}\right\rangle+\left\langle S_{ℓ=-1}\right\rangle$.
Equation (20) applies only for $V<t≪U$ where the formation of orbital current
is not favorable. In the opposite limit of large inter-site repulsion $V$, the
loop-orbital states $ℓ=±1,0$, rather than the localized site-states on the
triangle (Eq. 20), are a good starting point, as discussed above. The local
interaction Eq. (4) contains non-Hartree-Fock terms
$δH_{2}=(V-U)/6\,[S_{x,-1}S_{x,+1}+S_{y,-1}S_{y,+1}]$ omitted in the Hartree-
Fock energy Eq. (14). These can be rewritten as $δH_{2}=(V-U)/6\,[{\bf
S}_{-1}{\bf S}_{+1}-S_{z,-1}S_{z,+1}]$, which can be estimated as
$δE_{2}=J\,[(1/4)M^{2}+(1/4)J_{s}^{2}]$. The first term can be absorbed in the
existing expression for the energy, Eq. (II). The second term,

$\displaystyle H_{\text{exch-HF}}=\tilde{J}L_{s}^{2}\,,$ (22)

lifts the accidental degeneracy in the Hartree-Fock approximation between
states with nonzero orbital loop current or spin current states and it favors
the orbital magnetic moment over the spin-current moment. In the limit of
large $V$, parameter $J$ has to be found from a complete SU(2) analysis.

##  III Numerical results

The numerical calculations are also done with the $ℓ=0$ level far above the
chemical potential so that it is un-occupied. In Fig. (1), $ϵ_{d}/Δ=-5$ is
chosen so that the non-interacting triangle would have both $ℓ=±1$ fully
occupied and the ground state has four electrons. The panel on the left shows
the ground state charge $N$ as a function of $U/Δ$ and $V/Δ$. As expected, the
number goes down with these repulsions. The interesting thing is the stability
of $N=2$ over a large range of the interaction parameters. This can be traced
to the reduction in energy due to the maximum possible development of the spin
moment or orbital moment for $N=2$. The spin moment $M$, the orbital moment
$L_{o}$, and the spin current are shown in plots with varying $U/Δ$ and $V/Δ$.
The spin-moment, shown by the figure in the top right using a small $V$,
follows Anderson’s result PWA1961 , except that the value of $M$ is twice
larger because of the orbital degeneracy. The decrease in the local moment at
large $V$ and the “almost-steps” in the topography should be noted; this may
be attributed to lifting of the orbital degeneracy by $V$ and the consequent
decrease in $N$ noted in the top left-hand panel. But for small $U$, a weak
growth of the spin moment with $V$ is also noteworthy.

Figure 1: Top left: Total charge $N$ as a function of interaction $U/Δ$ and
$V/Δ$ for the orbitally degenerate non-interacting levels with $ϵ/Δ$ such that
$N$ would be $4$ for non-interacting electrons. Top right: Spin magnetic
moment as a function of the interaction parameters $U/Δ$ and $V/Δ$ for the
same $ϵ/Δ$ as in the top left. The bottom left and right present
$\sqrt{L^{2}_{o}+L^{2}_{s}}$ for the case where the exchange interaction
$J=0$. As explained in the text, numerical calculations give either $L_{s}\neq
0$ or $L_{o}\neq 0$ due to numerical noise but without any ambiguity for
$\sqrt{L^{2}_{o}+L^{2}_{s}}$. Including $J\neq 0$ removes the degeneracy and
favors $L_{o}$. That value is shown in the bottom right. Figure 2: Left: The
top figure shows the total charge for a modest value of $U/Δ$ in the $ϵ_{d}/V$
and $V/Δ$ plane, while the bottom figure shows the same for a fixed modest
$V/Δ$ in the $ϵ_{d}/U$ and $U/Δ$ plane. Right: The top figure shows the result
for $J=0$ when $L_{s}$ and $L_{o}$ are degenerate in our approximation. The
bottom shows $L_{o}$ for finite $J$ for which $L_{s}=0$.

We first show the calculations for $L_{o}$ and $L_{s}$ without including the
exchange interaction parameterized by ${\tilde{J}}$. As discussed in the
previous section, the Hartree-Fock approximation for ${\tilde{J}}=0$
introduces an “accidental” degeneracy between $L_{o}$ and $L_{s}$. This is
seen in the numerical results which give the state realized to be either
$L_{o}\neq 0$ with $L_{s}=0$ or vice-versa depending on the numerical noise.
One or the other is favored due to the repulsions seen already in the low
moment expansions of the energy above in Eq. (II) due to the cubic term. On
the other hand we find no ambiguity when we plot $\sqrt{L_{o}^{2}+L_{s}^{2}}$.
This is what is shown in the bottom left in Fig. (1) which is for
${\tilde{J}}=0$. On the bottom right, we plot the results for $\tilde{J}$ with
its value fixed by the other parameters as in Eq. (21). $L_{o}$ is favored,
although numerical noise appears to give also a much smaller value of $L_{s}$
for some range of parameters.

The figures at the bottom right in Fig. (1) gives the orbital magnetic moment
with a maximum amplitude $1$ at large $V$ corresponding to the complete
lifting of the degeneracy of the $ℓ=±1$ state. The value of $V$ required for
this is about 1/2 the value of $U$ for a maximum spin moment at small $V$, due
to the fact that the effective value of $V$ to generate the orbital moment
gets multiplied by the number of neighbors, two in this case. One finds that
the modest value of $U$ does not decrease the orbital moment. This is due to
the fact that both a spin-loop-current and a charge-loop-current are favored
for large $V$. At large values of $U$, the change of $N$ is responsible for
the decrease of $L_{o}$ in a step-like fashion. This is due to the fact that a
charge-loop-current is favored for large $V$. We have already discussed
following Eq. (II) the competition of $L$ with $M$ due to the fourth order
repulsive terms coupling $L_{o}$ and $M$ proportional to $c_{4}$. Further
details of the numerical results obtained are illustrated in Fig. 2.

##  IV General remarks

For simplicity we considered atoms on an equilateral triangle so that the
unperturbed states could be labeled by their angular momentum. Then one need
only consider interactions which lift the Kramers’ degeneracy between the $±1$
angular momentum so that if the chemical potential is not above the newly
diagonalized states, a moment due to loop-current is realized. This also
stresses the connection of the sign of the product of the transfer integral
between the three atoms and the average density of particles to have the
orbital moment.

These considerations can be generalized to “molecules” with different
symmetries. For problems in the lattice, the patterns which are realized must
satisfy the condition that the currents are conserved at each site. In a mean-
field approximation, the local order parameter $\left\langle C\right\rangle$
added to a real kinetic energy operator corresponds to a complex hopping
between $i$ and $j$ for $\left\langle C\right\rangle$ purely real and
therefore a current for $σ=σ^{\prime}$ and a spin-current for $σ\neq
σ^{\prime}$. For $\left\langle C\right\rangle$ purely imaginary, it represents
an excitonic charge transfer which would in general couple also to the lattice
and lead to a structural transformation. The identity can be transformed to
momentum space and one could do mean-field theory in the weak-coupling
approximation to look for all such instabilities. This was the procedure used
in Ref. simon-cmv  for cuprates.

The Hartree-Fock calculations, although revealing, are hardly the complete
story for the conditions of occurrence of orbital moments in a lattice. These
are considered in more detail in Refs. Weber-Giam-V ; Weber-Mila . A primary
consideration is that for a metallic state to exist requires charge
fluctuations and these cost energy due to repulsive interactions which are
usually ameliorated by screening, generally by relatively un-correlated $s-p$
electrons which in turn cost kinetic energy for them. The s-p electrons are
often not in the models and sometimes, as in the cuprates and other transition
metals to the right of the periodic table, too far in the majority of their
density of states above the chemical potential to be effective. A loop-current
ordered state allows a coherent screening within the correlated states
reducing the kinetic energy to screen. A significant next nearest neighbor
interaction, necessary for loop-current states, also requires the absence of
significant effects of $s-p$ electrons in actual materials.

There are two other possible states in the triangle when the total charge is
$2$: One is an in-homogeneous threefold degenerate state in which two of the
sites are occupied and the third is unoccupied. This should probably be called
a “Kekule” structure. With interactions eliminated to $O(t/(U,V)$ this state
is further stabilized by the exchange energy so that a singlet bond is formed
on a pair of sites with the third site unoccupied. Such a state would also
persist for $N$ close to $2$. Consideration of such inhomogeneous states
requires considering the $ℓ=0$ state which we have neglected. The spin-current
state discussed is also likely to be modified if exchange interactions are
considered. The state is one in which the Kekule structure is a linear
combination of spin-singlet bonds in the three pairs of sites which should be
appropriately called a “resonating valence bond” state on a triangle is
related to the orbital current state investigated here but for one important
difference. Since $SU(2)$ spin-symmetry must be maintained for such a state, a
simple Hartree-Fock approximation which is the basis of the results in this
paper is not appropriate. Retaining $SU(2)$ symmetry is also essential for a
proper theory of the spin-current state in a model including exchange
interactions.

Finally, we discuss the connection of the loop-currents generated by
interactions in the lattice version Varma-rmp2020  of the physical states such
as discussed in this paper, with the topological states with anomalous Hall
effects Haldane1988 , or anomalous spin-Hall effects KaneMele2005 ; NiuRMP2010
. Both kinds of effects depend on having a lattice with a basis (whether in
the lattice already or after the symmetry breaking induced by the
interactions.) The latter depend crucially on the topological effects of the
periodicity of the Brillouin zone in more than one dimension, and the former
do not. The Haldane state Haldane1988  on a hexagonal lattice can be realized
only through loop currents due to next nearest neighbor interactions, as
explained in Ref. Zhu-A-V2013  but it has the additional feature that
inversion is preserved. Alternate loop-currents on the hexagonal lattice
investigated in the same paper are also promoted by interactions but they
break inversion. Then the anomalous Hall effect cannot occur. The anomalous
spin-Hall currents require spin-orbit coupling which at least in non-
relativistic physics should be thought of as an intra-atomic interaction
effect and some specific conditions on symmetry at special points in the zone
of the conduction and valence band states. However, the spin-currents
discussed here together with spin-orbit couplings lead to the same physics if
those conditions are satisfied NiuRMP2010 . In general, the role of electron-
electron interactions in stabilizing orbital magnetism is less obvious in the
topological arguments in momentum space than in the real space analysis which
has been the emphasis in this paper.

Acknowledgements: Work at Los Alamos is supported by the NSF through
DMR-1644779 and the U.S. Department of Energy. A.S. acknowledges support from
the DOE/BES “Science of 100 T” grant. This work was performed in part at Aspen
Center for Physics, which is supported by National Science Foundation grant
No. PHY-1607611.

## References

  * (1) P. Bourges, D. Bounoua, and Y. Sidis, _Comptes Rendus Physique_ 22, 7 (2021).
  * (2) A. Kaminski, S. Rosenkranz, H. M. Fretwell, J. C. Campuzano, Z. Li, H. Raffy, W. G. Cullen, H. You, C. G. Olson, C. M. Varma, and H. Hochst, _Nature_ 416, 610 (2002).
  * (3) A. de la Torre, K. L. Seyler, L. Zhao, S. Di Matteo, M. S. Scheurer, Y. Li, B. Yu, M. Greven, S. Sachdev, M. R. Norman, and D. Hsieh, _Nature Physics_ 17, 777 (2021).
  * (4) L. Zhao, C. A. Belvin, R. Liang, D. A. Bonn, W. N. Hardy, N. P. Armitage, and D. Hsieh, _Nature Physics_ 13, 250 (2016).
  * (5) K. Jiang, T. Wu, J.-X. Yin, Z. Wang, M. Z. Hasan, S. D. Wilson, X. Chen, and J. Hu, National Science Review, nwac199 (2021).
  * (6) Y. Xu, Z. Ni, Y. Liu, B. R. Ortiz, S. D. Wilson, B. Yan, L. Balents, L. Wu, arXiv:2204.10116, (2022).
  * (7) L. Zhao, D. H. Torchinsky, H. Chu, V. Ivanov, R. Lifshitz, R. Flint, T. Qi, G. Cao, and D. Hsieh, _Nature Physics_ 12, 32 (2016).
  * (8) J. Jeong, Y. Sidis, A. Louat, V. Brouet, and P. Bourges, _Nature Communications_ 8, 15119 (2017).
  * (9) C. M. Varma, _Phys. Rev. B_ 55, 14554 (1997).
  * (10) M. E. Simon and C. M. Varma, _Phys. Rev. Lett._ 89, 247003 (2002).
  * (11) C. M. Varma, P. B. Littlewood, S. Schmitt-Rink, E. Abrahams, and A. E. Ruckenstein, _Phys. Rev. Lett._ 63, 1996 (1989).
  * (12) V. Aji and C. M. Varma, _Phys. Rev. Lett._ 99, 067003 (2007).
  * (13) C. M. Varma, _Rev. Mod. Phys._ 92, 031001 (2020).
  * (14) C. M. Varma, _Reports on Progress in Physics_ 79, 082501 (2016).
  * (15) L. Zhu, V. Aji, and C. M. Varma, _Phys. Rev. B_ 87, 035427 (2013).
  * (16) N. Bultinck, E. Khalaf, S. Liu, S. Chatterjee, A. Vishwanath, and M. P. Zaletel, _Phys. Rev. X_ 10, 031034 (2020).
  * (17) J. S. Hofmann, E. Khalaf, A. Vishwanath, E. Berg, and J. Y. Lee, arXiv:2105.12112 (2021).
  * (18) E. Persky, A. V. Bjørlig, I. Feldman, A. Almoalem, E. Altman, E. Berg, I. Kimchi, J. Ruhman, A. Kanigel, and B. Kalisky, _Nature (London)_ 607, 692 (2022).
  * (19) C. M. Varma, _Phys. Rev. Lett._ 128, 206601 (2022).
  * (20) J. Friedel, Canadian _Journal of Physics_ 34, 1190 (1956).
  * (21) P. W. Anderson, _Phys. Rev._ 124, 41 (1961).
  * (22) C. Weber, T. Giamarchi, and C. M. Varma, _Phys. Rev. Lett._ 112, 117001 (2014).
  * (23) C. Weber, A. Lauchli, F. Mila, and T. Giamarchi, _Phys. Rev. Lett._ 102, 017005 (2009).
  * (24) F.D.H. Haldane, _Phys. Rev. Lett._ 61, 2015 (1988).
  * (25) C.L. Kane and E.J. Mele, _Phys. Rev. Lett._ 93, 197402 (2005).
  * (26) D. Xiao, M.-C. Chang and Q. Niu, _Rev. Mod. Phys._ 82, 1959 (2010)


# On Sibson’s $\alpha$-Mutual Information

Amedeo Roberto Esposito, Adrien Vandenbroucque, Michael Gastpar  School of
Computer and Communication Sciences
EPFL, Lausanne, Switzerland
{amedeo.esposito<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

We explore a family of information measures that stems from Rényi’s
$\alpha$-Divergences with $\alpha<0$. In particular, we extend the definition
of Sibson’s $\alpha$-Mutual Information to negative values of $\alpha$ and
show several properties of these objects. Moreover, we highlight how this
family of information measures is related to functional inequalities that can
be employed in a variety of fields, including lower-bounds on the Risk in
Bayesian Estimation Procedures.

###### Index Terms:

Rényi-Divergence, Sibson’s Mutual Information, Information Measures, Bayesian
Risk, Estimation

##  I Introduction

Sibson’s $\alpha$-Mutual Information is a generalization of Shannon’s Mutual
Information with several applications in probability, information and learning
theory [1]. In particular, it has been used to provide concentration
inequalities in settings where the random variables are not independent, with
applications to learning theory [1]. The measure is also connected to
Gallager’s exponent function, a central object in the channel coding problem
both for rates below and above capacity [2, 3]. Moreover, a new operational
meaning has been given to the measure with $\alpha\\!=\\!+\infty$ when a novel
measure of information leakage has been proposed in [4], under the name of
Maximal leakage. Similarly to $I_{\alpha}$, Maximal Leakage has recently found
applications in learning and probability theory [1]. In this work we will look
at some properties of $I_{\alpha}$ with a different perspective and we will
extend the definition in order to include negative values of $\alpha$ as well.
The reason for this extension is strongly operationally driven and will be
tied to a family of functional-analytic bounds that we can provide for
Sibson’s $\alpha$ MI for every $\alpha\in\mathbb{R}$.

##  II Background and definitions

Throughout the paper we will often use the notion of $L^{p}$-norms: let $1\leq
p\leq\infty$ and consider the measurable space $(\Omega,\mathcal{F},\mu)$, let
$f$ be a measurable function with respect to the space, then

$\left\lVert
f\right\rVert_{L^{p}(\mu)}=\left(\int|f|^{p}d\mu\right)^{\frac{1}{p}}.$ (1)

The definition can be extended to values of $p<1$, however in those cases one
does not have norms anymore e.g., if $0<p<1$ one recovers a quasi-norm but not
a norm.

###  II-A Sibson’s $\alpha$-Mutual Information

Introduced by Rényi as a generalization of entropy and KL-divergence,
$\alpha$-divergence has found many applications ranging from hypothesis
testing to guessing and several other statistical inference and coding
problems [5]. Indeed, it has several useful operational interpretations (e.g.,
hypothesis testing, and the cut-off rate in block coding [6, 7]). It can be
defined as follows [6]:

######  Definition 1.

Let $(\Omega,\mathcal{F},\mathcal{P}),(\Omega,\mathcal{F},\mathcal{Q})$ be two
probability spaces. Let $\alpha>0$ be a positive real number different from
$1$. Consider a measure $\mu$ such that $\mathcal{P}\ll\mu$ and
$\mathcal{Q}\ll\mu$ (such a measure always exists, e.g.
$\mu=(\mathcal{P}+\mathcal{Q})/2$) and denote with $p,q$ the densities of
$\mathcal{P},\mathcal{Q}$ with respect to $\mu$. The $\alpha$-Divergence of
$\mathcal{P}$ from $\mathcal{Q}$ is defined as follows:

$\displaystyle D_{\alpha}(\mathcal{P}\|\mathcal{Q})=\frac{1}{\alpha-1}\log\int
p^{\alpha}q^{1-\alpha}d\mu.$ (2)

######  Remark 1.

The definition is independent of the chosen measure $\mu$. It is indeed
possible to show that $\int
p^{\alpha}q^{1-\alpha}d\mu=\int\left(\frac{q}{p}\right)^{1-\alpha}d\mathcal{P}$,
and that whenever $\mathcal{P}\ll\mathcal{Q}$ or $0<\alpha<1,$ we have $\int
p^{\alpha}q^{1-\alpha}d\mu=\int\left(\frac{p}{q}\right)^{\alpha}d\mathcal{Q}$,
see [6].

It can be shown that if $\alpha>1$ and $\mathcal{P}\not\ll\mathcal{Q}$ then
$D_{\alpha}(\mathcal{P}\|\mathcal{Q})=\infty$. The behaviour of the measure
for $\alpha\in\\{0,1,\infty\\}$ can be defined by continuity. In general, one
has that $D_{1}(\mathcal{P}\|\mathcal{Q})=D(\mathcal{P}\|\mathcal{Q})$ but if
$D(\mathcal{P}\|\mathcal{Q})=\infty$ or there exists $\beta$ such that
$D_{\beta}(\mathcal{P}\|\mathcal{Q})<\infty$ then $\lim_{\alpha\downarrow
1}D_{\alpha}(\mathcal{P}\|Q)=D(\mathcal{P}\|\mathcal{Q})$[6, Theorem 5]. For
an extensive treatment of $\alpha$-divergences and their properties we refer
the reader to [6]. Starting from Rényi’s Divergence and the geometric
averaging that it involves, Sibson built the notion of Information Radius [8]:

######  Definition 2.

Let $(\mu_{1},\ldots,\mu_{n})$ be a family of probability measures and
$(w_{1},\ldots,w_{n})$ be a set of weights s.t. $w_{i}\geq 0$ for
$i=1,\ldots,n$ and such that $\sum_{i=1}^{n}w_{i}>0$. Let $\alpha\geq 1$, the
information radius of order $\alpha$ is defined as:

$\displaystyle\frac{1}{\alpha-1}\min_{\nu\ll\sum_{i}w_{i}\mu_{i}}\log\left(\sum_{i}w_{i}\exp((\alpha-1)D_{\alpha}(\mu_{i}\|\nu))\right).$

Suppose now we have two random variables $X,Y$ jointly distributed according
to $\mathcal{P}_{XY}$. It is possible to generalise Def. 2 and see that the
information radius is a special case of the following quantity [5]:

$I_{\alpha}(X,Y)=\min_{\mathcal{Q}_{Y}}D_{\alpha}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}_{Y}).$
(3)

$I_{\alpha}(X,Y)$ represents a generalisation of Shannon’s Mutual Information
and possesses many interesting properties [5]. Indeed, $\lim_{\alpha\to
1}I_{\alpha}(X,Y)=I(X;Y)$. On the other hand when $\alpha\to\infty$, we get:
$I_{\infty}(X,Y)=\log\mathbb{E}_{\mathcal{P}_{Y}}\left[\sup_{x:\mathcal{P}_{X}(x)>0}\frac{\mathcal{P}_{XY}(\\{x,Y\\})}{\mathcal{P}_{X}(\\{x\\})\mathcal{P}_{Y}(\\{Y\\})}\right]=\mathcal{L}\left(X\\!\\!\to\\!\\!Y\right)$,
where $\mathcal{L}\left(X\\!\\!\to\\!\\!Y\right)$ denotes the Maximal Leakage
from $X$ to $Y$, a recently defined information measure with an operational
meaning in the context of privacy and security [4]. For more details on
Sibson’s $\alpha$-MI, as well as a closed-form expression, we refer the reader
to [5], as for Maximal Leakage the reader is referred to [4].

###  II-B Rényi’s $\alpha$-Divergence - Negative Orders

Not much is known or has been explored on Rényi’s $\alpha$-Divergence with
$\alpha<0$. According to Rényi himself [9] only positive orders can be
regarded as information measures. One of the reasons behind this statement is
probably the fact that, taking an axiomatic approach to measures of
information like the one undertaken in [9], $D_{\alpha}$ with $\alpha<0$
satisfies many of those properties with the wrong sign of inequality. In
fact[6]:

######  Proposition 1.

Let $\alpha<0$ and $\mu,\nu$ be two probability measures such that $\nu\ll\mu$
then:

  1. 1.

$D_{\alpha}(\nu\|\mu)\leq 0$ (as opposed to $\geq 0$);

  2. 2.

$D_{\alpha}(\nu\|\mu)$ is concave in $\nu$ (as opposed to convex in $\mu$);

  3. 3.

$D_{\alpha}$ is upper semi-continuous in the pair $(\nu,\mu)$ in the topology
of set-wise convergence (as opposed to lower semi-continuous) ;

  4. 4.

Let $K$ be a Markov Kernel, then $D_{\alpha}(K\nu\|K\mu)\geq
D_{\alpha}(\nu\|\mu)$ (as opposed to $D_{\alpha}(K\nu\|K\mu)\leq
D_{\alpha}(\nu\|\mu)$);

However, some of the properties are maintained, like the following:

######  Theorem ([6, Thm 39]).

Let $\alpha\in[-\infty,+\infty]$ and let $\nu,\mu$ be two probability measures
such that $\nu\ll\mu$, then $D_{\alpha}(\nu\|\mu)$ is non-decreasing in
$\alpha$.

And to conclude, let us present a result connecting negative and positive
orders:

######  Lemma 1 ([6, Lemma 10] \- Skew Symmetry).

For every $\alpha\in(-\infty,+\infty)\setminus\\{0,1\\}$, let let $\nu,\mu$ be
two probability measures such that $\nu\equiv\mu$

$D_{\alpha}(\nu\|\mu)=\frac{\alpha}{1-\alpha}D_{(1-\alpha)}(\mu\|\nu).$ (4)

Such a relationship, although very useful, is restricted to cases in which the
two measures are equivalent with respect to each other, i.e., absolute
continuity holds in both directions: $\nu\ll\mu$ and $\mu\ll\nu$ otherwise one
might incur in settings where one divergence is finite and the other is
infinite. However, $D_{\alpha}$ with $\alpha<0$ is not concave in the second
argument.

######  Counterexample 1.

Consider a discrete setting and point mass functions. Let
$\mu_{1}=(0.32,0.68),\mu_{2}=(0.5,0.5),\nu=(0.13,0.87)$ and $\lambda=0.4$. One
has that with $\alpha=-2$, $D_{\alpha}(\nu\|\mu_{1})=-0.2855$,
$D_{\alpha}(\nu\|\mu_{2})=-0.6744$ and, moreover,
$-0.5287=D_{\alpha}(\nu\|\lambda\mu_{1}+(1-\lambda)\mu_{2})<\lambda
D_{\alpha}(\nu\|\mu_{1})+(1-\lambda)D_{\alpha}(\nu\|\mu_{2})=-0.5188.$

##  III Definition

The starting point will, once again, be (3). From now on we will consider
$\alpha$ to be always strictly negative unless specified otherwise and we will
restrict ourselves to discrete probability measures for simplicity. We thus
bring forth the following definition:

######  Definition 3.

Let $X,Y$ be two random variables whose joint measure is $\mathcal{P}_{XY}$
and the corresponding marginals are given by $\mathcal{P}_{X}$ and
$\mathcal{P}_{Y}$. Let $\alpha\in(-\infty,0)$

$I_{\alpha}(X,Y)=-\max_{\mathcal{Q}_{Y}}D_{\alpha}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}_{Y}).$
(5)

######  Remark 2.

The minus in the definition is included in order to make sure that all the
properties of an information measures are satisfied (non-negativity, DPI,
etc.). A similar approach can be undertaken in order to define $D_{\alpha}$
and $H_{\alpha}$ with negative $\alpha$ by simply changing the multiplicative
constant from $\frac{1}{\alpha-1}$ to $\frac{1}{1-\alpha}$ for Rényi’s
divergence and from $\frac{1}{1-\alpha}$ to $\frac{1}{\alpha-1}$ for Rényi’s
entropy for negative orders.

This quantity has, much like Sibson’s $\alpha$-MI with $\alpha>0$, a closed-
form expression given by the following result.

######  Theorem 1.

Let $X,Y$ be two random variables whose joint measure is $\mathcal{P}_{XY}$
and the corresponding marginals are given by $\mathcal{P}_{X}$ and
$\mathcal{P}_{Y}$. Let $\alpha<0$

$I_{\alpha}(X,Y)=-\frac{\alpha}{\alpha-1}\log\mathbb{E}_{\mathcal{P}_{Y}}\left[\mathbb{E}_{\mathcal{P}_{X}}^{\frac{1}{\alpha}}\left[\left(\frac{d\mathcal{P}_{XY}}{d\mathcal{P}_{X}\mathcal{P}_{Y}}\right)^{\alpha}\right]\right].$
(6)

###### Proof.

We have that for every $\mathcal{Q}_{Y}$,

$\displaystyle D_{\alpha}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}_{Y})$
$\displaystyle=\frac{1}{\alpha-1}\log\mathbb{E}_{\mathcal{Q}_{Y}}\left[\mathbb{E}_{\mathcal{P}_{X}}\left[\mathcal{P}_{Y|X}^{\alpha}\mathcal{Q}_{Y}^{-\alpha}\right]\right]$
(7)
$\displaystyle=\frac{\alpha}{\alpha-1}\log\mathbb{E}^{\frac{1}{\alpha}}_{\mathcal{Q}_{Y}}\left[\frac{\mathbb{E}_{\mathcal{P}_{X}}\left[\mathcal{P}_{Y|X}^{\alpha}\right]}{Q_{Y}^{\alpha}}\right]$
(8)
$\displaystyle\leq\frac{\alpha}{\alpha-1}\log\mathbb{E}_{\mathcal{Q}_{Y}}\left[\frac{\mathbb{E}^{\frac{1}{\alpha}}_{\mathcal{P}_{X}}\left[\mathcal{P}_{Y|X}^{\alpha}\right]}{Q_{Y}}\right]$
(9)
$\displaystyle=\frac{\alpha}{\alpha-1}\log\sum_{y}\mathbb{E}^{\frac{1}{\alpha}}_{\mathcal{P}_{X}}\left[\mathcal{P}_{Y|X}^{\alpha}\right]$
(10)
$\displaystyle=\frac{\alpha}{\alpha-1}\log\sum_{y}\mathbb{E}^{\frac{1}{\alpha}}_{\mathcal{P}_{X}}\left[\mathcal{P}_{XY}^{\alpha}\mathcal{P}_{X}^{-\alpha}\right]$
(11)
$\displaystyle=\frac{\alpha}{\alpha-1}\log\mathbb{E}_{\mathcal{P}_{Y}}\left[\mathbb{E}^{\frac{1}{\alpha}}_{\mathcal{P}_{X}}\left[\left(\frac{\mathcal{P}_{X|Y}}{\mathcal{P}_{X}}\right)^{\alpha}\right]\right]$
(12) $\displaystyle=-I_{\alpha}(X,Y).$ (13)

Where (9) follows from the convexity of $x^{\frac{1}{k}},k<0$ and Jensen’s
inequality. This implies that for every $\mathcal{Q}_{Y}$:

$\displaystyle-I_{\alpha}(X,Y)\geq
D_{\alpha}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}_{Y}).$ (14)

Moreover one has that, given

$\mathcal{Q}_{Y}^{\star}(y)=\frac{\mathcal{P}_{Y}(y)\left(\sum_{x}\mathcal{P}_{X|Y=y}(x)^{\alpha}\mathcal{P}_{X}(x)^{1-\alpha}\right)^{\frac{1}{\alpha}}}{\mathbb{E}_{\mathcal{P}_{Y}}\left[\left(\sum_{x}\mathcal{P}_{X|Y=y}(x)^{\alpha}\mathcal{P}_{X}(x)^{1-\alpha}\right)^{\frac{1}{\alpha}}\right]}$
(15)

then,

$D_{\alpha}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}^{\star}_{Y})=-I_{\alpha}(X,Y).$
(16)

∎

######  Remark 3.

In a setting where both $X$ and $Y$ are discrete random variables then
denoting with $p_{XY}$, $p_{Y}$ and $p_{X}$ the joint point mass functions and
the corresponding marginals and denoting with $p_{X|Y=y}$ the conditional
point mass functions one has that:

$\displaystyle I_{\alpha}(X,Y)\\!$
$\displaystyle=\\!\frac{\alpha}{1-\alpha}\\!\log\\!\sum_{y}\\!p_{Y}(y)\\!\\!\left(\\!\sum_{x}\\!\left(\frac{p_{X|Y=y}(x)}{p_{X}(x)}\\!\right)^{\alpha}\\!\\!\\!p_{X}(x)\\!\\!\right)^{\\!\\!\frac{1}{\alpha}}$
$\displaystyle=\frac{\alpha}{1-\alpha}\\!\log\\!\sum_{y}\\!\\!\left(\\!\sum_{x}p_{Y|X=x}(y)^{\alpha}p_{X}(x)\right)^{\\!\\!\frac{1}{\alpha}}.$
(17)

######  Theorem 2.

Let $X,Y$ be two discrete random variables whose joint point mass function is
$p_{XY}$ and the corresponding marginals are given by $p_{X}$ and $p_{Y}$ then

$I_{-\infty}(X,Y)=-\log\sum_{y}\left(\min_{x:p_{X}(x)>0}\mathcal{P}_{Y|X=x}(y)\right).$
(18)

###### Proof.

The result follows from noticing that one can re-write $I_{\alpha}$ as follows

$-I_{\alpha}(X,Y)=\frac{\alpha}{\alpha-1}\log\mathbb{E}_{\mathcal{P}_{Y}}\left[\left\lVert\frac{\mathcal{P}_{X|Y}}{\mathcal{P}_{X}}\right\rVert_{L^{\alpha}(\mathcal{P}_{X})}\right],$
(19)

where
$\left\lVert\frac{\mathcal{P}_{X|Y}}{\mathcal{P}_{X}}\right\rVert_{L^{\alpha}(\mathcal{P}_{X})}$
denotes the ”$\alpha-$norm” with respect to $\mathcal{P}_{X}$. Taking the
limit of $\alpha\to-\infty$ one has that $\frac{\alpha}{\alpha-1}\to 1$ and
$\left\lVert\frac{\mathcal{P}_{X|Y}}{\mathcal{P}_{X}}\right\rVert_{L^{\alpha}(\mathcal{P}_{X})}\to\operatorname*{ess\,inf}_{\mathcal{P}_{X}}\frac{\mathcal{P}_{X|Y}}{\mathcal{P}_{X}}.$
The conclusion then follows from simple algebraic manipulations. ∎

######  Remark 4.

The quantity $I_{-\infty}$ has already appeared in the literature as “Maximal-
Cost Leakage” [4, Thm. 15, Eq. (95)]. Indeed, assuming equivalence between
$\mathcal{P}_{XY}$ and $\mathcal{P}_{X}\mathcal{Q}_{Y}$ and using the Skew
Symmetry of $D_{\alpha}$ one has that
$-D_{-\infty}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}_{Y})=D_{\infty}(\mathcal{P}_{X}\mathcal{Q}_{Y}\|\mathcal{P}_{XY})$
and consequently

$\displaystyle I_{-\infty}(X,Y)$
$\displaystyle=-\max_{\mathcal{Q}_{Y}}D_{-\infty}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}_{Y})$
(20)
$\displaystyle=\min_{\mathcal{Q}_{Y}}D_{\infty}(\mathcal{P}_{X}\mathcal{Q}_{Y}\|\mathcal{P}_{XY})$
(21) $\displaystyle=\mathcal{L}^{c}(X\\!\\!\to\\!\\!Y).$ (22)

##  IV Properties

For $I_{\alpha}$ with $\alpha<0$ we can show the following properties.

######  Theorem 3.

Let $X,Y$ be two random variables such that
$\mathcal{P}_{XY}\ll\mathcal{P}_{X}\mathcal{P}_{Y}$ and assume that
$\alpha<0$, then:

  1. 1.

$I_{\alpha}(X,Y)\geq 0$ with equality iff $X$ and $Y$ are independent;

  2. 2.

$I_{\alpha}(X,Y)\neq I_{\alpha}(Y,X)$;

  3. 3.

Let $0>\alpha_{1}>\alpha_{2}$ then $I_{\alpha_{1}}(X,Y)\leq
I_{\alpha_{2}}(X,Y)$;

  4. 4.

$I_{\alpha}(X,Y)\leq\mathcal{L}^{c}(X\\!\\!\to\\!\\!Y)$ for every
$-\infty<\alpha<0$;

  5. 5.

Let $X-Y-Z$ be a Markov Chain,
$I_{\alpha}(X,Z)\leq\min\\{I_{\alpha}(X,Y),I_{\alpha}(Y,Z)\\}$;

  6. 6.

$\exp\left(-\frac{\alpha-1}{\alpha}I_{\alpha}(X,Y)\right)$ is concave in
$\mathcal{P}_{Y|X}$ and $I_{\alpha}(X,Y)$ is convex in $\mathcal{P}_{Y|X}$.

###### Proof.

1): We have that
$I_{\alpha}(X,Y)=-D_{\alpha}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}_{Y}^{\star})$
and given that $D_{\alpha}\leq 0$ if $\alpha\leq 0$ we have the non-negativity
of $I_{\alpha}$. Moreover, if $X$ and $Y$ are independent then
$\mathcal{Q}^{\star}_{Y}(y)=\mathcal{P}_{Y}(y)$ and
$D_{\alpha}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{P}_{Y})=0$. On the other
hand, if $I_{\alpha}(X,Y)=0$ then
$D_{\alpha}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}^{\star}_{Y})=0$ which
means that $\mathcal{P}_{Y|X=x}=\mathcal{Q}_{Y}^{\star}$, hence $Y$ does not
depend on $X$.
2): It’s clear from the definition and shown more concretely in Example 2.
3): We have that, denoting with
$\mathcal{Q}_{Y}^{\alpha_{2}}=\operatorname*{arg\,max}_{\mathcal{Q}_{Y}}D_{\alpha_{2}}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}_{Y})$

$\displaystyle-I_{\alpha_{1}}(X,Y)$
$\displaystyle=\max_{Q_{Y}}D_{\alpha_{1}}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}_{Y})$
(23) $\displaystyle\geq
D_{\alpha_{1}}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}_{Y}^{\alpha_{2}})$
(24) $\displaystyle\geq
D_{\alpha_{2}}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}_{Y}^{\alpha_{2}})$
(25) $\displaystyle=-I_{\alpha_{2}}(X,Y)$ (26)

where (25) follows from the non-decreasability of $D_{\alpha}$ for
$\alpha\in[-\infty,\infty]$ [6, Thm. 39].
4): Follows from 3) and the fact that $\mathcal{L}^{c}(X\\!\\!\to\\!\\!Y)=$
$\lim_{\alpha\to-\infty}I_{\alpha}(X,Y)$.
5): let $X-Y-Z$ be a Markov chain and let $K_{Z|Y}$ denote the corresponding
Markov Kernel
$K_{Z|Y}:(\mathcal{Y},\mathcal{F})\to(\mathcal{Z},\mathcal{\hat{F}})$ induced
by the transition probabilities $\mathcal{P}_{Z|Y}$. Consequently, given any
probability measure $\mathcal{Q}_{Y}$ one can construct a corresponding
$\mathcal{Q}_{Z}=\mathcal{Q}_{Y}K_{Z|Y}$ where substantially
$\mathcal{Q}_{Z}(z)=\mathbb{E}_{\mathcal{Q}_{Y}}[\mathcal{P}_{Z|Y}(z)]$. By
the Data-Processing Inequality for $D_{\alpha}$ one has that for every
$\mathcal{Q}_{Y}$:

$-D_{\alpha}(\mathcal{P}_{XZ}\|\mathcal{P}_{X}\mathcal{Q}_{Z})\leq-
D_{\alpha}(\mathcal{P}_{XY}\|\mathcal{P}_{X}\mathcal{Q}_{Y})$ (27)

Taking the $\inf$ with respect to $\mathcal{Q}_{Y}$ leads to

$-D_{\alpha}(\mathcal{P}_{XZ}\|\mathcal{P}_{X}\mathcal{Q}^{\star}_{Z})\leq
I_{\alpha}(X,Y)$ (28)

and given that $I_{\alpha}(X,Z)\leq-
D_{\alpha}(\mathcal{P}_{XZ}\|\mathcal{P}_{X}\mathcal{Q}_{Z})$ for every
$\mathcal{Q}_{Z}$ the statement follows. The other inequality follows by
observing that considering the Kernel determined by $\mathcal{P}_{XZ|XY}$ and
denoted by $K_{XZ|YZ}$ one has that
$\mathcal{P}_{XZ}=\mathcal{P}_{YZ}K_{XZ|YZ}$ while
$\mathcal{P}_{X}\mathcal{Q}_{Z}=(\mathcal{P}_{X}\mathcal{Q}_{Y})K_{XZ|YZ}$,
consequently by the Data-Processing Inequality for $D_{\alpha}$ one has that

$-D_{\alpha}(\mathcal{P}_{XZ}\|\mathcal{P}_{X}\mathcal{Q}_{Z})\leq-
D_{\alpha}(\mathcal{P}_{YZ}\|\mathcal{P}_{Y}\mathcal{Q}_{Z}).$ (29)

The statement follows from a similar argument as above.
6): One can rewrite the expression as follows

$\exp\left(-\frac{\alpha-1}{\alpha}I_{\alpha}(X,Y)\right)=\sum_{y}\left\lVert\mathcal{P}_{Y|X}\right\rVert_{L^{\alpha}(\mathcal{P}_{X})}$
(30)

and concavity follows from the Reverse-Minkowski’s inequality. Convexity of
$I_{\alpha}(X,Y)$ follows from the concavity just proven and the fact that
$-\frac{\alpha}{\alpha-1}\log(x)$ is a non-increasing convex function for a
given $\alpha$ and composition of a non-increasing convex function with a
concave one gives rise to a convex function [10, Eq (3.10), pag. 84]. ∎

Following [5] we will now look at some specific choices of $X$ and $Y$ and
compute the corresponding values of $I_{\alpha}$. Given that the expression
for the information measure is essentially identical for $\alpha<0$ and
$\alpha$, the values in these examples match the one for $I_{\alpha}$ with
$\alpha>0$ except for a minus sign that ensures the non-negativity of the
measure.

######  Example 1.

Let $X,Y\sim\text{Ber}(1/2)$ and let $\mathbb{P}(Y\neq X)=\delta$ then if
$\alpha<0$

$\displaystyle I_{\alpha}(X,Y)=I_{\alpha}(Y,X)=-d_{\alpha}(\delta\|1/2),$ (31)

where
$d_{\alpha}(p\|q)=\frac{1}{\alpha-1}\log(p^{\alpha}q^{1-\alpha}+(1\\!-p)^{\alpha}(1\\!-q)^{1-\alpha})$
denotes the binary $\alpha$-divergence.

######  Example 2.

Let $X\sim\text{Ber}(1/2)$ and let $Y\in\\{0,1,e\\}$. Assume also that

$P_{Y|X=x}(y)=\begin{cases}1-\delta,\,\,x=y\\\ \delta,\,\,y=e\\\ 0,\text{
else}.\end{cases}$ (32)

Then, if $\alpha<0$

$\displaystyle I_{\alpha}(X,Y)$
$\displaystyle=-\frac{\alpha}{\alpha-1}\log_{2}\left(\delta+(1-\delta)2^{\frac{\alpha-1}{\alpha}}\right)$
(33) $\displaystyle I_{\alpha}(Y,X)$
$\displaystyle=-\frac{1}{\alpha-1}\log_{2}\left(\delta+(1-\delta)2^{\alpha-1}\right)$
(34)

######  Example 3.

Let $X\sim\mathcal{N}(0,\sigma^{2}_{X})$ and
$N\sim\mathcal{N}(0,\sigma^{2}_{N})$ with $N$ independent from $X$ then if
$-\frac{\sigma^{2}_{N}}{\sigma^{2}_{X}}<\alpha<0$

$\displaystyle I_{\alpha}(X,X+N)$
$\displaystyle=-\frac{1}{2}\log\left(1+\alpha\frac{\sigma^{2}_{X}}{\sigma^{2}_{N}}\right).$
(35)

This example shows a major difference in behaviour between the two families of
information measures: the value of $I_{\alpha}(X,X+N)$ has the familiar
expression only if $|\alpha|\leq\frac{\sigma^{2}_{N}}{\sigma^{2}_{X}}$.

##  V Bound

######  Theorem 4.

Let $X,Y$ be two random variables whose joint measure is $\mathcal{P}_{XY}$
and the corresponding marginals are given by $\mathcal{P}_{X}$ and
$\mathcal{P}_{Y}$. For every $f:\mathcal{X}\times\mathcal{Y}\to\mathbb{R}^{+}$
one has that

$\displaystyle\mathbb{E}_{\mathcal{P}_{XY}}[f(X,Y)]\geq$
$\displaystyle\mathbb{E}^{\frac{1}{\beta^{\prime}}}_{\mathcal{P}_{Y}}\left[\mathbb{E}^{\frac{\beta^{\prime}}{\beta}}_{\mathcal{P}_{X}}\left[f(X,Y)^{\beta}\right]\right]$
(36)
$\displaystyle\cdot\mathbb{E}^{\frac{1}{\alpha^{\prime}}}_{\mathcal{P}_{Y}}\left[\mathbb{E}^{\frac{\alpha^{\prime}}{\alpha}}_{\mathcal{P}_{X}}\left[\left(\frac{d\mathcal{P}_{XY}}{d\mathcal{P}_{X}\mathcal{P}_{Y}}\right)^{\alpha}\right]\right],$
(37)

where
$\frac{1}{\alpha}+\frac{1}{\beta}=1=\frac{1}{\alpha^{\prime}}+\frac{1}{\beta^{\prime}}$
and $\alpha,\alpha^{\prime}<1$.

###### Proof.

$\displaystyle\mathbb{E}_{\mathcal{P}_{XY}}[f(X,Y)]=\,$
$\displaystyle\mathbb{E}_{\mathcal{P}_{X}\mathcal{P}_{Y}}\left[f(X,Y)\left(\frac{d\mathcal{P}_{XY}}{d\mathcal{P}_{X}\mathcal{P}_{Y}}\right)\right]$
(38) $\displaystyle\geq\,$
$\displaystyle\mathbb{E}_{\mathcal{P}_{Y}}\Bigg{[}\mathbb{E}^{\frac{1}{\beta}}_{\mathcal{P}_{X}}\left[f(X,Y)^{\beta}\right]$
(39)
$\displaystyle\,\,\,\,\,\,\,\,\,\,\,\,\,\,\cdot\mathbb{E}^{\frac{1}{\alpha}}_{\mathcal{P}_{X}}\left[\left(\frac{d\mathcal{P}_{XY}}{d\mathcal{P}_{X}\mathcal{P}_{Y}}\right)^{\alpha}\right]\Bigg{]}$
(40) $\displaystyle\geq\,$
$\displaystyle\mathbb{E}^{\frac{1}{\beta^{\prime}}}_{\mathcal{P}_{Y}}\left[\mathbb{E}^{\frac{\beta^{\prime}}{\beta}}_{\mathcal{P}_{X}}\left[f(X,Y)^{\beta}\right]\right]$
(41) $\displaystyle\cdot$
$\displaystyle\,\mathbb{E}^{\frac{1}{\alpha^{\prime}}}_{\mathcal{P}_{Y}}\left[\mathbb{E}^{\frac{\alpha^{\prime}}{\alpha}}_{\mathcal{P}_{X}}\left[\left(\frac{d\mathcal{P}_{XY}}{d\mathcal{P}_{X}\mathcal{P}_{Y}}\right)^{\alpha}\right]\right],$
(42)

where each inequality follows from applying the reverse Hölder’s inequality
(which, in turn, requires the positivity of $f$). ∎

Taking the limit of $\alpha^{\prime}\to 1$ (which implies
$\beta^{\prime}\to-\infty$) leads to the following bound involving
$I_{\alpha}(X,Y)$ with $\alpha<1$:

######  Corollary 1.

Consider the same setting as in Theorem 4. For every
$f:\mathcal{X}\times\mathcal{Y}\to\mathbb{R}^{+}$ and for every $\alpha<1$ and
$\beta=\frac{\alpha-1}{\alpha}$ one has that

$\displaystyle\mathbb{E}_{\mathcal{P}_{XY}}[f(X,Y)]\geq$
$\displaystyle\operatorname*{ess\,inf}_{\mathcal{P}_{Y}}\left[\mathbb{E}^{\frac{1}{\beta}}_{\mathcal{P}_{X}}\left[f(X,Y)^{\beta}\right]\right]$
(43)
$\displaystyle\cdot\exp\left(\operatorname{sign}(\alpha)\cdot\frac{\alpha-1}{\alpha}I_{\alpha}(X,Y)\right).$
(44)

Corollary 1 holds for every non-negative function $f$. However, given
$0<\alpha<1$ (which in turn implies $\beta<0$), if there exists an $x$ with
positive measure with respect to $\mathcal{P}_{X}$ and such that for every $y$
such that $\mathcal{P}_{Y}(\\{y\\})>0$, $f(x,y)=0$, one recovers a trivial
lower-bound on $\mathbb{E}_{\mathcal{P}_{XY}}[f(X,Y)]$. This prevents us from
setting $f=\mathds{1}_{E}$ and recovering a bound that involves probabilities
in this case. However, whenever $\alpha<0$ (which implies $0<\beta<1$) we do
recover said functions and we can provide the following lower-bound connecting
the probability of any event $E$ with respect to the joint $\mathcal{P}_{XY}$
and the product of the marginals $\mathcal{P}_{X}\mathcal{P}_{Y}$:

######  Corollary 2.

Let $X,Y$ be two random variables and consider the probability spaces
$(\mathcal{X}\times\mathcal{Y},\mathcal{F},\mathcal{P}_{XY})$ and
$(\mathcal{X}\times\mathcal{Y},\mathcal{F},\mathcal{P}_{X}\mathcal{P}_{Y})$.
Let $E\in\mathcal{F}$ and, given $y\in\mathcal{Y}$, denote with
$E_{y}=\\{x:(x,y)\in E\\}$, then, for every $\alpha<0$

$\displaystyle\mathcal{P}_{XY}(E)$
$\displaystyle\geq\min_{y}\mathcal{P}_{X}(E_{y})^{\frac{1}{\beta}}\cdot\exp\left(-\frac{\alpha-1}{\alpha}I_{\alpha}(X,Y)\right)$
(45)
$\displaystyle=\exp\left(\frac{1}{\beta}\left(\log(\min_{y}\mathcal{P}_{X}(E_{y}))-I_{\alpha}(X,Y)\right)\right).$
(46)

Taking the limit of $\alpha\to-\infty$ which implies $\beta\to 1$ one recovers
the following:

$\displaystyle\mathcal{P}_{XY}(E)$
$\displaystyle\geq\min_{y}\mathcal{P}_{X}(E_{y})\exp(-I_{-\infty}(X,Y))$ (47)
$\displaystyle=\min_{y}\mathcal{P}_{X}(E_{y})\exp(-\mathcal{L}^{c}(X\\!\\!\to\\!\\!Y)).$
(48)

TABLE I: Behaviour of the bounds expressed in Corollary 1, Corollary 2,
Corollary 3 and [1, Corollary 1] Behaviour of the Bound
$\mathbb{E}_{\mathcal{P}_{XY}}[f(X,Y)]\lesseqgtr h_{\beta}(f(X,Y))\cdot
g(I_{\alpha}(X,Y))$
---
Admitted values of $\alpha$ |  $\alpha<0\implies 0<\beta<1$ |  $0<\alpha<1\implies\beta<0$ |  $\alpha>1\implies\beta>1$
Information-Measure $g(I_{\alpha})$ |  $\exp((1-\alpha)/\alpha\cdot I_{\alpha}(X,Y))$ |  $\exp((\alpha-1)/\alpha\cdot I_{\alpha}(X,Y))$ |  $\exp((\alpha-1)/\alpha\cdot I_{\alpha}(X,Y))$
Multiplicative Term $h_{\beta}(f)$ |  $\min_{y}\mathbb{E}^{\frac{1}{\beta}}_{P_{X}}\left[f(X,y)^{\beta}\right]$ |  $\min_{y}\mathbb{E}^{\frac{1}{\beta}}_{P_{X}}\left[f(X,y)^{\beta}\right]$ |  $\max_{y}\mathbb{E}^{\frac{1}{\beta}}_{P_{X}}\left[f(X,y)^{\beta}\right]$
Multiplicat. Term $h_{\beta}(\mathds{1}_{E})$ |  $\min_{y}(P_{X}(E_{y}))^{\frac{1}{\beta}}$ |  cannot be provided |  $\max_{y}(P_{X}(E_{y}))^{\frac{1}{\beta}}$
Inequality |  $\mathbb{E}_{\mathcal{P}_{XY}}[f]\geq h_{\beta}(f)\cdot g(I_{\alpha}(X,Y))$ |  $\mathbb{E}_{\mathcal{P}_{XY}}[f]\geq h_{\beta}(f)\cdot g(I_{\alpha}(X,Y))$ |  $\mathbb{E}_{\mathcal{P}_{XY}}[f]\leq h_{\beta}(f)\cdot g(I_{\alpha}(X,Y))$
References |  Corollary 1 and Corollary 2 |  Corollary 1 |  Corollary 3 and [1, Corollary 1]

Let us compare, through Table I, Corollary 1, Corollary 2 and a straight-
forward generalisation of [1, Corollary 1] that we will now state for
reference:

######  Corollary 3.

Consider the same setting as in Theorem 4. For every
$f:\mathcal{X}\times\mathcal{Y}\to\mathbb{R}^{+}$ and for every $\alpha>1$ and
$\beta=\frac{\alpha-1}{\alpha}$ one has that

$\displaystyle\mathbb{E}_{\mathcal{P}_{XY}}[f(X,Y)]\leq$
$\displaystyle\operatorname*{ess\,sup}_{\mathcal{P}_{Y}}\left[\mathbb{E}^{\frac{1}{\beta}}_{\mathcal{P}_{X}}\left[f(X,Y)^{\beta}\right]\right]$
(49)
$\displaystyle\cdot\exp\left(\frac{\alpha-1}{\alpha}I_{\alpha}(X,Y)\right).$
(50)

Differently from $I_{\alpha}$ with $\alpha>0$, $I_{\alpha}$ with $\alpha<0$ is
strongly connected to converse/negative results. For instance, it can find
applications in lower-bounding the Bayesian Risk in estimation procedures. In
these settings one often has a non-negative loss function $\ell(\cdot,\cdot)$
that measures how far a parameter is from its estimation. The purpose is often
to lower-bound the minimum expected value of this loss where the minimum is
over all the possible estimators of the parameter. In such a framework
Corollary 3 can be useful as we will see in the next section.

##  VI Bayesian Risk

Let $\mathcal{W}$ denote the parameter space and assume that we have access to
a prior distribution over this space $\mathcal{P}_{W}$. Suppose then that we
observe $W$ through the family of distributions
$\mathcal{P}=\\{\mathcal{P}_{X|W=w}:w\in\mathcal{W}\\}.$ Given a function
$\phi:\mathcal{X}\to\hat{\mathcal{W}}$, one can then estimate $W$ from the
observations $X\sim\mathcal{P}_{X|W}$ via $\phi(X)=\hat{W}$. Let us denote
with $\ell:\mathcal{W}\times\hat{\mathcal{W}}\to\mathbb{R}^{+}$ a loss
function, the Bayesian risk is defined as:

$R=\inf_{\phi}\mathbb{E}[\ell(W,\phi(X)]=\inf_{\phi}\mathbb{E}[\ell(W,\hat{W})].$
(51)

######  Corollary 4.

Consider the Bayesian framework just described, the following must hold for
every $\alpha<0$ and every $\hat{W}=\phi(X^{n})$:

$\displaystyle\mathbb{E}[\ell(W,\hat{W})]\geq\exp\Bigg{(}$
$\displaystyle\frac{1}{\beta}\Bigg{(}-I_{\alpha}(W,X^{n})$
$\displaystyle+\log\left(\operatorname*{ess\,inf}_{P_{\hat{W}}}\mathbb{E}_{P_{W}}\left[\ell(W,\hat{W})^{\beta}\right]\right)\\!\\!\Bigg{)}\\!\Bigg{)}.$
(52)

Moreover, taking the limit of $\alpha\to-\infty$ one recovers the following:

$\mathbb{E}[\ell(W,\hat{W})]\geq\rho\exp\left(-\mathcal{L}^{c}(W\\!\\!\to\\!\\!X^{n})\right)\\!\operatorname*{ess\,inf}_{\mathcal{P}_{\hat{W}}}\mathbb{E}_{P_{W}}\\!\left[\ell(W,\hat{W})\right]\\!\\!.$
(53)

###### Proof.

The result follows from Corollary 1, Property 5 and the fact that
$W-X^{n}-\hat{W}$ is a Markov Chain. ∎

Another result can be included that allows us to retrieve a lower-bound that
is independent of the choice of the estimator $\phi$.

######  Corollary 5.

Consider the Bayesian framework just described, the following must hold for
every $\rho>0$, $\alpha<0$ and every $\hat{W}=\phi(X^{n})$:

$\displaystyle\mathbb{E}[\ell(W,\hat{W})]\geq\rho\exp\left(-\frac{1}{\beta}I_{\alpha}(W,X^{n})\right)\left(1-L_{W}(\rho))\right)^{\frac{1}{\beta}},$
(54)

where $L_{W}(\rho)=\max_{\hat{w}}\mathcal{P}_{W}(\ell(W,\hat{w})\leq\rho)$,
also known in the literature as “small-ball probability”. Moreover, taking the
limit of $\alpha\to-\infty$ one recovers the following:

$\mathbb{E}[\ell(W,\hat{W})]\geq\rho\exp\left(-\mathcal{L}^{c}(W\\!\\!\to\\!\\!X^{n})\right)\left(1-L_{W}(\rho))\right).$
(55)

###### Proof.

We start by observing that for every $\rho$, by Markov’s inequality

$\mathbb{E}[\ell(W,\hat{W})]\geq\rho\left(\mathbb{P}(\ell(W,\hat{W})\geq\rho)\right).$
(56)

The statement then follows from further lower-bounding
$\mathbb{P}(\ell(W,\hat{W})\geq\rho)$ with Corollary 2 and noticing that
$\min_{\hat{w}}\mathcal{P}_{W}(\ell(W,\hat{w})\geq\rho)=\min_{\hat{w}}(1-\mathcal{P}_{W}(\ell(W,\hat{w})\leq\rho))=1-L_{W}(\rho).$
∎

##  VII Conclusion

In this work we extended the definition of Sibson’s $\alpha$-Mutual
Information to negative orders. In order to have a properly defined object and
to be consistent with the axiomatic properties that an information measure
should satisfy (according to Rényi [9]), we slightly adapted the original
definition from (3) for $\alpha>0$ to (5) for $\alpha<0$. We presented a
sequence of properties that these objects satisfy and we connected the
information-measures to a family of functional inequalities (similarly to
$I_{\alpha}$ with $\alpha>1$ [1, Thm 4, Remark 7, Cor 1]). This family of
inequalities exists, although not always with the same inequality sign, for
every $\alpha\in\mathbb{R}$ as shown in Table I. When $\alpha>1$ and through
these inequalities, we were able to connect the information measure to bounds
on the probability of having a large generalisation error [1], hypothesis
testing problems [11] and to lower-bounds on the Bayesian Risk in Estimation
Procedures [12]. Here, as an example of application, we considered once again
the Bayesian Risk setting and showed how this family of information measures
can be employed in such a framework even if $\alpha<1$.

## Acknowledgment

The work in this paper was supported in part by the Swiss National Science
Foundation under Grant 200364.

## References

  * [1] A. R. Esposito, M. Gastpar, and I. Issa, “Generalization error bounds via Rényi-, $f$-divergences and Maximal Leakage,” _IEEE Transactions on Information Theory_ , vol. 67, no. 8, pp. 4986–5004, 2021.
  * [2] R. Gallager, “A simple derivation of the coding theorem and some applications,” _IEEE Transactions on Information Theory_ , vol. 11, no. 1, pp. 3–18, 1965.
  * [3] R. G. Gallager, _Information Theory and Reliable Communication_. USA: John Wiley & Sons, Inc., 1968.
  * [4] I. Issa, A. B. Wagner, and S. Kamath, “An operational approach to information leakage,” _IEEE Transactions on Information Theory_ , vol. 66, no. 3, pp. 1625–1657, 2020.
  * [5] S. Verdú, “$\alpha$-mutual information,” in _2015 Information Theory and Applications Workshop, ITA 2015, San Diego, CA, USA, February 1-6, 2015_ , 2015, pp. 1–6.
  * [6] T. van Erven and P. Harremoës, “Rényi divergence and Kullback-Leibler divergence,” _IEEE Trans. Inf. Theory_ , vol. 60, no. 7, pp. 3797–3820, July 2014.
  * [7] I. Csiszar, “Generalized cutoff rates and Rényi’s information measures,” _IEEE Transactions on Information Theory_ , vol. 41, no. 1, pp. 26–34, Jan 1995.
  * [8] R. Sibson, “Information radius,” _Z. Wahrscheinlichkeitstheorie verw Gebiete 14_ , pp. 149–160, 1969.
  * [9] A. Rényi, “On measures of entropy and information,” _Proceedings of the 4th Berkeley Symposium on Mathematics, Statistics and Probability_ , vol. 1, pp. 547–561, 1960.
  * [10] S. Boyd and L. Vandenberghe, _Convex Optimization_. Cambridge University Press, March 2004.
  * [11] A. R. Esposito, D. Wu, and M. Gastpar, “On conditional Sibson’s $\alpha$-Mutual Information,” in _2021 IEEE International Symposium on Information Theory (ISIT)_ , 2021, pp. 1796–1801.
  * [12] A. R. Esposito and M. Gastpar, “Lower-bounds on the bayesian risk in estimation procedures via Sibson’s $\alpha$-mutual information,” in _2021 IEEE International Symposium on Information Theory (ISIT)_ , 2021, pp. 748–753.