# A note on no-hair properties of static black holes in four and higher
dimensional spacetimes with cosmological constant

Akihiro Ishibashi1,2, Satoshi Matsumoto1, and Yuichiro Yoneo1

###### Abstract

We study no-hair properties of static black holes in four and higher
dimensional spacetimes with a cosmological constant. For the vanishing
cosmological constant case, we show a no-hair theorem and also a no-short-hair
theorem under certain conditions for the energy-momentum of matter fields. For
the positive cosmological constant case, we discuss conditions for hairy
static black holes to exist in terms of the energy density of matter fields
evaluated at the black hole horizon and the cosmological horizon. For the
negative cosmological constant case, we study conditions for hairy black holes
by presenting a no-hair theorem in which the asymptotic structure is assumed
to be determined by the true cosmological constant.

##  1 Introduction

It is well known that a stationary black hole in general relativity has the
uniqueness or no-hair property: All asymptotically flat, stationary black
holes in $4$-dimensional Einstein-Maxwell theory are uniquely characterized by
their mass, angular momentum, and electric charge[1, 2, 3, 4, 5, 6] (see also
a review [7]). No-hair properties have so far been studied in a number of
models with various different matter fields, by either proving no-hair
theorems [8, 9, 10, 11, 12] or explicitly (analytically or numerically)
constructing hairy black hole solutions [13, 14, 15, 16, 17, 18, 19, 20, 21].
Most of such studies have focused mainly on the case of $4$-dimensional,
asymptotically flat spacetimes.

There have recently been plenty of astrophysical and cosmological evidence to
support unknown dark sector in our universe. The existence of dark matter and
dark energy (or a positive cosmological constant) modifies, in principle, the
spacetime structure near the event horizon, as well as asymptotic region,
which allows for a cosmological horizon. It is of considerable importance to
study no-hair properties of stationary black hole spacetimes with positive
cosmological constant (see, e.g., [22, 23]).

A black hole spacetime also plays an important role as a theoretical
laboratory for various theories of gravity, such as string theories, in which
the notion of higher dimensional spacetime or extra-dimension is essential. It
is known that in higher dimensions, the standard notion of black hole no-hair
properties no longer holds (see, e.g., reviews [24, 25, 26]). Another
interesting theoretical possibilities are black hole spacetimes with negative
cosmological constant, which have attracted much attention in the context of
AdS/CFT correspondence or more broadly holographic principle [27, 28, 29]. It
is therefore of great interest in considering no-hair properties of black
holes in higher dimensional spacetimes with cosmological constant either
positive or negative (see e.g., [30, 31, 32, 33, 34] for the study of no-hair
properties with a non-vanishing cosmological constant in $4$-dimensions).

The main purpose of this paper is to analyze no-hair properties of static
black holes in four and higher dimensional Einstein gravity with cosmological
constant. For asymptotically locally flat case, we show a no-hair theorem and
also a no-short-hair theorem, generalizing to higher-dimensions Pena-
Sudarsky’s no-hair theorem [9] and Nunez-Quevedo-Sudarsky’s no-short hair
theorem [10]. For positive cosmological constant case, we discuss conditions
for hairy static black holes to exist in terms of the energy density of matter
fields evaluated at the black hole horizon and the cosmological horizon. For
asymptotically AdS case, we study conditions for hairy black holes by
generalizing to higher-dimensions Sudarsky-Gonzalez’ theorem [34].

The outline of this paper is as follows. In the next section, we summarize our
geometric ansatz and basic requirements for matter fields. In Sec. 3, we
present our main theorems and their proofs. Section 4 is devoted to summary
and discussions.

##  2 Static black hole metrics and matter fields

We consider the $(2+n)$-dimensional metric of the following warped product
form

$ds^{2}=g_{\mu\nu}dx^{\mu}dx^{\nu}=-\mu(r)e^{-2\delta(r)}dt^{2}+\frac{1}{\mu(r)}dr^{2}+r^{2}\gamma_{ij}(z)dz^{i}dz^{j},$
(2.1)

where $\mu$ and $\delta$ are functions of $r$ only and $\gamma_{ij}$ denotes
the metric of the $n$-dimensional Einstein manifold ${\cal K}_{n}$ with
normalized sectional curvature $k=1,0,-1$. Here and hereafter the greek
indices $\mu,\nu,...$ denote, as usual, tensor on the $(2+n)$-dimensional
spacetime, while the latin indices $i,j,...$ tensors on ${\cal K}_{n}$.

We consider the Einstein equations with a cosmological constant $\Lambda$ so
that the above metric is a solution to

$\displaystyle\begin{split}G^{\mu}_{\nu}&=8\pi{T^{(\rm
tot)}}^{\mu}_{\nu}:=8\pi{T}^{\mu}_{\nu}-\Lambda\delta^{\mu}_{\nu},\end{split}$
(2.2)

where $T^{\mu}_{\nu}$ denotes the energy-momentum tensor (EMT) for matter
fields. Note that in general one cannot uniquely separate the EMT for matter
fields and cosmological constant term. In what follows we assume that any
constant part of matter EMT proportional to the metric is already absorbed in
$\Lambda$.

The above metric (2.1) can describe a fairly large class of static black
holes, including generalized Schwarzschild-(Anti-)de Sitter black holes. In
the following we assume that the metric function $\mu(r)$ admits a single or
multiple single-root(s), which corresponds to a non-degenerate Killing
horizon(s). When $\Lambda\leq 0$, we further assume that for the largest root
$r_{b}$ of $\mu(r)$, $\delta(r_{b})<\infty$ and $\mu(r)>0$ for $r>r_{b}$ so
that $r_{b}$ corresponds to a black hole event horizon. When $\Lambda>0$, we
assume that there exist at least two roots $r_{b},r_{c}(>r_{b})$ for which
$\delta(r_{b})$ and $\delta(r_{c})$ are both finite and $\mu(r)>0$ for
$r_{b}<r<r_{c}$ so that $r_{b}$ and $r_{c}$ correspond to the black hole
horizon and cosmological horizon, respectively, in an asymptotically de Sitter
spacetime. In the following, for either asymptotically flat, AdS, or de Sitter
case, we refer to the static region outside the black hole where
$r_{b}<r(<r_{c})$ and $\mu(r)>0$ as the domain of outer communication
(DOC)111DOC is usually defined by $J^{-}({\cal I}^{+})\cap J^{+}({\cal
I}^{-})$. We are, however, interested mainly in the static region bounded by
the black hole horizon and cosmological horizon when $\Lambda>0$. In this
paper, by DOC we mean the region $J^{-}(\gamma)\cap J^{+}(\gamma)$, where
$\gamma$ is any complete timelike orbit of the Killing vector field
$\partial/\partial t$. Note that this definition excludes the region outside
the cosmological horizon when our spacetime is asymptotically locally de
Sitter. . We often denote the horizon radii $r_{b}$ and $r_{c}$ collectively
by $r_{H}$.

It is convenient to describe the metric function $\mu(r)$ in terms of the
generalized Misner-Sharp mass function, $m(r)$, as [35],

$\mu(r)=k-\frac{2m(r)}{r^{n-1}}-\lambda r^{2},$ (2.3)

with $\lambda:=2\Lambda/n(n+1)$. We assume that at large distances
($r\rightarrow\infty$), the mass function behaves as

$m(r)=m_{0}+O(1/r^{\alpha}),$ (2.4)

with some constants, $m_{0}\geq 0$ and $\alpha>0$, so that the asymptotic
structure of (2.1) near infinity is determined solely by $\lambda$, $m_{0}$,
and ${\cal K}_{n}$. Then, for instance, when $\lambda=0$ and ${\cal K}_{n}$ is
the round sphere, the ADM mass is given by $M_{\rm ADM}=nm_{0}{A}_{n}/8\pi$,
where ${A}_{n}=2\pi^{(n+1)/2}/\Gamma[(n+1)/2]$ is the area of a unit
$n$-sphere. We also require that the generalized Misner-Sharp mass is positive
in DOC:

$m(r)>0\quad{\rm for}\quad r_{b}\leq r(\leq r_{c}).$ (2.5)

Note that there have been several different types of quasilocal masses
proposed so far, mainly in $4$-dimensions [36], of which positivity has also
been studied for certain cases, e.g., [37, 38].

We note that from the symmetry structure of the metric ansatz (2.1), the EMT
for matter fields takes the form

$T^{t}_{i}=T^{r}_{i}=0,\quad T^{i}_{j}=P\delta^{i}_{j},$ (2.6)

with $P$ being some function of $r$, corresponding to a tangential pressure
along ${\cal K}_{n}$. We also introduce the energy density $\rho$ and the
radial pressure $P_{r}$ by

$\rho:=-T^{t}_{t},\quad P_{r}:=T^{r}_{r}\,.$ (2.7)

In order for the asymptotic structure to be determined by $\Lambda$, we
require that

$\rho,P_{r},P\sim O(1/r^{n+2})\rightarrow 0\quad\mbox{as
$r\rightarrow\infty$}.$ (2.8)

Note that each mixed component $T^{\mu}_{\nu}$ of the EMT for static Maxwell
fields in $(2+n)$-dimensions decays faster than (2.8), except for the
$4$-dimension ($n=2$) case.

With the assumptions (2.3), (2.4), (2.8) above, the spacetime (2.1) approaches
asymptotically locally either flat, de Sitter, or AdS, depending on $\lambda$,
and one may anticipate that whenever the matter EMT identically vanishes, the
static black hole under consideration coincides with the generalized
Schwarzschild-(A)dS black hole specified by $\lambda$, $m_{0}$, ${\cal
K}_{n}$. This is indeed the case for $\lambda=0$ in general dimensions as
shown in [39]. Then, we can say that a static black hole has “hair” when it
allows to have a non-trivial matter and metric configurations outside the
black hole event horizon, being different from the generalized
Schwarzschild-(A)dS metric, but still asymptotically approaches the
corresponding generalized Schwarzschild-(A)dS metric.

We impose, as a physically reasonable requirement for EMT, the weak energy
condition (WEC) on the matter fields:

$\rho\geq 0,\quad\rho+P_{r}\geq 0,\ \rho+P\geq 0.$ (2.9)

From the Einstein equations (2.2), we obtain

$\displaystyle\mu^{\prime}$ $\displaystyle=-\frac{16\pi r}{n}\rho-(n+1)\lambda
r+\frac{\left(n-1\right)\left(k-\mu\right)}{r},$ (2.10)
$\displaystyle\delta^{\prime}$ $\displaystyle=-\frac{8\pi
r}{n\mu}\left(\rho+P_{r}\right),$ (2.11)

where the prime denotes the derivative with respect to $r$. From (2.3), (2.10)
and WEC, we can easily find

$m^{\prime}=\frac{8\pi r^{n}}{n}\rho\geq 0.$ (2.12)

Thus, under the WEC, the generalized Misner-Sharp mass, $m(r)$, is a non-
decreasing function of the area radius $r$.

From the regularity at the horizon $r_{H}$, we can derive a useful constraint
on $T^{\mu}_{\nu}$. We first note that the scalar
$T^{\mu}_{\nu}T_{\mu}^{\nu}=(T^{t}_{t})^{2}+(T^{r}_{r})^{2}+T^{i}_{j}T^{j}_{i}$
is regular on the horizon $r_{H}$, hence each of the mixed components
$T^{\mu}_{\nu}$ must also be regular at $r_{H}$. Next, in order to see the
relation between each components $T^{\mu}_{\nu}$ on the horizon, we introduce
the null vector fields defined by

$\begin{split}n^{(\pm)}_{\alpha}dx^{\alpha}&=dt\pm\frac{e^{\delta}}{\mu}dr.\end{split}$
(2.13)

On the horizon $r_{H}$, the null vector field $n_{+}^{\alpha}$ (resp.
$n_{-}^{\alpha}$) becomes tangent to the past-directed null geodesic
generators of the black hole horizon at $r_{b}$ (resp. cosmological horizon at
$r_{c}$). Now contracting the EMT with the null vector field $n^{\alpha}$, we
have

${T^{(\rm tot)}}_{\mu\nu}n^{\mu}n^{\nu}=\frac{e^{2\delta}}{\mu}\left({T^{(\rm
tot)}}^{r}_{r}-{T^{(\rm
tot)}}^{t}_{t}\right)=\frac{e^{2\delta}}{\mu}\left({T}^{r}_{r}-{T}^{t}_{t}\right).$
(2.14)

This quantity should be regular on the horizon $r_{H}$. Since each component
$T^{t}_{t}$ and $T^{r}_{r}$ must be regular at $r_{H}$ by itself, as noted
above, we find222 When $k\neq 0$, this can be also shown directly by examining
(2.17) as in [9]. One can reexpress (2.17) as
$\dfrac{dE}{dx}=\dfrac{e^{-\delta}}{2r}\left[\dfrac{(n-1)k}{\mu^{1/2}}\left({T}^{t}_{t}-{T}^{r}_{r}\right)+\mu^{1/2}\left\\{(3-n){T}^{t}_{t}-(n-1)T^{r}_{r}+2\left(2nP-{T}\right)\right\\}\right]$
(2.15) Here in the left-hand side the derivative is taken with respect to the
proper radial distance $x$ defined $dx:=\mu^{-1/2}dr$, which is well-defined
on the horizon $r_{H}$, and therefore the right-hand side must also be well-
defined on $r_{H}$. From the first term of the right-hand side, one finds
(2.16) when $k\neq 0$. , ${T}^{t}_{t}(r_{H})={T}^{r}_{r}(r_{H})$, namely,

$\rho(r_{H})=-P_{r}(r_{H})\geq 0.$ (2.16)

Combining (2.10) and (2.11) together with the $r$-component of the
conservation law $\nabla_{\mu}{T^{(\rm tot)}}^{\mu}_{\nu}=0$, we can derive
the following formula:

$\begin{split}E^{\prime}&=\frac{e^{-\delta}}{2\mu
r}\bigg{[}(n-1)(k+\mu)\left({T}^{t}_{t}-{T}^{r}_{r}\right)-2(n-2)\mu{T}^{t}_{t}+2\mu\left(2nP-{T}\right)\bigg{]},\end{split}$
(2.17)

where in the right-hand side $T$ denotes the trace of ${T}^{\mu}_{\nu}$, and
in the left-hand side, we have introduced

$E\equiv e^{-\delta}{T^{(\rm tot)}}^{r}_{r}.$ (2.18)

This formula (2.17) can be cast into the form:

$E^{\prime}=\frac{e^{-\delta}}{2\mu r}\left\\{{\cal
A}\left(\rho+P\right)-{\cal B}\left(P_{r}-P\right)\right\\},$ (2.19)

where

$\mathcal{A}\equiv(n-1)(\mu-k)=(n-1)\left(-\frac{2m(r)}{r^{n-1}}-\lambda
r^{2}\right),\quad\mathcal{B}\equiv(n-1)k+(n+1)\mu.$ (2.20)

##  3 No-hair properties with or without cosmological constant in
$(2+n)$-dimensions

Let us now turn to the study of no-hair properties of possible static black
holes described by the metric (2.1) with $n\geq 2$. We are concerned with DOC,
where $\mu(r)\geq 0$. We analyze the asymptotically flat, de Sitter, and AdS
case, separately.

###  A Asymptotically flat case ($\Lambda=0$)

We state our no-hair theorem for static black holes with $\Lambda=0$, which
corresponds to a generalization of Pena-Sudarsky’s theorem [9] to the higher
dimensional case.

Theorem 1. Consider a $(2+n)$-dimensional static black hole spacetime
$(M,g_{\mu\nu})$ described by the metric (2.1) with the metric function (2.3)
and which has a non-degenerate regular black hole horizon at $r_{b}$ and
satisfies the Einstein equations with vanishing cosmological constant
$\Lambda=0$ and matter fields with $T^{\mu}_{\nu}$. We assume that the
generalized Misner-Sharp mass $m(r)$ is positive, (2.5), and asymptotically
behaves as (2.4), and also that the matter fields satisfy the WEC (2.9) and
asymptotically behave as (2.8). If furthermore the following condition between
the radial pressure $P_{r}$ and the tangential pressure $P$:

$P_{r}\geq P,$ (3.1)

is satisfied, then $T^{\mu}_{\nu}$ must vanish identically and
$(M,g_{\mu\nu})$ corresponds to the generalized Schwarzschild solution with
$({\cal K}_{n},k=1)$.

Proof. For $k=0,-1$ cases, the assumption (2.5) implies $\mu<0$, and therefore
the only relevant case is $k=1$. In this case, for any $r\geq r_{b}$, we have
${\cal B}>0$, while ${\cal A}<0$ due to (2.5). From WEC and the condition
(3.1), it follows that $\rho+P\geq 0$ and $P_{r}-P\geq 0$. Therefore, (2.19)
immediately implies $E^{\prime}\leq 0$; namely that $E$ is a non-increasing
function of $r$, thus in particular $E(r_{b})\geq E(r=\infty)$. By a suitable
rescaling of the time coordinate $t$, without loss of generality, one can make
$\delta(r_{b})=0$. Then, on one hand, we have
$E(r_{b})=P_{r}(r_{b})=-\rho(r_{b})\leq 0$ by the regularity (2.16) and WEC.
On the other hand, by the asymptotic condition (2.8), as well as the
regularity $\delta(\infty)<\infty$, the limit
$E(\infty)=e^{-\delta(\infty)}P_{r}(\infty)$ must vanish. Therefore $E(r)$
must be identically vanishing $E=0$, thus $P_{r}=0$, for any $r\geq r_{b}$.
This also implies $E^{\prime}=0$ identically. It then follows from (2.19) and
(2.20) that ${\cal A}\rho+2n\mu P=0$ for any $r\geq r_{b}$. This equality
holds only when $\rho=P=0$ for any $r\geq r_{b}$, since ${\cal A}<0$ due to
(2.5), and the condition (3.1) with $P_{r}=0$ now implies $P\leq 0$, while
$\rho\geq 0$ due to WEC. $\Box$

Remarks.

  * •

The last part of the statement that the DOC with vanishing $T^{\mu}_{\nu}$
must corresponds to the generalized Schwarzschild spacetime in
$(2+n)$-dimensions conforms to the uniqueness theorem shown in [39].

  * •

The above condition (3.1) has been employed in [9]. This is, for example, the
case of a static configuration of minimally coupled real scalar field (see
also below for time-dependent complex scalar case). Note also that this is not
the case for Maxwell fields, in accord with the Reissner-Nordstrom black hole,
which is not viewed as a hairy black hole as it has the conserved charge
defined at infinity.

  * •

As in [9], Theorem 1 can also apply to harmonically time-dependent complex
scalar fields [11, 12], which can be expressed as

$\phi(t,r,z^{i})=e^{-i\omega t}\Phi(r)\mathbb{S}(z),$ (3.2)

where $\Phi(r)$ is a real function and $\mathbb{S}(z)$ denotes the scalar
harmonics on ${\cal K}_{n}$. The EMT for such a complex scalar field is given
by

$T_{\mu\nu}(\phi)=\dfrac{1}{2}\left(\nabla_{\mu}\phi^{*}\nabla_{\nu}\phi+\nabla_{\mu}\phi\nabla_{\nu}\phi^{*}\right)-\dfrac{1}{2}g_{\mu\nu}\left\\{\nabla^{\lambda}\phi^{*}\nabla_{\lambda}\phi+2V(|\phi|)\right\\},$
(3.3)

and we can read off

$\displaystyle\rho=\dfrac{\omega^{2}}{2\mu
e^{-2\delta}}\Phi^{2}+\dfrac{1}{2}\mu\Phi^{\prime}{}^{2}+V(\Phi),\quad
P_{r}=\rho-2V(\Phi),\quad P=P_{r}-\mu\Phi^{\prime}{}^{2}.$ (3.4)

Then, we can see that $\rho,P_{r},P$ in fact satisfy not only WEC but also the
requirement (3.1), and therefore apply Theorem 1.

  * •

In Table 1, we list some known hairy black hole solutions, which all satisfy
WEC but violate the condition (3.1) outside the horizon. For comparison, we
also include the Reissner-Nordstrom solution, which is not classified as a
hairy black hole.

Table 1: Energy conditions and some example of hairy black holes: Reissner-Nordstrom solution (RN), Bardeen solution, Hayward solution [40], Culetu-Simpson-Visser solution (CSV) [41, 42, 43], and modified-Culetu-Simpson-Visser solution (mCSV) [44]. They all satisfy WEC but violate the condition (3.1) outside the horizon.  |  | WEC |  Condition (3.1)
---|---|---|---
Solutions |  | $8\pi\rho$ | ${}\,8\pi(\rho+P_{r})\,$ | $8\pi(\rho+P)$ | $8\pi(P_{r}-P)$
RN |  | $\dfrac{Q^{2}}{r^{4}}$ | $0$ | $\dfrac{2Q^{2}}{r^{4}}$ | $-\dfrac{2Q^{2}}{r^{4}}$
Bardeen |  | $\dfrac{6MQ^{2}}{r^{5}}$ | $0$ | $\dfrac{15MQ^{2}}{r^{5}}$ | $-\dfrac{15MQ^{2}}{r^{5}}$
Hayward |  | $\dfrac{6ML^{3}}{(r^{3}+L^{3})^{2}}$ | $0$ | $\dfrac{18ML^{3}r^{3}}{(r^{3}+L^{3})^{3}}$ | $-\dfrac{18ML^{3}r^{3}}{(r^{3}+L^{3})^{3}}$
CSV |  | $\dfrac{2Ma}{r^{4}}e^{-{a}/{r}}$ | $0$ | $\dfrac{Ma(4r-a)}{r^{5}}e^{-{a}/{r}}$ | $-\dfrac{Ma(4r-a)}{r^{5}}e^{-{a}/{r}}$
mCSV |  | $\dfrac{4Ma^{2}}{r^{5}}e^{-{a^{2}}/{r^{2}}}$ | $0$ | $\dfrac{2Ma^{2}(5r^{2}-2a^{2})}{r^{7}}e^{-{a^{2}}/{r^{2}}}$ | $-\dfrac{2Ma^{2}(5r^{2}-2a^{2})}{r^{7}}e^{-{a^{2}}/{r^{2}}}$

We also have the following theorem concerning “hair length” as a
generalization of Nunez-Quevedo-Sudarsky’s no-short-hair theorem [10] to
higher dimensions.

Theorem 2. Consider the class of static black hole spacetimes considered in
Theorem 1. We assume that the generalized Misner-Sharp mass $m(r)$ is
positive, (2.5), and asymptotically behaves as (2.4) and also that the matter
fields satisfy WEC, (2.9), as in Theorem 1. We further require that $r^{n+2}E$
vanishes at infinity. If furthermore the trace of the EMT is negative, then
$r^{n+1}E$ is negative semidefinite at the horizon $r_{b}$ and is decreasing
between $r_{b}$ and $r_{0}$, where $r_{0}>r_{\star}:=[(n+1)/2]^{1/(n-1)}r_{b}$
and for some $r>r_{0}$, the function $r^{n+2}E$ begins to increase toward its
asymptotic value, 0.

Proof. As shown in the proof of Theorem 1, only for the case $k=1$, the metric
(2.1) with (2.3) admits a static black hole. From (2.19) we can obtain the
following formula

$(r^{n+2}E)^{\prime}=\frac{r^{n+1}e^{-\delta}}{2\mu}\left\\{{\cal
C}\left(\rho+P_{r}\right)+2\mu T\right\\},$ (3.5)

where ${\cal C}:=(n+1)\mu-(n-1)$ and $T$ denotes the trace of EMT. As has
already been shown in the proof of Theorem 1, $E(r_{b})\leq 0$, hence
$r_{b}^{n+2}E(r_{b})\leq 0$. By WEC $\rho+P_{r}\geq 0$ and by assumption
$T<0$, we find that $(r^{n+2}E)^{\prime}<0$, and hence $r^{n+2}E$ is a
decreasing function, unless ${\cal C}>0$. Since at the horizon $r_{b}$, the
regularity implies that $(\rho+P_{r})/\mu$ is finite, and ${\cal
C}(r_{b})=-(n-1)<0$, hence $(r^{n+2}E)^{\prime}\leq 0$ at $r_{b}$. Thus,
$r^{n+2}E$ is a nagative, decreasing function between $r_{b}$ and some
$r_{0}>r_{1}$, where $r_{1}$ is given by ${\cal C}(r_{1})=0$. By
straightforward calculation, one finds $r_{1}^{n-1}=(n+1)m(r_{1})$. Since the
generalized Misner-Sharp mass is non-decreasing as shown by (2.12), it follows
that
$r_{0}^{n-1}>r_{1}^{n-1}=(n+1)m(r_{1})\geq(n+1)m(r_{b})=(n+1)r_{b}^{n-1}/2$.
$\Box$

Remarks.

  * •

The requirement that $r^{n+2}E(=r^{n+2}e^{-\delta}P_{r})$ goes to zero at
infinity $r\rightarrow\infty$ may be somewhat more stringent condition than
our asymptotic condition (2.8), in particular, than the standard definition of
asymptotic flatness in $4$-dimensions. For example, $r^{4}e^{-\delta}P_{r}$
for Maxwell fields in $4$-dimensions remains to be non-vanishing at infinity,
as pointed out in [10]. Note however that in higher dimensions, Maxwell fields
satisfy this requirement.

  * •

From Theorem 2, we have $r_{\star}^{n+2}E(r_{\star})<r_{b}^{n+2}E(r_{b})\leq
0$, which is rewritten in terms of the radial pressure $P_{r}<0$ as

$|P_{r}(r_{\star})|>\left(\dfrac{r_{b}}{r_{\star}}\right)^{n+2}|P_{r}(r_{b})|e^{\delta(r_{\star})-\delta(r_{b})}.$
(3.6)

By integrating (2.11), we have

$\delta(r_{\star})-\delta(r_{b})=-\dfrac{8\pi}{n}\int_{r_{b}}^{r_{\star}}dr\dfrac{r(\rho+P_{r})}{\mu}.$
(3.7)

This provides a bound on $\delta(r_{\star})-\delta(r_{b})$ in terms of the
maximal value of the matter fields in the interval $[r_{b},r_{\star}]$.
Substituting this into the right-hand side of (3.6), we can, in principle,
obtain a bound on the radial pressure $P_{r}$ at the point $r_{\star}$.
Following a similar argument of [10], we can claim that a black hole hair, if
exists, must extend beyond the point $r_{\star}$.

###  B Asymptotically de Sitter case ($\Lambda>0$)

For $k=0,-1$, we have $\mu\leq 0$ due to (2.5) and the positivity $\lambda>0$.
Hence the only relevant case is $k=1$, for which it immediately follows from
(2.20) that ${\cal A}<0$ while ${\cal B}>0$ in DOC. Then, with WEC and the
condition (3.1), we find that $E^{\prime}\leq 0$, namely that $E$ is a non-
increasing function of $r$ in DOC, thus in particular $E(r_{b})\geq E(r_{c})$.
Note that as in the asymptotically flat case, one can rescale the time
coordinate $t$ so that $\delta(r_{b})=0$. It follows from (2.11) and WEC that
$\delta(r)$ becomes a non-increasing function in DOC, and hence,
$e^{-\delta(r_{c})}\geq e^{-\delta(r_{b})}=1$. Now let us introduce the non-
negative constant
$\epsilon:=[{n(n+1)}/{16\pi}]\lambda\left(e^{-\delta(r_{c})}-1\right)\geq 0$.
Then, we obtain the following theorem for static black holes with positive
cosmological constant:

Theorem 3. Consider a $(2+n)$-dimensional static black hole spacetime
$(M,g_{\mu\nu})$ described by the metric (2.1) which satisfies the Einstein
equations with a positive cosmological constant $\Lambda>0$ and which is
bounded by two non-degenerate regular Killing horizons, i.e., a black hole
horizon at $r_{b}$ and a cosmological horizon at $r_{c}>r_{b}$. If
furthermore, the positivity (2.5) of the generalized Misner-Sharp mass $m(r)$
holds and the matter fields satisfy the WEC (2.9) and the conditions (3.1),
then the matter fields must satisfy the following condition:

$\begin{split}\rho(r_{b})&\leq\rho(r_{c})e^{-\delta(r_{c})}+\epsilon.\end{split}$
(3.8)

Proof. By using the definition (2.18) of $E$ as well as the regularity
condition (2.16), the condition $E(r_{b})\geq E(r_{c})$ can be expressed in
terms of the energy density $\rho$ of the matter field at the horizons,
resulting in (3.8). $\Box$

Remarks.

  * •

Although the inequality (3.8) is a non-trivial condition due to the factor
$\delta(r_{c})$ and $\epsilon$, it can roughly be interpreted that in order
for a static black hole to have a non-trivial hair stemming from
$T^{\mu}_{\nu}$, the matter field should not decay so quickly toward the
cosmological horizon to meet (3.8). In other words, the hair of a static black
hole, if exists under WEC and (3.1), must be long enough so that it reaches
the cosmological horizon. In this sense, Theorem 3 above may be viewed as a
generalization of the no-short hair theorem of [10] to include a positive
cosmological constant. Note that some known example, e.g., MTZ solution [45]
and BdS solution [46] violate the condition (3.8) but have hair since they
violate the condition (3.1).

  * •

When $\Lambda$ vanishes, $r_{c}\rightarrow\infty$, $\rho(r_{c})\rightarrow 0$
for the asymptotic condition, the condition (3.8) indicates a possible
violation of WEC.

  * •

Theorem 3 indicates that if the condition (3.8) is not satisfied, then
$T^{\mu}_{\nu}$ would vanish identically in DOC and such a black hole
$(M,g_{\mu\nu})$ should correspond to the static region of a generalized
Schwarzschild-de Sitter spacetime with $({\cal K}_{n},k=1)$. In this respect,
we note that a uniqueness theorem for Schwarzschild-de Sitter spacetime in
$4$-dimensions ($n=2$) has recently been shown [22, 23].

###  C Asymptotically AdS case ($\Lambda<0$)

In the asymptotically AdS case, we have the following no-hair theorem, which
generalizes Sudarsky-Gonzalez’s theorem [34] to higher-dimensional case.

Theorem 4. Consider a $(2+n)$-dimensional static black hole spacetime
$(M,g_{\mu\nu})$ described by the metric (2.1) and which has a non-degenerate
regular black hole horizon at $r_{b}$ and satisfies the Einstein equations
with a negative cosmological constant $\Lambda<0$ and matter fields with
$T^{\mu}_{\nu}$. We assume that the generalized Misner-Sharp mass $m(r)$ is
positive, (2.5), and asymptotically behaves as (2.4), and also that the matter
fields satisfy the WEC (2.9) and asymptotically behave as (2.8). If further
the matter fields satisfy the following conditions,

$\displaystyle\rho+P_{r}\leq\dfrac{2n}{(n-1)(2-k)}(P_{r}-P),$ (3.9)

then there are no nontrivial static black hole solutions with $k=0,1$ in the
asymptotically AdS spacetime whose asymptotic behavior corresponds to the AdS
spacetime with the true cosmological constant $\lambda=2\Lambda/n(n+1)<0$.

Proof. The formula (2.17) can be cast into the following form,

$\displaystyle(e^{-\delta}P_{r})^{\prime}$ $\displaystyle=$
$\displaystyle\dfrac{e^{-\delta}}{2\mu
r}\left[-\\{(n-1)k+(n+1)(-\lambda)r^{2}\\}\left(\rho+P_{r}\right)\right.$
(3.10)
$\displaystyle\qquad\left.+\mu\left\\{(n-1)(2-k)(\rho+P_{r})-2n(P_{r}-P)\right\\}\right].$

It is straightforward to see that when $k=0,1$, the first term of the right-
hand side of (3.10) is always non-positive as $\lambda<0$ and WEC. The
condition (3.9) ensures the non-positivity of the the second term of the
right-hand side of (3.10). Then, $e^{-\delta}P_{r}$ is a non-increasing
function in $r\geq r_{b}$, and we can derive a contradiction by similar
argument in the proof of Theorem 1. $\Box$

Remarks.

  * •

When $\rho+P_{r}=P_{r}-P$, the condition (3.9) holds for $k=0,1$ in any
dimensions $n\geq 2$, while for $k=-1$, it holds only in the case $n=2,3$.
This is, for example, the case of a minimally coupled scalar field, as studied
in $4$-dimensional spherical black hole case [34].

  * •

If, for example, a scalar field admits non-vanishing potential $V(\phi)$ at
infinity, namely, if it does not satisfy our asymptotic conditions, then a
non-trivial hairy static black hole is allowed, in which the effective
cosmological constant $\Lambda_{\rm eff}:=\Lambda+8\pi V(\phi_{\rm\infty})$
determines its asymptotic structure, as explained in [34].

  * •

For static, vacuum spacetimes with a negative cosmological constant,
uniqueness theorems have been proposed with some further assumptions (see,
e.g., [47, 48]).

##  4 Summary

We have shown the four theorems concerning no-hair properties of static black
holes in four and higher dimensional asymptotically locally flat, de Sitter,
and AdS spacetimes. Theorems 1, 2, and 4 correspond, respectively, to higher-
dimensional generalizations of Pena-Sudarsky’s theorem [9], Nunez-Quevedo-
Sudarsky’s theorem [10], and Sudarsky-Gonzalez’ theorem [34]. All our theorems
assume that the static metric takes the warped product form of (2.1), admits
regular non-degenerate Killing horizons as a black hole event horizon and also
a cosmological horizon for $\Lambda>0$ case, the Misner-Sharp local mass
function is positive definite, and matter fields satisfy WEC. Then, each
theorem further requires its own additional assumption, such as (3.1) in
Theorem 1 and the negative trace of energy-momentum tensor in Theorem 2. As
shown in Table 1, some known hairy black hole solutions satisfy WEC but
violate our additional condition (3.1).

In the present paper, for simplicity we have assumed that the Killing horizons
are non-degenerate ones. There seems to be no serious obstruction to
generalize the present four theorems to involve degenerate horizons. It is
worth studying whether we can further relax our conditions to obtain similar
theorems for extremal black holes.

Acknowledgments

This work was supported in part by JSPS KAKENHI Grant No. 20K03938 and also
supported by MEXT KAKENHI Grant-in-Aid for Transformative Research Areas A
Extreme Universe No. 21H05182, 21H05186. We are grateful to the long term
workshop YITP-T-23-01 held at YITP, Kyoto University, where a part of this
work was done.

## References

  * [1] W. Israel, Phys. Rev. 164, 1776-1779 (1967) doi:10.1103/PhysRev.164.1776
  * [2] W. Israel, Commun. Math. Phys. 8, 245-260 (1968) doi:10.1007/BF01645859
  * [3] B. Carter, Phys. Rev. Lett. 26, 331-333 (1971) doi:10.1103/PhysRevLett.26.331
  * [4] D. C. Robinson, Phys. Rev. D 10, 458-460 (1974) doi:10.1103/PhysRevD.10.458
  * [5] D. C. Robinson, Phys. Rev. Lett. 34, 905-906 (1975) doi:10.1103/PhysRevLett.34.905
  * [6] P. O. Mazur, J. Phys. A 15, 3173-3180 (1982) doi:10.1088/0305-4470/15/10/021
  * [7] P. T. Chrusciel, J. Lopes Costa and M. Heusler, Living Rev. Rel. 15, 7 (2012) doi:10.12942/lrr-2012-7 [arXiv:1205.6112 [gr-qc]].
  * [8] J. D. Bekenstein, Phys. Rev. D 5, 1239-1246 (1972) doi:10.1103/PhysRevD.5.1239
  * [9] I. Pena and D. Sudarsky, Class. Quant. Grav. 14, 3131-3134 (1997) doi:10.1088/0264-9381/14/11/013
  * [10] D. Nunez, H. Quevedo and D. Sudarsky, Phys. Rev. Lett. 76, 571-574 (1996) doi:10.1103/PhysRevLett.76.571 [arXiv:gr-qc/9601020 [gr-qc]].
  * [11] K. Lin, S. Zhang, C. Zhang, X. Zhao, B. Wang and A. Wang, Phys. Rev. D 102, no.2, 024034 (2020) doi:10.1103/PhysRevD.102.024034 [arXiv:2004.04773 [gr-qc]].
  * [12] S. Matsumoto, Class. Quant. Grav. 40, no.17, 175011 (2023) doi:10.1088/1361-6382/ace94e [arXiv:2210.03966 [gr-qc]].
  * [13] P. Bizon, Phys. Rev. Lett. 64, 2844-2847 (1990) doi:10.1103/PhysRevLett.64.2844
  * [14] D. Garfinkle, G. T. Horowitz and A. Strominger, Phys. Rev. D 43, 3140 (1991) [erratum: Phys. Rev. D 45, 3888 (1992)] doi:10.1103/PhysRevD.43.3140
  * [15] P. Bizon and T. Chmaj, Phys. Lett. B 297, 55-62 (1992) doi:10.1016/0370-2693(92)91069-L
  * [16] G. V. Lavrelashvili and D. Maison, Nucl. Phys. B 410, 407-422 (1993) doi:10.1016/0550-3213(93)90441-Q
  * [17] B. R. Greene, S. D. Mathur and C. M. O’Neill, Phys. Rev. D 47, 2242-2259 (1993) doi:10.1103/PhysRevD.47.2242 [arXiv:hep-th/9211007 [hep-th]].
  * [18] U. Nucamendi and M. Salgado, Phys. Rev. D 68, 044026 (2003) doi:10.1103/PhysRevD.68.044026 [arXiv:gr-qc/0301062 [gr-qc]].
  * [19] C. A. R. Herdeiro, E. Radu, N. Sanchis-Gual and J. A. Font, Phys. Rev. Lett. 121, no.10, 101102 (2018) doi:10.1103/PhysRevLett.121.101102 [arXiv:1806.05190 [gr-qc]].
  * [20] P. G. S. Fernandes, C. A. R. Herdeiro, A. M. Pombo, E. Radu and N. Sanchis-Gual, Class. Quant. Grav. 36, no.13, 134002 (2019) [erratum: Class. Quant. Grav. 37, no.4, 049501 (2020)] doi:10.1088/1361-6382/ab23a1 [arXiv:1902.05079 [gr-qc]].
  * [21] N. M. Santos, C. L. Benone, L. C. B. Crispino, C. A. R. Herdeiro and E. Radu, JHEP 07, 010 (2020) doi:10.1007/JHEP07(2020)010 [arXiv:2004.09536 [gr-qc]].
  * [22] A.K.M. Masood-ul-Alam and W. Yu, Communications in Analysis and Geometry, 23, 377-387 (2015).
  * [23] S. Borghini, P. T. Chruściel and L. Mazzieri, Arch. Ration. Mech. Anal. 247, no.2, 1-35 (2023) doi:10.1007/s00205-023-01860-1 [arXiv:1909.05941 [math.DG]].
  * [24] R. Emparan and H. S. Reall, Living Rev. Rel. 11, 6 (2008) doi:10.12942/lrr-2008-6 [arXiv:0801.3471 [hep-th]].
  * [25] S. Hollands and A. Ishibashi, Class. Quant. Grav. 29, 163001 (2012) doi:10.1088/0264-9381/29/16/163001 [arXiv:1206.1164 [gr-qc]].
  * [26] D. Ida, A. Ishibashi and T. Shiromizu, Prog. Theor. Phys. Suppl. 189, 52-92 (2011) doi:10.1143/PTPS.189.52 [arXiv:1105.3491 [hep-th]].
  * [27] J. M. Maldacena, “The Large N limit of superconformal field theories and supergravity,” Adv. Theor. Math. Phys. 2, 231-252 (1998) [arXiv:hep-th/9711200 [hep-th]].
  * [28] S. S. Gubser, I. R. Klebanov and A. M. Polyakov, “Gauge theory correlators from noncritical string theory,” Phys. Lett. B 428, 105-114 (1998) [arXiv:hep-th/9802109 [hep-th]].
  * [29] E. Witten, “Anti-de Sitter space and holography,” Adv. Theor. Math. Phys. 2, 253-291 (1998) [arXiv:hep-th/9802150 [hep-th]].
  * [30] T. Torii, K. Maeda and M. Narita, Phys. Rev. D 64, 044007 (2001) doi:10.1103/PhysRevD.64.044007
  * [31] E. Winstanley, Class. Quant. Grav. 16, 1963-1978 (1999) doi:10.1088/0264-9381/16/6/325 [arXiv:gr-qc/9812064 [gr-qc]].
  * [32] E. Winstanley, Found. Phys. 33, 111-143 (2003) doi:10.1023/A:1022871809835 [arXiv:gr-qc/0205092 [gr-qc]].
  * [33] Y. Brihaye, C. Herdeiro and E. Radu, Phys. Lett. B 802, 135269 (2020) doi:10.1016/j.physletb.2020.135269 [arXiv:1910.05286 [gr-qc]].
  * [34] D. Sudarsky and J. A. Gonzalez, Phys. Rev. D 67, 024038 (2003) doi:10.1103/PhysRevD.67.024038 [arXiv:gr-qc/0207069 [gr-qc]].
  * [35] H. Maeda, Phys. Rev. D 73, 104004 (2006) doi:10.1103/PhysRevD.73.104004 [arXiv:gr-qc/0602109 [gr-qc]].
  * [36] L. B. Szabados, Living Rev. Rel. 12, 4 (2009) doi:10.12942/lrr-2009-4
  * [37] C. C. M. Liu and S. T. Yau, Phys. Rev. Lett. 90, 231102 (2003) doi:10.1103/PhysRevLett.90.231102 [arXiv:gr-qc/0303019 [gr-qc]].
  * [38] C. C. M. Liu and S. T. Yau, [arXiv:math/0412292 [math.DG]].
  * [39] G. W. Gibbons, D. Ida and T. Shiromizu, Phys. Rev. Lett. 89, 041101 (2002) doi:10.1103/PhysRevLett.89.041101 [arXiv:hep-th/0206049 [hep-th]].
  * [40] S. A. Hayward, Phys. Rev. Lett. 96, 031103 (2006) doi:10.1103/PhysRevLett.96.031103 [arXiv:gr-qc/0506126 [gr-qc]].
  * [41] H. Culetu, [arXiv:1305.5964 [gr-qc]].
  * [42] H. Culetu, Int. J. Theor. Phys. 54, no.8, 2855-2863 (2015) doi:10.1007/s10773-015-2521-6 [arXiv:1408.3334 [gr-qc]].
  * [43] A. Simpson and M. Visser, Universe 6, no.1, 8 (2019) doi:10.3390/universe6010008 [arXiv:1911.01020 [gr-qc]].
  * [44] L. Xiang, Y. Ling and Y. G. Shen, Int. J. Mod. Phys. D 22, 1342016 (2013) doi:10.1142/S0218271813420169 [arXiv:1305.3851 [gr-qc]].
  * [45] C. Martinez, R. Troncoso and J. Zanelli, Phys. Rev. D 67, 024008 (2003) doi:10.1103/PhysRevD.67.024008 [arXiv:hep-th/0205319 [hep-th]].
  * [46] S. Fernando, Int. J. Mod. Phys. D 26, no.07, 1750071 (2017) doi:10.1142/S0218271817500717 [arXiv:1611.05337 [gr-qc]].
  * [47] W. Boucher, G. W. Gibbons and G. T. Horowitz, Phys. Rev. D 30, 2447 (1984) doi:10.1103/PhysRevD.30.2447
  * [48] M. T. Anderson, P. T. Chrusciel and E. Delay, JHEP 10, 063 (2002) doi:10.1088/1126-6708/2002/10/063 [arXiv:gr-qc/0211006 [gr-qc]].


# R. A. Fisher’s Exact Test Revisited

Martin Mugnier
Department of Economics, University of Oxford  Mailing address: 10 Manor Road,
Oxford, OX1 3UQ, United Kingdom. Contact<EMAIL_ADDRESS>The author gratefully acknowledges Stéphane Bonhomme for his comments.

###### Abstract

This note provides a conceptual clarification of Ronald Aylmer Fisher’s (1935)
pioneering exact test in the context of the Lady Testing Tea experiment. It
unveils a critical implicit assumption in Fisher’s calibration: the taster
minimizes expected misclassification given fixed probabilistic information.
Without similar assumptions or an explicit alternative hypothesis, the
rationale behind Fisher’s specification of the rejection region remains
unclear.

Keywords: Hypothesis testing, Null hypothesis, Information theory, Shannon
entropy.

##  1 Introduction

A colleague of Ronald Aylmer Fisher asserts that they can distinguish tea
poured into milk from milk poured into tea. To convince himself, Fisher
prepares eight cups of tea, four with tea poured into milk (confidentially
labelled $TM$) and four with milk poured into tea (confidentially labelled
$MT$). He presents the cups in random order to his colleague, who is asked to
taste and assign one of the labels $\\{TM,MT\\}$ to each cup. Before tasting,
the colleague knows that there are four $TM$ cups and four $MT$ cups among the
eight, but they do not know the specific order in which Fisher arranged the
cups.111I am indebted to Ritzwoller et al. (2024) for inspiring this short
paper and introduction. This experiment is reported in Fisher (1935), p.13-29.

##  2 A disconnect between Fisher’s null hypothesis and exact test

### Fisher’s null hypothesis

Distinguishing $TM$ cups from $MT$ cups is a rather surprising skill. Fisher
considers the null hypothesis that all possible assignments of eight cups into
two groups of four are equally probable answers. He disbelieves that his
colleague can do better or worse than a fair coin toss prediction unless they
provide some experimental evidence. Specifically, Fisher states

> The two classes of results which are distinguished by our test of
> significance are, on the one hand, those which show a significant
> discrepancy from a certain hypothesis; namely, in this case, the hypothesis
> that the judgments given are in no way influenced by the order in which the
> ingredients have been added; and on the other hand, results which show no
> significant discrepancy from this hypothesis. (Fisher, 1935, p.18)

Without further assumption, experimental evidence can be understood as
observing some event reflecting a “sufficiently large deviation” from what the
uniform distribution over the set of all possible assignments would “typically
predict”. Such an event may be perfect distinction in the prediction sense,
which occurs if each time the experimenter presents $TM$ (resp. $MT$), the
taster claims $TM$ (resp. $MT$). Another one is perfect distinction in the
weak information sense, which occurs if each time the experimenter presents
$TM$ (resp. $MT$), the taster claims $MT$ (resp. $TM$). Note that perfect
distinction in the weak informative sense implies perfect distinction in the
predictive sense up to applying the deterministic bijective transformation
$g:\\{TM,MT\\}\to\\{TM,MT\\}$ defined as $g(TM)=MT$ and $g(MT)=TM$ to each
claim. In other words, both extreme events equally suggest that the taster can
“distinguish” by using some knowledge about $TM$ and $MT$ to recover the
correct equivalence classes of cups invariant to relabeling.

### Fisher’s exact test

Let $Y_{N}=(Y_{1N},\ldots,Y_{NN})^{\prime}$ denote the $N$ cup labels
corresponding to the random ordering chosen by Fisher, among which $n$ are
$TM$ and $N-n$ are $MT$. Let $X_{N}=(X_{1N},\ldots,X_{NN})^{\prime}$ denote
the (random) sequence of answers from his colleague. Fisher’s null hypothesis
is an assumption on the joint distribution $P_{(X_{N},Y_{N})}$. Specifically,
it says that the conditional distribution $P_{X_{N}|Y_{N}}$ is the uniform
distribution over the set

$\mathcal{X}_{N,n}:=\left\\{x\in\\{TM,MT\\}^{N}:\sum_{j=1}^{N}1\\{x_{j}=TM\\}=n\right\\}.$

In Fisher’s experiment, $N=8$, $n=4$, and $|\mathcal{X}_{8,4}|={8\choose
4}=70$ where for any finite set $A$, $|A|$ denotes the cardinality of $A$.
Fisher randomly draws $Y_{8}\in\mathcal{X}_{8,4}$ and his colleague answers
$X_{8}\in\mathcal{X}_{8,4}$. Under the null hypothesis, for any given
$Y_{8}\in\mathcal{X}_{8,4}$, there is a 1/70 chance of observing only right
answers, i.e., $X_{8}=Y_{8}$. Fisher rejects the null hypothesis if and only
if $X_{8}=Y_{8}$. The test is exact at the significance level $5\%$ since the
probability to reject under the null, i.e., to make a Type I error, is
$1/70\approx 0.014\leq 5\%$. This probability does not depend on $Y_{8}$: it
is the same both conditionally and unconditionally on $Y_{8}$.

### A disconnect

Though Fisher does not specify an alternative hypothesis, he chooses a
rejection region in favour of perfect distinction in the predictive sense. Why
ignore the region in favour of perfect distinction in the weak information
sense? Certainly not for significance-level considerations. Under the null,
the event of observing no right answer, $\cap_{j=1}^{8}\\{X_{j8}\neq
Y_{j8}\\}$, also occurs with probability 1/70 so that the exact test that
rejects the null if and only if $\cap_{j=1}^{8}\\{X_{j8}\neq
Y_{j8}\\}\cup\\{X_{8}=Y_{8}\\}$ is observed is at least as powerful against
all alternatives, is more powerful against some alternatives, and enjoys the
same significance level since $2/70\approx 0.029\leq 5\%$.

In this experiment, the colleague ends up making correct predictions for all
cups, which implies that both tests reject the null at the significance level
$5\%$ (the size of the more powerful test, however, is higher). Despite that
running the more powerful test does not change the conclusion in this
particular experiment, there is a priori no theoretical justification for
preferring the original exact test. More concerning, the exact $5\%$-level
test that rejects the null if and only if $\cap_{j=1}^{8}\\{X_{j8}\neq
Y_{j8}\\}$ is observed, although equally legitimate, does not reject the null
in this experiment.

Fisher is one of the most celebrated statisticians and it is of interest to
investigate what could have motivated the disconnect between his null
hypothesis and his proposed exact test. A natural justification is that Fisher
has an alternative hypothesis in mind. Others have gathered around the table,
and the only reason his colleague might challenge him is to demonstrate to the
community that they can do “better” than a random guess in the commonly
accepted sense, i.e., the prediction sense. Fisher is willing to reject only
based on evidence pointing to that direction. This justification, however, is
rather unsatisfactory for at least two reasons. First, Neyman & Pearson (1928)
are credited for introducing the concept of alternative hypothesis after
Fisher’s first formulation of his methodology (Fisher, 1925). Second, Fisher
himself later opposed the use of an alternative hypothesis in the 1966 edition
of his book (Fisher, 1935).

The next section provides another rigorous justification taking the form of an
explicit behavioural hypothesis on the taster that avoids explicitly invoking
any particular alternative hypothesis. It amounts to formalizing a single
crucially important sentence in the original text, though left over in the
sequel and often overlooked in modern expositions:

> Her task [the taster] is to divide the 8 cups into two sets of 4, agreeing,
> if possible, with the treatments received.222Italics and text into square
> brackets added. (Fisher, 1935, p.14)

Fisher implicitly assumes that he and the taster agree that the taster will
use their discriminating ability, if any (that is under any alternative), to
minimize misclassification. While this assumption is innocuous to stating the
null hypothesis, it is central to justifying Fisher’s test rejection region,
as it implies stochastic dominance relationships between distributions of
success under different levels of probabilistic information. Theorem 3.1
clarifies this point by formalizing the minimization problem within a
statistical decision and information-theoretic framework.

##  3 A reconciling statistical decision and information-theoretic framework

As emphasized in the previous section, a possible source of confusion
regarding the mapping from Fisher’s null hypothesis to his exact test is that
the ability to distinguish has more to do with information than correct
prediction. More than 70 years ago (and thus posterior to Fisher), Claude
Shannon introduced the concept of entropy or Shannon information.333See
Shannon (1948). The main contribution of this note is to employ this key
concept from information theory to eliminate any disconnect by giving a
mathematically precise, behaviourally sound and rigorous justification for
Fisher (1935)’s exact test in the Lady Testing Tea experiment.

### Misclassification loss

Consider the classical binary misclassification loss $\ell$, popular in the
statistical learning and decision literature: for all
$x_{N},y_{N}\in\mathcal{X}_{N,n}$,

$\ell(x_{N},y_{N}):=\sum_{j=1}^{N}1\\{x_{jN}\neq y_{jN}\\}.$

The theoretical expected predictive performance of the taster in terms of loss
$\ell$ and distribution of answers $P_{X_{N}|Y_{N}}$ can be visualized as an
arrow “indexing the joint distributions” from the left (systematically less
correct than random with the extremity being “always wrong” if almost surely
$\cap_{j=1}^{8}\\{X_{j8}\neq Y_{j8}\\}$ and thus $\ell(X_{8},Y_{8})=8$)
through the centre (as correct as random) to the right (systematically more
correct than random with the extremity being “always correct” if almost surely
$X_{8}=Y_{8}$ and thus $\ell(X_{8},Y_{8})=0$). Clearly, observing
$\cap_{j=1}^{8}\\{X_{j8}\neq Y_{j8}\\}$ provides evidence against the null in
the direction to the left (previously called “information sense”) while
observing $X_{8}=Y_{8}$ provides evidence against the null in the direction to
the right (previously called “prediction sense”).

### Misclassification minimization subject to an information constraint

For any distribution $P$ supported on a discrete set $\mathcal{X}$, define the
probabilistic entropy, or Shannon information, as

$H(P)=-\sum_{x\in\mathcal{X}}P(x)\log(P(x)).$

Note that $H(X)\in[0,\log(|\mathcal{X}|)]$ is uniquely maximized at the
uniform distribution $\overline{P}$ on $\mathcal{X}$, and minimized for Dirac
distributions
$\underline{P}\in\mathcal{D}(\mathcal{X}):=\\{\delta_{x}:x\in\mathcal{X}\\}$.
These facts are useful to show the main result of the paper stated below.

###### Theorem 1

Let $\overline{h}:=\log(|\mathcal{X}_{N,n}|)$. For $h\in(0,\overline{h}]$, let
$P_{X_{N}|Y_{N}}^{*}(h)$ denote an interior solution to the minimization
problem

$\begin{array}[]{ll}&\min_{P_{X_{N}|Y_{N}}}\mathbb{E}_{P_{X_{N}|Y_{N}}}[\ell(X_{N},Y_{N})]\\\
&\text{ subject to }\quad H(P_{X_{N}|Y_{N}})\geq h.\end{array}$

Then, $P_{X_{N}|Y_{N}}^{*}(h)$ is uniquely defined and such that

  1. 1.

$H(P^{*}_{X_{N}|Y_{N}}(h))=h$,

  2. 2.

$\lim_{h\to
0^{+}}\sup_{x\in\mathcal{X}_{N,n}}|P_{X_{N}|Y_{N}}^{*}(h)(x)-\delta_{Y_{N}}(x)|=0$,

  3. 3.

$P_{X_{N}|Y_{N}}^{*}(\overline{h})={\rm Uniform}(\mathcal{X}_{N,n})$,

  4. 4.

For all $h,h^{\prime}\in(0,\overline{h}]$ such that $h\leq h^{\prime}$,
$\ell(X_{N},Y_{N})$ under $P_{X_{N}|Y_{N}}^{*}(h^{\prime})$ first-order
stochastically dominates $\ell(X_{N},Y_{N})$ under $P_{X_{N}|Y_{N}}^{*}(h)$.

Points 1-3 of Theorem 1 say that if the taster chooses their answers
($P_{X_{N}|Y_{N}}^{*}(h)$) to minimize the expected misclassification error
$\mathbb{E}_{P_{X_{N}|Y_{N}}}[\ell(X_{N},Y_{N})]$ given a maximum level of
information $\overline{h}-h$, then i) the taster uses all available
information, ii) the taster with full information ($h=0^{+}$) will almost
surely give all answers right, and iii) Fisher’s null hypothesis corresponds
to a taster with no information ($h=\overline{h}$). Point 4 is the most
enlightening result since it justifies Fisher’s exact test irrelevant of any
alternative. Specifically, the more the taster deviates from the null
hypothesis of no information ($h=\overline{h}$), the more their answers
($P_{X_{N}|Y_{N}}^{*}(h)$) generate a distribution of successes
($N-\ell(X_{N},Y_{N})$) that has a heavy right tail. Such a minimizing
behaviour disciplines the taster’s response as a function of their information
and solves the previous inconsistency of “two-sided” testing with a one-sided
test by selecting a unique direction. Corollary 1 displays the main
implications of this result for Fisher’s exact test. Figure 1 provides an
illustration for binary and ternary outcome distributions. The optimal path as
a function of $h$ can be traced out starting from the red dot (maximum entropy
distribution) and minimizing the distance to the green dot (maximum payoff)
along the level curves. Analogous results hold for a maximizing taster.

###### Corollary 1

Suppose the taster minimizes expected misclassification given a maximum
probabilistic information level. Then,

  1. 1.

For all $x,y\in\mathcal{X}_{8,4}$,

$\displaystyle\ell(x,y)\leq\mathbb{E}_{\text{under the
null}}[\ell(X_{8},y)|Y_{8}=y]$ $\displaystyle\quad\iff\Pr_{\text{under any
alternative}}(X_{8}=x|Y_{8}=y)$ $\displaystyle\geq$
$\displaystyle\Pr_{\text{under the null}}(X_{8}=x|Y_{8}=y).$

  2. 2.

Fisher’s exact test has power 1 against the alternative of full information.

Corollary 1 shows that only events related to “higher success than random” are
more extreme under any violation of the null hypothesis than under the null.

Figure 1: Entropy for binary or ternary random variables with arbitrary
maximum payoff points.

### Connection to the maximum entropy, entropic regularization, multinomial
logit, and rational inattention literature

Theorem 1 can be interpreted as a reverse of the Bayesian principle of maximum
entropy, which dictates that among all prior distributions that incorporate
precise knowledge about data, the one which represents the best the current
knowledge is the one with maximal entropy. For instance, given knowledge of
the mean, the exponential distribution is the distribution with the largest
entropy. Given knowledge of mean and variance, the Gaussian distribution is
the distribution with maximum entropy. In contrast, Theorem 1 fixes the
entropy and considers the optimizing behaviour of the taster: among all
distributions with a given entropy, the taster chooses the one that minimizes
their expected prediction loss. In that sense, it is closer to entropic
regularization methods at the heart of the optimal transport and machine
learning literature (e.g. Wilson, 1969; Peyré & Cuturi, 2019). Its proof
provides another illustration of the deep mathematical connection between
entropy and the multinomial logit distribution (or the variational principle
and Gibbs measure), as already shown in Jaynes (1957) who reinterpreted
statistical mechanics in physics using information theory and the maximum
entropy principle, in Shannon (1959) and Anas (1983) in statistical estimation
and engineering applications, or more recently in the rational inattention
economic literature (e.g., Matějka & McKay, 2015) where the decision maker’s
optimal information-processing strategy results in probabilistic choices that
follow a logit model.

##  4 Proof of Theorem 1

Suppose one proves another version of Theorem 1 where the inequality
constraint is replaced by an equality constraint. Then, using the fourth
result of this new version, all results of the original Theorem 1 are
straightforward to obtain. Hence, the version with an equality constraint is
proved below.

1. $H(P^{*}_{X_{N}|Y_{N}}(h))=h$. Under an equality constraint, this is an immediate consequence of the existence and uniqueness of $P^{*}_{X_{N}|Y_{N}}(h)$ established in Point 4 below.

3. $P_{X_{N}|Y_{N}}^{*}(\overline{h})={\rm Uniform}(\mathcal{X}_{N,n})$. It suffices to show that for any finite set $\mathcal{W}$, the collection of distributions $P$ on $\mathcal{W}$ such that $H(P)=\log(|\mathcal{W}|)$ is the singleton $\\{{\rm Uniform}(\mathcal{W})\\}$. This is true since $H$ is uniquely maximized at ${\rm Uniform}(\mathcal{W})$ over the collection of distributions $P$ on $\mathcal{W}$, with maximum value $H({\rm Uniform}(\mathcal{W}))=\log(\left|\mathcal{W}\right|)$. A proof of this well-known result from information theory is given for completeness. Label $\mathcal{W}=\\{w_{1},\ldots,w_{\left|\mathcal{W}\right|}\\}$ and, for all $i\in\\{1,\ldots,\left|\mathcal{W}\right|\\}$, let $p_{i}=\Pr_{W\sim P}(W=w_{i})$, leaving the dependence on $P$ implicit. Note that the mapping $\phi:x\mapsto x\log x$ is strictly convex on $[0,1]$ and the entropy writes $H(P)=-\sum_{i=1}^{\left|\mathcal{W}\right|}\phi(p_{i})$. By Jensen’s inequality,

$\phi\left(\frac{\sum_{i=1}^{\left|\mathcal{W}\right|}p_{i}}{\left|\mathcal{W}\right|}\right)\leq\frac{\sum_{i=1}^{\left|\mathcal{W}\right|}\phi(p_{i})}{\left|\mathcal{W}\right|}=-\frac{H(P)}{\left|\mathcal{W}\right|}.$

Hence,
$H(P)\leq-\left|\mathcal{W}\right|\phi\left(\frac{\sum_{i=1}^{\left|\mathcal{W}\right|}p_{i}}{\left|\mathcal{W}\right|}\right)$.
From $\sum_{i=1}^{\left|\mathcal{W}\right|}p_{i}=1$, deduce
$H(P)\leq\log(\left|\mathcal{W}\right|)$. If $P={\rm Uniform}(\mathcal{W})$,
then $p_{i}=1/\left|\mathcal{W}\right|$ so that
$H(P)=\log(\left|\mathcal{W}\right|)$. That the maximum of $H$ is uniquely
attained at ${\rm Uniform}(\mathcal{W})$ follows from the fact that the
Hessian matrix of $H$ at $P$, denoted $\nabla_{P}^{2}H$, takes value

$\nabla_{P}^{2}H=\begin{pmatrix}-1/p_{1}&0&\cdots&0\\\
0&-1/p_{2}&\ddots&\vdots\\\ \vdots&\ddots&\ddots&0\\\
0&\cdots&0&-1/p_{\left|\mathcal{W}\right|}\end{pmatrix}=\begin{pmatrix}-1/\left|\mathcal{W}\right|&0&\cdots&0\\\
0&-1/\left|\mathcal{W}\right|&\ddots&\vdots\\\ \vdots&\ddots&\ddots&0\\\
0&\cdots&0&-1/\left|\mathcal{W}\right|\end{pmatrix}.$

This matrix is negative definite, and hence $H$ is strictly concave.

4. For all $h,h^{\prime}\in(0,\overline{h}]$ such that $h\leq h^{\prime}$, $\ell(X_{N},Y_{N})$ under $P_{X_{N}|Y_{N}}^{*}(h^{\prime})$ first-order stochastically dominates $\ell(X_{N},Y_{N})$ under $P_{X_{N}|Y_{N}}^{*}(h)$. Fix $y\in\mathcal{X}_{N,n}$ and let us characterize the interior solution of

$\begin{array}[]{cc}\min_{P_{X_{N}|Y_{N}=y}}\mathbb{E}_{P_{X_{N}|Y_{N}=y}}[\ell(X_{N},y)]\\\
\text{subject
to}\quad-\sum_{x\in\mathcal{X}_{N,n}}P_{X_{N}|Y_{N}=y}(x)\log(P_{X_{N}|Y_{N}=y}(x))=h.\end{array}$
(1)

To simplify notation, suppose w.l.o.g. that $2n=N$. Let $a_{k}={n\choose
n-k+1}{n\choose k-1}$, $b_{k}=n-k+1$,
$\boldsymbol{a}=(a_{1},\ldots,a_{n+1})^{\prime}$, and
$\boldsymbol{b}=(b_{1},\ldots,b_{n+1})^{\prime}$. The number $a_{k}$ is the
number of events associated with $b_{k}$ successes out of the first $n$
trials. Rewriting Problem (1) as a maximization problem, it is equivalent to
finding an interior solution
$\boldsymbol{p}:=(p_{1},\ldots,p_{n+1})^{\prime}>\boldsymbol{0}$ that
maximizes

$(\boldsymbol{a}\odot\boldsymbol{b})^{\top}\boldsymbol{p}=\sum_{k=1}^{n+1}(n-k+1)a_{k}p_{k}$

subject to

$\displaystyle\boldsymbol{a}^{\top}\boldsymbol{p}$
$\displaystyle=\sum_{k=1}^{n+1}a_{k}p_{k}=1,$
$\displaystyle-\boldsymbol{a}^{\top}(\boldsymbol{p}\odot\log(\boldsymbol{p}))$
$\displaystyle=-\sum_{k=1}^{n+1}a_{k}p_{k}\log(p_{k})=h,$

where $\odot$ denotes the Hadamard product. The Lagrangian associated with
this linear program with nonlinear constraints is

$\mathcal{L}(\boldsymbol{p},\lambda,\mu)=-(\boldsymbol{a}\odot\boldsymbol{b})^{\top}\boldsymbol{p}+\lambda[1-\boldsymbol{a}^{\top}\boldsymbol{p}]+\mu[h+\boldsymbol{a}^{\top}(\boldsymbol{p}\odot\log(\boldsymbol{p}))].$

For $h\in(0,\overline{h})$, the Karush-Kuhn-Tucker conditions for an interior
solution are

$\displaystyle\partial\mathcal{L}/\partial p_{k}=0$ $\displaystyle\iff-
a_{k}b_{k}-\lambda a_{k}+\mu a_{k}(\log(p_{k})+1)=0\quad\forall
k\in\\{1,\ldots,n+1\\},$ (2)
$\displaystyle\partial\mathcal{L}/\partial\lambda=0$
$\displaystyle\iff\boldsymbol{a}^{\top}\boldsymbol{p}=1,$ (3)
$\displaystyle\partial\mathcal{L}/\partial\mu=0$
$\displaystyle\iff-\boldsymbol{a}^{\top}(\boldsymbol{p}\odot\log(\boldsymbol{p}))=h.$
(4)

The first-order conditions displayed in Equation (2) rewrite

$p_{k}=\exp(b_{k}/\mu-1)\exp(\lambda/\mu)\quad\forall k\in\\{1,\ldots,n+1\\}.$
(5)

Equation (5) combined with the first-order condition (3) implies the logit
form

$p_{k}=\frac{\exp(b_{k}/\mu)}{\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)}\quad\forall
k\in\\{1,\ldots,n+1\\}.$ (6)

Conclude that
$P^{*}_{X_{N}|Y_{N}=y}(h)(x)=\sum_{k=0}^{n}1\\{\ell(x,y)=k\\}p_{k+1}$ is
unique. The desired first-order stochastic dominance result is equivalent to:
for any non-decreasing mapping $u:\mathbb{R}\to\mathbb{R}$,

$\sum_{k=1}^{n+1}u(b_{k})a_{k}p_{k}(\mu,h)\geq\sum_{k=1}^{n+1}u(b_{k})a_{k}p_{k}(\mu^{\prime},h^{\prime}).$
(7)

For $h=\overline{h}$ or $h=h^{\prime}$ the result is trivial. Suppose
$0<h<h^{\prime}\leq\overline{h}$. First, taking the derivative of Equation (6)
yields

$\frac{\partial
p_{k}}{\partial\mu}=\frac{p_{k}}{\mu^{2}}\left(\frac{\sum_{\ell=1}^{n+1}a_{\ell}b_{\ell}\exp(b_{\ell}/\mu)}{\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)}-b_{k}\right)=\frac{p_{k}}{\mu^{2}}\left(\mathbb{E}_{x\sim
P^{*}_{X_{N}|Y_{N}=y}(h)}[N-\ell(x,y)]-b_{k}\right).$ (8)

Second, let us show that ${\rm d}\mu/{\rm d}h>0$. First-order condition (4)
writes

$\underbrace{\frac{1}{{\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)}}\sum_{k=1}^{n+1}a_{k}\exp(b_{k}/\mu)\left[-b_{k}/\mu+\log\left(\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\right)\right]}_{=:f(\mu)}=h.$

Taking the derivative of $f$ yields

$\displaystyle f^{\prime}(\mu)$
$\displaystyle=\Big{(}\Big{[}\sum_{k=1}^{n+1}\left(\frac{-a_{k}b_{k}}{\mu^{2}}\right)\exp(b_{k}/\mu)\Big{\\{}-b_{k}/\mu+\log\left(\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\right)\Big{\\}}$
$\displaystyle\quad+a_{k}\exp(b_{k}/\mu)\Big{\\{}b_{k}/\mu^{2}+\frac{\sum_{\ell=1}^{n+1}\left(\frac{-a_{\ell}b_{\ell}}{\mu^{2}}\right)\exp(b_{\ell}/\mu)}{\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)}\Big{\\}}\Big{]}\Big{[}\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\Big{]}$
$\displaystyle\quad+\Big{[}\sum_{\ell=1}^{n+1}\frac{a_{\ell}b_{\ell}}{\mu^{2}}\exp(b_{\ell}/\mu)\Big{]}\Big{[}\sum_{k=1}^{n+1}a_{k}\exp(b_{k}/\mu)\Big{\\{}-b_{k}/\mu+\log\left(\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\right)\Big{\\}}\Big{]}\Big{)}$
$\displaystyle\quad\Big{/}\Big{(}\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\Big{)}^{2}$
$\displaystyle=\Big{(}\Big{[}\sum_{k=1}^{n+1}\left(\frac{a_{k}b_{k}^{2}}{\mu^{3}}\right)\exp(b_{k}/\mu)$
$\displaystyle\quad+a_{k}\exp(b_{k}/\mu)\Big{\\{}b_{k}/\mu^{2}+\frac{\sum_{\ell=1}^{n+1}\left(\frac{-a_{\ell}b_{\ell}}{\mu^{2}}\right)\exp(b_{\ell}/\mu)}{\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)}\Big{\\}}\Big{]}\Big{[}\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\Big{]}$
$\displaystyle\quad+\Big{[}\sum_{\ell=1}^{n+1}\frac{a_{\ell}b_{\ell}}{\mu^{2}}\exp(b_{\ell}/\mu)\Big{]}\Big{[}\sum_{k=1}^{n+1}\left(\frac{-a_{k}b_{k}}{\mu}\right)\exp(b_{k}/\mu)\Big{]}\Big{)}\Big{/}\Big{(}\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\Big{)}^{2}.$

This expression can be reorganized as

$\displaystyle f^{\prime}(\mu)$
$\displaystyle=\Big{(}\Big{[}\sum_{k=1}^{n+1}\left(\frac{a_{k}b_{k}^{2}}{\mu^{3}}\right)\exp(b_{k}/\mu)+a_{k}\exp(b_{k}/\mu)\Big{\\{}\frac{\sum_{\ell=1}^{n+1}\left(\frac{-a_{\ell}b_{\ell}}{\mu^{2}}\right)\exp(b_{\ell}/\mu)}{\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)}\Big{\\}}\Big{]}$
$\displaystyle\quad\times\Big{[}\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\Big{]}+\Big{[}\sum_{\ell=1}^{n+1}\frac{a_{\ell}b_{\ell}}{\mu^{2}}\exp(b_{\ell}/\mu)\Big{]}\Big{[}\sum_{k=1}^{n+1}a_{k}\exp(b_{k}/\mu)(1-b_{k}/\mu)\Big{]}\Big{)}$
$\displaystyle\quad\Big{/}\Big{(}\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\Big{)}^{2}.$

This expression further simplifies to

$\displaystyle f^{\prime}(\mu)$
$\displaystyle=\Big{(}\Big{[}\sum_{k=1}^{n+1}\frac{a_{k}b_{k}^{2}}{\mu^{3}}\exp(b_{k}/\mu)\Big{]}\Big{[}\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\Big{]}$
$\displaystyle\quad+\Big{[}\sum_{k=1}^{n+1}a_{k}\exp(b_{k}/\mu)\Big{]}\Big{[}\sum_{\ell=1}^{n+1}\left(\frac{-a_{\ell}b_{\ell}}{\mu^{2}}\right)\exp(b_{\ell}/\mu)\Big{]}\Big{)}$
$\displaystyle\quad+\Big{[}\sum_{\ell=1}^{n+1}\frac{a_{\ell}b_{\ell}}{\mu^{2}}\exp(b_{\ell}/\mu)\Big{]}\Big{[}\sum_{k=1}^{n+1}a_{k}\exp(b_{k}/\mu)(1-b_{k}/\mu)\Big{]}\Big{)}$
$\displaystyle\quad\Big{/}\Big{(}\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\Big{)}^{2}$
$\displaystyle=\Big{(}\Big{[}\sum_{k=1}^{n+1}\frac{a_{k}b_{k}^{2}}{\mu^{3}}\exp(b_{k}/\mu)\Big{]}\Big{[}\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\Big{]}$
$\displaystyle\quad-\Big{[}\sum_{\ell=1}^{n+1}\frac{a_{\ell}b_{\ell}}{\mu^{2}}\exp(b_{\ell}/\mu)\Big{]}\Big{[}\sum_{k=1}^{n+1}\frac{a_{k}b_{k}}{\mu}\exp(b_{k}/\mu)\Big{]}\Big{)}\Big{/}\Big{(}\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\Big{)}^{2}.$

Since $\mu>0$, the sign of $f^{\prime}(\mu)$ is the same as the sign of

$\underbrace{\Big{[}\sum_{k=1}^{n+1}a_{k}b_{k}^{2}\exp(b_{k}/\mu)\Big{]}\Big{[}\sum_{\ell=1}^{n+1}a_{\ell}\exp(b_{\ell}/\mu)\Big{]}}_{=:A(\mu)}-\underbrace{\Big{[}\sum_{\ell=1}^{n+1}a_{\ell}b_{\ell}\exp(b_{\ell}/\mu)\Big{]}^{2}}_{=:B(\mu)}.$

By developing products and using mathematical induction on $n\in\mathbb{N}$,
it can be shown that, for all $\mu\in\mathbb{R}$,

$\displaystyle A(\mu)-B(\mu)$
$\displaystyle=\sum_{k=1}^{n+1}\sum_{\ell=1}^{n+1}a_{k}a_{\ell}\exp(b_{k}/\mu)\exp(b_{\ell}/\mu)b_{k}(b_{k}-b_{\ell})\geq
0.$

Conclude

$\frac{{\rm d}\mu}{{\rm d}h}=[f^{\prime}(\mu)]^{-1}>0.$

Next, because $b_{1}>b_{2}>\cdots>b_{n+1}$ and $\mathbb{E}_{x\sim
P^{*}_{X_{N}|Y_{N}=y}(h)}[N-\ell(x,y)]\in(b_{n+1},b_{1})$, Equation (8)
implies that there exists some $\bar{k}\in\\{1,\ldots,n+1\\}$ such that

$\left\\{\begin{array}[]{ll}p_{k}(\mu,h)\geq
p_{k}(\mu^{\prime},h^{\prime})\quad\forall k\leq\bar{k},\\\ p_{k}(\mu,h)\leq
p_{k}(\mu^{\prime},h^{\prime})\quad\forall k>\bar{k}.\end{array}\right.$

Again, since $b_{1}>b_{2}>\cdots>b_{n+1}$ and $u$ is non-decreasing, the
decomposition

$\displaystyle\sum_{k=1}^{n+1}u(b_{k})a_{k}p_{k}(\mu,h)$
$\displaystyle\quad=\sum_{k=1}^{n+1}u(b_{k})a_{k}p_{k}(\mu^{\prime},h^{\prime})+\sum_{k=1}^{n+1}u(b_{k})a_{k}[p_{k}(\mu,h)-p_{k}(\mu^{\prime},h^{\prime})]$
$\displaystyle\quad=\sum_{k=1}^{n+1}u(b_{k})a_{k}p_{k}(\mu^{\prime},h^{\prime})+\sum_{k\leq\bar{k}}u(b_{k})\underbrace{a_{k}[p_{k}(\mu,h)-p_{k}(\mu^{\prime},h^{\prime})}_{\geq
0}+\sum_{k>\bar{k}}u(b_{k})\underbrace{a_{k}[p_{k}(\mu,h)-p_{k}(\mu^{\prime},h^{\prime})}_{\leq
0}$
$\displaystyle\quad\geq\sum_{k=1}^{n+1}u(b_{k})a_{k}p_{k}(\mu^{\prime},h^{\prime})+u(b_{\bar{k}})\sum_{k\leq\bar{k}}a_{k}[p_{k}(\mu,h)-p_{k}(\mu^{\prime},h^{\prime})]+u(b_{\bar{k}})\sum_{k>\bar{k}}a_{k}[p_{k}(\mu,h)-p_{k}(\mu^{\prime},h^{\prime})]$
$\displaystyle\quad=\sum_{k=1}^{n+1}u(b_{k})a_{k}p_{k}(\mu^{\prime},h^{\prime})+u(b_{\bar{k}})\left[\underbrace{\sum_{k=1}^{n+1}a_{k}p_{k}(\mu,h)}_{=1}-\underbrace{\sum_{k=1}^{n+1}a_{k}p_{k}(\mu^{\prime},h^{\prime})}_{=1}\right]$
$\displaystyle\quad=\sum_{k=1}^{n+1}u(b_{k})a_{k}p_{k}(\mu^{\prime},h^{\prime})$

completes the proof of (7).

2. $\lim_{h\to 0^{+}}\sup_{x\in\mathcal{X}_{N,n}}\left|P_{X_{N}|Y_{N}}^{*}(h)(x)-\delta_{Y_{N}}(x)\right|=0$. Since $\mu>0$ and $\frac{{\rm d}\mu}{{\rm d}h}>0$, the result follows by taking $\lim_{\mu\to 0^{+}}p_{k}(\mu)=1\\{k=1\\}/a_{1}=1\\{k=1\\}$ in Equation (6).

The proof of Theorem 1 is complete.

## References

  * (1)
  * Anas (1983) Anas, A. (1983), ‘Discrete choice theory, information theory and the multinomial logit and gravity models’, Transportation Research Part B: Methodological 17(1), 13–23.
  * Fisher (1925) Fisher, R. (1925), Statistical methods for research workers, Edinburgh Oliver & Boyd.
  * Fisher (1935) Fisher, R. A. (1935), The Design of Experiments, Oliver and Boyd, Edinburgh.
  * Jaynes (1957) Jaynes, E. T. (1957), ‘Information theory and statistical mechanics’, Phys. Rev. 106, 620–630.
  * Matějka & McKay (2015) Matějka, F. & McKay, A. (2015), ‘Rational inattention to discrete choices: A new foundation for the multinomial logit model’, The American Economic Review 105(1), 272–298.
  * Neyman & Pearson (1928) Neyman, J. & Pearson, E. S. (1928), ‘On the Use And Interpretation of Certain Test Criteria for Purpose of Statistical Inference’, Biometrika 20A(3-4), 263–294.
  * Peyré & Cuturi (2019) Peyré, G. & Cuturi, M. (2019), ‘Computational optimal transport’, Foundations and Trends® in Machine Learning 11(5-6), 355–607.
  * Ritzwoller et al. (2024) Ritzwoller, D. M., Romano, J. P. & Shaikh, A. M. (2024), ‘Randomization inference: Theory and applications’.
  * Shannon (1948) Shannon, C. E. (1948), ‘A mathematical theory of communication’, The Bell System Technical Journal 27, 379–423.
  * Shannon (1959) Shannon, C. E. (1959), ‘Coding theorems for a discrete source with a fidelity criterion’, Institute of Radio Engineers National Convention Record 7(4), 142–163.
  * Wilson (1969) Wilson, A. G. (1969), ‘The use of entropy maximizing models, in the theory of trip distribution, mode split and route split’, Journal of Transport Economics and Policy 3(1), 108–126.