# Generalized Fisher-Darmois-Koopman-Pitman Theorem and Rao-Blackwell Type
Estimators for Power-Law Distributions

Atin Gayen and M. Ashok Kumar
Atin Gayen is supported by an INSPIRE fellowship of the Department of Science
and Technology, Govt. of India.Atin Gayen (Email<EMAIL_ADDRESS>and M.
Ashok Kumar (Email<EMAIL_ADDRESS>are with the Department of
Mathematics of Indian Institute of Technology Palakkad, Kerala 678 557,
India.A part of this paper was presented at the 2021 IEEE International
Symposium on Information Theory [24].

###### Abstract

This paper generalizes the notion of sufficiency for estimation problems
beyond maximum likelihood. In particular, we consider estimation problems
based on Jones et al. and Basu et al. likelihood functions that are popular
among distance-based robust inference methods. We first characterize the
probability distributions that always have a fixed number of sufficient
statistics (independent of sample size) with respect to these likelihood
functions. These distributions are power-law extensions of the usual
exponential family and contain Student distributions as a special case. We
then extend the notion of minimal sufficient statistics and compute it for
these power-law families. Finally, we establish a Rao-Blackwell-type theorem
for finding the best estimators for a power-law family. This helps us
establish Cramér-Rao-type lower bounds for power-law families.

###### Index Terms:

Cramér-Rao bounds, Fisher-Darmois-Koopman-Pitman theorem, Rao-Blackwell
theorem, Student distribution, sufficient statistics.

##  I Introduction

Consider a parametric family of probability distributions, parameterized by
$\theta$, $\Pi=\\{p_{\theta}:\theta\in\Theta\\}$, where $\Theta$ is an open
subset of $\mathbb{R}^{k}$. We shall assume (unless explicitly specified) that
all distributions in the family have common support
$\mathbb{S}\subset\mathbb{R}$. Let $X_{1}^{n}\coloneqq X_{1},\dots,X_{n}$ be
an independent and identically distributed (i.i.d.) random sample drawn
according to some $p_{\theta}\in\Pi$. An estimator of $\theta$ is a function
of $X_{1}^{n}$. If the estimator is a function of some specific statistics of
the sample, then one can use these statistics to estimate $\theta$ without
knowing the entire sample. Fisher introduced this concept in his seminal paper
titled “On the mathematical foundations of theoretical statistics” and termed
it as _sufficient statistics_ for the parameter [19, Sec. 7]. He designated a
statistic $T$ to be _sufficient for estimating $\theta$_ if the conditional
distribution of the sample, given $T$, is independent of $\theta$. Formally,
if $T\coloneqq T(X_{1}^{n})$ is a statistic ($T$ could very well be a vector-
valued function), then $T$ is sufficient (for estimating $\theta$) if the
conditional distribution

$p_{\theta_{X_{1}^{n}|T}}(x_{1}^{n}|t)\quad\text{is independent
of}\quad\theta$ (1)

for every $t\in\mathscr{T}\coloneqq\\{T(y_{1}^{n})\colon
y_{1}^{n}\in\mathbb{S}^{n}\\}$. This definition, however, is helpful only if
one has a suitable guess for $T$. Observe also that the distribution of $T$ is
needed to find the conditional distribution in (1). Later, Koopman proposed
the following equivalent definition, where the distribution of $T$ is not
needed [31]. He called a statistic $T$ sufficient for $\theta$ if the
following holds. If $x_{1}^{n}$ and $y_{1}^{n}$ are two sets of i.i.d. samples
such that they are same under $T$, that is, $T(x_{1}^{n})=T(y_{1}^{n})$, then
the difference of their respective log-likelihoods,
$[L(x_{1}^{n};\theta)-L(y_{1}^{n};\theta)]$ is independent of $\theta$, where
$L(x_{1}^{n};\theta)\coloneqq\sum_{i=1}^{n}\log p_{\theta}(x_{i})$ denotes the
log-likelihood function of the sample $x_{1}^{n}$. Observe that, to use this
definition also, one needs to have a guess for $T$. Let us now suppose that a
sufficient statistic $T$ is known. A natural question that arises is, what
kind of estimators that can be found using $T$. Observe that both Fisher’s and
Koopman’s definitions of sufficiency make use of the log-likelihood function
$L$. However, it is still not clear if Fisher or Koopman had maximum
likelihood (ML)-estimation in mind while proposing the above definitions. The
following result, known as _factorization theorem_ [38, 26], which is
equivalent to both the above definitions, seems to answer this. A statistic
$T$ is sufficient for $\theta$ if and only if there exist two functions
$u:\Theta\times\mathscr{T}\to\mathbb{R}$ and $v:\mathbb{S}^{n}\to\mathbb{R}$
such that

$\displaystyle L(x_{1}^{n};\theta)=u(\theta,T(x_{1}^{n}))+v(x_{1}^{n})$ (2)

for all $x_{1}^{n}$ and all $\theta\in\Theta$ [10, Th. 6.2.6]. Observe that,
using this criterion, one can identify the sufficient statistic $T$ by simply
factorizing the log-likelihood function. Hence no prior guess for T is needed.
If $\theta_{\text{MLE}}$ denotes the maximum likelihood estimator (MLE) of
$\theta$, then from (2), we have

$\displaystyle\theta_{\text{MLE}}$ $\displaystyle=$
$\displaystyle\arg\max_{\theta}L({X}_{1}^{n};\theta)$ $\displaystyle=$
$\displaystyle\displaystyle\arg\max_{\theta}[u(\theta,T({X}_{1}^{n}))+v({X}_{1}^{n})]$
$\displaystyle=$
$\displaystyle\displaystyle\arg\max_{\theta}u(\theta,T({X}_{1}^{n})).$

This implies that the MLE of $\theta$ depends on the sample only through the
statistic $T$ (that is, knowledge of $T$ is sufficient for the estimation of
$\theta$). This seems to suggest that the classical notion of sufficiency is
based on ML estimation.

Observe that the ‘entire sample’ is trivially a sufficient statistic with
respect to any estimation. However, the notion of sufficient statistics is
helpful only when it exists in a fixed number, independent of the sample size.
_Fisher-Darmois-Koopman-Pitman theorem_ addresses this problem [20, 31, 41,
16, 1]. It states that, under certain regularity conditions, the existence of
such a set of sufficient statistics is possible if and only if the underlying
family is exponential. A set of probability distributions $\mathcal{E}$ is
said to be an _exponential family_ if there exist functions
$w=[w_{1},\dots,w_{s}]^{\top}$ and $f=$ $[f_{1},\dots,$ $f_{s}]^{\top}$ with
$w_{i}:\Theta\to\mathbb{R}$, $f_{i}:\mathbb{S}\to\mathbb{R}$, and
$h:\mathbb{R}\to\mathbb{R}$ such that each $p_{\theta}\in\mathcal{E}$ is given
by

$\displaystyle{p_{\theta}(x)}=\left\\{\begin{array}[]{ll}{Z(\theta)\exp\big{[}h(x)+w(\theta)^{\top}f(x)\big{]}}&\hbox{~{}if~{}}{x}\in\mathbb{S}\\\
{0}&\hbox{~{}otherwise},\end{array}\right.$ (5)

where $Z(\theta)$ is the normalizing factor [7]. Bernoulli, Binomial, Poisson,
and Normal distributions are some examples of exponential families. However, a
statistic that is sufficient for ML estimation might not be sufficient for
some other estimations. For example, consider the family of binomial
distributions with parameters, say $(m,\theta)$, where $m$ is a positive
integer denoting the number of trials (assumed known) and $\theta\in(0,1)$ is
the success probability, which is unknown. It is well-known that the sum of
the sample is always a sufficient statistic for $\theta$ (with respect to the
usual likelihood function) (see, for example, [10]).

Let us now suppose that we want to estimate $\theta$ by maximizing the
following likelihood function, called _Cauchy-Schwarz likelihood function_
(c.f. [9, Eq. (2.90)]):

$\displaystyle
L_{cs}(x_{1}^{n};\theta)\coloneqq\log\Bigg{[}~{}{\sum\limits_{x=0}^{m}p_{n}(x)p_{\theta}(x)}\Big{/}{\sum\limits_{x=0}^{m}p_{\theta}(x)^{2}}\Bigg{]},$
(6)

where $p_{\theta}(x)=\binom{m}{x}\theta^{x}(1-\theta)^{m-x}$ for $x=0,\dots,m$
and $p_{n}$ is the empirical distribution of the observed sample. Maximizing
the Cauchy-Schwarz likelihood function is the same as minimizing the well-
known Cauchy-Schwarz divergence. (This divergence is popular in machine
learning applications such as blind source separation, independent component
analysis, manifold learning, image segmentation, and so on [28, 45, 30, 27,
39, 40].) It is easy to check that the solutions of the estimating equation,
in this case, are the roots of a polynomial in $\theta$ whose coefficients
involve some of $p_{n}(x)$, $x=0,\dots,m$. For example, when $m=2$, it is a
polynomial of degree five, and the following are, respectively, the
coefficients in descending order of exponents of $\theta$:

$8p_{n}(1),~{}8p_{n}(0)-20p_{n}(1)-6p_{n}(2),~{}-20p_{n}(0)+16p_{n}(1)+12p_{n}(2),18p_{n}(0)-4p_{n}(1)-6p_{n}(2),$
$-7p_{n}(0)-2p_{n}(1)+p_{n}(2)$, and $p_{n}(0)+p_{n}(1)$.

This shows that at least two among $p_{n}(0),p_{n}(1)$ and $p_{n}(2)$ are
needed to find the estimator. Hence the sum of the sample is no longer
sufficient for estimating $\theta$.

Observe that the Cauchy-Schwarz likelihood function is a special case
($\alpha=2$) of the more general class of likelihood functions, called Jones
et al. likelihood function [29]:

$\displaystyle{L_{J}^{(\alpha)}(x_{1}^{n};\theta)}$ $\displaystyle\coloneqq$
$\displaystyle\dfrac{1}{\alpha-1}\log\left[\dfrac{1}{n}\sum\limits_{i=1}^{n}p_{\theta}({x}_{i})^{\alpha-1}\right]-\dfrac{1}{\alpha}\log\left[\int
p_{\theta}(x)^{\alpha}dx\right]$ (7) $\displaystyle=$
$\displaystyle\dfrac{1}{\alpha-1}\log\left[\dfrac{1}{n}\sum\limits_{i=1}^{n}p_{\theta}({x}_{i})^{\alpha-1}\right]-\log\|p_{\theta}\|,~{}~{}$

where $\alpha>0$, $\alpha\neq 1$, and $\|p_{\theta}\|\coloneqq\big{(}\int
p_{\theta}(x)^{\alpha}dx\big{)}^{\frac{1}{\alpha}}$.

Let us now consider the following family of Student distributions given by

$p_{\theta}(x)=\frac{2}{\sqrt{3}\pi\sigma}\Big{[}1+\frac{(x-\mu)^{2}}{3\sigma^{2}}\Big{]}^{-2}\quad\text{for}~{}x\in\mathbb{R},$
(8)

where $\theta=[\mu,\sigma^{2}]^{\top}$, $\mu\in(-\infty,\infty)$ and
$\sigma\in(0,\infty)$. Observe that (8) does not fall in the exponential
family as defined in (5). Hence, by the Fisher-Darmois-Koopman-Pitman theorem,
there does not exist any non-trivial sufficient statistics for $\mu$ and
$\sigma$. Indeed, it can be shown that the knowledge of the entire sample is
needed for estimating $\mu$ and $\sigma$ in this case (See [24]). However,
Eguchi et al. considered the estimation based on Jones et al. likelihood
function (7) with $\alpha=1/2$ and found closed-form solution for the
estimators (See [17]). The estimators are functions of the statistics
$\sum_{i=1}^{n}X_{i}$ and $\sum_{i=1}^{n}X_{i}^{2}$ [17, Th. 3] (c.f. [25, Th.
11]). This motivates us to generalize the concept of sufficiency based on an
estimation method. Let us consider the difference of Jones et al. likelihood
function $L_{J}^{(1/2)}$ evaluated at samples $x_{1}^{n}$ and $y_{1}^{n}$:

$\displaystyle{L_{J}^{(1/2)}(x_{1}^{n};\theta)-L_{J}^{(1/2)}(y_{1}^{n};\theta)}$
$\displaystyle=$ $\displaystyle
2\Big{[}\log\sum\limits_{i=1}^{n}p_{\theta}(y_{i})-\log\sum\limits_{i=1}^{n}p_{\theta}(x_{i})\Big{]}.$

For the family of Student distributions (8), we have

$\displaystyle{L_{J}^{(1/2)}(x_{1}^{n};\theta)-L_{J}^{(1/2)}(y_{1}^{n};\theta)}$
$\displaystyle=$ $\displaystyle
2\log\Bigg{[}\dfrac{1+(1/(\mu^{2}+3\sigma^{2}))}{1+(1/(\mu^{2}+3\sigma^{2}))}\cdot\dfrac{\sum\limits_{i=1}^{n}y_{i}^{2}-(2\mu/(\mu^{2}+3\sigma^{2}))\sum\limits_{i=1}^{n}y_{i}}{\sum\limits_{i=1}^{n}x_{i}^{2}-(2\mu/(\mu^{2}+3\sigma^{2}))\sum\limits_{i=1}^{n}x_{i}}\Bigg{]}.~{}$
(9)

(9) is independent of $\theta$ if

$\sum\limits_{i=1}^{n}x_{i}^{2}=\sum\limits_{i=1}^{n}y_{i}^{2}\quad\text{and}\quad\sum\limits_{i=1}^{n}x_{i}=\sum\limits_{i=1}^{n}y_{i}.$

Notice that the Jones et al. estimators for $\mu$ and $\sigma$ depend on the
sample only through the statistics $\sum_{i=1}^{n}X_{i}$ and
$\sum_{i=1}^{n}X_{i}^{2}$. This implies that, when the estimation is based on
Jones et al. likelihood function $L_{J}^{(1/2)}$, it is reasonable to call the
pair $\big{(}\sum X_{i},\sum X_{i}^{2}\big{)}$ sufficient for $\mu$ and
$\sigma$.

Jones et al. likelihood function is popular in the context of robust
estimation [29]. Observe that (7) coincides with the usual log-likelihood
function $L(\theta)$ as $\alpha\to 1$. Hence Jones et al. likelihood function
is considered a robust alternative to ML estimation. Motivated by the works of
Windham [48] and Field and Smith [18], Jones et al. proposed this method in
search of an estimating equation, an alternative to the one of MLE, that uses
a suitable power of the model density to down-weight the effect of _outliers_
(possibly) present in the data [29, 25, 21, 17]. It is well-known that ML
estimation is closely related to the minimization of the well-known Kullback-
Leibler divergence or relative entropy [15, Lem. 3.1]. Similarly, Jones et al.
estimation is related to the minimization of the so-called Jones et al.
divergence [22, 35, 46, 47, 25] (also known as $\gamma$-divergence [11, 22],
projective power divergence [17], logarithmic density power divergence [36],
relative $\alpha$-entropy [32, 33], Rényi pseudo-distance [5, 6]). While Jones
et al. divergence for $\alpha>1$ corresponds to robust inference, the same for
$\alpha<1$ has connections to problems in information theory, such as
mismatched guessing [46], source coding [33], encoding of tasks [8] and so on.
Though Jones et al. estimation for $\alpha<1$ does not have a meaning in terms
of robustness, it can be used to approximate MLE of a Student distribution
(c.f. [25, Rem. 7]).

Another likelihood function that is popular in the context of robust inference
is the so-called Basu et al. likelihood function given by

$\displaystyle{L^{(\alpha)}_{B}(x_{1}^{n};\theta)}$ $\displaystyle=$
$\displaystyle\frac{1}{n}\sum\limits_{i=1}^{n}\left[\frac{\alpha\cdot
p_{\theta}(x_{i})^{\alpha-1}-1}{\alpha-1}\right]-\int
p_{\theta}(x)^{\alpha}dx,~{}~{}~{}$ (10)

where $\alpha>0$, $\alpha\neq 1$ [3, Eq. (2.2)]. Maximization of
$L^{(\alpha)}_{B}$ results in a non-normalized version of the Jones et al.
estimating equation (see [25, Sec. 1]). Basu et al. estimation is closely
related to the minimization of the so-called Basu et al. divergence, which
belongs to the Bregman class [13]. Basu et al. divergence for $\alpha=2$
corresponds to the squared Euclidean distance between the empirical measure
and the model density. A comparison of Basu et al. and Jones et al.
estimations was studied in [29]. Unlike other likelihood functions such as the
Hellinger likelihood function, Basu et al. and Jones et al. likelihood
functions are special in robust inference, in the sense that smoothing of the
empirical probability measure is not needed (see, for example, [25, Sec. 1]
for details). In this paper, we propose a generalized notion of sufficiency
based on the likelihood function associated with an estimation problem.
Specifically, we will focus on Jones et al. and Basu et al. likelihood
functions. We will characterize the family of probability distributions that
has a fixed number of sufficient statistics independent of the sample size
with respect to these likelihood functions. It turns out that these are the
power-law families studied in [25, 33, 2, 37, 14]. Student distributions fall
in this family.

Sufficient statistics not only help in data reduction but also help in finding
an estimator better than the given one. Indeed, if one has an unbiased
estimator along with sufficient statistics, it is always possible to find
another unbiased estimator (usually a function of sufficient statistics) whose
variance is not more than the given one. This result is known as the _Rao-
Blackwell theorem_ [42, 4]. That is, conditional expectation of an unbiased
estimator given the sufficient statistics results in a uniform improvement (in
variance) of the estimator. This suggests one to consider only such
conditional expectations while searching for best unbiased estimators.

In this paper, we find Rao-Blackwell type estimators for power-law families of
the form (8) with respect to Jones et al. and Basu et al. likelihood
functions. We also find a lower bound for the variance of these estimators.
This generalizes the usual Cramér-Rao Lower Bound (CRLB) and results in a
sharper bound for the power-law families. We also apply these results to the
Student distributions.

Our main contributions in this paper are the following.

  * (i)

Propose a generalized notion of principle of sufficiency that is appropriate
when the underlying estimation is not necessarily ML estimation.

  * (ii)

Characterize the families of probability distributions that have a fixed
number of sufficient statistics independent of sample size with respect to
Basu et al. and Jones et al. likelihood functions.

  * (iii)

Generalize the notion of minimal sufficiency with respect to a likelihood
function and find generalized minimal sufficient statistics for the power-law
family.

  * (iv)

Extend Rao-Blackwell theorem to both Basu et al. and Jones et al. estimations
and find best estimators for the power-law family.

  * (v)

Derive a Cramér-Rao lower bound for the variance of an unbiased estimator (of
the expected value of the generalized sufficient statistic) of a power-law
family in terms of the asymptotic variance of the Jones et al. estimator.

The rest of the paper is organized as follows. In Section II, we introduce the
generalized notion of sufficiency. In Section III, we establish Fisher-
Darmois-Koopman-Pitman theorem for Jones et al. and Basu et al. likelihood
functions. In this section, we also find minimal sufficient statistics for the
mean and variance parameters of Student distributions. In Section IV, we find
generalized Rao-Blackwell type estimators for the power-law family. In Section
V, we establish generalized CRLB for Basu et al. and Jones et al. estimators
of power-law family. In this section, we also find the best estimators for the
mean and variance parameters of Student distributions. We end the paper with a
summary and concluding remarks in Section VI. Proofs of most of the results
are presented in the Appendix.

##  II A Generalized notion of the principle of sufficiency

In this section, we first propose a generalized notion of sufficiency
analogous to Koopman’s definition of sufficiency. We then derive a
factorization theorem associated with this generalized notion. We also show
that this is equivalent to generalized Fisher’s definition of sufficiency.

###### Definition 1

Let $\Pi=\\{p_{\theta}:\theta\in\Theta\\}$ be a family of probability
distributions. Suppose the underlying problem is to estimate $\theta$ by
maximizing some generalized likelihood function $L_{G}$. A set of statistics
$T$ is said to be a sufficient statistic for $\theta$ with respect to $L_{G}$
if $[L_{G}({x}_{1}^{n};\theta)-L_{G}({y}_{1}^{n};\theta)]$ is independent of
$\theta$ whenever $T({x}_{1}^{n})=T({y}_{1}^{n})$.

Observe that when $L_{G}$ is the log-likelihood function, the above definition
coincides with the usual Koopman’s definition of sufficiency. We now derive a
generalized factorization theorem.

######  Proposition 2 (Generalized Factorization Theorem)

Let $\Pi=\\{p_{\theta}:\theta\in\Theta\\}$ be a family of probability
distributions. Suppose the underlying problem is to estimate $\theta$ by
maximizing some generalized likelihood function $L_{G}$. Then a set of
statistics $T$ is a sufficient statistic for $\theta$ with respect to $L_{G}$
if and only if there exist functions $u:\Theta\times\mathscr{T}\to\mathbb{R}$
and $v:\mathbb{S}^{n}\to\mathbb{R}$ such that

$L_{G}({x}_{1}^{n};\theta)=u(\theta,T({x}_{1}^{n}))+v({x}_{1}^{n}),$ (11)

for all sample points ${x}_{1}^{n}$ and for all $\theta\in\Theta$.

###### Proof:

See Sec-A.1 of Appendix. ∎

Observe that when $L_{G}=L$, the above coincides with the usual factorization
theorem [38, 26]. We now show that this generalized notion of sufficiency is
also equivalent to a generalized version of Fisher’s definition of
sufficiency. Let $X_{1}^{n}$ be i.i.d. random variables according to some
$p_{\theta}\in\Pi$. Let $x_{1}^{n}\coloneqq x_{1},\dots,x_{n}$ be a sample.
Let us define the deformed probability distribution associated with the
likelihood function $L_{G}$ by

$\displaystyle
p_{\theta}^{*}(x_{1}^{n})\coloneqq\frac{\exp[L_{G}(x_{1}^{n};\theta)]}{\int\exp[L_{G}(y_{1}^{n};\theta)]dy_{1}^{n}},$

provided the denominator is finite. Observe that, if
$L_{G}(x_{1}^{n};\theta)=L(x_{1}^{n};\theta)$, then

$\displaystyle p_{\theta}^{*}(x_{1}^{n})$ $\displaystyle=$
$\displaystyle\dfrac{\exp[\log p_{\theta}(x_{1}^{n})]}{\int\exp[\log
p_{\theta}(y_{1}^{n})]d{y_{1}^{n}}}=\dfrac{p_{\theta}(x_{1}^{n})}{\int
p_{\theta}(y_{1}^{n})d{y_{1}^{n}}}=p_{\theta}(x_{1}^{n}).$ (12)

Let $T$ be a statistic of the sample $x_{1}^{n}$. For any value $t$ of $T$,
let $q_{\theta}^{*}(t)\coloneqq\int
p_{\theta}^{*}(y_{1}^{n})\textbf{1}(T(y_{1}^{n})=t)dy_{1}^{n}$, where
$\textbf{1}(\cdot)$ denotes the indicator function. This can be re-written as

$q_{\theta}^{*}(t)=\int\limits_{A_{t}}p_{\theta}^{*}(y_{1}^{n})dy_{1}^{n},$

where $A_{t}=\\{x_{1}^{n}:T(x_{1}^{n})=t\\}$. The associated conditional
distribution is given by

$\displaystyle p_{\theta{{}_{X_{1}^{n}|T}}}^{*}(x_{1}^{n}|t)$
$\displaystyle\coloneqq$
$\displaystyle\dfrac{p_{\theta}^{*}(x_{1}^{n})\textbf{1}(T(x_{1}^{n})=t)}{q_{\theta}^{*}(t)}$
(15) $\displaystyle=$
$\displaystyle\left\\{\begin{array}[]{ll}{\dfrac{p_{\theta}^{*}(x_{1}^{n})}{q_{\theta}^{*}(t)}}&\hbox{~{}if~{}}x_{1}^{n}\in
A_{t}\\\ {0}&\hbox{~{}otherwise}.\end{array}\right.$

Then

$\displaystyle\int
p_{\theta{{}_{X_{1}^{n}|T}}}^{*}(x_{1}^{n}|t)d{x_{1}^{n}}=\int\limits_{A_{t}}\dfrac{p_{\theta}^{*}(x_{1}^{n})}{q_{\theta}^{*}(t)}dx_{1}^{n}=\int\limits_{A_{t}}\dfrac{p_{\theta}^{*}(x_{1}^{n})}{q_{\theta}^{*}(T(x_{1}^{n}))}dx_{1}^{n}=\dfrac{\int\limits_{A_{t}}p_{\theta}^{*}(x_{1}^{n})dx_{1}^{n}}{\int\limits_{A_{t}}p_{\theta}^{*}(x_{1}^{n})dx_{1}^{n}}=1.$

Observe that, when $L_{G}=L$, we have

$p_{\theta{{}_{X_{1}^{n}|T}}}^{*}(x_{1}^{n}|t)=p_{\theta{{}_{X_{1}^{n}|T}}}(x_{1}^{n}|t),$

the usual conditional distribution of $x_{1}^{n}$ given $T=t$.

###### Proposition 3

$T$ is sufficient for $\theta$ with respect to a likelihood function $L_{G}$
if and only if $p_{\theta{{}_{X_{1}^{n}|T}}}^{*}(x_{1}^{n}|t)$ is independent
of $\theta$ for any value $t$ of $T$.

###### Proof:

See Sec-A.2 of Appendix. ∎

###### Remark 1

Fisher’s idea of sufficiency has one appealing interpretation that if we know
the value of the sufficient statistic $T$ without knowing the entire sample,
then we could (using the conditional distribution of the sample which is
independent of $\theta$ given the sufficient statistic) generate another
sample which has the same unconditional distribution as the original sample
and the same value of the sufficient statistic $T$. Hence the information
contained in both the samples are essentially the same. In this sense, the
statistic $T$ is “sufficient”. This interpretation continues to hold for the
generalized notion of sufficiency as well. Indeed, in the case of generalized
sufficient statistics $T$ with respect to $L_{G}$, one can generate a sample
using the generalized conditional distribution as defined in (15). It contains
the same information about $\theta$ as in the original sample (with respect to
$L_{G}$). Suppose that the given sample is $X_{1}^{n}=x_{1}^{n}$ and that
$T(x_{1}^{n})=t$. If we do not know $x_{1}^{n}$, but know only that
$T(X_{1}^{n})=t$, then we can generate $z_{1}^{n}$ using the generalized
conditional distribution of $X_{1}^{n}=z_{1}^{n}$, given $T(X_{1}^{n})=t$
(since the conditional distribution is independent of $\theta$ by Proposition
3). Then, using (64), the value of $L_{G}$ for the generated sample is given
by $L_{G}(z_{1}^{n};\theta)=u(t,\theta)+v(z_{1}^{n})$, where

$\displaystyle u(t,\theta)$ $\displaystyle=$
$\displaystyle\log\int\limits_{A_{t}}\exp[L_{G}(y_{1}^{n};\theta)]dy_{1}^{n}\quad\text{with}$
$\displaystyle A_{t}$ $\displaystyle=$
$\displaystyle\\{x_{1}^{n}:T(x_{1}^{n})=t\\},\text{ and }$ $\displaystyle
v(z_{1}^{n})$ $\displaystyle=$ $\displaystyle\log
p_{\theta_{X_{1}^{n}|T}}^{*}(z_{1}^{n}|t).$

Hence

$\displaystyle\arg\displaystyle\max_{\theta}L_{G}(z_{1}^{n};\theta)=\arg\displaystyle\max_{\theta}u(t,\theta)=\arg\displaystyle\max_{\theta}L_{G}(x_{1}^{n};\theta),$

since $x_{1}^{n}\in A_{t}$. Thus $x_{1}^{n}$ and $z_{1}^{n}$ have the same
information about $\theta$ as far as estimation with respect to $L_{G}$ is
concerned. Also, from (15), for any $y_{1}^{n}\in A_{t}$, the unconditional
distribution with respect to $p_{\theta}^{*}$ is given by
$p^{*}_{\theta}(y_{1}^{n})=p^{*}_{\theta_{X_{1}^{n}|T}}(y_{1}^{n}|t)q_{\theta}^{*}(t)$.
This implies that the generated sample $z_{1}^{n}$ has the same distribution
as the initial sample $x_{1}^{n}$ (with respect to $p_{\theta}^{*}$).

While sufficient statistics help in data compression, _minimal sufficient
statistics_ help maximum such compression. We now define the notion of minimal
sufficiency in a general setup. Let us first recall the definition of minimal
sufficiency in the classical sense.

###### Definition 4

[10, Def. 6.2.11] A statistic $T$ is said to be a _minimal sufficient
statistic_ if $T$ is a function of any other sufficient statistic. In other
words, $T$ is minimal if the following holds: For any sufficient statistic
$\widetilde{T}$ and for any two i.i.d. samples ${x}_{1}^{n}$ and
${y}_{1}^{n}$, $\widetilde{T}(x_{1}^{n})=\widetilde{T}(y_{1}^{n})$ implies
${T}({{x}_{1}^{n}})={T}({y}_{1}^{n})$.

There is an easy criterion to find minimal sufficient statistics for MLE due
to Lehmann and Scheffé [10, Th. 6.2.13]. Here we first generalize this
criterion to a generalized likelihood function. Later we will use it to find
minimal sufficient statistics for power-law families.

###### Proposition 5

Let $\Pi=\\{p_{\theta}:\theta\in\Theta\\}$ be a family of probability
distributions. Suppose the underlying problem is to estimate $\theta$ by
maximizing some likelihood function $L_{G}$. Assume that the following
condition holds for a statistic $T$. For any two i.i.d. samples $x_{1}^{n}$
and $y_{1}^{n}$, $T(x_{1}^{n})=T(y_{1}^{n})$ if and only if
$[L_{G}(x_{1}^{n};\theta)-L_{G}(y_{1}^{n};\theta)]$ is independent of $\theta$
for $\theta\in\Theta$. Then $T$ is a minimal sufficient statistic for $\theta$
with respect to $L_{G}$.

###### Proof:

See Sec-A.3 of Appendix. ∎

##  III Generalized Fisher-Darmois-Koopman-Pitman Theorem

In this section, we first characterize the probability distributions that have
a fixed number of sufficient statistics independent of sample size with
respect to Jones et al. and Basu et al. likelihood functions. We observe that
each of them is a power-law extension of the usual exponential family. We then
find sufficient statistics for the power-law families. We start by defining
the family associated with Jones et al. likelihood function.

###### Definition 6

A $k$-parameter $\alpha$-power-law family characterized by $h,w,f,\Theta$ and
$\mathbb{S}$, where $\Theta$ is an open subset of $\mathbb{R}^{k}$,
$w=[w_{1},\dots,w_{s}]^{\top}$, $f=$ $[f_{1},\dots,$ $f_{s}]^{\top}$, with
$w_{i}:\Theta\to\mathbb{R}$ is differentiable for $i=1,\dots,s$,
$f_{i}:\mathbb{R}\to\mathbb{R}$ for $i=1,\dots,s$,
$h:\mathbb{R}\to\mathbb{R}$, and $\mathbb{S}\subseteq\mathbb{R}$, is defined
by $\mathbb{M}^{(\alpha)}=\\{p_{\theta}\colon\theta\in\Theta\\}$, where

$\displaystyle{p_{\theta}(x)}=\left\\{\begin{array}[]{ll}{Z(\theta)\big{[}h(x)+w(\theta)^{\top}f(x)\big{]}^{\frac{1}{\alpha-1}}}&\hbox{~{}for~{}}x\in\mathbb{S}\\\
{0}&\hbox{~{}otherwise,}\end{array}\right.$ (18)

with
$Z(\theta)^{-1}=\int\limits_{\mathbb{S}}[h(x)+w(\theta)^{\top}f(x)]^{\frac{1}{\alpha-1}}d{x}$.

###### Remark 2

  * (i)

If $h(x)=q(x)^{\alpha-1}$ for some function $q(x)>0$ and
$f(x)=(1-\alpha)\tilde{f}(x)$, then $\mathbb{M}^{(\alpha)}$-family can be
shown to coincide with an exponential family when $\alpha=1$ (see [32, Def.
8]).

  * (ii)

In [33], this family was derived as projections of Jones et al. divergence on
a set of probability distributions determined by linear constraints. More
about $\mathbb{M}^{(\alpha)}$-family can be found in [2, 23, 25], and [33].

  * (iii)

Student distributions fall in $\mathbb{M}^{(\alpha)}$-family [25, Ex. 3].

We now state the main result of this section.

###### Theorem 7

Let $\Pi=\\{p_{\theta}:\theta\in\Theta\subset\mathbb{R}^{k}\\}$ be a family of
probability distributions with common support $\mathbb{S}$, an open subset of
$\mathbb{R}$. Let $x_{1}^{n}$ be an i.i.d. sample from some
$p_{\theta}\in\Pi$. For some $s$ $(k\leq s<n)$, let $T_{j}(x_{1}^{n})$ for
$j=1,\dots,s$ be differentiable functions on $\mathbb{S}^{n}$. If
$[T_{1},\dots,T_{s}]^{\top}$ is a sufficient statistic for $\theta$ with
respect to Jones et al. likelihood function $L_{J}^{(\alpha)}$, then there
exist functions $h:\mathbb{S}\to\mathbb{R}$, $f_{i}:\mathbb{S}\to\mathbb{R}$
and $w_{i}:\Theta\to\mathbb{R}$, $i=1,\dots,s$ such that $p_{\theta}(x)$ can
be expressed as (18) for $x\in\mathbb{S}$.

###### Proof:

The proof is divided into two parts. The first part deals with the case when
$k=s$ and the next one for $k<s$.

  * (a)

Let $k=s$. We only give the proof when $k=s=1$. Other cases can be proved
similarly.

Let us first consider the Jones et al. likelihood function (7). Taking
derivative with respect to $\theta$ on both sides of (7), we get

$\displaystyle{\frac{\partial}{\partial\theta}L_{J}^{(\alpha)}(x_{1}^{n};\theta)}$
$\displaystyle=$
$\displaystyle\frac{1}{\alpha-1}\frac{\sum_{i=1}^{n}\frac{\partial}{\partial\theta}\big{[}p_{\theta}(x_{i})^{\alpha-1}\big{]}}{\sum_{i=1}^{n}p_{\theta}(x_{i})^{\alpha-1}}-\frac{\partial}{\partial\theta}\log\|p_{\theta}\|.$

If $T$ is a sufficient statistic for $\theta$ with respect to
$L_{J}^{(\alpha)}$, then, from Proposition 2, we have

$L_{J}^{(\alpha)}(x_{1}^{n};\theta)=u(\theta,T)+v(x_{1}^{n})$ (20)

for some functions $u$ and $v$, where $v$ is independent of $\theta$. This
implies that

$\dfrac{\partial}{\partial\theta}L_{J}^{(\alpha)}(x_{1}^{n};\theta)=u_{1}(\theta,T),$
(21)

where
$u_{1}(\theta,T)\coloneqq\dfrac{\partial}{\partial\theta}\big{[}u(\theta,T)\big{]}$.
Comparing ((a)) and (21), we have

$\displaystyle{u_{1}(\theta,T)}$ $\displaystyle=$
$\displaystyle\frac{1}{\alpha-1}\frac{\sum_{i=1}^{n}\frac{\partial}{\partial\theta}\big{[}p_{\theta}(x_{i})^{\alpha-1}\big{]}}{\sum_{i=1}^{n}p_{\theta}(x_{i})^{\alpha-1}}-\frac{\partial}{\partial\theta}\log\|p_{\theta}\|.$

Now, for a particular value of $\theta\in\Theta$, say $\theta_{0}$, ((a)) can
be re-written as

$\displaystyle u_{1}(\theta_{0},T)$
$\displaystyle=\frac{1}{\alpha-1}\frac{\sum_{i=1}^{n}\Big{[}\frac{\partial}{\partial\theta}\big{\\{}p_{\theta}(x_{i})^{\alpha-1}\big{\\}}\Big{]}_{\theta_{0}}}{\sum_{i=1}^{n}p_{\theta_{0}}(x_{i})^{\alpha-1}}-\Big{[}\frac{\partial}{\partial\theta}\log\|p_{\theta}\|\Big{]}_{\theta_{0}}$
$\displaystyle=\frac{1}{\alpha-1}\frac{\sum_{i=1}^{n}g(x_{i})}{\sum_{i=1}^{n}h(x_{i})}-\Big{[}\frac{\partial}{\partial\theta}\log\|p_{\theta}\|\Big{]}_{\theta_{0}},$
(23)

where
$g(x_{i})=\big{[}\frac{\partial}{\partial\theta}\big{\\{}p_{\theta}(x_{i})^{\alpha-1}\big{\\}}\big{]}_{\theta_{0}}$,
and $h(x_{i})=p_{\theta_{0}}(x_{i})^{\alpha-1}$.

Let us denote $\widetilde{T}_{1}\coloneqq\sum_{i=1}^{n}g(x_{i})$ and
$\widetilde{T}_{2}\coloneqq\sum_{i=1}^{n}h(x_{i})$. Then, from ((a)), there
exists a function $\Phi$ such that

$T=\Phi(\widetilde{T}_{1},\widetilde{T}_{2}).$ (24)

Using this, (21) can be re-written as

$\dfrac{\partial}{\partial\theta}L_{J}^{(\alpha)}(x_{1}^{n};\theta)=u_{2}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})$
(25)

for some function $u_{2}$. Observe also that, (7) can be re-written as

$\displaystyle L_{J}^{(\alpha)}(x_{1}^{n};\theta)$
$\displaystyle=\dfrac{1}{\alpha-1}\log\bigg{[}\dfrac{1}{n}\sum\limits_{i=1}^{n}p_{\theta}({x}_{i})^{\alpha-1}\bigg{]}-\dfrac{1}{\alpha-1}\log\|p_{\theta}\|^{\alpha-1}$
$\displaystyle=\dfrac{1}{\alpha-1}\log\Bigg{[}\frac{1}{n}\sum_{i=1}^{n}\frac{p_{\theta}(x_{i})^{\alpha-1}}{\|p_{\theta}\|^{\alpha-1}}\Bigg{]}.$

This implies that

$\exp\Big{[}(\alpha-1)L_{J}^{(\alpha)}(x_{1}^{n};\theta)\Big{]}=\frac{1}{n}\sum_{i=1}^{n}\frac{p_{\theta}(x_{i})^{\alpha-1}}{\|p_{\theta}\|^{\alpha-1}}.$

Further (20) implies that

$\displaystyle{\exp\Big{[}(\alpha-1)L_{J}^{(\alpha)}(x_{1}^{n};\theta)\Big{]}}$
$\displaystyle=$
$\displaystyle\exp\Big{[}(\alpha-1)u(T,\theta)\Big{]}\cdot\exp\Big{[}(\alpha-1)v(x_{1}^{n})\Big{]}.$

Thus we have

$\displaystyle{\frac{1}{n}\sum_{i=1}^{n}\frac{p_{\theta}(x_{i})^{\alpha-1}}{\|p_{\theta}\|^{\alpha-1}}}$
$\displaystyle=$
$\displaystyle\exp\big{[}(\alpha-1)u(T,\theta)\big{]}\cdot\exp\big{[}(\alpha-1)v(x_{1}^{n})\big{]}.$

Fixing $\theta=\theta_{0}$, the above equation becomes

$\displaystyle\widetilde{T}_{2}$ $\displaystyle=$ $\displaystyle
n\|p_{\theta_{0}}\|^{\alpha-1}\exp\Big{[}(\alpha-1)u(T,\theta_{0})\Big{]}\exp\Big{[}(\alpha-1)v(x_{1}^{n})\Big{]}.$

Since $T=\Phi(\widetilde{T}_{1},\widetilde{T}_{2})$, this implies that
$v(x_{1}^{n})$ must be a function of $\widetilde{T}_{1}$ and
$\widetilde{T}_{2}$ only. Therefore, ((a)) can be re-written as

$\sum_{i=1}^{n}{p_{\theta}(x_{i})^{\alpha-1}}=u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2}),$
(27)

where $u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\coloneqq
n\|p_{\theta}\|^{\alpha-1}\exp\big{[}(\alpha-1)u\big{(}\Phi(\widetilde{T}_{1},\widetilde{T}_{2}),\theta\big{)}\big{]}\cdot\exp\big{[}(\alpha-1)v(x_{1}^{n})\big{]}$.
Now, taking partial derivative with respect to $x_{j}$ on both sides of (27),
we have

$\displaystyle\frac{\partial}{\partial x_{j}}p_{\theta}(x_{j})^{\alpha-1}$
$\displaystyle=\frac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\frac{\partial\widetilde{T}_{1}}{\partial
x_{j}}+\frac{\partial}{\partial\widetilde{T}_{2}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\frac{\partial\widetilde{T}_{2}}{\partial
x_{j}}$
$\displaystyle=\frac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\frac{\partial}{\partial
x_{j}}g(x_{j})+\frac{\partial}{\partial\widetilde{T}_{2}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\frac{\partial}{\partial
x_{j}}h(x_{j}).$ (28)

Since the left hand side of ((a)) is a function of $x_{j}$ and $\theta$ only,
we shall denote it by $A(x_{j},\theta)$. Thus taking partial derivative with
respect to $x_{i}$ for $i\neq j$ on both sides of ((a)), we get

$\displaystyle\frac{\partial}{\partial
x_{i}}\Big{[}\frac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\Big{]}~{}\frac{\partial}{\partial
x_{j}}[g(x_{j})]+\frac{\partial}{\partial
x_{i}}\Big{[}\frac{\partial}{\partial\widetilde{T}_{2}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\Big{]}~{}\frac{\partial}{\partial
x_{j}}[h(x_{j})]=0.~{}~{}~{}$ (29)

From (29), it is clear that

$\displaystyle\frac{\partial}{\partial
x_{i}}\Big{[}\frac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\Big{]}=0~{}\text{if
and only if}~{}\frac{\partial}{\partial
x_{i}}\Big{[}\frac{\partial}{\partial\widetilde{T}_{2}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\Big{]}=0,$
(30)

since neither $\frac{\partial}{\partial x_{j}}[g(x_{j})]$ nor
$\frac{\partial}{\partial x_{j}}[h(x_{j})]$ can be zero.

Equivalently,

$\displaystyle\frac{\partial}{\partial
x_{i}}\Big{[}\frac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\Big{]}\neq
0~{}\text{if and only if}~{}\frac{\partial}{\partial
x_{i}}\Big{[}\frac{\partial}{\partial\widetilde{T}_{2}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\Big{]}\neq
0.$

Suppose that $\dfrac{\partial}{\partial
x_{i}}\Big{[}\dfrac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\Big{]}\neq
0$. Then (29) can be re-written as

$\displaystyle\dfrac{\frac{\partial}{\partial
x_{i}}\Big{[}\frac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\Big{]}}{\frac{\partial}{\partial
x_{i}}\Big{[}\frac{\partial}{\partial\widetilde{T}_{2}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\Big{]}}$
$\displaystyle=$ $\displaystyle-\dfrac{\frac{\partial}{\partial
x_{j}}[h(x_{j})]}{\frac{\partial}{\partial
x_{j}}[g(x_{j})]}\eqqcolon\xi(x_{j}).$ (31)

Since the right hand side of (31) is a function of $x_{j}$ alone, both
$\frac{\partial}{\partial\widetilde{T}_{1}}[u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})]$
and
$\frac{\partial}{\partial\widetilde{T}_{2}}[u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})]$
must be of the following form

$\displaystyle\frac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})=\Psi(x_{1}^{n},\theta)S_{1}(x_{j})+V_{1}(x_{1},\dots,x_{i-1},x_{i+1},\dots,x_{n},\theta)$

and

$\displaystyle\frac{\partial}{\partial\widetilde{T}_{2}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})=\Psi(x_{1}^{n},\theta)S_{2}(x_{j})+V_{2}(x_{1},\dots,x_{i-1},x_{i+1},\dots,x_{n},\theta),$

for some real-valued functions $\Psi,S_{1},S_{2},V_{1}$, and $V_{2}$. Observe
that (31) holds for any $i\neq j$. This implies $V_{1}$ and $V_{2}$ must be
functions of $x_{j}$ and $\theta$ only. Using these in ((a)), we get

$A(x_{j},\theta)=\Psi(x_{1}^{n},\theta)K_{1}(x_{j})+K_{2}(x_{j},\theta),$ (32)

where

$\displaystyle{K_{1}(x_{j})}$ $\displaystyle\coloneqq$ $\displaystyle
S_{1}(x_{j})~{}\frac{\partial}{\partial
x_{j}}[g(x_{j})]-S_{2}(x_{j})~{}\frac{\partial}{\partial x_{j}}[h(x_{j})],$

and

$\displaystyle{K_{2}(x_{j},\theta)}$ $\displaystyle\coloneqq$ $\displaystyle
V_{1}(x_{j},\theta)~{}\frac{\partial}{\partial
x_{j}}[g(x_{j})]-V_{2}(x_{j},\theta)~{}\frac{\partial}{\partial
x_{j}}[h(x_{j})].$

In order for (32) to hold, $\Psi$ must be a function of $x_{j}$ and $\theta$
only. This implies that both

$\dfrac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\quad\text{and}\quad\dfrac{\partial}{\partial\widetilde{T}_{2}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})$
(33)

must be a function of $x_{j}$ and $\theta$ only.

On the other hand, let $\frac{\partial}{\partial
x_{i}}\Big{[}\frac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\Big{]}=0$
for some $i\neq j$. Then from (30), we have $\frac{\partial}{\partial
x_{i}}\Big{[}\frac{\partial}{\partial\widetilde{T}_{2}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})\Big{]}=0$.
This implies that both
$\frac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})$
and
$\frac{\partial}{\partial\widetilde{T}_{2}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})$
are independent of $x_{i}$’s where $i\neq j$. Thus, in this case also, (33)
holds.

Since $u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})$ is a function of
$\widetilde{T}_{1}$, $\widetilde{T}_{2}$ and $\theta$ only, both
$\frac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})$
and
$\frac{\partial}{\partial\widetilde{T}_{2}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})$
are also functions of $\widetilde{T}_{1},\widetilde{T}_{2}$ and $\theta$ only.
But both $\widetilde{T}_{1}$ and $\widetilde{T}_{2}$ are symmetric functions
of the $x_{i}$’s, $i=1,\dots,n$. This implies that both quantities in (33)
must only be a function of $\theta$. Let

$\displaystyle\frac{\partial}{\partial\widetilde{T}_{1}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})=:w_{1}(\theta)\quad\text{and}\quad\frac{\partial}{\partial\widetilde{T}_{2}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})=:w_{2}(\theta)$

for some $w_{1}$ and $w_{2}$ which are functions of $\theta$ only. Then we
have

$u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2})=w_{1}(\theta)\widetilde{T}_{1}+w_{2}(\theta)\widetilde{T}_{2}+w_{3}(\theta).$

Using (27), we get

$p_{\theta}(x_{i})^{\alpha-1}=w_{1}(\theta)g(x_{i})+w_{2}(\theta)h(x_{i})+w_{3}(\theta)$

for $i=1,\dots,n$.

  * (b)

Let $k<s$. The proof for this case is quite similar to that of case (a). For
the sake of simplicity, we give a sketch of proof for $k=1$ and $s=2$.

Let $(T_{1},T_{2})$ be a pair of sufficient statistic of $\theta$ with respect
to Jones et al. likelihood function $L_{J}^{(\alpha)}$. Then, proceeding as in
Case (a), one would get an equation similar to (24) with four symmetric
functions of $x_{i}$’s (instead of two), say,
$\widetilde{T}_{1}=\sum_{i=1}^{n}g_{1}(x_{i})$,
$\widetilde{T}_{2}=\sum_{i=1}^{n}g_{2}(x_{i})$,
$\widetilde{T}_{3}=\sum_{i=1}^{n}h_{1}(x_{i})$ and
$\widetilde{T}_{4}=\sum_{i=1}^{n}h_{2}(x_{i})$. Hence (27) holds with
$\widetilde{T}_{1},\widetilde{T}_{2},\widetilde{T}_{3}$, and
$\widetilde{T}_{4}$ instead of only $\widetilde{T}_{1}$ and
$\widetilde{T}_{2}$. Then a similar calculation yields that all
$\frac{\partial}{\partial\widetilde{T}_{i}}u_{3}(\theta,\widetilde{T}_{1},\widetilde{T}_{2},\widetilde{T}_{3},\widetilde{T}_{4})$
for $i=1,2,3,4$ are a function of $\theta$ only. This further implies

$\displaystyle{p_{\theta}(x_{i})^{\alpha-1}=\sum\limits_{j=1}^{2}w_{j}^{(1)}(\theta)g_{j}(x_{i})}+\sum\limits_{j=1}^{2}w_{j}^{(2)}(\theta)h_{j}(x_{i})+w^{(3)}(\theta)$

for $i=1,\dots,n$. This completes the proof.

∎

The converse of the above theorem is also true. Indeed, in the following, we
show that an $\mathbb{M}^{(\alpha)}$-family always has a fixed number of
sufficient statistics with respect to the Jones et al. likelihood function
$L_{J}^{(\alpha)}$.

###### Theorem 8

Consider a parametric family of probability distributions
$\Pi=\\{p_{\theta}:\theta\in\Theta\subset\mathbb{R}^{k}\\}$ having a common
support $\mathbb{S}$. Let $T=[T_{1},\dots,T_{s}]^{\top}$ be $s$ statistics
defined on $\mathbb{S}^{n}$ where $s\geq k$. $T$ is a sufficient statistic for
$\theta$ with respect to Jones et al. likelihood function $L_{J}^{(\alpha)}$
if the following hold.

  * (a)

$\Pi$ is a $k$-parameter $\mathbb{M}^{(\alpha)}$-family as in Definition 6,

  * (b)

there exist real valued functions $\psi_{1},\dots,\psi_{s}$ defined on
$\mathscr{T}$ such that for any i.i.d. sample $x_{1}^{n}$

$\psi_{i}(T({x}_{1}^{n}))=\overline{f_{i}}\big{/}\overline{h}$ (34)

for $i=1,\dots,s$, where
$\overline{f_{i}}=\frac{1}{n}\sum_{j=1}^{n}f_{i}({x}_{j})$ and
$\overline{h}\coloneqq\frac{1}{n}\sum_{j=1}^{n}h({x}_{j})$.

###### Proof:

See Sec-B.1 of Appendix. ∎

###### Remark 3

It is easy to see from the above theorem that
$\overline{f}/\overline{h}\coloneqq\big{[}\overline{f_{1}}/\overline{h},\dots,\overline{f_{s}}/\overline{h}\big{]}^{\top}$
is a sufficient statistic for the $k$-parameter $\mathbb{M}^{(\alpha)}$-family
with respect to Jones et al. likelihood function.

###### Example 1

Consider the Student distributions with location parameter $\mu$, scale
parameter $\sigma$ and degrees of freedom parameter $\nu\in(2,\infty):$

$p_{\mu,\sigma}(x)=N_{\sigma,\nu}\bigg{[}1+\frac{(x-\mu)^{2}}{\nu\sigma^{2}}\bigg{]}^{-\frac{\nu+1}{2}},$
(35)

for $x\in\mathbb{R}$, where
$N_{\sigma,\nu}\coloneqq\dfrac{\Gamma(\frac{\nu+1}{2})}{\sqrt{\nu\pi\sigma^{2}}~{}\Gamma(\frac{\nu}{2})}$
is the normalizing factor. Observe that (35) includes (8) for $\nu=3$.

Let $\nu$ be known, and $\mu$ and $\sigma^{2}$ be unknown. Letting
$\alpha=\dfrac{\nu-1}{\nu+1}$ and $\theta=[\mu,\sigma^{2}]^{\top}$, (35) can
be re-written as

$\displaystyle{p_{\theta}(x)=N_{\theta,\alpha}\Big{(}1+\frac{\mu^{2}b_{\alpha}}{\sigma^{2}}\Big{)}^{\frac{1}{\alpha-1}}}\bigg{[}1+\frac{b_{\alpha}x^{2}}{\sigma^{2}+b_{\alpha}\mu^{2}}-\frac{2\mu
b_{\alpha}x}{\sigma^{2}+b_{\alpha}\mu^{2}}\bigg{]}^{\frac{1}{\alpha-1}},~{}~{}$
(36)

where $b_{\alpha}=\dfrac{1}{\nu}=\dfrac{1-\alpha}{1+\alpha}$ and
$N_{\theta,\alpha}=N_{\sigma,\nu}$.

(36) implies that Student distributions form an $\mathbb{M}^{(\alpha)}$-family
with

$\displaystyle\begin{array}[]{ll}h(x)\equiv 1,\quad
Z(\theta)=N_{\theta,\alpha}\Big{(}1+\frac{\mu^{2}b_{\alpha}}{\sigma^{2}}\Big{)}^{\frac{1}{\alpha-1}},\\\
{w(\theta)=\big{[}w_{1}(\theta),w_{2}(\theta)\big{]}^{\top},\quad
f(x)=\big{[}f_{1}(x),f_{2}(x))\big{]}^{\top},}\\\
{w_{1}(\theta)={b_{\alpha}}\big{/}{(\sigma^{2}+b_{\alpha}\mu^{2})},\quad
f_{1}(x)=x^{2},}\\\ {w_{2}(\theta)={-2\mu
b_{\alpha}}\big{/}{(\sigma^{2}+b_{\alpha}\mu^{2})},\quad\text{and}\quad
f_{2}(x)=x}.\end{array}$

By Theorem 8, $\bar{f_{1}}=\frac{1}{n}\sum_{i=1}^{n}x_{i}^{2}$ and
$\bar{f_{2}}=\frac{1}{n}\sum_{i=1}^{n}x_{i}$ are sufficient statistics for
$\mu$ and $\sigma$ with respect to Jones et al. likelihood function.

###  III-A Generalized Fisher-Darmois-Koopman-Pitman theorem for Basu et al.
likelihood function

In [24], we characterized the probability distributions that have a fixed
number of sufficient statistics with respect to the Basu et al. likelihood
function (only for the case $\Theta\subset\mathbb{R}$ though). We termed the
associated family of probability distributions as
$\mathbb{B}^{(\alpha)}$-family. In the following, we first define the
$\mathbb{B}^{(\alpha)}$-family and then state the results concerning
sufficient statistics for the same.

###### Definition 9

Let $h,w,f,\Theta$ and $\mathbb{S}$ be as in Definition 6. A $k$-parameter
$\mathbb{B}^{(\alpha)}$-family characterized by $h,w,f,\Theta$ and
$\mathbb{S}$ is defined by
$\mathbb{B}^{(\alpha)}=\\{p_{\theta}\colon\theta\in\Theta\\}$, where

$\displaystyle{p_{\theta}(x)}=\left\\{\begin{array}[]{ll}{[h(x)+F(\theta)+w(\theta)^{\top}f(x)]^{\frac{1}{\alpha-1}}}\text{~{}for~{}}x\in\mathbb{S}\\\
{0}\hskip 128.0374pt\text{~{}otherwise}\end{array}\right.$ (39)

for some $F:\Theta\to\mathbb{R}$, the normalizing factor.

###### Remark 4

  * (i)

Student distributions fall in $\mathbb{B}^{(\alpha)}$-family [25, Ex. 1].

  * (ii)

$\mathbb{B}^{(\alpha)}$-family is also a generalization of the usual
exponential family [25, 24]. A more general form of this family can be found
in [14] in connection to the projection theorem of Bregman divergences (c.f.
[25]). This family was also studied in [2, 37]. A comparison between
$\mathbb{B}^{(\alpha)}$ and $\mathbb{M}^{(\alpha)}$ families can be found in
[25, Rem. 2].

The following is the generalized Fisher-Darmois-Koopman-Pitman theorem with
respect to Basu et al. likelihood function $L_{B}^{(\alpha)}$.

###### Theorem 10

Let $k\leq s<n$. If $[T_{1},\dots,T_{s}]^{\top}$ is a sufficient statistic for
$\theta$ with respect to the Basu et al. likelihood function
$L_{B}^{(\alpha)}$, then for $x\in\mathbb{S}$, there exist functions
$h:\mathbb{S}\to\mathbb{R}$, $f_{i}:\mathbb{S}\to\mathbb{R}$ and
$w_{i}:\Theta\to\mathbb{R}$, $i=1,\dots,s$ such that $p_{\theta}(x)$ can be
expressed as (39).

###### Proof:

The proof for the case $s=k$ can be found in [24, Th. 2]. On the other hand,
let $k<s$. We give the sketch of the proof for $k=1$ and $s=2$ as in Theorem
7. The other cases can be proved similarly. Following the same steps as in
Theorem 7, one gets (25) using $L_{B}^{(\alpha)}(x_{1}^{n};\theta)$ in place
of $L_{J}^{(\alpha)}(x_{1}^{n};\theta)$. This, upon taking partial derivative
with respect to $x_{j}$ first, and then with respect to some $x_{i}$, $i\neq
j$, implies (29). Then, proceeding as in Theorem 7, one can prove the result.
∎

The converse of the above theorem is also true.

###### Theorem 11

Consider a $k$-parameter $\mathbb{B}^{(\alpha)}$-family as in Definition 9.
Let $T=[T_{1},\dots,T_{s}]^{\top}$ be $s$ statistics defined on
$\mathbb{S}^{n}$ where $s\geq k$. Then $T$ is sufficient statistic for
$\theta$ with respect to $L_{B}^{(\alpha)}$ if there exist real valued
functions $\psi_{1},\dots,\psi_{s}$ defined on $\mathscr{T}$ such that
$\psi_{i}(T({x}_{1}^{n}))=\overline{f_{i}}$ for any i.i.d. sample $x_{1}^{n}$
and $i=1,\dots,s$. In particular,
$\overline{f}\coloneqq\big{[}\overline{f_{1}},\dots,\overline{f_{s}}\big{]}^{\top}$
is a sufficient statistic for $\theta$ with respect to $L_{B}^{(\alpha)}$.

###### Proof:

See [24, Th. 4]. ∎

###### Example 2

Consider the Student distributions as in Example 1. In [25], we showed that
these also form a 2-parameter $\mathbb{B}^{(\alpha)}$-family. Using Theorem
11, the same statistics as in Example 1 form sufficient statistics for their
parameters with respect to Basu et al. likelihood function.

###  III-B Minimal sufficiency with respect to Basu et al. and Jones et al.
likelihood functions

In the following, we show that $\bar{f}/\bar{h}$ and $\bar{f}$ are also
minimal, respectively, for regular $\mathbb{M}^{(\alpha)}$ and
$\mathbb{B}^{(\alpha)}$ families. An $\mathbb{M}^{(\alpha)}$-family as in
Definition 6 is said to be _regular_ if the following are satisfied (c.f. [25,
Def. 3]).

  * (i)

The support $\mathbb{S}$ does not depend on the parameter $\theta$.

  * (ii)

The number of unknown parameters is the same as the number of $w_{i}$’s, that
is, $s=k$.

  * (iii)

$1,w_{1},\dots,w_{k}$ are linearly independent.

  * (iv)

$f_{1},\dots,f_{k}$ are linearly independent.

Similarly, a $\mathbb{B}^{(\alpha)}$-family as in Definition 9 is said to be
_regular_ if $1,f_{1},\dots,f_{k}$ are linearly independent along with
conditions (i)-(iii) above (c.f. [25, Def. 1]).

###### Theorem 12

The statistics
$\overline{f}/\overline{h}=\big{[}\overline{f_{1}}/\overline{h},\dots,\overline{f_{k}}/\overline{h}\big{]}^{\top}$
and
$\overline{f}=\big{[}\overline{f_{1}},\dots,\overline{f_{k}}\big{]}^{\top}$
are minimal sufficient statistics, respectively, for a $k$-parameter regular
$\mathbb{M}^{(\alpha)}$-family and a $k$-parameter regular
$\mathbb{B}^{(\alpha)}$-family with respect to Jones et al. and Basu et al.
likelihood functions.

###### Proof:

See Sec-B.2 of Appendix. ∎

###### Remark 5

In view of Theorem 12, the sufficient statistics derived in Example 1 for
Student distributions are minimal too (with respect to both Basu et al. and
Jones et al. likelihood functions).

##  IV Rao-Blackwell theorem for Jones et al. estimation

Let $X_{1}^{n}=x_{1}^{n}$ be an i.i.d. sample drawn according to some
$p_{\theta}\in\Pi$. Here and afterward, we assume
$\theta\in\Theta\subset\mathbb{R}$ unless otherwise stated. Let
$\widehat{\theta}\coloneqq\widehat{\theta}(x_{1}^{n})$ be an unbiased
estimator of $\theta$. Let $T\coloneqq T(x_{1}^{n})$ be a sufficient statistic
for $\theta$ with respect to the usual log-likelihood function. Let us
consider the conditional expectation
$\mathbb{E}_{\theta}[\widehat{\theta}|T]$. It is easy to show from Fisher’s
definition of sufficiency that $\mathbb{E}_{\theta}[\widehat{\theta}|T]$ is
independent of $\theta$ [4]. Hence $\mathbb{E}_{\theta}[\widehat{\theta}|T]$
is a function of $T$ alone. Let
$\phi(T)\coloneqq\mathbb{E}_{\theta}[\widehat{\theta}|T]$. Then by Rao-
Blackwell theorem, $\phi(T)$ is a uniformly better unbiased estimator of
$\theta$ than $\widehat{\theta}$ (see [42, 4]). In other words,

  * 1.

$\mathbb{E}_{\theta}[\phi(T)]=\mathbb{E}_{\theta}[\widehat{\theta}]=\theta$,
and

  * 2.

$\text{Var}_{\theta}[\phi(T)]\leq\text{Var}_{\theta}[\widehat{\theta}]$

for all $\theta\in\Theta$, and equality holds if and only if
$\widehat{\theta}$ is a function of the sufficient statistic $T$. This implies
that the ‘best’ unbiased estimator of $\theta$ (the one with the least
variance) is a function of the sufficient statistic $T$ only, that is, one
from the family $\\{\psi(T):\mathbb{E}_{\theta}[\psi(T)]=\theta\\}$.

Let us consider an exponential family $\mathcal{E}$ as in (5).
$\bar{f}=\frac{1}{n}\sum_{i=1}^{n}f(x_{i})$ is a sufficient statistic for
$\theta$ with respect to the usual log-likelihood function [34] and
$\mathbb{E}_{\theta}[\bar{f}]=\mathbb{E}_{\theta}[f(X)]=:\tau(\theta)$, called
the _mean value parameter_. Let $\widehat{\theta}$ be any unbiased estimator
of $\tau(\theta)$. Then by Rao-Blackwell theorem, we have
$\phi(\bar{f})\coloneqq\mathbb{E}_{\theta}[\widehat{\theta}|\bar{f}]$ is also
an unbiased estimator of $\tau(\theta)$ and has variance not more than that of
$\widehat{\theta}$. That is,

$\displaystyle\mathbb{E}_{\theta}[\phi(\bar{f})]=\tau(\theta)\quad\text{and}\quad\text{Var}_{\theta}[\phi(\bar{f})]\leq\text{Var}_{\theta}[\widehat{\theta}].$
(40)

Now consider $\psi(\bar{f})=\phi(\bar{f})-\bar{f}$. Then

$\mathbb{E}_{\theta}[\psi(\bar{f})]=\mathbb{E}_{\theta}[\phi(\bar{f})]-\mathbb{E}_{\theta}[\bar{f}]=\tau(\theta)-\tau(\theta)=0,$

and

$\displaystyle{\text{Var}_{\theta}[\phi(\bar{f})]}$ $\displaystyle=$
$\displaystyle\text{Var}_{\theta}[\psi(\bar{f})+\bar{f}]$ (41)
$\displaystyle=$
$\displaystyle\text{Var}_{\theta}[\psi(\bar{f})]+\text{Var}_{\theta}[\bar{f}]+2~{}\text{Cov}_{\theta}[\psi(\bar{f}),\bar{f}].~{}~{}~{}~{}$

It can be shown, under some mild regularity conditions, that
$\text{Cov}_{\theta}[\psi(\bar{f}),\bar{f}]=0$. See Sec-C.1 of Appendix for
the detailed derivation. Hence (41) reduces to

$\text{Var}_{\theta}[\phi(\bar{f})]=\text{Var}_{\theta}[\psi(\bar{f})]+\text{Var}_{\theta}[\bar{f}]\geq\text{Var}_{\theta}[\bar{f}].$

Thus from (40), for any $\widehat{\theta}$ with
$\mathbb{E}_{\theta}[\widehat{\theta}]=\mathbb{E}_{\theta}[\bar{f}]$, we have

$\text{Var}_{\theta}[\widehat{\theta}]\geq\text{Var}_{\theta}[\phi(\bar{f})]\geq\text{Var}_{\theta}[\bar{f}].$

This shows that the variance of $\bar{f}$ is not more than that of any
unbiased estimator of $\tau(\theta)$. Thus we have the following (c.f. [44,
10]).

###### Proposition 13

Consider a regular exponential family as in (5). Let
$\mathbb{E}_{\theta}[f(X)]=\tau(\theta)$. Then there does not exist any
unbiased estimator of $\tau(\theta)$ that has lesser variance than that of
$\bar{f}$. That is, $\bar{f}$ is the best estimator of its expected value.

Further, it can be easily shown that (c.f. [43, 44])

$\text{Var}_{\theta}[\bar{f}]=\frac{1}{n}\text{Var}[f(X)]=\frac{[\tau^{\prime}(\theta)]^{2}}{n~{}I(\theta)},$

where $I(\theta)$ is the Fisher information, that is,
$I(\theta)=\mathbb{E}_{\theta}\Big{[}\big{(}\frac{\partial}{\partial\theta}\log
p_{\theta}(X)\big{)}^{2}\Big{]}$. This and Proposition 13 imply that

$\text{Var}_{\theta}[\widehat{\theta}]\geq\frac{[\tau^{\prime}(\theta)]^{2}}{n~{}I(\theta)}.$
(42)

Equality holds in (42) if and only if $\widehat{\theta}=\bar{f}$ almost
surely. Notice that (42) is the well-known _Cramér-Rao inequality_ [42, 12].

Observe that, in Rao-Blackwell theorem, instead of a sufficient statistic $T$,
if one conditions on some other statistic, then also the resulting estimator
has variance not more than that of the given estimator. However, conditioning
on any other statistic does not guarantee that the conditional expectation is
independent of $\theta$. Also by Fisher-Darmois-Koopman-Pitman theorem, a
sufficient statistic with respect to the usual log-likelihood function exists
if and only if the underlying model is exponential [31, 41, 16]. Hence Rao-
Blackwell theorem is not applicable outside the exponential family. However,
in Section III, we showed that there are models that have a fixed number of
sufficient statistics with respect to likelihood functions other than log-
likelihood. In particular, we showed that $\mathbb{M}^{(\alpha)}$-family is
one such model which has a fixed number of sufficient statistics with respect
to Jones et al. likelihood function (see Theorem 7 and Theorem 8). In the
following, we generalize the Rao-Blackwell theorem for Jones et al. likelihood
function. Later we will apply this to $\mathbb{M}^{(\alpha)}$-family and find
Rao-Blackwell type “best” estimator. This also helps us find a lower bound for
the variance of the estimators of $\mathbb{M}^{(\alpha)}$-family as an
alternative to the usual CRLB.

The Jones et al. likelihood function $L_{J}^{(\alpha)}$ as in (7) can be re-
written as

$\displaystyle L_{J}^{(\alpha)}(x_{1}^{n};\theta)$ $\displaystyle=$
$\displaystyle\log\Big{[}{\Big{(}\frac{1}{n}\sum\limits_{i=1}^{n}p_{\theta}(x_{i})^{\alpha-1}\Big{)}^{\frac{1}{\alpha-1}}}/{\|p_{\theta}\|}\Big{]}.$

Then the deformed probability distribution associated with $L_{J}^{(\alpha)}$
is given by

$\displaystyle p_{\theta}^{*}(x_{1}^{n})$ $\displaystyle\coloneqq$
$\displaystyle\frac{\exp[L_{J}^{(\alpha)}(x_{1}^{n};\theta)]}{\int\exp[L_{J}^{(\alpha)}(y_{1}^{n};\theta)]dy_{1}^{n}}$
(43) $\displaystyle~{}=$
$\displaystyle\dfrac{\bigg{(}\frac{1}{n}\sum\limits_{i=1}^{n}p_{\theta}(x_{i})^{\alpha-1}\bigg{)}^{\frac{1}{\alpha-1}}}{\int\bigg{(}\frac{1}{n}\sum\limits_{i=1}^{n}p_{\theta}(y_{i})^{\alpha-1}\bigg{)}^{\frac{1}{\alpha-1}}dy_{1}^{n}}.$

Observe that when $\alpha\to 1$, $L_{J}^{(\alpha)}(x_{1}^{n};\theta)$
coincides with the log-likelihood function $L(x_{1}^{n};\theta)$. Hence, in
this case, $p_{\theta}^{*}(x_{1}^{n})=p_{\theta}(x_{1}^{n})$.

Let $T$ be a statistic of the sample $x_{1}^{n}$. Then, from (15), we have

$p_{\theta{{}_{X_{1}^{n}|T}}}^{*}(x_{1}^{n}|t)=\begin{cases}\dfrac{p_{\theta}^{*}(x_{1}^{n})}{q_{\theta}^{*}(t)}&\text{if}~{}x_{1}^{n}\in
A_{t}\\\ 0&\text{otherwise}.\end{cases}$

Observe that, as $\alpha\to 1$, the above coincides with the usual conditional
distribution of $x_{1}^{n}$ given $T=t$.

Let $\widehat{\theta}\coloneqq\widehat{\theta}(x_{1}^{n})$ be an unbiased
estimator of
$\tau^{*}(\theta)\coloneqq\mathbb{E}_{\theta}^{*}[\widehat{\theta}]$, where
$\mathbb{E}_{\theta}^{*}[\cdots]$ denotes the expectation with respect to
$p_{\theta}^{*}$. Let $T\coloneqq T(x_{1}^{n})$ be a sufficient statistic for
$\theta$ with respect to Jones et al. likelihood function $L_{J}^{(\alpha)}$.
Let us consider the conditional expectation

$\displaystyle\mathbb{E}_{\theta}^{*}\big{[}\widehat{\theta}|T=T(x_{1}^{n})\big{]}=\int\widehat{\theta}(y_{1}^{n})p_{\theta}^{*}(y_{1}^{n}|T(x_{1}^{n}))dy_{1}^{n}.$
(44)

In the following, we first establish some properties of (44).

###### Proposition 14

The conditional expectation $\mathbb{E}_{\theta}^{*}[\widehat{\theta}|T]$ is
independent of $\theta$.

###### Proof:

See Sec-C.2 of Appendix. ∎

Let

$\mathbb{E}_{\theta}^{*}\big{[}\widehat{\theta}|T=T(x_{1}^{n})\big{]}\eqqcolon\phi^{*}(T(x_{1}^{n})).$

Let us consider the expectation of $\phi^{*}(T)$ with respect to
$p_{\theta}^{*}$. That is,

$\mathbb{E}_{\theta}^{*}[\phi^{*}(T(X_{1}^{n}))]=\int
p_{\theta}^{*}(x_{1}^{n})\phi^{*}(T(x_{1}^{n}))dx_{1}^{n}.$

Then we have the following.

###### Proposition 15

$\phi^{*}(T)$ is unbiased for $\tau^{*}(\theta)$, that is,
$\mathbb{E}_{\theta}^{*}[\phi^{*}(T(X_{1}^{n}))]=\mathbb{E}_{\theta}^{*}[\widehat{\theta}(X_{1}^{n}))]=\tau^{*}(\theta).$

###### Proof:

See Sec-C.3 of Appendix. ∎

Thus, if we have an unbiased estimator of $\tau^{*}(\theta)$ and a sufficient
statistic with respect to Jones et al. likelihood function, we can always get
another unbiased estimator for $\tau^{*}(\theta)$ that is based only on the
sufficient statistic. Let us now consider the following two classes of
estimators based on the sufficient statistic $T:$

$\displaystyle\mathcal{A}$ $\displaystyle=$
$\displaystyle\\{\mathbb{E}^{*}_{\theta}[\widehat{\theta}|T]:\quad\widehat{\theta}~{}\text{is
an unbiased estimator of}\tau^{*}(\theta),~{}\text{that
is,}~{}\mathbb{E}_{\theta}^{*}[\widehat{\theta}]=\tau^{*}(\theta)\\},$
$\displaystyle\mathcal{B}$ $\displaystyle=$
$\displaystyle\\{\psi(T):\mathbb{E}^{*}_{\theta}[\psi(T)]=\tau^{*}(\theta)\\}.$
(45)

###### Proposition 16

The above two classes of estimators are the same, that is
$\mathcal{A}=\mathcal{B}$.

###### Proof:

See Sec-C.4 of Appendix. ∎

The above proposition suggests that any unbiased estimator of
$\tau^{*}(\theta)$ that is a function of the sufficient statistic $T$ is of
the form $\mathbb{E}^{*}_{\theta}[\widehat{\theta}|T]$, where
$\widehat{\theta}$ is an unbiased estimator of $\tau^{*}(\theta)$. In the
following, we compare the variance of an estimator of the form (IV) and any
other unbiased estimator of $\tau^{*}(\theta)$.

###### Lemma 17

Let $\widehat{\theta}$ be any unbiased estimator of $\tau^{*}(\theta)$ and
$\phi^{*}(T)=\mathbb{E}_{\theta}^{*}[\widehat{\theta}|T]$. Let
$\psi^{*}(T)\in\mathcal{B}$. Then

$\displaystyle{\text{Var}^{*}_{\theta}[\widehat{\theta}]=\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\psi^{*}(T))^{2}]+\text{Var}^{*}_{\theta}[\psi^{*}(T)]}+2\mathbb{E}^{*}_{\theta}[\psi^{*}(T)\\{\phi^{*}(T)-\psi^{*}(T)\\}].~{}~{}$
(46)

###### Proof:

See Sec-C.5 of Appendix. ∎

The following (also the main result of this section) is a generalization of
Rao-Blackwell theorem for Jones et al. likelihood function.

###### Theorem 18

Let $\Pi=\\{p_{\theta}:\theta\in\Theta\subset\mathbb{R}\\}$ be a family of
probability distributions with common support $\mathbb{S}\subset\mathbb{R}$.
Consider the deformed family of probability distributions
$\Pi^{*}=\\{p_{\theta}^{*}\\}$ of $\Pi$, where $p_{\theta}^{*}$ is as in (43).
Let $\widehat{\theta}$ be an unbiased estimator for its expected value with
respect to $p_{\theta}^{*}$. Let
$\tau^{*}(\theta)\coloneqq\mathbb{E}_{\theta}^{*}[\widehat{\theta}]$. Suppose
$T$ is a sufficient statistic for $\theta$ with respect to the Jones et al.
likelihood function. Let
$\phi^{*}(T)\coloneqq\mathbb{E}_{\theta}^{*}[\widehat{\theta}|T]$. Then
$\phi^{*}(T)$ is a uniformly better estimator of $\tau^{*}(\theta)$ than
$\widehat{\theta}$, that is,

  * (a)

$\mathbb{E}_{\theta}^{*}[\phi^{*}(T)]=\tau^{*}(\theta)$,

  * (b)

$\text{Var}_{\theta}^{*}[\phi^{*}(T(X_{1}^{n}))]\leq\text{Var}_{\theta}^{*}[\widehat{\theta}(X_{1}^{n})]$,
and equality holds if and only if $\widehat{\theta}$ is a function of the
sufficient statistic $T$.

###### Proof:

See Sec-C.6 of Appendix. ∎

Theorem 18 implies that $\phi^{*}(T)$ is _uniformly a better estimator_ of
$\tau^{*}(\theta)$ than $\widehat{\theta}$. This means that the _best unbiased
estimator_ of $\tau^{*}(\theta)$ is a function of the sufficient statistic
$T$. The following result asserts that the estimator with minimum variance
(best estimator) is unique.

###### Proposition 19

The best unbiased estimator for $\tau^{*}(\theta)$ with respect to
$p_{\theta}^{*}$ is unique.

###### Proof:

See Sec-C.7 of Appendix. ∎

In the following, we apply the above results to the family of Bernoulli
distributions.

###  IV-A Example:

Observe that the family of Bernoulli distributions can be written as

$p_{\theta}(x)=(1-\theta)[1+x\cdot(2\theta-1)/(1-\theta)]\quad\text{for}\quad
x=0,1,$

where $\theta\in(0,1)$ is the unknown parameter. Thus Bernoulli distributions
form an $\mathbb{M}^{(\alpha)}$-family with $\alpha=2$, $h(x)\equiv 1$,
$Z(\theta)=1-\theta$, $w(\theta)=(2\theta-1)/(1-\theta)$ and $f(x)=x$. By
Theorem 8, $\sum_{i=1}^{n}x_{i}$ is a sufficient statistic for $\theta$ with
respect to the Jones et al. likelihood function.

The Jones et al. likelihood function for Bernoulli distributions is

$\displaystyle L_{J}^{(2)}(x_{1}^{n},\theta)$ $\displaystyle=$
$\displaystyle\log\Bigg{[}\frac{\frac{1}{n}\sum_{i=1}^{n}p_{\theta}(x_{i})}{(\mathbb{E}_{\theta}[p_{\theta}(X)])^{1/2}}\Bigg{]}$
$\displaystyle=$
$\displaystyle\log\Bigg{[}\frac{(1-\theta)^{1/2}\\{1+\overline{x}\cdot(2\theta-1)/(1-\theta)\\}}{(1+\theta(2\theta-1)/(1-\theta))^{1/2}}\Bigg{]}.$

The associated deformed probability distribution is given by

$\displaystyle p_{\theta}^{*}(x_{1}^{n})$ $\displaystyle=$
$\displaystyle\dfrac{\exp[L^{(2)}_{J}(x_{1}^{n},\theta)]}{\sum_{y_{1}^{n}}\exp[L^{(2)}_{J}(y_{1}^{n},\theta)]}$
$\displaystyle=$
$\displaystyle\frac{1+\overline{x}\cdot(2\theta-1)/(1-\theta)}{\sum_{y_{1}^{n}}[1+\overline{y}\cdot(2\theta-1)/(1-\theta)]}$
$\displaystyle=$ $\displaystyle
N(\theta)\bigg{[}1+\overline{x}\cdot\frac{2\theta-1}{1-\theta}\bigg{]},$

where

$\displaystyle N(\theta)^{-1}$ $\displaystyle\coloneqq$
$\displaystyle\sum_{y_{1}^{n}}\bigg{[}1+\overline{y}\cdot\frac{2\theta-1}{1-\theta}\bigg{]}$
$\displaystyle=$ $\displaystyle
2^{n}+\left[\frac{(2\theta-1)}{(1-\theta)}\cdot\frac{1}{n}\sum_{i=1}^{n}i\cdot\binom{n}{i}\right]$
$\displaystyle=$ $\displaystyle
2^{n}+\bigg{[}\frac{(2\theta-1)}{n(1-\theta)}\cdot\big{(}n2^{n-1}\big{)}\bigg{]}$
$\displaystyle=$ $\displaystyle\frac{2^{n-1}}{1-\theta}.$

Also

$\displaystyle
p_{\theta}^{*}\Bigg{(}X_{1}^{n}=x_{1}^{n}\Big{|}T=\sum_{i=1}^{n}x_{i}\Bigg{)}$
$\displaystyle=$
$\displaystyle\dfrac{1+\overline{x}\cdot{(2\theta-1)}/{(1-\theta)}}{\sum\limits_{y_{1}^{n}\in
A_{\sum x_{i}}}[1+\overline{y}\cdot(2\theta-1)/(1-\theta)]}$ $\displaystyle=$
$\displaystyle\dfrac{1}{\big{|}A_{\sum x_{i}}\big{|}}$ $\displaystyle=$
$\displaystyle\frac{1}{\binom{n}{\sum x_{i}}}.$

Let $\widehat{\theta}=x_{1}$. Then

$\displaystyle\tau^{*}(\theta)$ $\displaystyle=$
$\displaystyle\mathbb{E}_{\theta}^{*}[\widehat{\theta}]$ $\displaystyle=$
$\displaystyle\sum\limits_{x_{1}^{n}}x_{1}\cdot p_{\theta}^{*}(x_{1}^{n})$
$\displaystyle=$
$\displaystyle\sum\limits_{x_{1}}N(\theta)\Bigg{[}2^{n-1}+\frac{2\theta-1}{n(1-\theta)}\cdot\sum_{i=1}^{n}(x_{1}+i)\binom{n-1}{i}\Bigg{]}$
$\displaystyle=$ $\displaystyle N(\theta)\cdot
2^{n-1}\bigg{[}1+\frac{2\theta-1}{n(1-\theta)}\cdot\bigg{(}1+\frac{n+1}{2}\bigg{)}\bigg{]}$
$\displaystyle=$ $\displaystyle\frac{\theta}{n}+\frac{n-1}{2n}.$

The variance of $\widehat{\theta}$ with respect to $p_{\theta}^{*}$ is given
by

$\displaystyle\text{Var}_{\theta}^{*}[\widehat{\theta}]$ $\displaystyle=$
$\displaystyle\mathbb{E}_{\theta}^{*}\big{[}X_{1}^{2}\big{]}-\big{(}\mathbb{E}_{\theta}^{*}[X_{1}]\big{)}^{2}=\tau^{*}(\theta)-\tau^{*}(\theta)^{2}.$

Let us consider the generalized Rao-Blackwell estimator of $\tau^{*}(\theta)$.
Recall that $\sum_{i=1}^{n}x_{i}$ is a sufficient statistic for $\theta$ with
respect to Jones et al. likelihood function.

$\displaystyle\phi^{*}\big{(}T(x_{1}^{n})\big{)}$ $\displaystyle=$
$\displaystyle\phi^{*}\left(\sum_{i=1}^{n}x_{i}\right)$ $\displaystyle=$
$\displaystyle\sum\limits_{y_{1}^{n}}\widehat{\theta}(y_{1}^{n})\cdot~{}p_{\theta}^{*}\left(y_{1}^{n}~{}\Big{|}~{}T=\sum_{i=1}^{n}x_{i}\right)$
$\displaystyle=$ $\displaystyle\sum\limits_{y_{1}^{n}\in A_{\sum
x_{i}}}y_{1}\cdot~{}p_{\theta}^{*}\Bigg{(}y_{1}^{n}~{}\Big{|}~{}T=\sum_{i=1}^{n}x_{i}\Bigg{)}$
$\displaystyle=$ $\displaystyle{\binom{n}{\sum
x_{i}}}^{-1}\cdot\sum\limits_{y_{1}^{n}\in A_{\sum x_{i}}}y_{1}$
$\displaystyle=$ $\displaystyle{\binom{n}{\sum
x_{i}}}^{-1}\cdot\sum\limits_{y_{2}^{n}\in A_{(\sum_{i=1}^{n}x_{i}-1)}}1$
$\displaystyle=$ $\displaystyle\binom{n-1}{\sum x_{i}-1}\Bigg{/}\binom{n}{\sum
x_{i}}$ $\displaystyle=$ $\displaystyle\frac{1}{n}\sum_{i=1}^{n}x_{i}.$

Thus $\phi^{*}(T(x_{1}^{n}))=\frac{1}{n}\sum\limits_{i=1}^{n}x_{i}$ is
uniformly better estimator for $\tau^{*}(\theta)$ than $x_{1}$. It can also be
shown that

$\text{Var}_{\theta}^{*}[\phi^{*}(T)]=\frac{\Big{[}\frac{\partial}{\partial\theta}\tau^{*}(\theta)\Big{]}^{2}}{I_{\alpha,n}^{*}(\theta)},$

where $I_{\alpha,n}^{*}(\theta)$ is the inverse of the asymptotic variance of
Jones et al. estimator for $p_{\theta}^{*}(X_{1}^{n})$ [29]. This shows that
the Rao-Blackwell estimator derived from a sufficient statistic attains the
asymptotic variance of Jones et al. estimator. In the following section, we
elucidate this for a general $\mathbb{M}^{(\alpha)}$-family.

##  V Efficiency of Rao-Blackwell estimators of power-law family

In this section, we shall show the efficiency of Rao-Blackwell estimators for
a power-law family. For simplicity, we shall consider a regular
$\mathbb{M}^{(\alpha)}$-family with $h\equiv 1$. That is, for
$x\in\mathbb{S}$,

$p_{\theta}(x)=Z(\theta)[1+w(\theta)f(x)]^{\frac{1}{\alpha-1}}.$ (47)

Then

$\displaystyle{\left(\frac{1}{n}\sum\limits_{i=1}^{n}p_{\theta}(x_{i})^{\alpha-1}\right)^{\frac{1}{\alpha-1}}}$
$\displaystyle=$
$\displaystyle\left(\frac{1}{n}\sum\limits_{i=1}^{n}Z(\theta)^{\alpha-1}[1+w(\theta)f(x_{i})]\right)^{\frac{1}{\alpha-1}}$
$\displaystyle=$ $\displaystyle
Z(\theta)[1+w(\theta)\overline{f}]^{\frac{1}{\alpha-1}}.$

Thus, using (43), we get

$\displaystyle p_{\theta}^{*}(x_{1}^{n})$ $\displaystyle=$
$\displaystyle\frac{[1+w(\theta)\overline{f}]^{\frac{1}{\alpha-1}}}{\int\big{[}1+w(\theta)\bar{f}\big{]}^{\frac{1}{\alpha-1}}dx_{1}^{n}}=N(\theta)[1+w(\theta)\overline{f}]^{\frac{1}{\alpha-1}},$
(48)

where $\bar{f}\coloneqq\frac{1}{n}\sum_{i=1}^{n}f(x_{i})$ and
$N(\theta)^{-1}\coloneqq\int[1+w(\theta)\bar{f}]^{\frac{1}{\alpha-1}}dx_{1}^{n}$
is the normalizing constant. Comparing this with (18), we see that (48) also
forms an $\mathbb{M}^{(\alpha)}$-family with $h\equiv 1$ and
$f(x_{1}^{n})=\bar{f}$. Recall that, $\bar{f}$ is a sufficient statistic for
$\mathbb{M}^{(\alpha)}$ with respect to Jones et al. likelihood function
(Theorem 8). Let $\tau^{*}(\theta)\coloneqq\mathbb{E}_{\theta}^{*}[\bar{f}]$.
In the following, we show that $\bar{f}$ is the best estimator of
$\tau^{*}(\theta)$. That is the variance of any unbiased estimator of
$\tau^{*}(\theta)$ is bounded below by the variance of $\bar{f}$ with respect
to $p_{\theta}^{*}$.

Let $\widehat{\theta}$ be an unbiased estimator of $\tau^{*}(\theta)$ with
respect to $p_{\theta}^{*}$. Then by Theorem 18,
$\phi^{*}(\bar{f})=\mathbb{E}_{\theta}^{*}[\widehat{\theta}|\bar{f}]$ is also
an unbiased estimator of $\tau^{*}(\theta)$ and

$\text{Var}^{*}_{\theta}[\widehat{\theta}]\geq\text{Var}^{*}_{\theta}[\phi^{*}(\bar{f})].$
(49)

Let us consider $\psi^{*}(\bar{f})\coloneqq\phi^{*}(\bar{f})-\bar{f}$. Then
$\mathbb{E}_{\theta}^{*}[\psi^{*}(\bar{f})]=\tau^{*}(\theta)-\tau^{*}(\theta)=0.$
Also

$\displaystyle{\text{Var}_{\theta}^{*}[\phi^{*}(\bar{f})]}$ $\displaystyle=$
$\displaystyle\text{Var}_{\theta}^{*}[\psi^{*}(\bar{f})+\bar{f}]$ (50)
$\displaystyle=$
$\displaystyle\text{Var}_{\theta}^{*}[\psi^{*}(\bar{f})]+\text{Var}_{\theta}^{*}[\bar{f}]+2~{}\text{Cov}_{\theta}^{*}[\psi^{*}(\bar{f}),\bar{f}].~{}~{}~{}~{}~{}$

In Sec-D.1 of Appendix, we have shown that, if $w(\theta)>0$ for
$\theta\in\Theta$, then
$\text{Cov}_{\theta}^{*}[\psi^{*}(\bar{f}),\bar{f}]\geq 0$. Using this in
(50), we get

$\displaystyle{\text{Var}_{\theta}^{*}[\phi^{*}(\bar{f})]}$ $\displaystyle=$
$\displaystyle\text{Var}_{\theta}^{*}[\psi^{*}(\bar{f})]+\text{Var}_{\theta}^{*}[\bar{f}]+2~{}\text{Cov}_{\theta}^{*}[\psi^{*}(\bar{f}),\bar{f}]$
(51) $\displaystyle\geq$ $\displaystyle\text{Var}_{\theta}^{*}[\bar{f}].$

Using (49) and (51), we get

$\text{Var}^{*}_{\theta}[\widehat{\theta}]\geq\text{Var}_{\theta}^{*}[\bar{f}],$
(52)

where equality holds if and only if $\widehat{\theta}=\bar{f}$ almost surely.
Thus we have the following.

###### Theorem 20

Consider an $\mathbb{M}^{(\alpha)}$-family as in (47) with $w(\theta)>0$ for
$\theta\in\Theta\subset\mathbb{R}$. Let
$\tau^{*}(\theta)=\mathbb{E}^{*}_{\theta}[\bar{f}]$. Then the variance of any
unbiased estimator of $\tau^{*}(\theta)$ with respect to $p_{\theta}^{*}$ is
not lower than that of $\bar{f}$. That is, $\bar{f}$ is the best estimator of
its expected value with respect to $p_{\theta}^{*}$.

We now show that $\text{Var}_{\theta}^{*}[\bar{f}]$, indeed, equals to the
asymptotic variance of Jones et al. estimator for the model $p_{\theta}^{*}$.
The Jones et al. asymptotic variance for $p_{\theta}^{*}$ is given by (c.f.
[29])

$\displaystyle\frac{1}{I_{\alpha,n}^{*}(\theta)}\coloneqq\frac{\text{Var}_{\theta}^{*}[p_{\theta}^{*}(X_{1}^{n})^{\alpha-1}s^{*}(X_{1}^{n},\theta)]}{\text{Cov}_{\theta}^{*}[s^{*}(X_{1}^{n},\theta),p_{\theta}^{*}(X_{1}^{n})^{\alpha-1}s^{*}(X_{1}^{n},\theta)]^{2}},$
(53)

where

$s^{*}(x_{1}^{n},\theta)\coloneqq\frac{\partial}{\partial\theta}\log
p_{\theta}^{*}(x_{1}^{n}).$ (54)

It can be easily seen that $I_{\alpha,n}^{*}(\theta)$ coincides with the usual
Fisher information $I_{n}^{*}(\theta)$ for $\alpha=1$, where
$I_{n}^{*}(\theta)=\text{Var}_{\theta}^{*}[\frac{\partial}{\partial\theta}\log
p_{\theta}^{*}(X_{1}^{n})]$. Observe that (48) can be re-written as

$\displaystyle p_{\theta}^{*}(x_{1}^{n})$ $\displaystyle=$ $\displaystyle
N(\theta)[1+w(\theta)\overline{f}]^{\frac{1}{\alpha-1}}=\big{[}N(\theta)^{\alpha-1}+N(\theta)^{\alpha-1}w(\theta)\overline{f}\big{]}^{\frac{1}{\alpha-1}}.$

For convenience, let $F(\theta)\coloneqq N(\theta)^{\alpha-1}$ and
$\widetilde{w}(\theta)\coloneqq N(\theta)^{\alpha-1}w(\theta)$. Then

$p_{\theta}^{*}(x_{1}^{n})=[F(\theta)+\widetilde{w}(\theta)\overline{f}]^{\frac{1}{\alpha-1}}$

and hence

$\frac{\partial}{\partial\theta}p_{\theta}^{*}(x_{1}^{n})=\frac{1}{\alpha-1}p_{\theta}^{*}(x_{1}^{n})^{2-\alpha}[F^{\prime}(\theta)+\widetilde{w}^{\prime}(\theta)\overline{f}].$

Then

$\displaystyle{\text{Var}_{\theta}^{*}\big{[}p_{\theta}^{*}(X_{1}^{n})^{\alpha-1}s^{*}(X_{1}^{n},\theta)\big{]}}$
$\displaystyle=$
$\displaystyle\left(\frac{1}{\alpha-1}\right)^{2}\big{[}\widetilde{w}^{\prime}(\theta)\big{]}^{2}\text{Var}^{*}_{\theta}[\bar{f}],$
(55)

and

$\displaystyle\text{Cov}^{*}_{\theta}[s^{*}(X_{1}^{n},\theta),p_{\theta}^{*}(X_{1}^{n})^{\alpha-1}s^{*}(X_{1}^{n},\theta)]$
$\displaystyle=$
$\displaystyle\left(\frac{1}{\alpha-1}\right)\widetilde{w}^{\prime}(\theta)\frac{\partial}{\partial\theta}\tau^{*}(\theta).$
(56)

The detailed derivations of (55) and (56) can be found in Sec-D.2 of the
Appendix. Using these values, (53) becomes

$\displaystyle\frac{1}{I^{*}_{\alpha,n}(\theta)}=\frac{\text{Var}^{*}_{\theta}\big{[}\overline{f}\big{]}}{\Big{[}\frac{\partial}{\partial\theta}\mathbb{E}^{*}_{\theta}[\overline{f}]\Big{]}^{2}},$

that is,

$\displaystyle\dfrac{{\Big{[}\frac{\partial}{\partial\theta}\mathbb{E}^{*}_{\theta}[\overline{f}]\Big{]}^{2}}}{I^{*}_{\alpha,n}(\theta)}=\frac{{\Big{[}\frac{\partial}{\partial\theta}\tau^{*}(\theta)\Big{]}^{2}}}{I^{*}_{\alpha,n}(\theta)}={\text{Var}^{*}_{\theta}[\overline{f}]}.$

This, together with (52), yields the following result.

###### Corollary 21

Consider an $\mathbb{M}^{(\alpha)}$-family as in Theorem 20. Let
$\tau^{*}(\theta)=\mathbb{E}^{*}_{\theta}[\bar{f}]$. Let $\widehat{\theta}$ be
any unbiased estimator of $\tau^{*}(\theta)$ with respect to $p_{\theta}^{*}$.
Then

$\text{Var}_{\theta}^{*}[\widehat{\theta}]\geq\frac{{\Big{[}\frac{\partial}{\partial\theta}\tau^{*}(\theta)\Big{]}^{2}}}{I^{*}_{\alpha,n}(\theta)}.$
(57)

Equality holds in (57) if and only if $\widehat{\theta}=\bar{f}$ almost
surely.

###### Remark 6

  * (i)

Observe that

$\displaystyle{\text{Cov}_{\theta}^{*}[s^{*}(X_{1}^{n},\theta),p_{\theta}^{*}(X_{1}^{n})^{\alpha-1}s^{*}(X_{1}^{n},\theta)]^{2}}$
$\displaystyle\leq$
$\displaystyle\text{Var}_{\theta}^{*}[p_{\theta}^{*}(X_{1}^{n})^{\alpha-1}s^{*}(X_{1}^{n},\theta)]\text{Var}_{\theta}^{*}[s^{*}(X_{1}^{n},\theta)],$

by Cauchy-Schwarz inequality. This implies that

$\dfrac{1}{I_{n}^{*}(\theta)}\leq\dfrac{1}{I^{*}_{\alpha,n}(\theta)}.$

That means the bound given by Corollary 21 is sharper than the usual CRLB for
any unbiased estimator of $\tau^{*}(\theta)$. Observe that Jones et al.
estimators are, in general, more robust than MLE but less efficient. The
generalized bound in Corollary 21 might help in investigating the efficacies
of such robust estimators.

  * (ii)

In view of (12) and Remark 2(i), we see that (57) in Corollary 21 is
essentially a generalization of the usual Cramér-Rao inequality for an
exponential family.

###### Example 3

Consider the Student distributions with location parameter $\mu=0$, scale
parameter $\sigma^{2}$ (unknown) and a fixed degrees of freedom parameter
$\nu\in(2,\infty):$

$p_{\sigma}(x)=N_{\nu,\sigma}\bigg{[}1+\frac{x^{2}}{\nu\sigma^{2}}\bigg{]}^{-\frac{\nu+1}{2}},$
(58)

where $N_{\nu,\sigma}$ is the normalizing constant. Let
$\alpha=1-\frac{2}{\nu+1}$, $b_{\alpha}=1/\nu$ and
$N_{\alpha,\sigma}=N_{\nu,\sigma}$. Then (58) can be re-written as a one
parameter regular $\mathbb{M}^{(\alpha)}$-family with $h\equiv
1,w(\sigma)=b_{\alpha}/\sigma^{2}>0$ and $f(x)=x^{2}$ [25]. By Theorem 20,
$\overline{x^{2}}$ is the best estimator for
$\mathbb{E}_{\sigma}^{*}[\overline{X^{2}}]$. We shall now find
$\mathbb{E}_{\sigma}^{*}[\overline{X^{2}}]$ explicitly. Using (48), we have

$\displaystyle
p^{*}_{\sigma}(x_{1}^{n})=Z_{\alpha,\sigma}\Big{[}1+(b_{\alpha}/\sigma^{2})\overline{x^{2}}\Big{]}^{\frac{1}{\alpha-1}},$

where
$Z_{\alpha,\sigma}^{-1}\coloneqq\int[1+(b_{\alpha}/\sigma^{2})\overline{y^{2}}]^{\frac{1}{\alpha-1}}dy_{1}^{n}=\sigma^{n}\cdot
H_{n}{(\alpha)}$, provided $\alpha>1-\frac{2}{n}$. Here,

  * if $n=2k+1$, $k\geq 0$,

$H_{n}(\alpha)=\frac{(\pi
b_{\alpha})^{\frac{1}{2}}(n\pi/b_{\alpha})^{\frac{n}{2}}\Gamma\big{(}\frac{1+\alpha}{2(1-\alpha)}\big{)}A_{n}(\alpha)^{k}}{\Gamma(n/2)\Gamma(\frac{1}{1-\alpha})},\quad\text{and}$

  * if $n=2k$, $k\geq 1$,

$H_{n}(\alpha)=\frac{(n\pi/b_{\alpha})^{\frac{n}{2}}(1-\alpha)A_{n}(\alpha)^{k-1}}{\alpha\Gamma(n/2)}$

with $A_{n}(\alpha)\coloneqq\frac{(1-\alpha)(n-2)}{2-n(1-\alpha)}$. Let us now
denote $B_{n}(\alpha)=\frac{n(1-\alpha)}{2\alpha-n(1-\alpha)}$. Then
$\mathbb{E}_{\sigma}^{*}[\overline{X^{2}}]$ equals to
$\frac{A_{n}(\alpha)^{2-k}B_{n}(\alpha)^{k-1}}{b_{\alpha}}\sigma^{2}$ when
$n=2k$, and
$\frac{A_{n}(\alpha)^{1-k}B_{n}(\alpha)^{k}}{b_{\alpha}}\sigma^{2}$ when
$n=2k+1$, provided $\alpha>\frac{n}{n+2}$. Hence the generalized CRLB is given
by,

  * when $n=2k+1$,

$[\\{{A_{n}(\alpha)^{2-k}(C_{n}(\alpha)^{k}-A_{n}(\alpha)^{-k}B_{n}(\alpha)^{2k}})\\}\big{/}{b_{\alpha}^{2}}]\sigma^{4},$

  * and when $n=2k$,

$[\\{{A_{n}(\alpha)^{3-k}(C_{n}(\alpha)^{k-1}-A_{n}(\alpha)^{1-k}B_{n}(\alpha)^{(2k-1)}})\\}\big{/}{b_{\alpha}^{2}}]\sigma^{4}$

where $C_{n}(\alpha)\coloneqq\frac{(1-\alpha)(n+2)}{2\alpha+(n+2)(\alpha-1)}$
and $\alpha>\frac{n+2}{n+4}$.

###### Remark 7

Observe that Theorem 20 can not be applied, for example, to Student
distributions with unknown location parameter (scale parameter known), since
they do not form a regular $\mathbb{M}^{(\alpha)}$-family [25, Rem. 2(c)].
However, in the following, we address this problem by deriving a Rao-Blackwell
type theorem for Basu et al. likelihood function (10). Note that, for this
model, Basu et al. estimator and Jones et al. estimator are the same [23, 25].

###  V-A Analogous results for Basu et al. likelihood function

Consider the Basu et al. likelihood function $L^{(\alpha)}_{B}$ as in (10).
The associated deformed probability distribution is given by

$\displaystyle p_{\theta}^{*}(x_{1}^{n})$ $\displaystyle\coloneqq$
$\displaystyle\frac{\exp[L^{(\alpha)}_{B}(x_{1}^{n};\theta)]}{\int\exp[L^{(\alpha)}_{B}(y_{1}^{n};\theta)]d{y_{1}^{n}}}$
(59) $\displaystyle=$
$\displaystyle\frac{\exp[\frac{\alpha}{n(\alpha-1)}\sum_{i=1}^{n}p_{\theta}(x_{i})^{\alpha-1}]}{\int\exp[\frac{\alpha}{n(\alpha-1)}\sum_{i=1}^{n}p_{\theta}(y_{i})^{\alpha-1}]dy_{1}^{n}}.~{}~{}~{}~{}~{}$

Recall that, $\mathbb{B}^{(\alpha)}$-families are the ones that have a fixed
number of sufficient statistics with respect to Basu et al. likelihood
function (Theorem 10). Consider a $\mathbb{B}^{(\alpha)}$-family as in (39).
Then, using (59), we have

$\displaystyle
p_{\theta}^{*}(x_{1}^{n})=\exp\Big{[}\frac{\alpha}{\alpha-1}\overline{h}+\widetilde{F}(\theta)+\frac{\alpha}{\alpha-1}w(\theta)\overline{f}\Big{]},$
(60)

where
$\widetilde{F}(\theta)=-\log\int\exp\big{[}\frac{\alpha}{\alpha-1}(\overline{h}+w(\theta)\overline{f})\big{]}dx_{1}^{n}$,
is the normalizing constant. Observe that $p_{\theta}^{*}$ forms a one
parameter exponential family. Then we can derive the following result
concerning the best estimator for the parameter, analogous to Theorem 20 and
Corollary 21.

###### Theorem 22

Consider a regular $\mathbb{B}^{(\alpha)}$-family as in (39). Let
$\mathbb{E}^{*}_{\theta}[\bar{f}]=\tau(\theta)$. Then $\bar{f}$ is the best
estimator of its expected value with respect to $p_{\theta}^{*}$. Furthermore,
if $\widehat{\theta}$ is any estimator such that
$\mathbb{E}^{*}_{\theta}[\widehat{\theta}]=\tau(\theta)$, then

$\text{Var}_{\theta}^{*}[\widehat{\theta}]\geq\frac{[\tau^{\prime}(\theta)]^{2}}{I_{n}^{*}(\theta)}.$
(61)

###### Proof:

See Sec-D.3 of Appendix. ∎

###### Remark 8

Notice that the bound in (61) is given by the usual Fisher information (of
course, applied to the model $p_{\theta}^{*}$) unlike the one in Corollary 21
where the bound is given by the asymptotic variance of Jones et al.
estimators.

We now apply the above result to find best estimator for the mean parameter of
a Student distribution.

###### Example 4

Consider the Student distributions with degrees of freedom parameter
$\nu\in(2,\infty)$, location parameter $\mu$ and scale parameter
$\sigma^{2}=1:$

$p_{\mu}(x)=N_{\nu}\Big{[}1+\frac{1}{\nu}(x-\mu)^{2}\Big{]}^{-\frac{\nu+1}{2}},$
(62)

where $N_{\nu}$ is the normalizing constant. Letting
$\alpha=1-\frac{2}{\nu+1}$, $b_{\alpha}=1/\nu$ and $N_{\alpha}=N_{\nu}$, (62)
forms a regular $\mathbb{B}^{(\alpha)}$-family as in (39) with
$h(x)=N_{\alpha}^{\alpha-1}b_{\alpha}x^{2}$,
$F(\mu)=(1+b_{\alpha}\mu^{2})N_{\alpha}^{\alpha-1}$, $w(\mu)=-2\mu
N_{\alpha}^{\alpha-1}b_{\alpha}$ and $f(x)=x$. This implies that
$\overline{x}$ is a sufficient statistic for $\mu$ with respect to the Basu et
al. likelihood function [24, Th. 4] and $\overline{x}$ is the best estimator
for $\mathbb{E}^{*}_{\mu}[\overline{X}]$ (Theorem 22).

It can also be shown, in this case, that

$\displaystyle{p_{\mu}^{*}(x_{1}^{n})}=\exp\Big{[}\frac{\alpha}{\alpha-1}N_{\alpha}^{\alpha-1}b_{\alpha}\overline{x^{2}}+F_{\alpha}(\mu)-\frac{2\alpha}{\alpha-1}N_{\alpha}^{\alpha-1}b_{\alpha}\mu\overline{x}\Big{]},$

where

$F_{\alpha}(\mu)\coloneqq\frac{\alpha}{\alpha-1}N_{\alpha}^{\alpha-1}b_{\alpha}\mu^{2}-\frac{n}{2}\log\Big{[}\pi-\frac{\alpha}{n(\alpha-1)}N_{\alpha}^{\alpha-1}b_{\alpha}\Big{]},$
$\displaystyle\mathbb{E}^{*}_{\mu}[\overline{X}]$ $\displaystyle=$
$\displaystyle-\Big{(}\frac{\alpha-1}{\alpha}\Big{)}\frac{\frac{2\alpha}{(1-\alpha)}N_{\alpha}^{\alpha-1}b_{\alpha}\mu}{w^{\prime}(\mu)}=\mu,\quad\text{and}$
$\displaystyle\text{Var}_{\mu}^{*}[\overline{X}]$ $\displaystyle=$
$\displaystyle\frac{[\tau^{\prime}(\mu)]^{2}}{I_{n}^{*}(\mu)}=\frac{(1-\alpha)}{2\alpha
b_{\alpha}N_{\alpha}^{\alpha-1}}.$

##  VI Summary and concluding remarks

Fisher introduced the concept of sufficiency in connection with parameter
estimation problems in an attempt to estimate the unknown parameters of a
population without knowing the entire sample [19]. Fisher’s notion of
sufficiency is based on the usual log-likelihood function and the underlying
estimation is ML estimation. However, there are situations (for example,
robust inference) where we need to look beyond ML estimation. Hence the
classical notion of sufficiency is inadequate in such situations. In this
paper, we presented a notion of sufficiency based on a general likelihood
function. In particular, we considered two families of likelihood functions,
namely Basu et al. and Jones et al. $\alpha$-likelihood functions. These (for
$\alpha>1$) arise in distance-based robust inference methods and are
generalizations of the usual log-likelihood function. Also, these likelihood
functions are special as they do not require smoothing of empirical measure
(of the observed sample) [29]. We characterized the family of probability
distributions that have a fixed number of sufficient statistics (independent
of sample size) with respect to these likelihood functions. This family
generalizes the usual exponential family and has a power law. Student
distributions fall in this family. We also extended the notion of minimal
sufficiency and found minimal sufficient statistics for the power-law
families. This could potentially help when the data is modeled using one of
these power-law distributions. However, we would like to emphasize that the
established results have the limitation (when it comes to practical
applications) that, the tuning parameter $\alpha$ in Basu et al. or Jones et
al. likelihood function should be related to the degrees of freedom parameter
$\nu$ of the Student distribution by $\alpha={(\nu-1)}/{(\nu+1)}$. Basu et al.
and Jones et al. estimations for exponential families, in particular, for
normal distributions have been studied, for example, in [3, 29, 6]. However,
unlike these works, the present work focused on models that have closed-form
solutions for these estimations.

Another important application of sufficient statistics is that, given an
estimator, one can find another estimator that is uniformly better than the
given one. This result is known as the Rao-Blackwell theorem. We generalized
Rao-Blackwell theorem when one has sufficient statistics with respect to Basu
et al. or Jones et al. likelihood function. This helps us find the “best”
estimator for the parameters of a power-law family and hence for Student
distributions. Consequently, we derived a lower bound for the variance of
estimators of a deformed form of power-law family, $p_{\theta}^{*}$. In case
of Jones et al. likelihood function, the bound turns out to be its asymptotic
variance (with respect to $p_{\theta}^{*}$). The derivation of asymptotic
variance of Jones et al. estimator can be found in [29]. This bound is sharper
than the usual CRLB (See Corollary 21). Recall that the usual CRLB equals the
asymptotic variance of MLE.

We now make some general remarks on the deformed probability distribution
$p_{\theta}^{*}$. Recall that, it is the factorization theorem (Halmos and
Savage [26] in the classical case and Proposition 2 in the general case) that
makes it clear that the notion of sufficiency is with respect to a likelihood
function. However, the generalized notion of sufficiency is defined using the
deformed probability distribution $p_{\theta}^{*}$. The equivalence of
analogous factorization theorem and Fisher’s definition of sufficiency shows
the uniqueness of the usage of $p_{\theta}^{*}$ in the generalized notion of
sufficiency. This fact was further evidenced in the case of Jones et al.
likelihood function, where the lower bound of variance of estimators for
power-law families equals the asymptotic variance of the Jones et al.
estimator for $p_{\theta}^{*}$. However, for Basu et al. likelihood function,
the bound is given by the usual CRLB, though computed with respect to
$p_{\theta}^{*}$, does not equal the asymptotic variance of the Basu et al.
estimator (Theorem 22). This is surprising as one would expect the bound to
equal the Basu et al. asymptotic variance given by [3, Th. 2] (as in the case
of Jones et al. likelihood function). This will be the focus of one of our
forthcoming works. In any case, such an anomalous behavior of Basu et al.
likelihood function (from Jones et al. likelihood function) was also noticed
in the study of the geometry of the associated divergence functions in
connection with projection theorems for families of probability distributions
determined by linear constraints. While the geometry of the Basu et al.
divergence is similar to that of the Kullback-Leibler divergence, the same for
Jones et al. divergence was different [33].

## A. Proof of results of Section 2

  * A.1.

###### Proof:

Let $T$ be a sufficient statistic for $\theta$ with respect to $L_{G}$. Let
$x_{1}^{n}$ and $y_{1}^{n}$ be two samples such that
$T(x_{1}^{n})=T(y_{1}^{n})$. Then from generalized Koopman’s definition of
sufficiency, we have $[L_{G}(x_{1}^{n};\theta)-L_{G}(y_{1}^{n};\theta)]$ is
independent of $\theta$. Let us define a relation ‘$\sim$’ on the set of all
sample points of length $n$ by $x_{1}^{n}\sim y_{1}^{n}$ if and only if
$T(x_{1}^{n})=T(y_{1}^{n})$. Then ‘$\sim$’ is an equivalence relation. Let us
denote the equivalence classes by $\mathbb{S}^{t}$ for each
$t\in\mathscr{T}=\\{r:r=T(x_{1}^{n})~{}\text{for
some}~{}x_{1}^{n}\in\mathbb{S}^{n}\\}$. For each equivalence class
$\mathbb{S}^{t}$, designate an element ${x}_{1,t}^{n}\in\mathbb{S}^{t}$.

Let $x_{1}^{n}$ be a sample point such that
$T(x_{1}^{n})=t^{*}\in\mathscr{T}$. Then $x_{1}^{n}\in\mathbb{S}^{t^{*}}$ and
$T(x_{1}^{n})=T({x}_{1,t^{*}}^{n})$. This implies that
$[L_{G}(x_{1}^{n};\theta)-L_{G}({x}_{1,t^{*}}^{n};\theta)]$ is independent of
$\theta$ by hypothesis. Let
$v(x_{1}^{n}):=[L_{G}(x_{1}^{n};\theta)-L_{G}({x}_{1,t^{*}}^{n};\theta)]$.
Then

$\displaystyle
L_{G}(x_{1}^{n};\theta)=L_{G}({x}_{1,t^{*}}^{n};\theta)+v(x_{1}^{n})=u(\theta,t^{*})+v(x_{1}^{n}),$

where $u(\theta,t):=L_{G}({x}_{1,t}^{n};\theta)$ for each $t\in\mathscr{T}$.

Conversely, let us assume that there exist two functions $u$ and $v$ such that

$L_{G}(x_{1}^{n};\theta)=u(\theta,T(x_{1}^{n}))+v(x_{1}^{n})$

for all sample points $x_{1}^{n}$ and for all $\theta\in\Theta$. Thus for any
two sample points $x_{1}^{n}$ and $y_{1}^{n}$ with
$T(x_{1}^{n})=T(y_{1}^{n})$, we have

$\displaystyle{L_{G}(x_{1}^{n};\theta)-L_{G}(y_{1}^{n};\theta)}$
$\displaystyle=u(\theta,T(x_{1}^{n}))+v(x_{1}^{n})-u(\theta,T(y_{1}^{n}))-v(y_{1}^{n})$
$\displaystyle=v(x_{1}^{n})-v(y_{1}^{n}).$

This implies that $[L_{G}(x_{1}^{n};\theta)-L_{G}(y_{1}^{n};\theta)]$ is
independent of $\theta$. By generalized Koopman’s definition of sufficiency,
$T$ is a sufficient statistic for $\theta$ with respect to $L_{G}$. This
completes the proof. ∎

  * A.2.

###### Proof:

Let $T$ be sufficient for $\theta$ with respect to $L_{G}$. Then, from
Proposition 2, there exist functions $u$ and $v$ such that

$L_{G}(x_{1}^{n};\theta)=u(T(x_{1}^{n}),\theta)+v(x_{1}^{n})$ (63)

for all $x_{1}^{n}\in\mathbb{S}^{n}$. We need to only show
$p_{\theta{{}_{X_{1}^{n}|T}}}^{*}(x_{1}^{n}|t)$ is independent of $\theta$
when $t=T(x_{1}^{n})$. Observe that if $t\neq T(x_{1}^{n})$,
$p_{\theta{{}_{X_{1}^{n}|T}}}^{*}(x_{1}^{n}|t)=0$ which is independent of
$\theta$. Using (63), we have

$\displaystyle{p_{\theta{{}_{X_{1}^{n}|T}}}^{*}(x_{1}^{n}|t)}$
$\displaystyle=$
$\displaystyle\dfrac{p_{\theta}^{*}(x_{1}^{n})}{\int\limits_{A_{T(x_{1}^{n})}}p^{*}_{\theta}(y_{1}^{n})d{y_{1}^{n}}}$
$\displaystyle=$
$\displaystyle\dfrac{\exp[L_{G}(x_{1}^{n};\theta)]}{\int\limits_{A_{T(x_{1}^{n})}}\exp[L_{G}(y_{1}^{n};\theta)]dy_{1}^{n}}$
$\displaystyle=$
$\displaystyle\dfrac{\exp[u(T(x_{1}^{n}),\theta)]\exp[v(x_{1}^{n})]}{\int\limits_{A_{T(x_{1}^{n})}}\exp[u(T(x_{1}^{n}),\theta)]\exp[v(y_{1}^{n})]dy_{1}^{n}}$
$\displaystyle=$
$\displaystyle\dfrac{\exp[v(x_{1}^{n})]}{\int\limits_{A_{T(x_{1}^{n})}}\exp[v(y_{1}^{n})]dy_{1}^{n}},$

which is independent of $\theta$, where the first equality is by the
definition of the generalized conditional distribution.

Conversely, let $p_{\theta{{}_{X_{1}^{n}|T}}}^{*}(x_{1}^{n}|t)$ be independent
of $\theta$. Let $p_{\theta{{}_{X_{1}^{n}|T}}}^{*}(x_{1}^{n}|t)\coloneqq
v_{1}(x_{1}^{n})$. Then

$\displaystyle\dfrac{\exp[L_{G}(x_{1}^{n};\theta)]}{\int\limits_{A_{T(x_{1}^{n})}}\exp[L_{G}(y_{1}^{n};\theta)]dy_{1}^{n}}=v_{1}(x_{1}^{n}).$

This implies

$\displaystyle L_{G}(x_{1}^{n};\theta)=u(T(x_{1}^{n}),\theta)+v(x_{1}^{n}),$
(64)

where
$u(T(x_{1}^{n}),\theta)=\log\int\limits_{A_{T(x_{1}^{n})}}\exp[L_{G}(y_{1}^{n};\theta)]dy_{1}^{n}$
and $v(x_{1}^{n})=\log v_{1}(x_{1}^{n})$. Since the above is true for any
$x_{1}^{n}\in\mathbb{S}^{n}$, we have the result. ∎

  * A.3.

###### Proof:

The condition that if $T(x_{1}^{n})=T(y_{1}^{n})$ then
$[L_{G}(x_{1}^{n};\theta)-L_{G}(y_{1}^{n};\theta)]$ is independent of
$\theta$, for all $\theta\in\Theta$ implies that $T$ is a sufficient statistic
for $\theta$ with respect to $L_{G}$, by Definition 1. Thus the only thing we
need to prove is that $T$ is further minimal. Let $\widetilde{T}$ be a
sufficient statistic such that
$\widetilde{T}({x_{1}^{n}})=\widetilde{T}(y_{1}^{n})$. Then by Definition 1,
$[L_{G}(x_{1}^{n};\theta)-L_{G}(y_{1}^{n};\theta)]$ is independent of
$\theta$. Hence by hypothesis, we have $T(x_{1}^{n})=T(y_{1}^{n})$. Therefore
$T$ is minimal, by definition. ∎

## B. Proof of results of Section 3

  * B.1.

###### Proof:

Let $\Pi=\mathbb{M}^{(\alpha)}$ as in Definition 6. Let $x_{1}^{n}$ and
$y_{1}^{n}$ be two i.i.d. samples from some
$p_{\theta}\in\mathbb{M}^{(\alpha)}$ such that $T(x_{1}^{n})=T(y_{1}^{n})$.
Assume that, for $i=1,\dots,s$,

$\displaystyle{\overline{f_{i}(x_{1}^{n})}/\overline{h(x_{1}^{n})}=\psi_{i}(T(x_{1}^{n})),~{}\text{and}~{}}\overline{f_{i}(y_{1}^{n})}/\overline{h(y_{1}^{n})}=\psi_{i}(T(y_{1}^{n})).$

Then

$\displaystyle
L_{J}^{(\alpha)}(x_{1}^{n};\theta)-L_{J}^{(\alpha)}(y_{1}^{n};\theta)$
$\displaystyle=\frac{1}{\alpha-1}\log\Big{[}\frac{1}{n}\sum\limits_{i=1}^{n}p_{\theta}(x_{i})^{\alpha-1}\Big{]}-\frac{1}{\alpha-1}\log\Big{[}\frac{1}{n}\sum\limits_{i=1}^{n}p_{\theta}(y_{i})^{\alpha-1}\Big{]}$
$\displaystyle=\frac{1}{\alpha-1}\log\frac{Z(\theta)^{\alpha-1}\big{\\{}\overline{h(x_{1}^{n})}+w(\theta)^{T}\overline{f(x_{1}^{n})}\big{\\}}}{Z(\theta)^{\alpha-1}\big{\\{}\overline{h(y_{1}^{n})}+w(\theta)^{T}\overline{f(y_{1}^{n})}\big{\\}}}$
$\displaystyle=\frac{1}{\alpha-1}\log\frac{\overline{h(x_{1}^{n})}}{\overline{h(y_{1}^{n})}}+\frac{1}{\alpha-1}\log\frac{1+w(\theta)^{T}\big{\\{}\overline{f(x_{1}^{n})}\big{/}\overline{h(x_{1}^{n})}\big{\\}}}{1+w(\theta)^{T}\big{\\{}\overline{f(y_{1}^{n})}\big{/}\overline{h(y_{1}^{n})}\big{\\}}}$
$\displaystyle=\frac{1}{\alpha-1}\log\big{[}{\overline{h(x_{1}^{n})}}\big{/}{\overline{h(y_{1}^{n})}}\big{]},$

which is independent of $\theta$. The last equality holds since
$T(x_{1}^{n})=T(y_{1}^{n})$ and thus
$\overline{f_{i}(x_{1}^{n})}/\overline{h(x_{1}^{n})}=\overline{f_{i}(y_{1}^{n})}/\overline{h(y_{1}^{n})}$.
Hence $T$ is a sufficient statistic for $\theta$ with respect to the Jones et
al. likelihood function $L_{J}^{(\alpha)}$. ∎

  * B.2.

###### Proof:

We only give the proof for $\mathbb{M}^{(\alpha)}$-family. The result for
$\mathbb{B}^{(\alpha)}$-family can be proved similarly.

Let us consider an $\mathbb{M}^{(\alpha)}$-family as in Definition 6. Assume
that the family is regular. We will only show that the statistic

$\displaystyle\overline{f(x_{1}^{n})}\big{/}\overline{h(x_{1}^{n})}=\big{(}\overline{f_{1}(x_{1}^{n})}\big{/}\overline{h(x_{1}^{n})},\dots,\overline{f_{k}(x_{1}^{n})}\big{/}\overline{h(x_{1}^{n})}\big{)}$

is minimal. Let us consider two sample points $x_{1}^{n}$ and $y_{1}^{n}$ such
that $[L_{J}^{(\alpha)}(x_{1}^{n};\theta)-L_{J}^{(\alpha)}(y_{1}^{n};\theta)]$
is independent of $\theta$. Our aim is to prove that
$\overline{f(x_{1}^{n})}\big{/}\overline{h(x_{1}^{n})}=\overline{f(y_{1}^{n})}\big{/}\overline{h(y_{1}^{n})}$.

Consider
$[L_{J}^{(\alpha)}(x_{1}^{n};\theta)-L_{J}^{(\alpha)}(y_{1}^{n};\theta)]$.
Using (B.1.), we have

$\displaystyle{L_{J}^{(\alpha)}(x_{1}^{n};\theta)-L_{J}^{(\alpha)}(y_{1}^{n};\theta)=\frac{1}{\alpha-1}\log\frac{\overline{h(x_{1}^{n})}}{\overline{h(y_{1}^{n})}}}+\frac{1}{\alpha-1}\log\frac{1+w(\theta)^{T}\big{\\{}\overline{f(x_{1}^{n})}\big{/}\overline{h(x_{1}^{n})}\big{\\}}}{1+w(\theta)^{T}\big{\\{}\overline{f(y_{1}^{n})}\big{/}\overline{h(y_{1}^{n})}\big{\\}}}.$

Since the above is independent of $\theta$, we must have

$\log\frac{1+w(\theta)^{T}\big{\\{}\overline{f(x_{1}^{n})}\big{/}\overline{h(x_{1}^{n})}\big{\\}}}{1+w(\theta)^{T}\big{\\{}\overline{f(y_{1}^{n})}\big{/}\overline{h(y_{1}^{n})}\big{\\}}}=C,$

where $C$ is a constant independent of $\theta$ (may depend on $x_{1}^{n}$ and
$y_{1}^{n}$). This implies

$\displaystyle{(1-\exp(C))+w(\theta)^{T}\big{[}\overline{f(x_{1}^{n})}\big{/}\overline{h(x_{1}^{n})}}-\exp(C)\big{\\{}\overline{f(y_{1}^{n})}\big{/}\overline{h(y_{1}^{n})}\big{\\}}\big{]}=0.$

Since the given family is regular, the functions 1, $w_{1},\dots,w_{k}$ are
linearly independent. Hence we have $\exp(C)=1$ and therefore

$\big{[}\overline{f(x_{1}^{n})}\big{/}\overline{h(x_{1}^{n})}-\overline{f(y_{1}^{n})}\big{/}\overline{h(y_{1}^{n})}\big{]}=0.$

That is,

$\overline{f(x_{1}^{n})}\big{/}\overline{h(x_{1}^{n})}=\overline{f(y_{1}^{n})}\big{/}\overline{h(y_{1}^{n})}.$

Thus $\bar{f}/\bar{h}$ is a minimal sufficient statistic for a regular
$\mathbb{M}^{(\alpha)}$-family with respect to $L_{J}^{(\alpha)}$. ∎

## C. Proof of results of Section 4

  * C.1.

###### Proof:

Recall that $p_{\theta}(x)=Z(\theta)[h(x)+w(\theta)f(x)]$ for $x\in\mathbb{S}$
and $\mathbb{E}_{\theta}[\psi(\bar{f})]=0$. That is,

$\int\psi(\bar{f})p_{\theta}(x_{1}^{n})dx_{1}^{n}=0.$

Taking derivative with respect to $\theta$ on both sides, we get

$\int\psi(\bar{f})\frac{\partial}{\partial\theta}\bigg{\\{}\prod\limits_{i=1}^{n}\exp[h(x_{i})+w(\theta)f(x_{i})]\bigg{\\}}dx_{1}^{n}=0.$

This implies

$\int\psi(\bar{f})p_{\theta}(x_{1}^{n})w^{\prime}(\theta)n\bar{f}dx_{1}^{n}=0.$

Since $w^{\prime}(\theta)\neq 0$ for all $\theta\in\Theta$, we have
$\mathbb{E}_{\theta}[\psi(\bar{f})\bar{f}]=0.$ Therefore

$\displaystyle\text{Cov}_{\theta}[\psi(\bar{f}),\bar{f}]=\mathbb{E}_{\theta}[\psi(\bar{f})\bar{f}]-\mathbb{E}_{\theta}[\psi(\bar{f})]\mathbb{E}_{\theta}[\bar{f}]=0.$

∎

  * C.2.

###### Proof:

Since $T$ is a sufficient statistic with respect to the Jones et al.
likelihood function, by Proposition 3, the conditional distribution of
$x_{1}^{n}$ given $T$ with respect to $p_{\theta}^{*}$ is independent of
$\theta$. Hence the result follows as $\widehat{\theta}$ is a function of
$x_{1}^{n}$ only. ∎

  * C.3.

###### Proof:

For every partition set $A_{T(y_{1}^{n})}$ of $\mathbb{S}^{n}$, the space of
all possible $n$-length samples, we have

$\displaystyle{\mathbb{E}_{\theta}^{*}\Big{[}\phi^{*}(T(X_{1}^{n}))\cdot\mathbf{1}_{A_{T(y_{1}^{n})}}\Big{]}}$
$\displaystyle\coloneqq$
$\displaystyle\int\limits_{A_{T(y_{1}^{n})}}p_{\theta}^{*}(x_{1}^{n})\phi^{*}(T(x_{1}^{n}))dx_{1}^{n}$
(66) $\displaystyle=$
$\displaystyle\int\limits_{A_{T(y_{1}^{n})}}p_{\theta}^{*}(x_{1}^{n})\phi^{*}(T(y_{1}^{n}))dx_{1}^{n},$

since $x_{1}^{n}\in A_{T(y_{1}^{n})}$ implies $T(x_{1}^{n})=T(y_{1}^{n})$.

Observe that

$\displaystyle{\phi^{*}(T(x_{1}^{n}))}$ $\displaystyle=$
$\displaystyle\int\widehat{\theta}(y_{1}^{n})p_{\theta}^{*}(y_{1}^{n}|T(x_{1}^{n}))dy_{1}^{n}$
$\displaystyle=$
$\displaystyle\int\limits_{A_{T(x_{1}^{n})}}\widehat{\theta}(y_{1}^{n})p_{\theta}^{*}(y_{1}^{n}|T(x_{1}^{n}))dy_{1}^{n}$
$\displaystyle+\int\limits_{A^{c}_{T(x_{1}^{n})}}\widehat{\theta}(y_{1}^{n})p_{\theta}^{*}(y_{1}^{n}|T(x_{1}^{n}))dy_{1}^{n}$
$\displaystyle=$
$\displaystyle\int\limits_{A_{T(x_{1}^{n})}}\widehat{\theta}(y_{1}^{n})p_{\theta}^{*}(y_{1}^{n}|T(x_{1}^{n}))dy_{1}^{n}+0$
$\displaystyle=$
$\displaystyle\int\limits_{A_{T(x_{1}^{n})}}\widehat{\theta}(y_{1}^{n})\cdot\Bigg{[}\dfrac{p_{\theta}^{*}(y_{1}^{n})}{q_{\theta}^{*}(T(x_{1}^{n}))}\Bigg{]}dy_{1}^{n}$
$\displaystyle=$
$\displaystyle{\int\limits_{A_{T(x_{1}^{n})}}\widehat{\theta}(y_{1}^{n})p_{\theta}^{*}(y_{1}^{n})dy_{1}^{n}}\Big{/}{\int\limits_{A_{T(x_{1}^{n})}}p_{\theta}^{*}(y_{1}^{n})dy_{1}^{n}}.$

Thus (66) implies

$\displaystyle{\mathbb{E}_{\theta}^{*}\Big{[}\phi^{*}(T(X_{1}^{n}))\cdot\mathbf{1}_{A_{T(y_{1}^{n})}}\Big{]}}$
$\displaystyle=$
$\displaystyle\int\limits_{A_{T(y_{1}^{n})}}p_{\theta}^{*}(x_{1}^{n})\phi^{*}(T(y_{1}^{n}))dx_{1}^{n}$
(67) $\displaystyle=$
$\displaystyle\phi^{*}(T(y_{1}^{n}))\int\limits_{A_{T(y_{1}^{n})}}p_{\theta}^{*}(x_{1}^{n})dx_{1}^{n}$
$\displaystyle=$
$\displaystyle\frac{\int\limits_{A_{T(y_{1}^{n})}}\widehat{\theta}(x_{1}^{n})p_{\theta}^{*}(x_{1}^{n})dx_{1}^{n}}{\int\limits_{A_{T(y_{1}^{n})}}p_{\theta}^{*}(x_{1}^{n})dx_{1}^{n}}\Big{[}\int\limits_{A_{T(y_{1}^{n})}}p_{\theta}^{*}(x_{1}^{n})dx_{1}^{n}\Big{]}$
$\displaystyle=$
$\displaystyle\int\limits_{A_{T(y_{1}^{n})}}\widehat{\theta}(x_{1}^{n})p_{\theta}^{*}(x_{1}^{n})dx_{1}^{n}$
$\displaystyle=$
$\displaystyle\mathbb{E}_{\theta}^{*}\Big{[}\widehat{\theta}(X_{1}^{n}))\cdot\mathbf{1}_{A_{T(y_{1}^{n})}}\Big{]}.$

Observe that (67) holds for every partition set $A_{T(y_{1}^{n})}$ of
$\mathbb{S}^{n}$. This implies both the estimators $\widehat{\theta}$ and
$\phi^{*}(T)$ have the same expected value with respect to $p_{\theta}^{*}$.
That is,

$\mathbb{E}_{\theta}^{*}[\phi^{*}(T(X_{1}^{n}))]=\mathbb{E}_{\theta}^{*}[\widehat{\theta}(X_{1}^{n}))]=\tau^{*}(\theta).$

∎

  * C.4.

###### Proof:

$\mathcal{A}\subseteq\mathcal{B}$ is immediate from Propositions 14 and 15.
Conversely, let $\psi(T)\in\mathcal{B}$. Since
$\mathbb{E}^{*}_{\theta}[\psi(T)]=\tau^{*}(\theta)$, we have
$\mathbb{E}^{*}_{\theta}[\psi^{*}(T)|T]\in\mathcal{A}$. But
$\mathbb{E}^{*}_{\theta}[\psi^{*}(T)|T]=\psi^{*}(T)$. This implies that
$\psi^{*}(T)\in\mathcal{A}$. Hence $\mathcal{B}\subseteq\mathcal{A}$. This
completes the proof. ∎

  * C.5.

###### Proof:

We have

$\displaystyle{\text{Var}^{*}_{\theta}[\widehat{\theta}]}$ $\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\tau^{*}(\theta))^{2}]$
$\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\psi^{*}(T)+\psi^{*}(T)-\tau^{*}(\theta))^{2}]$
$\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\psi^{*}(T))^{2}]+\mathbb{E}^{*}_{\theta}[(\psi^{*}(T)-\tau^{*}(\theta))^{2}]$
$\displaystyle\hskip
28.45274pt+2\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\psi^{*}(T))(\psi^{*}(T)-\tau^{*}(\theta))]$
$\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\psi^{*}(T))^{2}]+\text{Var}^{*}_{\theta}[\psi^{*}(T)]$
$\displaystyle\hskip
28.45274pt+2\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\psi^{*}(T))(\psi^{*}(T)-\tau^{*}(\theta))].$

Now

$\displaystyle{\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\psi^{*}(T))(\psi^{*}(T)-\tau^{*}(\theta))]}$
$\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\psi^{*}(T))\psi^{*}(T)]-\tau^{*}(\theta)~{}\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\psi^{*}(T))]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{=}}$
$\displaystyle\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\psi^{*}(T))\psi^{*}(T)]$
$\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[\widehat{\theta}~{}\psi^{*}(T)]-\mathbb{E}^{*}_{\theta}[(\psi^{*}(T))^{2}]$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{=}}$
$\displaystyle\mathbb{E}^{*}_{\theta}[\mathbb{E}^{*}_{\theta}[\widehat{\theta}~{}\psi^{*}(T)|T]]-\mathbb{E}^{*}_{\theta}[(\psi^{*}(T))^{2}]$
$\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[\psi^{*}(T)~{}\mathbb{E}^{*}_{\theta}[\widehat{\theta}|T]]-\mathbb{E}^{*}_{\theta}[(\psi^{*}(T))^{2}]$
$\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[\psi^{*}(T)\big{\\{}\mathbb{E}^{*}_{\theta}[\widehat{\theta}|T]-\psi^{*}(T)\big{\\}}]$
$\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[\psi^{*}(T)\\{\phi^{*}(T)-\psi^{*}(T)\\}],$

where (a) follows from the fact that both $\widehat{\theta}$ and $\psi^{*}(T)$
are unbiased estimators of $\tau^{*}(\theta)$ and (b) follows from the law of
total expectation. Thus we have

$\displaystyle\text{Var}^{*}_{\theta}[\widehat{\theta}]$ $\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\psi^{*}(T))^{2}]+\text{Var}^{*}_{\theta}[\psi^{*}(T)]+2\mathbb{E}^{*}_{\theta}[\psi^{*}(T)\\{\phi^{*}(T)-\psi^{*}(T)\\}].$

∎

  * C.6.

###### Proof:

From Proposition 15, we have
$\mathbb{E}^{*}_{\theta}[\phi^{*}(T)]=\mathbb{E}^{*}_{\theta}[\widehat{\theta}]=\tau^{*}(\theta)$.
Now in Lemma 17, if we take $\psi^{*}(T)=\phi^{*}(T)$, we have

$\displaystyle\text{Var}^{*}_{\theta}[\widehat{\theta}]$ $\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\phi^{*}(T))^{2}]+\text{Var}^{*}_{\theta}[\phi^{*}(T)]$
$\displaystyle\geq$ $\displaystyle\text{Var}^{*}_{\theta}[\phi^{*}(T)].$

Equality holds if and only if
$\mathbb{E}^{*}_{\theta}[(\widehat{\theta}-\phi^{*}(T))^{2}]=0$, that is, if
and only if
$\widehat{\theta}=\phi^{*}(T)=\mathbb{E}^{*}_{\theta}[\widehat{\theta}|T]$
almost surely. This happens if and only if $\widehat{\theta}$ is a function of
$T$ only. This completes the proof. ∎

  * C.7.

###### Proof:

Let $\phi_{1}(T)$ and $\phi_{2}(T)$ be two unbiased estimators for
$\tau^{*}(\theta)$ with respect to $p_{\theta}^{*}$. Let $\sigma^{2}_{*}$ be
the minimum variance that any unbiased estimator for $\tau^{*}(\theta)$ can
achieve. Suppose
$\text{Var}_{\theta}^{*}[\phi_{1}(T)]=\text{Var}_{\theta}^{*}[\phi_{2}(T)]=\sigma^{2}_{*}$.
We show that $\phi_{1}(T)=\phi_{2}(T)$ almost surely.

Consider $\xi(T):=\dfrac{\phi_{1}(T)+\phi_{2}(T)}{2}$. Then
$\mathbb{E}_{\theta}^{*}[\xi(T)]=\tau^{*}(\theta)$ and

$\displaystyle\text{Var}^{*}_{\theta}[\xi(T)]\leq\sigma_{*}^{2},$

since $\text{Cov}_{\theta}^{*}[\phi_{1}(T),\phi_{2}(T)]\leq\sigma_{*}^{2}$, by
Cauchy-Schwarz inequality. As $\sigma^{2}_{*}$ is the minimum variance that
one unbiased estimator of $\tau^{*}(\theta)$ can achieve, we have
$\text{Var}^{*}_{\theta}[\xi(T)]\geq\sigma_{*}^{2}$. Therefore
$\text{Var}^{*}_{\theta}[\xi(T)]=\sigma_{*}^{2}$. This implies

$\text{Cov}_{\theta}^{*}[\phi_{1}(T),\phi_{2}(T)]=\sigma_{*}^{2}.$

That means $\phi_{1}(T)$ and $\phi_{2}(T)$ are positively correlated. Thus
there exist constants, say $\alpha>0$ and $\beta$, such that

$\phi_{1}(T)=\alpha\phi_{2}(T)+\beta\quad\text{almost surely.}$

Taking expectation and variance with respect to $p_{\theta}^{*}$ on both
sides, respectively, we get

$\tau^{*}(\theta)=\alpha\tau^{*}(\theta)+\beta\quad\text{and}\quad\sigma_{*}^{2}=\alpha^{2}\sigma_{*}^{2}.$

These together imply $\alpha=1$ and $\beta=0$. That is,
$\phi_{1}(T)=\phi_{2}(T)$ almost surely. ∎

## D. Proof of results of Section 5

  * D.1.

###### Proof:

Since $\mathbb{E}_{\theta}^{*}[\psi^{*}(\bar{f})]=0$,

$\int\psi^{*}(\bar{f})N(\theta)[1+w(\theta)\overline{f}]^{\frac{1}{\alpha-1}}dx_{1}^{n}=0.$

That is,

$\int\psi^{*}(\bar{f})[1+w(\theta)\overline{f}]^{\frac{1}{\alpha-1}}dx_{1}^{n}=0.$

Differentiating both sides with respect to $\theta$,

$\int\psi^{*}(\bar{f})\frac{1}{\alpha-1}[1+w(\theta)\overline{f}]^{\frac{2-\alpha}{\alpha-1}}w^{\prime}(\theta)\bar{f}dx_{1}^{n}=0.$

This implies

$\int\frac{\psi^{*}(\bar{f})\bar{f}}{[1+w(\theta)\overline{f}]}p_{\theta}^{*}(x_{1}^{n})dx_{1}^{n}=0.$

That is,

$\mathbb{E}^{*}_{\theta}\Bigg{[}\frac{\psi^{*}(\bar{f})\bar{f}}{1+w(\theta)\overline{f}}\Bigg{]}=0.$
(68)

We now claim that $\mathbb{E}_{\theta}^{*}[\psi^{*}(\bar{f})\bar{f}]\geq 0$.
Let

$k(\theta)\coloneqq\mathbb{E}_{\theta}^{*}[\psi^{*}(\bar{f})\bar{f}].$ (69)

Suppose that $k(\theta)\neq 0$ for all $\theta\in\Theta$. From (69),

$\int\psi^{*}(\bar{f})\bar{f}N(\theta)[1+w(\theta)\bar{f}]^{\frac{1}{\alpha-1}}dx_{1}^{n}=k(\theta).$

That is,

$\int\psi^{*}(\bar{f})\bar{f}[1+w(\theta)\bar{f}]^{\frac{1}{\alpha-1}}dx_{1}^{n}=\frac{k(\theta)}{N(\theta)}.$

Differentiating both sides with respect to $\theta$, we get

$\displaystyle\int\psi^{*}(\bar{f})\bar{f}\frac{1}{\alpha-1}[1+w(\theta)\bar{f}]^{\frac{2-\alpha}{\alpha-1}}w^{\prime}(\theta)\bar{f}dx_{1}^{n}$
$\displaystyle=$
$\displaystyle\frac{\partial}{\partial\theta}\Big{[}\frac{k(\theta)}{N(\theta)}\Big{]}.$

Hence

$\displaystyle{\int\psi^{*}(\bar{f})\frac{{\bar{f}}^{2}}{1+w(\theta)\bar{f}}p^{*}_{\theta}(x_{1}^{n})dx_{1}^{n}}$
$\displaystyle=$
$\displaystyle(\alpha-1)\frac{N(\theta)}{w^{\prime}(\theta)}\frac{\partial}{\partial\theta}\Big{[}\frac{k(\theta)}{N(\theta)}\Big{]}.$

Hence

$\displaystyle{\int\psi^{*}(\bar{f})\bar{f}\Big{[}\frac{1+w(\theta)\bar{f}-1}{1+w(\theta)\bar{f}}\Big{]}p_{\theta}^{*}(x_{1}^{n})dx_{1}^{n}}$
$\displaystyle=$
$\displaystyle(\alpha-1)\frac{w(\theta)N(\theta)}{w^{\prime}(\theta)}\frac{\partial}{\partial\theta}\Big{[}\frac{k(\theta)}{N(\theta)}\Big{]}.$

This implies

$\displaystyle{\mathbb{E}_{\theta}^{*}[\psi^{*}(\bar{f})\bar{f}]-\mathbb{E}^{*}_{\theta}\Big{[}\frac{\psi^{*}(\bar{f})\bar{f}}{1+w(\theta)\overline{f}}\Big{]}}$
$\displaystyle=$
$\displaystyle(\alpha-1)\frac{w(\theta)N(\theta)}{w^{\prime}(\theta)}\frac{\partial}{\partial\theta}\Big{[}\frac{k(\theta)}{N(\theta)}\Big{]}.$

Using (68), we get

$k(\theta)=(\alpha-1)\frac{w(\theta)N(\theta)}{w^{\prime}(\theta)}\frac{\partial}{\partial\theta}\Big{[}\frac{k(\theta)}{N(\theta)}\Big{]}.$

Since $k(\theta)\neq 0$, we have

$(\alpha-1)\frac{\partial}{\partial\theta}\Big{[}\frac{k(\theta)}{N(\theta)}\Big{]}\Big{/}\Big{[}\frac{k(\theta)}{N(\theta)}\Big{]}=\frac{w^{\prime}(\theta)}{w(\theta)}.$

That is,

$(\alpha-1)\frac{\partial}{\partial\theta}\log\Big{[}\frac{k(\theta)}{N(\theta)}\Big{]}=\frac{\partial}{\partial\theta}\log
w(\theta).$

This reduces to

$\frac{\partial}{\partial\theta}\log\Big{[}\frac{k(\theta)}{N(\theta)}\Big{]}^{\alpha-1}=\frac{\partial}{\partial\theta}\log
w(\theta).$

Integrating both sides with respect to $\theta$, we get

$\log\Big{[}\frac{k(\theta)}{N(\theta)}\Big{]}^{\alpha-1}=\log w(\theta)+\log
C$

where $\log C$ is the constant of integration. This implies

$k(\theta)^{\alpha-1}=C\cdot w(\theta)N(\theta)^{\alpha-1}.$

Observe that $C>0$ and $N(\theta)>0$ for all $\theta\in\Theta$. Hence, if
$w(\theta)>0$ for all $\theta\in\Theta$, we get $k(\theta)>0$. This, together
with the case $k(\theta)=0$, implies $k(\theta)\geq 0$. That means, in any
case, $\mathbb{E}^{*}_{\theta}[\psi^{*}(\bar{f})\bar{f}]\geq 0$. Therefore

$\displaystyle\text{Cov}_{\theta}^{*}[\psi^{*}(\bar{f}),\bar{f}]$
$\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[\psi^{*}(\bar{f})\bar{f}]-\mathbb{E}^{*}_{\theta}[\psi^{*}(\bar{f})]\mathbb{E}^{*}_{\theta}[\bar{f}]$
$\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[\psi^{*}(\bar{f})\bar{f}]-0$
$\displaystyle=$
$\displaystyle\mathbb{E}^{*}_{\theta}[\psi^{*}(\bar{f})\bar{f}]$
$\displaystyle\geq$ $\displaystyle 0.$

∎

  * D.2.

###### Proof:

Consider

$\displaystyle p_{\theta}^{*}(x_{1}^{n})$ $\displaystyle=$ $\displaystyle
N(\theta)[1+w(\theta)\overline{f}]^{\frac{1}{\alpha-1}}$ $\displaystyle=$
$\displaystyle[N(\theta)^{\alpha-1}+N(\theta)^{\alpha-1}w(\theta)\overline{f}]^{\frac{1}{\alpha-1}}.$

For convenience, let $F(\theta):=N(\theta)^{\alpha-1}$ and
$\widetilde{w}(\theta):=N(\theta)^{\alpha-1}w(\theta)$. Then

$p_{\theta}^{*}(x_{1}^{n})=[F(\theta)+\widetilde{w}(\theta)\overline{f}]^{\frac{1}{\alpha-1}},$

and thus

$\displaystyle\frac{\partial}{\partial\theta}p_{\theta}^{*}(x_{1}^{n})=\frac{1}{\alpha-1}p_{\theta}^{*}(x_{1}^{n})^{2-\alpha}[F^{\prime}(\theta)+\widetilde{w}^{\prime}(\theta)\overline{f}].~{}~{}$
(70)

Hence, from the definition of $s^{*}$,

$\displaystyle\text{Cov}^{*}_{\theta}[s^{*}(X_{1}^{n},\theta),p_{\theta}^{*}(X_{1}^{n})^{\alpha-1}s^{*}(X_{1}^{n},\theta)]$
$\displaystyle=\int
s^{*}(x_{1}^{n},\theta)^{2}p^{*}_{\theta}(x_{1}^{n})^{\alpha}dx_{1}^{n}$
$\displaystyle=\int
s^{*}(x_{1}^{n},\theta)\Big{[}\frac{1}{p^{*}_{\theta}(x_{1}^{n})}\frac{\partial}{\partial\theta}p^{*}_{\theta}(x_{1}^{n})\Big{]}p^{*}_{\theta}(x_{1}^{n})^{\alpha}dx_{1}^{n}$
$\displaystyle=\tfrac{1}{\alpha-1}\int
p^{*}_{\theta}(x_{1}^{n})s^{*}(x_{1}^{n},\theta)[F^{\prime}(\theta)+\widetilde{w}^{\prime}(\theta)\bar{f}]dx_{1}^{n}$
$\displaystyle=\frac{1}{\alpha-1}\int
p^{*}_{\theta}(x_{1}^{n})s^{*}(x_{1}^{n},\theta)F^{\prime}(\theta)dx_{1}^{n}+\frac{1}{\alpha-1}\int
p^{*}_{\theta}(x_{1}^{n})s^{*}(x_{1}^{n},\theta)\widetilde{w}^{\prime}(\theta)\bar{f}dx_{1}^{n}$
$\displaystyle=\frac{1}{\alpha-1}\widetilde{w}^{\prime}(\theta)\int
p^{*}_{\theta}(x_{1}^{n})\frac{1}{p^{*}_{\theta}(x_{1}^{n})}\frac{\partial}{\partial\theta}p^{*}_{\theta}(x_{1}^{n})\bar{f}dx_{1}^{n}$
$\displaystyle=\frac{1}{\alpha-1}\widetilde{w}^{\prime}(\theta)\frac{\partial}{\partial\theta}\int
p^{*}_{\theta}(x_{1}^{n})\bar{f}dx_{1}^{n}$
$\displaystyle=\frac{1}{\alpha-1}\widetilde{w}^{\prime}(\theta)\frac{\partial}{\partial\theta}\tau^{*}(\theta).$

Using (70), we get

$\displaystyle{\int
s^{*}(x_{1}^{n},\theta)p_{\theta}^{*}(x_{1}^{n})^{\alpha}dx_{1}^{n}}$
$\displaystyle=\int\frac{\partial}{\partial\theta}\log
p_{\theta}^{*}(x_{1}^{n})p_{\theta}^{*}(x_{1}^{n})^{\alpha}dx_{1}^{n}$
$\displaystyle=\frac{1}{\alpha-1}\int
p_{\theta}^{*}(x_{1}^{n})[F^{\prime}(\theta)+\widetilde{w}^{\prime}(\theta)\bar{f}]dx_{1}^{n}$

and

$\displaystyle{\int
s^{*}(x_{1}^{n},\theta)^{2}p_{\theta}^{*}(x_{1}^{n})^{2\alpha-1}dx_{1}^{n}}$
$\displaystyle=\int\Big{[}\frac{\partial}{\partial\theta}\log
p_{\theta}^{*}(x_{1}^{n})\Big{]}^{2}p_{\theta}^{*}(x_{1}^{n})^{2\alpha-1}dx_{1}^{n}$
$\displaystyle=\Big{(}\frac{1}{\alpha-1}\Big{)}^{2}\int
p_{\theta}^{*}(x_{1}^{n})[F^{\prime}(\theta)+\widetilde{w}^{\prime}(\theta)\bar{f}]^{2}dx_{1}^{n}.$

Thus

$\displaystyle{\text{Var}_{\theta}^{*}[p_{\theta}^{*}(X_{1}^{n})^{\alpha-1}s^{*}(X_{1}^{n},\theta)]}$
$\displaystyle=\int
s^{*}(x_{1}^{n},\theta)^{2}p_{\theta}^{*}(x_{1}^{n})^{2\alpha-1}dx_{1}^{n}-\Big{[}\int
s^{*}(x_{1}^{n},\theta)p_{\theta}^{*}(x_{1}^{n})^{\alpha}dx_{1}^{n}\Big{]}^{2}$
$\displaystyle=\Big{(}\frac{1}{\alpha-1}\Big{)}^{2}\\{\mathbb{E}^{*}_{\theta}[(F^{\prime}(\theta)+\widetilde{w}^{\prime}(\theta)\bar{f})^{2}]-(\mathbb{E}^{*}_{\theta}[F^{\prime}(\theta)+\widetilde{w}^{\prime}(\theta)\bar{f}])^{2}\\}$
$\displaystyle=\Big{(}\frac{1}{\alpha-1}\Big{)}^{2}\text{Var}^{*}_{\theta}[F^{\prime}(\theta)+\widetilde{w}^{\prime}(\theta)\bar{f}]$
$\displaystyle=\Big{(}\frac{1}{\alpha-1}\Big{)}^{2}[\widetilde{w}^{\prime}(\theta)]^{2}\text{Var}^{*}_{\theta}[\bar{f}].$

∎

  * D.3.

###### Proof:

The deformed probability distribution associated with Basu et al. likelihood
function is given by

$\displaystyle
p_{\theta}^{*}(x_{1}^{n})=\dfrac{\exp[\frac{\alpha}{n(\alpha-1)}\sum_{i=1}^{n}p_{\theta}(x_{i})^{\alpha-1}]}{\int\exp[\frac{\alpha}{n(\alpha-1)}\sum_{i=1}^{n}p_{\theta}(y_{i})^{\alpha-1}]dy_{1}^{n}}.~{}~{}$
(71)

$T:=T(x_{1}^{n})$ is a sufficient statistic for $\theta$ with respect to Basu
et al. likelihood function. Let
$\widehat{\theta}:=\widehat{\theta}(x_{1}^{n})$ be an unbiased estimator of
$\tau(\theta)$ with respect to $p_{\theta}^{*}$. Let us define

$\displaystyle\phi(T(x_{1}^{n}))$ $\displaystyle:=$
$\displaystyle\mathbb{E}_{\theta}^{*}[\widehat{\theta}|T=T(x_{1}^{n})]$
$\displaystyle=$
$\displaystyle\int\widehat{\theta}(y_{1}^{n})p_{\theta}^{*}(y_{1}^{n}|T(x_{1}^{n}))dy_{1}^{n},$

where the conditional distribution is as defined in (11) of the main article.
Then proceeding with similar steps as in Lemma 17 and Theorem 18, we have the
following.

    * 1.

$\mathbb{E}_{\theta}^{*}[\phi(T)]=\mathbb{E}_{\theta}^{*}[\widehat{\theta}]=\tau(\theta)$,

    * 2.

$\text{Var}_{\theta}^{*}[\phi(T)]\leq\text{Var}_{\theta}^{*}[\widehat{\theta}]$,
and equality holds if and only if $\widehat{\theta}$ is a function of $T$
only.

Now for the given $\mathbb{B}^{(\alpha)}$-family, using (71), we have

$p_{\theta}^{*}(x_{1}^{n})=\exp\Big{[}\frac{\alpha}{\alpha-1}\overline{h}+\widetilde{F}(\theta)+\frac{\alpha}{\alpha-1}w(\theta)\overline{f}\Big{]}.$

Recall that $T=\overline{f}$ is a sufficient statistic for $\theta$ with
respect to Basu et al. likelihood function. Using generalized Rao-Blackwell
theorem for Basu et al. likelihood function, we get

$\displaystyle\mathbb{E}_{\theta}^{*}[\phi(\overline{f})]=\mathbb{E}_{\theta}^{*}[\widehat{\theta}]=\tau(\theta)\quad\text{and}$
$\displaystyle\text{Var}_{\theta}^{*}[\phi(\bar{f})]\leq\text{Var}_{\theta}^{*}[\widehat{\theta}].$
(72)

Let us define $\psi(\bar{f})\coloneqq\phi(\bar{f})-\bar{f}$. Then
$\mathbb{E}_{\theta}^{*}[\psi(\overline{f})]=0$, since
$\mathbb{E}_{\theta}^{*}[\bar{f}]=\tau(\theta)$ by hypothesis. Also

$\displaystyle{\text{Var}_{\theta}^{*}[\phi(\bar{f})]}$ $\displaystyle=$
$\displaystyle\text{Var}_{\theta}^{*}[\psi(\bar{f})+\bar{f}]$ $\displaystyle=$
$\displaystyle\text{Var}_{\theta}^{*}[\psi(\bar{f})]+\text{Var}_{\theta}^{*}[\bar{f}]+2~{}\text{Cov}^{*}(\psi(\bar{f}),\bar{f}).$

Proceeding as in C.1., one can show that
$\text{Cov}^{*}(\psi(\bar{f}),\bar{f})=0$. Thus we get

$\text{Var}_{\theta}^{*}[\phi(\bar{f})]=\text{Var}_{\theta}^{*}[\psi(\bar{f})]+\text{Var}_{\theta}^{*}[\bar{f}]\geq\text{Var}_{\theta}^{*}[\bar{f}].$

Hence the inequality in (D.3.) reduces to

$\text{Var}_{\theta}^{*}[\widehat{\theta}]\geq\text{Var}_{\theta}^{*}[\bar{f}],$
(73)

that is, $\bar{f}$ is the best estimator of its expected value with respect to
$p_{\theta}^{*}$.

We next show that
$\text{Var}_{\theta}^{*}[\bar{f}]=\dfrac{[\tau^{\prime}(\theta)]^{2}}{I_{n}^{*}(\theta)}$.
Observe that $p_{\theta}^{*}$ forms a one-parameter exponential family and

$\displaystyle I_{n}^{*}(\theta)$ $\displaystyle=$
$\displaystyle\text{Var}_{\theta}^{*}[\frac{\partial}{\partial\theta}\log
p_{\theta}^{*}(X_{1}^{n})]$ (74) $\displaystyle=$
$\displaystyle\text{Var}_{\theta}^{*}[\widetilde{F}^{\prime}(\theta)+\frac{\alpha}{\alpha-1}w^{\prime}(\theta)\bar{f}]$
$\displaystyle=$
$\displaystyle\Big{(}\frac{\alpha}{\alpha-1}\Big{)}^{2}(w^{\prime}(\theta))^{2}~{}\text{Var}_{\theta}^{*}[\bar{f}].$

Using the fact $\mathbb{E}^{*}_{\theta}[\frac{\partial}{\partial\theta}\log
p_{\theta}^{*}(X_{1}^{n})]=0$ and
$\mathbb{E}_{\theta}^{*}[\bar{f}]=\tau(\theta)$, we get

$\tau(\theta)=-\frac{(\alpha-1)}{\alpha}\frac{\widetilde{F}^{\prime}(\theta)}{w^{\prime}(\theta)}.$
(75)

Further taking derivative on both sides of
$\mathbb{E}^{*}_{\theta}[\frac{\partial}{\partial\theta}\log
p_{\theta}^{*}(X_{1}^{n})]=0$, we get

$\tau^{\prime}(\theta)=\frac{\alpha}{\alpha-1}w^{\prime}(\theta)\text{Var}_{\theta}^{*}[\bar{f}].$
(76)

Thus, using (74), (75) and (76), we get

$\text{Var}_{\theta}^{*}[\overline{f}]=\frac{[\tau^{\prime}(\theta)]^{2}}{I_{n}^{*}(\theta)}.$

This completes the proof. ∎

## Acknowledgments

The authors would like to thank the Editor, Associate Editor, and Referees for
their valuable comments that improved the presentation of the paper.

## References

  * [1] E. W. Barankin and A. P. Maitra, “Generalization of the Fisher-Darmois-Koopman-Pitman theorem on sufficient statistics,” _Sankhya: The Indian Journal of Statistics, Series A_ , vol. 25(3), pp. 217–244, 1963.
  * [2] A. G. Bashkirov, “On maximum entropy principle, superstatistics, power-law distribution and Rényi parameter,” _Phys. A_ , vol. 340, pp. 153–162, 2004.
  * [3] A. Basu, I. R. Harris, N. L. Hjort, and M. C. Jones, “Robust and efficient estimation by minimizing a density power divergence,” _Biometrika_ , vol. 85, pp. 549–559, 1998.
  * [4] D. Blackwell, “Conditional expectation and unbiased sequential estimation,” _Ann. Math. Statist._ , vol. 18(1), pp. 105–110, 1947.
  * [5] M. Broniatowski, A. Toma, and I. Vajda, “Decomposable pseudo-distances and applications in statistical estimation,” _J. Statist. Plann. Inference_ , vol. 142, pp. 2574–2585, 2012.
  * [6] M. Broniatowski and I. Vajda, “Several applications of divergence criteria in continuous families,” _Kybernetika (Prague)_ , vol. 48, pp. 600–636, 2012\.
  * [7] L. D. Brown, _Fundamentals of Statistical Exponential Families: with Applications in Statistical Decision Theory_. Institute of Mathematical Statistics, Hayward, CA., 1986.
  * [8] C. Bunte and A. Lapidoth, “Encoding tasks and Rényi entropy,” _IEEE Trans. Inform. Theory_ , vol. 60, pp. 5065–5076, 2014.
  * [9] J. c Principe, _Information Theoretic Learning: Rényi’s Entropy and Kernel Perspectives_. Springer Science and Business Media, New York, USA., 2010.
  * [10] G. Casella and R. L. Berger, _Statistical Inference_. 2nd ed. Duxbury Press, 2002.
  * [11] A. Cichocki and S. Amari, “Families of alpha-beta-and gamma-divergences: Flexible and robust measure of similarities,” _Entropy_ , vol. 12, pp. 1532–1568, 2010.
  * [12] H. Cramér, _Mathematical methods of statistics_. Princeton: Princeton University Press, 1946.
  * [13] I. Csiszár, “Generalized cutoff rates and Rényi’s information measures,” _IEEE Trans. Inform. Theory._ , vol. 41, pp. 26–34, 1995.
  * [14] I. Csiszár and F. Matúš, “Generalized minimizers of convex integral functionals, Bergman distance, Pythagorean identities,” _Kybernetika (Prague)._ , vol. 48, pp. 637–689, 2012.
  * [15] I. Csiszár and P. C. Shields, _Information Theory and Statistics: A Tutorial_. Foundations and Trends in Communications and Information Theory, 2004.
  * [16] G. Darmois, “Sur les lois de probabilites a estimation exhaustive,” _C. R. Acad. Sci. Paris (in French)_ , vol. 200, pp. 1265–1266, 1935.
  * [17] S. Eguchi, O. Komori, and S. Kato, “Projective power entropy and maximum Tsallis entropy distributions,” _Entropy_ , vol. 13, pp. 1746–1764, 2011\.
  * [18] C. Field and B. Smith, “Robust estimation: A weighted maximum likelihood approach,” _Int. Stat. Rev._ , vol. 62, pp. 405–424, 1994.
  * [19] R. A. Fisher, “On the mathematical foundations of theoretical statistics,” _Philos. Trans. Roy. Soc. A._ , vol. 222, pp. 309–368, 1922.
  * [20] ——, “Two new properties of mathematical likelihood,” _Proceedings of the Royal Society, Series A_ , vol. 144, pp. 285–307, 1934.
  * [21] H. Fujisawa, “Normalized estimating equation for robust parameter estimation,” _Electron. J. Stat._ , vol. 7, pp. 1587–1606, 2013.
  * [22] H. Fujisawa and S. Eguchi, “Robust parameter estimation with a small bias against heavy contamination,” _J. Multivariate Anal._ , vol. 99, pp. 2053–2081, 2008.
  * [23] A. Gayen and M. A. Kumar, “Generalized estimating equation for the Student-t distributions,” in _2018 IEEE International Symposium on Information Theory (ISIT)_ , 2018, pp. 571–575.
  * [24] ——, “A generalized notion of sufficiency for power-law distributions,” in _2021 IEEE International Symposium on Information Theory (ISIT)_ , 2021, pp. 2185–2190.
  * [25] ——, “Projection theorems and estimating equations for power-law models,” _Journal of Multivariate Analysis_ , vol. 184, p. 104734, 2021.
  * [26] P. R. Halmos and L. J. Savage, “Application of the Radon-Nikodym theorem to the theory of sufficient statistics,” _The Annals of Mathematical Statistics_ , vol. 20, pp. 225 – 241, 1949.
  * [27] H. G. Hoang, B. Vo, B. T. Vo, and R. Mahler, “The Cauchy–Schwarz divergence for poisson point processes,” _IEEE Transactions on Information Theory_ , vol. 61, pp. 4475–4485, 2015.
  * [28] R. Jenssen, J. C. Principe, D. Erdogmus, and T. Eltoft, “The Cauchy–Schwarz divergence and Parzen windowing: Connections to graph theory and Mercer kernels,” _Journal of the Franklin Institute_ , vol. 343, pp. 614–629, 2006.
  * [29] M. C. Jones, N. L. Hjort, I. R. Harris, and A. Basu, “A comparison of related density based minimum divergence estimators,” _Biometrika_ , vol. 88, pp. 865–873, 2001.
  * [30] K. Kampa, E. Hasanbelliu, and J. C. Principe, “Closed-form Cauchy-Schwarz pdf divergence for mixture of gaussians,” _Proceedings of International Joint Conference on Neural Networks, San Jose, California, USA_ , pp. 2578–2585, 2011.
  * [31] B. O. Koopman, “On distributions admitting a sufficient statistic,” _Trans. Amer. Math. Soc._ , vol. 39, pp. 399–409, 1936.
  * [32] M. A. Kumar and R. Sundaresan, “Minimization problems based on relative $\alpha$-entropy I: Forward projection,” _IEEE Trans. Inform. Theory._ , vol. 61, pp. 5063–5080, 2015.
  * [33] ——, “Minimization problems based on relative $\alpha$-entropy II: Reverse projection,” _IEEE Trans. Inform. Theory._ , vol. 61, pp. 5081–5095, 2015.
  * [34] E. L. Lehmann and G. Casella, _Theory of point estimation_. Springer, 2nd ed., 1998.
  * [35] E. Lutwak, D. Yang, and G. Zhang, “Cramér-Rao and moment-entropy inequalities for Rényi entropy and generalized Fisher information,” _IEEE Trans. Inform. Theory._ , vol. 51, pp. 473–478, 2005.
  * [36] A. Maji, A. Ghosh, and A. Basu, “The logarithmic super divergence and asymptotic inference properties,” _AStA Adv. Stat. Anal_ , vol. 100, pp. 99–131, 2016.
  * [37] J. Naudts, “Estimators, escort probabilities, and $\phi$-exponential families in statistical physics,” _J. Inequal. Pure. Appl. Math._ , vol. 5, p. 102, 2004.
  * [38] J. Neyman, “Sur un teorema concernente la cosidette statisticle sufficienti,” _Giorn. Ist. Ital. Attuari_ , vol. 6, pp. 320–334, 1935.
  * [39] F. Nielsen, K. Sun, and S. Marchand-Maillet, “k-means clustering with Hölder divergences,” _Geometric Science of Information_ , pp. 856–863, 2017.
  * [40] ——, “On Hölder projective divergences,” _Entropy_ , vol. 19, pp. 1–28, 2017.
  * [41] E. J. G. Pitman, “Sufficient statistics and intrinsic accuracy,” _Mathematical Proceedings of the Cambridge Phil. Soc._ , vol. 32, pp. 567–579, 1936.
  * [42] C. R. Rao, “Information and accuracy attainable in the estimation of statistical parameters,” _Bulletin of the Calcutta Mathematical Society_ , vol. 37(3), pp. 81–91, 1945.
  * [43] ——, “Minimum variance and the estimation of several parameters,” _Mathematical Proceedings of the Cambridge Philosophical Society_ , vol. 43, pp. 280–283, 1947.
  * [44] ——, “Sufficient statistics and minimum variance estimates,” _Mathematical Proceedings of the Cambridge Philosophical Society_ , vol. 45(2), pp. 213–218, 1949.
  * [45] Z. Shanqing, Z. Kunlong, and X. Weibin, “Application of Cauchy-Schwarz divergence in image segmentation,” _Computer Engineering and Applications_ , vol. 49, pp. 129–131, 2013.
  * [46] R. Sundaresan, “A measure of discrimination and its geometric properties,” in _2002 IEEE International Symposium on Information Theory (ISIT)_ , 2002, p. 264.
  * [47] ——, “Guessing under source uncertainty,” _IEEE Trans. Inform. Theory._ , vol. 53, pp. 269–287, 2007.
  * [48] M. P. Windham, “Robustifying model fitting,” _J. R. Stat. Soc. Ser. B. Stat. Methodol._ , vol. 57, pp. 599–609, 1995.