# Generalized van Trees inequality:
Local minimax bounds for non-smooth functionals and
irregular statistical models

Kenta Takatsu  Department of Statistics and Data Science, Carnegie Mellon
University  Arun Kumar Kuchibhotla  Department of Statistics and Data Science,
Carnegie Mellon University

###### Abstract

In a decision-theoretic framework, minimax lower bound provides the worst-case
performance of estimators relative to a given class of statistical models. For
parametric and semiparametric models, the Hájek–Le Cam local asymptotic
minimax (LAM) theorem provides the optimal and sharp asymptotic lower bound.
Despite its relative generality, this result comes with limitations as it only
applies to the estimation of differentiable functionals under regular
statistical models. On the other hand, non-asymptotic minimax lower bounds,
such as those based on the reduction to hypothesis testing, do not often yield
sharp asymptotic constants. Inspired by the recent improvement of the van
Trees inequality and related methods in the literature, we provide new non-
asymptotic minimax lower bounds under minimal regularity assumptions, which
imply sharp asymptotic constants. The proposed lower bounds do not require the
differentiability of functionals or regularity of statistical models,
extending the efficiency theory to broader situations where standard
approaches fail. Additionally, new lower bounds provide non-asymptotic
constants, which can shed light on more refined fundamental limits of
estimation in finite samples. We demonstrate that new lower bounds recover
many classical results, including the LAM theorem and semiparametric
efficiency bounds. We also illustrate the use of the new lower bound by
deriving the local minimax lower bound for estimating the density at a point
and directionally differentiable parameters.

Keywords— van Trees inequality, Hammersley–Chapman–Robbins bound, Local
asymptotic minimax theorem, Local minimaxity, Efficiency theory, Nonsmooth
functional estimation, Irregular statistical models

###### Contents

  1. 1 Introduction
    1. 1.1 Motivation
    2. 1.2 Literature review
  2. 2 Setup and background
    1. 2.1 The local behavior of divergence metrics
    2. 2.2 Minimax lower bounds via parametric paths
  3. 3 General minimax lower bounds under weaker regularities
    1. 3.1 Lower bound based on the approximating functionals
    2. 3.2 Lower bound based on the chi-squared divergence
    3. 3.3 Lower bound based on the Hellinger distance
    4. 3.4 The extension from $\mathbb{R}^{d}$ to a general parameter space
    5. 3.5 Connection to the modulus of continuity
  4. 4 Asymptotic properties
    1. 4.1 Local asymptotic minimax lower bound
    2. 4.2 Semiparametric efficiency bound
    3. 4.3 Local minimax rate for irregular estimation
  5. 5 Applications: Local minimax lower bounds
    1. 5.1 Nonparametric density estimation
    2. 5.2 Simple directionally differentiable functionals
    3. 5.3 Parameter estimation under an irregular model
  6. 6 Visual illustration: Upper bounds attained by plug-in estimators
  7. 7 Concluding remarks
  8. A Proof of the lower bound based on the approximation
    1. A.1 Proof of Lemma 7
    2. A.2 Proof of Theorem 8
  9. B Proofs of the extensions of the van Trees inequality
    1. B.1 Proof of Theorem 9
    2. B.2 Proof of Theorem 10
    3. B.3 Proof of Lemma 11
  10. C Proofs of asymptotic properties
    1. C.1 Proofs of local asymptotic minimax theorem
    2. C.2 Proofs of semiparametric efficiency bound
    3. C.3 Proof of local minimax rate for irregular estimation
  11. D Nonparametric density estimation
    1. D.1 Local parameter space
    2. D.2 Approximation bias
    3. D.3 Surrogate efficiency
    4. D.4 Optimization
  12. E Optimization under absolute moment constraints
  13. F Additional derivations related to examples
    1. F.1 Example 3
    2. F.2 Example 4
  14. G Derivation of the risk of estimators
    1. G.1 Lower bound under Gaussian prior and arctan diffeomorphism
    2. G.2 Exact upper bounds
  15. H Supporting lemmas on divergence metrics

##  1 Introduction

###  1.1 Motivation

In a decision-theoretic framework, the optimality of an estimation procedure
is often motivated by the attainability to minimax lower bound. Specifically,
a proposed estimator is considered rate-optimal when its convergence rate
matches the rate of the corresponding minimax lower bound. A more refined
assessment is to examine whether an estimator is asymptotically efficient,
meaning the risk of the estimator coincides with the minimax lower bound
including the constant. If this is the case, the estimation procedure becomes
no longer improvable at least asymptotically. Hence, the construction of
precise minimax lower bounds has been a crucial step for evaluating any
statistical procedure.

The optimality of functional estimation has been extensively studied in both
parametric and nonparametric models (Ibragimov and Has’minskii, , 1981; Bickel
et al., , 1993; Lehmann and Casella, , 2006). Semiparametric efficiency
theory, in particular, is considered a cornerstone for understanding the
asymptotic optimality of functional estimation. The convolution theorem and
the local asymptotic minimax theorem are two fundamental results in the
development of semiparametric efficiency (Hájek, , 1970, 1972; Le Cam, ,
1972); however, these theorems only apply to smooth functionals and require
certain regularity conditions on the underlying statistical models. As optimal
estimation of non-smooth functionals has gained growing interest, extending
the classical efficiency theory to broader settings is an imminent task.

This manuscript aims to develop general minimax lower bounds under minimal
regularity assumptions imposed on functionals, estimators, and underlying
statistical models. We consider a (nonparametric) functional
$\psi:\mathcal{P}\mapsto\mathbb{R}^{k}$, where $\mathcal{P}$ is a collection
of probability measures containing the distribution from which the observation
is drawn. The unknown data-generating distribution is denoted by
$P_{\theta_{0}}$ and a local statistical model containing $P_{\theta_{0}}$ is
given by $\\{P_{\theta}:\theta\in\Theta_{0}\subseteq\Theta\\}$, satisfying
$\theta_{0}\in\Theta_{0}$. Here, the indexing set $\Theta_{0}$ is a subset of
a metric space equipped with an appropriate metric. Our primary focus is on
the following general minimax risk for an arbitrary parameter set
$\Theta_{0}\subseteq\Theta$:

$\inf_{T}\,\sup_{\theta\in\Theta_{0}}\,\operatorname{\mathbb{E}}_{P_{\theta}}\|T(X)-\psi(P_{\theta})\|^{2}$
(1)

where the infimum is taken over mesurable functions and
$\|\cdot\|:\mathbb{R}^{k}\mapsto\mathbb{R}_{+}$ for
$\mathbb{R}_{+}=[0,\infty)$ is any vector norm. We also denote by
$\operatorname{\mathbb{E}}_{P_{\theta}}$ the expectation under $P_{\theta}$.
It should be noted that this framework is not confined to a parametric model,
and it can be extended to a nonparametric model using the standard machinery
from semiparametric statistics (Bickel et al., , 1993; Van der Vaart, , 2002).

Classical local asymptotic results investigate when $\Theta_{0}$ forms a
shrinking ball around $\theta_{0}$ whose radius converges to zero at a
suitable rate, depending on the sample size $n$. For example, the risk
displayed by (1) becomes equivalent to the local asymptotic minimax risk under
the root-$n$ rate, which is given by

$\liminf_{c\longrightarrow\infty}\,\liminf_{n\longrightarrow\infty}\,\inf_{T}\,\sup_{\|\theta-\theta_{0}\|<cn^{-1/2}}\,n\operatorname{\mathbb{E}}_{P_{\theta}}\|T(X)-\psi(P_{\theta})\|^{2}.$
(2)

Particularly, the local asymptotic minimax (LAM) theorem (Hájek, , 1970, 1972;
Le Cam, , 1972) states that the lower bound to the above display is given by
the variance of the efficient influence function under fairly demanding
regularity conditions. On the other hand, little is known about the lower
bound when these regularity conditions fail to hold.

One immediate limitation of the LAM theorem is its asymptotic nature. To
address this limitation, numerous non-asymptotic minimax lower bounds have
been proposed in the literature. Many share the following principle:
maximizing the separation of functionals evaluated at two, or more, “similar”
distributions. The similarity of distributions is quantified by different
metrics and divergences such as the Hellinger distance, total variation, or
the $f$-divergences, which includes the KL and the chi-squared divergences.
While these non-asymptotic lower bounds may provide correct rates, they often
fall short of recovering the correct constant implied by the LAM theorem. To
the best of our knowledge, existing non-asymptotic lower bounds based on the
Hellinger distance or total variation (LeCam, , 1973; Ibragimov and
Has’minskii, , 1981; Assouad, , 1983; Donoho and Liu, , 1987; Birgé, , 1987)
cannot imply the LAM theorem even for regular parametric models. Some chi-
squared-based lower bounds (Gill and Levit, , 1995; Gassiat and Stoltz, ,
2024), on the other hand, can recover the LAM theorem. Cai and Low,  (2011)
proposed a chi-squared-based lower bound for non-smooth functionals, resulting
in a sharp constant for the problem they studied.

The second limitation of the LAM theorem is that it requires the existence of
an efficient influence function. This excludes two classes of estimation
problems: irregular distributions and non-smooth functionals. Under irregular
statistical models (e.g., $\mbox{Uniform}[0,\theta],\theta\geq 0$), the chi-
squared divergences are often infinite, implying trivial results from the
aforementioned minimax lower bounds using chi-squared divergences (Gill and
Levit, , 1995; Cai and Low, , 2011; Gassiat and Stoltz, , 2024). On the other
hand, other metrics such as the Hellinger or total variation distances, do not
suffer from this problem under irregularity. Thus, one of our main motivations
is to understand whether we can obtain lower bounds based on the Hellinger or
total variation distances that yield the LAM theorem for regular parametric
models. Regarding non-smooth functionals, there is barely any analogous
statement to the LAM theorem. A few works have extended local asymptotic
minimaxity in the context of plug-in estimators for directionally
differentiable functionals (Fang, , 2014) or specific non-smooth functionals
that are non-differentiable transforms of a regular parameter (Song, 2014a, ;
Song, 2014b, ). This manuscript also aims to develop new local asymptotic
results that apply to any, possibly non-smooth, functionals or estimators.

Before proceeding, we would like to acknowledge that general-purpose minimax
lower bounds can only go so far in the sense that their application to
specific instances requires additional work. We present some examples to show
how our results can be applied.

#### Summary of main results.

Our main results are summarized as follows. On a conceptual level, we
demonstrate that the general minimax risk for estimating non-smooth
functionals is characterized by the interplay between two non-negative terms
in the proceeding display:

$\displaystyle\inf_{T}\,\sup_{\theta\in\Theta_{0}}\,\operatorname{\mathbb{E}}_{P_{\theta}}\|T(X)-\psi(P_{\theta})\|^{2}\geq\sup_{\phi}\,\left[\left\\{\textrm{Term
I}(\phi;\Theta_{0})\right\\}^{1/2}-\left\\{\textrm{Term
II}(\psi,\phi;\Theta_{0})\right\\}^{1/2}\right]^{2}_{+}$

where $[t]_{+}:=\max(t,0)$ and $\phi$ is any functional that approximates the
original functional $\psi$. The first term, we call it surrogate efficiency,
is an efficiency bound for estimating the functional $\phi$. When $\phi$
provides sufficient structures such as smoothness, this term becomes well-
studied in the literature for both parametric and nonparametric models. The
second term, we call it approximation bias, quantifies the deviation of the
introduced functional $\phi$ from the original functional $\psi$. The overall
minimax risk is then obtained by optimizing the choice of $\phi$. Such an
interplay between surrogate efficiency and approximation bias resembles the
classical bias-variance decomposition of an estimation error. Hence, our
result can be seen as the minimax lower bound analog to the bias-variance
trade-off.

We provide two concrete instances of the above idea in our main theorems.
First, Theorem 8 shows that the minimax risk admits the following lower bound
based on the projection onto the set $\Phi_{\textrm{ac}}$, all absolutely
continuous functions of $\theta\in\Theta_{0}$. Let $\psi$ be a real-valued
functional of interest and $\phi\in\Phi_{\textrm{ac}}$ with almost-everywhere
derivative $\nabla\phi$. Our formal result in Section 3.3 holds for any
vector-valued functional $\psi$. Assume that the statistical model
$\\{P_{\theta}:\,\theta\in\Theta_{0}\\}$ is regular such that the Fisher
information $\mathcal{I}(\theta)$ at $P_{\theta}$ is well-defined (see our
definition in (3)). Then, for all “nice” priors $Q$ on $\Theta_{0}$ (see
Definition 1) with Fisher information $\mathcal{I}(Q)$, defined in (5), it
follows that:

$\displaystyle\sup_{\theta\in\Theta_{0}}\,\operatorname{\mathbb{E}}_{P_{\theta}}|T(X)-\psi(\theta)|^{2}\geq\left[\Gamma^{1/2}_{Q,\phi}-\left(\int_{\Theta_{0}}|\psi(\theta)-\phi(\theta)|^{2}dQ(\theta)\right)^{1/2}\right]_{+}^{2}$

where

$\displaystyle\Gamma_{Q,\phi}:=\left(\int_{\Theta_{0}}\nabla\phi(\theta)\,dQ(\theta)\right)^{\top}\left(\mathcal{I}(Q)+\int_{\Theta_{0}}\mathcal{I}(\theta)\,dQ(\theta)\right)^{-1}\left(\int_{\Theta_{0}}\nabla\phi(\theta)\,dQ(\theta)\right).$

The lower bound holds for any choice of $\phi\in\Phi_{\textrm{ac}}$ and any
“nice” prior $Q$, allowing for further optimization. Here, the surrogate
efficiency is obtained by the van Trees inequality (Gill and Levit, , 1995;
Gassiat and Stoltz, , 2024).

Second, we consider approximating $\psi(t)$ with $\psi(t-h)$ for $h\in\Theta$.
Such a local perturbation is well-defined, as long as $t,t-h\in\Theta_{0}$,
even when the functional $\psi$ is non-smooth. Then for any probability
measure $Q$ over $\mathbb{R}^{d}$, Theorem 10 states that

$\displaystyle\sup_{\theta\in\mathbb{R}^{d}}\,\operatorname{\mathbb{E}}_{P_{\theta}}\|T(X)-\psi(\theta)\|^{2}$
$\displaystyle\qquad\geq\sup_{h\in\mathbb{R}^{d},\,Q}\,\left[\left(\frac{\left\|\int_{\mathbb{R}^{d}}\psi(t)-\psi(t-h)\,dQ\right\|^{2}}{4H^{2}(\mathbb{P}_{0},\mathbb{P}_{h})}\right)^{1/2}-\left(\int_{\mathbb{R}^{d}}\left\|\psi(t)-\psi(t-h)\right\|^{2}\,dQ\right)^{1/2}\right]^{2}_{+}$

where $H^{2}(\mathbb{P}_{0},\mathbb{P}_{h})$ is the Hellinger distance between
two mixture distributions $\mathbb{P}_{0}$ and $\mathbb{P}_{h}$ indexed by
$h\in\mathbb{R}^{d}$ (see (7)). Informally, $\mathbb{P}_{h}$ is a probability
measure “slightly perturbed” from $\mathbb{P}_{0}$ by $h$, which is a mixture
between $P_{0}$ and $Q$. The sharpest minimax lower bound is obtained by
maximizing the difference of functionals $\psi(P_{t})-\psi(P_{t-h})$ while
minimizing the divergence between probability measures. This approach
conceptually aligns with classical methods for deriving minimax lower bounds
such as the Hellinger modulus of continuity (Donoho and Liu, , 1987) or the
general reduction to hypothesis testing. See, for instance, the list of
approaches in Chapter 2 of Tsybakov,  (2008).

The lower bounds given by Theorems 8 and 10 are non-asymptotic and require no
additional assumptions as long as all terms in the expression are well-
defined. In particular, both lower bounds are obtained from the worse Bayes
risk, optimized over the choice of a prior $Q$. This manuscript demonstrates
that the introduction of the prior $Q$ enables us to recover not only the
optimal rate but also the constant. Hence, these bounds possibly provide non-
asymptotic efficiency lower bounds, which generalize well-known asymptotic
results such as the Cramér-Rao and semiparametric efficiency bounds. Theorem
10 in particular holds under the irregularity exhibited through the underlying
statistical model, which heuristically results in the undefined Fisher
information and the failure of the LAM theorem. We address this irregularity
and derive the asymptotic minimax rate for irregular problems. Our conclusions
are consistent with the literature on irregular estimation (Ibragimov and
Has’minskii, , 1981; Shemyakin, , 2014; Lin et al., , 2019) and the asymptotic
theory developed by Hájek and Le Cam (Hájek, , 1970, 1972; Le Cam, , 1972;
Pollard, , 2023).

In summary, we focus on the construction of minimax lower bounds that satisfy
the following criteria: (i) they yield the general minimax risk defined in
equation (1) for an arbitrary $\Theta_{0}$, which can depend on the sample
size or the data-generating distribution; (ii) they require minimal regularity
conditions, allowing for the application to non-smooth functionals, irregular
estimators and irregular statistical models; and (iii) they provide a non-
asymptotic constant, which converges to the asymptotically optimal constant
for regular problems. To the best of our knowledge, there is limited work in
the minimax theory literature that considers all three requirements
simultaneously.

###  1.2 Literature review

There has been a growing interest in investigating the minimax optimality of
non-smooth functional estimation. These problems include the $L_{r}$-norm
estimation in a Gaussian sequence model (Lepski et al., , 1999; Cai and Low, ,
2011; Collier et al., , 2020; Han et al., , 2020), the property estimation of
high-dimensional discrete distributions (Valiant and Valiant, , 2011; Jiao et
al., , 2015; Wu and Yang, , 2019), the hypothesis testing for shape
constraints (Juditsky and Nemirovski, , 2002), the estimation of conditional
variance of a nonparametric regression function (Shen et al., , 2020), many
parameter estimation problems such as the maximum of average potential
outcomes (Fang, , 2014) and conditional average treatment effect (Kennedy et
al., , 2022), among others. For many such problems, minimax optimal rates are
not well understood, let alone the optimal constants.

One popular method for deriving minimax lower bounds involves reducing the
original estimation problem to a set of carefully designed testing problems.
It has been observed that non-smooth functional estimation often requires the
reduction to composite versus composite hypothesis testing, which is known as
the fuzzy hypothesis approach (Ibragimov et al., , 1987; Lepski et al., ,
1999; Nemirovski, , 2000; Tsybakov, , 2008). This reduction often leads to the
study of the divergence metrics between two mixture distributions. In view of
the Neyman-Pearson Lemma, a uniformly most powerful test is based on the
likelihood ratio, which has a natural connection to the total variation
distance between two hypotheses. Minimax lower bounds can then be obtained in
terms of various metrics, including the Hellinger distance, the chi-squared
divergence, the Kullback–Leibler divergence, and others, by invoking a series
of inequalities between metrics such as Le Cam’s and Pinsker’s inequalities.
See, for instance, Section 2.7.4 of Tsybakov,  (2008). However, it is
important to note that in the process of applying different inequalities
between metrics, the precision of the constant may be lost. As a result, the
minimax optimality of non-smooth functional estimation is often considered in
terms of rate, with no specific focus on the constant.

In a slightly different approach, Cai and Low,  (2011) extends the notion of
fuzzy hypotheses to the constrained risk inequality (Brown and Low, , 1996),
deriving a precise constant for the $L_{1}$-norm estimation under a Gaussian
sequence model. In this manuscript, we aim to explore similar extensions of
classical two-point risk lower bounds to mixture distributions in order to
obtain a sharp constant for the minimax lower bounds. Towards this goal, we
refer to the well-established literature on parametric estimation, which has
extensively investigated risk lower bounds with accurate non-asymptotic
constants. Although these results are not typically presented as minimax risk,
they are essential references regardless. Among the commonly used bounds are
the Cramér-Rao bound (Fisher, , 1922; Radhakrishna Rao, , 1945; Cramér, ,
1999), which was improved by Barankin,  (1949), the higher-order analogue
given by Bhattacharyya bound (Bhattacharyya, , 1946), the Hammersley-Chapman-
Robbins bounds (Hammersley, , 1950; Chapman and Robbins, , 1951), and the
Weiss-Weinstein bound (Weiss and Weinstein, , 1985). Each of these bounds
requires varying degrees of regularity conditions and is a variant of the
others, as unified by Weinstein and Weiss,  (1988). They all asymptotically
imply the Cramér-Rao bound under different levels of regularity conditions,
and they offer sharper non-asymptotic constants for different settings.
Notably, all of these bounds, including Cai and Low,  (2011), define the lower
bounds in terms of the likelihood ratio between local models, or some
analogous objects. This can be understood as a variant of the chi-squared
divergence between local models, which requires strong regularity assumptions
for its asymptotic behavior. This will be the focus of Section 2.1 of this
manuscript.

The lower bound presented here is also closely related to the technique of
obtaining a sharp minimax lower bound by exploring the least-favorable priors
of the Bayes risk such as the van Trees inequality (Van Trees, , 1968).
Traditionally, the literature on the van Trees inequality (Van Trees, , 1968;
Gill and Levit, , 1995; Jupp, , 2010) requires strong regularity assumptions
to guarantee the limiting behavior of the chi-squared divergence. Taking
inspiration from the recent development by Gassiat and Stoltz,  (2024), we
provide extensions of the van Trees inequality for non-smooth functionals
based on the Hellinger distance. The asymptotic behavior of the Hellinger
distance requires far weaker regularity conditions compared to the analogous
result under the chi-squared divergence (Pollard, , 2023). This property has
gathered a growing interest in the use of the Hellinger distance as the
desirable metric for studying minimax lower bounds (Ibragimov and Has’minskii,
, 1981; Donoho and Liu, , 1987; Chen, , 1997; Shemyakin, , 2014; Lin et al., ,
2019; Pollard, , 2023). Our lower bound is consistent with the message of
these proponents.

Finally, we discuss connections between the presented lower bounds and the
classical result in functional estimation, namely the modulus of continuity,
which was investigated by Donoho and Liu,  (1987). As a consequence of the new
minimax lower bounds for non-smooth functional estimation, we resolve one of
the open problems since Donoho and Liu,  (1987) to characterize the non-
asymptotic minimax lower bound that implies a precise limiting constant. For
linear functional estimation, the modulus of continuity has been analyzed with
an attempt to establish non-asymptotic efficiency theory (Mou et al., , 2022).
Our result can be seen as one of the first steps towards understanding non-
asymptotics for non-smooth functional estimation. The modulus of continuity is
also considered in the context of impossibility results, often known as ill-
posedness in econometrics literature (Pötscher, , 2002; Forchini and Hillier,
, 2005). The presented results generalize their asymptotic statements to the
non-asymptotic context.

#### Organization.

The remaining manuscript is organized as follows: Section 2 provides necessary
backgrounds and defines several existing lower bounds that largely inspired
this work. Section 3 presents new minimax lower bounds. Section 4 investigates
asymptotic properties of the proposed lower bounds with particular attention
to the preservation of sharp constants. Section 5 presents the application to
several estimation problems in the presence of non-smoothness or irregularity
and Section 6 provides a visual comparison of the new non-asymptotic lower
bounds to upper bounds exhibited by different estimators. Finally, concluding
remarks and a discussion of open problems are provided in Section 7.

#### Notation.

Throughout the manuscript, we adopt the following convention for notation. We
denote by $\|\cdot\|$ a general vector norm. Given $x\in\mathbb{R}$, we write
$[x]_{+}=\max(x,0)$. For $x\in\mathbb{R}^{d}$, $\|x\|_{2}$ denotes the
Euclidean norm in $\mathbb{R}^{d}$. For a univariate function $f$ from
$S\subseteq\mathbb{R}$ to $\mathbb{R}$, the supremum norm is defined as
$\|f\|_{\infty}=\sup_{x\in S}|f(x)|$. The open $\mathbb{R}^{d}$-ball centered
at $y_{0}\in\mathbb{R}^{d}$ with radius $\delta>0$ is denoted as
$B(y_{0},\delta):=\\{y\,:\|y_{0}-y\|_{2}<\delta\\}$. The unit sphere in
$\mathbb{R}^{d}$ is denoted by
$\mathbb{S}^{d-1}:=\\{u\in\mathbb{R}^{d}:\|u\|_{2}=1\\}$. We let
$\operatorname{\mathbb{E}}_{P}$ represent the expectation under $P$, whereas
$\operatorname{\mathbb{E}}_{\theta}$ denotes the expectation under
$P_{\theta}$. Furthermore, $L_{2}(P)$ is the set of $P$-measurable functions
that satisfy the condition

$L_{2}(P):=\left\\{f:\mathcal{X}\mapsto\mathbb{R}^{k}\,\bigg{|}\,\int\|f(x)\|_{2}^{2}\,dP(x)<\infty\right\\}.$

##  2 Setup and background

Throughout this section, we consider probability measures defined on a shared
measurable space $(\mathcal{X},\mathcal{A})$ with a $\sigma$-finite measure
$\nu$. We assume that any probability measure $P$ we consider is absolutely
continuous with respect to $\nu$ and has a well-defined density function,
denoted by $dP\equiv dP/d\nu$. We omit the specification of the base measure
$\nu$ when it is clear from the context. Suppose we observe $X$ from an
unknown distribution $P_{0}$, belonging to a possibly nonparametric model
$\mathcal{P}$. The parametric submodel
$\mathcal{P}_{\Theta}:=\\{P_{\theta}:\theta\in\Theta\\}\subset\mathcal{P}$ is
defined as a set of probability measures indexed by a parameter space
$\Theta\subseteq\mathbb{R}^{d}$, which contains the data-generating
distribution $P_{0}$. Without loss of generality, we assume that the true
parameter corresponds to $\theta=0$ in $\Theta$.

The local behavior of a path $t\mapsto P_{t}\in\mathcal{P}_{\Theta}$ as it
approaches $P_{0}$ is often of interest. A local path is said to be Hellinger
differentible at $t=0$ (Pollard, , 2023) if there exists a measurable vector-
valued function $\dot{\xi}_{0}:\mathcal{X}\mapsto\mathbb{R}^{d}$ that is
square integrable with respect to the $\sigma$-finite measure $\nu$, and as
$\|t\|_{2}\longrightarrow 0$,

$\displaystyle\int\left[dP_{t}^{1/2}-dP_{0}^{1/2}-t^{\top}\dot{\xi}_{0}\right]^{2}\,d\nu=o(\|t\|_{2}^{2}).$

The Fisher information matrix of $P_{t}$ at $t=0$ is defined as

$\displaystyle\mathcal{I}(0):=4\int\dot{\xi}_{0}\,\dot{\xi}_{0}^{\top}\,d\nu$
(3)

under the Hellinger differentiability. We also define a slightly stronger but
more commonly used notion of differentiability for parametric paths, called
quadratic mean differentiability (QMD). A local path is differentiable in
quadratic mean, or QMD, at $t=0$ if there exists a $P_{0}$-square integrable
vector-valued function $g_{0}:\mathcal{X}\mapsto\mathbb{R}^{d}$ that
satisfies, as $\|t\|_{2}\longrightarrow 0$,

$\displaystyle\int\left[dP_{t}^{1/2}-dP_{0}^{1/2}-\frac{1}{2}t^{\top}g_{0}\,dP_{0}^{1/2}\right]^{2}\,d\nu=o(\|t\|_{2}^{2}).$
(4)

The function $g_{0}$ is commonly referred to as the score function of the path
at $t=0$. When $P_{t}$ is QMD at $t=0$, it implies the Hellinger
differentiability with $\dot{\xi}_{0}=\frac{1}{2}g_{0}\,dP_{0}^{1/2}$ (See
Theorem 20 of Pollard,  (2023)) however the converse is not true. The
corresponding Fisher information under the QMD assumption is given by

$\mathcal{I}(0):=\int g_{0}\,g_{0}^{\top}\,dP_{0}.$

As the Hellinger differentiability is a weaker condition, we only use the QMD
assumption when necessary. Both definitions allow for the Fisher information
to exist for distributions with pointwise non-differentiable densities, which
is a crucial aspect of the asymptotic theory developed by Hàjek and Le Cam.
For instance, consider the double-exponential density
$t\mapsto\frac{1}{2}\exp(-|t-\theta|)$ with a parameter $\theta\in\Theta$.
This density function is not differentiable at $t=\theta$, yet its Fisher
information matrix exists.

When $\Theta\subseteq\mathbb{R}^{d}$ with some base measure $\mu$, we define a
location family induced by a probability measure $Q$ on $\Theta$ as
$\\{Q(\cdot-h):h\in\Theta\\}$. If we assume that $Q$ has an absolutely
continuous density function $q$ with an almost-everywhere derivative $\nabla
q$, the corresponding location family is differentiable in quadratic mean with
the Fisher information given by

$\mathcal{I}(Q)=\int\frac{\nabla q(t)\nabla
q(t)^{\top}}{q(t)}I(q(t)>0)\,d\mu.$ (5)

See Example 7.8 of Van der Vaart,  (2000). The Fisher information
$\mathcal{I}(Q)$ does not depend on the location parameter $h$, making this
notation a convenient reference to the Fisher information of the location
families. Finally, we impose the following regularity conditions on $Q$, under
which the priors are called “nice”.

######  Definition 1 (Nice priors).

Given an absolutely continuous mapping $\phi:\Theta_{0}\to\mathbb{R}^{k}$ and
a vector norm $\|\cdot\|:\mathbb{R}^{k}\mapsto\mathbb{R}_{+}$, a probability
measure $Q$ on $\Theta_{0}$ is “nice” if it satisfies the following
conditions:

  1. (1)

It has a Lebesgue density $q$ and $q$ is an absolutely continuous function
with a positive definite Fisher information $\mathcal{I}(Q)$,

  2. (2)

Both $\|\phi\|$ and $\|\nabla\phi\|$ is $Q$-integrable and
$\int_{\Theta_{0}}\,\mathrm{tr}\,(\mathcal{I}(t))\,dQ<\infty$, and

  3. (3)

It holds that $q(\theta)\to 0$ as $\theta$ approaches any boundary point of
$\Theta_{0}$ with finite norm along some canonical direction.

We note that this property of priors is defined for each particular functional
$\phi$.

###  2.1 The local behavior of divergence metrics

This manuscript primarily focuses on two divergence metrics, namely the chi-
squared divergence and the Hellinger distance. Consider two probability
measures, $P_{0}$ and $P_{1}$, defined on a common measurable space. Further,
assume that there exist densities $dP_{0}=dP_{0}/d\nu$ and
$dP_{1}=dP_{1}/d\nu$ with respect to a common $\sigma$-finite measure $\nu$.
The chi-squared divergence is defined as

$\displaystyle\chi^{2}(P_{1}\|P_{0}):=\begin{cases}\int\left(dP_{1}/dP_{0}-1\right)^{2}\,dP_{0}&\mbox{if}\quad
P_{1}\textrm{ is dominated by }P_{0}\\\ \infty&\mbox{otherwise}\end{cases}$

and the Hellinger distance is defined as

$\displaystyle
H^{2}(P_{0},P_{1}):=\int\left(dP_{0}^{1/2}-dP_{1}^{1/2}\right)^{2}.$

Below, we introduce conditions that imply certain limiting behaviors of these
metrics over a parametric path as it passes through $P_{0}$. We first extend
the standard notion of absolute continuity of a univariate function to a
multivariate function as follows:

######  Definition 2 (Multivariate absolute continuity).

A function $\omega:\mathbb{R}^{d}\mapsto\mathbb{R}$ is absolutely continuous
over an open $\mathbb{R}^{d}$-ball $B(\theta,\delta)$ if for all
$u\in\mathbb{S}^{d-1}$, the induced univariate function
$t\mapsto\omega(\theta+tu)$ is absolutely continuous over $0<t<\delta$.

The concept of multivariate absolute continuity has been explored in various
real analysis literature. For instance, Malỳ,  (1999) and Hencl,  (2004)
extend the classical $\delta$-$\varepsilon$ definition of absolute continuity
by considering the oscillation of the functions over $d$-dimensional balls.
Additionally, Šremr,  (2010) presents a similar extension using
$d$-dimensional hyper-cubes. We do not claim that Definition 2 is the most
general definition of multivariate absolutely continuous functions for our
purpose. Definition 2 is introduced solely for our complementary result
(Theorem 9), and it is not a necessary condition for our main results (Theorem
7 or Theorem 10).

We then introduce the following regularity conditions:

  1. (A1)

There exists $\delta>0$ such that

    1. (a)

for $\nu$-almost everywhere, the mapping $t\mapsto dP_{t}$ is absolutely
continuous over an open $\mathbb{R}^{d}$-ball $B([0],\delta)$ with the
gradient $t\mapsto u^{\top}\dot{\rho}_{t,u}$ for each $u\in\mathbb{S}^{d-1}$,

    2. (b)

for all $u\in\mathbb{S}^{d-1}$, the gradient is continuous such that
$\lim_{t\longrightarrow 0}u^{\top}\dot{\rho}_{t,u}=u^{\top}\dot{\rho}_{0}$ for
$\nu$-almost everywhere, and

    3. (c)

for all $0<t<\delta$ and $u\in\mathbb{S}^{d-1}$, $dP_{0}(x)=0$ implies
$u^{\top}\dot{\rho}_{t,u}(x)=0$, and

$\displaystyle\int\sup_{0<t_{1},t_{2}<\delta}\frac{\dot{\rho}_{t_{1},u}\dot{\rho}_{t_{2},u}^{\top}}{dP_{0}}\,d\nu<\infty.$

With this regularity condition in place, we obtain the following local
expansion of the chi-squared divergence:

######  Lemma 1 (Theorem 7.20 of Polyanskiy and Wu,  (2022)).

Assuming that the local path $\\{P_{t}:t\in\Theta\\}$ satisfies (A1) and
$t\mapsto\mathcal{I}(t)$ is continuous at $t=0$, then as
$\|t\|_{2}\longrightarrow 0$,

$\displaystyle\chi^{2}(P_{t}\|P_{0})=t^{\top}\mathcal{I}(0)t+o(\|t\|_{2}^{2}).$

The regularity condition (A1) is stronger than the Hellinger
differentiability. Gassiat and Stoltz,  (2024) shows that an analogous (but
weaker) condition to (A1) implies the Hellinger differentiability.
Furthermore, Example 7.1 of Polyanskiy and Wu,  (2022) provides a concrete
example where (A1) fails while Hellinger differentiability still holds. The
local expansion of the chi-squared divergence can hold under the following
weaker conditions:

  1. (A2)

There exits $\delta>0$ such that

    1. (a)

for $\nu$-almost everywhere, the mapping $t\mapsto dP^{1/2}_{t}$ is absolutely
continuous over an open $\mathbb{R}^{d}$-ball $B([0],\delta)$ with the
gradient $t\mapsto u^{\top}\dot{\gamma}_{t,u}$ for each
$u\in\mathbb{S}^{d-1}$,

    2. (b)

for all $u\in\mathbb{S}^{d-1}$, the gradient is continuous such that
$\lim_{t\longrightarrow 0}u^{\top}\dot{\gamma}_{t,u}=u^{\top}\dot{\gamma}_{0}$
for $\nu$-almost everywhere, and

$\displaystyle\int\sup_{0<t_{1},t_{2}<\delta}\dot{\gamma}_{t_{1},u}\dot{\gamma}_{t_{2},u}^{\top}\,d\nu<\infty.$

Roughly speaking, assumption (A1) pertains to the smoothness of density
functions, whereas assumption (A2) pertains to a similar smoothness of the
square roots of density functions. In fact, we can demonstrate that (A1)
implies (A2) (the proof is provided in Section H of Supplementary Material).
However, the converse is not true. For instance, location families with
compact support fail to satisfy (A1)(c), while they may satisfy (A2) under
certain conditions. See Example 7.1 of Polyanskiy and Wu,  (2022) for more
details. Under (A2), the following result holds:

######  Lemma 2 (Theorem 7.21 of Polyanskiy and Wu,  (2022)).

Assuming that the local path $\\{P_{t}:t\in\Theta\\}$ satisfies (A2) and
$t\mapsto\mathcal{I}(t)$ is continuous at $t=0$, then as
$\|t\|_{2}\longrightarrow 0$,

$\displaystyle\chi^{2}(P_{t}\|\lambda P_{0}+(1-\lambda)P_{t})$
$\displaystyle=(1-\lambda)^{2}t^{\top}\left(\mathcal{I}(0)-\frac{1-4\lambda}{4\lambda}\mathcal{I}^{\dagger}(0)\right)t+o(\|t\|_{2}^{2})\quad\mbox{for
all }\lambda\in(0,1)$

where
$\mathcal{I}^{\dagger}(0):=4\int\dot{\gamma}_{0}\,\dot{\gamma}_{0}^{\top}I(dP_{0}=0)\,d\nu$
is known as the Fisher defect.

######  Remark 1 (On the Fisher defect $\mathcal{I}^{\dagger}$).

If the parameter $t=0$ is an interior point of the parameter space $\Theta$,
then $\dot{\gamma}_{0}=0$ for $\nu$-almost all $x$ in $\\{x:dP_{0}(x)=0\\}$.
Consequently, the Fisher defect must be zero. However, even for a Hellinger
differentiable statistical model, it is still possible for the Fisher defect
to be non-zero on the boundary. An illustrative example is the Bernoulli
distribution with parameter $p^{2}$ at $p=0$. Example 7.2 of Polyanskiy and
Wu,  (2022) provides a formal derivation, and Example 18 of Pollard,  (2023)
offers an additional example.

As shown by Lemma 23 in Supplementary Material, the regularity condition (A2)
implies the Hellinger differentiability with $\dot{\gamma}_{0}=\dot{\xi}_{0}$
for $\nu$-almost everywhere and hence two definitions of the Fisher
information coincide under (A2). In contrast, building directly from the
Hellinger differentiability, we obtain the following result:

######  Lemma 3.

Assuming that the local path $\\{P_{t}:t\in\Theta\\}$ is Hellinger
differentiable and $t\mapsto\mathcal{I}(t)$ is continuous at $t=0$, then as
$\|t\|_{2}\longrightarrow 0$,

$\displaystyle H^{2}(P_{t},P_{0})$
$\displaystyle=\frac{1}{4}t^{\top}\mathcal{I}(0)t+o(\|t\|_{2}^{2}).$

Slightly generalized statements of Lemmas 1–3 are provided with proofs as
Lemmas 19–21 in Supplementary Material. The main takeaway of this section is
as follows: The local behavior of the Hellinger distance is easily understood
under weaker conditions, while an analogous result under the chi-squared
divergence requires more unpleasant regularity conditions. This is one of many
reasons why the asymptotic theory according to Hàjek and Le Cam promotes the
square roots of density functions as the primary object to investigate.

###  2.2 Minimax lower bounds via parametric paths

In this section, we introduce the local asymptotic minimax (LAM) theorem, a
fundamental result in the efficiency theory. We then provide several non-
asymptotic minimax lower bounds that asymptotically imply the best constant in
view of the LAM theorem. For clarity and ease of illustration, we focus on a
real-valued function $\psi:\Theta\mapsto\mathbb{R}$ where
$\Theta\subseteq\mathbb{R}^{d}$ under parametric models. Although the result
applies to nonparametric models, we defer a detailed discussion to Section
4.2. We assume that $X_{1},\dots,X_{n}\in\mathcal{X}^{n}$ are independent and
identically distributed (IID) observations from
$P_{\theta_{0}}\in\\{P_{\theta}:\theta\in\Theta\\}$ and $t\mapsto P_{t}$ is
Hellinger differentiable at all $\theta\in\Theta$. The joint distribution of
$n$ IID observations is denoted by $P_{\theta_{0}}^{n}$. We now state the LAM
theorem.

######  Theorem 4 (Local asymptotic minimax theorem (Hájek, , 1970, 1972; Le
Cam, , 1972)).

Assuming that the mappings $t\mapsto\psi(t)$ and $t\mapsto\mathcal{I}(t)$ are
continuously differentiable at $t=\theta_{0}$, then for any measurable
function $T:\mathcal{X}^{n}\mapsto\mathbb{R}$,

$\displaystyle\liminf_{c\longrightarrow\infty}\,\liminf_{n\longrightarrow\infty}\,\inf_{T}\,\sup_{\|\theta-\theta_{0}\|<cn^{-1/2}}\,n\operatorname{\mathbb{E}}_{P_{\theta}^{n}}|T(X)-\psi(\theta)|^{2}\geq\nabla\psi(\theta_{0})^{\top}\mathcal{I}(\theta_{0})^{-1}\nabla\psi(\theta_{0}).$
(6)

When the risk of an estimator matches the constant provided by the LAM
theorem, the estimation becomes no longer improvable asymptotically. This, for
instance, suggests the asymptotic efficiency of maximum likelihood estimators
(MLEs) for this specific estimation task. This feature of the precise constant
sets it apart from the minimax theory through the testing reduction, which
typically claims the optimality in terms of the rates alone. It is essential
to note that the first supremum over a neighborhood around $\theta_{0}$ is a
crucial feature that cannot be removed. This supremum over a small
neighborhood eliminates any estimator that performs exceptionally well on a
Lebesgue measure-zero set, known as superefficient estimators. Without the
supremum, the LAM theorem only applies to a family of regular estimators,
which is restrictive in the context of non-smooth functional estimation
(Hirano and Porter, , 2012). See Section 2.3 of Fang,  (2014) for further
discussion on the motivation behind the local asymptotic minimax risk.

One of our results is particularly inspired by the Hammersley-Chapman-Robbins
(HCR) bound, initially introduced by Hammersley,  (1950) and Chapman and
Robbins,  (1951). Although we present the results within the context of
parametric estimation, it is possible to establish analogous results for
nonparametric functionals using the argument to be discussed in Section 4.2.
We now present the HCR bound:

######  Theorem 5 (HCR bound (Hammersley, , 1950; Chapman and Robbins, ,
1951)).

Let $\psi:\Theta\mapsto\mathbb{R}$ be a real-valued mapping for
$\Theta\subseteq\mathbb{R}^{d}$.

$\displaystyle\inf_{T:\mathrm{unbiased}}\,\sup_{\theta\in\Theta}\,\operatorname{\mathbb{E}}_{\theta}|T(X)-\psi(\theta)|^{2}\geq\sup_{\theta_{0},\theta_{1}\in\Theta}\,\frac{|\psi(\theta_{1})-\psi(\theta_{0})|^{2}}{\chi^{2}\left(P_{\theta_{1}}\|P_{\theta_{0}}\right)}.$

The HCR bound is an elegant lower bound as it expresses the separation between
functions and the divergence between two underlying probability measures in a
concise manner. Assuming that $P_{\theta_{0}}$ is a fixed data generating
distribution and taking the limit as $\theta_{1}\rightarrow\theta_{0}$, the
HCR lower bound implies the Cramér–Rao bound under the regularity conditions
required for Lemma 1. The regularity conditions can be weakened to the
Hellinger differentiability by deriving an analogous lower bound in terms of
the Hellinger distance (see Simons and Woodroofe,  (1983) and Exercise VI.5 of
Polyanskiy and Wu,  (2022)). However, one major limitation of the HCR bound is
that it only holds for unbiased estimators, which can be overly restrictive
when analyzing minimax risks. In particular, if the function $\psi$ is non-
differentiable, no sequences of unbiased estimators exist (Hirano and Porter,
, 2012), indicating that the HCR lower bound cannot be directly applied in our
context.

The van Trees inequality differs from other approaches as it considers the
minimax lower bound in terms of the worst Bayes risk. It achieves a sharp
constant by seeking the supremum over all possible priors, known as the least-
favorable prior. Let $\Theta\subseteq\mathbb{R}^{d}$ and $Q$ be a probability
measure defined on $\Theta$ with a density function $dQ$. Further assuming
that $Q$ is “nice”, we state the following:

######  Theorem 6 (The van Trees inequality (Gassiat and Stoltz, , 2024)).

Suppose $\\{P_{\theta}:\theta\in\Theta\\}$ for $\Theta\subseteq\mathbb{R}^{d}$
is Hellinger differentiable for all $\Theta$ and the mapping $\psi$ is
absolutely continuous on $\Theta$. Then for any measurable function $T$ and a
“nice” prior distribution $Q$ (see Definition 1) ,

$\displaystyle\inf_{T}\,\sup_{\theta\in\Theta}\,\operatorname{\mathbb{E}}_{\theta}|T(X)-\psi(\theta)|^{2}\geq\left(\int_{\Theta}\nabla\psi(\theta)\,dQ(\theta)\right)^{\top}\left(\mathcal{I}(Q)+\int_{\Theta}\mathcal{I}(\theta)\,dQ(\theta)\right)^{-1}\left(\int_{\Theta}\nabla\psi(\theta)\,dQ(\theta)\right)$

where $\mathcal{I}(Q)$ is the Fisher information of the location family
induced by the prior $Q$ defined in (5).

Theorem 6 is stated under the Hellinger differentiability, and this is due to
the recent refinement by Gassiat and Stoltz,  (2024). The classical literature
on the van Trees inequality (Van Trees, , 1968; Gill and Levit, , 1995; Jupp,
, 2010) requires stronger regularity conditions to assert the local behavior
of the chi-squared divergence, as shown in Lemma 1. The van Trees inequality
is particularly desirable since it does not confine the choice of estimators
to unbiased estimators. Furthermore, it has been shown that by assuming $\psi$
is continuously differentiable at $\theta_{0}$ and selecting a prior
distribution $Q$ that concentrates at the true parameter $\theta_{0}$ with the
rate $n^{-1/2}$, the exact constant for the LAM theorem can be obtained
(Gassiat and Stoltz, , 2024). However, the van Trees inequality gives a
trivial lower bound for an irregular statistical model where the Fisher
information is undefined. Our main results, presented below, aim to remove
these limitations of the existing results.

##  3 General minimax lower bounds under weaker regularities

This section provides the main results of this manuscript. Section 3.1
provides a general minimax lower bound via smooth approximation. Sections 3.2
and 3.3 provide two extensions of the van Trees inequality that do not require
the differentiability of functionals. We first present our results for
parametric models with $\Theta=\mathbb{R}^{d}$ for ease of exposition, with an
extension to more general sets $\Theta\subseteq\mathbb{R}^{d}$ discussed in
Section 3.4. These results hold for nonparametric functionals and are achieved
through the standard least-favorable parametric paths argument from
semiparametric statistics, which is postponed to Section 4.2. Proofs of all
theorems in this section are provided in Supplementary Material.

###  3.1 Lower bound based on the approximating functionals

When the target functional $\psi$ does not possess certain properties, such as
smoothness, it is often useful to approximate $\psi$ with an alternative
functional $\phi$ at the expense of the bias introduced by approximation. This
is a common approach for non-smooth functional estimation where the target
functional is first smoothed by, for instance, convolution. We formalize this
idea in our first main result.

Suppose we jointly observe a random sequence of observations $X$ from an
unknown distribution $P_{\theta}$, which belongs to a statistical model
$\mathcal{P}:=\\{P_{t}:t\in\Theta\\}$ defined on a measurable space
$(\mathcal{X},\mathcal{A})$ with each possessing a density with respect to
$\sigma$-finite measure $\nu$. Let $\psi:\mathcal{P}\mapsto\mathbb{R}^{k}$
denote a vector-valued functional where the estimand of interest is the
evaluation of the functional at the population parameter
$\psi(P_{\theta_{0}})$. We state the following as a consequence of the reverse
triangle inequality:

######  Lemma 7.

Given a measure space $(\Theta,\mathcal{T},\mu)$, let
$\Theta_{0}\subseteq\Theta$ be any subset of $\Theta$ and let
$\mathcal{Q}\equiv\mathcal{Q}(\Theta_{0})$ be a collection of probability
measures on $\Theta_{0}$ equipped with a density function with respect to the
base measure $\mu$, denoted as $dQ:=dQ/d\mu$. Let $\Phi$ be any collection of
functionals $\Phi:=\\{\phi:\mathcal{P}\mapsto\mathbb{R}^{k}\\}$. For any
measurable function $T:\mathcal{X}\mapsto\mathbb{R}^{k}$ and vector norm
$\|\cdot\|:\mathbb{R}^{k}\mapsto\mathbb{R}_{+}$, it holds that

$\displaystyle\sup_{\theta\in\Theta_{0}}\,\operatorname{\mathbb{E}}_{\theta}\|T(X)-\psi(P_{\theta})\|^{2}$
$\displaystyle\qquad\geq\sup_{\phi\in\Phi,\,Q\in\mathcal{Q}}\,\left[\left(\int\,\operatorname{\mathbb{E}}_{\theta}\left\|T(X)-\phi(P_{\theta})\right\|^{2}\,dQ\right)^{1/2}-\left(\int\|\psi(\theta)-\phi(\theta)\|^{2}dQ(\theta)\right)^{1/2}\right]_{+}^{2}.$

This result states that the minimax lower bound for any functional estimation
can be expressed as the trade-off between the Bayes risk of estimating $\phi$
and the approximation error of $\psi$ by $\phi$. For a real-valued functional,
the approximation error is expressed as $L_{2}(Q)$-norm, hence the functional
$\phi$ needs not to approximate $\psi$ uniformly.

We present the application of this result by considering the projection of
$\psi$ onto the space of absolutely continuous functions. The Bayes risk for
absolutely continuous functionals can be characterized by the van Trees
inequality. We define a collection of probability measures
$\mathcal{Q}^{\dagger}$ supported on $\Theta_{0}$, satisfying Definition 1.
Note that $\mathcal{Q}^{\dagger}$ depends on $\phi$. We then state the
following result:

######  Theorem 8.

Suppose $\\{P_{\theta}:\theta\in\Theta_{0}\\}$ for any open subset
$\Theta_{0}\subseteq\mathbb{R}^{d}$ is Hellinger differentiable for all
$\Theta_{0}$ and $\phi:\mathbb{R}^{d}\mapsto\mathbb{R}^{k}$ is an absolutely
continuous vector-valued functions on $\Theta_{0}$ with almost-everywhere
derivative $\nabla\phi$. Then for any measurable function
$T:\mathcal{X}\mapsto\mathbb{R}^{k}$, vector norm
$\|\cdot\|:\mathbb{R}^{k}\mapsto\mathbb{R}_{+}$, its dual $\|\cdot\|_{*}$, and
$Q\in\mathcal{Q}^{\dagger}$,

$\displaystyle\inf_{T}\,\sup_{\theta\in\Theta_{0}}\,\operatorname{\mathbb{E}}_{\theta}\|T(X)-\psi(\theta)\|^{2}\geq\left[\sup_{\|u\|_{*}\leq
1}\,\|\Gamma^{1/2}_{Q,\phi}u\|-\left(\int\|\psi(\theta)-\phi(\theta)\|^{2}dQ(\theta)\right)^{1/2}\right]_{+}^{2}$

where

$\displaystyle\Gamma_{Q,\phi}:=\left(\int_{\Theta_{0}}\nabla\phi(t)\,dQ\right)^{\top}\left(\mathcal{I}(Q)+\int_{\Theta_{0}}\mathcal{I}(t)\,dQ\right)^{-1}\left(\int_{\Theta_{0}}\nabla\phi(t)\,dQ\right).$

Theorem 8 is a straightforward application of the multivariate van Trees
inequality (Theorem 12 of Gassiat and Stoltz,  (2024)) in conjunction with
Lemma 7. Here, $\mathcal{I}(t)$ is the Fisher information associated with the
data-generating distribution $P_{t}$. It is well-known that the Fisher
information for an $n$-fold product measure is given by $n\mathcal{I}(t)$
where $\mathcal{I}(t)$ is the Fisher information associated with a single
observation under $P_{t}$. Hence, under $n$ IID observations from $P_{t}$, the
theorem above yields the identical lower bound but replaces $\Gamma_{Q,\phi}$
with

$\displaystyle\left(\int_{\Theta_{0}}\nabla\phi(t)\,dQ\right)^{\top}\left(\mathcal{I}(Q)+n\int_{\Theta_{0}}\mathcal{I}(t)\,dQ\right)^{-1}\left(\int_{\Theta_{0}}\nabla\phi(t)\,dQ\right).$

###  3.2 Lower bound based on the chi-squared divergence

We now move on to the direct extension of the HCR inequality without the
earlier projection argument. Throughout, we assume that
$\Theta=\mathbb{R}^{d}$. To establish the extension of the van Trees
inequality, we define two probability measures on a product space
$(\mathcal{X}\times\mathbb{R}^{d})$. Specifically, we let

$\displaystyle
d\mathbb{P}_{0}(x,\theta):=dP_{\theta}(x)\,dQ(\theta)\quad\mbox{and}\quad
d\mathbb{P}_{h}(x,\theta):=dP_{\theta+h}(x)\,dQ(\theta+h).$ (7)

The measure under the translation $Q(\theta+h)$ is well-defined for all
$h\in\mathbb{R}^{d}$. We now provide an extension of the HCR inequality.

######  Theorem 9 (Mixture HCR inequality with the chi-squared divergence).

For any probability measure $Q$ on $\mathbb{R}^{d}$, define

$\displaystyle
A_{\lambda,\psi,Q,h}:=\frac{\left\|\int_{\mathbb{R}^{d}}\psi(t)-\psi(t-h)\,dQ(t)\right\|^{2}}{\chi^{2}(\mathbb{P}_{h}\|\lambda\mathbb{P}_{h}+(1-\lambda)\mathbb{P}_{0})}\quad\textrm{and}\quad
B_{\psi,Q,h}:=\int_{\mathbb{R}^{d}}\|\psi(t)-\psi(t-h)\|^{2}\,dQ(t).$

Then for any measurable function $T:\mathcal{X}\mapsto\mathbb{R}^{k}$, vector
norm $\|\cdot\|:\mathbb{R}^{k}\mapsto\mathbb{R}_{+}$, and $\lambda\in[0,1]$,

$\displaystyle\inf_{T}\,\sup_{\theta\in\mathbb{R}^{d}}\,\operatorname{\mathbb{E}}_{\theta}\|T(X)-\psi(\theta)\|^{2}$
$\displaystyle\geq\sup_{h\in\mathbb{R}^{d}}\,\left[\sqrt{(1-\lambda)^{2}A_{\lambda,\psi,Q,h}}-\sqrt{\lambda
B_{\psi,Q,h}}\right]_{+}^{2}.$

The above result holds for any choice of $Q$ and $\lambda$ allowing for the
derivation of the sharpest constant by taking supremum over them. The
additional parameter $\lambda$ is introduced to prevent a trivial constant of
zero under the chi-squared divergence. This might happen, for instance, when
the statistical model $\mathcal{P}$ exhibits some irregularity. Under
additional regularity conditions, the theorem can be significantly simplified
by choosing $\lambda=0$. This provides a non-trivial lower bound when the
necessary conditions for Lemma 1 hold.

######  Corollary 9.1.

When $\lambda=0$, Theorem 9 implies that

$\displaystyle\inf_{T}\,\sup_{\theta\in\mathbb{R}^{d}}\,\operatorname{\mathbb{E}}_{\theta}\|T(X)-\psi(\theta)\|^{2}~{}\geq~{}\sup_{h\in\mathbb{R}^{d}}\,\frac{\left\|\int_{\mathbb{R}^{d}}\psi(t)-\psi(t-h)\,dQ(t)\right\|^{2}}{\chi^{2}(\mathbb{P}_{h}\|\mathbb{P}_{0})}.$
(8)

This inequality can be seen as the modification of the classical HCR lower
bound from a two-point risk to mixture distributions $\mathbb{P}_{h}$ and
$\mathbb{P}_{0}$. We now provide a heuristic argument that Theorem 9 implies
the van Trees inequality as $\|h\|_{2}\longrightarrow 0$. Assuming the
conditions for Lemma 1 hold and for a suitably selected prior $Q$, the
following expansion of the chi-squared divergence can be obtained:

$\displaystyle\chi^{2}(\mathbb{P}_{h}\|\mathbb{P}_{0})=\chi^{2}(Q_{h}\|Q)+\int_{\mathbb{R}^{d}}\chi^{2}(P_{t+h}\|P_{t})\,\frac{dQ_{h}^{2}}{dQ}=h^{\top}\left(\mathcal{I}(Q)+\int_{\mathbb{R}^{d}}\mathcal{I}(t)\,dQ\right)h+o(\|h\|_{2}^{2})$
(9)

provided that we can exchange the limit and the integral under the dominated
convergence theorem. Hence, if $\psi$ is a continuously differentiable
function, the inequality (8) and the limit of its denominator given by (9)
together imply the van Trees inequality as $\|h\|_{2}\longrightarrow 0$. A
similar result holds for $\lambda>0$ as an application of Lemma 2 assuming
that the Fisher defect is zero. As discussed earlier, the local behavior of
the chi-squared divergence requires stronger assumptions than the Hellinger
distance. The additional parameter $\lambda\in[0,1]$ can be removed under a
similar result based on the Hellinger distance. Before presenting the
corresponding result, we provide a short discussion on the role of $\lambda$.

######  Remark 2 (The role of the mixing weight $\lambda$).

The mixing weight $\lambda$ is introduced to take advantage of Lemma 2, which
states the convergence of the chi-squared divergence under the Hellinger
differentiability. It also prevents a trivial lower bound of zero when the
denominator of the lower bound (8) diverge, in other words, $P_{\theta}$ and
$P_{\theta+h}$ are not absolutely continuous. This happens, for instance,
considering any location family induced by a density function with compact
support. By choosing $\lambda>0$, $P_{\theta}$ is absolutely continuous with
respect to $(1-\lambda)P_{\theta}+\lambda P_{\theta+h}$ and thus, it prevents
the chi-squared divergence from diverging. If the local expansion of the chi-
squared divergence is characterized by (9), we can always take $\lambda=0$.

###  3.3 Lower bound based on the Hellinger distance

This section presents our second main result: the mixture extension of the HCR
bound under the Hellinger distance. This extension is desirable since it does
not require additional regularity conditions, unlike the analogous statement
under the chi-squared divergence. Additionally, we discuss the connections
between the presented result and the classical minimax lower bound in terms of
the Hellinger distance by Donoho and Liu,  (1987), which has been frequently
considered for non-smooth functionals, irregular estimation, or non-asymptotic
minimax lower bound. Similar to Theorem 9, we first introduce the result for
$\Theta=\mathbb{R}^{d}$ and then discuss more generalized settings in the
proceeding section.

######  Theorem 10 (Mixture HCR inequality with the Hellinger distance).

For any probability measure $Q$ on $\mathbb{R}^{d}$, define

$\displaystyle
A_{\psi,Q,h}:=\frac{\left\|\int_{\mathbb{R}^{d}}\psi(t)-\psi(t-h)\,dQ(t)\right\|^{2}}{4H^{2}(\mathbb{P}_{0},\mathbb{P}_{h})}\quad\textrm{and}\quad
B_{\psi,Q,h}:=\int_{\mathbb{R}^{d}}\|\psi(t)-\psi(t-h)\|^{2}\,dQ(t).$

Then for any measurable function $T:\mathcal{X}\mapsto\mathbb{R}^{k}$ and
vector norm $\|\cdot\|:\mathbb{R}^{k}\mapsto\mathbb{R}_{+}$,

$\displaystyle\inf_{T}\,\sup_{\theta\in\mathbb{R}^{d}}\,\operatorname{\mathbb{E}}_{\theta}\|T(X)-\psi(\theta)\|^{2}\geq\sup_{h\in\mathbb{R}^{d}}\,\left[\sqrt{A_{\psi,Q,h}}-\sqrt{B_{\psi,Q,h}}\right]^{2}_{+}.$

Theorem 10 can be compared to Lemma 1 of Simons and Woodroofe,  (1983) which
restricts to unbiased estimators and does not consider priors. Similar to
Theorem 9, the sharpest constant can be achieved by taking the supremum over
all prior distributions $Q$. Under the Hellinger differentiability and
suitably selected $Q$, the Hellinger distance between two mixture
distributions admits the following expansion:

$\displaystyle H^{2}(\mathbb{P}_{0},\mathbb{P}_{h})$
$\displaystyle=H^{2}(Q_{h},Q)+\int_{\mathbb{R}^{d}}H^{2}(P_{t+h},P_{t})\,dQ_{h}^{1/2}\,dQ^{1/2}$
$\displaystyle=\frac{1}{4}h^{\top}\left(\mathcal{I}(Q)+\int_{\mathbb{R}^{d}}\mathcal{I}(t)\,dQ\right)h+o(\|h\|^{2})$

as $\|h\|_{2}\longrightarrow 0$. Therefore, assuming $\psi$ is continuously
differentiable, Theorem 10 implies the van Trees inequality under weaker
conditions than Theorem 9.

###  3.4 The extension from $\mathbb{R}^{d}$ to a general parameter space

Thus far, Theorems 9 and 10 are restricted to the case of
$\Theta_{0}=\Theta=\mathbb{R}^{d}$. These results may still be useful for
deriving a global minimax risk. However, it cannot be applied directly to a
local minimax risk, for instance, when $\Theta_{0}\subset\Theta$ is a
shrinking neighborhood of the parameter space around a particular value
$\theta_{0}$. In this section, Theorems 9 and 10 are extended to the general
space $\Theta_{0}\subseteq\mathbb{R}^{d}$ by constricting a diffeomorphism111A
diffeomorphism is an isomorphism of smooth manifolds. It is an invertible
function that maps one differentiable manifold to another such that both the
function and its inverse are differentiable.
$\varphi:\mathbb{R}^{d}\mapsto\Theta_{0}$. Although we only provide the
illustration using Theorem 10, an identical argument can be applied to Theorem
9.

Given a functional $\psi:\Theta_{0}\mapsto\mathbb{R}^{k}$, we define
$\widetilde{\psi}:\mathbb{R}^{d}\mapsto\mathbb{R}^{k}$, a composite function
$(\psi\circ\varphi)(t)$ for $t\in\mathbb{R}^{d}$ and also define respectively

$\displaystyle
d\widetilde{\mathbb{P}}_{0}(x,t):=dP_{\varphi(t)}(x)\,dQ(t)\quad\mbox{and}\quad
d\widetilde{\mathbb{P}}_{h}(x,t):=dP_{\varphi(t+h)}(x)\,dQ(t+h),$

where $Q(\cdot)$ is a probability measure on $\mathbb{R}^{d}$. The direct
application of Theorem 10 leads to the following corollary:

######  Corollary 10.1.

For any probability measure $Q$ on $\mathbb{R}^{d}$ and a diffeomorphism
$\varphi:\mathbb{R}^{d}\mapsto\Theta_{0}$, define a composite function
$\widetilde{\psi}:=(\psi\circ\varphi)$,

$\displaystyle
A_{\widetilde{\psi},Q,h}:=\frac{\left\|\int_{\mathbb{R}^{d}}\widetilde{\psi}(t)-\widetilde{\psi}(t-h)\,dQ(t)\right\|^{2}}{4H^{2}(\widetilde{\mathbb{P}}_{0},\widetilde{\mathbb{P}}_{h})}\quad\textrm{and}\quad
B_{\widetilde{\psi},Q,h}:=\int_{\mathbb{R}^{d}}\|\widetilde{\psi}(t)-\widetilde{\psi}(t-h)\|^{2}\,dQ(t).$

Then for any measurable function $T:\mathcal{X}\mapsto\mathbb{R}^{k}$ and
vector norm $\|\cdot\|:\mathbb{R}^{k}\mapsto\mathbb{R}_{+}$,

$\displaystyle\inf_{T}\,\sup_{\theta\in\Theta_{0}}\,\operatorname{\mathbb{E}}_{\theta}\|T(X)-\psi(\theta)\|^{2}\geq\sup_{\varphi}\,\sup_{h\in\mathbb{R}^{d}}\,\left[\sqrt{A_{\widetilde{\psi},Q,h}}-\sqrt{B_{\widetilde{\psi},Q,h}}\right]^{2}_{+}$

where the first supremum is over the diffeomorphism $\varphi$ between
$\mathbb{R}^{d}$ and $\Theta_{0}$.

We provide the intuition behind the role of the diffeomorphism with an
example. Suppose, for instance, the local parameter space $\Theta_{0}$ is an
open $\mathbb{R}^{d}$-ball around $\theta_{0}$ whose radius shrinks at the
rate $n^{-r}$ for $r>0$. For such $\Theta_{0}$, we may consider diffeomorphism
$\varphi$ in the form of

$\varphi(t):=\theta_{0}+n^{-r}\varphi_{0}(t)$

where $\varphi_{0}:\mathbb{R}^{d}\mapsto B([0],1)$ and $B([0],1)$ is an open
unit ball in $\mathbb{R}^{d}$. Without loss of generality, we assume that
$\varphi_{0}(0)=0$ so $\varphi(0)=\theta_{0}$. Then $\varphi$ admits the
following local expansion:

$\displaystyle\varphi(\delta)-\varphi(0)=n^{-r}\varphi_{0}(\delta)\approx
n^{-r}\nabla\varphi_{0}(\delta)\delta.$ (10)

Hence as $\|\delta\|\longrightarrow 0$, the functional $\psi(\varphi(\delta))$
approaches $\psi(\varphi(0))=\psi(\theta_{0})$ through a nonlinear path
uniquely defined by the diffeomorphism $\varphi_{0}$. We frequently use this
result in the later sections where we demonstrate the application of Theorems
9 and 10 to recover the constant implied by the LAM theorem.

######  Remark 3.

Another way to apply Theorems 9 and 10 is to use the fact that the supremum
over $\theta\in\Theta_{0}$ can be obtained by considering any two points
$\theta_{0},\theta_{1}\in\Theta_{0}$ and any smooth path
$\gamma:=\gamma_{\theta_{0},\theta_{1}}:(-\infty,\infty)\mapsto\Theta_{0}$
such that
$\lim_{u\to-\infty}\gamma(u)=\theta_{0},\lim_{u\to\infty}\gamma(u)=\theta_{1}$.
Let $\widetilde{\psi}(u):=\psi(\gamma(u))$ for $u\in\mathbb{R}$,

$d\mathbb{P}_{0}(x,u):=dP_{\gamma(u)}(x)dQ(u)\quad\mbox{and}\quad
d\mathbb{P}_{h}(x,u):=dP_{\gamma(u+h)}(x)dQ(u+h),$

where $Q$ is a probability measure on $\mathbb{R}$. Using this notation, we
obtain

$\displaystyle\inf_{T}\,\sup_{\theta\in\Theta_{0}}\,\mathbb{E}_{\theta}\|T(X)-\psi(\theta)\|^{2}$
$\displaystyle\quad=\inf_{T}\,\sup_{\theta_{0},\theta_{1}}\,\sup_{\gamma}\,\sup_{u\in\mathbb{R}}\,\mathbb{E}_{\gamma(u)}\|T(X)-\widetilde{\psi}(u)\|^{2}$
$\displaystyle\quad\geq\sup_{\theta_{0},\theta_{1}}\,\sup_{\gamma}\,\sup_{u\in\mathbb{R}}\,\left[\left(\frac{\|\int_{\mathbb{R}}(\widetilde{\psi}(u)-\widetilde{\psi}(u-h))\,dQ(u)\|^{2}}{4H^{2}(\mathbb{P}_{0},\mathbb{P}_{h})}\right)^{1/2}-\left(\int_{\mathbb{R}}\|\widetilde{\psi}(u)-\widetilde{\psi}(u-h)\|^{2}\,dQ(u)\right)^{1/2}\right]_{+}^{2}.$

As a concrete example, suppose we are interested in finding a lower bound when
$\Theta_{0}:=\\{\theta:\,\rho(\theta-\theta_{0})\leq\delta\\}$ for some
positive homogeneous norm function $\rho(\cdot)$, then one can take
$\theta_{1}=\theta_{0}+\delta h$ such that $\rho(h)\leq 1$ and consider
$\gamma(u)=\theta_{0}+\Phi(u)\delta h$ where $\Phi(\cdot)$ is the CDF of the
standard normal distribution. This approach of taking a smooth path between
two arbitrary points allows for the application of the results above to non-
convex, but continuous, parameter spaces $\Theta_{0}$.

###  3.5 Connection to the modulus of continuity

We now discuss the connections between Theorem 10 and existing minimax lower
bounds in terms of the Hellinger distance. Previous works by Donoho and Liu,
(1987, 1991) provide a geometric interpretation of the minimax rate through
the modulus of continuity. We define the modulus of continuity of a real-
valued functional $\psi$ with respect to the Helligner distance as

$\omega(\varepsilon):=\left\\{\sup_{\theta_{1},\theta_{2}\in\Theta_{0}}\,|\psi(P_{\theta_{1}})-\psi(P_{\theta_{2}})|\,:\,H^{2}(P_{\theta_{1}},P_{\theta_{2}})\leq\varepsilon^{2}\right\\}.$

This quantity captures the maximum fluctuation of functionals evaluated at
sufficiently “similar” distributions. We momentarily focus on estimating
$\psi(P_{0})$ based on $n$ IID observations $X_{1},\ldots,X_{n}$ drawn from
$P_{0}$. The minimax risk is considered for the supremum over a local model
$\mathcal{P}_{n}:=\\{P:H^{2}(P_{0},P)\leq n^{-1}\\}$, which is a set of
distributions concentrated at $P_{0}$, analogous to the setting of the LAM
theorem. Section 9.4 of Donoho and Liu,  (1987) proves that, under additional
assumptions, for any $n$ sufficiently large,

$\displaystyle\inf_{T}\,\sup_{P\in\mathcal{P}_{n}}\,n\operatorname{\mathbb{E}}_{P}|T(X)-\psi(P)|^{2}>\frac{1}{16}\left(\frac{\omega(n^{-1/2})}{n^{-1/2}}\right)^{2}.$
(11)

Donoho and Liu,  (1987) also shows that, under the setting of the LAM theorem
(Theorem 4) and $\psi(P_{\theta})=\theta$, it holds

$\displaystyle\liminf_{n\longrightarrow\infty}\,\frac{1}{16}\left(\frac{\omega(n^{-1/2})}{n^{-1/2}}\right)^{2}=\frac{1}{4\mathcal{I}(\theta_{0})}.$
(12)

Hence, the non-asymptotic minimax lower bound given by (11) converges to $1/4$
the optimal constant according to the LAM theorem. It may be tempting to
multiply the lower bound of (11) by $4$, that is, to consider the sequence
$4^{-1}\\{n^{1/2}\omega(n^{-1/2})\\}^{2}$. Although this sequence converges to
the correct constant as $n\longrightarrow\infty$, it is an invalid lower bound
as there exists an estimator that violates this inequality, provided in
Section 8.4 of Donoho and Liu,  (1987). Donoho and Liu,  (1987) conjectured
that a different approach is necessary to develop a sequence of non-asymptotic
minimax lower bounds that converge to the correct limit. This result is
provided by Theorem 10. Our result is similar to Donoho and Liu,  (1987) in
the sense that both define local models relative to the Hellinger distance;
however, they differ as we consider the mixture of distribution over the
Hellinger ball instead of two distributions. It is also worth noting that an
analogous non-asymptotic result in terms of the two-point modulus of
continuity is studied by Chen,  (1997) without particular focus on a sharp
constant.

The local behavior of the Hellinger distance has been a frequent tool in
analyzing irregular statistical models (Ibragimov and Has’minskii, , 1981;
Donoho and Liu, , 1987; Shemyakin, , 2014; Duchi and Ruan, , 2018; Lin et al.,
, 2019). One of the fundamental two-point risk bounds in terms of the
Hellinger distance is provided by Theorem 6.1 of Ibragimov and Has’minskii,
(1981); however, the original proof does not provide the optimal constant,
leaving room for a simple improvement. Here, we present a refined two-point
risk lower bound with an optimal constant. Although this improvement still
fails to recover the asymptotic constant of the LAM theorem as we later
demonstrate, it may still be of independent interest for its simplicity.

######  Lemma 11 (Refinement of Theorem 6.1 of Ibragimov and Has’minskii,
(1981) for real-valued functionals).

For any real-valued functional $\psi:\Theta\mapsto\mathbb{R}$, a measurable
function $T$ and $\theta_{1},\theta_{2}\in\Theta$, we have

$\displaystyle\frac{1}{2}\left\\{\operatorname{\mathbb{E}}_{\theta_{1}}|T(X)-\psi(\theta_{1})|^{2}+\operatorname{\mathbb{E}}_{\theta_{2}}|T(X)-\psi(\theta_{2})|^{2}\right\\}\geq\left[\frac{1-H^{2}(P_{\theta_{1}},P_{\theta_{2}})}{4}\right]_{+}|\psi(\theta_{1})-\psi(\theta_{2})|^{2}.$

This implies that

$\displaystyle\inf_{T}\,\sup_{\theta\in\Theta}\,\operatorname{\mathbb{E}}_{\theta}|T(X)-\psi(\theta)|^{2}\geq\sup_{\theta_{1},\theta_{2}\in\Theta}\left[\frac{1-H^{2}(P_{\theta_{1}},P_{\theta_{2}})}{4}\right]_{+}|\psi(\theta_{1})-\psi(\theta_{2})|^{2}.$

Lemma 11 can be compared to Lemma 1 of Simons and Woodroofe,  (1983). It
should be noted that analogous results as Lemma 11 have reported in the
literature as the optimal constant for two-point risk has been explored as
early as Donoho and Liu,  (1987). This result is presented to underscore the
insufficiency of the two-point risk inequality in recovering the asymptotic
constant—a conclusion also reached by Donoho and Liu,  (1987). Although Lemma
11 is stated for a real-valued functional, these results can be extended to
any vector-valued functional, with the leading constant naturally dependent on
the vector norm $\|\cdot\|$.

Finally, similar objects as the Hellinger modulus of continuity have been
considered in the econometrics literature. When the local asymptotic minimax
lower bound, such as the right-hand term of Theorem 10, is bounded away from
zero, the functional estimation is called ill-posed (Pötscher, , 2002;
Forchini and Hillier, , 2005). This precludes the existence of a (locally)
uniformly consistent estimator of $\psi(P_{0})$ in the nonparametric model.
Pötscher,  (2002) and Forchini and Hillier,  (2005) define the modulus of
continuity in the total variation distance as

$\omega(\varepsilon):=\left\\{\sup_{\theta_{1},\theta_{2}\in\Theta}|\psi(P_{\theta_{1}})-\psi(P_{\theta_{2}})|\,:\,\textrm{TV}(P_{\theta_{1}},P_{\theta_{2}})\leq\varepsilon\right\\}$

where the total variation distance is defined as

$\displaystyle\textrm{TV}(P_{0},P_{1}):=\sup_{A\in\mathcal{A}}|P_{0}(A)-P_{1}(A)|,$

with the supremum over all Borel measurable sets $A$. Theorem 2.1 of Pötscher,
(2002) provides the sufficient condition under which the following holds:

$\displaystyle\lim_{\varepsilon\longrightarrow
0}\,\inf_{T}\,\sup_{P\in\mathcal{P}(\varepsilon)}\,\operatorname{\mathbb{E}}_{P}|T(X)-\psi(P)|^{2}$
$\displaystyle\geq\lim_{\varepsilon\longrightarrow
0}\,\frac{1}{4}\omega(\varepsilon)^{2}.$

where $\mathcal{P}(\varepsilon):=\\{P:\mathrm{TV}(P_{0},P)\leq\varepsilon\\}$.
Pötscher,  (2002) and Forchini and Hillier,  (2005) analyze concrete problems
where the lower bound is bounded away from zero. We note that there is no
significant loss in considering the TV- or Hellinger-moduli as both distances
define the same topology of the space of probability measures. However, our
result also differs from Pötscher,  (2002) and Forchini and Hillier,  (2005)
as we focus on the non-asymptotic results as well as the optimal constant.
When the asymptotic lower bound of Theorem 10 is bounded away from zero, it
can be considered as the measure of ill-posedness. We defer the corresponding
analysis to future works.

##  4 Asymptotic properties

In this section, we examine the asymptotic properties of the general minimax
lower bounds, presented in Section 3. Specifically, we investigate whether
these bounds can recover established asymptotic constants, such as the LAM
theorem for both parametric and nonparametric models as well as the local
minimax rates for irregular estimation. These findings further reinforce our
general understanding that (1) the local behavior of the Hellinger distance is
easier to assert than that of the chi-squared divergence and (2) refined
constants may be obtained using the mixture-based method instead of the two-
point risk method.

The following two subsections are dedicated to recovering the classical LAM
theorem for parametric and semiparametric models. Interestingly, they can
imply a whole spectrum of such results; in classical LAM results, the
neighborhood around a parameter shrinks at an $n^{-1/2}$ rate while we can let
that neighborhood shrink at an arbitrary rate given the finite sample nature
of our results. This includes the superefficiency phenomenon as well. For
example, if the neighborhood is a singleton, then the lower bound should be
zero. See Section 6 for more details.

###  4.1 Local asymptotic minimax lower bound

We first investigate whether the proposed lower bounds can recover the exact
asymptotic constant provided by the LAM theorem for a parametric model. The
setting is identical to the LAM theorem, that is,
$X_{1},\dots,X_{n}\in\mathcal{X}$ are IID observations from
$P_{\theta_{0}}\in\\{P_{\theta}:\theta\in\Theta\subseteq\mathbb{R}^{d}\\}$
where $P_{t}$ is Hellinger differentiable at $t$. Let
$\psi:\Theta\mapsto\mathbb{R}$ be continuously differentiable at $\theta_{0}$
and $T:\mathcal{X}^{n}\mapsto\mathbb{R}$ be any sequence of measurable
functions. We can then state the following result:

######  Proposition 1.

Assuming the setting of Theorem 4, the following statements hold:

  1. (i)

Theorem 8 implies the LAM theorem,

  2. (ii)

if (A2) holds and the Fisher defect is zero at $\theta_{0}$, then Theorem 9
implies the LAM theorem,

  3. (iii)

Theorem 10 implies the LAM theorem, and

  4. (iv)

Lemma 11 implies the LAM theorem with its lower bound multiplied by the
constant $C\approx 0.28953$.

We provide proofs for (i)–(iv) in Supplementary Material. In particular,
Theorems 8 and 10 do not require additional conditions to imply the LAM
theorem, even though they hold for a broader class of estimation problems. The
LAM theorem remains valid for distributions with non-zero Fisher defect, which
can occur, for instance, when the parameter of interests lies on the boundary
(see Remark 1). Theorem 9 requires additional regularity conditions in order
to rule out this scenario since it leads to possible misbehavior of the chi-
squared divergence in its local limit. Although Proposition 1 (iv) slightly
improves the leading constant from 1/4, given by (12), it falls short of
recovering the optimal constant. This result highlights that the minimax lower
bound based on two distributions (i.e., Lemma 11) may not be sufficient to
recover the precise asymptotic constant.

###  4.2 Semiparametric efficiency bound

Although the results thus far may appear to be confined to parametric
settings, they can be extended to nonparametric functionals using the standard
argument in semiparametric statistics: the supremum over the collection of
parametric submodels. To begin, we introduce additional notation. We consider
the estimation of nonparametric functional
$\psi:\mathcal{P}\mapsto\mathbb{R}$, where $\mathcal{P}$ is a collection of
probability measures belonging to an infinite-dimensional set. We reduce the
study of nonparametric functionals to their behavior along parametric paths
$P_{t}$. Although a univariate path suffices for our purpose, all statements
in this section can be extended to a multivariate path.

In what follows, we define the notion of the smoothness of infinite-
dimensional functionals along the QMD path, which is defined as (4). We
emphasize that the theory remains intact under the weaker condition of the
Hellinger differentiability, and we only use the QMD since many objects in
semiparametric statistics are defined in terms of the score function $g_{0}$.
This allows for consistent use of terminology without modifying definitions. A
functional is called pathwise differentiable given a QMD path with the score
function $g_{0}$ if there exists a real-valued measurable function
$\dot{\psi}_{0}:\mathcal{X}\mapsto\mathbb{R}$ such that

$\left|\psi(P_{t})-\psi(P_{0})+t\int\dot{\psi}_{0}\,g_{0}\,dP_{0}\right|=o(t),\quad\mbox{as}\quad
t\longrightarrow 0.$ (13)

The definition of pathwise differentiability implies that the local behavior
of $\psi$ only depends on each path through the linear functional of
$\dot{\psi}_{0}$. Below, we denote each path by $P_{t,g}\in\mathcal{P}$ for a
given generic score function $g$ to make the dependence explicit. Then, we
observe that the minimax risk of functional estimation in a nonparametric
model must be at least larger than the supremum of the minimax risk along any
parametric paths indexed by $g$. Here, the supremum is taken over the entire
space of $g$, called the tangent set $\mathcal{T}_{P_{0}}$ of $\mathcal{P}$ at
$P_{0}$. A functional is pathwise differentiable relative to
$\mathcal{T}_{P_{0}}$ if equation (13) holds for all
$g\in\mathcal{T}_{P_{0}}$. Although the function $\dot{\psi}_{0}$ is not
generally unique, there is a unique projection of $\dot{\psi}_{0}$ onto the
closed linear span of the tangent set. The projected function is called the
efficient influence function and plays an important role in the semiparametric
efficiency theory. We consider the nonparametric model, containing all
distributions on the shared measurable space. The tangent space associated
with this model at $P_{0}$ corresponds to the collection of mean-zero
functions in $L_{2}(P_{0})$, i.e., the entire Hilbert space of mean-zero,
finite-variance functions. This is the maximal tangent space and we denote by
$L_{2}^{0}(P_{0})$. With this terminology in place, we present the following
nonparametric analog of the LAM theorem,

######  Theorem 12 (Local asymptotic minimax theorem II (Theorem 5.2 of Van
der Vaart,  (2002))).

Let the functional $\psi:\mathcal{P}\mapsto\mathbb{R}$ be pathwise
differentiable at $P_{0}$ relative to the maximal tangent space
$L_{2}^{0}(P_{0})$ with an efficient influence function $\dot{\psi}_{0}$.
Assuming that the tangent set is a linear closure, then for any measurable
function $T$ of the $n$ IID observations from $P_{t,g}$,

$\displaystyle\sup_{g\in
L_{2}^{0}(P_{0})}\,\liminf_{c\longrightarrow\infty}\,\liminf_{n\longrightarrow\infty}\,\inf_{T}\,\sup_{|t|<cn^{-1/2}}\,n\operatorname{\mathbb{E}}_{P^{n}_{t,g}}|T(X)-\psi(P_{t,g})|^{2}\geq\int\dot{\psi}^{2}_{0}\,dP_{0}.$
(14)

The resulting lower bound is often called the semiparametric efficiency bound
and it extends the LAM theorem to nonparametric contexts. We now demonstrate
the application of Theorems 8–10 and Lemma 11 to derive the semiparametric
efficiency bound. Before applying Theorem 9 in particular, we need to
introduce an additional condition. Following Example 25.16 of Van der Vaart,
(2000), we define smooth and bounded parametric paths as follows:

$dP_{t,g}(x)=\frac{1}{C_{t}}\kappa(tg(x))\,dP_{0}(x)$ (15)

where $C_{t}:=\int\kappa(tg(x))\,dP_{0}$. We assume that
$\kappa(0)=\kappa^{\prime}(0)=1$ and $\|\kappa^{\prime}\|_{\infty}\leq K$ and
$\|\kappa^{\prime\prime}\|_{\infty}\leq K$ for some constant $K$. For
instance, the function $\kappa(t):=2/(1+\exp(-2t))$ satisfies this condition.
This choice of parametric paths allows the score function to be unbounded but
the paths are bounded themselves. Crucially, this path asserts the local
behavior of the chi-squared divergence, as in Lemma 1, uniformly over
$g\in\mathcal{T}_{P_{0}}\subseteq L^{0}_{2}(P_{0})$ (See Lemma 1 of Duchi and
Ruan,  (2021)). We then state the following result.

######  Proposition 2.

Assuming the identical setting as Theorem 12, the following statements hold:

  1. (i)

Theorem 8 implies the semiparametric efficiency bound for any QMD parametric
path,

  2. (ii)

if each parametric path is defined as (15), then Theorem 9 implies the
semiparametric efficiency bound,

  3. (iii)

Theorem 10 implies the semiparametric efficiency bound for any QMD parametric
path, and

  4. (iv)

Lemma 11 implies the semiparametric efficiency bound for any QMD parametric
path, multiplied by the constant $C\approx 0.28953$.

We provide proofs for (i)–(iv) in Supplementary Material for completeness
although they follow naturally from Proposition 1. The proof proceeds roughly
as follows: We apply the results of Proposition 1 on each parametric path with
a fixed score function $g$, and then take the supremum of the score function
over the linear closure of the tangent set $\mathcal{T}_{P_{0}}$. The
statement (ii) of Proposition 2 shows that the lower bound based on the chi-
squared divergence restricts the choice of parametric paths beyond the
Hellinger differentiability. Such restriction can be undesirable for certain
cases, as we discuss in the following remark.

######  Remark 4 (The implication from restricting parametric paths).

When the linear closure of the tangent set spans the entire $L_{2}(P_{0})$,
then the choice of parametric path does not impact the lower bound. Therefore,
there is no loss in selecting a specific path such as the one given by (15).
However, if the tangent set is a strict subset of $L_{2}(P_{0})$, such as a
tangent cone, the constraint on the path becomes undesirable. In particular,
this limitation on paths can have significant implications for semiparametric
inference under shape constraints or parameters on bounded domains. For
instance, Kuchibhotla et al.,  (2021) considers the projection of arbitrary
QMD parametric paths onto a working statistical model in order to satisfy
certain shape constraints. The statement (ii) of Proposition 2 may not be
generally applicable in such cases.

######  Remark 5 (Non-pathwise differentiable functionals).

While we demonstrate the application of the proposed lower bounds to derive
the semiparametric efficiency bound for pathwise differentiable functionals,
the main results from Section 3 still hold for general non-pathwise
differentiable functionals. First, by taking $\Phi$ as a collection of
pathwise differentiable functionals, Lemma 7 implies

$\displaystyle\sup_{g\in\mathcal{T}_{P_{0}}}\,\inf_{T}\,\sup_{\theta\in\Theta}\,\operatorname{\mathbb{E}}_{\theta}|T(X)-\psi(P_{\theta,g})|^{2}$
$\displaystyle\,\,\geq\sup_{g\in\mathcal{T}_{P_{0}}}\,\sup_{\phi\in\Phi,\,Q\in\mathcal{Q}(\Theta)}\,\left[\left(\int\,\operatorname{\mathbb{E}}_{\theta}|T(X)-\phi(P_{\theta,g})|^{2}\,dQ\right)^{1/2}-\left(\int\|\psi(P_{\theta,g})-\phi(P_{\theta,g})\|^{2}dQ(\theta)\right)^{1/2}\right]_{+}^{2}.$

This suggests that the efficiency bound for non-pathwise differentiable
functionals can be analyzed by first placing a prior over the local parametric
paths and then deriving the Bayes risk for estimating $\phi$ at the expense of
the approximation error.

Alternatively, we can apply Theorem 10 to each parametric path and take the
supremum over any generic tangent set $\mathcal{T}_{P_{0}}\subseteq
L_{2}^{0}(P_{0})$. This implies

$\displaystyle\sup_{g\in\mathcal{T}_{P_{0}}}\,\inf_{T}\,\sup_{\theta\in\mathbb{R}^{d}}\,\operatorname{\mathbb{E}}_{\theta}|T(X)-\psi(P_{\theta,g})|^{2}$
$\displaystyle\,\,\geq\sup_{g\in\mathcal{T}_{P_{0}}}\,\sup_{h\in\mathbb{R}^{d}}\left[\left(\frac{|\int_{\mathbb{R}^{d}}\left(\psi(P_{t,g})-\psi(P_{t-h,g}\right)\,dQ(t)|^{2}}{4H^{2}(\mathbb{P}_{0,g},\mathbb{P}_{h,g})}\right)^{1/2}-\left(\int_{\mathbb{R}^{d}}|\psi(P_{t,g})-\psi(P_{t-h,g})|^{2}\,dQ(t)\right)^{1/2}\right]^{2}_{+}$

where

$d\mathbb{P}_{0,g}(x,\theta):=dP_{\theta,g}(x)\,dQ(\theta)\quad\mbox{and}\quad
d\mathbb{P}_{h,g}(x,\theta):=dP_{\theta+h,g}(x)\,dQ(\theta+h).$

In the above non-asymptotic lower bound, the limiting behavior of
$\psi(P_{t,g})-\psi(P_{t-h,g})$ as $h\longrightarrow 0$ is unspecified,
allowing $\psi$ to be non-smooth at $t$.

###  4.3 Local minimax rate for irregular estimation

The use of minimax lower bounds based on the Hellinger distance by Theorem 10
is also attractive in the context of irregular problems. Recently, Lin et al.,
(2019) has proposed using the Hölder smoothness of the local Hellinger
distance, relative to $P_{0}$, as the degrees of irregularity for the
corresponding estimation problem. This proposal is motivated by the classical
result in the literature, namely, Theorem 6.1 of Ibragimov and Has’minskii,
(1981). This result provides a two-point risk inequality in terms of the
Hölder smoothness of the functional and that of the local Hellinger distance
between two distributions in a model. In what follows, we demonstrate that
Theorem 10 also recovers the asymptotic minimax rate given by Theorem 6.1 of
Ibragimov and Has’minskii,  (1981).

######  Lemma 13 (Theorem 6.1 of Ibragimov and Has’minskii,  (1981)).

Consider the estimation of a real-valued functional $\psi(\theta)$ based on
the $n$ IID observations from $P_{\theta}$. We define
$\mathrm{sign}(x):=x/\|x\|$ for $x\in\mathbb{R}^{d}$. We assume that there
exits a constant $\delta>0$ such that for all
$t,t+h\in\\{\theta:\|\theta_{0}-\theta\|<\delta\\}$ and as
$\|h\|\longrightarrow 0$,

$\displaystyle H^{2}(P_{t},P_{t+h})$
$\displaystyle=C_{1}\|h\|^{\alpha}+o(\|h\|^{\alpha}),\quad\alpha\in(0,2],$
$\displaystyle\psi(t+h)-\psi(t)$
$\displaystyle=C_{2}\|h\|^{\beta}+o(\|h\|^{\beta}),\quad\beta>0,$

where $C_{1}\equiv C_{1,t,\mathrm{sign}(h)}$ and $C_{2}\equiv
C_{2,t,\mathrm{sign}(h)}$. Futhermore, these constants are assumed uniformly
bounded such that $\sup_{t,h}C_{1,t,\mathrm{sign}(h)}\leq\bar{C_{1}}$ and
$\sup_{t,h}C_{2,t,\mathrm{sign}(h)}\leq\bar{C_{2}}$ for some
$\overline{C}_{1},\overline{C}_{2}\in(0,\infty)$. Theorem 10 then implies that
there exists a constant $C$, depending on
$\overline{C}_{1},\overline{C}_{2},\alpha$, and $\beta$, satisfying

$\displaystyle\liminf_{c\longrightarrow\infty}\,\liminf_{n\longrightarrow\infty}\,\inf_{T}\,\sup_{\|\theta-\theta_{0}\|<cn^{-1/\alpha}}\,n^{2\beta/\alpha}\operatorname{\mathbb{E}}_{P_{\theta}^{n}}|T(X)-\psi(\theta)|^{2}\geq
C>0.$

The proof of this lemma can be found in Supplementary Material. We note that
$\alpha$ is defined on the interval $(0,2]$ with $\alpha=2$ corresponding to
the standard Hellinger differentiable models where the Fisher information is
well-defined. Hence the most regular setting corresponds to $\alpha=2$ and
$\beta=1$. Although the Hellinger differentiability fails for $\alpha<2$, the
Hellinger distance can still be well-defined. For instance, a set of uniform
distributions $\text{Unif}(0,\theta)$ indexed by $\theta>0$ corresponds to
$\alpha=1$. We refer to Lin et al.,  (2019) for more examples of irregular
models. While Theorem 10 can account for this type of irregularity, Theorems 8
and 9 yield trivial lower bounds of zero; Theorem 8 fails due to the lack of
the Fisher information and Theorem 9 fails as the chi-squared divergence for
an irregular model is often infinite for any $n\geq 1$ and $c>0$.

##  5 Applications: Local minimax lower bounds

In this section, we provide several application of the minimax lower bounds
for the estimation problems involving points of non-differentiability.

###  5.1 Nonparametric density estimation

The first application is nonparametric density estimation under the smoothness
assumption.

######  Example 1 (Nonparametric density estimation).

Let $X_{1},\ldots,X_{n}\in\mathcal{X}\subseteq\mathbb{R}$ be $n$ IID
observations from the unknown density function $f_{0}$. The functional of
interest is the density value at a pre-specified point $x_{0}\in\mathcal{X}$,
that is, $\psi(f):=f(x_{0})$.

We assume that the true density $f_{0}$ is $s$-times continuously
differentiable at $x_{0}$. We then analyze the following class of density
functions:

$\displaystyle\mathcal{F}(s,f_{0},x_{0},M):=\left\\{f\textrm{ is $s$-times
differentiable at $x_{0}$, satisfying
}|f^{(s)}(x_{0})|\leq(1+M)|f^{(s)}_{0}(x_{0})|\right\\}$

for fixed $M>0$. Furthermore, we define the following localized set of density
functions

$\displaystyle
U(\delta;\varepsilon):=\left\\{f\in\mathcal{F}(s,f_{0},x_{0},M)\,:\,\int_{|x-x_{0}|\leq\varepsilon}\,|f^{(k)}(x)-f^{(k)}_{0}(x)|\,dx\leq\delta\textrm{
for all }k\in\\{0,1,\ldots,s\\}\right\\}$

for fixed $\varepsilon>0$. The parameter $M$ is introduced in order to prevent
the true density $f_{0}$ from lying on the boundary of the local model
$U(\delta;\varepsilon)$. As $\psi(f)$ is nonsmooth functional, we consider the
following approximation via convolution:

$\displaystyle\phi(f)\equiv\phi(f;K,h):=\int
h^{-1}K\left(\frac{x-x_{0}}{h}\right)\,f(x)\,dx$

where $K$ is a kernel function and $h>0$ is a bandwidth parameter. The
collection of approximation functionals $\Phi$ is then indexed by the choice
of $K$ and the values of $h>0$. We consider any kernel function that satisfies
the following conditions:

  1. (A3)

A function $K$ is assumed to satisfy the following conditions:

    1. (a)

it is uniformly bounded,

    2. (b)

it is $s$-times differentiable with the uniformly bounded $s$th derivative,

    3. (c)

it integrates to one over its support, and

    4. (d)

for all integer $k$ where $0<k<s$, $\int_{-1}^{1}u^{k}K(u)\,du=0$ and
$\int_{-1}^{1}u^{s}K(u)\,du<\infty$.

We then define the following class of approximation functionals:

$\displaystyle\Phi:=\left\\{\phi(f;K,h):\textrm{ for all $h>0$ and $K$,
satisfying \ref{as:kernel}}\right\\}.$

We now present the application of Theorem 8 to the estimation of $\psi(f)$
approximated with $\phi(f)$ for any $\phi\in\Phi$:

######  Lemma 14.

Let $\delta_{n}:=c_{0}n^{-r}$ for $r\in[0,(2s+1)^{-1})$. Then Theorem 8
implies,

$\displaystyle\liminf_{n\longrightarrow\infty}\,\inf_{T}\sup_{f\in
U(\delta_{n})}n^{2s/(2s+1)}\operatorname{\mathbb{E}}_{f}|T(X)-f(x_{0})|^{2}$
$\displaystyle\qquad\geq\sup_{K}C(s,M,K)\,f_{0}(x_{0})^{2s/(2s+1)}|f_{0}^{(s)}(x_{0})|^{2/(2s+1)}$

where $C(s,M,K)$ is a constant only depending on $s,M$ and $K$.

The proof of this lemma is provided in Section D of Supplementary Material. To
the best of our knowledge, this is a new local asymptotic minimax constant in
the context of nonparametric density estimation. A similar object to the term
in the lower bound has appeared in the classical literature on kernel density
estimation (Devroye and Györfi, , 1985). Let $f_{0}$ be any density on $[0,1]$
with continuous $s$th derivative. Then Theorem 11 (Chapter 4, page 49) of
Devroye and Györfi,  (1985) states that

$\displaystyle\liminf_{n\longrightarrow\infty}\,\sup_{f\in
H(f_{0})}\,n^{2s/(2s+1)}\frac{\operatorname{\mathbb{E}}_{f}\int|T-f|^{p}}{(\int
f^{p/2})^{2s/(2s+1)}(\int|f^{(s)}|^{p})^{1/(2s+1)}}>0$

for any estimator $T$ where $H(f_{0})$ is a set of all densities of the form
$\sum_{i=1}^{\infty}\pi_{i}f_{0}(x+x_{i})$, $\pi_{i}$ is any probability
vector and $\\{x_{i}\\}$ is an increasing sequence of real numbers such that
$x_{i+1}-x_{i}>1$. Simple algebra also suggests that the kernel density
estimator, defined as

$\displaystyle\widehat{f}:=\frac{1}{nh}\sum_{i=1}^{n}K\left(\frac{X_{i}-x_{0}}{h}\right)$

with the following “theoretical” choice of bandwidth

$\displaystyle
h^{*}\propto\left(\frac{f_{0}(x_{0})}{n|f_{0}^{(s)}(x_{0})|^{2}}\right)^{1/(2s+1)}$

attains the matching upper bound including the dependency on $f_{0}$
(Abramson, , 1982; Woodroofe, , 1970; Hall, , 1993; Brown et al., , 1997).
Such an estimator is often called adaptive in the nonparametric density
estimation literature222This notion of adaptivity is completely separate from
the adaptation to unknown smoothness $s$.. A reasonable estimate of $h^{*}$ is
obtained by the “plug-in” rule where $f_{0}$ and $f_{0}^{(s)}$ in $h^{*}$ are
replaced with pilot estimators $\widetilde{f}(x_{0})$ and
$\widetilde{f}^{(s)}(x_{0})$, constructed from a separate data. The final
estimator is defined as

$\displaystyle\check{f}:=\frac{1}{nh_{\textrm{plug-
in}}}\sum_{i=1}^{n}K\left(\frac{X_{i}-x_{0}}{h_{\textrm{plug-
in}}}\right)\quad\text{where}\quad h_{\textrm{plug-
in}}\propto\left(\frac{\widetilde{f}(x_{0})}{n|\widetilde{f}^{(s)}(x_{0})|^{2}}\right)^{1/(2s+1)}.$

Assuming these pilot estimators converge in quadratic mean, Brown et al.,
(1997) states that the estimator $\check{f}$ asymptotically achieves the
minimum pointwise risk, i.e., the risk under fixed $f_{0}$, among all choice
of bandwidth parameters (See equation (2.5) of Brown et al.,  (1997)). Brown
et al.,  (1997) however argues that such a pointwise assessment of the
nonparametric estimator is often misleading in view of superefficiency,
necessitating the evaluation under “uniform” risk. Our result does not
contradict the message by Brown et al.,  (1997). The pointwise risk of the
estimator is equivalent to the local minimiax risk over a singleton
$\\{f_{0}\\}$, and this roughly corresponds to taking $r\longrightarrow\infty$
in our result. Lemma 14 does not allow the neighborhood $U(n^{-r})$ to shrink
too fast, slower than $n^{-1/(1+s)}$ to be precise, which prevents the issue
of superefficiency. Hence, the adaptivity to the unknown $f_{0}$ is not purely
an artifact of superefficiency.

The van Trees inequality has been previously applied to analyze minimax lower
bounds in nonparametric problems. See, for instance, Theorem 4 of Anevski and
Soulier,  (2011) and Example 2.3 of Tsybakov,  (2008). In order to apply the
classical van Trees inequality, these approaches typically reduce the original
nonparametric problem to parameter estimation along a parametric submodel.
This differs from our “direct” approach based on Theorem 8. Consequently,
existing results do not account for risk over a shrinking neighborhood around
$f_{0}$, and do not provide the dependency of the lower bound on $f_{0}$.
Therefore, these results do not fully comply with the classical notion of
local asymptotic minimaxity.

###  5.2 Simple directionally differentiable functionals

We now present the application of Theorem 10 to the estimation of
$\psi(P_{\theta}):=\max(\theta,0)$ for $\theta\in\mathbb{R}$. This is one of
the canonical examples of directionally differentiable functionals. More
complex problems such as interval regression (Fang, , 2014) and testing of the
shape of regression (Juditsky and Nemirovski, , 2002), can be reduced to this
form.

######  Example 2 (Estimating $\max(\theta,0)$ when $\theta=0$).

Suppose $X_{1},\ldots,X_{n}$ are $n$ IID observations drawn from $P_{0}$ that
belongs to a local model
$\\{P_{\theta}:|\theta|<\delta,\theta\in\mathbb{R}\\}$ for fixed $\delta>0$.
We assume that this model is Hellinger differentiable with the Fisher
information $\mathcal{I}(t)$ for $t\in(-\delta,\delta)$. The functional of
interest is $\psi(P_{\theta})=\max(0,\theta)$.

Then Theorem 8 implies the following non-asymptotic local minimax lower bound:

######  Lemma 15.

Suppose that $\mathcal{Q}^{\dagger}\equiv\mathcal{Q}^{\dagger}(-1,1)$ is a
collection of “nice” priors defined on $[-1,1]$ (see Definition 1) then
Theorem 8 implies that

$\displaystyle\inf_{T}\,\sup_{|\theta|<\delta}\,\operatorname{\mathbb{E}}_{\theta}|T(X)-\psi(P_{\theta})|^{2}$
$\displaystyle\geq\sup_{Q\in\mathcal{Q}^{\dagger}}\frac{\left(\int_{0}^{1}q(t)\,dt\right)^{2}}{\delta^{-2}\mathcal{I}(Q)+n\int_{-1}^{1}\mathcal{I}(\delta
t)q(t)\,dt}$ (16)
$\displaystyle\geq\sup_{a\in[0,1]}\,\frac{a^{2}}{4\pi^{2}\delta^{-2}W_{a}^{-2}+n\sup_{|t|<\delta}\mathcal{I}(t)}$
(17)

where $q$ is a density function of $Q$ for any $Q\in\mathcal{Q}^{\dagger}$,
$W_{a}:=2/(Y_{a}+1)$ and $Y_{a}$ is the inverse of the following equation:

$\displaystyle Y_{a}-\sin(-\pi Y_{a})/\pi=|2a-1|.$

The first result of Lemma 15, given by (16), is a direct consequence of the
van Trees inequality since the functional $\psi(\theta)=\max(\theta,0)$ is
absolutely continuous. The second result of Lemma 15, given by (17),
investigates the least-favorable prior using variational calculus and
optimization under absolute moment constraints (Ernst, , 2017). The
corresponding derivation can be found in Section E of Supplementary Material.
Our lower bound involves the inverse of the Kepler equation from Celestial
Mechanics, which does not have a closed-form solution (Kepler, , 1609). Hence,
our lower bound relies on a computational method. This technique seems to be
less known in the literature but has been mentioned in the context of non-
asymptotic minimax lower bound for Gaussian mean estimation under bounded
constraints (Levit, , 2010).

Next, we provide a non-asymptotic lower bound for estimating
$\max(\theta^{\alpha},0)$ for $0<\alpha\leq 1$ and $\theta\in\mathbb{R}$ based
on Theorem 10. This function is also non-smooth at $\theta=0$ and the
corresponding lower bound behaves differently depending on the location of the
true parameter.

######  Example 3 (Estimating $\max(\theta^{\alpha},0)$ for $\alpha>0$).

Consider the identical settings as Example 2, except the functional of
interest is now $\psi(P_{\theta})=\max(0,\theta^{\alpha})$ for $\alpha>0$.

The following lemma provides the application of Theorem 10 to this problem:

######  Lemma 16.

Theorem 10 implies the following non-asymptotic local minimax lower bounds:

  1. (i)

When $\theta_{0}<0$, it holds that

$\displaystyle\inf_{T}\,\sup_{|\theta-\theta_{0}|<\delta}\,\operatorname{\mathbb{E}}_{\theta}|T(X)-\psi(P_{\theta})|^{2}\geq\sup_{Q,\varphi_{0}}\,\frac{\delta^{2}\alpha^{2}|\operatorname{\mathbb{E}}_{Q}\\{\theta_{0}+\delta\varphi_{0}(t)\\}^{\alpha-1}\varphi^{\prime}_{0}(t)\,I\\{t:|\theta_{0}|/\delta<\varphi_{0}(t)\\}|^{2}}{\mathcal{I}(Q)+n\delta^{2}\operatorname{\mathbb{E}}_{Q}[\\{\varphi_{0}^{\prime}(t)\\}^{2}\,\mathcal{I}(\theta_{0}+\delta\varphi_{0}(t))]}.$

  2. (ii)

When $\theta_{0}>0$, it holds that

$\displaystyle\inf_{T}\,\sup_{|\theta-\theta_{0}|<\delta}\,\operatorname{\mathbb{E}}_{\theta}|T(X)-\psi(P_{\theta})|^{2}\geq\sup_{Q,\varphi_{0}}\,\frac{\delta^{2}\alpha^{2}|\operatorname{\mathbb{E}}_{Q}\\{\theta_{0}+\delta\varphi_{0}(t)\\}^{\alpha-1}\varphi^{\prime}_{0}(t)\,I\\{t:\varphi_{0}(t)<\theta_{0}/\delta\\}|^{2}}{\mathcal{I}(Q)+n\delta^{2}\operatorname{\mathbb{E}}_{Q}[\\{\varphi_{0}^{\prime}(t)\\}^{2}\,\mathcal{I}(\theta_{0}+\delta\varphi_{0}(t))]}.$

  3. (iii)

When $\theta_{0}=0$, it holds that

$\displaystyle\inf_{T}\,\sup_{|\theta|<\delta}\,\operatorname{\mathbb{E}}_{\theta}|T(X)-\psi(P_{\theta})|^{2}\geq\sup_{Q,\varphi_{0}}\,\frac{\delta^{2\alpha}\alpha^{2}|\operatorname{\mathbb{E}}_{Q}\varphi_{0}(t)^{\alpha-1}\varphi^{\prime}_{0}(t)\,I\\{t:\varphi_{0}(t)>0\\}|^{2}}{\mathcal{I}(Q)+n\delta^{2}\operatorname{\mathbb{E}}_{Q}[\\{\varphi_{0}^{\prime}(t)\\}^{2}\,\mathcal{I}(\delta\varphi_{0}(t))]}.$

The supremums are taken over any probability measure $Q$ on $\mathbb{R}$ and
any increasing deffeomorphism $\varphi_{0}:\mathbb{R}\mapsto(-1,1)$ such that
$\varphi_{0}(0)=0$ and $\|\varphi^{\prime}_{0}\|_{\infty}<C$ for some
constant.

Here, the choice of priors no longer needs to satisfy Definition 1. Instead,
Lemma 16 posits certain requirements over the choice of diffeomorphism.

######  Remark 6 (Minimax rates of convergence under IID observations).

From the expressions above, we can deduce the local minimax rates of
convergence. When $\theta_{0}>0$, the lower bound (ii) of Lemma 16 above
implies that

$\displaystyle\frac{\delta^{2}O(1)}{\mathcal{I}(Q)+n\delta^{2}O(1)}=\frac{O(1)}{\delta^{-2}\mathcal{I}(Q)+nO(1)}.$

Balancing two terms in the denominator, we choose $\delta=O(n^{-1/2})$; the
overall rate of convergence is $n^{-1}$, or so-called parametric rate,
provided $\theta_{0}>0$. This is reasonable given that $\psi(\theta_{0})$ is a
smooth parameter when $\theta_{0}$ is bounded away from zero.

Similarly when $\theta_{0}=0$, the lower bound (iii) of Lemma 16 implies that

$\displaystyle\frac{\delta^{2\alpha}O(1)}{\mathcal{I}(Q)+n\delta^{2}O(1)}=\frac{O(1)}{\delta^{-2\alpha}\mathcal{I}(Q)+n\delta^{2-2\alpha}O(1)}.$

Balancing two terms in the denominator, we choose $\delta=O(n^{-1/2})$ again;
the overall rate of convergence is now $n^{-\alpha}$ when $\theta_{0}=0$,
which is strictly slower than the parametric rate when $\alpha<1$. We thus
conclude that the minimax rates of convergence for estimating
$\max(\theta^{\alpha},0)$ remain the same as the rates for $\theta^{\alpha}$
discussed in Gill and Levit,  (1995).

######  Remark 7 (Local asymptotic minimax constants).

Lemma 16 also recovers the precise constants for the local asymptotic minimax
lower bound. In addition to the setting of Lemma 16, we further assume that
$\mathcal{I}(t)$ is continuous at $\theta_{0}$. Given the observation from
Remark 6, we replace $\delta$ with $cn^{-1/2}$ and analyze their limits. When
$\theta_{0}<0$, there exists $n$ large enough that $|\theta_{0}|\geq
cn^{-1/2}$, and hence we have

$\displaystyle\liminf_{c\longrightarrow\infty}\,\liminf_{n\longrightarrow\infty}\,\inf_{T}\,\sup_{|\theta-\theta_{0}|<cn^{-1/2}}n\operatorname{\mathbb{E}}_{P^{n}_{\theta}}|T(X)-\psi(P_{\theta})|^{2}\geq
0.$

When $\theta_{0}>0$, there exists $n$ large enough such that $\theta_{0}\geq
cn^{-1/2}$, and thus $I\\{t:\varphi_{0}(t)<\theta_{0}/(cn^{-1/2})\\}=1$ for
all $t\in\mathbb{R}$. We then have,

$\displaystyle\liminf_{c\longrightarrow\infty}\,\liminf_{n\longrightarrow\infty}\,\inf_{T}\,\sup_{|\theta-\theta_{0}|<cn^{-1/2}}n\operatorname{\mathbb{E}}_{P^{n}_{\theta}}|T(X)-\psi(P_{\theta})|^{2}\geq\frac{\alpha^{2}\theta_{0}^{2\alpha-2}}{\mathcal{I}(\theta_{0})}\frac{|\operatorname{\mathbb{E}}_{Q}\varphi_{0}^{\prime}(t)|^{2}}{\operatorname{\mathbb{E}}_{Q}\\{\varphi_{0}^{\prime}(t)\\}^{2}}$

where we invoke the dominated convergence theorem in view of the continuity of
$\mathcal{I}(t)$ at $t=\theta_{0}$ and the uniform boundedness of
$\varphi^{\prime}_{0}$. The supremum of the last quantity involving
$\varphi_{0}$ is $1$ as shown in the proof of Proposition 1 (iii) (See Section
C of Supplementary Material). Hence the local asymptotic minimax constant is
$\alpha^{2}\theta_{0}^{2\alpha-2}\mathcal{I}(\theta_{0})^{-1}$. The case with
$\theta_{0}=0$ is more involved. By the analogous argument,

$\displaystyle\liminf_{c\longrightarrow\infty}\,\liminf_{n\longrightarrow\infty}\,\inf_{T}\,\sup_{|\theta-\theta_{0}|<cn^{-1/2}}n^{\alpha}\operatorname{\mathbb{E}}_{P^{n}_{\theta}}|T(X)-\psi(P_{\theta})|^{2}$
$\displaystyle\qquad\geq\liminf_{c\longrightarrow\infty}\,\frac{c^{2\alpha}\alpha^{2}|\operatorname{\mathbb{E}}_{Q}\varphi_{0}(t)^{\alpha-1}\varphi^{\prime}_{0}(t)\,I\\{t>0\\}|^{2}}{\mathcal{I}(Q)+c^{2}\mathcal{I}(\theta_{0})\operatorname{\mathbb{E}}_{Q}\\{\varphi_{0}^{\prime}(t)\\}^{2}}.$

The $\liminf$ may not be achieved by the extreme values of $c$, and it depends
on specific choices of $Q$ and $\varphi_{0}$. Taking the supremum of this
object over the choice of $Q$ and $\varphi_{0}$ requires more involved
analysis, which we do not pursue in this manuscript.

###  5.3 Parameter estimation under an irregular model

The final illustration provides a local asymptotic minimax result where the
Hellinger differentiability fails. This is a quintessential example where the
Fisher information is undefined. We demonstrate that Theorem 10 recovers a
correct local minimax rate of convergence as well as local parameter
dependence. We also observe that the simple choice of diffeomorphism fails to
recover the correct constant and may require more delicate analyses.

######  Example 4 (Estimating $\theta_{0}$ from $\mathrm{Unif}(0,\theta_{0})$
for $\theta_{0}>0$).

Suppose $X_{1},\dots,X_{n}$ are $n$ IID observations from
$\mathrm{Unif}(0,\theta_{0})$, which belong to a statistical model
$\\{\mathrm{Unif}(0,\theta):0<\theta\\}$. The functional of interest is
$\psi(P_{\theta})=\theta$.

Then we state the following result:

######  Lemma 17.

Theorem 10 implies the following local asymptotic minimax lower bound:

$\displaystyle\liminf_{c\longrightarrow\infty}\,\liminf_{n\longrightarrow\infty}\,\inf_{T}\,\sup_{|\theta-\theta_{0}|<cn^{-1}}\operatorname{\mathbb{E}}_{P_{\theta}^{n}}n^{2}|T(X)-\theta|^{2}$
$\displaystyle\qquad\geq\theta_{0}^{2}\sup_{\eta,\,Q,\,\varphi_{0}}\,\left[\frac{\left|\int\eta\varphi_{0}^{\prime}(t)\,dQ(t)\right|}{2\left(2-2\int\exp\left(-\eta\varphi^{\prime}_{0}(t)/2\right)\,dQ(t)\right)^{1/2}}-\left(\int\\{\eta\varphi_{0}^{\prime}(t)\\}^{2}\,dQ(t)\right)^{1/2}\right]^{2}_{+}=C\theta_{0}^{2}$

for $C>0$ where the supremum is over $\eta\in\mathbb{R}$, diffeomorphism
$\varphi_{0}$ from $\mathbb{R}$ to $(-1,1)$ and any prior distributions $Q$
over $\mathbb{R}$.

The preceding display shows that the local minimax risk for parameter
estimation under uniform distribution behaves as $O(\theta_{0}^{2}/n^{2})$.
Without taking into account the constant, this is the correct known
dependency. Theorem 4.9 and Proposition 4.5 of Korostelev and Korosteleva,
(2011) further prove that the sharpest constant is $1$ based on a similar
approach using the Bayes risk with a uniform prior. This implies that Theorem
7, which is also based on the worst Bayes risk, can recover the correct
constant for irregular problems. However, Theorem 10 does not readily
accommodate a prior with compact support. In what follows, we present a
corresponding constant based on a simple choice of diffeomorphism, which fails
to recover the asymptotic constant.

######  Proposition 3.

Consider a sequence of diffeomorphism $\varphi_{0}(t;\eta)$, which is indexed
by $\eta>0$. Assuming that $\eta\varphi_{0}^{\prime}(t;\eta)\longrightarrow C$
as $\eta\longrightarrow\infty$ for some finite constant, we obtain

$\displaystyle\liminf_{c\longrightarrow\infty}\,\liminf_{n\longrightarrow\infty}\,\inf_{T}\,\sup_{|\theta-\theta_{0}|<cn^{-1}}\operatorname{\mathbb{E}}_{P_{\theta}^{n}}n^{2}|T(X)-\theta|^{2}\geq
C^{*}\theta_{0}^{2}$

where $C^{*}\approx 0.0635^{2}$.

The proceeding proposition demonstrates that while lemma 11 enables simpler
analysis and a sharper constant compared to Proposition 3, it still falls
short of achieving the theoretically optimal constant.

######  Proposition 4.

Under the settings as Example 4, Lemma 11 provides the following local
asymptotic minimax lower bound:

$\displaystyle\liminf_{c\longrightarrow\infty}\,\liminf_{n\longrightarrow\infty}\,\inf_{T}\,\sup_{|\theta-\theta_{0}|<cn^{-1}}\operatorname{\mathbb{E}}_{P_{\theta}^{n}}n^{2}|T(X)-\theta|^{2}\geq
C^{*}\theta_{0}^{2}$

where $C^{*}\approx 0.0558$.

##  6 Visual illustration: Upper bounds attained by plug-in estimators

In this section, we investigate the precise gaps between the risk of
estimators and the non-asymptotic local minimax lower bounds for fixed
$\delta>0$ and $n\geq 1$. The results in this section are provided mostly for
illustration purposes, demonstrating how non-asymptotic analysis of estimators
beyond the efficiency bound may look like. Consider the following estimation
problem:

$\displaystyle\sup_{|\theta|<\delta}\,n\operatorname{\mathbb{E}}_{\theta}|T(X)-\max(\theta,0)|^{2}$
(18)

where we observe $n$ IID observations $X:=(X_{1},\ldots,X_{n})$ from
$N(\theta,1)$. We define the local model as
$\\{N(\theta,1):|\theta|<\delta\\}$. We construct lower bounds to (18) using
two methods. The first lower bound (vT) is due to equation (17) of Lemma 15,
which is given by

$\inf_{T}\,\sup_{|\theta|<\delta}\,n\operatorname{\mathbb{E}}_{\theta}|T(X)-\psi(P_{\theta})|^{2}\geq\sup_{a\in[0,1]}\,\frac{na^{2}}{4\pi^{2}\delta^{-2}W_{a}^{-2}+n},$
(vT)

where $W_{a}$ is defined in Lemma 15. The second lower bound (diffeo) is based
on the equation Lemma 16 (iii) when $\alpha=1$. As computing the supremum over
$\varphi_{0}$ and $Q$ is challenging, we simplify the problem by focusing on
the following choices:

$\displaystyle
Q\in\\{N(\mu,\sigma^{2}):(\mu,\sigma)\in\mathbb{R}\times\mathbb{R}_{+}\\}\textrm{
and }\varphi_{0}\in\\{t\mapsto\pi/2\arctan(t/\eta):\eta>0\\},$ (diffeo)

and optimize the parameters $(\eta,\mu,\sigma)$. In Section G of Supplementary
Material, we show that this reduces to the following simpler optimization:

$\displaystyle\inf_{T}\,\sup_{|\theta|<\delta}\,n\operatorname{\mathbb{E}}_{\theta}|T(X)-\max(\theta,0)|^{2}\geq\sup_{\xi_{1},\xi_{2}}\,\frac{4n\xi_{2}^{2}\left|\operatorname{\mathbb{E}}\left[\left(1+(\xi_{1}+Z\xi_{2})^{2}\right)^{-1}I(Z>-\xi_{1}/\xi_{2})\right]\right|^{2}}{\pi^{2}\delta^{-2}+4n\operatorname{\mathbb{E}}\left[\left(1+(\xi_{1}+Z\xi_{2})^{2}\right)^{-2}\right]}$

where $Z\overset{d}{=}N(0,1)$ and
$\xi_{1},\xi_{2}\in\mathbb{R}\times\mathbb{R}_{+}$.

The exact local minimax risk of three estimators is considered: the constant
estimator, the plug-in maximum likelihood estimator (MLE), and the plug-in
preliminary-test (pre-test) estimator. Specifically, the plug-in pre-test
estimator is an example of an irregular estimator, which may exhibit super-
efficiency at a Lebesgue measure zero set. We now define the estimator. The
constant estimator returns a predetermined value regardless of the
observation, and we consider the case where $S_{n}^{\text{const}}:=\delta/2$.
This is the best constant estimator that minimizes the local minimax risk,
which is given by

$\displaystyle\sup_{|\theta|<\delta}\,n\operatorname{\mathbb{E}}_{\theta}|S_{n}^{\text{const}}-\max(\theta,0)|^{2}=n\delta^{2}/4.$

The remaining two estimators are defined as follows:

$\displaystyle S_{n}^{\text{plug-in}}$
$\displaystyle:=\max(\widehat{\theta}_{\text{MLE}},0)\quad\text{where}\quad\widehat{\theta}_{\text{MLE}}:=\overline{X}_{n}=\frac{1}{n}\sum_{i=1}^{n}X_{i},\quad\text{and}$
(19) $\displaystyle S_{n}^{\text{pre-test}}$
$\displaystyle:=\max(\widehat{\theta}_{\text{pre-
test}},0)\quad\text{where}\quad\widehat{\theta}_{\text{pre-
test}}:=\begin{cases}\overline{X}_{n}&\text{If}\quad|\overline{X}_{n}|\geq
C_{n}\\\ 0&\text{otherwise}.\end{cases}$ (20)

This family of preliminary-test estimators (Sclove et al., , 1972) is known to
be super-efficient when $\theta_{0}=0$. For example, we consider the case when
$C_{n}=n^{-1/4}$, which is equivalent to the Hodges’ estimator. The following
proposition provides the exact local minimax risks of two estimators.

######  Proposition 5.

Assuming $X_{1},\dots,X_{n}$ are $n$ IID observation from $N(0,1)$ and the
plug-in MLE is defined as (19), the local minimax risk of this estimator is
given by

$\displaystyle\sup_{|\theta|<\delta}\,n\operatorname{\mathbb{E}}_{\theta}|S_{n}^{\mathrm{plug-
in}}-\max(\theta,0)|^{2}=\sup_{0\leq\theta<\delta}\,\left\\{\operatorname{\mathbb{E}}\left[Z^{2}I\left(Z\geq-n^{1/2}\theta\right)\right]+n\theta^{2}P\left(Z<-n^{1/2}\theta\right)\right\\}.$

where $Z\overset{d}{=}N(0,1)$. Similarly for the pre-test estimator defined as
(20), the local minimax risk of this estimator is given by

$\displaystyle\sup_{|\theta|<\delta}\,n\operatorname{\mathbb{E}}_{\theta}|S_{n}^{\mathrm{pre-
test}}-\max(\theta,0)|^{2}$
$\displaystyle\qquad=\sup_{0\leq\theta<\delta}\,\left\\{\operatorname{\mathbb{E}}\left[Z^{2}I\left(Z\geq
n^{1/4}-n^{1/2}\theta\right)\right]+n\theta^{2}P\left(Z<n^{1/4}-n^{1/2}\theta\right)\right\\}.$

Figure 1 shows that for any fixed sample size $n$ as $\delta$ increases, the
lower bounds tend to $\sigma^{2}=1$ and the best among the estimators
considered also has risk tending to $\sigma^{2}$. For any fixed sample size
$n$ and “small” $\delta$, the lower bound is close to zero and the best among
the estimators considered also has risk close to zero. This is expected
because an estimator that is always zero will have zero risk at
$\theta_{0}=0$. Finally, Figure 1 also shows an interesting comparison between
different lower bounds. Neither (vT) nor (diffeo) is a clear winner. Figure 2
shows the same phenomena by fixing $\delta$ and varying the sample size $n$.

Figure 1: The non-asymptotic local minimax lower bounds and the risk given by
different estimators. Figure 2: The non-asymptotic local minimax lower bounds
and the risk given by different estimators.

##  7 Concluding remarks

This manuscript presents new general minimax lower bound techniques for
functional estimation without requiring the differentiability of functionals
or the regularity of statistical models. We focus on the local minimax lower
bounds based on approximation via absolutely continuous functionals (Theprem
8), the extensions of the HCR bound to the mixture model based on two
divergence metrics, the chi-squared divergence (Theorem 9) and the Hellinger
distance (Theorem 10). The minimax lower bound based on the Hellinger distance
is more applicable to irregular estimation problems and only requires mild
regularity conditions. Unlike standard minimax analysis that relies on testing
reduction, this manuscript focuses on preserving a precise asymptotic. The
manuscript provides conditions under which the proposed lower bounds recover
the well-known local asymptotic minimax theorem for both parametric
(Proposition 1) and semiparametric models (Proposition 2) as well as local
minimax results for irregular models (Lemma 13).

The flexibility of the proposed lower bounds offers many potential
applications, especially for non-smooth functionals or estimation under
irregularity. For example, one may consider the minimax lower bound for non-
pathwise differentiable functionals or the irregularity that arises on the
boundary of projection operations. A similar local asymptotic minimax lower
bound was recently derived in the context of plug-in estimators (Fang, ,
2014), but there remain many open problems for general estimators. Finally,
the potential application to the minimax lower bound under non-IID
observations would also be of interest.

This manuscript primarily focuses on the extension of the van Trees inequality
and the HCR bound. However, other Bayes risk lower bounds exist, such as the
Ziv-Zakai bound (Ziv and Zakai, , 1969; Bell et al., , 1997). Recently, Jeong
et al.,  (2023) studied the extension of this bound in the context of
parameter estimation where density does not exist. It would be interesting to
explore the application of Ziv-Zakai bound to non-smooth functionals and its
connection to the semiparametric efficiency theory.

As we discussed in Section 1.2, a precise constant associated with the minimax
lower bound for non-smooth functional estimation was also discovered by Cai
and Low,  (2011) using a slightly different approach. Their technique is based
on the extension of the constrained risk inequality (Brown and Low, , 1996) to
two mixture distributions. Their method is also attractive as two mixture
distributions are constructed based on moment-matching priors, enabling
various analytic tools from approximation theory (Wu and Yang, , 2020). Our
current result only considers a single prior over a parametric path, and it
would be interesting to explore a technique that takes advantage of such
moment-matching priors.

Finally, Levit,  (2010) also discusses the importance of non-asymptotic
constants in the minimax paradigm. In particular, Levit,  (2010) observes that
different lower bounding methods provide sharp constants according to small,
moderate, or large sample sizes. Although the results in this manuscript are
valid non-asymptotically and converge to the correct constant in the limit, we
did not study the sharpness of the derived constants for different finite
sample sizes. While Levit,  (2010) focuses on bounded Gaussian mean
estimations, our lower bounds can be extended to non-asymptotic analysis for
more complex functional estimation problems, which is an important direction
for future work.

## References

  * Abramson, (1982) Abramson, I. S. (1982).  On bandwidth variation in kernel estimates-a square root law.  The annals of Statistics, pages 1217–1223.
  * Anevski and Soulier, (2011) Anevski, D. and Soulier, P. (2011).  Monotone spectral density estimation.  The Annals of Statistics.
  * Assouad, (1983) Assouad, P. (1983).  Deux remarques sur l’estimation.  Comptes rendus des séances de l’Académie des sciences. Série 1, Mathématique, 296(23):1021–1024.
  * Barankin, (1949) Barankin, E. W. (1949).  Locally best unbiased estimates.  The Annals of Mathematical Statistics, 20(4):477–501.
  * Bell et al., (1997) Bell, K. L., Steinberg, Y., Ephraim, Y., and Van Trees, H. L. (1997).  Extended ziv-zakai lower bound for vector parameter estimation.  IEEE Transactions on information theory, 43(2):624–637.
  * Bhattacharyya, (1946) Bhattacharyya, A. (1946).  On some analogues of the amount of information and their use in statistical estimation.  Sankhyā: The Indian Journal of Statistics, pages 1–14.
  * Bickel et al., (1993) Bickel, P. J., Klaassen, C. A., Bickel, P. J., Ritov, Y., Klaassen, J., Wellner, J. A., and Ritov, Y. (1993).  Efficient and adaptive estimation for semiparametric models, volume 4.  Springer.
  * Birgé, (1987) Birgé, L. (1987).  Estimating a density under order restrictions: Nonasymptotic minimax risk.  The Annals of Statistics, pages 995–1012.
  * Brown and Low, (1996) Brown, L. D. and Low, M. G. (1996).  A constrained risk inequality with applications to nonparametric functional estimation.  The annals of Statistics, 24(6):2524–2535.
  * Brown et al., (1997) Brown, L. D., Low, M. G., and Zhao, L. H. (1997).  Superefficiency in nonparametric function estimation.  The Annals of Statistics, 25(6):2607–2625.
  * Cai and Low, (2011) Cai, T. T. and Low, M. G. (2011).  Testing composite hypotheses, hermite polynomials and optimal estimation of a nonsmooth functional.  The Annals of Statistics, pages 1012–1041.
  * Chapman and Robbins, (1951) Chapman, D. G. and Robbins, H. (1951).  Minimum variance estimation without regularity assumptions.  The Annals of Mathematical Statistics, pages 581–586.
  * Chen, (1997) Chen, J. (1997).  A general lower bound of minimax risk for absolute-error loss.  Canadian Journal of Statistics, 25(4):545–558.
  * Collier et al., (2020) Collier, O., Comminges, L., and Tsybakov, A. B. (2020).  On estimation of nonsmooth functionals of sparse normal means.
  * Cramér, (1999) Cramér, H. (1999).  Mathematical methods of statistics, volume 26.  Princeton university press.
  * Devroye and Györfi, (1985) Devroye, L. and Györfi, L. (1985).  Nonparametric Density Estimation: The $L_{1}$ view.  Wiley, New York.
  * Donoho and Liu, (1987) Donoho, D. L. and Liu, R. C. (1987).  Geometrizing rates of convergence.  Preprint.
  * Donoho and Liu, (1991) Donoho, D. L. and Liu, R. C. (1991).  Geometrizing rates of convergence, ii.  The Annals of Statistics, pages 633–667.
  * Duchi and Ruan, (2021) Duchi, J. and Ruan, F. (2021).  A constrained risk inequality for general losses.  In International Conference on Artificial Intelligence and Statistics, pages 802–810. PMLR.
  * Duchi and Ruan, (2018) Duchi, J. C. and Ruan, F. (2018).  The right complexity measure in locally private estimation: It is not the fisher information.  arXiv preprint arXiv:1806.05756.
  * Ernst, (2017) Ernst, P. A. (2017).  Minimizing fisher information with absolute moment constraints.  Statistics & Probability Letters, 129:167–170.
  * Fang, (2014) Fang, Z. (2014).  Optimal plug-in estimators of directionally differentiable functionals.  Unpublished Manuscript.
  * Fisher, (1922) Fisher, R. A. (1922).  On the mathematical foundations of theoretical statistics.  Philosophical transactions of the Royal Society of London. Series A, containing papers of a mathematical or physical character, 222(594-604):309–368.
  * Forchini and Hillier, (2005) Forchini, G. and Hillier, G. H. (2005).  Ill-conditioned problems, fisher information, and weak instruments.  Technical report, cemmap working paper.
  * Gassiat and Stoltz, (2024) Gassiat, E. and Stoltz, G. (2024).  The van trees inequality in the spirit of hajek and le cam.  arXiv preprint arXiv:2402.06431.
  * Gill and Levit, (1995) Gill, R. D. and Levit, B. Y. (1995).  Applications of the van trees inequality: a bayesian cramér-rao bound.  Bernoulli, pages 59–79.
  * Hájek, (1970) Hájek, J. (1970).  A characterization of limiting distributions of regular estimates.  Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 14(4):323–330.
  * Hájek, (1972) Hájek, J. (1972).  Local asymptotic minimax and admissibility in estimation.  In Proceedings of the sixth Berkeley symposium on mathematical statistics and probability, volume 1, pages 175–194.
  * Hall, (1993) Hall, P. (1993).  On plug-in rules for local smoothing of density estimators.  The Annals of Statistics, pages 694–710.
  * Hammersley, (1950) Hammersley, J. M. (1950).  On estimating restricted parameters.  Journal of the Royal Statistical Society. Series B (Methodological), 12(2):192–240.
  * Han et al., (2020) Han, Y., Jiao, J., and Mukherjee, R. (2020).  On estimation of l r-norms in gaussian white noise models.  Probability Theory and Related Fields, 177(3-4):1243–1294.
  * Hencl, (2004) Hencl, S. (2004).  Notes on absolutely continuous functions of several variables.
  * Hirano and Porter, (2012) Hirano, K. and Porter, J. R. (2012).  Impossibility results for nondifferentiable functionals.  Econometrica, 80(4):1769–1790.
  * Ibragimov and Has’minskii, (1981) Ibragimov, I. A. and Has’minskii, R. Z. (1981).  Statistical estimation: asymptotic theory, volume 16.  Springer Science & Business Media.
  * Ibragimov et al., (1987) Ibragimov, I. A., Nemirovskii, A. S., and Khas’ minskii, R. (1987).  Some problems on nonparametric estimation in gaussian white noise.  Theory of Probability & Its Applications, 31(3):391–406.
  * Jeong et al., (2023) Jeong, M., Dytso, A., and Cardone, M. (2023).  Functional properties of the ziv-zakai bound with arbitrary inputs.  arXiv preprint arXiv:2305.02970.
  * Jiao et al., (2015) Jiao, J., Venkat, K., Han, Y., and Weissman, T. (2015).  Minimax estimation of functionals of discrete distributions.  IEEE Transactions on Information Theory, 61(5):2835–2885.
  * Juditsky and Nemirovski, (2002) Juditsky, A. and Nemirovski, A. (2002).  On nonparametric tests of positivity/monotonicity/convexity.  The Annals of Statistics, 30(2):498–527.
  * Jupp, (2010) Jupp, P. E. (2010).  A van trees inequality for estimators on manifolds.  Journal of multivariate analysis, 101(8):1814–1825.
  * Kennedy et al., (2022) Kennedy, E. H., Balakrishnan, S., and Wasserman, L. (2022).  Minimax rates for heterogeneous causal effect estimation.  arXiv preprint arXiv:2203.00837.
  * Kepler, (1609) Kepler, J. (1609).  Astronomia Nova.  Paris, A. Blanchard.
  * Korostelev and Korosteleva, (2011) Korostelev, A. P. and Korosteleva, O. (2011).  Mathematical statistics: asymptotic Minimax theory, volume 119.  American Mathematical Soc.
  * Kuchibhotla et al., (2021) Kuchibhotla, A. K., Patra, R. K., and Sen, B. (2021).  Semiparametric efficiency in convexity constrained single-index model.  Journal of the American Statistical Association, pages 1–15.
  * Le Cam, (1972) Le Cam, L. (1972).  Limits of experiments.  In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, volume 1, pages 245–261. University of California Press Berkeley-Los Angeles.
  * LeCam, (1973) LeCam, L. (1973).  Convergence of estimates under dimensionality restrictions.  The Annals of Statistics, pages 38–53.
  * Lehmann and Casella, (2006) Lehmann, E. L. and Casella, G. (2006).  Theory of point estimation.  Springer Science & Business Media.
  * Lepski et al., (1999) Lepski, O., Nemirovski, A., and Spokoiny, V. (1999).  On estimation of the l r norm of a regression function.  Probability theory and related fields, 113:221–253.
  * Levit, (2010) Levit, B. (2010).  Minimax revisited. i.  Mathematical Methods of Statistics, 19:283–297.
  * Lin et al., (2019) Lin, Y., Martin, R., and Yang, M. (2019).  On optimal designs for nonregular models.  The Annals of Statistics, 47(6):3335–3359.
  * Malỳ, (1999) Malỳ, J. (1999).  Absolutely continuous functions of several variables.  Journal of mathematical analysis and applications, 231(2):492–508.
  * Mou et al., (2022) Mou, W., Wainwright, M. J., and Bartlett, P. L. (2022).  Off-policy estimation of linear functionals: Non-asymptotic theory for semi-parametric efficiency.  arXiv preprint arXiv:2209.13075.
  * Nemirovski, (2000) Nemirovski, A. (2000).  Topics in non-parametric.  Ecole d’Eté de Probabilités de Saint-Flour, 28:85.
  * Pollard, (2023) Pollard, D. (2023+).  Asymptotia (book in progress), Chapter 6 on “Hellinger differentiability”.
  * Polyanskiy and Wu, (2022) Polyanskiy, Y. and Wu, Y. (2022).  Information Theory: From Coding to Learning.  Cambridge University Press.
  * Pötscher, (2002) Pötscher, B. M. (2002).  Lower risk bounds and properties of confidence sets for ill-posed estimation problems with applications to spectral density and persistence estimation, unit roots, and estimation of long memory parameters.  Econometrica, 70(3):1035–1065.
  * Radhakrishna Rao, (1945) Radhakrishna Rao, C. (1945).  Information and accuracy attainable in the estimation of statistical parameters.  Bulletin of the Calcutta Mathematical Society, 37(3):81–91.
  * Sclove et al., (1972) Sclove, S. L., Morris, C., and Radhakrishnan, R. (1972).  Non-optimality of preliminary-test estimators for the mean of a multivariate normal distribution.  The Annals of Mathematical Statistics, pages 1481–1490.
  * Shemyakin, (2014) Shemyakin, A. (2014).  Hellinger distance and non-informative priors.
  * Shen et al., (2020) Shen, Y., Gao, C., Witten, D., and Han, F. (2020).  Optimal estimation of variance in nonparametric regression with random design.
  * Simons and Woodroofe, (1983) Simons, G. and Woodroofe, M. (1983).  The cramér-rao inequality holds almost everywhere.  In Recent Advances in Statistics, pages 69–93. Elsevier.
  * (61) Song, K. (2014a).  Local asymptotic minimax estimation of nonregular parameters with translation-scale equivariant maps.  Journal of Multivariate Analysis, 125:136–158.
  * (62) Song, K. (2014b).  Minimax estimation of nonregular parameters and discontinuity in minimax risk.  In Essays in Honor of Peter CB Phillips, volume 33, pages 557–585. Emerald Group Publishing Limited.
  * Šremr, (2010) Šremr, J. (2010).  Absolutely continuous functions of two variables in the sense of carathéodory.  Electronic Journal of Differential Equations (EJDE)[electronic only], 2010:Paper–No.
  * Tsybakov, (2008) Tsybakov, A. B. (2008).  Introduction to Nonparametric Estimation.  Springer Publishing Company, Incorporated, 1st edition.
  * Uhrmann-Klingen, (1995) Uhrmann-Klingen, E. (1995).  Minimal fisher information distributions with compact-supports.  Sankhyā: The Indian Journal of Statistics, Series A, pages 360–374.
  * Valiant and Valiant, (2011) Valiant, G. and Valiant, P. (2011).  Estimating the unseen: an n/log (n)-sample estimator for entropy and support size, shown optimal via new clts.  In Proceedings of the forty-third annual ACM symposium on Theory of computing, pages 685–694.
  * Van der Vaart, (2000) Van der Vaart, A. W. (2000).  Asymptotic statistics, volume 3.  Cambridge university press.
  * Van der Vaart, (2002) Van der Vaart, A. W. (2002).  Semiparametric statistics.  Lecture Notes in Math.
  * Van Trees, (1968) Van Trees, H. L. (1968).  Detection, estimation, and modulation theory, part I: detection, estimation, and linear modulation theory.  John Wiley & Sons.
  * Weinstein and Weiss, (1988) Weinstein, E. and Weiss, A. J. (1988).  A general class of lower bounds in parameter estimation.  IEEE Trans. Inf. Theory, 34:338–342.
  * Weiss and Weinstein, (1985) Weiss, A. and Weinstein, E. (1985).  A lower bound on the mean-square error in random parameter estimation (corresp.).  IEEE Transactions on information theory, 31(5):680–682.
  * Woodroofe, (1970) Woodroofe, M. (1970).  On choosing a delta-sequence.  The Annals of Mathematical Statistics, 41(5):1665–1671.
  * Wu and Yang, (2019) Wu, Y. and Yang, P. (2019).  Chebyshev polynomials, moment matching, and optimal estimation of the unseen.  The Annals of Statistics, 47(2):857–883.
  * Wu and Yang, (2020) Wu, Y. and Yang, P. (2020).  Polynomial methods in statistical inference: Theory and practice.  Foundations and Trends in Communications and Information Theory, 17(4):402–586.
  * Ziv and Zakai, (1969) Ziv, J. and Zakai, M. (1969).  Some lower bounds on signal parameter estimation.  IEEE transactions on Information Theory, 15(3):386–391.

##  Appendix A Proof of the lower bound based on the approximation

We briefly review the setting and notation to which we frequently refer.
Suppose we observe a random vector $X$ from an unknown distribution
$P_{\theta}$, which belongs to a statistical model
$\mathcal{P}:=\\{P_{t}:t\in\Theta\\}$ defined on a measurable space
$(\mathcal{X},\mathcal{A})$. Here, $\mathcal{A}$ denotes $\sigma$-algebra on
$\mathcal{X}$. Let $\psi:\mathcal{P}\mapsto\mathbb{R}^{k}$ denote the vector-
valued function of interest. In other words, the target estimand is described
as the evaluation of the functional at the population parameter
$\psi(P_{\theta})$.

###  A.1 Proof of Lemma 7

Let $(\Theta,\mathcal{T})$ be a measurable space with respect to the base
measure $\nu$ where $\mathcal{T}$ is a $\sigma$-algebra on $\Theta$. We denote
by $\mathcal{Q}\equiv\mathcal{Q}(\Theta)$ be a collection of probability
measures on $\Theta$ equipped with a density function with respect to $\mu$.
We define $\Phi:=\\{\phi:\mathcal{P}\mapsto\mathbb{R}^{k}\\}$ to be a
collection of arbitrary functionals that maps from $\mathcal{P}$ to
$\mathbb{R}^{k}$. For each $Q\in\mathcal{Q}$ and $\phi\in\Phi$, it follows
that

$\displaystyle\left(\int\operatorname{\mathbb{E}}_{\theta}\|T(X)-\psi(P_{\theta})\|^{2}\,dQ\right)^{1/2}$
$\displaystyle\qquad=\left(\int\operatorname{\mathbb{E}}_{\theta}\|T(X)-\phi(P_{\theta})+\phi(P_{\theta})-\psi(P_{\theta})\|^{2}\,dQ\right)^{1/2}$
$\displaystyle\qquad\geq\left(\int\operatorname{\mathbb{E}}_{\theta}\|T(X)-\phi(P_{\theta})\|^{2}\,dQ\right)^{1/2}-\left(\int\|\phi(P_{\theta})-\psi(P_{\theta})\|^{2}\,dQ\right)^{1/2}$

by the reverse triangle inequality. When the lower bound is negative, we
replace with the trivial lower bound of zero. As the choice of $\phi$ was
arbitrary, we conclude by taking the supremum over $\phi$ such that

$\displaystyle\sup_{\theta\in\Theta}\,\operatorname{\mathbb{E}}_{\theta}\|T(X)-\psi(P_{\theta})\|^{2}$
$\displaystyle\qquad\geq\sup_{\phi\in\Phi,\,Q\in\mathcal{Q}}\,\left[\left(\int\,\operatorname{\mathbb{E}}_{\theta}\left\|T(X)-\phi(P_{\theta})\right\|^{2}\,dQ\right)^{1/2}-\left(\int\|\psi(\theta)-\phi(\theta)\|^{2}dQ(\theta)\right)^{1/2}\right]_{+}^{2}.$

###  A.2 Proof of Theorem 8

Next, we prove Theorem 8 which follows directly by applying the van Trees
inequality. We recall the necessary regularity conditions for the van Trees
inequality according to Gassiat and Stoltz,  (2024). Finally, $q(\theta)\to 0$
as $\theta$ approaches any boundary point of $\Theta_{0}$ with finite norm
along some canonical direction. A collection of probability measures
$\mathcal{Q}^{\dagger}$ supported on $\Theta_{0}$ satisfies Definition 1. Note
that $\mathcal{Q}^{\dagger}$ depends on $\phi$. We then state the following
result:

###### Proof of Theorem 8.