# Parametrization, Prior Independence, and the Semiparametric Bernstein-von
Mises Theorem for the Partially Linear Model

Christopher D. Walker111Harvard University, Department of Economics:
<EMAIL_ADDRESS>I thank Isaiah Andrews, Nick Cao, Kevin Chen, Anna
Mikusheva, Bas Sanders, Neil Shephard, James Stratton, Rahul Singh, and Elie
Tamer for their helpful comments. All errors are mine.

###### Abstract

I prove a semiparametric Bernstein-von Mises theorem for a partially linear
regression model with independent priors for the low-dimensional parameter of
interest and the infinite-dimensional nuisance parameters. My result avoids a
prior invariance condition that arises from a loss of information in not
knowing the nuisance parameter. The key idea is a feasible reparametrization
of the regression function that mimics the Gaussian profile likelihood. This
allows a researcher to assume independent priors for the model parameters
while automatically accounting for the loss of information associated with not
knowing the nuisance parameter. As these prior stability conditions often
impose strong restrictions on the underlying data-generating process, my
results provide a more robust asymptotic normality theorem than the original
parametrization of the partially linear model.

##  1 Introduction

###  1.1 Overview

This paper is concerned with Bayesian inference for a partially linear
regression model. An analyst observes an independent and identically
distributed (i.i.d) sample
$\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n}$ from some probability
distribution $P_{0}$, where $Y_{i}\in\mathbb{R}$ is an outcome variable, and
$X_{i}\in\mathbb{R}$ and $W_{i}\in\mathbb{R}^{d}$ are covariates. The
partially linear model states that the outcome $Y_{i}$ is related to
covariates $X_{i}$ and $W_{i}$ via the regression model

$\displaystyle Y_{i}=X_{i}\beta_{0}+\eta_{0}(W_{i})+U_{i},\quad i=1,...,n,$
(1)

where $\beta_{0}$ is a scalar parameter, $\eta_{0}$ is a function-valued
parameter, and $U_{i}$ is a scalar unobservable that satisfies
$U_{i}|\\{(X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n}\overset{ind}{\sim}\mathcal{N}(0,\sigma_{01}^{2})$
for $i=1,...,n$ and some scalar $\sigma_{01}^{2}\in(0,\infty)$. This
regression model appears in a variety of applications including modeling the
relationship between temperature and electricity demand (e.g., Engle et al.
(1986)), economic models of production with endogenous input choices (e.g.,
Olley and Pakes (1996), Levinsohn and Petrin (2003), Wooldridge (2009),
Ackerberg et al. (2015)), and semiparametric sample selection models (e.g.,
Ahn and Powell (1993), Das et al. (2003)). The partially linear model has also
gained recent theoretical attention as a leading example for which
double/debiased machine learning methods are applied (e.g., Chernozhukov et
al. (2018)).

My analysis views $\beta_{0}$ as the parameter of interest while treating
$\eta_{0}$ as an infinite-dimensional nuisance parameter. A key challenge is
that estimation of the nuisance parameter may severely bias estimates of the
parameter of interest to the extent that desirable properties such as
$\sqrt{n}$-consistency and asymptotic normality are lost. In the Bayesian
context, the common assumption that $\beta$ and $\eta$ are independent
generally leads to posterior asymptotic normality for $\beta$ being subject to
a prior invariance condition that is related to the loss of information
associated with not knowing the nuisance parameter $\eta$. To be concrete, let
$m_{02}$ denote the conditional expectation of $X_{i}$ given $W_{i}$. Since
the ordinary score for $\beta$ is not an efficient score, a change of
variables from $(\beta,\eta)$ to $(\beta,\eta-(\beta-\beta_{0})m_{02})$ must
be applied to obtain asymptotic expansions that are compatible with
semiparametric efficiency. For strict semiparametric models with Gaussian
process priors, Castillo (2012a, b) demonstrated that these transformations
imply that the prior for $\eta$ must simultaneously be suitable for estimation
of the nuisance parameter $\eta_{0}$ and the infinite-dimensional parameter
$m_{02}$ that characterizes the least favorable direction.222See Chapter 25 of
Van der Vaart (1998) for an introduction to semiparametric estimation. The
latter demands that the nuisance prior exhibit a strong lack of sensitivity to
shifts in the least favorable direction and motivates the terminology ‘prior
invariance’. Moreover, these important papers offer evidence that the
condition is essentially necessary for the semiparametric Bernstein-von Mises
theorem and suggest that the condition is hard to verify for general classes
of nonparametric priors (i.e., beyond Gaussian processes). Similar conditions
are encountered in the context of statistical functional estimation (e.g.,
Rivoirard and Rousseau (2012), Castillo and Rousseau (2015), Ray and van der
Vaart (2020)) and lead to a general takeaway that bias may invalidate the
semiparametric Bernstein-von Mises theorem unless strong restrictions are
placed on the prior and/or the statistical model.

Given these concerns, I propose a fully Bayesian inference procedure for the
partially linear model that bypasses prior invariance even with independent
priors for the parameter of interest and the nuisance parameters. The starting
point is an interest-respecting transformation of the regression function

$\displaystyle Y_{i}=m_{01}(W_{i})+[X_{i}-m_{02}(W_{i})]\beta_{0}+U_{i},$ (2)

where $m_{01}(W_{i})$ and $m_{02}(W_{i})$ denote $E_{P_{0}}[Y_{i}|W_{i}]$ and
$E_{P_{0}}[X_{i}|W_{i}]$, respectively. Equation (2) holds because $\eta_{0}$
is nonparametrically identified as $m_{01}-\beta_{0}m_{02}$ and is often
referred to as the ‘Robinson transformation’ because it was first utilized in
Robinson (1988). Next, a robustness property of the Gaussian distribution is
invoked to form a model for the joint distribution of $Y_{i}$ and $X_{i}$
given $W_{i}$ that imposes the Robinson transformation and enables consistent
estimation of the parameter of interest $\beta_{0}$ and the nuisance functions
$m_{0}=(m_{01},m_{02})$. I call this sampling model the
$(\beta,m)$-parametrization while (1) is the $(\beta,\eta)$-parametrization. A
notable property of the $(\beta,m)$-parametrization is that it is adaptive (in
the sense of Bickel (1982)) meaning that there is no cost in not knowing the
nuisance parameter. For that reason, the ordinary score for $\beta$ coincides
with the efficient score in the Gaussian model, and, as a result, a
semiparametric Bernstein-von Mises theorem effectively reduces to local
asymptotic normality (LAN) expansions along locally misspecified parametric
submodels. Since there is no need to change variables, the marginal posterior
for $\beta$ satisfies a Bernstein-von Mises theorem that completely bypasses
the prior invariance condition.

To my knowledge, I present the first Bernstein-von Mises theorem for the
$(\beta,m)$-parametrization of the partially linear model. It has several
important takeaways. First, the findings demonstrate that the choice of
parametrization can influence the conditions under which the semiparametric
Bernstein-von Mises theorem holds. Since the $(\beta,m)$-parametrization is
orthogonal, this lesson is reminiscent of classical results on parametric
likelihood estimation where parameter orthogonality was used to develop
procedures that obtained higher-order improvements (e.g., Cox and Reid
(1987)). The notable difference is that the orthogonality of the
$(\beta,m)$-parametrization has an effect on the first-order asymptotics via
the mitigation of the prior invariance condition. Second, the use of
reparametrization to satisfy prior invariance conditions avoids data-driven
prior adjustments (e.g., Yang et al. (2015), Ray and Szabó (2019), Ray and van
der Vaart (2020), Breunig et al. (2022)), and, for that reason, more closely
adheres to traditional Bayesian analysis. Moreover, the discussion in Section
4 illustrates that my procedure can be implemented using standard Gibbs
sampling techniques. This is a useful observation because one reason for data-
driven prior adjustments is the computational burden associated with
hierarchical priors. Finally, the general theorem is valid even if the
distribution of $Y_{i}$ and $X_{i}$ given $W_{i}$ is not normal despite the
reliance on the Gaussian likelihood for deriving the posterior distribution. A
comparison with the semiparametric efficiency bound in Chamberlain (1992)
leads to the conclusion that my approach enjoys some local efficiency
guarantees under misspecification. The intuition is that the Gaussian
distribution is adequate for estimating parameters that are identified via
mean restrictions because the log-likelihood function resembles a least
squares objective function. This is practically important because researchers
may be concerned about certain violations of normality.

More generally, the results in this paper are suggestive of an approach to
posterior asymptotic normality that bypasses prior invariance and may extend
to a large class of semiparametric models. At the heart of the
$(\beta,m)$-parametrization is profile likelihood (e.g., Severini and Wong
(1992), Murphy and Van der Vaart (2000)). For a fixed value of the low-
dimensional parameter $\beta$, the maximization of the population Gaussian
likelihood with respect to $\eta$ generates the least favorable curve
$\eta_{\beta}=m_{01}-\beta m_{02}$. Consequently, the Robinson transformation
(2) mimics the structure of the Gaussian profile likelihood problem because
the conditional mean $E_{P_{0}}[Y_{i}|X_{i},W_{i}]$ can be rearranged as
$E_{P_{0}}[Y_{i}|X_{i},W_{i}]=X_{i}\beta_{0}+\eta_{\beta_{0}}(W_{i})$. A
parametrization that mimics profile likelihood explicitly incorporates the
behavior of the least favorable parametric submodel, and, as a result, a
researcher can assume independent priors for the model parameters while
simultaneously respecting the semiparametric structure of the model. Although
this paper focuses on the partially linear model, an important direction for
future research is to determine the extent to which this profiling approach
applies to a more general class of semiparametric models.

###  1.2 Related Literature

This paper is related to several active lines of research in statistics and
econometrics. One strand is the literature on semiparametric Bernstein-von
Mises theorems. A semiparametric Bernstein-von Mises theorem for the
$(\beta,\eta)$-parametrization of the partially linear regression model with
independent priors for $\beta$ and $\eta$ is studied in Shen (2002), Bickel
and Kleijn (2012), Yang et al. (2015), and Xie and Xu (2020). Castillo (2012a,
b) emphasized the importance of the prior invariance condition for
semiparametric Bernstein-von Mises theorems. de Jonge and van Zanten (2013)
proved a Bernstein-von Mises theorem for the error standard deviation in a
Gaussian nonparametric regression model. de Jonge and van Zanten’s results are
useful for demonstrating how the assumptions that underpin the general theorem
may be verified for certain classes of priors because there is a connection
between the sampling model and covariance matrix recovery. Other contributions
to this literature include Kim (2006), Rivoirard and Rousseau (2012), Castillo
and Rousseau (2015), Norets (2015), and Chae et al. (2019).

The findings are also related to the literature on semiparametric Bayesian
inference procedures designed for the estimation of low-dimensional target
parameters. Yang et al. (2015) circumvent prior invariance in the
$(\beta,\eta)$-parametrization by shifting the center of the prior for $\eta$
using a suitably consistent frequentist estimator of $m_{02}$. Ray and Szabó
(2019), Ray and van der Vaart (2020), and Breunig et al. (2022) focus on
estimating the average treatment effect (ATE) in a binary treatment potential
outcomes model with a selection-on-observables assumption. These papers debias
the ATE posterior using priors for the treatment and control group conditional
means that depend on plug-in estimators for the propensity score. In a similar
spirit to the prior correction in Yang et al. (2015), a frequentist estimator
of the propensity score shifts the center of the prior for the treatment and
control group conditional means by a scaled estimate of the propensity score
so as to mimic shifts in the direction of the least favorable direction. Hahn
et al. (2018) present simulation evidence in favor of the
$(\beta,m)$-parametrization for the case where $\eta_{0}$, $m_{01}$, and
$m_{02}$ are affine functions of $W_{i}$ and shrinkage priors are used for the
nuisance function coefficients. Yang (2019) combines a different type of
reparametrization and a prior correction to derive a Bernstein-von Mises
theorem for a single coordinate in a high-dimensional linear regression model
under sparsity.

Since Bernstein-von Mises theorems justify the use of Bayesian procedures for
frequentist uncertainty quantification, my paper dovetails some literature on
frequentist semiparametric estimation. The transformed regression model (2)
mimics the profile maximum likelihood problem for (1) under Gaussian errors,
and, as a result, the asymptotic expansions of the posterior are similar to
seminal papers on profile maximum likelihood estimation (e.g., Severini and
Wong (1992), Murphy and Van der Vaart (2000)). Moreover, the
$(\beta,m)$-parametrization was first utilized in Robinson (1988) to derive a
$\sqrt{n}$-consistent and asymptotically normal least squares estimator of
$\beta_{0}$ based on kernel estimators of $m_{01}$ and $m_{02}$. The
connection to the ‘Robinson transformation’ means that the findings are also
related to the literature on two-step semiparametric estimation based on
Neyman orthogonal moment conditions (e.g., Andrews (1994); Newey (1994);
Chernozhukov et al. (2018)).

###  1.3 Outline of Paper

The paper is organized as follows. Section 2 formalizes the data-generating
process and proposes a Bayesian inference procedure targeted toward the
estimation of the low-dimensional parameter of interest. Section 3 derives
conditions under which the marginal posterior for $\beta$ satisfies a
Bernstein-von Mises theorem and uses the normal model with Gaussian process
priors to further illustrate these conditions. Section 4 provides an extensive
discussion of the findings from Section 3. Section 5 concludes. The proofs of
the theorems and propositions are in Appendix A while the proofs of technical
lemmas are in Appendix B.

##  2 Data Generating Process and Bayesian Inference

###  2.1 Data Generating Process

The observed data is an i.i.d sample
$\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n}$ from some distribution
$P_{0}$, where $Y_{i}\in\mathcal{Y}\subseteq\mathbb{R}$,
$X_{i}\in\mathcal{X}\subseteq\mathbb{R}$, and
$W_{i}\in\mathcal{W}\subseteq\mathbb{R}^{d}$ for $i=1,...,n$. It is maintained
that the conditional distribution of $(Y_{i},X_{i})$ given $W_{i}$ has a
density $p_{0}(\cdot|W_{i})$ with respect to the Lebesgue measure. Throughout
the paper, I use $L_{2}(\mathcal{W})$ to denote the space of measurable
functions $f:\mathcal{W}\rightarrow\mathbb{R}$ with
$E_{P_{0}}|f(W_{i})|^{2}<\infty$.

######  Assumption 1.

The distribution of the data $P_{0}$ satisfies the following restrictions:

  1. 1.

$Y_{i}|X_{i},W_{i}\sim\mathcal{N}(X_{i}\beta_{0}+\eta_{0}(W_{i}),\sigma_{01}^{2})$
for
$(\beta_{0},\eta_{0},\sigma_{01}^{2})\in\mathcal{B}\times\mathcal{H}\times\mathbb{R}_{++}$
with $\mathcal{B}\subseteq\mathbb{R}$ and $\mathcal{H}\subseteq
L_{2}(\mathcal{W})$.

  2. 2.

$E_{P_{0}}[(X_{i}-E_{P_{0}}[X_{i}|W_{i}])^{2}]\in[\underline{c},\overline{c}]$
for some $0<\underline{c}<\overline{c}<\infty$.

  3. 3.

$E_{P_{0}}[Y_{i}^{2}],E_{P_{0}}[X_{i}^{2}],E_{P_{0}}[(X_{i}-E_{P_{0}}[X_{i}|W_{i}])^{4}],E_{P_{0}}|\log
p_{0}(Y_{i},X_{i}|W_{i})|\in(0,\infty)$.

I briefly discuss Assumption 1. The first part of the assumption states that
$P_{0}$ is compatible with the homoskedastic Gaussian partially linear model.
For expositional reasons, I assume that $\sigma_{01}^{2}$ is known and discuss
the possibility of unknown $\sigma_{01}^{2}$ in Section 4. The existence of
constants $0<\underline{c}<\overline{c}<\infty$ such that
$E_{P_{0}}[(X_{i}-E_{P_{0}}[X_{i}|W_{i}])^{2}]\in[\underline{c},\overline{c}]$
ensures that $\beta_{0}$ is point identified (Robinson, 1988). The final
restriction is comprised of standard regularity conditions that are sufficient
for the existence of population objective functions and the application of
stochastic limit theorems.

###  2.2 Bayesian Inference

A Bayesian statistical model is comprised of a sampling model and a prior
distribution over the parameter space. I discuss each in turn.

####  2.2.1 Sampling Model

The sampling model is based on the ‘Robinson transformation’ of the regression
function:

$\displaystyle
E_{P_{0}}[Y_{i}|X_{i},W_{i}]=m_{01}(W_{i})+[X_{i}-m_{02}(W_{i})]\beta_{0},$
(3)

where $m_{01}(W_{i})$ and $m_{02}(W_{i})$ denote $E_{P_{0}}[Y_{i}|W_{i}]$ and
$E_{P_{0}}[X_{i}|W_{i}]$, respectively. Equation (3) characterizes an
interest-respecting reparametrization of the partially linear model that was
first applied in Robinson (1988) to derive a $\sqrt{n}$-consistent and
asymptotically normal least squares estimator of $\beta_{0}$. Since $m_{01}$
and $m_{02}$ are treated as free parameters, a model for the joint
distribution of $(Y_{i},X_{i})$ given $W_{i}$ is required to ensure
identifiability of the nuisance functions $m_{0}=(m_{01},m_{02})$. To that
end, the following conditional statistical model serves as the basis for
inference:

$\displaystyle(Y_{i},X_{i})^{\prime}|\\{W_{i}\\}_{i=1}^{n},\beta,m\overset{ind}{\sim}\mathcal{N}(m(W_{i}),V(\beta))$

for $i=1,...,n$, where the parameter $\beta$ is an element of
$\mathcal{B}\subseteq\mathbb{R}$, the parameter $m=(m_{1},m_{2})$ is an
element of $\mathcal{M}\subseteq L_{2}(\mathcal{W})\times L_{2}(\mathcal{W})$
with $\mathcal{M}$ defined so that $m_{0}\in\mathcal{M}$, and
$\mathcal{N}(m(W_{i}),V(\beta))$ is a bivariate normal distribution with mean
$m(W_{i})=(m_{1}(W_{i}),m_{2}(W_{i}))^{\prime}$ and covariance matrix

$\displaystyle
V(\beta)=\begin{pmatrix}\sigma_{01}^{2}+\beta^{2}\sigma_{02}^{2}&\beta\sigma_{02}^{2}\\\
\beta\sigma_{02}^{2}&\sigma_{02}^{2}\end{pmatrix}$

with $\sigma_{01}^{2}\in(0,\infty)$ defined in Assumption 1 and
$\sigma_{02}^{2}\in(0,\infty)$ being a known scalar. Under i.i.d sampling, the
likelihood function is

$\displaystyle L_{n}(\beta,m)=\prod_{i=1}^{n}p_{\beta,m}(Y_{i},X_{i}|W_{i}),$

where $p_{\beta,m}(Y_{i},X_{i}|W_{i})$ is the density of
$\mathcal{N}(m(W_{i}),V(\beta))$.

There are several features of the sampling model that are worth highlighting.
The covariance matrix $V(\beta)$ is chosen so that the model-implied
distribution of $Y_{i}$ given $(X_{i},W_{i}^{\prime})^{\prime}$ is the
Gaussian distribution
$\mathcal{N}(m_{1}(W_{i})+(X_{i}-m_{2}(W_{i}))\beta,\sigma_{01}^{2})$ for
every $(\beta,m)$-pair. As such, the sampling model imposes the Robinson
transformation. Combining this with the model implication that $X_{i}$ given
$W_{i}$ is distributed according to
$\mathcal{N}(m_{2}(W_{i}),\sigma_{01}^{2})$ for every $(\beta,m)$-pair, I
invoke a well-known robustness property of the Gaussian likelihood to conclude
that the model is suitable for consistent estimation of $\beta_{0}$ even if
the conditional density $p_{0}(\cdot|W_{i})$ is not Gaussian (i.e., due to
non-normality of the distribution of $X_{i}$ given $W_{i}$). Theorem 1
verifies this claim. In fact, the proof of the theorem does not rely on the
normality assumption
$Y_{i}|X_{i},W_{i}\sim\mathcal{N}(X_{i}\beta_{0}+\eta_{0}(W_{i}),\sigma_{01}^{2})$
but only that the conditional mean satisfies
$E_{P_{0}}[Y_{i}|X_{i},W_{i}]=X_{i}\beta_{0}+\eta_{0}(W_{i})$ with
$\eta_{0}(W_{i})=m_{01}(W_{i})-\beta_{0}m_{02}(W_{i})$. This leads to a
stronger conclusion that the sampling model is appropriate for consistent
estimation of $\beta_{0}$ even when the Gaussian model for the distribution of
$Y_{i}$ given $X_{i}$ and $W_{i}$ is misspecified.

######  Theorem 1.

Suppose that Assumption 1 holds. Then, for any $\beta\in\mathcal{B}$ fixed,
$m_{0}=(m_{01},m_{02})$ is the unique solution to the minimization problem

$\displaystyle\min_{m\in\mathcal{M}}E_{P_{0}}\left[KL\left(p_{0}(\cdot|W_{i}),p_{\beta,m}(\cdot|W_{i})\right)\right],$

where $KL\left(p_{0}(\cdot|W_{i}),p_{\beta,m}(\cdot|W_{i}))\right)$ is the
Kullback-Leibler divergence divergence between $p_{0}(\cdot|W_{i})$ and
$p_{\beta,m}(\cdot|W_{i})$. Moreover, $(\beta_{0},m_{0})$ is the unique
solution to the minimization problem

$\displaystyle\min_{(\beta,m)\in\mathcal{B}\times\mathcal{M}}E_{P_{0}}\left[KL\left(p_{0}(\cdot|W_{i}),p_{\beta,m}(\cdot|W_{i})\right)\right].$

Theorem 1 also implies that the sampling model is engineered for efficient
estimation of $\beta_{0}$. Since the conditional mean $m_{0}$ maximizes the
population log-likelihood function for any fixed value of $\beta$, the
collection of densities
$\\{p_{\beta,m_{0}}(\cdot|X_{i},W_{i}):\beta\in\mathcal{B}\\}$ characterizes
the least favorable parametric submodel. Here
$p_{\beta,m_{0}}(\cdot|X_{i},W_{i})$ is the probability density function of
$\mathcal{N}(m_{01}(W_{i})+(X_{i}-m_{02}(W_{i}))\beta,\sigma_{01}^{2})$. The
least favorable parametric submodel indicates that there is no loss of
information associated with not knowing the nuisance parameter meaning that
the $(\beta,m)$-parametrization is adaptive in the sense of Bickel (1982).
This implies that the model is targeted towards efficient estimation of
$\beta_{0}$ because the ordinary derivative $\tilde{\ell}_{n}(\beta,m)$ of the
log-likelihood function $\ell_{n}(\beta,m)=\log L_{n}(\beta,m)$ satisfies
$\tilde{\ell}_{n}(\beta,m)=\sum_{i=1}^{n}\tilde{\ell}(Y_{i},X_{i},W_{i},\beta,m)$,
where
$\tilde{\ell}(Y_{i},X_{i},W_{i},\beta,m)=(Y_{i}-m_{1}(W_{i})-(X_{i}-m_{2}(W_{i}))\beta)(X_{i}-m_{2}(W_{i}))/\sigma_{01}^{2}$
is the efficient score for the homoskedastic Gaussian partially linear model.

####  2.2.2 Prior and Posterior

The Bayesian model is completed with a prior distribution over the parameter
space. Besides some regularity conditions, the key restriction for the prior
is that the parameter of interest $\beta$ and the nuisance functions $m$ are
statistically independent. Formally, I endow the parameter space
$\mathcal{B}\times\mathcal{M}$ with a product probability measure
$\Pi=\Pi_{\mathcal{B}}\times\Pi_{\mathcal{M}}$ that defined on a
$\sigma$-algebra that contains the Borel $\sigma$-algebra generated by the
product topology.

Given the prior and the sampling model, the analyst draws inferences about the
parameter using the posterior distribution
$\Pi((\beta,m)\in\cdot|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$.
Since the Gaussian model is dominated, the posterior distribution can be
expressed using Bayes formula

$\displaystyle\Pi((\beta,m)\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})=\int_{A}L_{n}(\beta,m)d\Pi(\beta,m)\bigg{/}\int_{\mathcal{B}\times\mathcal{M}}L_{n}(\beta,m)d\Pi(\beta,m),$

where $A$ is any measurable set. Section 3 studies the large sample properties
of the posterior distribution while Section 4 provides some discussion about
how to obtain samples from the posterior distribution using a default class of
priors for the nuisance functions.

##  3 Bernstein-von Mises Theorem

###  3.1 Main Result

I list sufficient conditions for a semiparametric Bernstein-von Mises theorem
for data-generating processes compatible with Assumption 1. To that end, it is
useful to define the product supremum norm
$||m||_{\mathcal{M},\infty}=\max\\{||m_{1}||_{\infty},||m_{2}||_{\infty}\\}$
for any $m\in\mathcal{M}$, where $||f||_{\infty}=\sup_{w\in\mathcal{W}}|f(w)|$
for any measurable $f:\mathcal{W}\rightarrow\mathbb{R}$.

######  Assumption 2.

The following conditions hold:

  1. 1.

$\int_{\mathcal{B}\times\mathcal{M}}\exp(\ell_{n}(\beta,m))d\Pi(\beta,m)<\infty$
with $P_{0}$-probability equal to one.

  2. 2.

$\Pi_{\mathcal{B}}$ has a density $\pi_{\mathcal{B}}$ (with respect to
Lebesgue measure) that is continuous and positive over a neighborhood
$\mathcal{B}_{0}$ of $\beta_{0}$.

######  Assumption 3.

For each $\delta>0$, let
$B_{\mathcal{M},\infty}(m_{0},\delta)=\\{m\in\mathcal{M}:||m-m_{0}||_{\mathcal{M},\infty}<\delta\\}$.
There exists sets $\\{\mathcal{M}\\}_{n=1}^{\infty}$ with
$\mathcal{M}_{n}\subseteq\mathcal{M}$ and a sequence
$\\{\delta_{n}\\}_{n=1}^{\infty}$ with $\delta_{n}\downarrow 0$ such that the
following holds as $n\rightarrow\infty$:

  1. 1.

$E_{P_{0}}\left(\Pi\left(\mathcal{M}_{n}^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n}\right)\right)\rightarrow
0$ as $n\rightarrow\infty$.

  2. 2.

$E_{P_{0}}\left(\Pi\left(\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})^{c}\middle|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n}\right)\right)\rightarrow
0$ for some large constant $D>0$.

######  Assumption 4.

The sequence $\\{\delta_{n}\\}_{n=1}^{\infty}$ satisfies
$\delta_{n}=o(n^{-\frac{1}{4}})$.

######  Assumption 5.

Let $G_{n}^{(1)}:\mathcal{M}\rightarrow\mathbb{R}$ and
$G_{n}^{(2)}:\mathcal{M}\rightarrow\mathbb{R}$ be empirical processes given by

$\displaystyle
G_{n}^{(1)}(m)=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\varepsilon_{i2}(m_{1}(W_{i})-m_{01}(W_{i}))$

and

$\displaystyle
G_{n}^{(2)}(m)=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}(U_{i}-\beta_{0}\varepsilon_{i2})(m_{2}(W_{i})-m_{02}(W_{i})),$

where $\varepsilon_{i2}=X_{i}-m_{02}(W_{i})$ and
$U_{i}=Y_{i}-m_{01}(W_{i})-(X_{i}-m_{02}(W_{i}))\beta_{0}$. The following
conditions hold:

$\displaystyle E_{P_{0}}\left(\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\left|G_{n}^{(1)}(m)\right|\right)\longrightarrow
0$

and

$\displaystyle E_{P_{0}}\left(\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\left|G_{n}^{(2)}(m)\right|\right)\longrightarrow
0$

as $n\rightarrow\infty$.

A discussion of Assumptions 2–5 is provided. Assumption 2 imposes some
regularity conditions on the prior. The first part ensures that the posterior
distribution is well-defined while the second states that the prior for
$\beta$ has a density satisfying some continuity and positivity restrictions.
Restrictions like Part 2 of Assumption 2 are routinely applied for proving
semiparametric Bernstein-von Mises theorems (see Chapter 12 of Ghosal and Van
der Vaart (2017)). Assumption 3 states that the nonparametric part of the
posterior concentrates its mass on sets of the form $\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})$ while Assumption 4 restricts the
nuisance posterior contraction rate $\delta_{n}$. Requiring that the posterior
is consistent in the supremum norm topology is overly strong. The proof of the
Bernstein-von Mises theorem requires that $\sqrt{n}\langle
m_{1}-m_{01},m_{2}-m_{02}\rangle_{n,2}=o_{P_{0}}(1)$ and
$\sqrt{n}||m_{2}-m_{02}||_{n,2}^{2}=o_{P_{0}}(1)$ uniformly over a shrinking
neighborhood of $m_{0}$, where $\langle\cdot,\cdot\rangle_{n,2}$ and
$||\cdot||_{n,2}$ are the empirical $L_{2}$ inner product and induced norm,
respectively.333For any measurable $f,g:\mathcal{W}\rightarrow\mathbb{R}$,
define $\langle f,g\rangle_{n,2}=\frac{1}{n}\sum_{i=1}^{n}f(W_{i})g(W_{i})$
and $||f||_{n,2}^{2}=\langle f,f\rangle_{n,2}$. An application of the Cauchy-
Schwarz inequality indicates that Assumption 3 could be stated in terms of the
product empirical $L_{2}$-norm (i.e.,
$\max\\{||m_{1}||_{n,2},||m_{2}||_{n,2}\\}$), however, I elect to work with a
stronger norm to eliminate the dependence on the data. Finally, Assumption 5
concerns the complexity of the sets $\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})$ on which the posterior concentrates
because it is effectively a stochastic equicontinuity restriction on the
sequences of empirical processes $\\{G_{n}^{(1)}\\}_{n=1}^{\infty}$ and
$\\{G_{n}^{(2)}\\}_{n=1}^{\infty}$. Asymptotic equicontinuity conditions are
typically required for large sample analyses of semiparametric likelihood
estimators. For instance, Murphy and Van der Vaart (2000) assume that the
score functions are Donsker in a neighborhood of the true parameter to prove
large sample properties of profile maximum likelihood estimators. More
recently, Ray and van der Vaart (2020) impose stochastic equicontinuity to
derive a Bernstein-von-Mises theorem for the ATE under selection-on-
observables (see (3.5) and (3.12) in their paper).

Combined with Assumption 1, Assumptions 2–5 imply the following theorem.

######  Theorem 2.

Let $\tilde{I}_{n}:\mathcal{M}\rightarrow\mathbb{R}$ be given by

$\displaystyle\tilde{I}_{n}(m)=\frac{1}{n}\sum_{i=1}^{n}\left(\frac{X_{i}-m_{2}(W_{i})}{\sigma_{01}}\right)^{2}$

and let $\tilde{\Delta}_{n,0}$ satisfy

$\displaystyle\tilde{\Delta}_{n,0}=\tilde{I}_{n}(m_{0})^{-1}\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0}).$

If Assumptions 1, 2, 3, 4, and 5 hold and
$\beta_{0}\in\text{int}(\mathcal{B})$, then

$\displaystyle\left|\left|\Pi\left(\beta\in\cdot|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n}\right)-\mathcal{N}\left(\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}\right)\right|\right|_{TV}\overset{P_{0}}{\longrightarrow}0,$

as $n\rightarrow\infty$, where $||\cdot||_{TV}$ is the total variation
distance.

I offer some intuition for Theorem 2. The argument is similar to two-step
semiparametric estimation. Assumption 3 guarantees that $\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$ equals

$\displaystyle\int_{\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})+o_{P_{0}}(1)$
(4)

for all events $A$, where

$\displaystyle\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)=\frac{\int_{A}L_{n}(\beta,m)d\Pi_{\mathcal{B}}(\beta)}{\int_{\mathcal{B}}L_{n}(\beta,m)d\Pi_{\mathcal{B}}(\beta)}$

is the posterior in the $m$-indexed parametric submodel
$\\{\bigotimes_{i=1}^{n}\mathcal{N}(m(W_{i}),V(\beta)):\beta\in\mathcal{B}\\}$
with prior $\Pi_{\mathcal{B}}$. Consequently, the proof reduces to showing
$\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)$ is asymptotically
equivalent to the desired normal distribution. This is analogous to the final
stage of two-step estimation because the parametric problem is essentially the
recovery of the low-dimensional parameter of interest under local
perturbations of the nuisance functions. From that perspective, Assumptions 4
and 5 essentially do the rest of the ‘hard work’. First, the conditions allow
me to show that for any sequence $M_{n}\rightarrow\infty$,

$\displaystyle\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi(\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)\overset{P_{0}}{\rightarrow}0,$
(5)

where
$B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})=\\{\beta\in\mathcal{B}:|\beta-\beta_{0}|<M_{n}/\sqrt{n}\\}$.
Second, Assumptions 4 and 5 are sufficient for the local asymptotic normality
(LAN) condition

$\displaystyle\ell_{n}(\beta,m)-\ell_{n}(\beta_{0},m)=\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}+o_{P_{0}}(1)$
(6)

for all $(\beta,m)\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})\times(\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n}))$ with $M_{n}\rightarrow\infty$
arbitrarily slowly. Combined with (4) and Part 2 of Assumption 2, the
$\sqrt{n}$-consistency condition (5) and LAN condition (6) guarantee that
$\Pi(\beta\in A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$ is
asymptotically equivalent to

$\displaystyle\frac{\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}\right)d\beta}{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}\right)d\beta}$
(7)

for all events $A$. Since $M_{n}\rightarrow\infty$, it is possible to show
that (7) is asymptotically equivalent to the desired normal distribution in
total variation and we obtain the conclusion of Theorem 2. The result implies
that Bayesian credible sets are asymptotically efficient confidence sets when
the true distribution of $Y_{i}$ given $X_{i}$ and $W_{i}$ is
$\mathcal{N}(X_{i}\beta_{0}+\eta_{0}(W_{i}),\sigma_{01}^{2})$. The theorem
also has some efficiency implications in settings where the conditional
distribution of $Y_{i}$ given $X_{i}$ and $W_{i}$ is non-Gaussian. I discuss
the important issue of model misspecification in Section 4.

###  3.2 Specialization to Gaussian Process Priors

An example is provided to offer a lower-level depiction of the conditions that
imply the Bernstein-von Mises theorem. I maintain that the conditional
probability density function $p_{0}(\cdot|W_{i})$ is that of the Gaussian
distribution $\mathcal{N}(m_{0}(W_{i}),V(\beta_{0}))$ and that the $W_{i}$ has
a density on $[0,1]$ with respect to the Lebesgue measure that is bounded away
from zero and infinity. The normality assumption is useful because the problem
of learning $\beta$ in the $(\beta,m)$-parametrization becomes covariance
matrix recovery in a random design multivariate Gaussian regression model.
Consequently, I can borrow ideas from de Jonge and van Zanten (2013) to
provide a lower-level discussion of the assumptions that underpin Theorem 2.
They derive a Bernstein-von Mises theorem for the error standard deviation in
a fixed design Gaussian nonparametric regression model, and, as a result, the
problem of learning $\beta$ in the $(\beta,m)$-parametrization is essentially
an extension of the problem encountered in their paper to the setting where
the covariates are random and there are multiple regression functions to
estimate. Although strong relative to Assumption 1, I expect the practical
lessons from this discussion to be relatively unchanged with a relaxation of
normality based on the conclusion of Theorem 1 and the results from Kleijn and
van der Vaart (2006) on misspecified nonparametric regression models.

In line with Section 2, I impose $\beta\sim\Pi_{\mathcal{B}}$,
$m\sim\Pi_{\mathcal{M}}$, and $\beta\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}m$ so that the prior for $(\beta,m)$ is
$\Pi=\Pi_{\mathcal{B}}\times\Pi_{\mathcal{M}}$. I assume that
$\Pi_{\mathcal{B}}$ is a distribution supported on a compact subset
$\mathcal{B}$ of $\mathbb{R}$ and admits a strictly positive and continuous
Lebesgue density $\pi_{\mathcal{B}}$. An example is the uniform distribution
on $\mathcal{B}$. I set $\Pi_{\mathcal{M}}=\text{P}_{1}\times\text{P}_{2}$,
where $\text{P}_{j}$ is the law of a suitably smooth mean-zero Gaussian
process in the space of continuous functions $(C([0,1]),||\cdot||_{\infty})$.
The specific prior smoothness restriction is that draws from $\text{P}_{j}$
are $\alpha_{j}$-regular for some $\alpha_{j}>1/2$ in the sense that they take
values in the Hölder space $(C^{a_{j}}([0,1]),||\cdot||_{a_{j}})$ for all
$a_{j}\in[0,\alpha_{j})$ and that the reproducing kernel Hilbert space (RKHS)
associated with the Gaussian process $\text{P}_{j}$, denoted
$(\mathbb{H}_{j},||\cdot||_{\mathbb{H}_{j}})$, has a unit ball
$\mathbb{H}_{j,1}$ with metric entropy $\log
N(\tau,\mathbb{H}_{j,1},||\cdot||_{a_{j}})$ that satisfies the inequality
$\log
N(\tau,\mathbb{H}_{j,1},||\cdot||_{a_{j}})\lesssim\tau^{-2/(1+2(\alpha_{j}-a_{j}))}$
for any $\tau>0$.444A function $f:[0,1]\rightarrow\mathbb{R}$ belongs the the
Hölder space $(C^{a}([0,1]),||\cdot||_{a})$ if and only if
$||f||_{a}=\max_{k\in\mathbb{Z}_{+}:|k|\leq\underline{a}}\sup_{w\in[0,1]}|D^{k}f(w)|+\max_{k\in\mathbb{Z}_{+}:|k|=\underline{a}}\sup_{w,w^{\prime}\in[0,1]:w\neq
w^{\prime}}\frac{|D^{k}f(w)-D^{k}f(w^{\prime})|}{|w-w^{\prime}|^{a-\underline{a}}}<\infty$,
where $\underline{a}$ denotes the largest integer strictly smaller than $a$,
and $D^{k}$ denotes the differential operator (i.e.,
$D^{k}f=d^{k}f(w)/dw^{k}$). Mean-zero Gaussian processes that satisfy these
restrictions include Matérn processes, integrated Brownian motion released at
zero, and Riemann-Liouville processes released at zero (see Chapter 11 of
Ghosal and Van der Vaart (2017) for precise definitions).

For Gaussian process priors, the contraction rate $\delta_{n}$ is typically
obtained via the concentration function. Since $\text{P}_{1}$ and
$\text{P}_{2}$ are the laws of Gaussian process, the product
$\text{P}=\text{P}_{1}\times\text{P}_{2}$ is also the law of a Gaussian
process. Consequently, the Gaussian process $\\{m(w):w\in[0,1]\\}$ has an
associated RKHS $(\mathbb{H},||\cdot||_{\mathbb{H}})$ and takes values in
$(C([0,1])\times C([0,1]),||\cdot||_{\mathcal{M},\infty})$. The concentration
function $\varphi_{m_{0}}$ measures the mass that P places around $m_{0}$ and
satisfies

$\displaystyle\varphi_{m_{0}}(\delta)=\inf_{h\in\mathbb{H}:||h-m_{0}||_{\mathcal{M},\infty}<\delta}\frac{1}{2}||h||_{\mathbb{H}}^{2}-\log\text{P}\left(||m||_{\mathcal{M},\infty}<\delta\right)$

for $\delta>0$. General results on posterior contraction rates with Gaussian
process priors state that if a sequence $\\{\delta_{n}\\}_{n=1}^{\infty}$
satisfies $\delta_{n}\rightarrow 0$, $n\delta_{n}^{2}\rightarrow\infty$, and
$\varphi_{m_{0}}(\delta_{n})\leq n\delta_{n}^{2}$, then the posterior
concentrates around the true parameter at rate $\delta_{n}$ (see Theorem 2.1
of van der Vaart and van Zanten (2008a)). This motivates a preliminary
posterior contraction assumption, which may be interpreted as the
specialization of Assumption 3 to Gaussian process priors.

######  Assumption 6.

Let $\\{\delta_{n}\\}_{n=1}^{\infty}$ be a sequence such that
$\delta_{n}\rightarrow 0$ as $n\rightarrow\infty$,
$n\delta_{n}^{2}\rightarrow\infty$ as $n\rightarrow\infty$, and
$\varphi_{m_{0}}(\delta_{n})\leq n\delta_{n}^{2}$ for large $n$. The posterior
satisfies
$\Pi(B_{\mathcal{M},\infty}(m_{0},D\delta_{n})^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\overset{P_{0}}{\rightarrow}0$
as $n\rightarrow\infty$ under the prior
$\Pi=\Pi_{\mathcal{B}}\times\Pi_{\mathcal{M}}$ described above.

Some further justification for Assumption 6 is provided. van der Vaart and van
Zanten (2011) use the supremum norm concentration function inequality to
derive $L_{2}([0,1])$-contraction rates for random design regression with
Gaussian errors and a Hölder smooth Gaussian process prior for the regression
function. Since the $(\beta,m)$-parametrization corresponds to a multivariate
random design regression with Gaussian errors and the Gaussian process priors
used to construct $\Pi_{\mathcal{M}}$ take values in Hölder space, I expect
the theorems from that paper to extend to this setting to obtain a contraction
rate $\overline{\delta}_{n}$ with respect to the product $L_{2}$-pseudometric
$||\cdot||_{\mathcal{M},2}=\max\\{||\cdot||_{L_{2([0,1])}},||\cdot||_{L_{2}([0,1])}\\}$
so long as the true regression $m_{0}$ satisfies $m_{0j}\in
C^{\alpha_{0,j}}([0,1])$ for $\alpha_{0,j}>1/2$ and $j\in\\{1,2\\}$.555Here
$||f||_{L_{2}([0,1])}^{2}=\langle f,f\rangle_{2}$ and $\langle
f,g\rangle_{2}=E_{P_{0}}[f(W_{i})g(W_{i})]$ for
$f,g:[0,1]\rightarrow\mathbb{R}$. Proposition 2 of the supplement to Castillo
and Rousseau (2015) then provides a method of extending $L_{2}$-norm posterior
contraction rates to supremum norm rates when a Gaussian process prior with
Hölder smooth sample paths is assumed. The intuition is that $m_{j}-m_{0j}$
can be viewed as an element of the separable Hilbert space
$(L_{2}([0,1]),||\cdot||_{L_{2}([0,1])})$, and, as a result, admits an
abstract Fourier series representation in terms of the Cohen-Daubechies-Vial
(CDV) wavelet basis (see Cohen et al. (1993) for definitions). Localization
properties of the CDV wavelet system and the Hölder smoothness conditions are
then combined to obtain a slower supremum norm rate of convergence
$\delta_{n}$ from a preliminary $L_{2}([0,1])$\- contraction rate
$\overline{\delta}_{n}$. Although Castillo and Rousseau’s argument may result
in a suboptimal uniform contraction rate, this is not necessarily a concern
because I only require that $\delta_{n}=o(n^{-1/4})$. I also note that Lemma 3
of Castillo (2008) implies that the resulting uniform rate $\delta_{n}$
satisfies $\varphi_{m_{0}}(\delta_{n})\leq n\delta_{n}^{2}$ for large $n$
because $\delta\mapsto\varphi_{m_{0}}(\delta)$ is strictly decreasing for
$\delta>0$.

I describe the sieves $\\{\mathcal{M}_{n}\\}_{n=1}^{\infty}$. Under the
smoothness restrictions of the prior, a candidate sieve is
$\mathcal{M}_{n}=\mathcal{M}_{n1}\times\mathcal{M}_{n2}$, where
$\mathcal{M}_{nj}=M_{j}\sqrt{n}\delta_{n}\mathbb{H}_{j,1}+\gamma_{n,j}C_{1}^{a_{j}}([0,1])$,
$C_{1}^{a_{j}}([0,1])$ is the unit ball in the Hölder space
$(C^{a_{j}}([0,1]),||\cdot||_{a_{j}})$ for some $a_{j}\in(1/2,\alpha_{j})$,
$\gamma_{n,j}=n^{-(\alpha_{j}-a_{j})/(2\alpha_{j}+1)}$, and $M_{j}>0$ is a
sufficiently large constant for each $j\in\\{1,2\\}$. The sets
$\mathcal{M}_{n}$ extend those encountered in Theorem 3 of de Jonge and van
Zanten (2013) to the case where there are multiple regression functions to
estimate. Lemma 3 formally proves that
$\Pi(\mathcal{M}_{n}^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\overset{P_{0}}{\rightarrow}0$
when combined with the conditions of Assumption 6. Consequently, this lemma
and Assumption 6 are sufficient for Assumption 3 under this class of priors. I
then recognize that, conditional on $\\{W_{i}\\}_{i=1}^{n}$, the empirical
processes $G_{n}^{(1)}$ and $G_{n}^{(2)}$ are Gaussian processes, and, as a
result, a standard maximal inequality for sub-Gaussian processes from van der
Vaart and Wellner (1996) and the assumption that $\delta_{n}=o(n^{-1/4})$
combine to verify Assumption 5. Lemma 4 provides a formal statement of this
result. Putting this all together, I arrive at the following proposition.

######  Proposition 1.

Suppose that $p_{0}(\cdot|W_{i})$ is that of
$\mathcal{N}(m_{0}(W_{i}),V(\beta_{0}))$ and the prior $\Pi$ satisfies the
conditions above. Then Assumption 6 with $\delta_{n}=o(n^{-1/4})$ implies the
conclusion of Theorem 2.

##  4 Discussion of the Results

###  4.1 Comparison of Parametrizations and Adaptation

I compare the two parametrizations under prior independence. Let
$q_{\beta,\eta}(\cdot|X_{i},W_{i})$ denote the density function of
$\mathcal{N}(X_{i}\beta+\eta(W_{i}),\sigma_{01}^{2})$. Bayesian inference for
the $(\beta,\eta)$-parametrization typically uses
$q_{\beta,\eta}(\cdot|X_{i},W_{i})$ to form a likelihood function and assumes
$\beta\sim\Pi_{\mathcal{B}}$, $\eta\sim\Pi_{\mathcal{H}}$, and
$\beta\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}\eta$ so that the joint prior for
$(\beta,\eta)$ is $\Pi_{\mathcal{B}}\times\Pi_{\mathcal{H}}$ (e.g., Section 7
of Bickel and Kleijn (2012)).666It suffices to focus on the conditional
distribution of $Y_{i}$ given $W_{i}$ and $X_{i}$ because a factorization of
the likelihood and prior independence implies that $(\beta,\eta)$ is a
posteriori independent to the parameters that index the distribution of
$X_{i}$ given $W_{i}$. A conceptual challenge is that the
$(\beta,\eta)$-parametrization is a semiparametric model with information loss
in that the efficient Fisher information does not coincide with the ordinary
Fisher information in the associated parametric model. Consequently, a change
of variables must be applied to account for the loss of information associated
with not knowing the nuisance parameter $\eta_{0}$ to obtain asymptotic
expansions compatible with semiparametric efficiency.

This change of variables means that a researcher is confronted with a prior
invariance condition that is sensitive to the fine details of the prior and
statistical model. If $\Pi_{\mathcal{H}}$ is the law of a mean-zero Gaussian
process, then the results in Section 1.4 of Castillo (2012a) suggest a change
of variables $(\beta,\eta)\mapsto(\beta,\eta-(\beta-\beta_{0})m_{n2})$, where
$\\{m_{n2}\\}_{n=1}^{\infty}$ is a sequence in the associated RKHS that
approximates the conditional expectation $m_{02}$. Consequently, the nuisance
prior $\Pi_{\mathcal{H}}$ must be suitable for both the estimation of
$\eta_{0}$ and the approximation of the least favorable direction $m_{02}$.
Requiring the RKHS to approximate $m_{02}$ well has important practical
implications because the RKHS may form a relatively small class of functions.
Consequently, a researcher may need to impose strong restrictions on the model
and/or the prior distribution to achieve asymptotic normality. Similar
concerns have been documented in other papers. For instance, Ray and van der
Vaart (2020) argue that the prior stability requirement may impose harsh
smoothness conditions on the propensity score in a heterogeneous treatment
effects model. Moreover, there is evidence that prior invariance conditions
can pose challenges for Bayesian procedures based on priors that estimate the
unknown regularity of the nuisance functions (e.g., Rivoirard and Rousseau
(2012), Castillo and Rousseau (2015), Yang et al. (2015)).

Theorem 1 establishes that the $(\beta,m)$-parametrization is adaptive because
the least favorable parametric submodel is $\beta\mapsto
p_{\beta,m_{0}}(\cdot|X_{i},W_{i})$. Consequently, there is no loss of
information in not knowing the nuisance parameter in the
$(\beta,m)$-parametrization and the semiparametric Bernstein-von Mises theorem
effectively reduces to an ordinary LAN condition along locally misspecified
parametric models $\beta\mapsto p_{\beta,m}(\cdot|X_{i},W_{i})$. This leads to
two conditions for posterior asymptotic normality: 1. a preliminary posterior
consistency requirement, and 2. a stochastic equicontinuity restriction. These
should be interpreted as a reflection of the semiparametric model rather than
the Bayesian approach because similar conditions are encountered in
frequentist treatments of the partially linear model (e.g., Chernozhukov et
al. (2018)). There is no prior invariance requirement at all because the
$(\beta,m)$-parametrization automatically incorporates the structure of the
least favorable submodel. For this reason, Theorem 2 can be interpreted as
providing a more robust semiparametric Bernstein-von Mises theorem for the
partially linear model.

Finally, it is worth discussing priors that estimate the unknown regularity of
the nuisance functions given the earlier discussion. I conjecture that my
approach can handle such priors. Since the example in the previous section
essentially extends Section 3 of de Jonge and van Zanten (2013) to the case
with two regression functions, it is plausible that a similar exercise may be
applied to the hierarchical spline priors in Section 4 of their paper.
Assuming $m_{1}\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}m_{2}$, the stochastic equicontinuity
restriction would be verified using
$\mathcal{M}_{n}=\mathcal{M}_{n1}\times\mathcal{M}_{n2}$ with
$\mathcal{M}_{nj}$ constructed in a similar manner to $\mathcal{F}_{n}$ in
Theorem 4.1 of their paper. Consequently, a technical extension of the
contraction rate results from de Jonge and van Zanten (2012) to multiple
functions and the product supremum norm delivers a similar conclusion to
Proposition 1 for this class of priors. Since these priors are known to
deliver rate-optimal Bayesian procedures that adapt to the unknown regularity
of the regression functions, this makes me confident that the
$(\beta,m)$-parametrization can handle priors that adapt unknown regularity of
the nuisance functions while the earlier discussion indicates that these
priors may threaten the Bernstein-von Mises theorem in the
$(\beta,\eta)$-parametrization.

###  4.2 Prior Corrections and Implementation

Recent literature has proposed dependent priors that incorporate auxiliary
estimators of the least favorable direction to satisfy prior invariance. Yang
et al. (2015) consider the partially linear model and propose a dependent
prior for $\eta$ given $\beta$ that incorporates information about $m_{02}$.
Specifically, $\eta|\beta\sim\tilde{\eta}-\hat{m}_{n2}\beta$ with
$\tilde{\eta}\sim\Pi_{\mathcal{H}}$, $\beta\sim\Pi_{\mathcal{B}}$,
$\tilde{\eta}\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}\beta$, and $\hat{m}_{n2}$ is a plug-in
estimator for $m_{02}$ (e.g., Nadaraya-Watson estimator). The sequence of
estimators $\\{\hat{m}_{n2}\\}_{n=1}^{\infty}$ effectively mimics the
approximating sequence $\\{m_{n2}\\}_{n=1}^{\infty}$, and, as a result, the
prior invariance condition holds as long as the estimator $\hat{m}_{n2}$ is
consistent for $m_{02}$ at an appropriate rate (see Lemma 4.2 of Yang et al.
(2015)). More recently, Ray and van der Vaart (2020) consider a binary
treatment potential outcome model under selection-on-observables and propose a
correction to the priors for the treatment and control group conditional means
that incorporates a plug-in estimator of the propensity score to mitigate the
prior invariance condition (see the comparison of Theorem 1 and 2 in their
paper). In an accompanying paper, Ray and Szabó (2019) propose a data-driven
prior correction despite an admission that a hierarchical prior ‘may be
philosophically appealing’.

The $(\beta,m)$-parametrization of the partially linear model is equivalent to
the $(\beta,\eta)$-parametrization with a dependent prior
$\eta|\beta,m_{2}\sim m_{1}-m_{2}\beta$ and sampling from the posterior is
possible using a standard Gibbs sampling algorithm. The sampling model implies

$\displaystyle
Y_{i}|\\{(X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},\beta,m\overset{ind}{\sim}\mathcal{N}(m_{1}(W_{i})+(X_{i}-m_{2}(W_{i}))\beta,\sigma_{01}^{2})$
(8)

and

$\displaystyle
X_{i}|\\{(Y_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},\beta,m\overset{ind}{\sim}\mathcal{N}\left(m_{2}(W_{i})+\frac{\beta\sigma_{02}^{2}}{\sigma_{01}^{2}+\beta^{2}\sigma_{02}^{2}}\left(Y_{i}-m_{1}(W_{i})\right),\frac{\sigma_{01}^{2}\sigma_{02}^{2}}{\sigma_{01}^{2}+\beta^{2}\sigma_{02}^{2}}\right)$
(9)

for $i=1,...,n$. If $\beta$, $m_{1}$, and $m_{2}$ are mutually independent,
(8) indicates that the problem of learning $m_{1}$ while holding
$(\beta,m_{2})$ fixed is that of a Gaussian nonparametric regression of
$Y_{i}-(X_{i}-m_{2}(W_{i}))\beta$ on $W_{i}$. A similar statement applies to
Bayesian updating of $m_{2}$ while holding $(\beta,m_{1})$ fixed except using
(9). In both cases, Gaussian processes form a conjugate family of priors for
which a large literature is devoted to computation (e.g., Rasmussen and
Williams (2005) and the references therein). Using (8), Bayesian updating of
$\beta$ with $(m_{1},m_{2})$ fixed corresponds to that of a Gaussian linear
regression of a centered outcome $Y_{i}-m_{1}(W_{i})$ on a centered predictor
$X_{i}-m_{2}(W_{i})$, another setting in which conjugate priors are available.

The above discussion illustrates that the profile likelihood-inspired
reparametrization offers the implementation benefits associated with
independent priors while simultaneously respecting the semiparametric
structure of the model. It should be viewed as a prototype for an approach to
Bayesian inference in semiparametric models that complements the recent
literature on data-driven corrections. Indeed, there may be settings in which
a reparametrization that resembles the least favorable submodel is not as
easily available. I am currently working to determine the extent to which the
lessons from the partially linear model apply to a broader class of models.

###  4.3 Interpretation under Misspecification

Theorem 2 implies that Bayesian credible sets are asymptotically efficient
frequentist confidence sets in the homoskedastic Gaussian partially linear
model. The efficiency claim holds because the derivative
$\tilde{\ell}(Y_{i},X_{i},W_{i},\beta,m)$ of $\log
p_{\beta,m}(Y_{i},X_{i}|W_{i})$ with respect to $\beta$ constitutes an
efficient score and $\tilde{I}_{0}(m_{0})$ is the efficient Fisher information
in this semiparametric model. Since normality may fail, it is important to
acknowledge how my results are affected by model misspecification.

The first takeaway is that the statement of Theorem 2 is valid even when Part
1 of Assumption 1 is weakened. Given the other assumptions, an inspection of
the proof reveals that I only use
$E_{P_{0}}[Y_{i}|X_{i},W_{i}]=X_{i}\beta_{0}+\eta_{0}(W_{i})$ and
$Var_{P_{0}}(Y_{i}|X_{i},W_{i})=\sigma_{01}^{2}$ to arrive at the asymptotic
normality result.777Two points should be clarified here. First,
$\eta_{0}(W_{i})=m_{01}(W_{i})-\beta_{0}m_{02}(W_{i})$ means
$E_{P_{0}}[Y_{i}|X_{i},W_{i}]=X_{i}\beta_{0}+\eta_{0}(W_{i})$ is the same as
$E_{P_{0}}[Y_{i}|X_{i},W_{i}]=m_{01}(W_{i})+(X_{i}-m_{02}(W_{i}))\beta_{0}$.
Second, the homoskedasticity assumption should be viewed as convenient rather
than essential. A strengthening of the moment conditions in Part 4 of
Assumption 1 to ensure that $Var_{P_{0}}(U_{i}\varepsilon_{i2})\in(0,\infty)$
will deliver the same result provided that the conditional mean restriction
$E_{P_{0}}[Y_{i}|X_{i},W_{i}]=X_{i}\beta_{0}+\eta_{0}(W_{i})$ is satisfied.
The intuition is that the Gaussian likelihood possesses the same degree of
robustness to misspecification as least squares, and, as a result, we only
require that the mean restrictions hold to obtain consistent and
asymptotically normal estimators. Of course, one may be concerned that
normality is embedded in the other assumptions. However, it is well-known that
posteriors concentrate around the parameter value that minimizes the Kullback-
Leibler divergence between the true distribution and the misspecified model.
Moreover, the rate of convergence is unaffected by misspecification of the
error distribution in Gaussian regression contexts provided that the tails of
the true error distribution are not too thick (e.g., Section 4 of Kleijn and
van der Vaart (2006)). Given the conclusion of Theorem 1 and the observation
that the proof only requires
$E_{P_{0}}[Y_{i}|X_{i},W_{i}]=X_{i}\beta_{0}+\eta_{0}(W_{i})$, it is
reasonable that Assumption 3 and Assumption 4 still hold even if the true
distribution of $(Y_{i},X_{i})^{\prime}$ given $W_{i}$ is not Gaussian.
Similarly, Assumption 5 is a stochastic equicontinuity restriction on two
multiplier empirical processes that may be verified under certain violations
of normality. For instance, the case where the random variables
$\varepsilon_{i2}$ and $U_{i}-\beta_{0}\varepsilon_{i2}$ are subgaussian
conditional on $\\{W_{i}\\}_{i=1}^{n}$.

The second takeaway is that the asymptotic efficiency claim may be sensitive
to violations of normality. Chamberlain (1992) derives the semiparametric
efficiency bound for $\beta_{0}$ in the case where Part 1 of Assumption 1 is
weakened to $E_{P_{0}}[Y_{i}|X_{i},W_{i}]=X_{i}\beta_{0}+\eta_{0}(W_{i})$.
Although the efficiency bound does not impose homoskedasticity, Chamberlain’s
result implies that an efficient estimator $\hat{\beta}_{n}$ of $\beta_{0}$
satisfies
$\sqrt{n}(\hat{\beta}_{n}-\beta_{0})=\tilde{I}_{n}(m_{0})^{-1}\tilde{\ell}_{n}(\beta_{0},m_{0})/\sqrt{n}+o_{P_{0}}(1)$
in the special case that $Var_{P_{0}}(Y_{i}|X_{i},W_{i})=\sigma_{01}^{2}$.
This leads to an interpretation of Theorem 2 under misspecification. It
guarantees that Bayesian credible intervals are asymptotically efficient
Frequentist confidence intervals in the model where the conditional mean
restriction $E_{P_{0}}[Y_{i}|X_{i},W_{i}]=X_{i}\beta_{0}+\eta_{0}(W_{i})$ is
imposed and the true data-generating process satisfies
$Var_{P_{0}}[Y_{i}|X_{i},W_{i}]=\sigma_{01}^{2}$. However, my theorem does not
necessarily imply asymptotically efficient Frequentist uncertainty
quantification in the case where
$Var_{P_{0}}[Y_{i}|X_{i},W_{i}]=\sigma_{01}^{2}$ is imposed because a sharper
bound may be attainable that exploits the information contained in the
conditional variance restriction. For this reason, I interpret Theorem 2 as
offering a local efficiency guarantee in the case where the Gaussian model is
misspecified but the mean and variance restrictions
$E_{P_{0}}[Y_{i}|X_{i},W_{i}]=X_{i}\beta_{0}+\eta_{0}(W_{i})$ and
$Var_{P_{0}}[Y_{i}|X_{i},W_{i}]=\sigma_{01}^{2}$ are satisfied (see p. 114 of
Newey (1990) for a definition of local efficiency).

Finally, I discuss the possibility that $\sigma_{01}^{2}$ is unknown and the
Gaussian model for the conditional distribution of $X_{i}$ given $W_{i}$ is
misspecified. An extension that allows $\sigma_{01}^{2}$ to be unknown should
not change the results in a meaningful way provided that the prior for
$\sigma_{1}^{2}$ is supported on a compact interval $[a,b]\subset(0,\infty)$
that contains $\sigma_{01}^{2}$ and admits a Lebesgue density that is bounded
away from zero and infinity (i.e., conditions that ensure $\sigma_{01}^{2}$ is
consistently estimable). Indeed, the least favorable curve is
$\eta_{\beta,\sigma_{1}^{2}}(W_{i})=m_{01}(W_{i})-\beta m_{02}(W_{i})$ meaning
that the Robinson transformation may be applied to bypass the prior invariance
condition. Consequently, the extension to unknown $\sigma_{01}^{2}$ is likely
to only add technical difficulties because it makes the LAN expansions used to
obtain the Bernstein-von Mises theorem slightly more complicated.
Heteroskedasticity is beyond the scope of this article. Since I only require
that $m_{02}$ is consistently estimable (at an appropriate rate), the
previously discussed robustness properties of the Gaussian likelihood
guarantee that the modeling choice
$X_{i}|\\{W_{i}\\}_{i=1}^{n},\beta,m\overset{ind}{\sim}\mathcal{N}(m_{2}(W_{i}),\sigma_{02}^{2})$
is appropriate because the only requirement is that the conditional mean of
$X_{i}$ given $W_{i}$ is an unknown function of $W_{i}$.

##  5 Conclusion

I prove a semiparametric Bernstein-von Mises theorem for the partially linear
model that does not require a prior invariance condition. At the heart of the
approach is the observation that the structure of the profile likelihood
problem leads to parametrization of the partially linear model that
simultaneously respects the semiparametric structure and allows for
independent priors. My findings are useful because the prior invariance
conditions that arise in the original parametrization of the model may impose
harsh restrictions on the nuisance prior and/or surrounding model parameters.
Moreover, the results complement recent literature that uses data-driven
dependent priors to overcome these prior stability conditions. Finally, the
main theorem offers an example of a situation in which Bayesian credible sets
may be asymptotically efficient frequentist confidence sets even though the
sampling model is misspecified. I expect that the lessons from the partially
linear model apply to a broader class of semiparametric statistical models and
view this as an important direction for future research.

##  Appendix A Proofs of Theorems 1, 2, and Proposition 1

###### Proof of Theorem 1.

The proof has three steps. Step 1 relates the objective function to a weighted
least squares criterion. Step 2 argues that the least squares criterion is
uniquely minimized at $m_{0}$. Step 3 uses the results from the first two
steps to prove the second statement of the theorem.

Step 1. First, I note that

$\displaystyle
E_{P_{0}}\left[KL\left(p_{0}(\cdot|W_{i}),p_{\beta,m}(\cdot|W_{i})\right)\right]$
$\displaystyle=E_{P_{0}}\left[\log\frac{p_{0}(Y_{i},X_{i}|W_{i})}{p_{\beta,m}(Y_{i},X_{i}|W_{i})}\right]$
$\displaystyle=E_{P_{0}}[\log p_{0}(Y_{i},X_{i}|W_{i})]-E_{P_{0}}[\log
p_{\beta,m}(Y_{i},X_{i}|W_{i})]$

Let $Z_{i}=(Y_{i},X_{i})^{\prime}$. Applying the formula for a bivariate
normal probability density function, I know that

$\displaystyle p_{\beta,m}(Y_{i},X_{i}|W_{i})=\frac{1}{\sqrt{(2\pi)^{2}\det
V(\beta)}}\exp\left(-\frac{1}{2}(Z_{i}-m(W_{i}))^{\prime}V(\beta)^{-1}(Z_{i}-m(W_{i}))\right)$

which implies that $E_{P_{0}}[\log p_{\beta,m}(Y_{i},X_{i}|W_{i})]$ is equal
to

$\displaystyle-\frac{1}{2}\log 4\pi^{2}-\frac{1}{2}\log\det
V(\beta)-\frac{1}{2}E_{P_{0}}[(Z_{i}-m(W_{i}))^{\prime}V(\beta)^{-1}(Z_{i}-m(W_{i}))].$

Assumption 1 and $\mathcal{M}\subseteq L_{2}(\mathcal{W})\times
L_{2}(\mathcal{W})$ ensures the two conditions $E_{P_{0}}|\log
p_{0}(Y_{i},X_{i}|W_{i})|\in(0,\infty)$ and $E_{P_{0}}|\log
p_{\beta,m}(Y_{i},X_{i}|W_{i})|\in(0,\infty)$. The former is a direct
application of Part 4 of Assumption 1 and the latter holds because
$E_{P_{0}}[Y_{i}^{2}]\in(0,\infty)$, $E_{P_{0}}[X_{i}^{2}]\in(0,\infty)$, and
$\det V(\beta)>0$ for any $\beta\in\mathcal{B}$ as
$\sigma_{01}^{2},\sigma_{02}^{2}\in(0,\infty)$. Consequently, the objective
function is well-defined. Next, I note that
$Z_{i}=m_{0}(W_{i})+\varepsilon_{i}$ with $\varepsilon_{i}=Z_{i}-m_{0}(W_{i})$
satisfies $E_{P_{0}}[\varepsilon_{i}|W_{i}]=0$. As such, I can apply the law
of iterated expectations to conclude

$\displaystyle\frac{1}{2}E_{P_{0}}\left[(Z_{i}-m(W_{i}))^{\prime}V(\beta)^{-1}(Z_{i}-m(W_{i}))\right]$
$\displaystyle\quad=\frac{1}{2}E_{P_{0}}\left[(\varepsilon_{i}+m_{0}(W_{i})-m(W_{i}))^{\prime}V(\beta)^{-1}(\varepsilon_{i}+m_{0}(W_{i})-m(W_{i}))\right]$
$\displaystyle\quad=\frac{1}{2}E_{P_{0}}[(m(W_{i})-m_{0}(W_{i}))^{\prime}V(\beta)^{-1}(m(W_{i})-m_{0}(W_{i}))]$
$\displaystyle\quad\quad+\frac{1}{2}E_{P_{0}}[\varepsilon_{i}^{\prime}V(\beta)^{-1}\varepsilon_{i}].$

Part 4 of Assumption 1 and the observation that $V(\beta)^{-1}$ is symmetric
and positive-definite guarantees that
$E_{P_{0}}[\varepsilon_{i}^{\prime}V(\beta)^{-1}\varepsilon_{i}]\in(0,\infty)$.
Positive-definiteness of $V(\beta)$ follows from Sylvester’s criterion because
$\sigma_{01}^{2}+\beta^{2}\sigma_{01}^{2}>0$ and $\det
V(\beta)=\sigma_{01}^{2}\sigma_{02}^{2}>0$. Consequently, I can combine the
above with the expression for $E_{P_{0}}[\log p_{\beta,m}(Y_{i},X_{i}|W_{i})]$
to conclude that there exists a constant $C(\beta)\in(-\infty,\infty)$ such
that

$\displaystyle
E_{P_{0}}\left[KL\left(p_{0}(\cdot|W_{i}),p_{\beta,m}(\cdot|W_{i})\right)\right]$
$\displaystyle\quad=C(\beta)+\frac{1}{2}E_{P_{0}}[(m(W_{i})-m_{0}(W_{i}))^{\prime}V(\beta)^{-1}(m(W_{i})-m_{0}(W_{i}))].$

Consequently, it suffices to solve

$\displaystyle\min_{m\in\mathcal{M}}E_{P_{0}}\left[(m(W_{i})-m_{0}(W_{i}))^{\prime}V(\beta)^{-1}(m(W_{i})-m_{0}(W_{i}))\right].$
(10)

Step 2. I establish that $m_{0}$ uniquely solves (10). First, $m_{0}$
minimizes the objective function because
$E_{P_{0}}\left[(m(W_{i})-m_{0}(W_{i}))^{\prime}V(\beta)^{-1}(m(W_{i})-m_{0}(W_{i}))\right]\geq
0$ for all $m\in\mathcal{M}$ with equality at $m=m_{0}$ and the nuisance
parameter space is chosen so that $m_{0}\in\mathcal{M}$ (i.e., $m_{0}$ is
feasible). For uniqueness, we note that the positive-definiteness of
$V(\beta)^{-1}$ implies

$\displaystyle
E_{P_{0}}\left[(m(W_{i})-m_{0}(W_{i}))^{\prime}V(\beta)^{-1}(m(W_{i})-m_{0}(W_{i}))\right]$
$\displaystyle\quad\geq\lambda_{min}(V(\beta)^{-1})\left\\{E_{P_{0}}[(m_{1}(W_{i})-m_{01}(W_{i}))^{2}]+E_{P_{0}}[(m_{2}(W_{i})-m_{02}(W_{i}))^{2}]\right\\}$
(11)

and

$\displaystyle
E_{P_{0}}\left[(m(W_{i})-m_{0}(W_{i}))^{\prime}V(\beta)^{-1}(m(W_{i})-m_{0}(W_{i}))\right]$
$\displaystyle\quad\leq\lambda_{max}(V(\beta)^{-1})\left\\{E_{P_{0}}[(m_{1}(W_{i})-m_{01}(W_{i}))^{2}]+E_{P_{0}}[(m_{2}(W_{i})-m_{02}(W_{i}))^{2}]\right\\},$
(12)

where $\lambda_{min}(V(\beta)^{-1})>0$ is the smallest eigenvalue of
$V(\beta)^{-1}$ and $\lambda_{max}(V(\beta)^{-1})>0$ is the largest eigenvalue
of $V(\beta)^{-1}$. Using the definition of the $L_{2}$-norm, it follows that

$\displaystyle
E_{P_{0}}[(m_{1}(W_{i})-m_{01}(W_{i}))^{2}]+E_{P_{0}}[(m_{2}(W_{i})-m_{02}(W_{i}))^{2}]>0$

if and only if there exists $j\in\\{1,2\\}$ such that

$\displaystyle P_{0}(m_{j}(W_{i})\neq m_{0j}(W_{i}))>0.$

From this, I conclude that $m_{0}$ is the unique solution to (10) in an
almost-sure sense.

Step 3. Based on the results of Step 1 and Step 2, it is sufficient to solve

$\displaystyle\min_{\beta\in\mathcal{B}}E_{P_{0}}[KL(p_{0}(\cdot|W_{i}),p_{\beta,m_{0}}(\cdot|W_{i}))].$

to obtain the second assertion of the theorem. The factorization of a
bivariate normal distribution into its conditional and marginal distributions
offers further simplification because it reduces the problem to solving

$\displaystyle\min_{\beta\in\mathcal{B}}E_{P_{0}}\left[\frac{1}{2\sigma_{01}^{2}}\left(Y_{i}-m_{01}(W_{i})-(X_{i}-m_{02}(W_{i}))\beta\right)^{2}\right].$

Adding and subtracting $(X_{i}-m_{02}(W_{i}))\beta_{0}$ and invoking Part 4 of
Assumption 1, I can find a constant $C\in(-\infty,\infty)$ that does not
depend on $\beta$ such that

$\displaystyle
E_{P_{0}}\left[\frac{1}{2\sigma_{01}^{2}}\left(Y_{i}-m_{01}(W_{i})-(X_{i}-m_{02}(W_{i}))\beta\right)^{2}\right]$
$\displaystyle\quad=C+\frac{1}{2\sigma_{01}^{2}}(\beta-\beta_{0})^{2}E_{P_{0}}[(X_{i}-m_{02}(W_{i}))^{2}],$

and, as a result, it suffices to solve the least squares problem

$\displaystyle\min_{\beta\in\mathcal{B}}\left\\{\frac{1}{2\sigma_{01}^{2}}(\beta-\beta_{0})^{2}E_{P_{0}}[(X_{i}-m_{02}(W_{i}))^{2}]\right\\}.$
(13)

As the objective function is nonnegative and equal to zero when
$\beta=\beta_{0}$ (a feasible solution because $\beta_{0}\in\mathcal{B}$), it
follows that $\beta_{0}$ is a solution to (13). Uniqueness follows from Part 2
of Assumption 1 because the restriction
$E_{P_{0}}[(X_{i}-m_{02}(W_{i}))^{2}]\in[\underline{c},\overline{c}]$ implies
that the objective function is equal to $0$ if and only if $\beta=\beta_{0}$.
This completes the proof. ∎

######  Lemma 1.

Let $\\{\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\\}_{n=1}^{\infty}$ be the sets
defined in Assumption 3. If Assumptions 1, 4, and 5 hold, then

  1. 1.

For any $\xi>0$,

$\displaystyle P_{0}\left(\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)-\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right|>\xi\right)\rightarrow
0$

as $n\rightarrow\infty$.

  2. 2.

For any $\xi>0$,

$\displaystyle P_{0}\left(\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|\tilde{I}_{n}(m)-\tilde{I}_{n}(m_{0})\right|>\xi\right)\rightarrow
0$

as $n\rightarrow\infty$.

  3. 3.

Let $\underline{c}_{0}=\underline{c}/\sigma_{01}^{2}$ and
$\overline{c}_{0}=\overline{c}/\sigma_{01}^{2}$, where $\overline{c}$ and
$\underline{c}$ are the constants Part 2 of Assumption 1. The following
statements hold:

$\displaystyle P_{0}\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\tilde{I}_{n}(m)\geq\underline{c}_{0}\right)\rightarrow
1$

and

$\displaystyle P_{0}\left(\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\tilde{I}_{n}(m)\leq\overline{c}_{0}\right)\rightarrow$

as $n\rightarrow\infty$.

  4. 4.

Let $R_{n}(\beta,m)$ be given by

$\displaystyle
R_{n}(\beta,m)=\ell_{n}(\beta,m)-\ell_{n}(\beta_{0},m)-\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})+\frac{n}{2}(\beta-\beta_{0})^{2}\tilde{I}_{n}(m_{0})$

for each $(\beta,m)\in\mathcal{B}\times\mathcal{M}$. If
$M_{n}\rightarrow\infty$, then the following holds for any $\xi>0$:

$\displaystyle P_{0}\left(\sup_{(\beta,m)\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})\times(B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n})}\left|R_{n}(\beta,m)\right|>\xi\right)\rightarrow
0$

as $n\rightarrow\infty$.

######  Lemma 2.

Let $\\{\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\\}_{n=1}^{\infty}$ be the sets
defined in Assumption 3. If Assumptions 1, 2, 4, and 5 hold, then for every
$M_{n}\rightarrow\infty$

$\displaystyle\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi\left(\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m\right)\overset{P_{0}}{\rightarrow}0$

as $n\rightarrow\infty$.

###### Proof of Theorem 2.

The proof is comprised of four steps. Step 1 and Step 2 utilize Assumption 3
and Lemma 2 to conclude it is enough to show that
$\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n}),m)$ and
$\mathcal{N}(\tilde{\Delta}_{n,0},\tilde{I}_{0}(m_{0})^{-1})(A)$ are first-
order asymptotically equivalent for all measurable sets $A$. Step 3 uses the
assumption about the density of $\Pi_{\mathcal{B}}$ and Part 2 of Lemma 1 to
further simplify the problem to one in which $\beta$ is the only free
parameter. Step 4 concludes by showing asymptotic normality.

Step 1. Applying the law of total probability,

$\displaystyle\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$
$\displaystyle\quad=\int_{\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$
$\displaystyle\quad\quad+\int_{\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})^{c}}\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$
$\displaystyle\quad\quad+\int_{\mathcal{M}_{n}^{c}}\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n}).$

for any event $A$. As
$0\leq\Pi(\beta\in\cdot|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)\leq
1$, it follows that

$\displaystyle\sup_{A}\int_{\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})^{c}}\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$
$\displaystyle\quad\leq\Pi(m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$

and

$\displaystyle\sup_{A}\int_{\mathcal{M}_{n}^{c}}\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$
$\displaystyle\quad\leq\Pi(m\in\mathcal{M}_{n}^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n}).$

Applying Assumption 3, it then follows that

$\displaystyle\sup_{A}\int_{\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})^{c}}\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\overset{P_{0}}{\longrightarrow}0$

and

$\displaystyle\sup_{A}\int_{\mathcal{M}_{n}^{c}}\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\overset{P_{0}}{\longrightarrow}0.$

As a result,

$\displaystyle\sup_{A}\left|\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})-F_{n,1}(A)\right|\overset{P_{0}}{\rightarrow}0$

as $n\rightarrow\infty$, where $A\mapsto F_{n,1}(A)$ is a set function that
satisfies

$\displaystyle F_{n,1}(A)=\int_{\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n}).$

Step 2. Let $M_{n}\rightarrow\infty$ arbitrarily slowly. Since $A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})$ and $A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}$ define a partition of $A$, it
follows

$\displaystyle\sup_{A}|F_{n,1}(A)-F_{n,1}(A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n}))|$
$\displaystyle\quad=\sup_{A}F_{n,1}(A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c})$
$\displaystyle\quad\leq\sup_{A}\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi(A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)$
$\displaystyle\quad\quad\times\Pi(\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$
$\displaystyle\quad\leq\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi(B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)$

with the first inequality applies Hölder’s inequality, and the second
inequality uses that $A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}\subseteq
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}$ for all $A$ and that
probabilities are bounded from above by one. Applying Lemma 2, I then obtain

$\displaystyle\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi(B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)\overset{P_{0}}{\rightarrow}0$

as $n\rightarrow\infty$, and, as a result,

$\displaystyle\sup_{A}|F_{n,1}(A)-F_{n,1}(A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n}))|\overset{P_{0}}{\rightarrow}0$

as $n\rightarrow\infty$. Moreover, I know that

$\displaystyle\left|\int_{\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\left(\frac{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}L_{n}(\beta,m)d\Pi_{\mathcal{B}}(\beta)}{\int_{\mathcal{B}}L_{n}(\beta,m)d\Pi_{\mathcal{B}}(\beta)}-1\right)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\right|$
$\displaystyle\quad=\int_{\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi(\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$
$\displaystyle\quad\leq\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi(\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)$

with the inequality applying Hölder’s inequality and that probabilities are
bounded by $1$. Applying Lemma 2,

$\displaystyle\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi(\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n},m)\overset{P_{0}}{\rightarrow}0$

as $n\rightarrow\infty$, and, as a result,

$\displaystyle\left|\int_{\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\left(\frac{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}L_{n}(\beta,m)d\Pi_{\mathcal{B}}(\beta)}{\int_{\mathcal{B}}L_{n}(\beta,m)d\Pi_{\mathcal{B}}(\beta)}-1\right)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n}\right|\overset{P_{0}}{\rightarrow}0$

as $n\rightarrow\infty$. Consequently, I am able to conclude that

$\displaystyle\sup_{A}\left|F_{n,1}(A)-F_{n,2}(A)\right|\overset{P_{0}}{\rightarrow}0,$

where $A\mapsto F_{n,2}(A)$ is a set function given by

$\displaystyle F_{n,2}(A)=\int_{\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\left(\frac{\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}L_{n}(\beta,m)d\Pi_{\mathcal{B}}(\beta)}{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}L_{n}(\beta,m)d\Pi_{\mathcal{B}}(\beta)}\right)d\Pi(m|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$

for all events $A$.

Step 3. Exponentiating and adding/subtracting $\ell_{n}(\beta_{0},m)$, we have
that

$\displaystyle\frac{\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}L_{n}(\beta,m)d\Pi_{\mathcal{B}}(\beta)}{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}L_{n}(\beta,m)d\Pi_{\mathcal{B}}(\beta)}=\frac{\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp(\ell_{n}(\beta,m)-\ell_{n}(\beta_{0},m))d\Pi_{\mathcal{B}}(\beta)}{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp(\ell_{n}(\beta,m)-\ell_{n}(\beta_{0},m))d\Pi_{\mathcal{B}}(\beta)}$

for all events $A$. Recall the definition

$\displaystyle
R_{n}(\beta,m)=\ell_{n}(\beta,m)-\ell_{n}(\beta_{0},m)-\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})+\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}$

for $(\beta,m)\in\mathcal{B}\times\mathcal{M}$. Part 4 of Lemma 1 implies that

$\displaystyle\overline{R}_{n}:=\sup_{(\beta,m)\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})\times
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\exp(R_{n}(\beta,m))\overset{P_{0}}{\longrightarrow}1$

and

$\displaystyle\underline{R}_{n}:=\inf_{(\beta,m)\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})\times
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\exp(R_{n}(\beta,m))\overset{P_{0}}{\longrightarrow}1.$

Consequently, there exists a positive sequence $\\{c_{n,1}\\}_{n=1}^{\infty}$
such that $c_{n,1}\rightarrow 0$ and

$\displaystyle\left|\frac{\overline{R}_{n}}{\underline{R}_{n}}-1\right|\leq
c_{n,1}$

with $P_{0}$-probability approaching $1$. Next, the assumption that
$\Pi_{\mathcal{B}}$ has a density that is continuous and positive in a
neighborhood of $\beta_{0}$ guarantees that there exists a positive sequence
$c_{n,2}$ such that $c_{n,2}\rightarrow 0$ and

$\displaystyle\left|\frac{\sup_{\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\pi_{\mathcal{B}}(\beta)}{\inf_{\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\pi_{\mathcal{B}}(\beta)}-1\right|\leq
c_{n,2}.$

As a result, with $P_{0}$-probability approaching one, the following
quantities

$\displaystyle(1+c_{n,1})(1+c_{n,2})\frac{\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}\right)d\beta}{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}\right)d\beta}$

and

$\displaystyle[(1+c_{n,1})(1+c_{n,2})]^{-1}\frac{\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp\left(\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}\right)d\beta}{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp\left(\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}\right)d\beta}$

are upper and lower bounds, respectively, for $F_{n,2}(A)$. Since
$c_{n,1},c_{n,2}\rightarrow 0$ as $n\rightarrow\infty$, it follows that

$\displaystyle\sup_{A}\left|F_{n,2}(A)-\frac{\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}\right)d\beta}{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}\right)d\beta}\right|\overset{P_{0}}{\rightarrow}0$

as $n\rightarrow\infty$.

Step 4. Recalling that
$\tilde{\Delta}_{n,0}=\tilde{I}_{n}(m_{0})^{-1}\tilde{\ell}_{n}(\beta_{0},m_{0})/\sqrt{n}$,
I complete the square to obtain

$\displaystyle\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}\right)$
$\displaystyle\quad=\exp\left(-\frac{1}{2}\tilde{I}_{n}(m_{0})\left(\sqrt{n}(\beta-\beta_{0})-\tilde{\Delta}_{n,0}\right)^{2}\right)\exp\left(\frac{1}{2}\tilde{I}_{n}(m_{0})\tilde{\Delta}_{n,0}^{2}\right)$
$\displaystyle=\exp\left(-\frac{1}{2}n\tilde{I}_{n}(m_{0})\left(\beta-\beta_{0}-\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}}\right)^{2}\right)\exp\left(\frac{1}{2}\tilde{I}_{n}(m_{0})\tilde{\Delta}_{n,0}^{2}\right).$

It then follows that

$\displaystyle\frac{\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}\right)d\beta}{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}\tilde{I}_{n}(m_{0})n(\beta-\beta_{0})^{2}\right)d\beta}$
$\displaystyle\quad=\frac{\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp\left(-\frac{1}{2}n\tilde{I}_{n}(m_{0})\left(\beta-\beta_{0}-\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}}\right)^{2}\right)d\beta}{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\exp\left(-\frac{1}{2}n\tilde{I}_{n}(m_{0})\left(\beta-\beta_{0}-\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}}\right)^{2}\right)d\beta}$
$\displaystyle\quad=\frac{\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta}{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta}$
(14)

for all events $A$, where
$\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}$
is the probability density function of the Gaussian distribution
$\mathcal{N}(\beta_{0}+\tilde{\Delta}_{n,0}/\sqrt{n},(n\tilde{I}_{n}(m_{0}))^{-1})$.
The denominator satisfies

$\displaystyle\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta=\Phi\left(\frac{M_{n}-\tilde{\Delta}_{n,0}}{\sqrt{\tilde{I}_{n}(m_{0})^{-1}}}\right)-\Phi\left(\frac{-M_{n}-\tilde{\Delta}_{n,0}}{\sqrt{\tilde{I}_{n}(m_{0})^{-1}}}\right)$

where $\Phi(\cdot)$ is standard normal cumulative distribution function.
Following Part 4 of Assumption 1, we can apply the Central Limit Theorem, the
Law of Large Numbers, and the Continuous Mapping Theorem to conclude that
$\tilde{\Delta}_{n,0}\overset{d}{\rightarrow}\mathcal{N}(0,\tilde{I}(m_{0})^{-1})$
as $n\rightarrow\infty$. Similarly, the Law of Large Numbers and the
Continuous Mapping Theorem guarantee that
$\tilde{I}_{n}(m_{0})^{-1}\overset{P_{0}}{\rightarrow}\tilde{I}(m_{0})^{-1}$
as $n\rightarrow\infty$. As $M_{n}\rightarrow\infty$,
$\tilde{\Delta}_{n,0}=O_{P_{0}}(1)$, and
$\tilde{I}_{n}(m_{0})^{-1}=O_{P_{0}}(1)$, it follows that

$\displaystyle\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta=1+o_{P_{0}}(1).$

Consequently,

$\displaystyle\sup_{A}\left|\frac{\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta}{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta}-\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta\right|\overset{P_{0}}{\rightarrow}0$

as $n\rightarrow\infty$. Since

$\displaystyle\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta\leq\int_{A}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta$

for all events $A$, I need to show that, with $P_{0}$-probability approaching
one,

$\displaystyle\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta\geq\int_{A}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta$

for all events $A$. Indeed, this implies that

$\displaystyle\sup_{A}\left|\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta-\int_{A}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta\right|\overset{P_{0}}{\rightarrow}0,$

and, as a result, an application of the triangle inequality leads to the
conclusion that

$\displaystyle\left|\left|\Pi(\beta\in
A|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})-\mathcal{N}\left(\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}\right)\right|\right|_{TV}\overset{P_{0}}{\rightarrow}0.$

The events $A\cap B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})$ and $A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}$ define a partition of $A$
meaning that

$\displaystyle\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta$
$\displaystyle\quad=\int_{A}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta-\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta$
$\displaystyle\quad\geq\int_{A}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta-\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta$

for all events $A$, where the lower bound holds because

$\displaystyle\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta\leq\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta$

for all events $A$ using the monotonicity of probability measures. An earlier
argument showed that

$\displaystyle\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta=1+o_{P_{0}}(1),$

so

$\displaystyle\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta=o_{P_{0}}(1)$

by the complement rule for probability measures. As such, I conclude that with
$P_{0}$-probability approaching one,

$\displaystyle\int_{A\cap
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta\geq\int_{A}\phi_{\beta_{0}+\frac{\tilde{\Delta}_{n,0}}{\sqrt{n}},\frac{1}{n}\tilde{I}_{n}(m_{0})^{-1}}(\beta)d\beta$

for all events $A$. This completes the proof. ∎

######  Lemma 3.

Suppose that $p_{0}(\cdot|W_{i})$ is that of
$\mathcal{N}(m_{0}(W_{i}),V(\beta_{0}))$, the distribution of $W_{i}$ has a
density on $[0,1]$ that is bounded away from zero and infinity, and the prior
satisfies the conditions from Section 3.2. If Assumption 6 holds, then the
sets $\\{\mathcal{M}\\}_{n=1}^{\infty}$ from Section 3.2 satisfy
$\Pi(\mathcal{M}_{n}^{c}|\\{\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\overset{P_{0}}{\longrightarrow}0$
as $n\rightarrow\infty$

######  Lemma 4.

Suppose that $p_{0}(\cdot|W_{i})$ is that of
$\mathcal{N}(m_{0}(W_{i}),V(\beta_{0}))$, the distribution of $W_{i}$ has a
density on $[0,1]$ that is bounded away from zero and infinity, and the prior
satisfies the conditions from Section 3.2. If the sequence
$\\{\delta_{n}\\}_{n=1}^{\infty}$ satisfies $\delta_{n}=o(n^{-1/4})$, then
Assumption 5 holds for $\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})$, where
$\\{\mathcal{M}_{n}\\}_{n\geq 1}$ are the sets described in Section 3.2.

###### Proof of Proposition 1.

Since the choice of prior guarantees Assumption 2 and I impose that
$\delta_{n}=o(n^{-1/4})$ (i.e., Assumption 4), I need to verify Assumption 3
and 5. Step 1 and Step 2 perform these tasks, respectively.

Step 1. Lemma 3 implies Part 1 of Assumption 3. Since $\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})^{c}\subseteq
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})^{c}$ implies

$\displaystyle\Pi(\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\leq\Pi(B_{\mathcal{M},\infty}(m_{0},D\delta_{n})^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n}),$

it follows from Assumption 6 that Part 2 of Assumption 3 holds. In both cases,
the convergence in probability statements can be upgraded to convergence in
mean because the sequence of posterior distributions is bounded (and therefore
uniformly integrable).

Step 2. I invoke Lemma 4 directly to verify Assumption 5. ∎

##  Appendix B Proofs of Lemmas

For the proof of Lemma 1, I recall the definitions of the empirical
$L_{2}$-norm $||\cdot||_{n,2}$ and inner product
$\langle\cdot,\cdot\rangle_{n,2}$. Specifically, $\langle
f,g\rangle_{n,2}=\frac{1}{n}\sum_{i=1}^{n}f(W_{i})g(W_{i})$ and
$||f||_{n,2}^{2}=\langle f,f\rangle_{n,2}$ for measurable
$f,g:\mathcal{W}\rightarrow\mathbb{R}$.

###### Proof of Lemma 1.

The proof has four steps with each step corresponding to the appropriate part
of the lemma (i.e., Step 1 verifies Part 1 of the lemma).

Step 1. Since $Y_{i}=m_{01}(W_{i})+(X_{i}-m_{02}(W_{i}))\beta_{0}+U_{i}$ and
$X_{i}=m_{02}(W_{i})+\varepsilon_{i2}$, I know that

$\displaystyle\tilde{\ell}(Y_{i},X_{i},W_{i},\beta_{0},m)$
$\displaystyle=\frac{1}{\sigma_{01}^{2}}\left(Y_{i}-m_{1}(W_{i})-(X_{i}-m_{2}(W_{i}))\beta_{0}\right)\left(X_{i}-m_{2}(W_{i})\right)$
$\displaystyle=\frac{1}{\sigma_{01}^{2}}\left(m_{01}(W_{i})-m_{1}(W_{i})-(m_{02}(W_{i})-m_{2}(W_{i}))\beta_{0}+U_{i}\right)\left(m_{02}(W_{i})-m_{2}(W_{i})+\varepsilon_{i2}\right)$
$\displaystyle=\frac{1}{\sigma_{01}^{2}}\left\\{(m_{01}(W_{i})-m_{1}(W_{i}))(m_{02}(W_{i})-m_{2}(W_{i}))-(m_{02}(W_{i})-m_{2}(W_{i}))^{2}\beta_{0}\right.$
$\displaystyle\quad\quad\quad\left.+(m_{02}(W_{i})-m_{2}(W_{i}))(U_{i}-\beta_{0}\varepsilon_{i2})+(m_{01}(W_{i})-m_{1}(W_{i}))\varepsilon_{i2}+U_{i}\varepsilon_{i2}\right\\}.$

Observing that
$\tilde{\ell}(Y_{i},X_{i},W_{i},\beta_{0},m_{0})=U_{i}\varepsilon_{i2}/\sigma_{01}^{2}$,
the previous derivation implies that

$\displaystyle\frac{\sigma_{01}^{2}}{\sqrt{n}}\left(\tilde{\ell}_{n}(\beta_{0},m)-\tilde{\ell}_{n}(\beta_{0},m_{0})\right)$
$\displaystyle\quad=\sqrt{n}\langle
m_{1}-m_{01},m_{2}-m_{02}\rangle_{n,2}-\sqrt{n}||m_{2}-m_{02}||_{n,2}^{2}\beta_{0}-G_{n}^{(1)}(m)-G_{n}^{(2)}(m),$

where we recall that the empirical processes
$\\{G_{n}^{(1)}\\}_{n=1}^{\infty}$ and $\\{G_{n}^{(2)}\\}_{n=1}^{\infty}$ are
given by

$\displaystyle
G_{n}^{(1)}(m)=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}(m_{1}(W_{i})-m_{01}(W_{i}))\varepsilon_{i2}$

and

$\displaystyle
G_{n}^{(2)}(m)=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left\\{\left(m_{2}(W_{i})-m_{02}(W_{i})\right)\left(U_{i}-\beta_{0}\varepsilon_{i2}\right)\right\\}$

for $m\in\mathcal{M}$. The Cauchy-Schwarz inequality, the inequality
$||\cdot||_{n,2}\leq||\cdot||_{\infty}$, and the definition of
$B_{\mathcal{M},\infty}(m_{0},D\delta_{n})$ reveals that

$\displaystyle\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty,2}(m_{0},D\delta_{n})}\left|\sqrt{n}\langle
m_{1}-m_{01},m_{2}-m_{02}\rangle_{n,2}\right|$
$\displaystyle\quad\leq\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty,2}(m_{0},D\delta_{n})}\left(\sqrt{n}||m_{1}-m_{01}||_{n,2}\cdot||m_{2}-m_{02}||_{n,2}\right)$
$\displaystyle\quad\leq\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty,2}(m_{0},D\delta_{n})}\left(\sqrt{n}||m_{1}-m_{01}||_{\infty}\cdot||m_{2}-m_{02}||_{\infty}\right)$
$\displaystyle\quad\lesssim\sqrt{n}\delta_{n}^{2}$

and

$\displaystyle\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty,2}(m_{0},D\delta_{n})}\sqrt{n}||m_{2}-m_{02}||_{n,2}^{2}\leq\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty,2}(m_{0},D\delta_{n})}\sqrt{n}||m_{2}-m_{02}||_{\infty}^{2}\lesssim\sqrt{n}\delta_{n}^{2}.$

with $P_{0}$-probability equal to $1$. Consequently, there exists a constant
$C_{0}>0$ such that for every $\xi>0$,

$\displaystyle P_{0}\left(\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)-\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right|>\xi\right)$
$\displaystyle\quad\leq P_{0}\left(\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|G_{n}^{(1)}(m)\right|+\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|G_{n}^{(2)}(m)\right|>\xi-
C_{0}\sqrt{n}\delta_{n}^{2}\right).$

Applying Markov’s inequality,

$\displaystyle P_{0}\left(\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|G_{n}^{(1)}(m)\right|+\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|G_{n}^{(2)}(m)\right|>\xi-
C_{0}\sqrt{n}\delta_{n}^{2}\right)$ $\displaystyle\quad\leq\frac{1}{\xi-
C_{0}\sqrt{n}\delta_{n}^{2}}\left\\{E_{P_{0}}\left(\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\left|G_{n}^{(1)}(m)\right|\right)+E_{P_{0}}\left(\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\left|G_{n}^{(2)}(m)\right|\right)\right\\}.$

Applying Assumptions 4 and 5, I then conclude that

$\displaystyle\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)-\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right|\overset{P_{0}}{\longrightarrow}0$

as $n\rightarrow\infty$.

Step 2. Since $X_{i}=m_{02}(W_{i})+\varepsilon_{i2}$, it follows that

$\displaystyle(X_{i}-m_{2}(W_{i}))^{2}=\left(m_{2}(W_{i})-m_{02}(W_{i})\right)^{2}-2(m_{2}(W_{i})-m_{02}(W_{i}))\varepsilon_{i2}+\varepsilon_{i2}^{2},$

and, as a result, I obtain

$\displaystyle\tilde{I}_{n}(m)-\tilde{I}_{n}(m_{0})=\frac{1}{\sigma_{01}^{2}}||m_{2}-m_{02}||_{n,2}^{2}-\frac{2}{\sigma_{01}^{2}}\frac{1}{n}\sum_{i=1}^{n}(m_{2}(W_{i})-m_{02}(W_{i}))\varepsilon_{i2}.$

Using the definition of $B_{\mathcal{M},\infty}(m_{0},D\delta_{n})$,

$\displaystyle\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}||m_{2}-m_{02}||_{n,2}^{2}\leq\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}||m_{2}-m_{02}||_{\infty}^{2}\lesssim\delta_{n}^{2},$

and, as a result, the triangle inequality reveals that there exists a constant
$C_{1}>0$ such that for every $\xi>0$,

$\displaystyle P_{0}\left(\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}|\tilde{I}_{n}(m)-\tilde{I}_{n}(m_{0})|>\xi\right)$
$\displaystyle\quad\leq
P_{0}\left(\sup_{m\in\mathcal{B}_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|\frac{1}{n}\sum_{i=1}^{n}(m_{2}(W_{i})-m_{02}(W_{i}))\varepsilon_{i2}\right|>\frac{\sigma_{01}^{2}}{2}\left(\xi-
C_{1}\delta_{n}^{2}\right)\right).$

Applying Markov’s inequality and using the definition of $G_{n}^{(2)}(m)$,

$\displaystyle
P_{0}\left(\sup_{m\in\mathcal{B}_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|\frac{1}{n}\sum_{i=1}^{n}(m_{2}(W_{i})-m_{02}(W_{i}))\varepsilon_{i2}\right|>\frac{\sigma_{01}^{2}}{2}\left(\xi-
C_{1}\delta_{n}^{2}\right)\right)$
$\displaystyle\quad\lesssim\frac{1}{\sqrt{n}(\xi-
C_{1}\delta_{n}^{2})}E_{P_{0}}\left(\sup_{m\in\mathcal{B}_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}|G_{n}^{(2)}(m)|\right).$

Assumptions 4 and 5 guarantee that

$\displaystyle\frac{1}{\sqrt{n}(\xi-
C_{1}\delta_{n}^{2})}E_{P_{0}}\left(\sup_{m\in\mathcal{B}_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}|G_{n}^{(2)}(m)|\right)\rightarrow
0$

as $n\rightarrow\infty$ for any $\xi>0$, which establishes the claim of Part 2
of the lemma.

Step 3. Part 2 of Assumption 1 and the Weak Law of Large Numbers reveals that
$\tilde{I}_{n}(m_{0})\in[\underline{c}_{0},\overline{c}_{0}]$ with
$P_{0}$-probability approaching $1$. As such, I add and subtract
$\tilde{I}_{n}(m_{0})$ and apply the law of total probability to conclude that

$\displaystyle P_{0}\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\tilde{I}_{n}(m)<\underline{c}_{0}\right)$
$\displaystyle=P_{0}\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\tilde{I}_{n}(m)<\underline{c}_{0},\
\tilde{I}_{n}(m_{0})\geq\overline{c}_{0}\right)$
$\displaystyle\quad+P_{0}\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\tilde{I}_{n}(m)<\underline{c}_{0},\
\tilde{I}_{n}(m_{0})<\overline{c}_{0}\right)$ $\displaystyle\leq
P_{0}\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}(\tilde{I}_{n}(m)-\tilde{I}_{n}(m_{0}))<0\right)$
$\displaystyle\quad+P_{0}\left(\tilde{I}_{n}(m_{0})<\underline{c}_{0}\right)$
$\displaystyle=P_{0}\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}(\tilde{I}_{n}(m)-\tilde{I}_{n}(m_{0}))<0\right)+o(1)$

with the equality using that
$\tilde{I}_{n}(m_{0})\in[\underline{c}_{0},\overline{c}_{0}]$ with
$P_{0}$-probability approaching one. Applying Part 2 of the lemma, I deduce
that

$\displaystyle P_{0}\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}(\tilde{I}_{n}(m)-\tilde{I}_{n}(m_{0}))<0\right)\rightarrow
0$

as $n\rightarrow\infty$. It then follows that

$\displaystyle P_{0}\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\tilde{I}_{n}(m)<\underline{c}_{0}\right)\rightarrow
0$

as $n\rightarrow\infty$. A symmetric argument establishes that

$\displaystyle P_{0}\left(\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\tilde{I}_{n}(m)>\overline{c}_{0}\right)\rightarrow
0$

as $n\rightarrow\infty$.

Step 4. Since $p_{\beta,m}(Y_{i},X_{i}|W_{i})$ is the density of
$\mathcal{N}(m(W_{i}),V(\beta))$, we obtain the following expansion of the
log-likelihood function around the true parameter $\beta_{0}$:

$\displaystyle\ell_{n}(\beta,m)-\ell_{n}(\beta_{0},m)=\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}n(\beta-\beta_{0})^{2}\tilde{I}_{n}(m),$

and, as a result, we apply the triangle inequality to conclude

$\displaystyle\left|\ell_{n}(\beta,m)-\ell_{n}(\beta_{0},m)-\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\sqrt{n}(\beta-\beta_{0})+\frac{1}{2}n(\beta-\beta_{0})^{2}\tilde{I}_{n}(m_{0})\right|$
$\displaystyle\quad\leq\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)-\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right|\cdot\left|\sqrt{n}(\beta-\beta_{0})\right|+\frac{n}{2}(\beta-\beta_{0})^{2}\left|\tilde{I}_{n}(m)-\tilde{I}_{n}(m_{0})\right|.$

Consequently, for any $\xi>0$, we have that

$\displaystyle P_{0}\left(\sup_{(\beta,m)\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})\times(B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n})}|R_{n}(\beta,m)|>\xi\right)$
$\displaystyle\quad\leq P_{0}\left(M_{n}\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)-\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right|>\frac{\xi}{2}\right)$
$\displaystyle\quad\quad+P_{0}\left(\frac{M_{n}^{2}}{2}\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|\tilde{I}_{n}(m)-\tilde{I}_{n}(m_{0})\right|>\frac{\xi}{2}\right).$

Since $M_{n}\rightarrow\infty$ arbitrarily slowly as $n\rightarrow\infty$, it
follows from Part 1 and Part 2 of the lemma that

$\displaystyle P_{0}\left(M_{n}\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)-\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right|>\frac{\xi}{2}\right)\rightarrow
0$

and

$\displaystyle P_{0}\left(\frac{M_{n}^{2}}{2}\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|\tilde{I}_{n}(m)-\tilde{I}_{n}(m_{0})\right|>\frac{\xi}{2}\right)\rightarrow
0$

as $n\rightarrow\infty$. This establishes that for any $\xi>0$,

$\displaystyle P_{0}\left(\sup_{(\beta,m)\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})\times(B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n})}|R_{n}(\beta,m)|>\xi\right)\rightarrow
0$

as $n\rightarrow\infty$ if $M_{n}\rightarrow\infty$ arbitrarily slowly. ∎

###### Proof of Lemma 2.

The proof is comprised of three steps. Step 1 obtains an expression for the
posterior based on a quadratic expansion of the log-likelihood function. Step
2 shows that the numerator is uniformly bounded from above by
$\exp(-(\underline{c}_{0}/4)M_{n}^{2})$ with $P_{0}$-probability approaching
$1$, where $\underline{c}_{0}>0$ is the constant defined in Part 3 of Lemma 1.
Step 3 shows that the denominator is uniformly bounded from below by
$\exp(-(\underline{c}_{0}/8)M_{n}^{2})$ with $P_{0}$-probability approaching
$1$. The arguments for Step 2 and Step 3 are similar to those encountered on
pp. 233-234 and the proof of Lemma 6.3, respectively, in Bickel and Kleijn
(2012) with appropriate modifications to account for the different
parametrization.

Step 1. We start with the definition of the posterior:

$\displaystyle\Pi\left(\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}|\\{(Y_{i},X_{i},W_{i})\\}_{i=1}^{n},m\right)$
$\displaystyle\quad=\frac{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\exp\left(\ell_{n}(\beta,m)-\ell_{n}(\beta_{0},m)\right)d\Pi_{\mathcal{B}}(\beta)}{\int_{\mathcal{B}}\exp\left(\ell_{n}(\beta,m)-\ell_{n}(\beta_{0},m)\right)d\Pi_{\mathcal{B}}(\beta)}.$

As $p_{\beta,m}(Y_{i},X_{i}|W_{i})$ is the density of
$\mathcal{N}(m(W_{i}),V(\beta))$, we obtain the following expansion of the
log-likelihood function around the true parameter $\beta_{0}$

$\displaystyle\ell_{n}(\beta,m)-\ell_{n}(\beta_{0},m)=\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}n(\beta-\beta_{0})^{2}\tilde{I}_{n}(m).$

Consequently,

$\displaystyle\Pi\left(\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}|\\{(Y_{i},X_{i},W_{i})\\}_{i=1}^{n},m\right)$
$\displaystyle=\frac{\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}n(\beta-\beta_{0})^{2}\tilde{I}_{n}(m)\right)d\Pi_{\mathcal{B}}(\beta)}{\int_{\mathcal{B}}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}n(\beta-\beta_{0})^{2}\tilde{I}_{n}(m)\right)d\Pi_{\mathcal{B}}(\beta)}.$

Step 2. Let $A_{n,1}$ be the event that

$\displaystyle\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}n(\beta-\beta_{0})^{2}\tilde{I}_{n}(m)\right)d\Pi_{\mathcal{B}}(\beta)$

is less than or equal to $\exp(-\underline{c}_{0}M_{n}^{2}/4)$. The first task
is to show that $P_{0}(A_{n,1}^{c})\rightarrow 0$ as $n\rightarrow\infty$.
Using the definition of the supremum and that probabilities are bounded from
above by $1$, I obtain the following string of inequalities

$\displaystyle\int_{B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}n(\beta-\beta_{0})^{2}\tilde{I}_{n}(m)\right)d\Pi_{\mathcal{B}}(\beta)$
$\displaystyle\quad\leq\sup_{\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}n(\beta-\beta_{0})^{2}\tilde{I}_{n}(m)\right)$
$\displaystyle\quad\leq\exp\left(\sup_{\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}n(\beta-\beta_{0})^{2}\tilde{I}_{n}(m)\right)\right)$
$\displaystyle\quad\leq\exp\left(\sup_{\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\left(\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\right|\cdot|\sqrt{n}(\beta-\beta_{0})|-\frac{1}{2}n(\beta-\beta_{0})^{2}\tilde{I}_{n}(m)\right)\right)$

for all $m\in\mathcal{M}_{n}\cap B_{\mathcal{M},\infty}(m_{0},D\delta_{n})$.
From Part 1 of Lemma 1, I know that

$\displaystyle\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)-\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right|\overset{P_{0}}{\rightarrow}0.$

Since $(1/\sqrt{n})\tilde{\ell}_{n}(\beta_{0},m_{0})=O_{P_{0}}(1)$ by the
Lindeberg-Lévy Central Limit Theorem (CLT), it follows that

$\displaystyle P_{0}\left(\sup_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\right|<\frac{\underline{c}_{0}}{4}M_{n}\right)\rightarrow
1.$

The CLT is valid because
$E_{P_{0}}(\tilde{\ell}(Y_{i},X_{i},W_{i},\beta_{0},m_{0}))=0$ and
$Var_{P_{0}}(\tilde{\ell}(Y_{i},X_{i},W_{i},\beta_{0},m_{0}))=\tilde{I}_{0}(m_{0})\in[\underline{c}_{0},\overline{c}_{0}]$
by Part 2 of Assumption 1. Moreover, Part 3 of Lemma 1 implies that

$\displaystyle P_{0}\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\tilde{I}_{n}(m)\geq\underline{c}_{0}\right)\rightarrow
1.$

Consequently, I obtain the inequality

$\displaystyle
P_{0}(A_{n,1}^{c})\leq\mathbf{1}\left\\{\underline{c}_{0}\sup_{\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\left(\frac{1}{4}M_{n}\cdot|\sqrt{n}(\beta-\beta_{0})|-\frac{1}{2}n(\beta-\beta_{0})^{2}\right)>-\frac{\underline{c}_{0}}{4}M_{n}^{2}\right\\}+o(1).$

Defining $t=\sqrt{n}|\beta-\beta_{0}|$, it follows that

$\displaystyle\sup_{\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\left[\frac{M_{n}}{4}\sqrt{n}|\beta-\beta_{0}|-\frac{1}{2}n(\beta-\beta_{0})^{2}\right]=\sup_{t\in
B_{\mathcal{B}}(0,M_{n})^{c}}\left[\frac{M_{n}}{4}t-\frac{1}{2}t^{2}\right]=-\frac{M_{n}^{2}}{4}$

as a concave quadratic function maximized subject to the constraint that
$t\geq M_{n}$. This implies that

$\displaystyle\mathbf{1}\left\\{\underline{c}_{0}\sup_{\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}}\left(\frac{1}{4}M_{n}\cdot|\sqrt{n}(\beta-\beta_{0})|-\frac{1}{2}n(\beta-\beta_{0})^{2}\right)>-\frac{\underline{c}_{0}}{4}M_{n}^{2}\right\\}=0$

and leads to the conclusion that $P_{0}(A_{n,1}^{c})\rightarrow 0$, and, as a
result, the complement rule implies $P_{0}(A_{n,1})\rightarrow 1$ as
$n\rightarrow\infty$.

Step 3. Let $A_{n,2}$ be the event that

$\displaystyle\inf_{m\in
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})\cap\mathcal{M}_{n}}\int_{\mathcal{B}}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}n(\beta-\beta_{0})^{2}\tilde{I}_{n}(m)\right)d\Pi_{\mathcal{B}}(\beta)$

is bounded from below by $\exp\left(-\underline{c}_{0}M_{n}^{2}/{8}\right)$. I
want to show that $P_{0}(A_{n,2}^{c})\rightarrow 0$ because combined with the
result from Step 2, it is sufficient to conclude the result of the lemma
because it implies

$\displaystyle P_{0}\left(\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\Pi\left(\beta\in
B_{\mathcal{B}}(\beta_{0},M_{n}/\sqrt{n})^{c}|\\{(Y_{i},X_{i},W_{i})\\}_{i=1}^{n},m\right)\leq\exp\left(-\frac{\underline{c}_{0}}{8}M_{n}^{2}\right)\right)\rightarrow
1.$

Let $C=\underline{c}_{0}/8\overline{c}_{0}$ with
$0<\underline{c}_{0}<\overline{c}_{0}<\infty$ defined as in Part 3 of Lemma 1.
Since I modify the proof of Lemma 6.3 in Bickel and Kleijn (2012) for this
step, I elect to work in the local parametrization
$s=\sqrt{n}(\beta-\beta_{0})$. Consequently, I let $\Pi_{n}$ be the induced
prior for $s$ (i.e., the pushforward of $\Pi_{\mathcal{B}}$ through the map
$\beta\mapsto\sqrt{n}(\beta-\beta_{0})$), let $\mathcal{S}$ be the support of
$\Pi_{n}$, and let $\mathcal{S}_{C}=\\{s:|s|\leq\sqrt{C}\\}$. Under Assumption
2, for large $n$, the induced prior $\Pi_{n}$ admits a Lebesgue density
$\pi_{n}$ such that $\inf_{s\in\mathcal{S}_{C}}\pi_{n}(s)\geq\tilde{C}$ for
some large constant $\tilde{C}>0$. Consequently, I apply a change of variables
and conclude that for $n$ large,

$\displaystyle\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\int_{\mathcal{B}}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\sqrt{n}(\beta-\beta_{0})-\frac{1}{2}n(\beta-\beta_{0})^{2}\tilde{I}_{n}(m)\right)d\Pi_{\mathcal{B}}(\beta)$
$\displaystyle\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\int_{\mathcal{S}}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)s-\frac{1}{2}s^{2}\tilde{I}_{n}(m)\right)d\Pi_{n}(s)$
$\displaystyle\geq\tilde{C}\lambda(\mathcal{S}_{C})\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\int\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)s-\frac{1}{2}s^{2}\tilde{I}_{n}(m)\right)\pi_{\mathcal{S}_{C}}(s)ds,$

where $\lambda$ is the Lebesgue measure on $\mathbb{R}$ and
$\pi_{\mathcal{S}_{C}}(s)=\mathbf{1}\\{s\in\mathcal{S}_{C}\\}/\lambda(\mathcal{S}_{C})$
is the density of a uniform distribution on $\mathcal{S}_{C}$. Applying Part 3
of Lemma 1, it follows that

$\displaystyle P_{0}\left(\sup_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\tilde{I}_{n}(m)\leq\overline{c}_{0}M_{n}^{2}\right)\rightarrow
1$

for any $M_{n}\rightarrow\infty$. Using that
$C=\underline{c}_{0}/8\overline{c}_{0}$, it then follows that

$\displaystyle-\frac{1}{2}s^{2}\tilde{I}_{n}(m)\geq-\frac{\underline{c}_{0}}{16}M_{n}^{2}$

for all $(s,m)\in\mathcal{S}_{C}\times(\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n}))$ holds with $P_{0}$-probability
approaching $1$ as $n\rightarrow\infty$. Consequently,
$P_{0}\left(A_{n,2}^{c}\right)\leq P_{0}(\tilde{A}_{n,2}^{c})+o(1)$, where
$\tilde{A}_{n,2}$ is the event

$\displaystyle\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\int\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)s\right)\pi_{\mathcal{S}_{C}}(s)ds\geq\left(\lambda(\mathcal{S}_{C})\tilde{C}\right)^{-1}\exp\left(-\frac{\underline{c}_{0}}{16}M_{n}^{2}\right).$

Applying Jensen’s inequality,

$\displaystyle\int\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)s\right)\pi_{\mathcal{S}_{C}}(s)ds\geq\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\int
s\pi_{\mathcal{S}_{C}}(s)ds\right)$

for all $m\in\mathcal{M}_{n}\cap B_{\mathcal{M},\infty}(m_{0},D\delta_{n})$.
As the infimum is the greatest lower bound, I then obtain

$\displaystyle\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\int\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)s\right)\pi_{\mathcal{S}_{C}}(s)ds$
$\displaystyle\quad\geq\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\int
s\pi_{\mathcal{S}_{C}}(s)ds\right)$
$\displaystyle\quad\geq\exp\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)\int
s\pi_{\mathcal{S}_{C}}(s)ds\right)$
$\displaystyle\quad\geq\exp\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\left(\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)-\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right)\int
s\pi_{\mathcal{S}_{C}}(s)ds\right)\right)$
$\displaystyle\quad\quad\times\exp\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\int
s\pi_{\mathcal{S}_{C}}(s)ds\right),$

where the last inequality adds/subtracts
$(1/\sqrt{n})\tilde{\ell}_{n}(\beta_{0},m_{0})$ and uses that the sum of the
infima is less than or equal to the infimum of the sum. Since

$\displaystyle\left|\int
s\pi_{\mathcal{S}_{C}}(s)ds\right|\leq\int|s|\pi_{\mathcal{S}_{C}}(s)ds\leq\sqrt{C}$

because $\pi_{\mathcal{S}_{C}}$ is supported on $\\{s:|s|\leq\sqrt{C}\\}$ and
$\pi_{\mathcal{S}_{C}}$ integrates to $1$ (i.e., it is a probability density
function), it follows from Part 1 of Lemma 1 that

$\displaystyle\exp\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\left(\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m)-\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right)\int
s\pi_{\mathcal{S}_{C}}(s)ds\right)\right)=1+o_{P_{0}}(1)$

As such, I arrive at the conclusion

$\displaystyle P_{0}\left(\inf_{m\in\mathcal{M}_{n}\cap
B_{\mathcal{M},\infty}(m_{0},D\delta_{n})}\int_{\mathcal{B}}\exp(\ell_{n}(\beta,m)-\ell_{n}(\beta_{0},m))d\Pi_{\mathcal{B}}(\beta)<\exp\left(-\frac{\underline{c}_{0}}{8}M_{n}^{2}\right)\right)$
$\displaystyle\quad\leq
P_{0}\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\int
s\pi_{\mathcal{S}_{C}}(s)ds<-\log\tilde{C}-\log\lambda(\mathcal{S}_{C})-\frac{\underline{c}_{0}}{16}M_{n}^{2}\right)+o(1).$

Since
$-\log\tilde{C}-\log\lambda(\mathcal{S}_{C})\leq(\underline{c}_{0}/32)M_{n}^{2}$
for $n$ large, I can further conclude that

$\displaystyle P_{0}\left(A_{n,2}^{c}\right)\leq
P_{0}\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\int
s\pi_{\mathcal{S}_{C}}(s)ds<-\frac{\underline{c}_{0}}{32}M_{n}^{2}\right)+o(1).$

Next, the event

$\displaystyle\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\int
s\pi_{\mathcal{S}_{C}}(s)ds<-\frac{\underline{c}_{0}}{32}M_{n}^{2}$

implies

$\displaystyle\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\int
s\pi_{\mathcal{S}_{C}}(s)ds\right|>\frac{\underline{c}_{0}}{32}M_{n}^{2},$

and, as a result, I can apply Chebyshev’s inequality to conclude that

$\displaystyle
P_{0}\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\int
s\pi_{\mathcal{S}_{C}}(s)ds<-\frac{\underline{c}_{0}}{32}M_{n}^{2}\right)$
$\displaystyle\quad\lesssim\frac{1}{M_{n}^{4}}E_{P_{0}}\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right|^{2}\left|\int
s\pi_{\mathcal{S}_{C}}(s)ds\right|^{2}$
$\displaystyle\quad\lesssim\frac{1}{M_{n}^{4}}E_{P_{0}}\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right|^{2}$

with the second inequality using Jensen’s inequality and that
$\sup_{s\in\mathcal{S}_{C}}|s|^{2}\leq C.$ Since
$E_{P_{0}}[\tilde{\ell}_{n}(\beta_{0},m_{0})]=0$, it follows that

$\displaystyle
E_{P_{0}}\left|\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right|^{2}=Var_{P_{0}}\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\right)=Var_{P_{0}}\left(\tilde{\ell}(Y_{i},X_{i},W_{i},\beta_{0},m_{0}))\right)=\tilde{I}_{0}(m_{0})\leq\overline{c}_{0}$

with the inequality reflecting Part 2 of Assumption 1. Consequently, I obtain

$\displaystyle
P_{0}\left(\frac{1}{\sqrt{n}}\tilde{\ell}_{n}(\beta_{0},m_{0})\int
s\pi_{\mathcal{S}_{C}}(s)ds<-\frac{\underline{c}_{0}}{32}M_{n}^{2}\right)=O(M_{n}^{-4}),$

and, as a result, $P_{0}(A_{n,2}^{c})\rightarrow 0$ because
$M_{n}\rightarrow\infty$. Since I chose the sequence
$\\{M_{n}\\}_{n=1}^{\infty}$ with $M_{n}\rightarrow\infty$ arbitrarily, the
argument holds for any such choice and I conclude the proof. ∎

###### Proof of Lemma 3.

The proof has three steps. Step 1 establishes an exponentially small prior
mass condition by modifying the arguments from Theorem 3.1 of de Jonge and van
Zanten (2013). Step 2 establishes that the posterior probability of
$\mathcal{M}_{n}$ converges in $P_{0}$-probability to $1$ as
$n\rightarrow\infty$. Step 3 verifies a claim that is asserted in Step 2.

Step 1. Since $m_{j}$ takes values in $C^{a_{j}}([0,1])$ for any
$a_{j}\in[0,\alpha_{j})$, I can view $m_{j}$ as a Gaussian element in the
Banach space $(C^{a_{j}}[0,1],||\cdot||_{a_{j}})$ for
$a_{j}\in(1/2,\alpha_{j})$. Such an $a_{j}$ exists because $\alpha_{j}>1/2$.
The RKHS is unchanged because $C([0,1])$ is the completion of
$C^{a_{j}}([0,1])$ with respect to the uniform norm $||\cdot||_{\infty}$ and
$||\cdot||_{\infty}\leq||\cdot||_{a_{j}}$ (see Lemma 8.1 in van der Vaart and
van Zanten (2008b)). I then apply Borell’s inequality (i.e., Theorem 5.1 in
van der Vaart and van Zanten (2008b)) to conclude that the law $\text{P}_{j}$
satisfies the inequality

$\displaystyle\text{P}_{j}(\mathcal{M}_{nj}^{c})\leq
1-\Phi\left(\Phi^{-1}\left(\text{P}_{j}(m_{j}\in\gamma_{n,j}C_{1}^{a_{j}}([0,1]))\right)+M_{j}\sqrt{n}\delta_{n}\right),$

where $\Phi(\cdot)$ denotes the standard normal cumulative distribution
function. Without loss of generality, I can assume that $\delta_{n}\geq
n^{-\alpha_{1}/(2\alpha_{1}+1)}$ and $\delta_{n}\geq
n^{-\alpha_{2}/(2\alpha_{2}+1)}$. Indeed, I can always redefine the sequence
$\delta_{n}$ as
$\max\\{\delta_{n},n^{-\alpha_{1}/(2\alpha_{1}+1)},n^{-\alpha_{2}/(2\alpha_{2}+1)}\\}$
without affecting the crucial $o(n^{-1/4})$ requirement. Since the unit ball
$\mathbb{H}_{1,j}$ in $(\mathbb{H}_{j},||\cdot||_{\mathbb{H}_{j}})$ satisfies
the metric entropy condition $\log
N(\tau,\mathbb{H}_{1,j},||\cdot||_{a_{j}})\lesssim\tau^{-2/[1+2(\alpha_{j}-a_{j})]}$
for any $\tau>0$, I can invoke the relationship between the metric entropy of
the unit ball in RKHS and small ball probabilities established in Li and Linde
(1999) to find a constant $L_{j}>0$ such that

$\displaystyle\text{P}_{j}(||m_{j}||_{a_{j}}<\gamma_{n,j})\geq\exp(-L_{j}\gamma_{n,j}^{-1/(\alpha_{j}-a_{j})})$

for each $j\in\\{1,2\\}$. Since the sequence
$\gamma_{n,j}=n^{-(\alpha_{j}-a_{j})/(2\alpha_{j}+1)}$ satisfies
$\gamma_{n,j}^{-1/(\alpha_{j}-a_{j})}=n^{1/(2\alpha_{j}+1)}\leq
n\delta_{n}^{2}$ because of the lower bound $\delta_{n}\geq
n^{-\alpha_{j}/(2\alpha_{j}+1)}$, it follows that

$\displaystyle\text{P}_{j}(||m_{j}||_{a_{j}}<\gamma_{n,j})\geq\exp(-L_{j}n\delta_{n}^{2}).$

Consequently,

$\displaystyle\text{P}_{j}(\mathcal{M}_{nj}^{c})\leq
1-\Phi\left(\Phi^{-1}\left(\exp(-L_{j}n\delta_{n}^{2})\right)+M_{j}\sqrt{n}\delta_{n}\right)$

for each $j\in\\{1,2\\}$. Using the inequality
$\Phi^{-1}(p)\geq-\sqrt{2\log(1/p)}$ for any $p\in(0,1)$, it follows that

$\displaystyle\Phi^{-1}\left(\exp(-L_{j}n\delta_{n}^{2})\right)\geq-\sqrt{2L_{j}n\delta_{n}^{2}}=-\tilde{L}_{j}\sqrt{n}\delta_{n}$

for $\tilde{L}_{j}=\sqrt{2L_{j}}$. Consequently,

$\displaystyle\text{P}_{j}(\mathcal{M}_{nj}^{c})\lesssim
1-\Phi((M_{j}-\tilde{L}_{j})\sqrt{n}\delta_{n}).$

By choosing $M_{j}>\tilde{L}_{j}$ arbitrarily large and using the inequality
$\Phi(z)\geq 1-\exp(-z^{2}/2)$, it follows that

$\displaystyle\text{P}_{j}(\mathcal{M}_{nj}^{c})\lesssim\exp(-\frac{1}{2}(M_{j}-\tilde{L}_{j})^{2}n\delta_{n}^{2})$

for each $j\in\\{1,2\\}$. Since
$\Pi_{\mathcal{M}}(\mathcal{M}_{n}^{c})\lesssim\max\\{\text{P}_{1}(\mathcal{M}_{n1}^{c}),\text{P}_{2}(\mathcal{M}_{n2}^{c})\\}$,
it follows that there exists a large constant $M>0$ such that
$\Pi_{\mathcal{M}}(\mathcal{M}_{n}^{c})\lesssim\exp(-Mn\delta_{n}^{2})$.

Step 2. Let $A_{n}$ be the event that

$\displaystyle\int_{\mathcal{B}\times\mathcal{M}}\frac{L_{n}(\beta,m)}{L_{n}(\beta_{0},m_{0})}d\Pi(\beta,m)\geq\exp\left(-Cn\delta_{n}^{2}\right).$

for some constant $C>0$. Since probability measures are bounded between zero
and one,

$\displaystyle\Pi(\mathcal{M}_{n}^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})$
$\displaystyle=\Pi(\mathcal{M}_{n}^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\mathbf{1}\\{A_{n}\\}+\Pi(\mathcal{M}_{n}^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\mathbf{1}\\{A_{n}^{c}\\}$
$\displaystyle\leq\Pi(\mathcal{M}_{n}^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\mathbf{1}\\{A_{n}\\}+\mathbf{1}\\{A_{n}^{c}\\},$

and, taking expectations, it follows that

$\displaystyle
E_{P_{0}}\left(\Pi(\mathcal{M}_{n}^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\right)\leq
E_{P_{0}}\left(\Pi(\mathcal{M}_{n}^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\mathbf{1}\\{A_{n}\\}\right)+P_{0}(A_{n}^{c}).$

If $P_{0}(A_{n}^{c})\rightarrow 0$ as $n\rightarrow\infty$, it remains to show
that

$\displaystyle
E_{P_{0}}\left(\Pi(\mathcal{M}_{n}^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\mathbf{1}\\{A_{n}\\}\right)\rightarrow
0$

as $n\rightarrow\infty$. Using the definition of the posterior,

$\displaystyle
E_{P_{0}}\left(\Pi(\mathcal{M}_{n}^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\mathbf{1}\\{A_{n}\\}\right)$
$\displaystyle\leq
E_{P_{0}}\left(\int_{\mathcal{B}\times\mathcal{M}_{n}^{c}}\frac{L_{n}(\beta,m)}{L_{n}(\beta_{0},m_{0})}d\Pi(\beta,m)\right)\exp\left(Cn\delta_{n}^{2}\right)$
$\displaystyle=\int_{\mathcal{B}\times\mathcal{M}_{n}^{c}}E_{P_{0}}\left(\frac{L_{n}(\beta,m)}{L_{n}(\beta_{0},m_{0})}\right)d\Pi(\beta,m)\exp\left(Cn\delta_{n}^{2}\right)$
$\displaystyle=\Pi(\mathcal{M}_{n}^{c})\exp\left(Cn\delta_{n}^{2}\right),$

where the first equality applies Fubini’s theorem while the second equality
uses the assumption of i.i.d sampling to obtain

$\displaystyle
E_{P_{0}}\left(\frac{L_{n}(\beta,m)}{L_{n}(\beta_{0},m_{0})}\right)=\left(E_{P_{0}}\left(\frac{p_{\beta,m}(Y_{i},X_{i}|W_{i})}{p_{\beta_{0},m_{0}}(Y_{i},X_{i}|W_{i})}\right)\right)^{n}=1$

under the assumption that $p_{0}(\cdot|W_{i})$ is that of
$\mathcal{N}(m_{0}(W_{i}),V(\beta_{0}))$. Using the result from Step 1 and
that $n\delta_{n}^{2}\rightarrow\infty$ as $n\rightarrow\infty$, it follows
that

$\displaystyle
E_{P_{0}}\left(\Pi(\mathcal{M}_{n}^{c}|\\{(Y_{i},X_{i},W_{i}^{\prime})^{\prime}\\}_{i=1}^{n})\mathbf{1}\\{A_{n}\\}\right)\rightarrow
0$

as $n\rightarrow\infty$. Indeed, I can choose $M_{1},M_{2}>0$ arbitrarily
large so that $-M+C<0$ and, as a result, implies that
$\Pi(\mathcal{M}_{n}^{c})\exp\left(Cn\delta_{n}^{2}\right)\leq\exp(-(M+C)n\delta_{n}^{2})\rightarrow
0$ as $n\rightarrow\infty$. Since I asserted that $P_{0}(A_{n})\rightarrow 1$
as $n\rightarrow\infty$, it remains to verify that the claim holds.

Step 3. For each $\delta>0$, I define neighborhoods

$\displaystyle\mathcal{V}_{2}(\delta)=\left\\{(\beta,m):E_{P_{0}}\left(\log\frac{p_{\beta_{0},m_{0}}(Y_{i},X_{i}|W_{i})}{p_{\beta,m}(Y_{i},X_{i}|W_{i})}\right)\leq\delta^{2},\
E_{P_{0}}\left(\left(\log\frac{p_{\beta_{0},m_{0}}(Y_{i},X_{i}|W_{i})}{p_{\beta,m}(Y_{i},X_{i}|W_{i})}\right)^{2}\right)\leq\delta^{2}\right\\}$

Applying Lemma 8.1 of Ghosal et al. (2000), for any constant
$\tilde{C}_{1}>0$, the event

$\displaystyle\int_{\mathcal{B}\times\mathcal{M}}\frac{L_{n}(\beta,m)}{L_{n}(\beta_{0},m_{0})}d\Pi(\beta,m)\geq\Pi\left(\mathcal{V}_{2}(\delta_{n})\right)\exp\left(-(1+\tilde{C}_{1})n\delta_{n}^{2}\right)$

occurs with $P_{0}$-probability approaching one as $n\rightarrow\infty$.
Consequently, I need to show that there exists a constant $\tilde{C}_{2}>0$
such that
$\Pi(\mathcal{V}_{2}(\delta_{n}))\gtrsim\exp(-\tilde{C}_{2}n\delta_{n}^{2})$
for $n$ large because we can set $C=1+\tilde{C}_{1}+\tilde{C}_{2}$ and
conclude that $P_{0}(A_{n})\rightarrow 1$ as $n\rightarrow\infty$. Consider
the event
$\\{\max\\{|\beta-\beta_{0}|,||m-m_{0}||_{\infty}\\}<\delta_{n}/\sqrt{2}\\}$.
Since $\\{\delta_{n}\\}_{n=1}^{\infty}$ satisfies $\delta_{n}\rightarrow 0$ as
$n\rightarrow\infty$, there exists a constant $\tilde{\delta}>0$ such that
$\delta_{n}\leq\tilde{\delta}$ for each $n\geq 1$. Consequently, this event
implies that pairs $(\beta,m)$ satisfy
$|\beta-\beta_{0}|^{q}\leq\tilde{\delta}^{q-2}|\beta-\beta_{0}|^{2}$ and
$||m-m_{0}||_{\mathcal{M},\infty}^{q}\leq\tilde{\delta}^{q-2}||m-m_{0}||_{\mathcal{M},\infty}^{2}$
for any $q\geq 2$. We now use these inequalities to conclude that the moments
in $V_{2}(\delta_{n})$ are bounded from above by a sequence that is
proportional to $\delta_{n}^{2}$. Using the expression for the Gaussian
density and the observation that $\det
V(\beta)=\sigma_{01}^{2}\sigma_{02}^{2}$ for all $\beta\in\mathcal{B}$,

$\displaystyle\log\frac{p_{\beta_{0},m_{0}}(Y_{i},X_{i}|W_{i})}{p_{\beta,m}(Y_{i},X_{i}|W_{i})}$
$\displaystyle\quad=\log\frac{p_{\beta_{0},m_{0}}(Y_{i},X_{i}|W_{i})}{p_{\beta,m_{0}}(Y_{i},X_{i}|W_{i})}+\log\frac{p_{\beta,m_{0}}(Y_{i},X_{i}|W_{i})}{p_{\beta,m}(Y_{i},X_{i}|W_{i})}$
$\displaystyle\quad=\frac{1}{2}\varepsilon_{i2}^{2}(\beta-\beta_{0})^{2}-U_{i}\varepsilon_{i2}(\beta-\beta_{0})-(m(W_{i})-m_{0}(W_{i}))^{\prime}V(\beta)^{-1}\varepsilon_{i}$
$\displaystyle\quad\quad+\frac{1}{2}(m(W_{i})-m_{0}(W_{i}))^{\prime}V(\beta)^{-1}(m(W_{i})-m_{0}(W_{i})),$

where $\varepsilon_{i}=(\varepsilon_{i1},\varepsilon_{i2})^{\prime}$ satisfies
$\varepsilon_{i}|W_{i}\sim\mathcal{N}(0,V(\beta_{0}))$, and $U_{i}$ satisfies
$U_{i}|W_{i},X_{i}\sim\mathcal{N}(0,\sigma_{01}^{2})$ with
$U_{i}\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}\varepsilon_{i2}$. Compactness of
$\mathcal{B}$ implies that $\\{V(\beta):\beta\in\mathcal{B}\\}$ forms a
compact set of symmetric and positive definite matrices, and, as a result,
$\inf_{\beta\in\mathcal{B}}\lambda_{min}(V(\beta))>0$. This uniform positive
definiteness condition, the distributional properties that $\varepsilon_{i}$
and $U_{i}$ satisfy, the assumption that distribution of $W_{i}$ has a density
that is bounded away from zero and infinity on $[0,1]$, and the inequalities
above then lead me to conclude that

$\displaystyle
E_{P_{0}}\left(\log\frac{p_{\beta_{0},m_{0}}(Y_{i},X_{i}|W_{i})}{p_{\beta,m}(Y_{i},X_{i}|W_{i})}\right)$
$\displaystyle\lesssim|\beta-\beta_{0}|^{2}+||m-m_{0}||_{\mathcal{M},\infty}^{2}\lesssim\delta_{n}^{2}$

and

$\displaystyle
E_{P_{0}}\left(\left(\log\frac{p_{\beta_{0},m_{0}}(Y_{i},X_{i}|W_{i})}{p_{\beta,m}(Y_{i},X_{i}|W_{i})}\right)^{2}\right)$
$\displaystyle\lesssim|\beta-\beta_{0}|^{2}+||m-m_{0}||_{\mathcal{M},\infty}^{2}\lesssim\delta_{n}^{2}$

on the event
$\\{(\beta,m):|\beta-\beta_{0}|<\delta_{n}/\sqrt{2},||m-m_{0}||_{\infty}<\delta_{n}/\sqrt{2}\\}$.
In other words, there exists a constant $C^{\dagger}>0$ such that the open
ball
$\\{(\beta,m):\max\\{|\beta-\beta_{0}|,||m-m_{0}||_{\infty}\\}<C^{\dagger}\delta_{n}\\}$
is contained in $\mathcal{V}_{2}(\delta_{n})$ for $n$ large. As such, I need
to show that
$\Pi(\max\\{|\beta-\beta_{0}|,||m-m_{0}||_{\infty}\\}<C^{\dagger}\delta_{n})\gtrsim\exp(-\tilde{C}_{2}n\delta_{n}^{2})$
for $n$ large. Since $\Pi=\Pi_{\mathcal{B}}\times\Pi_{\mathcal{M}}$,

$\displaystyle\Pi\left(\max\\{|\beta-\beta_{0}|,||m-m_{0}||_{\infty}\\}<C^{\dagger}\delta_{n}\right)=\Pi_{\mathcal{B}}\left(|\beta-\beta_{0}|<C^{\dagger}\delta_{n}\right)\Pi_{\mathcal{M}}\left(||m-m_{0}||_{\mathcal{M},\infty}<C^{\dagger}\delta_{n}\right).$

Moreover, the assumption that $\Pi_{\mathcal{B}}$ admits a strictly positive
and continuous Lebesgue density supported on a compact set $\mathcal{B}$
guarantees that

$\displaystyle\Pi_{\mathcal{B}}(|\beta-\beta_{0}|<C^{\dagger}\delta_{n})\gtrsim\delta_{n}$

Applying an inequality for small ball probabilities for Gaussian processes
(i.e., Lemma 5.3 of van der Vaart and van Zanten (2008b)) and the inequality
$\varphi_{m_{0}}(\delta_{n})\leq n\delta_{n}^{2}$, I conclude that there
exists a constant $\tilde{C}_{2}>0$ such that

$\displaystyle\Pi\left(\max\\{|\beta-\beta_{0}|,||m-m_{0}||_{\infty}\\}<C^{\dagger}\delta_{n}\right)\gtrsim\exp(-\tilde{C}_{2}n\delta_{n}^{2})$

for $n$ large. Consequently,