u^{j+k-2}K(u)f_{0}(a_{0}+uh)\,du.$

By (A5), $f_{0}(a_{0}+uh)=f_{0}(a_{0})+O(uh)$, and since $K$ has support
$[-1,1]$ by (A1), we then have

$\int u^{j+k-2}K(u)f_{0}(a_{0}+uh)\,du=f_{0}(a_{0})\int
u^{j+k-2}K(u)\,du+O(h)=f_{0}(a_{0})c_{j+k-2}+O(h).$

Hence, $\mathbf{D}_{0,h,a_{0},1}=f_{0}(a_{0})\mathbf{S}_{2}+O(h)$, so
$\mathbf{D}_{0,h,a_{0},1}^{-1}=f_{0}(a)^{-1}\mathbf{S}_{2}^{-1}+O(h)$ since
$f(a_{0})>0$. The proof for $\mathbf{D}_{0,b,a_{0},3}$ is analogous. For the
uniform result, we have for each $j,k\in\\{1,2\\}$,

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbf{D}_{0,h,a_{0},1}[j,k]-f_{0}(a_{0})c_{j+k+2}\right|$
$\displaystyle=\sup_{a_{0}\in\mathcal{A}_{0}}\left|\int
u^{j+k-2}K(u)\left[f_{0}(a_{0}+uh)-f_{0}(a_{0})\right]\,du\right|$
$\displaystyle\leq\sup_{a_{0}\in\mathcal{A}_{0}}\int\left|u\right|^{j+k-2}K(u)\left|f_{0}(a_{0}+uh)-f_{0}(a_{0})\right|\,du$
$\displaystyle\leq Ch$

for some $C<\infty$ because $f_{0}$ is Lipschitz on $\mathcal{A}_{\delta_{3}}$
and $K$ is uniformly bounded with compact support. The result follows, and a
similar argument yields the result for $\mathbf{D}_{0,b,a_{0},3}^{-1}$. ∎

######  Lemma 5.

If (A1)–(A2), (A4)(a), and (A5) hold, then

$(nh)^{1/2}\mathbb{P}_{n}\phi^{*}_{\infty,h,b,a_{0}}\operatorname*{\stackrel{{\scriptstyle
d}}{{\longrightarrow}}}\mathcal{N}\left(0,V_{K,\tau}f_{0}(a_{0})^{-1}\sigma_{0}^{2}(a_{0})\right),$

where

$V_{K,\tau}:=\int\left\\{K(u)-\tau^{3}c_{2}\frac{(\tau
u)^{2}-c_{2}}{c_{4}-c_{2}^{2}}K(\tau
u)\right\\}^{2}\,du=c_{0}^{*}-2\tau^{3}c_{2}\frac{\tau^{2}c_{2,\tau}^{*}-c_{2}c_{0,\tau}^{*}}{c_{4}-c_{2}^{2}}+\tau^{5}c_{2}^{2}\frac{c_{4}^{*}-2c_{2}c_{2}^{*}+c_{2}^{2}c_{0}^{*}}{\left(c_{4}-c_{2}\right)^{2}}$

is a positive, finite constant for all $\tau\in(0,\infty)$ and kernels $K$
satisfying (A1). Furthermore, $hP_{0}(\phi^{*}_{\infty,h,b,a_{0}})^{2}$
converges to $V_{K,\tau}\sigma_{0}^{2}(a_{0})/f_{0}(a_{0})$ as $h\to 0$.

###### Proof of Lemma 5.

By adding and subtracting $\mathbb{P}_{n}\Gamma_{0,h,b,a_{0}}\theta_{0}$, we
can rewrite $\mathbb{P}_{n}\phi^{*}_{\infty,h,b,a_{0}}$ as

$\displaystyle\mathbb{P}_{n}\phi_{\infty,h,b,a_{0}}^{*}$
$\displaystyle=\mathbb{P}_{n}\left\\{\Gamma_{0,h,b,a_{0}}(\xi_{\infty}-\theta_{0})\right\\}+\mathbb{P}_{n}\left\\{\Gamma_{0,h,b,a_{0}}\theta_{0}-\gamma_{0,h,b,a_{0}}\right\\}+\mathbb{P}_{n}\left\\{\int\Gamma_{0,h,b,a_{0}}\left(\mu_{\infty}-\int\mu_{\infty}\,dQ_{0}\right)\,dF_{0}\right\\}$

We show that only the first term contributes to the limit distribution; the
remaining two terms are each $o_{p}(\\{nh\\}^{-1/2})$. We use the Lyapunov
central limit theorem for triangular arrays to demonstrate that the first term
is asymptotically normal. The Lyapunov CLT implies that if for each $n$,
$X_{n,1},X_{n,2},\dots X_{n,n}$ are IID, mean-zero random variables that
satisfy (1) $\lim_{n\to\infty}\mathrm{Var}(X_{n,i})=\Sigma>0$, and (2) there
exists $\delta>0$ such that $n^{-\delta/2}E|X_{n,i}|^{2+\delta}=o(1)$, then
$n^{-1/2}\sum^{n}_{i=1}X_{n,i}$ converges in distribution to
$\mathcal{N}(0,\Sigma)$. We can write
$(nh)^{1/2}\mathbb{P}_{n}\left\\{\Gamma_{0,h,b,a_{0}}(\xi_{\infty}-\theta_{0})\right\\}=n^{-1/2}\sum_{i=1}^{n}X_{n,i}$
for

$X_{n,i}:=h^{1/2}\Gamma_{0,h,b,a_{0}}(A_{i})\left\\{\xi_{\infty}(Y_{i},A_{i},W_{i})-\theta_{0}(A_{i})\right\\}.$

Since (A4)(a) holds, $E_{0}\left[\xi_{\infty}(Y,A,W)\mid
A=a\right]=\theta_{0}(a)$ for $F_{0}$-almost every $a\in
B_{\delta_{1}}(a_{0})$ by Lemma 2, and by (A1), $\Gamma_{0,h,b,a_{0}}(a)=0$
for all $a\notin B_{\delta_{1}}(a_{0})$ and $h\leq\delta_{1}$. Hence, for all
$n$ large enough, $E_{0}[X_{n,i}]=0$ for all $i$. We thus have

$\displaystyle\mathrm{Var}\left(X_{n,i}\right)$
$\displaystyle=hE_{0}\left[\Gamma_{0,h,b,a_{0}}(A)^{2}\left\\{\xi_{\infty}(Y,A,W)-\theta_{0}(A)\right\\}^{2}\right]$
$\displaystyle=h\int\Gamma_{0,h,b,a_{0}}(a)^{2}\sigma_{0}^{2}(a)\,dF_{0}(a)$
$\displaystyle=h\int\left\\{e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}w_{h,a_{0},1}(a)K_{h,a_{0}}(a)-e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{0},3}^{-1}w_{b,a_{0},3}(a)K_{b,a_{0}}(a)\right\\}^{2}\sigma_{0}^{2}(a)f_{0}(a)\,da$
$\displaystyle=\int\left\\{e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}(1,u)^{T}K(u)-e_{3}^{T}c_{2}\tau_{n}^{3}\mathbf{D}_{0,b,a_{0},3}^{-1}(1,\\{\tau_{n}u\\},\\{\tau_{n}u\\}^{2},\\{\tau_{n}u\\}^{3})K(\tau_{n}u)\right\\}^{2}\sigma_{0}^{2}(a_{0}+uh)f_{0}(a_{0}+uh)\,du.$

By (A5), $\sigma_{0}$ and $f_{0}$ are continuous at $a_{0}$. Also,
$\mathbf{D}_{0,h,a_{0},1}^{-1}$ and $\mathbf{D}_{0,b,a_{0},3}^{-1}$ converge
to $f_{0}(a_{0})^{-1}\mathbf{S}_{2}^{-1}$ and
$f_{0}(a_{0})^{-1}\mathbf{S}_{4}^{-1}$, respectively, as $b,h\longrightarrow
0$ by Lemma 4, and $\tau_{n}\longrightarrow\tau\in(0,\infty)$. The preceding
display thus converges to

$\displaystyle\sigma_{0}^{2}(a_{0})f_{0}(a_{0})^{-1}\int\left\\{e_{1}^{T}\mathbf{S}_{2}^{-1}(1,u)^{T}K(u)-e_{3}^{T}c_{2}\tau^{3}\mathbf{S}_{4}^{-1}(1,\\{\tau
u\\},\\{\tau u\\}^{2},\\{\tau u\\}^{3})K(\tau u)\right\\}^{2}\,du.$

Now, $e_{1}^{T}\mathbf{S}_{2}^{-1}(1,u)^{T}=1$, and using the block structure
of $\mathbf{S}_{4}$, we find that
$e_{3}^{T}\mathbf{S}_{4}^{-1}=\frac{1}{c_{4}-c_{2}^{2}}\left(-c_{2},0,1,0\right)$.
Therefore, we can simplify the above to
$\sigma_{0}^{2}(a_{0})f_{0}(a_{0})^{-1}V_{K,\tau}$ for

$\displaystyle V_{K,\tau}$
$\displaystyle=\int\left\\{K(u)-\tau^{3}c_{2}\frac{(\tau
u)^{2}-c_{2}}{c_{4}-c_{2}^{2}}K(\tau u)\right\\}^{2}\,du$ $\displaystyle=\int
K(u)^{2}\,du-2\tau^{3}c_{2}\frac{\int(\tau u)^{2}K(u)K(\tau u)\,du-c_{2}\int
K(u)K(\tau u)\,du}{c_{4}-c_{2}^{2}}$
$\displaystyle\qquad+\tau^{6}c_{2}^{2}\frac{\int(\tau u)^{4}K(\tau
u)^{2}\,du-2c_{2}\int(\tau u)^{2}K(\tau u)^{2}\,du+c_{2}^{2}\int K(\tau
u)^{2}\,du}{\left(c_{4}-c_{2}\right)^{2}}$
$\displaystyle=c_{0}^{*}-2\tau^{3}c_{2}\frac{\tau^{2}c_{2,\tau}^{*}-c_{2}c_{0,\tau}^{*}}{c_{4}-c_{2}^{2}}+\tau^{5}c_{2}^{2}\frac{c_{4}^{*}-2c_{2}c_{2}^{*}+c_{2}^{2}c_{0}^{*}}{\left(c_{4}-c_{2}\right)^{2}}.$

Hence, $\mathrm{Var}(X_{n,i})$ converges to
$\sigma_{0}^{2}(a_{0})f_{0}(a_{0})^{-1}V_{K,\tau}$ as claimed. If $\tau=1$,
then the above simplifies to

$V_{K,1}=\int\left\\{\frac{c_{4}-c_{2}u^{2}}{c_{4}-c_{2}^{2}}K(u)\right\\}^{2}\,du=\frac{c_{0}^{*}c_{4}^{2}-2c_{2}c_{4}c_{2}^{*}+c_{2}^{2}c_{4}^{*}}{(c_{4}-c_{2}^{2})^{2}}.$

Clearly, $V_{K,\tau}\geq 0$, with equality if and only if the expression in
the integral is zero identically. When $\tau\neq 1$, the differing supports of
$K(u)$ and $K(\tau u)$ guarantees that this is not the case, and if $\tau=1$
then the expression is zero if and only if $c_{2}=c_{4}=0$, which is not the
case. Hence, $V_{K,\tau}>0$ for any $\tau\in(0,\infty)$. Furthermore, by the
boundedness of $K$ and since $c_{2}^{2}<c_{4}$ by Jensen’s inequality,
$V_{K,\tau}$ is finite for every $\tau\in(0,\infty)$.

For the second condition of the Lyapunov CLT, we first note that

$\displaystyle\sup_{|a-a_{0}|<\delta_{1}}E_{0}\left[\left|\xi_{\infty}(Y,A,W)-\theta_{0}(A)\right|^{2+\delta_{2}}\mid
A=a\right]\leq C.$

for some $C<\infty$ since $\mu_{\infty}$ is uniformly bounded and $g_{\infty}$
is uniformly bounded away from zero by (A4)(a) and
$E_{0}(|Y|^{2+\delta_{2}}\mid A=a,W=w)$ is uniformly bounded over $a\in
B_{\delta_{1}}(a_{0})$ and $w\in\mathcal{W}$ by (A5). Therefore, for all $n$
large enough,

$\displaystyle E|X_{n,i}|^{2+\delta_{2}}$
$\displaystyle=h^{1+\delta_{2}/2}\int\left|\Gamma_{0}(a)\right|^{2+\delta_{2}}E_{0}\left[\left|\xi_{\infty}(y,a,w)-\theta_{0}(a)\right|^{2+\delta_{2}}\mid
A=a\right]f_{0}(a)\,da$ $\displaystyle\leq
Ch^{1+\delta_{2}/2}\int\left|\Gamma_{0}(a)\right|^{2+\delta_{2}}f_{0}(a)\,da.$

By the triangle inequality, we then have

$\displaystyle\left\\{\int\left|\Gamma_{0}(a)\right|^{2+\delta_{2}}f_{0}(a)\,da\right\\}^{1/(2+\delta_{2})}$
$\displaystyle\leq\left\\{\int\left|e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}w_{h,a_{0},1}(a)K_{h,a_{0}}(a)\right|^{2+\delta_{2}}f_{0}(a)\,da\right\\}^{1/(2+\delta_{2})}$
$\displaystyle\qquad+\left\\{\int\left|e_{3}^{T}c_{2}\tau^{2}\mathbf{D}_{0,b,a_{0},3}^{-1}w_{b,a_{0},3}(a)K_{b,a_{0}}(a)\right|^{2+\delta_{2}}f_{0}(a)\,da\right\\}^{1/(2+\delta_{2})}$
$\displaystyle=\left\\{\int\left|e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}(1,u)^{T}K(u)\right|^{2+\delta_{2}}h^{-(1+\delta_{2})}f_{0}(a_{0}+uh)\,du\right\\}^{1/(2+\delta_{2})}$
$\displaystyle\qquad+\left\\{\int\left|e_{3}^{T}c_{2}\tau^{2}\mathbf{D}_{0,b,a_{0},3}^{-1}(1,u,u^{2},u^{3})^{T}K(u)\right|^{2+\delta_{2}}h^{-(1+\delta_{2})}f_{0}(a_{0}+uh)\,du\right\\}^{1/(2+\delta_{2})}.$

By Lemma 4, (A1), (A2), and (A5), the preceding display is bounded up to a
constant by $h^{-(1+\delta_{2})/(2+\delta_{2})}$. Hence, for all $n$ large
enough, $E|X_{n,i}|^{2+\delta_{2}}$ is bounded up to a constant by
$h^{-\delta_{2}/2}$. Therefore,

$n^{-\delta_{2}/2}E|X_{n,i}|^{2+\delta_{2}}=O\left(\left\\{nh\right\\}^{-\delta_{2}/2}\right)=o(1)$

since $nh\longrightarrow\infty$ by assumption.

Next, we show that
$\mathbb{P}_{n}(\Gamma_{0,h,b,a_{0}}\theta_{0}-\gamma_{0,h,b,a_{0}})=o_{p}(\left\\{n/h\right\\}^{-1/2})$,
which implies that it is $o_{p}(\left\\{nh\right\\}^{-1/2})$. We note that
since $P_{0}(\Gamma_{0,h,b,a_{0}}\theta_{0})=P_{0}\gamma_{0,h,b,a_{0}}$,
$E_{0}\left[\mathbb{P}_{n}(\Gamma_{0,h,b,a_{0}}\theta_{0}-\gamma_{0,h,b,a_{0}})\right]=0$
and
$\mathrm{Var}\left[\mathbb{P}_{n}\left(\Gamma_{0,h,b,a_{0}}\theta_{0}-\gamma_{0,h,b,a_{0}}\right)\right]=n^{-1}P_{0}\left(\Gamma_{0,h,b,a_{0}}\theta_{0}-\gamma_{0,h,b,a_{0}}\right)^{2}$.
Thus, it is sufficient to show that
$P_{0}(\Gamma_{0,h,b,a_{0}}\theta_{0}-\gamma_{0,h,b,a_{0}})^{2}=o(h)$. We have

$\displaystyle P_{0}(\Gamma_{0,h,b,a_{0}}\theta_{0}-\gamma_{0,h,b,a_{0}})^{2}$
$\displaystyle\qquad\leq
2\int\left[e_{1}^{T}\mathbf{D}_{0,h,a_{0}}^{-1}w_{h,a_{0},1}(a)K_{h,a_{0}}(a)\left\\{\theta_{0}(a)-w_{h,a_{0},1}(a)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)\right\\}\right]^{2}f_{0}(a)\,da$
$\displaystyle\qquad\qquad+2\int\left[e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{0}}^{-1}w_{b,a_{0},3}(a)K_{b,a_{0}}(a)\left\\{\theta_{0}(a)-w_{b,a_{0},3}(a)^{T}\mathbf{D}^{-1}_{b,a_{0},3}P_{0}\left(w_{b,a_{0},3}K_{b,a_{0}}\theta_{0}\right)\right\\}\right]^{2}f_{0}(a)\,da$
$\displaystyle\qquad=2h^{-1}\int\left[e_{1}^{T}\mathbf{D}_{0,h,a_{0}}^{-1}(1,u)^{T}K(u)\left\\{\theta_{0}(a_{0}+uh)-(1,u)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)\right\\}\right]^{2}f_{0}(a_{0}+uh)\,du$
$\displaystyle\qquad\qquad+2b^{-1}\int\left[e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{0}}^{-1}\mathbf{u}_{3}^{T}K(u)\left\\{\theta_{0}(a_{0}+ub)-\mathbf{u}_{3}^{T}\mathbf{D}^{-1}_{b,a_{0},3}P_{0}\left(w_{b,a_{0},3}K_{b,a_{0}}\theta_{0}\right)\right\\}\right]^{2}f_{0}(a_{0}+ub)\,du,$

where $\mathbf{u}_{3}$ denotes the vector $(1,u,u^{2},u^{3})$. For the first
term, we define
$\beta_{h,a_{0},1}:=(\theta_{0}(a_{0}),h\theta^{(1)}_{0}(a_{0}))^{T}$. We then
write

$\displaystyle(1,u)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)-\theta_{0}(a_{0})-uh\theta_{0}^{(1)}(a_{0})$
$\displaystyle\qquad=(1,u)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)-(1,u)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}\mathbf{D}_{0,h,a_{0}}\beta_{h,a_{0},1}$
$\displaystyle\qquad=(1,u)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}\left[P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)-P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}w_{h,a_{0},1}^{T}\right)\beta_{h,a_{0},1}\right]$
$\displaystyle\qquad=(1,u)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\left[\theta_{0}-w_{h,a_{0},1}^{T}\beta_{h,a_{0},1}\right]\right)$
$\displaystyle\qquad=(1,u)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}\int
w_{h,a_{0},1}(a)K_{h,a_{0}}(a)\left[\theta_{0}(a)-w_{h,a_{0},1}(a)^{T}\beta_{h,a_{0},1}\right]f_{0}(a)\,da$
$\displaystyle\qquad=(1,u)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}\int(1,v)^{T}K(v)\left[\theta_{0}(a_{0}+vh)-(1,v)^{T}\beta_{h,a_{0},1}\right]f_{0}(a_{0}+vh)\,dv$
$\displaystyle\qquad=(1,u)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}\int(1,v)^{T}K(v)\left[\theta_{0}(a_{0}+vh)-\theta_{0}(a_{0})-vh\theta_{0}^{(1)}(a_{0})\right]f_{0}(a_{0}+vh)\,dv$
$\displaystyle\qquad=(1,u)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}o(h),$

where we have also used (A1) and (A5). Therefore,

$\displaystyle
h^{-1}\int\left[e_{1}^{T}\mathbf{D}_{0,h,a_{0}}^{-1}(1,u)^{T}K(u)\left\\{\theta_{0}(a_{0}+uh)-(1,u)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)\right\\}\right]^{2}f_{0}(a_{0}+uh)\,du$
$\displaystyle\qquad=h^{-1}\int\left[e_{1}^{T}\mathbf{D}_{0,h,a_{0}}^{-1}(1,u)^{T}K(u)\left\\{\left[\theta_{0}(a_{0}+uh)-\theta_{0}(a_{0})-uh\theta_{0}^{(1)}(a_{0})\right]\right.\right.$
$\displaystyle\qquad\qquad\qquad\left.\left.-\left[(1,u)^{T}\mathbf{D}^{-1}_{0,h,a_{0}}P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)-\theta_{0}(a_{0})-uh\theta_{0}^{(1)}(a_{0})\right]\right\\}\right]^{2}f_{0}(a_{0}+uh)\,du$
$\displaystyle\qquad=h^{-1}\int\left[e_{1}^{T}\mathbf{D}_{0,h,a_{0}}^{-1}(1,u)^{T}K(u)o(h)\right]^{2}f_{0}(a_{0}+uh)\,du$
$\displaystyle\qquad=o(h),$

using Lemma 4, (A1), (A2), and (A5).

For the second term, we similarly define
$\beta_{b,a_{0},3}:=(\theta_{0}(a_{0}),b\theta^{(1)}_{0}(a_{0}),0,0)^{T}$. By
an identical calculation, we can then show that

$\displaystyle\mathbf{u}_{3}^{T}\mathbf{D}^{-1}_{b,a_{0},3}P_{0}\left(w_{b,a_{0},3}K_{b,a_{0}}\theta_{0}\right)-\theta_{0}(a_{0})-ub\theta_{0}^{(1)}(a_{0})=\mathbf{u}_{3}^{T}\mathbf{D}^{-1}_{b,a_{0},3}o(b)$

and hence

$\displaystyle
b^{-1}\int\left[e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{0}}^{-1}\mathbf{u}_{3}^{T}K(u)\left\\{\theta_{0}(a_{0}+ub)-\mathbf{u}_{3}^{T}\mathbf{D}^{-1}_{b,a_{0},3}P_{0}\left(w_{b,a_{0},3}K_{b,a_{0}}\theta_{0}\right)\right\\}\right]^{2}f_{0}(a_{0}+ub)\,du=o(b).$

Note that we can even show that this term is $o(b^{3})$ using a second-order
Taylor expansion, though this is unnecessary for the purposes of our proof. We
now have

$P_{0}(\Gamma_{0,h,b,a_{0}}\theta_{0}-\gamma_{0,h,b,a_{0}})^{2}=o(h),$

which yields the result.

Finally, we claim that

$\mathbb{P}_{n}\left\\{\int\Gamma_{0}\left(\mu_{\infty}-\int\mu_{\infty}\,dQ_{0}\right)\,dF_{0}\right\\}=O_{p}\left(n^{-1/2}\right),$

which implies in particular that it is $o_{p}\left(\\{nh\\}^{-1/2}\right)$.
Since the function inside the empirical mean has mean zero, it is sufficient
to show that

$\int\left[\int\Gamma_{0}\left(\mu_{\infty}-\int\mu_{\infty}\,dQ_{0}\right)\,dF_{0}\right]^{2}\,dQ_{0}=O(1).$

Since $\mu_{\infty}$ is uniformly bounded by (A4)(a), we have

$\displaystyle\int\left[\int\Gamma_{0}\left(\mu_{\infty}-\int\mu_{\infty}\,dQ_{0}\right)\,dF_{0}\right]^{2}\,dQ_{0}$
$\displaystyle\leq\int\left[\int\Gamma_{0}\mu_{\infty}\,dF_{0}\right]^{2}\,dQ_{0}$
$\displaystyle\leq\int\left[\int\left|e_{1}^{T}\mathbf{D}_{0,h,a_{0}}^{-1}w_{h,a_{0},1}(a)K_{h,a_{0}}(a)\mu_{\infty}(a,w)\right|\,dF_{0}(a)\right.$
$\displaystyle\qquad\left.+\int\left|e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{0}}^{-1}w_{b,a_{0},3}(a)K_{b,a_{0}}(a)\mu_{\infty}(a,w)\right|\,dF_{0}(a)\right]^{2}\,dQ_{0}(w)$
$\displaystyle\lesssim\left[\int\left|e_{1}^{T}\mathbf{D}_{0,h,a_{0}}^{-1}(1,u)^{T}K(u)f_{0}(a_{0}+uh)\right|\,du\right.$
$\displaystyle\qquad\left.+\int\left|e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{0}}^{-1}(1,u,u^{2},u^{3})^{T}K(u)f_{0}(a_{0}+uh)\right|\,du\right]^{2}.$

This latter expression is $O(1)$ by Lemma 4, (A1), (A2), and (A5).

For the final statement of the result, using the same decomposition of
$\phi^{*}_{\infty,h,b,a_{0}}$ as above and the fact that
$E_{0}[\xi_{\infty}\mid A=a]=\theta_{0}(a)$ for $a\in B_{\delta_{1}}(a_{0})$,
we can write

$\displaystyle\sigma_{\infty,h,b}^{2}(a_{0})$
$\displaystyle=hP_{0}{\phi^{*2}_{\infty,h,b,a_{0}}}$
$\displaystyle=hP_{0}\left[\Gamma_{0,h,b,a_{0}}^{2}\left(\xi_{\infty}-\theta_{0}\right)^{2}\right]+hP_{0}\left(\Gamma_{0,h,b,a_{0}}\theta_{0}-\gamma_{0,h,b,a_{0}}\right)^{2}+hP_{0}\left[\int\Gamma_{0,h,b,a_{0}}(\mu_{\infty}-\smallint\mu_{\infty}\,dQ_{0})\,dF_{0}\right]^{2}$
$\displaystyle\qquad+2hP_{0}\left[\Gamma_{0,h,b,a_{0}}\left(\xi_{\infty}-\theta_{0}\right)\int\Gamma_{0,h,b,a_{0}}(\mu_{\infty}-\smallint\mu_{\infty}\,dQ_{0})\,dF_{0}\right]$
$\displaystyle\qquad+2hP_{0}\left[\left(\Gamma_{0,h,b,a_{0}}\theta_{0}-\gamma_{0,h,b,a_{0}}\right)\int\Gamma_{0,h,b,a_{0}}(\mu_{\infty}-\smallint\mu_{\infty}\,dQ_{0})\,dF_{0}\right].$

We have showed that
$hP_{0}\left[\Gamma_{0,h,b,a_{0}}^{2}\left(\xi_{\infty}-\theta_{0}\right)^{2}\right]$
converges to $V_{K,\tau}\sigma_{0}^{2}(a_{0})/f_{0}(a_{0})$ as $h\to 0$. We
have also showed that
$hP_{0}\left(\Gamma_{0,h,b,a_{0}}\theta_{0}-\gamma_{0,h,b,a_{0}}\right)^{2}=o(h^{2})$
and
$hP_{0}\left[\int\Gamma_{0,h,b,a_{0}}(\mu_{\infty}-\smallint\mu_{\infty}\,dQ_{0})\,dF_{0}\right]^{2}=O(h)$,
so the statement follows.

∎

######  Lemma 6.

If the assumptions of Lemma 5 hold for each $a_{0}$ in
$\\{a_{1},\dotsc,a_{m}\\}$, then

$(nh)^{1/2}\mathbb{P}_{n}\begin{pmatrix}\phi_{\infty,h,b,a_{1}}^{*}\\\
\vdots\\\ \phi_{\infty,h,b,a_{m}}^{*}\end{pmatrix}$

converges in distribution to a mean-zero multivariate normal distribution with
zero off-diagonal elements and diagonal elements
$V_{K,\tau}f_{0}(a_{1})^{-1}\sigma_{0}^{2}(a_{1}),\dotsc,V_{K,\tau}f_{0}(a_{m})^{-1}\sigma_{0}^{2}(a_{m})$.
Furthermore, $hP_{0}(\phi^{*}_{\infty,h,b,a_{j}}\phi^{*}_{\infty,h,b,a_{k}})$
converges to zero for $j\neq k$.

###### Proof of Lemma 6.

We focus on the case of $m=2$ for simplicity; the proof for $m>2$ is entirely
analogous. We will show that for any $t_{1},t_{2}\in\mathbb{R}$,

$t_{1}(nh)^{1/2}\mathbb{P}_{n}\phi^{*}_{\infty,h,b,a_{1}}+t_{2}(nh)^{1/2}\mathbb{P}_{n}\phi^{*}_{\infty,h,b,a_{2}}\operatorname*{\stackrel{{\scriptstyle
d}}{{\longrightarrow}}}t_{1}^{2}V_{K,\tau}f_{0}(a_{1})^{-1}\sigma_{0}^{2}(a_{1})Z_{1}+t_{2}^{2}V_{K,\tau}f_{0}(a_{2})^{-1}\sigma_{0}^{2}(a_{2})Z_{2}$

where $Z_{1}$ and $Z_{2}$ are independent standard normal random variables. By
the Cramer-Wold device, this yields the result. First, by the derivations in
the proof of Lemma 5,

$\displaystyle t_{1}\mathbb{P}_{n}\phi^{*}_{\infty,h,b,a_{1}}$
$\displaystyle=\mathbb{P}_{n}\left\\{t_{1}\Gamma_{0,h,b,a_{1}}(\xi_{\infty}-\theta_{0})\right\\}+o_{p}(\\{nh\\}^{-1/2})\qquad\text{and}$
$\displaystyle t_{2}\mathbb{P}_{n}\phi^{*}_{\infty,h,b,a_{2}}$
$\displaystyle=\mathbb{P}_{n}\left\\{t_{2}\Gamma_{0,h,b,a_{2}}(\xi_{\infty}-\theta_{0})\right\\}+o_{p}(\\{nh\\}^{-1/2}).$

Hence,

$t_{1}(nh)^{1/2}\mathbb{P}_{n}\phi^{*}_{\infty,h,b,a_{1}}+t_{2}(nh)^{1/2}\mathbb{P}_{n}\phi^{*}_{\infty,h,b,a_{2}}=(nh)^{1/2}\mathbb{P}_{n}\left\\{(t_{1}\Gamma_{0,h,b,a_{1}}+t_{2}\Gamma_{0,h,b,a_{2}})(\xi_{\infty}-\theta_{0})\right\\}+o_{p}(\\{nh\\}^{-1/2}).$

As in the proof of Lemma 5, we demonstrate the convergence of the expression
on the right using the Lyapunov CLT for triangular arrays. We define
$X_{n,1},X_{n,2},\dotsc,X_{n,n}$ as

$X_{n,i}:=h^{1/2}\left\\{t_{1}\Gamma_{0,h,b,a_{1}}(A_{i})+t_{2}\Gamma_{0,h,b,a_{2}}(A_{i})\right\\}\left\\{\xi_{\infty}(Y_{i},A_{i},W_{i})-\theta_{0}(A_{i})\right\\}.$

We can then write
$(nh)^{1/2}\mathbb{P}_{n}\left\\{(t_{1}\Gamma_{0,h,b,a_{1}}+t_{2}\Gamma_{0,h,b,a_{2}})(\xi_{\infty}-\theta_{0})\right\\}$
as $n^{-1/2}\sum_{i=1}^{n}\tilde{X}_{n,i}$. Hence, the claim follows by
showing that $n^{-1/2}\sum_{i=1}^{n}X_{n,i}$ converges in distribution to the
claimed limit, which we do using the Lyapunov CLT.

The main condition of the Lyapunov CLT that differs from Lemma 5 is
convergence of the variance. By linearity of expectation and since
$P_{0}\phi^{*}_{\infty,h,b,a_{1}}=P_{0}\phi^{*}_{\infty,h,b,a_{2}}=0$, we have
$E_{0}[X_{n,i}]=0$ for all $i$. We also have

$\displaystyle\mathrm{Var}\left(X_{n,i}\right)$
$\displaystyle=hE_{0}\left[\left\\{t_{1}\Gamma_{0,h,b,a_{1}}(A)+t_{2}\Gamma_{0,h,b,a_{2}}(A)\right\\}^{2}\left\\{\xi_{\infty}(Y,A,W)-\theta_{0}(A)\right\\}^{2}\right]$
$\displaystyle=ht_{1}^{2}E_{0}\left[\Gamma_{0,h,b,a_{1}}(A)^{2}\left\\{\xi_{\infty}(Y,A,W)-\theta_{0}(A)\right\\}^{2}\right]$
$\displaystyle\qquad+ht_{2}^{2}E_{0}\left[\Gamma_{0,h,b,a_{2}}(A)^{2}\left\\{\xi_{\infty}(Y,A,W)-\theta_{0}(A)\right\\}^{2}\right]$
$\displaystyle\qquad+2ht_{1}t_{2}E_{0}\left[\Gamma_{0,h,b,a_{1}}(A)\Gamma_{0,h,b,a_{2}}(A)\left\\{\xi_{\infty}(Y,A,W)-\theta_{0}(A)\right\\}^{2}\right].$

The first two terms converge to
$t_{1}^{2}V_{K,\tau}f_{0}(a_{1})^{-1}\sigma_{0}^{2}(a_{1})$ and
$t_{2}^{2}V_{K,\tau}f_{0}(a_{2})^{-1}\sigma_{0}^{2}(a_{2})$ by Lemma 5. We can
write out the third term as

$\displaystyle
E_{0}\left[\Gamma_{0,h,b,a_{1}}(A)\Gamma_{0,h,b,a_{2}}(A)\left\\{\xi_{\infty}(Y,A,W)-\theta_{0}(A)\right\\}^{2}\right]$
$\displaystyle\qquad=\int\left\\{e_{1}^{T}\mathbf{D}_{0,h,a_{1},1}^{-1}w_{h,a_{1},1}(a)K_{h,a_{1}}(a)e_{1}^{T}\mathbf{D}_{0,h,a_{2},1}^{-1}w_{h,a_{2},1}(a)K_{h,a_{2}}(a)\right.$
$\displaystyle\qquad\qquad\left.-e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{1},3}^{-1}w_{b,a_{1},3}(a)K_{b,a_{1}}(a)e_{1}^{T}\mathbf{D}_{0,h,a_{2},1}^{-1}w_{h,a_{2},1}(a)K_{h,a_{2}}(a)\right.$
$\displaystyle\qquad\qquad\left.-e_{1}^{T}\mathbf{D}_{0,h,a_{1},1}^{-1}w_{h,a_{1},1}(a)K_{h,a_{1}}(a)e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{2},3}^{-1}w_{b,a_{2},3}(a)K_{b,a_{2}}(a)\right.$
$\displaystyle\qquad\qquad\left.+e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{1},3}^{-1}w_{b,a_{1},3}(a)K_{b,a_{1}}(a)e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{2},3}^{-1}w_{b,a_{2},3}(a)K_{b,a_{2}}(a)\right\\}\sigma_{0}^{2}(a)f_{0}(a)\,da.$

The four summands in the above display involve the products
$K_{h,a_{1}}K_{h,a_{2}}$, $K_{h,a_{1}}K_{b,a_{2}}$, $K_{b,a_{1}}K_{h,a_{2}}$,
and $K_{b,a_{1}}K_{b,a_{2}}$. Since the support of $K$ is contained in
$[-1,1]$, each of these products is zero for all $h,b$ small enough.
Specifically, if $\max\\{h,b\\}<|a_{1}-a_{2}|/2$, then
$\\{a:K_{h,a_{1}}(a)>0\\}\cap\\{a:K_{h,a_{2}}(a)>0\\}=\\{a:|a-a_{1}|\leq
h\\}\cap\\{a:|a-a_{2}|\leq h\\}=\emptyset$. Therefore, the variance converges
to
$t_{1}^{2}V_{K,\tau}f_{0}(a_{1})^{-1}\sigma_{0}^{2}(a_{1})+t_{2}^{2}V_{K,\tau}f_{0}(a_{2})^{-1}\sigma_{0}^{2}(a_{2})$.
Each term is bounded away from zero by Lemma 5. The remainder of the
conditions of the Lyapunov CLT can be checked using the same derivations as in
Lemma 5 and using the triangle inequality. ∎

##  Supplement G Analysis of remainder term $R_{n,h,b,a_{0},1}$

######  Lemma 7.

If (A1) and (A5) hold, then
$\theta_{0,h,b}(a_{0})-\theta_{0}(a_{0})=o(h^{2})$. If (A1) and (A8) hold,
then for some $\delta_{4}>0$,
$sup_{a_{0}\in\mathcal{A}_{0}}\left|\theta_{0,h,b}(a_{0})-\theta_{0}(a_{0})\right|=O(h^{2+\delta_{4}})$.

###### Proof of Lemma 7.

As in the proof of Lemma 1, we write
$\theta_{0,h,b}(a_{0})=\theta_{0,h}(a_{0})-\tfrac{1}{2}c_{2}h^{2}\theta_{0,b}^{(2)}(a_{0})$
for

$\displaystyle\theta_{0,h}(a_{0})$
$\displaystyle:=e_{1}^{T}\mathbf{D}^{-1}_{0,h,a_{0},1}P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)\text{
and
}\theta_{0,b}^{(2)}(a_{0}):=2b^{-2}e_{3}^{T}\mathbf{D}^{-1}_{0,b,a_{0},3}P_{0}\left(w_{b,a_{0},3}K_{b,a_{0}}\theta_{0}\right).$

Since $e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}\mathbf{D}_{0,h,a_{0},1}e_{1}=1$
and $w_{h,a_{0},1}^{T}e_{1}=1$, we can write

$\displaystyle\theta_{0}(a_{0})$
$\displaystyle=e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}\mathbf{D}_{0,h,a_{0},1}e_{1}\theta_{0}(a_{0})=e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}P_{0}\left\\{w_{h,a_{0},1}K_{h,a_{0}}w_{h,a_{0},1}^{T}e_{1}\theta_{0}(a_{0})\right\\}$
$\displaystyle=e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}P_{0}\left\\{w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}(a_{0})\right\\}.$

We then have

$\displaystyle\theta_{0,h}(a_{0})-\theta_{0}(a_{0})$
$\displaystyle=e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}P_{0}(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0})-e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}P_{0}\left\\{w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}(a_{0})\right\\}$
$\displaystyle=e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}P_{0}\left\\{w_{h,a_{0},1}K_{h,a_{0}}\left[\theta_{0}-\theta_{0}(a_{0})\right]\right\\}.$

By (A5), the second derivative of $a\mapsto\theta_{0}(a)$ is continuous in a
neighborhood of $a=a_{0}$, so by the mean value form of Taylor’s theorem, for
each $a$ in a neighborhood of $a_{0}$, there exists $a_{*}$ (depending on $a$)
between $a_{0}$ and $a$ such that

$\theta_{0}(a)-\theta_{0}(a_{0})=\theta_{0}^{(1)}(a_{0})(a-a_{0})+\tfrac{1}{2}\theta_{0}^{(2)}(a_{*})(a-a_{0})^{2}.$

We then have

$\displaystyle e_{1}^{T}$
$\displaystyle\mathbf{D}_{0,h,a_{0},1}^{-1}P_{0}\left\\{w_{h,a_{0},1}K_{h,a_{0}}\left[\theta_{0}-\theta_{0}(a_{0})\right]\right\\}$
$\displaystyle=e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}E_{0}\left[w_{h,a_{0},1}(A)K_{h,a_{0}}(A)\\{\theta_{0}(A)-\theta_{0}(a_{0})\\}\right]$
$\displaystyle=e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}E_{0}\left[w_{h,a_{0},1}(A)K_{h,a_{0}}(A)\left\\{\theta_{0}^{(1)}(a_{0})(A-a_{0})+\tfrac{1}{2}\theta_{0}^{(2)}(A_{*})(A-a_{0})^{2}\right\\}\right]$
$\displaystyle=e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}E_{0}\left[w_{h,a_{0},1}(A)K_{h,a_{0}}(A)(A-a_{0})\right]\theta_{0}^{(1)}(a_{0})+\tfrac{1}{2}e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}E_{0}\left[w_{h,a_{0},1}(A)K_{h,a_{0}}(A)\theta_{0}^{(2)}(A_{*})(A-a_{0})^{2}\right].$

We then note that we can write $A-a_{0}=hw_{h,a_{0},1}^{T}(A)e_{2}$, so that

$\displaystyle
e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}E_{0}\left[w_{h,a_{0},1}(A)K_{h,a_{0}}(A)(A-a_{0})\right]$
$\displaystyle=he_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}E_{0}\left[w_{h,a_{0},1}(A)K_{h,a_{0}}(A)w_{h,a_{0},1}^{T}(A)\right]e_{2}$
$\displaystyle=he_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}\mathbf{D}_{0,h,a_{0},1}e_{2}=he_{1}^{T}e_{2}=0.$

Thus,

$e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}P_{0}\left\\{w_{h,a_{0},1}K_{h,a_{0}}\left[\theta_{0}-\theta_{0}(a_{0})\right]\right\\}=\tfrac{1}{2}e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}E_{0}\left[w_{h,a_{0},1}(A)K_{h,a_{0}}(A)\theta_{0}^{(2)}(A_{*})(A-a_{0})^{2}\right].$

Now we define $u_{*}=(a_{*}-a_{0})/h$, and we use the change of variables
$u=(a-a_{0})/h$ and continuity of $\theta_{0}^{(2)}$ in a neighborhood of
$a_{0}$ to see that

$\displaystyle
E_{0}\left[w_{h,a_{0},1}(A)K_{h,a_{0}}(A)\theta_{0}^{(2)}(A_{*})(A-a_{0})^{2}\right]$
$\displaystyle=\int
w_{h,a_{0},1}(a)K_{h,a_{0}}(a)\theta_{0}^{(2)}(a_{*})(a-a_{0})^{2}f_{0}(a)\,da$
$\displaystyle=\int(1,u)^{T}K(u)\theta_{0}^{(2)}(a_{0}+u_{*}h)(uh)^{2}f_{0}(a_{0}+uh)\,du.$

By adding and subtracting terms and noting that
$e_{1}^{T}\mathbf{S}_{2}^{-1}(c_{2},0)^{T}=1$, we can then write

$\displaystyle\theta_{0,h}(a_{0})-\theta_{0}(a_{0})$
$\displaystyle=\tfrac{1}{2}h^{2}e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}\int(u^{2},u^{3})^{T}K(u)\theta_{0}^{(2)}(a_{0}+u_{*}h)f_{0}(a_{0}+uh)\,du$
$\displaystyle=\tfrac{1}{2}h^{2}c_{2}\theta_{0}^{(2)}(a_{0})+\tfrac{1}{2}h^{2}e_{1}^{T}\left[\mathbf{D}_{0,h,a_{0},1}^{-1}-\mathbf{S}_{2}^{-1}f_{0}(a_{0})^{-1}\right](c_{2},0)^{T}f_{0}(a_{0})\theta_{0}^{(2)}(a_{0})$
$\displaystyle\qquad+\tfrac{1}{2}h^{2}e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}\int(u^{2},u^{3})^{T}K(u)\left[\theta_{0}^{(2)}(a_{0}+u_{*}h)-\theta_{0}^{(2)}(a_{0})\right]f_{0}(a_{0})\,du$
$\displaystyle\qquad+\tfrac{1}{2}h^{2}e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}\int(u^{2},u^{3})^{T}K(u)\theta_{0}^{(2)}(a_{0}+u_{*}h)\left[f_{0}(a_{0}+uh)-f_{0}(a_{0})\right]\,du.$

By Lemma 4,
$\mathbf{D}_{0,h,a_{0},1}^{-1}-\mathbf{S}_{2}^{-1}f_{0}(a_{0})^{-1}=O(h)$. The
expression on the second line is $o(h^{2})$ by continuity of
$\theta_{0}^{(2)}$ at $a_{0}$. The final expression is $o(h^{2})$ by
continuity of $f_{0}$ at $a_{0}$. Hence,
$\theta_{0,h}(a_{0})-\theta_{0}(a_{0})=\tfrac{1}{2}h^{2}c_{2}\theta_{0}^{(2)}(a_{0})+o(h^{2})$.

Next, we study $\theta_{0,b}^{(2)}(a_{0})$. We first note that for
$j\in\\{0,1\\}$, since $(A-a_{0})^{j}=b^{j}w_{b,a_{0},3}^{T}(A)e_{j+1}$,

$\displaystyle
e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}E_{0}\left\\{w_{b,a_{0},3}(A)K_{b,a_{0}}(A)(A-a_{0})^{j}\right\\}$
$\displaystyle=b^{j}e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}E_{0}\left\\{w_{b,a_{0},3}(A)K_{b,a_{0}}(A)w_{b,a_{0},3}^{T}(A)\right\\}e_{j+1}$
$\displaystyle=b^{j}e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}\mathbf{D}_{0,b,a_{0},3}e_{j+1}=b^{j}e_{3}^{T}e_{j+1}=0.$

Similarly, since $(A-a_{0})^{2}=b^{2}w_{b,a_{0},3}^{T}(A)e_{3}$,
$e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}E_{0}\left\\{w_{b,a_{0},3}(A)K_{b,a_{0}}(A)(A-a_{0})^{2}\right\\}=b^{2}e_{3}^{T}e_{3}=b^{2}$.
Using these results in conjunction with the second-order Taylor expansion and
continuity of $\theta_{0}^{(2)}$, we have

$\displaystyle e_{3}^{T}$
$\displaystyle\mathbf{D}_{0,b,a_{0},3}^{-1}P_{0}(w_{b,a_{0},3}K_{b,a_{0}}\theta_{0})$
$\displaystyle=e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}\int
w_{b,a_{0},3}(a)K_{b,a_{0}}(a)\theta_{0}(a)f_{0}(a)\,da$
$\displaystyle=e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}\int
w_{b,a_{0},3}(a)K_{b,a_{0}}(a)\left\\{\theta_{0}(a_{0})+\theta_{0}^{(1)}(a_{0})(a-a_{0})+\tfrac{1}{2}\theta_{0}^{(2)}(a_{*})(a-a_{0})^{2}\right\\}f_{0}(a)\,da$
$\displaystyle=\tfrac{1}{2}e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}\int
w_{b,a_{0},3}(a)K_{b,a_{0}}(a)\theta^{(2)}_{0}(a_{*})\left(a-a_{0}\right)^{2}f_{0}(a)\,da$
$\displaystyle=\tfrac{1}{2}b^{2}e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}\int(u^{2},u^{3},u^{4},u^{5})^{T}K(u)\theta^{(2)}_{0}(a_{0}+u_{*}b)f_{0}(a_{0}+ub)\,du.$

By adding and subtracting terms and noting that
$e_{3}^{T}\mathbf{S}_{4}^{-1}(c_{2},0,c_{4},0)^{T}=1$, we then have

$\displaystyle\theta_{0,b}^{(2)}(a_{0})$
$\displaystyle=e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}\int(u^{2},u^{3},u^{4},u^{5})^{T}K(u)\theta^{(2)}_{0}(a_{0}+u_{*}b)f_{0}(a_{0}+ub)\,du$
$\displaystyle=\theta_{0}^{(2)}(a_{0})+\theta_{0}^{(2)}(a_{0})e_{3}^{T}\left[\mathbf{D}_{0,b,a_{0},3}^{-1}-\mathbf{S}_{4}^{-1}f_{0}(a_{0})^{-1}\right](c_{2},0,c_{4},0)^{T}f_{0}(a_{0})$
$\displaystyle\qquad+e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}\int(u^{2},u^{3},u^{4},u^{5})^{T}K(u)\left[\theta^{(2)}_{0}(a_{0}+u_{*}b)-\theta_{0}^{(2)}(a_{0})\right]f_{0}(a_{0})\,du$
$\displaystyle\qquad+e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}\int(u^{2},u^{3},u^{4},u^{5})^{T}K(u)\theta^{(2)}_{0}(a_{0}+u_{*}b)\left[f_{0}(a_{0}+ub)-f_{0}(a_{0})\right]\,du.$

As above, Lemma 4, continuity of $\theta_{0}^{(2)}$, and continuity of $f_{0}$
then imply that $\theta^{(2)}_{0,b}(a_{0})=\theta_{0}^{(2)}(a_{0})+o(1)$ as
$b\longrightarrow 0$.

Putting it together, we conclude that

$\displaystyle\theta_{0,h,b}(a_{0})-\theta_{0}(a_{0})$
$\displaystyle=\left[\theta_{0,h}(a_{0})-\theta_{0}(a_{0})\right]-\tfrac{1}{2}c_{2}h^{2}\theta^{(2)}_{0,b}(a_{0})$
$\displaystyle=\tfrac{1}{2}h^{2}c_{2}\theta_{0}^{(2)}(a_{0})+o(h^{2})-\tfrac{1}{2}c_{2}h^{2}\theta_{0}^{(2)}(a_{0})+o(h^{2})$
$\displaystyle=o(h^{2}).$

For the uniform statement, using the above derivations, for each $a_{0}$ we
have

$\displaystyle\theta_{0,h,b}(a_{0})-\theta_{0}(a_{0})$
$\displaystyle=\tfrac{1}{2}h^{2}e_{1}^{T}\left[\mathbf{D}_{0,h,a_{0},1}^{-1}-\mathbf{S}_{2}^{-1}f_{0}(a_{0})^{-1}\right](c_{2},0)^{T}f_{0}(a_{0})\theta_{0}^{(2)}(a_{0})$
$\displaystyle\qquad+\tfrac{1}{2}h^{2}e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}\int(u^{2},u^{3})^{T}K(u)\left[\theta_{0}^{(2)}(a_{0}+u_{*}h)-\theta_{0}^{(2)}(a_{0})\right]f_{0}(a_{0})\,du$
$\displaystyle\qquad+\tfrac{1}{2}h^{2}e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}\int(u^{2},u^{3})^{T}K(u)\theta_{0}^{(2)}(a_{0}+u_{*}h)\left[f_{0}(a_{0}+uh)-f_{0}(a_{0})\right]\,du$
$\displaystyle\qquad-\theta_{0}^{(2)}(a_{0})e_{3}^{T}\left[\mathbf{D}_{0,b,a_{0},3}^{-1}-\mathbf{S}_{4}^{-1}f_{0}(a_{0})^{-1}\right](c_{2},0,c_{4},0)^{T}f_{0}(s)$
$\displaystyle\qquad-
e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}\int(u^{2},u^{3},u^{4},u^{5})^{T}K(u)\left[\theta^{(2)}_{0}(a_{0}+u_{*}b)-\theta_{0}^{(2)}(a_{0})\right]f_{0}(a_{0})\,du$
$\displaystyle\qquad-
e_{3}^{T}\mathbf{D}_{0,b,a_{0},3}^{-1}\int(u^{2},u^{3},u^{4},u^{5})^{T}K(u)\theta^{(2)}_{0}(a_{0}+u_{*}b)\left[f_{0}(a_{0}+ub)-f_{0}(a_{0})\right]\,du.$

By Lemma 4, the first and fourth lines on the right hand side are both
$O(h^{3})$ uniformly over $a_{0}\in\mathcal{A}_{0}$. By the smoothness of
$\theta_{0}^{(2)}$ imposed by (A8), the second and fifth lines are both
$O(h^{2+\delta_{4}})$ uniformly over $a_{0}$, where $\delta_{4}$ is the Hölder
exponent of $\theta_{0}^{(2)}$. By the smoothness of $f_{0}$ imposed by (A8)
the third and sixth lines are $O(h^{3})$. The result follows. ∎

##  Supplement H Analysis of remainder terms $R_{n,h,b,a_{0},2}$ and
$R_{n,h,b,a_{0},3}$

Before presenting results regarding the remainder terms $R_{n,h,b,a_{0},2}$
and $R_{n,h,b,a_{0},3}$, we state a Lemma regarding the rate of convergence of
$\mathbf{D}_{n,h,a_{0},1}-\mathbf{D}_{0,h,a_{0},1}$.

######  Lemma 8.

Denote by $\|\cdot\|_{\infty}$ the element-wise maximum norm. If (A1) and (A5)
hold and $nh\longrightarrow\infty$, then
$\|\mathbf{D}_{n,h,a_{0},1}-\mathbf{D}_{0,h,a_{0},1}\|_{\infty}$ and
$\|\mathbf{D}^{-1}_{n,h,a_{0},1}-\mathbf{D}^{-1}_{0,h,a_{0},1}\|_{\infty}$ are
$O_{p}(\\{nh\\}^{-1/2})$, and
$\|\mathbf{D}_{n,b,a_{0},3}-\mathbf{D}_{0,b,a_{0},3}\|_{\infty}$ and
$\|\mathbf{D}^{-1}_{n,b,a_{0},3}-\mathbf{D}^{-1}_{0,b,a_{0},3}\|_{\infty}$ are
$O_{p}(\\{nb\\}^{-1/2})$.

###### Proof of Lemma 8.

We first note that for all $(j,k)$,
$E_{0}\left(\mathbf{D}_{n,h,a_{0},1}[j,k]\right)=\mathbf{D}_{0,h,a_{0},1}[j,k]$.
Using the change of variables $u=(a-a_{0})/h$, we have

$\displaystyle\mathrm{Var}\left(\mathbf{D}_{n,h,a_{0},1}[j,k]\right)$
$\displaystyle=\mathrm{Var}\left\\{n^{-1}\sum_{i=1}^{n}\left(\frac{A_{i}-a_{0}}{h}\right)^{j+k-2}h^{-1}K\left(\frac{A_{i}-a_{0}}{h}\right)\right\\}$
$\displaystyle=(nh)^{-2}\sum_{i=1}^{n}\mathrm{Var}\left\\{\left(\frac{A_{i}-a_{0}}{h}\right)^{j+k-2}K\left(\frac{A_{i}-a_{0}}{h}\right)\right\\}$
$\displaystyle\leq(nh)^{-2}\sum_{i=1}^{n}E_{0}\left[\left\\{\left(\frac{A_{i}-a_{0}}{h}\right)^{j+k-2}K\left(\frac{A_{i}-a_{0}}{h}\right)\right\\}^{2}\right]$
$\displaystyle=n^{-1}h^{-2}\int\left\\{\left(\frac{a-a_{0}}{h}\right)^{j+k-2}K\left(\frac{a-a_{0}}{h}\right)\right\\}^{2}\,dF_{0}(a)$
$\displaystyle=(nh)^{-1}\int
u^{2(j+k-2)}K^{2}\left(u\right)f_{0}(a_{0}+uh)\,du.$

Since $f_{0}(a_{0}+uh)=f_{0}(a_{0})+O(uh)$ by (A5) and $K$ is uniformly
bounded with compact support by (A1), we have

$\displaystyle(nh)^{-1}\int
u^{2(j+k-2)}K^{2}\left(u\right)f_{0}(a_{0}+uh)\,du$
$\displaystyle=(nh)^{-1}f_{0}(a_{0})\left\\{\int
u^{2(j+k-2)}K^{2}\left(u\right)\,du+O(h)\right\\}$
$\displaystyle=(nh)^{-1}f_{0}(a_{0})c_{2(i+j-2),2}+o(\\{nh\\}^{-1})$

for $1\leq j,k,\leq 2$. By Chebyshev’s inequality, we then have
$\mathbf{D}_{n,h,a_{0},1}[j,k]-\mathbf{D}_{0,h,a_{0},1}[j,k]=O_{p}(\\{nh\\}^{-1/2})$.
Since this holds for all $1\leq j,k,\leq 2$, we conclude that
$\|\mathbf{D}_{n,h,a_{0},1}-\mathbf{D}_{0,h,a_{0},1}\|_{\infty}=O_{p}(\\{nh\\}^{-1/2})$.
The result for
$\|\mathbf{D}_{n,b,a_{0},3}-\mathbf{D}_{0,b,a_{0},3}\|_{\infty}=O_{p}(\\{nb\\}^{-1/2})$
can be shown analogously.

To show that
$\|\mathbf{D}_{n,h,a_{0},1}^{-1}-\mathbf{D}_{0,h,a_{0},1}^{-1}\|_{\infty}=O_{p}(\\{nh\\}^{-1})$,
we first write
$\mathbf{D}_{n,h,a_{0},1}^{-1}-\mathbf{D}_{0,h,a_{0},1}^{-1}=\mathbf{D}^{-1}_{n,h,a_{0},1}(\mathbf{D}_{0,h,a_{0},1}-\mathbf{D}_{n,h,a_{0},1})\mathbf{D}^{-1}_{0,h,a_{0},1}$.
Denoting by $\|\cdot\|_{1}$ the $L_{1}$ matrix operator norm, it follows from
the fact that the $L_{1}$ norm is sub-multiplicative,

$\displaystyle\|\mathbf{D}_{n,h,a_{0},1}^{-1}-\mathbf{D}_{0,h,a_{0},1}^{-1}\|_{\infty}$
$\displaystyle=\|\mathbf{D}^{-1}_{n,h,a_{0},1}(\mathbf{D}_{0,h,a_{0},1}-\mathbf{D}_{n,h,a_{0},1})\mathbf{D}^{-1}_{0,h,a_{0},1}\|_{\infty}$
$\displaystyle\leq
C\|\mathbf{D}^{-1}_{n,h,a_{0},1}(\mathbf{D}_{0,h,a_{0},1}-\mathbf{D}_{n,h,a_{0},1})\mathbf{D}^{-1}_{0,h,a_{0},1}\|_{1}$
$\displaystyle\leq
C\|\mathbf{D}^{-1}_{n,h,a_{0},1}\|_{1}\|\mathbf{D}_{0,h,a_{0},1}-\mathbf{D}_{n,h,a_{0},1}\|_{1}\|\mathbf{D}^{-1}_{0,h,a_{0},1}\|_{1}.$

We then note that $f_{0}(a_{0})>0$ implies that
$\|\mathbf{D}^{-1}_{0,h,a_{0},1}\|_{1}=O(1)$ by Lemma 4. Since
$nh\longrightarrow\infty$, Lemma 4 also implies that
$\mathbf{D}_{n,h,a_{0},1}^{-1}\operatorname*{\stackrel{{\scriptstyle
P}}{{\longrightarrow}}}f_{0}(a_{0})^{-1}\mathbf{S}_{2}^{-1}$, so by the
continuous mapping theorem,
$\|\mathbf{D}^{-1}_{n,h,a_{0},1}\|_{1}\operatorname*{\stackrel{{\scriptstyle
P}}{{\longrightarrow}}}\|f_{0}(a_{0})^{-1}\mathbf{S}_{2}^{-1}\|_{1}$ and hence
$\|\mathbf{D}^{-1}_{n,h,a_{0},1}\|_{1}=O_{p}(1)$. We conclude that
$\|\mathbf{D}^{-1}_{n,h,a_{0},1}-\mathbf{D}^{-1}_{0,h,a_{0},1}\|_{\infty}=O_{p}(\\{nh\\}^{-1/2})$.
The derivation above holds for finite dimensional matrices, and thus a similar
argument yields that
$\|\mathbf{D}^{-1}_{n,b,a_{0},3}-\mathbf{D}^{-1}_{0,b,a_{0},3}\|_{\infty}=O_{p}(\\{nb\\}^{-1/2})$.

∎

We establish rates of convergence of $R_{n,h,b,a_{0},2}$ and
$R_{n,h,b,a_{0},3}$ using empirical process theory. Before presenting
supporting lemmas, we review key definitions and notation used in empirical
process theory. For a class of functions $\mathcal{H}$ defined on a domain
$\mathcal{X}$, a probability measure $Q$ on $\mathcal{X}$, and any
$\varepsilon>0$, the $\varepsilon$-covering number
$N(\varepsilon,\mathcal{H},L_{2}(Q))$ of $\mathcal{H}$ relative to the
$L_{2}(Q)$ metric is the smallest number of $L_{2}(Q)$-balls of radius less
than or equal to $\varepsilon$ needed to cover $\mathcal{H}$. A function $H$
on $\mathcal{X}$ is called an _envelope function_ for $\mathcal{H}$ if
$\sup_{\eta\in\mathcal{H}}|\eta(x)|\leq H(x)$ for all $x\in\mathcal{X}$.

We first consider the sequence of stochastic processes
$\\{V_{n}(\lambda):\lambda\in\Lambda\\}$, for
$V_{n}(\lambda):=\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda}$, where

$\eta_{h,a_{0},j,\lambda}(y,a,w):=h^{1/2}\left(\frac{a-a_{0}}{h}\right)^{j-1}K_{h,a_{0}}(a)\lambda(y,a,w).$

We note that $\eta$ depends on $n$ through $h$. We also make use of the
following semi-metric on $\Lambda$:

$\rho_{a_{0},\varepsilon}(\lambda_{1},\lambda_{2}):=\sup_{|a-a_{0}|<\varepsilon}\left(E_{0}\left[\left\\{\lambda_{1}(Y,A,W)-\lambda_{2}(Y,A,W)\right\\}^{2}\mid
A=a\right]\right)^{1/2}.$

We then have the following lemma.

######  Lemma 9.

Suppose (A1), (A2), and (A5) hold, and that $\Lambda$ is a class of functions
with envelope $L$ satisfying:

  1. 1.

$\sup_{|a-a_{0}|<\delta_{1}}E_{0}[L^{2+\delta_{2}}\mid A=a]<\infty$ for some
$\delta_{1},\delta_{2}>0$; and

  2. 2.

$\int_{0}^{1}\left\\{\sup_{Q}\log
N\left(\varepsilon\|L\|_{Q,2},\Lambda,L_{2}(Q)\right)\right\\}^{1/2}\,d\varepsilon<\infty$.

Then for each $j\in\\{1,2,3,4\\}$,
$\sup_{\lambda\in\Lambda}|\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda}|=O_{p}(1)$,
and for any possibly random sequences $\lambda_{n1},\lambda_{n2}\in\Lambda$
such that $\rho_{a_{0},\varepsilon}(\lambda_{n1},\lambda_{n2})=o_{p}(1)$ for
some $\varepsilon>0$,
$\mathbb{G}_{n}\\{\eta_{h,a_{0},j,\lambda_{n1}}-\eta_{h,a_{0},j,\lambda_{n2}}\\}=o_{p}(1)$,
so
$\mathbb{G}_{n}\left\\{h^{1/2}\Gamma_{0,h,b,a_{0}}(\lambda_{1n}-\lambda_{2n})\right\\}=o_{p}(1)$
as well.

###### Proof of Lemma 9.

We first establish properties of the class
$\mathcal{H}_{h,a_{0},j}:=\\{\eta_{h,a_{0},j,\lambda}:\lambda\in\Lambda\\}$.
Since $L$ is an envelope for $\Lambda$,

$H_{h,a_{0},j}(y,a,w):=h^{1/2}\left(\frac{a-a_{0}}{h}\right)^{j-1}K_{h,a_{0}}(a)L(y,a,w)$

is an envelope for $\mathcal{H}_{h,a_{0},j}$. By the tower property and a
change of variables, we then have for all $h<\delta_{1}$

$\displaystyle P_{0}H_{h,a_{0},j}^{2}$ $\displaystyle=\int
h\left(\frac{a-a_{0}}{h}\right)^{2(j-1)}K_{h,a_{0}}(a)^{2}E_{0}\left[L^{2}\mid
A=a\right]\,f_{0}(a)\,da$ $\displaystyle=\int
u^{2(j-1)}K(u)^{2}E_{0}\left[L^{2}\mid A=a_{0}+uh\right]\,f_{0}(a_{0}+uh)\,du$
$\displaystyle\leq\sup_{|a-a_{0}|<\delta_{1}}E_{0}\left[L^{2}\mid
A=a\right]\int u^{2(j-1)}K(u)^{2}\,f_{0}(a_{0}+uh)\,du.$

The last expression above is $O(1)$ by assumption, (A1), and (A5).

Next, we study the uniform entropy of the class $\mathcal{H}_{h,a_{0},j}$. We
clearly have $\mathcal{H}_{h,a_{0},j}=\mathcal{G}_{h,a_{0},j}\Lambda$, where
$\mathcal{G}_{h,a_{0},j}$ is the class consisting of the single function
$a\mapsto h^{1/2}[(a-a_{0})/h]^{j}K_{h,a_{0}}(a)$. We then have by, e.g. Lemma
5.1 of van der Vaart and van der Laan, (2006), that

$\sup_{Q}N\left(\varepsilon\|H_{h,a_{0},j}\|_{Q,2},\mathcal{H}_{h,a_{0},j},L_{2}(Q)\right)\leq\sup_{Q}N\left(\varepsilon\|L\|_{Q,2},\Lambda,L_{2}(Q)\right)$

for any $\varepsilon>0$. We can now establish the first claim of Lemma 9. By
Theorem 2.14.1 of van der Vaart and Wellner, (1996), we have

$\displaystyle
E_{0}\left[\sup_{\lambda\in\Lambda}\left|\mathbb{G}_{n}\eta_{h,a_{0},h,\lambda}\right|\right]$
$\displaystyle=E_{0}\left[\sup_{\eta\in\mathcal{H}_{h,a_{0},j}}\left|\mathbb{G}_{n}\eta\right|\right]$
$\displaystyle\lesssim\left\\{P_{0}H_{h,a_{0},j}^{2}\right\\}^{1/2}\sup_{Q}\int_{0}^{1}\left\\{1+\log
N\left(\varepsilon\|H_{h,a_{0},j}\|_{Q,2},\mathcal{H}_{h,a_{0},j},L_{2}(Q)\right)\right\\}^{1/2}\,d\varepsilon$
$\displaystyle\leq O(1)\int_{0}^{1}\left\\{\sup_{Q}\log
N\left(\varepsilon\|L\|_{Q,2},\Lambda,L_{2}(Q)\right)\right\\}^{1/2}\,d\varepsilon.$

The integral is finite by assumption, which establishes the claim.

We will establish the second statement by showing that the sequence of
processes is _asymptotically uniformly
$\rho_{a_{0},\varepsilon}$-equicontinuous_ (Pollard,, 1984; van der Vaart and
Wellner,, 1996). To do so, we will use Theorem 2.11.1 (see also Theorem
2.11.22) of van der Vaart and Wellner, (1996), which implies that
$\\{V_{n}:n=1,2,\dots\\}$ is asymptotically uniformly
$\rho_{a_{0},\varepsilon}$-equicontinuous if the following conditions hold:

  * (a)

$P_{0}H_{h,a_{0},j}^{2}=O(1)$;

  * (b)

$P_{0}\left[H_{h,a_{0},j}^{2}I\\{H_{h,a_{0},j}>\varepsilon
n^{1/2}\\}\right]=o(1)$ for every $\varepsilon>0$;

  * (c)

$\sup_{\rho_{a_{0},\varepsilon}(\lambda_{1},\lambda_{2})<\delta_{n}}P_{0}\\{\eta_{h,a_{0},j,\lambda_{1}}-\eta_{h,a_{0},j,\lambda_{2}}\\}^{2}=o(1)$
for every $\delta_{n}\longrightarrow 0$; and

  * (d)

for every $\delta_{n}=o(1)$, $\sup_{Q}\int_{0}^{\delta_{n}}\left\\{\log
N(\varepsilon\|H_{h,a_{0},j}\|_{Q,2},\mathcal{H}_{h,a_{0},j},L_{2}(Q))\right\\}^{1/2}\,d\varepsilon=o(1)$.

We prove that $\\{\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda}:\lambda\in\Lambda\\}$
is $\rho_{a_{0},\varepsilon}$-equicontinuous by establishing these conditions,
and the result follows.

We showed condition (a) above. For (b), we have again by the tower property
and a change of variables

$\displaystyle P_{0}\left\\{H_{h,a_{0},j}^{2}I\left(H_{h,a_{0},j}>\varepsilon
n^{1/2}\right)\right\\}$ $\displaystyle=\int
h\left(\frac{a-a_{0}}{h}\right)^{2(j-1)}K_{h,a_{0}}(a)^{2}E_{0}\left[L^{2}I\left(H_{h,a_{0},j}>\varepsilon
n^{1/2}\right)\mid A=a\right]\,f_{0}(a)\,da$ $\displaystyle=\int
u^{2(j-1)}K(u)^{2}E_{0}\left[L^{2}I\left(H_{h,a_{0},j}>\varepsilon
n^{1/2}\right)\mid A=a_{0}+uh\right]\,f_{0}(a_{0}+uh)\,du.$

Now we note that since the kernel $K$ is uniformly bounded by, say
$\bar{K}<\infty$, and has support contained in $[-1,1]$,

$I\left(H_{h,a_{0},j}>\varepsilon
n^{1/2}\right)=I\left(h^{1/2}[(A-a_{0})/h]^{j-1}K_{h,a_{0}}(A)L>\varepsilon
n^{1/2}\right)\leq I\left(\bar{K}L>\varepsilon(nh)^{1/2}\right).$

Hence, for all $h<\delta_{1}$,

$\displaystyle P_{0}\left\\{H_{h,a_{0},j}^{2}I\left(H_{h,a_{0},j}>\varepsilon
n^{1/2}\right)\right\\}$
$\displaystyle\leq\bar{K}^{2}\int_{-1}^{1}E_{0}\left[L^{2}I\left(L>\varepsilon(nh)^{1/2}/\bar{K}\right)\mid
A=a_{0}+uh\right]\,f_{0}(a_{0}+uh)\,du$
$\displaystyle\lesssim\sup_{|a-a_{0}|<\delta_{1}}E_{0}\left[L^{2}I\left(L>\varepsilon(nh)^{1/2}/\bar{K}\right)\mid
A=a\right].$

Then by Hölder’s inequality,

$\displaystyle
E_{0}\left[L^{2}I\left(L>\varepsilon(nh)^{1/2}/\bar{K}\right)\mid A=a\right]$
$\displaystyle\leq\left\\{E_{0}\left[L^{2+\delta_{2}}\mid
A=a\right]\right\\}^{2/(2+\delta_{2})}\left\\{P_{0}\left(L>\varepsilon(nh)^{1/2}/\bar{K}\mid
A=a\right)\right\\}^{\delta_{2}/(2+\delta_{2})},$

and by Markov’s inequality,

$\displaystyle P_{0}\left(L>\varepsilon(nh)^{1/2}/\bar{K}\mid A=a\right)$
$\displaystyle\leq E_{0}\left[L^{2+\delta_{2}}\mid
A=a\right]/\left[\varepsilon(nh)^{1/2}/\bar{K}\right]^{2+\delta_{2}}.$

Thus,

$\displaystyle\sup_{|a-a_{0}|<\delta_{1}}E_{0}\left[L^{2}I\left(L>\varepsilon(nh)^{1/2}/\bar{K}\right)\mid
A=a\right]$
$\displaystyle\leq\left[\varepsilon(nh)^{1/2}/\bar{K}\right]^{-\delta_{2}}\sup_{|a-a_{0}|<\delta_{1}}E_{0}\left[L^{2+\delta_{2}}\mid
A=a\right].$

Since $nh\longrightarrow\infty$, this latter expression tends to zero by
assumption for every $\varepsilon>0$, which verifies (b).

For (c), by a similar calculation we have for all $h$ small enough that

$\displaystyle
P_{0}\left\\{\eta_{h,a_{0},j,\lambda_{1}}-\eta_{h,a_{0},j,\lambda_{2}}\right\\}^{2}$
$\displaystyle=\int
h\left(\frac{a-a_{0}}{h}\right)^{2(j-1)}K_{h,a_{0}}(a)^{2}E_{0}\left[\left\\{\lambda_{1}-\lambda_{2}\right\\}^{2}\mid
A=a\right]f_{0}(a)\,da$ $\displaystyle=\int
u^{2(j-1)}K(u)^{2}E_{0}\left[\left\\{\lambda_{1}-\lambda_{2}\right\\}^{2}\mid
A=a_{0}+uh\right]f_{0}(a_{0}+uh)\,du$ $\displaystyle\lesssim
C\sup_{|a-a_{0}|<\varepsilon}E_{0}\left[\left\\{\lambda_{1}-\lambda_{2}\right\\}^{2}\mid
A=a\right]$
$\displaystyle=\rho_{a_{0},\varepsilon}(\lambda_{1},\lambda_{2})^{2}.$

Hence,

$\sup_{\rho(\lambda_{1},\lambda_{2})<\delta_{n}}P_{0}\\{\eta_{h,a_{0},j,\lambda_{1}}-\eta_{h,a_{0},j,\lambda_{2}}\\}^{2}\lesssim\delta_{n}^{2},$

so that (c) is satisfied.

For (d), using the entropy bound established above, we have

$\sup_{Q}\int_{0}^{\delta}\left\\{\log
N\left(\varepsilon\|H_{h,a_{0},j}\|_{Q,2},\mathcal{H}_{h,a_{0},j},L_{2}(Q)\right)\right\\}^{1/2}\,d\varepsilon\leq\int_{0}^{\delta}\left\\{\sup_{Q}\log
N\left(\varepsilon\|L\|_{Q,2},\Lambda,L_{2}(Q)\right)\right\\}^{1/2}\,d\varepsilon.$

Since the integral on the right is convergent by assumption, the expression
converges to zero as $\delta\to 0$, which verifies (d).

For the final statement, we expand $h^{1/2}\Gamma_{0,h,b,a_{0}}$ using Lemma 4
as follows:

$\displaystyle h^{1/2}\Gamma_{0,h,b,a_{0}}\lambda$
$\displaystyle=\left\\{e_{1}^{T}f_{0}(a_{0})^{-1}\mathbf{S}_{2}^{-1}+O(h)\right\\}h^{1/2}w_{h,a_{0},1}K_{h,a_{0}}\lambda-\left\\{e_{3}^{T}f_{0}(a_{0})^{-1}\mathbf{S}_{4}^{-1}+O(b)\right\\}h^{1/2}w_{b,a_{0},3}K_{b,a_{0}}\lambda$
$\displaystyle=\sum_{j=1}^{4}C_{h,a_{0},j}\eta_{h,a_{0},j,\lambda}$

for some constants $C_{h,a_{0},j}=O(1)$. By the result above, we then have

$\displaystyle\mathbb{G}_{n}\left\\{h^{1/2}\Gamma_{0,h,b,a_{0}}(\lambda_{1n}-\lambda_{2n})\right\\}$
$\displaystyle=\sum_{j=1}^{4}C_{h,a_{0},j}\mathbb{G}_{n}\left\\{\eta_{h,a_{0},j,\lambda_{n1}}-\eta_{h,a_{0},j,\lambda_{n2}}\right\\}=\sum_{j=1}^{4}O(1)o_{p}(1)=o_{p}(1).$

∎

Next, we consider the sequence of stochastic processes
$\\{\bar{V}_{n}(\lambda):\lambda\in\Lambda\\}$, for
$\bar{V}_{n}(\lambda):=\mathbb{G}_{n}\bar{\eta}_{h,a_{0},j,\lambda}$, where

$\bar{\eta}_{h,a_{0},j,\lambda}(y,a,w):=\int
h^{1/2}\left(\frac{a-a_{0}}{h}\right)^{j-1}K_{h,a_{0}}(a)\lambda(y,a,w)\,dF_{0}(a).$

######  Lemma 10.

Suppose (A1), (A2), and (A5) hold, and that $\Lambda$ is a class of functions
uniformly bounded by $L<\infty$ and satisfying $\sup_{Q}\log
N\left(\varepsilon,\Lambda,L_{2}(Q)\right)\leq C\varepsilon^{-V}$ for some
$C<\infty$ and $V\in(0,2)$. Then
$\sup_{\lambda\in\Lambda}\left|\mathbb{G}_{n}\bar{\eta}_{h,a_{0},j,\lambda}\right|=o_{p}(h^{(2-s)/(2s)})$
for every $s\in(V\vee 1,2)$ and each $j\in\\{1,2,3,4\\}$. Hence,
$\sup_{\lambda\in\Lambda}\left|\mathbb{G}_{n}\left(\int
h^{1/2}\Gamma_{0,h,b,a_{0}}\lambda\,dF_{0}\right)\right|=o_{p}(h^{(2-s)/(2s)})$
for every $s\in(V,2)$ as well.

###### Proof of Lemma 10.

We let
$\bar{\mathcal{H}}_{h,a_{0},j}:=\\{\bar{\eta}_{h,a_{0},j,\lambda}:\lambda\in\Lambda\\}$.
We use Theorem 2.14.1 of van der Vaart and Wellner, (1996). We first bound the
uniform entropy of $\bar{\mathcal{H}}_{h,a_{0},j}$. Lemma 5.1 of van der Vaart
and van der Laan, (2006) implies that the class of functions
$\mathcal{F}_{h,a_{0},j}:=\left\\{(y,a,w)\mapsto
h^{1/2}\left(\frac{a-a_{0}}{h}\right)^{j-1}K_{h,a_{0}}(a)\lambda(y,a,w):\lambda\in\Lambda\right\\}$
satisfies

$\sup_{Q}N\left(\varepsilon\|F_{h,a_{0},j}\|_{Q,2},\mathcal{F}_{h,a_{0},j},L_{2}(Q)\right)\leq\sup_{Q}N\left(\varepsilon
L,\Lambda,L_{2}(Q)\right)$

relative to the envelope function $F_{h,a_{0},j}:a\mapsto
Lh^{1/2}|(a-a_{0})/h|^{j}K_{h,a_{0}}(a)$. We note that for $s\geq 1$,

$\left[\int
F_{h,a_{0},j}(a)^{s}\,dF_{0}(a)\right]^{1/s}=Lh^{1/2+(1-s)/s}\left[\int|u|^{s(j-1)}K(u)^{s}f_{0}(a_{0}+uh)\,du\right]^{1/s}\lesssim
h^{(2-s)/(2s)}$

by (A1) and (A5). Hence, $\bar{H}_{h,s}:=Ch^{(2-s)/(2s)}$ is an envelope for
$\bar{\mathcal{H}}_{h,a_{0},j}$ for some $C<\infty$. By Lemma 5.2 of van der
Vaart and van der Laan, (2006) (with $r=t=2$), the above bound for the uniform
entropy of $\mathcal{F}_{h,a_{0},j}$, and the assumed bound for the uniform
entropy of $\Lambda$, we then have

$\displaystyle\sup_{Q}\log
N\left(\varepsilon\|\bar{H}_{h,s}\|_{Q,2},\mathcal{H}_{h,a_{0},j},L_{2}(Q)\right)$
$\displaystyle\leq\sup_{Q}\log
N\left(\varepsilon^{2/s}\|F_{h,a_{0},j}\|_{Q,2}/2^{r/s},\mathcal{F}_{h,a_{0},j},L_{2}(Q)\right)$
$\displaystyle\leq\sup_{Q}\log
N\left(\varepsilon^{2/s}L/2^{r/s},\Lambda,L_{2}(Q)\right)$
$\displaystyle\lesssim\varepsilon^{-2V/s}.$

Theorem 2.14.1 of van der Vaart and Wellner, (1996) then gives

$\displaystyle
E_{0}\left[\sup_{\bar{\eta}\in\bar{\mathcal{H}}_{h,a_{0},j}}\left|\mathbb{G}_{n}\bar{\eta}\right|\right]$
$\displaystyle\lesssim\|\bar{H}_{h,s}\|_{P,2}\int_{0}^{1}\left\\{1+\sup_{Q}\log
N\left(\varepsilon\|\bar{H}_{h,s}\|_{Q,2},\mathcal{H}_{h,a_{0},j},L_{2}(Q)\right)\right\\}^{1/2}\,d\varepsilon$
$\displaystyle\lesssim
h^{(2-s)/(2s)}\int_{0}^{1}\left\\{1+\varepsilon^{-2V/s}\right\\}^{1/2}\,d\varepsilon.$

The integral is finite so long as $V/s<1$. Thus,

$E_{0}\left[\sup_{\bar{\eta}\in\bar{\mathcal{H}}_{h,a_{0},j}}\left|\mathbb{G}_{n}\bar{\eta}\right|\right]=O\left(h^{(2-s)/(2s)}\right)$

for all $s>V$. Furthermore, for any $s>V$, there exists $s^{\prime}\in(V,s)$,
so that

$E_{0}\left[\sup_{\bar{\eta}\in\bar{\mathcal{H}}_{h,a_{0},j}}\left|\mathbb{G}_{n}\bar{\eta}\right|\right]=O\left(h^{(2-s^{\prime})/(2s^{\prime})}\right)=o\left(h^{(2-s)/(2s)}\right).$

The final statement regarding $\\{\mathbb{G}_{n}\left(\int
h^{1/2}\Gamma_{0,h,b,a_{0}}\lambda\,dF_{0}\right):\lambda\in\Lambda\\}$
follows by decomposing $\Gamma_{0,h,b,a_{0}}$ and using Lemma 8, as in the
proof of Lemma 9. ∎

We now use these results to establish rates of convergence of the remainder
terms $R_{n,h,b,a_{0},2}$ and $R_{n,h,b,a_{0},3}$.

######  Corollary 4.

If (A1)–(A5) hold, then $R_{n,h,b,a_{0},2}=o_{p}\left(\\{nh\\}^{-1/2}\right)$
and $R_{n,h,b,a_{0},3}=O_{p}\left(\\{nh\\}^{-1}\right)$.

###### Proof of Corollary 4.

We can write

$\displaystyle(nh)^{1/2}R_{n,h,b,a_{0},2}$
$\displaystyle=\mathbb{G}_{n}\left\\{h^{1/2}\Gamma_{0,h,b,a_{0}}\left(\psi_{n}-\psi_{\infty}\right)\right\\}+\mathbb{G}_{n}\left\\{h^{1/2}\Gamma_{0,h,b,a_{0}}\left(\int\mu_{n}\,dQ_{0}-\int\mu_{\infty}\,dQ_{0}\right)\right\\}$
$\displaystyle\qquad+\mathbb{G}_{n}\left\\{\int
h^{1/2}\Gamma_{0,h,b,a_{0}}\mu_{n}\,dF_{0}\right\\}-\mathbb{G}_{n}\left\\{\int
h^{1/2}\Gamma_{0,h,b,a_{0}}\mu_{\infty}\,dF_{0}\right\\}$ (9)

and

$\displaystyle(nh)^{1/2}R_{n,h,b,a_{0},3}$
$\displaystyle=\mathbb{G}_{n}\left\\{h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\psi_{n}\right\\}+\mathbb{G}_{n}\left\\{h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\int\mu_{n}\,dQ_{0}\right\\}$
$\displaystyle\qquad+\mathbb{G}_{n}\left\\{\int
h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\mu_{n}\,dF_{0}\right\\}$
(10)

We address the first two terms in each expansion using Lemma 9. For the first
term, we set
$\Lambda:=\\{o\mapsto[y-\mu(a,w)]/g(a,w):\mu\in\mathcal{F}_{\mu},g\in\mathcal{F}_{g}\\}$.
By (A3), an envelope function for $\Lambda$ is $L(y,a,w)=C+C^{\prime}|y|$ for
some $C,C^{\prime}<\infty$. Hence, by the triangle inequality,

$\displaystyle\sup_{|a-a_{0}|<\delta_{1}}E_{0}\left[L^{2+\delta_{2}}\mid
A=a\right]$
$\displaystyle=\sup_{|a-a_{0}|<\delta_{1}}E_{0}\left[\left\\{C+C^{\prime}|Y|\right\\}^{2+\delta_{2}}\mid
A=a\right]$
$\displaystyle\leq\left[C+C^{\prime}\sup_{|a-a_{0}|<\delta_{1}}\left(E_{0}\left\\{|Y|^{2+\delta_{2}}\mid
A=a\right\\}\right)^{1/(2+\delta_{2})}\right]^{2+\delta_{2}},$

which is finite by (A5). Therefore, condition 1 of Lemma 9 holds. For
condition 2 of Lemma 9, we note that permanence properties of uniform entropy
numbers and uniform entropy integrals (see, e.g., Theorem 2.10.20 of van der
Vaart and Wellner,, 1996 and Lemma 5.1 of van der Vaart and van der Laan,,
2006) in conjunction with (A3) implies that $\Lambda$ possesses finite uniform
entropy integral. Finally, we show that
$\rho_{a_{0},\delta_{1}}(\psi_{n},\psi_{\infty})=o_{p}(1)$. By Jensen’s
inequality,

$E_{0}\left(Y^{2}\mid
A=a,W=w\right)\leq\left[E_{0}\left(|Y|^{2+\delta_{2}}\mid
A=a,W=w\right)\right]^{2/(2+\delta_{2})}$

which is uniformly bounded for $a\in B_{\delta_{1}}(a_{0})$ and $P_{0}$-almost
every $w$ by (A5). We also note that by (A3) and (A4), for any functions
$\gamma_{1},\gamma_{2}$ of $(a,w)$,

$\displaystyle\sup_{|a-a_{0}|<\delta_{1}}\left\\{E_{0}\left[\left\\{\gamma_{1}(a,W)-\gamma_{2}(a,W)\right\\}^{2}\mid
A=a\right]\right\\}^{1/2}$
$\displaystyle=\sup_{|a-a_{0}|<\delta_{1}}\left\\{E_{0}\left[g_{0}(a,W)\left\\{\gamma_{1}(a,W)-\gamma_{2}(a,W)\right\\}^{2}\right]\right\\}^{1/2}$
$\displaystyle\lesssim\sup_{|a-a_{0}|<\delta_{1}}\left\\{E_{0}\left[\left\\{\gamma_{1}(a,W)-\gamma_{2}(a,W)\right\\}^{2}\right]\right\\}^{1/2}$
$\displaystyle=\sup_{|a-a_{0}|<\delta_{1}}\left\\{E_{0}\left[I_{B_{\delta_{1}}(a_{0})\times\mathcal{W}}(a,W)\left\\{\gamma_{1}(a,W)-\gamma_{2}(a,W)\right\\}^{2}\right]\right\\}^{1/2}$
$\displaystyle\leq\sum_{j=1}^{3}\sup_{|a-a_{0}|<\delta_{1}}\left\\{E_{0}\left[I_{\mathcal{S}_{j}}(a,W)\left\\{\gamma_{1}(a,W)-\gamma_{2}(a,W)\right\\}^{2}\right]\right\\}^{1/2}$
$\displaystyle=\sum_{j=1}^{3}d\left(\gamma_{1},\gamma_{2};B_{\delta_{1}}(a_{0}),\mathcal{S}_{j}\right).$

We then have

$\displaystyle\rho_{a_{0},\delta_{1}}(\psi_{n},\psi_{\infty})$
$\displaystyle=\sup_{|a-a_{0}|<\delta_{1}}\left\\{E_{0}\left[\left(\psi_{n}-\psi_{0}\right)^{2}\mid
A=a\right]\right\\}^{1/2}$
$\displaystyle=\sup_{|a-a_{0}|<\delta_{1}}\left\\{E_{0}\left[\left(\\{1/(g_{n}g_{\infty})\\}\left\\{Y-\mu_{n}\right\\}\left\\{g_{\infty}-g_{n}\right\\}-\\{1/g_{\infty}\\}\\{\mu_{n}-\mu_{\infty}\\}\right)^{2}\mid
A=a\right]\right\\}^{1/2}$
$\displaystyle\lesssim\sup_{|a-a_{0}|<\delta_{1}}\left\\{E_{0}\left[Y^{2}\left(g_{n}-g_{\infty}\right)^{2}\mid
A=a\right]\right\\}^{1/2}+\sup_{|a-a_{0}|<\delta_{1}}\left\\{E_{0}\left[\mu_{n}^{2}\left(g_{n}-g_{\infty}\right)^{2}\mid
A=a\right]\right\\}^{1/2}$
$\displaystyle\qquad+\sup_{|a-a_{0}|<\delta_{1}}\left\\{E_{0}\left[\left(\mu_{n}-\mu_{\infty}\right)^{2}\mid
A=a\right]\right\\}^{1/2}$
$\displaystyle\lesssim\sup_{|a-a_{0}|<\delta_{1}}\left\\{E_{0}\left[E_{0}\left(Y^{2}\mid
A=a,W\right)\left(g_{n}-g_{\infty}\right)^{2}\mid
A=a\right]\right\\}^{1/2}+\sum_{j=1}^{3}d\left(g_{n},g_{\infty};B_{\delta_{1}}(a_{0}),\mathcal{S}_{j}\right)$
$\displaystyle\qquad+\sum_{j=1}^{3}d\left(\mu_{n},\mu_{\infty};B_{\delta_{1}}(a_{0}),\mathcal{S}_{j}\right)$
$\displaystyle\lesssim\sum_{j=1}^{3}d\left(g_{n},g_{\infty};B_{\delta_{1}}(a_{0}),\mathcal{S}_{j}\right)+\sum_{j=1}^{3}d\left(\mu_{n},\mu_{\infty};B_{\delta_{1}}(a_{0}),\mathcal{S}_{j}\right).$

Hence, by (A4), $\rho_{a_{0},\delta_{1}}(\psi_{n},\psi_{\infty})=o_{p}(1)$.
Lemma 9 thus implies that
$\mathbb{G}_{n}\\{h^{1/2}\Gamma_{0,h,b,a_{0}}(\psi_{n}-\psi_{\infty})\\}=o_{p}(1)$,
which addresses the first term in equation (9). For the first term in equation
(10), we note that

$\displaystyle\mathbb{G}_{n}\left\\{h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\psi_{n}\right\\}$
$\displaystyle=e_{1}^{T}\left(\mathbf{D}_{n,h,a_{0},1}^{-1}-\mathbf{D}_{0,h,a_{0},1}^{-1}\right)\mathbb{G}_{n}\left\\{h^{1/2}w_{h,a_{0},1}K_{h,a_{0}}\psi_{n}\right\\}$
$\displaystyle\qquad-
e_{3}^{T}c_{2}(h/b)^{2}\left(\mathbf{D}_{n,h,a_{0},3}^{-1}-\mathbf{D}_{0,h,a_{0},3}^{-1}\right)\mathbb{G}_{n}\left\\{h^{1/2}w_{b,a_{0},3}K_{b,a_{0}}\psi_{n}\right\\}.$

Since we have established that the conditions of Lemma 9 hold the class
$\Lambda$ that $\psi_{n}$ falls in, Lemma 9 implies that all elements of
$\mathbb{G}_{n}\left\\{h^{1/2}w_{h,a_{0},1}K_{h,a_{0}}\psi_{n}\right\\}$ and
$\mathbb{G}_{n}\left\\{h^{1/2}w_{b,a_{0},3}K_{b,a_{0}}\psi_{n}\right\\}$ are
$O_{p}(1)$. In conjunction with Lemma 8, we then have

$\mathbb{G}_{n}\left\\{h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\psi_{n}\right\\}=O_{p}\left(\\{nh\\}^{-1/2}\right).$

For the second term in equation (9), we use Lemma 9 with
$\Lambda:=\left\\{a\mapsto\int\mu(a,w)\,dQ_{0}(w):\mu\in\mathcal{F}_{\mu}\right\\}$,
which is a uniformly bounded class with envelope $L=C_{1}$ by (A3). Hence,
condition (1) of Lemma 9 hold trivially for this class. For condition (2),
(A3) and Lemma 5.2 of van der Vaart and van der Laan, (2006) (with $r=s=t=2$)
together imply that condition (2) of Lemma 9 is satisfied. Therefore, if
$\rho_{a_{0},\delta_{1}}(\lambda_{n},\lambda_{\infty})=o_{p}(1)$ for
$\lambda_{n}:a\mapsto\int\mu_{n}(a,w)\,dQ_{0}(w)$ and
$\lambda_{\infty}:a\mapsto\int\mu_{\infty}(a,w)\,dQ_{0}(w)$, Lemma 9 implies
that

$\mathbb{G}_{n}\left\\{h^{1/2}\Gamma_{0,h,b,a_{0}}\left(\int\mu_{n}\,dQ_{0}-\int\mu_{\infty}\,dQ_{0}\right)\right\\}=o_{p}(1).$

Since $\lambda_{n}$ and $\lambda_{\infty}$ are functions just of $a$, we have

$\displaystyle\rho_{a_{0},\delta_{1}}(\lambda_{n},\lambda_{\infty})^{2}$
$\displaystyle=\sup_{|a-a_{0}|<\delta_{1}}\left\\{\int\left[\mu_{n}(a,w)-\mu_{\infty}(a,w)\right]\,dQ_{0}(w)\right\\}^{2}\leq\sup_{|a-a_{0}|<\delta_{1}}\int\left[\mu_{n}(a,w)-\mu_{\infty}(a,w)\right]^{2}\,dQ_{0}(w),$

which is $o_{p}(1)$ by (A4) as noted above. A similar argument yields that the
second term of (10) is $O_{p}\left(\\{nh\\}^{-1/2}\right)$.

For the third and fourth terms in equation (9), we use Lemma 10 with
$\Lambda=\mathcal{F}_{\mu}$. The conditions of the lemma are satisfied by
(A3), so that

$\sup_{\mu\in\mathcal{F}_{\mu}}\left|\mathbb{G}_{n}\left\\{\int
h^{1/2}\Gamma_{0,h,b,a_{0}}\mu\,dF_{0}\right\\}\right|=o_{p}\left(h^{(2-s)/(2s)}\right)$

for every $s\in(V_{\mu},2)$. Since $\mu_{n},\mu_{\infty}\in\mathcal{F}_{\mu}$
almost surely for all $n$ large enough, the result follows.

For the third term in equation (10), we note that

$\displaystyle\mathbb{G}_{n}\left\\{\int
h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\mu_{n}\,dF_{0}\right\\}$
$\displaystyle=e_{1}^{T}\left(\mathbf{D}_{n,h,a_{0},1}^{-1}-\mathbf{D}_{0,h,a_{0},1}^{-1}\right)\mathbb{G}_{n}\left\\{\int
h^{1/2}w_{h,a_{0},1}K_{h,a_{0}}\mu_{n}\,dF_{0}\right\\}$ $\displaystyle\quad-
e_{3}^{T}c_{2}(h/b)^{2}\left(\mathbf{D}_{n,b,a_{0},3}^{-1}-\mathbf{D}_{0,b,a_{0},3}^{-1}\right)\mathbb{G}_{n}\left\\{\int
h^{1/2}w_{h,a_{0},3}K_{h,a_{0}}\mu_{n}\,dF_{0}\right\\}.$

Each of the terms in $\mathbb{G}_{n}\left\\{\int
h^{1/2}w_{h,a_{0},1}K_{h,a_{0}}\mu_{n}\,dF_{0}\right\\}$ and
$\mathbb{G}_{n}\left\\{\int
h^{1/2}w_{b,a_{0},3}K_{b,a_{0}}\mu_{n}\,dF_{0}\right\\}$ are
$o_{p}\left(h^{(2-s)/(2s)}\right)$ by Lemma 10, so they are also $O_{p}(1)$.
In conjunction with Lemma 8, we then have

$\mathbb{G}_{n}\left\\{\int
h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\mu_{n}\,dF_{0}\right\\}=O_{p}\left(\\{nh\\}^{-1/2}\right),$

as claimed. ∎

We now turn to uniform control of $R_{n,h,b,s,2}$ and $R_{n,h,b,,3}$ over
$a_{0}\in\mathcal{A}_{0}$. Let $\mathcal{F}$ be a class of measurable
functions equipped with a measurable envelope $F$. We say that $\mathcal{F}$
is _VC type_ with envelope $F$ if there exists a constant $V>0$ such that
$\sup_{Q}N(\varepsilon\|F\|_{Q,2},\mathcal{F},L_{2}(Q))\lesssim\varepsilon^{-V}$
when $\varepsilon<1$ and the supremum is taken over all probability measures.

######  Lemma 11.

If (A1) holds, then
$\\{a\mapsto(\tfrac{a-a_{0}}{h})^{j-1}K(\tfrac{a-a_{0}}{h}):a_{0}\in\mathcal{A}_{0}\\}$
is VC type for each fixed $h>0$ and positive integer $j$.

###### Proof of Lemma 11.

By (A1), $K$ is supported on $[-1,1]$ and is uniformly bounded. Hence,

$(\tfrac{a-a_{0}}{h})^{j-1}K(\tfrac{a-a_{0}}{h})=I\left(\left|\tfrac{a-a_{0}}{h}\right|\leq
1\right)(\tfrac{a-a_{0}}{h})^{j-1}K(\tfrac{a-a_{0}}{h}),$

which implies that the class in question is contained in the product of the
two classes of functions

$\displaystyle\left\\{K\left(\frac{a-a_{0}}{h}\right):a_{0}\in\mathcal{A}_{0}\right\\}\text{
and
}\left\\{\left(\frac{a-a_{0}}{h}\right)^{j-1}I\left(\left|\frac{a-a_{0}}{h}\right|\leq
1\right):a_{0}\in\mathcal{A}_{0}\right\\}.$

The first class is VC type by (A1) and the results of Giné and Guillou,
(2002). For the second class, we first rewrite this class as

$\left\\{\phi\circ
p_{j}\left(\frac{a-a_{0}}{h}\right):a_{0}\in\mathcal{A}_{0}\right\\},$

where where $\phi:=u\mapsto uI(|u|\leq 1)$ and $p_{j}:=u\mapsto u^{j}$. Since
$\phi$ is a bounded real function of bounded variation and $p_{j}$ is a
polynomial, Giné and Guillou, (2002) implies that this class is also VC. Since
the product of VC classes is also VC, (e.g., Lemma 5.1 of van der Vaart and
van der Laan,, 2006), the result follows. ∎

######  Corollary 5.

For $a_{0}\in\mathcal{A}_{0}$ and $h,b>0$, consider the following function:

$\eta_{h,b,a_{0}}:(y,a,w)\mapsto
h^{1/2}\sigma^{-1}_{\infty,h,b}(a_{0})\phi_{\infty,h,b,a_{0}}^{*}(y,a,w).$

If (A1) holds, then the class of functions
$\mathcal{H}_{h,b}:=\\{\eta_{h,b,a_{0}}:a_{0}\in\mathcal{A}_{0}\\}$ is VC
type.

###### Proof of Corollary 5.

For each $a_{0}\in\mathcal{A}_{0}$, $\eta_{h,b,a_{0}}$ is defined as

$\displaystyle
h^{1/2}\sigma^{-1}_{\infty,h,b}(a_{0})\left[\Gamma_{0,h,b,a_{0}}(a)\xi_{\infty}(y,a,w)-\gamma_{0,h,b,a_{0}}(a)+\int\Gamma_{0,h,b,a_{0}}(\bar{a})\left\\{\mu_{\infty}(\bar{a},w)-\int\mu_{\infty}(\bar{a},\bar{w})\,dQ_{0}(\bar{w})\right\\}\,dF_{0}(\bar{a})\right].$

Any collection of constants $\\{c_{a_{0}}:a_{0}\in\mathcal{A}_{0}\\}$ is a VC
class with VC index 1 because no set of size 2 can be shattered. This implies
that $\\{\sigma_{\infty,h,b}(a_{0}):a_{0}\in\mathcal{A}_{0}\\}$,
$\\{P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right):a_{0}\in\mathcal{A}_{0}\\}$,
$\\{\mathbf{D}^{-1}_{0,h,a_{0},1}[i,j]:a_{0}\in\mathcal{A}_{0}\\}$ for each
$(i,j)$, and
$\\{\iint\Gamma_{0,h,b,a_{0}}(\bar{a})\mu_{\infty}(\bar{a},\bar{w})\,dQ_{0}(\bar{w})\,dF_{0}(\bar{a}):a_{0}\in\mathcal{A}_{0}\\}$
are all VC type. We note that by permanence properties of uniform covering
numbers, sums and products of VC classes of functions are also VC type. It
thus remains to show that
$\\{\Gamma_{0,h,b,a_{0}}(a)\xi_{\infty}(y,a,w):a_{0}\in\mathcal{A}_{0}\\}$,
$\\{\gamma_{0,h,b,a_{0}}(a):a_{0}\in\mathcal{A}_{0}\\}$ and
$\\{\int\Gamma_{0,h,b,a_{0}}(\bar{a})\mu_{\infty}(\bar{a},w)\,dF_{0}(\bar{a}):a_{0}\in\mathcal{A}_{0}\\}$
are VC-type.

We recall that

$\displaystyle\Gamma_{0,h,b,a_{0}}(a)=e_{1}^{T}\mathbf{D}_{0,h,a_{0},1}^{-1}w_{h,a_{0},1}(a)K_{h,a_{0}}(a)-e_{3}^{T}c_{2}(h/b)^{2}\mathbf{D}_{0,b,a_{0},3}^{-1}w_{b,a_{0},3}(a)K_{b,a_{0}}(a).$

By Lemma 11, each entry of
$\\{w_{h,a_{0},1}(a)K_{h,a_{0}}(a):a_{0}\in\mathcal{A}_{0}\\}$ and
$\\{w_{b,a_{0},3}(a)K_{b,a_{0}}(a):a_{0}\in\mathcal{A}_{0}\\}$ are VC type.
Combined with the discussion of sets of constants above, this implies that
$\\{\Gamma_{0,h,b,a_{0}}(a):a_{0}\in\mathcal{A}_{0}\\}$ is VC type for each
$h,b>0$. By Lemma 2.6.18 of van der Vaart and Wellner, (1996), element-wise
products of a fixed function and a VC class of functions are also VC type.
Hence, it follows that
$\\{\Gamma_{0,h,b,a_{0}}(a)\xi_{\infty}(y,a,w):a_{0}\in\mathcal{A}_{0}\\}$ is
also VC type. Similarly,
$\\{\Gamma_{0,h,b,a_{0}}(a)\mu_{\infty}(a,w):a_{0}\in\mathcal{A}_{0}\\}$ is VC
type. Lemma 5.2 of van der Vaart and van der Laan, (2006) (with $r=s=t=2$)
then implies that
$\\{\int\Gamma_{0,h,b,s}(\bar{a})\mu_{\infty}(\bar{a},w)\,dF_{0}(\bar{a}):a_{0}\in\mathcal{A}_{0}\\}$
is also VC type.

Finally, we recall that

$\displaystyle\gamma_{0,h,b,a_{0}}(a)$
$\displaystyle=e_{1}^{T}\mathbf{D}^{-1}_{0,h,a_{0},1}w_{h,a_{0},1}(a)K_{h,a_{0}}(a)w^{T}_{h,s,1}(a)\mathbf{D}^{-1}_{0,h,a_{0},1}P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)$
$\displaystyle\qquad-
e_{3}^{T}c_{2}(h/b)^{2}\mathbf{D}_{0,b,a_{0},3}^{-1}w_{b,a_{0},3}(a)K_{b,a_{0}}(a)w^{T}_{b,a_{0},3}(a)\mathbf{D}^{-1}_{0,b,a_{0},3}P_{0}\left(w_{b,a_{0},3}K_{b,a_{0}}\theta_{0}\right).$

As in Lemma 11, each entry of
$\\{w_{h,a_{0},1}(a)K_{h,s}(a)w^{T}_{h,a_{0},1}(a):a_{0}\in\mathcal{A}_{0}\\}$
and
$\\{w_{b,a_{0},3}(a)K_{b,a_{0}}(a)w^{T}_{b,a_{0},3}(a):a_{0}\in\mathcal{A}_{0}\\}$
are VC type. Thus by the permanence property of VC-type function classes, we
conclude that $\\{\gamma_{0,h,b,a_{0}}(a):a_{0}\in\mathcal{A}_{0}\\}$ is also
VC type. ∎

######  Lemma 12.

If (A1) and (A8) hold, then
$\sup_{a_{0}\in\mathcal{A}_{0}}\|\mathbf{D}_{n,h,a_{0},1}-\mathbf{D}_{0,h,a_{0},1}\|_{\infty}$
and
$\sup_{a_{0}\in\mathcal{A}_{0}}\|\mathbf{D}^{-1}_{n,h,a_{0},1}-\mathbf{D}^{-1}_{0,h,a_{0},1}\|_{\infty}$
are $O_{p}(\\{nh/\log h^{-1}\\}^{-1/2})$. If in addition $nh/\log
h^{-1}\longrightarrow\infty$, then
$\sup_{a_{0}\in\mathcal{A}_{0}}\|\mathbf{D}_{n,b,a_{0},3}-\mathbf{D}_{0,b,a_{0},3}\|_{\infty}$
and
$\sup_{a_{0}\in\mathcal{A}_{0}}\|\mathbf{D}^{-1}_{n,b,a_{0},3}-\mathbf{D}^{-1}_{0,b,a_{0},3}\|_{\infty}$
are $O_{p}(\\{nh/\log h^{-1}\\}^{-1/2})$.

###### Proof of Lemma 12.

For all $1\leq j,k\leq 2$, we can write

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbf{D}_{n,h,a_{0},1}[j,k]-\mathbf{D}_{0,h,a_{0},1}[j,k]\right|$
$\displaystyle=h^{-1}\sup_{a_{0}\in\mathcal{A}_{0}}\left|\int\left(\frac{a-a_{0}}{h}\right)^{j+k-2}K\left(\frac{a-a_{0}}{h}\right)\,\left\\{dF_{n}(a)-\,dF_{0}(a)\right\\}\right|$
$\displaystyle=n^{-1/2}h^{-1}\sup_{f\in\mathcal{F}_{j+k-2,h}}\left|\mathbb{G}_{n}f\right|$

for

$\mathcal{F}_{j+k-2,h}:=\left\\{a\mapsto\left(\frac{a-a_{0}}{h}\right)^{j+k-2}K\left(\frac{a-a_{0}}{h}\right)\,:\,a_{0}\in\mathcal{A}_{0}\right\\}.$

By (A1) and Lemma 11, $\mathcal{F}_{j+k-2,h}$ is a uniformly bounded VC class.
For all $a_{0}\in\mathcal{A}_{0}$,

$\displaystyle\mathrm{Var}\left\\{\left(\frac{A-a_{0}}{h}\right)^{j+k-2}K\left(\frac{A-a_{0}}{h}\right)\right\\}$
$\displaystyle\leq
E_{0}\left[\left\\{\left(\frac{A-a_{0}}{h}\right)^{j+k-2}K\left(\frac{A-a_{0}}{h}\right)\right\\}^{2}\right]$
$\displaystyle=\int\left\\{\left(\frac{a-a_{0}}{h}\right)^{j+k-2}K\left(\frac{a-a_{0}}{h}\right)\right\\}^{2}f_{0}(a)\,da$
$\displaystyle=\int hu^{2(j+k-2)}K(u)^{2}f_{0}(uh+a_{0})\,du$
$\displaystyle\leq h\|f_{0}\|_{\infty}c_{2(j+k-2)}.$

Additionally, we have that
$\sup_{a_{0}\in\mathcal{A}_{0}}|\left(\frac{a-a_{0}}{h}\right)^{j+k-2}K\left(\frac{a-a_{0}}{h}\right)|\leq\|K\|_{\infty}$
by (A1). Hence, $\mathcal{F}_{j+k-2,h}$ satisfies the conditions of Theorem
2.1 of Giné and Guillou, (2002) with $\sigma^{2}$ proportional to $h$, which
implies that that

$\sup_{f\in\mathcal{F}_{j+k-2,h}}\left|\mathbb{G}_{n}f\right|=O_{p}\left(\left\\{h\log
h^{-1}\right\\}^{1/2}\right).$

Hence,

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbf{D}_{n,h,a_{0},1}[j,k]-\mathbf{D}_{0,h,a_{0},1}[j,k]\right|=O_{p}\left(\left\\{\frac{\log
h^{-1}}{nh}\right\\}^{1/2}\right)$

for each $1\leq j,k\leq 2$. We conclude that
$\sup_{a_{0}\in\mathcal{A}_{0}}\|\mathbf{D}_{n,h,a_{0},1}-\mathbf{D}_{0,h,a_{0},1}\|_{\infty}=O_{p}(\\{nh/\log
h^{-1}\\}^{-1/2})$. The remainder of the proof is analogous to that of Lemma
8, where we use the assumption that $nh/\log h^{-1}\longrightarrow\infty$ to
conclude that
$\sup_{a_{0}\in\mathcal{A}_{0}}\|\mathbf{D}_{n,h,a_{0},1}-\mathbf{D}_{0,h,a_{0},1}\|_{\infty}=o_{p}(1)$.
∎

As above, we define

$\eta_{h,a_{0},j,\lambda}(y,a,w):=h^{1/2}\left(\frac{a-a_{0}}{h}\right)^{j-1}K_{h,a_{0}}\left(a\right)\lambda(y,a,w),$

and we let $\lambda_{\infty}$ be a fixed function in $\Lambda$. We also define
the following semi-metric on $\Lambda$:

$\rho_{\mathcal{A}_{\delta}}(\lambda_{1},\lambda_{2}):=\sup_{a_{0}\in\mathcal{A}_{\delta}}\left(E_{0}\left[\left\\{\lambda_{1}(Y,A,W)-\lambda_{2}(Y,A,W)\right\\}^{2}\mid
A=a_{0}\right]\right)^{1/2},$

where $\mathcal{A}_{\delta}:=\\{a:\exists
a_{0}\in\mathcal{A}_{0},|a-a_{0}|\leq\delta\\}$. For each $r>0$, we then
define
$\Lambda_{r}:=\\{\lambda\in\Lambda:\rho_{\mathcal{A}_{\delta}}(\lambda,\lambda_{\infty})\leq
r\\}$ and
$\mathcal{H}_{h,j,r}:=\\{\eta_{h,a_{0},j,\lambda}-\eta_{h,a_{0},j,\lambda_{\infty}}:a_{0}\in\mathcal{A}_{0},\lambda\in\Lambda_{r}\\}$.
If $\Lambda$ is equipped with an envelope function $L$, then
$H_{h}:=2h^{-1/2}\|K\|_{\infty}L$ is an envelope for $\mathcal{H}_{h,j,r}$.
The next Lemma controls the uniform entropy of $\mathcal{H}_{h,j,r}$ in terms
of that of $\Lambda$.

######  Lemma 13.

Suppose $\Lambda$ has envelope function $L$ with $PL^{2}<\infty$ and such that
$\sup_{Q}\log N(\varepsilon\|L\|_{Q,2},\Lambda,L_{2}(Q))\leq
C\varepsilon^{-V}$ for some $C,V<\infty$. If (A1) also holds, then

$\sup_{Q}\log N(\varepsilon\|H_{h}\|_{Q,2},\mathcal{H}_{h,j,r},L_{2}(Q))\leq
C^{\prime}\varepsilon^{-V}$

for all $\varepsilon\leq 1$ and some $C^{\prime}<\infty$.

###### Proof of Lemma 13.

For each $h>0$ and $j\in\\{1,\dotsc,4\\}$, we define

$\displaystyle\mathcal{K}_{h,j}:=\left\\{\left(\frac{a-a_{0}}{h}\right)^{j-1}K\left(\frac{a-a_{0}}{h}\right):a_{0}\in\mathcal{A}_{0}\right\\},$

which is uniformly bounded by $\|K\|_{\infty}$. We have that
$\mathcal{H}_{h,j,r}$ is then contained in the product of
$h^{-1/2}\mathcal{K}_{h,j}$ and $\Lambda-\lambda_{\infty}$, which has envelope
$2L$. Lemma 5.1 of van der Vaart and van der Laan, (2006) implies that

$\displaystyle\sup_{Q}\log
N\left(\varepsilon\|H_{h,j,r}\|_{Q,2},\mathcal{H}_{h,j,r},L_{2}(Q)\right)\leq\sup_{Q}\log
N\left(\varepsilon
h^{-1/2}\|K\|_{\infty},h^{-1/2}\mathcal{K}_{h,j},L_{2}(Q)\right)+\sup_{Q}\log
N\left(\varepsilon\|L\|_{Q,2},\Lambda,L_{2}(Q)\right)$

for any $\varepsilon>0$. By Lemma 11, $\mathcal{K}_{h,j}$ is VC type, which
implies by Theorem 2.6.7 of van der Vaart and Wellner, (1996) that

$\sup_{Q}\log N\left(\varepsilon
h^{-1/2}\|K\|_{\infty},h^{-1/2}\mathcal{K}_{h,j},L_{2}(Q)\right)=\sup_{Q}\log
N\left(\varepsilon\|K\|_{\infty},\mathcal{K}_{h,j},L_{2}(Q)\right)\lesssim\log\varepsilon^{-1},$

which is bounded up to a constant by $\varepsilon^{-V}$. ∎

######  Lemma 14.

Suppose $\Lambda$ is a class of functions uniformly bounded by $L<\infty$ and
satisfying $\sup_{Q}\log N\left(\varepsilon,\Lambda,L_{2}(Q)\right)\leq
C\varepsilon^{-V}$ for some $C<\infty$ and $V\in(0,2)$ such that
$n\left[h/(\log n)\right]^{\frac{2+V}{2-V}}\longrightarrow\infty$. If
$r=r_{n}>0$ is a sequence satisfying

$r=o\left(h^{\frac{V}{2(2-V)}}\\{\log n\\}^{-\frac{1}{2-V}}\right)$

and (A1) holds, then

$E_{0}\left[\sup_{\zeta\in\mathcal{H}_{h,j,r}}\left|\mathbb{G}_{n}\zeta\right|\right]=o(\\{\log
n\\}^{-1/2})$

for each $j\in\\{1,2,3,4\\}$. Consequently, if the above conditions hold and
$\lambda_{n}\in\Lambda$ is a sequence of possibly random functions satisfying

$\rho_{\mathcal{A}_{\delta}}(\lambda_{n},\lambda_{\infty})=o_{p}\left(h^{\frac{V}{2(2-V)}}\\{\log
n\\}^{-\frac{1}{2-V}}\right)$

for some $\delta>0$ and (A8) also holds, then

$\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}\Gamma_{0,h,b,a_{0}}(\lambda_{n}-\lambda_{\infty})\right\\}\right|=o_{p}\left(\\{\log
n\\}^{-1/2}\right).$

###### Proof of Lemma 14.

For each $\eta_{h,a_{0},j,\lambda}$ such that $h\leq\delta$, we have

$\displaystyle
P_{0}(\eta_{h,a_{0},j,\lambda}-\eta_{h,a_{0},j,\lambda_{\infty}})^{2}$
$\displaystyle=h^{-1}\int\left\\{\left(\frac{a-a_{0}}{h}\right)^{j-1}K\left(\frac{a-a_{0}}{h}\right)(\lambda-\lambda_{\infty})(y,a,w)\right\\}^{2}\,dP_{0}(y,a,w)$
$\displaystyle=h^{-1}\int\left(\frac{a-a_{0}}{h}\right)^{2(j-1)}\left\\{K\left(\frac{a-a_{0}}{h}\right)\right\\}^{2}E_{0}\left\\{(\lambda-\lambda_{\infty})^{2}\mid
A=a\right\\}f_{0}(a)\,da$ $\displaystyle=\int
u^{2(j-1)}K(u)^{2}E_{0}\\{(\lambda-\lambda_{\infty})^{2}\mid
A=a_{0}+uh\\}f_{0}(a_{0}+uh)\,du$
$\displaystyle\leq\|K\|_{\infty}^{2}\sup_{a\in\mathcal{A}_{\delta}}E_{0}\\{(\lambda-\lambda_{\infty})^{2}\mid
A=a\\}\|f_{0}\|_{\infty}$
$\displaystyle=\|K\|_{\infty}^{2}\rho_{\mathcal{A}_{\delta}}(\lambda,\lambda_{\infty})^{2}\|f_{0}\|_{\infty}.$

Hence, for all $\zeta\in\mathcal{H}_{h,j,r}$ with $h\leq\delta$, we have
$P_{0}\zeta^{2}\leq r^{2}\|K\|_{\infty}^{2}\|f_{0}\|_{\infty}$. Since
$\Lambda$ is uniformly bounded by $L$, $\mathcal{H}_{h,j,r}$ is uniformly
bounded by $H_{h,j,r}=2h^{-1/2}L\|K\|_{\infty}$, and hence
$P_{0}H^{2}_{h,j,r}=4h^{-1}L^{2}\|K\|_{\infty}^{2}$. Therefore,
$P_{0}\zeta^{2}\leq C^{2}hr^{2}P_{0}H^{2}_{h,j,r}$ for all
$\zeta\in\mathcal{H}_{h,j,r}$ with $h\leq\delta$, where
$C=\|f_{0}\|_{\infty}^{1/2}/(2L)$ does not depend on $h$ or $r$. Theorem 2.1
of van der Vaart and Wellner, (2011) (applied to
$\mathcal{F}=\mathcal{H}_{h,j,r}/H_{h,j,r}$, which is uniformly bounded by 1)
then implies that for all $n$ large enough,

$\displaystyle
E_{0}\left[\sup_{\zeta\in\mathcal{H}_{h,j,r}}|\mathbb{G}_{n}\zeta|\right]\lesssim
h^{-1/2}J(Crh^{1/2},\mathcal{H}_{h,j,r},L_{2})\left\\{1+\frac{J(Crh^{1/2},\mathcal{H}_{h,j,r},L_{2})}{C^{2}hr^{2}n^{1/2}}\right\\},$

where

$\displaystyle J(x,\mathcal{H}_{h,j,r},L_{2})$
$\displaystyle:=\sup_{Q}\int_{0}^{x}\sqrt{1+\log
N\left(\varepsilon\|H_{h,j,r}\|_{Q,2},\mathcal{H}_{h,j,r},L_{2}(Q)\right)}\,d\varepsilon\lesssim\int_{0}^{x}\varepsilon^{-V/2}\,d\varepsilon\lesssim
x^{1-V/2}$

for all $x\in(0,1]$ by Lemma 13. Therefore,

$\displaystyle\\{\log
n\\}^{1/2}E_{0}\left[\sup_{\zeta\in\mathcal{H}_{h,j,r}}|\mathbb{G}_{n}\zeta|\right]$
$\displaystyle\lesssim\\{\log
n\\}^{1/2}h^{-1/2}r^{\frac{2-V}{2}}h^{\frac{2-V}{4}}\left\\{1+\frac{r^{\frac{2-V}{2}}h^{\frac{2-V}{4}}}{hr^{2}n^{1/2}}\right\\}$
$\displaystyle=\left\\{r\left[\log
n\right]^{\frac{1}{2-V}}h^{-\frac{V}{2(2-V)}}\right\\}^{\frac{2-V}{2}}+\left\\{r\left[\log
n\right]^{-\frac{1}{2V}}n^{\frac{1}{2V}}h^{\frac{1+V}{2V}}\right\\}^{-V}$
$\displaystyle=\left\\{rs\right\\}^{\frac{2-V}{2}}+\left\\{rt\right\\}^{-V},$

where $s:=\left[\log n\right]^{\frac{1}{2-V}}h^{-\frac{V}{2(2-V)}}$ and
$t:=\left[\log n\right]^{-\frac{1}{2V}}n^{\frac{1}{2V}}h^{\frac{1+V}{2V}}$. We
note that

$t/s=\left\\{n\left[h/(\log
n)\right]^{\frac{2+V}{2-V}}\right\\}^{\frac{1}{2V}}\longrightarrow\infty$

by assumption. We now define $\bar{r}:=\max\\{r,[st]^{-1/2}\\}$. Since
$r\leq\bar{r}$, we then have

$\sup_{\zeta\in\mathcal{H}_{h,j,r}}|\mathbb{G}_{n}\zeta|\leq\sup_{\zeta\in\mathcal{H}_{h,j,\bar{r}}}|\mathbb{G}_{n}\zeta|$

almost surely by the increasing nature of the sets $\mathcal{H}_{h,j,r}$.
Using the above bound applied to $\bar{r}$, we also have

$\displaystyle\\{\log
n\\}^{1/2}E_{0}\left[\sup_{\zeta\in\mathcal{H}_{h,j,\bar{r}}}|\mathbb{G}_{n}\zeta|\right]$
$\displaystyle\lesssim\left\\{\bar{r}s\right\\}^{\frac{2-V}{2}}+\left\\{\bar{r}t\right\\}^{-V},$

By assumption, $rs=o(1)$, and $[st]^{-1/2}s=[s/t]^{1/2}$, which tends to zero
since $t/s\longrightarrow\infty$ as noted above. Therefore, $\bar{r}s=o(1)$.
In addition, $\bar{r}t\geq[st]^{-1/2}t=[t/s]^{1/2}\longrightarrow\infty$.
Hence,
$\left\\{\bar{r}s\right\\}^{\frac{2-V}{2}}+\left\\{\bar{r}t\right\\}^{-V}=o(1)$.
Putting it together, we have

$\\{\log
n\\}^{1/2}E_{0}\left[\sup_{\zeta\in\mathcal{H}_{h,j,r}}|\mathbb{G}_{n}\zeta|\right]\leq\\{\log
n\\}^{1/2}E_{0}\left[\sup_{\zeta\in\mathcal{H}_{h,j,\bar{r}}}|\mathbb{G}_{n}\zeta|\right]\lesssim\left\\{\bar{r}s\right\\}^{\frac{2-V}{2}}+\left\\{\bar{r}t\right\\}^{-V}=o(1),$

which proves the first claim.

For the second claim, as in the proof of Lemma 9, we can write
$h^{1/2}\Gamma_{0,h,b,a_{0}}\lambda=\sum_{j=1}^{4}C_{h,a_{0},j}\eta_{h,a_{0},j,\lambda}$
for some constants $C_{h,a_{0},j}$ such that
$\sup_{a_{0}\in\mathcal{A}_{0}}|C_{h,a_{0},j}|=O(1)$ by the uniform statements
of Lemma 4. Thus,

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}\Gamma_{0,h,b,a_{0}}(\lambda_{n}-\lambda_{\infty})\right\\}\right|$
$\displaystyle\leq\sum_{j=1}^{4}\sup_{a_{0}\in\mathcal{A}_{0}}\left|C_{h,a_{0},j}\right|\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda_{n}}\right|=O(1)\sum_{j=1}^{4}\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda_{n}}\right|.$

Hence, if we can show that
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda_{n}}\right|=o_{p}\left(\\{\log
n\\}^{-1/2}\right)$ for each $j\in\\{1,2,3,4\\}$, then the claim follows.

For any $\nu,\gamma>0$, we have

$\displaystyle
P_{0}\left(\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda_{n}}\right|>\nu/\\{\log
n\\}^{1/2}\right)$ $\displaystyle\leq
P_{0}\left(\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda_{n}}\right|>\nu/\\{\log
n\\}^{1/2},\rho_{\mathcal{A}_{\delta}}(\lambda_{n},\lambda_{\infty})<\gamma/s\right)$
$\displaystyle\qquad+P_{0}\left(\rho_{\mathcal{A}_{\delta}}(\lambda_{n},\lambda_{\infty})\geq\gamma/s\right).$

Now, $\rho_{\mathcal{A}_{\delta}}(\lambda_{n},\lambda_{\infty})<\gamma/s$
implies that $\eta_{h,a_{0},j,\lambda_{n}}\in\mathcal{H}_{h,j,\gamma/s}$, so
by Markov’s inequality,

$\displaystyle
P_{0}\left(\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda_{n}}\right|>\nu/\\{\log
n\\}^{1/2},\rho_{\mathcal{A}_{0},\delta}(\lambda_{n},\lambda_{\infty})<\gamma/s\right)$
$\displaystyle\leq
P_{0}\left(\sup_{\zeta\in\mathcal{H}_{h,j,\gamma/a}}\left|\mathbb{G}_{n}\zeta\right|>\nu/\\{\log
n\\}^{1/2}\right)$ $\displaystyle\leq\nu^{-1}\\{\log
n\\}^{1/2}E_{0}\left[\sup_{\zeta\in\mathcal{H}_{h,j,\gamma/s}}\left|\mathbb{G}_{n}\zeta\right|\right].$

Applying the same the same technique as used above, we then have

$\displaystyle\\{\log
n\\}^{1/2}E_{0}\left[\sup_{\zeta\in\mathcal{H}_{h,j,\gamma/s}}\left|\mathbb{G}_{n}\zeta\right|\right]$
$\displaystyle\lesssim\left[\max\\{\gamma/s(st)^{-1/2}\\}s\right]^{\frac{2-V}{2}}+\left[\max\\{\gamma/s,(st)^{-1/2}\\}t\right]^{-V}$
$\displaystyle=\left[\max\\{\gamma,(s/t)^{1/2}\\}\right]^{\frac{2-V}{2}}+\left[\max\\{\gamma(t/s),(t/s)^{1/2}\\}\right]^{-V}.$

Therefore, for any $\nu,\gamma>0$, we have

$\displaystyle
P_{0}\left(\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda_{n}}\right|>\nu/\\{\log
n\\}^{1/2}\right)$
$\displaystyle\lesssim\nu^{-1}\left[\max\\{\gamma,(s/t)^{1/2}\\}\right]^{\frac{2-V}{2}}+\nu^{-1}\left[\max\\{\gamma(t/s),(t/s)^{1/2}\\}\right]^{-V}$
$\displaystyle\qquad+P_{0}\left(\rho_{\mathcal{A}_{\delta}}(\lambda_{n},\lambda_{\infty})\geq\gamma/s\right)$

Since $s/t\longrightarrow 0$,
$\rho_{\mathcal{A}_{\delta}}(\lambda_{n},\lambda_{\infty})=o_{p}(s^{-1})$, and
$\gamma$ was arbitrary, for any fixed $\nu>0$, we can choose $\gamma$ to make
the above expression as small as we like for all $n$ large enough. This
implies that

$P_{0}\left(\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda_{n}}\right|>\nu/\\{\log
n\\}^{1/2}\right)\longrightarrow 0$

for any $\nu>0$, so that
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda_{n}}\right|=o_{p}\left(\\{\log
n\\}^{-1/2}\right)$ for each $j\in\\{1,2,3,4\\}$, which completes the proof.

∎

Next, we define

$\bar{\eta}_{h,a_{0},j,\lambda}(y,a,w):=\int\eta_{h,a_{0},j,\lambda}(y,a,w)\,dF_{0}(a)=\int
h^{1/2}\left(\frac{a-a_{0}}{h}\right)^{j-1}K_{h,a_{0}}(a)\lambda(y,a,w)\,dF_{0}(a).$

The next lemma provides a rate of convergence for
$\sup_{s\in\mathcal{S},\lambda\in\Lambda}\left|\mathbb{G}_{n}\bar{\eta}_{h,s,j,\lambda}\right|$,
which is sufficient to obtain a rate of convergence of
$\sup_{a_{0}\in\mathcal{A}_{0}}|R_{n,h,b,s,2}|$ and
$\sup_{a_{0}\in\mathcal{A}_{0}}|R_{n,h,b,s,3}|$. We note that we could obtain
an even faster rate of convergence for
$\sup_{s\in\mathcal{S},\lambda\in\Lambda}\left|\mathbb{G}_{n}\left\\{\bar{\eta}_{h,s,j,\lambda}-\bar{\eta}_{h,s,j,\lambda_{\infty}}\right\\}\right|$
using similar techniques to those above for
$\sup_{s\in\mathcal{S},\lambda\in\Lambda}\left|\mathbb{G}_{n}\left\\{\eta_{h,s,j,\lambda}-\eta_{h,s,j,\lambda_{\infty}}\right\\}\right|$.
However, the integration over $a$ in $\bar{\eta}_{h,s,j,\lambda}$ is
sufficient to obtain a rate of convergence without localizing around
$\lambda_{\infty}$, which provides a sufficient rate for our purposes.

######  Lemma 15.

Suppose $\Lambda$ is a class of functions uniformly bounded by $L<\infty$ and
satisfying $\sup_{Q}\log N\left(\varepsilon,\Lambda,L_{2}(Q)\right)\leq
C\varepsilon^{-V}$ for some $C<\infty$ and $V\in(0,2)$. If (A1) holds, then
$\sup_{a_{0}\in\mathcal{A}_{0},\lambda\in\Lambda}\left|\mathbb{G}_{n}\bar{\eta}_{h,a_{0},j,\lambda}\right|=O_{p}\left(h^{\frac{1-V}{2}}+\left\\{nh^{1+2V}\right\\}^{-1/2}\right)$
for every $j\in\\{1,2,3,4\\}$. Consequently, if (A8) holds as well, then

$\sup_{a_{0}\in\mathcal{A}_{0},\lambda\in\Lambda}\left|\mathbb{G}_{n}\left\\{\int
h^{1/2}\Gamma_{0,h,b,a_{0}}\lambda\,dF_{0}\right\\}\right|=O_{p}\left(h^{\frac{1-V}{2}}+\left\\{nh^{1+2V}\right\\}^{-1/2}\right).$

###### Proof of Lemma 15.

We consider the class of functions
$\bar{\mathcal{H}}_{h,j}:=\\{\bar{\eta}_{h,a_{0},j,\lambda}:a_{0}\in\mathcal{A}_{0},\lambda\in\Lambda\\}$.
We equip this class with the envelope function
$H_{h}=h^{-1/2}L\|K\|_{\infty}$. We also define
$\mathcal{H}_{h,j}:=\\{\eta_{h,a_{0},j,\lambda}:a_{0}\in\mathcal{A}_{0},\lambda\in\Lambda\\}$,
and we note that $H_{h}$ is also an envelope for $\mathcal{H}_{h,j}$. Hence,
by Lemma 5.2 of van der Vaart and van der Laan, (2006) with $s=t=r=2$ and
Lemma 13, we have

$\sup_{Q}\log N\left(\varepsilon
H_{h},\bar{\mathcal{H}}_{h,j},L_{2}(Q)\right)\leq\sup_{Q}\log
N\left(\varepsilon
H_{h}/2,\mathcal{H}_{h,j},L_{2}(Q)\right)\lesssim\varepsilon^{-V}.$

Therefore,

$\displaystyle J(x,\bar{\mathcal{H}}_{h,j},L_{2})$
$\displaystyle:=\sup_{Q}\int_{0}^{x}\sqrt{1+\log
N\left(\varepsilon\bar{H}_{h},\bar{\mathcal{H}}_{h,j},L_{2}(Q)\right)}\,d\varepsilon\lesssim
x^{1-V/2}$

for all $x\in(0,1]$. We also have

$\displaystyle P_{0}\bar{\eta}_{h,a_{0},j,\lambda}^{2}$ $\displaystyle\leq
h^{-1}L^{2}\left\\{\int\left|\frac{a-a_{0}}{h}\right|^{j-1}K\left(\frac{a-a_{0}}{h}\right)\,dF_{0}(a)\right\\}^{2}$
$\displaystyle=hL^{2}\left\\{\int\left|u\right|^{j-1}K\left(u\right)f_{0}(a_{0}+uh)\,du\right\\}^{2}$
$\displaystyle\leq hL^{2}\|f_{0}\|_{\infty}^{2}\|K\|_{\infty}^{2}$

for every $a_{0}\in\mathcal{A}_{0}$ and $\lambda\in\Lambda$. Hence,
$P_{0}\bar{\eta}_{h,a_{0},j,\lambda}^{2}\leq\|f_{0}\|_{\infty}^{2}h^{2}P_{0}H_{h}^{2}$
for every $\bar{\eta}_{h,a_{0},j,\lambda}\in\bar{\mathcal{H}}_{h,j}$. By
Theorem 2.1 of van der Vaart and Wellner, (2011), we then have

$\displaystyle
E_{0}\left[\sup_{\zeta\in\bar{\mathcal{H}}_{h,j}}|\mathbb{G}_{n}\zeta|\right]$
$\displaystyle\lesssim
h^{-1/2}J\left(\|f_{0}\|_{\infty}h,\bar{\mathcal{H}}_{h,j},L_{2}\right)\left\\{1+\frac{J\left(\|f_{0}\|_{\infty}h,\bar{\mathcal{H}}_{h,j},L_{2}\right)}{n^{1/2}\|f_{0}\|_{\infty}^{2}h^{2}}\right\\}$
$\displaystyle\lesssim
h^{\frac{1-V}{2}}\left\\{1+n^{-1/2}h^{-\frac{2+V}{2}}\right\\}$
$\displaystyle=h^{\frac{1-V}{2}}+\left\\{nh^{1+2V}\right\\}^{-1/2}.$

The second claim follows as in the proof of Lemma 14. We can write
$h^{1/2}\Gamma_{0,h,b,a_{0}}\lambda=\sum_{j=1}^{4}C_{h,a_{0},j}\eta_{h,a_{0},j,\lambda}$
for some constants $C_{h,a_{0},j}$ such that
$\sup_{a_{0}\in\mathcal{A}_{0}}|C_{h,a_{0},j}|=O(1)$ by the uniform statements
of Lemma 4. Thus,

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{\int
h^{1/2}\Gamma_{0,h,b,a_{0}}\lambda\,dF_{0}\right\\}\right|$
$\displaystyle\leq\sum_{j=1}^{4}\sup_{a_{0}\in\mathcal{A}_{0}}\left|C_{h,a_{0},j}\right|\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\eta_{h,a_{0},j,\lambda_{n}}\right|\leq
O(1)\sum_{j=1}^{4}\sup_{\zeta\in\bar{\mathcal{H}}_{h,j}}\left|\mathbb{G}_{n}\zeta\right|,$

and the result follows.

∎

######  Lemma 16.

Suppose $\Lambda$ is a class of functions uniformly bounded by $L<\infty$ such
that $\sup_{Q}\log N\left(\varepsilon,\Lambda,L_{2}(Q)\right)\leq
C\varepsilon^{-V}$ for some $C<\infty$ and $V\in(0,2)$. If (A1) holds, then
for each $j\in\\{1,2,3,4\\}$,
$\sup_{a_{0}\in\mathcal{A}_{0},\lambda\in\Lambda}\mathbb{G}_{n}\left|\eta_{h,a_{0},j,\lambda}\right|=O_{p}\left(h^{-1/2}\right)$.

###### Proof of Lemma 16.

Since $L$ is an envelope for $\Lambda$, $h^{-1/2}\|K\|_{\infty}L$ is an
envelope for
$\\{\eta_{h,a_{0},j,\lambda}:a_{0}\in\mathcal{A}_{0},\lambda\in\Lambda\\}$ by
(A1). By Lemma 13, the uniform entropy integral of this class is finite.
Hence, by Theorem 2.14.1 of van der Vaart and Wellner, (1996), we have

$\displaystyle
E_{0}\left[\sup_{a_{0}\in\mathcal{A}_{0},\lambda\in\Lambda}\mathbb{G}_{n}\left|\eta_{h,a_{0},j,\lambda}\right|\right]$
$\displaystyle\lesssim h^{-1/2},$

and the result follows. ∎

######  Corollary 6.

If (A1)–(A3), (A6), (A7)(d), and (A8) hold and $nh^{p}\longrightarrow 0$ for
some $p>0$, then
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|R_{n,h,b,a_{0},2}\right|=o_{p}\left(\\{nh\log
n\\}^{-1/2}\right)$ and
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|R_{n,h,b,a_{0},3}\right|=o_{p}\left(\\{nh\log
n\\}^{-1/2}\right)$.

###### Proof of Corollary 6.

As in the proof of Corollary 4, we can write

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left|\\{nh\\}^{1/2}R_{n,h,b,a_{0},2}\right|$
$\displaystyle\leq\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}\Gamma_{0,h,b,a_{0}}\left(\psi_{n}-\psi_{\infty}\right)\right\\}\right|$
(11)
$\displaystyle\qquad+\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}\Gamma_{0,h,b,a_{0}}\left(\int\mu_{n}\,dQ_{0}-\int\mu_{\infty}\,dQ_{0}\right)\right\\}\right|$
$\displaystyle\qquad+\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{\int
h^{1/2}\Gamma_{0,h,b,a_{0}}\mu_{n}\,dF_{0}\right\\}\right|+\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{\int
h^{1/2}\Gamma_{0,h,b,a_{0}}\mu_{\infty}\,dF_{0}\right\\}\right|.$ (12)

For the first term, we use Lemma 14 with
$\Lambda:=\\{o\mapsto[y-\mu(a,w)]/g(a,w):\mu\in\mathcal{F}_{\mu},g\in\mathcal{F}_{g}\\}$.
By (A3) and (A8), $\Lambda$ is uniformly bounded by some $L<\infty$. By (A3)
and permanence properties of uniform entropy integrals, $\sup_{Q}\log
N(\varepsilon,\Lambda,L_{2}(Q))\leq C\varepsilon^{-V}$ for
$V=\max\\{V_{\mu},V_{g}\\}\in(0,2)$, and by (A6), $n[h/(\log
n)]^{\frac{2+V}{2-V}}\longrightarrow\infty$. Using a similar argument to that
used in the proof of Corollary 4 and using the assumption that $|Y|$ is
bounded almost surely, we can show that

$\displaystyle\rho_{\mathcal{A}_{\delta_{3}}}(\psi_{n},\psi_{\infty})\lesssim
d(g_{n},g_{\infty};\mathcal{A}_{\delta_{3}},\mathcal{A}\times\mathcal{W})+d(\mu_{n},\mu_{\infty};\mathcal{A}_{\delta_{3}},\mathcal{A}\times\mathcal{W}).$

Hence, by (A7)(d),
$\rho_{\mathcal{A}_{\delta_{3}}}(\psi_{n},\psi_{\infty})=o_{p}\left(h^{\frac{V}{2(2-V)}}\\{\log
n\\}^{-\frac{1}{2-V}}\right)$. Thus, the conditions of Lemma 14 hold, and it
follows that
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\\{h^{1/2}\Gamma_{0,h,b,a_{0}}(\psi_{n}-\psi_{\infty})\\}\right|=o_{p}(\\{\log
n\\}^{-1/2})$.

For the second term in equation (12), we also use Lemma 14 with
$\Lambda=\left\\{a\mapsto\int\mu(a,w)\,dQ_{0}(w):\mu\in\mathcal{F}_{\mu}\right\\}$.
This class is uniformly bounded since $\mathcal{F}_{\mu}$ is uniformly bounded
by (A3). By (A3) and Lemma 5.2 of van der Vaart and van der Laan, (2006),
$\sup_{Q}\log
N(\varepsilon,\Lambda,L_{2}(Q))\lesssim\varepsilon^{-V_{\mu}}\lesssim\varepsilon^{-V}$,
and by (A6) $n[h/(\log n)]^{\frac{2+V}{2-V}}\longrightarrow\infty$. By
Jensen’s inequality and (A7)(d),

$\rho_{\mathcal{A}_{\delta_{3}}}\left(\int\mu_{n}\,dQ_{0},\int\mu_{\infty}\,dQ_{0}\right)\leq
d(\mu_{n},\mu_{\infty};\mathcal{A}_{\delta_{3}},\mathcal{A}\times\mathcal{W})=o_{p}\left(h^{\frac{V}{2(2-V)}}\\{\log
n\\}^{-\frac{1}{2-V}}\right).$

The conditions of Lemma 14 are satisfied, and it follows that

$\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}\Gamma_{0,h,b,a_{0}}\left(\int\mu_{n}\,dQ_{0}-\int\mu_{\infty}\,dQ_{0}\right)\right\\}\right|=o_{p}\left(\\{\log
n\\}^{-1/2}\right).$

For the third and fourth terms in equation (12), we use Lemma 15 with
$\Lambda=\mathcal{F}_{\mu}$. The conditions of the lemma are satisfied by
(A3), so that

$\\{\log
n\\}^{1/2}\sup_{a_{0}\in\mathcal{A}_{0},\mu\in\mathcal{F}_{\mu}}\left|\mathbb{G}_{n}\left\\{\int
h^{1/2}\Gamma_{0,h,b,a_{0}}\mu\,dF_{0}\right\\}\right|=O_{p}\left(\\{\log
n\\}^{1/2}h^{\frac{1-V_{\mu}}{2}}+\left\\{nh^{1+2V_{\mu}}/\log
n\right\\}^{-1/2}\right).$

By (A6), $nh^{3}\longrightarrow\infty$, which implies that
$nh^{1+2V_{\mu}}/\log n\longrightarrow\infty$ since $V_{\mu}\in(0,1)$. By
assumption, $nh^{p}\longrightarrow 0$ for some $p>0$, and since
$V_{\mu}\in(0,1)$, this implies that $\\{\log
n\\}^{1/2}h^{\frac{1-V_{\mu}}{2}}=o(1)$ as well. Hence,

$\sup_{a_{0}\in\mathcal{A}_{0},\mu\in\mathcal{F}_{\mu}}\left|\mathbb{G}_{n}\left\\{\int
h^{1/2}\Gamma_{0,h,b,a_{0}}\mu\,dF_{0}\right\\}\right|=o_{p}\left(\\{\log
n\\}^{-1/2}\right).$

We have now shown that every term of equation (12) is $o_{p}\left(\\{\log
n\\}^{-1/2}\right)$, so we conclude that
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|R_{n,h,b,a_{0},2}\right|=o_{p}\left(\left\\{nh\log
n\right\\}^{-1/2}\right)$.

We can similarly decompose $R_{n,h,b,a_{0},3}$ as

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left|(nh)^{1/2}R_{n,h,b,a_{0},3}\right|$
$\displaystyle\leq\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\psi_{n}\right\\}\right|$
$\displaystyle\qquad+\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\int\mu_{n}\,dQ_{0}\right\\}\right|$
$\displaystyle\qquad+\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{\int
h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\mu_{n}\,dF_{0}\right\\}\right|.$
(13)

For the first term in equation (13), we note that

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\psi_{n}\right\\}\right|$
$\displaystyle=\sup_{a_{0}\in\mathcal{A}_{0}}\left|e_{1}^{T}\left(\mathbf{D}_{n,h,a_{0},1}^{-1}-\mathbf{D}_{0,h,a_{0},1}^{-1}\right)\mathbb{G}_{n}\left\\{h^{1/2}w_{h,a_{0},1}K_{h,a_{0}}\psi_{n}\right\\}\right.$
$\displaystyle\qquad\left.-e_{3}^{T}c_{2}(h/b)^{2}\left(\mathbf{D}_{n,b,a_{0},3}^{-1}-\mathbf{D}_{0,b,a_{0},3}^{-1}\right)\mathbb{G}_{n}\left\\{h^{1/2}w_{b,a_{0},3}K_{b,a_{0}}\psi_{n}\right\\}\right|$
$\displaystyle\lesssim\sup_{a_{0}\in\mathcal{A}_{0}}\left\|\mathbf{D}_{n,h,a_{0},1}^{-1}-\mathbf{D}_{0,h,a_{0},1}^{-1}\right\|_{\infty}\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}w_{h,a_{0},1}\mathbf{1}^{T}K_{h,a_{0}}\psi_{n}\right\\}\right|$
$\displaystyle\qquad+\sup_{a_{0}\in\mathcal{A}_{0}}\left\|\mathbf{D}_{n,b,a_{0},3}^{-1}-\mathbf{D}_{0,b,a_{0},3}^{-1}\right\|_{\infty}\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}w_{b,a_{0},3}\mathbf{1}^{T}K_{b,a_{0}}\psi_{n}\right\\}\right|.$

By (A6), $nh^{3}\longrightarrow\infty$, which implies that $nh/\log
h^{-1}\longrightarrow\infty$. Hence, by Lemma 12,
$\sup_{a_{0}\in\mathcal{A}_{0}}\left\|\mathbf{D}_{n,h,a_{0},1}^{-1}-\mathbf{D}_{0,h,a_{0},1}^{-1}\right\|_{\infty}$
and
$\sup_{a_{0}\in\mathcal{A}_{0}}\left\|\mathbf{D}_{n,b,a_{0},3}^{-1}-\mathbf{D}_{0,b,a_{0},3}^{-1}\right\|_{\infty}$
are both $O_{p}(\\{nh/\log h^{-1}\\}^{-1/2})$. In addition, by Lemma 16, we
have that
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}w_{h,a_{0},1}^{T}\mathbf{1}K_{h,a_{0}}\psi_{n}\right\\}\right|$
and
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}w_{b,a_{0},3}^{T}\mathbf{1}K_{b,a_{0}}\psi_{n}\right\\}\right|$
are both $O_{p}(h^{-1/2})$. We then have

$\\{\log
n\\}^{1/2}\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\psi_{n}\right\\}\right|=O_{p}\left(\left\\{\frac{nh^{2}}{\log
h^{-1}\log n}\right\\}^{-1/2}\right).$

Since $nh^{3}\longrightarrow\infty$ and $nh^{p}\longrightarrow 0$ for some
$p>0$, $\frac{nh^{2}}{\log h^{-1}\log n}\longrightarrow\infty$. The second
term in equation (13) can be addressed in the same way using Lemma 5.2 of van
der Vaart and van der Laan, (2006).

We can similarly bound the third term in equation (13) up to a constant by

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left\|\mathbf{D}_{n,h,a_{0},1}^{-1}-\mathbf{D}_{0,h,a_{0},1}^{-1}\right\|_{\infty}\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{\int
h^{1/2}w_{h,a_{0},1}^{T}\mathbf{1}K_{h,a_{0}}\mu_{n}\,dF_{0}\right\\}\right|$
$\displaystyle\qquad+\sup_{a_{0}\in\mathcal{A}_{0}}\left\|\mathbf{D}_{n,b,a_{0},3}^{-1}-\mathbf{D}_{0,b,a_{0},3}^{-1}\right\|_{\infty}\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{\int
h^{1/2}w_{b,a_{0},3}^{T}\mathbf{1}K_{b,a_{0}}\mu_{n}\,dF_{0}\right\\}\right|.$

By Lemmas 12 and 15, we then have

$\displaystyle\\{\log
n\\}^{1/2}\sup_{a_{0}\in\mathcal{A}_{0}}\left|\mathbb{G}_{n}\left\\{\int
h^{1/2}\left(\Gamma_{n,h,b,a_{0}}-\Gamma_{0,h,b,a_{0}}\right)\mu_{n}\,dF_{0}\right\\}\right|$
$\displaystyle\qquad=O_{p}\left(\left\\{\frac{nh}{\log h^{-1}\log
n}\right\\}^{-1/2}\left\\{h^{(1-V_{\mu})/2}+\left(nh^{1+2V_{\mu}}\right)^{-1/2}\right\\}\right)$
$\displaystyle\qquad=O_{p}\left(\left\\{\frac{nh^{V_{\mu}}}{\log h^{-1}\log
n}\right\\}^{-1/2}+\left\\{\frac{nh^{1+V_{\mu}}}{\left(\log h^{-1}\log
n\right)^{1/2}}\right\\}^{-1}\right).$

By (A6), both terms are $o(1)$. ∎

##  Supplement I Analysis of remainder term $R_{n,h,b,a_{0},4}$

######  Lemma 17.

If (A1)–(A4) hold, then $R_{n,h,b,a_{0},4}=o_{p}(\\{nh\\}^{-1/2})$. If
(A1)–(A3), (A7), and (A8) hold, then
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|R_{n,h,b,a_{0},4}\right|=o_{p}\left(\\{nh\log
n\\}^{-1/2}\right)$.

###### Proof of Lemma 17.

We note that $\Gamma_{n,h,b,a_{0}}(a)=0$ for all $a$ such that
$|a-a_{0}|>\max\\{h,b\\}$ by (A1). Hence, for $\max\\{h,b\\}\leq\delta_{1}$,
$\Gamma_{n,h,b,a_{0}}(a)=0$ for $a\notin B_{\delta_{1}}(a_{0})$. Therefore,
using (A4), for all $\max\\{h,b\\}\leq\delta_{1}$, we can write

$\displaystyle\left|\mathrel{R}_{n,h,b,a_{0},4}\right|$
$\displaystyle=\left|\iint_{B_{\delta_{1}}(a_{0})\times\mathcal{W}}\Gamma_{n,h,b,a_{0}}(a)\left\\{\mu_{n}(a,w)-\mu_{0}(a,w)\right\\}\left\\{1-\frac{g_{0}(a,w)}{g_{n}(a,w)}\right\\}\,dF_{0}(a)\,dQ_{0}(w)\right|$
$\displaystyle=\left|\sum_{j=1}^{3}\iint_{\mathcal{S}_{j}}\Gamma_{n,h,b,a_{0}}(a)\left\\{\mu_{n}(a,w)-\mu_{0}(a,w)\right\\}\left\\{1-\frac{g_{0}(a,w)}{g_{n}(a,w)}\right\\}\,dF_{0}(a)\,dQ_{0}(w)\right|$
$\displaystyle\leq\sum_{j=1}^{3}\iint_{\mathcal{S}_{j}}\left|\Gamma_{n,h,b,a_{0}}(a)\right|\left|\mu_{n}(a,w)-\mu_{0}(a,w)\right|\left|1-\frac{g_{0}(a,w)}{g_{n}(a,w)}\right|\,dF_{0}(a)\,dQ_{0}(w).$

Now we note that for any uniformly bounded function
$\lambda:\mathcal{A}\times\mathcal{W}\to\mathbb{R}$ and
$\mathcal{S}\subset\mathcal{A}\times\mathcal{W}$, we have

$\displaystyle\iint_{S}\left|w_{h,a_{0},j}(a)K_{h,a_{0}}(a)\lambda(a,w)\right|\,dF_{0}(a)\,dQ_{0}(w)$
$\displaystyle\qquad=\int_{-1}^{1}\left|(1,u,\dotsc,u^{j})^{T}\right|K(u)\int
I_{\mathcal{S}}(a_{0}+uh,w)\left|\lambda(a_{0}+uh,w)\right|\,dQ_{0}(w)f_{0}(a_{0}+uh)\,du$
$\displaystyle\qquad\lesssim\sup_{|a-a_{0}|\leq h}\int
I_{\mathcal{S}}(a,w)\left|\lambda(a,w)\right|\,dQ_{0}(w).$

Hence, for all $n$ large enough, we have

$\displaystyle\iint_{\mathcal{S}_{j}}\left|\Gamma_{n,h,b,a_{0}}(a)\right|\left|\mu_{n}(a,w)-\mu_{0}(a,w)\right|\left|1-\frac{g_{0}(a,w)}{g_{n}(a,w)}\right|\,dF_{0}(a)\,dQ_{0}(w)$
$\displaystyle\qquad\lesssim\left[\left\|\mathbf{D}_{n,h,a_{0},1}^{-1}\right\|_{\infty}+\left\|\mathbf{D}_{n,h,a_{0},3}^{-1}\right\|_{\infty}\right]\sup_{|a-a_{0}|<\delta_{1}}E_{0}\left\\{I_{\mathcal{S}_{j}}(a,W)\left|\mu_{n}(a,w)-\mu_{0}(a,w)\right|\left|1-\frac{g_{0}(a,W)}{g_{n}(a,W)}\right|\right\\}$
$\displaystyle\qquad\lesssim
O_{p}(1)d\left(\mu_{n},\mu_{0};B_{\delta_{1}}(a_{0}),\mathcal{S}_{j}\right)d\left(g_{n},g_{0};B_{\delta_{1}}(a_{0}),\mathcal{S}_{j}\right),$

where for the last inequality we used Lemma 8, the Cauchy-Schwartz inequality,
and the uniform boundedness of $1/g_{n}$ guaranteed by (A3). We now use (A4)
to see that

$\displaystyle
d\left(\mu_{n},\mu_{0};B_{\delta_{1}}(a_{0}),\mathcal{S}_{1}\right)$
$\displaystyle=o_{p}\left(\\{nh\\}^{-1/2}\right),$ $\displaystyle
d\left(g_{n},g_{0};B_{\delta_{1}}(a_{0}),\mathcal{S}_{2}\right)$
$\displaystyle=o_{p}\left(\\{nh\\}^{-1/2}\right),\text{ and}$ $\displaystyle
d\left(\mu_{n},\mu_{0};B_{\delta_{1}}(a_{0}),\mathcal{S}_{3}\right)d\left(g_{n},g_{0};B_{\delta_{1}}(a_{0}),\mathcal{S}_{3}\right)$
$\displaystyle=o_{p}\left(\\{nh\\}^{-1/2}\right).$

Thus, $R_{n,h,b,a_{0},4}=o_{p}\left(\\{nh\\}^{-1/2}\right)$.

For the uniform statement, we note that $\Gamma_{n,h,b,a_{0}}(a)=0$ for all
$a_{0}\in\mathcal{A}_{0}$, $a\notin\mathcal{A}_{\varepsilon_{3}}$, and
$\max\\{h,b\\}<\varepsilon_{3}$. Hence, for $\max\\{h,b\\}<\varepsilon_{3}$,
we can write

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left|R_{n,h,b,a_{0},4}\right|$
$\displaystyle=\sup_{a_{0}\in\mathcal{A}_{0}}\left|\iint_{\mathcal{A}_{3}\times\mathcal{W}}\Gamma_{n,h,b,a_{0}}(a)\left\\{\mu_{n}(a,w)-\mu_{0}(a,w)\right\\}\left\\{1-\frac{g_{0}(a,w)}{g_{n}(a,w)}\right\\}\,dF_{0}(a)\,dQ_{0}(w)\right|$
$\displaystyle\leq\sum_{j=1}^{3}\sup_{a_{0}\in\mathcal{A}_{0}}\iint_{\mathcal{S}_{j}^{\prime}}\left|\Gamma_{n,h,b,a_{0}}(a)\right|\left|\mu_{n}(a,w)-\mu_{0}(a,w)\right|\left|1-\frac{g_{0}(a,w)}{g_{n}(a,w)}\right|\,dF_{0}(a)\,dQ_{0}(w).$

Using the derivations above, we then have

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\iint_{\mathcal{S}_{j}^{\prime}}\left|\Gamma_{n,h,b,a_{0}}(a)\right|\left|\mu_{n}(a,w)-\mu_{0}(a,w)\right|\left|1-\frac{g_{0}(a,w)}{g_{n}(a,w)}\right|\,dF_{0}(a)\,dQ_{0}(w)$
$\displaystyle\qquad\lesssim\sup_{a_{0}\in\mathcal{A}_{0}}\left[\left\|\mathbf{D}_{n,h,a_{0},1}^{-1}\right\|_{\infty}+\left\|\mathbf{D}_{n,h,a_{0},3}^{-1}\right\|_{\infty}\right]\sup_{a_{0}\in\mathcal{A}_{\varepsilon_{3}}}E_{0}\left\\{I_{\mathcal{S}_{j}^{\prime}}(a,W)\left|\mu_{n}(a,w)-\mu_{0}(a,w)\right|\left|1-\frac{g_{0}(a,W)}{g_{n}(a,W)}\right|\right\\}$
$\displaystyle\qquad\lesssim
O_{p}(1)d\left(\mu_{n},\mu_{0};\mathcal{A}_{\varepsilon_{3}},\mathcal{S}_{j}^{\prime}\right)d\left(g_{n},g_{0};\mathcal{A}_{\varepsilon_{3}},\mathcal{S}_{j}^{\prime}\right),$

where the last inequality uses Lemma 12 and the uniform boundedness of
$1/g_{n}$ guaranteed by (A3). We now use the faster rates guaranteed by (A7)
to see that

$\displaystyle
d(\mu_{n},\mu_{0};\mathcal{A}_{\varepsilon_{3}},\mathcal{S}_{1}^{\prime})$
$\displaystyle=o_{p}\left(\\{nh\log n\\}^{-1/2}\right),$ $\displaystyle
d(g_{n},g_{0};\mathcal{A}_{\varepsilon_{3}},\mathcal{S}_{2}^{\prime})$
$\displaystyle=o_{p}\left(\\{nh\log n\\}^{-1/2}\right),\text{ and}$
$\displaystyle
d(\mu_{n},\mu_{0};\mathcal{A}_{\varepsilon_{3}},\mathcal{S}_{3}^{\prime})d_{2}(g_{n},g_{0};\mathcal{A}_{\varepsilon_{3}},\mathcal{S}_{3}^{\prime})$
$\displaystyle=o_{p}\left(\\{nh\log n\\}^{-1/2}\right).$

Thus,
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|R_{n,h,b,a_{0},4}\right|=o_{p}\left(\\{nh\log
n\\}^{-1/2}\right)$. ∎

##  Supplement J Analysis of remainder term $R_{n,h,b,a_{0},5}$

######  Lemma 18.

Suppose $X_{1},X_{2},\dots$ is a sequence of IID random variables on sample
space $\mathcal{X}$ with marginal distribution $P_{0}$. Let $\mathbb{P}_{n}$
be the empirical distribution corresponding to $(X_{1},\dotsc,X_{n})$, and
$\mathcal{F}$ be a collection of measurable functions from
$\mathcal{X}\times\mathcal{X}$ to $\mathbb{R}$ with envelope function $F$.
Then

$\displaystyle\sup_{f\in\mathcal{F}}\left|\iint
f(x_{1},x_{2})\,d(\mathbb{P}_{n}-P_{0})(x_{1})\,d(\mathbb{P}_{n}-P_{0})(x_{2})\right|$
$\displaystyle\lesssim n^{-1}\left\|F\right\|_{P_{0}\times
P_{0},2}\int_{0}^{1}\left[1+\log\sup_{Q}N(\varepsilon\|F\|_{Q,2},\mathcal{F},L_{2}(Q))\right]\,d\varepsilon$
$\displaystyle\quad+n^{-3/2}\|F\|_{P_{0}\times
P_{0},2}\int_{0}^{1}\left[1+\log\sup_{Q}N\left(\varepsilon\|F\|_{Q,2}/2,\mathcal{F},L_{2}(Q)\right)\right]^{1/2}\,d\varepsilon.$

###### Proof of Lemma 18.

For each $f\in\mathcal{F}$, define the symmetrized, centered version of $f$ as

$\displaystyle f^{\circ}(x_{1},x_{2})$
$\displaystyle:=f(x_{1},x_{2})+f(x_{2},x_{1})-\int\left[f(u,x_{1})+f(u,x_{2})+f(x_{1},u)+f(x_{2},u)\right]\,dP_{0}(u)+2\iint
f(u,v)\,dP_{0}(u)\,dP_{0}(v).$

We note that $f^{\circ}$ is symmetric in its arguments, meaning
$f^{\circ}(x_{1},x_{2})=f^{\circ}(x_{2},x_{1})$ for all
$x_{1},x_{2}\in\mathcal{X}$, and $\int f(x_{1},u)\,dP_{0}(u)=0$ for all
$x_{1}\in\mathcal{X}$. We let
$\mathcal{F}^{\circ}:=\\{f^{\circ}:f\in\mathcal{F}\\}$, and we note that an
envelope function for $\mathcal{F}^{\circ}$ is $F^{\circ}$ for

$\displaystyle F^{\circ}(x_{1},x_{2})$
$\displaystyle:=F(x_{1},x_{2})+F(x_{2},x_{1})+\int\left[F(u,x_{1})+F(u,x_{2})+F(x_{1},u)+F(x_{2},u)\right]\,dP_{0}(u)+2\iint
F(u,v)\,dP_{0}(u)\,dP_{0}(v).$

By adding and subtracting terms, we can write

$\displaystyle\iint
f(x_{1},x_{2})\,d(\mathbb{P}_{n}-P_{0})(x_{1})\,d(\mathbb{P}_{n}-P_{0})(x_{2})$
$\displaystyle=\frac{1}{2n^{2}}\sum_{\begin{subarray}{c}i,j=1\\\ i\neq
j\end{subarray}}^{n}f^{\circ}(X_{i},X_{j})-\frac{1}{n}\iint
f(x_{1},x_{2})\,dP_{0}(x_{1})\,d(\mathbb{P}_{n}-P_{0})(x_{2})$
$\displaystyle\qquad-\frac{1}{n}\iint
f(x_{1},x_{2})\,d(\mathbb{P}_{n}-P_{0})(x_{1})\,dP_{0}(x_{2})$
$\displaystyle\qquad-\frac{1}{n}\iint
f(x_{1},x_{2})\,dP_{0}(x_{1})\,dP_{0}(x_{2})$

By Lemma 3 of Westling et al., (2020), we have

$\displaystyle
E_{0}\left[\frac{1}{2n^{2}}\sup_{f\in\mathcal{F}^{\circ}}\left|\sum_{\begin{subarray}{c}i,j=1\\\
i\neq
j\end{subarray}}^{n}f^{\circ}(X_{i},X_{j})\right|\right]\lesssim\frac{1}{n}\left\|F^{\circ}\right\|_{P_{0}\times
P_{0},2}\int_{0}^{1}\left[1+\log\sup_{Q}N(\varepsilon\|F^{\circ}\|_{Q,2},\mathcal{F}^{\circ},L_{2}(Q))\right]\,d\varepsilon.$

By the triangle inequality and Jensen’s inequality,
$\left\|F^{\circ}\right\|_{P_{0}\times P_{0},2}\leq
8\left\|F\right\|_{P_{0}\times P_{0},2}$. We also note that by definition of
$f^{\circ}$, $\mathcal{F}^{\circ}$ is contained in the sum of $\mathcal{F}$
and the following classes:

$\displaystyle\mathcal{F}_{r}$ $\displaystyle:=\left\\{(x_{1},x_{2})\mapsto
f(x_{2},x_{1}):f\in\mathcal{F}\right\\},$
$\displaystyle\bar{\mathcal{F}}_{11}$
$\displaystyle:=\left\\{x_{1}\mapsto-\int
f(x_{1},u)\,dP_{0}(u):f\in\mathcal{F}\right\\},$
$\displaystyle\bar{\mathcal{F}}_{12}$
$\displaystyle:=\left\\{x_{1}\mapsto-\int
f(u,x_{1})\,dP_{0}(u):f\in\mathcal{F}\right\\},$
$\displaystyle\bar{\mathcal{F}}_{21}$
$\displaystyle:=\left\\{x_{2}\mapsto-\int
f(x_{2},u)\,dP_{0}(u):f\in\mathcal{F}\right\\},$
$\displaystyle\bar{\mathcal{F}}_{22}$
$\displaystyle:=\left\\{x_{2}\mapsto-\int
f(u,x_{2})\,dP_{0}(u):f\in\mathcal{F}\right\\},$ $\displaystyle\text{and
}\mathcal{F}_{m}$ $\displaystyle:=\left\\{2\iint
f(u,v)\,dP_{0}(u)\,dP_{0}(v):f\in\mathcal{F}\right\\}.$

By Lemma 5.1 of van der Vaart and van der Laan, (2006),
$\log\sup_{Q}N(\varepsilon\|F^{\circ}\|_{Q,2},\mathcal{F}^{\circ},L_{2}(Q))$
is bounded by the sum of the uniform entropies of each of the above classes.
The uniform entropy of $\mathcal{F}_{r}$ is the same as that of $\mathcal{F}$
because for any measure $Q$ on $\mathcal{X}\times\mathcal{X}$,

$\int[f_{1}(x_{1},x_{2})-f_{2}(x_{1},x_{2})]^{2}\,dQ(x_{1},x_{2})=\int[f_{1}(x_{2},x_{1})-f_{2}(x_{2},x_{1})]^{2}\,dQ_{r}(x_{1},x_{2}),$

where $Q_{r}$ is defined as $dQ_{r}(x_{1},x_{2}):=dQ(x_{2},x_{1})$. We equip
$\bar{\mathcal{F}}_{11}$ with the envelope function

$\bar{F}_{11}:x_{1}\mapsto\left[\int F(x_{1},u)^{2}\,dP_{0}(u)\right]^{1/2}.$

We note that $\|\bar{F}_{11}\|_{P_{0},2}=\|F\|_{P_{0}\times P_{0},2}$. By
Lemma 5.2 of van der Vaart and van der Laan, (2006) (with $r=s=t=2$), we have

$\sup_{Q}N\left(\varepsilon\|\bar{F}_{11}\|_{Q,2},\bar{\mathcal{F}}_{11},L_{2}(Q)\right)\leq\sup_{Q}N\left(\varepsilon\|F\|_{Q,2}/2,\mathcal{F},L_{2}(Q)\right).$

Identical results hold for $\bar{F}_{12}$, $\bar{F}_{21}$, and $\bar{F}_{22}$.
For $\mathcal{F}_{m}$ equipped with envelope $\|F\|_{P_{0}\times P_{0},2}$, we
have

$\sup_{Q}N\left(\varepsilon\|F\|_{P_{0}\times
P_{0},2},\mathcal{F}_{m},L_{2}(Q)\right)\leq\sup_{Q}N\left(\varepsilon\|F\|_{P_{0}\times
P_{0},2},\mathcal{F},L_{2}(Q)\right)$

since

$\left\|\int f_{1}\,d(P_{0}\times P_{0})-\int f_{2}\,d(P_{0}\times
P_{0})\right\|_{Q,2}=\left|\int f_{1}\,d(P_{0}\times P_{0})-\int
f_{2}\,d(P_{0}\times P_{0})\right|\leq\|f_{1}-f_{2}\|_{P_{0}\times P_{0},2}$

for any probability measure $Q$. Therefore, we have

$\displaystyle\left\|F^{\circ}\right\|_{P_{0}\times
P_{0},2}\int_{0}^{1}\left[1+\log\sup_{Q}N(\varepsilon\|F^{\circ}\|_{Q,2},\mathcal{F}^{\circ},L_{2}(Q))\right]\,d\varepsilon\lesssim\left\|F\right\|_{P_{0}\times
P_{0},2}\int_{0}^{1}\left[1+\log\sup_{Q}N(\varepsilon\|F\|_{Q,2},\mathcal{F},L_{2}(Q))\right]\,d\varepsilon$

Next, we have

$\displaystyle E_{0}\left[\sup_{f\in\mathcal{F}}n^{-1}\left|\iint
f(x_{1},x_{2})\,dP_{0}(x_{1})\,d(\mathbb{P}_{n}-P_{0})(x_{2})\right|\right]$
$\displaystyle=n^{-3/2}E_{0}\left[\sup_{\bar{f}\in\bar{\mathcal{F}}_{22}}\left|\mathbb{G}_{n}\bar{f}\right|\right].$

By Theorem 2.14.1 of van der Vaart and Wellner, (1996),

$\displaystyle
E_{0}\left[\sup_{\bar{f}\in\bar{\mathcal{F}}_{22}}\left|\mathbb{G}_{n}\bar{f}\right|\right]$
$\displaystyle\lesssim\|\bar{F}_{22}\|_{P_{0},2}\int_{0}^{1}\left[1+\log\sup_{Q}N\left(\varepsilon\|\bar{F}_{22}\|_{Q,2},\bar{\mathcal{F}}_{22},L_{2}(Q)\right)\right]^{1/2}\,d\varepsilon.$

Hence,

$\displaystyle E_{0}\left[\sup_{f\in\mathcal{F}}n^{-1}\left|\iint
f(x_{1},x_{2})\,dP_{0}(x_{1})\,d(\mathbb{P}_{n}-P_{0})(x_{2})\right|\right]\lesssim
n^{-3/2}\|F\|_{P_{0}\times
P_{0},2}\int_{0}^{1}\left[1+\log\sup_{Q}N\left(\varepsilon\|F\|_{Q,2}/2,\mathcal{F},L_{2}(Q)\right)\right]^{1/2}\,d\varepsilon.$

By an identical argument, we also have

$\displaystyle E_{0}\left[\sup_{f\in\mathcal{F}}n^{-1}\left|\iint
f(x_{1},x_{2})\,d(\mathbb{P}_{n}-P_{0})(x_{1})\,dP_{0}(x_{2})\right|\right]\lesssim
n^{-3/2}\|F\|_{P_{0}\times
P_{0},2}\int_{0}^{1}\left[1+\log\sup_{Q}N\left(\varepsilon\|F\|_{Q,2}/2,\mathcal{F},L_{2}(Q)\right)\right]^{1/2}\,d\varepsilon.$

Finally,

$\sup_{f\in\mathcal{F}}n^{-1}\left|\iint
f(x_{1},x_{2})\,dP_{0}(x_{1})\,dP_{0}(x_{2})\right|\leq
n^{-1}\|F\|_{P_{0}\times P_{0},2}.$

Putting together the pieces yields the result. ∎

######  Lemma 19.

If (A1)–(A3), and (A5) hold, then
$\mathrel{R}_{n,h,b,a_{0},5}=O_{p}(\\{nh^{1/2}\\}^{-1})$. If (A1), (A3), and
(A8) hold and $nh/\log h^{-1}\longrightarrow\infty$, then
$\sup_{a_{0}\in\mathcal{A}_{0}}|R_{n,h,b,a_{0},5}|=O_{p}(\\{nh\\}^{-1})$.

###### Proof of Lemma 19.

For the pointwise statement, we write

$\displaystyle\mathrel{R}_{n,h,b,a_{0},5}$
$\displaystyle=\iint\Gamma_{n,h,b,a_{0}}(a)\mu_{n}(a,w)\,d(Q_{n}-Q_{0})(w)\,d(F_{n}-F_{0})(a)$
$\displaystyle=e_{1}^{T}\mathbf{D}_{n,h,a_{0},1}^{-1}\iint
w_{h,a_{0},1}(a)K_{h,a_{0}}(a)\mu_{n}(a,w)\,d(Q_{n}-Q_{0})(w)\,d(F_{n}-F_{0})(a)$
$\displaystyle\qquad-
e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{n,b,a_{0},3}^{-1}\iint
w_{b,a_{0},3}(a)K_{b,a_{0}}(a)\mu_{n}(a,w)\,d(Q_{n}-Q_{0})(w)\,d(F_{n}-F_{0})(a).$

By Lemmas 4 and 8, $\left\|\mathbf{D}_{n,h,a_{0},1}^{-1}\right\|_{\infty}$ and
$\left\|\mathbf{D}_{n,h,a_{0},3}^{-1}\right\|_{\infty}$ are both $O_{p}(1)$.
Hence, there exist constants $C_{n,j}=O_{p}(1)$ such that

$\displaystyle\left|R_{n,h,b,a_{0},5}\right|$
$\displaystyle\leq\sum_{j=0}^{4}C_{n,j}\left|\iint[(a-a_{0})/h]^{j}K_{h,a_{0}}(a)\mu_{n}(a,w)\,d(Q_{n}-Q_{0})(w)\,d(F_{n}-F_{0})(a)\right|.$

For each $j$, we then write

$\displaystyle\left|\iint[(a-a_{0})/h]^{j}K_{h,a_{0}}(a)\mu_{n}(a,w)\,d(Q_{n}-Q_{0})(w)\,d(F_{n}-F_{0})(a)\right|$
$\displaystyle\qquad=\left|\iint
f_{n,h,a_{0},j}(a_{1},w_{1},a_{2},w_{2})\,d(\mathbb{P}_{n}-P_{0})(a_{1},w_{1})\,d(\mathbb{P}_{n}-P_{0})(a_{2},w_{2})\right|$
$\displaystyle\qquad\leq\sup_{f\in\mathcal{F}_{h,a_{0},j}}\left|\iint
f(a_{1},w_{1},a_{2},w_{2})\,d(\mathbb{P}_{n}-P_{0})(a_{1},w_{1})\,d(\mathbb{P}_{n}-P_{0})(a_{2},w_{2})\right|,$

where

$\displaystyle f_{n,h,a_{0},j}(a_{1},w_{1},a_{2},w_{2})$
$\displaystyle:=[(a_{1}-a_{0})/h]^{j}K_{h,a_{0}}(a_{1})\mu_{n}(a_{1},w_{2})\text{
and}$ $\displaystyle\mathcal{F}_{h,a_{0},j}$
$\displaystyle=\left\\{(a_{1},w_{1},a_{2},w_{2})\mapsto[(a_{1}-a_{0})/h]^{j}K_{h,a_{0}}(a_{1})\mu(a_{1},w_{2}):\mu\in\mathcal{F}_{\mu}\right\\}.$

By (A1) and (A3), an envelope function $F_{h,a_{0}}$ for
$\mathcal{F}_{h,a_{0},j}$ is given by a constant times $K_{h,a_{0}}(a_{1})$,
and $\|F_{h,a_{0}}\|_{(P_{0}\times P_{0}),2}\lesssim h^{-1/2}$ by the standard
change of variables. In addition, by (A3),

$\sup_{Q}\log N\left(\varepsilon\|F_{h,a_{0}}\|_{(P_{0}\times
P_{0}),2},\mathcal{F}_{h,a_{0},j},L_{2}(Q)\right)\lesssim\varepsilon^{-V_{\mu}},$

where $V_{\mu}\in(0,1)$. Hence, by Lemma 18,

$\displaystyle E_{0}\left[\sup_{f\in\mathcal{F}_{h,a_{0},j}}\left|\iint
f(a_{1},w_{1},a_{2},w_{2})\,d(\mathbb{P}_{n}-P_{0})(a_{1},w_{1})\,d(\mathbb{P}_{n}-P_{0})(a_{2},w_{2})\right|\right]\lesssim
n^{-1}h^{-1/2}$

for each $j$, and the pointwise result follows.

For the uniform result, by Lemmas 4 and 12,
$\sup_{a_{0}\in\mathcal{A}_{0}}\left\|\mathbf{D}_{n,h,a_{0},1}^{-1}\right\|_{\infty}$
and
$\sup_{a_{0}\in\mathcal{A}_{0}}\left\|\mathbf{D}_{n,h,a_{0},3}^{-1}\right\|_{\infty}$
are both $O_{p}(1)$. Hence, there exist constants $C_{n,j}^{\prime}=O_{p}(1)$
such that

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left|R_{n,h,b,a_{0},5}\right|$
$\displaystyle\leq\sum_{j=0}^{4}C_{n,j}^{\prime}\sup_{a_{0}\in\mathcal{A}_{0}}\left|\iint[(a-a_{0})/h]^{j}K_{h,a_{0}}(a)\mu_{n}(a,w)\,d(Q_{n}-Q_{0})(w)\,d(F_{n}-F_{0})(a)\right|.$

As above, for each $j$ we then write

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left|\iint[(a-a_{0})/h]^{j}K_{h,a_{0}}(a)\mu_{n}(a,w)\,d(Q_{n}-Q_{0})(w)\,d(F_{n}-F_{0})(a)\right|$
$\displaystyle\qquad=\sup_{a_{0}\in\mathcal{A}_{0}}\left|\iint
f_{n,h,a_{0},j}(a_{1},w_{1},a_{2},w_{2})\,d(\mathbb{P}_{n}-P_{0})(a_{1},w_{1})\,d(\mathbb{P}_{n}-P_{0})(a_{2},w_{2})\right|$
$\displaystyle\qquad\leq\sup_{f\in\mathcal{F}_{h,j}}\left|\iint
f(a_{1},w_{1},a_{2},w_{2})\,d(\mathbb{P}_{n}-P_{0})(a_{1},w_{1})\,d(\mathbb{P}_{n}-P_{0})(a_{2},w_{2})\right|,$

where

$\displaystyle\mathcal{F}_{h,j}$
$\displaystyle=\left\\{(a_{1},w_{1},a_{2},w_{2})\mapsto[(a_{1}-a_{0})/h]^{j}K_{h,a_{0}}(a_{1})\mu(a_{1},w_{2}):\mu\in\mathcal{F}_{\mu},a_{0}\in\mathcal{A}_{0}\right\\}.$

By (A1) and (A3), $\mathcal{F}_{h,j}$ is uniformly bounded up to a constant by
$h^{-1}$, and by Lemma 13 and (A3), $\mathcal{F}_{h,j}$ has uniform entropy
bounded up to a constant by $\varepsilon^{-V_{\mu}}$ relative to this
envelope. Thus, by Lemma 18,

$\displaystyle E_{0}\left[\sup_{f\in\mathcal{F}_{h,j}}\left|\iint
f(a_{1},w_{1},a_{2},w_{2})\,d(\mathbb{P}_{n}-P_{0})(a_{1},w_{1})\,d(\mathbb{P}_{n}-P_{0})(a_{2},w_{2})\right|\right]\lesssim(nh)^{-1}$

for each $j$, and the result follows. ∎

##  Supplement K Analysis of remainder term $R_{n,h,b,a_{0},6}$

######  Lemma 20.

If (A1), (A2), and (A5) hold, then $|R_{n,h,b,a_{0},6}|=O_{p}(\\{nh\\}^{-1})$.
If (A1), (A2), and (A8) hold and $nh/\log h^{-1}\longrightarrow\infty$, then
$\sup_{a_{0}\in\mathcal{A}_{0}}|R_{n,h,b,a_{0},6}|=O_{p}(\\{nh/\log
h^{-1}\\}^{-1})$.

###### Proof of Lemma 20.

We have

$\displaystyle\left|R_{n,h,b,a_{0},6}\right|$
$\displaystyle\lesssim\left\|\mathbf{D}^{-1}_{0,h,a_{0},1}\right\|_{\infty}\left\|\mathbf{D}_{0,h,a_{0},1}-\mathbf{D}_{n,h,a_{0},1}\right\|_{\infty}\left\|\mathbf{D}^{-1}_{n,h,a_{0},1}-\mathbf{D}^{-1}_{0,h,a_{0},1}\right\|_{\infty}\left|P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)\right|$
$\displaystyle\qquad+\left\|\mathbf{D}^{-1}_{0,b,a_{0},3}\right\|_{\infty}\left\|\mathbf{D}_{0,b,a_{0},3}-\mathbf{D}_{n,b,a_{0},3}\right\|_{\infty}\left\|\mathbf{D}^{-1}_{n,b,a_{0},3}-\mathbf{D}^{-1}_{0,b,a_{0},3}\right\|_{\infty}\left|P_{0}\left(w_{b,a_{0},3}K_{b,a_{0}}\theta_{0}\right)\right|.$

By (A1) and (A5),
$\left|P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)\right|$ and
$\left|P_{0}\left(w_{b,a_{0},3}K_{b,a_{0}}\theta_{0}\right)\right|$ are both
$O(1)$. By Lemma 4, $\left\|\mathbf{D}^{-1}_{0,h,a_{0},1}\right\|_{\infty}$
and $\left\|\mathbf{D}^{-1}_{0,b,a_{0},3}\right\|_{\infty}$ are also $O(1)$.
Finally, by Lemma 8, each of the four differences is $O_{p}(\\{nh\\}^{-1/2})$.
Thus, $R_{n,h,b,a_{0},6}=O_{p}(\\{nh\\}^{-1})$.

For the uniform statement, as above, we have

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left|R_{n,h,b,a_{0},6}\right|$
$\displaystyle\lesssim\sup_{a_{0}\in\mathcal{A}_{0}}\left\\{\left\|\mathbf{D}^{-1}_{0,h,a_{0},1}\right\|_{\infty}\left\|\mathbf{D}_{0,h,a_{0},1}-\mathbf{D}_{n,h,a_{0},1}\right\|_{\infty}\left\|\mathbf{D}^{-1}_{n,h,a_{0},1}-\mathbf{D}^{-1}_{0,h,a_{0},1}\right\|_{\infty}\left|P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)\right|\right\\}$
$\displaystyle\qquad+\sup_{a_{0}\in\mathcal{A}_{0}}\left\\{\|\mathbf{D}^{-1}_{0,b,a_{0},3}\|_{\infty}\left\|\mathbf{D}_{0,b,a_{0},3}-\mathbf{D}_{n,b,a_{0},3}\right\|_{\infty}\left\|\mathbf{D}^{-1}_{n,b,a_{0},3}-\mathbf{D}^{-1}_{0,b,s,3}\right\|_{\infty}\left|P_{0}\left(w_{b,s,a_{0}}K_{b,a_{0}}\theta_{0}\right)\right|\right\\}.$

By (A1) and (A8),
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|P_{0}\left(w_{h,a_{0},1}K_{h,a_{0}}\theta_{0}\right)\right|$
and
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|P_{0}\left(w_{b,a_{0},3}K_{b,a_{0}}\theta_{0}\right)\right|$
are both $O(1)$. By Lemma 4 and (A8),
$\sup_{a_{0}\in\mathcal{A}_{0}}\left\|\mathbf{D}^{-1}_{0,h,a_{0},1}\right\|_{\infty}$
and
$\sup_{a_{0}\in\mathcal{A}_{0}}\left\|\mathbf{D}^{-1}_{0,b,a_{0},3}\right\|_{\infty}$
are also $O(1)$. Finally, by Lemma 12 both products of differences ares
$O_{p}(\\{nh/\log h^{-1}\\}^{-1/2})$. We thus conclude
$\sup_{a_{0}\in\mathcal{A}_{0}}\left|R_{n,h,b,a_{0},6}\right|=O_{p}(\\{nh/\log
h^{-1}\\}^{-1})$. ∎

##  Supplement L Analysis of the covariance estimator

######  Lemma 21.

If (A1)–(A5) hold for $a_{0}=u$ and $a_{0}=v$, then

$\displaystyle
h\mathbb{P}_{n}(\phi_{n,h,b,u}^{*}\phi_{n,h,b,v}^{*})-hP_{0}(\phi_{\infty,h,b,u}^{*}\phi_{\infty,h,b,v}^{*})=o_{p}(1).$

If (A1)–(A3) and (A6)–(A8) hold and $nh^{5}=O(1)$, then

$\displaystyle\sup_{u,v\in\mathcal{A}_{0}}\left|h\mathbb{P}_{n}(\phi_{n,h,b,u}^{*}\phi_{n,h,b,v}^{*})-hP_{0}(\phi_{\infty,h,b,u}^{*}\phi_{\infty,h,b,v}^{*})\right|=O_{p}(n^{-p})$

for some $p>0$.

###### Proof of Lemma 21.

Analysis of this expression uses many of the techniques developed throughout
this document. Hence, for brevity, we omit some of the details in this proof.

For convenience, we define

$\displaystyle\eta_{n,h,b,u}(w)$
$\displaystyle:=\int\Gamma_{n,h,b,u}(a)\left[\mu_{n}(a,w)-\smallint\mu_{n}\,dQ_{n}\right]\,dF_{n}(a),\text{
and}$ $\displaystyle\eta_{0,h,b,u}(w)$
$\displaystyle:=\int\Gamma_{0,h,b,u}(a)\left[\mu_{\infty}(a,w)-\smallint\mu_{\infty}\,dQ_{0}\right]\,dF_{0}(a).$

Up to terms with $u$ and $v$ swapped, we then expand
$h\mathbb{P}_{n}(\phi_{n,h,b,u}^{*}\phi_{n,h,b,v}^{*})-hP_{0}(\phi_{\infty,h,b,u}^{*}\phi_{\infty,h,b,v}^{*})$
as

$\displaystyle\mathbb{G}_{n}\left(h\Gamma_{n,h,b,u}\Gamma_{n,h,b,v}\xi_{n}^{2}\right)+\mathbb{G}_{n}\left(h\gamma_{n,h,b,u}\gamma_{n,h,b,v}\right)-\mathbb{G}_{n}\left(h\Gamma_{n,h,b,u}\gamma_{n,h,b,v}\xi_{n}\right)$
$\displaystyle\qquad+P_{0}\left[h\left(\Gamma_{n,h,b,u}-\Gamma_{0,h,b,u}\right)\left(\Gamma_{n,h,b,v}\xi_{n}-\gamma_{n,h,b,v}\right)\xi_{n}\right]+P_{0}\left[h\Gamma_{0,h,b,u}\left\\{\Gamma_{0,h,b,v}\left(\xi_{n}+\xi_{\infty}\right)-\gamma_{0,h,b,v}\right\\}\left\\{\xi_{n}-\xi_{\infty}\right\\}\right]$
$\displaystyle\qquad-
P_{0}\left[h\left(\Gamma_{0,h,b,u}\xi_{n}-\gamma_{0,h,b,u}\right)\left(\gamma_{n,h,b,v}-\gamma_{0,h,b,v}\right)\right]$
$\displaystyle\qquad+\mathbb{P}_{n}\left(h\eta_{n,h,b,u}\eta_{n,h,b,v}\right)+\mathbb{P}_{n}\left(h\Gamma_{n,h,b,u}\eta_{n,h,b,v}\xi_{n}\right)-\mathbb{P}_{n}\left(h\gamma_{n,h,b,u}\eta_{n,h,b,v}\right)$
$\displaystyle\qquad-
P_{0}\left(h\eta_{0,h,b,u}\eta_{0,h,b,v}\right)-P_{0}\left(h\Gamma_{0,h,b,u}\eta_{0,h,b,v}\xi_{\infty}\right)+P_{0}\left(h\gamma_{0,h,b,u}\eta_{0,h,b,v}\right).$
(14)

For the first term in (14), we note that
$\|\mathbf{D}_{n,h,b,a,1}^{-1}\|_{\infty}=O_{p}(1)$ and
$\|\mathbf{D}_{n,h,b,a,3}^{-1}\|_{\infty}=O_{p}(1)$ by Lemmas 4 and 8 for each
$a\in\\{u,v\\}$. Hence, by expanding $\Gamma_{n,h,b,u}\Gamma_{n,h,b,v}$,
$\mathbb{G}_{n}\left(h\Gamma_{n,h,b,u}\Gamma_{n,h,b,v}\xi_{n}^{2}\right)$ can
be decomposed into $O_{p}(1)$ times terms of the form

$\displaystyle\mathbb{G}_{n}\left(hw_{h,u,j}w_{h,v,k}K_{h,u}K_{b,v}\xi_{n}^{2}\right).$

By the bounded fourth moment of $Y$ and condition (A3), the class

$\left\\{hw_{h,u,j}w_{h,v,k}K_{h,u}K_{b,v}\xi_{n}^{2}:\mu_{n}\in\mathcal{F}_{\mu},g_{n}\in\mathcal{F}_{g}\right\\}$

has finite uniform entropy integral and an envelope that is a square-
integrable function times $h^{-1}$. Hence, by Theorem 2.14.1 of van der Vaart
and Wellner, (1996),
$\mathbb{G}_{n}\left(h\Gamma_{n,h,b,u}\Gamma_{n,h,b,v}\xi_{n}^{2}\right)=O_{p}(h^{-1})$.
Similarly, by Lemmas 4 and 12,
$\sup_{u\in\mathcal{A}_{0}}\|\mathbf{D}_{n,h,b,u,1}^{-1}\|_{\infty}=O_{p}(1)$
and
$\sup_{u\in\mathcal{A}_{0}}\|\mathbf{D}_{n,h,b,u,3}^{-1}\|_{\infty}=O_{p}(1)$,
and by conditions (A1) and (A3), the class

$\left\\{hw_{h,u,j}w_{h,v,k}K_{h,u}K_{b,v}\xi_{n}^{2}:\mu_{n}\in\mathcal{F}_{\mu},g_{n}\in\mathcal{F}_{g},u\in\mathcal{A}_{0},v\in\mathcal{A}_{0}\right\\}$

possesses finite uniform entropy integral and under (A8), an envelope that is
bounded up to a constant by $h^{-1}$. Therefore,
$\sup_{u,v\in\mathcal{A}_{0}}|\mathbb{G}_{n}\left(h\Gamma_{n,h,b,u}\Gamma_{n,h,b,v}\xi_{n}^{2}\right)|=O_{p}(h^{-1})$
as well. Using an analogous argument, we can show that
$\mathbb{G}_{n}\left(h\gamma_{n,h,b,u}\gamma_{n,h,b,v}\right)$ and
$\mathbb{G}_{n}\left(h\Gamma_{n,h,b,u}\gamma_{h,h,b,v}\xi_{n}\right)$, which
are the second and third terms in (14) are $O_{p}(h^{-1})$ both pointwise and
uniformly.

The fourth term in (14) can be written as a constant times a sum of terms of
the form

$h\left(\mathbf{D}_{n,h,u,j}^{-1}-\mathbf{D}_{0,h,u,j}^{-1}\right)P_{0}\left[w_{h,u,k}K_{h,u}\left(\Gamma_{n,h,b,v}\xi_{n}-\gamma_{n,h,b,v}\right)\xi_{n}\right].$

By Lemma 8,
$\|\mathbf{D}_{n,h,u,j}^{-1}-\mathbf{D}_{0,h,u,j}^{-1}\|_{\infty}=O_{p}(\\{nh\\}^{-1/2})$.
Using a change of variables, we can also show that
$\|w_{h,u,k}K_{h,u}\xi_{n}\|_{L_{2}(P_{0})}=O_{p}(h^{-1/2})$ and
$\|\Gamma_{n,h,b,v}\xi_{n}-\gamma_{n,h,b,v}\|_{L_{2}(P_{0})}=O_{p}(h^{-1/2})$.
Hence, under the pointwise conditions this term is
$O_{p}(\\{nh\\}^{-1/2})=o_{p}(1)$. Under the uniform results, the same rates
hold except that Lemma 12 includes an extra $\log h^{-1}$ term. Hence, under
the uniform conditions, this term is $O_{p}(\\{nh/\log
h^{-1}\\}^{-1/2})=O_{p}(h)$ uniformly over $u,v\in\mathcal{A}_{0}$. A similar
analysis applies to the sixth term in (14).

For the fifth term in (14), we can show that
$\|\Gamma_{0,h,b,u}(\xi_{n}-\xi_{\infty})\|_{L_{2}(P_{0})}=o_{p}(h^{-1/2})$
and
$\|\Gamma_{0,h,b,v}\left(\xi_{n}+\xi_{\infty}\right)-\gamma_{0,h,b,v}\|_{L_{2}(P_{0})}=O_{p}(h^{-1/2})$,
so the pointwise result follows by the Cauchy-Schwarz inequality. For the
uniform result, by (A6)(e),

$\sup_{u\in\mathcal{A}_{0}}\|\Gamma_{0,h,b,u}(\xi_{n}-\xi_{\infty})\|_{L_{2}(P_{0})}=o_{p}\left(h^{-1/2}h^{\frac{V}{2(2-V)}}\\{\log
n\\}^{-\frac{1}{2-V}}\right),$

so we obtain a uniform rate of $o_{p}\left(h^{\frac{V}{2(2-V)}}\\{\log
n\\}^{-\frac{1}{2-V}}\right)$ for this term.

For the seventh term in (14), by the boundedness of $\mu_{n}$ we have for any
$u,v$

$\displaystyle\left|\mathbb{P}_{n}\left(h\eta_{n,h,b,u}\eta_{n,h,b,v}\right)\right|$
$\displaystyle\lesssim
h\int\left|\Gamma_{n,h,b,u}\right|\,dF_{n}\int\left|\Gamma_{n,h,b,v}\right|\,dF_{n}.$

As in the proof of Lemmas 8 and 12, we can show that each of these terms is
$O_{p}(1)$ pointwise and uniformly under the appropriate conditions. The same
holds when $\mathbb{P}_{n}$ is replaced with $P_{0}$ above. Similarly, for the
eighth term in (14), we have

$\displaystyle\left|\mathbb{P}_{n}\left(h\Gamma_{n,h,b,u}\eta_{n,h,b,v}\xi_{n}\right)\right|$
$\displaystyle\lesssim
h\mathbb{P}_{n}\left|\Gamma_{n,h,b,u}\xi_{n}\right|\int\left|\Gamma_{n,h,b,v}\right|\,dF_{n}.$

We can show that both terms are $O_{p}(1)$ pointwise and uniformly. The
remainder of the terms in (14) can be handled using a similar argument.

We have now shown that

$\displaystyle
hP_{0}\left[\left(\phi_{n,h,b,u}^{*}-\phi_{\infty,h,b,u}^{*}\right)\phi_{n,h,b,v}^{*}\right]=o_{p}(1)$

under the pointwise conditions, and

$\displaystyle\sup_{u,v\in\mathcal{A}_{0}}h\left|P_{0}\left[\left(\phi_{n,h,b,u}^{*}-\phi_{\infty,h,b,u}^{*}\right)\phi_{n,h,b,v}^{*}\right]\right|=o_{p}\left(n^{-p}\right)$

for some $p>0$ since $nh^{5}=O(1)$.

∎

##  Supplement M Lemmas supporting uniform result

We now present a series of supporting lemmas towards the construction of
uniform bands for $\theta_{0}(a_{0})$.

######  Lemma 22.

Let $\sigma^{2}_{\infty,h,b}(a_{0}):=hP_{0}(\phi^{*}_{\infty,h,b,a_{0}})^{2}$.
If (A1)–(A2), (A7)(a), and (A8) hold, then there exist constants
$c,C\in(0,\infty)$ such that $c\leq\sigma^{2}_{\infty,h,b}(a_{0})\leq C$ for
all $a_{0}\in\mathcal{A}_{0}$ and all h small enough.

###### Proof of Lemma 22.

We note that (A8) implies that (A5) holds for all $a_{0}\in\mathcal{A}_{0}$,
and (A7)(a) implies that (A4)(a) holds for all $a_{0}\in\mathcal{A}_{0}$.
Therefore, the conditions of Lemma 5 are met for every
$a_{0}\in\mathcal{A}_{0}$, so $\sigma_{\infty,h,b}^{2}(a_{0})$ converges to
$V_{K,\tau}\sigma_{0}^{2}(a_{0})/f_{0}(a_{0})$ for each
$a_{0}\in\mathcal{A}_{0}$. Since $\mathcal{A}_{0}$ is compact, this
convergence is also uniform in $a_{0}$. By Lemma 5, $V_{K,\tau}\in(0,\infty)$,
so by (A8), there exist $c,C\in(0,\infty)$ such that $c\leq
V_{K,\tau}\sigma_{0}^{2}(a_{0})/f_{0}(a_{0})\leq C$ for all
$a_{0}\in\mathcal{A}_{0}$. Hence, for all $h$ small enough,
$c\leq\sigma_{\infty,h,b}^{2}(a_{0})\leq C$ for all $a_{0}\in\mathcal{A}_{0}$.
∎

######  Lemma 23.

If (A1), (A7)(a), and (A8) hold, then
$\sup_{a_{0}\in\mathcal{A}_{0}}P_{0}|h\phi^{*}_{\infty,h,b,a_{0}}|^{k}\lesssim
h$ for all $h$ small enough.

###### Proof of Lemma 23.

First by the triangle inequality, we have that

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left[P_{0}|h\phi^{*}_{\infty,h,b,a_{0}}|^{k}\right]^{1/k}$
$\displaystyle\leq\sup_{a_{0}\in\mathcal{A}_{0}}\left[P_{0}\left|h\Gamma_{0,h,b,a_{0}}\xi_{\infty}\right|^{k}\right]^{1/k}+\sup_{a_{0}\in\mathcal{A}_{0}}\left[P_{0}\left|h\gamma_{0,h,b,a_{0}}\right|^{k}\right]^{1/k}$
$\displaystyle\qquad+\sup_{a_{0}\in\mathcal{A}_{0}}\left[P_{0}\left|h\int\Gamma_{0,h,b,a_{0}}(\bar{a})\left\\{\mu_{\infty}(\bar{a},w)-\int\mu_{\infty}(\bar{a},\bar{w})\,dQ_{0}(\bar{w})\right\\}\,dF_{0}(\bar{a})\right|^{k}\right]^{1/k}.$
(15)

We show that three terms on the right hand side of the above display are
bounded up to a constant by $h^{1/k}$, which implies that
$\sup_{a_{0}\in\mathcal{A}_{0}}P_{0}|h\phi^{*}_{\infty,h,b,a_{0}}|^{k}$ is
bounded up to a constant by $h$. Noting that $|\xi_{\infty}|$ is
$P_{0}$-almost surely uniformly bounded by (A7)(a) and (A8), we have

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\left[P_{0}\left|h\Gamma_{0,h,b,a_{0}}\xi_{\infty}\right|^{k}\right]^{1/k}$
$\displaystyle\lesssim\sup_{a_{0}\in\mathcal{A}_{0}}\left[\int\left|h\Gamma_{0,h,b,a_{0}}(a)\right|^{k}\,dF_{0}(a)\right]^{1/k}$
$\displaystyle\leq\sup_{a_{0}\in\mathcal{A}_{0}}\left\\{\int\left|he_{1}^{T}\mathbf{D}_{0,h,a_{0}}^{-1}w_{h,a_{0},1}(a)K_{h,a_{0}}(a)\right|^{k}\,dF_{0}(a)\right\\}^{1/k}$
$\displaystyle\qquad+\sup_{a_{0}\in\mathcal{A}_{0}}\left\\{\int\left|he_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{0}}^{-1}w_{b,a_{0},3}(a)K_{b,a_{0}}(a)\right|^{k}\,dF_{0}(a)\right\\}^{1/k}$
$\displaystyle\lesssim
h^{1/k}\sup_{a_{0}\in\mathcal{A}_{0}}\left|e_{1}^{T}\mathbf{D}_{0,h,a_{0}}^{-1}\mathbf{1}\right|+h^{1/k}\tau_{n}^{-1/k}\sup_{a_{0}\in\mathcal{A}_{0}}\left|e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{0}}^{-1}\mathbf{1}\right|$

By Lemma 4 and (A8),
$\sup_{a_{0}\in\mathcal{A}_{0}}|e_{1}^{T}\mathbf{D}_{0,h,a_{0}}^{-1}|$ and
$\sup_{a_{0}\in\mathcal{A}_{0}}|e_{3}^{T}c_{2}\tau_{n}^{2}\mathbf{D}_{0,b,a_{0}}^{-1}|$
are both $O(1)$. Therefore, the first term on the right hand side of (15) is
bounded up to a constant by $h^{1/k}$ as $h\longrightarrow 0$. The second term
on the right hand side of (15) can similarly be shown to be bounded up to a
constant by $h^{1/k}$ using Lemma 4 and the fact that $\mu_{0}$ is uniformly
bounded since $|Y|$ is $P_{0}$-almost surely uniformly bounded by (A8).

Finally, the last term of (15) can be bounded using Jensen’s inequality as
follows:

$\displaystyle\sup_{a_{0}\in\mathcal{A}_{0}}\int\left|h\int\Gamma_{0,h,b,a_{0}}(\bar{a})\left\\{\mu_{\infty}(\bar{a},w)-\smallint\mu_{\infty}(\bar{a},\bar{w})\,dQ_{0}(\bar{w})\right\\}\,dF_{0}(\bar{a})\right|^{k}\,dQ_{0}(w)$
$\displaystyle\qquad\leq\sup_{a_{0}\in\mathcal{A}_{0}}\iint\left|h\Gamma_{0,h,b,a_{0}}(\bar{a})\left\\{\mu_{\infty}(\bar{a},w)-\int\mu_{\infty}(\bar{a},\bar{w})\,dQ_{0}(\bar{w})\right\\}\right|^{k}\,dF_{0}(\bar{a})\,dQ_{0}(w)$
$\displaystyle\qquad\lesssim\sup_{a_{0}\in\mathcal{A}_{0}}\int\left|h\Gamma_{0,h,b,a_{0}}(\bar{a})\right|^{k}\,dF_{0}(\bar{a}),$

where the final inequality follows from the uniform bound on $\mu_{\infty}$
assumed by (A7)(a). The final integral is bounded up to a constant by $h$ as
shown above. ∎

######  Lemma 24.

Let $\\{Z_{\infty,h,b}(a_{0}):a_{0}\in\mathcal{A}_{0}\\}$ be the mean-zero
Gaussian process with covariance function

$\Sigma_{\infty,h,b}:(u,v)\mapsto
hP_{0}(\phi^{*}_{\infty,h,b,u}\phi^{*}_{\infty,h,b,v})/[\sigma_{\infty,h,b}(u)\sigma_{\infty,h,b}(v)].$

If (A1)–(A2), (A7)(a), and (A8) hold, then
$\\{Z_{\infty,h,b}:a_{0}\in\mathcal{A}_{0}\\}$ is tight in
$\ell^{\infty}(\mathcal{A}_{0})$,
$E_{0}\left[\sup_{a_{0}\in\mathcal{A}_{0}}|Z_{\infty,h,b}(a_{0})|\right]\leq
C\sqrt{\log(1/h)}$ for a constant $C$ not depending on $h$, and

$E\left[\sup_{|u-v|<\delta}\left|Z_{\infty,h,b}(u)-Z_{\infty,h,b}(v)\right|\right]\leq
C^{\prime}h^{-1/2}([2\delta]\wedge
h)^{1/2}\left[\log\frac{|\mathcal{A}_{0}|}{2([2\delta]\wedge h)}\right]^{1/2}$

for all $[2\delta]\wedge h\leq|\mathcal{A}_{0}|/8$ and a constant $C^{\prime}$
not depending on $h$.

###### Proof of Lemma 24.

Let $\rho_{\infty,h,b}$ be the standard deviation semi-metric corresponding to
$Z_{\infty,h,b}$; i.e.

$\rho_{\infty,h,b}^{2}(u,v):=E\left[Z_{\infty,h,b}(u)-Z_{\infty,h,b}(v)\right]^{2}=|\Sigma_{\infty,h,b}(u,u)-2\Sigma_{\infty,h,b}(u,v)+\Sigma_{\infty,h,b}(v,v)|=2\left|1-\Sigma_{\infty,h,b}(u,v)\right|$

because $\Sigma_{\infty,h,b}(u,u)=1$ for any $u$ by definition. We first
establish that $\rho_{\infty,h,b}$ is Lipschitz in $|u-v|$. By Lemma 22, there
exist constants $c,C\in(0,\infty)$ such that for all $h$ small enough and all
$a_{0}\in\mathcal{A}_{0}$, $c\leq\sigma^{2}_{\infty,h,b}(a_{0})\leq C$. We can
then bound $|1-\Sigma_{\infty,h,b}(u,v)|$ as follows:

$\displaystyle|1-\Sigma_{\infty,h,b}(u,v)|$
$\displaystyle=\left|1-hP_{0}(\phi^{*}_{\infty,h,b,u}\phi^{*}_{\infty,h,b,v})/[\sigma_{\infty,h,b}(u)\sigma_{\infty,h,b}(v)]\right|$
$\displaystyle=\left|1-\frac{\sigma_{\infty,h,b}(u)}{\sigma_{\infty,h,b}(v)}-\frac{hP_{0}\phi^{*}_{\infty,h,b,u}(\phi^{*}_{\infty,h,b,v}-\phi^{*}_{\infty,h,b,u})}{\sigma_{\infty,h,b}(u)\sigma_{\infty,h,b}(v)}\right|$
$\displaystyle\leq\frac{\left|\sigma_{\infty,h,b}(u)-\sigma_{\infty,h,b}(v)\right|}{\sigma_{\infty,h,b}(v)}+\frac{h\left|P_{0}\phi^{*}_{\infty,h,b,u}(\phi^{*}_{\infty,h,b,v}-\phi^{*}_{\infty,h,b,u})\right|}{\sigma_{\infty,h,b}(u)\sigma_{\infty,h,b}(v)}$
$\displaystyle\leq\frac{\left|\sigma_{\infty,h,b}(u)-\sigma_{\infty,h,b}(v)\right|+h^{1/2}\left[P_{0}(\phi^{*}_{\infty,h,b,u}-\phi^{*}_{\infty,h,b,v})^{2}\right]^{1/2}}{\sigma_{\infty,h,b}(v)}$
$\displaystyle\leq
c^{-1/2}\left|\sigma_{\infty,h,b}(u)-\sigma_{\infty,h,b}(v)\right|+c^{-1/2}h^{1/2}\left[P_{0}(\phi^{*}_{\infty,h,b,u}-\phi^{*}_{\infty,h,b,v})^{2}\right]^{1/2}.$

Since $\sigma_{\infty,h,b}(u)$ and $\sigma_{\infty,h,b}(v)$ are bounded away
from zero for all $h$ small enough and $x\mapsto x^{1/2}$ is Lipschitz on
$[\varepsilon,\infty)$ for any $\varepsilon>0$, we also have

$\displaystyle\left|\sigma_{\infty,h,b}(u)-\sigma_{\infty,h,b}(v)\right|$
$\displaystyle=\left|\left[hP_{0}(\phi^{*}_{\infty,h,b,u})^{2}\right]^{1/2}-\left[hP_{0}(\phi^{*}_{\infty,h,b,v})^{2}\right]^{1/2}\right|$
$\displaystyle\lesssim