$\displaystyle-\frac{1}{2}\left[(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))^{\top}\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))-(\mathbf{x}^{\prime}-\bm{\mu}(\mathbf{z}^{\prime};\bm{\vartheta}))^{\top}\mathbf{M}^{-1}(\mathbf{z}^{\prime};\bm{\vartheta})(\mathbf{x}^{\prime}-\bm{\mu}(\mathbf{z}^{\prime};\bm{\vartheta}))\right]$
$\displaystyle-\frac{1}{2}\left[\log\det\left(\mathbf{M}(\mathbf{z};\bm{\vartheta})\right)-\log\det\left(\mathbf{M}(\mathbf{z}^{\prime};\bm{\vartheta})\right)\right]$
$\displaystyle:=$ $\displaystyle-\frac{1}{2}(I_{1}+I_{2}),$

where the definitions of $I_{1}$ and $I_{2}$ should be obvious, and
$\mathbf{M}(\mathbf{z};\bm{\vartheta})=\mathbf{H}(\mathbf{z};\bm{\vartheta})\mathbf{H}(\mathbf{z};\bm{\vartheta})^{\top}$.

For $I_{2}$, we have

$\displaystyle|\mathbf{M}(\mathbf{z};\bm{\vartheta})-\mathbf{M}(\mathbf{z}^{\prime};\bm{\vartheta})|$
$\displaystyle\leq$
$\displaystyle|\mathbf{H}(\mathbf{z};\bm{\vartheta})-\mathbf{H}(\mathbf{z}^{\prime};\bm{\vartheta})|\left(|\mathbf{H}(\mathbf{z};\bm{\vartheta})|+|\mathbf{H}(\mathbf{z}^{\prime};\bm{\vartheta})|\right)$
$\displaystyle\leq$ $\displaystyle
M|\mathbf{z}-\mathbf{z}^{\prime}|_{\bm{\chi}}(2+|\mathbf{z}|_{\bm{\chi}}+|\mathbf{z}^{\prime}|_{\bm{\chi}}),$

where the second inequality follows from the facts that

$\displaystyle|\mathbf{H}(\mathbf{z};\bm{\vartheta})-\mathbf{H}(\mathbf{z}^{\prime};\bm{\vartheta})|=O(|\mathbf{z}-\mathbf{z}^{\prime}|_{\bm{\chi}}),$
$\displaystyle|\mathbf{H}(\mathbf{z};\bm{\vartheta})|\leq|\mathbf{H}(\mathbf{z};\bm{\vartheta})-\mathbf{H}(\mathbf{0};\bm{\vartheta})|+|\mathbf{H}(\mathbf{0};\bm{\vartheta})|=O(1+|\mathbf{z}|_{\bm{\chi}})$

by using Assumption 1.2 twice.

By Assumption 3, it is easy to know that
$\det(\mathbf{M}(\mathbf{z};\bm{\vartheta}))\geq\underline{H}>0$, which in
connection with the fact $\log(\cdot)$ is Lipschitz continuous on
$[\underline{H},\infty)$ yields that

$\displaystyle I_{2}\leq
M|\det\left(\mathbf{M}(\mathbf{z};\bm{\vartheta})\right)-\det\left(\mathbf{M}(\mathbf{z}^{\prime};\bm{\vartheta})\right)|.$

In addition, for an invertible matrix $\mathbf{A}$,
$\det(\mathbf{A}+\mathbf{B})=\det(\mathbf{A})+\mathrm{tr}(\mathbf{A}^{-1,\top}\mathbf{B})+o(|\mathbf{B}|)$,
and for a positive definite matrix $\mathbf{A}$ and symmetric matrix
$\mathbf{B}$,
$|\mathrm{tr}(\mathbf{A}^{-1,\top}\mathbf{B})|\leq|\mathbf{B}|\mathrm{tr}(\mathbf{A}^{-1})$.
Hence, we have

$\displaystyle I_{2}$ $\displaystyle\leq$ $\displaystyle
M|\det\left(\mathbf{M}(\mathbf{z};\bm{\vartheta})\right)-\det\left(\mathbf{M}(\mathbf{z}^{\prime};\bm{\vartheta})\right)|$
$\displaystyle\leq$ $\displaystyle
M\mathrm{tr}(\mathbf{M}^{-1}(\mathbf{z}^{\prime};\bm{\vartheta}))\cdot|\mathbf{M}(\mathbf{z};\bm{\vartheta})-\mathbf{M}(\mathbf{z}^{\prime};\bm{\vartheta})|$
$\displaystyle=$ $\displaystyle
O(|\mathbf{z}-\mathbf{z}^{\prime}|_{\bm{\chi}}(1+|\mathbf{z}|_{\bm{\chi}}+|\mathbf{z}^{\prime}|_{\bm{\chi}})).$

Note that if
$\mathbf{H}(\mathbf{z};\bm{\vartheta})\equiv\mathbf{H}(\mathbf{0};\bm{\vartheta})$,
$I_{2}=0$.

For $I_{1}$, since $|\mathbf{H}^{-1}(\mathbf{z};\bm{\vartheta})|$ is bounded
by Assumption 3, we can obtain that

$\displaystyle I_{1}$ $\displaystyle\leq$
$\displaystyle|\mathbf{H}^{-1}(\mathbf{z};\bm{\vartheta})(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))-\mathbf{H}^{-1}(\mathbf{z}^{\prime};\bm{\vartheta})(\mathbf{x}^{\prime}-\bm{\mu}(\mathbf{z}^{\prime};\bm{\vartheta}))|$
$\displaystyle\cdot(|\mathbf{H}^{-1}(\mathbf{z};\bm{\vartheta})(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))|+|\mathbf{H}^{-1}(\mathbf{z}^{\prime};\bm{\vartheta})(\mathbf{x}^{\prime}-\bm{\mu}(\mathbf{z}^{\prime};\bm{\vartheta}))|)$
$\displaystyle=$ $\displaystyle
O(|\mathbf{y}-\mathbf{y}^{\prime}|_{\bm{\chi}}\cdot(1+|\mathbf{y}|_{\bm{\chi}}^{2}+|\mathbf{y}^{\prime}|_{\bm{\chi}}^{2})),$

where $\mathbf{y}=(\mathbf{x},\mathbf{z})$. Similarly, if
$\mathbf{H}(\mathbf{z};\bm{\vartheta})\equiv\mathbf{H}(\mathbf{0};\bm{\vartheta})$,
$I_{1}=O(|\mathbf{y}-\mathbf{y}^{\prime}|_{\bm{\chi}}\cdot(1+|\mathbf{y}|_{\bm{\chi}}+|\mathbf{y}^{\prime}|_{\bm{\chi}}))$.

For
$\mathcal{l}(\mathbf{x},\mathbf{z};\bm{\vartheta})-\mathcal{l}(\mathbf{x},\mathbf{z};\bm{\vartheta}^{\prime})$,
write

$\displaystyle\mathcal{l}(\mathbf{x},\mathbf{z};\bm{\vartheta})-\mathcal{l}(\mathbf{x},\mathbf{z};\bm{\vartheta}^{\prime})$
$\displaystyle=$
$\displaystyle-\frac{1}{2}\left[(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))^{\top}\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))-(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}^{\prime}))^{\top}\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta}^{\prime})(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}^{\prime}))\right]$
$\displaystyle-\frac{1}{2}\left[\log\det\left(\mathbf{M}(\mathbf{z};\bm{\vartheta})\right)-\log\det\left(\mathbf{M}(\mathbf{z};\bm{\vartheta}^{\prime})\right)\right]$
$\displaystyle:=$ $\displaystyle-\frac{1}{2}(I_{3}+I_{4}).$

Similar to the development for $I_{1}$ and $I_{2}$, we can obtain that

$\displaystyle
I_{3}=O(|\bm{\vartheta}-\bm{\vartheta}^{\prime}|(1+|\mathbf{y}|_{\bm{\chi}}^{3})\quad\text{and}\quad
I_{4}=O(|\bm{\vartheta}-\bm{\vartheta}^{\prime}|(1+|\mathbf{z}|_{\bm{\chi}}^{2})),$

where we again let $\mathbf{y}=(\mathbf{x},\mathbf{z})$. Also if
$\mathbf{H}(\mathbf{z};\bm{\vartheta})\equiv\mathbf{H}(\mathbf{0};\bm{\vartheta})$,

$\displaystyle
I_{3}=O(|\bm{\vartheta}-\bm{\vartheta}^{\prime}|(1+|\mathbf{y}|_{\bm{\chi}}^{2})\quad\text{and}\quad
I_{4}=O(|\bm{\vartheta}-\bm{\vartheta}^{\prime}|).$

Combing the above analysis, we have shown
$\mathcal{l}\in\mathcal{H}(3,\bm{\chi},M)$. In addition, if
$\mathbf{H}(\mathbf{z};\bm{\vartheta})\equiv\mathbf{H}(\mathbf{0};\bm{\vartheta})$,
$\mathcal{l}\in\mathcal{H}(2,\bm{\chi},M)$.

Similar to the development for $\mathcal{l}$, we can show
$\gradient\mathcal{l}$,
$\gradient^{2}\mathcal{l}\in\mathcal{H}(3,\bm{\chi},M)$ and
$\gradient\mathcal{l},\gradient^{2}\mathcal{l}\in\mathcal{H}(2,\bm{\chi},M)$
if
$\mathbf{H}(\mathbf{z};\bm{\vartheta})\equiv\mathbf{H}(\mathbf{0};\bm{\vartheta})$.

The proof is now complete. ∎

###### Proof of Lemma B.3.

(1). By Proposition 2.1.1, we have
$\sup_{\tau\in[0,1]}\|\widetilde{\mathbf{x}}_{t}(\tau)\|_{C}<\infty$. Since
$g\in\mathcal{H}(C,\bm{\chi},M)$, we have

$\displaystyle\sup_{\bm{\eta}\neq\bm{\eta}^{\prime}}\frac{|\widetilde{G}_{\tau}(\bm{\eta})-\widetilde{G}_{\tau}(\bm{\eta}^{\prime})|}{|\bm{\eta}-\bm{\eta}^{\prime}|}\leq
M(Th)^{-1}\sum_{t=1}^{T}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)\left[2+|\widetilde{\mathbf{y}}_{t}(\tau_{t})|_{\bm{\chi}}^{C}+\||\widetilde{\mathbf{y}}_{t}(\tau_{t})|_{\bm{\chi}}^{C}\|_{1}\right].$

Using
$(Th)^{-1}\sum_{t=1}^{T}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)<\infty$,
we have

$\displaystyle\left\|\sup_{\bm{\eta}\neq\bm{\eta}^{\prime}}\frac{|\widetilde{G}_{\tau}(\bm{\eta})-\widetilde{G}_{\tau}(\bm{\eta}^{\prime})|}{|\bm{\eta}-\bm{\eta}^{\prime}|}\right\|_{1}\leq
M\max_{t}\||\widetilde{\mathbf{y}}_{t}(\tau_{t})|_{\bm{\chi}}^{C}\|_{1}<\infty.$

In addition, by using the Lipschitz property of $\widehat{K}(\cdot)$, we have

$\displaystyle|\widetilde{G}_{\tau}(\bm{\eta})-\widetilde{G}_{\tau^{\prime}}(\bm{\eta}^{\prime})|$
$\displaystyle\leq$
$\displaystyle(Th)^{-1}\sum_{t=1}^{T}\left|\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)-\widehat{K}\left(\frac{\tau_{t}-\tau^{\prime}}{h}\right)\right|\cdot\sup_{\bm{\vartheta}}(|g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\vartheta})|+\|g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\vartheta})\|_{1})$
$\displaystyle+(Th)^{-1}\sum_{t=1}^{T}\widehat{K}\left(\frac{\tau_{t}-\tau^{\prime}}{h}\right)\cdot|g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}(\tau_{t}-\tau)/h)-g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}^{\prime}+\bm{\eta}_{2}^{\prime}(\tau_{t}-\tau^{\prime})/h)|$
$\displaystyle\leq$ $\displaystyle
M\left(h^{-2}|\tau-\tau^{\prime}|+h^{-1}|\bm{\eta}-\bm{\eta}^{\prime}|+h^{-2}|\bm{\eta}_{2}|\cdot|\tau-\tau^{\prime}|\right)\cdot\frac{1}{T}\sum_{t=1}^{T}(2+|\widetilde{\mathbf{y}}_{t}(\tau_{t})|_{\bm{\chi}}^{C}+\||\widetilde{\mathbf{y}}_{t}(\tau_{t})|_{\bm{\chi}}^{C}\|_{1}).$

Combing the above analyses, the first result follows.

(2). By Lemma B.8.1 and Proposition 2.2, for $|\tau_{t}-\tau|\leq h$, we have

$\displaystyle\|g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)-g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\|_{1}$
$\displaystyle\leq$ $\displaystyle
M\sum_{j=0}^{\infty}\chi_{j}\|\widetilde{\mathbf{x}}_{t-j}(\tau_{t})-\widetilde{\mathbf{x}}_{t-j}(\tau)\|_{C}=O(h).$

Hence, we have

$\displaystyle\left\|\widetilde{B}_{\tau}(\bm{\eta})-\frac{1}{Th}\sum_{t=1}^{T}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\right\|_{1}$
$\displaystyle\leq$ $\displaystyle
M\frac{1}{Th}\sum_{t=1}^{T}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)\sum_{j=0}^{\infty}\chi_{j}\|\widetilde{\mathbf{x}}_{t-j}(\tau_{t})-\widetilde{\mathbf{x}}_{t-j}(\tau)\|_{C}=O(h)$

and

$\displaystyle\frac{1}{Th}\sum_{t=1}^{T}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)E\left[g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\right]$
$\displaystyle=$
$\displaystyle\int_{-\tau/h}^{(1-\tau)/h}\widehat{K}(u)E(g(\widetilde{\mathbf{y}}_{0}(\tau),\bm{\eta}_{1}+\bm{\eta}_{2}u))\mathrm{d}u+O((Th)^{-1})$

by the definition of Riemann integral and the stationarity of
$\widetilde{\mathbf{y}}_{t}(\tau)$.

(3). Write

$\displaystyle|\mathbf{m}_{t}^{(2)}(u,\tau)-\mathbf{m}_{t}^{(2)}(u,\tau^{\prime})|$
$\displaystyle\leq$
$\displaystyle|\widehat{K}\left(\frac{\tau-u}{h}\right)-\widehat{K}\left(\frac{\tau^{\prime}-u}{h}\right)|\cdot|g(\widetilde{\mathbf{y}}_{t}(u),\bm{\theta}(\tau)-v\mathbf{d}(u,\tau))|\cdot|\mathbf{d}(u,\tau)|$
$\displaystyle+|\widehat{K}\left(\frac{\tau^{\prime}-u}{h}\right)|\cdot|g(\widetilde{\mathbf{y}}_{t}(u),\bm{\theta}(\tau)-v\mathbf{d}(u,\tau))-g(\widetilde{\mathbf{y}}_{t}(u),\bm{\theta}(\tau^{\prime})-v\mathbf{d}(u,\tau^{\prime}))|\cdot|\mathbf{d}(u,\tau)|$
$\displaystyle+|\widehat{K}\left(\frac{\tau^{\prime}-u}{h}\right)|\cdot|g(\widetilde{\mathbf{y}}_{t}(u),\bm{\theta}(\tau^{\prime})-v\mathbf{d}(u,\tau^{\prime}))|\cdot|\mathbf{d}(u,\tau)-\mathbf{d}(u,\tau^{\prime})|$
$\displaystyle:=$ $\displaystyle I_{1}+I_{2}+I_{3}.$

By the Lipschitz continuity of $\widehat{K}(\cdot)$ and
$\|\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}(u),\bm{\vartheta})|\|_{1}=O(1)$
(by Lemma B.8.3), we have

$\displaystyle E(I_{1})=O(h^{-1}|\tau-\tau^{\prime}|).$

Similarly, by Lemma B.8.2 and $|\mathbf{d}(u,\tau)|=O(1)$, we have

$\displaystyle E(I_{2})=O(|\tau-\tau^{\prime}|).$

By the Lipschitz continuity of $\mathbf{d}(u,\cdot)$, we have
$E(I_{3})=O(|\tau-\tau^{\prime}|)$. Hence,

$\displaystyle\left|\frac{1}{Th}\sum_{t=1}^{T}[\mathbf{m}_{t}^{(2)}(\tau,\tau_{t})-\mathbf{m}_{t}^{(2)}(\tau^{\prime},\tau_{t})]\right|\leq\frac{1}{Th}\sum_{t=1}^{T}|\mathbf{m}_{t}^{(2)}(\tau,\tau_{t})-\mathbf{m}_{t}^{(2)}(\tau^{\prime},\tau_{t})|=O(h^{-2}|\tau-\tau^{\prime}|).$

The proof is now complete.

(4). By Propositions 2.1.1 and 2.2.2, we have
$\sup_{\tau\in[0,1]}\left\|\widetilde{\mathbf{x}}_{t}(\tau)\right\|_{C}<\infty$
and
$\max_{t}\left\|\mathbf{x}_{t}-\widetilde{\mathbf{x}}_{t}(\tau_{t})\right\|_{C}=O(T^{-1})$.
Hence, we have $\max_{t}\left\|\mathbf{x}_{t}\right\|_{C}\leq M$.

By Lemma B.8 and the definitions of $\mathbf{y}_{t}$ and $\mathbf{y}_{t}^{c}$,
we have

$\displaystyle\|\sup_{\bm{\vartheta}\in\bm{\Theta}_{r}}|g(\mathbf{y}_{t},\bm{\vartheta})-g(\mathbf{y}_{t}^{c},\bm{\vartheta})|\|_{1}\leq
M\sum_{j=t}^{\infty}\chi_{j}\|\mathbf{x}_{t-j}\|_{C}=O\left(\sum_{j=t}^{\infty}\chi_{j}\right).$

In addition, by Proposition 2.2.1 and $\chi_{j}=O(j^{-(2+s)})$ for some $s>0$,
we have

$\displaystyle\|\sup_{\bm{\vartheta}\in\bm{\Theta}_{r}}|g(\mathbf{y}_{t},\bm{\vartheta})-g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\vartheta})|\|_{1}\leq
M\sum_{j=0}^{\infty}\chi_{j}\|\mathbf{x}_{t-j}-\widetilde{\mathbf{x}}_{t-j}(\tau_{t})\|_{C}$
$\displaystyle\leq$ $\displaystyle
M\sum_{j=0}^{\infty}\chi_{j}\|\mathbf{x}_{t-j}-\widetilde{\mathbf{x}}_{t-j}(\tau_{t-j})\|_{C}+M\sum_{j=0}^{\infty}\chi_{j}\|\widetilde{\mathbf{x}}_{t-j}(\tau_{t})-\widetilde{\mathbf{x}}_{t-j}(\tau_{t-j})\|_{C}$
$\displaystyle=$ $\displaystyle
O(\sum_{j=0}^{\infty}\chi_{j}/T)+O(\sum_{j=0}^{\infty}j\chi_{j}/T)=O(T^{-1}).$

Hence, we have

$\displaystyle\|\sup_{\tau\in[0,1]}\sup_{\bm{\eta}\in\mathbf{E}_{T}(r)}|\widetilde{G}_{\tau}(\bm{\eta})-G_{\tau}^{c}(\bm{\eta})|\|_{1}$
$\displaystyle\leq$ $\displaystyle
M(Th)^{-1}\sum_{t=1}^{T}\sup_{\bm{\vartheta}\in\bm{\Theta}_{r}}\|g(\widetilde{\mathbf{y}}_{t},\bm{\vartheta})-g(\mathbf{y}_{t}^{c},\bm{\vartheta})\|_{1}$
$\displaystyle\leq$ $\displaystyle
M(Th)^{-1}\sum_{t=1}^{T}\sum_{j=t}^{\infty}\chi_{j}\leq
M(Th)^{-1}\sum_{j=1}^{\infty}j\chi_{j}=O((Th)^{-1}).$

The proof of the fourth result is now complete. ∎

###### Proof of Lemma B.4.

(1). Let $\widetilde{\mathbf{y}}_{t}^{*}(\tau)$ be a coupled version of
$\widetilde{\mathbf{y}}_{t}(\tau)$ with $\bm{\varepsilon}_{0}$ replaced by
$\bm{\varepsilon}_{0}^{*}$. By Lemma B.8, we have

$\displaystyle\delta_{q}^{\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\vartheta})|}(t)$
$\displaystyle=$
$\displaystyle\|\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\vartheta})|-\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}^{*}(\tau),\bm{\vartheta})|\|_{q}$
$\displaystyle\leq$
$\displaystyle\|\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\vartheta})-g(\widetilde{\mathbf{y}}_{t}^{*}(\tau),\bm{\vartheta})|\|_{q}$
$\displaystyle\leq$ $\displaystyle
M\sum_{j=0}^{\infty}\chi_{j}\|\widetilde{\mathbf{x}}_{t-j}(\tau)-\widetilde{\mathbf{x}}_{t-j}^{*}(\tau)\|_{qC}$
$\displaystyle=$ $\displaystyle
M\sum_{j=0}^{t}\chi_{j}\delta_{r}^{\widetilde{\mathbf{x}}(\tau)}(t-j).$

By Proposition 2.1.2 and the conditions on $\alpha_{j}(\bm{\theta}(\tau))$ and
$\beta_{j}(\bm{\theta}(\tau))$ in the body of this lemma, we have
$\delta_{r}^{\widetilde{\mathbf{x}}(\tau)}(j)=O(j^{-(a+s)})$ for some $s>0$.
Hence, we have

$\displaystyle\sum_{j=0}^{t}\chi_{j}\delta_{r}^{\widetilde{\mathbf{x}}(\tau)}(t-j)$
$\displaystyle\leq$ $\displaystyle\sum_{j\geq
t/2}\chi_{j}\delta_{r}^{\widetilde{\mathbf{x}}(\tau)}(t-j)+\sum_{0\leq j\leq
t/2}\chi_{j}\delta_{r}^{\widetilde{\mathbf{x}}(\tau)}(t-j)$
$\displaystyle\leq$ $\displaystyle(t/2)^{-(a+s)}\sum_{j\geq
t/2}\delta_{r}^{\widetilde{\mathbf{x}}(\tau)}(t-j)+(t/2)^{-(a+s)}\sum_{0\leq
j\leq t/2}\chi_{j}$ $\displaystyle=$ $\displaystyle O(t^{-(a+s)}).$

The proof of the first result of this lemma is now complete.

(2)–(3). Since

$\displaystyle|\sup_{u,\bm{\eta}}|m(\tau,\bm{\eta},u)|-\sup_{u,\bm{\eta}}|m^{*}(\tau,\bm{\eta},u)||$
$\displaystyle\leq$
$\displaystyle\sup_{u,\bm{\eta}}|m(\tau,\bm{\eta},u)-m^{*}(\tau,\bm{\eta},u)|$
$\displaystyle\leq$ $\displaystyle
M\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\vartheta})-g(\widetilde{\mathbf{y}}_{t}^{*}(\tau),\bm{\vartheta})|,$

the second result follows directly from the first result.

Since $\mathbf{d}(u,\tau)=O(h^{2})$ when $|\tau-u|\leq h$, for each element of
$\mathbf{m}_{t}^{(2)}(u,\tau)$, we have

$\displaystyle|\sup_{\tau}|m_{t,i}^{(2)}(u,\tau)|-\sup_{\tau}|m_{t,i}^{(2)}(\tau,u)^{*}||$
$\displaystyle\leq$
$\displaystyle\sup_{\tau}|m_{t,i}^{(2)}(\tau,u)-m_{t,i}^{(2)}(\tau,u)^{*}|$
$\displaystyle\leq$ $\displaystyle
Mh^{2}\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\vartheta})-g(\widetilde{\mathbf{y}}_{t}^{*}(\tau),\bm{\vartheta})|,$

where $m_{t,i}^{(2)}(\tau,u)$ is yielded by the coupled version.

The proof is now complete. ∎

###### Proof of Lemma B.5.

(1). Note that

$\displaystyle
g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)-E(g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h))$
$\displaystyle=$
$\displaystyle\sum_{l=0}^{\infty}\mathcal{P}_{t-l}g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h),$

in which
$\\{\mathcal{P}_{t-l}(g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h))\\}_{t=1}^{T}$
is a sequence of martingale differences.

If $1<q\leq 2$, by the Burkholder’s inequality,
$|\sum_{i=1}^{d}a_{i}|^{r}\leq\sum_{i=1}^{d}|a_{i}|^{r}$ for $r\in(0,1]$ and
Lemma B.4.1, we have

$\displaystyle\|\widetilde{G}_{\tau}(\bm{\eta})\|_{q}$ $\displaystyle\leq$
$\displaystyle\sum_{l=0}^{\infty}\left\|\sum_{t=1}^{T}\frac{1}{Th}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)\mathcal{P}_{t-l}g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\right\|_{q}$
$\displaystyle\leq$ $\displaystyle
O(1)\sum_{l=0}^{\infty}\left\\{E\left[\sum_{t=1}^{T}\left(\frac{1}{Th}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)\mathcal{P}_{t-l}g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\right)^{2}\right]^{q/2}\right\\}^{1/q}$
$\displaystyle\leq$ $\displaystyle
O(1)\sum_{l=0}^{\infty}\left\\{E\left[\sum_{t=1}^{T}\left(\frac{1}{Th}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)\mathcal{P}_{t-l}g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\right)^{q}\right]\right\\}^{1/q}$
$\displaystyle\leq$ $\displaystyle
O(1)(Th)^{-(q-1)/q}\sum_{l=0}^{\infty}\sup_{\tau\in[0,1]}\delta_{q}^{\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\vartheta})|}(l)\left(\frac{1}{Th}\sum_{t=1}^{T}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)^{q}\right)^{1/q}$
$\displaystyle=$ $\displaystyle O((Th)^{-(q-1)/q}).$

Similarly, for $q\geq 2$, by the Burkholder’s inequality and the Minkowski
inequality, we have

$\displaystyle\|\widetilde{G}_{\tau}(\bm{\eta})\|_{q}$ $\displaystyle=$
$\displaystyle\left\|\frac{1}{Th}\sum_{t=1}^{T}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)\sum_{l=0}^{\infty}\mathcal{P}_{t-l}g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\right\|_{q}$
$\displaystyle\leq$
$\displaystyle\sum_{l=0}^{\infty}\left\|\sum_{t=1}^{T}\frac{1}{Th}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)\mathcal{P}_{t-l}g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\right\|_{q}$
$\displaystyle\leq$ $\displaystyle
O(1)\sum_{l=0}^{\infty}\left\\{E\left[\sum_{t=1}^{T}\left(\frac{1}{Th}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)\mathcal{P}_{t-l}g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\right)^{2}\right]^{q/2}\right\\}^{1/q}$
$\displaystyle\leq$ $\displaystyle
O(1)\sum_{l=0}^{\infty}\left\\{\sum_{t=1}^{T}\left[E\left(\frac{1}{Th}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)\mathcal{P}_{t-l}g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\right)^{q}\right]^{2/q}\right\\}^{1/2}$
$\displaystyle=$ $\displaystyle
O(1)(Th)^{-1/2}\sum_{l=0}^{\infty}\sup_{\tau\in[0,1]}\delta_{q}^{\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\vartheta})|}(l)=O((Th)^{-1/2}).$

The proof of the first result is now complete.

(2). For any fixed $v>0$, let $\kappa>0$ and $\mathbf{E}_{T}^{\kappa}(r)$ be a
discretization of $\mathbf{E}_{T}(r)$ such that for each
$\bm{\eta}\in\mathbf{E}_{T}(r)$ one can find
$\bm{\eta}^{\prime}\in\mathbf{E}_{T}^{\kappa}(r)$ satisfying
$|\bm{\eta}-\bm{\eta}^{\prime}|\leq\kappa$. Let
$\\#\mathbf{E}_{T}^{\kappa}(r)$ denote the numbers of sets in
$\mathbf{E}_{T}^{\kappa}(r)$. Write

$\displaystyle\Pr(\sup_{\bm{\eta}\in\mathbf{E}_{T}(r)}|\widetilde{G}_{\tau}(\bm{\eta})|>v)$
$\displaystyle\leq$
$\displaystyle\\#\mathbf{E}_{T}^{\kappa}(r)\sup_{\bm{\eta}\in\mathbf{E}_{T}(r)}\Pr(|\widetilde{G}_{\tau}(\bm{\eta})|>v/2)$
$\displaystyle+\Pr(\sup_{|\bm{\eta}-\bm{\eta}^{\prime}|\leq\kappa}|\widetilde{G}_{\tau}(\bm{\eta})-\widetilde{G}_{\tau}(\bm{\eta}^{\prime})|>v/2).$

By the Markov inequality, we have

$\displaystyle\Pr(|\widetilde{G}_{\tau}(\bm{\eta})|>v/2)\leq\frac{\|\widetilde{G}\tau(\bm{\eta})\|_{q}^{q}}{(v/2)^{q}}.$

Note that
$\\{\mathcal{P}_{t-j}g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\\}_{t}$
forms a sequence of martingale differences. By the Burkholder’s inequality and
Lemma B.4.1, we have

$\displaystyle\|\widetilde{G}_{\tau}(\bm{\eta})\|_{q}$ $\displaystyle\leq$
$\displaystyle(Th)^{-1}\sum_{j=0}^{\infty}\|\sum_{t=1}^{T}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)\mathcal{P}_{t-j}g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\|_{q}$
$\displaystyle\leq$
$\displaystyle(q-1)^{-1}(Th)^{-1}\sum_{j=0}^{\infty}\left(\|\sum_{t=1}^{T}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)^{2}\mathcal{P}_{t-j}^{2}g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)\|_{q/2}^{q/2}\right)^{1/q}$
$\displaystyle\leq$ $\displaystyle
M(Th)^{-(q-1)/q}\sum_{j=0}^{\infty}\sup_{\tau\in[0,1]}\delta_{q}^{\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\vartheta})|}(j)=O((Th)^{-(q-1)/q}),$

which in connection with the fact $\\#\mathbf{E}_{T}^{\kappa}(r)$ is
independent of $T$ yields that

$\displaystyle\\#\mathbf{E}_{T}^{\kappa}(r)\sup_{\bm{\eta}\in\mathbf{E}_{T}(r)}\Pr(|\widetilde{G}_{\tau}(\bm{\eta})|>v/2)=o(1).$

In addition, by Lemma B.3.1, we have

$\displaystyle\Pr(\sup_{|\bm{\eta}-\bm{\eta}^{\prime}|\leq\kappa}|\widetilde{G}_{\tau}(\bm{\eta})-\widetilde{G}_{\tau}(\bm{\eta}^{\prime})|>v/2)\leq
M\kappa\to 0$

by choosing $\kappa$ small enough. Hence,
$\Pr(\sup_{\bm{\eta}\in\mathbf{E}_{T}(r)}|\widetilde{G}_{\tau}(\bm{\eta})|>v)\to
0$ as $T\to\infty$.

(3). Let $\beta_{T}:=(\log T)^{1/2}(Th)^{-1/2}h^{-1/2}$ for short. Let further
$\mathbf{E}_{T,\kappa}(r)$ be a discretization of $\mathbf{E}_{T}(r)$ such
that for each $\bm{\eta}\in\mathbf{E}_{T}(r)$ one can find
$\bm{\eta}^{\prime}\in\mathbf{E}_{T,\kappa}(r)$ satisfying
$|\bm{\eta}-\bm{\eta}^{\prime}|\leq\kappa_{T}^{-1}$. Define
$\mathcal{T}_{T,\kappa}=\\{t/\kappa_{T}:t=1,2,...,\kappa_{T}\\}$ as a
discretization of $[0,1]$. For some constant $M>0$, we have

$\displaystyle\Pr(\sup_{\tau\in[0,1]}\sup_{\bm{\eta}\in\mathbf{E}_{T}(r)}|\widetilde{G}_{\tau}(\bm{\eta})|>M\beta_{T})$
$\displaystyle\leq$
$\displaystyle\Pr(\sup_{\tau\in\mathcal{T}_{T,\kappa}}\sup_{\bm{\eta}\in\mathbf{E}_{T,\kappa}(r)}|\widetilde{G}_{\tau}(\bm{\eta})|>\beta_{T}M/2)$
$\displaystyle+\Pr(\sup_{|\tau-\tau^{\prime}|\leq\kappa_{T}^{-1},|\bm{\eta}-\bm{\eta}^{\prime}|\leq\kappa_{T}^{-1}}|\widetilde{G}_{\tau}(\bm{\eta})-\widetilde{G}_{\tau^{\prime}}(\bm{\eta}^{\prime})|>\beta_{T}M/2).$

Let
$m_{t}(\tau,\bm{\eta},u):=\widehat{K}\left(\frac{\tau-u}{h}\right)g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau-u)/h)$,
we have
$\sup_{\tau\in[0,1]}\sup_{u,\bm{\eta}}\delta_{q}^{m(\tau,\bm{\eta},u)}(j)=O(j^{-(a+s)})$
and
$\sup_{\tau\in[0,1]}\delta_{q}^{\sup_{u,\bm{\eta}}|m(\tau,\bm{\eta},u)|}(j)=O(j^{-(a+s)})$
for some $a\geq 3/2$ by Lemma B.4.2. Let $\alpha=1/2$, we have

$W_{q,\alpha}:=\max_{k\geq
0}(k+1)^{\alpha}\sup_{\tau\in[0,1]}\sum_{j=k}^{\infty}\delta_{q}^{\sup_{u,\bm{\eta}}|m(\tau,\bm{\eta},u)|}(j)\leq
M\max_{k}k^{-(a-3/2+s)}<\infty$

and

$W_{2,\alpha}:=\max_{k\geq
0}(k+1)^{\alpha}\sup_{\tau\in[0,1]}\sup_{u,\bm{\eta}}\sum_{j=k}^{\infty}\delta_{2}^{m(\tau,\bm{\eta},u)}(j)\leq
M\max_{k}k^{-(a-3/2+s)}<\infty.$

Note that
$l=\min\\{1,\log(\\#\mathbf{E}_{T,\kappa}(r)\times\mathcal{T}_{T,\kappa})\\}\leq
3(2d+1)\log(T)$ and $M\beta_{T}Th=MT^{1/2}(\log
T)^{1/2}\geq\sqrt{Tl}W_{2,\alpha}+T^{1/q}l^{3/2}W_{q,\alpha}\geq T^{1/2}(\log
T)^{1/2}+T^{1/q}(\log T)^{3/2}$ for some $M$ large enough. By using Theorem
6.2 of Zhang & Wu (2017) (the proof therein also works for the uniform
functional dependence measure) with $q>2$ and $\alpha=1/2$ to
$\\{m_{t}(\tau,\bm{\eta},\tau_{t})\\}_{\tau\in\mathcal{T}_{T,\kappa},\bm{\eta}\in\mathbf{E}_{T,\kappa}(r)}$,
we have

$\displaystyle\Pr(\sup_{\tau\in\mathcal{T}_{T,\kappa}}\sup_{\bm{\eta}\in\mathbf{E}_{T,\kappa}(r)}|\widetilde{G}_{\tau}(\bm{\eta})|>\beta_{T}M/2)$
$\displaystyle\leq$
$\displaystyle\frac{MTl^{q/2}}{(\beta_{T}Th)^{q}}+M\exp\left(-\frac{M(\beta_{T}Th)^{2}}{T}\right)$
$\displaystyle\leq$ $\displaystyle M\left(T^{-(q-2)/2}+\exp(-\log T)\right)\to
0.$

In addition, by the Markov inequality and Lemma B.3.1, we have

$\displaystyle\Pr(\sup_{|\tau-\tau^{\prime}|\leq\kappa_{T}^{-1},|\bm{\eta}-\bm{\eta}^{\prime}|\leq\kappa_{T}^{-1}}|\widetilde{G}_{\tau}(\bm{\eta})-\widetilde{G}_{\tau^{\prime}}(\bm{\eta}^{\prime})|>\beta_{T}M/2)=O(h^{-2}T^{-3}/\beta_{T})\to
0.$

The proof is now complete. ∎

###### Proof of Lemma B.6.

For notational simplicity, we let
$\bm{\eta}(\tau)=[\bm{\theta}(\tau),h\bm{\theta}^{(1)}(\tau)]$ in what
follows, and define

$\displaystyle\bm{\Gamma}(\tau)$ $\displaystyle:=$
$\displaystyle\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))-E[\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))]$
$\displaystyle-\frac{1}{Th}\sum_{t=1}^{T}\widehat{\bm{K}}((\tau_{t}-\tau)/h)\otimes\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t})).$

Due to
$E(\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t})))=0$
by Lemma B.1, we have

$\displaystyle\bm{\Gamma}(\tau)$ $\displaystyle=$
$\displaystyle\frac{1}{Th}\sum_{t=1}^{T}\widehat{\bm{K}}((\tau_{t}-\tau)/h)\otimes[\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau)+\bm{\theta}^{(1)}(\tau)(\tau_{t}-\tau))$
$\displaystyle-\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))]$
$\displaystyle-\frac{1}{Th}\sum_{t=1}^{T}\widehat{\bm{K}}((\tau_{t}-\tau)/h)\otimes
E[\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau)+\bm{\theta}^{(1)}(\tau)(\tau_{t}-\tau))$
$\displaystyle-\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t})].$

By the Mean Value Theorem, we have

$\displaystyle\bm{\Gamma}(\tau)=(Th)^{-1}\sum_{t=1}^{T}[\mathbf{M}_{t}^{(2)}(\tau,\tau_{t})-E(\mathbf{M}_{t}^{(2)}(\tau,\tau_{t}))],$

where

$\displaystyle\mathbf{M}_{t}^{(2)}(\tau,u):=\widehat{\bm{K}}((\tau_{t}-\tau)/h)\otimes\gradient_{\bm{\vartheta}}^{2}\mathcal{l}(\widetilde{\bm{x}}_{t}(u),\widetilde{\bm{z}}_{t-1}(u);\bm{\theta}(\tau)-v\mathbf{r}(u))\mathbf{r}(u)\text{
for some $v\in[0,1]$},$
$\displaystyle\mathbf{r}(u)=\frac{1}{2}\bm{\theta}^{(2)}(\tau)(u-\tau)^{2}+\frac{1}{6}\bm{\theta}^{(3)}(\overline{\tau})(u-\tau)^{3}\text{
with $\overline{\tau}$ between $u$ and $\tau$}.$

We then use a similar argument as in the proof of Lemma B.5 to prove

$\displaystyle\Pr(\sup_{\tau\in[0,1]}|\bm{\Gamma}(\tau)|>M\beta_{T}h^{2})\to
0.$

Define $\kappa_{T}=T^{5}$ and
$\mathcal{T}_{T,\kappa}=\\{t/\kappa_{T}:t=1,2,...,\kappa_{T}\\}$ as a
discretization of $[0,1]$. For some constant $M>0$, we have

$\displaystyle\Pr(\sup_{\tau\in[0,1]}|\bm{\Gamma}(\tau)|>M\beta_{T}h^{2})$
$\displaystyle\leq$
$\displaystyle\Pr(\sup_{\tau\in\mathcal{T}_{T,\kappa}}|\bm{\Gamma}(\tau)|>\beta_{T}h^{2}M/2)$
$\displaystyle+\Pr(\sup_{|\tau-\tau^{\prime}|\leq\kappa_{T}^{-1}}|\bm{\Gamma}(\tau)-\bm{\Gamma}(\tau^{\prime})|>\beta_{T}h^{2}M/2).$

By Lemma B.3.3 and the Markov inequality, we have

$\displaystyle\Pr(\sup_{|\tau-\tau^{\prime}|\leq\kappa_{T}^{-1}}|\bm{\Gamma}(\tau)-\bm{\Gamma}(\tau^{\prime})|>\beta_{T}h^{2}M/2)=O\left(\frac{h^{-2}\kappa_{T}^{-1}}{\beta_{T}h^{2}M/2}\right)\to
0.$

By Lemma B.4.3, we have

$\displaystyle\sup_{u,\tau\in[0,1]}\delta_{q}^{|\mathbf{M}^{(2)}(\tau,u)|}(j)=O(h^{2}j^{-(a+s)})\quad\text{and}\quad\sup_{\tau\in[0,1]}\delta_{q}^{\sup_{u}|\mathbf{M}^{(2)}(\tau,u)|}(j)=O(h^{2}j^{-(a+s)})$

for some $a\geq 3/2$ . Let $\alpha=1/2$, we have

$\displaystyle\widetilde{W}_{q,\alpha}:=\max_{k\geq
0}(k+1)^{\alpha}\sup_{\tau\in[0,1]}\sum_{j=k}^{\infty}\delta_{q}^{\sup_{u}|\mathbf{M}^{(2)}(\tau,u)|}(j)=O(h^{2})$

and

$\displaystyle\widetilde{W}_{2,\alpha}:=\max_{k\geq
0}(k+1)^{\alpha}\sup_{\tau,u\in[0,1]}\sum_{j=k}^{\infty}\delta_{2}^{|\mathbf{M}^{(2)}(\tau,u)|}(j)=O(h^{2}).$

Using Theorem 6.2 of Zhang & Wu (2017) with $q>2$, $\alpha=1/2$ and
$l=\min\\{1,\log(\\#\mathcal{T}_{T,\kappa})\\}\leq 5\log(T)$ to
$\\{\mathbf{M}_{t}^{(2)}(\tau,\tau_{t})\\}_{\tau\in\mathcal{T}_{T,\kappa}}$,
we have

$\displaystyle\Pr(\sup_{\tau\in\mathcal{T}_{T,\kappa}}|\bm{\Gamma}(\tau)|>h^{2}\beta_{T}M/2)$
$\displaystyle\leq$
$\displaystyle\frac{MTl^{q/2}\widetilde{W}_{q,\alpha}^{q}}{(\beta_{T}h^{2}Th)^{q}}+M\exp\left(-\frac{M(\beta_{T}h^{2}Th)^{2}}{T\widetilde{W}_{2,\alpha}^{2}}\right)$
$\displaystyle\leq$ $\displaystyle M\left(T^{-(q-2)/2}+\exp(-\log T)\right)\to
0.$

The proof is now complete. ∎

###### Proof of Lemma B.7.

(1). Let
$\widehat{\bm{\eta}}(\tau):=[\widehat{\bm{\theta}}(\tau)^{\top},\widehat{\bm{\theta}}^{\star}(\tau)^{\top}]^{\top}$
and
$\bm{\eta}(\tau):=[\bm{\theta}(\tau)^{\top},h\bm{\theta}^{(1)}(\tau)^{\top}]^{\top}$.
By Lemma B.5 and the proof of Theorem 2.1, we have

$\displaystyle\sup_{\tau\in[0,1]}|\widehat{\bm{\eta}}(\tau)-\bm{\eta}(\tau)|=o_{P}(1).$

By the Taylor expansion, we have

$\displaystyle\widehat{\bm{\eta}}(\tau)-\bm{\eta}(\tau)=-(\widetilde{\bm{\Sigma}}(\tau)+\mathbf{R}_{T}(\tau))^{-1}\gradient\mathcal{L}_{\tau}(\bm{\eta}(\tau)),$

where
$\mathbf{R}_{T}(\tau):=\gradient^{2}\mathcal{L}_{\tau}(\overline{\bm{\eta}})-\widetilde{\bm{\Sigma}}(\tau)$
and $\widetilde{\bm{\Sigma}}(\tau):=\left[\begin{matrix}1&0\\\
0&\widetilde{c}_{2}\end{matrix}\right]\otimes\bm{\Sigma}(\tau)$ with
$\overline{\bm{\eta}}$ between $\widehat{\bm{\eta}}(\tau)$ and
$\bm{\eta}(\tau)$. By Lemma B.3 and Lemma B.5, we have

$\displaystyle\sup_{\tau\in[0,1],\bm{\eta}\in\mathbf{E}_{T}(r)}|\gradient^{2}\mathcal{L}_{\tau}(\bm{\eta})-\widetilde{\bm{\Sigma}}(\tau,\bm{\eta})|$
$\displaystyle=$
$\displaystyle\sup_{\tau\in[0,1],\bm{\eta}\in\mathbf{E}_{T}(r)}|\gradient^{2}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta})-\widetilde{\bm{\Sigma}}(\tau,\bm{\eta})|+O_{P}((Th)^{-1})$
$\displaystyle=$
$\displaystyle\sup_{\tau\in[0,1],\bm{\eta}\in\mathbf{E}_{T}(r)}|E[\gradient^{2}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta})]-\widetilde{\bm{\Sigma}}(\tau,\bm{\eta})|+O_{P}((Th)^{-1}+\beta_{T})$
$\displaystyle=$ $\displaystyle O_{P}(\beta_{T}+(Th)^{-1})+O(h),$

where
$\widetilde{\bm{\Sigma}}(\tau,\bm{\eta}):=\int_{-\tau/h}^{(1-\tau)/h}K(u)\left[\begin{matrix}1&u\\\
u&u^{2}\end{matrix}\right]\otimes\bm{\Sigma}(\tau,\bm{\eta}_{1}+\bm{\eta}_{2}u)\mathrm{d}u$
and

$\bm{\Sigma}(\tau,\bm{\eta}_{1}+\bm{\eta}_{2}u):=E\left(\gradient_{\bm{\vartheta}}^{2}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\eta}_{1}+\bm{\eta}_{2}u)\right).$

By Lemma B.4.3 and the condition

$\displaystyle\sup_{\tau\in[0,1]}[\alpha_{j}(\bm{\theta}(\tau))+\beta_{j}(\bm{\theta}(\tau))]=O(j^{-(3+s)})$

for some $s>0$, we have
$\sup_{\tau\in[0,1]}\delta_{q}^{\gradient_{\vartheta_{j}}\mathcal{l}}(j)=O(j^{-(2+s)})$
for some $s>0$. By Lemma B.10, we have

$\sup_{\tau\in[0,1]}\left|T^{-1}\sum_{t=1}^{T}K_{h}(\tau_{t}-\tau)\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau_{t}),\widetilde{\mathbf{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))\right|=O_{P}((Th)^{-1/2}\log
T).$

Since
$E\left(\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\theta}(\tau))\right)=0$,
we further obtain that

$\displaystyle\sup_{\tau\in[0,1]}|\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))-E[\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))]|$
$\displaystyle\leq$
$\displaystyle\sup_{\tau\in[0,1]}|\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))-E[\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))]$
$\displaystyle-\frac{1}{Th}\sum_{t=1}^{T}\widehat{\bm{K}}((\tau_{t}-\tau)/h)\otimes\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))|$
$\displaystyle+\sup_{\tau\in[0,1]}\left|T^{-1}\sum_{t=1}^{T}K_{h}(\tau_{t}-\tau)\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau_{t}),\widetilde{\mathbf{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))\right|$
$\displaystyle=$ $\displaystyle O_{P}(h^{2}\beta_{T}+(Th)^{-1/2}\log T)$

using Lemma B.6.

Hence, by Lemma B.3.4, we have

$\displaystyle\sup_{\tau\in[0,1]}|\gradient\mathcal{L}_{\tau}(\bm{\eta}(\tau))|$
$\displaystyle\leq$
$\displaystyle\sup_{\tau\in[0,1]}|\gradient\mathcal{L}_{\tau}(\bm{\eta}(\tau))-\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))|+\sup_{\tau\in[0,1]}|\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))-E(\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau)))|$
$\displaystyle+\sup_{\tau\in[0,1]}|E(\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau)))|$
$\displaystyle=$
$\displaystyle\sup_{\tau\in[0,1]}|E(\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau)))|+O_{P}(h^{2}\beta_{T}+(Th)^{-1/2}\log
T+(Th)^{-1}).$

Since

$\displaystyle
E\left[\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))-\frac{1}{Th}\sum_{t=1}^{T}\widehat{\bm{K}}((\tau_{t}-\tau)/h)\otimes\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))\right]$
$\displaystyle=$
$\displaystyle-\frac{1}{2}h^{2}\frac{1}{Th}\sum_{t=1}^{T}\widehat{\bm{K}}((\tau_{t}-\tau)/h)\otimes\left[E[\gradient_{\bm{\vartheta}}^{2}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau),\widetilde{\bm{z}}_{t-1}(\tau);\bm{\theta}(\tau))]\bm{\theta}^{(2)}(\tau)\left(\frac{\tau_{t}-\tau}{h}\right)^{2}\right]+O(h^{3})$
$\displaystyle=$
$\displaystyle\frac{1}{2}h^{2}\int_{-\tau/h}^{(1-\tau)/h}K(u)[u^{2},u^{3}]^{\top}\mathrm{d}u\otimes\left(-\bm{\Sigma}(\tau)\bm{\theta}^{(2)}(\tau)\right)+O((Th)^{-1}+h^{3}),$

we have

$\sup_{\tau\in[0,1]}|\gradient_{\bm{\eta}_{j}}\mathcal{L}_{\tau}(\bm{\eta}(\tau))|=O_{P}(h^{2}\beta_{T}+(Th)^{-1/2}\log
T+(Th)^{-1}+h^{1+j}).$

for $j=1,2$. Hence, we have
$\sup_{\tau\in[0,1]}|\widehat{\bm{\eta}}_{j}(\tau)-\bm{\eta}_{j}(\tau)|=O_{P}(h^{2}\beta_{T}+(Th)^{-1/2}\log
T+(Th)^{-1}+h^{1+j})$ and
$\sup_{\tau\in[0,1]}|\mathbf{R}_{T}(\tau)|=O_{P}(\beta_{T}+h+(Th)^{-1})$,
where $\widehat{\bm{\eta}}_{j}(\tau)$ and $\bm{\eta}_{j}(\tau)$ are
corresponding to the $j^{th}$ part in their definitions given in the beginning
of this proof.

Write

$\displaystyle\left|-\widetilde{\bm{\Sigma}}(\tau)(\widehat{\bm{\eta}}(\tau)-\bm{\eta}(\tau))-\gradient\mathcal{L}_{\tau}(\bm{\eta}(\tau))\right|$
$\displaystyle\leq$
$\displaystyle|[\mathbf{I}_{2d}+\widetilde{\bm{\Sigma}}^{-1}(\tau)\mathbf{R}_{T}(\tau)]^{-1}-\mathbf{I}_{2d}^{-1}|\cdot|\gradient\mathcal{L}_{\tau}(\bm{\eta}(\tau))|$
$\displaystyle\leq$
$\displaystyle|[\mathbf{I}_{2d}+\widetilde{\bm{\Sigma}}^{-1}(\tau)\mathbf{R}_{T}(\tau)]^{-1}|\cdot|\widetilde{\bm{\Sigma}}^{-1}(\tau)\mathbf{R}_{T}(\tau)|\cdot|\gradient\mathcal{L}_{\tau}(\bm{\eta}(\tau))|$
$\displaystyle=$ $\displaystyle O_{P}(\gamma_{T}).$

The proof of the first result is now complete.

(2). By Lemma B.3 and Lemma B.6, we have

$\displaystyle\sup_{\tau\in[h,1-h]}\Big{|}\gradient_{\bm{\vartheta}}\mathcal{L}_{\tau}(\bm{\theta}(\tau),h\bm{\theta}^{(1)}(\tau))+\frac{1}{2}h^{2}\widetilde{c}_{2}\bm{\Sigma}(\tau)\bm{\theta}^{(2)}(\tau)$
$\displaystyle\quad-\frac{1}{T}\sum_{t=1}^{T}\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau_{t}),\widetilde{\mathbf{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))K_{h}(\tau_{t}-\tau)\Big{|}$
$\displaystyle\leq$
$\displaystyle\sup_{\tau\in[h,1-h]}|\gradient_{\bm{\eta}_{1}}\mathcal{L}_{\tau}(\bm{\eta}(\tau))-\gradient_{\bm{\eta}_{1}}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))|$
$\displaystyle+\sup_{\tau\in[h,1-h]}|\gradient_{\bm{\eta}_{1}}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))-E(\gradient_{\bm{\eta}_{1}}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau)))-\frac{1}{T}\sum_{t=1}^{T}\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau_{t}),\widetilde{\mathbf{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))K_{h}(\tau_{t}-\tau)|$
$\displaystyle+\sup_{\tau\in[h,1-h]}|E(\gradient_{\bm{\eta}_{1}}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau)))+\frac{1}{2}h^{2}\widetilde{c}_{2}\bm{\Sigma}(\tau)\bm{\theta}^{(2)}(\tau)|$
$\displaystyle=$
$\displaystyle\sup_{\tau\in[h,1-h]}|E(\gradient_{\bm{\eta}_{1}}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau)))+\frac{1}{2}h^{2}\widetilde{c}_{2}\bm{\Sigma}(\tau)\bm{\theta}^{(2)}(\tau)|+O_{P}((Th)^{-1}+\beta_{T}h^{2}).$

In addition, by the proof of the first result of this lemma, we have

$\sup_{\tau\in[h,1-h]}|E(\gradient_{\bm{\eta}_{1}}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau)))+\frac{1}{2}h^{2}\widetilde{c}_{2}\bm{\Sigma}(\tau)\bm{\theta}^{(2)}(\tau)|=O(h^{3}+(Th)^{-1}).$

The proof is now complete. ∎

###  B.4 Computation of the Local Linear ML Estimates

In our numerical studies, we use the function fminunc in programming language
MATLAB to minimize the negative of log-likelihood function. The initial guess
is important when using optimization functions because these optimizers are
trying to find a local minimum, i.e. the one closest to the initial guess that
can be achieved using derivatives. In this section, we give a possible choice
of initial estimates.

We could estimate the coefficients of time-varying VARMA($p,q$) model

$\mathbf{x}_{t}=\sum_{j=1}^{p}\mathbf{A}_{j}(\tau_{t})\mathbf{x}_{t-j}+\bm{\eta}_{t}+\sum_{j=1}^{q}\mathbf{B}_{j}(\tau_{t})\bm{\eta}_{t-j}\quad\text{with}\quad\bm{\eta}_{t}=\bm{\omega}(\tau_{t})\bm{\varepsilon}_{t},$

by kernel-weighted least squares method if the lagged $\bm{\eta}_{t}$ were
given. To obtain a preliminary estimator, we first fit a long VAR model and
then use estimated residuals in place of true residuals. Consider the
VAR$(p_{T})$ model

$\mathbf{x}_{t}=\sum_{j=1}^{p_{T}}\bm{\Gamma}_{j}(\tau_{t})\mathbf{x}_{t-j}+\bm{\eta}_{t},$

where $p_{T}$ is set to be $2(Th)^{1/3}$ in our numerical studies. Then, we
compute
$\widehat{\bm{\eta}}_{t}=\mathbf{x}_{t}-\sum_{j=1}^{p_{T}}\widehat{\bm{\Gamma}}_{j}(\tau_{t})\mathbf{x}_{t-j}$,
where $\\{\widehat{\bm{\Gamma}}_{j}(\tau)\\}$ are the local linear least
squares estimators. Given $\widehat{\bm{\eta}}_{t}$, we are able to estimate
$\\{\mathbf{A}_{j}(\tau)\\}$, $\\{\mathbf{B}_{j}(\tau)\\}$ and
$\bm{\Omega}(\tau)$ as well as their derivatives by local linear least squares
method.

In order to achieve identifications, certain restrictions should be imposed on
the coefficients of the VARMA model. Suppose there exists a known matrix
$\mathbf{R}$ and a vector $\bm{\gamma}(\tau)$ satisfying

$\mathrm{vec}(\mathbf{A}_{1}(\tau),...,\mathbf{A}_{p}(\tau),\mathbf{B}_{1}(\tau),...,\mathbf{B}_{q}(\tau))=\mathbf{R}\bm{\gamma}(\tau),$

which follows that

$\mathbf{x}_{t}\simeq(\mathbf{z}_{t-1}^{\top}\otimes\mathbf{I}_{m})\mathbf{R}[\bm{\gamma}(\tau)+\bm{\gamma}^{(1)}(\tau)(\tau_{t}-\tau)]+\bm{\eta}_{t},$

where
$\mathbf{z}_{t}=[\mathbf{x}_{t}^{\top},...,\mathbf{x}_{t-p+1}^{\top},\widehat{\bm{\eta}}_{t}^{\top},...,\widehat{\bm{\eta}}_{t-q}^{\top}]^{\top}$.
Then the local linear estimator of
$(\bm{\gamma}(\tau),\bm{\gamma}^{(1)}(\tau))$ is given by

$\left(\begin{matrix}\widehat{\bm{\gamma}}(\tau)\\\
h\widehat{\bm{\gamma}}^{(1)}(\tau)\end{matrix}\right)=\left(\sum_{t=1}^{T}\mathbf{R}^{\top}\mathbf{Z}_{t-1}^{*}\mathbf{Z}_{t-1}^{*\top}\mathbf{R}K_{h}(\tau_{t}-\tau)\right)^{-1}\sum_{t=1}^{T}\mathbf{R}^{\top}\mathbf{Z}_{t-1}^{*}\mathbf{x}_{t}K_{h}(\tau_{t}-\tau),$

where
$\mathbf{Z}_{t}^{*}=\mathbf{z}_{t}\otimes\mathbf{I}_{m}\otimes[1,\frac{\tau_{t+1}-\tau}{h}]^{\top}$.
Similarly, the local linear estimator of
$(\mathrm{vech}(\bm{\Omega}(\tau)),\mathrm{vech}(\bm{\Omega}^{(1)}(\tau)))$ is
given by

$\left(\begin{matrix}\mathrm{vech}(\widehat{\bm{\Omega}}(\tau))\\\
h\mathrm{vech}(\widehat{\bm{\Omega}}^{(1)}(\tau))\end{matrix}\right)=\left(\sum_{t=1}^{T}\mathbf{Z}_{t}\mathbf{Z}_{t}^{\top}K_{h}(\tau_{t}-\tau)\right)^{-1}\sum_{t=1}^{T}\mathbf{Z}_{t}\mathrm{vech}(\widehat{\bm{\eta}}_{t}\widehat{\bm{\eta}}_{t}^{\top})K_{h}(\tau_{t}-\tau),$

where
$\mathbf{Z}_{t}=[1,\frac{\tau_{t}-\tau}{h}]^{\top}\otimes\mathbf{I}_{m(m+1)/2}$.

We next consider the preliminary Estimation of Multivariate GARCH Models.
Define $\mathbf{y}_{t}=\mathbf{x}_{t}\odot\mathbf{x}_{t}$ and
$\mathbf{v}_{t}=\mathbf{y}_{t}-\mathbf{h}_{t}$. We can rewrite model (1.4) as

$\mathbf{y}_{t}=\mathbf{c}_{0}(\tau_{t})+\sum_{j=1}^{\max(p,q)}(\mathbf{C}_{j}(\tau_{t})+\mathbf{D}_{j}(\tau_{t}))\mathbf{y}_{t-j}+\mathbf{v}_{t}+\sum_{j=1}^{q}(-\mathbf{D}_{j}(\tau_{t}))\mathbf{v}_{t-j}$

with $E(\mathbf{v}_{t}|\mathcal{F}_{t-1})=0$. Similar to the VARMA model, we
are able to estimate $\mathbf{c}_{0}(\tau)$, $\\{\mathbf{C}_{j}(\tau)\\}$ and
$\\{\mathbf{D}_{j}(\tau)\\}$ as well as their derivatives by local linear
least squares method. Consider the VAR$(p_{T})$ model

$\mathbf{y}_{t}=\sum_{j=1}^{p_{T}}\bm{\Phi}_{j}(\tau_{t})\mathbf{y}_{t-j}+\mathbf{v}_{t},$

where $p_{T}$ is set to be $2(Th)^{1/3}$ in our numerical studies. Then, we
compute
$\widehat{\mathbf{v}}_{t}=\mathbf{y}_{t}-\sum_{j=1}^{p_{T}}\widehat{\bm{\Phi}}_{j}(\tau_{t})\mathbf{p}_{t-j}$,
$\widehat{\mathbf{h}}_{t}=\mathbf{y}_{t}-\widehat{\mathbf{v}}_{t}$ and
$\widehat{\bm{\eta}}_{t}=\mathrm{diag}^{-1/2}(\widehat{\mathbf{h}}_{t})\mathbf{x}_{t}$.
Hence, the local linear estimator of
$(\mathrm{vechl}(\bm{\Omega}(\tau)),\mathrm{vechl}(\bm{\Omega}^{(1)}(\tau)))$
is given by

$\left(\begin{matrix}\mathrm{vechl}(\widehat{\bm{\Omega}}(\tau))\\\
h\mathrm{vechl}(\widehat{\bm{\Omega}}^{(1)}(\tau))\end{matrix}\right)=\left(\sum_{t=1}^{T}\mathbf{Z}_{t}\mathbf{Z}_{t}^{\top}K_{h}(\tau_{t}-\tau)\right)^{-1}\sum_{t=1}^{T}\mathbf{Z}_{t}\mathrm{vechl}(\widehat{\bm{\eta}}_{t}\widehat{\bm{\eta}}_{t}^{\top})K_{h}(\tau_{t}-\tau),$

where
$\mathbf{Z}_{t}=[1,\frac{\tau_{t}-\tau}{h}]^{\top}\otimes\mathbf{I}_{m(m-1)/2}$
and $\mathrm{vechl}(\cdot)$ stacks the lower triangular part of a square
matrix excluding the diagonal.