$\displaystyle\big{\|}\sup_{\theta\in\Theta}|g(\hat{Z},\theta)-g(\hat{Z}^{\prime},\theta)|\big{\|}_{q}$
$\displaystyle\leq$
$\displaystyle\bar{C}^{(s)}\big{\|}|\hat{Z}-\hat{Z}^{\prime}|_{\chi^{(s)},s}\cdot(R_{M,M}(\hat{Z})+R_{M,M}(\hat{Z}^{\prime}))\big{\|}_{q}$
$\displaystyle\quad\quad+\bar{C}^{(s)}\big{\|}|\hat{Z}-\hat{Z}^{\prime}|_{\chi^{(s)},1}\cdot(R_{M-1,M-1}(\hat{Z})^{1+s}+R_{M-1,M-1}(\hat{Z}^{\prime})^{1+s})\big{\|}_{q}.$

The second summand can be dealt with as in the proof of Lemma 8.6(i), giving
the upper bound

$2\bar{C}^{(s)}(1+D(|\chi^{(s)}|_{1}+1))^{(M-1)(1+s)}\big{(}\|\hat{Y}-\hat{Y}^{\prime}\|_{qM}+\sum_{j=1}^{\infty}\chi_{j}^{(s)}\|\hat{X}_{j}-\hat{X}_{j}^{\prime}\|_{qM}\big{)}.$

For the first summand, we obtain with Hölder’s inequality:

$\displaystyle\big{\|}|\hat{Z}-\hat{Z}^{\prime}|_{\hat{\chi}^{(s)},s}^{s}\cdot(R_{M,M}(\hat{Z})+R_{M,M}(\hat{Z}^{\prime}))\big{\|}_{q}$
$\displaystyle\leq$
$\displaystyle\sum_{j=0}^{\infty}\hat{\chi}_{j}^{(s)}\|\hat{Z}_{j}-\hat{Z}_{j}^{\prime}\|_{q(M+s)}^{s}\cdot\big{(}\|R_{M,M}(\hat{Z})\|_{q(M+s)/M}+\|R_{M,M}(\hat{Z}^{\prime})\|_{q(M+s)/M}\big{)},$

giving the result since $M+s\leq M(1+s)$ and thus
$\|R_{M,M}(\hat{Z})\|_{q(M+s)/M}\leq(1+D(|\chi^{(s)}|_{1}+1))^{M}$.

######  Lemma 8.10 (for tvGARCH).

Let $q>0,s>0,\iota>0$. Let $\hat{X},\hat{X}^{\prime},\hat{Y},\hat{Y}^{\prime}$
be as in Lemma 8.6 satisfying (8.31). Let
$g\in\mathcal{H}_{s,\iota}^{mult}(M_{y},M_{x},\chi,\bar{C})$. Let $\zeta_{0}$
be independent of $\hat{X}$, $\hat{X}^{\prime}$ with $\|\zeta_{0}\|_{qM}\leq
D$. Then there exists some constant $C>0$ only dependent on $M$, $D$, $\chi$,
$\bar{C}$ such that

$\displaystyle\|\sup_{|\theta-\tilde{\theta}|<\iota}|\tilde{g}_{\tilde{\theta}}(\zeta_{0},\hat{X},\theta)-\tilde{g}_{\tilde{\theta}}(\zeta_{0},\hat{X}^{\prime},\theta)|\|_{q}$
$\displaystyle\leq$ $\displaystyle\bar{C}\cdot
C\sum_{j=1}^{\infty}\chi_{j}\|\hat{X}_{j}-\hat{X}_{j}^{\prime}\|_{qM},$ (8.41)
$\displaystyle\big{\|}\sup_{\theta\not=\theta^{\prime},|\theta-\tilde{\theta}|_{1}<\iota,|\theta^{\prime}-\tilde{\theta}|_{1}<\iota}\frac{|\tilde{g}_{\tilde{\theta}}(\zeta_{0},\hat{X},\theta)-\tilde{g}_{\tilde{\theta}}(\zeta_{0},\hat{X},\theta^{\prime})|}{|\theta-\theta^{\prime}|_{1}}\big{\|}_{q}$
$\displaystyle\leq$ $\displaystyle\bar{C}\cdot C,$ (8.42)
$\displaystyle\|\sup_{|\theta-\tilde{\theta}|_{1}<\iota}|\tilde{g}(\zeta_{0},\hat{X},\theta)|\|_{q}$
$\displaystyle\leq$ $\displaystyle\bar{C}\cdot C.$ (8.43)

######  Proof 8.11 (Proof of Lemma 8.10).

As in the proof of Lemma 8.6(i), we have:

$\|R_{M_{y}-1,M_{x}-1}(1,\hat{X})\|_{q(1+s)M/(M-1)}\leq(1+D(|\chi|_{1}+1))^{M-1}.$

Thus, with Hölder’s inequality,

$\displaystyle\big{\|}\sup_{|\theta-\tilde{\theta}|<\iota}|g(\zeta_{0},\hat{X},\theta)-g(\zeta_{0},\hat{X}^{\prime},\theta)|\big{\|}_{q}$
$\displaystyle\leq$ $\displaystyle
2\bar{C}(1+D(|\chi|_{1}+1))^{(M-1)(1+s)}(1+\|\zeta_{0}\|_{qM})\sum_{j=1}^{\infty}\chi_{j}\|\hat{X}_{j}-\hat{X}_{j}^{\prime}\|_{qM}.$

This shows (8.41). (8.43) follows from (8.41) since

$\|\sup_{|\theta-\tilde{\theta}|<\iota}|\tilde{g}_{\tilde{\theta}}(\zeta_{0},0,\theta)|\|_{q}\leq\bar{C}(1+\|\zeta_{0}\|_{qM}^{M})\leq\bar{C}(1+D^{M}).$

######  Lemma 8.12.

Assume that $\theta(\cdot)\in C^{2}[0,1]$. Let
$\chi^{\prime}=(\tilde{\chi}_{i}^{\prime})_{i\in\mathbb{N}}$ be an absolutely
summable sequence. Let
$g:\mathbb{R}\times\mathbb{R}^{\mathbb{N}}\times\Theta\to\mathbb{R}$ be
continuously differentiable. Suppose that either

  * (a)

Assumption 7.1(A5) holds with some $r\geq 2$ and Assumption 7.2(B3) holds.
Additionally, $g\in\mathcal{H}(M_{y}^{\prime},M_{x}^{\prime},\chi,\bar{C})$,
$\nabla_{\theta}g\in\mathcal{H}(M_{y}^{\prime},M_{x}^{\prime},\chi,\bar{C})$
and for all $l\in\mathbb{N}$,
$\partial_{x_{l}}g\in\mathcal{H}(M_{y}^{\prime}-1,M_{x}^{\prime}-1,\tilde{\chi},\bar{C}\chi_{l})$.

or

  * (b)

(for tvGARCH) There exists $s\geq 0$ such that Assumption 7.3(A5’) holds with
$r\geq 2(1+s)$ and Assumption 7.4(B3’) holds. Additionally,
$g\in\mathcal{H}_{s,\iota}^{mult}(M_{y}^{\prime},M_{x}^{\prime},\chi,\bar{C})$,
$\nabla_{\theta}g\in\mathcal{H}_{s,\iota}^{mult}(M_{y}^{\prime},M_{x}^{\prime},\chi,\bar{C})$
and for all $l\in\mathbb{N}$,
$\partial_{x_{l}}g\in\mathcal{H}_{s,\iota}^{mult}(M_{y}^{\prime}-1,M_{x}^{\prime}-1,\tilde{\chi},\bar{C}\chi_{l})$.

Define $M^{\prime}:=\max\\{M_{y}^{\prime},M_{x}^{\prime}\\}$. Then

  * (i)

$\sup_{t\in[0,1]}\|\partial_{t}g(\tilde{Z}_{0}(t),\theta(t))\|_{1}<\infty$,

  * (ii)

$\sup_{t\not=t^{\prime}}\frac{\|\partial_{t}g(\tilde{Z}_{0}(t),\theta(t))-\partial_{t}g(\tilde{Z}_{0}(t^{\prime}),\theta(t^{\prime}))\|_{1}}{|t-t^{\prime}|}<\infty.$

######  Proof 8.13 (Proof of Lemma 8.12).

(i) Note that

$\partial_{t}g(\tilde{Z}_{0}(t),\theta(t))=\partial_{z}g(\tilde{Z}_{0}(t),\theta(t))\partial_{t}\tilde{Z}_{0}(t)+\nabla_{\theta}g(\tilde{Z}_{0}(t),\theta(t))\theta^{\prime}(t).$
(8.44)

By Lemma 8.6 (if Assumption 7.1 holds) or Lemma 8.10 (if Assumption 7.3
holds),

$\sup_{t}\|\nabla_{\theta}g(\tilde{Z}_{0}(t),\theta(t))\|_{1}<\infty.$

and there exists a constant $C>0$ such that for each $j\in\mathbb{N}_{0}$,

$\|\partial_{z_{j}}g(\tilde{Z}_{0}(t),\theta(t))\|_{M^{\prime}/(M^{\prime}-1)}\leq
C\bar{C}\hat{\chi}_{j}.$ (8.45)

It follows that

$\displaystyle\|\partial_{z}g(\tilde{Z}_{0}(t),\theta(t))\partial_{t}\tilde{Z}_{0}(t)\|_{1}$
$\displaystyle\leq$
$\displaystyle\sum_{j=0}^{\infty}\|\partial_{z_{j}}g(\tilde{Z}_{0}(t),\theta(t))\|_{M^{\prime}/(M^{\prime}-1)}\cdot\|\partial_{t}\tilde{Z}_{0j}(t)\|_{M^{\prime}}$
$\displaystyle\leq$ $\displaystyle
CD\bar{C}\sum_{j=0}^{\infty}\hat{\chi}_{j}<\infty,$

which shows the assertion.

(ii) Let $t,t^{\prime}\in[0,1]$. From Lemma 8.6 (if Assumption 7.1 holds) or
Lemma 8.10 (if Assumption 7.3 holds) we obtain with some constant $C>0$, for
each $k=1,...,d_{\Theta}$,

$\displaystyle\|\nabla_{\theta_{k}}g(\tilde{Z}_{0}(t),\theta(t))\|_{1}$
$\displaystyle\leq$ $\displaystyle C,$
$\displaystyle\|\nabla_{\theta_{k}}g(\tilde{Z}_{0}(t),\theta(t))-\nabla_{\theta_{k}}g(\tilde{Z}_{0}(t),\theta(t^{\prime}))\|_{1}$
$\displaystyle\leq$ $\displaystyle C|\theta(t)-\theta(t^{\prime})|_{1}.$

From Lemma 8.6 (if Assumption 7.1 holds) or Lemma 8.10 (if Assumption 7.3
holds) we obtain for each $k=1,...,d_{\Theta}$ (note that $rM\geq M^{\prime}$
in Assumption 7.2):

$\displaystyle\|\nabla_{\theta_{k}}g(\tilde{Z}_{0}(t),\theta(t^{\prime}))-\nabla_{\theta_{k}}g(\tilde{Z}_{0}(t^{\prime}),\theta(t^{\prime}))\|_{1}$
(8.46) $\displaystyle\leq$
$\displaystyle\bar{C}C\big{(}\|\tilde{Y}_{0}(t)-\tilde{Y}_{0}(t^{\prime})\|_{M^{\prime}}+\sum_{j=1}^{\infty}\chi_{j}\|\tilde{X}_{0}(t)-\tilde{X}_{0}(t^{\prime})\|_{M^{\prime}}\big{)}$
$\displaystyle\leq$ $\displaystyle\bar{C}CC_{B}|t-t^{\prime}|(1+|\chi|_{1}).$

By Lipschitz continuity of $\theta$, $\theta^{\prime}$, the above results
imply that the second summand in (8.44) fulfills the assertion,

$\sup_{t\not=t^{\prime}}\frac{\|\nabla_{\theta}g(\tilde{Z}_{0}(t),\theta(t))\theta^{\prime}(t)-\nabla_{\theta}g(\tilde{Z}_{0}(t^{\prime}),\theta(t^{\prime}))\theta^{\prime}(t^{\prime})\|_{1}}{|t-t^{\prime}|}<\infty.$

It remains to show the same for the first summand in (8.44). By (8.45) and
$\|\partial_{t}\tilde{Z}_{0j}(t)-\partial_{t}\tilde{Z}_{0j}(t^{\prime})\|_{M^{\prime}}\leq
C_{B}|t-t^{\prime}|$ from Assumption 7.2(B3), we have

$\|\partial_{z}g(\tilde{Z}_{0}(t),\theta(t))(\partial_{t}\tilde{Z}_{0}(t)-\partial_{t}\tilde{Z}_{0}(t^{\prime}))\|_{1}\leq
C\bar{C}C_{B}|\chi|_{1}|t-t^{\prime}|.$ (8.47)

Similar as in (8.46), we see by Lemma 8.6 (if Assumption 7.1 holds) or Lemma
8.10 (if Assumption 7.3 holds) that

$\|\partial_{z_{j}}g(\tilde{Z}_{0}(t),\theta(t))-\partial_{z_{j}}g(\tilde{Z}_{0}(t^{\prime}),\theta(t))\|_{M^{\prime}/(M^{\prime}-1)}\leq\chi_{j}\bar{C}CC_{B}(1+|\chi^{\prime}|_{1})|t-t^{\prime}|.$
(8.48)

Finally, by Lemma 8.6 (if Assumption 7.1 holds) or Lemma 8.10 (if Assumption
7.3 holds) and Lipschitz continuity of $\theta$, we have

$\|\partial_{z_{j}}g(\tilde{Z}_{0}(t^{\prime}),\theta(t))-\partial_{z_{j}}g(\tilde{Z}_{0}(t^{\prime}),\theta(t^{\prime}))\|_{M^{\prime}/(M^{\prime}-1)}\leq\chi_{j}\bar{C}C|\theta(t)-\theta(t^{\prime})|_{1}=O(|t-t^{\prime}|).$
(8.49)

By Hölder’s inequality, we conclude from (8.48) and (8.49) that

$\|(\partial_{z}g(\tilde{Z}_{0}(t),\theta(t))-\partial_{z}g(\tilde{Z}_{0}(t^{\prime}),\theta(t^{\prime})))\partial_{t}\tilde{Z}_{0}(t^{\prime})\|_{1}=O(|t-t^{\prime}|),$

which together with (8.47), finishes the proof.

######  Lemma 8.14.

Let $q\geq 1$. Suppose that Assumption 7.1(A5), (A7) hold with some $r\geq q$.
Let $g\in\mathcal{H}(M_{y},M_{x},\chi,\bar{C})$, where
$\chi_{i}=O(i^{-(1+\gamma)})$. Then it holds that

  1. (i)

$\sup_{t\in[0,1]}\delta_{q}^{\sup_{\theta}|g(\tilde{Z}(t),\theta)|}(j)=O(j^{-(1+\gamma)})$.

  2. (ii)

For
$M_{i}(t,\eta,u):=\hat{K}_{b_{n}}(u-t)g(\tilde{Z}_{i}(u),\eta_{1}+\eta_{2}(u-t)b_{n}^{-1})$,
we have

$\sup_{u\in[0,1]}\sup_{t,\eta}\delta_{q}^{M(t,\eta,u)}(j)=O(j^{-(1+\gamma)}),\quad\quad\sup_{u\in[0,1]}\delta^{\sup_{t,\eta}|M(t,\eta,u)|}_{q}(j)=O(j^{-(1+\gamma)}).$

  3. (iii)

Let $d_{u}(t)=\theta(u)-\theta(t)-(u-t)\theta^{\prime}(t)$ and
$M_{i}^{(2)}(t,u):=\hat{K}_{b_{n}}(u-t)\\{\int_{0}^{1}g(\tilde{Z}_{i}(u),\theta(t)+sd_{u}(t))ds\\}\cdot
d_{u}(t)$. Then it holds for each component that

$\sup_{u\in[0,1]}\delta_{q}^{M^{(2)}(t,u)}(j)=O(b_{n}^{2}j^{-(1+\gamma)}),\quad\quad\sup_{u\in[0,1]}\delta^{\sup_{t}|M^{(2)}(t,u)|}_{q}(j)=O(b_{n}^{2}j^{-(1+\gamma)}).$

(*) If instead Assumption 7.3(A5’), (A7’) hold with some $r>q$ and $g=\ell$
fulfills (7.6) for all $s>0$ small enough,then the statements above remain
valid.

######  Proof 8.15.

(i) Let $\tilde{Z}_{j}(t)^{*}$ be a coupled version of $\tilde{Z}_{j}(t)$
where $\zeta_{0}$ is replaced by $\zeta_{0}^{*}$. By Lemma 8.6 we obtain in
case (7.4) that with some constant $\tilde{C}>0$:

$\displaystyle\delta^{\sup_{\theta}|g(\tilde{Z}(t),\theta)|}_{q}(j)$ (8.50)
$\displaystyle=$
$\displaystyle\|\sup_{\theta}|g(\tilde{Z}_{j}(t),\theta)|-\sup_{\theta}|g(\tilde{Z}_{j}(t)^{*},\theta)|\|_{q}$
$\displaystyle\leq$
$\displaystyle\|\sup_{\theta}|g(\tilde{Z}_{j}(t),\theta)-g(\tilde{Z}_{j}(t)^{*},\theta)|\|_{q}$
$\displaystyle\leq$
$\displaystyle\tilde{C}\Big{(}\|\tilde{Y}_{j}(t)-\tilde{Y}_{j}(t)^{*}\|_{qM}+\sum_{i=1}^{\infty}\chi_{i}\|\tilde{X}_{j-i+1}(t)-\tilde{X}_{j-i+1}(t)^{*}\|_{qM}\Big{)}$
$\displaystyle\leq$
$\displaystyle\tilde{C}\Big{(}\delta_{qM}^{\tilde{Y}(t)}(j)+\sum_{i=1}^{\infty}\chi_{i}\delta_{qM}^{\tilde{X}(t)}(j-i+1)\Big{)},$

and in case (7.5), similarly

$\delta_{q}^{g(\tilde{Z}(t),\theta)}(j)\leq\tilde{C}\sum_{i=1}^{\infty}\chi_{i}\delta_{qM}^{\tilde{X}(t)}(j-i+1).$
(8.51)

In case (*), let $s>0$ be such that $q(1+s)<r$. Then we have by Lemma 8.8:

$\displaystyle\delta_{q}^{g(\tilde{Z}(t),\theta)}(j)$ $\displaystyle\leq$
$\displaystyle\tilde{C}\sum_{i=0}^{\infty}\hat{\hat{\chi}}_{i}^{(s)}\big{(}\|\tilde{Z}_{i}(t)-\tilde{Z}_{i}(t)^{*}\|_{qM}+\|\tilde{Z}_{i}(t)-\tilde{Z}_{i}(t)^{*}\|_{qM(1+s)}^{s}\big{)}$
(8.52) $\displaystyle\leq$
$\displaystyle\tilde{C}\sum_{i=0}^{\infty}\chi_{i}^{(s)}\big{(}\delta_{qM}^{\tilde{X}(t)}(j-i+1)+[\delta_{qM(1+s)}^{\tilde{X}(t)}(j-i+1)]^{s}\big{)}.$

Note that if two sequences $a_{i},b_{i}$ with $a_{i}=b_{i}=0$ for $i<0$ obey
$a_{i},b_{i}=O(i^{-(1+\gamma)})$ then the convolution
$c_{j}=\sum_{i=1}^{\infty}a_{i}b_{j-i+1}$ still obeys
$c_{j}=O(j^{-(1+\gamma)})$ due to

$\displaystyle|c_{j}|$ $\displaystyle\leq$
$\displaystyle\sum_{i=1,i\geq(j+1)/2}^{j+1}|a_{i}|\cdot|b_{j-i+1}|+\sum_{i=1,|j-i|\geq(j+1)/2}^{j+1}|a_{i}||b_{j-i+1}|$
$\displaystyle\leq$
$\displaystyle\big{(}\frac{j+1}{2}\big{)}^{-(1+\gamma)}\sum_{i=1}^{j+1}|b_{j-i+1}|+\big{(}\frac{j+1}{2}\big{)}^{-(1+\gamma)}\sum_{i=1}^{j+1}|a_{i}|=O(j^{-(1+\gamma)}).$

Together with Assumption (A7) and (8.50), (8.51) or (in case (*)) Assumption
7.3(A7’) and (8.52), this shows
$\sup_{t\in[0,1]}\delta_{r}^{g(\tilde{Z}(t),\theta)}(j)=O(j^{-(1+\gamma)})$.

The proof for (ii),(iii) is the same since

$\displaystyle\big{|}\sup_{t,\eta}|M_{i}(t,\eta,u)|-\sup_{t,\eta}|M_{i}(t,\eta,u)^{*}|\big{|}$
$\displaystyle\leq$
$\displaystyle\sup_{t,\eta}|M_{i}(t,\eta,u)-M_{i}(t,\eta,u)^{*}|$
$\displaystyle\leq$
$\displaystyle|\hat{K}|_{\infty}\sup_{\theta}|g(\tilde{Z}_{i}(u),\theta)-g(\tilde{Z}_{i}(u)^{*},\theta)|$

and (since
$|d_{u}(t)|_{\infty}\leq\sup_{s}|\theta^{\prime\prime}(s)|_{\infty}\cdot
b_{n}^{2}$ if $|t-u|\leq b_{n}$), for each $l=1,\ldots,k$,

$\displaystyle\big{|}\sup_{t}|\tilde{M}_{i}^{(2)}(t,u)_{l}|-\sup_{t}|\tilde{M}_{i}^{(2)}(t,u)_{l}^{*}|\big{|}\leq\sup_{t}|M_{i}^{(2)}(t,u)_{l}-M_{i}^{(2)}(t,u)_{l}^{*}|$
$\displaystyle\leq$
$\displaystyle|\hat{K}|_{\infty}\sup_{s}|\theta^{\prime\prime}(s)|_{\infty}b_{n}^{2}$
$\displaystyle\quad\quad\times\sup_{t}\int_{0}^{1}|g(\tilde{Z}_{i}(u),\theta(t)+sd_{u}(t))-g(\tilde{Z}_{i}(u)^{*},\theta(t)+sd_{u}(t))|ds$
$\displaystyle\leq$
$\displaystyle|\hat{K}|_{\infty}\sup_{s}|\theta^{\prime\prime}(s)|_{\infty}b_{n}^{2}\sup_{\theta\in\Theta}|g(\tilde{Z}_{i}(u),\theta)-g(\tilde{Z}_{i}(u)^{*},\theta)|.$

######  Lemma 8.16 (for tvGARCH).

Let $q\geq 1$. Suppose that Assumption 7.3(A5’), (A7’) hold with some $r>q$.
For $s>0$, let $\chi^{(s)}=(\chi_{i}^{(s)})_{i\in\mathbb{N}}$ be such that
$\chi_{i}^{(s)}=O(i^{-(1+\gamma)})$. Let $g$ be such that
$\tilde{g}_{\tilde{\theta}}(y,x,\theta):=g(F(x,\tilde{\theta},y),x,\theta)$
fulfills
$\tilde{g}\in\mathcal{H}_{s,\iota}^{mult}(M_{y},M_{x},\chi^{(s)},\bar{C}^{(s)})$
for all $s>0$ small enough. Then

  1. (i)

$\sup_{t\in[0,1]}\delta_{q}^{\sup_{|\theta-\theta(t)|_{1}<\iota}|g(\tilde{Z}(t),\theta)|}(j)=O(j^{-(1+\gamma)})$.

  2. (ii)

For $n$ large enough,

$\sup_{u\in[0,1]}\sup_{t,|\eta-\eta_{b_{n}}(t)|_{1}<\iota/2}\delta_{q}^{M(t,\eta,u)}(j)=O(j^{-(1+\gamma)}),\quad\quad\sup_{u\in[0,1]}\delta^{\sup_{t,|\eta-\eta_{b_{n}}(t)|_{1}<\iota/2}|M(t,\eta,u)|}_{q}(j)=O(j^{-(1+\gamma)}).$

  3. (iii)

For $n$ large enough,
$\sup_{u\in[0,1]}\delta_{q}^{M^{(2)}(t,u)}(j)=O(b_{n}^{2}j^{-(1+\gamma)})$,
and
$\sup_{u\in[0,1]}\delta^{\sup_{t}|M^{(2)}(t,u)|}_{q}(j)=O(b_{n}^{2}j^{-(1+\gamma)})$.

######  Proof 8.17 (Proof of Lemma 8.16).

(i) Let $\tilde{Z}_{j}(t)^{*}$ be a coupled version of $\tilde{Z}_{j}(t)$
where $\zeta_{0}$ is replaced by $\zeta_{0}^{*}$. By Lemma 8.10 we obtain that
with some constant $\tilde{C}>0$:

$\displaystyle\delta^{\sup_{|\theta-\theta(t)|_{1}<\iota}|g(\tilde{Z}(t),\theta)|}_{q}(j)$
$\displaystyle\leq$
$\displaystyle\|\sup_{|\theta-\theta(t)|_{1}<\iota}|\tilde{g}_{\theta(t)}(\zeta_{j},\tilde{X}_{j}(t),\theta)-\tilde{g}_{\theta(t)}(\zeta_{j},\tilde{X}_{j}(t)^{*},\theta)|\|_{q}$
$\displaystyle\leq$
$\displaystyle\tilde{C}\sum_{i=1}^{\infty}\chi_{i}\|\tilde{X}_{j-i+1}(t)-\tilde{X}_{j-i+1}(t)^{*}\|_{qM}$
$\displaystyle\leq$
$\displaystyle\tilde{C}\sum_{i=1}^{\infty}\chi_{i}\delta_{qM}^{\tilde{X}(t)}(j-i+1).$

The result now follows as in the proof of Lemma 8.14(i) with Assumption
7.3(A7’).

(ii) We have for $n$ large enough that

$|\eta-\eta_{b_{n}}(t)|_{1}=|\eta_{1}-\theta(t)|_{1}+|\eta_{2}-b_{n}\theta^{\prime}(t)|_{1}<\iota/2\quad\text{
implies
}\quad|(\eta_{1}+\eta_{2}(u-t)b_{n}^{-1})-\theta(t)|_{1}\leq|\eta_{1}-\theta(t)|_{1}+|\eta_{2}|_{1}<\iota$

and $|\theta-\theta(t)|_{1}<\iota$, $|u-t|\leq b_{n}$ implies
$|\theta-\theta(u)|_{1}<\iota$ due to uniform continuity of $\theta(\cdot)$.
Therefore, we have for $n$ large enough:

$\displaystyle\big{|}\sup_{t,|\eta-\eta_{b_{n}}(t)|_{1}<\iota/2}|M_{i}(t,\eta,u)|-\sup_{t,|\eta-\eta_{b_{n}}(t)|_{1}<\iota/2}|M_{i}(t,\eta,u)^{*}|\big{|}$
$\displaystyle\leq$
$\displaystyle\sup_{t,|\eta-\eta_{b_{n}}(t)|_{1}<\iota/2}|\hat{K}_{b_{n}}(u-t)|\cdot|g(\tilde{Z}_{i}(u),\eta_{1}+\eta_{2}(u-t)b_{n}^{-1})-g(\tilde{Z}_{i}(u),\eta_{1}+\eta_{2}(u-t)b_{n}^{-1})|$
$\displaystyle\leq$
$\displaystyle\sup_{t,|\theta-\theta(t)|_{1}<\iota}|\hat{K}_{b_{n}}(u-t)|\cdot|\tilde{g}_{\theta(u)}(\zeta_{i},\tilde{X}_{i}(u),\theta)-\tilde{g}_{\theta(u)}(\zeta_{i},\tilde{X}_{i}(u)^{*},\theta)|$
$\displaystyle\leq$
$\displaystyle|\hat{K}|_{\infty}\cdot\sup_{|\theta-\theta(u)|_{1}<\iota}|\tilde{g}_{\theta(u)}(\zeta_{i},\tilde{X}_{i}(u),\theta)-\tilde{g}_{\theta(u)}(\zeta_{i},\tilde{X}_{i}(u)^{*},\theta)|.$

The rest works as in (i).
(iii) For $n$ large enough, it holds that $|u-t|\leq b_{n}$ implies that
$\sup_{s\in[0,1]}|\theta(t)+sd_{u}(t)-\theta(u)|<\iota$ due to uniform
continuity of $\theta(\cdot)$. Thus

$\displaystyle\big{|}\sup_{t}|M_{i}^{(2)}(t,u)|-\sup_{t}|M_{i}^{(2)}(t,u)^{*}|\big{|}$
$\displaystyle\leq$
$\displaystyle|\hat{K}|_{\infty}\sup_{s}|\theta^{\prime\prime}(s)|_{\infty}b_{n}^{2}\sup_{|\theta-\theta(u)|<\iota}|\tilde{g}_{\theta(u)}(\zeta_{i},\tilde{X}_{i}(u),\theta)-\tilde{g}_{\theta(u)}(\zeta_{i},\tilde{X}_{i}(u)^{*},\theta)|.$

The rest works as in (i).

######  Lemma 8.18 (Lipschitz properties of $\hat{G}_{n}$).

Let $s\geq 0$.

  1. (i)

Let $g\in\mathcal{H}_{s}(M_{y},M_{x},\chi,\bar{C})$. Let Assumption 7.1(A5)
hold with $r\geq 1+s$. Then there exists some constant $\tilde{C}>0$ such that

$\sup_{t\in[0,1]}\Big{\|}\sup_{\eta\not=\eta^{\prime}}\frac{|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t,\eta^{\prime})|}{|\eta-\eta^{\prime}|_{1}}\Big{\|}_{1}\leq\tilde{C},$

and

$\Big{\|}\sup_{t\not=t^{\prime}}\sup_{\eta\not=\eta^{\prime}}\frac{|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t^{\prime},\eta^{\prime})|}{|t-t^{\prime}|+|\eta-\eta^{\prime}|_{1}}\Big{\|}_{1}\leq\tilde{C}b_{n}^{-2},$

  2. (ii)

(for tvGARCH) Let $g$ be such that
$\tilde{g}_{\tilde{\theta}}(y,x,\theta):=g(F(x,\tilde{\theta},y),x,\theta)$
fulfills
$\tilde{g}\in\mathcal{H}_{s,\iota}^{mult}(M_{y},M_{x},\chi^{(s)},\bar{C}^{(s)})$
with $\chi_{i}^{(s)}=O(i^{-(1+\gamma)})$. Let Assumption 7.3(A5’) hold with
$r\geq 1+s$ and let $\theta(\cdot)$ be continuous. Then there exists some
constant $\tilde{C}^{(s)}>0$ such that

$\sup_{t\in[0,1]}\Big{\|}\underset{|\eta-\eta_{b_{n}}(t)|_{1}<\iota/2,|\eta^{\prime}-\eta_{b_{n}}(t)|_{1}<\iota/2}{\sup_{\eta\not=\eta^{\prime}}}\frac{|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t,\eta^{\prime})|}{|\eta-\eta^{\prime}|_{1}}\Big{\|}_{1}\leq\tilde{C}^{(s)},$

and

$\Big{\|}\sup_{t\not=t^{\prime}}\underset{|\eta-\eta_{b_{n}}(t)|_{1}<\iota/2,|\eta^{\prime}-\eta_{b_{n}}(t^{\prime})|_{1}<\iota/2}{\sup_{\eta\not=\eta^{\prime}}}\frac{|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t^{\prime},\eta^{\prime})|}{|t-t^{\prime}|+|\eta-\eta^{\prime}|_{1}}\Big{\|}_{1}\leq\tilde{C}^{(s)}b_{n}^{-2},$

######  Proof 8.19 (Proof of Lemma 8.18).

By Lemma 8.6(i),
$\sup_{t\in[0,1]}\|R_{M_{y},M_{x}}(\tilde{Z}_{0}(t))^{1+s}\|_{1}<\infty$. This
is needed several times in the following.
(i) Since $g\in\mathcal{H}_{s}(M_{y},M_{x},\chi,\bar{C})$ and $|i/n-t|\leq
b_{n}$ inside the sum, it holds that

$|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t,\eta^{\prime})|\leq\bar{C}|\eta-\eta^{\prime}|_{1}\cdot(nb_{n})^{-1}\sum_{i=1}^{n}|\hat{K}_{b_{n}}(i/n-t)|\\{R_{M_{y},M_{x}}(\tilde{Z}_{i}(i/n))^{1+s}+\|R_{M_{y},M_{x}}(\tilde{Z}_{i}(i/n))^{1+s}\|_{1}\\}.$
(8.53)

Furthermore,
$(nb_{n})^{-1}\sum_{i=1}^{n}|\hat{K}_{b_{n}}(i/n-t)|\leq|\hat{K}|_{\infty}$.
This yields the assertion.
Since $g\in\mathcal{H}_{s}(M_{y},M_{x},\chi,\bar{C})$, we have with some
constant $\tilde{C}>0$:

$\displaystyle|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t^{\prime},\eta^{\prime})|$
$\displaystyle\leq$
$\displaystyle(nb_{n})^{-1}\sum_{i=1}^{n}|\hat{K}_{b_{n}}(i/n-t)-\hat{K}_{b_{n}}(i/n-t^{\prime})|\cdot\sup_{\theta}\\{|g(\tilde{Z}_{i}(i/n),\theta)|+\|g(\tilde{Z}_{i}(i/n),\theta)\|_{1}\\}$
$\displaystyle+(nb_{n})^{-1}\sum_{i=1}^{n}|\hat{K}_{b_{n}}(i/n-t^{\prime})|\cdot|g(\tilde{Z}_{i}(i/n),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})-g(\tilde{Z}_{i}(i/n),\eta_{1}^{\prime}+\eta_{2}^{\prime}(i/n-t^{\prime})b_{n}^{-1})$
$\displaystyle\leq$
$\displaystyle\big{[}b_{n}^{-2}L_{\hat{K}}|t-t^{\prime}|+b_{n}^{-1}|\hat{K}|_{\infty}\\{|\eta-\eta^{\prime}|_{1}+|\eta_{2}|\cdot|t-t^{\prime}|b_{n}^{-1}\big{]}\cdot\frac{1}{n}\sum_{i=1}^{n}\\{R_{M_{y},M_{x}}(\tilde{Z}_{i}(i/n))^{1+s}$
$\displaystyle\quad\quad\quad\quad\quad\quad\quad\quad\quad+\|R_{M_{y},M_{x}}(\tilde{Z}_{i}(i/n))^{1+s}\|_{1}\\}.$

Since $\sup_{\eta\in E_{n}}|\eta_{2}|_{1}<\infty$ is compact, this gives the
result.
(ii) We now have

$\displaystyle|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t,\eta^{\prime})|$
$\displaystyle\leq$
$\displaystyle(nb_{n})^{-1}\sum_{i=1}^{n}|\hat{K}_{b_{n}}(i/n-t)|\cdot\big{|}\tilde{g}_{\theta(i/n)}(\zeta_{i},\tilde{X}_{i}(i/n),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})$
$\displaystyle\quad\quad\quad\quad\quad-\tilde{g}_{\theta(i/n)}(\zeta_{i},\tilde{X}_{i}(i/n),\eta_{1}^{\prime}+\eta_{2}^{\prime}(i/n-t)b_{n}^{-1})\big{|}.$

Here, $|\eta-\eta_{b_{n}}(t)|<\iota/2$ implies
$|(\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})-\theta(t)|<\iota$ for $n$ large enough.
Since $\theta(\cdot)$ is uniformly continuous, $|\theta-\theta(t)|_{1}<\iota$,
$|i/n-t|\leq b_{n}$ imply $|\theta-\theta(i/n)|_{1}<\iota$ for $n$ large
enough. Since
$\tilde{g}_{\tilde{\theta}}\in\mathcal{H}_{s,\iota}^{mult}(M_{y},M_{x},\chi^{(s)},\bar{C}^{(s)})$,
we obtain

$\displaystyle|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t,\eta^{\prime})|$
$\displaystyle\leq$
$\displaystyle\bar{C}^{(s)}|\eta-\eta^{\prime}|_{1}(nb_{n})^{-1}\sum_{i=1}^{n}|\hat{K}_{b_{n}}(i/n-t)|\cdot\\{R_{M,M}(1,\tilde{X}_{i}(i/n))^{1+s}(1+|\zeta_{i}|^{M})^{1+s}$
$\displaystyle\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad+\|R_{M,M}(1,\tilde{X}_{i}(i/n))^{1+s}(1+|\zeta_{i}|^{M})^{1+s}\|_{1}\\},$

giving the result.
We have

$\displaystyle|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t^{\prime},\eta^{\prime})|$
$\displaystyle\leq$
$\displaystyle(nb_{n})^{-1}\sum_{i=1}^{n}|\hat{K}_{b_{n}}(i/n-t)-\hat{K}_{b_{n}}(i/n-t^{\prime})|$
$\displaystyle\quad\quad\times\sup_{|\eta-\eta_{b_{n}}(t)|<\iota/2}\\{|\tilde{g}_{\theta(i/n)}(\zeta_{i},\tilde{X}_{i}(i/n),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})|+\|\tilde{g}_{\theta(i/n)}(\zeta_{i},\tilde{X}_{i}(i/n),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})\|_{1}\\}$
$\displaystyle+(nb_{n})^{-1}\sum_{i=1}^{n}|\hat{K}_{b_{n}}(i/n-t^{\prime})|\cdot|\tilde{g}_{\theta(i/n)}(\zeta_{i},\tilde{X}_{i}(i/n),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})$
$\displaystyle\quad\quad\quad\quad\quad-\tilde{g}_{\theta(i/n)}(\zeta_{i},\tilde{X}_{i}(i/n),\eta_{1}^{\prime}+\eta_{2}^{\prime}(i/n-t^{\prime})b_{n}^{-1}).$

The same argumentation as before allows us to use the Lipschitz properties of
$\tilde{g}_{\theta(i/n)}$ w.r.t. $\theta$, giving the result.

For the proof of the following lemma, we will make use of the adjusted
dependence measure $\|\cdot\|_{q,\alpha}$ which is defined as follows (cf.
[57]): For some zero-mean random variable $Z$, let
$\|Z\|_{q,\alpha}:=\sup_{m\geq 0}(m+1)^{\alpha}\Delta^{Z}_{q}(m)$.

######  Lemma 8.20.

Let $\gamma>1$. For $s\geq 0$, let
$\chi_{i}^{(s)}=(\chi_{i}^{(s)})_{i\in\mathbb{N}}$ be a sequence with
$\chi_{i}^{(s)}=O(i^{-(1+\gamma)})$. Recall the notation from (8.1). Assume
that either (in the case (a)) Assumption 7.1(A5), (A6), (A7) or (in the cases
(b),(c)) Assumption 7.3(A5’), (A6’), (A7’) hold with some $r$ specified below.

  1. (i)

Let $r\geq 1+\varsigma$, $\varsigma\geq 0$ and assume either that
$\varsigma=0$ and $g\in\mathcal{H}_{0}(M_{y},M_{x},\chi^{(0)},\bar{C}^{(0)})$
or $\varsigma>0$ and for all $s>0$ small enough,
$g\in\mathcal{H}_{s}(M_{y},M_{x},\chi^{(s)},\bar{C}^{(s)})$. Then

$\|\sup_{t\in(0,1)}\sup_{\eta\in
E_{n}}|\hat{G}_{n}(t,\eta)-G_{n}^{c}(t,\eta)|\|_{1}=O((nb_{n})^{-1}).$

  2. (ii)

Fix $t\in[0,1]$ and assume that $nb_{n}\to\infty$. Let $r\geq 1+\varsigma$,
$\varsigma>0$.
(a) If for all $s>0$ small enough,
$g\in\mathcal{H}_{s}(M_{y},M_{x},\chi^{(s)},\bar{C}^{(s)})$, then

$\sup_{\eta\in E_{n}}|\hat{G}_{n}(t,\eta)|=o_{\mathbb{P}}(1).$

(b) If for all $s>0$ small enough,
$\tilde{g}_{\tilde{\theta}}(y,x,\theta):=g(F(y,x,\tilde{\theta}),x,\theta)$
fulfills
$\tilde{g}\in\mathcal{H}^{mult}_{s,\iota}(M,\chi^{(s)},\bar{C}^{(s)})$, then

$\sup_{|\eta-\eta_{b_{n}}(t)|<\iota}|\hat{G}_{n}(t,\eta)|=o_{\mathbb{P}}(1)\quad\quad\text{if}\quad
b_{n}\to 0.$

(c) If for all $s>0$ small enough, $g$ fulfills (7.6) and
$g\in\mathcal{H}_{s}(2M_{y},2M_{x},\chi^{(s)},\bar{C}^{(s)})$, then

$\sup_{\eta\in E_{n}}|\hat{G}_{n}(t,\eta)|=o_{\mathbb{P}}(1).$

  3. (iii)

Let $r\geq 2+\varsigma$, $\varsigma>0$. Define
$\beta_{n}=\log(n)^{1/2}(nb_{n})^{-1/2}b_{n}^{-1/2}$.
(a) If for all $s>0$ small enough,
$g\in\mathcal{H}_{s}(M_{y},M_{x},\chi^{(s)},\bar{C}^{(s)})$, then

$\sup_{t\in(0,1)}\sup_{\eta\in
E_{n}}|\hat{G}_{n}(t,\eta)|=O_{\mathbb{P}}(\beta_{n}).$

(b) If $g$ is such that
$\tilde{g}_{\tilde{\theta}}(y,x,\theta):=g(F(y,x,\tilde{\theta}),x,\theta)$
and for all $s>0$ small enough,
$\tilde{g}\in\mathcal{H}^{mult}_{s,\iota}(M,\chi^{(s)},\bar{C}^{(s)})$, then

$\sup_{t\in(0,1)}\sup_{|\eta-\eta_{b_{n}}(t)|_{1}<\iota}|\hat{G}(t,\eta)|=O_{\mathbb{P}}(\beta_{n}).$

(c) If for all $s>0$ small enough, $g$ fulfills (7.6) and
$g\in\mathcal{H}_{s}(2M_{y},2M_{x},\chi^{(s)},\bar{C}^{(s)})$, then

$\sup_{t\in(0,1)}\sup_{\eta\in
E_{n}}|\hat{G}(t,\eta)|=O_{\mathbb{P}}(\beta_{n}).$

######  Proof 8.21 (Proof of Lemma 8.20).

We abbreviate $\chi=\chi^{(s)}$ and $\bar{C}=\bar{C}^{(s)}$.
(i) By Lemma 8.6(i),(ii) and by Assumption 7.1(A6), we obtain (independent of
(7.4) or (7.5)) that for some $C>0$:

$\displaystyle\|\sup_{\theta\in\Theta}|g(Z_{i},\theta)-g(Z_{i}^{c},\theta)|\|_{1}$
$\displaystyle\leq$ $\displaystyle
C\sum_{j=0}^{\infty}\hat{\chi}_{j}\|Z_{ij}-Z_{ij}^{c}\|_{M}\leq
2C\sum_{j=i}^{\infty}\chi_{j}\|Z_{ij}\|_{M}\leq
2CD\sum_{j=i}^{\infty}\chi_{j}.$

Similarly, we have in the case (7.4) that

$\displaystyle\|\sup_{\theta\in\Theta}|g(Z_{i},\theta)-g(\tilde{Z}_{i}(i/n),\theta)|\|_{1}$
$\displaystyle\leq$ $\displaystyle
C\Big{(}\|Y_{i}-\tilde{Y}_{i}(i/n)\|_{M}+\sum_{j=1}^{\infty}\chi_{j}\|X_{ij}-\tilde{X}_{ij}(i/n)\|_{M}\Big{)}$
$\displaystyle\leq$ $\displaystyle CC_{A}|\chi|_{1}n^{-1},$

while in the case (7.5) there exists $C_{2}>0$ such that

$\displaystyle\|\sup_{\theta\in\Theta}|g(Z_{i},\theta)-g(\tilde{Z}_{i}(i/n),\theta)|\|_{1}$
$\displaystyle\leq$ $\displaystyle
C_{2}\sum_{j=1}^{\infty}\chi_{j}\|X_{ij}-\tilde{X}_{ij}(i/n)\|_{M}\leq
C_{2}C_{A}|\chi|_{1}n^{-1}.$

Thus

$\displaystyle\|\sup_{t\in(0,1)}\sup_{\eta\in
E_{n}}|G_{n}(t,\eta)-G_{n}^{c}(t,\eta)|\|_{1}$ $\displaystyle\leq$
$\displaystyle|K|_{\infty}(nb_{n})^{-1}\sum_{i=1}^{n}\|\sup_{\theta\in\Theta}|g(Z_{i},\theta)-g(Z_{i}^{c},\theta)|\|_{1}$
$\displaystyle\leq$ $\displaystyle
2CD|K|_{\infty}(nb_{n})^{-1}\sum_{i=1}^{n}\sum_{j=i}^{\infty}\chi_{j}+|K|_{\infty}(C\vee
C_{2})|\chi|_{1}(nb_{n})^{-1}.$

Since $\chi_{j}=O(j^{-1+\gamma})$, it holds that
$\sum_{i=1}^{n}\sum_{j=i}^{\infty}\chi_{j}=O(1)$ and the assertion is proved.
The proofs under Assumption 7.3 are similar in view of Lemma 8.8.

(ii) (a) Fix $Q>0$. Let $\kappa>0$. Let $E_{n}^{(\kappa)}$ be a discretization
of $E_{n}$ such that for each $\eta\in E_{n}$ one can find $\eta^{\prime}\in
E_{n}^{(\kappa)}$ with $|\eta-\eta^{\prime}|_{1}\leq\kappa$. Note that
$\\#E_{n}^{(\kappa)}$ does not need to depend on $n$. Then

$\displaystyle\mathbb{P}\big{(}\sup_{\eta\in
E_{n}}|\hat{G}_{n}(t,\eta)|>Q\big{)}$ $\displaystyle\leq$
$\displaystyle\\#E_{n}^{(\kappa)}\sup_{\eta\in
E_{n}}\mathbb{P}\big{(}|\hat{G}_{n}(t,\eta)|>Q/2\big{)}$ (8.54)
$\displaystyle\quad\quad\quad+\mathbb{P}(\sup_{|\eta-\eta^{\prime}|_{1}\leq\kappa}|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t,\eta^{\prime})|>Q/2).$

By Markov’s inequality, we have for $0\leq s\leq\varsigma$,

$\mathbb{P}\big{(}|\hat{G}_{n}(t,\eta)|>Q/2\big{)}\leq\frac{\|\hat{G}_{n}(t,\eta)\|_{1+s}^{1+s}}{(Q/2)^{1+s}}.$

Using Burkholder’s moment inequality (cf. [7]) and Lemma 8.14(i) applied for
$q=1+s$, $s>0$ small enough, the computation

$\displaystyle\|\hat{G}_{n}(t,\eta)\|_{1+s}$ $\displaystyle\leq$
$\displaystyle(nb_{n})^{-1}\sum_{l=0}^{\infty}\Big{\|}\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)P_{i-l}g(\tilde{Z}_{i}(i/n),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})\Big{\|}_{1+s}$
$\displaystyle\leq$ $\displaystyle
s^{-1}(nb_{n})^{-1}\sum_{l=0}^{\infty}\Big{(}\Big{\|}\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)^{2}P_{i-l}^{2}g(\tilde{Z}_{i}(i/n),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})\Big{\|}_{(1+s)/2}^{(1+s)/2}\Big{)}^{1/(1+s)}$
$\displaystyle\leq$ $\displaystyle
s^{-1}(nb_{n})^{-s/(1+s)}|\hat{K}|_{\infty}\sum_{l=0}^{\infty}\sup_{t\in[0,1]}\delta_{1+s}^{\sup_{\theta\in\Theta}|g(\tilde{Z}(t),\theta)|}(l)=O((nb_{n})^{-s/(1+s)}),$

shows that the first summand in (8.54) tends to zero. For the second summand,
Lemma 8.18(i) implies

$\mathbb{P}(\sup_{|\eta-\eta^{\prime}|_{1}\leq\kappa}|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t,\eta^{\prime})|>Q/2)\leq\frac{2\tilde{C}\kappa}{Q},$

which can be made arbitrary small by choosing $\iota$ small enough. So we have
shown that (8.54) tends to zero for $n\to\infty$.
(b) The proof is similar to (a) by using 8.18(ii) and Lemma 8.16(i) instead of
Lemma 8.18(i) and Lemma 8.14(i).
(c) The proof is similar to (a) by using Lemma 8.14(i)(*) instead of Lemma
8.14(i).

(iii) (a) We use a chaining argument. Let $r=n^{3}$ and let $E_{n,r}$ be a
discretization of $E_{n}$ such that for each $\eta\in E_{n}$ one can find
$\eta^{\prime}\in E_{n,r}$ with $|\eta-\eta^{\prime}|\leq r^{-1}$. Define
$\mathcal{T}_{n,r}:=\\{i/r:i=1,\ldots,r\\}$ as a discretization of $(0,1)$.
Then $\\#(E_{n,r}\times\mathcal{T}_{n,r})=O(r^{2d_{\Theta}+1})$. For some
constant $Q>0$, we have

$\displaystyle\mathbb{P}\Big{(}\sup_{\eta\in
E_{n},t\in(0,1)}|\hat{G}_{n}(t,\eta)|>Q\beta_{n}\Big{)}$ (8.56)
$\displaystyle\leq$ $\displaystyle\mathbb{P}\Big{(}\sup_{\eta\in
E_{n,r},t\in\mathcal{T}_{n,r}}|\hat{G}_{n}(t,\eta)|>Q\beta_{n}/2\Big{)}$
$\displaystyle\quad\quad+\mathbb{P}\Big{(}\sup_{|\eta-\eta^{\prime}|\leq
r^{-1},|t-t^{\prime}|\leq
r^{-1}}|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t^{\prime},\eta^{\prime})|>Q\beta_{n}/2\Big{)}.$

Let $\alpha=1/2$. Let
$M_{i}(t,\eta,u):=\hat{K}_{b_{n}}(u-t)g(\tilde{Z}_{i}(u),\eta_{1}+\eta_{2}(u-t)b_{n}^{-1})$.
By Lemma 8.14(ii) applied with $q=2+s$, $s>0$ small enough, we have
$\sup_{u}\Delta^{\sup_{t,\eta}|M(t,\eta,u)|}_{2+s}(k)=O(k^{-(1+\gamma)})$.
Thus

$\displaystyle W_{2+s,\alpha}$ $\displaystyle:=$
$\displaystyle\sup_{u\in[0,1]}\|\sup_{t,\eta}|M_{i}(t,\eta,u)|\|_{2+s,\alpha}=\sup_{m\geq
0}(m+1)^{\alpha}\sup_{u\in[0,1]}\sup_{t,\eta}\Delta_{2+s}^{\sup_{t,\eta}|M(t,\eta,u)|}(m)<\infty.$

(independent of $n$) and

$\displaystyle W_{2,\alpha}$ $\displaystyle:=$
$\displaystyle\sup_{u\in[0,1]}\sup_{t,\eta}\|M_{i}(t,\eta,u)\|_{2,\alpha}=\sup_{m\geq
0}(m+1)^{\alpha}\sup_{u\in[0,1]}\sup_{t,\eta}\Delta_{2}^{M(t,\eta,u)}(m)<\infty$

(independent of $n$). Note that
$l=1\wedge\log\\#(E_{n,r}\times\mathcal{T}_{n,r})\leq 3(2d_{\Theta}+1)\log(n)$
and
$Q\beta_{n}(nb_{n})=Qn^{1/2}\log(n)^{1/2}\geq\sqrt{nl}W_{2,\alpha}+n^{1/(2+s)}l^{3/2}W_{2+s,\alpha}\gtrsim
n^{1/2}\log(n)^{1/2}+n^{1/(2+s)}\log(n)^{3/2}$ for $Q$ large enough. By
applying Theorem 6.2 of [57] (the proof therein also works for the uniform
functional dependence measure) with $q=2+s$ and $\alpha=1/2$ to
$(M_{i}(t,\eta,i/n))_{t\in\mathcal{T}_{n,r},\eta\in E_{n,r}}$, we have with
some constant $C_{\alpha}>0$:

$\displaystyle\mathbb{P}\big{(}\sup_{\eta^{\prime}\in
E_{n,r},t^{\prime}\in\mathcal{T}_{n,r}}|\hat{G}_{n}(t^{\prime},\eta^{\prime})|\geq
Q\beta_{n}/2\big{)}$ (8.57) $\displaystyle\leq$
$\displaystyle\frac{C_{\alpha}n\cdot
l^{1+s/2}W_{2+s,\alpha}^{2+s}}{(Q/2)^{2+s}(\delta_{n}(nb_{n}))^{2+s}}+C_{\alpha}\exp\Big{(}-\frac{C_{\alpha}(Q/2)^{2}(\beta_{n}(nb_{n}))^{2}}{nW_{2,\alpha}^{2}}\Big{)}$
$\displaystyle\lesssim$ $\displaystyle
n^{-s/2}+\exp\big{(}-\frac{(nb_{n})b_{n}^{-1}\log(n)}{n}\big{)}$
$\displaystyle\to$ $\displaystyle 0.$

By Markov’s inequality and Lemma 8.18(i),

$\mathbb{P}\big{(}\sup_{|\eta-\eta^{\prime}|_{1}\leq r^{-1},|t-t^{\prime}|\leq
r^{-1}}|\hat{G}_{n}(t,\eta)-\hat{G}_{n}(t^{\prime},\eta^{\prime})|\geq
C\beta_{n}/2\big{)}=O\Big{(}\frac{b_{n}^{-2}r^{-1}}{\beta_{n}}\Big{)}.$ (8.58)

We have
$b_{n}^{-2}r^{-1}\beta_{n}^{-1}=b_{n}^{-2}n^{-3}(nb_{n})^{1/2}b_{n}^{1/2}\log(n)^{-1/2}\to
0$. Inserting (8.57) and (8.58) into (8.56), we obtain the result.
(b) The proof is similar to (a) by using 8.18(ii) and Lemma 8.16(ii) instead
of Lemma 8.18(i) and Lemma 8.14(ii).
(c) The proof is similar to (a) by using 8.18(ii)(*) instead of Lemma
8.18(ii).

######  Lemma 8.22.

For $g:\mathbb{R}\times\mathbb{R}^{\mathbb{N}}\times\Theta\to\mathbb{R}$. Let

$\hat{B}_{n}(t,\eta)=(nb_{n})^{-1}\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)g(\tilde{Z}_{i}(i/n),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1}).$

  1. (a)

If Assumption 7.1(A5) is fulfilled with $r\geq 1+s$, $s\geq 0$ and
$g\in\mathcal{H}_{s}(M_{y},M_{x},\chi,\bar{C})$, then

$\sup_{t\in(0,1)}\sup_{\eta\in
E_{n}}|\mathbb{E}\hat{B}_{n}(t,\eta)-\int_{-t/b_{n}}^{(1-t)/b_{n}}\hat{K}(x)\mathbb{E}g(\tilde{Z}_{0}(t),\eta_{1}+\eta_{2}x)dx|=O((nb_{n})^{-1}+b_{n}).$

  2. (b)

If Assumption 7.3(A5’) is fulfilled with $r\geq 1+s$ and $g$ is such that
$\tilde{g}_{\tilde{\theta}}(y,x,\theta):=g(F(y,x,\tilde{\theta}),x,\theta)$
fulfills $\tilde{g}\in\mathcal{H}^{mult}_{s,\iota}(M,\chi,\bar{C})$, then

$\sup_{t\in(0,1)}\sup_{|\eta-\eta_{b_{n}}(t)|<\iota}|\mathbb{E}\hat{B}_{n}(t,\eta)-\int_{-t/b_{n}}^{(1-t)/b_{n}}\hat{K}(x)\mathbb{E}g(\tilde{Z}_{0}(t),\eta_{1}+\eta_{2}x)dx|=O((nb_{n})^{-1}+b_{n}).$

If the supremum is taken over $t\in\mathcal{T}_{n}$ instead of $t\in(0,1)$,
then $\int_{-t/b_{n}}^{(1-t)/b_{n}}$ can be replaced by $\int_{-1}^{1}$.

######  Proof 8.23 (Proof of Lemma 8.22).

(a) Let
$\tilde{B}_{n}(t,\eta):=(nb_{n})^{-1}\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)g(\tilde{Z}_{i}(t),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})$.
By Lemma 8.6(i), we have with some constant $\tilde{C}>0$ that either in the
case of (7.4),

$\displaystyle\|g(\tilde{Z}_{0}(i/n),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})-g(\tilde{Z}_{0}(t),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})\|_{1}$
$\displaystyle\leq$
$\displaystyle\tilde{C}\Big{(}\|\tilde{Y}_{0}(i/n)-\tilde{Y}_{0}(t)\|_{M}+\sum_{i=1}^{\infty}\chi_{i}\|\tilde{X}_{-i}(i/n)-\tilde{X}_{-i}(t)\|_{M}\Big{)}\leq\tilde{C}C_{B}(1+|\chi|_{1})b_{n}$

or in the case of (7.5),

$\displaystyle\|g(\tilde{Z}_{0}(i/n),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})-g(\tilde{Z}_{0}(t),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})\|_{1}$
$\displaystyle\leq$
$\displaystyle\tilde{C}\sum_{i=1}^{\infty}\chi_{i}\|\tilde{X}_{-i}(i/n)-\tilde{X}_{-i}(t)\|_{M}\leq\tilde{C}C_{B}|\chi|_{1}b_{n}.$

Thus

$\displaystyle\|\hat{B}_{n}(t,\eta)-\tilde{B}_{n}(t,\eta)\|_{1}$
$\displaystyle\leq$
$\displaystyle(nb_{n})^{-1}\sum_{i=1}^{n}|\hat{K}_{b_{n}}(i/n-t)|$
$\displaystyle\quad\quad\quad\quad\quad\quad\times\|g(\tilde{Z}_{i}(i/n),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})-g(\tilde{Z}_{i}(t),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})\|_{1}$
$\displaystyle\leq$
$\displaystyle\tilde{C}|\hat{K}|_{\infty}C_{B}(1+|\chi|_{1})b_{n}.$

Since $\hat{K}$ is of bounded variation and
$\theta\mapsto\mathbb{E}g(\tilde{Z}_{0}(t),\theta)$ is Lipschitz continuous
due to $g\in\mathcal{H}(M_{y},M_{x},\chi,\bar{C})$ and Lemma 8.6, a Riemannian
sum argument yields

$\displaystyle\tilde{B}_{n}(t,\eta)$ $\displaystyle=$
$\displaystyle(nb_{n})^{-1}\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)\mathbb{E}g(\tilde{Z}_{0}(t),\eta_{1}+\eta_{2}(i/n-t)b_{n}^{-1})$
$\displaystyle=$
$\displaystyle\int_{-t/b_{n}}^{(1-t)/b_{n}}\hat{K}(x)\mathbb{E}g(\tilde{Z}_{0}(t),\eta_{1}+\eta_{2}x)dx+O((nb_{n})^{-1}),$

uniformly in $t\in(0,1)$, $\eta\in E_{n}$.
(b) The proof is the same by using Lemma 8.10 with $q=1$ instead of Lemma 8.6.

######  Lemma 8.24.

Let
$\eta_{b_{n}}(t)=(\theta(t)^{\mkern-1.5mu\mathsf{T}},b_{n}\theta^{\prime}(t)^{\mkern-1.5mu\mathsf{T}})^{\mkern-1.5mu\mathsf{T}}$.
Let Assumption 7.1 hold with $r=1$ or let Assumption 7.3 hold with
$r=2+\varsigma$, $\varsigma>0$.

  1. (i)

Then uniformly in $t\in\mathcal{T}_{n}$,

$\mathbb{E}\nabla_{\eta_{1}}\hat{L}_{n,b_{n}}^{\circ}(t,\eta_{b_{n}}(t))=b_{n}^{2}\frac{\mu_{K,2}}{2}V(t)\theta^{\prime\prime}(t)+O(b_{n}^{3}+(nb_{n})^{-1}).$
(8.59)

Furthermore, it holds uniformly in $t\in(0,1)$ that

$\mathbb{E}\nabla_{\eta}\hat{L}_{n,b_{n}}^{\circ}(t,\eta_{b_{n}}(t))=\frac{b_{n}^{2}}{2}\int_{-t/b_{n}}^{(1-t)/b_{n}}K(x)\begin{pmatrix}x^{2}\\\
x^{3}\end{pmatrix}dx\otimes[V(t)\theta^{\prime\prime}(t)]+O(b_{n}^{3}+(nb_{n})^{-1}).$
(8.60)

  2. (ii)

If Assumptions 7.1, 7.2 or Assumptions 7.3, 7.4 hold with the $r$ specified
above, then uniformly in $t\in\mathcal{T}_{n}$,

$\mathbb{E}\nabla_{\eta_{2}}\hat{L}_{n,b_{n}}^{\circ}(t,\eta_{b_{n}}(t))=b_{n}^{3}\frac{\mu_{K,4}}{2}V(t)\text{bias}(t)+O(b_{n}^{4}+(nb_{n})^{-1}),$

where
$\text{bias}(t)=\frac{1}{3}\theta^{(3)}(t)+V(t)^{-1}\mathbb{E}[\partial_{t}\nabla_{\theta}^{2}\ell(\tilde{Z}_{0}(t),\theta(t))]\cdot\theta^{\prime\prime}(t)$,
and the term $O(b_{n}^{3})$ in (8.59) can be replaced by $O(b_{n}^{4})$.

######  Proof 8.25 (Proof of Lemma 8.24).

(i) Let
$U_{i,n}(t)=(K_{b_{n}}(i/n-t),K_{b_{n}}(i/n-t)(i/n-t)b_{n}^{-1})^{\mkern-1.5mu\mathsf{T}}$.
By a Taylor expansion of $\theta(i/n)$ around $t$, we have

$\theta(i/n)=\theta(t)+\theta^{\prime}(t)(i/n-t)+r_{n}(t),$

where
$r_{n}(t)=\theta^{\prime\prime}(t)\frac{(i/n-t)^{2}}{2}+\theta^{\prime\prime\prime}(\tilde{t})\frac{(i/n-t)^{3}}{6}$
and $\tilde{t}$ is between $t$ and $i/n$. We conclude that

$\displaystyle\nabla_{\eta}\hat{L}_{n,b_{n}}^{\circ}(t,\eta_{b_{n}}(t))-(nb_{n})^{-1}\sum_{i=1}^{n}U_{i,n}(t)\otimes\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))$
(8.61) $\displaystyle=$
$\displaystyle(nb_{n})^{-1}\sum_{i=1}^{n}U_{i,n}(t)\otimes\big{\\{}\int_{0}^{1}\nabla_{\theta}^{2}\ell(\tilde{Z}_{i}(i/n),\theta(i/n)+sr_{n}(t))ds\cdot
r_{n}(t)\big{\\}}.$

Using $\nabla_{\theta}^{2}\ell\in\mathcal{H}(M_{y},M_{x},\chi,\bar{C})$ (if
Assumption 7.1 holds) or
$\nabla_{\theta}^{2}\ell\in\mathcal{H}_{s}(2M_{y},2M_{x},\chi,\bar{C})$ with
$s>0$ small enough (if Assumption 7.3 holds), we obtain with Lemma 8.6 for
$|i/n-t|\leq b_{n}$:

$\|\nabla_{\theta}^{2}\ell(\tilde{Z}_{i}(i/n),\theta(i/n)+sr_{n}(t))-\nabla_{\theta}^{2}\ell(\tilde{Z}_{i}(t),\theta(t))\|_{1}=O(b_{n}+n^{-1}).$
(8.62)

Using (8.61),
$\mathbb{E}\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))=0$ (by
Assumption 7.1(A1),(A3) or Assumption 7.3(A1’), (A3’)) and (8.62), we obtain

$\displaystyle\mathbb{E}\nabla_{\eta}\hat{L}_{n,b_{n}}^{\circ}(t,\eta_{b_{n}}(t))$
(8.63) $\displaystyle=$
$\displaystyle(nb_{n})^{-1}\sum_{i=1}^{n}U_{i,n}(t)\otimes\big{\\{}\mathbb{E}\nabla_{\theta}^{2}\ell(\tilde{Z}_{i}(t),\theta(t))\cdot\theta^{\prime\prime}(t)\frac{(i/n-t)^{2}}{2}\big{\\}}+O(b_{n}^{3}+n^{-1})$
$\displaystyle=$
$\displaystyle\begin{pmatrix}b_{n}^{2}\frac{\mu_{K,2}}{2}V(t)\theta^{\prime\prime}(t)\\\
0\end{pmatrix}+O(b_{n}^{3}+n^{-1}+(nb_{n})^{-1}),$

which shows (8.59).
(8.60) follows by a more careful examination of the above Riemannian sum:
Under Assumption 7.2, we have
$r_{n}(t)=\theta^{\prime\prime}(t)\frac{(i/n-t)^{2}}{2}+\theta^{(3)}(t)\frac{(i/n-t)^{3}}{6}+\theta^{(4)}(\tilde{t})\frac{(i/n-t)^{4}}{24}$,
where $\tilde{t}$ is between $t$ and $i/n$. We now use a more precise Taylor
argument as in (8.61). We have

$\displaystyle\nabla_{\eta}\hat{L}_{n,b_{n}}^{\circ}(t,\eta_{b_{n}}(t))-(nb_{n})^{-1}\sum_{i=1}^{n}U_{i,n}(t)\otimes\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))$
$\displaystyle=$
$\displaystyle(nb_{n})^{-1}\sum_{i=1}^{n}U_{i,n}(t)\otimes\nabla_{\theta}^{2}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))r_{n}(t)$
$\displaystyle\quad\quad+(nb_{n})^{-1}\sum_{i=1}^{n}U_{i,n}(t)\otimes\big{\\{}\int_{0}^{1}\nabla_{\theta}^{2}\ell(\tilde{Z}_{i}(i/n),\theta(i/n)+sr_{n}(t))$
$\displaystyle\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad-\nabla^{2}_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))ds\cdot
r_{n}(t)\big{\\}}.$

Since $\nabla_{\theta}^{2}\ell\in\mathcal{H}(M_{y},M_{x},\chi,\bar{C})$ (if
Assumption 7.1 holds) or
$\nabla_{\theta}^{2}\ell\in\mathcal{H}_{s}(2M_{y},2M_{x},\chi,\bar{C})$ for
$s>0$ small enough (if Assumption 7.3 holds), we have by Lemma 8.6:

$\big{\|}\nabla_{\theta}^{2}\ell(\tilde{Z}_{i}(i/n),\theta(i/n)+sr_{n}(t))-\nabla_{\theta}^{2}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))\big{\|}_{1}=O(r_{n}(t))=O(|i/n-t|^{2}).$
(8.65)

This shows that the expectation of the second summand in (8.25) is
$O(b_{n}^{4})$. We now discuss the first term in (8.25). Put
$v_{i}(t):=\nabla_{\theta}^{2}\ell(\tilde{Z}_{i}(t),\theta(t))$. By Assumption
7.2, $t\mapsto v_{i}(t)$ is continuously differentiable. By Taylor’s
expansion,
$v_{i}(i/n)=v_{i}(t)+(i/n-t)\partial_{t}v_{i}(t)+(i/n-t)\int_{0}^{1}\partial_{t}v_{i}(t+s(i/n-t))-\partial_{t}v_{i}(t)ds$.
We have

$\mathbb{E}\big{[}(nb_{n})^{-1}\sum_{i=1}^{n}U_{i,n}(t)\otimes
v_{i}(t)r_{n}(t)\big{]}=\begin{pmatrix}b_{n}^{2}\frac{\mu_{K,2}}{2}V(t)\theta^{\prime\prime}(t)\\\
b_{n}^{3}\frac{\mu_{K,4}}{6}V(t)\theta^{(3)}(t)\end{pmatrix}+O(n^{-1}+b_{n}^{4}),$
(8.66)

since $K$ has bounded variation and $\int K(x)x^{3}dx=0$ by symmetry.
Similarly,

$\displaystyle\mathbb{E}\big{[}(nb_{n})^{-1}\sum_{i=1}^{n}U_{i,n}(t)\otimes\partial_{t}v_{i}(t)r_{n}(t)\big{]}$
(8.67) $\displaystyle=$ $\displaystyle\begin{pmatrix}0\\\
b_{n}^{3}\frac{\mu_{K,4}}{2}\mathbb{E}[\partial_{t}\nabla_{\theta}^{2}\ell(\tilde{Z}_{0}(t),\theta(t))]\theta^{\prime\prime}(t)\end{pmatrix}+O(n^{-1}+b_{n}^{4}).$

Finally, Lemma 8.12 applied to $g=\nabla_{\theta}^{2}\ell$ (use Assumption 7.2
or 7.4) yields:

$\|\partial_{t}v_{i}(t+s(i/n-t))-\partial_{t}v_{i}(t)\|_{1}=O(|i/n-t|).$
(8.68)

The results (8.66), (8.67) and (8.68) imply

$\displaystyle\mathbb{E}\big{[}(nb_{n})^{-1}\sum_{i=1}^{n}U_{i,n}(t)\otimes\nabla_{\theta}^{2}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))r_{n}(t)]$
$\displaystyle=$
$\displaystyle\begin{pmatrix}b_{n}^{2}\frac{\mu_{K,2}}{2}V(t)\theta^{\prime\prime}(t)\\\
b_{n}^{3}\mu_{K,4}\cdot\big{\\{}\frac{1}{6}V(t)\theta^{(3)}(t)+\frac{1}{2}\mathbb{E}[\partial_{t}\nabla_{\theta}^{2}\ell(\tilde{Z}_{0}(t),\theta(t))]\cdot\theta^{\prime\prime}(t)\big{\\}}\end{pmatrix}+O(n^{-1}+b_{n}^{4}),$

which together with (8.25) gives the result.

######  Lemma 8.26 (Lipschitz properties of $\Pi_{n}$).

Let $s\geq 0$. Suppose that Assumption 7.1 holds with $r\geq 1$ or Assumption
7.3 holds with $r>1$. Define

$\Pi_{n}(t):=(nb_{n})^{-1}\sum_{i=1}^{n}(M_{i}^{(2)}(t,i/n)-\mathbb{E}M_{i}^{(2)}(t,i/n)),$

where

$M_{i}^{(2)}(t,u)=\hat{K}_{b_{n}}(u-t)\cdot\int_{0}^{1}M_{i}(t,u)ds\cdot
d_{u}(t),$

$M_{i}(u,t)=\nabla_{\theta}^{2}\ell(\tilde{Z}_{i}(u),\theta(t)+sd_{u}(t))$ and
$d_{u}(t)=\theta(u)-\theta(t)-(u-t)\theta^{\prime}(t)$. Then there exist come
constants $\tilde{C},\iota^{\prime}>0$ such that

$\Big{\|}\sup_{t\not=t^{\prime},|t-t^{\prime}|<\iota^{\prime}}\frac{|\Pi_{n}(t)-\Pi_{n}(t^{\prime})|}{|t-t^{\prime}|_{1}}\Big{\|}_{1}\leq\tilde{C}.$

######  Proof 8.27 (Proof of Lemma 8.26).

We have

$\displaystyle|M_{i}^{(2)}(t,u)-M_{i}^{(2)}(t^{\prime},u)|$
$\displaystyle\leq$
$\displaystyle|\hat{K}_{b_{n}}(u-t)-\hat{K}_{b_{n}}(u-t^{\prime})|\cdot|M_{i}(t,u)|\cdot|d_{u}(t)|$
$\displaystyle\quad\quad+|\hat{K}_{b_{n}}(u-t^{\prime})|\cdot|M_{i}(t,u)-M_{i}(t^{\prime},u)|\cdot|d_{u}(t)|$
$\displaystyle\quad\quad+|\hat{K}_{b_{n}}(u-t^{\prime})|\cdot|M_{i}(t^{\prime},u)|\cdot|d_{u}(t)-d_{u}(t^{\prime})|.$

If Assumption 7.1 holds, we have

$\displaystyle|M_{i}(t,u)|$ $\displaystyle\leq$
$\displaystyle\sup_{\theta\in\Theta}|g(\tilde{Z}_{i}(u),\theta)|,$
$\displaystyle|M_{i}(t,u)-M_{i}(t^{\prime},u)|$ $\displaystyle\leq$
$\displaystyle\sup_{\theta\in\Theta}\frac{|g(\tilde{Z}_{i}(u),\theta)-g(\tilde{Z}_{i}(u),\theta^{\prime})|}{|\theta-\theta^{\prime}|_{1}}\cdot\\{|\theta(t)-\theta(t^{\prime})|_{1}+|d_{u}(t)-d_{u}(t^{\prime})|_{1}\\},$

As long as $|t-u|<1$ and $|t-t^{\prime}|$ is small enough, we obtain
$|t^{\prime}-u|\leq 1$. So in the case that either $|t-u|<1$ or
$|t^{\prime}-u|<1$, Lipschitz continuity of
$\theta(\cdot),\theta^{\prime}(\cdot)$ implies that there exists some constant
$\tilde{C}>0$ such that
$|d_{u}(t)-d_{u}(t^{\prime})|_{1}\leq\tilde{C}|t-t^{\prime}|,|\theta(t)-\theta(t^{\prime})|_{1}\leq\tilde{C}|t-t^{\prime}|$,
$|d_{u}(t)|_{1}\leq\tilde{C}$.

This implies

$\displaystyle|M_{i}^{(2)}(t,u)-M_{i}^{(2)}(t^{\prime},u)|$
$\displaystyle\leq$
$\displaystyle\tilde{C}b_{n}^{-1}L_{\hat{K}}\sup_{\theta\in\Theta}|g(\tilde{Z}_{i}(u),\theta)|\cdot|t-t^{\prime}|$
(8.69)
$\displaystyle\quad+2|\hat{K}|_{\infty}\tilde{C}^{2}\cdot\sup_{\theta\in\Theta}\frac{|g(\tilde{Z}_{i}(u),\theta)-g(\tilde{Z}_{i}(u),\theta^{\prime})|}{|\theta-\theta^{\prime}|_{1}}|t-t^{\prime}|$
$\displaystyle\quad+|\hat{K}|_{\infty}\tilde{C}\cdot\sup_{\theta\in\Theta}|g(\tilde{Z}_{i}(u),\theta)|\cdot|t-t^{\prime}|.$

With Lemma 8.6(i) we obtain the result.

Suppose now that Assumption 7.3 holds. As long as $|t-t^{\prime}|$ is small
enough and $n$ is large enough, $|u-t|\leq b_{n}$ (or $|u-t^{\prime}|\leq
b_{n}$) and the twice differentiability of $\theta(\cdot)$ imply that
$\sup_{\nu\in[0,1]}|\theta(u)-(\theta(t)+\nu d_{u}(t))|_{1}<\iota$,
$\sup_{\nu\in[0,1]}|\theta(u)-(\theta(t^{\prime})+\nu
d_{u}(t^{\prime}))|_{1}<\iota$. We then obtain

$\displaystyle|M_{i}(t,u)|$ $\displaystyle\leq$
$\displaystyle\sup_{|\theta-\theta(u)|_{1}<\iota}|\tilde{g}_{\theta(u)}(\zeta_{i},\tilde{X}_{i}(u),\theta)|,$
$\displaystyle|M_{i}(t,u)-M_{i}(t^{\prime},u)|$ $\displaystyle\leq$
$\displaystyle\bar{C}\cdot\sup_{\theta\not=\theta^{\prime},|\theta-\theta(u)|_{1}<\iota,|\theta^{\prime}-\theta(u)|_{1}<\iota}\frac{|\tilde{g}_{\theta(u)}(\zeta_{i},\tilde{X}_{i}(u),\theta)-\tilde{g}_{\theta(u)}(\zeta_{i},\tilde{X}_{i}(u),\theta^{\prime})|}{|\theta-\theta^{\prime}|_{1}}$
$\displaystyle\quad\quad\times\\{|\theta(t)-\theta(t^{\prime})|_{1}+|d_{u}(t)-d_{u}(t^{\prime})|_{1},$

giving appropriate results for (8.69) and thus the assertion with Lemma 8.10.

######  Lemma 8.28.

Let
$U_{i,n}(t):=K_{b_{n}}(i/n-t)\cdot(1,(i/n-t)b_{n}^{-1})^{\mkern-1.5mu\mathsf{T}}$.
Let Assumption 7.1 or 7.3 hold with some $r=2+\varsigma$, $\varsigma>0$. Then
it holds that

$\displaystyle\sup_{t\in(0,1)}\big{|}\nabla_{\eta}\hat{L}_{n,b_{n}}^{\circ}(t,\eta_{b_{n}}(t))-\mathbb{E}\nabla_{\eta}\hat{L}_{n,b_{n}}^{\circ}(t,\eta_{b_{n}}(t))$
$\displaystyle\quad\quad\quad\quad\quad-(nb_{n})^{-1}\sum_{i=1}^{n}U_{i,n}(t)\otimes\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))\big{|}=O_{\mathbb{P}}(\beta_{n}b_{n}^{2}).$

######  Proof 8.29.

Note that $\mathbb{E}\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))=0$ by
Assumption 7.1(A1),(A3) or Assumption 7.3(A1’),(A3’). Put

$\displaystyle\Pi_{n}(t)$ $\displaystyle:=$
$\displaystyle(nb_{n})^{-1}\sum_{i=1}^{n}U_{i,n}(t)\otimes\big{\\{}[\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(t)+(i/n-t)\theta^{\prime}(t))-\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))]$
$\displaystyle\quad\quad\quad\quad\quad\quad\quad\quad-\mathbb{E}[\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(t)+(i/n-t)\theta^{\prime}(t))-\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))]\\}.$

We have to prove that
$\sup_{t\in\mathcal{T}_{n}}\big{|}\Pi_{n}(t)\big{|}=O_{\mathbb{P}}(\delta_{n}b_{n}^{2}$).
Define
$M_{i}(t,u):=\int_{0}^{1}\nabla_{\theta}^{2}\ell(\tilde{Z}_{i}(u),\theta(t)+s(\theta(u)-\theta(t)-(u-t)\theta^{\prime}(t)))ds$
and
$M_{i}^{(2)}(t,u)=U_{i,n}(t)\otimes\big{\\{}M_{i}(t,u)\\{\theta(u)-\theta(t)-(u-t)\theta^{\prime}(t)\\}\big{\\}}$.
By a Taylor expansion of $\nabla_{\theta}\ell$ w.r.t. $\theta$, we have

$\Pi_{n}(t)=(nb_{n})^{-1}\sum_{i=1}^{n}(M_{i}^{(2)}(t,i/n)-\mathbb{E}M_{i}^{(2)}(t,i/n)).$

We now apply a similar technique as in the proof of Lemma 8.20(iii), namely we
use a chaining argument similar to (8.56) to prove

$\mathbb{P}\big{(}\sup_{t\in(0,1)}|\Pi_{n}(t)|>Q\beta_{n}b_{n}^{2}\big{)}\to
0,$

for some $Q>0$ large enough. Define the discretization
$\mathcal{T}_{n,r}:=\\{l/r:l=1,\ldots,r\\}$ with $r=n^{5}$. By Lemma 8.26, we
have with Markov’s inequality for $Q>0$:

$\mathbb{P}\big{(}\sup_{|t-t^{\prime}|\leq
r^{-1}}|\Pi_{n}(t)-\Pi_{n}(t^{\prime})|>Q\beta_{n}b_{n}^{2}/2\big{)}=O\big{(}\frac{b_{n}^{-2}r^{-1}}{\beta_{n}b_{n}^{2}}\big{)},$

which converges to 0. Choose $\alpha=1/2$. By Lemma 8.14(iii) or Lemma
8.16(iii) applied with $q=2+s$ ($s$ small enough), we obtain that
$\sup_{u}\Delta_{2+s}^{\sup_{t}|M^{(2)}(t,u)|}(k)=O(k^{-(1+\gamma)})$. Thus

$\displaystyle\tilde{W}_{2+s,\alpha}$ $\displaystyle:=$
$\displaystyle\sup_{u\in[0,1]}\sup_{t\in[0,1]}\|\sup_{t,\eta}|M_{i}^{(2)}(t,u)|\|_{2+\varsigma,\alpha}=\sup_{m\geq
0}(m+1)^{\alpha}\Delta_{2+s}^{\sup_{t}|M^{(2)}(t,u)|}(m)$ (8.70)
$\displaystyle=$ $\displaystyle O(b_{n}^{2})$

(the constant being independent of $n$) and

$\displaystyle\tilde{W}_{2,\alpha}$ $\displaystyle:=$
$\displaystyle\sup_{t,u}\|M_{i}^{(2)}(t,u)\|_{2,\alpha}=\sup_{m\geq
0}(m+1)^{\alpha}\sup_{u\in[0,1]}\sup_{t}\Delta_{2}^{M^{(2)}(t,u)}(m)$ (8.71)
$\displaystyle=$ $\displaystyle O(b_{n}^{2})$

(the constant being independent of $n$). We now apply Theorem 6.2 of [57] (the
proof therein also works for the uniform functional dependence measure) with
$q=2+s$, $\alpha=1/2$ to $(M_{i}^{(2)}(t,i/n))_{t\in\mathcal{T}_{n,r}}$, where
$l=1\vee\\#(\mathcal{T}_{n,r})\leq 5\log(n)$. For $Q$ large enough, we obtain
with some constant $C_{\alpha,s}>0$:

$\displaystyle\mathbb{P}\big{(}\sup_{t^{\prime}\in\mathcal{T}_{n,r}}|\Pi_{n}(t^{\prime})|\geq
Q\beta_{n}b_{n}^{2}/2\big{)}$ $\displaystyle\leq$
$\displaystyle\frac{C_{\alpha,s}n\cdot
l^{1+s/2}\tilde{W}_{2+s,\alpha}^{2+s}}{(Q/2)^{2+s}(\beta_{n}b_{n}^{2}(nb_{n}))^{2+s}}+C_{\alpha,s}\exp\Big{(}-\frac{C_{\alpha,s}(Q/2)^{2}(\beta_{n}b_{n}^{2}(nb_{n}))^{2}}{n\tilde{W}_{2,\alpha}^{2}}\Big{)}$
$\displaystyle\lesssim$ $\displaystyle
n^{-\varsigma/2}+\exp\big{(}-\frac{(nb_{n})b_{n}^{-1}\log(n)}{n}\big{)}$
$\displaystyle\to$ $\displaystyle 0,$

which finishes the proof.

###  8.2 Proofs and Lemmas for the SCB

From Lemma 1 in [59], we adopt the following result:

######  Lemma 8.30.

Let $F_{n}(t)=\sum_{i=1}^{n}\hat{K}_{b_{n}}(t_{i}-t)V_{i}$, where
$V_{i},i\in\mathbb{Z}$ are i.i.d. $N(0,I_{s\times s})$. $b_{n}\to 0$ and
$nb_{n}/\log^{2}(n)\to\infty$. Let $m^{*}=1/b_{n}.$ Then

$\lim_{n\to\infty}\mathbb{P}\Big{(}\frac{1}{\sigma_{\hat{K},0}\sqrt{nb_{n}}}\sup_{t\in\mathcal{T}_{n}}|F_{n}(t)|-B_{\hat{K}}(m^{*})\leq\frac{u}{\sqrt{2\log(m^{*})}}\Big{)}=\exp(-2\exp(-u)).$
(8.72)

where $B_{\hat{K}}$ is defined in (3.7).

The following lemma is an analogue of Lemma 2 in [59]. Since we use other
Gaussian approximation rates from Theorem 8.33, we shortly state the proof for
completeness.

######  Lemma 8.31.

Let the assumptions and notations from Theorem 8.33 hold. Define

$D_{\tilde{h}}(t):=(nb_{n})^{-1}\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)\tilde{h}_{i}(i/n).$

Assume that $\Sigma_{\tilde{h}}(t)$ is Lipschitz-continuous and that its
smallest eigenvalue is bounded away from 0 uniformly on $[0,1]$. Assume that
$\log(n)^{4}\big{(}b_{n}n^{(2\gamma+\varsigma\gamma-\varsigma)/(\varsigma+4\gamma+2\gamma\varsigma)}\big{)}^{-1}\to
0$ and $b_{n}\log(n)^{3/2}\to 0$. Then

$\lim_{n\to\infty}\mathbb{P}\Big{(}\frac{\sqrt{nb_{n}}}{\sigma_{\hat{K},0}}\sup_{t\in\mathcal{T}_{n}}\Big{|}\Sigma_{\tilde{h}}^{-1}(t)D_{\tilde{h}}(t)\Big{|}-B_{\hat{K}}(m^{*})\leq\frac{u}{\sqrt{2\log(m^{*})}}\Big{)}=\exp(-2\exp(-u)),$
(8.73)

######  Proof 8.32 (Proof of Lemma 8.31).

By Theorem 8.33 and summation-by-parts, there exist i.i.d. $V_{i}\sim
N(0,I_{s\times s})$ such that

$\sup_{t\in(0,1)}|D_{\tilde{h}}(t)-\Xi(t)|=O_{\mathbb{P}}\Big{(}\frac{n^{\frac{2\varsigma+2\gamma+\gamma\varsigma}{2\varsigma+8\gamma+4\gamma\varsigma}}\log(n)^{\frac{2\gamma(3+\varsigma)}{\varsigma+4\gamma+2\gamma\varsigma}}}{nb_{n}}\Big{)}=O_{\mathbb{P}}\Big{(}\frac{\log(n)^{2}\big{(}b_{n}n^{\frac{2\gamma+\varsigma\gamma-\varsigma}{\varsigma+4\gamma+2\gamma\varsigma}}\big{)}^{-1/2}}{(nb_{n})^{1/2}\log(n)^{1/2}}\Big{)},$
(8.74)

where
$\Xi(t)=(nb_{n})^{-1}\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)\Sigma_{\tilde{h}}(i/n)V_{i}$.
Here, (8.74) is $o_{\mathbb{P}}((nb_{n})^{-1/2}\log(n)^{-1/2})$ due to

$\log(n)^{4}\big{(}b_{n}n^{(2\gamma+\varsigma\gamma-\varsigma)/(\varsigma+4\gamma+2\gamma\varsigma)}\big{)}^{-1}\to
0.$

Since $\Sigma_{\tilde{h}}(\cdot)$ is Lipschitz continuous by Assumption (b),
we can use a standard chaining argument in $t$ (as it was done in Lemma 8.28
for $\Pi_{n}(t)$) and the fact that
$(nb_{n})^{-1}\sum_{i=1}^{n}(\Sigma_{\tilde{h}}(i/n)-\Sigma_{\tilde{h}}(t))\hat{K}_{b_{n}}(i/n-t)V_{i}\sim
N(0,v_{n})$, with $|v_{n}|_{\infty}\leq C\frac{b_{n}}{n}$ for some constant
$C>0$ to obtain

$\displaystyle\sup_{t\in(0,1)}|\Xi(t)-(nb_{n})^{-1}\Sigma_{\tilde{h}}(t)\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)V_{i}|$
(8.75) $\displaystyle=$
$\displaystyle\sup_{t\in(0,1)}\big{|}(nb_{n})^{-1}\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)(\Sigma_{\tilde{h}}(i/n)-\Sigma_{\tilde{h}}(t))V_{i}\big{|}$
$\displaystyle=$ $\displaystyle
O_{\mathbb{P}}\Big{(}\frac{b_{n}\log(n)}{(nb_{n})^{1/2}}\Big{)}=O_{\mathbb{P}}\Big{(}\frac{b_{n}\log(n)^{3/2}}{(nb_{n})^{1/2}\log(n)^{1/2}}\Big{)},$

which is $o_{\mathbb{P}}((nb_{n})^{-1/2}\log(n)^{-1/2})$ due to
$b_{n}\log(n)^{3/2}\to 0$. So the result follows from Lemma 8.30 in view of
(8.74) and (8.75).

For the following results, let us assume that there exists some measurable
function $\tilde{H}(\cdot,\cdot)$ such that for each $t\in[0,1]$,
$\tilde{h}_{i}(t)=\tilde{H}(t,\mathcal{F}_{i})\in\mathbb{R}^{s}$ is well-
defined. Put $S_{\tilde{h}}(i):=\sum_{j=1}^{i}\tilde{h}_{j}(j/n)$.

######  Theorem 8.33 (Theorem 1 and Corollary 2 from Wu and Zhou [53]).

Assume that for each component $j=1,\ldots,s$:

  1. (a)

$\sup_{t\in[0,1]}\|\tilde{h}_{0}(t)_{j}\|_{2+\varsigma}<\infty$,

  2. (b)

$\sup_{t\not=t^{\prime}\in[0,1]}\|\tilde{h}_{0}(t)_{j}-\tilde{h}_{0}(t^{\prime})_{j}\|_{2}/|t-t^{\prime}|<\infty$,

  3. (c)

$\sup_{t\in[0,1]}\delta_{2+\varsigma}^{\tilde{h}(t)_{j}}(k)=O(k^{-(\gamma+1)})$
with some $\gamma\geq 1$.

for some $\varsigma\leq 2$. Then on a richer probability space, there are
i.i.d. $V_{1},V_{2},\ldots\sim N(0,I_{s\times s})$ and a process
$S_{\tilde{h}}^{0}(i)=\sum_{j=1}^{i}\Sigma_{\tilde{h}}(j/n)V_{j}$ such that
$(S_{\tilde{h}}(i))_{i=1,\ldots,n}\overset{d}{=}(S_{\tilde{h}}^{0}(i))_{i=1,\ldots,n}$
and

$\max_{i=1,\ldots,n}|S_{\tilde{h}}(i)-S_{\tilde{h}}^{0}(i)|=O_{\mathbb{P}}(\pi_{n}).$

where

$\pi_{n}=n^{(2\varsigma+2\gamma+\gamma\varsigma)/(2\varsigma+8\gamma+4\gamma\varsigma)}\log(n)^{2\gamma(3+\varsigma)/(\varsigma+4\gamma+2\gamma\varsigma)}$
(8.76)

and

$\Sigma_{\tilde{h}}(t)=\big{(}\sum_{j\in\mathbb{Z}}\mathbb{E}[\tilde{h}_{0}(t)\tilde{h}_{j}(t)^{\mkern-1.5mu\mathsf{T}}]\big{)}^{1/2}.$

######  Proof 8.34 (Proof of Theorem 3.3).

By Proposition 9.1. Assumption 2.2 implies Assumption 7.1, 7.2. By Proposition
9.3 or Assumption 2.3 implies Assumption 7.3, 7.4. The results now follow from
Theorem 8.35.

######  Theorem 8.35 (Simultaneous confidence bands for $\theta(\cdot)$ and
$\theta^{\prime}(\cdot)$).

Let $C$ be a fixed $k\times s$ matrix with rank $s\leq k$. Define
$\hat{\theta}_{b_{n},C}(t):=C^{\mkern-1.5mu\mathsf{T}}\hat{\theta}_{b_{n}}(t)$,
$\hat{\theta}_{b_{n},C}^{\prime}(t):=C^{\mkern-1.5mu\mathsf{T}}\widehat{\theta}_{b_{n}}^{\prime}(t)$
and $\theta_{C}(t):=C^{\mkern-1.5mu\mathsf{T}}\theta(t)$,
$A_{C}(t):=V(t)^{-1}C$,
$\Sigma_{C}^{2}(t):=A_{C}^{\mkern-1.5mu\mathsf{T}}(t)\Lambda(t)A_{C}(t)$.

Let Assumption 7.1 be fulfilled with $r=2+\varsigma$ for some $\varsigma>0$.
Assume that, for
$\alpha_{exp}=(2\gamma+\varsigma\gamma-\varsigma)/(\varsigma+4\gamma+2\gamma\varsigma)$,
$\log(n)\big{(}b_{n}n^{\alpha_{exp}}\big{)}^{-1}\to 0$.

  1. (i)

If $nb_{n}^{7}\log(n)\to 0$, then

$\displaystyle\lim_{n\to\infty}\mathbb{P}\Big{(}\frac{\sqrt{nb_{n}}}{\sigma_{K,0}}\sup_{t\in\mathcal{T}_{n}}\Big{|}\Sigma_{C}^{-1}(t)\Big{\\{}\hat{\theta}_{b_{n},C}(t)-\theta_{C}(t)-b_{n}^{2}\frac{\mu_{K,2}}{2}\theta_{C}^{\prime\prime}(t)\Big{\\}}\Big{|}$
$\displaystyle\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad-
B_{K}(m^{*})\leq\frac{u}{\sqrt{2\log(m^{*})}}\Big{)}=\exp(-2\exp(-u)),$ (8.77)

  2. (ii)

If additionally, Assumption 7.2 is fulfilled and $nb_{n}^{9}\log(n)\to 0$,
then with $\hat{K}(x)=K(x)x$,

$\displaystyle\lim_{n\to\infty}\mathbb{P}\Big{(}\frac{\sqrt{nb_{n}^{3}}\mu_{K,2}}{\sigma_{K,2}}\sup_{t\in\mathcal{T}_{n}}\Big{|}\Sigma_{C}^{-1}(t)\Big{\\{}\hat{\theta}_{b_{n},C}^{\prime}(t)-\theta_{C}^{\prime}(t)-b_{n}^{2}\frac{\mu_{\hat{K},4}}{2\mu_{\hat{K},2}}C^{\mkern-1.5mu\mathsf{T}}\text{bias}(t)\Big{\\}}\Big{|}$
$\displaystyle\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad-
B_{\hat{K}}(m^{*})\leq\frac{u}{\sqrt{2\log(m^{*})}}\Big{)}=\exp(-2\exp(-u)),$
(8.78)

where in both cases $\mathcal{T}_{n}=[b_{n},1-b_{n}]$, $m^{*}=1/b_{n}$ and

$B_{K}(m^{*})=\sqrt{2\log(m^{*})}+\frac{\log(C_{K})+(s/2-1/2)\log(\log(m^{*}))-\log(2)}{\sqrt{2\log(m^{*})}},$
(8.79)

with

$C_{K}=\frac{\Big{\\{}\int_{-1}^{1}|K^{\prime}(u)|^{2}du/\sigma_{K,0}^{2}\pi\Big{\\}}^{1/2}}{\Gamma(s/2)}.$

The results hold true if instead of Assumption 7.1, 7.2, Assumption 7.3, 7.4
with some $r>2$ is assumed.

######  Proof 8.36 (Proof of Theorem 8.35).

Let $\tilde{k}_{i}(t):=\nabla_{\theta}\ell(\tilde{Z}_{i}(t),\theta(t))$ and
$\hat{K}(x)=K(x)$ or $\hat{K}(x)=K(x)x$, respectively. Define

$\Omega_{C}(t):=(nb_{n})^{-1}\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)A_{C}(i/n)^{\mkern-1.5mu\mathsf{T}}\tilde{k}_{i}(i/n)$

and
$D_{\tilde{k}}(t)=(nb_{n})^{-1}\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)\tilde{k}_{i}(i/n)$.
Similar to the discussion of $\Pi_{n}(t)$ in the proof of Lemma 8.28 (note
that the rates in (8.70) and (8.71) then change to $O(b_{n})$ instead of
$O(b_{n}^{2})$), we can show that

$\sup_{t\in(0,1)}|\Omega_{C}(t)-A_{C}(t)^{\mkern-1.5mu\mathsf{T}}\cdot
D_{\tilde{k}}(t)|=O_{\mathbb{P}}(\beta_{n}b_{n})=O_{\mathbb{P}}\Big{(}\frac{b_{n}^{1/2}\log(n)}{(nb_{n})^{1/2}\log(n)^{1/2}}\Big{)},$
(8.80)

which is $o_{\mathbb{P}}((nb_{n})^{-1/2}\log(n)^{-1/2})$ since
$b_{n}\log(n)^{2}\to 0$.

$\tilde{h}_{i}(t):=A_{C}(t)^{\mkern-1.5mu\mathsf{T}}\tilde{k}_{i}(t)$ is a
locally stationary process with long-run variance
$\Sigma^{2}_{\tilde{h}}(t)=\Sigma^{2}_{C}(t)$. By the result of Lemma 8.31, we
have that

$\lim_{n\to\infty}\mathbb{P}\Big{(}\frac{\sqrt{nb_{n}}}{\sigma_{\hat{K},0}}\sup_{t\in\mathcal{T}_{n}}\big{|}\Sigma_{C}^{-1}(t)\Omega_{C}(t)\big{|}-B_{\hat{K}}(m^{*})\leq\frac{u}{\sqrt{2\log(m^{*})}}\Big{)}=\exp(-2\exp(-u)).$
(8.81)

(i) By Theorem 8.4(i), we have

$\displaystyle\sup_{t\in\mathcal{T}_{n}}\big{|}V(t)\\{\hat{\theta}_{b_{n}}(t)-\theta(t)\\}-b_{n}^{2}\frac{\mu_{K,2}}{2}V(t)\theta^{\prime\prime}(t)-D_{\tilde{k}}(t)\big{|}$
(8.82) $\displaystyle=$ $\displaystyle
O_{\mathbb{P}}\big{(}b_{n}^{3}+(nb_{n})^{-1}b_{n}^{-1/2}\log(n)^{3/2}+(nb_{n})^{-1/2}b_{n}\log(n)\big{)}$
$\displaystyle=$ $\displaystyle
O_{\mathbb{P}}\Big{(}\frac{(nb_{n}^{7}\log(n))^{1/2}+(nb_{n}^{2}\log(n)^{-4})^{-1/2}+b_{n}\log(n)^{3/2}}{(nb_{n})^{1/2}\log(n)^{1/2}}\Big{)},$

which is $o_{\mathbb{P}}((nb_{n})^{-1/2}\log(n)^{-1/2})$ since
$nb_{n}^{7}\log(n)\to 0$, $nb_{n}^{2}\log(n)^{-4}\to\infty$ and
$b_{n}\log(n)^{2}\to 0$. Together with (8.80) and (8.81) (with $\hat{K}=K$),
this implies (8.77).

(ii) By Theorem 8.4(ii), we have

$\displaystyle\sup_{t\in\mathcal{T}_{n}}\big{|}\mu_{K,2}V(t)b_{n}\\{\widehat{\theta}_{b_{n}}^{\prime}(t)-\theta^{\prime}(t)\\}-b_{n}^{3}\frac{\mu_{K,4}}{2}V(t)\text{bias}(t)-D_{\tilde{k}}(t)\big{|}$
$\displaystyle=$ $\displaystyle
O_{\mathbb{P}}\big{(}b_{n}^{4}+(nb_{n})^{-1}b_{n}^{-1/2}\log(n)^{3/2}+(nb_{n})^{-1/2}b_{n}\log(n)\big{)}$
$\displaystyle=$ $\displaystyle
o_{\mathbb{P}}((nb_{n})^{-1/2}\log(n)^{-1/2}),$

as above. Together with (8.80) and (8.81) (with $\hat{K}(x)=K(x)x$), this
implies (8.78).

###  8.3 Proofs of Section 4

######  Proof 8.37 (Proof of Proposition 4.1).

(i) Lemma 8.20(i),(iii), Lemma 8.22 and the notation therein applied to
$g=\nabla_{\theta}^{2}\ell$ imply

$\displaystyle\sup_{t\in\mathcal{T}_{n}}|\hat{\mu}_{K,0,b_{n}}(t)\hat{V}_{b_{n}}(t)-\hat{\mu}_{K,0,b_{n}}(t)V(t)|$
(8.83) $\displaystyle\leq$ $\displaystyle\sup_{t\in\mathcal{T}_{n},\eta\in
E_{n}}|G_{n}^{c}(t,\eta)-\hat{G}_{n}(t,\eta)|+\sup_{t\in\mathcal{T}_{n},\eta\in
E_{n}}|\hat{G}_{n}(t,\eta)|$
$\displaystyle\quad\quad+\sup_{t\in\mathcal{T}_{n},\eta\in
E_{n}}|\mathbb{E}\hat{B}_{n}(t,\eta)-V^{\circ}(t,\eta)|+\sup_{t\in\mathcal{T}_{n}}|V^{\circ}(t,\hat{\eta}_{b_{n}})-\hat{\mu}_{K,0,b_{n}}(t)V(t)|$
$\displaystyle=$ $\displaystyle
O_{\mathbb{P}}((nb_{n})^{-1})+o_{\mathbb{P}}(\beta_{n})+O(b_{n})+\sup_{t\in\mathcal{T}_{n}}|V^{\circ}(t,\hat{\eta}_{b_{n}})-\hat{\mu}_{K,0,b_{n}}(t)V(t)|.$

We obtain similar as in the proof of Theorem 8.4(i) ((8.30) therein) that

$\sup_{t\in\mathcal{T}_{n}}|\hat{\eta}_{b_{n}}(t)-\eta_{b_{n}}(t)|=O_{\mathbb{P}}((nb_{n})^{-1/2}\log(n)+(nb_{n})^{-1}+\beta_{n}b_{n}^{2}+b_{n}^{2}).$

Since $\eta\mapsto V^{\circ}(t,\eta)$ is Lipschitz continuous by Lemma 8.6,
the result follows from (8.83) and $b_{n}\log(n)\to 0$.

(ii) follows similarly due to
$\nabla_{\theta}\ell\cdot\nabla_{\theta}\ell^{\mkern-1.5mu\mathsf{T}}\in\mathcal{H}(2M_{y},2M_{y},\chi,\bar{\bar{C}})$
with some $\bar{\bar{C}}>0$.

######  Proof 8.38 (Proof of Proposition 4.3).

Similar as in the proof of Theorem 8.4(i) by now using the explicit result of
Lemma 8.22(a) applied to $g=\ell$ (both for Assumption 7.1 and 7.3), we obtain

$\sup_{t\in(0,1)}\sup_{\eta\in
E_{n}}|L_{n,b_{n}}^{\circ}(t,\eta)-\tilde{L}_{b_{n}}^{\circ}(t,\eta)|=O_{\mathbb{P}}(\beta_{n}+(nb_{n})^{-1})+O(b_{n}),$

where
$\tilde{L}_{b_{n}}^{\circ}(t,\eta)=\int_{-t/b_{n}}^{(1-t)/b_{n}}K(x)L(t,\eta_{1}+\eta_{2}x)dx$.
By optimality of $\hat{\eta}_{b_{n}}(t)$,

$\displaystyle 0$ $\displaystyle\leq$ $\displaystyle
L_{n,b_{n}}^{\circ}(t,\theta(t))-L_{n,b_{n}}^{\circ}(t,\hat{\eta}_{b_{n}}(t))$
$\displaystyle\leq$
$\displaystyle\tilde{L}_{b_{n}}^{\circ}(t,\theta(t))-\tilde{L}_{b_{n}}^{\circ}(t,\hat{\eta}_{b_{n}}(t))+2\sup_{\eta\in
E_{n}}|L_{n,b_{n}}^{\circ}(t,\eta)-\tilde{L}_{b_{n}}^{\circ}(t,\eta)|.$

This implies

$\displaystyle\min\Big{\\{}\int_{-1}^{0}K(x)\big{\\{}L(t,\hat{\theta}_{b_{n}}(t)+b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)x)-L(t,\theta(t))\big{\\}}dx,$
$\displaystyle\quad\quad\int_{0}^{1}K(x)\big{\\{}L(t,\hat{\theta}_{b_{n}}(t)+b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)x)-L(t,\theta(t))\big{\\}}dx\Big{\\}}\leq
2\sup_{\eta\in
E_{n}}|L_{n,b_{n}}^{\circ}(t,\eta)-\tilde{L}_{b_{n}}^{\circ}(t,\eta)|.$ (8.84)

Assume that for some $\iota>0$,
$\limsup_{n\to\infty}\sup_{t\in(0,1)}|\hat{\eta}_{b_{n}}(t)-(\theta(t)^{\mkern-1.5mu\mathsf{T}},0)^{\mkern-1.5mu\mathsf{T}}|\geq\iota$.
Then there exists $t\in(0,1)$ such that either (c1)

$|\hat{\theta}_{b_{n}}(t)-\theta(t)|\geq\frac{1}{2}|b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)|$

and thus $|\hat{\theta}_{b_{n}}(t)-\theta(t)|>\iota/3$, or (c2)

$|\hat{\theta}_{b_{n}}(t)-\theta(t)|<\frac{1}{2}|b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)|,$

and thus $|b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)|>2\iota/3$.
In case (c1), we have
$|\hat{\theta}_{b_{n}}(t)+b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)x-\theta(t)|\geq|\hat{\theta}_{b_{n}}(t)-\theta(t)|-|x||b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)|\geq\frac{\iota}{6}$
for $x\in[0,\frac{1}{4}]$, thus with some $c_{0}>0$,

$\int_{0}^{1}K(x)\big{\\{}L(t,\hat{\theta}_{b_{n}}(t)+b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)x)-L(t,\theta(t))\big{\\}}dx\geq\int_{0}^{1/4}K(x)\big{\\{}L(t,\hat{\theta}_{b_{n}}(t)+b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)x)-L(t,\theta(t))\big{\\}}dx\geq
c_{0}$

since $\theta\mapsto L(t,\theta)$ is continuous and attains its unique minimum
at $\theta=\theta(t)$.
In case (c2), we have
$|\hat{\theta}_{b_{n}}(t)+b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)x-\theta(t)|\geq|x||b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)|-|\hat{\theta}_{b_{n}}(t)-\theta(t)|\geq\frac{\iota}{6}$
for $x\in[\frac{3}{4},1]$, thus with some $c_{0}>0$,

$\int_{0}^{1}K(x)\big{\\{}L(t,\hat{\theta}_{b_{n}}(t)+b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)x)-L(t,\theta(t))\big{\\}}dx\geq\int_{3/4}^{1}K(x)\big{\\{}L(t,\hat{\theta}_{b_{n}}(t)+b_{n}\widehat{\theta}_{b_{n}}^{\prime}(t)x)-L(t,\theta(t))\big{\\}}dx\geq
c_{0}.$

In both cases, (8.84) becomes a contradiction. Therefore,

$\sup_{t\in(0,1)}|\hat{\eta}_{b_{n}}(t)-\eta_{b_{n}}(t)|=o_{\mathbb{P}}(1).$

Using summation-by-parts and Gaussian approximation similar to that presented
in Theorem 8.33 for the process
$\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))$, there exists i.i.d.
$V_{1},V_{2},\ldots\sim N(0,I_{s\times s})$ on a richer probability space such
that, for $\pi_{n}$ as in (8.76)

$\sup_{t\in(0,1)}\big{|}(nb_{n})^{-1}\sum_{i=1}^{n}K_{b_{n}}(i/n-t)(\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))-V_{i})\big{\\}}|=O_{\mathbb{P}}((nb_{n})^{-1}\pi_{n})=O_{\mathbb{P}}((nb_{n})^{-1/2}\log(n)).$
(8.85)

Thus one can replace $\sup_{t\in\mathcal{T}_{n}}$ by $\sup_{t\in(0,1)}$ in
(8.27). A careful examination of the rest of the proof of Theorem 8.4(i) (with
Lemma 8.24(8.59) replaced by Lemma 8.24(8.60)) now yields the result

$\sup_{t\in(0,1)}|\tilde{V}_{b_{n}}^{\circ}(t)\cdot(\hat{\eta}_{b_{n}}(t)-\eta_{b_{n}}(t))-\nabla_{\eta}L_{n,b_{n}}^{\circ,c}(t,\eta_{b_{n}}(t))\big{|}=O_{\mathbb{P}}(\tau_{n}^{(1)}),$
(8.86)

where (we shortly write $\hat{\mu}_{K,j}(t)=\hat{\mu}_{K,j,b_{n}}(t)$)

$\tilde{V}_{b_{n}}^{\circ}(t)=\begin{pmatrix}\hat{\mu}_{K,0}(t)&\hat{\mu}_{K,1}(t)\\\
\hat{\mu}_{K,1}(t)&\hat{\mu}_{K,2}(t)\end{pmatrix}\otimes V(t).$

By Lemma 8.20(i), Lemma 8.24 and Lemma 8.28, we obtain furthermore with
$U_{i,n}(t)=(K_{b_{n}}(i/n-t),K_{b_{n}}(i/n-t)\cdot(i/n-t)b_{n}^{-1})^{\mkern-1.5mu\mathsf{T}}$:

$\displaystyle\sup_{t\in(0,1)}\big{|}\nabla_{\eta}L_{n,b_{n}}^{\circ,c}(t,\eta_{b_{n}}(t))-b_{n}^{2}\begin{pmatrix}\hat{\mu}_{K,2}(t)\\\
\hat{\mu}_{K,3}(t)\end{pmatrix}\otimes[V(t)\theta^{\prime\prime}(t)]$
$\displaystyle\quad\quad\quad\quad-(nb_{n})^{-1}\sum_{i=1}^{n}U_{i,n}(t)\otimes\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))\big{|}=O_{\mathbb{P}}(\beta_{n}b_{n}^{2}+b_{n}^{3}+(nb_{n})^{-1}).$
(8.87)

Recalling the proof of Lemma 8.31, (8.74) and (8.75) and the proof of Theorem
8.35, (8.80) we see that there exist i.i.d. $V_{i}\sim N(0,I_{s\times s})$
such that both for $\hat{K}=K$ and $\hat{K}(x)=K(x)\cdot x$,

$\displaystyle\sup_{t\in(0,1)}\big{|}A_{C}(t)^{\mkern-1.5mu\mathsf{T}}(nb_{n})^{-1}\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)\nabla_{\theta}\ell(\tilde{Z}_{i}(i/n),\theta(i/n))-\Sigma_{C}(t)(nb_{n})^{-1}\sum_{i=1}^{n}\hat{K}_{b_{n}}(i/n-t)V_{i}\big{|}$
(8.88) $\displaystyle=$ $\displaystyle
O_{\mathbb{P}}\Big{(}\frac{\log(n)^{2}\big{(}b_{n}n^{\frac{2\gamma+\varsigma\gamma-\varsigma}{\varsigma+4\gamma+2\gamma\varsigma}}\big{)}^{-1/2}}{(nb_{n})^{1/2}\log(n)^{1/2}}+\frac{b_{n}\log(n)^{3/2}}{(nb_{n})^{1/2}\log(n)^{1/2}}+\frac{b_{n}^{1/2}\log(n)}{(nb_{n})^{1/2}\log(n)^{1/2}}\Big{)}=:O_{\mathbb{P}}(w_{n}).$

With (8.86) and

$\displaystyle\tilde{V}_{b_{n}}^{\circ}(t)^{-1}$ $\displaystyle=$
$\displaystyle\begin{pmatrix}\hat{\mu}_{K,0}(t)&\hat{\mu}_{K,1}(t)\\\
\hat{\mu}_{K,1}(t)&\hat{\mu}_{K,2}(t)\end{pmatrix}^{-1}\otimes V(t)^{-1}$
$\displaystyle=$
$\displaystyle\frac{1}{\hat{\mu}_{K,2}(t)N_{b_{n}}^{(0)}(t)}\begin{pmatrix}\hat{\mu}_{K,2}(t)V(t)^{-1}&-\hat{\mu}_{K,1}(t)V(t)^{-1}\\\
-\hat{\mu}_{K,1}(t)V(t)^{-1}&\hat{\mu}_{K,0}(t)V(t)^{-1}\end{pmatrix},$

we obtain:

$\displaystyle\sup_{t\in(0,1)}\Big{|}N_{b_{n}}^{(0)}(t)\cdot\\{\hat{\theta}_{b_{n},C}(t)-\theta_{C}(t)\\}$
$\displaystyle\quad\quad\quad\quad-\Big{[}A_{C}(t)^{\mkern-1.5mu\mathsf{T}}\nabla_{\eta_{1}}L_{n,b_{n}}^{\circ,c}(t,\eta_{b_{n}}(t))-\frac{\hat{\mu}_{K,1}(t)}{\hat{\mu}_{K,2}(t)}A_{C}(t)^{\mkern-1.5mu\mathsf{T}}\nabla_{\eta_{2}}L_{n,b_{n}}^{\circ,c}(t,\eta_{b_{n}}(t))\Big{]}\Big{|}=O_{\mathbb{P}}(\tau_{n}^{(1)}).$

With (8.87) and (8.88), we have

$\displaystyle\sup_{t\in(0,1)}\Big{|}N^{(0)}_{b_{n}}(t)\cdot\\{\hat{\theta}_{b_{n},C}(t)-\theta_{C}(t)\\}$
$\displaystyle\quad\quad\quad\quad\quad+b_{n}^{2}N^{(1)}_{b_{n}}(t)\theta_{C}^{\prime\prime}(t)-\Sigma_{C}(t)\big{\\{}Q_{b_{n}}^{(0)}(t)-\frac{\hat{\mu}_{K,1}(t)}{\hat{\mu}_{K,2}(t)}Q_{b_{n}}^{(1)}(t)\big{\\}}\Big{|}$
$\displaystyle=$ $\displaystyle
O_{\mathbb{P}}(\tau_{n}^{(1)}+(\beta_{n}b_{n}^{2}+b_{n}^{3}+(nb_{n})^{-1})+w_{n}),$

which finishes the proof.

##  9 Proofs of Assumption sets

###  9.1 Case 1: Recursively defined models

######  Proposition 9.1.

If Assumption 2.2 holds, then Assumption 7.1, 7.2 is fulfilled with every
$r=2+\tilde{a}$, $\tilde{a}<a$, the corresponding $M$ and $\gamma>2$
arbitrarily large.

It holds that $V(t)=\Lambda(t)$. If (i) $\mathbb{E}\zeta_{0}^{3}=0$, or (ii)
$\mu(x,\theta)\equiv 0$ or (iii) $\sigma(x,\theta)\equiv\beta_{0}$ and
$\mathbb{E}m(\tilde{X}_{0}(t))=0$, then

$I(t)=\big{(}\begin{smallmatrix}I_{k}&0\\\
0&(\mathbb{E}\zeta_{0}^{4}-1)I_{l+1}/2\end{smallmatrix}\big{)}\cdot V(t),$

where $I_{d}$ denotes the $d$-dimensional identity matrix.

######  Proof 9.2 (Proof of Proposition 9.1).

Choose $0<\tilde{a}<a$ small enough such that (2.5) holds with
$\|\zeta_{0}\|_{(2+\tilde{a})M}$ replaced by $\|\zeta_{0}\|_{2M}$ (this is
possible due to continuity of the term in $\tilde{a}=0$). Let
$q=(2+\tilde{a})M$. Let
$\nu=(\nu_{0},\ldots,\nu_{l})^{\mkern-1.5mu\mathsf{T}}$ and
$m=(m_{1},\ldots,m_{k})^{\mkern-1.5mu\mathsf{T}}$. As known from Proposition
4.3 and Lemma 4.4 in [13] the process $(Y_{i})_{i=1,\ldots,n}$ described by
(2.1) exists and fulfills Assumption 7.1(A5), (A7) with
$\delta_{q}^{Y}(i)=O(c^{i})$ for some $0<c<1$ and $q\geq 1$ if the recursion
function $G_{\zeta}(y,t):=\mu(y,\theta(t))+\sigma(y,\theta(t))\zeta$ obeys

$\Big{\|}\sup_{t\in[0,1]}\sup_{y\not=y^{\prime}}\frac{|G_{\zeta_{0}}(y,t)-G_{\zeta_{0}}(y^{\prime},t)|}{|y-y^{\prime}|_{\chi,1}}\Big{\|}_{q}\leq
1$ (9.1)

and

$\sup_{t\in[0,1]}\|C(\tilde{X}_{t}(t))\|_{q}<\infty,\quad
C(y):=\sup_{t\not=t^{\prime}}\frac{\|G_{\zeta_{0}}(y,t)-G_{\zeta_{0}}(y,t^{\prime})\|_{q}}{|t-t^{\prime}|},$
(9.2)

where $|z|_{\chi,1}:=\sum_{i=1}^{p}|z_{i}|\chi_{i}$ for some
$\chi=(\chi_{i})_{i=1,\ldots,p}\in\mathbb{R}_{\geq 0}^{p}$ with
$|\chi|_{1}=\sum_{i=1}^{p}\chi_{i}<1$. Here, we can bound

$|\mu(y,\theta)-\mu(y^{\prime},\theta)|\leq\sum_{i=1}^{k}|\alpha_{i}||y-y^{\prime}|_{\kappa_{i\cdot},1}\leq|y-y^{\prime}|_{\kappa^{(\mu)}(\alpha),1},$
(9.3)

where $\chi^{(\mu)}(\alpha):=\sum_{i=1}^{k}|\alpha_{i}|\kappa_{i\cdot}$.
Furthermore,

$\displaystyle|\sigma(y,\theta)^{2}-\sigma(y^{\prime},\theta)^{2}|$
$\displaystyle\leq$
$\displaystyle\sum_{i=0}^{l}\beta_{i}|\nu_{i}(y)-\nu_{i}(y^{\prime})|$
$\displaystyle\leq$
$\displaystyle\sum_{i=0}^{l}\sqrt{\beta_{i}}|y-y^{\prime}|_{\rho_{i\cdot},1}\cdot\big{(}\sqrt{\beta_{i}\nu_{i}(y)}+\sqrt{\beta_{i}\nu_{i}(y^{\prime})}\big{)}$
$\displaystyle\leq$
$\displaystyle\sum_{i=0}^{l}\sqrt{\beta_{i}}|y-y^{\prime}|_{\rho_{i\cdot},1}\cdot\big{(}\sigma(y,\theta)+\sigma(y^{\prime},\theta)\big{)},$

i.e.

$|\sigma(y,\theta)-\sigma(y^{\prime},\theta)|\leq|y-y^{\prime}|_{\chi^{(\sigma)}(\beta),1},$
(9.4)

where $\chi^{(\sigma)}(\beta):=\sum_{i=1}^{l}\sqrt{\beta_{i}}\rho_{i\cdot}$.
Define

$\chi^{(\mu,max)}_{j}:=\sup_{t}|\chi^{(\mu)}(\alpha(t))_{j}|,\quad\quad\chi^{(\sigma,max)}_{j}:=\sup_{t}|\chi^{(\sigma)}(\beta(t))_{j}|.$

Since
$\theta(t)=(\alpha(t)^{\mkern-1.5mu\mathsf{T}},\beta(t)^{\mkern-1.5mu\mathsf{T}})^{\mkern-1.5mu\mathsf{T}}\in\Theta$,
we have that

$\sum_{j=1}^{p}(\chi_{j}^{(\mu,max)}+\|\zeta_{0}\|_{q}\chi_{j}^{(\sigma,max)})=\sum_{j=1}^{p}\big{(}\sup_{t}|\chi^{(\mu)}(\alpha(t))_{j}|+\|\zeta_{0}\|_{q}\sup_{t}|\chi^{(\sigma)}(\beta(t))_{j}|\big{)}<1.$

Define
$\chi_{j}:=\chi^{(\mu,max)}_{j}+\|\zeta_{0}\|_{q}\chi^{(\sigma,max)}_{j}$.
Then we have for all $t,y\not=y^{\prime}$:

$|\mu(y,\theta(t))-\mu(y^{\prime},\theta(t))|+\|\zeta_{0}\|_{q}|\sigma(y,\theta(t))-\sigma(y^{\prime},\theta(t))|\leq|y-y^{\prime}|_{\chi,1},$

which implies (9.1). Proposition 4.3 from [13] now implies the existence of
$Y_{i}$, the stationary approximation $\tilde{Y}_{i}(t)$ and
$\sup_{t}\|\tilde{Y}_{0}(t)\|_{q}<\infty$. By Lipschitz continuity of $\theta$
with constant $L_{\theta}$, we have

$|\mu(y,\theta(t))-\mu(y,\theta(t^{\prime}))|\leq
L_{\theta}|t-t^{\prime}|\sum_{i=1}^{k}|m_{i}(y)|,$ (9.5)

and

$\displaystyle|\sigma(y,\theta(t))^{2}-\sigma(y,\theta(t^{\prime}))^{2}|$
$\displaystyle\leq$ $\displaystyle
L_{\theta}|t-t^{\prime}|\sum_{i=0}^{l}\sqrt{\nu_{i}(y)}\frac{1}{2\beta_{min}^{1/2}}\big{(}\sqrt{\beta_{i}(t)\nu_{i}(y)}+\sqrt{\beta_{i}(t^{\prime})\nu_{i}(y)}\big{)}$
$\displaystyle\leq$
$\displaystyle\frac{L_{\theta}}{2\beta_{min}^{1/2}}|t-t^{\prime}|\sum_{i=0}^{l}\sqrt{\nu_{i}(y)}(\sigma(y,\theta(t))+\sigma(y,\theta(t^{\prime}))),$

which shows that

$|\sigma(y,\theta(t))-\sigma(y,\theta(t^{\prime}))|\leq\frac{L_{\theta}}{2\beta_{min}^{1/2}}\sum_{i=0}^{l}\sqrt{\nu_{i}(y)}.$
(9.6)

Note that (2.4) implies

$m_{i}(y),\sqrt{\nu_{i}(y)}\leq C_{1}|y|_{1}+C_{2},$

with some constants $C_{1},C_{2}>0$. By (9.5), (9.6), we have for
$t\not=t^{\prime}$

$\displaystyle\|G_{\zeta_{0}}(y,t)-G_{\zeta_{0}}(y,t^{\prime})\|_{q}$
$\displaystyle\leq$
$\displaystyle|\mu(y,\theta(t))-\mu(y,\theta(t^{\prime}))|+\|\zeta_{0}\|_{q}|\sigma(y,\theta(t))-\sigma(y,\theta(t^{\prime}))|$
$\displaystyle\leq$ $\displaystyle
C_{3}|t-t^{\prime}|\big{(}1+|y|_{1}\big{)},$

with some constant $C_{3}>0$. Since $\sup_{t}\|\tilde{Y}_{0}(t)\|_{q}<\infty$,
(9.2) follows.

We now inspect the properties of the function $\ell$. First note that the
recursion of the stationary approximation,

$\tilde{Y}_{i}(t)=\mu(\tilde{X}_{i}(t),\theta(t))+\sigma(\tilde{X}_{i}(t),\theta(t))\zeta_{i},$

implies $\mathbb{E}\tilde{Y}_{0}(t)=0$ and
$\mathbb{E}\tilde{Y}_{0}(t)^{2}=\mathbb{E}\mu(\tilde{X}_{0}(t),\theta(t))^{2}+\mathbb{E}\sigma(\tilde{X}_{0}(t),\theta(t))^{2}\geq\beta_{min}\nu_{min}>0$.
Furthermore, for $L(t,\theta):=\mathbb{E}\ell(\tilde{Z}_{0}(t),\theta)$ it
holds that

$\displaystyle L(t,\theta)-L(t,\theta(t))$ $\displaystyle=$
$\displaystyle\mathbb{E}\Big{(}\frac{\mu(\tilde{X}_{0}(t),\theta)-\mu(\tilde{X}_{0}(t),\theta(t))}{\sigma(\tilde{X}_{0}(t),\theta)}\Big{)}^{2}$
(9.7)
$\displaystyle\quad\quad\quad\quad+\mathbb{E}\Big{[}\frac{\sigma(\tilde{X}_{0}(t),\theta(t))^{2}}{\sigma(\tilde{X}_{0}(t),\theta)^{2}}-\log\frac{\sigma(\tilde{X}_{0}(t),\theta(t))^{2}}{\sigma(\tilde{X}_{0}(t),\theta)^{2}}-1\Big{]}.$

In the following we use the notation
$|x|_{A}^{2}:=x^{\mkern-1.5mu\mathsf{T}}Ax$ for a weighted vector norm. Note
that

$\mathbb{E}\Big{(}\frac{\mu(\tilde{X}_{0}(t),\theta)-\mu(\tilde{X}_{0}(t),\theta(t))}{\sigma(\tilde{X}_{0}(t),\theta)}\Big{)}^{2}\geq
c_{0}|\alpha-\alpha(t)|_{M_{1}(t)}^{2},$ (9.8)

with $c_{0}=(\max_{\theta\in\Theta}\max_{i}\theta_{i}^{2})^{-1}$ and
$M_{1}(t):=\mathbb{E}[\frac{m(\tilde{X}_{0}(t))m(\tilde{X}_{0}(t))^{\mkern-1.5mu\mathsf{T}}}{\mathbf{1}\nu(\tilde{X}(t))\nu(\tilde{X}(t))^{\mkern-1.5mu\mathsf{T}}\mathbf{1}}]$.
If $M_{1}(t)$ was not positive definite, this would imply that there exists
$v\in\mathbb{R}^{k}$ such that $v^{\prime}M(t)v=0$, which in turn would imply
$v^{\prime}\mu(\tilde{X}_{0}(t))\mu(\tilde{X}_{0}(t))v=0$ a.s. and thus non-
positive definiteness of
$\mathbb{E}[\mu(\tilde{X}_{0}(t))\mu(\tilde{X}_{0}(t))^{\mkern-1.5mu\mathsf{T}}]$
which is a contradiction to the assumption.
By a Taylor expansion of $f(x)=x-\log(x)-1$, we obtain

$\displaystyle\mathbb{E}\Big{[}\frac{\sigma(\tilde{X}_{0}(t),\theta(t))^{2}}{\sigma(\tilde{X}_{0}(t),\theta)^{2}}-\log\frac{\sigma(\tilde{X}_{0}(t),\theta(t))^{2}}{\sigma(\tilde{X}_{0}(t),\theta)^{2}}-1\Big{]}$
(9.9) $\displaystyle\geq$
$\displaystyle\frac{1}{2}\mathbb{E}\Big{[}\frac{(\sigma(\tilde{X}_{0}(t),\theta)^{2}-\sigma(\tilde{X}_{0}(t),\theta(t))^{2})^{2}}{(\sigma(\tilde{X}_{0}(t),\theta)^{2}-\sigma(\tilde{X}_{0}(t),\theta(t))^{2})^{2}+\sigma(\tilde{X}_{0}(t),\theta)^{4}}\Big{]}$
$\displaystyle\geq$
$\displaystyle\frac{c_{0}}{10}|\beta-\beta(t)|_{M_{2}(t)}^{2},$

where
$M_{2}(t)=\mathbb{E}[\frac{\nu(\tilde{X}_{0}(t))\nu(\tilde{X}_{0}(t))^{\mkern-1.5mu\mathsf{T}}}{\mathbf{1}\nu(\tilde{X}(t))\nu(\tilde{X}(t))^{\mkern-1.5mu\mathsf{T}}\mathbf{1}}]$
is positive definite by assumption (use a similar argumentation as above). By
(9.7), (9.8) and (9.9) we conclude that $\theta\mapsto L(t,\theta)$ is
uniquely minimized in $\theta=\theta(t)$. This shows 7.1(A3).

Omitting the arguments $z=(y,x)$ and $\theta$, we have

$\displaystyle\ell$ $\displaystyle=$
$\displaystyle\frac{1}{2}\Big{[}\frac{(y-\langle\alpha,m\rangle)^{2}}{\langle\beta,\nu\rangle}+\log\langle\beta,\nu\rangle\Big{]},$
(9.10) $\displaystyle\nabla_{\theta}\ell$ $\displaystyle=$
$\displaystyle-\frac{\nabla_{\theta}m}{\sigma}\Big{(}\frac{y-m}{\sigma}\Big{)}+\frac{\nabla_{\theta}(\sigma^{2})}{2\sigma^{2}}\Big{[}1-\Big{(}\frac{y-m}{\sigma}\Big{)}^{2}\Big{]}$
(9.11) $\displaystyle=$
$\displaystyle\begin{pmatrix}-\frac{m}{\sigma}\Big{(}\frac{y-m}{\sigma}\Big{)}\\\
\frac{\nu}{2\sigma^{2}}\big{[}1-\big{(}\frac{y-m}{\sigma}\big{)}^{2}\big{]}\end{pmatrix}=\begin{pmatrix}\frac{m}{\langle\beta,\nu\rangle}(y-\langle\alpha,m\rangle)\\\
\frac{\nu}{2\langle\beta,\nu\rangle}\big{(}1-\frac{(y-\langle\alpha,m\rangle)^{2}}{\langle\beta,\nu\rangle}\big{)}\end{pmatrix},$
$\displaystyle\nabla_{\theta}^{2}\ell$ $\displaystyle=$
$\displaystyle\frac{\nabla_{\theta}m\nabla_{\theta}m^{\mkern-1.5mu\mathsf{T}}}{\sigma^{2}}+\big{(}\frac{y-m}{\sigma}\big{)}\cdot\Big{[}\frac{\nabla_{\theta}m\nabla_{\theta}(\sigma^{2})^{\mkern-1.5mu\mathsf{T}}+\nabla_{\theta}(\sigma^{2})\nabla_{\theta}m^{\mkern-1.5mu\mathsf{T}}}{\sigma^{3}}-\frac{\nabla_{\theta}^{2}m}{\sigma}\Big{]}$
(9.12)
$\displaystyle\quad\quad\quad+\frac{\nabla_{\theta}^{2}(\sigma^{2})}{2\sigma^{2}}\Big{[}1-\big{(}\frac{y-m}{\sigma}\big{)}^{2}\Big{]}+\frac{\nabla_{\theta}(\sigma^{2})\nabla_{\theta}(\sigma^{2})^{\mkern-1.5mu\mathsf{T}}}{2\sigma^{4}}\Big{[}2\Big{(}\frac{y-m}{\sigma}\big{)}^{2}-1\Big{]}$
$\displaystyle=$
$\displaystyle\begin{pmatrix}\frac{mm^{\mkern-1.5mu\mathsf{T}}}{\sigma^{2}}&\frac{y-m}{\sigma^{2}}\cdot
m\nu^{\mkern-1.5mu\mathsf{T}}\\\ \frac{y-m}{\sigma^{2}}\cdot\nu
m^{\mkern-1.5mu\mathsf{T}}&\frac{\nu\nu^{\mkern-1.5mu\mathsf{T}}}{2\sigma^{4}}\big{[}2\big{(}\frac{y-m}{\sigma}\big{)}^{2}-1\big{]}\end{pmatrix}$
$\displaystyle=$
$\displaystyle\begin{pmatrix}\frac{mm^{\mkern-1.5mu\mathsf{T}}}{\langle\beta,\nu\rangle}&\frac{y-\langle\alpha,m\rangle}{\langle\beta,\nu\rangle^{2}}\cdot
m\nu^{\mkern-1.5mu\mathsf{T}}\\\
\frac{y-\langle\alpha,m\rangle}{\langle\beta,\nu\rangle^{2}}\cdot\nu
m^{\mkern-1.5mu\mathsf{T}}&\frac{\nu\nu^{\mkern-1.5mu\mathsf{T}}}{2\langle\beta,\nu\rangle^{2}}\big{[}2\frac{(y-\langle\alpha,m\rangle)^{2}}{\langle\beta,\nu\rangle}-1\big{]}\end{pmatrix}.$

Since $\zeta_{1}$ is independent of $\tilde{X}_{0}(t)\in\mathcal{F}_{0}$ and
$\mathbb{E}\zeta_{1}=0$, $\mathbb{E}\zeta_{1}^{2}=1$, we conclude that

$\mathbb{E}[\nabla_{\theta}\ell(\tilde{Z}_{0}(t),\theta(t))|\mathcal{F}_{t-1}]=\mathbb{E}\Big{[}-\frac{\mu(\tilde{X}_{j}(t),\theta(t))}{\sigma(\tilde{X}_{0}(t),\theta(t))}\zeta_{0}+\frac{\nu(\tilde{X}_{0}(t),\theta(t))}{2\sigma(\tilde{X}_{j}(t),\theta(t))^{2}}(1-\zeta_{0}^{2})\big{|}\mathcal{F}_{t-1}\Big{]}=0,$

i.e. $\nabla_{\theta}\ell(\tilde{Z}_{1}(t),\theta(t))$ is a martingale
difference sequence, showing that $V(t)=\Lambda(t)$. We furthermore have that
(we omit the arguments $(\tilde{X}_{0}(t),\theta(t))$ of $\mu$, $\sigma$ in
the following):

$V(t)=\mathbb{E}\nabla_{\theta}^{2}\ell(\tilde{Z}_{0}(t),\theta(t))=\begin{pmatrix}\mathbb{E}\big{[}\frac{mm^{\mkern-1.5mu\mathsf{T}}}{\langle\beta,\nu\rangle}\big{]}&0\\\
0&\mathbb{E}\big{[}\frac{\nu\nu^{\mkern-1.5mu\mathsf{T}}}{2\langle\beta,\nu\rangle^{2}}\big{]}\end{pmatrix}.$

With a similar argumentation as above, we conclude that $V(t)$ is positive
definite (which then implies by continuity that the smallest eigenvalue of
$V(t)$ is bounded away from 0 uniformly in $t$). By the martingale difference
property, $I(t)=\Lambda(t)$. Omitting the arguments
$(\tilde{X}_{0}(t),\theta(t))$,

$\displaystyle I(t)$ $\displaystyle=$
$\displaystyle\mathbb{E}[\nabla_{\theta}(\tilde{Z}_{j}(t),\theta(t))\nabla_{\theta}(\tilde{Z}_{0}(t),\theta(t))^{\mkern-1.5mu\mathsf{T}}]$
$\displaystyle=$
$\displaystyle\begin{pmatrix}\mathbb{E}\big{[}\frac{mm^{\mkern-1.5mu\mathsf{T}}}{\sigma^{2}}\big{]}&\mathbb{E}[\zeta_{0}^{3}]\cdot\mathbb{E}\big{[}\frac{m\nu^{\mkern-1.5mu\mathsf{T}}}{2\sigma^{3}}\big{]}\\\
\mathbb{E}[\zeta_{0}^{3}]\cdot\mathbb{E}\big{[}\frac{\nu
m^{\mkern-1.5mu\mathsf{T}}}{2\sigma^{3}}\big{]}&\frac{\mathbb{E}[\zeta_{0}^{4}]-1}{4}\cdot\mathbb{E}\big{[}\frac{\nu\nu^{\mkern-1.5mu\mathsf{T}}}{\sigma^{4}}\big{]}\end{pmatrix}$
$\displaystyle=$
$\displaystyle\mathbb{E}\Big{[}\frac{1}{\sigma^{2}}\begin{pmatrix}m\\\
\frac{\mathbb{E}[\zeta_{0}^{3}]}{2\sigma}\nu\end{pmatrix}^{\mkern-1.5mu\mathsf{T}}\begin{pmatrix}m\\\
\frac{\mathbb{E}[\zeta_{0}^{3}]}{2\sigma}\nu\end{pmatrix}\Big{]}+\begin{pmatrix}0&0\\\
0&\big{(}\frac{\mathbb{E}[\zeta_{0}^{4}]-\mathbb{E}[\zeta_{0}^{3}]^{2}-1}{4}\big{)}\mathbb{E}\big{[}\frac{\nu\nu^{\mkern-1.5mu\mathsf{T}}}{\sigma^{4}}\big{]}\end{pmatrix},$

which is positive semidefinite since
$\mathbb{E}[\zeta_{0}^{3}]=\mathbb{E}[\zeta_{0}(\zeta_{0}^{2}-1)]\leq\mathbb{E}[\zeta_{0}^{2}]^{1/2}\mathbb{E}[(\zeta_{0}^{2}-1)^{2}]^{1/2}=(\mathbb{E}[\zeta_{0}^{4}]-1)^{1/2}$.
Positive definiteness follows from the fact that
$(v_{1},v_{2})^{\mkern-1.5mu\mathsf{T}}I(t)(v_{1},v_{2})=0$ implies
$\nu^{\mkern-1.5mu\mathsf{T}}v_{2}=0$ a.s. from the last summand and
$v_{1}^{\mkern-1.5mu\mathsf{T}}m+\frac{\mathbb{E}[\zeta_{0}^{3}]}{2\sigma}v_{2}^{\mkern-1.5mu\mathsf{T}}\nu=0$
a.s. from the first summand, i.e. $v_{1}^{\mkern-1.5mu\mathsf{T}}m=0$ a.s.
which leads to a contradiction to either the positive definiteness of
$\mathbb{E}[\nu\nu^{\mkern-1.5mu\mathsf{T}}]$ or
$\mathbb{E}[mm^{\mkern-1.5mu\mathsf{T}}]$. So we obtain that Assumption
7.1(A4) is fulfilled.

A careful inspection of (9.10), (9.11) and (9.12) shows that
$\ell,\nabla_{\theta}\ell,\nabla_{\theta}^{2}\ell\in\mathcal{H}(2,3,\tilde{\chi},\tilde{C})$
with some $\tilde{C}>0$ and $\tilde{\chi}=(1,\ldots,1,0,0,\ldots)$ consisting
of $\max\\{k,l\\}$ ones followed by zeros, which shows Assumption 7.1(A1). In
the special case $\mu(x,\theta)\equiv 0$, it seems as if no direct improvement
of the value $M$ is possible. In the special case of
$\sigma(x,\theta)^{2}\equiv\beta_{0}$, we have

$\displaystyle\ell$ $\displaystyle=$
$\displaystyle\frac{1}{2}\Big{[}\frac{(y-\langle\alpha,m\rangle)^{2}}{\beta_{0}}+\log\beta_{0}\Big{]},$
$\displaystyle\nabla_{\theta}\ell$ $\displaystyle=$
$\displaystyle\begin{pmatrix}\frac{m}{\beta_{0}}(y-\langle\alpha,m\rangle)\\\
\frac{1}{2\beta_{0}}(1-\frac{(y-\langle\alpha,m\rangle)^{2}}{\beta_{0}})\end{pmatrix},$
$\displaystyle\nabla_{\theta}^{2}\ell$ $\displaystyle=$
$\displaystyle\begin{pmatrix}\frac{mm^{\mkern-1.5mu\mathsf{T}}}{\beta_{0}}&\frac{y-\langle\alpha,m\rangle}{\beta_{0}^{2}}m\\\
\frac{y-\langle\alpha,m\rangle}{\beta_{0}^{2}}m^{\mkern-1.5mu\mathsf{T}}&\frac{1}{2\beta_{0}^{2}}\big{[}2\frac{(y-\langle\alpha,m\rangle)^{2}}{\beta_{0}}-1\big{]}\end{pmatrix},$

which implies that
$\ell,\nabla_{\theta}\ell,\nabla_{\theta}^{2}\ell\in\mathcal{H}(2,2,\tilde{\chi},\tilde{C})$.

Now suppose that Assumption 7.2(B1) is fulfilled. We use results from Section
4 in [13] to show that the first derivative process
$\partial_{t}\tilde{Y}_{i}(t)$ exists and fulfills a Lipschitz condition. By
assumption, with some constant $C>0$,

$\displaystyle|\partial_{x_{j}}G_{\zeta_{0}}(x,t)-\partial_{x_{j}}G_{\zeta_{0}}(x^{\prime},t)|$
$\displaystyle\leq$
$\displaystyle|\langle\alpha(t),\partial_{x_{j}}m(x)-\partial_{x_{j}}m(x^{\prime})\rangle|$
$\displaystyle\quad\quad+\big{|}\frac{\langle\beta(t),\partial_{x_{j}}\nu(x)\rangle}{2\langle\beta(t),\nu(x)\rangle^{1/2}}-\frac{\langle\beta(t),\partial_{x_{j}}\nu(x^{\prime})\rangle}{2\langle\beta(t),\nu(x^{\prime})\rangle^{1/2}}\big{|}\cdot|\zeta_{0}|$
$\displaystyle\leq$ $\displaystyle|\alpha(t)|_{\infty}C|x-x^{\prime}|_{1}$
$\displaystyle\quad\quad+|\zeta_{0}|\cdot\Big{(}\frac{1}{2\beta_{min}^{1/2}}|\langle\beta(t),\partial_{x_{j}}\nu(x)-\partial_{x_{j}}\nu(x^{\prime})\rangle|$
$\displaystyle+\frac{|\langle\beta(t),\partial_{x_{j}}\nu(x^{\prime})\rangle|}{2}\frac{|\langle\beta(t),\nu(x)-\nu(x^{\prime})\rangle|}{\langle\beta(t),\nu(x)\rangle^{1/2}\langle\beta(t),\nu(x^{\prime})\rangle^{1/2}(\langle\beta(t),\nu(x)\rangle^{1/2}+\langle\beta(t),\nu(x^{\prime})\rangle^{1/2})}\Big{)}.$

By assumption,

$|\langle\beta(t),\partial_{x_{j}}\nu(x)-\partial_{x_{j}}\nu(x^{\prime})\rangle|\leq
C|\beta(t)|_{\infty}|x-x^{\prime}|_{1}.$

Furthermore,

$\displaystyle|\langle\beta(t),\nu(x)-\nu(x^{\prime})\rangle|$
$\displaystyle\leq$
$\displaystyle\sum_{i=1}^{l}\beta_{i}(t)|\sqrt{\nu_{i}(x)}-\sqrt{\nu_{i}(x^{\prime})}|\cdot|\sqrt{\nu_{i}(x)}+\sqrt{\nu_{i}(x^{\prime})}|$
$\displaystyle\leq$
$\displaystyle|\beta(t)|_{\infty}|x-x^{\prime}|_{1}\sum_{i=1}^{l}\big{(}\sqrt{\nu_{i}(x)}+\sqrt{\nu_{i}(x^{\prime})}\big{)}.$

Since each component of $\beta(t)$ is lower bounded by $\beta_{min}$ and
therefore
$\langle\beta(t),\nu(x)\rangle^{1/2}\geq\beta_{min}\sqrt{\nu_{i}(x)}$ for each
$i$, we conclude that

$\frac{|\langle\beta(t),\nu(x)-\nu(x^{\prime})\rangle|}{\langle\beta(t),\nu(x)\rangle^{1/2}(\langle\beta(t),\nu(x)\rangle^{1/2}+\langle\beta(t),\nu(x^{\prime})\rangle^{1/2})}\leq
C|x-x^{\prime}|_{1},$

with some constant $C>0$. Finally, let $e_{j}$ be the $j$-th unit vector in
$\mathbb{R}^{k}$. Notice that for all $i$,

$\frac{|\partial_{x_{j}}\nu_{i}(x)|}{\nu_{i}(x)^{1/2}}=2|\partial_{x_{j}}\sqrt{\nu_{i}(x)}|\leq
2\lim_{h\to
0}\frac{|\sqrt{\nu_{i}(x)}-\sqrt{\nu_{i}(x+he_{j})}|}{|h|}\leq\lim_{h\to
0}\frac{|he_{j}|_{1}}{|h|}\leq 1.$

This shows that
$\frac{|\langle\beta(t),\partial_{x_{j}}\nu(x^{\prime})\rangle|}{\langle\beta(t),\nu(x^{\prime})\rangle^{1/2}}$
is bounded and we obtain that for some constant $>0$,

$|\partial_{x_{j}}G_{\zeta_{0}}(x,t)-\partial_{x_{j}}G_{\zeta_{0}}(x^{\prime},t)|\leq
C(1+|\zeta_{0}|)|x-x^{\prime}|_{1}.$

With similar but simpler arguments we obtain that

$|\partial_{t}G_{\zeta_{0}}(x,t)-\partial_{t}G_{\zeta_{0}}(x^{\prime},t)|\leq
C(1+|\zeta_{0}|)|x-x^{\prime}|_{1}$

and

$\frac{|\partial_{x_{j}}G_{\zeta_{0}}(x,t)-\partial_{x_{j}}G_{\zeta_{0}}(x,t^{\prime})|}{|t-t^{\prime}|\cdot|x|_{1}}\leq
C(1+|\zeta_{0}|),\quad\quad\frac{|\partial_{t}G_{\zeta_{0}}(x,t)-\partial_{t}G_{\zeta_{0}}(x,t^{\prime})|}{|t-t^{\prime}|\cdot|x|_{1}}\leq
C(1+|\zeta_{0}|).$

By Theorem 4.8 and Proposition 4.11 in [13], we obtain 7.2(B3) with
$M_{2}=2M$.
Finally, straightforward calculations show that each component of
$\partial_{x_{j}}\nabla_{\theta}^{2}\ell$ and $\nabla_{\theta}^{3}\ell$ is in
$\mathcal{H}(M_{2},M_{2},\tilde{\chi},\bar{C})$ with
$\tilde{\chi}=(1,\ldots,1,0,\ldots,0)$ consisting of $p$ ones. This shows
Assumption 7.2(B2).

###  9.2 Case 2: tvGARCH

######  Proposition 9.3.

Let Assumption 2.3 hold. Then Assumption 7.3, 7.4 is fulfilled with every
$r=2+\tilde{a}$, $\tilde{a}<\frac{a}{2}$, $M=2$ and $\gamma>2$ arbitrarily
large. It holds that $\Lambda(t)=I(t)=((\mathbb{E}\zeta_{0}^{4}-1)/2)V(t)$.

######  Proof 9.4 (Proof of Proposition 9.3).

We abbreviate $M_{i}(t):=M_{i}(\theta(t))$. Let $M=1$. Fix $t\in[0,1]$.
Consider the recursion of the corresponding stationary approximation

$\displaystyle\tilde{Y}_{i}(t)$ $\displaystyle=$
$\displaystyle\tilde{\sigma}_{i}(t)^{2}\zeta_{i}^{2},$
$\displaystyle\tilde{\sigma}_{i}(t)^{2}$ $\displaystyle=$
$\displaystyle\alpha_{0}(t)+\sum_{j=1}^{m}\alpha_{j}(t)\tilde{Y}_{i-j}(t)+\sum_{j=1}^{l}\beta_{j}(t)\tilde{\sigma}_{i-j}(t)^{2}.$
(9.13)

Define

$\displaystyle\tilde{P}_{i}(t)$ $\displaystyle:=$
$\displaystyle(\tilde{Y}_{i}(t),\ldots,\tilde{Y}_{i-m+1}(t),\tilde{\sigma}_{i}(t)^{2},\ldots,\tilde{\sigma}_{i-l+1}(t)^{2})^{\mkern-1.5mu\mathsf{T}},$
$\displaystyle a_{i}(t)$ $\displaystyle:=$
$\displaystyle(\alpha_{0}(t)\zeta_{i}^{2},0,\ldots,0,\alpha_{0}(t),0,\ldots,0)^{\mkern-1.5mu\mathsf{T}}.$

For brevity, let $M_{i}(t)=M_{i}(\theta(t))$. Following Section 3.1 in [51],
the model (9.13) admits the representation

$\tilde{P}_{i}(t)=M_{i}(t)\tilde{P}_{i-1}(t)+a_{i}(t).$ (9.14)

Therefore, $\tilde{P}_{i}(t)=G_{\zeta_{i}}(\tilde{P}_{i-1}(t),t)$ with
$G_{\zeta_{i}}(y,t)=M_{i}(t)\cdot y+a_{i}(t)$. Let
$W_{n}(y,t):=G_{\zeta_{n}}(G_{\zeta_{n-1}}(...G_{\zeta_{1}}(y,t)...))$. Then
we have

$W_{n}(y,t)-W_{n}(y^{\prime},t)=M_{n}(t)M_{n-1}(t)\cdot...\cdot
M_{1}(t)\cdot(y-y^{\prime}).$

Note that $M_{i}(t)$ are i.i.d. matrices and for $i\geq 1$ they are also
independent of $y$. Thus

$\mathbb{E}((W_{n}(y,t)-W_{n}(y^{\prime},t))^{\otimes
2})=\mathbb{E}(M_{0}(t)^{\otimes 2})^{n}\mathbb{E}((y-y^{\prime})^{\otimes
2}).$

From the assumption $\rho(\mathbb{E}[M_{0}(\theta)^{\otimes 2}])<1$ and
Theorem 2 in [52], we obtain existence and a.s. uniqueness of
$\tilde{Y}_{i}(t)=H(t,\mathcal{F}_{i})$,
$\sup_{t\in[0,1]}\|\tilde{Y}_{0}(t)\|_{q}<\infty$ and
$\sup_{t\in[0,1]}\delta_{q}^{\tilde{Y}(t)}(k)=\|\tilde{Y}_{i}(t)-\tilde{Y}_{i}(t)^{*}\|_{q}=O(c^{k})$
for some $0<c<1$ and $q=2+\tilde{a},\tilde{a}<a/2$. We can choose $q$ to be
slightly higher than 2 using continuity on moment index. This shows Assumption
7.3(A7’).

(9.14) implies the explicit representation

$\tilde{P}_{i}(t)=\sum_{k=0}^{\infty}\Big{(}\prod_{j=0}^{k-1}M_{i-j}(t)\Big{)}a_{i-k}(t).$
(9.15)

We therefore have for $t,t^{\prime}\in[0,1]$:

$\displaystyle\|\tilde{P}_{i}(t)-\tilde{P}_{i}(t^{\prime})\|_{q}$
$\displaystyle\leq$
$\displaystyle\sum_{k=0}^{\infty}\sum_{l=0}^{k-1}\|\Big{(}\prod_{0\leq
j<l}M_{i-j}(t)\Big{)}\|_{q}\|M_{i-l}(t)-M_{i-l}(t^{\prime})\|_{q}$ (9.16)
$\displaystyle\quad\quad\quad\quad\quad\times\|\Big{(}\prod_{l<j\leq
k-1}M_{i-j}(t)\Big{)}\|_{q}\cdot\|a_{i-k}(t)\|_{q}$
$\displaystyle\quad+\sum_{k=0}^{\infty}\|\Big{(}\prod_{j=0}^{k-1}M_{i-j}(t)\Big{)}\|_{q}\|a_{i-k}(t)-a_{i-k}(t^{\prime})\|_{q}.$

By Lipschitz continuity of $\theta(\cdot)$ we have

$\|a_{0}(t)-a_{0}(t^{\prime})\|_{q}=|\alpha_{0}(t)-\alpha_{0}(t^{\prime})|(\|\zeta_{0}^{2}\|_{q},0,\ldots,0,1,0,\ldots,0)^{\mkern-1.5mu\mathsf{T}}=O(|t-t^{\prime}|)$

and

$\|M_{0}(t)-M_{0}(t^{\prime})\|_{q}=(\|\zeta_{0}^{2}\|_{q}|f(\theta(t))-f(\theta(t^{\prime}))|,0,\ldots,0,|f(\theta(t))-f(\theta(t^{\prime}))|,0,\ldots,0)^{\mkern-1.5mu\mathsf{T}}=O(|t-t^{\prime}|).$

Finally for the products of $M_{i}$ matrices again note that they are
independent and thus we can apply the same kronecker product technique along
with the assumption $\rho(\mathbb{E}[M_{0}(\theta)^{\otimes 2}])<1$ as above
and use continuity on the moment to deduce $\|\Big{(}\prod_{a\leq
j<b}M_{i-j}(t)\Big{)}\|_{q}\leq c^{b-a+1}$ for some $0<c<1$. We conclude from
the first component of (9.16), that for all $t,t^{\prime}\in[0,1]$:

$\|\tilde{Y}_{i}(t)-\tilde{Y}_{i}(t^{\prime})\|_{q}\leq C\cdot|t-t^{\prime}|,$
(9.17)

with some constant $C>0$.

Put
$P_{i}=(Y_{i},\ldots,Y_{i-m+1},\sigma_{i}^{2},\ldots,\sigma_{i-l+1}^{2})^{\mkern-1.5mu\mathsf{T}}$.
Similarly to (9.14), we have

$P_{i}=M_{i}(i/n)P_{i-1}+a_{i}(i/n),\quad i=1,\ldots,n.$ (9.18)

Note that $i$ iterations of (9.18) lead to $P_{0}=\tilde{P}_{0}(0)$, thus
existence of $Y_{i}$ follows from existence of $\tilde{Y}_{i}(0)$. We have

$\displaystyle\|P_{i}-\tilde{P}_{i}(i/n)\|_{q}$ $\displaystyle\leq$
$\displaystyle\|M_{i}(i/n)\|_{q}\|P_{i-1}-\tilde{P}_{i-1}(i/n)\|_{q}$
$\displaystyle\leq$
$\displaystyle\|M_{0}(i/n)\|_{q}\|P_{i-1}-\tilde{P}_{i-1}((i-1)/n)\|_{q}$
$\displaystyle\quad\quad\quad+\|M_{0}(i/n)\|_{q}\|\tilde{P}_{0}(i/n)-\tilde{P}_{0}((i-1)/n)\|_{q}.$

Iteration of this inequality leads to

$\|P_{i}-\tilde{P}_{i}(i/n)\|_{q}\leq\sum_{k=1}^{i}\|\Big{(}\prod_{j=0}^{k}M_{0}((i-j)/n)\Big{)}\|_{q}\|\tilde{P}_{0}((i-k)/n)-\tilde{P}_{0}((i-k-1)/n)\|_{q}.$

For the product of $M_{i}$ matrices the above technique is used. With that and
(9.17), we conclude from the first component that
$\|Y_{i}-\tilde{Y}_{i}(i/n)\|_{q}=O(n^{-1})$. This shows Assumption 7.3(A5’).

Let
$\Sigma(x,\theta):=(\sigma(x,\theta)^{2},\ldots,\sigma(x_{(l-1)\rightarrow},\theta)^{2})^{\mkern-1.5mu\mathsf{T}}$
and
$A(x,\theta):=(\alpha_{0}+\sum_{j=1}^{m}\alpha_{j}x_{j},\ldots,\alpha_{0}+\sum_{j=1}^{m}\alpha_{j}x_{j+l-1})^{\mkern-1.5mu\mathsf{T}}$,
and

$B(\theta)=\begin{pmatrix}\beta_{1}&\dots&\dots&\dots&\beta_{l}\\\
1&0&\dots&\dots&0\\\ 0&\ddots&\ddots&&\vdots\\\
\vdots&\ddots&\ddots&\ddots&0\\\ 0&\dots&0&1&0\end{pmatrix}.$

As said in Theorem 2.1 in [36], $\rho(\mathbb{E}M_{0}(\theta)^{\otimes 2})<1$
is a necessary and sufficient condition for the corresponding GARCH process
with parameters $\theta$ to have 4-th moments. We conclude that
$\rho(\mathbb{E}M_{0}(\theta))<1$ which by Proposition 1 in [19] implies
$\rho(B(\theta))<1$. We have the explicit representation

$\sigma(x,\theta)^{2}=\sum_{k=0}^{\infty}\big{(}B(\theta)^{k}A(x_{k\rightarrow},\theta)\big{)}_{1}.$
(9.19)

Since $A(0,\theta)=(\alpha_{0},0,...,0)^{\mkern-1.5mu\mathsf{T}}$, we have

$\sigma(0,\theta)^{2}=\alpha_{0}\sum_{k=0}^{\infty}(B(\theta)^{k})_{11}.$

From (9.19) we also obtain that

$\sigma(x,\theta)^{2}=c_{0}(\theta)+\sum_{j=1}^{\infty}c_{j}(\theta)\cdot
x_{j},$ (9.20)

where $c_{j}(\theta)\geq 0$ satisfies

$\sup_{\theta\in\Theta}|c_{j}(\theta)|\leq C\cdot\rho^{j}$ (9.21)

with some $\rho\in(0,1)$ and $c_{0}(\theta)\geq\sigma_{min}^{2}>0$ (due to
$\alpha_{0}\geq\alpha_{min}>0$). Due to the explicit representation (9.19)
with geometrically decaying summands, it is easy to see that
$\sigma(x,\theta)^{2}$ is four times continuously differentiable w.r.t.
$\theta$ with

$\nabla_{\theta}^{k}(\sigma(x,\theta)^{2})=\nabla_{\theta}^{k}c_{0}(\theta)+\sum_{j=1}^{\infty}\nabla_{\theta}^{k}c_{j}(\theta)\cdot
x_{j},\quad\quad k\in\\{0,1,2,3,4\\},$ (9.22)

where $(\nabla_{\theta}^{k}c_{j}(\theta))_{j}$ is still geometrically decaying
with $\sup_{\theta\in\Theta}|\nabla_{\theta}^{k}c_{j}(\theta)|_{\infty}\leq
C\cdot\rho^{j}$, say.

From (9.22) we conclude that (component-wise) for $k=0,1,2,3$:

$\displaystyle|\nabla_{\theta}^{k}(\sigma(x,\theta)^{2})-\nabla_{\theta}^{k}(\sigma(x^{\prime},\theta)^{2})|$
$\displaystyle\leq$ $\displaystyle C|x-x^{\prime}|_{(\rho^{j})_{j},1},$
$\displaystyle|\nabla_{\theta}^{k}(\sigma(x,\theta)^{2})-\nabla_{\theta}^{k}(\sigma(x,\theta^{\prime})^{2})|$
$\displaystyle\leq$
$\displaystyle|\theta-\theta^{\prime}|_{1}\cdot\sup_{\theta\in\Theta}|\nabla_{\theta}^{k+1}(\sigma(x,\theta)^{2})|_{\infty}\leq
C|\theta-\theta^{\prime}|_{1}\cdot|x|_{(\rho^{j})_{j},1}.$

We obtain that $\ell(y,x,\theta)$ is four times continuously differentiable
and

$\displaystyle\ell(y,x,\theta)$ $\displaystyle=$
$\displaystyle\frac{1}{2}\Big{(}\frac{y}{\sigma(x,\theta)^{2}}+\log(\sigma(x,\theta)^{2})\Big{)},$
$\displaystyle\nabla_{\theta}\ell(y,x,\theta)$ $\displaystyle=$
$\displaystyle\frac{\nabla_{\theta}(\sigma(x,\theta)^{2})}{2\sigma(x,\theta)^{2}}\Big{(}1-\frac{y}{\sigma(x,\theta)^{2}}\Big{)},$
$\displaystyle\nabla_{\theta}^{2}\ell(y,x,\theta)$ $\displaystyle=$
$\displaystyle\Big{[}-\frac{\nabla_{\theta}(\sigma(x,\theta)^{2})\nabla_{\theta}(\sigma(x,\theta)^{2})^{\mkern-1.5mu\mathsf{T}}}{2\sigma(x,\theta)^{4}}+\frac{\nabla_{\theta}^{2}(\sigma(x,\theta)^{2})}{2\sigma(x,\theta)^{2}}\Big{]}\Big{(}1-\frac{y}{\sigma(x,\theta)^{2}}\Big{)}$
$\displaystyle\quad\quad\quad+\frac{\nabla_{\theta}(\sigma(x,\theta)^{2})\nabla_{\theta}(\sigma(x,\theta)^{2})^{\mkern-1.5mu\mathsf{T}}}{2\sigma(x,\theta)^{4}}\cdot\frac{y}{\sigma(x,\theta)^{2}}.$

It was shown in the proof of Theorem 2.1 in [19], that $\theta\mapsto
L(t,\theta)=\mathbb{E}\ell(\tilde{Z}_{0}(t),\theta)$ is uniquely minimized in
$\theta=\theta(t)$, which shows Assumption 7.3(A3’). As in the proof of
example Proposition 9.1, we obtain that

$\displaystyle
V(t)=\mathbb{E}\big{[}\frac{\nabla_{\theta}(\sigma(\tilde{X}_{0}(t),\theta(t))^{2})\nabla_{\theta}(\sigma(\tilde{X}_{0}(t),\theta(t))^{2})^{\mkern-1.5mu\mathsf{T}}}{2\sigma(\tilde{X}_{0}(t),\theta(t))^{4}}\big{]}=I(t)\frac{2}{\mathbb{E}\zeta_{0}^{4}-1}.$

Furthermore,

$\nabla_{\theta}\ell(\tilde{Z}_{i}(t),\theta(t))=\frac{\nabla_{\theta}(\sigma(\tilde{X}_{i}(t),\theta(t))^{2})}{2\sigma(\tilde{X}_{i}(t),\theta(t))^{2}}\\{1-\zeta_{i}^{2}\\},$

which shows that $\nabla_{\theta}\ell(\tilde{Z}_{i}(t),\theta(t))$ is a
martingale difference sequence w.r.t. $\mathcal{F}_{i}$. Thus
$\Lambda(t)=I(t)$. It was shown in the proof of Theorem 2.2 in [19] that
$V(t)$ is positive definite for each $t\in[0,1]$. By continuity, we conclude
that Assumption 7.3(A4’) is fulfilled.

Proof of Assumption 7.3(A1’): It holds that

$2|\ell(y,x,\theta)-\ell(y^{\prime},x^{\prime},\theta)|\leq|y-y^{\prime}|\cdot\frac{1}{\sigma(x,\theta)^{2}}+|y^{\prime}|\cdot\Big{|}\frac{1}{\sigma(x,\theta)^{2}}-\frac{1}{\sigma(x^{\prime},\theta)^{2}}\Big{|}+|\log(\sigma(x,\theta)^{2})-\log(\sigma(x^{\prime},\theta)^{2})|.$

Since $\sigma(x,\theta)^{2}\geq\sigma_{min}^{2}>0$, Lipschitz continuity of
$\log$ on $[\sigma_{min},\infty)$ and (9.21), there exists some constant
$C^{\prime}>0$ such that

$2|\ell(y,x,\theta)-\ell(y^{\prime},x^{\prime},\theta)|\leq
C^{\prime}(|y-y^{\prime}|+|x-x^{\prime}|_{(\rho^{j})_{j},1})+|y^{\prime}|\cdot\Big{|}\frac{1}{\sigma(x,\theta)^{2}}-\frac{1}{\sigma(x^{\prime},\theta)^{2}}\Big{|}.$
(9.24)

Note that

$\displaystyle\Big{|}\frac{1}{\sigma(x,\theta)^{2}}-\frac{1}{\sigma(x^{\prime},\theta)^{2}}\Big{|}\leq\frac{\sum_{j=0}^{\infty}c_{j}(\theta)|x_{j}.x_{j}^{\prime}|}{\sigma(x,\theta)^{2}\sigma(x^{\prime},\theta)^{2}}\leq\sum_{j=1}^{\infty}\frac{c_{j}(\theta)|x_{j}-x_{j}^{\prime}|}{(\sigma_{min}^{2}+c_{j}(\theta)x_{j})(\sigma_{min}^{2}+c_{j}(\theta)x_{j}^{\prime})}$
$\displaystyle\leq$
$\displaystyle\frac{1}{\sigma_{min}^{2}}\sum_{j=1}^{\infty}\frac{c_{j}(\theta)|x_{j}-x_{j}^{\prime}|}{\sigma_{min}^{2}+c_{j}(\theta)|x_{j}-x_{j}^{\prime}|.}$

The last step holds due to the following argument: It holds either
$|x_{j}-x_{j}^{\prime}|\leq x_{j}$ or $|x_{j}-x_{j}^{\prime}|\leq
x_{j}^{\prime}$ since $x_{j},x_{j}^{\prime}\geq 0$. Therefore, one factor in
the denominator can be lower bounded by $\sigma_{min}$ and the other one by
$\sigma_{min}+c_{j}(\theta)|x_{j}-x_{j}^{\prime}|$. Following the ideas of
[19], for arbitrarily small $s>0$ we use the inequality $\frac{x}{1+x}\leq
x^{s}$ to obtain

$\displaystyle\Big{|}\frac{1}{\sigma(x,\theta)^{2}}-\frac{1}{\sigma(x^{\prime},\theta)^{2}}\Big{|}\leq\frac{1}{\sigma_{min}^{4+2s}}\sum_{j=1}^{\infty}c_{j}(\theta)^{s}|x_{j}-x_{j}^{\prime}|^{s}\leq\frac{C}{\sigma_{min}^{4+2s}}|x-x|_{(\rho^{js})_{j},s}$

Together with (9.24), we obtain (7.6).

Using directly (9.24) and (9.4), we obtain

$\sup_{\theta\in\Theta}\sup_{z\not=z^{\prime}}\frac{|\ell(z,\theta)-\ell(z^{\prime},\theta)|}{|z-z^{\prime}|_{(\rho^{j})_{j},1}\cdot(1+R_{2M-1,2M-1}(z)+R_{2M-1,2M-1}(z^{\prime}))}<\infty.$

Note that with some constant $C^{\prime}>0$,

$\displaystyle 2|\ell(z,\theta)-\ell(z,\theta^{\prime})|$ $\displaystyle\leq$
$\displaystyle|y|\cdot\Big{[}\frac{1}{\sigma(x,\theta)^{2}}-\frac{1}{\sigma(x,\theta^{\prime})^{2}}\Big{]}+|\log(\sigma(x,\theta)^{2})-\log(\sigma(x,\theta^{\prime})^{2})|$
$\displaystyle\leq$ $\displaystyle
C^{\prime}(1+|y|)\cdot|\sigma(x,\theta)^{2}-\sigma(x,\theta^{\prime})^{2}|.$

Together with (LABEL:eq:sigmagarch_thetadiff), we obtain

$\sup_{\theta\not=\theta^{\prime}}\sup_{z}\frac{|\ell(z,\theta)-\ell(z,\theta^{\prime})|}{|\theta-\theta^{\prime}|_{1}\cdot(1+R_{2M,2M}(z)+R_{2M,2M}(z^{\prime}))}<\infty.$

This shows $\ell\in\mathcal{H}_{0}(2M,2M,(\rho^{j})_{j},\bar{C})$ with some
suitably chosen $\bar{C}>0$.

Let $s>0$ be arbitrary. It was shown in [19], (4.25) therein that with some
small $\iota>0$ only depending on $s,\Theta$, it holds that

$\sup_{|\tilde{\theta}-\theta|<\iota}\frac{\sigma(x,\tilde{\theta})^{2}}{\sigma(x,\theta)^{2}}\leq\bar{C}(1+|x|_{(\rho^{js})_{j},s}^{s}).$
(9.25)

Similarly, one can obtain for $k=1,2,3$ that

$\sup_{|\tilde{\theta}-\theta|<\iota}\frac{\nabla_{\theta}^{k}\sigma(x,\tilde{\theta})^{2}}{\sigma(x,\theta)^{2}}\leq\bar{C}(1+|x|_{(\rho^{js})_{j},s}^{s}).$
(9.26)

In the following we show that
$\nabla_{\theta}\ell\in\mathcal{H}_{s}(2M,2M,(\rho^{j})_{j},\bar{C})$ with
some suitably chosen $\bar{C}>0$. We have (component-wise):

$\displaystyle
2|\nabla_{\theta}\ell(y,x,\theta)-\nabla_{\theta}\ell(y^{\prime},x^{\prime},\theta)|$
$\displaystyle\leq$
$\displaystyle|y-y^{\prime}|\cdot\frac{1}{\sigma_{min}^{2}}\frac{|\nabla_{\theta}(\sigma(x,\theta)^{2})|}{\sigma(x,\theta)^{2}}+|y^{\prime}|\cdot\Big{|}\frac{1}{\sigma(x,\theta)^{2}}-\frac{1}{\sigma(x^{\prime},\theta)}\Big{|}\cdot\frac{|\nabla_{\theta}(\sigma(x,\theta)^{2})|}{\sigma(x,\theta)^{2}}$
$\displaystyle\quad+\Big{(}1+\frac{|y^{\prime}|}{\sigma_{min}^{2}}\Big{)}\cdot\Big{(}\frac{|\nabla_{\theta}(\sigma(x,\theta)^{2})-\nabla_{\theta}(\sigma(x^{\prime},\theta)^{2})|}{\sigma_{min}^{2}}+\frac{|\nabla_{\theta}(\sigma(x^{\prime},\theta)^{2})|}{\sigma(x^{\prime},\theta)^{2}\sigma_{min}^{2}}|\sigma(x,\theta)^{2}-\sigma(x,\theta^{\prime})^{2}|\Big{)}.$

Using (9.4) and (9.26), we obtain (component-wise) with some suitably chosen
$\bar{C}>0$:

$2|\nabla_{\theta}\ell(y,x,\theta)-\nabla_{\theta}\ell(y^{\prime},x^{\prime},\theta)|\leq\bar{C}|z-z^{\prime}|_{(\rho^{j})_{j},1}\cdot(1+R_{2M-1,2M-1}(z)+R_{2M-1,2M-1}(z^{\prime}))^{1+s}.$
(9.27)

We have (component-wise):

$\displaystyle
2|\nabla_{\theta}\ell(z,\theta)-\nabla_{\theta}\ell(z,\theta^{\prime})|$
$\displaystyle\leq$
$\displaystyle|y|\cdot\Big{|}\frac{1}{\sigma(x,\theta)^{2}}-\frac{1}{\sigma(x,\theta^{\prime})}\Big{|}\cdot\frac{|\nabla_{\theta}(\sigma(x,\theta)^{2})|}{\sigma(x,\theta)^{2}}$
$\displaystyle\quad+\Big{(}1+\frac{|y|}{\sigma_{min}^{2}}\Big{)}\cdot\Big{(}\frac{|\nabla_{\theta}(\sigma(x,\theta)^{2})-\nabla_{\theta}(\sigma(x,\theta^{\prime})^{2})|}{\sigma_{min}^{2}}+\frac{|\nabla_{\theta}(\sigma(x,\theta^{\prime})^{2})|}{\sigma(x,\theta^{\prime})^{2}\sigma_{min}^{2}}|\sigma(x,\theta)^{2}-\sigma(x,\theta^{\prime})^{2}|\Big{)}.$

Using (9.4) and (9.26), we obtain (component-wise) with some suitably chosen
$\bar{C}>0$:

$2|\nabla_{\theta}\ell(z,\theta)-\nabla_{\theta}\ell(z,\theta^{\prime})|\leq\bar{C}|\theta-\theta^{\prime}|_{1}\cdot(1+R_{2M,2M}(z)+R_{2M,2M}(z^{\prime}))^{1+s}.$
(9.28)

We conclude from (9.27) and (9.28) that
$\nabla_{\theta}\ell\in\mathcal{H}_{s}(2M,2M,(\rho^{j})_{j},\bar{C})$. The
proof for $\nabla_{\theta}^{2}\ell$ is similar in view of (9.4),
(LABEL:eq:sigmagarch_thetadiff) and (9.26) and therefore omitted.

Let $s>0$ be arbitrary and $\iota>0$ such that (9.25) and (9.26) hold. In the
following we show that
$\nabla_{\theta}\tilde{\ell}\in\mathcal{H}_{s,\iota}^{mult}(M,M,(\rho^{j})_{j},\bar{C})$
with some suitable chosen $\bar{C}>0$. It holds that

$\nabla_{\theta}\tilde{\ell}_{\tilde{\theta}}(y,x,\theta)=\frac{\nabla_{\theta}(\sigma(x,\theta)^{2})}{2\sigma(x,\theta)^{2}}\Big{(}1-y\frac{\sigma(x,\tilde{\theta})^{2}}{\sigma(x,\theta)^{2}}\Big{)}.$

We have for $|\theta-\tilde{\theta}|_{1}<\iota$:

$\displaystyle
2|\nabla_{\theta}\tilde{\ell}_{\tilde{\theta}}(y,x,\theta)-\nabla_{\theta}\tilde{\ell}_{\tilde{\theta}}(y,x^{\prime},\theta)|$
$\displaystyle\leq$
$\displaystyle|y|\cdot\Big{[}\frac{|\sigma(x,\tilde{\theta})^{2}-\sigma(x^{\prime},\tilde{\theta})^{2}|}{\sigma_{min}^{2}}+\frac{\sigma(x^{\prime},\tilde{\theta})^{2}}{\sigma(x^{\prime},\theta)^{2}\sigma_{min}^{2}}|\sigma(x,\theta)^{2}-\sigma(x^{\prime},\theta)^{2}|\Big{]}\cdot\frac{|\nabla_{\theta}(\sigma(x,\theta)^{2})|}{\sigma(x,\theta)^{2}}$
$\displaystyle\quad+\Big{(}1+|y|\cdot\frac{\sigma(x^{\prime},\tilde{\theta})^{2}}{\sigma(x^{\prime},\theta)^{2}}\Big{)}\cdot\Big{(}\frac{|\nabla_{\theta}(\sigma(x,\theta)^{2})-\nabla_{\theta}(\sigma(x^{\prime},\theta)^{2})|}{\sigma_{min}^{2}}+\frac{|\nabla_{\theta}(\sigma(x^{\prime},\theta)^{2})|}{\sigma(x^{\prime},\theta)^{2}\sigma_{min}^{2}}|\sigma(x,\theta)^{2}-\sigma(x,\theta^{\prime})^{2}|\Big{)}.$

Using (9.4) and (9.26), we obtain (component-wise) with some suitably chosen
$\bar{C}>0$:

$2|\nabla_{\theta}\tilde{\ell}_{\tilde{\theta}}(y,x,\theta)-\nabla_{\theta}\tilde{\ell}_{\tilde{\theta}}(y,x^{\prime},\theta)|\leq\bar{C}(1+|y|)\cdot|x-x^{\prime}|_{(\rho^{j})_{j},1}\cdot|x|_{(\rho^{js})_{j},s}^{s}.$
(9.29)

We have for
$|\theta-\tilde{\theta}|_{1},|\theta^{\prime}-\tilde{\theta}|_{1}<\iota$:

$\displaystyle
2|\nabla_{\theta}\tilde{\ell}_{\tilde{\theta}}(y,x,\theta)-\nabla_{\theta}\tilde{\ell}_{\tilde{\theta}}(y,x,\theta^{\prime})|$
$\displaystyle\leq$
$\displaystyle|y|\cdot\frac{\sigma(x,\tilde{\theta})^{2}}{\sigma(x,\theta)^{2}\sigma_{min}^{2}}|\sigma(x,\theta)^{2}-\sigma(x,\theta^{\prime})^{2}|\cdot\frac{|\nabla_{\theta}(\sigma(x,\theta)^{2})|}{\sigma(x,\theta)^{2}}$
$\displaystyle\quad+\Big{(}1+|y|\cdot\frac{\sigma(x,\tilde{\theta})^{2}}{\sigma(x,\theta^{\prime})^{2}}\Big{)}\cdot\Big{(}\frac{|\nabla_{\theta}(\sigma(x,\theta)^{2})-\nabla_{\theta}(\sigma(x,\theta^{\prime})^{2})|}{\sigma_{min}^{2}}+\frac{|\nabla_{\theta}(\sigma(x,\theta^{\prime})^{2})|}{\sigma(x,\theta^{\prime})^{2}\sigma_{min}^{2}}|\sigma(x,\theta)^{2}-\sigma(x,\theta^{\prime})^{2}|\Big{)}.$

Using (LABEL:eq:sigmagarch_thetadiff) and (9.26), we obtain (component-wise)
with some suitably chosen $\bar{C}>0$:

$2|\nabla_{\theta}\tilde{\ell}_{\tilde{\theta}}(y,x,\theta)-\nabla_{\theta}\tilde{\ell}_{\tilde{\theta}}(y,x,\theta^{\prime})|\leq\bar{C}(1+|y|)\cdot|\theta-\theta^{\prime}|_{1}\cdot
R_{M,M}(1,x)\cdot|x|_{(\rho^{js})_{j},s}^{s}.$ (9.30)

We conclude from (9.29) and (9.30) that
$\nabla_{\theta}\tilde{\ell}\in\mathcal{H}_{s,\iota}^{mult}(M,M,(\rho^{j})_{j},\bar{C})$.
The proof for $\nabla_{\theta}^{2}\tilde{\ell}$ is similar in view of (9.4),
(LABEL:eq:sigmagarch_thetadiff) and (9.26) and therefore omitted.

Regarding Assumption 7.2, notice that from the explicit representation
uniformly in $t$ we have that

$\displaystyle\partial_{t}\tilde{P}_{i}(t)$ $\displaystyle=$
$\displaystyle\sum_{k=0}^{\infty}\sum_{l=0}^{k-1}\Big{(}\prod_{0\leq
j<l}M_{i-j}(t)\Big{)}\partial_{t}M_{i-l}(t)\Big{(}\prod_{l<j\leq
k-1}M_{i-j}(t)\Big{)}a_{i-k}(t)$
$\displaystyle\quad\quad\quad+\sum_{k=0}^{\infty}\Big{(}\prod_{j=0}^{k-1}M_{i-j}(t)\Big{)}\partial_{t}a_{i-k}(t)$

exists a.s. and has $q$-th moments, so does its first component
$\partial_{t}\tilde{Y}_{t}(t)$. Similar arguments that were used to prove
(9.17) can be applied here and yield for $t,t^{\prime}\in[0,1]$:

$\|\partial_{t}\tilde{Y}_{i}(t)-\partial_{t}\tilde{Y}_{i}(t^{\prime})\|_{q}\leq
C^{\prime}\cdot|t-t^{\prime}|,$

with some constant $C^{\prime}>0$, i.e. Assumption 7.4(B3’) is shown.

From (9.20) and $\sup_{\theta\in\Theta}\rho(B(\theta))<1$, it follows that
$x_{i}\mapsto\sigma(x,\theta)^{2}$ is differentiable for all $i\in\mathbb{N}$
and

$\partial_{x_{i}}\nabla_{\theta}^{k}(\sigma(x,\theta)^{2})=\nabla_{\theta}^{k}c_{i}(\theta),\quad
k=0,1,2.$

Let $M^{\prime}=1$. Similar as above it can be seen that
$\nabla_{\theta}^{3}\tilde{\ell}\in\mathcal{H}_{s,\iota}^{mult}(M^{\prime},M^{\prime},(\rho^{j})_{j},\bar{C})$.
Note that

$\displaystyle\partial_{x_{i}}\nabla_{\theta}^{2}\tilde{\ell}_{\tilde{\theta}}(y,x,\theta)$
$\displaystyle=$
$\displaystyle\Big{[}-\frac{\nabla_{\theta}c_{i}(\theta)}{\sigma(x,\theta)^{2}}\frac{\nabla_{\theta}(\sigma(x,\theta)^{2})^{\mkern-1.5mu\mathsf{T}}}{2\sigma(x,\theta)^{2}}-\frac{\nabla_{\theta}(2\sigma(x,\theta)^{2})}{\sigma(x,\theta)^{2}}\cdot\frac{\nabla_{\theta}c_{i}(\theta)^{\mkern-1.5mu\mathsf{T}}}{\sigma(x,\theta)^{2}}$
$\displaystyle\quad\quad+\frac{c_{i}(\theta)}{\sigma(x,\theta)^{2}}\frac{\nabla_{\theta}(\sigma(x,\theta)^{2})\nabla_{\theta}(\sigma(x,\theta)^{2})^{\mkern-1.5mu\mathsf{T}}}{\sigma(x,\theta)^{4}}$
$\displaystyle\quad\quad+\frac{\nabla_{\theta}^{2}c_{i}(\theta)}{2\sigma(x,\theta)^{2}}-\frac{c_{i}(\theta)}{\sigma(x,\theta)^{2}}\frac{\nabla_{\theta}^{2}(\sigma(x,\theta)^{2})}{2\sigma(x,\theta)^{2}}\Big{]}\cdot(1-y\frac{\sigma(x,\tilde{\theta})^{2}}{\sigma(x,\theta)^{2}}\Big{)}$
$\displaystyle\quad+\Big{[}-\frac{\nabla_{\theta}(\sigma(x,\theta)^{2})\nabla_{\theta}(\sigma(x,\theta)^{2})^{\mkern-1.5mu\mathsf{T}}}{2\sigma(x,\theta)^{4}}+\frac{\nabla_{\theta}^{2}(\sigma(x,\theta)^{2})}{2\sigma(x,\theta)^{2}}\Big{]}\cdot\frac{c_{i}(\theta)}{\sigma(x,\theta)^{2}}\frac{\sigma(x,\tilde{\theta})^{2}}{\sigma(x,\theta)^{2}}\cdot
y$ $\displaystyle\quad\quad+\text{similar terms (derivative of second
summand)}$

Each summand contains a factor $\nabla_{\theta}^{k}c_{i}(\theta)$ which is
geometrically decaying by (9.26). Similar as above one can therefore see that
$\nabla_{\theta}^{2}\tilde{\ell}_{\tilde{\theta}}\in\mathcal{H}_{s,\iota}^{mult}(M^{\prime},M^{\prime},(\rho^{j})_{j},\bar{C}\rho^{i})$.
This shows Assumption 7.4(B2’).