$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2})\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\left(\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right)\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2})\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\left(\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right)^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2})\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{4}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}\stackrel{{\scriptstyle(c)}}{{<}}\infty.$

(a) Apply the Cauchy-Schwarz inequality, and note that the supremum of the
square of a positive function is the square of the supremum of the function.
(b) Firstly, under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11),
$\mathbf{S}$ contains an open ball centered at
$(\theta_{0},\tau(\theta_{0}))$, so that, for $\overline{B_{L}}$ of
sufficiently small radius,
$\overline{B_{L}}\subset\\{(\theta,\tau):\theta\in\mathcal{N}\wedge\tau\in\mathbf{T}(\theta)\\}\subset\mathbf{S}\subset\mathbf{S}^{\epsilon}$.
Secondly, as the second supremum does not depend on $\tau$,
$\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\leqslant\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}$
because $\overline{B_{L}}\subset\mathbf{S}$, for $\overline{B_{L}}$ of radius
small enough. (c) Firstly, because $\overline{B_{L}}$ is bounded,
$(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2})<\infty$. Secondly, by
Assumption 2(b), the first expectation is bounded. Thirdly, by Assumption
1(g), the second expectation is also bounded. ∎

######  Lemma 19 (Finiteness of the expectations of the supremum of the terms
from
$\frac{\partial^{2}L_{T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{j}}$).

Under Assumptions 1 and 2, there exists a closed ball $\overline{B_{L}}$
centered at $(\theta_{0},\tau(\theta_{0}))$ with strictly positive radius
s.t., for all $(k,j)\in[\\![1,m]\\!]^{2}$,

  1. (i)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\right]<\infty$;

  2. (ii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\psi_{k}(X_{1},\theta)|\right]<\infty$;

  3. (iii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi_{k}(X_{1},\theta)}{\partial\theta_{\ell}}|\right]<\infty$;

  4. (iv)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}|\right]<\infty$;

  5. (v)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)|\right]<\infty$;

  6. (vi)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$;

  7. (vii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$;

  8. (viii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right]<\infty$;

  9. (ix)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta^{\prime}}|\right]<\infty$;

  10. (x)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right]<\infty$;

  11. (xi)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta^{\prime}}|\right]<\infty$;

  12. (xii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\frac{\partial\psi_{k}(X_{1},\theta)}{\partial\theta_{j}}|\right]<\infty$;

  13. (xiii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$;

  14. (xiv)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$;

  15. (xv)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$;

  16. (xvi)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\psi(X_{1},\theta)^{\prime}|\right]<\infty$;

  17. (xvii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$;

  18. (xviii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\psi(X_{1},\theta)^{\prime}|\right]<\infty$;
and

  19. (xix)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi_{k}(X_{1},\theta)}{\partial\theta_{j}}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$.

###### Proof.

The proofs are similar to the ones of Lemma 18 (p. 18): We only use more often
the inequality that states that the norm of a component of a vector is smaller
than the norm of the vector (e.g.,
$|\psi_{k}(X_{1},\theta)|\leqslant\sqrt{\sum_{l=1}^{m}\psi_{l}(X_{1},\theta)^{2}}=|\psi(X_{1},\theta)|$).
Thus, we only provide proof sketches.

(i) See Lemma 18i p. 18.

(ii) For $\overline{B_{L}}$ of sufficiently small radius, for all
$(k,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\psi_{k}(X_{1},\theta)|\right]$
$\displaystyle\leqslant$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi_{k}(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi_{k}(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)^{\prime}|^{2}\right]}\stackrel{{\scriptstyle}}{{<}}\infty,$

where the last inequality follows from Assumption 2(b), and Lemma 9i (p. 9),
under Assumption 1(a)(b)(g).

(iii) For $\overline{B_{L}}$ of sufficiently small radius, for all
$(k,j)\in[\\![1,m]\\!]^{2}$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi_{k}(X_{1},\theta)}{\partial\theta_{j}}|\right]\leqslant\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi_{k}(X_{1},\theta)}{\partial\theta^{\prime}}|\right]\leqslant\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})\right]<\infty$,
where the last inequality follows from Assumption 2(b).

(iv) See Lemma 18iv p. 18.

(v) For $\overline{B_{L}}$ of sufficiently small radius, for all
$k\in[\\![1,m]\\!]$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)|\right]\leqslant\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)|\right]\leqslant\mathbb{E}\left[\sup_{(\theta,\tau)\in\mathbf{S}^{\epsilon}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)|\right]<\infty$
where the last inequality follows from Lemma 9ii (p. 9) under Assumption
1(a)(b)(e)(g).

(vi) See Lemma 18v p. 18.

(vii) For $\overline{B_{L}}$ of sufficiently small radius, for all
$k\in[\\![1,m]\\!]$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]$
$\displaystyle\leqslant$
$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi_{k}(X_{1},\theta)|\right]$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi_{k}(X_{1},\theta)|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)|^{2}\right]}\stackrel{{\scriptstyle}}{{<}}\infty,$

where the last inequality follows from Assumption 2(b) and Lemma 9i (p. 9)
under Assumption 1(a)(b)(g).

(viii) For all $j\in[\\![1,m]\\!]$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right]<(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\\\
\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]\leqslant(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]<\infty$
where the two last inequalities follow from Assumption 2(b) and the
boundedness of $\overline{B_{L}}$.

(ix) See Lemma 18vi p. 18.

(x) Under Assumption 1(a)(b), for $\overline{B_{L}}$ of sufficiently small
radius, for all $(j,k)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right]<\infty$
$\displaystyle\leqslant$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\left(\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right)\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi_{k}(X_{1},\theta)|\right]$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\left(\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right)^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi_{k}(X_{1},\theta)|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{4}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)|^{2}\right]}\stackrel{{\scriptstyle}}{{<}}\infty,$

where the last inequality follows from the boundedness of $\overline{B_{L}}$,
Assumption 2(b) and Lemma 9i (p. 9) under Assumption 1(a)(b)(g) and (e).

(xi) Under Assumption 1(a)(b), for $\overline{B_{L}}$ of sufficiently small
radius, for all $(j,k)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta^{\prime}}|\right]$
$\displaystyle\leqslant$
$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta^{\prime}}|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi_{k}(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta^{\prime}}|^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi_{k}(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)^{\prime}|^{2}\right]}\stackrel{{\scriptstyle}}{{<}}\infty,$

where the last inequality follows from Assumption 2(b) and Lemma 9 (p. 9),
under Assumption 1(a)(b)(g) and (e).

(xii) Under Assumption 1(a)-(e) and (g)-(h), for $\overline{B_{L}}$ of
sufficiently small radius, for all $(j,k)\in[\\![1,m]\\!]^{2}$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\frac{\partial\psi_{k}(X_{1},\theta)}{\partial\theta_{j}}|\right]<\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right]\leqslant\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]<\infty$
where the two last inequalities follow from Assumption 2(b).

(xiii) See Lemma 18xii p. 18.

(xiv) Under Assumption 1(a)-(e) and (g)-(h), for $\overline{B_{L}}$ of
sufficiently small radius, for all $k\in[\\![1,m]\\!]$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\leqslant$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2})\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\left(\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\psi_{k}(X_{1},\theta)|\right)\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2})\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\left(\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\psi_{k}(X_{1},\theta)|\right)^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2})\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}\stackrel{{\scriptstyle}}{{<}}\infty,$

where the last inequality follows from the boundedness of $\overline{B_{L}}$,
Assumptions 1(g) and 2(b).

(xv) See Lemma 18xiv p. 18.

(xvi) See Lemma 18xiii p. 18.

(xvii) Under Assumption 1(a)(b), for $\overline{B_{L}}$ of sufficiently small
radius, for all $(k,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\leqslant$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\left(\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\psi_{k}(X_{1},\theta)||\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right)\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\left(\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\psi_{k}(X_{1},\theta)||\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right)^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle\negthickspace(\negthickspace\negthickspace\sup_{(\theta,\tau)\in\overline{B_{L}}}\negthickspace\negthickspace\negthickspace|\tau|)\sqrt{\mathbb{E}\left[\negthickspace\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\negthickspace\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X)^{4}\right]}\negthickspace\sqrt{\mathbb{E}\left[\negthickspace\sup_{(\theta,\tau)\in\overline{B_{L}}}\negthickspace|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle}}{{<}}$ $\displaystyle\infty,$

where the last inequality follows from the boundedness of $\overline{B_{L}}$,
Assumption 2(b) and Assumption 1(g).

(xviii) Under Assumption 1(a)(b), for $\overline{B_{L}}$ of sufficiently small
radius, for all $(k,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\leqslant$
$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{4}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)|^{2}\right]}\stackrel{{\scriptstyle}}{{<}}\infty,$

where the last inequality follows from Assumption 2(b), and Lemma 9i (p. 9),
under Assumption 1(a)(b)(g).

(xix) Under Assumption 1(a)(b), for $\overline{B_{L}}$ of sufficiently small
radius, for all $(k,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi_{k}(X_{1},\theta)}{\partial\theta_{j}}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\leqslant$
$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi_{k}(X_{1},\theta)}{\partial\theta_{j}}|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi_{k}(X_{1},\theta)}{\partial\theta_{j}}|^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}\stackrel{{\scriptstyle}}{{<}}\infty,$

where the last inequality follows from Assumption 2(b) and Assumption 1(g). ∎

######  Lemma 20.

Under Assumptions 1 and 2, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$\frac{\partial
L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}=O(T^{-1})$.

###### Proof.

Unlike in most of the rest of the paper, for clarity, in this proof we do not
use the potentially ambiguous notation that denotes $\left.\frac{\partial
L_{T}(\theta,\tau)}{\partial\theta}\right|_{(\theta,\tau)=(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}$
with $\frac{\partial
L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}$.141414This
is a potentially ambiguous notation in the sense that $\frac{\partial
L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}$ could
also denote $\left.\frac{\partial
L_{T}(\theta,\tau_{T}(\theta))}{\partial\theta}\right|_{\theta=\hat{\theta}_{T}}$.
Except when indicated otherwise, such an ambiguity cannot occur because we
never use derivatives of $\theta\mapsto L_{T}(\theta,\tau_{T}(\theta))$.

Under Assumptions 1 and 2(a), by subsection B.2 (p. B.2), the function
$L_{T}(\theta,\tau)$ is well-defined and twice continuously differentiable in
a neighborhood of $(\theta_{0}^{\prime}\;\tau(\theta_{0})^{\prime})$
$\mathbb{P}$-a.s. for $T$ big enough. Moreover, under Assumption 1(a)(b) and
(d)-(h), by Lemma 21i (p. 21), $\tau_{T}(.)$ is continuously differentiable in
$\mathbf{\Theta}$. Now, under Assumption 1, by Theorem 1i (p. 1) and Lemma
2iii (p. 2), $\mathbb{P}$-a.s., $\hat{\theta}_{T}\rightarrow\theta_{0}$ and
$\tau_{T}(\hat{\theta}_{T})\rightarrow\tau(\theta_{0})$, so that
$\mathbb{P}$-a.s. for $T$ big enough,
$(\hat{\theta}_{T}^{\prime}\;\tau_{T}(\hat{\theta}_{T})^{\prime})$ is in any
arbitrary small neighborhood of
$(\theta_{0}^{\prime}\;\tau(\theta_{0})^{\prime})$. Therefore, under
Assumption 1 and 2(a), by the chain rule theorem (e.g., Magnus and Neudecker
1999/1988, Chap. 5 sec. 11), $\mathbb{P}$-a.s. for $T$ big enough,
$\theta\mapsto L_{T}(\theta,\tau_{T}(\theta))$ is continuously differentiable
in a neighborhood of $\hat{\theta}_{T}$, and, for all $j\in[\\![1,m]\\!]$,

$\displaystyle 0=\left.\frac{\partial
L_{T}(\theta,\tau_{T}(\theta))}{\partial\theta_{j}}\right|_{\theta=\hat{\theta}_{T}}$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\Leftrightarrow}}$ $\displaystyle
0=\left.\frac{\partial
L_{T}(\theta,\tau)}{\partial\theta_{j}}\right|_{(\theta,\tau)=(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}+\left.\frac{\partial
L_{T}(\theta,\tau)}{\partial\tau^{\prime}}\right|_{(\theta,\tau)=(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}\left.\frac{\partial\tau(\theta)}{\partial\theta_{j}}\right|_{\theta=\hat{\theta}_{T}}$
$\displaystyle\stackrel{{\scriptstyle}}{{\Leftrightarrow}}$
$\displaystyle\left.\frac{\partial
L_{T}(\theta,\tau)}{\partial\theta_{j}}\right|_{(\theta,\tau)=(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}=-\left.\frac{\partial
L_{T}(\theta,\tau)}{\partial\tau^{\prime}}\right|_{(\theta,\tau)=(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}\left.\frac{\partial\tau(\theta)}{\partial\theta_{j}}\right|_{\theta=\hat{\theta}_{T}}$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{\Leftrightarrow}}$
$\displaystyle\left.\frac{\partial
L_{T}(\theta,\tau)}{\partial\theta_{j}}\right|_{(\theta,\tau)=(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}=O(T^{-1})O(1)=O(T^{-1}).$

(a) It is an immediate and standard implication of the chain rule (e.g.,
Magnus and Neudecker 1999/1988, chap. 5, sec. 12, exercise 3). (b) Firstly,
under Assumptions 1 and 2, by Lemma 22iv (p. 22), $\mathbb{P}$-a.s. as
$T\rightarrow\infty$, $\left.\frac{\partial
L_{T}(\theta,\tau)}{\partial\tau^{\prime}}\right|_{(\theta,\tau)=(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}=O(T^{-1})$
because
$(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))\rightarrow(\theta_{0},\tau(\theta_{0}))$,
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, by Theorem 1i (p. 1) and Lemma 2iii
(p. 2). Secondly, under Assumptions 1 and 2, by Theorem 1i (p. 1) and Lemma
21iii (p. 21), $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,$\left.\frac{\partial\tau(\theta)}{\partial\theta_{j}}\right|_{(\theta,\tau)=(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}=O(1)$.
∎

######  Lemma 21 (First Derivative of the implicit function $\tau_{T}(.)$).

Under Assumption 1(a)(b) and (d)-(h),

  1. (i)

$\mathbb{P}$-a.s. for $T$ big enough, the function
$\tau_{T}:\mathbf{\Theta}\rightarrow\mathbf{R}^{m}$ is continuously
differentiable in $\mathbf{\Theta}$ and its first derivative is

$\displaystyle\frac{\partial\tau_{T}(\theta)}{\partial\theta^{\prime}}=\negthickspace-\negthickspace\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}(\theta)^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\negthickspace\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}(\theta)^{\prime}\psi_{t}(\theta)}\negthickspace\left(\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\negthickspace+\negthickspace\psi_{t}(\theta)\tau_{T}(\theta)^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right)\right];$

  2. (ii)

for any sequence
$(\theta_{T})_{T\in\mathbf{N}}\in\mathbf{\Theta}^{\mathbf{N}}$ converging to
$\theta_{0}$, $\mathbb{P}$-a.s. for $T$ big enough, there exists
$\bar{\theta}_{T}$ between $\theta_{T}$ and $\theta_{0}$ s.t.
$\sqrt{T}[\tau_{T}(\theta_{T})-\tau_{T}(\theta_{0})]=\frac{\partial\tau_{T}(\bar{\theta}_{T})}{\partial\theta^{\prime}}\sqrt{T}(\theta_{T}-\theta_{0})$;

  3. (iii)

under additional Assumptions 1(c) and 2(b), for any sequence
$(\theta_{T})_{T\in\mathbf{N}}\in\mathbf{\Theta}^{\mathbf{N}}$ converging to
$\theta_{0}$, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$\frac{\partial\tau_{T}(\theta_{T})}{\partial\theta^{\prime}}\rightarrow-\mathbb{E}[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]^{-1}\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta}\right]$;
and

  4. (iv)

under additional Assumptions 1(c) and 2(b), for any sequence
$(\theta_{T})_{T\in\mathbf{N}}\in\mathbf{\Theta}^{\mathbf{N}}$ converging to
$\theta_{0}$ s.t., as $T\rightarrow\infty$,
$\sqrt{T}(\theta_{T}-\theta_{0})=O_{\mathbb{P}}(1)$ $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,
$\tau_{T}(\theta_{T})-\tau_{T}(\theta_{0})=-V^{-1}M(\theta_{T}-\theta_{0})+o_{\mathbb{P}}(T^{-1/2})$,
where $V:=\mathbb{E}[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]$
and
$M:=\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta}\right]$.

###### Proof.

(i) Under Assumption 1(a)(b) and (d)-(h), by Lemma 1ii (p. 1) and its proof,
$\mathbb{P}$-a.s. for $T$ big enough, the assumptions of the standard implicit
function theorem hold and $\tau_{T}(.)$ is continuously differentiable. Thus,
under Assumption 1(a)(b) and (d)-(h), $\mathbb{P}$-a.s. for $T$ big enough,
application of the implicit function theorem yields

$\displaystyle\frac{\partial\tau_{T}(\theta)}{\partial\theta^{\prime}}$
$\displaystyle=$
$\displaystyle\left.-\left[\frac{\partial\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\right]}{\partial\tau^{\prime}}\right]^{-1}\left[\frac{\partial\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\right]}{\partial\theta^{\prime}}\right]\right|_{\tau=\tau_{T}(\theta)}$
$\displaystyle=$
$\displaystyle\left.-\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}+\psi_{t}(\theta)\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right)\right]\right|_{\tau=\tau_{T}(\theta)}$
$\displaystyle=$
$\displaystyle\negthickspace-\negthickspace\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}(\theta)^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\negthickspace\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}(\theta)^{\prime}\psi_{t}(\theta)}\negthickspace\left(\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\negthickspace+\negthickspace\psi_{t}(\theta)\tau_{T}(\theta)^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right)\right].$

(ii) Again, under Assumption 1(a)(b) and (d)-(h), by Lemma 1ii (p. 1),
$\mathbb{P}$-a.s. for $T$ big enough, $\tau_{T}(.)$ is continuously
differentiable, so that the result follows from a first-order stochastic
Taylor-Lagrange expansion (Jennrich 1969, Lemma 3).

(iii) Firstly, under Assumption 1(a)(b)(d)(e)(g)(h), by Lemma 2iii (p. 2),
$\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$\sup_{\theta\in\mathbf{\Theta}}|\tau_{T}(\theta)-\tau(\theta)|=o(1)$, so that
$\tau_{T}(\theta_{T})\rightarrow\tau(\theta_{0})$. Secondly, under Assumptions
1 and 2, by Lemma 23iv, vii and x (p. 23), for $\overline{B_{L}}$ a ball
around $(\theta_{0},\tau(\theta_{0}))$ of sufficiently small radius,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$,
and
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$.
Thus, by Assumptions 1(a)(b) and (d), the ULLN (uniform law of large numbers)
à la Wald (e.g., Ghosh and Ramamoorthi 2003, pp. 24-25, Theorem 1.3.3),
implies that, for all $k\in[\\![1,m]\\!]$, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,

$\displaystyle\ \frac{\partial\tau_{T}(\theta_{T})}{\partial\theta^{\prime}}$
$\displaystyle\rightarrow$
$\displaystyle-\mathbb{E}[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]^{-1}\left\\{\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta}\right]\right.$
$\displaystyle\
\left.+\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi(X_{1},\theta)\tau(\theta_{0})^{\prime}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta}\right]\right\\}$
$\displaystyle=$
$\displaystyle-\mathbb{E}[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]^{-1}\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta}\right]$

because $\tau(\theta_{0})=0_{m\times 1}$ by Lemma 10iv (p. 10) under
Assumption 1(a)-(e) and (g)-(h).

(iv) Under Assumption 1 and 2(b), by the statement (ii) of the present lemma,
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, there exists $\bar{\theta}_{T}$
between $\theta_{T}$ and $\theta_{0}$ s.t.

$\displaystyle\tau_{T}(\theta_{T})-\tau_{T}(\theta_{0})$ $\displaystyle=$
$\displaystyle\frac{\partial\tau_{T}(\bar{\theta}_{T})}{\partial\theta^{\prime}}(\theta_{T}-\theta_{0})$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{=}}$
$\displaystyle-V^{-1}M(\theta_{T}-\theta_{0})+\left[\frac{\partial\tau_{T}(\bar{\theta}_{T})}{\partial\theta^{\prime}}+V^{-1}M\right](\theta_{T}-\theta_{0})$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{=}}$
$\displaystyle-V^{-1}M(\theta_{T}-\theta_{0})+o_{\mathbb{P}}(T^{-1/2})$

(a) Add and subtract $V^{-1}M(\theta_{T}-\theta_{0})$. (b) Under Assumption 1
and 2(b), by the statement (ii) of the present lemma, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,
$\frac{\partial\tau_{T}(\bar{\theta}_{T})}{\partial\theta^{\prime}}+V^{-1}M=o(1)$.
Moreover, by assumption, as $T\rightarrow\infty$,
$\theta_{T}-\theta_{0}=O_{\mathbb{P}}(T^{-1/2})$, so that
$\left[\frac{\partial\tau_{T}(\bar{\theta}_{T})}{\partial\theta^{\prime}}+V^{-1}M\right](\theta_{T}-\theta_{0})=o_{\mathbb{P}}(T^{-1/2})$.
∎

######  Remark 3.

As notation indicates, $\frac{\partial\tau_{T}(.)}{\partial\theta^{\prime}}$
corresponds to a partial derivative as $\tau_{T}(.)$ is also a function of the
data. $\diamond$

######  Lemma 22 (Asymptotic limit of $\frac{\partial
L_{T}(\theta_{T},\tau_{T}(\theta_{T}))}{\partial\tau^{\prime}}$).

Under Assumptions 1 and 2, for any sequence
$(\theta_{T})_{T\in\mathbf{N}}\in\mathbf{\Theta}^{\mathbf{N}}$ converging to
$\theta_{0}$, for all $k\in[\\![1,m]\\!]$, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,

  1. (i)

$\frac{\partial
M_{1,T}(\theta_{T},\tau_{T}(\theta_{T}))}{\partial\tau_{k}}=0$;

  2. (ii)

$\frac{\partial
M_{2,T}(\theta_{T},\tau_{T}(\theta_{T}))}{\partial\tau_{k}}=O(T^{-1})$;

  3. (iii)

$\frac{\partial
M_{3,T}(\theta_{T},\tau_{T}(\theta_{T}))}{\partial\tau_{k}}=O(T^{-1})$; and

  4. (iv)

$\frac{\partial
L_{T}(\theta_{T},\tau_{T}(\theta_{T}))}{\partial\tau^{\prime}}=O(T^{-1})$.

###### Proof.

(i) Under Assumption 1(a)(b) and (d)-(h), by Lemma 2ii (p. 2),
$\mathbb{P}$-a.s. for $T$ big enough, $\tau_{T}(\theta_{T})$ exists, so that,
by equation (28) on p. 28, $\mathbb{P}$-a.s. for $T$ big enough, for all
$k\in[\\![1,m]\\!]$,

$\displaystyle\frac{\partial
M_{1,T}(\theta_{T},\tau_{T}(\theta_{T}))}{\partial\tau_{k}}$ $\displaystyle=$
$\displaystyle\left(1-\frac{m}{2T}\right)\frac{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}(\theta_{T})^{\prime}\psi_{t}(\theta_{T})}\psi_{t,k}(\theta_{T})}{\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau_{T}(\theta_{T})^{\prime}\psi_{i}(\theta_{T})}}$
$\displaystyle=$ $\displaystyle 0$

because, by definition of $\tau_{T}(\theta)$ in equation (15) on p. 15,
$\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}(\theta_{T})^{\prime}\psi_{t}(\theta_{T})}\psi_{t,k}(\theta_{T})=0$.

(ii) Similarly, under Assumption 1, by equation (34) on p. 34,
$\mathbb{P}$-a.s. for $T$ big enough, for all $k\in[\\![1,m]\\!]$,

$\displaystyle\frac{\partial
M_{2,T}(\theta_{T},\tau_{T}(\theta_{T}))}{\partial\tau_{k}}=\frac{1}{T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}(\theta_{T})^{\prime}\psi_{t}(\theta_{T})}\frac{\partial\psi_{t}(\theta_{T})}{\partial\theta^{\prime}}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}(\theta_{T})^{\prime}\psi_{t}(\theta_{T})}\psi_{t,k}(\theta_{T})\frac{\partial\psi_{t}(\theta_{T})}{\partial\theta^{\prime}}\right]\right\\}$

where $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$(\theta_{T}^{\prime}\quad\tau_{T}(\theta_{T})^{\prime})\rightarrow(\theta_{0}^{\prime}\quad\tau(\theta_{0})^{\prime})$
by the lemma’s assumption and Lemma 2iii (p. 2). Now, under Assumptions 1 and
2, by Lemma 23iv and v (p. 23), for $\overline{B_{L}}$ a ball around
$(\theta_{0},\tau(\theta_{0}))$ of sufficiently small radius,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$,
and, for all $k\in[\\![1,m]\\!]$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$.
Thus, by Assumptions 1(a)(b) and (d), the ULLN (uniform law of large numbers)
à la Wald (e.g., Ghosh and Ramamoorthi 2003, pp. 24-25, Theorem 1.3.3),
implies that, for all $k\in[\\![1,m]\\!]$, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,

$\displaystyle T\frac{\partial
M_{2,T}(\theta_{T},\tau_{T}(\theta_{T}))}{\partial\tau_{k}}$
$\displaystyle\rightarrow$ $\displaystyle{\rm
tr}\left\\{\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi_{k}(X_{1},\theta_{0})\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]\right\\}$
$\displaystyle=$ $\displaystyle{\rm
tr}\left\\{\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\mathbb{E}\left[\psi_{k}(X_{1},\theta_{0})\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]\right\\},$

because $\tau(\theta_{0})=0_{m\times 1}$ by Lemma 10iv (p. 10) under
Assumption 1(a)-(e) and (g)-(h). Therefore, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$, $\frac{\partial
M_{2,T}(\theta_{T},\tau_{T}(\theta_{T}))}{\partial\tau_{k}}=O(T^{-1})$.

(iii) Under Assumption 1, by equation (39) (p. 39), for all
$k\in[\\![1,m]\\!]$,

$\displaystyle\frac{\partial
M_{3,T}(\theta_{T},\tau_{T}(\theta_{T}))}{\partial\tau_{k}}$ $\displaystyle=$
$\displaystyle-\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}(\theta_{T})^{\prime}\psi_{t}(\theta_{T})}\psi_{t}(\theta_{T})\psi_{t}(\theta_{T})^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}(\theta_{T})^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta_{T})\psi_{t}(\theta_{T})\psi_{t}(\theta_{T})^{\prime}\right]\right\\}$

where $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$(\theta_{T}^{\prime}\quad\tau_{T}(\theta_{T})^{\prime})\rightarrow(\theta_{0}^{\prime}\quad\tau(\theta_{0})^{\prime})$
by Theorem 1i (p. 1). Now, under Assumptions 1 and 2, by Lemma 23vii and viii
(p. 23), there exists a closed ball $\overline{B_{L}}\subset\mathbf{S}$
centered at $(\theta_{0},\tau(\theta_{0}))$ with strictly positive radius
s.t., for all $k\in[\\![1,m]\\!]$,

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$
and

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$.
Thus, under Assumptions 1 and 2, by ULLN (uniform law of large numbers) à la
Wald (e.g., Ghosh and Ramamoorthi 2003, pp. 24-25, Theorem 1.3.3), for all
$k\in[\\![1,m]\\!]$, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,

$\displaystyle T\frac{\partial
M_{3,T}(\theta_{T},\tau_{T}(\theta_{T}))}{\partial\tau_{k}}$
$\displaystyle\rightarrow$ $\displaystyle-\frac{1}{2}{\rm
tr}\left\\{\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]^{-1}\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi_{k}(X_{1},\theta_{0})\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\right\\}$
$\displaystyle=$ $\displaystyle-\frac{1}{2}{\rm
tr}\left\\{\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]^{-1}\mathbb{E}\left[\psi_{k}(X_{1},\theta_{0})\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\right\\},$

because $\tau(\theta_{0})=0_{m\times 1}$ by Lemma 10iv (p. 10) under
Assumption 1(a)-(e) and (g)-(h). Therefore, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$, $\frac{\partial
M_{3,T}(\theta_{T},\tau_{T}(\theta_{T}))}{\partial\tau_{k}}=O(T^{-1})$.

(iv) Under Assumption 1(a)-(b) and (d)-(h), by Lemma 12 (p. 12),
$L_{T}(\theta,\tau)=M_{1,T}(\theta,\tau)+M_{2,T}(\theta,\tau)+M_{3,T}(\theta,\tau)$,
so that the result follows from the statement (i)-(iii) of the present lemma.
∎

######  Remark 4.

In the case in which $\theta_{T}=\hat{\theta}_{T}$, there exist at least one
other way to prove Lemma 22 that do not require Assumption 2. This way follows
an approach à la Newey and Smith (2004), which relies on ULLN with
$\mathbf{T}_{T}(\theta)=\\{\tau\in\mathbf{R}^{m}:|\tau|\leqslant
T^{-\zeta}\\}$ and $\zeta>0$. We do not follow this ways because (i) Other
parts of the proof of Theorem 1ii (p. 1) require the asymptotic normality of
$\hat{\theta}_{T}$ and thus Assumption 2; (ii) It would lengthen the proofs
and complicate their logic; (iii) We later use Lemma 22 with
$\theta_{T}=\check{\theta}_{T}$, where $\check{\theta}_{T}$ is a constrained
estimator. $\diamond$

######  Lemma 23 (Finiteness of the expectations of supremum of the terms from
$\frac{\partial L_{T}(\theta,\tau)}{\partial\tau}$ and
$\frac{\partial^{2}L_{T}(\theta,\tau)}{\partial\tau^{\prime}\partial\tau}$).

Under Assumptions 1 and 2, there exists a closed ball
$\overline{B_{L}}\subset\mathbf{S}$ centered at
$(\theta_{0},\tau(\theta_{0}))$ with strictly positive radius s.t., for all
$(h,k)\in[\\![1,m]\\!]^{2}$,

  * (i)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\right]<\infty$;

  * (ii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)|\right]<\infty$;

  * (iii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)|\right]<\infty$;

  * (iv)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$;

  * (v)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$;

  * (vi)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$;

  * (vii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$;

  * (viii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$;

  * (ix)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$;
and

  * (x)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$.

###### Proof.

(i) Apply Lemma 18i (p. 18) under Assumptions 1 and 2. Note that it does not
immediately follow from Assumption 1(e) and the Cauchy-Schwarz inequality
because we need additional assumptions to ensures that there exists
$\overline{B_{L}}\subset\mathbf{S}$: See Lemma 11(ii) on p. 11.

(ii) For all $(h,k)\in[\\![1,m]\\!]^{2}$, for all
$(\theta,\tau)\in\overline{B_{L}}$,
$\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}|\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)|=\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\\\
\sqrt{[\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)]^{2}}\leqslant\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\sqrt{\sum_{(i,j)\in[\\![1,m]\\!]^{2}}[\psi_{i}(X_{1},\theta)\psi_{j}(X_{1},\theta)]^{2}}=\
\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|$,
so that
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)|\right]\leqslant\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)|\right]<\infty$,
where the last inequality follows from Lemma 18xii (p. 18) under Assumptions 1
and 2.

(iii) Apply Lemma 19v (p. 19) under Assumptions 1 and 2.

(iv) Apply Lemma 18v (p. 18) under Assumptions 1 and 2.

(v) Apply Lemma 19vii (p. 19) under Assumptions 1 and 2.

(vi) Proof similar to the one of Lemma 18xiii (p. 18). The supremum of the
absolute value of the product is smaller than the product of the suprema of
the absolute values. Thus, under Assumption 1(a)(b), for $\overline{B_{L}}$ of
sufficiently small radius, for all $(h,k)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]$
$\displaystyle\leqslant$
$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)|\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leqslant}}$
$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle(c)}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}\stackrel{{\scriptstyle(d)}}{{<}}\infty.$

(a) As in the proof of statement (ii), for all $(h,k)\in[\\![1,m]\\!]^{2}$,
for all

$(\theta,\tau)\in\overline{B_{L}}$,
$|\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)|\leqslant|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|$.
(b) Apply the Cauchy-Schwarz inequality, and note that the supremum of the
square of a positive function is the square of the supremum of the function.
(c) Firstly, under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11),
$\mathbf{S}$ contains an open ball centered at
$(\theta_{0},\tau(\theta_{0}))$, so that, for $\overline{B_{L}}$ of
sufficiently small radius,
$\overline{B_{L}}\subset\\{(\theta,\tau):\theta\in\mathcal{N}\wedge\tau\in\mathbf{T}(\theta)\\}\subset\mathbf{S}\subset\mathbf{S}^{\epsilon}$.
Secondly, as the second supremum does not depend on $\tau$,
$\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\leqslant\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}$
because $\overline{B_{L}}\subset\mathbf{S}$, for $\overline{B_{L}}$ of radius
small enough. (d) Firstly, by Assumption 2(b), the first expectation is
bounded. Secondly, by Assumption 1(g), the second expectation is also bounded.

(vii) Apply Lemma 18xii (p. 18) under Assumptions 1 and 2.

(viii) Proof similar to the one of Lemma 18xiii (p. 18) and to the statement
(vi) of the present lemma. The supremum of the absolute value of the product
is smaller than the product of the suprema of the absolute values. Thus, under
Assumption 1(a)(b), for $\overline{B_{L}}$ of sufficiently small radius, for
all $(h,k)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\psi(X_{1},\theta)\psi(X_{1},\theta)|\right]$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)|^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}\stackrel{{\scriptstyle(c)}}{{<}}\infty.$

(a) Apply the Cauchy-Schwarz inequality, and note that the supremum of the
square of a positive function is the square of the supremum of the function.
(b) Firstly, under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11),
$\mathbf{S}$ contains an open ball centered at
$(\theta_{0},\tau(\theta_{0}))$, so that, for $\overline{B_{L}}$ of
sufficiently small radius,
$\overline{B_{L}}\subset\\{(\theta,\tau):\theta\in\mathcal{N}\wedge\tau\in\mathbf{T}(\theta)\\}\subset\mathbf{S}\subset\mathbf{S}^{\epsilon}$.
Moreover, for all $k\in[\\![1,m]\\!]$, for all $\theta\in\mathbf{\Theta}$,
$|\psi_{k}(X_{1},\theta)|\leqslant|\psi(X_{1},\theta)|\leqslant b(X)$, where
the last inequality follows from Assumption 2(b). Secondly, as the second
supremum does not depend on $\tau$,
$\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\leqslant\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}$
because $\overline{B_{L}}\subset\mathbf{S}$, for $\overline{B_{L}}$ of radius
small enough. (c) Firstly, by Assumption 2(b), the first expectation is
bounded. Secondly, by Assumption 1(g), the second expectation is also bounded.

(ix) Proof similar to the one of Lemma 18xiii (p. 18) and to the statement
(vi) of the present lemma. The supremum of the absolute value of the product
is smaller than the product of the suprema of the absolute values. Thus, under
Assumption 1(a)(b), for $\overline{B_{L}}$ of sufficiently small radius, for
all $(h,k)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\psi(X_{1},\theta)\psi(X_{1},\theta)|\right]$
$\displaystyle\stackrel{{\scriptstyle}}{{\leqslant}}$
$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)|^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{4}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}\stackrel{{\scriptstyle(c)}}{{<}}\infty.$

(a) Apply the Cauchy-Schwarz inequality, and note that the supremum of the
square of a positive function is the square of the supremum of the function.
(b) Firstly, under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11),
$\mathbf{S}$ contains an open ball centered at
$(\theta_{0},\tau(\theta_{0}))$, so that, for $\overline{B_{L}}$ of
sufficiently small radius,
$\overline{B_{L}}\subset\\{(\theta,\tau):\theta\in\mathcal{N}\wedge\tau\in\mathbf{T}(\theta)\\}\subset\mathbf{S}\subset\mathbf{S}^{\epsilon}$.
Moreover, for all $k\in[\\![1,m]\\!]$, for all $\theta\in\mathbf{\Theta}$,
$|\psi_{k}(X_{1},\theta)|\leqslant|\psi(X_{1},\theta)|\leqslant b(X)$ where
the last inequality follows from Assumption 2(b). Secondly, as the second
supremum does not depend on $\tau$,
$\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\leqslant\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}$
because $\overline{B_{L}}\subset\mathbf{S}$, for $\overline{B_{L}}$ of radius
small enough. (c) Firstly, by Assumption 2(b), the first expectation is
bounded. Secondly, by Assumption 1(g), the second expectation is also bounded.

(x) The norm of a product of matrices is smaller than the product of the norms
(e.g., Rudin 1953, Theorem 9.7 and note that all norms are equivalent on
finite dimensional spaces). Thus, for $\overline{B_{L}}$ of sufficiently small
radius, for all $(\ell,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\
\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]$
$\displaystyle\leqslant$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\psi(X_{1},\theta)||\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]\stackrel{{\scriptstyle(b)}}{{<}}\infty.$

(a) Firstly, under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11),
$\mathbf{S}$ contains an open ball centered at $(\theta_{0},\tau(\theta_{0}))$
Thus, under Assumption 1(a)-(e) and (g)-(h), for $\overline{B_{L}}$ of
sufficiently small radius, by definition of $\mathbf{S}$,
$\overline{B_{L}}\subset\\{(\theta,\tau):\theta\in\mathcal{N}\wedge\tau\in\mathbf{T}(\theta)\\}\subset\mathbf{S}$,
because $\mathcal{N}\subset\mathbf{\Theta}$ by Assumption 2(a). Secondly, by
Assumption 2(b), $\sup_{\theta\in\mathcal{N}}|\psi(X_{1},\theta)|\leqslant
b(X)$ and
$\sup_{\theta\in\mathcal{N}}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\leqslant
b(X)$. (b) Firstly, $\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2}<\infty$
because $\overline{B_{L}}$ is bounded. Secondly, by Assumption 2(b),
$\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]<\infty$.
∎

###  B.4. Proof of Theorem 2: Trinity$+1$

The proof of Theorem 2 adapts the traditional way of deriving the trinity
along the lines of Smith (2011). As in the proof of Theorem 1, the main
difference comes from the complexity of the variance term
$|\Sigma_{T}(\theta)|_{\det}$.

###### Core of the proof of Theorem 2.

Asymptotic distribution of $\mathrm{Wald}_{T}$. By Assumption 3(a),
$r:\mathbf{\Theta}\rightarrow\mathbf{R}^{q}$ is continuously differentiable.
Thus, under Assumptions 1 and 2, if the test hypothesis (9) on p. 9 holds, a
first-order Taylor-Lagrange expansion at $\theta_{0}$ evaluated at
$\hat{\theta}_{T}$, $\omega$ by $\omega$, yields, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,

$\displaystyle r(\hat{\theta}_{T})$ $\displaystyle=$ $\displaystyle
r(\theta_{0})+R(\bar{\theta}_{T})(\hat{\theta}_{T}-\theta_{0})\text{ , where
$\bar{\theta}_{T}$ is between $\hat{\theta}_{T}$ and $\theta_{0}$; }$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{=}}$ $\displaystyle
R(\bar{\theta}_{T})(\hat{\theta}_{T}-\theta_{0})$
$\displaystyle\underset{(b)}{\stackrel{{\scriptstyle D}}{{\rightarrow}}}$
$\displaystyle R(\theta_{0})\mathcal{N}(0,\Sigma(\theta_{0}))$
$\displaystyle\stackrel{{\scriptstyle D}}{{=}}$
$\displaystyle\mathcal{N}\left(0,R(\theta_{0})\Sigma(\theta_{0})R(\theta_{0})^{\prime}\right).$

(a) By definition, if the test hypothesis (9) on p. 9 holds,
$r(\theta_{0})=0_{q\times 1}$. (b) Under Assumptions 1 and 2, by Theorem 1ii
(p. 1), $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$\sqrt{T}(\hat{\theta}_{T}-\theta_{0})\stackrel{{\scriptstyle
D}}{{\rightarrow}}\mathcal{N}(0,\Sigma(\theta_{0}))$, which also implies that
$\bar{\theta}_{T}\rightarrow\theta_{0}$. Thus, under Assumptions 1, 2 and
3(a), by continuity of $R(.)$, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$R(\bar{\theta})\rightarrow R(\theta_{0})$, so that the result follows by the
Slutsky’s theorem.

Now, under Assumptions 1, 2 and 3(a), by Lemma 28i (p. 28), $\mathbb{P}$-a.s.
as $T\rightarrow\infty$, $\check{\theta}_{T}\rightarrow\theta_{0}$, so that
$R(\check{\theta}_{T})\rightarrow R(\theta_{0})$ by Assumption 3(a). Moreover,
by the theorem’s assumption, as $T\rightarrow\infty$,
$\widehat{\Sigma(\theta_{0})}\stackrel{{\scriptstyle\mathbb{P}}}{{\rightarrow}}\Sigma(\theta_{0})$.
In addition, by Assumption 1(h) and 3(b), $\Sigma(\theta_{0})$ and
$R(\theta_{0})$ are full rank, so that $\widehat{\Sigma(\theta_{0})}$ and
$R(\check{\theta}_{T})$ are full rank w.p.a.1 as $T\rightarrow\infty$ (Lemma
30 p. 30). Then, the result follows from the Cochran’s theorem.

Asymptotic distribution of $\mathrm{LM}_{T}$. Under Assumptions 1, 2 and 3, by
Proposition 2iii (p. 2), if the test hypothesis (9) on p. 9 holds, as
$T\rightarrow\infty$, $\check{\gamma}_{T}\stackrel{{\scriptstyle
D}}{{\rightarrow}}\mathcal{N}(0,(R(\theta_{0})\Sigma(\theta_{0})R(\theta_{0})^{\prime})^{-1})$.
Now, under Assumptions 1, 2 and 3(a), by Lemma 28i (p. 28), $\mathbb{P}$-a.s.
as $T\rightarrow\infty$, $\check{\theta}_{T}\rightarrow\theta_{0}$, so that
$R(\check{\theta}_{T})\rightarrow R(\theta_{0})$ by Assumption 3(a). Moreover,
by the theorem’s assumption, as $T\rightarrow\infty$,
$\widehat{\Sigma(\theta_{0})}\stackrel{{\scriptstyle\mathbb{P}}}{{\rightarrow}}\Sigma(\theta_{0})$.
In addition, by Assumption 1(h) and 3(b), $\Sigma(\theta_{0})$ and
$R(\theta_{0})$ are full rank, so that $\widehat{\Sigma(\theta_{0})}$ and
$R(\check{\theta}_{T})$ are full rank w.p.a.1 as $T\rightarrow\infty$ (Lemma
30 p. 30). Then, by the Cochran’s theorem, as $T\rightarrow\infty$,
$T\check{\gamma}_{T}^{\prime}[R(\check{\theta}_{T})\widehat{\Sigma(\theta_{0})}R(\check{\theta}_{T})^{\prime}]\check{\gamma}_{T}\stackrel{{\scriptstyle
D}}{{\rightarrow}}\chi_{q}^{2}$. Finally, under Assumptions 1, 2 and 3, by
Lemma 28iii (p. 28),
$R(\check{\theta}_{T})^{\prime}\check{\gamma}_{T}=-\left.\frac{\partial
L_{T}(\theta,\tau_{T}(\theta))}{\partial\theta}\right|_{\theta=\check{\theta}_{T}}$,
so
$T\check{\gamma}_{T}^{\prime}[R(\check{\theta}_{T})\widehat{\Sigma(\theta_{0})}R(\check{\theta}_{T})^{\prime}]\check{\gamma}_{T}=T[R(\check{\theta}_{T})^{\prime}\check{\gamma}_{T}]^{\prime}\widehat{\Sigma(\theta_{0})}[R(\check{\theta}_{T})^{\prime}\check{\gamma}_{T}]=\negthickspace
T\left.\frac{\partial
L_{T}(\theta,\tau_{T}(\theta))}{\partial\theta}\right|_{\theta=\check{\theta}_{T}}^{\prime}\negthickspace\widehat{\Sigma(\theta_{0})}\left.\frac{\partial
L_{T}(\theta,\tau_{T}(\theta))}{\partial\theta}\right|_{\theta=\check{\theta}_{T}}\negthickspace\negthickspace\negthickspace\negthickspace=\frac{\partial\ln[\hat{f}_{\theta^{*}_{T}}(\check{\theta}_{T})]}{\partial\theta^{\prime}}\widehat{\Sigma(\theta_{0})}\frac{\partial\ln[\hat{f}_{\theta^{*}_{T}}(\check{\theta}_{T})]}{\partial\theta}$,
where the last equality follows from the definition of the LogESP in Lemma 12
(p. 12), i.e.,

$L_{T}(\theta,\tau_{T}(\theta)):=\ln\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}(\theta)^{\prime}\psi_{t}(\theta)}\right]-\frac{1}{2T}\ln|\Sigma_{T}(\theta)|_{\det}=\frac{1}{T}[\ln(\hat{f}_{\theta^{*}_{T}}(\theta))-\frac{m}{2}\ln(\frac{T}{2\pi})].$

Asymptotic distribution of $\mathrm{ALR}_{T}$. Under Assumptions 1,2 and 3, if
the test hypothesis (9) on p. 9 holds, by Lemma 24 (p. 24), $\mathbb{P}$-a.s.
as $T\rightarrow\infty$,

$\displaystyle
2\\{\ln[\hat{f}_{\theta^{*}_{T}}(\hat{\theta}_{T})]-\ln[\hat{f}_{\theta^{*}_{T}}(\check{\theta}_{T})]\\}$
$\displaystyle=$
$\displaystyle-\left[\sqrt{T}(\hat{\theta}_{T}-\check{\theta}_{T})\right]^{\prime}\Sigma^{-1}\left[\sqrt{T}(\hat{\theta}_{T}-\check{\theta}_{T})\right]+o_{\mathbb{P}}(1)$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{=}}$ $\displaystyle-\left[\Sigma
R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\negthickspace+\negthickspace
o_{\mathbb{P}}(1)\right]^{\prime}\negthickspace\negthickspace\Sigma^{-1}\negthickspace\left[\Sigma
R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\negthickspace+\negthickspace
o_{\mathbb{P}}(1)\right]\negthickspace+o_{\mathbb{P}}(1)$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{=}}$
$\displaystyle-\left[\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]^{\prime}(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma\Sigma^{-1}\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\left[\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]+o_{\mathbb{P}}(1)$
$\displaystyle\stackrel{{\scriptstyle(c)}}{{=}}$
$\displaystyle-\left[\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]^{\prime}(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\left[\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]\negthickspace+o_{\mathbb{P}}(1)$
$\displaystyle\stackrel{{\scriptstyle(d)}}{{=}}$
$\displaystyle-\left[V^{-1/2}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]^{\prime}V^{1/2^{\prime}}(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}V^{1/2}\left[V^{-1/2}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]\negthickspace+o_{\mathbb{P}}(1)$
$\displaystyle\stackrel{{\scriptstyle(e)}}{{=}}$
$\displaystyle-\left[V^{-1/2}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]^{\prime}P_{\Sigma^{1/2}R^{\prime}}\left[V^{-1/2}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]\negthickspace+o_{\mathbb{P}}(1)$
$\displaystyle\underset{(f)}{\stackrel{{\scriptstyle D}}{{\rightarrow}}}$
$\displaystyle\chi^{2}_{q}$

(a) Under Assumptions 1, 2 and 3, if the test hypothesis (9) on p. 9 holds, by
Lemma 24ii (p. 24), $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,$\sqrt{T}(\hat{\theta}_{T}-\check{\theta}_{T})=\Sigma
R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$.
(b) Transpose the content of the first square bracket, and then note that
$\Sigma=\Sigma^{\prime}$ by symmetry. (c) Note that $R\Sigma\Sigma^{-1}\Sigma
R^{\prime}(R\Sigma R^{\prime})^{-1}=I$. (d) Use that $V^{1/2}V^{-1/2}=I$. (e)
Note that
$P_{\Sigma^{1/2}R^{\prime}}=V^{1/2^{\prime}}(M^{\prime})^{-1}R(R^{\prime}\Sigma
R)^{-1}R^{\prime}M^{-1}V^{1/2}$. (f) Under Assumption 1(a)-(c) and (g), by the
Lindeberg-Lévy CLT theorem, as $T\rightarrow\infty$,
$\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\stackrel{{\scriptstyle
D}}{{\rightarrow}}\mathcal{N}(0,V)$ where
$V:=\mathbb{E}[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]$.
Moreover, the orthogonal projection matrix $P_{\Sigma^{1/2}R^{\prime}}$ has
rank $q$ because $R$ is of rank $q$ and $\Sigma$ has full rank by Assumptions
3(b) and 1(h), respectively. Thus, the result follows from the Cochran’s
theorem.

Asymptotic distribution of $\mathrm{ET}_{T}$. Under Assumptions 1, 2 and 3, if
the test hypothesis (9) on p. 9 holds, by Proposition 2ii (p. 2),
$\mathbb{P}$-a.s. as $T\rightarrow\infty$,

$\displaystyle\sqrt{T}\tau_{T}(\check{\theta}_{T})$ $\displaystyle=$
$\displaystyle(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{=}}$
$\displaystyle(M^{\prime})^{-1}[V^{1/2}(M^{\prime})^{-1}]^{-1}P_{\Sigma^{1/2}R^{\prime}}[M^{-1}V^{1/2^{\prime}}]^{-1}M^{-1}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{=}}$
$\displaystyle(M^{\prime})^{-1}M^{\prime}V^{-1/2}P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}MM^{-1}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$
$\displaystyle=$ $\displaystyle
V^{-1/2}P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$

(a) By definition,
$M^{-1}V(M^{\prime})^{-1}=:\Sigma=\Sigma^{1/2^{\prime}}\Sigma^{1/2}$, so that
$\Sigma^{-1/2}:=(\Sigma^{1/2})^{-1}=[V^{1/2}(M^{\prime})^{-1}]^{-1}$ and
$\Sigma^{-1/2^{\prime}}:=(\Sigma^{1/2^{\prime}})^{-1}=[M^{-1}V^{1/2^{\prime}}]^{-1}$.
(b) By standard property of inverses, $[V^{1/2}(M^{\prime})^{-1}]^{-1}$ and
$[M^{-1}V^{1/2^{\prime}}]^{-1}=V^{-1/2^{\prime}}M$.

Thus, under Assumptions 1, 2 and 3, if the test hypothesis (9) on p. 9 holds,
by Proposition 2, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,

$\displaystyle
T\tau_{T}(\check{\theta}_{T})^{\prime}\widehat{V}\tau_{T}(\check{\theta}_{T})$
$\displaystyle=$
$\displaystyle\left[V^{-1/2}P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)\right]^{\prime}\widehat{V}_{T}\left[V^{-1/2}P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{=}}$
$\displaystyle\left[V^{-1/2}P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]^{\prime}\widehat{V}_{T}\left[V^{-1/2}P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]$
$\displaystyle
2\left[V^{-1/2}P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]^{\prime}\widehat{V}_{T}\left[o_{\mathbb{P}}(1)\right]+\left[o_{\mathbb{P}}(1)\right]^{\prime}\widehat{V}\left[o_{\mathbb{P}}(1)\right]$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{=}}$
$\displaystyle\left[P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]^{\prime}V^{-1/2^{\prime}}\widehat{V}_{T}V^{-1/2}\left[P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]+o_{\mathbb{P}}(1)$
$\displaystyle\stackrel{{\scriptstyle(c)}}{{=}}$
$\displaystyle\left[P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]^{\prime}\left[P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]$
$\displaystyle\left[P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]^{\prime}\left(V^{-1/2^{\prime}}\widehat{V}_{T}V^{-1/2}-I\right)\left[P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]+o_{\mathbb{P}}(1)$
$\displaystyle\stackrel{{\scriptstyle(d)}}{{=}}$
$\displaystyle\left[P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]^{\prime}\left[P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]+o_{\mathbb{P}}(1)$
$\displaystyle\underset{(d)}{\stackrel{{\scriptstyle D}}{{\rightarrow}}}$
$\displaystyle\chi^{2}_{q}$

(a) Use the bilinearity and symmetry of the quadratic form defined by the
matrix $\widehat{V}_{T}$, which is symmetric by the theorem’s assumption. (b)
Firstly, by theorem’s assumption, as $T\rightarrow\infty$,
$\widehat{V}_{T}\stackrel{{\scriptstyle\mathbb{P}}}{{\rightarrow}}V$, so that
$\widehat{V}_{T}=O_{\mathbb{P}}(1)$. Secondly, under Assumption 1(a)-(c) and
(g), by the Lindeberg-Lévy CLT theorem, as $T\rightarrow\infty$,
$\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\stackrel{{\scriptstyle
D}}{{\rightarrow}}\mathcal{N}(0,V)$ where
$V:=\mathbb{E}[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]$, so
that, as $T\rightarrow\infty$,
$\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})=O_{\mathbb{P}}(1)$.
Thus, the second and third terms are $o_{\mathbb{P}}(1)$. (c) Add and subtract
$\left[P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]^{\prime}\left[P_{\Sigma^{1/2}R^{\prime}}V^{-1/2^{\prime}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\right]$.
(d) Denoting the convergence in probability with
$\stackrel{{\scriptstyle\mathbb{P}}}{{\rightarrow}}$, by the present theorem
assumption, as $T\rightarrow\infty$,
$\widehat{V}_{T}\stackrel{{\scriptstyle\mathbb{P}}}{{\rightarrow}}V$, where
$V$ is a positive definite symmetric matrix by Assumption 1(h). Thus, by Lemma
31 (p. 31), w.p.a.1 as $T\rightarrow\infty$, $\hat{V}_{T}$ is p-d.m, so that
it has a square root s.t.
$\hat{V}_{T}=\hat{V}_{T}^{1/2^{\prime}}\hat{V}_{T}^{1/2}$, where
$\hat{V}_{T}^{1/2}\stackrel{{\scriptstyle\mathbb{P}}}{{\rightarrow}}V^{1/2}$.(d)
Under Assumption 1(a)-(c) and (g), by the Lindeberg-Lévy CLT theorem, as
$T\rightarrow\infty$,
$\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\stackrel{{\scriptstyle
D}}{{\rightarrow}}\mathcal{N}(0,V)$ where
$V:=\mathbb{E}[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]$.
Moreover, the orthogonal projection matrix $P_{\Sigma^{1/2}R^{\prime}}$ has
rank $q$ because $R$ is of rank $q$ and $\Sigma$ has full rank by Assumptions
3(b) and 1(h), respectively. Thus, the result follows from the Cochran’s
theorem. ∎

######  Lemma 24 (Asymptotic expansions for $\mathrm{ALR}_{T}$).

Under Assumptions 1,2 and 3, if the test hypothesis (9) on p. 9 holds,
$\mathbb{P}$-a.s. as $T\rightarrow\infty$,

  * (i)

$\displaystyle
2\\{\ln[\hat{f}_{\theta^{*}_{T}}(\hat{\theta}_{T})]-\ln[\hat{f}_{\theta^{*}_{T}}(\check{\theta}_{T})]\\}=T(\hat{\theta}_{T}-\check{\theta}_{T})^{\prime}\Sigma^{-1}(\hat{\theta}_{T}-\check{\theta}_{T})+o_{\mathbb{P}}(1)$;

  * (ii)

$\sqrt{T}(\hat{\theta}_{T}-\check{\theta}_{T})=\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$

where $\Sigma:=\Sigma(\theta_{0}):=M^{-1}V(M^{\prime})^{-1}$,
$M:=\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]$,
$V:=\mathbb{E}[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]$, and
$R:=\frac{\partial r(\theta_{0})}{\partial\theta^{\prime}}$.

###### Proof.

(i) Under Assumption 1, by Lemma 12 (p. 12), $\mathbb{P}$-a.s. for $T$ big
enough, for all $(\theta,\tau)$ in a neighborhood of
$(\theta_{0},\tau(\theta_{0}))$, $L_{T}(\theta,\tau)$ exists. Moreover, under
Assumptions 1, 2 and 3(a), if the test hypothesis (9) on p. 9 holds, by
Theorem 1i (p. 1), Lemma 28i (p. 28) and Lemma 2iii (p. 2),
$\hat{\theta}_{T}\rightarrow\theta_{0}$,
$\check{\theta}_{T}\rightarrow\theta_{0}$,
$\tau_{T}(\hat{\theta}_{T})\rightarrow\tau(\theta_{0})$ and
$\tau_{T}(\check{\theta}_{T})\rightarrow\tau(\theta_{0})$, $\mathbb{P}$-a.s.
as $T\rightarrow\infty$. Thus, noting that
$\ln[\hat{f}_{\theta^{*}_{T}}(\theta)]=L_{T}(\theta,\tau_{T}(\theta))$,under
Assumptions 1, 2 and 3(a), if the test hypothesis (9) on p. 9 holds,
$\mathbb{P}$-a.s. for $T$ big enough,

$\displaystyle
2\\{\ln[\hat{f}_{\theta^{*}_{T}}(\hat{\theta}_{T})]-\ln[\hat{f}_{\theta^{*}_{T}}(\check{\theta}_{T})]\\}=-2T[L_{T}(\check{\theta}_{T},\tau_{T}(\check{\theta}_{T}))-L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))].$

Now, under Assumptions 1 and 2(a), by subsection B.2 (p. B.2), $L_{T}(.,.)$ is
twice continuously differentiable in a neighborhood of
$(\theta_{0}^{\prime}\;\tau(\theta_{0})^{\prime})$ $\mathbb{P}$-a.s. for $T$
big enough, so that a stochastic second-order Taylor-Lagrange expansion (e.g.,
Aliprantis and Border 2006/1999, Theorem 18.18) around
$(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))$ and evaluated
$(\check{\theta}_{T},\tau_{T}(\check{\theta}_{T}))$ yields, $\mathbb{P}$-a.s.
for $T$ big enough,

$\displaystyle
L_{T}(\check{\theta}_{T},\tau_{T}(\check{\theta}_{T}))=L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))+\begin{bmatrix}\frac{\partial
L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta^{\prime}}&\frac{\partial
L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\tau^{\prime}}\\\
\end{bmatrix}\begin{bmatrix}\check{\theta}_{T}-\hat{\theta}_{T}\\\
\tau_{T}(\check{\theta}_{T})-\tau_{T}(\hat{\theta}_{T})\\\ \end{bmatrix}$
$\displaystyle\
+\frac{1}{2}\begin{bmatrix}(\check{\theta}_{T}-\hat{\theta}_{T})^{\prime}&(\tau_{T}(\check{\theta}_{T})-\tau_{T}(\hat{\theta}_{T}))^{\prime}\\\
\end{bmatrix}\left[\begin{array}[]{c c
}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}\\\
\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\tau}\end{array}\right]\begin{bmatrix}\check{\theta}_{T}-\hat{\theta}_{T}\\\
\tau_{T}(\check{\theta}_{T})-\tau_{T}(\hat{\theta}_{T})\\\ \end{bmatrix}$
where $(\bar{\theta}_{T},\bar{\tau}_{T})$ is between
$(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))$ and
$(\check{\theta}_{T},\tau(\check{\theta}_{T}))$ ; $\displaystyle\Rightarrow$
$\displaystyle
L_{T}(\check{\theta}_{T},\tau_{T}(\check{\theta}_{T}))-L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))$
$\displaystyle=\frac{\partial
L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta^{\prime}}(\check{\theta}_{T}-\hat{\theta}_{T})+\frac{\partial
L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\tau^{\prime}}(\tau_{T}(\check{\theta}_{T})-\tau_{T}(\hat{\theta}_{T}))$
$\displaystyle+\frac{1}{2}(\check{\theta}_{T}-\hat{\theta}_{T})^{\prime}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}(\check{\theta}_{T}-\hat{\theta}_{T})+\frac{1}{2}(\tau_{T}(\check{\theta}_{T})-\tau_{T}(\hat{\theta}_{T}))^{\prime}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\tau}(\tau_{T}(\check{\theta}_{T})-\tau_{T}(\hat{\theta}_{T}))$
$\displaystyle+(\check{\theta}_{T}-\hat{\theta}_{T})^{\prime}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}(\tau_{T}(\check{\theta}_{T})-\tau_{T}(\hat{\theta}_{T})),$

where

  * •

Under Assumptions 1, 2 and 3, if the test hypothesis (9) on p. 9 holds, by
Theorem 1ii (p. 1) and Proposition 2iii (p. 2), $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,
$\check{\theta}_{T}-\hat{\theta}_{T}=(\check{\theta}_{T}-\theta_{0})-(\hat{\theta}_{T}-\theta_{0})=O_{\mathbb{P}}(T^{-\frac{1}{2}})+O_{\mathbb{P}}(T^{-\frac{1}{2}})=O_{\mathbb{P}}(T^{-\frac{1}{2}})$;

  * •

Under Assumptions 1 and 2, by Lemma 20 (p. 20), $\mathbb{P}$-a.s. as
$T\rightarrow\infty$, $\frac{\partial
L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta^{\prime}}=O(T^{-1})$,
so that $\frac{\partial
L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta^{\prime}}(\check{\theta}_{T}-\hat{\theta}_{T})=O_{\mathbb{P}}(T^{-\frac{3}{2}})$
by the first bullet point;

  * •

Under Assumptions 1, 2 and 3, if the test hypothesis (9) on p. 9 holds, by
Lemma 21iv (p. 21) and Theorem 1i (p. 1) and Lemma 28i (p. 28),
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, there exists $\tilde{\theta}_{T}$
between $\hat{\theta}_{T}$ and $\theta_{0}$ s.t.
$\sqrt{T}[\tau_{T}(\check{\theta}_{T})-\tau_{T}(\hat{\theta}_{T})]=\sqrt{T}[\tau_{T}(\check{\theta}_{T})-\tau_{T}(\theta_{0})]-\sqrt{T}[\tau_{T}(\hat{\theta}_{T})-\tau_{T}(\theta_{0})]=-V^{-1}M(\check{\theta}_{T}-\theta_{0})+o_{\mathbb{P}}(1)-\left[-V^{-1}M(\hat{\theta}_{T}-\theta_{0})+o_{\mathbb{P}}(T^{-1/2})\right]=-V^{-1}M(\check{\theta}_{T}-\theta_{0})+V^{-1}M(\hat{\theta}_{T}-\theta_{0})+o_{\mathbb{P}}(1)=-V^{-1}M(\check{\theta}_{T}-\hat{\theta}_{T})+o_{\mathbb{P}}(1)$;

  * •

Under Assumptions 1 and 2, by Lemma 22iv (p. 22), $\mathbb{P}$-a.s. as
$T\rightarrow\infty$, $\frac{\partial
L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\tau^{\prime}}=O(T^{-1})$,
so that, by the first and third bullet point, $\frac{\partial
L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\tau^{\prime}}(\tau_{T}(\check{\theta}_{T})-\tau_{T}(\hat{\theta}_{T}))=O_{\mathbb{P}}(T^{-3/2})$,
under Assumptions 1, 2 and 3, if the test hypothesis (9) on p. 9 holds;

  * •

Under Assumptions 1, 2 and 3, if the test hypothesis (9) on p. 9 holds, by
Lemma 21iv (p. 21) and Theorem 1i (p. 1) and Lemma 28i (p. 28),
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, there exists $\tilde{\theta}_{T}$
between $\hat{\theta}_{T}$ and $\theta_{0}$ s.t.
$\sqrt{T}[\tau_{T}(\check{\theta}_{T})-\tau_{T}(\hat{\theta}_{T})]=\sqrt{T}[\tau_{T}(\check{\theta}_{T})-\tau_{T}(\theta_{0})]-\sqrt{T}[\tau_{T}(\hat{\theta}_{T})-\tau_{T}(\theta_{0})]=-V^{-1}M(\check{\theta}_{T}-\theta_{0})+o_{\mathbb{P}}(1)-\left[-V^{-1}M(\hat{\theta}_{T}-\theta_{0})+o_{\mathbb{P}}(T^{-1/2})\right]=-V^{-1}M(\check{\theta}_{T}-\theta_{0})+V^{-1}M(\hat{\theta}_{T}-\theta_{0})+o_{\mathbb{P}}(1)=-V^{-1}M(\check{\theta}_{T}-\hat{\theta}_{T})+o_{\mathbb{P}}(1)$;
and

  * •

under Assumptions 1 and 2, by Lemma 14ii (p. 14), $\mathbb{P}$-a.s. as
$T\rightarrow\infty$
$\displaystyle\left|\frac{\partial^{2}L_{T}(\theta_{T},\tau_{T})}{\partial\theta_{j}\partial\theta_{\ell}}\right|=o(1)$,
so that, by Theorem 1ii (p. 1),
$(\check{\theta}_{T}-\hat{\theta}_{T})^{\prime}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}(\check{\theta}_{T}-\hat{\theta}_{T})=o_{\mathbb{P}}(T^{-1})$
by the first bullet point.

Therefore, Assumptions 1, 2 and 3, if the test hypothesis (9) on p. 9 holds,

$\displaystyle
2\\{\ln[\hat{f}_{\theta^{*}_{T}}(\hat{\theta}_{T})]-\ln[\hat{f}_{\theta^{*}_{T}}(\check{\theta}_{T})]\\}=-2T\left[L_{T}(\check{\theta}_{T},\tau_{T}(\check{\theta}_{T}))-L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))\right]$
$\displaystyle=$
$\displaystyle-\left[V^{-1}M\sqrt{T}(\check{\theta}_{T}-\hat{\theta}_{T})\right]^{\prime}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\tau}\left[V^{-1}M\sqrt{T}(\check{\theta}_{T}-\hat{\theta}_{T})\right]$
$\displaystyle+2\sqrt{T}(\theta_{0}-\hat{\theta}_{T})^{\prime}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}\left[V^{-1}M\sqrt{T}(\theta_{0}-\hat{\theta}_{T})\right]+o_{\mathbb{P}}(1)$
$\displaystyle=$
$\displaystyle-\sqrt{T}(\check{\theta}_{T}-\hat{\theta}_{T})^{\prime}\left[M^{\prime}V^{-1}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\tau}V^{-1}M-2\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}V^{-1}M\right]\sqrt{T}(\check{\theta}_{T}-\hat{\theta}_{T})+o_{\mathbb{P}}(1)$
$\displaystyle=$
$\displaystyle\sqrt{T}(\check{\theta}_{T}-\hat{\theta}_{T})^{\prime}\Sigma(\theta_{0})^{-1}\sqrt{T}(\check{\theta}_{T}-\hat{\theta}_{T})+o_{\mathbb{P}}(1),$

where the explanations for the convergence are as follow. Firstly, under
Assumptions 1 and 2, by Lemma 14ii (p. 14), for any sequence
$(\theta_{T},\tau_{T})_{T\in\mathbf{N}}$ converging to
$(\theta_{0},\tau(\theta_{0}))$, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$\frac{\partial^{2}L_{T}(\theta_{T},\tau_{T})}{\partial\theta^{\prime}\partial\tau}\rightarrow\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]=:M$.
Secondly, under Assumptions 1 and 2, by Lemma 25iv (p. 25), $\mathbb{P}$-a.s.
as $T\rightarrow\infty$, $\frac{\partial
L_{T}(\theta_{T},\tau_{T})}{\partial\tau\partial\tau^{\prime}}\rightarrow\mathbb{E}[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]=:V$.
Therefore, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,

$\displaystyle\
M^{\prime}(V^{\prime})^{-1}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\tau}V^{-1}M-2\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}V^{-1}M$
$\displaystyle\rightarrow$ $\displaystyle
M^{\prime}(V^{\prime})^{-1}VV^{-1}M-2M^{\prime}V^{-1}M$ $\displaystyle=$
$\displaystyle-M^{\prime}V^{-1}M=-\Sigma(\theta_{0})^{-1}.$

(ii) Under Assumptions 1, 2 and 3, if the test hypothesis (9) on p. 9 holds,
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, addition and subtraction of
$\sqrt{T}\theta_{0}$ yield

$\displaystyle\sqrt{T}(\hat{\theta}_{T}-\check{\theta}_{T})$ $\displaystyle=$
$\displaystyle\sqrt{T}(\hat{\theta}_{T}-\theta_{0})-\sqrt{T}(\check{\theta}_{T}-\theta_{0})$
$\displaystyle\stackrel{{\scriptstyle}}{{=}}$
$\displaystyle-M^{-1}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)-\left[M^{-1}-\Sigma
R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)\right]$
$\displaystyle\stackrel{{\scriptstyle}}{{=}}$ $\displaystyle\Sigma
R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$

where the explanations for the second equality are the following. Firstly,
under Assumptions 1 and 2, by Proposition 1 (p. 1), $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,
$\sqrt{T}(\hat{\theta}_{T}-\theta_{0})=-M\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$.
Secondly, under Assumptions 1, 2 and 3, by Proposition 2i (p. 2), if the test
hypothesis (9) on p. 9 holds, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$\sqrt{T}(\check{\theta}_{T}-\theta_{0})=M^{-1}-\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$.
∎

######  Lemma 25 (Asymptotic limit of
$\frac{\partial^{2}L_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\tau^{\prime}\partial\tau}$).

Under Assumptions 1 and 2, for any sequence
$(\theta_{T},\tau_{T})_{T\in\mathbf{N}}$ converging to
$(\theta_{0},\tau(\theta_{0}))$, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
for all $(h,k)\in[\\![1,m]\\!]^{2}$, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,

  1. (i)

$\frac{\partial^{2}M_{1,T}(\theta_{T},\tau_{T})}{\partial\tau_{h}\partial\tau_{k}}\rightarrow\mathbb{E}[\psi_{k}(X_{1},\theta_{0})\psi_{h}(X_{1},\theta_{0})]$;

  2. (ii)

$\frac{\partial^{2}M_{2,T}(\theta_{T},\tau_{T})}{\partial\tau_{h}\partial\tau_{k}}=O(T^{-1})$;

  3. (iii)

$\frac{\partial^{2}M_{3,T}(\theta_{T},\tau_{T})}{\partial\tau_{h}\partial\tau_{k}}=O(T^{-1})$;
and

  4. (iv)

$\frac{\partial^{2}L_{T}(\theta_{T},\tau_{T})}{\partial\tau\partial\tau^{\prime}}\rightarrow\mathbb{E}[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]$.

###### Proof.

(i) By equation (29) on p. 29, for all $(h,k)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\frac{\partial^{2}M_{1,T}(\theta_{T},\tau_{T})}{\partial\tau_{h}\partial\tau_{k}}$
$\displaystyle=$
$\displaystyle\negthickspace\left(1-\frac{m}{2T}\right)\frac{1}{\left[\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{i}(\theta_{T})}\right]^{2}}\left\\{\negthickspace\left[\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{i}(\theta_{T})}\right]\negthickspace\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\psi_{t,h}(\theta_{T})\psi_{t,k}(\theta_{T})\right]\right.$
$\displaystyle\left.\negthickspace-\negthickspace\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\psi_{t,h}(\theta_{T})\right]\left[\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{i}(\theta_{T})}\psi_{i,k}(\theta_{T})\right]\negthickspace\right\\}.$

where, as $T\rightarrow\infty$,
$(\theta_{T}\quad\tau_{T})\rightarrow(\theta_{0}\quad\tau(\theta_{0}))$ by
assumption. Now, under Assumptions 1 and 2, by Lemma 23i-iii (p. 23), for
$\overline{B_{L}}$ a ball around $(\theta_{0},\tau(\theta_{0}))$ of
sufficiently small radius,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\right]<\infty$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)|\right]<\infty$,
and
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)|\right]$.
Thus, by Assumption 1(a)(b) and (d), the ULLN (uniform law of large numbers) à
la Wald (e.g., Ghosh and Ramamoorthi 2003, pp. 24-25, Theorem 1.3.3), implies
that, for all $(h,k)\in[\\![1,m]\\!]^{2}$, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,

$\displaystyle\frac{\partial^{2}M_{1,T}(\theta_{T},\tau_{T})}{\partial\tau_{k}\partial\tau_{h}}$
$\displaystyle\rightarrow$
$\displaystyle\frac{1}{\mathbb{E}[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1},\theta_{0})}]^{2}}\left\\{\mathbb{E}[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1},\theta_{0})}]\mathbb{E}[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1},\theta_{0})}\psi_{k}(X_{1},\theta_{0})\psi_{h}(X_{1},\theta_{0})]\right.$
$\displaystyle-\left.\mathbb{E}[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1},\theta_{0})}\psi_{h}(X_{1},\theta_{0})]\mathbb{E}[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1},\theta_{0})}\psi_{k}(X_{1},\theta_{0})]\right\\}$
$\displaystyle=$
$\displaystyle\mathbb{E}[\psi_{k}(X_{1},\theta_{0})\psi_{h}(X_{1},\theta_{0})]$

because $\mathbb{E}[\psi(X_{1},\theta_{0})]=0_{m\times 1}$ by Assumption 1(c),
and $\tau(\theta_{0})=0_{m\times 1}$ by Lemma 10iv (p. 10) under Assumption
1(a)-(e) and (g)-(h).

(ii) Under Assumptions 1, by equation (35) on p. 35, $\mathbb{P}$-a.s. for $T$
big enough, for all $(h,k)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\frac{\partial^{2}M_{2,T}(\theta_{T},\tau_{T})}{\partial\tau_{h}\partial\tau_{k}}$
$\displaystyle=-\frac{1}{T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\frac{\partial\psi_{t}(\theta_{T})}{\partial\theta^{\prime}}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\psi_{t,k}(\theta_{T})\frac{\partial\psi_{t}(\theta_{T})}{\partial\theta^{\prime}}\right]\right.$
$\displaystyle\hskip
36.135pt\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\frac{\partial\psi_{t}(\theta_{T})}{\partial\theta^{\prime}}\right]^{-1}\left.\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\psi_{t,h}(\theta_{T})\frac{\partial\psi_{t}(\theta_{T})}{\partial\theta^{\prime}}\right]\right\\}$
$\displaystyle+\frac{1}{T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\frac{\partial\psi_{t}(\theta_{T})}{\partial\theta^{\prime}}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\psi_{t,k}(\theta_{T})\psi_{t,h}(\theta_{T})\frac{\partial\psi_{t}(\theta_{T})}{\partial\theta^{\prime}}\right]\right\\}.\
$

where, as $T\rightarrow\infty$,
$(\theta_{T}\quad\tau_{T})\rightarrow(\theta_{0}\quad\tau(\theta_{0}))$ by
assumption. Now, under Assumptions 1 and 2, by Lemma 23iv-vi (p. 23), for
$\overline{B_{L}}$ a ball around $(\theta_{0},\tau(\theta_{0}))$ of
sufficiently small radius,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]$,
and
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\psi_{h}(X_{1},\theta)\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]$.
Thus, by Assumptions 1(a)(b) and (d), the ULLN (uniform law of large numbers)
à la Wald (e.g., Ghosh and Ramamoorthi 2003, pp. 24-25, Theorem 1.3.3),
implies that, for all $(h,k)\in[\\![1,m]\\!]^{2}$, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,

$\displaystyle
T\frac{\partial^{2}M_{2,T}(\theta_{T},\tau_{T})}{\partial\tau_{h}\partial\tau_{k}}$
$\displaystyle\rightarrow$ $\displaystyle{\rm
tr}\left\\{\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi_{k}(X_{1},\theta_{0})\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]\right.$
$\displaystyle\left.\times\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi_{h}(X_{1},\theta_{0})\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]\right\\}$
$\displaystyle+{\rm
tr}\left\\{\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi_{k}(X_{1},\theta_{0})\psi_{h}(X_{1},\theta_{0})\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]\right\\}$
$\displaystyle=$ $\displaystyle{\rm
tr}\left\\{\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\mathbb{E}\left[\psi_{k}(X_{1},\theta_{0})\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]\right.$
$\displaystyle\left.\times\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\mathbb{E}\left[\psi_{h}(X_{1},\theta_{0})\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]\right\\}$
$\displaystyle+{\rm
tr}\left\\{\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\mathbb{E}\left[\psi_{k}(X_{1},\theta_{0})\psi_{h}(X_{1},\theta_{0})\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]\right\\}$

because $\tau(\theta_{0})=0_{m\times 1}$ by Lemma 10iv (p. 10) under
Assumption 1(a)-(e) and (g)-(h). Therefore, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,
$\frac{\partial^{2}M_{2,T}(\theta_{T},\tau_{T})}{\partial\tau_{h}\partial\tau_{k}}=O(T^{-1})$.

(iii) Under Assumptions 1(a)(b)(e)(g)(h), by equation (40) (p. 40), for all
$(h,k)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\frac{\partial^{2}M_{3,T}(\theta_{T},\tau_{T})}{\partial\tau_{h}\partial\tau_{k}}$
$\displaystyle=$ $\displaystyle\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\psi_{t}(\theta_{T})\psi_{t}(\theta_{T})^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\psi_{t,k}(\theta_{T})\psi_{t}(\theta_{T})\psi_{t}(\theta_{T})^{\prime}\right]\right.$
$\displaystyle\hskip
36.135pt\left.\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\psi_{t}(\theta_{T})\psi_{t}(\theta_{T})^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\psi_{t,h}(\theta_{T})\psi_{t}(\theta_{T})\psi_{t}(\theta_{T})^{\prime}\right]\right\\}$
$\displaystyle-\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\psi_{t}(\theta_{T})\psi_{t}(\theta_{T})^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau_{T}^{\prime}\psi_{t}(\theta_{T})}\psi_{t,k}(\theta_{T})\psi_{t,h}(\theta_{T})\psi_{t}(\theta_{T})\psi_{t}(\theta_{T})^{\prime}\right]\right\\}$

where, as $T\rightarrow\infty$,
$(\theta_{T}\quad\tau_{T})\rightarrow(\theta_{0}\quad\tau(\theta_{0}))$ by
assumption. Now, under Assumptions 1 and 2, by Lemma 23vii-ix (p. 23), there
exists a closed ball $\overline{B_{L}}\subset\mathbf{S}$ centered at
$(\theta_{0},\tau(\theta_{0}))$ with strictly positive radius s.t., for all
$k\in[\\![1,m]\\!]$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{k}(X_{1},\theta)\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$,
and
$\mathbb{E}[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\psi_{h}(X_{1},\theta)\\\
\psi_{k}(X_{1},\theta)\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|]<\infty$.
Thus, by Assumptions 1(a)(b) and (d), the ULLN (uniform law of large numbers)
à la Wald (e.g., Ghosh and Ramamoorthi 2003, pp. 24-25, Theorem 1.3.3),
implies that, for all $(h,k)\in[\\![1,m]\\!]^{2}$, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,

$\displaystyle
T\frac{\partial^{2}M_{3,T}(\theta_{T},\tau_{T})}{\partial\tau_{h}\partial\tau_{k}}$
$\displaystyle\rightarrow$ $\displaystyle\frac{1}{2}{\rm
tr}\left\\{\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]^{-1}\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi_{k}(X_{1},\theta_{0})\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\right.$
$\displaystyle\left.\times\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]^{-1}\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi_{h}(X_{1},\theta_{0})\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\right\\}$
$\displaystyle-\frac{1}{2}{\rm
tr}\left\\{\mathbb{E}\negthickspace\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]^{-1}\right.$
$\displaystyle\hskip 130.08621pt\left.\times
E\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1,}\theta_{0})}\psi_{k}(X_{1},\theta_{0})\psi_{h}(X_{1},\theta_{0})\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\right\\}$
$\displaystyle=$ $\displaystyle\frac{1}{2}{\rm
tr}\left\\{\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]^{-1}\mathbb{E}\left[\psi_{k}(X_{1},\theta_{0})\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\right.$
$\displaystyle\left.\times\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]^{-1}\mathbb{E}\left[\psi_{h}(X_{1},\theta_{0})\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\right\\}$
$\displaystyle-\frac{1}{2}{\rm
tr}\left\\{\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]^{-1}\mathbb{E}\left[\psi_{k}(X_{1},\theta_{0})\psi_{h}(X_{1},\theta_{0})\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\right\\}$

because $\tau(\theta_{0})=0_{m\times 1}$ by Lemma 10iv (p. 10) under
Assumption 1(a)-(e) and (g)-(h). Therefore, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$, $\frac{\partial
M_{3,T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\tau_{k}}=O(T^{-1})$.

(iv) Under Assumption 1(a)-(b) and (d)-(h), by Lemma 12 (p. 12), for all
$(\theta,\tau)$ in a neighborhood of $(\theta_{0},\tau(\theta_{0}))$,
$L_{T}(\theta,\tau)=M_{1,T}(\theta,\tau)+M_{2,T}(\theta,\tau)+M_{3,T}(\theta,\tau)$,
so that the result follows from the statement (i)-(iii) of the present lemma.
∎

######  Proposition 2 (Asymptotic normality of $\check{\theta}_{T}$,
$\tau_{T}(\check{\theta}_{T})$ and $\check{\gamma}_{T}$).

Under Assumptions 1, 2 and 3, if the test hypothesis (9) on p. 9 holds,
$\mathbb{P}$-a.s. as $T\rightarrow\infty$,

  1. (i)

$\displaystyle\sqrt{T}\begin{bmatrix}\check{\theta}_{T}-\theta_{0}\\\
\tau_{T}(\check{\theta}_{T})\\\ \check{\gamma}_{T}\\\
\end{bmatrix}=\begin{bmatrix}\quad M^{-1}-\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\\\ (M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\\\ -(R\Sigma R^{\prime})^{-1}RM^{-1}\\\
\end{bmatrix}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$;
and

  2. (ii)

$\displaystyle\sqrt{T}\begin{bmatrix}\check{\theta}_{T}-\theta_{0}\\\
\tau_{T}(\check{\theta}_{T})\\\ \check{\gamma}_{T}\\\
\end{bmatrix}=\begin{bmatrix}\quad\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{-1/2^{\prime}}M^{-1}\\\
(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}&\\\
-(R\Sigma R^{\prime})^{-1}RM^{-1}\\\
\end{bmatrix}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$

  3. (iii)

$\displaystyle\sqrt{T}\begin{bmatrix}\check{\theta}_{T}-\theta_{0}\\\
\tau_{T}(\check{\theta}_{T})\\\ \check{\gamma}_{T}\\\
\end{bmatrix}\stackrel{{\scriptstyle
D}}{{\rightarrow}}\mathcal{N}\left(0,\begin{bmatrix}(\Sigma^{1/2})^{\prime}P^{\perp}_{\Sigma^{1/2}R^{\prime}}\Sigma^{1/2}&0_{m\times
m}&0_{m\times q}\\\ 0_{m\times
m}&(V^{1/2})^{-1}P_{\Sigma^{1/2}R^{\prime}}(V^{1/2^{\prime}})^{-1}&-(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\ 0_{q\times m}&-(R\Sigma R^{\prime})^{-1}RM^{-1}&(R\Sigma
R^{\prime})^{-1}\\\ \end{bmatrix}\right)$,

where $\Sigma:=\Sigma(\theta_{0}):=M^{-1}V(M^{\prime})^{-1}$,
$M:=\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]$,
$V:=\mathbb{E}[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]$, and
$R:=\frac{\partial r(\theta_{0})}{\partial\theta^{\prime}}$.

###### Proof.

(i)-(ii) The function $L_{T}(\theta,\tau)$ is well-defined and twice
continuously differentiable in a neighborhood of
$(\theta_{0}^{\prime}\;\tau(\theta_{0})^{\prime})$ $\mathbb{P}$-a.s. for $T$
big enough by subsection B.2 (p. B.2), under Assumptions 1 and 2(a).
Similarly, the function
$S_{T}(\theta,\tau):=\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)$
and $\theta\mapsto r(\theta)$ are continuously differentiable in a
neighborhood of $(\theta_{0}^{\prime}\;\tau(\theta_{0})^{\prime})$ by
Assumption 1(a)(b) and 3(a). Now, under Assumptions 1, 2, and 3(a), by Lemma
28i (p. 28), Lemma 2iii (p. 2), $\mathbb{P}$-a.s.,
$\check{\theta}_{T}\rightarrow\theta_{0}$ and
$\tau_{T}(\check{\theta}_{T})\rightarrow\tau(\theta_{0})$, so that
$\mathbb{P}$-a.s. for $T$ big enough,
$(\check{\theta}_{T}^{\prime}\;\tau_{T}(\check{\theta}_{T})^{\prime})$ is in
any arbitrary small neighborhood of
$(\theta_{0}^{\prime}\;\tau(\theta_{0})^{\prime})$. Therefore, under
Assumptions 1, 2 and 3 (a), stochastic first-order Taylor-Lagrange expansions
(Jennrich 1969, Lemma 3) around $(\theta_{0},\tau(\theta_{0}))$ evaluated at
$(\check{\theta}_{T},\tau_{T}(\check{\theta}_{T}))$ yield, $\mathbb{P}$-a.s.
for $T$ big enough

$\displaystyle\frac{\partial{L}_{T}(\check{\theta}_{T},\tau_{T}(\check{\theta}_{T}))}{\partial\theta}$
$\displaystyle=$
$\displaystyle\frac{\partial{L}_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta}+\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}\left(\check{\theta}_{T}-\theta_{0}\right)+\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}\tau_{T}(\check{\theta}_{T})$
$\displaystyle S_{T}(\check{\theta}_{T},\tau_{T}(\check{\theta}_{T}))$
$\displaystyle=$ $\displaystyle
S_{T}(\theta_{0},\tau(\theta_{0}))+\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}\left(\check{\theta}_{T}-\theta_{0}\right)+\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}\tau_{T}(\check{\theta}_{T})$
$\displaystyle r(\check{\theta}_{T})$ $\displaystyle=$ $\displaystyle
r(\theta_{0})+\frac{\partial
r(\bar{\theta}_{T})}{\partial\theta^{\prime}}\left(\check{\theta}_{T}-\theta_{0}\right)$

because $\tau(\theta_{0})=0_{m\times 1}$ by Lemma 10iv (p. 10), and where
$\bar{\theta}_{T}$ and $\bar{\tau}_{T}$ are between $\check{\theta}_{T}$ and
$\theta_{0}$, and between $\tau_{T}(\check{\theta}_{T})$ and
$\tau(\theta_{0})$, respectively. Now, under Assumptions 1 and 2, by
definition of $\check{\theta}_{T}$ and by definition of $\tau_{T}(.)$
(equation 15 on p. 15), $r(\check{\theta}_{T})=0_{q\times 1}$ and
$S_{T}(\check{\theta}_{T},\tau_{T}(\check{\theta}_{T}))=0$, respectively.
Moreover, under Assumptions 1, 2 and 3, by Lemma 28iv (p. 28),
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, $\frac{\partial
L_{T}(\check{\theta}_{T},\tau_{T}(\check{\theta}_{T}))}{\partial\theta}=-\frac{\partial
r(\check{\theta}_{T})^{\prime}}{\partial\theta}\check{\gamma}_{T}+O(T^{-1})$.
Therefore, under Assumptions 1, 2 and 3, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,

$\displaystyle\begin{cases}O(T^{-1})=\frac{\partial{L}_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta}+\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}\left(\check{\theta}_{T}-\theta_{0}\right)+\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}\tau_{T}(\check{\theta}_{T})+\frac{\partial
r(\check{\theta}_{T})^{\prime}}{\partial\theta}\check{\gamma}_{T}\\\
0_{m\times 1}=S_{T}(\theta_{0},\tau(\theta_{0}))+\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}\left(\check{\theta}_{T}-\theta_{0}\right)+\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}\tau_{T}(\check{\theta}_{T})\\\
0_{q\times 1}=r(\theta_{0})+\frac{\partial
r(\bar{\theta}_{T})}{\partial\theta^{\prime}}\left(\check{\theta}_{T}-\theta_{0}\right)\end{cases},$

which in matrix form is

$\displaystyle\begin{bmatrix}O(T^{-1})\\\ 0_{m\times 1}\\\ 0_{q\times 1}\\\
\end{bmatrix}=\begin{bmatrix}\frac{\partial{L}_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta}\\\
S_{T}(\theta_{0},\tau(\theta_{0}))\\\ r(\theta_{0})\\\
\end{bmatrix}+\begin{bmatrix}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}&\frac{\partial
r(\check{\theta}_{T})^{\prime}}{\partial\theta^{\prime}}\\\ \frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}&0\\\
\frac{\partial r(\bar{\theta}_{T})}{\partial\theta^{\prime}}&0&0\\\
\end{bmatrix}\begin{bmatrix}\check{\theta}_{T}-\theta_{0}\\\
\tau_{T}(\check{\theta}_{T})\\\ \check{\gamma}_{T}\\\ \end{bmatrix}.$

Now, under Assumptions 1, 2 and 3, by Lemma 26ii (p. 26), $\mathbb{P}$-a.s.
for $T$ big enough, the matrix
$\begin{bmatrix}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}&\frac{\partial
r(\check{\theta}_{T})^{\prime}}{\partial\theta^{\prime}}\\\ \frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}&0\\\
\frac{\partial r(\bar{\theta}_{T})}{\partial\theta^{\prime}}&0&0\\\
\end{bmatrix}$ is invertible. Then, under Assumptions 1, 2 and 3, solving for
the parameters and multiplying by $\sqrt{T}$ yield, $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,

$\displaystyle\sqrt{T}\begin{bmatrix}\check{\theta}_{T}-\theta_{0}\\\
\tau_{T}(\check{\theta}_{T})\\\ \check{\gamma}_{T}\\\ \end{bmatrix}$
$\displaystyle=$
$\displaystyle-\begin{bmatrix}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}&\frac{\partial
r(\check{\theta}_{T})^{\prime}}{\partial\theta}\\\ \frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}&0\\\
\frac{\partial r(\bar{\theta}_{T})}{\partial\theta^{\prime}}&0&0\\\
\end{bmatrix}^{-1}\sqrt{T}\begin{bmatrix}\frac{\partial{L}_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta}+O(T^{-1})\\\
S_{T}(\theta_{0},\tau(\theta_{0}))\\\ r(\theta_{0})\\\ \end{bmatrix}$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{=}}$
$\displaystyle-\begin{bmatrix}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}&\frac{\partial
r(\check{\theta}_{T})^{\prime}}{\partial\theta}\\\ \frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}&0\\\
\frac{\partial r(\bar{\theta}_{T})}{\partial\theta^{\prime}}&0&0\\\
\end{bmatrix}^{-1}\begin{bmatrix}O(T^{-\frac{1}{2}})\\\
\sqrt{T}\frac{1}{T}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\\\ 0\\\ \end{bmatrix}$
$\displaystyle\underset{}{\stackrel{{\scriptstyle(b)}}{{=}}}$
$\displaystyle\negthickspace-\negthickspace\begin{bmatrix}-\Sigma+\Sigma
R^{\prime}(R\Sigma R^{\prime})^{-1}R\Sigma&\quad
M^{-1}\negthickspace-\negthickspace\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}&\Sigma R^{\prime}(R\Sigma R^{\prime})^{-1}\\\
(M^{\prime})^{-1}\negthickspace-\negthickspace(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma&(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}&\;-(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\ (R\Sigma R^{\prime})^{-1}R\Sigma&-(R\Sigma
R^{\prime})^{-1}RM^{-1}&(R\Sigma R^{\prime})^{-1}\\\
\end{bmatrix}\negthickspace$ $\displaystyle\hskip
289.07999pt\times\begin{bmatrix}O(T^{-\frac{1}{2}})\\\
\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\\\ 0\\\ \end{bmatrix}$
$\displaystyle+\left\\{\begin{bmatrix}-\Sigma+\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma&\quad M^{-1}\negthickspace-\negthickspace\Sigma
R^{\prime}(R\Sigma R^{\prime})^{-1}RM^{-1}&\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\
(M^{\prime})^{-1}\negthickspace-\negthickspace(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma&(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}&\;-(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\ (R\Sigma R^{\prime})^{-1}R\Sigma&-(R\Sigma
R^{\prime})^{-1}RM^{-1}&(R\Sigma R^{\prime})^{-1}\\\ \end{bmatrix}\right.$
$\displaystyle-\left.\begin{bmatrix}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}&\frac{\partial
r(\check{\theta}_{T})^{\prime}}{\partial\theta}\\\ \frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}&0\\\
\frac{\partial r(\bar{\theta}_{T})}{\partial\theta^{\prime}}&0&0\\\
\end{bmatrix}^{-1}\right\\}\begin{bmatrix}O(T^{-\frac{1}{2}})\\\
\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\\\ 0\\\ \end{bmatrix}$
$\displaystyle\underset{}{\stackrel{{\scriptstyle(c)}}{{=}}}$
$\displaystyle\begin{bmatrix}\quad M^{-1}-\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\\\ (M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\\\ -(R\Sigma R^{\prime})^{-1}RM^{-1}\\\
\end{bmatrix}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$
$\displaystyle\underset{}{\stackrel{{\scriptstyle(d)}}{{=}}}$
$\displaystyle\begin{bmatrix}\quad\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{-1/2^{\prime}}M^{-1}\\\
(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}&\\\
-(R\Sigma R^{\prime})^{-1}RM^{-1}\\\
\end{bmatrix}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$

(a) Firstly, under Assumptions 1 and 2, by Lemma 14i (p. 14),
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, $\frac{\partial
L_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta_{j}}=O(T^{-1})$, so that
$\sqrt{T}\left[\frac{\partial
L_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta_{j}}+O(T^{-1})\right]=O(T^{-\frac{1}{2}})$.
Secondly, note that
$S_{T}(\theta_{0},\tau(\theta_{0}))=\frac{1}{T}\sum_{t=1}^{T}\psi_{t}(\theta_{0})$
because $\tau(\theta_{0})=0_{m\times 1}$ by Lemma 10iv (p. 10) under
Assumption 1(a)-(e) and (g)-(h). Finally, if the test hypothesis (9) on p. 9
holds, then $r(\theta_{0})=0_{q\times 1}$. (b) Add and subtract the matrix
$\begin{bmatrix}-\Sigma+\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma&\quad M^{-1}\negthickspace-\negthickspace\Sigma
R^{\prime}(R\Sigma R^{\prime})^{-1}RM^{-1}&\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\
(M^{\prime})^{-1}\negthickspace-\negthickspace(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma&(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}&\;-(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\ (R\Sigma R^{\prime})^{-1}R\Sigma&-(R\Sigma
R^{\prime})^{-1}RM^{-1}&(R\Sigma R^{\prime})^{-1}\\\ \end{bmatrix}$. (c)
Firstly, the first and third column of the first square matrix cancel out
because the first element and third element of the vector are zeros. Secondly,
under Assumptions 1, 2 and 3, by Lemma 26iii (p. 26) and Theorem 1i (p. 1),
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, the curly bracket is $o(1)$, and,
under Assumption 1(a)-(c) and (g), by the Lindeberg-Lévy CLT,
$\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})=O_{\mathbb{P}}(1)$, as
$T\rightarrow\infty$. (d) By definition
$\Sigma=\Sigma^{1/2^{\prime}}\Sigma^{1/2}$ and
$\Sigma^{-1/2^{\prime}}=[\Sigma^{1/2^{\prime}}]^{-1}$. Thus,

  * •

$M^{-1}-\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}=\Sigma^{1/2^{\prime}}[I-\Sigma^{1/2}R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma^{1/2^{\prime}}]\Sigma^{-1/2^{\prime}}M^{-1}$

$=\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{-1/2^{\prime}}M^{-1}$
where $P_{\Sigma^{1/2}R^{\prime}}^{\perp}$ denotes the orthogonal projection
on the orthogonal of the space spanned by the columns of
$\Sigma^{1/2}R^{\prime}$.

  * •

$(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}=(M^{\prime})^{-1}\Sigma^{-1/2}[\Sigma^{1/2}R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma^{1/2^{\prime}}]\Sigma^{-1/2^{\prime}}M^{-1}\\\
=(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}=(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}\Sigma\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}\\\
=(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}=(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}\\\
=(M^{\prime})^{-1}[V^{1/2}(M^{\prime})^{-1}]^{-1}P_{\Sigma^{1/2}R^{\prime}}[M^{-1}V^{1/2^{\prime}}]^{-1}M^{-1}=(V^{1/2})^{-1}P_{\Sigma^{1/2}R^{\prime}}(V^{1/2^{\prime}})^{-1}$
because $M^{-1}V(M^{\prime})^{-1}=:\Sigma=\Sigma^{1/2^{\prime}}\Sigma^{1/2}$,
so that
$\Sigma^{-1/2}:=(\Sigma^{1/2})^{-1}=[V^{1/2}(M^{\prime})^{-1}]^{-1}=M^{\prime}V^{-1/2}$
and
$\Sigma^{-1/2^{\prime}}:=(\Sigma^{1/2^{\prime}})^{-1}=[M^{-1}V^{1/2^{\prime}}]^{-1}=V^{-1/2^{\prime}}M$.

(iii) Under Assumptions 1, 2 and 3, by the statement (ii) of the present
proposition, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,

$\displaystyle\sqrt{T}\begin{bmatrix}\check{\theta}_{T}-\theta_{0}\\\
\tau_{T}(\check{\theta}_{T})\\\ \check{\gamma}_{T}\\\ \end{bmatrix}$
$\displaystyle\underset{}{\stackrel{{\scriptstyle}}{{=}}}$
$\displaystyle\begin{bmatrix}\quad\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{-1/2^{\prime}}M^{-1}\\\
(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}&\quad\\\
-(R\Sigma R^{\prime})^{-1}RM^{-1}\\\
\end{bmatrix}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o(1)$
$\displaystyle\underset{(a)}{\stackrel{{\scriptstyle D}}{{\rightarrow}}}$
$\displaystyle-\begin{bmatrix}\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{-1/2^{\prime}}M^{-1}\\\
(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}&\quad\\\
-(R\Sigma R^{\prime})^{-1}RM^{-1}\\\ \end{bmatrix}\mathcal{N}(0,V)$
$\displaystyle\underset{(b)}{\stackrel{{\scriptstyle D}}{{=}}}$
$\displaystyle\mathcal{N}\left(0,\begin{bmatrix}\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{-1/2^{\prime}}M^{-1}\\\
(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}&\quad\\\
-(R\Sigma R^{\prime})^{-1}RM^{-1}\\\ \end{bmatrix}V\right.$
$\displaystyle\left.\begin{array}[]{c}\\\ \\\ \\\
\end{array}\times\begin{bmatrix}(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{1/2}&\quad(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}&\quad-(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\ \end{bmatrix}\right)$
$\displaystyle\underset{(c)}{\stackrel{{\scriptstyle D}}{{=}}}$
$\displaystyle\mathcal{N}\left(0,\begin{bmatrix}(\Sigma^{1/2})^{\prime}P^{\perp}_{\Sigma^{1/2}R^{\prime}}\Sigma^{1/2}&0_{m\times
m}&0_{m\times q}\\\ 0_{m\times
m}&(V^{1/2})^{-1}P_{\Sigma^{1/2}R^{\prime}}(V^{1/2^{\prime}})^{-1}&-(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\ 0_{q\times m}&-(R\Sigma R^{\prime})^{-1}RM^{-1}&(R\Sigma
R^{\prime})^{-1}\\\ \end{bmatrix}\right)$

(a) Under Assumption 1(a)-(c) and (g), by the Lindeberg-Lévy CLT theorem, as
$T\rightarrow\infty$,
$\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\stackrel{{\scriptstyle
D}}{{\rightarrow}}\mathcal{N}(0,V)$ where
$V:=\mathbb{E}[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]$. (b)
Firstly, the minus sign can be discarded because of the symmetry of the
Gaussian distribution. Secondly, if $X$ is a random vector and $F$ is a
matrix, then $\mathbb{V}(FX)=F\mathbb{V}(X)F^{\prime}$. (c) Denote the final
asymptotic variance matrix with $\Gamma$, and its $(i,j)$ block components
with $\Gamma_{i,j}$. Then,

  * •

$\Gamma_{1,1}=\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{-1/2^{\prime}}M^{-1}V(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{1/2}\\\
=\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{-1/2^{\prime}}\Sigma\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{1/2}=(\Sigma^{1/2})^{\prime}P^{\perp}_{\Sigma^{1/2}R^{\prime}}\Sigma^{1/2}$
because $M^{-1}V(M^{\prime})^{-1}=:\Sigma=\Sigma^{1/2^{\prime}}\Sigma^{1/2}$,
$\Sigma^{-1/2}:=(\Sigma^{1/2})^{-1}$,
$\Sigma^{-1/2^{\prime}}:=(\Sigma^{1/2^{\prime}})^{-1}$, and
$P^{\perp}_{\Sigma^{1/2}R^{\prime}}P^{\perp}_{\Sigma^{1/2}R^{\prime}}=P^{\perp}_{\Sigma^{1/2}R^{\prime}}$
by idempotence of projections on linear spaces;

  * •

$\Gamma_{2,2}=(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}V(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}=\\\
(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}\Sigma\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}\\\
=(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}=(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}\\\
=(M^{\prime})^{-1}[V^{1/2}(M^{\prime})^{-1}]^{-1}P_{\Sigma^{1/2}R^{\prime}}[M^{-1}V^{1/2^{\prime}}]^{-1}M^{-1}\\\
=(V^{1/2})^{-1}P_{\Sigma^{1/2}R^{\prime}}(V^{1/2^{\prime}})^{-1}$ because
$M^{-1}V(M^{\prime})^{-1}=:\Sigma=\Sigma^{1/2^{\prime}}\Sigma^{1/2}$,
$\Sigma^{-1/2}:=(\Sigma^{1/2})^{-1}=[V^{1/2}(M^{\prime})^{-1}]^{-1}=M^{\prime}V^{-1/2}$,
$\Sigma^{-1/2^{\prime}}:=(\Sigma^{1/2^{\prime}})^{-1}=[M^{-1}V^{1/2^{\prime}}]^{-1}=V^{-1/2^{\prime}}M$,
and
$P_{\Sigma^{1/2}R^{\prime}}P_{\Sigma^{1/2}R^{\prime}}=P^{\perp}_{\Sigma^{1/2}R^{\prime}}$
by idempotence;

  * •

$\Gamma_{3,3}=(R\Sigma
R^{\prime})^{-1}RM^{-1}V(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}=(R\Sigma R^{\prime})^{-1}R\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}=(R\Sigma R^{\prime})^{-1}$ because
$M^{-1}V(M^{\prime})^{-1}=:\Sigma$;

  * •

$\Gamma_{1,2}=\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{-1/2^{\prime}}M^{-1}V(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}=\\\
\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{-1/2^{\prime}}\Sigma\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}=\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}=0$
because $M^{-1}V(M^{\prime})^{-1}=:\Sigma=\Sigma^{1/2^{\prime}}\Sigma^{1/2}$,
$\Sigma^{-1/2}:=(\Sigma^{1/2})^{-1}$,
$\Sigma^{-1/2^{\prime}}:=(\Sigma^{1/2^{\prime}})^{-1}$, and
$P^{\perp}_{\Sigma^{1/2}R^{\prime}}P_{\Sigma^{1/2}R^{\prime}}=0_{m\times m}$;

  * •

$\Gamma_{1,3}=-\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{-1/2^{\prime}}M^{-1}V(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\
=-\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{-1/2^{\prime}}\Sigma
R^{\prime}(R\Sigma
R^{\prime})^{-1}=-\Sigma^{1/2^{\prime}}P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{1/2}R^{\prime}(R\Sigma
R^{\prime})^{-1}=0$ because
$M^{-1}V(M^{\prime})^{-1}=:\Sigma=\Sigma^{1/2^{\prime}}\Sigma^{1/2}$,
$\Sigma^{-1/2^{\prime}}:=(\Sigma^{1/2^{\prime}})^{-1}$, and
$P_{\Sigma^{1/2}R^{\prime}}^{\perp}\Sigma^{1/2}R^{\prime}=0_{m\times q}$;

  * •

$\Gamma_{2,3}=-(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}M^{-1}V(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\
=-(M^{\prime})^{-1}\Sigma^{-1/2}P_{\Sigma^{1/2}R^{\prime}}\Sigma^{-1/2^{\prime}}\Sigma
R^{\prime}(R\Sigma R^{\prime})^{-1}\\\
=-(M^{\prime})^{-1}\Sigma^{-1/2}[\Sigma^{1/2}R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma^{1/2^{\prime}}]\Sigma^{-1/2^{\prime}}\Sigma
R^{\prime}(R\Sigma R^{\prime})^{-1}\\\ =-(M^{\prime})^{-1}[R^{\prime}(R\Sigma
R^{\prime})^{-1}R]\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}=-(M^{\prime})^{-1}R^{\prime}(R\Sigma R^{\prime})^{-1}$

because $M^{-1}V(M^{\prime})^{-1}=:\Sigma$ and
$P_{\Sigma^{1/2}R^{\prime}}=[\Sigma^{1/2}R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma^{1/2^{\prime}}]$.

∎

######  Lemma 26.

Using the notation of Proposition 2 (p. 2), under Assumptions 1, 2 and 3,

  1. (i)

for any sequence $(\theta_{T},\tau_{T})_{T\in\mathbf{N}}$ converging to
$(\theta_{0},\tau(\theta_{0}))$, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$\begin{bmatrix}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}&\frac{\partial
r(\bar{\theta}_{T})^{\prime}}{\partial\theta}\\\ \frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}&0\\\
\frac{\partial r(\bar{\theta}_{T})}{\partial\theta^{\prime}}&0&0\\\
\end{bmatrix}\rightarrow\begin{bmatrix}0_{m\times
m}&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{\prime}&\frac{\partial
r(\theta_{0})^{\prime}}{\partial\theta}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]&\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]&0\\\
\frac{\partial r(\theta_{0})}{\partial\theta^{\prime}}&0&0\\\ \end{bmatrix}$ ;

  2. (ii)

$\begin{bmatrix}0_{m\times
m}&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{\prime}&\frac{\partial
r(\theta_{0})^{\prime}}{\partial\theta}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]&\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]&0\\\
\frac{\partial r(\theta_{0})}{\partial\theta^{\prime}}&0&0\\\ \end{bmatrix}$
is invertible, so that, for any sequence
$(\theta_{T},\tau_{T})_{T\in\mathbf{N}}$ converging to
$(\theta_{0},\tau(\theta_{0}))$, $\mathbb{P}$-a.s., for $T$ big enough, the
matrix

$\begin{bmatrix}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}&\frac{\partial
r(\bar{\theta}_{T})^{\prime}}{\partial\theta}\\\ \frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}&0\\\
\frac{\partial r(\bar{\theta}_{T})}{\partial\theta^{\prime}}&0&0\\\
\end{bmatrix}$ is invertible; and

  3. (iii)

for any sequence $(\theta_{T},\tau_{T})_{T\in\mathbf{N}}$ converging to
$(\theta_{0},\tau(\theta_{0}))$, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$\begin{bmatrix}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}&\frac{\partial
r(\bar{\theta}_{T})^{\prime}}{\partial\theta}\\\ \frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}&0\\\
\frac{\partial r(\bar{\theta}_{T})}{\partial\theta^{\prime}}&0&0\\\
\end{bmatrix}^{-1}\\\ \rightarrow\begin{bmatrix}-\Sigma+\Sigma
R^{\prime}(R\Sigma R^{\prime})^{-1}R\Sigma&\quad
M^{-1}\negthickspace-\negthickspace\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}&\Sigma R^{\prime}(R\Sigma R^{\prime})^{-1}\\\
(M^{\prime})^{-1}\negthickspace-\negthickspace(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma&(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}&\;-(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\ (R\Sigma R^{\prime})^{-1}R\Sigma&-(R\Sigma
R^{\prime})^{-1}RM^{-1}&(R\Sigma R^{\prime})^{-1}\\\ \end{bmatrix}$.

###### Proof.

(i) Under Assumptions 1, 2 and 3(a), it follows from the continuity of
$\frac{\partial r(.)}{\partial\theta^{\prime}}$, which is implied by
Assumption 3(a), and Lemma 14ii and iii (p. 14) and Lemma 17 (p. 17), given
that $\tau(\theta_{0})=0_{m\times 1}$ by Lemma 10ii (p. 10) and Assumption
1(c), under Assumption 1(a)(b)(d)(e)(g) and (h).

(ii) It is sufficient to check the assumptions of Corollary 2i (p. 2) with
$A=\left[\begin{array}[]{c c }0_{m\times m}&M^{\prime}\\\ M&\
V\end{array}\right]$ and $B=\begin{bmatrix}R^{\prime}\\\ 0_{m\times q}\\\
\end{bmatrix}$ in order to establish the first part of the statement. Firstly,
under Assumptions 1 and 2, by Lemma 13iii (p. 13), $A=\left[\begin{array}[]{c
c }0_{m\times m}&M^{\prime}\\\ M&\ V\end{array}\right]$ is invertible.
Secondly, by Assumptions 1(h) and 3(b), $(B^{\prime}A^{-1}B)=-(R\Sigma
R^{\prime})$ is also invertible. Then, the second part of the statement
follows from a trivial case of the Lemma 30 (p. 30).
(iii) Under Assumption 1(a)(b)(c)(d)(e)(g)(h), by the statement (ii) of the
present lemma, the limiting matrix is invertible. Thus, using the notation of
Proposition 2 (p. 2),

$\displaystyle\ \begin{bmatrix}0_{m\times
m}&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{\prime}&\frac{\partial
r(\theta_{0})^{\prime}}{\partial\theta}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]&\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]&0\\\
\frac{\partial r(\theta_{0})}{\partial\theta^{\prime}}&0&0\\\
\end{bmatrix}^{-1}$ $\displaystyle=$ $\displaystyle\begin{bmatrix}0_{m\times
m}&M^{\prime}&R^{\prime}\\\ M&V&0_{m\times q}\\\ R&0_{q\times m}&0_{q\times
q}\\\ \end{bmatrix}^{-1}$ $\displaystyle\stackrel{{\scriptstyle}}{{=}}$
$\displaystyle\begin{bmatrix}-\Sigma+\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma&\quad M^{-1}-\Sigma R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}&\Sigma R^{\prime}(R\Sigma R^{\prime})^{-1}\\\
(M^{\prime})^{-1}-(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma&(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}&\quad-(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\ (R\Sigma R^{\prime})^{-1}R\Sigma&-(R\Sigma
R^{\prime})^{-1}RM^{-1}&(R\Sigma R^{\prime})^{-1}\\\ \end{bmatrix}$

where the explanation for the last equality is as follows. Apply Corollary 2ii
(p. 2) with $A=\begin{bmatrix}0_{m\times m}&M^{\prime}\\\ M&V\end{bmatrix}$
and $B=\begin{bmatrix}R^{\prime}\\\ 0_{m\times q}\\\ \end{bmatrix}$, and note
that, by Lemma 27iii, iv and vi (p. 27),

$\displaystyle A^{-1}-A^{-1}B(B^{\prime}A^{-1}B)B^{\prime}A^{-1}$
$\displaystyle=$ $\displaystyle\begin{bmatrix}-\Sigma+\Sigma
R^{\prime}(R\Sigma R^{\prime})^{-1}R\Sigma&\quad M^{-1}-\Sigma
R^{\prime}(R\Sigma R^{\prime})^{-1}RM^{-1}\\\
(M^{\prime})^{-1}-(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}R\Sigma&\quad(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}RM^{-1}\\\ \end{bmatrix}$ $\displaystyle
A^{-1}B(B^{\prime}A^{-1}B)^{-1}$ $\displaystyle=$
$\displaystyle\begin{bmatrix}\Sigma R^{\prime}(R\Sigma R^{\prime})^{-1}\\\
-(M^{\prime})^{-1}R^{\prime}(R\Sigma R^{\prime})^{-1}\\\ \end{bmatrix}$
$\displaystyle(B^{\prime}A^{-1}B)^{-1}$ $\displaystyle=$
$\displaystyle-(R\Sigma R^{\prime})^{-1}.$

Then, the result follows from the continuity of the inverse transformation
(e.g., Rudin 1953, Theorem 9.8). ∎

######  Lemma 27.

Let $A=\begin{bmatrix}0_{m\times m}&M^{\prime}\\\ M&V\end{bmatrix}$ and
$B=\begin{bmatrix}R^{\prime}\\\ 0_{m\times q}\\\ \end{bmatrix}$ where
$\Sigma:=\Sigma(\theta_{0}):=M^{-1}V(M^{\prime})^{-1}$,
$M:=\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]$,
$V:=\mathbb{E}[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]$, and
$R=\frac{\partial r(\theta_{0})}{\partial\theta^{\prime}}$. Then, under
Assumption 1(a)(b)(h) and 3(b), the following equalities hold

  1. (i)

$\displaystyle A^{-1}=\begin{bmatrix}-\Sigma&M^{-1}\\\
(M^{\prime})^{-1}&0_{m\times m}\\\ \end{bmatrix}$;

  2. (ii)

$\displaystyle A^{-1}B=\begin{bmatrix}-\Sigma R^{\prime}\\\
(M^{\prime})^{-1}R^{\prime}\\\ \end{bmatrix}$, so that
$B^{\prime}A^{-1}=\begin{bmatrix}-R\Sigma&\quad RM^{-1}\\\ \end{bmatrix}$;

  3. (iii)

$\displaystyle(B^{\prime}A^{-1}B)^{-1}=-(R\Sigma R^{\prime})^{-1}$;

  4. (iv)

$\displaystyle A^{-1}B(B^{\prime}A^{-1}B)^{-1}=\begin{bmatrix}\Sigma
R^{\prime}(R\Sigma R^{\prime})^{-1}\\\ -(M^{\prime})^{-1}R^{\prime}(R\Sigma
R^{\prime})^{-1}\\\ \end{bmatrix}$;

  5. (v)

$\displaystyle
A^{-1}B(B^{\prime}A^{-1}B)B^{\prime}A^{-1}=\begin{bmatrix}-\Sigma
R^{\prime}(R\Sigma R^{\prime})^{-1}R\Sigma&\qquad\Sigma R^{\prime}(R\Sigma