(ii) The supremum of the absolute value of the product is smaller than the
product of the suprema of the absolute values. Thus,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\mathbf{S}^{\epsilon}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)|\right]$
(22) $\displaystyle\leqslant$
$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\mathbf{S}^{\epsilon}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\sup_{(\theta,\tau)\in\mathbf{S}^{\epsilon}}\left|\psi(X_{1},\theta)\right|\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leqslant}}$
$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\mathbf{S}^{\epsilon}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|^{2}\right]^{1/2}\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)|^{2}\right]^{1/2}$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{<}}$ $\displaystyle\infty$

(a) Firstly, for any $(\theta,\tau)\in\mathbf{S}^{\epsilon}$,
$\theta\in\mathbf{\Theta}^{\epsilon}$ because, for all
$(\tilde{\tau},\tilde{\theta})\in\mathbf{S}$,
$|\theta-\tilde{\theta}|=\sqrt{\sum_{k=1}^{m}(\theta_{k}-\tilde{\theta}_{k})^{2}}\leqslant\sqrt{\sum_{k=1}^{m}(\theta_{k}-\tilde{\theta}_{k})^{2}+\sum_{k=1}^{m}(\tau_{k}-\tilde{\tau}_{k})^{2}}=|(\theta,\tau)-(\tilde{\tau},\tilde{\theta})|<\epsilon$.
Thus, as the expression in the second supremum does not depend on $\tau$,
$\sup_{(\theta,\tau)\in\mathbf{S}^{\epsilon}}\left|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}\right|\leqslant\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}\left|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}\right|$.
Secondly apply the Cauchy-Schwarz inequality. Finally, note that
$[\sup_{(\theta,\tau)\in\mathbf{S}^{\epsilon}}|\mathrm{e}^{\tau(\theta)^{\prime}\psi(X_{1},\theta)}|]^{2}=\sup_{(\theta,\tau)\in\mathbf{S}^{\epsilon}}|\mathrm{e}^{\tau(\theta)^{\prime}\psi(X_{1},\theta)}|^{2}$
and
$[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}\left|\psi(X_{1},\theta)\right|]^{2}=\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}\left|\psi(X_{1},\theta)\right|^{2}$
because $x\mapsto x^{2}$ is increasing on $\mathbf{R}_{+}$. (b) Note that
$|\mathrm{e}^{\tau(\theta)^{\prime}\psi(X_{1},\theta)}|^{2}=\mathrm{e}^{2\tau(\theta)^{\prime}\psi(X_{1},\theta)}$,
and then apply Assumption 1(e) to the first term. Then, application of the
statement (i) of the present lemma to the second term yields the result. ∎

######  Remark 1.

The first step of the proof shows that even the fourth moment is uniformly
bounded. $\diamond$

######  Lemma 10 (Implicit function $\tau(.)$).

Under Assumption 1 (a)(b)(e)(g) and (h),

  * (i)

for all $\theta\in\mathbf{\Theta}$,
$\tau\mapsto\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\right]$
is a strictly convex function s.t.
$\frac{\partial\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\right]}{\partial\tau}=\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\right]$;

  * (ii)

under additional Assumption 1(d), for all $\theta\in\mathbf{\Theta}$, there
exists a unique $\tau(\theta)$ such that
$\mathbb{E}\left[\mathrm{e}^{\tau(\theta)^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\right]=0$;
and

  * (iii)

under additional Assumption 1(d),
$\tau:\mathbf{\Theta}\rightarrow\mathbf{R}^{m}$ is continuous; and

  * (iv)

under additional Assumption 1(c) and (d), for all
$\theta\in\mathbf{\Theta}\setminus\\{\theta_{0}\\}$,
$\mathbb{E}\left[\mathrm{e}^{\tau(\theta)^{\prime}\psi(X_{1},\theta)}\right]<\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1},\theta_{0})}\right]=1$
where $\tau(\theta_{0})=0_{m\times 1}$.

###### Proof.

(i) Under Assumption 1(a) and (b), by the Cauchy-Schwarz inequality,

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\mathbf{S}^{\epsilon}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\right]\leqslant\mathbb{E}\left[\sup_{(\theta,\tau)\in\mathbf{S}^{\epsilon}}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}\right]^{1/2}$,
which is finite by Assumption 1(e). Now, by Assumption 1(e), for all
$\dot{\theta}\in\mathbf{\Theta}$,
$\tau(\dot{\theta})\in\mathrm{int}[\mathbf{T}(\dot{\theta})]$. Then, by a
standard result on Laplace’s transform (e.g., Monfort (1980, Theorems 3 on p.
183),
$\tau\mapsto\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\dot{\theta})}\right]$
is $C^{\infty}$ in a neighborhood of $\tau(\dot{\theta})$, and
$\tau\mapsto\frac{\partial\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\dot{\theta})}\right]}{\partial\tau}=\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\dot{\theta})}\psi(X_{1},\theta)\right]$
and
$\tau\mapsto\frac{\partial^{2}\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\dot{\theta})}\right]}{\partial\tau\partial\tau^{\prime}}=\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\dot{\theta})}\psi(X_{1},\dot{\theta})\psi(X_{1},\dot{\theta})^{\prime}\right]$.
Moreover, under Assumptions 1(a)-(b), (e) and (g), Assumption 1(h) implies
that,

$\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\dot{\theta})}\psi(X_{1},\dot{\theta})\psi(X_{1},\dot{\theta})^{\prime}\right]$
is a symmetric positive-definite matrix because a well-defined covariance
matrix is invertible iff it is invertible under an equivalent probability
measure (Lemma 29 and Corollary 1i on p. 1).

(ii) Assumption 1(d) ensures existence, while the statement (i) of the present
lemma ensures that $\tau(\theta)$ is the solution of a strictly convex
problem, so that it is unique.

(iii) Note that, under our assumptions, an application of the standard
implicit function (e.g., Rudin 1953, Theorem 9.28) is not directly possible as
it requires
$(\theta,\tau)\mapsto\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\right]$
to be continuously differentiable in $\mathbf{S}^{\epsilon}$, which, in turn,
typically requires to uniformly bound the derivative of the latter in
$\mathbf{S}^{\epsilon}$ (e.g., Davidson 1994, Theorem 9.31). Thus, we apply
the sufficiency part of Kumagai’s implicit function theorem (Kumagai 1980).
Check its assumptions. Firstly, under Assumptions 1(a)(b)(e) and (g), by Lemma
9ii (p. 9) and the Lebesgue dominated convergence theorem,
$(\theta,\tau)\mapsto\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\right]$
is continuous in $\mathbf{S}^{\epsilon}$, i.e., in an open neighborhood of
every $(\theta,\tau)\in\mathbf{S}$. Secondly, by the inverse function theorem
applied to
$\tau\mapsto\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\right]$
(e.g., Rudin 1953, Theorem 9.24), for all
$\theta\in\mathbf{\Theta}^{\epsilon}$,
$\tau\mapsto\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\right]$
is locally one-to-one :121212Here it is necessary to work in an
$\epsilon$-neighborhood of $\mathbf{\Theta}$ in order to satisfy the
assumption of Kumagai’s implicit function theorem (Kumagai 1980). The standard
implicit function theorem would also require the existence of open
neighborhoods around the parameter values at which the function is zero. As
explained in the proof of (i), under Assumption 1(a)(b)(e) and (h),
$\tau\mapsto\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\right]$
is continuously differentiable and, under Assumption 1(a)(b)(e)(g) and (h),
for all $\theta\in\mathbf{\Theta}$,
$\frac{\partial\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\right]}{\partial\tau^{\prime}}=\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}\right]$
is invertible, so that the assumptions of the inverse function theorem are
valid.

(iv) By the statements (i) and (ii) of the present lemma, for all
$\theta\in\mathbf{\Theta}$, for all $\tau\neq\tau(\theta)$,
$\mathbb{E}[\mathrm{e}^{\tau(\theta)^{\prime}\psi(X_{1},\theta)}]<\mathbb{E}[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}]$.
Now, for all $\theta\in\mathbf{\Theta}\setminus\\{\theta_{0}\\}$,
$\tau(\theta)\neq 0_{m\times 1}$: If there existed
$\dot{\theta}\in\mathbf{\Theta}\setminus\\{\theta_{0}\\}$ s.t.
$\tau(\dot{\theta})=0_{m\times 1}$, then
$0=\mathbb{E}[\mathrm{e}^{\tau(\dot{\theta})^{\prime}\psi(X_{1},\dot{\theta})}\psi(X_{1},\dot{\theta})]=\mathbb{E}[\psi(X_{1},\dot{\theta})]$,
which would contradict Assumption 1(c). Thus, for all
$\theta\in\mathbf{\Theta}\setminus\\{\theta_{0}\\}$,
$\mathbb{E}[\mathrm{e}^{\tau(\theta)^{\prime}\psi(X_{1},\theta)}]<\mathbb{E}[\mathrm{e}^{0_{1\times
m}\psi(X_{1},\theta)}]=1$. Then, the result follows by the statement (ii) of
the present lemma because $0_{m\times
1}=\mathbb{E}[\psi(X_{1},\theta_{0})]=\mathbb{E}[\mathrm{e}^{0_{1\times
m}\psi(X_{1},\theta)}\psi(X_{1},\theta_{0})]$. ∎

###  B.2. Decomposition and derivatives of the log-ESP $L_{T}(.,.)$

In this section, we simplify $L_{T}(\theta,\tau)$ and study its derivatives.
Such results are needed for the proof of Theorem 1ii and other results
afterwards.

######  Lemma 11.

Under Assumption 1(a)-(e) and (g)(h), by Lemma 10 (p. 10), define
$\tau(\theta_{0})=\tau_{0}=0_{m\times 1}$. Under Assumption 1(a)-(b), (e) and
(h),

  1. (i)

under additional Assumption 1(d) and (g), there exist
$(\underline{M}_{\mathrm{e}},\overline{M}_{\mathrm{e}})\in\mathbf{R}_{+}\setminus\\{0\\}$
s.t. $\mathbb{P}$-a.s. for $T$ big enough,
$\underline{M}_{\mathrm{e}}<\inf_{(\theta,\tau)\in\mathbf{S}}\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}$
and
$\sup_{(\theta,\tau)\in\mathbf{S}}\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}<\overline{M}_{\mathrm{e}}$;

  2. (ii)

under additional Assumption 1 (c)(d) and (g), there exists an open ball
$B_{r}(\theta_{0},\tau_{0})$ centered at $(\theta_{0},\tau_{0})$ of radius
$r>0$, which is a subset of $\mathbf{S}$;

  3. (iii)

under additional Assumption 1(c)(d)(f) and (g), for all $(\theta,\tau)$ in a
closed ball
$\overline{B_{r_{\partial}}(\theta_{0},\tau_{0})}\subset\mathbf{S}$ centered
at $(\theta_{0},\tau_{0})$ with radius $r_{\partial}>0$,
$|\mathbb{E}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|_{\det}^{2}>0$,
so that, $\mathbb{P}$-a.s. for $T$ big enough,
$|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}|_{\det}^{2}>0$;

  4. (iv)

under additional Assumption 1(g), $\mathbb{P}$-a.s. for $T$ big enough,

$\inf_{(\theta,\tau)\in\mathbf{S}}\negthickspace|\frac{1}{T}\negthickspace\sum_{t=1}^{T}\negthickspace\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}|_{\det}>0$.

###### Proof.

(i) Under Assumption 1(a)(b)(d)(e)(g) and (h), by Lemma 2i (p. 2), which
states that, $\mathbb{P}\text{-a.s.}$ as $T\rightarrow\infty$,
$\sup_{(\theta,\tau)\in\mathbf{S}}\left|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}-\mathbb{E}[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}]\right|=o(1)$,
and Lemma 3 (p. 3) with $\mathrm{P}=\mathbb{P}$, which states that
$0<\inf_{(\theta,\tau)\in\mathbf{S}}\mathbb{E}[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}]$,
the result follows.

(ii) First of all, note that the result is not completely immediate, as
$\mathbf{S}:=\\{(\theta,\tau):\theta\in\mathbf{\Theta}\wedge\tau\in\overline{B_{\epsilon_{\mathbf{T}}}(\tau(\theta))}\\}$
is not a Cartesian product. Under Assumption 1 (a)(b)(d)(e)(g) and (h), by
Lemma 10iii (p. 10), $\tau:\mathbf{\Theta}\rightarrow\mathbf{R}^{m}$ is
continuous. Thus, by the topological definition of continuity,
$\tau^{-1}[B_{\epsilon_{\mathbf{T}}/2}(\tau_{0})]$ is an open set of
$\mathbf{\Theta}$. Moreover, by the definition of $\tau_{0}$,
$\theta_{0}\in\tau^{-1}[B_{\epsilon_{\mathbf{T}}/2}(\tau_{0})]$, and, by
Assumption 1(c), $\theta_{0}\in\mathrm{int}(\mathbf{\Theta})$,131313This
assumption forbids $\theta_{0}$ to be on the boundary of
$\tau^{-1}[B_{\epsilon_{\mathbf{T}}/2}(\tau_{0})]$, which is an open set of
$\mathbf{\Theta}$, but not necessarily of $\mathbf{R}^{m}$. so that there
exists $r_{0}>0$ s.t.
$B_{r_{0}}(\theta_{0})\subset\tau^{-1}[B_{\epsilon_{\mathbf{T}}/2}(\tau_{0})]$
and $B_{r_{0}}(\theta_{0})\subset\mathbf{\Theta}$. Now, for this proof, put
$r=\min\\{r_{0},\epsilon_{\mathbf{T}}/2\\}$. Then, it remains to show that
$B_{r}(\theta_{0},\tau_{0})\subset\mathbf{S}$, i.e., for all
$(\dot{\theta},\dot{\tau})\in B_{r}(\theta_{0},\tau_{0})$,
$|\dot{\tau}-\tau(\dot{\theta})|\leqslant\epsilon_{\mathbf{T}}$. By the
triangle inequality, for any $(\dot{\theta},\dot{\tau})\in
B_{r}(\theta_{0},\tau_{0})$,

$\displaystyle|\dot{\tau}-\tau(\dot{\theta})|$ $\displaystyle\leqslant$
$\displaystyle|\dot{\tau}-\tau_{0}|+|\tau_{0}-\tau(\dot{\theta})|$
$\displaystyle\leqslant$
$\displaystyle\frac{\epsilon_{\mathbf{T}}}{2}+\frac{\epsilon_{\mathbf{T}}}{2}=\epsilon_{\mathbf{T}}$

where the explanations for the last inequality are as follows. Firstly,
$|\dot{\tau}-\tau_{0}|<\sqrt{\sum_{k=1}^{m}(\dot{\tau}_{k}-\tau_{0,k})^{2}}\leqslant\sqrt{\sum_{k=1}^{m}(\dot{\theta}_{k}-\theta_{0,k})^{2}+\sum_{k=1}^{m}(\dot{\tau}_{k}-\tau_{0,k})^{2}}<r\leqslant\frac{\epsilon_{\mathbf{T}}}{2}$
by definition of $r$. Secondly, and
similarly,$|\dot{\theta}-\theta_{0}|<\sqrt{\sum_{k=1}^{m}(\dot{\theta}_{k}-\theta_{0,k})^{2}}\leqslant\sqrt{\sum_{k=1}^{m}(\dot{\theta}_{k}-\theta_{0,k})^{2}+\sum_{k=1}^{m}(\dot{\tau}_{k}-\tau_{0,k})^{2}}<r\leqslant
r_{0}$, so that
$|\tau_{0}-\tau(\dot{\theta})|<\frac{\epsilon_{\mathbf{T}}}{2}$ because
$B_{r_{0}}(\theta_{0})\subset\tau^{-1}[B_{\epsilon_{\mathbf{T}}/2}(\tau_{0})]$.

(iii) Under Assumption 1 (a)-(b) and (e)-(f), by Lemma 7i (p. 7), Assumption
1(b) and the Lebesgue dominated convergence theorem,
$(\theta,\tau)\mapsto\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\right]$
is continuous in $\mathbf{S}$, and thus in a neighborhood of
$(\theta_{0},\tau_{0})$ in $\mathbf{S}$ by Assumption 1(c) and (e). Then,
$(\theta,\tau)\mapsto|\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\right]|_{\det}^{2}$
is also continuous. Now, by Assumption 1(h),

$|\mathbb{E}[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1},\theta_{0})}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}]|_{\det}^{2}>0$,
so that, under Assumption 1(a)-(e) and (g)-(h), by the statement (ii) of the
present lemma, there exists a closed ball
$\overline{B_{r_{\partial}}(\theta_{0},\tau_{0})}\subset\mathbf{S}$ centered
at $(\theta_{0},\tau_{0})$ with radius $r_{\partial}>0$, s.t., for all
$(\theta,\tau)\in\overline{B_{r_{\partial}}(\theta_{0},\tau_{0})}$,
$0<|\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\right]|_{\det}^{2}$,
which is the first part of the result. By Lemma 30 (p. 30), the second part of
the result follows from the continuity of
$(\theta,\tau)\mapsto\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\right]$,
the invertibility of
$\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\right]$
for all $(\theta,\tau)\in\overline{B_{r_{\partial}}(\theta_{0},\tau_{0})}$,
and Lemma 7ii (p. 7), which, under Assumption 1(a)-(b) and (e)-(f), implies
that
$\sup_{(\theta,\tau)\in\overline{B_{r_{\partial}}(\theta_{0},\tau_{0})}}|\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta}\right]-\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)^{\prime}}{\partial\theta}\right]|=o(1)$,
$\mathbb{P}$-a.s. as $T\rightarrow\infty$.

(iv) It follows from Lemma 30 (p. 30), so that it is sufficient to check its
assumptions. Firstly, under Assumptions 1(a)-(b), (e), (g) and (h), by
Corollary 1 (p. 1), for all $(\theta,\tau)\in\mathbf{S}$,
$\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}\right]$
is a positive definite symmetric matrix, and thus it is invertible. Secondly,
under Assumption 1(a)-(b), (e) and (g), by Lemma 8i (p. 8),

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\mathbf{S}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$,
so that by the Lebesgue dominated convergence theorem and Assumption 1(b),
$(\theta,\tau)\mapsto\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}\right]$
is continuous in $\mathbf{S}$. Finally, under Assumptions 1(a)-(b), (d), (e),
(g) and (h), $\mathbb{P}$-a.s. as $T\rightarrow\infty$,

$\sup_{(\theta,\tau)\in\mathbf{S}}\left|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}-\mathbb{E}[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}]\right|=o(1)$.

∎

In order to simplify the analysis of the asymptotic properties of the ESP
estimator, we decompose the LogESP into three terms.

######  Lemma 12 (LogESP decomposition).

Under Assumption 1, $\mathbb{P}$-a.s. for $T$ big enough, define, for all
$(\theta,\tau)\in\overline{B_{r_{\partial}}(\theta_{0},\tau_{0})}$,
$L_{T}(\theta,\tau):=\ln\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\right]\\\
-\frac{1}{2T}\ln\negthickspace\left\\{\left|\left[\sum_{t=1}^{T}\frac{\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}}{\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\negthickspace\left[\sum_{t=1}^{T}\frac{\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}}{\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]\negthickspace\left[\sum_{t=1}^{T}\frac{\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}}{\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}}\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta}\right]^{-1}\right|_{\det}\negthickspace\right\\}$,
which exists by Lemma 11 (p. 11), and where
$\overline{B_{r_{\partial}}(\theta_{0},\tau_{0})}$ is defined as in the
aforementioned lemma. Then, under Assumption 1(a)(b) and (d)-(h),
$\mathbb{P}$-a.s. for $T$ big enough, for all
$(\theta,\tau)\in\overline{B_{r_{\partial}}(\theta_{0},\tau_{0})}$,

$\displaystyle
L_{T}(\theta,\tau)=M_{1,T}(\theta,\tau)+M_{2,T}(\theta,\tau)+M_{3,T}(\theta,\tau)\text{
where}$

$M_{1,T}(\theta,\tau)\negthickspace:=\negthickspace\left(1\negthickspace-\negthickspace\frac{m}{2T}\right)\ln\negthickspace\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\right]$,
$M_{2,T}(\theta,\tau)\negthickspace:=\negthickspace\frac{1}{2T}\ln\negthickspace\left[\left|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right|_{\det}^{2}\right]$,
and

$M_{3,T}(\theta,\tau)\negthickspace:=\negthickspace-\frac{1}{2T}\ln\negthickspace\left[\left|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right|_{\det}\right]$.

###### Proof.

First of all, note that, under Assumption 1, by Lemma 11 (p. 11), $L_{T}(.)$
is well-defined $\mathbb{P}$-a.s. for $T$ big enough, for all
$(\theta,\tau)\in\overline{B_{r_{\partial}}(\theta_{0},\tau_{0})}$. Thus,
under Assumption 1, $\mathbb{P}$-a.s. for $T$ big enough, for all
$(\theta,\tau)\in\overline{B_{r_{\partial}}(\theta_{0},\tau_{0})}$.

$\displaystyle L_{T}(\theta,\tau)$ (24) $\displaystyle=$
$\displaystyle\ln\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\right]$
$\displaystyle\negthickspace-\frac{1}{2T}\ln\negthickspace\left\\{\left|\left[\sum_{t=1}^{T}\frac{\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}}{\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\negthickspace\left[\sum_{t=1}^{T}\frac{\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}}{\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]\right.\right.$
$\displaystyle\hskip
158.99377pt\left.\left.\times\left[\sum_{t=1}^{T}\frac{\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}}{\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}}\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta}\right]^{-1}\right|_{\det}\negthickspace\right\\}$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{=}}$
$\displaystyle\ln\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\right]+\frac{1}{2T}\ln\left[\left|\sum_{t=1}^{T}\frac{\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}}{\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right|_{\det}^{2}\right]$
$\displaystyle\
-\frac{1}{2T}\ln\left[\left|\frac{1}{T}\sum_{t=1}^{T}\frac{\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}}{\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right|_{\det}\right]$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{=}}$
$\displaystyle\ln\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\right]$
$\displaystyle+\frac{1}{2T}\ln\left[\left(\frac{1}{\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}}\right)^{2m}\left|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right|_{\det}^{2}\right]$
$\displaystyle-\frac{1}{2T}\ln\left[\left(\frac{1}{\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}}\right)^{m}\left|\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right|_{\det}\right]\
$ $\displaystyle\stackrel{{\scriptstyle(c)}}{{=}}$
$\displaystyle\left(1-\frac{m}{2T}\right)\
\ln\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\right]+\frac{1}{2T}\ln\left[\left|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right|_{\det}^{2}\right]$
$\displaystyle-\frac{1}{2T}\ln\left[\left|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right|_{\det}\right].$

(a) Firstly, use that the determinant of the product is the product of the
determinants (e.g. Rudin 1953, Theorem 9.35). Secondly, the determinant of an
inverse is the inverse of the determinant (e.g. Rudin 1953, p. 233). Finally,
use basic properties of the logarithm, and note that we keep the square in the
second logarithm in order to ensure the positivity of the argument (then the
strict positivity is ensured by Lemma 11 on p. 11). (b) Use multilinearity of
determinant. (c) Note that $1+\frac{-2m}{2T}-\frac{-m}{2T}=1-\frac{m}{2T}$. ∎

####  B.2.1. Derivatives of
$M_{1,T}(\theta,\tau):=\left(1-\frac{m}{2T}\right)\ln\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\right]$

First derivative $\frac{\partial M_{1,T}(\theta,\tau)}{\partial\theta_{j}}$.
By Assumption 1(b), $\theta\mapsto\psi(X_{1},\theta)$ is differentiable in
$\mathbf{\Theta}$ $\mathbb{P}$-a.s. Thus, for all
$(\theta,\tau)\in\mathbf{S}$, for all $j\in[\\![1,m]\\!]$,

$\displaystyle\frac{\partial
M_{1,T}(\theta,\tau)}{\partial\theta_{j}}=\left(1-\frac{m}{2T}\right)\frac{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}}{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}}.$
(25)

Second derivative
$\frac{\partial^{2}M_{1,T}(\theta,\tau)}{\partial\theta_{\ell}\partial\theta_{j}}$.
By Assumption 2(a), $\theta\mapsto\psi(X_{1},\theta)$ are three times
continuously differentiable in a neighborhood of $\theta_{0}$
$\mathbb{P}$-a.s. Thus, by equation (25) on p. 25, under Assumptions 1(a)-(e),
(g)-(h) and 2(a), by Lemma 11ii (p. 11), $\mathbb{P}$-a.s., for all
$(\theta,\tau)$ in a neighborhood of $(\theta_{0},\tau_{0})$, for all
$(\ell,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\frac{\partial^{2}M_{1,T}(\theta,\tau)}{\partial\theta_{\ell}\partial\theta_{j}}$
(26) $\displaystyle=$
$\displaystyle\frac{(1-\frac{m}{2T})}{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\right]^{2}}\left\\{\left\\{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left[\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\right]\left[\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right]+\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left[\tau^{\prime}\frac{\partial^{2}\psi_{t}(\theta)}{\partial\theta_{j}\partial\theta_{\ell}}\right]\right\\}\right.$
$\displaystyle\times\left\\{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\right\\}-\left\\{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left[\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right]\right\\}$
$\displaystyle\left.\times\left\\{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left[\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\right]\right\\}\right\\}$
$\displaystyle=$
$\displaystyle\frac{(1-\frac{m}{2T})}{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\right]}\left\\{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left[\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\right]\left[\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right]+\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left[\tau^{\prime}\frac{\partial^{2}\psi_{t}(\theta)}{\partial\theta_{j}\partial\theta_{\ell}}\right]\right\\}$
$\displaystyle-\frac{(1-\frac{m}{2T})}{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\right]^{2}}\left\\{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right\\}\times\left\\{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\right\\}.$

Second derivative
$\frac{\partial^{2}M_{1,T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{j}}$.
Under Assumption 1(a)-(b), by equation (25) on p. 25, $\mathbb{P}$-a.s., for
all $(\theta,\tau)\in\mathbf{S}$, for all $(k,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\frac{\partial^{2}M_{1,T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{j}}$
(27) $\displaystyle=$
$\displaystyle\left(1-\frac{m}{2T}\right)\frac{1}{\left[\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}\right]^{2}}$
$\displaystyle\times\left\\{\left[\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}\right]\frac{1}{T}\sum_{t=1}^{T}\left\\{\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\psi_{t,k}(\theta)+\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t,k}(\theta)}{\partial\theta_{j}}\right\\}\right.$
$\displaystyle\left.\hskip
36.135pt-\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right]\left[\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}\psi_{i,k}(\theta)\right]\right\\}.$

First derivative $\frac{\partial M_{1,T}(\theta,\tau)}{\partial\tau_{k}}$. By
definition of $M_{1,T}(\theta,\tau)$ in Lemma 12 (p. 12), for all
$(\theta,\tau)\in\mathbf{S}$, for all $k\in[\\![1,m]\\!]$,

$\displaystyle\frac{\partial M_{1,T}(\theta,\tau)}{\partial\tau_{k}}$
$\displaystyle=$
$\displaystyle\left(1-\frac{m}{2T}\right)\frac{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta)}{\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}}.$
(28)

Second derivative
$\frac{\partial^{2}M_{1,T}(\theta,\tau)}{\partial\tau_{h}\partial\tau_{k}}$.
By the above equation (28), for all $(\theta,\tau)\in\mathbf{S}$, for all
$(h,k)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\frac{\partial^{2}M_{1,T}(\theta,\tau)}{\partial\tau_{h}\partial\tau_{k}}$
(29) $\displaystyle=$
$\displaystyle\left(1-\frac{m}{2T}\right)\frac{1}{\left[\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}\right]^{2}}\times\left\\{\left[\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}\right]\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,h}(\theta)\psi_{t,k}(\theta)\right]\right.$
$\displaystyle\left.-\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,h}(\theta)\right]\left[\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}\psi_{i,k}(\theta)\right]\right\\}.$

####  B.2.2. Derivatives of
$M_{2,T}(\theta,\tau):=\frac{1}{2T}\ln\left[\left|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right|_{\det}^{2}\right]$

First derivative $\frac{\partial M_{2,T}(\theta,\tau)}{\partial\theta_{j}}$.
If $F(.)$ is a differentiable matrix function s.t. $|F(x)|_{\det}\neq 0$, then
$D\ln[|F(x)|_{\det}^{2}]=2{\rm tr}[F(x)^{-1}DF(x)]$ (Lemma 32ii on p. 32)
where $DF(x)$ denotes the derivative of $F(.)$ at $x$. Now, under Assumption
1, by Lemma 11iii (p. 11), $\mathbb{P}$-a.s. for $T$ big enough, for all
$(\theta,\tau)$ in a neighborhood of $(\theta_{0},\tau_{0})$,
$\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}$
is invertible. In addition, under Assumption 1(a), by Assumption 2(a),
$\theta\mapsto\psi(X_{1},\theta)$ is twice differentiable in a neighborhood of
$\theta_{0}$ $\mathbb{P}$-a.s., so that, under Assumption 1(a)-(e) and
(g)-(h), by Lemma 11ii (p. 11),
$(\theta,\tau)\mapsto\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}$
is also differentiable in a neighborhood of $(\theta_{0},\tau_{0})$
$\mathbb{P}$-a.s. Thus, under Assumptions 1 and 2(a), $\mathbb{P}$-a.s. for
$T$ big enough, for all $(\theta,\tau)$ in a neighborhood of
$(\theta_{0},\tau_{0})$, for all $j\in[\\![1,m]\\!]$,

$\displaystyle\frac{\partial
M_{2,T}(\theta,\tau)}{\partial\theta_{j}}=\frac{1}{T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\right.$
$\displaystyle\negthickspace\times\negthickspace\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial^{2}\psi_{t}(\theta)}{\partial\theta_{j}\partial\theta^{\prime}}\left.\negthickspace+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\negthickspace\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right)\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]\right\\}$
(30)

Second derivative
$\frac{\partial^{2}M_{2,T}(\theta,\tau)}{\partial\theta_{\ell}\partial\theta_{j}}$.
The trace of a derivative is the derivative of the trace because both the
trace and derivative operators are linear (e.g., Magnus and Neudecker
1999/1988, chap. 9 sec. 9). Moreover, if $F(.)$ is a differentiable matrix
function s.t., for all $x$ in a neighborhood of $\dot{x}$, $|F(x)|_{\det}\neq
0$, then
$D\left[F(\dot{x})^{-1}\right]=-F(\dot{x})^{-1}[DF(\dot{x})]F(\dot{x})^{-1}$
(e.g., Magnus and Neudecker 1999/1988, chap. 8 sec. 4). Now, as explained for
the first derivative, under Assumption 1, by Lemma 11iii (p. 11),
$\mathbb{P}$-a.s. for $T$ big enough, for all $(\theta,\tau)$ in a
neighborhood of $(\theta_{0},\tau_{0})$,
$\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}$
is invertible. In addition, by Assumption 2(a), $\mathbb{P}$-a.s.
$\theta\mapsto\psi(X_{1},\theta)$ is three times continuously differentiable
in a neighborhood of $\theta_{0}$, so that, under Assumption 1 and 2(a),
$\theta\mapsto\frac{\partial M_{2,T}(\theta,\tau)}{\partial\theta_{j}}$ is
differentiable in a neighborhood of $(\theta_{0},\tau_{0})$. Thus, under
Assumptions 1 and 2(a), by the above equation (30), $\mathbb{P}$-a.s. for $T$
big enough, for all $(\theta,\tau)$ in a neighborhood of
$(\theta_{0},\tau_{0})$, for all $(\ell,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\frac{\partial^{2}M_{2,T}(\theta,\tau)}{\partial\theta_{\ell}\partial\theta_{j}}=\frac{1}{T}{\rm
tr}\left\\{-\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\right.$
$\displaystyle\hskip
7.22743pt\left\\{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial^{2}\psi_{t}(\theta)^{\prime}}{\partial\theta_{\ell}\partial\theta^{\prime}}+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\right)\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right\\}$
$\displaystyle\hskip
7.22743pt\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}$
$\displaystyle\hskip
14.45377pt\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial^{2}\psi_{t}(\theta)}{\partial\theta_{j}\partial\theta^{\prime}}+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right)\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]$
$\displaystyle\hskip
1.4457pt+\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}$
$\displaystyle\hskip
14.45377pt\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial^{3}\psi_{t}(\theta)}{\partial\theta_{\ell}\partial\theta_{j}\partial\theta^{\prime}}\right.+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\right)\frac{\partial^{2}\psi_{t}(\theta)}{\partial\theta_{j}\partial\theta^{\prime}}$
$\displaystyle\hskip
36.135pt+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right)\frac{\partial^{2}\psi_{t}(\theta)}{\partial\theta_{\ell}\partial\theta^{\prime}}$
$\displaystyle\hskip
36.135pt+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\tau^{\prime}\frac{\partial^{2}\psi_{t}(\theta)}{\partial\theta_{\ell}\partial\theta_{j}}\right)\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}$
$\displaystyle\hskip
36.135pt\left.\left.+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\negthickspace\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\right)\negthickspace\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right)\negthickspace\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]\right\\}$
(31)

Second derivative
$\frac{\partial^{2}M_{2,T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{j}}$.
By a reasoning similar to the one for the derivative
$\frac{\partial^{2}M_{2,T}(\theta,\tau)}{\partial\theta_{\ell}\partial\theta_{j}}$,
under Assumptions 1 and 2(a), by the above equation (30), $\mathbb{P}$-a.s.
for $T$ big enough, for all $(\theta,\tau)$ in a neighborhood of
$(\theta_{0},\tau_{0})$, for all $(k,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\frac{\partial^{2}M_{2,T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{j}}$
(32) $\displaystyle=\frac{1}{T}{\rm
tr}\left\\{-\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta)\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\right.$
$\displaystyle\hskip
36.135pt\left.\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}+\frac{\partial^{2}\psi_{t}(\theta)}{\partial\theta_{j}\partial\theta^{\prime}}\right)\right]\right\\}$
$\displaystyle+\frac{1}{T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta)\left(\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}+\frac{\partial^{2}\psi_{t}(\theta)}{\partial\theta_{j}\partial\theta^{\prime}}\right)\right]\right\\}$
$\displaystyle+\frac{1}{T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\frac{\partial\psi_{t,k}(\theta)}{\partial\theta_{j}}\right)\right]\right\\}.$
(33)

First derivative $\frac{\partial M_{2,T}(\theta,\tau)}{\partial\tau_{k}}$. If
$F(.)$ is a differentiable matrix function s.t. $|F(x)|_{\det}\neq 0$, then
$D\ln[|F(x)|_{\det}^{2}]=2{\rm tr}[F(x)^{-1}DF(x)]$ (Lemma 32ii on p. 32)
where $DF(x)$ denotes the derivative of $F(.)$ at $x$. Now, under Assumption
1, by Lemma 11iii (p. 11), $\mathbb{P}$-a.s. for $T$ big enough, for all
$(\theta,\tau)$ in a neighborhood of $(\theta_{0},\tau_{0})$,
$\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}$
is invertible. Thus, under Assumption 1, by definition of
$M_{2,T}(\theta,\tau)$ in Lemma 12 (p. 12), $\mathbb{P}$-a.s. for $T$ big
enough, for all $(\theta,\tau)$ in a neighborhood of $(\theta_{0},\tau_{0})$,
for all $k\in[\\![1,m]\\!]$,

$\displaystyle\frac{\partial M_{2,T}(\theta,\tau)}{\partial\tau_{k}}$
$\displaystyle=$ $\displaystyle\frac{1}{T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta)\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]\right\\}.$
(34)

Second derivative
$\frac{\partial^{2}M_{2,T}(\theta,\tau)}{\partial\tau_{h}\partial\tau_{k}}$.
The trace of a derivative is the derivative of the trace because both the
trace and derivative operators are linear (e.g., Magnus and Neudecker
1999/1988, chap. 9 sec. 9). Moreover, if $F(.)$ is a differentiable matrix
function s.t., for all $x$ in a neighborhood of $\dot{x}$, $|F(x)|_{\det}\neq
0$, then
$D\left[F(\dot{x})^{-1}\right]=-F(\dot{x})^{-1}[DF(\dot{x})]F(\dot{x})^{-1}$
(e.g., Magnus and Neudecker 1999/1988, chap. 8 sec. 4). Now, as explained for
the first derivative, under Assumption 1, by Lemma 11iii (p. 11),
$\mathbb{P}$-a.s. for $T$ big enough, for all $(\theta,\tau)$ in a
neighborhood of $(\theta_{0},\tau_{0})$,
$\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}$
is invertible. Thus, under Assumption 1, by the above equation (34),
$\mathbb{P}$-a.s. for $T$ big enough, for all $(\theta,\tau)$ in a
neighborhood of $(\theta_{0},\tau_{0})$, for all $(h,k)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\frac{\partial^{2}M_{2,T}(\theta,\tau)}{\partial\tau_{h}\partial\tau_{k}}$
$\displaystyle=-\frac{1}{T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,h}(\theta)\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\right.$
$\displaystyle\hskip
36.135pt\times\left.\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta)\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]\right\\}$
$\displaystyle+\frac{1}{T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta)\psi_{t,h}(\theta)\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]\right\\}.$
(35)

####  B.2.3. Derivatives of
$M_{3,T}(\theta,\tau)=-\frac{1}{2T}\ln\left[\left|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right|_{\det}\right]$

First derivative

$\frac{\partial M_{3,T}(\theta,\tau)}{\partial\theta_{j}}$. If $F(.)$ is a
differentiable matrix function s.t. $|F(x)|_{\det}>0$, then
$D\ln[|F(x)|_{\det}]={\rm tr}[F(x)^{-1}DF(x)]$ (e.g., Magnus and Neudecker
1999/1988, chap. 8 sec. 3). Now, under Assumption 1(a)-(b), (e) and (g)(h), by
Lemma 11iv (p. 11), $\mathbb{P}$-a.s. for $T$ big enough, for all
$(\theta,\tau)\in\mathbf{S}$,
$|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}|_{\det}>0$.
In addition, by Assumption 1(b), $\mathbb{P}$-a.s.
$\theta\mapsto\psi(X_{1},\theta)$ is continuously differentiable in
$\mathbf{\Theta}$, so that, $\mathbb{P}$-a.s. for $T$ big enough,
$\theta\mapsto M_{3,T}(\theta,\tau)$ is differentiable in $\mathbf{\Theta}$,
for all $(\theta,\tau)\in\mathbf{S}$. Thus, under Assumption 1(a)-(b) and
(e)(g)(h), $\mathbb{P}$-a.s. for $T$ big enough, for all
$(\theta,\tau)\in\mathbf{S}$, for all $j\in[\\![1,m]\\!]$,

$\displaystyle\frac{\partial M_{3,T}(\theta,\tau)}{\partial\theta_{j}}$
$\displaystyle=$ $\displaystyle-\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\right.$
(36) $\displaystyle\hskip
14.45377pt\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left\\{\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\psi_{t}(\theta)^{\prime}+\psi_{t}(\theta)\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta_{j}}\right\\}\right.$
$\displaystyle\hskip
18.06749pt\left.\left.+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\negthickspace\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right)\negthickspace\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]\right\\}$

Second derivative
$\frac{\partial^{2}M_{3,T}(\theta,\tau)}{\partial\theta_{\ell}\partial\theta_{j}}$.
The trace of a derivative is the derivative of the trace because both the
trace and differentiation operators are linear (e.g., Magnus and Neudecker
1999/1988, chap. 9 sec. 9). Moreover, if $F(.)$ is a differentiable matrix
function s.t., for all $x$ in a neighborhood of $\dot{x}$, $|F(x)|_{\det}\neq
0$, then
$D\left[F(\dot{x})^{-1}\right]=-F(\dot{x})^{-1}[DF(\dot{x})]F(\dot{x})^{-1}$
(e.g., Magnus and Neudecker 1999/1988, chap. 8 sec. 4). Now, under Assumption
1(a)-(b), (e) and (g)-(h), by Lemma 11iv (p. 11), $\mathbb{P}$-a.s. for $T$
big enough, for all $(\theta,\tau)\in\mathbf{S}$,
$\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}$
is invertible. In addition, by Assumption 2(a), $\mathbb{P}$-a.s.
$\theta\mapsto\psi(X_{1},\theta)$ is three times continuously differentiable
in a neighborhood of $\theta_{0}$, so that, under Assumption 1(a)-(e) and
(g)(h), by Lemma 11ii (p. 11), $\theta\mapsto\frac{\partial
M_{3,T}(\theta,\tau)}{\partial\theta_{j}}$ is differentiable in a neighborhood
of $(\theta_{0},\tau_{0})$. Thus, under Assumptions 1(a)(b), (e) and (g)(h),
and 2(a), by the above equation (36), $\mathbb{P}$-a.s. for $T$ big enough,
for all $(\theta,\tau)$ in a neighborhood of $(\theta_{0},\tau_{0})$, for all
$(\ell,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\frac{\partial^{2}M_{3,T}(\theta,\tau)}{\partial\theta_{\ell}\partial\theta_{j}}=-\frac{1}{2T}{\rm
tr}\left\\{-\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\right.$
$\displaystyle\times\negthickspace\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\negthickspace\left(\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\psi_{t}(\theta)^{\prime}\negthickspace+\negthickspace\psi_{t}(\theta)\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta_{\ell}}\right)\right.$
$\displaystyle\hskip
36.135pt\left.+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\negthickspace\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\right)\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]$
$\displaystyle\hskip
7.22743pt\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}$
$\displaystyle\hskip
14.45377pt\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\psi_{t}(\theta)^{\prime}+\psi_{t}(\theta)\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta_{j}}\right)\right.$
$\displaystyle\hskip
36.135pt\left.+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right)\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]$
$\displaystyle+\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}$
$\displaystyle\hskip
14.45377pt\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\frac{\partial^{2}\psi_{t}(\theta)}{\partial\theta_{\ell}\partial\theta_{j}}\psi_{t}(\theta)^{\prime}+\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta_{\ell}}\right.+\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta_{j}}+\psi_{t}(\theta)\frac{\partial^{2}\psi_{t}(\theta)^{\prime}}{\partial\theta_{\ell}\partial\theta_{j}}\right)$
$\displaystyle\hskip
36.135pt+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\right)\left(\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\psi_{t}(\theta)^{\prime}+\psi_{t}(\theta)\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta_{j}}\right)$
$\displaystyle\hskip
36.135pt+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right)\left(\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\psi_{t}(\theta)^{\prime}+\psi_{t}(\theta)\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta_{\ell}}\right)$
$\displaystyle\hskip
36.135pt+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\negthickspace\left(\tau^{\prime}\frac{\partial^{2}\psi_{t}(\theta)^{\prime}}{\partial\theta_{\ell}\partial\theta_{j}}\right)\negthickspace\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}$
$\displaystyle\hskip
36.135pt+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\negthickspace\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{\ell}}\right)\negthickspace\negthickspace\left.\left.\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\right)\negthickspace\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]\right\\}$
(37)

Second derivative
$\frac{\partial^{2}M_{3,T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{j}}$.
Follow a reasoning similar to the one for the derivative
$\frac{\partial^{2}M_{2,T}(\theta,\tau)}{\partial\theta_{\ell}\partial\theta_{j}}$.
The trace of a derivative is the derivative of the trace because both the
trace and differentiation operators are linear (e.g., Magnus and Neudecker
1999/1988, chap. 9 sec. 9). Moreover, if $F(.)$ is a differentiable matrix
function s.t., for all $x$ in a neighborhood of $\dot{x}$, $|F(x)|_{\det}\neq
0$, then
$D\left[F(\dot{x})^{-1}\right]=-F(\dot{x})^{-1}[DF(\dot{x})]F(\dot{x})^{-1}$
(e.g., Magnus and Neudecker 1999/1988, chap. 8 sec. 4). Now, under Assumption
1(a)-(b), (e) and (g)-(h), by Lemma 11iv (p. 11), $\mathbb{P}$-a.s. for $T$
big enough, for all $(\theta,\tau)\in\mathbf{S}$,
$\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}$
is invertible. Thus, under Assumptions 1 (a)-(b), (e), (g)(h), by the above
equation (36), $\mathbb{P}$-a.s. for $T$ big enough, for all
$(\theta,\tau)\in\mathbf{S}$, for all $(k,j)\in[\\![1,m]\\!]^{2}$

$\displaystyle\frac{\partial^{2}M_{3,T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{j}}$
(38) $\displaystyle=$ $\displaystyle\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta)\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\right.$
$\displaystyle\hskip
36.135pt\left.\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}+\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\psi_{t}(\theta)^{\prime}+\psi_{t}(\theta)\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta_{j}}\right)\right]\right\\}$
$\displaystyle-\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\right.$
$\displaystyle\hskip
36.135pt\left.\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta)\left(\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}+\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\psi_{t}(\theta)^{\prime}+\psi_{t}(\theta)\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta_{j}}\right)\right]\right\\}$
$\displaystyle-\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t,k}(\theta)}{\partial\theta_{j}}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]\right\\}.$

First derivative $\frac{\partial M_{3,T}(\theta,\tau)}{\partial\tau_{k}}$. If
$F(.)$ is a differentiable matrix function s.t. $|F(x)|_{\det}>0$, then
$D\ln[|F(x)|_{\det}]={\rm tr}[F(x)^{-1}DF(x)]$ (e.g., Magnus and Neudecker
1999/1988, chap. 8 sec. 3). Now, under Assumption 1(a)-(b)(e)(g)(h), by Lemma
11iv (p. 11), $\mathbb{P}$-a.s. for $T$ big enough, for all
$(\theta,\tau)\in\mathbf{S}$,
$|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}|_{\det}>0$.
Thus, under Assumption 1(a)-(b)(e)(g)(h), by definition of
$M_{3,T}(\theta,\tau)$ in Lemma 12 (p. 12), $\mathbb{P}$-a.s. for $T$ big
enough, for all $(\theta,\tau)\in\mathbf{S}$, for all $k\in[\\![1,m]\\!]$,

$\displaystyle\frac{\partial
M_{3,T}(\theta,\tau)}{\partial\tau_{k}}=-\frac{1}{2T}\frac{1}{\left|\frac{1}{T}\sum_{i=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{i}(\theta)}\psi_{i}(\theta)\psi_{i}(\theta)^{\prime}\right|_{\det}}$
(39)
$\displaystyle\times\left|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right|_{\det}$
$\displaystyle\times{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta)\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]\right\\}$
$\displaystyle=$ $\displaystyle-\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta)\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]\right\\}$

Second derivative
$\frac{\partial^{2}M_{3,T}(\theta,\tau)}{\partial\tau_{h}\partial\tau_{k}}$.
The trace of a derivative is the derivative of the trace because both the
trace and differentiation operators are linear (e.g., Magnus and Neudecker
1999/1988, chap. 9 sec. 9). Moreover, if $F(.)$ is a differentiable matrix
function s.t., for all $x$ in a neighborhood of $\dot{x}$, $|F(x)|_{\det}\neq
0$, then
$D\left[F(\dot{x})^{-1}\right]=-F(\dot{x})^{-1}[DF(\dot{x})]F(\dot{x})^{-1}$
(e.g., Magnus and Neudecker 1999/1988, chap. 8 sec. 4). Now, under Assumption
1(a)-(b)(e)(g)-(h), by Lemma 11iv (p. 11), $\mathbb{P}$-a.s. for $T$ big
enough, for all $(\theta,\tau)\in\mathbf{S}$,
$\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}$
is invertible. Thus, under Assumptions 1(a)-(b)(e)(g)(h), by the above
equation (39), $\mathbb{P}$-a.s. for $T$ big enough, for all $(\theta,\tau)$
in a neighborhood of $(\theta_{0},\tau_{0})$, for all
$(h,k)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\frac{\partial^{2}M_{3,T}(\theta,\tau)}{\partial\tau_{h}\partial\tau_{k}}$
(40) $\displaystyle=$ $\displaystyle\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta)\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]\right.$
$\displaystyle\hskip
36.135pt\left.\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,h}(\theta)\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]\right\\}$
$\displaystyle-\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t,k}(\theta)\psi_{t,h}(\theta)\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]\right\\}.$

####  B.2.4. Derivatives of $\theta\mapsto L_{T}(\theta,\tau)$

First derivative. Under Assumption 1(a)-(e) and (g)-(h) and 2(a), by Lemma
11ii (p. 11), $\mathbf{S}$ contains an open neighborhood of
$(\theta_{0},\tau_{0})$, so that the derivatives derived in $\mathbf{S}$ also
hold in a neighborhood of $(\theta_{0},\tau_{0})$. Thus, by equations (25),
(30) and (36) on pp. 25-36. Therefore, under Assumptions 1 and 2(a),
$\mathbb{P}$-a.s. for $T$ big enough, for all $(\theta,\tau)$ in a
neighborhood of $(\theta_{0},\tau_{0})$,

$\displaystyle\frac{\partial L_{T}(\theta,\tau)}{\partial\theta_{j}}$
$\displaystyle=$
$\displaystyle\left(1-\frac{m}{2T}\right)\frac{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}}{\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}}$
$\displaystyle\hskip 7.22743pt+\frac{1}{T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]^{-1}\right.$
$\displaystyle\hskip
14.45377pt\times\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial^{2}\psi_{t}(\theta)}{\partial\theta_{j}\partial\theta^{\prime}}\left.+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}\right]\right\\}$
$\displaystyle\hskip 7.22743pt-\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]^{-1}\right.$
$\displaystyle\hskip
14.45377pt\negthickspace\times\negthickspace\left[\frac{1}{T}\sum_{t=1}^{T}\negthickspace\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\negthickspace\left\\{\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\psi_{t}(\theta)^{\prime}\negthickspace+\negthickspace\psi_{t}(\theta)\frac{\partial\psi_{t}(\theta)^{\prime}}{\partial\theta_{j}}\right\\}\left.\negthickspace+\frac{1}{T}\sum_{t=1}^{T}\negthickspace\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta_{j}}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}\right]\negthickspace\right\\}$

Thus, evaluated at $(\theta_{0},\tau(\theta_{0}))$,

$\displaystyle\frac{\partial L_{T}(\theta_{0},\tau_{0})}{\partial\theta_{j}}$
(41) $\displaystyle=$ $\displaystyle\frac{1}{T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\frac{\partial\psi_{t}(\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\frac{\partial^{2}\psi_{t}(\theta_{0})}{\partial\theta_{j}\partial\theta^{\prime}}\right]\right\\}$
$\displaystyle\hskip 7.22743pt-\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\psi_{t}(\theta_{0})^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\negthickspace\left\\{\frac{\partial\psi_{t}(\theta_{0})}{\partial\theta_{j}}\psi_{t}(\theta_{0})^{\prime}\negthickspace+\negthickspace\psi_{t}(\theta_{0})\frac{\partial\psi_{t}(\theta_{0})^{\prime}}{\partial\theta_{j}}\right\\}\right]\right\\}$

because $\tau(\theta_{0})=0_{m\times 1}$ by Lemma 10iv (p. 10).

###  B.3. Proof of Theorem 1(ii) : Asymptotic normality

The proof of Theorem 1(ii) (i.e., asymptotic normality) adapts the traditional
approach of expanding the FOCs (first order conditions). The two main
differences w.r.t. the proofs in the entropy literature are the following.
Firstly, instead of expanding the FOC $\left.\frac{\partial
L_{T}(\theta,\tau_{T}(\theta))}{\partial\theta}\right|_{\theta=\hat{\theta}_{T}}$,
we expand the approximate FOC $\left.\frac{\partial
L_{T}(\theta,\tau)}{\partial\theta}\right|_{(\theta,\tau)=(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}$
combined with the FOC (15) for $\tau$ on p. 15. Secondly, we need to control
the asymptotic behaviour of the derivatives that come from
$\ln|\Sigma_{T}(\theta)|_{\det}$.

###### Core of the proof of Theorem 1(ii).

We prove asymptotic normality adapting the traditional approach of expanding
the FOCs (first order conditions). Note that our approximate FOCs are written
as a functionof the $2m$ variables $\theta$ and $\tau$. In other words,
instead of using the implicit function $\tau_{T}(\theta),$ $\tau$ is an
estimated parameter and hence the ET equation (15) on p. 15 is also included
in the expansion.

Under Assumptions 1 and 2, by Proposition 1 (p. 1), $\mathbb{P}$-a.s. as
$T\rightarrow\infty$,

$\displaystyle\sqrt{T}\left[\begin{array}[]{c}\left(\hat{\theta}_{T}-\theta_{0}\right)\\\
\tau_{T}(\hat{\theta}_{T})\end{array}\right]$ $\displaystyle=$
$\displaystyle-\begin{bmatrix}\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\\\
0_{m\times m}\\\
\end{bmatrix}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$
$\displaystyle\underset{(a)}{\stackrel{{\scriptstyle D}}{{\rightarrow}}}$
$\displaystyle-\begin{bmatrix}\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\\\
0_{m\times m}\\\
\end{bmatrix}\mathcal{N}(0,\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right])$
$\displaystyle\underset{(b)}{\stackrel{{\scriptstyle D}}{{=}}}$
$\displaystyle\mathcal{N}\left(0,\begin{bmatrix}\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\\\
0_{m\times m}\\\
\end{bmatrix}\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\begin{bmatrix}\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})^{\prime}}{\partial\theta}\right]^{-1}&0_{m\times
m}\\\ \end{bmatrix}\right)$ $\displaystyle\underset{}{\stackrel{{\scriptstyle
D}}{{=}}}$ $\displaystyle\mathcal{N}\left(0,\left(\begin{array}[]{c c
}\Sigma(\theta_{0})&0_{m\times m}\\\ 0_{m\times m}&0_{m\times
m}\end{array}\right)\right)\text{ }$ (47)

where
$\Sigma(\theta_{0})=\left[\mathbb{E}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\negthickspace\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\left[\mathbb{E}\frac{\partial\psi(X_{1},\theta_{0})^{\prime}}{\partial\theta}\right]^{-1}$.
(a) Under Assumption 1(a)-(c) and (g), by the Lindeberg-Lévy CLT theorem,

$\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\stackrel{{\scriptstyle
D}}{{\rightarrow}}\mathcal{N}(0,\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}]\right)$,
as $T\rightarrow\infty$. (b) Firstly, the minus sign can be discarded because
of the symmetry of the Gaussian distribution. Secondly, if $X$ is a random
vector and $F$ is a (deterministic) matrix, then
$\mathbb{V}(FX)=F\mathbb{V}(X)F^{\prime}$. ∎

######  Proposition 1 (Asymptotic expansion of
$\sqrt{T}(\hat{\theta}_{T}-\theta_{0})$).

Under Assumptions 1 and 2, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,

$\displaystyle\sqrt{T}\left[\begin{array}[]{c}\left(\hat{\theta}_{T}-\theta_{0}\right)\\\
\tau_{T}(\hat{\theta}_{T})\end{array}\right]=-\begin{bmatrix}\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\\\
0_{m\times m}\\\
\end{bmatrix}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$

###### Proof.

The function $L_{T}(\theta,\tau)$ is well-defined and twice continuously
differentiable in a neighborhood of
$(\theta_{0}^{\prime}\;\tau(\theta_{0})^{\prime})$ $\mathbb{P}$-a.s. for $T$
big enough by subsection B.2 (p. B.2), under Assumptions 1 and 2(a).
Similarly, let
$S_{T}(\theta,\tau):=\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)$,
which is continuously differentiable in a neighborhood of
$(\theta_{0}^{\prime}\;\tau(\theta_{0})^{\prime})$ by Assumption 1(a)(b). Now,
under Assumption 1, by Theorem 1i (p. 1), Lemma 2iii (p. 2) and Lemma 10iv (p.
10), $\mathbb{P}$-a.s., $\hat{\theta}_{T}\rightarrow\theta_{0}$ and
$\tau_{T}(\hat{\theta}_{T})\rightarrow\tau(\theta_{0})$, where
$\tau(\theta_{0})=0_{m\times 1}$, so that $\mathbb{P}$-a.s. for $T$ big
enough, $(\hat{\theta}_{T}^{\prime}\;\tau_{T}(\hat{\theta}_{T})^{\prime})$ is
in any arbitrary small neighborhood of
$(\theta_{0}^{\prime}\;\tau(\theta_{0})^{\prime})$. Therefore, under
Assumption 1 and 2(a), a stochastic first-order Taylor-Lagrange expansion
(Jennrich 1969, Lemma 3) around $(\theta_{0},\tau(\theta_{0}))$ evaluated at
$(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))$ yields, $\mathbb{P}$-a.s. for
$T$ big enough

$\displaystyle\left[\begin{array}[]{c}\frac{\partial{L}_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}\\\
S_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))\end{array}\right]=\left[\begin{array}[]{c}\frac{\partial{L}_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta}\\\
S_{T}(\theta_{0},\tau(\theta_{0}))\end{array}\right]+\left[\begin{array}[]{c c
}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}\\\
\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}\end{array}\right]\left[\begin{array}[]{c}\left(\hat{\theta}_{T}-\theta_{0}\right)\\\
\tau_{T}(\hat{\theta}_{T})\end{array}\right]$ (57)

where $\bar{\theta}_{T}$ and $\bar{\tau}_{T}$ are between $\hat{\theta}_{T}$
and $\theta_{0}$, and between $\tau_{T}(\hat{\theta}_{T})$ and
$\tau(\theta_{0})$, respectively. Under Assumptions 1 and 2, by Lemma 20 (p.
20) and by definition of $\tau_{T}(.)$ (equation 15 on p. 15),
$\frac{\partial{L}_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}=O(T^{-1})$
and $S_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))=0$, respectively.
Moreover, under Assumptions 1 and 2, by Theorem 1i, Lemma 2iii (p. 2) and
Lemma 13ii (p. 13), $\mathbb{P}$-a.s. for $T$ big enough,
$\left[\begin{array}[]{c c
}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}\\\
\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}\end{array}\right]$
is invertible. Thus, under Assumptions 1 and 2, $\mathbb{P}$-a.s. for $T$ big
enough,

$\displaystyle\sqrt{T}\left[\begin{array}[]{c}\left(\hat{\theta}_{T}-\theta_{0}\right)\\\
\tau_{T}(\hat{\theta}_{T})\end{array}\right]$ $\displaystyle=$
$\displaystyle-\left[\begin{array}[]{c c
}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}\\\
\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}\end{array}\right]^{-1}\sqrt{T}\left[\begin{array}[]{c}\frac{\partial{L}_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta}+O(T^{-1})\\\
S_{T}(\theta_{0},\tau(\theta_{0}))\end{array}\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{=}}$
$\displaystyle-\left[\begin{array}[]{c c
}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}\\\
\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}\end{array}\right]^{-1}\left[\begin{array}[]{c}O(T^{-\frac{1}{2}})\\\
\sqrt{T}\frac{1}{T}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\end{array}\right]$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{=}}$
$\displaystyle-\left[\begin{array}[]{c c
}-\Sigma(\theta_{0})&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})^{\prime}}{\partial\theta}\right]^{-1}&\
0_{m\times m}\end{array}\right]\left[\begin{array}[]{c}O(T^{-\frac{1}{2}})\\\
\sqrt{T}\frac{1}{T}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\end{array}\right]$
$\displaystyle-\left\\{\left[\begin{array}[]{c c
}\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}\\\
\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}\end{array}\right]^{-1}-\left[\begin{array}[]{c
c
}-\Sigma(\theta_{0})&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})^{\prime}}{\partial\theta}\right]^{-1}&\
0_{m\times
m}\end{array}\right]\right\\}\left[\begin{array}[]{c}O(T^{-\frac{1}{2}})\\\
\sqrt{T}\frac{1}{T}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\end{array}\right]$
$\displaystyle\underset{}{\stackrel{{\scriptstyle(c)}}{{=-}}}$
$\displaystyle\begin{bmatrix}\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\\\
0_{m\times m}\\\
\end{bmatrix}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})+o_{\mathbb{P}}(1)$

where
$\Sigma(\theta_{0})=\left[\mathbb{E}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\negthickspace\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\left[\mathbb{E}\frac{\partial\psi(X_{1},\theta_{0})^{\prime}}{\partial\theta}\right]^{-1}$.
(a) Firstly, under Assumptions 1 and 2, by Lemma 14i (p. 14),
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, $\frac{\partial
L_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta_{j}}=O(T^{-1})$, so that
$\sqrt{T}\left[\frac{\partial
L_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta_{j}}+O(T^{-1})\right]=O(T^{-\frac{1}{2}})$.
Secondly, note that
$S_{T}(\theta_{0},\tau(\theta_{0}))=\frac{1}{T}\sum_{t=1}^{T}\psi_{t}(\theta_{0})$.
(b) Add and subtract the matrix $\left[\begin{array}[]{c c
}-\Sigma(\theta_{0})&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})^{\prime}}{\partial\theta}\right]^{-1}&\
0_{m\times m}\end{array}\right]$. (c) Firstly, the first column of the first
square matrix cancels out because the first element of the vector is zero.
Secondly, under Assumptions 1 and 2, by Lemma 13iii (p. 13) and Theorem 1i (p.
1), $\mathbb{P}$-a.s. as $T\rightarrow\infty$, the curly bracket is $o(1)$,
and, under Assumption 1(a)-(c) and (g), by the Lindeberg-L vy CLT,
$\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\psi_{t}(\theta_{0})=O_{\mathbb{P}}(1)$, as
$T\rightarrow\infty$. ∎

######  Remark 2 (Alternative approximate FOC).

In the proof of Theorem 1ii, it is possible to use the approximate FOC
$\frac{\partial
M_{1,T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}=O(T^{-1})$
instead of the approximate FOC
$\frac{\partial{L}_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}=O(T^{-1})$.
Under Assumption 1 and 2 (with $k_{2}\in[\\![1,3]\\!]$ and $j\in[\\![0,2]\\!]$
in its part b), by Lemma 12 (p. 12) and 18v-vii,xii-xiv (p. 18) and the ULLN à
la Wald,
$\frac{\partial{L}_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}=\frac{\partial
M_{1,T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}+\frac{\partial
M_{2,T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}+\frac{\partial
M_{3,T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}=\frac{\partial
M_{1,T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}+O(T^{-1})$.
The approximate FOC $\frac{\partial
M_{1,T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}=O(T^{-1})$
would lead to replace expansion (57) on p. 57 with the following expansion

$\displaystyle\left[\begin{array}[]{c}\frac{\partial
M_{1,T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}\\\
S_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))\end{array}\right]=\left[\begin{array}[]{c}\frac{\partial
M_{1,T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta}\\\
S_{T}(\theta_{0},\tau(\theta_{0}))\end{array}\right]+\left[\begin{array}[]{c c
}\frac{\partial^{2}M_{1,T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}M_{1,T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}\\\
\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\tau^{\prime}}\end{array}\right]\left[\begin{array}[]{c}\left(\hat{\theta}_{T}-\theta_{0}\right)\\\
\tau_{T}(\hat{\theta}_{T})\end{array}\right]$

where
$\frac{\partial^{2}M_{1,T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}$
and
$\frac{\partial^{2}M_{1,T}(\bar{\theta}_{T},\bar{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}$
can easily be controlled by Lemma 18i-iv (p. 18), Lemma 19i-v (p. 19), Lemma
23i-iii (p. 23) and ULLN à la Wald under Assumptions 1 and 2 (with
$k_{2}\in[\\![1,3]\\!]$ and $j\in[\\![0,2]\\!]$ in its part b). The
approximate FOC $\frac{\partial
M_{1,T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}=O(T^{-1})$
requires less assumptions than the approximate FOC
$\frac{\partial{L}_{T}(\hat{\theta}_{T},\tau_{T}(\hat{\theta}_{T}))}{\partial\theta}=O(T^{-1})$
because it does not require to control the 2nd derivatives of
$M_{2,T}(\theta,\tau)$ and $M_{3,T}(\theta,\tau)$. However, it would not save
space and it would require to add one more block of assumptions because our
proof of Theorem 2 requires the full Assumption 2. $\diamond$

######  Lemma 13.

Under Assumptions 1 and 2,

  1. (i)

for any sequence $(\theta_{T},\tau_{T})_{T\in\mathbf{N}}$ converging to
$(\theta_{0},\tau(\theta_{0}))$, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$\left[\begin{array}[]{c c
}\frac{\partial^{2}{L}_{T}({\theta}_{T},{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}({\theta}_{T},{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}\\\
\frac{\partial
S_{T}({\theta}_{T},{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}({\theta}_{T},{\tau}_{T})}{\partial\tau^{\prime}}\end{array}\right]\rightarrow\left[\begin{array}[]{c
c }0_{m\times
m}&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{\prime}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]&\
\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\end{array}\right]$;

  2. (ii)

$\left[\begin{array}[]{c c }0_{m\times
m}&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{\prime}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]&\
\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\end{array}\right]$
is invertible, so that, for any sequence
$(\theta_{T},\tau_{T})_{T\in\mathbf{N}}$ converging to
$(\theta_{0},\tau(\theta_{0}))$, $\mathbb{P}$-a.s., for $T$ big enough, the
matrix $\left[\begin{array}[]{c c
}\frac{\partial^{2}{L}_{T}({\theta}_{T},{\tau}_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}({\theta}_{T},{\tau}_{T})}{\partial\tau^{\prime}\partial\theta}\\\
\frac{\partial
S_{T}({\theta}_{T},{\tau}_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}({\theta}_{T},{\tau}_{T})}{\partial\tau^{\prime}}\end{array}\right]$ is
invertible; and

  3. (iii)

for any sequence $(\theta_{T},\tau_{T})_{T\in\mathbf{N}}$ converging to
$(\theta_{0},\tau(\theta_{0}))$, $\mathbb{P}$-a.s. as $T\rightarrow\infty$,
$\left[\begin{array}[]{c c
}\frac{\partial^{2}{L}_{T}(\theta_{T},\tau_{T})}{\partial\theta^{\prime}\partial\theta}&\frac{\partial^{2}{L}_{T}(\theta_{T},\tau_{T})}{\partial\tau^{\prime}\partial\theta}\\\
\frac{\partial
S_{T}(\theta_{T},\tau_{T})}{\partial\theta^{\prime}}&\frac{\partial
S_{T}(\theta_{T},\tau_{T})}{\partial\tau^{\prime}}\end{array}\right]^{-1}\rightarrow\left[\begin{array}[]{c
c
}-\Sigma(\theta_{0})&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})^{\prime}}{\partial\theta}\right]^{-1}&\
0_{m\times m}\end{array}\right]$, where
$\displaystyle\left[\begin{array}[]{c c
}-\Sigma(\theta_{0})&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})^{\prime}}{\partial\theta}\right]^{-1}&\
0_{m\times m}\end{array}\right]=\left[\begin{array}[]{c c }0_{m\times
m}&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{\prime}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]&\
\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\end{array}\right]^{-1}$
.

###### Proof.

(i) Under Assumptions 1 and 2, it follows from Lemma 14ii and iii (p. 14) and
Lemma 17 (p. 17), given that $\tau(\theta_{0})=0_{m\times 1}$ by Lemma 10ii
(p. 10) and Assumption 1(c), under Assumption 1(a)(b)(d)(e)(g) and (h).

(ii) Assumption 1(h) implies the invertibility of

$\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1},\theta_{0})}\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]=\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]$
and
$\mathbb{E}\left[\mathrm{e}^{\tau(\theta_{0})^{\prime}\psi(X_{1},\theta_{0})}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]=\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]$
because $\tau(\theta_{0})=0_{m\times 1}$ by Lemma 10iv (p. 10) under
Assumption 1(a)-(e)(g)-(h). Thus,
$\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})^{\prime}}{\partial\theta}\right]\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]^{-1}\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]$
is also invertible, so that the first part of the statement (ii) follows from
Lemma 33ii (p. 33) with $A=0_{m\times m}$,
$B=\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})^{\prime}}{\partial\theta}\right]$,
$C=\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]$
and
$D=\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]$
. Then, the second part of the statement follows from a trivial case of the
Lemma 30 (p. 30).
(iii) Under under Assumption 1(a)(b)(c)(d)(e)(g)(h), by the statement (ii) of
the present lemma, the limiting matrix is invertible. Thus, by the inverse
formula for partitioned matrices (e.g., Magnus and Neudecker 1999/1988, Chap.
1 Sec. 11),

$\displaystyle\left[\begin{array}[]{c c }0_{m\times
m}&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{\prime}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]&\
\mathbb{E}\left[\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]\end{array}\right]^{-1}=\left[\begin{array}[]{c
c
}-\Sigma(\theta_{0})&\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\\\
\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})^{\prime}}{\partial\theta}\right]^{-1}&\
0_{m\times 1}\end{array}\right]$

because
$(-M^{\prime}V^{-1}M)^{-1}=-M^{-1}V(M^{\prime})^{-1}:=-\Sigma(\theta_{0})$.
Then, the result follows from the continuity of the inverse transformation
(e.g., Rudin 1953, Theorem 9.8). ∎

######  Lemma 14.

Under Assumptions 1 and 2,

  1. (i)

$\mathbb{P}$-a.s. as $T\rightarrow\infty$,

$T\frac{\partial
L_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta_{j}}\rightarrow{\rm
tr}\left\\{\left[\mathbb{E}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\mathbb{E}\left[\frac{\partial^{2}\psi_{t}(\theta_{0})}{\partial\theta_{j}\partial\theta^{\prime}}\right]\right\\}-\frac{1}{2}{\rm
tr}\Big{\\{}\left[\mathbb{E}\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]^{-1}\\\
\times\Big{[}\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta_{j}}\psi(X_{1},\theta_{0})^{\prime}\right]+\mathbb{E}\left[\psi(X_{1},\theta_{0})\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta_{j}}^{\prime}\right]\Big{]}\Big{\\}}$,
so that $\frac{\partial
L_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta_{j}}=O(T^{-1})$;

  2. (ii)

for any sequence $(\theta_{T},\tau_{T})_{T\in\mathbf{N}}$ converging to
$(\theta_{0},\tau(\theta_{0}))$,
$\displaystyle\left|\frac{\partial^{2}L_{T}(\theta_{T},\tau_{T})}{\partial\theta_{j}\partial\theta_{\ell}}\right|=o(1)$,
$\mathbb{P}$-a.s. as $T\rightarrow\infty$;

  3. (iii)

for any sequence $(\theta_{T},\tau_{T})_{T\in\mathbf{N}}$ converging to
$(\theta_{0},\tau(\theta_{0}))$,
$\displaystyle\left|\frac{\partial^{2}L_{T}(\theta_{T},\tau_{T})}{\partial\theta^{\prime}\partial\tau}-\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]\right|=o(1)$,
$\mathbb{P}$-a.s. as $T\rightarrow\infty$.

###### Proof.

(i) By equation (41) on p. 41, under Assumptions 1 and 2(a), for all
$j\in[\\![1,m]\\!]$, $\mathbb{P}$-a.s. for $T$ big enough, evaluating
$\frac{\partial L_{T}(\theta,\tau)}{\partial\theta_{j}}$ at
$(\theta_{0},\tau(\theta_{0}))$ yields

$\displaystyle\frac{\partial
L_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta_{j}}$ (65) $\displaystyle=$
$\displaystyle\frac{1}{T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\frac{\partial\psi_{t}(\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\frac{\partial^{2}\psi_{t}(\theta_{0})}{\partial\theta_{j}\partial\theta^{\prime}}\right]\right\\}$
$\displaystyle\hskip 7.22743pt-\frac{1}{2T}{\rm
tr}\left\\{\left[\frac{1}{T}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\psi_{t}(\theta_{0})^{\prime}\right]^{-1}\left[\frac{1}{T}\sum_{t=1}^{T}\negthickspace\left\\{\frac{\partial\psi_{t}(\theta_{0})}{\partial\theta_{j}}\psi_{t}(\theta_{0})^{\prime}\negthickspace+\negthickspace\psi_{t}(\theta_{0})\frac{\partial\psi_{t}(\theta_{0})^{\prime}}{\partial\theta_{j}}\right\\}\right]\right\\}.$

Now, under Assumption 1(a)(b),

  * •

under additional Assumption 1(h), by the LLN and Lemma 30 (p. 30),
$\mathbb{P}$-a.s. for $T$ big enough,
$\frac{1}{T}\sum_{t=1}^{T}\frac{\partial\psi_{t}(\theta_{0})}{\partial\theta^{\prime}}$
is invertible, so that
$\left[\frac{1}{T}\sum_{t=1}^{T}\frac{\partial\psi_{t}(\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\rightarrow\left[\mathbb{E}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}$;

  * •

under additional Assumption 2(b), by the LLN,
$\frac{1}{T}\sum_{t=1}^{T}\frac{\partial^{2}\psi_{t}(\theta_{0})}{\partial\theta_{j}\partial\theta^{\prime}}\rightarrow\mathbb{E}\left[\frac{\partial^{2}\psi_{t}(\theta_{0})}{\partial\theta_{j}\partial\theta^{\prime}}\right]$;

  * •

under additional Assumption 1(h), by the LLN and Lemma 30 (p. 30),
$\mathbb{P}$-a.s. for $T$ big enough,
$\frac{1}{T}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\psi_{t}(\theta_{0})^{\prime}$
is invertible, so that
$\left[\frac{1}{T}\sum_{t=1}^{T}\psi_{t}(\theta_{0})\psi_{t}(\theta_{0})^{\prime}\right]^{-1}\rightarrow\left[\mathbb{E}\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]^{-1}$;
and

  * •

under additional Assumption 1(f)(g), by the Cauchy-Schwarz inequality and the
monotonicity of integration,

$\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta_{j}}\psi(X_{1},\theta_{0})\right]\leqslant\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|^{2}\right]\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}}|\psi(X_{1},\theta)|^{2}\right]}<\infty$,
so that, by the LLN,

$\frac{1}{T}\sum_{t=1}^{T}\negthickspace\left\\{\frac{\partial\psi_{t}(\theta_{0})}{\partial\theta_{j}}\psi_{t}(\theta_{0})^{\prime}\negthickspace+\negthickspace\psi_{t}(\theta_{0})\frac{\partial\psi_{t}(\theta_{0})^{\prime}}{\partial\theta_{j}}\right\\}\rightarrow\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta_{j}}\psi(X_{1},\theta_{0})^{\prime}\right]+\mathbb{E}\left[\psi(X_{1},\theta_{0})\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta_{j}}^{\prime}\right]$
$\mathbb{P}$-a.s. as $T\rightarrow\infty$.

Thus, under Assumptions 1 and 2, for all $j\in[\\![1,m]\\!]$,
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, $T\frac{\partial
L_{T}(\theta_{0},\tau(\theta_{0}))}{\partial\theta_{j}}\rightarrow{\rm
tr}\left\\{\left[\mathbb{E}\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta^{\prime}}\right]^{-1}\mathbb{E}\left[\frac{\partial^{2}\psi_{t}(\theta_{0})}{\partial\theta_{j}\partial\theta^{\prime}}\right]\right\\}-\frac{1}{2}{\rm
tr}\Big{\\{}\left[\mathbb{E}\psi(X_{1},\theta_{0})\psi(X_{1},\theta_{0})^{\prime}\right]^{-1}\Big{[}\mathbb{E}\left[\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta_{j}}\psi(X_{1},\theta_{0})^{\prime}\right]\\\
+\mathbb{E}\left[\psi(X_{1},\theta_{0})\frac{\partial\psi(X_{1},\theta_{0})}{\partial\theta_{j}}^{\prime}\right]\Big{]}\Big{\\}}$,
so that $\frac{\partial
L_{T}(\theta_{0},\tau_{0})}{\partial\theta_{j}}=\frac{1}{T}O(1)=O(T^{-1})$.

(ii) Under Assumptions 1 and 2, by Lemma 15 (p. 15) and Lemma 12 (p. 12),
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, uniformly over a closed ball around
$(\theta_{0},\tau(\theta_{0}))$ with strictly positive radius,
$\frac{\partial^{2}L_{T}(\theta,\tau)}{\partial\theta_{j}\partial\theta_{\ell}}\rightarrow\frac{1}{\left[\mathbb{E}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\right]}\mathbb{E}\left\\{\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\left[\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\right]\left[\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\right]+\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\left[\tau^{\prime}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta_{\ell}}\right]\right\\}\\\
-\frac{1\frac{}{}}{\left[\mathbb{E}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\right]^{2}}\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\right]\times\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\right]$.
Now, under Assumption 1(a)(b)(d) (e)(g) and (h), by Lemma 10ii (p. 10) and
Assumption 1(c), put $\tau(\theta_{0})=0_{m\times 1}$, so that the result
follows.

(iii) Under Assumptions 1 and 2, by Lemma 16 (p. 16) and Lemma 12 (p. 12),
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, uniformly over a closed ball around
$(\theta_{0},\tau(\theta_{0}))$ with strictly positive radius,
$\frac{\partial^{2}L_{T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{\ell}}\rightarrow\frac{1}{\left[\mathbb{E}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\right]^{2}}\times\left\\{\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\right]\mathbb{E}\left\\{\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\psi_{k}(X_{1,}\theta)+\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi_{k}(X_{1,}\theta)}{\partial\theta_{\ell}}\right\\}\right.\\\
\left.\hskip
36.135pt-\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\right]\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1,}\theta)\right]\right\\}$.
Now, under Assumption 1(a)(b)(d)(e)(g) and (h), by Lemma 10ii (p. 10) and
Assumption 1(c), $\tau(\theta_{0})=0_{m\times 1}$, so that $\mathbb{P}$-a.s.
as $T\rightarrow\infty$,
$\frac{\partial^{2}L_{T}(\theta_{T},\tau_{T})}{\partial\tau_{k}\partial\theta_{\ell}}\rightarrow\mathbb{E}\left[\frac{\partial\psi_{k}(X_{1,}\theta)}{\partial\theta_{\ell}}\right]$.
Stack the components together in order to obtain the result.

∎

######  Lemma 15 (Uniform limit of
$\frac{\partial^{2}L_{T}(\theta,\tau)}{\partial\theta_{j}\partial\theta_{\ell}}$
in a neighborhood of $(\theta_{0},\tau(\theta_{0}))$).

Under Assumptions 1 and 2, for all $(j,\ell)\in[\\![1,m]\\!]^{2}$,
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, uniformly over a closed ball around
$(\theta_{0},\tau(\theta_{0}))$ with strictly positive radius,

  1. (i)

$\frac{\partial^{2}M_{1,T}(\theta,\tau)}{\partial\theta_{j}\partial\theta_{\ell}}\rightarrow\frac{1}{\left[\mathbb{E}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\right]}\mathbb{E}\left\\{\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\left[\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\right]\left[\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\right]+\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\left[\tau^{\prime}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta_{\ell}}\right]\right\\}\\\
-\frac{1\frac{}{}}{\left[\mathbb{E}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\right]^{2}}\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\right]\times\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\right]$;

  2. (ii)

$\frac{\partial^{2}M_{2,T}(\theta,\tau)}{\partial\theta_{j}\partial\theta_{\ell}}\rightarrow
0$;

  3. (iii)

$\frac{\partial^{2}M_{3,T}(\theta,\tau)}{\partial\theta_{j}\partial\theta_{\ell}}\rightarrow
0$.

###### Proof.

(i) Under Assumptions 1 and 2, by Lemma 18i-iv (p. 18), Assumption 1(a) and
(b), all the averages in
$\frac{\partial^{2}M_{1,T}(\theta,\tau)}{\partial\theta_{j}\partial\theta_{\ell}}$
(equation (26) on p. 26) satisfy the assumptions of the ULLN à la Wald.
Moreover, under Assumption 1(a)-(b) (d)(e)(g) and (h), by Lemma 11i (p. 11)
the averages in the denominators are bounded away from zero. Thus, the result
follows from the ULLN à la Wald. Note that the coefficient $\frac{m}{2T}$
vanishes as it goes to zero, as $T\rightarrow\infty$.

(ii) Under Assumptions 1 and 2, by Lemma 18v-xi (p. 18), Assumption 1(a) and
(b), all the averages in
$\frac{\partial^{2}M_{2,T}(\theta,\tau)}{\partial\theta_{j}\partial\theta_{\ell}}$
(equation (31) on p. 31) satisfy the assumptions of the ULLN à la Wald.
Moreover, under Assumption 1, by Lemma 11iii (p. 11) the averages in the
inverted matrices are invertible in a neighborhood of
$(\theta_{0},\tau(\theta_{0}))$ $\mathbb{P}$-a.s. for $T$ big enough. Thus,
the result follows from the ULLN à la Wald, the linearity of the trace
operator and the scaling by $\frac{1}{T}$.

(iii) Under Assumptions 1 and 2, by Lemma 18xii-xix (p. 18), Assumption 1(a)
and (b), all the averages in
$\frac{\partial^{2}M_{3,T}(\theta,\tau)}{\partial\theta_{j}\partial\theta_{\ell}}$
(equation (37) on p. 37) satisfy the assumptions of the ULLN à la Wald.
Moreover, under Assumption 1(a)(b)(e)(g) and (h), by Lemma 11iv (p. 11) the
averages in the inverted matrices are invertible in a neighborhood of
$(\theta_{0},\tau(\theta_{0}))$, $\mathbb{P}$-a.s. for $T$ big enough. Thus,
the result follows from the ULLN à la Wald, the linearity of the trace
operator and the scaling by $\frac{1}{T}$. ∎

######  Lemma 16 (Uniform limit of
$\frac{\partial^{2}L_{T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{l}}$ in
a neighborhood of $(\theta_{0},\tau(\theta_{0}))$).

Under Assumptions 1 and 2, for all $(k,\ell)\in[\\![1,m]\\!]^{2}$,
$\mathbb{P}$-a.s. as $T\rightarrow\infty$, uniformly over a closed ball around
$(\theta_{0},\tau(\theta_{0}))$ with strictly positive radius,

  1. (i)

$\frac{\partial^{2}M_{1,T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{\ell}}\rightarrow\frac{1}{\left[\mathbb{E}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\right]^{2}}\times\left\\{\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\right]\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\psi_{k}(X_{1,}\theta)+\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi_{k}(X_{1,}\theta)}{\partial\theta_{\ell}}\right]\right.\\\
\left.\hskip
36.135pt-\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\right]\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi_{k}(X_{1,}\theta)\right]\right\\}$;

  2. (ii)

$\frac{\partial^{2}M_{2,T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{\ell}}\rightarrow
0$.

  3. (iii)

$\frac{\partial^{2}M_{3,T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{\ell}}\rightarrow
0$.

###### Proof.

The proof is similar to the one of Lemma 15 (p. 15). (i) Under Assumptions 1
and 2, by Lemma 19i-v (p. 19), Assumption 1(a) and (b), all the averages in
$\frac{\partial^{2}M_{1,T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{\ell}}$
(equation (27) on p. 27) satisfy the assumptions of the ULLN à la Wald.
Moreover, under Assumption 1(a)-(b) (d)(e)(g) and (h), by Lemma 11i (p. 11)
the averages in the denominators are bounded away from zero. Thus, the result
follows from the ULLN à la Wald. Note that the coefficient $\frac{m}{2T}$
vanishes as it goes to zero, as $T\rightarrow\infty$.

(ii) Under Assumptions 1 and 2, by Lemma 19vi-xii (p. 19), Assumption 1(a) and
(b), all the averages in
$\frac{\partial^{2}M_{2,T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{\ell}}$
(equation (33) on p. 33) satisfy the assumptions of the ULLN à la Wald.
Moreover, under Assumption 1, by Lemma 11iii (p. 11) the averages in the
inverted matrices are invertible in a neighborhood of
$(\theta_{0},\tau(\theta_{0}))$ $\mathbb{P}$-a.s. for $T$ big enough. Thus,
the result follows from the ULLN à la Wald, the linearity of the trace
operator and the scaling by $\frac{1}{T}$.

(iii) Under Assumptions 1 and 2, by Lemma 19xiii-xix (p. 19), Assumption 1(a)
and (b), all the averages in
$\frac{\partial^{2}M_{3,T}(\theta,\tau)}{\partial\tau_{k}\partial\theta_{\ell}}$
(equation (35)on p. 38) satisfy the assumptions of the ULLN à la Wald.
Moreover, under Assumption 1(a)(b)(e)(g) and (h), by Lemma 11iv (p. 11) the
averages in the inverted matrices are invertible in a neighborhood of
$(\theta_{0},\tau(\theta_{0}))$, $\mathbb{P}$-a.s. for $T$ big enough. Thus,
the result follows from the ULLN à la Wald, the linearity of the trace
operator and the scaling by $\frac{1}{T}$. ∎

######  Lemma 17.

Put
$S_{T}(\theta,\tau):=\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)$.
Under Assumptions 1 and 2, there exists a closed ball centered at
$(\theta_{0}^{\prime}\,\tau(\theta_{0})^{\prime})$ with strictly positive
radius s.t., $\mathbb{P}$-a.s. as $T\rightarrow\infty$,

  1. (i)

$\sup_{(\theta,\tau)\in\overline{B_{r_{L}}((\theta_{0},\tau_{0}))}}\left|\frac{\partial
S_{T}(\theta,\tau)}{\partial\theta^{\prime}}-\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\right]-\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\right]\right|=o(1)$;

  2. (ii)

$\sup_{(\theta,\tau)\in\overline{B_{r_{L}}((\theta_{0},\tau_{0}))}}\left|\frac{\partial
S_{T}(\theta,\tau)}{\partial\tau^{\prime}}-\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}\right]\right|=o(1)$.

###### Proof.

(i) By definition of $S_{T}(\theta,\tau)$,

$\displaystyle\frac{\partial S(\theta,\tau)}{\partial\theta^{\prime}}$
$\displaystyle=$
$\displaystyle\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}+\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}.$

Thus, by the triangle inequality,

$\displaystyle\sup_{(\theta,\tau)\in\overline{B_{r_{L}}((\theta_{0},\tau_{0}))}}\left|\frac{\partial
S_{T}(\theta,\tau)}{\partial\theta^{\prime}}-\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\right]-\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\right]\right|$
$\displaystyle\leqslant$
$\displaystyle\sup_{(\theta,\tau)\in\overline{B_{r_{L}}((\theta_{0},\tau_{0}))}}\left|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\tau^{\prime}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}-\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\right]\right|$
$\displaystyle+\sup_{(\theta,\tau)\in\overline{B_{r_{L}}((\theta_{0},\tau_{0}))}}\left|\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\frac{\partial\psi_{t}(\theta)}{\partial\theta^{\prime}}-\mathbb{E}\left[\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}\right]\right|$
$\displaystyle=$ $\displaystyle o(1)\text{ $\mathbb{P}$-a.s. as
$T\rightarrow\infty$}$

where the last equality follows from the ULLN à la Wald by Assumption 1(a)(b)
and Lemma 18iv-v (p. 18), under Assumptions 1 and 2.

(ii) By definition of $S_{T}(\theta,\tau)$, $\frac{\partial
S_{T}(\theta,\tau)}{\partial\tau^{\prime}}=\frac{1}{T}\sum_{t=1}^{T}\mathrm{e}^{\tau^{\prime}\psi_{t}(\theta)}\psi_{t}(\theta)\psi_{t}(\theta)^{\prime}$.
Now, under Assumption 1(a)-(b)(e) and (g), by Lemma 8i (p. 8) and Assumption
1(a)(b), the assumptions of the ULLN à la Wald are satisfied, so that the
result follows from the latter. ∎

######  Lemma 18 (Finiteness of the expectations of the supremum of the terms
from
$\frac{\partial^{2}L_{T}(\theta,\tau)}{\partial\theta_{\ell}\partial\theta_{j}}$).

Under Assumptions 1 and 2, there exists a closed ball
$\overline{B_{L}}\subset\mathbf{S}$ centered at
$(\theta_{0},\tau(\theta_{0}))$ with strictly positive radius s.t., for all
$(\ell,j)\in[\\![1,m]\\!]^{2}$,

  1. (i)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\right]<\infty$;

  2. (ii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right]<\infty$;

  3. (iii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta_{\ell}}|\right]<\infty$;

  4. (iv)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}|\right]<\infty$;

  5. (v)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$;

  6. (vi)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{l}\partial\theta^{\prime}}|\right]<\infty$;

  7. (vii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$;

  8. (viii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{3}\psi(X_{1},\theta)}{\partial\theta_{l}\partial\theta_{j}\partial\theta^{\prime}}|\right]<\infty$;

  9. (ix)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta^{\prime}}|\right]<\infty$;

  10. (x)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{\ell}\partial\theta_{j}}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$;

  11. (xi)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<\infty$;

  12. (xii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$;

  13. (xiii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\psi(X_{1},\theta)^{\prime}|\right]<\infty$;

  14. (xiv)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$;

  15. (xv)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{l}\partial\theta_{j}}\psi(X_{1},\theta)^{\prime}|\right]<\infty$;

  16. (xvi)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right]<\infty$;

  17. (xvii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}(\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}})\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\psi(X_{1},\theta)^{\prime}|\right]<\infty$;

  18. (xviii)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}(\tau^{\prime}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{\ell}\partial\theta_{j}})\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$;
and

  19. (xix)

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}(\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}})(\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}})\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$.

###### Proof.

(i) Under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11), $\mathbf{S}$
contains an open ball centered at $(\theta_{0},\tau(\theta_{0}))$, so that, by
the Cauchy-Schwarz inequality, for $\overline{B_{L}}$ of sufficiently small
radius,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)}\right]\leqslant\sqrt{\mathbb{E}\left[(\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1,}\theta)})^{2}\right]}=\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\mathbf{S}}\mathrm{e}^{2\tau^{\prime}\psi(X_{1,}\theta)}\right]}<\infty$
where the equality follows from the fact that supremum of the square of a
positive function is the square of the supremum of the function, and the last
inequality from Assumption 1(e).

(ii) The norm of a product of matrices is smaller than the product of the
norms (e.g., Rudin 1953, Theorem 9.7 and note that all norms are equivalent on
finite dimensional spaces). Thus,
for $\overline{B_{L}}$ of sufficiently small radius, for all
$(\ell,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\
\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right]$
$\displaystyle\leqslant$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2})\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2})\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]\stackrel{{\scriptstyle(b)}}{{<}}\infty.$

(a) Firstly, under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11),
$\mathbf{S}$ contains an open ball centered at $(\theta_{0},\tau(\theta_{0}))$
Thus, under Assumption 1(a)-(e) and (g)-(h), for $\overline{B_{L}}$ of
sufficiently small radius, by definition of $\mathbf{S}$,
$\overline{B_{L}}\subset\\{(\theta,\tau):\theta\in\mathcal{N}\wedge\tau\in\mathbf{T}(\theta)\\}\subset\mathbf{S}$,
because $\mathcal{N}\subset\mathbf{\Theta}$ by Assumption 2(a). Secondly, by
Assumption 2(b),
$\sup_{\theta\in\mathcal{N}}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}|\leqslant
b(X)$ and
$\sup_{\theta\in\mathcal{N}}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\leqslant
b(X)$. (b) Firstly, $\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2}<\infty$
because $\overline{B_{L}}$ is bounded. Secondly, by Assumption 2(b),
$\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]<\infty$.
(iii) Similarly to the proof of statement (ii), under Assumption 1(a)-(e) and
(g)-(h), for $\overline{B_{L}}$ of sufficiently small radius, for all
$(\ell,j)\in[\\![1,m]\\!]^{2}$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta_{\ell}}|\right]\leqslant(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\\\
\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta_{\ell}}|\right]\leqslant(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})\right]<\infty$,
where the two last inequalities follow from Assumption 2(b) and the
boundedness of $\overline{B_{L}}$.

(iv) Similarly to the proof of statement (ii), under Assumption 1(a)-(e) and
(g)-(h), for $\overline{B_{L}}$ of sufficiently small radius, for all
$\ell\in[\\![1,m]\\!]$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}|\right]\leqslant(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\\\
\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}|\right]\leqslant(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})\right]<\infty$,
where the two last inequalities follow from Assumption 2(b) and the
boundedness of $\overline{B_{L}}$.

(v) Similarly to the proof of statement (ii), under Assumption 1(a)-(e) and
(g)-(h), for $\overline{B_{L}}$ of sufficiently small radius,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]\leqslant\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})\right]<\infty$,
where the last inequality follows from Assumption 2(b).

(vi) Similarly to the proof of statement (ii), under Assumption 1(a)-(e) and
(g)-(h), for $\overline{B_{L}}$ of sufficiently small radius, for all
$\ell\in[\\![1,m]\\!]$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{\ell}\partial\theta^{\prime}}|\right]\\\
\leqslant\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})\right]<\infty$,
where the last inequality follows from Assumption 2(b).

(vii) Similarly to the proof of statement (ii), under Assumption 1(a)-(e) and
(g)-(h), for $\overline{B_{L}}$ of sufficiently small radius, for all
$\ell\in[\\![1,m]\\!]$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]\leqslant(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\\\
\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]<\infty$
where the two last inequalities follow from Assumption 2(b) and the
boundedness of $\overline{B_{L}}$.

(viii) Similarly to the proof of statement (ii), under Assumption 1(a)-(e) and
(g)-(h), for $\overline{B_{L}}$ of sufficiently small radius, for all
$(\ell,j)\in[\\![1,m]\\!]^{2}$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{3}\psi(X_{1},\theta)}{\partial\theta_{l}\partial\theta_{j}\partial\theta^{\prime}}|\right]\leqslant\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})\right]<\infty$,
where the last inequality follows from Assumption 2(b).

(ix) Similarly to the proof of statement (ii), under Assumption 1(a)-(e) and
(g)-(h), for $\overline{B_{L}}$ of sufficiently small radius, for all
$(\ell,j)\in[\\![1,m]\\!]^{2}$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\tau^{\prime}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta^{\prime}}|\right]<(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2})\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}||\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{j}\partial\theta^{\prime}}|\right]\leqslant(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)^{2}\\\
\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]<\infty$
where the two last inequalities follow from Assumption 2(b) and the
boundedness of $\overline{B_{L}}$.

(x) Similarly to the proof of statement (ii), under Assumption 1(a)-(e) and
(g)-(h), for $\overline{B_{L}}$ of sufficiently small radius, for all
$(\ell,j)\in[\\![1,m]\\!]^{2}$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{\ell}\partial\theta_{j}}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{\ell}\partial\theta_{j}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]\\\
\leqslant(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]<\infty$
where the two last inequalities follow from Assumption 2(b) and the
boundedness of $\overline{B_{L}}$.

(xi) Similarly to the proof of statement (ii), under Assumption 1(a)-(e) and
(g)-(h), for $\overline{B_{L}}$ of sufficiently small radius, for all
$(\ell,j)\in[\\![1,m]\\!]^{2}$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]<(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2})\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta^{\prime}}|\right]\leqslant(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|^{2})\\\
\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{3}\right]<\infty$
where the two last inequalities follow from Assumption 2(b) and the
boundedness of $\overline{B_{L}}$.

(xii) Under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11),
$\mathbf{S}$ contains an open ball centered at
$(\theta_{0},\tau(\theta_{0}))$, so that, for $\overline{B_{L}}$ of
sufficiently small radius,

$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]\leqslant\mathbb{E}\left[\sup_{(\theta,\tau)\in\mathbf{S}^{\epsilon}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]<\infty$
where the last inequality follows from Lemma 8i (p. 8) under Assumption
1(a)-(b)(e)(g).

(xiii) The supremum of the absolute value of the product is smaller than the
product of the suprema of the absolute values. Thus, under Assumption 1(a)(b),
for $\overline{B_{L}}$ of sufficiently small radius, for all
$\ell\in[\\![1,m]\\!]$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\leqslant$
$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}|^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)^{\prime}|^{2}\right]}\stackrel{{\scriptstyle(c)}}{{<}}\infty.$

(a) Apply the Cauchy-Schwarz inequality, and note that the supremum of the
square of a positive function is the square of the supremum of the function.
(b) Firstly, under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11),
$\mathbf{S}$ contains an open ball centered at
$(\theta_{0},\tau(\theta_{0}))$, so that, for $\overline{B_{L}}$ of
sufficiently small radius,
$\overline{B_{L}}\subset\\{(\theta,\tau):\theta\in\mathcal{N}\wedge\tau\in\mathbf{T}(\theta)\\}\subset\mathbf{S}\subset\mathbf{S}^{\epsilon}$
because $\mathcal{N}\subset\mathbf{\Theta}$ and
$\mathbf{S}=\\{(\theta,\tau):\theta\in\mathbf{\Theta}\wedge\tau\in\mathbf{T}(\theta)\\}$.
Secondly, as the second supremum does not depend on $\tau$,
$\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)^{\prime}|^{2}\leqslant\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)^{\prime}|^{2}$
because $\overline{B_{L}}\subset\mathbf{S}$, for $\overline{B_{L}}$ of radius
small enough. (c) By Assumption 2(b), the first expectation is bounded. Under
Assumption 1(a)(b)(g), by Lemma 9i (p. 9), the second expectation is also
bounded.

(xiv) Proof similar to the one of statement (xiii). The supremum of the
absolute value of the product is smaller than the product of the suprema of
the absolute values. Thus, under Assumption 1(a)(b), for $\overline{B_{L}}$ of
sufficiently small radius, for all $\ell\in[\\![1,m]\\!]$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\leqslant$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}|^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}\stackrel{{\scriptstyle(c)}}{{<}}\infty.$

(a) Apply the Cauchy-Schwarz inequality, and note that the supremum of the
square of a positive function is the square of the supremum of the function.
(b) Firstly, under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11),
$\mathbf{S}$ contains an open ball centered at
$(\theta_{0},\tau(\theta_{0}))$, so that, for $\overline{B_{L}}$ of
sufficiently small radius,
$\overline{B_{L}}\subset\\{(\theta,\tau):\theta\in\mathcal{N}\wedge\tau\in\mathbf{T}(\theta)\\}\subset\mathbf{S}\subset\mathbf{S}^{\epsilon}$.
Secondly, as the second supremum does not depend on $\tau$,
$\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\leqslant\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}$
because $\overline{B_{L}}\subset\mathbf{S}$, for $\overline{B_{L}}$ of radius
small enough. (c) Firstly, because $\overline{B_{L}}$ is bounded,
$(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)<\infty$. Secondly, by
Assumption 2(b), the first expectation is bounded. Thirdly, by Assumption
1(g), the second expectation is also bounded.

(xv) The proof is the same as for statement (xiii) with
$\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{l}\partial\theta_{j}}$
instead of $\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}$. The
supremum of the absolute value of the product is smaller than the product of
the suprema of the absolute values. Thus, under Assumption 1(a)(b), for
$\overline{B_{L}}$ of sufficiently small radius, for all
$(\ell,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{l}\partial\theta_{j}}\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\leqslant$
$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{l}\partial\theta_{j}}|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{l}\partial\theta_{j}}|^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{\leqslant}}$
$\displaystyle\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)^{\prime}|^{2}\right]}\stackrel{{\scriptstyle(c)}}{{<}}\infty.$

(a) Apply the Cauchy-Schwarz inequality, and note that the supremum of the
square of a positive function is the square of the supremum of the function.
(b) Firstly, under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11),
$\mathbf{S}$ contains an open ball centered at
$(\theta_{0},\tau(\theta_{0}))$, so that, for $\overline{B_{L}}$ of
sufficiently small radius,
$\overline{B_{L}}\subset\\{(\theta,\tau):\theta\in\mathcal{N}\wedge\tau\in\mathbf{T}(\theta)\\}\subset\mathbf{S}\subset\mathbf{S}^{\epsilon}$.
Secondly, as the second supremum does not depend on $\tau$,
$\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)^{\prime}|^{2}\leqslant\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)^{\prime}|^{2}$
because $\overline{B_{L}}\subset\mathbf{S}$, for $\overline{B_{L}}$ of radius
small enough. (c) By Assumption 2(b), the first expectation is bounded. Under
Assumption 1(a)(b)(g), by Lemma 9i (p. 9), the second expectation is also
bounded.

(xvi) Similarly to the proof of statement (ii), under Assumption 1(a)-(e) and
(g)-(h), for $\overline{B_{L}}$ of sufficiently small radius, for all
$(\ell,j)\in[\\![1,m]\\!]^{2}$,
$\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right]<\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right]\leqslant\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]<\infty$
where the last inequality follows from Assumption 2(b).

(xvii) Proof similar to the one of statement (xiii). The norm of a product of
matrices is smaller than the product of the norms (e.g., Rudin 1953, Theorem
9.7 and note that all norms are equivalent on finite dimensional spaces).
Moreover, the supremum of the absolute value of the product is smaller than
the product of the suprema of the absolute values. Thus, under Assumption
1(a)(b), for $\overline{B_{L}}$ of sufficiently small radius, for all
$(\ell,j)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}(\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}})\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}\psi(X_{1},\theta)^{\prime}|\right]<\infty$
$\displaystyle\leqslant$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\left(\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right)\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)|\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}\left(\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}|\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}}||\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}}|\right)^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{4}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)|^{2}\right]}\stackrel{{\scriptstyle(c)}}{{<}}\infty.$

(a) Apply the Cauchy-Schwarz inequality, and note that the supremum of the
square of a positive function is the square of the supremum of the function.
(b) Firstly, under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11),
$\mathbf{S}$ contains an open ball centered at
$(\theta_{0},\tau(\theta_{0}))$, so that, for $\overline{B_{L}}$ of
sufficiently small radius,
$\overline{B_{L}}\subset\\{(\theta,\tau):\theta\in\mathcal{N}\wedge\tau\in\mathbf{T}(\theta)\\}\subset\mathbf{S}\subset\mathbf{S}^{\epsilon}$.
Secondly, as the second supremum does not depend on $\tau$,
$\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)^{\prime}|^{2}\leqslant\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)^{\prime}|^{2}$
because $\overline{B_{L}}\subset\mathbf{S}$, for $\overline{B_{L}}$ of radius
small enough. (c) Firstly, because $\overline{B_{L}}$ is bounded,
$(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)<\infty$. Secondly, by
Assumption 2(b), the first expectation is bounded. Thirdly, under Assumption
1(a)(b)(g), by Lemma 9 (p. 9), the second expectation is also bounded.

(xviii) Proof similar to the one of statement (xiii). The supremum of the
absolute value of the product is smaller than the product of the suprema of
the absolute values. Thus, under Assumption 1(a)(b), for $\overline{B_{L}}$ of
sufficiently small radius, for all $(j,\ell)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}(\tau^{\prime}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{\ell}\partial\theta_{j}})\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\leqslant$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{\ell}\partial\theta_{j}}|\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\stackrel{{\scriptstyle(a)}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}\frac{\partial^{2}\psi(X_{1},\theta)}{\partial\theta_{\ell}\partial\theta_{j}}|^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}$
$\displaystyle\stackrel{{\scriptstyle(b)}}{{\leqslant}}$
$\displaystyle(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathcal{N}}\sup_{\tau\in\mathbf{T}(\theta)}\mathrm{e}^{2\tau^{\prime}\psi(X_{1},\theta)}b(X_{1})^{2}\right]}\sqrt{\mathbb{E}\left[\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|^{2}\right]}\stackrel{{\scriptstyle(c)}}{{<}}\infty.$

(a) Apply the Cauchy-Schwarz inequality, and note that the supremum of the
square of a positive function is the square of the supremum of the function.
(b) Firstly, under Assumption 1(a)-(e) and (g)-(h), by Lemma 11ii (p. 11),
$\mathbf{S}$ contains an open ball centered at
$(\theta_{0},\tau(\theta_{0}))$, so that, for $\overline{B_{L}}$ of
sufficiently small radius,
$\overline{B_{L}}\subset\\{(\theta,\tau):\theta\in\mathcal{N}\wedge\tau\in\mathbf{T}(\theta)\\}\subset\mathbf{S}\subset\mathbf{S}^{\epsilon}$.
Secondly, as the second supremum does not depend on $\tau$,
$\sup_{(\theta,\tau)\in\overline{B_{L}}}|\psi(X_{1},\theta)^{\prime}|^{2}\leqslant\sup_{\theta\in\mathbf{\Theta}^{\epsilon}}|\psi(X_{1},\theta)^{\prime}|^{2}$
because $\overline{B_{L}}\subset\mathbf{S}$, for $\overline{B_{L}}$ of radius
small enough. (c) Firstly, because $\overline{B_{L}}$ is bounded,
$(\sup_{(\theta,\tau)\in\overline{B_{L}}}|\tau|)<\infty$. Secondly, by
Assumption 2(b), the first expectation is bounded. Thirdly, by Assumption
1(g), the second expectation is also bounded.

(xix) Proof similar to the one of statement (xiii). The supremum of the
absolute value of the product is smaller than the product of the suprema of
the absolute values. Thus, under Assumption 1(a)(b), for $\overline{B_{L}}$ of
sufficiently small radius, for all $(j,\ell)\in[\\![1,m]\\!]^{2}$,

$\displaystyle\mathbb{E}\left[\sup_{(\theta,\tau)\in\overline{B_{L}}}|\mathrm{e}^{\tau^{\prime}\psi(X_{1},\theta)}(\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{\ell}})(\tau^{\prime}\frac{\partial\psi(X_{1},\theta)}{\partial\theta_{j}})\psi(X_{1},\theta)\psi(X_{1},\theta)^{\prime}|\right]$
$\displaystyle\leqslant$