# Inference and model selection in general causal time series with exogenous
covariates

Mamadou Lamine DIOP 111Supported by the MME-DII center of excellence
(ANR-11-LABEX-0023-01)  and William KENGNE 222Developed within the ANR
BREAKRISK: ANR-17-CE26-0001-01 and the CY Initiative of Excellence (grant
”Investissements d’Avenir” ANR-16-IDEX-0008), Project ”EcoDep” PSI-
AAP2020-0000000013

THEMA, CY Cergy Paris Université, 33 Boulevard du Port, 95011 Cergy-Pontoise
Cedex, France.
E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>

Abstract: In this paper, we study a general class of causal processes with
exogenous covariates, including many classical processes such as the ARMA-
GARCH, APARCH, ARMAX, GARCH-X and APARCH-X processes. Under some Lipschitz-
type conditions, the existence of a $\tau$-weakly dependent strictly
stationary and ergodic solution is established. We provide conditions for the
strong consistency and derive the asymptotic distribution of the quasi-maximum
likelihood estimator (QMLE), both when the true parameter is an interior point
of the parameter’s space and when it belongs to the boundary. A significance
Wald-type test of parameter is developed. This test is quite extensive and
includes the test of nullity of the parameter’s components, which in
particular, allows us to assess the relevance of the exogenous covariates.
Relying on the QMLE of the model, we also propose a penalized criterion to
address the problem of the model selection for this class. The weak and the
strong consistency of the procedure are established. Finally, Monte Carlo
simulations are conducted to numerically illustrate the main results.

Keywords: Causal processes, exogenous covariates, quasi-maximum likelihood
estimator, consistency, boundary, significance test, model selection,
penalized criterion.

##  1 Introduction

Autoregressive time series with exogenous covariates provide effective ways to
take into account some available extra information in the models. The well
known example that has been widely studied is the ARMAX model, see Hannan
(1976), Hannan and Deistler (2012). The GARCH-type models with exogenous
covariates have recently attracted much attention in the literature, see for
instance Han and Kristensen (2014) for GARCH-X, Francq and Thieu (2019) for
APARCH-X. Guo et al. (2014) considered the factor double autoregressive model,
whose ARX and ARCH-X are particular cases. We consider a large class of causal
time series models, whose ARMAX and GARCH-X type models are specific examples.

Let $X_{t}=(X_{1,t},X_{2,t},\ldots,X_{d_{x},t})\in\mathbb{R}^{d_{x}}$ be a
vector of covariates, with $d_{x}\in\mathbb{N}$. Consider the class of affine
causal models with exogenous covariates,

Class $\mathcal{AC}$-$X(M_{\theta},f_{\theta}):$ A process
$\\{Y_{t},\,t\in\mathbb{Z}\\}$ belongs to
$\mathcal{AC}$-$X(M_{\theta},f_{\theta})$ if it satisfies:

$Y_{t}=M_{\theta}(Y_{t-1},Y_{t-2},\ldots;X_{t-1},X_{t-2},\ldots)\xi_{t}+f_{\theta}(Y_{t-1},Y_{t-2},\ldots;X_{t-1},X_{t-2},\ldots),$
(1.1)

where
$M_{\theta},~{}f_{\theta}:\mathbb{R}^{\mathbb{N}}\times(\mathbb{R}^{d_{x}})^{\mathbb{N}}\rightarrow\mathbb{R}$
are two measurable functions and assumed to be known up to the parameter
$\theta$, which belongs in a compact subset $\Theta\subset\mathbb{R}^{d}$
($d\in\mathbb{N}$); and $(\xi_{t})_{t\in\mathbb{Z}}$ is a sequence of zero-
mean independent, identically distributed (i.i.d) random variable satisfying
$\mathbb{E}(\xi^{r}_{0})<\infty$ for some $r\geq 2$ and
$\mathbb{E}(\xi^{2}_{0})=1$. Remark that, if $X_{t}\equiv C$ for some constant
$C$ (absence of covariates), then (1.1) reduces to the classical affine causal
models that has already been considered in the literature (see, for instance,
Bardet and Wintenberger (2009), Bardet et al. (2012), Bardet et al. (2020)).
One can see that, the ARMAX, GARCH-X, APARCH-X models belong to the class
$\mathcal{AC}$-$X(M_{\theta},f_{\theta})$.

There exist several important contributions devoted to autoregressive models
with covariates; we refer to Hannan and Deistler (2012), Han and Kristensen
(2014), Sucarrat et al. (2016), Francq and Sucarrat (2017), Pedersen and
Rahbek (2018), Francq and Thieu (2019), Grønneberg and Holcblat (2019), Zambom
and Gel (2020) and the references therein for some developments on ARMAX and
conditional volatility type models with exogenous covariates. The class
$\mathcal{AC}$-$X(M_{\theta},f_{\theta})$ is more general than the models
considered in the aforementioned works, as well as the factor double
autoregressive model proposed by Guo et al. (2014) which is a particular case
of the model (1.1). Note as well that, the class
$\mathcal{AC}$-$X(M_{\theta},f_{\theta})$ provides a more general way to take
into account covariates in the model, and one can see that the linear
covariates regressors considered by Francq and Thieu (2019) and by many other
works is a specific case. Compared to Bardet and Wintenberger (2009), besides
taking into account covariates in the model (1.1), we address the inference
when the true parameter belongs to the boundary of the parameter set $\Theta$
and the model selection question.

In this new contribution, we consider the class of model (1.1) and address the
following issues.

  1. (i)

Existence of a stationary solution. We provide sufficient conditions that
ensure the existence of a $\tau$-weakly dependent stationary and ergodic
solution $Z_{t}=(Y_{t},X_{t})$ of (1.1). At a first glance, one might think
that these conditions are the same as those obtained by Bardet and
Wintenberger (2009), but in our case, the existence of the covariates must be
taken into account.

  2. (ii)

Inference for the class $\mathcal{AC}$-$X(M_{\theta^{*}},f_{\theta^{*}})$. An
inference based on the quasi likelihood of the model is carried out. The
consistency of the quasi-maximum likelihood estimator (QMLE) is established
and we derived the asymptotic distribution of this estimator (even when
$\theta^{*}$ belongs to the boundary of $\Theta$).

  3. (iii)

Significance test of parameter. A Wald-type significance test of parameter of
the model (1.1) is conducted. The proposed test is quite extensive and
includes the test of nullity of the parameter’s components. An asymptotic
study is carried out, which shows in particular that, when the true parameter
belongs to the boundary of $\Theta$, the asymptotic distribution of the test
statistic under the null hypothesis is quite different from the classical chi-
square distribution.

  4. (iv)

Model selection. A penalized criterion based on the quasi likelihood of the
model is proposed for model selection in the class
$\mathcal{AC}$-$X(M_{\theta^{*}},f_{\theta^{*}})$. We provides conditions that
ensure the weak and the strong consistency of the proposed procedure. These
conditions shows in particular that, the Hannan-Quinn information Criterion
(HQC) with a regularization parameter $\kappa_{n}=c\log\log n$ (see (3.2)) is
strongly consistent for sufficiently large $c$.

The article is organized as follows. In Section 2, firstly, we provide
conditions for stability properties. Secondly, we give the definition of the
QMLE and study its asymptotic properties; a significance test of parameter
with an asymptotic study is also addressed. Section 3 focuses on the model
selection and the consistency of the proposed procedure. Some classical
examples of processes belonging to the class
$\mathcal{AC}$-$X(M_{\theta^{*}},f_{\theta^{*}})$ are detailed in Section 4.
Section 5 gives some empirical results, whereas Section 6 is devoted to a
summary and conclusion. Section 7 contains the proofs of the main results.

##  2 Assumptions, inference and test of the parameters

###  2.1 Assumptions

Throughout the sequel, the following norms will be used:

  * •

$\|x\|\coloneqq\sqrt{\sum_{i=1}^{p}x_{i}^{2}}$, for any $x\in\mathbb{R}^{p}$,
$p\in\mathbb{N}$;

  * •

$\|V\|\coloneqq\sqrt{\sum_{i=1}^{p}\sum_{j=1}^{q}v_{i,j}^{2}}$, for any matrix
$V\in M_{p,q}(\mathbb{R})$, where $M_{p,q}(\mathbb{R})$ denotes the set of
matrices of dimension $p\times q$ with coefficients in $\mathbb{R}$, for
$p,q\in\mathbb{N}$;

  * •

$\left\|g\right\|_{\mathcal{K}}\coloneqq\sup_{\theta\in\mathcal{K}}\left(\left\|g(\theta)\right\|\right)$
for any compact set $\mathcal{K}\subseteq\mathbb{R}^{d}$ and function
$g:\mathcal{K}\longrightarrow M_{p,q}(\mathbb{R})$;

  * •

$\left\|Y\right\|_{r}\coloneqq\mathbb{E}\left(\left\|Y\right\|^{r}\right)^{1/r}$,
if $Y$ is a random vector with finite $r-$order moments, for $r>0$.

We will denote by $0$ the null vector of any vector space. Let $\Psi_{\theta}$
be the generic symbol for any of the functions $f_{\theta}$ or $M_{\theta}$.
We set the following classical Lipschitz-type conditions for any compact set
$\mathcal{K}\subseteq\Theta$.

Assumption A${}_{i}(\Psi_{\theta},\mathcal{K})$ ($i=0,1,2$): For any
$(y,x)\in\mathbb{R}^{\mathbb{N}}\times(\mathbb{R}^{d_{x}})^{\mathbb{N}}$, the
function $\theta\mapsto\Psi_{\theta}(y)$ is $i$ times continuously
differentiable on $\mathcal{K}$ with
$\big{\|}\frac{\partial^{i}\Psi_{\theta}(0)}{\partial\theta^{i}}\big{\|}_{\mathcal{K}}<\infty$;
and there exists two sequences of non-negative real numbers
$(\alpha^{(i)}_{k,Y}(\Psi_{\theta},\mathcal{K}))_{k\geq 1}$ and
$(\alpha^{(i)}_{k,X}(\Psi_{\theta},\mathcal{K}))_{k\geq 1}$ satisfying:
$\sum\limits_{k=1}^{\infty}\alpha^{(i)}_{k,Y}(\Psi_{\theta},\mathcal{K})<\infty$,
$\sum\limits_{k=1}^{\infty}\alpha^{(i)}_{k,X}(\Psi_{\theta},\mathcal{K})<\infty$
for $i=0,1,2$; such that for any
$(y,x),(y^{\prime},x^{\prime})\in\mathbb{R}^{\mathbb{N}}\times(\mathbb{R}^{d_{x}})^{\mathbb{N}}$,

$\Big{\|}\frac{\partial^{i}\Psi_{\theta}(y,x)}{\partial\theta^{i}}-\frac{\partial^{i}\Psi_{\theta}(y^{\prime},x^{\prime})}{\partial\theta^{i}}\Big{\|}_{\mathcal{K}}\leq\sum\limits_{k=1}^{\infty}\alpha^{(i)}_{k,Y}(\Psi_{\theta},\mathcal{K})|y_{k}-y^{\prime}_{k}|+\sum\limits_{k=1}^{\infty}\alpha^{(i)}_{k,X}(\Psi_{\theta},\mathcal{K})\|x_{k}-x^{\prime}_{k}\|),$

where $\|\cdot\|$ denotes any vector, matrix norm.

The following assumption is considered on the function
$H_{\theta}=M^{2}_{\theta}$ in the cases of ARCH-X type process.

Assumption A${}_{i}(H_{\theta},\mathcal{K})$ ($i=0,1,2$): Assume that
$f_{\theta}=0$. There exists two sequences of non-negative real numbers
$(\alpha^{(i)}_{k,Y}(H_{\theta},\mathcal{K}))_{k\geq 1}$ and
$(\alpha^{(i)}_{k,X}(H_{\theta},\mathcal{K}))_{k\geq 1}$ satisfying:
$\sum\limits_{k=1}^{\infty}\alpha^{(i)}_{k,Y}(H_{\theta},\mathcal{K})<\infty$,
$\sum\limits_{k=1}^{\infty}\alpha^{(i)}_{k,X}(H_{\theta},\mathcal{K})<\infty$
for $i=0,1,2$; such that for any
$(y,x),(y^{\prime},x^{\prime})\in\mathbb{R}^{\infty}\times(\mathbb{R}^{d_{x}})^{\infty}$,

$\Big{\|}\frac{\partial^{i}H_{\theta}(y,x)}{\partial\theta^{i}}-\frac{\partial^{i}H_{\theta}(y^{\prime},x^{\prime})}{\partial\theta^{i}}\Big{\|}_{\mathcal{K}}\leq\sum\limits_{k=1}^{\infty}\alpha^{(i)}_{k,Y}(H_{\theta},\mathcal{K})|y^{2}_{k}-{y^{\prime}}^{2}_{k}|+\sum\limits_{k=1}^{\infty}\alpha^{(i)}_{k,X}(H_{\theta},\mathcal{K})\|x_{k}-x^{\prime}_{k}\|.$

In the whole paper, we impose an autoregressive-type structure on the
covariates:

$X_{t}=g(X_{t-1},X_{t-2},\ldots;\eta_{t}),$ (2.1)

where $(\eta_{t})_{t\in\mathbb{Z}}$ is a sequence of random variables such as
$(\eta_{t},\xi_{t})_{t\in\mathbb{Z}}$ is i.i.d and $g$ is a function with
values in $\mathbb{R}^{d_{x}}$ satisfying

$\mathbb{E}\left[\left\|g(0,\eta_{0})\right\|^{r}\right]<\infty~{}\text{ and
}~{}\left\|g(x;\eta_{0})-g(x^{\prime};\eta_{0})\right\|_{r}\leq\sum\limits_{k=1}^{\infty}\alpha_{k}(g)\left\|x_{k}-x^{\prime}_{k}\right\|~{}\text{
for all }x,x^{\prime}\in(\mathbb{R}^{d_{x}})^{\mathbb{N}},$ (2.2)

for some $r\geq 1$ and non-negative sequence $(\alpha_{k}(g))_{k\geq 1}$ such
that $\sum\limits_{k=1}^{\infty}\alpha_{k}(g)<1$.

For $r\geq 1$, when (2.2) holds, we define the set

$\Theta(r)=\Big{\\{}\theta\in\mathbb{R}^{d}\,\big{/}\,\textbf{A}_{0}(f_{\theta},\\{\theta\\})\
\text{and}\ \textbf{A}_{0}(M_{\theta},\\{\theta\\})\ \text{hold with}\\\
\hskip
199.45366pt\sum\limits_{k=1}^{\infty}\max\left\\{\alpha_{k}(g),\,\alpha^{(0)}_{k,Y}(f_{\theta},\\{\theta\\})+\|\xi_{0}\|_{r}\alpha^{(0)}_{k,Y}(M_{\theta},\\{\theta\\}\right\\}<1\Big{\\}}\\\
\bigcup\Big{\\{}\theta\in\mathbb{R}^{d}\ \big{/}\ f_{\theta}=0\text{ and
}\textbf{A}_{0}(H_{\theta},\\{\theta\\})\text{ holds with
}\|\xi_{0}\|^{2}_{r}\sum\limits_{k=1}^{\infty}\max\left\\{\alpha_{k}(g),\,\alpha^{(0)}_{k,Y}(H_{\theta},\\{\theta\\})\right\\}<1\Big{\\}}.$

In the sequel, we make the convention that if A${}_{i}(M_{\theta},\Theta)$
holds then
$\alpha^{(i)}_{k,Y}(H_{\theta},\Theta)=\alpha^{(i)}_{k,X}(H_{\theta},\Theta)=0$
for all $k\in\mathbb{N}$ and if A${}_{i}(H_{\theta},\Theta)$ holds then
$\alpha^{(i)}_{k,Y}(M_{\theta},\Theta)=\alpha^{(i)}_{k,X}(M_{\theta},\Theta)=0$
for all $k\in\mathbb{N}$.

The condition (2.2) allows to assure the stability of the process $X_{t}$.
Together the aforementioned assumptions assure the existence of a stationary
and weakly dependent solution of order $r$ to the model (1.1), as shown in the
following proposition.

###### Proposition 2.1

Assume that A${}_{0}(f_{\theta},\Theta)$, A${}_{0}(M_{\theta},\Theta)$ (or
A${}_{0}(H_{\theta},\Theta)$) and (2.2) hold. If
$\theta^{*}\in\Theta\cap\Theta(r)$ with $r\geq 1$, then there exists a
$\tau$-weakly dependent stationary, ergodic and non anticipative solution
$(Z_{t})_{t\in\mathbb{Z}}$ $Z_{t}=(Y_{t},X_{t})$, to (1.1), satisfying
$\mathbb{E}[{\|Z_{0}\|}^{r}]<\infty$.

###  2.2 Inference and significance test of parameter

In this paragraph, we describe the use of the Gaussian quasi-maximum
likelihood to obtain an estimator of the parameters of the model (1.1). The
main asymptotic properties of this estimator are also established. Assume that
the observations $(Y_{1},X_{1}),\ldots,(Y_{n},X_{n})$ are generated from (1.1)
and (2.1) according to the true parameter $\theta^{*}\in\Theta$ which is
unknown. For all $t\in\mathbb{Z}$, denote by
$\mathcal{F}_{t}=\sigma((Y_{s},X_{s}),\,s\leq t)$ the $\sigma$-field generated
by the whole past at time $t$. The mean and the variance of
$Y_{t}|\mathcal{F}_{t-1}$ and $f_{\theta^{*}}(Y_{t-1},\ldots;X_{t-1},\ldots)$
and variance $M^{2}_{\theta^{*}}(Y_{t-1},\ldots;X_{t-1},\ldots)$ respectively.
For any $\theta\in\Theta$, the conditional Gaussian quasi-log-likelihood is
given by (up to an additional constant)

$L_{n}(\theta):=-\frac{1}{2}\sum_{t=1}^{n}q_{t}(\theta)~{}~{}~{}\text{ with
}~{}~{}~{}q_{t}(\theta)=\frac{(Y_{t}-f^{t}_{\theta})^{2}}{H^{t}_{\theta}}+\log
H^{t}_{\theta},$

where
$f^{t}_{\theta}:=f_{\theta}(Y_{t-1},Y_{t-2}\ldots;X_{t-1},X_{t-2},\ldots)$,
$M^{t}_{\theta}:=M_{\theta}(Y_{t-1},Y_{t-2}\ldots;X_{t-1},X_{t-2},\ldots)$ and
$H^{t}_{\theta}:=(M^{t}_{\theta})^{2}$.
Since $(Y_{0},X_{0}),(Y_{-1},X_{-1}),\ldots$ are not observed, $L_{n}(\theta)$
is approximated by

$\widehat{L}_{n}(\theta)=-\frac{1}{2}\sum_{t=1}^{n}\widehat{q}_{t}(\theta)~{}~{}~{}\text{
with
}~{}~{}~{}\widehat{q}_{t}(\theta)=\frac{(Y_{t}-\widehat{f}^{t}_{\theta})^{2}}{\widehat{H}^{t}_{\theta}}+\log\widehat{H}^{t}_{\theta},$

where
$\widehat{f}^{t}_{\theta}:=f_{\theta}(Y_{t-1},\ldots,Y_{1},0;X_{t-1},\ldots,X_{1},0)$,
$\widehat{M}^{t}_{\theta}:=M_{\theta}(Y_{t-1},\ldots,Y_{1},0;X_{t-1},\ldots,X_{1},0)$
and $\widehat{H}^{t}_{\theta}:=(\widehat{M}^{t}_{\theta})^{2}$. Thus, the QMLE
of $\theta^{*}$ is defined by

$\widehat{\theta}_{n}=\underset{\theta\in\Theta}{\text{argmax}}\big{(}\widehat{L}_{n}(\theta)\big{)}.$

We set the following regularity conditions to assure the identifiability of
the model and to derive the asymptotic behavior of the QMLE.

  1. (A0):

for all $\theta\in\Theta$ and some $t\in\mathbb{Z}$,
$\big{(}f^{t}_{\theta^{*}}=f^{t}_{\theta}\ \text{and}\
H^{t}_{\theta^{*}}=H^{t}_{\theta}\ \
a.s.\big{)}\Rightarrow~{}\theta=\theta^{*}$;

  2. (A1):

$\exists\underline{h}>0$ such that
$\displaystyle\inf_{\theta\in\Theta}H_{\theta}(y,x)\geq\underline{h}$, for all
$(y,x)\in\mathbb{R}^{\mathbb{N}}\times(\mathbb{R}^{d_{x}})^{\mathbb{N}}$;

  3. (A2):

for all $\theta\in\Theta$, $c\in\mathbb{R}^{d}$, $\Big{(}$
$c^{\prime}\frac{\partial}{\partial\theta}f^{0}_{\theta^{*}}=0$ or
$c^{\prime}\frac{\partial}{\partial\theta}H^{0}_{\theta^{*}}=0$
$\Big{)}~{}a.s.$ $\Longrightarrow~{}c=0$, where ′ denotes the transpose.

Assumption (A0) is an identifiability condition and it will be discussed in
detail for each of the examples of processes studied in the paper. From (A1),
the quasi likelihood is well defined, whereas (A2), which is classical (see
for instance Bardet and Wintenberger (2009)) allows to derive the asymptotic
distribution of the QMLE. The following theorem addresses the strong
consistency of the QMLE.

###### Theorem 2.2

Assume that (A0), (A1), A${}_{0}(f_{\theta},\Theta)$,
A${}_{0}(M_{\theta},\Theta)$ and (2.2) (with $r\geq 2$) hold with

$\alpha^{(0)}_{k,Y}(f_{\theta},\Theta)+\alpha^{(0)}_{k,X}(f_{\theta},\Theta)+\alpha^{(0)}_{k,Y}(M_{\theta},\Theta)+\alpha^{(0)}_{k,X}(M_{\theta},\Theta)+\alpha^{(0)}_{k,Y}(H_{\theta},\Theta)+\alpha^{(0)}_{k,X}(H_{\theta},\Theta)=O(k^{-\gamma}),$
(2.3)

for some $\gamma>3/2$.
If $\theta^{*}\in\Theta\cap\Theta(r)$ with $r\geq 2$, then

$\widehat{\theta}_{n}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}\theta^{*}.$

To derive the asymptotic distribution of the QMLE, it is necessary to take
into account the constraints in the parameter space $\Theta$ corresponding to
the model. For example, in some processes belonging to (1.1), such as the
ARCH-X models (see below), the components of $\theta^{*}$ are constrained to
be positive or equal to zero. In order to propose a parsimonious
representation, it is often required to test whether or not the exogenous
covariates are relevant. For example, in an ARCH$(1)$-X model defined by
$Y_{t}=\xi_{t}\sigma_{t}\text{ with
}\sigma^{2}_{t}=\alpha^{*}_{0}+\alpha^{*}_{1}Y^{2}_{t-k}+{\gamma^{*}}^{\prime}X_{t-1}$,
the true parameter vector is
$\theta^{*}=(\alpha^{*}_{0},\alpha^{*}_{1},\gamma^{*})\in\Theta\subset]0,\infty[\times{[0,\infty[}^{dx+1}$.
The significance test of the covariate $X_{t}$ consists to verify the nullity
of the parameter $\gamma^{*}$; that is, if the true parameter vector can be of
the form $\theta_{0}=(\alpha_{0},\alpha_{1},0)$ which is not an interior point
of $\Theta$. In this situation, it is impossible to apply the asymptotic
normality results based on the classical assumption of ”interior point” to
derive the asymptotic behavior of the test statistic used. To take into
account such a scenario in the general class (1.1), we will consider that the
component $i$ of $\theta^{*}$ is constrained if the $i$-th section of $\Theta$
is of the form $[\underline{\theta}_{i},\overline{\theta}_{i}]$ with
$\underline{\theta}_{i}<\overline{\theta}_{i}$. Assume that the $d_{2}$ (with
$d_{2}\in\\{0,\ldots,d\\}$) last components of $\theta^{*}$ are constrained,
and let $d_{1}=d-d_{2}$. Therefore, if $d_{2}\geq 1$ and
$\theta^{*}_{i}\in\\{\underline{\theta}_{i},\overline{\theta}_{i}\\}$ with
$i>d_{1}$, then $\theta^{*}$ is not an interior point of $\Theta$. For
instance, in a scenario where $\theta^{*}_{i}=\underline{\theta}_{i}$, with
the QMLE
$\widehat{\theta}_{n}=(\widehat{\theta}_{1,n},\ldots,\widehat{\theta}_{d,n})$,
it holds that $\sqrt{n}(\widehat{\theta}_{i,n}-\theta^{*}_{i})\in[0,\infty)$
which cannot tend to a Gaussian distribution with mean $0$. By convention, it
is assumed that $\theta^{*}\in\overset{\circ}{\Theta}$ if $d_{2}=0$. When
$d_{2}\geq 1$ and the set $\Theta$ is assumed to be large enough, then the
following relation holds:

$\bigcup_{n\geq 1}\left\\{\sqrt{n}\left(\theta-\theta^{*}\right),\
\theta\in\Theta\right\\}=\mathcal{C}~{}~{}\text{ with
}~{}~{}\mathcal{C}=\prod_{i=1}^{d}\mathcal{C}_{i},$ (2.4)

where $\mathcal{C}_{i}=[0,\infty[$ when $i>d_{1}$ and
$\theta^{*}_{i}=\underline{\theta}_{i}$, $\mathcal{C}_{i}=]\infty,0]$ when
$i>d_{1}$ and $\theta^{*}_{i}=\overline{\theta}_{i}$, and
$\mathcal{C}_{i}=\mathbb{R}$ otherwise. The set $\mathcal{C}$ is a convex cone
which is equal to $\mathbb{R}^{d}$ if $\theta^{*}\in\overset{\circ}{\Theta}$.

Let us define the following matrices

$F=\mathbb{E}\Big{[}\frac{\partial^{2}q_{0}(\theta^{*})}{\partial\theta\partial\theta^{\prime}}\Big{]}~{}~{}~{}\text{
and }~{}~{}~{}G=\mathbb{E}\Big{[}\frac{\partial
q_{0}(\theta^{*})}{\partial\theta}\frac{\partial
q_{0}(\theta^{*})}{\partial\theta^{\prime}}\Big{]}.$ (2.5)

Under the assumptions A${}_{i}(f_{\theta},\Theta)$,
A${}_{i}(M_{\theta},\Theta)$ (with $i=0,1,2$), one can show the existence of
$F$ and $G$. In addition, in view to (A2), the same arguments as in Bardet and
Wintenberger (2009) allow to establish that matrix $F$ is positive definite.
Consider then the $F$-scalar product $\langle x,y\rangle_{F}=x^{\prime}Fy$ and
the norm ${\|x\|}^{2}_{F}=x^{\prime}Fx$ for $x,y\in\mathbb{R}^{d}$. Let us
define the $F$-orthogonal projection of a vector $Z\in\mathbb{R}^{d}$ on the
cone $\mathcal{C}$ as follows:

$Z^{\mathcal{C}}=\text{arg}\inf_{C\in\mathcal{C}}\left\|C-Z\right\|_{F}.$

This definition is equivalent to

$Z^{\mathcal{C}}\in\mathcal{C}~{}~{}\text{ with }~{}~{}\left\langle
Z-Z^{\mathcal{C}},C-Z^{\mathcal{C}}\right\rangle\leq 0,~{}\ \forall
C\in\mathcal{C}.$ (2.6)

Note that, when $\theta^{*}\in\overset{\circ}{\Theta}$, we have
$Z^{\mathcal{C}}=Z$. Combining all the regularity conditions and definitions
given above, we obtain the following main result.

###### Theorem 2.3

Assume that (A0)-(A2), (A${}_{i}(f_{\theta},\Theta)$),
(A${}_{i}(M_{\theta},\Theta)$) (for $i=0,1,2$) and (2.2) (with $r\geq 4$) hold
with

$\alpha^{(i)}_{k,Y}(f_{\theta},\Theta)+\alpha^{(i)}_{k,X}(f_{\theta},\Theta)+\alpha^{(i)}_{k_{\theta},Y}(M_{\theta},\Theta)+\alpha^{(i)}_{k,X}(M_{\theta},\Theta)+\alpha^{(i)}_{k_{\theta},Y}(H_{\theta},\Theta)+\alpha^{(i)}_{k,X}(H_{\theta},\Theta)=O(k^{-\gamma}),$
(2.7)

for $i=0,1,2$ and some $\gamma>3/2$.

  * •

If $\theta^{*}\in\Theta\cap\Theta(r)$ with $r\geq 4$, then

$\sqrt{n}\left(\widehat{\theta}_{n}-\theta^{*}\right)\begin{array}[t]{c}\stackrel{{\scriptstyle{\cal
D}}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}Z^{\mathcal{C}}~{}~{}~{}with~{}~{}~{}Z\sim\mathcal{N}_{d}\left(0,\Sigma\right),~{}~{}~{}where~{}~{}~{}\Sigma:=F^{-1}GF^{-1}.$

  * •

If $\theta^{*}\in\overset{\circ}{\Theta}\cap\Theta(r)$ with $r\geq 4$, then

$\widehat{\theta}_{n}-\theta^{*}=O\bigg{(}\sqrt{\frac{\log\log
n}{n}}\bigg{)}~{}~{}a.s.$

The matrix $\Sigma$ can be consistently estimated by
$\widehat{\Sigma}_{n}=F_{n}(\widehat{\theta}_{n})^{-1}G_{n}(\widehat{\theta}_{n})F_{n}(\widehat{\theta}_{n})^{-1}$,
where

$F_{n}(\widehat{\theta}_{n})=\frac{1}{n}\sum_{t=1}^{n}\frac{\partial^{2}q_{t}(\widehat{\theta}_{n})}{\partial\theta\partial\theta^{\prime}}~{}~{}\text{
and }~{}~{}G_{n}(\widehat{\theta}_{n})=\frac{1}{n}\sum_{t=1}^{n}\frac{\partial
q_{t}(\widehat{\theta}_{n})}{\partial\theta}\frac{\partial
q_{t}(\widehat{\theta}_{n})}{\partial\theta^{\prime}}.$

Now, we are interested to investigate whether or not a given subset of
components of $\theta^{*}$ are equal to some fixed vector. To do so, consider
the following hypothesis testing:

$H_{0}:\,\Gamma\theta^{*}=\vartheta_{0}~{}~{}~{}~{}~{}~{}\text{ against
}~{}~{}~{}~{}~{}~{}H_{1}:\,\Gamma\theta^{*}\neq\vartheta_{0},$ (2.8)

where $\Gamma$ is a $d_{0}\times d$ full-rank matrix and $\vartheta_{0}$ is a
vector of dimension $d_{0}$. Define the Wald-type test statistic given by

$W_{n}=n(\Gamma\widehat{\theta}_{n}-\vartheta_{0})^{\prime}(\Gamma\widehat{\Sigma}_{n}\Gamma^{\prime})^{-1}(\Gamma\widehat{\theta}_{n}-\vartheta_{0}).$
(2.9)

Under $H_{0}$, the asymptotic behavior of $W_{n}$ is given by the following
theorem.

###### Theorem 2.4

Under $H_{0}$, assume that the assumptions of Theorem 2.3 hold. Then

$W_{n}\begin{array}[t]{c}\stackrel{{\scriptstyle{\cal
D}}}{{\longrightarrow}}\\\ {\scriptstyle n\rightarrow\infty}\end{array}(\Gamma
Z^{\mathcal{C}})^{\prime}(\Gamma\Sigma\Gamma^{\prime})^{-1}\Gamma
Z^{\mathcal{C}}~{}~{}~{}~{}\text{with}~{}~{}~{}~{}Z\sim\mathcal{N}_{d}\left(0,\Sigma\right).$

By the above theorem, at a nominal level $\alpha\in(0,1)$, the critical region
of the test is $(W_{n}>q_{\alpha})$, where $q_{\alpha}$ is the
$(1-\alpha)$-quantile of the distribution of $(\Gamma
Z^{\mathcal{C}})^{\prime}(\Gamma\Sigma\Gamma^{\prime})^{-1}\Gamma
Z^{\mathcal{C}}$. The critical value $q_{\alpha}$ can be computed through
Monte-Carlo simulations. The following corollary follows immediately when
$\theta^{*}$ belongs to the interior of the parameter space.

###### Corollary 2.5

Assume that the conditions of Theorem 2.4 hold. If
$\theta^{*}\in\overset{\circ}{\Theta}$, then $W_{n}$ converges to a chi-square
distribution with $d_{0}$ degrees of freedom.

Under $H_{1}$, one can easily see that
$W_{n}\underset{n\rightarrow\infty}{\overset{a.s}{\longrightarrow}}+\infty$;
which shows that the test is consistent in power. In the empirical studies, we
will restrict our attention to test the relevance of the exogenous covariates
by using the hypothesis (2.8) with $\vartheta_{0}=0$ and an appropriate matrix
$\Gamma$.

##  3 Model selection

###  3.1 Model selection framework

Assume that $(Y_{1},\ldots,Y_{n})$ is a trajectory of the process
$Y=\\{Y_{t},\,t\in\mathbb{Z}\\}$ satisfying
$\mathcal{AC}$-$X(M_{\theta^{*}},f_{\theta^{*}})$ (defined as in (1.1)), where
the true parameter $\theta^{*}$ is unknown. Let $\mathcal{M}$ be a finite
collection of models belonging to $\mathcal{AC}$-$X(M_{\theta},f_{\theta})$
with $\theta\in\Theta$. Assume that $\mathcal{M}$ contains at least the true
model $m^{*}$ corresponding to the parameter $\theta^{*}$. Our objective is to
develop a procedure that allows to select the ”best model” (that we denote by
$\widehat{m}_{n}$) among the collection $\mathcal{M}$ such that it is ”close”
to $m^{*}$ for $n$ large enough. To this end, we consider the following
definitions and notations in the sequel:

  * •

a model $m\in\mathcal{M}$ is considered as a subset of $\\{1,\ldots,d\\}$ and
denote by $|m|$ the dimension of $m$ (i.e, $|m|=\\#(m)$);

  * •

for $m\in\mathcal{M}$, $\Theta_{m}=\\{(\theta_{i})_{1\leq i\leq
d}\in\Theta\text{ with }\theta_{i}=0\text{ if }i\notin m\\}$ is a compact set
containing $\theta(m)$, where $\theta(m)$ denotes the parameter vector
associated to the model $m$;

  * •

$\mathcal{M}$ is considered as a subset of the power set of
$\\{1,\ldots,d\\}$; that is, $\mathcal{M}\in\mathcal{P}(\\{1,\ldots,d\\})$.

For instance, when the observations $Y_{1},\ldots,Y_{n}$ are generated from a
ARMAX$(p^{*},q^{*},s^{*})$ model (defined below), the collection $\mathcal{M}$
of the competing models could be considered as a family of ARMAX$(p,q,s)$ with
$(p,q,s)\in\\{0,1,\ldots,p_{max}\\}\times\\{0,1,\ldots,q_{max}\\}\times\\{0,1,\ldots,s_{max}\\}$,
where $p_{max},q_{max},s_{max}$ are the fixed upper bounds of the orders
satisfying $p_{max}\geq p^{*}$, $q_{max}\geq q^{*}$, $s_{max}\geq s^{*}$. The
parameter space $\Theta$ is a compact subset of
$\mathbb{R}^{p_{max}+q_{max}+s_{max}}$, and thus a model $m$ is a subset of
$\\{1,2,\ldots,p_{max}+q_{max}+s_{max}\\}$.

###  3.2 Model selection criterion and asymptotic results

Note that, under the identifiability assumption (A0), one can show that, for
all $m\in\mathcal{M}$, the function $\theta\mapsto-\mathbb{E}[q_{0}(\theta)]$
has a unique maximum in $\Theta_{m}$ (see proof of Theorem 2.2). Let us thus
define the ”best” parameter associated to the model $m$ as

$\theta^{*}(m):=\underset{\theta\in\Theta_{m}}{\text{argmin}}\left(\mathbb{E}[q_{0}(\theta)]\right).$

When $m\supseteq m^{*}$, we have $\theta^{*}(m)=\theta^{*}(m^{*})=\theta^{*}$;
that is, $\theta^{*}(m)$ will play the role of the true parameter $\theta^{*}$
in cases of ”true” or overfitted model. For $m\in\mathcal{M}$, we define the
QMLE of $\theta^{*}(m)$ as

$\widehat{\theta}(m):=\underset{\theta\in\Theta_{m}}{\text{argmax}}\left(\widehat{L}_{n}(\theta)\right).$
(3.1)

Now, define the penalized criteria given by

$\widehat{C}_{n}(m):=-2\widehat{L}_{n}(\widehat{\theta}(m))+\kappa_{n}|m|,\text{
for all }m\in\mathcal{M},$ (3.2)

where $(\kappa_{n})_{n\in\mathbb{N}}$ is an increasing sequence of the
regularization parameter (possibly data-dependent) that will be used to
calibrate the penalty term, and $|m|$ is the number of non-zero components of
$\theta^{*}(m)\in\Theta_{m}$ that will be called the dimension of the model
$m$. The selection of the ”best” model $\widehat{m}_{n}$ is then obtained by
minimizing the penalized contrast; that is,

$\widehat{m}_{n}:=\underset{m\in\mathcal{M}}{\text{argmin}}\left(\widehat{C}_{n}(m)\right).$
(3.3)

Using the results of Theorems 2.2 and 2.3, we establish the asymptotic
behavior of the model selection procedure, as shown in the following theorem.

###### Theorem 3.1

Let $(Y_{1},\ldots,Y_{n})$ be a trajectory of a process belonging to
$\mathcal{AC}$-$X(M_{\theta^{*}},f_{\theta^{*}})$, where
$\theta^{*}\in\Theta\cap\Theta(r)$ with $r>4$. Assume that (A0)-(A2),
(A${}_{i}(f_{\theta},\Theta)$), (A${}_{i}(M_{\theta},\Theta)$) (or
(A${}_{i}(H_{\theta},\Theta)$)) (for $i=0,1,2$) and (2.2) (with $r>4$) hold
with
$\kappa_{n}/n\begin{array}[t]{c}\stackrel{{\scriptstyle}}{{\longrightarrow}}\\\
{\scriptstyle n\rightarrow\infty}\end{array}0$. Suppose that when
$\theta^{*}\in\overset{\circ}{\Theta}$,

$\sum_{k\geq 1}\frac{1}{\sqrt{k\log\log k}}\sum_{j\geq
k}\sum_{i=0}^{2}\big{\\{}\alpha^{(i)}_{j,Y}(f_{\theta},\Theta)+\alpha^{(i)}_{j,X}(f_{\theta},\Theta)+\alpha^{(i)}_{j,Y}(M_{\theta},\Theta)+\alpha^{(i)}_{j,X}(M_{\theta},\Theta)\\\
+\alpha^{(i)}_{j,Y}(H_{\theta},\Theta)+\alpha^{(i)}_{j,X}(H_{\theta},\Theta)\big{\\}}<\infty.$
(3.4)

  1. (i.)

If $\kappa_{n}/\sqrt{\log\log
n}\begin{array}[t]{c}\stackrel{{\scriptstyle}}{{\longrightarrow}}\\\
{\scriptstyle n\rightarrow\infty}\end{array}\infty$, then

$\widehat{m}_{n}\begin{array}[t]{c}\stackrel{{\scriptstyle{\cal
P}}}{{\longrightarrow}}\\\ {\scriptstyle n\rightarrow\infty}\end{array}m^{*}.$

  2. (ii.)

When $\theta^{*}\in\overset{\circ}{\Theta}$, there exists a constant $c$ such
that if $~{}\underset{n\rightarrow\infty}{\liminf}(\kappa_{n}/\log\log n)>c$,
then

$\widehat{m}_{n}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}m^{*}.$

  3. (iii.)

If $\theta^{*}\in\overset{\circ}{\Theta}$ and (3.4) holds, then

$\widehat{\theta}(\widehat{m}_{n})-\theta^{*}=O\bigg{(}\sqrt{\frac{\log\log
n}{n}}\bigg{)}.$

Remark that, if
$\sum_{i=0}^{2}\big{\\{}\alpha^{(i)}_{j,Y}(f_{\theta},\Theta)+\alpha^{(i)}_{j,X}(f_{\theta},\Theta)+\alpha^{(i)}_{j,Y}(M_{\theta},\Theta)+\alpha^{(i)}_{j,X}(M_{\theta},\Theta)+\alpha^{(i)}_{j,Y}(H_{\theta},\Theta)+\alpha^{(i)}_{j,X}(H_{\theta},\Theta)\big{\\}}=O(j^{-\gamma})$
for some $\gamma>3/2$, then (3.4) is satisfied. The first and second parts of
Theorem 3.1 show the consistency of the selection procedure; in particular,
the second part provides sufficient conditions for the consistency of the HQC
procedure. The last part establishes that the estimator of the parameter of
the selected model $\widehat{\theta}(\widehat{m}_{n})$ obeys the law of
iterated logarithm.

##  4 Some examples

In this section, we detail some particular processes satisfying the class
(1.1). We show that the regularity conditions required for the main results
are satisfied for these processes, with a particular emphasis on the
identifiability assumption. For each example discussed, we consider that
$X_{t}=(X_{1,t},X_{2,t},\ldots,X_{d_{x},t})\in\mathbb{R}^{d_{x}}$
($d_{x}\in\mathbb{N}$) represents a vector of covariates; and
$(\xi_{t})_{t\in\mathbb{Z}}$ is a sequence of zero-mean i.i.d. random variable
satisfying $\mathbb{E}(\xi^{r}_{0})<\infty$ for some $r\geq 2$ and
$\mathbb{E}(\xi^{2}_{0})=1$.

###  4.1 ARMAX and ARX$(\infty)$ models

As first example, consider a linear ARMAX$(p^{*},q^{*},s^{*})$ model defined
by

$Y_{t}=\sum_{i=1}^{p^{*}}\alpha^{*}_{i}Y_{t-i}+\xi_{t}+\sum_{i=1}^{q^{*}}\beta^{*}_{i}\xi_{t-i}+\sum_{i=1}^{s^{*}}{\gamma^{*}}^{\prime}_{i}X_{t-i},~{}\forall
t\in\mathbb{Z},$ (4.1)

where $\alpha^{*}_{i},\beta^{*}_{i}\in\mathbb{R}$ (for $1\leq i\leq p^{*}$,
for $1\leq i\leq q^{*}$) and $\gamma^{*}_{i}\in\mathbb{R}^{d_{x}}$ (for $1\leq
i\leq s^{*}$). This model has been investigated by several authors and various
issues have been addressed; see, among others, Hannan et al. (1980), Hannan
and Deistle (1988), Hannan and Deistler (2012) (within the multivariate
framework), and Bierens (1994) (within the linear and nonlinear frameworks).
We also refer to Hoque and Peters (1986) who carried out the ARMAX$(1,0,1)$
process. The true parameter of the model is
$\theta^{*}=(\alpha^{*}_{1},\ldots,\alpha^{*}_{p^{*}},\beta^{*}_{1},\ldots,\beta^{*}_{q^{*}},\gamma^{*}_{1},\ldots,\gamma^{*}_{s^{*}})\in\Theta$,
where $\Theta$ is a compact set such as: for all
$\theta=\left(\alpha_{1},\ldots,\alpha_{p^{*}},\beta_{1},\ldots,\beta_{q^{*}},\gamma_{1},\ldots,\gamma_{s^{*}}\right)\in\Theta$,
$\sum\limits_{i=1}^{p^{*}}|\alpha_{i}|+\sum\limits_{i=1}^{q^{*}}|\beta_{i}|<1$.
Let us define the following lag polynomials for any $\theta\in\Theta$:

$A_{\theta}(L)=1-\sum\limits_{i=1}^{p^{*}}\alpha_{i}L^{i},~{}~{}B_{\theta}(L)=1+\sum\limits_{j=1}^{q^{*}}\beta_{j}L^{j}~{}~{}\text{and}~{}~{}C_{\theta}(L)=\sum\limits_{k=1}^{s^{*}}{\gamma^{*}}^{\prime}_{k}L^{k},$

where $L$ denotes the lag operator. The polynomial $B_{\theta^{*}}(L)$ is
invertible and the model (4.1) can be rewritten as

$\frac{A_{\theta^{*}}(L)}{B_{\theta^{*}}(L)}Y_{t}=\xi_{t}+\frac{C_{\theta^{*}}(L)}{B_{\theta^{*}}(L)}X_{t},\
\forall t\in\mathbb{Z}.$ (4.2)

For any $\theta\in\Theta$, again from the invertibility of $B_{\theta}(L)$,
one can find two sequences $(\phi_{k}(\theta))_{k\in\mathbb{N}}$ and
$(\varphi_{k}(\theta))_{k\in\mathbb{N}}$ such that

$\frac{A_{\theta}(L)}{B_{\theta}(L)}=1-\sum\limits_{k\geq
1}\phi_{k}(\theta)L^{k}~{}~{}\text{and}~{}~{}\frac{C_{\theta}(L)}{B_{\theta}(L)}=\sum\limits_{k\geq
1}\varphi^{\prime}_{k}(\theta)L^{k}.$

Thus, (4.2) is equivalent to

$Y_{t}=\sum\limits_{k\geq 1}\phi_{k}(\theta^{*})Y_{t-k}+\sum\limits_{k\geq
1}\varphi^{\prime}_{k}(\theta^{*})X_{t-k}+\xi_{t},\ \forall t\in\mathbb{Z},$
(4.3)

which shows that this process belongs to the class
$\mathcal{AC}$-$X(M_{\theta^{*}},f_{\theta^{*}})$, with $M^{t}_{\theta}\equiv
1$ and $f^{t}_{\theta}=\sum\limits_{k\geq
1}\phi_{i}(\theta)Y_{t-k}+\sum\limits_{i\geq
k}\varphi^{\prime}_{k}(\theta)X_{t-k}$, for any $\theta\in\Theta$. We deduce
that the Lipschitz coefficients are
$\alpha^{(0)}_{k,Y}(M_{\theta},\\{\theta\\})=\alpha^{(0)}_{k,X}(M_{\theta},\\{\theta\\})=0$,
$\alpha^{(0)}_{k,Y}(f_{\theta},\\{\theta\\})=|\phi_{k}(\theta)|$ and
$\alpha^{(0)}_{k,X}(f_{\theta},\\{\theta\\})=\|\varphi_{k}(\theta)\|$ for all
$k\geq 1$. The stationarity set for any $r\geq 1$ is

$\Theta(r)=\big{\\{}\theta\in\mathbb{R}^{p^{*}+q^{*}+s^{*}\times d_{x}}\
\big{/}\ \sum\limits_{k\geq
1}\max\left\\{\alpha_{k}(g),|\phi_{k}(\theta)|\right\\}<1\big{\\}}.$

The assumption (A1) holds with $\underline{h}=1$. To assure the correctness of
this ARMAX model specification, we impose the following additional conditions:

  1. (B0):

$\mathbb{E}\left[\xi_{t}X_{t^{\prime}}\right]=0$ for all
$(t,t^{\prime})\in\mathbb{Z}^{2}$;

  2. (B1):

$A_{\theta^{*}}(z)$ and $B_{\theta^{*}}(z)$ have no common root;

  3. (B2):

if $(c_{k})_{k\in\mathbb{N}}$ is a sequence of vector of $\mathbb{R}^{d_{x}}$
such as $\exists c_{k_{0}}\neq 0$ (with $k_{0}\in\mathbb{N}$), then
$\sum\limits_{k\geq 1}c^{\prime}_{k}X_{t-k}$ is non-degenerate.

Observe that, the assumption (B0) excludes situations where the covariate
$c^{\prime}X_{t}$ is replaced by $Y_{\nu(t)}$ for some constant
$c\in\mathbb{R}^{d_{x}}$ and a function $\nu:\mathbb{Z}\rightarrow\mathbb{Z}$
satisfying $\nu(t)\leq t$ for all $t\in\mathbb{Z}$; in particular, the case
$c^{\prime}X_{t}=Y_{t}$ is excluded. Under the conditions (B0)-(B2), one can
easily prove that this model is identifiable. Indeed, let
$(t,\theta)\in\mathbb{Z}\times\Theta$ such as
$f^{t}_{\theta^{*}}=f^{t}_{\theta}$ a.s. Let us show that $\theta=\theta^{*}$.
When $f^{t}_{\theta^{*}}=f^{t}_{\theta}$ a.s., it follows that

$\sum\limits_{k\geq
1}\big{(}\phi_{k}(\theta^{*})-\phi_{k}(\theta)\big{)}Y_{t-k}=\sum\limits_{k\geq
1}\big{(}\varphi^{\prime}_{k}(\theta)-\varphi^{\prime}_{k}(\theta^{*})\big{)}X_{t-k}.$
(4.4)

Let us first prove by contradiction that
$\phi_{k}(\theta^{*})=\phi_{k}(\theta),\ \forall k\geq 1$. Let $m\geq 1$ be
the smallest integer satisfying $\phi_{m}(\theta^{*})\neq\phi_{m}(\theta)$.
Then, (4.4) gives

$Y_{t-m}=\frac{1}{\phi_{m}(\theta)-\phi_{m}(\theta^{*})}\bigg{\\{}\sum\limits_{k>m}\big{(}\phi_{k}(\theta^{*})-\phi_{k}(\theta)\big{)}Y_{t-k}+\sum\limits_{k\geq
1}\big{(}\varphi^{\prime}_{k}(\theta^{*})-\varphi^{\prime}_{k}(\theta)\big{)}X_{t-k}\bigg{\\}}.$

This implies that from (B0),

$\mathbb{E}\left[\xi_{t-m}Y_{t-m}\right]=\frac{1}{\phi_{m}(\theta)-\phi_{m}(\theta^{*})}\sum\limits_{k>m}\big{(}\phi_{k}(\theta^{*})-\phi_{k}(\theta)\big{)}\mathbb{E}\left[\xi_{t-m}Y_{t-k}\right].$
(4.5)

Note that $A_{\theta^{*}}(L)$ is also invertible since $\theta^{*}\in\Theta$.
Therefore, (4.1) is equivalent to

$Y_{t}=\frac{B_{\theta^{*}}(L)}{A_{\theta^{*}}(L)}\xi_{t}+\frac{C_{\theta^{*}}(L)}{A_{\theta^{*}}(L)}X_{t}=\xi_{t}+\sum\limits_{i\geq
1}a_{i}(\theta^{*})\xi_{t-i}+\sum\limits_{i\geq
1}{b_{i}}^{\prime}(\theta^{*})X_{t-i},\ \forall t\in\mathbb{Z},$

where $(a_{i}(\theta^{*}))_{i\in\mathbb{N}}$ and
$(b_{i}(\theta^{*}))_{i\in\mathbb{N}}$ are two sequences of real numbers and
vectors. Thus, from (B0) and the proprieties of $(\xi_{t})_{t\in\mathbb{Z}}$,
we deduce: $\mathbb{E}[\xi_{t}Y_{t}]=1$ for any $t\in\mathbb{Z}$ and
$\mathbb{E}[\xi_{t}Y_{t^{\prime}}]=0$ for any $t^{\prime}<t$. It comes from
(4.5) that $\mathbb{E}\left[\xi_{t-m}Y_{t-m}\right]=0$, which is impossible.
This establishes that $\phi_{k}(\theta^{*})=\phi_{k}(\theta),\ \forall k\geq
1$. Consequently,

$\frac{A_{\theta^{*}}(L)}{B_{\theta^{*}}(L)}=\frac{A_{\theta}(L)}{B_{\theta}(L)}~{}~{}\text{and}~{}~{}\sum\limits_{i\geq
1}\big{(}\varphi^{\prime}_{i}(\theta^{*})-\varphi^{\prime}_{i}(\theta)\big{)}X_{t-i}=0,$

where the last equality is obtained from the relation (4.4). In view of (B1)
and (B2), this implies that $A_{\theta^{*}}(L)=A_{\theta}(L)$,
$B_{\theta^{*}}(L)=B_{\theta}(L)$ and $C_{\theta^{*}}(L)=C_{\theta}(L)$, which
proves that $\theta=\theta^{*}$.

In general, one can define the ARX$(\infty)$ process as follows:

$Y_{t}=\psi_{0}(\theta^{*})+\sum\limits_{i\geq
1}\psi_{i}(\theta^{*})Y_{t-i}+\sum\limits_{i\geq
1}\gamma^{\prime}_{i}(\theta^{*})X_{t-i}+\xi_{t},\ \forall t\in\mathbb{Z},$
(4.6)

where $(\psi_{i}(\theta^{*}))_{i\geq 0}$ and
$(\gamma^{\prime}_{i}(\theta^{*}))_{i\geq 1}$ are two sequences, and
$\theta^{*}$ is the true parameter. If $\sum\limits_{k\geq
1}\max\left\\{\alpha_{k}(g),|\psi_{k}(\theta^{*})|\right\\}<1$, there exists a
stationary and ergodic solution with $r$-order moment. If the assumption (B0)
holds for (4.6), then one can go along similar lines as in the model (4.2) to
show that a sufficient condition for the model (4.6) to be identifiable is
that the function $\theta\mapsto\psi_{i}(\theta)$ (for $i\geq 0$) is
injective. The condition (A1) holds if
$\inf_{\theta\in\Theta}\psi_{0}(\theta)>0$.

###  4.2 ARCH-X$(\infty)$ models

Consider an ARCH-X$(\infty)$ model defined by:

$Y_{t}=\xi_{t}\sigma_{t}(\theta^{*})\ \text{ with }\
\sigma^{2}_{t}(\theta^{*})=\phi_{0}(\theta^{*})+\sum\limits_{k=1}^{\infty}\phi_{k}(\theta^{*})Y^{2}_{t-k}+\sum\limits_{k=1}^{\infty}\gamma^{\prime}_{k}(\theta^{*})X_{t-k},\
\forall t\in\mathbb{Z},$ (4.7)

where $\theta^{*}$ is the true parameter, $(\phi_{k}(\theta^{*}))_{k\geq 0}$
and $(\gamma_{k}(\theta^{*}))_{k\geq 1}$ are two non-negative (componentwise
for $\gamma_{k}(\theta^{*})$) sequences with $(\phi_{0}(\theta^{*}))\neq 0$,
and $X_{t}$ is a vector of non-negative (componentwise) covariates. This model
is an example of the class (1.1) with $f^{t}_{\theta}=0$ and
$M^{t}_{\theta}=\sigma_{t}(\theta)$. Assume that $\theta^{*}\in\Theta$, where
$\Theta$ is a compact set defined by

$\Theta=\big{\\{}\theta\in\mathbb{R}^{d}\ \big{/}\ \sum\limits_{k\geq
1}\max\left\\{\alpha_{k}(g),|\phi_{k}(\theta)|\right\\}<1\text{ and
}\sum\limits_{k\geq 1}\|\gamma_{k}(\theta)\|<\alpha_{U}\text{ for some
}\alpha_{U}>0\big{\\}}.$

Therefore, A${}_{0}(H_{\theta},\Theta)$ holds and $\Theta(2)=\Theta$. Assume
that $\inf_{\theta}\phi_{0}(\theta)>0$, which ensures that assumption (A1) is
satisfied. Now, denote by $\mathcal{F}_{t,i}$ the $\sigma-$field generated by
$\left\\{\xi_{t-j},j>i,\ X_{t-k},k>0\right\\}$. The following assumptions are
needed to ensure the identifiability.

  1. (B3):

the support of the distribution of $\xi_{t-i}$ given $\mathcal{F}_{t,i}$
contains at least three points;

  2. (B4):

$\theta\mapsto\phi_{k_{0}}(\theta)$ is an injective function for some $\exists
k_{0}>0$;

  3. (B5):

$\theta\mapsto\phi_{0}(\theta)$ (or $\theta\mapsto\gamma_{k_{0}}(\theta^{*})$
for some $\exists k_{0}>0$) is an injective function, and the condition (B2)
holds for this model.

Note that (B3) is a required identifiability condition which entails that
$\xi^{2}_{t-i}$ given $\mathcal{F}_{t,i}$ is a non-degenerate random variable.
But, both the assumptions (B4) and (B5) are not necessary; that is, the model
is identifiable if B3 and ((B4) or (B5)) holds. Indeed, let $\theta\in\Theta$
is such that $H_{\theta}=H_{\theta}^{*}$. Then

$\sum\limits_{k=1}^{\infty}(\phi_{k}(\theta^{*})-\phi_{k}(\theta))Y^{2}_{t-k}=\phi_{0}(\theta)-\phi_{0}(\theta^{*})+\sum\limits_{k=1}^{\infty}(\gamma^{\prime}_{k}(\theta)-\gamma^{\prime}_{k}(\theta^{*}))X_{t-k}.$
(4.8)

By contradiction, assume that $\phi_{k}(\theta^{*})\neq\phi_{k}(\theta)$ for
some $k>0$ and denote by $m>0$ the smallest integer satisfying
$\phi_{m}(\theta^{*})\neq\phi_{m}(\theta)$. Therefore,

$\left(\phi_{m}(\theta^{*})-\phi_{m}(\theta)\right)Y^{2}_{t-m}=\phi_{0}(\theta)-\phi_{0}(\theta^{*})+\sum\limits_{k>m}(\phi_{k}(\theta)-\phi_{k}(\theta^{*}))Y^{2}_{t-k}+\sum\limits_{k=1}^{\infty}(\gamma^{\prime}_{k}(\theta)-\gamma^{\prime}_{k}(\theta^{*}))X_{t-k},$

and (4.7) yields that

$\xi^{2}_{t-m}=\frac{1}{\left(\phi_{m}(\theta^{*})-\phi_{m}(\theta)\right)\sigma^{2}_{t-m}(\theta^{*})}\big{[}\phi_{0}(\theta)-\phi_{0}(\theta^{*})+\sum\limits_{k>m}(\phi_{k}(\theta)-\phi_{k}(\theta^{*}))Y^{2}_{t-k}+\sum\limits_{k=1}^{\infty}(\gamma^{\prime}_{k}(\theta)-\gamma^{\prime}_{k}(\theta^{*}))X_{t-k}\big{]}.$
(4.9)

By a recursive substitution, one can easily see that for all $k>m$, $Y_{t-k}$
is $\mathcal{F}_{t,m}-$measurable, and thus the right-hand side of (4.9). This
implies that $\xi^{2}_{t-m}$ given $\mathcal{F}_{t,m}$ is degenerate, which
contradicts the assumption (B3); hence,
$\phi_{k}(\theta^{*})=\phi_{k}(\theta),\forall k>0$. We deduce that:

  * •

if (B4) holds, then $\theta=\theta^{*}$;

  * •

else, if (B5) holds, we have from (4.8),

$\sum\limits_{k=1}^{\infty}(\gamma^{\prime}_{k}(\theta^{*})-\gamma^{\prime}_{k}(\theta))X_{t-k}=\phi_{0}(\theta)-\phi_{0}(\theta^{*}),$

which implies that $\gamma_{k}(\theta^{*})=\gamma_{k}(\theta)\ \forall k\geq
1$ (by the assumption (B2) in (B5)); and consequently,
$\phi_{0}(\theta)=\phi_{0}(\theta^{*})$. Hence, $\theta=\theta^{*}$.

Note that the GARCH-X process already introduced in the literature and widely
studied by some authors (see, for instance, Han and Kristensen (2014), Nana et
al. (2013) and Han (2015)) is a particular case of the ARCH-X$(\infty)$
process that we have considered here. Let us stress that, the i.i.d.
assumption for $(\xi_{t})_{t\in\mathbb{Z}}$ is a bit strong for the model
(4.7). This assumption, which is needed to the large class
$\mathcal{AC}$-$X(M_{\theta^{*}},f_{\theta^{*}})$ can be relaxed to
$(\xi_{t},\mathcal{F}_{t,0})_{t\in\mathbb{Z}}$ is a martingale difference
sequence (see for instance Francq and Thieu (2019) in the case of GARCH-X
model) when checking the identifiability. In the absence of covariates (i.e.,
the case of the standard ARCH$(\infty)$ model), the assumption (B3) can be
automatically reduced to: ”$\xi^{2}_{t}$ is a non-degenerate random variable”;
see, for instance, Franck and Zakoïan (2004) and Berkes et al. (2003) for some
works where such condition is assumed to address the identifiability for the
standard GARCH model. With the assumption (B3), the feedback from $Y_{t}$ to
$X_{t}$ (for instance, $c^{\prime}X_{t-1}=Y^{2}_{t-i}$ with
$c\in\mathbb{R}^{d_{x}}$) is excluded, because this would contradict the non-
degeneration of the variable $\xi^{2}_{t-i}|\mathcal{F}_{t,i}$.

###  4.3 ARX-GARCH models

Now, we propose an extension of the AR-GARCH processes (see Ling and McAleer
(2003)) to processes with exogenous covariates. Define the
ARX$(1)$-GARCH$(1,1)$ process by

$\left\\{\begin{array}[]{l}Y_{t}=a^{*}Y_{t-1}+\gamma^{*^{\prime}}X_{t-1}+\varepsilon_{t}\\\
\rule[0.0pt]{0.0pt}{14.22636pt}\varepsilon_{t}=\xi_{t}\sigma_{t}~{}\text{ with
}~{}\sigma^{2}_{t}=c^{*}_{0}+c^{*}_{1}\varepsilon^{2}_{t-1}+d^{*}\sigma^{2}_{t-1},\end{array}\right.$
(4.10)

where $|a^{*}|<1$, $c^{*}_{0}>0$, $0\leq c^{*}_{1}+d^{*}<1$ and $\gamma^{*}$
is a vector of a compact set included in $\mathbb{R}^{dx}$
($d_{x}\in\mathbb{N}$). The true parameter of the model is
$\theta^{*}=(a^{*},c^{*}_{0},c^{*}_{1},d^{*},\gamma^{*})$. Since
$d^{*}\in(0,1)$, we get
$\sigma^{2}_{t}=\frac{c^{*}_{0}}{1-d^{*}}+c^{*}_{1}\sum_{k=1}^{\infty}d^{*{k-1}}\varepsilon^{2}_{t-k}$.
Then, one can see that the model (4.10) belongs to the class
$\mathcal{AC}$-$X(M_{\theta^{*}},f_{\theta^{*}})$ with

$f^{t}_{\theta}=aY_{t-1}+\gamma^{\prime}X_{t-1}~{}~{}\text{ and
}~{}~{}M^{t}_{\theta}=\sqrt{\frac{c_{0}}{1-d}+c_{1}\sum_{k\geq
1}d^{k-1}(Y_{t-k}-aY_{t-k-1}-\gamma^{\prime}X_{t-k-1})^{2}},$

for all $\theta=(a,c_{0},c_{1},d,\gamma)\in\Theta\subset\mathbb{R}^{d_{x}+4}$.
The assumption A${}_{0}(f_{\theta},\\{\theta\\})$ holds with
$\alpha^{(0)}_{1,Y}(f_{\theta},\\{\theta\\})=|a|$,
$\alpha^{(0)}_{1,X}(f_{\theta},\\{\theta\\})=\|\gamma\|$ and
$\alpha^{(0)}_{k,Y}(f_{\theta},\\{\theta\\})=\alpha^{(0)}_{k,X}(f_{\theta},\\{\theta\\})=0$
for $k\geq 2$. Simple computations show that, the Lipschitz coefficients of
$M^{t}_{\theta}$ are:

$\left\\{\begin{array}[]{l}\alpha^{(0)}_{1,Y}(M_{\theta},\\{\theta\\})=c_{1}~{}\text{
and
}~{}\alpha^{(0)}_{k,Y}(M_{\theta},\\{\theta\\})=c_{1}(d^{k-1}+|a|d^{k-2})~{}~{}\text{
for }k\geq 2;\\\
\rule[0.0pt]{0.0pt}{17.07182pt}\par\alpha^{(0)}_{1,X}(M_{\theta},\\{\theta\\})=0~{}\text{
and
}~{}\alpha^{(0)}_{k,X}(M_{\theta},\\{\theta\\})=c_{1}\|\gamma\|d^{k-2}~{}~{}\text{
for }k\geq 2.\end{array}\right.$

For example, if the components of $X_{t}$ satisfy a Markov-structure as
follows:

$X_{i,t}=\varphi_{i}X_{i,t-1}+\eta_{i,t}~{}\text{ for all }1\leq i\leq
d_{x}\text{ and }t\in\mathbb{Z},$

where $|\varphi_{i}|<1$ and $\eta_{i,t}$ is a white noise, then the stability
condition (2.2) is satisfied with $\alpha_{1}(g)=\max_{1\leq i\leq
d_{x}}|\varphi_{i}|$ and $\alpha_{k}(g)=0$ for $k\geq 2$. In this case, the
stationarity set $\Theta(r)$ is defined by

$\Theta(r)=\left\\{\theta=(a,c_{0},c_{1},d,\gamma)\in\mathbb{R}^{d_{x}+4}\
\big{/}\
\max\left\\{\alpha_{1}(g),|a|+c_{1}\|\xi_{0}\|_{r}\right\\}+c_{1}(1-d)^{-1}(d+|a|)\|\xi_{0}\|_{r}<1\right\\}.$

In addition to the assumptions (B0) and (B3), we assume that

(B6): $c^{\prime}X_{1}$ is not degenerated if $c$ is a non zero vector of
$\mathbb{R}^{d_{x}}$.

Then, the identifiability of this model can be established as in the model
(4.7).

##  5 Simulation study

In this section, we consider a double autoregressive model with exogenous
covariates, defined by

$Y_{t}=\phi_{0}+\phi_{1}Y_{t-1}+\sum_{i=1}^{q}\psi_{i}X_{t-i}+\xi_{t}\sqrt{\alpha_{0}+\alpha_{1}Y^{2}_{t-1}+\sum^{q}_{i=1}\beta_{i}X^{2}_{t-i}},$
(5.1)

where $q\in\mathbb{N}$,
$~{}\alpha_{0},\alpha_{1},\beta_{1},\ldots,\beta_{q}>0$,
$\phi_{0},\phi_{1},\psi_{1},\ldots,\psi_{q}\in\mathbb{R}$, $\xi_{t}$ is a
white noise and $X_{t}$ is a real valued exogenous covariates. This model is a
particular case of the a factor double autoregressive (FDAR) process
introduced by Guo et al. (2014) to extend the double AR$(p)$ model proposed by
Ling (2017). We assume that $X_{t}$ is an AR$(1)$ process:

$X_{t}=\varphi_{0}+\varphi_{1}X_{t-1}+\eta_{t}~{}\text{ for all
}t\in\mathbb{Z},$

where $\eta_{t}$ is a Gaussian white noise. The AR parameter is set to
$(\varphi_{0},\varphi_{1})=(0.5,0.5)$ and $X_{t}$ is initialized at
$X_{0}\sim\mathcal{N}\left(\varphi_{0}/(1-\varphi_{1}),1/(1-\varphi^{2}_{1})\right)$.
Set $\psi=(\psi_{1},\ldots,\psi_{q})$ and
$\beta=(\beta_{1},\ldots,\beta_{q})$; the true parameter is
$\theta^{*}=(\phi_{0},\phi_{1},\alpha_{0},\alpha_{1},\psi,\beta)$. Based on
the examples discussed in Section 4, if the conditions (B0) and (B3) hold for
(5.1), then to satisfy the identifiability condition, it suffices to impose
the following assumption on the covariate:

(B7): if $(c_{k})_{1\leq k\leq q}$ is a sequence of vector of
$\mathbb{R}^{d_{x}}$ such as $\exists c_{k_{0}}\neq 0$ (with $1\leq k_{0}\leq
q$), then $\sum^{q}_{i=1}c_{k}X^{2}_{t-k}$ is not degenerated.

We first present some results from Monte Carlo simulations to assess the
asymptotic properties of the QMLE. We also investigate the empirical size and
power of the proposed procedure on testing the significance of the covariate
$X_{t}$. With $q=1$, consider the model (5.1) in the following situations:

  * •

scenario S0: $\theta^{*}=(0.15,-0.2,0.4,0.3,0,0)$;

  * •

scenario S1: $\theta^{*}=(0.15,-0.2,0.4,0.3,0.08,0)$;

  * •

scenario S’0: $\theta^{*}=(1,0.4,0.5,0.2,0,0)$;

  * •

scenario S’1: $\theta^{*}=(1,0.4,0.5,0.2,0.07,0.07)$.

The scenarios S0 and S’0 correspond to cases where the covariate is absent. We
consider the following significance tests:

$H_{0}:\,\theta^{*}=(0.15,-0.2,0.4,0.3,0,0)~{}~{}(S_{0})~{}~{}~{}~{}~{}~{}\text{
against }~{}~{}~{}~{}~{}~{}H_{1}:\,\theta^{*}\neq(0.15,-0.2,0.4,0.3,0,0);$
(5.2)

$H_{0}:\,\theta^{*}=(1,0.4,0.5,0.2,0,0)~{}~{}(S^{\prime}_{0})~{}~{}~{}~{}~{}~{}\text{
against }~{}~{}~{}~{}~{}~{}H_{1}:\,\theta^{*}\neq(1,0.4,0.5,0.2,0,0).$ (5.3)

In each scenario S0, S1, S’0 and S’1, we simulate $200$ replications with the
sample size $n=500,\,1000$ and test the nullity of the vector $(\psi,\beta)$
after estimating the parameters of interest. Table 1 contains the empirical
mean and root mean square error (RMSE) of each component of the estimator. The
last two columns of Table 1 indicate the empirical levels and powers of the
above tests at the nominal level $\alpha=0.05$, where the empirical powers are
computed under the alternative $H_{1}$ respectively in the scenario S1 and
S’1. For the scenario S’1, the histograms and estimated densities of the
estimates are plotted in Figure 1.

From these findings, one can see that, in all scenarios, the performance of
the QMLE is satisfactory in terms of the mean and that, the RMSE of the
estimators decreases when $n$ increases. This is consistent with the results
of Theorem 2.2. Also remark that, the fact of computing the QMLE with
$(\widehat{\psi},\widehat{\beta})$ for trajectories generated without
covariates (see the scenarios S0 and S’0) does not affect the performance of
the QMLE, which again confirms its good theoretical properties. As seen in
Figure 1, for each component of $\widehat{\theta}_{n}$, the estimated density
is very close to that of the normal distribution; which is in accordance with
the asymptotic results obtained from Theorem 2.3 when
$\theta^{*}\in\overset{\circ}{\Theta}$. The results of the test (see Table 1)
show that, the statistic $W_{n}$ is slightly undersized for a sample size of
$500$ observations, but the empirical levels are reasonable when $n=1000$ in
the sense that, they are very close to the nominal one. Further, the empirical
powers displayed increases with the sample size and are quite accurate.

Now, we are going to carry out other simulation experiments aimed at
evaluating the effectiveness of the proposed model selection procedure for
choosing the order $q$ in the model (5.1). To this end, consider the case
where $q=2$ as the ”true” model $m^{*}$. We generate trajectories in the
following situations:

  * •

scenario S${}^{*}_{1}$: $\theta^{*}=(0.6,0.45,0.5,0.15,1,0.7,0.6,0.35)$;

  * •

scenario S${}^{*}_{2}$: $\theta^{*}=(0.15,0.4,0.5,0.2,0.1,0.1,0.03,0.3)$.

For $n=100,125,150,\ldots,1000$, we simulate $100$ independent replications in
each of these two scenarios. The competing models considered are all process
satisfying (5.1) with $q\in\\{0,1,\ldots,9\\}$, which leads us to a collection
of 10 models. We compare the performances of the procedure with
$\kappa_{n}=\log n$ (see (3.2)) linked to the Bayesian Information Criteria
(BIC) and the procedure with $\kappa_{n}=c\log\log n$
($c\in\\{2,\,3.5,\,5\\}$) linked to the Hannan-Quinn information Criterion
(HQC). For each penalty, Figures 2 and 3 display the points
$(n,\widehat{q}_{n})$, where $\widehat{q}_{n}$ denotes the average of the
orders selected with trajectories of length $n$, as well as the curve of the
proportions of number of replications (frequencies) where the associated
criterion selects the true order.

From these figures, the first remark is that, for all the penalties, the
performances of the procedure increase with $n$ in each scenario. Further, the
probability of selecting the true order is very close to $1$ when $n=1000$.
This shows that, these procedures are in accordance with the results of
Theorem 3.1. One can notice that, in the scenario ${\bf S}^{*}_{1}$, the $\log
n$ penalty is more interesting for selecting the true order than the others
penalties for a small sample size (see Figure 2 for $n\leq 250$), while in the
scenario ${\bf S}^{*}_{2}$, the HQC with $c=2$ slightly outperforms the BIC
penalization when $n\leq 350$ (see Figure 3). However, the larger the sample
size, the $c\log\log n$ penalty (except for the case where $c=2$) provides the
same accuracies in comparison with the $\log n$ penalty, and displays
satisfactory results. The results also show that, as $c$ increases, the
performances of the $c\log\log n$ penalty increase, which reveals that the
common use of the classical HQC penalization (i.e, the $c\log\log n$ penalty
with $c=2$) is not always the optimal choice to select the best model with
this information criterion.

Table 1: Sample mean and RMSE$\times 10$ of the QMLE for the model (5.1) following the scenarios S0, S1, S’0 and S’1. The last two columns show the empirical levels and powers at the nominal level $0.05$ for the test of the relevance of the exogenous covariates. |  |  |  | QMLE |  |  Statistic $W_{n}$
---|---|---|---|---|---|---
Scenarios | $n$ |  |  | $\widehat{\phi}_{0}$ | $\widehat{\phi}_{1}$ | $\widehat{\alpha}_{0}$ | $\widehat{\alpha}_{1}$ | $\widehat{\psi}$ | $\widehat{\beta}$ |  | Levels | Powers
S0 | $500$ | Mean |  | $0.1488$ | $-0.2001$ | $0.3887$ | $0.2987$ | $0.0019$ | $0.0033$ |  | $0.035$ |
|  | Rmse |  | $0.0189$ | $0.0369$ | $0.0177$ | $0.0853$ | $0.0076$ | $0.0004$ |  |
| $1000$ | Mean |  | $0.1502$ | $-0.2023$ | $0.3911$ | $0.2979$ | $-0.0006$ | $0.0032$ |  | $0.045$ |
|  | Rmse |  | $0.0106$ | $0.0137$ | $0.0083$ | $0.0496$ | $0.0693$ | $0.0003$ |  |
[2pt/3pt]  S1 | $500$ | Mean |  | $0.1520$ | $-0.2012$ | $0.3857$ | $0.2997$ | $0.0806$ | $0.0049$ |  |  | $0.750$
|  | Rmse |  | $0.0238$ | $0.0299$ | $0.0193$ | $0.0814$ | $0.0100$ | $0.0009$ |  |
| $1000$ | Mean |  | $0.1511$ | $-0.1996$ | $0.3889$ | $0.3006$ | $0.0793$ | $0.0031$ |  |  | $0.970$
|  | Rmse |  | $0.0107$ | $0.0133$ | $0.0087$ | $0.0452$ | $0.0045$ | $0.0003$ |  |
S’0 | $500$ | Mean |  | $1.0003$ | $0.4009$ | $0.4793$ | $0.2004$ | $0.0020$ | $0.0065$ |  | $0.030$ |
|  | Rmse |  | $0.0664$ | $0.0200$ | $0.0452$ | $0.0081$ | $0.0147$ | $0.0016$ |  |
| $1000$ | Mean |  | $1.0015$ | $0.4005$ | $0.4854$ | $0.1986$ | $-0.0012$ | $0.0050$ |  | $0.055$ |
|  | Rmse |  | $0.0231$ | $0.0077$ | $0.0235$ | $0.0040$ | $0.0087$ | $0.0009$ |  |
[2pt/3pt]  S’1 | $500$ | Mean |  | $1.0022$ | $0.3993$ | $0.4943$ | $0.2005$ | $0.0704$ | $0.0702$ |  |  | $0.795$
|  | Rmse |  | $0.0674$ | $0.0196$ | $0.0645$ | $0.0086$ | $0.0202$ | $0.0099$ |  |
| $1000$ | Mean |  | $1.0019$ | $0.3970$ | $0.5052$ | $0.1979$ | $0.0680$ | $0.0672$ |  |  | $0.975$
|  | Rmse |  | $0.0345$ | $0.0096$ | $0.0300$ | $0.0047$ | $0.0104$ | $0.0050$ |  |
|  |  |  |  |  |  |  |  |  |  |  |
Figure 1: Histograms of the components of $\widehat{\theta}_{n}$ in the
scenario S’1 with sample size $n=1000$. The overlaying curves are the density
estimates and the dotted vertical lines represent the true values of the
parameters. Figure 2: The averages of the orders selected and the frequencies
of selecting the true order based on 100 independent replications depending on
sample’s length in the scenario ${\bf S}^{*}_{1}$. Figure 3: The averages of
the orders selected and the frequencies of selecting the true order based on
100 independent replications depending on sample’s length in the scenario
${\bf S}^{*}_{2}$.

##  6 Summary and conclusion

This paper considers a general class of causal processes with exogenous
covariates in a semiparametric framework. This class is quite extensive and
many classical processes such as ARMA-GARCH, ARMAX-GARCH-X, APARCH-X,$\cdots$
are particular cases. Sufficient conditions for the existence of a stationary
and ergodic solution are provided.
A quasi likelihood estimator is performed for inference; the consistency of
this estimator is established and the asymptotic distribution is derived. This
distribution coincides with the Gaussian one, when the true parameter is an
interior point of the parameter’s space.
A Wald-type statistic is proposed for testing the significance test of
parameter. The asymptotic studies show that, this test has correct size
asymptotically and is consistent in power. In certain cases, this test can be
used in particular to test the relevance of the exogenous covariates.
The model selection question for the class
$\mathcal{AC}$-$X(M_{\theta^{*}},f_{\theta^{*}})$ is carried out by a
penalized quasi likelihood contrast. The weak and the strong consistency of
the proposed procedure is established. These results provide sufficient
conditions for the consistency of the BIC and the HQC procedures. Simulation
study shows that, the empirical and the theoretical results are overall in
accordance.
An extension of this works is to address the inference, the significance test
of parameter, the model selection problem for the class
$\mathcal{AC}$-$X(M_{\theta^{*}},f_{\theta^{*}})$ with a non Gaussian quasi
likelihood. For instance, as pointed out by Kengne (2021), the use of the
Laplacian quasi likelihood will allow to reduce the order of moments imposed
to the process. Another topic of a research project, is the change-point
detection in this class of models.

##  7 Proofs of the main results

To simplify the expressions, in the proofs of Theorems 2.2, 2.3 and 3.1, we
will use the conditional Gaussian quasi-log-likelihood given by
$L_{n}(\theta)=-\sum\limits_{t=1}^{n}q_{t}(\theta)$ and
$\widehat{L}_{n}(\theta)=-\sum\limits_{t=1}^{n}\widehat{q}_{t}(\theta)$.
Throughout the sequel, $C$ denotes a positive constant whom value may differ
from an inequality to another.

###  7.1 Proof of Proposition 2.1

We verify that the process $Z_{t}:=(Y_{t};X_{t})$ satisfies the conditions
required for the Theorem 3.1 in Doukhan and Wintenberger [6]. According to
(1.1), for all $t\in\mathbb{Z}$,

$\displaystyle Z_{t}$
$\displaystyle=\big{(}M_{\theta^{*}}(Y_{t-1},\ldots;X_{t-1},\ldots)\xi_{t}+f_{\theta^{*}}(Y_{t-1},\ldots;X_{t-1},\ldots);\,g(X_{t-1},\ldots;\eta_{t})\big{)}$
$\displaystyle=F(Z_{t-1},Z_{t-2},\ldots;U_{t}),$

with $U_{t}=(\xi_{t},\eta_{t})$ and
$F(\boldsymbol{z};U_{t})=\big{(}M_{\theta^{*}}(y_{1},\ldots;x_{1},\ldots)\xi_{t}+f_{\theta^{*}}(y_{1},\ldots;x_{1},\ldots);\,g(x_{1},\ldots;\eta_{t})\big{)}$
for all
$\boldsymbol{z}=\big{(}(y_{k},x_{k})\big{)}_{k\in\mathbb{N}}\in(\mathbb{R}^{d_{x}+1})^{\mathbb{N}}$.
Thus, the equation (1.1) of [6] holds for $(Z_{t})_{t\in\mathbb{Z}}$. For a
vector $z=(y,x)\in\mathbb{R}^{d_{x}+1}$, define the norm
$\|z\|_{w}=|y|+w_{x}\|x\|$ for some $w_{x}>0$. According to Doukhan and
Wintenberger (2008), it suffices to show that:

  * (i)

$\mathbb{E}\|F(\boldsymbol{z};U_{0})\|^{r}_{w}<\infty$ for some
$\boldsymbol{z}\in(\mathbb{R}^{d_{x}+1})^{\mathbb{N}}$;

  * (ii)

there exists a non-negative sequence $(\alpha_{k}(F))_{k\geq 1}$ satisfying
$\sum_{k\geq 1}\alpha_{k}(F)<1$ such that, for all
$\boldsymbol{z},\tilde{\boldsymbol{z}}\in(\mathbb{R}^{d_{x}+1})^{\mathbb{N}}$,

$\mathbb{E}\|F(\boldsymbol{z};U_{0})-F(\tilde{\boldsymbol{z}};U_{0})\|^{r}_{w}\leq\sum_{k\geq
1}\alpha_{k}(F)\|z_{k}-\tilde{z}_{k}\|_{w}.$

Using the condition (2.2), the part (i) is directly obtained from the
assumptions A${}_{0}(f_{\theta},\Theta)$, A${}_{0}(M_{\theta},\Theta)$.
To prove (ii), let $\boldsymbol{z}=(z_{1},\ldots),\
\tilde{\boldsymbol{z}}=(\tilde{z}_{1},\ldots)\in(\mathbb{R}^{d_{x}+1})^{\mathbb{N}}$
such that $z_{k}=(y_{k},x_{k})$ and
$\tilde{z}_{k}=(\tilde{y}_{k},\tilde{x}_{k})$ for all $k\geq 1$. From
A${}_{0}(f_{\theta},\Theta)$, A${}_{0}(M_{\theta},\Theta)$ and (2.2), we get

$\displaystyle\left\|\left\|F(\boldsymbol{z};U_{0})-F(\tilde{\boldsymbol{z}};U_{0})\right\|_{w}\right\|_{r}$
$\displaystyle\hskip
5.69046pt\leq\big{\|}\left\|\left(M_{\theta}(y_{1},\ldots;x_{1},\ldots)-M_{\theta}(\tilde{y}_{1},\ldots;\tilde{x}_{1},\ldots)\right)\xi_{0}\right\|_{\Theta}+\left\|f_{\theta}(y_{1},\ldots;x_{1},\ldots)-f_{\theta}(\tilde{y}_{1},\ldots;\tilde{x}_{1},\ldots)\right\|_{\Theta}\big{\|}_{r}$
$\displaystyle\hskip
42.67912pt+w_{x}\left\|g(x_{1},\ldots;\eta_{0})-g(\tilde{x}_{1},\ldots;\eta_{0})\right\|_{r}$
$\displaystyle\hskip
5.69046pt\leq\sum\limits_{k=1}^{\infty}\left(\alpha^{(0)}_{k,Y}(f_{\theta},\Theta)+\|\xi_{0}\|_{r}\alpha^{(0)}_{k,Y}(M_{\theta},\Theta)\right)|y_{k}-\tilde{y}_{k}|+\sum\limits_{k=1}^{\infty}\left(\alpha^{(0)}_{k,X}(f_{\theta},\Theta)+\|\xi_{0}\|_{r}\alpha^{(0)}_{k,X}(M_{\theta},\Theta)\right)\|x_{k}-\tilde{x}_{k}\|$
$\displaystyle\hskip
42.67912pt+w_{x}\sum\limits_{k=1}^{\infty}\alpha_{k}(g)\left\|x_{k}-\tilde{x}_{k}\right\|$
$\displaystyle\hskip
5.69046pt\leq\sum\limits_{k=1}^{\infty}\left(\alpha^{(0)}_{k,Y}(f_{\theta},\Theta)+\|\xi_{0}\|_{r}\alpha^{(0)}_{k,Y}(M_{\theta},\Theta)\right)|y_{k}-\tilde{y}_{k}|$
$\displaystyle\hskip
42.67912pt+w_{x}\sum\limits_{k=1}^{\infty}\Big{(}\frac{1}{w_{x}}\big{(}\alpha^{(0)}_{k,X}(f_{\theta},\Theta)+\|\xi_{0}\|_{r}\alpha^{(0)}_{k,X}(M_{\theta},\Theta)\big{)}+\alpha_{k}(g)\Big{)}\|x_{k}-\tilde{x}_{k}\|$
$\displaystyle\hskip
5.69046pt\leq\sum\limits_{k=1}^{\infty}\alpha_{k}(F)\|z_{k}-\tilde{z}_{k}\|_{w}$

with
$\alpha_{k}(F)=\max\left\\{\alpha^{(0)}_{k,Y}(f_{\theta},\Theta)+\|\xi_{0}\|_{r}\alpha^{(0)}_{k,Y}(M_{\theta},\Theta),\
\frac{1}{w_{x}}\big{(}\alpha^{(0)}_{k,X}(f_{\theta},\Theta)+\|\xi_{0}\|_{r}\alpha^{(0)}_{k,X}(M_{\theta},\Theta)\big{)}+\alpha_{k}(g)\right\\}.$
Thus, to get $\sum\limits_{k=1}^{\infty}\alpha_{k}(F)<1$, it suffices to
choose $w_{x}$ sufficiently large, such that

$w_{x}>\frac{\sum_{k\geq
1}\left\\{\alpha^{(0)}_{k,X}(f_{\theta},\Theta)+\|\xi_{0}\|_{r}\alpha^{(0)}_{k,X}(M_{\theta},\Theta)\right\\}}{1-\sum_{k\geq
1}\max\left\\{\alpha_{k}(g),\,\alpha^{(0)}_{k,Y}(f_{\theta},\Theta)+\|\xi_{0}\|_{r}\alpha^{(0)}_{k,Y}(M_{\theta},\Theta)\right\\}}.$

This completes the proof of the proposition.

$\blacksquare$

###  7.2 Proof of Theorem 2.2

We consider the following lemma.

###### Lemma 7.1

Assume that the assumptions of Theorem 2.2 hold. Then

$\frac{1}{n}\big{\|}\widehat{L}_{n}(\theta)-L_{n}(\theta)\big{\|}_{\Theta}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle n\rightarrow\infty}\end{array}0.$

_Proof of Lemma 7.1_
Remark that

$\displaystyle\frac{1}{n}\|\widehat{L}_{n}(\theta)-L_{n}(\theta)\|_{\Theta}$
$\displaystyle\leq\frac{1}{n}\sum_{t=1}^{n}\|\widehat{q}_{t}(\theta)-q_{t}(\theta)\|_{\Theta}.$

Hence, by Corollary 1 of Kounias and Weng (1969), with
$2\leq\tilde{r}\leq\min\\{3,r\\}$ (without loss of generality), it suffices to
show that

$\sum_{\ell\geq
1}\frac{1}{\ell^{\tilde{r}/3}}\mathbb{E}\left(\|\widehat{q}_{\ell}(\theta)-q_{\ell}(\theta)\|^{\tilde{r}/3}_{\Theta}\right)<\infty.$
(7.1)

For any $\theta\in\Theta$, by applying the mean value theorem at the functions
$x\mapsto\frac{1}{x^{2}}$ and $x\mapsto\log x$, we have

$\displaystyle|\widehat{q}_{t}(\theta)-q_{t}(\theta)|$
$\displaystyle\leq\Big{|}\frac{(Y_{t}-\widehat{f}^{t}_{\theta})^{2}}{\widehat{H}^{t}_{\theta}}-\frac{(Y_{t}-f^{t}_{\theta})^{2}}{H^{t}_{\theta}}\Big{|}+|\log\widehat{H}^{t}_{\theta}-\log
H^{t}_{\theta}|$
$\displaystyle\leq\Big{|}\frac{(Y_{t}-\widehat{f}^{t}_{\theta})^{2}}{(\widehat{M}^{t}_{\theta})^{2}}-\frac{(Y_{t}-f^{t}_{\theta})^{2}}{(M^{t}_{\theta})^{2}}\Big{|}+2\big{|}\log|\widehat{M}^{t}_{\theta}|-\log|M^{t}_{\theta}|\big{|}$
$\displaystyle\leq\Big{|}(Y_{t}-\widehat{f}^{t}_{\theta})^{2}\Big{(}\frac{1}{(\widehat{M}^{t}_{\theta})^{2}}-\frac{1}{(M^{t}_{\theta})^{2}}\Big{)}+\frac{1}{(M^{t}_{\theta})^{2}}\left((Y_{t}-\widehat{f}^{t}_{\theta})^{2}-(Y_{t}-f^{t}_{\theta})^{2}\right)\Big{|}+\frac{2}{\underline{h}^{1/2}}|\widehat{M}^{t}_{\theta}-M^{t}_{\theta}|$
$\displaystyle\leq\frac{2}{\underline{h}^{3/2}}|Y_{t}-\widehat{f}^{t}_{\theta}|^{2}|\widehat{M}^{t}_{\theta}-M^{t}_{\theta}|+\frac{1}{\underline{h}}|\widehat{f}^{t}_{\theta}-f^{t}_{\theta}||\widehat{f}^{t}_{\theta}+f^{t}_{\theta}-2Y_{t}|+\frac{2}{\underline{h}^{1/2}}|\widehat{M}^{t}_{\theta}-M^{t}_{\theta}|$
$\displaystyle\leq
C\big{(}(|Y_{t}-f^{t}_{\theta}|^{2}+1)|\widehat{M}^{t}_{\theta}-M^{t}_{\theta}|+|\widehat{f}^{t}_{\theta}-f^{t}_{\theta}||\widehat{f}^{t}_{\theta}+f^{t}_{\theta}-2Y_{t}|\big{)}.$

This implies

$\mathbb{E}\big{[}\|\widehat{q}_{t}(\theta)-q_{t}(\theta)\|^{\tilde{r}/3}_{\Theta}\big{]}\leq
C\Big{(}\mathbb{E}\Big{[}\big{(}\|Y_{t}-f^{t}_{\theta}\|^{2}_{\Theta}+1\big{)}^{\tilde{r}/3}\|\widehat{M}^{t}_{\theta}-M^{t}_{\theta}\|^{\tilde{r}/3}_{\Theta}\Big{]}\\\
+\mathbb{E}\Big{[}\|\widehat{f}^{t}_{\theta}-f^{t}_{\theta}\|^{\tilde{r}/3}_{\Theta}\big{(}\|\widehat{f}^{t}_{\theta}\|_{\Theta}+\|f^{t}_{\theta}\|_{\Theta}+2|Y_{t}|\big{)}^{\tilde{r}/3}\Big{]}\Big{)}.$

Moreover, since $\theta^{*}\in\Theta(r)$ for some $r\geq 2$, by Assumption
A${}_{0}(\Psi_{\theta},\Theta)$, one can easily show that:

$\displaystyle\bullet~{}~{}\mathbb{E}\left[|Y_{t}|^{r}+\|f^{t}_{\theta}\|^{r}_{\Theta}+\|\widehat{f}^{t}_{\theta}\|^{r}_{\Theta}+\|M^{t}_{\theta}\|^{r}_{\Theta}+\|\widehat{M}^{t}_{\theta}\|^{r}_{\Theta}+\|H^{t}_{\theta}\|^{r/2}_{\Theta}+\|\widehat{H}^{t}_{\theta}\|^{r/2}_{\Theta}\right]<\infty;$
(7.2)
$\displaystyle\bullet~{}~{}\left\\{\begin{array}[]{l}\mathbb{E}(\|\widehat{f}^{t}_{\theta}-f^{t}_{\theta}\|^{r}_{\Theta})\leq
C\Big{(}\sum\limits_{k\geq
t}\big{\\{}\alpha^{(0)}_{k,Y}(f_{\theta},\Theta)+\alpha^{(0)}_{k,X}(f_{\theta},\Theta)\big{\\}}\Big{)}^{r};\\\
\rule[0.0pt]{0.0pt}{17.07182pt}\mathbb{E}(\|\widehat{M}^{t}_{\theta}-M^{t}_{\theta}\|^{r}_{\Theta})\leq
C\Big{(}\sum\limits_{k\geq
t}\big{\\{}\alpha^{(0)}_{k,Y}(M_{\theta},\Theta)+\alpha^{(0)}_{k,X}(M_{\theta},\Theta)\big{\\}}\Big{)}^{r}.\end{array}\right.$
(7.5)

Then, by the Hölder’s inequality, we have

$\displaystyle\mathbb{E}\Big{[}\|\widehat{f}^{\ell}_{\theta}-f^{\ell}_{\theta}\|^{\tilde{r}/3}_{\Theta}\big{(}\|\widehat{f}^{\ell}_{\theta}\|_{\Theta}+\|f^{\ell}_{\theta}\|_{\Theta}+2|Y_{\ell}|\big{)}^{\tilde{r}/3}\Big{]}$
$\displaystyle\hskip
113.81102pt\leq\Big{(}\mathbb{E}\big{[}\|\widehat{f}^{\ell}_{\theta}-f^{\ell}_{\theta}\|^{\tilde{r}}_{\Theta}\big{]}\Big{)}^{1/3}\Big{(}\mathbb{E}\big{[}\|\widehat{f}^{\ell}_{\theta}\|_{\Theta}+\|f^{\ell}_{\theta}\|_{\Theta}+2|Y_{\ell}|\big{]}^{\tilde{r}/2}\Big{)}^{2/3}$
$\displaystyle\hskip 113.81102pt\leq C\Big{(}\sum\limits_{k\geq
t}\big{\\{}\alpha^{(0)}_{k,Y}(f_{\theta},\Theta)+\alpha^{(0)}_{k,X}(f_{\theta},\Theta)\big{\\}}\Big{)}^{\tilde{r}/3}.$

Again, by the Hölder’s inequality, from (7.2) and (7.5), we obtain

$\displaystyle\mathbb{E}\Big{[}\big{(}\|Y_{t}-f^{\ell}_{\theta}\|^{2}_{\Theta}+1\big{)}^{\tilde{r}/3}\|\widehat{M}^{\ell}_{\theta}-M^{\ell}_{\theta}\|^{\tilde{r}/3}_{\Theta}\Big{]}$
$\displaystyle\leq\mathbb{E}\Big{[}\|Y_{t}+f^{\ell}_{\theta}+1\|^{2\tilde{r}/3}_{\Theta}\|\widehat{M}^{\ell}_{\theta}-M^{\ell}_{\theta}\|^{\tilde{r}/3}_{\Theta}\Big{]}$
$\displaystyle\leq\big{(}\mathbb{E}\big{[}\|Y_{t}+f^{\ell}_{\theta}+1\|^{\tilde{r}}_{\Theta}\big{]}\big{)}^{2/3}\big{(}\mathbb{E}\big{[}\|\widehat{M}^{\ell}_{\theta}-M^{\ell}_{\theta}\|^{\tilde{r}}_{\Theta}\big{]}\big{)}^{1/3}$
$\displaystyle\leq C\Big{(}\sum\limits_{k\geq
t}\big{\\{}\alpha^{(0)}_{k,Y}(M_{\theta},\Theta)+\alpha^{(0)}_{k,X}(M_{\theta},\Theta)\big{\\}}\Big{)}^{\tilde{r}/3}.$

Hence, from (2.3), we deduce

$\displaystyle\sum_{\ell\geq
1}\frac{1}{\ell^{\tilde{r}/3}}\mathbb{E}\left(\|\widehat{q}_{\ell}(\theta)-q_{\ell}(\theta)\|^{\tilde{r}/3}_{\Theta}\right)$
$\displaystyle\hskip 56.9055pt\leq C\sum_{\ell\geq
1}\frac{1}{\ell^{\tilde{r}/3}}\Big{(}\sum\limits_{k\geq\ell}\big{\\{}\alpha^{(0)}_{k,Y}(f_{\theta},\Theta)+\alpha^{(0)}_{k,X}(f_{\theta},\Theta)+\alpha^{(0)}_{k,Y}(M_{\theta},\Theta)+\alpha^{(0)}_{k,X}(M_{\theta},\Theta)\big{\\}}\Big{)}^{\tilde{r}/3}$
$\displaystyle\hskip 56.9055pt\leq C\sum_{\ell\geq
1}\frac{1}{\ell^{\tilde{r}/3}}\Big{(}\dfrac{1}{\ell^{\gamma-1}}\Big{)}^{\tilde{r}/3}\leq
C\sum_{\ell\geq 1}\frac{1}{\ell^{\tilde{r}\gamma/3}}<\infty,$

where the last inequality holds since $\gamma>3/2$. Thus, the condition (7.1)
is satisfied. This completes the proof Lemma 7.1.

$~{}~{}~{}~{}~{}~{}\blacksquare$

To complete the proof of Theorem 2.2, we will show that: (1.)
$\mathbb{E}\left[\left\|q_{t}(\theta)\right\|_{\Theta}\right]<\infty$ and (2.)
the function $\theta\mapsto-\mathbb{E}[q_{0}(\theta)]$ has a unique maximum at
$\theta^{*}$.

  1. (1.)

For all $\theta\in\Theta$, using the inequality $|\log(x)|\leq|x-1|$ for all
$x>1$, we have

$\displaystyle|q_{t}(\theta)|$
$\displaystyle\leq\frac{1}{H^{t}_{\theta}}|Y_{t}-f^{t}_{\theta})|^{2}+\big{|}\log\big{(}\frac{H^{t}_{\theta}}{\underline{h}}\big{)}+\log(\underline{h})\big{|}$
$\displaystyle\leq\frac{1}{\underline{h}}(Y^{2}_{t}+(f^{t}_{\theta})^{2}+2Y_{t}f^{t}_{\theta})+\big{|}\frac{H^{t}_{\theta}}{\underline{h}}-1\big{|}+|\log(\underline{h})|$
$\displaystyle\leq
C\left(Y^{2}_{t}+(f^{t}_{\theta})^{2}+2Y_{t}f^{t}_{\theta}+|M^{t}_{\theta}|^{2}\right)+C.$

Hence, from (7.2), we deduce

$\mathbb{E}\left[\left\|q_{t}\right\|_{\Theta}\right]\leq
C\Big{(}\mathbb{E}[Y^{2}_{t}]+\mathbb{E}\|f^{t}_{\theta}\|^{2}_{\Theta}+2\big{(}(\mathbb{E}[Y^{2}_{t}])^{1/2}(\mathbb{E}[\|f^{t}_{\theta}\|^{2}_{\Theta}])^{1/2}\big{)}+\mathbb{E}[\|M^{t}_{\theta}\|^{2}_{\Theta}]\Big{)}+C<\infty,$

which shows that (1.) holds.

  2. (2.)

Let $\theta\in\Theta$ with $\theta\neq\theta^{*}$. We have

$\mathbb{E}[q_{0}(\theta)]-\mathbb{E}[q_{0}(\theta^{*})]=\mathbb{E}\big{[}\mathbb{E}\left[\left(q_{0}(\theta)-q_{0}(\theta^{*})\right)|\mathcal{F}_{-1}\right]\big{]}.$
(7.6)

Moreover,

$\displaystyle\mathbb{E}\left[\left(q_{0}(\theta)-q_{0}(\theta^{*})\right)|\mathcal{F}_{-1}\right]$
$\displaystyle=\mathbb{E}\big{[}\frac{(Y_{0}-f^{0}_{\theta})^{2}}{H^{0}_{\theta}}+\log
H^{0}_{\theta}-\frac{(Y_{0}-f^{0}_{\theta^{*}})^{2}}{H^{0}_{\theta^{*}}}-\log
H^{0}_{\theta^{*}}\big{|}\mathcal{F}_{-1}\big{]}$
$\displaystyle=-\log\big{(}\frac{H^{0}_{\theta^{*}}}{H^{0}_{\theta}}\big{)}+\frac{\mathbb{E}\left[(Y_{0}-f^{0}_{\theta})^{2}\big{|}\mathcal{F}_{-1}\right]}{H^{0}_{\theta}}-\frac{\mathbb{E}\left[(Y_{0}-f^{0}_{\theta^{*}})^{2}\big{|}\mathcal{F}_{-1}\right]}{H^{0}_{\theta^{*}}}$
$\displaystyle=-\log\big{(}\frac{H^{0}_{\theta^{*}}}{H^{0}_{\theta}}\big{)}-1+\frac{\mathbb{E}\left[(Y_{0}-f^{0}_{\theta^{*}}+f^{0}_{\theta^{*}}-f^{0}_{\theta})^{2}\big{|}\mathcal{F}_{-1}\right]}{H^{0}_{\theta}}$
$\displaystyle=\frac{H^{0}_{\theta^{*}}}{H^{0}_{\theta}}-\log\big{(}\frac{H^{0}_{\theta^{*}}}{H^{0}_{\theta}}\big{)}-1+\frac{(f^{0}_{\theta^{*}}-f^{0}_{\theta})^{2}}{H^{0}_{\theta}}.$

Therefore, using (7.6) and by applying the Jensen’s inequality, we get

$\displaystyle\mathbb{E}[q_{0}(\theta)]-\mathbb{E}[q_{0}(\theta^{*})]$
$\displaystyle=\mathbb{E}\big{[}\frac{H^{0}_{\theta^{*}}}{H^{0}_{\theta}}-\log\big{(}\frac{H^{0}_{\theta^{*}}}{H^{0}_{\theta}}\big{)}-1+\frac{(f^{0}_{\theta^{*}}-f^{0}_{\theta})^{2}}{H^{0}_{\theta}}\big{]}$
$\displaystyle\geq\mathbb{E}\big{[}\frac{H^{0}_{\theta^{*}}}{H^{0}_{\theta}}\big{]}-\log\big{(}\mathbb{E}\big{[}\frac{H^{0}_{\theta^{*}}}{H^{0}_{\theta}}\big{]}\big{)}-1+\mathbb{E}\big{[}\frac{(f^{0}_{\theta^{*}}-f^{0}_{\theta})^{2}}{H^{0}_{\theta}}\big{]}.$

Since $x-\log(x)-1>0$ for any $x>0,\ x\neq 1$; and $x-\log(x)-1=0$ for $x=1$,
we deduce:

     * •

if $f^{0}_{\theta^{*}}\neq f^{0}_{\theta}~{}a.s.$, then
$\mathbb{E}\big{[}\frac{(f^{0}_{\theta^{*}}-f^{0}_{\theta})^{2}}{H^{0}_{\theta}}\big{]}>0$
and $\mathbb{E}[q_{0}(\theta)]-\mathbb{E}[q_{0}(\theta^{*})]>0$,

     * •

if $f^{0}_{\theta^{*}}=f^{0}_{\theta}~{}a.s.$, then

$\mathbb{E}[q_{0}(\theta)]-\mathbb{E}[q_{0}(\theta^{*})]=\mathbb{E}\big{[}\frac{H^{0}_{\theta^{*}}}{H^{0}_{\theta}}-\log\big{(}\frac{H^{0}_{\theta^{*}}}{H^{0}_{\theta}}\big{)}-1\big{]}$

From the identifiability condition (A0), when $\theta^{*}\neq\theta$ and
$f^{0}_{\theta^{*}}=f^{0}_{\theta}~{}a.s.$, we necessarily have
$H^{0}_{\theta^{*}}\neq H^{0}_{\theta}~{}a.s.$. This implies
$\frac{H^{0}_{\theta^{*}}}{H^{0}_{\theta}}\neq 1~{}a.s.$, and thus
$\mathbb{E}[q_{0}(\theta)]-\mathbb{E}[q_{0}(\theta^{*})]>0$.

The equality $\mathbb{E}[q_{0}(\theta)]=\mathbb{E}[q_{0}(\theta^{*})]$ holds
a.s. if and only if $\theta^{*}=\theta$. This achieves the proof of (2.).

Since $\\{(Y_{t},X_{t}),\,t\in\mathbb{Z}\\}$ is stationary and ergodic, the
process $\\{q_{t}(\theta),~{}t\in\mathbb{Z}\\}$ is also a stationary and
ergodic sequence. Then, according to (1.), by the uniform strong law of large
number applied on the process $\\{q_{t}(\theta),~{}t\in\mathbb{Z}\\}$, it
holds that

$\big{\|}\frac{1}{n}L_{n}(\theta)+\mathbb{E}(q_{0}(\theta))\big{\|}_{\Theta}=\big{\|}\frac{1}{n}\sum_{t=1}^{n}q_{t}(\theta)-\mathbb{E}(q_{0}(\theta))\big{\|}_{\Theta}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle n\rightarrow\infty}\end{array}0.$

Then, by Lemma 7.1, we obtain

$\displaystyle\big{\|}\frac{1}{n}\widehat{L}_{n}(\theta)+\mathbb{E}(q_{0}(\theta))\big{\|}_{\Theta}$
$\displaystyle\leq\frac{1}{n}\big{\|}\widehat{L}_{n}(\theta)-L_{n}(\theta)\big{\|}_{\Theta}+\big{\|}\frac{1}{n}L_{n}(\theta)+\mathbb{E}(q_{0}(\theta))\big{\|}_{\Theta}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle n\rightarrow\infty}\end{array}0.$
(7.9)

The part (2.) and (7.9) lead to conclude the proof of the theorem.

$\blacksquare$

###  7.3 Proof of Theorem 2.3

The following lemma is needed.

###### Lemma 7.2

Assume that the conditions of Theorem 2.3 hold. Then

  1. (i.)

$\mathbb{E}\big{[}\frac{1}{\sqrt{n}}\big{\|}\frac{\partial\widehat{L}_{n}(\theta)}{\partial\theta}-\frac{\partial
L_{n}(\theta)}{\partial\theta}\big{\|}_{\Theta}\big{]}\begin{array}[t]{c}\stackrel{{\scriptstyle}}{{\longrightarrow}}\\\
{\scriptstyle n\rightarrow\infty}\end{array}0$;

  2. (ii.)

$\frac{1}{n}\big{\|}\frac{\partial^{2}\widehat{L}_{n}(\theta)}{\partial\theta\partial\theta^{\prime}}-\frac{\partial^{2}L_{n}(\theta)}{\partial\theta\partial\theta^{\prime}}\big{\|}_{\Theta}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle n\rightarrow\infty}\end{array}0$;

  3. (iii.)

$\big{\|}\frac{1}{n}\sum\limits_{t=1}^{n}\frac{\partial^{2}q_{t}(\theta)}{\partial\theta\partial\theta^{\prime}}-\mathbb{E}\big{(}\frac{\partial^{2}q_{0}(\theta)}{\partial\theta\partial\theta^{\prime}}\big{)}\big{\|}_{\Theta}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle n\rightarrow\infty}\end{array}0.$

_Proof of Lemma 7.2_

  1. (i.)

Remark that

$\Big{\|}\frac{\partial\widehat{L}_{n}(\theta)}{\partial\theta}-\frac{\partial
L_{n}(\theta)}{\partial\theta}\Big{\|}_{\Theta}\leq\sum_{t=1}^{n}\Big{\|}\frac{\partial\widehat{q}_{t}(\theta)}{\partial\theta}-\frac{\partial
q_{t}(\theta)}{\partial\theta}\Big{\|}_{\Theta}.$ (7.10)

Moreover, for all $\theta\in\Theta$,

$\displaystyle\frac{\partial q_{t}(\theta)}{\partial\theta}$
$\displaystyle=-(H^{t}_{\theta})^{-2}\big{(}2H^{t}_{\theta}(Y_{t}-f^{t}_{\theta})\frac{\partial
f^{t}_{\theta}}{\partial\theta}+(Y_{t}-f^{t}_{\theta})^{2}\frac{\partial
H^{t}_{\theta}}{\partial\theta}\big{)}+(H^{t}_{\theta})^{-1}\frac{H^{t}_{\theta}}{\partial\theta}$
$\displaystyle=-2(H^{t}_{\theta})^{-1}(Y_{t}-f^{t}_{\theta})\frac{\partial
f^{t}_{\theta}}{\partial\theta}+(Y_{t}-f^{t}_{\theta})^{2}\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta}+(H^{t}_{\theta})^{-1}\frac{\partial
H^{t}_{\theta}}{\partial\theta},$ (7.11)

which implies

$\displaystyle\Big{|}\frac{\partial\widehat{q}_{t}(\theta)}{\partial\theta}-\frac{\partial
q_{t}(\theta)}{\partial\theta}\Big{|}$ $\displaystyle\leq
2\Big{|}(\widehat{H}^{t}_{\theta})^{-1}(Y_{t}-\widehat{f}^{t}_{\theta})\frac{\partial\widehat{f}^{t}_{\theta}}{\partial\theta}-(H^{t}_{\theta})^{-1}(Y_{t}-f^{t}_{\theta})\frac{\partial
f^{t}_{\theta}}{\partial\theta}\Big{|}$ $\displaystyle\hskip
14.22636pt+\Big{|}(Y_{t}-\widehat{f}^{t}_{\theta})^{2}\frac{\partial(\widehat{H}^{t}_{\theta})^{-1}}{\partial\theta}-(Y_{t}-f^{t}_{\theta})^{2}\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta}\Big{|}+\Big{|}(\widehat{H}^{t}_{\theta})^{-1}\frac{\partial\widehat{H}^{t}_{\theta}}{\partial\theta}-(H^{t}_{\theta})^{-1}\frac{\partial
H^{t}_{\theta}}{\partial\theta}\Big{|}.$

Using the relation
$|a_{1}b_{1}c_{1}-a_{2}b_{2}c_{2}|\leq|a_{1}-a_{2}||b_{2}||c_{2}|+|a_{1}||b_{1}-b_{2}||c_{2}|+|a_{1}||b_{1}||c_{1}-c_{2}|,~{}\forall
a_{1},a_{2},b_{1},b_{2},c_{1},c_{2},\in\mathbb{R}$, we get

$\displaystyle\Big{\|}\frac{\partial\widehat{q}_{t}(\theta)}{\partial\theta}-\frac{\partial
q_{t}(\theta)}{\partial\theta}\Big{\|}_{\Theta}$ $\displaystyle\hskip
14.22636pt\leq
2\Big{(}\big{\|}(\widehat{H}^{t}_{\theta})^{-1}-(H^{t}_{\theta})^{-1}\big{\|}_{\Theta}\|Y_{t}-\widehat{f}^{t}_{\theta}\|_{\Theta}\Big{\|}\frac{\partial
f^{t}_{\theta}}{\partial\theta}\Big{\|}_{\Theta}+\big{\|}(\widehat{H}^{t}_{\theta})^{-1}\big{\|}_{\Theta}\|\widehat{f}^{t}_{\theta}-f^{t}_{\theta}\|_{\Theta}\Big{\|}\frac{\partial\widehat{f}^{t}_{\theta}}{\partial\theta}\Big{\|}_{\Theta}$
$\displaystyle\hskip
28.45274pt+\big{\|}(\widehat{H}^{t}_{\theta})^{-1}\big{\|}_{\Theta}\|Y_{t}-f^{t}_{\theta}\|_{\Theta}\Big{\|}\frac{\partial\widehat{f}^{t}_{\theta}}{\partial\theta}-\frac{\partial
f^{t}_{\theta}}{\partial\theta}\Big{\|}_{\Theta}\Big{)}+\|(Y_{t}-\widehat{f}^{t}_{\theta})\|^{2}_{\Theta}\Big{\|}\frac{\partial(\widehat{H}^{t}_{\theta})^{-1}}{\partial\theta}-\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta}\Big{\|}_{\Theta}$
$\displaystyle\hskip
28.45274pt+2|Y_{t}|\|\widehat{f}^{t}_{\theta}-f^{t}_{\theta}\|_{\Theta}\Big{\|}\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta}\Big{\|}_{\Theta}+\big{\|}(\widehat{H}^{t}_{\theta})^{-1}\big{\|}_{\Theta}\Big{\|}\frac{\partial\widehat{H}^{t}_{\theta}}{\partial\theta}-\frac{\partial
H^{t}_{\theta}}{\partial\theta}\Big{\|}_{\Theta}+\big{\|}(\widehat{H}^{t}_{\theta})^{-1}-(H^{t}_{\theta})^{-1}\big{\|}_{\Theta}\Big{\|}\frac{\partial
H^{t}_{\theta}}{\partial\theta}\Big{\|}_{\Theta}$ $\displaystyle\hskip
14.22636pt\leq
2(\underline{h})^{-1}\Big{(}\|\widehat{f}^{t}_{\theta}-f^{t}_{\theta}\|_{\Theta}\Big{\|}\frac{\partial\widehat{f}^{t}_{\theta}}{\partial\theta}\Big{\|}_{\Theta}+\|Y_{t}-f^{t}_{\theta}\|_{\Theta}\Big{\|}\frac{\partial\widehat{f}^{t}_{\theta}}{\partial\theta}-\frac{\partial
f^{t}_{\theta}}{\partial\theta}\Big{\|}_{\Theta}+\frac{1}{2}\Big{\|}\frac{\partial\widehat{H}^{t}_{\theta}}{\partial\theta}-\frac{\partial
H^{t}_{\theta}}{\partial\theta}\Big{\|}_{\Theta}\Big{)}$ $\displaystyle\hskip
28.45274pt+2\big{\|}(\widehat{H}^{t}_{\theta})^{-1}-(H^{t}_{\theta})^{-1}\big{\|}_{\Theta}\|Y_{t}-\widehat{f}^{t}_{\theta}\|_{\Theta}\Big{\|}\frac{\partial
f^{t}_{\theta}}{\partial\theta}\Big{\|}_{\Theta}+\|(Y_{t}-\widehat{f}^{t}_{\theta})\|^{2}_{\Theta}\Big{\|}\frac{\partial(\widehat{H}^{t}_{\theta})^{-1}}{\partial\theta}-\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta}\Big{\|}_{\Theta}$
$\displaystyle\hskip
28.45274pt+2|Y_{t}|\|\widehat{f}^{t}_{\theta}-f^{t}_{\theta}\|_{\Theta}\Big{\|}\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta}\Big{\|}_{\Theta}+\big{\|}(\widehat{H}^{t}_{\theta})^{-1}-(H^{t}_{\theta})^{-1}\big{\|}_{\Theta}\Big{\|}\frac{\partial
H^{t}_{\theta}}{\partial\theta}\Big{\|}_{\Theta}.$ (7.12)

By applying the Hölder’s inequality to the terms of the right hand side of
((i.)), we have

$\displaystyle\mathbb{E}\Big{[}\Big{\|}\frac{\partial\widehat{q}_{t}(\theta)}{\partial\theta}-\frac{\partial
q_{t}(\theta)}{\partial\theta}\Big{\|}_{\Theta}\Big{]}$ $\displaystyle\hskip
14.22636pt\leq
C\Bigg{[}\left(\mathbb{E}[\|\widehat{f}^{t}_{\theta}-f^{t}_{\theta}\|^{4}_{\Theta}]\right)^{1/4}\Big{(}\mathbb{E}\Big{[}\Big{\|}\frac{\partial\widehat{f}^{t}_{\theta}}{\partial\theta}\Big{\|}^{4/3}_{\Theta}\Big{]}\Big{)}^{3/4}+\big{(}\mathbb{E}[\|Y_{t}-f^{t}_{\theta}\|^{4/3}_{\Theta}]\big{)}^{3/4}\Big{(}\mathbb{E}\Big{[}\Big{\|}\frac{\partial\widehat{f}^{t}_{\theta}}{\partial\theta}-\frac{\partial
f^{t}_{\theta}}{\partial\theta}\Big{\|}^{4}_{\Theta}\Big{]}\Big{)}^{1/4}$
$\displaystyle\hskip
28.45274pt+\Big{(}\mathbb{E}\Big{[}\Big{\|}\frac{\partial\widehat{H}^{t}_{\theta}}{\partial\theta}-\frac{\partial
H^{t}_{\theta}}{\partial\theta}\Big{\|}^{2}_{\Theta}\Big{]}\Big{)}^{1/2}+\Big{(}\mathbb{E}\Big{[}\big{\|}(\widehat{H}^{t}_{\theta})^{-1}-(H^{t}_{\theta})^{-1}\big{\|}^{4}_{\Theta}\Big{]}\Big{)}^{1/4}\left(\mathbb{E}[\|Y_{t}-\widehat{f}^{t}_{\theta}\|^{4}_{\Theta}]\right)^{1/4}\Big{(}\mathbb{E}\Big{[}\Big{\|}\frac{\partial
f^{t}_{\theta}}{\partial\theta}\Big{\|}^{2}_{\Theta}\Big{]}\Big{)}^{1/2}$
$\displaystyle\hskip
42.67912pt+\left(\mathbb{E}[\|(Y_{t}-\widehat{f}^{t}_{\theta})\|^{4}_{\Theta}]\right)^{1/2}\Big{(}\mathbb{E}\Big{[}\Big{\|}\frac{\partial(\widehat{H}^{t}_{\theta})^{-1}}{\partial\theta}-\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta}\Big{\|}^{2}_{\Theta}\Big{]}\Big{)}^{1/2}+\left(\mathbb{E}[|Y_{t}|^{4}]\right)^{1/4}\left(\mathbb{E}[\|\widehat{f}^{t}_{\theta}-f^{t}_{\theta}\|^{4}_{\Theta}]\right)^{1/4}$
$\displaystyle\hskip
56.9055pt\times\Big{(}\mathbb{E}\Big{[}\Big{\|}\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta}\Big{\|}^{2}_{\Theta}\Big{]}\Big{)}^{1/2}+\left(\mathbb{E}\big{[}\big{\|}(\widehat{H}^{t}_{\theta})^{-1}-(H^{t}_{\theta})^{-1}\big{\|}^{4}_{\Theta}\big{]}\right)^{1/4}\Big{(}\mathbb{E}\Big{[}\Big{\|}\frac{\partial
H^{t}_{\theta}}{\partial\theta}\Big{\|}^{4/3}_{\Theta}\Big{]}\Big{)}^{3/4}\Bigg{]}.$

Moreover, since $\theta^{*}\in\Theta(r)$, using A${}_{i}(f_{\theta},\Theta)$
and A${}_{i}(M_{\theta},\Theta)$ (with $i=0,1$), one can go along similar
lines as in Bardet and Wintenberger (2009) to establish the following results:

$\displaystyle\bullet~{}\mathbb{E}\Big{[}\Big{\|}\frac{\partial
f^{t}_{\theta}}{\partial\theta}\Big{\|}^{r}_{\Theta}+\Big{\|}\frac{\partial\widehat{f}^{t}_{\theta}}{\partial\theta}\Big{\|}^{r}_{\Theta}+\Big{\|}\frac{\partial
M^{t}_{\theta}}{\partial\theta}\Big{\|}^{r}_{\Theta}+\Big{\|}\frac{\partial\widehat{M}^{t}_{\theta}}{\partial\theta}\Big{\|}^{r}_{\Theta}+\Big{\|}\frac{\partial
H^{t}_{\theta}}{\partial\theta}\Big{\|}^{r/2}_{\Theta}+\Big{\|}\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta}\Big{\|}^{r/2}_{\Theta}\Big{]}<\infty,$
(7.13)
$\displaystyle\bullet~{}\left\\{\begin{array}[]{l}\mathbb{E}\Big{[}\Big{\|}\frac{\partial\widehat{f}^{t}_{\theta}}{\partial\theta}-\frac{\partial
f^{t}_{\theta}}{\partial\theta}\Big{\|}^{r}_{\Theta}\Big{]}\leq
C\Big{(}\sum\limits_{k\geq
t}\big{\\{}\alpha^{(1)}_{k,Y}(f_{\theta},\Theta)+\alpha^{(1)}_{k,X}(f_{\theta},\Theta)\big{\\}}\Big{)}^{r},\\\
\mathbb{E}\Big{[}\big{\|}(\widehat{H}^{t}_{\theta})^{-1}-(H^{t}_{\theta})^{-1}\big{\|}^{r}_{\Theta}\Big{]}\leq
C\Big{(}\sum\limits_{k\geq
t}\big{\\{}\alpha^{(0)}_{k,Y}(M_{\theta},\Theta)+\alpha^{(0)}_{k,X}(M_{\theta},\Theta)\big{\\}}\Big{)}^{r},\\\
\mathbb{E}\Big{[}\Big{\|}\frac{\partial\widehat{H}^{t}_{\theta}}{\partial\theta}-\frac{\partial
H^{t}_{\theta}}{\partial\theta}\Big{\|}^{r/2}_{\Theta}\Big{]}\leq
C\Big{(}\sum\limits_{k\geq
t}\big{\\{}\alpha^{(0)}_{k,Y}(M_{\theta},\Theta)+\alpha^{(1)}_{k,X}(M_{\theta},\Theta)\big{\\}}\Big{)}^{r/2},\\\
\mathbb{E}\Big{[}\Big{\|}\frac{\partial(\widehat{H}^{t}_{\theta})^{-1}}{\partial\theta}-\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta}\Big{\|}^{r/2}_{\Theta}\Big{]}\leq
C\Big{(}\sum\limits_{k\geq
t}\big{\\{}\alpha^{(0)}_{k,Y}(M_{\theta},\Theta)+\alpha^{(1)}_{k,X}(M_{\theta},\Theta)\big{\\}}\Big{)}^{r/2}.\end{array}\right.$
(7.18)

Thus, using (7.2), (7.5), (7.13) and (7.18) with $r=4$, we obtain

$\displaystyle\mathbb{E}\Big{[}\Big{\|}\frac{\partial\widehat{q}_{t}(\theta)}{\partial\theta}-\frac{\partial
q_{t}(\theta)}{\partial\theta}\Big{\|}_{\Theta}\Big{]}$ $\displaystyle\hskip
28.45274pt\leq
C\Bigg{[}\left(\mathbb{E}[\|\widehat{f}^{t}_{\theta}-f^{t}_{\theta}\|^{4}_{\Theta}]\right)^{1/4}+\Big{(}\mathbb{E}\Big{[}\Big{\|}\frac{\partial\widehat{f}^{t}_{\theta}}{\partial\theta}-\frac{\partial
f^{t}_{\theta}}{\partial\theta}\Big{\|}^{4}_{\Theta}\Big{]}\Big{)}^{1/4}+\Big{(}\mathbb{E}\Big{[}\Big{\|}\frac{\partial\widehat{H}^{t}_{\theta}}{\partial\theta}-\frac{\partial
H^{t}_{\theta}}{\partial\theta}\Big{\|}^{2}_{\Theta}\Big{]}\Big{)}^{1/2}$
$\displaystyle\hskip
56.9055pt+\Big{(}\mathbb{E}\Big{[}\big{\|}(\widehat{H}^{t}_{\theta})^{-1}-(H^{t}_{\theta})^{-1}\big{\|}^{4}_{\Theta}\Big{]}\Big{)}^{1/4}+\Big{(}\mathbb{E}\Big{[}\Big{\|}\frac{\partial(\widehat{H}^{t}_{\theta})^{-1}}{\partial\theta}-\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta}\Big{\|}^{2}_{\Theta}\Big{]}\Big{)}^{1/2}+\left(\mathbb{E}[\|\widehat{f}^{t}_{\theta}-f^{t}_{\theta}\|^{4}_{\Theta}]\right)^{1/4}$
$\displaystyle\hskip
71.13188pt+\left(\mathbb{E}\big{[}\big{\|}(\widehat{H}^{t}_{\theta})^{-1}-(H^{t}_{\theta})^{-1}\big{\|}^{4}_{\Theta}\big{]}\right)^{1/4}\Bigg{]}$
$\displaystyle\hskip 28.45274pt\leq C\sum\limits_{k\geq
t}\Big{\\{}\alpha^{(0)}_{k,Y}(f_{\theta},\Theta)+\alpha^{(0)}_{k,X}(f_{\theta},\Theta)+\alpha^{(1)}_{k,Y}(f_{\theta},\Theta)+\alpha^{(1)}_{k,X}(f_{\theta},\Theta)$
$\displaystyle\hskip
85.35826pt+\alpha^{(0)}_{k,Y}(M_{\theta},\Theta)+\alpha^{(0)}_{k,X}(M_{\theta},\Theta)+\alpha^{(1)}_{k,Y}(M_{\theta},\Theta)+\alpha^{(1)}_{k,X}(M_{\theta},\Theta)\Big{\\}}.$

Therefore, in view of the condition (2.7), it holds that

$\displaystyle\mathbb{E}\Big{[}\Big{\|}\frac{\partial\widehat{q}_{t}(\theta)}{\partial\theta}-\frac{\partial
q_{t}(\theta)}{\partial\theta}\Big{\|}_{\Theta}\Big{]}$ $\displaystyle\leq
C\sum\limits_{k\geq t}k^{-\gamma}=C\frac{1}{t^{\gamma-1}}.$

By the inequality (7.10), we deduce

$\displaystyle\mathbb{E}\Big{[}\frac{1}{\sqrt{n}}\Big{\|}\frac{\partial\widehat{L}_{n}(\theta)}{\partial\theta}-\frac{\partial
L_{n}(\theta)}{\partial\theta}\Big{\|}_{\Theta}\Big{]}$ $\displaystyle\leq
C\frac{1}{\sqrt{n}}\sum\limits_{t=1}^{n}\frac{1}{t^{\gamma-1}}=C\frac{1}{\sqrt{n}}(1+n^{2-\gamma})\begin{array}[t]{c}\stackrel{{\scriptstyle}}{{\longrightarrow}}\\\
{\scriptstyle n\rightarrow\infty}\end{array}0.$

This proves the part (i.) of Lemma 7.2.

  2. (ii.)

This part can be established by using the same arguments as in the proof of
Lemma 7.1.

  3. (iii.)

Let us show that
$\mathbb{E}\Big{[}\Big{\|}\frac{\partial^{2}q_{t}(\theta)}{\partial\theta_{i}\partial\theta_{j}}\Big{\|}_{\Theta}\Big{]}<\infty$,
for all $i,j\in\\{1,\ldots,d\\}$.
From ((i.)), for any $i,j\in\\{1,\ldots,d\\}$, we have

$\displaystyle\frac{\partial^{2}q_{t}(\theta)}{\partial\theta\partial_{i}\theta_{j}}$
$\displaystyle=-2(H^{t}_{\theta})^{-1}(Y_{t}-f^{t}_{\theta})\frac{\partial^{2}f^{t}_{\theta}}{\partial\theta_{i}\partial\theta_{j}}+(Y_{t}-f^{t}_{\theta})^{2}\frac{\partial^{2}(H^{t}_{\theta})^{-1}}{\partial\theta_{i}\partial\theta_{j}}-2(Y_{t}-f^{t}_{\theta})\Big{(}\frac{\partial
f^{t}_{\theta}}{\partial\theta_{i}}\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta_{j}}+\frac{\partial
f^{t}_{\theta}}{\partial\theta_{j}}\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta_{i}}\Big{)}$
$\displaystyle\hskip 14.22636pt+2(H^{t}_{\theta})^{-1}\frac{\partial
f^{t}_{\theta}}{\partial\theta_{i}}\frac{\partial
f^{t}_{\theta}}{\partial\theta_{j}}+\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta_{j}}\frac{\partial
H^{t}_{\theta}}{\partial\theta_{i}}+(H^{t}_{\theta})^{-1}\frac{\partial^{2}H^{t}_{\theta}}{\partial\theta_{i}\partial\theta_{j}}.$

Therefore, according to (A1), we get

$\displaystyle\Big{\|}\frac{\partial^{2}q_{t}(\theta)}{\partial\theta\partial_{i}\theta_{j}}\Big{\|}_{\Theta}$
$\displaystyle\hskip 5.69046pt\leq
C\|(Y_{t}-f^{t}_{\theta})\|_{\Theta}\Big{(}\Big{\|}\frac{\partial^{2}f^{t}_{\theta}}{\partial\theta_{i}\partial\theta_{j}}\Big{\|}_{\Theta}+\Big{\|}\frac{\partial
f^{t}_{\theta}}{\partial\theta_{i}}\Big{\|}_{\Theta}\Big{\|}\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta_{j}}\Big{\|}_{\Theta}+\Big{\|}\frac{\partial
f^{t}_{\theta}}{\partial\theta_{j}}\Big{\|}_{\Theta}\Big{\|}\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta_{i}}\Big{\|}_{\Theta}\Big{)}$
$\displaystyle\hskip 14.22636pt+C\Big{(}\Big{\|}\frac{\partial
f^{t}_{\theta}}{\partial\theta_{i}}\Big{\|}_{\Theta}\Big{\|}\frac{\partial
f^{t}_{\theta}}{\partial\theta_{j}}\Big{\|}_{\Theta}+\Big{\|}\frac{\partial^{2}H^{t}_{\theta}}{\partial\theta_{i}\partial\theta_{j}}\Big{\|}_{\Theta}\Big{)}+{\|(Y_{t}-f^{t}_{\theta})\|}^{2}_{\Theta}\Big{\|}\frac{\partial^{2}(H^{t}_{\theta})^{-1}}{\partial\theta_{i}\partial\theta_{j}}\Big{\|}_{\Theta}+\Big{\|}\frac{\partial(H^{t}_{\theta})^{-1}}{\partial\theta_{j}}\Big{\|}_{\Theta}\Big{\|}\frac{\partial
H^{t}_{\theta}}{\partial\theta_{i}}\Big{\|}_{\Theta}.$ (7.19)

Moreover, by A${}_{2}(f_{\theta},\Theta)$ and A${}_{2}(M_{\theta},\Theta)$,
one can show that

$\mathbb{E}\Big{[}\Big{\|}\frac{\partial^{2}f^{t}_{\theta}}{\partial\theta_{i}\partial\theta_{j}}\Big{\|}^{4}_{\Theta}+\Big{\|}\frac{\partial^{2}H^{t}_{\theta}}{\partial\theta_{i}\partial\theta_{j}}\Big{\|}^{2}_{\Theta}+\Big{\|}\frac{\partial^{2}(H^{t}_{\theta})^{-1}}{\partial\theta_{i}\partial\theta_{j}}\Big{\|}^{2}_{\Theta}\Big{]}<\infty.$

Thus, by applying the Hölder’s inequality to the terms of the right hand side
of ((iii.)), it suffices to use (7.2) and (7.13) to obtain
$\mathbb{E}\Big{[}\Big{\|}\frac{\partial^{2}q_{t}(\theta)}{\partial\theta_{i}\partial\theta_{j}}\Big{\|}_{\Theta}\Big{]}<\infty$.
Since
$\mathbb{E}\Big{[}\Big{\|}\frac{\partial^{2}q_{t}(\theta)}{\partial\theta_{i}\partial\theta_{j}}\Big{\|}_{\Theta}\Big{]}<\infty$
for all $i,j\in\\{1,\ldots,d\\}$, from the stationarity and ergodicity
properties of
$\big{\\{}\frac{\partial^{2}q_{t}(\theta)}{\partial\theta\partial\theta^{\prime}},\
t\in\mathbb{Z}\big{\\}}$ and the uniform strong law of large numbers, it holds
that

$\Big{\|}\frac{1}{n}\sum_{t=1}^{n}\frac{\partial^{2}q_{t}(\theta)}{\partial\theta\partial\theta^{\prime}}-\mathbb{E}\Big{(}\frac{\partial^{2}q_{0}(\theta)}{\partial\theta\partial\theta^{\prime}}\Big{)}\Big{\|}_{\Theta}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle n\rightarrow\infty}\end{array}0.$

This completes the proof of Lemma 7.2.
$~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\blacksquare$

The following lemma is also needed.

###### Lemma 7.3

Assume that the conditions of Theorem 2.3 hold. Then

  1. (i.)

$\big{\\{}\frac{\partial
q_{t}(\theta^{*})}{\partial\theta}|\mathcal{F}_{t-1},\,t\in\mathbb{Z}\big{\\}}$
is a stationary ergodic martingale difference sequence with covariance matrix
$G$,

  2. (ii.)

$-\frac{1}{n}\frac{\partial^{2}}{\partial\theta\partial\theta^{\prime}}\widehat{L}_{n}(\tilde{\theta}_{n})\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle n\rightarrow\infty}\end{array}F$,
for any sequence $(\tilde{\theta}_{n})_{n\geq 1}$ with values in $\Theta$ and
satisfying $\tilde{\theta}_{n}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}\theta^{*}$,

where $G$ and $F$ are defined in (2.5).

_Proof of Lemma 7.3_

  1. (i.)

Recall that $G=\mathbb{E}\Big{[}\frac{\partial
q_{t}(\theta^{*})}{\partial\theta}\frac{\partial
q_{t}(\theta^{*})}{\partial\theta^{\prime}}\Big{]}$ and that for all
$\theta\in\Theta$,

$\frac{\partial
q_{t}(\theta)}{\partial\theta}=-2(H^{t}_{\theta})^{-1}(Y_{t}-f^{t}_{\theta})\frac{\partial
f^{t}_{\theta}}{\partial\theta}-\Big{(}\frac{Y_{t}-f^{t}_{\theta}}{H^{t}_{\theta}}\Big{)}^{2}\frac{\partial
H^{t}_{\theta}}{\partial\theta}+(H^{t}_{\theta})^{-1}\frac{\partial
H^{t}_{\theta}}{\partial\theta}.$

Since the functions $f^{t}_{\theta}$, $H^{t}_{\theta}$, $\frac{\partial
f^{t}_{\theta}}{\partial\theta}$ and $\frac{\partial
H^{t}_{\theta}}{\partial\theta}$ are $\mathcal{F}_{t-1}$-measurable, we have

$\mathbb{E}\Big{[}\frac{\partial
q_{t}(\theta^{*})}{\partial\theta}\big{|}\mathcal{F}_{t-1}\Big{]}=-(H^{t}_{\theta^{*}})^{-1}\frac{\partial
H^{t}_{\theta^{*}}}{\partial\theta}\left((H^{t}_{\theta^{*}})^{-1}\mathbb{E}\left[(Y_{t}-f^{t}_{\theta^{*}})^{2}|\mathcal{F}_{t-1}\right]-1\right)=0,$

which shows that (i.) holds.

  2. (ii.)

Let $(\tilde{\theta}_{n})_{n\in\mathbb{N}}$ be a sequence satisfying
$\tilde{\theta}_{n}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}\theta^{*}$. For any $i,j=1,\ldots,d$, we have

$\displaystyle\Big{|}\frac{1}{n}\sum\limits_{t=1}^{n}\frac{\partial}{\partial\theta_{j}\partial\theta_{i}}q_{t}(\tilde{\theta}_{n})-\mathbb{E}\Big{(}\frac{\partial}{\partial\theta_{j}\partial\theta_{i}}q_{0}(\theta^{*})\Big{)}\Big{|}$
$\displaystyle\leq\Big{|}\frac{1}{n}\sum\limits_{t=1}^{n}\frac{\partial}{\partial\theta_{j}\partial\theta_{i}}q_{t}(\tilde{\theta}_{n})-\mathbb{E}\Big{(}\frac{\partial}{\partial\theta_{j}\partial\theta_{i}}q_{0}(\tilde{\theta}_{n})\Big{)}\Big{|}+\Big{|}\mathbb{E}\Big{(}\frac{\partial}{\partial\theta_{j}\partial\theta_{i}}q_{0}(\tilde{\theta}_{n})\Big{)}-\mathbb{E}\Big{(}\frac{\partial}{\partial\theta_{j}\partial\theta_{i}}q_{0}(\theta^{*})\Big{)}\Big{|}$
$\displaystyle\leq\Big{\|}\frac{1}{n}\sum\limits_{t=1}^{n}\frac{\partial}{\partial\theta_{j}\partial\theta_{i}}q_{t}(\theta)-\mathbb{E}\Big{(}\frac{\partial}{\partial\theta_{j}\partial\theta_{i}}q_{0}(\theta)\Big{)}\Big{\|}_{\Theta}+\Big{|}\mathbb{E}\Big{(}\frac{\partial}{\partial\theta_{j}\partial\theta_{i}}q_{0}(\tilde{\theta}_{n})\Big{)}-\mathbb{E}\Big{(}\frac{\partial}{\partial\theta_{j}\partial\theta_{i}}q_{0}(\theta^{*})\Big{)}\Big{|}$
$\displaystyle\begin{array}[t]{c}\stackrel{{\scriptstyle}}{{\longrightarrow}}\\\
{\scriptstyle n\rightarrow\infty}\end{array}0~{}~{}\text{(by virtue of Lemma
\ref{lem2} (iii.))}.$

Thus,

$-\frac{1}{n}\frac{\partial^{2}}{\partial\theta\partial\theta^{\prime}}L_{n}(\tilde{\theta}_{n})=\frac{1}{n}\sum\limits_{t=1}^{n}\frac{\partial^{2}}{\partial\theta\partial\theta^{\prime}}q_{t}(\tilde{\theta}_{n})\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}\mathbb{E}\Big{(}\frac{\partial^{2}}{\partial\theta\partial\theta^{\prime}}q_{0}(\theta^{*})\Big{)}=F.$

We conclude the proof of the part (ii.) by using Lemma 7.2
(ii.).$~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\blacksquare$

Now, we use the results of Lemma 7.2 and 7.3 to prove the first part of
Theorem 2.3. The second part can be established by going along similar lines
as in Kengne (2021).
By applying a second-order Taylor expansion to the function
$\theta\mapsto\widehat{L}_{n}(\theta)$, for all $\theta\in\Theta$, there
exists $\tilde{\theta}$ between $\theta$ and $\theta^{*}$ such that

$\frac{1}{n}\left\\{\widehat{L}_{n}(\theta)-\widehat{L}_{n}(\theta^{*})\right\\}=\frac{1}{n}\frac{\partial
L_{n}(\theta^{*})}{\partial\theta^{\prime}}(\theta-\theta^{*})-\frac{1}{2}(\theta-\theta^{*})^{\prime}F(\theta-\theta^{*})+R_{n}(\theta),$
(7.20)

where

$R_{n}(\theta)=\frac{1}{n}\Big{\\{}\frac{\partial\widehat{L}_{n}(\theta^{*})}{\partial\theta^{\prime}}-\frac{\partial
L_{n}(\theta^{*})}{\partial\theta^{\prime}}\Big{\\}}(\theta-\theta^{*})+\frac{1}{2}(\theta-\theta^{*})^{\prime}\Big{(}\frac{1}{n}\frac{\partial^{2}}{\partial\theta\partial\theta^{\prime}}\widehat{L}_{n}(\tilde{\theta})+F\Big{)}(\theta-\theta^{*}).$

Let us define the vector

$Z_{n}=F^{-1}\frac{1}{\sqrt{n}}\frac{\partial
L_{n}(\theta^{*})}{\partial\theta}.$

Then, we can rewrite (7.20) as

$\frac{1}{n}\left\\{\widehat{L}_{n}(\theta)-\widehat{L}_{n}(\theta^{*})\right\\}=\frac{1}{2n}\left\|Z_{n}\right\|^{2}_{F}-\frac{1}{2n}\left\|Z_{n}-\sqrt{n}(\theta-\theta^{*})\right\|^{2}_{F}+R_{n}(\theta).$
(7.21)

Define also

$\theta_{Z_{n}}=\text{arg}\inf_{\theta\in\Theta}\left\|Z_{n}-\sqrt{n}(\theta-\theta^{*})\right\|_{F}.$

Then, by (2.4), for $n$ large enough, we have

$\sqrt{n}(\theta_{Z_{n}}-\theta^{*})=Z^{\mathcal{C}}_{n},$

where $Z^{\mathcal{C}}_{n}$ is the $F$-projection of $Z_{n}$ on $\mathcal{C}$.
Using this relation and the definition of $\theta_{Z_{n}}$, we have

$\big{\|}Z_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}-\big{\|}Z_{n}-Z^{\mathcal{C}}_{n}\big{\|}^{2}_{F}=\big{\|}Z_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}-\left\|Z_{n}-\sqrt{n}(\theta_{Z_{n}}-\theta^{*})\right\|^{2}_{F}\geq
0.$

Furthermore, from (7.21) and the definition of $\widehat{\theta}_{n}$, it
holds that

$\displaystyle\big{\|}Z_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}-\left\|Z_{n}-\sqrt{n}(\theta_{Z_{n}}-\theta^{*})\right\|^{2}_{F}$
$\displaystyle=\\{\widehat{L}_{n}(\theta_{Z_{n}})-\widehat{L}_{n}(\widehat{\theta}_{n})\\}+2n\\{R_{n}(\widehat{\theta}_{n})-R_{n}(\theta_{Z_{n}})\\}$
$\displaystyle\leq 2n\\{R_{n}(\widehat{\theta}_{n})-R_{n}(\theta_{Z_{n}})\\}.$

Therefore,

$\left|\big{\|}Z_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}-\big{\|}Z_{n}-Z^{\mathcal{C}}_{n}\big{\|}^{2}_{F}\right|\leq
2n\\{R_{n}(\widehat{\theta}_{n})-R_{n}(\theta_{Z_{n}})\\}.$ (7.22)

Let us consider the following Lemma.

###### Lemma 7.4

Assume that the conditions of Theorem 2.3 hold. Then

$n\\{R_{n}(\widehat{\theta}_{n})-R_{n}(\theta_{Z_{n}})\\}=o_{P}(1).$

By Lemma 7.4 and (7.22), it follows that

$\big{\|}Z_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}-\big{\|}Z^{\mathcal{C}}_{n}-Z_{n}\big{\|}^{2}_{F}=o_{P}(1).$
(7.23)

Moreover, according to the equivalent definition of the $F$-orthogonal
projection in (2.6), we get

$\displaystyle\big{\|}Z_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}$
$\displaystyle=\big{\|}Z^{\mathcal{C}}_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}+\big{\|}Z^{\mathcal{C}}_{n}-Z_{n}\big{\|}^{2}_{F}-2\left\langle
Z^{\mathcal{C}}_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*}),Z^{\mathcal{C}}_{n}-Z_{n}\right\rangle$
$\displaystyle\geq\big{\|}Z^{\mathcal{C}}_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}+\big{\|}Z^{\mathcal{C}}_{n}-Z_{n}\big{\|}^{2}_{F}.$

Therefore, from (7.23), we obtain

$\big{\|}Z^{\mathcal{C}}_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}\leq\big{\|}Z_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}-\big{\|}Z^{\mathcal{C}}_{n}-Z_{n}\big{\|}^{2}_{F}=o_{P}(1).$
(7.24)

Now, using Lemma 7.3 (i.), we apply the central limit theorem for the
stationary ergodic martingale difference sequence $\big{\\{}\frac{\partial
q_{t}(\theta^{*})}{\partial\theta}|\mathcal{F}_{t-1},\,t\in\mathbb{Z}\big{\\}}$.
It follows that

$\frac{1}{\sqrt{n}}\frac{\partial
L_{n}(\theta^{*})}{\partial\theta}=\frac{1}{\sqrt{n}}\sum\limits_{t=1}^{n}\frac{\partial
q_{t}(\theta^{*})}{\partial\theta}\begin{array}[t]{c}\stackrel{{\scriptstyle{\cal
D}}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}\mathcal{N}_{d}\left(0,G\right),$ (7.25)

and thus

$\displaystyle Z_{n}=F^{-1}\frac{1}{\sqrt{n}}\frac{\partial
L_{n}(\theta^{*})}{\partial\theta}\begin{array}[t]{c}\stackrel{{\scriptstyle{\cal
D}}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}Z\sim\mathcal{N}_{d}\left(0,F^{-1}GF^{-1}\right).$
(7.28)

Hence, $Z^{\mathcal{C}}_{n}\begin{array}[t]{c}\stackrel{{\scriptstyle{\cal
D}}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}Z^{\mathcal{C}}$. From this, it suffices to use
(7.24) to conclude the proof of Theorem 2.3.

$\blacksquare$

Proof of Lemma 7.4.
Recall that

$R_{n}(\theta)=\frac{1}{n}\Big{\\{}\frac{\partial\widehat{L}_{n}(\theta^{*})}{\partial\theta^{\prime}}-\frac{\partial
L_{n}(\theta^{*})}{\partial\theta^{\prime}}\Big{\\}}(\theta-\theta^{*})+\frac{1}{2}(\theta-\theta^{*})^{\prime}\Big{(}\frac{1}{n}\frac{\partial^{2}}{\partial\theta\partial\theta^{\prime}}\widehat{L}_{n}(\tilde{\theta})+F\Big{)}(\theta-\theta^{*}).$

According to Lemmas 7.2 (i.) and 7.3 (ii.), when
$\tilde{\theta}_{n}-\theta^{*}=o_{P}(1)$, we have

$nR_{n}(\tilde{\theta}_{n})=o_{P}(\sqrt{n}(\tilde{\theta}_{n}-\theta^{*}))+o_{P}(n\|\tilde{\theta}_{n}-\theta^{*}\|^{2}).$
(7.29)

This implies

$nR_{n}(\tilde{\theta}_{n})=o_{P}(1)~{}\text{ when
}~{}\sqrt{n}(\tilde{\theta}_{n}-\theta^{*})=O_{P}(1).$ (7.30)

It comes from the definition of $\theta_{Z_{n}}$ that

$\left\|\sqrt{n}(\theta_{Z_{n}}-\theta^{*})\right\|_{F}\leq\left\|\sqrt{n}(\theta_{Z_{n}}-\theta^{*})-Z_{n}\right\|_{F}+\left\|Z_{n}\right\|_{F}\leq
2\left\|Z_{n}\right\|_{F}.$

Moreover, the convergence in (7.28) implies
$\left\|Z_{n}\right\|_{F}=O_{P}(1)$; and consequently,
$\sqrt{n}(\theta_{Z_{n}}-\theta^{*})=O_{P}(1)$. Thus,
$nR_{n}(\theta_{Z_{n}})=o_{P}(1)$ by virtue (7.30).

We now show that, it also holds $nR_{n}(\widehat{\theta}_{n})=o_{P}(1)$. From
(7.21), we have

$\left\|Z_{n}\right\|^{2}_{F}-\big{\|}Z_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}+2nR_{n}(\widehat{\theta}_{n})=2\\{\widehat{L}_{n}(\widehat{\theta}_{n})-\widehat{L}_{n}(\theta^{*})\\}\geq
0,$

where the inequality holds since
$\widehat{\theta}_{n}=\underset{\theta\in\Theta}{\text{argmax}}(\widehat{L}_{n}(\theta))$.
Thus, it holds that

$\displaystyle\big{\|}\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}$
$\displaystyle\leq
2\big{(}\big{\|}Z_{n}-\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}+\left\|Z_{n}\right\|^{2}_{F}\big{)}$
$\displaystyle\leq
4\left\|Z_{n}\right\|^{2}_{F}+4nR_{n}(\widehat{\theta}_{n}).$

Furthermore, since
$\widehat{\theta}_{n}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}\theta^{*}$, by (7.29), it follows that
$nR_{n}(\widehat{\theta}_{n})=o_{P}\big{(}\big{\|}\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\big{\|}^{2}_{F}\big{)}$.
Consequently, $\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})=O_{P}(1)$, and
$nR_{n}(\widehat{\theta}_{n})=o_{P}(1)$ holds according to (7.30). This
achieves the proof of the lemma.

$~{}~{}~{}~{}~{}~{}\blacksquare$

###  7.4 Proof of Theorem 2.4

Under $H_{0}$, we have
$\Gamma\widehat{\theta}_{n}-\vartheta_{0}=\Gamma(\widehat{\theta}_{n}-\theta^{*})$.
Then, we get

$\displaystyle W_{n}$
$\displaystyle=n(\Gamma\widehat{\theta}_{n}-\vartheta_{0})^{\prime}(\Gamma\widehat{\Sigma}_{n}\Gamma^{\prime})^{-1}(\Gamma\widehat{\theta}_{n}-\vartheta_{0})$
$\displaystyle=n(\widehat{\theta}_{n}-\theta^{*})^{\prime}\Gamma^{\prime}(\Gamma\widehat{\Sigma}_{n}\Gamma^{\prime})^{-1}\Gamma(\widehat{\theta}_{n}-\theta^{*})$
$\displaystyle=\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})^{\prime}\Gamma^{\prime}(\Gamma\Sigma\Gamma^{\prime})^{-1}\Gamma\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})$
$\displaystyle\hskip
142.26378pt+\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})^{\prime}\Gamma^{\prime}\left((\Gamma\widehat{\Sigma}_{n}\Gamma^{\prime})^{-1}-(\Gamma\Sigma\Gamma^{\prime})^{-1}\right)\Gamma\sqrt{n}(\widehat{\theta}_{n}-\theta^{*}).$
(7.31)

Recall that, by Theorem 2.4, we have
$\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})\begin{array}[t]{c}\stackrel{{\scriptstyle{\cal
D}}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}Z^{\mathcal{C}}~{}$ with
$~{}Z\sim\mathcal{N}_{d}\left(0,\Sigma\right)$.
Furthermore,
$(\Gamma\widehat{\Sigma}_{n}\Gamma^{\prime})^{-1}-(\Gamma\Sigma\Gamma^{\prime})^{-1}=o_{P}(1)$.
Thus, from (7.4), it holds that

$\displaystyle W_{n}$
$\displaystyle=\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})^{\prime}\Gamma^{\prime}(\Gamma\Sigma\Gamma^{\prime})^{-1}\Gamma\sqrt{n}(\widehat{\theta}_{n}-\theta^{*})+o_{P}(1)$
$\displaystyle~{}~{}\begin{array}[t]{c}\stackrel{{\scriptstyle{\cal
D}}}{{\longrightarrow}}\\\ {\scriptstyle n\rightarrow\infty}\end{array}(\Gamma
Z^{\mathcal{C}})^{\prime}(\Gamma\Sigma\Gamma^{\prime})^{-1}\Gamma Z,$

which establishes the theorem.

$\blacksquare$

Proof of Corollary 2.5.
When $\theta^{*}\in\overset{\circ}{\Theta}$, we have
$W_{n}\begin{array}[t]{c}\stackrel{{\scriptstyle{\cal
D}}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}Z^{\prime}\Gamma^{\prime}(\Gamma\Sigma\Gamma^{\prime})^{-1}\Gamma
Z={\left\|U\right\|}^{2}$ with $U=(\Gamma\Sigma\Gamma^{\prime})^{-1/2}\Gamma
Z$ and $Z\sim\mathcal{N}_{d}\left(0,\Sigma\right)$. Since $\Sigma$ is
symmetric, the vector $U$ follows a multivariate Gaussian distribution with
mean $0$ and covariance matrix $I_{d_{0}}$, where $I_{d_{0}}$ is the identity
matrix of size $d_{0}$. Therefore, all components of $U$ are independent,
standard normal distributed random variables. This leads to the conclusion.

$\blacksquare$

###  7.5 Proof of Theorem 3.1

Consider the following lemma.

###### Lemma 7.5

Assume that the conditions of Theorem 3.1 hold. Then

$\frac{1}{\sqrt{n\log\log
n}}\Big{\|}\frac{\partial\widehat{L}_{n}(\theta)}{\partial\theta}-\frac{\partial
L_{n}(\theta)}{\partial\theta}\Big{\|}_{\Theta}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle n\rightarrow\infty}\end{array}0.$

_Proof of Lemma 7.5._
Using the inequality (7.10) and Corollary 1 of of Kounias and Weng (1969), it
suffices to show that

$\sum_{k\geq 2}\frac{1}{\sqrt{k\log\log
k}}\mathbb{E}\left[\Big{\|}\frac{\partial\widehat{q}_{k}(\theta)}{\partial\theta}-\frac{\partial
q_{k}(\theta)}{\partial\theta}\Big{\|}_{\Theta}\right]<\infty.$ (7.32)

In the proof of Lemma 7.3, we have established that

$\displaystyle\Big{\|}\frac{\partial\widehat{q}_{k}(\theta)}{\partial\theta}-\frac{\partial
q_{k}(\theta)}{\partial\theta}\Big{\|}_{\Theta}$ $\displaystyle\leq
C\sum\limits_{j\geq
k}\Big{\\{}\alpha^{(0)}_{j,Y}(f_{\theta},\Theta)+\alpha^{(0)}_{j,X}(f_{\theta},\Theta)+\alpha^{(1)}_{j,Y}(f_{\theta},\Theta)+\alpha^{(1)}_{k,X}(f_{\theta},\Theta)$
$\displaystyle\hskip
85.35826pt+\alpha^{(0)}_{j,Y}(M_{\theta},\Theta)+\alpha^{(0)}_{j,X}(M_{\theta},\Theta)+\alpha^{(1)}_{j,Y}(M_{\theta},\Theta)+\alpha^{(1)}_{j,X}(M_{\theta},\Theta)\Big{\\}}$
$\displaystyle=C\sum\limits_{j\geq
k}\sum_{i=0}^{1}\left\\{\alpha^{(i)}_{j,Y}(f_{\theta},\Theta)+\alpha^{(i)}_{j,X}(f_{\theta},\Theta)+\alpha^{(i)}_{j,Y}(M_{\theta},\Theta)+\alpha^{(i)}_{j,X}(M_{\theta},\Theta)\right\\}.$

Then, from the condition (3.4), we obtain

$\sum_{k\geq 2}\frac{1}{\sqrt{k\log\log
k}}\mathbb{E}\left[\Big{\|}\frac{\partial\widehat{q}_{k}(\theta)}{\partial\theta}-\frac{\partial
q_{k}(\theta)}{\partial\theta}\Big{\|}_{\Theta}\right]\\\ \leq\sum_{k\geq
2}\frac{1}{\sqrt{k\log\log k}}\sum_{j\geq
k}\sum_{i=0}^{1}\big{\\{}\alpha^{(i)}_{j,Y}(f_{\theta},\Theta)+\alpha^{(i)}_{j,X}(f_{\theta},\Theta)+\alpha^{(i)}_{j,Y}(M_{\theta},\Theta)+\alpha^{(i)}_{j,X}(M_{\theta},\Theta)\big{\\}}<\infty.$

Hence, (7.32) is satisfied, and Lemma 7.5 holds.

$\blacksquare$

Let us prove the part (i.) of the theorem.

  1. (i.)

We have

$P(\widehat{m}_{n}=m^{*})=1-P(\widehat{m}_{n}\supsetneq
m^{*})-P(\widehat{m}_{n}\nsupseteq m^{*}).$

Therefore, it suffices to show that

$\lim\limits_{n\rightarrow\infty}P(\widehat{m}_{n}\supsetneq
m^{*})=\lim\limits_{n\rightarrow\infty}P(\widehat{m}_{n}\nsupseteq m^{*})=0.$
(7.33)

1\. Let $m\in{\cal M}$ such as $m\supsetneq m^{*}$. We have,

$\frac{1}{\sqrt{\log\log
n}}\big{(}\widehat{C}(m^{*})-\widehat{C}(m)\big{)}=\frac{2}{\sqrt{\log\log
n}}\big{(}\widehat{L}_{n}\big{(}\widehat{\theta}(m)\big{)}-\widehat{L}_{n}\big{(}\widehat{\theta}(m^{*})\big{)}-\frac{\kappa_{n}}{\sqrt{\log\log
n}}(|m|-|m^{*}|).$ (7.34)

Let us establish that

$\frac{1}{\sqrt{\log\log
n}}\big{(}\widehat{L}_{n}\big{(}\widehat{\theta}(m)\big{)}-\widehat{L}_{n}\big{(}\widehat{\theta}(m^{*})\big{)}\big{)}=O_{P}(1).$
(7.35)

From the Taylor expansion of $\widehat{L}_{n}$, we can find
$\overline{\theta}(m)$ between $\widehat{\theta}(m)$ and $\theta^{*}$ such
that

$\widehat{L}_{n}\big{(}\widehat{\theta}(m)\big{)}-\widehat{L}_{n}\big{(}\theta^{*}\big{)}=\dfrac{\partial
L_{n}(\theta^{*})}{\partial\theta}\big{(}\widehat{\theta}(m)-\theta^{*}\big{)}-\frac{1}{2}\sqrt{n}\big{(}\widehat{\theta}(m)-\theta^{*}\big{)}^{\prime}F(\theta^{*},m)\sqrt{n}\big{(}\widehat{\theta}(m)-\theta^{*}\big{)}+nR^{\prime}_{n}(m),$
(7.36)

where

$R^{\prime}_{n}(m)=\frac{1}{n}\Big{\\{}\frac{\partial\widehat{L}_{n}(\theta^{*})}{\partial\theta^{\prime}}-\frac{\partial
L_{n}(\theta^{*})}{\partial\theta^{\prime}}\Big{\\}}\big{(}\widehat{\theta}(m)-\theta^{*}\big{)}\\\
+\frac{1}{2}\big{(}\widehat{\theta}(m)-\theta^{*}\big{)}^{\prime}\Big{(}\frac{1}{n}\frac{\partial^{2}}{\partial\theta\partial\theta^{\prime}}\widehat{L}_{n}(\overline{\theta}(m))+F(\theta^{*},m)\Big{)}\big{(}\widehat{\theta}(m)-\theta^{*}\big{)}$

and

$\displaystyle
F(\theta^{*},m)=\Big{(}\mathbb{E}\Big{[}\dfrac{\partial^{2}q_{0}(\theta^{*})}{\partial\theta_{i}\partial\theta_{j}}\Big{]}\Big{)}_{i,j\in
m}.$

Moreover, since
$\widehat{\theta}(m),~{}\overline{\theta}(m)\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}\theta^{*}$, in this case of overfitting, the
same arguments as in the proof of Lemma 7.3 (ii.) lead to

$-\dfrac{1}{n}\dfrac{\partial^{2}\widehat{L}_{n}\big{(}\overline{\theta}(m)\big{)}}{\partial\theta\partial\theta^{\prime}}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}F(\theta^{*},m).$

Then, one can show as in the proof of Theorem 2.3 that
$nR^{\prime}_{n}(m)=o_{P}(1)$. Also, we have
$\sqrt{n}\big{(}\widehat{\theta}(m)-\theta^{*}\big{)}=O_{P}(1)$. In addition,
$\big{\\{}\frac{\partial
q_{t}(\theta^{*})}{\partial\theta}|\mathcal{F}_{t-1},\,t\in\mathbb{Z}\big{\\}}$
is a stationary ergodic square integrable martingale difference sequence (see
above). Hence, from the law of iterative logarithm for martingales (see for
instance [24, 25]), we get,

$\frac{1}{\sqrt{n\log\log n}}\dfrac{\partial
L_{n}(\theta^{*})}{\partial\theta}=O(1).$

Thus, we have from (7.36),

$\displaystyle\frac{1}{\sqrt{\log\log
n}}\big{(}\widehat{L}_{n}\big{(}\widehat{\theta}(m)\big{)}-\widehat{L}_{n}\big{(}\theta^{*}\big{)}\big{)}$
$\displaystyle=\frac{1}{\sqrt{n\log\log n}}\dfrac{\partial
L_{n}(\theta^{*})}{\partial\theta}\sqrt{n}\big{(}\widehat{\theta}(m)-\theta^{*}\big{)}+\frac{1}{2\sqrt{\log\log
n}}\sqrt{n}\big{(}\widehat{\theta}(m)-\theta^{*}\big{)}^{\prime}F(\theta^{*},m)\sqrt{n}\big{(}\widehat{\theta}(m)-\theta^{*}\big{)}$
$\displaystyle\hskip 341.71744pt+\dfrac{1}{\sqrt{\log\log
n}}nR^{\prime}_{n}(m)$
$\displaystyle=O(1)O_{P}(1)+o(1)O_{P}(1)O_{P}(1)+o_{P}(1)=O_{P}(1).$ (7.37)

By using the same arguments with $m=m^{*}$, we get

$\frac{1}{\sqrt{\log\log
n}}\big{(}\widehat{L}_{n}\big{(}\widehat{\theta}(m^{*})\big{)}-\widehat{L}_{n}\big{(}\theta^{*})\big{)}=O_{P}(1).$
(7.38)

Hence, (7.35) holds from ((i.)) and (7.38).

Therefore, since $\kappa_{n}/\sqrt{\log\log
n}\begin{array}[t]{c}\stackrel{{\scriptstyle}}{{\longrightarrow}}\\\
{\scriptstyle n\rightarrow\infty}\end{array}\infty$ and $|m|>|m^{*}|$, then
(7.34) and (7.35) lead to

$\frac{1}{\sqrt{\log\log
n}}\big{(}\widehat{C}(m^{*})-\widehat{C}(m)\big{)}\begin{array}[t]{c}\stackrel{{\scriptstyle{\cal
P}}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}-\infty~{}~{}.$

This implies that, for large $n$,

$\widehat{C}(m)-\widehat{C}(m^{*})>0$

with probability one; that is, $P(\widehat{m}_{n}\supsetneq
m^{*})\begin{array}[t]{c}\stackrel{{\scriptstyle}}{{\longrightarrow}}\\\
{\scriptstyle n\rightarrow\infty}\end{array}0$.

2\. Let $m\in{\cal M}$ such as $m\nsupseteq m^{*}$. We have,

$\frac{1}{n}\big{(}\widehat{C}(m^{*})-\widehat{C}(m)\big{)}=\frac{2}{n}\big{(}\widehat{L}_{n}\big{(}\widehat{\theta}(m)\big{)}-\widehat{L}_{n}\big{(}\widehat{\theta}(m^{*})\big{)}\big{)}-\frac{\kappa_{n}}{n}(|m|-|m^{*}|).$
(7.39)

Using the same arguments in the proof of Theorem 3.1 of Bardet et al. (2020),
we get

$\frac{1}{n}\big{(}\widehat{L}_{n}\big{(}\widehat{\theta}(m)\big{)}-\widehat{L}_{n}\big{(}\widehat{\theta}(m^{*})\big{)}\big{)}=L(\theta^{*}(m))-L(\theta^{*})+o(1)~{}~{}a.s.,$

where $L(\theta)=-\mathbb{E}[q_{0}(\theta)]$, for all $\theta\in\Theta$. Note
that, the function $L:\Theta\rightarrow\mathbb{R}$ has a unique maximum at
$\theta^{*}$ (see the proof of Theorem 2.2). Since $m\nsupseteq m^{*}$, it
holds that $\theta^{*}\notin\Theta(m)$; and consequently,
$L(\theta^{*}(m))-L(\theta^{*})<0~{}~{}a.s.$. Thus, according to (7.39) and
since
$\kappa_{n}/n\begin{array}[t]{c}\stackrel{{\scriptstyle}}{{\longrightarrow}}\\\
{\scriptstyle n\rightarrow\infty}\end{array}0$, we get

$\lim_{n\rightarrow\infty}\frac{1}{n}\big{(}\widehat{C}(m^{*})-\widehat{C}(m)\big{)}<0~{}~{}~{}a.s.~{}\text{
and }~{}~{}\widehat{C}(m)-\widehat{C}(m^{*})>0~{}~{}~{}a.s.~{}\text{ for large
}n.$

This implies that $P(\widehat{m}_{n}\nsupseteq
m^{*})\begin{array}[t]{c}\stackrel{{\scriptstyle}}{{\longrightarrow}}\\\
{\scriptstyle n\rightarrow\infty}\end{array}0$. Hence, the condition (7.33)
holds; and the part (i.) of the theorem is established.

  2. (ii.)

Let $m\in{\cal M}$ such as $m\supsetneq m^{*}$. We have

$\frac{1}{\log\log
n}\big{(}\widehat{C}(m)-\widehat{C}(m^{*})\big{)}=\frac{2}{\log\log
n}\big{(}\widehat{L}_{n}\big{(}\widehat{\theta}(m^{*})\big{)}-\widehat{L}_{n}\big{(}\widehat{\theta}(m)\big{)}\big{)}+\frac{\kappa_{n}}{\log\log
n}(|m|-|m^{*}|).$

Moreover, from the same arguments as in the proof of Theorem 3.1 in Kengne
(2021), one can show that

$\frac{1}{\log\log
n}\big{(}\widehat{L}_{n}\big{(}\widehat{\theta}(m^{*})\big{)}-\widehat{L}_{n}\big{(}\widehat{\theta}(m)\big{)}\big{)}=O(1)~{}~{}a.s.$

Thus, we can find a constant $c$ such that if
$\underset{n\rightarrow\infty}{\liminf}\kappa_{n}/\log\log n>c$, then

$\liminf_{n\rightarrow\infty}\frac{1}{\log\log
n}\big{(}\widehat{C}(m)-\widehat{C}(m^{*})\big{)}>0~{}~{}a.s.$

This implies that

$\widehat{C}(m)-\widehat{C}(m^{*})>0~{}~{}a.s.~{}~{}\text{for large }n.$
(7.40)

Note that, the inequality (7.40) also holds when $m\nsupseteq m^{*}$ (see the
part 2. of the proof of (i.)). Hence, we deduce that
$\widehat{m}_{n}=\underset{m\in\mathcal{M}}{\text{argmin}}~{}\widehat{C}(m)=\underset{m\in\mathcal{M}}{\text{argmin}}\big{(}\widehat{C}(m)-\widehat{C}(m^{*})\big{)}\begin{array}[t]{c}\stackrel{{\scriptstyle
a.s.}}{{\longrightarrow}}\\\ {\scriptstyle
n\rightarrow\infty}\end{array}m^{*}$; which establishes the strong consistency
of $\widehat{m}_{n}$.

  3. (iii.)

Using Lemma 7.5, this part can be proved by going along similar lines as in
Kengne (2021).

$\blacksquare$

## References

  * [1] Bardet, J. M., Kengne, K. and Wintenberger, O. Multiple breaks detection in general causal time series using penalized quasi-likelihood.  Electronic Journal of Statistics 6, (2012), 435-477.
  * [2] Bardet, J.-M. and Wintenberger, O. Asymptotic normality of the quasi-maximum likelihood estimator for multidimensional causal processes.  The Annals of Statistics 37, 5B, (2009), 2730-2759.
  * [3] Bardet, J.-M., Kamila, K. and Kengne, W. Consistent model selection criteria and goodness-of-fit test for common time series models.  Electron. J. Stat. 14, (2020), 2009-2052.
  * [4] Berkes, I. Horváth, L. and Kokoszka, P. GARCH processes: Structure and estimation. Bernoulli 9, (2003), 201-227.
  * [5] Bierens H.J. Estimation, testing and specification of cross-section and time series models.  Cambridge University Press, (1994).
  * [6] Doukhan, P. and Wintenberger, O. Weakly dependent chains with infinite memory.  Stochastic Process. Appl. 118, (2008), 1997-2013.
  * [7] Francq, C. and Sucarrat, G. An equation-by-equation estimator of a multivariate log-GARCH-X model of financial returns. Journal of Multivariate Analysis, 153, (2017), 16-32.
  * [8] Francq, C. and Thieu, L.Q. QML inference for volatility models with covariates. Econometric Theory, 35, (2019), 37-72.
  * [9] Francq, C. and Zakoïan, J.M. Maximum likelihood estimation of pure GARCH and ARMA-GARCH processes. Bernoulli 10, (2004), 605-637.
  * [10] Grønneberg, S. and Holcblat, B. Factor double autoregressive models with application to simultaneous causality testing. Annals of Statistics 47(6), (2019), 3216–3243.
  * [11] Guo, S., Ling S. and Zhu, K. Factor double autoregressive models with application to simultaneous causality testing. Journal of Statistical Planning and Inference 148, (2014), 82-94.
  * [12] Han, H. and Kristensen, D. Asymptotic Theory for the QMLE in GARCHX Models With Stationary and Nonstationary Covariates.  Journal of Business $\&$ Economic Statistics 32, (2014) , 416-429.
  * [13] Han, H. Asymptotic properties of GARCH-X processes.  Journal of Financial Econometrics, 13, (2015), 188-221
  * [14] Hannan, E.J. The identification and parameterization of ARMAX and state space forms.  Econometrica: Journal of the Econometric Society, (1976), 713–723.
  * [15] Hannan, E.J., Dunsmuir, W.T.M. and Deistler, M. Estimation of Vector ARMAX Models.  Journal of Multivariate Analysis, 10, (1980), 275-295.
  * [16] Hannan, E. J. and Deistler, M. The statistical theory of linear systems.  SIAM, (2012).
  * [17] Hoque, A. and Peters, T.A. Finite Sample Analysis of Least Squares in ARMAX Models.  The Indian Journal of Statistics, B 48, (1986), 266-283.
  * [18] Kengne, W. Strongly consistent model selection for general causal time series.  Statistics and Probability Letters, (2021).
  * [19] Kounias, E.G. and Weng, T.-S.  An inequality and almost sure convergence.  Annals of Mathematical Statistics 33, (1969), 1091-1093.
  * [20] Ling, S. and McAleer, M. Asymptotic theory for a vector ARMA-GARCH model.  Econometric theory 19(02) (2003), 280-310.
  * [21] Ling, S. Adouble AR$(p)$ model: structure and estimation.  Statist. Sinica 17, (2007), 161-175.
  * [22] Nana, G.N., Korn, R. and Elwein-Sayer, C. GARCH-extended models: theoretical properties and applications.  arXiv:1307.6685v1, (2013).
  * [23] Pedersen, R.S. and Rahbek, A. Testing Garch-X Type Models.  Econometric Theory 35(5), (2018), 1-36.
  * [24] Stout, W. F. The Hartman-Wintner law of the iterated logarithm for martingales.  The Annals of Mathematical Statistics 41 (1970), 2158-2160.
  * [25] Stout, W. F. Almost sure convergence.  Academic press  (1974).
  * [26] Sucarrat, Genaro and Grønneberg, S. and Escribano, A. Testing for local covariate trend effects in volatility models.  Computational Statistics & Data Analysis 100 (2016), 582-594.
  * [27] Zambom, A. Z. and Gel, Y. R. Testing for local covariate trend effects in volatility models.  Electronic Journal of Statistics 14(2) (2020), 2529-2550.