# Limit Theory under Network Dependence and Nonstationarity111These lecture
notes were prepared during my Ph.D. studies at the Department of Economics of
the School of Economic, Social and Political Sciences, University of
Southampton, Southampton SO17 1BJ, United Kingdom. This draft was updated
during the academic year 2022-2023, Department of Economics, University of
Exeter.
Dr. Christis Katsouris is a Lecturer in Economics, University of Exeter
Business School, Exeter EX4 4PU, United Kingdom. Email:
<EMAIL_ADDRESS>
Christis Katsouris
Department of Economics, University of Southampton

###### Abstract

These lecture notes represent supplementary material for a short course on
time series econometrics and network econometrics. We give emphasis on limit
theory for time series regression models as well as the use of the local-to-
unity parametrization when modeling time series nonstationarity. Moreover, we
present various non-asymptotic theory results for moderate deviation
principles when considering the eigenvalues of covariance matrices as well as
asymptotics for unit root moderate deviations in nonstationary autoregressive
processes. Although not all applications from the literature are covered we
also discuss some open problems in the time series and network econometrics
literature.

###### Contents

  1. 1\. Introduction
    1. 1.1. Time Series as a Stochastic process
    2. 1.2. Martingale Theorems
    3. 1.3. Covariance Matrix Estimation
      1. 1.3.1. Long-Run variance
      2. 1.3.2. Consistent Covariance Matrix Estimation for Linear Processes
    4. 1.4. Continuous Asymptotics
    5. 1.5. The Ornstein–Uhlenbeck process
    6. 1.6. Higher-Order Autoregressive Processes
      1. 1.6.1. AR(p) Time Series Regression
      2. 1.6.2. The Autoregressive and Moving Average Processes (ARMA)
    7. 1.7. Moderate Deviations for Multivariate Martingales
    8. 1.8. Estimators for Gaussian Autoregressive Process
    9. 1.9. The Exogeneity Assumption in Time Series Models
    10. 1.10. Autoregressive Processes with Exogenous Variables
  2. 2\. Temporal Dependence
    1. 2.1. Setting the time series framework
    2. 2.2. Example: Nonlinear time series models
    3. 2.3. Heavy Tail Time series
      1. 2.3.1. Max-stable processes
      2. 2.3.2. Tail behaviour of sample quantiles
    4. 2.4. Extreme Value Theory for nonstationary time series
  3. 3\. Limit Theory under Network Dependence
    1. 3.1. Random Graphs and Moderate Deviations Theory
      1. 3.1.1. Spectral Edge in Sparse Random Graphs: Tail Large Deviations
    2. 3.2. Limit theorems for network dependent processes
      1. 3.2.1. Network dependence condition
      2. 3.2.2. Law of Large numbers
      3. 3.2.3. Central limit theorem
      4. 3.2.4. Network HAC
    3. 3.3. Stable Limit Theorems under Network Dependence
      1. 3.3.1. Stable Convergence of an Empirical Process
      2. 3.3.2. Maximal Inequality
      3. 3.3.3. Stable Central Limit Theorem
    4. 3.4. Moderate Deviations: Eigenvalues of Covariance Matrices
      1. 3.4.1. Limit of the smallest eigenvalue of a Large Sample Covariance Matrix
      2. 3.4.2. Limit Theory for the largest eigenvalues of sample covariance matrices with heavy-tails
      3. 3.4.3. Limit Spectral Distribution for symmetric random matrices with correlated entries
    5. 3.5. Moderate Deviations for Extreme Eigenvalues
      1. 3.5.1. Moderate Deviations for $\lambda_{\text{max}}$, $\lambda_{\text{min}}$ with fixed $k$
      2. 3.5.2. Discussion on High Dimensional Results
  4. 4\. Nonstationary Time Series Regressions
    1. 4.1. Testing for unit root in time series regression
      1. 4.1.1. Power Functions for Unit Root Tests
      2. 4.1.2. Time Series Regression with a Unit Root
    2. 4.2. Regression Asymptotics using Martingale Convergence Methods
      1. 4.2.1. Unification of the Limit theory of Autoregression
    3. 4.3. Time Series Regression
      1. 4.3.1. Unit Roots versus Deterministic Trends
      2. 4.3.2. The Engle-Granger Approach
      3. 4.3.3. Testing the hypothesis of no serial correlation
      4. 4.3.4. A Cointegrated VAR
    4. 4.4. Cointegrating Regression
      1. 4.4.1. Distribution of the OLS Estimate for a Special Case
      2. 4.4.2. Estimation of cointegrating vectors
      3. 4.4.3. Cointegrating Vector and Equilibrium Points
    5. 4.5. Unit Roots, Cointegration and Structural Breaks
      1. 4.5.1. Testing for Structural Change in Cointegrated Systems
      2. 4.5.2. Trend Stationarity and Structural Break Testing
      3. 4.5.3. Cointegration Testing under Structural Breaks
      4. 4.5.4. Testing for a Structural Break in Cointegrating Regression
      5. 4.5.5. Unit root and Cointegrating limit theory
      6. 4.5.6. FM transformation in Cointegrating Regression
      7. 4.5.7. FM estimation with nearly integrated regressors
    6. 4.6. Dynamic Seemingly Unrelated Cointegrating Regression
    7. 4.7. Predictive Regression Models
      1. 4.7.1. Regression with Integrated Regressors
      2. 4.7.2. Optimal Inference in Predictive Regressions with Persistent Regressors
      3. 4.7.3. Moderate Deviations from a Unit Root in Autoregressive Processes
      4. 4.7.4. Multivariate Predictive Regression Model
      5. 4.7.5. Predictive Regression with Multiple Regressors
      6. 4.7.6. Testing for Structural Break in Predictive Regression
      7. 4.7.7. Main Aspects on Predictive Regressions
  5. 5\. Resampling for Time Series Regressions
    1. 5.1. Preliminary Theory on Bootstrap Methodologies
      1. 5.1.1. Bootstrap in regression models
    2. 5.2. On Bootstrap Asymptotics
      1. 5.2.1. Why the bootstrap provides Asymptotic Refinements
      2. 5.2.2. The Block Bootstrap
      3. 5.2.3. Studentization
      4. 5.2.4. The Sieve Bootstrap for Linear Processes
      5. 5.2.5. Discussion on Bootstrap Asymptotics
    3. 5.3. Bootstrapping in Time Series Regressions
  6. 6\. Open Problems
    1. 6.1. Literature on Measures of Connectedness
    2. 6.2. Further Research
  7. A Elements of Stochastic Processes
    1. A1. Probability Theory
    2. A2. Elements of Large Deviations Principles
  8. B Elements of Real Analysis
    1. B1. Convergence and Cauchy Sequences
    2. B2. Polynomial and Power Series Operators
    3. B3. Dynamic Multipliers Models
  9. C Elements of Weak Convergence
    1. C1. Joint Weak Convergence in $J_{1}$ topology
      1. C1.1. Functional Central Limit Theorems
    2. C2. Uniform Inference in the $J_{1}$ topology
      1. C2.1. Testing Methodology
      2. C2.2. Class of Test Statistics
      3. C2.3. Validity of Stock’s Method
    3. C3. Sub-Gaussian processes
    4. C4. Clivenko-Cantelli theorems
    5. C5. Donsker Theorems
  10. D Elements of Graph Limits Theory
  11. References

##  1\. Introduction

This set of lecture notes is about systems of predictive regressions under
network dependence. The first question that we will address is: why should we
model network dependence? Our answer to this issue is threefold. First, under
both network dependence and time series nonstationarity (using a local-to-
unity parametrization) conventional asymptotic theory results are not directly
applicable unless we understand the required regularity conditions and
modifications that are needed.

###  1.1. Time Series as a Stochastic process

Consider $\left\\{X_{\theta}\right\\}_{\theta\in\Theta}$, a stochastic process
with index set $\Theta$. This is a family of random elements defined on a
probability space $(\Omega,\mathcal{F},\mathbb{P})$. More formally, we have a
mapping $X:\Theta\times\Omega\mapsto S$, and thus we can write
$X(\theta,\omega)$ so that the element is a function of $\theta$ (possibly
time or space) and $\omega$ is randomness. As we shall see in the next
section, we can write down a coordinate representation of the probability
space so that $\Omega=R_{\infty}\left(\times_{0}^{\infty}R\right)$ such that
$\omega=\left(x_{0},x_{1},...\right)\in R_{\infty}$. Here $\omega$ is a
typical trajectory or history of the process. We have the following natural
maps

$\displaystyle\text{given}\ \omega:\ t\mapsto X(t,\omega)=x_{t}$ (1.1)

The above map picks off the $t^{\prime}$th element and records a sample path
as we change $t$.

$\displaystyle\text{given}\ t:\ \omega\mapsto X(t,\omega)=X_{t}$ (1.2)

The above map produces a random variable that lives on $R$ (the state space)
which in other words is a real valued random variable. This perspective is
useful because it makes explicit what is the most distinctive feature of time
series analysis. The fact that in making inferences about the probability law
$\mathbb{P}$ that governs the process the evidence we have available takes the
form of a single history $\omega$. In practise, of course, the situation is
even worse: we only have a finite stretch of $\omega$ such as
$\left(x_{0},...,x_{n}\right)$ on which to base inference.

  1. (i)

Weak dependence: As we move along a history we accumulate new information
about the process, because events separated by long enough stretches of time
are nearly independent. This is what we mean by weak independence, asymptotic
independence.

  2. (ii)

Stationarity: The probability law remains unchanged as we move along in time.
Thus, there is enough stability about the law to ensure that information
within a given trajectory can be usefully employed to estimate some aspect of
the data generating mechanism. The assumption of strict stationarity implies
that the probability law is fixed and all marginal distributions are the same.

######  Theorem 1.

(Necessary and sufficient condition for ergodicity) Let
$\left(\Omega,\mathcal{F},\mathbb{P}\right)$ is a probability space and
$S:\Omega\times\Omega$ be a map on $\Omega$ and $S$ is ergodic iff

$\displaystyle\frac{1}{n}\sum_{k=0}^{n-1}\mathbb{P}\left(F\subset
S^{-k}G\right)\mapsto\mathbb{P}(F)\mathbb{P}(G),\ \ \ \forall\ \
F,G\in\mathcal{F}$ (1.3)

Proof. Suppose S is ergodic and take the random variable $1_{G}$. Then, by the
ergodic theorem,

$\displaystyle\frac{1}{n}\sum_{k=0}^{n-1}1_{g}\left(S^{k}\omega\right)\to
E(1_{G})=\int_{\omega}dP=P(G),$ (1.4)

######  Definition 1.

Let $\left\\{X_{t}\in\mathbb{N}\right\\}$, for $\mathbb{N}=0,1,2,...$ be a
stochastic process with mean function

$\displaystyle\mathbb{E}\left[X_{t}\right]=\mu,\ \ \ \text{for all}\ \
t\in\mathbb{N}$ (1.5)

Then, the sequence

$\displaystyle\bar{X}_{T}=\frac{1}{T}\sum_{t=1}^{T}X_{t},\ \ \ T=1,2,...$
(1.6)

is said to be ergodic if and only if

$\displaystyle\underset{T\to\infty}{\mathsf{lim}}\left(\bar{X}_{T}-\mu\right)=\underset{T\to\infty}{\mathsf{lim}}\mathbb{E}\left[\left|\bar{X}_{T}-\mu\right|^{2}\right]=0.$
(1.7)

######  Definition 2.

The autocovariance of a time series at some lag or interval, $k$, is defined
to be the covariance between $X_{t}$ and $X_{t+k}$ such that

$\displaystyle\gamma_{k}=\mathsf{Cov}\left(X_{t},X_{t+k}\right)=\mathbb{E}\big{[}\left(X_{t}-\mu\right)\left(X_{t+k}-\mu\right)\big{]}$
(1.8)

######  Definition 3.

The autocorrelation of a time series is the standardization of the
autocovariance of a time series relative to the variance of a time series, and
the autocorrelation at lag $k$, $\rho_{k}$, is bounded between $+1$ and $-1$
such that

$\displaystyle\rho_{k}=\frac{\mathbb{E}\big{[}\left(X_{t}-\mu\right)\left(X_{t+k}-\mu\right)\big{]}}{\sqrt{\mathbb{E}\left[(X_{t}-\mu)^{2}\right]\mathbb{E}\left[(X_{t+k}-\mu)^{2}\right]}}$
(1.9)

######  Remark 1.

Both the sample autocovariance (acf) and sample autocorrelation functions are
considered as measured of dependency for stationary time-series data. A simple
structural time series model, is the additive decomposition model. Usually, in
financial time series there are often changes which are long term. Thus, the
long term level of at time t is a stochastic process and is assumed to be a
function of present and past values of $\left\\{X_{t},X_{t-1},...\right\\}$.
At time $t$ there will also be a fluctuation component $f_{t}$ which
represents new behaviour at time additional to the existing previous level at
time $(t-1)$, which is assumed to be stationary, that is, none of its
statistical properties change with time.

The above aspects are considered to be the fundamental assumptions in time
series models. However, often these assumptions could vary depending on both
the econometric model under consideration as well as on the empirical data
application. For example, when modelling stock returns the presence of serial
correlation has an important interpretation in the financial economics
literature. Specifically, incorporating serial correlation in models of stock
prices and expected returns can provide additional evidence on the validity of
the efficient market hypothesis and explain other market anomalies.

Further stylized facts of financial returns include the presence of heavy-
tailedness and volatility clustering which implies that related econometric
assumptions require the existence of fourth moments for the underline
distribution function of the model innovations. Further assumptions can
include the presence of heteroscedastic errors by imposing a parametric GARCH
specification as well as other type of structures such as autoregressive
errors. Although the main assumption regarding the innovation sequence of the
the stationary AR$(1)$ time series model is that $u_{t}$ is an i.i.d sequence
with mean zero and known variance, this assumption can be further relaxed to
incorporate weakly or strongly dependent error sequences. By imposing a linear
process representation then we can introduce the notation of weakly dependent
errors.

Following the work of Granger and Andersen,  (1978) for a linear stationary
process it holds that

$\displaystyle\mathsf{Corr}\left(X^{2}_{t},X^{2}_{t-k}\right)=[\mathsf{Corr}\left(X_{t},X_{t-k}\right)]^{2}.$

for some time series $X_{t}$ across all $k$.

Therefore, departures from the above expression indicate the presence of
nonlinearity in time series. In particular, using the squared residuals from a
linear model we can apply a standard Box-Ljung Portmanteau test for serial
correlation (see, Lee et al.,  (1993)). In other words, the BLP test is
sensitive to departures from linearity in mean.

######  Definition 4.

A process $\left\\{X_{t}\in\mathbb{N}\right\\}$ is said to be a moving average
process of order $m$ if it can be expressed

$\displaystyle X_{t}=\sum_{j=0}^{m}c_{j}\varepsilon_{t-j}$ (1.10)

Thus, an autoregressive process of order $p$, provided that certain conditions
are satisfied concerning the roots of the associated polynomial, can be
represented by a moving average process of infinite order.

######  Remark 2.

Notice that many time series exhibit "long memory", which implies that the
autocorrelation function decays slowly with respect to the lag. In general
such a time series characteristic traditionally has been modelled in the
literature with unit roots (using nonstationary asymptotics), nonlinear
dynamics (or regime switching) as well as structural breaks.

###  1.2. Martingale Theorems

######  Theorem 2 (CLT for linear processes).

Consider the stationary linear process $X_{t}$ with $\left(c_{j}\right)$
satisfying

$\displaystyle\frac{1}{\sqrt{n}}\sum_{t=1}^{n}\varepsilon_{t}\to_{d}\mathcal{N}\left(0,\sigma^{2}\right)\
\ \ \text{as}\ \ n\to\infty,\ \ \text{with}\ \ \ \sum_{i=0}^{\infty}c_{i}\neq
0.$ (1.11)

Then, it holds that

$\displaystyle\frac{1}{\sqrt{n}}\sum_{t=1}^{n}X_{t}\to_{d}\mathcal{N}\left(0,\omega^{2}\right)\
\ \ \text{as}\ n\to\infty.$ (1.12)

######  Theorem 3 (Martingale CLT).

Let $\left\\{\xi_{n,j}:1\leq j\leq k_{n},n\in\mathbb{N}\right\\}$ be an
$\mathcal{F}_{n,j}$ martingale difference array that satisfies the Lindeberg
condition. Then, it holds that

$\displaystyle\sum_{j=1}^{k_{n}}\mathbb{E}\left[\xi^{2}_{n,j}|\mathcal{F}_{n,j-1}\right]\overset{p}{\to}\eta^{2}>0,\
\ \ \text{as}\ n\to\infty,$ (1.13)

which implies that
$\sum_{j=1}^{k_{n}}\xi_{n,j}\to_{d}Z\sim\mathcal{N}(0,\eta^{2})$.

######  Remark 3.

Notice that the sample mean of the linear process $X_{t}$ can be written as

$\displaystyle\frac{1}{\sqrt{n}}\sum_{t=1}^{n}X_{t}=\tilde{c}_{-1}\frac{1}{\sqrt{n}}\sum_{t=1}^{n}\varepsilon_{t}-\frac{1}{\sqrt{n}}\sum_{t=1}^{n}\Delta\tilde{\varepsilon}_{t}$
$\displaystyle=\tilde{c}_{-1}\frac{1}{\sqrt{n}}\sum_{t=1}^{n}\varepsilon_{t}-\frac{1}{\sqrt{n}}\left(\tilde{\varepsilon}_{n}+\tilde{\varepsilon}_{0}\right)$
$\displaystyle=\tilde{c}_{-1}\frac{1}{\sqrt{n}}\sum_{t=1}^{n}\varepsilon_{t}+\mathcal{O}_{p}\left(\frac{1}{\sqrt{n}}\right).$

In other words, as long as the sequence $\varepsilon_{t}$ satisfies a CL(e.g.,
when $\varepsilon_{t}$ are i.i.d $(0,\sigma^{2})$, it follows

$\displaystyle\frac{1}{\sqrt{n}}\sum_{t=1}^{n}\varepsilon_{t}\to_{d}\mathcal{N}\left(0,\sigma^{2}\right)\Rightarrow\frac{1}{\sqrt{n}}\sum_{t=1}^{n}X_{t}\to_{d}\mathcal{N}\left(0,\omega^{2}\right).$
(1.14)

where we have that

$\displaystyle\omega^{2}=\tilde{c}_{-1}\sigma^{2}=\left(\sum_{j=0}^{\infty}c_{j}\right)^{2}\sigma^{2}.$
(1.15)

is called the long run variance of $X_{t}$. The term emphasises the fact that,
under short memory correlation, the variance of $X_{t}$ is

$\displaystyle\mathbb{E}\left(X_{t}^{2}\right)=\underset{n\to\infty}{\mathsf{plim}}\frac{1}{n}\sum_{t=1}^{n}X_{t}^{2}\neq\omega^{2}=\underset{n\to\infty}{\mathsf{lim}}\
\mathbb{E}\left(\frac{1}{\sqrt{n}}\sum_{t=1}^{n}X_{t}\right)^{2},$ (1.16)

Thus, the probability limit of the WLLN for the sample variance is not equal
to the asymptotic variance of the CLT as in the case of i.i.d random
variables. Moreover, it can be shown that
$\omega^{2}=\sum_{h=-\infty}^{\infty}\gamma(h)$,

$\displaystyle\gamma(h)=\mathbb{E}\left(X_{t}X_{t-h}\right)=\sigma^{2}\sum_{j=0}^{\infty}c_{j}c_{j+|h|},\
\ \ h\in\mathbb{Z}.$ (1.17)

######  Example 1.

(CLT for sums of linear processes under SM) Consider the linear process
$X_{t}=\sum_{j=0}^{\infty}c_{j}\varepsilon_{t-j}$,

$\displaystyle\sum_{j=0}^{\infty}|c_{j}|<\infty\ \ \ \text{and}\ \ \
\mathbb{E}\left(\varepsilon_{t}^{2}\right)=\sigma^{2}\in(0,\sigma)\ \ \
\text{for all}\ t.$ (1.18)

Then, it holds that

$\displaystyle\left\lVert\frac{1}{\sqrt{n}}\sum_{=1}^{n}-\left(\sum_{j=0}^{n-1}\frac{1}{\sqrt{n}}\varepsilon_{t}\right)\right\rVert_{L_{1}}\to
0\ \ \text{as}\ n\to\infty.$ (1.19)

To see this, we write

$\displaystyle\frac{1}{\sqrt{n}}\sum_{t=1}^{n}X_{t}=\frac{1}{\sqrt{n}}\sum_{t=1}^{n}\sum_{j=0}^{t-1}c_{j}\varepsilon_{t-j}+\frac{1}{\sqrt{n}}\sum_{t=1}^{n}\sum_{j=t}^{\infty}c_{j}\varepsilon_{t-j}=:S_{n1}+S_{n1}.$
(1.20)

######  Example 2.

$\displaystyle X_{t}=\rho X_{t-1}\epsilon_{t},\ \ \
\epsilon_{t}\sim(0,\sigma^{2}).$ (1.21)

For a stationary process we assume that the first two moments are constant,
such that $\mathbb{E}(X_{t})=\mu$ and $\mathsf{Var}(X_{t})=\sigma^{2}$.
Moreover, the covariance depends on $h$. Consider the martingale transform

$\displaystyle M_{n}=\sum_{t=1}^{n}X_{t-1}\varepsilon_{t}$ (1.22)

Moreover, it holds that $\mathbb{E}\left(M_{n}^{2}\right)=O(n)$. Thus,
consider

$\displaystyle\frac{1}{\sqrt{n}}\sum_{t=1}^{n}X_{t-1}\varepsilon_{t}=\sum_{t=1}^{n}\xi_{n,t}\
\ \ \text{with}\ \ \ \xi_{n,t}=\frac{1}{\sqrt{n}}X_{t-1}\varepsilon_{t}$
(1.23)

First we compute the probability limit of the conditional variance for the
martingale CLT

$\displaystyle\sum_{t=1}^{n}\mathbb{E}\left(\xi^{2}_{n,t}|\mathcal{F}_{t-1}\right)=\frac{1}{n}\sum_{t=1}^{n}X_{t-1}^{2}\mathbb{E}\left(\varepsilon^{2}_{t}|\mathcal{F}_{t-1}\right)=\frac{\sigma^{2}}{n}\sum_{t=1}^{n}X_{t-1}^{2}\overset{p}{\to}\sigma^{2}\gamma_{X}(0)=\frac{\sigma^{4}}{1-\rho^{2}}\
\ \ \text{as}\ \ n\to\infty.$ (1.24)

by $\mathcal{F}_{t-1}$ measurability of $X_{t-1}$ and WLLN for the sample ACF
of linear processes. The conditional variance probability limit is given by
$\tilde{\sigma}=\frac{\displaystyle\sigma^{4}}{\displaystyle 1-\rho^{2}}$,
which implies a distribution limit by the MCLT:

$\displaystyle\sum_{t=1}^{n}\xi_{n,t}=\frac{1}{\sqrt{n}}\sum_{t=1}^{n}X_{t-1}\varepsilon_{t}\overset{d}{\to}\mathcal{N}\left(0,\frac{\sigma^{4}}{1-\rho^{2}}\right)$
(1.25)

To prove that indeed the above convergence in distribution holds, it remains
to prove the Lindeberg condition for the CLT such that

$\displaystyle\mathcal{L}_{n}\left(\delta\right)=\frac{1}{n}\sum_{t=1}^{n}\mathbb{E}\big{[}X_{t-1}^{2}\varepsilon^{2}_{t}\mathbf{1}\left\\{|X_{t-1}||\varepsilon_{t}|>\sqrt{n\delta}\right\\}\big{]}\to
0.$ (1.26)

Assuming that the above result can be proved then, we can conclude that using
the Slutsky theorem, the asymptotic distribution of the OLS estimator is given
by

$\displaystyle\sqrt{n}\left(\hat{\rho}_{n}-\rho\right)=\frac{\displaystyle\frac{1}{\sqrt{n}}\sum_{t=1}^{n}X_{t-1}\varepsilon_{t}}{\displaystyle\frac{1}{n}\sum_{t=1}^{n}X_{t-1}^{2}}\overset{d}{\to}\frac{\mathcal{N}\left(0,\frac{\sigma^{4}}{1-\rho^{2}}\right)}{\frac{\sigma^{2}}{1-\rho^{2}}}=_{d}\mathcal{N}\left(0,1-\rho^{2}\right).$
(1.27)

###  1.3. Covariance Matrix Estimation

####  1.3.1. Long-Run variance

Denote with $\bar{w}=n^{-1}\sum_{t=1}^{n}w_{t}$. When the series is strict
white noise (that is, zero mean iid), then the central limit theorem implies
that $n^{1/2}\bar{w}\overset{d}{\to}\mathcal{N}(0,V_{w})$, where
$V_{w}=\mathsf{Var}(w)$. Furthermore, when the series $w_{t}$ exhibit serial
dependence but is a martingale difference with respect to its own past then
this convergence in distribution still holds. However, if the series $w_{t}$
exhibits serial correlation, the asymptotic normality still holds, but the
asymptotic variance becomes

$\displaystyle V_{w}=\underset{n\to\infty}{\mathsf{lim}}\
\mathsf{Var}\left(\sqrt{n}\bar{w}\right),$ (1.28)

which differs from the variance var$(w)$. Since we have that,

$\displaystyle V_{w}=\underset{n\to\infty}{\mathsf{lim}}\
\mathsf{Var}\left(\sqrt{n}\bar{w}\right)=\underset{n\to\infty}{\mathsf{lim}}\
\mathbb{E}\left[\left(\sqrt{n}\bar{w}\right)\left(\sqrt{n}\bar{w}\right)^{\top}\right]=\underset{n\to\infty}{\mathsf{lim}}\
\mathbb{E}\left[\frac{1}{n}\sum_{t=1}^{n}\sum_{s=1}^{n}\bar{w}_{t}\bar{w}_{s}^{\top}\right]=\underset{n\to\infty}{\mathsf{lim}}\
\frac{1}{n}\sum_{t=1}^{n}\sum_{s=1}^{n}\mathbb{E}\big{[}\bar{w}_{t}\bar{w}_{s}^{\top}\big{]}.$

Moreover, we denote the autocovariance of $w_{t}$ at delay $j$ by
$\rho(j)=\mathbb{E}\big{[}w_{t}w_{t-j}^{\top}\big{]}$. Using that
$\rho(-j)=\rho(j)^{\top}$ and the Toeplitz lemma, we obtain

$\displaystyle V_{w}=\underset{n\to\infty}{\mathsf{lim}}\
\frac{1}{n}\left[n\rho(0)+\sum_{j=1}^{n-1}(n-j)\big{(}\rho(j)+\rho(j)^{\top}\big{)}\right]=\underset{n\to\infty}{\mathsf{lim}}\
\left[n\rho(0)+\sum_{j=1}^{n-1}\left(1-\frac{j}{n}\right)\big{(}\rho(j)+\rho(j)^{\top}\big{)}\right]=\sum_{j=-\infty}^{\infty}\rho(j).$

Notice that the above expression for $V_{w}$ is termed the long-run variance.
If instead of zero $\mathbb{E}[w_{t}]$ we consider possibly non-zero
$\mathbb{E}\big{[}m(w_{t})\big{]}$ for some function $m(.)$, the long-run
variance

$\displaystyle
V_{m}=\underset{n\to\infty}{\mathsf{lim}}\mathsf{Var}\left(\sqrt{n}\sum_{t=1}^{n}m(w_{t})\right)$
(1.29)

has the same representation as before, but the autocovariance of $m(w_{t})$ at
delay $j$ takes the form
$\rho(j):=\mathbb{E}\big{[}\big{(}m(w_{t})-\mathbb{E}\left[m(w_{t})\right]\big{)}\big{(}m(w_{t-j})-\mathbb{E}\left[m(w_{t})\right]\big{)}^{\prime}\big{]}$.
Then, the long-run covariance between, say, $m(w_{t})$ and $\mu(w_{t})$ is
defined similarly such that

$\displaystyle C_{m,\mu}$ $\displaystyle:=\underset{n\to\infty}{\mathsf{lim}}\
\mathsf{Cov}\left(\frac{1}{\sqrt{n}}\sum_{t=1}^{n}m(w_{t}),\frac{1}{\sqrt{n}}\sum_{t=1}^{n}\mu(w_{t})\right)=\sum_{j=-\infty}^{\infty}\gamma(j).$
$\displaystyle\gamma(j)$
$\displaystyle:=\mathbb{E}\big{[}\big{(}m(w_{t})-\mathbb{E}\left[m(w_{t})\right]\big{)}\big{(}\mu(w_{t-j})-\mathbb{E}\left[\mu(w_{t-j})\right]\big{)}^{\prime}\big{]}$

is the cross-covariance between $m(w_{t})$ and $\mu(w_{t})$ at delay $j$.
Consistent estimation of the long-run variance is not straightforward.
Specifically, each autocovariance $\rho(j)$ can be consistently estimated by

$\displaystyle\widehat{\rho(j)}=\frac{1}{n}\sum_{t=j+1}^{n}\left(m(w_{t})-\frac{1}{n}\sum_{s=1}^{n}m(w_{s})\right)\left(m(w_{t-j})-\frac{1}{n}\sum_{s=1}^{n}m(w_{s})\right)$
(1.30)

for non-negative $j$, and by
$\widehat{\rho(j)}=\left(\widehat{\rho(-j)}\right)^{\prime}$ for negative $j$.
A natural estimator of $V_{m}$ is

$\displaystyle\widehat{V}^{0}_{m}=\sum_{j=-(n-1)}^{n-1}\widehat{\rho(j)}.$
(1.31)

However, this estimator is not consistent because it is constructed using
covariance estimates whose number increases proportionately with the sample
size. Therefore, a possible solution is to truncate the summation at $-p$ and
$p$, where $p<n-1$ such that

$\displaystyle\widehat{V}_{m}^{HH}=\sum_{j=-p}^{p}\widehat{\rho(j)}.$ (1.32)

Due to an ill-posed estimation problem, since the positive-definiteness
property of the sample covariance matrix is not guaranteed, NW proposed the
following modified version of the covariance matrix:

$\displaystyle\widehat{V}_{m}^{NW}=\sum_{j=-p}^{p}\left(1-\frac{|j|}{p+1}\right)\widehat{\rho(j)}.$
(1.33)

where the weights ensure that this NW estimator is positive semi-definite. The
use of the long-run variance estimators is widespread in estimation and
inference for models with dependent data. In particular, $V_{n}(\bar{\theta})$
is a consistent estimator of the long-run variance of the moment function
$m(w_{t},\theta_{0})$

Define with

$\displaystyle V_{m}=\underset{n\to\infty}{\mathsf{lim}}\
\mathsf{var}\big{(}\sqrt{n}m_{n}(\theta_{0})\big{)}=\sum_{j=-\infty}^{\infty}\mathbb{E}\big{[}m(w_{t},\theta_{0})m(w_{t-j},\theta_{0})^{\top}\big{]}.$
(1.34)

Then, the feasible optimal weighting matrix is then an estimate of $V_{m}$
which also uses a preliminary estimator $\bar{\theta}$ in place of
$\theta_{0}$. Similarly, for the OLS estimator in a linear time series
regression
$y_{t}=x_{t}^{\prime}\theta_{0}+e_{t},\mathbb{E}\big{[}e_{t}|x_{t}\big{]}=0$,
the asymptotic variance of $\hat{\theta}$ can be estimated using the long-run
variance estimator of $x_{t}\hat{e}_{t}$, where $\hat{e}_{t}$ are the OLS
residuals of the model. Moreover because, the long-run variance estimators
automatically take into account the serial correlation and conditional
heteroscedasticity in $e_{t}$, these estimators are often referred to as HAC
estimators (see, Newey and West,  (1986), Andrews,  (1991) and Andrews and
Monahan,  (1992)).

####  1.3.2. Consistent Covariance Matrix Estimation for Linear Processes

Consistency of kernel estimators of the long-run covariance matrix of a linear
process is established under weak moment and memory conditions.

$\displaystyle V_{t}=\sum_{j=0}^{\infty}C_{j}e_{t-j}.$ (1.35)

We consider the estimation of
$\Omega=\displaystyle\underset{T\to\infty}{\text{lim}}\frac{1}{T}\sum_{t=1}^{T}\sum_{s=1}^{T}\mathbb{E}\left(V_{t}V_{t}^{\prime}\right)$,
the long-run covariance matrix of $V_{t}$.

Consistency of kernel estimators of $\Omega$ is established under weak
conditions on $\left\\{C_{j}:j\geq 1\right\\}$ and
$\left\\{e_{t}:t\in\mathbb{Z}\right\\}$. We consider a kernel estimator of the
form

$\displaystyle\hat{\Gamma}_{T}=T^{-1}\sum_{t=s}^{T}\sum_{s=1}^{t-1}k\left(\frac{|t-s|}{b_{T}}\right)V_{t}V_{s}^{\prime},$
(1.36)

where $k(.)$ is a measurable kernel function and $\left\\{b_{T}:T\geq
1\right\\}$ is a sequence of bandwidth parameters. The corresponding estimator
of $\Omega$ is given by

$\displaystyle\hat{\Omega}_{T}=T^{-1}\sum_{t=1}^{T}\sum_{s=1}^{T}\left(\frac{|t-s|}{b_{T}}\right)V_{t}V_{s}^{\prime},$
(1.37)

which can be written as
$\hat{\Gamma}_{T}+\hat{\Gamma}_{T}^{\prime}+\hat{\Sigma}_{T}$, where
$\hat{\Sigma}_{T}=\sum_{t=1}^{T}V_{t}V_{t}^{\prime}$. Notice that
$\sqrt{n}\left(\hat{\theta}_{n}-\theta\right)=\mathcal{O}_{p}(1)$ implies that
$\hat{\theta}_{n}\overset{p}{\to}\theta$ with a $\sqrt{n}-$rate of
convergence. The order of convergence indicates the rate at which the distance
of the sample estimator to the population parameter get smaller as the sample
size increases, $n\to\infty$, i.e.,
$\hat{\theta}_{n}=\mathcal{O}_{p}(\frac{1}{\sqrt{n}})$. According to Kiefer et
al.,  (2000) and Kiefer and Vogelsang,  (2002),

$\displaystyle\sqrt{T}\left(\hat{\beta}-\beta_{0}\right)\Rightarrow
Q^{-1}\Lambda
W_{k}(1)\sim\mathcal{N}\big{(}0,Q^{-1}\Lambda\Lambda^{\prime}Q^{-1}\big{)}\equiv\mathcal{N}\big{(}0,Q^{-1}\Omega
Q^{-1}\big{)}=\mathcal{N}(0,V).$

In other words, the asymptotic distribution is a $k-$variate normal
distribution with mean 0 and variance-covariance matrix $V=Q^{-1}\Omega
Q^{-1}$. Then, the asymptotic distribution of $\hat{\beta}$ can be used to
test hypothesis about $\beta$. However, to do this an estimate of $V$ is
required. In particular, a natural estimate of $Q$ is

$\displaystyle\widehat{Q}=\frac{1}{T}\boldsymbol{F}^{\prime}(\hat{\beta})W\boldsymbol{F}^{\prime}(\hat{\beta})$
(1.38)

Therefore, $\Omega$ can be estimated by a HAC estimator, $\widehat{\Omega}$.
Letting $\hat{u}_{t}$ be the residuals of the transformed model, the HAC
estimate would use $\hat{v}_{t}=\tilde{F}_{t}(\hat{\beta})\hat{u}_{t}$ to
estimate nonparametrically the spectral density of $v_{t}$ at frequency 0, and
hence $\Omega$. A typical estimator takes the form

$\displaystyle\widehat{\Omega}$
$\displaystyle=\sum_{j=-(T-1)}^{T-1}k\left(\frac{j}{s(T)}\right)\widehat{\Gamma}_{j}.$
(1.39) $\displaystyle\widehat{\Gamma}_{j}$
$\displaystyle=\frac{1}{T}\sum_{t=j+1}^{T}\hat{v}_{t}\hat{v}_{t-j}^{\prime},\
\ \text{for}\ \ j\geq 0.$ (1.40) $\displaystyle\widehat{\Gamma}_{j}$
$\displaystyle=\frac{1}{T}\sum_{t=-j+1}^{T}\hat{v}_{t}\hat{v}_{t-j}^{\prime},\
\ \text{for}\ \ j\geq 0.$ (1.41)

where $k(x)$ is a kernel function satisfying $k(x)=k(-x)$, $k(0)=1$,
$|k(x)|\leq 1$ continuous at $x=0$ and
$\int_{-\infty}^{\infty}k^{2}(x)dx<\infty$. Moreover, the tuning parameter,
$s(T)$, is often called the truncation lag or bandwidth. A typical condition
for consistency of $\widehat{\Omega}$ is that $s(T)\to\infty$ as $T\to\infty$
but $s(T)/T\to 0$. Therefore, to test hypotheses about $\beta$ in the standard
approach, $\widehat{V}=\widehat{Q}^{-1}\widehat{\Omega}\widehat{Q}^{-1}$ is
used to transform $\sqrt{T}\left(\hat{\beta}-\beta_{0}\right)$ to obtain

$\displaystyle\widehat{V}^{-\frac{1}{2}}\sqrt{T}\left(\hat{\beta}-\beta_{0}\right)\Rightarrow\Omega^{-\frac{1}{2}}QQ^{-1}\Lambda
W_{k}(1)=W_{k}(1)\sim\mathcal{N}\big{(}0,\boldsymbol{I}_{k}\big{)}.$

In a new approach Hong et al.,  (2023) (see also Sun et al.,  (2022)), use a
similar method, but transform $\sqrt{T}\left(\hat{\beta}-\beta_{0}\right)$ in
such a manner that the asymptotic distribution no longer depends on unknown
parameters. The essential difference between the two approaches is that their
approach does not require an explicit estimate of $\Omega$ and takes the
additional sampling variation associated with not knowing the covariance
matrix into account in the asymptotic approximation. On the other hand, HAC
estimates treat the variance-covariance matrix as known asymptotically and
ignore the impact of finite-sample variability from $\widehat{\Omega}$ on the
distribution of test statistics. Thus, a practical limitation of the HAC
approach is that to obtain HAC estimates, a truncation lag for a spectral
density estimator must be chosen. Although asymptotic theory dictates the rate
at which the truncation lag much increase as the sample size grows, no
concrete guidance is provided. Consider the following scaled partial sum
empirical process:

$\displaystyle T^{-1/2}\widehat{S}_{\lfloor
Tr\rfloor}=T^{-1/2}\sum_{t=1}^{\lfloor
Tr\rfloor}\hat{v}_{t}=T^{-1/2}\sum_{t=1}^{\lfloor
Tr\rfloor}\tilde{F}_{t}\left(\hat{\beta}\right)\hat{u}_{t}.$ (1.42)

Their approach provides an elegant solution to this practical problem by
avoiding a consistently estimate of the variance-covariance matrix and thus
removing the need to choose a truncation lag. In other words, a data-dependent
transformation is applied to the NLS estimates of the parameters of interest.
This transformation is chosen such that it ensures that the asymptotic
distribution of the transformed estimator does not depend on nuisance
parameters. The transformed estimator can then be used to construct a test for
general hypotheses on the parameters of interest.

###  1.4. Continuous Asymptotics

Before considering the continuous time asymptotics we consider the following
example which can be found on page 328 in the book of Csorgo and Horvath,
(1997).

######  Example 3.

$\displaystyle X_{t}=\rho X_{t-1}+\epsilon_{t},$ (1.43)

Let $|\rho|\neq 1$, then the partial sums of the residuals
$\displaystyle\sum_{t=1}^{n}\hat{\epsilon}_{t}$ have asymptotic behaviour
similar to the partial sums of the i.i.d errors. On the other hand when the
autocorrelation coefficient is within the unit root boundary, i.e.,
$|\rho|=1$, then the limit of the suitably normalized residual sum
$\displaystyle\sum_{t=1}^{n}\hat{\epsilon}_{t}$ can be proved that is not a
Wiener process. The particular result proved in various studies appeared in
the literature is of particular importance when considering persistence
processes with predictive regression models.

Consider $X_{t}=S(t)$ the corresponding partial sum process such that
$S(t)=\sum_{j=1}^{t}\epsilon_{j}$. Furthermore, consider that $\hat{\rho}$ is
the maximum likelihood estimator for $\rho=1$, then we have that

$\displaystyle\hat{\rho}-1=\frac{\displaystyle\sum_{t=1}^{T}\epsilon_{j}S(t-1)}{\displaystyle\sum_{t=1}^{T}S^{2}(t-1)}.$
(1.44)

Hence, we have that the partial sum process is given by

$\displaystyle\sum_{j=1}^{t}\hat{\epsilon}_{j}=\big{(}1-\hat{\rho}\big{)}\sum_{j=1}^{k}S(j-1)+\sum_{j=1}^{k}\epsilon_{j}=S(k)-\frac{\displaystyle\sum_{j=1}^{n}\epsilon_{j}S(j-1)}{\displaystyle\sum_{j=1}^{n}S^{2}(j-1)}\sum_{j=1}^{k}S(j-1).$

Since, we have that

$\displaystyle\frac{1}{\sqrt{n}\uptau}S(nt)\to W(t),\ \ \text{where}\ \
\big{\\{}W(t),0\leq t\leq 1\big{\\}}\ \text{is a Wiener process}.$ (1.45)

Consider a triangular array of random variables
$\left\\{\left\\{y_{nt}\right\\}_{t=1}^{T_{n}}\right\\}_{n=1}^{\infty}$. The
triangular array then provides a formal framework within which $h$ may vary
and by means of which we may investigate limiting behaviour as $h_{n}\to 0$.
Let $S_{ni}=\sum_{j=1}^{i}u_{nj}$ such that $1\leq i\leq T_{n}$ with
$S_{n0}=0$.

Furthermore, we form the random function as below

$\displaystyle Y_{n}$ $\displaystyle=\sigma^{-1}S_{ni-1},\ \ (i-1)/T_{n}\leq
t\leq i/T_{n},\ \ i=1,...,T_{n}$ (1.46) $\displaystyle Y_{n}(1)$
$\displaystyle=\sigma^{-1}S_{nT_{n}}$ (1.47)

As $n\to\infty,Y_{n}(r)$ converges weakly to a constant multiple of a standard
Wiener process.

######  Lemma 1.

If the following conditions hold:

  * (a)

$\left\\{\left\\{u_{nt}\right\\}\right\\}$ is a triangular array of i.i.d
$(0,\sigma^{2}h_{n})$,

  * (b)

$T_{n}\in Z^{+}$, $T_{n}\to\infty$, and $h_{n}\to 0$ as $n\to\infty$ in such a
way that the product $T_{n}h_{n}=N>0$ remains constant,

then $Y_{n}(r)\Rightarrow N^{1/2}W(r)$ as $n\to\infty$ where $W(r)$ as
$n\to\infty$, where $W(r)$ is a standard Wiener process.

We define with

$\displaystyle\hat{\alpha}_{n}=\frac{\displaystyle\sum_{t=1}^{T_{n}}y_{nt}y_{nt-1}}{\displaystyle\sum_{t=1}^{T_{n}}y_{nt-1}^{2}}$
(1.48)

Then, we have the t-statistic given by

$\displaystyle
t_{\alpha_{n}}=\left(\sum_{t=1}^{T_{n}}y_{nt-1}^{2}\right)\left(\hat{\alpha}_{n}-1\right)/s_{n}$
(1.49)

where an unbiased estimator for the variance is given by

$\displaystyle
s_{n}=\left\\{T_{n}^{-1}\sum_{t=1}^{T_{n}}\left(y_{nt}-\hat{\alpha}_{n}y_{nt-1}\right)^{2}\right\\}^{1/2}$
(1.50)

Furthermore, we have that

$\displaystyle\int_{0}^{1}X_{T}(r)^{2}dr\equiv\int_{0}^{1}W(r)^{2}dr+O_{p}(T^{-1})$
(1.51)

and this expansion may be verified directly by developing an expansion for the
characteristic function of $\int_{0}^{1}X_{T}(r)^{2}dr$.

######  Theorem 4.

If $y_{t}$ is generated from the random walk with $\alpha=1$ and initial value
$y_{0}=0$ and if the $u_{i}$ are i.i.d $\mathcal{N}(0,1)$, then we have that

$\displaystyle
T\left(\hat{\alpha}-1\right)\equiv\frac{\displaystyle(1/2)\left(W(1)^{2}-1\right)-\left(1/\sqrt{2T}\right)\xi}{\displaystyle\int_{0}^{1}W(r)^{2}dr}$
(1.52)

where $W(r)$ is a standard Wiener process and $\xi$ is $\mathcal{N}(0,1)$ and
independent of $W(r)$.

###  1.5. The Ornstein–Uhlenbeck process

Hazard rates processes are functionals of piecewise-continuous Gaussian
martingales, such as the OU processes and the pure jump process.

##### Modelling the item state by a stochastic process

$\displaystyle
X(n+1)-X(n)=\sigma\left(X(n)\right)\epsilon_{n}+\mu\left(X(n)\right)h,$ (1.53)

The resulting stochastic differential equation is given by

$\displaystyle dX(t)=\sigma X(t)d\gamma(t)+\mu\left[X(t)\right]dt,$ (1.54)

##### Modelling item failure rate by a stochastic process

Dynamic environments exert random stresses on the item. An approach for
modelling the lifelength of items in a dynamic environment is to describe the
item’s failure rate by a stochastic process. Moreover, there is an interesting
relation between a hazard rate process and a doubly stochastic Poisson process
(known as the Cox process).

Let T denote the lifelength of the item and let $\left\\{\lambda(s),s\leq
0\right\\}$ be a non-negative, real-valued, right-continuous process.Then,
$\left\\{\lambda(s),s\leq 0\right\\}$ is said to be the hazard rate process of
T if

$\displaystyle\mathbb{P}\big{(}T\leq t|\lambda(s),0\leq s\leq
t\big{)}=\text{exp}\left\\{-\int_{0}^{t}\lambda(s)ds\right\\}$ (1.55)

Consequently,

$\displaystyle\mathbb{P}\big{(}T\leq
t\big{)}=\mathbb{E}\left[\text{exp}\left\\{-\int_{0}^{t}\lambda(s)ds\right\\}\right],t\geq
0$ (1.56)

###  1.6. Higher-Order Autoregressive Processes

####  1.6.1. AR(p) Time Series Regression

An autoregressive process of order AR$(p)$ is defined by

$\displaystyle X_{t}=\theta_{1}X_{t-1}+...+\theta_{p}X_{t-p}+\varepsilon_{t}$
(1.57)

for any $t\geq 1$ or equivalently, in a compact form we have that
$X_{t}=\theta^{\top}\Phi_{t-1}+\varepsilon_{t}$, where
$\theta=\big{(}\theta_{1},...,\theta_{p}\big{)}^{\top}$ is a vector parameter,
$\Phi_{0}$ is an arbitrary initial random vector and
$\Phi_{t}=\big{(}X_{t},...,X_{t-p+1}\big{)}^{\top}$ and $\varepsilon_{t}$ is a
strong white noise having a finite variance $\sigma^{2}$. Then, the
corresponding characteristic polynomial is defined by

$\displaystyle\Theta(z)=1-\theta_{1}z-...-\theta_{p}z^{p}\ \ \ \text{and}\ \ \
\Phi_{t}=C_{\theta}\Phi_{t-1}+E_{t},$ (1.58)

where $E_{t}=\big{(}\varepsilon_{t},0,...,0\big{)}^{\top}$ is a
$p-$dimensional noise. Notice that it is well-known that the stability of this
$p-$dimensional process is closely related to the eigenvalues of the companion
matrix that we will denote and arrange in increasing order as below

$\displaystyle\rho(C_{\theta})=|\lambda_{1}|\geq|\lambda_{2}|\geq...\geq|\lambda_{p}|.$
(1.59)

####  1.6.2. The Autoregressive and Moving Average Processes (ARMA)

A stochastic process, or time series, can be repeated as the output resulting
from a white noise input, $\varepsilon_{t}$,

$\displaystyle
X_{t}=\varepsilon_{t}+\phi_{1}\varepsilon_{t-1}+\phi_{2}\varepsilon_{t-2}+...\Rightarrow
X_{t}=\varepsilon_{t}+\sum_{j=1}^{\infty}\phi_{j}\varepsilon_{t-j}$

where $X_{t}$ is considered to be the output series after demeaning e.g.,
$X_{t}=\tilde{X}_{t}-\mu$. In other words, the general linear process
representation allows to represent the output of a time series, as a function
of the current and previous value of the white noise process,
$\varepsilon_{t}$, which may be represented as a series of shocks. The
autocorrelation function of a linear process is given by

$\displaystyle\gamma_{k}=\sigma_{\varepsilon}^{2}\sum_{j=0}^{\infty}\phi_{j}\phi_{j+k}$
(1.60)

Thus, the backward shift operator, $B$, is defined as $BX_{t}=X_{t-1}$ and
$B^{j}X_{t}=X_{t-j}$. The autocorrelation generating function may be written
as $\gamma(B)=\sum_{j=-\infty}^{+\infty}\gamma_{k}B^{k}$. Therefore, the
strict stationarity assumption holds if and only if the $\phi$ weights of a
linear process must satisfy that $\phi(B)$ converges on or lies within the
unit circle. Then, an autoregressive, AR, model, the current value of the time
series and a random shock $\varepsilon_{t}$, such that

$\displaystyle
X_{t}=\phi_{1}X_{t-1}+\phi_{2}X_{t-2}+...+\phi_{p}X_{t-p}+\varepsilon_{t}$
(1.61)

Therefore, the autoregressive operator of order $P$ is given by

$\displaystyle\phi(B)=1-\phi_{1}B^{2}-\phi_{2}B^{2}-...-\phi_{p}B^{p}.$ (1.62)

######  Example 4.

Consider the following $m-$dimensional vector ARMA model (see, the study of
Grønneberg and Holcblat,  (2019))

$\displaystyle\boldsymbol{\Phi}(B)\boldsymbol{X}_{t}=\boldsymbol{\Theta}(B)\boldsymbol{\epsilon}_{t},$
(1.63)

where $B$ is the back-shift operator, and $\boldsymbol{\Phi}(B)$ and
$\boldsymbol{\Theta}(B)$ are defined as below

$\displaystyle\boldsymbol{\Phi}(B)=\boldsymbol{I}-\sum_{i=1}^{p}\boldsymbol{\Phi}_{i}B^{i},\
\ \ \ \
\boldsymbol{\Theta}(B)=\boldsymbol{I}+\sum_{j=1}^{q}\boldsymbol{\Theta}_{j}B^{j}$
(1.64)

where $\boldsymbol{\Phi}_{i}$ and $\boldsymbol{\Theta}_{j}$ are $(m\times m)$
matrices with $1\leq i\leq p$ and $1\leq j\leq q$. Therefore, we denote the
unknown parameters such that
$\boldsymbol{\beta}=\mathsf{vec}\big{(}\boldsymbol{\Phi}_{1},...,\boldsymbol{\Phi}_{p},\boldsymbol{\Theta}_{1},...,\boldsymbol{\Theta}_{q}\big{)}$
and its true parameters by

$\displaystyle\boldsymbol{\beta}_{0}=\mathsf{vec}\big{(}\boldsymbol{\Phi}_{01},...,\boldsymbol{\Phi}_{0p},\boldsymbol{\Theta}_{01},...,\boldsymbol{\Theta}_{0q}\big{)}$
(1.65)

Furthermore, assume that $\boldsymbol{\beta}\in\mathcal{C}$, a compact
subspace in $\mathbb{R}^{(p+q)m^{2}}$, and $\boldsymbol{\beta}_{0}$ is an
interior point in $\mathcal{C}$.

######  Assumption 1.

All roots of $\mathsf{det}\big{[}\boldsymbol{\Phi}(z)\big{]}=0$ and all the
roots of $\mathsf{det}\big{[}\boldsymbol{\Theta}(z)\big{]}=0$ lie outside the
unit circle.

In particular, the above assumption is a sufficient condition for stationarity
and invertability, and it implies that the time series
$\left\\{\boldsymbol{X}_{t}\right\\}$ has the following representation:

$\displaystyle\boldsymbol{X}_{t}=\boldsymbol{\Psi}(B)\boldsymbol{\epsilon}_{t},$
(1.66)

where
$\boldsymbol{\Psi}(z):=\boldsymbol{\Phi}^{-1}(z)\boldsymbol{\Theta}(z)=\boldsymbol{I}+\sum_{j=1}^{\infty}\boldsymbol{\Psi}_{j}z^{j}$
with $\boldsymbol{\Psi}_{i}=\mathcal{O}_{p}\left(\rho^{i}\right)$ for some
$\rho\in(0,1)$. Then, the periodgram function of
$\left\\{\boldsymbol{X}_{1},...,\boldsymbol{X}_{n}\right\\}$ is defined as
below

$\displaystyle\boldsymbol{I}_{x}=\boldsymbol{J}_{x}(\lambda)\boldsymbol{J}^{\prime}_{x}(-\lambda)=\sum_{|k|<n}\boldsymbol{\Gamma}_{X}(k)e^{-i\lambda
k},$ (1.67)

where $\lambda\in[-\pi,\pi]$ and
$\boldsymbol{J}_{x}(\lambda)=\sum_{t=1}^{n}\boldsymbol{X}_{t}e^{-i\lambda
t}/\sqrt{n}$, and for $k\geq 0$,

$\displaystyle\Gamma_{X}(k)=\sum_{t=1}^{n-k}\boldsymbol{X}_{t+k}\boldsymbol{X}_{t}^{\prime}/n.$
(1.68)

Furthermore, the device proposed byPhillips and Solo,  (1992), is suitable to
derive invariance principles and functional central limit theorems in
different econometric environments. In particular, Grønneberg and Holcblat,
(2019) focus on establishing limit results for partial sum processes based on
ARMAX residuals. The following lemma from Grønneberg and Holcblat,  (2019)
adapts the Phillips-Solo device although unlike Phillips and Solo,  (1992),
the authors do not impose a time dependence structure, and allow two
dimensional arrays.

######  Lemma 2 (Grønneberg and Holcblat,  (2019)).

Let $C(L):=\sum_{k=0}^{\infty}c_{k}L^{k}$ be a lag series with uniformly
geometrically decreasing coefficients, that is, there exists $M\in\mathbb{R}$
and $\rho\in[0,1]$ such that for all $k\in\mathbb{N}$, $|c_{k}|M\rho^{k}$. If
a constant $\epsilon>0$, then

$\displaystyle\underset{(n,t)\in[1,\infty]\times\mathbb{Z}:t\leq
n}{\mathsf{sup}}\ \mathbb{E}|Z_{n,t}|^{1+\epsilon}<\infty$ (1.69)

Therefore, by the Beveridge-Nelson decomposition as stated in Lemma 1 of
Phillips and Solo,  (1992),

$\displaystyle\tilde{Z}_{n,t}:=\sum_{j=0}^{\infty}\left(\sum_{k=j+1}^{\infty}c_{k}\right)Z_{n,t-j},\
\ \text{as}\ \ n\to\infty,$ (1.70) $\displaystyle C(B)Z_{n,t}$
$\displaystyle=C(1)Z_{n,t}-(1-B)\tilde{Z}_{n,t}=C(1)Z_{n,t}-\left(\tilde{Z}_{n,t}-\tilde{Z}_{n,t-1}\right)$
$\displaystyle\Rightarrow\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}C(B)Z_{n,t}=C(1)\left[\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}Z_{n,t}\right]+\frac{\tilde{Z}_{n,0}}{n}-\frac{\tilde{Z}_{n,\lfloor\lambda
n\rfloor}}{n}$
$\displaystyle\Rightarrow\underset{\lambda\in[0,1]}{\mathsf{sup}}\left|\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}C(B)Z_{n,t}-C(1)\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}Z_{n,t}\right|=\underset{\lambda\in[0,1]}{\mathsf{sup}}\left|\frac{\tilde{Z}_{n,0}}{n}-\frac{\tilde{Z}_{n,\lfloor\lambda
n\rfloor}}{n}\right|$
$\displaystyle\leq\frac{2}{n}\underset{\lambda\in[0,1]}{\mathsf{sup}}\left|\tilde{Z}_{n,t}\right|=o_{p}(1).$

The second result is stated as below

$\displaystyle\underset{\lambda\in[0,1]}{\mathsf{sup}}\left|\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}\sum_{j=0}^{t}C(B)Z_{n,j}-C(1)\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}\sum_{j=0}^{t}Z_{n,j}\right|=o_{p}(1),\ \ \ \text{as}\ \ n\to\infty.$
(1.71)

Denote with
$\tilde{Z}_{n,t}:=\sum_{j=0}^{\infty}\left(\sum_{k=j+1}^{\infty}c_{k}\right)Z_{n,t-j}$.
Then, we have that

$\displaystyle C(B)Z_{n,t}$
$\displaystyle=C(1)Z_{n,t}-(1-B)\tilde{Z}_{n,t}=C(1)Z_{n,t}-\left(\tilde{Z}_{n,t}-\tilde{Z}_{n,t-1}\right)$
$\displaystyle\Rightarrow\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}\sum_{j=0}^{t}C(B)Z_{n,t}=C(1)\left[\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}\sum_{j=0}^{t}Z_{n,t}\right]+\frac{\tilde{Z}_{n,0}}{n}-\frac{\tilde{Z}_{n,\lfloor\lambda
n\rfloor}}{n}$
$\displaystyle\Rightarrow\underset{\lambda\in[0,1]}{\mathsf{sup}}\left|\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}\sum_{j=0}^{t}C(B)Z_{n,t}-C(1)\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}\sum_{j=0}^{t}Z_{n,t}\right|=\underset{\lambda\in[0,1]}{\mathsf{sup}}\left|\frac{\tilde{Z}_{n,0}}{n}-\frac{\tilde{Z}_{n,\lfloor\lambda
n\rfloor}}{n}\right|\leq\frac{2}{n}\underset{\lambda\in[0,1]}{\mathsf{sup}}\left|\tilde{Z}_{n,t}\right|=o_{p}(1).$

The above result holds due to the following property

$\displaystyle\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}\sum_{j=0}^{t}\left(\tilde{Z}_{n,t}-\tilde{Z}_{n,t-1}\right)=\frac{\tilde{Z}_{n,t}}{n}-\frac{\tilde{Z}_{n,\lfloor\lambda
n\rfloor}}{n}$ (1.72)

Let $\eta>0$ be given. Then, by definition of supremuem, as $n\to\infty$, we
have that

$\displaystyle\mathbb{P}\left(\frac{1}{n}\underset{t\in[0,n]}{\mathsf{sup}}\
\left|\tilde{Z}_{n,t}\right|>\eta\right)$
$\displaystyle=\mathbb{P}\left(\bigcup_{t=0}^{n}\left\\{\frac{1}{n}\left|\tilde{Z}_{n,t}\right|>\eta\right\\}\right)$
$\displaystyle\leq\sum_{t=0}^{n}\mathbb{P}\bigg{(}\frac{1}{n}\left|\tilde{Z}_{n,t}\right|>\eta\bigg{)}=\sum_{t=0}^{n}\mathbb{P}\bigg{(}\left(\frac{1}{n}\left|\tilde{Z}_{n,t}\right|\right)^{1+\epsilon}>\eta^{1+\epsilon}\bigg{)}$
$\displaystyle=\sum_{t=0}^{n}\mathbb{P}\bigg{(}\frac{1}{n^{1+\epsilon}}\left|\tilde{Z}_{n,t}\right|^{1+\epsilon}>\eta^{1+\epsilon}\bigg{)}$
$\displaystyle\leq\left(\underset{(n,t)\in[1,\infty]\times\mathbb{N}:t\leq
n}{\mathsf{sup}}\
\mathbb{E}\left|\tilde{Z}_{n,t}\right|^{1+\epsilon}\right)\frac{1}{\eta^{1+\epsilon}}\sum_{t=0}^{n}\frac{1}{n^{1+\epsilon}}$
$\displaystyle=\left(\underset{(n,t)\in[1,\infty]\times\mathbb{N}:t\leq
n}{\mathsf{sup}}\
\mathbb{E}\left|\tilde{Z}_{n,t}\right|^{1+\epsilon}\right)\frac{1}{\eta^{1+\epsilon}}\frac{2}{n^{\epsilon}}=o(1).$

######  Lemma 3.

(Stability of uniform partial-sum LLN under linear filtering). Let
$\left(\tilde{S}_{n,t}\right)$ be a stochastic array

  * (a)

for a constant $\epsilon>0$,
$\underset{(n,t)\in[1,\infty]\times\mathbb{Z}:t\leq n}{\mathsf{sup}}\
\mathbb{E}|Z_{n,t}|^{1+\epsilon}<\infty$, and that

  * (b)

$\underset{\lambda\in[0,1]}{\mathsf{sup}}\left|\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}\tilde{S}_{n,t}\right|=o_{p}(1)$.

Then, for all sequences $\left(\omega_{j}\right)_{j\in\mathbb{N}}$ such that
there exist $M>0$ and $\rho\in[0,1]$ for all $j\in\mathbb{N}$ such that
$|\omega_{j}|\leq M\rho^{j}$ and

$\displaystyle\underset{\lambda\in[0,1]}{\mathsf{sup}}\
\left|\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}\sum_{j=0}^{\infty}\omega_{j}\tilde{S}_{n,t-j}\right|=o_{p}(1)\ \
\text{as}\ \ n\to\infty.$ (1.73)

Let $\Gamma(B)=\sum_{j=0}^{\infty}\omega_{j}B^{j}$ such that

$\displaystyle\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}\sum_{j=0}^{\infty}\omega_{j}\tilde{S}_{n,t-j}=\frac{1}{n}\sum_{t=1}^{\lfloor\lambda
n\rfloor}\Gamma(B)\tilde{S}_{n,t}$ (1.74)

where $\Gamma(B)\tilde{S}_{n,t}$ is well-defined. Then, addition and
subtraction of $\Gamma(1)\sum_{t=1}^{\lfloor\lambda n\rfloor}\tilde{S}_{n,t}$
with the triangle inequality yields the following result

###  1.7. Moderate Deviations for Multivariate Martingales

According to Grama and Haeusler,  (2006), limit theorems for probabilities of
moderate deviations for sums of independent random variables have been
extensively examined before. For martingales, however the number of results on
moderate deviations in the literature is still rather limited. Moderate
deviation results are related to bounds on the rate of convergence in the
central limit theorem. In the multivariate case, quadratic characteristics are
matrix valued processes. Therefore, to establish uniform estimates of the rate
of convergence in the multivariate martingale CLT, moderate deviations require
special handling.

##### Univariate Case

Let $\left(\xi_{nk},\mathcal{F}_{nk}\right)_{0\leq k\leq n}$ be a square
integrable martingale difference sequence with $\xi_{n0}=0$. We set

$\displaystyle X_{k}^{n}=\sum_{i=1}^{k}\xi_{ni},\ \ 1\leq k\leq n.$ (1.75)

Denote by $\langle X^{n}\rangle$ the quadratic characteristic of the
martingale $X^{n}$, such that

$\displaystyle\langle X^{n}\rangle_{k}=\sum_{i=1}^{k}a_{ni},\ \
a_{nk}=\mathbb{E}\left(\xi_{nk}^{2}|\mathcal{F}_{n,k-1}\right),\ \ 1\leq k\leq
n.$ (1.76)

##### Multivariate Case

Denote by $\langle X^{n}\rangle$ the quadratic characteristic of the
martingale $X^{n}$, i.e.,

$\displaystyle\langle X^{n}\rangle_{k}=\sum_{i=1}^{k}a_{ni},\ \
a_{nk}=\mathbb{E}\left(\xi_{nk}\xi_{nk}^{\prime}|\mathcal{F}_{n,k-1}\right),\
\ 1\leq k\leq n.$ (1.77)

######  Example 5.

Consider the autoregressive model given by

$\displaystyle X_{t}=\rho_{1}X_{t-1}+...+\rho_{d}X_{t-d}+\epsilon_{t},$ (1.78)

Then, the OLS estimator is given by

$\displaystyle\hat{\rho}_{n}=\left(\sum_{t=1}^{n}\mathbf{X}_{t-1}\mathbf{X}_{t-1}^{\prime}\right)^{-1}\sum_{t=1}^{n}\mathbf{X}_{t-1}X_{t}$
(1.79)

Therefore, we have that

$\displaystyle\frac{1}{\sqrt{n}}\sum_{t=1}^{n}\mathbf{X}_{t-1}\mathbf{X}_{t-1}^{\prime}\left(\hat{\rho}_{n}-\rho\right)=\frac{1}{\sqrt{n}}\sum_{t=1}^{n}\mathbf{X}_{t-1}\epsilon_{t},$
(1.80)

Thus, the $d-$dimensional random vectors
$\xi_{nt}=\mathbf{X}_{t-1}\epsilon_{t}/\sqrt{n}$ form a square integrable
martingale difference array with respect to the $\sigma-$fields
$\mathcal{F}_{nt}$, which implies that asymptotic results for the OLS
estimator $\hat{\rho}_{n}$ may be derived from martingale limit theory.

Moreover, it is well known that the distributional properties of the time
series $(X_{t})_{t\geq-d+1}$ are strongly dependent on the value of
$\rho\in\mathbb{R}^{d}$ are are entirely different for different
$\rho-$regions in $\mathbb{R}^{d}$. One of the basic results states that there
exists a stationary distribution of
$\mathbf{X}_{0}=\left(X_{0},...,X_{-d+1}\right)^{\prime}$, that is, a
distribution which results in a strictly stationary process
$(X_{t})_{t\geq-d+1}$ , if and only if all (possible complex) roots of the
equation

$\displaystyle\lambda^{d}-\rho_{1}\lambda^{d-1}-\rho_{2}\lambda^{d-2}-...-\rho_{d-1}\lambda-\rho_{d}=0$
(1.81)

have modules strictly less than one. Then, the quadratic characteristic of the
martingale difference array $\left(\xi_{nt},\mathcal{F}_{nt}\right)$ satisfies

$\displaystyle\sum_{t=1}^{n}\mathbb{E}\left(\xi_{nt}\xi_{nt}^{\prime}|\mathcal{F}_{n,t-1}\right)=\frac{\sigma^{2}}{n}\sum_{t=1}^{n}\mathbf{X}_{t-1}\mathbf{X}_{t-1}^{\prime}\to\sigma^{4}\Sigma$
(1.82)

in probability as $n\to\infty$, since $\mathbf{X}_{t-1}$ is measurable w.r.t
$\mathcal{F}_{n,t-1}$ and $\epsilon_{t}$ and $\mathcal{F}_{n,t-1}$ are
independent. Moreover, the Lindeberg condition is satisfied, and the
multivariate CLT implies that

$\displaystyle\sum_{t=1}^{n}\xi_{nt}\to\mathcal{N}(0,\sigma^{4}\Sigma),\ \
\text{as}\ n\to\infty,$ (1.83)

Therefore, we obtain the CLT for the least squares estimator $\hat{\rho}_{n}$
as below

$\displaystyle\sqrt{n}\left(\hat{\rho}_{n}-\rho\right)\to\mathcal{N}\left(0,\Sigma^{-1}\right),\
\ \text{as}\ \ n\to\infty,$ (1.84)

Consider also the $\left\lVert\ .\ \right\rVert_{\Sigma}$ to be the norm on
$\mathbb{R}^{d}$ pertaining to the inner product induced by $\Sigma$, that is,
$\left\lVert x\right\rVert_{\Sigma}=\left\lVert\Sigma^{1/2}x\right\rVert$ for
all $x\in\mathbb{R}^{d}$, where $\left\lVert.\right\rVert$ is the Euclidean
norm in $\mathbb{R}^{d}$.

Then, for any sequence $x_{n}\to\infty$ with
$x_{n}=\mathcal{O}(n^{\delta-\epsilon})$ for some $\epsilon>0$ as
$n\to\infty$, we have the moderate deviations limit theorem

$\displaystyle\mathbb{P}\left(\left\lVert\sqrt{n}\left(\widehat{\rho}_{n}-\rho\right)\right\rVert_{\Sigma}\geq\sqrt{2\text{log}x_{n}}\right)$
$\displaystyle=\frac{1}{\Gamma(d/2)}x_{n}^{-1}\left(\text{log}x_{n}\right)^{d/2-1}\left(1+\mathcal{O}\left(\frac{1}{\text{log}x_{n}}\right)\right).$

Consider the martingale difference array such that
$\tilde{\xi}_{nt}=\sigma^{-2}\Sigma^{-1/2}\xi_{nt}$. Using this martingale
difference array we have that

$\displaystyle\tilde{L}_{\delta}=\sum_{t=1}^{k}\mathbb{E}\left\lVert\tilde{\xi}_{nt}\right\rVert^{2+2\delta}=\mathcal{O}(n^{-\delta}).$
(1.85)

###  1.8. Estimators for Gaussian Autoregressive Process

Consider the OU diffusion solution of the stochastic differential equation

$\displaystyle dX_{t}=\theta X_{t}dt+dW_{t},\ \ X_{0}=x,$ (1.86)

where $W$ is a standard Brownian motion and $\theta$ is an unknown parameter
in $\mathbb{R}$. For a continuous observation of $X$ over $[0,T]$, it is usual
to consider an estimating function, the score function, that is, the
derivative of the log-likelihood, which is given by

$\displaystyle
Y_{T}(\theta)=\frac{1}{2}\left(X_{T}^{2}-x^{2}\right)-\int_{0}^{T}\left(\theta
X_{t}^{2}+\frac{1}{2}\right)dt.$ (1.87)

We denote by $\hat{\theta}_{T}$ the maximum likelihood estimator (MLE),
solution of $Y_{T}(\theta)=0$, explicitly as

$\displaystyle\hat{\theta}_{T}=\frac{\displaystyle\int_{0}^{T}X_{t}dX_{t}}{\displaystyle\int_{0}^{T}X_{t}^{2}dt}.$
(1.88)

######  Example 6.

Consider $(X_{n})$ to be the stable autoregressive model of order $p$, and
dimensions $d$

$\displaystyle X_{n}=A_{1}X_{n-1}+A_{2}X_{n-2}+...+A_{p}X_{n-p}+\epsilon_{n}$
(1.89)

with an initial state $X_{0}^{(p)}=\left(X_{0},X_{-1},...,X_{-p+1}\right)$
independent from the noise. By stable model we mean that all roots of the
polynomial $z\mapsto\text{det}\left(I-A_{1}z-...-A_{p}z^{p}\right)$ have their
modules $>1$.

######  Example 7.

Consider the linear autoregressive model in $\mathbb{R}^{d}$

$\displaystyle X_{n}=\theta X_{n-1}+\xi_{n}$ (1.90)

Notice that a stationary solution of the autoregressive equation is given by

$\displaystyle X_{n}=\sum_{p=0}^{\infty}\theta^{p}\xi_{n-p},\ \ \ n\geq 0$
(1.91)

Thus, as in Yu and Si,  (2009), for estimating $\theta$, the following two
estimators are widely used

$\displaystyle\widehat{\theta}^{ols}_{n}$
$\displaystyle=\left(\sum_{k=1}^{n}X_{k}X_{k-1}^{\top}\right)\left(\sum_{k=1}^{n}X_{k-1}X_{k-1}^{\top}\right).$
(1.92) $\displaystyle\widetilde{\theta}^{YW}_{n}$
$\displaystyle=\left(\sum_{k=1}^{n}X_{k}X_{k-1}^{\top}\right)\left(\sum_{k=0}^{n}X_{k}X_{k}^{\top}\right).$
(1.93)

Consider the least-squares estimator $\theta_{n}$ of the parameter $\theta$
which is defined as below (see, Worms,  (2001))

$\displaystyle\hat{\theta}_{n}=\left(\sum_{j=1}^{n}X_{j-1}^{2}\right)\sum_{j=1}^{n}X_{j-1}X_{j}.$
(1.94)

It is well-known that $\left(\hat{\theta}_{n}\right)$ is strongly consistent,
but the behaviour in distribution and the corresponding speeds are different
according to the true parameter value of $\theta$, when $X_{0}=0$.

  * •

in the stable case $\left(|\theta|<1\right)$,
$\sqrt{n}\tilde{\theta}/\sqrt{1-\theta^{2}}$ converges in distribution to the
Gaussian law $\mathcal{N}(0,1)$,

  * •

in the explosive case $\left(|\theta|>1\right)$,
$\sqrt{n}\tilde{\theta}/\sqrt{1-\theta^{2}}$ converges in distribution to a
Gauchy distribution,

  * •

in the unstable case $\left(|\theta|=1\right)$, $\tilde{\theta}$ converges in
distribution to $B_{1}/\int_{0}^{1}B_{s}^{2}ds$, where $\left(B_{t}\right)$ is
some standard Brownian motion.

###  1.9. The Exogeneity Assumption in Time Series Models

The second important issue when considering econometric modelling is the
aspect of exogeneity. In other words, the concept of exogeneity is essential
to uniquely characterize the implications that certain variables are exogenous
according to particular definitions.

  * •

Weak exogeneity:

A good intuition of the concept of weak exogeneity, is that it guarantees that
the parameters of the conditional model and those of the marginal model are
variation free, it offers a natural framework for analyzing the structural
invariance of parameters of conditional models. However, by itslef, weak
exogeneity is neither necessary nor sufficient for structural invariance of a
conditional model.

  * •

Strong exogeneity:

If in addition to being weakly exogenous, $z_{t}$, is not caused in the sence
of Granger by any of the endogenous variables in the system, then $z_{t}$ is
defined to be strongly exogenous.

The concept of structurally invariant conditional models characterizes the
conditions which guarantee the appropriateness of "policy implications" or
other control exercises, since any change in the distribution of the
conditioning variables has no effect on the conditional model and thus on the
conditional forecasts of the endogenous variables (see, Ericsson et al.,
(1998), White and Pettenuzzo,  (2014)). It might be more helpful to understand
what is not causality, thus we consider the definition of Granger
Noncausality.

######  Definition 5.

$Y_{t-1}$ does not Granger cause $z_{t}$ with respect to $X_{t-1}$ if and only
if

$\displaystyle
D(z_{t}|X_{t-1},\theta)=D\left(z_{t}|Z_{t-1},Y_{0},\theta\right)$ (1.95)

######  Remark 4.

Granger noncausality is neither necessary nor sufficient for weak exogeneity.
Granger noncausality in combination with weak exogeneity, however, defines
strong exogeneity.

######  Example 8.

(time-varying covariates: endogeneity versus exogeneity) Consider for
simplicity a generalized linear model, in which case we assume that
$\left\\{X_{i}\right\\}$ is stochastic which implies that
$\left(Y_{i1},X_{i1}\right)$,…,$\left(Y_{in_{i}},X_{in_{i}}\right)$ are
independently distributed. Furthermore, this assumption implies

$\displaystyle\mathbb{E}\left[Y_{ij}|X_{i1},X_{i2},...,X_{in_{i}}\right]=\mathbb{E}\left[Y_{ij}|X_{ij}\right],$
(1.96)

which is called the Full Covariate Conditional Mean (FCCM) assumption. In
other words, if current outcomes predict future values of the covariates, then
the above equality does not hold. In particular, if
$f_{X}\left(X_{ij+1}|Y_{ij},X_{ij}\right)\neq
f_{X}\left(X_{ij+1}|X_{ij}\right)$, then we have that

$\displaystyle f_{Y}\left(Y_{ij+1}|X_{ij},X_{ij+1}\right)\neq
f_{Y}\left(Y_{ij+1}|X_{ij}\right)$ (1.97)

and so the expectation $\mathbb{E}\left[Y_{ij}|X_{ij}\right]$ may be
incorrect. Therefore, it is important to distinguish between the presence of
endogeneity versus exogeneity for time-varying covariates. Assume that we have
follow-up times which are discrete, e.g., $t=1,...,T$. Let
$\mathcal{H}_{it}^{Y}:\left\\{Y_{i1},....,Y_{it}\right\\}$ and
$\mathcal{H}_{it}^{X}:\left\\{X_{i1},....,X_{it}\right\\}$, to denote the
response history of subject $i$ at time $t$, and the covariate history of
subject $i$ at time $t$ (filtration) respectively, where $Z_{i}$ to denote the
baseline covariates of subject $i$. Then, exogeneity implies

$\displaystyle
f_{X}\left(X_{it}|\mathcal{H}_{it}^{Y},\mathcal{H}_{it-1}^{X},Z_{i}\right)=f_{X}\left(X_{it}|\mathcal{H}_{it-1}^{X},Z_{i}\right)$
(1.98)

and endogeneity gives that

$\displaystyle
f_{X}\left(X_{it}|\mathcal{H}_{it}^{Y},\mathcal{H}_{it-1}^{X},Z_{i}\right)\neq
f_{X}\left(X_{it}|\mathcal{H}_{it-1}^{X},Z_{i}\right)$ (1.99)

where $f_{X}$ is the pdf/cdf of $X_{it}$. Exogeneity assumption implies that
$f_{X,Y}\left(Y_{it},X_{it}|Z_{i},\theta\right)=\ell_{Y}(\theta).\ell(\theta)$,
which implies that the likelihood of $Y$ can be maximized independently of the
likelihood of $X$. Also, if $\theta=\left(\theta_{1},\theta_{2}\right)$ and
$f_{X,Y}\left(Y_{it},X_{it}|Z_{i},\theta\right)=\ell_{Y}(\theta_{1}).\ell(\theta_{2})$,
there will be no loss of information/no loss of efficiency if we condition on
$X$ instead of modelling the joint density. Exogeneity further implies

$\displaystyle\mathbb{E}\left[Y_{it}|X_{i1},...,X_{iT},Z_{i}\right]=\mathbb{E}\left[Y_{it}|X_{i1},...,X_{iT},Z_{i}\right]$
(1.100)

that is, $Y_{it}$ is conditionally independent of all future outcomes given
the covariate history.

######  Remark 5.

Conditional invariance is a property of the data generating process, defined
without regard to any model, unlike super exogeneity (see, White,  (1996)).
Thus, conditional invariance can be viewed as a form of necessary condition
for examining the effect of $Y_{t}$ of policy changes (i.e., changes in the
distribution of $W_{t}$), because if $\left\\{Y_{t}|W_{t}\right\\}$ is not
conditionally invariant, then changing the distribution of $W_{t}$ may change
the relationship between $Y_{t}$ and $W_{t}$ in unpredictable ways. In this
situation, even a correctly specified model of the old relationship is likely
to be of little value in describing the behaviour of $Y_{t}$ under the new
regime. Numerous examples can be given to show that dynamic misspecification
may adversely affect consistency, and if not consistency, then attainment of
the asymptotic variance bound. Furthermore, neglected heteroscedasticity may
also adversely affect attainment of the asymptotic variance bound.

###  1.10. Autoregressive Processes with Exogenous Variables

Following the framework of Damon and Guillas,  (2005), we consider the
following autoregressive Hilbertian with exogenous variables of order one
model, denoted by ARHX(1) such that

$\displaystyle
Y_{t}=\rho(Y_{t-1})+\beta_{1}(X_{t,1})+...+\beta_{p}(X_{t,p})+\varepsilon_{t},\
\ \ t\in\mathbb{Z},$ (1.101)

where $X_{t,1}+...+X_{t,p}$ are $p$ autoregressive of order one exogenous
variables associated respectively with operators $\varphi_{1},...,\varphi_{p}$
and strong white noises $u_{t,1},...,u_{t,p}$ such that

$\displaystyle X_{t,i}=\varphi_{j}(X_{t-1,i})+u_{t,i},\ \ \ \text{for all}\ \
i\in\left\\{1,...,p\right\\}.$ (1.102)

######  Remark 6.

Notice that regressors of ’mixed integration order’ it should represent the
case where the autoregressive model includes regressors of different
integration order such as stationary versus nonstationary rather regressors
which are LUR but fall in a different persistence class due to different
values and sign of the localizing coefficients of persistence.

Furthermore, define with

$\displaystyle\mathcal{Y}_{t}=\begin{bmatrix}Y_{t}\\\ X_{t+1,1}\\\ \vdots\\\
X_{t+1,p}\end{bmatrix},\ \ \mathcal{E}_{t}=\begin{bmatrix}\varepsilon_{t}\\\
u_{t,1}\\\ \vdots\\\ u_{t,p}\end{bmatrix}\ \ \ \text{and}\ \ \
\mathcal{R}=\begin{bmatrix}\rho&\beta_{1}&\ldots&\ldots&\beta_{p}\\\
0&\varphi_{1}&0&\ldots&0\\\ 0&0&\varphi_{2}&0&\vdots\\\
\vdots&\vdots&&\ddots&0\\\ 0&0&0&0&\varphi_{p}\end{bmatrix}$ (1.103)

In particular, when the sequence $\left\\{Y_{t}\right\\}$ is an ARHX (1)
model, then $\left\\{\mathcal{Y}_{t}\right\\}$ is an $H^{p+1}-$valued ARHX(1)
process, as seen by the following autoregressive representation

$\displaystyle\mathcal{Y}_{t}=\mathcal{R}(\mathcal{Y}_{t-1})+\mathcal{E}_{t},\
t\in\mathbb{Z}.$ (1.104)

We suppose that the noises $\varepsilon_{t}$ and $u_{t,1},...,u_{t,p}$ are
independent which ensures that for all $i\in\left\\{1,...,p\right\\}$ the
whole processes $(\varepsilon_{t})$ and $\left(X_{t,i}\right)$ are independent
since $X_{t,i}$ can be expressed as an infinite moving average of the
$u_{t-q,1}$ where $q=0,...,\infty$.

######  Proposition 1.

Equation (1) has a unique stationary solution given by

$\displaystyle
X_{k}=\sum_{j=0}^{\infty}\left(P_{1}\mathcal{R}^{j}\right)\mathcal{E}_{k-j},\
\ \ t\in\mathbb{Z}.$ (1.105)

Denote by $P_{i}$ the projection operators
$\big{(}X_{1},...,X_{q+1}\big{)}\mapsto x_{i}.$ Then, the series converge
almost surely and in
$\mathcal{L}^{2}_{H}\left(\Omega,\mathcal{A},\mathcal{P}\right)$.

######  Definition 6.

The covariance operator $C^{X,Y}$ of two $H-$valued random variables $X$ and
$Y$ is

$\displaystyle C^{X,Y}(x):=\mathbb{E}\big{[}\langle X,x\rangle Y\big{]},\ \ \
x\in H,$ (1.106)

and $C^{X}$ stands for $C^{X,Y}$. The autocovariance of a stationary process
$(X_{n})$ is the sequence of operators defined by
$C_{h}=\big{(}C^{X_{0},X_{h}},h\in\mathbb{Z}\big{)}$.

Consider for example the framework given by Seo and Beare,  (2019). A
cointegrated linear process in Bayes Hilbert space is isomorphic to a
cointegrated linear process in a Hilbert space of centered square-integrable
real functions. We illustrate the use of this isomorphism for modeling
nonstationary time series of probability densities. In particular, a recent
literature on functional data analysis deals with datasets whose observations
take values in an infinite dimensional Banach or Hilbert space, typically a
space of functions.

######  Proposition 2.

Let $H_{1}$ and $H_{2}$ be real separable Hilbert spaces, and $S:H_{1}\to
H_{2}$ an isomorphism between them. If $X$ is a cointegrated linear process in
$H_{2}$, then $S^{-1}X=S^{-1}X_{t},t\geq 0$, is a cointegrated linear process
in $H_{1}$, and its attractor space is the inverse image of the attractor
space of $X$ under $S$.

###### Proof.

Each $S^{-1}X_{t}$ is a random element of $H_{1}$ (since $S^{-1}$ is Borel
measurable) that satisfies the finite moment condition $\mathbb{E}\left\lVert
S^{-1}X_{t}\right\rVert_{H_{1}}^{2}<+\infty$ (since $S^{-1}$ is norm-
preserving) and thus it holds that $\mathbb{E}S^{-1}X_{t}=0$ (since $S^{-1}$
is linear). Therefore, $S^{-1}X_{t}$ is indeed a sequence in $L_{H_{1}}^{2}$.

Furthermore, its differences $\Delta S^{-1}X_{t}$ satisfy

$\displaystyle\Delta
S^{-1}X_{t}=\sum_{j=0}^{+\infty}S^{-1}\Psi_{k}S\big{(}S^{-1}\left(\epsilon_{t-k}\right)\big{)},\
\ \ t\geq 1.$ (1.107)

Thus, we will show that expression (1.107) above constitutes a valid
representation for $\Delta S^{-1}X_{t}$ with coefficients $S^{-1}\Psi_{k}S$
and innovations $S^{-1}\left(\epsilon_{t}\right)$. In particular, the argument
used above to show that $S^{-1}X_{t}\in L_{H_{1}}^{2}$ also shows that the
innovations $S^{-1}\epsilon_{t}$ belong to $L_{H_{1}}^{2}$. Moreover, they are
i.i.d due to the i.i.d property of the $\epsilon_{t}$’s and the Borel
measurability of $S^{-1}$. They have covariance operator $S^{-1}\Sigma S$
(since $S$ is the adjoint to its inverse), which is positive definite since

$\displaystyle\langle S^{-1}\Sigma S(f),f\rangle_{H_{1}}=\langle\Sigma
S(f),S(f)\rangle_{H_{2}}>0$ (1.108)

for any nonzero $f\in H_{1}$ due to the positive definiteness of $\Sigma$.

∎

##  2\. Temporal Dependence

Econometric models provide parsimonious representations of economic phenomena;
thus the development of robust statistical methods is of paramount importance
in improving our understanding of financial markets and economic decision
making. In this section we study some aspects of estimation and inference in
time series regression under temporal dependence.

###  2.1. Setting the time series framework

Consider the following model

$\displaystyle X_{t}=F\left(X_{t-1},M_{t-1},\epsilon_{t}\right),\ \
t\in\mathbb{Z},$ (2.1)

where $\left(X_{t}\right)_{t\in\mathbb{Z}}$ is the stochastic time series we
aim to model, $\left(M_{t}\right)_{t\in\mathbb{Z}}$ is a covariate process and
$\left(\epsilon_{t}\right)_{t\in\mathbb{Z}}$ a noise process. Here, we notice
that $X_{t}=f_{t}\left(X_{t-1}\right)$ holds for the random function defined
by $f_{t}(x)=F\left(x,M_{t-1},\epsilon_{t}\right)$. Therefore, the stochastic
time series model (2.1) shows that the stochastic sequence
$\left\\{X_{t}\right\\}$ transforms a sequence of i.i.d random innovations
$\epsilon_{t}$ into a time series sequence of dependent random variables
(maps) as described by the random function
$\left(f_{t}\right)_{t\in\mathbb{Z}}$. Moreover, the stochastic process
$\left(M_{t}\right)_{t\in\mathbb{Z}}$ allows to incorporate in the model
exogenous covariates which typically exhibit temporal dependence. Under this
general setting we can examine both linear and nonlinear time series models
without restricting the estimation methodology employed.

###  2.2. Example: Nonlinear time series models

A nonlinear time series methodology which is commonly employed to capture
conditional heteroscedasticity and dynamic variances in financial markets
includes the ARCH and GARCH models. Specifically, these two nonlinear type of
models have been the cornerstone of financial econometrics and statistical
risk management fields. The ARCH model has been generalized to to GARCH by
Bollerslev,  (1986). A GARCH$(p,q)$ sequence
$\left\\{X_{t},-\infty<t<\infty\right\\}$ has the following form

$\displaystyle X_{t}=\sigma_{t}\epsilon_{t}$ (2.2)
$\displaystyle\sigma_{t}^{2}=\alpha_{0}+\sum_{i=1}^{p}\alpha_{i}X^{2}_{t-i}+\sum_{j=1}^{q}\beta_{j}\sigma^{2}_{t-j}$
(2.3)

The literature has evolved considerably since the introduction of the
GARCH$(p,q)$ model. The aforementioned stochastic linear and nonlinear time
series models exhibit different asymptotic behaviour under the presence of
exogenous covariates, especially considering these time series models as
iterations of dependent random variables capturing linear and nonlinear
dynamics.

Furthermore, considering these time series models as stochastically recursive
sequences, when the independence assumption is removed under certain
regulatory conditions, certain interesting asymptotic cases appear. For
instance, we can study stationarity of asymptotic power GARCH processes which
encompasses the case of exogenous regressors in nonlinear dynamics. Notice
that for strictly exogenous regressors, the processes
$\left(M_{t}\right)_{t\in\mathbb{Z}}$ and
$\left(\epsilon_{t}\right)_{t\in\mathbb{Z}}$ are independent.

We are interested for example in the dynamics of predictive regression models
described by (2.1) with regressors that are not necessarily strictly
exogenous, assuming that at any time $t$, the noise $\epsilon_{t}$ is
independent from the past information
$\sigma\left(\left(M_{s},\epsilon_{s}\right):s\leq t-1\right)$. Therefore,
this independence assumption is weaker than the independence assumption
between the two processes $\left(\epsilon_{t}\right)_{t\in\mathbb{Z}}$ and
$\left(M_{t}\right)_{t\in\mathbb{Z}}$. The latter independence condition
implies strict exogeneity, a terminology intially defined by Sims. Strict
exogeneity is useful for deriving the conditional likelihood of the $X_{t}$’s
conditionally on the $M_{t}$’s. However, strict exogeneity is a strong
assumption. Moreover, Chamberlain,  (1982) shown has shown that this
assumption is equivalent to the non-Granger causality, that is, $M_{t}$ is
independent of $\left(X_{s}\right)_{s\leq t}$ conditionally on
$\left(X_{s}\right)_{s\leq t-1}$. This idea means that the covariate process
$\left(M_{t}\right)_{t\in\mathbb{Z}}$ evolves in a totally autonomous way. On
the other hand, our exogeneity condition allows general covariates of the form
$M_{t}=H\left(\eta_{t},\eta_{t-1},....,\right)$ with $H$ a measurable function
and a sequence $\left(\eta_{t},\epsilon_{t}\right)_{t\in\mathbb{Z}}$ of i.i.d
random vectors, $\epsilon_{t}$ being possibly correlated with $\eta_{t}$.
Thus, the error $\epsilon_{t}$ can then still have an influence on future
values of the covariates. For linear models, the two technical independence
conditions discussed above between the noise and the covariate processes are
often used as a distinction between weak and strict exogeneity (see, Debaly
and Truquet,  (2021)).

The family of ARCH and GARCH models are commonly used parametric models for
capturing important stylized features of financial time series such as long
memory, volatility clustering, fat tails in the distribution of stock returns
etc. In particular, the GARCH(1,1) model or AR(1)-GARCH(1,1) model have
repeatedly proved to provide robust parsimonious representations of market
volatility (see Berkes et al.,  (2003)). A GARCH(1,1) regression process is
given by the following equations

$\displaystyle y_{t}$ $\displaystyle=\mu+\eta_{t},\
\eta_{t}=\sigma_{t}\epsilon_{t},\ \ \ \epsilon_{t}\sim N(0,1)$
$\displaystyle\sigma^{2}_{t}$
$\displaystyle=\omega+\alpha\epsilon^{2}_{t-1}+\beta\sigma^{2}_{t-1}$

An AR(1)-GARCH(1,1) regression process is given by

$\displaystyle y_{t}$ $\displaystyle=\mu+\rho y_{t-1}+\eta_{t},\
\eta_{t}=\sigma_{t}\epsilon_{t}\ \ epsilon_{t}\sim N(0,1)$
$\displaystyle\sigma^{2}_{t}$
$\displaystyle=\omega+\alpha\epsilon^{2}_{t-1}+\beta\sigma^{2}_{t-1}$

The above non-linear models are valid under certain regulatory conditions
which include $\omega,\alpha,\beta\geq 0$ and $\alpha+\beta<1$ in order to
avoid the existence of IGARCH effects. The recursively estimated sequences
$(\sigma^{2}_{t})$ for $t=0,...,+\infty$ are assumed to be non negative with
probability 1 and common unconditional local mean given by
$\sigma^{2}=\frac{\omega}{1-\alpha-\beta}$. Statistical estimation can be done
by imposing certain parametric assumptions. A commonly used methodology in the
literature is the implementation of the QMLE.

The use of QML estimation can restore any inefficient model estimations (i.e.,
existence of negative coefficients or out of bounds persistence) and allows
for higher estimation precision since the proposed methodology is less
restrictive in the moment assumptions of the observed process (no restriction
on normality assumption - which induces distribution-free inference).

$\displaystyle L_{N}(\theta)\equiv
L_{N}(\theta;\epsilon_{1},...,\epsilon_{N})=\prod_{t=1}^{N}\frac{1}{\sqrt{2\pi\hat{\sigma}^{2}_{t}}}\exp\bigg{(}-\frac{\epsilon^{2}_{t}}{2\hat{\sigma}^{2}_{t}}\bigg{)},\hat{\sigma}^{2}_{t}=\omega+\alpha\epsilon^{2}_{t-1}+\beta\hat{\sigma}^{2}_{t-1}.$
(2.4)

and the QMLE for the set of model parameters
$\theta=\\{\omega,\alpha,\beta\\}$ of GARCH(1,1) to be

$\displaystyle\hat{\theta}_{N}=\underset{\theta\in\Theta}{\text{argmax}}\
L_{N}(\theta)$

which is equivalent to

$\displaystyle\underset{\theta\in\Theta}{\text{argmin}}\ \hat{l}_{N}(\theta),\
\text{where}\
\hat{\ell}_{N}(\theta)=\sum_{t}\bigg{(}\log[\hat{\sigma}^{2}_{t}]+\frac{\epsilon^{2}_{t}}{\hat{\sigma^{2}_{t}}}\bigg{)}.$

Then asymptotic normality holds, which implies that,

$\displaystyle\displaystyle{\sqrt{N}(\hat{\theta}_{N}-\theta_{0}^{*})\overset{d}{\to}N(\mathbf{0},\mathbf{V_{0}})},\
\mathbf{V_{0}}=\mathbf{B_{0}^{-1}A_{0}B_{0}^{-1}}.$ (2.5)

###  2.3. Heavy Tail Time series

####  2.3.1. Max-stable processes

######  Definition 7.

A time series $\left\\{\mathbf{X}_{j},j\in\mathbb{Z}\right\\}$ is max-stable
if all its finite dimensional distributions are max stable.

######  Definition 8.

We call a Lebesgue measurable function $L$ from $\mathbb{R}$ into $(0,\infty)$
slowly varying iff

$\displaystyle\underset{\lambda\to\infty}{\text{lim}}\frac{L(\lambda
t)}{L(\lambda)}=1,\ \ \text{for every}\ \ t>0.$ (2.6)

####  2.3.2. Tail behaviour of sample quantiles

######  Proposition 3.

Let $\mathbf{Y}_{j}=\left(Y_{j}(t),t\in T\right)$, $j=1,2,...$ be a sequence
of i.i.d measurable stochastic processes. The stochastic processes

$\displaystyle X_{j}(t)=\int_{\mathcal{S}}f(t,s)M_{j}(ds),\ \ t\in T,$ (2.7)

$j\geq 1$, are well defined and independent.

Moreover, we have that

$\displaystyle\bigg{(}X(t),t\in
T\bigg{)}\overset{d}{=}\bigg{(}\sum_{j=1}^{\infty}X_{j}(t),t\in T\bigg{)},$
(2.8)

with the sum converging a.s for every $t\in T$. We have already proved that
each $\left(X_{j}(t),t\in T\right)$ has a measurable version. Since the
pointwise limit of measurable functions is measurable, we have constructed our
measurable version of $\left(X(t),t\in T\right)$.

###  2.4. Extreme Value Theory for nonstationary time series

Consider the limit theory for extreme values of a class of nonstationary time
series with the form

$\displaystyle Y_{t}=\mu_{t}+\xi_{t},\ \ \
\xi_{t}=\sum_{j=0}^{\infty}c_{j}Z_{t-j}$ (2.9)

where $\left\\{Z_{t}=\sigma_{t}\eta_{t},-\infty<t<\infty\right\\}$ and
$\left\\{\eta_{t};-\infty<t<\infty\right\\}$ is a sequence of i.i.d random
variables with regularly varying tail probabilities. Some convergence results
for point processes based on one-sided moving averages
$\left\\{\xi_{t}=\sum_{j=0}^{\infty}c_{j}Z_{t-j}\right\\}$ are derived.
Extreme properties of the nonstationary sequence $\left\\{Y_{t}\right\\}$ are
then obtained from the convergence results (see, Niu,  (1997)).

######  Lemma 4.

Suppose for each $n\geq 1$, $\left\\{W_{n,j},j\geq 1\right\\}$ is a sequence
of independent nonidentically distributed random elements of
$(E,\mathcal{E})$. Define with
$N_{n}=\sum_{j=1}^{\infty}\epsilon_{(j/n,W_{n,j})}$. Then, $N_{n}\Rightarrow
N$ iff

$\displaystyle\sum_{j=1}^{\infty}\epsilon_{j/n}(.)\mathbb{P}\left\\{W_{n,j}\in.\right\\}\to_{\nu}\nu\times\mu.$
(2.10)

We assume that

$\displaystyle\mathbb{P}\left\\{|\eta_{1}|>x\right\\}\in RV_{-\alpha},\ \ \
\alpha>0$ (2.11)

and

$\displaystyle\underset{x\to\infty}{\text{lim}}\frac{\mathbb{P}(\eta_{1}>x)}{\mathbb{P}(|\eta_{1}|>x)}=\pi_{0},\
\ \
\underset{x\to\infty}{\text{lim}}\frac{\mathbb{P}(\eta_{1}<-x)}{\mathbb{P}(|\eta_{1}|>x)}=1-\pi_{0},$
(2.12)

where $0\leq\pi_{0}\leq 1$. Consider one-sided moving averages of the form

$\displaystyle\xi_{t}=\sum_{j=0}^{\infty}c_{j}Z_{t-j}=\sum_{j=0}^{\infty}c_{j}\sigma_{t-j}\eta_{t-j},\
\ \ -\infty<t<\infty$ (2.13)

where $\left\\{c_{j}\right\\}$ is a sequence of real constants with $c_{0}=1$

where

$\displaystyle\sum_{j=0}^{\infty}|c_{j}|^{\gamma}<\infty,\ \ \ \text{for
some}\ \ 0<\gamma<1$ (2.14)

Furthermore, we assume that

$\displaystyle\frac{1}{n}\sum_{t=1}^{n}\sigma_{t}^{\alpha}\to\sigma^{\alpha},\
\ \text{as}\ n\to\infty,$ (2.15)

######  Definition 9.

Let $\left\\{X_{nk},u_{n}\leq k\leq v_{n},n\geq 1\right\\}$ be an array of
random variables and $\left\\{a_{nk},u_{n}\leq k\leq v_{n},n\geq 1\right\\}$
an array of constants with $\sum_{k=v_{n}}^{v_{n}}|a_{nk}|\leq C$ for all
$n\in\mathbb{N}$ and some constant $C>0$. Moreover, let $\left\\{h(n),n\geq
1\right\\}$ be an increasing sequence of positive constants, with
$h(n)\to\infty$ as $n\to\infty$. The array $\left\\{X_{nk}\right\\}$ is said
to be $h-$integrable with respect to the array of constants
$\left\\{a_{nk}\right\\}$ if the following conditions hold:

$\displaystyle\underset{n\geq
1}{\text{sup}}\sum_{k=v_{n}}^{v_{n}}|a_{nk}|\mathbf{E}|X_{nk}|<\infty,\ \ \
\text{and}\ \ \
\underset{n\to\infty}{\text{lim}}\sum_{k=v_{n}}^{v_{n}}|a_{nk}|\mathbf{E}|X_{nk}|\mathbf{1}\left\\{|X_{nk}|>h(n)\right\\}=0.$
(2.16)

A linear process can be written in the following form

$\displaystyle X_{t}=\sum_{j}\psi_{j}\epsilon_{t-j},\ \ \ t\in\mathbb{Z}$
(2.17)

is regularly varying with index $\alpha>0$ if the i.i.d sequence
$\left(\epsilon_{t}\right)$ is regularly varying with index $\alpha$. Under
mildly conditions on $\left(\psi_{j}\right)$ we have that

$\displaystyle\frac{\mathbb{P}\left(X_{0}>x\right)}{\mathbb{P}\left(|\epsilon_{0}|>x\right)}\sim\sum_{j}|\psi_{j}|^{\alpha}\left(p\mathbf{I}\left\\{\psi_{j}>0\right\\}+\mathbf{I}\left\\{\psi_{j}<0\right\\}\right)=\left\lVert\psi\right\rVert_{\alpha}^{a},\
\ x\to\infty$ (2.18)

For an i.i.d sequence $\left(A_{t},B_{t}\right)_{t\in\mathbb{Z}}$, $A,B>0$,
the stochastic recurrence equation can be written as below

$\displaystyle X_{t}=A_{t}X_{t-1}+B_{t},\ \ t\in\mathbb{Z},$ (2.19)

has a unique stationary solution given by the following expression

$\displaystyle X_{t}=B_{t}+\sum_{j=-\infty}^{t-1}A_{t}....A_{j+1}B_{j},\ \
t\in\mathbb{Z},$ (2.20)

######  Remark 7.

Overall, the nature of dependence can vary and unless specific assumptions are
made about the dependence between random variables, no meaningful statistical
model can be assumed. A measure of dependence indicates how closely two random
variables $X$ and $Y$ are. Measures of dependence could be conditions based on
order or time between random variables, or could be conditions expressed in
terms of a covariance or a correlation coefficient. Distance is also
considered as a measure of dependence.

##  3\. Limit Theory under Network Dependence

###  3.1. Random Graphs and Moderate Deviations Theory

####  3.1.1. Spectral Edge in Sparse Random Graphs: Tail Large Deviations

Understanding asymptotic properties of spectral statistics arising from random
matrices has been the subject of intense study in recent years. Within this
theme, a particularly important research direction is in deriving large
deviation principles (rare event probabilities) for spectral functionals, such
as the empirical spectral measure and the extreme eigenvalues. In the results
above, we consider the large deviations of the edge eigenvalues of
$\mathcal{G}_{n,p}$, in the regime
$p<<\sqrt{\text{log}n/\text{log}\text{log}n}/n$. Notice for example that the
largest eigenvalue $\lambda_{1}\left(\mathcal{G}_{n,p}\right)$ is
asymptotically $\sqrt{L}_{p}$, which is governed by the maximum degree of the
graph, and corresponding eigenvector is localized; whereas above this
threshold the typical value is asymptotically $np$, which is dictated by the
total number of edges in the graph, and the corresponding eigenvector is
completely delocalized.

######  Proposition 4.

The space $\big{(}\mathcal{F},\left\lVert.\right\rVert_{n}\big{)}$ is a
pseudo-normed space.

###### Proof.

Note that $\left\lVert
f\right\rVert_{n}=\left(\frac{1}{n}\sum_{i=1}^{n}f^{2}\big{(}\boldsymbol{x}_{i}\big{)}\right)^{1/2}$.

(i) Based on the definition of $\left\lVert.\right\rVert_{n}$, it is clear
that $\left\lVert.\right\rVert_{n}\geq 0$, for any $f\in\mathcal{F}$.

(ii) For any $\lambda\in\mathbb{R}$ and $f\in\mathcal{F}$,

$\displaystyle\left\lVert\lambda
f\right\rVert_{n}=\left(\frac{1}{n}\sum_{i=1}^{n}\lambda^{2}f^{2}\big{(}\boldsymbol{x}_{i}\big{)}\right)^{1/2}=|\lambda|\left\lVert
f\right\rVert_{n}.$ (3.1)

(iii) For any $f,g\in\mathcal{F}$,

$\displaystyle\left\lVert f+g\right\rVert_{n}$
$\displaystyle=\left(\frac{1}{n}\sum_{i=1}^{n}\big{[}f\left(\boldsymbol{x}_{i}\right)+g\left(\boldsymbol{x}_{i}\right)\big{]}^{2}\right)^{1/2}=\left(\sum_{i=1}^{n}\left[\frac{1}{\sqrt{n}}f\left(\boldsymbol{x}_{i}\right)+\frac{1}{\sqrt{n}}g\left(\boldsymbol{x}_{i}\right)\right]^{2}\right)^{1/2}$
$\displaystyle\leq\left(\sum_{i=1}^{n}\left[\frac{1}{\sqrt{n}}f\left(\boldsymbol{x}_{i}\right)\right]^{2}\right)^{1/2}+\left(\sum_{i=1}^{n}\left[\frac{1}{\sqrt{n}}g\left(\boldsymbol{x}_{i}\right)\right]^{2}\right)^{1/2}$
$\displaystyle=\left\lVert f\right\rVert_{n}+\left\lVert g\right\rVert_{n},$

where we applied the triangle inequality to the classical Euclidean norm.
Therefore, we showed that
$\big{(}\mathcal{F},\left\lVert.\right\rVert_{n}\big{)}$ is a pseudo-normed
space.

∎

###  3.2. Limit theorems for network dependent processes

####  3.2.1. Network dependence condition

We discuss in details the framework proposed by Kojevnikov et al.,  (2021) as
this is useful in understanding the relevant limit theory. In particular, we
provide a sufficient condition for the shape of the network that ensures that
our limit theorems hold (LLN and CLT). A crucial aspect for the network based
limit theorems to hold, the number of neighbours at distance $s$ should not
grow too fast as $s$ increases. The precise condition for such neighbourhood
shells depends on the dependence coefficients $\theta_{n,s}$, so that if
$\theta_{n,s}$ decreases fast as $s$ increases, the requirement for the
neighbourhood shells can be weakened. To introduce sufficient conditions, let

$\displaystyle\delta_{n}^{\partial}(s;k)=\frac{1}{n}\sum_{i\in
N_{n}}|N_{n}^{\partial}(i;s)|^{k},$ (3.2)

where $N_{n}^{\partial}(i;s)$. When $k=1$, we simply write that
$\delta_{n}^{\partial}(s;1)=\delta^{\partial}_{n}(s)$. This quantity measures
the denseness of a network. Let us introduce a further notation. Define the
following expression

$\displaystyle\Delta_{n}(s,m;k)=\frac{1}{n}\sum_{i\in N_{n}}\underset{j\in
N_{n}^{\partial}(i;s)}{\text{max}}|N_{n}(i;m)\backslash N_{n}(j;s-1)|^{k}$
(3.3)

where $N_{n}^{\partial}(i;s)$ is defined above, and we take $N_{n}(j;s-1)=0$
if $s=0$. We also define the following expression

$\displaystyle
c_{n}(s,m;k)=\underset{a>1}{\text{inf}}\left[\Delta_{n}(s,m;ka)\right]^{\frac{1}{a}}\left[\delta_{n}^{\partial}\left(s;\frac{\alpha}{1-\alpha}\right)\right]^{1-\frac{1}{\alpha}}$
(3.4)

The quantity $c_{n}(s,m;k)$ is easy to compute when a network is given, and it
captures the network properties that are relevant for the limit theorems. It
consists of two components: $\Delta_{n}(s,m;k\alpha)$ and
$\delta_{n}^{\partial}\left(s;\alpha/(\alpha-1)\right)$. These capture the
denseness of the network through the average neighbourhood sizes and the
average neighbourhood shell size. We summarize a sufficient condition for the
network and the weak dependence coefficient as follows

######  Assumption 2 (Kojevnikov et al.,  (2021)).

(Condition ND.) There exists $p>4$ and a sequence $m_{n}\to\infty$ s.t

  * (a)

$\theta^{1-1/p}_{n,m_{n}}=o\left(n^{-3/2}\right)$,

  * (b)

for each $k\in\left\\{1,2\right\\}$,

$\displaystyle\frac{1}{n^{k/2}}\sum_{s\geq
0}c_{n}\left(s,m_{n};k\right)\theta_{n,s}^{1-\frac{k+2}{p}}=o_{a.s}(1),\
\text{and}$ (3.5)

  * (c)

sup${}_{n\geq 1}\text{max}_{i\in
N_{n}}\mathbf{E}\left[|Y_{n,i}|^{p}|\mathcal{C}_{n}\right]<\infty$ almost
surely.

Below we show that Condition ND (i) is sufficient for the LLN and CLT. Notice
that $\Delta_{n}(s,m_{n};k)$ tends to decrease fast to zero as $s$ goes beyond
a certain level, because the set $N_{n}(j;s-1)$ quickly becomes large.

######  Lemma 5 (Kojevnikov et al.,  (2021)).

Suppose that network $G_{n}$ is generated as above, and let $c_{n}$ be a
$\sigma-$field such that the adjacency matrix of network $G_{n}$ is
$\mathcal{C}_{n}-$measurable. Suppose further that $\left\\{Y_{n,i}\right\\}$
is conditionally $\psi-$dependent given $\left\\{\mathcal{C}_{n}\right\\}$,
with dependent coefficients $\left\\{\theta_{n}\right\\}$ satisfying Condition
NF. Then Conditions ND(a) and (b) hold.

####  3.2.2. Law of Large numbers

Let $\left\\{Y_{n,i}\right\\}$ be conditionally $\psi-$dependent given
$\left\\{\mathcal{C}_{n}\right\\}$. Since a LLN can be applied element-by-
element in the vector case, without loss of generality we can assume that
$Y_{n,i}\in\mathbb{R}$ in this sections, that is, $v=1$.

Define with,

$\displaystyle\left\lVert
Y_{n,i}\right\rVert_{\mathcal{C}_{n,p}}=\left(\mathbf{E}\left[|Y_{n,i}|^{p}|\mathcal{C}_{n}\right]\right)^{1/p}$
(3.6)

Assume the following moment condition holds.

######  Assumption 3 (Kojevnikov et al.,  (2021)).

For some $\epsilon>0$,

$\displaystyle\underset{n\geq 1}{\mathsf{sup}}\ \underset{i\in
N_{n}}{\mathsf{max}}\ \left\lVert
Y_{n,i}\right\rVert_{\mathcal{C}_{n,1+\epsilon}}<\infty,$ (3.7)

almost surely.

The next assumption puts a restriction on the denseness of the network and the
rate of decay of dependence with the network distance.

######  Assumption 4 (Kojevnikov et al.,  (2021)).

$n^{-1}\sum_{s\geq 1}\delta_{n}^{\partial}(s)\theta_{n,s}\to 0$ almost surely.

For example, Assumption 4 above can fail, for example, if there is a node
connected to almost every other node in the network as in the following
example. Consider a network with the star topology, which has a central node
or hub connected to every other node.

######  Theorem 5 (Kojevnikov et al.,  (2021)).

Suppose that $\left\\{Y_{n,i}\right\\}$ is conditionally $\psi-$dependent
given $\left\\{\mathcal{C}_{n}\right\\}$ and that the Assumptions above hold.
Then, as $n\to\infty$,

$\displaystyle\left\lVert\frac{1}{n}\sum_{i\in
N_{n}}\left(Y_{n,i}-\mathbf{E}\left[Y_{n,i}|\mathcal{C}_{n}\right]\right)\right\rVert_{\mathcal{C}_{n,1}}\to
0.$ (3.8)

An unconditional version of the result, which replaces the conditional norm in
Theorem 3.1 with the unconditional norm, can be established in a similar
manner by replacing the conditional moment with the unconditional moment.

Next, we discuss LNNs for nonlinear functions of $\left\\{Y_{n,i}\right\\}$.
When $f\in\mathcal{L}_{\nu,1}$, a LLN for a nonlinear transformation
$f\left(Y_{n,i}\right)$ follows immediately from the definition of the $\psi-$
dependence.

In that case, we have that

$\displaystyle\left\lVert\frac{1}{n}\sum_{i\in
N_{n}}\big{(}f\left(Y_{n,i}\right)-\mathbf{E}\left[f\left(Y_{n,i}\right)|\mathcal{C}_{n}\right]\big{)}\right\rVert^{2}_{\mathcal{C}_{n,2}}\leq\frac{2}{n}\left\lVert
f\right\rVert_{\infty}^{2}+\psi_{1,1}(f,f)\frac{1}{n}\sum_{s\geq
1}\delta_{n}^{\partial}(s)\theta_{n,s}.$ (3.9)

We have the following result.

######  Proposition 5 (Kojevnikov et al.,  (2021)).

Suppose that $\left\\{Y_{n,i}\right\\}$ is conditionally $\psi-$dependent
given $\left\\{\mathcal{C}_{n}\right\\}$, then if Assumption 3.2., holds and
$f\in\mathcal{L}_{v,1}$ as $n\to\infty$ we have that

$\displaystyle\left\lVert\frac{1}{n}\sum_{i\in
N_{n}}\big{(}f\left(Y_{n,i}\right)-\mathbf{E}\left[f\left(Y_{n,i}\right)|\mathcal{C}_{n}\right]\big{)}\right\rVert_{\mathcal{C}_{n,2}}\to
0\ \textit{almost surely}.$ (3.10)

However, in general nonlinear transformations of $\psi-$dependent processes
are not necessarily $\psi-$dependent. In such cases, LLNs for nonlinear
transformations can be established using the covariance inequalities for
transformation functions. For example, suppose that for some nonlinear
function $h(.)$ of a $\psi-$dependent process $\left\\{Y_{n,i}\right\\}$, and
that $\theta_{n,s}$ is bounded by a constant uniformly over $s\geq 1$ and
$n\geq 1$.

In that case for some constants $C>0$ and $p>2$, the conditional covariance
given $\mathcal{C}_{n}$ between
$\big{\\{}h(Y_{i})-\mathbf{E}\left[h(Y_{n,i})|\mathcal{C}_{n}\right]\big{\\}}$
and
$\big{\\{}h(Y_{n,j})-\mathbf{E}\left[h(Y_{n,j})|\mathcal{C}_{n}\right]\big{\\}}$
is bounded by

$\displaystyle C.\ \underset{n,i}{\text{sup}}\left\lVert
h\left(Y_{n,i}\right)\right\rVert^{2}_{\mathcal{C}_{n,p}}.\theta_{n,d_{n}(i,j)}^{1-\frac{2}{p}}.$
(3.11)

Therefore, as $n\to\infty$,

$\displaystyle\left\lVert\frac{1}{n}\sum_{i\in
N_{n}}\big{(}h\left(Y_{n,i}\right)-\mathbf{E}\left[h\left(Y_{n,i}\right)|\mathcal{C}_{n}\right]\big{)}\right\rVert_{\mathcal{C}_{n,2}}\to
0\ \text{almost surely}.$ (3.12)

provided that sup${}_{n,i}\left\lVert
h\left(Y_{n,i}\right)\right\rVert_{\mathcal{C}_{n,p}}<\infty$ almost surely.

####  3.2.3. Central limit theorem

In this section, we study the CLT for a sum of random variables that are
conditionally $\psi-$dependent. Define with

$\displaystyle\sigma_{n}^{2}=\text{Var}\left(S_{n}|\mathcal{C}_{n}\right),\ \
\ S_{n}=\sum_{i\in N_{n}}Y_{n,i}.$ (3.13)

######  Assumption 5 (Kojevnikov et al.,  (2021)).

There exists a positive sequence $m_{n}\to\infty$ such that for $k=1,2$,

$\displaystyle\frac{n}{\sigma^{2+k}}\sum_{s\geq
0}c_{n}(s,m_{n};k)\theta_{n,s}^{1-\frac{2+k}{p}}\to 0\ \ \textit{almost
surely}$ (3.14)

and

$\displaystyle\frac{n^{2}\theta_{n,m_{n}}^{1-\frac{2+k}{p}}}{\sigma_{n}}\to 0\
\ \textit{almost surely},$ (3.15)

as $n\to\infty$, where $p>4$.

It is not hard to see that Condition ND is a sufficient condition for this
assumption, when $\sigma_{n}\geq c\sqrt{n}$ with probability one, for some
constant $c>0$ that does not depend on $n$. The latter condition is satisfied
if the "long-run variance", Var $\left(S_{n}|\mathcal{C}_{n}\right)/n$ is
bounded away from $c^{2}>0$ for all $n\geq 1$. The theorem below establishes
the CLT for the normalized sum $S_{n}/\sigma_{n}$.

######  Theorem 6 (Kojevnikov et al.,  (2021)).

Suppose that Assumptions hold, and that
$\mathbf{E}\left[Y_{n,i}|\mathcal{C}_{n}\right]=0$ almost surely. Then, we
have that

$\displaystyle\underset{t\in\mathbb{R}}{\text{sup}}\left|\mathbb{P}\left\\{\frac{S_{n}}{\sigma_{n}}\leq
t\big{|}\mathcal{C}_{n}\right\\}-\Phi(t)\right|\to 0,\ \text{as}\ n\to\infty\
\ \textit{almost surely}.$ (3.16)

where $\Phi$ denotes the distribution function of $\mathcal{N}(0,1)$.

The proof of the CLT uses Stein’s Lemma and the CLT immediately gives a stable
convergence of a normalized sum of random variables under appropriate
conditions. Suppose that

$\displaystyle\sigma_{n}^{2}/\left(nv^{2}\right)\to 1\ \textit{almost
surely},$ (3.17)

where $v^{2}$ is a random variable that is $\mathcal{C}-$measurable and
$\mathcal{C}$ is a sub $\sigma-$field of $\mathcal{C}_{n}$ for all $n\geq 1$.
Then, it follows that $S_{n}/\sqrt{n}$ converges stably to a mixture normal
random variable.

####  3.2.4. Network HAC

In this section, we develop network HAC estimation of the conditional variance
of $S_{n}/\sqrt{n}$ given $\mathcal{C}_{n}$, where $S_{n}=\sum_{i\in
N_{n}}Y_{n,i}$. We assume that
$\mathbf{E}\left[Y_{n,i}|\mathcal{C}_{n}\right]=0$ almost surely for all $i\in
N_{n}$. Let

$\displaystyle\Omega_{n}(s)=n^{-1}\sum_{i\in N_{n}}\sum_{j\in
N_{n}^{\partial}(i;s)}\mathbf{E}\left[Y_{n,i}Y^{\prime}_{n,j}\right].$ (3.18)

Then, the conditional variance of $S_{n}/\sqrt{n}$ given $\mathcal{C}_{n}$ is
expressed as below

$\displaystyle
V_{n}=\text{Var}\left(S_{n}/\sqrt{n}|\mathcal{C}_{n}\right)=\sum_{s\geq
0}\Omega_{n}(s)\ \textit{almost surely}$ (3.19)

Similarly to the time-series case, the asymptotic consistency of an estimator
$V_{n}$ requires a restriction on weights given to the estimated
"autocovariance" terms $\Omega_{n}(.)$. Consider a kernel function
$\omega:\mathbb{R}\to[-1,1]$ such that $\omega(0)=1$, $\omega(z)=0$ for
$|z|>1$, and $\omega(z)=\omega(-z)$ for all $z\in\mathbb{R}$. Let $b_{n}$
denote the bandwidth or the lag truncation parameter. Then, the kernel HAC
estimator of $V_{n}$ is given by

$\displaystyle\widetilde{V}_{n}=\sum_{s\geq
0}\omega_{n}(s)\widetilde{\Omega}_{n}(s),$ (3.20)

where $\omega_{n}(s)=\omega(s/b_{n})$, and

$\displaystyle\widetilde{\Omega}_{n}(s)=n^{-1}\sum_{i\in N_{n}}\sum_{j\in
N_{n}^{\partial}(i;s)}Y_{n,i}Y_{n,j}^{\prime}.$ (3.21)

######  Remark 8.

Notice that the weight given for each sample covariance term
$\widetilde{\Omega}_{n}(s)$ is a function of distance $s$ implied by the
structure of a network. Moreover, notice that if nodes $i$ and $j$ are
disconnected then $d_{n}(i,j)=\infty$ so that
$\omega_{n}\left(d_{n}(i,j)\right)=0$. Moreover, unlike the time series case,
the number of terms included in the double sum depends on the shape of the
network. Hence, if there are many empty neighbourhood shells, a large value of
the bandwidth can still produce a HAC estimator that performs well in finite
samples. We assume that
$\mathbf{E}\left[Y_{n,i}|\mathcal{C}_{n}\right]=\Lambda_{n}$ almost surely for
all $i\in N_{n}$ and the sequence of common conditional expectations
$\left\\{\Lambda_{n}\right\\}$ is unknown.

We have that $\bar{Y}=S_{n}/n$ is a consistent estimator of $\Lambda_{n}$ such
that

$\displaystyle\mathbf{E}\left[\left\lVert\bar{Y}_{n}-\Lambda_{n}\right\rVert\big{|}\mathcal{C}_{n}\right]\to
0\ \ \textit{almost surely}.$ (3.22)

We redefine the kernel HAC estimator as follows

$\displaystyle\widehat{V}_{n}=\sum_{s\geq
0}\omega_{n}(s)\widehat{\Omega}_{n}(s),$ (3.23)

where

$\displaystyle\widetilde{\Omega}_{n}(s)=n^{-1}\sum_{i\in N_{n}}\sum_{j\in
N_{n}^{\partial}(i;s)}\left(Y_{n,i}-\bar{Y}_{n}\right)\left(Y_{n,j}-\bar{Y}_{n}\right)^{\prime}.$
(3.24)

###  3.3. Stable Limit Theorems under Network Dependence

We borrow some of the derivations presented in the framework of Lee and Song,
(2019). A relevant question of concern is: Does imposing conditions on the
degree of connectedness in the network allows to control the stability of the
nodes?

  * •

Consider a new notion of stochastic dependence among a set of random
variables. Suppose that we are given a set of random variables
$\left\\{Y_{i}\right\\}_{i\in N_{n}}$ indexed by a set $N_{n}$, where the set
$N_{n}$ is endowed with a neigborhood system so that each $i\in N_{n}$ is
associated with a subset $\nu_{n}(i)\subset N_{n}\ \left\\{i\right\\}$ called
the neighborhood of $i$. In this paper, we call the map $\nu_{n}:N_{n}\to
2^{N_{n}}$ a neighborhood system.

  * •

Given a neighborhood system $\nu_{n}$ and a set of $\sigma-$fields
$\mathcal{M}\equiv\left(\mathcal{M}_{i}\right)_{i\in N_{n}}$, we say that
$\left\\{Y_{i}\right\\}_{i\in N_{n}}$ is conditionally neighborhood dependent
(CND) with respect to $\left(\nu_{n},\mathcal{M}\right)$ if for any two non-
adjacent subsets $A$ and $B$ of $N_{n}$, $\left(\left(Y_{i}\right)_{i\in
A},\left(\mathcal{M}_{i}\right)_{i\in A}\right)$ and
$\left(\left(Y_{i}\right)_{i\in B},\left(\mathcal{M}_{i}\right)_{i\in
B}\right)$ are conditionally independent given $\left(\mathcal{M}_{i}\right)$,
where $\nu_{n}\left(A\right)$ is the union of the neighbourhoods of $i\in A$
with the set $A$ left removed.

  * •

The CND property is a generalization of both dependency graphs and Markov
random fields with a global Markov property. A set of random variables have a
graph as a dependency graph, if two sets of random variables are allowed to be
dependent only when the two sets are adjacent in the graph. This dependence
can be viewed as restrictive in many applications, as it requires that any
random variables be independent even if their indices are indirectly connected
in the graph. In contrast, CND random variables are allowed to be dependent
even if they are not adjacent in the graph.

  * •

The CND property captures the notation that "any two random variables are
independent once we condition on the source of their joint dependence". In
this sense, the CND property is closely related to a Markov property in the
literature of random fields. However, in contrast to the Markov property, the
CND property does not require that the $\sigma-$fields $\mathcal{M}_{i}$ be
generated by $Y_{i}$ itself.

####  3.3.1. Stable Convergence of an Empirical Process

Suppose that $\left\\{Y_{i}\right\\}_{i\in N_{n}}$ is a given triangular array
of $\mathbb{R}-$valued random variables with is CND with respect to
$\left(\nu_{n},\mathcal{M}\right)$. Let $\mathcal{H}$ be a given class of real
measurable functions on $\mathbb{R}$, having a measurable envelope H. Then, we
consider the following empirical process:

$\displaystyle\big{\\{}\mathbb{G}_{n}(h):h\in\mathcal{H}\big{\\}},$ (3.25)

where for each $h\in\mathcal{H}$, we have that

$\displaystyle\mathbb{G}_{n}(h)=\frac{1}{\sqrt{n}}\sum_{i\in
N_{n}}\bigg{(}h(Y_{i})-\mathbf{E}\left[h(Y_{i})|\mathcal{M}_{\nu_{n}(i)}\right]\bigg{)}.$
(3.26)

The empirical process $\nu_{n}$ takes a value in
$\ell^{\infty}\left(\mathcal{H}\right)$, the collection of bounded functions
on $\mathcal{H}$ which is endowed with the sup norm so that
$\big{(}\ell^{\infty}\left(\mathcal{H}\right),\left\lVert.\right\rVert_{\infty}\big{)}$
forms the metric space $\left(\mathbb{D},d\right)$ with the sup norm
$\left\lVert
h\right\rVert_{\infty}\equiv\text{sup}_{y\in\mathbb{R}}\left|h(y)\right|$. Lee
and Song,  (2019) explore conditions for the class $\mathcal{H}$ and the joint
distribution of the triangular array $\left\\{Y_{i}\right\\}_{i\in N_{n}}$
which delivers the stable convergence of the empirical process. Stable
convergence in completer separable metric spaces can be defined as a weak
convergence of Markov kernels. However, this definition does not extend to the
case of empirical processes taking values in $\mathbb{D}$ that is endowed with
the sup norm, due to the non-measurability. Weak convergence of an empirical
process to a Gaussian process is often established in three steps. First, they
show that the class of functions is totally bounded with respect to a certain
pseudo-metric $\rho$. Second, they show that each finite dimensional
projection of the empirical process converges in distribution to a
multivariate normal random vector. Third, they establish the asymptotic
$\rho-$equicontinuity of the empirical process.

####  3.3.2. Maximal Inequality

This subsection presents a maximal inequality in terms of bracketing entropy
bounds. The maximal inequality is useful primarily for establishing asymptotic
$\rho-$equicontinuity of the empirical process. We begin with a tail bound for
a sum of CND random variables. The following exponential tail bound is crucial
for our maximal inequality.

######  Lemma 6 (Lee and Song,  (2019)).

Suppose that $\left\\{X_{i}\right\\}_{i\in N_{n}}$ is a triangular array of
random variables that take values in $[-M,M]$ and are CND with respect to
$(\nu_{n},\mathcal{M})$, with
$\mathbf{E}\left[X_{i}|\mathcal{M}_{\nu_{n}(i)}\right]=0$, and let
$\sigma_{i}^{2}=\text{Var}\left(X_{i}|\mathcal{M}_{\nu_{n}(i)}\right)$ and
$V_{n}=\sum_{i\in N_{n}}\mathbf{E}\left[\sigma_{i}^{2}|\mathcal{G}\right]$
with $\mathcal{G}$ defined above.

Then, for any $\eta>0$ we have that

$\displaystyle\mathbb{P}\left\\{\left|\sum_{i\in
N_{n}}X_{i}\right|\geq\eta|\mathcal{G}\right\\}\leq
2\text{exp}\left(-\frac{\eta^{2}}{2(d_{mx}+1)\left[2(d_{mx}+1)V_{n}+M\frac{\eta}{3}\right]}\right)$
(3.27)

almost surely for all $n\geq 1$.

Furthermore, if Condition A holds and the condition
$\mathbf{E}\left[X_{i}|\mathcal{M}_{\nu_{n}(i)}\right]=0$ is replaced by
$\mathbf{E}\left[X_{i}|\mathcal{G}\right]=0$ and the $\sigma-$fields
$\mathcal{M}_{\nu)n(i)}$ in $\sigma_{i}^{\prime}$s are replace by
$\mathcal{G}$, then the following hods for any $\eta>0$,

$\displaystyle\mathbb{P}\left\\{\left|\sum_{i\in
N_{n}}X_{i}\right|\geq\eta|\mathcal{G}\right\\}\leq
8\text{exp}\left(-\frac{\eta^{2}}{25(d_{mx}+1)\left(V_{n}+M\frac{\eta}{3}\right)}\right)$
(3.28)

######  Corollary 1 (Lee and Song,  (2019)).

Suppose that $\left\\{Y_{i}\right\\}_{i\in N_{n}}$ is a triangular array of
random variables that are CND with respect to
$\left(\nu_{n},\mathcal{M}\right)$. Let for each $h\in\mathcal{H}$,

$\displaystyle V_{n}(h)$ $\displaystyle=n^{-1}\sum_{i\in
N_{n}}\mathbf{E}\left[\sigma_{i}^{2}(h)|\mathcal{G}\right]$ (3.29)
$\displaystyle\sigma_{i}^{2}$
$\displaystyle=\text{Var}\left(h(Y_{i})|\mathcal{M}_{\nu_{n}(i)}\right)$
(3.30)

with $\mathcal{G}$ as defined above. Then, there exists an absolute constant
$C>0$ such that

$\displaystyle\mathbf{E}$ $\displaystyle\bigg{[}\underset{1\leq s\leq
m}{\text{max}}|\mathbb{G}_{n}(h_{s})|\bigg{|}\mathcal{G}\bigg{]}\leq
C\left(d_{mx}+1\right)\left(\frac{J}{\sqrt{n}}\text{log}(1+m)+\sqrt{\text{log}(1+m)\underset{1\leq
s\leq m}{\text{max}}V_{n}(h_{s})}\right)$ (3.31)

almost surely for any $n\leq 1$ and any $m\geq 1$ with a finite subset
$\left\\{h_{1},...,h_{m}\right\\}$ of $\mathcal{H}$ such that for some
constant $J>0$, it holds that

$\displaystyle\underset{1\leq s\leq m}{\mathsf{max}}\
\underset{x\in\mathbb{R}}{\mathsf{sup}}\ \big{|}h_{s}(x)\big{|}\leq J.$

####  3.3.3. Stable Central Limit Theorem

######  Definition 10 (Lee and Song,  (2019)).

Assume that a stochastic process
$\left\\{\mathbb{G}(h):h\in\mathcal{H}\right\\}$ is a $\mathcal{G}-$mixture
Gaussian process if for any finite collection
$\left\\{h_{1},...,h_{m}\right\\}\subset\mathcal{H}$, the distribution of
random vectors

$\displaystyle\big{[}\mathbb{G}(h_{1}),...,\mathbb{G}(h_{m})\big{]}$

conditional on $\mathcal{G}$ is a multivariate normal distribution.

######  Theorem 7 (Lee and Song,  (2019)).

Suppose that $\left\\{Y_{i}\right\\}_{i\in N_{n}}$ is a triangular array of
random variables that are CND with respect to
$\left(\nu_{n},\mathcal{M}\right)$, satisfying Assumption 3.1. Suppose further
that there exists $C>0$ such that for each $n\geq 1$,
$\text{max}_{i\in\mathcal{N}_{n}}\ \mathbf{E}\left[H(Y_{i})^{4}\right]<C$,
where $H$ is an envelope of $\mathcal{H}$.

Then $\nu_{n}$ converges to a $\mathcal{G}-$mixture Gaussian process
$\mathbb{G}$ in $\ell^{\infty}\left(\mathcal{H}\right)$, $\mathcal{G}-$stably,
such that for any

$\displaystyle
h_{1},h_{2}\in\mathcal{H},\mathbf{E}\left[\mathbb{G}(h_{1})\mathbb{G}(h_{2})|\mathcal{G}\right]=K\left(h_{1},h_{2}|\mathcal{G}\right)$
(3.32)

almost surely.

######  Remark 9.

The two main theorems which are considered to be the building tools when
developing asymptotic theory in time series regression models, are the
continuous mapping theorem as well as the implementation of a suitable
invariance law for partial sum processes; both are used to establish the weak
convergence theory. Furthermore, empirical processes have been extensively
used the past decades to help in the development of asymptotic statistics of
various inference problems. However, one area which has not been studied in
the literature in great length is a modification for empirical processes to
accommodate network dependence in data.

###  3.4. Moderate Deviations: Eigenvalues of Covariance Matrices

Although due to the convenient structure of the covariance matrix, the
estimation procedure of the entries of the matrix is many times overlooked
therefore the spectral analysis of high dimensional matrices mainly focused on
the asymptotic behaviour of the eigenvalues without taking into consideration
how the entries are estimated. In our framework we give emphasis on the
estimation of the risk matrix with the use of quantile predictive regression
models which clearly accounts for more features regarding the stationarity and
persistence properties of the time series. In practise, the estimation of the
covariance matrix with the conventional methodologies which consider the use
of a high dimensional vector does not account for much features of the time
series under examination (see, Katsouris,  (2021); Katsouris, 2023c ).

According to Zhang et al.,  (2020), in recent years, large sample properties
for high-dimensional sample covariance matrices, including their eigenvalues
and eigenvectors, have been proved to be useful. In fact, random matrix theory
(RMT) provides a plethora of useful methods for estimation and testing in high
dimensional environments. Currently, there are two steams of literature about
asymptotic theory of the largest eigenvalues of high dimensional random
matrices. The first stream of literature is concerned with the Tracey-Widom
law. In particular, it is well known that the limiting distribution of the
largest eigenvalues of high-dimensional random matrices, such as Wigner
matrices, follow the Tracy-Widom law which holds for Gaussian Wigner
ensembles. Moreover, the second stream of literature is interested in the
asymptotic behaviour of spiked eigenvalues.

Most of the existing studies operate under the assumption that observations of
high dimensional data are independent. However, applications in finance and
economics with high dimensional settings involve data which can be temporally
dependent or even nonstationary. For instance, the first framework that
examines the asymptotic behaviour of the largest eigenvalues of sample
covariance matrices generated from high-dimensional nonstationary time series
is proposed by Zhang et al.,  (2018). Investigating the asymptotic behaviour
of the largest eigenvalues of random matrix models is important in
understanding the stability behaviour of these systems. See Remark 4 (page 34)
of Grama and Haeusler,  (2006) who mention: "The key point in the proof when
considering the eigenvalues of a covariance or state matrix, is the fact that
the eigenvalues of a matrix depend continuously on its elements. In practise,
this means that the maximal eigenvalue of a matrix is a predictable discrete
time process in $t$ and therefore the random times $\tau_{m}$ are predictable
stopping times".

####  3.4.1. Limit of the smallest eigenvalue of a Large Sample Covariance
Matrix

The related framework is presented by Bai and Silverstein,  (1999).

######  Lemma 7.

We assume that the entries of $X_{n}$ have already been truncated at
$\delta\sqrt{n}$ for some slowly varying $\delta=\delta_{n}\to 0$. We define
with

$\displaystyle
V_{ij}=X_{ij}\mathbf{1}\big{(}\left|X_{ij}\right|\leq\delta\sqrt{n}\big{)}-\mathbb{E}\left[X_{ij}\mathbf{1}\big{(}\left|X_{ij}\right|\leq\delta\sqrt{n}\big{)}\right].$
(3.33)

Moreover, it has been proved that

$\displaystyle\sum_{i=1}^{p}\lambda_{i}\eta_{i}\geq\text{trace}\left(A^{\prime}B\right),$
(3.34)

if $A$ and $B$ are $p\times n$ matrices with singular values
$\lambda_{1}\geq...\geq\lambda_{p}$ and $\eta_{1}\geq...\geq\eta_{p}$,
respectively. Therefore, using the von Neumann’s inequality, we have that

$\displaystyle\left|\lambda_{\text{min}}^{1/2}\left(n^{1/2}\hat{X}_{n}\hat{X}^{\prime}_{n}\right)-\lambda_{\text{min}}^{1/2}\left(n^{1/2}\hat{V}_{n}\hat{V}^{\prime}_{n}\right)\right|$
$\displaystyle\leq\sum_{i=1}^{p}\left\\{\lambda_{k}^{1/2}\left(n^{1/2}\hat{X}_{n}\hat{X}^{\prime}_{n}\right)-\lambda_{k}^{1/2}\left(n^{1/2}\hat{V}_{n}\hat{V}^{\prime}_{n}\right)\right\\}^{2}$
$\displaystyle\leq\frac{1}{n}\text{trace}\left(\hat{X}_{n}-V_{n}\right)\left(\hat{X}_{n}-V_{n}\right)^{\prime}$
$\displaystyle\leq
p\mathbb{E}^{2}\left|X_{11}\right|\mathbf{1}\left(\left|X_{11}\right|>\delta\sqrt{n}\right)\to
0,$

where $\hat{X}_{n}$ and $V$ are $n\times p$ matrices with $(u,v)-$th entries
$X_{u,v}\mathbf{1}\left(\left|X_{11}\right|\leq\delta\sqrt{n}\right)$ and
$V_{uv}$, respectively. The above convergence is true provided that
$n\delta^{3}\to 0$. Therefore, we assume that for each $n$ the entries
$X_{uv}=X_{uv}(n)$ of the matrix $X_{n}$ are i.i.d and satisfy

$\displaystyle\mathbb{E}X_{uv}=0,\ \ \mathbb{E}X^{2}_{uv}\leq 1\ \ \text{and}\
\ \mathbb{E}X^{2}_{uv}\to 1\ \ \text{as}\ \ n\to\infty,$ (3.35)

##### Proof of Lemma 3 of the paper

The following inequality holds,

$\displaystyle\left\lVert\frac{1}{n}X^{(1)}X^{(1)\prime}\right\rVert$
$\displaystyle\leq\left\lVert
T(1)\right\rVert+\left\lVert\text{diag}\left[\frac{1}{n}\sum_{j=1}^{n}X_{ij}^{2},\
i=1,...,p\right]\right\rVert$ $\displaystyle\leq\left\lVert
T(1)\right\rVert+\frac{1}{n}\underset{i\leq
p}{\text{max}}\sum_{j=1}^{n}X_{ij}^{2}.$

####  3.4.2. Limit Theory for the largest eigenvalues of sample covariance
matrices with heavy-tails

Theory that consistently estimates the spectrum of a large dimensional
covariance matrix using RMT. Thus, statistical considerations will be our
motivation for a random matrix model with heavy-tailed and dependent entries.
Notice that results on the global behaviour of the eigenvalues of $XX^{\top}$
mostly concern the spectral distribution, that is, the random probability
measure of its eigenvalues
$p^{-1}\sum_{i=1}^{p}\epsilon_{n^{-1}\lambda_{(i)}}$, where $\epsilon$ denotes
the Dirac measure. The spectral distribution converges, as $n,p\to\infty$ with
$p/n\to\gamma\in(0,1]$, to a deterministic measure with density function

$\displaystyle\frac{1}{2\pi
x\gamma}\sqrt{(x_{+}-x)(x-x_{-})}\mathbf{1}(x_{-},x_{+})(x),$ (3.36)

This is the so-called Marcenko-Pastur law. Therefore, one obtains a different
result if $XX^{\top}$ is perturbed via an affine transformation. Although the
eigenvalues of $XX^{\top}$ offer various interesting local properties to be
examined, we will only focus on the joint asymptotic behaviour of the $k$
largest eigenvalues
$\left(\lambda_{(1)},...,\lambda_{(k)}\right),k\in\mathbb{N}$. This is
motivated from a statistical point of view since the variances of the first
$k$ principal components are given by the $k$ largest eigenvalues of the
covariance matrix. Therefore, a Marcenko-Pastur type result holds for the
asymptotics of the spectrum of heavy-tailed random matrices.

####  3.4.3. Limit Spectral Distribution for symmetric random matrices with
correlated entries

Let $\left(X_{k,\ell}\right)_{(k,\ell)\in\mathbb{Z}^{2}}$ be an array of real-
valued random variables, and consider its associated systemic random matrix
$\mathbf{X}_{n}$ of order $n$ defined by

$\displaystyle\left(\mathbf{X}_{n}\right)=X_{i,j},\ \text{if}\ 1\leq j\leq
i\leq n\ \ \ \text{and}\ \ \ \left(\mathbf{X}_{n}\right)=X_{j,i},\ \text{if}\
1\leq i\leq j\leq n$ (3.37)

We then define

$\displaystyle\mathbb{X}_{n}:=n^{-1/2}\mathbf{X}_{n}.$ (3.38)

The aim of this section is to study the limiting spectral empirical
distribution function of the symmetric matrix $\mathbf{X}_{n}$ when the
process $\left(X_{k,\ell}\right)_{(k,\ell)\in\mathbb{Z}^{2}}$ has the
following dependence structure: for any $(k,\ell)\in\mathbb{Z}^{2}$,

$\displaystyle
X_{k,\ell}=g\left(\xi_{k-i,\ell-j}:(i,j)\in\mathbb{Z}^{2}\right),$ (3.39)

where $\left(\xi_{i,j}\right)_{(i,j)\in\mathbb{Z}^{2}}$ is an array of i.i.d
real-valued random variables given on a common probability space
$\left(\Omega,\mathcal{F},\mathbb{P}\right)$ and $g$ is a measurable function
$\mathbb{R}^{\mathbb{Z}^{2}}\mapsto\mathbb{R}$ such that
$\mathbb{E}\left(X_{0,0}\right)=0$ and $\left\lVert
X_{0,0}\right\rVert_{2}<\infty$.

The Theorem below shows a universality scheme for the random matrix
$\mathbb{X}_{n}$ when the entries of the symmetric matrix
$\sqrt{n}\mathbb{X}_{n}$ have the above dependence structure. Notice that the
particular result does not require rate of convergence to zero of the
correlation between the entries of the risk matrix.

######  Theorem 8.

Let $\left(X_{k,\ell}\right)_{(k,\ell)\in\mathbb{Z}^{2}}$ be a real-valued
stationary random field. Define the symmetric matrix $\mathbf{X}_{n}$. let
$\left(G_{k,\ell}\right)_{(k,\ell)\in\mathbb{Z}^{2}}$ be a real-valued centred
Gaussian random field, with covariance function given by

$\displaystyle\mathbb{E}\left(G_{k,\ell}G_{i,j}\right)=\mathbb{E}\left(X_{k,\ell}X_{i,j}\right)\
\ \text{for any}\ \ (k,\ell)\ \text{and}\ (i,j)\ \text{in}\ \mathbb{Z}^{2}.$
(3.40)

Let $\mathbf{G}_{n}$ be the symmetric random matrix defined by
$\left(\mathbf{G}_{n}\right)_{i,j}=G_{i,j}$ if $1\leq j\leq i\leq n$ and
$\left(\mathbf{G}_{n}\right)_{i,j}=G_{j,i}$ if $1\leq i\leq j\leq n$. Denote
with $\mathbb{G}_{n}=\frac{1}{\sqrt{n}}\mathbf{G}_{n}$. Then, for any
$z\in\mathbb{C}^{+}$,

$\displaystyle\underset{n\to\infty}{\text{lim}}\left|S_{\mathbb{X}_{n}}(z)-\mathbb{E}\left(S_{\mathbb{X}_{n}}(z)\right)\right|=0,\
\text{almost surely}.$ (3.41)

The above theorem is important since it shows that the study of the limiting
spectral distribution function of a symmetric matrix whose entries are
functions of i.i.d random variables can be reduced to studying the same
problem as for a Gaussian matrix with the same covariance structure.

##### Proof of Theorem 8

###### Proof.

For $m$ a positive integer (fixed for the moment) and for any
$(u,v)\in\mathbb{Z}^{2}$ define

$\displaystyle
X_{u,v}^{(m)}=\mathbb{E}\left(X_{u,v}|\mathcal{F}_{u,v}^{(m)}\right)$ (3.42)

where $mathcal{F}_{u,v}^{(m)}:=\sigma\left(\xi_{i,j}:u-m\leq i\leq u+m,v-m\leq
j\leq v+m\right)$.

Let $\mathbf{X}_{n}^{(m)}$ be the symmetric random matrix of order $n$
associated with $\left(X_{u,v}^{(m)}\right)_{(u,v)\in\mathbb{Z}^{2}}$ and
defined by $\left(\mathbf{X}_{n}^{(m)}\right)_{i,j}=X_{i,j}^{(m)}$ if $1\leq
j\leq i\leq n$ and $\left(\mathbf{X}_{n}^{(m)}\right)_{i,j}=X_{j,i}^{(m)}$ if
$1\leq i\leq j\leq n$. Let
$\mathbb{X}_{n}^{(m)}=n^{-1/2}\mathbf{X}_{n}^{(m)}$.

We first show that, for any $z\in\mathbb{C}^{+}$,

$\displaystyle\underset{m\to\infty}{\text{lim}}\underset{n\to\infty}{\text{lim
sup}}\big{|}S_{\mathbb{X}_{n}}(z)-\mathbb{E}\left(S_{\mathbb{X}_{n}}(z)\right)\big{|}=0,\
\text{almost surely},$ (3.43)

Moreover, we have that

$\displaystyle\big{|}S_{\mathbb{X}_{n}}(z)-\mathbb{E}\left(S_{\mathbb{X}_{n}}(z)\right)\big{|}^{2}\leq\frac{2}{n^{2}v^{4}}\sum_{1\leq\ell\leq
k\leq n}\left(X_{k,\ell}-X_{k,\ell}^{(m)}\right)^{2}.$ (3.44)

Since the shift is ergodic with respect to the measure generated by a sequence
of i.i.d random variables and the sets of summations are on regular sets, the
ergodic theorem entails that

$\displaystyle\underset{n\to\infty}{\text{lim}}\sum_{1\leq k,\ell\leq
n}\left(X_{k,\ell}-X_{k,\ell}^{(m)}\right)^{2}=\mathbb{E}\left(\left(X_{0,0}-X_{0,0}^{(m)}\right)^{2}\right)\
\ \text{almost surely}.$ (3.45)

Therefore,

$\displaystyle\underset{n\to\infty}{\text{lim
sup}}\left|S_{\mathbb{X}_{n}}(z)-\mathbb{E}\left(S_{\mathbb{X}_{n}}(z)\right)\right|^{2}\leq
2v^{-4}\left\lVert X_{0,0}-X_{0,0}^{(m)}\right\rVert_{2}^{2}\ \text{almost
surely}.$ (3.46)

But by the martingale convergence theorem

$\displaystyle\left\lVert X_{0,0}-X_{0,0}^{(m)}\right\rVert_{2}\to 0\
\text{as}\to\infty,$ (3.47)

∎

###  3.5. Moderate Deviations for Extreme Eigenvalues

According to Jiang et al.,  (2021) the Marcehnko-Pastur Law implies that the
empirical spectral distribution of $W$ converges to a deterministic
distribution with support
$\left[\left(1-\sqrt{\beta}\right)_{+}^{2},\left(1+\sqrt{\beta}\right)^{2}\right]$,
where $x_{+}=\text{max}\left\\{0,x\right\\}$.

######  Theorem 9.

Let $k\to\infty$ as $n\to\infty$. Moreover, suppose that each of the entries
$c_{ij}$ is symmetric around 0 and $|c_{ij}|<M<\infty$ almost surely or
$c_{ij}$ is standard normal. If $\text{Var}\left(c_{11}^{2}\right)>0$, we have

  1. (a)

(Moderate deviation for $\lambda_{\text{min}}$) For any $\nu\leq 0$,

$\displaystyle\underset{n\to\infty}{\text{lim}}\frac{1}{\ell_{n}^{2}}\text{log}\mathbb{P}\left(\ell_{n}^{-1}n^{1/2}\left(\lambda_{\text{min}}-1\right)\leq\nu\right)=-\frac{1}{2\sigma_{\infty}}\nu^{2};$
(3.48)

  2. (a)

(Moderate deviation for $\lambda_{\text{max}}$) For any $\nu\geq 0$,

$\displaystyle\underset{n\to\infty}{\text{lim}}\frac{1}{\ell_{n}^{2}}\text{log}\mathbb{P}\left(\ell_{n}^{-1}n^{1/2}\left(\lambda_{\text{max}}-1\right)\geq\nu\right)=-\frac{1}{2\sigma_{\infty}}\nu^{2}.$
(3.49)

In both cases,
$\sigma_{\infty}^{2}=\text{lim}_{k\to\infty}\sigma_{k}^{2}=\text{max}\left\\{2,\text{Var}(c_{11}^{2})\right\\}$.

As a corollary we can obtain the following moderate deviation for the
condition number $\lambda_{\text{max}}/\lambda_{\text{min}}$, which is
important quantity that can indicate whether the matrix is ill-conditioned.

####  3.5.1. Moderate Deviations for $\lambda_{\text{max}}$,
$\lambda_{\text{min}}$ with fixed $k$

Suppose there exist two sequences $b_{n}\left(b_{n}\to\infty\right)$, $r_{n}$
so that a family of random variables $\left\\{Z_{n},n\geq 0\right\\}$ with
values in topological vector space $\mathcal{X}$ (equipped with $\sigma-$field
$\mathcal{B}$) satisfies a fluctuation theorem, say $b_{n}(Z_{n}-r_{n})$
converges to some nontrivial distribution. We say $Z_{n}$ satisfies the
moderate deviation principle with speed $\lambda_{n}\to\infty$ and with good
rate function $I(.)$ if the level sets $\left\\{I\leq\ell\right\\}$ are
compact for all $\ell>0$ and for any closed set $F$ and open set
$G\in\mathcal{B}$,

$\displaystyle\underset{n\to\infty}{\text{lim
sup}}\frac{1}{\lambda_{n}}\text{log}\mathbb{P}\left(\tilde{b}_{n}(Z_{n}-r_{n})\in
F\right)\leq-\underset{x\in F}{\text{inf}}I(x)$
$\displaystyle\underset{n\to\infty}{\text{lim
sup}}\frac{1}{\lambda_{n}}\text{log}\mathbb{P}\left(\tilde{b}_{n}(Z_{n}-r_{n})\in
G\right)\geq-\underset{x\in G}{\text{inf}}I(x).$

Here, $\tilde{b}_{n}$ is a sequence satisfying $\tilde{b}_{n}\to\infty$ and
$\tilde{b}_{n}/b_{n}\to 0$. The form of rate function $I(.)$ is closely
related to the limit distribution of $b_{n}(Z_{n}-r_{n})$. Therefore, if the
rate function is continuous we have that

$\displaystyle\underset{n\to\infty}{\text{lim}}\frac{1}{\lambda_{n}}\text{log}\mathbb{P}\left(\tilde{b}_{n}(Z_{n}-r_{n})\leq
x\right)=-\underset{u\leq x}{\text{inf}}I(u)$
$\displaystyle\underset{n\to\infty}{\text{lim}}\frac{1}{\lambda_{n}}\text{log}\mathbb{P}\left(\tilde{b}_{n}(Z_{n}-r_{n})\geq
y\right)=-\underset{u\geq x}{\text{inf}}I(u)$

####  3.5.2. Discussion on High Dimensional Results

The literature has indeed documented the inconsistency of sample covariance
matrices for especially high-dimensional settings. In order to be able to
investigate the features of our novel tail risk matrix, it is necessary to
consider the distribution theory of the entries of the matrix. Overall to
assess the quality of a matrix estimate, we can use the norm operator such as
the spectral radius. Below we present some useful results (e.g. see, Chen et
al.,  (2013)).

######  Lemma 8.

Let $Z_{i}$ be i.i.d
$\mathcal{N}\left(\mathbf{0},\boldsymbol{\Sigma}_{p}\right)$ and
$\lambda_{\text{max}}\left(\boldsymbol{\Sigma}_{p}\right)\leq\bar{k}<\infty$.
Then if $\boldsymbol{\Sigma}_{p}=[\sigma_{ab}]$,

$\displaystyle\mathbb{P}\left[\left|\sum_{i=1}^{n}\left(Z_{ij}Z_{ik}-\sigma_{jk}\right)\right|\geq
n\nu\right]\leq c_{1}\text{exp}\left(-c_{2}n\nu^{2}\right)\ \ \ \text{for}\ \
\ |\nu|\leq\delta$ (3.50)

where $c_{1},c_{2}$ and $\delta$ depend on $\bar{k}$ only.

Suppose we have $n$ temporally observed $p-$dimensional vectors
$\left(\mathbf{z}_{i}\right)_{i=1}^{n}$ having mean zero and covariance matrix
$\Sigma_{i}=\mathbb{E}\left(\mathbf{z}_{i},\mathbf{z}_{i}\right)$ whose
dimension is $p\times p$. Our goal is to estimate the covariance matrices
$\Sigma_{i}$ and their inverses $\Omega_{i}=\Sigma_{i}^{-1}$ based on the data
matrix $Z_{p\times n}=\left(\mathbf{z}_{1},...,\mathbf{z}_{n}\right)$. In the
classical situation where $p$ is fixed, $n\to\infty$ and $\mathbf{z}_{i}$ are
mean zero independent and identically distributed i.i.d random vectors.

In particular, it is well known that the sample covariance matrix
$\hat{\boldsymbol{\Sigma}}_{n}=\frac{1}{n}\sum_{i=1}^{n}\boldsymbol{X}_{i}\boldsymbol{X}_{i}^{\top}$,
is a consistent and well behaved estimator of $\boldsymbol{\Sigma}$, and
$\hat{\boldsymbol{\Omega}}_{n}=\hat{\boldsymbol{\Sigma}}_{n}^{-1}$ is a
natural and good estimator of $\boldsymbol{\Omega}$. However, when the
dimensionality $p$ grows with $n$, then $\hat{\boldsymbol{\Sigma}}_{n}$ is no
longer a consistent estimate of $\boldsymbol{\Sigma}$ in the sense that its
eigenvalues do not converge to those of $\boldsymbol{\Sigma}$. Moreover, it is
clear that $\hat{\boldsymbol{\Omega}}_{n}$ is not defined when
$\hat{\boldsymbol{\Sigma}}_{n}$ is not invertible in the high-dimensional case
with $p>>n$. Define with

$\displaystyle
T_{u}\left(\widehat{\boldsymbol{\Sigma}}_{u}\right)=\mathbf{Q}\hat{\boldsymbol{\Lambda}}\mathbf{Q}^{\top}=\sum_{j=1}^{p}\hat{\lambda}_{j}\mathbf{q}_{j}\mathbf{q}_{j}^{\top}$

the eigen-decomposition, where $\mathbf{Q}$ is an orthonormal matrix and
$\hat{\boldsymbol{\Lambda}}$ is a diagonal matrix. For $v>0$,

$\displaystyle\tilde{S}_{v}=\sum_{j=1}^{p}\left(\hat{\lambda}_{j}\vee
v\right)\mathbf{q}_{j}\mathbf{q}_{j}^{\top},$ (3.51)

where $0<v\leq\sqrt{p}\bar{\omega}$ and $\omega^{2}$ is the rate of
convergence. Let $\mu_{1},...,\mu_{p}$ be the diagonal elements of
$\mathbf{Q}^{\top}\boldsymbol{\Sigma}\mathbf{Q}$. Then, we have by Theorem 2.1
that $\sum_{j=1}^{p}\left(\hat{\lambda}_{j}-\mu_{j}\right)^{2}\leq
p^{2}\bar{\omega}^{2}$, and consequently

$\displaystyle\left|\tilde{S}_{v}-\boldsymbol{\Sigma}\right|^{2}_{F}$
$\displaystyle\leq
2\left|\tilde{S}_{v}-T_{u}\left(\hat{\boldsymbol{\Sigma}}\right)\right|^{2}_{F}+2\left|T_{u}\left(\hat{\boldsymbol{\Sigma}}_{u}\right)-\boldsymbol{\Sigma}\right|_{F}^{2}$
$\displaystyle\leq
2\sum_{j=1}^{p}\left(\hat{\lambda}_{j}-\left(\hat{\lambda}_{j}\vee
v\right)\right)^{2}+2\bar{\omega}^{2}p^{2}$ $\displaystyle\leq
2\sum_{j=1}^{p}\left(2\hat{\lambda}_{j}^{2}\mathbf{1}\left\\{\hat{\lambda}_{j}\leq
0\right\\}+2v^{2}\right)+2\bar{\omega}^{2}p^{2}.$

If $\hat{\lambda}_{j}\leq 0$, since $\mu_{i}\geq 0$, we have that
$\left|\hat{\lambda}_{j}\right|\leq\left|\hat{\lambda}_{j}-\mu_{i}\right|$.
Then,

$\displaystyle\left|\tilde{S}_{v}-\Sigma\right|_{F}^{2}\leq
4v^{2}p+6\bar{\omega}p^{2}\leq 10\bar{\omega}^{2}p^{2}.$ (3.52)

The eigenvalues of $\tilde{S}_{v}$ are bounded below by $v$, and thus it is
positive definite. Suppose that

$\displaystyle
v=\left(p^{-1}\sum_{j,k=1}^{p}u^{2}\times\mathbf{1}\left\\{\left|\hat{\sigma}_{jk}\right|\geq
u\right\\}\right)^{1/2}$

Thus, the same positive-definization procedure also applies to the spectral
norm. We obtain that the differences between the eigenvalues and eigenvectors
of $\Sigma_{m}$ and $\hat{\Sigma}_{m}$ can be bounded by the following

$\displaystyle\big{(}\hat{\lambda}_{r,m}-\lambda_{r}\big{)}=\text{trace}\big{\\{}e_{r,m}e_{r,m}^{\top}\left(\hat{\boldsymbol{\Sigma}}_{m}-\boldsymbol{\Sigma}\right)\big{\\}}+\tilde{R}_{r,m},\
\ \ \tilde{R}_{r,m}\leq\frac{\displaystyle 6\underset{\left\lVert
a\right\rVert=1}{\text{sup}}a^{\top}\left(\hat{\Sigma}_{m}-\Sigma\right)^{2}a}{\displaystyle\underset{s}{\text{min}}\left|\lambda_{s}-\lambda_{r}\right|}$
(3.53)

Moreover, we have that

$\displaystyle\big{(}\hat{\zeta}_{r,m}-e_{r,m}\big{)}=-S_{r,m}\left(\hat{\Sigma}_{m}-\Sigma\right)e_{r,m}+R_{r,m}^{*},$
(3.54)

with

$\displaystyle\left\lVert R_{r,m}^{*}\right\rVert\leq\frac{\displaystyle
6\underset{\left\lVert
a\right\rVert=1}{\text{sup}}a^{\top}\left(\hat{\Sigma}_{m}-\Sigma\right)^{2}a}{\displaystyle\underset{s}{\text{min}}\left|\lambda_{s}-\lambda_{r}\right|},$
(3.55)

where we denote with $S_{r,m}=\sum_{s\neq
r}\frac{1}{\lambda_{s}-\lambda_{r}}e_{s,m}e_{s,m}^{\top}$. Assumption 1
implies that $\mathbb{E}\left(\hat{\beta}_{r}\right)=0$ and
Var$\left(\hat{\beta}_{r}\right)=\frac{\lambda_{r}}{r}$ and with
$\delta_{ii}=1$ as well as $\delta_{ij}=0$ for $i\neq j$, we obtain that

$\displaystyle\mathbb{E}\left\\{\underset{\left\lVert
a\right\rVert=1}{\text{sup}}a^{\top}\left(\hat{\boldsymbol{\Sigma}}_{m}-\boldsymbol{\Sigma}\right)\right\\}$
$\displaystyle\leq\mathbb{E}\bigg{\\{}\text{trace}\left[\left(\hat{\boldsymbol{\Sigma}}_{m}-\boldsymbol{\Sigma}\right)^{2}\right]\bigg{\\}}$
$\displaystyle=\mathbb{E}\left\\{\sum_{j,k=1}^{m}\left[\frac{1}{n}\left(\beta_{ji}-\bar{\beta}_{j}\right)\left(\beta_{ki}-\bar{\beta}_{k}\right)-\delta_{jk}\lambda_{j}\right]^{2}\right\\}$
$\displaystyle\leq\mathbb{E}\left\\{\sum_{j,k=1}^{\infty}\left[\frac{1}{n}\left(\beta_{ji}-\bar{\beta}_{j}\right)\left(\beta_{ki}-\bar{\beta}_{k}\right)-\delta_{jk}\lambda_{j}\right]^{2}\right\\}$
$\displaystyle=\frac{1}{n}\left(\sum_{j}\sum_{k}\mathbb{E}\left(\beta_{ji}^{2}\beta_{ki}^{2}\right)\right)+o(n^{-1})=\mathcal{O}\left(n^{-1}\right)$

for all $m$.

Furthermore, since

$\displaystyle\mathsf{trace}\big{\\{}e_{r,m}e_{r,m}^{\top}\left(\hat{\boldsymbol{\Sigma}}_{m}-\boldsymbol{\Sigma}_{m}\right)\big{\\}}=\frac{1}{n}\sum_{i=1}^{n}\left(\beta_{ri}-\bar{\beta}_{r}\right)^{2}-\lambda_{r}$
(3.56)

Therefore after applying the central limit theorem we obtain that

$\displaystyle\sqrt{n}\left(\hat{\lambda}_{r}-\lambda_{r}\right)$
$\displaystyle=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left(\beta_{ri}-\hat{\beta}_{r}\right)^{2}-\lambda_{r}+\mathcal{O}_{p}\left(n^{-1/2}\right)$
$\displaystyle=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left\\{\left(\beta_{ri}\right)^{2}-\mathbb{E}\left[\left(\beta_{ri}\right)^{2}\right]\right\\}+\mathcal{O}_{p}\left(n^{-1/2}\right)\to\mathcal{N}\left(0,\boldsymbol{\Lambda}_{r}\right).$

Obviously the event
$\hat{\lambda}_{r-1}>\hat{\lambda}_{r}>\hat{\lambda}_{r+1}$ occurs with
probability 1.

##  4\. Nonstationary Time Series Regressions

###  4.1. Testing for unit root in time series regression

######  Example 9.

Consider the zero-mean Gaussian AR(1) model where $\left\\{y_{t}:1\leq t\leq
T\right\\}$ generated as below

$\displaystyle y_{t}=\rho y_{t-1}+\epsilon_{t},$ (4.1)