#  BOOTSTRAPPING NONSTATIONARY AUTOREGRESSIVE PROCESSES
WITH PREDICTIVE REGRESSION MODELS††thanks: Article history: November 2020,
February 2022, July 2023. MSC2020 classifications: 62F05, 62F12, 62M10.
Keywords: nonstationary autoregression, IVX filtration, bootstrap asymptotics,
mixed gaussian distribution.

BY CHRISTIS KATSOURIS111Lecturer in Economics, Department of Economics,
University of Exeter Business School, Exeter EX4 4PU, United Kingdom. E-mail
Address<EMAIL_ADDRESS>
University of Southampton and University of Exeter

We establish the asymptotic validity of the bootstrap-based IVX estimator
proposed by Phillips and Magdalinos,  (2009) for the predictive regression
model parameter based on a local-to-unity specification of the autoregressive
coefficient which covers both nearly nonstationary and nearly stationary
processes. A mixed Gaussian limit distribution is obtained for the bootstrap-
based IVX estimator. The statistical validity of the theoretical results are
illustrated by Monte Carlo experiments for various statistical inference
problems.

##  Introduction.

Consider the first-order autoregressive process
$\left\\{X_{t}\right\\}_{t=1}^{+\infty}$, defined by the following recursive
process

(1.1) $\displaystyle X_{t}=\vartheta_{n}X_{t-1}+\varepsilon_{t},\ \ \ \
X_{0}=0,$

where $\left\\{\varepsilon_{t}\right\\}$ are independent $\mathcal{N}(0,1)$
random sequences. The least squares estimator $\hat{\vartheta}_{n}$ of
$\vartheta$, based on a sample of $n$ observations
$\left\\{X_{1},...,X_{n}\right\\}$ is given by the following expression

(1.2)
$\displaystyle\hat{\vartheta}_{n}=\left(\sum_{t=1}^{n}X^{2}_{t-1}\right)^{-1}\left(\sum_{t=1}^{n}X_{t-1}X_{t}\right).$

A well known asymptotic result is that $\hat{\vartheta}_{n}$ is a consistent
estimator for $\vartheta_{n}$ for all values of
$\vartheta_{n}\in\left(-\infty,+\infty\right)$. Specifically, the asymptotic
distribution of $\hat{\vartheta}_{n}$ depends on restrictions imposed on the
admissible parameter space of $\vartheta$. In particular, in the stable case,
which implies that the parameter space of $\vartheta$ takes values within the
unit circle, that is, $|\vartheta|<1$, various seminal studies such as Mann
and Wald,  (1943) and Anderson,  (1959) among others, have shown that

(1.3)
$\displaystyle\sqrt{n}\left(\hat{\vartheta}_{n}-\vartheta_{n}\right)\overset{d}{\to}\mathcal{N}\big{(}0,1-\vartheta^{2}\big{)}\
\ \ \text{as}\ \ n\to\infty.$

When the true parameter lies on the boundary of the parameter space, such that
$|\vartheta|=1$, it has been shown by White,  (1958) and Phillips, 1987b that
the following limiting distribution holds

(1.4) $\displaystyle
n\left(\hat{\vartheta}_{n}-\vartheta_{n}\right)\overset{d}{\to}\left(\int_{0}^{1}W(r)^{2}ds\right)^{-1}\left(\int_{0}^{1}W(r)dW(r)\right)\
\ \ \text{as}\ \ n\to\infty,$

where $W(r)$ for some $0\leq r\leq 1$ is a standard Wiener process within the
probability space $\left(\Omega,\mathbb{P},\mathcal{F}_{t}\right)$. On the
other hand, when the true parameter of the autoregressive process given by
expression (1.1) is outside the unit circle (explosive parameter region), such
that $|\vartheta|>1$, it has been shown by White,  (1958) and Anderson,
(1959) that the following asymptotic result holds

(1.5)
$\displaystyle\sqrt{n}\left(\hat{\vartheta}_{n}-\vartheta_{n}\right)\overset{d}{\to}\text{Cauchy}\left(0,\vartheta^{2}-1\right)\
\ \ \text{as}\ \ n\to\infty.$

The particular limiting distribution discontinuity with respect to the
parameter space of the autoregressive model of order one has been extensively
discussed in the literature (see, Bercu,  (2001), Jeganathan,  (1991),
Phillips and Magdalinos,  (2007), Proïa,  (2020), Benke and Pap,  (2021)).
Without loss of generality, considering for each $n\in\mathbb{N}$, the
statistical experiment $\mathcal{E}_{n}$ corresponding to the observations
$\left\\{X_{0},...,X_{n}\right\\}$, then the sequence
$\left(\mathcal{E}_{n}\right)_{n\in\mathbb{N}}$ is locally asymptotically
normal (LAN) when $|\vartheta|<1$, it is locally asymptotically Brownian
functional (LABF) when $|\vartheta|=1$, and it is locally asymptotically mixed
normal (LAMN) when $|\vartheta|>1$ (see, Le Cam,  (1986), Le Cam and Yang,
(2000) and Phillips,  (1989)). Thus, statistical inference procedures can
crucially depend on the true value of the population parameter $\theta_{n}$ in
certain regions of the parameter space.

Motivated by these observations various studies have focus on the development
of estimation and inference methodologies for nonstationary time series models
(see, Rao,  (1978), Dickey and Fuller,  (1979), Chan and Wei,  (1987, 1988),
Phillips, 1987b ; Phillips, 1987a ; Phillips, 1988a ; Phillips, 1988b ,
Phillips and Perron,  (1988), Abadir,  (1993), Larsson,  (1995), Buchmann et
al.,  (2007) and Phillips and Magdalinos,  (2007) and Horváth and Kokoszka,
(2003)). Specifically, the class of nearly nonstationary time series models as
proposed by the seminal studies of Phillips, 1987b ; Phillips, 1987a who
developed asymptotics for near-integrated time series provide a unified
approach to inference. In practise, expressing the autoregressive coefficient
with the local-unit-root specification, such that,
$\vartheta_{n}=\big{(}1+c/n\big{)}$, where $n$ the sample size and $c$ denotes
the nuisance parameter of persistence, allows to encompass such moderate
deviations from the unit boundary (e.g., unit root, near-integrated or
explosive, see Stoykov,  (2019)). Our main research interest is to investigate
the finite and large sample properties of bootstrap-based estimators and test
statistics (e.g., Moreno and Romo,  (2012)) for predictive regression models
when regressors are generated as nonstationary autoregressive processes.

Predictive regression models are commonly used for modelling macroeconomic
data which exhibit high persistence (see, Zhu et al.,  (2014)). Furthermore,
the nonstandard nature of the inference problem in predictive regressions has
been previously discussed in the literature such as by the studies of Phillips
and Hansen,  (1990), Cavanagh et al.,  (1995) and Jansson and Moreira,
(2006), with the main challenge being the robustness of inference
methodologies to the nuisance parameter of persistence (see, Astill et al.,
(2023))). A unified framework for robust estimation and inference regardless
of the abstract degree of persistence has been proposed by Phillips and
Magdalinos,  (2009) and Kostakis et al.,  (2015) (see, also Chen et al.,
(2013), Breitung and Demetrescu,  (2015), Li et al.,  (2017); Liu et al.,
(2019), Yang et al.,  (2021) and Jayetileke et al.,  (2021)). We argue that
the implementation of resampling methodologies for nonstationary time series
models provides a way for conducting inference, when non-pivotal quantities
are involved, which can ensure accurate asymptotic approximations (see more
recently Demetrescu et al.,  (2022)). Thus, establishing the asymptotic
validity (in the spirit of Cavaliere and Georgiev,  (2020)) of these
bootstrapped approximations is crucial in other testing problems as well such
as when considering structural break (see, Georgiev et al.,  (2018),
Katsouris,  (2022); Katsouris, 2023b ; Katsouris, 2023a and Fei,  (2023)).

A commonly used resampling methodology used to study the distributional
properties of a statistic of interest is based on the well known Efron’s
bootstrap method as in Efron and Tibshirani,  (1986). Various studies consider
the asymptotic validity of the bootstrap-based estimator in (1.1). More
specifically, the validity of explosive autoregressive processes is studied by
Basawa et al.,  (1989, 1991) while in general asymptotics for the bootstrap in
stationary models is predominately concerned with independent observations
(see, Freedman et al.,  (1981, 1984)). Although it has been initially
documented in the literature that the bootstrap cannot work for dependent
processes (see, Bose,  (1988)) it was anticipated that it would work if the
dependence is taken care of while resampling (see, Politis and Romano,  (1994)
and Paparoditis and Politis,  (2003, 2005)). Therefore, the dependence
structure of innovations found in predictive regression models has to be
correctly implemented when considering a bootstrap procedure.

Our framework considers that the dependence structure is preserved due to the
joint Gaussianity assumption imposed on the error sequences of the predictive
regression model along with the nonstationary autoregressive process. In
particular, motivated by the limit theory of cointegrating regressions the
bootstrap implementation is adjusted to capture the dependence induced by the
linear process representation of the innovation sequence (see, Li and Maddala,
(1997), Li and Xiao,  (2001), Psaradakis, 2001b , Psaradakis, 2001a and Inoue
and Kilian,  (2002)). Furthermore, the optimization method employed can affect
the limiting distribution of related test statistics. The main quantity of
interest is the risk of the normalized error, such that,
$r_{n}\left(\hat{\theta}_{n}-\theta\right)$ of an estimator $\hat{\theta}_{n}$
with a suitable slow varying rate function $r_{n}$ where $r_{n}\to\infty$ as
$n\to\infty$. The exact form of the particular rate function depends on the
properties of the estimator under consideration; such as
$\sqrt{n}-$consistency is only ensured for the OLS estimator while a different
rate of convergence holds for the IVX estimator. Consequently, deriving
probability bounds for the specific risk quantity using the Berry-Essen
approach, as well as for its bootstrapped counterpart, in order to assess the
efficiency of finite-sample approximations depend on the chosen estimation
methodology222Related statistical theory is discussed in the books of Tanaka,
(2017) and Hamilton,  (2020) as well as in the studies of Hall and Heyde,
(2014) and Billingsley,  (2013). Moreover, Bickel and Freedman,  (1981)
presents some asymptotic theory aspects for the bootstrap while Mykland,
(1992) proposes relevant asymptotic expansions..

In general, by the classical CLT, the distribution of a pivotal quantity

(1.6) $\displaystyle Q_{n}=\sqrt{n}\big{(}\mu_{n}-\mu\big{)}/s_{n}$

tends weakly to $\mathcal{N}(0,1)$. Let $F_{n}$ be the empirical distribution
of $X_{1},...,X_{n}$, putting mass $1/n$ on each $X_{t}$. When one considers
the bootstrap resampling, the $n$ data points $X_{1},...,X_{n}$ are treated as
a population, with distribution function $F_{n}$ and mean $\mu_{n}$, while
$\mu_{n}^{*}$ is considered as an estimator of $\mu_{n}$. In practise, the
idea is that the behaviour of the quantity $Q_{n}^{*}$ mimics that of $Q_{n}$.
Thus, the distribution of $Q_{n}^{*}$ could be computed from the data and used
to approximate the unknown sampling distribution of $Q_{n}$ (see, Præstgaard
and Wellner,  (1993)).

This paper develops limit theory for the asymptotic validity of the
bootstrapped based IVX estimator in predictive regression models that
satisfies the regularity conditions of valid instrumental based estimators
such as relevance and orthogonality while preserving the robust properties to
the nuisance parameter of persistence. To the best of our knowledge this is a
novel implementation and further establishes the robust properties of the IVX
estimator for bootstrap-based inference.

###  Preliminary Theory.

Concurrent with our work, and independently, Georgiev et al.,  (2021) also
develop asymptotic results for the bootstrap test based on the IVX estimator
who focus on predictability tests based on the sub-sample approach. Another
key point here is that a comparison of the asymptotic behaviour of various
bootstrapping procedures especially for the case of predictive regression
models with regressors of abstract degree of persistence will be essential to
obtain insights regarding the accuracy of bootstrap approximations. Notice
that the IVX estimator is a consistent estimator of the predictive regression
model coefficient although it has a different convergence rate in comparison
to the OLS estimator. Therefore, the bootstrap which is well known to deliver
asymptotic refinements over first-order asymptotic approximations as well as
bias corrections can be applied to solve the asymptotic bias problem (see
Cavaliere et al.,  (2022) for the counterargument). On the other hand, any
additional bias term that appears in the asymptotic expansion reduces the
precision of the test.

Consider the partial-sum process below:

(1.7) $\displaystyle W_{n}(t)=\frac{1}{\sqrt{n}}\sum_{i=1}^{\lfloor
nt\rfloor}Z_{i}^{n},$

then it holds that

(1.8) $\displaystyle X_{n}(t)=X_{n}(0)+\int_{0}^{\lfloor
nt\rfloor/n}b\big{(}X_{n}(s)\big{)}ds+\int_{0}^{t}\sqrt{a\big{(}X_{n}(s-)\big{)}}dW_{n}(s)+error.$

where $W$ is the standard Brownian motion.

Moreover this convergence suggests that $X_{n}$ should converge to a solution
of the limiting stochastic differential equation. Thus, a key step in the
application of the stochastic differential equation approach is to show that
the sequence of stochastic integrals in the approximating equation converges
to the corresponding stochastic integral in the limit333A specific example is
the case in which one considers the random weighted bootstrap. In this case it
is more interesting to study the Marcinkiewicz-Zygmund strong law for randomly
weighted sums of dependent random variables under dependent random designs. .
Let $D$ be the space of cadlag functions $f:[0,1]\to\mathbb{R}$ equipped with
the Skorokhod topology. As a measurable structure on $D$ we consider the
corresponding Borel $\sigma-$algebra $\mathcal{B}(D)$. We write
$X_{n}\overset{d}{\to}X$ whenever $X_{n}$, X are random variables taking
values in $\big{(}D,\mathcal{B}(D)\big{)}$ such that $X_{n}$ converges weakly
to $X$ in $D$ as $n\to\infty$, that is,
$\mathsf{lim}_{n\to\infty}\mathbb{E}\big{[}f(X_{n})\big{]}\to\mathbb{E}\big{[}f(X)\big{]}$
for all bounded continuous functions $f:D\to\mathbb{R}$.

We have that

(1.9) $\displaystyle\frac{1}{a_{n}}\sum_{k=1}^{\lfloor
n.\rfloor}\varepsilon_{k}\overset{d}{\to}J,\ \ \ \text{as}\ \ n\to\infty.$

Therefore, it holds that

(1.10) $\displaystyle\mathbb{P}\big{(}J(t)=0\ \forall t\in[0,1]\big{)}=0.$

We define the Ornstein-Uhlenbeck process
$J_{c}=\big{(}J_{c}(t)\big{)}_{t\in[0,1]}$ driven by $J$ by setting the
following stochastic expression

(1.11) $\displaystyle J_{c}(t)=J(t)-c\int_{0}^{t}e^{-c(t-s)}J(s)ds,\ \ \
t\in[0,1],c\in\mathbb{R}.$

The usefulness of the bootstrap resampling method is exactly to provide a way
to ensure uniform size that is robust to the presence of effects (such as
conditional heteroscedasticity) that can affect sufficient control of the type
of errors when conducting hypothesis testing. Moreover, exact calculation of
the critical value of the test statistic is approximated based on Monte Carlo
simulations. Due to the computational burden of the bootstrap procedure we use
relative small sample sizes. However, by taking a sufficiently large number of
Monte Carlo samples the approximation error can be made arbitrary small thus
ensuring that the bootstrap approximation will work well given regularity
conditions hold that we discuss in more details below.

Consider a test statistic $\mathcal{T}_{n}$ of a parameter estimator such that
$\mathcal{T}_{n}=\mathsf{\eta}(n)\left(\hat{\theta}_{n}-\theta_{0}\right)$.
Let $\mathcal{T}^{*}_{n}$ denote the bootstrap version of $\mathcal{T}_{n}$
such that
$\mathcal{T}^{*}_{n}=\mathsf{\eta}(n)\left(\hat{\theta}^{*}_{n}-\theta_{0}\right)$.
Let $\hat{F}_{n}(u):=\mathbb{P}^{*}\left(\mathcal{T}^{*}_{n}\leq
u\right),u\in\mathbb{R}$ denote the distribution function conditional on the
original pair of data.The bootstrap $\mathsf{p-value}$ is defined as below:

(1.12)
$\displaystyle\hat{p}_{n}:=\mathbb{P}^{*}\left(\mathcal{T}^{*}_{n}\leq\mathcal{T}_{n}\right)=\hat{F}_{n}(\mathcal{T}_{n})$

First-order asymptotic validity of $\hat{p}^{*}_{n}$ requires that
$\hat{p}_{n}$ converges in distribution to a standard uniform distribution
such that $\hat{p}_{n}\overset{d}{\to}\mathsf{Unif}[0,1]$.

##### Minimax Optimality

A complementary aim of this paper is to show that the sufficient conditions
for the error bounds are indeed necessary in some applications. Let us define
a test $\phi$, which is a Borel measurable map,
$\phi:\mathcal{X}_{n}:\mapsto\left\\{0,1\right\\}$. Then, for a class of null
distributions $\mathcal{P}_{0}$, we denote the set of all level $\alpha$ tests
by

(1.13)
$\displaystyle\Phi_{n,\alpha}:=\left\\{\phi:\underset{P\in\mathcal{P}_{0}}{\mathsf{sup}}\
\mathbb{P}_{P}^{(n)}\left(\phi=1\right)\leq\alpha\right\\}.$

We focus on the minimax framework to evaluate the performance of test
statistics and model estimators which enables us to study the fundamental
limits of hypothesis testing while ensuring a (strong) uniform guarantee over
a large class of (null and alternative) distributions. The notion of minimax
performance is widely used to quantify the difficulty of a statistical
problem. More specifically, in the hypothesis testing literature, it is also
common to study the performance of tests against fixed or directional
alternatives such as cases in which
$\beta_{n}=\beta_{0}+\frac{\delta}{\sqrt{n}}$ for some $\delta>0$. For
example, the problem of severe empirical size distortions when $\rho$ is large
it has been also documented in the case of structural break tests in dynamic
econometric models (see, Kuan and Chen,  (1994)). Therefore, this implies that
we are likely to obtain false rejection of constant coefficients in dynamic
models when variables have high persistence and endogeneity.

###  Outline of the paper.

The rest of the paper is organized as follows. In Section 2, we introduce the
predictive regression model and model assumptions. We motivate the
implementation of the IV based estimator which is robust to the nuisance
parameters of persistence. In Section 3, we prove the asymptotic validity of
the bootstrap IVX estimator and verify that we obtain an identical limit
distribution as in the standard case. Section 4, provides a Monte Carlo
Simulation study. In particular, the asymptotic properties of the different
bootstrap proposals under the null and under the alternative are investigated
and their relative power performance is analyzed. Section 5, concludes.

###  Notation.

Throughout the paper, all limits are taken as $n\to\infty$, where $n$ is the
sample size. Then, the symbol $"\Rightarrow"$ is used to denote the weak
convergence of the associated probability measures as $n\to\infty$ (as defined
in Billingsley,  (2013)) which implies convergence of sequences of random
vectors of continuous càdlàg functions on $\mathcal{D}\left([0,1]\right)$
equipped with the Skorokhod topology. The symbol $\overset{d}{\to}$ denotes
convergence in distribution and $\overset{p}{\to}$ denotes convergence in
probability within a suitable probability space
$(\Omega,\mathcal{F}_{t},\mathbb{P})$. Finally, $\mathcal{O}(a_{n})$ denotes
the rate of convergence of estimators, such that if
$\theta_{n}=\mathcal{O}(a_{n})$, implies that, $\theta_{n}/a_{n}$ is bounded.
Also we denote with $\mathbb{P}^{*}$, $\mathbb{E}^{*}$ and $\mathsf{Var}^{*}$
the probability measure, expected value and variance operators induced by the
bootstrap resampling procedure conditional on the time series observations of
the original sample.

##  The model, main assumptions and estimators.

In this section we introduce the model under consideration in this paper, the
main modelling assumptions as well as two different estimation methodologies
proposed in the literature.

###  Predictive regression model.

Consider the predictive regression model given by

(2.1) $\displaystyle Y_{t}$ $\displaystyle=\beta_{n}X_{t-1}+u_{t},\ \
t\in\left\\{1,...,n\right\\},$ (2.2) $\displaystyle X_{t}$
$\displaystyle=\rho_{n}X_{t-1}+v_{t},\ \ \text{with}\ \ X_{0}=0.$

Then, the regressor $X_{t}$ of the predictive regression model is assumed to
be generated via the following process

(2.3) $\displaystyle X_{t}=\varrho_{n}X_{t-1}+v_{t},\ \ \ X_{0}=0,\ \
\varrho_{n}=\left(1+\frac{c}{n^{\gamma}}\right),\ \ \
t\in\left\\{1,...,n\right\\},$

where $\upbeta_{n}\in\mathbb{R}$ is the (model) parameter of interest while
the autoregressive coefficient $\varrho_{n}\in\mathbb{R}$ is defined such that
$\varrho_{n}=\big{(}1+\frac{\displaystyle c}{\displaystyle n^{\gamma}}\big{)}$
with $c\in(-\infty,+\infty)$ and $\gamma\in(0,1]$.

Notice that $v_{t}$ is assumed to be a zero mean, stationary and ergodic
process with finite autocovariances given by
$\tilde{\sigma}:=\mathbb{E}\left(v_{t}v_{t-j}\right)$ and
$\omega^{2}=\sum_{j=-\infty}^{\infty}\tilde{\sigma}$ is finite and nonzero.
Furthermore, the nuisance coefficients $c$ and $\gamma$ are considered as
tuning parameters which determine the degree of persistence for the regressors
and therefore without loss of generality the particular novel feature proposed
by Phillips, 1987b ; Phillips, 1987a provides a tractable way to consider the
nature of possible nonstatioary regressors when estimating predictive
regressions. In terms of distributional assumptions, we assume that the
predictive regression model is generated from a bivariate Gaussian
distribution, such that
$e_{t}=\left(u_{t},v_{t}\right)^{\prime}\overset{\textit{i.i.d}}{\sim}\
\mathcal{N}\big{(}0,\boldsymbol{\Sigma}\big{)}$ with
$t\in\left\\{1,...,n\right\\}$. Further related theoretical results which
apply to the innovation sequence of the system can be found in Phillips and
Solo,  (1992).

######  Assumption 1.

Let $\left\\{\epsilon_{t}\right\\}_{t\in\mathbb{Z}}$ be a sequence of i.i.d
continuous random variables with $\mathbb{E}\left(\epsilon_{1}\right)=0$ and
$\mathbb{E}\left(\epsilon^{2}_{1}\right)=1$, and with a characteristic
function $\phi(\tau)$. Then, the regressor $X_{t}$ of the predictive
regression model is assumed to be generated via the following process

(2.4) $\displaystyle X_{t}=\varrho_{n}X_{t-1}+v_{t},\ \ \ X_{0}=0,\ \
\varrho_{n}=\left(1+\frac{c}{n^{\gamma}}\right),\ \ \
t\in\left\\{1,...,n\right\\},$

with $\gamma=1$, where $\kappa$ is a real constant such that $c\in(-\infty,0)$
and $v_{t}=\sum_{j=0}^{\infty}c_{j}\epsilon_{t-j}$, with
$\sum_{j=0}^{\infty}c_{j}\neq 0$ and the summability condition
$\sum_{j=0}^{\infty}j^{1+\delta}|c_{j}|<\infty$ holds for some $\delta>0$.

######  Assumption 2.

(i) Let $\left\\{u_{t},\mathcal{F}_{t}\right\\}_{t\geq 1}$, where
$\mathcal{F}_{t}$ is a sequence of increasing $\sigma-$fields which is
independent of $\epsilon_{j}$, $j>t$, forms a martingale difference sequence
satisfying
$\mathbb{E}\big{(}u^{2}_{t}|\mathcal{F}_{t-1}\big{)}\to\sigma^{2}>0$, as
$t\to\infty$ and $\underset{t\geq 1}{\mathsf{sup}}\
\mathbb{E}\big{(}|u_{t}|^{4}\big{|}\mathcal{F}_{t-1}\big{)}<\infty$.

(ii) $X_{t}$ is adapted to $\mathcal{F}_{t}$, and there exists a correlated
Brownian motion $\big{(}W,V\big{)}$ such that the following joint weakly
convergence holds

(2.5)
$\displaystyle\left(\frac{1}{\sqrt{n}}\sum_{j=1}^{[nr]}\epsilon_{j-1},\frac{1}{\sqrt{n}}\sum_{j=1}^{[nr]}u_{j}\right)\Rightarrow\big{(}W(r),V(r)\big{)}$

on $\mathcal{D}_{\mathbb{R}^{2}}\left([0,1]\right)^{2}$ as $n\to\infty$.

Define the partial sum process $V_{n}(r)$ and $U_{n}(r)$ as below

(2.6)
$\displaystyle\big{(}V_{n}(r),U_{n}(r)\big{)}=\frac{1}{\sqrt{n}}\sum_{t=1}^{\lfloor
nr\rfloor}\big{(}v_{t},u_{t}\big{)},\ \ \ \ r\in[0,1].$

######  Remark 1.

Assumption 1 and 2 provide necessary conditions for the development of the
asymptotic theory (similarly in Wang and Phillips,  (2012)) for the
corresponding bootstrap IV based estimator we are aiming to examine its
statistical validity. In terms of the estimation procedure for the parameter
of the predictive regression model we consider both the least squares
estimation as well as the endogenous instrumentation method proposed by
Phillips and Magdalinos,  (2009).

###  Main Results

We are interested to examine the asymptotic behaviour for the
$\hat{\upbeta}_{n}$ estimator, in the case of both least square estimation as
well as the two-step endogenous instrumental variable estimation approach
followed for constructing the IVX filtration. To derive the asymptotic
distribution of both estimators we consider the Ornstein-Uhlenbeck (OU)
process

(2.7) $\displaystyle J_{c}(r)=\int_{0}^{r}e^{(r-s)c}dW(s),\ \ \ \ \
\text{with}\ 0\leq r\leq 1\ \text{and}\ c\in(0,+\infty).$

which satisfies the stochastic differential equation
$dJ_{c}(r)=cJ_{c}(r)dr+dW(r)$. Then, the following asymptotic results hold,
based on the local to unity limit law proposed by Phillips, 1987a ; Phillips,
1987b such that $\frac{X_{[nr]}}{\sqrt{n}}\Rightarrow J_{c}(r)$, where
$J_{c}(r)$ is the OU process defined in expression (2.7).

(2.8) $\displaystyle\frac{1}{n\sqrt{n}}\sum_{t=1}^{[nr]}X_{t}$
$\displaystyle\Rightarrow\int_{0}^{r}J_{c}(s)ds,\ \ \ \ \ \ \ \ \ \text{with}\
0\leq s\leq r,$ (2.9) $\displaystyle\frac{1}{n}\sum_{t=1}^{[nr]}X_{t}u_{t}$
$\displaystyle\Rightarrow\int_{0}^{r}J_{c}(s)dW(s),\ \text{with}\ 0\leq s\leq
r.$

######  Remark 2.

The convergence of the sample moments to their stochastic integral
counterparts are proved in the seminal study of Phillips, 1988a who also
derived invariance principles for the multivariate model. In our study we
consider the corresponding univariate limit results since we consider the
univariate predictive regression model with persistent regressors.
Furthermore, we consider the bootstrap invariance principle to obtain the weak
convergence of the sample moments for the bootstrapped-based test statistics.

The ordinary least squares estimate of $\upbeta_{n}$ denoted with
$\hat{\upbeta}_{n}^{ols}$ is given by

(2.10)
$\displaystyle\hat{\upbeta}_{n}^{ols}=\left(\sum_{t=1}^{n}X^{2}_{t-1}\right)^{-1}\left(\sum_{t=1}^{n}X_{t-1}Y_{t}\right)$

Therefore, since
$\sqrt{n}\left(\hat{\upbeta}_{n}^{ols}-\upbeta\right)=\mathcal{O}_{p}(\frac{1}{\sqrt{n}})$
then it converges in distribution which implies

$\displaystyle n\left(\hat{\upbeta}_{n}^{ols}-\upbeta\right)$
$\displaystyle=\left(n^{-2}\sum_{t=1}^{n}X_{t-1}^{2}\right)^{-1}\left(n^{-1}\sum_{t=1}^{n}X_{t-1}u_{t}\right)$
(2.11)
$\displaystyle\Rightarrow\left(\int_{0}^{1}J^{2}_{c}(s)ds\right)^{-1}\left(\int_{0}^{1}J_{c}(s)dW(s)\right)$

######  Remark 3.

The limit distribution given by (2.2) shows that the existence of the nuisance
parameter $c$ in the underline stochastic process, when the parameter
$\upbeta_{n}$ is obtained based on the least squares estimation weakly
convergence to a nonstandard limit. On the other hand this is not the case for
the corresponding IVX estimator as we demonstrate in the next section.

###  IVX estimator

In this section we discuss the main methodology employed for the estimating
the nonstationary time series model. Specifically, the IVX instrumentation
proposed by Phillips and Magdalinos,  (2009) provides important advantages
rather than using the conventional differencing approach prior to fitting the
model. Although employing the difference operator is a common approach for
ensuring covariance stationarity in time series observations due to the fact
that it can remove trends and seasonality it can also contribute to loss of
information such as the long-memory properties444In particular, Kasparis et
al.,  (2015) consider how the interplay of the long-memory properties of
innovations and the persistence properties of regressors in a nonparametric
predictive regression affect the limiting distributions of test statistics and
corresponding model estimators. of these observations which are eliminated
when differencing is applied. Therefore, the IVX estimator in contrast to its
OLS counterpart is found to be robust to the persistence properties of the
regressors as captured by the autoregressive specification without altering
the persistence properties of regressors. The particular estimation approach
can accommodate both nearly nonstationary and nearly stationary processes,
that is, generalizing this way the integration order of regressors for the
predictive regression.

Consider the instrumental variable $Z_{tn}$ which is based on the IVX
methodology of Phillips and Magdalinos,  (2009). Then, the IVX instrumental
variable is constructed

(2.12) $\displaystyle
Z_{tn}=\sum_{j=0}^{t-1}\left(1+\frac{c_{z}}{n^{\upgamma_{z}}}\right)^{j}\big{(}X_{t-j}-X_{t-j-1}\big{)},\
\ \ \ \text{where}\ c_{z}\in(-\infty,0),\upgamma_{z}\in(0,1).$

Denote with $\hat{\upbeta}_{n}^{ivx}$ the corresponding IVX based estimator
for the parameter of the predictive regression given by the system of
equations in expressions (2.1) and (2.2). Then, standard IV estimation
arguments provide an expression for the IVX estimator

(2.13)
$\displaystyle\hat{\upbeta}_{n}^{ivx}=\left(\sum_{t=1}^{n}Z_{t-1}X_{t-1}\right)^{-1}\left(\sum_{t=1}^{n}Z_{t-1}Y_{t}\right)$

Thus, we consider the limit distribution of the random variable $\psi_{n}$ as
defined below

(2.14)
$\displaystyle\psi_{n}:=n^{\frac{1+\upgamma_{z}}{2}}\left(\hat{\upbeta}_{n}^{ivx}-\upbeta_{n}\right)\equiv\left(\frac{1}{n^{1+\upgamma_{x}}}\sum_{t=1}^{n}Z_{t-1}X_{t-1}\right)^{-1}\left(\frac{1}{n^{\frac{1+\upgamma_{x}}{2}}}\sum_{t=1}^{n}Z_{t-1}u_{t}\right)$

######  Lemma 1.

Under Assumption 1 and 2, the following invariance principle holds

(2.15)
$\displaystyle\sum_{t=1}^{[nr]}Z_{t-1}X_{t-1}\Rightarrow-\frac{1}{c_{z}}\left(r\omega_{xx}^{2}+\int_{0}^{r}J_{c}(s)dJ_{c}(s)\right),\
\ \text{with}\ \ r\in[0,1].$

######  Remark 4.

Related weakly convergence arguments which can be employed for the proof of
Lemma 1 can be found in Phillips and Magdalinos,  (2009). The proof of the
particular result consists of a key limit result for deriving the asymptotic
distribution of the bootstrap based IVX estimator which is the main goal of
this paper.

A simple implementation of the limit given by Lemma 1 after appropriate
normalization shows that the following nuisance-free limiting distribution
holds

(2.16)
$\displaystyle\psi_{n}:=n^{(1+\upgamma_{z})/2}\left(\hat{\upbeta}_{n}^{ivx}-\upbeta_{n}\right)\Rightarrow\psi\equiv\mathcal{MN}\left(0,\tilde{\boldsymbol{\Sigma}}\right)$

where $\psi$ follows a mixed Gaussian distribution. The mixed Gaussian
property of the $\psi_{n}$ random variate, is instrumental in developing
inference procedures robust to abstract degree of persistence (see, Phillips
and Lee,  (2013, 2016)).

Therefore, it remains to determine the components of the stochastic covariance
matrix $\tilde{\boldsymbol{\Sigma}}$. More precisely, to determine the
covariance matrix of the random variable $\psi$, we consider the martingale
difference sequence
$\xi_{nt}:=\left(\frac{1}{n^{(1+\upgamma_{z})/2}}Z_{t-1}u_{t},\frac{1}{\sqrt{n}}v_{t}\right)^{\prime}$.
Then the martingale conditional variance of $\xi_{nt}$ has the following
structure

(2.17)
$\displaystyle\sum_{t=1}^{n}\mathbb{E}\big{(}\xi_{nt}\xi_{nt}^{\prime}|\mathcal{F}_{nt-1}\big{)}=\begin{bmatrix}\displaystyle\left(\frac{1}{n^{1+\upgamma_{z}}}\sum_{t=1}^{n}Z_{t-1}^{2}\right)\sigma^{2}_{uu}\
\ &\ \
\displaystyle\left(\frac{1}{n^{1+\frac{\upgamma_{z}}{2}}}\sum_{t=1}^{n}Z_{t-1}\right)\sigma_{uv}\\\
\\\
\displaystyle\left(\frac{1}{n^{1+\frac{\upgamma_{z}}{2}}}\sum_{t=1}^{n}Z_{t-1}\right)\sigma_{vu}\
\ &\ \ \displaystyle\sigma^{2}_{vv}\end{bmatrix}$

######  Remark 5.

Note that as shown by Phillips and Magdalinos,  (2007)
$n^{-\left(1+\upgamma_{z}/2\right)}\sum_{t=1}^{n}z_{t-1}=o_{p}(1)$, which
implies that the martingale conditional covariance (2.17) has off-diagonal
elements which converge in probability to zero. Furthermore, note that without
loss of generality, we operate under the assumption that $\upgamma_{x}=1$ and
$c\in(-\infty,0)$. Relaxing the parameter space of these two quantities allows
to consider regressors generated from general nonstationary autoregressive
processes, covering both near nonstationary and near stationary processes.
Nevertheless, the robust property of the IVX estimator implies that regardless
of the persistence properties of regressors asymptotically converges to a
pivotal mixed Gaussian distribution. Therefore, the particular large sample
property result to conventional statistical inference when considering Wald
type statistics after suitable normalization (see, Phillips and Lee,  (2013,
2016)).

Notice that it has been shown that when the residual process $u_{t}$ is
correlated with the regressor $x_{t}$, the limiting distribution of the
$t-$ratio is no longer normal (see, Phillips and Durlauf,  (1986)). As a
remedy to the particular bias, the fully modified regression proposed by
Phillips and Hansen,  (1990) as an alternative way of constructing the test
statistic as we present later. Overall a key component of our framework is
that it establishes the invariance principle which concerns the weak
convergence (within the Shorokhod topology) of the bootstrap partial sum
process to Brownian motion. Then, based on the continuous mapping theorem, it
can be used to obtain asymptotic distributions of various bootstrapped
statistics without making parametric assumptions on the underlying model. In
particular this approach is build upon the strong approximation and the
Beveridge-Nelson representation of linear processes. The invariance principles
for the i.i.d innovation and the bootstrapped counterpart are first developed
using the strong approximation of the partial sum process by the standard
Brownian motion, and subsequently the invariance principles for the general
linear process with i.i.d innovations and the corresponding bootstrapped
process are established by their Beveridge-Nelson representations as in
Phillips and Solo,  (1992).

Let the partial sum process of $(\varepsilon_{t})$ be defined by

(2.18) $\displaystyle W_{n}(t)=\frac{1}{\sigma\sqrt{n}}\sum_{j=1}^{\lfloor
nt\rfloor}\varepsilon_{j}.$

Then, we have that $W_{n}\overset{d}{\to}W$ in the space of $\mathcal{D}[0,1]$
of cadlag functions, where $W$ is the standard Brownian motion. The space
$\mathcal{D}[0,1]$ is equipped with the Skorokhod topology.

In particular for the bootstrap, we first obtain or estimate
$\left\\{\varepsilon_{t}\right\\}_{t=1}^{n}$ from the sample of size $n$ and
get $\left\\{\hat{\varepsilon}_{t}\right\\}_{t=1}^{n}$ . then, we resample
from the empirical distribution of
$\left\\{\hat{\varepsilon}_{t}\right\\}_{t=1}^{n}$, that is, the distribution
with point probability mass $1/n$ based on the time series observations of
size $n$, to get the bootstrap sample
$\left\\{\varepsilon_{t}^{*}\right\\}_{t=1}^{n}$ as the i.i.d samples from the
empirical distribution of $\left\\{\hat{\varepsilon}_{t}\right\\}_{t=1}^{n}$.
Both $(\hat{\varepsilon}_{t})$ and $(\varepsilon_{t}^{*})$ are dependent upon
the sample size $n$, and we may more precisely denote them as triangular
arrays $(\hat{\varepsilon}_{nt})$ and $(\varepsilon_{nt}^{*})$.

Furthermore, certain studies have previously suggested the inconsistency of
the bootstrap estimator when the parameter is on the boundary of the parameter
space (see, Andrews,  (2000)). However, notice within our setting the
particular consideration has a different interpretation as such a phenomenon
occurs via the local unit root process. In other words, the local-to-unity
specification provides a natural bound on these type of processes when
modelling dependent time series and therefore the asymptotic behaviour of
estimators based on predictive regression models are analytically tractable.
Therefore, when considering bootstrapping nonstationary autoregressive
processes using predictive regression models, in practise we are mainly
concern about deriving an identical weakly convergence argument similar to the
non-bootstrapped conventional limit result.

Our main goal is to develop a consistent bootstrap estimator of the
distribution of the stochastic quantity
$\psi_{n}:=n^{\frac{1+\upgamma_{z}}{2}}\left(\hat{\upbeta}_{n}^{ivx}-\upbeta_{n}\right)$,
regardless of the persistence properties of the regressors of the predictive
regression model as captured by the autoregressive process with a LUR
coefficient. More precisely, under the assumption that the regressors are
generated by the nonstationary process $X_{t}=\varrho_{n}X_{t-1}+v_{t}$ with
$\varrho_{n}=\left(1-\frac{\kappa}{n^{\upgamma_{x}}}\right)$ in Assumption 1,
then we aim to investigate whether $\psi_{n}$ and
$\psi_{n}^{\star}:=n^{\frac{1+\upgamma_{z}}{2}}\left(\hat{\upbeta}_{n}^{\star
ivx}-\upbeta_{n}\right)$ converge to the same distribution limit within a
suitable probability space. To do this, we consider all features of the
distribution of $\psi_{n}$ such as the covariance matrix in (2.17).

##  The Bootstrap and Hypothesis testing.

A commonly used approach to evaluate the limiting distributions in
nonstationary time series models based on asymptotic approximations relies on
finite samples bootstrapped samples. The exact type of the bootstrap procedure
follwed to replicate the dependence structure of the model can affect both the
validity of the results as well as the accuracy of those approximations. In
this section, we study the asymptotic behaviour of the bootstrap based IVX
estimator for different bootstrap schemes. To do this, we impose additional
assumptions for the associated partial sum processes. Let $\mathcal{D}$ be the
space of cadlag functions, $f:[0,1]\to\mathbb{R}$ equipped with the Shorokhod
topology (e.g., see Csörgő et al.,  (2003)).

##### Notation

Denote with $\mathcal{Z}_{nt}^{*}$ a sequence of bootstrap statistics, then
the following modes of convergence hold (see related definitions in Goncalves
and Kaffo,  (2015))

  * •

$\mathcal{Z}_{t}^{*}=o_{P^{*}}(1)$ in probability, or
$\mathcal{Z}_{t}^{*}\overset{P^{*}}{\to}0$ in probability, if for any

(3.1) $\displaystyle\epsilon>0,\delta>0\ \
\underset{n\to\infty}{\mathsf{lim}}\mathbb{P}\big{[}\mathbb{P}^{*}\big{(}\left|\mathcal{Z}_{n}^{*}\right|>\delta\big{)}>\epsilon\big{]}=0.$

  * •

$\mathcal{Z}_{t}^{*}=\mathcal{O}_{P^{*}}(1)$ in probability, if for all
$\epsilon>0$ there exists a $M_{\epsilon}<\infty$ such that

(3.2)
$\displaystyle\underset{n\to\infty}{\mathsf{lim}}\mathbb{P}\big{[}\mathbb{P}^{*}\big{(}\left|\mathcal{Z}_{n}^{*}\right|>M_{\epsilon}\big{)}>\epsilon\big{]}=0.$

  * •

$\mathcal{Z}_{n}^{*}\overset{d^{*}}{\to}\mathcal{Z}$ in probability if,
conditional on the sample, $\mathcal{Z}_{n}^{*}$, weakly converges to
$\mathcal{Z}$ under $P^{*}$, for all samples contained in a set with
probability converging to one. Specifically, we write
$\mathcal{Z}_{n}^{*}\overset{d^{*}}{\to}\mathcal{Z}$ in probability if and
only if
$\mathbb{E}^{*}\big{(}f\left(\mathcal{Z}_{n}^{*}\right)\big{)}\to\mathbb{E}\left(f(\mathcal{Z})\right)$
in probability for any bounded and uniformly continuous function $f$.

Therefore, for the asymptotic validity of the bootstrap algorithm one needs to
consider similar arguments as when deriving the limiting distribution of
nonstationary time series models. In particular, Müller,  (2011) discusses
various aspects of asymptotic efficiency with respect of the weak convergence
of functionals to their brownian motion counterparts.

###  Bootstrap Procedure.

Similar investigation in the literature with respect to the bootstrap
procedure include the study of Davidson and Flachaire,  (2008) who consider a
framework for the Wild bootstrap although not in a nonstationary time series
regression environment. In this paper we apply the Wild bootstrap algorithm
using the OLS as well as the IVX residuals. Another example include the study
of Phillips and Ouliaris,  (1990) who develop asymptotic theory for residual-
based tests of cointegration. Next, we describe the estimation procedure for
the bootstrap based IVX estimator, which can help us shed light on the
specific asymptotic expressions and properties we need to consider in order to
derive its limiting distribution.

Consider again the predictive regression with multiple predictors. Simulate
the following DGP

(3.3) $\displaystyle Y_{t}$ $\displaystyle=\upbeta_{n}X_{t-1}+u_{t},\ \
t\in\left\\{1,...,n\right\\},$ (3.4) $\displaystyle X_{t}$
$\displaystyle=\varrho_{n}X_{t-1}+v_{t},\ \ \text{with}\ \ X_{0}=0.$

where the parameters of interest are $\upbeta_{n}$ and $\varrho_{n}$. Denote
with $u_{t}=\left(u_{t},v_{t}\right)^{\prime}$ the martingale difference
sequence vector with variance-covariance matrix given by

(3.5)
$\displaystyle\boldsymbol{\Sigma}:=\text{Var}(u_{t})=\begin{bmatrix}\sigma^{2}_{u}&\
\sigma_{uv}\\\ \sigma_{vu}&\ \sigma_{vv}\end{bmatrix}$

where $\boldsymbol{\Sigma}$ is a known positive-definite matrix.

Specifically, we are motivated in establishing the asymptotic validity of the
bootstrap for the IVX estimator since its is employed in statistical problems
for which the limiting distribution is non-standard such as when testing for
structural breaks in predictive regression models and the bootstrap is
necessary for deriving critical values. While for instance, we focus in the
case of IVX-based predictability tests, the limit results we derive in this
paper can be generalized for the supremum IVX-Wald statistic when testing for
parameter instability in predictive regression models. The first Bootstrap
algorithm we consider is described below.

Bootstrap Algorithm

  * Step 1.

Estimation Step

    * 1.1.

Fit the predictive regression to the sample data
$\left(Y_{t},X_{t-1}\right)^{\prime}$ to obtain the OLS residuals
$\hat{u}_{t}$, for $t\in\left\\{1,...,n\right\\}$. Obtain the estimate of
$\hat{\upbeta}_{n}$.

    * 1.2.

Fit by OLS the AR(1) model with the LUR coefficient, to the regressor $X_{t}$,
to obtain the OLS residuals $\hat{v}_{t}$, for $t\in\left\\{1,...,n\right\\}$.
Set $\hat{v}_{t}=0$. Obtain $\hat{\varrho}$.

In this section, we assume that the autoregressive coefficient is given by
$\rho_{n}=\left(1-\frac{\kappa}{n}\right)$, which allows to examine the limit
theory of the estimate of the autoregressive coefficient for moderate
deviations from unity. Note that the estimate of $\varrho_{n}$ is obtained by
ordinary least squares estimation which is expressed as

(3.6)
$\displaystyle\hat{\varrho}^{ols}_{n}=\left(\sum_{t=1}^{n}X^{2}_{t-1}\right)^{-1}\left(\sum_{t=1}^{n}X_{t}X_{t-1}\right)$

  * Step 2.

Generate the bootstrap sample
$\left\\{Y_{t}^{*},X_{t}^{*},t=1,...,n\right\\}$.

    * 2.1.

Generate the bootstrap innovations
$w_{t}^{*}=\left(u_{t}^{*},v_{t}^{*\prime}\right)^{\prime}:=\left(e_{t}\otimes\hat{u}_{t},e_{t}\otimes\hat{v}_{t}\right)$,
where the symbol $\otimes$ denotes element-wise vector multiplication, such
that $e_{t}$ for $t\in\left\\{1,...,n\right\\}$, is a scalar sequence
independent from the data such that
$e_{t}\overset{\textit{i.i.d}}{\sim}\mathcal{N}(0,1)$.

    * 2.2.

Generate $X_{t}^{*}$ such that

(3.7) $\displaystyle
X_{t_{b}}^{*}=\hat{\varrho}X_{t_{b}-1}^{*}+u_{t_{b}}^{*},\ \ \ \ \ \text{for}\
t_{b}\in\left\\{1,...,n\right\\},$

with initial conditions $X_{0}^{*}=0$ and $b\in\left\\{1,...,B\right\\}$.

    * 2.3.

Generate the associated bootstrap IVX instrument $z_{t}^{*}$ as below

(3.8) $\displaystyle Z_{0}^{*}=0\ \ \ \text{and}\ \ \
Z_{t}^{*}=\sum_{j=0}^{t_{b}-1}\left(1+\frac{\kappa_{z}}{n}\right)^{j}\Delta
X_{t_{b}-j}^{*},\ \ \text{for}\ \ t\in\left\\{1,...,n\right\\},$

where $\kappa_{z}$ is the coefficient matrix for the persistence of the
bootstrap IVX instruments that contains the same values as the original IVX
instruments.

    * 2.4.

Generate $Y_{t_{b}}^{*}$ such that
$y_{t_{b}}^{*}=\hat{\upbeta}X_{t_{b}-1}^{*}+u_{t_{b}}^{*}$. Set
$Y_{1}^{*}=Y_{1}$.

    * 2.5.

Repeat Steps 2.1 to 2.4 $B$ times (e.g., $B=1000$).

######  Remark 6.

Notice that the asymptotic validity of the bootstrap IVX estimator is an
important property of general interest with applications in various inference
problems. For example, Katsouris,  (2022) considers a framework for structural
break testing in predictive regression models with persistent regressors. In
the particular framework the bootstrap validity of the IVX estimator is
necessary to ensure the consistent estimation of critical values from the
bootstrap limiting distribution of the corresponding structural break test
statistics. Furthermore, Georgiev et al.,  (2021) study the implementation of
both the residual wild bootstrap as well as the fixed regressor wild bootstrap
for the IVX estimator which is useful when constructing predictability tests
(e.g., t-tests or Wald type statistics) based on the predictive regression
model.

Then, for the estimation step one can consider estimating $\beta_{n}$ either
via the classical least squares estimation, denoted with
$\hat{\beta}_{n}^{LS}$, or via the IVX instrumentation methodology proposed by
Phillips and Magdalinos,  (2009), denoted with $\hat{\beta}_{n}^{IVX}$. The
least squares estimate of $\beta_{n}$ is

(3.9)
$\displaystyle\hat{\beta}_{n}^{LS}=\left(\sum_{t=1}^{n}X^{2}_{t-1}\right)^{-1}\left(\sum_{t=1}^{n}X_{t-1}Y_{t}\right)$

The corresponding IVX estimate of $\beta_{n}$ is

(3.10)
$\displaystyle\hat{\beta}_{n}^{IVX}=\left(\sum_{t=1}^{n}Z_{t-1}X_{t-1}\right)^{-1}\left(\sum_{t=1}^{n}Z_{t-1}Y_{t}\right)$

where

(3.11) $\displaystyle
Z_{t}=\sum_{j=0}^{t-1}\left(1-\frac{\kappa_{z}}{n^{\gamma_{z}}}\right)^{j}\left(X_{t-j}-X_{t-j-1}\right),\
\ \ \ \text{with}\ \kappa_{z}\in(0,+\infty),\gamma_{z}\in(0,1).$

We use for example, $\kappa_{z}=1$ and $\gamma_{z}=0.95$.

From Step1A and Step1B, we can obtain the residual sequences,
$\left\\{\hat{u}_{t}^{IVX},\hat{v}_{t}^{LS}\right\\}_{t=1}^{n}$. To simplify
the notation, for the remaining of this Section, we simply denote with
$\hat{u}_{t}$ the corresponding residual sequence obtained via the IVX
instrumentation procedure and $\hat{v}_{t}$ the residual sequence obtained via
Least squares estimation. That is,

(3.12)
$\displaystyle\hat{u}_{t}=\left(Y_{t}-\hat{\beta}_{n}^{IVX}X_{t-1}\right)$

Next, we define the centered residuals for both sequences, such that

(3.13)
$\displaystyle\tilde{u}_{t}=\hat{u}_{t}-\frac{1}{n}\sum_{t=1}^{n}\hat{u}_{t}$
(3.14)
$\displaystyle\tilde{v}_{t}=\hat{v}_{t}-\frac{1}{n}\sum_{t=1}^{n}\hat{v}_{t}$

Denote with $\tilde{F}_{n}$ the empirical distribution function based on
$\left\\{\tilde{e}_{t}\right\\}_{t=1}^{n}$ where
$\tilde{e}_{t}=\left(\tilde{u}_{t},\tilde{v}_{t}\right)^{\prime}$. Thus,
$\tilde{F}_{n}$ associates mass $n^{-1}$ to each of
$\tilde{e}_{t},t=1,....,n$. Now, assuming that $\tilde{F}_{n}$ is the true
distribution, draw a random sample $\left\\{e^{*}_{t},t=1,...,n\right\\}$ from
$\tilde{F}_{n}$. Therefore, conditionally on $\left(X_{1},...,X_{n-1}\right)$,
the random variables $\left\\{e^{*}_{t},t=1,...,n\right\\}$ are i.i.d with
distribution function $\tilde{F}_{n}$.

Step 2: (Bootstrap Step)

We can now construct the bootstrap sample
$\left\\{X_{t}^{*},t=1,...,n\right\\}$ recursively via the following
expression

(3.15) $\displaystyle X_{t}^{*}=\hat{\rho}_{n}X_{t-1}^{*}+v^{*}_{t},\ \ \ \
t=1,...,n,$

with $X_{0}=0$, where $\hat{\rho}_{n}$ it has been estimated in Step 1 by
Least squares estimation.

Next, we construct the bootstrap sample $\left\\{Y_{t}^{*},t=1,...,n\right\\}$
recursively via the following expression

(3.16) $\displaystyle Y_{t}^{*}=\hat{\beta}_{n}X_{t-1}^{*}+u^{*}_{t},\ \ \ \
t=1,...,n,$

where $\hat{\beta}_{n}$ it has been estimated in Step 1 by the IVX
instrumentation methodology.

Next, we re-estimate the model parameter of interest, denoted with
$\hat{\hat{\beta}}_{n}$, by applying again the IVX instrumentation on the
generated sequence of $\left\\{Y_{t}^{*},X_{t}^{*}\right\\}$, from Step 1.
That is,

(3.17)
$\displaystyle\hat{\hat{\beta}}_{n}^{*}=\left(\sum_{t=1}^{n}Z^{*}_{t-1}X^{*}_{t-1}\right)^{-1}\left(\sum_{t=1}^{n}Z^{*}_{t-1}Y^{*}_{t}\right)$

where

(3.18) $\displaystyle
Z^{*}_{t}=\sum_{j=0}^{t-1}\left(1-\frac{\kappa_{z}}{n^{\gamma_{z}}}\right)^{j}\left(X^{*}_{t-j}-X^{*}_{t-j-1}\right),\
\ \ \ \text{with}\ \kappa_{z}\in(0,+\infty),\gamma_{z}\in(0,1).$

We use for example, the same values as in Step 1, that is, $\kappa_{z}=1$ and
$\gamma_{z}=0.95$.

Step 3: (and onwards)

For the remaining bootstrap steps, that is, $b=\left\\{1,...,B\right\\}$ we
generate a sequence of IVX estimates that is
$\underline{\beta}^{*IVX}=\left(\beta^{*IVX}_{n1},....,\beta^{*IVX}_{nB}\right)$.

The main aim of this Section, is to derive the limit distribution of
$\underline{\beta}^{*IVX}$ given $\left(X_{1},...X_{n}\right)$ and to show
that it is the same as the original limit distribution of the IVX estimator.
Notice that in practise the bootstrap procedure should be able to capture the
time series dependence structure. It is worth mentioning that we need to
employ some autoregressive robust covariance matrix estimation methodology
when applying our proposed bootstrap algorithm to ensure that the dependence
structure of the nonstationary time series model is preserved which safeguards
the consistent estimation of the Wald-type statistic (Jansson,  (2004) and
Jansson and Nielsen,  (2012)).

###  Asymptotic distribution result.

We consider a triangular array of paired random variables such that
$\left\\{X_{k,n},Y_{k,n}\right\\}$ with $n\geq 1$ and $k\geq 1$ satisfying

(3.19) $\displaystyle Y_{k,n}$ $\displaystyle=\upbeta_{n}X_{k-1,n}+u_{k},$
(3.20) $\displaystyle X_{k,n}$ $\displaystyle=\varrho_{n}X_{k-1,n}+v_{k},\ \
\text{with}\ \ X_{0}=0.$

where $e_{k}=\left(u_{k},v_{k}\right)^{\prime}\sim\
\mathcal{N}\big{(}0,\boldsymbol{\Sigma}\big{)}$. Then, we construct the
following triangular array using endogenous information generated by the
triangular array $\left\\{X_{k,n}\right\\}_{n,k\geq 1}$ given by

(3.21) $\displaystyle
Z_{k,n}=\sum_{j=0}^{t-1}\left(1-\frac{\kappa_{z}}{n^{\upgamma_{z}}}\right)^{j}\big{(}X_{k-j,n}-X_{k-j-1,n}\big{)},\
\ \ \ \text{with}\ \kappa_{z}\in(0,+\infty),\upgamma_{z}\in(0,1).$

We define the following random sequence

(3.22)
$\displaystyle\mathcal{J}_{n}:=\left(\sum_{t=1}^{n}Z_{t-1}X_{t-1}\right)^{2}\left(\hat{\upbeta}_{n}^{ivx}-\upbeta_{n}\right)$

where $\hat{\upbeta}_{n}^{ivx}$ is the IVX estimator of $\upbeta_{n}$.
Therefore, the random measure $\mathcal{J}_{n}$ can be shown to have a
limiting distribution which is given by the following expression

(3.23) $\displaystyle\mathcal{J}_{n}\overset{d}{\to}\mathcal{J}:=-\kappa_{z}\
\mathcal{U}\left(.\right)\left\\{\omega_{xx}^{2}+\int_{0}^{1}J_{\kappa}(s)dJ_{\kappa}(s)\right\\}$

where $\mathcal{U}\left(.\right)$ is Brownian motion with variance
$\sigma^{2}_{uu}\tilde{V}$ with
$\tilde{V}\equiv-\omega^{2}_{xx}\big{/}2\kappa_{z}$, for
$\upgamma_{x}>\upgamma_{z}$ since $\upgamma_{z}\in(0,1)$. Then, the bootstrap
estimator $\hat{\upbeta}_{n}^{*ivx}$ of $\upbeta_{n}$ is obtained from the
corresponding bootstrap sequence $\left\\{Y_{t}^{*},X_{t}^{*}\right\\}$ which
implies the following random sequence

(3.24)
$\displaystyle\mathcal{J}_{n}^{*}=\left(\sum_{t=1}^{n}Z^{*}_{t-1}X^{*}_{t-1}\right)^{2}\left(\hat{\upbeta}_{n}^{*ivx}-\upbeta_{n}\right)$

where

(3.25) $\displaystyle
Z^{*}_{t}=\sum_{j=0}^{t-1}\left(1-\frac{\kappa_{z}}{n^{\upgamma_{z}}}\right)^{j}\left(X^{*}_{t-j}-X^{*}_{t-j-1}\right),\
\ \ \ \text{with}\ \kappa_{z}\in(0,+\infty),\upgamma_{z}\in(0,1).$

Therefore, in practise we need to show that $\mathcal{J}_{n}$ and
$\mathcal{J}_{n}^{*}$ have the same limit distribution (we define this step as
Condition 1). To do this, we consider the random measure $\mathcal{\zeta}_{n}$
that corresponds to the paired triangular array (3.19)-(3.20) and the
generated instrument is given in expression (3.21). Sweeting,  (1989) proposed
a framework for evaluating conditional weak convergence. Specifically, when
considering the asymptotic validity of the bootstrap algorithm it is necessary
to consider the conditional weak convergence based on the bootstrapped
invariance principles. On the other hand, the dependence structure of the
predictive regression model remains fixed therefore the invariance principles
based for the probability space that corresponds to the bootstrapped
observations is valid due to similar conditions.

Notice also that although the practitioner needs to choose the nuisance
parameter of persistence for the instrumental variable. In other words, the
statistics of interest such as the model estimator as well as the Wald test
are calculated based on the following expression:

(3.26)
$\displaystyle\mathcal{\zeta}_{n}:=\left(\sum_{k=1}^{n}Z_{k-1,n}X_{k-1,n}\right)^{2}\left\\{\frac{\displaystyle\sum_{k=1}^{n}Z_{k-1,n}Y_{k,n}}{\displaystyle\sum_{k=1}^{n}Z_{k-1,n}X_{k-1,n}}-\upbeta_{n}\right\\}$

Following a similar approach in the literature (to add references), we need to
specify an auxiliary functional of the form $\Psi(.)$ such that

(3.27) $\displaystyle\mathcal{H}(.,x)=\mathbb{P}\big{(}\mathcal{J}_{n}\leq
x\big{)},$

so that we can derive an associated result that shows a probability limit of
the form

(3.28)
$\displaystyle\underset{n\to\infty}{\mathsf{lim}}\mathbb{P}_{\upbeta_{n}}\big{(}\mathcal{\zeta}_{n}\leq
x\big{)}=\mathcal{H}(.,x)$

To do that, we can define for example with

(3.29)
$\displaystyle\mathcal{H}_{n}\left(\hat{\upbeta}^{ivx}_{n},x\right)=\mathbb{P}\bigg{(}\mathcal{J}_{n}^{*}\leq
x\big{|}X_{1},...,X_{n}\bigg{)}$

which is taken to be a regular conditional probability distribution function.

######  Remark 7.

The above statements imply that to show that the bootstrap approximation is
valid, then we shall show that along almost all paths of $H_{n}$ given by
expression (3.29) converges in distribution to the distribution of
$\mathcal{J}$ given by expression (3.23) (we define this step as Condition 2).
Therefore, to prove the result we are aiming we can try to prove that a
corresponding weakly convergence result holds for the quantity
$\mathcal{J}_{n}^{*}$.

Notice that in the literature for bootstrap based inference of time series
models (see, Basawa et al.,  (1991)) the authors prove that the bootstrap
estimator of the autoregressive coefficient in an AR(1) model in the explosive
case will not have the same limiting distribution as the original estimator,
within our framework this might not be affecting the result which we are
aiming to prove. In other words, due to the structure of the predictive
regression system, in each bootstrap step, to generate the pair
$\left\\{X_{t}^{*},Y_{t}^{*}\right\\}$ we use the residuals $u_{t}^{*}$
obtained from the first step regression and then construct the observations
$Y_{t}^{*}$ in the second step regression. Therefore, in practise since we are
not interested whether or not the bootstrap estimator of the autoregressive
coefficient (under various nonstationary processes) is valid or not, we can
still use the particular residual sequences. Therefore, after fitting the
predictive regression we can obtain the bootstrap IVX estimator conditional on
the nonstationary process. More precisely, the statistical validity of the
bootstrap residuals that correspond to the autoregressive process is proved to
hold by Paparoditis and Politis,  (2003) (see, Appendix E.3).

Therefore, we can consider the covariance matrix of the random variable
$\psi^{*}$, where

(3.30)
$\displaystyle\psi_{n}^{*}:=n^{(1+\upgamma_{z})/2}\left(\hat{\upbeta}_{n}^{*IVX}-\upbeta_{n}\right)\Rightarrow\psi^{*}:=\mathcal{MN}\left(0,\tilde{\boldsymbol{\Sigma}}^{*}\right)$

Therefore, we consider the martingale difference sequence
$\xi^{*}_{nt}:=\left(\frac{1}{n^{(1+\upgamma_{z})/2}}Z^{*}_{t-1}u^{*}_{t},\frac{1}{\sqrt{n}}v^{*}_{t}\right)^{\prime}$.

Then the martingale conditional variance has the following structure

(3.31)
$\displaystyle\sum_{t=1}^{n}\mathbb{E}\big{(}\xi^{*}_{nt}\xi_{nt}^{*\prime}|\mathcal{F}_{nt-1}\big{)}=\begin{bmatrix}\displaystyle\left(\frac{1}{n^{1+\gamma_{z}}}\sum_{t=1}^{n}Z^{*2}_{t-1}\right)\sigma^{*2}_{uu}\
\ &\ \
\displaystyle\left(\frac{1}{n^{1+\frac{\gamma_{z}}{2}}}\sum_{t=1}^{n}Z^{*}_{t-1}\right)\sigma^{*}_{uv}\\\
\\\
\displaystyle\left(\frac{1}{n^{1+\frac{\gamma_{z}}{2}}}\sum_{t=1}^{n}Z^{*}_{t-1}\right)\sigma^{*}_{vu}\
\ &\ \ \displaystyle\sigma^{*2}_{vv}\end{bmatrix}$

######  Remark 8.

In this paper, in contrary to some related studies in the literature such as
Georgiev et al.,  (2018), Cavaliere and Georgiev,  (2020) and Georgiev et al.,
(2021) we do not consider the implementation of the fixed regressor bootstrap.
The reason for this is simple, since we study estimators for the predictive
regression model based on the paired data
$\left\\{Y_{t},X_{t}\right\\}_{t=1}^{n}$, then keeping the regressors fixed in
each bootstrap iteration asymptotically might not preserve the limiting
distribution of the original pair.

##  Monte Carlo Simulations.

######  Example 1.

We consider the following simple data-generating process (DGP):

(4.1) $\displaystyle y_{t}$ $\displaystyle=\beta_{0}x_{t}+u_{t},\ \ \
t=1,2,...,n$ (4.2) $\displaystyle x_{t}$
$\displaystyle=\left(1-\frac{c}{n}\right)x_{t-1}+v_{t},\ \ \ \text{for some}\
c>0.$

where $v_{t}\sim\mathcal{N}\left(0,\sigma_{v}^{2}\right)$ and $u_{t}=\rho
u_{t-1}+\epsilon_{t}$ where
$\epsilon_{t}\sim\mathcal{N}\left(0,\sigma_{\epsilon}^{2}\right)$. In
practise, when $\rho=0$ then $u_{t}=\epsilon_{t}$ is the i.i.d residual
process.

The particular example can demonstrate our asymptotic results presented in
this paper. Then the performance of our bootstrap procedure can be assessed
using total variation distances applied to the empirical distribution function
and the corresponding empirical distribution function of the residual process
(see related limit theory in Stroud,  (1972) and Mykland,  (1992)). According
to Horowitz,  (2003) bootstrap resampling is a method for estimating the
distribution of an estimator or test statistic by resampling one’s data or a
model estimated from the data. Under conditions that hold in a variety of
econometric applications, the bootstrap provides approximations to
distributions of statistics, coverage probabilities of confidence intervals,
and rejection probabilities of tests that are more accurate than the
approximations of first-order asymptotic distribution theory.

##  Conclusion.

In this paper we contribute to the literature of bootstrap methodologies for
nonstationary time series models. Specifically, we study the bootstrapping
problem of nonstationary autoregressive processes with predictive regression
models with near unit root (persistent) regressors. These are regressors which
are near-integrated, that is, close to the unit boundary. We establish the
asymptotic validity of the bootstrap based estimators and investigate the
finite-sample behaviour of these bootstrap procedures by Monte Carlo
experiments. The related theory to the bootstrap aspects of the paper is
presented in the Appendix. Moreover, the IVX instruments are constructed based
on endogenous information therefore satisfying the relevance condition for
valid instruments. Therefore, the role of the bootstrap is to mimic the
dependence structure in the predictive regression model. A discussion
regarding the relevance condition of instrumental variables can be found in
the paper of Hall et al.,  (1996).

Therefore, one of the applications of our methodology is to employ the
bootstrap for nonstandard and nonpivotal testing problems. Generally a Wald-
type statistic is a pivotal function, as occurs for instance when testing
linear restrictions on the coefficients of linear regressions. On the other
hand, when testing in the predictive regression model we employ the IVX-Wald
test which is robust to the nuisance parameter of persistence. We have
investigated two estimation methodologies when bootstrapping nonstationary
autoregressive processes with predictive regression models for the purpose of
obtaining the bootstrapped distribution of test statistics. On the other hand,
bootstraps with exchangeable weights are not so wide spread in the
econometrics and statistics literature. Thus considering the random weight
bootstrap in the context of nonstationary autoregressive processes is a novel
contribution to the literature.

As future research we aim to investigate the bootstrap validity of the
framework that corresponds to the uniform inference methodology in predictive
regression. In particular, the quasi restricted likelihood ratio test (QRLRT)
shows to have desirabe properties. More precisely, the restricted likelihood
has been found to provide a well-behaved likelihood ratio test in the
predictive regression model even when the regressor exhibits almost unit root
behaviour. Another interesting further research worth mentioning include to
consider the bootstrap validity of additional persistence classes or different
functional forms for the predictive regression model as proposed in the
framework of Duffy and Kasparis,  (2021). Our proposed bootstrap algorithm can
be also employed in different settings as well such that within a sequential
monitoring framework for structural breaks in Garch $(p,q)$ models (see,
Berkes et al.,  (2004)). Furthermore, although in this paper we consider the
construction of Wald-type statistics based on linear restrictions other
transformations can be also considered such as non-linear restrictions which
are found to have better power performance (see also Heimann and Kreiss,
(1996)).

##### Acknowledgements

I wish to thank Jose Olmo, Tassos Magdalinos and Jean-Yves Pitarakis for
helpful discussions during my PhD studies as well as Stathis Paparoditis and
Giuseppe Cavaliere. The author of this article acknowledge the use of the
IRIDIS High Performance Computing Facility and associated support services at
the University of Southampton, in the completion of this work. Financial
support from the VC PhD studentship of the University of Southampton is also
gratefully acknowledged.

APPENDIX

##  Appendix A Technical Proofs

(page S.23 in Georgiev et al.,  (2021))

We have that

(A.1) $\displaystyle\frac{1}{n^{1+\upgamma_{z}}}\sum_{t=1}^{\lfloor
nr\rfloor}Z_{t-1}X_{t-1}=\frac{\omega}{a}\frac{1}{n}\sum_{t=2}^{\lfloor
nr\rfloor}v_{t-1}\xi_{t-1}-\frac{c}{a}\frac{1}{n^{2}}\sum_{t=2}^{\lfloor
nr\rfloor}\xi_{t-2}\xi_{t-1}+o_{p}(1),$

It then follows by near-integration asymptotics that

$\displaystyle\frac{1}{n^{1+\upgamma_{z}}}\sum_{t=1}^{\lfloor
nr\rfloor}Z_{t-1}X_{t-1}$
$\displaystyle\Rightarrow\frac{\omega^{2}}{a}\left(\int_{0}^{\pi}JdM_{c}+\left[M_{v}\right]_{\pi}-c\int_{0}^{\pi}J^{2}_{c}(r)\right)$
$\displaystyle=\frac{\omega^{2}}{a}\left(\int_{0}^{\pi}JdM_{c}+\left[M_{v}\right]_{\pi}\right),$

Notice that it holds that $\left[M_{v}\right]_{\pi}=\left[J_{c}\right]_{\pi}$,
that is, the two stochastic processes have the same quadratic variation.
Furthermore, it holds that

(A.2) $\displaystyle
J^{2}_{c}(\pi)-\int_{0}^{\pi}J_{c}(r)dJ_{c}(r)=\int_{0}^{\pi}J_{c}(r)dJ_{c}(r)+\left[J_{c}\right]_{\pi}$

which holds by the semi-martingale property of $J_{c}(r)$. Moreover, notice
these stochastic processes are monotonically increasing with a continuous
quadratic variation function $\left[M_{v}\right](\pi)$ on a compact set
$\mathcal{B}(0,1)$, then it is sufficient to show that the asserted weakly
convergence of the sample moments to the corresponding stochastic integrals
holds point-wise in probability for $\pi\in[0,1]$.

Consider the proof from Li and Xiao,  (2001). In particular, we are interested
to show the invariance principles for the partial sums of
$\varepsilon_{t}^{*}$ and $u_{t}^{*}$. Thus, first we show that

(A.3) $\displaystyle\frac{1}{\sqrt{T}}\sum_{t=1}^{\lfloor
nr\rfloor}\varepsilon_{t}^{*}\Rightarrow B_{\varepsilon}(r)$

Let EDF denote the empirical distribution function $d_{2}(.,.)$ be the Mallows
metric defined as $d_{p}(\nu,\mu)=\mathsf{inf}\ \mathbb{E}\left(\left\lVert
V-U\right\rVert^{p}\right)^{1/p}$. Thus, we denote with

(A.4) $\displaystyle S_{\lfloor
nr\rfloor}:=\frac{1}{\sqrt{T}}\sum_{t=1}^{\lfloor
nr\rfloor}\varepsilon_{t}^{*}\Rightarrow B_{\varepsilon}(r),\ \ S_{\lfloor
nr\rfloor}^{*}:=\frac{1}{\sqrt{T}}\sum_{t=1}^{\lfloor
nr\rfloor}\varepsilon_{t}^{*}\Rightarrow B_{\varepsilon}(r)$

Thus, we need to show that

(A.5) $\displaystyle d_{2}(F,\hat{F}_{T})\to 0$

Moreover, we need to establish convergence of finite-dimensional distributions
and verify the tightness condition. In particular, for the finite-dimensional
convergence, it is sufficient to show that for any finite $k$ and
$0<r_{1}<...<r_{k}<1$, if

(A.6) $\displaystyle V_{k}=\big{[}S_{\lfloor nr_{1}\rfloor},S_{\lfloor
nr_{2}\rfloor}-S_{\lfloor nr_{1}\rfloor},....,S_{\lfloor
nr_{k}\rfloor}-S_{\lfloor nr_{k-1}\rfloor}\big{]}$

and

(A.7) $\displaystyle V_{k}^{*}=\big{[}S^{*}_{\lfloor
nr_{1}\rfloor},S^{*}_{\lfloor nr_{2}\rfloor}-S^{*}_{\lfloor
nr_{1}\rfloor},....,S^{*}_{\lfloor nr_{k}\rfloor}-S^{*}_{\lfloor
nr_{k-1}\rfloor}\big{]}$

then $d_{2}\left(V_{k}^{*},V_{k}\right)\to 0$. Thus, it can be shown that
$d_{2}\left(V_{k}^{*},V_{k}\right)^{2}\leq
d_{2}\left(F_{T},\tilde{F}_{T}\right)^{2}\to 0$.

For tightness, it suffices to show that there exists a non-decreasing function
$\psi$ such that, for almost all sample paths, for $0\leq r_{1}\leq r\leq
r_{2}\leq 1$,

(A.8) $\displaystyle\mathbb{P}^{*}\left(\left|S^{*}_{\lfloor
nr\rfloor}-S^{*}_{\lfloor nr_{1}\rfloor}\right|\right)$

From the paper: Testing for parameter instability in predictive regression
models

######  Remark 9.

Notice that considering for example the test statistic $\mathcal{W}^{*}_{ivx}$
we observe that bootstrap invariance principles hold jointly along with stated
convergence results. In practise, in terms of convergence in topological
spaces, we have joint weak convergence of random measures. The concept is
weaker than weak convergence in probability, although it reduces to the latter
when the limit distribution is non-random.

##  Appendix B Smoothing Local-to-moderate unit root theory.

Consider an autoregressive process with local-to-moderate deviations from unit
root of the form $\rho_{n}=\left(1+\frac{c}{K}\right)$, where $K$ converges to
infinity with the sample size $n$ and $\rho_{n}$ approaches unity from the
stationary or the explosive side according to the sign of $c$ (see, Phillips
et al.,  (2010)).

We consider that such a time series process constitutes of $m$ blocks of $K$
observations with total sample size $n=mK$. Then, partitioning the
chronological sequence $\left\\{t=1,...,n\right\\}$ by setting $t=[kj]+k$ for
$k\in\left\\{1,...,K\right\\}$ and $j\in\left\\{0,...,m-1\right\\}$, it is
possible to study the asymptotic behaviour of the time series
$\left\\{X_{t}:t=1,...,n\right\\}$ via the asymptotic properties of the time
series $\left\\{X_{Kj+k}:j=0,...,m-1,k=1,...,K\right\\}$. The latter
representation is particularly useful for revealing the transition from non-
stationary to stationary autoregression as the number of blocks $m$ increases.

More formally, the process with the above characteristics may be written in
the form

(B.1) $\displaystyle X_{t}$ $\displaystyle=\rho_{n,m}X_{t-1}+u_{t},\ \ \
u_{t}\sim_{iid}\left(0,\sigma^{2}\right),$ (B.2) $\displaystyle\rho_{n,m}$
$\displaystyle=\left(1+\frac{c}{K}\right)=\left(1+\frac{cm}{n}\right)$

since we have $m$ blocks with $K$ observations in each block, giving us a
total of $n=mK$.

Therefore, under this setting when $m=1$, then the usual local to unity model
applies, and when $m\to\infty$, then the moderate deviation theory of PM and
GP holds.

Let $W$ be a standard Brownian motion and
$J_{c}(t)=\displaystyle\int_{0}^{t}e^{c(t-s)}dW(s)$ be a corresponding OU
process. For each $m$, letting

(B.3) $\displaystyle\widetilde{W}(t)=\sqrt{m}W\left(\frac{t}{m}\right),$

we observe that W also a standard Brownian motion and we denote by
$\widetilde{J}_{c}(t)=\int_{0}^{t}e^{c(t-s)}d\widetilde{W}(s)$ the associated
OU process. Thus, for given $m\geq 1$, we may derive a limit theory for the
least squares estimate $\hat{\rho}_{n,m}$ of $\rho_{n,m}$ as $n\to\infty$
using earlier results from standard local to unity asympotics.

Thus, using the identities above we obtain

(B.4) $\displaystyle\int_{0}^{1}J_{cm}(s)dW(s)$
$\displaystyle=\frac{1}{m}\int_{0}^{m}\widetilde{J}_{c}(s)d\widetilde{W}(s),$
(B.5) $\displaystyle\int_{0}^{1}J_{cm}(s)^{2}ds$
$\displaystyle=\frac{1}{m^{2}}\int_{0}^{m}\widetilde{J}_{c}(s)^{2}ds.$

the results derived in Phillips imply that, for fixed $m$ and $n\to\infty$,
the asymptotic distribution of the least squares estimator takes the following
form

(B.6) $\displaystyle
n\left(\hat{\rho}_{n,m}-\rho_{n,m}\right)\Rightarrow\frac{\displaystyle\int_{0}^{1}J_{cm}(s)dW(s)}{\displaystyle\int_{0}^{1}J_{cm}(s)^{2}ds}=m\frac{\displaystyle\int_{0}^{m}\widetilde{J}_{c}(s)d\widetilde{W}(s)}{\displaystyle\int_{0}^{m}\widetilde{J}_{c}(s)^{2}ds}.$

When $c<0$, sequential limits with $n\to\infty$ followed by $m\to\infty$ lead
to the normal asymptotic theory given in PM and GP:

(B.7)
$\displaystyle\frac{n}{\sqrt{m}}\left(\hat{\rho}_{n,m}-\rho_{n,m}\right)$
$\displaystyle\Rightarrow\frac{\displaystyle\frac{1}{\sqrt{m}}\int_{0}^{m}\widetilde{J}_{c}(s)d\widetilde{W}(s)}{\displaystyle\frac{1}{m}\int_{0}^{m}\widetilde{J}_{c}(s)^{2}ds}\
\ \text{for fixed}\ m$ (B.8)
$\displaystyle=\frac{\displaystyle\frac{1}{\sqrt{m}}\sum_{j=1}^{m}\int_{j-1}^{j}\widetilde{J}_{c}(s)d\widetilde{W}(s)}{\displaystyle\frac{1}{m}\sum_{j=1}^{m}\int_{j-1}^{j}\widetilde{J}_{c}(s)^{2}ds}$
(B.9)
$\displaystyle\Rightarrow\frac{\mathcal{N}\left(0,-\frac{1}{2c}\right)}{-\frac{1}{2c}}\equiv\mathcal{N}(0,-2c)\
\ \text{as}\ \ m\to\infty.$

##  Appendix C Bootstrapping Unstable Autoregressive Processes.

###  Theoretical Background.

In this section, we present related asymptotic theory examples as preliminary
theory to the framework proposed in this paper.

Consider the first-order autoregressive process $\left\\{X_{t}\right\\}$, for
$t=1,2,...$,

(C.1) $\displaystyle X_{t}=\beta X_{t-1}+\epsilon_{t},\ \ \ X_{0}=0,$

where $\left\\{\epsilon_{t}\right\\}$ are independent $\mathcal{N}(0,1)$
random variables.

The least squares estimator $\hat{\beta}_{n}$ of $\beta$, based on a sample of
$n$ observations $\left(X_{1},...,X_{n}\right)$, is given by the following
expression

(C.2)
$\displaystyle\hat{\beta}_{n}=\left(\sum_{t=1}^{n}X^{2}_{t-1}\right)^{-1}\left(\sum_{t=1}^{n}X_{t}X_{t-1}\right)$

Note that the asymptotic validity of the bootstrap estimator corresponding to
$\hat{\beta}_{n}$ for the stationary case, viz., $|\beta|<1$, follows from the
work of Bose,  (1988) and the validity for the explosive case, viz.,
$|\beta|>1$ has recently established by Basawa et al.,  (1989). Both of these
papers consider the general case when the distribution of
$\left\\{\epsilon_{t}\right\\}$ is not necessarily known. The limit
distribution of $\hat{\beta}_{n}$ in the unstable case $|\beta|=1$ is known to
be nonnormal. It is therefore, of special interest to consider the bootstrap
approximation for the distribution of $\hat{\beta}_{n}$ for the unstable case.

###  Invalidity of the bootstrap estimator.

Let
$\xi_{n}=\left(\sum_{t=1}^{n}X^{2}_{t-1}\right)^{1/2}\left(\hat{\beta}_{n}-\beta\right)$,
where $\hat{\beta}_{n}$ is defined as above. It is then, well known that when
$\beta=1$,

(C.3)
$\displaystyle\xi_{n}\overset{d}{\to}\xi=\frac{1}{2}\left[W^{2}(1)-1\right]\left\\{\int_{0}^{1}W^{2}(t)dt\right\\}^{-1/2}\
\ \text{as}\ \ n\to\infty,$

where $W(t)$ is a standard Wiener process. Then, the corresponding bootstrap
sample $\left\\{X_{t}^{*}\right\\}$ is obtained recursively from the following
relation

(C.4) $\displaystyle X_{t}^{*}=\hat{\beta}_{n}X_{t-1}^{*}+\epsilon_{t}^{*},\ \
X_{0}^{*}=0.$

where $\left\\{\epsilon_{t}^{*}\right\\}$ constitutes a random sample from
$\mathcal{N}\left(0,1\right)$. Then, the bootstrap estimator
$\hat{\beta}^{*}_{n}$ of $\beta$ is then defined as in (C.2) with $X$’s
replaced by $X^{*}$’s.

Let
$\xi_{n}^{*}=\left(\sum_{t=1}^{n}X^{*2}_{t-1}\right)^{1/2}\left(\hat{\beta}^{*}_{n}-\beta\right)$
denote the bootstrap version of $\xi_{n}$. It will be shown that $\xi_{n}$ and
$\xi_{n}^{*}$ do not have the same limit distribution, thus invalidating the
bootstrap.

Therefore, we consider a triangular array $\left\\{X_{k,n},k\leq 1,n\leq
1\right\\}$ satisfying

(C.5) $\displaystyle X_{k,n}=b_{n}X_{k-1,n}+\epsilon_{k},\ \ X_{0}=0,$

with independent $\epsilon_{k}\sim\mathcal{N}\left(0,1\right)$ and where
$\left\\{b_{n}\right\\}$ is a sequence of numbers such that
$n\left(b_{n}-1\right)\to\gamma$. Let

(C.6)
$\displaystyle\Psi(\gamma)=\frac{\displaystyle\int_{0}^{1}\left(1-t+te^{-2\gamma}\right)^{-1}W(t)dW(t)}{\displaystyle\left\\{\int_{0}^{1}\left(1-t+te^{-2\gamma}\right)^{-2}W^{2}(t)dt\right\\}^{1/2}}$

where $\left\\{W(t):0\leq t\leq 1\right\\}$ is a standard Brownian motion and

(C.7) $\displaystyle H(\gamma,x)=\mathbb{P}\left(\Psi(\gamma)\leq x\right).$

Then, by Theorem 1 of Chan and Wei,  (1987) we have that

(C.8)
$\displaystyle\underset{n\to\infty}{\text{lim}}\mathbb{P}_{b_{n}}\left(\zeta_{n}\leq
x\right)=H(\gamma,x)$

where

(C.9)
$\displaystyle\zeta_{n}=\left(\sum_{k=1}^{n}X^{2}_{k-1,n}\right)^{1/2}\left(\frac{\sum_{k=1}^{n}X_{k,n}X_{k-1,n}}{\sum_{k=1}^{n}X^{2}_{k-1,n}}-b_{n}\right)$

and where $\mathbb{P}_{b_{n}}$ indicates the probability distribution induced
by the model in (C.5). Define

(C.10) $\displaystyle
H_{n}\left(\hat{b}_{n},x\right)=\mathbb{P}\bigg{(}\xi_{n}^{*}\leq
x\big{|}X_{1},...,X_{n}\bigg{)}$

which is taken to be a regular conditional probability distribution function.
Therefore, we define a random measure

(C.11) $\displaystyle\eta_{n}(A)=\int_{A}H_{n}\left(\hat{b}_{n},dx\right).$

Since $H(\gamma,x)$ given by (D.2) is continuous in $\gamma$ for each fixed
$x$, we have

(C.12) $\displaystyle\eta(A)=\int_{A}H\left(\xi^{\prime},dx\right)$ (C.13)
$\displaystyle\xi^{\prime}=\frac{1}{2}\left[W^{2}(1)-1\right]\left\\{\int_{0}^{1}W^{2}(t)dt\right\\}^{-1}\
\ \text{as}\ \ n\to\infty,$

Therefore, if the bootstrap approximation were valid then along almost all
paths $H_{n}$ given by (C.10) would converge in distribution to the
distribution of $\xi$ given in (C.3). However, we have in fact that

(C.14) $\displaystyle\eta_{n}\Rightarrow\eta,\ \ \text{as}\ \ n\to\infty.$

###### Proof.

By almost sure representativeness of convergent laws, it is possible to define
$\tilde{\beta}_{n},n\leq 1$, and $\tilde{\xi}$ with
$\hat{\beta}_{n}=\tilde{\beta}_{n}$, $\xi^{\prime}=_{d}\tilde{\xi}$ and

(C.15) $\displaystyle
n\left(\tilde{\beta}_{n}-1\right)\overset{d}{\to}\tilde{\xi}\ \ \textit{almost
surely}\ \text{as}\ n\to\infty.$

Recall that,

(C.16) $\displaystyle
n\left(\hat{\beta}_{n}-1\right)\overset{d}{\to}\xi^{\prime}\ \text{as}\
n\to\infty$

Therefore, we have that

(C.17) $\displaystyle H_{n}\left(\tilde{\beta}_{n},x\right)\to
H(\tilde{\xi},x)\ \ \textit{almost surely}\ \text{as}\ n\to\infty.$

Hence for sets of the form

(C.18) $\displaystyle A_{j}=\bigcup_{i=1}^{m_{j}}\big{(}x_{ij},y_{ij}\big{)}$

representing a disjoint union of intervals, (C.17) implies that

(C.19)
$\displaystyle\bigg{(}H_{n}\left(\tilde{\beta}_{n},A_{1}\right),...,H_{n}\left(\tilde{\beta}_{n},A_{k}\right)\bigg{)}\to\bigg{(}H_{n}\left(\tilde{\xi},A_{1}\right),...,H_{n}\left(\tilde{\xi},A_{k}\right)\bigg{)}$

almost surely as $n\to\infty$.

Since,

(C.20)
$\displaystyle\bigg{(}\eta_{n}(A_{1}),....,\eta_{n}(A_{k})\bigg{)}=_{d}\bigg{(}H_{n}\left(\tilde{\beta}_{n},A_{1}\right),...,H_{n}\left(\tilde{\beta}_{n},A_{k}\right)\bigg{)}$

and

(C.21)
$\displaystyle\bigg{(}\eta(A_{1}),....,\eta(A_{k})\bigg{)}=_{d}\bigg{(}H_{n}\left(\tilde{\xi},A_{1}\right),...,H_{n}\left(\tilde{\xi},A_{k}\right)\bigg{)},$

which implies that (C.14) follows from (C.19). ∎

The above example, demonstrates that the least squares estimate of the
autoregressive coefficient in the case of the unstable autoregression model
($\beta=1$), induces an invalidity of the bootstrap procedure. A similar
invalidity of the bootstrap procedure occurs when considering the distribution
of $n\left(\hat{\beta}_{n}-\beta\right)$ as examined by Chan and Wei,  (1987).
In this paper, we aim to demonstrate that in the case of predictive regression
model, when we estimate the unknown parameter using the IVX instrumentation
procedure, then the limiting distribution of the original estimate is
preserved which verifies the validity of the corresponding IVX bootstrap
estimate.

##  Appendix D Bootstrapping Explosive Autoregressive Processes.

###  Preliminary results and the bootstrap estimate.

First, we review some basic limit results for the explosive case as in Bose,
(1988). We assume that $|\beta|>1$ and define with

(D.1) $\displaystyle U_{n}=\sum_{t=1}^{n}\beta^{-(t-1)}\epsilon_{t}\ \
\text{and}\ \ V_{n}=\sum_{t=1}^{n}\beta^{-(n-t)}\epsilon_{t}$

where $\left\\{\epsilon_{t}\right\\}$ are i.i.d random variables with
$\mathbf{E}\left(\epsilon_{j}\right)=0$ and
Var$\left(\epsilon_{j}\right)=\sigma^{2}$. Moreover, it can be shown that
$U_{n}$ and $V_{n}$ are identically distributed for each $n$ and there exist
random variables $U$ and $V$ such that

(D.2) $\displaystyle\left(U_{n},V_{n}\right)\to_{d}\left(U,V\right)\ \
\text{as}\ \ n\to\infty,$

where $U$ and $V$ are independent and identically distributed. If
$\left\\{\epsilon_{t}\right\\}$ are normal, $U$ and $V$ are normal each with
mean 0 and variance $\left(1-\beta^{-2}\right)^{-1}$. In the general case, $U$
and $V$ can be represented by

(D.3) $\displaystyle
U\overset{d}{=}\sum_{t=1}^{\infty}\beta^{-(t-1)}\epsilon_{t}\overset{d}{=}V,$

Note that the common characteristic function of $U$ and $V$ is

(D.4)
$\displaystyle\phi(\tau)=\prod_{t=1}^{\infty}\phi_{\epsilon}\left(\beta^{-(t-1)}\tau\right)$

where $\phi_{\epsilon}(.)$ is the characteristic function of $\epsilon_{t}$.

Let $\hat{\beta}_{n}$ denote the least squares estimate of $\beta$. The
following Theorem summarizes the limit distribution of $\hat{\beta}_{n}$ for
the case $|\beta|>1$.

######  Theorem 1.

For $\hat{\beta}_{n}$ defined in the autoregressive model, we have that for
$|\beta|>1$,

(D.5)
$\displaystyle\left(\beta^{2}-1\right)^{-1}|\beta|^{n}\left(\hat{\beta}_{n}-\beta\right)\to_{d}V\big{/}U,$

where $U$ and $V$ are defined in (D.1).

We now describe the corresponding bootstrap estimate. Let
$\hat{\epsilon}_{t}=X_{t}-\hat{\beta}_{n}X_{t-1}$ and define
$\tilde{\epsilon}_{t}=\hat{\epsilon}_{t}-n^{-1}\sum_{t=1}^{n}\hat{\epsilon}_{t}$,
the centered residuals. Denote by $\tilde{F}_{n}$ the empirical distribution
function based on $\left\\{\tilde{\epsilon}_{t},t=1,...,n\right\\}$. Thus,
$\tilde{F}_{n}$ associates mass $n^{-1}$ to each of
$\tilde{\epsilon}_{t},t=1,....,n$. Now, assuming that $\tilde{F}_{n}$ is the
true distribution, draw a random sample
$\left\\{\epsilon^{*}_{t},t=1,...,n\right\\}$ from $\tilde{F}_{n}$. Therefore,
conditionally on $\left(X_{1},...,X_{n}\right)$, the random variables
$\left\\{\epsilon^{*}_{t},t=1,...,n\right\\}$ are i.i.d with distribution
function $\tilde{F}_{n}$. This pseudo-series allow us to construct the
bootstrap sample $\left\\{X_{t}^{*},t=1,...,n\right\\}$ recursively by the
following expression

(D.6) $\displaystyle X_{t}^{*}=\hat{\beta}_{n}X_{t-1}^{*}+\epsilon^{*}_{t},\ \
\ \ t=1,...,n,$

with $X_{0}=0$. The bootstrap least squares estimate is then given by

(D.7)
$\displaystyle\hat{\beta}_{n}^{*}=\left(\sum_{t=1}^{n}X^{*2}_{t-1}\right)^{-1}\left(\sum_{t=1}^{n}X^{*}_{t}X^{*}_{t-1}\right)$

The main aim of this section (see, also Bose,  (1988)), is to derive the limit
distribution of $\hat{\beta}_{n}^{*}$ given $\left(X_{1},X_{2},...\right)$ and
to show that it is the same as the limit distribution given in Theorem (1).
Before doing that, we present below a useful lemma for the purpose of proving
the aforementioned result.

######  Lemma 2.

For the autoregressive model with $\mathbf{E}\epsilon_{t}=0$, Var
$\epsilon_{t}=\sigma^{2}<\infty$ and $|\beta|>1$, we have

(D.8)
$\displaystyle\frac{1}{n}\sum_{t=1}^{n}\left(\tilde{\epsilon}_{t}-\epsilon_{t}\right)\overset{a.s}{\to}0,\
\ \text{as}\ n\to\infty.$

Moreover,

(D.9) $\displaystyle\hat{\beta}_{n}\overset{a.s}{\to}\beta\ \ \text{as}\
n\to\infty.$

In practise, we denote the empirical distribution of the centred residuals
$\tilde{\epsilon}_{t}$ as $\tilde{F}_{t}$. Notice that the bootstrap
innovations $\left\\{\epsilon_{t}^{*}\right\\}$ are conditionally independent
with common distribution $\tilde{F}_{T}$.

###  Limit distribution of the bootstrap estimate.

We now state the main result of this section.

######  Theorem 2.

Conditionally on $\left(X_{1},...,X_{n}\right)$ as $n\to\infty$ we have, for
$|\beta|>1$,

(D.10)
$\displaystyle\left(\hat{\beta}_{n}^{2}-1\right)^{-1}|\hat{\beta}_{n}|^{n}\left(\hat{\beta}_{n}^{*}-\hat{\beta}_{n}\right)\to_{d}V\big{/}U$

for almost all sample paths $\left(X_{1},X_{2},...\right)$, where $U$ and $V$
are defined in (D.1), and $\left(\hat{\beta}_{n},\hat{\beta}_{n}^{*}\right)$
are defined, above.

###### Proof.

Define,

(D.11) $\displaystyle
U_{n}^{*}=\sum_{t=1}^{n}\hat{\beta}^{-(t-1)}\epsilon_{t}^{*}\ \ \text{and}\ \
V_{n}^{*}=\sum_{t=1}^{n}\hat{\beta}^{-(n-t)}\epsilon_{t}^{*}$

The result in the theorem can be deduced analogously to that of Theorem (1)
provided we show that, conditionally on $\left(X_{1},...X_{n}\right)$,
$\left(U_{n}^{*},V_{n}^{*}\right)\to_{d}\left(U,V\right)$ for almost all
sample paths, where $U_{n}^{*}$, $V_{n}^{*}$, $U$ and $V$ are as defined
above. This is due to the fact that the limit distributions of
$\hat{\beta}_{n}^{*}$ and $\hat{\beta}_{n}$ are determined respectively by
those of $\left(U_{n}^{*},V_{n}^{*}\right)$ and $\left(U,V\right)$. Thus, for
the remaining derivations we shall show that the conditionally characteristic
function of $\left(U_{n}^{*},V_{n}^{*}\right)$ given
$\left(X_{1},...X_{n}\right)$ converges to the characteristic function of
$\left(U,V\right)$, for almost all sample paths
$\left(X_{1},X_{2},...\right)$. Let $\mathbf{E}^{*}(.)$ denote the expectation
with respect to the distribution $\tilde{F}_{n}$ (of $\epsilon_{t}^{*}$ )
conditional on $\left(X_{1},...X_{n}\right)$. Let
$\phi_{U_{n}^{*}(\tau)}=\mathbf{E}^{*}\left(\text{exp}\ i\tau
U_{n}^{*}\right)$.

Thus, it is shown that

(D.12)
$\displaystyle\phi_{U_{n}^{*}(\tau)}=\prod_{t=1}^{n}\mathbf{E}^{*}\left(\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\epsilon_{t}^{*}\right)$

converges a.s to $\phi_{U(\tau)}$ for each $\tau\in\mathbb{R}$.

To accomplish this it suffices to show that

(D.13) $\displaystyle\underset{m\to\infty}{\text{lim}}\ \underset{n\leq
m}{\text{sup}}\sum_{t=m}^{n}\bigg{|}\mathbf{E}^{*}\left(\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\epsilon_{t}^{*}\right)-1\bigg{|}=0\ \ \
\text{a.s},$

where the convergence in (D.2) is uniform on compact sets in $\mathbb{R}$ and
it holds that

(D.14)
$\displaystyle\underset{n\to\infty}{\text{lim}}\bigg{|}\mathbf{E}^{*}\left(\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\epsilon_{t}^{*}\right)-\phi_{\epsilon_{1}}\left(\tau\beta^{-(t-1)}\right)\bigg{|}=0\
\ \ \text{a.s}.$

for each $t\geq 1$ and uniformly in $\tau$ in bounded intervals.

First, note that

$\displaystyle\bigg{|}\mathbf{E}^{*}\left(\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\epsilon_{t}^{*}\right)-1\bigg{|}$
$\displaystyle\leq\mathbf{E}^{*}\left(|\tau\hat{\beta}_{n}^{(t-1)}\epsilon_{t}^{*}|\right)$
(D.15)
$\displaystyle=|\tau||\hat{\beta}_{n}|^{-(t-1)}\frac{1}{n}\sum_{j=1}^{n}|\tilde{\epsilon}_{j}^{*}|.$

Since by Lemma (2) $\hat{\beta}_{n}\overset{\text{a.s}}{\to}\beta$,
$|\beta|>1$ and
$(1/n)\sum_{j=1}^{n}|\tilde{\epsilon}_{t}|\overset{\text{a.s}}{\to}\mathbb{E}|\epsilon_{1}|<\infty$,
it follows that almost for almost each sample path there exist positive
integers $C$ and $m$ such that $(1/n)\sum_{j=1}^{n}|\tilde{\epsilon}_{j}|\leq
C$ and $|\hat{\beta}_{n}|\geq\delta>1$ for all $n\geq m$. Thus, for almost
each sample path

(D.16) $\displaystyle\bigg{|}\mathbf{E}^{*}\left(\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\epsilon_{t}^{*}\right)-1\bigg{|}\leq|\tau|\delta^{-(t-1)}C$

for all $n\geq m$, and is established. Next, we have that

$\displaystyle\bigg{|}\mathbf{E}^{*}$ $\displaystyle\left(\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\epsilon_{t}^{*}\right)-\phi_{\epsilon_{1}}\left(\tau\beta^{-(t-1)}\right)\bigg{|}$
$\displaystyle\leq\bigg{|}\frac{1}{n}\sum_{j=1}^{n}\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\tilde{\epsilon}_{j}-\frac{1}{n}\sum_{j=1}^{n}\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\epsilon_{j}\bigg{|}$
$\displaystyle+\bigg{|}\frac{1}{n}\sum_{j=1}^{n}\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\epsilon_{j}-\frac{1}{n}\sum_{j=1}^{n}\text{exp}\
i\tau\beta_{n}^{-(t-1)}\epsilon_{j}\bigg{|}$ (D.17)
$\displaystyle+\bigg{|}\frac{1}{n}\sum_{j=1}^{n}\text{exp}\
i\tau\beta_{n}^{-(t-1)}\epsilon_{j}-\mathbf{E}\left(\text{exp}\
i\tau\beta^{-(t-1)}\epsilon_{1}\right)\bigg{|}.$

Therefore, for the first term in (D.2) we obtain,

$\displaystyle\bigg{|}\frac{1}{n}$ $\displaystyle\sum_{j=1}^{n}\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\tilde{\epsilon}_{j}-\frac{1}{n}\sum_{j=1}^{n}\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\epsilon_{j}\bigg{|}$
$\displaystyle\leq\frac{1}{n}\sum_{j=1}^{n}\big{|}\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\epsilon_{j}\big{|}\big{|}\text{exp}\
i\tau\hat{\beta}_{n}^{-(t-1)}\left(\tilde{\epsilon}_{j}-\epsilon_{j}\right)-1\big{|}$
$\displaystyle\leq\frac{|\tau|}{n}\sum_{j=1}^{n}\big{|}\hat{\beta}_{n}^{-(t-1)}\big{|}\big{|}\tilde{\epsilon}_{j}-\epsilon_{j}\big{|}$
$\displaystyle\leq\left(|\tau|\hat{\beta}_{n}^{-2(t-1)}\right)^{1/2}\left(\frac{|\tau|}{n}\sum_{j=1}^{n}\left(\tilde{\epsilon}_{j}-\epsilon_{j}\right)^{2}\right)^{1/2}\to
0\ \ \ \text{a.s}.$

uniformly on $\tau$ on compact subsets by Lemma 2. Similarly, we obtain that

(D.18)
$\displaystyle\big{|}\hat{\beta}_{n}^{-(t-1)}-\beta^{-(t-1)}\big{|}\frac{|\tau|}{n}\sum_{j=1}^{n}|\epsilon_{j}|\to
0\ \ \ \text{a.s}.$

uniformly in $\tau$ on compact subsets, which implies that the second term in
(D.2) goes to zero almost surely.

By the Glivenko-Cantelli theorem and the convergence theorem for
characteristic functions, the third term in (D.2) goes to 0 a.s. and uniformly
for $\tau$ in a compact set. Hence, (D.14) is established, and it follows that

(D.19)
$\displaystyle\phi_{U^{*}_{n}}(\tau)=\prod_{t=1}^{n}\phi_{\epsilon^{*}_{1}}\left(\tau\hat{\beta}_{n}^{-(t-1)}\right)\overset{\text{a.s}}{\to}\prod_{t=1}^{\infty}\phi_{\epsilon_{1}}\left(\tau\beta^{-(t-1)}\right)=\phi_{U}(\tau).$

Similarly, it can be shown that

(D.20)
$\displaystyle\phi_{U^{*}_{n},V^{*}_{n}}(\tau,s)\overset{\text{a.s}}{\to}\prod_{t=1}^{\infty}\phi_{\epsilon^{*}_{1}}\left(\tau\hat{\beta}^{-(t-1)}\right)\prod_{t=1}^{\infty}\phi_{\epsilon^{*}_{1}}\left(s\hat{\beta}^{-(t-1)}\right)=\phi_{U}(\tau)\phi_{V}(s).$

∎

######  Corollary 1.

If $\left\\{\epsilon_{t}\right\\}$ are assumed normal, we have, for
$|\beta|>1$, conditionally on $\left(X_{1},...,X_{n}\right)$,

(D.21) $\displaystyle
T_{n}=\hat{\sigma}_{n}^{-1}\left(\sum_{t=1}^{n}X^{*2}_{t-1}\right)^{1/2}\left(\hat{\beta}_{n}^{*}-\hat{\beta}_{n}\right)\to_{d}\mathcal{N}\left(0,1\right),$

for almost all sample paths, where

(D.22)
$\displaystyle\hat{\sigma}_{n}^{2}=\sum_{t=1}^{n}\left(X_{t}-\hat{\beta}_{n}X_{t-1}\right)^{2}.$

##  Appendix E Residual based Block Bootstrap for unit root testing.

###  Estimation framework.

Consider the autoregressive model given by

(E.1) $\displaystyle X_{t}=\rho_{n}X_{t-1}+U_{t},\ \ \ t=1,2,...$

where $\rho_{n}=1+c/n$, $c<0$ and the stationary process
$\left\\{U_{t}\right\\}$ satisfies: $E\left(U_{t}\right)=0$,
$E\left(|U_{t}|^{\nu}\right)<\infty$ for some $\nu>2$, $f_{U}(0)>0$ and
$\sum_{k=1}^{\infty}\alpha(k)^{1-2/\nu}<\infty$, where $\alpha(.)$ denotes the
strong mixing coefficient of $\left\\{U_{t}\right\\}$. In this Section, we
summarize the RBB testing Algorithm as proposed by Paparoditis and Politis,
(2003). More specifically, the algorithm is carried out conditionally on the
original data $\left\\{X_{1},....,X_{n}\right\\}$ and implicitly defines a
bootstrap probability mechanism denoted by $P^{*}$ that is capable of
generating bootstrap pseudo-series of the type
$\left\\{X_{t}^{*},t=1,2,...\right\\}$.

###  RBB Testing Algorithm.

The RBB procedure could be indeed an interesting application to examine, but
we first consider the validity of the bootstrap IVX estimate constructed via
the simple bootstrap procedure for obtaining the residual sequence. Then after
establishing this asymptotic equivalence, we could examine the implementation
of the RBB algorithm for obtaining the IVX bootstrap estimate.

  1. Step 1.

First calculate the centered residuals given by

(E.2)
$\displaystyle\hat{v}_{t}=\left(X_{t}-\tilde{\rho}_{n}X_{t-1}\right)-\frac{1}{n-1}\sum_{j=2}^{n}\big{(}X_{j}-\tilde{\rho}_{n}X_{j-1}\big{)}$

for $t=2,3,...,n$ where
$\tilde{\rho}_{n}=\tilde{\rho}_{n}\big{(}X_{1},X_{2},...,X_{n}\big{)}$ is a
consistent estimator of $\rho_{n}$ based on the observations
$\left\\{X_{1},...,X_{n}\right\\}$.

  2. Step 2.

Choose a positive integer $b$, where $b<n$ and let $i_{0},...,i_{k-1}$ be
drawn i.i.d with distribution uniform on the set
$\left\\{1,2,...,n-b\right\\}$, such that for example,
$k=\left[\frac{(n-1)}{b}\right]$. Then, the procedure constructs a bootstrap
pseudo-series $\left\\{X_{1}^{*},....,X_{l}^{*}\right\\}$ where $l=kb+1$, as
follows

(E.3) $X_{t}^{*}=\begin{cases}X_{1}&,\text{for}\ t=1,\\\
\hat{\mu}+X^{*}_{t-1}+\hat{v}_{i_{m}+s}&,\text{for}\ t=2,3,...,l,\end{cases}$

where $m=\left[\frac{(t-2)}{b}\right]$, $s=t-mb-1$, and $\hat{\mu}$ is a drift
parameter that is either equal to zero or represents a consistent estimator of
$\mu$.

  3. Step 3.

Let $\hat{\rho}_{n}$ be the estimator used to perform the unit root test.
Compute the pseudo-statistic $\rho^{*}$ which is the test statistic
$\hat{\rho}_{l}$ based on the pseudo-data
$\left\\{X_{1}^{*},....,X_{l}^{*}\right\\}$.

Taking into account the asymptotic theory of the regression statistic
$n\left(\hat{\rho}_{LS}-1\right)$ for near integrated processes, the following
theorem about the asymptotic local power behavior of the RBB based test can be
established.

######  Theorem 3.

Let

(E.4)
$\displaystyle\mathcal{B}_{RBB,n}\left(\rho_{n};\alpha\right)\to\mathbb{P}\left(J\leq\mathcal{C}_{\alpha}-c\right)$

convergence in probability, where $\mathcal{C}_{\alpha}$ is the $\alpha$
quantile of the distribution of

(E.5)
$\displaystyle\bigg{(}W^{2}(1)-\sigma_{U}^{2}/\sigma\bigg{)}\bigg{(}2\int_{0}^{1}W^{2}(r)dr\bigg{)}^{-1}$

where J is a random variable which has the following distribution

(E.6)
$\displaystyle\bigg{(}\int_{0}^{1}J_{c}(r)dW(r)+\left(1-\frac{\sigma^{2}_{U}}{\sigma^{2}}\right)\bigg{)}\bigg{(}\int_{0}^{1}J_{c}^{2}(r)dr\bigg{)}^{-1}$

and $J_{c}(r)=\int_{0}^{1}e^{(r-s)c}dW(s)$ is the OU process generated by the
stochastic differential equation $dJ_{c}(r)=cJ_{c}(r)dr+dW(r)$ with initial
condition $J_{c}(0)=0$.

###  FCLT for the Bootstrap Partial Sum process.

The asymptotic properties of the RBB testing procedure are based on the
stochastic behaviour of the standardized partial sum process
$\left\\{S_{m}^{*}(r),0\leq r\leq 1\right\\}$, defined by

(E.7) $\displaystyle
S_{m}^{*}(r)=\frac{1}{m}\sum_{t=1}^{j-1}\frac{v_{t}^{*}}{\sigma^{*}},\ \ \ \ \
\text{for}\ \ \frac{(j-1)}{m}\leq r\leq\frac{j}{m},\ \ \ j=2,...,m.$

and also

(E.8) $\displaystyle
S_{m}^{*}(1)=\frac{1}{m}\sum_{t=1}^{m}\frac{v_{t}^{*}}{\sigma^{*}},\ \ \ \ \
\text{for}\ \ \frac{(j-1)}{m}\leq r\leq\frac{j}{m},\ \ \ j=2,...,m.$

where for example, we define that $v_{1}^{*}\equiv X_{1}$,
$v_{t}^{*}=X_{t}^{*}-\hat{\beta}-X_{t-1}^{*}$, for $t=2,3,...,m$, and
$\sigma^{*2}=\text{var}\left(m^{-1/2}\sum_{j=1}^{m}v_{j}^{*}\right)$.

Note that, the partial sum process $S_{m}^{*}(r)$ is considered to be a random
element in the function space $\mathcal{D}[0,1]$, which represent the space of
all real valued functions on the interval $[0,1]$ that are right continuous at
each point and have finite left limits.

The following theorem, given by Paparoditis and Politis,  (2003), shows that
under a general set of assumptions on the process $\left\\{X_{t}\right\\}$,
and conditionally on the observed series $X_{1},X_{2},...,X_{n}$, the
bootstrap partial sum process defined by (E.7) and (E.8) converges weakly to
the standard Wiener process on $[0,1]$. Moreover, we denote with
$T_{n}^{*}=T_{n}^{*}\left(X_{1}^{*},X_{2}^{*},....,X_{n}^{*}\right)$ is a
random sequence based on the bootstrap sample
$X_{1}^{*},X_{2}^{*},....,X_{n}^{*}$ and $G$ is a random measure, then the we
use $T_{n}^{*}\Rightarrow G$ to denote the convergence in probability, which
means that the distance between the law of $T_{n}^{*}$ and the law of $G$
tends to zero in probability for any distance metrizing weak convergence.

######  Theorem 4.

Let $\left\\{X_{t}\right\\}$ be a stochastic process, and assume that the
process $\left\\{v_{t}\right\\}$ defined by $v_{t}=X_{t}-\rho X_{t-1}$
satisfies the above regulatory conditions, and let $\tilde{\rho}_{n}$ be an
estimator of $\rho$. Moreover, if $b\to\infty$ such that $b\big{/}\sqrt{n}\to
0$ as $n\to\infty$, then

(E.9) $\displaystyle S_{m}^{*}\Rightarrow W\ \ \ \text{in probability}.$

Therefore, the above result along with a bootstrap version of the continuous
mapping theorem enables us to apply the block bootstrap proposal of this paper
in order to approximate the null distribution of a variety of different test
statistics.

######  Theorem 5.

Assume that the process $\left\\{X_{t}\right\\}$ satisfies the above
conditions with $\beta=0$. If $b\to\infty$ but $b\big{/}\sqrt{n}$ as
$n\to\infty$, then we have that

(E.10) $\displaystyle\underset{x\in\mathbb{R}}{\text{sup}}\
\bigg{|}\mathbb{P}^{*}\left(l\left(\hat{\rho}^{*LS}_{n}-1\right)\leq
x\big{|}X_{1},...,X_{n}\right)-\mathbb{P}_{0}\left(\left(\hat{\rho}^{LS}-1\right)\leq
x\right)\bigg{|}\to 0$

in probability, and

(E.11) $\displaystyle\underset{x\in\mathbb{R}}{\text{sup}}\
\bigg{|}\mathbb{P}^{*}\left(l\left(\hat{\rho}^{*LS}_{c,n}-1\right)\leq
x\big{|}X_{1},...,X_{n}\right)-\mathbb{P}_{0}\left(\left(\hat{\rho}_{c,n}^{LS}-1\right)\leq
x\right)\bigg{|}\to 0$

in probability, where $\mathbb{P}_{0}$ denotes the probability measure
corresponding to the case where the statistics $\hat{\rho}^{*LS}_{n}$ and
$\hat{\rho}^{*LS}_{c,n}$ are computed from a stretch of size $n$ from the unit
root process obtained by integrating $\left\\{U_{t}\right\\}$.

##  Appendix F Supplementary Limit Theorems.

In this section we explain in more details the implications of having a random
limit distribution in the related theory for bootstrapping. In other words,
the theoretical result which we aim to prove involves inference based on
stochastic limit bootstrap measures555Notice that our approach in this paper
is different from examining the conditional versus unconditional validity of
the bootstrap for predictive regression models. . To begin with, we assume
that the standard conditional weak convergence result applies,
$\underset{x\in\mathbb{R}}{\text{sup}}\big{|}F_{n}^{*}(x)-F(x)\big{|}\to_{p}0$.
Furthermore, denote with $\mathbb{E}^{*}$ the expectation under the
probability measure induced by the standard bootstrap.

######  Theorem 6.

Suppose that $\left\\{X_{n,j},\mathcal{F}_{n,j}\right\\}$ is a martingale
difference array. Let $\left\\{\mathcal{J}_{n}(r),r\in[0,1]\right\\}$ be a
sequence of adapted time scales and
$\left\\{\mathcal{J}(r),r\in[0,1]\right\\}$ a continuous, nonrandom function.
If the following holds,

(F.1) $\displaystyle\forall\epsilon>0,\
\sum_{j=1}^{\mathcal{J}(1)}\mathbf{E}\left(X_{n,j}^{2}\mathbf{1}\left\\{|X_{n,j}|>\epsilon\right\\}\big{|}\mathcal{F}_{n,j-1}\right)$
$\displaystyle\to_{p}0,\ \ \text{as}\ \ n\to\infty$ (F.2)
$\displaystyle\sum_{j=1}^{\mathcal{J}_{n}(r)}\mathbf{E}\left(X^{2}_{n,j}\big{|}\mathcal{F}_{n,j-1}\right)$
$\displaystyle\to_{p}\mathcal{J}(r),\ \ \text{as}\ \ n\to\infty,r\in[0,1],$

Then

(F.3)
$\displaystyle\sum_{j=1}^{\mathcal{J}_{n}(r)}X_{n,j}\to_{d}W\left(\mathcal{J}(r)\right),\
\ \text{as}\ \ n\to\infty,\ \ \text{in}\ \ \mathcal{D}[0,1].$

A sequence $\left\\{P,P_{n}\right\\}$ of probability measures on the metric
space $(S,d)$ converges weakly, when

(F.4) $\displaystyle\int\phi(x)dP_{n}(x)\to\int\phi(x)dP(x),\ \ n\to\infty,$

holds true for all $\phi\in C_{b}(S,\mathbb{R})$.

######  Theorem 7.

(Continuous Mapping Theorem) Let $\left\\{X,X_{n}\right\\}$ be a sequence of
random elements taking values in some metric space $(S,d)$ equipped with the
associated Borel $\sigma-$field. Assume that

(F.5) $\displaystyle X_{n}\Rightarrow X,\ \ \text{as}\ \ n\to\infty,$

If $\phi:S\to S^{\prime}$, is a mapping into another metric space $S^{\prime}$
with metric $d^{\prime}$ that is almost surely, continuous on
$X(\Omega)\subset S$, then

(F.6) $\displaystyle\phi(X_{n})\overset{d}{\to}\phi(X),\ \ \text{as}\ \
n\to\infty,$

######  Theorem 8.

(Joint Weak Convergence)

Let $\left\\{X,X_{n}\right\\}$ and $\left\\{Y,Y_{n}\right\\}$ be two sequences
taking values in $(S_{1},d_{1})$, respectively, $(S_{2},d_{2})$, such that
some conditions hold. Then,

(F.7) $\displaystyle(X_{n},Y_{n})\Rightarrow(X,Y),$

as $n\to\infty$, provided at least one of the following conditions is
satisfied

  1. (i)

$Y=c\in S_{2}$ is a constant, that is, non-random.

  2. (ii)

$X_{n}$ and $Y_{n}$ are independent for all $n$ as well as $X$ and $Y$ are
independent.

Recall that we are given a metric space $(S,d)$ such that as
$\mathcal{D}\left([0,1],\mathbb{R}\right)$ equipped with the Borel
$\sigma-$field and the associated set $\mathcal{P}(S)$ of probability
measures. Then, the space $\mathcal{P}(S)$ can be metrized by the Prohorov
metric $\pi$ and the convergence with respect to the Prohorov metric is the
weak convergence, that is,

(F.8) $\displaystyle P_{n}\Rightarrow P\ \ \text{if and only if}\
\phi\left(P_{n},P\right)\to 0.$

In a metric space convergence can be characterized by a sequence criterion: A
sequence converges if and only if any subsequence contains a further
convergent subsequence. Therefore, the weak convergence of a sequence
$\left\\{P_{n},P\right\\}$ of probability measures can be characterized in the
following way:

(F.9) $\displaystyle P_{n}\Rightarrow P\iff\pi(P_{n},P)\to 0,\ \ \text{as}\
n\to\infty$

if and only if any subsequence $\left\\{P_{n_{k}}:k\geq 1\right\\}$ contains a
further subsequence $\left\\{P_{n_{k^{\prime}}}:k\geq 1\right\\}$ such that
$P_{n_{k^{\prime}}}\Rightarrow P\iff\pi(P_{n_{k^{\prime}}},P)\to 0$, as
$k\to\infty$, which is equivalent to $P_{n_{k^{\prime}}}\Rightarrow P$, as
$k\to\infty$.

Further, in any metric space a subset $A$ is relatively compact, that is, has
a compact closure, if every subsequence $\left\\{P_{n}\right\\}\subset A$ has
a subsequence $\left\\{P_{n_{k^{\prime}}}:k\geq 1\right\\}$ with
$P_{n_{k^{\prime}}}\to P^{\prime}$ as $k\to\infty$, where the limit
$P^{\prime}$ is in the closure of $A$ is relatively compact. Applied to out
setting this means: A subset $A\subset\mathcal{P}(\mathcal{S})$ of probability
measures has compact closure $\bar{A}$ if and only if every sequence
$\left\\{P_{n}\right\\}\subset A$ has a subsequence
$\left\\{P_{n_{k}}\right\\}$ with converges weakly to some
$P^{\prime}\in\bar{A}$, such that, $P_{n_{k}}\Rightarrow P^{\prime}$ as
$k\to\infty$.

Here the limit $P^{\prime}$ may depend on the subsequence. Therefore, the weak
convergence $P_{n}\Rightarrow P$ as $n\to\infty$. First, one shows that
$\left\\{P_{n}\right\\}$ is relatively compact and then one verifies that all
possible limits are equal to $P$. A theorem due to Prohorov allows us to
relate the compact sets of $\mathcal{P}(S)$ to the compact sets of $S$. This
is achieved by the concept of tightness. A subset $A\subset\mathcal{P}(S)$ of
probability measures in $\mathcal{P}(S)$ is called tight if for all
$\epsilon>0$ there exists a compact subset $K_{\epsilon}\subset S$ such that

(F.10) $\displaystyle P\left(K_{\epsilon}\right)>1-\epsilon,\ \ \text{for
all}\ \ P\in A.$

##### Functional Central Limit Theorems

######  Theorem 9.

(Donsker i.i.d case)

Let $\xi_{1},\xi_{2}...$ be a sequence of i.i.d random variables with
$\mathbb{E}\left(\xi_{1}\right)=0$ and
$\sigma^{2}=\mathbb{E}\left(\xi_{1}^{2}\right)<\infty$. Then,

(F.11) $\displaystyle\frac{1}{\sqrt{T}}\sum_{t=1}^{\lfloor
Ts\rfloor}\xi_{t}\Rightarrow\sigma B(s),$

as $T\to\infty$, where $B$ denotes the standard Brownian motion and
$\Rightarrow$ signifies weak convergence in the Skorohod space
$\mathcal{D}\left([0,1],\mathbb{R}\right)$.

######  Theorem 10.

Suppose $\xi_{1},\xi_{2},...$ satisfies a weak invariance principle. Then,

(F.12) $\displaystyle S_{T}(u)=\frac{1}{\sqrt{T}}S\left(\lfloor
Tu\rfloor\right)\Rightarrow B(u),$

as $T\to\infty$.

###### Proof.

We have that $\left\\{B(u):u\geq 0\right\\}$ is equal in distribution to
$\left\\{\frac{1}{\sqrt{T}}B(Tu):u\geq 0\right\\}$ for each $T$. Therefore,

(F.13)
$\displaystyle\underset{u\in[0,1]}{\mathsf{sup}}\left|S_{T}(u)-B(u)\right|\overset{d}{=}\underset{u\in[0,1]}{\mathsf{sup}}\left|\frac{1}{\sqrt{T}}\sum_{t=1}^{\lfloor
Tu\rfloor}\xi_{t}-\frac{1}{\sqrt{T}}B(Tu)\right|.$

Therefore, we can conclude that on a new probability space,

$\displaystyle\underset{u\in[0,1]}{\mathsf{sup}}\left|S_{T}(u)-B(u)\right|$
$\displaystyle\overset{d}{=}\underset{u\in[0,1]}{\mathsf{sup}}\frac{1}{\sqrt{T}}\left|\sum_{t=1}^{\lfloor
Tu\rfloor}\xi_{t}-B(Tu)\right|$ $\displaystyle=\frac{1}{\sqrt{T}}\
\underset{n\leq T}{\mathsf{max}}\ \left|S(n)-B(n)\right|\overset{p}{\to}0$

as $T\to\infty$, which implies that $S_{T}\Rightarrow B$ as $T\to\infty$, for
the original processes. ∎

Reference: Nuisance-parameter-free changepoint detection in non-stationary
series

##### Proof of Theorem 3

Let $\left\\{X_{t}\right\\}_{n=1}^{\infty}$ be i.i.d random variables. Then,
the bootstrap partial sum process

(F.14) $\displaystyle
S_{n}^{\star}(t):=\frac{1}{\sqrt{n}}V_{n}^{\star}\left(\lfloor
nt\rfloor\right)=\frac{1}{\sqrt{n}}\sum_{k=1}^{\lfloor
nt\rfloor}Y_{n,k}^{\star}$

for $t\in[0,1]$ has conditionally on $\left\\{Y_{n,k}\right\\}_{k=1}^{n}$ the
same distribution as
$\big{(}W\left(\nu_{n}^{\star}(t)\right)\big{)}_{t\in[0,1]}$ for some standard
Wiener process $W$ and

(F.15)
$\displaystyle\nu_{n}^{\star}(t)=\mathsf{Var}\left[\frac{1}{\sqrt{n}}\sum_{k=1}^{\lfloor
nt\rfloor}Y_{n,k}^{\star}\big{|}\left\\{Y_{n,k}\right\\}_{k\leq n}\right].$

Then, if $\delta_{n}\to$ as $n\to\infty$, we have that $\nu_{n}^{\star}\to\nu$
uniformly in $t\in[0,1]$ almost surely.

###  Additional Proofs

Next following the excellent statistical framework of Li and Xiao,  (2001) we
establish the convergence of finite-dimensional distributions and verify the
tightness condition. In particular, for the finite-dimensional convergence, it
is sufficient to show that for any finite $k$ and $0<r_{1}<...<r_{k}<1$, if

(F.16) $\displaystyle V_{k}$ $\displaystyle=\bigg{[}S_{\lfloor
Tr_{1}\rfloor},S_{\lfloor Tr_{2}\rfloor},...,S_{\lfloor
Tr_{k-1}\rfloor},S_{\lfloor Tr_{k}\rfloor}\bigg{]}$ (F.17) $\displaystyle
V^{*}_{k}$ $\displaystyle=\bigg{[}S^{*}_{\lfloor Tr_{1}\rfloor},S^{*}_{\lfloor
Tr_{2}\rfloor},...,S^{*}_{\lfloor Tr_{k-1}\rfloor},S^{*}_{\lfloor
Tr_{k}\rfloor}\bigg{]}$

then, it holds that $d_{2}\left(V^{*}_{k},V_{k}\right)\to 0$. Following the
same arguments as in the literature it can be shown that

(F.18) $\displaystyle d_{2}\left(V^{*}_{k},V_{k}\right)^{2}\leq
d_{2}\left(F_{T},\widetilde{F}_{T}\right)\to 0.$

Furthermore, for tightness it suffices to show that there exists a non-
decreasing function $\psi$ such that, for almost all sample paths, for $0\leq
r_{1}\leq r\leq r_{2}\leq 1$,

(F.19) $\displaystyle\mathbb{P}^{*}\bigg{(}\bigg{\\{}\bigg{|}S^{*}_{\lfloor
Tr\rfloor}-S^{*}_{\lfloor
Tr_{1}\rfloor}\bigg{|}\geq\lambda\bigg{\\}}\bigcap\bigg{\\{}\bigg{|}S^{*}_{\lfloor
Tr_{2}\rfloor}-S^{*}_{\lfloor
Tr\rfloor}\bigg{|}\geq\lambda\bigg{\\}}\bigg{)}\leq\frac{1}{\lambda^{4}}\bigg{[}\psi(r_{2})-\psi(r_{1})\bigg{]}$

Notice that $\epsilon_{t}^{*}$ are i.i.d draws from
$\left\\{\tilde{\epsilon}_{t}\right\\}$. Then, by Markov’s inequality we have
that

$\displaystyle\mathbb{P}^{*}\bigg{(}\bigg{\\{}\bigg{|}S^{*}_{\lfloor
Tr\rfloor}-S^{*}_{\lfloor
Tr_{1}\rfloor}\bigg{|}\geq\lambda\bigg{\\}}\bigcap\bigg{\\{}\bigg{|}S^{*}_{\lfloor
Tr_{2}\rfloor}-S^{*}_{\lfloor
Tr\rfloor}\bigg{|}\geq\lambda\bigg{\\}}\bigg{)}\leq\frac{\mathbb{E}^{*}\bigg{|}S^{*}_{\lfloor
Tr\rfloor}-S^{*}_{\lfloor
Tr_{1}\rfloor}\bigg{|}^{2}}{\lambda^{2}}\frac{\mathbb{E}^{*}\bigg{|}S^{*}_{\lfloor
Tr_{2}\rfloor}-S^{*}_{\lfloor Tr\rfloor}\bigg{|}^{2}}{\lambda^{2}}$

Notice that the above bound in probability holds because

(F.20)
$\displaystyle\left\\{\int_{0}^{1}B_{v}(r)^{2}dr\right\\}^{1/2}\left\\{\int_{0}^{1}B_{v}(r)dW_{\epsilon}(r)\right\\}\equiv\mathcal{N}(0,1).$

####  Unit Root bootstrap tests for AR(1) models

Consider the autoregressive model

(F.21) $\displaystyle X_{t}=\beta X_{t-1}+u_{t},\ \ \ X_{0}=0,$

Then, $\hat{\beta}_{n}$ is the OLS estimator of $\beta$ based on a sample of
$n$ observations $(X_{1},...,X_{n})$. Although the OLS estimator
$\hat{\beta}_{n}$ is consistent, its limit distribution is different for the
three possible cases: stationary, unstable and explosive. In particular, for
the unstable case such that $\beta=1$, is is known that the risk of the
normalized error of the estimator given by

(F.22) $\displaystyle
K_{n}:=\frac{1}{\sigma_{u}}\left(\sum_{t=1}^{n}X_{t-1}^{2}\right)^{1/2}\left(\hat{\beta}_{n}-1\right)$

converges weakly to

(F.23) $\displaystyle
K:=\frac{1}{2}\bigg{[}W^{2}(1)-1\bigg{]}\left\\{\int_{0}^{1}W^{2}(t)dt\right\\}^{-1/2},\
\ \ \text{as}\ \ n\to\infty$

where $W(t)$ is the standard Brownian motion on $[0,1]$.

Therefore, in the particular paper the authors prove that their resampling
algorithm is asymptotically correct under the null hypothesis, $H_{0}$, in the
sense that it converges weakly to the limit distribution for almost all
samples $(X_{1},...,X_{n})$.

####  A bootstrap invariance principle

The study of the asymptotic behaviour of the bootstrap least squares estimate
relies on a bootstrap invariance principle, that is, a functional central
limit theorem for a stochastic process constructed from the sequence of
partial sums corresponding to the bootstrap resamples.

Consider the sequence of partial sums

(F.24) $\displaystyle S_{n,0}^{*}=0,\ \
S_{n,k}^{*}\sum_{j=1}^{k}\varepsilon_{n,j}^{*},\ \ \ k=1,...,n,\
n\in\mathbb{N}.$

Then a sequence of continuous-time processes such that
$\left\\{Y^{*}_{n}(s):s\in[0,1],n\in\mathbb{N}\right\\}$

  * •

Thus, for the predictive regression model since the parameter of interest is
the coefficient $\beta$ such that $H_{0}:\beta=0$. Therefore, in other words
in the spirit of Jeganthan (1980) we have proved that the nonstandard problem
of testing for structural break in predictive regression models at an unknown
location, has a limiting distribution with a discontinuity (a critical point)
in the case of persistent predictors or more specifically when the exponent
rate of persistence equals to one.

Furthermore, we introduce some precise bootstrap terminology on bootstrap
convergence that we need. Let $Y=\left(Y_{1},Y_{2},...,Y_{n}\right)$ be a
random sample from a distribution $G$ and let $\uptau(Y;G)$ be the statistic
of interest. A general goal of bootstrap resampling is to approximate the
distribution function $\mathbb{P}\big{(}\uptau(Y;G)\leq x\big{)}$ of the
statistic $\uptau(Y;G)$ by using the distribution function

(F.25) $\displaystyle\mathbb{P}\bigg{(}\uptau(Y^{*};G)\leq x\bigg{)}\ \ \
\text{of}\ \ \ \uptau(Y^{*};G),$

where $Y^{*}=\left(Y^{*}_{1},Y^{*}_{2},...,Y^{*}_{n}\right)$ is the bootstrap
resample.

Therefore, if $\uptau(Y;G)$ converges weakly to a distribution $S(G)$, it is
enough to show that $\uptau(Y^{*};G)$ converges weakly to $S(G)$ for almost
all samples $(Y_{1},Y_{2},)$ or to establish that the distance between the law
of $\uptau(Y^{*};G)$ and the law of $S(G)$ tends to zero in probability for
any distance metrising weak convergence. Therefore, our first lemma
establishes the weak convergence of the finite dimensional distributions of
the processes $Y^{*}_{n}(s)$ for almost all samples $(X_{1},...,X_{n})$.

######  Lemma 3.

Conditionally on $(X_{1},...,X_{n})$ and for almost all sample paths we have
that $(X_{1},X_{2},...)$,
$\big{(}Y^{*}_{1}(s_{1}),Y^{*}_{2}(s_{2}),...,Y^{*}_{n}(s_{d})\big{)}$
converges weakly to $\big{(}W(s_{1}),...,W(s_{d})\big{)}$ as
$n\to\infty,\forall\ (s_{1},...,s_{d})\in[0,1]^{d}$. Furthermore, the
tightness of our sequence of stochastic processes is implemented by the
following results.

Next, we establish the bootstrap invariance principle.

######  Proposition 1.

Let $\left\\{\epsilon_{n}:n\in\mathbb{N}\right\\}$ be a sequence of residuals.
Define with $\widehat{F}_{n}$ to be the empirical distribution of
$\hat{\epsilon}_{t}:=\epsilon_{t}-\frac{1}{n}\sum_{j=1}^{n}\epsilon_{t}$, for
$t=1,...,n$ and let $\hat{\epsilon}^{*}_{n,t}$ for $t=1,...,n$ be independent
random variables with distribution $\widehat{F}_{n}$. Furthermore, define with
$\big{\\{}Y_{n}^{*}(s):s\in[0,1]\big{\\}}$ for $n\in\mathbb{N}$. Then,
$Y_{n}^{*}$ converges weakly to $W$ in $C[0,1]$ almost surely as $n\to\infty$,
where $W$ is the standard one-dimensional Brownian motion on $[0,1]$.

####  Asymptotic behaviour of the bootstrap statistic

Define the following statistic

(F.26) $\displaystyle
Z_{n}^{*}:=\frac{1}{\widehat{\sigma}_{n}}\left(\sum_{t=1}^{n}X_{n,t-1}^{*2}\right)^{1/2}\left(\widehat{\beta}_{n}^{*}-1\right),$

to be the bootstrap version of $Z_{n}$ under $\beta=1$. We derive the limiting
distribution of $Z_{n}^{*}$ in the theorem below. In order to prove the
particular result we need to employ the following lemma.

######  Lemma 4.

Define with

(F.27) $\displaystyle
r_{n}^{*}=\frac{1}{n}\sum_{i=1}^{n}Y_{n}^{*2}\left(\frac{i}{n}\right)-\int_{0}^{1}Y_{n}^{*2}(s)ds.$

Then $r_{n}^{*}$ converges in probability to $0$ as $n\to\infty$,
conditionally on $\left(X_{1},...,X_{n}\right)$, and for almost all sample
paths $\left(X_{1},X_{2},...\right)$.

Our goal is to show that $Z_{n}^{*}$ converges weakly to $Z$ as $n\to\infty$
almost surely and so this bootstrap resampling approaches properly the correct
limiting distribution.

######  Theorem 11.

Under the AR(1) model with $\beta=1$, $Z_{n}^{*}$ defined in (2.3) converges
weakly to Z as $n\to\infty$ for almost all sample
$\left(X_{1},X_{2},...,\right)$, where $Z$ is defined in (1.2).

Consider the following statistic

(F.28)
$\displaystyle\mathcal{T}_{n}:=\frac{1}{s_{n}}\left(\sum_{t=1}^{n}X_{n,t-1}^{2}\right)^{1/2}\left(\widehat{\beta}_{n}-1\right),$

where

(F.29) $\displaystyle
s_{n}^{2}=\frac{1}{n}\sum_{t=1}^{n}\left(X_{t}-\widehat{\beta}_{n}X_{t-1}\right)^{2}.$

In particular, Phillips, 1987a showed that, under $H_{0}$,
$\widehat{\beta}_{n}$ converges in probability to 1, as $n\to\infty$ and
$\mathcal{T}_{n}$ converges weakly to

(F.30)
$\displaystyle\mathcal{T}:=\frac{\sigma}{2\sigma_{u}}\left\\{W^{2}(1)-\frac{\sigma_{u}^{2}}{\sigma^{2}}\right\\}\left\\{\int_{0}^{1}W^{2}(t)dt\right\\}^{-1/2},\
\ \text{as}\ \ n\to\infty.$

with
$\sigma_{u}^{2}=\underset{n\to\infty}{\mathsf{lim}}\frac{1}{n}\sum_{t=1}^{n}\mathbb{E}\left(u_{t}^{2}\right)$.

####  Bootstrap Inference in Cointegrating Regressions

Consider the following system studied by Psaradakis, 2001b

(F.31) $\displaystyle y_{t}$
$\displaystyle=\boldsymbol{\beta}^{\top}\boldsymbol{x}_{t}+u_{t},$ (F.32)
$\displaystyle\boldsymbol{x}_{t}$
$\displaystyle=\boldsymbol{x}_{t-1}+\boldsymbol{v}_{t}$

where $\boldsymbol{\beta}$ is $d-$dimensional parameter vector and
$\boldsymbol{\eta}_{t}=\left(u_{t},\boldsymbol{v}_{t}^{\top}\right)^{\top}\in\mathbb{R}^{d+1}$
is a strictly stationary and ergodic random process with zero mean, finite
covariance matrix, and continuous spectral density matrix
$f_{\boldsymbol{\eta}}(.)$ which is positive definite at zero. For the above
system we are interested in conducting inference about $\boldsymbol{\beta}$
based on the FM-OLS and CCR estimators.

In order to define these estimators we consider the following matrices

(F.33)
$\displaystyle\boldsymbol{\Lambda}=\sum_{j=1}^{\infty}\mathbb{E}\left(\boldsymbol{u}_{t}\boldsymbol{u}_{t+j}^{\top}\right)\
\ \ \text{and}\ \ \
\boldsymbol{\Sigma}=\mathbb{E}\left(\boldsymbol{u}_{t}\boldsymbol{u}_{t}^{\top}\right)$

and decompose the long-run covariance matrix $\boldsymbol{\Omega}=2\pi
f_{\boldsymbol{\eta}}(0)$ of the innovation sequence
$\left\\{\boldsymbol{\eta}\right\\}$ as

(F.34)
$\displaystyle\boldsymbol{\Omega}=\boldsymbol{\Lambda}^{\top}+\boldsymbol{\Delta},\
\ \ \text{where}\ \ \
\boldsymbol{\Delta}=\boldsymbol{\Lambda}+\boldsymbol{\Sigma}$

such that

(F.35)
$\displaystyle\boldsymbol{\Omega}=\begin{pmatrix}\omega_{11}&\boldsymbol{\omega}_{21}^{\top}\\\
\boldsymbol{\omega}_{21}&\boldsymbol{\Omega}_{22}\end{pmatrix},\ \ \ \ \ \
\boldsymbol{\Delta}=\begin{pmatrix}\delta_{11}&\boldsymbol{\delta}_{12}\\\
\boldsymbol{\delta}_{12}^{\top}&\boldsymbol{\Delta}_{22}\end{pmatrix},\ \ \ \
\boldsymbol{\Delta}_{2}=\left(\boldsymbol{\delta}_{12}^{\top},\boldsymbol{\Delta}_{22}^{\top}\right)^{\top}$

Then, the FM-OLS estimator is defined as below

(F.36)
$\displaystyle\tilde{\boldsymbol{\beta}}=\left(\sum_{t=1}^{n}\boldsymbol{x}_{t}\boldsymbol{x}_{t}^{\top}\right)\left(\sum_{t=1}^{n}\boldsymbol{x}_{t}\tilde{y}_{t}-n\tilde{\boldsymbol{\delta}}\right),$

where we have that

(F.37) $\displaystyle\tilde{y}_{t}$
$\displaystyle=y_{t}-\hat{\boldsymbol{\omega}}_{21}^{\top}\hat{\boldsymbol{\Omega}}^{-1}_{22}\Delta\boldsymbol{x}_{t},\
\ \
\Delta\boldsymbol{x}_{t}=\big{(}\boldsymbol{x}_{t}-\boldsymbol{x}_{t-1}\big{)}$
(F.38) $\displaystyle\tilde{\boldsymbol{\delta}}$
$\displaystyle=\left(\hat{\boldsymbol{\delta}}_{12}^{\top},\hat{\boldsymbol{\Delta}}_{22}^{\top}\right)\left(1,-\hat{\boldsymbol{\omega}}_{21}^{\top}\hat{\boldsymbol{\Omega}}^{-1}_{22}\right)^{\top}\equiv\hat{\boldsymbol{\delta}}_{12}^{\top}-\hat{\boldsymbol{\omega}}_{21}^{\top}\hat{\boldsymbol{\Omega}}^{-1}_{22}\hat{\boldsymbol{\Delta}}_{22}^{\top}$

where $\hat{\boldsymbol{\Delta}}$ and $\hat{\boldsymbol{\Omega}}$ are
consistent estimators of $\boldsymbol{\Delta}$ and $\boldsymbol{\Omega}$
respectively. Therefore, testing hypotheses regarding the individual elements
of $\boldsymbol{\beta}$ are based on fully modified t-type statistics which
are constructed based on the estimator $\tilde{\boldsymbol{\beta}}$ and the
diagonal elements of the covariance matrix given by

(F.39)
$\displaystyle\tilde{\boldsymbol{V}}=\left(\hat{\omega}_{11}-\hat{\boldsymbol{\omega}}_{21}^{\top}\hat{\boldsymbol{\Omega}}_{22}^{-1}\hat{\boldsymbol{\omega}}_{21}\right)\left(\sum_{t=1}^{n}\boldsymbol{x}_{t}\boldsymbol{x}_{t}^{\top}\right)^{-1}.$

Notice that the nonsingularity of $\boldsymbol{\Omega}$ is equivalent to

(F.40)
$\displaystyle\omega_{11.2}=\omega_{11}-\omega_{12}\boldsymbol{\Omega}_{22}^{-1}\omega_{21}$

Next we consider the canonical cointegrating regression estimator of
$\boldsymbol{\beta}$ proposed by Park,  (1992), defined as below

(F.41)
$\displaystyle\bar{\boldsymbol{\beta}}=\left(\sum_{t=1}^{n}\bar{\boldsymbol{x}}_{t}\bar{\boldsymbol{x}}_{t}^{\top}\right)^{-1}\left(\sum_{t=1}^{n}\bar{\boldsymbol{x}}_{t}^{\top}\bar{y}_{t}\right),$

Therefore, in the sequel we take
$\hat{\boldsymbol{\Sigma}}=n^{-1}\sum_{t=1}^{n}\hat{\boldsymbol{u}}_{t}\hat{\boldsymbol{u}}_{t}^{\top}$
and estimate the matrices $\boldsymbol{\Omega}$ and $\boldsymbol{\Delta}$
nonparametrically using Parzen lag window, a plug-in bandwidth estimator, and
the least-squares residuals
$\left\\{\hat{\boldsymbol{u}}_{t}\right\\}_{t=1}^{n}$ prewhitened via a first-
order autoregression.

##### Bootstrap Algorithm

Our bootstrap methodology is based on the assumption that
$\left\\{\boldsymbol{\eta}_{t}\right\\}$ admits an AR$(\infty)$ representation
such that

(F.42)
$\displaystyle\boldsymbol{\eta}_{t}=\sum_{j=1}^{\infty}\boldsymbol{\varphi}_{j}\boldsymbol{\eta}_{t-j}+\boldsymbol{\varepsilon}_{t},$

where $\left\\{\boldsymbol{\varepsilon}_{t}\right\\}$ is a zero-mean white-
noise process and an absolutely summable sequence of matrices
$\big{\\{}\boldsymbol{\varphi}_{j}\big{\\}}$ that satisfies the following
regularity condition

(F.43)
$\displaystyle\mathsf{det}\left(\boldsymbol{I}_{p}-\sum_{j=1}^{\infty}\boldsymbol{\varphi}_{j}z^{j}\right)\neq
0\ \ \ \forall|z|\leq 1.$

The particular condition is an assumption that has been extensively used in
the cointegration literature and which is satisfied by a large class of random
processes, including a causal and invertible multivariate ARMA processes.
Then, the idea is to approximate the above linear process representation of
the innovation sequence of the predictive regression model by a finite-order
autoregressive model for a suitably proxy for $\boldsymbol{u}_{t}$ and use
this to generate bootstrap observations which is the common practise when
implementing residual-based bootstrap schemes.

Consider testing the following hypothesis using the bootstrap procedure

(F.44) $\displaystyle\mathcal{H}_{0}:\beta_{i}=\beta_{i,0}\ \ \ \text{versus}\
\ \ \mathcal{H}_{1}:\beta_{i}\neq\beta_{i,0}$

Then, the bootstrap procedure is described as below:

  * Step 1.

Calculate the residuals such that

(F.45)
$\displaystyle\boldsymbol{\eta}^{o}_{t}=\left(u_{t}^{o},\Delta\boldsymbol{x}_{t}^{\top}\right)^{\top}\equiv\left(y_{t}-\boldsymbol{\beta}^{o\top}\boldsymbol{x}_{t},\Delta\boldsymbol{x}_{t}^{\top}\right)^{\top},\
\ \text{for}\ \ t=1,...,n$

where $\boldsymbol{\beta}^{o}$ denotes the FM-OLS or the CCR estimator of
$\boldsymbol{\beta}$ obtained under the null hypothesis which implies that the
$i-$th element of the parameter vector
$\left[\boldsymbol{\beta}^{o}\right]_{i}\equiv\beta_{i,0}$.

  * Step 2.

For a fixed positive integer $p$, obtain the estimates
$\left\\{\hat{\boldsymbol{\Phi}}_{1},...,\hat{\boldsymbol{\Phi}}_{p}\right\\}$
of the coefficient matrices of a $(d+1)-$variate AR$(p)$ model for the
innovation sequence $\left\\{\boldsymbol{\eta}_{t}^{o}\right\\}_{t=1}^{n}$. In
particular, these may be Yule-Walker estimates, least-squares estimates or
Burg-type estimates. Furthermore, define with
$\left\\{\boldsymbol{e}_{t}\right\\}_{t=p+1}^{n}$ to be the corresponding
residuals.

  * Step 3.

For a given positive integer $r$, take a random sample
$\left\\{\boldsymbol{e}^{\star}_{t}\right\\}_{t=-r+1}^{n}$ from the empirical
distribution that puts mass $(n-p)^{-1}$ on each of the centered residuals
such that

(F.46)
$\displaystyle\bar{\hat{\boldsymbol{e}}}_{t}=\left(\hat{\boldsymbol{e}}_{t}-\frac{1}{n-p}\sum_{t=p+1}^{n}\boldsymbol{e}_{t}\right),\
\ \ \text{for}\ t=p+1,...,n$

Then, construct a bootstrap noise series
$\left\\{\boldsymbol{\eta}_{t}^{\star}\right\\}_{t=-r+1}^{n}$ via the
recursion below

(F.47)
$\displaystyle\boldsymbol{\eta}_{t}^{\star}=\sum_{j=1}^{p}\hat{\boldsymbol{\Phi}}_{j}\boldsymbol{\eta}_{t-j}^{\star}+\boldsymbol{e}_{t}^{\star},\
\ \ t=-r+1,...,n,$

where $\boldsymbol{\eta}_{t}^{\star}=\boldsymbol{0}$ for $t\leq-r$.

  * Step 4.

Generate bootstrap replicates
$\left\\{\boldsymbol{x}_{t}^{\star}\right\\}_{t=1}^{n}$ and
$\left\\{\boldsymbol{y}_{t}^{\star}\right\\}_{t=1}^{n}$ by setting

(F.48)
$\displaystyle\boldsymbol{x}_{t}^{\star}=\boldsymbol{x}_{t-1}^{\star}+\boldsymbol{v}_{t}^{\star}\
\ \ \text{and}\ \ \
y_{t}^{\star}=\boldsymbol{\beta}^{o\top}\boldsymbol{x}_{t}^{\star}+u_{t}^{\star},\
\ \ \text{for}\ t=1,...,n$

where we have partitioned the innovation sequence
$\boldsymbol{\eta}_{t}^{\star}$ such that
$\boldsymbol{\eta}_{t}^{\star}=\left(u_{t}^{\star},\boldsymbol{v}_{t}^{\star\top}\right)^{\top}$
and set $\boldsymbol{x}_{0}^{\star}=\boldsymbol{0}$.

  * Step 5.

Estimate $\boldsymbol{\beta}$ by FM-OLS or CCM using the bootstrap sample
$\left\\{\left(y_{t}^{\star},\boldsymbol{x}_{t}^{\star}\right)\right\\}_{t=1}^{n}$
and compute the value of the associated $t-$statistic for
$\mathcal{H}_{0}:\beta_{i}=\beta_{i,0}$, say
$\mathcal{T}^{\star}(\beta_{i,0})$.

  * Step 6.

Repeat steps 3-5 independently $B$ times to obtain sa sample
$\big{\\{}\mathcal{T}_{b}^{\star}(\beta_{i,0})\big{\\}}_{b=1}^{B}$ of
$\mathcal{T}(\beta_{i,0})$ values.

  * Step 7.

Approximate the null sampling distribution of $\mathcal{T}(\beta_{i,0})$ by
the empirical distribution of
$\big{\\{}\mathcal{T}_{b}^{\star}(\beta_{i,0})\big{\\}}_{b=1}^{B}$. The
bootstrap p-value of the observed realization $\hat{\mathcal{T}}(\beta_{i,0})$
of $\mathcal{T}(\beta_{i,0})$ is given by

(F.49) $\displaystyle
p-value=\frac{1}{B}\sum_{b=1}^{B}\mathbf{1}\left\\{\big{|}\mathcal{T}_{b}^{\star}(\beta_{i,0})\big{|}\leq\left|\hat{\mathcal{T}}(\beta_{i,0})\right|\right\\}$

######  Remark 10.

The bootstrap procedure described above uses the residuals obtained under the
null hypothesis based on the Studentized difference between $\beta_{i,0}$ and
an estimate of $\beta_{i}$. In particular, bootstrap tests based on such a
resampling scheme have generally been found to have good small-sample size and
power properties.Furthermore, for the implementation of the sieve bootstrap a
necessary step is the selection of an appropriate value for the order $p$ of
the approximating autoregression for the innovation sequence
$\boldsymbol{\eta}_{t}$.

######  Remark 11.

Notice that we are particularly interested to obtain a good enough
autoregressive approximation for tests to have the correct empirical Type I
error probability. More precisely, as it has been shown in the literature, the
closure with respect to certain metrics of the class of stationary linear
processes and AR$(\infty)$ processes is fairly large. In particular, this
implies that the sieve bootstrap is likely to work reasonably well even when
the data-generating mechanism does not belong to the AR$(\infty)$ family.

In other words, the empirical distribution of
$n^{-1/2}\mathcal{T}_{b}^{\star}(\beta_{i,0})$ yields a conservative estimate
of the empirical distribution of $n^{-1/2}\mathcal{T}(\beta_{i,0})$ as
$n,B\to\infty$. Although, we do not consider formal cross-validation methods
in this paper to evaluate the performance of the bootstrap procedure such
methodologies are discussed in the paper of Efron,  (1983). Furthermore, our
goal is to investigate whether our bootstrap implementation indeed leads to an
asymptotically valid test. In particular, this can be seen by examining
whether the bootstrap consistently estimates the desired quantile.

####  Unit root and Cointegrating limit theory

Consider the cointegrating system as below

(F.50) $\displaystyle\boldsymbol{y}_{t}$
$\displaystyle=\boldsymbol{A}x_{t}+\boldsymbol{u}_{t},$ (F.51)
$\displaystyle\boldsymbol{x}_{t}$
$\displaystyle=\boldsymbol{x}_{t-1}+\boldsymbol{v}_{t}$

where $\boldsymbol{y}_{t}$ is an $m-$dimensional vector and
$\boldsymbol{x}_{t}$ is $K-$dimensional such that
$\boldsymbol{\eta}_{t}=\left(\boldsymbol{u}_{t}^{\prime},\boldsymbol{v}_{t}^{\prime}\right)^{\prime}$
is an $(m+K)-$dimensional vector of innovations and $\boldsymbol{A}$ is an
$(m\times K)$ matrix of cointegrating coefficients.

The mixture process of the limit theory is given by

(F.52)
$\displaystyle\mathsf{vec}\left\\{n\left(\hat{\boldsymbol{A}}^{+}-\boldsymbol{A}\right)\right\\}\Rightarrow\mathcal{MN}\left(\boldsymbol{0},\left(\int_{0}^{1}\boldsymbol{B}_{\uptau}^{+}\boldsymbol{B}_{\uptau}^{+\prime}\right)^{-1}\otimes\boldsymbol{\Omega}_{yy.x}\right),$

where $\hat{A}^{+}$ is the FM regression estimator with a conditional long-run
covariance matrix of $u_{t}$ given $v_{t}$ expressed as below

(F.53)
$\displaystyle\boldsymbol{\Omega}_{yy.x}:=\boldsymbol{\Omega}_{yy}-\boldsymbol{\Omega}_{yx}\boldsymbol{\Omega}_{xx}^{-1}\boldsymbol{\Omega}_{xy}$

Then, the FM regression estimator has the explicit form given by

(F.54)
$\displaystyle\hat{\boldsymbol{A}}^{+}=\left(\hat{\boldsymbol{Y}}^{+\prime}\boldsymbol{X}-n\hat{\boldsymbol{\Delta}}^{+}_{yx}\right)\times\left(\boldsymbol{X}^{\prime}\boldsymbol{X}\right)^{-1}$

where

(F.55)
$\displaystyle\boldsymbol{X}=\big{[}x_{1}^{\prime},...,x_{n}^{\prime}\big{]},\
\ \
\hat{\boldsymbol{Y}}^{+}=\big{[}\hat{y}_{1}^{+\prime},...,\hat{y}_{n}^{+\prime}\big{]}^{\prime}\in\mathbb{R}^{n\times
m}$

such that

(F.56)
$\displaystyle\hat{\boldsymbol{y}}_{t}^{+}=\boldsymbol{y}_{t}-\hat{\boldsymbol{\Omega}}_{yx}\hat{\boldsymbol{\Omega}}_{xx}^{-1}\Delta\boldsymbol{x}_{t},\
\ \ \text{and}\ \ \
\boldsymbol{\Delta}_{yx}^{+}=\boldsymbol{\Delta}_{yx}-\boldsymbol{\Omega}_{yx}\boldsymbol{\Omega}_{xx}^{-1}\boldsymbol{\Delta}_{xx}$

where $\hat{\boldsymbol{\Omega}}_{yx}\hat{\boldsymbol{\Omega}}_{xx}^{-1}$ and
$\hat{\boldsymbol{\Delta}}_{yx}^{+}$ are consistent estimates of
$\boldsymbol{\Omega}_{yx}\hat{\boldsymbol{\Omega}}_{xx}^{-1}$ and
$\boldsymbol{\Delta}_{yx}^{+}$ respectively, which may be constructed in the
familiar fashion using semiparametric lag kernel methods with residuals from a
preliminary cointegrating least squares regression.

###### Proof.

Setting with $u_{y.xt}=u_{yt}-\Omega_{yx}\Omega_{xx}^{-1}\Delta x_{t}$ and
$U_{y.x}=\big{[}u_{y.x1}^{\prime},...,u_{y.xn}^{\prime}\big{]}^{\prime}$ as
the corresponding data matrix, we have that

(F.57)
$\displaystyle\hat{\boldsymbol{y}}_{t}^{+}=\boldsymbol{y}_{t}-\hat{\boldsymbol{\Omega}}_{yx}\hat{\boldsymbol{\Omega}}_{xx}^{-1}\Delta\boldsymbol{x}_{t}=\boldsymbol{A}\boldsymbol{x}_{t}+\boldsymbol{u}_{0.xt}$

∎

In terms of asymptotic theory we have that

(F.58)
$\displaystyle\xi_{y.xn}^{+}(s):=\xi_{yn}(s)-\boldsymbol{\Omega}_{yx}\boldsymbol{\Omega}_{xx}^{-1}\xi_{xn}(s)\Rightarrow\boldsymbol{B}_{y.x}(s)\equiv
BM\big{(}\boldsymbol{\Omega}_{y.xx}\big{)}$

######  Remark 12.

Notice that the unit root limit theory given by expressions (17) and (18) of
PM (2009) involves the demeaned process $B^{\mu}(s)$ although there is no
intercept in the regression. In particular, the demeaning effects arises
because as shown in expression (16), in the direction of the initial
condition, the time series is dominated by a component that behaves like a
constant.

Therefore, the sample moment matrix is no longer asymptotically singular such
that

(F.59)
$\displaystyle\frac{1}{n^{2}}\sum_{t=1}^{n}\boldsymbol{x}_{t-1}\boldsymbol{x}_{t-1}^{\prime}\Rightarrow\int_{0}^{1}J_{c}^{*}(r)J_{c}^{*\prime}(r)$

Related theory regarding the weak convergence to the matrix stochastic
integral is presented by Phillips, 1988b . We briefly summarize the main
results below. We define the partial sum process such that
$S_{t}=\sum_{j=1}^{t}u_{j}$ and construct the following random element of
$D^{n}[0,1]$ such that

(F.60) $\displaystyle X_{T}(r)=T^{-1/2}S_{\lfloor Tr\rfloor}=T^{-1/2}S_{j-1},\
\ \ \ \text{with}\ \ \frac{(j-1)}{T}\leq r\leq\frac{j}{T}.$

Then as $T\to\infty$, we have that

(F.61) $\displaystyle X_{T}(r)\Rightarrow B(r),$

where $B(r)$ is vector Brownian motion on $C^{n}[0,1]$, with covariance matrix
$\Omega$. Therefore, one major time series application is to the theory of
regression for integrated processes. In particular,
$\left\\{u_{t}\right\\}_{t=1}^{\infty}$ is generated by a linear process such
as a finite-order stationary and invertible vector ARMA model then $y_{t}$ is
known as an integrated process of order one. Furthermore, we are often
interested in the asymptotic behaviour of statistics from linear least-squares
regressions with integrated processes. Then, from the first-order vector
autoregression of $y_{t}$ on $y_{t-1}$ we obtain the regression coefficient
matrix such that

(F.62)
$\displaystyle\hat{A}=\left(\sum_{t=1}^{T}y_{t}y_{t-1}^{\prime}\right)\left(\sum_{t=1}^{T}y_{t-1}y_{t-1}^{\prime}\right)^{-1}.$

The asymptotic behaviour of $\hat{A}$ is described by a corresponding
functional Brownian motion. To be more precise consider standardized
deviations of $\hat{A}$ about $I_{n}$ such that

(F.63) $\displaystyle
T\left(\hat{A}-I_{n}\right)=\left(\frac{1}{T}\sum_{t=1}^{T}u_{t}y_{t-1}^{\prime}\right)\left(\frac{1}{T^{2}}\sum_{t=1}^{T}y_{t-1}y_{t-1}^{\prime}\right)^{-1}.$

Therefore, to obtain the asymptotic behaviour of the above statistic we write
the sample second moment
$\left(\frac{1}{T^{2}}\sum_{t=1}^{T}y_{t-1}y_{t-1}^{\prime}\right)$ as a
quadratic functional of the random element $X_{T}(r)$, at least up to a term
of $o_{p}(1)$. That is,

(F.64)