# Transformed-Linear Innovations Algorithm for Modeling and Forecasting of
Time Series Extremes

Nehali Mhatre
Department of Statistics
Colorado State University
<EMAIL_ADDRESS>
&Daniel Cooley
Department of Statistics
Colorado State University
<EMAIL_ADDRESS>

###### Abstract

The innovations algorithm is a classical recursive forecasting algorithm used
in time series analysis. We develop the innovations algorithm for a class of
nonnegative regularly varying time series models constructed via transformed-
linear arithmetic. In addition to providing the best linear predictor, the
algorithm also enables us to estimate parameters of transformed-linear
regularly-varying moving average (MA) models, thus providing a tool for
modeling.

We first construct an inner product space of transformed-linear combinations
of nonnegative regularly-varying random variables and prove its link to a
Hilbert space which allows us to employ the projection theorem, from which we
develop the transformed-linear innovations algorithm. Turning our attention to
the class of transformed linear MA($\infty$) models, we give results on
parameter estimation and also show that this class of models is dense in the
class of possible tail pairwise dependence functions (TPDFs). We also develop
an extremes analogue of the classical Wold decomposition. Simulation study
shows that our class of models captures tail dependence for the GARCH(1,1)
model and a Markov time series model, both of which are outside our class of
models.

We also develop prediction intervals based on the geometry of regular
variation. Simulation study shows that we obtain good coverage rates for
prediction errors. We perform modeling and prediction for hourly windspeed
data by applying the innovations algorithm to the estimated TPDF.

_K_ eywords Transformed-linear regularly-varying time series models $\cdot$
Innovations algorithm $\cdot$ Stationary $\cdot$ Tail pairwise dependence
function $\cdot$ ARMA models.

##  1 Introduction

A primary aim of time series analysis is forecasting. Mhatre and Cooley
(2020+) developed transformed-linear time series models, a class of time
series which are nonnegative and regularly-varying and which are similar to
familiar ARMA models in the non-extreme setting. Mhatre and Cooley (2020+)
showed that these relatively simple models can capture dependence in a time
series’ upper tail. We now address the problem of forecasting specifically
when values are large.

Our approach for forecasting is to develop the innovations algorithm for
transformed-linear time series. The innovations algorithm, a well known
forecasting method in classical time series, relies on the autocovariance
function. In traditional time series analysis and elsewhere, the best linear
predictor $\hat{X}_{n+1}$ minimizes mean squared prediction error (MSPE),
$E[(X_{n+1}-\hat{X}_{n+1})^{2}]$ and Gaussian assumptions are usually used to
create prediction intervals. However, the autocovariance function is not well-
suited for describing tail dependence and expected-squared error is not a
natural or intuitive measure of loss for extremes. The innovations algorithm
we develop uses the tail pairwise dependence function (TPDF) to characterize
extremal dependence, and minimizes a quantity describing tail behavior.
Despite these differences, we show that the form of the transformed-linear
predictor is of the same form as in the non-extreme setting.

To develop the innovations algorithm, we construct a vector space $\mathbb{V}$
of a series of absolutely summable transformed-linear combinations of
nonnegative regularly-varying random variables. We show that $\mathbb{V}$ is
an inner product space and is isomorphic to $\ell^{1}$, the space of
absolutely summable sequences. Although $\mathbb{V}$ itself is not a Hilbert
space, we show that the set of predictors based on previous $n$ observations
is isomorphic to a closed linear subspace of $\ell^{2}$, the space of square
summable sequences, and we can employ the projection theorem. Using the
properties of the projection theorem we develop a transformed-linear analogue
of the classical innovations algorithm that allows us to do modeling and
prediction iteratively.

The innovations algorithm gives us more than just a method for prediction, as
we also use it as a tool to demonstrate properties of this modeling framework.
Using the innovations algorithm we show that if the true model is in our
transformed-linear space then applying the innovations algorithm iteratively
yields parameter estimates that converge to the true parameters. Furthermore
we show that even if the underlying model is not a transformed-linear model,
applying the innovations algorithm will yield a transformed-linear model whose
TPDF matches closely the estimated TPDF of the underlying model. We go on to
show that the class of transformed-linear regularly-varying MA($\infty$) time
series is dense in the class of possible TPDFs. We also develop a transformed-
linear analogue of the Wold decomposition. To demonstrate the richness of the
class of transformed-linear regularly-varying MA($\infty$) models we run the
innovations algorithm on data simulated from two different models from outside
our class of transform linear models: the GARCH(1,1) process and a first-order
Markov chain.Neither of these models is in the family of transformed-linear
time series. We show for both these models that by running the innovations
algorithm on the estimated TPDF we can get estimates for coefficients of a
transformed-linear regularly-varying MA time series whose TPDF closely matches
the estimated TPDF of the simulated data and well represents summary measures
of tail dependence.

We also develop a method based on the polar geometry of regular variation for
producing prediction intervals for the case when predictands are large.
Because the regular variation geometry differs from the elliptical geometry
typically assumed in standard linear prediction settings, uncertainty
quantification is significantly different from the non-extreme setting. We
perform modeling and prediction for the windspeed anomalies data discussed in
Mhatre and Cooley (2020+) by applying the innovations algorithm to the
estimated TPDF.

##  2 Background: Transformed-Linear Time Series Models for Extremes

Mhatre and Cooley (2020+) use transformed-linear arithmetic to produce
nonnegative regularly varying time series models for capturing extremal
dependence. Here we review the basics of these models. We consider time series
$\\{X_{t}\\},t\in\mathbb{(}Z)$ whose finite-dimensional distributions are
multivariate regularly varying. Let
${\bm{X}}_{t,p}=(X_{t},X_{t+1},\ldots,X_{t+p-1})^{T}$ for any $t$ and $p>0$.
Then, there exists a function $b(s)\rightarrow\infty$ as $s\rightarrow\infty$
and a non-trivial limit measure $\nu_{\bm{X}_{t,p}}$ such that

$\displaystyle s\text{
pr}\left\\{\frac{{\bm{X}_{t,p}}}{b(s)}\in\cdot\right\\}\xrightarrow{v}\nu_{\bm{X}_{t,p}}(\cdot)\mbox{
as }s\rightarrow\infty,$ (1)

where $\xrightarrow{v}$ denotes vague convergence in
$M_{+}(\mathbb{R}^{p}\setminus\\{0\\})$, the space of nonnegative Radon
measures on $\mathbb{R}^{p}\setminus\\{0\\}$ (Resnick, 2007, Section 6). The
normalizing function $b(s)=s^{1/\alpha}U(s)$ for $\alpha>0$ and some slowly
varying function $U$. The scaling property
$\nu_{\bm{X}_{t,p}}(aC)=a^{-\alpha}\nu_{\bm{X}_{t,p}}(C)$ for any $a>0$ and
any set $C\subset\mathbb{R}^{p}\setminus\\{0\\}$ implies that the measure is
more easily understood via polar coordinates. In particular, for a radially-
defined set
$C(r,B)=\\{{\bm{X}_{t,p}}\in\mathbb{R}^{p}:\|{\bm{X}_{t,p}}\|>r,\|{\bm{X}_{t,p}}\|^{-1}{\bm{X}_{t,p}}\in
B\\}$ where Borel set
$B\subset\mathbb{S}^{p-1}=\\{\bm{x}\in\mathbb{R}^{p}:\|\bm{x}\|=1\\}$,
$\nu_{\bm{X}_{t,p}}(C(r,B))=r^{-\alpha}H_{\bm{X}_{t,p}}(B)$ where
$H_{\bm{X}_{t,p}}$ is the angular measure taking values on $\mathbb{S}^{p-1}$.

Kulik and Soulier (2020) provide a comprehensive treatment of regularly-
varying time series, beginning with the finite-dimensional distribution
definition and developing the limit measure of the process. Fully
characterizing the limit measure is challenging, particularly because only a
subset of extreme data are used for estimation. Mhatre and Cooley (2020+)
instead develop the notion of tail stationarity, an extremal analogue to
second-order stationarity. Characterization of the tail of a time series is
simplified to characterizing the pairwise dependence via the TPDF, which takes
the place of the autocovariance function in standard time series analysis. The
TPDF is the functional extension of the tail pairwise dependence matrix
(TPDM), introduced in Cooley and Thibaud (2019). Kiriliouk and Zhou (2022)
define the TPDM for regularly-varying random vectors with tail index $\alpha$,
but we will follow both Cooley and Thibaud (2019) and Mhatre and Cooley
(2020+) and further assume $\\{X_{t}\\}$ has tail index $\alpha=2$, so that in
Section 5 we can connect the TPDF to the inner product we introduce in Section
3.1. In practice, this assumption generally requires a marginal
transformation. The TPDF is given by

$\displaystyle\sigma(X_{t},X_{t+h})=\int_{\Theta_{1}}w_{t}w_{t+h}\text{d}H_{X_{t},X_{t+h}}(\bm{w}),$
(2)

where $\Theta_{1}=\\{\bm{x}\in\mathbb{R}^{2}\mid\|\bm{x}\|_{2}=1\\}$. If the
time series is stationary the TPDF can be viewed as a function of the lag $h$.
Like the autocovariance function in standard time series analysis, the TPDF
evaluated at lag 0 provides a measure of the marginal ‘scale’ of the time
series, as $\lim\limits_{s\to\infty}s\text{
pr}\left\\{\frac{|X_{1}|}{b(s)}>c\right\\}=c^{-2}\sigma(0).$ As
$\sigma(h)/\sigma(0)\in[0,1]$, this ratio provides an interpretable number for
dependence strength at lag $h$.

Linear time series models of the form
$X_{t}=\sum_{j=-\infty}^{\infty}\psi_{j}Z_{t-j},$ where $Z_{t-j}$ is a white
noise sequence make up a large portion of classical time series analysis, and
include the familiar ARMA models. For time series with finite second moments,
$n$-step predictors (predictors of $X_{t+1}$ based on $X_{t},\ldots
X_{t-n+1})$ minimizing mean square prediction error (MSPE) can be constructed
via the projection theorem. It is straightforward to construct linear
regularly varying time series models by assuming the noise sequence is
regularly varying. As in Mhatre and Cooley (2020+), we are motivated to
construct nonnegative time series models, which enables one to focus attention
on the time series upper tail. If one wants to construct a nonnegative linear
time series using traditional arithmetic, one needs to restrict $\psi_{j}>0$
for all $j$ and employ a nonnegative noise sequence. Mhatre and Cooley (2020+)
show that transformed-linear time series can more flexibly fit data than
traditional linear time series models restricted to be nonnegative. Important
for this work, allowing the $\psi_{j}$’s to take negative values will allow us
to show the elements of transformed-linear $\\{X_{t}\\}$ can be thought of as
members of a vector space.

Mhatre and Cooley (2020+) employ the transformed-linear arithmetic of Cooley
and Thibaud (2019) to construct nonnegative time series models. Given
bijective transform $\tau:\mathbb{R}\mapsto\mathbb{R}_{+}$, define
$X_{1}\oplus X_{2}=\tau(\tau^{-1}(X_{1})+\tau^{-1}(X_{2}))$ and $a\circ
X_{1}=\tau(a\tau^{-1}(X_{1})$ for $a\in\mathbb{R}$. Cooley and Thibaud (2019)
show that if $X_{1},X_{2}$ are independent regularly varying with index
$\alpha$ with respective measures $\nu_{X_{1}},\nu_{X_{2}}$, if
$\lim_{y\rightarrow\infty}\tau(y)/y=1$, and if $X_{1},X_{2}$ meet a lower tail
condition associated with the particular $\tau$, then $\nu_{X_{1}\oplus
X_{2}}(\cdot)=\nu_{X_{1}}(\cdot)+\nu_{X_{2}}(\cdot)$ and $\nu_{a\circ
X_{1}}(\cdot)=(a^{(0)})^{\alpha}\nu_{X_{1}}(\cdot)$, where
$a^{(0)}=\max(a,0)$. If $\tau(y)=\log{\\{1+\exp{(y)}\\}}$, the lower tail
condition $s\text{ pr}\left[X_{i}\leq\exp{\\{-kb(s)\\}}\right]\to
0,\;k>0,\;i=1,2\;s\rightarrow\infty$ is sufficient.

Mhatre and Cooley (2020+) largely focus on causal transformed linear time
series models

$\displaystyle X_{t}=\bigoplus_{j=0}^{\infty}\psi_{j}\circ Z_{t-j},$ (3)

where $\sum^{\infty}_{j=-\infty}|\psi_{j}|<\infty$, and $\\{Z_{t}\\}$ is a
noise sequence of independent and tail stationary $RV_{+}(2)$ random
variables. The focus of Mhatre and Cooley (2020+) is to develop transformed
linear ARMA models and to develop their properties, particularly the TPDF, as
this is the critical parameter from the tail stationarity standpoint. The
model in (3) is known as the transformed-linear MA($\infty$) and has TPDF
$\sigma(h)=\sum^{\infty}_{j=0}\psi^{(0)}_{j}\psi^{(0)}_{j+h}$.

##  3 Space $\mathbb{V}$ and Innovations Algorithm

###  3.1 Inner Product Space $\mathbb{V}$

We begin by considering the space
$\mathbb{V}=\\{X_{t}:X_{t}=\bigoplus^{\infty}_{j=0}\psi_{t,j}\circ
Z_{j},\sum^{\infty}_{j=0}|\psi_{t,j}|<\infty$ where $Z_{j}$’s are independent
and tail stationary $RV_{+}(2)$ random variables with
$\lim_{x\to\infty}\text{Pr}(Z_{j}>x)/\\{x^{-2}L(x)\\}=1$ for some slowly-
varying function $L(x)$, $\psi_{t,j}\in\mathbb{R}$, and $t\in\mathbb{Z}$.
Mhatre and Cooley (2020+) show that $X_{t}$ converges with probability one.
Letting $X_{t}=\bigoplus^{\infty}_{j=0}\psi_{t,j}\circ
Z_{j},X_{s}=\bigoplus^{\infty}_{j=0}\psi_{s,j}\circ Z_{j}\in\mathbb{V}$, then
$X_{t}\oplus X_{s}=\bigoplus^{\infty}_{j=0}(\psi_{t,j}+\psi_{s,j})\circ Z_{j}$
and $a\circ X_{t}=\bigoplus^{\infty}_{j=0}(a^{(0)}\psi_{t,j})\circ Z_{j}$. The
space $\mathbb{V}$ equipped with transformed linear operations is a vector
space. The details of this and other properties of $\mathbb{V}$ described in
this section are provided in the supplementary materials.

Define the inner product between $X_{t}$ and $X_{s}$ as,

$\displaystyle\langle
X_{t},X_{s}\rangle:=\sum^{\infty}_{j=0}\psi_{t,j}\psi_{s,j}.$ (4)

We define the norm $\|X_{t}\|=\sqrt{\langle
X_{t},X_{t}\rangle}=\sqrt{\sum^{\infty}_{j=0}\psi_{t,j}^{2}}$ and say $X_{t}$
and $X_{s}$ are orthogonal if $\langle X_{t},X_{s}\rangle=0$. Note the space
$\mathbb{V}$ is more general than the stationary time series construction
given in (3). We will return to the stationary time series setting in Section
4, and we will relate the inner product to the TPDF in Section 5.

It will be useful to simultaneously consider the infinite-dimensional space of
absolutely summable sequences,

$\ell^{1}=\left\\{\\{a_{j}\\}^{\infty}_{j=0},a_{j}\in\mathbb{R}:\sum^{\infty}_{j=0}|a_{j}|<\infty\right\\},$

equipped with the standard vector addition and scalar multiplication. For any
$X_{t}\in\mathbb{V}$ we can define a mapping $T:\mathbb{V}\to\ell^{1}$ such
that $T(X_{t})=\\{\psi_{t,j}\\}_{j=0}^{\infty}\in\ell^{1}$. The mapping $T$ is
a linear map and an isomorphism.We know that vector space
$\ell^{1}\subset\ell^{2}$, where
$\ell^{2}=\\{\\{d_{j}\\}:\sum^{\infty}_{j=0}d_{j}^{2}<\infty\\}$, the space of
square-summable sequences. The inner product defined in (4) is isomorphic to
the usual inner product on $\ell^{2}$.

###  3.2 Best Transformed-Linear Prediction in $\mathbb{V}$

We want to use the projection theorem to perform prediction. Unfortunately,
$\mathbb{V}$ is not itself a Hilbert space as $\mathbb{V}$ is isomorphic to
$\ell^{1}$, and $\ell^{1}$ is not complete in the metric induced by the
$\ell^{2}$ inner product in (4). We consider the sequence
$\\{X_{t}\\},t\in\mathbb{Z}$ and consider transformed-linear prediction in
terms of the previous $n$ steps; that is we consider predictors

$\displaystyle\hat{X}_{n+1}(\bm{b}_{n})=\bigoplus^{n}_{j=1}b_{nj}\circ
X_{n+1-j},$ (5)

where $\bm{b}_{n}=(b_{n1},\cdots,b_{nn})^{T}\in\mathbb{R}^{n}.$ Let
$\mathbb{V}^{n}$ be the set of all such predictors. Let us consider the
analogous problem in $\ell^{1}$. Consider sequences
$\\{a_{j}\\}=T(X_{n+1-j}),j=1,\ldots,n$ in $\ell^{1}$. Let $\mathbb{C}^{n}$ be
the set of sequences
$c(\bm{b}_{n})=b_{n1}\\{a_{1}\\}+\cdots+b_{nn}\\{a_{n}\\}.$ $\mathbb{C}^{n}$
is the space spanned by $\\{a_{1}\\},\cdots,\\{a_{n}\\}$, and
dim($\mathbb{C}^{n})\leq n$. Since any $n$-dimensional subspace of a complex
topological vector space is closed (Rudin, 1991, Theorem 1.21),
$\mathbb{C}^{n}$ is a closed subspace of $\ell^{1}\subset\ell^{2}$. By the
projection theorem, there is a unique $\hat{c}\in\mathbb{C}^{n}$ such that
$\|x-\hat{c}\|=\inf_{c\in\mathbb{C}^{n}}\|x-c\|$, for every $x$ in $\ell^{2}$.
Thus, the set of predictors $\mathbb{V}^{n}$ based on previous $n$
observations is isomorphic to a closed linear subspace of $\ell^{2}$ and we
can employ the projection theorem since $\ell^{2}$ is known to be a Hilbert
space.

Armed now with the projection theorem, the best linear one-step predictor
$\hat{X}_{n+1}$ is given by

$\hat{X}_{n+1}=\left\\{\begin{array}[]{l l}0,&\text{ if }n=0,\\\
P_{\mathbb{V}^{n}}X_{n+1}&\text{ if }n\geq 1,\end{array}\right.$ (6)

where $P_{\mathbb{V}^{n}}$ denotes the projection mapping onto
$\mathbb{V}^{n}$. Thus, $\hat{X}_{n+1}$ is a transformed-linear combination of
$\\{X_{1},...,X_{n}\\}$ as given in (5). Define
$\hat{\bm{b}}_{n}=(\hat{b}_{n1},\ldots,\hat{b}_{nn})^{T}$ to be the solutions
to the prediction equations given by the projection theorem

$\displaystyle\left\langle X_{n+1}\ominus\bigoplus^{n}_{j=1}\hat{b}_{nj}\circ
X_{n+1-j},\;\;X_{n+1-k}\right\rangle$ $\displaystyle=0,\;\;k=1,\cdots,n.$ (7)

Equivalently,

$\displaystyle\left\langle\bigoplus^{n}_{j=1}\hat{b}_{nj}\circ
X_{n+1-j},\;\;X_{n+1-k}\right\rangle$ $\displaystyle=\langle
X_{n+1},\;X_{n+1-k}\rangle,\;\;k=1,\cdots,n.$ (8)

By linearity of the inner product, the prediction equations can be rewritten
in matrix form as

$\displaystyle\Gamma_{n}\hat{\bm{b}}_{n}$ $\displaystyle=\bm{\gamma}_{n}$ (9)

where $\Gamma_{n}=\left[\langle
X_{n+1-j},X_{n+1-k}\rangle\right]^{n}_{j,k=1}$, and
$\bm{\gamma}_{n}=\left[\langle X_{n+1},X_{n+1-k}\rangle\right]^{n}_{k=1}$. If
$\Gamma_{n}$ is non-singular, then the solution is given as

$\displaystyle\hat{\bm{b}}_{n}$
$\displaystyle=\Gamma_{n}^{-1}\bm{\gamma}_{n}.$ (10)

It can be shown that the above is equivalent to minimizing the squared norm
$\|X_{n+1}\ominus\hat{X}_{n+1}\|^{2}$ by setting the appropriate derivative to
zero. We see that (10) is of the familiar form for linear prediction in the
non-extreme setting where the inner product terms are autocovariances.

###  3.3 Transformed-Linear Innovations

Following Brockwell and Davis (1991), we develop a transformed-linear analogue
of the recursive innovations algorithm to obtain the one-step predictors
$\hat{X}_{n+1}$, $n\geq 1$, defined in (7), without having to perform matrix
inversion of $\Gamma_{n}$.

Consider the transformed-linear innovation, $(X_{n+1}\ominus\hat{X}_{n+1})$,
$n\geq 1$. Since $\mathbb{V}^{n}=\bar{\text{sp}}\\{X_{1},\cdots,X_{n}\\}$,
letting $\hat{X_{1}}:=\tau^{-1}(0)$,
$\mathbb{V}^{n}=\bar{\text{sp}}\\{X_{1}\ominus\hat{X_{1}},\cdots,X_{n}\ominus\hat{X}_{n}\\}$.
We can rewrite the predictor in (5) in terms of the innovations as,

$\displaystyle\hat{X}_{n+1}=\bigoplus_{j=1}^{n}\theta_{nj}\circ\left(X_{n+1-j}\ominus\hat{X}_{n+1-j}\right).$

By properties of projection mappings, $\hat{X}_{n+1}\in\mathbb{V}^{n}$ and by
(8),

$\displaystyle\langle X_{n+1}\ominus\hat{X}_{n+1},\hat{X}_{n+1}\rangle=0.$

That is, the transformed-linear innovation $(X_{n+1}\ominus\hat{X}_{n+1})$ is
orthogonal to a transformed-linear combination of $X_{1},...,X_{n}$. Thus, the
innovation is orthogonal to each of $X_{1},...,X_{n}$.

Consider the set of transformed-linear innovations,
$\\{X_{n+1-j}\ominus\hat{X}_{n+1-j}\\}_{j=1,...,n}$. The innovation
$(X_{i}\ominus\hat{X}_{i})\in\mathbb{V}^{j-1}$ for $i<j$, as
$(X_{i}\ominus\hat{X}_{i})$ is a transformed-linear combination of
$X_{1},...,X_{i}$. Also, by (8),
$(X_{j}\ominus\hat{X}_{j})\perp\mathbb{V}^{j-1}$. Thus, elements of the set
$\\{X_{1}\ominus\hat{X}_{1},X_{2}\ominus\hat{X}_{2},...,X_{n}\ominus\hat{X}_{n}\\}$
are mutually orthogonal. In fact,
$\\{X_{n+1-j}\ominus\hat{X}_{n+1-j}\\}_{j=1,...,n}$ is an orthogonal basis of
$\mathbb{V}^{n}$. Let the squared distance of prediction be denoted by
$\nu_{n}$, that is, $\nu_{n}=\|X_{n+1}\ominus\hat{X}_{n+1}\|^{2}$. The
innovations algorithm for a transformed-linear time series in $\mathbb{V}$ is
given as follows:

######  Proposition 1 (The Transformed-Linear Innovations Algorithm).

If $\\{X_{t}\\}$ is a transformed-linear time series in $\mathbb{V}$, where
the matrix $\Gamma_{n}=\left[\langle X_{i},X_{j}\rangle\right]^{n}_{i,j=1}$ is
non-singular for each $n\geq 1$, then the one-step predictors $\hat{X}_{n+1}$,
$n\geq 0$, and their squared distances of prediction $\nu_{n}$, $n\geq 1$, are
given by

$\displaystyle\hat{X}_{n+1}=\begin{cases}0&\text{if }n=0\\\
\bigoplus_{j=1}^{n}\theta_{nj}\circ(X_{n+1-j}\ominus\hat{X}_{n+1-j})&\text{if
}n\geq 1,\end{cases}$ (11)

and

$\displaystyle\begin{cases}\nu_{0}&=\langle X_{1},X_{1}\rangle\\\
\theta_{n,n-k}&=\nu_{k}^{-1}\Big{(}\langle
X_{n+1},X_{k+1}\rangle-\sum_{j=0}^{k-1}\theta_{k,k-j}\theta_{n,n-j}\nu_{j}\Big{)},\quad
k=0,1,...,n-1,\\\ \nu_{n}&=\langle
X_{n+1},X_{n+1}\rangle-\sum_{j=0}^{n-1}\theta_{n,n-j}^{2}\nu_{j},\end{cases}.$
(12)

Proof: Taking the inner product on both sides of (11) with
$(X_{k+1}\ominus\hat{X}_{k+1})$, $0\leq k<n$, we get

$\displaystyle\left\langle\hat{X}_{n+1},(X_{k+1}\ominus\hat{X}_{k+1})\right\rangle$
$\displaystyle=\left\langle\left\\{\bigoplus_{j=1}^{n}\theta_{nj}\circ(X_{n+1-j}\ominus\hat{X}_{n+1-j})\right\\},(X_{k+1}\ominus\hat{X}_{k+1})\right\rangle$
$\displaystyle=\sum_{j=1}^{n}\theta_{nj}\left\langle(X_{n+1-j}\ominus\hat{X}_{n+1-j}),(X_{k+1}\ominus\hat{X}_{k+1})\right\rangle$
$\displaystyle=\theta_{n,n-k}\nu_{k},$

since $(X_{n+1-j}\ominus\hat{X}_{n+1-j})\perp(X_{k+1}\ominus\hat{X}_{k+1})$
for all $j\neq n-k$.

Also, since $(X_{n+1}\ominus\hat{X}_{n+1})\perp(X_{k+1}\ominus\hat{X}_{k+1})$
for $k=0,\cdots,n-1$, we get,

$\langle\hat{X}_{n+1},(X_{k+1}\ominus\hat{X}_{k+1})\rangle=\langle
X_{n+1},(X_{k+1}\ominus\hat{X}_{k+1})\rangle.$

Hence, the coefficients $\theta_{n,n-k}$, $k=0,...,n-1$ are given by

$\displaystyle\theta_{n,n-k}=\nu_{k}^{-1}\langle
X_{n+1},(X_{k+1}\ominus\hat{X}_{k+1})\rangle.$ (13)

Using the representation in (11) with $n$ replaced by $k$, we get

$\displaystyle\theta_{n,n-k}=\nu_{k}^{-1}\Big{(}\langle
X_{n+1},X_{k+1}\rangle-\sum_{j=0}^{k-1}\theta_{k,k-j}\langle
X_{n+1},(X_{j+1}\ominus\hat{X}_{j+1})\rangle\Big{)}.$ (14)

Since by (13), $\langle
X_{n+1},(X_{j+1}\ominus\hat{X}_{j+1})\rangle=\nu_{j}\theta_{n,n-j}$, $0\leq
j<n$, we can rewrite (14) as

$\displaystyle\theta_{n,n-k}=\nu_{k}^{-1}\Big{(}\langle
X_{n+1},X_{k+1}\rangle-\sum_{j=0}^{k-1}\theta_{k,k-j}\theta_{n,n-j}\nu_{j}\Big{)}.$

By properties of projection mapping, we have

$\displaystyle\nu_{n}=\|X_{n+1}\ominus\hat{X}_{n+1}\|^{2}=\|X_{n+1}\|^{2}-\|\hat{X}_{n+1}\|^{2}=\langle
X_{n+1},X_{n+1}\rangle-\sum_{k=0}^{n-1}\theta_{n,n-k}^{2}\nu_{k}.$

$\square$

We will return to forecasting in Section 7 where we will propose a method for
quantifying prediction uncertainty. For now, we will turn our attention to
using the innovations algorithm as a tool to better understand the richness of
our class of models and also as a tool for model fitting.

##  4 Implications for Modeling of Stationary Time Series

Rather than the general setting described by the class $\mathbb{V}$, we now
focus on stationary time series. If $\\{X_{t}\\}$ is an MA($\infty$) time
series (3), $X_{t}\in\mathbb{V}$ for all $t$. As $\\{X_{t}\\}$ is stationary,
it is natural to think of the inner product as a function of lag:

$\gamma(h)=\langle
X_{t},X_{t+h}\rangle=\sum^{\infty}_{j=0}\psi_{j}\psi_{j+h}.$

Being an inner product, $\gamma(.)$ is nonnegative definite and by the Cauchy-
Schwarz inequality, $|\gamma(h)|\leq\gamma(0)$.

The following corollary shows that given an invertible transformed-linear
regularly-varying MA time series, iterating the transformed-linear innovations
algorithm yields coefficient estimates that converge to the true MA
coefficients.

######  Corollary 1.

If $\\{X_{t}\\}$ is an invertible MA process in $\mathbb{V}$, that is,

$Z_{t}=X_{t}\oplus\bigoplus^{\infty}_{j=1}\pi_{j}\circ X_{t-j},$

with $\lim_{x\to\infty}\text{Pr}(Z_{j}>x)/\\{x^{-2}L(x)\\}=1$, then as
$n\to\infty$,
(i) $\nu_{n}\to 1$,
(ii) $\|(X_{n}\ominus\hat{X}_{n})\ominus Z_{n}\|^{2}\to 0$, and
(iii) $\theta_{nj}\to\psi_{j},j=1,2,\cdots$.

Proof:
Let $\mathbb{M}_{n}=\bar{\text{sp}}\\{X_{s},-\infty<s\leq n\\}$ and
$\mathbb{V}^{n}=\bar{\text{sp}}\\{X_{1},\cdots,X_{n}\\}$. Because
$\\{X_{t}\\}$ is invertible,

$Z_{n+1}\ominus X_{n+1}=\bigoplus^{\infty}_{j=1}\pi_{j}\circ
X_{n+1-j}=P_{\mathbb{M}_{n}}(Z_{n+1}\ominus X_{n+1})=\ominus
P_{\mathbb{M}_{n}}X_{n+1},$

since $Z_{n+1}\perp\mathbb{M}_{n}.$ Also, we can think of $Z_{k}$ as
$Z_{k}=\bigoplus^{\infty}_{j=0}\psi_{j}\circ Z_{j}$, where $\psi_{j}=1$ for
$j=k$ and $\psi_{j}=0$ for all $j\neq k$. Thus, $Z_{k}\in\mathbb{V}$ and
subsequently, $\|Z_{k}\|^{2}=1$ for all $k$. Then,

$\displaystyle 1=\|Z_{n+1}\|^{2}$ $\displaystyle=$
$\displaystyle\|X_{n+1}\oplus\bigoplus^{\infty}_{j=1}\pi_{j}\circ
X_{n+1-j}\|^{2}=\|X_{n+1}\ominus P_{\mathbb{M}_{n}}X_{n+1}\|^{2}$
$\displaystyle\leq$ $\displaystyle\|X_{n+1}\ominus
P_{\mathbb{V}^{n}}X_{n+1}\|^{2}=\nu_{n}$ $\displaystyle\leq$
$\displaystyle\|X_{n+1}\oplus\bigoplus^{n}_{j=1}\pi_{j}\circ
X_{n+1-j}\|^{2}=\|Z_{n+1}\ominus\bigoplus^{\infty}_{j=n+1}\pi_{j}\circ
X_{n+1-j}\|^{2}$ $\displaystyle=$
$\displaystyle\|Z_{n+1}\|^{2}+\|\bigoplus^{\infty}_{j=n+1}\pi_{j}\circ
X_{n+1-j}\|^{2}=1+\sum^{\infty}_{i,j=n}\pi_{i}\pi_{j}\langle
X_{n+1-i},X_{n+1-j}\rangle$ $\displaystyle\leq$ $\displaystyle
1+\left(\sum^{\infty}_{j=n+1}\pi_{j}\right)^{2}\gamma(0).$

Thus (i) is established since,

$1\leq\nu_{n}\leq
1+\left(\sum^{\infty}_{j=n+1}\pi_{j}\right)^{2}\gamma(0)\implies\nu_{n}\to
1\text{ as }n\to\infty.$

Consider,

$\displaystyle\|X_{n}\ominus\hat{X}_{n}\ominus Z_{n}\|^{2}$ $\displaystyle=$
$\displaystyle\|X_{n}\ominus\hat{X}_{n}\|^{2}-2\langle
Z_{n},X_{n}\ominus\hat{X}_{n}\rangle+\|Z_{n}\|^{2}$ (15) $\displaystyle=$
$\displaystyle v_{n-1}-2\left[\langle Z_{n},X_{n}\rangle-\langle
Z_{n},\hat{X}_{n}\rangle\right]+1$ $\displaystyle=$ $\displaystyle
v_{n-1}-2\left[\langle Z_{n},\bigoplus^{\infty}_{j=0}\psi_{j}\circ
Z_{n-j}\rangle-\langle Z_{n},\bigoplus^{n-1}_{j=1}b_{nj}\circ
X_{n-j}\rangle\right]+1$ $\displaystyle=$ $\displaystyle
v_{n-1}+2[\|Z_{n}\|^{2}-0]+1$ $\displaystyle=$ $\displaystyle v_{n-1}-1,$

where (15) converges to $0\text{ as }n\to\infty$ by (i), thus proving (ii).

Since $X_{n+1}=\bigoplus^{\infty}_{j=0}\psi_{j}\circ Z_{n+1-j}$, we have that

$\psi_{j}=\langle X_{n+1},Z_{n+1-j}\rangle.$

Also, by (13),

$\theta_{nj}=\nu_{n-j}^{-1}\langle
X_{n+1},(X_{n+1-j}\ominus\hat{X}_{n+1-j})\rangle.$

Then,

$\displaystyle|\theta_{nj}-\psi_{j}|$ $\displaystyle=$
$\displaystyle\left|\theta_{nj}-\langle
X_{n+1},(X_{n+1-j}\ominus\hat{X}_{n+1-j})\rangle+\langle
X_{n+1},(X_{n+1-j}\ominus\hat{X}_{n+1-j})\rangle-\psi_{j}\right|$
$\displaystyle\leq$ $\displaystyle\left|\theta_{nj}-\langle
X_{n+1},(X_{n+1-j}\ominus\hat{X}_{n+1-j})\rangle\right|+\left|\langle
X_{n+1},(X_{n+1-j}\ominus\hat{X}_{n+1-j})\rangle-\langle
X_{n+1},Z_{n+1-j}\rangle\right|$ $\displaystyle=$
$\displaystyle\left|\theta_{nj}-\theta_{nj}v_{n-j}\right|+\left|\langle
X_{n+1},(X_{n+1-j}\ominus\hat{X}_{n+1-j}\ominus Z_{n+1-j})\rangle\right|$
$\displaystyle\leq$
$\displaystyle\left|\theta_{nj}-\theta_{nj}v_{n-j}\right|+\sqrt{\gamma(0)}\left\|(X_{n+1-j}\ominus\hat{X}_{n+1-j}\ominus
Z_{n+1-j})\right\|,$ (17)

where the inequalities in (4) and (17) hold by the triangle inequality and the
Cauchy-Schwarz inequality, respectively. Since $\theta_{nj}$ and $\gamma(0)$
are bounded, as $n\to\infty$, the first term on the right-hand side of (17)
converges to $0$ by (i) and the second term on the right-hand side of (17)
converges to $0$ by (ii). Thus, $\theta_{nj}\to\psi_{j}$ as $n\to\infty$,
proving (iii). $\square$

Analogous to Proposition 3.2.1 in Brockwell and Davis (1991), Proposition 2
shows that a $q$-tail-dependent tail stationary regularly-varying time series
can be represented as a transformed-linear regularly-varying MA($q$) process.
This proposition is a beginning step in showing the richness of the class of
transformed-linear time series, which we build on in Section 6.

######  Proposition 2.

If $\\{X_{t}\\}$ is a regularly-varying tail stationary process in
$\mathbb{V}$ with inner product function $\gamma(.)$ such that $\gamma(h)=0$
for $|h|>q$ and $\gamma(q)\neq 0$, then $\\{X_{t}\\}$ is a transformed-linear
regularly-varying MA($q$) process, i.e. there exists a regularly-varying noise
sequence $\\{Z_{t}\\}$ of independent and tail stationary $Z_{t}$’s such that

$X_{t}=Z_{t}\oplus\theta_{1}\circ Z_{t-1}\oplus\cdots\oplus\theta_{q}\circ
Z_{t-q}.$

Before we prove Proposition 2, we prove the following Lemma which requires the
notion of convergence in tail ratio. Let $X_{t}$ be the MA($\infty$) in (3)
and for some $q>0$ define $X_{t}^{(q)}=\bigoplus_{j=0}^{q}\psi_{j}\circ
Z_{t-j}$ and $X_{t}^{(q)^{\prime}}=\bigoplus_{j=q+1}^{\infty}\psi_{j}\circ
Z_{t-j}$. Mhatre and Cooley (2020+, Section 3.4) say that $X_{t}^{(q)}$
converges to $X_{t}$ in tail ratio if
$\lim\limits_{x\to\infty}\frac{\text{pr}(X_{t}^{(q)^{\prime}}>x)}{x^{-2}L(x)}=0$
as $q\rightarrow\infty$. The proofs below are extreme analogues to the steps
in Brockwell and Davis (1991, Proposition 3.2.1), with tail ratio convergence
replacing mean-square convergence.

######  Lemma 1.

If $X_{t}$ is a tail stationary process in $\mathbb{V}$, then

$P_{\bar{\text{sp}}\\{X_{j},t-n\leq j\leq
t-1\\}}X_{t}\stackrel{{\scriptstyle\text{tail
ratio}}}{{\rightarrow}}P_{\bar{\text{sp}}\\{X_{j},-\infty<j\leq
t-1\\}}X_{t},\;\;\text{as }n\to\infty.$

Proof: Consider the transformed-linear combination

$\bigoplus^{\infty}_{j=n+1}b_{j}\circ X_{t-j}.$

As $X_{t}$ is a tail stationary process,

$\text{pr}(X_{t}>x)\sim x^{-2}L(x)\sigma(0),$

where $\sigma(0)=\sigma(X_{t},X_{t})$. As shown in Mhatre and Cooley (2020+,
Section 3.4),

$\displaystyle\text{pr}\left(\bigoplus^{\infty}_{j=n+1}b_{j}\circ
X_{t-j}>x\right)$ $\displaystyle\sim
x^{-2}L(x)\sigma(0)\sum^{\infty}_{j=n+1}|b_{j}|^{2},\;\;\text{as }x\to\infty.$
(18)

Taking limit on both sides of (18) we get, as $x\to\infty$,

$\displaystyle\lim\limits_{n\to\infty}\text{pr}\left(\bigoplus^{\infty}_{j=n+1}b_{j}\circ
X_{t-j}>x\right)$ $\displaystyle\sim$
$\displaystyle\lim\limits_{n\to\infty}x^{-2}L(x)\sigma(0)\sum^{\infty}_{j=n+1}|b_{j}|^{2}=0,$
$\displaystyle\implies\lim\limits_{x\to\infty}\frac{\text{pr}\left(\bigoplus^{\infty}_{j=n+1}b_{j}\circ
X_{t-j}>x\right)}{x^{-2}L(x)}$ $\displaystyle\to$ $\displaystyle
0,\;\;\text{as }n\to\infty.$

Thus

$\displaystyle\bigoplus^{n}_{j=1}b_{j}\circ X_{t-j}$
$\displaystyle\stackrel{{\scriptstyle\text{tail ratio}}}{{\rightarrow}}$
$\displaystyle\bigoplus^{\infty}_{j=1}b_{j}\circ X_{t-j},\mbox{ as
}n\rightarrow\infty,\mbox{ and }$ $\displaystyle
P_{\bar{\text{sp}}\\{X_{j},t-n\leq j\leq t-1\\}}X_{t}$
$\displaystyle\stackrel{{\scriptstyle\text{tail ratio}}}{{\rightarrow}}$
$\displaystyle P_{\bar{\text{sp}}\\{X_{j},-\infty<j\leq
t-1\\}}X_{t},\;\;\text{as }n\to\infty.$

$\square$

Proof of Proposition 2: For each $t$, let $\mathbb{M}_{t}$ be the closed
transformed-linear subspace $\bar{\text{sp}}\\{X_{s},s\leq t\\}$ of
$\mathbb{V}$ and set

$\displaystyle Z_{t}$ $\displaystyle=X_{t}\ominus P_{\mathbb{M}_{t-1}}X_{t}.$
(19)

That is,

$Z_{t}=X_{t}\ominus\bigoplus^{\infty}_{j=1}b_{j}\circ
X_{t-j},\;\;a_{j}\in\mathbb{R}.$

Thus, $Z_{t}\in\mathbb{M}_{t}$. By definition of $P_{\mathbb{M}_{t-1}}$,
$P_{\mathbb{M}_{t-1}}X_{t}\in\mathbb{M}_{t-1}$ and $Z_{t}=X_{t}\ominus
P_{\mathbb{M}_{t-1}}X_{t}\in\mathbb{M}_{t-1}^{\perp}$. Thus
$Z_{s}\in\mathbb{M}_{s}\subset\mathbb{M}_{t-1}$ and hence $\langle
Z_{s},Z_{t}\rangle=0$ for $s<t$. Also, by Lemma 1

$P_{\bar{\text{sp}}\\{X_{s},t-n\leq s\leq
t-1\\}}X_{t}\stackrel{{\scriptstyle\text{tail
ratio}}}{{\rightarrow}}P_{\mathbb{M}_{t-1}}X_{t},\;\;\text{as }n\to\infty.$

By stationarity and continuity of norm,

$\displaystyle\|Z_{t+1}\|$ $\displaystyle=$ $\displaystyle\|X_{t+1}\ominus
P_{\mathbb{M}_{t}}X_{t+1}\|$ $\displaystyle=$
$\displaystyle\lim\limits_{n\to\infty}\|X_{t+1}\ominus
P_{\bar{\text{sp}}\\{X_{s},s=t+1-n,\cdots,t\\}}X_{t+1}\|$ $\displaystyle=$
$\displaystyle\lim\limits_{n\to\infty}\|X_{t}\ominus
P_{\bar{\text{sp}}\\{X_{s},s=t-n,\cdots,t-1\\}}X_{t}\|$ $\displaystyle=$
$\displaystyle\|X_{t}\ominus P_{\mathbb{M}_{t-1}}X_{t}\|=\|Z_{t}\|.$

Letting $c^{2}=\|Z_{t}\|^{2}$, $\\{Z_{t}\\}$ is a sequence of independent and
tail stationary regularly-varying random variables with scale $c$, that is,
$\text{Pr}(Z_{t}>x)/\\{x^{-2}L(x)\\}=c^{2}$.

By (19),

$X_{t-1}=Z_{t-1}\oplus P_{\mathbb{M}_{t-2}}X_{t-1}.$

Consequently,

$\displaystyle\mathbb{M}_{t-1}$ $\displaystyle=$
$\displaystyle\bar{\text{sp}}\\{X_{s},s\leq t-1\\}$ $\displaystyle=$
$\displaystyle\bar{\text{sp}}\\{X_{s},s<t-1,Z_{t-1}\\}$ $\displaystyle=$
$\displaystyle\bar{\text{sp}}\\{X_{s},s<t-q,Z_{t-q},\cdots,Z_{t-1}\\}.$

Therefore, $\mathbb{M}_{t-1}$ can be decomposed into two orthogonal subspaces,
$\mathbb{M}_{t-q-1}$ and $\bar{\text{sp}}\\{Z_{t-q},\cdots,Z_{t-1}\\}$. Since
$\gamma(h)=0$ for $|h|>q$, it follows that $X_{t}\perp\mathbb{M}_{t-q-1}$ and
since $\bar{\text{sp}}\\{c^{-2}\circ Z_{t-q},\cdots,c^{-2}\circ Z_{t-1}\\}$ is
an orthonormal set, by properties of projection mappings,

$\displaystyle P_{\mathbb{M}_{t-1}}X_{t}$ $\displaystyle=$ $\displaystyle
P_{\mathbb{M}_{t-q-1}}X_{t}\oplus P_{\bar{\text{sp}}\\{Z_{t-q},\cdots
Z_{t-1}\\}}X_{t}$ (20) $\displaystyle=$ $\displaystyle
0\;\oplus\;\left(c^{-2}\langle X_{t},Z_{t-1}\rangle\right)\circ
Z_{t-1}\;\oplus\cdots\oplus\;\left(c^{-2}\langle
X_{t},Z_{t-q}\rangle\right)\circ Z_{t-q}$ $\displaystyle=$
$\displaystyle\theta_{1}\circ Z_{t-1}\oplus\cdots\oplus\theta_{q}\circ
Z_{t-q},$

where $\theta_{j}:=c^{-2}\langle X_{t},Z_{t-j}\rangle$, which by stationarity
is independent of $t$ for $j=1,\cdots,q$. By (19) and (20),

$X_{t}=Z_{t}\oplus\theta_{1}\circ Z_{t-1}\oplus\cdots\oplus\theta_{q}\circ
Z_{t-q}.$

$\square$

##  5 Modeling in Subset $\mathbb{V}_{+}$

We have defined $\mathbb{V}$ allowing for negative $\psi_{j}$’s as they are
needed to have a vector space and to create orthogonal elements. In this
section we will show that the negative coefficients defining $\\{X_{t}\\}$ are
academic in the sense that a time series that has negative coefficients is
indistinguishable in terms of tail behavior from a time series that has zeroes
in place of those coefficients. Rather than being a problem, this will allow
us to restrict our attention to a subset $\mathbb{V}_{+}$ for which the TPDF
is equivalent to the inner product function.

Consider a subset of $\mathbb{V}$ defined as

$\mathbb{V}_{+}=\\{X_{t}:X_{t}=\bigoplus^{\infty}_{j=0}\psi_{j}\circ
Z_{t-j},\;\psi_{j}\geq 0,\sum^{\infty}_{j=0}\psi_{j}<\infty\\}.$

Proposition 3 below follows from the definition of TPDF.

######  Proposition 3.

If a transformed-linear MA($\infty$) time series in $\mathbb{V}$ has TPDF
$\sigma(h)$, then there exists a transformed-linear MA($\infty$) time series
in subset $\mathbb{V}_{+}$ which has the same TPDF $\sigma(h)$, for all lag
$h$.

Proof: Let $X_{t}=\bigoplus^{\infty}_{j=0}\psi_{j}\circ Z_{t-j}\in\mathbb{V}$.
The TPDF of $X_{t}$ is given by
$\sigma(h)=\sum^{\infty}_{j=0}\psi^{(0)}_{j}\psi^{(0)}_{j+h}$, which is equal
to the TPDF of $X_{t}^{*}=\bigoplus^{\infty}_{j=0}\psi_{j}^{(0)}\circ
Z_{t-j}\in\mathbb{V}_{+}.$ $\square$

In other words, $X_{t}$ and $X_{t}^{*}$ are indistinguishable in terms of tail
dependence. $X_{t}$ and $X_{t}^{*}$ are also indistinguishable in terms of the
tail ratio since recall that tail ratio is equal to $\sigma(0)$. Furthermore,
the TPDF gives full information for a stationary time series in
$\mathbb{V}_{+}$ and unlike the inner product function, the TPDF is estimable.
Also, it can be clearly seen that $\gamma(h)=\sigma(h)$, for
$X_{t}^{*},X^{*}_{t+h}\in\mathbb{V}_{+}$, for all lag $h$. Hence it is
reasonable to restrict our attention to $\mathbb{V}_{+}$.

As the inner product is equivalent to the TPDF in $\mathbb{V}_{+}$, equation
10 can be rewritten as

$\displaystyle\hat{\bm{b}}_{n}=\Sigma_{n}^{-1}\bm{\sigma}_{n}.$ (21)

where $\Sigma_{n}=\left[\sigma(i-j)\right]^{n}_{i,j=1}$ and
$\bm{\sigma}_{n}=\left[\sigma(i)\right]^{n}_{i=1}$.

Also, if our time series is in $\mathbb{V}_{+}$, we can rewrite the equations
(12) of the innovations algorithm in terms of the TPDF $\sigma(\cdot)$ instead
of the inner product as

$\displaystyle\begin{cases}\nu_{0}&=\sigma(0)\\\
\theta_{n,n-k}&=\nu_{k}^{-1}\left(\sigma(n-k)-\sum_{j=0}^{k-1}\theta_{k,k-j}\theta_{n,n-j}\nu_{j}\right),\quad
k=0,1,...,n-1,\\\
\nu_{n}&=\sigma(0)-\sum_{j=0}^{n-1}\theta_{n,n-j}^{2}\nu_{j},\end{cases}.$
(22)

Rewriting Corollary 1 for $\mathbb{V}_{+}$, we get the following corollary.

######  Corollary 2.

If $\\{X_{t}\\}$ is an invertible MA process in $\mathbb{V}_{+}$ with
$\lim_{x\to\infty}\text{Pr}(Z_{j}>x)/\\{x^{-2}L(x)\\}=1$, then as
$n\to\infty$,
(i) $\nu_{n}\to 1$,
(ii) $\|X_{n}\ominus\hat{X}_{n}\ominus Z_{n}\|^{2}\to 0$, and
(iii) $\theta_{nj}\to\psi_{j},j=1,2,\cdots$; $\psi_{j}\geq 0$.

Also, rewriting Proposition 2 for $\mathbb{V}_{+}$,

######  Corollary 3.

If $\\{X_{t}\\}$ is a regularly-varying tail stationary process in
$\mathbb{V}_{+}$ with tail pairwise dependence function $\sigma(.)$ such that
$\sigma(h)=0$ for $|h|>q$ and $\sigma(q)\neq 0$, then $\\{X_{t}\\}$ is an
transformed-linear regularly-varying MA($q$) process, i.e. there exists a
regularly-varying noise sequence $\\{Z_{t}\\}$ of independent and tail
stationary $Z_{t}$’s such that

$X_{t}=Z_{t}\oplus\theta_{1}\circ Z_{t-1}\oplus\cdots\oplus\theta_{q}\circ
Z_{t-q}.$

The relation between the TPDF and the inner product gives an important result
described in the following remark:

######  Remark 1.

If $X_{t}$ is an MA($\infty$) time series in $\mathbb{V}$ and
$X_{t}\notin\mathbb{V}_{+}$, then by Proposition 3 there exists
$X_{t}^{*}\in\mathbb{V}_{+}$, obtained by applying the zero-operator on the
coefficients of $X_{t}$, which has the same TPDF as $X_{t}$. Thus, the
innovations algorithm applied to the TPDF of $X_{t}\in\mathbb{V}$ will give us
the one-step predictors for the corresponding $X_{t}^{*}\in\mathbb{V}_{+}$.

##  6 Flexibility of the MA($\infty$) Class for Modeling

In this section, we show that the class of MA($\infty$) models is a rich class
for modeling.

###  6.1 Richness of the MA($\infty$) Class in terms of the TPDF

Given a valid TPDF (that is, a completely positive function) that converges to
$0$ as lag increases, we can run the innovations algorithm to get the
$\theta_{nj}$’s and $\nu_{n}$ as defined in (12). If we apply the TPDF formula
to these $\theta_{nj}$’s we will get a TPDF that gets arbitrarily close to the
given TPDF. In other words, if we consider random noise terms $\\{Z_{j}\\}$
that are Frèchet with $\alpha=2$ and scale $\sqrt{\nu_{n}}$, and generate a
process applying the coefficients $\theta_{nj}$ to the $Z$’s, the TPDF of this
generated process will be arbitrarily close to the given TPDF. Thus, given any
valid TPDF, we can run the transformed-linear innovations algorithm long
enough to find a transformed-linear regularly-varying MA($\infty$) time series
whose TPDF will get arbitrarily close to the given TPDF. As such, the class of
MA($\infty$) time series is rich in the class of possible TPDFs that converge
to $0$.

To show this, first, we prove the result for a $q$-tail-dependent TPDF in the
following corollary.

######  Corollary 4.

If $\\{X_{t}\\}$ is any regularly-varying tail stationary process with TPDF
$\sigma(.)$ such that $\sigma(h)=0$ for $|h|>q$ and $\sigma(q)\neq 0$, then as
$n\to\infty$, the $\theta_{nj}$s generated from the transformed-linear
innovations algorithm approach $\theta_{1},\cdots,\theta_{q}$ of an MA($q$)
whose TPDF matches the given TPDF.

Proof: Let us consider the form for the $\theta_{nj}$s given by the
transformed-linear innovations algorithm in 22:

$\displaystyle\theta_{n,n-k}=\nu_{k}^{-1}\left(\sigma(n-k)-\sum_{l=0}^{k-1}\theta_{k,k-l}\theta_{n,n-l}\nu_{l}\right),\quad
k=0,1,...,n-1.$ (23)

Rewriting (23) by letting $h=n-k$,

$\displaystyle\theta_{n,h}$ $\displaystyle=$
$\displaystyle\nu_{n-h}^{-1}\left(\sigma(h)-\sum_{l=0}^{n-h-1}\theta_{n-h,n-h-l}\theta_{n,n-l}\nu_{l}\right),$
(24) $\displaystyle=$
$\displaystyle\nu_{n-h}^{-1}\left(\sigma(h)-\sum_{l=n-h-q}^{n-h-1}\theta_{n-h,n-h-l}\theta_{n,n-l}\nu_{l}\right),\quad
h=0,1,...,q,$

since $\theta_{n,n-l}=0$ for all $l=0,1,\cdots,n-h-q-1$.

Rewriting (24) by letting $j=n-h-l$,

$\displaystyle\theta_{n,h}=\nu_{n-h}^{-1}\left(\sigma(h)-\sum_{j=1}^{q}\theta_{n-h,j}\theta_{n,j+h}\nu_{n-h-j}\right).$
(25)

As $n\to\infty$, let $\theta_{n,h}\to\theta_{h}$ and $\nu_{n}\to c^{2}$. Then
(25) becomes,

$\displaystyle\theta_{h}$
$\displaystyle=c^{-2}\left(\sigma(h)-\sum_{j=1}^{q}\theta_{j}\theta_{j+h}c^{2}\right).$
(26)

Rearranging (26),

$\displaystyle\sigma(h)$ $\displaystyle=$
$\displaystyle\theta_{h}c^{2}+\sum_{j=1}^{q}\theta_{j}\theta_{j+h}c^{2}$ (27)
$\displaystyle=$ $\displaystyle
c^{2}\sum_{j=0}^{q}\theta_{j}\theta_{j+h},\quad h=0,1,...,q,$

which is the form for the TPDF at lag $h$ for a regularly-varying tail
stationary MA($q$) with $\theta_{j}\geq 0$ for $j=0,1,\cdots,q$ and
$\lim_{x\to\infty}\text{pr}(Z_{j}>x)/\\{x^{-2}L(x)\\}=c^{2}$. Thus, the TPDF
of this MA($q$) matches the given TPDF $\sigma(h)$. $\square$

We are extending the above result to the MA($\infty$) case.

###  6.2 Transformed-Linear Wold Decomposition

If the TPDF of a time series does not converge to $0$, analogous to the Wold
decomposition discussed in Brockwell and Davis (1991) we can decompose the
time series into an MA($\infty$) process and a deterministic process.
Following Brockwell and Davis (1991) and Sargent (1979) we prove our
Transformed-Linear Wold Decomposition as follows:

######  Theorem 1 (The Transformed-Linear Wold Decomposition).

If $c^{2}=\|X_{n+1}\ominus\hat{X}_{n+1}\|^{2}>0$, then $X_{t}$ can be
expressed as

$\displaystyle X_{t}=\bigoplus_{j=0}^{\infty}\psi_{j}\circ
Z_{t-j}\;\oplus\;U_{t},$ (28)

where

$\displaystyle\text{(i) }\psi_{0}=1\text{ and
}\sum_{j=0}^{\infty}\psi_{j}^{2}<\infty,$ $\displaystyle\text{(ii)
}\\{Z_{t}\\}\text{ is a sequence of independent and tail stationary regularly-
varying random variables with scale }c,$ $\displaystyle\text{(iii)
}Z_{t}\in\mathbb{V}^{t}\text{ for each }t\in\mathbb{Z},$
$\displaystyle\text{(iv) }\langle Z_{t},U_{s}\rangle=0\text{ for all
}s,t\in\mathbb{Z},$ $\displaystyle\text{(v) }\\{U_{t}\\}\text{ is
deterministic.}$

The sequences $\\{\psi_{j}\\}$, $\\{Z_{j}\\}$, and $\\{U_{j}\\}$ are uniquely
determined by (28) and the conditions (i) - (v).

Proof: Consider the sequences

$\displaystyle Z_{t}=X_{t}\ominus P_{\mathbb{V}^{t-1}}X_{t},$ (29)
$\displaystyle\psi_{j}=c^{-2}\langle X_{t},Z_{t-j}\rangle,$ (30)
$\displaystyle U_{t}=X_{t}\ominus\bigoplus_{j=0}^{\infty}\psi_{j}\circ
Z_{t-j}.$ (31)

That is,

$Z_{t}=X_{t}\ominus\bigoplus^{\infty}_{j=1}a_{j}\circ
X_{t-j},\;\;a_{j}\in\mathbb{R},\;j=1,\cdots,t-1.$

Thus, $Z_{t}\in\mathbb{V}^{t}$, establishing (iii). By definition of
$P_{\mathbb{V}^{t-1}}$, $P_{\mathbb{V}^{t-1}}X_{t}\in\mathbb{V}^{t-1}$ and
$Z_{t}=X_{t}\ominus P_{\mathbb{V}^{t-1}}X_{t}\in\mathbb{V}^{{t-1}{\perp}}$.
Thus,

$Z_{t}\in\mathbb{V}^{{t-1}{\perp}}\subset\mathbb{V}^{{t-2}{\perp}}\subset\cdots$

Hence for $s<t$, $\langle Z_{s},Z_{t}\rangle=0$. By Lemma 1

$P_{\bar{\text{sp}}\\{X_{s},t-n\leq s\leq
t-1\\}}X_{t}\stackrel{{\scriptstyle\text{tail
ratio}}}{{\rightarrow}}P_{\mathbb{V}^{t-1}}X_{t},\;\;\text{as }n\to\infty.$

By stationarity and continuity of norm,

$\displaystyle\|Z_{t+1}\|$ $\displaystyle=$ $\displaystyle\|X_{t+1}\ominus
P_{\mathbb{V}^{t}}X_{t+1}\|$ $\displaystyle=$
$\displaystyle\lim\limits_{n\to\infty}\|X_{t+1}\ominus
P_{\bar{\text{sp}}\\{X_{s},s=t+1-n,\cdots,t\\}}X_{t+1}\|$ $\displaystyle=$
$\displaystyle\lim\limits_{n\to\infty}\|X_{t}\ominus
P_{\bar{\text{sp}}\\{X_{s},s=t-n,\cdots,t-1\\}}X_{t}\|$ $\displaystyle=$
$\displaystyle\|X_{t}\ominus P_{\mathbb{V}^{t-1}}X_{t}\|=\|Z_{t}\|.$

Letting $c^{2}=\|Z_{t}\|^{2}$, $\\{Z_{t}\\}$ is a sequence of independent and
tail stationary regularly-varying random variables with scale $c$, thus
establishing (ii).

By equation (20) in the proof of Proposition 2,

$P_{\bar{\text{sp}}\\{Z_{j},j\leq t\\}}X_{t}=\sum_{j=0}^{\infty}\psi_{j}\circ
Z_{t-j},$

where $\psi_{j}=c^{-2}\langle X_{t},Z_{t-j}\rangle$ and
$\sum_{j=0}^{\infty}\psi_{j}^{2}<\infty$. By stationarity, the coefficients
$\psi_{j}$ are independent of $t$. Also,

$\psi_{0}=c^{-2}\langle X_{t},X_{t}\ominus
P_{\mathbb{V}^{t-1}}X_{t}\rangle=c^{-2}\|X_{t}\ominus
P_{\mathbb{V}^{t-1}}X_{t}\|^{2}=c^{-2}\|Z_{t}\|^{2}=1,$

thus proving (i). From equation (30) and (31), for $s\leq t$,

$\displaystyle\langle U_{t},Z_{s}\rangle$ $\displaystyle=$
$\displaystyle\left\langle X_{t}\ominus\bigoplus_{j=0}^{\infty}\psi_{j}\circ
Z_{t-j},\;Z_{s}\right\rangle$ $\displaystyle=$ $\displaystyle\left\langle
X_{t},Z_{s}\right\rangle-\left\langle\bigoplus_{j=0}^{\infty}\psi_{j}\circ
Z_{t-j},\;Z_{s}\right\rangle$ $\displaystyle=$ $\displaystyle\left\langle
X_{t},Z_{s}\right\rangle-\psi_{t-s}\left\langle Z_{s},Z_{s}\right\rangle$
$\displaystyle=$ $\displaystyle\left\langle
X_{t},Z_{s}\right\rangle-\|Z_{s}\|^{-2}\left\langle
X_{t},Z_{s}\right\rangle\|Z_{s}\|^{2}$ $\displaystyle=$ $\displaystyle 0.$

In addition, if $s>t$,
$Z_{s}\in\mathbb{V}^{{s-1}{\perp}}\subset\mathbb{V}^{{t}{\perp}}$. But
$U_{t}\in\mathbb{V}^{t}$. Hence $\langle U_{t},Z_{s}\rangle=0$ for $s>t$. Thus
(iv) is proved.

Since $U_{t}$ is orthogonal to $Z_{t}$, $U_{t}\in\mathbb{V}^{t-1}$, that is
$U_{t}$ can be predicted perfectly from previous $X$’s. To see this clearly,
consider the projection of $U_{t}$ on $\mathbb{V}^{t-1}$ to get

$P_{\mathbb{V}^{t-1}}U_{t}=P_{\mathbb{V}^{t-1}}X_{t}\ominus
P_{\mathbb{V}^{t-1}}\bigoplus_{j=0}^{\infty}\psi_{j}\circ Z_{t-j}\\\
=P_{\mathbb{V}^{t-1}}X_{t}\ominus\;\bigoplus_{j=1}^{\infty}\psi_{j}\circ
Z_{t-j},$

since $P_{\mathbb{V}^{t-1}}Z_{t}=0$ and $P_{\mathbb{V}^{t-1}}Z_{t-k}=Z_{t-k}$
for $k\geq 1$. Transformed-linearly subtracting above equation from (31) gives

$U_{t}\ominus P_{\mathbb{V}^{t-1}}U_{t}=\left(X_{t}\ominus
P^{\mathbb{V}^{t-1}}X_{t}\right)\;\ominus\;\psi_{0}\circ
Z_{t}=0_{\mathbb{V}},$

since the one-step ahead prediction error for $X_{t}$ is $\psi_{0}\circ
Z_{t}$. Hence, $U_{t}=P_{\mathbb{V}^{t-1}}U_{t}$. In general,

$P_{\mathbb{V}^{t-k}}U_{t}=P_{\mathbb{V}^{t-k}}X_{t}\ominus\;\bigoplus_{j=k}^{\infty}\psi_{j}\circ
Z_{t-j}.$

Transformed-linearly subtracting above equation from (31) gives

$U_{t}\ominus P_{\mathbb{V}^{t-k}}U_{t}=\left(X_{t}\ominus
P_{\mathbb{V}^{t-k}}X_{t}\right)\;\ominus\;\bigoplus_{j=0}^{k-1}\psi_{j}\circ
Z_{t-j}=0_{\mathbb{V}},$

since the k-step ahead prediction error for $X_{t}$ is
$\bigoplus_{j=0}^{k-1}\psi_{j}\circ Z_{t-j}$. Thus $\\{U_{t}\\}$ is
deterministic as it can be predicted from past $X$’s. $\square$

###  6.3 Simulation Study

We conduct a simulation study that corroborates the richness of the class of
transformed-linear regularly-varying MA($\infty$) models. We simulate data
from two models. The first model is a GARCH(1,1) process (Bollerslev (1986))
with Gaussian noise terms and parameters $\alpha_{0}=0.2$, $\alpha_{1}=0.5$,
and $\beta_{1}=0.3$. We consider the time series of absolute values of this
GARCH process, which we denote $x_{t}^{(orig)}$ A chi-plot (not shown) for the
upper tail shows asymptotic dependence with $\hat{\chi}(1)\approx 0.34$. The
Hill estimator (Hill (1975)) at the empirical $0.99$ quantile of this
transformed data gives an estimate $\hat{\alpha}=3.27$ of the tail index. The
scale is estimated to be $\hat{c}=0.47$. We further transform the data into
$x_{t}=\hat{c}^{-1/2}(x_{t}^{(orig)})^{\hat{\alpha}/2}$ so that our marginal
now can be assumed to have. $\alpha=2$ and $\sigma(0)=1$. As discussed in
Mhatre and Cooley (2020+), preprocessing the data to have $\sigma(0)=1$ allows
us to reduce bias in TPDF estimation. Note that by doing this the noise terms
$Z_{j}$ are no longer such that $\sigma_{Z_{j}}(0)=1$. As done in Mhatre and
Cooley (2020+), to reduce bias in TPDF estimation, we subtract off the mean of
the transformed data and replace the negative observations with $0$. We
estimate the TPDF up to 500 lags using data whose radial components exceeds
the $0.99$ quantile. The squared distance of prediction $\nu_{n}$ converges to
0.65. Running the innovations algorithm on the estimated TPDF gives us
converged $\theta$ estimates of an MA model. We consider the first 25
$\hat{\theta}$’s as we deem the $\theta$ estimates to be negligible beyond
that. We then generate Fréchet noise terms with $\alpha=2$ and scale
$\sqrt{\nu_{n}}=\sqrt{0.65}$, and simulate a transformed-linear regularly-
varying MA($25$) time series using the estimated $\theta$’s from the
innovations algorithm. We then back-transform the simulated time series to the
original marginals. The average difference for the first 25 lags between the
estimated TPDF from the original GARCH data and the estimated TPDF from our
fitted model is $-0.01$ (se = $0.01$). Figure 1 shows the estimated TPDF’s for
both the GARCH model data and the data simulated from the fitted MA(25).

Figure 1: Left panel: estimated TPDF’s for the original GARCH data and for the
data simulated from the fitted transformed-linear MA(25) model. Right panel:
same but for the original Markov logistic model and the fitted transformed-
linear MA(30).

The second model is a first-order Markov chain such that each pair of
consecutive observations has a bivariate logistic distribution with a
dependence parameter of 0.4 and common unit-Fréchet marginals (refer Smith et
al. (1997)). A chi-plot (not shown) for the upper tail shows asymptotic
dependence with $\hat{\chi}(1)\approx 0.7$. We perform the square-root
transformation on the data so that $\alpha=2$. Following the same process as
for the first model, we simulate a transformed-linear regularly-varying
MA($30$) time series using the estimated $\theta$’s from the innovations
algorithm and back-transform the simulated time series to the original
marginals. The average difference for the first 30 lags between the estimated
TPDF from the original logistic data and the estimated TPDF from our fitted
model is $0.05$ (se = $0.03$).

Table 1 gives the average length of run above higher quantiles for the
original time series data and the fitted time series data (using coefficient
estimates from the innovations algorithm) for the GARCH model and the logistic
model. Table 2 gives the higher quantiles for sum of three consecutive time
series terms. The fitted models seem to produce reasonable estimates of these
tail quantities. Interestingly, for the fitted MA model in the logistic case
(last column in Table 1), there is an increasing trend in the estimates and we
do not see the “threshold stability” as exhibited by the fitted GARCH
model.Tables 1 and 2 show that a model which captures the level of dependence
in the TPDF can adequately estimate quantities of interest like length of run
or sums of consecutive terms, despite not fully characterizing the angular
measure of the true model. In fact, lag-1 scatterplots of the target and
fitted models (available in Mhatre (2022)) show distinct differences in the
nature of the bivariate dependence structure.

Table 1: Average length (standard error) of run above a threshold for the simulation study Threshold | GARCH |  | Logistic
---|---|---|---
quantile | Original | Fitted |  | Original | Fitted
0.95 | 1.57 (0.02) | 1.71 (0.03) |  | 3.02 (0.09) | 3.88 (0.11)
0.98 | 1.57 (0.03) | 1.60 (0.04) |  | 3.21 (0.15) | 3.87 (0.16)
0.99 | 1.56 (0.05) | 1.62 (0.06) |  | 3.44 (0.22) | 4.46 (0.26)
0.995 | 1.56 (0.06) | 1.66 (0.09) |  | 3.45 (0.29) | 4.63 (0.35)
0.999 | 1.47 (0.11) | 1.52 (0.15) |  | 2.86 (0.48) | 4.76 (0.76)
Table 2: Quantiles for sum (standard error) of twelve consecutive terms for the simulation study Quantile | GARCH |  | Logistic
---|---|---|---
| Original | Fitted |  | Original | Fitted
0.95 | 16.23 (0.20) | 17.49 (0.16) |  | 223.19 (8.29) | 233.21 (8.26)
0.98 | 21.36 (0.31) | 22.05 (0.34) |  | 565.32 (37.79) | 589.23 (37.77)
0.99 | 25.72 (0.68) | 26.72 (0.58) |  | 1252.96 (184.83) | 1250.13 (118.99)
0.995 | 32.08 (1.24) | 32.09 (1.03) |  | 2789.53 (420.92) | 2672.24 (338.49)
0.999 | 50.67 (3.67) | 48.90 (2.98) |  | 12833.18 (3736.45) | 12732.06 (3705.63)

##  7 Prediction Error

We now return our attention to prediction and investigate the problem of
assessing prediction uncertainty. Because the geometry of regular variation is
very different from the elliptical geometry assumed in many non-extreme
settings, we need to deal with uncertainty in prediction differently.

###  7.1 Completely Positive Decomposition of the Prediction TPDM

The squared distance of prediction, $\nu_{n}$, is the analogue to mean square
prediction error. In the finite-dimensional multivariate case Lee (2021) have
shown that squared-norm prediction error $\nu_{n}$ is not useful to construct
a prediction interval in the polar geometry of regular variation because the
magnitude of error is dependent on the magnitude of the predicted value. We
follow Lee (2021) and apply their method to construct prediction intervals
when $\hat{X}_{n+1}$ is large.

The tail dependence between $\hat{X}_{n+1}$ and $X_{n+1}$ can be characterized
by the bivariate angular measure $H_{\hat{X}_{n+1},X_{n+1}}$. As shown in Lee
(2021), the dependence of $H_{\hat{X}_{n+1},X_{n+1}}$ is summarized by the
prediction TPDM

$\displaystyle\Sigma_{\hat{X}_{n+1},X_{n+1}}=\begin{bmatrix}\bm{\sigma}_{n}^{T}\Sigma_{n}^{-1}\bm{\sigma}_{n}&\bm{\sigma}_{n}^{T}\Sigma_{n}^{-1}\bm{\sigma}_{n}\\\
\bm{\sigma}_{n}^{T}\Sigma_{n}^{-1}\bm{\sigma}_{n}&\sigma(0)\end{bmatrix},$
(32)

where $\Sigma_{n}=\left[\sigma(i-j)\right]^{n}_{i,j=1}$ and
$\bm{\sigma}_{n}=\left[\sigma(i)\right]^{n}_{i=1}$. Since
$\Sigma_{\hat{X}_{n+1},X_{n+1}}$ is a $2\times 2$ completely positive matrix,
given any $q_{*}\geq 2$, there exist nonnegative matrices
$B\in\mathbb{R}^{2\times q_{*}}$ such that
$BB^{T}=\Sigma_{\hat{X}_{n+1},X_{n+1}}$. For feasible computation, Lee (2021)
choose a moderate $q_{*}$ and apply the algorithm in Groetzner (2020)
repeatedly to get $n_{decomp}$ nonnegative $B^{(k)}$ matrices,
$k=1,\cdots,n_{decomp}$, such that
$B^{(k)}B^{(k)^{T}}=\Sigma_{\hat{X}_{n+1},X_{n+1}}$ for all $k$. Then,

$\hat{H}_{\hat{X}_{n+1},X_{n+1}}=n_{decomp}^{-1}\sum_{k=1}^{n_{decomp}}\sum_{j=1}^{q_{*}}\|b^{(0)}_{kj}\|_{2}^{2}\;\delta_{b^{(0)}_{kj}/\|b^{(0)}_{kj}\|_{2}}(\cdot),$

where $b_{kj}$ is the $j^{\text{th}}$ column of the $k^{\text{th}}$ matrix
$B^{(k)}$ and $\delta$ is the Dirac mass function, and

$\Sigma_{\hat{H}}=n_{decomp}^{-1}\sum_{k=1}^{n_{decomp}}B^{(k)}B^{(k)^{T}}=\Sigma_{\hat{X}_{n+1},X_{n+1}}.$

Defined this way, $\hat{H}_{\hat{X}_{n+1},X_{n+1}}$ consists of
$n_{decomp}\times q_{*}$ point masses.

As in Section 6.3, we simulate 100,000 random observations of a first order
Markov chain model such that each pair of consecutive observations has a
bivariate logistic distribution with dependence parameter of 0.4 and common
unit-Fréchet marginals. We perform the square-root transformation on the data
so that $\alpha=2$. We consider the first 70,000 observations as training data
and the remaining as test data. In Section 6.3 we fitted a transformed-linear
regularly-varying MA($30$) time series to data simulated from the logistic
model because the converged innovations algorithm gave negligible $\theta$
estimates beyond $\theta_{30}$. Hence we consider the problem of predicting
any observation $X_{n+1}$, $n\geq 30$, based on the previous $30$
observations. Let us denote this predicted observation as
$\hat{X}_{n+1|n:n-29}$. Using equation (21) we obtain $\hat{\bm{b}}$ and the
prediction TPDM $\Sigma_{\hat{X}_{n+1|n:n-29},X_{n+1}}$ is obtained from
equation (32). We apply the algorithm given in Groetzner (2020) repeatedly to
compute $2\times 5$ matrices $B^{(k)}$, $k=1,\cdots,100$, each of which is a
completely positive decomposition of $\Sigma_{\hat{X}_{n+1|n:n-29},X_{n+1}}$.
Thus our estimated angular measure $\hat{H}_{\hat{X}_{n+1|n:n-29},X_{n+1}}$
has $500$ point masses. The $0.025$ and $0.975$ quantiles of
$\hat{H}_{\hat{X}_{n+1|n:n-29},X_{n+1}}$ give us a $95\%$ joint region. The
left panel of Figure 2 gives $95\%$ joint region on the 30,000 test data.
Thresholding at the $0.95$ quantile of $\|\hat{X}_{n+1|n:n-29},X_{n+1}\|$,
$99.6\%$ of the large data points fall within this joint region.

###  7.2 Conditional Prediction Intervals

The conditional density of $X_{2}|X_{1}=x_{1}$ if $x_{1}$ is large is given in
Lee (2021) as approximately

$\displaystyle
f_{X_{2}|X_{1}}(x_{2}|x_{1})=2c^{-1}\|(x_{1},x_{2})\|^{-5}_{2}x_{2}h\left(\frac{(x_{1},x_{2})}{\|(x_{1},x_{2})\|_{2}}\right),$
(33)

where
$c=\int^{\infty}_{0}2\|(x_{1},x_{2})\|^{-5}_{2}x_{2}h\left(\frac{(x_{1},x_{2})}{\|(x_{1},x_{2})\|_{2}}\right)\text{d}x_{2}$.
We obtain an estimate of the conditional density of $X_{n+1}$ given a large
value of $\hat{X}_{n+1}$ using equation (33). The angular density $h$ is
estimated through a kernel density estimate of
$\hat{H}_{\hat{X}_{n+1},X_{n+1}}$. The $0.025$ and $0.975$ quantiles of the
estimated conditional density in equation (33) give us a $95\%$ conditional
prediction interval. The right panel of Figure 2 gives the scatterplot after
thresholding the test data at the $0.95$ quantile of $\hat{X}_{n+1|n:n-29}$,
along with the $95\%$ conditional prediction intervals. These prediction
intervals have a coverage rate of $0.975$.

Figure 2: Scatterplot of the Logistic model test data, on the transformed
Fréchet scale with $\alpha=2$, with the estimated 95% joint prediction region
(left panel). 95% conditional prediction intervals given each large value of
$\hat{X}_{n+1|n:n-29}$ of the Logistic model test data, on the transformed
Fréchet scale with $\alpha=2$ (right panel).

##  8 Application to Santa Ana Winds

We return to the March AFB hourly windspeed data that we fitted a transformed-
linear regularly-varying ARMA(1,1) time series to in Mhatre and Cooley
(2020+). We marginally transformed the data to have regularly-varying tails
with $\alpha=2$ and $\sigma(0)=1$. After subtracting off the mean of the
transformed data and replacing the negative observations with $0$, we estimate
the TPDF up to 500 lags using data whose radial components exceeds the $0.99$
quantile. Running the innovations algorithm on the estimated TPDF gives us
converged $\theta$ estimates of an MA model. The squared distance of
prediction, $\nu_{n}$, converges to We consider the first 40 $\hat{\theta}$’s
since the $\theta$ estimates are negligible beyond that. We then generate
Fréchet noise terms, with $\alpha=2$ and scale $\sqrt{\nu_{n}}=\sqrt{0.65}$,
and simulate a transformed-linear regularly-varying MA($40$) time series using
the estimated $\theta$’s from the innovations algorithm and back transform the
simulated time series to the original marginals. The average difference
between the estimated TPDF from the original windspeed anomalies data and the
estimated TPDF from our fitted model is $-0.02$ (se = $0.01$). Table 3 gives
average length of run above higher quantiles for the original windspeed
anomalies time series data and the fitted time series data (using coefficient
estimates from the innovations algorithm). Table 4 gives the higher quantiles
for sum of three consecutive time series terms. The fitted models seem to
produce reasonable estimates of these tail quantities.

Table 3: Average length (standard error) of run above a threshold for the windspeed data Threshold | Original | Fitted
---|---|---
quantile |  |
0.95 | 2.43 (0.06) | 2.32 (0.07)
0.98 | 2.35 (0.09) | 2.27 (0.11)
0.99 | 2.10 (0.10) | 2.46 (0.18)
0.995 | 1.77 (0.11) | 2.35 (0.23)
0.999 | 1.40 (0.10) | 2.04 (0.33)
Table 4: Quantiles for sum (standard error) of twelve consecutive terms for the windspeed data Quantile | Original | Fitted
---|---|---
0.95 | 27.70 (0.72) | 28.44 (0.52)
0.98 | 43.74 (1.19) | 42.93 (1.07)
0.99 | 56.65 (1.66) | 54.82 (1.62)
0.995 | 69.67 (2.11) | 68.91 (2.50)
0.999 | 91.89 (3.60) | 97.93 (4.49)

We now perform prediction of the windspeed anomalies time series. Out of the
103,630 observations, we consider the first 70,000 observations as training
data and the remaining as test data. We consider the problem of predicting an
observation $X_{n+1}$, $n\geq 40$, based on the previous 40 observations. Let
us denote this predicted observation as $\hat{X}_{n+1|n:n-39}$. Using equation
(21) we obtain $\hat{\bm{b}}$ and the prediction TPDM
$\Sigma_{\hat{X}_{n+1|n:n-39},X_{n+1}}$ is obtained from equation (32). We
apply the algorithm given in Groetzner (2020) repeatedly to compute $2\times
5$ matrices $B^{(k)}$, $k=1,\cdots,100$, each of which is a completely
positive decomposition of $\Sigma_{\hat{X}_{n+1|n:n-39},X_{n+1}}$. Thus our
estimated angular measure $\hat{H}_{\hat{X}_{n+1|n:n-39},X_{n+1}}$ has $500$
point masses. The $0.025$ and $0.975$ quantiles of
$\hat{H}_{\hat{X}_{n+1|n:n-39},X_{n+1}}$ give us a $95\%$ joint region. The
left panel of Figure 3 gives $95\%$ joint region on the test data.
Thresholding at the $0.95$ quantile of $\|\hat{X}_{n+1|n:n-39},X_{n+1}\|$,
$98.99\%$ of the large data points fall within this joint region.

We obtain an estimate of the conditional density of $X_{n+1}$ given a large
value of $\hat{X}_{n+1}$ using equation (33). The $0.025$ and $0.975$
quantiles of the estimated conditional density in equation (33) give us a
$95\%$ conditional prediction interval. The center panel of Figure 3 gives the
scatterplot after thresholding at the $0.95$ quantile of
$\hat{X}_{n+1|n:n-39}$ of the test data along with the $95\%$ conditional
prediction bounds. These prediction intervals have a coverage rate of $0.96$.
The right panel of Figure 3 gives the prediction intervals on the original
scale of the anomalies obtained by taking the inverse of the marginal
transformation. We compare our prediction intervals to the standard Gaussian
method. We transform the marginal of the original windspeed anomalies data to
be standard normal and estimate the ACVF. We use the estimated covariance
matrix to find the best linear unbiased predictor and to estimate the MSPE. We
then create 95% Gaussian prediction intervals from the estimated MSPE and get
a coverage rate of 0.94. For the windspeed anomalies data, our prediction
intervals do not show significant advantage over the standard Gaussian based
prediction intervals because our data is not too heavy-tailed, resulting into
a negligible difference between the corresponding predicted weight vectors
$\hat{\bm{b}}$. We are investigating a heavy-tailed precipitation data set
where we suspect the difference between the corresponding predicted weight
vectors will be more significant.

Figure 3: Scatterplot of the windspeed anomalies test data on the Fréchet
scale with the estimated 95% joint prediction region (left panel). 95%
conditional prediction intervals given each large value of
$\hat{X}_{n+1|n:n-39}$ of the windspeed anomalies test data on the Fréchet
scale (center panel). 95% conditional prediction intervals given each large
value of $\hat{X}_{n+1|n:n-39}$ of the windspeed anomalies test data on the
original scale (right panel).

##  9 Summary

This paper extends the work of Mhatre and Cooley (2020+), which introduced the
transformed linear time series models. That paper laid out the foundation of
these models, introducing the notion of tail stationarity which enables
characterization of tail dependence in time series by the TPDF. Mhatre and
Cooley (2020+) introduced the transformed linear backshift operator which
allows the AR and ARMA models to be defined as transformed linear solutions to
the model equations. They showed via example that the transformed linear
models were more flexible than nonnegative traditional linear models in
capturing tail dependence.

This paper introduces the transformed linear innovations algorithm with the
aim of performing prediction when the previous time series terms are large.
Not only does the innovations algorithm enable iterative prediction, but it
also provides us a tool demonstrating the richness of the class of models.
Perhaps the most important result is that one can fit a transformed-linear
regularly-varying MA($\infty$) arbitrarily close to any valid TPDF. Using the
polar geometry of regular variation, we develop prediction intervals when
predicted values are large.

There are many avenues for future work. A prevalent method for identifying the
orders of AR and MA in classical time series is through looking at
autocorrelation function (ACF) and partial autocorrelation function (PACF)
plots. This necessitates the development of a PACF analogue for our models.
Some initial results on partial tail correlation in the finite dimensions have
been worked on by Lee & Cooley in their upcoming paper.

There might also be motivation in extremes to think about a non-causal time
series. The innovations algorithm looks only backward in time, by definition.
In the simulated time series in our causal setting, we see that extreme
behavior is characterized by an asymmetric spike, where large values have
subsequent large values, but no large values precede the spike. Although these
transformed linear values have shown good ability to capture dependence
summary measures, behavior of the simulated series differs from that of the
environmental time series we have so far explored. A relaxation of causality
could likely address this.

## Acknowledgement

Nehali Mhatre and Daniel Cooley were both partially supported by National
Science Foundation award DMS-1811657.

## Supplementary Material

Supplementary materials demonstrate that $\mathbb{V}$ is a vector space, that
the defined inner product meets the considitions of inner product, and that
$T$ is an linear map creatig the isomorphism between $\mathbb{V}$ and
$\ell_{1}$. The windspeed anomalies data and the R codes for the simulation
study results (Section 6.3) and the application to the windspeed anomalies
data (Section 8) are available at (https://www.stat.colostate.edu/
cooleyd/TransLinTS/).

## References

  * (1)
  * Bollerslev (1986) Bollerslev, T. (1986), ‘Generalized autoregressive conditional heteroskedasticity’, Journal of Econometrics 31, 307–327.
  * Brockwell and Davis (1991) Brockwell, P. J. and Davis, R. A. (1991), Time series: theory and methods, Springer, New York.
  * Cooley and Thibaud (2019) Cooley, D. and Thibaud, E. (2019), ‘Decompositions of dependence for high-dimensional extremes’, Biometrika 106(3), 587–604.
  * Groetzner (2020) Groetzner, P. & Dür, M. (2020), ‘A factorization method for completely positive matrices’, Linear Algebra and its Applications 591, 1–24.
  * Hill (1975) Hill, B. M. (1975), ‘A simple general approach to inference about the tail of a distribution’, The Annals of Statistics 3, 1163–1174.
  * Kiriliouk and Zhou (2022) Kiriliouk, A. and Zhou, C. (2022), ‘Estimating probabilities of multivariate failure sets based on pairwise tail dependence coefficients’.
  * Kulik and Soulier (2020) Kulik, R. and Soulier, P. (2020), Heavy-tailed Time Series, Springer-Verlag, New York.
  * Lee (2021) Lee, J. & Cooley, D. (2021), ‘Transformed-linear prediction for extremes’, arXiv preprint .
arXiv:2111.03754v4

  * Mhatre (2022) Mhatre, N. (2022), Transformed-linear models for time series extremes, PhD thesis, Colorado State University.
  * Mhatre and Cooley (2020+) Mhatre, N. and Cooley, D. (2020+), ‘Transformed-linear models for time series extremes’, arXiv Preprint .
arXiv:2012.06705v3

  * Resnick (2007) Resnick, S. I. (2007), Heavy-tail phenomena: probabilistic and statistical modeling, Springer-Verlag, New York.
  * Rudin (1991) Rudin, W. (1991), Functional analysis, 2nd ed., McGraw-Hill, New York.
  * Sargent (1979) Sargent, T. J. (1979), Macroeconomic theory, Economic theory, econometrics, and mathematical economics, Academic Press.
  * Smith et al. (1997) Smith, R. L., Tawn, J. and Coles, S. (1997), ‘Markov chain models for threshold exceedances’, Biometrika 84, 249–268.