Minimax-robust estimation problems for stationary stochastic sequencesMikhail
Moklyachuk

Mikhail Moklyachuk (Email: Moklyachuk@gmail.com). Department of Probability
Theory, Statistics and Actuarial Mathematics, Taras Shevchenko National
University of Kyiv, Volodymyrs’ka 64 Str., Kyiv 01601, Ukraine.

# Minimax-robust estimation problems for stationary stochastic sequences

Mikhail Moklyachuk , 1  11affiliationmark:  Department of Probability Theory,
Statistics and Actuarial Mathematics, Taras Shevchenko National University of
Kyiv, Kyiv, Ukraine

###### Abstract

This survey provides an overview of optimal estimation of linear functionals
which depend on the unknown values of a stationary stochastic sequence. Based
on observations of the sequence without noise as well as observations of the
sequence with a stationary noise, estimates could be obtained. Formulas for
calculating the spectral characteristics and the mean-square errors of the
optimal estimates of functionals are derived in the case of spectral
certainty, where spectral densities of the sequences are exactly known. In the
case of spectral uncertainty, where spectral densities of the sequences are
not known exactly while sets of admissible spectral densities are given, the
minimax-robust method of estimation is applied. Formulas that determine the
least favourable spectral densities and the minimax spectral characteristics
of estimates are presented for some special classes of admissible spectral
densities.

###### keywords:

Stochastic sequence, minimax-robust estimate, mean square error, least
favourable spectral density, minimax-robust spectral characteristic

AMS 2010 subject classifications. Primary: 60G10, 60G25, 60G35, Secondary:
62M20, 62P20, 93E10, 93E11

DOI: 10.19139/soic.v3i4.173

###### Contents

  1. 1 Introduction
  2. 2 Stationary stochastic sequences. Spectral representation
    1. 2.1 Definitions. Examples
    2. 2.2 Spectral representation of stationary sequences
    3. 2.3 Linear filters of stationary sequences
    4. 2.4 Wold expansion of stationary sequences
    5. 2.5 Conclusions
  3. 3 Estimates for functionals of stationary sequences
    1. 3.1 The maximum value of the mean-square error of estimate of the functional $A_{N}{\xi}$
    2. 3.2 The maximum value of the mean square error of estimate of the functional $A{\xi}$
    3. 3.3 Conclusions
  4. 4 Extrapolation problem for functionals of stationary sequences
    1. 4.1 The classical Hilbert space projection method of linear extrapolation
    2. 4.2 Minimax-robust method of linear extrapolation of functionals
    3. 4.3 Least favourable spectral densities in the class $D_{0}$
    4. 4.4 Least favourable spectral densities in the class $D_{M}$
    5. 4.5 Least favourable spectral densities in the class $D_{v}^{u}$
    6. 4.6 Least favourable spectral densities in the class $D_{\varepsilon}$
    7. 4.7 Least favourable spectral densities in the class $D_{1\varepsilon}$
    8. 4.8 Least favourable spectral densities in the class $D_{2\varepsilon}$
    9. 4.9 Conclusions
  5. 5 Extrapolation problem for stationary sequences from observations with noise
    1. 5.1 The classical Hilbert space projection method of linear extrapolation
    2. 5.2 Minimax-robust method of extrapolation
    3. 5.3 Least favourable spectral densities in the class $D_{f}^{0}\times D_{g}^{0}$
    4. 5.4 Least favourable spectral densities in the class $D_{v}^{u}\times D_{\varepsilon}$
    5. 5.5 Least favourable spectral densities in the class $D_{2\varepsilon_{1}}\times D_{1\varepsilon_{2}}$
    6. 5.6 Conclusions
  6. 6 Interpolation problem for stationary sequences
    1. 6.1 The classical Hilbert space projection method of linear interpolation
    2. 6.2 Minimax-robust method of interpolation
    3. 6.3 Least favourable spectral densities in the class $D_{0}^{-}$
    4. 6.4 Least favourable spectral densities in the class $D_{M}$
    5. 6.5 Least favourable spectral densities in the class $D_{v}^{u-}$
    6. 6.6 Conclusions
  7. 7 Interpolation problem for stationary sequences from observations with noise
    1. 7.1 The classical Hilbert space projection method of linear interpolation
    2. 7.2 Minimax-robust method of interpolation
    3. 7.3 Least favourable spectral densities in the class $D_{f}^{0}\times D_{g}^{0}$
    4. 7.4 Least favourable spectral densities in the class $D_{v}^{u}\times D_{\varepsilon}$
    5. 7.5 Least favourable spectral densities in the class $D_{2\varepsilon_{1}}\times D_{1\varepsilon_{2}}$
    6. 7.6 Conclusions
  8. 8 Conclusion Remarks

##  1 Introduction

Theory of estimation of the unknown values of stationary stochastic processes
based on a set of observations of the processes plays an important role in
many practical applications. The development of the theory started from the
classical works of Kolmogorov [24] and Wiener [77], in which they presented
methods of solution of the extrapolation and interpolation problems for
stationary processes. The interpolation problem considered by Kolmogorov means
estimation of the missed values of stochastic sequences. The prediction
problem consists in estimation the future values of the process based on
observations of the process in the past. The third classical problem is
filtering of stochastic processes which consists in estimation the original
values of the signal process from observations of the process with noise. All
these problems for stationary sequences and processes are clearly described in
books by Gikhman and Skorokhod [12], Hannan [17], Rozanov [73], Yaglom [80,
81].

One of the fields of practical applications of the stationary and related
stochastic sequences is economical modelling and financial time series. Most
simple examples of stationary linear models are moving average (MA) sequences,
autoregressive (AR) and autoregressive-moving average (ARMA) sequences, state
space models, all of which refer to stationary sequences with rational
spectral function without unit AR-roots. The main results concerning the model
description, parameter estimation, forecasting and further investigations are
described in the classical book by Box, Jenkins and Reinsel [2]. Most of
results which have appeared since that time were based on the assumption that
the spectral structure of the stationary sequence is known.

Traditional methods of solution of linear extrapolation, interpolation and
filtering problems for stationary stochastic sequences may be employed under
the condition that spectral densities of the sequences are known exactly (see,
for example, selected works of Kolmogorov [24], survey article by Kailath
[21], books by Rozanov [73], Wiener [77], Yaglom [80, 81]). In practice,
however, complete information on the spectral densities is impossible in most
cases. To solve the problem the parametric or nonparametric estimates of the
unknown spectral densities are found or these densities are selected by other
reasoning. Then the classical estimation method is applied provided that the
estimated or selected densities are the true one. This procedure can result in
a significant increasing of the value of the error as Vastola and Poor [76]
have demonstrated with the help of some examples. This is a reason to search
estimates which are optimal for all densities from a certain class of the
admissible spectral densities. These estimates are called minimax since they
minimize the maximal value of the error. Many investigators have been
interested in minimax extrapolation, interpolation and filtering problems for
stationary stochastic sequences. See, for example, the papers by Hosoya [19]
and Taniguchi [75]. A survey of results in minimax (robust) methods of data
processing can be found in the paper by Kassam and Poor [23]. The paper by
Grenander [15] should be marked as the first one where the minimax approach to
extrapolation problem for stationary processes was proposed. Franke [9, 10],
Franke and Poor [11] investigated the minimax extrapolation and filtering
problems for stationary sequences with the help of convex optimization
methods. This approach makes it possible to find equations that determine the
least favourable spectral densities for various classes of admissible
densities.

In the works by Moklyachuk [35] – [58] problems of extrapolation,
interpolation and filtering for stationary processes and sequences were
studied. The corresponding problems for vector-valued stationary sequences and
processes were investigated by Moklyachuk and Masyutka [59] – [64]. In the
articles by Dubovets’ka and Moklyachuk [3] \- [7] and in the book by
Golichenko and Moklyachuk [13] the minimax estimation problems were
investigated for another generalization of stationary processes – periodically
correlated stochastic sequences and processes. Luz and Moklyachuk [26] – [32],
[34] investigated the classical and minimax extrapolation, interpolation and
filtering problems for sequences and processes with $n$th stationary
increments.

In the proposed paper we deal with the problem of estimation of the functional
$A\xi=\sum_{k=0}^{\infty}a(k)\xi(k)$ which depends on the unknown values of a
stationary stochastic sequence $\xi(k)$ based on observations of the sequence
$\xi(k)+\eta(k)$, where $\eta(k)$ is an uncorrelated with the sequence
$\xi(k)$ stationary stochastic sequence. Formulas for calculating the mean-
square errors and the spectral characteristics of the optimal estimate of the
functional are derived in the case of spectral certainty, where the spectral
densities of the sequences $\xi(n)$ and $\eta(n)$ are exactly known. The
estimation problem is solved in the case of spectral uncertainty, where the
spectral densities of sequences are not exactly known, but, instead, a set of
admissible spectral densities is given. The minimax-robust estimation method
is applied in this case. Formulas that determine the least favourable spectral
densities and the minimax-robust spectral characteristic of the optimal linear
estimates of the functional $A\xi$ are derived in the case of spectral
uncertainty for some concrete classes of admissible spectral densities.

The paper is organized as follows. The spectral representations of stationary
stochastic sequences and their correlation functions are described in Section
2. Factorizations of spectral densities with relations to one-sided moving
average representations and applications to the problem of prediction of
stationary sequences are discussed in this section. In section 3 the problem
of the mean square optimal linear estimation of the functionals which depend
on the unknown ‘future’ values of a stationary stochastic sequence from a
class of stationary stochastic sequences is investigated. Estimates are based
on observations of the sequence in the ‘past’. Following the Grenander [15]
approach to investigation the problem of optimal linear estimation of the
linear functional we consider the problem as a two-person zero-sum game. It is
show that this game has equilibrium point. In Section 4 the extrapolation
problem for functionals of stationary sequences is investigated in the case of
spectral certainty, where the spectral density of the stationary stochastic
sequence is exactly known, as well as in the case of spectral uncertainty,
where the spectral density of the sequence is not exactly known, but a class
of admissible spectral densities is given. The classical Hilbert space
projection method and the minimax-robust procedure to extrapolation of the
functional are applied. The corresponding results of investigation of the
extrapolation problem for functionals of stationary sequences from
observations with noise are presented in Section 5. In Section 6 and Section 7
results of investigation of the interpolation problem for functionals of
stationary sequences from observations without noise and with noise are
presented. The problem is investigated in the case of spectral certainty as
well as in the case of spectral uncertainty. Formulas are proposed for
calculation the value of the mean square errors and the spectral
characteristics of the interpolation in the case of spectral certainty.
Relations which determine the least favourable spectral densities and the
minimax spectral characteristics are proposed in the case of spectral
uncertainty for some special classes of spectral densities.

##  2 Stationary stochastic sequences. Spectral representation

In this section properties of stationary stochastic sequences illustrated with
the help of some examples are presented. The spectral representation of
stationary stochastic sequences and their correlation functions are described.
Basic properties of linear transformations of stationary stochastic sequences
are proposed. The Wold representation of stationary stochastic sequences is
studied with application to the problem of prediction of stationary sequences.

###  2.1 Definitions. Examples

Let $H=L_{2}(\Omega,{\mathcal{F}},P)$ be the space of all complex valued
random variables $\xi=\xi(\omega),\omega\in\Omega,$ such that
${E}|\xi|^{2}<\infty$. Covariation of the random variables $\xi,\eta\in H$ is
defined as the quantity

$\text{cov}\,(\xi,\eta)=E(\xi-E\xi)\overline{(\eta-E\eta)}.$ (1)

######  Definition 2.1.

A sequence of complex valued random variables $\xi(n)$, $n\in\mathbb{Z}$, with
values in the space $H=L_{2}(\Omega,{\mathcal{F}},P)$ is called wide sense
stationary if for all $n\in\mathbb{Z}$ and $k\in\mathbb{Z}$ the following
relations hold true

$E\xi(n)=E\xi(0)=C=\text{const},$
${\text{cov}}(\xi(k+n),\xi(k))={\text{cov}}(\xi(n),\xi(0)).$ (2)

In the following we will assume that $E\xi(n)=0$, $n\in\mathbb{Z}$. This
assumption gives us a possibility to consider the covariation as the inner
product in the Hilbert space $H=L_{2}(\Omega,{\mathcal{F}},P)$ and use the
powerful methods and results of the Hilbert space theory.

######  Definition 2.2.

The function

$R(n)={\text{cov}}(\xi(n),\xi(0)),\,\,\,n\in\mathbb{Z},$ (3)

is called the correlation function of the wide sense stationary sequence
$\xi(n)$.

######  Example 1.

White noise sequence. The white noise sequence
$\varepsilon=\\{\varepsilon(n),n\in\mathbb{Z}\\}$ is a sequence of orthogonal
random variables such that
$E\varepsilon(n)=0,E\varepsilon(i)\overline{\varepsilon(j)}=\delta_{ij}$,
where $\delta_{ij}$ is the Kroneker symbol. This sequence is stationary with
the correlation function

$R(n)=\int\limits_{-\pi}^{\pi}{e^{i\lambda n}dF(\lambda)},$ (4)
$F(\lambda)=\frac{1}{2\pi}\int\limits_{-\pi}^{\lambda}f(v)dv,\quad
f(v)=1,\quad-\pi<v<\pi.$ (5)

The white noise sequence has the absolutely continuous spectral function
$F(\lambda)$ with the constant spectral density function $f(v)=1$.

######  Example 2.

Moving average sequence. Let $\varepsilon=\\{\varepsilon(n),n\in\mathbb{Z}\\}$
be a white noise sequence. Consider the sequence

$\xi(n)=\sum\limits_{k=-\infty}^{\infty}{a(k)\varepsilon(n-k),}$ (6)

where $a(k)$ are complex valued numbers such that
$\sum_{k=-\infty}^{\infty}{\left|{a(k)}\right|^{2}<\infty}$. By the Parseval
equality

${\text{cov}}(\xi(n+m),\xi(m))={\text{cov}}(\xi(n),\xi(0))=\sum\limits_{k=-\infty}^{\infty}{a(n+k)\overline{a(k)}}.$

For this reason $\xi=\\{\xi(n),n\in\mathbb{Z}\\}$ is a stationary sequence,
which is called (two-sided) moving average (MA) sequence generated by the
white noise sequence $\varepsilon=\\{\varepsilon(n),n\in\mathbb{Z}\\}$.

In the case where $a(k)=0$ for all $k=-1,-2,\dots$, the sequence

$\xi(n)=\sum\limits_{k=0}^{\infty}a(k)\varepsilon(n-k),$ (7)

is called one-sided moving average (MA) sequence generated by the white noise
sequence $\varepsilon=\\{\varepsilon(n),n\in\mathbb{Z}\\}$.

In the case where $a(k)=0$ for all $k=-1,-2,\dots$ and all $k=p+1,p+2,\dots$,
the sequence

$\xi(n)=a(0)\varepsilon(n)+a(1)\varepsilon(n-1)+\cdots+a(p)\varepsilon(n-p),$
(8)

is called moving average $MA(p)$ sequence of order $p$ generated by the white
noise sequence $\varepsilon=\\{\varepsilon(n),n\in\mathbb{Z}\\}$.

The correlation function $R(n)$ of the moving average sequence $MA(p)$ is of
the form

$R(n)=\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}{e^{i\lambda
n}f(\lambda)d\lambda},$

with the spectral density function

$f(\lambda)=\left|{P(e^{-i\lambda})}\right|^{2},$ (9)

where

$P(z)=a(0)+a(1)z+\cdots+a(p)z^{p}.$

is a polynomial of order $p$.

######  Example 3.

The autoregressive sequence $AR(q)$. Let
$\varepsilon=\\{\varepsilon(n),n\in\mathbb{Z}\\}$ be a white noise sequence. A
sequence $\xi=\\{\xi(n),n\in\mathbb{Z}\\}$ is called the autoregressive
sequence of order $q$ if it satisfies the equation

$\xi(n)+b(1)\xi(n-1)+\cdots+b(q)\xi(n-q)=\varepsilon(n).$ (10)

In the case where all zeros of the polynomial

$Q(z)=1+b_{1}z+\cdots+b_{q}z^{q}$ (11)

are outside the unit disk $D=\\{z:|z|<1\\}$, equation (10) has a stationary
solution which can be represented as one-sided moving average sequence. In
this case the correlation function $R(n)$ is of the form

$R(n)=\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}{e^{i\lambda
n}f(\lambda)d\lambda},$ (12)
$f(\lambda)=\frac{1}{\left|{Q(e^{-i\lambda})}\right|^{2}}.$

######  Example 4.

The autoregressive and moving average $ARMA(p,q)$ sequence. Suppose that in
the right-hand side of the equation (10) instead of $\varepsilon(n)$ we put
the sequence
$a_{0}\varepsilon(n)+a(1)\varepsilon(n-1)+\cdots+a(p)\varepsilon(n-p)$. We get
the so called autoregressive and moving average $ARMA(p,q)$ sequence of order
$(p,q)$

$\xi(n)+b(1)\xi(n-1)+\cdots+b(q)\xi(n-q)=a(0)\varepsilon(n)+a(1)\varepsilon(n-1)+\cdots+a(p)\varepsilon(n-p).$
(13)

Under the same conditions on zeros of the polynomial $Q(z)$ as in the previous
example, equation (13) has a stationary solution
$\xi=\\{\xi(n),n\in\mathbb{Z}\\}$. The correlation function $R(n)$ of the
sequence $\xi=\\{\xi(n),n\in\mathbb{Z}\\}$ is of the form

$R(n)=\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}{e^{i\lambda
n}f(\lambda)d\lambda},$
$f(\lambda)=\left|{\frac{P(e^{-i\lambda})}{Q(e^{-i\lambda})}}\right|^{2}.$
(14)

###  2.2 Spectral representation of stationary sequences

######  Theorem 1.

Herglotz theorem. Let $R(n)$ be the covariance function of a (wide sense)
stationary stochastic sequence $\xi=\\{\xi(n),n\in\mathbb{Z}\\}$ with zero
mean value. Then on the space
$\left([-\pi,\pi),{\mathcal{B}}([-\pi,\pi))\right)$ there is a finite measure
$F=F(B)$, $B\in{\cal B}([-\pi,\pi))$, such that for every $n\in\mathbb{Z}$

$R(n)=\int\limits_{-\pi}^{\pi}{e^{i\lambda n}F(d\lambda),}$ (15)

where $\int\limits_{-\pi}^{\pi}{e^{i\lambda n}F(d\lambda)}$ is the Lebesgue-
Stieltjes integral.

######  Remark 2.1.

The measure $F=F(B)$ in the representation (15) is called the spectral
measure, and the function $F(\lambda)=F([-\pi,\lambda])$ is called the
spectral function of the stationary sequence with the covariance function
$R(n)$.

######  Remark 2.2.

In the case where $\sum_{n=-\infty}^{\infty}|R(n)|<\infty$ the spectral
function $F(\lambda)$ has the spectral density $f(\lambda)$, which is
determined by the formula

$f(\lambda)=\sum_{n=-\infty}^{\infty}e^{-in\lambda}R(n).$

The following theorem provides the corresponding spectral representation of
the stationary stochastic sequence $\xi=\\{\xi(n),n\in\mathbb{Z}\\}$ itself.

######  Theorem 2.

Let $\xi=\\{\xi(n),n\in\mathbb{Z}\\}$ be a stationary stochastic sequence with
zero mean value $E\xi(n)=0$, $n\in\mathbb{Z}$. There is an orthogonal
stochastic measure $Z=Z(\Delta)$, $\Delta\in{\cal B}([-\pi,\pi))$, such that
for every $n\in\mathbb{Z}$ the following spectral representation is valid

$\xi(n)=\int\limits_{-\pi}^{\pi}{e^{i\lambda
n}Z(d\lambda)\left({=\int\limits_{\left[{-\pi,\pi}\right)}{e^{i\lambda
n}Z(d\lambda)}}\right)}.$ (16)

Moreover, $E\left|{Z(\Delta)}\right|^{2}=F(\Delta)$.

Let $\xi=\\{\xi(n),n\in\mathbb{Z}\\}$ be a stationary stochastic sequence with
the spectral representation (16). Let $H(\xi)$ be the closed in the mean-
square sense linear manifold spanned by the random variables $\xi(n)$,
$n\in\mathbb{Z}$. Let $\eta\in H(\xi)$. The following theorem describes the
structure of such random variables.

######  Theorem 3.

For any $\eta\in H(\xi)$ there is a function $\varphi\in L^{2}(F)$ such that

$\eta=\int\limits_{-\pi}^{\pi}{\varphi(\lambda)Z(d\lambda)}.$ (17)

######  Remark 2.3.

Let $H_{0}(\xi)$ and $L^{2}_{0}(F)$ be the corresponding closed linear
manifolds spanned by the variables $\\{\xi(n),n=0,-1,-2,\dots\\}$ and by the
functions $\\{e_{n}=e^{in\lambda},n=0,-1,-2,\dots\\}$. Then for $\eta\in
H_{0}(\xi)$ there is a function $\varphi\in L^{2}_{0}(F)$ such that

$\eta=\int_{-\pi}^{\pi}{\varphi(\lambda)Z(d\lambda)}.$

###  2.3 Linear filters of stationary sequences

Consider a special but important class of linear transformations that are
defined by means of the so called linear filters. Suppose that, at instant of
time $m$, a system (filter) receives as input a signal $x(m)$, and that the
output of the system is, at instant of time $n$, the signal $h(n-m)x(m)$,
where $h=h(m)$, $m\in\mathbb{Z}$, is a complex valued function called the
impulse response function of the filter. The total signal $y(n)$ obtained from
the input $x(m)$, $m\in\mathbb{Z}$, can be represented in the form

$y(n)=\sum\limits_{m=-\infty}^{\infty}{h(n-m)x(m)}.$ (18)

For physically realizable systems values of the output at instant of time $n$
are determined only by the ”past” values of the signal, i.e. the values $x(m)$
at instants $m\leq n$. It is therefore natural to call a filter with the
impulse response function $h=h(m)$ physically realizable (causal filter), if
$h(m)=0$ for all $m<0$, in other words if

$y(n)=\sum\limits_{m=-\infty}^{n}h(n-m)x(m)=\sum\limits_{m=0}^{\infty}h(m)x(n-m).$
(19)

An important spectral characteristic of a filter with the impulse response
function $h=h(m)$ is its Fourier transform

$\varphi(\lambda)=\sum\limits_{m=-\infty}^{\infty}{e^{-im\lambda}h(m)},$ (20)

which is called the spectral characteristic of the filter.

Let us now describe conditions under which the series in (18) and (20)
convergence. Let us suppose that the input is a stationary stochastic sequence
$\xi=\\{\xi(n),n\in\mathbb{Z}\\}$ with the covariance function $R(n)$ and
spectral decomposition (16). If the following condition holds true

$\sum\limits_{k,l=-\infty}^{\infty}{h(k)R(k-l)\overline{h(l)}<\infty},$ (21)

then the series $\sum_{m=-\infty}^{\infty}{h(n-m)\xi(m)}$ converges in the
mean-square sense and therefore there is a stationary sequence
$\eta=\\{\eta(n),n\in\mathbb{Z}\\}$ such that

$\eta(n)=\sum\limits_{m=-\infty}^{\infty}{h(n-m)\xi(m)}=\sum\limits_{m=-\infty}^{\infty}{h(m)\xi({n-m})}.$
(22)

In terms of the spectral measure condition (21) is equivalent to the condition
that the spectral characteristic of the filter $\varphi(\lambda)\in L^{2}(F)$,
that is

$\int\limits_{-\pi}^{\pi}{\left|{\varphi(\lambda)}\right|^{2}F(d\lambda)<\infty}.$
(23)

Under the condition (21) or condition (23) from (22) and (16) we find the
spectral representation of the sequence $\eta=\\{\eta(n),n\in\mathbb{Z}\\}$:

$\eta(n)=\int\limits_{-\pi}^{\pi}{e^{i\lambda n}\varphi(\lambda)Z(d\lambda)}.$
(24)

The correlation function $R_{\eta}(n)$ of the sequence
$\eta=\\{\eta(n),n\in\mathbb{Z}\\}$ is determined by the formula

$R_{\eta}(n)=\int\limits_{-\pi}^{\pi}{e^{i\lambda
n}\left|{\varphi(\lambda)}\right|^{2}F(d\lambda)}.$ (25)

In particular, if the input to a filter with the spectral characteristic
$\varphi=\varphi(\lambda)$ is taken to be white noise
$\varepsilon=\\{\varepsilon(n)\\},n\in\mathbb{Z}\\}$, then the output will be
a moving average stationary sequence

$\eta(n)=\sum\limits_{m=-\infty}^{\infty}{h(m)\varepsilon(n-m)}$ (26)

with the spectral density

$f_{\eta}(\lambda)=\left|{\varphi(\lambda)}\right|^{2}.$

The following theorem shows that every stationary stochastic sequence with a
spectral density can be represented as a moving average sequence.

######  Theorem 4.

Let $\eta=\\{\eta(n),n\in\mathbb{Z}\\}$ be a stationary stochastic sequence
with the spectral density $f_{\eta}(\lambda)$. Then (possibly on the extended
probability space) we can find a white noise sequence
$\varepsilon=\\{\varepsilon(n),n\in\mathbb{Z}\\}$ and a filter such that the
representation (26) is valid.

###### Proof.

For a given (nonnegative) spectral density function $f_{\eta}(\lambda)$ we can
find a function ${\varphi(\lambda)}$ such that

$f_{\eta}(\lambda)=\left|{\varphi(\lambda)}\right|^{2}.$

Since

$\int_{-\pi}^{\pi}f_{\eta}(\lambda)d\lambda<\infty,$

we have ${\varphi(\lambda)}\in L^{2}(\lambda)$, where $\lambda$ is the
Lebesgue measure on $[-\pi,\pi]$.

Hence the function ${\varphi(\lambda)}$ can be represented as a Fourier series
(20) with

$h(m)=\frac{1}{2\pi}\int_{-\pi}^{\pi}e^{im\lambda}{\varphi(\lambda)}d\lambda,$

where convergence is understood in the sense that

$\int_{-\pi}^{\pi}\left|{\varphi(\lambda)}-\sum_{|m|\leq
n}e^{-im\lambda}{h(m)}\right|^{2}d\lambda\to 0,\quad n\to\infty.$

Let

$\eta(n)=\int_{-\pi}^{\pi}e^{in\lambda}{Z(d\lambda)},\quad n\in\mathbb{Z}.$

In addition to the random measure $Z=Z(\Delta)$ we introduce another
(independent) orthogonal stochastic measure $\tilde{Z}=\tilde{Z}(\Delta)$ with
$M|\tilde{Z}(a,b]|^{2}=\frac{b-a}{2\pi}$ and construct the measure

$\bar{Z}(\Delta)=\int\limits_{\Delta}{\varphi^{\oplus}(\lambda)}{Z(d\lambda)}+\int\limits_{\Delta}[1-{\varphi^{\oplus}(\lambda)}{\varphi(\lambda)}]{\tilde{Z}(d\lambda)},$

where $a^{\oplus}$ is the pseudoinverse operation

$a^{\oplus}=\left\\{{\begin{array}[]{rl}a^{-1},&a\not=0,\\\ 0,&a=0.\\\
\end{array}}\right.$

The stochastic measure $\bar{Z}=\bar{Z}(\Delta)$ is a measure with orthogonal
values, and for every $\Delta=(a,b]$ we have

$E|\bar{Z}(\Delta)|^{2}=\frac{1}{2\pi}\int\limits_{\Delta}|{\varphi^{\oplus}(\lambda)}|^{2}|{\varphi(\lambda)}|^{2}d\lambda+\frac{1}{2\pi}\int\limits_{\Delta}|1-{\varphi^{\oplus}(\lambda)}{\varphi(\lambda)}|^{2}d\lambda=\frac{|\Delta|}{2\pi},$

where $|\Delta|=b-a$. Therefore the stationary sequence

$\varepsilon(n)=\int_{-\pi}^{\pi}e^{in\lambda}{\bar{Z}(d\lambda)},\quad
n\in\mathbb{Z},$

is a white noise sequence. Note that

$\int_{-\pi}^{\pi}e^{in\lambda}{\varphi(\lambda)}{\bar{Z}(d\lambda)}=\int_{-\pi}^{\pi}e^{in\lambda}{Z}(d\lambda)=\eta(n),$
(27)

and, from the other hand,

$\int_{-\pi}^{\pi}e^{in\lambda}{\varphi(\lambda)}{\bar{Z}(d\lambda)}=\int_{-\pi}^{\pi}e^{in\lambda}\left(\sum\limits_{m=-\infty}^{\infty}{e^{-im\lambda}h(m)}\right){\bar{Z}(d\lambda)}=$
$=\sum\limits_{m=-\infty}^{\infty}h(m)\int_{-\pi}^{\pi}e^{i(n-m)\lambda}{\bar{Z}(d\lambda)}=\sum\limits_{m=-\infty}^{\infty}h(m)\varepsilon(n-m).$

This equality, together with (27), establishes the representation (26). This
completes the proof of the theorem. ∎

######  Corrolary 2.1.

Let the spectral density function $f_{\eta}(\lambda)>0$ (almost everywhere
with respect to the Lebesgue measure) and let the following factorization of
the function $f_{\eta}(\lambda)>0$ holds true

$f_{\eta}(\lambda)=\left|{\varphi(\lambda)}\right|^{2},$ (28)
$\varphi(\lambda)=\sum\limits_{k=0}^{\infty}{e^{-i\lambda
k}h(k)},\quad\sum\limits_{k=0}^{\infty}{\left|{h(k)}\right|^{2}<\infty}.$ (29)

Then the sequence $\eta=\\{\eta(n),n\in\mathbb{Z}\\}$ admits the
representation as one-sided moving average sequence

$\eta(n)=\sum\limits_{m=0}^{\infty}{h(m)\varepsilon(n-m)}.$

In particular, if $P(z)=a(0)+a(1)z+\cdots+a(p)z^{p}$ is a polynomial that has
no zeros on $\\{z:\left|z\right|=1\\}$, then the sequence
$\eta=\\{\eta(n),n\in\mathbb{Z}\\}$ with the spectral density

$f_{\eta}(\lambda)=\left|{P(e^{-i\lambda})}\right|^{2}$

can be represented in the form

$\eta(n)=a(0)\varepsilon(n)+a(1)\varepsilon(n-1)+\cdots+a(p)\varepsilon(n-p),$

where $\varepsilon=\\{\varepsilon(n),n\in\mathbb{Z}\\}$ is a white noise
sequence.

######  Corrolary 2.2.

Let $\xi=\\{\xi(n),n\in\mathbb{Z}\\}$ be a sequence with rational spectral
density

$f(\lambda)=\left|\frac{{P(e^{-i\lambda})}}{{Q(e^{-i\lambda})}}\right|^{2},$
(30)

where polynomials $P(z)=a(0)+a(1)z+\cdots+a(p)z^{p}$ and
$Q(z)=1+b(1)z+\cdots+b(q)z^{q}$ have no zeros on $\\{z:\left|z\right|=1\\}$.
Then there is a white noise sequence
$\varepsilon=\\{\varepsilon(n),n\in\mathbb{Z}\\}$ such that the following
equation holds true

$\xi(n)+b(1)\xi(n-1)+\cdots+b(q)\xi(n-q)=a(0)\varepsilon(n)+a(1)\varepsilon(n-1)+\cdots+a(p)\varepsilon(n-p).$
(31)

Conversely, every stationary stochastic sequence
$\xi=\\{\xi(n),n\in\mathbb{Z}\\}$ that satisfies this equation with a white
noise $\varepsilon=\\{\varepsilon(n),n\in\mathbb{Z}\\}$ and polynomial
$Q(z)=1+b(1)z+\cdots+b(q)z^{q}$ with no zeros on $\\{z:\left|z\right|=1\\}$
has the spectral density (30).

Let us describe conditions under which a function $f(\lambda)$ admits the
representation (28), (29).

First, denote by $H_{2}$ the set of all analytic in the unit disk
$D=\\{z:|z|<1\\}$ functions $f(z)$ such that

$|f(z)|^{2}=\lim_{r\uparrow
1}\int_{-\pi}^{\pi}|f(re^{i\theta})|^{2}d\theta<\infty.$

For the function

$f(z)=\sum_{n=0}^{\infty}a_{n}z^{n}$

we have

$f(re^{i\theta})=\sum_{n=0}^{\infty}a_{n}r^{n}e^{in\theta}.$

It means that $a_{n}r^{n}$ are the Fourier coefficients of the function
$f(re^{i\theta})$. From the Parseval equality it follows that

$\int_{-\pi}^{\pi}|f(re^{i\theta})|^{2}d\theta=2\pi\sum_{n=0}^{\infty}|a_{n}|^{2}r^{2n}.$

We can conclude that $f(z)\in H_{2}$ if and only if

$\sum_{n=0}^{\infty}|a_{n}|^{2}<\infty.$

For any function $f(z)\in H_{2}$ we can determine a series

$f(e^{i\theta})=\sum_{n=0}^{\infty}a_{n}e^{in\theta},$

which converges in $L_{2}[-\pi,\pi]$. The function $f(z),|z|<1$, restores from
the function $f(e^{i\theta})$ with the help of the Poisson formula

$f(re^{i\theta})=\frac{1}{2\pi}\int_{-\pi}^{\pi}f(e^{iu})P(r,\theta,u)du,$

where

$P(r,\theta,u)=\frac{1-r^{2}}{1-2r\cos(\theta-u)+r^{2}}=\sum_{n=-\infty}^{\infty}r^{|n|}e^{in(\theta-u)}.$

######  Theorem 5.

Let $f(\lambda)$ be a nonnegative and integrable with respect to the Lebesgue
measure on $[-\pi,\pi)$ function. In order that a function $g(z)\in H_{2}$
exists such that

$f(\lambda)=|g(e^{i\lambda})|^{2},$

it is necessary and sufficient that the following condition holds

$\int_{-\pi}^{\pi}|\ln{f(\lambda)}|d\lambda<\infty.$ (32)

###### Proof.

Let condition (32) holds true. Then the function

$v(r,\theta)=\frac{1}{2\pi}\int_{-\pi}^{\pi}\ln{f(\lambda)}P(r,\theta,\lambda)d\lambda$

is harmonic in the unit disk $D=\\{z:|z|<1\\}$. From the Jensen inequality it
follows that

$v(r,\theta)\leq\ln\left\\{\frac{1}{2\pi}\int_{-\pi}^{\pi}{f(\lambda)}P(r,\theta,\lambda)d\lambda\right\\}.$

Denote by $\varphi(z)$ the analytic in the unit disk $D=\\{z:|z|<1\\}$
function with the real part $v(r,\theta)$. Consider the function
$g(z)=e^{\varphi(z)/2}$. For this function

$|g(re^{i\theta})|^{2}=e^{\text{Re}\varphi(z)}=e^{v(r,\theta)}\leq\frac{1}{2\pi}\int_{-\pi}^{\pi}{f(\lambda)}P(r,\theta,\lambda)d\lambda,$
$\int_{-\pi}^{\pi}|g(re^{i\theta})|^{2}d\theta\leq\int_{-\pi}^{\pi}f(\lambda)d\lambda.$

So, the function $g(z)\in H_{2}$ and

$\lim_{r\uparrow 1}|g(re^{i\theta})|^{2}=e^{\lim_{r\uparrow
1}v(r,\theta)}=f(\theta)$

almost everywhere. ∎

######  Corrolary 2.3.

In order that a stochastic sequence $\eta=\\{\eta(n),n\in\mathbb{Z}\\}$ admits
the representation as one-sided moving average sequence

$\eta(n)=\sum\limits_{m=0}^{\infty}{a(m)\varepsilon(n-m)},\quad\sum_{m=0}^{\infty}|a(m)|^{2}<\infty.$

where $\varepsilon=\\{\varepsilon(n),n\in\mathbb{Z}\\}$ is a white noise
sequence, it is necessary and sufficient that the sequence
$\eta=\\{\eta(n),n\in\mathbb{Z}\\}$ has an absolutely continuous spectral
measure and its spectral density $f(\lambda)$ satisfies the condition

$\int_{-\pi}^{\pi}\ln{f(\lambda)}d\lambda>-\infty.$ (33)

Determined in the theorem analytical in the unit disk $D=\\{z:|z|<1\\}$
function $\varphi(z)$ with the real part $v(r,\theta)$ has the boundary value
$\ln{f(\lambda)}$. By the Schwartz formula we have

$\varphi(z)=\frac{1}{2\pi}\int_{-\pi}^{\pi}\ln{f(\lambda)}\frac{e^{i\lambda}+z}{e^{i\lambda}-z}d\lambda.$
(34)

It follows from the decomposition of the function
$g(z)=\exp{\\{\frac{\varphi(z)}{2}\\}}$ into a power series
$g(z)=\sum_{n=0}^{\infty}b_{n}z^{n}$ that the coefficients
$a_{n}=\sqrt{2\pi}b_{n}$. From the other hand, the function $g(z)$ can be
represented in a different form. Since

$\frac{e^{i\lambda}+z}{e^{i\lambda}-z}=1+\frac{2ze^{-i\lambda}}{1-ze^{-i\lambda}}=1+2\sum\limits_{k=1}^{\infty}z^{k}e^{-ik\lambda},$

we have

$\overline{g(z)}=\exp\left\\{\frac{1}{4\pi}\int_{-\pi}^{\pi}\ln{f(\lambda)}d\lambda+\frac{1}{2\pi}\sum\limits_{k=1}^{\infty}d_{k}\overline{z}^{k}\right\\},$
$d_{k}=\int_{-\pi}^{\pi}e^{ik\lambda}\ln{f(\lambda)}d\lambda.$

If we introduce the notation

$P=\exp\left\\{\frac{1}{4\pi}\int_{-\pi}^{\pi}\ln{f(\lambda)}d\lambda\right\\},$
$\exp\left\\{\frac{1}{2\pi}\sum\limits_{k=1}^{\infty}d_{k}{z}^{k}\right\\}=\sum\limits_{n=0}^{\infty}c_{n}{z}^{n},\quad
c_{0}=1,$

we will have

$\overline{g(z)}=P\sum\limits_{n=0}^{\infty}c_{n}\overline{z}^{n}.$

So

$a_{n}=\sqrt{2\pi}Pc_{n}.$

###  2.4 Wold expansion of stationary sequences

In contrast to the representation (16) which gives an expansion of a
stationary sequence in the frequency domain, the Wold [78, 78] expansion
operates in the time domain. The main point of this expansion is that a
stationary sequence $\xi(n)$ can be represented as a sum of two stationary
sequences, one of which is completely predictable (in the sense that its
values are completely determined by its ”past”), whereas the second does not
have this property.

First we give some definitions. Let $H(\xi)$ and $H_{n}(\xi)$ be closed linear
manifolds, spanned by all values of the stationary sequence
$\\{\xi(k),k\in\mathbb{Z}\\}$ and values $\\{\xi(k),k=n,n-1,n-2,\dots\\}$
respectively. Let

$S(\xi)={\mathop{\cap}\limits_{n}}H_{n}(\xi).$

For every element $\eta\in H(\xi)$ denote by

$\hat{\eta}_{n}=\text{Proj}\,(\eta|H_{n}(\xi))$

the projection of the element $\eta$ on the subspace $H_{n}(\xi)$. Denote also

$\hat{\eta}_{S}=\text{Proj}\,(\eta|S(\xi)),$

Every element $\eta\in H(\xi)$ can be represented in the form

$\eta=\hat{\eta}_{S}+(\eta-\hat{\eta}_{S}),$

where $\eta-\hat{\eta}_{S}\bot\hat{\eta}_{S}$. Therefore the space $H(\xi)$
can be represented as the orthogonal sum

$H(\xi)=S(\xi)\oplus R(\xi),$

where $S(\xi)$ consists of the elements $\hat{\eta}_{S}$ with $\eta\in
H(\xi)$, and $R(\xi)$ consists of the elements of the form
$\eta-\hat{\eta}_{S}$.

######  Definition 2.3.

A stationary sequence $\xi(n)$ is called regular, if

$H(\xi)=R(\xi),$

and singular, if

$H(\xi)=S(\xi).$

######  Remark 2.4.

Singular sequence is also called deterministic, and regular sequence is called
purely or completely nondeterministic. If $S(\xi)$ is a proper subspace of the
space $H(\xi)$, then the sequence $\xi(n)$ is called nondeterministic.

######  Theorem 6.

Every wide sense stationary random sequence $\xi(n)$ has a unique
decomposition

$\xi(n)=\xi^{r}(n)+\xi^{s}(n),$ (35)

where $\xi^{r}(n)$ is regular sequence and $\xi^{s}(n)$ is singular sequence.
The sequences $\xi^{r}(n)$ and nd $\xi^{s}(n)$ are orthogonal.

###### Proof.

Define

$\xi^{s}(n)=\text{Proj}\,(\xi(n)|S(\xi)),\quad\xi^{r}(n)=\xi(n)-\xi^{s}(n).$

Since $\xi^{r}(n)\bot S(\xi)$ for every $n$, we have $S(\xi^{r})\bot S(\xi)$.
On the other hand, $S(\xi^{r})\subseteq S(\xi)$ and therefore $S(\xi^{r})$ is
trivial (contains only random sequences that coincide almost surely with
zero). Consequently $\xi^{r}(n)$ is regular.

Moreover, $H_{n}(\xi)\subseteq H_{n}(\xi^{s})\oplus H_{n}(\xi^{r})$ and
$H_{n}(\xi^{s})\subseteq H_{n}(\xi)$, $H_{n}(\xi^{r})\subseteq H_{n}(\xi)$.
Therefore $H_{n}(\xi)=H_{n}(\xi^{s})\oplus H_{n}(\xi^{r})$, and hence for
every $n$

$S(\xi)\subseteq H_{n}(\xi^{s})\oplus H_{n}(\xi^{r}).$ (36)

Since $\xi_{n}^{r}\bot S(\xi)$, it follows from (36) that

$S(\xi)\subseteq H_{n}(\xi^{s}),$

and therefore $S(\xi)\subseteq S(\xi^{s})\subseteq H(\xi^{s})$. But
$\xi_{n}^{s}\in S(\xi)$, hence $H(\xi^{s})\subseteq S(\xi)$, and consequently

$S(\xi)=S(\xi^{s})=H(\xi^{s}),$

which means that $\xi^{s}(n)$ is singular.

The orthogonality of $\xi^{s}(n)$ and $\xi^{r}(n)$ follows in an obvious way
from $\xi^{s}(n)\in S(\xi)$ and $\xi^{r}(n)\bot S(\xi)$. ∎

######  Definition 2.4.

Let $\xi(n)$ be a nondegenerate stationary sequence. A random sequence
$\varepsilon(n)$ is called the innovation sequence for $\xi(n)$, if the
following conditions holds true:

1) $\varepsilon(n)$ consists of pairwise orthogonal random variables with
$E\varepsilon(n)=0$, $E{\left|{\varepsilon(n)}\right|}^{2}=1$;

2) $H_{n}(\xi)=H_{n}(\varepsilon)$ for all $n\in\mathbb{Z}$.

######  Remark 2.5.

The reason for the term “innovation” is that $\varepsilon(n+1)$ provides new
”information”, not contained in $H_{n}(\xi)$, that is needed for forming
$H_{n+1}(\xi)$.

The following fundamental theorem establishes a connection between one-sided
moving average sequences and regular sequences.

######  Theorem 7.

The necessary and sufficient condition for a nondegenerate stationary sequence
$\xi(n)$ to be regular is that there exist an innovation sequence
$\varepsilon=\\{\varepsilon(n)\\}$ and a sequence of complex numbers
$\\{a(n),n\geq 0\\}$, with
${\sum\limits_{n=0}^{\infty}{{\left|{a(n)}\right|}^{2}}}<\infty,$ such that

$\xi(n)={\sum\limits_{k=0}^{\infty}{a(k)\varepsilon(n-k)}}.$ (37)

###### Proof.

Necessity. Represent $H_{n}(\xi)$ in the form

$H_{n}(\xi)=H_{n-1}(\xi)\oplus B_{n}(\xi),$

where $B_{n}(\xi)$ is the space of random variables of the form
$\beta\cdot\xi(n)$, where $\beta$ is a complex number. Since $H_{n}(\xi)$ is
generated by elements from $H_{n-1}(\xi)$ and elements of the form
$\beta\cdot\xi(n)$, the dimension (dim) of the space $B_{n}(\xi)$ is either
zero or one. But the space $H_{n}(\xi)$ cannot coincide with $H_{n-1}(\xi)$
for any value of $n$. In fact, if for some $n$ the space $B_{n}(\xi)$ is
trivial, then by stationarity $B_{n}(\xi)$ is trivial for all $n$, and
therefore $H(\xi)=S(\xi)$, contradicting the assumption that the sequence
$\xi(n)$ is regular. Thus the space $B_{n}(\xi)$ has the dimension $1$. Let
$\eta(n)$ be a nonzero element of $B_{n}(\xi)$. Take

$\varepsilon_{n}={\frac{{\eta(n)}}{{{\left\|{\eta(n)}\right\|}}}},$

where ${\left\|{\eta(n)}\right\|}^{2}=E{\left|{\eta(n)}\right|}^{2}>0$. For
fixed $n$ and $k\geq 0$ consider the decomposition

$H_{n}(\xi)=H_{n-k}(\xi)\oplus B_{n-k+1}(\xi)\oplus\cdots\oplus B_{n}(\xi).$

The elements $\varepsilon(n-k),\dots,\varepsilon(n)$ is an orthogonal basis in
$B_{n-k+1}(\xi)\oplus\cdots\oplus B_{n}(\xi)$ and

$\xi(n)=\sum\limits_{j=0}^{k-1}a(j)\varepsilon(n-j)+\text{Proj}\,({\xi}(n)|H_{n-k}(\xi)),\quad
a(j)=E\xi(n)\overline{\varepsilon(n-j)}.$ (38)

By the Bessel inequality

${\sum\limits_{j=0}^{\infty}{{\left|{a(j)}\right|}^{2}}}\leq{\left\|{\xi(n)}\right\|}^{2}<\infty.$

The series

${\sum_{j=0}^{\infty}{a(j)\varepsilon(n-j)}}$

converges in the mean square, and then, by (38), relation (37) will be
established as soon as we show that

$\text{Proj}\,({\xi}(n)|H_{n-k}(\xi)){\buildrel{L^{2}}\over{\longrightarrow}}0,\quad
k\to\infty.$

Consider the case $n=0$. Denote
$\hat{\xi}_{i}=\text{Proj}\,({\xi}(0)|H_{i}(\xi))$. Since

$\hat{\xi}_{-k}=\hat{\xi}_{0}+{\sum\limits_{i=1}^{k}{[\hat{\xi}_{-i}-\hat{\xi}_{-i+1}]}},$

and the terms that appear in this sum are orthogonal, we have for every $k\geq
0$

${\sum\limits_{i=1}^{k}{{\left\|{\hat{\xi}_{-i}-\hat{\xi}_{-i+1}}\right\|}^{2}}}={\left\|{{\sum\limits_{i=1}^{k}{(\hat{\xi}_{-i}-\hat{\xi}_{-i+1})}}}\right\|}^{2}=$
$={\left\|{\hat{\xi}_{-k}-\hat{\xi}_{0}}\right\|}^{2}\leq
4{\left\|{\xi(0)}\right\|}^{2}<\infty.$

Therefore the limit ${\mathop{\lim}\limits_{k\to\infty}}\hat{\xi}_{-k}$ exists
(in the mean square sense). For each $k$ the value $\hat{\xi}_{-k}\in
H_{-k}(\xi)$, and therefore the limit in question must belong to the space
${\mathop{\cap}\limits_{k\geq 0}}H_{-k}(\xi)=S(\xi)$. But, by assumption,
$S(\xi)$ is trivial, and therefore

$\text{Proj}\,({\xi}(n)|H_{n-k}(\xi)){\buildrel{L^{2}}\over{\longrightarrow}}0,\quad
k\to\infty.$

Sufficiency. Let the nondegenerate stationary sequence $\xi(n)$ have a
representation (37), where $\varepsilon=\\{\varepsilon(n)\\}$ is an
orthonormal system (not necessarily satisfying the condition
$H_{n}(\xi)=H_{n}(\varepsilon)$, $n\in\mathbb{Z})$. Then $H_{n}(\xi)\subseteq
H_{n}(\varepsilon)$ and therefore
$S(\xi)={\mathop{\cap}\limits_{k}}H_{k}(\xi)\subseteq H_{n}(\varepsilon)$ for
every $n$. But $\varepsilon_{n+1}\bot H_{n}(\varepsilon)$, and therefore
$\varepsilon_{n+1}\bot S(\xi)$ and at the same time
$\varepsilon=\\{\varepsilon(n)\\}$ is a basis in $H(\xi)$. lt follows that
$S(\xi)$ is trivial, and consequently $\xi(n)$ is regular. This completes the
proof of the theorem. ∎

######  Remark 2.6.

It follows from the proof of the theorem 7 that a nondegenerate sequence
$\xi(n)$ is regular if and only if it admits a representation as one-sided
moving average

$\xi(n)={\sum\limits_{k=0}^{\infty}{\tilde{a}_{k}\tilde{\varepsilon}_{n-k}}}$
(39)

where $\tilde{\varepsilon}=\\{\tilde{\varepsilon}(n)\\}$ is an orthonormal
system which (it is important to emphasize this !) does not necessarily
satisfy the condition $H_{n}(\xi)=H_{n}(\tilde{\varepsilon})$,
$n\in\mathbb{Z}$. In this sense the conclusion of the theorem 7 says more, and
specifically that for a regular stationary sequence $\xi(n)$ there exist a
sequence of numbers $a=\\{a(n)\\}$ and an orthonormal system of random
variables $\varepsilon=\\{{\varepsilon}(n)\\}$, such that not only (39), but
also (37), is satisfied, with $H_{n}(\xi)=H_{n}({\varepsilon})$,
$n\in\mathbb{Z}$.

######  Theorem 8.

Wold expansion. A nondegenerate stationary sequence $\xi=\\{\xi(n)\\}$ can be
represented in the form

$\xi(n)=\xi^{s}(n)+{\sum\limits_{k=0}^{\infty}{a(k)\varepsilon(n-k)}},$ (40)

where

${\sum_{k=0}^{\infty}{{\left|{a(k)}\right|}^{2}}}<\infty,$

and $\varepsilon=\\{\varepsilon(n)\\}$ is the innovation sequence (for
$\xi^{r}(n))$.

The significance of the concepts of regular and singular sequences becomes
clear if we consider the following (linear) extrapolation problem, for whose
solution the Wold expansion (40) is especially useful.

Let $H_{0}(\xi)$ be the closed linear manifold spanned by values of the
stationary sequence $\\{\xi(k),k=0,-1,-2,\dots\\}$. Consider the problem of
constructing an optimal (least-squares) linear estimator $\hat{\xi}(n)$ of the
value $\xi(n)$ of the sequence at point $n>0$ based on observations of the
“past” $\\{\xi(k),k=0,-1,-2,\dots\\}$. The estimate $\hat{\xi}(n)$ is a
projection of $\xi(n)$ on the manifold $H_{0}(\xi)$:

$\hat{\xi}(n)=\text{Proj}\,({\xi}(n)|H_{0}(\xi)).$ (41)

Since the sequences $\xi^{s}(n)$ and $\xi^{r}(n)$ are orthogonal and
$H_{0}(\xi)\subseteq H_{0}(\xi^{s})\oplus H_{0}(\xi^{r})$, we obtain, by using
(40)

$\begin{array}[]{rl}\hat{\xi}(n)=&\text{Proj}\,(\xi^{s}(n)+\xi^{r}(n)|H_{0}(\xi))=\text{Proj}\,(\xi^{s}(n)|H_{0}(\xi))+\text{Proj}\,(\xi^{r}(n)|H_{0}(\xi))=\\\
=&\text{Proj}\,(\xi^{s}(n)|H_{0}(\xi^{s})\oplus
H_{0}(\xi^{r}))+\text{Proj}\,(\xi_{n}^{r}|H_{0}(\xi^{s})\oplus
H_{0}(\xi^{r}))=\\\
=&\text{Proj}\,(\xi^{s}(n)|H_{0}(\xi^{s}))+\text{Proj}\,(\xi^{r}(n)|H_{0}(\xi^{r}))=\\\
=&\xi^{s}(n)+\text{Proj}\,({\sum\limits_{k=0}^{\infty}{a(k)\varepsilon(n-k)}}|H_{0}(\xi^{r})).\\\
\end{array}$

In (40) the sequence $\varepsilon=\\{\varepsilon(n)\\}$ is an innovation
sequence for $\xi^{r}=\\{\xi^{r}(n)\\}$ and
$H_{0}(\xi^{r})=H_{0}(\varepsilon)$. Therefore

$\hat{\xi}(n)=\xi^{s}(n)+\text{Proj}\,\left({\sum\limits_{k=0}^{\infty}{a(k)\varepsilon(n-k)}}|H_{0}(\varepsilon)\right)=\xi^{s}(n)+{\sum\limits_{k=n}^{\infty}{a(k)\varepsilon(n-k)}}$
(42)

and the mean-square error of prediction $\xi(n)$ based on observations
$\\{\xi(k),k=0,-1,-2,\dots\\}$ equals

$\sigma_{n}^{2}=E{\left|{\xi(n)-\hat{\xi}(n)}\right|}^{2}={\sum\limits_{k=0}^{n-1}{{\left|{a(k)}\right|}^{2}}}.$
(43)

Two important conclusions follows from the presented results.

1) If the sequence $\xi(n)$ is singular, then for every $n\geq 1$ the error of
the extrapolation $\sigma_{n}^{2}$ is zero; in other words, we can predict
$\xi(n)$ without error from its ”past” $\\{\xi(k),k=0,-1,-2,\dots\\}$.

2) If the sequence $\xi(n)$ is regular, then
$\sigma_{n}^{2}\leq\sigma_{n+1}^{2}$ and

${\mathop{\lim}\limits_{n\to\infty}}\sigma_{n}^{2}={\sum\limits_{k=0}^{\infty}{{\left|{a(k)}\right|}^{2}}}.$
(44)

Since

${\sum\limits_{k=0}^{\infty}{{\left|{a(k)}\right|}^{2}}}=E{\left|{\xi(n)}\right|}^{2},$

it follows from (44) and (43) that if $n$ increases, the prediction of
$\xi(n)$ based on observations $\\{\xi(k),k=0,-1,-2,\dots\\}$ becomes trivial
(reducing simply to $E\xi(n)=0)$.

###  2.5 Conclusions

In this section we describe properties of stationary stochastic sequence
illustrated with the help of some examples. The spectral representation of
stationary stochastic sequences and their correlation functions are described.
Properties of linear transformations of stationary stochastic sequences are
described. The Wold representation of stationary stochastic sequences is
analysed. Application to the problem of prediction of stationary sequences is
described.

For the detailed exposition of results of the theory of stationary stochastic
sequences see books by Gikhman and Skorokhod [12], Hannan[17], Rozanov[73],
Yaglom [80, 81].

##  3 Estimates for functionals of stationary sequences

In this section we deal with the problem of the mean square optimal linear
estimation of the functionals

$A_{N}{\xi}=\sum_{j=0}^{N}{a}(j){\xi}(j),$
$A{\xi}=\sum_{j=0}^{\infty}{a}(j){\xi}(j)$

which depend on the unknown values of a stationary stochastic sequence
${\xi}(j)$ from the class $\Xi$ of stationary stochastic sequences satisfying
the conditions

$E{\xi}(j)={0},\quad E|{\xi}(j)|^{2}\leq P.$ (45)

Estimates are based on results of observations of the sequence ${\xi}(j)$ at
points of time $j=-1,-2,\dots$.

Following the Ulf Grenander [15] approach to investigation the problem of
optimal linear estimation of the functional $A_{N}{\xi}$ (as well estimation
of the functional $A{\xi}$) we consider the problem as a two-person zero-sum
game in which the first player chooses a stationary stochastic sequence
$\xi(j)$ from the class $\Xi$ of stationary stochastic sequences such that the
value of the mean square error of estimate of the functional attains its
maximum. The second player is looking for an estimate of the linear functional
which minimizes the value of the mean square error. It is show that this game
has equilibrium point. The maximum error gives a moving average stationary
sequence which is least favourable in the given class of stationary sequences.
The greatest value of the error and the least favourable sequence are
determined by the largest eigenvalue and the corresponding eigenvector of the
operator determined by the coefficients $a(j)$ which determine the
functionals.

###  3.1 The maximum value of the mean-square error of estimate of the
functional $A_{N}{\xi}$

In this subsection the maximum value of the mean-square error of the optimal
linear estimate of the functional $A_{N}{\xi}$ which depend on the unknown
values of a stationary stochastic sequence ${\xi}(j)$ from the class $\Xi$ is
found.

Let $\Delta(\xi,\hat{A}_{N})=E\left|A_{N}{\xi}-\hat{A}_{N}{\xi}\right|^{2}$
denotes the mean-square error of an estimate $\hat{A}_{N}{\xi}$ of the
functional $A_{N}{\xi}$. Denote by $\Lambda$ the class of all linear estimates
of the functional $A_{N}{\xi}$.

######  Theorem 1.

The function $\Delta(\xi,\hat{A}_{N})$ has a saddle point on the set
$\Xi\times\Lambda$. The following relation holds true

$\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda}\mathop{\max}\limits_{\xi\in\Xi}\Delta(\xi,\hat{A}_{N})=\mathop{\max}\limits_{\xi\in\Xi}\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda}\Delta(\xi,\hat{A}_{N})=P\nu_{N}^{2}.$

The least favourable in the class $\Xi$ of stationary stochastic sequences
satisfying conditions (45) for the optimal linear estimation of the functional
$A_{N}{\xi}$ is one-sided moving average sequence of order $N$, which is
determined by the formula

${\xi}(j)=\sum_{u=j-N}^{j}\varphi(j-u){\varepsilon}(u).$

Here $\nu_{N}^{2}$ is the largest eigenvalue of the selfadjoint compact
operator $Q_{N}$ in the space ${\mathbb{C}}^{(N+1)}$, determined by the matrix
$Q_{N}=\left\\{Q_{N}(p,q)\right\\}_{p,q=0}^{N}$ that is constructed with the
elements

$Q_{N}(p,q)=\sum_{u=0}^{\min(N-p,N-q)}{a}(p+u)\,\overline{a}(q+u),\quad
p,q=0,1,\dots,N,$

${\varepsilon}(u)$ is a stationary stochastic sequence with orthogonal values:

$E{\varepsilon}(i)\overline{\varepsilon}(j)=\delta_{ij},$

where $\delta_{ij}$ is the Kronecker symbol; $\\{\varphi(u),u={0,\dots,N}\\}$
is determined by the eigenvector that corresponds to $\nu_{N}^{2}$, and the
condition $\left\|{\xi}(j)\right\|^{2}=P.$

Proof. Lower bound. Denote by $\Xi_{R}$ the class of all regular stationary
sequences which satisfy conditions (45). Since $\Xi_{R}\subset\Xi$, we have

$\mathop{\max}\limits_{\xi\in\Xi}\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda}\Delta(\xi,\hat{A}_{N})\geq\mathop{\max}\limits_{\xi\in\Xi_{R}}\mathop{\min}\limits_{\hat{A}_{N}\in\Xi_{R}}\Delta(\xi,\hat{A}_{N}).$
(46)

A regular stationary sequence ${\xi}(j)$ admits the canonical representation
as an one-sided moving average sequence

${\xi}(j)=\sum_{u=-\infty}^{j}\varphi(j-u){\varepsilon}(u),$ (47)

where ${\varepsilon}(u)$ is a standard stationary stochastic sequence with
orthogonal values, $\\{\varphi(u):u=0,1,\ldots\\}$ are coefficients of the
canonical representation (47). The sequence ${\xi}(j)\in\Xi_{R}$ is determined
if there are determined coefficients
$\left\\{\varphi(u):u=0,1,\ldots\right\\}$ such that

$E|{\xi}(j)|^{2}=\left|\sum_{u=-\infty}^{j}\varphi(j-u)\varepsilon(u)\right|^{2}=$
$=\sum_{u,v=-\infty}^{j}\varphi(j-u)\overline{\varphi(j-v)}E\varepsilon(u)\overline{\varepsilon(v)}=$
$=\sum_{u=-\infty}^{j}\left|\varphi(j-u)\right|^{2}=\sum_{u=0}^{\infty}\left|\varphi(u)\right|^{2}\leq
P.$ (48)

The value of the mean-square error
$E\left|A_{N}{\xi}-\hat{A}_{N}{\xi}\right|^{2}$ attains its minimum if we
choose an estimate $\hat{A}_{N}{\xi}$ of the form

$\hat{A}_{N}{\xi}=\sum_{j=0}^{N}{a}(j)\hat{\xi}(j),$

where $\hat{{\xi}}(j)$ is the optimal estimate of the value of ${\xi}(j)$
based on observations of the sequence ${\xi}(p)$ at points $p=-1,-2,\dots$.
Taking into consideration the canonical representation (47) of the regular
sequence and the form of the optimal estimates of its values

$\hat{{\xi}}(j)=\sum_{u=-\infty}^{-1}\varphi(j-u){\varepsilon}(u),$ (49)

we can write

$\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda}E\left|A_{N}{\xi}-\hat{A}_{N}{\xi}\right|^{2}=$
$=\sum_{i,j=0}^{N}a(i)\overline{a(j)}\sum_{u=0}^{i}\sum_{v=0}^{j}\;\varphi(i-u)\overline{\varphi(j-v)}\,E\varepsilon(u)\overline{\varepsilon(v)}$
$=\sum_{i,j=0}^{N}\;a(i)\overline{a(j)}\;\sum_{u=0}^{\min(i,j)}\varphi(i-u)\overline{\varphi(j-u)}=$
$=\sum_{i,j=0}^{N}a(i)\overline{a(j)}R(i,j),$ (50)

where

$R(i,j)=\sum_{u=0}^{\min(i,j)}\varphi(i-u)\overline{\varphi(j-u)}.$

The change of variables $p=i-u,\;q=j-u$, gives us a possibility to write (50)
in a different form

$\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda}E\left|A_{N}{\xi}-\hat{A}_{N}{\xi}\right|^{2}=\sum_{p,q=0}^{N}\varphi(p)\overline{\varphi(q)}\;Q_{N}(p,q),$
(51)

where

$Q_{N}(p,q)=\sum_{u=0}^{\min(N-p,N-q)}a(p+u)\overline{a(q+u)}.$ (52)

Denote by $Q_{N}$ the operator in the space ${\mathbb{C}}^{(N+1)}$, determined
by the matrix $\left\\{Q_{N}(p,q)\right\\}_{p,q=0}^{N}$. The operator $Q_{N}$
is selfadjoint (its matrix is Hermitian) bounded operator. It can be
represented in the form $Q_{N}=A_{N}\cdot A_{N}^{*}$, where the operator
$A_{N}$ is determined by the matrix $\left\\{A_{N}(p,q)\right\\}_{p,q=0}^{N}$
with

$A_{N}(p,q)=\left\\{\begin{array}[]{cc}{a}(p+q),&\;p+q\leq N,\\\
{0},&\;p+q>N.\end{array}\right.$

For this reason the operator $Q_{N}$ has nonnegative eigenvalues [1], [71]. It
follows from (51), that $\varphi(p)=0$ for $p>N$. Denote by

$\vec{{\varphi}}=\\{\tilde{\varphi}(0),\tilde{\varphi}(1),\dots,\tilde{\varphi}(N)\\},\quad\tilde{\varphi}(p)=P^{-1/2}\varphi(p).$

Condition (48) with this notation is of the form

$\left\|\vec{\varphi}\right\|^{2}=\sum_{p=0}^{N}|\tilde{\varphi}(p)|^{2}\leq
1,$ (53)

where $\left\|\vec{\varphi}\right\|$ is the norm in the space
${\mathbb{C}}^{(N+1)}$. Taking into consideration (51) and (53), we can write

$\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda}\Delta(\xi,\hat{A}_{N})=P\left\langle
Q_{N}\vec{\varphi},\vec{\varphi}\right\rangle,$

where $\left\langle\cdot,\cdot\right\rangle$ is the inner product in the space
${\mathbb{C}}^{(N+1)}$.

Taking into account (46), we get the following bound from below for the
maximum value of the error [8], [14], [71]

$\mathop{\max}\limits_{\xi\in\Xi}\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda}\Delta(\xi,\hat{A}_{N})\geq
P\mathop{\max}\limits_{\left\|\vec{\varphi}\right\|\leq 1}\left\langle
Q_{N}\vec{\varphi},\vec{\varphi}\right\rangle=P\nu_{N}^{2},$ (54)

where $\nu_{N}^{2}$ is the greatest eigenvalue of the operator $Q_{N}$.

Upper bound. To find the upper bound of the minimax values of the error we use
the inequality

$\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda}\mathop{\max}\limits_{\xi\in\Xi}\Delta(\xi,\hat{A}_{N})\leq\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda_{1}}\mathop{\max}\limits_{\xi\in\Xi}\Delta(\xi,\hat{A}_{N}),$
(55)

where $\Lambda_{1}$ is the class of all linear estimates of the functional
$A_{N}{\xi}$, which have the form

$\hat{A}_{N}{\xi}=\sum_{j=-\infty}^{-1}{c}(j){\xi}(j).$ (56)

Here ${c}(j)$ are complex-valued coefficients such that

$\sum_{j=-\infty}^{-1}|{c}(j)|^{2}<\infty.$

Taking into consideration the spectral representations of the stationary
stochastic sequence $\xi(j)$ and the correlation function of the sequence
$\xi(j)$ , we can write

$\Delta(\xi,\hat{A}_{N})=E\left|A_{N}{\xi}-\hat{A}_{N}{\xi}\right|^{2}=E\left|\sum_{j=0}^{N}{a}(j){\xi}(j)-\sum_{j=-\infty}^{-1}{c}(j){\xi}(j)\right|^{2}=$
$=\int_{-\pi}^{\pi}|A_{N}(e^{i\lambda})-C(e^{i\lambda})|^{2}F_{\xi}(d\lambda),$

where

$A_{N}(e^{i\lambda})=\sum_{j=0}^{N}{a}(j)e^{ij\lambda},\quad
C(e^{i\lambda})=\sum_{j=-\infty}^{-1}{c}(j)e^{ij\lambda}.$

Here $F_{\xi}(d\lambda)$ is the spectral measure of the stationary stochastic
sequence. Restriction (45) is equivalent to the restriction

$\mathop{\max}\limits_{\xi\in\Xi}\int_{-\pi}^{\pi}\,F_{\xi}(d\lambda)\leq P.$
(57)

So we can write

$\mathop{\max}\limits_{\xi\in\Xi}\Delta(\xi,\hat{A}_{N})=\mathop{\max}\limits_{\xi\in\Xi}\int_{-\pi}^{\pi}|A_{N}(e^{i\lambda})-C(e^{i\lambda})|^{2}F_{\xi}(d\lambda)\leq$
$\leq\left(\mathop{\max}\limits_{\lambda\in[-\pi,\pi]}|A_{N}(e^{i\lambda})-C(e^{i\lambda})|^{2}\right)\mathop{\max}\limits_{\xi\in\Xi}\int_{-\pi}^{\pi}F_{\xi}(d\lambda)$
$\leq
P\mathop{\max}\limits_{\lambda\in[-\pi,\pi]}|A_{N}(e^{i\lambda})-C(e^{i\lambda})|^{2}.$

To calculate

$\mathop{\max}\limits_{\lambda\in[-\pi,\pi]}|A_{N}(e^{i\lambda})-C(e^{i\lambda})|^{2}$

consider the class of all power series

${f}(z)=\sum_{n=0}^{\infty}{\alpha}(n)z^{n},$

which are regular in the region $\left|z\right|<1$ and have fixed first $N+1$
coefficients

${\alpha}(n)={d}(n),\;n=0,1,\dots,N.$

Denote by $\rho_{N}^{2}$ the greatest eigenvalue of the matrix

$H=\left\\{H(p,q)\right\\}_{p,q=0}^{N},$
$H(p,q)=\sum_{j=0}^{\min(p,q)}{d}(p-j)\overline{d}(q-j),\quad
p,q=0,1,\dots,N.$

It follows from the properties of the power series, that [16]

$\mathop{\min}\limits_{\left\\{{\alpha}(n):n\geq
N+1\right\\}}\mathop{\max}\limits_{\left|z\right|=1}\left|{f}(z)\right|^{2}=\rho_{N}^{2}.$

Since in our case

${d}(p)={a}(N-p),\;p=0,1,\dots,N,$

we have to determine the greatest eigenvalue of the matrix

$G_{N}=\left\\{G_{N}(p,q)\right\\}_{p,q=0}^{N},$
$G_{N}(p,q)=\sum_{u=0}^{\min(p,q)}{a}(N-p+u)\overline{a}(N-q+u).$

Denote this greatest eigenvalue by $\omega_{N}^{2}$. With this notations we
have

$\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda_{1}}\mathop{\max}\limits_{\xi\in\Xi}\Delta(\xi,\hat{A}_{N})\leq
P\omega_{N}^{2}.$

Taking into account (55), we get

$\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda}\mathop{\max}\limits_{\xi\in\Xi}\Delta(\xi,\hat{A}_{N})\leq
P\omega_{N}^{2}.$ (58)

Note, that

$G_{N}(N-p,N-q)=Q_{N}(p,q).$

For this reason $\omega_{N}^{2}=\nu_{N}^{2}$. Comparing (54) and (58), we get

$\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda}\mathop{\max}\limits_{\xi\in\Xi}\Delta(\xi,\hat{A}_{N})\leq\mathop{\max}\limits_{\xi\in\Xi}\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda}\Delta(\xi,\hat{A}_{N}).$
(59)

Since the opposite inequality always holds true, only equality is possible in
(59). Proof is complete.

From the proof of the theorem a construction of the optimal minimax estimate
follows.

######  Corollary 1.

The optimal minimax estimate $\hat{A}_{N}{\xi}$ of the functional $A_{N}{\xi}$
is of the form

$\hat{A}_{N}{\xi}=\sum_{j=0}^{N}{a}(j)\left(\sum_{u=j-N}^{-1}\varphi(j-u){\varepsilon}(u)\right),$

where ${\varepsilon}(u)$ is a stationary sequence with orthogonal values, the
sequence $\\{\varphi(u),u=0,1,\dots,N\\}$ is uniquely determined by
coordinates of the eigenvector of the operator $Q_{N}$ that corresponds to the
greatest eigenvalue $\nu_{N}^{2}$ and condition
$E\left|{\xi}(j)\right|^{2}=P$.

######  Example 1.

Consider the problem of optimal linear stimulation of the functional

$A_{1}{\xi}=\xi(0)+\xi(1)$

that depends on the unknown values of a stationary sequence ${\xi}(j)$, that
satisfies the conditions

$E{\xi}(j)=0,\quad E\left|{\xi}(j)\right|^{2}\leq 1,$

based on observations of the sequence ${\xi}(j)$ at points $j=-1,-2,\dots$.

Eigenvalues of the operator $Q_{1}$, determined by equation (52), are equal to
$3\pm\sqrt{5}$. So the greatest eigenvalue is $\nu_{1}^{2}=3+\sqrt{5}$. The
eigenvector corresponding to the eigenvalue $\nu_{1}^{2}=3+\sqrt{5}$ is of the
form $\vec{\varphi}=\left\\{\varphi(0),\varphi(1)\right\\}$, where

$\varphi(0)=\sqrt{(5+\sqrt{5})/10},\quad\varphi(1)=\sqrt{(5-\sqrt{5})/10}.$

The least favourable stationary sequence ${\xi}(j)$ is a moving average
sequence of the form

${\xi}(j)=\varphi(0)\varepsilon(j)+\varphi(1)\varepsilon(j-1)=$
$=\sqrt{(5+\sqrt{5})/10}\,\,\varepsilon(j)+\sqrt{(5-\sqrt{5})/10}\,\,\varepsilon(j+1).$

The optimal linear minimax estimate $\hat{A}_{1}{\xi}$ of the functional
$A_{1}{\xi}$ is of the form

$\hat{A}_{1}{\xi}=\varphi(1)\,\,\varepsilon(-1)=\sqrt{(5-\sqrt{5})/10}\,\,\varepsilon(-1).$

The mean-square error of the optimal estimate of the functional $A_{1}{\xi}$
does not exceed $3+\sqrt{5}$.

###  3.2 The maximum value of the mean square error of estimate of the
functional $A{\xi}$

In this subsection the maximum value of the mean square error of the optimal
linear estimate of the functional

$A{\xi}=\sum_{j=0}^{\infty}{a}(j){\xi}(j)$

which depend on unknown values of a stationary stochastic sequence ${\xi}(j)$
from the class $\Xi$ is found.

We will suppose that the sequence $\\{{a}(j):j=0,1,\ldots\\}$ which determines
the functional $A{\xi}$ satisfies conditions

$\sum_{j=0}^{\infty}\left|a(j)\right|<\infty,\quad\sum_{j=0}^{\infty}(j+1)\left|{a}(j)\right|^{2}<\infty.$
(60)

######  Theorem 9.

The function $\Delta(\xi,\hat{A})=E\left|A{\xi}-\hat{A}{\xi}\right|^{2}$ has a
saddle point on the set $\Xi\times\Lambda$. The following relation holds true

$\mathop{\min}\limits_{\hat{A}\in\Lambda}\mathop{\max}\limits_{\xi\in\Xi}\Delta(\xi,\hat{A})=\mathop{\max}\limits_{\xi\in\Xi}\mathop{\min}\limits_{\hat{A}\in\Lambda}\Delta(\xi,\hat{A})=P\nu^{2}.$

The least favourable in the class $\Xi$ of stationary stochastic sequences
satisfying conditions (45) for the optimal linear estimation of the functional
$A{\xi}$ is a moving average sequence

${\xi}(j)=\sum_{u=-\infty}^{j}\varphi(j-u){\varepsilon}(u).$

Here $\nu^{2}$ is the greatest eigenvalue and
$\vec{\varphi}=\left\\{\varphi(u):u=0,1,\dots\right\\}$ is the corresponding
eigenvector of the selfadjoint compact operator in the space $\ell_{2}$
determined by the matrix

$Q=\left\\{Q(p,q)\right\\}_{p,q=0}^{\infty},\quad
Q(p,q)=\sum_{u=0}^{\infty}{a}(p+u)\overline{a}(q+u),$

${\varepsilon}(u)$ is a stationary sequence with orthogonal values.

Proof. Lower bound. Let $\xi\in\Xi_{R}$. In this case the following inequality
holds true

$\mathop{\max}\limits_{\xi\in\Xi}\mathop{\min}\limits_{\hat{A}\in\Lambda}\Delta(\xi,\hat{A})\geq\mathop{\max}\limits_{\xi\in\Xi_{R}}\mathop{\min}\limits_{\hat{A}\in\Lambda}\Delta(\xi,\hat{A}).$
(61)

Making use the canonical representation of the regular stationary sequence
(47) as a moving average sequence and form (49) of the optimal estimate we
have

$\mathop{\min}\limits_{\hat{A}\in\Lambda}\Delta(\xi,\hat{A})=\mathop{\min}\limits_{\hat{A}\in\Lambda}E\left|A{\xi}-\hat{A}{\xi}\right|^{2}=$
$=E\left|\sum_{j=0}^{\infty}{a}(j)\sum_{u=0}^{j}\varphi(j-u){\varepsilon}(u)\right|^{2}=$
$=\sum_{p,q=0}^{\infty}\;\varphi(p)\overline{\varphi(q)}Q(p,q),$ (62)

where

$Q(p,q)=\sum_{u=0}^{\infty}{a}(p+u)\overline{a(q+u)}.$ (63)

Denote by $Q$ the operator in the space $\ell_{2}$, determined by the matrix
$Q=\left\\{Q(p,q)\right\\}_{p,q=0}^{\infty}.$ Since the second condition from
(60) is satisfied and

$\sum_{p,q=0}^{\infty}\left|Q(p,q)\right|^{2}=\sum_{p,q=0}^{\infty}\;\left|\sum_{u=0}^{\infty}a(p+u)\overline{a(q+u)}\right|^{2}\leq$
$\leq\sum_{p,q=0}^{\infty}\;\left(\sum_{u=0}^{\infty}\left|a(p+u)\right|^{2}\cdot\sum_{u=0}^{\infty}\left|a(q+u)\right|^{2}\right)=$
$=\left(\sum_{p=0}^{\infty}\sum_{u=0}^{\infty}\left|a(p+u)\right|^{2}\right)^{2}=\left(\sum_{p=0}^{\infty}(p+1)\left|{a}(p)\right|^{2}\right)^{2},$

we have

$\left\|Q\right\|\leq
N(Q)\leq\sum_{p=0}^{\infty}(p+1)\left|{a}(p)\right|^{2}<\infty,$

where $N(Q)$ is the Hilbert-Schmidt norm of the operator $Q$. The operator $Q$
is selfadjoint (its matrix is Hermitian) Hilbert-Schmidt operator. For these
reasons the operator $Q$ is selfadjoint continuous operator [1],[71]. It can
be represented in the form $Q=A\cdot A^{*}$, where the operator $A$ is
determined by the matrix

$A=\left\\{A(p,q)\right\\}_{p,q=0}^{\infty},\quad
A(p,q)={a}(p+q),\,p,q=0,1,\dots,\infty.$

The operator $Q$ has real nonnegative eigenvalues. Note that the operator $A$
is a Hilbert-Schmidt operator and its Hilbert-Schmidt norm equals to

$N(A)=\left(\sum_{p=0}^{\infty}(p+1)\left|{a}(p)\right|^{2}\right)^{1/2}.$

Let us introduce the notation

$\vec{\varphi}=\\{\tilde{\varphi}(u):u=0,1,\dots\\},\quad\tilde{\varphi}(u)=P^{-1/2}\varphi(u).$

Making use the introduced notations (62) can be written in the form

$\mathop{\min}\limits_{\hat{A}\in\Lambda}\Delta(\xi,\hat{A})=P\left\langle
Q\vec{\varphi},\vec{\varphi}\right\rangle.$

Taking into consideration (48), we get

$\mathop{\max}\limits_{\xi\in\Xi_{R}}\mathop{\min}\limits_{\hat{A}\in\Lambda}\Delta(\xi,\hat{A})=P\mathop{\max}\limits_{\left\|\vec{\varphi}\right\|=1}\left\langle
Q\vec{\varphi},\vec{\varphi}\right\rangle=P\nu^{2},$

where $\nu^{2}$ is the greatest eigenvalue of the operator $Q$ and
$\left\langle\cdot,\cdot\right\rangle$ is the inner product in the space
$\ell_{2}$. Making use (61), we can estimate the maximin value of the error

$\mathop{\max}\limits_{\xi\in\Xi}\mathop{\min}\limits_{\hat{A}\in\Lambda}\Delta(\xi,\hat{A})\geq
P\nu^{2}.$ (64)

Upper bound. Consider the sequence of operators $Q_{N}$, determined by
matrices (52), and the operator $Q$ determined by the matrix with elements
(63). Since the second condition from (60) is satisfied, then

$N(Q-Q_{N})=\sum_{p=N+1}^{\infty}(p+1)\left|{a}(p)\right|^{2}\to 0,$

as $N\to\infty$. Having in mind that

$\left\|Q-Q_{N}\right\|\leq N(Q-Q_{N}),$

we have

$\mathop{\lim}\limits_{N\to\infty}\left\|Q-Q_{N}\right\|=0.$

It means that the sequence of operators $Q_{N}$ converge uniformly to the
operator $Q$. That is why [8], [14]

$\mathop{\lim}\limits_{N\to\infty}\nu_{N}^{2}=\nu^{2},$

where $\nu_{N}^{2}$ is the greatest eigenvalue of the operator $Q_{N}$, and
$\nu^{2}$ is the greatest eigenvalue of the operator $Q$. Making use the
statement of Theorem 1, we can write

$\mathop{\min}\limits_{\hat{A}\in\Lambda}\mathop{\max}\limits_{\xi\in\Xi}\Delta(\xi,\hat{A})=$
(65)
$=\mathop{\lim}\limits_{N\to\infty}\mathop{\min}\limits_{\hat{A}_{N}\in\Lambda}\mathop{\max}\limits_{\xi\in\Xi}\Delta(\xi,\hat{A}_{N})=\mathop{\lim}\limits_{N\to\infty}P\nu_{N}^{2}=P\nu^{2}.$

Comparing (65) and (64), we get

$\mathop{\min}\limits_{\hat{A}\in\Lambda}\mathop{\max}\limits_{\xi\in\Xi}\Delta(\xi,\hat{A})=P\nu^{2}\leq\mathop{\max}\limits_{\xi\in\Xi}\mathop{\min}\limits_{\hat{A}\in\Lambda}\Delta(\xi,\hat{A}),$

where only equality is possible.

This completes the proof of the theorem.

######  Corollary 2.

The optimal minimax linear estimate $\hat{A}{\xi}$ of the functional $A{\xi}$
is of the form

$\hat{A}{\xi}=\sum_{j=0}^{\infty}{a}(j)\left[\sum_{u=-\infty}^{-1}\varphi(j-u){\varepsilon}(u)\right],$

where $\varepsilon(u)$ is a standard stationary sequence with orthogonal
values, the sequence $\\{\varphi(u):u=0,1,\dots\\}$ is uniquely determined by
coordinates of the eigenvector of the operator $Q$ that corresponds to the
greatest eigenvalue $\nu^{2}$ and condition $E\left\|{\xi}(j)\right\|^{2}=P$.

######  Example 2.

Consider the problem of optimal linear estimation of the functional

$A{\xi}=\sum_{j=0}^{\infty}e^{-\lambda j}\xi(j),$

where $\lambda>0$, which depends on the unknown values of a stationary
sequence ${\xi}(j)$, that satisfies the condition

$E{\xi}(j)=0,\quad E\left|\xi(j)\right|^{2}\leq 1,$

based on observations of the sequence at points $j=-1,-2,\dots$. Conditions
(60) are satisfied. Elements of the matrix which determines the operator $Q$,
determined by the equation (63), are of the form

$Q(p,q)=\sum_{u=0}^{\infty}a(p+u)\overline{a(q+u)}=e^{-\lambda(p+q)}(1-e^{-2\lambda})^{-1}.$

Eigenvalues of the operator $Q$ are determined by the system of equations

$\mu\varphi(p)=\sum_{s=0}^{\infty}e^{-\lambda(p+s)}(1-e^{-2\lambda})^{-1}\varphi(s),\;\;p=0,1,\ldots.$

It follows from this system of equations that $\varphi(p)$ are of the form

$\varphi(p)=Ce^{-\lambda p},\,\,p=0,1,\ldots$

The constant $C$ is determined by the condition

$\sum_{p=0}^{\infty}|\varphi(p)|^{2}=1.$

So we have

$C=(1-e^{-2\lambda})^{1/2},\quad\varphi(p)=(1-e^{-2\lambda})^{1/2}e^{-\lambda
p}.$

Substitution of these expressions to the system of equations gives us

$\mu=(1-e^{-2\lambda})^{-2}.$

We can conclude that the least favourable in the class $\Xi$ stationary
sequence ${\xi}(j)$ is a moving average sequence of the form

${\xi}(j)=(1-e^{-2\lambda})^{1/2}e^{-\lambda j}\sum_{u=-\infty}^{j}e^{\lambda
u}\varepsilon(u),$

where $\varepsilon(u)$ is a stationary sequence with orthogonal values.

The optimal linear minimax estimate $\hat{A}{\xi}$ of the functional $A{\xi}$
is as follows

$\hat{A}{\xi}=(1-e^{-2\lambda})^{1/2}\sum_{j=0}^{\infty}e^{-2\lambda
j}\left[\sum_{u=-\infty}^{-1}e^{\lambda u}\varepsilon(u)\right].$

The value of the mean-square error does not exceed

$\Delta=(1-e^{-2\lambda})^{-2}.$

This value of the mean-square error gives the least favourable stationary
sequence.

###  3.3 Conclusions

In this section we propose a method of solution of the mean square optimal
linear estimation of the functionals $A_{N}{\xi}=\sum_{j=0}^{N}{a}(j){\xi}(j)$
and $A{\xi}=\sum_{j=0}^{\infty}{a}(j){\xi}(j)$ which depend on the unknown
values of a stationary stochastic sequence ${\xi}(j)$ from the class $\Xi$ of
stationary stochastic sequences satisfying the conditions $E{\xi}(j)={0},$
$E|{\xi}(j)|^{2}\leq P.$ Estimates are based on results of observations of the
sequence ${\xi}(j)$ at points of time $j=-1,-2,\dots$.

Inspired by the Ulf Grenander [15] approach to investigation the problem of
optimal linear estimation of the functionals which depend on the unknown
values of a stationary stochastic process we consider the problem as a two-
person zero-sum game. It is show that this game has an equilibrium point. The
maximum error gives a moving average stationary sequence which is least
favourable in the given class of stationary sequences. The greatest value of
the error and the least favourable sequence are determined by the greatest
eigenvalue and the corresponding eigenvector of the operator determined by the
coefficients $a(j)$ which determine the functional.

For the corresponding results for stationary stochastic processes with values
in a Hilbert space see papers by Moklyachuk [35] – [39], [57].

##  4 Extrapolation problem for functionals of stationary sequences

In this section we deal with the problem of the mean-square optimal estimation
of the linear functionals

$A_{N}\xi=\sum\limits_{j=0}^{N}a(j)\xi(j),\quad
A{\xi}=\sum_{j=0}^{\infty}{a}(j){\xi}(j),$

which depend on the unknown values of a stationary stochastic sequence
$\xi(j),j\in\mathbb{Z},$ based on observations of the sequence ${\xi}(j)$ at
points of time $j=-1,-2,\dots$.

The problem is investigated in the case of spectral certainty, where the
spectral density of the stationary stochastic sequence $\xi(j)$ is exactly
known. In this case the classical Hilbert space projection method of linear
estimation of the functional is applied. Formulas are derived for calculation
the value of the mean square error and the spectral characteristic of the
mean-square optimal estimate of the linear functional. In the case of spectral
uncertainty, where the spectral density of the stationary stochastic sequence
is not exactly known, but a class of admissible spectral densities is given,
the minimax-robust procedure to linear estimation of the functional is
applied. Relations which determine the least favourable spectral densities and
the minimax spectral characteristics are proposed for some special sets of
admissible spectral densities.

###  4.1 The classical Hilbert space projection method of linear extrapolation

Let $\xi(j),\,j\in\mathbb{Z}$, $E\xi(j)=0$, be a (wide sense) stationary
stochastic sequence. We will consider values of $\xi(j),\,j\in\mathbb{Z}$, as
elements of the Hilbert space $H=L_{2}(\Omega,\mathcal{F},P)$ of complex
valued random variables with zero first moment, $E\xi=0$, finite second
moment, $E|\xi|^{2}<\infty$, and the inner product
$\left(\xi,{\eta}\right)=E\xi\overline{\eta}$. The correlation function
$R(k)=\left(\xi(j+k),\xi(j)\right)=E\xi(j+k)\overline{\xi(j)}$ of the
stationary stochastic sequence $\xi(j),\,j\in\mathbb{Z}$, admits the spectral
representation

$R(k)=\int\limits_{-\pi}^{\pi}e^{ik\lambda}F(d\lambda),$

where $F(d\lambda)$ is the spectral measure of the sequence. We will consider
regular stationary stochastic sequences with absolutely continuous spectral
measures and the correlation functions of the form

$R(k)=\,\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}e^{ik\lambda}f(\lambda)d\lambda,$

where $f(\lambda)$ is the spectral density function of the sequence $\xi(j)$
that satisfies the regularity condition

$\int\limits_{-\pi}^{\pi}\ln{(f(\lambda))}d\lambda>-\infty.$ (66)

This condition is necessary and sufficient in order that the error-free
extrapolation of the unknown values of the sequence is impossible [12].

Suppose that coefficients ${a}(j)$, which determine the functional $A{\xi}$,
satisfy conditions

$\sum_{j=0}^{\infty}\left|a(j)\right|<\infty,\quad\sum_{j=0}^{\infty}(j+1)\left|{a}(j)\right|^{2}<\infty.$
(67)

In this case the functional $A{\xi}$ has the second moment and the Hilbert-
Schmidt operator $Q$, determined in the previous section, has its Hilbert-
Schmidt norm finite.

Let the sequence ${\xi}(j)$ admits the canonical representation as a moving
average sequence

${\xi}(j)=\sum_{u=-\infty}^{j}d(j-u){\varepsilon}(u),$ (68)

where the sequence $d(u)$ satisfies condition

$\sum_{u=0}^{\infty}|d(u)|^{2}<\infty,$

and where ${\varepsilon}(u)$ is a standard stationary white noise sequence

$E{\varepsilon}(i)\overline{\varepsilon}(j)=\delta_{ij},$

$\delta_{ij}$ is the Kronecker symbol.

In this case the spectral density $f(\lambda)$ of the stationary sequence
${\xi}(j)$ admits the canonical factorization

$f(\lambda)=\varphi(\lambda)\overline{\varphi(\lambda)},\quad\varphi(\lambda)=\sum_{k=0}^{\infty}d(k)e^{-ik\lambda}.$
(69)

Denote by $L_{2}(f)$ the Hilbert space of functions ${a}(\lambda)$ such that

$\int_{-\pi}^{\pi}\,a(\lambda)\overline{a(\lambda)}\,f(\lambda)\,d\lambda<\infty.$

Denote by $L_{2}^{-}(f)$ the subspace of the space $L_{2}(f)$ generated by
functions

$\\{e^{in\lambda},n=-1,-2,\dots\\}.$

Every linear estimate $\hat{A}{\xi}$ of the functional $A{\xi}$ based on
observations of the sequence ${\xi}(j)$ at points $j=-1,-2,\dots$ is of the
form

$\hat{A}{\xi}=\int_{-\pi}^{\pi}h(e^{i\lambda})Z_{\xi}(d\lambda),$

where $Z_{\xi}(\Delta)$ is the orthogonal random measure of the sequence
${\xi}(j)$:

$E\left(Z_{\xi}(\Delta_{1})\overline{Z_{\xi}(\Delta_{2})}\right)=\frac{1}{2\pi}\int_{\Delta_{1}\cap\Delta_{2}}f(\lambda)d\lambda,$

$h(e^{i\lambda})$ is the spectral characteristic of the estimate
$\hat{A}{\xi}$ which belongs to the subspace $L_{2}^{-}(f)$. The mean-square
error of the linear estimate $\hat{A}{\xi}$ of the functional $A{\xi}$ based
on observations of the sequence ${\xi}(j)$ at points $j=-1,-2,\dots$, is
calculated by the formula

$\Delta(h,f)=M\left|A{\xi}-\hat{A}{\xi}\right|^{2}=$
$=\frac{1}{2\pi}\int_{-\pi}^{\pi}\left|A(e^{i\lambda})-h(e^{i\lambda})\right|^{2}f(\lambda)d\lambda,$

where

$A(e^{i\lambda})=\sum_{j=0}^{\infty}{a}(j)e^{ij\lambda}.$

If the stationary stochastic sequence ${\xi}(j)$ admits the canonical
representation in the form of a moving average sequence (68), then the optimal
estimate of the functional $A{\xi}$ is determined by the spectral
characteristic $h(f)\in L_{2}^{-}(f)$ such that

$\Delta(h(f),f)=\mathop{\min}\limits_{h\in
L_{2}^{-}(f)}\Delta(h,f)=\left\|Ad\right\|^{2},$ (70)

where

$\left\|Ad\right\|^{2}=\sum_{k=0}^{\infty}\left\|(Ad)_{k}\right\|^{2},\quad\left(Ad\right)_{k}=\sum_{l=0}^{\infty}{a}(k+l)d(l).$

Note, that $\left\|Ad\right\|^{2}<\infty$ under the conditions (67). The
spectral characteristic $h(f)$ of the optimal estimate is calculated by the
formula

$h(f)=A(e^{i\lambda})-{\varphi}^{-1}(\lambda)\,r(e^{i\lambda}),$ (71)

where

$r(e^{i\lambda})=\sum_{k=0}^{\infty}(Ad)_{k}e^{ik\lambda}.$ (72)

For the functional $A_{N}{\xi}$ the mean-square error and the spectral
characteristic of the optimal estimate of the functional is calculated by the
formulas

$\Delta_{N}(h(f),f)=\left\|A_{N}d\right\|^{2},$ (73)
$h_{N}(f)=A_{N}(e^{i\lambda})-{\varphi}^{-1}(\lambda)\,r_{N}(e^{i\lambda}),$
(74)

where

$A_{N}(e^{i\lambda})=\sum_{j=0}^{N}{a}(j)e^{ij\lambda},\quad
r_{N}(e^{i\lambda})=\sum_{k=0}^{N}(A_{N}d)_{k}e^{ik\lambda},$ (75)
$\left\|A_{N}d\right\|^{2}=\sum_{k=0}^{N}\left\|(A_{N}d)_{k}\right\|^{2},\quad(A_{N}d)_{k}=\sum_{l=0}^{N-k}{a}(k+l)d(l),k=0,1,\dots,N.$

As a corollary, from the formula (73) one can find the following formula for
calculation the mean-square error of optimal estimates $\hat{\xi}(j)$ of the
unknown values $\xi(j)$

$E\left|\xi(j)-\hat{\xi}(j)\right|^{2}=\sum_{u=0}^{j}\left|d(u)\right|^{2},$
(76)

where $d(u)$ are determined from equations of factorization (69) of the
density $f(\lambda)$.

Thus we came to conclusion that the following theorem holds true.

######  Theorem 10.

If conditions (67) are satisfied and the density $f(\lambda)$ admits the
canonical factorization (69), then the mean-square error of the optimal linear
estimate of the functional $A{\xi}$ based on observations of the sequence
${\xi}(j)$ at points $j=-1,-2,\dots$, is calculated by formula (70) (by
formula (73) for the functional $A_{N}{\xi}$). The spectral characteristic of
the optimal linear estimate of the functional is calculated by formula (71)
$($ by formula (74) for the functional $A_{N}{\xi}$ $)$.

###  4.2 Minimax-robust method of linear extrapolation of functionals

Formulas (70) – (76) can be applied to calculate the spectral characteristic
and the mean-square error of the optimal linear estimate of the functional
$A{\xi}$ only in the case where the spectral density $f(\lambda)$ of the
stationary stochastic sequence ${\xi}(j)$ is exactly known. In the case where
the spectral density $f(\lambda)$ is not exactly known, but, instead, a set
$D$ of admissible spectral densities is specified, the minimax approach to the
problem of the optimal linear estimate of the functional which depends on the
unknown values of the stationary sequence is reasonable. Under this approach
one finds an estimate of the functional which is optimal and minimize the
mean-square error for all spectral densities from a given class $D$
simultaneously.

######  Definition 1.

A spectral density $f^{0}(\lambda)$ is called the least favourable in the
class $D$ for the optimal linear extrapolation of the functional $A{\xi}$ if
the following relation holds true

$\Delta(h(f^{0}),f^{0})=\mathop{\max}\limits_{f\in
D}\Delta(h(f),f)=\mathop{\max}\limits_{f\in D}\mathop{\min}\limits_{h\in
L_{2}^{-}(f)}\Delta(h,f).$

Taking into consideration relations (70) – (76), we can verify that the
following statements hold true

######  Theorem 11.

The spectral density $f^{0}(\lambda)\in D$ is the least favourable in the
class $D$ for the optimal linear extrapolation of the functional $A{\xi}$, if
it admits the canonical factorization

$f^{0}(\lambda)=\left(\sum_{k=0}^{\infty}d^{0}(k)e^{-ik\lambda}\right)\cdot\overline{\left(\sum_{k=0}^{\infty}d^{0}(k)e^{-ik\lambda}\right)},$
(77)

where $d^{0}=\left\\{d^{0}(k):k=0,\,1,\,\ldots\right\\}$ is a solution to the
constrained optimization problem

$\left\|Ad\right\|^{2}\to\max,$ (78)
$f(\lambda)=\left(\sum_{k=0}^{\infty}d(k)e^{-ik\lambda}\right)\cdot\overline{\left(\sum_{k=0}^{\infty}d(k)e^{-ik\lambda}\right)}\in
D.$

The sequence ${\xi}(j)$ in this case admits the canonical one-sided moving
average representation

${\xi}(j)=\sum_{u=-\infty}^{j}d^{0}(j-u){\varepsilon}(u).$ (79)

######  Theorem 12.

The spectral density $f^{0}(\lambda)\in D$ is the least favourable in the
class $D$ for the optimal linear extrapolation of the functional $A_{N}{\xi}$,
if it admits the canonical factorization

$f^{0}(\lambda)=\left(\sum_{k=0}^{N}d^{0}(k)e^{-ik\lambda}\right)\cdot\overline{\left(\sum_{k=0}^{N}d^{0}(k)e^{-ik\lambda}\right)},$
(80)

where $d^{0}=\left\\{d^{0}(k):k=0,1,\dots,N\right\\}$ is a solution to the
constrained optimization problem

$\left\|A_{N}d\right\|^{2}\to\max,$ (81)
$f(\lambda)=\left(\sum_{k=0}^{N}d(k)e^{-ik\lambda}\right)\cdot\overline{\left(\sum_{k=0}^{N}d(k)e^{-ik\lambda}\right)}\in
D.$

The sequence ${\xi}(j)$ in this case admits the canonical one-sided moving
average representation of order $N+1$:

${\xi}(j)=\sum_{u=j-N}^{j}d^{0}(j-u){\varepsilon}(u).$ (82)

######  Definition 2.

The spectral characteristic $h^{0}(e^{i\lambda})$ of the optimal linear
extrapolation of the functional $A{\xi}$ is called minimax (robust) if the
following conditions hold true

$h^{0}(e^{i\lambda})\in H_{D}=\bigcap_{f\in
D}L_{2}^{-}(f),\quad\mathop{\min}\limits_{h\in
H_{D}}\mathop{\max}\limits_{f\in D}\Delta(h,f)=\mathop{\max}\limits_{f\in
D}\Delta(h^{0},f).$

The least favourable spectral density $f^{0}(\lambda)\in D$ and the minimax
(robust) spectral characteristic $h^{0}(e^{i\lambda})\in H_{D}$ form a saddle
point of the function $\Delta(h,f)$ on the set $H_{D}\times{D}$. The saddle
point inequalities

$\Delta(h,f^{0})\geq\Delta(h^{0},f^{0})\geq\Delta(h^{0},f)\quad\forall f\in
D\;\forall h\in H_{D}$

hold true if $h^{0}=h(f^{0})$ and $h(f^{0})\in H_{D}$, where $f^{0}$ is a
solution to the constrained optimization problem

$\Delta(h(f^{0}),f^{0})=\mathop{\max}\limits_{f\in D}\Delta(h(f^{0}),f).$

If we have found a solution $f^{0}(\lambda)$ to this problem, then the
minimax(robust) spectral characteristics can be calculated by formulas (71),
(74) if the condition $h(f^{0})\in H_{D}$ holds true.

The spectral density $f^{0}(\lambda)$ is a solution to the following
constrained optimization problem

$\Delta(f)=-\Delta(h(f^{0}),f)\to\inf,\quad f(\lambda)\in D,$ (83)
$\Delta(h(f^{0}),f)=\frac{1}{2\pi}\int_{-\pi}^{\pi}\frac{|r(e^{i\lambda})|^{2}}{|\varphi^{0}(\lambda)|^{2}}f(\lambda)d\lambda,$

where $r(e^{i\lambda})$ is calculated by formulas (72), (75) with
$f(\lambda)=f^{0}(\lambda)$.

The constrained optimization problem (83) is equivalent to the following
unconstrained optimization problem [58],[70]

$\Delta_{D}(f)=-\Delta(h(f^{0}),f)+\delta(f|D)\to\inf,$ (84)

where $\delta(f|D)$ is the indicator function of the set $D$. Solution of the
problem (84) is determined by the condition $0\in\partial\Delta_{D}(f^{0})$,
where $\partial\Delta_{D}(f^{0})$ is the subdifferential on the convex
functional $\Delta_{D}(f)$ at the point $f^{0}$. With the help of conditions
(83), (84) we can find the least favourable spectral densities for concrete
classes of spectral densities.

Note, that the form of the functional $\Delta(h(f_{0}),f)$ is convenient for
application the Lagrange method of indefinite multipliers for finding solution
to the problem (83). Making use the method of Lagrange multipliers and the
form of subdifferentials of the indicator functions of certain classes of
spectral densities we describe relations that determine least favourable
spectral densities in some special classes of spectral densities [57], [64].

###  4.3 Least favourable spectral densities in the class $D_{0}$

Consider the problem of the optimal estimation of the functionals
$A\xi=\sum_{j=1}^{\infty}a(j)\xi(j)$ and $A_{N}\xi=\sum_{j=1}^{N}a(j)\xi(j)$
which depends on the unknown values of a stationary stochastic sequence
$\xi(j)$ from observations of the sequence $\xi(j)$ at points of time
$j=-1,-2,\dots$ in the case where the spectral density $f(\lambda)$ is from
the class $D_{0}$ of spectral densities which are characterized by
restrictions on the first moment of the density

$D_{0}=\left\\{f(\lambda):\,\frac{1}{2\pi}\int_{-\pi}^{\pi}\;f(\lambda)d\lambda\leq
P_{0}\right\\},$

where $P_{0},P_{0}>0,$ is a given number. This class of spectral densities
describes stationary sequences with restriction on the dispersion
$E|\xi(j)|^{2}\leq P_{0}$.

We can apply the method of Lagrange multipliers to find solution to the
optimization problem (83). We get the following relations that determine the
least favourable spectral density $f^{0}\in D_{0}$

$|r(e^{i\lambda})|^{2}=\alpha^{2}|\varphi^{0}(\lambda)|^{2},$ (85)

where $\alpha^{2}$ is the Lagrange multiplier.

This relation can be rewritten in the following way

$\left(\sum_{k=0}^{\infty}(Ad^{0})_{k}e^{ik\lambda}\right)\cdot\overline{\left(\sum_{k=0}^{\infty}(Ad^{0})_{k}e^{ik\lambda}\right)}=\alpha^{2}\left(\sum_{k=0}^{\infty}d^{0}(k)e^{-ik\lambda}\right)\cdot\overline{\left(\sum_{k=0}^{\infty}d^{0}(k)e^{-ik\lambda}\right)}.$
(86)

The unknown $\alpha^{2}$ and $d^{0}=\left\\{d^{0}(k):k=0,1,\ldots\right\\}$
are determined with the help of equations of the canonical factorization (77)
of the density $f^{0}(\lambda)$, solution of the constrained optimization
problem (78) and restrictions imposed on densities from the class of
admissible spectral densities $D_{0}$.

For all solutions $d=\left\\{d(k):k=0,1,\ldots\right\\}$ of the equation

$Ad=\alpha\overline{d},\,\alpha\in\mathbb{C},$ (87)

which satisfy condition

$\|d\|^{2}=\sum_{k=0}^{\infty}|d(k)|^{2}=P_{0},$ (88)

the following equality holds true

$f^{0}(\lambda)=\left|\sum_{k=0}^{\infty}d(k)e^{-ik\lambda}\right|^{2}=\left|c\sum_{k=0}^{\infty}(Ad)_{k}e^{ik\lambda}\right|^{2}.$
(89)

Solution $d^{0}=\left\\{d^{0}(k):k=0,1,\ldots\right\\}$ of the equation (87)
which satisfies condition (88) and gives the maximum value
$\|Ad^{0}\|^{2}=\nu_{0}P_{0}$ of the quantity $\|Ad\|^{2}$ determines the
least favourable spectral density

$f^{0}(\lambda)=\left|\sum_{k=0}^{\infty}d^{0}(k)e^{-ik\lambda}\right|^{2},$
(90)

which is the spectral density of the one-sided moving-average sequence

${\xi}(j)=\sum_{u=-\infty}^{j}d^{0}(j-u){\varepsilon}(u).$ (91)

Thus the following statement fulfilled.

######  Theorem 13.

The least favourable in the class $D_{0}$ for the optimal extrapolation of the
functional $A{\xi}$ is the spectral density (90) of the one-sided moving-
average sequence (91) which is determined by solution
$d^{0}=\left\\{d^{0}(k):k=0,1,\ldots\right\\}$ of the equation (87) which
satisfies condition (88) and gives the maximum value
$\|Ad^{0}\|^{2}=\nu^{2}P_{0}$ of the quantity $\|Ad\|^{2}$. The minimax
spectral characteristic of the optimal estimate of the functional $A{\xi}$ is
calculated by the formula (71).

For the functional $A_{N}\xi=\sum_{j=1}^{N}a(j)\xi(j)$ the corresponding
relation has the following form

$\left|r_{N}(e^{i\lambda})\right|^{2}=\alpha^{2}\left|\varphi^{0}(\lambda)\right|^{2},$
(92)

and the equality

$\left|r_{N}(e^{i\lambda})\right|^{2}=\left|\sum_{k=0}^{N}(A_{N}d)_{k}e^{ik\lambda}\right|^{2}=\left|\sum_{k=0}^{N}(\tilde{A}_{N}d)_{k}e^{-ik\lambda}\right|^{2},$
(93)

holds true, where

$(\tilde{A}_{N}d)_{k}=\sum_{u=0}^{k}{a}(N-k+u)d(u),\;k=0,1,\dots,N.$

For all solutions $d=\left\\{d(k):k=0,1,\dots,N\right\\}$ to equations

$A_{N}d=\alpha\overline{d},\,\alpha\in\mathbb{C},$ (94)
$\tilde{A}_{N}d=\beta{d},\,\beta\in\mathbb{C},$ (95)

which satisfy condition

$\|d\|^{2}=\sum_{k=0}^{N}|d(k)|^{2}=P_{0},$ (96)

the following equality holds true

$f^{0}(\lambda)=\left|\sum_{k=0}^{N}d(k)e^{-ik\lambda}\right|^{2}=\left|c\sum_{k=0}^{N}(Ad)_{k}e^{ik\lambda}\right|^{2}.$
(97)

Solutions $d^{0}=\left\\{d^{0}(k):k=0,1,\ldots,N\right\\}$ of the equations
(94), (95) which satisfy condition (96) and gives the maximum values
$\|A_{N}d^{0}\|^{2}=\|\tilde{A}_{N}d^{0}\|^{2}=\nu_{N}^{2}P_{0}$ of the
quantity $\|A_{N}d\|^{2}$ determines the least favourable spectral density

$f^{0}(\lambda)=\left|\sum_{k=0}^{N}d^{0}(k)e^{-ik\lambda}\right|^{2},$ (98)

which is the spectral density of the one-sided moving-average sequence of
order $N$

${\xi}(j)=\sum_{u=j-N}^{j}d^{0}(j-u){\varepsilon}(u).$ (99)

Thus the following statement holds true.

######  Theorem 14.

The least favourable in the class $D_{0}$ for the optimal extrapolation of the
functional $A_{N}{\xi}$ is the spectral density (98) of the one-sided moving-
average sequence (99) which is determined by solutions
$d^{0}=\left\\{d^{0}(k):k=0,1,\ldots,N\right\\}$ of the equations (94), (95)
which satisfy condition (96) and gives the maximum value
$\|A_{N}d^{0}\|^{2}=\|\tilde{A}_{N}d^{0}\|^{2}=\nu_{N}^{2}P_{0}$ of the
quantity $\|A_{N}d\|^{2}$. The minimax spectral characteristic of the optimal
estimate of the functional $A_{N}{\xi}$ is calculated by the formula (74).

######  Corollary 3.

The spectral density (98) of the one-sided moving-average sequence (99), where
the sequence $d^{0}=\left\\{d^{0}(k):k=0,1,\ldots,N\right\\}$ satisfy
condition (96), is the least favourable in the class $D_{0}$ for the optimal
extrapolation of the functional ${a}(N){\xi}(N)$.

###  4.4 Least favourable spectral densities in the class $D_{M}$

Consider the problem of the optimal estimation of the functionals
$A\xi=\sum_{j=1}^{\infty}a(j)\xi(j)$ and $A_{N}\xi=\sum_{j=1}^{N}a(j)\xi(j)$
which depends on the unknown values of a stationary stochastic sequence
$\xi(j)$ from observations of the sequence $\xi(j)$ at points of time
$j=-1,-2,\dots$ in the case where the spectral density $f(\lambda)$ is from
the class $D_{M}$ of spectral densities which are characterized by
restrictions on the moments of the density

$D_{M}=\left\\{f(\lambda)\left|\,\frac{1}{2\pi}\int_{-\pi}^{\pi}f(\lambda)\cos(m\lambda)d\lambda=r(m),\,m=0,1,\dots,M\right.\right\\},$

where $r(0)=P_{0}$ and $r(0),r(1),\dots,r(M)$ is a strictly positive sequence.
The moment problem in this case has not uniquely determined solutions and the
class $D_{M}$ contains an infinite number of densities [25].

We can apply the method of Lagrange multipliers to find solution to the
optimization problem (83) in the case $D=D_{M}$. We get the following
relations that determine the least favourable spectral density $f^{0}\in
D_{M}$

$|r(e^{i\lambda})|^{2}(f^{0}(\lambda))^{-1}=\sum\limits_{m=0}^{M}\alpha_{m}\cos(m\lambda)=\left|\sum\limits_{m=0}^{M}p(m)e^{im\lambda}\right|^{2},$
(100)

where $\alpha_{m},m=0,1,\ldots,M$ are the Lagrange multipliers.

It follows from this relation that the least favourable spectral density
$f^{0}\in D_{M}$ is of the form

$f_{0}(\lambda)=\frac{\left|\sum\limits_{k=0}^{\infty}(Ad)_{k}e^{ik\lambda}\right|^{2}}{\left|\sum\limits_{m=0}^{M}p(m)e^{-im\lambda}\right|^{2}}.$
(101)

The unknown $\\{p(m):m=0,1,\dots,M\\}$ and
$d=\left\\{d(k):k=0,\,1,\,\ldots\right\\}$ are determined by equations of the
canonical factorization (77) of the density $f^{0}(\lambda)$, solution of the
constrained optimization problem (78) and restrictions imposed on densities
from the class of admissible spectral densities $D_{M}$.

Denote by $\nu_{M}P_{0}$ the maximum value $\|Ad^{0}\|^{2}$ of the quantity
$\|Ad\|^{2}$, where $d=\left\\{d(k):k=0,\,1,\,\ldots\right\\}$ satisfies
condition (88) and are determined by equations of the canonical factorization
(77) of the density $f^{0}(\lambda)$ and restrictions imposed on densities
from the class of admissible spectral densities $D_{M}$.

Denote by $\nu_{M}^{+}P_{0}$ the maximum value $\|Ad^{0}\|^{2}$ of the
quantity $\|Ad\|^{2}$, where $d=\left\\{d(k):k=0,1,\ldots\right\\}$ satisfies
condition (88) and are determined by equations of canonical factorization (77)
of the density (101) and restrictions imposed on densities from the class of
admissible spectral densities $D_{M}$.

The following statement is true.

######  Theorem 15.

If there exists a sequence $d^{0}=\left\\{d^{0}(k):k=0,1,\ldots\right\\}$
which satisfies condition (88) and such that
$\nu_{M}P_{0}=\nu_{M}^{+}P_{0}=\|Ad^{0}\|^{2}$, then the least favourable in
the class $D_{M}$ for the optimal extrapolation of the functional $A{\xi}$ is
the spectral density (90) of the one-sided moving-average sequence (91). If
$\nu_{M}<\nu_{M}^{+}$, then the least favourable in the class $D_{M}$ for the
optimal extrapolation of the functional $A{\xi}$ is the spectral density (101)
which admits the canonical factorization (77). The unknown
$\\{p(m):m=0,1,\dots,M\\}$ and $d=\left\\{d(k):k=0,\,1,\,\ldots\right\\}$ are
determined by equations of canonical factorization (77) of the density
$f^{0}(\lambda)$, solution of the constrained optimization problem (78) and
restrictions imposed on densities from the class of admissible spectral
densities $D_{M}$. The minimax spectral characteristic of the optimal estimate
of the functional $A{\xi}$ is calculated by the formula (71).

Consider now the problem of the optimal estimation of the functional
$A_{N}\xi=\sum_{j=1}^{N}a(j)\xi(j)$. It follows from the relation (100) that
in this case the least favourable spectral density $f^{0}(\lambda)\in D_{M}$
is of the form

$f_{0}(\lambda)=\frac{\left|\sum\limits_{k=0}^{N}(A_{N}d)_{k}e^{ik\lambda}\right|^{2}}{\left|\sum\limits_{m=0}^{M}p(m)e^{-im\lambda}\right|^{2}}.$
(102)

These densities are spectral densities of the autoregressive-moving-average
$ARMA(M,N)$ sequences

$\sum\limits_{m=0}^{M}p(m)\xi(n-m)=\sum_{k=0}^{N}(A_{N}d)_{k}{\varepsilon}(n-k).$
(103)

The unknown $\\{p(m):m=0,1,\dots,M\\}$ and
$d=\left\\{d(k):k=0,1,\ldots,N\right\\}$ are determined by equations of
canonical factorization (80) of the density $f^{0}(\lambda)$, solution of the
constrained optimization problem (81) and restrictions imposed on densities
from the class of admissible spectral densities $D_{M}$.

Denote by $\nu_{MN}P_{0}$ the maximum value $\|A_{N}d^{0}\|^{2}$ of the
quantity $\|A_{N}d\|^{2}$, where $d=\left\\{d(k):k=0,1,\ldots,N\right\\}$ are
determined by equations (94), (95), condition (96) and equations of canonical
factorization (80) of the density $f^{0}(\lambda)\in D_{M}$.

Denote by $\nu_{MN}^{+}P_{0}$ the maximum value $\|A_{N}d^{0}\|^{2}$ of the
quantity $\|A_{N}d\|^{2}$, where $d=\left\\{d(k):k=0,1,\ldots,N\right\\}$
satisfies condition (96), equations of canonical factorization (77) of the
density (102) and restrictions imposed on densities from the class of
admissible spectral densities $f^{0}(\lambda)\in D_{M}$.

Thus the following statement holds true.

######  Theorem 16.

If there exists a solution $d^{0}=\left\\{d^{0}(k):k=0,1,\ldots,N\right\\}$ of
the equation (94), or the equation (95), which satisfy condition (96) and such
that $\nu_{MN}P_{0}=\nu_{MN}^{+}P_{0}=\|A_{N}d^{0}\|^{2}$, then the least
favourable in the class $D_{M}$ for the optimal extrapolation of the
functional $A_{N}{\xi}$ is the spectral density (98) of the one-sided moving-
average sequence (99). If $\nu_{MN}<\nu_{MN}^{+}$, then the least favourable
in the class $D_{M}$ for the optimal extrapolation of the functional
$A_{N}{\xi}$ is the spectral density (102) of the autoregressive-moving-
average $ARMA(M,N)$ sequences (103). The unknown $\\{p(m):m=0,1,\dots,M\\}$
and $d=\left\\{d(k):k=0,\,1,\,\ldots\right\\}$ are determined by equations of
canonical factorization (77) of the density $f^{0}(\lambda)$, solution of the
constrained optimization problem (78) and restrictions imposed on densities
from the class of admissible spectral densities $D_{M}$. The minimax spectral
characteristic of the optimal estimate of the functional $A_{N}{\xi}$ is
calculated by the formula (74).

###  4.5 Least favourable spectral densities in the class $D_{v}^{u}$

Consider the problem of minimax estimation of the functionals $A{\xi}$ and
$A_{N}{\xi}$ which depend on the unknown values of a stationary stochastic
sequence ${\xi}(j)$ for the sets of spectral densities that describe the
“strip” model of stationary stochastic sequences

$D_{v}^{u}=\left\\{f(\lambda)\left|v(\lambda)\leq f(\lambda)\leq
u(\lambda),\;\frac{1}{2\pi}\int_{-\pi}^{\pi}f(\lambda)d\lambda=P_{0}\right.\right\\},$

where $v(\lambda),u(\lambda)$ are given bounded spectral densities.

From the condition $0\in\partial\Delta_{D}(f_{0})$ for $D=D_{v}^{u}$ we find
the following equation which determines the least favourable spectral density
for the optimal estimation of the functional $A{\xi}$

$\left|\sum\limits_{k=0}^{\infty}(Ad)_{k}e^{ik\lambda}\right|^{2}=(\psi_{1}(\lambda)+\psi_{2}(\lambda)+\alpha_{0})\left|\varphi^{0}(\lambda)\right|^{2},$
(104)

where $\psi_{1}(\lambda)\geq 0$ and $\psi_{1}(\lambda)=0$ if
$f_{0}(\lambda)\geq v(\lambda);$ $\psi_{2}(\lambda)\leq 0$ and
$\psi_{2}(\lambda)=0$ if $f_{0}(\lambda)\leq u(\lambda).$

From this equation we find that the least favourable spectral density for the
optimal estimation of the functional $A{\xi}$ is of the form

$f_{0}(\lambda)=\max\left\\{v(\lambda),\min\left\\{u(\lambda),c\left|\sum\limits_{k=0}^{\infty}(Ad)_{k}e^{ik\lambda}\right|^{2}\right\\}\right\\}.$
(105)

Denote by $\nu_{u}P_{0}$ the maximum value $\|Ad^{0}\|^{2}$ of the quantity
$\|Ad\|^{2}$, where $d=\left\\{d(k):k=0,\,1,\,\ldots\right\\}$ are solutions
of the equation (87) which satisfy condition (88), the inequality

$v(\lambda)\leq\left|\sum\limits_{k=0}^{\infty}d(k)e^{-ik\lambda}\right|^{2}\leq
u(\lambda),$ (106)

and determine the canonical factorization (77) of the density
$f^{0}(\lambda)\in D_{v}^{u}$.

Denote by $\nu_{u}^{+}P_{0}$ the maximum value $\|Ad^{0}\|^{2}$ of the
quantity $\|Ad\|^{2}$, where $d=\left\\{d(k):k=0,1,\ldots\right\\}$ satisfies
condition (88) and determine the canonical factorization (77) of the density
(105) from the class of admissible spectral densities $D_{v}^{u}$.

Thus the following theorem holds true.

######  Theorem 17.

If there exists a solution of the equation (87) which satisfy condition (88)
and such that $\nu_{u}P_{0}=\nu_{u}^{+}P_{0}=\|Ad^{0}\|^{2}$, then the
spectral density (90) of the one-sided moving-average sequence (91) is the
least favourable in the set $D_{v}^{u}$ for the optimal extrapolation of the
functional $A{\xi}$. If $\nu_{u}<\nu_{u}^{+}$, then the least favourable in
the class $D_{v}^{u}$ for the optimal extrapolation of the functional $A{\xi}$
is the spectral density (105) which admits the canonical factorization (90).
The sequence $d=\left\\{d(k):k=0,1,\ldots\right\\}$ is determined by the
optimisation problem (78) and restrictions imposed on densities by the given
set of admissible spectral densities. The minimax spectral characteristic of
the optimal estimate of the functional $A{\xi}$ is calculated by the formula
(71).

For the functional $A_{N}{\xi}$ the least favourable spectral density for the
optimal estimation of the functional is of the form

$f_{0}(\lambda)=\max\left\\{v(\lambda),\min\left\\{u(\lambda),c\left|\sum\limits_{k=0}^{N}(A_{N}d)_{k}e^{ik\lambda}\right|^{2}\right\\}\right\\}.$
(107)

Denote by $\nu_{uN}P_{0}$ the maximum value $\|A_{N}d^{0}\|^{2}$ of the
quantity $\|A_{N}d\|^{2}$, where $d=\left\\{d(k):k=0,1,\ldots,N\right\\}$ are
solutions of the equations (94), (95), which satisfy condition (96), the
inequality

$v(\lambda)\leq\left|\sum\limits_{k=0}^{N}d(k)e^{-ik\lambda}\right|^{2}\leq
u(\lambda),$ (108)

and determine the canonical factorization (80) of the density
$f^{0}(\lambda)\in D_{v}^{u}$.

Denote by $\nu_{uN}^{+}P_{0}$ the maximum value $\|A_{N}d^{0}\|^{2}$ of the
quantity $\|A_{N}d\|^{2}$, where $d=\left\\{d(k):k=0,1,\ldots\right\\}$
satisfies condition (88) and determine the canonical factorization (77) of the
density (107) from the class of admissible spectral densities $D_{v}^{u}$.

The following theorem holds true.

######  Theorem 18.

If there exists a solution of the equation (94), or equation (95), which
satisfy condition (96) and such that
$\nu_{uN}P_{0}=\nu_{uN}^{+}P_{0}=\|A_{N}d^{0}\|^{2}$, then the spectral
density (98) of the one-sided moving-average sequence (99) is the least
favourable in the set $D_{v}^{u}$ for the optimal extrapolation of the
functional $A_{N}{\xi}$. If $\nu_{uN}<\nu_{uN}^{+}$, then the least favourable
in the class $D_{v}^{u}$ for the optimal extrapolation of the functional
$A_{N}{\xi}$ is the spectral density (107) which admits the canonical
factorization (90). The sequence $d=\left\\{d(k):k=0,1,\ldots\right\\}$ is
determined by the optimisation problem (78) and restrictions imposed on
densities by the given set of admissible spectral densities. The minimax
spectral characteristic of the optimal estimate of the functional $A_{N}{\xi}$
is calculated by the formula (74).

###  4.6 Least favourable spectral densities in the class $D_{\varepsilon}$

Consider the problem of minimax estimation of the functionals $A{\xi}$ and
$A_{N}{\xi}$ which depend on the unknown values of a stationary stochastic
sequence ${\xi}(j)$ for the set of spectral densities that describes the
“$\varepsilon$– contamination” model of stationary stochastic sequences

$D_{\varepsilon}=\left\\{f(\lambda)\left|\;f(\lambda)=(1-\varepsilon)w(\lambda)+\varepsilon
u(\lambda),\,\,\frac{1}{2\pi}\int_{-\pi}^{\pi}f(\lambda)d\lambda=P_{0}\right.\right\\},$

where $w(\lambda)$ is a known spectral density, and $u(\lambda)$ is an unknown
spectral density. From the condition $0\in\partial\Delta_{D}(f^{0})$ for the
functional $A{\xi}$ we find the following equations which determine the least
favourable spectral densities for the optimal estimation of the functional
$A{\xi}$ for the given set of admissible spectral densities

$\left|\sum\limits_{k=0}^{\infty}(Ad)_{k}e^{ik\lambda}\right|^{2}=(\psi_{1}(\lambda)+\alpha_{0}^{-1})\left|\varphi^{0}(\lambda)\right|^{2},$
(109)

where $\psi_{1}(\lambda)\geq 0$ and $\psi_{1}(\lambda)=0$ if
$f_{0}(\lambda)\geq(1-\varepsilon)w(\lambda).$

From this equation we find that the least favourable spectral density for the
optimal estimation of the functional $A{\xi}$ is of the form

$f_{0}(\lambda)=\max\left\\{(1-\varepsilon)w(\lambda),\alpha_{0}\left|\sum\limits_{k=0}^{\infty}(Ad)_{k}e^{ik\lambda}\right|^{2}\right\\}.$
(110)

Denote by $\nu_{\varepsilon}P_{0}$ the maximum value $\|Ad^{0}\|^{2}$ of the
quantity $\|Ad\|^{2}$, where $d=\left\\{d(k):k=0,\,1,\,\ldots\right\\}$ are
solutions of the equation (87) which satisfy condition (88), the inequality

$\left|\sum\limits_{k=0}^{\infty}d(k)e^{-ik\lambda}\right|^{2}\geq(1-\varepsilon)w(\lambda),$
(111)

and determine the canonical factorization (77) of the density
$f^{0}(\lambda)\in D_{\varepsilon}$.

Denote by $\nu_{\varepsilon}^{+}P_{0}$ the maximum value $\|Ad^{0}\|^{2}$ of
the quantity $\|Ad\|^{2}$, where $d=\left\\{d(k):k=0,1,\ldots\right\\}$
satisfies condition (88) and determine the canonical factorization (77) of the
density (110) from the class of admissible spectral densities
$D_{\varepsilon}$.

The following theorem holds true.

######  Theorem 19.

If there exists a solution of the equation (87) which satisfy condition (88)
and such that
$\nu_{\varepsilon}P_{0}=\nu_{\varepsilon}^{+}P_{0}=\|Ad^{0}\|^{2}$, then the
spectral density (90) of the one-sided moving-average sequence (91) is the
least favourable in the set $D_{\varepsilon}$ for the optimal extrapolation of
the functional $A{\xi}$. If $\nu_{\varepsilon}<\nu_{\varepsilon}^{+}$, then
the least favourable in the class $D_{\varepsilon}$ for the optimal
extrapolation of the functional $A{\xi}$ is the spectral density (110) which
admits the canonical factorization (90). The sequence
$d=\left\\{d(k):k=0,1,\ldots\right\\}$ is determined by the optimisation
problem (78) and restrictions imposed on densities by the given set of
admissible spectral densities. The minimax spectral characteristic of the
optimal estimate of the functional $A{\xi}$ is calculated by the formula (71).

For the functional $A_{N}{\xi}$ the least favourable spectral density for the
optimal estimation of the functional is of the form

$f_{0}(\lambda)=\max\left\\{(1-\varepsilon)w(\lambda),\alpha_{0}\left|\sum\limits_{k=0}^{N}(A_{N}d)_{k}e^{ik\lambda}\right|^{2}\right\\}.$
(112)

Denote by $\nu_{\varepsilon}^{N}P_{0}$ the maximum value $\|A_{N}d^{0}\|^{2}$
of the quantity $\|A_{N}d\|^{2}$, where
$d=\left\\{d(k):k=0,1,\ldots,N\right\\}$ are solutions of the equations (94),
(95), which satisfy condition (96), the inequality

$\left|\sum\limits_{k=0}^{N}d(k)e^{-ik\lambda}\right|^{2}\geq(1-\varepsilon)w(\lambda),$
(113)

and determine the canonical factorization (80) of the density
$f^{0}(\lambda)\in D_{\varepsilon}$.

Denote by $\nu_{\varepsilon}^{N+}P_{0}$ the maximum value $\|A_{N}d^{0}\|^{2}$
of the quantity $\|A_{N}d\|^{2}$, where $d=\left\\{d(k):k=0,1,\ldots\right\\}$
satisfies condition (88) and determine the canonical factorization (77) of the
density (107) from the class of admissible spectral densities
$D_{\varepsilon}$.

The following theorem holds true.

######  Theorem 20.

If there exists a solution of the equation (94), or the equation, (95), which
satisfy condition (96) and such that
$\nu_{\varepsilon}^{N}P_{0}=\nu_{\varepsilon}^{N+}P_{0}=\|A_{N}d^{0}\|^{2}$,
then the spectral density (98) of the one-sided moving-average sequence (99)
is the least favourable in the set $D_{\varepsilon}$ for the optimal
extrapolation of the functional $A_{N}{\xi}$. If
$\nu_{\varepsilon}^{N}<\nu_{\varepsilon}^{N+}$, then the least favourable in
the class $D_{\varepsilon}$ for the optimal extrapolation of the functional
$A_{N}{\xi}$ is the spectral density (112) which admits the canonical
factorization (90). The sequence $d=\left\\{d(k):k=0,1,\ldots\right\\}$ is
determined by the optimisation problem (78) and restrictions imposed on
densities by the given set of admissible spectral densities. The minimax
spectral characteristic of the optimal estimate of the functional $A_{N}{\xi}$
is calculated by the formula (74).

###  4.7 Least favourable spectral densities in the class $D_{1\varepsilon}$

Consider the problem of minimax estimation of the functionals $A{\xi}$ and
$A_{N}{\xi}$ which depend on the unknown values of a stationary stochastic
sequence ${\xi}(j)$ for the set of spectral densities that describes the model
of ”$\varepsilon$– neighborhood” in the space $L_{1}$ of a stationary
stochastic sequence

$D_{1\varepsilon}=\left\\{f(\lambda)\left|\,\frac{1}{2\pi}\int_{-\pi}^{\pi}\left|f(\lambda)-v(\lambda)\right|d\lambda\leq\varepsilon\right.\right\\},$

where $\varepsilon$ is a given numbers, $v(\lambda)$ is a given spectral
density.

From the condition $0\in\partial\Delta_{D}(f^{0})$ we find that the least
favourable spectral density for the optimal estimation of the functional
$A{\xi}$ is of the form

$f_{0}(\lambda)=\max\left\\{v(\lambda),c\left|\sum\limits_{k=0}^{\infty}(Ad)_{k}e^{ik\lambda}\right|^{2}\right\\}.$
(114)

Denote by $\nu_{1\varepsilon}P_{0}$ the maximum value $\|Ad^{0}\|^{2}$ of the
quantity $\|Ad\|^{2}$, where $d=\left\\{d(k):k=0,\,1,\,\ldots\right\\}$ are
solutions of the equation (87) which satisfy condition (88), the inequality

$\left|\sum\limits_{k=0}^{\infty}d(k)e^{-ik\lambda}\right|^{2}\geq
v(\lambda),$ (115)

and determine the canonical factorization (77) of the density
$f^{0}(\lambda)\in D_{1\varepsilon}$.

Denote by $\nu_{1\varepsilon}^{+}P_{0}$ the maximum value $\|Ad^{0}\|^{2}$ of
the quantity $\|Ad\|^{2}$, where $d=\left\\{d(k):k=0,1,\ldots\right\\}$
satisfies condition (88) and determine the canonical factorization (77) of the
density (114) from the class of admissible spectral densities
$D_{1\varepsilon}$.

The following theorem holds true.

######  Theorem 21.

If there exists a solution of the equation (87) which satisfy condition (88)
and such that
$\nu_{1\varepsilon}P_{0}=\nu_{1\varepsilon}^{+}P_{0}=\|Ad^{0}\|^{2}$, then the
spectral density (90) of the one-sided moving-average sequence (91) is the
least favourable in the set $D_{1\varepsilon}$ for the optimal extrapolation
of the functional $A{\xi}$. If $\nu_{1\varepsilon}<\nu_{1\varepsilon}^{+}$,
then the least favourable in the class $D_{1\varepsilon}$ for the optimal
extrapolation of the functional $A{\xi}$ is the spectral density (114) which
admits the canonical factorization (90). The sequence
$d=\left\\{d(k):k=0,1,\ldots\right\\}$ is determined by the optimisation
problem (78) and restrictions imposed on densities by the given set of
admissible spectral densities. The minimax spectral characteristic of the
optimal estimate of the functional $A{\xi}$ is calculated by the formula (71).

For the functional $A_{N}{\xi}$ the least favourable spectral density for the
optimal estimation of the functional is of the form

$f_{0}(\lambda)=\max\left\\{v(\lambda),c\left|\sum\limits_{k=0}^{N}(A_{N}d)_{k}e^{ik\lambda}\right|^{2}\right\\}.$
(116)

Denote by $\nu_{1\varepsilon}^{N}P_{0}$ the maximum value $\|A_{N}d^{0}\|^{2}$
of the quantity $\|A_{N}d\|^{2}$, where
$d=\left\\{d(k):k=0,1,\ldots,N\right\\}$ are solutions of the equations (94),
(95), which satisfy condition (96), the inequality

$\left|\sum\limits_{k=0}^{N}d(k)e^{-ik\lambda}\right|^{2}\geq v(\lambda),$
(117)

and determine the canonical factorization (80) of the density
$f^{0}(\lambda)\in D_{1\varepsilon}$.

Denote by $\nu_{1\varepsilon}^{N+}P_{0}$ the maximum value
$\|A_{N}d^{0}\|^{2}$ of the quantity $\|A_{N}d\|^{2}$, where
$d=\left\\{d(k):k=0,1,\ldots\right\\}$ satisfies condition (88) and determine
the canonical factorization (77) of the density (116) from the class of
admissible spectral densities $D_{1\varepsilon}$.

The following theorem holds true.

######  Theorem 22.

If there exists a solution of the equation (94), or equation (95), which
satisfy condition (96) and such that
$\nu_{1\varepsilon}^{N}P_{0}=\nu_{1\varepsilon}^{N+}P_{0}=\|A_{N}d^{0}\|^{2}$,
then the spectral density (98) of the one-sided moving-average sequence (99)
is the least favourable in the set $D_{1\varepsilon}$ for the optimal
extrapolation of the functional $A_{N}{\xi}$. If
$\nu_{1\varepsilon}^{N}<\nu_{1\varepsilon}^{N+}$, then the least favourable in
the class $D_{1\varepsilon}$ for the optimal extrapolation of the functional
$A_{N}{\xi}$ is the spectral density (116) which admits the canonical
factorization (90). The sequence $d=\left\\{d(k):k=0,1,\ldots\right\\}$ is
determined by the optimisation problem (78) and restrictions imposed on
densities by the given set of admissible spectral densities. The minimax
spectral characteristic of the optimal estimate of the functional $A_{N}{\xi}$
is calculated by the formula (74).

###  4.8 Least favourable spectral densities in the class $D_{2\varepsilon}$

Consider the problem of minimax estimation of the functionals $A{\xi}$ and
$A_{N}{\xi}$ which depend on the unknown values of a stationary stochastic
sequence ${\xi}(j)$ for the set of spectral densities that describes the model
of ”$\varepsilon$– neighborhood” in the space $L_{2}$ of a stationary
stochastic sequence

$D_{2\varepsilon}=\left\\{f(\lambda)\left|\,\frac{1}{2\pi}\int_{-\pi}^{\pi}\left|f(\lambda)-v(\lambda)\right|^{2}d\lambda\leq\varepsilon\right.\right\\},$

where $\varepsilon$ is a given numbers, $v(\lambda)$ is a given spectral
density.

From the condition $0\in\partial\Delta_{D}(f^{0})$ we find that the least
favourable spectral density for the optimal estimation of the functional
$A{\xi}$ is of the form

$f_{0}(\lambda)=\frac{v(\lambda)}{2}+\left(\frac{(v(\lambda))^{2}}{4}+\left|\sum\limits_{k=0}^{\infty}(Ad)_{k}e^{ik\lambda}\right|^{2}\right)^{1/2}.$
(118)

The sequence $d=\left\\{d(k):k=0,1,\ldots\right\\}$ is determined by equations
of the canonical factorization (90) of the density (118), optimisation problem
(78) and restriction

$\frac{1}{2\pi}\int_{-\pi}^{\pi}\left|f(\lambda)-v(\lambda)\right|^{2}d\lambda=\varepsilon.$
(119)

The following theorem holds true.

######  Theorem 23.

The least favourable in the set $D_{2\varepsilon}$ spectral density for the
optimal extrapolation of the functional $A{\xi}$ is determined by equation
(118). The sequence $d=\left\\{d(k):k=0,1,\ldots\right\\}$ is determined by
equations of the canonical factorization (90) of the density (118),
optimisation problem (78) and restriction (119) imposed on densities by the
given set of admissible spectral densities. The minimax spectral
characteristic of the optimal estimate of the functional $A{\xi}$ is
calculated by the formula (71).

For the functional $A_{N}{\xi}$ the least favourable spectral density for the
optimal estimation of the functional is of the form

$f_{0}(\lambda)=\frac{v(\lambda)}{2}+\left(\frac{(v(\lambda))^{2}}{4}+\left|\sum\limits_{k=0}^{N}(A_{N}d)_{k}e^{ik\lambda}\right|^{2}\right)^{1/2}.$
(120)

The sequence $d=\left\\{d(k):k=0,1,\ldots,N\right\\}$ is determined by
equations of the canonical factorization (90) of the density (120),
optimisation problem (78) and restriction (119).

######  Theorem 24.

The least favourable in the set $D_{2\varepsilon}$ spectral density for the
optimal extrapolation of the functional $A_{N}{\xi}$ is determined by equation
(120). The sequence $d=\left\\{d(k):k=0,1,\ldots,N\right\\}$ is determined by
equations of the canonical factorization (90) of the density (120),
optimisation problem (78) and restriction (119) imposed on densities by the
given set of admissible spectral densities. The minimax spectral
characteristic of the optimal estimate of the functional $A{\xi}$ is
calculated by the formula (74).

###  4.9 Conclusions

In this section we propose methods of solution of the problem of the mean-
square optimal linear estimation of the functionals
$A\xi=\sum\limits_{j=0}^{\infty}a(j)\xi(j)$ and
$A_{N}\xi=\sum\limits_{j=0}^{N}a(j)\xi(j)$ which depend on the unknown values
of the stationary stochastic sequence $\xi(j)$. Estimates are based on
observations of the sequence $\xi(j)$ at points of time $j=-1,-2,\dots$. The
problem is investigated in the case of spectral certainty, where the spectral
densities of the stationary stochastic sequence $\xi(j)$ is exactly known. In
this case the classical Hilbert space projection method of linear estimation
is applied. Formulas are derived for calculation the value of the mean square
errors and the spectral characteristics of the mean-square optimal estimates
of the linear functionals. In the case of spectral uncertainty, where the
spectral density of the stationary stochastic sequence is not exactly known,
but a class of admissible spectral densities is given the minimax-robust
procedure to linear estimation of the functionals is applied. Relations which
determine the least favourable spectral densities and the minimax spectral
characteristics are proposed for some given sets of admissible spectral
densities.

The minimax-robust approach to the problem of one step ahead optimal
prediction of the stationary stochastic sequences as well as estimation of one
missed value of the sequences based on convex optimization methods was
initiated in papers by Franke [9, 10], Franke and Poor [11]. See also papers
by Hosoya [19], Taniguchi [75], and survey paper by Kassam and Poor [23].

For the relative results on the mean-square optimal linear extrapolation of
linear functionals for stationary stochastic sequences and processes see
papers by Moklyachuk [39] – [45], [57], book by Moklyachuk and Masyutka [64].

##  5 Extrapolation problem for stationary sequences from observations with
noise

In this section we deal with the problem of the mean-square optimal estimation
of the linear functionals $A\xi=\sum\limits_{j=0}^{\infty}a(j)\xi(j)$ and
$A_{N}\xi=\sum\limits_{j=0}^{N}a(j)\xi(j)$ which depend on the unknown values
of a stationary stochastic sequence $\xi(j),j\in\mathbb{Z},$ from observations
of the sequence $\xi(j)+\eta(j)$ at points of time $j=-1,-2,\dots$, where
$\eta(j)$ is an uncorrelated with $\xi(j)$ stationary stochastic sequence. The
problem is investigated in the case of spectral certainty, where the spectral
densities of the stationary stochastic sequences $\xi(j)$ and $\eta(j)$ are
exactly known. In this case the classical Hilbert space projection method of
linear estimation is applied. Formulas are derived for calculation the value
of the mean square errors and the spectral characteristics of the mean-square
optimal estimates of the linear functionals. In the case of spectral
uncertainty, where the spectral densities of the stationary stochastic
sequences are not exactly known, but a class of admissible spectral densities
is given the minimax-robust procedure to linear estimation of the functional
is applied. Relations which determine the least favourable spectral densities
and the minimax spectral characteristics are proposed for some special sets of
admissible spectral densities.

###  5.1 The classical Hilbert space projection method of linear extrapolation

Let $\xi(j),j\in\mathbb{Z}$, and $\eta(j),j\in\mathbb{Z}$, be (wide sense)
stationary stochastic sequences with zero mathematical expectations
$E\xi(j)=0$, $E\eta(j)=0$. The correlation functions
$R_{\xi}(k)=E\xi(j+k)\overline{\xi(j)}$ and
$R_{\eta}(k)=E\eta(j+k)\overline{\eta(j)}$ of stationary stochastic sequences
$\xi(j),j\in\mathbb{Z}$, and $\eta(j),j\in\mathbb{Z}$, admit the spectral
representations

$R_{\xi}(k)=\int\limits_{-\pi}^{\pi}e^{ik\lambda}F(d\lambda),\quad
R_{\eta}(k)=\int\limits_{-\pi}^{\pi}e^{ik\lambda}G(d\lambda),$

where $F(d\lambda)$ and $G(d\lambda)$ are spectral measures of the sequences.
We will consider stationary stochastic sequences with absolutely continuous
spectral measures $F(d\lambda)$ and $G(d\lambda)$ and the correlation
functions of the form

$R_{\xi}(k)=\,\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}e^{ik\lambda}f(\lambda)d\lambda,\quad
R_{\eta}(k)=\,\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}e^{ik\lambda}g(\lambda)d\lambda,$

where $f(\lambda)$ and $g(\lambda)$ are the spectral density functions of the
sequences $\xi(j),j\in\mathbb{Z}$, and $\eta(j),j\in\mathbb{Z}$,
correspondingly.

We will suppose that the spectral density functions $f(\lambda)$ and
$g(\lambda)$ satisfy the minimality condition

$\int\limits_{-\pi}^{\pi}\frac{1}{f(\lambda)+g(\lambda)}d\lambda<\infty.$
(121)

Under this condition the error-free extrapolation of the unknown values of the
sequence $\xi(j)+\eta(j)$ is impossible.

The stationary stochastic sequences $\xi(j)$ and $\eta(j)$ admit the spectral
representations

$\xi(j)=\int\limits_{-\pi}^{\pi}e^{ij\lambda}dZ_{\xi}(\lambda),\hskip
28.45274pt\eta(j)=\int\limits_{-\pi}^{\pi}e^{ij\lambda}dZ_{\eta}(\lambda),$

where $Z_{\xi}(d\lambda)$ and $Z_{\eta}(d\lambda)$ are orthogonal stochastic
measures of the sequences $\xi(j)$ and $\eta(j)$ such that

$E{Z_{\xi}}(\Delta_{1})\overline{{Z_{\xi}}(\Delta_{2})}=F(\Delta_{1}\cap\Delta_{2})=\,\frac{1}{2\pi}\int_{\Delta_{1}\cap\Delta_{2}}f(\lambda)d\lambda,$
$E{Z_{\eta}}(\Delta_{1})\overline{{Z_{\eta}}(\Delta_{2})}=G(\Delta_{1}\cap\Delta_{2})=\,\frac{1}{2\pi}\int_{\Delta_{1}\cap\Delta_{2}}g(\lambda)d\lambda.$

Consider the problem of the mean-square optimal estimation of the linear
functional

$A\xi=\sum\limits_{j=0}^{\infty}a(j)\xi(j)$

which depends on the unknown values of a stationary stochastic sequence
$\xi(j),j=0,1,\dots,$ based on observations of the sequence $\xi(j)+\eta(j)$
at points of time $j=-1,-2,\dots$.

We will suppose that the sequence $\\{{a}(j):j=0,1,\ldots\\}$ which determines
the functional $A{\xi}$ satisfies the following conditions

$\sum_{j=0}^{\infty}\left|a(j)\right|<\infty,\quad\sum_{j=0}^{\infty}(j+1)\left|{a}(j)\right|^{2}<\infty.$
(122)

It follows from the spectral representation of the sequence $\xi(j)$ that we
can represent the functional $A\xi$ in the form

$A\xi=\int\limits_{-\pi}^{\pi}A(e^{i\lambda}){Z}_{\xi}(d\lambda),$ (123)

where

$A(e^{i\lambda})=\sum\limits_{j=0}^{\infty}a(j)e^{ij\lambda}.$

Denote by $\hat{A}\xi$ the mean square optimal linear estimate of the
functional $A\xi$ from observations of the sequence $\xi(j)+\eta(j)$ at points
of time $j=-1,-2,\dots$. Denote by
$\Delta(f,g)=E\left|A\xi-\hat{A}\xi\right|^{2}$ the mean square error of the
estimate $\hat{A}\xi$. To find the estimate $\hat{A}\xi$ we will use the
Hilbert space projection method proposed by Kolmogorov [24]. We will consider
$\xi(j),j\in\mathbb{Z}$, and $\eta(j),j\in\mathbb{Z}$, as elements of the
Hilbert space $H=L_{2}(\Omega,\mathcal{F},P)$ of complex valued random
variables with zero first moment, $E\xi=0$, finite second moment,
$E|\xi|^{2}<\infty$, and the inner product
$\left(\xi,{\eta}\right)=E\xi\overline{\eta}$.

Denote by $H^{0}(\xi+\eta)$ the subspace of the Hilbert space
$H=L_{2}(\Omega,\mathcal{F},P)$ generated by elements
$\\{\xi(j)+\eta(j):j=-1,-2,\dots\\}$. Denote by $L_{2}(f+g)$ the Hilbert space
of complex-valued functions that are square-integrable with respect to the
measure whose density is $f(\lambda)+g(\lambda).$ Denote by $L_{2}^{0}(f+g)$
the subspace of $L_{2}(f+g)$ generated by functions
$\\{e^{ij\lambda}:j=-1,-2,\dots\\}.$

The mean square optimal linear estimate $\hat{A}\xi$ of the functional $A\xi$
based on observations of the sequence $\xi(j)+\eta(j)$ at points of time
$j=-1,-2,\dots$ is an element of the $H^{0}(\xi+\eta)$. It can be represented
in the form

$\hat{A}\xi=\,\int\limits_{-\pi}^{\pi}h(e^{i\lambda})(Z_{\xi}(d\lambda)+Z_{\eta}(d\lambda)),$
(124)

where $h(e^{i\lambda})\in L_{2}^{0}(f+g)$ is the spectral characteristic of
the estimate $\hat{A}\xi.$

The mean square error $\Delta(h;f,g)$ of the estimate $\hat{A}\xi$ is given by
the formula

$\Delta(h;f,g)=E\left|A\xi-\hat{A}\xi\right|^{2}=\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}\left|A(e^{i\lambda})-h(e^{i\lambda})\right|^{2}f(\lambda)d\lambda+\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}\left|h(e^{i\lambda})\right|^{2}g(\lambda)d\lambda.$

The Hilbert space projection method proposed by A. N. Kolmogorov [24] makes it
possible to find the spectral characteristic $h(e^{i\lambda})$ and the mean
square error $\Delta(h;f,g)$ of the optimal linear estimate of the functional
$A\xi$ in the case where the spectral densities $f(\lambda)$ and $g(\lambda)$
of the sequences $\xi(j),j\in\mathbb{Z},$ and $\eta(j),j\in\mathbb{Z},$ are
exactly known and the minimality condition (121) is satisfied. The spectral
characteristic is determined by the following conditions:

$\begin{split}1)&\hat{A}\xi\in H^{0}(\xi+\eta),\\\ 2)&A\xi-\hat{A}\xi\bot
H^{0}(\xi+\eta).\end{split}$

It follows from the second condition that the following equations should be
satisfied

$\begin{split}&E\left(A\xi-\hat{A}\xi\right)\left(\overline{\xi(j)}+\overline{\eta(j)}\right)=\\\
&=\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}\left(A(e^{i\lambda})-h(e^{i\lambda})\right)e^{-ij\lambda}f(\lambda)d\lambda-\int\limits_{-\pi}^{\pi}h(e^{i\lambda})e^{-ij\lambda}g(\lambda)d\lambda=0,\,\,j=-1,-2,\dots.\end{split}$

The last equations are equivalent to equations

$\int\limits_{-\pi}^{\pi}\left[A(e^{i\lambda})f(\lambda)-h(e^{i\lambda})(f(\lambda)+g(\lambda))\right]e^{-ij\lambda}d\lambda=0,\,\,j=-1,-2,\dots.$

It follows from these equations that the function
$\left[A(e^{i\lambda})f(\lambda)-h(e^{i\lambda})(f(\lambda)+g(\lambda))\right]$
is of the form

$A(e^{i\lambda})f(\lambda)-h(e^{i\lambda})(f(\lambda)+g(\lambda))=C(e^{i\lambda}),$
(125) $C(e^{i\lambda})=\sum\limits_{j=0}^{\infty}c(j)e^{ij\lambda},$

where $c(j),j=0,1,\dots$ are unknown coefficients that we have to find.

From the relation (125) we deduce that the spectral characteristic
$h(e^{i\lambda})$ of the optimal linear estimate of the functional $A\xi$ is
of the form

$\begin{split}h(e^{i\lambda})=&\frac{A(e^{i\lambda})f(\lambda)-C(e^{i\lambda})}{f(\lambda)+g(\lambda)}=\\\
=A(e^{i\lambda})-&\frac{A(e^{i\lambda})g(\lambda)+C(e^{i\lambda})}{f(\lambda)+g(\lambda)}.\end{split}$
(126)

It follows from the first condition, which determines the spectral
characteristic $h(e^{i\lambda})\in L_{2}^{0}(f+g)$ of the optimal linear
estimate of the functional $A\xi$ that the Fourier coefficients of the
function $h(e^{i\lambda})$ are equal to zero for $j=0,1,\dots$, namely

$\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}h(e^{i\lambda})e^{-ij\lambda}d\lambda=0,\,\,j=0,1,\dots$

Using the last relations and (126) we get the following system of equations

$\begin{split}\int\limits_{-\pi}^{\pi}\left(A(e^{i\lambda})\frac{f(\lambda)}{f(\lambda)+g(\lambda)}-\frac{C(e^{i\lambda})}{f(\lambda)+g(\lambda)}\right)e^{-ij\lambda}d\lambda=0,\,\,j=0,1,\dots{}\end{split}$

These equations can be represented in the form

$\begin{split}\sum_{k=0}^{\infty}a(k)\int\limits_{-\pi}^{\pi}\frac{e^{i(k-j)\lambda}f(\lambda)}{f(\lambda)+g(\lambda)}d\lambda-\sum\limits_{k=0}^{\infty}c(k)\int\limits_{-\pi}^{\pi}\frac{e^{i(k-j)\lambda}}{f(\lambda)+g(\lambda)}d\lambda=0,\,\,j=0,1,\dots{}\end{split}$
(127)

Let us introduce the following notations

$R_{j,k}=\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}e^{-i(j-k)\lambda}\frac{f(\lambda)}{f(\lambda)+g(\lambda)}d\lambda;$
$B_{j,k}=\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}e^{-i(j-k)\lambda}\frac{1}{f(\lambda)+g(\lambda)}d\lambda;$
$Q_{j,k}=\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}e^{-i(j-k)\lambda}\frac{f(\lambda)g(\lambda)}{f(\lambda)+g(\lambda)}d\lambda.$

Making use the introduced notations we can write equations (127) in the form

$\begin{split}\sum\limits_{k=0}^{\infty}R_{j,k}a(k)=\sum\limits_{k=0}^{\infty}B_{j,k}c(k),\,\,j=0,1,\dots{}\end{split}$

The derived equations can be written in the form

$\begin{split}\mathbb{R}\mathbb{a}=\mathbb{B}\mathbb{c},\end{split}$

where $\mathbb{a}=(a(0),a(1),\dots)$ is a vector constructed from the
coefficients that determine the functional $A\xi$,
$\mathbb{c}=(c(0),c(1),\dots)$ is a vector constructed from the unknown
coefficients $c(k),k=0,1,\dots$, $\mathbb{B}$ and $\mathbb{R}$ are linear
operators in $\ell_{2}$, which are determined by matrices with elements
$(\mathbb{B})_{j,k}=B_{j,k}$, $(\mathbb{R})_{j,k}=R_{j,k}$, $j,k=0,1,\dots$

We get the formula

$\mathbb{c}=\mathbb{B}^{-1}\mathbb{R}\mathbb{a},$ (128)

Hence, the unknown coefficients $c(j),j=0,1,\dots$, are calculated by the
formula

$c(j)=\left(\mathbb{B}^{-1}\mathbb{R}\mathbb{a}\right)_{j},$

where $\left(\mathbb{B}^{-1}\mathbb{R}\mathbb{a}\right)_{j}$ is the $j$-th
component of the vector $\mathbb{B}^{-1}\mathbb{R}\mathbb{a},$ and the formula
for calculating the spectral characteristic of the estimate $\hat{A}\xi$ is of
the form

$\begin{split}h(e^{i\lambda})=A(e^{i\lambda})\frac{f(\lambda)}{f(\lambda)+g(\lambda)}-\frac{\sum\limits_{k=0}^{\infty}(\mathbb{B}^{-1}\mathbb{R}\mathbb{a})_{k}e^{ik\lambda}}{f(\lambda)+g(\lambda)}.\end{split}$
(129)

The mean square error of the estimate of the function can be calculated by the
formula

$\begin{split}\Delta(h;f,g)=E\left|A\xi-\hat{A}\xi\right|^{2}&=\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}\frac{\left|A(e^{i\lambda})g(\lambda)+\sum\limits_{k=0}^{\infty}(\mathbb{B}^{-1}\mathbb{R}\mathbb{a})_{k}e^{ik\lambda}\right|^{2}}{(f(\lambda)+g(\lambda))^{2}}f(\lambda)d\lambda\\\
&+\frac{1}{2\pi}\int\limits_{-\pi}^{\pi}\frac{\left|A(e^{i\lambda})f(\lambda)-\sum\limits_{k=0}^{\infty}(\mathbb{B}^{-1}\mathbb{R}\mathbb{a})_{k}e^{ik\lambda}\right|^{2}}{(f(\lambda)+g(\lambda))^{2}}g(\lambda)d\lambda\\\
&=\langle\mathbb{R}\mathbb{a},\mathbb{B}^{-1}\mathbb{R}\mathbb{a}\rangle+\langle\mathbb{Q}\mathbb{a},\mathbb{a}\rangle,\end{split}$
(130)

where
$\langle\mathbb{a},\mathbb{c}\rangle=\sum_{k=0}^{\infty}a(k)\overline{c(k)}$
is the inner product in the space $\ell_{2}$ and $\mathbb{Q}$ is a linear
operator in $\ell_{2}$, which is determined by the matrix with elements
$(\mathbb{Q})_{j,k}=Q_{j,k}$, $j,k=0,1,\dots$

Let us summarize our results and present them in the form of a theorem.

######  Theorem 5.1.

Let $\xi(j)$ and $\eta(j)$ be stationary stochastic sequences with the
spectral densities $f(\lambda)$ and $g(\lambda)$ that satisfy the minimality
condition (121). Let conditions (122) be satisfied. The spectral
characteristic $h(e^{i\lambda})$ and the mean square error $\Delta(h;f,g)$ of
the optimal linear estimate $\hat{A}\xi$ of the functional $A\xi$ from
observations of the sequence $\xi(j)+\eta(j)$ at points of time
$j=-1,-2,\dots$ can be calculated by formulas (129), (130).

###  5.2 Minimax-robust method of extrapolation

The traditional methods of estimation of the functional $A\xi$ which depends
on the unknown values of a stationary stochastic sequence $\xi(j)$ can be
applied in the case where the spectral densities $f(\lambda)$ and $g(\lambda)$
of the considered stochastic sequences $\xi(j)$ and $\eta(j)$ are exactly
known. In practise, however, we do not have complete information on spectral
densities of the sequences. For this reason we apply the minimax(robust)
method of estimation of the functional $A\xi$, that is we find an estimate
that minimizes the maximum of the mean square errors for all spectral
densities from the given class of admissible spectral densities $D$.

######  Definition 5.1.

For a given class of spectral densities $D=D_{f}\times D_{g}$ the spectral
densities $f_{0}(\lambda)\in D_{f}$, $g_{0}(\lambda)\in D_{g}$ are called the
least favourable in $D$ for the optimal linear estimation of the functional
$A\xi$ if the following relation holds true

$\Delta\left(f_{0},g_{0}\right)=\Delta\left(h\left(f_{0},g_{0}\right);f_{0},g_{0}\right)=\max\limits_{(f,g)\in
D_{f}\times D_{g}}\Delta\left(h\left(f,g\right);f,g\right).$

######  Definition 5.2.

For a given class of spectral densities $D=D_{f}\times D_{g}$ the spectral
characteristic $h^{0}(e^{i\lambda})$ of the optimal linear estimate of the
functional $A\xi$ is called minimax-robust if

$h^{0}(e^{i\lambda})\in H_{D}=\bigcap\limits_{(f,g)\in D_{f}\times
D_{g}}L_{2}^{0}(f+g),$ $\min\limits_{h\in H_{D}}\max\limits_{(f,g)\in
D}\Delta\left(h;f,g\right)=\sup\limits_{(f,g)\in
D}\Delta\left(h^{0};f,g\right).$

It follows from the introduced definitions and the obtained formulas that the
following statement holds true.

######  Lemma 5.1.

The spectral densities $f_{0}(\lambda)\in D_{f}$, $g_{0}(\lambda)\in D_{g}$
are the least favourable in the class of admissible spectral densities
$D=D_{f}\times D_{g}$ for the optimal linear estimate of the functional $A\xi$
if the Fourier coefficients of the functions

$(f_{0}(\lambda)+g_{0}(\lambda))^{-1},\quad
f_{0}(\lambda)(f_{0}(\lambda)+g_{0}(\lambda))^{-1},\quad
f_{0}(\lambda)g_{0}(\lambda)(f_{0}(\lambda)+g_{0}(\lambda))^{-1}$

define operators $\mathbb{B}^{0},\mathbb{R}^{0},\mathbb{Q}^{0}$ that determine
a solution to the optimization problem

$\begin{split}\max\limits_{(f,g)\in D_{f}\times
D_{g}}\langle\mathbb{R}\mathbb{a},\mathbb{B}^{-1}\mathbb{R}\mathbb{a}\rangle+\langle\mathbb{Q}\mathbb{a},\mathbb{a}\rangle\\\