$\bm{\varsigma}^{\prime}\in\mathbb{Z}^{3}$. From (2), we have
$\mathcal{J}\left[(\delta_{\ell 1}\ \cdots\ \delta_{\ell
n})e^{i\langle\bm{\varsigma},\cdot\rangle}\right]=x_{\ell\bm{\varsigma}}$, by
linearity and for a sequence of complex numbers
$\\{\alpha_{\bm{\varsigma}}\\}$, we have

$\mathcal{J}\left[\sum_{\bm{\varsigma}}\alpha_{\bm{\varsigma}}(\delta_{\ell
1}\ \cdots\ \delta_{\ell
n})e^{i\langle\bm{\varsigma},\cdot\rangle}\right]=\sum_{\bm{\varsigma}}\alpha_{\bm{\varsigma}}x_{\ell\bm{\varsigma}},$

which is an onto linear mapping between two spaces: the collection of finite
combinations of $\\{(\delta_{\ell 1}\ \cdots\ \delta_{\ell
n})e^{i\langle\bm{\varsigma},\cdot\rangle}:\bm{\varsigma}\in\mathbb{Z}^{3}\\}$
and all finite linear combinations of
$\\{x_{\ell\bm{\varsigma}}:\bm{\varsigma}\in\mathbb{Z}^{3}\\}$. We remark that
these two spaces are dense manifolds of
$L^{\infty}_{2}(\bm{\Theta},\mathbb{C},\bm{\Sigma}^{x})$ and
$\bm{\mathcal{X}}$, respectively.

To show that $\mathcal{J}$ preserves the inner product, let us consider the
complex sequences $\\{\alpha_{\bm{\varsigma}}\\}$ and
$\\{\beta_{\bm{\varsigma}^{\prime}}\\}$ and define the $n$ dimensional vector
$\bm{A}_{n\bm{\varsigma}}=(0\ \ \cdots\ \alpha_{\bm{\varsigma}}\ \cdots\
0)^{\top}$, which has all entries equal to zero, but the $\ell$-th entry which
is equal to $\alpha_{\bm{\varsigma}}$. Similarly, we define the $n$
dimensional vector $\bm{B}_{n\bm{\varsigma}^{\prime}}=(0\ \ \cdots\
\beta_{\bm{\varsigma}^{\prime}}\ \cdots\ 0)^{\top}$, which has all entries
equal to zero, but the $\ell$-th entry which is equal to
$\beta_{\bm{\varsigma}^{\prime}}$. Then, we have

$\displaystyle\left\langle\mathcal{J}\left[\sum_{\bm{\varsigma}}\alpha_{\bm{\varsigma}}(\delta_{\ell
1}\ \cdots\ \delta_{\ell
n})e^{i\langle\bm{\varsigma},\cdot\rangle}\right],\mathcal{J}\left[\sum_{\bm{\varsigma}}\beta_{\bm{\varsigma}}(\delta_{\ell
1}\ \cdots\ \delta_{\ell
n})e^{i\langle\bm{\varsigma},\cdot\rangle}\right]\right\rangle$ (A.1)
$\displaystyle=$
$\displaystyle\left\langle\sum_{\bm{\varsigma}}\alpha_{\bm{\varsigma}}{x}_{\ell\bm{\varsigma}},\sum_{\bm{\varsigma}}\beta_{\bm{\varsigma}}{x}_{\ell\bm{\varsigma}}\right\rangle=\sum_{\bm{\varsigma}}\sum_{\bm{\varsigma}^{\prime}}\alpha_{\bm{\varsigma}}{\beta^{{\dagger}}_{\bm{\varsigma}^{\prime}}}\left\langle{x}_{\ell\bm{\varsigma}},{x}_{\ell\bm{\varsigma}^{\prime}}\right\rangle$
$\displaystyle=$
$\displaystyle\sum_{\bm{\varsigma}}\sum_{\bm{\varsigma}^{\prime}}\bm{A}_{n\bm{\varsigma}}\bm{\Gamma}^{x}_{n}(\bm{\varsigma}-\bm{\varsigma}^{\prime}){\bm{B}_{n\bm{\varsigma}^{\prime}}^{{\dagger}}}$
$\displaystyle=$
$\displaystyle\frac{1}{8\pi^{3}}\sum_{\bm{\varsigma}}\sum_{\bm{\varsigma}^{\prime}}\bm{A}_{n\bm{\varsigma}}\left[\int_{\bm{\Theta}}e^{i\langle\bm{\varsigma}-\bm{\varsigma}^{\prime},\bm{\theta}\rangle}\bm{\Sigma}_{n}^{x}(\bm{\theta})d\bm{\theta}\right]{\bm{B}_{n\bm{\varsigma}^{\prime}}^{{\dagger}}}$
$\displaystyle=$
$\displaystyle\left\langle\sum_{\bm{\varsigma}}\alpha_{\bm{\varsigma}}e^{i\langle\bm{\varsigma},\bm{\theta}\rangle},\sum_{\bm{\varsigma}}{\beta_{\bm{\varsigma}}}e^{i\langle\bm{\varsigma},\bm{\theta}\rangle}\right\rangle_{\bm{\Sigma}_{n}^{x}(\bm{\theta})},$

which implies that $\mathcal{J}$ is a one-to-one mapping and it preserves the
inner product. The above properties hold for $\mathcal{J}$ applied just to
$(\delta_{\ell 1}\ \cdots\ \delta_{\ell
n})e^{i\langle\bm{\varsigma},\cdot\rangle}$, which yields
$x_{\ell\bm{\varsigma}}$. Nevertheless, $\mathcal{J}$ is linear thus the same
properties hold when applying it to get all the elements of
$\bm{\mathcal{X}}$.

We have shown that $\mathcal{J}$ is an isometric isomorphism between two dense
linear manifolds. Lemma 4.1 in Rozanov (1967, p.14) implies that one can
always extend the isomorphism to the closed linear manifolds generated by
these manifolds. With a slight abuse of notation, we call $\mathcal{J}$ the
extended isomorphism such that
$\mathcal{J}:L^{\infty}_{2}(\bm{\Theta},\mathbb{C},\bm{\Sigma}^{x})\to\bm{\mathcal{X}}$.
That concludes the proof. ∎

######  Remark A.1.

A consequence of Lemma A.2 is that also the inverse mapping of $\mathcal{J}$,
let us call it $\mathcal{J}^{-1}:\bm{\mathcal{X}}\to
L^{\infty}_{2}(\bm{\Theta},\mathbb{C},\bm{\Sigma}^{x})$, is an isomorphism,
such that for $\bm{\varsigma}\in\mathbb{Z}^{3}$, any $n\in\mathbb{N}$, and any
$\ell\leq n$,

$\mathcal{J}^{-1}\left[x_{\ell\bm{\varsigma}}\right]=(\delta_{\ell 1}\
\cdots\delta_{\ell k}\cdots\delta_{\ell
n})e^{i\langle\bm{\varsigma},\cdot\rangle}.$ (A.2)

Moreover, Lemma A.2 implies that for any $n\in\mathbb{N}$, the process
$\bm{x}_{n}$ is harmonizable, namely for any
$\bm{\varsigma}\in\mathbb{Z}^{3}$, we can write

$\bm{x}_{n\bm{\varsigma}}=\int_{\bm{\Theta}}e^{i\langle\bm{\varsigma},\bm{\theta}\rangle}\bm{\mathcal{M}}_{n}(\bm{\theta})\mathrm{d}\bm{\theta},\qquad\text{w.p.
1},$ (A.3)

where $\bm{\mathcal{M}}_{n}$ is a complex random measure on (the Borel
$\sigma$-field of) $\bm{\Theta}$, which is such that
$\bm{\mathcal{M}}_{n}(\bm{\Delta_{1}}\cup\bm{\Delta_{2}})=\bm{\mathcal{M}}_{n}(\bm{\Delta_{1}})+\bm{\mathcal{M}}_{n}(\bm{\Delta_{2}})$,
for disjoint Borel sets $\bm{\Delta_{1}}$ and $\bm{\Delta_{2}}$. In addition,
for any Borel set $\bm{\Delta}$, we have that
$\textrm{E}(\bm{\mathcal{M}}_{n}(\bm{\Delta}))=\bm{0}_{n}$,
$\textrm{E}(|\bm{\mathcal{M}}_{n}(\bm{\Delta})|^{2})=\int_{\bm{\Delta}}\bm{\Sigma}_{n}^{x}(d\bm{\theta})/8\pi^{3}$
and
$\textrm{E}(\bm{\mathcal{M}}_{n}(\bm{\Delta_{1}})\bm{\mathcal{M}}^{{\dagger}}_{n}(\bm{\Delta_{2}}))=\bm{0}_{n\times
n}$, for all disjoint Borel sets $\bm{\Delta_{1}}$ and $\bm{\Delta_{2}}$; see,
e.g., Stein (2012), p.21 or Cressie and Wikle (2015) Ch.6, section 6.1.6, for
book-length discussions. The result in (A.3) is analogous to the standard
spectral representation of time series, where $\bm{\mathcal{M}}_{n}$ is the
measure related to an orthogonal increment process; see e.g. Brockwell and
Davis (2006), Ch.4.

######  Remark A.2.

To understand the importance of linear filter, notice that any scalar rf of
the form $y_{\bm{\varsigma}}=\bm{\underline{b}}(L)\bm{x}_{\bm{\varsigma}}$ is
such that $y\in\bm{\mathcal{X}}$ and therefore is co-homostationary with
$x_{\ell}$ for any $\ell\in\mathbb{N}$. Moreover, $y$ has a scalar spectral
density $\sigma^{y}(\bm{\theta})$. In particular, recalling the definition of
inner product in the Hilbert space $L_{2}(\mathcal{P},\mathbb{C})$, we have by
Lemma A.2 and (7), for any $\bm{\varsigma},\bm{h}\in\mathbb{Z}^{3}$

$\displaystyle\textrm{E}(y_{\bm{\varsigma}}y_{\bm{\varsigma}-\bm{h}}^{\dagger})$
$\displaystyle=\langle
y_{\bm{\varsigma}},y_{\bm{\varsigma}-\bm{h}}\rangle=\langle\mathcal{J}^{-1}[y_{\bm{\varsigma}}],\mathcal{J}^{-1}[y_{\bm{\varsigma}-\bm{h}}]\rangle_{\bm{\Sigma}^{x}}$
$\displaystyle=\frac{1}{8\pi^{3}}\int_{\bm{\Theta}}e^{i\langle\bm{\varsigma},\bm{\theta}\rangle}\left[\left(\sum_{\bm{\kappa}}\mathbf{b}_{\bm{\kappa}}e^{-i\langle\bm{\kappa},\bm{\theta}\rangle}\right)\bm{\Sigma}^{x}(\bm{\theta})\left(\sum_{\bm{\kappa}}\mathbf{b}_{\bm{\kappa}}e^{-i\langle\bm{\kappa},\bm{\theta}\rangle}\right)^{\dagger}\right]e^{-i\langle\bm{\varsigma}-\bm{h},\bm{\theta}\rangle}\mathrm{d}\bm{\theta}$
$\displaystyle=\frac{1}{8\pi^{3}}\int_{\bm{\Theta}}\left[\left(\sum_{\bm{\kappa}}\mathbf{b}_{\bm{\kappa}}e^{-i\langle\bm{\kappa},\bm{\theta}\rangle}\right)\bm{\Sigma}^{x}(\bm{\theta})\left(\sum_{\bm{\kappa}}\mathbf{b}_{\bm{\kappa}}e^{-i\langle\bm{\kappa},\bm{\theta}\rangle}\right)^{\dagger}\right]e^{i\langle\bm{h},\bm{\theta}\rangle}\mathrm{d}\bm{\theta}$
$\displaystyle=\frac{1}{8\pi^{3}}\int_{\bm{\Theta}}\sigma^{y}(\bm{\theta})e^{i\langle\bm{h},\bm{\theta}\rangle}\mathrm{d}\bm{\theta},$

which shows that, for any $\bm{\theta}\in\bm{\Theta}$, the spectral density of
$y$ is

$\sigma^{y}(\bm{\theta})=\left(\sum_{\bm{\kappa}}\mathbf{b}_{\bm{\kappa}}e^{-i\langle\bm{\kappa},\bm{\theta}\rangle}\right)\bm{\Sigma}^{x}(\bm{\theta})\left(\sum_{\bm{\kappa}}\mathbf{b}_{\bm{\kappa}}e^{-i\langle\bm{\kappa},\bm{\theta}\rangle}\right)^{\dagger}=\left(\sum_{\bm{\kappa}}\mathbf{b}_{\bm{\kappa}}e^{-i\langle\bm{\kappa},\bm{\theta}\rangle}\right)\bm{\Sigma}^{x}(\bm{\theta})\left(\sum_{\bm{\kappa}}\mathbf{b}_{\bm{\kappa}}e^{i\langle\bm{\kappa},\bm{\theta}\rangle}\right).$

The same reasoning can be easily generalized to the case of $m$-dimensional
linear filters, for any $m\in\mathbb{N}$, defining filtered rf as
$\bm{y}_{m\bm{\varsigma}}=\underline{\bm{B}}(L)\bm{x}_{\bm{\varsigma}}=\sum_{\bm{\kappa}}\bm{B}_{\bm{\kappa}}L^{\bm{\kappa}}\bm{x}_{\bm{\varsigma}}$,
where $\bm{B}_{\bm{\kappa}}$ is a complex matrix with $m$ rows and infinite
columns and such that $\bm{y}_{m\bm{\varsigma}}\in\bm{\mathcal{X}}$ for any
$\bm{\varsigma}\in\mathbb{Z}^{3}$. For $j=1,\ldots,m$, each component
$y_{j\bm{\varsigma}}=\underline{\bm{b}}_{j}(L)\bm{x}_{\bm{\varsigma}}$
satisfies the above properties.

######  Remark A.3.

If $\bm{x}_{n}$ is a white noise rf for any $n\in\mathbb{N}$, then it has a
constant spectral density matrix.

##  Appendix B Proof of results of Section 4

###  B.1 Weyl’s inequality

For the sake of completeness, we recall some properties of the eigenvalues of
Hermitian nonnegative definite matrices, that go under the name of Weyl’s
inequality.

  * (a)

Let $\mathbf{D}$ and $\mathbf{E}$ be $m\times m$ Hermitian nonnegative
definite and $\bm{F}=\bm{D}+\bm{E}$. Then

$\nu_{s}(\bm{F})\leq\nu_{s}(\bm{D})+\nu_{1}(\bm{E}),\quad\nu_{s}(\bm{F})\leq\nu_{1}(\bm{D})+\nu_{s}(\bm{E}),\quad\nu_{s}(\bm{F})\geq\nu_{s}(\bm{D}),\quad\nu_{s}(\bm{F})\geq\nu_{s}(\bm{E})$

for any $s=1,\ldots,m$.

  * (b)

Let $\mathbf{D}$ be as in (a) and let $\mathbf{G}$ be the top-left
$(m-1)\times(m-1)$ submatrix of $\bm{D}$. Then
$\nu_{s}(\bm{D})\geq\nu_{s}(\bm{G})$ for $s=1,\ldots,m-1$.

###  B.2 Proof of Theorem 4.1 \- sufficient condition

The proof of the sufficient condition is based on a series of intermediate
results. In the sequel, for ease of notation, when we write matrix products
$\bm{A}\bm{B}$ but the number of columns of $\bm{A}$ is smaller than the
number of rows of $\bm{B}$ we mean that $\bm{A}$ has been augmented with
columns of zeros to match the number of rows of $\bm{B}$. For example, for
$m<n$, $\underline{\bm{P}}_{m}(L)\bm{x}_{n\bm{\varsigma}}$ means nothing but
$\underline{\bm{P}}_{m}(L)\bm{x}_{m\bm{\varsigma}}$.

####  B.2.1 Construction of $q$-dimensional orthonormal white noise rf

We start proving that there exists a converging sequence of $q$-dimensional
orthonormal white noise rf which belongs to $\bm{\mathcal{X}}$. For a given
integer $q\leq n$ and any $\bm{\theta}\in\bm{\Theta}$, let
$\bm{P}_{n}(\bm{\theta})=(\bm{p}_{n1}^{x\top}(\bm{\theta})\cdots\bm{p}_{nq}^{x\top}(\bm{\theta}))^{\top}$
be the $q\times n$ matrix having as rows the normalized eigenvectors of
$\bm{\Sigma}_{n}^{x}(\bm{\theta})$ corresponding to the $q$ largest
eigenvectors. Similarly let
$\bm{Q}_{n}(\bm{\theta})=(\bm{p}_{nq+1}^{x\top}(\bm{\theta})\cdots\bm{p}_{nn}^{x\top}(\bm{\theta}))^{\top}$
which is $(n-q)\times n$. Define also $\bm{\Lambda}_{n}(\bm{\theta})$ as the
$q\times q$ diagonal matrix containing on the diagonal the largest $q$
eigenvalues $\lambda^{x}_{nj}(\bm{\theta})$, $j=1,\ldots,q$ and denote by
$\bm{\Phi}_{n}(\bm{\theta})$ the $(n-q)\times(n-q)$ diagonal matrix containing
on the diagonal the remaining eigenvalues $\lambda^{x}_{nj}(\bm{\theta})$,
$j=q+1,\ldots,n$. The spectral decomposition yields

$\displaystyle\bm{\Sigma}_{n}^{x}(\bm{\theta})$ $\displaystyle=$
$\displaystyle{\bm{P}}^{\dagger}_{n}(\bm{\theta})\bm{\Lambda}_{n}(\bm{\theta})\bm{P}_{n}(\bm{\theta})+{\bm{Q}}^{\dagger}_{n}(\bm{\theta})\bm{\Phi}_{n}(\bm{\theta})\bm{Q}_{n}(\bm{\theta}),$
$\displaystyle\mathbf{I}_{n}$ $\displaystyle=$
$\displaystyle{\bm{P}}^{\dagger}_{n}(\bm{\theta}){\bm{P}_{n}}(\bm{\theta})+{\bm{Q}}^{\dagger}_{n}(\bm{\theta}){\bm{Q}}_{n}(\bm{\theta}),$
(B.4) $\displaystyle\mathbf{I}_{q}$ $\displaystyle=$
$\displaystyle{\bm{P}_{n}}(\bm{\theta}){\bm{P}^{\dagger}_{n}}(\bm{\theta}).$

Now, let $\underline{\bm{P}}_{n}(L)$ and
$\underline{\bm{\Lambda}}^{-1/2}_{n}(L)$ be the spatio-temporal linear filters
built as in (7) from $\bm{P}_{n}(\bm{\theta})$ and
$\bm{\Lambda}^{-1/2}_{n}(\bm{\theta})$, respectively. Notice that since
$\bm{\Sigma}_{n}^{x}(\bm{\theta})$ is positive definite by construction, then
$\bm{\Lambda}_{n}(\bm{\theta})$ is bounded away from zero for all
$n\in\mathbb{N}$ and all $\bm{\theta}\in\bm{\Theta}$ and also bounded for any
fixed $n$. So, recall the notation in (8),

${\bm{\psi}}^{n}_{\bm{\varsigma}}=\left(\psi^{n}_{1\bm{\varsigma}}\cdots\psi^{n}_{q\bm{\varsigma}}\right)^{\top}=\underline{\bm{\Lambda}}^{-1/2}_{n}(L)\star\underline{\bm{P}}_{n}(L)\bm{x}_{n\bm{\varsigma}}$
(B.5)

is well defined and it is a $q$-dimensional orthonormal white noise rf since
by (B.4) its spectral density is (see Remark A.2):
$\bm{\Sigma}^{\bm{\psi}^{n}}(\bm{\theta})=\mathbf{I}_{q}$.

For $\bm{M}\subset\bm{\Theta}$, let $\bm{K}_{M}\subset L^{q\times
q}_{\infty}(\bm{\Theta},\mathbb{C})$ whose elements $\bm{C}$ are $q\times q$
complex matrices with elements being functions defined on $\bm{\Theta}$ and
such that: (a) $\bm{C}(\bm{\theta})=\bm{0}$ for $\bm{\theta}\notin\bm{M}$, (b)
$\bm{C}(\bm{\theta}){\bm{C}}^{\dagger}(\bm{\theta})=\mathbf{I}_{q}$ for
$\bm{\theta}\in\bm{M}$. For any $m,n\in\mathbb{N}$, let $\bm{C}\in\bm{K}_{M}$,
so that
$\underline{\bm{C}}(L){\bm{\psi}}^{m}_{\bm{\varsigma}}\in\bm{\mathcal{X}}$ and
it is still a $q$-dimensional orthonormal white noise rf. Let us consider the
orthogonal projection of this new white noise on the space
$\overline{\text{span}}(\\{\psi^{n}_{j\bm{\varsigma}},j=1,\ldots,q,\bm{\varsigma}\in\mathbb{Z}^{3}\\})$.
To this end, note that (B.4) yields
$\bm{x}_{n\bm{\varsigma}}=\underline{{\bm{P}}}^{\dagger}_{n}(L)\star\underline{{\bm{P}}}_{n}(L)\bm{x}_{n\bm{\varsigma}}+\underline{{\bm{Q}}}^{\dagger}_{n}(L)\star\underline{{\bm{Q}}}_{n}(L)\bm{x}_{n\bm{\varsigma}}$,
thus

$\bm{x}_{n\bm{\varsigma}}=\underline{{\bm{P}}}^{\dagger}_{n}(L)\star\underline{\bm{\Lambda}}^{1/2}_{n}(L){\bm{\psi}}^{n}_{\bm{\varsigma}}+\underline{{\bm{Q}}}^{\dagger}_{n}(L)\star\underline{{\bm{Q}}}_{n}(L)\bm{x}_{n\bm{\varsigma}},$
(B.6)

where ${{\bm{P}}}^{\dagger}_{n}{\bm{\Lambda}}^{1/2}_{n}\in L^{n\times
q}_{2}(\bm{\Theta},\mathbb{C})$ due to integrability of the dynamic spatio-
temporal eigenvalues (see Remark 3.1). Moreover, since
$\bm{Q}_{n}(\bm{\theta})\bm{\Sigma}_{n}^{x}(\bm{\theta}){{\bm{P}}}^{\dagger}_{n}(\bm{\theta})=\bm{\Phi}_{n}(\bm{\theta})\bm{Q}_{n}(\bm{\theta}){{\bm{P}}}^{\dagger}_{n}(\bm{\theta})=\bm{0}$
for all $\bm{\theta}\in\bm{\Theta}$, the two terms on the right-hand side of
(B.6) are orthogonal at any lead and lag element by element. Therefore, the
first term is the projection of $\bm{x}_{n\bm{\varsigma}}$ on
$\overline{\text{span}}(\\{\psi^{n}_{j\bm{\varsigma}},j=1,\ldots,q,\bm{\varsigma}\in\mathbb{Z}^{3}\\})$
and the second term is the residual. By applying
$\underline{\bm{C}}(L)\star\underline{\bm{\Lambda}}^{-1/2}_{m}(L)\star\underline{\bm{P}}_{m}(L)$
to both sides of (B.6) and using that
$\underline{\bm{\Lambda}}^{-1/2}_{m}(L)\star\underline{\bm{P}}_{m}(L)\bm{x}_{n\bm{\varsigma}}=\underline{\bm{\Lambda}}^{-1/2}_{m}(L)\star\underline{\bm{P}}_{m}(L)\bm{x}_{m\bm{\varsigma}}={\bm{\psi}}^{m}_{\bm{\varsigma}}$,
we obtain the projection as follows

$\underline{\bm{C}}(L){\bm{\psi}}^{m}_{\bm{\varsigma}}=\underline{\bm{D}}(L){\bm{\psi}}^{n}_{\bm{\varsigma}}+\underline{\bm{R}}(L)\bm{x}_{n\bm{\varsigma}},$
(B.7)

where

${\bm{D}}(\bm{\theta})={\bm{C}}(\bm{\theta}){\bm{\Lambda}}^{-1/2}_{m}(\bm{\theta}){\bm{P}}_{m}(\bm{\theta}){\bm{P}}^{\dagger}_{n}(\bm{\theta}){\bm{\Lambda}}^{1/2}_{n}(\bm{\theta}),\quad{\bm{R}}(\bm{\theta})={\bm{C}}(\bm{\theta})\bm{\Lambda}_{m}^{-1/2}(\bm{\theta})\bm{P}_{m}(\bm{\theta}){\bm{Q}}^{\dagger}_{n}(\bm{\theta})\bm{Q}_{n}(\bm{\theta}).$
(B.8)

Notice that by taking the spectral density matrices of both sides of (B.7)
yields

$\mathbf{I}_{q}=\bm{D}(\bm{\theta}){\bm{D}}^{\dagger}(\bm{\theta})+\bm{R}(\bm{\theta})\bm{\Sigma}_{n}^{x}(\bm{\theta}){\bm{R}}^{\dagger}(\bm{\theta}).$
(B.9)

Now, let us denote by $\mu(\bm{\theta})$ the largest eigenvalue of the
spectral density matrix of $\underline{\bm{R}}(L)\bm{x}_{n\bm{s}t}$. Then, we
have

######  Lemma B.1.

Suppose that (i) and (ii) of Theorem 4.1, and Assumptions 3.1 and 3.2 hold.
Then, for $m<n$ and $\bm{C}\in\bm{K}_{M}$,
$\mu(\bm{\theta})\leq\lambda_{n,q+1}^{x}(\bm{\theta})/\lambda_{mq}^{x}(\bm{\theta}).$

###### Proof.

Due to (B.4) both
$\mathbf{I}_{n}-{\bm{Q}}^{\dagger}_{n}(\bm{\theta}){\bm{Q}}_{n}(\bm{\theta})$
and
$\lambda_{n,q+1}^{x}(\bm{\theta}){\bm{Q}}^{\dagger}_{n}(\bm{\theta}){\bm{Q}}_{n}(\bm{\theta})-{\bm{Q}}^{\dagger}_{n}(\bm{\theta})\bm{\Phi}_{n}(\bm{\theta}){\bm{Q}}_{n}(\bm{\theta})$
are non-negative definite. Therefore,
$\lambda_{n,q+1}^{x}(\bm{\theta})\mathbf{I}_{n}-{\bm{Q}}^{\dagger}_{n}(\bm{\theta})\bm{\Phi}_{n}(\bm{\theta}){\bm{Q}}_{n}(\bm{\theta})$
is non-negative definite too, which implies that

$\bm{C}(\bm{\theta}){\bm{\Lambda}}^{-1/2}_{m}(\bm{\theta})\bm{P}_{m}(\bm{\theta})(\lambda_{n,q+1}^{x}(\bm{\theta})\mathbf{I}_{n}-{\bm{Q}}^{\dagger}_{n}(\bm{\theta})\bm{\Phi}_{n}(\bm{\theta}){\bm{Q}}_{n}(\bm{\theta})){\bm{P}}^{\dagger}_{m}(\bm{\theta}){\bm{\Lambda}}^{-1/2}_{m}(\bm{\theta}){\bm{C}}^{\dagger}(\bm{\theta})$

is also non-negative definite. Moreover,

$\displaystyle\bm{C}(\bm{\theta}){\bm{\Lambda}}^{-1/2}_{m}(\bm{\theta})\bm{P}_{m}(\bm{\theta})(\lambda_{n,q+1}^{x}(\bm{\theta})\mathbf{I}_{n}$
$\displaystyle-{\bm{Q}}^{\dagger}_{n}(\bm{\theta})\bm{\Phi}_{n}(\bm{\theta}){\bm{Q}}_{n}(\bm{\theta})){\bm{P}}^{\dagger}_{m}(\bm{\theta}){\bm{\Lambda}}^{-1/2}_{m}(\bm{\theta}){\bm{C}}^{\dagger}(\bm{\theta})$
$\displaystyle=\lambda_{n,q+1}^{x}(\bm{\theta})\bm{C}(\bm{\theta}){\bm{\Lambda}}^{-1}_{m}(\bm{\theta}){\bm{C}}^{\dagger}(\bm{\theta})-\bm{R}(\bm{\theta})\bm{\Sigma}_{n}^{x}(\bm{\theta}){\bm{R}}^{\dagger}(\bm{\theta}),$
(B.10)

where we obtain the second term making use of
$\bm{\Phi}_{n}(\bm{\theta})=\bm{Q}_{n}(\bm{\theta})\bm{\Sigma}_{n}^{x}(\bm{\theta}){\bm{Q}}^{\dagger}_{n}(\bm{\theta})$.
Letting
$\bm{A}(\bm{\theta})=\lambda_{n,q+1}^{x}(\bm{\theta})\bm{C}(\bm{\theta}){\bm{\Lambda}}^{-1}_{m}(\bm{\theta}){\bm{C}}^{\dagger}(\bm{\theta})$
and
$\bm{B}(\bm{\theta})=\bm{R}(\bm{\theta})\bm{\Sigma}_{n}^{x}(\bm{\theta}){\bm{R}}^{\dagger}(\bm{\theta})$
we have that

$\nu_{\max}\left(\bm{A}(\bm{\theta})\right)-\nu_{\max}(\bm{B}(\bm{\theta}))\geq\nu_{\min}\left(\bm{A}(\bm{\theta})\right)-\nu_{\max}(\bm{B}(\bm{\theta}))\geq\nu_{\min}\left(\bm{A}(\bm{\theta})-\bm{B}(\bm{\theta})\right)\geq
0,$

where the left inequality follows from Weyl’s inequality; see Appendix B.1.
Then,

$0\leq\nu_{\max}\left(\bm{A}(\bm{\theta})\right)-\nu_{\max}(\bm{B}(\bm{\theta}))=\lambda_{n,q+1}^{x}(\bm{\theta})\nu_{\max}\left(\bm{C}(\bm{\theta}){\bm{\Lambda}}^{-1}_{m}(\bm{\theta}){\bm{C}}^{\dagger}(\bm{\theta})\right)-\mu(\bm{\theta}).$

Using
$\nu_{\max}\left(\bm{C}(\bm{\theta}){\bm{\Lambda}}^{-1}_{m}(\bm{\theta}){\bm{C}}^{\dagger}(\bm{\theta})\right)=1/\nu_{\min}({\bm{\Lambda}}_{m}(\bm{\theta}))=1/\lambda_{mq}^{x}(\bm{\theta})$
concludes the proof. ∎

With the projection as in (B.7), we are now ready to construct our converging
sequence. Under (i) and (ii) of Theorem 4.1, there exists a set
$\bm{\Pi}\subseteq\bm{\Theta}$ and a real number $W$ such that
$\bm{\Theta}\setminus\bm{\Pi}$ has null measure and (1)
$\lambda_{nq+1}^{x}(\bm{\theta})\leq W$ for any $n\in\mathbb{N}$ and any
$\bm{\theta}\in\bm{\Pi}$; (2)
$\lim_{n\to\infty}\lambda_{nq}^{x}(\bm{\theta})=\infty$ for any
$\bm{\theta}\in\bm{\Pi}$. Let $\bm{M}$ be a positive measure subset of
$\bm{\Pi}$ and $\\{\alpha_{n},n\in\mathbb{N}\\}$ a real positive nondecreasing
sequence such that $\lim_{n\to\infty}\alpha_{n}=\infty$ and
$\lambda_{nq}^{x}(\bm{\theta})\geq\alpha_{n}$ for $\bm{\theta}\in\bm{M}$.
Then, for $\bm{\theta}\in\bm{M}$ according to Lemma B.1,
$\mu(\bm{\theta})\leq\lambda_{n,q+1}^{x}(\bm{\theta})/\lambda_{mq}^{x}(\bm{\theta})\leq
W/\alpha_{m}$. Denote by $\Delta_{j}(\bm{\theta}),j=1,\ldots,q$ the
eigenvalues of $\bm{D}(\bm{\theta}){\bm{D}}^{\dagger}(\bm{\theta})$ in
descending order. By (B.9) and Weyl’s inequality, we have

$1\geq\Delta_{q}(\bm{\theta})\geq 1-W/\alpha_{m}$ (B.11)

for any $\bm{\theta}\in\bm{M}$. Hence, if $m^{*}$ is such that
$W/\alpha_{m^{*}}<1$, we have

$\Delta_{q}(\bm{\theta})\geq 1-W/\alpha_{m^{*}}>0$ (B.12)

for any $\bm{\theta}\in\bm{M}$ and $m\geq m^{*}$.

Assuming $m\geq m^{*}$, we denote by $\bm{\Delta}(\bm{\theta})$ the $q\times
q$ diagonal matrix with $\Delta_{j}(\bm{\theta})$, $j=1,\ldots,q$, on the
diagonal. Let $\bm{H}(\bm{\theta})$ be a matrix that is measurable in $\bm{M}$
and satisfies that for any $\bm{\theta}\in\bm{M}$,
$\bm{H}(\bm{\theta}){\bm{H}}^{\dagger}(\bm{\theta})=\bm{I}_{q}$ and
$\bm{H}(\bm{\theta})\bm{\Delta}(\bm{\theta}){\bm{H}}^{\dagger}(\bm{\theta})=\bm{D}(\bm{\theta}){\bm{D}}^{\dagger}(\bm{\theta})$.
Notice that due to (B.12), $\Delta_{j}^{-1/2}(\bm{\theta})$ is bounded for all
$\bm{\theta}\in\bm{M}$ and all $j=1,\ldots,q$. Therefore,

$\bm{F}(\bm{\theta})=\begin{cases}\bm{H}(\bm{\theta})\bm{\Delta}^{-1/2}(\bm{\theta}){\bm{H}}^{\dagger}(\bm{\theta})\bm{D}(\bm{\theta})&\text{if
$\bm{\theta}\in\bm{M}$}\\\ \bm{0}&\text{if
$\bm{\theta}\notin\bm{M}$}\end{cases}$ (B.13)

is well defined and, clearly, it belongs to $\bm{K}_{M}$. We then have the
following result:

######  Lemma B.2.

Suppose that (i) and (ii) of Theorem 4.1 hold. Then given $\tau$ such that
$0<\tau<2$, there exists an integer $m_{\tau}$ such that: (1)
$W/\alpha_{m_{\tau}}<1$; (2) for $n>m>m_{\tau}$, the largest eigenvalue of the
spectral density matrix of
$\underline{\bm{C}}(L){\bm{\psi}}^{m}_{\bm{\varsigma}}-\underline{\bm{F}}(L){\bm{\psi}}^{n}_{\bm{\varsigma}}$
is less than $\tau$ for any $\bm{\theta}\in\bm{\Pi}$.

###### Proof.

Denote by $\bm{S}(\bm{\theta})$ the spectral density matrix of
$\underline{\bm{C}}(L){\bm{\psi}}^{m}_{\bm{\varsigma}}-\underline{\bm{F}}(L){\bm{\psi}}^{n}_{\bm{\varsigma}}$.
Note that due to (B.7),

$\underline{\bm{C}}(L){\bm{\psi}}^{m}_{\bm{\varsigma}}-\underline{\bm{F}}(L){\bm{\psi}}^{n}_{\bm{\varsigma}}=\underline{\bm{R}}(L)\bm{x}_{n\bm{\varsigma}}+(\underline{\bm{D}}(L)-\underline{\bm{F}}(L)){\bm{\psi}}^{n}_{\bm{\varsigma}},$

where two terms on the right-hand side are orthogonal at any lead and lag.
Denoting by $\bm{S}_{1}(\bm{\theta})$ and $\bm{S}_{2}(\bm{\theta})$ the
spectral density matrices of these terms respectively, then, for any
$\bm{\theta}\in\bm{M}$, we have

$\displaystyle\bm{S}(\bm{\theta})$
$\displaystyle=\bm{S}_{1}(\bm{\theta})+\bm{S}_{2}(\bm{\theta})$
$\displaystyle=\bm{R}(\bm{\theta})\bm{\Sigma}_{n}^{x}(\bm{\theta}){\bm{R}}^{\dagger}(\bm{\theta})+\bm{D}(\bm{\theta}){\bm{D}}^{\dagger}(\bm{\theta})+\bm{F}(\bm{\theta}){\bm{F}}^{\dagger}(\bm{\theta})-\bm{D}(\bm{\theta}){\bm{F}}^{\dagger}(\bm{\theta})-\bm{F}(\bm{\theta}){\bm{D}}^{\dagger}(\bm{\theta})$
$\displaystyle=2\mathbf{I}_{q}-\bm{D}(\bm{\theta}){\bm{F}}^{\dagger}(\bm{\theta})-\bm{F}(\bm{\theta}){\bm{D}}^{\dagger}(\bm{\theta})$
$\displaystyle=2\mathbf{I}_{q}-2\bm{H}(\bm{\theta})\bm{\Delta}^{1/2}(\bm{\theta}){\bm{H}}^{\dagger}(\bm{\theta})$
$\displaystyle=2\bm{H}(\bm{\theta})(\mathbf{I}_{q}-\bm{\Delta}^{1/2}(\bm{\theta})){\bm{H}}^{\dagger}(\bm{\theta}),$

where the third equality is due to (B.9). Therefore, the largest eigenvalue of
$\bm{S}(\bm{\theta})$ is $2-2\sqrt{\Delta_{q}(\bm{\theta})}$, which is (recall
(B.11)) less than or equal to $2-2\Delta_{q}(\bm{\theta})\leq 2W/\alpha_{m}$.
Since $\tau<2$, the result follows from taking a positive integer $m_{\tau}$
such that

$2W/\alpha_{m_{\tau}}<\tau.$ (B.14)

This completes the proof. ∎

Denote by $\mathcal{S}(x_{i\bm{\varsigma}},x_{j\bm{\varsigma}};\bm{\theta})$
the cross-spectrum between $x_{i\bm{\varsigma}},x_{j\bm{\varsigma}}$ for
$i,j\in\mathbb{N}$. The following intermediate lemmas will be used for proving
further results.

######  Lemma B.3.

Suppose that Assumptions 3.1 and 3.2 hold. Consider the scalar sequences
$\\{A_{n\bm{\varsigma}},n\in\mathbb{N}\\}$ and
$\\{B_{n\bm{\varsigma}},n\in\mathbb{N}\\}$ such that
$\lim_{n\to\infty}A_{n\bm{\varsigma}}=A_{\bm{\varsigma}}$ and
$\lim_{n\to\infty}B_{n\bm{\varsigma}}=B_{\bm{\varsigma}}$, for any
$\bm{\varsigma}\in\mathbb{Z}^{3}$, with
$A_{\bm{\varsigma}},B_{\bm{\varsigma}}\in\bm{\mathcal{X}}$. Suppose also that
the rf $A_{n},B_{n},A,$ and $B$ are co-homostationary with $x_{\ell}$,
$\ell\in\mathbb{N}$. Then, there exists a sequence
$\\{s_{i},i\in\mathbb{N},\,s_{i}<s_{i+1}\\}$, such that
$\\{A_{s_{i}\bm{\varsigma}},i\in\mathbb{N}\\}$ and
$\\{B_{s_{i}\bm{\varsigma}},i\in\mathbb{N}\\}$ satisfy
$\underset{i\to\infty}{\lim}\
\mathcal{S}(A_{s_{i}\bm{\varsigma}},B_{s_{i}\bm{\varsigma}};\bm{\theta})=\mathcal{S}(A_{\bm{\varsigma}},B_{\bm{\varsigma}};\bm{\theta})$,
$\mathcal{L}$-a.e. in $\bm{\Theta}$.

###### Proof.

First, we have $\langle
A_{n\bm{\varsigma}},B_{n\bm{\varsigma}}\rangle=(8\pi^{3})^{-1}\int_{\bm{\Theta}}\mathcal{S}(A_{n\bm{\varsigma}},B_{n\bm{\varsigma}};\bm{\theta}){\mathrm{d}}\bm{\theta}$.
Then, due to continuity of the inner product and convergence of
$A_{n\bm{\varsigma}}$ and $B_{n\bm{\varsigma}}$, we have

$\lim_{n\to\infty}\frac{1}{8\pi^{3}}\int_{\bm{\Theta}}|\mathcal{S}(A_{n\bm{\varsigma}},B_{n\bm{\varsigma}};\bm{\theta})-\mathcal{S}(A_{\bm{\varsigma}},B_{\bm{\varsigma}};\bm{\theta})|{\mathrm{d}}\bm{\theta}=0.$

The desired result follows from Royden and Fitzpatrick (1988, p.145). ∎

######  Lemma B.4.

Suppose that (i) and (ii) of Theorem 4.1 and Assumptions 3.1-3.3 hold. Then
there exists a $q$-dimensional orthonormal rf $\bm{v}$ such that

  1. (a)

$v_{j\bm{\varsigma}}\in\mathcal{G}(\bm{x})$ for all $j=1,\ldots,q$;

  2. (b)

the spectral density matrix of $\bm{v}$ is $\mathbf{I}_{q}$ $\mathcal{L}$-a.e.
in $\bm{M}$, whilst it is ${\mathbf{0}}$ for $\bm{\theta}\notin\bm{M}$.

###### Proof.

The proof goes through by repeatedly applying Lemma B.2 to construct a Cauchy
sequence whose limit satisfies the desired properties.

Let $\bm{F}_{1}(\bm{\theta})$ be an element of $\bm{K}_{M}$. Let also
$\tau=1/2^{2}$, $n_{1}=m_{\tau}$, where $m_{\tau}$ satisfies (B.14),
$\bm{G}_{1}(\bm{\theta})=\bm{F}_{1}(\bm{\theta})\bm{\Lambda}_{n_{1}}^{-1/2}(\bm{\theta})\bm{P}_{n_{1}}(\bm{\theta})$,
and
$\bm{v}_{\bm{\varsigma}}^{(1)}=\underline{\bm{G}}_{1}(L)\bm{x}_{n\bm{\varsigma}}$.
One can easily check that the spectral density matrix of $\bm{v}^{(1)}$ equals
$\mathbf{I}_{q}$ for $\bm{\theta}\in\bm{M}$, ${\mathbf{0}}_{q}$ for
$\bm{\theta}\notin\bm{M}$.

In the similar way, set $\tau=1/2^{4}$ and $n_{2}=m_{\tau}$, where $m_{\tau}$
satisfies (B.14) and $m_{\tau}\geq n_{1}$. Set $\bm{D}(\bm{\theta})$ in (B.8)
by replacing $\bm{C}(\bm{\theta})$, $n$, $m$ with $\bm{F}_{1}(\bm{\theta})$,
$n_{2}$, $n_{1}$, respectively, and set $\bm{F}_{2}(\bm{\theta})$ as in
(B.13). Then set
$\bm{G}_{2}(\bm{\theta})=\bm{F}_{2}(\bm{\theta})\bm{\Lambda}_{n_{2}}^{-1/2}(\bm{\theta})\bm{P}_{n_{2}}(\bm{\theta})$
and
$\bm{v}_{\bm{\varsigma}}^{(2)}=\underline{\bm{G}}_{2}(L)\bm{x}_{n\bm{\varsigma}}$.
The spectral density matrix of $\bm{v}^{(2)}$ equals $\mathbf{I}_{q}$ for
$\bm{\theta}\in\bm{M}$, ${\mathbf{0}}_{q}$ for $\bm{\theta}\notin\bm{M}$.
Denote by $\mathcal{A}_{1}(\bm{\theta})$ the largest eigenvalue of the
spectral matrix of $\bm{v}^{(1)}-\bm{v}^{(2)}$. According to the definition of
$n_{1}$ and Lemma B.2, $\mathcal{A}_{1}(\bm{\theta})<1/2^{2}$ for any
$\bm{\theta}\in\bm{\Pi}$, which entails
$\|v_{j\bm{\varsigma}}^{(1)}-v_{j\bm{\varsigma}}^{(2)}\|<1/2$ for all
$j=1,\ldots,q$ and any $\bm{\varsigma}\in\mathbb{Z}^{3}$.

By recursion, set $\tau=1/2^{2k}$ and $n_{k}=m_{\tau}$, where $m_{\tau}$
satisfies (B.14) and $m_{\tau}\geq n_{k-1}$. Set $\bm{D}(\bm{\theta})$ in
(B.8) by replacing $\bm{C}(\bm{\theta})$, $n$, $m$ with
$\bm{F}_{k-1}(\bm{\theta})$, $n_{k}$, $n_{k-1}$, respectively, and set
$\bm{F}_{k}(\bm{\theta})$ as in (B.13). Then set
$\bm{G}_{k}(\bm{\theta})=\bm{F}_{k}(\bm{\theta})\bm{\Lambda}_{n_{k}}^{-1/2}(\bm{\theta})\bm{P}_{n_{k}}(\bm{\theta})$
and
$\bm{v}_{\bm{\varsigma}}^{(k)}=\underline{\bm{G}}_{k}(L)\bm{x}_{n\bm{\varsigma}}$.
The spectral density matrix of $\bm{v}^{(k)}$ equals $\mathbf{I}_{q}$ for
$\bm{\theta}\in\bm{M}$, ${\mathbf{0}}_{q}$ for $\bm{\theta}\notin\bm{M}$.
Denote by $\mathcal{A}_{k-1}(\bm{\theta})$ the largest eigenvalue of the
spectral matrix of $\bm{v}^{(k-1)}-\bm{v}^{(k)}$. According to the definition
of $n_{k-1}$ and Lemma B.2, $\mathcal{A}_{k-1}(\bm{\theta})<1/2^{2(k-1)}$ for
any $\bm{\theta}\in\bm{\Pi}$, which entails
$\|v_{j\bm{\varsigma}}^{(k-1)}-v_{j\bm{\varsigma}}^{(k)}\|<1/2^{k-1}$ for all
$j=1,\ldots,q$ and any $\bm{\varsigma}\in\mathbb{Z}^{3}$.

Hence, for all $j=1,\ldots,q$ and any $\bm{\varsigma}\in\mathbb{Z}^{3}$, we
have

$\|v_{j\bm{\varsigma}}^{(k)}-v_{j\bm{\varsigma}}^{(k+h)}\|\leq\|v_{j\bm{\varsigma}}^{(k)}-v_{j\bm{\varsigma}}^{(k+1)}\|+\cdots+\|v_{j\bm{\varsigma}}^{(k+h-1)}-v_{j\bm{\varsigma}}^{(k+h)}\|<\sum_{j=k}^{k+h-1}\frac{1}{2^{j}}<\frac{1}{2^{k-1}},$

which implies that for all $j=1,\ldots,q$,
$\\{{v}_{j\bm{\varsigma}}^{(k)},k\in\mathbb{N}\\}$ is a Cauchy sequence.
Denote by
$\bm{v}_{\bm{\varsigma}}=\lim_{k\to\infty}\bm{v}_{\bm{\varsigma}}^{(k)}$. Then
(b) follows from Lemma B.3 and the fact that the spectral density matrix of
$\bm{v}^{(k)}$ equals $\mathbf{I}_{q}$ for $\bm{\theta}\in\bm{M}$,
${\mathbf{0}}_{q}$ for $\bm{\theta}\notin\bm{M}$.

Now it remains to prove (a), for which it suffices to show that each row of
$\\{\bm{G}_{k},k\in\mathbb{N}\\}$ is a STDAS (see Definition 4.1). Notice that
$\bm{G}_{k}(\bm{\theta})\bm{G}^{\dagger}_{k}(\bm{\theta})=\bm{F}_{k}(\bm{\theta})\bm{\Lambda}_{n_{k}}^{-1}(\bm{\theta})\bm{F}^{\dagger}_{k}(\bm{\theta})$,
whose diagonal entries are equal or less than
$1/\lambda_{n_{k}q}^{x}(\bm{\theta})$ since
$\bm{F}_{k}(\bm{\theta})\in\bm{K}_{M}$ because
$1/\lambda_{n_{k}q}^{x}(\bm{\theta})$ converges to zero
$\mathcal{L}\text{-a.e.}$ in $\bm{\Theta}$ by (ii) of Theorem 4.1. Moreover,
without loss of generality we can always restrict Assumption 3.3 to assume
$\lambda_{n_{k}q}^{x}(\bm{\theta})>1$ for all $\bm{\theta}\in\bm{\Theta}$ (see
the arguments in Forni and Lippi, 2001, Section 4.2). Then,
$1/\lambda_{n_{k}q}^{x}(\bm{\theta})<1$ and by Lebesgue’s dominated
convergence theorem, its integral over $\bm{\Theta}$ converges to zero. This
concludes the proof of (a). ∎

Now, we apply the results in Lemma B.4 to define a $q$-dimensional white noise
rf over all $\bm{\Theta}$.

######  Proposition B.5.

Suppose that (i) and (ii) of Theorem 4.1 and Assumptions 3.1-3.3 hold. There
exists a $q$-dimensional orthonormal white noise rf $\bm{z}$ such that, for
all $j=1,\ldots,q$, $z_{j\bm{\varsigma}}\in\mathcal{G}(\bm{x})$.

###### Proof.

The proof goes through by choosing a set $\bm{N}\in\bm{\Theta}$ with Lebesgue
measure $\mathcal{L}(\bm{N})=8\pi^{3}=\mathcal{L}(\bm{\Theta})$ and by
obtaining a sequence of $q$-dimensional vector rf, which satisfy (a) in Lemma
B.4 and have spectral density matrix equal to $\mathbf{I}_{q}$ for
$\mathcal{L}\text{-a.e.}$ in a partition of $\bm{N}$, and is ${\mathbf{0}}$
otherwise.

Now, define $\bm{M}_{0}^{(1)}=\bm{\Pi}$. Then, by recursion, define $\nu_{a}$,
$a\in\mathbb{N}$, as the smallest among the integer $m$ such that

$\mathcal{L}(\\{\bm{\theta}\in\bm{M}_{a-1}^{(1)},\lambda_{mq}^{x}(\bm{\theta})>a\\})>4\pi^{3}$

and define
$\bm{M}_{a}^{(1)}=\\{\bm{\theta}\in\bm{M}_{a-1}^{(1)},\lambda_{\nu_{a}q}^{x}(\bm{\theta})>a\\}.$
Clearly, the Lebesgue measure of the set

$\bm{N}_{1}=\bm{M}_{1}^{(1)}\cap\bm{M}_{2}^{(1)}\cap\cdots\cap\bm{M}_{a}^{(1)}\cap\cdots$

is not less than $4\pi^{3}$. In the similar fashion, define $\bm{N}_{2}$
starting with $\bm{M}_{0}^{(2)}=\bm{\Pi}\setminus\bm{N}_{1}$ instead of
$\bm{\Pi}$, and using $\mathcal{L}(\bm{\Pi}\setminus\bm{N}_{1})/2$ instead of
$4\pi^{3}$. Also, for $b>2$, define $\bm{N}_{b}$ starting with
$\bm{M}_{0}^{(b)}=\bm{\Pi}\setminus\bm{N}_{1}\setminus\bm{N}_{2}\setminus\cdots\setminus\bm{N}_{b-1}$,
and using
$\mathcal{L}(\bm{\Pi}\setminus\bm{N}_{1}\setminus\bm{N}_{2}\setminus\cdots\setminus\bm{N}_{b-1})/2$.
Letting $\bm{N}=\bm{N}_{1}\cup\bm{N}_{2}\cup\cdots$, we have

$\mathcal{L}(\bm{N})=\mathcal{L}(\bm{N}_{1})+\mathcal{L}(\bm{N}_{2})+\cdots+\mathcal{L}(\bm{N}_{b})+\cdots=8\pi^{3},$

since by construction $\bm{N}_{i}\cap\bm{N}_{j}=\emptyset$, for $i\neq j$ and
$i,j\in\mathbb{N}$.

Lemma B.4 can be applied to the subset $\bm{N}_{b}$, with the sequence
$\alpha_{n}$ defined as $\alpha_{n}=a$, where $a$ is the only integer such
that $\nu_{a}\leq n<\nu_{a+1}$. Hence, we obtain a $q$-dimensional vector rf
$\\{\bm{v}_{\bm{\varsigma}}^{b}=(v_{1\bm{\varsigma}}^{b}\
v_{2\bm{\varsigma}}^{b}\ \cdots\
v_{q\bm{\varsigma}}^{b})^{\top},\,\bm{\varsigma}\in\mathbb{Z}^{3}\\}$ such
that (i) $v_{j\bm{\varsigma}}^{b}\in\mathcal{G}(\bm{x})$ for all
$j=1,\ldots,q$; (ii) its spectral density matrix equals $\mathbf{I}_{q}$ for
$\mathcal{L}\text{-a.e.}$ in $\bm{N}_{b}$, and is ${\mathbf{0}}_{q}$ for
$\bm{\theta}\notin\bm{N}_{b}$.

Finally, set
$\bm{z}_{\bm{\varsigma}}=\sum_{b=1}^{\infty}\bm{v}_{\bm{\varsigma}}^{b}$. It
is easy to see that $z_{j\bm{\varsigma}}\in\mathcal{G}(\bm{x})$ for all
$j=1,\ldots,q$ and the spectral density matrix of $\bm{z}$ equals
$\mathbf{I}_{q}$ $\mathcal{L}\text{-a.e.}$ in $\bm{\Theta}$. Therefore,
$\bm{z}$ is a $q$-dimensional orthonormal white noise rf. ∎

Considering the $q$-dimensional orthonormal white noise rf $\bm{z}$ in
Proposition B.5, we have the following

######  Proposition B.6.

Suppose that (i) and (ii) of Theorem 4.1 and Assumptions 3.1-3.3 hold. Then
$\text{\rm$\overline{\text{span}}$}(\bm{z})=\mathcal{G}(\bm{x})$.

###### Proof.

Consider a scalar rf $y_{\bm{\varsigma}}\in\mathcal{G}(\bm{x})$ and consider
the projection

$y_{\bm{\varsigma}}=\text{proj}(y_{\bm{\varsigma}}|\overline{\text{span}}(\bm{z}))+r_{\bm{\varsigma}}.$

It suffices to prove that $r_{\bm{\varsigma}}=0$. Let $\bm{W}(\bm{\theta})$
denote the spectral density matrix of the $(q+1)$-dimensional rf
$\\{(\bm{z}_{\bm{\varsigma}}\
r_{\bm{\varsigma}})^{\top},\,\bm{\varsigma}\in\mathbb{Z}^{3}\\}$. According to
the proof of Proposition B.5, $\bm{W}(\bm{\theta})$ is diagonal with
$\mathbf{I}_{q}$ in the $q\times q$ upper-left submatrix and
$\text{det}(\bm{W}(\bm{\theta}))=\mathcal{S}(r_{\bm{\varsigma}},r_{\bm{\varsigma}};\bm{\theta}).$
Since both $\bm{z}$ and $r$ belong to $\mathcal{G}(\bm{x})$, there exist
STDASs $\\{\bm{a}_{nj},n\in\mathbb{N}\\}$, for $j=1,\ldots,q+1$ such that,

$\displaystyle\underset{n\to\infty}{\lim}\
\bm{a}_{nj}(L)\bm{x}_{n\bm{\varsigma}}=z_{j\bm{\varsigma}},\quad
j=1,\ldots,q,$ $\displaystyle\underset{n\to\infty}{\lim}\
\bm{a}_{n,q+1}(L)\bm{x}_{n\bm{\varsigma}}=r_{\bm{\varsigma}}.$

Now, for all $j=1,\ldots,q+1$, by Definition 4.1 of STDAS, we must have
$\lim_{n\to\infty}\int_{\bm{\Theta}}\bm{a}_{nj}(\bm{\theta})\bm{a}_{nj}^{\dagger}(\bm{\theta}){\mathrm{d}}\bm{\theta}=0$,
it follows that

$\lim_{n\to\infty}\bm{a}_{nj}(\bm{\theta})\bm{a}_{nj}^{\dagger}(\bm{\theta})=\lim_{n\to\infty}|\bm{a}_{nj}(\bm{\theta})|^{2}=0,\quad\mathcal{L}\;\text{a.e.
in}\;\bm{\Theta}.$ (B.15)

Therefore (see Royden and Fitzpatrick (1988, p.145)) there exists a sub-
sequence $\\{s_{k},\,k\in\mathbb{N},s_{k}<s_{k+1}\\}$, defining a
corresponding sub-set of $s_{k}$ elements of $\bm{a}_{nj}(\bm{\theta})$
collected into the $s_{k}$-dimensional row vector
$\bm{a}_{s_{k}j}(\bm{\theta})$, which is such that

$\lim_{k\to\infty}|\bm{a}_{s_{k}j}(\bm{\theta})|^{2}=0,\quad\mathcal{L}\;\text{a.e.
in}\;\bm{\Theta}.$ (B.16)

Define also the rf $\bm{x}_{s_{k}\bm{\varsigma}}$ obtained from
$\bm{x}_{n\bm{\varsigma}}$ by setting to zero all entries with the exception
of the $s_{k}$ elements corresponding to the sub-sequence $\\{s_{k}\\}$, and
let $\bm{Z}_{n}(\bm{\theta})$ denote the spectral density matrix of
$\\{(\bm{a}_{n1}(L)\bm{x}_{s_{k}\bm{\varsigma}}\cdots\bm{a}_{n,q+1}(L)\bm{x}_{s_{k}\bm{\varsigma}})^{\top},\,\bm{\varsigma}\in\mathbb{Z}^{3}\\}$.
In view of Lemma B.3, there exists a sub-sequence of $\bm{Z}_{n}(\bm{\theta})$
converging to $\bm{W}(\bm{\theta})$, $\mathcal{L}$ a.e. in $\bm{\Theta}$.
Therefore, without loss of generality, we can assume that
$\bm{Z}_{n}(\bm{\theta})$ converges to $\bm{W}(\bm{\theta})$, $\mathcal{L}$
a.e. in $\bm{\Theta}$.

Let
$\bm{f}_{nj}(\bm{\theta})=\bm{a}_{nj}(\bm{\theta})\bm{P}_{s_{k}}^{\dagger}(\bm{\theta})$
and
$\bm{g}_{nj}(\bm{\theta})=\bm{a}_{nj}(\bm{\theta})-\bm{f}_{nj}(\bm{\theta})\bm{P}_{s_{k}}(\bm{\theta})$
for $j=1,\ldots,q+1$ and $\bm{\theta}\in\bm{\Theta}$. Hence,
$\bm{a}_{nj}(\bm{\theta})=\bm{f}_{nj}(\bm{\theta})\bm{P}_{s_{k}}(\bm{\theta})+\bm{g}_{nj}(\bm{\theta})$
and, for all $\bm{\theta}\in\bm{\Theta}$,

$|\bm{a}_{nj}(\bm{\theta})|^{2}=|\bm{f}_{nj}(\bm{\theta})|^{2}+|\bm{g}_{nj}(\bm{\theta})|^{2}.$
(B.17)

Indeed, by definition,

$\bm{f}_{nj}(\bm{\theta})\bm{P}_{s_{k}}(\bm{\theta})\bm{g}_{nj}^{\dagger}(\bm{\theta})=\bm{a}_{nj}(\bm{\theta})\bm{P}_{s_{k}}^{\dagger}(\bm{\theta})\bm{P}_{s_{k}}(\bm{\theta})\left(\bm{a}_{nj}(\bm{\theta})-\bm{a}_{nj}(\bm{\theta})\bm{P}_{s_{k}}^{\dagger}(\bm{\theta})\bm{P}_{s_{k}}(\bm{\theta})\right)^{\dagger}=0.$

Now, (B.15) implies

$\lim_{n\to\infty}|\bm{g}_{nj}(\bm{\theta})|^{2}=0\quad\mathcal{L}\;\text{a.e.
in}\;\bm{\Theta},$ (B.18)

and we have also that

$\lim_{n\to\infty}|\bm{f}_{nj}(\bm{\theta})|^{2}=\lim_{n\to\infty}\bm{a}_{nj}(\bm{\theta})\bm{P}_{s_{k}}^{\dagger}(\bm{\theta})\bm{P}_{s_{k}}(\bm{\theta})\bm{a}^{\dagger}_{nj}(\bm{\theta})=\lim_{k\to\infty}|\bm{a}_{s_{k}j}(\bm{\theta})|^{2}=0,\quad\mathcal{L}\;\text{a.e.
in}\;\bm{\Theta}.$ (B.19)

It follows that the following orthogonal decomposition holds:

$\underline{\bm{a}}_{nj}(L)\bm{x}_{s_{k}\bm{\varsigma}}=\underline{\bm{f}}_{nj}(L)\star\underline{\bm{P}}_{s_{k}}(L)\bm{x}_{s_{k}\bm{\varsigma}}+\underline{\bm{g}}_{nj}(L)\bm{x}_{s_{k}\bm{\varsigma}},\quad
j=1,\ldots,q+1.$ (B.20)

Denote by $\bm{Z}_{n}^{1}(\bm{\theta})$ and $\bm{Z}_{n}^{2}(\bm{\theta})$ the
spectral density matrices of the rf

$\left\\{\left(\underline{\bm{f}}_{n1}(L)\star\underline{\bm{P}}_{s_{k}}(L)\bm{x}_{s_{k}\bm{\varsigma}}\cdots\underline{\bm{f}}_{n,q+1}(L)\star\underline{\bm{P}}_{s_{k}}(L)\bm{x}_{s_{k}\bm{\varsigma}}\right)^{\dagger},\,\bm{\varsigma}\in\mathbb{Z}^{3}\right\\}$

and

$\left\\{\left((\underline{\bm{g}}_{n1}(L)\bm{x}_{s_{k}\bm{\varsigma}}\cdots\underline{\bm{g}}_{n,q+1}(L)\bm{x}_{s_{k}\bm{\varsigma}}\right)^{\top},\,\bm{\varsigma}\in\mathbb{Z}^{3}\right\\},$

respectively. Because of (B.17) and (B.20), we then have

$\bm{Z}_{n}(\bm{\theta})=\bm{Z}_{n}^{1}(\bm{\theta})+\bm{Z}_{n}^{2}(\bm{\theta}).$

Notice that $\bm{Z}_{n}^{1}(\bm{\theta})$ is singular for all
$\bm{\theta}\in\bm{\Theta}$, as $k\to\infty$, because
$\bm{P}_{s_{k}}(\bm{\theta})$ is $q+1\times s_{k}$. Hence,

$\lim_{n\to\infty}\text{det}(\bm{Z}_{n}^{1}(\bm{\theta}))=0,\quad\text{for all
}\bm{\theta}\in\bm{\Theta}.$ (B.21)

Since $\bm{g}_{nj}(\bm{\theta})$ is orthogonal to
${\bm{p}}_{s_{k}i}^{x}(\bm{\theta})$ for $i=1,\ldots,q$, we have

$\bm{Z}_{n}^{2}(\bm{\theta}))=\bm{g}_{nj}(\bm{\theta})\bm{\Sigma}_{s_{k}}^{x}(\bm{\theta})\bm{g}_{nj}^{\dagger}(\bm{\theta})\leq\lambda_{s_{k}q+1}^{x}(\bm{\theta})|\bm{g}_{nj}(\bm{\theta})|^{2}$

(Lancaster and Tismenetsky, 1985, Exercise 1, p. 287). Now, because of (i) in
Theorem 4.1 and by (B.18) we have that $\bm{Z}_{n}^{2}(\bm{\theta})$ converges
to zero $\mathcal{L}\text{-a.e.}$ in $\bm{\Theta}$ as $n\to\infty$. Therefore,
by (B.21),

$\lim_{n\to\infty}\text{det}(\bm{Z}_{n}(\bm{\theta}))=0,\quad\mathcal{L}\text{
a.e. in }\bm{\Theta},$ (B.22)

which entails that
$\text{det}(\bm{W}(\bm{\theta}))=\mathcal{S}(r_{\bm{\varsigma}},r_{\bm{\varsigma}};\bm{\theta})=0$,
$\mathcal{L}\text{-a.e.}$ in $\bm{\Theta}$ and, thus, $r_{\bm{\varsigma}}=0$.
∎

####  B.2.2 Canonical decomposition into common and idiosyncratic component

Consider the canonical decomposition

$\displaystyle x_{\ell\bm{\varsigma}}$
$\displaystyle=\text{proj}(x_{\ell\bm{\varsigma}}|\mathcal{G}(\bm{x}))+\delta_{\ell\bm{\varsigma}}=\gamma_{\ell\bm{\varsigma}}+\delta_{\ell\bm{\varsigma}},\;\text{
say.}$ (B.23)

So far by means of Propositions B.5 and B.6, we have shown that if (i) and
(ii) of Theorem 4.1 hold, then, there exists a $q$-dimensional orthonormal
white noise rf $\bm{z}$ such that

$\displaystyle\gamma_{\ell\bm{\varsigma}}$
$\displaystyle=\underline{\bm{c}}_{\ell}(L)\bm{z}_{\bm{\varsigma}},$ (B.24)

with ${\bm{c}}_{\ell}\in L_{2}^{q}(\bm{\Theta},\mathbb{C}).$

Now, consider a generic $n$-dimensional rf $\bm{w}_{n}$ satisfying Assumptions
3.1-3.2 with dynamic spatio-temporal eigenvalues
$\lambda_{nj}^{w}(\bm{\theta})$, $j=1,\ldots,n$, $\bm{\theta}\in\bm{\Theta}$.
Then, for any $\bm{\theta}\in\bm{\Theta}$ let
$\lambda_{j}^{w}(\bm{\theta})=\sup_{n\in\mathbb{N}}\lambda_{nj}^{w}(\bm{\theta})$
and recall that, since $\lambda_{nj}^{w}(\bm{\theta})$ is an increasing
sequence in $n$ then
$\sup_{n\in\mathbb{N}}\lambda_{nj}^{w}(\bm{\theta})=\lim_{n\to\infty}\lambda_{nj}^{w}(\bm{\theta})$.
So $\lambda_{j}^{w}(\bm{\theta})$ is the $j$-th largest dynamic spatio-
temporal eigenvalue of the infinite dimensional spectral density matrix
$\bm{\Sigma}^{w}(\bm{\theta})$ of the infinite dimensional rf $\bm{w}$. The
proof of the sufficient condition is concluded by means of the next two
results.

######  Proposition B.7.

Under Assumptions 3.1-3.2, the following statements are equivalent:

  1. (a)

$\bm{w}=\\{(w_{1\bm{\varsigma}}\ w_{2\bm{\varsigma}}\cdots
w_{\ell\bm{\varsigma}}\cdots)^{\top},\bm{\varsigma}\in\mathbb{Z}^{3}\\}$ is
idiosyncratic;

  2. (b)

the function $\lambda^{w}_{1}:\bm{\Theta}\to\mathbb{R}^{+}$ is essentially
bounded, i.e., $\text{ess}\sup(\lambda_{1}^{w})<\infty$ where
$\text{ess}\sup(\lambda_{1}^{w})=\inf\\{M:\mathcal{L}[\bm{\theta}:\lambda_{1}^{w}(\bm{\theta})>M]=0\\}$;

  3. (c)

Define $\bm{\Upsilon}:\bm{\Psi}\to
L^{\infty}_{2}(\bm{\Theta},\mathbb{C},\bm{\Sigma}^{w}_{n})$ as
$\bm{\Upsilon}(\bm{f})=\bm{f}$, the mapping $\bm{\Upsilon}$ is continuous.

###### Proof.

We first show that (a) and (c) are equivalent, $(a)\Leftrightarrow(c)$. To
this end, notice that

$\|\underline{\bm{a}}_{n}(L)\bm{w}_{\bm{s}t}\|=\|\bm{a}_{n}\|_{\bm{\Sigma}^{w}}=\|\bm{\Upsilon}({\bm{a}}_{n})\|_{\bm{\Sigma}^{w}},$
(B.25)

where the first equality follows from the application of the isometric
isomorphism $\mathcal{J}^{-1}$ (see Remarks A.1 and A.2), while the second
from the definition of the mapping $\bm{\Upsilon}$. Then, by definition of
idiosyncratic process
$\lim_{n\to\infty}\|\underline{\bm{a}}_{n}(L)\bm{w}_{\bm{s}t}\|=0$, thus by
(B.25) it follows that
$\lim_{n\to\infty}\|\bm{\Upsilon}({\bm{a}}_{n})\|_{\bm{\Sigma}^{w}}=0$, which
implies that the linear mapping $\bm{\Upsilon}$ is continuous at zero. From
Conway (1985, Proposition 1.1, p.26), it follows that $\bm{\Upsilon}$ is
continuous everywhere. This proves $(a)\Leftrightarrow(c)$.

To prove that (b) and (c) are equivalent, $(b)\Leftrightarrow(c)$, first
notice that continuity and boundedness are equivalent for linear maps between
normed vector spaces (Royden and Fitzpatrick, 1988, Theorem 1, p. 257). Then,
consider the definition of operator norm:

$\|\bm{\Upsilon}\|=\sup_{\bm{f}\in\bm{\Psi},\|\bm{f}\|=1}\left\|\bm{\Upsilon}(\bm{f})\right\|_{\bm{\Sigma}^{w}}$

and notice that boundedness of $\bm{\Upsilon}$ means that
$\|\bm{\Upsilon}\|\leq c<\infty$, for $c\in\mathbb{R}^{+}$. We now show that
$\|\bm{\Upsilon}\|=\sqrt{\text{ess}\sup(\lambda_{1}^{w})}$. This would imply
that $(b)\Leftrightarrow(c)$.

Let us define, $\bm{f}^{[n]}$ as the infinite dimensional vector with
$f_{j}^{[n]}=f_{j}$ for $j\leq n$ and $f_{j}^{[n]}=0$ for $j>n$,
$\bm{f}^{\\{n\\}}$ as the $n$-dimensional sub-vector made of the first $n$
entries of $\bm{f}^{[n]}$, and

$\psi_{n}=\sup_{\bm{f}\in\bm{\Psi},\|\bm{f}\|=1}\left\|\bm{\Upsilon}(\bm{f}^{[n]})\right\|_{\bm{\Sigma}^{w}},$

so, for $\bm{\Psi}_{n}=L^{\infty}_{2}(\bm{\Theta},\mathbb{C})\cap
L^{\infty}_{2}(\bm{\Theta},\mathbb{C},\lambda^{w}_{1n})$, we have

$\displaystyle\psi^{2}_{n}$ $\displaystyle=$
$\displaystyle\sup_{\bm{f}\in\bm{\Psi},\|\bm{f}\|=1}\frac{1}{8\pi^{3}}\int_{\bm{\Theta}}\bm{f}^{\\{n\\}}(\bm{\theta})\bm{\Sigma}^{w}_{n}(\bm{\theta}){\bm{f}}^{\\{n\\}{\dagger}}(\bm{\theta})\mathrm{d}\bm{\theta}$
$\displaystyle=$
$\displaystyle\sup_{h\in\bm{\Psi}_{n},\|\bm{h}\|=1}\frac{1}{8\pi^{3}}\int_{\bm{\Theta}}\|h(\bm{\theta})\|^{2}\lambda^{w}_{1n}(\bm{\theta})\mathrm{d}\bm{\theta},$

where the last equality follows form Lancaster and Tismenetsky (1985, Theorem
4, p.285). Moreover, Conway (1985, Theorem 1.5, p.28) implies that
$\psi_{n}^{2}=\text{ess}\sup\lambda_{1n}^{w}(\bm{\theta})$. Finally we notice
that
$\|\bm{\Upsilon}\|^{2}=\lim_{n\to\infty}\psi^{2}_{n}=\lim_{n\to\infty}{\text{ess}\sup\lambda^{w}_{1n}(\bm{\theta})}$
$=\text{ess}\sup\lim_{n\to\infty}\lambda^{w}_{1n}(\bm{\theta})=\text{ess}\sup\lambda^{w}_{1}(\bm{\theta})$.
∎

######  Proposition B.8.

Under Assumptions 3.1-3.2, $\bm{\delta}=\\{(\delta_{1\bm{\varsigma}}\
\delta_{2\bm{\varsigma}}\cdots\delta_{\ell\bm{\varsigma}}\cdots)^{\top},\bm{\varsigma}\in\mathbb{Z}^{3}\\}$
is idiosyncratic, where $\delta_{\ell\bm{\varsigma}}$, $\ell\in\mathbb{N}$,
are defined in (B.23).

###### Proof.

Start by considering again (B.6) and let, $\underline{\bm{\pi}}_{n\ell}(L)$
and $\underline{\bm{q}}_{n\ell}(L)$ be the $\ell$-th $q$-dimensional and
$(n-q)$-dimensional rows of $\underline{{\bm{P}}}^{\dagger}_{n}(L)$ and
$\underline{{\bm{Q}}}^{\dagger}_{n}(L)$, respectively, for any given $\ell\leq
n$. Then,

$x_{\ell\bm{\varsigma}}=\underline{\bm{\pi}}_{n\ell}(L)\star\underline{{\bm{P}}}(L)\bm{x}_{n\bm{\varsigma}}+\underline{\bm{q}}_{n\ell}(L)\star\underline{{\bm{Q}}}(L)\bm{x}_{n\bm{\varsigma}}=\underline{\bm{\pi}}_{n\ell}(L)\star\underline{\bm{\Lambda}}_{n}^{1/2}(L)\bm{\psi}_{\bm{\varsigma}}^{n}+\underline{\bm{q}}_{n\ell}(L)\star\underline{{\bm{Q}}}(L)\bm{x}_{n\bm{\varsigma}}=\gamma_{\ell\bm{\varsigma}}^{n}+\delta_{\ell\bm{\varsigma}}^{n}.$
(B.26)

For any given $m\in\mathbb{N}$, let $\bm{\Sigma}_{m}^{\delta}(\bm{\theta})$ be
the spectral density matrix of
$\bm{\delta}_{m}=\\{(\delta_{1\bm{\varsigma}}\cdots\delta_{m\bm{\varsigma}})^{\top},\bm{\varsigma}\in\mathbb{Z}^{3}\\}$
and for $n>m$ let $\bm{\Sigma}_{m}^{\delta^{n}}(\bm{\theta})$ be the spectral
density matrix of
$\bm{\delta}_{m}^{n}=\\{(\delta_{1\bm{\varsigma}}^{n}\cdots\delta_{m\bm{\varsigma}}^{n})^{\top},\bm{\varsigma}\in\mathbb{Z}^{3}\\}$,
where
$\delta_{\ell\bm{\varsigma}}^{n}=x_{\ell\bm{\varsigma}}-\gamma_{\ell\bm{\varsigma}}^{n}$
(see (B.26)). Then, from Theorem 5.1, we have that

$\lim_{n\to\infty}\gamma_{\ell\bm{\varsigma}}^{n}=\gamma_{\ell\bm{\varsigma}},$

in mean-square, with $\gamma_{\ell\bm{\varsigma}}$, $\ell\in\mathbb{N}$,
defined in (B.23). Thus,

$\lim_{n\to\infty}\delta_{\ell\bm{\varsigma}}^{n}=\delta_{\ell\bm{\varsigma}}$

in mean-square for any $\ell\leq m$. Notice that, although Theorem 5.1 is
proved in the next section, its proof only requires (i) and (ii) in Theorem
4.1 to hold as in this proof, so there is no feedback loop between the two
theorems.

By Lemma B.3, a sub-sequence of $\bm{\Sigma}_{m}^{\delta^{n}}(\bm{\theta})$
converges to $\bm{\Sigma}_{m}^{\delta}(\bm{\theta})$, $\mathcal{L}\text{-a.e.
in }\bm{\Theta}$.

$\lim_{n\to\infty}\left\|\bm{\Sigma}_{m}^{\delta^{n}}(\bm{\theta})-\bm{\Sigma}_{m}^{\delta}(\bm{\theta})\right\|_{F}=0,\quad\mathcal{L}\text{-a.e.
in }\bm{\Theta}.$

where $\|\cdot\|_{F}$ denotes the Frobenius norm. Then, by definition of the
spectral norm, denoted as $\|\cdot\|$,

$\lim_{n\to\infty}\left|\lambda_{m1}^{\delta^{n}}(\bm{\theta})-\lambda_{m1}^{\delta}(\bm{\theta})\right|=\lim_{n\to\infty}\left\|\bm{\Sigma}_{m}^{\delta^{n}}(\bm{\theta})-\bm{\Sigma}_{m}^{\delta}(\bm{\theta})\right\|\leq\lim_{n\to\infty}\left\|\bm{\Sigma}_{m}^{\delta^{n}}(\bm{\theta})-\bm{\Sigma}_{m}^{\delta}(\bm{\theta})\right\|_{F}=0.$
(B.27)

Moreover, since $\bm{\Sigma}_{m}^{\delta^{n}}(\bm{\theta})$ is the upper-left
$m\times m$ submatrix of $\bm{\Sigma}_{n}^{\delta^{n}}(\bm{\theta})$, we have
by Weyl’s inequality and definition of $\delta_{n}^{n}$ in (B.26),

$\lambda_{m1}^{\delta^{n}}(\bm{\theta})\leq\lambda_{n1}^{\delta^{n}}(\bm{\theta})=\lambda_{n,q+1}^{x}(\bm{\theta}),$

for any $m\leq n$, any $n\in\mathbb{N}$, and any $\bm{\theta}\in\bm{\Theta}$.
Therefore, letting
$\lambda_{q+1}^{x}(\bm{\theta})=\lim_{n\to\infty}\lambda_{n,q+1}^{x}(\bm{\theta})$,
from (B.27)

$\lambda_{m1}^{\delta}(\bm{\theta})\leq\lambda_{q+1}^{x}(\bm{\theta}),\quad\mathcal{L}\text{-a.e.
in }\bm{\Theta},$

and since this is true for any $m\in\mathbb{N}$, then, letting
$\lambda_{1}^{\delta}(\bm{\theta})=\lim_{m\to\infty}\lambda_{m1}^{\delta}(\bm{\theta})$,

$\lambda_{1}^{\delta}(\bm{\theta})\leq\lambda_{q+1}^{x}(\bm{\theta}),\quad\mathcal{L}\text{-a.e.
in }\bm{\Theta}.$ (B.28)

So by (i) in Theorem 4.1, $\lambda_{1}^{\delta}$ is essentially bounded, and
by Proposition B.7, $\bm{\delta}$ is idiosyncratic. ∎

To conclude, by Weyl’s inequality and Propositions B.7 and B.8, we have

$\lambda_{nq}^{\gamma}(\bm{\theta})\geq\lambda_{nq}^{x}(\bm{\theta})-\lambda_{n1}^{\delta}(\bm{\theta})\geq\lambda_{nq}^{x}(\bm{\theta}),$

for any given $n\in\mathbb{N}$ and $\bm{\theta}\in\bm{\Theta}$. Therefore,
given (ii) of Theorem 4.1,

$\lambda_{q}^{\gamma}(\bm{\theta})=\lim_{n\to\infty}\lambda_{nq}^{\gamma}(\bm{\theta})=\infty,\quad\mathcal{L}\text{-a.e.
in }\bm{\Theta}.$ (B.29)

By (B.28) and (B.29), we showed that if (i) and (ii) in Theorem 4.1 hold then
(iv) and (v) in Definition 4.5 hold and we can write decomposition (11) with
idiosyncratic component $\xi_{\ell\bm{\varsigma}}=\delta_{\ell\bm{\varsigma}}$
and common component
$\chi_{\ell\bm{\varsigma}}=\gamma_{\ell\bm{\varsigma}}=\underline{\bm{c}}_{\ell}(L)\bm{z}_{\bm{\varsigma}}$
with $\bm{c}_{\ell}\in L_{2}^{q}(\bm{\Theta},\mathbb{C})$. Because of Remark
4.4 we can always find a transformation such that we can also write
$\chi_{\ell\bm{\varsigma}}=\underline{\bm{b}}_{\ell}(L)\bm{u}_{\bm{\varsigma}}$
as in (12) this proves part (i) and (ii) in Definition 4.5. Finally, part
(iii) in Definition 4.5 follows from orthogonality of
$\gamma_{\ell\bm{\varsigma}}$ and $\delta_{\ell\bm{\varsigma}}$ in the
canonical decomposition (B.23). This completes the proof of the sufficient
condition.

###  B.3 Proof of Theorem 4.1 \- necessary condition

From (iii) in Definition 4.5

$\bm{\Sigma}_{n}^{x}(\bm{\theta})=\bm{\Sigma}_{n}^{\chi}(\bm{\theta})+\bm{\Sigma}_{n}^{\xi}(\bm{\theta}).$

Then, for all $\bm{\theta}\in\bm{\Theta}$ and any $n\in\mathbb{N}$, by Weyl’s
inequality and since, by definition, $\bm{\Sigma}_{n}^{\xi}(\bm{\theta})$ is
positive semi-definite for all $n\in\mathbb{N}$,

$\lambda_{nq}^{x}(\bm{\theta})\geq\lambda_{nq}^{\chi}(\bm{\theta})+\lambda_{nn}^{\xi}(\bm{\theta})\geq\lambda_{nq}^{\chi}(\bm{\theta}).$

By taking the limit for $n\to\infty$ and because of (v) in Definition 4.5, we
prove (ii) in Theorem 4.1. Again by Weyl’s inequality, for all
$\bm{\theta}\in\bm{\Theta}$ and any $n\in\mathbb{N}$, we also have

$\lambda_{n,q+1}^{x}(\bm{\theta})\leq\lambda_{n,q+1}^{\chi}(\bm{\theta})+\lambda_{n1}^{\xi}(\bm{\theta})=\lambda_{n1}^{\xi}(\bm{\theta})$

By taking the limit for $n\to\infty$ and because of (iv) in Definition 4.5, we
prove (i) in Theorem 4.1. This completes the proof of the necessary condition.

###  B.4 Proof of Corollary 4.2

Supposing that $\bm{x}$ is a $q$-GSTFM with representation (11)-(12), as we
have shown, $\bm{x}$ also has the canonical representation

$x_{\ell\bm{\varsigma}}=\gamma_{\ell\bm{\varsigma}}+\delta_{\ell\bm{\varsigma}},$

where
$\gamma_{\ell\bm{\varsigma}}=\text{proj}(x_{\ell\bm{\varsigma}}|\mathcal{G}(\bm{x}))=\underline{\bm{c}}_{\ell}(L)\bm{z}_{\bm{\varsigma}}$
and $\bm{z}_{\bm{\varsigma}}$ is a $q$-dimensional orthonormal white noise rf
and $\overline{\text{span}}(\bm{z})=\mathcal{G}(\bm{x})$. Since $\bm{\xi}$ is
idiosyncratic, we have
$\mathcal{G}(\bm{x})\subseteq\overline{\text{span}}(\bm{\chi})$, which, by
noting that
$\overline{\text{span}}(\bm{\chi})\subseteq\overline{\text{span}}(\bm{u})$,
entails
$\overline{\text{span}}(\bm{z})\subseteq\overline{\text{span}}(\bm{u})$. On
the other hand, since both $\bm{z}$ and $\bm{u}$ are $q$-dimensional white
noise rf, we have
$\overline{\text{span}}(\bm{z})=\overline{\text{span}}(\bm{u})$. Therefore,
$\mathcal{G}(\bm{x})=\overline{\text{span}}(\bm{\chi})=\overline{\text{span}}(\bm{u}).$
This implies that $\chi_{\ell\bm{\varsigma}}\in\mathcal{G}(\bm{x})$ and
$\xi_{\ell\bm{\varsigma}}\perp\mathcal{G}(\bm{x})$, so that
$\chi_{\ell\bm{\varsigma}}=\gamma_{\ell\bm{\varsigma}}=\text{\rm
proj}(x_{\ell\bm{\varsigma}}|\mathcal{G}(\bm{x}))$ and
$\xi_{\ell\bm{\varsigma}}=\delta_{\ell\bm{\varsigma}}$. Uniqueness follows
from uniqueness of the canonical representation. This completes the proof.

##  Appendix C Proof of results of Section 5

###  C.1 Proof of Theorem 5.1

The proof requires the following definition and preliminary lemmas.

######  Definition C.1 (Cauchy sequence of spaces).

For any $n\in\mathbb{N}$, let
$\bm{v}_{n}=\\{\bm{v}_{n\bm{\varsigma}},\bm{\varsigma}\in\mathbb{Z}^{3}\\}$ be
a $q$-dimensional orthonormal white noise rf such that
$\bm{v}_{n\bm{\varsigma}}\in\bm{\mathcal{X}}$ and is co-homostationary with
$x_{\ell}$, $\ell\in\mathbb{N}$, so that $\bm{v}_{n}$ and $\bm{v}_{m}$ are co-
homostationary for any $n$ and $m$. Denote by $\bm{A}^{mn}(\bm{\theta})$ the
$q\times q$ matrix whose $(h,k)$ entry is the cross spectrum
$\mathcal{S}(v_{mh\bm{\varsigma}},v_{nk\bm{\varsigma}};\bm{\theta})$, for
$\bm{\theta}\in\bm{\Theta}$. The orthogonal projection, element by element, of
$\bm{v}_{m\bm{\varsigma}}$ on the process $\bm{v}_{n}$ is
$\underline{\bm{A}}^{mn}(L)\bm{v}_{n\bm{\varsigma}}$. Consider the orthogonal
decomposition

$\bm{v}_{m\bm{\varsigma}}=\underline{\bm{A}}^{mn}(L)\bm{v}_{n\bm{\varsigma}}+\bm{\rho}_{\bm{\varsigma}}^{mn},$
(C.30)

and let $\bm{\varrho}^{mn}(\bm{\theta})$ denote the spectral density matrix of
$\bm{\rho}_{\bm{\varsigma}}^{mn}$. The sequence
$\\{\bm{v}_{n},n\in\mathbb{N}\\}$ generates a Cauchy sequence of spaces if,
for a given $\epsilon>0$ and $\mathcal{L}$-a.e. in $\bm{\Theta}$, there exists
an integer $m_{\epsilon}(\bm{\theta})$ such that for
$n,m>m_{\epsilon}(\bm{\theta})$, $\text{\rm
trace}(\bm{\varrho}^{mn}(\bm{\theta}))<\epsilon$.

######  Lemma C.1.

Assume that $\\{\bm{v}_{n},n\in\mathbb{N}\\}$ fulfills Definition C.1 and
$y=\\{y_{\bm{\varsigma}},\bm{\varsigma}\in\mathbb{Z}^{3}\\}$ is such that
$y_{\bm{\varsigma}}\in\bm{\mathcal{X}}$ and is co-homostationary with
$x_{\ell}$, $\ell\in\mathbb{N}$. Let $Y_{n\bm{\varsigma}}$ be the orthogonal
projection of $y_{\bm{\varsigma}}$ on the process $\bm{v}_{n}$, i.e.,
$Y_{n\bm{\varsigma}}=\text{\rm
proj}(y_{\bm{\varsigma}}|\text{\rm$\overline{\text{span}}$}(\bm{v}_{n}))$.
Then, $Y_{n\bm{\varsigma}}$ converges in $\bm{\mathcal{X}}$ in mean-square, as
$n\to\infty$.

###### Proof.

Considering projections

$y_{\bm{\varsigma}}=Y_{n\bm{\varsigma}}+r_{n\bm{\varsigma}}=\underline{\bm{b}}_{n}(L)\bm{v}_{n\bm{\varsigma}}+r_{n\bm{\varsigma}},$
$y_{\bm{\varsigma}}=Y_{m\bm{\varsigma}}+r_{m\bm{\varsigma}}=\underline{\bm{b}}_{m}(L)\bm{v}_{m\bm{\varsigma}}+r_{m\bm{\varsigma}},$

where ${\bm{b}}_{n},{\bm{b}}_{m}\in L_{2}^{q}(\bm{\Theta},\mathbb{C})$, this
yields

$\underline{\bm{b}}_{n}(L)\bm{v}_{n\bm{\varsigma}}-\underline{\bm{b}}_{m}(L)\bm{v}_{m\bm{\varsigma}}=r_{m\bm{\varsigma}}-r_{n\bm{\varsigma}}.$

Now we show the spectral density of the rf on the left-hand side converges to
zero $\mathcal{L}$-a.e. in $\bm{\Theta}$. Note that the spectral density of
the rf on left-hand side is the cross spectrum between the left- and right-
hand sides, which, due to the definition of $r_{n\bm{\varsigma}}$ and
$r_{m\bm{\varsigma}}$, is the sum of two cross spectra:
$\mathcal{S}(r_{n\bm{\varsigma}},\underline{\bm{b}}_{m}(L)\bm{v}_{m\bm{\varsigma}};\bm{\theta})+\mathcal{S}(r_{m\bm{\varsigma}},\underline{\bm{b}}_{n}(L)\bm{v}_{n\bm{\varsigma}};\bm{\theta}).$
In view of (C.30), we have

$\mathcal{S}(r_{n\bm{\varsigma}},\underline{\bm{b}}_{m}(L)\bm{v}_{m\bm{\varsigma}};\bm{\theta})=\mathcal{S}(r_{n\bm{\varsigma}},\underline{\bm{b}}_{m}(L)\star\underline{\bm{A}}^{mn}(L)\bm{v}_{n\bm{\varsigma}}+\underline{\bm{b}}_{m}(L)\bm{\rho}_{\bm{\varsigma}}^{mn};\bm{\theta})=\mathcal{S}(r_{n\bm{\varsigma}},\underline{\bm{b}}_{m}(L)\bm{\rho}_{\bm{\varsigma}}^{mn};\bm{\theta}).$

Note that both the spectral density of $r_{n\bm{\varsigma}}$ and the squared
entries of ${\bm{b}}_{m}$ are bounded in modulus by the spectral density of
$y_{\bm{\varsigma}}$. Hence, the fact that $\\{\bm{v}_{n},n\in\mathbb{N}\\}$
generates a Cauchy sequence of spaces implies that
$\mathcal{S}(r_{n\bm{\varsigma}},\underline{\bm{b}}_{m}(L)\bm{\rho}_{\bm{\varsigma}}^{mn};\bm{\theta})$
converges to zero $\mathcal{L}$-a.e. in $\bm{\Theta}$ as
$m,n\rightarrow\infty$. Similar argument holds also for
$\mathcal{S}(r_{m\bm{\varsigma}},\underline{\bm{b}}_{n}(L)\bm{v}_{n\bm{\varsigma}};\bm{\theta})$.
Therefore, the spectral density of $Y_{n\bm{\varsigma}}-Y_{m\bm{\varsigma}}$
converges to zero $\mathcal{L}$-a.e. in $\bm{\Theta}$ as
$m,n\rightarrow\infty$. Since the spectral densities of $Y_{n\bm{\varsigma}}$
and $Y_{m\bm{\varsigma}}$ are dominated by that of $y_{\bm{\varsigma}}$, by
Lebesgue’s dominated convergence theorem, the integral of the spectral density
of $Y_{n\bm{\varsigma}}-Y_{m\bm{\varsigma}}$ converges to zero as
$m,n\rightarrow\infty$, which implies that $Y_{n\bm{\varsigma}}$ is a Cauchy
sequence and thus converges in $\bm{\mathcal{X}}$, as $n\to\infty$. ∎

######  Lemma C.2.

Suppose that (i) and (ii) of Theorem 4.1 and Assumptions 3.1-3.3 hold. Then,
$\\{{\bm{\psi}}^{n},n\in\mathbb{N}\\}$, as defined in (B.5), generates a
Cauchy sequence of spaces.

###### Proof.

For $n>m$, in (B.7), letting $\bm{C}=\bm{I}_{q}$ yields

${\bm{\psi}}^{m}_{\bm{\varsigma}}=\underline{\bm{D}}(L){\bm{\psi}}^{n}_{\bm{\varsigma}}+\bm{\rho}_{\bm{\varsigma}}^{mn}.$
(C.31)

where
${\bm{D}}(\bm{\theta})={\bm{\Lambda}}^{-1/2}_{m}(\bm{\theta}){\bm{P}}_{m}(\bm{\theta}){\bm{P}}^{\dagger}_{n}(\bm{\theta}){\bm{\Lambda}}^{1/2}_{n}(\bm{\theta}).$
Let $\bm{\varrho}^{mn}(\bm{\theta})$ denote the spectral density matrix of
$\bm{\rho}_{\bm{\varsigma}}^{mn}$. Lemma B.1 and (ii) of Theorem 4.1 imply
that $\text{trace}(\bm{\varrho}^{mn}(\bm{\theta}))$ converges to zero
$\mathcal{L}$-a.e. in $\bm{\Theta}$.

On the other hand,

${\bm{\psi}}^{n}_{\bm{\varsigma}}=\underline{\bm{D}}^{\dagger}(L){\bm{\psi}}^{m}_{\bm{\varsigma}}+\bm{\rho}_{\bm{\varsigma}}^{nm}.$
(C.32)

Using (C.31) and (C.32), we have

$\bm{I}_{q}={\bm{D}}(\bm{\theta}){\bm{D}}^{\dagger}(\bm{\theta})+\bm{\varrho}^{mn}(\bm{\theta})={\bm{D}}^{\dagger}(\bm{\theta}){\bm{D}}(\bm{\theta})+\bm{\varrho}^{nm}(\bm{\theta})$

$\mathcal{L}$-a.e. in $\bm{\Theta}$. Taking the trace on both sides and noting
that the trace of ${\bm{D}}(\bm{\theta}){\bm{D}}^{\dagger}(\bm{\theta})$ is
equal to that of ${\bm{D}}^{\dagger}(\bm{\theta}){\bm{D}}(\bm{\theta})$, we
have
$\text{trace}(\bm{\varrho}^{mn}(\bm{\theta}))=\text{trace}(\bm{\varrho}^{nm}(\bm{\theta}))$
$\mathcal{L}$-a.e. in $\bm{\Theta}$. Finally,
$\text{trace}(\bm{\varrho}^{mm}(\bm{\theta}))=0$. Therefore,
$\text{trace}(\bm{\varrho}^{mn}(\bm{\theta}))$ converges to zero
$\mathcal{L}$-a.e. in $\bm{\Theta}$ for any $m,n\rightarrow\infty$. ∎

Finally, let us consider again the orthogonal decomposition in (B.26) for any
$\ell\leq n$, i.e.,

$x_{\ell\bm{\varsigma}}=\underline{{\bm{\pi}}}_{n\ell}(L)\star\underline{\bm{\Lambda}}^{1/2}_{n}(L){\bm{\psi}}^{n}_{\bm{\varsigma}}+\underline{{\bm{q}}}_{n\ell}(L)\star\underline{{\bm{Q}}}_{n}(L)\bm{x}_{n\bm{\varsigma}}=\gamma_{\ell\bm{\varsigma}}^{n}+\delta_{\ell\bm{\varsigma}}^{n}.$

Due to Lemma C.1 and Lemma C.2,

$\lim_{n\to\infty}\gamma_{\ell\bm{\varsigma}}^{n}=\gamma_{\ell\bm{\varsigma}}$

in mean-square and $\gamma_{\ell\bm{\varsigma}}\in\bm{\mathcal{X}}$. This
entails that

$\lim_{n\to\infty}\delta_{\ell\bm{\varsigma}}^{n}=x_{\ell\bm{\varsigma}}-\gamma_{\ell\bm{\varsigma}}=\delta_{\ell\bm{\varsigma}}$

in mean-square, with $\delta_{\ell\bm{\varsigma}}\in\bm{\mathcal{X}}$.

Moreover, $\gamma_{\ell\bm{\varsigma}}$ is an aggregate, i.e., it belongs to
$\mathcal{G}(\bm{x})$, since ${\bm{\pi}}_{n\ell}{\bm{P}}_{n}$ is a STDAS. To
see this, note that the spectral density matrix of
$\gamma_{\ell\bm{\varsigma}}^{n}$, which is
${\bm{\pi}}_{n\ell}(\bm{\theta}){\bm{\Lambda}}_{n}(\bm{\theta}){\bm{\pi}}^{\dagger}_{n\ell}(\bm{\theta})$,
is not smaller than
$\lambda_{nq}^{x}(\bm{\theta}){\bm{\pi}}_{n\ell}(\bm{\theta}){\bm{\pi}}^{\dagger}_{n\ell}(\bm{\theta})$
and it is bounded above by the spectral density of $x_{\ell\bm{\varsigma}}$,
call it $\sigma^{2}_{\ell}(\bm{\theta})$. Therefore, we have
${\bm{\pi}}_{n\ell}(\bm{\theta}){\bm{\pi}}^{\dagger}_{n\ell}(\bm{\theta})\leq\sigma^{2}_{\ell}(\bm{\theta})/\lambda_{nq}^{x}(\bm{\theta})$.
Assuming again without loss of generality that
$\lambda_{nq}^{x}(\bm{\theta})>1$ for all $\bm{\theta}\in\bm{\Theta}$ (see the
arguments in Forni and Lippi, 2001, Section 4.2), it follows that
$\sigma_{\ell}^{2}(\bm{\theta})/\lambda_{nq}^{x}(\bm{\theta})$ is bounded
above by $\sigma^{2}_{\ell}(\bm{\theta})$ and it converges to zero
$\mathcal{L}$-a.e. in $\bm{\Theta}$, because of (ii) in Theorem 4.1. Hence, by
Lebesgue’s dominated convergence theorem,
$\int_{\bm{\Theta}}\sigma_{\ell}(\bm{\theta})/\lambda_{nq}^{x}(\bm{\theta})\mathrm{d}\bm{\theta}$
converges to zero.

By construction, $\delta_{\ell\bm{\varsigma}}^{n}$ is orthogonal to
${\bm{\psi}}^{n}_{\bm{\varsigma}-\bm{h}}$ for any $\bm{h}\in\mathbb{Z}^{3}$.
Since $\mathcal{G}(\bm{x})=\overline{\text{span}}(\bm{z})$ by Proposition B.6
and the process $\bm{z}$ has been obtained by taking limits of linear
combinations of the elements of $\bm{\psi}^{n}$, continuity of the inner
product implies that $\delta_{\ell\bm{\varsigma}}\perp\mathcal{G}(\bm{x})$.
Then, by uniqueness of the canonical decomposition we have that
$\gamma_{\ell\bm{\varsigma}}=\chi_{\ell\bm{\varsigma}}$ for all
$\bm{\varsigma}\in\mathbb{Z}^{3}$. This completes the proof.

##  Appendix D Some novel results on spectral density matrix estimation for
spatio-temporal random fields

In this section we derive some novel results about spectral density estimation
of spatio-temporal rf. Our theory builds on and extends the one already
available in Deb, Pourahmadi and Wu (2017), which is available for spatial
processes only, and the one of Wu and Zaffaroni (2018) and Wu (2005), which
study the time series case.

Consider a generic $n$-dimensional rf:
${\bm{Y}}_{n}=\\{\bm{Y}_{n\bm{\varsigma}}=(Y_{1\bm{\varsigma}}\cdots
Y_{n\bm{\varsigma}})^{\top},\bm{\varsigma}\in\mathbb{Z}^{3}\\}$ where, for any
$\ell=1,\ldots,n$

$Y_{\ell\bm{\varsigma}}=F_{\ell}(\bm{\epsilon}_{\bm{\varsigma}-{\bm{k}}};{\bm{k}}\in\mathbb{Z}^{2}\times\mathbb{N}_{0}),$
(D.33)

for some measurable function $F_{\ell}(\cdot)$ a such that
$Y_{\ell\bm{\varsigma}}$ exists and a, possibly infinite dimensional, zero-
mean and i.i.d. rf
$\\{\bm{\epsilon}_{\bm{\varsigma}}=(\epsilon_{1\bm{\varsigma}}\
\epsilon_{2\bm{\varsigma}}\cdots)^{\top};\bm{\varsigma}\in\mathbb{Z}^{3}\\}$.
Let $Y_{\ell\bm{\varsigma}}\in L_{p}$, for some $p\geq 1$, and denote as
$Y^{*}_{\ell\bm{\varsigma}}=F_{\ell}(\bm{\epsilon}^{*}_{\bm{\varsigma}-{\bm{k}}};{\bm{k}}\in\mathbb{Z}^{2}\times\mathbb{N}_{0})$,
with $\bm{\epsilon}^{*}_{\bm{\varsigma}}=\bm{\epsilon}_{\bm{\varsigma}}$ if
$\bm{\varsigma}\neq\bm{0}$ and
$\bm{\epsilon}^{*}_{{\bm{0}}}=\widetilde{\bm{\epsilon}}_{{\bm{0}}}$, where
$\widetilde{\bm{\epsilon}}_{\bm{\varsigma}_{1}},\bm{\epsilon}_{\bm{\varsigma}_{2}}$
are i.i.d. for $\bm{\varsigma}_{1},\bm{\varsigma}_{2}\in\mathbb{Z}^{3}$.
Recall that we can arbitrarily set the location of the origin $\bm{0}$ because
of the Assumption 3.1 (homostationarity). Then, we consider the following
functional dependence measure

$\delta_{\bm{\varsigma},p}^{[\ell]}=\left({\rm
E}|Y_{\ell\bm{\varsigma}}-Y^{*}_{\ell\bm{\varsigma}}|^{p}\right)^{1/p}.$
(D.34)

Let also ${\bm{m}}=(m_{1}\ m_{2}\ m_{3})^{\top}$ and define

$\varphi_{{\bm{m}},p}^{[\ell]}=\sum_{|s_{1}|>m_{1}}\sum_{|s_{2}|>m_{2}}\sum_{t>m_{3}}\delta_{\bm{\varsigma},p}^{[\ell]}.$

The definition of $\varphi_{{\bm{m}},p}^{[\ell]}$ is natural in our spatio-
temporal rf setting: it measures two-sided dependence over space and one-sided
dependence over time for $Y_{\ell\bm{\varsigma}}$. Therefore,
$\varphi_{{\bm{m}},p}^{[\ell]}$ considers a form of dependence which acts in
each direction of $\mathbb{Z}^{3}$: this is different from the approach of
(Deb, Pourahmadi and Wu, 2017, Definition 2.1 and see also their discussion on
p.4315), where the dependence can be only in one spatial dimension.

Given the sample $\\{Y_{\ell\bm{\varsigma}}=x_{\ell(s_{1}\ s_{2}\ t)},\
\ell=1,\ldots,n,\ s_{1}=1,\ldots,S_{1},\ s_{2}=1,\ldots,S_{2},\
t=1,\ldots,T\\}$, we consider the estimator of the spectral spectral density
matrix $\widehat{\bm{\Sigma}}_{n}^{y}(\bm{\theta})$ of $\bm{Y}_{n}$ with
$(i,j)$-th, $i,j=1.\ldots,n$, generic entry

$\widehat{\sigma}_{ij}^{y}(\bm{\theta})=\frac{1}{S_{1}S_{2}T}\sum_{\bm{\varsigma}_{1},\bm{\varsigma}_{2}=(1\
1\ 1)^{\top}}^{(S_{1}\ S_{2}\
T)^{\top}}\\!\\!\\!\\!Y_{i\bm{\varsigma}_{1}}Y_{j\bm{\varsigma}_{2}}K_{1}\left(\frac{s_{11}-s_{21}}{B_{S_{1}}}\right)K_{2}\left(\frac{s_{12}-s_{22}}{B_{S_{2}}}\right)K_{3}\left(\frac{t_{1}-t_{2}}{B_{T}}\right)e^{-i\left\langle\bm{\varsigma}_{1}-\bm{\varsigma}_{2},\bm{\theta}\right\rangle}$
(D.35)

with $\bm{\varsigma}_{1}=(s_{11}\ s_{12}\ t_{1}),\bm{\varsigma}_{2}=(s_{21}\
s_{22}\ t_{2})$, $K_{1}(\cdot),K_{2}(\cdot),K_{3}(\cdot)$ being kernel
functions and $B_{S_{1}},B_{S_{2}},B_{T}$ being bandwidths satisfying
Assumption 6.5(i) and (ii).

The following conditions are imposed

######  Condition 1.

For all $\ell\in\mathbb{N}$, ${\rm E}(Y_{\ell{\bm{0}}})=0$,
$Y_{\ell{\bm{0}}}\in L_{p}$, for some $p\geq 4$.

######  Condition 2.

For all $\ell\in\mathbb{N}$, $\delta_{\bm{\varsigma},p}^{[\ell]}\leq
A_{1}\rho_{1}^{|s_{1}|}\rho_{2}^{|s_{2}|}\rho_{3}^{|t|}$, for some finite
$\rho_{1},\rho_{2},\rho_{3}\in(0,1)$ and $A_{1}>0$, independent of $\ell$.

######  Condition 3.

For all $i,j\in\mathbb{N}$, $|{\rm E}(Y_{i\bm{\varsigma}}Y_{j{\bm{0}}})|\leq
A_{2}\rho_{4}^{|s_{1}|}\rho_{5}^{|s_{2}|}\rho_{6}^{|t|}$, for some finite
$\rho_{4},\rho_{5},\rho_{6}\in(0,1)$ and $A_{2}\in(0,\infty)$, independent of
$i$ and $j$.

Then, we have the following result

######  Proposition D.1.

Let Conditions 1 and 2, and Assumption 6.5 hold. Then, there exists a finite
$C>0$ depending on $p$ such that

$\max_{1\leq i,j\leq n}\sup_{\bm{\theta}\in\bm{\Theta}}\left[{\rm
E}\left|\widehat{\sigma}_{ij}^{y}(\bm{\theta})-{\rm
E}(\widehat{\sigma}_{ij}^{y}(\bm{\theta}))\right|^{p/2}\right]^{2/p}\leq
C\left[\frac{(\log B_{S_{1}}\log B_{S_{2}}\log
B_{T})^{2}B_{S_{1}}B_{S_{2}}B_{T}}{S_{1}S_{2}T}\right]^{1/2}.$

For $p=4$, Proposition D.1 bounds the variance of
$\widehat{\sigma}_{ij}^{y}(\bm{\theta})$. The following result bounds the mean
square error

######  Proposition D.2.

Let Conditions 1, 2, and 3, and Assumption 6.5 hold. Then, there exists a
finite $C^{*}>0$ such that

$\displaystyle\quad\max_{1\leq i,j\leq n}\sup_{\bm{\theta}\in\bm{\Theta}}{\rm
E}\left|\widehat{\sigma}_{ij}^{y}(\bm{\theta})-\sigma_{ij}^{y}(\bm{\theta})\right|^{2}\leq
C^{*}\max\left\\{\frac{(\log B_{S_{1}}\log B_{S_{2}}\log
B_{T})^{2}B_{S_{1}}B_{S_{2}}B_{T}}{S_{1}S_{2}T},\frac{1}{B_{S_{1}}^{2\vartheta_{1}}},\frac{1}{B_{S_{2}}^{2\vartheta_{2}}},\frac{1}{B_{T}^{2\vartheta_{3}}}\right\\}.$

This implies mean-square consistency of
$\widehat{\sigma}_{ij}^{y}(\bm{\theta})$, as $S_{1},S_{2},T\to\infty$.

###  D.1 Proof of Proposition D.1

The following preliminary result is needed to prove Proposition D.1.

######  Lemma D.3.

For any $i=1,\ldots,n$, let $Y_{i\bm{\varsigma}}\in L_{p}$ for some $p>1$,
${\rm E}Y_{i\bm{\varsigma}}=0$, $\alpha_{\bm{\varsigma}}\in\mathbb{C}$ for
$\bm{\varsigma}\in\mathbb{Z}^{3}$,
$A_{\bar{\bm{\varsigma}}}=(\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}|\alpha_{\bm{\varsigma}}|^{2})^{1/2}$
and $C_{p}=18p^{3/2}(p-1)^{-1/2}$. Then, for $\bar{\bm{\varsigma}}=(S_{1}\
S_{2}\ T)^{\top}$, we have

$\left({\rm
E}\left|\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}Y_{i\bm{\varsigma}}\right|^{p}\right)^{1/p}\leq
C_{p}A_{\bar{\bm{\varsigma}}}\varphi_{{\bm{0}},p}^{[i]},\quad\left({\rm
E}\left|\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}\widetilde{Y}_{i\bm{\varsigma}}\right|^{p}\right)^{1/p}\leq
C_{p}A_{\bar{\bm{\varsigma}}}\varphi_{{\bm{0}},p}^{[i]}$

and

$\left({\rm
E}\left|\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}(Y_{i\bm{\varsigma}}-\widetilde{Y}_{i\bm{\varsigma}})\right|^{p}\right)^{1/p}\leq
C_{p}A_{\bar{\bm{\varsigma}}}\varphi_{{\bm{m}}+{\bm{1}},p}^{[i]}.$

###### Proof.

Let $\bm{\tau}:\mathbb{Z}\rightarrow\mathbb{Z}^{3}$ be a bijection, which is
non-decreasing in the third coordinate. Define for $l\in\mathbb{Z}$ and
$(\bm{\tau}(l))_{3}\leq t$, where $(\bm{\tau}(l))_{3}$ means the third
coordinate of $\bm{\tau}(l)$, the projection

$P_{l}(Y_{i\bm{\varsigma}})={\rm E}(Y_{i\bm{\varsigma}}|\mathcal{F}_{l})-{\rm
E}(Y_{i\bm{\varsigma}}|\mathcal{F}_{l-1}),$

where $\mathcal{F}_{l}=\sigma(\bm{\epsilon}_{\bm{\tau}(h)};h\leq l).$ Define
for ${\bm{k}}\in\mathbb{Z}^{2}\times\mathbb{N}_{0}$, the shift
$\mathcal{T}^{{\bm{k}}}\mathcal{F}_{l}=\sigma(\bm{\epsilon}_{\bm{\tau}(h)-{\bm{k}}};h\leq
l).$ By (D.33), we then have

$\displaystyle\quad\left({\rm
E}\left|P_{l}(Y_{i\bm{\varsigma}})\right|^{p}\right)^{1/p}$
$\displaystyle=\left({\rm E}\left|{\rm
E}(Y_{i\bm{\varsigma}}|\mathcal{F}_{l})-{\rm
E}(Y_{i\bm{\varsigma}}|\mathcal{F}_{l-1})\right|^{p}\right)^{1/p}$
$\displaystyle=\left({\rm E}\left|{\rm
E}(Y_{i{\bm{0}}}|\mathcal{T}^{\bm{\varsigma}}\mathcal{F}_{l})-{\rm
E}(Y_{i{\bm{0}}}|\mathcal{T}^{\bm{\varsigma}}\mathcal{F}_{l-1})\right|^{p}\right)^{1/p}$
$\displaystyle=\left\\{{\rm E}\left|{\rm
E}[F_{i}(\bm{\epsilon}_{-{\bm{k}}};{\bm{k}}\in\mathbb{Z}\times\mathbb{Z}\times\mathbb{N}_{0})|\mathcal{T}^{\bm{\varsigma}}\mathcal{F}_{l}]-{\rm
E}[F_{i}(\bm{\epsilon}_{-{\bm{k}}},\widetilde{\bm{\epsilon}}_{\bm{\tau}(l)-\bm{\varsigma}};{\bm{k}}\in\mathbb{Z}\times\mathbb{Z}\times\mathbb{N}_{0}\setminus\\{\bm{\varsigma}-\bm{\tau}(l)\\})|\mathcal{T}^{\bm{\varsigma}}\mathcal{F}_{l}]\right|^{p}\right\\}^{1/p}$
$\displaystyle\leq\left({\rm
E}\left|F_{i}(\bm{\epsilon}_{-{\bm{k}}};{\bm{k}}\in\mathbb{Z}\times\mathbb{Z}\times\mathbb{N}_{0})-F_{i}(\bm{\epsilon}_{-{\bm{k}}},\widetilde{\bm{\epsilon}}_{\bm{\tau}(l)-\bm{\varsigma}};{\bm{k}}\in\mathbb{Z}\times\mathbb{Z}\times\mathbb{N}_{0}\setminus\\{\bm{\varsigma}-\bm{\tau}(l)\\})\right|^{p}\right)^{1/p}$
$\displaystyle=\left({\rm
E}\left|F_{i}(\bm{\epsilon}_{\bm{\varsigma}-\bm{\tau}(l)-{\bm{k}}};{\bm{k}}\in\mathbb{Z}\times\mathbb{Z}\times\mathbb{N}_{0})-F_{i}(\bm{\epsilon}_{\bm{\varsigma}-\bm{\tau}(l)-{\bm{k}}},\widetilde{\bm{\epsilon}}_{{\bm{0}}};{\bm{k}}\in\mathbb{Z}\times\mathbb{Z}\times\mathbb{N}_{0}\setminus\\{\bm{\varsigma}-\bm{\tau}(l)\\})\right|^{p}\right)^{1/p}$
$\displaystyle=\left({\rm
E}\left|Y_{i\bm{\varsigma}-\bm{\tau}(l)}-Y^{*}_{i\bm{\varsigma}-\bm{\tau}(l)}\right|^{p}\right)^{1/p}$
$\displaystyle=\delta_{\bm{\varsigma}-\bm{\tau}(l),p}^{[i]}.$

Let $\mathbb{E}_{t}=\\{l\in\mathbb{Z}:(\bm{\tau}(l))_{3}\leq t\\}$. Noting
that
$\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}Y_{i\bm{\varsigma}}=\sum_{l\in\mathbb{E}_{T}}P_{l}(\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}Y_{i\bm{\varsigma}})$,
we have

$\left({\rm
E}\left|\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}Y_{i\bm{\varsigma}}\right|^{p}\right)^{1/p}=\left({\rm
E}\left|\sum_{l\in\mathbb{E}_{T}}P_{l}\left(\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}Y_{i\bm{\varsigma}}\right)\right|^{p}\right)^{1/p},$

where
$\\{P_{l}\left(\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}Y_{i\bm{\varsigma}}\right);l\in\mathbb{E}_{T}\\}$
forms a martingale difference sequence in time, that is,

${\rm
E}\left[P_{l}\left(\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}Y_{i\bm{\varsigma}}\right)|\mathcal{F}_{l-1}\right]=0.$

Hence, by Minkowski’s, Cauchy-Schwarz’s and Burkholder’s inequalities (see Wu
and Shao 2007, Lemma 1),

$\displaystyle\left({\rm
E}\left|\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}Y_{i\bm{\varsigma}}\right|^{p}\right)^{2/p}$
$\displaystyle\leq C_{p}^{2}\sum_{l\in\mathbb{E}_{T}}\left({\rm
E}\left|P_{l}\left(\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}Y_{i\bm{\varsigma}}\right)\right|^{p}\right)^{2/p}$
$\displaystyle\leq
C_{p}^{2}\sum_{l\in\mathbb{E}_{T}}\left(\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\left|\alpha_{\bm{\varsigma}}\right|\delta_{\bm{\varsigma}-\bm{\tau}(l),p}^{[i]}\right)^{2}$
$\displaystyle\leq
C_{p}^{2}\sum_{l\in\mathbb{E}_{T}}\left(\left(\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\left|\alpha_{\bm{\varsigma}}\right|^{2}\delta_{\bm{\varsigma}-\bm{\tau}(l),p}^{[i]}\right)\left(\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\delta_{\bm{\varsigma}-\bm{\tau}(l),p}^{[i]}\right)\right)$
$\displaystyle\leq
C_{p}^{2}\sum_{l\in\mathbb{E}_{T}}\left(\left(\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\left|\alpha_{\bm{\varsigma}}\right|^{2}\delta_{\bm{\varsigma}-\bm{\tau}(l),p}^{[i]}\right)\varphi_{{\bm{0}},p}^{[i]}\right)$
$\displaystyle\leq
C_{p}^{2}\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}^{2}\left(\varphi_{{\bm{0}},p}^{[i]}\right)^{2},$

which implies $\left({\rm
E}\left|\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}Y_{i\bm{\varsigma}}\right|^{p}\right)^{1/p}\leq
C_{p}A_{\bar{\bm{\varsigma}}}\varphi_{{\bm{0}},p}^{[i]}.$

Repeating the similar arguments for $\widetilde{Y}_{i\bm{\varsigma}}$ yields
$\left({\rm
E}\left|\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}\widetilde{Y}_{i\bm{\varsigma}}\right|^{p}\right)^{1/p}\leq
C_{p}A_{\bar{\bm{\varsigma}}}\varphi_{{\bm{0}},p}^{[i]}$. Noting that

$\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}(Y_{i\bm{\varsigma}}-\widetilde{Y}_{i\bm{\varsigma}})=\sum_{l\in\mathbb{F}_{T}}P_{l}\left(\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\alpha_{\bm{\varsigma}}(Y_{i\bm{\varsigma}}-\widetilde{Y}_{i\bm{\varsigma}})\right),$

where
$\mathbb{F}_{T}=\\{l\in\mathbb{Z}:|\bm{\tau}(l))_{1}|>m_{1},|\bm{\tau}(l))_{2}|>m_{2},T\geq(\bm{\tau}(l))_{3}>m_{3}\\}$,
the last inequality follows from repeating the similar arguments as above. ∎

In the sequel, $A_{1},A_{2},...$ will denote generic positive constants.
Letting

$a_{\bar{\bm{\varsigma}},{\bm{r}}}=K_{1}(r_{1}/B_{S_{1}})K_{2}(r_{2}/B_{S_{2}})K_{3}(r_{3}/B_{T})e^{-i\left\langle{\bm{r}},\bm{\theta}\right\rangle},$

where ${\bm{r}}=(r_{1}\ r_{2}\ r_{3})^{\top}\in\mathbb{Z}^{3}$, then (D.35)
can be written as

$\widehat{\sigma}_{ij}^{y}(\bm{\theta})=\frac{1}{\mathcal{R}}\sum_{\bm{\varsigma}_{1}={\bm{1}}}^{\bar{\bm{\varsigma}}}\sum_{\bm{\varsigma}_{2}={\bm{1}}}^{\bar{\bm{\varsigma}}}Y_{i\bm{\varsigma}_{1}}Y_{j\bm{\varsigma}_{2}}a_{\bar{\bm{\varsigma}},\bm{\varsigma}_{1}-\bm{\varsigma}_{2}}.$

Assumption 6.5 implies that
$\sum_{{\bm{r}}\in\mathbb{Z}^{3}}|a_{\bar{\bm{\varsigma}},{\bm{r}}}|^{2}/(B_{S_{1}}B_{S_{2}}B_{T})<\infty$.
Let $\bm{h}=(h_{1}\ h_{2}\ h_{3})^{\top}\in\mathbb{Z}^{3}$ and
$\mathcal{R}=S_{1}S_{2}T$. Then, we consider an approximation of
$\widehat{\sigma}_{ij}^{y}(\bm{\theta})$ by

$\displaystyle\widetilde{\sigma}_{ij}^{y}(\bm{\theta})$
$\displaystyle=\frac{1}{\mathcal{R}}\sum_{\bm{\varsigma}_{1}={\bm{1}}}^{\bar{\bm{\varsigma}}}\sum_{\bm{\varsigma}_{2}={\bm{1}}}^{\bar{\bm{\varsigma}}}\widetilde{Y}_{i\bm{\varsigma}_{1}}\widetilde{Y}_{j\bm{\varsigma}_{2}}a_{\bar{\bm{\varsigma}},\bm{\varsigma}_{1}-\bm{\varsigma}_{2}}$
$\displaystyle=\frac{1}{\mathcal{R}}\sum_{\bm{\varsigma}_{1}={\bm{1}}}^{\bar{\bm{\varsigma}}}\widetilde{Y}_{i\bm{\varsigma}_{1}}\widetilde{Y}_{j\bm{\varsigma}_{1}}+\frac{2}{\mathcal{R}}\sum_{\bm{\varsigma}_{2}={\bm{2}}}^{\bar{\bm{\varsigma}}}\widetilde{Y}_{j\bm{\varsigma}_{2}}\sum_{s_{11}=\max(1,s_{21}-h_{1})}^{s_{21}-1}\sum_{s_{12}=\max(1,s_{22}-h_{2})}^{s_{22}-1}\sum_{t_{1}=\max(1,t_{2}-h_{3})}^{t_{2}-1}a_{\bar{\bm{\varsigma}},\bm{\varsigma}_{1}-\bm{\varsigma}_{2}}\widetilde{Y}_{i\bm{\varsigma}_{1}}$
$\displaystyle\quad+\frac{2}{\mathcal{R}}\sum_{\bm{\varsigma}_{2}={\bm{h}}+{\bm{1}}}^{\bar{\bm{\varsigma}}}\widetilde{Z}_{ij,\bm{\varsigma}_{2}},$
(D.36)

where
$\widetilde{Z}_{ij,\bm{\varsigma}_{2}}=\widetilde{Y}_{j\bm{\varsigma}_{2}}\sum_{\bm{\varsigma}_{1}={\bm{1}}}^{\bm{\varsigma}_{2}-{\bm{h}}}a_{\bar{\bm{\varsigma}},\bm{\varsigma}_{1}-\bm{\varsigma}_{2}}\widetilde{Y}_{i\bm{\varsigma}_{1}}$.

Now let us consider the $\sigma$-field

$\mathcal{F}_{{\bm{m}},\bm{\varsigma}}=\sigma(\bm{\epsilon}_{\bm{\varsigma}-{\bm{k}}};|k_{1}|\leq
m_{1},|k_{2}|\leq m_{2},0\leq k_{3}\leq m_{3})$

with ${\bm{k}}=(k_{1}\ k_{2}\
k_{3})^{\top}\in\mathbb{Z}^{2}\times\mathbb{N}_{0}$ and ${\bm{m}}=(m_{1}\
m_{2}\ m_{3})^{\top}\in\mathbb{N}_{0}^{3}$. The $\sigma$-field
$\mathcal{F}_{{\bm{m}},\bm{\varsigma}}$ is based on a hyperrectangle in
$\mathbb{Z}^{3}$, where for each coordinate defining the spatio-temporal point
$\bm{\varsigma}-{\bm{k}}$, we consider its distance form $\bm{\varsigma}$. We
refer to El Machkouri and Volnỳ (2003) p. 328 for a discussion on other
options which can be considered to define this $\sigma$-field. For the sake of
completeness of our argument, here we emphasize that our choice for
$\mathcal{F}_{{\bm{m}},\bm{\varsigma}}$ is the natural one to consider in our
spatio-temporal setting, where we impose homostationarity (see Assumption 3.1)
and we are interested in controlling the rf functional dependence in each
direction of $\mathbb{Z}^{3}$.

Note that
$\\{\widetilde{Z}_{ij,\bm{\varsigma}_{2}+l\bm{h}};l\in\mathbb{N}_{0}\\}$ are
martingale differences when $|h_{1}|>m_{1},|h_{2}|>m_{2},h_{3}>m_{3}$, that
is, ${\rm
E}[\widetilde{Z}_{ij,\bm{\varsigma}_{2}+l\bm{h}}|\mathcal{F}_{{\bm{m}},\bm{\varsigma}_{2}+(l-1)\bm{h}}]=0$.
Taking $|h_{1}|=2m_{1},|h_{2}|=2m_{2},h_{3}=2m_{3}$ and using Lemma D.3, there
exists $A_{3}>0$ such that

$\displaystyle\frac{\max_{1\leq i,j\leq n}\left({\rm
E}\left|\widetilde{Z}_{ij,\bm{\varsigma}_{2}}\right|^{p}\right)^{1/p}}{(B_{S_{1}}B_{S_{2}}B_{T})^{1/2}}$
$\displaystyle=\max_{1\leq i,j\leq n}\left({\rm
E}\left|\widetilde{Y}_{j\bm{\varsigma}_{2}}\right|^{p}\right)^{1/p}\frac{\left({\rm
E}\left|\sum_{\bm{\varsigma}_{1}={\bm{1}}}^{\bm{\varsigma}_{2}-{\bm{h}}}a_{\bar{\bm{\varsigma}},\bm{\varsigma}_{1}-\bm{\varsigma}_{2}}\widetilde{Y}_{i\bm{\varsigma}_{1}}\right|^{p}\right)^{1/p}}{(B_{S_{1}}B_{S_{2}}B_{T})^{1/2}}$
$\displaystyle\leq\max_{1\leq i,j\leq n}\left({\rm
E}\left|\widetilde{Y}_{j{\bm{0}}}\right|^{p}\right)^{1/p}C_{p}\varphi_{{\bm{0}},p}^{[i]}\left(\frac{\sum_{\bm{\varsigma}_{1}={\bm{1}}}^{\bm{\varsigma}_{2}-{\bm{h}}}|a_{\bar{\bm{\varsigma}},\bm{\varsigma}_{1}-\bm{\varsigma}_{2}}|^{2}}{B_{S_{1}}B_{S_{2}}B_{T}}\right)^{1/2}\leq
A_{3}$

due to Conditions 1 and 2. For the last term in (D.36), we consider splitting
the interval $[h_{1}+1,S_{1}]\times[h_{2}+1,S_{2}]\times[h_{3}+1,T]$ into
consecutive blocks, each of which has same size $2m_{1}\times 2m_{2}\times
m_{3}$ (here, without loss of generality, we assume
$(S_{1}-h_{1}-1)/(2m_{1}),(S_{2}-h_{2}-1)/(2m_{2}),(T-h_{3}-1)/m_{3}\in\mathbb{N}$).
There are
$C^{*}=(S_{1}-h_{1}-1)(S_{2}-h_{2}-1)(T-h_{3}-1)/(4m_{1}m_{2}m_{3})=O(\mathcal{R}/(4m_{1}m_{2}m_{3}))$
number of such blocks. In this manner, for $\bm{\varsigma}_{2}$ in the non-
consecutive blocks,
$\\{\widetilde{Z}_{ij,\bm{\varsigma}_{2}};{\bm{h}}+{\bm{1}}\leq\bm{\varsigma}_{2}\leq\bar{\bm{\varsigma}}\\}$
forms a martingale difference sequence. Therefore, we have

$\displaystyle\quad\frac{\max_{1\leq i,j\leq n}\left({\rm
E}\left|\frac{2}{\mathcal{R}}\sum_{\bm{\varsigma}_{2}={\bm{h}}+{\bm{1}}}^{\bar{\bm{\varsigma}}}\widetilde{Z}_{ij,\bm{\varsigma}_{2}}\right|^{p}\right)^{1/p}}{\left(m_{1}m_{2}m_{3}B_{S_{1}}B_{S_{2}}B_{T}/\mathcal{R}\right)^{1/2}}$
$\displaystyle\leq\max_{1\leq i,j\leq
n}\frac{4}{4m_{1}m_{2}m_{3}}\sum_{a_{1}=1}^{2m_{1}}\sum_{a_{2}=1}^{2m_{2}}\sum_{a_{3}=1}^{m_{3}}\frac{\left({\rm
E}\left|\left(\frac{\mathcal{R}}{4m_{1}m_{2}m_{3}}\right)^{-1/2}\sum_{l=1}^{C^{*}}\widetilde{Z}_{ij,\bm{\varsigma}_{2}+l\bm{h}}\right|^{p}\right)^{1/p}}{(B_{S_{1}}B_{S_{2}}B_{T})^{1/2}}\leq
A_{4}.$ (D.37)

Now, Let $\widetilde{\Gamma}_{ij,\bm{\varsigma}_{1}}={\rm
E}(\widetilde{Y}_{i{\bm{0}}}\widetilde{Y}_{j\bm{\varsigma}_{1}})$ and consider
the first two terms in (D.36) together in view of the fact that for any
$\bm{\varsigma}_{1}\in\mathbb{Z}^{3}$,

$\underset{\mathcal{R}\rightarrow\infty}{\lim}\frac{\max_{1\leq i,j\leq
n}\left({\rm
E}\left|\mathcal{R}^{-1}\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\widetilde{Y}_{i\bm{\varsigma}}\widetilde{Y}_{j\bm{\varsigma}+\bm{\varsigma}_{1}}-\widetilde{\Gamma}_{ij,\bm{\varsigma}_{1}}\right|^{p/2}\right)^{2/p}}{(m_{1}m_{2}m_{3}/\mathcal{R})^{1/2}}<\infty$

due to the central limit theorem for $m$-dependent process (see, e.g., Lehmann
1999, Theorem 2.8.1) and Conditions 1 and 2. Note that in the first two terms,
we are combining all the terms of the form
$\widetilde{Y}_{i\bm{\varsigma}_{1}}\widetilde{Y}_{j\bm{\varsigma}_{1}+{\bm{k}}}$
where $|k_{1}|\leq h_{1},|k_{2}|\leq h_{2},0\leq k_{3}\leq h_{3}$. Denoting by
$\widetilde{W}_{ij,\mathcal{R}}/\mathcal{R}$ the last term in (D.36) and
considering that $K_{1},K_{2},K_{3}$ are bounded with support $[-1,1]$ because
of Assumption 6.5, we then have

$\displaystyle\quad\frac{\max_{1\leq i,j\leq n}\left({\rm
E}\left|\widetilde{\sigma}_{ij}^{y}(\bm{\theta})-\widetilde{W}_{ij,\mathcal{R}}/\mathcal{R}-{\rm
E}[\widetilde{\sigma}_{ij}^{y}(\bm{\theta})-\widetilde{W}_{ij,\mathcal{R}}/\mathcal{R}]\right|^{p/2}\right)^{2/p}}{\left(m_{1}^{2}m_{2}^{2}m_{3}^{2}B_{S_{1}}B_{S_{2}}B_{T}/\mathcal{R}\right)^{1/2}}$
$\displaystyle\leq\frac{1}{(m_{1}m_{2}m_{3}B_{S_{1}}B_{S_{2}}B_{T})^{1/2}}\sum_{k_{1}=1}^{2m_{1}}\sum_{k_{2}=1}^{2m_{2}}\sum_{k_{3}=1}^{2m_{3}}|a_{\bar{\bm{\varsigma}},{\bm{k}}}|A_{5}$
$\displaystyle\leq\frac{A_{6}\min(8m_{1}m_{2}m_{3},B_{S_{1}}B_{S_{2}}B_{T})}{(m_{1}m_{2}m_{3}B_{S_{1}}B_{S_{2}}B_{T})^{1/2}}A_{5}\leq
A_{7}.$

Now, due to (D.37) and the above inequality, there exists $A_{8}>0$ such that

$\frac{\max_{1\leq i,j\leq n}\left({\rm
E}\left|\widetilde{\sigma}_{ij}^{y}(\bm{\theta})-{\rm
E}[\widetilde{\sigma}_{ij}^{y}(\bm{\theta})]\right|^{p/2}\right)^{2/p}}{\left(m_{1}^{2}m_{2}^{2}m_{3}^{2}B_{S_{1}}B_{S_{2}}B_{T}/\mathcal{R}\right)^{1/2}}\leq
A_{8}.$

Define
$U_{\bar{\bm{\varsigma}}}^{[i]}(\bm{\theta})=\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}Y_{i\bm{\varsigma}}e^{-i\left\langle\bm{\varsigma},\bm{\theta}\right\rangle}$
and
$\widetilde{U}_{\bar{\bm{\varsigma}}}^{[i]}(\bm{\theta})=\sum_{\bm{\varsigma}={\bm{1}}}^{\bar{\bm{\varsigma}}}\widetilde{Y}_{i\bm{\varsigma}}e^{-i\left\langle\bm{\varsigma},\bm{\theta}\right\rangle}$.
Lemma D.3 then implies that

$\left({\rm
E}\left|U_{\bar{\bm{\varsigma}}}^{[i]}(\bm{\theta})-\widetilde{U}_{\bar{\bm{\varsigma}}}^{[i]}(\bm{\theta})\right|^{p}\right)^{1/p}\leq
C_{p}\mathcal{R}^{1/2}\varphi_{{\bm{m}}+{\bm{1}},p}^{[i]}\ \text{and}\
\left({\rm
E}\left|U_{\bar{\bm{\varsigma}}}^{[i]}(\bm{\theta})\right|^{p}\right)^{1/p}+\left({\rm
E}\left|\widetilde{U}_{\bar{\bm{\varsigma}}}^{[i]}(\bm{\theta})\right|^{p}\right)^{1/p}\leq
2C_{p}\mathcal{R}^{1/2}\varphi_{{\bm{0}},p}^{[i]}.$

Denote by $\widehat{K}_{1},\widehat{K}_{2},\widehat{K}_{3}$ the Fourier
transforms of $K_{1},K_{2},K_{3}$, respectively, we then have

$\widehat{\sigma}_{ij}^{y}(\bm{\theta})=\frac{1}{\mathcal{R}}\int_{\mathbb{R}^{3}}\widehat{K}_{1}(u_{1})\widehat{K}_{2}(u_{2})\widehat{K}_{3}(u_{3})U_{\bar{\bm{\varsigma}}}^{[i]}(\bm{\theta}^{*}_{{\bm{u}}})\bar{U}_{\bar{\bm{\varsigma}}}^{[j]}(\bm{\theta}^{*}_{{\bm{u}}}){\rm
d}{\bm{u}},$

where $\bm{\theta}^{*}_{{\bm{u}}}=(B_{S_{1}}^{-1}u_{1}\ B_{S_{2}}^{-1}u_{2}\
B_{T}^{-1}u_{3})+\bm{\theta}$, ${\bm{u}}=(u_{1}\ u_{2}\ u_{3})$ and
$\bar{U}_{\bar{\bm{\varsigma}}}^{[j]}$ denotes the conjugate of
$U_{\bar{\bm{\varsigma}}}^{[j]}$. Hence, we have

$\displaystyle\quad\left({\rm
E}\left|\widehat{\sigma}_{ij}^{y}(\bm{\theta})-\widetilde{\sigma}_{ij}^{y}(\bm{\theta})\right|^{p/2}\right)^{2/p}$
$\displaystyle\leq\frac{1}{\mathcal{R}}\int_{\mathbb{R}^{3}}|\widehat{K}_{1}(u_{1})\widehat{K}_{2}(u_{2})\widehat{K}_{3}(u_{3})|\left({\rm
E}\left|\left(U_{\bar{\bm{\varsigma}}}^{[i]}(\bm{\theta}^{*}_{{\bm{u}}})-\widetilde{U}_{\bar{\bm{\varsigma}}}^{[i]}(\bm{\theta}^{*}_{{\bm{u}}})\right)\bar{U}_{\bar{\bm{\varsigma}}}^{[j]}(\bm{\theta}^{*}_{{\bm{u}}})\right|^{p/2}\right)^{2/p}{\rm
d}{\bm{u}}$
$\displaystyle\quad+\frac{1}{\mathcal{R}}\int_{\mathbb{R}^{3}}|\widehat{K}_{1}(u_{1})\widehat{K}_{2}(u_{2})\widehat{K}_{3}(u_{3})|\left({\rm
E}\left|\widetilde{U}_{\bar{\bm{\varsigma}}}^{[i]}(\bm{\theta}^{*}_{{\bm{u}}})\left(\bar{U}_{\bar{\bm{\varsigma}}}^{[j]}(\bm{\theta}^{*}_{{\bm{u}}})-\bar{\widetilde{U}}_{\bar{\bm{\varsigma}}}^{[j]}(\bm{\theta}^{*}_{{\bm{u}}})\right)\right|^{p/2}\right)^{2/p}{\rm
d}{\bm{u}}$
$\displaystyle\leq\int_{\mathbb{R}^{3}}|\widehat{K}_{1}(u_{1})\widehat{K}_{2}(u_{2})\widehat{K}_{3}(u_{3})|C_{p}^{2}\left(\varphi_{{\bm{m}}+{\bm{1}},p}^{[i]}\varphi_{{\bm{0}},p}^{[j]}+\varphi_{{\bm{0}},p}^{[i]}\varphi_{{\bm{m}}+{\bm{1}},p}^{[j]}\right){\rm
d}{\bm{u}},$ $\displaystyle\leq
A_{9}\left(\varphi_{{\bm{m}}+{\bm{1}},p}^{[i]}\varphi_{{\bm{0}},p}^{[j]}+\varphi_{{\bm{0}},p}^{[i]}\varphi_{{\bm{m}}+{\bm{1}},p}^{[j]}\right).$

Now, combining all the results from above, we have

$\displaystyle\quad\max_{1\leq i,j\leq
n}\sup_{\bm{\theta}\in\bm{\Theta}}\left({\rm
E}\left|\widehat{\sigma}_{ij}^{y}(\bm{\theta})-{\rm
E}\widehat{\sigma}_{ij}^{y}(\bm{\theta})\right|^{p/2}\right)^{2/p}$
$\displaystyle\leq\max_{1\leq i,j\leq
n}\sup_{\bm{\theta}\in\bm{\Theta}}\left[\left({\rm
E}\left|\widehat{\sigma}_{ij}^{y}(\bm{\theta})-\widetilde{\sigma}_{ij}^{y}(\bm{\theta})\right|^{p/2}\right)^{2/p}+|{\rm
E}\left(\widehat{\sigma}_{ij}^{y}(\bm{\theta})-\widetilde{\sigma}_{ij}^{y}(\bm{\theta})\right)|+\left({\rm
E}\left|\widetilde{\sigma}_{ij}^{y}(\bm{\theta})-{\rm
E}\widetilde{\sigma}_{ij}^{y}(\bm{\theta})\right|^{p/2}\right)^{2/p}\right]$
$\displaystyle\leq A_{10}\max_{1\leq i\leq
n}\varphi_{{\bm{m}}+{\bm{1}},p}^{[i]}+A_{8}\left(m_{1}^{2}m_{2}^{2}m_{3}^{2}B_{S_{1}}B_{S_{2}}B_{T}/\mathcal{R}\right)^{1/2}.$

Under Conditions 1 and 2, there exists $A_{11}\in(0,\infty)$ such that
$\varphi_{{\bm{m}}+{\bm{1}},p}^{[i]}<A_{11}\rho_{1}^{m_{1}}\rho_{2}^{m_{2}}\rho_{3}^{m_{3}}$.
Take
$m_{1}=-\log_{\rho_{1}}(B_{S_{1}}^{a_{1}}),m_{2}=-\log_{\rho_{2}}(B_{S_{2}}^{a_{2}}),m_{3}=-\log_{\rho_{3}}(B_{T}^{a_{3}})$
such that $a_{1},a_{2},a_{3}>0$ and
$a_{1}>(1/b_{1}-1)/2,a_{2}>(1/b_{1}^{*}-1)/2,a_{3}>(1/b_{1}^{**}-1)/2$. This
yields

$\varphi_{{\bm{m}}+{\bm{1}},p}^{[i]}<A_{11}B_{S_{1}}^{-a_{1}}B_{S_{2}}^{-a_{2}}B_{T}^{-a_{3}}=o(B_{S_{1}}^{(1-1/b_{1})/2}B_{S_{2}}^{(1-1/b_{1}^{*})/2}B_{T}^{(1-1/b_{1}^{**})/2})=o((B_{S_{1}}B_{S_{2}}B_{T}/\mathcal{R})^{1/2}),$

and $m_{1}m_{2}m_{3}=A_{12}\log(B_{S_{1}})\log(B_{S_{2}})\log(B_{T})$ for a
positive constant $A_{12}$. Hence, there exists $C>0$ such that

$\displaystyle\frac{\max_{1\leq i,j\leq
n}\sup_{\bm{\theta}\in\bm{\Theta}}\left({\rm
E}\left|\widehat{\sigma}_{ij}^{y}(\bm{\theta})-{\rm
E}\widehat{\sigma}_{ij}^{y}(\bm{\theta})\right|^{p/2}\right)^{2/p}}{\left(\log(B_{S_{1}})\log(B_{S_{2}})\log(B_{T}))^{2}B_{S_{1}}B_{S_{2}}B_{T}/\mathcal{R}\right)^{1/2}}$
$\displaystyle\leq\frac{A_{10}\max_{1\leq i\leq
n}\varphi_{{\bm{m}}+{\bm{1}},p}^{[i]}+A_{8}\left(m_{1}^{2}m_{2}^{2}m_{3}^{2}B_{S_{1}}B_{S_{2}}B_{T}/\mathcal{R}\right)^{1/2}}{A_{12}^{-1}\left(m_{1}^{2}m_{2}^{2}m_{3}^{2}B_{S_{1}}B_{S_{2}}B_{T}/\mathcal{R}\right)^{1/2}}<C.$

This completes the proof.

###  D.2 Proof of Proposition D.2

Noting that

${\rm
E}|\widehat{\sigma}_{ij}^{y}(\bm{\theta})-\sigma_{ij}^{y}(\bm{\theta})|^{2}\leq
2\left[{\rm E}|\widehat{\sigma}_{ij}^{y}(\bm{\theta})-{\rm
E}\widehat{\sigma}_{ij}^{y}(\bm{\theta})|^{2}+{\rm E}|{\rm
E}\widehat{\sigma}_{ij}^{y}(\bm{\theta})-\sigma_{ij}^{y}(\bm{\theta})|^{2}\right],$

where, because of Proposition D.1 when $p=4$, the first term on the right-hand
side satisfies

$\max_{1\leq i,j\leq n}\sup_{\bm{\theta}\in\bm{\Theta}}{\rm
E}|\widehat{\sigma}_{ij}^{y}(\bm{\theta})-{\rm
E}\widehat{\sigma}_{ij}^{y}(\bm{\theta})|^{2}\leq
C^{2}(\log(B_{S_{1}})\log(B_{S_{2}})\log(B_{T}))^{2}B_{S_{1}}B_{S_{2}}B_{T}/\mathcal{R},$
(D.38)

it only remains to deal with the second term.

Since

$\displaystyle{\rm E}\widehat{\sigma}_{ij}^{y}(\bm{\theta})$
$\displaystyle=\frac{1}{\mathcal{R}}\sum_{\bm{\varsigma}_{1}={\bm{1}}}^{\bar{\bm{\varsigma}}}\sum_{\bm{\varsigma}_{2}={\bm{1}}}^{\bar{\bm{\varsigma}}}\Gamma_{ij,\bm{\varsigma}_{1}-\bm{\varsigma}_{2}}^{y}K_{1}\left(\frac{s_{11}-s_{21}}{B_{S_{1}}}\right)K_{2}\left(\frac{s_{12}-s_{22}}{B_{S_{2}}}\right)K_{3}\left(\frac{t_{1}-t_{2}}{B_{T}}\right)e^{-i\left\langle\bm{\varsigma}_{1}-\bm{\varsigma}_{2},\bm{\theta}\right\rangle}$
$\displaystyle=\frac{1}{\mathcal{R}}\sum_{s_{1}=1-S_{1}}^{S_{1}-1}\sum_{s_{2}=1-S_{2}}^{S_{2}-1}\sum_{t=1-T}^{T-1}(S_{1}-|s_{1}|)(S_{2}-|s_{2}|)(T-|t|)\Gamma_{ij,\bm{\varsigma}}K_{1}\left(\frac{s_{1}}{B_{S_{1}}}\right)K_{2}\left(\frac{s_{2}}{B_{S_{2}}}\right)K_{3}\left(\frac{t}{B_{T}}\right)e^{-i\left\langle\bm{\varsigma},\bm{\theta}\right\rangle}$
$\displaystyle=\sum_{s_{1}=1-S_{1}}^{S_{1}-1}\sum_{s_{2}=1-S_{2}}^{S_{2}-1}\sum_{t=1-T}^{T-1}\left(1-\frac{|s_{1}|}{S_{1}}\right)\left(1-\frac{|s_{2}|}{S_{2}}\right)\left(1-\frac{|t|}{T}\right)\Gamma_{ij,\bm{\varsigma}}K_{1}\left(\frac{s_{1}}{B_{S_{1}}}\right)K_{2}\left(\frac{s_{2}}{B_{S_{2}}}\right)K_{3}\left(\frac{t}{B_{T}}\right)e^{-i\left\langle\bm{\varsigma},\bm{\theta}\right\rangle},$

we have

$\displaystyle\left|\left({\rm
E}\widehat{\sigma}_{ij}^{y}(\bm{\theta})-\sigma_{ij}^{y}(\bm{\theta})\right)\right|$
$\displaystyle\leq\left|\sum_{s_{1}=1-S_{1}}^{S_{1}-1}\sum_{s_{2}=1-S_{2}}^{S_{2}-1}\sum_{t=1-T}^{T-1}\left(K_{1}\left(\frac{s_{1}}{B_{S_{1}}}\right)K_{2}\left(\frac{s_{2}}{B_{S_{2}}}\right)K_{3}\left(\frac{t}{B_{T}}\right)-1\right)\Gamma_{ij,\bm{\varsigma}}e^{-i\left\langle\bm{\varsigma},\bm{\theta}\right\rangle}\right|$
$\displaystyle+\left|\sum_{s_{1}=1-S_{1}}^{S_{1}-1}\sum_{s_{2}=1-S_{2}}^{S_{2}-1}\sum_{t=1-T}^{T-1}\left[\left(1-\frac{|s_{1}|}{S_{1}}\right)\left(1-\frac{|s_{2}|}{S_{2}}\right)\left(1-\frac{|t|}{T}\right)-1\right]\Gamma_{ij,\bm{\varsigma}}\right.$
$\displaystyle\quad\times\left.K_{1}\left(\frac{s_{1}}{B_{S_{1}}}\right)K_{2}\left(\frac{s_{2}}{B_{S_{2}}}\right)K_{3}\left(\frac{t}{B_{T}}\right)e^{-i\left\langle\bm{\varsigma},\bm{\theta}\right\rangle}\right|$
$\displaystyle+\left|\sum_{|s_{1}|\geq S_{1}}\sum_{|s_{2}|\geq
S_{2}}\sum_{|t|\geq
T}\Gamma_{ij,\bm{\varsigma}}e^{-i\left\langle\bm{\varsigma},\bm{\theta}\right\rangle}\right|$
$\displaystyle=:\mathcal{C}_{ij,\bar{\bm{\varsigma}}}^{[1]}(\bm{\theta})+\mathcal{C}_{ij,\bar{\bm{\varsigma}}}^{[2]}(\bm{\theta})+\mathcal{C}_{ij,\bar{\bm{\varsigma}}}^{[3]}(\bm{\theta}).$

Under Assumptions 6.5 and Condition 3, there exists some $A_{3},A_{4}>0$ such
that, for all $i,j$ and $\bm{\theta}$,

$\displaystyle\mathcal{C}_{ij,\bar{\bm{\varsigma}}}^{[1]}(\bm{\theta})$
$\displaystyle\leq
A_{3}\sum_{s_{1}=-\infty}^{\infty}\sum_{s_{2}=-\infty}^{\infty}\sum_{t=-\infty}^{\infty}\rho_{4}^{|s_{1}|}\rho_{5}^{|s_{2}|}\rho_{6}^{|t|}\left\\{\left(\frac{|s_{1}|}{B_{S_{1}}}\right)^{\vartheta_{1}}+\left(\frac{|s_{2}|}{B_{S_{2}}}\right)^{\vartheta_{2}}+\left(\frac{|t|}{B_{T}}\right)^{\vartheta_{3}}\right.$
$\displaystyle\left.+\left(\frac{|s_{1}|}{B_{S_{1}}}\right)^{\vartheta_{1}}\left(\frac{|s_{2}|}{B_{S_{2}}}\right)^{\vartheta_{2}}+\left(\frac{|s_{1}|}{B_{S_{1}}}\right)^{\vartheta_{1}}\left(\frac{|t|}{B_{T}}\right)^{\vartheta_{3}}+\left(\frac{|s_{2}|}{B_{S_{2}}}\right)^{\vartheta_{2}}\left(\frac{|t|}{B_{T}}\right)^{\vartheta_{3}}+\left(\frac{|s_{1}|}{B_{S_{1}}}\right)^{\vartheta_{1}}\left(\frac{|s_{2}|}{B_{S_{2}}}\right)^{\vartheta_{2}}\left(\frac{|t|}{B_{T}}\right)^{\vartheta_{3}}\right\\}$
$\displaystyle\leq
A_{4}\left\\{B_{S_{1}}^{-\vartheta_{1}}+B_{S_{2}}^{-\vartheta_{2}}+B_{T}^{-\vartheta_{3}}\right\\}.$

Moreover, there exist $A_{5},...,A_{13}>0$ such that, for all $i,j$ and
$\bm{\theta}$,

$\displaystyle\mathcal{C}_{ij,\bar{\bm{\varsigma}}}^{[2]}(\bm{\theta})$
$\displaystyle\leq
A_{5}\sum_{s_{1}=-\infty}^{\infty}\sum_{s_{2}=-\infty}^{\infty}\sum_{t=-\infty}^{\infty}\left(\frac{|s_{1}|}{S_{1}}+\frac{|s_{2}|}{S_{2}}+\frac{|t|}{T}+\frac{|s_{1}s_{2}|}{S_{1}S_{2}}+\frac{|s_{1}t|}{S_{1}T}+\frac{|s_{2}t|}{S_{2}T}+\frac{|s_{1}s_{2}t|}{S_{1}S_{2}T}\right)\rho_{4}^{|s_{1}|}\rho_{5}^{|s_{2}|}\rho_{6}^{|t|}$
$\displaystyle\leq
A_{6}S_{1}^{-1}+A_{7}S_{2}^{-1}+A_{8}T^{-1}+A_{9}(S_{1}S_{2})^{-1}+A_{10}(S_{1}T)^{-1}+A_{11}(S_{2}T)^{-1}+A_{12}(S_{1}S_{2}T)^{-1}$
$\displaystyle\leq A_{13}\max(S_{1}^{-1},S_{2}^{-1},T^{-1})$

and

$\displaystyle\mathcal{C}_{ij,\bar{\bm{\varsigma}}}^{[3]}(\bm{\theta})\leq
A_{2}\sum_{|s_{1}|\geq S_{1}}\sum_{|s_{2}|\geq S_{2}}\sum_{|t|\geq
T}\rho_{4}^{|s_{1}|}\rho_{5}^{|s_{2}|}\rho_{6}^{|t|}=o(\max(S_{1}^{-1},S_{2}^{-1},T^{-1})).$

Therefore,

$\max_{1\leq i,j\leq n}\sup_{\bm{\theta}\in\bm{\Theta}}\left|{\rm
E}\widehat{\sigma}_{ij}^{y}(\bm{\theta})-\sigma_{ij}^{y}(\bm{\theta})\right|^{2}=O\left\\{\max\left[B_{S_{1}}B_{S_{2}}B_{T}/\mathcal{R},B_{S_{1}}^{-2\vartheta_{1}},B_{S_{2}}^{-2\vartheta_{2}},B_{T}^{-2\vartheta_{3}}\right]\right\\},$

which, together with (D.38), completes the proof.

##  Appendix E Proof of results of Section 6

###  E.1 Proof of Theorem 6.1

We first prove the following three propositions.

######  Proposition E.1.

Let Assumption 3.1, 3.2, 6.1, and 6.2 hold. Then, there exists a finite $C>0$
such that
$\sup_{\bm{\theta}\in\bm{\Theta}}\sup_{n\in\mathbb{N}}\lambda_{n1}^{\xi}(\bm{\theta})\leq
C$.

###### Proof.

Let $\sigma_{ij}^{\xi}(\bm{\theta})$ denote the cross-spectral density of
$\xi_{i}$ and $\xi_{j}$. Then, by Assumptions 6.1 and 6.2, for all
$j=1,\ldots,n$ and all $\bm{\theta}\in\bm{\Theta}$,

$\displaystyle\sum_{i=1}^{\infty}|\sigma_{ij}^{\xi}(\bm{\theta})|$
$\displaystyle\leq\sum_{i=1}^{\infty}\sum_{\ell=1}^{\infty}\left|\beta_{i\ell}(\bm{\theta})\bar{\beta}_{j\ell}(\bm{\theta})\right|\leq\sum_{i=1}^{\infty}\sum_{\ell=1}^{\infty}\left(\sum_{\kappa_{1},\kappa_{2}\in\mathbb{Z}}\sum_{\kappa_{3}=0}^{\infty}|\beta_{i\ell,\bm{\kappa}}|\right)\left(\sum_{\kappa_{1},\kappa_{2}\in\mathbb{Z}}\sum_{\kappa_{3}=0}^{\infty}|\bar{\beta}_{j\ell,\bm{\kappa}}|\right)$
$\displaystyle\leq\sum_{i=1}^{\infty}\sum_{\ell=1}^{\infty}\left(\sum_{\kappa_{1},\kappa_{2}\in\mathbb{Z}}\sum_{\kappa_{3}=0}^{\infty}A^{\xi}_{i\ell}{\rho_{1}^{\xi|\kappa_{1}|}}{\rho_{2}^{\xi|\kappa_{2}|}}{\rho_{3}^{\xi\kappa_{3}}}\right)\left(\sum_{\kappa_{1},\kappa_{2}\in\mathbb{Z}}\sum_{\kappa_{3}=0}^{\infty}A^{\xi}_{j\ell}{\rho_{1}^{\xi|\kappa_{1}|}}{\rho_{2}^{\xi|\kappa_{2}|}}{\rho_{3}^{\xi\kappa_{3}}}\right)$
$\displaystyle\leq(A^{\xi})^{2}C_{{\rho_{1}^{\xi}}{\rho_{2}^{\xi}}{\rho_{3}^{\xi}}},$

for some finite $C_{{\rho_{1}^{\xi}}{\rho_{2}^{\xi}}{\rho_{3}^{\xi}}}>0$
depending only on ${\rho_{1}^{\xi}}$, ${\rho_{2}^{\xi}}$, and
${\rho_{3}^{\xi}}$. This yields that

$\sup_{\bm{\theta}\in\bm{\Theta}}\max_{j=1,\ldots,n}\sum_{i=1}^{n}|\sigma_{ij}^{\xi}(\bm{\theta})|\leq\sup_{\bm{\theta}\in\bm{\Theta}}\max_{j=1,\ldots,n}\sum_{i=1}^{\infty}|\sigma_{ij}^{\xi}(\bm{\theta})|\leq(A^{\xi})^{2}C_{{\rho_{1}^{\xi}}{\rho_{2}^{\xi}}{\rho_{3}^{\xi}}}.$

Let $\bm{\Sigma}_{n}^{\xi}(\bm{\theta})$ be the spectral density matrix of
$\bm{\xi}_{n}$. By Hölder inequality, for all $n\in\mathbb{N}$ and all
$\bm{\theta}\in\bm{\Theta}$,

$[\lambda_{n1}^{\xi}(\bm{\theta})]^{2}=\|\bm{\Sigma}_{n}^{\xi}(\bm{\theta})\|^{2}\leq\|\bm{\Sigma}_{n}^{\xi}(\bm{\theta})\|^{2}_{F}\leq\|\bm{\Sigma}_{n}^{\xi}(\bm{\theta})\|_{1}\|\bm{\Sigma}_{n}^{\xi}(\bm{\theta})\|_{\infty}=\|\bm{\Sigma}_{n}^{\xi}(\bm{\theta})\|_{1}^{2}=\left[\max_{j=1,\ldots,n}\sum_{i=1}^{n}|\sigma_{ij}^{\xi}(\bm{\theta})|\right]^{2},$

where $\|\cdot\|$ denotes the spectral norm, $\|\cdot\|_{F}$ denotes the
Frobenius norm, and $\|\cdot\|_{1}$ and $\|\cdot\|_{\infty}$ denote the
column-wise and row-wise norms, respectively. It follows that,

$\sup_{\bm{\theta}\in\bm{\Theta}}\sup_{n\in\mathbb{N}}\lambda_{n1}^{\xi}(\bm{\theta})\leq\sup_{\bm{\theta}\in\bm{\Theta}}\sup_{n\in\mathbb{N}}\max_{j=1,\ldots,n}\sum_{i=1}^{n}|\sigma_{ij}^{\xi}(\bm{\theta})|\leq(A^{\xi})^{2}C_{{\rho_{1}^{\xi}}{\rho_{2}^{\xi}}{\rho_{3}^{\xi}}}.$

This completes the proof. ∎

######  Proposition E.2.

Let Assumptions 3.1, 6.1, 6.2, and 6.4 hold. Then, for all
$\ell\in\mathbb{N}$, there exist a finite $p>4$, and
$\widetilde{\rho}_{1},\widetilde{\rho}_{2},\widetilde{\rho}_{3}\in(0,1)$, and
$\widetilde{A}_{1},\widetilde{A}_{2}>0$, such that

  1. (i)

${\rm E}\left(|x_{\ell\bm{\varsigma}}|^{p}\right)\leq\widetilde{A}_{1}$;

  2. (ii)

$\delta^{[\ell]}_{\bm{\varsigma},p}({\bm{x}})\leq\widetilde{A}_{2}\widetilde{\rho}_{1}^{|s_{1}|}\widetilde{\rho}_{2}^{|s_{2}|}\widetilde{\rho}_{3}^{|t|}.$

###### Proof.

The Minkowski inequality yields

$\left\\{{\rm
E}\left(|x_{\ell\bm{\varsigma}}|^{p}\right)\right\\}^{1/p}\leq\left\\{{\rm
E}\left(|\chi_{\ell\bm{\varsigma}}|^{p}\right)\right\\}^{1/p}+\left\\{{\rm
E}\left(|\xi_{\ell\bm{\varsigma}}|^{p}\right)\right\\}^{1/p},$

so it suffices to bound the two terms on the RHS. Using again the Minkowski
inequality and Assumptions 6.1, 6.2 and 6.4, there is a constant
$\widetilde{A}_{1}>0$ such that

$\displaystyle\left\\{{\rm
E}\left(|\xi_{\ell\bm{\varsigma}}|^{p}\right)\right\\}^{1/p}$
$\displaystyle=\left\\{{\rm
E}\left(\left|\sum_{\kappa_{1},\kappa_{2}\in\mathbb{Z}}\sum_{\kappa_{3}=0}^{\infty}\sum_{j=1}^{\infty}\beta_{\ell
j,\bm{\kappa}}\varepsilon_{j,\bm{\varsigma}-\bm{\kappa}}\right|^{p}\right)\right\\}^{1/p}$
$\displaystyle\leq\sum_{\kappa_{1},\kappa_{2}\in\mathbb{Z}}\sum_{\kappa_{3}=0}^{\infty}\sum_{j=1}^{\infty}\left\\{{\rm
E}\left(\left|\beta_{\ell
j,\bm{\kappa}}\varepsilon_{j,\bm{\varsigma}-\bm{\kappa}}\right|^{p}\right)\right\\}^{1/p}$
$\displaystyle\leq\sum_{\kappa_{1},\kappa_{2}\in\mathbb{Z}}\sum_{\kappa_{3}=0}^{\infty}\sum_{j=1}^{\infty}|\beta_{\ell
j,\bm{\kappa}}|\left\\{{\rm
E}\left(\left|\varepsilon_{j,\bm{\varsigma}-\bm{\kappa}}\right|^{p}\right)\right\\}^{1/p}$
$\displaystyle\leq
A^{\xi}\widetilde{A}_{{\rho_{1}^{\xi}}{\rho_{2}^{\xi}}{\rho_{3}^{\xi}}},$

for some finite
$\widetilde{A}_{{\rho_{1}^{\xi}}{\rho_{2}^{\xi}}{\rho_{3}^{\xi}}}>0$ depending
only on ${\rho_{1}^{\xi}}$, ${\rho_{2}^{\xi}}$, and ${\rho_{3}^{\xi}}$.
Similarly, we have $\left\\{{\rm
E}\left(|\chi_{\ell\bm{\varsigma}}|^{p}\right)\right\\}^{1/p}\leq{A}^{\chi}\widetilde{A}_{{\rho_{1}^{\chi}}{\rho_{2}^{\chi}}{\rho_{3}^{\chi}}}$
for some finite
$\widetilde{A}_{{\rho_{1}^{\chi}}{\rho_{2}^{\chi}}{\rho_{3}^{\chi}}}>0$
depending only on ${\rho_{1}^{\chi}}$, ${\rho_{2}^{\chi}}$, and
${\rho_{3}^{\chi}}$. Letting
$\widetilde{A}_{1}=(A^{\xi}\widetilde{A}_{{\rho_{1}^{\xi}}{\rho_{2}^{\xi}}{\rho_{3}^{\xi}}}+{A}^{\chi}\widetilde{A}_{{\rho_{1}^{\chi}}{\rho_{2}^{\chi}}{\rho_{3}^{\chi}}})^{p}$
yields ${\rm
E}\left(|x_{\ell\bm{\varsigma}}|^{p}\right)\leq\widetilde{A}_{1}$. This proves
part (i).

Turning to part (ii), for $t<0$, the inequality holds trivially since
$\delta^{[\ell]}_{\bm{\varsigma},p}({\bm{x}})=0$. For $t\geq 0$, notice that

$\displaystyle\delta^{[\ell]}_{\bm{\varsigma},p}({\bm{x}})$
$\displaystyle=\\{{\rm
E}|x_{\ell\bm{\varsigma}}-x^{*}_{\ell\bm{\varsigma}}|^{p}\\}^{1/p}$
$\displaystyle\leq\\{{\rm
E}|\chi_{\ell\bm{\varsigma}}-\chi^{*}_{\ell\bm{\varsigma}}|^{p}\\}^{1/p}+\\{{\rm
E}|\xi_{\ell\bm{\varsigma}}-\xi^{*}_{\ell\bm{\varsigma}}|^{p}\\}^{1/p}$
$\displaystyle=\delta^{[\ell]}_{\bm{\varsigma},p}(\bm{\chi})+\delta^{[\ell]}_{\bm{\varsigma},p}(\bm{\xi}).$

Assumptions 6.1, 6.2 and 6.4 entail that

$\displaystyle\delta^{[\ell]}_{\bm{\varsigma},p}(\bm{\xi})$
$\displaystyle=\left\\{{\rm E}\left|\sum_{j=1}^{\infty}\beta_{\ell
j,\bm{\varsigma}}(\varepsilon_{j,{\bm{0}}}-\varepsilon_{j,{\bm{0}}}^{*})\right|^{p}\right\\}^{1/p}$
$\displaystyle\leq\sum_{j=1}^{\infty}|\beta_{\ell
j,\bm{\varsigma}}|\left\\{{\rm
E}\left|(\varepsilon_{j,{\bm{0}}}-\varepsilon_{j,{\bm{0}}}^{*})\right|^{p}\right\\}^{1/p}\leq\bar{A}A^{\xi}{\rho_{1}^{\xi}}^{|s_{1}|}{\rho_{2}^{\xi}}^{|s_{2}|}{\rho_{3}^{\xi}}^{t}.$

Similarly, we have that
$\delta^{[\ell]}_{\bm{\varsigma},p}(\bm{\chi})\leq\bar{A}A^{\chi}{\rho_{1}^{\chi}}^{|s_{1}|}{\rho_{2}^{\chi}}^{|s_{2}|}{\rho_{3}^{\chi}}^{t}$.
The result then follows by letting
$\widetilde{A}_{2}=\bar{A}\max(A^{\chi},A^{\xi})$ and
$\widetilde{\rho}_{h}=\max({\rho_{h}^{\xi}},{\rho_{h}^{\chi}})$, $h=1,2,3$. ∎

######  Proposition E.3.

Let Assumptions 3.1, 6.1, 6.2, and 6.4 hold. Then, letting
$\Gamma_{ij,\bm{\varsigma}}^{x}={\rm E}(x_{i\bm{\varsigma}}x_{j{\bm{0}}})$,
for all $i,j\in\mathbb{N}$, there exist finite
$\widetilde{\rho}_{4},\widetilde{\rho}_{5},\widetilde{\rho}_{6}\in(0,1)$ and
$\widetilde{A}^{*}>0$, such that
$|\Gamma_{ij,\bm{\varsigma}}^{x}|\leq\widetilde{A}^{*}\widetilde{\rho}_{4}^{|s_{1}|}\widetilde{\rho}_{5}^{|s_{2}|}\widetilde{\rho}_{6}^{|t|}$.

###### Proof.

We have
$|\Gamma_{ij,\bm{\varsigma}}^{x}|=|\Gamma_{ij,\bm{\varsigma}}^{\chi}+\Gamma_{ij,\bm{\varsigma}}^{\xi}|\leq|\Gamma_{ij,\bm{\varsigma}}^{\chi}|+|\Gamma_{ij,\bm{\varsigma}}^{\xi}|$.
Since $\\{{\bm{\varepsilon}}_{\bm{\varsigma}}=(\varepsilon_{1\bm{\varsigma}}\
\varepsilon_{2\bm{\varsigma}}\
\cdots)^{\top},\bm{\varsigma}\in\mathbb{Z}^{3}\\}$ is an i.i.d. infinite
dimensional zero-mean orthonormal rf, by Assumptions 6.1 and 6.2 we have

$\displaystyle|\Gamma_{ij,\bm{\varsigma}}^{\xi}|$ $\displaystyle=\left|{\rm
E}\left\\{\left(\sum_{\kappa_{1},\kappa_{2}\in\mathbb{Z}}\sum_{\kappa_{3}=0}^{\infty}\sum_{h=1}^{\infty}\beta_{ih,\bm{\kappa}}\varepsilon_{h,\bm{\varsigma}-\bm{\kappa}}\right)\left(\sum_{\tilde{\kappa}_{1},\tilde{\kappa}_{2}\in\mathbb{Z}}\sum_{\tilde{\kappa}_{3}=0}^{\infty}\sum_{\tilde{h}=1}^{\infty}\beta_{j\tilde{h},\tilde{\bm{\kappa}}}\varepsilon_{\tilde{h},-\tilde{\bm{\kappa}}}\right)\right\\}\right|$
$\displaystyle=\left|{\rm
E}\left\\{\left(\sum_{\kappa_{1},\kappa_{2}\in\mathbb{Z}}\sum_{\kappa_{3}=0}^{\infty}\sum_{h=1}^{\infty}\beta_{ih,\bm{\kappa}}\beta_{jh,{\bm{\kappa}}-\bm{\varsigma}}\varepsilon_{h,\bm{\varsigma}-\bm{\kappa}}^{2}\right)\right\\}\right|$
$\displaystyle\leq\sum_{\kappa_{1},\kappa_{2}\in\mathbb{Z}}\sum_{\kappa_{3}=0}^{\infty}\sum_{h=1}^{\infty}\left|\beta_{ih,\bm{\kappa}}\beta_{jh,{\bm{\kappa}}-\bm{\varsigma}}\right|{\rm
E}(\varepsilon_{h,\bm{\varsigma}-\bm{\kappa}}^{2})$
$\displaystyle\leq\sum_{\kappa_{1},\kappa_{2}\in\mathbb{Z}}\sum_{\kappa_{3}=0}^{\infty}\sum_{h=1}^{\infty}A_{ih}^{\xi}A_{jh}^{\xi}\rho_{1}^{\xi,|\kappa_{1}|+|\kappa_{1}-s_{1}|}\rho_{2}^{\xi,|\kappa_{2}|+|\kappa_{2}-s_{2}|}\rho_{3}^{\xi,\kappa_{3}+\kappa_{3}-t}$
$\displaystyle\leq\left\\{\sum_{h=1}^{\infty}A_{ih}^{\xi}A_{jh}^{\xi}\sum_{\kappa_{1},\kappa_{2}\in\mathbb{Z}}\sum_{\kappa_{3}=0}^{\infty}\rho_{1}^{\xi,2|\kappa_{1}|}\rho_{2}^{\xi,2|\kappa_{2}|}\rho_{3}^{\xi,2\kappa_{3}}\right\\}{\rho^{\xi
s_{1}|}_{1}}{\rho_{2}^{\xi|s_{2}|}}{\rho^{\xi}_{3}}^{|t|}$
$\displaystyle\leq(A^{\xi})^{2}\widetilde{A}^{*}_{\rho_{1}^{\xi},\rho_{2}^{\xi},\rho_{3}^{\xi}}{\rho^{\xi
s_{1}|}_{1}}{\rho_{2}^{\xi|s_{2}|}}{\rho^{\xi}_{3}}^{|t|}$

for some finite
$\widetilde{A}^{*}_{\rho_{1}^{\xi},\rho_{2}^{\xi},\rho_{3}^{\xi}}>0$ depending
only on ${\rho_{1}^{\xi}}$, ${\rho_{2}^{\xi}}$, and ${\rho_{3}^{\xi}}$.

Similarly, there exists a finite
$\widetilde{A}^{*}_{\rho_{1}^{\chi},\rho_{2}^{\chi},\rho_{3}^{\chi}}>0$
depending only on ${\rho_{1}^{\chi}}$, ${\rho_{2}^{\chi}}$, and
${\rho_{3}^{\chi}}$, such that
$|\Gamma_{ij,\bm{\varsigma}}^{\chi}|\leq\widetilde{A}^{*}_{\rho_{1}^{\chi},\rho_{2}^{\chi},\rho_{3}^{\chi}}{\rho^{\chi}_{1}}^{|s_{1}|}{\rho^{\chi}_{2}}^{|s_{2}|}{\rho^{\chi}_{3}}^{|t|}$.
Letting
$\widetilde{A}^{*}=\max(\widetilde{A}^{*}_{\rho_{1}^{\chi},\rho_{2}^{\chi},\rho_{3}^{\chi}},\widetilde{A}^{*}_{\rho_{1}^{\xi},\rho_{2}^{\xi},\rho_{3}^{\xi}})$
and $\widetilde{\rho}_{4}=\max({\rho_{1}^{\xi}},{\rho^{\chi}_{1}})$,
$\widetilde{\rho}_{5}=\max({\rho^{\xi}_{2}},{\rho^{\chi}_{2}})$, and
$\widetilde{\rho}_{6}=\max({\rho^{\xi}_{3}},{\rho^{\chi}_{3}})$ yields the
desired result. ∎

It immediately follows from Assumption 3.1 and Proposition E.2, that for any
$n\in\mathbb{N}$ each component $x_{\ell}$, $\ell=1,\ldots,n$ of the rf
$\bm{x}_{n}$ satisfies Conditions 1 and 2 in Appendix D. Moreover, by
Proposition E.3, we have that also Condition 3 is satisfied by all couples
$x_{i}$ and $x_{j}$, $i,j=1,\ldots,n$. Therefore, we can apply Propositions
D.1 and D.2 to the entries of the estimator of the spectral density matrix of
$\bm{x}_{n}$, defined in (17). This yields the desired result.

###  E.2 Proof of Theorem 6.2

Hereafter, let

$\alpha_{\bar{\bm{\varsigma}}}=\max\left\\{\frac{(\log B_{S_{1}}\log
B_{S_{2}}\log
B_{T})^{2}B_{S_{1}}B_{S_{2}}B_{T}}{S_{1}S_{2}T},\frac{1}{B_{S_{1}}^{2\vartheta_{1}}},\frac{1}{B_{S_{2}}^{2\vartheta_{2}}},\frac{1}{B_{T}^{2\vartheta_{3}}}\right\\}.$

Moreover, we let $C_{1},C_{2},\ldots$ and $C_{1}^{*},C_{2}^{*},\ldots$ denote
generic finite positive constants independent of $n$. We let also
$\bm{e}_{ni}$, $i=1,\ldots,n,$ denote the $i$-th canonical basis of
$\mathbb{R}^{n}$. Throughout, we make use of the spectral norm, denoted as
$\|\cdot\|$, and the Frobenius norm, denoted as $\|\cdot\|_{F}$. The proof
relies on some lemmas introduced below.

######  Lemma E.4.

Let Assumptions 3.1, 3.2, 6.1, 6.2, 6.4, and 6.5 hold. Then, for all
$n\in\mathbb{N}$,

  1. (i)

$\sup_{\bm{\theta}\in\bm{\Theta}}n^{-2}{\rm
E}\left\|\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{x}(\bm{\theta})\right\|^{2}\leq
C\alpha_{\bar{\bm{\varsigma}}}$, where $C$ is the same constant as in Theorem
6.1;

  2. (ii)

$\max_{1\leq i\leq n}\sup_{\bm{\theta}\in\bm{\Theta}}n^{-1}{\rm
E}\left\|\bm{e}_{ni}^{\top}\left(\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{x}(\bm{\theta})\right)\right\|^{2}\leq
C\alpha_{\bar{\bm{\varsigma}}}$, where $C$ is the same constant as in Theorem
6.1;

  3. (iii)

$\sup_{\bm{\theta}\in\bm{\Theta}}n^{-1}\left\|{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\right\|\leq
C_{1}^{*}n^{-1}$;

  4. (iv)

$\sup_{\bm{\theta}\in\bm{\Theta}}n^{-2}{\rm
E}\left\|\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\right\|^{2}\leq
C_{1}\max(n^{-2},\alpha_{\bar{\bm{\varsigma}}})$;

  5. (v)

$\max_{1\leq i\leq n}\sup_{\bm{\theta}\in\bm{\Theta}}n^{-1}{\rm
E}\left\|\bm{e}_{ni}^{\top}\left(\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\right)\right\|^{2}\leq
C_{2}\max(n^{-1},\alpha_{\bar{\bm{\varsigma}}})$.

###### Proof.

Part (i). For any $\bm{\theta}\in\bm{\Theta}$ and any $n\in\mathbb{N}$, we
have

$\displaystyle\left\|\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{x}(\bm{\theta})\right\|^{2}$
$\displaystyle\leq\left\|\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{x}(\bm{\theta})\right\|^{2}_{F}$
$\displaystyle={\rm
trace}\left[\left(\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{x}(\bm{\theta})\right)^{\dagger}\left(\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{x}(\bm{\theta})\right)\right]=\sum_{i=1}^{n}\sum_{j=1}^{n}\left|\widehat{\sigma}_{ij}^{x}(\bm{\theta})-\sigma_{ij}^{x}(\bm{\theta})\right|^{2}.$

The result then follows from Theorem 6.1.

Part (ii). Noticing that

$n^{-1}{\rm
E}\left\|\bm{e}_{ni}^{\top}\left(\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{x}(\bm{\theta})\right)\right\|^{2}=n^{-1}\sum_{j=1}^{n}{\rm
E}\left|\widehat{\sigma}_{ij}^{x}(\bm{\theta})-\sigma_{ij}^{x}(\bm{\theta})\right|^{2},$

the result then follows from Theorem 6.1.

Part (iii). Since
$\left\|{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\right\|=\left\|{\bm{\Sigma}}_{n}^{\xi}(\bm{\theta})\right\|$,
the result follows immediately from Proposition E.1.

Part (iv). Denote by ${\bm{\Sigma}}_{n}^{\xi}(\bm{\theta})$ the $n\times n$
spectral density matrix of ${\bm{\xi}}_{n}$. We have

$\displaystyle\left\|\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\right\|$
$\displaystyle=\left\|\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-\left({\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\xi}(\bm{\theta})\right)\right\|\leq\left\|\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{x}(\bm{\theta})\right\|+\left\|{\bm{\Sigma}}_{n}^{\xi}(\bm{\theta})\right\|.$

The result then follows immediately from part (i) and Proposition E.1.

Part (v). Note that

$\left\|\bm{e}_{ni}^{\top}\left(\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\right)\right\|\leq\left\|\bm{e}_{ni}^{\top}\left(\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{x}(\bm{\theta})\right)\right\|+\left\|\bm{e}_{ni}^{\top}{\bm{\Sigma}}_{n}^{\xi}(\bm{\theta})\right\|.$

The result then follows immediately from part (ii) and Proposition E.1. ∎

######  Lemma E.5.

Let Assumptions 3.1, 3.2, 6.1, 6.2, 6.4, and 6.5 hold. Then, for all
$n\in\mathbb{N}$ and all $j=1,\ldots,q$,

  1. (i)

$\sup_{\bm{\theta}\in\bm{\Theta}}n^{-2}{\rm
E}|\widehat{\lambda}_{nj}^{x}(\bm{\theta})-\lambda_{nj}^{x}(\bm{\theta})|^{2}\leq
C\alpha_{\bar{\bm{\varsigma}}}$, where $C$ is the same constant as in Lemma
E.4(i);

  2. (ii)

$\sup_{\bm{\theta}\in\bm{\Theta}}n^{-1}|{\lambda}_{nj}^{x}(\bm{\theta})-\lambda_{nj}^{\chi}(\bm{\theta})|\leq
C_{1}^{*}n^{-1}$, where $C_{1}^{*}$ is the same constant as in Lemma E.4(iii);

  3. (iii)

$\sup_{\bm{\theta}\in\bm{\Theta}}n^{-2}{\rm
E}|\widehat{\lambda}_{nj}^{x}(\bm{\theta})-\lambda_{nj}^{\chi}(\bm{\theta})|^{2}\leq
C_{1}\max(n^{-2},\alpha_{\bar{\bm{\varsigma}}}),$ where $C_{1}$ is the same
constant as in Lemma E.4(iv);

###### Proof.

For any two $n\times n$ matrices $\bm{A}_{1}$ and $\bm{A}_{2}$, Wely’s
inequality (see Appendix B.1) implies that

$|\nu_{\ell}(\bm{A}_{1}+\bm{A}_{2})-\nu_{\ell}(\bm{A}_{1})|\leq\|\bm{A}_{2}\|,\ell=1,\ldots,n.$
(E.39)

Part (i). Letting $\bm{A}_{1}={\bm{\Sigma}}_{n}^{x}(\bm{\theta})$,
$\bm{A}_{2}=\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{x}(\bm{\theta})$,
the result then follows from (E.39), which implies
$|\widehat{\lambda}_{nj}^{x}(\bm{\theta})-\lambda_{nj}^{x}(\bm{\theta})|\leq\|\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{x}(\bm{\theta})\|$
and Lemma E.4(i).

Part (ii). Letting $\bm{A}_{1}={\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})$,
$\bm{A}_{2}={\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})$
yields, because of (E.39),
$|{\lambda}_{nj}^{x}(\bm{\theta})-\lambda_{nj}^{\chi}(\bm{\theta})|\leq\left\|{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\right\|.$
The result hence follows from Lemma E.4(iii).

Part (iii). Letting $\bm{A}_{1}={\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})$,
$\bm{A}_{2}=\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})$,
the result then follows from (E.39), which implies
$|\widehat{\lambda}_{nj}^{x}(\bm{\theta})-\lambda_{nj}^{\chi}(\bm{\theta})|\leq\|\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\|$
and Lemma E.4(iv). ∎

######  Lemma E.6.

Let Assumptions 3.1, 3.2, 6.2, and 6.3 hold. Then, for all $n\in\mathbb{N}$,

$\max_{1\leq i\leq
n}\sup_{\bm{\theta}\in\bm{\Theta}}\left\|\bm{e}_{ni}^{\top}n^{1/2}({{\bm{P}}^{\chi}_{n}}(\bm{\theta}))^{\dagger}\right\|\leq
C_{2}.$

###### Proof.

Let ${p}_{n,jk}^{\chi}(\bm{\theta})$, $j=1,\ldots,q$, $k=1,\ldots,n,$ denote
$(j,k)$-th entry of ${\bm{P}}_{n}^{\chi}(\bm{\theta})$. Then,

$\max_{1\leq k\leq
n}\sup_{\bm{\theta}\in\bm{\Theta}}\sigma^{\chi}_{kk}(\bm{\theta})=\max_{1\leq
k\leq
n}\sup_{\bm{\theta}\in\bm{\Theta}}\sum_{j=1}^{q}\lambda_{nj}^{\chi}(\bm{\theta})|{p}_{n,jk}^{\chi}(\bm{\theta})|^{2}<\infty.$

Indeed, by Assumptions 6.1 and 6.2

$\sigma^{\chi}_{kk}(\bm{\theta})=\sum_{\bm{h}}e^{-i\langle\bm{h},\bm{\theta}\rangle}\text{E}[\chi_{k\bm{\varsigma}}\chi_{k\bm{\varsigma}-\bm{h}}]\leq\sum_{\bm{h}}|\text{E}[\chi_{k\bm{\varsigma}}\chi_{k\bm{\varsigma}-\bm{h}}]|\leq\sum_{h_{1}}\sum_{h_{2}}\sum_{h_{3}}\widetilde{A}^{*}_{\rho_{1}^{\chi},\rho_{2}^{\chi},\rho_{3}^{\chi}}{\rho^{\chi}_{1}}^{|h_{1}|}{\rho^{\chi}_{2}}^{|h_{2}|}{\rho^{\chi}_{3}}^{|h_{3}|}<\infty,$

where $\widetilde{A}^{*}_{\rho_{1}^{\chi},\rho_{2}^{\chi},\rho_{3}^{\chi}}$,
$\rho^{\chi}_{1},\rho^{\chi}_{2}$, and $\rho^{\chi}_{3}$ are defined in the
proof of Proposition E.3. Moreover, $\sigma^{\chi}_{kk}(\bm{\theta})\geq 0$
(Brockwell and Davis, 2006, Corollary 4.3.2, p.120).

Now, since, by Assumption 6.3, for all $j=1,\ldots,q$,

$\sup_{\bm{\theta}\in\bm{\Theta}}\frac{\lambda_{nj}^{\chi}(\bm{\theta})}{n}>\sup_{\bm{\theta}\in\bm{\Theta}}\undertilde{\omega}(\bm{\theta})_{j}>0,$

we must have
$\sup_{\bm{\theta}\in\bm{\Theta}}n|{p}_{n,jk}^{\chi}(\bm{\theta})|^{2}<C_{2}$,
where $C_{2}$ is independent of $j$ and $k$. This completes the proof. ∎

######  Lemma E.7.

Let Assumptions 3.1, 3.2, 6.2, and 6.3 hold. Then, for all $n\in\mathbb{N}$,

For all $j=1,\ldots,q-1$ there exist continuous functions
$\bm{\theta}\mapsto\widetilde{\omega}_{j}^{*}(\bm{\theta})$ and
$\bm{\theta}\mapsto\undertilde{\omega}_{j}^{*}(\bm{\theta})$ such that for all
$\bm{\theta}\in\bm{\Theta}$

$0<\undertilde{\omega}_{j+1}^{*}(\bm{\theta})\leq\lim_{n\to\infty}\frac{\lambda_{n,j+1}^{x}(\bm{\theta})}{n}\leq\widetilde{\omega}_{j+1}^{*}(\bm{\theta})<\undertilde{\omega}_{j}^{*}(\bm{\theta})\leq\lim_{n\to\infty}\frac{\lambda_{nj}^{x}(\bm{\theta})}{n}\leq\widetilde{\omega}_{j}^{*}(\bm{\theta})<\infty.$

###### Proof.

The result follows immediately from Lemma E.5(ii) and Assumption 6.3. ∎

######  Lemma E.8.

Let Assumptions 3.1, 3.2, 6.1, 6.2, 6.3, 6.4, 6.5, and 6.6 hold. Then, for all
$n\in\mathbb{N}$,

$\max_{1\leq\ell\leq n}\sup_{\bm{\theta}\in\bm{\Theta}}{\rm
E}\left\|\bm{e}_{n\ell}^{\top}n^{1/2}\left[{{\bm{K}}}^{\chi}_{n}(\bm{\theta})-\widehat{{\bm{K}}}^{x}_{n}(\bm{\theta})\right]\right\|^{2}\leq
C_{3}^{*}\max(n^{-1},\alpha_{\bar{\bm{\varsigma}}}).$

###### Proof.

Theorem 2 in Yu, Wang and Samworth (2015) implies that there exists a $q\times
q$ orthogonal matrix ${\bm{O}}_{q}$ such that

$\displaystyle\left\|{\bm{P}}_{n}^{\chi}(\bm{\theta})-{\bm{O}}_{q}\widehat{\bm{P}}_{n}^{x}(\bm{\theta})\right\|$
$\displaystyle\leq\left\|{\bm{P}}_{n}^{\chi}(\bm{\theta})-{\bm{O}}_{q}\widehat{\bm{P}}_{n}^{x}(\bm{\theta})\right\|_{\rm
F}$ $\displaystyle\leq
C_{4}^{*}\frac{\left\|\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\right\|}{{\lambda}_{nq}^{\chi}(\bm{\theta})-{\lambda}_{n,q+1}^{\chi}(\bm{\theta})}\leq
C_{5}^{*}n^{-1}\left\|\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\right\|,$
(E.40)

where we used the fact that ${\lambda}_{n,q+1}^{\chi}(\bm{\theta})=0$ and the
last inequality is due to Lemma E.7. Note that

$\displaystyle\bm{e}_{n\ell}^{\top}n^{1/2}\left[{{\bm{K}}}^{\chi}_{n}(\bm{\theta})-\widehat{{\bm{K}}}^{x}_{n}(\bm{\theta})\right]$
$\displaystyle=$
$\displaystyle\,\bm{e}_{n\ell}^{\top}n^{1/2}{\bm{P}}_{n}^{\chi{\dagger}}(\bm{\theta})\left[{\bm{P}}_{n}^{\chi}(\bm{\theta})-{\bm{O}}_{q}\widehat{\bm{P}}_{n}^{x}(\bm{\theta})\right]+\bm{e}_{n\ell}^{\top}n^{1/2}\left[{\bm{P}}_{n}^{\chi{\dagger}}(\bm{\theta}){\bm{O}}_{q}-\widehat{\bm{P}}_{n}^{x{\dagger}}(\bm{\theta})\right]\widehat{\bm{P}}_{n}^{x}(\bm{\theta})$
$\displaystyle=$
$\displaystyle\,{\bm{d}}_{n\ell}^{(1)}(\bm{\theta})+{\bm{d}}_{n\ell}^{(2)}(\bm{\theta}),\qquad\text{say}.$

For ${\bm{d}}_{n\ell}^{(1)}(\bm{\theta})$, in view of (E.2) and Lemmas E.4(iv)
and E.6, we have

$\max_{1\leq\ell\leq n}\sup_{\bm{\theta}\in\bm{\Theta}}{\rm
E}\left\|{\bm{d}}_{n\ell}^{(1)}(\bm{\theta})\right\|^{2}\leq
C_{6}^{*}\max(n^{-2},\alpha_{\bar{\bm{\varsigma}}}).$

As for ${\bm{d}}_{n\ell}^{(2)}(\bm{\theta})$, note that

$\displaystyle\sum_{\ell=1}^{n}\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\left[{\bm{P}}_{n}^{\chi}(\bm{\theta})-{\bm{O}}_{q}\widehat{\bm{P}}_{n}^{x}(\bm{\theta})\right]\right\|^{2}$
$\displaystyle=\left\|n^{1/2}\left[{\bm{P}}_{n}^{\chi}(\bm{\theta})-{\bm{O}}_{q}\widehat{\bm{P}}_{n}^{x}(\bm{\theta})\right]\right\|_{\rm
F}^{2}$
$\displaystyle\leq(C_{5}^{*})^{2}n^{-2}\left\|n^{1/2}\left[\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\right]\right\|_{\rm
F}^{2}$
$\displaystyle=(C_{5}^{*})^{2}n^{-1}\sum_{\ell=1}^{n}\left\|\bm{e}_{n\ell}^{\top}\left[\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\right]\right\|^{2}.$

Hence, for all $\ell=1,\ldots,n$, ${\rm
E}\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\left[{\bm{P}}_{n}^{\chi}(\bm{\theta})-{\bm{O}}_{q}\widehat{\bm{P}}_{n}^{x}(\bm{\theta})\right]\right\|^{2}$
is of order no greater than

$\max_{1\leq i\leq n}n^{-1}{\rm
E}\left\|\bm{e}_{ni}^{\top}\left[\widehat{\bm{\Sigma}}_{n}^{x}(\bm{\theta})-{\bm{\Sigma}}_{n}^{\chi}(\bm{\theta})\right]\right\|^{2},$

which, in view of Lemma E.4(v), entails

$\max_{1\leq\ell\leq n}\sup_{\bm{\theta}\in\bm{\Theta}}{\rm
E}\left\|{\bm{d}}_{n\ell}^{(2)}(\bm{\theta})\right\|^{2}\leq
C_{7}^{*}\max(n^{-1},\alpha_{\bar{\bm{\varsigma}}}).$

The result then follows. ∎

We can then prove the theorem as follows. Let
$\bm{\chi}_{n\bm{\varsigma}}=(\chi_{1\bm{\varsigma}},\ldots,\chi_{n\bm{\varsigma}})^{\top}$
and
$\bm{\xi}_{n\bm{\varsigma}}=(\xi_{1\bm{\varsigma}},\ldots,\xi_{n\bm{\varsigma}})^{\top}$.
Recall from (16) in Remark 5.1 that

$\displaystyle{\underline{{\bm{K}}}}^{\chi}_{n}(L){\bm{x}}_{n\bm{\varsigma}}={\underline{{\bm{K}}}}^{\chi}_{n}(L)\bm{\chi}_{n\bm{\varsigma}}+{\underline{{\bm{K}}}}^{\chi}_{n}(L)\bm{\xi}_{n\bm{\varsigma}}={\underline{{\bm{K}}}}^{\chi}_{n}(L)\bm{\chi}_{n\bm{\varsigma}}=\bm{\chi}_{n\bm{\varsigma}}.$

Letting

$\mathcal{D}_{{\bm{\varsigma}}}=\\{(s_{1},s_{2},t):\underline{\kappa}_{1}(s_{1})\leq
s_{1}\leq\overline{\kappa}_{1}(s_{1}),\underline{\kappa}_{2}(s_{2})\leq
s_{2}\leq\overline{\kappa}_{2}(s_{2}),\underline{\kappa}_{3}(t)\leq
t\leq\overline{\kappa}_{3}(t)\\},$

we then have

$\displaystyle\widehat{\chi}^{(n)}_{\ell\bm{\varsigma}}-\chi^{n}_{\ell\bm{\varsigma}}$
$\displaystyle=\sum_{\bm{\varsigma}^{\prime}\in\mathcal{D}_{{\bm{\varsigma}}}}\bm{e}_{n\ell}^{\top}\left(\widehat{\underline{{\bm{K}}}}^{x}_{n\bm{\varsigma}^{\prime}}-\underline{{\bm{K}}}^{\chi}_{n\bm{\varsigma}^{\prime}}\right){\bm{x}}_{n\bm{\varsigma}-\bm{\varsigma}^{\prime}}+\sum_{\bm{\varsigma}^{\prime}\in\mathbb{Z}^{3}\setminus\mathcal{D}_{{\bm{\varsigma}}}}\bm{e}_{n\ell}^{\top}\underline{{\bm{K}}}^{\chi}_{n\bm{\varsigma}^{\prime}}{\bm{x}}_{n\bm{\varsigma}-\bm{\varsigma}^{\prime}}={a}_{n\ell\bm{\varsigma}}+\widetilde{a}_{n\ell\bm{\varsigma}},\;\text{say}.$

We first derive convergence rate of ${a}_{n\ell\bm{\varsigma}}$. Letting
$\bm{C}_{n\bm{\kappa}}$ be the $n\times q$ matrix with entries $c_{\ell
j,\bm{\kappa}}$, $\ell=1,\ldots,n$, $j=1,\ldots,q$,
$\bm{\kappa}\in\mathbb{Z}^{2}\times\mathbb{N}_{0}$, from Assumption 6.1, we
have

$\bm{\chi}_{n\bm{\varsigma}-\bm{\varsigma}^{\prime}}=\sum_{\bm{\kappa}}\bm{C}_{n\bm{\kappa}}\bm{v}_{\bm{\varsigma}-\bm{\varsigma}^{\prime}-\bm{\kappa}}.$

Therefore,

$\displaystyle|{a}_{n\ell\bm{\varsigma}}|$
$\displaystyle\leq\left|\sum_{\bm{\varsigma}^{\prime}\in\mathcal{D}_{{\bm{\varsigma}}}}\bm{e}_{n\ell}^{\top}\left(\widehat{\underline{{\bm{K}}}}^{x}_{n\bm{\varsigma}^{\prime}}-\underline{{\bm{K}}}^{\chi}_{n\bm{\varsigma}^{\prime}}\right)\left(\sum_{\bm{\kappa}}\bm{C}_{n\bm{\kappa}}\bm{v}_{\bm{\varsigma}-\bm{\varsigma}^{\prime}-\bm{\kappa}}\right)\right|+\left|\sum_{\bm{\varsigma}^{\prime}\in\mathcal{D}_{{\bm{\varsigma}}}}\bm{e}_{n\ell}^{\top}\left(\widehat{\underline{{\bm{K}}}}^{x}_{n\bm{\varsigma}^{\prime}}-\underline{{\bm{K}}}^{\chi}_{n\bm{\varsigma}^{\prime}}\right)\bm{\xi}_{n\bm{\varsigma}-\bm{\varsigma}^{\prime}}\right|=a_{n\ell\bm{\varsigma}}^{(1)}+a_{n\ell\bm{\varsigma}}^{(2)},\;\text{say}.$

Consider $a_{n\ell\bm{\varsigma}}^{(1)}$. By Cauchy-Schwarz inequality

$\displaystyle{\rm E}(a_{n\ell\bm{\varsigma}}^{(1)})$
$\displaystyle\leq\sum_{\bm{\varsigma}^{\prime}\in\mathcal{D}_{{\bm{\varsigma}}}}{\rm
E}\left|\bm{e}_{n\ell}^{\top}\left(\widehat{\underline{{\bm{K}}}}^{x}_{n\bm{\varsigma}^{\prime}}-\underline{{\bm{K}}}^{\chi}_{n\bm{\varsigma}^{\prime}}\right)\left(\sum_{\bm{\kappa}}\bm{C}_{n\bm{\kappa}}\bm{v}_{\bm{\varsigma}-\bm{\varsigma}^{\prime}-\bm{\kappa}}\right)\right|$
$\displaystyle\leq\sum_{\bm{\varsigma}^{\prime}\in\mathcal{D}_{{\bm{\varsigma}}}}\left\\{{\rm
E}\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\left(\widehat{\underline{{\bm{K}}}}^{x}_{n\bm{\varsigma}^{\prime}}-\underline{{\bm{K}}}^{\chi}_{n\bm{\varsigma}^{\prime}}\right)\right\|^{2}\right\\}^{1/2}\left\\{n^{-1}{\rm
E}\left\|\sum_{\bm{\kappa}}\bm{C}_{n\bm{\kappa}}\bm{v}_{\bm{\varsigma}-\bm{\varsigma}^{\prime}-\bm{\kappa}}\right\|^{2}\right\\}^{1/2}.$
(E.41)

Now, because of Assumption 6.1, which implies that $\bm{v}$ is a white noise
rf, and Assumption 6.2

$\displaystyle\frac{1}{n}{\rm
E}\left\|\sum_{\bm{\kappa}}\bm{C}_{n\bm{\kappa}}\bm{v}_{\bm{\varsigma}-\bm{\varsigma}^{\prime}-\bm{\kappa}}\right\|^{2}$
$\displaystyle\leq\frac{1}{n}{\rm
E}\left\|\sum_{\bm{\kappa}}\bm{C}_{n\bm{\kappa}}\bm{v}_{\bm{\varsigma}-\bm{\varsigma}^{\prime}-\bm{\kappa}}\right\|^{2}_{F}$
$\displaystyle=\frac{1}{n}\sum_{\bm{\kappa}}\sum_{\bm{\kappa}^{\prime}}{\rm
trace}\left(\bm{C}_{n\bm{\kappa}}{\rm
E}(\bm{v}_{\bm{\varsigma}-\bm{\varsigma}^{\prime}-\bm{\kappa}}\bm{v}_{\bm{\varsigma}-\bm{\varsigma}^{\prime}-\bm{\kappa}^{\prime}}^{\dagger})\bm{C}_{n\bm{\kappa}^{\prime}}^{\dagger}\right)$
$\displaystyle=\frac{1}{n}\sum_{\bm{\kappa}}{\rm
trace}\left(\bm{C}_{n\bm{\kappa}}\bm{C}_{n\bm{\kappa}}^{\dagger}\right)=\frac{1}{n}\sum_{\bm{\kappa}}\sum_{j=1}^{q}\sum_{\ell=1}^{n}|c_{\ell
j,\bm{\kappa}}|^{2}$
$\displaystyle\leq\frac{1}{n}\sum_{\bm{\kappa}}\sum_{j=1}^{q}\sum_{\ell=1}^{n}\left(A^{\chi}_{\ell
j}{\rho_{1}^{\chi|\kappa_{1}|}}{\rho_{2}^{\chi|\kappa_{2}|}}{\rho_{3}^{\chi\kappa_{3}}}\right)^{2}\leq
q\max_{1\leq\ell\leq n}\max_{1\leq j\leq q}(A^{\chi}_{\ell
j})^{2}\widetilde{A}_{{\rho_{1}^{\chi}}{\rho_{2}^{\chi}}{\rho_{3}^{\chi}}}$
$\displaystyle\leq
q(A^{\chi})^{2}\widetilde{A}_{{\rho_{1}^{\chi}}{\rho_{2}^{\chi}}{\rho_{3}^{\chi}}}=C_{8}^{*},\;\text{say},$
(E.42)

for some finite
$\widetilde{A}_{{\rho_{1}^{\chi}}{\rho_{2}^{\chi}}{\rho_{3}^{\chi}}}>0$
depending only on ${\rho_{1}^{\chi}}$, ${\rho_{2}^{\chi}}$, and
${\rho_{3}^{\chi}}$. Furthermore, noting that

$\displaystyle\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\left(\widehat{\underline{{\bm{K}}}}^{x}_{n\bm{\varsigma}^{\prime}}-\underline{{\bm{K}}}^{\chi}_{n\bm{\varsigma}^{\prime}}\right)\right\|$
$\displaystyle=\left\|\frac{1}{8\pi^{3}}\int_{\bm{\Theta}}n^{1/2}\bm{e}_{n\ell}^{\top}\left[\widehat{{\bm{K}}}^{x}_{n}(\bm{\theta})-{{\bm{K}}}^{\chi}_{n}(\bm{\theta})\right]e^{i\langle\bm{\varsigma}^{\prime},\bm{\theta}\rangle}{\rm
d}\bm{\theta}\right\|$
$\displaystyle\leq\frac{1}{8\pi^{3}}\int_{\bm{\Theta}}\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\left[\widehat{{\bm{K}}}^{x}_{n}(\bm{\theta})-{{\bm{K}}}^{\chi}_{n}(\bm{\theta})\right]\right\|{\rm
d}\bm{\theta},$

we have

$\displaystyle\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\left(\widehat{\underline{{\bm{K}}}}^{x}_{n\bm{\varsigma}^{\prime}}-\underline{{\bm{K}}}^{\chi}_{n\bm{\varsigma}^{\prime}}\right)\right\|^{2}$
$\displaystyle\leq\left(\frac{1}{8\pi^{3}}\right)^{2}\left\\{\int_{\bm{\Theta}}\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\left[\widehat{{\bm{K}}}^{x}_{n}(\bm{\theta})-{{\bm{K}}}^{\chi}_{n}(\bm{\theta})\right]\right\|{\rm
d}\bm{\theta}\right\\}^{2}$
$\displaystyle\leq\frac{1}{8\pi^{3}}\int_{\bm{\Theta}}\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\left[\widehat{{\bm{K}}}^{x}_{n}(\bm{\theta})-{{\bm{K}}}^{\chi}_{n}(\bm{\theta})\right]\right\|^{2}{\rm
d}\bm{\theta},$

which, because of Lemma E.8, entails that

$\displaystyle{\rm
E}\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\left(\widehat{\underline{{\bm{K}}}}^{x}_{n\bm{\varsigma}^{\prime}}-\underline{{\bm{K}}}^{\chi}_{n\bm{\varsigma}^{\prime}}\right)\right\|^{2}$
$\displaystyle\leq\frac{1}{8\pi^{3}}\int_{\bm{\Theta}}{\rm
E}\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\left[\widehat{{\bm{K}}}^{x}_{n}(\bm{\theta})-{{\bm{K}}}^{\chi}_{n}(\bm{\theta})\right]\right\|^{2}{\rm
d}\bm{\theta}$ $\displaystyle\leq\sup_{\bm{\theta}\in\bm{\Theta}}{\rm
E}\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\left[\widehat{{\bm{K}}}^{x}_{n}(\bm{\theta})-{{\bm{K}}}^{\chi}_{n}(\bm{\theta})\right]\right\|^{2}$
$\displaystyle\leq C_{3}^{*}\max(n^{-1},\alpha_{\bar{\bm{\varsigma}}}).$
(E.43)

By using (E.42) and (E.43), into (E.41), and recalling (18) and (19), we have

$\displaystyle\max_{1\leq\ell\leq n}{\rm E}(a_{n\ell\bm{\varsigma}}^{(1)})$
$\displaystyle\leq\sqrt{C_{3}^{*}C_{8}^{*}}\max(n^{-1/2},\alpha_{\bar{\bm{\varsigma}}}^{1/2})(\overline{\kappa}_{1}(s_{1})-\underline{\kappa}_{1}(s_{1}))(\overline{\kappa}_{2}(s_{2})-\underline{\kappa}_{2}(s_{2}))(\overline{\kappa}_{3}(t)-\underline{\kappa}_{3}(t))$
$\displaystyle\leq\sqrt{C_{3}^{*}C_{8}^{*}}\max(n^{-1/2},\alpha_{\bar{\bm{\varsigma}}}^{1/2})M_{S_{1}}M_{S_{2}}M_{T}.$

Applying the same arguments will yield the same convergence rate of
$a_{n\ell\bm{\varsigma}}^{(2)}$. We hence have

$\max_{1\leq\ell\leq n}{\rm E}(a_{n\ell\bm{\varsigma}})\leq
C\max(n^{-1/2},\alpha_{\bar{\bm{\varsigma}}}^{1/2})M_{S_{1}}M_{S_{2}}M_{T},$

for some finite $C>0$ independent of $n,S_{1},S_{2}$ and $T$.

Turning to $\widetilde{a}_{n\ell\bm{\varsigma}}$, we have

$\displaystyle|\widetilde{a}_{n\ell\bm{\varsigma}}|$
$\displaystyle\leq\left|\sum_{\bm{\varsigma}^{\prime}\in\mathbb{Z}^{3}\setminus\mathcal{D}_{{\bm{\varsigma}}}}\bm{e}_{n\ell}^{\top}\underline{{\bm{K}}}^{\chi}_{n\bm{\varsigma}^{\prime}}\left(\sum_{\bm{\kappa}}\bm{C}_{n\bm{\kappa}}\bm{v}_{\bm{\varsigma}-\bm{\varsigma}^{\prime}-\bm{\kappa}}\right)\right|+\left|\sum_{\bm{\varsigma}^{\prime}\in\mathbb{Z}^{3}\setminus\mathcal{D}_{{\bm{\varsigma}}}}\bm{e}_{n\ell}^{\top}\underline{{\bm{K}}}^{\chi}_{n\bm{\varsigma}^{\prime}}\bm{\xi}_{n\bm{\varsigma}-\bm{\varsigma}^{\prime}}\right|=\widetilde{a}_{n\ell\bm{\varsigma}}^{(1)}+\widetilde{a}_{n\ell\bm{\varsigma}}^{(2)},\;\text{say}.$

Consider $\widetilde{a}_{n\ell\bm{\varsigma}}^{(1)}$. By repeating the same
arguments for bounding ${\rm E}(a_{n\ell\bm{\varsigma}}^{(1)})$, and using
Assumption 6.6 and Lemma E.6, we have

$\displaystyle{\rm E}(\widetilde{a}_{n\ell\bm{\varsigma}}^{(1)})$
$\displaystyle\leq\sum_{\bm{\varsigma}^{\prime}\in\mathbb{Z}^{3}\setminus\mathcal{D}_{{\bm{\varsigma}}}}\left\\{{\rm
E}\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\underline{{\bm{K}}}^{\chi}_{n\bm{\varsigma}^{\prime}}\right\|^{2}\right\\}^{1/2}\left\\{n^{-1}{\rm
E}\left\|\sum_{\bm{\kappa}}\bm{C}_{n\bm{\kappa}}\bm{v}_{\bm{\varsigma}-\bm{\varsigma}^{\prime}-\bm{\kappa}}\right\|^{2}\right\\}^{1/2}$
$\displaystyle\leq\sqrt{C_{8}^{*}}\left\\{{\rm
E}\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\underline{{\bm{K}}}^{\chi}_{n(0\ 0\
0)}\right\|^{2}\right\\}^{1/2}\sum_{\bm{\varsigma}^{\prime}\in\mathbb{Z}^{3}\setminus\mathcal{D}_{{\bm{\varsigma}}}}(1+\varepsilon_{1})^{-|s_{1}^{\prime}|}(1+\varepsilon_{2})^{-|s_{2}^{\prime}|}(1+\varepsilon_{3})^{-|t^{\prime}|}$
$\displaystyle\leq{C}_{9}^{*}\left\\{{\rm
E}\left\|n^{1/2}\bm{e}_{n\ell}^{\top}\underline{{\bm{K}}}^{\chi}_{n(0\ 0\
0)}\right\|^{2}\right\\}^{1/2}(1+\varepsilon_{1})^{-\kappa^{*}_{1}(s_{1})}(1+\varepsilon_{2})^{-\kappa^{*}_{2}(s_{2})}(1+\varepsilon_{3})^{-\kappa^{*}_{3}(t)}$
$\displaystyle\leq
C^{*}(1+\varepsilon_{1})^{-\kappa^{*}_{1}(s_{1})}(1+\varepsilon_{2})^{-\kappa^{*}_{2}(s_{2})}(1+\varepsilon_{3})^{-\kappa^{*}_{3}(t)},$

for some finite $C^{*}>0$ independent of $\ell,n,S_{1},S_{2}$, and $T$.
Applying the same arguments will yield the same convergence rate of
$\widetilde{a}_{n\ell\bm{\varsigma}}^{(2)}$.

All of the above arguments yield