# Nonparametric regression for locally stationary random fields under
stochastic sampling design

Daisuke Kurisu  Department of Industrial Engineering and Economics, School of
Engineering, Tokyo Institute of Technology
2-12-1 Ookayama, Meguro-ku, Tokyo 152-8552, Japan<EMAIL_ADDRESS>
(Date: First version: April 16, 2020. This version: )

###### Abstract.

In this study, we develop an asymptotic theory of nonparametric regression for
locally stationary random fields (LSRFs) $\\{\bm{X}_{\bm{s},A_{n}}:\bm{s}\in
R_{n}\\}$ in $\mathbb{R}^{p}$ observed at irregularly spaced locations in
$R_{n}=[0,A_{n}]^{d}\subset\mathbb{R}^{d}$. We first derive the uniform
convergence rate of general kernel estimators, followed by the asymptotic
normality of an estimator for the mean function of the model. Moreover, we
consider additive models to avoid the curse of dimensionality arising from the
dependence of the convergence rate of estimators on the number of covariates.
Subsequently, we derive the uniform convergence rate and joint asymptotic
normality of the estimators for additive functions. We also introduce
approximately $m_{n}$-dependent RFs to provide examples of LSRFs. We find that
these RFs include a wide class of Lévy-driven moving average RFs.

Keywords: nonparametric regression, locally stationary random field,
irregularly spaced data, additive model, Lévy-driven moving average random
field.

D. Kurisu is partially supported by JSPS KAKENHI Grant Numbers 17H02513 and
20K13468. The author would like to thank Paul Doukhan, Kengo Kato, Yasumasa
Matsuda and Taisuke Otsu for their helpful comments and suggestions.

##  1\. Introduction

In this study, we consider the following model:

$\displaystyle Y_{\bm{s}_{j},A_{n}}$ $\displaystyle=m\left({\bm{s}_{j}\over
A_{n}},\bm{X}_{\bm{s}_{j},A_{n}}\right)+\epsilon_{\bm{s}_{j},A_{n}},\
\bm{s}_{j}\in R_{n},\ j=1,\ldots,n,$ (1.1)

where $E[\epsilon_{\bm{s},A_{n}}|\bm{X}_{\bm{s},A_{n}}]=0$ and
$R_{n}=[0,A_{n}]^{d}\subset\mathbb{R}^{d}$ is a sampling region with
$A_{n}\to\infty$ as $n\to\infty$. Here, $Y_{\bm{s}_{j},A_{n}}$ and
$\bm{X}_{\bm{s}_{j},A_{n}}$ are random variables of dimensions $1$ and $p$,
respectively. We assume that $\\{\bm{X}_{\bm{s},A_{n}}:\bm{s}\in R_{n}\\}$ is
a locally stationary random field on $R_{n}\subset\mathbb{R}^{d}$ ($d\geq 2$).
Locally stationary processes, as proposed by Dahlhaus (1997), are
nonstationary time series that allow parameters of the time series to be time-
dependent. They can be approximated by a stationary time series locally in
time, which enables asymptotic theories to be established for the estimation
of time-dependent characteristics. In time series analysis, locally stationary
models are mainly considered in a parametric framework with time-varying
coefficients. For example, we refer to Dahlhaus et al. (1999), Dahlhaus and
Subba Rao (2006), Fryzlewicz et al. (2008), Hafner and Linton (2010), Koo and
Linton (2012), Zhou (2014) and Truquet (2017). Nonparametric methods for
stationary and nonstationary time series models have also been developed. We
refer to Masry (1996), Fan and Yao (2003) and Hansen (2008) for stationary
time series and Kristensen (2009), Zhou and Wu (2009), Kristensen (2011), Vogt
(2012), Zhang and Wu (2015), Truquet (2019), Kurisu (2021a), and Kurisu
(2021b) for recent contributions on locally stationary (functional) time
series. Furthermore, we refer to Dahlhaus et al. (2019) for general theory in
the literature of locally stationary processes.

Recently, statistical inference of spatial regression models for
geostatistical data has drawn considerable attention in certain economic and
scientific fields, such as spatial econometrics, ecology, and seismology.
Nonparametric methods for spatial data have also been the focus of attention.
Recent contributions include those by Hallin et al. (2004), Machkouri and
Stoica (2008), Hallin et al. (2009), Robinson (2011), Jenish (2012), Lu and
Tjøstheim (2014), Li (2016), Machkouri et al. (2017), and Kurisu (2019), who
studied nonparametric inference and estimation of mean functions of spatial
regression models. Compared with the abovementioned studies regarding time
series analysis, studies pertaining to nonparametric methods for locally
stationary random fields are scarce even for nonparametric density estimation
as well as nonparametric regression despite empirical interest in modeling
spatially varying dependence structures. Moreover, for the analysis of spatial
data on $\mathbb{R}^{d}$, it is generally assumed that the underlying model is
a parametric Gaussian process. We refer to Fuglstad et al. (2015) for the
empirical motivation and discussion regarding the modeling of nonstationary
random fields and Steel and Fuentes (2010) for discussion regarding the
importance of nonparametric, nonstationary, and non-Gaussian spatial models.
To bridge the gap between theory and practice, we first extend the concept of
a locally stationary time series by Dahlhaus (1997) to random fields and
consider a locally stationary random field $\\{\bm{X}_{\bm{s},A_{n}}:\bm{s}\in
R_{n}\\}$ ($R_{n}\subset\mathbb{R}^{d}$) as a nonstationary random field that
can be approximated by a stationary random field locally at each spatial point
$\bm{s}\in R_{n}$. A detailed definition of locally stationary random fields
is provided in Section 2. Next, we provide a complete asymptotic theory for
the nonparametric estimators of the model (1.1). The model (1.1) can be
regarded as a natural extension of (i) nonparametric regression to the locally
stationary time series considered in Vogt (2012) and Zhang and Wu (2015), and
(ii) nonparametric spatial regression investigated in Hallin et al. (2004),
Robinson (2011), Jenish (2012), Li (2016), and Kurisu (2019) to locally
stationary random fields, and (iii) linear regression models with spatially
varying coefficients to a nonlinear framework. Therefore, the model includes a
wide range of key models for nonstationary random fields.

The objectives of this study are to (i) derive the uniform convergence rate of
kernel estimators for the density function of $\bm{X}_{\bm{s},A_{n}}$ and the
mean function $m$ in the model (1.1) over compact sets; (ii) derive the
asymptotic normality of the estimators at a specified point; and (iii) provide
examples of locally stationary random fields on $\mathbb{R}^{d}$ with a
detailed discussion of their properties. To attain the first and second
objectives, we first derive the uniform convergence rate of the important
general kernel estimators; the result is crucial for demonstrating our main
results. As general estimators include a wide range of kernel-based estimators
such as the Nadaraya-Watson estimators, the general results are of independent
interest. Although these results are general, the estimators are affected by
dimensionality because their convergence rate depends on the number of
covariates. Hence, we consider additive models and derive the uniform
convergence rate and joint asymptotic normality of kernel estimators for
additive functions based on the backfitting method developed by Mammen et al.
(1999) and Vogt (2012). Our results are extensions of the results for time
series in Vogt (2012) to random fields with irregularly spaced observations,
which include irregularly spaced time series as a special case. To the best of
our knowledge, this is the first study where a complete asymptotic theory of
density estimation, nonparametric regression and additive models for locally
stationary random fields on $\mathbb{R}^{d}$ are developed with a detailed
discussion on concrete examples of locally stationary random fields.

Technically, the proofs of our results are significantly different from those
for equally distant time series (mixing sequence) or spatial data observed on
lattice points, as our focus is on spatial dependence and irregularly spaced
sampling points. In many scientific fields, such as ecology, meteorology,
seismology, and spatial econometrics, sampling points are naturally irregular.
In fact, measurement stations cannot be placed on a regular grid owing to
physical constraints. The stochastic sampling design assumed in this study
allows the sampling sites to have a possibly non-uniform density across the
sampling region, thereby enabling the number of sampling sites to increase at
different rates with respect to the volume of the region $O(A_{n}^{d})$ . In
this study, we work with both the pure increasing domain case
($\lim_{n\to\infty}nA_{n}^{-d}=\kappa\in(0,\infty)$) and the mixed increasing
domain case ($\lim_{n\to\infty}nA_{n}^{-d}=\infty$). As recent contributions
pertaining to the design of stochastic spatial sampling, we refer to Shao
(2010) for time series and Lahiri (2003b), Lahiri and Zhu (2006), and
Bandyopadhyay et al. (2015) for random fields. In Lu and Tjøstheim (2014),
they show asymptotic normality of a kernel density estimator of a strictly
stationary random field on $\mathbb{R}^{2}$ under non-stochastic irregularly
spaced observations based on the technique developed in Bolthausen (1982). Our
approach is quite different from theirs and their sampling framework seems not
compatible to develop asymptotic theory for locally stationary random fields.
More precisely, in the proofs of our main results, we combine the big and
small block techniques of Bernstein (1926) and the coupling technique for
mixing sequences of Yu (1994) to construct a sequence of independent blocks in
a finite sample size. However, their applications are nontrivial for
addressing irregularly spaced data and spatial dependence simultaneously.
Discussion on the mixing condition is provided in Section 2.

To attain the third objective of our study, we discuss examples of locally
stationary random fields on $\mathbb{R}^{d}$ that satisfy our mixing
conditions and the other regularity conditions mentioned in Section 5. For
this, we introduce the concept of approximately $m_{n}$-dependent locally
stationary random fields ($m_{n}\to\infty$ as $n\to\infty$) and we extend
continuous autoregressive and moving average (CARMA)-type random fields
developed in Brockwell and Matsuda (2017) and Matsuda and Yuan (2020) to
locally stationary CARMA-type random fields. CARMA random fields are
characterized by solutions of (fractional) stochastic partial differential
equations (cf. Berger (2020)) and are known as a rich class of models for
spatial data (cf. Brockwell and Matsuda (2017), Matsuda and Yajima (2018) and
Matsuda and Yuan (2020)). However, their mixing properties have not been
investigated. We can show that a wide class of Lévy-driven moving average
random fields, which includes the locally stationary CARMA-type random fields,
comprise (approximately $m_{n}$-dependent) locally stationary random fields.
One of the key features of CARMA random fields is that they can represent non-
Gaussian random fields as well as Gaussian random fields if the driving Lévy
random measures are purely non-Gaussian. On the other hand, the statistical
models in most of existing papers for spatial data on $\mathbb{R}^{2}$ are
based on Gaussian processes and for non-Gaussian processes, it is often
difficult to check mixing conditions as considered in Lahiri and Zhu (2006)
and Bandyopadhyay et al. (2015), for example. Moreover, in many empirical
applications, the assumption of Gaussianity in spatial models is not
necessarily adequate (see Steel and Fuentes (2010) for example). As a result,
this study also contributes to the flexible modeling of nonparametric,
nonstationary and possibly non-Gaussian random fields on $\mathbb{R}^{d}$.
Therefore, this paper not only extends the scope of empirical analysis for
spatial and spatio-temporal data (discussed in Section 6) but also addresses
open questions on the dependence structure of statistical models built on
CARMA random fields.

The rest of this paper is organized as follows. In Section 2, we define
locally stationary random fields and describe our sampling scheme to consider
irregularly spaced data and mixing conditions for random fields. In Section 3,
we present the uniform convergence rate of general kernel estimators as well
as the estimators of the mean function $m$ in the model (1.1) over compact
sets. The asymptotic normality of the estimator of the mean function is also
provided. In Section 4, we consider the additive models and provide the
uniform convergence rate and joint asymptotic normality of the kernel
estimators for the additive functions. In Section 5, we discuss examples of
locally stationary random fields by introducing the concept of
$m_{n}$-dependent random fields; we also provide examples of locally
stationary Lévy-driven moving average random fields. We also discuss some
extensions of our results and applications to spatio-temporal data in Section
6. All the proofs are deferred to Appendix.

###  1.1. Notations

For $\bm{x}=(x_{1},\ldots,x_{d})^{\prime}\in\mathbb{R}^{d}$, let
$|\bm{x}|=|x_{1}|+\cdots+|x_{d}|$ and
$\|\bm{x}\|=(x_{1}^{2}+\cdots+x_{d}^{2})^{1/2}$ denote the $\ell^{1}$ and
$\ell^{2}$ norm on $\mathbb{R}^{d}$. For
$\bm{x}=(x_{1},\ldots,x_{d})^{\prime},\bm{y}=(y_{1},\ldots,y_{d})\in\mathbb{R}^{d}$,
the notation $\bm{x}\leq\bm{y}$ means that $x_{j}\leq y_{j}$ for all
$j=1,\ldots,d$. For any set $A\subset\mathbb{R}^{d}$, let $|A|$ denote the
Lebesgue measure of $A$ and $[\\![A]\\!]$ denote the number of elements in
$A$. For any positive sequences $a_{n}$ and $b_{n}$, we write $a_{n}\lesssim
b_{n}$ if a constant $C>0$ independent of $n$ exists such that $a_{n}\leq
Cb_{n}$ for all $n$, $a_{n}\sim b_{n}$ if $a_{n}\lesssim b_{n}$ and
$b_{n}\lesssim a_{n}$, and $a_{n}\ll b_{n}$ if $a_{n}/b_{n}\to 0$ as
$n\to\infty$. For $a\in\mathbb{R}$ and $b>0$, we use the shorthand notation
$[a\pm b]=[a-b,a+b]$. We use the notation $\stackrel{{\scriptstyle d}}{{\to}}$
to denote convergence in distribution. For random variables $X$ and $Y$, we
write $X\stackrel{{\scriptstyle d}}{{=}}Y$ if they have the same distribution.
$N(\mu,\Sigma)$ denotes a (multivariate) normal distribution with mean $\mu$
and a covariance matrix $\Sigma$. Let $P_{\bm{S}}$ denote the joint
probability distribution of the sequence of independent and identically
distributed (i.i.d.) random vectors $\\{\bm{S}_{0,j}\\}_{j\geq 1}$, and let
$P_{\cdot|\bm{S}}$ denote the conditional probability distribution given
$\\{\bm{S}_{0,j}\\}_{j\geq 1}$. Let $E_{\cdot|\bm{S}}$ and
$\operatorname{Var}_{\cdot|\bm{S}}$ denote the conditional expectation and
variance given $\\{\bm{S}_{0,j}\\}_{j\geq 1}$, respectively.
$\mathcal{B}(\mathbb{R}^{d})$ denotes the Borel $\sigma$-field on
$\mathbb{R}^{d}$.

##  2\. Settings

In this section, we introduce locally stationary random fields on
$\mathbb{R}^{d}$ that extend the concept of the locally stationary process on
$\mathbb{R}$. Furthermore, we discuss the sampling design of irregularly
spaced locations and the dependence structures for random fields.

###  2.1. Local stationarity

Intuitively, a random field $\\{\bm{X}_{\bm{s}_{j},A_{n}}:\bm{s}\in R_{n}\\}$
($A_{n}\to\infty$ as $n\to\infty$) is locally stationary if it behaves
approximately stationary in local space. To ensure that it is locally
stationary around each rescaled space point $\bm{u}$, a process
$\\{\bm{X}_{\bm{s},A_{n}}\\}$ can be approximated by a stationary random field
$\\{\bm{X}_{\bm{u}}(\bm{s}):\bm{s}\in\mathbb{R}^{d}\\}$ stochastically. See
Dahlhaus and Subba Rao (2006) for an example. This concept can be defined as
follows.

######  Definition 2.1.

The process
$\\{\bm{X}_{\bm{s},A_{n}}=(X_{\bm{s},A_{j}}^{1},\ldots,X_{\bm{s},A_{n}}^{p})^{\prime}:\bm{s}\in
R_{n}\\}$ is locally stationary if for each rescaled space point
$\bm{u}\in[0,1]^{d}$, there exists an associated random field
$\\{\bm{X}_{\bm{u}}(\bm{s})=(X_{\bm{u}}^{1}(\bm{s}),\ldots,X_{\bm{u}}^{p}(\bm{s}))^{\prime}:\bm{s}\in\mathbb{R}^{d}\\}$
with the following properties:

  * (i)

$\\{\bm{X}_{\bm{u}}(\bm{s}):\bm{s}\in\mathbb{R}^{d}\\}$ is strictly stationary
with density $f_{\bm{X}_{\bm{u}}(\bm{s})}$.

  * (ii)

It holds that

$\displaystyle\|\bm{X}_{\bm{s},A_{n}}-\bm{X}_{\bm{u}}(\bm{s})\|_{1}\leq\left(\left\|{\bm{s}\over
A_{n}}-\bm{u}\right\|_{2}+{1\over A_{n}^{d}}\right)U_{\bm{s},A_{n}}(\bm{u})\
a.s.,$ (2.1)

where $\\{U_{\bm{s},A_{n}}(\bm{u})\\}$ is a process of positive variables
satisfying $E[(U_{\bm{s},A_{n}}(\bm{u}))^{\rho}]<C$ for some $\rho>0$,
$C<\infty$ that is independent of $\bm{u},\bm{s}$, and $A_{n}$, and where
$\|\cdot\|_{1}$ and $\|\cdot\|_{2}$ denote arbitrary norms on $\mathbb{R}^{p}$
and $\mathbb{R}^{d}$, respectively.

Definition 2.1 is a natural extension of the concept of‘ local stationarity
for time series introduced in Dahlhaus (1997). We discuss examples of locally
stationary random fields in Section 5. In particular, we demonstrate that a
wide class of random fields, which includes locally stationary versions of
Lévy-driven moving average random fields, satisfy the Condition (2.1). We also
refer to Pezo (2018) and Matsuda and Yajima (2018) for other definitions of
locally stationary spatial and spatio-temporal processes by using their
spectral representations, respectively.

###  2.2. Sampling design

To account for irregularly spaced data, we consider the stochastic sampling
design. First, we define the sampling region $R_{n}$. Let $\\{A_{n}\\}_{n\geq
1}$ be a sequence of positive numbers such that $A_{n}\to\infty$ as
$n\to\infty$. We consider the following set as the sampling region.

$\displaystyle R_{n}=[0,A_{n}]^{d}.$ (2.2)

Next, we introduce our (stochastic) sampling designs. Let
$f_{\bm{S}}(\bm{s}_{0})$ be a continuous, everywhere positive probability
density function on $R_{0}=[0,1]^{d}$, and let $\\{\bm{S}_{0,j}\\}_{j\geq 1}$
be a sequence of i.i.d. random vectors with probability density
$f_{\bm{S}}(\bm{s}_{0})$ such that $\\{\bm{S}_{0,j}\\}_{j\geq 1}$ and
$\\{\bm{X}_{\bm{s},A}:\bm{s}\in R_{n}\\}$ are defined on a common probability
space $(\Omega,\mathcal{F},P)$ and are independent. We assume that the
sampling sites $\bm{s}_{1},\ldots,\bm{s}_{n}$ are obtained from realizations
$\bm{s}_{0,1},\ldots,\bm{s}_{0,n}$ of random vectors
$\bm{S}_{0,1},\ldots,\bm{S}_{0,n}$ by the following relation:

$\bm{s}_{j}=A_{n}\bm{s}_{0,j},\ j=1,\ldots,n.$

Herein, we assume that $nA_{n}^{-d}\to\infty$ as $n\to\infty$. We also assume
the following conditions on the sampling scheme.

######  Assumption 2.1.

  1. (S1)

For any $\bm{\alpha}\in\mathbb{N}^{d}$ with $|\bm{\alpha}|=1,2$,
$\partial^{\bm{\alpha}}f_{\bm{S}}(\bm{s})$ exists and is continuous on
$(0,1)^{d}$.

  2. (S2)

$C_{0}\leq nA_{n}^{-d}\leq C_{1}n^{\eta_{1}}$ for some $0<C_{0}<C_{1}<\infty$
and small $\eta_{1}\in[0,1)$.

  3. (S3)

Let $A_{1,n}$ and $A_{2,n}$ be two positive numbers such that as $n\to\infty$,
$A_{1,n},A_{2,n}\to\infty$ and ${A_{1,n}\over A_{n}}+{A_{2,n}\over
A_{1,n}}\leq C_{0}^{-1}n^{-\eta}\to 0$ for some $C_{0}>0$ and $\eta>0$.

Condition (S2) implies that our sampling design allows both the pure
increasing domain case ($\lim_{n\to\infty}nA_{n}^{-d}=\kappa\in(0,\infty)$)
and the mixed increasing domain case ($\lim_{n\to\infty}nA_{n}^{-d}=\infty$).
This implies that our study addresses the infill sampling criteria in the
stochastic design case (cf. Cressie (1993) and Lahiri (2003b)), which is of
interest in geostatistical and environmental monitoring applications (cf.
Lahiri (2003b) and Lahiri et al. (1999)). Moreover, in stochastic sampling
design, the sampling density can be nonuniform. There is another approach for
irregularly spaced sampling sites based on a homogeneous Poisson point process
(cf. Chapter 8 in Cressie (1993)). For a sample size $n$, the approach allows
the sampling sites to have only a uniform distribution over the sampling
region. Therefore, the stochastic sampling design in this study is flexible
compared with that based on homogeneous Poisson point processes and is
therefore useful for practical applications. It may be worthwhile to consider
sampling designs based on point processes that allow nonuniform sampling sites
such as Cox or Hawkes point processes; however, we did not consider them
because we believe that our sampling design is sufficiently practical for
numerous applications. Condition (S3) is considered to decompose the sampling
region $R_{n}$ into big and small blocks. See also the proof of Proposition
3.1 in Appendix A for details.

######  Remark 2.1.

In practice, $A_{n}$ can be determined using the diameter of the sampling
region. See Hall and Patil (1994) and Matsuda and Yajima (2009) for examples.
We can relax the assumption (2.2) on $R_{n}$ to a more general situation,
i.e.,

$R_{n}=\prod_{j=1}^{d}[0,A_{j,n}],$

where $A_{j,n}$ are sequences of positive constants with $A_{j,n}\to\infty$ as
$n\to\infty$. To avoid complicated results, we assumed (2.2). See also Section
6.1 for further discussion on the general sampling region and extensions of
our results to the general cases.

###  2.3. Mixing condition

Now we define $\beta$-mixing coefficients for a random field
$\widetilde{\bm{X}}$. Let
$\sigma_{\widetilde{\bm{X}}}(T)=\sigma(\\{\widetilde{\bm{X}}(\bm{s}):\bm{s}\in
T\\})$ be the $\sigma$-field generated by variables
$\\{\widetilde{\bm{X}(}\bm{s}):\bm{s}\in T\\}$, $T\subset\mathbb{R}^{d}$. For
subsets $T_{1}$ and $T_{2}$ of $\mathbb{R}^{d}$, let

$\bar{\beta}(T_{1},T_{2})=\sup{1\over
2}\sum_{j=1}^{J}\sum_{k=1}^{K}|P(A_{j}\cap B_{k})-P(A_{j})P(B_{k})|,$

where the supremum is taken over all pairs of (finite) partitions
$\\{A_{1},\ldots,A_{J}\\}$ and $\\{B_{1},\ldots,B_{K}\\}$ of $\mathbb{R}^{d}$
such that $A_{j}\in\sigma_{\widetilde{\bm{X}}}(T_{1})$ and
$B_{k}\in\sigma_{\widetilde{\bm{X}}}(T_{2})$. Furthermore, let
$d(T_{1},T_{2})=\inf\\{|\bm{x}-\bm{y}|:\bm{x}\in T_{1},\bm{y}\in T_{2}\\}$,
where $|\bm{x}|=\sum_{j=1}^{d}|x_{j}|$ for $\bm{x}\in\mathbb{R}^{d}$, and let
$\mathcal{R}(b)$ be the collection of all finite disjoint unions of cubes in
$\mathbb{R}^{d}$ with a total volume not exceeding $b$. Subsequently, the
$\beta$-mixing coefficients for the random field $\widetilde{\bm{X}}$ can be
defined as

$\displaystyle\beta(a;b)=\sup\\{\bar{\beta}(T_{1},T_{2}):d(T_{1},T_{2})\geq
a,T_{1},T_{2}\in\mathcal{R}(b)\\}.$ (2.3)

We assume that a non-increasing function $\beta_{1}$ with
$\lim_{a\to\infty}\beta_{1}(a)=0$ and a non-decreasing function $g_{1}$ exist
such that the $\beta$-mixing coefficient $\beta(a;b)$ satisfies the following
inequality:

$\beta(a;b)\leq\beta_{1}(a)g_{1}(b),\ a>0,b>0,$

where $g_{1}$ may be unbounded for $d\geq 2$.

In Section 5, we can show that our results also hold for a locally stationary
random field $\\{\bm{X}_{\bm{s},A_{n}}:\bm{s}\in R_{n}\\}$ which is
approximated by a $\beta$-mixing random field, i.e.,

$\displaystyle\bm{X}_{\bm{s},A_{n}}$
$\displaystyle=\widetilde{\bm{X}}_{\bm{s},A_{n}}+\widetilde{\bm{\epsilon}}_{\bm{s},A_{n}},$
(2.4)

where $\widetilde{\bm{X}}_{\bm{s},A_{n}}$ is a $\beta$-mixing random field
with a $\beta$-mixing coefficient such that $\beta(a;b)=0$ for $a\geq A_{2,n}$
(i.e. $A_{2,n}$-dependent) and $\widetilde{\bm{\epsilon}}_{\bm{s},A_{n}}$ is a
“residual” random field which is asymptotically negligible. See also
Definition 5.1 in Section 5 for the meaning of asymptotic negligibility.

We can also define $\alpha$-mixing coefficients $\alpha(a;b)$ for a random
field $\bm{X}$ similar to the way we defined $\beta(a;b)$ and it is known that
$\beta$-mixing implies $\alpha$-mixing in general. However, it is difficult to
compute $\beta$\- (and even for $\alpha$-)mixing coefficients of a given
random field on $\mathbb{R}^{d}$, which is a key different point from time
series data. Moreover, almost all existing papers assume technical conditions
on mixing coefficients which are difficult to check except for some Gaussian
random fields, $m$-dependent random fields on $\mathbb{R}^{2}$ ($m>0$: fix)
and linear random fields on $\mathbb{Z}^{2}$ (cf. Hallin et al. (2004), Lahiri
and Zhu (2006), Lu and Tjøstheim (2014), Bandyopadhyay et al. (2015) and
references therein). In Section 5, we introduce a concept of approximately
$m_{n}$-dependent random fields. We can check that a wide class of random
fields on $\mathbb{R}^{d}$ which includes non-Gaussian random fields as well
as Gaussian random fields satisfies (2.4) and we can show that our results
hold for such a class of random fields. See Section 5.3 for details.

######  Remark 2.2.

It is important to restrict the size of index sets $T_{1}$ and $T_{2}$ in the
definition of $\beta(a;b)$. To see this, we define the $\beta$-mixing
coefficients of a random field $\bm{X}$ as a natural extension of the
$\beta$-mixing coefficients for the time series as follows: Let
$\mathcal{O}_{1}$ and $\mathcal{O}_{2}$ be half-planes with boundaries $L_{1}$
and $L_{2}$, respectively. For each real number $a>0$, we define the following
quantity

$\displaystyle\beta(a)=\sup\left\\{\bar{\beta}(\mathcal{O}_{1},\mathcal{O}_{2}):d(\mathcal{O}_{1},\mathcal{O}_{2})\geq
a\right\\},$

where $\sup$ is taken over all pairs of parallel lines $L_{1}$ and $L_{2}$
such that $d(L_{1},L_{2})\geq a$. Subsequently, we obtain the following
result:

######  Theorem 2.1 (Theorem 1 in Bradley (1989)).

Suppose $\\{\bm{X}(s):\bm{s}\in\mathbb{R}^{2}\\}$ is a strictly stationary
mixing random field, and $a>0$ is a real number. Then $\beta(a)=1\ \text{or}\
0$.

This implies that if a random field $\bm{X}$ is $\beta$-mixing with respect to
the $\beta$-mixing coefficient $\beta(\cdot)$ ($\lim_{a\to\infty}\beta(a)=0$),
then the random field $\bm{X}$ is “$m$”-dependent, i.e., $\beta(a)=0$ for some
$a>m$, where $m$ is a fixed positive constant. For practical purposes, this is
extremely restrictive. Therefore, we adopted the definition (2.3) for
$\beta$-mixing random fields. We refer to Bradley (1993), Doukhan (1994), and
Dedecker et al. (2007) for details regarding the mixing coefficients for
random fields.

###  2.4. Discussion on mixing conditions

It is often assumed that $\bm{X}$ is $\alpha$-mixing and blocking techniques
are applied to construct asymptotically independent blocks of observations and
to show asymptotic normality of estimators based on the convergence of
characteristic functions (cf. Proposition 2.6 in Fan and Yao (2003)). See also
Lahiri (2003b), Lahiri and Zhu (2006) and Bandyopadhyay et al. (2015) for
example. Vogt (2012) used an exponential inequality for an equidistant
$\alpha$-mixing sequence (Theorem 2.1 in Liebscher (1996)) to derive uniform
convergence rates of kernel estimators for locally stationary time series. We
can show pointwise convergence results under $\alpha$-mixing conditions
without changing proofs. Precisely, the central limit theorems (Theorems 3.2
and 4.2) below hold under the same Assumptions by replacing conditions on
$\beta$-mixing coefficient $\beta(a;b)$ with conditions on $\alpha$-mixing
coefficient $\alpha(a;b)$ that is defined as follows: Let $\widetilde{\bm{X}}$
be a random field on $\mathbb{R}^{d}$. For any two subsets $T_{1}$ and $T_{2}$
of $\mathbb{R}^{d}$, let $\widetilde{\alpha}(T_{1},T_{2})=\sup\\{|P(A\cap
B)-P(A)P(B)|:A\in\sigma_{\widetilde{\bm{X}}}(T_{1}),B\in\sigma_{\widetilde{\bm{X}}}(T_{2})\\}$.
Then the $\alpha$-mixing coefficient of the random field $\widetilde{\bm{X}}$
is defined as

$\displaystyle\alpha(a;b)=\sup\\{\widetilde{\alpha}(T_{1},T_{2}):d(T_{1},T_{2})\geq
a,T_{2},T_{2}\in\mathcal{R}(b)\\}.$ (2.5)

As with the $\beta$-mixing coefficients, we also assume that a non-increasing
function $\alpha_{1}$ with $\lim_{a\to\infty}\alpha_{1}(a)=0$ and a non-
decreasing function $g^{\prime}_{1}$ exist such that the $\alpha$-mixing
coefficient $\alpha(a;b)$ satisfies the following inequality:

$\alpha(a;b)\leq\alpha_{1}(a)g^{\prime}_{1}(b),\ a>0,b>0,$

where $g^{\prime}_{1}$ may be unbounded for $d\geq 2$. We refer to Lahiri and
Zhu (2006) and Bandyopadhyay et al. (2015) for the definition and discussion
of $\alpha$-mixing coefficients for random fields on $\mathbb{R}^{d}$.

On the other hand, the proofs of main results in this paper are based on a
general blocking technique designed for irregularly spaced sampling sites.
Moreover, to derive the uniform convergence rates of the kernel estimators, we
need to care about the effect of non-equidistant sampling sites when applying
a maximal inequality and it requires additional work compared with the case
that sampling sites are equidistant. Indeed, in place of using results for
(regularly spaced) stationary sequence, which cannot be applied to the
analysis of irregularly spaced non-stationary data, we construct “exactly”
independent blocks of observations and apply results for independent data to
the independent blocks since there is no practical guidance for introducing an
order to spatial points as opposed to time series. Precisely, we first reduce
the dependent data to not asymptotically but exactly independent blocks in
finite sample by using the key result in Yu (1994)(Corollary 2.7) which does
not require any dependence structure and regularly spaced sampling sites. Then
we apply the Bernstein’s inequality for independent and possibly not
identically distributed random variables to the independent blocks. It also
should be noted that according to Remarks (ii) after the proof Lemma 4.1 in Yu
(1994), the result on construction of independent blocks for $\beta$-mixing
sequence would not hold for $\alpha$-mixing sequences. Therefore, we work with
$\beta$-mixing sequence for uniform estimation of regression functions.

As a recent contribution in econometrics, Chernozhukov et al. (2019) extended
high-dimensional central limit theorems for independent data developed by
Chernozhukov et al. (2013, 2017) to possibly nonstationary $\beta$-mixing time
series under a nonstochastic sampling design using the technique in Yu (1994).
See also Section 6.2 for further discussion on the mixing conditions.

##  3\. Main results

In this section, we consider general kernel estimators and derive their
uniform convergence rates. Based on the result, we derive the uniform
convergence rate and asymptotic normality of an estimator for the mean
function in the model (1.1).

###  3.1. Kernel estimation for regression functions

We consider the following kernel estimator for $m(\bm{u},\bm{x})$ in the model
(1.1):

$\displaystyle\widehat{m}(\bm{u},\bm{x})$
$\displaystyle={\sum_{j=1}^{n}\bar{K}_{h}(\bm{u}-\bm{s}_{j}/A_{n})\prod_{\ell=1}^{p}K_{h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})Y_{\bm{s}_{j},A_{n}}\over\sum_{j=1}^{n}\bar{K}_{h}(\bm{u}-\bm{s}_{j}/A_{n})\prod_{\ell=1}^{p}K_{h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})}.$
(3.1)

Here,
$\bm{X}_{\bm{s},A_{n}}=(X_{\bm{s},A_{n}}^{1},\ldots,X_{\bm{s},A_{n}}^{p})^{\prime}$,
$\bm{x}=(x_{1},\ldots,x_{p})^{\prime}$ and
$\bm{u}=(u_{1},\ldots,u_{p})^{\prime}\in\mathbb{R}^{p}$. $K$ denotes a one-
dimensional kernel function, and we used the notations $K_{h}(v)=K(v/h)$,
$\bar{K}_{h}(\bm{u})=\bar{K}(\bm{u}/h)$ where
$\bar{K}(\bm{u})=\prod_{j=1}^{d}K(u_{j})$ and
$\bm{u}/h=(u_{1}/h,\ldots,u_{d}/h)^{\prime}$.

Before we state the main results, we summarize the assumptions made for the
model (1.1) and kernel functions. These assumptions are standard; similar
assumptions are made in Vogt (2012) and Zhang and Wu (2015).

######  Assumption 3.1.

  1. (M1)

The process $\\{\bm{X}_{\bm{s},A_{n}}\\}$ is locally stationary. Hence, for
each space point $\bm{u}\in[0,1]^{d}$, a strictly stationary random field
$\\{\bm{X}_{\bm{u}}(\bm{s})\\}$ exists such that

$\|\bm{X}_{\bm{s},A_{n}}-\bm{X}_{\bm{u}}(\bm{s})\|\leq\left(\left\|{\bm{s}\over
A_{n}}-\bm{u}\right\|+{1\over A_{n}^{d}}\right)U_{\bm{s},A_{n}}(\bm{u})\
\text{a.s.}$

with $E[(U_{\bm{s},A_{n}}(\bm{u}))^{\rho}]\leq C$ for some $\rho>0$.

  2. (M2)

The density $f(\bm{u},\bm{x})=f_{\bm{X}_{\bm{u}}(\bm{s})}(\bm{x})$ of the
variable $\bm{X}_{\bm{u}}(\bm{s})$ is smooth in $\bm{u}$. In particular,
$f(\bm{u},\bm{x})$ is partially differentiable with respect to (w.r.t.)
$\bm{u}\in(0,1)^{d}$ for each $\bm{x}\in\mathbb{R}^{p}$, and the derivatives
$\partial_{u_{i}}f(\bm{u},\bm{x})={\partial\over\partial
u_{i}}f(\bm{u},\bm{x})$, $1\leq i\leq d$ are continuous.

  3. (M3)

The $\beta$-mixing coefficients of the array
$\\{\bm{X}_{\bm{s},A_{n}},\epsilon_{\bm{s},A_{n}}\\}$ satisfy
$\beta(a;b)\leq\beta_{1}(a)g_{1}(b)$ with $\beta_{1}(a)\to 0$ as $a\to\infty$.

  4. (M4)

$f(\bm{u},\bm{x})$ is partially differentiable w.r.t. $\bm{x}$ for each
$\bm{u}\in[0,1]^{d}$. The derivatives
$\partial_{x_{i}}f(\bm{u},\bm{x}):={\partial\over\partial
x_{i}}f(\bm{u},\bm{x})$, $1\leq i\leq p$ are continuous.

  5. (M5)

$m(\bm{u},\bm{x})$ is twice continuously partially differentiable with first
derivatives $\partial_{u_{i}}m(\bm{u},\bm{x})={\partial\over\partial
u_{i}}m(\bm{u},\bm{x})$,
$\partial_{x_{i}}m(\bm{u},\bm{x})={\partial\over\partial
x_{i}}m(\bm{u},\bm{x})$ and second derivatives
$\partial_{u_{i}u_{j}}^{2}m(\bm{u},\bm{x})={\partial^{2}\over\partial
u_{i}\partial u_{j}}m(\bm{u},\bm{x})$,
$\partial_{u_{i}x_{j}}^{2}m(\bm{u},\bm{x})={\partial^{2}\over\partial
u_{i}\partial x_{j}}m(\bm{u},\bm{x})$,
$\partial_{x_{i}x_{j}}^{2}m(\bm{u},\bm{x})={\partial^{2}\over\partial
x_{i}\partial x_{j}}m(\bm{u},\bm{x})$.

######  Remark 3.1.

We can verify that a wide class of random fields satisfies Condition (M1).
Indeed, our results can be applied to a class of locally stationary Lévy-
driven moving average random fields that include non-stationary CARMA random
fields and CARMA random fields are known as a rich class of models for spatial
data (cf. Brockwell and Matsuda (2017), Matsuda and Yajima (2018) and Matsuda
and Yuan (2020)). See also Section 5 for detailed discussion on the properties
of locally stationary Lévy-driven moving average random fields.

######  Assumption 3.2.

  1. (KB1)

The kernel $K$ is symmetric around zero, bounded, and has a compact support,
i.e., $K(v)=0$ for all $|v|>C_{1}$ for some $C_{1}<\infty$. Moreover, $K$ is
Lipschitz continuous, i.e., $|K(v_{1})-K(v_{2})|\leq C_{2}|v_{1}-v_{2}|$ for
some $C_{2}<\infty$ and all $v_{1},v_{2}\in\mathbb{R}$.

  2. (KB2)

The bandwidth $h$ is assumed to converge to zero at least at a polynomial
rate, that is, there exists a small $\xi_{1}>0$ such that $h\leq
Cn^{-\xi_{1}}$ for some constant $0<C<\infty$.

###  3.2. Uniform convergence rates for general kernel estimators

As a first step to study the asymptotic properties of estimators (3.1), we
analyze the following general kernel estimator:

$\displaystyle\widehat{\psi}(\bm{u},\bm{x})$ $\displaystyle={1\over
nh^{p+d}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell}\right)W_{\bm{s}_{j},A_{n}},$
(3.2)

where $\bar{K}(\bm{u})=\prod_{j=1}^{d}K(u_{j})$ and
$\\{W_{\bm{s}_{j},A_{n}}\\}$ is an array of one-dimensional random variables.
Many kernel estimators, such as Nadaraya–Watson estimators, can be represented
by (3.2). In this study, we use the results with $W_{\bm{s},A_{n}}=1$ and
$W_{\bm{s},A_{n}}=\epsilon_{\bm{s},A_{n}}$.

Next, we derived the uniform convergence rate of
$\widehat{\psi}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{\psi}(\bm{u},\bm{x})]$.
We assumed the following for the components in (3.2). Similar assumptions are
made in Hansen (2008), Kristensen (2009), and Vogt (2012).

######  Assumption 3.3.

  1. (U1)

It holds that $E[|W_{\bm{s},A_{n}}|^{\zeta}]\leq C$ for some $\zeta>2$ and
$C<\infty$.

  2. (U2)

The $\beta$-mixing coefficients of the array
$\\{\bm{X}_{\bm{s},A_{n}},W_{\bm{s},A_{n}}\\}$ satisfy
$\beta(a;b)\leq\beta_{1}(a)g_{1}(b)$ with $\beta_{1}(a)\to 0$ as $a\to\infty$.

  3. (U3)

Let $f_{\bm{X}_{\bm{s},A_{n}}}$ be the density of $\bm{X}_{\bm{s},A_{n}}$. For
any compact set $S_{c}\subset\mathbb{R}^{p}$, a constant $C=C(S_{c})$ exists
such that

$\displaystyle\sup_{\bm{s},A_{n}}\sup_{\bm{x}\in
S_{c}}f_{\bm{X}_{\bm{s},A_{n}}}(\bm{x})\leq C\ \text{and}\
\sup_{\bm{s},A_{n}}\sup_{\bm{x}\in
S_{c}}E[|W_{\bm{s},A_{n}}|^{\zeta}|\bm{X}_{\bm{s},A_{n}}=\bm{x}]\leq C.$

Moreover, for all distinct $\bm{s}_{1},\bm{s}_{2}\in R_{n}$,

$\displaystyle\sup_{\bm{s}_{1},\bm{s}_{2},A_{n}}\sup_{\bm{x}_{1},\bm{x}_{2}\in
S_{c}}E[|W_{\bm{s}_{1},A_{n}}||W_{\bm{s}_{2},A_{n}}||\bm{X}_{\bm{s}_{1},A_{n}}=\bm{x}_{1},\bm{X}_{\bm{s}_{2},A_{n}}=\bm{x}_{2}]\leq
C.$

Furthermore, we assume the following regularity conditions.

######  Assumption 3.4.

Let $a_{n}=\sqrt{{\log n}\over nh^{d+p}}$. As $n\to\infty$,

  1. (R1)

$h^{-(d+p)}a_{n}^{d+p}A_{n}^{d}A_{1,n}^{-d}\beta(A_{2,n};A_{n}^{d})\to 0$ and
$A_{1,n}^{d}A_{n}^{-d}nh^{d+p}(\log n)\to 0$,

  2. (R2)

${n^{1/2}h^{(d+p)/2}\over A_{1,n}^{d}n^{1/\zeta}}\geq C_{0}n^{\eta}$ for some
$0<C_{0}<\infty$ and $\eta>0$,

  3. (R3)

$A_{n}^{dr}h^{p}\to\infty$,

where $\zeta$ is a positive constant that appears in Assumption 3.3.

Discussions on the assumptions regarding the mixing condition in Assumptions
3.4 and 3.6 (introduced in Section 3.3) are given in Section 5 and Appendix A.

######  Proposition 3.1.

Let $S_{c}$ be a compact subset of $\mathbb{R}^{p}$. Then, under Assumptions
2.1, 3.2, 3.3 and 3.4, the following results hold for $P_{\bm{S}}$ almost
surely:

$\displaystyle\sup_{\bm{u}\in[0,1]^{d},\bm{x}\in
S_{c}}|\widehat{\psi}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{\psi}(\bm{u},\bm{x})]|$
$\displaystyle=O_{P_{\cdot|\bm{S}}}\left(\sqrt{\log n\over nh^{d+p}}\right).$

###  3.3. Asymptotic properties of $\widehat{m}$

Next, we present our main results for the estimation of mean function $m$ for
the model (1.1).

######  Theorem 3.1.

Let $I_{h}=[C_{1}h,1-C_{1}h]^{d}$ and let $S_{c}$ be a compact subset of
$\mathbb{R}^{p}$. Suppose that $\inf_{\bm{u}\in[0,1]^{d},\bm{x}\in
S_{c}}f(\bm{u},\bm{x})>0$. Then, under Assumptions 2.1, 3.1, 3.2, 3.3 (with
$W_{\bm{s}_{j},A_{n}}=1$ and $\epsilon_{\bm{s}_{j},A_{n}}$) and 3.4, the
following result holds for $P_{\bm{S}}$ almost surely:

$\displaystyle\sup_{\bm{u}\in I_{h},\bm{x}\in
S_{c}}\left|\widehat{m}(\bm{u},\bm{x})-m(\bm{u},\bm{x})\right|$
$\displaystyle=O_{P_{\cdot|\bm{S}}}\left(\sqrt{{\log n\over
nh^{d+p}}}+h^{2}+{1\over A_{n}^{dr}h^{p}}\right),$

where $r=\min\\{1,\rho\\}$.

The term $A_{n}^{-dr}h^{-p}$ in the convergence rate arises from the local
stationarity of $\bm{X}_{\bm{s},A_{n}}$, i.e., the approximation error of
$\bm{X}_{\bm{s},A_{n}}$ by a stationary random field
$\bm{X}_{\bm{u}}(\bm{s})$.

To demonstrate the asymptotic normality of the estimator, we additionally
assume the following conditions:

######  Assumption 3.5.

  1. (Ua1)

(U1), (U2) and (U3) in Assumption 3.3 hold.

  2. (Ua2)

for all distinct $\bm{s}_{1},\bm{s}_{2},\bm{s}_{3}\in R_{n}$, there exists a
constant $C<\infty$ such that

$\displaystyle\sup_{\bm{s}_{1},\bm{s}_{2},\bm{s}_{3},A_{n}}\sup_{\bm{x}_{1},\bm{x}_{2}\bm{x}_{3}\in
S_{c}}E[|W_{\bm{s}_{1},A_{n}}||W_{\bm{s}_{2},A_{n}}||W_{\bm{s}_{3},A_{n}}||\bm{X}_{\bm{s}_{1},A_{n}}=\bm{x}_{1},\bm{X}_{\bm{s}_{2},A_{n}}=\bm{x}_{2},\bm{X}_{\bm{s}_{3},A_{n}}=\bm{x}_{3}]\leq
C.$

######  Assumption 3.6.

As $n\to\infty$,

  1. (Ra1)

(R1) and (R2) in Assumption 3.4 hold.

  2. (Ra2)

$A_{n}^{dr}h^{p+2}\to\infty$.

  3. (Ra3)

$\left({1\over nh^{p+d}}\right)^{1/3}\left({A_{1,n}\over
A_{n}}\right)^{2d/3}\left({A_{2,n}\over
A_{1,n}}\right)^{2/3}g^{1/3}_{1}(A_{1,n}^{d})\sum_{k=1}^{A_{n}/A_{1,n}}k^{d-1}\beta_{1}^{1/3}(kA_{1,n}+A_{2,n})\to
0.$

The asymptotic normality of the kernel estimators can be established under
$\alpha$-mixing that is weaker than $\beta$-mixing. See also (2.5) for the
definition of $\alpha$-mixing coefficients for random fields.

######  Theorem 3.2.

Suppose that $f(\bm{u},\bm{x})>0$, $f_{\bm{S}}(\bm{u})>0$ and
$\epsilon_{\bm{s}_{j},A_{n}}=\sigma\left({\bm{s}_{j}\over
A_{n}},\bm{x}\right)\epsilon_{j}$, where $\sigma(\cdot,\cdot)$ is continuous
and $\\{\epsilon_{j}\\}_{j=1}^{n}$ is a sequence of i.i.d. random variables
with mean zero and variance $1$. Moreover, suppose $nh^{d+p+4}\to c_{0}$ for a
constant $c_{0}$. Then under Assumptions 2.1, 3.1, 3.2, 3.5 and 3.6 with
$W_{\bm{s}_{j},A_{n}}=1$ and $\epsilon_{\bm{s}_{j},A_{n}}$in Assumption 3.5,
the following result holds for $P_{\bm{S}}$ almost surely:

$\displaystyle\sqrt{nh^{d+p}}(\widehat{m}(\bm{u},\bm{x})-m(\bm{u},\bm{x}))\stackrel{{\scriptstyle
d}}{{\longrightarrow}}N(B_{\bm{u},\bm{x}},V_{\bm{u},\bm{x}}),$

where

$\displaystyle B_{\bm{u},\bm{x}}$ $\displaystyle=\sqrt{c_{0}}{\kappa_{2}\over
2}\left\\{\sum_{i=1}^{d}\left(2\partial_{u_{i}}m(\bm{u},\bm{x})\partial_{u_{i}}f(\bm{u},\bm{x})+\partial_{u_{i}u_{i}}^{2}m(\bm{u},\bm{x})f(\bm{u},\bm{x})\right)\right.$
$\displaystyle\left.\quad+\sum_{k=1}^{p}\left(2\partial_{x_{k}}m(\bm{u},\bm{x})\partial_{x_{k}}f(\bm{u},\bm{x})+\partial_{x_{k}x_{k}}^{2}m(\bm{u},\bm{x})f(\bm{u},\bm{x})\right)\right\\}$

and
$V_{\bm{u},\bm{x}}=\kappa_{0}^{d+p}\sigma^{2}(\bm{u},\bm{x})/(f_{\bm{S}}(\bm{u})f(\bm{u},\bm{x}))$
with $\kappa_{0}=\int_{\mathbb{R}}K^{2}(x)dx$ and
$\kappa_{2}=\int_{\mathbb{R}}x^{2}K(x)dx$. The same result holds true even if
we replace the conditions on $\beta$-mixing coefficients in Assumptions 3.1,
3.5 and 3.6 with those on $\alpha$-mixing coefficients defined by (2.5).

The asymptotic variance $V_{\bm{u},\bm{x}}$ depends on the sampling density
$f_{\bm{S}}(\bm{u})$ through $1/f_{S}(\bm{u})$. This dependence of variance on
sampling density differs in stochastic and nonstochastic (or equidistant)
sampling designs. Intuitively, the result implies that around “hot-spots,”
where more sampling sites tend to be selected, the asymptotic variance of the
estimator is smaller than that of the other spatial points. The same applies
to Theorem 4.2.

######  Remark 3.2.

As shown in Theorem 3.2, the expressions of asymptotic bias and variance of
the kernel estimator are very similar in structure to those from a standard
random design for stationary time series and random fields. Therefore, we
conjecture that the methods to choose the bandwidth in such a design can be
adapted to our setting. In particular, using the formulas for the asymptotic
bias and variance from Theorem 3.2, it should be possible to select the
bandwidth via plug-in methods.

######  Remark 3.3.

Set $A_{n}^{d}=O(n^{1-\bar{\eta}_{1}})$ for some $\bar{\eta}_{1}\in[0,1)$,
$A_{1,n}=O(A_{n}^{\gamma_{A_{1}}})$, $A_{2,n}=O(A_{n}^{\gamma_{A_{2}}})$ with
$0<\gamma_{A_{2}}<\gamma_{A_{1}}<1/3$ and $r=\min\\{1,\rho\\}=1$. Assume that
we can take a sufficiently large $\zeta>2$ such that
${2\over\zeta}<(1-\bar{\eta}_{1})(1-3\gamma_{A_{1}})$. Then, Assumption 3.6 is
satisfied for $d\geq 1$ and $p\geq 1$. See Remarks A.4 and B.1 for details.

##  4\. Additive models

In the previous section, we considered general kernel estimators and discussed
their asymptotic properties; however, the estimators are adversely affected by
dimensionality. In particular, the convergence rate
$O_{P_{\cdot|\bm{S}}}\left(\sqrt{\log n/nh^{d+p}}\right)$ of the estimators
deteriorated as the dimension of the covariates $p$ increases. Hence, we
consider additive models inspired by the idea presented in Vogt (2012), which
is based on the smooth backfitting method developed by Mammen et al. (1999),
and studied the asymptotic properties of the estimators of additive functions.

###  4.1. Construction of estimators

We place the following structural constraint on $m(\bm{u},\bm{x})$:

$\displaystyle E[Y_{\bm{s},A_{n}}|\bm{X}_{\bm{s},A_{n}}=\bm{x}]$
$\displaystyle=m\left({\bm{s}\over A_{n}},\bm{x}\right)$
$\displaystyle=m_{0}\left({\bm{s}\over
A_{n}}\right)+\sum_{\ell=1}^{p}m_{\ell}\left({\bm{s}\over
A_{n}},x_{\ell}\right).$ (4.1)

Model (4.1) is a natural extension of the following linear regression models
with spatially varying coefficients (cf. Gelfand et al. (2003)):

$Y_{\bm{s},A_{n}}=\beta_{0}\left({\bm{s}\over
A_{n}}\right)+\sum_{\ell=1}^{p}\beta_{\ell}\left({\bm{s}\over
A_{n}}\right)X_{\bm{s},A_{n}}^{\ell}+\epsilon_{\bm{s},A_{n}}.$

######  Remark 4.1.

In this paper, we focus on the circumvention of the curse of dimensionality
that comes from the number of covariates since in some applications it seems
not suitable to consider additive components over the spatial coordinates. For
example, many environmental and climate data such as temperature and
precipitation would have not spatial coordinate-wise but location specific
features. Moreover, one of our motivation is extending the spatially varying
linear regression models in Gelfand et al. (2003), which is one of the
influential papers in spatial statistics, to nonparametric, non-stationary and
non-Gaussian settings. They apply their methods to the analysis of log selling
price of single family homes and their regression model does not include
additive components over spatial coordinates. Brockwell and Matsuda (2017)
also considers regression models that do not include additive components over
spatial coordinates and estimating parameters of a CARMA random field with an
application of their methods to land price data. Therefore, we believe our
modeling would not be restrictive in many empirical applications.

To identify the additive function of the model (4.1) within a unit cube
$[0,1]^{p}$, we impose the condition that

$\displaystyle\int
m_{\ell}(\bm{u},x_{\ell})p_{\ell}(\bm{u},x_{\ell})dx_{\ell}=0,\
\ell=1,\ldots,p$

and all rescaled space points $\bm{u}\in[0,1]^{d}$. Here, the functions
$p_{\ell}(\bm{u},x_{\ell})=\int_{\mathbb{R}^{p}}p(\bm{u},\bm{x})d\bm{x}_{-\ell}$
are the marginals of the density

$\displaystyle p(\bm{u},\bm{x})$
$\displaystyle={I(\bm{x}\in[0,1]^{p})f(\bm{u},\bm{x})\over
P(X_{\bm{u}}(\bm{0})\in[0,1]^{p})},$

where $f(\bm{u},\cdot)$ is the density of the strictly stationary random field
$\\{X_{\bm{u}}(\bm{s})\\}$.

To estimate the functions $m_{0},\ldots,m_{p}$, we apply the strategy used in
Vogt (2012), which is based on the smooth backfitting technique developed in
Mammen et al. (1999). First, we introduce the auxiliary estimates.

$\displaystyle\widehat{p}(\bm{u},\bm{x})$ $\displaystyle={1\over
n_{[0,1]^{p}}}\sum_{j=1}^{n}I(\bm{X}_{\bm{s}_{j},A_{n}}\in[0,1]^{p})\bar{K}_{h}\left(\bm{u},{\bm{s}_{j}\over
A_{n}}\right)\prod_{\ell=1}^{p}K_{h}\left(x_{\ell},X_{\bm{s}_{j},A_{n}}^{\ell}\right),$
$\displaystyle\widehat{m}(\bm{u},\bm{x})$ $\displaystyle={1\over
n_{[0,1]^{p}}}\sum_{j=1}^{n}I(\bm{X}_{\bm{s}_{j},A_{n}}\in[0,1]^{p})\bar{K}_{h}\left(\bm{u},{\bm{s}_{j}\over
A_{n}}\right)\prod_{\ell=1}^{p}K_{h}\left(x_{\ell},X_{\bm{s}_{j},A_{n}}^{\ell}\right)Y_{\bm{s}_{j},A_{n}}/\widehat{p}(\bm{u},\bm{x}),$

where $\bar{K}_{h}\left(\bm{u},{\bm{s}_{j}\over
A_{n}}\right)=\prod_{\ell=1}^{d}K_{h}\left(u_{\ell},{s_{j,\ell}\over
A_{n}}\right)$, $\widehat{p}(\bm{u},\bm{x})$ is a kernel estimator for the
function $f_{\bm{S}}(\bm{u})p(\bm{u},\bm{x})$, and
$\widehat{m}(\bm{u},\bm{x})$ is a $(p+d)$-dimensional kernel estimator that
estimates $m(\bm{u},\bm{x})$ for $\bm{x}\in[0,1]^{p}$. In the aforementioned
definitions,

$\displaystyle n_{[0,1]^{p}}$
$\displaystyle=\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u},{\bm{s}_{j}\over
A_{n}}\right)I(\bm{X}_{\bm{s}_{j},A_{n}}\in[0,1]^{p})/\widetilde{f}_{S}(\bm{u})$

is the number of observations in the unit cube $[0,1]^{p}$, where only space
points close to $\bm{u}$ are considered. Furthermore,
$\widetilde{f}_{\bm{S}}(\bm{u})={1\over
n}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u},{\bm{s}_{j}\over A_{n}}\right)$, and

$\displaystyle K_{h}(v,w)$
$\displaystyle=I(v,w\in[0,1]){K_{h}(v-w)\over\int_{[0,1]}K_{h}(s-w)ds}$

is a modified kernel weight. This weight possesses the property that
$\int_{[0,1]}K_{h}(v,w)dv=1$ for all $w\in[0,1]$, which is required to derive
the asymptotic properties of the backfitting estimates.

Given the estimators $\widehat{p}$ and $\widehat{m}$, we define the smooth
backfitting estimates
$\widetilde{m}_{0}(\bm{u}),\widetilde{m}_{1}(\bm{u},\cdot),\ldots,\widetilde{m}_{d}(\bm{u},\cdot)$
of the functions
$m_{0}(\bm{u}),m_{1}(\bm{u},\cdot),\ldots,m_{d}(\bm{u},\cdot)$ at the space
point $\bm{u}\in[0,1]^{d}$ as the minimizer of the criterion

$\displaystyle\int_{[0,1]^{p}}\left\\{\widehat{m}(\bm{u},\bm{w})-\left(g_{0}+\sum_{\ell=1}^{p}g_{\ell}(w_{\ell})\right)\right\\}^{2}\widehat{p}(\bm{u},\bm{w})d\bm{w},$
(4.2)

where the minimization runs over all additive functions
$g(\bm{x})=g_{0}+\sum_{\ell=1}^{p}g_{\ell}(x^{\ell})$, whose components are
normalized to satisfy

$\displaystyle\int
g_{\ell}(w_{\ell})\widehat{p}_{\ell}(\bm{u},w_{\ell})dw_{\ell}=0,\
\ell=1,\ldots,p.$

Here,
$\widehat{p}_{\ell}(\bm{u},x_{\ell})=\int_{[0,1]^{p-1}}\widehat{p}(\bm{u},\bm{x})d\bm{x}_{-\ell}$
is the marginal of the kernel density $\widehat{p}(\bm{u},\cdot)$ at point
$x_{\ell}$. According to (4.2), the estimate
$\widetilde{m}(\bm{u},\cdot)=\widetilde{m}_{0}(\bm{u})+\sum_{\ell=1}^{p}\widetilde{m}_{\ell}(\bm{u},\cdot)$
is an $L^{2}$-projection of the full dimensional kernel estimate
$\widehat{m}(\bm{u},\cdot)$ on to the subspace of additive functions, where
the projection is performed with respect to the density estimate
$\widehat{p}(\bm{u},\cdot)$.

By differentiation, we can demonstrate that the solution of (4.2) is
characterized by the system of equations

$\displaystyle\widetilde{m}_{\ell}(\bm{u},x_{\ell})$
$\displaystyle=\widehat{m}_{\ell}(\bm{u},x_{\ell})-\sum_{k\neq\ell}\int_{[0,1]}\widetilde{m}_{k}(\bm{u},x_{k}){\widehat{p}_{\ell,k}(\bm{u},x_{\ell},x_{k})\over\widehat{p}_{\ell}(\bm{u},x_{\ell})}dx_{k}-\widetilde{m}_{0}(\bm{u})$

together with

$\displaystyle\int\widetilde{m}_{\ell}(\bm{u},w_{\ell})\widehat{p}_{\ell}(\bm{u},w_{\ell})dw_{\ell}=0,\
\ell=1,\ldots,p,$

where $\widehat{p}_{\ell}$ and $\widehat{p}_{\ell,k}$ are kernel density
estimates, and $\widehat{m}_{\ell}$ is a kernel estimator defined as

$\displaystyle\widehat{p}_{\ell}(\bm{u},x_{\ell})$ $\displaystyle={1\over
n_{[0,1]^{p}}}\sum_{j=1}^{n}I\left(\bm{X}_{\bm{s}_{j},A_{n}}\in[0,1]^{p}\right)\bar{K}_{h}\left(\bm{u},{\bm{s}_{j}\over
A_{n}}\right)K_{h}\left(x_{\ell},X_{\bm{s}_{j},A_{n}}^{\ell}\right),$
$\displaystyle\widehat{p}_{\ell,k}(\bm{u},x_{\ell},x_{k})$
$\displaystyle={1\over
n_{[0,1]^{p}}}\sum_{j=1}^{n}I\left(\bm{X}_{\bm{s}_{j},A_{n}}\in[0,1]^{p}\right)\bar{K}_{h}\left(\bm{u},{\bm{s}_{j}\over
A_{n}}\right)K_{h}\left(x_{\ell},X_{\bm{s}_{j},A_{n}}^{\ell}\right)K_{h}\left(x_{k},X_{\bm{s}_{j},A_{n}}^{k}\right),$
$\displaystyle\widehat{m}_{\ell}(\bm{u},x_{\ell})$ $\displaystyle={1\over
n_{[0,1]^{p}}}\sum_{j=1}^{n}I\left(\bm{X}_{\bm{s}_{j},A_{n}}\in[0,1]^{p}\right)\bar{K}_{h}\left(\bm{u},{\bm{s}_{j}\over
A_{n}}\right)K_{h}\left(x_{\ell},X_{\bm{s}_{j},A_{n}}^{\ell}\right)Y_{\bm{s}_{j},A_{n}}/\widehat{p}_{\ell}(\bm{u},x_{\ell}).$

Moreover, the estimate $\widetilde{m}_{0}(\bm{u})$ of the model constant at
space point $\bm{u}$ is expressed as

$\widetilde{m}_{0}(\bm{u})={1\over
n_{[0,1]^{p}}}\sum_{j=1}^{n}I\left(\bm{X}_{\bm{s}_{j},A_{n}}\in[0,1]^{p}\right)\bar{K}_{h}\left(\bm{u},{\bm{s}_{j}\over
A_{n}}\right)Y_{\bm{s}_{j},A_{n}}/\bar{f}_{\bm{S}}(\bm{u}),$

where $\bar{f}_{\bm{S}}(\bm{u})={1\over
n_{[0,1]^{p}}}\sum_{j=1}^{n}I\left(\bm{X}_{\bm{s}_{j},A_{n}}\in[0,1]^{p}\right)\bar{K}_{h}\left(\bm{u},{\bm{s}_{j}\over
A_{n}}\right)$.

###  4.2. Asymptotic properties of estimators

Now we present the uniform convergence rate and joint asymptotic normality of
the estimators. Before describing the results, we summarize the set of
assumptions made.

######  Assumption 4.1.

Let $a_{n}=\sqrt{{\log n}\over nh^{d+1}}$ and $r=\min\\{\rho,1\\}$, where
$\rho$ is a positive constant that appears in Assumption 3.1. As $n\to\infty$,

  1. (Rb1)

$h^{-(d+1)}a_{n}^{d+1}A_{n}^{d}A_{1,n}^{-d}\beta(A_{2,n};A_{n}^{d})\to 0$ and
$A_{1,n}^{d}A_{n}^{-d}nh^{d+1}(\log n)\to 0$,

  2. (Rb2)

${n^{1/2}h^{(d+1)/2}\over A_{1,n}^{d}n^{1/\zeta}}\geq C_{0}n^{\eta}$ for some
$C_{0}>0$ and $\eta>0$,

  3. (Rb3)

$A_{n}^{dr}h^{3}\to\infty$ and $A_{n}^{dr/(1+r)}h^{2}\to\infty$,

  4. (Rb4)

$nh^{d+4}\to\infty$.

  5. (Rb5)

(Ra3) in Assumption 3.6 holds with $p=1$,

where $\zeta$ is a positive constant that appears in Assumption 3.3.

The following results are extension of Theorems 5.1 and 5.2 in Vogt (2012) to
locally stationary random fields with irregularly spaced observations.

######  Theorem 4.1.

Let $I_{h,0}=[2C_{1}h,1-2C_{1}h]$ and $I_{h}=[2C_{1}h,1-2C_{1}h]^{d}$. Suppose
that $\inf_{\bm{u}\in[0,1]^{d},\bm{x}\in[0,1]^{p}}f(\bm{u},\bm{x})>0$ and
$\inf_{\bm{u}\in[0,1]^{d}}f_{\bm{S}}(\bm{u})>0$. Then, under Assumptions 2.1,
3.1, 3.2, 3.5 (with $W_{\bm{s}_{j},A_{n}}=1$ and
$\epsilon_{\bm{s}_{j},A_{n}}$) and 4.1, the following result holds for
$P_{\bm{S}}$ almost surely:

$\displaystyle\sup_{\bm{u}\in I_{h},x_{\ell}\in
I_{h,0}}\left|\widetilde{m}_{\ell}(\bm{u},x_{\ell})-m_{\ell}(\bm{u},x_{\ell})\right|=O_{P_{\cdot|\bm{S}}}\left(\sqrt{{\log
n\over nh^{d+1}}}+h^{2}\right),\ \ell=1,\ldots,p.$

As is the case with general regression function, the asymptotic normality of
the kernel estimators for additive functions can be established under
$\alpha$-mixing.

######  Theorem 4.2.

Suppose that $\inf_{\bm{u}\in[0,1]^{d},\bm{x}\in[0,1]^{p}}f(\bm{u},\bm{x})>0$
and $\inf_{\bm{u}\in[0,1]^{d}}f_{\bm{S}}(\bm{u})>0$. Moreover, suppose that
$\epsilon_{\bm{s}_{j},A_{n}}$ given $\bm{X}_{\bm{s}_{j},A_{n}}$ are i.i.d.
random variables and continuous functions $\sigma_{\ell}(\bm{u},x_{\ell})$,
$1\leq\ell\leq p$ exist such that $\sigma^{2}_{\ell}({\bm{s}_{j}\over
A_{n}},x_{\ell})=E_{\cdot|\bm{S}}[\epsilon_{\bm{s}_{j},A_{n}}^{2}|X_{\bm{s}_{j},A_{n}}^{\ell}=x_{\ell}]$
and that $n_{[0,1]^{p}}h^{d+5}-c_{0}=o_{P_{\cdot|S}}(1)$ for a constant
$c_{0}$, $P_{\bm{S}}$ almost surely. Then, under Assumptions 2.1, 3.1, 3.2,
3.5 and 4.1 with $W_{\bm{s}_{j},A_{n}}=1$ and $\epsilon_{\bm{s}_{j},A_{n}}$ in
Assumption 3.5, for any $\bm{u}\in(0,1)^{d}$, $x_{1},\ldots,x_{p}\in(0,1)$,
the following result holds for $P_{\bm{S}}$ almost surely:

$\displaystyle\sqrt{n_{[0,1]^{p}}h^{d+1}}\left(\begin{array}[]{c}\widetilde{m}_{1}(\bm{u},x_{1})-m_{1}(\bm{u},x_{1})\\\
\vdots\\\
\widetilde{m}_{p}(\bm{u},x_{p})-m_{p}(\bm{u},x_{p})\end{array}\right)$
$\displaystyle\stackrel{{\scriptstyle
d}}{{\longrightarrow}}N(B_{\bm{u},\bm{x}},V_{\bm{u},\bm{x}}).$

Here,
$V_{\bm{u},\bm{x}}=\text{diag}(v_{1}(\bm{u},x_{1}),\ldots,v_{p}(\bm{u},x_{p}))$
is a $p\times p$ diagonal matrix with

$v_{\ell}(\bm{u},x_{\ell})=\kappa_{0}^{d+1}\sigma_{\ell}^{2}(\bm{u},x_{\ell})/(f_{\bm{S}}(\bm{u})p_{\ell}(\bm{u},x_{\ell})),$

where $\kappa_{0}=\int_{\mathbb{R}}K^{2}(x)dx$. $B_{\bm{u},\bm{x}}$ has of the
form

$B_{\bm{u},\bm{x}}=\sqrt{c_{0}}(\beta_{1}(\bm{u},x_{1})-\gamma_{1}(\bm{u}),\ldots,\beta_{p}(\bm{u},x_{p})-\gamma_{p}(\bm{u}))^{\prime}.$

The functions $\beta_{\ell}(\bm{u},\cdot)$ are defined as the minimizer of the
problem

$\int_{\mathbb{R}^{p}}\left\\{\beta(\bm{u},\bm{x})-\left(b_{0}+\sum_{\ell=1}^{p}b_{\ell}(x_{\ell})\right)\right\\}^{2}p(\bm{u},\bm{x})d\bm{x},$

where the minimization runs over all additive functions
$b(x)=b_{0}+\sum_{\ell=1}^{p}b_{\ell}(x_{\ell})$ with
$\int_{\mathbb{R}}b_{\ell}(x_{\ell})p_{\ell}(\bm{u},x_{\ell})dx_{\ell}=0$; the
function $\beta(\bm{u},\bm{x})$ is provided in Lemma B.6 in Appendix.
$\gamma_{\ell}(\bm{u})$ can be characterized by the equation
$\int_{\mathbb{R}}\alpha_{\ell}(\bm{u},x_{\ell})\widehat{p}_{\ell}(\bm{u},x_{\ell})dx_{\ell}=h^{2}\gamma_{\ell}(\bm{u})+o_{P_{\cdot|\bm{S}}}(h^{2})$,
where $\alpha_{\ell}(\bm{u},x_{\ell})$ are defined in Lemma B.6.

The same result holds true even if we replace the conditions on $\beta$-mixing
coefficients in Assumptions 3.1, 3.5 and 4.1 with those on $\alpha$-mixing
coefficients defined by (2.5).

######  Remark 4.2.

Set $A_{n}^{d}={n^{1-\bar{\eta}_{1}}\over(\log n)^{\bar{\eta}_{2}}}$ for some
$\bar{\eta}_{1}\in[0,1)$ and $\bar{\eta}_{2}\geq 1$,
$A_{1,n}=O(A_{n}^{\gamma_{A_{1}}})$ with $\gamma_{A_{1}}\in(0,1/3)$ and $r=1$.
Assume that we can take a sufficiently large $\zeta>2$. In this case,
Assumption 4.1 is satisfied for $d\geq 1$ and $p\geq 1$.

##  5\. Examples of locally stationary random fields on $\mathbb{R}^{d}$

In this section, we present examples of locally stationary random fields. In
particular, we discuss Lévy-driven moving average (MA) random fields, which
represent a wide class of random fields. One of the important features of this
class is that it includes non-Gaussian random fields in addition to Gaussian
random fields. Moreover, we provide sufficient conditions to demonstrate that
a random field is locally stationary and can be approximated by a stationary
Lévy-driven MA random field. We focus on the case of $p=1$ to simplify our
discussion. The results in this section can be extended to the multivariate
case ($p\geq 2$). See Appendix D herein for the definition of multivariate
Lévy-driven MA random fields on $\mathbb{R}^{d}$ and for a discussion on their
properties.

###  5.1. Lévy-driven MA random fields

Brockwell and Matsuda (2017) generalized CARMA($p,q$) processes on
$\mathbb{R}$ to CARMA ($p,q$) random fields on $\mathbb{R}^{d}$, which is a
special class of Lévy-driven MA random fields. See also Brockwell (2000),
Brockwell (2001), Marquardt and Stelzer (2007) and Schlemm and Stelzer (2012)
for examples of the CARMA process on $\mathbb{R}$ and Matsuda and Yuan (2020)
for multivariate extension of CARMA random fields on $\mathbb{R}^{2}$ and
their parametric inference. Let $L=\\{L(A):A\in\mathcal{B}(\mathbb{R}^{d})\\}$
be an infinitely divisible random measure on some probability space
$(\Omega,\mathcal{A},P)$, i.e., a random measure such that

  * 1.

for each sequence $(E_{m})_{m\in\mathbb{N}}$ of disjoint sets in
$\mathcal{B}(\mathbb{R}^{d})$,

    * (a)

$L(\cup_{m=1}^{\infty}E_{m})=\sum_{m=1}^{\infty}L(E_{m})$ a.s. whenever
$\cup_{m=1}^{\infty}E_{m}\in\mathcal{B}(\mathbb{R}^{d})$,

    * (b)

$(L(E_{m}))_{m\in\mathbb{N}}$ is a sequence of independent random variables.

  * 2.

the random variable $L(A)$ has an infinitely divisible distribution for any
$A\in\mathcal{B}(\mathbb{R}^{d})$.

The characteristic function of $L(A)$ which will be denoted by
$\varphi_{L(A)}(t)$, has a Lévy–Khintchine representation of the form
$\varphi_{L(A)}(t)=\exp\left(|A|\psi(t)\right)$ with

$\displaystyle\psi(t)$ $\displaystyle=it\gamma_{0}-{1\over
2}t^{2}\sigma_{0}+\int_{\mathbb{R}}\left\\{e^{itx}-1-itxI(x\in[-1,1])\right\\}\nu_{0}(x)dx$

where $i=\sqrt{-1}$, $\gamma_{0}\in\mathbb{R}$, $0\leq\sigma_{0}<\infty$,
$\nu_{0}$ is a Lévy density with
$\int_{\mathbb{R}}\min\\{1,x^{2}\\}\nu_{0}(x)dx<\infty$, and $|A|$ is the
Lebesgue measure of $A$. The triplet $(\gamma_{0},\sigma_{0},\nu_{0})$ is
called the Lévy characteristic of $L$; it uniquely determines the distribution
of random measure $L$. We refer to Sato (1999) and Bertoin (1996) as standard
references on Lévy processes and Rajput and Rosinski (1989) for details on the
theory of infinitely divisible measures and fields. Let
$a(z)=z^{p}+a_{1}z^{p-1}+\cdots+a_{p}=\prod_{i=1}^{p}(z-\lambda_{i})$ be a
polynomial of degree $p$ with real coefficients and distinct negative zeros
$\lambda_{1},\ldots,\lambda_{p}$, and let
$b(z)=b_{0}+b_{1}z+\cdots+b_{q}z^{q}=\prod_{i=1}^{q}(z-\xi_{i})$ be a
polynomial with real coefficients and real zeros $\xi_{1},\ldots,\xi_{q}$ such
that $b_{q}=1$ and $0\leq q<p$ and $\lambda_{i}^{2}\neq\xi_{j}^{2}$ for all
$i$ and $j$. Define $a(z)=\prod_{i=1}^{p}(z^{2}-\lambda_{i}^{2})$ and
$b(z)=\prod_{i=1}^{q}(z^{2}-\xi_{i}^{2})$. A Lévy-driven MA random field
driven by an infinitely divisible random measure $L$, which we call Lévy
random measure, is defined by

$\displaystyle X(\bm{s})=\int_{\mathbb{R}^{d}}g(\bm{s}-\bm{v})L(d\bm{v})$
(5.1)

for every $\bm{s}\in\mathbb{R}^{d}$. In particular, when $g(\cdot)$ is a
kernel function of the form

$\displaystyle g(\bm{s})=\sum_{i=1}^{p}{b(\lambda_{i})\over
a^{\prime}(\lambda_{i})}e^{\lambda_{i}\|\bm{s}\|},$ (5.2)

where $a^{\prime}$ denotes the derivative of the polynomial $a$, $X(\bm{s})$
is a univariate (isotropic) CARMA($p,q$) random field.

######  Remark 5.1 (Connections to SPDEs).

Berger (2020) characterizes (5.1) as a solution of a (fractional) stochastic
partial differential equation (SPDE). The author also extends the concept of
CARMA($p,q$) random fields as a strictly stationary solution of an SPDE. The
uniqueness of the solution is discussed in Berger (2019).

###  5.2. Stationary distribution of Lévy-driven MA random fields

When the Lévy-driven MA random field (5.1) is strictly stationary, the
characteristic function of the stationary distribution of $X$ is expressed as

$\displaystyle\varphi_{X(\bm{0})}(t)$
$\displaystyle=E[e^{itX(\bm{0})}]=\exp\left(\int_{\mathbb{R}^{d}}H(tg(\bm{s}))d\bm{s}\right),$
(5.3)

where

$\displaystyle\int_{\mathbb{R}^{d}}H(tg(\bm{s}))d\bm{s}$
$\displaystyle=it\gamma_{1}-{1\over
2}t^{2}\sigma_{1}+\int_{\mathbb{R}}\left\\{e^{itx}-1-itxI(x\in[-1,1])\right\\}\nu_{1}(x)dx,$

with

$\displaystyle\gamma_{1}$
$\displaystyle=\int_{\mathbb{R}^{d}}U(g(\bm{s}))d\bm{s},\
\sigma_{1}=\sigma_{0}\int_{\mathbb{R}^{d}}g^{2}(\bm{s})d\bm{s},\
\nu_{1}(x)=\int_{S_{g}}{1\over|g(\bm{s})|}\nu_{0}\left({x\over
g(\bm{s})}\right)d\bm{s}.$

Here, $S_{g}=\operatorname{supp}(g)=\\{\bm{s}\in\mathbb{R}^{d}:g(\bm{s})\neq
0\\}$ denotes the support of $g$; the function $U$ is defined as follows:

$\displaystyle U(u)$
$\displaystyle=u\left(\gamma_{0}+\int_{\mathbb{R}}x\left\\{I(ux\in[-1,1])-I(x\in[-1,1])\right\\}\nu_{0}(x)dx\right).$

The triplet $(\gamma_{1},\sigma_{1},\nu_{1})$ is again referred to as the Lévy
characteristic of $X(\bm{0})$ and determines the distribution of $X(\bm{0})$
uniquely. See Karcher et al. (2019) for details. The representation of (5.3)
implies that the stationary distribution of $X(\bm{s})$ has a density function
when the Lévy random measure $L$ is Gaussian, i.e.,
$(\gamma_{0},\sigma_{0},\nu_{0})=(\gamma_{0},\sigma_{0},0)$. When $L$ is
purely non-Gaussian, i.e.,
$(\gamma_{0},\sigma_{0},\nu_{0})=(\gamma_{0},0,\nu_{0})$, we can demonstrate
that $X(\bm{s})$ has a bounded continuous, infinitely differentiable density
if

$\displaystyle\nu_{0}(x)$
$\displaystyle={c_{\beta}\over|x|^{1+\beta}}g_{0}(x),\ x\neq 0$ (5.4)

where $c_{\beta}>0$, $g_{0}$ is a positive, continuous, and bounded function
on $\mathbb{R}$ with $\lim_{|x|\downarrow 0}g_{0}(x)=1$ and $\beta\in(0,1)$.
In fact, we can verify that a constant $\widetilde{C}>0$ exists such that

$\displaystyle
u^{2}\int_{\mathbb{R}^{d}}g^{2}(\bm{v})\int_{|x|\leq{1\over|g(\bm{v})||u|}}x^{2}\nu_{0}(x)dxd\bm{v}\geq\widetilde{C}|u|^{2-\alpha}$
(5.5)

for any $|u|\geq c_{0}>1$, where $\alpha=2-\beta\in(0,2)$. This implies the
existence of the density function of $X(\bm{s})$. See Lemma B.9 for the proof
of (5.5) and Lemma D.1 for the sufficient condition for the existence of the
density of $\bm{X}(\bm{s})$ with a purely non-Gaussian Lévy random measure in
a multivariate case. A Lévy process with a Lévy density of the form (5.4) is
called a locally stable Lévy process and it can represent important Lévy
processes such as tempered stable and normal inverse Gaussian processes (cf.
Section 2 in Masuda (2019), Section 5 in Kato and Kurisu (2020) and Kurisu et
al. (2021)).

###  5.3. Locally stationary Lévy-driven MA random fields

Next, we present examples of locally stationary Lévy-driven MA random fields.
Consider the following processes

$\displaystyle X_{\bm{s},A_{n}}$
$\displaystyle=\int_{\mathbb{R}^{d}}g\left({\bm{s}\over
A_{n}},\|\bm{s}-\bm{v}\|\right)L(d\bm{v})\ \text{and}\
X_{\bm{u}}(\bm{s})=\int_{\mathbb{R}^{d}}g\left(\bm{u},\|\bm{s}-\bm{v}\|\right)L(d\bm{v}),$

where $g:[0,1]^{d}\times[0,\infty)\to\mathbb{R}$ is a bounded function such
that $|g(\bm{u},\cdot)-g(\bm{v},\cdot)|\leq C\|\bm{u}-\bm{v}\|\bar{g}(\cdot)$
with $C<\infty$ and for any $\bm{u}\in[0,1]^{d}$,
$\int_{\mathbb{R}^{d}}(|g(\bm{u},\|\bm{s}\|)|+|\bar{g}(\bm{s})|)d\bm{s}<\infty$
and
$\int_{\mathbb{R}^{d}}(g^{2}(\bm{u},\|\bm{s}\|)+\bar{g}^{2}(\bm{s}))d\bm{s}<\infty$.
Notably, $X_{\bm{u}}(\bm{s})$ is a strictly stationary random field for each
$\bm{u}$. If $E[|L(A)|^{q^{\prime}}]<\infty$ for any
$A\in\mathcal{B}(\mathbb{R}^{d})$ with a bounded Lebesgue measure $|A|$ and
for $q^{\prime}=1,2$, we have that

$E[X_{\bm{u}}(\bm{s})]=\mu_{0}\int_{\mathbb{R}^{d}}g\left(\bm{u},\|\bm{s}\|\right)d\bm{s}\
\text{and}\
E\left[(\bm{X}_{\bm{u}}(\bm{s}))^{2}\right]=\bar{\sigma}_{0}^{2}\int_{\mathbb{R}^{d}}g^{2}\left(\bm{u},\|\bm{s}\|\right)d\bm{s},$

where $\mu_{0}=-i\psi^{\prime}(0)$ and
$\bar{\sigma}_{0}^{2}=-\psi^{\prime\prime}(0)$. Let $m>0$. Define the function
$\iota(\cdot:m):[0,\infty)\to[0,1]$ as

$\iota(x:m)=\begin{cases}1&\text{if $0\leq x\leq{m\over 2}$},\\\ -{2\over
m}x+2&\text{if ${m\over 2}<x\leq m$},\\\ 0&\text{if $m<x$}\end{cases}$

and consider the process
$X_{\bm{u}}(\bm{s}:A_{2,n})=\int_{\mathbb{R}^{d}}g\left(\bm{u},\|\bm{s}-\bm{v}\|\right)\iota(\|\bm{s}-\bm{v}\|:A_{2,n})L(d\bm{v})$.
Notably, $X_{\bm{u}}(\bm{s}:A_{2,n})$ is also an $2A_{2,n}$-dependent strictly
stationary random field, i.e., the $\beta$-mixing coefficients
$\beta(a;b)=\beta_{1}(a)g_{1}(b)$ of $X_{\bm{u}}(\bm{s}:A_{2,n})$ satisfy
$\beta_{1}(a)=0$ for $a\geq 2A_{2,n}$. Observe that

$\displaystyle\left|X_{\bm{s},A_{n}}-X_{\bm{u}}(\bm{s}:A_{2,n})\right|$
$\displaystyle\leq\left|X_{\bm{s},A_{n}}-X_{\bm{u}}(\bm{s})\right|+\left|X_{\bm{u}}(\bm{s})-X_{\bm{u}}(\bm{s}:A_{2,n})\right|$
$\displaystyle\leq\int_{\mathbb{R}^{d}}\left|g\left({\bm{s}\over
A_{n}},\|\bm{s}-\bm{v}\|\right)-g\left(\bm{u},\|\bm{s}-\bm{v}\|\right)\right||L(d\bm{v})|$
$\displaystyle\quad+{1\over
A_{n}^{d}}\int_{\mathbb{R}^{d}}A_{n}^{d}|g\left(\bm{u},\|\bm{s}-\bm{v}\|\right)|(1-\iota(\|\bm{s}-\bm{v}\|:A_{2,n}))|L(d\bm{v})|$
$\displaystyle\leq\left\|{\bm{s}\over
A_{n}}-\bm{u}\right\|\int_{\mathbb{R}^{d}}C\bar{g}(\|\bm{s}-\bm{v}\|)|L(d\bm{v})|$
$\displaystyle\quad+{1\over
A_{n}^{d}}\int_{\mathbb{R}^{d}}A_{n}^{d}\underbrace{|g\left(\bm{u},\|\bm{s}-\bm{v}\|\right)|(1-\iota(\|\bm{s}-\bm{v}\|:A_{2,n}))}_{=:g_{\bm{u}}(\|\bm{s}-\bm{v}\|:A_{2,n})}|L(d\bm{v})|$
$\displaystyle\leq\left(\left\|{\bm{s}\over A_{n}}-\bm{u}\right\|+{1\over
A_{n}^{d}}\right)\int_{\mathbb{R}^{d}}(\underbrace{C\bar{g}(\|\bm{s}-\bm{v}\|)+A_{n}^{d}g_{\bm{u}}(\|\bm{s}-\bm{v}\|:A_{2,n})}_{=:g_{\bm{u},A_{n},A_{2,n}}(\|\bm{s}-\bm{v}\|)})|L(d\bm{v})|$
$\displaystyle=:\left(\left\|{\bm{s}\over A_{n}}-\bm{u}\right\|+{1\over
A_{n}^{d}}\right)U_{\bm{s},A_{n}}(\bm{u}).$ (5.6)

Here, we define $|L(A)|$ as the absolute value of the random variable $L(A)$
for any $A\in\mathcal{B}(\mathbb{R}^{d})$. If we assume that

$\displaystyle\sup_{n\geq
1}\sup_{\bm{u}\in[0,1]^{d}}\int_{\mathbb{R}^{d}}(A_{n}^{d}g_{\bm{u}}(\|\bm{s}\|:A_{2,n})+A_{n}^{2d}g_{\bm{u}}^{2}(\|\bm{s}\|:A_{2,n}))d\bm{s}<\infty,$
(5.7)

we obtain

$\displaystyle E[U_{\bm{s},A_{n}}^{2}(\bm{u})]$ $\displaystyle\leq
C\sup_{n\geq
1}\sup_{\bm{u}\in[0,1]^{d}}\left(\int_{\mathbb{R}^{d}}g_{\bm{u},A_{n},A_{2,n}}^{2}(\|\bm{s}\|)d\bm{s}+\left(\int_{\mathbb{R}^{d}}g_{\bm{u},A_{n},A_{2,n}}(\|\bm{s}\|)d\bm{s}\right)^{2}\right)<\infty.$
(5.8)

See Brockwell and Matsuda (2017) for details regarding the computation of the
moments of Lévy-driven MA random fields. Note that (5.6) and (5.8) imply that
$X_{\bm{s},A_{n}}$ is a locally stationary random field.

######  Remark 5.2.

We also note that (5.6) and (5.8) imply that $X_{\bm{s},A_{n}}$ can be
approximated by an $A_{2,n}$-dependent stationary random field under the
condition (5.7). In this case, the random field $X_{\bm{s},A_{n}}$ is an
approximately $A_{2,n}$-dependent locally stationary random field. Let
$c_{1},\ldots,c_{p_{1}}$ be positive constants and
$r_{1}(\cdot),\ldots,r_{p_{1}}(\cdot)$ be continuous functions on $[0,1]^{d}$
such that $|r_{k}(\bm{u})-r_{k}(\bm{v})|\leq C\|\bm{u}-\bm{v}\|$, $1\leq k\leq
p_{1}$ with $C<\infty$. Consider the function

$\displaystyle g(\bm{u},\|\bm{s}\|)$
$\displaystyle=\sum_{k=1}^{p_{1}}r_{k}\left(\bm{u}\right)e^{-c_{k}\|\bm{s}\|}.$
(5.9)

We can demonstrate that a Lévy-driven MA random field with the kernel function
(5.9) satisfies (5.6); furthermore, locally stationary random fields include a
wide class of CARMA-type random fields. For a discussion on general
multivariate cases and examples that satisfy our regularity conditions, see
also Appendix D herein.

Next, we introduce the concept of approximately $m_{n}$-dependence for locally
stationary random fields.

######  Definition 5.1 (Approximately $m_{n}$-dependence for locally
stationary random field).

Let $m_{n}$ be a sequence of positive constants with $m_{n}\to\infty$ as
$n\to\infty$. We define a locally stationary random field
$\bm{X}_{A_{n}}=\\{\bm{X}_{\bm{s},A_{n}}=(X^{1}_{\bm{s},A_{n}},\ldots,X^{p}_{\bm{s},A_{n}})^{\prime}:\bm{s}\in
R_{n}\\}$ in $\mathbb{R}^{p}$ to be an approximately $m_{n}$-dependent random
field if $\bm{X}_{A_{n}}$ can be represented as a sum of $m_{n}$-dependent
random field $\\{\bm{X}_{\bm{s},A_{n}:m_{n}}\\}$ and the “residual” random
field
$\\{\bm{\epsilon}_{\bm{s},A_{n}:m_{n}}=(\epsilon_{\bm{s},A_{n}:m_{n}}^{1},\ldots,\epsilon_{\bm{s},A_{n}:m_{n}}^{p})^{\prime}\\}$,
which satisfies the following condition:

  1. (Ma0)

For some $q>1$ such that ${q\zeta\over\zeta-1}$ is an even integer,

$\displaystyle\max_{1\leq j\leq n}\max_{1\leq\ell\leq
p}E_{\cdot|\bm{S}}\left[\left|\epsilon_{\bm{s}_{j},A_{n}:m_{n}}^{\ell}\right|^{{q\zeta\over\zeta-1}}\right]^{{\zeta-1\over
q\zeta}}\leq\gamma_{\epsilon}(m_{n}),\ \text{$P_{\bm{S}}$-a.s.}$

where $\zeta>2$ is the constant in Assumption 3.3 and
$\gamma_{\epsilon}(\cdot)$ is a decreasing function such that for some
$\eta_{2}\in(0,1)$

$\displaystyle{\gamma_{\epsilon}(m_{n})(n+A_{n}^{d}\log n)\over
n^{(1-\eta_{2})\over 2}h^{{d+p\over 2}+1}}\to 0\ \text{as $n\to\infty$}.$
(5.10)

We can show that a univariate locally stationary CARMA-type random field with
an exponential decay kernel such as (5.9) and a Lévy random measure that has
finite moments up to the $q\zeta/(\zeta-1)$th moment, where $q\zeta/(\zeta-1)$
is an even integer, i.e., $E[|L([0,1]^{d})|^{q^{\prime}}]<\infty$ for $1\leq
q^{\prime}\leq q\zeta/(\zeta-1)$, satisfies Condition (Ma0) with

$\displaystyle\gamma_{\epsilon}(x)\leq Cx^{{(d-1)(\zeta-1)\over
q\zeta}}e^{-{c_{0}x\over 2}}$ (5.11)

for a constant $C<\infty$, where $c_{0}=\min_{1\leq\ell\leq p_{1}}c_{\ell}$.
This implies (5.10). In this case, the CARMA-type process is approximately
$D(\log n)$-dependent for a sufficiently large $D>0$. See Lemma B.8 for the
proof of (5.11). See also Appendix D for a discussion on multivariate cases.

######  Remark 5.3.

We introduced the notion of approximately $m_{n}$-dependent random fields to
give some examples of random fields that satisfy our assumptions. For this, it
could be possible to adopt the $m$-dependence approximation technique in
Machkouri et al. (2013) and Machkouri (2014), for example. On the other hand,
the proofs of our results are based on a general blocking technique designed
for irregularly spaced sampling sites. Moreover, to derive the uniform
convergence rate of the kernel estimator, we need to care about the effect of
non-equidistant sampling sites when applying a maximal inequality, which
requires additional work compared with the case that sampling sites are
equidistant. See also Section 2.4 for a discussion of mixing conditions.

Now we summarize our discussion in this subsection.

######  Assumption 5.1.

  1. (Ma1)

The process $\\{\bm{X}_{\bm{s},A_{n}}\\}$ is an approximately
$A_{2,n}$-dependent random field. Hence, $\bm{X}_{\bm{s},A_{n}}$ can be
decomposed into a sum of $A_{2,n}$-dependent random fields
$\\{\bm{X}_{\bm{s},A_{n}:A_{2,n}}\\}$ and residual random fields
$\\{\bm{\epsilon}_{\bm{s},A_{n}:A_{2,n}}\\}$, which satisfies Condition (Ma0).

  2. (Ma2)

The process $\\{\bm{X}_{\bm{s},A_{n}}\\}$ is an approximately
$A_{2,n}$-dependent locally stationary random field. Therefore, for each space
point $\bm{u}\in[0,1]^{d}$, an $A_{2,n}$-dependent strictly stationary random
field $\\{\bm{X}_{\bm{u}}(\bm{s}:A_{2,n})\\}$ exists such that

$\|\bm{X}_{\bm{s},A_{n}}-\bm{X}_{\bm{u}}(\bm{s}:A_{2,n})\|\leq\left(\left\|{\bm{s}\over
A_{n}}-\bm{u}\right\|+{1\over
A_{n}^{d}}\right)U_{\bm{s},A_{n}:A_{2,n}}(\bm{u})\ \text{a.s.}$

with $E[(U_{\bm{s},A_{n}:A_{2,n}}(\bm{u}))^{\rho}]\leq C$ for some $\rho>0$.

  3. (Ma3)

(M2) and (M4) in Assumption 3.1 hold by replacing $\bm{X}_{\bm{u}}(\bm{s})$
with $\bm{X}_{\bm{u}}(\bm{s}:A_{2,n})$.

  4. (Ma4)

(M3) and (M5) in Assumption 3.1 hold.

######  Corollary 5.1.

Proposition 3.1, Theorems 3.1, 3.2 holds even if we replace Assumption 3.1
with Assumption 5.1.

######  Corollary 5.2.

Theorems 4.1 and 4.2 hold even if we replace Assumption 3.1 with Assumption
5.1 (with $p=1$ in (5.10)).

######  Remark 5.4.

Let $\widetilde{\beta}(a;b)=\widetilde{\beta}_{1}(a)\widetilde{g}_{1}(b)$
denote the $\beta$-mixing coefficients of $\bm{X}_{\bm{s},A_{n}:A_{2,n}}$.
When the process $\\{\bm{X}_{\bm{s},A_{n}}\\}$ is an approximately
$A_{2,n}$-dependent random field and an approximately $A_{2,n}$-dependent
locally stationary random field, we can replace $\bm{X}_{\bm{s},A_{n}}$ with
$\bm{X}_{\bm{s},A_{n}:A_{2,n}}$ in our analysis. In this case, because the
$\beta$-mixing coefficients of $\bm{X}_{\bm{s},A_{n}:A_{2,n}}$ satisfy
$\widetilde{\beta}(A_{2,n};A_{n}^{d})=\widetilde{\beta}_{1}(A_{2,n})\widetilde{g}_{1}(A_{n}^{d})=0$,
conditions on $\beta$-mixing coefficients (R1), (Ra3), (Rb1) are satisfied
automatically. See also Remark A.3 in the Appendix.

Brockwell and Matsuda (2017) developed CARMA random fields and they discuss in
the introduction of their paper that even for CARMA random fields, which is a
special class of Lévy-driven moving average random fields, the random fields
are already generates a much rich class of random fields. Therefore, Lévy-
driven moving average random fields are one of the most flexible and concrete
class of random fields from both theoretical and practical point of view.
Moreover, it is difficult to compute $\alpha$\- or $\beta$-mixing coefficients
for non-Gaussian random fields on $\mathbb{R}^{d}$ and there would be no
general tools for the computation. The dependence structure of CARMA random
fields on $\mathbb{R}^{d}$ have not been studied although there are some
results on $\beta$-mixing properties of multivariate CARMA processes on
$\mathbb{R}$ in Schlemm and Stelzer (2012) for example. Indeed, existing
papers on statistical analysis of CARMA random fields on $\mathbb{R}^{d}$ (cf.
Brockwell and Matsuda (2017) and Matsuda and Yuan (2020)) consider parametric
estimation of second-order stationary CARMA random fields but their
($\alpha$\- or $\beta$-) mixing properties are not addressed. Hence it is
still an open question that CARMA random fields are $\beta$\- (or $\alpha$-)
mixing. Then, our results in Section 5 are important contributions in the
sense that our results in Sections 3 and 4 can be applied to Lévy-driven
moving average random fields that is a more flexible class of random fields
than CARMA random fields by providing sufficient conditions for
$m_{n}$-dependent approximation of locally stationary random fields and
showing that locally stationary CARMA-type random fields can be approximated
by $m_{n}$-dependent random fields.

##  6\. Discussion

In this section we discuss some directions of possible extensions of our
results and the model (1.1) for applications to spatio-temporal data.

###  6.1. Extensions

####  6.1.1. On sampling region

It is possible to extend the definition of the sampling region $R_{n}$ to a
more general case that includes nonstandard shapes. For example, we can adopt
the definition of sampling regions in Lahiri (2003b) as follows: First, we
define the sampling region $R_{n}$. Let $R_{0}^{\ast}$ be an open connected
subset of $(-2,2]^{d}$ containing $[-1,1]^{d}$ and let $R_{0}$ be a Borel set
satisfying $R_{0}^{\ast}\subset R_{0}\subset\overline{R}_{0}^{\ast}$, where
for any set $S\subset\mathbb{R}^{d}$, $\overline{S}$ denotes its closure. Let
$\\{A_{n}\\}_{n\geq 1}$ be a sequence of positive numbers such that
$A_{n}\to\infty$ as $n\to\infty$ and consider $R_{n}=A_{n}R_{0}$ as a sampling
region. We also assume that for any sequence of positive numbers
$\\{a_{n}\\}_{n\geq 1}$ with $a_{n}\to 0$ as $n\to\infty$, the number of cubes
of the form $a_{n}(\bm{i}+[0,1)^{d})$, $\bm{i}\in\mathbb{Z}^{d}$ with their
lower left corner $a_{n}\bm{i}$ on the lattice $a_{n}\mathbb{Z}^{d}$ that
intersect both $R_{0}$ and $R_{0}^{c}$ is $O(a_{n}^{-d+1})$ as $n\to\infty$.
Moreover, let $f$ be a continuous, everywhere positive probability density
function on $R_{0}$, and let $\\{\bm{S}_{0,i}\\}_{i\geq 1}$ be a sequence of
i.i.d. random vectors with density $f$. Assume that $\\{\bm{S}_{0,i}\\}_{i\geq
1}$ and $\bm{X}_{\bm{s},A_{n}}$ are independent.

The boundary condition on the prototype set $R_{0}$ holds in many practical
situations, including many convex subsets in $\mathbb{R}^{d}$, such as
spheres, ellipsoids, polyhedrons, as well as many nonconvex sets in
$\mathbb{R}^{d}$. See also Lahiri (2003b) and Chapter 12 in Lahiri (2003a) for
further discussion on the boundary condition.

Under this setting on the sampling region, our results hold under the same
assumptions and the proofs are the same. Furthermore, for any set
$S\subset\mathbb{R}^{d}$, let $\text{int}(S)=S\backslash\overline{S}$ be the
internal part of $S$ and for any $\delta>0$, let
$S^{\delta}=\\{\bm{s}\in\mathbb{R}^{d}:d(\bm{s},S)\leq\delta\\}$ where
$d(\bm{s},S)=\inf_{\bm{x}\in S}\|\bm{s}-\bm{x}\|$. Then by replacing
$[0,1]^{d}$ and $(0,1)^{d}$ that appear in Assumptions 2.1 and 3.1 with
$R_{0}$ and $\text{int}(R_{0})$, it is also possible to show uniform
convergence results that correspond to Proposition 3.1, Theorem 3.1 and
Theorem 4.1 over $(\bm{u},\bm{x})\in V_{0}\times S_{c}$, $V_{0,h}\times S_{c}$
and $V_{0,2h}\times I_{h,0}$, respectively. Here, $S_{c}$ is a compact subset
of $\mathbb{R}^{p}$, and $V_{0,h}$, $V_{0,2h}$ and $V_{0}$ are compact subsets
of $R_{0}$ such that $V_{0,2h}\subset V_{0,h}$ and
$V_{0,2h}^{2C_{1}h}=V_{0,h}^{C_{1}h}=V_{0}$. We omit the proofs since they are
almost the same as those of Proposition 3.1, Theorem 3.1 and Theorem 4.1.

####  6.1.2. Constructing confidence bands for regression functions

As an extension of Theorem 3.2, it is straightforward to show joint asymptotic
normality of $\widehat{m}$ over finite number of design points
$\\{(\bm{u}_{j},\bm{x}_{j})\\}_{j=1}^{L}$ and verify that
$\widehat{m}(\bm{u}_{j},\bm{x}_{j})$ are asymptotically independent. Building
on the result, it should be possible to construct simple confidence bands by
plug-in methods and linear interpolations of the following joint confidence
intervals: Let $\xi_{1},\ldots,\xi_{L}$ be i.i.d. standard normal random
variables, and let $q_{\tau}$ satisfy $P\left(\max_{1\leq j\leq
L}|\xi_{j}|>q_{\tau}\right)=\tau$ for $\tau\in(0,1)$ and take a bandwidth $h$
such that $h^{2}\ll 1/\sqrt{nh^{d+p}}$. Then,

$C_{j}(1-\tau)=\left[\widehat{m}(\bm{u}_{j},\bm{x}_{j})\pm\sqrt{{V_{\bm{u}_{j},\bm{x}_{j}}\over
nh^{d+p}}}q_{\tau}\right],\ j=1,\ldots,L$

are joint asymptotic $100(1-\tau)$% confidence intervals of $m$. Here, $[a\pm
b]=[a-b,a+b]$ for $a\in\mathbb{R}$ and $b>0$. More generally, there could be
two possible ways to construct confidence bands of the regression function.
The first way is based on a Gumbel approximation as considered in Zhao and Wu
(2008). The second way is based on intermediate (high-dimensional) Gaussian
approximations as considered in Horowitz and Lee (2012). However, we believe
that both approaches require additional substantial work and there would be no
previous studies on the construction of uniform confidence bands for locally
stationary time series or random fields. Therefore, we only explain the idea
of both approaches formally to keep the tight focus of the present paper and
leave the extension as a future research topic. The idea of both approaches is
as follows. Instead of constructing uniform confidence bands of the regression
function over $I_{h}$ and $S_{c}$, we first consider a discretized version of
$I_{h}$ and $S_{c}$ in Theorem 3.1 such that
$\\{\bm{u}_{n,j}\\}_{j=1}^{N_{u}}$, $\\{\bm{x}_{n,k}\\}_{k=1}^{N_{x}}$ where
$\bm{u}_{n,j}\in I_{h}$ and $\bm{x}_{n,k}\in S_{c}$ and the design points
$\\{\bm{u}_{n,j}\\}_{j=1}^{N_{u}}$, $\\{\bm{x}_{n,k}\\}_{k=1}^{N_{x}}$ are
asymptotically dense in $I_{h}$ and $S_{c}$, i.e. $N_{u},N_{x}\to\infty$,
$\max_{1\leq j_{1}\neq j_{2}\leq
N_{u},}\|\bm{u}_{n,j_{1}}-\bm{u}_{n,j_{2}}\|\to 0$ and $\max_{1\leq k_{1}\neq
k_{2}\leq N_{x},}\|\bm{x}_{n,k_{1}}-\bm{x}_{n,k_{2}}\|\to 0$ as $n\to\infty$.
Then we could construct “asymptotically” uniform confidence bands of the
regression function by linear interpolations of joint confidence intervals
over the design points combining Gumbel or intermediate Gaussian
approximations.

###  6.2. Further discussion on mixing conditions

Here we discuss further on the mixing conditions in this paper. See also
Section 2.4 for discussion on mixing conditions from another technical
aspects. Consider the following decomposition of a random field
$\bm{X}_{\bm{s},A_{n}}$:

$\displaystyle\bm{X}_{\bm{s},A_{n}}=\widetilde{\bm{X}}_{\bm{s},A_{n}}+\widetilde{\bm{\epsilon}}_{\bm{s},A_{n}},$
(6.1)

where $\widetilde{\bm{X}}_{\bm{s},A_{n}}$ is a locally stationary
$\beta$-mixing (for uniform estimation) or $\alpha$-mixing (for asymptotic
normality) random field that satisfies Assumptions 3.1, 3.5 and 3.6 (for
general case) or Assumptions 3.1, 3.5 and 4.1 (for additive models), and
$\widetilde{\bm{\epsilon}}_{\bm{s},A_{n}}=(\widetilde{\epsilon}^{1}_{\bm{s},A_{n}},\ldots,\widetilde{\epsilon}^{p}_{\bm{s},A_{n}})^{\prime}$
is a residual random field which is asymptotically negligible. The
decomposition (6.1) implies that $\widetilde{\bm{X}}_{\bm{s},A_{n}}$ is not
necessarily $m_{n}$-dependent. A similar argument in Remark A.3 in Appendix A
yields that to show the asymptotic negligibility of
$\widetilde{\bm{\epsilon}}_{\bm{s},A_{n}}$ for our results to hold, it is
sufficient to verify the following condition: For some $q>1$ such that
${q\zeta\over\zeta-1}$ is an even integer,

$\displaystyle\max_{1\leq j\leq n}\max_{1\leq\ell\leq
p}E_{\cdot|\bm{S}}\left[\left|\widetilde{\epsilon}_{\bm{s}_{j},A_{n}}^{\ell}\right|^{{q\zeta\over\zeta-1}}\right]^{{\zeta-1\over
q\zeta}}\leq\iota_{\widetilde{\epsilon}}(n),\ \text{$P_{\bm{S}}$-a.s.}$

where $\zeta>2$ is the constant in Assumption 3.3 and
$\iota_{\widetilde{\epsilon}}(\cdot)$ is a decreasing function such that for
some $\eta_{2}\in(0,1)$

$\displaystyle{\iota_{\widetilde{\epsilon}}(n)(n+A_{n}^{d}\log n)\over
n^{(1-\eta_{2})\over 2}h^{{d+p\over 2}+1}}\to 0\ \text{as $n\to\infty$}.$
(6.2)

As far as (6.2) is satisfied, the random field $\bm{X}_{\bm{s},A_{n}}$ itself
is not necessarily $\beta$-mixing (or $\alpha$-mixing) in general and
$\widetilde{\bm{\epsilon}}_{\bm{s},A_{n}}$ can accommodate complex dependence
structures of $\bm{X}_{\bm{s},A_{n}}$ that cannot be captured by
$\beta$-mixing (or $\alpha$-mixing) random fields. As a special case,
$\widetilde{\bm{X}}_{\bm{s},A_{n}}$ can be an approximately $m_{n}$-dependent
locally stationary random field and in this case
$\widetilde{\bm{X}}_{\bm{s},A_{n}}$ also can be decomposed into
$m_{n}$-dependent random field $\widetilde{\bm{X}}_{\bm{s},A_{n}:m_{n}}$ and a
residual random field $\widetilde{\bm{\epsilon}}_{\bm{s},A_{n}:m_{n}}$ that
corresponds to the approximation error. Then $\bm{X}_{\bm{s},A_{n}}$ can be
represented as follows:

$\bm{X}_{\bm{s},A_{n}}=\underbrace{\widetilde{\bm{X}}_{\bm{s},A_{n}:m_{n}}}_{\text{$m_{n}$-dependent}}+\underbrace{\widetilde{\bm{\epsilon}}_{\bm{s},A_{n}:m_{n}}}_{\text{approximation
error}}+\underbrace{\widetilde{\bm{\epsilon}}_{\bm{s},A_{n}}}_{\text{complexly
dependent}}.$

The Lévy-driven moving average random fields considered in Section 5 of the
revised manuscript are examples when
$\widetilde{\bm{\epsilon}}_{\bm{s},A_{n}}=\bm{0}$.

###  6.3. Spatio-temporal modeling

Consider a spatially locally stationary spatio-temporal data
$\\{\bm{X}_{\bm{s}_{j},A_{n}}:\bm{s}_{j}\in R_{n},\\}$ where
$\bm{X}_{\bm{s}_{j},A_{n}}=(X_{\bm{s}_{j},A_{n}}(t_{1}),\cdots,X_{\bm{s}_{j},A_{n}}(t_{p+1}))^{\prime}:\bm{s}\in
R_{n}\\}$, that is, at each location $\bm{s}_{j}$, $\bm{X}_{\bm{s}_{j},A_{n}}$
is a time series observed at time points $t_{1}<\ldots<t_{p+1}$ with $p\geq 1$
fixed. Our results can be applied to the following spatio-temporal model:

$\displaystyle X_{\bm{s}_{j},A_{n}}(t_{p+1})$
$\displaystyle=m\left({\bm{s}_{j}\over
A_{n}},X_{\bm{s}_{j},A_{n}}(t_{1}),\cdots,X_{\bm{s}_{j},A_{n}}(t_{p})\right)+\epsilon_{\bm{s}_{j},A_{n}},\
j=1,\ldots,n.$

In our framework, the time points can be non-equidistant and the process can
be temporally nonstationary since we consider finite number of time points.
Additionally, our sampling design is related to that of Altmeyer and Reiß
(2019). They study nonparametric estimation of a linear SPDE with a spatially-
varying coefficient under the asymptotic regime with fixed time horizon and
with the spatial resolution of the observations tending to zero.

##  Appendix A Proofs

###  A.1. Proofs for Section 3

###### Proof of Proposition 3.1.

Define $B=\\{(\bm{u},\bm{x})\in\mathbb{R}^{d+p}:\bm{u}\in[0,1]^{d},\bm{x}\in
S_{c}\\}$, $a_{n}=\sqrt{\log n/nh^{d+p}}$ and $\tau_{n}=\rho_{n}n^{1/\zeta}$
with $\rho_{n}=(\log n)^{\zeta_{0}}$ for some $\zeta_{0}>0$. Define

$\displaystyle\widehat{\psi}_{1}(\bm{u},\bm{x})$ $\displaystyle={1\over
nh^{p+d}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\prod_{\ell_{2}}^{p}K_{h}\left(x_{\ell_{2}}-X_{\bm{s}_{j},A_{n}}^{\ell_{2}}\right)W_{\bm{s}_{j},A_{n}}I(|W_{\bm{s}_{j},A_{n}}|\leq\tau_{n}),$
$\displaystyle\widehat{\psi}_{2}(\bm{u},\bm{x})$ $\displaystyle={1\over
nh^{p+d}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\prod_{\ell_{2}}^{p}K_{h}\left(x_{\ell_{2}}-X_{\bm{s}_{j},A_{n}}^{\ell_{2}}\right)W_{\bm{s}_{j},A_{n}}I(|W_{\bm{s}_{j},A_{n}}|>\tau_{n}).$

Note that
$\widehat{\psi}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{\psi}(\bm{u},\bm{x})]=\widehat{\psi}_{1}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{\psi}_{1}(\bm{u},\bm{x})]+\widehat{\psi}_{2}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{\psi}_{2}(\bm{u},\bm{x})]$.

(Step1) First we consider the term
$\widehat{\psi}_{2}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{\psi}_{2}(\bm{u},\bm{x})]$.

$\displaystyle P_{\cdot|\bm{S}}\left(\sup_{(\bm{u},\bm{x})\in
B}|\widehat{\psi}_{2}(\bm{u},\bm{x})|>Ca_{n}\right)$ $\displaystyle\leq
P_{\cdot|\bm{s}}\left(|W_{\bm{s}_{j},A_{n}}|>\tau_{n}\ \text{for some
$j=1,\ldots,n$}\right)$
$\displaystyle\leq\tau_{n}^{-\zeta}\sum_{j=1}^{n}E_{\cdot|\bm{S}}[|W_{\bm{s}_{j},A_{n}}|^{\zeta}]\leq
Cn\tau_{n}^{-\zeta}=\rho_{n}^{-\zeta}\to 0\ P_{\bm{S}}-a.s.$

$\displaystyle
E_{\cdot|\bm{S}}\left[|\widehat{\psi}_{2}(\bm{u},\bm{x})|\right]$
$\displaystyle\leq{1\over
nh^{d+p}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\int_{\mathbb{R}^{d}}\prod_{\ell=1}^{p}K_{h}(x_{\ell}-w_{\ell})$
$\displaystyle\quad\times
E_{\cdot|\bm{S}}[|W_{\bm{s}_{j},A_{n}}|I(|W_{\bm{s}_{j},A_{n}}|>\tau_{n})|X_{\bm{s}_{j},A_{n}}=\bm{w}]f_{\bm{X}_{\bm{s}_{j},A_{n}}}(\bm{w})d\bm{w}$
$\displaystyle={1\over
nh^{d}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\int_{\mathbb{R}^{d}}\prod_{\ell=1}^{p}K(\varphi_{\ell})$
$\displaystyle\quad\times
E_{\cdot|\bm{S}}[|W_{\bm{s}_{j},A_{n}}|I(|W_{\bm{s}_{j},A_{n}}|>\tau_{n})|X_{\bm{s}_{j},A_{n}}=\bm{x}-h\bm{\varphi}]f_{\bm{X}_{\bm{s}_{j},A_{n}}}(\bm{x}-h\bm{\varphi})d\bm{\varphi}$
$\displaystyle={1\over
nh^{d}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right){1\over\tau_{n}^{\zeta-1}}\int_{\mathbb{R}^{d}}\prod_{\ell=1}^{p}K(\varphi_{\ell})$
$\displaystyle\quad\times
E_{\cdot|\bm{S}}[|W_{\bm{s}_{j},A_{n}}|^{\zeta}I(|W_{\bm{s}_{j},A_{n}}|>\tau_{n})|X_{\bm{s}_{j},A_{n}}=\bm{x}-h\bm{\varphi}]f_{\bm{X}_{\bm{s}_{j},A_{n}}}(\bm{x}-h\bm{\varphi})d\bm{\varphi}$
$\displaystyle\leq{C\over\tau_{n}^{\zeta-1}}{1\over
nh^{d}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over A_{n}}\right)$
$\displaystyle={C\over\tau_{n}^{\zeta-1}}\left(f_{\bm{S}}(\bm{u})+O\left({\sqrt{\log
n\over
nh^{d}}}+h^{2}\right)\right)\leq{C\over\tau_{n}^{\zeta-1}}=C\rho_{n}^{-(\zeta-1)}n^{-(\zeta-1)/\zeta}\leq
Ca_{n}\ P_{\bm{S}}-a.s.$

In the last equation, we used Lemma C.3. As a result,

$\sup_{(\bm{u},\bm{x})\in
B}|\widehat{\psi}_{2}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{\psi}_{2}(\bm{u},\bm{x})]|=O_{P_{\cdot|\bm{S}}}(a_{n}).$

(Step2) Now we consider the term
$\widehat{\psi}_{1}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{\psi}_{1}(\bm{u},\bm{x})]$.
First we introduce some notations. For
$\bm{\ell}=(\ell_{1},\ldots,\ell_{d})^{\prime}\in\mathbb{Z}^{d}$, let
$\Gamma_{n}(\bm{\ell};\bm{0})=(\bm{\ell}+(0,1]^{d})A_{3,n}$ where
$A_{3,n}=A_{1,n}+A_{2,n}$ and divide $\Gamma_{n}(\bm{\ell};\bm{0})$ into
$2^{d}$ hypercubes as follows:

$\Gamma_{n}(\bm{\ell};\bm{\epsilon})=\prod_{j=1}^{d}I_{j}(\epsilon_{j}),\
\bm{\epsilon}=(\epsilon_{1},\ldots,\epsilon_{d})^{\prime}\in\\{1,2\\}^{d},$

where, for $1\leq j\leq d$,

$I_{j}(\epsilon_{j})=\begin{cases}(\ell_{j}\lambda_{3,n},\ell_{j}\lambda_{3,n}+\lambda_{1,n}]&\text{if
$\epsilon_{1}=1$},\\\
(\ell_{j}\lambda_{3,n}+\lambda_{1,n},(\ell_{j}+1)\lambda_{3,n}]&\text{if
$\epsilon_{1}=2$}.\end{cases}$

Note that

$\displaystyle|\Gamma_{n}(\bm{\ell};\bm{\epsilon})|=\lambda_{1,n}^{q(\bm{\epsilon})}\lambda_{2,n}^{d-q(\bm{\epsilon})}$
(A.1)

for any $\bm{\ell}\in\mathbb{Z}^{d}$ and $\bm{\epsilon}\in\\{1,2\\}^{d}$,
where $q(\bm{\epsilon})=[\\![\\{1\leq j\leq d:\epsilon_{j}=1\\}]\\!]$.

Let $L_{n}=\\{\bm{\ell}\in\mathbb{Z}^{d}:\Gamma_{n}(\bm{\ell},\bm{0})\cap
R_{n}\neq\emptyset\\}$ denote the index set of all hypercubes
$\Gamma_{n}(\bm{\ell},\bm{0})$ that are contained in or boundary of $R_{n}$.
Moreover, let
$L_{1,n}=\\{\bm{\ell}\in\mathbb{Z}^{d}:\Gamma_{n}(\bm{\ell},\bm{0})\subset
R_{n}\\}$ denote the index set of all hypercubes
$\Gamma_{n}(\bm{\ell},\bm{0})$ that are contained in $R_{n}$, and let
$L_{2,n}=\\{\bm{\ell}\in\mathbb{Z}^{d}:\Gamma_{n}(\bm{\ell},\bm{0})\cap
R_{n}\neq 0,\Gamma_{n}(\bm{\ell},\bm{0})\cap R_{n}^{c}\neq\emptyset\\}$ be the
index set of boundary hypercubes.

######  Remark A.1.

Let $\bm{\epsilon}_{0}=(1,\ldots,1)^{\prime}$. The partitions
$\Gamma_{n}(\bm{\ell};\bm{\epsilon}_{0})$ correspond to “big blocks” and the
partitions $\Gamma(\bm{\ell};\bm{\epsilon})$ for
$\bm{\epsilon}\neq\bm{\epsilon}_{0}$ correspond to “small blocks”.

Define

$\displaystyle Z^{\prime}_{\bm{s},A_{n}}(\bm{u},\bm{x})$
$\displaystyle=\bar{K}_{h}\left(\bm{u}-{\bm{s}\over
A_{n}}\right)\left\\{\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{\bm{s},A_{n}}^{\ell}\right)W_{\bm{s}_{j},A_{n}}I(|W_{\bm{s},A_{n}}|\leq\tau_{n})\right.$
$\displaystyle\left.\quad-
E_{\cdot|\bm{S}}\left[\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell}\right)W_{\bm{s}_{j},A_{n}}I(|W_{\bm{s}_{j},A_{n}}|\leq\tau_{n})\right]\right\\}.$

Observe that

$\displaystyle\sum_{j=1}^{n}Z^{\prime}_{\bm{s}_{j},A_{n}}(\bm{u},\bm{x})$
$\displaystyle=\underbrace{\sum_{\ell\in L_{1,n}\cup
L_{2,n}}Z_{A_{n}}^{{}^{\prime}(\bm{\ell};\bm{\epsilon}_{0})}(\bm{u},\bm{x})}_{\text{big
blocks}}+\underbrace{\sum_{\bm{\epsilon}\neq\bm{\epsilon}_{0}}\sum_{\ell\in
L_{1,n}}Z_{A_{n}}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})+\sum_{\bm{\epsilon}\neq\bm{\epsilon}_{0}}\sum_{\ell\in
L_{2,n}}Z_{A_{n}}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})}_{\text{small
blocks}},$ (A.2)

where
$Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})=\sum_{j:\bm{s}_{j}\in\Gamma_{n}(\bm{\ell};\bm{\epsilon})}Z^{\prime}_{\bm{s},A_{n}}(\bm{u},\bm{x})$.
Let
$\\{\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})\\}_{\bm{\ell}\in
L_{1,n}\cup L_{2,n}}$ be independent random variables such that
$\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})\stackrel{{\scriptstyle
d}}{{=}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})$. Note
that for distinct $\bm{\ell}_{1}$ and $\bm{\ell}_{2}$,
$d(\Gamma_{n}(\bm{\ell}_{1};\bm{\epsilon}),\Gamma_{n}(\bm{\ell}_{2};\bm{\epsilon}))\geq
A_{2,n}$. Applying Corollary 2.7 in Yu (1994) (Lemma C.2 in this paper) with
$m=\left({A_{n}\over A_{1,n}}\right)^{d}$ and
$\beta(Q)=\beta(A_{2,n};A_{n}^{d})$, we have that

$\displaystyle\sup_{t>0}\left|P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in
L_{1,n}\cup
L_{2,n}}\\!\\!\\!\\!\\!\\!\\!Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon}_{0})}_{A_{n}}(\bm{u},\bm{x})\right|>t\right)-P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in
L_{1,n}\cup
L_{2,n}}\\!\\!\\!\\!\\!\\!\\!\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon}_{0})}_{A_{n}}(\bm{u},\bm{x})\right|>t\right)\right|$
$\displaystyle\quad\leq C\left({A_{n}\over
A_{1,n}}\right)^{d}\\!\\!\\!\beta(A_{2,n};A_{n}^{d})\ P_{\bm{S}}-a.s.$
$\displaystyle\sup_{t>0}\left|P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in
L_{1,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})\right|>t\right)-P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in
L_{1,n}}\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})\right|>t\right)\right|$
$\displaystyle\quad\leq C\left({A_{n}\over
A_{1,n}}\right)^{d}\beta(A_{2,n};A_{n}^{d})\ P_{\bm{S}}-a.s.$
$\displaystyle\sup_{t>0}\left|P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})\right|>t\right)-P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in
L_{2,n}}\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})\right|>t\right)\right|$
$\displaystyle\leq C\left({A_{n}\over
A_{1,n}}\right)^{d}\beta(A_{2,n};A_{n}^{d})\ P_{\bm{S}}-a.s.$

Since $A_{n}^{d}A_{1,n}^{-d}\beta(A_{2,n};A_{n}^{d})\to 0$ as $n\to\infty$,
these results imply that

$\displaystyle\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon}_{0})}_{A_{n}}(\bm{u},\bm{x})$
$\displaystyle=O\left(\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon}_{0})}_{A_{n}}(\bm{u},\bm{x})\right)\
P_{\bm{S}}-a.s.$ $\displaystyle\sum_{\bm{\ell}\in
L_{1,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})$
$\displaystyle=O\left(\sum_{\bm{\ell}\in
L_{1,n}}\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})\right)\
P_{\bm{S}}-a.s.$ $\displaystyle\sum_{\bm{\ell}\in
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})$
$\displaystyle=O\left(\sum_{\bm{\ell}\in
L_{2,n}}\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})\right)\
P_{\bm{S}}-a.s.$

Now we show $\sup_{(\bm{u},\bm{x})\in
B}\left|\widehat{\psi}_{1}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{\psi}_{1}(\bm{u},\bm{x})]\right|=O_{P_{\cdot|\bm{S}}}\left(a_{n}\right)$.
Cover the region $B$ with $N\leq Ch^{-(d+p)}a_{n}^{-(d+p)}$ balls
$B_{n}=\\{(\bm{u},\bm{x})\in\mathbb{R}^{d+p}:\|(\bm{u},\bm{x})-(\bm{u}_{n},\bm{x}_{n})\|_{\infty}\leq
a_{n}h\\}$ and use $(\bm{u}_{n},\bm{x}_{n})$ to denote the mid point of
$B_{n}$, where $\|\bm{x}_{1}-\bm{x}_{2}\|_{\infty}:=\max_{1\leq j\leq
d}|x_{1,j}-x_{2,j}|$. In addition, let
$K^{\ast}(\bm{w},\bm{v})=C\prod_{k=1}^{d}\widetilde{K}(w_{k})\prod_{j=1}^{p}I(|v_{j}|\leq
2C_{1})$ for $(\bm{w},\bm{v})\in\mathbb{R}^{d+p}$, where $\widetilde{K}$
satisfies Assumption 3.2 (KB1) with $\widetilde{K}^{\ast}(\bm{w},\bm{v})\geq
C_{0}\prod_{k=1}^{d}I(|w_{k}|\leq 2C_{1})\prod_{j=1}^{p}I(|v_{j}|\leq
2C_{1})$. Note that for $(\bm{u},\bm{x})\in B_{n}$ and sufficiently large n,

$\displaystyle\left|\bar{K}_{h}\left(\bm{u}-{\bm{s}\over
A_{n}}\right)\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{\bm{s},A_{n}}^{\ell}\right)-\bar{K}_{h}\left(\bm{u}_{n}-{\bm{s}\over
A_{n}}\right)\prod_{\ell=1}^{p}K_{h}\left(x_{\ell,n}-X_{\bm{s},A_{n}}^{\ell}\right)\right|$
$\displaystyle\quad\leq a_{n}K_{h}^{\ast}\left(\bm{u}_{n}-{\bm{s}\over
A_{n}},\bm{x}_{n}-\bm{X}_{\bm{s},A_{n}}\right)$

with $K_{h}^{\ast}(v)=K^{\ast}(v/h)$. For $\bm{\ell}\in L_{1,n}\cup L_{2,n}$
and $\bm{\epsilon}\in\\{1,2\\}^{d}$, define
$Z_{A_{n}}^{{}^{\prime\prime}(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})$ by
replacing

$\displaystyle\bar{K}_{h}\left(\bm{u}-{\bm{s}\over
A_{n}}\right)\prod_{\ell_{2}}^{p}K_{h}\left(x_{\ell_{2}}-X_{\bm{s},A_{n}}^{\ell_{2}}\right)W_{\bm{s}_{j},A_{n}}I(|W_{\bm{s},A_{n}}|\leq\tau_{n})$

with

$\displaystyle K_{h}^{\ast}\left(\bm{u}_{n}-{\bm{s}_{j}\over
A_{n}},\bm{x}_{n}-\bm{X}_{\bm{s}_{j},A_{n}}\right)|W_{\bm{s}_{j},A_{n}}|I(|W_{\bm{s}_{j},A_{n}}|\leq\tau_{n})$

in the definition of $Z^{\prime}_{\bm{s}_{j},A_{n}}(\bm{u},\bm{x})$ and define

$\displaystyle\bar{\psi}_{1}(\bm{u},\bm{x})$ $\displaystyle={1\over
nh^{p+d}}\sum_{j=1}^{n}K_{h}^{\ast}\left(\bm{u}_{n}-{\bm{s}_{j}\over
A_{n}},\bm{x}_{n}-\bm{X}_{\bm{s}_{j},A_{n}}\right)|W_{\bm{s}_{j},A_{n}}|I(|W_{\bm{s}_{j},A_{n}}|\leq\tau_{n}).$

Note that
$E_{\cdot|\bm{S}}\left[\left|\bar{\psi}_{1}(\bm{u},\bm{x})\right|\right]\leq
M<\infty$ for some sufficiently large $M$, $P_{\bm{S}}$-a.s. Then we obtain

$\displaystyle\sup_{(\bm{u},\bm{x})\in
B}\left|\widehat{\psi}_{1}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{\psi}_{1}(\bm{u},\bm{x})]\right|$
$\displaystyle\leq\left|\widehat{\psi}_{1}(\bm{u}_{n},\bm{x}_{n})-E_{\cdot|\bm{S}}[\widehat{\psi}_{1}(\bm{u}_{n},\bm{x}_{n})]\right|+a_{n}\left(\left|\bar{\psi}_{1}(\bm{u}_{n},\bm{x}_{n})\right|+E_{\cdot|\bm{S}}\left[\left|\bar{\psi}_{1}(\bm{u}_{n},\bm{x}_{n})\right|\right]\right)$
$\displaystyle\leq\left|\widehat{\psi}_{1}(\bm{u}_{n},\bm{x}_{n})-E_{\cdot|\bm{S}}[\widehat{\psi}_{1}(\bm{u}_{n},\bm{x}_{n})]\right|+\left|\bar{\psi}_{1}(\bm{u}_{n},\bm{x}_{n})-E_{\cdot|\bm{S}}[\bar{\psi}_{1}(\bm{u}_{n},\bm{x}_{n})]\right|+2Ma_{n}$
$\displaystyle\leq\left|\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon}_{0})}_{A_{n}}(\bm{u}_{n},\bm{x}_{n})\right|+\sum_{\bm{\epsilon}\neq\bm{\epsilon}_{0}}\left|\sum_{\bm{\ell}\in
L_{1,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u}_{n},\bm{x}_{n})\right|+\sum_{\bm{\epsilon}\neq\bm{\epsilon}_{0}}\left|\sum_{\bm{\ell}\in
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u}_{n},\bm{x}_{n})\right|$
$\displaystyle+\left|\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime\prime}(\bm{\ell};\bm{\epsilon}_{0})}_{A_{n}}(\bm{u}_{n},\bm{x}_{n})\right|+\sum_{\bm{\epsilon}\neq\bm{\epsilon}_{0}}\left|\sum_{\bm{\ell}\in
L_{1,n}}Z^{{}^{\prime\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u}_{n},\bm{x}_{n})\right|+\sum_{\bm{\epsilon}\neq\bm{\epsilon}_{0}}\left|\sum_{\bm{\ell}\in
L_{2,n}}Z^{{}^{\prime\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u}_{n},\bm{x}_{n})\right|$
$\displaystyle+2Ma_{n}.$

Moreover, let
$\\{\widetilde{Z}_{A_{n}}^{{}^{\prime\prime}(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})\\}_{\bm{\ell}\in
L_{1,n}\cup L_{2,n}}$ be independent random variables where
$\widetilde{Z}_{A_{n}}^{{}^{\prime\prime}(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})\stackrel{{\scriptstyle
d}}{{=}}Z_{A_{n}}^{{}^{\prime\prime}(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})$.
Then applying Corollary 2.7 in Yu (1994) to
$\\{\widetilde{Z}_{A_{n}}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})\\}_{\bm{\ell}\in
L_{1,n}\cup L_{2,n}}$ and
$\\{\widetilde{Z}_{A_{n}}^{{}^{\prime\prime}(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})\\}_{\bm{\ell}\in
L_{1,n}\cup L_{2,n}}$, we have that

$\displaystyle P_{\cdot|\bm{S}}\left(\sup_{(\bm{u},\bm{x})\in
B}\left|\widehat{\psi}_{1}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{\psi}_{1}(\bm{u},\bm{x})]\right|>2^{d+1}Ma_{n}\right)$
$\displaystyle\leq N\max_{1\leq k\leq
N}P_{\cdot|\bm{S}}\left(\sup_{(\bm{u},\bm{x})\in
B_{k}}\left|\widehat{\psi}_{1}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{\psi}_{1}(\bm{u},\bm{x})]\right|>2^{d+1}Ma_{n}\right)$
$\displaystyle\leq\sum_{\bm{\epsilon}\in\\{1,2\\}^{d}}\widehat{Q}_{n}(\bm{\epsilon})+\sum_{\bm{\epsilon}\in\\{1,2\\}^{d}}\bar{Q}_{n}(\bm{\epsilon})+2^{d+1}N\left({A_{n}\over
A_{1,n}}\right)^{d}\beta(A_{2,n};A_{n}^{d}),$

where

$\displaystyle\widehat{Q}_{n}(\bm{\epsilon}_{0})$ $\displaystyle=N\max_{1\leq
k\leq N}P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon}_{0})}_{A_{n}}(\bm{u}_{k},\bm{x}_{k})\right|>Ma_{n}nh^{d+p}\right),$
$\displaystyle\bar{Q}_{n}(\bm{\epsilon}_{0})$ $\displaystyle=N\max_{1\leq
k\leq N}P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}\widetilde{Z}^{{}^{\prime\prime}(\bm{\ell};\bm{\epsilon}_{0})}_{A_{n}}(\bm{u}_{k},\bm{x}_{k})\right|>Ma_{n}nh^{d+p}\right),$

and for $\bm{\epsilon}\neq\bm{\epsilon}_{0}$,

$\displaystyle\widehat{Q}_{n}(\bm{\epsilon})$ $\displaystyle=N\max_{1\leq
k\leq N}P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u}_{k},\bm{x}_{k})\right|>Ma_{n}nh^{d+p}\right),$
$\displaystyle\bar{Q}_{n}(\bm{\epsilon})$ $\displaystyle=N\max_{1\leq k\leq
N}P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}\widetilde{Z}^{{}^{\prime\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u}_{k},\bm{x}_{k})\right|>Ma_{n}nh^{d+p}\right).$

Since the proof is similar, we restrict our attention to
$\widehat{Q}_{n}(\bm{\epsilon})$, $\bm{\epsilon}\neq\bm{\epsilon}_{0}$. Note
that

$\displaystyle P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u}_{k},\bm{x}_{k})\right|>Ma_{n}nh^{d+p}\right)\leq
2P_{\cdot|\bm{S}}\left(\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u}_{k},\bm{x}_{k})>Ma_{n}nh^{d+p}\right)$

Since
$\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u}_{k},\bm{x}_{k})$
are zero-mean random variables with

$\displaystyle\left|\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u}_{k},\bm{x}_{k})\right|$
$\displaystyle\leq CA_{1,n}^{d-1}A_{2,n}(\log n)\tau_{n},\ P_{\bm{S}}-a.s.\
(\text{from Lemma \ref{n summands}})$ $\displaystyle
E_{\cdot|\bm{S}}\left[\left(\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u}_{k},\bm{x}_{k})\right)^{2}\right]$
$\displaystyle\leq Ch^{d+p}A_{1,n}^{d-1}A_{2,n}(\log n),\ P_{\bm{S}}-a.s.,$
(A.3)

Lemma C.6 yields that

$\displaystyle P_{\cdot|\bm{S}}\left(\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u}_{k},\bm{x}_{k})>Ma_{n}nh^{d+p}\right)$
$\displaystyle\quad\leq\exp\left(-{{Mnh^{d+p}\log n\over
2}\over\left({A_{n}\over A_{1,n}}\right)^{d}A_{1,n}^{d-1}A_{2,n}h^{d+p}(\log
n)+{M^{1/2}n^{1/2}h^{(d+p)/2}(\log n)^{1/2}A_{1,n}^{d-1}A_{2,n}\tau_{n}\over
3}}\right).$

For (A.3), see Lemma B.1. Observe that

$\displaystyle{nh^{d+p}\log n\over\left({A_{n}\over
A_{1,n}}\right)^{d}A_{1,n}^{d-1}A_{2,n}h^{d+p}(\log n)}$
$\displaystyle=nA_{n}^{-d}\left({A_{1,n}\over
A_{2,n}}\right)\gtrsim{A_{1,n}\over A_{2,n}}\gtrsim n^{\eta},$
$\displaystyle{nh^{d+p}\log n\over n^{1/2}h^{(d+p)/2}(\log
n)^{1/2}A_{1,n}^{d-1}A_{2,n}\tau_{n}}$ $\displaystyle={n^{1/2}h^{(d+p)/2}(\log
n)^{1/2}\over A_{1,n}^{d}\left({A_{2,n}\over
A_{1,n}}\right)\rho_{n}n^{1/\zeta}}\geq C_{0}n^{\eta/2}.$

Taking $M>0$ sufficiently large, this shows the desired result. ∎

######  Remark A.2.

Let $\eta_{1}\in[0,1),\gamma_{2},\gamma_{A_{1}},\gamma_{A_{2}}\in(0,1)$ with
$\gamma_{A_{1}}>\gamma_{A_{2}}$. Define

$\displaystyle
A_{n}^{d}=n^{1-\eta_{1}},nh^{p+d}=n^{\gamma_{2}},A_{1,n}=A_{n}^{\gamma_{A_{1}}},A_{2,n}=A_{n}^{\gamma_{A_{2}}}.$

Note that

$\displaystyle{n^{1/2}h^{(d+p)/2}(\log n)^{1/2}\over
A_{1,n}^{d}\left({A_{2,n}\over A_{1,n}}\right)\rho_{n}n^{1/\zeta}}$
$\displaystyle\geq{n^{1/2}h^{(d+p)/2}(\log n)^{1/2}\over
A_{1,n}^{d}\rho_{n}n^{1/\zeta}}={n^{\gamma_{2}/2}(\log
n)^{1/2}\over\rho_{n}n^{(1-\eta_{1})\gamma_{A_{1}}+1/\zeta}}$
$\displaystyle={(\log n)^{1/2}\over\rho_{n}}n^{{\gamma_{2}\over
2}-(1-\eta_{1})\gamma_{A_{1}}-{1\over\zeta}}$

For $n^{{\gamma_{2}\over 2}-(1-\eta_{1})\gamma_{A_{1}}-{1\over\zeta}}\gtrsim
n^{\eta}$ for some $\eta>0$, we need

$\displaystyle(1-\eta_{1})\gamma_{A_{1}}+{1\over\zeta}<{\gamma_{2}\over 2}.$
(A.4)

######  Remark A.3.

Assume that $\bm{X}_{\bm{s},A_{n}}$ satisfies Conditions (Ma1) and (Ma2) in
Assumption 5.1. Observe that

$\displaystyle Z_{A_{n}}^{(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})$
$\displaystyle=Z_{1,A_{n}}^{(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x}:A_{2,n})+Z_{2,A_{n}}^{(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x}:A_{2,n})$
$\displaystyle=\sum_{j:\bm{s}_{j}\in\Gamma_{n}(\bm{\ell};\bm{\epsilon})\cap
R_{n}}Z_{1,\bm{s}_{j},A_{n}}(\bm{u},\bm{x}:A_{2,n})+\sum_{j:\bm{s}_{j}\in\Gamma_{n}(\bm{\ell};\bm{\epsilon})\cap
R_{n}}Z_{2,\bm{s}_{j},A_{n}}(\bm{u},\bm{x}:A_{2,n}),$

where

$\displaystyle Z_{1,\bm{s}_{j},A_{n}}(\bm{u},\bm{x}:A_{2,n})$
$\displaystyle=\bar{K}_{h}\left(\bm{u}-{\bm{s}\over
A_{n}}\right)\left\\{\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{\bm{s}_{j},A_{n}:A_{2,n}}^{\ell}\right)W_{\bm{s},A_{n}}\right.$
$\displaystyle\left.\quad\quad-
E_{\cdot|\bm{S}}\left[\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{\bm{s}_{j},A_{n}:A_{2,n}}^{\ell}\right)W_{\bm{s},A_{n}}\right]\right\\}$

and
$Z_{2,\bm{s}_{j},A_{n}}(\bm{u},\bm{x}:A_{2,n})=\bar{Z}_{\bm{s}_{j},A_{n}}(\bm{u},\bm{x})-Z_{1,\bm{s}_{j},A_{n}}(\bm{u},\bm{x}:A_{2,n})$.
Note that

$\displaystyle|E_{\cdot|\bm{S}}\left[Z_{2,\bm{s}_{j},A_{n}}(\bm{u},\bm{x}:A_{2,n})\right]|$
$\displaystyle\leq 2\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)E_{\cdot|\bm{S}}\left[\left|\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{\bm{s}_{j},A_{n}:A_{2,n}}^{\ell}\right)-\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{\bm{s},A_{n}}^{\ell}\right)\right||W_{\bm{s}_{j},A_{n}}|\right]$
$\displaystyle\leq
2C\sum_{\ell=1}^{p}E_{\cdot|\bm{S}}\left[\left|K_{h}\left(x_{\ell}-X_{\bm{s}_{j},A_{n}:A_{2,n}}^{\ell}\right)-K_{h}\left(x_{\ell}-X_{\bm{s},A_{n}}^{\ell}\right)\right||W_{\bm{s}_{j},A_{n}}|\right]$
$\displaystyle\leq 2CpE_{\cdot|\bm{S}}\left[\max_{1\leq\ell\leq
p}\left|{X_{\bm{s},A_{n}}^{\ell}-X_{\bm{s}_{j},A_{n}:A_{2,n}}^{\ell}\over
h}\right||W_{\bm{s}_{j},A_{n}}|\right]={2Cp\over
h}E_{\cdot|\bm{S}}\left[\max_{1\leq\ell\leq
p}\left|\epsilon_{\bm{s}_{j},A_{n}:A_{2,n}}\right||W_{\bm{s}_{j},A_{n}}|\right]$
$\displaystyle\leq{2Cp\over h}E_{\cdot|\bm{S}}\left[\max_{1\leq\ell\leq
p}\left|\epsilon_{\bm{s}_{j},A_{n}:A_{2,n}}\right|^{{\zeta\over\zeta-1}}\right]^{1-1/\zeta}E_{\cdot|\bm{S}}\left[|W_{\bm{s}_{j},A_{n}}|^{\zeta}\right]^{1/\zeta}$
$\displaystyle\leq{2Cp^{1+1/q}\over h}\max_{1\leq\ell\leq
p}E_{\cdot|\bm{S}}\left[\left|\epsilon_{\bm{s}_{j},A_{n}:A_{2,n}}\right|^{{q\zeta\over\zeta-1}}\right]^{{\zeta-1\over
q\zeta}}\leq 2Cp^{1+1/q}h^{-1}\gamma_{\epsilon}(A_{2,n}).$

Here, we used Lipschitz continuity of $K$ in the third inequality and Lemma
2.2.2 in van der Vaart and Wellner (1996) in the fifth inequality. Applying
Markov’s inequality and Lemma C.1, this yields that

$\displaystyle P_{\cdot|\bm{S}}\left(\left|\sum_{\ell\in L_{1,n}\cup
L_{2,n}}Z_{2,A_{n}}^{(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x}:A_{2,n})\right|>\sqrt{n^{1-\eta_{2}}h^{d+p}}\right)$
$\displaystyle\leq{\sum_{\ell\in L_{1,n}\cup
L_{2,n}}\sum_{j:\bm{s}_{j}\in\Gamma_{n}(\bm{\ell};\bm{\epsilon})\cap
R_{n}}E_{\cdot|\bm{S}}\left[\left|Z_{2,\bm{s}_{j},A_{n}}(\bm{u},\bm{x}:A_{2,n})\right|\right]\over\sqrt{n^{1-\eta_{2}}h^{d+p}}}$
$\displaystyle\leq C{(A_{n}/A_{1,n})^{d}A_{1,n}^{d}(nA_{n}^{-d}+\log
n)\gamma_{\epsilon}(A_{2,n})\over n^{{1-\eta_{2}\over 2}}h^{{d+p\over
2}+1}}\to 0\ \text{as $n\to\infty$}.$ (A.5)

Define $Z_{1,A_{n}}^{{}^{\prime}(\bm{\ell;\bm{\epsilon}})}(\bm{u},\bm{x})$ by
replacing $X_{\bm{s}_{j},A_{n}}^{\ell}$ in definition of
$Z_{A_{n}}^{{}^{\prime}(\bm{\ell;\bm{\epsilon}})}(\bm{u},\bm{x})$ with
$X_{\bm{s}_{j},A_{n}:A_{2,n}}$ and define
$Z_{2,A_{n}}^{{}^{\prime}(\bm{\ell;\bm{\epsilon}})}(\bm{u},\bm{x})=Z_{A_{n}}^{{}^{\prime}(\bm{\ell;\bm{\epsilon}})}(\bm{u},\bm{x})-Z_{1,A_{n}}^{{}^{\prime}(\bm{\ell;\bm{\epsilon}})}(\bm{u},\bm{x})$.
Observe that

$\displaystyle P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})\right|>t\right)$
$\displaystyle=P_{\cdot|\bm{S}}\left(\left\\{\left|\sum_{\bm{\ell}\in
L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})\right|>t\right\\}\cap\left\\{\left|\sum_{\bm{\ell}\in
L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{2,A_{n}}(\bm{u},\bm{x})\right|>t_{0}\right\\}\right)$
$\displaystyle\quad+P_{\cdot|\bm{S}}\left(\left\\{\left|\sum_{\bm{\ell}\in
L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{1,A_{n}}(\bm{u},\bm{x})+\sum_{\bm{\ell}\in
L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{2,A_{n}}(\bm{u},\bm{x})\right|>t\right\\}\cap\left\\{\left|\sum_{\bm{\ell}\in
L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{2,A_{n}}(\bm{u},\bm{x})\right|\leq
t_{0}\right\\}\right)$ $\displaystyle\leq
P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{2,A_{n}}(\bm{u},\bm{x})\right|>t_{0}\right)+P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in
L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{1,A_{n}}(\bm{u},\bm{x})\right|>t-t_{0}\right)$

Set $t=Ma_{n}nh^{d+p}$ and
$t_{0}=\sqrt{n^{1-\eta_{2}}h^{d+p}}\lesssim\sqrt{nh^{d+p}}$. Let
$\widetilde{\beta}(a;b)$ be the $\beta$-mixing coefficients of
$X_{\bm{s},A_{n}:A_{2,n}}$. Applying (A.3) and Lemma C.2 (Corollary 2.7 in Yu
(1994)), we have that

$\displaystyle P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{A_{n}}(\bm{u},\bm{x})\right|>Ma_{n}nh^{d+p}\right)\leq
P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{1,A_{n}}(\bm{u},\bm{x})\right|>2Ma_{n}nh^{d+p}\right)+o(1)$
$\displaystyle\leq P_{\cdot|\bm{S}}\left(\left|\sum_{\bm{\ell}\in L_{1,n}\cup
L_{2,n}}\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{1,A_{n}}(\bm{u},\bm{x})\right|>2Ma_{n}nh^{d+p}\right)+\left({A_{n}\over
A_{1,n}}\right)^{d}\underbrace{\widetilde{\beta}(A_{2,n};A_{n}^{d})}_{=0}+o(1),$

where
$\\{\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{1,A_{n}}(\bm{u},\bm{x})\\}$
are independent random variables with
$\widetilde{Z}^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{1,A_{n}}(\bm{u},\bm{x})\stackrel{{\scriptstyle
d}}{{=}}Z^{{}^{\prime}(\bm{\ell};\bm{\epsilon})}_{1,A_{n}}(\bm{u},\bm{x})$.
This implies that under (Ma1) and (Ma2), we can replace
$\bm{X}_{\bm{s},A_{n}}$ with $\bm{X}_{\bm{s},A_{n}:A_{2,n}}$ in our analysis.

###### Proof of Theorem 3.1.

Observe that

$\displaystyle\widehat{m}(\bm{u},\bm{x})-m(\bm{u},\bm{x})$
$\displaystyle={1\over\widehat{f}(\bm{u},\bm{x})}\left(\widehat{g}_{1}(\bm{u},\bm{x})+\widehat{g}_{2}(\bm{u},\bm{x})-m(\bm{u},\bm{x})\widehat{f}(\bm{u},\bm{x})\right),$

where

$\displaystyle\widehat{f}(\bm{u},\bm{x})$ $\displaystyle={1\over
nh^{d+p}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell}\right),$
$\displaystyle\widehat{g}_{1}(\bm{u},\bm{x})$ $\displaystyle={1\over
nh^{d+p}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell}\right)\epsilon_{\bm{s}_{j},A_{n}},$
$\displaystyle\widehat{g}_{2}(\bm{u},\bm{x})$ $\displaystyle={1\over
nh^{d+p}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell}\right)m\left({\bm{s}_{j}\over
A_{n}},\bm{X}_{\bm{s}_{j},A_{n}}\right).$

(Step1) First we give a sketch of the proof. In Steps and 2, we show the
following four results:

  * (i)

$\sup_{\bm{u}\in[0,1]^{d},\bm{x}\in
S_{c}}\left|\widehat{g}_{1}(\bm{u},\bm{x})\right|=O_{P_{\cdot|\bm{S}}}\left(\sqrt{(\log
n)/nh^{d+p}}\right)$, $P_{\bm{S}}$-a.s.

  * (ii)

$\displaystyle\sup_{\bm{u}\in[0,1]^{d},\bm{x}\in
S_{c}}\left|\widehat{g}_{2}(\bm{u},\bm{x})-m(\bm{u},\bm{x})\widehat{f}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}\left[\widehat{g}_{2}(\bm{u},\bm{x})-m(\bm{u},\bm{x})\widehat{f}(\bm{u},\bm{x})\right]\right|$
$\displaystyle\quad=O_{P_{\cdot|\bm{S}}}\left(\sqrt{(\log
n)/nh^{d+p}}\right),\ P_{\bm{S}}-a.s.$

  * (iii)

Let $\kappa_{2}=\int_{\mathbb{R}}x^{2}K(x)dx$.

$\displaystyle\sup_{\bm{u}\in I_{h},\bm{x}\in
S_{c}}\left|E_{\cdot|\bm{S}}\left[\widehat{g}_{2}(\bm{u},\bm{x})-m(\bm{u},\bm{x})\widehat{f}(\bm{u},\bm{x})\right]\right|$
$\displaystyle\quad=h^{2}{\kappa_{2}\over
2}f_{\bm{S}}(\bm{u})\left\\{\sum_{i=1}^{d}\left(2\partial_{u_{i}}m(\bm{u},\bm{x})\partial_{u_{i}}f(\bm{u},\bm{x})+\partial_{u_{i}u_{i}}^{2}m(\bm{u},\bm{x})f(\bm{u},\bm{x})\right)\right.$
$\displaystyle\left.\quad\quad+\sum_{k=1}^{p}\left(2\partial_{x_{k}}m(\bm{u},\bm{x})\partial_{x_{k}}f(\bm{u},\bm{x})+\partial_{x_{k}x_{k}}^{2}m(\bm{u},\bm{x})f(\bm{u},\bm{x})\right)\right\\}+O\left({1\over
A_{n}^{dr}h^{p}}\right)+o(h^{2}),\ P_{\bm{S}}-a.s.$

  * (iv)

$\sup_{\bm{u}\in I_{h},\bm{x}\in
S_{c}}\left|\widehat{f}(\bm{u},\bm{x})-f(\bm{u},\bm{x})\right|=o_{P_{\cdot|\bm{S}}}(1)$,
$P_{\bm{S}}$-a.s.

(i) can be shown by applying Proposition 3.1 with
$W_{\bm{s}_{j},A_{n}}=\epsilon_{\bm{s}_{j},A}$. (ii) can be shown by applying
Proposition 3.1 to
$\widehat{g}_{2}(\bm{u},\bm{x})-m(\bm{u},\bm{x})\widehat{f}(\bm{u},\bm{x})$.
For the proof of (iv), we decompose
$\widehat{f}(\bm{u},\bm{x})-f(\bm{u},\bm{x})$ into a variance part
$\widehat{f}(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[\widehat{f}(\bm{u},\bm{x})]$ and
a bias part $E_{\cdot|\bm{S}}[\widehat{f}(\bm{u},\bm{x})]-f(\bm{u},\bm{x})$.
Applying Proposition 3.1 with $W_{\bm{s}_{j},A_{n}}=1$, we have that the
variance part is $o_{P_{\cdot|\bm{S}}}(1)$ uniformly in $\bm{u}$ and
$\bm{x}\in S_{c}$. The bias part can be evaluated by similar arguments used to
prove (iii). Combining the results (i), (ii) and (iii), we have that

$\displaystyle\sup_{\bm{u}\in I_{h},\bm{x}\in
S_{c}}\left|\widehat{m}(\bm{u},\bm{x})-m(\bm{u},\bm{x})\right|$
$\displaystyle\leq{1\over\inf_{\bm{u}\in I_{h},\bm{x}\in
S_{c}}\widehat{f}(\bm{u},\bm{x})}\left(\sup_{\bm{u}\in I_{h},\bm{x}\in
S_{c}}\left|\widehat{g}_{1}(\bm{u},\bm{x})\right|+\sup_{\bm{u}\in
I_{h},\bm{x}\in
S_{c}}\left|\widehat{g}_{2}(\bm{u},\bm{x})-m(\bm{u},\bm{x})\widehat{f}(\bm{u},\bm{x})\right|\right)$
$\displaystyle\leq{1\over\inf_{\bm{u}\in I_{h},\bm{x}\in
S_{c}}\widehat{f}(\bm{u},\bm{x})}O_{P_{\cdot|\bm{S}}}\left(\sqrt{\log n\over
nh^{d+p}}+{1\over A_{n}^{dr}h^{p}}+h^{2}\right).$

The result (iv) and $\inf_{\bm{u}\in[0,1]^{d},\bm{x}\in
S_{c}}f(\bm{u},\bm{x})>0$ imply that

$\displaystyle{1\over\inf_{\bm{u}\in I_{h},\bm{x}\in
S_{c}}\widehat{f}(\bm{u},\bm{x})}$
$\displaystyle\lesssim{1\over\inf_{\bm{u}\in[0,1]^{d},\bm{x}\in
S_{c}}f(\bm{u},\bm{x})-\inf_{\bm{u}\in I_{h},\bm{x}\in
S_{c}}|\widehat{f}(\bm{u},\bm{x})-f(\bm{u},\bm{x})|}=O_{P_{\cdot|\bm{S}}}(1).$

Therefore, we complete the proof.

(Step2) In this step, we show (iii). Let $K_{0}:\mathbb{R}\to\mathbb{R}$ be a
Lipschitz continuous function with support $[-qC_{1},qC_{1}]$ for some $q>1$.
Assume that $K_{0}(x)=1$ for all $x\in[-C_{1},C_{1}]$ and write
$K_{0,h}(x)=K_{0}(x/h)$. Observe that

$\displaystyle
E_{\cdot|\bm{S}}\left[\widehat{g}_{2}(\bm{u},\bm{x})-m(\bm{u},\bm{x})\widehat{f}(\bm{u},\bm{x}))\right]$
$\displaystyle=\sum_{i=1}^{4}Q_{i}(\bm{u},\bm{x}),$

where

$\displaystyle Q_{i}(\bm{u},\bm{x})$ $\displaystyle={1\over
nh^{d+p}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)q_{i}(\bm{u},\bm{x})$

and

$\displaystyle q_{1}(\bm{u},\bm{x})$
$\displaystyle=E_{\cdot|\bm{S}}\left[\prod_{\ell=1}^{p}K_{0,h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})\left\\{\prod_{\ell=1}^{p}K_{h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})-\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\right\\}\right.$
$\displaystyle\left.\quad\quad\times\left\\{m\left({\bm{s}_{j}\over
A_{n}},\bm{X}_{\bm{s}_{j},A_{n}}\right)-m(\bm{u},\bm{x})\right\\}\right],$
$\displaystyle q_{2}(\bm{u},\bm{x})$
$\displaystyle=E_{\cdot|\bm{S}}\left[\prod_{\ell=1}^{p}K_{0,h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\left\\{m\left({\bm{s}_{j}\over
A_{n}},\bm{X}_{\bm{s}_{j},A_{n}}\right)-m\left({\bm{s}_{j}\over
A_{n}},\bm{X}_{{\bm{s}_{j}\over A_{n}}}(\bm{s}_{j})\right)\right\\}\right],$
$\displaystyle q_{3}(\bm{u},\bm{x})$
$\displaystyle=E_{\cdot|\bm{S}}\left[\left\\{\prod_{\ell=1}^{p}K_{0,h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})-\prod_{\ell=1}^{p}K_{0,h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\right\\}\right.$
$\displaystyle\left.\quad\quad\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\left\\{m\left({\bm{s}_{j}\over
A_{n}},\bm{X}_{{\bm{s}_{j}\over
A_{n}}}(\bm{s}_{j})\right)-m(\bm{u},\bm{x})\right\\}\right],$ $\displaystyle
q_{4}(\bm{u},\bm{x})$
$\displaystyle=E_{\cdot|\bm{S}}\left[\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\left\\{m\left({\bm{s}_{j}\over
A_{n}},\bm{X}_{{\bm{s}_{j}\over
A_{n}}}(\bm{s}_{j})\right)-m(\bm{u},\bm{x})\right\\}\right].$

We first consider $Q_{1}(\bm{u},\bm{x})$. Since the kernel $K$ is bounded, we
can use the telescoping argument to get that

$\displaystyle\left|\prod_{\ell=1}^{p}K_{h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})-\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\right|\leq
C\sum_{\ell=1}^{p}\left|K_{h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})-K_{h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\right|.$

Once again using the boundedness of $K$, we can find a constant $C<\infty$
such that

$\displaystyle\left|K_{h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})-K_{h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\right|$
$\displaystyle\leq\left|K_{h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})-K_{h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\right|^{r},$

where $r=\min\\{\rho,1\\}$. Therefore,

$\displaystyle\left|\prod_{\ell=1}^{p}K_{h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})-\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\right|\leq
C\sum_{\ell=1}^{p}\left|K_{h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})-K_{h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\right|^{r}.$

Applying this inequality, we have that

$\displaystyle Q_{1}(\bm{u},\bm{x})$ $\displaystyle\leq{C\over
nh^{d+p}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)E_{\cdot|\bm{S}}\left[\sum_{\ell=1}^{p}\left|K_{h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})-K_{h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\right|^{r}\right.$
$\displaystyle\left.\quad\times\prod_{\ell=1}^{p}K_{0,h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})\left|m\left({\bm{s}_{j}\over
A_{n}},\bm{X}_{\bm{s}_{j},A_{n}}\right)-m(\bm{u},\bm{x})\right|\right].$

Note that
$\prod_{\ell=1}^{p}K_{0,h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})\left|m\left({\bm{s}_{j}\over
A_{n}},\bm{X}_{\bm{s}_{j},A_{n}}\right)-m(\bm{u},\bm{x})\right|\leq Ch$. Since
$K$ is Lipschitz, $\left|X_{\bm{s}_{j},A_{n}}^{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right|\leq{C\over
A_{n}^{d}}U_{\bm{s}_{j},A_{n}}(\bm{s}_{j}/A_{n})$ and the variable
$U_{\bm{s}_{j},A_{n}}(\bm{s}_{j}/A_{n})$ have finite $r$-th moment, we have
that

$\displaystyle Q_{1}(\bm{u},\bm{x})$ $\displaystyle\quad\leq{C\over
nh^{d+p-1}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)E_{\cdot|\bm{S}}\left[\sum_{\ell=1}^{p}\left|K_{h}(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell})-K_{h}\left(x_{\ell}-X_{{\bm{s}_{j}\over
A_{n}}}^{\ell}(\bm{s}_{j})\right)\right|^{r}\right]$
$\displaystyle\quad\leq{C\over
nh^{d+p-1}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)E_{\cdot|\bm{S}}\left[\sum_{k=1}^{p}\left|{1\over
A_{n}^{d}h}U_{\bm{s}_{j},A_{n}}\left({\bm{s}_{j}\over
A_{n}}\right)\right|^{r}\right]\leq{C\over A_{n}^{dr}h^{p-1+r}}\ (\text{from
Lemma \ref{Masry-thm}})$

uniformly in $\bm{u}$ and $\bm{x}$. Using similar arguments, we can also show
that

$\displaystyle\sup_{\bm{u}\in I_{h},\bm{x}\in S_{c}}|Q_{2}(\bm{u},\bm{x})|$
$\displaystyle\leq{C\over A_{n}^{dr}h^{p}},\ \sup_{\bm{u}\in I_{h},\bm{x}\in
S_{c}}|Q_{3}(\bm{u},\bm{x})|\leq{C\over A_{n}^{dr}h^{p-1+r}}.$

Finally, applying Lemmas C.4 and C.5 and using the assumptions on the
smoothness of $m$ and $f$, we have that

$\displaystyle Q_{4}(\bm{u},\bm{x})$ $\displaystyle=h^{2}{\kappa_{2}\over
2}f_{\bm{S}}(\bm{u})\left\\{\sum_{i=1}^{d}\left(2\partial_{u_{i}}m(\bm{u},\bm{x})\partial_{u_{i}}f(\bm{u},\bm{x})+\partial_{u_{i}u_{i}}^{2}m(\bm{u},\bm{x})f(\bm{u},\bm{x})\right)\right.$
$\displaystyle\left.\quad+\sum_{k=1}^{p}\left(2\partial_{x_{k}}m(\bm{u},\bm{x})\partial_{x_{k}}f(\bm{u},\bm{x})+\partial_{x_{k}x_{k}}^{2}m(\bm{u},\bm{x})f(\bm{u},\bm{x})\right)\right\\}+o(h^{2})$

uniformly in $\bm{u}$ and $\bm{x}$. Combining the results on
$Q_{i}(\bm{u},\bm{x})$, $1\leq i\leq 4$ yields (iii). ∎

######  Remark A.4.

Let $h\sim n^{1/(d+p+4)}$ and $A_{n}^{d}=n^{1-\eta_{1}}$ for some
$\eta_{1}\in[0,1)$.

$\displaystyle{1\over A_{n}^{dr}h^{p}}\lesssim h^{2}\Leftrightarrow 1\lesssim
A_{n}^{dr}h^{p+2}=n^{r(1-\eta_{1})-{p+2\over d+p+4}}(\log n)^{{r\eta_{2}\over
d}}\Leftarrow 1-\eta_{1}\geq{p+2\over r(d+p+4)}.$

If $r=1$,

$\displaystyle 1-\eta_{1}\geq{p+2\over d+p+4}\Leftrightarrow{d+2\over
d+p+4}\geq\eta_{1}.$ (A.6)

(A.6) is satisfied for $d\geq 1$ and $p\geq 1$.

###### Proof of Theorem 3.2.

Observe that

$\displaystyle\sqrt{nh^{d+p}}(\widehat{m}(\bm{u},\bm{x})-m(\bm{u},\bm{x}))={\sqrt{nh^{d+p}}\over\widehat{f}(\bm{u},\bm{x})}\left(\widehat{g}_{1}(\bm{u},\bm{x})+\widehat{g}_{2}(\bm{u},\bm{x})-m(\bm{u},\bm{x})\widehat{f}(\bm{u},\bm{x}))\right).$

Define

$\displaystyle B(\bm{u},\bm{x})$
$\displaystyle=\sqrt{nh^{p+d}}\left(\widehat{g}_{2}(\bm{u},\bm{x})-m(\bm{u},\bm{x})\widehat{f}(\bm{u},\bm{x}))\right),\
V(\bm{u},\bm{x})=\sqrt{nh^{d+p}}\widehat{g}_{1}(\bm{u},\bm{x}).$

$B(\bm{u},\bm{x})$ converge in $P_{\cdot|\bm{S}}$-probability to
$B_{\bm{u},\bm{x}}$ defined in Theorem 3.2. This follows from (iii) in the
proof of Theorem 3.1 and the fact that
$B(\bm{u},\bm{x})-E_{\cdot|\bm{S}}[B(\bm{u},\bm{x})]=o_{P_{\cdot|\bm{S}}}(1)$.
To prove the latter, it is sufficient to prove
$\operatorname{Var}_{\cdot|\bm{S}}(B(\bm{u},\bm{x}))=o(1)$, $P_{\bm{S}}$-a.s.,
which can be shown by similar arguments used in the proof Lemma B.2. Using the
i.i.d. assumption on $\\{\epsilon_{j}\\}$,
$\bar{K}(\bm{u})=\prod_{\ell=1}^{d}K(u_{\ell})$ and Lemma C.4, the asymptotic
variance in (A.7) can be computed as follows:

$\displaystyle\operatorname{Var}_{\cdot|\bm{S}}\left(V(\bm{u},\bm{x})\right)$
$\displaystyle=\operatorname{Var}_{\cdot|\bm{S}}\left({1\over\sqrt{nh^{d+p}}}\sum_{j=1}^{n}\bar{K}_{h}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-X_{\bm{s}_{j},A_{n}}^{\ell}\right)\epsilon_{\bm{s}_{j},A_{n}}\right)$
$\displaystyle={1\over
nh^{d+p}}\sum_{j=1}^{n}\bar{K}_{h}^{2}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\int_{\mathbb{R}^{p}}\prod_{\ell=1}^{p}K_{h}\left(x_{\ell}-w_{\ell}\right)E_{\cdot|\bm{S}}\left[\epsilon_{\bm{s}_{j},A_{n}}^{2}|\bm{X}_{\bm{s}_{j},A_{n}}=\bm{w}\right]f_{\bm{X}_{\bm{s}_{j},A_{n}}}(\bm{w})d\bm{w}$
$\displaystyle={1\over
nh^{d}}\sum_{j=1}^{n}\bar{K}_{h}^{2}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\int_{\mathbb{R}^{p}}\prod_{\ell=1}^{p}K\left(\varphi_{\ell}\right)\sigma^{2}\left({\bm{s}_{j}\over
A_{n}},\bm{x}-h\bm{\varphi}\right)f_{\bm{X}_{\bm{s}_{j},A_{n}}}(\bm{x}-h\bm{\varphi})d\bm{\varphi}$
$\displaystyle={1\over
nh^{d}}\sum_{j=1}^{n}\bar{K}_{h}^{2}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\left(\kappa_{0}^{p}\sigma^{2}\left({\bm{s}_{j}\over
A_{n}},\bm{x}\right)f\left({\bm{s}_{j}\over A_{n}},\bm{x}\right)\right)+o(1)$
$\displaystyle=\kappa_{0}^{d+p}f_{\bm{S}}(\bm{u})\sigma^{2}\left(\bm{u},\bm{x}\right)f\left(\bm{u},\bm{x}\right)+o(1)\
P_{\bm{S}}-a.s.$

Moreover, $V(\bm{u},\bm{x})$ is asymptotically normal. In particular,

$\displaystyle V(\bm{u},\bm{x})\stackrel{{\scriptstyle
d}}{{\to}}N(0,\kappa_{0}^{d+p}f_{\bm{S}}(\bm{u})\sigma^{2}(\bm{u},\bm{x})f(\bm{u},\bm{x})).$
(A.7)

We can show (A.7) by applying blocking arguments. Decompose $V(\bm{u},\bm{x})$
into some big-blocks and small-blocks as in (A.2). We can neglect the small
blocks by applying Lemma B.2 and use mixing conditions to replace the big
blocks by independent random variables. This allows us to apply a Lyapunov’s
condition for the central limit theorem for sum of independent random
variables to get the result. We omit the details as the proof is similar to
that of Theorem 3.1 in Lahiri (2003b) under the standard strictly stationary
$\alpha$-mixing settings for random fields. ∎

###  A.2. Proofs for Section 4

###### Proof of Theorems 4.1 and 4.2.

Since we can prove the desired result by applying almost the same strategy in
the proof of Theorems 5.1 and 5.2 in Vogt (2012), we omit the proof. We note
that it suffices to check conditions (A1)-(A6), (A8) and (A9) in Mammen et al.
(1999) to obtain the desired results. We can check those conditions by
applying almost the same argument in the proof of Theorems 5.1 and 5.2 in Vogt
(2012) and applying Proposition 3.1, Theorems 3.1, 3.2 and Lemmas B.4-B.6 in
this paper, which correspond to Lemmas C.2-C.4 in Vogt (2012). ∎

##  Appendix B Auxiliary lemmas

Define

$\displaystyle\bar{Z}_{\bm{s},A_{n}}(\bm{u},\bm{x})$
$\displaystyle=\bar{K}_{h}\left(\bm{u}-{\bm{s}\over
A_{n}}\right)\left\\{\prod_{\ell_{2}=1}^{p}K_{h}\left(x_{\ell_{2}}-X_{\bm{s},A_{n}}^{\ell_{2}}\right)W_{\bm{s},A_{n}}-E_{\cdot|\bm{S}}\left[\prod_{\ell_{2}=1}^{p}K_{h}\left(x_{\ell_{2}}-X_{\bm{s},A_{n}}^{\ell_{2}}\right)W_{\bm{s},A_{n}}\right]\right\\},$
(B.1)

$\displaystyle Z_{A_{n}}(\bm{u},\bm{x})$
$\displaystyle=\sum_{j=1}^{n}\bar{Z}_{\bm{s}_{j},A_{n}}(\bm{u},\bm{x})$
$\displaystyle=\sum_{\ell\in L_{1,n}\cup
L_{2,n}}Z_{A_{n}}^{(\bm{\ell};\bm{\epsilon}_{0})}(\bm{u},\bm{x})+\sum_{\bm{\epsilon}\neq\bm{\epsilon}_{0}}\sum_{\ell\in
L_{1,n}}Z_{A_{n}}^{(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})+\sum_{\bm{\epsilon}\neq\bm{\epsilon}_{0}}\sum_{\ell\in
L_{2,n}}Z_{A_{n}}^{(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x}),$

where
$Z_{A_{n}}^{(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})=\sum_{j:\bm{s}_{j}\in\Gamma_{n}(\bm{\ell};\bm{\epsilon})\cap
R_{n}}\bar{Z}_{\bm{s}_{j},A_{n}}(\bm{u},\bm{x})$.

######  Lemma B.1.

Under Assumptions 2.1, 3.2, 3.3 and 3.4,

$\displaystyle
E_{\cdot|\bm{S}}\left[\left(Z_{A_{n}}^{(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})\right)^{2}\right]$
$\displaystyle\leq CA_{1,n}^{d-1}A_{2,n}(nA_{n}^{-d}+\log n)h^{p+d}\
P_{\bm{S}}-a.s.$

###### Proof.

Observe that

$\displaystyle
E_{\cdot|\bm{S}}\left[\left(Z_{A_{n}}^{(\bm{\ell};\bm{\epsilon})}(\bm{u},\bm{x})\right)^{2}\right]$
$\displaystyle=\sum_{j:\bm{s}_{j}\in\Gamma_{n}(\bm{\ell};\bm{\epsilon})\cap
R_{n}}\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!E_{\cdot|\bm{S}}\left[\bar{Z}_{\bm{s}_{j},A_{n}}^{2}(\bm{u},\bm{x})\right]+\sum_{j_{1}\neq
j_{2}:\bm{s}_{j_{1}},\bm{s}_{j_{2}}\in\Gamma_{n}(\bm{\ell};\bm{\epsilon})\cap
R_{n}}\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!E_{\cdot|\bm{S}}\left[\bar{Z}_{\bm{s}_{j_{1}},A_{n}}(\bm{u},\bm{x})\bar{Z}_{\bm{s}_{j_{2}},A_{n}}(\bm{u},\bm{x})\right].$

Note that

$\displaystyle
E_{\cdot|\bm{S}}\left[\bar{Z}_{\bm{s}_{j},A_{n}}^{2}(\bm{u},\bm{x})\right]$
$\displaystyle=\bar{K}_{h}^{2}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\left(E_{\cdot|\bm{S}}\left[\prod_{\ell_{2}=1}^{p}K_{h}^{2}\left(x_{\ell_{2}}-X_{\bm{s}_{j},A_{n}}^{\ell_{2}}\right)W_{\bm{s}_{j},A_{n}}^{2}\right]-\left(E_{\cdot|\bm{S}}\left[\prod_{\ell_{2}=1}^{p}K_{h}\left(x_{\ell_{2}}-X_{\bm{s}_{j},A_{n}}^{\ell_{2}}\right)W_{\bm{s}_{j},A_{n}}\right]\right)^{2}\right)$
$\displaystyle\leq\bar{K}_{h}^{2}\left(\bm{u}-{\bm{s}_{j}\over
A_{n}}\right)\left\\{E_{\cdot|\bm{S}}\left[\prod_{\ell_{2}=1}^{p}K_{h}^{2}\left(x_{\ell_{2}}-X_{\bm{s}_{j},A_{n}}^{\ell_{2}}\right)W_{\bm{s}_{j},A_{n}}^{2}\right]+\left(E_{\cdot|\bm{S}}\left[\prod_{\ell_{2}=1}^{p}K_{h}\left(x_{\ell_{2}}-X_{\bm{s}_{j},A_{n}}^{\ell_{2}}\right)\left|W_{\bm{s}_{j},A_{n}}\right|\right]\right)^{2}\right\\}.$

Observe that

$\displaystyle
E_{\cdot|\bm{S}}\left[\prod_{\ell_{2}=1}^{p}K_{h}\left(x_{\ell_{2}}-X_{\bm{s},A_{n}}^{\ell_{2}}\right)\left|W_{\bm{s},A_{n}}\right|\right]$