# Bernstein-type Inequalities and Nonparametric Estimation under Near-Epoch
Dependence

Zihao Yuan  Chair of Statistics and Econometrics, Faculty of Business
Administration, University of Hamburg  Martin Spindler  Chair of Statistics
and Econometrics, Faculty of Business Administration, University of Hamburg

###### Abstract

The major contributions of this paper lie in two aspects. Firstly, we focus on
deriving Bernstein-type inequalities for both geometric and algebraic
irregularly-spaced NED random fields, which contain time series as special
case. Furthermore, by introducing the idea of ”effective dimension” to the
index set of random field, our results reflect that the sharpness of
inequalities are only associated with this ”effective dimension”. Up to the
best of our knowledge, our paper may be the first one reflecting this
phenomenon. Hence, the first contribution of this paper can be more or less
regarded as an update of the pioneering work from xu2018sieve. Additionally,
as a corollary of our first contribution, a Bernstein-type inequality for
geometric irregularly-spaced $\alpha$-mixing random fields is also obtained.
The second aspect of our contributions is that, based on the inequalities
mentioned above, we show the $L_{\infty}$ convergence rate of the many
interesting kernel-based nonparametric estimators. To do this, two deviation
inequalities for the supreme of empirical process are derived under NED and
$\alpha$-mixing conditions respectively. Then, for irregularly-spaced NED
random fields, we prove the attainability of optimal rate for local linear
estimator of nonparametric regression, which refreshes another pioneering work
on this topic, jenish2012nonparametric. Subsequently, we analyze the uniform
convergence rate of uni-modal regression under the same NED conditions as
well. Furthermore, by following the guide of rigollet2009optimal, we also
prove that the kernel-based plug-in density level set estimator could be
optimal up to a logarithm factor. Meanwhile, when the data is collected from
$\alpha$-mixing random fields, we also derive the uniform convergence rate of
a simple local polynomial density estimator Cattaneo . (2020).

##  1 Introduction

In time and spatial series analysis, we often encounter different kinds of
dependence. A natural idea to simplify the problem is to introduce some
measurement of dependence which is sufficiently general such that a large
class of time or spatial series models can be included as special cases. The
idea of ” _Near-Epoch-Dependence_ ” (NED) was firstly initiated by
ibragimov1962some to model the dependence of regularly-spaced time series.
This concept is further utilized by billingsley2013convergence and
mcleish1974dependent,mcleish1975maximal. The NED condition is very general
which allows for non-identically distributed sequences and covers many
important processes that are not mixing, e.g., autoregressive (AR) and
infinite moving average (MA($\infty$)) processes under very general
conditions. It also includes some dynamic simultaneous equation models (cf
bierens2012robust, Chapter 5 and gallant2009nonlinear, pp 481, 502, 539).
Particularly, $\alpha$-mixing may even fail under relatively regular
conditions. gorodetskii1978strong demonstrated that the $\alpha$-mixing
property fails in the case of continuously distributed innovations when the
coefficients of linear process do not fall sufficiently fast.
andrews1985nearly also proved that the AR(1) process with independent
Bernoulli innovations is not $\alpha$-mixing. But the NED condition is
satisfied in these two situations based on Proposition 1 of
jenish2012nonparametric. There are also many other interesting regularly-
spaced time series models that satisfy the NED condition. hansen1991garch
proved that GARCH(1,1) process is NED. In Sections 4.3, 4.5 and 4.6 lilocal
shows that AR(1)-NARCH(1,1), multivariate MA($\infty$) and semiparametric
ARCH($\infty$) proposed by Linton Mammen (2005) are also NED processes.

Unlike the regularly-spaced NED time series mentioned above, with the
development of spatial econometrics, irregularly-spaced NED random fields
became more crucial in many non-linear spatial processes. jenish2012spatial
proved that both linear autoregressive random field and nonlinear infinite
moving average random field are NED random fields. jenish2012nonparametric
showed that, under mild moment and Lipschitz conditions, the nonlinear
autoregressive random field is a NED random field as well, which includes many
widely used spatial auto-regressive(SAR) models as special cases. More
recently, xu2015spatial studied the NED properties of SAR models with a
nonlinear transformation of the dependent variable. xu2015maximum indicates
that under very mild conditions on disturbance and weight matrix, the
dependent variable in the SAR Tobit model is a $L^{2}$-NED random field on the
disturbance.

There has been some work on nonparametric estimation and prediction under NED
dependence. andrews1995nonparametric established uniform convergence rate of
kernel density estimator and Nadayara-Watson estimator under NED conditions.
lu2001asymptotic obtained asymptotic normality of kernel density estimator
under the regularly-spaced NED process. A strong law of large number and a
strong invariance principle for NED process were obtained by ling2007testing.
For local regression on regularly-spaced NED time series, lu2007local and
li2012local investigate pointwise and uniform convergence rates of local
linear fitting and proved that, under some mild conditions of dependence,
stone1980optimal’s optimal bound is attainable. More recently, ren2020local
showed the asymptotic normality of local linear estimator for quantile
regression and establish the Bahadur representation of this estimator in a
weak convergence sense.

However, all of the work mentioned above are based on regularly-spaced time
series. For NED random fields results are surprisingly missing in the
literature. hallin2001density analyze marginal density estimation of
regularly-spaced spatial linear processes, which are only a special example of
NED processes. Later, hallin2004kernel extended the previous research to
regularly-spaced nonlinear spatial process with iid innovations. But this
setting is still more restrictive compared with Definition 1 in Section 2. For
irregularly-spaced NED random fields, the ground breaking pioneering work
belongs to jenish2012spatial, which thoroughly derives uniform law of large
number, central limit theorem and their applications to asymptotic behaviors
of general GMM estimators. Based on this, jenish2012nonparametric investigated
the asymptotic normality and uniform convergence rate of local linear fitting
under nonstationary irregularly-spaced NED random fields.

Generally speaking, almost all of the previous research on nonparametric
regression under the NED condition is kernel-based. The methods used to obtain
pointwise or uniform theoretical guarantees of estimators can be generally
categorized into two groups: The first approach is to construct an
”approximate” process whose properties are already known. Due to the
definition of NED condition, the approximation error between the approximate
process and original NED process convergences to 0 in $L^{p}$ sense at a
higher order. Thus, we can focus solely on the approximate process, which is
assumed to be $\alpha$-mixing in this paper. (cf. lu2007local, li2012local,
xu2015maximum and xu2018sieve). Another approach is motivated by
parzen1962estimation, bierens1983uniform and andrews1995nonparametric. Since
kernel is often assumed to be a probabilistic density function as well, we can
make use of the inverse formula of the characteristic function to rewrite the
kernel based estimator. This method is also used in jenish2012nonparametric.
But it seems that the second approach does not allow Jenish to obtain a
sufficiently sharp uniform convergence rate of local linear estimators like
li2012local. Additionally, considering the use of inverse formula is strongly
supported by the assumption that the kernel is a density, the applicability of
the second method is not so strong as the first one. However, neither of the
two methods can be easily applied to any kind of estimators.

xu2018sieve has pointed out that having a exponential inequality on
irregularly-spaced NED random fields is vital for proving the consistency of
sieve-based estimators in many interesting spatial regression models. Indeed
things would be much easier if we can discover some concentration inequalities
directly about NED process, particularly irregularly-spaced NED random fields.
Lemma 5.1 in gerstenberger2018robust offers us a probabilistic inequality for
NED processes under very weak dependence assumption. But it only considers the
case where the data is a regularly-spaced time series and the approximate
process is $\beta$-mixing, which is not general enough to cover our setting.
xu2018sieve did ground breaking work on this topic. By employing
doukhan1999new’s combinatoric techniques, they proved an exponential
inequality on geometric irregularly-spaced NED random fields at the order of
$e$(see Definition 1). Equipped with this result, they prove the consistency
of sieve maximum likelihood estimator of spatial auto-regressive Tobit model.

Unfortunately, there are still some possible shortcomings of their results.

  1. S1

Their method heavily relies on the assumption that the NED condition is
geometric which excludes many interesting time and spatial series.

  2. S2

Their result may not be able to obtain sufficiently sharp uniform convergence
rates of many widely used estimators. For example, according to li2012local,
optimality of the local linear estimator could be obtained even under
algebraic regularly-spaced NED time series. But xu2018sieve’s result could not
provide this.

  3. S3

The decreasing rate of their inequality is
$\exp\\{-\text{constant}N^{1/2d+2}\epsilon^{2}\\}$, for $\forall\ \epsilon>0$,
where $d$ denotes the dimension of the index set. This directly implies that
the convergence rate is affected by the dimension of the index set of the
random field, which may not be true. For example, when $d=2$ and we
concentrate solely on selecting the locations along X-axis in $R^{2}$, similar
to time series, the index set now has linear order. Hence, the upper bound of
exponential inequality of averaged partial sums should be at the same level as
in time series case.

Therefore, in order to extend the work done by Xu Lee and important previous
research about nonparametric estimation under NED condition, the contributions
of this paper lie in two directions: Bernstein-type inequalities for both
geometric and algebraic NED random fields and theoretical guarantees of some
interesting nonparametric estimators beyond locally weighted least square
estimator for conditional mean function. More specifically, They can be
summarized as follows,

  1. 1.

By adding mild assumption(Assumption 3, 4), we use an alternative approach to
derive Bernstein-type inequalities for both geometric and algebraic
irregularly-spaced NED random fields. For the geometric case, our result holds
for arbitrary $L^{p}$-NED condition, $p\geq 1$. All of the new results would
be adaptive to the situation similar to the example in S3. More specifically,
our result shows that, the decreasing rate of probabilistic upper bound would
only be weakly affected by ”effective dimension”(see. Assumption 3).
Additionally, we also demonstrated a Bernstein-type inequality for geometric
irregularly-spaced $\alpha$-mixing random field, which refreshes the previous
results like adamczak2008tail, modha1996minimum and merlevede2011bernstein.

  2. 2.

To honor the previous research, we revisit local linear fitting of the
conditional mean function under irregularly-spaced NED random fields. Uniform
convergence rates are obtained under algebraic NED conditions. Our result
indicates that, under standard smoothing conditions, stone1982optimal’s
optimal rate could be attained. This section takes up and extends the results
from jenish2012nonparametric, li2012local and lu2007local.

  3. 3.

As an excellent application of the idea of local polynomial fitting,
cattaneo2020simple proposed a density estimator which generically solves the
problem of ”boundary effects”. Hence, in this paper, we investigate its
uniform convergence rate under geometrical irregularly-spaced $\alpha$-mixing
random fields.

  4. 4.

Considering that, at a high-level, conditional modes can reveal structure that
is missed by the conditional mean, we firstly obtain a non-asymptotic uniform
convergence rate of kernel density estimator for geometric irregularly-spaced
NED random fields. Then, based on this, uniform convergence rate of kernel-
based uni-modal regression is derived.

  5. 5.

rigollet2009optimal shew the attainability of optimal rate of kernel-based
plug-in density level estimator for iid data. We here prove that even when the
data is collected from irregularly-spaced geometric NED random field, the rate
is at least optimal up to a logrithm factor.

The rest part of this paper is organized as follow: Section 2 introduces the
definition of irregularly-spaced NED random fields and some useful properties
of them. In Section 3, contribution 1 will be thoroughly discussed. Section 4
to 6 exhibit the application of the inequalities shown in Section 3, which
will cover the contribution 2 to 5. At last, the proof of all the results
demonstrated in this paper is given in Appendix.

##  2 Irregularly-Spaced NED Random Fields

Before we start to display the definition of irregularly-spaced NED random
fields, we firstly introduce the following assumptions about the ”choice of
locations”.

###### Assumption 1

Individual units are located at
$\Gamma_{N}\subset\Gamma\subset\textbf{R}^{d}$, where $d$ is any positive
integer. Here $\Gamma_{N}:=\\{s_{iN},i=1,2,...,N\\}$ is the set of chosen
locations.

Apparently, assumption 1 defines irregularly-spaced locations. The set
$\Gamma$ here could be any region of interest on some specific time period,
geographical space or more abstract economic characteristics. For example,
when $d=1$, $\Gamma$ could be an union of multiple intervals; when $d\geq 2$,
$\Gamma$ might become an union of many geographical areas. In a nutshell, the
reason why we introduce $\Gamma$ here is to rigorously define ”available
region” for sampling and $\Gamma_{N}\subset\Gamma$ directly represents this
idea. But until now, we only know $\Gamma$ is a subset of $\mathbf{R}^{d}$. A
natural question is whether the area of $\Gamma$ can be finite or not. In this
paper, the answer is ”No” since we need the following assumption 2.

###### Assumption 2

For any $s_{iN},s_{jN}\in\Gamma_{N}$ and $i\neq j$, we assume,
$||s_{iN}-s_{jN}||>d_{0}$ for some $d_{0}>0$.

Here, $||\cdot||$ could be any norm defined on $\mathbf{R}^{d}$. Without loss
of generality, in this paper it is assumed as Euclidean norm. This assumption
actually indicates that, infinitely many selected locations could never be
contained by a bounded subset of $\mathbf{R}^{d}$. In the literature of
spatial statistics, this is often addressed as ”domain-expand asymptotics”Lu
Tjøstheim (2014). A formal statement of this property is given in the
following proposition, which is a modification of Lemma A.1 in
jenish2009central.

###### Proposition 1

Suppose Assumptions 1 and 2 hold. Then,

  1. (i)

By denoting $I(s,\frac{\sqrt{2}d_{0}}{2})$ as a closed cube centered at
location $s\in\Gamma$ with length of each edge as $\frac{\sqrt{2}d_{0}}{2}$,
we have $|I(s,\frac{\sqrt{2}d_{0}}{2})\cap\Gamma_{N}|\leq 1$.

  2. (ii)

There exists a constanct $C<+\infty$ such that for $h\geq 1$

$\sup_{s\in\Gamma}|I(s,h)\cap\Gamma_{N}|\leq Ch^{d},\
C=(2\sqrt{2}/d_{0})^{d}.$

###### Assumption 3

Based on the $N$ locations in $\Gamma_{N}$, we assume that, for any $N$, no
more than $m_{0}$-many closed rectangles are needed to contain set
$\Gamma_{N}$, $m_{0}\in\mathbf{N}^{+}$. For each of these rectangles, say
$R_{N}^{l}$, $l=1,\dots,m_{0}$, we also assume there are
$d_{1}^{l}$($d_{1}^{l}\geq 0$) out of $d$ edges whose length are
asymptotically bounded and the length of the rest $d-d_{1}^{l}:=d_{2}^{l}$
edges will diverge to infinite as $N\rightarrow+\infty$. More specifically,
suppose these $d_{1}^{l}$ edges are the $k_{i}$-th edges of $R_{N}^{l}$,
$i=1,..,d_{1}^{l}$. Without loss of generality, we assume for $\forall\
i=1,..,d_{1}^{l}$,$H_{k_{i}}^{N}(l)=H_{0}(l)\in[1,+\infty)$, where
$H_{k_{i}}^{N}(l)$ denotes the length of the $k_{i}$-th edge for every N. In
this paper, we define $d_{2}:=\max_{1\leq l\leq m_{0}}d_{2}^{l}$ as the
”effective dimension” of index set $\Gamma_{N}$. Additionally, for each $l$,
by denoting the length of these $d_{2}^{l}$ edges as
$H_{k_{j}}^{N}(l),j=1,\dots,d_{2}^{l}$, we assume $\max_{1\leq j\leq
d_{2}^{l}}H_{k_{j}}^{N}(l)\leq\max_{i\neq j}||s_{iN}-s_{jN}||$ and
$\min_{1\leq j\leq d_{2}^{l}}H_{k_{j}}^{N}(l)\geq H_{0}$, provided that
$s_{iN},s_{jN}\in\Gamma_{N}\cap R_{N}^{l}$.

Please note, according to Assumption 3, we actually get multiple sequences of
closed rectangles, $\\{R_{N}^{l}\\}_{N\in\mathbf{N}^{+}}$,
$l=1,2,\dots,m_{0}$. But for any given $l,\ N,\ M\in\mathbf{N}^{+}$,
$R_{N}^{l}$ does not need to be geometrically parallel to $R_{M}^{l}$.
Subsequently, the reason why we introduce this rectangle is because, unlike
regularly-spaced data, irregularity here forbids us to capture the exact
information about the locations. Hence, the major tool of our proof, Bernstein
Blocking technique could not be directly applied. Due to Assumption 2, by
introducing the basic cubes to ”group” the locations, we can rewrite the
partial sum of random variables as summation of ”groups”. This simple trick
easily changes irregularly-spaced setting into regularly-spaced setting.
Obviously, $\min_{j}H_{k_{j}}^{N}\geq H_{0}$ is a condition without loss if
generality since $\lim\min_{j}H_{k_{j}}^{N}=+\infty$. As for the restriction
$\max_{j}H_{k_{j}}^{N}\leq\max_{i\neq j}||s_{iN}-s_{jN}||$, it means the
volume of rectangle $R_{N}$ is no larger than $H_{0}^{d_{1}}(\max_{i\neq
j}||s_{iN}-s_{jN}||)^{d_{2}}$. By combing it with Assumption 4 below, we can
easily figure out $Vol(R_{N})=O(N)$, which builds a bridge between $R_{N}$ and
sample size.

As for the reason why we define the ”effective dimension” $d_{2}$ in this way,
note we have assumed that, for any $N$, no more than $m_{0}$-many rectangles
are needed to contain all the selected locations, for some positive integer
$m_{0}$. Then, based on Definition 1, we have

$\displaystyle\mathbb{P}:=\mathrm{P}\left(\left|\frac{1}{N}\sum_{i=1}^{N}Z_{s_{iN}}\right|>t\right)\leq\sum_{l=1}^{m_{0}}\mathrm{P}\left(\left|\frac{1}{N}\sum_{s_{i}\in
R_{N}^{l}}Z_{s_{iN}}\right|>\frac{t}{m_{0}}\right):=\sum_{l=1}^{m_{0}}\mathbb{P}_{l}.$

Since $m_{0}$ is a fixed number, we have $\mathbb{P}\leq m_{0}\max_{1\leq
l\leq m_{0}}\mathbb{P}_{l}$. However, please note that, for any $l$, the
cardinality of set $\\{s_{iN}\in R_{N}^{l}\\}$, denoted as $|R_{N}^{l}|$, is
strictly smaller than $N$ or even $|R_{N}^{l}|/N=o(1)$, which indicates that
$\max_{l}\mathbb{P}_{l}\lesssim\mathbb{P}$.111$A\lesssim B$ indicates $A\leq
CB$ for any $N$ and some positive constant $C$. Hence,
$\mathbb{P}=O(\max_{l}\mathbb{P}_{l})$, which indicates that, given $N$, we
only need concern the ”effective dimension” of one specific rectangle.
Additionally, a good example is as Figure 1. As we can see, the selected
locations(black dots) are distributed along the three lines A, B and C.
Suppose the length of the three lines will diverge to infinitely long. Then we
can use three rectangles whose ”effective dimension” is one to contain all of
these locations. This indicates that the ”effective dimension” of the set of
these locations is 1. Together with Theorem 1 and Theorem 2, if the index set
of a NED random fields are distributed as Figure 1, we know the sharpness of
Bernstein-type inequalities would be like time series, in which the ”effective
dimension” is always one. Of course, this example is only a tip of the ice
berg. Many other interesting examples can be discovered.

Figure 1: 1

###### Assumption 4

Given $\Gamma_{N}$, we assume $\max_{i\neq j}||s_{iN}-s_{jN}||\leq
KN^{\frac{1}{d_{2}}}$, for some positive constant K. Here, without loss of
generality, for convenience in the proof, we assume $K=H_{0}/2$.

Assumption 4 is dedicated to ensure that the distribution of the chosen
locations is not too sparse. Note that, the $K$ here could be arbitrary fixed
positive constant. Changing $K$ donesn’t affect the sharpness of the
inequalities in Theorems 1 and 2 but only the constant factors.

For any random vector $v\in\mathbf{R}^{D}$,
$||v||_{p}=[E||v||^{p}]^{\frac{1}{p}}$ denotes the $L^{p}$-norm of Euclidean
norm of vector $v$ with respect to probabilistic measure. For any subset
$E\subset\mathbf{R}^{d}$, denote $|E|$ its cardinality. Based on assumptions
1, We introduce the definitions and properties of $\mathbf{R}^{D}$-valued
($D\geq 1$) NED random fields as follow.

###### Definition 1

Let $Z=\\{Z_{s_{iN}},s_{iN}\in\Gamma_{N}\\}$ be a $\mathbf{R}^{D}$-valued
random field and $\epsilon=\\{\epsilon_{s_{iN}},s_{iN}\in\Gamma_{N}\\}$ also
an random field, where $\Gamma_{N}$ satisfies Assumption 1.
$\mathcal{F}_{i,N}(r)$ is denoted as a $\sigma$-field generated by the random
vectors $\epsilon_{s_{jN}}$ satisfying that $||s_{jN}-s_{iN}||\leq r$. $Z$ is
said to be $L^{p}$-near-epoch dependent ($L^{p}$-NED) on $\epsilon$ if, for
some $p>0$, we have

$||Z_{s_{iN}}-E(Z_{s_{iN}}|\mathcal{F}_{iN}(r)||_{p}\leq\alpha_{iN}\psi_{Z}(r).$
(1)

Here $\\{\alpha_{iN}\\}$ is some array of finite positive constants. Function
$\psi_{Z}(r)$ is a positive-valued non-increasing function that satisfies
$\lim_{r\rightarrow+\infty}\psi_{Z}(r)=0$. We call $Z$ a geometric or
algebraic irregularly-spaced $L^{p}$ NED random field on $\epsilon$ at the
order of $\nu$, when $\psi_{Z}(r)\leq\nu^{-br^{\gamma}}$($\nu>1,\ b,\
\gamma>0$) or $\psi_{Z}(r)\leq r^{-\nu}$($\nu>0$).

Definition 1 is directly borrowed from jenish2012spatial. As we mentioned in
Section 1, a widely used setting is to assume random field $\epsilon$ as a
strong mixing random field. In this paper, this setting will also be adopted.
Thus, in order to prepare the next sections, we review the definition of a
strong mixing random field first.

###### Definition 2

Given $\Gamma_{N}$ satisfying Assumption 1, we say a process
$\epsilon:=\\{\epsilon_{s_{iN}},s_{iN}\in\Gamma_{N}\\}$ is an $\alpha$-mixing
random field, if $\forall\ U,\ V\subset\Gamma_{N}$, such that
$\max\\{|U|,|V|\\}<+\infty$, the following condition is satisfied,

$\displaystyle\alpha(\sigma(U),\sigma(V))$
$\displaystyle:=\sup\\{|P(AB)-P(A)P(B)|,A\in\sigma(U),B\in\sigma(V)\\}$
$\displaystyle\leq\phi(|U|,|V|)\psi_{\epsilon}(\rho(U,V)),$

where $\sigma(U)$ denotes the sigma-algebra generated by $\epsilon_{s_{iN}}$s
whose coordinates belong to set $U$. Function
$\phi:\mathbf{N}^{2}\rightarrow\mathbf{R}^{+}$ is symmetric and increasing in
each of its two arguments and $\psi_{\epsilon}(\cdot)$ shares the same
property as function $\psi_{Z}(\cdot)$ defined in Definition 1. Further,
$\rho(U,V)=\inf\\{||u-v||,u\in U,v\in V\\}$, where $||\cdot||$ denotes the
Euclidean norm.

Based on the two definitions above, we can obtain the following propositions
which will be very useful in deriving concentration inequalities for NED
random fields later.

###### Proposition 2

Given random fields $Z=\\{Z_{s_{iN}},s_{iN}\in\Gamma_{N}\\}$ and
$\epsilon=\\{\epsilon_{s_{iN}},s_{iN}\in\Gamma_{N}\\}$, where $\Gamma_{N}$
satisfies Assumption 1 and
$Z_{s_{iN}}=(Z^{1}_{s_{iN}},...,Z^{D}_{s_{iN}})\in\mathbf{R}^{D}$, $Z$ is
$L^{p}$-NED on $\epsilon$ if and only if for every $k=1,...,D$,
$\\{Z^{k}_{s_{iN}},s_{iN}\in\Gamma_{N}\\}$ is also $L^{p}$-NED on $\epsilon$.

###### Proposition 3

Suppose random field $Z$ is $L^{p}$-NED on random field $\epsilon$ and for any
$s_{iN}\in\Gamma_{N}$, $Z_{s_{iN}}\in\mathcal{Z}\subset\mathbf{R}^{D}$. Let
$f$ be any real-valued measurable function whose domain is $\mathcal{Z}$. If
$f$ is Lipschitz continuous on $\mathcal{Z}$ with respect to Euclidean norm
and $Lip(f)$ as the Lipschitz constant, random field
$f(Z):=\\{f(Z_{s_{iN}}),s_{iN}\in\Gamma_{N}\\}$ is also $L^{p}$-NED on
$\epsilon$. Furthermore, we have

$||f(Z_{s_{iN}})-E[f(z_{s_{iN}})|\mathcal{F}_{iN}(r)]||_{p}\leq
2Lip(f)\alpha_{iN}\psi_{Z}(r).$ (2)

###### Proposition 4

Suppose random field $Z$ is $L^{p}$-NED on strong mixing random field
$\epsilon$ satisfying Definition 2 and for any $s_{iN}\in\Gamma_{N}$,
$Z_{s_{iN}}\in\mathcal{Z}\subset\mathbf{R}$. For $\forall\ U,\
V\subset\Gamma_{N}$, such that $\max\\{|U|,|V|\\}<+\infty$, denote
$U=\\{s_{iN},i=1,...,|U|\\}$, $V=\\{t_{jN},j=1,...,|V|\\}$. Let
$g:\mathcal{Z}^{|U|}\rightarrow R$, $h:\mathcal{Z}^{|V|}\rightarrow R$ be two
coordinate wise Lipschitz functions with $Lip(g)$ and $Lip(h)$ as Lipschitz
constants. When $p\geq 2$, for some $\delta>0$ and $r<\rho(U,V)/2$, we have

$\displaystyle\left|Cov(g(Z_{s_{1N}},...,Z_{s_{|U|N}}),h(Z_{t_{1N}},...,Z_{t_{|V|N}}))\right|$
$\displaystyle\leq
4Lip(f)Lip(g)\begin{matrix}(\sum_{i=1}^{|U|}\alpha_{s_{iN}})((\sum_{j=1}^{|V|}\alpha_{t_{jN}}))\end{matrix}\psi_{Z}^{2}(r)$
$\displaystyle+4Lip(g)\begin{matrix}(\sum_{i=1}^{|U|}\end{matrix}\alpha_{s_{iN}})\psi_{Z}(r)||h||_{p}$
$\displaystyle+4Lip(f)\begin{matrix}(\sum_{j=1}^{|V|}\end{matrix}\alpha_{t_{jN}})\psi_{Z}(r)||g||_{p}$
$\displaystyle+\phi(|U|,|V|)\psi_{\epsilon}^{\frac{p+\delta-2}{p+\delta}}(\rho(U,V)-2r)||g||_{p+\delta}||h||_{p+\delta}.$
(3)

Proposition 2 is obvious and easy to be proved by only using the basic
property of conditional expectations and the relationship between $L^{1}$ and
Euclidean norm in $\mathbf{R}^{D}$. Proposition 3 works as a foundation of
Proposition 4 and allows us to concentrate solely on real-valued NED processes
when the statistic can be described as an empirical process whose index set is
Lipschitz class. Proposition 4 is actually a refreshed result of Lemma A.2 in
xu2018sieve. The only difference is that our (3) relies on $L^{p}$-norm while
their result depends on $L^{\infty}$-norm. Proposition 2 allows us to directly
apply concentration inequalities displayed in Section 3 to many interesting
estimators of regression functions. After all, under mild conditions, most of
them can be regarded as a Lipschtiz function on the data, like kernel or kNN
smoother, hermite polynomial or many other orthogonal series estimator.

##  3 Bernstein-type Inequalities for NED

This section introduces Bernstein-type inequalities for geometric and
algebraic irregularly-spaced NED random fields. Compared to a Hoeffding-type
inequality, a significant advantage of a Bernstein-type inequality is that it
allows for localization, due to its specific dependence on the variance term
of random variables. Thus, in the asymptotic or non-asymptotic analysis of
many estimators, optimal rate is often achieved by applying Bernstein-type
inequalities.

Starting with carbon1983inegalite and collomb1984proprietes, during the last
three decades, there have been various generalizations of Bernstein-type
inequalities to different types of dependent stochastic process. Since only
irregularly-spaced NED random fields with respect to $\alpha$-mixing process
is discussed in this paper , here we only review the previous research about
Bernstein-type inequalities on $\alpha$-mixing processes. Based on coupling
and blocking techniques, rio1995functional and liebscher1996strong derived
Bernstein-type inequalities for stationary real-valued $\alpha$-mixing
processes under very general assumption of decay of dependence. bryc1996large
investigated the large deviation phenomenon for $\alpha$-mixing process under
exponential decay. According to a combination of Cantor set and Bernstein
Blocking technique, merlevede2009bernstein explored a Bernstein-type
inequality under geometric $\alpha$-mixing conditions. More recently,
valenzuela2017bernstein and krebs2018bernstein extended this work to geometric
strong mixing spatial lattice process and exponential graph. However, all of
the previous results only focus on the regularly-spaced case. Unfortunately,
research about Bernstein-type inequalities or more general concentration
inequalities for irregularly-spaced data are surprisingly few. According to
martingale approach, delyon2009exponential developed an exponential inequality
for irregularly-spaced mixing random fields. But the preconditions of this
result are hard to be specified and does not include the NED condition as
well. Then, as mentioned in Section 1, xu2018sieve delivered the first direct
result about exponential inequality for geometric irregularly-spaced NED
random fields.

Our inequalities generally share the same form as rio1995functional and
liebscher1996strong, where the upper bound is composed of an exponential part
and a remainder. Together with the blocking technique, the major idea of our
proof of Theorem 1 and 2 is making use of the definition of NED. This is quite
similar to the tricks demonstrated in rio1995functional. More specifically, by
following Definition 1, suppose $Z$ is an irregularly-spaced $L^{p}$ NED
random field on an $\alpha$-mixing random field $\epsilon$. Our result
indicates that the probabilistic upper bound of averaged partial sum process
is actually governed by the concentration phenomenon of the random field
$\epsilon$. Indeed this is intuitive according to the Definition 1. Note, for
each $Z_{s_{iN}}\in Z$, its projection error, i.e. projection on the
orthogonal complete space of $\mathcal{M}(\mathcal{F}_{iN}(r))$, will converge
to 0 with respect to $L^{p}$ norm, as $r\nearrow+\infty$, where
$\mathcal{M}(\mathcal{F}_{iN}(r))$ denotes the collection of all real-valued
$\mathcal{F}_{iN}(r)$ measurable functions. If we let $r$ be non-decreasing
with respect to sample size $N$, we thus ensure random field(array) $\epsilon$
is an approximation of field(array) $Z$ under $L^{p}$-norm.

Therefore, the assumption of the quantity of p becomes crucial. Many previous
literature assumes $p\geq 2$, e.g. lu2007local, li2012local and xu2018sieve.
Recently, ren2020local investigates the asymptotic properties of local linear
quantile regression only under the condition $p=1$. In this paper, except for
the theorems in Sections 3.2 and 4.1, we always assume $p\geq 1$, which
includes most of the previous research.

###  3.1 Geometric NED

Now we present the Bernstein-type inequality for irregularly-spaced geometric
NED random fields. Our method can be considered as a modification of the proof
of Theorem 3.1 in krebs2018bernstein.

###### Theorem 1

(Geometric NED) According to Definition 1, suppose
$Z:=\\{Z_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$ is a real-valued geometric
irregularly-spaced $L^{p}$($p\geq 1$) NED random field on
$\epsilon:=\\{\epsilon_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$ at the order of
$\nu$. For any $i$ and $N$, assume $||Z_{s_{iN}}||_{\infty}\leq A$,
$E[Z_{s_{iN}}^{2}]\leq\sigma^{2}$ and $E[Z_{s_{iN}}]=0$. $\epsilon$ is defined
as Definition 2. Furthermore, assume $\max_{i,N}\alpha_{iN}\leq N^{\kappa}$,
for some $\kappa>0$, $\phi(x,y)=(x+y)^{\tau},\ \tau>0$ and
$\psi_{Z}(t)=\psi_{\epsilon}(t)=\nu^{-bt^{\gamma}}$. Then, under Assumptions 1
to 4, for $\forall\ N\geq N_{0}\in\mathbf{N}^{+}$ and $\forall\ \beta\geq 0$,
we have,

$\displaystyle\mathrm{P}\left(\left|N^{-1}\sum_{i=1}^{N}Z_{s_{iN}}\right|\geq
t\right)$ $\displaystyle\leq
2K_{1}\left(\frac{1}{N^{\kappa+2\beta+\tau+1}t}\right)^{p}$
$\displaystyle+4\exp\left(-\frac{Nt^{2}}{K_{2}(\log_{\nu}N)^{\frac{d_{2}}{\gamma}}((C_{0}\sigma)^{2}+\frac{4C_{0}At}{3})}\right)$
(4)

where positive integer $N_{0}$ is specified in the proof and

$\displaystyle
K_{1}=\left(\frac{C_{0}}{H_{0}^{d_{2}}}\right)^{\frac{2(\kappa+\beta)+\tau+1}{p}},\
K_{2}=2^{8+d_{2}}\left(C_{0}\left(\frac{2(\kappa+\beta)+\tau+1}{b}\right)^{\frac{1}{\gamma}}\right)^{d_{2}},$
$\displaystyle
C_{0}=\left(\left[\frac{H_{0}}{\sqrt{2}d_{0}/2}\right]+1\right)^{d}.$

###### Remark 1

Please note that even though the first term of the upper bound seems to be at
polynomial rate, it actually plays a role as a higher order term in deriving
the convergence rate of many estimators. The reason for it is that Theorem 1
is valid for arbitrary $\beta\geq 0$. When we apply the Bernstein-inequality
to investigate the convergence rate of estimators, say $a_{N}$, a common step
is to replace $t$ with $a_{N}$. Until now, almost all the estimators converge
no faster than $N^{-\eta}$, for some $\eta>0$. In fact, according to to the
second term, $\eta$ can not be set as some real number larger than $1/2$,
which has already been at parametric rate. Thus, we can easily find a
sufficiently large $\beta$ such that, after setting $t=a_{N}$, the first term
is at a higher order rate compared with the second term. However, no matter
how larger the $\beta$ we select, it only enlarges the constant $K_{2}$ in the
second term and does not change the sharpness of the second term
asymptotically.

###### Remark 2

Unlike usually defined uniform NED condition (see Definition 1,
jenish2012spatial), the assumption $\max_{i,N}\alpha_{iN}\leq N^{\kappa}$ is
more general and could be more useful in some applications. For example,
suppose $Z=\\{Z_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$ is a $\mathcal{Z}$-valued
$L^{p}$ irregularly-spaced NED random field on
$\epsilon=\\{\epsilon_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$,
$\mathcal{Z}\subset\mathbf{R}^{D}$. Let $f:\mathcal{Z}\rightarrow\mathbf{R}$
be a Lipschitz continuous function with module $Lip(f)$. Then according to
Proposition 2, we know $\\{f(Z_{s_{iN}})\\}_{s_{iN}\in\Gamma_{N}}$ can be
regarded as a real-valued NED process on $\epsilon$ and
$||f(Z_{s_{iN}})-E[f(Z_{s_{iN}})|\mathcal{F}_{iN}(r)]||_{p}\leq
2Lip(f)\psi_{Z}(r)$. But sometimes $Lip(f)$ may diverge to infinity as
$N\rightarrow+\infty$. A typical example is smoothing kernel function from
Lipschitz class, $K(Z_{s_{iN}}-z/h)$, whose Lipschitz module is $O(1/h)$.

Actually the first term on the right side of 1 denotes the approximation error
between process $Z$ and $\epsilon$ while the second one is an upper bound of
concentration phenomenon of projection process
$\\{E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]\\}_{{s_{iN}}\in\Gamma_{N}}$, which is an
$\alpha$-mixing sequence. Therefore, a natural corollary of Theorem 1 is as
follow,

###### Corollary 1

(Geometric $\alpha$-Mixing) Suppose
$\epsilon:=\\{\epsilon_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$ is a mean-zero and
real-valued $\alpha$-mixing random field that satisfies, for any $i$ and $N$,
$||\epsilon_{s_{iN}}||_{\infty}\leq A$ and
$E[\epsilon_{s_{iN}}^{2}]\leq\sigma^{2}$. Assume $\phi(x,y)=(x+y)^{\tau}$ and
$\psi_{\epsilon}(t)=\nu^{-bt^{\gamma}}$, for some $\tau,\ \gamma,\ b>0$ .
Then, under Assumptions 1 to 4, we have

$\displaystyle\mathrm{P}\left(\left|N^{-1}\sum_{i=1}^{N}\epsilon_{s_{iN}}N\right|\geq
t\right)\leq
4\exp\left(-\frac{Nt^{2}}{K_{2}(\log_{\nu}N)^{\frac{d_{2}}{\gamma}}((C_{0}\sigma)^{2}+\frac{4C_{0}At}{3})}\right),$
(5)

where $K_{2}$ and $C_{0}$ maintain the same compared with Theorem 1.

In order to be better compared with the pioneering work from Xu Lee, based on
our notation, we firstly brief their result as follow.

(Theorem A.1 xu2018sieve) Under Assumptions 1 and 2, suppose random field
$\\{Z_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$ is a real-valued random field
satisfying $E[Z_{s_{iN}}]=0$ and $\sup_{i,N}||Z_{s_{iN}}||_{\infty}\leq A$.
Also assume for any $U=\\{s_{iN},i=1,...,|U|\\}$,
$V=\\{t_{jN},j=1,..,|V|\\}\subset\Gamma_{N}$,

$\displaystyle
Cov\left(\begin{matrix}\prod_{i=1}^{|U|}Z_{s_{iN}}\end{matrix},\
\begin{matrix}\prod_{i=1}^{|V|}Z_{t_{jN}}\end{matrix}\right)\leq
C^{*}e^{|U|+|V|}\exp(-a\rho(U,V))A^{|U|+|V|},$ (6)

for some $C^{*}$ and $a>0$. Then we have

$\displaystyle\mathrm{P}\left(\left|N^{-1}\sum_{i=1}^{N}Z_{s_{iN}}N\right|\geq
t\right)\leq C_{1}^{*}\exp\left(-C_{2}^{*}N^{\frac{1}{2d+2}}t^{2}\right),$ (7)

where both $C_{1}^{*}$ and $C_{2}^{*}$ are positive constants but $C_{2}^{*}$
has nothing to do with variance information.

The advantage of (1) and (5) are relatively obvious. Firstly, they are
adaptive to the counterexamples mentioned in the third limitation of (7) in
Section 1. More specifically, only the ”effective dimension” would really
affect the sharpness of inequality up to a $\log_{\nu}N$ level, which is
apparently sharper than (7). Secondly, Theorem 1 holds for NED condition under
$L^{p}$-norm for any $p\geq 1$, while (7) is only valid for the case $p\geq
2$. Meanwhile, as shown in the applications, (1) and (5) can be easily used to
investigate of the attainability of optimality for many widely used
estimators. Furthermore, unlike (7) disallowing for localization while our
results would not reject us to obtain sufficiently sharp convergence rate of
estimators whose variance becomes arbitrary small asymptotically, like kernel
density estimator. At last, (1) and (5) are valid for any $\nu>0$, while
according to the proof of (7), $\nu=e$ is a necessary setting. Generally
speaking, Theorem 1 and Corollary 1 almost solve the mentioned shortcomings of
xu2018sieve(see S1-S3).

Unfortunately, most of time, there is no ”free lunch”. xu2018sieve directly
obtained an exponential inequality on a irregularly-spaced weakly-dependent
random field with zero mean and bounded support, which contains both NED and
$\alpha$-mixing random fields as special cases. The only preconditions about
the sampling of locations are Assumptions 1 and 2. Therefore, compared with
(1) and (5), (7) coincides with more general setting.

However, an important criterion for evaluating a ”trade-off” is to check
whether the additional restriction added here is sufficiently mild while the
gain is significant. In this paper, these additional restrictions are from two
directions. Firstly, compared with (1), (7) does not rely on Assumptions 3 and
4. But as we argued in Section 2, Assumption 3 may be seen as a natural
consequence of Assumption 2 and Assumption 4 holds for many practical
examples. Secondly, Theorem 1 and Corollary 1 concentrate solely on NED and
$\alpha$-mixing conditions while Xu Lee’s result is valid for any weakly-
dependent random field satisfying (6). But note the motivation for both (1)
and (7) is to derive a useful concentration inequality for geometric
irregularly-spaced NED random field. Hence our result automatically fits in
well with all the applications mentioned in xu2018sieve, particularly spatial
autoregressive Tobit model(SART). This, to some degree, reflects the
generality of our results (1) and (5) as well.

As for Corollary 1, compared with adamczak2008tail, modha1996minimum and
merlevede2011bernstein, our result is either sharper or valid for much more
general settings. It seems that delyon2009exponential’s result could be as
sharp as iid setting. But note Corollary 1 is derived solely based the
covariance inequality in Lemma 2, which is totally different from the
preconditions of Delyon’s work.

###  3.2 Algebraic NED

In addition to geometric NED, according to Proposition 1, (2) in
xu2015maximum, SART sometimes may only satisfy the algebraic NED condition,
which enlightens us to discover a Bernstein-type inequality for algebraic
irregularly-spaced NED random field. Before the exhibition Theorem 2, we
introduce the following assumptions and proposition.

###### Assumption 5

We assume $H_{0}$ defined by Assumption 3 satisfies $H_{0}\geq 4C_{0}$.

###### Proposition 5

Based on Assumption 3, suppose the edges which are going to be infinitely long
are the $k_{j}$-th edges, $j=1,2,...,d_{2}$. Let
$\hat{N}=\prod_{j=1}^{d_{2}}H_{k_{j}}^{N}$, where $H_{k_{j}}^{N}$ denotes the
length of the $k_{j}$-th edge of $R_{N}$. Then, under Assumptions 2 and 4, we
have,

$\displaystyle\frac{H_{0}^{d_{2}}N}{C_{0}}\leq\hat{N},\
\hat{N}\leq\frac{H_{0}^{d_{2}-d_{1}}}{2^{d_{2}}}N.$

Assumption 5 is just an assumption without loss of generality designed to make
the result more compact. Proposition 5 is nothing else but a direct result of
Assumption 2 and 4.

###### Theorem 2

(Algebraic NED) Suppose $Z:=\\{Z_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$ is a real-
valued algebraic irregularly-spaced $L^{s}$($s\geq 2$) NED random field on
$\epsilon:=\\{\epsilon_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$ at the order of
$\nu_{1}$, which indicates $\psi_{Z}(t)=t^{-\nu_{1}}$, for some $\nu_{1}>0$.
For any $i$ and $N$, $||Z_{s_{iN}}||_{\infty}\leq A$,
$||Z_{s_{iN}}||_{2}\leq\sigma$,
$||Z_{s_{iN}}||_{2+\delta}\leq\sigma_{2+\delta}$, and $E[Z_{s_{iN}}]=0$.
Denote $\max_{ij}|E[Z_{s_{iN}}Z_{s_{jN}}]|:=\overline{\sigma}_{ij}$.
$\epsilon$ is defined as Definition 2. Assume
$\max_{i,N}\alpha_{iN}\leq\alpha(N)$, $\phi(x,y)=(x+y)^{\tau}$ and
$\psi_{\epsilon}(t)=t^{-\nu_{2}}$, for some $\tau$ and $\nu_{2}>0$.
$\alpha(N)$ is a non-decreasing with respect to $N$. Set $q\in\mathbf{N}$,
where $H_{k_{j}}^{N}$’s are defined as Proposition 5. Then, under Assumptions
1-5, for some $C>0$, $\forall\ t>0$, we have,

$\displaystyle\mathrm{P}\left(\left|\frac{\sum_{i=1}^{N}Z_{s_{iN}}}{N}\right|\geq
t\right)\leq
2\exp\left(-\frac{H_{0}^{d_{1}}Nt^{2}}{32C_{0}\left(2^{d_{2}-1}v(q)+\frac{AH_{0}^{d_{1}}Nt}{12(2q)^{d_{2}}C_{0}}\right)}\right)$
$\displaystyle+11K_{3}\left(1+\frac{8AH_{0}^{d_{2}-d_{1}}}{t}\right)^{\frac{1}{2}}q^{d_{2}}\left(\frac{N}{q^{d_{2}}}\right)^{-\frac{\nu_{2}}{d_{2}}+\tau}+K_{4}\left(\frac{N}{q^{d_{2}}}\right)^{-\frac{s\nu_{1}}{d_{2}}}\left(\frac{\alpha(N)}{t}\right)^{s},$
(8)

where

$\displaystyle v(q)=\max\left\\{\sigma^{2}+\sigma_{2+\delta}^{2}B(N,q),\
3\left(C_{0}C^{**}\left(\frac{2}{3H_{0}}\right)^{d_{2}}\right)\frac{N}{q^{d_{2}}}(\overline{\sigma}_{ij}\lor\overline{\sigma}_{ij}^{2}\lor\frac{2}{3^{d_{2}}}\sigma\alpha(N)(\frac{N}{q^{d_{2}}})^{-\frac{\nu_{1}}{d_{2}}})\right\\},$
$\displaystyle
B(N,q)=C^{*}\left(\frac{N}{q^{d_{2}}}\right)^{\frac{-\nu_{2}\delta}{d_{2}(2+\delta)}+1},\
C_{p}\leq\frac{C_{0}}{2^{d_{2}}H_{0}^{d_{1}}}\left(\frac{N}{q^{d_{2}}}\right)\
\mbox{and}\ C_{p}=O\left(\frac{N}{q^{d_{2}}}\right).$ $\displaystyle K_{3},\
K_{4},\ C^{*},\ C^{**}\in\mathbf{R}^{+}\ \mbox{are specified in the proof}.$

The core idea for obtaining the Bernstein-type inequality under algebraic NED
condition is motivated by yao2003exponential, in which, blocking technique and
Bradley’s coupling lemma are combined to obtain an exponential inequality for
regularly-spaced $\alpha$-mixing random field. Unfortunately, Theorem 2 can
not generally help us to attain the optimality of $L^{\infty}$ convergence
rate of arbitrary estimator under NED condition. The major reason is that
$v(q)$ may not be sharp enough. We suppose this can not be improved based on
our approach. But, things can be much better if we focus on kernel smoothing.
The term $\overline{\sigma}_{ij}:=\max_{i,j}|E[Z_{s_{iN}}Z_{s_{jN}}]|$ in
$v(p)$ could explain the reason for that. Because, according to lemmas A.2 and
A.3 in lu2007local, lemma A.5 and Theorem 3.1 in li2012local, by letting
$Z_{s_{iN}}=K(X_{s_{iN}}-x/h)Y_{s_{iN}}$, we have
$E[Z_{s_{iN}}Z_{s_{jN}}]=O(h^{2D})$, provided $X\in\mathbf{R}^{D}$. This
result heavily relies on the convolution property of kernel smoother, which is
satisfied by many other estimators.

Due to the second and third term on the right side of (2), another significant
phenomenon displayed by (2) is the effect of $d_{2}$, which is defined as
”effective dimension”(see. Assumption 3). Meanwhile, note the rate of the
first term within the upper bound is independent of $d_{2}$. Therefore, under
algebraic NED condition, inequality (2) is also adaptive to effective
dimension $d_{2}$. Moreover, as shown later in Section 4.1, Theorem 2 is able
to lead us to obtain the optimality of local linear fitting for algebraic
irregularly-spaced NED random field, which too some degree extends the result
from jenish2012nonparametric.

##  4 Locally Polynomial Regression

Within this section, we will revisit local polynomial fitting for irregularly-
spaced dependent data. Section 4.1 is dedicated to discuss the uniform
convergence rate of local linear fitting of conditional mean function under
the algebraic NED condition, which completes or refreshes the results from
li2012local and jenish2012nonparametric. In Section 4.2, cattaneo2020simple
recently proposed a generically unbiased density estimator by using local
polynomial regression to the smooth empirical cumulative distribution function
of random variable. We will investigate its uniform convergence rate for the
geometric irregularly-spaced $\alpha$-mixing random field.

###  4.1 Conditional Mean Function Estimation

Nonparametric modeling on spatial data has attracted much attention in the
past two decades, particularly kernel-based local estimator. Within this
section, we will investigate the uniform convergence rate of local linear
fitting for algebraic irregularly-spaced NED random field. More specifically,
suppose we have the following regression model,

$\displaystyle Y_{s_{iN}}=m(X_{s_{iN}})+e_{s_{iN}},$ $\displaystyle
E[e_{s_{iN}}|X_{s_{iN}}]=0\ \mbox{and}\
E[e_{s_{iN}}^{2}|X_{s_{iN}}]=\sigma^{2}.$

According to lu2007local,li2012local and jenish2012nonparametric, given
$X_{s_{iN}}=x$, local linear estimator of $m(x)$ is defined as,

$\displaystyle\hat{m}(x)=(1,\textbf{0})\hat{\theta}(x),\
\hat{\theta}(x)=S_{N}(x)^{-1}T_{N}(x),$ $\displaystyle
S_{N}(x)=\frac{1}{N}\sum_{i=1}^{N}K_{ih}(x)U_{i}(x)U_{i}(x)^{T};\
T_{N}(x)=\frac{1}{N}\sum_{i=1}^{N}K_{ih}(x)U_{i}(x)Y_{s_{iN}};$ $\displaystyle
K_{ih}(x)=\frac{1}{h}K\left(\frac{X_{s_{iN}}-x}{h}\right);\
U_{i}(x)=\left(1,\left(\frac{X_{s_{iN}}-x}{h}\right)^{T}\right)^{T},$

where 0 is a D-mensional zero vector.

###### Assumption 6

$Z:=\\{X_{s_{iN}},Y_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$ is an algebraic and
irregularly-spaced $L^{p}$-NED ($p\geq 2$) process on process
$\epsilon:=\\{\epsilon_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$. Here $\epsilon$ is
an $\alpha$-mixing random field defined as Definition 2. Here
$\Gamma_{N}\subset\mathbf{R}^{d}$ and $\max\alpha_{iN}\leq 1$.

###### Assumption 7

Let $\psi_{Z}(t)=t^{-\nu_{1}}$ and $\psi_{\epsilon}(t)=t^{-\nu_{2}}$, for some
$\nu_{1}$ and $\nu_{2}>0$. For any finite subset $U$ and $V$ in $\Gamma_{N}$,
$\phi(|U|,|V|)=(|U|+|V|)^{\tau}$, where $\tau>0$.

###### Assumption 8

$(X_{s_{iN}},Y_{s_{iN}})$ is identical to random vector
$(X,Y)\in\mathcal{X}\times\mathbf{R}$, where $\mathcal{X}$ is compact in
$\mathbf{R}^{D}$. Assume $E[Y^{2}|X=x]\leq\sigma_{Y}^{2}$,
$E[Y^{p_{0}}|X=x]<+\infty$, for some $p_{0}>2$ and $\forall\ x\in\mathcal{X}$.
Assume conditional mean function $m(x)$ is Hölder continuous on $\mathcal{X}$
with exponent $\beta>0$.

###### Assumption 9

The kernel function $K(u)$ is Lipschitz continuous with compact support. With
out loss of generality, we assume $K(u)\in Lip([-1,1]^{D})$.

###### Assumption 10

We assume $p_{0}$ is large enough such that

  1. (1)

There exists some $\alpha>0$, such that, $\log
N/N^{1-\frac{2}{p_{0}}}h^{D}=O(h^{\alpha})$.

  2. (2)

$N^{\frac{1}{2}-\frac{1}{p_{0}}}h^{\frac{D}{2}+1}=O(1)$.

###### Assumption 11

Based on the $p_{0}$ and $\alpha$ given in Assumption 10, $\nu_{1}$ is
sufficiently large such that

$\displaystyle\mbox{(1)}\
N^{\frac{1}{p_{0}}}h^{\frac{\alpha\nu_{1}}{2d_{2}}+D-1}=O(h^{D}),\ \mbox{(2)}\
\left(h^{\frac{\alpha\nu_{1}}{2d_{2}}}\frac{N^{2}}{\log N}\right)^{s}=o(1).$

###### Assumption 12

Based on the $p_{0}$ and $\alpha$ given in Assumption 10, $\nu_{2}$ is
sufficiently large such that

$\displaystyle\mbox{(1)}\
\left(\frac{\alpha\nu_{2}(p_{0}-2)}{2d_{2}}+2D\right)\frac{1}{p_{0}}>D+\frac{1}{2},\
\mbox{(2)}\
h^{\frac{\alpha\nu_{2}}{2d_{2}}-(\frac{\alpha\tau}{2}+\frac{5D}{4}+1)}(N^{5}\log
N)^{\frac{1}{4}}=o(1).$

###### Theorem 3

Under Assumptions 1-4 and 6-11, we have

$\displaystyle\sup_{x\in\mathcal{X}}|\hat{m}(x)-m(x)|=O_{p}\left(\sqrt{\frac{\log
N}{Nh^{D}}}+h^{\beta}\right).$

Assumption 10 requires $p_{0}$ to be sufficiently large. More specifically,(1)
in Assumption 10 is easy to be achieved. For example, by setting that
$h=O(\log N/N)^{\frac{1}{2+D}}$ and conditional mean function as Lipshitz
continuous function ($\beta=1$), we have

$\displaystyle\frac{\log N}{N^{1-\frac{2}{p_{0}}}h^{D}}=\frac{(\log
N)^{1-\frac{1}{2+D}}}{N^{1-\frac{1}{p_{0}}-\frac{1}{2+D}}}.$

Since $p_{0}>2$ and $h=O(\log N/N)^{\frac{1}{2+D}}$, we instantly know that
there exists an $\alpha>0$ such that (1) of Assumption 10 is satisfied. Under
these similar settings about $h$ and $\beta$, (2) in Assumption 10 can be
easily satisfied as well. These settings are widely accepted when we try to
obtain the optimal uniform convergence rate. Furthermore, to check whether
$h=O(\log N/N)^{\frac{1}{2+D}}$ is feasible, we ony need to check whether it
can satisfy Assumption 11 and Assumption 12 (2). Actually, for any given
$\alpha$, $D$, $d_{2}$, $\tau$ and $p_{0}$, there always exist $\nu_{1}$ and
$\nu_{2}$ satisfying Assumptions 11 and 12. Therefore, according to Theorem 3,
after some simple calculation, we know the local linear estimator of
conditional mean function for algebraic irregularly-spaced NED random fields
could obtain the minimax optimal rate proved by stone1982optimal.

Compared with Theorem 3 in jenish2012nonparametric, our result is obviously
sharper but seems only cover the stationary case. However our result can be
easily extended to nonstationary case by employing Assumption 5 from
jenish2012nonparametric, since this assumption is not involved in deriving the
uniform convergence rate. Moreover, Theorem 3 also extends li2012local’s
result to irregularly-spaced setting.

###  4.2 Density Estimation

As one of the most widely used density estimator, kernel density
estimator(KDE) has been thoroughly investigated. It is widely known that, KDE
suffers from the boundary effect, which means KDE is inconsistent when the
evaluation point is closed to boundary of the support set. Many nice
modification methods have been proposed to solve this problem.
karunamuni2005boundary did a nice literature review on this topic.
cattaneo2020simple proposed a method called simple local polynomial density
estimator(SLPDE), which generically solve this problem. In particular,
compared with the methods mentioned in karunamuni2005boundary, except for
being adaptive to the boundaries of the support of the density without
specific data modification or additional tuning parameter, SLPDE inherits all
the advantages from local polynomial fitting. Thus, in this section, we will
focus on its uniform convergence rate for SLPDE for real-valued geometric
irregularly-spaced $\alpha$-mixing random field.

Firstly, we brief the main idea of SLPDE. It is obvious that, for any point
$x$ in the support set, by denoting $F(x)$ as cumulative distribution
function(CDF), we have

$\displaystyle F(x)=E[F(X)|X=x]:=E[U|X=x].$

Hence, if we consider $F(X)$ as another random variable $U$(uniformly
distributed over [0,1]), $F(x)$ becomes a conditional mean function $E[U|X=x]$
and density function $f(x)$ is the derivative of conditional mean function.
Obviously, local polynomial regression can be used to fit these functions.
However, this is not a feasible estimator since we do not know the true CDF. A
natural idea is to use empirical distribution function $F_{N}$ to replace it.
Thus, SLPDE can be constructed as follow,

$\displaystyle\hat{\theta}(x)=U_{N}(x)^{-1}V_{N}(x),$

where

$\displaystyle
U_{N}(x)=\frac{1}{N}\sum_{i=1}^{N}K_{ih}U\left(\frac{X_{i}-x}{h}\right)U\left(\frac{X_{i}-x}{h}\right)^{T}$
$\displaystyle
V_{N}(x)=\frac{1}{N}\sum_{i=1}^{N}K_{ih}U\left(\frac{X_{i}-x}{h}\right)F_{N}(X_{i}),$
$\displaystyle X_{i}=X_{s_{iN}},\
K_{ih}=\frac{1}{h}K\left(\frac{X_{i}-x}{h}\right),\
U(x)=(1,x,x^{2},\dots,x^{p})^{T},$
$\displaystyle\hat{\theta}(x)=(\hat{F}(x),\hat{F}^{\prime}(x)h,\dots,\frac{1}{p!}\hat{F}^{(p)}(x)h^{p})^{T}.$

Denote $\theta(x)=(F(x),F^{\prime}(x)h,\dots,\frac{1}{p!}F^{(p)}(x)h^{p})^{T}$
and $v_{k}$ as the k-th element of vector $v$. A natural inequality is

$\displaystyle||\hat{F}^{(k-1)}(x)-F^{(k-1)}(x)||_{\infty}\leq
h^{-(k-1)}||\hat{\theta}(x)_{k}-\theta(x)_{k}||_{\infty}.$

Therefore, to obtain the uniform convergence rate of SLPDE, it suffice to
prove the convergence rate of
$||\hat{\theta}(x)_{2}-\theta(x)_{2}||_{\infty}$. Additionally, before
demonstrating our result, we exhibit the following propositions which are
crucial to our proof and may be of independent interest.

###### Proposition 6

Based on Definition 2 and Assumptions 1-4, suppose
$X:=\\{X_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$ is an $\mathbf{R}^{D}$-valued
irregularly-spaced $\alpha$-mixing random field. Assume
$\phi(x,y)=(x+y)^{\tau}$ and $\psi_{\epsilon}(t)=e^{-bt^{\gamma}}$, for some
$\tau,\ \gamma,\ b>0$. Suppose $\mathcal{G}$ is a class of measurable
functions and the lower graph set of each function has finite VC-dimension,
say $V_{\mathcal{G}^{+}}$. Furthermore, assume for $\forall\ g\in\mathcal{G}$,
we have $||g||_{\infty}\leq A$, for some $A>0$. Then, under Assumptions 1 to
4, if $V_{\mathcal{G}}\geq 2$, $\exists\ N^{*}\in\mathbf{N}$, such that for
$\forall\ N\geq N^{*}$, we have

$\displaystyle\mathrm{P}$
$\displaystyle\left(\sup_{g\in\mathcal{G}}\left|N^{-1}\sum_{i=1}^{N}(g(X_{s_{iN}})-E[g(X_{s_{iN}}])\right|>t\right)\leq
40\exp\left(-\frac{Nt^{2}}{(48A)^{2}}\right),$ (9)

where $N^{*}=N_{0}\lor N_{1}\lor N_{2}$,
$N_{1}=\min\\{N\in\mathbf{N}:\sqrt{N}t\geq 36A\\}$ and

$\displaystyle N_{0}=\min\left\\{N\in\mathbf{N}:\sqrt{N}t\geq
48A\int_{\frac{t}{16A}}^{\frac{1}{2}}\sqrt{V_{\mathcal{G}}^{+}\log
3\left(\frac{8e}{u^{2}}\log\frac{12e}{u^{2}}\right)}du\right\\},$
$\displaystyle N_{2}=\min\\{N\in\mathbf{N}:\frac{N}{(\log
N)^{d_{2}/\gamma}}\geq 4K_{2}\log 8C_{0}A((C_{0}A)+8t/3)/t^{2}\\}.$

A natural corollary of this proposition is the following weak-version of DKW
inequality for geometric irregularly-spaced $\alpha$-mixing random field,
which will be very useful in the proof of Theorem 4 later.

###### Corollary 2

Based on Definition 2, suppose $X:=\\{X_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$ is
a strict stationary $\mathbf{R}^{D}$-valued irregularly-spaced $\alpha$-mixing
random field. Assume $\phi(x,y)=(x+y)^{\tau}$ and
$\psi_{\epsilon}(t)=e^{-bt^{\gamma}}$, for some $\tau,\ \gamma,\ b>0$. Let
$\mathcal{A}$ be the collection of all rectangles in $\mathbf{R}^{D}$. Let
$P_{N}$ and $P$ denotes the empirical and true probability measure
respectively. Then, for sufficiently large N, we have

$\displaystyle\mathrm{P}\left(\sup_{A\in\mathcal{A}}\left|P_{N}(A)-P(A)\right|\geq
t\right)\leq 40\exp\left(-\frac{Nt^{2}}{2304}\right).$ (10)

What is surprising here is that, both (9) and (10) share the same deceasing
rate as Hoeffding inequality for iid case. Unfortunately, this still can not
lead us to obtain the optimal rate of estimators if the variance would be
sufficiently small as sample diverges to be infinitely large, since we only
consider the uniform upper bound here. Therefore, in order to solve this
problem, we take one step further and deliver the following proposition.

###### Proposition 7

Based on Definition 2 and Assumptions 1-4, suppose
$\\{X_{s_{iN}},Y_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$ is an
$\mathcal{X}\times[-L,L]$-valued irregularly-spaced $\alpha$-mixing random
field, for some $L>0$ and $\mathcal{X}\times[-L,L]\subset\mathbf{R}^{D+1}$.
Assume $\phi(x,y)=(x+y)^{\tau}$ and $\psi(t)=e^{-bt^{\gamma}}$, for some
$\tau,\ \gamma,\ b>0$. Let $\mathcal{G}$ a class of real-valued measurable
functions $g:\mathcal{X}\rightarrow\mathbf{R}$. Assume for any
$g\in\mathcal{G}$, $||g||_{\infty}\leq A$ and $||g||_{2}\leq\sigma$. Then, for
any $N,t$ that satisfy

C1. $tA\leq\frac{3\sigma^{2}L}{40}$,

C2.

$\sqrt{\frac{Nt^{2}}{(\log N)^{\frac{d_{2}}{\gamma}}}}\geq
2\sqrt{10K_{2}\sigma}C_{0}L\int_{\frac{1}{2}(\frac{t}{L})^{1/4}}^{\sigma^{1/4}}\sqrt{\log\mathcal{N}(u^{2},\mathcal{G},||\cdot||_{\infty})}du,$

C3.

$\sqrt{\frac{Nt^{2}}{(\log N)^{\frac{d_{2}}{\gamma}}}}\geq
24\sqrt{10K_{2}}C_{0}L\sigma,$

we have

$\displaystyle\mathrm{P}\left(\sup_{g\in\mathcal{G}}\left|P_{N}(gY)-P(gY)\right|\geq
t\right)\leq 8\exp\left(-\frac{Nt^{2}}{1440C_{0}^{2}K_{2}(\log
N)^{\frac{d_{2}}{\gamma}}L^{2}\sigma^{2}}\right),$ (11)

where $P_{N}(gY)=N^{-1}\sum_{i=1}^{N}g(X_{s_{iN}})Y_{s_{iN}}$ and
$P(gY)=E[N^{-1}\sum_{i=1}^{N}g(X_{s_{iN}})Y_{s_{iN}}]$.

Condition C1 is the reason why only variance term is involved in the upper
bound. For the case where the variance is asymptotically larger than 0, C1 is
trivial. When the variance term convergence to 0 as sample size grows, this
condition C1 is also suitable for many situations, like kernel-based
estimators. This can be checked in the proof of Proposition 8 and Theorem 4.

###### Assumption 13

The kernel function $K(u)$ is a Lipschitz continuous,non-negative and
symmetric function with compact support $[-1,1]$.

###### Assumption 14

Assume $\\{X_{s_{iN}}\\}_{s_{iN}\in\Gamma_{N}}$ is a real-valued strict
stationary geometric $\alpha$-mixing random field sharing the same marginal
distribution as random variable $X$. For any subsets $U$ and $V$ of
$\Gamma_{N}$ such that $|U|\lor|V|<+\infty$,
$\alpha(\sigma(U),\sigma(V))\leq(|U|+|V|)^{\tau}\exp(-b\rho(U,V)^{\gamma})$,
for some $b,\tau>0$ and $\gamma>d_{2}$. $d_{2}$ here is the ”effective
dimension” defined in Assumption 3.

###### Assumption 15

ASsume $X$ has bounded support $[a,b]$. The CDF of $X$, say $F(x)$, is $p+1$
times continuously differentiable for some $p\geq 1$ and its probability
density function, $f(x)$, satisfies $\inf_{x\in[a,b]}f(x)>0$.

###### Assumption 16

For sample size $N$ and bandwith $h$, we assume , for any $l=0,1,...,p$,

$\displaystyle\lim_{N\rightarrow+\infty}h=0,\
\lim_{N\rightarrow+\infty}\frac{\log N}{Nh^{2l+1}}=o(1).$

###### Proposition 8

Under Assumptions 1-4 and 13-16, we have,

$\displaystyle\sup_{x\in[a,b]}\left|\frac{1}{N}\sum_{i=1}^{N}g^{l}_{h}(X_{s_{iN}})F_{N}(X_{s_{iN}})-E\left[g^{l}_{h}(X)F_{N}(X)\right]\right|=O_{p}\left(\sqrt{\frac{\log
N}{Nh}}\right),$

where

$g^{l}_{h}(X)=\left(\frac{X-x}{h}\right)^{l}K_{h}\left(\frac{X-x}{h}\right),\
K_{h}\left(\frac{u}{h}\right)=\frac{1}{h}K\left(\frac{u}{h}\right).$

###### Theorem 4

Under Assumptions 1-4 and 13-16, we have, for any $v=0,1,2,\dots,p$ and
$\forall\ x\in[a,b]$

$\displaystyle\sup_{x\in[a,b]}|\hat{F}^{(v)}(x)-F^{(v)}(x)|=O_{p}\left(\sqrt{\frac{\log
N}{Nh^{2v+1}}}+h^{p+1-v}\right).$

More particularly, for density function,

$\displaystyle\sup_{x\in[a,b]}|\hat{f}(x)-f(x)|=\sup_{x\in[a,b]}|\hat{F}^{{}^{\prime}}(x)-F^{{}^{\prime}}(x)|=O\left(\sqrt{\frac{\log
N}{Nh^{3}}}+h^{p}\right).$

Assumptions 13, 15 and 16 are directly borrowed from cattaneo2020simple. Note
that Assumption 15 actually indicates that density function of $X$ belongs to
$p-th$ order of Holder class. Theorem 4 reflects that, unlike kernel density
estimator, SLPDE may not be able to achieve the optimal uniform convergence
rate. But as shown by the Theorem 4 and simulation part of cattaneo2020simple,
SLPDE is asymptotically unbiased for any $x\in[a,b]$ which significantly
outperforms the kernel density estimator near the boundary points.

##  5 Modal Regression

In standard prediction problem, we are interested in predicting the value of
output $Y$ given $X=x$. Conditional mean regression is a standard method which
has been widely used and thoroughly studied. But this method may fail when the
conditional density function $f_{Y|X}(y|x)$ is significantly asymmetric. After
all, prediction via conditional mean function only tells us an ”average-value”
of random variable Y given $X=x$, while what we are interested in is to
investigate, on condition of $X=x$, which value the output $Y$ may be as much
as possible. Under this circumstance, a more natural idea should be global or
local modal regression.

On this topic, there has been many outstanding previous research. Here we only
name a few. For global modal regression, lee1989mode, sager1982maximum,
yao2014new assume a parametric linear model,
$Mode(Y|X=x)=\beta_{0}+\beta^{T}x$, where $\beta_{0}\in\mathbf{R}$ and
$\beta\in\mathbf{R}^{D}$ are unkonwn coefficients and $Mode(Y|X=x)$ denotes
the global mode of $Y$ given $X=x$. Nonparametric global modal regression is
more flexible since the linearity of conditional mode function is not in need.
collomb1986note investigate the uniform convergence rate of conditional mode
function estimator based on kernel density estimation under $\phi$-mixing
process. More recently, yao2012local propose a mode estimation based on local
polynomial smoothing. Unlike global modal regression which seeks for the
global maximum point of conditional density $f_{Y|X}(y|x)$ (or equally, joint
density, $f(x,y)$) given $X=x$, local modal regression tends to detect the
local extreme points satisfying, $\partial f(x,y)/\partial y=0$ and
$\partial^{2}f(x,y)/\partial y^{2}<0$. scott2015multivariate proposed an
estimator of this according commonly used kernel-based joint density
estimates. Motivated by this, more recently, chen2016nonparametric thoroughly
investigate the uniform convergence rate and confidence set of local modal
regression based on kernel density estimator. Here, we focus on nonparametric
global modal regression like Collomb .’s setting.

More particularly, our object is to detect

$\displaystyle\arg\max_{y\in\mathcal{Y}_{x}}f(y|x):=y_{x},$

where $\mathcal{Y}_{x}$ is the support of $Y$ given $X=x$. Hence a natural
estimator is

$\displaystyle\arg\max_{y\in\mathcal{Y}_{x}}\hat{f}(y|x):=\hat{y}_{x},$
$\displaystyle\hat{f}(y|x)=\frac{(Nh^{D+1})^{-1}\sum_{i=1}^{N}K\left(\frac{X_{s_{iN}}-x}{h}\right)L\left(\frac{Y_{s_{iN}}-y}{h}\right)}{(Nh^{D})^{-1}\sum_{i=1}^{N}K\left(\frac{X_{s_{iN}}-x}{h}\right)}.$

###### Assumption 17

Based on Definition 1, let $Z:=(X_{s_{iN}},Y_{s_{iN}})_{s_{iN}\in\Gamma_{N}}$
be a geometric irregularly-spaced $L^{p}$ NED random field on random field
$\epsilon:=(\epsilon_{s_{iN}})_{s_{iN}\in\Gamma_{N}}$ at the order of $e$,
where $p\geq 1$. Assume $\epsilon$ is an $\alpha$-mixing random field defined
as Definition 2. Assume $\max_{i,N}\alpha_{iN}<+\infty$. More specifically,
with out loss of generality, let $\max_{i,N}\alpha_{iN}=1$. Let
$\phi(a,b)=(a+b)^{\tau}$ and
$\psi_{Z}(t)=\psi_{\epsilon}(t)=\exp(-bt^{\gamma})$, for some positive real
numbers $\kappa,\tau,b$ and $\gamma>d_{2}$, where $d_{2}$ is the ”effective
dimension” defined in Assumption 3.

###### Assumption 18

For each $s_{iN}$, let $(X_{s_{iN}},Y_{s_{iN}})$ be identically distributed as
$(X,Y)\in\mathcal{X}\times\mathcal{Y}$, where $\mathcal{X}\times\mathcal{Y}$
is compact subset of $\mathbf{R}^{D+1}$. Assume the joint density $f(x,y)$ and
marginal densities $f(x)$ and $f(y)$ are all well defined. Moreover, set
$f(x,y)$ is Hölder-continuous on support $\mathcal{X}\times\mathcal{Y}$ with
exponent $\alpha\in(0,1]$ and $\inf_{x\in\mathcal{X}}f(x)>0$.

###### Assumption 19

(Uniqueness) For the conditional density $f(y|x)$, we assume for any fixed
$x\in\mathcal{X}$, for $\forall\ \xi>0$, $\exists\ a>0$, such that, for any
mapping $t:x\rightarrow y$, $\sup_{x\in\mathcal{X}}|y_{x}-t(x)|>\xi$ always
implies $\sup_{x\in\mathcal{X}}|f(y_{x}|x)-f(t(x)|x)|>a$.

###### Assumption 20

Let the kernel functions $K(u)$ and $L(v)$ be non-negative, uniformly bounded
and Lipschitz continuous on $\mathbf{R}^{D}$ and $\mathbf{R}$ respectively.

As we mentioned previously, our goal is to obtain the uniform convergence rate
of estimator $\hat{y}_{x}$. Unfortunately, based on naive union bound
argument, (1) can not lead us to sufficiently sharp rate. Thus, we here still
choose ”chaining” as the major tool. However, talagrand1996majorizing pointed
out that, (generic) chaining often requires the increment of empirical process
is sub-Gaussian, where the increment is measured by $L^{p}$ distance of some
probabilistic measure. The sub-Gaussianity here actually offers us exponential
level tail probability to control the chaining Pollard (2015). This point can
also be shown by the proof of Theorem 19.1 in gyorfi2002distribution and
Proposition 6 and 7 in this paper. Thus it seems that Theorem 1 may not be
suitable for this argument. However, things are actually not that severe for
geometric NED. Because, as we mentioned in Remark 1, a key advantage of
Theorem 1 is that it is valid for any $\beta>0$. That is to say, setting
$\beta$ as arbitrary large positive number does not change the decreasing rate
of the exponential part of (1) while makes the polynomial part arbitrary
sharper. This flexibility can allow us to continue using chaining argument.
More specifically, similar to Proposition 6, we have the following Theorem 5,
which exhibit the non-asymptotic uniform convergence rate of kernel density
estimator for geometric irregularly-spaced NED process.

###### Theorem 5

Let $Z_{i}:=(X_{s_{iN}},Y_{s_{iN}})$, $s_{iN}\in\Gamma_{N}$ be a sequence of
$\mathcal{Z}:=\mathcal{X}\times\mathcal{Y}$-valued random vectors satisfying
Assumptions 17 and 18. Denote $Z=(X,Y)$ and $G(u)$ is a density function which
is uniformly upper bounded by $A>0$ and Lipschitz continuous on
$\mathbf{R}^{D+1}$. Then, together with Assumptions 1-4, for sufficiently
large $N$ and any $\beta>\max\\{11/4,\frac{3}{2}+\frac{3}{D+1}\\}$, we have,
with probability no less than $1-\triangle$,

$\displaystyle\sup_{z\in\mathcal{Z}}\left|\hat{f}(z)-f(z)\right|\leq\sqrt{\frac{\log
N}{Nh^{D+1}}}+h^{\alpha},$
$\displaystyle\hat{f}(z)=\frac{1}{Nh^{D+1}}\sum_{i=1}^{N}G\left(\frac{Z_{i}-z}{h}\right),$
$\displaystyle\triangle=8\exp\left(-\frac{(\log
N)^{1-\frac{d_{2}}{\gamma}}}{40K_{2}C_{0}^{2}}\right)+\frac{2K_{1}}{N^{\kappa+\tau+1}}.$

Then as a natural corollary of Theorem 5, we instantly obtain the uniform
convergence rate of uni-modal regression as follow,

###### Theorem 6

Under Assumption 1-4, 13 and 17-20, by assuming $\mathcal{X}$ and
$\mathcal{Y}$ as compact sets, we have

$\displaystyle\sup_{x\in\mathcal{X}}|\hat{y}_{x}-y_{x}|=O_{p}\left(\sqrt{\frac{\log
N}{Nh^{D+1}}}+h^{\alpha}\right).$

Moreover, by choosing $h=(\log N/N)^{\frac{1}{2\alpha+D+1}}$,

$\displaystyle\sup_{x\in\mathcal{X}}|\hat{y}_{x}-y_{x}|=O_{p}\left(\frac{\log
N}{N}\right)^{\frac{\alpha}{2\alpha+D+1}}.$

Assumption 13 is directly borrowed from collomb1986note which can be easily
satisfied. For example, assume for each $x\in\mathcal{X}$,
$\sup_{y\in\mathcal{Y}_{x}}|f(y|x)|$ uniquely exists. Without loss of
generality, suppose there exists $x^{*}\in\mathcal{X}$ such that,
$|y_{x^{*}}-t(x^{*})|=\sup_{x\in\mathcal{X}}|y_{x}-t(x)|$. Then, according to
Assumption 12, when $|y_{x^{*}}-t(x)|>\xi$, we know, there always exists
$a>0$, such that, $|f(y|x^{*})-f(y|t(x^{*}))|>a$. Otherwise, the uniqueness
assumption is violated. As for the compactness of $\mathcal{X}$ and
$\mathcal{Y}$, this condition can be released to unbounded support by using
the truncation method from hansen2008uniform. But here we only consider the
simpler case.

Theorem 5 shows that the optimal rate of estimator $\hat{y}_{x}$ is $(\log
N/N)^{\alpha/2\alpha+D+1}$ and can not be promoted. This is because, due to
the objective function, in order to detect $y_{x}$, a step we can not avoid is
to estimate the joint density $f(x,y)$, while, for kernel density estimator,
$(\log N/N)^{\alpha/2\alpha+D+1}$ has already achieved the mini-max lower
bound for $L^{\infty}$ distance shown by stone1982optimal.

##  6 Density Level Set Estimation

Set $\mu$ as a sigma finite measure on $Z\subset\mathbf{R}^{D}$. Consider
$(Z_{s_{iN}})$ as a strict-stationary $Z$-valued geometric $L^{p}$ NED process
on random field $\epsilon$ at order of $e$. Here $\epsilon$ is defined as
Definition 2. For each $Z_{s_{iN}}$, assume there is an unknown marginal
probability density $f$ with respect to measure $\mu$. For fixed $\lambda>0$,
our goal is to estimate the $\lambda$-level set of density $f$ defined by

$\displaystyle\Gamma_{f}(\lambda):=\\{z\in Z:f(z)>\lambda\\}.$ (12)

Another widely used definition of $\lambda$-level set is as follow,

$\displaystyle\bar{\Gamma}_{f}(\lambda):=\\{z\in Z:f(z)\geq\lambda\\}.$ (13)

Thorough out this subsection, we always consider $\lambda$ to be fixed. When
no confusion is possible, we also use notations $\Gamma(\lambda)$ or $\Gamma$
($\bar{\Gamma}(\lambda)$ or $\Gamma$) to represent $\Gamma_{f}(\lambda)$
($\bar{\Gamma}_{f}(\lambda)$). If we allow some part of the density function
to be ”flat”, then the two definitions are different from each other. But we
only focus on estimating (12) here.

Level set estimation is a core tool in many branches of machine learning and
statistics. A typical example is anomaly detection. In this research area, as
pointed out by steinwart2005classification and references therein, an approach
is to assume that observations are abnormal if they are observed outside of
”high-probability area”. This ”high-probability area” is the region where most
of the probability mass lies in. Obviously, to detect this ”high-probability
area”, a straightforward idea is to use $\lambda$-level set of density
function, provided that the density function of data has been well defined.
When $\lambda=0$, this questions corresponds to support estimation which has
been examined by devroye1980detection.

To estimate the $\lambda$-level set of $f$, we here introduce a plug-in
estimator with offset $l$. That is to say, based on sample,
$(Z_{s_{iN}})_{i=1}^{N}$, given a product kernel density estimator $\hat{f}$,
we estimate (12) by

$\displaystyle\hat{\Gamma}:=\\{z\in Z:\hat{f}(z)\geq\lambda+l_{n}\\},$ (14)
$\displaystyle\hat{f}=\frac{1}{Nh^{D}}\sum_{i=1}^{N}\prod_{k=1}^{D}K\left(\frac{Z_{i}^{k}-z^{k}}{h}\right),$
(15) $\displaystyle Z_{i}:=(Z_{i}^{1},\dots,Z_{i}^{D}),\
Z_{i}^{k}:=Z_{s_{iN}}^{k},$

where $\\{l_{n}\\}_{n\in\mathbf{N}}$ is a positive and monotonically non-
increasing number sequence such that $\lim l_{n}=0$. For estimator (14),
rigollet2009optimal had thoroughly investigated its convergence rates under
two different measures of performance. By introducing some local smoothness
assumptions of density function within the neighborhood of ”level $\lambda$”,
they shew their rates are minimax optimal. Hence, here we follow the guide of
this paper and extend the dependence condition of their discussion from iid to
NED.

Firstly, we introduce the two measures of performance of estimator (14).
Recall that $\mu$ is a sigma finite measure on $Z$ and define
$\nu(G)=\int_{G}|f(x)-\lambda|\mu(dx)$. To measure the performance of 14, we
use the following two pseudo-distances between $G_{1}$ and $G_{2}\in
Z\cap\mathcal{B}_{\mathbf{R}^{D}}$, where $\mathcal{B}_{\mathbf{R}^{D}}$
denotes the Borel field on $\mathbf{R}^{D}$:

$\displaystyle d_{\triangle}(G_{1},G_{2})=\mu(G_{1}\triangle G_{2}),$
$\displaystyle d_{H}(G_{1},G_{2})=\nu(G_{1}\triangle
G_{2})=\int_{G_{1}\triangle G_{2}}|f(x)-\lambda|\mu(dx).$

Obviously, compared with with $d_{\triangle}(G_{1},G_{2})$, an advantage of
$d_{H}(G_{1},G_{2})$ is that, it is invariant if we replace $\Gamma$ with
$\bar{\Gamma}$(see rigollet2009optimal). However,
$d_{\triangle}(\dot{,}\dot{)}$ is quite sensitive to the difference between
$\Gamma$ and $\bar{\Gamma}$, that is to say, $\\{f=\lambda\\}$. Actually, by
using idea of ”$\rho$-exponent at level $\lambda$ with respect to measure
$\mu$” defined as follow, we can build a link between the two measurements.

###### Definition 3

For any given $\rho,\ \lambda\geq 0$, a function $f:Z\rightarrow\ \mathbf{R}$
is said to be $\rho$-exponent at level $\lambda$ with respect to $\mu$ if
there exists some positive constants $c_{0}$ and $\epsilon_{0}$, for $\forall\
0<\epsilon\leq\epsilon_{0}$,

$\displaystyle\mu\\{z\in Z:0<|f(z)-\lambda|\leq\epsilon\\}\leq
c_{0}\epsilon^{\rho}.$

This definition is introduced by polonik1995measuring. For $\rho=0$, the
existence of $c_{0}$ and $\epsilon_{0}$ is trivial. When $\rho>0$, it measures
the concentration of the value of function. Based on Definition 3, Proposition
2.1 in rigollet2009optimal shows that for any fixed $\lambda,\rho\geq 0$, if
the density function $f$ satisfies Definition 3, then for any $L_{\mu}>0$.
there exists a $C>0$ such that for any $G_{1}$ and $G_{2}$ satisfying
$\mu(G_{1}\triangle G_{2})\leq L_{\mu}$, we have

$\displaystyle d_{\triangle}(G_{1},G_{2})\leq\mu(G_{1}\triangle
G_{2}\cap\\{f=\lambda\\})+C(d_{H}(G_{1},G_{2}))^{\frac{\rho}{1+\rho}}.$ (16)

Meanwhile, an assumption tightly connected with the parameter
”exponent-$\gamma$” is the ”$L^{p}$-NED condition”. More specifically, we need
the following assumption.

###### Assumption 21

Based on Definitions 1 and 2, assume $1\leq d^{\prime}\leq d$, $p>1+\rho$,
$\max\alpha_{iN}\leq 1$, $\psi_{Z}(r)=\psi_{\epsilon}(r)=\exp(-br^{\gamma})$
and $\gamma>d^{\prime}$.

Please note that even we assume $p>1+\rho$, since our result holds for any
$\rho>0$, when $0<\rho<1$, $L^{2}$-NED random field is sufficiently general to
cover this requirement. Additionally, the requirement $1<\rho<1$ can be easily
fulfilled when (4) in Assumption 22 is satisfied, provided that $\beta\geq 1$.
Due to xu2015maximum, xu2018sieve and jenish2012spatial, $L^{2}$-NED process
has been general enough to cover many interesting processes that are widely
used in time(spatial) series analysis.

Except for the choice of measure of performance, another key character
influencing the convergence rate of level set estimator is the smoothness of
density function. rigollet2009optimal shew that the fast rate of plug-in
estimator of density level set with respect to (12) and (13) are only
associated with the convergence rate of density estimator, $\hat{f}$, of the
neighborhood around level-$\lambda$, which is defined as

$\displaystyle D(\eta)=\\{z\in Z:f(z)\in(\lambda-\eta,\lambda+\eta)\\}.$

Since it is widely known that smoothing conditions are crucial to the
convergence rate of density estimator, Rigollet Vert’s discovery allows us to
ensure the consistency of plug-in level set estimator by only depending on the
assumptions about smoothness of the neighborhood and for the points belonging
to $D(\eta)^{c}$, $\hat{f}$ converges at least $O((\log n)^{-1/2})$-fast. In
this section, since we aim to discuss the attainability of optimality of
convergence rates of $\hat{\Gamma}_{f}(\lambda)$ with respect to
$d_{\triangle}$ and $d_{H}$ respectively, we deliver the following assumption
about the smoothness of density $f$ on $D(\eta)$ and $D(\eta)^{c}$.

###### Definition 4

(Locally Hölder Class) Fix $L>0$, $r>0$ and denote by
$\Sigma(\beta,L,r,x_{0})$ the set of functions $g:Z\rightarrow\mathbf{R}$ that
are $[\beta]$-times continuously differentiable at point $x_{0}$ and satisfy,
for any $s=(s_{1},\dots,s_{D})\in\mathbf{N}^{D}$,

$\displaystyle|g(z)-g_{z_{0}}^{(\beta)}(z)|\leq L||z-z_{0}||^{\beta},\
\forall\ z\in B(z_{0},r),$ $\displaystyle
g_{z_{0}}^{(\beta)}(z)=\sum_{|s|\leq[\beta]}\frac{(z-z_{0})^{s}}{s!}\frac{\partial^{s_{1}+\dots+s_{D}}}{\partial
z_{1}^{s_{1}}\dots\partial_{z_{D}}^{s_{D}}}g(z_{0}),$
$\displaystyle|s|=s_{1}+\dots+s_{D},\ s!=s_{1}!\dots s_{D}!,\
z^{s}=z_{1}^{s_{1}}\dots z_{D}^{s_{D}}.$

The set $\Sigma(\beta,L,r,x_{0})$ is called locally Hölder class of functions.

###### Assumption 22

Fix measure $\mu$ to be Lebesgue measure on $\mathbf{R}^{D}$. Fix $\beta>0$,
$L>0$, $r>0$, $\lambda>0$ and $\gamma>0$. Let
$\mathcal{P}_{\Sigma}(\beta,L,r,\lambda,\gamma,\beta^{\prime},L^{*}):=\mathcal{P}_{\Sigma}$
denote the class of probability density functions $f$ on $Z$ for which
$\exists\ \eta>0$ such that:

  1. (1)

$f\in\Sigma(\beta,L,r,x_{0})$ almost surely for all $x_{0}\in D(\eta)$.
$\exists\ \beta^{\prime}>0$ such that $f\in\Sigma(\beta^{\prime},L,r,x_{0})$
almost surely for all $x_{0}\in D(\eta)^{c}$.

  2. (2)

f has $\rho$-exponent at level $\lambda$ with respect to the measure $\mu$.

  3. (3)

f is uniformly bounded by a constant $L^{\star}$.

  4. (4)

$\gamma(\beta\land 1)\leq 1$.

Assumption 22 is directly inherited from Definition 4.1 in
rigollet2009optimal. According to (1) in Assumption 22, we know that the
smoothness of density function is allowed to be different. It is widely known
that parameters $\beta$ and $\beta^{\prime}$ here are usually expected to
control the convergence rate of density estimator. Actually, as we will show
later, $\beta^{\prime}$ could be allowed to be arbitrarily close to $0$ while
this won’t affect the convergence rate of level set estimator. (4) indicates
that there exits densities such that both $\Gamma$ and $\Gamma^{c}$ have non-
empty interior Audibert Tsybakov (2005).

Additionally, we also admit the following assumption on the kernel function
used in (15).

###### Assumption 23

Let K be a real-valued function on $\mathbf{R}$ with support $[-1,1]$. For
fixed $\alpha>0$, the function $K$ satisfies the following conditions:

  1. (1)

$\int K=1$, $||K||_{p}<\infty$ for any $1\leq p\leq\infty$.

  2. (2)

$\int|t|^{\alpha}|K(t)|dt<\infty$, for any $\alpha>0$. In the case
$[\alpha]\geq 1$, we instantly have, for any $s>0$ such that $s\leq[\alpha]$,
$\int t^{s}K(t)dt=0$.

  3. (3)

The first-order derivative of $K$ is uniformly bounded on $[-1,1]$.

Assumption 23 coincides the definition of ”$\alpha$-valid” kernel from
tsybakov2004optimal. Section 1.22 in tsybakov2004introduction clearly
demonstrates how to construct such kind of kernel by using the orthonormal
property of Legendre polynomial.

Generally speaking, we follow the method of rigollet2009optimal to prove
Theorem 7. Therefore, we demonstrate the following result about the ”fast”
rate of kernel-based plug-in estimator. This can be regarded as a direct
extension of Theorem 3.1 in Rigollet Vert.

###### Definition 5

Let $\mathcal{P}$ be a family of probability densities of
$Z\subset\mathbf{R}^{D}$ and fix an $\triangle>0$. Let $\varphi=(\varphi_{N})$
and $\phi=(\phi_{N})$ be two monotonically non-increasing number sequences.

  1. 1.

We say density estimator $\hat{f}$ is ”pointwise convergent at a rate
$\phi_{N}$ with remainder $N^{\theta p}$ uniformly over $\mathcal{P}$” if
there exist positive constants $c_{1},\ c_{2},\ c_{\phi}$ such that, for
almost all $z\in Z$ with respect to measure $\mu$, we have, for $\forall\
\theta>0,\ c_{\phi}\phi_{N}<\delta<\triangle$ and some constant $B>0$

$\displaystyle\sup_{f\in\mathcal{P}}\mathrm{P}\left(\left|\hat{f}(z)-f(z)\right|\geq\delta\right)\leq
c_{1}\exp\left(-c_{2}(\delta/\phi_{N})^{2}\right)+\left(\frac{B}{N^{\theta}\delta}\right)^{p}.$
(17)

  2. 2.

We say density estimator $\hat{f}$ is ”$(\varphi,\phi)$-locally pointwise
convergent with remainder $N^{\theta p}$ in $D(\eta)$ uniformly over
$\mathcal{P}$” if it is ”pointwise convergent at a rate $\phi_{N}$ with
remainder $N^{\theta p}$ uniformly over $\mathcal{P}$” and $\exists\ c_{3},\
c_{4},\ c_{\varphi}>0$, for $\mu$-almost all $z\in D(\eta)$ we have

$\displaystyle\sup_{f\in\mathcal{P}}\mathrm{P}\left(\left|\hat{f}(z)-f(z)\right|\geq\delta\right)\leq
c_{3}\exp\left(-c_{4}(\delta/\varphi_{N})^{2}\right)+\left(\frac{B}{N^{\theta}\delta}\right)^{p}.$
(18)

$p$ here is the $L^{p}$ condition of NED.(see Definition 1).

Definition 5 is a direct variation of Definition 3.1 in rigollet2009optimal
which plays a key role in obtaining the fast rate of kernel-based plug-in
density level set estimator, $\hat{\Gamma}$, under near epoch dependence.
Compared with Rigollet’s definition, a core difference here is the additional
polynomial part $(1/N^{\theta}\delta)^{p}$. Fortunately, similar to the proof
of Theorem 5 in Appendix D, by making use of the property that $\theta$
defined in Definition 5 is allowed to be arbitrary large, we can still obtain
a result which is nearly the same as Theorem 3.1 in rigollet2009optimal.

###### Theorem 7

Fix $\lambda>0$, $\triangle>0$ and let $\mathcal{P}$ be a class of densities
on $Z$. Let $\varphi_{N}\lor\phi_{N}=o(1/\sqrt{\log N})$,
$l_{N}\land\phi_{N}\land\varphi_{N}\geq CN^{-\nu}$, for some $C,\ \nu>0$. Let
$\hat{f}$ be a density estimator of $f\in\mathcal{P}$ such that
$\mu(\hat{f}\geq\lambda)\leq M$ holds for some $M>0$ almost surely with
respect to $\mathrm{P}$. Assume $\hat{f}$ is $(\varphi,\phi)$-locally
convergent with remainder $N^{\theta p}$ in $D(\eta)$ uniformly over
$\mathcal{P}$. Then if $f$ is $\rho$-exponent at level $\lambda$. Then we have

$\displaystyle\sup_{f\in\mathcal{P}}E\left[d_{H}(\hat{\Gamma},\Gamma)\right]\lesssim\varphi_{N}^{1+\rho},\
$ $\displaystyle l_{N}\lesssim\varphi_{N}.$ (19)
$\displaystyle\sup_{f\in\mathcal{P}}E\left[d_{\triangle}(\hat{\Gamma},\Gamma)\right]\lesssim(\varphi_{N}\sqrt{\log
N})^{\rho},\ $ $\displaystyle l_{N}=O(\varphi_{N}\sqrt{\log N}).$ (20)

(19) and (20) show that, under two different measures of performance, the
convergence rates have nothing to do with $\phi_{N}$. Considering that the
smoothness of density function significantly affects the local convergence
rate of density estimator, Theorem 7 too some degree indicates that, to obtain
the convergence rate of level set estimator, only assumptions about local
smoothness are in demand. Thus, by combining Definition 5 and Theorem 7, we
immediately have the following theorem which indicates that, estimator
$\hat{\Gamma}$ is optimal with respect to ”$d_{\triangle}$” and ”$d_{H}$” when
the density function $f$ satisfies Assumption 22.

###### Theorem 8

Suppose Assumptions 1-4, 21-23 have been satisfied. Define
$\beta^{*}=\beta\lor\beta^{\prime}$ and let density estimator $\hat{f}$ has
form (15) whose kernel function satisfies Assumption 23 with
$\alpha=\beta^{*}$. Then we have

  1. 1.

When $l_{N}\lesssim N^{-\frac{2\beta}{2\beta+D}}$ and
$h=O(N^{-\frac{1}{2\beta+D}})$,

$\displaystyle\sup_{f\in\mathcal{P}}E\left[d_{H}(\hat{\Gamma},\Gamma)\right]\lesssim(N^{-\frac{\beta}{2\beta+D}}\log
N^{\frac{d_{2}}{2\gamma}})^{1+\rho}.$ (21)

  2. 2.

When $l_{N}=O(N^{-\frac{2\beta}{2\beta+D}}\sqrt{\log N})$ and $h=O(N/\log
N)^{-\frac{1}{2\beta+D}}$,

$\displaystyle\sup_{f\in\mathcal{P}}E\left[d_{\triangle}(\hat{\Gamma},\Gamma)\right]\lesssim
N^{-\frac{\rho\beta}{2\beta+D}}(\log
N)^{\rho(\frac{\beta}{2\beta+D}+\frac{d_{2}}{2\gamma})}.$ (22)

Compared with Theorem 5.1 in rigollet2009optimal, we can instantly know that
(21) and (22) are optimal up to a logarithm factor. More specifically, when
the data is iid, that is to say, $\gamma=+\infty$, Theorem 8 coincides with
Corollary 4.1 in rigollet2009optimal under nearly the same conditions.

##  7 Conclusion and Discussion

This paper is dedicated to focus on two topics under different Near-Epoch
Dependent conditions. Firstly, we focus on deriving Bernstein-type
inequalities for geometric and algebraic irregularly-spaced NED process. As a
corollary, a new Bernstein-type inequality for geometric irregularly-spaced
$\alpha$-mixing process is also obtained. All of our inequalities are adaptive
to the ”effective dimension” of index set, which would allow us to obtain much
sharper upper bound when the ”effective dimension” is significantly smaller
than the dimension of index set. According to our knowledge, this may be the
first result reflecting this phenomenon. Major tools of proof are Bernstein
blocking technique and making use of the definition of NED, which indicates
that NED process is a functional version of a given process and the orthogonal
complement of its projection on the given process will vanish with respect to
$L^{p}$-norm, $p\geq 1$. However, an inevitable disadvantage of Bernstein
blocking technique is that the sharpness of inequality is determined by the
design of blocks. We suppose this is purely artificial. It is widely known
that NED process is generically connected with martingale process. Hence, a
possible alternative approach may be martingale method (e.g.
kontorovich2017concentration). We leave this topic for our future research.
Additionally, even though we have too some degree obtained Bernstein-type
inequalities for irregularly-spaced NED random field, our results heavily rely
on pure domain expand asymptotics. Thus, another important question is what
would happen if things become worse, like the ”domain expand and infill
asymptotics” Lu Tjøstheim (2014) or ”stochastic sampling with mixed
asymptotics” Lahiri (2003))?

For applications, based on these inequalities, we investigate uniform
convergence of local linear and uni-modal regression estimators for algebraic
and geometric NED process respectively. Additionally, we also investigate the
uniform convergence rate of simple local polynomial density estimator for
geometric $\alpha$-mixing random field. Our result shows that local linear
estimator could achieve optimal rate demonstrated by stone1982optimal, which
refreshes the result from jenish2012nonparametric. Another potential
interesting and direct application is to prove the consistency of sieve
maximum likelihood of spatial auto-regressive Tobit model when SART only
satisfies algebraic NED condition. This could significantly extend the
generality of the result from xu2018sieve.

## References

  * Adamczak (2008) adamczak2008tailAdamczak, R.  2008\.  A tail inequality for suprema of unbounded empirical processes with applications to Markov chains A tail inequality for suprema of unbounded empirical processes with applications to markov chains.  Electronic Journal of Probability131000–1034.
  * Andrews (1985) andrews1985nearlyAndrews, DW.  1985\.  A nearly independent, but non-strong mixing, triangular array A nearly independent, but non-strong mixing, triangular array.  Journal of Applied Probability223729–731.
  * Andrews (1995) andrews1995nonparametricAndrews, DW.  1995\.  Nonparametric kernel estimation for semiparametric models Nonparametric kernel estimation for semiparametric models.  Econometric Theory113560–586.
  * Audibert Tsybakov (2005) audibert2005fastAudibert, JY. Tsybakov, AB.  2005\.  Fast learning rates for plug-in classifiers under the margin condition Fast learning rates for plug-in classifiers under the margin condition.  arXiv preprint math/0507180.
  * Bierens (1983) bierens1983uniformBierens, HJ.  1983\.  Uniform consistency of kernel estimators of a regression function under generalized conditions Uniform consistency of kernel estimators of a regression function under generalized conditions.  Journal of the American Statistical Association78383699–707.
  * Bierens (2012) bierens2012robustBierens, HJ.  2012\.  Robust methods and asymptotic theory in nonlinear econometrics Robust methods and asymptotic theory in nonlinear econometrics ( 192).  Springer Science & Business Media.
  * Billingsley (2013) billingsley2013convergenceBillingsley, P.  2013\.  Convergence of probability measures Convergence of probability measures.  John Wiley & Sons.
  * Bosq (2012) bosq2012nonparametricBosq, D.  2012\.  Nonparametric statistics for stochastic processes: estimation and prediction Nonparametric statistics for stochastic processes: estimation and prediction ( 110).  Springer Science & Business Media.
  * Bryc Dembo (1996) bryc1996largeBryc, W. Dembo, A.  1996\.  Large deviations and strong mixing Large deviations and strong mixing.  Annales de l’IHP Probabilités et statistiques Annales de l’ihp probabilités et statistiques ( 32, 549–569).
  * Carbon (1983) carbon1983inegaliteCarbon, M.  1983\.  Inégalité de Bernstein pour les processus fortement mélangeants, non nécessairement stationnaires. Applications Inégalité de bernstein pour les processus fortement mélangeants, non nécessairement stationnaires. applications.  Comptes rendus des séances de l’Académie des sciences. Série 1, Mathématique2975303–306.
  * Cattaneo . (2020) cattaneo2020simpleCattaneo, MD., Jansson, M. Ma, X.  2020\.  Simple local polynomial density estimators Simple local polynomial density estimators.  Journal of the American Statistical Association1155311449–1455.
  * Chen . (2016) chen2016nonparametricChen, YC., Genovese, CR., Tibshirani, RJ. Wasserman, L.  2016\.  Nonparametric modal regression Nonparametric modal regression.  The Annals of Statistics442489–514.
  * Collomb (1984) collomb1984proprietesCollomb, G.  1984\.  Propriétés de convergence presque complète du prédicteur à noyau Propriétés de convergence presque complète du prédicteur à noyau.  Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete663441–460.
  * Collomb . (1986) collomb1986noteCollomb, G., Härdle, W. Hassani, S.  1986\.  A note on prediction via estimation of the conditional mode function A note on prediction via estimation of the conditional mode function.  Journal of Statistical Planning and Inference15227–236.
  * Davydov (1968) davydov1968convergenceDavydov, YA.  1968\.  Convergence of distributions generated by stationary stochastic processes Convergence of distributions generated by stationary stochastic processes.  Theory of Probability & Its Applications134691–696.
  * Delyon (2009) delyon2009exponentialDelyon, B.  2009\.  Exponential inequalities for sums of weakly dependent variables Exponential inequalities for sums of weakly dependent variables.  Electronic Journal of Probability14752–779.
  * Devroye Wise (1980) devroye1980detectionDevroye, L. Wise, GL.  1980\.  Detection of abnormal behavior via nonparametric estimation of the support Detection of abnormal behavior via nonparametric estimation of the support.  SIAM Journal on Applied Mathematics383480–488.
  * Doukhan Louhichi (1999) doukhan1999newDoukhan, P. Louhichi, S.  1999\.  A new weak dependence condition and applications to moment inequalities A new weak dependence condition and applications to moment inequalities.  Stochastic processes and their applications842313–342.
  * Gallant (2009) gallant2009nonlinearGallant, AR.  2009\.  Nonlinear statistical models Nonlinear statistical models ( 310).  John Wiley & Sons.
  * Gerstenberger (2018) gerstenberger2018robustGerstenberger, C.  2018\.  Robust Wilcoxon-Type Estimation of Change-Point Location Under Short-Range Dependence Robust wilcoxon-type estimation of change-point location under short-range dependence.  Journal of Time Series Analysis39190–104.
  * Gorodetskii (1978) gorodetskii1978strongGorodetskii, V.  1978\.  On the strong mixing property for linear sequences On the strong mixing property for linear sequences.  Theory of Probability & Its Applications222411–413.
  * Györfi . (2002) gyorfi2002distributionGyörfi, L., Kohler, M., Krzyżak, A. Walk, H.  2002\.  A distribution-free theory of nonparametric regression A distribution-free theory of nonparametric regression ( 1).  Springer.
  * Hallin . (2001) hallin2001densityHallin, M., Lu, Z. Tran, LT.  2001\.  Density estimation for spatial linear processes Density estimation for spatial linear processes.  Bernoulli657–668.
  * Hallin . (2004) hallin2004kernelHallin, M., Lu, Z. Tran, LT.  2004\.  Kernel density estimation for spatial processes: the L1 theory Kernel density estimation for spatial processes: the l1 theory.  Journal of Multivariate Analysis88161–75.
  * Hang Steinwart (2017) hang2017bernsteinHang, H. Steinwart, I.  2017\.  A Bernstein-type inequality for some mixing processes and dynamical systems with an application to learning A bernstein-type inequality for some mixing processes and dynamical systems with an application to learning.  The Annals of Statistics452708–743.
  * Hang . (2018) hang2018kernelHang, H., Steinwart, I., Feng, Y. Suykens, JA.  2018\.  Kernel density estimation for dynamical systems Kernel density estimation for dynamical systems.  The Journal of Machine Learning Research1911260–1308.
  * Hansen (1991) hansen1991garchHansen, BE.  1991\.  GARCH (1, 1) processes are near epoch dependent Garch (1, 1) processes are near epoch dependent.  Economics Letters362181–186.
  * Hansen (2008) hansen2008uniformHansen, BE.  2008\.  Uniform convergence rates for kernel estimation with dependent data Uniform convergence rates for kernel estimation with dependent data.  Econometric Theory243726–748.
  * Ibragimov (1962) ibragimov1962someIbragimov, IA.  1962\.  Some limit theorems for stationary processes Some limit theorems for stationary processes.  Theory of Probability & Its Applications74349–382.
  * Jenish (2012) jenish2012nonparametricJenish, N.  2012\.  Nonparametric spatial regression under near-epoch dependence Nonparametric spatial regression under near-epoch dependence.  Journal of Econometrics1671224–239.
  * Jenish Prucha (2009) jenish2009centralJenish, N. Prucha, IR.  2009\.  Central limit theorems and uniform laws of large numbers for arrays of random fields Central limit theorems and uniform laws of large numbers for arrays of random fields.  Journal of econometrics150186–98.
  * Jenish Prucha (2012) jenish2012spatialJenish, N. Prucha, IR.  2012\.  On spatial processes and asymptotic inference under near-epoch dependence On spatial processes and asymptotic inference under near-epoch dependence.  Journal of econometrics1701178–190.
  * Karunamuni Alberts (2005) karunamuni2005boundaryKarunamuni, RJ. Alberts, T.  2005\.  On boundary correction in kernel density estimation On boundary correction in kernel density estimation.  Statistical Methodology23191–212.
  * Kontorovich Raginsky (2017) kontorovich2017concentrationKontorovich, A. Raginsky, M.  2017\.  Concentration of measure without independence: a unified approach via the martingale method Concentration of measure without independence: a unified approach via the martingale method.  Convexity and Concentration Convexity and concentration ( 183–210).  Springer.
  * Krebs (2018) krebs2018bernsteinKrebs, JT.  2018\.  A Bernstein inequality for exponentially growing graphs A bernstein inequality for exponentially growing graphs.  Communications in Statistics-Theory and Methods47205097–5106.
  * Lahiri (2003) lahiri2003centralLahiri, S.  2003\.  Central limit theorems for weighted sums of a spatial process under a class of stochastic and fixed designs Central limit theorems for weighted sums of a spatial process under a class of stochastic and fixed designs.  Sankhyā: The Indian Journal of Statistics356–388.
  * Lee (1989) lee1989modeLee, Mj.  1989\.  Mode regression Mode regression.  Journal of Econometrics423337–349.
  * Li . (2010) lilocalLi, D., Lu, Z. Linton, O.  2010\.  Local Linear Fitting under Near Epoch Dependence: Uniform Consistency with Convergence Rate Local linear fitting under near epoch dependence: Uniform consistency with convergence rate.  Discussion PaperNo:EM/2010/549https://sticerd.lse.ac.uk/dps/em/em549.pdf.
  * Li . (2012) li2012localLi, D., Lu, Z. Linton, O.  2012\.  Local linear fitting under near epoch dependence: uniform consistency with convergence rates Local linear fitting under near epoch dependence: uniform consistency with convergence rates.  Econometric Theory285935–958.
  * Liebscher (1996) liebscher1996strongLiebscher, E.  1996\.  Strong convergence of sums of $\alpha$-mixing random variables with applications to density estimation Strong convergence of sums of $\alpha$-mixing random variables with applications to density estimation.  Stochastic Processes and Their Applications65169–80.
  * Ling (2007) ling2007testingLing, S.  2007\.  Testing for change points in time series models and limiting theorems for NED sequences Testing for change points in time series models and limiting theorems for ned sequences.  The Annals of Statistics3531213–1237.
  * Linton Mammen (2005) linton2005estimatingLinton, O. Mammen, E.  2005\.  Estimating Semiparametric ARCH($\infty$) Models by Kernel Smoothing Methods Estimating semiparametric arch($\infty$) models by kernel smoothing methods.  Econometrica733771–836.
  * Lu (2001) lu2001asymptoticLu, Z.  2001\.  Asymptotic normality of kernel density estimators under dependence Asymptotic normality of kernel density estimators under dependence.  Annals of the Institute of Statistical Mathematics533447–468.
  * Lu Linton (2007) lu2007localLu, Z. Linton, O.  2007\.  Local linear fitting under near epoch dependence Local linear fitting under near epoch dependence.  Econometric Theory23137–70.
  * Lu Tjøstheim (2014) lu2014nonparametricLu, Z. Tjøstheim, D.  2014\.  Nonparametric estimation of probability density functions for irregularly observed spatial data Nonparametric estimation of probability density functions for irregularly observed spatial data.  Journal of the American Statistical Association1095081546–1564.
  * McLeish (1974) mcleish1974dependentMcLeish, DL.  1974\.  Dependent central limit theorems and invariance principles Dependent central limit theorems and invariance principles.  the Annals of Probability24620–628.
  * McLeish (1975) mcleish1975maximalMcLeish, DL.  1975\.  A maximal inequality and dependent strong laws A maximal inequality and dependent strong laws.  The Annals of probability35829–839.
  * Merlevède . (2009) merlevede2009bernsteinMerlevède, F., Peligrad, M. Rio, E.  2009\.  Bernstein inequality and moderate deviations under strong mixing conditions Bernstein inequality and moderate deviations under strong mixing conditions.  High dimensional probability V: the Luminy volume High dimensional probability v: the luminy volume ( 273–292).  Institute of Mathematical Statistics.
  * Merlevède . (2011) merlevede2011bernsteinMerlevède, F., Peligrad, M. Rio, E.  2011\.  A Bernstein type inequality and moderate deviations for weakly dependent sequences A bernstein type inequality and moderate deviations for weakly dependent sequences.  Probability Theory and Related Fields1513-4435–474.
  * Modha Masry (1996) modha1996minimumModha, DS. Masry, E.  1996\.  Minimum complexity regression estimation with weakly dependent observations Minimum complexity regression estimation with weakly dependent observations.  IEEE Transactions on Information Theory4262133–2145.
  * Parzen (1962) parzen1962estimationParzen, E.  1962\.  On estimation of a probability density function and mode On estimation of a probability density function and mode.  The annals of mathematical statistics3331065–1076.
  * Pollard (2015) PollardPollard, D.  2015\.  Chaining Chaining.  http://www.stat.yale.edu/ pollard/Books/Mini/Chaining.pdf.
  * Polonik (1995) polonik1995measuringPolonik, W.  1995\.  Measuring mass concentrations and estimating density contour clusters-an excess mass approach Measuring mass concentrations and estimating density contour clusters-an excess mass approach.  The annals of Statistics855–881.
  * Ren Lu (2020) ren2020localRen, X. Lu, Z.  2020\.  Local Linear Quantile Regression for Time Series Under Near Epoch Dependence Local linear quantile regression for time series under near epoch dependence.  Available at SSRN 3555740.
  * Rigollet Vert (2009) rigollet2009optimalRigollet, P. Vert, R.  2009\.  Optimal rates for plug-in estimators of density level sets Optimal rates for plug-in estimators of density level sets.  Bernoulli1541154–1178.
  * Rio (1995) rio1995functionalRio, E.  1995\.  The functional law of the iterated logarithm for stationary strongly mixing sequences The functional law of the iterated logarithm for stationary strongly mixing sequences.  The Annals of Probability1188–1203.
  * Sager Thisted (1982) sager1982maximumSager, TW. Thisted, RA.  1982\.  Maximum likelihood estimation of isotonic modal regression Maximum likelihood estimation of isotonic modal regression.  The Annals of Statistics690–707.
  * Scott (2015) scott2015multivariateScott, DW.  2015\.  Multivariate density estimation: theory, practice, and visualization Multivariate density estimation: theory, practice, and visualization.  John Wiley & Sons.
  * Steinwart . (2005) steinwart2005classificationSteinwart, I., Hush, D. Scovel, C.  2005\.  A Classification Framework for Anomaly Detection. A classification framework for anomaly detection.  Journal of Machine Learning Research62.
  * Stone (1980) stone1980optimalStone, CJ.  1980\.  Optimal rates of convergence for nonparametric estimators Optimal rates of convergence for nonparametric estimators.  The annals of Statistics1348–1360.
  * Stone (1982) stone1982optimalStone, CJ.  1982\.  Optimal global rates of convergence for nonparametric regression Optimal global rates of convergence for nonparametric regression.  The annals of statistics1040–1053.
  * Talagrand (1996) talagrand1996majorizingTalagrand, M.  1996\.  Majorizing measures: the generic chaining Majorizing measures: the generic chaining.  The Annals of Probability2431049–1103.
  * Tsybakov (20041) tsybakov2004introductionTsybakov, AB.  20041\.  Introduction to nonparametric estimation, 2009 Introduction to nonparametric estimation, 2009.  URL https://doi. org/10.1007/b13794. Revised and extended from the910.
  * Tsybakov (20042) tsybakov2004optimalTsybakov, AB.  20042\.  Optimal aggregation of classifiers in statistical learning Optimal aggregation of classifiers in statistical learning.  The Annals of Statistics321135–166.
  * Valenzuela-Domínguez . (2017) valenzuela2017bernsteinValenzuela-Domínguez, E., Krebs, JT. Franke, JE.  2017\.  A Bernstein inequality for spatial lattice processes A bernstein inequality for spatial lattice processes.  arXiv preprint arXiv:1702.02023.
  * van der Vaart Weller (1996) van1996weakvan der Vaart, A. Weller, J.  1996\.  Weak convergence and empirical processes: with applications to statistics Weak convergence and empirical processes: with applications to statistics.  Springer Science & Business Media.
  * Xu Lee (20151) xu2015maximumXu, X. Lee, Lf.  20151\.  Maximum likelihood estimation of a spatial autoregressive Tobit model Maximum likelihood estimation of a spatial autoregressive tobit model.  Journal of Econometrics1881264–280.
  * Xu Lee (20152) xu2015spatialXu, X. Lee, Lf.  20152\.  A spatial autoregressive model with a nonlinear transformation of the dependent variable A spatial autoregressive model with a nonlinear transformation of the dependent variable.  Journal of Econometrics18611–18.
  * Xu Lee (2018) xu2018sieveXu, X. Lee, Lf.  2018\.  Sieve maximum likelihood estimation of the spatial autoregressive Tobit model Sieve maximum likelihood estimation of the spatial autoregressive tobit model.  Journal of Econometrics203196–112.
  * Q. Yao (2003) yao2003exponentialYao, Q.  2003\.  Exponential inequalities for spatial processes and uniform convergence rates for density estimation Exponential inequalities for spatial processes and uniform convergence rates for density estimation.  Development Of Modern Statistics And Related Topics: In Celebration of Professor Yaoting Zhang’s 70th Birthday Development of modern statistics and related topics: In celebration of professor yaoting zhang’s 70th birthday ( 118–128).  World Scientific.
  * W. Yao Li (2014) yao2014newYao, W. Li, L.  2014\.  A new regression model: modal linear regression A new regression model: modal linear regression.  Scandinavian Journal of Statistics413656–671.
  * W. Yao . (2012) yao2012localYao, W., Lindsay, BG. Li, R.  2012\.  Local modal regression Local modal regression.  Journal of nonparametric statistics243647–663.

##  Appendix A Proof Related to Section 2

Proof of Proposition 1
For (i), we prove it by contradiction. Set $B(s,d_{0}/2)$ as an open ball in
$\mathbf{R}^{d}$ whose center is $s\in\Gamma$ and radius is $d_{0}/2$.
Obviously, $I(s,\sqrt{2}d_{0}/2)\subset B(s,d_{0}/2)$. If $B(s,d_{0}/2)$
contains two elements in $\Gamma_{N}$, say $s_{iN}$ and $s_{jN}$, a direct
consequence is $||s_{jN}-s||\lor||s_{iN}-s||\leq d_{0}/2$. Thus, due to
triangle inequality, $||s_{iN}-s_{jN}||\leq||s_{jN}-s||+||s_{iN}-s||\leq
d_{0}$, which violates Assumption 2. Since $I(s,\sqrt{2}d_{0}/2)\subset
B(s,d_{0}/2)$, we finish the proof.
For (ii), note that for $\forall h>0$,
$\sqrt{2}h/d_{0}\leq[\sqrt{2}h/d_{0}]+1$. Thus $I(s,h)\subset
I(s,([\sqrt{2}h/d_{0}]+1)\sqrt{2}d_{0}/2)$. Clearly,
$I(s,([\sqrt{2}h/d_{0}]+1)\sqrt{2}d_{0}/2)$ can be partitioned into
$([\sqrt{2}h/d_{0}]+1)^{d}$ cubes whose length of each is $\sqrt{2}d_{0}/2$.
Then in light of Proposition 1 (i),
$|I(s,h)\cap\Gamma_{N}|\leq([\sqrt{2}h/d_{0}]+1)^{d}\leq(2\sqrt{2}h/d_{0})^{d}.$
Proof of Proposition 2
Given a $s_{iN}\in\Gamma_{N}$, for $\forall k=1,...,D$, let $Z_{s_{iN}}^{l}$
be the k-th element of random vector $Z_{s_{iN}}$. Due to some geometrical
property of Euclidean space, we have
$|Z_{s_{iN}}^{k}-E[Z_{s_{iN}}^{k}|\mathcal{F}_{iN}(r)]|\leq||Z_{s_{iN}}-E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]||\leq\begin{matrix}\sum_{k=1}^{D}\end{matrix}|Z_{s_{iN}}^{k}-E[Z_{s_{iN}}^{k}|\mathcal{F}_{iN}(r)]|.$
Together with the monotonicity of $L^{p}$-norm and Minkovski inequality, we
finish the proof of Proposition 2.
Proof of Proposition 3
Since $f$ is Lipschtz on $\mathcal{Z}$, we have

$\displaystyle|f(Z_{s_{iN}})-E[f(Z_{s_{iN}})|\mathcal{F}_{iN}(r)]|$
$\displaystyle\leq$
$\displaystyle|f(Z_{s_{iN}})-f(E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)])|+$
$\displaystyle|f(E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)])-E[f(Z_{s_{iN}})|\mathcal{F}_{iN}(r)]|$
$\displaystyle\leq$ $\displaystyle
Lip(f)||Z_{s_{iN}}-E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]||+$
$\displaystyle\left|E\left\\{f(E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)])-f(Z_{s_{iN}})|\mathcal{F}_{iN}(r)\right\\}\right|$
$\displaystyle:=$ $\displaystyle A_{1}+A_{2}$

Apparently, due to the Lipschitz property of $f$,

$\displaystyle||A_{1}||_{p}\leq
Lip(f)||Z_{s_{iN}}-E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]||_{p}.$

Similarly, based on basic properties of conditional expectaion,

$\displaystyle||A_{2}||_{p}$
$\displaystyle=||E\left\\{f(E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)])-f(Z_{s_{iN}})|\mathcal{F}_{iN}(r)\right\\}||_{p}$
$\displaystyle=\left\\{E|E\left\\{f(E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)])-f(Z_{s_{iN}})|\mathcal{F}_{iN}(r)\right\\}|^{p}\right\\}^{1/p}$
$\displaystyle\leq\left\\{EE\left\\{|f(E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)])-f(Z_{s_{iN}})|^{p}|\mathcal{F}_{iN}(r)\right\\}\right\\}^{1/p}$
$\displaystyle\leq
Lip(f)\left\\{E\left\\{E\left\\{||E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]-Z_{s_{iN}}||^{p}|\mathcal{F}_{iN}(r)\right\\}\right\\}\right\\}^{1/p}$
$\displaystyle=Lip(f)||E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]-Z_{s_{iN}}||_{p}$

Finally, by applying Minkovski inequality and definition of NED, we obtain

$\displaystyle||f(Z_{s_{iN}})-E[f(Z_{s_{iN}})|\mathcal{F}_{iN}(r)]||_{p}\leq||A_{1}||_{p}+||A_{2}||_{p}\leq
2Lip(f)\alpha_{iN}\psi_{Z}(r).$

Proof of Proposition 4
Define $g^{r}=E[g(Z_{s_{1N}},..,Z_{s_{|U|N}})|\mathcal{F}_{U}(r)]$ and
$h^{r}=E[h(Z_{t_{1N}},..,Z_{t_{|V|N}})|\mathcal{F}_{V}(r)]$, where
$\mathcal{F}_{U}(r)=\sigma(\bigcup_{i=1}^{|U|}\mathcal{F}_{iN}(r))$,
$\mathcal{F}_{V}(r)=\sigma(\bigcup_{j=1}^{|V|}\mathcal{F}_{jN}(r))$. Then, by
denoting $\triangle g=g-g^{r}$, $\triangle h=h-h^{r}$, we have

$\displaystyle
Cov(g(Z_{s_{1N}},...,Z_{s_{|U|N}}),h(Z_{t_{1N}},...,Z_{t_{|V|N}}))$
$\displaystyle:=Cov(f,g)=Cov(\triangle g+g^{r},\triangle h+h^{r})$
$\displaystyle=Cov(\triangle g,\triangle h)+Cov(g^{r},\triangle
h)+Cov(\triangle g,h^{r})+Cov(g^{r},h^{r}).$

Note $Cov(\triangle g,\triangle h)=E(\triangle g\triangle h)$, since
$E\triangle h=E\triangle g=0$. Meanwhile,

$\displaystyle Cov(\triangle g,h^{r})$ $\displaystyle=E[(\triangle
g-E\triangle g)(h^{r}-Eh^{r})]=E[\triangle g(h^{r}-Eh^{r})]$
$\displaystyle=E[\triangle g(h^{r}+\triangle h-\triangle h+E\triangle
h-E\triangle h-Eh^{r})]$ $\displaystyle=E[\triangle g(h-Eh)]-E[\triangle
g\triangle h]$

Similarly, $Cov(g^{r},\triangle h)=E[(g-Eg)\triangle h]-E[\triangle g\triangle
h]$.
$\therefore$ $Cov(g,h)=E[\triangle h(g-Eg)]+E[\triangle g(h-Eh)]-E[\triangle
g\triangle h]+Cov(g^{r},h^{r})$.
Therefore, for $\forall p\geq 2$ and $q=\frac{p}{p-1}$,

$\displaystyle|Cov(g,h)|$ $\displaystyle\leq E|\triangle g\triangle
h|+E|\triangle h(g-Eg)|+E|\triangle g(h-Eh)|+|Cov(g^{r},h^{r})|$
$\displaystyle\leq||\triangle g||_{q}||\triangle h||_{p}+||\triangle
h||_{q}||g-Eg||_{p}+||\triangle g||_{q}||h-Eh||_{p}+|Cov(g^{r},h^{r})|$
$\displaystyle:=I_{1}+I_{2}+I_{3}+I_{4}$

For $I_{4}$, since $g^{r}$ and $h^{r}$ are $\mathcal{F}_{U}(r)$ and
$\mathcal{F}_{V}(r)$ measurable, according to davydov1968convergence, for
$\forall\delta>0$, we have

$\displaystyle
I_{4}\leq\phi(|U|,|V|)\psi_{\epsilon}^{\frac{p+\delta-2}{p+\delta}}(\rho(U,V))||g^{r}||_{p+\delta}||h^{r}||_{p+\delta}.$

Also note that,

$\displaystyle||g^{r}||_{p}$
$\displaystyle=(E|E[g|\mathcal{F}_{U}(r)]|^{p})^{1/p}\leq(E[E[|g|^{p}|\mathcal{F}_{U}(r)]])^{1/p}=||g||_{p}\leq||g||_{p+\delta}.$
$\therefore
I_{4}\leq\phi(|U|,|V|)\psi_{\epsilon}^{\frac{p+\delta-2}{p+\delta}}(\rho(U,V))||g||_{p+\delta}||h||_{p+\delta}.$

Please remark $||g-Eg||_{p}\leq||g||_{p}+\int|Eg|^{p}dP\leq 2||g||_{p}$, where
the second inequality is due to Jensen inequality. Similarly, we have
$||h-Eh||_{p}\leq 2||h||_{p}$. Since $p\geq 2$, $q=p/p-1\leq p$. Thus
$||\triangle g||_{q}\leq||\triangle g||_{p}$ and $||\triangle
h||_{q}\leq||\triangle h||_{p}$. Moreover, to bound $||\triangle g||_{p}$, by
applying Minkovski inequality, we have

$\displaystyle||\triangle
g||_{p}\leq||g-\bar{g}||_{p}+||\bar{g}-g^{r}||_{p}:=G_{1}+G_{2},$

where $\bar{g}=g(E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)],i=1,..,|U|)$. For $G_{1}$,
since g is coordinate wise Lipschitz continuous on $\mathcal{Z}^{|U|}$, we
have

$\displaystyle G_{1}\leq
Lip(g)\begin{matrix}\sum_{i=1}^{|U|}\end{matrix}||Z_{s_{iN}}-E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]||_{p}\leq
Lip(g)(\begin{matrix}\sum_{i=1}^{|U|}\end{matrix}\alpha_{iN})\psi_{Z}(r).$

As for $G_{2}$, similarly to bounding ”$||A_{2}||_{p}$” in Proposition 3,

$\displaystyle\because|\bar{g}-g^{r}|$
$\displaystyle=|E[\bar{g}-g^{r}|\mathcal{F}_{U}(r)]|\leq
E[|\bar{g}-g^{r}||\mathcal{F}_{U}(r)]$ $\displaystyle\leq
Lip(g)\begin{matrix}\sum_{i=1}^{|U|}\end{matrix}E[|Z_{s_{iN}}-E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]||\mathcal{F}_{U}(r)],$
$\displaystyle\therefore G_{2}$ $\displaystyle\leq
Lip(g)||\begin{matrix}\sum_{i=1}^{|U|}\end{matrix}E[|Z_{s_{iN}}-E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]||\mathcal{F}_{U}(r)]||_{p}$
$\displaystyle\leq
Lip(g)\begin{matrix}\sum_{i=1}^{|U|}\end{matrix}\left(E|E[|Z_{s_{iN}}-E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]||\mathcal{F}_{U}(r)]|^{p}\right)^{\frac{1}{p}}$
$\displaystyle\leq
Lip(g)\begin{matrix}\sum_{i=1}^{|U|}\end{matrix}\left(E(E[|Z_{s_{iN}}-E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]|^{p}|\mathcal{F}_{U}(r)])\right)^{\frac{1}{p}}$
$\displaystyle=Lip(g)\begin{matrix}\sum_{i=1}^{|U|}\end{matrix}||Z_{s_{iN}}-E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]||_{p}$
$\displaystyle\leq
Lip(g)(\begin{matrix}\sum_{i=1}^{|U|}\end{matrix}\alpha_{iN})\psi_{Z}(r).$

Therefore, we prove that $||\triangle g||_{p}\leq
2Lip(g)(\begin{matrix}\sum_{j=1}^{|V|}\end{matrix}\alpha_{jN})\psi_{Z}(r)$.
Similarly, we can obtain $||\triangle h||_{p}\leq
2Lip(h)(\begin{matrix}\sum_{j=1}^{|V|}\end{matrix}\alpha_{jN})\psi_{Z}(r)$ as
well. Together with all the results above, we obtain

$\displaystyle I_{1}\leq||\triangle g||_{p}||\triangle h||_{p}\leq
4Lip(g)Lip(h)(\begin{matrix}\sum_{i=1}^{|U|}\end{matrix}\alpha_{iN})(\begin{matrix}\sum_{j=1}^{|V|}\end{matrix}\alpha_{jN})\psi_{Z}^{2}(r),$
$\displaystyle I_{2}\leq||\triangle g||_{p}||h-Eh||_{p}\leq 2||\triangle
g||_{p}||h||_{p}\leq
4Lip(g)(\begin{matrix}\sum_{i=1}^{|U|}\end{matrix})\psi_{Z}(r),$
$\displaystyle I_{3}\leq
4Lip(h)(\begin{matrix}\sum_{j=1}^{|V|}\end{matrix}\alpha_{jN})\psi_{Z}(r).$

Q.D.E

##  Appendix B Proof Related to Section 3

###### Lemma 1

Suppose process $Z$ is an $L^{p}$-NED random field on process $\epsilon$.
Provided that for $\forall s_{iN}\in\Gamma_{N}$, $||Z_{s_{iN}}||_{\infty}\leq
A$ and $||Z_{s_{iN}}||_{2}\leq\sigma$, for some $A$ and $\sigma$ strictly
larger 0, we have, for $\forall\ r>0$,

$||E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)||_{2}\leq\sigma,\
||E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)||_{\infty}\leq A.$

Proof: Note for any $p\geq 1$, since Jensen inequality still holds for
conditional expectation, we have

$\displaystyle||E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]||_{p}=\\{E|E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]|^{p}\\}^{\frac{1}{p}}$
$\displaystyle\leq$
$\displaystyle\\{E[E[|Z_{s_{iN}}|^{p}|\mathcal{F}_{iN}(r)]]\\}^{\frac{1}{p}}=||Z_{s_{iN}}||_{p}.$

Thus, the results when $p=2$ and $\infty$ respectively.

###### Lemma 2

(davydov1968convergence) Suppose $\epsilon$ is an $\alpha$-mixing random field
defined as Definition 2. For any finite subset $U,\ V\subset\Gamma_{N}$, let
$\xi$ and $\eta$ be $\sigma(U)$ and $\sigma(V)$ measurable respectively. Then,
for any $p_{1},\ p_{2}\geq 1$ and
$\frac{1}{p_{2}}+\frac{1}{p_{1}}+\frac{1}{p_{3}}=1$, we have the following
covariance inequality,

$\displaystyle|Cov(\xi,\eta)|\leq\phi(|U|,|V|)^{\frac{1}{p_{1}}}||\xi||_{p_{2}}||\eta||_{p_{3}}.$

Proof of Theorem 1
The whole proof will be generally decomposed into three steps. The first step
is to introduce the blocking technique which is essentially the same as
hang2017bernstein. The second step is to bound the sum of the difference
between $Z_{s_{iN}}$ and its projection on sigma field $\mathcal{F}_{iN}(r)$
of any given $r>0$. The last step actually aims at deriving a concentration
inequality for $\alpha$-mixing random field. The method used in this step is
strongly motivated by valenzuela2017bernstein.

Step 1(Blocking)

Firstly, without loss of generality, we assume
$R_{N}=\prod_{k=1}^{d}(0,H_{k}^{N}]$ and
$H_{1}^{N}=H_{2}^{N}=\dots=H_{d_{1}}^{N}=H_{0}$, where $0\leq d_{1}<d$. Thus,
according to Assumption 3, we have , for $\forall k=d_{1}+1,...,d$, we have
$H_{k}^{N}\nearrow+\infty$, as $N\nearrow\infty.$ By denoting
$\hat{N}=\prod_{k=d_{1}+1}^{d}H_{k}^{N}$, Assumption 4 implies $Vol(R_{N})\leq
K^{d_{2}}N$, where $Vol(R_{N})=H_{0}^{d_{1}}\hat{N}$. Now, for
$k=d_{1}+1,...,d$, let

$m_{k}=\begin{cases}\frac{H_{k}^{N}}{H_{0}},&\mbox{if
}\frac{H_{k}^{N}}{H_{0}}\in\mathbf{N}\\\
\left[\frac{H_{k}^{N}}{H_{0}}\right]+1,&\mbox{else }\end{cases}.$

We thus partition $R_{N}$ into $\prod_{k=d_{1}+1}^{d}m_{k}$ many smaller cubes
whose length of each edge is $H_{0}$. Here we name these cubes as $B_{l}$,
$l=1,...,M$, $M=\prod_{k=d_{1}+1}^{d}m_{k}$. Obviously, based on Proposition
1, for each $l$,
$|B_{l}\cap\Gamma_{N}|\leq(\left[\frac{H_{0}}{\sqrt{2}d_{0}/2}\right]+1)^{d}:=C_{0}$.
By defining $G_{lN}=\sum_{i=1}^{N}\textbf{1}[s_{iN}\in B_{l}]Z_{s_{iN}}$,
conditions of $Z_{s_{iN}}$ indicate that, for $\forall\ l=1,...,M$,
$E[G_{lN}]=0,\ ||G_{lN}||_{\infty}\leq A\ \mbox{and}\ E[G_{lN}^{2}]\leq
C_{0}^{2}\sigma^{2}.$

Secondly, note that $\\{G_{lN}\\}$ can be regarded as ”area-located” spatial
data. Hence, index $l$ can be replaced by vector
$\textbf{j}\in\mathbf{Z}^{d_{2}+}$, $\textbf{j}=(j_{d_{1}+1},\dots,j_{d})$, in
which, for $\forall\ k=d_{1}+1,...,d$, $1\leq j_{k}\leq m_{k}$. Now, suppose
$\\{P_{\hat{N}}\\}$ is a positive-integer valued number sequence such that
$P_{\hat{N}}\nearrow+\infty$, as $\hat{N}\nearrow+\infty$. Denote
$[m_{k}/P_{\hat{N}}]$ as $L_{\hat{N}}^{k}$ and let
$r_{k}=m_{k}-P_{\hat{N}}L_{\hat{N}}^{k}$. Furthermore, define

$I_{jk}=\begin{cases}j_{k},\
j_{k}+P_{\hat{N}},...,j_{k}+L_{\hat{N}}^{k}P_{\hat{N}}\ &1\leq j_{k}\leq
r_{k}\\\ j_{k},\ j_{k}+P_{\hat{N}},...,j_{k}+(L_{\hat{N}}^{k}-1)P_{\hat{N}}\
&r_{k}<j_{k}\leq P_{\hat{N}},\end{cases}$

and $I_{\textbf{j}}=\prod_{k=d_{1}+1}^{d}I_{jk}$. Therefore
$\textbf{j}\in[1,P_{\hat{N}}]^{d_{2}}\cap\mathbf{Z}^{d_{2}+}$. Additionally,
for each vector j, equip it with a scalar j for later notation,
$j=1,2,...,P_{\hat{n}}^{d_{2}}$.

Above all, we rewrite the partial sum of random variables as follow,

$\displaystyle\frac{S_{N}}{N}$
$\displaystyle=\frac{\sum_{i=1}^{N}Z_{s_{iN}}}{N}=\frac{\sum_{l=1}^{M}\sum_{i=1}^{N}\textbf{1}[s_{iN}\in
B_{l}]Z_{s_{iN}}}{N}$
$\displaystyle=\frac{\sum_{\textbf{j}\in[1,P_{\hat{N}}]^{d_{2}}\cap\mathbf{Z}^{d_{2}+}}\sum_{\textbf{j}^{{}^{\prime}}\in
I_{\textbf{j}}}G_{\textbf{j}^{\prime}N}}{N}=\frac{\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\sum_{l=1}^{|I_{j}|}G_{lN}}{N}$
$\displaystyle=\left(\frac{\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\sum_{l=1}^{|I_{j}|}G_{lN}}{|I_{j}|}\right)\left(\frac{|I_{j}|}{N}\right):=\begin{matrix}\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\end{matrix}p_{j}\frac{\sum_{l=1}^{|I_{j}|}G_{lN}}{|I_{j}|}.$

Since
$\prod_{k-d_{1}+1}^{d}L_{\hat{N}}^{k}\leq|I_{j}|\leq\prod_{k-d_{1}+1}^{d}L_{\hat{N}}^{k}$,
according to the definition of $L_{\hat{N}}^{k}$, it can be easily specified
that

$\displaystyle
p_{j}=\left(\frac{2}{P_{\hat{N}}H_{0}}\right)^{d_{2}}\left(\frac{\hat{N}}{N}\right)\leq\left(\frac{2}{P_{\hat{N}}H_{0}}\right)^{d_{2}}\frac{Vol(R_{N})}{N}\leq\left(\frac{2}{P_{\hat{N}}H_{0}}\right)^{d_{2}}\frac{(KN^{\frac{1}{d_{2}}})^{d_{2}}}{N}=\frac{1}{P_{\hat{N}}^{d_{2}}},$

which implies $\sum p_{j}\leq 1$. Now we have

$\mathrm{P}\left(\left|\frac{S_{N}}{N}\right|\geq
t\right)=\mathrm{P}\left(\left|\begin{matrix}\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\end{matrix}p_{j}\begin{matrix}\sum_{l=1}^{|I_{j}|}\end{matrix}\frac{G_{lN}}{|I_{j}|}\right|\geq
t\right).$ $None$

Let $G_{lN}^{r}=\sum_{i=1}^{N}\textbf{1}[s_{iN}\in
B_{l}]E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]$ for any $0<r<P_{\hat{N}}/2$. Then we
bound (B.1) as follow,

$\displaystyle\mathrm{P}\left(\left|\frac{S_{N}}{N}\right|\geq t\right)$
$\displaystyle\leq\mathrm{P}\left(\left|\begin{matrix}\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\end{matrix}p_{j}\begin{matrix}\sum_{l=1}^{|I_{j}|}\end{matrix}\frac{G_{lN}-G_{lN}^{r}}{|I_{j}|}\right|\geq\frac{t}{2}\right)$
$\displaystyle+\mathrm{P}\left(\left|\begin{matrix}\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\end{matrix}p_{j}\begin{matrix}\sum_{l=1}^{|I_{j}|}\end{matrix}\frac{G_{lN}^{r}}{|I_{j}|}\right|\geq\frac{t}{2}\right)$
$\displaystyle:=Q_{1}+Q_{2}.$ (B.2)

Therefore, according to (B.2), it suffice to give upper bounds to $Q_{1}$ and
$Q_{2}$ respectively. As we mentioned before, $Q_{1}$ essentially means the
summation of projection error between $Z_{s_{iN}}$’s and
$E[Z_{s_{iN}}|\mathcal{F}_{iN}(r)]$. To bound $Q_{2}$ is actually equal to
bound the partial sums of some measurable functions of $\alpha$-mixing random
field.

Step 2(Projection Error)

Proof of this step is very simple since we only make use of Markov inequality
the definition of NED condition. More particularly, we have,

$\displaystyle Q_{1}$
$\displaystyle\leq\frac{2^{p}E\left|\begin{matrix}\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\end{matrix}p_{j}\begin{matrix}\sum_{l=1}^{|I_{j}|}\end{matrix}\frac{G_{lN}-G_{lN}^{r}}{|I_{j}|}\right|^{p}}{t^{p}}$
$\displaystyle\leq\frac{\left(2||\begin{matrix}\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\end{matrix}p_{j}\begin{matrix}\sum_{l=1}^{|I_{j}|}\end{matrix}\frac{G_{lN}-G_{lN}^{r}}{|I_{j}|}||_{p}\right)^{p}}{t^{p}}$
$\displaystyle\leq\frac{\left(2\begin{matrix}\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\end{matrix}p_{j}\begin{matrix}\sum_{l=1}^{|I_{j}|}\frac{1}{|I_{j}|}\end{matrix}||G_{lN}-G_{lN}^{r}||_{p}\right)^{p}}{t^{p}}$
$\displaystyle\leq\frac{\left(2\begin{matrix}\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\end{matrix}p_{j}\max_{1\leq
l\leq|I_{j}|}||G_{lN}-G_{lN}^{r}||_{p}\right)^{p}}{t^{p}}$
$\displaystyle\leq\left(\frac{2\left(N^{\kappa}\nu^{-br^{\gamma}}\right)}{t}\right)^{p},\
\forall\ p\geq 1,\ 0\leq r<\frac{P_{\hat{N}}}{2}.$

The third and the fifth inequalities are due to Minkovski inequality of
$L^{p}$-norm and definition of $L^{p}$ NED condition.

Step 3(Bernstein-type Inequality for Process $\\{G_{lN}^{r}\\}$)

Due to Chernoff’s approach, we have, for $\forall\ \lambda>0$,

$\displaystyle\mathrm{P}\left(\begin{matrix}\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\end{matrix}p_{j}\begin{matrix}\sum_{l=1}^{|I_{j}|}\end{matrix}\frac{G_{lN}^{r}}{|I_{j}|}\geq\frac{t}{2}\right)\leq\exp\left(-\frac{\lambda
t}{2}\right)E\left[\exp\left(\lambda\frac{\begin{matrix}\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\end{matrix}p_{j}\begin{matrix}\sum_{l=1}^{|I_{j}|}\end{matrix}G_{lN}^{r}}{|I_{j}|}\right)\right].$

Recall for each $j$, $p_{j}\leq\frac{1}{P_{\hat{N}}^{d_{2}}}$. Due to the
monotonicity of exponential function and Jensen inequality,

$\displaystyle
E\left[\exp\left(\lambda\frac{\begin{matrix}\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\end{matrix}p_{j}\begin{matrix}\sum_{l=1}^{|I_{j}|}\end{matrix}G_{lN}^{r}}{|I_{j}|}\right)\right]\leq\begin{matrix}\sum_{j=1}^{P_{\hat{N}}^{d_{2}}}\end{matrix}\frac{1}{P_{\hat{N}}^{d_{2}}}E\left[\exp\left(\lambda\frac{\begin{matrix}\sum_{l=1}^{|I_{j}|}\end{matrix}G_{lN}^{r}}{|I_{j}|}\right)\right].$

Thus, it suffice to obtain an upper bound of the Laplace transformation of
$\sum_{l=1}^{|I_{j}|}G_{lN}^{r}/|I_{j}|$ uniformly over $j$.

Given $0\leq r<P_{\hat{N}}/2$, define $U_{l}=\cup_{s_{iN}\in
B_{l}}B(s_{iN},r)$ and $\sigma(U_{l})=\sigma(\epsilon_{s_{jN}},s_{jN}\in
U_{l})$. Then $G_{lN}^{r}$ is $\sigma(U_{l})$ measurable and

$|\\{s_{jN}\in U_{l}\\}|\leq([\sqrt{2}(H_{0}+2r)/d_{0}]+1)^{d}:=C(r).$

According to Lemma 1 and definition of $G_{lN}^{r}$, it’s easy to obtain, for
any $l=1,\dots,|I_{j}|$, $||G_{lN}^{r}||_{\infty}\leq C_{0}A$,
$||G_{lN}^{r}]||_{2}\leq\sigma$. Furthermore, $G_{lN}^{r}$ can be regarded as
a measurable function of $\epsilon_{s_{iN}}$’s whose $s_{iN}\in U_{l}$. Since
random field $\epsilon$ is an $\alpha$-mixing random field(array), by applying
Lemma 2, we can do the following calculation.

For $\forall\ p_{2},\ p_{3}\geq 1$ and $\sum_{i=1}^{3}1/p_{i}=1$, by letting
$\sum_{l=1}^{n}G_{lN}^{r}/{|I_{j}|}=S_{nN}^{r}$, $n=2,\dots,|I_{j}|$,

$\displaystyle
E\left[\exp\left(\frac{\lambda\sum_{l=1}^{|I_{j}|}G_{lN}^{r}}{|I_{j}|}\right)\right]$
$\displaystyle=$ $\displaystyle E\left[\exp\left(\lambda
S_{|I_{j}|N}^{r}\right)\right]=E\left[\exp\left(\lambda
S_{|I_{j}|-1,N}^{r}\right)\exp\left(\frac{\lambda
G_{|I_{j}|N}^{r}}{|I_{j}|}\right)\right]$ $\displaystyle\leq$
$\displaystyle\left|Cov\left(\exp\left(\lambda
S_{|I_{j}|-1,N}^{r}\right),\exp\left(\frac{\lambda
G_{|I_{j}|N}^{r}}{|I_{j}|}\right)\right)\right|+E\left[\exp\left(\lambda
S_{|I_{j}|-1,N}^{r}\right)\right]E\left[\exp\left(\frac{\lambda
G_{|I_{j}|N}^{r}}{|I_{j}|}\right)\right]$ $\displaystyle\leq$
$\displaystyle\left\\{\left[(|I_{j}|C(r))^{\tau}\nu^{-b(P_{\hat{N}}-2r)^{\gamma}}\right]^{\frac{1}{p_{1}}}\left\|\exp\left(\frac{\lambda
G_{|I_{j}|N}^{r}}{|I_{j}|}\right)\right\|_{p_{3}}+E\left[\exp\left(\frac{\lambda
G_{|I_{j}|N}^{r}}{|I_{j}|}\right)\right]\right\\}\left\|\exp\left(\lambda
S_{|I_{j}|-1,N}^{r}\right)\right\|_{p_{2}}$
$\displaystyle\textrm{let}p_{3}=\infty,\ \textrm{thus}\
\frac{1}{p_{1}}+\frac{1}{p_{2}}=1,$ $\displaystyle\leq$
$\displaystyle\left\\{\left[(|I_{j}|C(r))^{\tau}\nu^{-b(P_{\hat{N}}-2r)^{\gamma}}\right]^{\frac{1}{p_{1}}}\left\|\exp\left(\frac{\lambda
G_{|I_{j}|N}^{r}}{|I_{j}|}\right)\right\|_{\infty}+E\left[\exp\left(\frac{\lambda
G_{|I_{j}|N}^{r}}{|I_{j}|}\right)\right]\right\\}\left\|\exp\left(\lambda
S_{|I_{j}|-1,N}^{r}\right)\right\|_{p_{2}},$

repeat the calculation above $|I_{j}|-1$ times,

$\displaystyle\leq$
$\displaystyle\prod_{l=0}^{|I_{j}|-2}\left\\{\begin{matrix}\underbrace{\left(C(r)^{\tau}(|I_{j}|-l)^{\tau}\nu^{-b(P_{\hat{N}}-2r)^{\gamma}}\right)^{\frac{1}{p_{1}}}}\\\
:=E_{0}\end{matrix}\begin{matrix}\underbrace{\left\|\exp\left(\frac{\lambda
p_{2}^{l}G_{|I_{j}|-l,N}^{r}}{|I_{j}|}\right)\right\|_{\infty}}\\\
:=E_{1}\end{matrix}+\begin{matrix}\underbrace{E\left[\exp\left(\frac{\lambda
p_{2}^{2l}G_{|I_{j}|-l,N}^{r}}{|I_{j}|}\right)\right]}\\\
:=E_{2}\end{matrix}\right\\}^{\frac{1}{p_{2}^{l}}}$
$\displaystyle\times\begin{matrix}\underbrace{\left\|\exp\left(\frac{\lambda
G_{1,N}^{r}}{|I_{j}|}\right)\right\|_{p_{2}}}\\\ :=E_{3}\end{matrix}.$

Let $p_{2}=2^{\frac{1}{|I_{j}|}}$ and
$\frac{1}{p_{1}}=1-\frac{1}{p_{2}}=1-2^{-\frac{1}{|I_{j}|}}$.
$\because|I_{j}|\geq 1$, $\therefore 1<p_{2}\leq 2$,
$\frac{1}{2}\leq\frac{1}{p_{1}}<1$ and for $\forall l=0,\dots,|I_{j}|-2$,
$p_{2}^{l}\leq 2$. Now we bound $E_{k}$ uniformly over $l$, $k=0,1,2,3$. For
$E_{0}$, since $\nu>1$ and $\frac{1}{p_{1}}<1$, we have

$E_{0}\leq(C(r)|I_{j}|)^{\tau}\nu^{-\frac{b(P_{\hat{N}}-2r)^{\gamma}}{2}}$

For $E_{1}$, by assuming $\lambda<\frac{3|I_{j}|}{4C_{0}A}$, we obtain
$E_{1}\leq e^{\frac{3}{2}}$. Considering probabilistic measure is a finite
measure, $E_{3}$ can be bounded by $e$. At last, we focus on deriving an upper
bound of $E_{2}$. Since $\max_{1\leq l\leq|I_{j}||}p_{2}^{2l}\leq 4$,

$\displaystyle\exp\left(\frac{\lambda
p_{2}^{2l}G_{|I_{j}|-l,N}^{r}}{|I_{j}|}\right)\leq\exp\left(\frac{4\lambda
G_{|I_{j}|-l,N}^{r}}{|I_{j}|}\right)$ $\displaystyle(\delta:=4\lambda)\ $
$\displaystyle\leq 1+\frac{\delta G_{|I_{j}|-l,N}^{r}}{|I_{j}|}+\frac{(\delta
G_{|I_{j}|-l,N}^{r})^{2}}{2(|I_{j}|)^{2}}+\sum_{q=3}^{+\infty}\frac{1}{q!}\left(\frac{\delta
G_{|I_{j}|-l,N}^{r}}{|I_{j}|}\right)^{q}$ $\displaystyle\leq 1+\frac{\delta
G_{|I_{j}|-l,N}^{r}}{|I_{j}|}+\frac{(\delta
G_{|I_{j}|-l,N}^{r})^{2}}{2(|I_{j}|)^{2}}\left(1+\sum_{q=1}^{+\infty}\left(\frac{\delta
C_{0}A}{3|I_{j}|}\right)^{q}\right)$ $\displaystyle\leq 1+\frac{\delta
G_{|I_{j}|-l,N}^{r}}{|I_{j}|}+\frac{(\delta
G_{|I_{j}|-l,N}^{r})^{2}}{2(|I_{j}|)^{2}}\left(\frac{1}{1-\frac{\delta
C_{0}A}{3|I_{j}|}}\right).$

Therefore, we obtain that,

$\displaystyle E_{2}$ $\displaystyle\leq
1+\frac{1}{2}\left(\frac{\delta\sigma}{(|I_{j}|)}\right)^{2}\left(\frac{1}{1-\frac{\delta
C_{0}A}{3|I_{j}|}}\right)\leq\exp\left(\frac{8\lambda^{2}(C_{0}\sigma)^{2}}{|I_{j}|(|I_{j}|-\frac{4\lambda
C_{0}A}{3})}\right).$

Above all,

$\displaystyle
E\left[\exp\left(\frac{\lambda\sum_{l=1}^{|I_{j}|}G_{lN}^{r}}{|I_{j}|}\right)\right]$
$\displaystyle\leq$
$\displaystyle\prod_{l=0}^{|I_{j}|-2}\left\\{\sqrt{e}(C(r)|I_{j}|)^{\tau}\nu^{-\frac{b(P_{\hat{N}}-2r)^{\gamma}}{2}}+\exp\left(\frac{8\lambda^{2}(C_{0}\sigma)^{2}}{|I_{j}|(|I_{j}|-\frac{4\lambda
C_{0}A}{3})}\right)\right\\}^{\frac{1}{p_{2}^{l}}}\cdot e$ $\displaystyle\leq$
$\displaystyle
e\left\\{e^{\frac{3}{2}}(C(r)|I_{j}|)^{\tau}\nu^{-\frac{b(P_{\hat{N}}-2r)^{\gamma}}{2}}+\exp\left(\frac{8\lambda^{2}(C_{0}\sigma)^{2}}{|I_{j}|(|I_{j}|-\frac{4\lambda
C_{0}A}{3})}\right)\right\\}^{|I_{j}|}$ $\displaystyle\leq$ $\displaystyle
e\exp\left(e^{\frac{3}{2}}C(r)^{\tau}|I_{j}|^{\tau+1}\nu^{-\frac{b(P_{\hat{N}}-2r)^{\gamma}}{2}}\right)\exp\left(\frac{8\lambda^{2}(C_{0}\sigma)^{2}}{(|I_{j}|-\frac{4\lambda
C_{0}A}{3})}\right).$

The third inequality holds because for each $l$, $1/p_{2}^{l}\leq 1$. The
fourth inequality holds since $\lambda<3|I_{j}|/4C_{0}A$, which indicates
$|I_{j}|-3|I_{j}|/4C_{0}A>0$. Furthermore, note that

$\displaystyle C(r)$
$\displaystyle=\left(\left[\frac{H_{0}+2r}{\sqrt{2}d_{0}/2}\right]+1\right)^{d}\leq\left(\frac{2^{\frac{3}{2}}}{d_{0}}\right)^{d}(H_{0}+2r)^{d},$
$\displaystyle|I_{j}|$
$\displaystyle\leq\prod_{k=d_{1}+1}^{d}(L_{\hat{N}}^{k}+1)\leq
2^{d_{2}}\prod_{k=d_{1}+1}^{d}L_{\hat{N}}^{k}\leq
2^{d_{2}}\prod_{k=d_{1}+1}^{d}\frac{m_{k}}{P_{\hat{N}}}=\left(\frac{2}{P_{\hat{N}}}H_{0}\right)^{d_{2}}\hat{N}.$

Hence, by setting $r=\frac{P_{\hat{N}}}{3}$, by setting
$H_{0}\leq\frac{P_{\hat{N}}}{3}$ and some simple algebra,

$\displaystyle
E\left[\exp\left(\frac{\lambda\sum_{l=1}^{|I_{j}|}G_{lN}^{r}}{|I_{j}|}\right)\right]$
$\displaystyle\leq$ $\displaystyle
e\exp\left\\{e^{\frac{3}{2}}\frac{2^{\frac{3}{2}\tau
d+d_{2}(\tau+1)}}{H_{0}^{d_{2}(\tau+1)}}P_{\hat{N}}^{\tau
d-d_{2}(\tau+1)}\nu^{(\tau+1)\log_{\nu}\hat{N}-\frac{b}{2}\left(\frac{P_{\hat{N}}}{3}\right)^{\gamma}}\right\\}\exp\left(\frac{8\lambda^{2}(C_{0}\sigma)^{2}}{(|I_{j}|-\frac{4\lambda
C_{0}A}{3})}\right).$

Now, we let
$P_{\hat{N}}=3\left(\frac{[2(\kappa+\beta)+\tau+1]\log_{\nu}\hat{N}}{b}\right)^{\frac{1}{\gamma}}$
for any given $\beta\geq 0$, we obtain

$\displaystyle
E\left[\exp\left(\frac{\lambda\sum_{l=1}^{|I_{j}|}G_{lN}^{r}}{|I_{j}|}\right)\right]$
$\displaystyle\leq$ $\displaystyle
e\exp\left\\{\begin{matrix}\underbrace{e^{\frac{3}{2}}\frac{2^{\frac{3}{2}\tau
d+d_{2}(\tau+1)}}{H_{0}^{d_{2}(\tau+1)}}\left(3\left(\frac{2[\kappa+\beta]+\tau+1}{b}\right)^{\frac{1}{\gamma}}\right)^{\tau
d-d_{2}(\tau+1)}}\\\ :=C_{1}\end{matrix}\frac{(\log_{\nu}\hat{N})^{\frac{\tau
d-d_{2}(\tau+1)}{\gamma}}}{\hat{N}^{\kappa+\beta}}\right\\}\exp\left(\frac{8\lambda^{2}(C_{0}\sigma)^{2}}{(|I_{j}|-\frac{4\lambda
C_{0}A}{3})}\right).$

Let $\hat{N}\geq\hat{N}_{0}$, where

$\displaystyle\hat{N}_{0}:=\min\left\\{\hat{N}\in\mathbf{R}^{+},\
\frac{(\log_{\nu}\hat{N})^{\frac{\tau