Now we show that such consistent frequentist estimators for $\alpha$ and $\beta$ should not exist. By 42 and 51, the Gaussian measures induced by two Gaussian processes are either equivalent (absolutely continuous) or orthogonal to each other. For $\gp(m_0,\theta_0K_{\alpha_0,\nu})$ and $\gp(m_1,\theta_1K_{\alpha_1,\nu})$ with the same smoothness parameter $\nu$, the theory in Chapter 4 of 42 (Section 4.2 and Corollary 5) and Theorem 2 of 51 have shown that when $d=1,2,3$, their induced Gaussian measures are equivalent to each other if: (a) $\theta_0=\theta_1$; (b) Both mean functions $m_i$ lie in the reproducing kernel Hilbert space of $\theta_iK_{\alpha_i,\nu}$ for $i=0,1$. Let $L_2([0,1]^d)$ be the space of square integrable functions on $[0,1]^d$ with the norm $\|\cdot\|_2$). Then the Sobolev space of order $k$ for any $k>0$, denoted by $\Wcal_2^k([0,1]^d)$, is $\Wcal_2^k([0,1]^d) = \Big\{f\in L_2([0,1]^d): \|f\|^2_{\Wcal_2^k} = \sum_{\sj\in \NN^d: |\sj|\leq k} \left\|\sD^{\sj}f\right\|_2^2 <\infty\Big\}$. By Assumption <ref>, we have that $m(\cdot) = \beta^\T \ff (\cdot)$ belongs to $\Wcal_2^{\nu+d/2}([0,1]^d)$ for any $\beta\in \RR^p$, and hence belongs to the reproducing kernel Hilbert space of $\theta_iK_{\alpha_i,\nu}$ for $i=0,1$. By Corollary 10.48 of 49, the reproducing kernel Hilbert space of the isotropic Matérn covariance function $\theta_iK_{\alpha_i,\nu}$ for $i=0,1$ are norm equivalent to the Sobolev space of order $\nu+d/2$.

We choose $\theta_0=\theta_1$, $\alpha_0\neq \alpha_1$, $\beta_0\neq \beta_1$ and let $m_i(\cdot ) = \beta_i^\T \ff(\cdot)$ for $i=0,1$. Then both the conditions (a) and (b) above hold true, such that the two Gaussian measures $\gp(m_0,\theta_0 K_{\alpha_0,\nu})$ and $\gp(m_1,\theta_1K_{\alpha_1,\nu})$ are equivalent. Using similar argument to Corollary 1 of 51, if $\widetilde\alpha_n$ and $\widetilde\beta_n$ are consistent for $\alpha_0$ and $\beta_0$, then we can always find almost-surely convergence subsequences $\{\widetilde\alpha_{n_k}\}$ and $\{\widetilde\beta_{n_j}\}$, such that
$$\PP_{(\theta_0,\alpha_0,\tau_0,\beta_0)}\left(\lim_{k\to\infty}\widetilde\alpha_{n_k}= \alpha_0\right)=1, \text{ and } \PP_{(\theta_0,\alpha_0,\tau_0,\beta_0)}\left(\lim_{j\to\infty}\widetilde\beta_{n_j}= \beta_0\right)=1 .$$

But given the equivalence of the two Gaussian measures, this implies that
$$\PP_{(\theta_1,\alpha_1,\tau_1,\beta_1)}\left(\lim_{k\to\infty}\widetilde\alpha_{n_k}= \alpha_0\right)=1 \text{ and } \PP_{(\theta_1,\alpha_1,\tau_1,\beta_1)}\left(\lim_{j\to\infty}\widetilde\beta_{n_j}= \beta_0\right)=1.$$

However, under $\PP_{(\theta_1,\alpha_1,\tau_1,\beta_1)}$, the limits of $\{\widetilde\alpha_{n_k}\}$ and $\{\widetilde\beta_{n_j}\}$ should be $\alpha_1$ and $\beta_1$, respectively, which are different from $\alpha_0$ and $\beta_0$. Therefore, this is a contradiction, and such consistent estimators $\widetilde\alpha_n $ and $\widetilde\beta_n$ cannot exist. This further implies that the posterior $\Pi(\cdot \mid Y_n, F_n )$ is inconsistent for $\alpha$ and $\beta$.

§ PROOF OF PROPOSITION <REF>

We check (i) and (ii) in Assumption <ref> for the four covariance functions in Examples <ref>–<ref>, respectively.

Checking (i) of Assumption <ref>:

Since for all $\alpha$ that satisfies $|\alpha/\alpha_0 -1|\leq r_0$,

\begin{align} \label{eq:assump1.deriv}
&\quad~ \sup_{w\in \RR^d} \left|f_{\theta_0,\alpha,\nu}(w)/f_{\theta_0,\alpha_0,\nu}(w)-1\right| \nonumber \\
& \leq  \sup_{w\in \RR^d} \frac{\left|f_{\theta_0,\alpha,\nu}(w) - f_{\theta_0,\alpha_0,\nu}(w)\right|}{ f_{\theta_0,\alpha_0,\nu}(w)}  \leq  \left\{\sup_{|\alpha/\alpha_0 -1|\leq r_0} \sup_{w\in \RR^d} \left\|\frac{\frac{\partial f_{\theta_0,\alpha,\nu}(w)}{\partial \alpha}}{f_{\theta_0,\alpha_0,\nu}(w)} \right\|\right\} \cdot \alpha_0\left|\frac{\alpha}{\alpha_0}-1\right|,
\end{align}

we can see that it suffices to show that the double supremum in the first term is a continuous function on the set $\{\alpha:|\alpha/\alpha_0 -1|\leq r_0\}$ for some $r_0\in (0,1/2)$ and is upper bounded by a constant that may depend on $\nu,d,\theta_0,\alpha_0$. We can simply set $r_0=1/4$ and $\kappa=1$.

(i) The isotropic Matérn covariance function has the following spectral density (42) for any $(\theta,\alpha,\nu)\in \RR_+^{3}$:

\begin{align} \label{eq:Matern.spectral}
& f_{\theta,\alpha,\nu}(w) = \frac{\Gamma(\nu+d/2)}{\Gamma(\nu)\pi^{d/2}} \frac{\theta}{(\alpha^2+\|w\|^2)^{\nu+d/2}}.
\end{align}

\begin{align}
& \left|\frac{\frac{\partial f_{\theta_0,\alpha,\nu}(w)}{\partial \alpha}}{f_{\theta_0,\alpha_0,\nu}(w)}\right| = \left| \frac{\frac{\Gamma(\nu+d/2)}{\Gamma(\nu)\pi^{d/2}} \frac{- \theta\cdot 2(\nu+d/2) \alpha}{(\alpha^2+\|w\|^2)^{\nu+d/2+1}}} {\frac{\Gamma(\nu+d/2)}{\Gamma(\nu)\pi^{d/2}} \frac{\theta}{(\alpha^2+\|w\|^2)^{\nu+d/2}}} \right| = \frac{2(\nu+d/2)\alpha}{\alpha^2+\|w\|^2}. \nonumber
\end{align}

This implies that

\begin{align}
& \sup_{|\alpha/\alpha_0 -1|\leq r_0} \sup_{w\in \RR^d} \left\|\frac{\frac{\partial f_{\theta_0,\alpha,\nu}(w)}{\partial \alpha}}{f_{\theta_0,\alpha_0,\nu}(w)} \right\| \leq \sup_{|\alpha/\alpha_0 -1|\leq 1/4} \frac{2(\nu+d/2)}{\alpha} \leq \frac{4(2\nu+d)}{3\alpha_0}, \nonumber
\end{align}

which is a constant that depends only on $\alpha_0,\nu,d$. Thus the double supremum in (<ref>) is upper bounded by a constant for the isotropic Matérn covariance function in Example <ref>.

(ii) The tapered isotropic Matérn covariance function (22) has the following spectral density for any $(\theta,\alpha,\nu)\in \RR_+^{3}$:

\begin{align} \label{eq:taper.spectral}
& f_{\theta,\alpha,\nu}(w) = f_{\tap}(w)\cdot  \frac{\Gamma(\nu+d/2)}{\Gamma(\nu)\pi^{d/2}} \frac{\theta}{(\alpha^2+\|w\|^2)^{\nu+d/2}},
\end{align}

where $f_{\tap}$ satisfies $0< f_{\tap}(w)\leq C_{\tap} (1+\|w\|^2)^{-(\nu+d/2+\eta_{\tap})}$ for some constants $C_{\tap}>0,\eta_{\tap}>0$ and all $w\in \RR^d$. The function $f_{\tap}(w)$ only depends on $\nu,d$ but not $\alpha$ and $\theta$. Therefore,

\begin{align}
& \left|\frac{\frac{\partial f_{\theta_0,\alpha,\nu}(w)}{\partial \alpha}}{f_{\theta_0,\alpha_0,\nu}(w)}\right| = \frac{2(\nu+d/2)\alpha}{\alpha^2+\|w\|^2}, \nonumber
\end{align}

and hence
\begin{align}
& \sup_{|\alpha/\alpha_0 -1|\leq r_0} \sup_{w\in \RR^d} \left\|\frac{\frac{\partial f_{\theta_0,\alpha,\nu}(w)}{\partial \alpha}}{f_{\theta_0,\alpha_0,\nu}(w)} \right\| \leq \sup_{|\alpha/\alpha_0 -1|\leq 1/4} \frac{2(\nu+d/2)}{\alpha} \leq \frac{4(2\nu+d)}{3\alpha_0}, \nonumber
\end{align}

which is a constant that depends only on $\alpha_0,\nu,d$. Thus the double supremum in (<ref>) is upper bounded by a constant for the tapered isotropic Matérn covariance function in Example <ref>.

(iii) The isotropic generalized Wendland covariance function has the following spectral density for any $\mu > \nu + d$, $\nu \geq 1/2$, and $(\theta,\alpha)\in \RR_+^2$ (Theorem 1 of 7):

\begin{align} \label{eq:GW.spectral}
f_{\theta,\alpha,\nu}(w)& = \theta\cdot \frac{\Gamma(\mu+1)\Gamma(2\nu+d-1)}{2^{d+\nu-3/2}\pi^{d/2}\Gamma(\nu+(d-1)/2)\Gamma(\mu+2\nu+d) } \cdot \frac{\Gamma(\nu-1/2)}{2^{3/2-\nu}B(2\nu-1,\mu+1)} \nonumber \\
&\quad \cdot \alpha^{-(2\nu+d)} {}_1F_2\left(\nu+\frac{d}{2};\nu+\frac{d+\mu}{2},\nu+\frac{d+\mu+1}{2};-\frac{\|w\|^2}{4\alpha^2}\right) ,
\end{align}

where $\Gamma(\nu-1/2)/\big\{2^{3/2-\nu}B(2\nu-1,\mu+1)\big\}$ is defined to be 1 if $\nu=1/2$, and for any $a,b,c\in \RR_+$,

$${}_1F_2(a;b,c;z) = \sum_{k=0}^{\infty} \frac{(a)_k z^k}{(b)_k(c)_kk!}, \quad \text{for } z\in \RR, $$

with $(x)_k=\Gamma(x+k)/\Gamma(x)$ for any $x>0$ being the Pochhammer symbol. The derivative of ${}_1F_2(a;b,c;z)$ is

$$\frac{\ud}{\ud z}{}_1F_2(a;b,c;z) = \frac{a}{bc}{}_1F_2(a+1;b+1,c+1;z).$$

Therefore, we can calculate that

\begin{align*}
\frac{\frac{\partial f_{\theta_0,\alpha,\nu}(w)}{\partial \alpha}}{f_{\theta_0,\alpha_0,\nu}(w)} &=
\frac{ -(2\nu+d)\alpha^{-(2\nu+d+1)} {}_1F_2\left(\nu+\frac{d}{2};\nu+\frac{d+\mu}{2},\nu+\frac{d+\mu+1}{2};-\frac{\|w\|^2}{4\alpha^2}\right)} { \alpha^{-(2\nu+d)} {}_1F_2\left(\nu+\frac{d}{2};\nu+\frac{d+\mu}{2},\nu+\frac{d+\mu+1}{2};-\frac{\|w\|^2}{4\alpha^2}\right)} \\
&\quad+ \frac{\|w\|^2\alpha^{-(2\nu+d+3)} \frac{2\nu+d}{(2\nu+d+\mu)(2\nu+d+\mu+1)} {}_1F_2\left(\nu+\frac{d}{2}+1;\nu+\frac{d+\mu}{2}+1,\nu+\frac{d+\mu+3}{2};-\frac{\|w\|^2}{4\alpha^2}\right)}{\alpha^{-(2\nu+d)} {}_1F_2\left(\nu+\frac{d}{2};\nu+\frac{d+\mu}{2},\nu+\frac{d+\mu+1}{2};-\frac{\|w\|^2}{4\alpha^2}\right)} .
\end{align*}

Part (iii) in Theorems 1 and 2 of 7 have shown that as $\|w\|\to\infty$, $f_{\theta,\alpha,\nu}(w) \asymp \|w\|^{-(2\nu+d)}$ for a given set of $\theta,\alpha,\nu,\mu$. Furthermore, the function ${}_1F_2(a;b,c;z)$ is strictly positive and takes finite values for all $z\in [0,+\infty)$. Therefore, there exist $0< c_1(\mu,\nu,d,\alpha) < C_1(\mu,\nu,d,\alpha) <\infty$ and $0< c_2(\mu,\nu,d,\alpha) < C_2(\mu,\nu,d,\alpha) <\infty$ which are all continuous functions in $\alpha$, such that

\begin{align*}
& c_1(\mu,\nu,d,\alpha)\|w\|^{-(2\nu+d+2)} \leq {}_1F_2\left(\nu+\frac{d}{2}+1;\nu+\frac{d+\mu}{2}+1,\nu+\frac{d+\mu+3}{2};-\frac{\|w\|^2}{4\alpha^2}\right)\\
& \leq C_1(\mu,\nu,d,\alpha)\|w\|^{-(2\nu+d+2)}, \\
& c_2(\mu,\nu,d,\alpha)\|w\|^{-(2\nu+d)} \leq {}_1F_2\left(\nu+\frac{d}{2};\nu+\frac{d+\mu}{2},\nu+\frac{d+\mu+1}{2};-\frac{\|w\|^2}{4\alpha^2}\right)\\
& \leq C_2(\mu,\nu,d,\alpha)\|w\|^{-(2\nu+d)}.
\end{align*}

Therefore, for all $\alpha$ that satisfies $|\alpha/\alpha_0-1|\leq 1/4$, for all $w\in \RR^d$,

\begin{align*}
\left| \frac{\frac{\partial f_{\theta_0,\alpha,\nu}(w)}{\partial \alpha}}{f_{\theta_0,\alpha_0,\nu}(w)}  \right|
&\leq \frac{4(2\nu+d)}{3\alpha_0} + \frac{64\sup_{|\alpha/\alpha_0-1|\leq 1/4}C_1(\mu,\nu,d,\alpha)}{27\alpha_0^3 \inf_{|\alpha/\alpha_0-1|\leq 1/4}c_2(\mu,\nu,d,\alpha)},
\end{align*}

which is a finite number that depends on $\mu,\nu,d,\alpha_0$. Thus the double supremum in (<ref>) is upper bounded by a constant for the isotropic generalized Wendland covariance function in Example <ref>.

(iv) The isotropic confluent hypergeometric covariance function (31) has the following spectral density for any $(\theta,\alpha,\nu,\mu)\in \RR_+^4$:

\begin{align} \label{eq:CH.spectral}
f_{\theta,\alpha,\nu,\mu}(w)& = \frac{\theta\cdot 2^{\nu-\mu}\nu^{\nu}}{\Gamma(\nu+\mu)\pi^{d/2} \alpha^{2(\nu+\mu)}} \int_0^{\infty} \left(2\nu /t + \|w\|^2 \right)^{-(\nu+d/2)} t^{-(\nu+\mu+1)}\exp\left\{-1/(2\alpha^2 t)\right\} \ud t.
\end{align}

We highlight the dependence on $\mu$ in the spectral density for convenience of expressions below, though we have assumed that $\mu$ is known. The proof of Proposition 1 in 31 has shown that

\begin{align} \label{eq:MaBha.prop1}
\lim_{\|w\|\to\infty} \frac{f_{\theta,\alpha,\nu,\mu}(w)}{\frac{2^{2\nu}\nu^{\nu}}{\pi^{d/2}} \theta \|w\|^{-(2\nu+d)} \left(\frac{2\nu\alpha^2\|w\|^2}{2\nu\alpha^2\|w\|^2+1}\right)^{\nu+d/2}} = 1.
\end{align}

For the derivative with respect to $\alpha$, we can calculate from (<ref>) that

\begin{align} \label{eq:CH.deriv1}
\left| \frac{\frac{\partial f_{\theta_0,\alpha,\nu,\mu}(w)}{\partial \alpha}}{f_{\theta_0,\alpha_0,\nu,\mu}(w)}  \right|
& \leq \frac{2(\nu+\mu)}{\alpha} + \frac{2(\nu+\mu+1)}{\alpha}\cdot \frac{f_{\theta_0,\alpha,\nu,\mu+1}(w)}{f_{\theta_0,\alpha,\nu,\mu}(w)} .
\end{align}

From (<ref>), we clearly have that $0 < f_{\theta,\alpha,\nu,\mu}(w) <\infty $ for all $w\in \RR^d$ and $f_{\theta,\alpha,\nu,\mu}(w)$ is always a continuous function in $\|w\|$ and $\alpha$. Therefore, from (<ref>), there exists a function $0<C_1(\alpha,\theta_0,\nu,\mu)<\infty$ continuous in $\alpha$, such that $\sup_{w\in\RR^d }f_{\theta_0,\alpha,\nu,\mu+1}(w)/f_{\theta_0,\alpha,\nu,\mu}(w) \leq C_1(\alpha,\theta_0,\nu,\mu)$. Hence, we obtain from (<ref>) that for all $\alpha$ that satisfies $|\alpha/\alpha_0-1|\leq 1/4$, for all $w\in \RR^d$,

\begin{align}
\left| \frac{\frac{\partial f_{\theta_0,\alpha,\nu,\mu}(w)}{\partial \alpha}}{f_{\theta_0,\alpha_0,\nu,\mu}(w)}  \right|
& \leq \frac{8(\nu+\mu)}{3\alpha_0} + \frac{8(\nu+\mu+1)}{3\alpha_0}\cdot \sup_{|\alpha/\alpha_0-1|\leq 1/4}C_1(\alpha,\theta_0,\nu,\mu) , \nonumber
\end{align}

which is a finite number that depends on $\mu,\nu,d,\alpha_0$. Thus the double supremum in (<ref>) is upper bounded by a constant for the isotropic confluent hypergeometric covariance function in Example <ref>.

Checking (ii) of Assumption <ref>:

From (<ref>), the spectral density of isotropic Matérn covariance function is clearly a decreasing function in $\alpha$. From (<ref>), the spectral density of the tapered isotropic Matérn is also decreasing in $\alpha$ since $f_{\tap}$ does not depend on $\alpha$. The proof of Lemma 1 in 7 has shown that under our condition $\mu>\nu + d$, the spectral density of generalized Wendland in (<ref>) is a decreasing function in $\alpha$. The proof of Lemma 4 in 31 has shown that for fixed $\theta,\nu,\mu$, the spectral density of confluent hypergeometric covariance function in (<ref>) is a decreasing function in $\alpha$. This completes the proof of Proposition <ref>.

§ PROOF OF THEOREM <REF>

We first recall from Theorem <ref> that for the higher-order quadratic variation estimators $\widehat\theta_n$ and $\widehat\tau_n$, they satisfy Assumption <ref> with $b_1$ and $b_2$ defined by

\begin{align} \label{eq:b12.recall}
b_1& = \min\Bigg\{\frac{1}{2}  - \frac{2(1-\gamma)\nu}{d}- \rho_{22} , ~ \frac{1-\gamma}{2} - \varsigma , ~ \frac{1}{4} + \frac{(1-\gamma)(\ell_{\star}-2\nu)}{d} - \frac{\rho_{1} + \rho_{22}}{2}, \nonumber \\
& \qquad \qquad \frac{1-\gamma}{4} + \frac{(1-\gamma)(\ell_{\star}-\nu)}{d} - \frac{\rho_{1}}{2} - \varsigma \Bigg\}  ,  \nonumber \\
b_2& =  \min\Bigg\{\frac{1}{2} , ~ \frac{(1-\gamma)(4\nu+d)}{2d}  - \rho_{21} - \varsigma , ~ \frac{1}{4} +\frac{ (1-\gamma)\ell_{\star}}{d} - \frac{\rho_{1} + \rho_{21}}{2}, ~ \nonumber \\
& \qquad \qquad \frac{(1-\gamma)(4\nu+d +4\ell_{\star})}{4d} - \frac{\rho_{1} + 2\rho_{21}}{2} - \varsigma  \Bigg\} .
\end{align}

And recall that $\gamma$ needs to satisfy $\max\big\{1-d/(4\nu),0\big\} < \gamma < 1$.

In the expression of $b_1$ above, we first consider the third and fourth terms inside the minimum. Given that $\ell_{\star} = \lceil \nu + d/2 \rceil \geq \nu+d/2$, the fourth term satisfies

\begin{align*}
& \frac{1-\gamma}{4} + \frac{(1-\gamma)(\ell_{\star}-\nu)}{d} - \frac{\rho_{1}}{2} - \varsigma \geq \frac{1-\gamma}{4} + \frac{1-\gamma}{2} - \frac{\rho_{1}}{2} - \varsigma.
\end{align*}

Therefore, when $\rho_1$ and $\varsigma$ are sufficiently small, the fourth term is lower bounded by the second term inside the minimum in the expression of $b_1$.

For the third term inside the minimum in the expression of $b_1$, we have

\begin{align*}
& \frac{1}{4} + \frac{(1-\gamma)(\ell_{\star}-2\nu)}{d} - \frac{\rho_{1} + \rho_{22}}{2} \geq \frac{1}{4} + \frac{(1-\gamma)(\nu+d/2-2\nu)}{d} - \frac{\rho_{1} + \rho_{22}}{2} \\
&= \frac{1}{2} \left[\frac{1}{2}  - \frac{2(1-\gamma)\nu}{d} \right] + \frac{1-\gamma}{2} - \frac{\rho_{1} + \rho_{22}}{2} .
\end{align*}

In other words, when $\rho_1,\rho_{22},\varsigma$ are sufficiently small, the third term is approximately a linear combination of the first and second terms inside the minimum in the expression of $b_1$. As a result, it is sufficient to balance the first and second terms inside the minimum in the expression of $b_1$ in (<ref>). We set them equal and obtain that $\gamma = 4\nu/(4\nu+d)$. Then we have from (<ref>) that when $\gamma = 4\nu/(4\nu+d)$,

\begin{align} \label{eq:b1.lower}
b_1 & \geq \min\Bigg\{\frac{1}{2}  - \frac{2(1-\gamma)\nu}{d} - \rho_{22} , ~ \frac{1-\gamma}{2}- \varsigma  , ~ \frac{1}{2} \left[\frac{1}{2}  - \frac{2(1-\gamma)\nu}{d} \right] + \frac{1-\gamma}{2} - \frac{\rho_{1} + \rho_{22}}{2}, \nonumber \\
& \qquad \qquad \frac{3(1-\gamma)}{4} - \frac{\rho_{1}}{2} - \varsigma   \Bigg\}   \nonumber  \\
& = \min\Bigg\{\frac{d}{2(4\nu+d)}- \rho_{22} , ~~ \frac{d}{2(4\nu+d)} - \varsigma , ~~ \frac{3d}{4(4\nu+d)} - \frac{\rho_{1} + \rho_{22}}{2}, \nonumber \\
& \qquad \qquad \frac{3d}{4(4\nu+d)} - \frac{\rho_{1}}{2} - \varsigma  \Bigg\}  , \nonumber  \\
&\geq \frac{1}{2(4\nu/d+1)} - \varrho,
\end{align}

where $\varrho  =  \max(\rho_{22}, \rho_1/2)$ since $\varsigma>0$ can be arbitrarily small. As such, the condition in (<ref>) becomes

\begin{align}
& 0< \rho_1 < \frac{d}{4\nu+d},  \quad 0<\rho_{21} < \frac{2\nu}{4\nu+d}, \quad 0<\rho_{22} < \frac{d}{2(4\nu+d)}, \quad \rho_{31}>0, \quad 0<\rho_{32} < \frac{1}{4\nu+d}, \nonumber
\end{align}

which are satisfied by (<ref>) of Theorem <ref>.

In the expression of $b_2$ above, since $0<1-\gamma < \min\{1,d/(4\nu)\} $, we can choose $1-\gamma$ sufficiently close to its upper bound, say $1-\gamma = \min\{1,d/(4\nu)\}-\eta$ for some small $\eta \in (0,\min\{1,d/(4\nu)\})$, such that

\begin{align}
&\quad~\frac{(1-\gamma)(4\nu+d)}{2d}  - \rho_{21}  - \varsigma \nonumber \\
&= \frac{\min\{1,d/(4\nu)\}(4\nu+d)}{2d} - \frac{\eta(4\nu+d)}{2d} - \rho_{21}  - \varsigma  \nonumber  \\
&= \frac{1}{2} + \min\left\{\frac{2\nu}{d}, \frac{d}{8\nu} \right\} -  \frac{\eta(4\nu+d)}{2d}  - \rho_{21} - \varsigma , \label{eq:b2.rho1} \\
&\quad~ \frac{1}{4} +\frac{ (1-\gamma)\ell_{\star}}{d} - \frac{\rho_{1} + \rho_{21}}{2}  \nonumber \\
&\geq \frac{1}{4} +\frac{(\nu+d/2) \min\{1,d/(4\nu)\}}{d} - \frac{\eta (\nu+d/2)}{d} - \frac{\rho_{1} + \rho_{21}}{2}  \nonumber \\
&\geq \frac{1}{2} + \min\left\{\frac{1}{4}+\frac{\nu}{d}, \frac{d}{8\nu}\right\}  - \frac{\eta (\nu+d/2)}{d} - \frac{\rho_{1} + \rho_{21}}{2} , \label{eq:b2.rho2} \\
&\quad~ \frac{(1-\gamma)(4\nu+d +4\ell_{\star})}{4d} - \frac{\rho_{1} + 2\rho_{21} }{2}  - \varsigma  \nonumber \\
&\geq \frac{(8\nu+3d)\min\{1,d/(4\nu)\}}{4d} - \frac{\eta(8\nu+3d) }{4d} - \frac{\rho_{1} + 2\rho_{21} }{2}  - \varsigma  \nonumber \\
&= \frac{1}{2} + \min\left\{\frac{1}{4}+\frac{2\nu}{d}, \frac{3d}{16\nu}\right\}  - \frac{\eta(8\nu+3d) }{4d}  - \frac{\rho_{1} + 2\rho_{21} }{2} - \varsigma . \label{eq:b2.rho3}
\end{align}

In order to make the right-hand sides of (<ref>), (<ref>), and (<ref>) strictly larger than $1/2$, it is sufficient to have

\begin{align} \label{eq:rho.b2.1}
&  \rho_{21} < \min\left\{\frac{2\nu}{d}, \frac{d}{8\nu} \right\} , ~~ \frac{\rho_{1} + \rho_{21}}{2} < \min\left\{\frac{1}{4}+\frac{\nu}{d}, \frac{d}{8\nu}\right\} , \nonumber \\
& \frac{\rho_{1} + 2\rho_{21}}{2} < \min\left\{\frac{1}{4}+\frac{2\nu}{d}, \frac{3d}{16\nu}\right\},
\end{align}

since both $\eta>0$ and $\varsigma>0$ can be chosen as arbitrarily small. We notice that the third relations in (<ref>) is implied by adding up the first two relations in (<ref>), which are included in (<ref>) of Theorem <ref>. Therefore, with such choice of $\gamma$, we can set $b_2=1/2$ in (<ref>). Finally, the conclusion of Theorem <ref> follows by combining the lower bound of $b_1$ in (<ref>) and $b_2=1/2$ together with the posterior convergence in Theorem <ref>.

§ PROOF OF PROPOSITION <REF>

First, we verify Assumption <ref>. Clearly the joint prior density $\pi(\theta,\alpha,\tau,\beta)$ is continuous everywhere and satisfies $\pi(\theta_0,\alpha_0,\tau_0,\beta_0)>0$. Furthermore, since $\tau\sim \textup{IG}(a_2,b_2)$, we have that for any $n\in \ZZ_+$,

\begin{align*}
\int_0^{\infty} \tau^{-n/2}\pi(\tau) \ud \tau & = \frac{b_2^{a_2}}{\Gamma(a_2)} \int_0^{\infty} \tau^{-n/2} \cdot \tau^{-(a_2+1)} \exp(-b_2/\tau) \ud \tau = \frac{\Gamma(a_2+n/2)}{b_2^{n/2} \Gamma(a_2)} <\infty.
\end{align*}

Therefore Assumption <ref> is satisfied.

Next, we verify Assumption <ref>. For the isotropic Matérn covariance function, Assumption <ref> is satisfied with $\kappa=1$. We first decompose the set $\Ecal_{n}^c$ based on Equation <ref> into several sets:

\begin{align} \label{eq:Ecal.complement}
\Ecal_{n}^c & \subseteq \{\|\beta\|^2/\theta > n^{\rho_1}\} \cup \{ \tau/\theta < n^{-\rho_{21}}\} \cup \{\tau/\theta > n^{\rho_{22}} \}  \cup \{\alpha < n^{-\rho_{31}}\} \cup \{ \alpha > n^{\rho_{32}}\}.
\end{align}

We show each of the set on the right-hand side of (<ref>) satisfies Assumption <ref> with the prior specified in Proposition <ref>. Recall that for the isotropic Matérn in Example <ref>, we have $\kappa=1$ from the proof of Proposition <ref>.

First, given $\beta\sim \Ncal(0,a_0I_p)$, $\theta\sim \text{IG}(a_1,b_1)$, we have that $\|\beta\|^2/a_0 \sim \chi^2_{p}$, and hence for sufficiently large $n$,

\begin{align} \label{eq:Ecal.c.1}
\Pi(\|\beta\|^2/\theta > n^{\rho_1}) &\leq \int_0^{\infty} \frac{b_1^{a_1}}{\Gamma(a_1)}\theta^{-(a_1+1)} \ee^{-b_1/\theta} \int_{n^{\rho_1} \theta/a_0}^{\infty} \frac{1}{2^p\Gamma(p/2)} z^{p/2-1} \ee^{-z/2} \ud z \ud \theta \nonumber \\
&\leq \int_0^{n^{-\rho_1/2}} \frac{b_1^{a_1}}{\Gamma(a_1)}\theta^{-(a_1+1)} \ee^{-b_1/\theta} \int_0^{\infty} \frac{1}{2^p\Gamma(p/2)} z^{p/2-1} \ee^{-z/2} \ud z \ud \theta  \nonumber \\
&\quad + \int_{n^{-\rho_1/2}}^{\infty} \frac{b_1^{a_1}}{\Gamma(a_1)}\theta^{-(a_1+1)} \ee^{-b_1/\theta} \int_{n^{\rho_1/2}/a_0}^{\infty} \frac{1}{2^p\Gamma(p/2)} z^{p/2-1} \ee^{-z/2} \ud z \ud \theta  \nonumber \\
&\stackrel{(i)}{\leq} \int_{n^{\rho_1/2}}^{\infty}  \frac{b_1^{a_1}}{\Gamma(a_1)} t^{a_1-1} \ee^{-b_1t} \ud t  + \int_{n^{\rho_1/2}/a_0}^{\infty} \frac{1}{2^p\Gamma(p/2)} z^{p/2-1} \ee^{-z/2} \ud z  \nonumber \\
&\stackrel{(ii)}{<} \int_{n^{\rho_1/2}}^{\infty}  \frac{b_1^{a_1}}{\Gamma(a_1)} \ee^{-b_1t/2 } \ud t  + \int_{n^{\rho_1/2}/a_0}^{\infty} \frac{1}{2^p\Gamma(p/2)}  \ee^{-z/4} \ud z  \nonumber \\
&= \frac{2b_1^{a_1-1}}{\Gamma(a_1)} \exp\big(-b_1n^{\rho_1/2}/2\big) + \frac{1}{2^{p-2}\Gamma(p/2)} \exp\big\{-n^{\rho_1/2}/(4a_0)\big\}  \nonumber \\
&\prec n^{-(3p+6)},
\end{align}

where for (i) we use a change of variable $t=1/\theta$, and (ii) follows when $n$ is sufficiently large such that $z^{p/2-1}<\ee^{z/4}$ for all $z\geq n^{\rho_1/2}/a_0$.

Second, given $\tau\sim \text{IG}(a_2,b_2)$, $\theta\sim \text{IG}(a_1,b_1)$, we have that

\begin{align} \label{eq:Ecal.c.2}
\Pi(\tau/\theta < n^{-\rho_{21}}) &\leq \int_0^{\infty} \frac{b_1^{a_1}}{\Gamma(a_1)}\theta^{-(a_1+1)} \ee^{-b_1/\theta} \int_0^{n^{-\rho_{21}} \theta} \frac{b_2^{a_2}}{\Gamma(a_2)} \tau^{-(a_2+1)} \ee^{-b_2/\tau} \ud \tau \ud \theta \nonumber \\
&\leq \int_0^{n^{\rho_{21}/2}} \frac{b_1^{a_1}}{\Gamma(a_1)}\theta^{-(a_1+1)} \ee^{-b_1/\theta} \int_0^{n^{-\rho_{21}/2}} \frac{b_2^{a_2}}{\Gamma(a_2)} \tau^{-(a_2+1)} \ee^{-b_2/\tau} \ud \tau \ud \theta   \nonumber \\
&\quad + \int_{n^{\rho_{21}/2}}^{\infty} \frac{b_1^{a_1}}{\Gamma(a_1)}\theta^{-(a_1+1)} \ee^{-b_1/\theta} \int_0^{n^{-\rho_{21}} \theta} \frac{b_2^{a_2}}{\Gamma(a_2)} \tau^{-(a_2+1)} \ee^{-b_2/\tau} \ud \tau \ud \theta   \nonumber \\
&\leq \int_0^{n^{-\rho_{21}/2}} \frac{b_2^{a_2}}{\Gamma(a_2)} \tau^{-(a_2+1)} \ee^{-b_2/\tau} \ud \tau  + \int_{n^{\rho_{21}/2}}^{\infty} \frac{b_1^{a_1}}{\Gamma(a_1)}\theta^{-(a_1+1)} \ee^{-b_1/\theta} \ud \theta   \nonumber \\
&\stackrel{(i)}{=}  \int_{n^{\rho_{21}/2}}^{\infty} \frac{b_2^{a_2}}{\Gamma(a_2)} t^{a_2-1} \ee^{-b_2 t} \ud t  + \int_{n^{\rho_{21}/2}}^{\infty} \frac{b_1^{a_1}}{\Gamma(a_1)}\theta^{-(a_1+1)} \ee^{-b_1/\theta} \ud \theta   \nonumber \\
&\leq \int_{n^{\rho_{21}/2}}^{\infty} \frac{b_2^{a_2}}{\Gamma(a_2)} \ee^{-b_2 t/2} \ud t  + \int_{n^{\rho_{21}/2}}^{\infty} \frac{b_1^{a_1}}{\Gamma(a_1)} \theta^{-(a_1+1)}  \ud \theta   \nonumber \\
&= \frac{2b_2^{a_2-1}}{\Gamma(a_2)} \exp\big(-b_2 n^{\rho_{21}/2}/2 \big) + \frac{b_1^{a_1}}{a_1\Gamma(a_1)}n^{ -a_1\rho_{21}/2}  \nonumber \\
&\prec n^{-(3p+6)},
\end{align}

if $a_1\rho_{21}/2> 3p+6$, or $a_1 > 2(3p+6)/\rho_{21}$, where in (i) we use the change of variable $t=1/\tau$.

Third, for the right tail of $\tau/\theta$, we have that

\begin{align} \label{eq:Ecal.c.3}
\Pi(\tau/\theta > n^{\rho_{22}}) &\leq \int_0^{\infty} \frac{b_1^{a_1}}{\Gamma(a_1)}\theta^{-(a_1+1)} \ee^{-b_1/\theta} \int_{n^{\rho_{22}}\theta}^{\infty} \frac{b_2^{a_2}}{\Gamma(a_2)} \tau^{-(a_2+1)} \ee^{-b_2/\tau} \ud \tau \ud \theta \nonumber \\
&\leq \int_0^{n^{-\rho_{22}/2}} \frac{b_1^{a_1}}{\Gamma(a_1)}\theta^{-(a_1+1)} \ee^{-b_1/\theta} \int_{0}^{\infty} \frac{b_2^{a_2}}{\Gamma(a_2)} \tau^{-(a_2+1)} \ee^{-b_2/\tau} \ud \tau \ud \theta   \nonumber \\
&\quad + \int_{n^{-\rho_{22}/2}}^{\infty} \frac{b_1^{a_1}}{\Gamma(a_1)}\theta^{-(a_1+1)} \ee^{-b_1/\theta} \int_{n^{\rho_{22}/2}}^{\infty} \frac{b_2^{a_2}}{\Gamma(a_2)} \tau^{-(a_2+1)} \ee^{-b_2/\tau} \ud \tau \ud \theta   \nonumber \\
&\leq \int_0^{n^{-\rho_{22}/2}} \frac{b_1^{a_1}}{\Gamma(a_1)}\theta^{-(a_1+1)} \ee^{-b_1/\theta} \ud \theta  + \int_{n^{\rho_{22}/2}}^{\infty} \frac{b_2^{a_2}}{\Gamma(a_2)} \tau^{-(a_2+1)} \ee^{-b_2/\tau} \ud \tau   \nonumber \\
&\stackrel{(i)}{=}  \int_{n^{\rho_{22}/2}}^{\infty} \frac{b_1^{a_1}}{\Gamma(a_1)} t^{a_1-1} \ee^{-b_1 t} \ud t  + \int_{n^{\rho_{22}/2}}^{\infty} \frac{b_2^{a_2}}{\Gamma(a_2)}\tau^{-(a_2+1)} \ee^{-b_2/\tau} \ud \tau   \nonumber \\
&\leq \int_{n^{\rho_{22}/2}}^{\infty} \frac{b_1^{a_1}}{\Gamma(a_1)} \ee^{-b_1 t/2} \ud t  + \int_{n^{\rho_{22}/2}}^{\infty} \frac{b_2^{a_2}}{\Gamma(a_2)} \tau^{-(a_2+1)}  \ud \tau   \nonumber \\
&= \frac{2b_1^{a_1-1}}{\Gamma(a_1)} \exp\big(-b_1 n^{\rho_{22}/2}/2 \big) + \frac{b_2^{a_2}}{a_2\Gamma(a_2)}n^{ -a_2\rho_{22}/2}  \nonumber \\
&\prec n^{-(3p+6)},
\end{align}

if $a_2\rho_{22}/2 > 3p+6$, or $a_2 > 2(3p+6)/\rho_{22}$, where in (i) we use the change of variable $t=1/\theta$.

Fourth, the inverse Gaussian prior on $\alpha$ has the density for $\alpha\in \RR_+$,

\[
\pi(\alpha) = \left(\frac{\lambda}{2\pi\alpha^3}\right)^{1/2} \exp\left\{-\frac{\lambda(\alpha-\mu)^2}{2\mu^2\alpha} \right\}.
\]

For the left tail of $\alpha$, we have that

\begin{align} \label{eq:Ecal.c.4}
\Pi(\alpha < n^{-\rho_{31}}) &= \int_0^{n^{-\rho_{31}}} \left(\frac{\lambda}{2\pi\alpha^3}\right)^{1/2} \exp\left(-\frac{\lambda\alpha}{2\mu^2} + \frac{\lambda}{\mu} - \frac{\lambda}{2\alpha} \right) \ud \alpha\nonumber  \\
&\leq \{\lambda/(2\pi)\}^{1/2}\ee^{\lambda/\mu} \int_0^{n^{-\rho_{31}}}  \alpha^{-3/2} \exp \big\{-\lambda/(2\alpha) \big\} \ud \alpha   \nonumber \\
&\stackrel{(i)}{=}  \{\lambda/(2\pi)\}^{1/2}\ee^{\lambda/\mu} \int_{n^{\rho_{31}}}^{\infty}  t^{-1/2} \exp \big\{-\lambda t/2 \big\} \ud t   \nonumber \\
&\leq \{2/(\lambda\pi)\}^{1/2}\ee^{\lambda/\mu} \exp \big(-\lambda n^{\rho_{31}}/2\big) \nonumber \\
&\prec n^{-(3p+6)},
\end{align}

where in (i) we use the change of variable $t=1/\alpha$.

Similarly, for the right tail of $\alpha$, we have that

\begin{align} \label{eq:Ecal.c.5}
\Pi(\alpha > n^{\rho_{32}}) &= \int_{n^{\rho_{32}}}^{\infty} \left(\frac{\lambda}{2\pi\alpha^3}\right)^{1/2} \exp\left(-\frac{\lambda\alpha}{2\mu^2} + \frac{\lambda}{\mu} - \frac{\lambda}{2\alpha} \right) \ud \alpha   \nonumber  \\
&\leq \{\lambda/(2\pi)\}^{1/2} \ee^{\lambda/\mu} \int_{n^{\rho_{32}}}^{\infty} \exp \big\{-\lambda\alpha/(2\mu^2) \big\} \ud \alpha   \nonumber \\
&\leq \{2/(\lambda\pi)\}^{1/2} \mu^2 \ee^{\lambda/\mu} \exp \big\{ -\lambda n^{\rho_{32}}/(2\mu^2) \big\} \nonumber \\
&\prec n^{-(3p+6)}.
\end{align}

Finally, we combine (<ref>), (<ref>), (<ref>), (<ref>) and (<ref>) with (<ref>) to conclude that $\Pi(\Ecal_{n}^c)\leq n^{-(3p+6)}$ for all sufficiently large $n$. Hence, Assumption <ref> is satisfied.

§ PROOF OF THEOREM <REF>

The proof of Theorem <ref> is long and we proceed in several steps. We first present the Taylor series expansion for Matérn covariance function. The Matérn covariance function $\theta K_{\alpha,\nu}$ defined in (<ref>) can be expressed as the sum of an infinite series, whose formula depends whether or not $\nu$ is an integer. The following expansion of $\theta K_{\alpha,\nu}(\bfs-\bft)$ for $\bfs,\bft \in [0,1]^d$ can be found on page 2772 of 28.

\begin{align}\label{eq:K expansion}
& \theta K_{\alpha,\nu}(\bfs-\bft) = \textstyle \sum_{j=0}^{\infty}\big\{ \zeta_j\|\bfs-\bft\|^{2j} + \zeta^*_{\nu+j}G_{\nu+j}(\|\bfs-\bft\|) \big\},
\end{align}

where $\ZZ_+$ is the set of all positive integers, and

\begin{align} \label{eq:G.nu}
G_s (t) & = \left\{ \begin{array}{ll} t^{2 s} \log(t), & \text{ if } s \in \ZZ_+,
\\
t^{2 s}, & \text{ if } s \notin \ZZ_+.
\end{array}
\right.
\end{align}

The terms of $\zeta_j$ and $\zeta^*_{\nu+j}$ for $j=0,1,2,\ldots$ are defined as follows:

If $\nu\not\in\mathbb{Z}$, then

\begin{align}\label{eq:zeta.xi1}
\zeta_j &= \theta\alpha^{2j-2\nu} \xi_j, \qquad \xi_j = 1/ \big\{2^{2j}j!\textstyle \prod_{i=1}^{j}(i-\nu) \big\}, \nonumber \\
\zeta^*_{\nu+j} &= \theta \alpha^{2j} \xi^*_{\nu+j}, \qquad  \xi^*_{\nu+j} = -\pi/ \left\{2^{2j+2\nu}j!\Gamma(j+1+\nu) \Gamma(\nu)\sin(\nu\pi)\right\} .
\end{align}

If $\nu\in\mathbb{Z}_+$, then

\begin{align}\label{eq:zeta.xi2}
&\zeta_j=  \theta \alpha^{2j-2\nu} (-1)^j \frac{(\nu-j-1)!}{2^{2j}(\nu-1)!j!}, \quad \text{if ~} j<\nu, \nonumber \\
&\zeta_j=  \theta \alpha^{2j-2\nu}\left\{\xi_{1,j}+\xi_{2,j}\log(\alpha)\right\}, ~{\rm if ~} j\ge \nu, ~ \text{ where } \nonumber \\
&\xi_{1,j}  = \frac{(-1)^{\nu}\left[\psi(j-\nu+1)+\psi(j+1)+2\log 2\right]}{2^{2j}(j-\nu)!j!(\nu-1)!}, \quad \xi_{2,j}  = \frac{(-1)^{\nu+1}}{2^{2j-1}(j-\nu)!j!(\nu-1)!}, \nonumber \\
&\zeta^*_{\nu+j} = \theta \alpha^{2j} \xi^*_{\nu+j}, \quad \xi^*_{\nu+j} = \frac{(-1)^{\nu+1}}{2^{2\nu+2j-1}(\nu-1)!j!(\nu+j)!} ,
\end{align}

where $\psi(\cdot)$ is the digamma function. For both cases, the coefficients $\xi^*_{\nu+j},\xi_{1,j},\xi_{2,j}$ for all $j=0,1,\ldots$ are all upper bounded by constant.

We then cite an important lemma about the series of $m$-dependent constants $c_{\bfi,d,\ell}^{(k_1,\ldots,k_d)}$ and $c_{d,\ell}^{(k_1,\ldots,k_d)}$ as defined in Lemma <ref> of the main text. As $n=m^d\to\infty$ (and so $m\to\infty$), the $m$-dependent constants $c_{\bfi,d,\ell}^{(k_1,\ldots,k_d)}$ are uniformly close to a deterministic sequence of $m$-independent constants $c_{d,\ell}^{(k_1,\ldots,k_d)}$ which satisfy certain relations, as shown by Lemma 2 and Corollary 2 of 30.

(Lemma 2 and Corollary 2 of 30) Let $d,\ell\in \ZZ_+$. Let $\bfi=(i_1,\ldots,i_d)^\T $ where $1\leq i_1,\ldots,i_d\leq m - \ell \omega_m$. Then there exists a deterministic sequence of $m$-independent constants $c_{d,\ell}^{(k_1,\ldots,k_d)}$, such that $c_{\bfi,d,\ell}^{(k_1,\ldots,k_d)}$ in Lemma <ref> of the main text satisfies

\begin{align*}
& c_{\bfi,d,\ell}^{(k_1,\ldots,k_d)} = c_{d,\ell}^{(k_1,\ldots,k_d)} + O(\omega_m^{-1}), \quad \text{for all } 0\leq k_1,\ldots,k_d\leq \ell,
\end{align*}

as $m\to\infty$, where the term $O(\omega_m^{-1})$ is uniform over all $1\leq i_1,\ldots,i_d\leq m- \ell\omega_m$ and any $\delta_{\bfi;k}\in [0,1)$. Furthermore,

\begin{align*}
& \sum_{0\leq k_1,\ldots,k_d\leq \ell} c_{d,\ell}^{(k_1,\ldots,k_d)} k_d^{\ell} = \ell!, \\
\text{and } & \sum_{0\leq k_1,\ldots,k_d\leq \ell} c_{d,\ell}^{(k_1,\ldots,k_d)} \cdot \prod_{j=1}^d k_j^{l_j} = 0,
\end{align*}

for all integers $l_1,\ldots,l_d$ satisfying $0\leq l_1,\ldots,l_{d-1}\leq \ell$, $0\leq l_d\leq \ell-1$, and $0\leq l_1+\ldots + l_d\leq \ell$.

In the rest of the proof, we use $C$ to denote a generic positive constant that can take different values at different places, and $C$ only depends on $\nu,d,p,\theta_0,\alpha_0,\tau_0,\beta_0$. The order notation $O(\cdot)$ and $o(\cdot)$ is uniform over all $m$. For abbreviation, we write $\PP_0 = \PP_{(\theta_0,\alpha_0,\tau_0,\beta_0)}$, use $\EE_0$ to denote the expectation under $\PP_0$, and use $\EE$ to denote the expectation under $\PP_{(\theta,\alpha,\tau,\beta)}$ for a generic parameter vector $(\theta,\alpha,\tau,\beta)$. We will assume that Assumptions <ref>, <ref> and <ref> hold throughout this section.

Proof of Part (i):

By definition, we have that $(\theta_0,\alpha_0,\tau_0,\beta_0)\in \Ecal_{n}$ as $n\to\infty$. From (<ref>) and (<ref>) in Section <ref>, for any given $\epsilon_1>0$, we have that uniformly for all $(\theta,\alpha,\tau,\beta)\in \Ecal_{n}$, for all sufficiently large $n$,

\begin{align} \label{eq:V1.e4}
\left|\EE\left(V_{1,d,\ell_{\star}}\right)/(\theta g_{\ell_{\star},\nu}) - 1\right| < \epsilon_1/4,
\end{align}

with the expectation taken under $\PP_{(\theta,\alpha,\tau,\beta)}$.

Under the true measure $\PP_0$, for any $\epsilon_1\in (0,1)$,

\begin{align} \label{eq:V1.ineq1}
&\quad~ \PP_0 \left(\left|\frac{\widehat\theta_n}{\theta_0}-1\right|\geq \frac{\epsilon_1}{2} \right) \nonumber \\
&\leq \PP_0\left(\left|\frac{\EE_0\left(V_{1,d,\ell_{\star}}\right)}{\theta_0 g_{\ell_{\star},\nu}}\right| \cdot \left|\frac{V_{1,d,\ell_{\star}}}{\EE_0\left(V_{1,d,\ell_{\star}}\right)} -1 \right|  +  \left|\frac{\EE_0\left(V_{1,d,\ell_{\star}}\right)}{\theta_0 g_{\ell_{\star},\nu}} -1\right|\geq \frac{\epsilon_1}{2} \right) \nonumber \\
&\leq  \PP_0\left( \left|\frac{V_{1,d,\ell_{\star}}}{\EE_0\left(V_{1,d,\ell_{\star}}\right)} -1\right| > \frac{\epsilon_1}{5} \right).
\end{align}

Similarly, on the set $\Bcal^c(\epsilon_1,\epsilon_2)\cap \Ecal_{n}$, since $|\theta/\theta_0-1|\geq \epsilon_1$, for all sufficiently large $n$ such that (<ref>) holds for all $(\theta,\alpha,\tau,\beta)\in \Ecal_{n}$, we have that for any $\epsilon_1\in (0,1/2)$,

\begin{align} \label{eq:V1.ineq2}
&\quad~ \sup_{\Bcal^c(\epsilon_1,\epsilon_2)\cap \Ecal_{n}} \PP_{(\theta,\alpha,\tau,\beta)} \left(\left|\frac{\widehat\theta_n}{\theta_0}-1\right|\leq \frac{\epsilon_1}{2} \right) \nonumber \\
&\leq \sup_{\Bcal^c(\epsilon_1,\epsilon_2)\cap \Ecal_{n}} \PP_{(\theta,\alpha,\tau,\beta)} \left(\left|\frac{\theta}{\theta_0}-1\right| - \frac{\theta}{\theta_0}\left|\frac{\widehat\theta_n}{\theta}-1\right| \leq \frac{\epsilon_1}{2} \right) \nonumber \\
&\leq \sup_{\Bcal^c(\epsilon_1,\epsilon_2)\cap \Ecal_{n}} \PP_{(\theta,\alpha,\tau,\beta)} \left(\left|\frac{\widehat\theta_n}{\theta}-1\right| \geq \frac{\epsilon_1}{2(1+\epsilon_1)} \right) \nonumber \\
&\leq \sup_{\Bcal^c(\epsilon_1,\epsilon_2)\cap \Ecal_{n}} \PP_{(\theta,\alpha,\tau,\beta)} \left( \left|\frac{\EE\left(V_{1,d,\ell_{\star}}\right)}{\theta g_{\ell_{\star},\nu}}\right|  \cdot \left|\frac{V_{1,d,\ell_{\star}}}{\EE\left(V_{1,d,\ell_{\star}}\right)} -1\right|  +  \left|\frac{\EE(V_{1,d,\ell_{\star}})}{\theta g_{\ell_{\star},\nu}} -1\right|\geq \frac{\epsilon_1}{2} \right) \nonumber \\
&\leq  \sup_{\Bcal^c(\epsilon_1,\epsilon_2)\cap \Ecal_{n}} \PP_{(\theta,\alpha,\tau,\beta)}  \left( \left|\frac{V_{1,d,\ell_{\star}}}{\EE\left(V_{1,d,\ell_{\star}}\right)} -1\right| > \frac{\epsilon_1}{15} \right).
\end{align}

From (<ref>) in Section <ref>, we have that uniformly for all $(\theta,\alpha,\tau,\beta)\in \Ecal_{n}$, there exists $c_1>0$, such that

\begin{align} \label{eq:theta.E1n.concentration}
&\quad~  \PP_{(\theta,\alpha,\tau,\beta)}  \left( \left|\frac{V_{1,d,\ell_{\star}}}{\EE\left(V_{1,d,\ell_{\star}}\right)} -1\right| > \frac{\epsilon_1}{15} \right) \leq \exp \left\{- c_1 \varphi\left(n^{b_1'}\epsilon_1\right) \right\}, \nonumber \\
\text{where } b_1' &= \min\Bigg\{\frac{1}{2}  - \frac{2(1-\gamma)\nu}{d}- \rho_{22} , ~ \frac{1-\gamma}{2} - \frac{\log\log n}{\log n}, ~ \frac{1}{4} + \frac{(1-\gamma)(\ell_{\star}-2\nu)}{d}- \frac{\rho_{1} +\rho_{22}}{2}, \nonumber \\
& \qquad \qquad \frac{1-\gamma}{4} + \frac{(1-\gamma)(\ell_{\star}-\nu)}{d} - \frac{\rho_{1}}{2} - \frac{\log\log n}{2\log n} \Bigg\} \nonumber \\
&\geq \min\Bigg\{\frac{1}{2}  - \frac{2(1-\gamma)\nu}{d}- \rho_{22} , ~ \frac{1-\gamma}{2} -\varsigma, ~  \frac{1}{4} + \frac{(1-\gamma)(\ell_{\star}-2\nu)}{d}- \frac{\rho_{1} +\rho_{22}}{2} , \nonumber \\
& \qquad \qquad \frac{1-\gamma}{4} + \frac{(1-\gamma)(\ell_{\star}-\nu)}{d} - \frac{\rho_{1}}{2} - \varsigma \Bigg\} = b_1,
\end{align}

for an arbitrarily small $\varsigma>0$ as $n\to\infty$.
Therefore, (<ref>), (<ref>) and (<ref>) implies that (<ref>) and (<ref>) in Assumption <ref> are satisfied with $b_1$ given in (<ref>).

Proof of Part (ii):

Similar to the proof of Part (i), for $\widehat\tau_n$, by using (<ref>) and (<ref>) from Section <ref>, we obtain that for any $\epsilon_2\in (0,1)$,

\begin{align} \label{eq:V0.ineq1}
&\PP_0 \left(\left|\frac{\widehat\tau_n}{\tau_0}-1\right|\geq \frac{\epsilon_2}{2} \right) \leq  \PP_0\left( \left|\frac{V_{0,d,\ell_{\star}}}{\EE_0\left(V_{0,d,\ell_{\star}}\right)} -1\right| > \frac{\epsilon_2}{5} \right), \nonumber \\
&\sup_{\Bcal^c(\epsilon_1,\epsilon_2)\cap \Ecal_{n}} \PP_{(\theta,\alpha,\tau,\beta)} \left(\left|\frac{\widehat\tau_n}{\tau_0}-1\right|\leq \frac{\epsilon_2}{2} \right) \leq  \sup_{\Bcal^c(\epsilon_1,\epsilon_2)\cap \Ecal_{n}} \PP_{(\theta,\alpha,\tau,\beta)}  \left( \left|\frac{V_{0,d,\ell_{\star}}}{\EE\left(V_{0,d,\ell_{\star}}\right)} -1\right| > \frac{\epsilon_2}{15} \right).
\end{align}

From (<ref>) in Section <ref>, we have that for any $\epsilon_2\in (0,1/2)$, uniformly for all $(\theta,\alpha,\tau,\beta)\in \Ecal_{n}$, there exists $c_2>0$, such that

\begin{align} \label{eq:tau.E1n.concentration}
&\quad~  \PP_{(\theta,\alpha,\tau,\beta)}  \left( \left|\frac{V_{0,d,\ell_{\star}}}{\EE\left(V_{0,d,\ell_{\star}}\right)} -1\right| > \frac{\epsilon_2}{15} \right) \leq \exp \left\{- c_2 \varphi\left(n^{b_2'}\epsilon_2\right) \right\}, \nonumber \\
\text{where } b_2' &= \min\Bigg\{\frac{1}{2} , ~ \frac{(1-\gamma)(4\nu+d)}{2d}  - \rho_{21} - \frac{\log\log n}{\log n}, \nonumber \\
& \qquad~~ \frac{1}{4} +\frac{ (1-\gamma)\ell_{\star}}{d} - \frac{\rho_{1} + \rho_{21}}{2}, ~ \frac{(1-\gamma)(4\nu+d +4\ell_{\star})}{4d} - \frac{\rho_{1} + 2\rho_{21} }{2} - \frac{\log\log n}{2\log n} \Bigg\} \nonumber \\
&\geq \min\Bigg\{\frac{1}{2} , ~ \frac{(1-\gamma)(4\nu+d)}{2d}  - \rho_{21} - \varsigma, \nonumber \\
& \qquad~~ \frac{1}{4} +\frac{ (1-\gamma)\ell_{\star}}{d} - \frac{\rho_{1} + \rho_{21}}{2}, ~ \frac{(1-\gamma)(4\nu+d +4\ell_{\star})}{4d} - \frac{\rho_{1} + 2\rho_{21} }{2} - \varsigma \Bigg\} = b_2,
\end{align}

for an arbitrarily small $\varsigma>0$ as $n\to\infty$.
Therefore, (<ref>) and (<ref>) implies that (<ref>) and (<ref>) in Assumption <ref> are satisfied with $b_2$ given in (<ref>).

The rest of this section includes several auxiliary technical results. Section <ref> presents the derivation for the uniform upper bounds of $\EE(V_{u,d,\ell_{\star}})$ for $u\in\{0,1\}$ and all $(\theta,\alpha,\tau,\beta)\in \Ecal_{n}$. Section <ref> presents the uniform error bounds for $V_{u,d,\ell_{\star}}/\EE\left(V_{u,d,\ell_{\star}}\right)-1$ for $u\in\{0,1\}$ and all $(\theta,\alpha,\tau,\beta)\in \Ecal_{n}$, which has been used in the proof of Theorem <ref> above. Section <ref> includes the derivation for the Frobenius norms of some matrices that are used in Section <ref>. Section <ref> presents a technical lemma on the derivative of Matérn covariance function that is used in Section <ref>.

§.§ Uniform Bounds for $\EE(V_{u,d,\ell_{\star}})$ on $\Ecal_{n}$

We first derive some useful bounds for $\EE(V_{u,d,\ell_{\star}})$ for $u\in \{0,1\}$ and all $(\theta,\alpha,\tau,\beta)\in \Ecal_{n}$. Recall that $\ell_{\star}=\lceil \nu+d/2 \rceil$. For short, we write $\mm(\bfs)=\ff(\bfs)^\T  \beta$ for $\bfs\in [0,1]^d$ as the mean function of $Y(\cdot)$. In the following derivation, we always write $\bfk_1=(k_1,\ldots,k_d)^\T $ and $\bfk_2=(k_{d+1},\ldots,k_{2d})^\T $ for $k_1,\ldots,k_{2d}\in \NN$.

From Lemma <ref> of the main text, we have that for any integer $0\leq \ell \leq \ell_{\star}-1$, for any $\bfi\in \Xi_{u,m}$ for $u\in\{0,1\}$,

\begin{align}
& \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi +u \bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \left\|\bfs(\bfi+\bfk_1\omega_m)-\bfs(\bfi+u\bfe_1+\bfk_1\omega_m)\right\|^{2\ell} \nonumber \\
={}& \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi +u \bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \left\{\sum_{j=1}^d \left[ s_j(\bfi+\bfk_1\omega_m)-s_j(\bfi+u\bfe_1+\bfk_1\omega_m) \right]^2 \right\}^{\ell} \nonumber \\
={}& 0, \label{eq:2equiv.x1} \\
& \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi +u \bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \left\|\bfs(\bfi+\bfk_1\omega_m)-\bfs(\bfi+u\bfe_1+\bfk_1\omega_m)\right\|^{2\ell_{\star}} \nonumber \\
={}& \frac{(-1)^{\ell_{\star}}(2\ell_{\star})!}{(\ell_{\star}!)^2} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi +u \bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} s_j(\bfi+\bfk_1\omega_m)^{\ell_{\star}} s_j(\bfi+u\bfe_1+\bfk_1\omega_m)^{\ell_{\star}} \nonumber \\
={}& (-1)^{\ell_{\star}}(2\ell_{\star})! \left(\frac{\omega_m}{m}\right)^{2\ell_{\star}}. \label{eq:2equiv.x2}
\end{align}

These two formulas will be useful in the following derivation.

We observe that

\begin{align} \label{eq:E1}
& \quad ~ \EE (V_{u,d,\ell_{\star}}) \nonumber \\
&= \sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi +u \bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})}   \EE \left\{  Y(\bfs(\bfi + \bfk_1 \omega_m)) Y(\bfs(\bfi + u\bfe_1+ \bfk_2\omega_m))  \right\} \nonumber \\
&= \sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+u \bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \mm \big(\bfs(\bfi + \bfk_1\omega_m) \big) \mm\big(\bfs(\bfi + u \bfe_1+ \bfk_2 \omega_m) \big) \nonumber \\
&\quad + \tau(1-u) \sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_d\leq \ell_{\star}} \left(c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}\right)^2 \nonumber \\
&\quad +\sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}}
c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi +u \bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \theta K_{\alpha,\nu} \Big(  \bfs(\bfi+ \bfk_1 \omega_m )-	\bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \Big).
\end{align}

For the first term in (<ref>), since the partial derivatives of $\ff_1,\ldots,\ff_p$ up to the order $\lceil \nu+d/2 \rceil \geq \ell_{\star}$ are all upper bounded by $C_{\ff}'$ by Assumption <ref>, we can apply the Taylor series expansion to the mean function $\mm(\cdot)$ to obtain that

\begin{align}\label{eq:mm.bound}
&\quad~\left|\sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_d\leq \ell_{\star}}  c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)} \mm\big(\bfs(\bfi+\bfk_1\omega_m) \big)\right|\nonumber\\
&= \Bigg|\sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_d\leq \ell_{\star}}
c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}  \nonumber\\
&\quad \times \Bigg\{\sum_{a_1+\ldots + a_d \le \ell_{\star}-1}\frac{\mm^{(a_1,\ldots,a_d)}(\bfs(\bfi) ) }{a_1!\ldots a_d!}\prod_{j=1}^{d}[s_j(\bfi+\bfk_1\omega_m)-s_j(\bfi) ]^{a_j} \nonumber\\
&\quad +\sum_{a_1+\ldots + a_d =\ell_{\star}}\frac{\ell_{\star}}{a_1!\ldots a_d!} \prod_{j=1}^{d}[s_j(\bfi+\bfk_1\omega_m)-s_j(\bfi) ]^{a_j} \int_0^1 (1-t)^{\ell_{\star}-1}   \nonumber\\
&\quad\times \mm^{(a_1,\ldots,a_d)}\Big(\bfs(\bfi)+t \big( \bfs(\bfi+\bfk_1\omega_m)-\bfs(\bfi) \big)  \Big) \ud t \Bigg\} \Bigg| \nonumber \\
&\stackrel{(i)}{=} \Bigg|\sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_d\leq \ell_{\star}}
c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}  \nonumber\\
&\quad \times \Bigg\{\sum_{a_1+\ldots + a_d =\ell_{\star}}\frac{\ell_{\star}}{a_1!\ldots a_d!} \prod_{j=1}^{d}[s_j(\bfi+\bfk_1\omega_m)-s_j(\bfi) ]^{a_j} \int_0^1 (1-t)^{\ell_{\star}-1}   \nonumber\\
&\quad\times \mm^{(a_1,\ldots,a_d)}\Big(\bfs(\bfi)+t \big( \bfs(\bfi+\bfk_1\omega_m)-\bfs(\bfi) \big)  \Big) \ud t \Bigg\}\nonumber\\
&\quad \times \Bigg\{\sum_{a_1+\ldots + a_d =\ell_{\star}}\frac{\ell_{\star}}{a_1!\ldots a_d!} \prod_{j=1}^{d}[s_j(\bfi+u \bfe_1+\bfk_2\omega_m)-s_j(\bfi+u \bfe_1)]^{a_j} \int_0^1 (1-t)^{\ell_{\star}-1}   \nonumber\\
&\quad\times \mm^{(a_1,\ldots,a_d)}\Big(\bfs(\bfi+u \bfe_1)+t \big( \bfs(\bfi+u \bfe_1+\bfk_2\omega_m)-\bfs(\bfi+u \bfe_1) \big)  \Big)\ud t  \Bigg\} \Bigg| \nonumber \\
&\stackrel{(ii)}{\leq} C C_{\ff}' \|\beta\|_{1} \left(\frac{\omega_m}{m}\right)^{\ell_{\star}},
\end{align}

as $n\rightarrow\infty$ for some $C>0$ dependent on $\ell_{\star},\nu,d$, where $\|\beta\|_1=\sum_{j=1}^{p}|\beta_j|$; the equation (i) follows from Lemma <ref> of the main text; the inequality (ii) follows from that $\big|\mm^{(a_1,\ldots,a_d)}(\bfs)\big|\leq C_{\ff}' \|\beta\|_1$ by Assumption <ref>, $\int_0^1 (1-t)^{\ell_{\star}-1}\ud t\leq 1$, and Lemma <ref> of the main text. Therefore,

\begin{align} \label{eq:E2}
&\quad~\left|\sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d} \leq \ell_{\star}}  c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)} c_{\bfi, d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \mm\big(\bfs(\bfi+\bfk_1\omega_m) \big)\mm\big(\bfs(\bfi+u\bfe_1+\bfk_1\omega_m) \big)  \right|\nonumber\\
&\leq C C_{\ff}^{'2}\|\beta\|_{1}^2 |\Xi_{u,m}|\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}} \leq C C_{\ff}^{'2}\|\beta\|_{1}^2 m^d \left(\frac{\omega_m}{m}\right)^{2\ell_{\star}} .
\end{align}

Now we turn to the second and third terms in (<ref>). We consider two cases, either $\nu\notin \ZZ$ or $\nu\in\ZZ_+$. Define

\begin{align}
\mathcal{F}_{u,d,\ell} (\nu)
&= \zeta_\nu^*\sum_{\bfi  \in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell}  	c_{\bfi, d, \ell}^{(k_1, \ldots, k_d)}c_{\bfi + u\bfe_1, d, \ell}^{(k_{d+1}, \ldots, k_{2d})}
\nonumber \\
& \quad \times
G_\nu\Big(\|\bfs( \bfi+ \bfk_1 \omega_m ) - \bfs(\bfi+u\bfe_1+ \bfk_2 \omega_m ) \|\Big),
\end{align}

where $G_\nu$ is as defined in (<ref>).

Case A1. If $\nu\notin\ZZ$, then using (<ref>), we observe that

\begin{align}\label{eq:EV_u 1}
&\quad~ \tau(1-u) \sum_{\bfi\in \Xi_{0,m}} \sum_{0\leq k_1, \ldots, k_{d} \leq \ell_{\star}} (c_{\bfi,d,\ell_{\star}}^{k_1,\ldots,k_d})^2  +  \sum_{\bfi \in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+ u\bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \nonumber \\
&\quad \times \theta K_{\alpha,\nu} \left(  \bfs(\bfi+ \bfk_1 \omega_m )-\bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \right) \nonumber\\
&= \tau(1-u) \sum_{\bfi\in \Xi_{0,m}} \sum_{0\leq k_1, \ldots, k_{d} \leq \ell_{\star}} (c_{\bfi,d,\ell_{\star}}^{k_1,\ldots,k_d})^2     + \sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+ u\bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \nonumber \\
&\quad \times \sum_{j=0}^\infty \Big\{ \zeta_j \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|^{2j}
\nonumber \\
& \quad+ \zeta^*_{\nu+j} G_{\nu+j} \Big(\|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|\Big) \Big\}\nonumber \\
&= \tau(1-u) \sum_{\bfi\in \Xi_{0,m}} \sum_{0\leq k_1, \ldots, k_{d} \leq \ell_{\star}} (c_{\bfi,d,\ell_{\star}}^{k_1,\ldots,k_d})^2 + \zeta_{\ell_{\star}} |\Xi_{u,m}| (-1)^{\ell_{\star}}(2\ell_{\star})!\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}}  \nonumber\\
&\quad +\sum_{\bfi \in \Xi_{u,m}}  \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+u \bfe_1, d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \Big[ \zeta_\nu^* \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|^{2\nu} \nonumber \\
&\quad +  \sum_{j=\ell_{\star}+1}^{\infty}\zeta_j \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|^{2j} \nonumber\\
&\quad + \sum_{j=1}^{\infty}\zeta^*_{\nu+j} \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|^{2\nu+2j}  \Big] \nonumber \\
&= \tau(1-u) \sum_{\bfi\in \Xi_{0,m}} \sum_{0\leq k_1, \ldots, k_{d} \leq \ell_{\star}} (c_{\bfi,d,\ell_{\star}}^{k_1,\ldots,k_d})^2  + \theta \alpha^{2\ell_{\star}-2\nu}\xi_{\ell_{\star}} |\Xi_{u,m}| (-1)^{\ell_{\star}}(2\ell_{\star})!\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}} \nonumber\\
&\quad +  \mathcal{F}_{u,d,\ell_{\star}} (\nu) +\theta\sum_{\bfi \in \Xi_{u,m}}  \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+u \bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \nonumber \\
&\quad \times \Bigg\{ \sum_{j=\ell_{\star}+1}^{\infty}\alpha^{2j-2\nu}\xi_j \|\bfs( \bfi+ \bfk_1 \omega_m )-  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|^{2j}\nonumber\\
&\qquad + \sum_{j=1}^{\infty}\alpha^{2j}\xi^*_{\nu+j} \|\bfs( \bfi+ \bfk_1 \omega_m )-  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|^{2\nu+2j}  \Bigg\}.
\end{align}

For $\bfk_1 \neq \bfk_2$, we use Lemma <ref> to obtain that

\begin{align*}
&\quad~ \left\|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u \bfe_1+ \bfk_2 \omega_m ) \right\|^{2\nu} = \left(\frac{\omega_m }{m}\right)^{2\nu}  \left\{ \sum_{j=1}^d \left[ k_j - k_{d+j} + O(\omega_m^{-1}) \right]^2 \right\}^\nu  \nonumber \\
&= \left(\frac{\omega_m }{m}\right)^{2\nu}  \left\{ \sum_{j=1}^d ( k_j - k_{d+j} )^2 + O(\omega_m^{-1})  \right\}^\nu  = \left(\frac{\omega_m }{m}\right)^{2\nu}  \left\{ \left[\sum_{j=1}^d ( k_j - k_{d+j} )^2 \right]^\nu + O(\omega_m^{-1})  \right\},
\end{align*}

as $m\rightarrow \infty$ uniformly over $\bfi\in \Xi_{u,m}$.

For $\bfk_1 = \bfk_2$, we have that

\begin{align*}
&\quad~ \left\|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \right\|^{2\nu} \\
&= \left\{\sum_{j=1}^d \left[\frac{\delta_{\bfi+ \bfk_1 \omega_m;j}-ue_j-\delta_{\bfi+u\bfe_1+ \bfk_2 \omega_m;j}}{m}\right]\right\}^{\nu} = u O \left(\frac{1}{m^{2\nu}}\right),
\end{align*}

as $m\rightarrow \infty$ uniformly over $\bfi\in \Xi_{u,m}$. Therefore, by Lemma <ref>,

\begin{align}\label{eq:F.udl}
\mathcal{F}_{u,d,\ell_{\star}} (\nu) &= \zeta_\nu^* \sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}}  c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+u \bfe_1, d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})}  \nonumber \\
&\times  \left\|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \right\|^{2\nu} \nonumber \\
&= \zeta_\nu^* \sum_{\bfi\in \Xi_{u,m}}  \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}: \bfk_1 \neq \bfk_2}  \left\{ c_{d, \ell_{\star}}^{(k_1, \ldots, k_d)} + O(\omega_m^{-1}) \right\}  \nonumber \\
&\quad \times \left\{ c_{d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} + O(\omega_m^{-1}) \right\}\left(\frac{\omega_m }{m}\right)^{2\nu}  \left\{ \left[\sum_{j=1}^d ( k_j - k_{d+j} )^2 \right]^\nu + O(\omega_m^{-1})  \right\}  \nonumber \\
&\quad +\zeta_\nu^* \sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}: \bfk_1 = \bfk_2}
\left\{ c_{d, \ell_{\star}}^{(k_1, \ldots, k_d)} + O(\omega_m^{-1}) \right\} \nonumber \\
&\quad \times \left\{ c_{d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} + O(\omega_m^{-1}) \right\} uO\left(\frac{1}{m^{2\nu}}\right)  \nonumber \\
&= \zeta_\nu^*\left(\frac{\omega_m}{m}\right)^{2\nu} |\Xi_{u,m}| \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}}c_{d, \ell_{\star}}^{(k_1, \ldots, k_d)} c_{d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \left\| \bfk_1- \bfk_2\right\|^{2\nu}	\nonumber \\
&\quad    +\zeta^*_\nu |\Xi_{u,m}| O \left\{ \left(\frac{\omega_m}{m}\right)^{2\nu} \omega_m^{-1} + u\left(\frac{ \omega_m}{m}\right)^{2 \nu}\omega_m^{-2 \nu} \right\}  \nonumber \\
&= \theta \xi_\nu^* \left(\frac{\omega_m }{m}\right)^{2\nu} |\Xi_{u,m}| H_{\ell_{\star},\nu} + \theta \xi_\nu^* |\Xi_{u,m}|\left(\frac{\omega_m}{m}\right)^{2\nu} O\left\{\omega_m^{-1}+u\omega_m^{-2\nu} \right\} \\
&\asymp  \theta m^d \left(\frac{\omega_m}{m}\right)^{2\nu},\nonumber
\end{align}

as $m\rightarrow \infty$, where $H_{\ell_{\star},\nu}$ defined in (<ref>) satisfies $H_{\ell_{\star},\nu}>0$, and $O(\cdot)$ in the last equality only depends on $\nu$. Combining (<ref>) with (<ref>), (<ref>) and (<ref>), we conclude that

\begin{align}\label{eq:EV.case1}
&\quad~ \EE \left(V_{u,d,\ell_{\star}}\right) \nonumber \\
&= \tau(1-u) \sum_{\bfi\in \Xi_{0,m}} \sum_{0\leq k_1, \ldots, k_{d} \leq \ell_{\star}} \left(c_{\bfi,d,\ell_{\star}}^{(k_1,\ldots,k_d)}\right)^2 + \theta \alpha^{2\ell_{\star}-2\nu}\xi_{\ell_{\star}} |\Xi_{u,m}| (-1)^{\ell_{\star}}(2\ell_{\star})!\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}} \nonumber \\
&\quad +\zeta_\nu^* \left(\frac{\omega_m }{m}\right)^{2\nu} |\Xi_{u,m}| H_{\ell_{\star},\nu} +\zeta^*_\nu |\Xi_{u,m}|\left(\frac{\omega_m}{m}\right)^{2\nu} O\{\omega_m^{-1}+u\omega_m^{-2\nu} \}  \nonumber\\
&\quad +\theta\sum_{\bfi \in \Xi_{u,m}}  \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+u \bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})}  \nonumber \\
&\qquad \times \Bigg[ \sum_{j=\ell_{\star}+1}^{\infty} \alpha^{2j-2\nu}\xi_j \left\|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \right\|^{2j} \nonumber\\
&\qquad + \sum_{j=1}^{\infty} \alpha^{2j} \xi^*_{\nu+j} \left\|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \right\|^{2\nu+2j}  \Bigg] + O(1)  \|\beta\|_1^2 |\Xi_{u,m}|\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}}  \nonumber \\
&= \tau(1-u) \sum_{\bfi\in \Xi_{0,m}} \sum_{0\leq k_1, \ldots, k_{d} \leq \ell_{\star}} \left(c_{\bfi,d,\ell_{\star}}^{(k_1,\ldots,k_d)}\right)^2  + \theta \alpha^{2\ell_{\star}-2\nu}\xi_{\ell_{\star}} |\Xi_{u,m}| (-1)^{\ell_{\star}}(2\ell_{\star})!\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}}  \nonumber \\
&\quad + \theta\xi_\nu^* \left(\frac{\omega_m }{m}\right)^{2\nu} |\Xi_{u,m}| H_{\ell_{\star},\nu} +\theta\xi^*_\nu |\Xi_{u,m}|\left(\frac{\omega_m}{m}\right)^{2\nu} O\left(\omega_m^{-1}+u\omega_m^{-2\nu} \right) \nonumber\\
&\quad +\theta\left(\frac{\omega_m}{m}\right)^{2\nu}|\Xi_{u,m}|O(1)\left\{\sum_{j=\ell_{\star}+1}^{\infty} \xi_j \left(\alpha \frac{\omega_m}{m}\right)^{2j-2\nu}+\sum_{j=1}^{\infty}\xi^*_{\nu+j} \left(\alpha \frac{\omega_m}{m}\right)^{2j} \right\} \nonumber \\
&\quad + O(1) \|\beta\|_1^2 |\Xi_{u,m}|\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}},
\end{align}

as $m\rightarrow \infty$, and $O(\cdot)$ only depends on $\ell_{\star},\nu,d,C_{\ff}'$ and not on the parameters $\beta,\theta,\tau,\alpha$.

Hence, using the definitions of $C_{V,0},g_{\ell_{\star},\nu},H_{\ell_{\star},\nu}$ in (<ref>), we can derive from (<ref>) that when $u=1$,

\begin{align}
\frac{\EE \left(V_{1,d,\ell_{\star}}\right)}{\theta g_{\ell_{\star},\nu}} &= \frac{\EE\left(V_{1,d,\ell_{\star}}\right)}{\zeta_\nu^* (\frac{\omega_m }{m})^{2\nu} |\Xi_{1,m}| H_{\ell_{\star},\nu}}  \nonumber \\
&= 1+ O\left(\omega_m^{-1}+\omega_m^{-2\nu} \right) + O(1)\frac{\|\beta\|_1^2}{\theta} |\Xi_{u,m}|\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu} \nonumber \\
&\quad +  O(1) \left[ \sum_{j=\ell_{\star}}^{\infty}\frac{\xi_j}{\xi^*_{\nu}}\left(\alpha\frac{\omega_m }{m}\right)^{2j-2\nu}
+ \sum_{j=1}^{\infty}\frac{\xi^*_{\nu+j}}{\xi^*_{\nu}}\left(\alpha\frac{\omega_m }{m}\right)^{2j} \right] \label{eq:EV.case1.ratio1} \\
&\stackrel{(i)}{=} 1+o(1), \nonumber
\end{align}

as $m\to\infty$, where the relation (i) holds uniformly on the set $\Ecal_{n}$, because according to (<ref>), for all $(\theta,\alpha,\tau,\beta)\in \Ecal_{n}$, as $n\to\infty$,

\begin{align*}
& \frac{\|\beta\|_1^2}{\theta}\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu} \leq \frac{p\|\beta\|^2}{\theta}\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu} \leq pn^{\rho_{1} - (2\ell_{\star}-2\nu)(1-\gamma)/d} = o(1),  \\
& \alpha \frac{\omega_m}{m} \leq n^{\rho_{32} - (1 - \gamma)/d} =o(1),
\end{align*}

and that the two summations in (<ref>) are all finite given the definition of $\xi_j,\xi^*_{\nu+j}$ for $j\in\NN$ in (<ref>). The $O(\cdot)$ and $o(\cdot)$ only depend on $\ell_{\star},\nu,d,C_{\ff}'$ and not on $\beta,\theta,\tau,\alpha$, and the same applies to the derivation below as well.

For $u=0$, we have that

\begin{align}
\frac{\EE\left(V_{0,d,\ell_{\star}}\right)-\tau C_{V,0}}{\zeta_\nu^* (\frac{\omega_m }{m})^{2\nu} |\Xi_{0,m}| H_{\ell_{\star},\nu}}
&= 1+ O\left(\omega_m^{-1}+\omega_m^{-2\nu} \right)  +O(1)\frac{\|\beta\|_1^2}{\theta} |\Xi_{u,m}|\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu} \nonumber \\
&\quad + O(1) \left[ \sum_{j=\ell_{\star}}^{\infty}\frac{\xi_j}{\xi^*_{\nu}}\left(\alpha\frac{\omega_m }{m}\right)^{2j-2\nu}
+ \sum_{j=1}^{\infty}\frac{\xi^*_{\nu+j}}{\xi^*_{\nu}}\left(\alpha\frac{\omega_m }{m}\right)^{2j} \right] . \label{eq:EV.case1.ratio2.1}
\end{align}

The summations in (<ref>) are all finite given the definition of $\xi_j,\xi^*_{\nu+j}$ for $j\in\NN$ in (<ref>).

Furthermore, since as $m\to\infty$,

\begin{align} \label{eq:tau.CV0}
\tau C_{V,0} &= \tau \sum_{\bfi \in \Xi_{0,m}} \sum_{0\leq k_1, \ldots, k_{d}\leq \ell} \left(c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)} \right)^2 \nonumber \\
&\stackrel{(i)}{=} \tau |\Xi_{0,m}| \left\{\sum_{0\leq k_1, \ldots, k_{d}\leq \ell_{\star}} \left(c_{d, \ell_{\star}}^{(k_1, \ldots, k_d)} \right)^2 + O(\omega_m^{-1}) \right\}  \\
&\stackrel{(ii)}{\asymp} \tau m^d, \nonumber
\end{align}

where (i) follows from Lemma <ref> and the $O(\cdot)$ is uniform over all $\bfi \in \Xi_{0,m}$; (ii) follows because the constants $c_{d, \ell_{\star}}^{(k_1, \ldots, k_d)}$ do not depend on $\bfi$ and $m$, and that $\ell_{\star}$ is a constant.

From (<ref>) and (<ref>), it follows that

\begin{align}\label{eq:EV.case1.ratio2}
\frac{\EE\left(V_{0,d,\ell_{\star}}\right)}{\tau C_{V,0}}&= \frac{\EE\left(V_{0,d,\ell_{\star}}\right)-\tau C_{V,0}}{\zeta_\nu^* (\frac{\omega_m}{m})^{2\nu} |\Xi_{0,m}| H_{\ell_{\star},\nu}} \frac{\zeta_\nu^* \left(\frac{\omega_m }{m}\right)^{2\nu} |\Xi_{0,m}| H_{\ell_{\star},\nu}}{\tau C_{V,0}} + 1  \nonumber  \\
&= 1+ O(1)\frac{\theta}{\tau} \left(\frac{\omega_m}{m}\right)^{2\nu} \Bigg\{1+ O\left(\omega_m^{-1}+\omega_m^{-2\nu} \right)  +\frac{\|\beta\|_1^2}{\theta} |\Xi_{u,m}|\left(\frac{\omega_m}{m}\right)^{2\ell-2\nu}   \nonumber \\
&\quad + O(1) \left[ \sum_{j=\ell}^{\infty}\frac{\xi_j}{\xi^*_{\nu}}\left(\alpha\frac{\omega_m}{m}\right)^{2j-2\nu} + \sum_{j=1}^{\infty}\frac{\xi^*_{\nu+j}}{\xi^*_{\nu}}\left(\alpha\frac{\omega_m}{m}\right)^{2j} \right]  \Bigg\} \\
&\stackrel{(i)}{=} 1+o(1), \nonumber
\end{align}

as $m\rightarrow\infty$, where the relation (i) holds uniformly on the set $\Ecal_{n}$, because according to (<ref>), for all $(\theta,\alpha,\tau,\beta)\in \Ecal_{n}$, as $n\to\infty$,

\begin{align*}
& \frac{\theta}{\tau} \left(\frac{\omega_m}{m}\right)^{2\nu} \leq n^{\rho_{21} - 2\nu(1-\gamma)/d} = o(1), \\
& \frac{\|\beta\|_1^2}{\theta}\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu} \leq \frac{p\|\beta\|^2}{\theta}\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu} \leq pn^{\rho_{1} - (2\ell_{\star}-2\nu)(1-\gamma)/d} = o(1),  \\
& \alpha \frac{\omega_m}{m} \leq n^{\rho_{32} - (1 - \gamma)/d} =o(1),
\end{align*}

and that the two summations in (<ref>) are all finite given the definition of $\xi_j,\xi^*_{\nu+j}$ for $j\in\NN$ in (<ref>).

Case A2. If $\nu\in\ZZ_+$, then the bound in (<ref>) still applies, and we only need to derive the order for the second and the third terms in (<ref>).

We observe that

\begin{align}\label{eq:EV_u 2}
&\quad~ \tau(1-u) \sum_{\bfi\in \Xi_{0,m}} \sum_{0\leq k_1, \ldots, k_{d} \leq \ell_{\star}} \left(c_{\bfi,d,\ell_{\star}}^{(k_1,\ldots,k_d)}\right)^2  + \sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}}  c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+u \bfe_1, d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})}
\nonumber \\
&\quad \times \theta K_{\alpha,\nu} \left(\bfs(\bfi+ \bfk_1 \omega_m )- \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \right)
\nonumber\\
&= \tau(1-u) \sum_{\bfi\in \Xi_{0,m}} \sum_{0\leq k_1, \ldots, k_{d} \leq \ell_{\star}}\left(c_{\bfi,d,\ell_{\star}}^{(k_1,\ldots,k_d)}\right)^2   + \sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}}  c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+u \bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \nonumber \\
& \quad \times \sum_{j=0}^\infty \Big\{ \zeta_j \|\bfs(\bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \|^{2j} \nonumber \\
& \quad+ \zeta^*_{\nu+j} \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \|^{2\nu+2j}  \log \left(\|\bfs(\bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \| \right) \Big\} \nonumber \\
&=  \tau(1-u) \sum_{\bfi\in \Xi_{0,m}} \sum_{0\leq k_1, \ldots, k_{d} \leq \ell_{\star}} \left(c_{\bfi,d,\ell_{\star}}^{(k_1,\ldots,k_d)}\right)^2   + \zeta_{\ell_{\star}} |\Xi_{u,m}| (-1)^{\ell_{\star}}(2\ell_{\star})!\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}} \nonumber\\
&\quad +\sum_{\bfi\in \Xi_{u,m}}\sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi,  d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+ u\bfe_1, d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \nonumber \\
&\quad \times \Big[ \zeta_\nu^* \|\bfs(\bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \|^{2\nu}
\log \left(\frac{m}{\omega_m } \|\bfs(\bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \| \right)	\nonumber \\
&\quad +\zeta^*_{\nu+1} \|\bfs(\bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \|^{2\nu+2} \nonumber \\
&\quad \times  \log \left(\frac{m}{\omega_m } \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \| \right) \nonumber \\
&\quad -\zeta^*_{\nu+1} \|\bfs(\bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \|^{2\nu+2} \log \left(\frac{m}{\omega_m} \right)\nonumber \\
&\quad  +  \sum_{j=\ell_{\star}+1}^{\infty}\zeta_j \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|^{2j}  \nonumber\\
&\quad + \sum_{j=2}^{\infty}\zeta^*_{\nu+j} \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|^{2\nu+2j} \nonumber\\
&\qquad \times\log (\|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|) \Big] \nonumber \\
&=  \tau(1-u) \sum_{\bfi\in \Xi_{0,m}} \sum_{0\leq k_1, \ldots, k_{d} \leq \ell_{\star}} \left(c_{\bfi,d,\ell_{\star}}^{(k_1,\ldots,k_d)}\right)^2 \nonumber\\
&\quad + \theta \alpha^{2\ell_{\star}-2\nu}\{\xi_{1,\ell_{\star}}+\xi_{2,\ell_{\star}}\log(\alpha)\} |\Xi_{u,m}| (-1)^{\ell_{\star}}(2\ell_{\star})!\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}} \nonumber\\
&\quad + \mathcal{F}_{u,d,\ell_{\star}}(\nu)-\theta \alpha^2\xi^*_{\nu+1}|\Xi_{u,m}| (-1)^{\ell_{\star}}(2\ell_{\star})!\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}}\log \left(\frac{m}{\omega_m } \right) \Ical\{\ell_{\star}=\nu+1 \}\nonumber\\
&\quad +\theta \sum_{\bfi\in \Xi_{u,m}}\sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi,  d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+ u\bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \nonumber\\
&\quad \times \Big[ \alpha^2\xi^*_{\nu+1} \|\bfs(\bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \|^{2\nu+2} \nonumber \\
&\qquad \times  \log \left(\frac{m}{\omega_m} \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \| \right) \nonumber \\
&\qquad +  \sum_{j=\ell_{\star}+1}^{\infty}\alpha^{2j-2\nu}\xi_{1,j} \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|^{2j}  \nonumber\\
&\qquad  +  \sum_{j=\ell_{\star}+1}^{\infty}\alpha^{2j-2\nu}\log(\alpha)\xi_{2,j} \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|^{2j}   \nonumber\\
&\qquad + \sum_{j=2}^{\infty}\alpha^{2j}\xi^*_{\nu+j} \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|^{2\nu+2j}  \nonumber\\
&\qquad \times\log \left(\|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m ) \|\right) \Big].
\end{align}

Similar to Case A1, we can use Lemma <ref> to derive the order for $\mathcal{F}_{u,d,\ell_{\star}} (\nu)$:

\begin{align}\label{eq:F.udl2}
& \quad~ \mathcal{F}_{u,d,\ell_{\star}} (\nu) \nonumber \\
&= \zeta_\nu^* \sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi,  d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+u\bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})}  \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \|^{2\nu} \nonumber \\
&\times \log \left( \|\bfs( \bfi+ \bfk_1 \omega_m ) -  \bfs( \bfi+u\bfe_1+ \bfk_2 \omega_m) \| \right) \nonumber \\
&=\zeta_\nu^*  \sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}: \bfk_1\neq \bfk_2} \left\{ c_{d, \ell_{\star}}^{(k_1, \ldots, k_d)} + O(\omega_m^{-1}) \right\} \left \{ c_{d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} + O(\omega_m^{-1}) \right\} \left(\frac{\omega_m}{m}\right)^{2\nu}   \nonumber \\
& \qquad\times \left\{ \left[\sum_{j=1}^d ( k_j - k_{d+j} )^2 \right]^\nu + O(\omega_m^{-1})  \right\}  \log \left\{ \left[\sum_{j=1}^d ( k_j - k_{d+j} )^2 \right]^{1/2} + O(\omega_m^{-1})  \right\} \nonumber \\
&+ \zeta_\nu^* \sum_{\bfi\in \Xi_{u,m}} \sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}: \bfk_1= \bfk_2} \left\{ c_{d, \ell_{\star}}^{(k_1, \ldots, k_d)} + O(\omega_m^{-1}) \right\} \nonumber \\
& \qquad \times \left\{ c_{d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} + O(\omega_m^{-1}) \right\} uO \left\{  \left(\frac{1}{m}\right)^{2\nu} \log m   \right\} \nonumber \\
&= \zeta_\nu^* \left(\frac{\omega_m}{m}\right)^{2\nu}|\Xi_{u,m}| H_{\ell_{\star},\nu} + \zeta_\nu^*\left(\frac{\omega_m}{m}\right)^{2\nu} |\Xi_{u,m}| O\left\{ \omega_m^{-1}  + u\omega_m^{-2\nu} \log m \right\} \nonumber \\
&= \theta\xi_\nu^* \left(\frac{\omega_m}{m}\right)^{2\nu}|\Xi_{u,m}| H_{\ell_{\star},\nu} + \theta\xi_\nu^*\left(\frac{\omega_m}{m}\right)^{2\nu} |\Xi_{u,m}| O( \omega_m^{-1} + u\omega_m^{-2\nu} \log m )  \\
&\asymp  \theta m^d \left(\frac{\omega_m}{m}\right)^{2\nu},  \nonumber
\end{align}

as $m\rightarrow \infty$. Combining (<ref>) with (<ref>), (<ref>) and (<ref>), we conclude that

\begin{align}\label{eq:EV.case2}
&\quad ~ \EE \left(V_{u,d,\ell_{\star}}\right)  \nonumber \\
&= \tau(1-u) \sum_{\bfi\in \Xi_{0,m}} \sum_{0\leq k_1, \ldots, k_{d} \leq \ell_{\star}} \left(c_{\bfi,d,\ell_{\star}}^{(k_1,\ldots,k_d)}\right)^2  + \theta\xi_\nu^* \left(\frac{\omega_m}{m}\right)^{2\nu} |\Xi_{u,m}| H_{\ell_{\star},\nu}  \nonumber \\
&\quad + \theta\xi^*_\nu |\Xi_{u,m}|\left(\frac{\omega_m}{m}\right)^{2\nu} O(\omega_m^{-1}) + \theta \alpha^{2\ell_{\star}-2\nu}\{\xi_{1,\ell_{\star}}+\xi_{2,\ell_{\star}}\log(\alpha)\} |\Xi_{u,m}| (-1)^{\ell_{\star}}(2\ell_{\star})!\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}}  \nonumber\\
&\quad -\theta \alpha^2\xi^*_{\nu+1}|\Xi_{u,m}| (-1)^{\ell_{\star}}(2\ell_{\star})!\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}}\log \left(\frac{m}{\omega_m}\right) \Ical\{\ell_{\star}=\nu+1 \}  \nonumber\\
&\quad +\theta\left(\frac{\omega_m}{m}\right)^{2\nu}|\Xi_{u,m}|O(1)\Bigg\{\xi^*_{\nu+1}(\alpha \frac{\omega_m}{m})^2+ \sum_{j=\ell_{\star}+1}^{\infty} \xi_{1,j}\left(\alpha \frac{\omega_m}{m}\right)^{2j-2\nu}  \nonumber \\
&\quad+\sum_{j=\ell_{\star}+1}^{\infty} \log(\alpha)\xi_{2,j}\left(\alpha \frac{\omega_m}{m}\right)^{2j-2\nu} +\sum_{j=1}^{\infty}\xi^*_{\nu+j}(\alpha \frac{\omega_m}{m})^{2j}\log\left(\frac{m}{\omega_m}\right)  \Bigg\}  \nonumber \\
&\quad+O(1) \|\beta\|_1^2 |\Xi_{u,m}|\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}},
\end{align}

as $m\rightarrow \infty$.

Hence, using the definitions of $C_{V,0},g_{\ell_{\star},\nu},H_{\ell_{\star},\nu}$ in (<ref>), we can derive from (<ref>) that for $u=1$,

\begin{align}
\frac{\EE V_{1,d,\ell_{\star}}}{\theta g_{\ell_{\star},\nu}}
&= \frac{\EE V_{1,d,\ell_{\star}}}{\zeta_\nu^* \left(\frac{\omega_m}{m}\right)^{2\nu} |\Xi_{1,m}| H_{\ell_{\star},\nu}}  \nonumber \\
&= 1+ O(\omega_m^{-1})+O(1)\frac{\|\beta\|_1^2}{\theta}\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu} \nonumber \\
&\quad + O(1) \frac{\xi^*_{\nu+1}}{\xi^*_{\nu}}\left(\alpha \frac{\omega_m}{m}\right)^{2}\log\left(\frac{m}{\omega_m}\right) \Ical\{ \ell_{\star}=\nu+1\} \nonumber  \\
&\quad +  O(1) \Bigg[ \frac{\xi^*_{\nu+1}}{\xi^*_{\nu}}\left(\alpha \frac{\omega_m}{m}\right)^{2}+\sum_{j=\ell_{\star}}^{\infty}\frac{\xi_{1,j}}{\xi^*_{\nu}}
\left(\alpha\frac{\omega_m}{m}\right)^{2j-2\nu}  \nonumber \\
&\quad +\sum_{j=\ell_{\star}}^{\infty}\frac{\xi_{2,j}}{\xi^*_{\nu}}\log(\alpha)\left(\alpha\frac{\omega_m}{m}\right)^{2j-2\nu}
+ \sum_{j=2}^{\infty}\frac{\xi^*_{\nu+j}}{\xi^*_{\nu}}\left(\alpha\frac{\omega_m}{m}\right)^{2j}\log\left(\frac{m}{\omega_m}\right) \Bigg] \label{eq:EV.case2.ratio1} \\
&\stackrel{(i)}{=} 1+ o(1), \nonumber
\end{align}

where the last relation (i) holds uniformly on the set $\Ecal_{n}$, because according to (<ref>), for all $(\theta,\alpha,\tau,\beta)\in \Ecal_{n}$, as $n\to\infty$,

\begin{align} \label{eq:case2.rho.order1}
& \frac{\|\beta\|_1^2}{\theta}\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu} \leq \frac{p\|\beta\|^2}{\theta}\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu} \leq pn^{\rho_{1} - (2\ell_{\star}-2\nu)(1-\gamma)/d} = o(1),  \nonumber \\
& \alpha \frac{\omega_m}{m} \leq n^{\rho_{32} - (1 - \gamma)/d} =o(1), \nonumber \\
& \left(\alpha \frac{\omega_m}{m} \right)^2 |\log \alpha| \leq n^{2\rho_{32} - 2(1 - \gamma)/d} \cdot \frac{\rho_{31}+\rho_{32}}{d}\log n = o(1),  \nonumber \\
& \left(\alpha \frac{\omega_m}{m} \right)^{2(\ell_{\star}-\nu)} \log \left(\frac{m}{\omega_m}\right) \leq   n^{2(\ell_{\star}-\nu) [2\rho_{32} - 2(1 - \gamma)/d]} \cdot \frac{1-\gamma}{d}\log n = o(1),
\end{align}

and that the three summations in (<ref>) are all finite given the definition of $\xi_{1,j},\xi_{2,j},\xi^*_{\nu+j}$ for $j\in\NN$ in (<ref>).

For $u=0$, we have that

\begin{align}
\frac{\EE \left(V_{0,d,\ell_{\star}}\right)-\tau C_{V,0}}{\zeta_\nu^* \left(\frac{\omega_m}{m}\right)^{2\nu} |\Xi_{0,m}| H_{\ell_{\star},\nu}}
&= 1+ O(\omega_m^{-1})+O(1)\frac{\|\beta\|_1^2}{\theta}\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu}  \nonumber  \\
&\quad + O(1) \frac{\xi^*_{\nu+1}}{\xi^*_{\nu}}\left(\alpha \frac{\omega_m}{m}\right)^{2}\log\left(\frac{m}{\omega_m}\right)\Ical\{ \ell_{\star}=\nu+1\}  \nonumber  \\
&\quad +  O(1) \Bigg[ \frac{\xi^*_{\nu+1}}{\xi^*_{\nu}}\left(\alpha \frac{\omega_m}{m}\right)^{2}+\sum_{j=\ell_{\star}}^{\infty}\frac{\xi_{1,j}}{\xi^*_{\nu}}
\left(\alpha\frac{\omega_m}{m}\right)^{2j-2\nu} \nonumber \\
&\quad +\sum_{j=\ell_{\star}}^{\infty}\frac{\xi_{2,j}}{\xi^*_{\nu}}\log(\alpha)\left(\alpha\frac{\omega_m}{m}\right)^{2j-2\nu}
+ \sum_{j=2}^{\infty}\frac{\xi^*_{\nu+j}}{\xi^*_{\nu}}\left(\alpha\frac{\omega_m}{m}\right)^{2j}
\log\left(\frac{m}{\omega_m}\right) \Bigg]. \nonumber
\end{align}

\begin{align}
\frac{\EE \left(V_{0,d,\ell_{\star}}\right)}{\tau C_{V,0}} &= \frac{\EE \left(V_{0,d,\ell_{\star}}\right)-\tau C_{V,0}}{\zeta_\nu^* \left(\frac{\omega_m}{m}\right)^{2\nu} |\Xi_{0,m}| H_{\ell_{\star},\nu}} \frac{\zeta_\nu^* \left(\frac{\omega_m}{m}\right)^{2\nu} |\Xi_{0,m}| H_{\ell_{\star},\nu}}{\tau C_{V,0}} + 1 \nonumber \\
&= 1+ O(1)\frac{\theta}{\tau} \left(\frac{\omega_m}{m}\right)^{2\nu} \Bigg\{1+ O(\omega_m^{-1})+O(1)\frac{\|\beta\|_1^2}{\theta}\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu}  \nonumber \\
&\quad + O(1) \frac{\xi^*_{\nu+1}}{\xi^*_{\nu}}\left(\alpha \frac{\omega_m}{m}\right)^{2}\log\left(\frac{m}{\omega_m}\right)\Ical\{ \ell_{\star}=\nu+1\} \nonumber \\
&\quad +  O(1) \Bigg[ \frac{\xi^*_{\nu+1}}{\xi^*_{\nu}}\left(\alpha \frac{\omega_m}{m}\right)^{2}+\sum_{j=\ell_{\star}}^{\infty}\frac{\xi_{1,j}}{\xi^*_{\nu}}
\left(\alpha\frac{\omega_m}{m}\right)^{2j-2\nu} \nonumber \\ &\quad+\sum_{j=\ell_{\star}}^{\infty}\frac{\xi_{2,j}}{\xi^*_{\nu}}\log(\alpha)\left(\alpha\frac{\omega_m}{m}\right)^{2j-2\nu}
+ \sum_{j=2}^{\infty}\frac{\xi^*_{\nu+j}}{\xi^*_{\nu}}\left(\alpha\frac{\omega_m}{m}\right)^{2j}
\log\left(\frac{m}{\omega_m}\right) \Bigg]  \Bigg\} \label{eq:EV.case2.ratio2} \\
&\stackrel{(i)}{=} 1+ o(1), \nonumber
\end{align}

as $m\to\infty$, where the relation (i) holds uniformly on the set $\Ecal_{n}$ because of the relations in (<ref>) on $\Ecal_{n}$,

\begin{align*}
& \frac{\theta}{\tau} \left(\frac{\omega_m}{m}\right)^{2\nu} \leq n^{\rho_{21} - 2\nu(1-\gamma)/d} = o(1)
\end{align*}

on $\Ecal_{n}$, and that the three summations in (<ref>) are all finite given the definition of $\xi_{1,j},\xi_{2,j},\xi^*_{\nu+j}$ for $j\in\NN$ in (<ref>).

§.§ Uniform Error bounds for $V_{u,d,\ell_{\star}}/\EE\left(V_{u,d,\ell_{\star}}\right)-1$ on $\Ecal_{n}$

We consider the two cases $u=0$ and $u=1$ separately. The $u=0$ case is used for showing the convergence of $\widehat\tau_n$ in (<ref>) and the $u=1$ case is used for showing the convergence of $\widehat\theta_n$ in (<ref>).

Case B1. If $u=0$, we write $\widetilde{W}=\big(W_1,\ldots,W_{|\Xi_{0,m}|}\big)^\T  $ where

\begin{align*}
\left\{W_1,\ldots,  W_{|\Xi_{0,m}|}\right\} &= \left\{ \frac{\nabla_{d,\ell_{\star}} Y\left(\bfs(\bfi)\right)}{\sqrt{ \EE  \left(V_{0,d,\ell_{\star}}\right)}} :
\bfi=(i_1,\ldots, i_d)^\T   \in \Xi_{0,m} \right\}.
\end{align*}

Define $\mu_W=\EE \big(\widetilde{W}\big)$, $\Sigma_W= \EE \Big[\big(\widetilde{W}-\mu_W\big)\big(\widetilde{W}-\mu_W\big)^\T   \Big]$. We can write

\begin{align} \label{eq:V0.decomp}
\frac{V_{0,d,\ell_{\star}}}{\EE \left(V_{0,d,\ell_{\star}}\right)}= \widetilde{Z}^\T  \Sigma_W \widetilde{Z}+2\mu^\T  _W\Sigma_W^{1/2}\widetilde{Z}+\mu_W^\T  \mu_W,
\end{align}

where $\widetilde{Z} = (Z_1,\ldots, Z_{|\Xi_{0,m}|} )^\T   \sim \Ncal \left({0}_{|\Xi_{0,m}|}, I_{|\Xi_{0,m}|}\right)$.

Therefore, using the upper bounds of $\mu_W^\T  \mu_W$ and $\|\Sigma_W\|_F$ proved in the later Section <ref>, (<ref>), (<ref>) and (<ref>) imply that on the set $\Ecal_{n}$, as $m\to\infty$ (equivalently $n=m^d\to\infty$),

\begin{align}\label{eq:mu2.SigmaW.o1}
\mu_W^\T  \mu_W &=  \frac{1}{\EE  (V_{0,d,\ell_{\star}})}\sum_{\bfi \in \Xi_{0,m}}\sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \mm \big(\bfs(\bfi+\bfk_1\omega_m) \big) \mm \big(\bfs(\bfi+ \bfk_2 \omega_m) \big) \nonumber \\
&=   O(1) \frac{\|\beta\|_1^2}{\tau}\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}}  \nonumber \\
&\leq O(1)\frac{p\|\beta\|^2}{\theta} \frac{\theta}{\tau} \left(\frac{\omega_m}{m}\right)^{2\ell_{\star}}  \nonumber \\
&\leq C n^{\rho_{1} + \rho_{21} - 2(1-\gamma)\ell_{\star}/d} = o(1) , \nonumber \\
\|\Sigma_W\|_F^2  &\leq  Cm^{-d} + C\frac{\theta^2}{\tau^2}\left(\frac{\omega_m}{m}\right)^{4\nu+d} \left(1 + [1+|\log(\alpha)|]^2 \right) \nonumber \\
&\leq C \left\{n^{-1} + n^{2\rho_{21} - (1-\gamma)(4\nu+d)/d} \log^2 n \right\} =o(1) ,
\end{align}

where the two $o(1)$'s are based on the order of $\omega_m=\lfloor m^{\gamma} \rfloor$, the upper bounds of $\|\beta_1\|^2/\theta$, $\tau/\theta$ and $\alpha$ in the set $\Ecal_{n}$, and the condition (<ref>).
Hence $\mu_W^\T  \Sigma_W\mu_W\le \mu_W^\T  \mu_W\big\|\Sigma_W\big\|_F=o(1)$ as well.

Notice that taking expectation on both sides of (<ref>) implies that $1= \EE\left(\widetilde{Z}^\T  \Sigma_W \widetilde{Z}\right) + \mu_W^\T  \mu_W$, and hence $\EE\left(\widetilde{Z}^\T  \Sigma_W \widetilde{Z}\right) =1- \mu_W^\T  \mu_W$. We conclude that for any $\epsilon>0$, for all sufficiently large $n$,

\begin{align} \label{eq:V0.concentration}
&\quad ~ \PP\left(\left|\frac{V_{0,d,\ell_{\star}}}{\EE V_{0,d,\ell_{\star}}}-1\right|>\epsilon\right) \nonumber \\
&\leq  \PP\left(\left|\widetilde{Z}^\T  \Sigma_W\widetilde{Z} + \mu_W^\T  \mu_W - 1\right| > \frac{\epsilon}{2}\right)  +  \PP\left(\left|\mu_W^\T  \Sigma_W^{1/2}\widetilde{Z}\right| > \frac{\epsilon}{4}\right) \nonumber \\
&=\PP\left(\left|\widetilde{Z}^\T  \Sigma_W\widetilde{Z}- \EE\left(\widetilde{Z}^\T  \Sigma_W\widetilde{Z}\right)\right| > \frac{\epsilon}{4}\right) + \PP\left(\left|\mu_W^\T  \Sigma_W^{1/2}\widetilde{Z}\right| > \frac{\epsilon}{4} \right) \nonumber \\
&\stackrel{(i)}{\leq}  2\exp\left\{-\frac{C_{\HW}}{16}\min\left( \frac{\epsilon^2}{\|\Sigma_W\|_F^2},\frac{\epsilon}{\|\Sigma_W\|_F} \right) \right\}
+ 2 \exp\left\{-\frac{\epsilon^2}{16\mu_W^\T  \Sigma_W\mu_W} \right\} \nonumber \\
&\leq 2\exp\left\{-\frac{C_{\HW}}{16}\min\left( \frac{\epsilon^2}{\|\Sigma_W\|_F^2},\frac{\epsilon}{\|\Sigma_W\|_F} \right) \right\}  +  2\exp\left\{-\frac{\epsilon^2}{16\mu_W^\T  \mu_W\big\|\Sigma_W\big\|_F} \right\} \nonumber \\
&\stackrel{(ii)}{\leq} 2\exp\left\{-C \varphi\left(\min\left\{n^{1/2}, n^{(1-\gamma)(4\nu+d)/(2d) - \rho_{21}} /\log n\right\} \epsilon \right) \right\} \nonumber \\
&\quad +  2\exp\left\{- Cn^{2(1-\gamma)\ell_{\star}/d - \rho_{1} -\rho_{21}} \min\left\{n^{1/2} , n^{(1-\gamma)(4\nu+d)/(2d) - \rho_{21}} /\log n \right\}  \epsilon^2 \right\} \nonumber \\
&\leq 2\exp\Big\{ -C \varphi\Big( \min\Big\{ n^{1/2}, ~~ n^{(1-\gamma)(4\nu+d)/(2d) - \rho_{21}} /\log n,  \nonumber \\
&\qquad  n^{1/4 + (1-\gamma)\ell_{\star}/d - (\rho_1+\rho_{21})/2} , ~~ n^{(1-\gamma)(4\nu+d+4\ell_{\star})/(4d) - (\rho_1+2\rho_{21})/2} /\log^{1/2} n \Big\}  \epsilon \Big)\Big\},
\end{align}

for some constant $C>0$, where in the inequality (i), the first term follows from the Hanson-Wright inequality in Lemma <ref> and $\|\Sigma_W\|_{\op}\leq \|\Sigma_W\|_F$, and the second term follows from the sub-Gaussian concentration inequality for the Gaussian random variable $\mu_W^\T  \Sigma_W^{1/2}\widetilde{Z}$; the inequality (ii) follows from (<ref>) and (<ref>) in Section <ref>.

Case B2. If $u=1$, we write $\widetilde{U} = \big(U_1,\ldots, U_{2|\Xi_{1,m}|}\big)^\T  $,  where

\begin{align*}
\left\{ U_1,\ldots, U_{|\Xi_{1,m}| } \right\} &=   \left\{ \frac{\nabla_{d,\ell_{\star}} Y(\bfs(\bfi))}{\sqrt{ \EE  \left(V_{1,d,\ell_{\star}}\right)}} :  \bfi=(i_1,\ldots, i_d)^\T   \in \Xi_{1,m} \right\}, \nonumber \\
\left\{ U_{|\Xi_{1,m}| +1},\ldots, U_{2 |\Xi_{1,m}| } \right\} &=   \left\{ \frac{\nabla_{d,\ell_{\star}} Y(\bfs(\bfi+\bfe_1))}{\sqrt{\EE \left(V_{1,d,\ell_{\star}}\right)}} : \bfi=(i_1,\ldots, i_d)^\T   \in \Xi_{1,m} \right\}.
\end{align*}

Define the  $2|\Xi_{1,m}| \times 2 |\Xi_{1,m}|$ symmetric matrix $A = (a_{i,k})_{1\leq i, k\leq 2|\Xi_{1,m}|}$ by

\begin{align*}
a_{j, |\Xi_{1,m}| +j} &=   a_{|\Xi_{1,m}| + j, j} = 1/2, \qquad \text{for } j= 1,\ldots, |\Xi_{1,m}|,\nonumber \\
a_{j,k} &=   0, \qquad \mbox{otherwise.}
\end{align*}

\begin{align}
\frac{ V_{1,d,\ell_{\star}} }{\EE  ( V_{1,d,\ell_{\star}} )} = \widetilde{U}^\T   A \widetilde{U}. \nonumber
\end{align}

Define $\mu_U=\EE \left(\widetilde{U}\right)$ and $\Sigma_U = \EE \left[(\widetilde{U}-\mu_U) (\widetilde{U}-\mu_U)^\T  \right]$.
We observe that

\begin{align}\label{eq:V1.decomp}
\frac{ V_{1,d,\ell_{\star}} }{\EE  \left(V_{1,d,\ell_{\star}} \right)}  = \widetilde{Z}^\T   \Sigma_U^{1/2} A \Sigma_U^{1/2} \widetilde{Z}+ 2\mu_U^\T  A \Sigma_U^{1/2}\widetilde{Z} + \mu_U^\T  A A^\T  \mu_U,
\end{align}

where $\widetilde{Z} = \big(Z_1,\ldots, Z_{2|\Xi_{1,m}|} \big)^\T   \sim \Ncal \left(0_{|\Xi_{0,m}|}, I_{2 |\Xi_{1,m}|}\right)$.

Therefore, using the upper bounds of $\mu_U^\T  AA^\T  \mu_U$, $\left\| \Sigma^{1/2}_U A \Sigma^{1/2}_U \right\|_F$, and $\left\|\Sigma_U \right\|_F$ proved in the later Section <ref>, and from (<ref>), (<ref>) and (<ref>), we have that on the event $\Ecal_{n}$, as $m\to\infty$ (equivalently $n=m^d\to\infty$),

\begin{align}\label{eq:mu2.SigmaU.o1}
\mu_U^\T  AA^\T  \mu_U &=  \frac{1}{4\EE \left(V_{1,d,\ell_{\star}}\right)}\sum_{\bfi \in \Xi_{1,m}}\Bigg\{\sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}}   c_{\bfi, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \nonumber\\
&\quad\times \mm \big(\bfs(\bfi+\bfk_1\omega_m) \big) \mm \big(\bfs(\bfi+ \bfk_2 \omega_m) \big)  \nonumber \\
& \quad +\sum_{0\leq k_1, \ldots, k_{2d}\leq \ell_{\star}}c_{\bfi+\bfe_1, d, \ell_{\star}}^{(k_1, \ldots, k_d)}c_{\bfi+ \bfe_1,  d, \ell_{\star}}^{(k_{d+1}, \ldots, k_{2d})} \mm \big(\bfs(\bfi+\bfe_1+\bfk_1\omega_m) \big) \mm \big(\bfs(\bfi+\bfe_1+ \bfk_2 \omega_m) \big)\Bigg\} \nonumber \\
&=  O(1)\frac{\|\beta\|_1^2}{\theta}\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu}   \nonumber \\
&\leq O(1)\frac{p\|\beta\|^2}{\theta}\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}-2\nu}  \nonumber \\
&\leq C n^{\rho_{1} - (1-\gamma)(2\ell_{\star}-2\nu)/d} = o(1) , \nonumber \\
\left\| \Sigma^{1/2}_U A \Sigma^{1/2}_U \right\|_F^2  &\leq C \Bigg\{\frac{\tau^2}{\theta^2} m^{-d}\left(\frac{\omega_m}{m}\right)^{-4\nu} + \frac{\tau}{\theta} m^{-d} \left(\frac{\omega_m}{m}\right)^{-2\nu}  \nonumber \\
&\qquad + \left(1+ [1+|\log(\alpha)|]^2\right)\left(\frac{\omega_m}{m}\right)^{d} \Bigg\} \nonumber \\
&\leq C\left\{n^{2\rho_{22} - 1 + 4(1-\gamma)\nu/d}  + n^{- (1-\gamma)} \log^2 n\right\} = o(1), \nonumber \\
\left\|\Sigma_U \right\|_F^2 &\leq  C \Bigg\{\frac{\tau^2}{\theta^2} m^{-d}\left(\frac{\omega_m}{m}\right)^{-4\nu} + \frac{\tau}{\theta} m^{-d} \left(\frac{\omega_m}{m}\right)^{-2\nu}  \nonumber \\
&\qquad + \left(1+[1+|\log(\alpha)|]^2\right) \left(\frac{\omega_m}{m}\right)^{d} \Bigg\} \nonumber \\
&\leq C\left\{ n^{2\rho_{22} - 1 + 4(1-\gamma)\nu/d}  + n^{- (1-\gamma)} \log^2 n \right\}  = o(1),
\end{align}

where the three $o(1)$'s are based on the order of $\omega_m=\lfloor m^{\gamma} \rfloor$, the upper bounds of $\|\beta_1\|^2/\theta$, $\tau/\theta$ and $\alpha$ in the set $\Ecal_{n}$, and the condition (<ref>).
Hence $\mu_U^\T  A \Sigma_U A^\T  \mu_U\leq \mu_U^\T  AA^\T  \mu_U \big\|\Sigma_U\big\|_F=o(1)$ as well.
Notice that taking expectation on both sides of (<ref>) implies that $1= \EE\left(\widetilde{Z}^\T  \Sigma_U^{1/2}A \Sigma_U^{1/2} \widetilde{Z}\right) + \mu_U^\T  AA^\T  \mu_U$, and hence $\EE\left(\widetilde{Z}^\T  \Sigma_U^{1/2}A \Sigma_U^{1/2} \widetilde{Z}\right) = 1- \mu_U^\T  AA^\T  \mu_U$. Then similar to the derivation of (<ref>), we conclude from (<ref>), (<ref>) and (<ref>) in Section <ref> that for any $\epsilon>0$, for all sufficiently large $n$,

\begin{align} \label{eq:V1.concentration}
&\quad~ \PP \left(\left|\frac{V_{1,d,\ell_{\star}}}{\EE \left(V_{1,d,\ell_{\star}}\right)}-1\right|>\epsilon\right)  \nonumber \\
&\leq  \PP\left(\left|\widetilde{Z}^\T  \Sigma_U^{1/2}A\Sigma_U\widetilde{Z}-\mu_U^\T  AA^\T  \mu_U-1\right| >\frac{\epsilon}{2}\right) +  \PP\left(\left|\mu^\T _U A\Sigma_U^{1/2}\widetilde{Z}\right|>\frac{\epsilon}{4}\right)  \nonumber \\
&= \PP\left(\left|\widetilde{Z}^\T  \Sigma_U^{1/2}A\Sigma_U\widetilde{Z}- \EE\left(\widetilde{Z}^\T  \Sigma_U^{1/2}A\Sigma_U\widetilde{Z}\right)\right|>\frac{\epsilon}{2}\right) + \PP\left(\left|\mu^\T _U A\Sigma_U^{1/2}\widetilde{Z}\right|>\frac{\epsilon}{4}\right) \nonumber \\
&\leq  2\exp\left\{- \frac{C_{\HW}}{16} \min\left( \frac{\epsilon^2}{\big\| \Sigma^{1/2}_U A \Sigma^{1/2}_U \big\|_F^2}, \frac{\epsilon}{\big\| \Sigma^{1/2}_U A \Sigma^{1/2}_U \big\|_F} \right) \right\}  \nonumber \\
&\quad + 2 \exp\left\{-\frac{\epsilon^2}{16\mu_U^\T  A\Sigma_U A^\T  \mu_U} \right\}  \nonumber \\
&\leq  2\exp\left\{- \frac{C_{\HW}}{16}\min\left( \frac{\epsilon^2}{\big\| \Sigma^{1/2}_U A \Sigma^{1/2}_U \big\|_F^2}, \frac{\epsilon}{\big\| \Sigma^{1/2}_U A \Sigma^{1/2}_U \big\|_F} \right) \right\}  \nonumber \\
&\quad + 2 \exp\left\{-\frac{\epsilon^2}{16\mu_U^\T  A A^\T  \mu_U \left\|\Sigma_U\right\|_F} \right\}  \nonumber \\
&\stackrel{(i)}{\leq}  2\exp\left\{- C \varphi\left(\min\left\{n^{1/2 - \rho_{22} - 2(1-\gamma)\nu/d } , n^{(1-\gamma)/2 } /\log n \right\} \epsilon\right) \right\}  \nonumber \\
&\quad + 2 \exp\left\{-C n^{(1-\gamma)(2\ell_{\star}-2\nu)/d - \rho_{1}} \min\left\{ n^{ 1/2 - 2(1-\gamma)\nu/d - \rho_{22}} , n^{(1-\gamma)/2} /\log n \right\}\epsilon^2 \right\}  \nonumber \\
&\leq  2\exp\Big\{- C \varphi\Big(\min\Big\{ n^{1/2 - \rho_{22} - 2(1-\gamma)\nu/d } , ~~ n^{(1-\gamma)/2} /\log n ,  \nonumber \\
&\qquad   n^{ 1/4 + (1-\gamma)(\ell_{\star}-2\nu)/d - (\rho_{1} + \rho_{22})/2} , ~~
n^{(1-\gamma)/4 + (1-\gamma)(\ell_{\star}-\nu)/d - \rho_{1}/2 }  /\log^{1/2} n \Big\} \epsilon \Big) \Big\} ,
\end{align}

for some constant $C>0$, where the inequality (i) follows from (<ref>) and (<ref>) in Section <ref>.

§.§ Bounds for the Frobenius Norms of $\big\|\Sigma_W\big\|_F^2$, $\big\| \Sigma^{1/2}_U A \Sigma^{1/2}_W \big\|_F^2$ and $\big\|\Sigma_U\big\|_F^2$

This section provides the detailed derivation of upper bounds for $\big\|\Sigma_W\big\|_F^2$, $\big\| \Sigma^{1/2}_U A \Sigma^{1/2}_W \big\|_F^2$, and $\big\|\Sigma_U\big\|_F^2$.

(1) Upper bound for $\|\Sigma_W\|_F^2$.

We first use $(a+b)^2\leq 2(a^2+b^2)$ for any $a,b\in \RR$ to obtain that

\begin{align} \label{eq:Sigma.W.1}
\|\Sigma_W\|_F^2&=  \frac{1}{\left\{\EE \left(V_{0,d,\ell_{\star}}\right)\right\}^2} \sum_{\bfi, \bfj\in \Xi_{0,m}}
\Bigg\{ \sum_{0\leq k_1, \ldots, k_{2d} \leq \ell_{\star}} c_{\bfi,d,\ell_{\star}}^{(k_1,\ldots,k_d)}
c_{\bfj,d,\ell_{\star}}^{(k_{d+1},\ldots,k_{2d})} \nonumber \\
& \quad \times \Big[ \tau \Ical \left\{ \bfs(\bfi+ \bfk_1 \omega_m  )=  \bfs (\bfj+ \bfk_2 \omega_m  ) \right\}  + \theta K_{\alpha,\nu}  \left( \bfs(\bfi+ \bfk_1 \omega_m  )-  \bfs (\bfj+ \bfk_2 \omega_m  ) \right) \Big] \Bigg\}^2 \nonumber \\
&\leq  \frac{2\tau^2}{\left\{\EE \left(V_{0,d,\ell_{\star}}\right)\right\}^2} \sum_{\bfi, \bfj\in \Xi_{0,m}}
\Bigg\{ \sum_{0\leq k_1, \ldots, k_{2d} \leq \ell_{\star}} c_{\bfi,  d,\ell_{\star}}^{(k_1,\ldots,k_d)}
c_{\bfj,  d,\ell_{\star}}^{(k_{d+1},\ldots,k_{2d})} \nonumber \\
&\qquad \times \Ical \{ \bfs(\bfi+ \bfk_1 \omega_m )=  \bfs (\bfj+ \bfk_2 \omega_m  ) \} \Bigg\}^2 \nonumber \\
&\quad  + \frac{2}{\left\{\EE \left(V_{0,d,\ell_{\star}}\right)\right\}^2}  \sum_{\bfi, \bfj\in \Xi_{0,m}} \Bigg\{ \sum_{0\leq k_1, \ldots, k_{2d} \leq \ell_{\star}} c_{\bfi,  d,\ell_{\star}}^{(k_1,\ldots,k_d)} c_{\bfj,  d,\ell_{\star}}^{(k_{d+1},\ldots,k_{2d})}  \nonumber \\
&  \qquad\times \theta K_{\alpha,\nu}  \left( \bfs(\bfi+ \bfk_1 \omega_m )-  \bfs (\bfj+ \bfk_2 \omega_m  ) \right) \Bigg\}^2.
\end{align}

For the first term in (<ref>), from Lemma <ref>, we observe that as $m\to\infty$,

\begin{align} \label{eq:Sigma.W.2}
&\quad~ \sum_{\bfi, \bfj\in \Xi_{0,m}} \left\{ \sum_{0\leq k_1, \ldots, k_{2d} \leq \ell_{\star}} c_{\bfi, d,\ell_{\star}}^{(k_1,\ldots,k_d)} c_{\bfj,  d,\ell_{\star}}^{(k_{d+1},\ldots,k_{2d})} \Ical \{ \bfs(\bfi+ \bfk_1 \omega_m  )=  \bfs (\bfj+ \bfk_2 \omega_m  ) \} \right\}^2  \nonumber \\
&= \sum_{\bfi, \bfj\in \Xi_{0,m}} \Bigg\{ \sum_{0\leq k_1, \ldots, k_{2d} \leq \ell_{\star}} \left[c_{d,\ell_{\star}}^{(k_1,\ldots,k_d)}+O(\omega_m^{-1}) \right] \left[c_{d,\ell_{\star}}^{(k_{d+1},\ldots,k_{2d})} +O(\omega_m^{-1}) \right] \nonumber \\
&  \qquad\times \Ical \{ \bfs(\bfi+ \bfk_1 \omega_m  )=  \bfs (\bfj+ \bfk_2 \omega_m  ) \} \Bigg\}^2 \nonumber \\
&\leq  C m^d,
\end{align}

for some $C>0$ that only depends on $\ell_{\star},\nu,d$.

For the second term in (<ref>), we consider two cases, $\|\bfi-\bfj\|\leq 5\ell_{\star}\omega_m\sqrt{d}$ and $\|\bfi-\bfj\|>5\ell_{\star}\omega_m\sqrt{d}$. For the first case, the number of such $(\bfi,\bfj)$ pairs is at most of order $m^d \omega_m^d$, which implies that

\begin{align} \label{eq:Sigma.W.3}
&\quad~ \sum_{\bfi, \bfj\in \Xi_{0,m}: \| \bfi - \bfj \| \leq 5\ell_{\star} \omega_m \sqrt{d}} \Bigg\{ \sum_{0\leq k_1, \ldots, k_{2d} \leq \ell_{\star}} c_{\bfi,d,\ell_{\star}}^{(k_1,\ldots,k_d)} c_{\bfj, d,\ell_{\star}}^{(k_{d+1},\ldots,k_{2d})} \nonumber \\
&  \qquad\times \theta K_{\alpha,\nu}  \left( \bfs( \bfi+ \bfk_1 \omega_m  )- \bfs (\bfj+ \bfk_2 \omega_m ) \right) \Bigg\}^2 \nonumber \\
&= \sum_{\bfi, \bfj\in \Xi_{0,m}: \| \bfi - \bfj \| \leq 5\ell_{\star} \omega_m \sqrt{d}} \Big\{ \zeta_{\ell_{\star}}(-1)^{\ell_{\star}}(2\ell_{\star})!\left(\frac{\omega_m}{m}\right)^{2\ell_{\star}} \nonumber \\
&\qquad  + \sum_{0\leq k_1, \ldots, k_{2d} \leq \ell_{\star}} c_{\bfi, d,\ell_{\star}}^{(k_1,\ldots,k_d)}c_{\bfj,d,\ell_{\star}}^{(k_{d+1},\ldots,k_{2d})} \nonumber \\
&\qquad \times \Bigg[\sum_{j=\ell_{\star}+1}^{\infty}\zeta_{j} \|\bfs( \bfi+ \bfk_1 \omega_m  )-  \bfs (\bfj+ \bfk_2 \omega_m ) \|^{2j} \nonumber \\
& \qquad+ \sum_{j=0}^{\infty} \zeta^*_{\nu+j} G_{\nu+j}(\|\bfs( \bfi+ \bfk_1 \omega_m  )- \bfs (\bfj+ \bfk_2 \omega_m ) \|) \Bigg] \Bigg\}^2 \nonumber \\
&\leq  C \theta^2 m^d \omega_m^d\left(\frac{\omega_m}{m}\right)^{4\nu},
\end{align}

as $m\to\infty$ for some $C>0$ dependent only on $\ell_{\star},\nu,d$.

For the second case of $\|\bfi-\bfj\|>5\ell_{\star}\omega_m\sqrt{d}$, if we let $\sa=(a_1,\ldots,a_d)\in \NN^d$, then we have that

\begin{align}\label{eq:Sigma.W.4}
&\quad~  \sum_{\bfi, \bfj\in \Xi_{0,m}: \| \bfi-\bfj\| > 5 \ell_{\star} \omega_m \sqrt{d}}
\Bigg\{  \sum_{0\leq k_1,\ldots, k_{2d}\leq \ell_{\star}}  c_{\bfi, d, \ell_{\star}}^{(k_1,\ldots, k_d)}  c_{\bfj,  d, \ell_{\star}}^{(k_{d+1},\ldots, k_{2d})} \nonumber \\
&\qquad \times \theta K_{\alpha,\nu} \left(\bfs( \bfi + \bfk_1 \omega_m ) - \bfs(\bfj + \bfk_2 \omega_m ) \right) \Bigg\}^2 \nonumber \\
&= \sum_{\bfi, \bfj\in \Xi_{0,m}: \| \bfi-\bfj\| > 5 \ell_{\star} \omega_m \sqrt{d}} \Bigg\{\sum_{0\leq k_1,\ldots, k_{2d}\leq \ell_{\star}}  c_{\bfi,d,\ell_{\star}}^{(k_1,\ldots, k_d)}  c_{\bfj,d,\ell_{\star}}^{(k_{d+1},\ldots, k_{2d})} \nonumber \\
&\quad \times \Bigg[ \sum_{0\leq a_1+\ldots + a_d \leq 2\ell_{\star} -1} \frac{\sD^{\sa} \theta K_{\alpha,\nu}  (\bfs( \bfi ) - \bfs(\bfj ) ) }{a_1! \ldots a_d!} \nonumber \\
&\qquad \times \prod_{q=1}^d \left[ s_q(\bfi +\bfk_1 \omega_m ) - s_q(\bfj +\bfk_2 \omega_m ) - s_q (\bfi) + s_q(\bfj) \right]^{a_q} \nonumber \\
&\qquad  + \sum_{a_1+\ldots + a_d = 2\ell_{\star}} \frac{2 \ell_{\star}}{a_1! \ldots a_d!}  \prod_{q=1}^d \left[s_q(\bfi +\bfk_1 \omega_m ) - s_q(\bfj +\bfk_2 \omega_m ) - s_q (\bfi) + s_q(\bfj)\right]^{a_q} \nonumber \\
&\qquad \times \int_0^1 (1-t)^{2 \ell_{\star}-1} \sD^{\sa} \theta K_{\alpha,\nu}  \Big( \bfs (\bfi) - \bfs(\bfj) \nonumber \\
&\qquad + t \{ \bfs (\bfi +\bfk_1 \omega_m ) - \bfs (\bfj +\bfk_2 \omega_m) -\bfs(\bfi) + \bfs(\bfj) \} \Big) \ud t \Bigg] \Bigg\}^2 \nonumber \\
&= \sum_{\bfi, \bfj\in \Xi_{0,m}: \| \bfi-\bfj\| > 5 \ell_{\star} \omega_m \sqrt{d}} \Bigg\{
\sum_{0\leq k_1,\ldots, k_{2d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1,\ldots, k_d)}  c_{\bfj,  d, \ell_{\star}}^{(k_{d+1},\ldots, k_{2d})} \nonumber \\
&\quad  \times \sum_{a_1+\ldots + a_d = 2\ell_{\star}} \frac{2 \ell_{\star}}{a_1! \ldots a_d!}  \prod_{q=1}^d \left[s_q(\bfi +\bfk_1 \omega_m ) - s_q(\bfj +\bfk_2 \omega_m ) -s_q (\bfi) + s_q(\bfj) \right]^{a_q}  \nonumber \\
&\qquad  \times \int_0^1 (1-t)^{2 \ell_{\star}-1} \sD^{\sa}  \theta K_{\alpha,\nu} \Big( \bfs (\bfi) - \bfs(\bfj) \nonumber \\
&  \qquad + t \{ \bfs (\bfi +\bfk_1 \omega_m) - \bfs (\bfj +\bfk_2 \omega_m )
-\bfs(\bfi) + \bfs(\bfj) \} \Big) \ud t \Bigg\}^2 \nonumber \\
&\stackrel{(i)}{\leq} C \left(\frac{\omega_m}{m}\right)^{4 \ell_{\star}}  \sum_{\bfi, \bfj\in \Xi_{0,m}: \| \bfi-\bfj\| > 5 \ell_{\star} \omega_m \sqrt{d}} \Bigg\{\sum_{0\leq k_1,\ldots, k_{2d}\leq \ell_{\star}} \sup_{\substack{0\leq t\leq 1 \\ a_1+\ldots a_d=2\ell_{\star}}} \sD^{(a_1,\ldots,a_d)} \theta K_{\alpha,\nu}  \Big( \bfs (\bfi) - \bfs(\bfj) \nonumber \\
&  \qquad  + t \{ \bfs (\bfi +\bfk_1 \omega_m ) - \bfs (\bfj +\bfk_2 \omega_m ) -\bfs(\bfi) + \bfs(\bfj) \} \Big) \Bigg\}^2 \nonumber \\
&\stackrel{(ii)}{\leq}  C \theta^2 (1+|\log(\alpha)|)^2 \left(\frac{\omega_m}{m}\right)^{4 \ell_{\star}}  \nonumber\\
&\quad \times \sum_{\bfi, \bfj\in \Xi_{0,m}: \| \bfi-\bfj\| > 5 \ell_{\star} \omega_m \sqrt{d}}  \left\{ \|\bfs (\bfi) - \bfs(\bfj) \| -\frac{ (4\ell_{\star} \omega_m +2) \sqrt{d} }{m} \right\}^{4\nu -4\ell_{\star}}   \nonumber \\
&\leq  C \theta^2 (1+|\log(\alpha)|)^2 \left(\frac{\omega_m}{m}\right)^{4 \ell_{\star}}\sum_{\bfi, \bfj\in \Xi_{0,m}: \| \bfi-\bfj\| > 5 \ell_{\star} \omega_m \sqrt{d}} \|\bfs (\bfi) - \bfs(\bfj) \|^{4\nu -4\ell_{\star}}  \nonumber \\
&\leq C\theta^2 (1+|\log(\alpha)|)^2 \left(\frac{\omega_m}{m}\right)^{4 \ell_{\star}} \sum_{\bfi, \bfj\in \Xi_{0,m}: \| \bfi-\bfj\| > 5 \ell_{\star} \omega_m \sqrt{d}} \left(\frac{ \|\bfi - \bfj \| }{m}\right)^{4\nu -4\ell_{\star}} \nonumber \\
&\leq C \theta^2 (1+|\log(\alpha)|)^2 \left(\frac{\omega_m}{m}\right)^{4\ell_{\star}} \sum_{\bfi\in \Xi_{0,m}} m^d \int_{\omega_m/m}^{\sqrt{d}} s^{4 \nu -4\ell_{\star} +d-1} \ud s  \nonumber \\
&\stackrel{(iii)}{\leq} C \theta^2  (1+|\log(\alpha)|)^2 m^{2d} \left(\frac{\omega_m}{m}\right)^{4\nu+d},
\end{align}

as $m\to\infty$, where the inequality (i) follows because all $s_q(\bfi +\bfk_1 \omega_m ) , s_q(\bfj +\bfk_2 \omega_m ) , s_q (\bfi) ,  s_q(\bfj) \in [0,1]$ and the power $2\ell_{\star}-1 =2 \lceil \nu+d/2 \rceil - 1 \geq 2 -1 = 1$,
the inequality (ii) follows from Lemma <ref>, and (iii) follows from $|\Xi_{0,m}|\leq m^d$ and the definition that $\ell_{\star}=\lceil \nu+d/2 \rceil \geq \nu+d/2 > \nu+d/4$ so $4\nu-4\ell_{\star} +d <0$.

Therefore, we combine (<ref>), (<ref>), (<ref>), (<ref>), (<ref>) and (<ref>) to conclude that for all sufficiently large $m$,

\begin{align}\label{eq:Sigma.W.5}
\big\|\Sigma_W\big\|_F^2 &\leq \frac{(\tau C_{V,0})^2}{\left\{\EE \left(V_{0,d,\ell_{\star}}\right)\right\}^2} \cdot \frac{2}{(\tau C_{V,0})^2} \Bigg\{C\tau^2 m^d + C\theta^2 m^d \omega_m^d \left(\frac{\omega_m}{m}\right)^{4\nu} \nonumber \\
&\quad +  C \theta^2 (1+|\log(\alpha)|)^2 m^{2d} \left(\frac{\omega_m}{m}\right)^{4\nu+d}\Bigg\}  \nonumber \\
&\stackrel{(i)}{\leq} \frac{C}{\tau^2 m^{2d}} \Bigg\{\tau^2 m^d + \theta^2 m^d \omega_m^d \left(\frac{\omega_m}{m}\right)^{4\nu} \nonumber \\
&\quad +  \theta^2 (1+|\log(\alpha)|)^2 m^{2d} \left(\frac{\omega_m}{m}\right)^{4\nu+d}\Bigg\} \nonumber \\
&= C \left\{m^{-d} + \frac{\theta^2}{\tau^2}\left(\frac{\omega_m}{m}\right)^{4\nu+d}  \left(1+(1+|\log(\alpha)|)^2\right)  \right\} ,
\end{align}

where the inequality (i) follows from (<ref>), (<ref>), and (<ref>).

(2) Upper bound for $\|\Sigma^{1/2}_U A \Sigma^{1/2}_U\|_F^2$ and $\|\Sigma_U\|_F^2$.

Based on the definition of $\Sigma_U$ and $A$, we partition them into $2\times 2$ blocks of size $|\Xi_{1,m}|\times |\Xi_{1,m}|$ as follows:

\begin{align*}
& \Sigma_U = \begin{pmatrix}
\Sigma_U^{1,1} & \Sigma_U^{1,2} \\
\Sigma_U^{2,1} & \Sigma_U^{2,2}
\end{pmatrix},
\qquad
A = \begin{pmatrix}
A^{1,1} & A^{1,2} \\
A^{2,1} & A^{2,2}
\end{pmatrix},
\end{align*}

where $\Sigma_U^{j,k}$ and $A^{j,k}$ are $|\Xi_{1,m}|\times |\Xi_{1,m}|$ matrices for $j,k\in \{1,2\}$. From the definition of $A$, we have $A^{1,1}=A^{2,2}=0\times I_{|\Xi_{1,m}|}$ and $A^{1,2}=A^{2,1}=I_{|\Xi_{1,m}|}$. Consequently, we observe that

\begin{align}\label{eq:SAS.1}
\left\|\Sigma_U^{1/2} A \Sigma_U^{1/2} \right\|_F^2 & = \tr\left(A \Sigma_U A \Sigma_U\right) \nonumber \\
&=\frac{1}{4} \tr\left\{
\begin{pmatrix}
\Sigma_U^{2,1} & \Sigma_U^{2,2} \\
\Sigma_U^{1,1} & \Sigma_U^{1,2}
\end{pmatrix}
\begin{pmatrix}
\Sigma_U^{2,1} & \Sigma_U^{2,2} \\
\Sigma_U^{1,1} & \Sigma_U^{1,2}
\end{pmatrix}
\right\} \nonumber \\
&=\frac{1}{4} \tr \left\{
\begin{pmatrix}
\Sigma_U^{2,1}\Sigma_U^{2,1} + \Sigma_U^{1,1}\Sigma_U^{2,2} & \Sigma_U^{2,1}\Sigma_U^{2,2}+\Sigma_U^{2,2}\Sigma_U^{1,2} \\
\Sigma_U^{1,1}\Sigma_U^{2,1} + \Sigma_U^{1,2}\Sigma_U^{1,1} & \Sigma_U^{1,2}\Sigma_U^{1,2}+\Sigma_U^{1,1}\Sigma_U^{2,2}
\end{pmatrix}
\right\} \nonumber \\
&= \frac{1}{2}\left\{ \tr\left(\Sigma_U^{1,1}\Sigma_U^{2,2}\right) + \tr\left(\Sigma_U^{1,2}\Sigma_U^{1,2}\right) \right\}.
\end{align}

We further observe that

\begin{align} \label{eq:Sig11.Sig22}
&\quad ~ \tr\left( \Sigma_U^{1,1}\Sigma_U^{2,2}\right)  =   \sum_{1\leq i,j\leq |\Xi_{1,m}|} \left(\Sigma_U^{1,1}\right)_{ij}\left(\Sigma_U^{2,2}\right)_{ij}  \nonumber \\
&=   \frac{1}{ \left\{ \EE \left( V_{1, d, \ell_{\star}} \right) \right\}^2}   \sum_{ \bfi, \bfj \in \Xi_{1,m}}
\EE \left\{ \left[\nabla_{d,\ell_{\star}} Y(\bfs(\bfi))\right] \left[\nabla_{d,\ell_{\star}} Y(\bfs(\bfj)) \right]\right\} \nonumber\\
&\qquad\times \EE \left\{ \left[\nabla_{d,\ell_{\star}} Y(\bfs(\bfi+\bfe_1))\right] \left[\nabla_{d,\ell_{\star}} Y(\bfs(\bfj+\bfe_1))\right]\right\}  \nonumber \\
&=  \frac{1}{ \left\{ \EE \left( V_{1,  d, \ell_{\star}} \right) \right\}^2}  \sum_{\bfi, \bfj \in \Xi_{1,m}}
\sum_{0\leq k_1,\ldots, k_{4d}\leq \ell_{\star}} c_{\bfi,d,\ell_{\star}}^{(k_1,\ldots, k_d)}
c_{\bfj,  d, \ell_{\star}}^{(k_{d+1},\ldots, k_{2d})} c_{\bfi+\bfe_1,  d, \ell_{\star}}^{(k_{2d+1},\ldots, k_{3d})}
c_{\bfj+\bfe_1, d, \ell_{\star}}^{(k_{3d+1},\ldots, k_{4d})}    \nonumber \\
& \quad\times  \left[\tau \Ical \left\{ \bfs(\bfi+\bfk_1 \omega_m )= \bfs(\bfj+ \bfk_2 \omega_m ) \right\}
+ \theta K_{\alpha,\nu}  \left( \bfs(\bfi+\bfk_1 \omega_m )- \bfs(\bfj+ \bfk_2 \omega_m )  \right) \right]  \nonumber \\
& \quad\times  \Big[\tau \Ical \left\{ \bfs(\bfi+\bfe_1 +\bfk_3 \omega_m )= \bfs(\bfj+\bfe_1 + \bfk_4 \omega_m ) \right\} \nonumber \\
&\qquad + \theta K_{\alpha,\nu} \left( \bfs(\bfi+\bfe_1 +\bfk_3 \omega_m )- \bfs(\bfj+\bfe_1 + \bfk_4 \omega_m ) \right) \Big] ,
\end{align}

and similarly

\begin{align} \label{eq:Sig12.Sig12}
&\quad~ \tr\left( \Sigma_U^{1,2}\Sigma_U^{1,2}\right) =   \sum_{1\leq i,j\leq |\Xi_{1,m}|} \left(\Sigma_U^{1,2}\right)_{ij} \left(\Sigma_U^{1,2}\right)_{ji} =  \sum_{1\leq i,j\leq |\Xi_{1,m}|} \left(\Sigma_U^{1,2}\right)_{ij} \left(\Sigma_U^{2,1}\right)_{ji} \nonumber \\
&=   \frac{1}{ \left\{ \EE \left(V_{1,  d, \ell_{\star}} \right) \right\}^2} \sum_{\bfi, \bfj \in \Xi_{1,m}}
\EE \left\{ \left[\nabla_{d,\ell_{\star}} Y(\bfs(\bfi)) \right] \left[\nabla_{d,\ell_{\star}} Y(\bfs(\bfj+\bfe_1)) \right] \right\} \nonumber \\
&\qquad\times \EE \left\{ \left[\nabla_{d,\ell_{\star}} Y(\bfs(\bfi+\bfe_1))\right] \left[\nabla_{d,\ell_{\star}} Y(\bfs(\bfj))\right] \right\} \nonumber \\
&=  \frac{1}{\left\{ \EE \left( V_{1, d, \ell_{\star}} \right) \right\}^2}   \sum_{\bfi, \bfj \in \Xi_{1,m}}  \sum_{0\leq k_1,\ldots, k_{4d}\leq \ell_{\star}} c_{\bfi,d,\ell_{\star}}^{(k_1,\ldots, k_d)}
c_{\bfj+\bfe_1,  d, \ell_{\star}}^{(k_{d+1},\ldots, k_{2d})} c_{\bfi+\bfe_1,  d, \ell_{\star}}^{(k_{2d+1},\ldots, k_{3d})}
c_{\bfj, d, \ell_{\star}}^{(k_{3d+1},\ldots, k_{4d})}    \nonumber \\
&\quad \times  \left[\tau \Ical \left\{ \bfs(\bfi+\bfk_1 \omega_m )= \bfs(\bfj+\bfe_1+ \bfk_2 \omega_m ) \right\}
+ \theta K_{\alpha,\nu}  \left( \bfs(\bfi+\bfk_1 \omega_m )- \bfs(\bfj+\bfe_1+ \bfk_2 \omega_m )  \right) \right]  \nonumber \\
&\quad \times  \Big[\tau \Ical \left\{ \bfs(\bfi+\bfe_1 +\bfk_3 \omega_m )= \bfs(\bfj + \bfk_4 \omega_m ) \right\} \nonumber \\
&\qquad + \theta K_{\alpha,\nu}  \left( \bfs(\bfi+\bfe_1 +\bfk_3 \omega_m )- \bfs(\bfj + \bfk_4 \omega_m )  \right) \Big].
\end{align}

Let $\{\bfb_1, \bfb_2 \} = \{ 0_{|\Xi_{1,m}|}, \bfe _1 \}$. Then by (<ref>), we have that

\begin{align} \label{eq:6.4}
& \tau^2 \sum_{\bfi, \bfj \in \Xi_{1,m}}  \sum_{0\leq k_1,\ldots, k_{4d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1,\ldots, k_d)}  c_{\bfj+\bfb_1,  d, \ell_{\star}}^{(k_{d+1},\ldots, k_{2d})} c_{\bfi+\bfe_1,  d, \ell_{\star}}^{(k_{2d+1},\ldots, k_{3d})} c_{\bfj+\bfb_2,  d, \ell_{\star}}^{(k_{3d+1},\ldots, k_{4d})}  \nonumber \\
&\times \Ical\{ \bfs (\bfi + \bfk_1 \omega_m)  = \bfs(\bfj + \bfb_1 + \bfk_2 \omega_m ) \} \Ical\{ \bfs(\bfi+\bfe_1+ \bfk_3  \omega_m)  = \bfs(\bfj+\bfb_2 + \bfk_4 \omega_m ) \}   \nonumber \\
\leq{}& C \tau^2 m^d,   \nonumber \\
&  \tau \sum_{\bfi, \bfj \in \Xi_{1,m}}  \sum_{0\leq k_1,\ldots, k_{4d}\leq \ell_{\star}} c_{\bfi, d, \ell_{\star}}^{(k_1,\ldots, k_d)} c_{\bfj+\bfb_1,  d, \ell_{\star}}^{(k_{d+1},\ldots, k_{2d})} c_{\bfi+\bfe_1,  d, \ell_{\star}}^{(k_{2d+1},\ldots, k_{3d})}  c_{\bfj+\bfb_2,  d, \ell_{\star}}^{(k_{3d+1},\ldots, k_{4d})} \nonumber \\
&\times  \Ical \{ \bfs(\bfi+ \bfk_1 \omega_m) = \bfs(\bfj +\bfb_1 + \bfk_2\omega_m ) \}
\theta K_{\alpha,\nu}  \left( \bfs(\bfi+\bfe_1 + \bfk_3 \omega_m )- \bfs(\bfj+\bfb_2 + \bfk_4 \omega_m ) \right)  \nonumber \\
\leq{}& C \tau \theta m^d \left(\frac{\omega_m}{m}\right)^{2\nu},  \nonumber \\
&  \tau \sum_{\bfi, \bfj \in \Xi_{1,m}}  \sum_{0\leq k_1,\ldots, k_{4d}\leq \ell_{\star}} c_{\bfi,  d, \ell_{\star}}^{(k_1,\ldots, k_d)} c_{\bfj+\bfb_1,  d, \ell_{\star}}^{(k_{d+1},\ldots, k_{2d})} c_{\bfi+\bfe_1,  d, \ell_{\star}}^{(k_{2d+1},\ldots, k_{3d})} c_{\bfj+\bfb_2,  d, \ell_{\star}}^{(k_{3d+1},\ldots, k_{4d})} \nonumber \\
& \times \theta K_{\alpha,\nu}  \left( \bfs(\bfi+ \bfk_1 \omega_m )- \bfs(\bfj + \bfb_1 + \bfk_2 \omega_m ) \right)  \Ical \{ \bfs(\bfi+\bfe_1+ \bfk_3  \omega_m) =\bfs( \bfj+\bfb_2 + \bfk_4 \omega_m ) \} \nonumber \\
\leq{}& C\tau \theta m^d\left(\frac{\omega_m}{m}\right)^{2\nu},
\end{align}

By the Cauchy-Schwarz inequality, we can obtain from (<ref>), (<ref>) and (<ref>) that for all sufficiently large $m$,

\begin{align}
&  \sum_{\bfi, \bfj \in \Xi_{1,m}: \|\bfi-\bfj\| \leq 5 \ell \omega_m \sqrt{d}} \sum_{0\leq k_1,\ldots, k_{4d}\leq \ell_{\star}} c_{\bfi, d, \ell}^{(k_1,\ldots, k_d)} c_{\bfj+\bfb_1,  d, \ell}^{(k_{d+1},\ldots, k_{2d})}  c_{\bfi+\bfe _1,  d, \ell}^{(k_{2d+1},\ldots, k_{3d})} c_{\bfj+\bfb_2,  d, \ell}^{(k_{3d+1},\ldots, k_{4d})} \nonumber \\
&\quad \times  \theta K_{\alpha,\nu}  \left( \bfs(\bfi + \bfk_1 \omega_m )- \bfs(\bfj + \bfb_1 + \bfk_2 \omega_m ) \right) \nonumber\\
&\quad \times \theta K_{\alpha,\nu}  \left( \bfs( \bfi+ \bfe_1 + \bfk_3 \omega_m )- \bfs(\bfj+\bfb_2 + \bfk_4 \omega_m) \right)  \nonumber \\
&\leq   C\theta^2  m^d \omega_m^d \left(\frac{\omega_m}{m}\right)^{4\nu}, \label{eq:Sigma.U.1} \\
& \sum_{\bfi, \bfj \in \Xi_{1,m}: \|\bfi-\bfj\| > 5 \ell \omega_m \sqrt{d}} \sum_{0\leq k_1,\ldots, k_{4d}\leq \ell_{\star}} c_{\bfi, d, \ell}^{(k_1,\ldots, k_d)} c_{\bfj+\bfb_1,  d, \ell}^{(k_{d+1},\ldots, k_{2d})}  c_{\bfi+\bfe _1,  d, \ell}^{(k_{2d+1},\ldots, k_{3d})} c_{\bfj+\bfb_2,  d, \ell}^{(k_{3d+1},\ldots, k_{4d})} \nonumber \\
&\quad \times  \theta K_{\alpha,\nu}  \left( \bfs(\bfi + \bfk_1 \omega_m )- \bfs(\bfj + \bfb_1 + \bfk_2 \omega_m ) \right) \nonumber\\
&\quad \times  \theta K_{\alpha,\nu}  \left( \bfs( \bfi+ \bfe_1 + \bfk_3 \omega_m )- \bfs(\bfj+\bfb_2 + \bfk_4 \omega_m) \right)  \nonumber \\
&\leq C \theta^2 [1+|\log(\alpha)|]^2  m^{2d} \left(\frac{\omega_m}{m}\right)^{4\nu+d} . \label{eq:Sigma.U.2}
\end{align}

We combine (<ref>), (<ref>), (<ref>), (<ref>), (<ref>), (<ref>), (<ref>), (<ref>) to conclude that for all sufficiently large $m$,

\begin{align} \label{eq:Sigma.U.3}
\left\|\Sigma_U^{1/2} A \Sigma_U^{1/2} \right\|_F^2 & \leq   \frac{\left[\theta g_{\ell_{\star},\nu}\right]^2}{\EE \left(V_{1,d,\ell_{\star}}^2\right)} \cdot \frac{1}{\left[\theta g_{\ell_{\star},\nu}\right]^2} \times \Bigg\{ C\tau^2 m^d + C\tau\theta m^d \left(\frac{\omega_m}{m}\right)^{2\nu} \nonumber \\
&\qquad + C \theta^2  m^d \omega_m^d \left(\frac{\omega_m}{m}\right)^{4\nu}  + C \theta^2 [1+|\log(\alpha)|]^2  m^{2d} \left(\frac{\omega_m}{m}\right)^{4\nu+d} \Bigg\} \nonumber \\
&\leq \frac{C(1+o(1))}{m^{2d} \left(\frac{\omega_m}{m}\right)^{4\nu}\left[\theta g_{\ell_{\star},\nu}\right]^2} \Bigg\{\tau^2 m^d + \tau\theta m^d \left(\frac{\omega_m}{m}\right)^{2\nu} + \theta^2  m^d \omega_m^d \left(\frac{\omega_m}{m}\right)^{4\nu}  \nonumber \\
&\qquad +\theta^2 [1+|\log(\alpha)|]^2 m^{2d} \left(\frac{\omega_m}{m}\right)^{4\nu+d} \Bigg\} \nonumber \\
&\leq C \Bigg\{\frac{\tau^2}{\theta^2} m^{-d}\left(\frac{\omega_m}{m}\right)^{-4\nu} + \frac{\tau}{\theta} m^{-d} \left(\frac{\omega_m}{m}\right)^{-2\nu}  \nonumber \\
&\qquad +  \left( 1+ [1+|\log(\alpha)|]^2 \right) \left(\frac{\omega_m}{m}\right)^{d} \Bigg\} .
\end{align}

For $\|\Sigma_U\|_F^2 = \tr\big[(\Sigma_U^{1,1})^2\big]+\tr\big[(\Sigma_U^{2,2})^2\big]+\tr\big[(\Sigma_U^{1,2})^2\big]+\tr\big[(\Sigma_U^{2,1})^2\big]$, by the similar argument to above, we have that for all sufficiently large $m$,

\begin{align} \label{eq:Sigma.U.4}
\left\|\Sigma_U \right\|_F^2 &\leq C \Bigg\{\frac{\tau^2}{\theta^2} m^{-d}\left(\frac{\omega_m}{m}\right)^{-4\nu} + \frac{\tau}{\theta} m^{-d} \left(\frac{\omega_m}{m}\right)^{-2\nu}  \nonumber \\
&\qquad + \left( 1+ [1+|\log(\alpha)|]^2 \right) \left(\frac{\omega_m}{m}\right)^{d} \Bigg\} .
\end{align}

§.§ A Lemma on the Derivatives of Matérn Covariance Function

For any $\sa=(a_1,\ldots,a_d)\in \NN^d$, the partial derivative $\sD^{\sa} \theta K_{\alpha,\nu} (\bfx)$ is a continuous function. For any $\ell\in \NN$, there exists a positive constant $C_{\sD,\ell}$ that only depends on $\ell,\nu,d$, such that

\begin{align*}
& \left|\sD^{\sa} \theta K_{\alpha,\nu} (\bfs-\bft)\right| \leq C_{\sD,\ell} \theta \left(1+|\log \alpha| \right) \|\bfs-\bft \|^{2\nu - 2\ell},
\end{align*}

for all $\sa=(a_1,\ldots,a_d)\in \NN^d$ satisfying $a_1+\ldots + a_d = 2\ell$ and all $\bfs,\bft \in [0,1]^d, \bfs\neq \bft$.

We consider two different cases below. For short, we write $r=\|\bfs-\bft\|$ and $\overline \sK(r;\theta,\alpha,\nu) = \theta K_{\alpha,\nu} (\bfs-\bft)$. As such, $\sD^{\sa} \theta K_{\alpha,\nu} (\bfs-\bft)$ is the derivative of the composite function $\overline \sK\circ r$.

We first cite an important result about the higher-order derivatives of composite functions. Proposition 1 of 16 has generalized this formula to the functions with multivariate arguments: for the composite function $f(g(\bfx))$ with $\bfx=(x_1,\ldots,x_d)$ and any $k\in \NN$, its $k$th partial derivative satisfies

\begin{align} \label{eq:hardy}
& \frac{\partial^k f(g(\bfx))}{\partial \tilde x_1\ldots \partial \tilde x_k} = \sum_{\pi} \left(\frac{\ud^{|\pi|} f}{\ud y^{|\pi|}}\right) \Bigg|_{y=g(\bfx)}  \prod_{B\in \pi} \frac{\partial^{|B|}g(\bfx)}{\prod_{j\in B} \partial \tilde x_j},
\end{align}

where each $\tilde x_j$'s represents some component in $\bfx=(x_1,\ldots,x_d)$, the sum is over all partition $\pi$ of $\{1,\ldots,k\}$, the product is over all blocks $B$ in the partition $\pi$, and $|\pi|$ denotes the number of blocks in $\pi$. Proposition 2 of 16 has shown that even if some $\tilde x_j$'s correspond to the same component in $\bfx=(x_1,\ldots,x_d)$, (<ref>) still holds true.

In this case, we take $f(\cdot)$ to be $\overline \sK(\cdot;\theta,\alpha,\nu)$ and $g(\cdot)$ to be the radius function $r(\cdot)$. By induction, it is straightforward to verify that for any $d\in \ZZ_+$, for any $k\in \NN$, for all $\sa=(a_1,\ldots,a_d)\in \NN^d$ satisfying $a_1+\ldots + a_d = k$, the partial derivatives of $r(\cdot)$ satisfies

\begin{align} \label{eq:Rad.deriv}
& \left| \sD^{\sa} r \right| \leq C_{k}'/r^{k-1},
\end{align}

for some absolute constant $C_{k}'>0$ that only depends on $k,d$. Therefore, we conclude from (<ref>) and (<ref>) that for any $\ell\in \NN$, for all $\sa=(a_1,\ldots,a_d)\in \NN^d$ satisfying $a_1+\ldots + a_d = 2\ell$ and all $\bfs,\bft \in [0,1]^d, \bfs\neq \bft$,

\begin{align} \label{eq:DK.composite}
& \left|\sD^{\sa} \theta K_{\alpha,\nu} (\bfs-\bft)\right| \leq \sum_{k=1}^{2\ell} C_{1k} \left| \frac{\ud^{k}}{\ud r^k} \overline \sK(r;\theta,\alpha,\nu)\right| \cdot r^{-(2\ell-k)} \nonumber \\
&=   \sum_{k=1}^{2\ell} C_{1k} r^{k-2\ell} \left| \frac{\ud^{k}}{\ud r^k} \overline \sK(r;\theta,\alpha,\nu)\right| ,
\end{align}

where $C_{1k}$'s are some positive constants that only depend on $\ell$, $0\leq k\leq \ell$, and $d$.

Next we bound the derivative of $\overline \sK(\cdot)$ in (<ref>). Since the series expansion in (<ref>) is convergent for all radius $r\in \RR_+$, we can differentiate term-by-term. We proceed in two different cases.

Case 1. If $\alpha\leq 2$, then $\alpha\|\bfs-\bft\|\le 2\sqrt{d}$. First consider the case $\nu\notin\ZZ_+$. Then for any $r>0$, any $k\in \NN$,

\begin{align}\label{eq:case1.alpha<2.notZ}
&\quad~  \left| \frac{\ud^{k}}{\ud r^{k}} \overline \sK(r;\theta,\alpha,\nu) \right|    \nonumber \\
&\leq \theta \left|\sum_{j=\lceil k/2\rceil}^{\infty}\xi_j\frac{(2j)!}{(2j-k)!} \alpha^{2j-2\nu} r^{2j-k}\right| +  \theta \left|\sum_{j=0}^{\infty} \xi_{\nu+j}^* \alpha^{2j} r^{2\nu-k+2j} \prod_{l=1}^{k}(2\nu+2j+1-l)\right| \nonumber \\
&= \theta r^{2\nu-k} \left\{\left|\sum_{j=\lceil k/2\rceil}^{\infty}\xi_j\frac{(2j)!}{(2j-k)!} (\alpha r)^{2j-2\nu} \right| +   \left|\sum_{j=0}^{\infty} \xi_{\nu+j}^* (\alpha r)^{2j}\prod_{l=1}^{k}(2\nu+2j+1-l)\right|\right\} \nonumber  \\
&\leq \theta r^{2\nu-k} \Bigg\{ \sum_{j=\lceil k/2\rceil}^{\infty} |\xi_j| \frac{(2j)!}{(2j-k)!} \left(2\sqrt{d}\right)^{2j-2\nu} +  \sum_{j=0}^{\infty} |\xi_{\nu+j}^*| \left(2\sqrt{d}\right)^{2j} \prod_{l=1}^{k}|2\nu+2j+1-l| \Bigg\} \nonumber \\
&\leq C_{2k} \theta r^{2\nu-k},
\end{align}

for some large constant $C_{2k}$ that depends only on $k,\nu,d$, where the last inequality follows because the two sequences are both convergent given the definition of $\xi_j$ in (<ref>) and $\xi_{\nu+j}^*$ and (<ref>).

If $\nu\in\ZZ_+$, then we can similarly obtain that for a large absolute constant $C_{2k}$, for any $r>0$,

\begin{align}\label{eq:case1.alpha<2.Z}
& \left| \frac{\ud^{k}}{\ud r^{k}} \overline \sK(r;\theta,\alpha,\nu) \right| \leq C_{2k} \theta (1+|\log(\alpha)|) r^{2\nu-k}.
\end{align}

We combine (<ref>), (<ref>), and (<ref>) to conclude that when $\alpha\leq 2$, for any $\ell\in \NN$, for all $\sa=(a_1,\ldots,a_d)\in \NN^d$ satisfying $a_1+\ldots + a_d = 2\ell$ and all $\bfs,\bft \in [0,1]^d, \bfs\neq \bft$,

\begin{align} \label{eq:DK.deriv.case1}
&\quad~ \left|\sD^{\sa} \theta K_{\alpha,\nu} (\bfs-\bft)\right| \nonumber \\
&\leq \sum_{k=1}^{2\ell} C_{1k} r^{k-2\ell} \cdot C_{2k} \theta (1+|\log(\alpha)|) r^{2\nu-k} \nonumber \\
&= \theta (1+|\log(\alpha)|) \sum_{k=1}^{2\ell} C_{1k} C_{2k} r^{2\nu-2\ell}  = C_{3\ell}  \theta (1+|\log(\alpha)|) r^{2\nu-2\ell},
\end{align}

where $C_{3\ell}=\sum_{k=1}^{2\ell} C_{1k} C_{2k}$.

Case 2. If $\alpha\geq 2$, then we consider two different subcases below.

(i)  If $\alpha\|\bfs-\bft\|\leq 2\sqrt{d}$, then similarly to Case 1, (<ref>) holds for any $r>0$. Hence (<ref>) remains true for this subcase.

(ii) For $\alpha\|\bfs-\bft\|> 2\sqrt{d}$, 33 has shown that for the modified Bessel function of the second kind $\Kcal_{\nu}$, it satisfies

\begin{align*}
&\frac{\Kcal_{\nu}(x) }{\Kcal_{\nu}(y)} > \ee^{y-x}(x/y)^{\nu},\quad \text{for } \nu>-\frac{1}{2}, ~0<x<y.
\end{align*}

Since $\alpha r=\alpha\|\bfs-\bft\|> 2\sqrt{d} \geq 2$, we set $x=2$ and $y=\alpha r$ to obtain that

\begin{align}\label{eq:Bessel.1}
& \Kcal_{\nu}(\alpha r) < (\alpha r/2)^{\nu} \ee^{2-\alpha r} \Kcal_{\nu}(2) = 2^{-\nu}\ee^2 \Kcal_{\nu}(2) (\alpha r)^{\nu} \ee^{-\alpha r}.
\end{align}

On the other hand, 48 has proved that (on page 79)

\begin{align*}
& [x^{\nu}\Kcal_{\nu}(x)]^{(1)}=-x^{\nu}\Kcal_{\nu-1}(x), \qquad\text{for any }~\nu\in\RR, ~ x\in \RR_+.
\end{align*}

where the superscript $(k)$ denotes the $k$th derivative for $k\in \ZZ_+$. Let $h_{\nu}(r)=r^{\nu}\Kcal_{\nu}(\alpha r)$ for $r\in \RR_+$. By induction, we can see that for any integer $s\in \NN$ and any $r\in \RR_+$,

\begin{align}\label{eq:Bessel.h.deriv}
h^{(2s)}_{\nu}(r)&=  \sum_{k=0}^{s}c_{1,k,s} \alpha^{s+k} r^{\nu-s+k} \Kcal_{\nu-s-k}(\alpha r), \nonumber \\
h^{(2s+1)}_{\nu}(r)&= \sum_{k=0}^{s}c_{2,k,s}\alpha^{s+k+1} r^{\nu-s+k}\Kcal_{\nu-s-k-1}(\alpha r),
\end{align}

where $c_{1,k,s}$'s and $c_{2,k,s}$'s are finite constants dependent on $k,s,\nu$ only.

From (<ref>), for $\alpha r>2$ and any $s\in \NN$, we can find a finite constant $C_{4s}=\sum_{k=0}^s |c_{1,k,s}|>0$ that depends only on $s,\nu,d$, such that

\begin{align}\label{eq:Bessel.2}
\left|h_{\nu}^{(2s)}(r)\right| &\leq  \sum_{k=0}^{s} |c_{1,k,s}| \alpha^{s+k} r^{\nu-s+k} \Kcal_{\nu-s-k}(\alpha r) \stackrel{(i)}{=} \sum_{k=0}^{s} |c_{1,k,s}| \alpha^{s+k} r^{\nu-s+k} \Kcal_{|s+k-\nu|}(\alpha r) \nonumber \\
&\stackrel{(ii)}{\leq} 2^{-\nu}\ee^2  \sum_{k=0}^{s} |c_{1,k,s}| \alpha^{s+k} r^{\nu-s+k} \cdot \Kcal_{|s+k-\nu|}(2) (\alpha r)^{|s+k-\nu|} \ee^{-\alpha r} \nonumber \\
&=  2^{-\nu} \ee^2 \alpha^{\nu} r^{2\nu-2s} \sum_{k=0}^{s} |c_{1,k,s}| \Kcal_{|s+k-\nu|}(2) (\alpha r)^{|s+k-\nu|+(s+k-\nu)} \ee^{-\alpha r}  \nonumber \\
&\stackrel{(iii)}{\leq} C_{4s} 2^{-\nu} \ee^2 \alpha^{\nu} r^{2\nu-2s} (s+1)  \left\{\max_{t\in \{0,1,\ldots, |s-\nu| + |2s-\nu|\}}\Kcal_{t}(2)\right\} (\alpha r)^{2(|s-\nu|+|2s-\nu|)} \ee^{-\alpha r}  \nonumber \\
&\stackrel{(iv)}{\leq} C_{4s} 2^{-\nu} \ee^2 (s+1)\lceil 2(|s-\nu| +|2s-\nu|) \rceil! \alpha^{\nu} r^{2\nu-2s} \left\{\max_{t\in \{0,1,\ldots,\lceil|s-\nu| +|2s-\nu|\rceil\}}\Kcal_{t}(2)\right\} \nonumber \\
&\stackrel{(v)}{\leq} C_{5s} \alpha^{\nu} r^{2\nu-2s} ,
\end{align}

where (i) follows from the relation $\Kcal_{-s}(r)=\Kcal_s(r)$ for any $s\in \RR$ and $r\in \RR_+$; (ii) follows from the upper bound in (<ref>); (iii) follows from $\alpha r > 2$ and $|s+k-\nu|+(s+k-\nu)\leq 2|s+k-\nu| \leq 2(|s-\nu| +|2s-\nu|)$; (iv) follows because $\exp(x)>x^s/s!$ for all $x>0$ and $s\in \ZZ_+$, we can set $x=\lceil 2(|s-\nu| +|2s-\nu|) \rceil$, such that

\begin{align*}
& \exp(\alpha r)> (\alpha r)^{\lceil 2(|s-\nu| +|2s-\nu|) \rceil} / \lceil 2(|s-\nu| +|2s-\nu|) \rceil! \\
\text{and hence } & (\alpha r)^{2(|s-\nu| +|2s-\nu|)} \ee^{-\alpha r} \leq \lceil 2(|s-\nu| +|2s-\nu|) \rceil! ;
\end{align*}

and the constant $C_{5s}$ in (v) only depends on $s,\nu,d$ but not $\alpha$ and $r$.

Similarly, using the second relation in (<ref>), we can show that for $\alpha r>2$ and any $s\in \NN$, we can find a finite constant $C_{6s}>0$ that depends on $c_{2,k,s},s,\nu,d$, such that

\begin{align}\label{eq:Bessel.3}
\left|h_{\nu}^{(2s+1)}(r)\right| &\leq C_{6s} \alpha^{\nu} r^{2\nu-2s-1} .
\end{align}

Therefore, using the definition $\overline \sK(r;\theta,\alpha,\nu)= 2^{1-\nu}\Gamma(\nu)^{-1}\theta \alpha^{-\nu} r^{\nu} \Kcal_{\nu}(\alpha r) =  2^{1-\nu}\Gamma(\nu)^{-1}\theta \alpha^{-\nu} h_{\nu}(r)$, we combine (<ref>), (<ref>), and (<ref>) to conclude that when $\alpha\|\bfs-\bft\|> 2\sqrt{d}$, for any $\ell\in \NN$, for all $\sa=(a_1,\ldots,a_d)\in \NN^d$ satisfying $a_1+\ldots + a_d = 2\ell$ and all $\bfs,\bft \in [0,1]^d, \bfs\neq \bft$,

\begin{align} \label{eq:DK.deriv.case2}
&\quad~ \left|\sD^{\sa} \theta K_{\alpha,\nu} (\bfs-\bft)\right| \nonumber \\
&\leq \sum_{s=1}^{\ell} C_{7s} r^{2s-2\ell} \cdot \frac{2^{1-\nu}\theta \alpha^{-\nu}}{\Gamma(\nu)} \left|h_{\nu}^{(2s)}(r)\right| + \sum_{s=0}^{\ell-1} C_{8s} r^{2s+1-2\ell} \cdot \frac{2^{1-\nu}\theta \alpha^{-\nu}}{\Gamma(\nu)} \left|h_{\nu}^{(2s+1)}(r)\right| \nonumber \\
&\leq \sum_{s=1}^{\ell}\frac{2^{1-\nu} C_{7s}}{\Gamma(\nu)} \theta  \alpha^{-\nu}  r^{2s-2\ell} \cdot \alpha^{\nu} r^{2\nu-2s}  + \sum_{s=0}^{\ell-1}\frac{2^{1-\nu} C_{8s}}{\Gamma(\nu)} \theta  \alpha^{-\nu} r^{2s+1-2\ell} \cdot \alpha^{\nu} r^{2\nu-2s-1} \nonumber \\
&\leq C_{9\ell} \theta r^{2\nu-2\ell},
\end{align}

where $C_{7s},C_{8s},C_{9\ell}$ are constants that depends only on $0\leq s\leq \ell$ and $\ell,\nu,d$.

The conclusion is proved by combining Case 1 and Case 2.

§ ADDITIONAL NUMERICAL RESULTS

Before we present the simulation results for $d=1$, we first explain the formulas for calculating the mean squared error for Bayesian posterior prediction and the oracle prediction. In the matrix format, our model can be written as $Y_n = F_n\beta + X_n + e_n$, where $X_n=(X(\bfs_1),\ldots,X(\bfs_n))^\T$ and $e_n=(\varepsilon(\bfs_1),\ldots,\varepsilon(\bfs_n))^\T$. Since in the simulations, we assign the normal prior $\beta \sim \Ncal(0,a_0I_p)$ as in Proposition <ref>, we can obtain that the conditional posterior distribution of $\beta$ is

\begin{align} \label{eq:beta.cond.post}
& \beta \mid \theta,\alpha,\tau, Y_n,F_n \sim \Ncal\left(\tilde \beta, ~\big[ F_n^\T \big\{\theta K_{\alpha}(S_n)+\tau I_n\big\}^{-1} F_n + a_0 I_p \big]^{-1} \right),  \nonumber \\
\text{where } & \tilde \beta  =  \big[ F_n^\T \big\{\theta K_{\alpha}(S_n)+\tau I_n\big\}^{-1} F_n + a_0 I_p \big]^{-1} F_n^\T \big\{\theta K_{\alpha}(S_n)+\tau I_n\big\}^{-1} Y_n.
\end{align}

Under the model $Y\sim \gp (\beta^\T \ff, \theta K_{\alpha} + \tau \delta_0)$, by Section 1.5 of 42, the best linear unbiased predictor for $Y(\bfs^*)$ at a new location $\bfs^*\in [0,1]^d$ based on the data $(Y_n,F_n)$ is

\begin{align}\label{eq:BLUP}
\widehat Y(\bfs^*;\theta,\alpha,\tau,\beta) & =
\beta^\T \ff(\bfs^*) + \theta K_{\alpha}(S_n, \bfs^*)^\T \big\{\theta K_{\alpha}(S_n)+\tau I_n\big\}^{-1} \left(Y_n - F_n \beta\right) ,
\end{align}

where $K(S_n,\bfs^*)=\{K(\bfs_1,\bfs^*),\ldots,K(\bfs_n,\bfs^*)\}^\T$ is an $n \times 1$ vector.

Under the Bayesian setup, we randomly draw $(\theta,\alpha,\tau,\beta)$ from the posterior $\Pi(\cdot\mid Y_n,F_n)$ to predict $Y(\bfs^*)$. We denote the predicted variable as $\tilde Y(\bfs^*)$. The Gaussian process predictive distribution implies that

\begin{align} \label{eq:GP.predict}
&\quad~ \tilde Y(\bfs^*) \mid Y_n,F_n, \theta,\alpha,\tau,\beta \nonumber \\
&\sim \Ncal \left(\widehat Y(\bfs^*;\theta,\alpha,\tau,\beta), ~ \theta \left\{K_{\alpha}(\bfs^*,\bfs^*)- \theta K_{\alpha}(S_n, \bfs^*)^\T  \big\{\theta K_{\alpha}(S_n)+\tau I_n\big\}^{-1}  K_{\alpha}(S_n, \bfs^*) \right\} \right).
\end{align}

From (<ref>), (<ref>) and (<ref>), with some algebra, we can integrate out $\beta$ from the posterior distribution and obtain that for any $\bfs^*\in [0,1]^d$,

\begin{align}\label{eq:BLUP.pred}
&\tilde Y(\bfs^*)\mid Y_n,F_n,\theta,\alpha,\tau \sim \Ncal \Big( Y^{\dagger}(\bfs^*;\theta,\alpha,\tau), \vv_n(\bfs^*;\theta,\alpha,\tau) \Big), \quad \text{where } \nonumber \\
& Y^{\dagger}(\bfs^*;\theta,\alpha,\tau) = \theta K_{\alpha}(S_n,\bfs^*)^\T \big\{\theta K_{\alpha}(S_n)+\tau I_n\big\}^{-1} Y_n \nonumber \\
&\quad\quad + \bb(\bfs^*;\theta,\alpha,\tau)^\T \left[ F_n^\T \big\{\theta K_{\alpha}(S_n)+\tau I_n\big\}^{-1} F_n  + a_0I_p \right]^{-1} F_n^\T \big\{\theta K_{\alpha}(S_n)+\tau I_n\big\}^{-1} Y_n, \nonumber \\
&{\vv}_n(\bfs^*;\theta,\alpha,\tau) = \theta \left\{K_{\alpha}(\bfs^*,\bfs^*)- \theta K_{\alpha}(S_n, \bfs^*)^\T  \big\{\theta K_{\alpha}(S_n)+\tau I_n\big\}^{-1}  K_{\alpha}(S_n, \bfs^*) \right\} \nonumber \\
&\quad\quad +  \bb(\bfs^*;\theta,\alpha,\tau)^\T \left[ F_n^\T \big\{\theta K_{\alpha}(S_n)+\tau I_n\big\}^{-1} F_n  + a_0I_p \right]^{-1} \bb(\bfs^*;\theta,\alpha,\tau), \text{ and } \nonumber \\
& \bb(\bfs^*;\theta,\alpha,\tau) = \ff(\bfs^*) - F_n^\T \big\{\theta K_{\alpha}(S_n)+\tau I_n\big\}^{-1} \theta K_{\alpha}(S_n,\bfs^*) .
\end{align}

Let $Y_0(\bfs^*) = \beta_0^\T \ff(\bfs^*) + X(\bfs^*)$ be the true mean function at $\bfs^*$ without measurement error. Then, for the Bayesian posterior prediction $\tilde Y(\bfs^*)$ given a random draw $(\theta,\alpha,\tau)$ from the posterior density $\pi(\theta,\alpha,\tau\mid Y_n, F_n)$, the mean squared error has the formula

\begin{align} \label{eq:mse.bayes}
M_{\text{post}}(\bfs^*) &= \left[{\EE}_{\tilde Y(\bfs^*) \mid \theta,\alpha,\tau} \left\{\tilde Y(\bfs^*) \right\} - Y_0(\bfs^*)\right]^2 + {\Var}_{\tilde Y(\bfs^*) \mid \theta,\alpha,\tau} \left\{ \tilde Y(\bfs^*) \right\} \nonumber \\
&= \left\{ Y^{\dagger}(\bfs^*;\theta,\alpha,\tau) - \beta_0^\T \ff(\bfs^*) - X(\bfs^*) \right\}^2 + {\vv}_n(\bfs^*;\theta,\alpha,\tau),
\end{align}

where $Y^{\dagger}(\bfs^*;\theta,\alpha,\tau)$ and ${\vv}_n(\bfs^*;\theta,\alpha,\tau)$ are defined in (<ref>).

For the oracle prediction mean squared error of the predicted value $\tilde Y_0(\bfs^*)$, we simply replace all $(\theta,\alpha,\tau)$ in (<ref>) by the true parameters $(\theta_0,\alpha_0,\tau_0)$ to obtain that

\begin{align} \label{eq:mse.oracle}
M_{0}(\bfs^*) &= \left[{\EE}_{\tilde Y_0(\bfs^*) \mid \theta_0,\alpha_0,\tau_0} \left\{\tilde Y_0(\bfs^*) \right\} - Y_0(\bfs^*)\right]^2 + {\Var}_{\tilde Y_0(\bfs^*) \mid \theta_0,\alpha_0,\tau_0} \left\{ \tilde Y_0(\bfs^*) \right\} \nonumber \\
&= \left\{ Y^{\dagger}(\bfs^*;\theta_0,\alpha_0,\tau_0) - \beta_0^\T \ff(\bfs^*) - X(\bfs^*) \right\}^2 + {\vv}_n(\bfs^*;\theta_0,\alpha_0,\tau_0),
\end{align}

where $Y^{\dagger}(\bfs^*;\theta_0,\alpha_0,\tau_0)$ and ${\vv}_n(\bfs^*;\theta_0,\alpha_0,\tau_0)$ are defined in (<ref>) with $(\theta,\alpha,\tau)$ replaced by $(\theta_0,\alpha_0,\tau_0)$.

Now we present additional simulation results for the spatial Gaussian process model (<ref>) with domain dimension $d=1$ and the isotropic Matérn covariance function in (<ref>). We still consider two values of the smoothness parameter $\nu=1/2$ and $\nu=1/4$. The true covariance parameters are set to be $\theta_0=5,\alpha_0=1,\tau_0=0.5$ for both $\nu=1/2$ and $\nu=1/4$. For $d=1$ and the domain $[0,1]$, we choose the sampling points $S_n$ to be the regular grid $\bfs_i=(2i-1)/(2n)$ for $i=1,\ldots,n$, where we choose $n=400, 500, 625, 781, 976, 1220, 1525, 1907, 2384, 2980$, such that the sample size roughly follows the geometric sequence $n\approx 400\times 1.25^{k-1}$ for $k=1,\ldots,10$. For the regression functions, we let $\ff(\bfs)=\left(1,\bfs,\bfs^2,\bfs^3\right)^\T$ for $\bfs \in [0,1]$ and $\beta_0=(1,0.66,-1.5,1)^\T$. Other aspects of the Bayesian simulation setup, including the prior specification and the posterior sampling procedures are all the same as the $d=2$ case described in the main text.

The results are summarized in Figure <ref> for $\nu=1/2$ and Figure <ref> for $\nu=1/4$. In the left panels, the boxplots of both the marginal posterior distributions of $\theta$ and $\tau$ contract to the true parameter values $\theta_0=5$ and $\tau_0=0.5$ as $n$ increases. The right panels show that the means of absolute differences from all posterior draws of $(\theta,\tau)$ to the true parameters $(\theta_0,\tau_0)$ decreases approximately linearly in the sample size on the logarithm scale, indicating polynomial posterior contraction rates for $\theta$ and $\alpha$.

Posterior contraction for $d=1$ and $\nu=1/2$. Left column: Boxplots for the marginal posterior densities of $\theta$ and $\tau$ versus the increasing sample size $n$. The grey dashed lines are the true parameters $\theta_0=5$ and $\tau_0=0.5$. Right column: Posterior means of $|\theta-\theta_0|$ and $|\tau-\tau_0|$ versus the increasing sample size $n$, on the logarithm scale. The dashed lines are the linear regression fits, and the grey shaded areas are the 95% confidence bands. All posterior summaries are averaged over 40 macro Monte Carlo replications.

Posterior contraction for $d=1$ and $\nu=1/4$. Left column: Boxplots for the marginal posterior densities of $\theta$ and $\tau$ versus the increasing sample size $n$. The grey dashed lines are the true parameters $\theta_0=5$ and $\tau_0=0.5$. Right column: Posterior means of $|\theta-\theta_0|$ and $|\tau-\tau_0|$ versus the increasing sample size $n$, on the logarithm scale. The dashed lines are the linear regression fits, and the grey shaded areas are the 95% confidence bands. All posterior summaries are averaged over 40 macro Monte Carlo replications.

Prediction mean squared errors for $\nu=1/2$ and $\nu=1/4$ when $d=1$. Left column: The prediction mean squared errors under both the Bayesian posterior prediction and the oracle prediction based on the true parameters. Right column: Ratios of the Bayesian prediction mean squared error and the oracle prediction mean squared error. All posterior summaries are averaged over 2000 testing locations in $[0,1]$ and 40 macro Monte Carlo replications.

We also investigate the Bayesian posterior prediction performance for the $d=1$ case and compare with the best possible prediction. We draw another $N=2000$ points $\bfs_1^*,\ldots,\bfs_N^*$ uniformly from the domain $[0,1]$ as the testing locations, and compute the prediction mean squared error $\EE_{Y_n}  \big\{N^{-1}\sum_{l=1}^N M_{\text{post}}(\bfs_l^*)\big\} $ versus the oracle prediction mean squared error $\EE_{Y_n}  \big\{N^{-1}\sum_{l=1}^N M_{0}(\bfs_l^*)\big\}$. The results are shown in Figure <ref>. The left panel shows that the Bayesian predictive posterior has almost the same mean square errors as the oracle prediction for both $\nu=1/2$ and $\nu=1/4$. The right panel shows that their ratio averaged over $N=2000$ testing locations decreases towards 1 as the sample size $n$ increases, indicating the asymptotic efficiency of Bayesian posterior prediction.

[1]
Anderes, E. (2010).
On the consistent separation of scale and variance in Gaussian
random fields.
The Annals of Statistics 38(2), 870–893.

[2]
Arafat, A., E. Porcu, M. Bevilacqua, and J. Mateu (2018).
Equivalence and orthogonality of Gaussian measures on spheres.
Journal of Multivariate Analysis 167, 306–318.

[3]
Bachoc, F., M. Bevilacqua, and D. Velandia (2019).
Composite likelihood estimation for a Gaussian process under fixed
domain asymptotics.
Journal of Multivariate Analysis 174, 104–534.

[4]
Bachoc, F. and A. Lagnoux (2020).
Fixed-domain asymptotic properties of maximum composite likelihood
estimators for Gaussian processes.
Journal of Statistical Planning and Inference 209,

[5]
Bachoc, F., A. Lagnoux, and T. M. N. Nguyen (2017).
Cross-validation estimation of covariance parameters under
fixed-domain asymptotics.
Journal of Multivariate Analysis 160, 42–67.

[6]
Banerjee, S., A. E. Gelfand, A. O. Finley, and H. Sang (2008).
Gaussian predictive process models for large spatial data sets.
Journal of the Royal Statistical Society: Series B 70(4), 825–848.

[7]
Bevilacqua, M., T. Faouzi, R. Furrer, and E. Porcu (2019).
Estimation and prediction using generalized Wendland covariance
functions under fixed domain asymptotics.
The Annals of Statistics 47(2), 828–856.

[8]
Bolin, D. and K. Kirchner (2023).
Equivalence of measures and asymptotically optimal linear prediction
for Gaussian random fields with fractional-order covariance operators.
Bernoulli 29(2), 1476–1504.

[9]
Chen, H.-S., D. G. Simpson, and Z. Ying (2000).
Infill asymptotics for a stochastic process model with measurement
Statistica Sinica 10, 141–156.

[10]
Datta, A., S. Banerjee, A. O. Finley, and A. E. Gelfand (2016).
Hierarchical nearest-neighbor Gaussian process models for large
geostatistical datasets.
Journal of the American Statistical Association 111(514), 800–812.

[11]
De Oliveira, V., B. Kedem, and D. A. Short (1997).
Bayesian prediction of transformed Gaussian random fields.
Journal of the American Statistical Association 92(440), 1422–1433.

[12]
Du, J., H. Zhang, and V. S. Mandrekar (2009).
Fixed-domain asymptotic properties of tapered maximum likelihood
The Annals of Statistics 37(6A), 3330–3361.

[13]
Ghosal, S. and A. W. van der Vaart (2017).
Fundamentals of Nonparametric Bayesian Inference.
Cambridge University Press.

[14]
Guhaniyogi, R., C. Li, T. D. Savitsky, and S. Srivastava (2022).
Distributed Bayesian inference in massive spatial data.
Statistical Science (forthcoming).

[15]
Handcock, M. S. and M. L. Stein (1993).
A Bayesian analysis of kriging.
Technometrics 35(4), 403–410.

[16]
Hardy, M. (2006).
Combinatorics of partial derivatives.
The Electronic Journal of Combinatorics 13, R1.

[17]
Heaton, J. H., A. Datta, A. O. Finley, R. Furrer, J. Guinness, R. Guhaniyogi,
F. Gerber, R. B. Gramacy, D. Hammerling, M. Katzfuss, F. Lindgren, D. W.
Nychka, F. Sun, and A. Zammit-Mangion (2019).
A case study competition among methods for analyzing large spatial
Journal of Agricultural, Biological and Environmental
Statistics 24, 398–425.

[18]
Horn, R. A. and C. R. Johnson (1985).
Matrix Analysis.
Cambrige University Press.

[19]
Hsu, D., S. M. Kakade, and T. Zhang (2012).
A tail inequality for quadratic forms of subgaussian random vectors.
Electronic Communications in Probability 17(52),

[20]
Ibragimov, I. A. and Y. A. Rozanov (1978).
Gaussian Random Processes (translated by A. B. Aries).
Springer, New York.

[21]
Katzfuss, M. and J. Guinness (2021).
A general framework for Vecchia approximations of Gaussian
Statistical Science 36(1), 124–141.

[22]
Kaufman, C. G., M. J. Schervish, and D. W. Nychka (2008).
Covariance tapering for likelihood-based estimation in large spatial
data sets.
Journal of the American Statistical Association 103(484), 1545–1555.

[23]
Kaufman, C. G. and B. A. Shaby (2013).
The role of the range parameter for estimation and prediction in
Biometrika 100(2), 473–484.

[24]
Li, C. (2022).
Bayesian fixed-domain asymptotics for covariance parameters in a
Gaussian process model.
The Annals of Statistics 6(50), 3334–3363.

[25]
Li, C., S. Srivastava, and D. B. Dunson (2017).
Simple, scalable and accurate posterior interval estimation.
Biometrika 104(3), 665–680.

[26]
Li, D., W. Tang, and B. Sudipto (2021).
Inference for Gaussian processes with Matérn covariogram on
compact Riemannian manifolds.
arXiv preprint arXiv:2104.03529v2.

[27]
Loh, W.-L. (2005).
Fixed-domain asymptotics for a subclass of Matérn-type Gaussian
random fields.
The Annals of Statistics 33(5), 2344–2394.

[28]
Loh, W.-L. (2015).
Estimating the smoothness of a Gaussian random field from
irregularly spaced data via higher-order quadratic variations.
The Annals of Statistics 43(6), 2766–2794.

[29]
Loh, W.-L. and S. Sun (2023).