A Stable Jacobi polynomials based least squares regression estimator
associated with an ANOVA decomposition model.

Mohamed Jebaliaa and Abderrazek Karouib *** Emails:
<EMAIL_ADDRESS>(A. Karoui, corresponding author),
<EMAIL_ADDRESS>(M. Jebalia).
This work was supported in part by the DGRST research grant LR21ES10 and the
PHC-Utique research project 20G1503.

a University of Carthage, National School of Engineering of Bizerte, Menzel
Abderrahman 7035, Tunisia.
b University of Carthage, Faculty of Sciences of Bizerte, Jarzouna 7021,
Tunisia.

Abstract— In this work, we construct a stable and fairly fast estimator for
solving non-parametric multidimensional regression problems. The proposed
estimator is based on the use of multivariate Jacobi polynomials that generate
a basis for a reduced size of $d-$variate finite dimensional polynomial space.
An ANOVA decomposition trick has been used for building this later polynomial
space. Also, by using some results from the theory of positive definite random
matrices, we show that the proposed estimator is stable under the condition
that the i.i.d. random sampling points for the different covariates of the
regression problem, follow a $d-$dimensional Beta distribution. Also, we
provide the reader with an estimate for the $L^{2}-$risk error of the
estimator. Moreover, a more precise estimate of the quality of the
approximation is provided under the condition that the regression function
belongs to some weighted Sobolev space. Finally, the various theoretical
results of this work are supported by numerical simulations.

Keywords: Non-parametric Regression, Jacobi polynomials, generalized
polynomials chaos, ANOVA decomposition, least squares, stable regression
estimator, risk error.

##  1 Introduction

In this work, we combine the popular technique of generalized polynomials
chaos (gPC) [39, 43] and a special family of $d-$variate Jacobi polynomials in
order to solve a $d-$dimensional non-parametric regression problem. This last
problem is one of the important as well as an active research topic from the
machine learning area, see for example [39, 40]. Note that a machine learning
algorithm can be briefly described as an algorithm for the approximation of an
unknown function $f$ that maps in general a random vectors
$X\in\mathbb{R}^{d}$ to an observed real valued variable $Y.$ An estimator or
an approximation $\widehat{f}$ of $f$ is constructed by the use a training
data set $\\{(X_{i},Y_{i}),\,1\leq i\leq n\\}.$ Note that unlike a parametric
learning algorithm, where $\widehat{f}$ is given in terms of a set of fixed
size of parameters, a non-parametric learning algorithm does not require any
assumption about the function $f$ or its estimator $\widehat{f}.$ Usually, for
a non-parametric (NP) model, the function $f$ lies in an infinite dimensional
functional space. Consequently, the NP models have the advantage to better fit
a wide range of the true functions $f$. We should mention that the
multidimensional NP regression problem is frequently encountered in a wide
range of scientific fields. An NP learning algorithm for solving this problem
aims to provide a convenient estimate $\widehat{f}$ for the true regression
function $f$ associated with the regression problem,

$Y_{i}=f(X_{i})+\varepsilon_{i},\ i=1,\ldots,n\,.$ (1)

Here, the $X_{i}\in\mathbb{R}^{d}$ are assumed to be i.i.d. random vectors and
the $(\varepsilon_{i})_{1\leq i\leq n}$ are centered i.i.d. random variables
with a finite variance $\sigma^{2}=\mathbb{E}[\varepsilon_{1}^{2}]$ and
independent from the $X_{i}.$ The goal of a learning algorithm is to minimize
the empirical risk $\mathcal{R}_{emp}(f)$ over a given class of functional
space $\mathcal{H}.$ That is to solve the minimization problem

$\widehat{f}=\arg\min_{f\in\mathcal{H}}\mathcal{R}_{emp}(f)=\arg\min_{f\in\mathcal{H}}\frac{1}{n}\sum_{i=1}^{n}L(f(X_{i}),Y_{i}),$
(2)

where, $L(\cdot,\cdot)$ is a non-negative loss function. Among the frequently
used loss functions from the literature, we cite the Tikhonov regularized loss
[35] and the weighted $\ell_{2}$ loss function [33], given respectively by

$L_{\lambda}(F(X_{i}),Y_{i})=\big{(}f(X_{i})-Y_{i}\big{)}^{2}+\lambda\|f\|^{2}_{\mathcal{H}},\qquad
L_{\omega}(f(X_{i}),Y_{i})=\omega_{i}\big{(}f(X_{i})-Y_{i}\big{)}^{2},$

for some convenient regularization parameter $\lambda>0$ and finite weight
sequence $(\omega_{i})_{i}.$ In general, the least squares scheme is used to
solve the minimization problem (2).

It is well known that despite the superiority of the NP model in terms of
quality of approximation of the true functional $f$, it suffers from the curse
of dimensionality for large or even moderate values of $d,$ the number of
covariates. The computational load by an NP algorithm grows fast with the
dimension $d$ and its convergence rate or its associated risk error rate slows
drastically. For instance, it has been shown in [37], see also [5, 15] that if
$f$ is of class $C^{p}$ with $p-$th derivative being Hölder continuous, then
the optimal convergence rate of any NP least-squares estimator
$\widehat{f}_{n},$ solution of the minimization problem (2) is given by
${\displaystyle\mathbb{E}\Big{[}\|f-\widehat{f}_{n}\|_{\mathcal{H}}^{2}\Big{]}=O\Big{(}n^{-2p/(2p+d)}\Big{)}.}$
In order to reduce the computational load required by an algorithm for solving
more general models with random inputs or models from uncertainty
quantification (UQ) area, a popular technique of polynomial chaos expansion
(PCE) is successfully used in the literature. This technique aims to
approximate the output variable $Y$ by using a projection over a reduced size
of orthogonal polynomial basis. The PCE scheme has been first introduced by N.
Wiener in his pioneer work [42], for the Hermite polynomials and Gaussian
random variables. Recently, there is a growing interest in the study and the
use for UQ applications of a generalized version of PCE, called generalized
polynomial chaos (gPC). The gPC was first introduced by [43] and it aims to
extend the PCE to various discrete and continuous probability distributions
associated with the set of weight functions for the family of orthogonal
polynomials of the Askey–scheme. For more details on PCE and gPC schemes and
their associated UQ related applications, the reader is refereed to [16, 17,
21, 22, 26, 40, 44]. Nonetheless, the gPC based learning algorithm still has
the limitation to be slow for moderate large values of the dimension $d$. To
overcome this problem, various solutions have been considered in the
literature. Among the popular adopted solutions, we cite the use of sparsity
and optimal sampling techniques, see for example [7, 9, 14, 18, 22, 24, 29,
34], dimension reduction through a sensitivity analysis techniques, [2, 8], as
well as the use of partial functional ANOVA decomposition technique, see for
example [20, 25, 30, 32, 36, 38]. More precisely, a partial functional ANOVA
decomposition consists in the approximation of a real valued function
$f(x_{1},\cdots,x_{d})$ by a sum of functions with reduced number of variables
of the form $f_{i}(x_{i_{1}},\ldots,x_{i_{m}})$ where $1\leq m\leq d$ and the
$i_{j}\in[[1,d]].$ Note that for the full ANOVA decomposition, that is $m=d,$
the uniqueness of the ANOVA decomposition of a function $f\in
L^{1}(J^{d}),\,J=[0,1]$ has been shown in [36]. Also, among the popular
reduced size multivariate polynomials spaces used by a gPC scheme, we cite the
total degree space of degree $N,$ given by
$\mathcal{P}_{N}^{TD}=\mbox{Span}\\{\boldsymbol{x}^{\boldsymbol{i}}=x_{1}^{i_{1}}\cdots
x_{d}^{i_{d}},\quad\|\boldsymbol{i}\|_{1}=\sum_{j=1}^{d}|i_{j}|\leq N\\}$ and
the hyperbolic cross space of degree $N,$ given by
$\mathcal{P}_{q,N}^{HC}=\mbox{Span}\\{\boldsymbol{x}^{\boldsymbol{i}}=x_{1}^{i_{1}}\cdots
x_{d}^{i_{d}},\quad\|\boldsymbol{i}\|_{q}=\Big{(}\sum_{j=1}^{d}|i_{j}|^{q}\Big{)}^{1/q}\leq
N\\},$ $0<q<1.$ For more details on these polynomials spaces, the reader is
refereed to [7].

In this work, we introduce a new $d-$variate polynomial space constructed from
uni-variate Jacobi polynomials associated with parameters
$\beta=\alpha\geq-\frac{1}{2}$ and orthonormal over$I^{d}=[-1,1]^{d}.$ More
precisely, for two positive integers $N\geq 1$ and $1\leq m\leq\min(d,N),$ we
let the ANOVA type Jacobi polynomials space

$\mathcal{P}_{N,m,d}=Span\left\\{\prod_{i\in\boldsymbol{u}}\widetilde{P}_{k_{i}}^{(\alpha)}(x_{i})\
;\ \boldsymbol{u}\subset[[1,d]]\ ;\ \ |\boldsymbol{u}|\leq m;\
\boldsymbol{k}\in\mathbb{N}_{0}^{d},\ \ ||\boldsymbol{k}||_{1}\leq
N\right\\}\,.$ (3)

Here, the $\widetilde{P}_{k_{i}}^{(\alpha)}$ are the orthonormal uni-variate
Jacobi polynomials , associated with a parameter $\alpha\geq-\frac{1}{2}.$ The
dimension of $\mathcal{P}_{N,m,d}$ is given by ${\displaystyle
M_{N,m,d}=\dim\mathcal{P}_{N,m,d}=\sum_{k=0}^{m}{d\choose k}{N\choose k}.}$
For the extreme case $m=\min(d,N),$ the space $\mathcal{P}_{N,m,d}$ is reduced
to the total degree space polynomials of degree $N,$ given by
$\mathcal{P}_{N}^{TD}.$ One of the main results of this work is to prove the
stability of our proposed least-squares estimator
$\widehat{f}_{N,n,m}^{(\alpha)}$ which is the solution of the minimization
problem (2) with the functional space $\mathcal{H}=\mathcal{P}_{N,m,d}$ and a
loss function $L$ given by the Euclidean distance of $\mathbb{R}^{n}.$ Note
that there is a growing interest in the study of the stability issue of
estimators of functions with random inputs, see for example [1, 11, 12, 27,
28]. In the present work, we show that under the condition that the $X_{i}$
follow a $d-$variate $\mbox{Beta}(\alpha+1,\alpha+1)$ distribution with
support $I^{d},$ the positive definite $n\times n$ random matrix involved in
the construction of our proposed least-squares based estimator
$\widehat{f}_{N,n,m}^{(\alpha)}$ is with high probability well conditioned in
the $2-$norm. Moreover, we give an estimate for the $L^{2}-$risk error of a
truncated version of the $\widehat{f}_{N,n,m}^{(\alpha)}$ which we denote by
$\widehat{F}_{N,n,m}^{(\alpha)}.$ More precisely, if $\|\cdot\|_{\alpha}$
denotes the weighted $L^{2}-$norm associated with the weight
$\boldsymbol{\omega}_{\alpha}=\prod_{i=1}^{d}(1-x_{i}^{2})^{\alpha},$ then we
give an estimate of the $L^{2}-$risk error
$\mathbb{E}\Big{[}\|f-\widehat{F}_{N,n,m}^{(\alpha)}\|^{2}_{\alpha}\Big{]}.$
This later is given in terms of the classical bias-variance decomposition. The
variance term decays at a rate of $O\big{(}\frac{M_{N,m,d}}{n}\big{)}.$ Here,
$M_{N,m,d}$ is the dimension of our proposed polynomial space
$\mathcal{P}_{N,m,d}.$ The bias term f the $L^{2}-$risk involves the quantity
$\|f-\Pi_{N,m}f\|_{\alpha},$ where $\Pi_{N,m}$ denotes the orthogonal
projection over $\mathcal{P}_{N,m,d}.$ An estimate of this last quantity is
given under the hypothesis that the true regression function lies in a
weighted Sobolev space with given Sobolev smoothness property.

This work is organized as follows. In section 2, we give some mathematical
preliminaries on Matrix Chernoff eigenvalues bounds, the Gershgorin circle
theorem for bounding the spectrum of a square matrix, as well as some
properties of the Jacobi polynomials. In particular, we give some bounds of
these polynomials that will be used for proving different results of this
work. In section 3, we describe the new adopted and reduced size ANOVA type
multivariate Jacobi polynomial space $\mathcal{P}_{N,m,d}.$ Moreover, we
provide the reader with an estimate of the dimension of this later. Section 4
of this work is devoted to the proof of the stability of the proposed least-
squares and gPC based NP regression estimator
$\widehat{f}_{N,n,m}^{(\alpha)}.$ In section 5, we give an estimate for the
weighted $L^{2}-$risk error of a truncation version of the estimator
$\widehat{f}_{N,n,m}^{(\alpha)}.$ Moreover, in section 6, we give an estimate
for the bias term of the previous weighted $L^{2}-$risk error, when the true
regression function belongs to some weighted Sobolev space. Finally, in
section 7, we give some numerical simulations that illustrate the different
results of this work.

##  2 Mathematical Preliminaries and estimates for Jacobi polynomials

In this paragraph, we first give some mathematical preliminaries from the
literature that will be used frequently in this work. Then, we give some
useful estimates for the Jacobi polynomials. These estimates are needed for
the proof of the stability property of our proposed special Jacobi polynomials
multivariate non-parametric (NP) regression estimator.

###  2.1 Mathematical preliminaries

We first recall the Matrix Chernoff Theorem (see for example [41]) and the
Gershgorin circle Theorem (see for example [19]) that will be useful for
proving the stability of our NP regression estimator.

Matrix Chernoff Theorem: Consider a sequence of $n$ independent $D\times D$
random Hermitian matrices $\\{\boldsymbol{Z}_{k}\\}$. Assume that for some
$L>0$, we have

$0\preccurlyeq\boldsymbol{Z}_{k}\preccurlyeq L.\boldsymbol{I}_{D}\,.$

Let

$\boldsymbol{A}=\sum_{k=1}^{n}\boldsymbol{Z}_{k},\
\mu_{\min}=\lambda_{\min}(\mathbb{E}(\boldsymbol{A})).\
\mu_{\max}=\lambda_{\max}(\mathbb{E}(\boldsymbol{A}))\,.$

Then, for any $\delta\in(0,1]$, we have

$\mathbb{P}\left(\lambda_{\min}(\boldsymbol{A}\leq(1-\delta)\mu_{\min}\right)\leq
D.\exp\left(-\frac{\delta^{2}\mu_{\min}}{2L}\right),\
\mathbb{P}\left(\lambda_{\max}(\boldsymbol{A}\geq(1+\delta)\mu_{\max}\right)\leq
D.\exp\left(-\frac{\delta^{2}\mu_{\max}}{3L}\right)$ (4)

Gershgorin circle Theorem: Let $A=[a_{i,j}]_{1\leq i,j\leq n}$ be a complex
matrix. For $1\leq i\leq n$, let $R_{i}=\sum_{j\neq i}|a_{i,j}|$. Then every
eigenvalue of $A$ lies within at least one of the discs $D(a_{ii},R_{i})$.

In the sequel, we let $\Gamma(a)$ and $\mbox{B}(a,b)$ respectively denote the
usual Gamma and Beta functions with $a,b>0$.
For an integer $k\geq 0$ and $\alpha\geq-\frac{1}{2}$, let
$\widetilde{P}_{k}^{(\alpha,\alpha)}$ denote the normalized Jacobi polynomial
defined on $I=[-1,1]$ of degree $k$ and parameters $(\alpha,\alpha)$. We have:

$\widetilde{P}_{k}^{(\alpha,\alpha)}(x)=\frac{1}{\sqrt{h_{k}^{(\alpha)}}}P_{k}^{(\alpha)}(x),\
h_{k}^{(\alpha)}=\frac{2^{2\alpha+1}\Gamma^{2}(k+\alpha+1)}{k!(2k+2\alpha+1)\Gamma(k+2\alpha+1)}\,.$
(5)

The polynomials $\widetilde{P}_{k}^{(\alpha,\alpha)},\ k\geq 0$ satisfy the
orthonormality relation

$\int_{I}\widetilde{P}_{j}^{(\alpha,\alpha)}\widetilde{P}_{k}^{(\alpha,\alpha)}w_{(}\alpha)(x)dx=\delta_{j,k},\,\
w_{\alpha}(x)=(1-x^{2})^{\alpha}\,.$

In the sequel, in order to alleviate notations, we will use the notation
$\widetilde{P}_{k}^{(\alpha)}$ instead of
$\widetilde{P}_{k}^{(\alpha,\alpha)}.$

The following Lemma regroups different useful identities and inequalities that
can be easily found in the literature, see for example [3, 4, 23].

######  Lemma 1.

Let $J_{a}$ be the Bessel function of the first kind and order $a>-1$. Then,
we have

  1. 1.

For any $x\in\mathbb{R}$ and for any integer $m\geq 0,$

$\int_{-1}^{1}e^{ixy}\widetilde{P}_{m}^{(\alpha)}(y)w_{\alpha}(y)dy=i^{m}\sqrt{\pi}.\sqrt{2m+2\alpha+1}\sqrt{\frac{\Gamma(m+2\alpha+1)}{\Gamma(m+1)}}\frac{J_{m+\alpha+1/2}(x)}{x^{\alpha+1/2}}\,.$
(6)

  2. 2.

For any $x\in\mathbb{R}$ and any real $\mu>-1$, we have

$|J_{\mu}(x)|\leq\frac{|x|^{\mu}}{2^{\mu}\Gamma(\mu+1)}\,.$ (7)

  3. 3.

For $x>-\frac{1}{2}$,

$\sqrt{2e}\left(\frac{x+\frac{1}{2}}{e}\right)^{x+\frac{1}{2}}\leq\Gamma(x+1)\leq\sqrt{2\pi}\left(\frac{x+\frac{1}{2}}{e}\right)^{x+\frac{1}{2}}\,.$
(8)

###  2.2 Estimates for Jacobi polynomials

In order to provide estimates for Jacobi polynomials, we will need the
following result.

######  Lemma 2.

For $\alpha\geq-\frac{1}{2},$ the function
$h_{0}^{(\alpha)}=2^{2\alpha+1}\text{Beta}(\alpha+1,\alpha+1)$ is bounded as
follows

$\frac{1}{C^{2}(\alpha)}\leq h_{0}^{(\alpha)}\leq 2\pi\,,$ (9)

where
$C(\alpha):=\left[\frac{\pi}{e^{2}}\left(\alpha+\frac{3}{4}\right)\right]^{\frac{1}{4}}$.

###### Proof.

: We first establish the lower bound of $h_{0}^{(\alpha)}$ using (8).

$\displaystyle\frac{1}{h_{0}^{(\alpha)}}$ $\displaystyle=$
$\displaystyle\frac{1}{2^{2\alpha+1}}\frac{\Gamma(2\alpha+2)}{\Gamma^{2}(\alpha+1)}$
$\displaystyle\leq$
$\displaystyle\sqrt{\pi}{e}^{-\frac{3}{2}}\left(\alpha+\frac{3}{4}\right)^{2\alpha+\frac{3}{2}}\left(\frac{1}{\alpha+\frac{1}{2}}\right)^{2\alpha+1}$
$\displaystyle\leq$
$\displaystyle\sqrt{\pi}{e}^{-\frac{3}{2}}\left(\alpha+\frac{3}{4}\right)^{\frac{1}{2}}\left(1+\frac{1}{2}\frac{1}{2\alpha+1}\right)^{2\alpha+1}\leq\sqrt{\pi}{e}^{-1}\left(\alpha+\frac{3}{4}\right)^{\frac{1}{2}}.$

For the upper bound of $h_{0}^{(\alpha)}$, we will use again (8). We get

$\displaystyle
h_{0}^{(\alpha)}=2^{2\alpha+1}\frac{\Gamma^{2}(\alpha+1)}{\Gamma(2\alpha+2)}$
$\displaystyle\leq$ $\displaystyle
2^{2\alpha+2}\pi\left(\frac{\alpha+\frac{1}{2}}{e}\right)^{2\alpha+1}\frac{1}{\sqrt{2e}}\left(\frac{e}{2\alpha+\frac{3}{2}}\right)^{2\alpha+\frac{3}{2}}$
$\displaystyle=$ $\displaystyle
2^{2\alpha+\frac{3}{2}}\pi\left(\frac{\alpha+\frac{1}{2}}{e}\right)^{2\alpha+1}\frac{1}{\sqrt{e}}\left(\frac{e}{2\alpha+\frac{3}{2}}\right)^{2\alpha+\frac{3}{2}}$
$\displaystyle=$
$\displaystyle\pi\left(\frac{1}{\alpha+\frac{3}{4}}\right)^{\frac{1}{2}}\left(\frac{\alpha+\frac{1}{2}}{\alpha+\frac{3}{4}}\right)^{2\alpha+1}\leq
2\pi.$

∎

The following proposition provides us with some useful estimates for the
normalized Jacobi polynomials $\widetilde{P}_{k}^{(\alpha)}.$

######  Proposition 1 (Bounds for
$||\widetilde{P}_{k}^{(\alpha)}||_{\infty}$).

Under the previous notation, let $\alpha\geq-\frac{1}{2},$ then for any
integer $k\geq 1,$ we have

$\big{\|}\widetilde{P}_{k}^{(\alpha)}\big{\|}_{\infty}\leq\left\\{\begin{array}[]{ll}\frac{\pi^{\frac{1}{4}}e^{\alpha+\frac{3}{4}}}{\sqrt{2}}k^{\alpha+\frac{1}{2}}&\mbox{
if }\alpha>-\frac{1}{2}\\\ {}&{}\\\ \frac{2}{\sqrt{\pi}}&\mbox{ if
}\alpha=-\frac{1}{2}.\end{array}\right.$ (10)

Moreover, for $k=0,$

$\big{\|}\widetilde{P}_{0}^{(\alpha)}\big{\|}_{\infty}\leq\
C(\alpha),\quad\alpha\geq-\frac{1}{2}\,,$ (11)

where $C(\alpha)$ is the quantity defined in Lemma 2.

###### Proof.

: Let $k\geq 1$, since

$\frac{1}{\sqrt{h_{k}^{(\alpha)}}}=\frac{1}{2^{\alpha}\Gamma(k+\alpha+1)}\sqrt{\Gamma(k+1)\Gamma(k+2\alpha+1)}\sqrt{k+\alpha+\frac{1}{2}},$

then

$\frac{||P_{k}^{(\alpha)}||_{\infty}}{\sqrt{h_{k}^{(\alpha)}}\sqrt{k+\alpha+\frac{1}{2}}}=\frac{1}{2^{\alpha}\Gamma(\alpha+1)}\sqrt{\frac{\Gamma(k+2\alpha+1)}{\Gamma(k+1)}}\,.$

For $\alpha=-\frac{1}{2}$, one gets

$\|\widetilde{P}_{k}^{(-\frac{1}{2})}||_{\infty}=\frac{||P_{k}^{(-\frac{1}{2})}\|_{\infty}}{\sqrt{h_{k}^{(1/2)}}}=\frac{\sqrt{2}}{\Gamma(1/2)}\sqrt{\frac{\Gamma(k)}{\Gamma(k+1)}}\sqrt{k+1}=\sqrt{\frac{2}{\pi}}\sqrt{1+\frac{1}{k}}\leq\frac{2}{\sqrt{\pi}}.$

Next, for the case $\alpha>-\frac{1}{2}$ and by Using the bounds of the Gamma
function (8), one gets

$\frac{\Gamma(k+2\alpha+1)}{\Gamma(k+1)}\leq\left(\frac{\pi}{e}\right)^{\frac{1}{2}}\left(\frac{k}{e}\right)^{2\alpha}\left(1+\frac{2\alpha}{k+\frac{1}{2}}\right)^{k+\frac{1}{2}}\left(1+\frac{2\alpha+\frac{1}{2}}{k}\right)^{2\alpha}\,.$

Thus,

$\frac{||P_{k}^{(\alpha)}||_{\infty}}{\sqrt{h_{k}^{(\alpha)}}\sqrt{k+\alpha+\frac{1}{2}}}\leq\frac{1}{2^{\alpha}\Gamma(\alpha+1)}\left(\frac{\pi}{e}\right)^{\frac{1}{4}}\left(\frac{k}{e}\right)^{\alpha}\left(1+\frac{2\alpha}{k+\frac{1}{2}}\right)^{\frac{k+\frac{1}{2}}{2}}\left(1+\frac{2\alpha+\frac{1}{2}}{k}\right)^{\alpha}\,.$

Since
$\left(1+\frac{2\alpha}{k+\frac{1}{2}}\right)^{\frac{k+\frac{1}{2}}{2}}\leq
e^{\alpha}$, then we get

$\frac{||P_{k}^{(\alpha)}||_{\infty}}{\sqrt{h_{k}^{(\alpha)}}}\leq\frac{k^{\alpha+\frac{1}{2}}}{2^{\alpha}\Gamma(\alpha+1)}\left(\frac{\pi}{e}\right)^{\frac{1}{4}}\left(2\alpha+\frac{3}{2}\right)^{\alpha}\sqrt{\alpha+\frac{3}{2}}=\frac{k^{\alpha+\frac{1}{2}}}{\Gamma(\alpha+1)}\left(\frac{\pi}{e}\right)^{\frac{1}{4}}\left(\alpha+\frac{3}{4}\right)^{\alpha}\sqrt{\alpha+\frac{3}{2}}\,.$

Using again (8), we get
$\frac{1}{\Gamma(\alpha+1)}\leq\frac{1}{\sqrt{2e}}\left(\frac{e}{\alpha+\frac{1}{2}}\right)^{\alpha+\frac{1}{2}}.$
Consequently, one gets

$\frac{\|P_{k}^{(\alpha)}\|_{\infty}}{\sqrt{h_{k}^{(\alpha)}}}\leq\frac{\pi^{\frac{1}{4}}k^{\alpha+\frac{1}{2}}e^{\alpha-\frac{1}{4}}}{\sqrt{2}}\left(1+\frac{1}{\alpha+\frac{1}{2}}\right)^{\alpha+\frac{1}{2}}\leq\frac{\pi^{\frac{1}{4}}e^{\alpha+\frac{3}{4}}}{\sqrt{2}}k^{\alpha+\frac{1}{2}}.$

Finally, for $k=0$, we have
$\|\widetilde{P}_{0}^{(\alpha)}\|_{\infty}^{2}=\frac{1}{h_{0}^{(\alpha)}}$
which is, according to Lemma 2, upper bounded by $C^{2}(\alpha)$.

######  Corollary 1.

Under the same hypothesis and notations of the previous proposition, for any
$\alpha\geq-\frac{1}{2}$ and for any integer $N\geq 1,$ we have

$\sum_{k=0}^{N}\|\widetilde{P}_{k}^{(\alpha)}\|_{\infty}\leq\eta_{\alpha}\frac{\left(N+1\right)^{\alpha+\frac{3}{2}}}{\alpha+\frac{3}{2}},\qquad\eta_{\alpha}=\frac{\pi^{\frac{1}{4}}e^{\alpha+\frac{3}{4}}}{\sqrt{2}}.$
(12)

###### Proof.

From the previous proposition, we can write

$\displaystyle\sum_{k=0}^{N}\|\widetilde{P}_{k}^{(\alpha)}\|_{\infty}$
$\displaystyle=\|\widetilde{P}_{0}^{(\alpha)}\|_{\infty}+\sum_{k=1}^{N}\|\widetilde{P}_{k}^{(\alpha)}\|_{\infty}\leq\frac{\eta_{\alpha}}{\alpha+\frac{3}{2}}+\sum_{k=1}^{N}\eta_{\alpha}k^{\alpha+\frac{1}{2}}$
(13)
$\displaystyle\leq\eta_{\alpha}\int_{0}^{N+1}x^{\alpha+\frac{1}{2}}dx=\eta_{\alpha}\frac{\left(N+1\right)^{\alpha+\frac{3}{2}}}{\alpha+\frac{3}{2}}\,.$

∎

Let $L^{2}(I,w_{\alpha})$ be the Hilbert space associated to the inner product
$<f,g>_{\alpha}=\int_{I}f\cdot g\cdot w_{\alpha}(x)\,dx.$ Note that the family
$\left\\{\widetilde{P}_{k}^{(\alpha)},k\geq 0\right\\}$ is an orthonormal
basis of $L^{2}(I,w_{\alpha}).$

##  3 An ANOVA type space based on multivariate Jacobi polynoimals

In this paragraph, we describe a reduced size multidimensional polynomials
space. The construction of this space is based on combining the ANOVA
decomposition technique see for example [20, 25, 30, 32, 36, 38] and the total
degree polynomial space, see for example [7]. For this purpose, let
$\mathcal{D}=[[1,d]]=\\{1,2,\ldots,d\\}$ and we will adopt the notations
$\boldsymbol{u}\subset\mathcal{D}$ for subsets of $\mathcal{D},$
$x_{\boldsymbol{u}}=(x_{i})_{i\in\boldsymbol{u}}$ and $|\boldsymbol{u}|$ for
the length of the vector $\boldsymbol{u}$. For a given
$\boldsymbol{u}\subset\mathcal{D}$, we let $F_{\boldsymbol{u}}$ denote the
subset of $\mathbb{Z}^{d}$ defined by

$F_{\boldsymbol{u}}=\left\\{\boldsymbol{k}\in\mathbb{Z}^{d}/k_{\boldsymbol{u}^{c}}=0_{\mathbb{Z}^{d-|u|}}\,;\
k_{j}\neq 0\ \forall j\,\in\boldsymbol{u}\right\\}\,.$

First, we describe a Jacobi polynomials orthonormal basis of
$L^{2}(I^{d},\boldsymbol{w}_{\alpha})$ based on the ANOVA decomposition. Here,
the $d-$variate weight function $\boldsymbol{w}$ is defined on $I^{d}$ by

$\forall\ \boldsymbol{x}=(x_{1},\ldots,x_{d})\in I^{d},\
\boldsymbol{w}_{\alpha}(\boldsymbol{x})=\prod_{i=1}^{d}w_{\alpha}(x_{i})=\left(\prod_{i=1}^{d}\left(1-x_{i}^{2}\right)\right)^{\alpha}\,.$

The usual inner product associated with $L^{2}(I^{d},\boldsymbol{w}_{\alpha})$
is defined by

$<f,g>_{\boldsymbol{w}}=\int_{[-1,1]^{d}}f(\boldsymbol{x})\cdot
g(\boldsymbol{x})\,\boldsymbol{w}_{\alpha}(\boldsymbol{x})\,d\boldsymbol{x}.$

Let
$\Phi_{\boldsymbol{0}}^{(\alpha)}(\boldsymbol{x})=\frac{1}{\sqrt{h_{0}^{(\alpha)}}^{d}}$
and for $\boldsymbol{u}\subset\mathcal{D}$ such that $|\boldsymbol{u}|\geq 1$
and $\boldsymbol{k}\in F_{\boldsymbol{u}}$, let

$\Phi_{\boldsymbol{u,k}}^{(\alpha)}(\boldsymbol{x}):=\frac{1}{\sqrt{h_{0}^{(\alpha)}}^{d-|\boldsymbol{u}|}}\prod_{i\in\boldsymbol{u}}\widetilde{P}_{k_{i}}^{(\alpha)}(x_{i})\,.$
(14)

######  Lemma 3.

The family

$\left\\{\Phi_{\boldsymbol{0}}^{(\alpha)}\right\\}\cup\left\\{\Phi_{\boldsymbol{u,k}}^{(\alpha)},\
\boldsymbol{u}\subset\mathcal{D},\ \ \boldsymbol{k}\in
F_{\boldsymbol{u}}\right\\}$

is an orthonormal basis of $L^{2}(I^{d},\boldsymbol{w}_{\alpha})$.

###### Proof.

Since

$\mbox{Span}\left\\{\Phi_{\boldsymbol{0}}^{(\alpha)}\right\\}\cup\left\\{\Phi_{\boldsymbol{u,k}}^{(\alpha)},\
\boldsymbol{u}\subset\mathcal{D},\ \ \boldsymbol{k}\in
F_{\boldsymbol{u}}\right\\}=\mbox{Span}\left\\{\prod_{i=1}^{d}P^{(\alpha)}_{k_{i}},\,k_{i}\geq
0\right\\}$

and since this later is dense in $L^{2}(I^{d},\boldsymbol{w}_{\alpha}),$ then
it suffices to establish the orthornormality of the vectors
$\Phi_{\boldsymbol{0}}^{(\alpha)}$ and $\Phi_{\boldsymbol{u,k}}^{(\alpha)}$,
we consider the following three cases.

  1. 1.

Computation of
$<\Phi_{\boldsymbol{u,k}}^{(\alpha)},\Phi_{\boldsymbol{u,k}}^{(\alpha)}>_{\alpha}$.
We first assume that $|\boldsymbol{u}|\geq 1$, then we have

$\displaystyle<\Phi_{\boldsymbol{u,k}}^{(\alpha)},\Phi_{\boldsymbol{u,k}}^{(\alpha)}>_{\alpha}$
$\displaystyle=\int_{[-1,1]^{d}}\Phi_{\boldsymbol{u,k}}^{(\alpha)}(\boldsymbol{x})\prod_{i=1}^{d}w_{\alpha}(x_{i})dx_{1}\ldots
dx_{d}$ (15)
$\displaystyle=\int_{[-1,1]^{d}}\left(\widetilde{P}_{0}^{(\alpha)}\right)^{2d-2|\boldsymbol{u}|}\prod_{i\in\boldsymbol{u}}\left(\widetilde{P}_{k_{i}}^{(\alpha)}(x_{i})\right)^{2}\prod_{i=1}^{d}w_{\alpha}(x_{i})dx_{1}\ldots
dx_{d}$
$\displaystyle=\left(\prod_{i\in\boldsymbol{u}}\int_{-1}^{1}\left(\widetilde{P}_{k_{i}}^{(\alpha)}(x_{i})\right)^{2}w_{\alpha}(x_{i})dx_{i}\right)\left(\prod_{1\leq
j\leq
d;j\notin\boldsymbol{u}}\int_{-1}^{1}\left(\widetilde{P}_{0}^{(\alpha)}(x_{j})\right)^{2}w_{\alpha}(x_{j})dx_{j}\right)=1\,.$

In a similar manner, we have
$<\Phi_{\boldsymbol{0}}^{(\alpha)},\Phi_{\boldsymbol{0}}^{(\alpha)}>_{\alpha}=1$.

  2. 2.

Computation of
$<\Phi_{\boldsymbol{u,k}}^{(\alpha)},\Phi_{\boldsymbol{u,l}}^{(\alpha)}>_{\alpha}$
in the case where $|\boldsymbol{u}|\geq 1$ and
$\boldsymbol{k}\neq\boldsymbol{l}$
As $\boldsymbol{k}\neq\boldsymbol{l}$ then $\exists\ i_{0}\in\boldsymbol{u}$
such that $\boldsymbol{k}_{i_{0}}\neq\boldsymbol{l}_{i_{0}}$ .

$\displaystyle<\Phi_{\boldsymbol{u,k}}^{(\alpha)},\Phi_{\boldsymbol{u,l}}^{(\alpha)}>_{\alpha}$
$\displaystyle=\int_{[-1,1]^{d}}\Phi_{\boldsymbol{u,k}}^{(\alpha)}(\boldsymbol{x})\Phi_{\boldsymbol{u,l}}^{(\alpha)}(\boldsymbol{x})\boldsymbol{w}_{\alpha}(\boldsymbol{x})d\boldsymbol{x}$
(16)
$\displaystyle=\left(\frac{1}{h_{0}^{\alpha}}\right)^{d-|\boldsymbol{u}|}\left(\int_{-1}^{1}\widetilde{P}_{k_{i_{0}}}^{(\alpha)}(x_{i_{0}})\widetilde{P}_{l_{i_{0}}}^{(\alpha)}(x_{i_{0}})w_{\alpha}(x_{i_{0}})dx_{i_{0}}\right)\prod_{j\neq
i_{0};j\in\boldsymbol{u}}\left(\int_{-1}^{1}\widetilde{P}_{k_{j}}^{(\alpha)}(x_{j})\widetilde{P}_{l_{j}}^{(\alpha)}(x_{j})w_{\alpha}(x_{j})dx_{j}\right)$
$\displaystyle=0$

  3. 3.

Computation of
$<\Phi_{\boldsymbol{u,k}}^{(\alpha)},\Phi_{\boldsymbol{v,l}}^{(\alpha)}>_{\alpha}$.
Let us consider the case where $\boldsymbol{u}\neq\boldsymbol{v}$,
$|\boldsymbol{u}|\geq 1$ and $|\boldsymbol{v}|\geq 1$
As $\boldsymbol{u}\neq\boldsymbol{v}$ then : (1)
$\boldsymbol{k}\neq\boldsymbol{l}$ and (2) $\exists\
i_{0}\in\boldsymbol{u}\backslash\boldsymbol{v}$ (or $\boldsymbol{v}\backslash
v$). Without loss of generality, we will suppose that
$i_{0}\in\boldsymbol{u}\backslash\boldsymbol{v}$.

$\displaystyle<\Phi_{\boldsymbol{u,k}}^{(\alpha)},\Phi_{\boldsymbol{v,l}}^{(\alpha)}>_{\alpha}$
(17)
$\displaystyle=\int_{[-1,1]^{d}}\Phi_{\boldsymbol{u,k}}^{(\alpha)}(\boldsymbol{x})\Phi_{\boldsymbol{v,l}}^{(\alpha)}(\boldsymbol{x})\boldsymbol{w}_{\alpha}(\boldsymbol{x})d\boldsymbol{x}$
$\displaystyle=\int_{[-1,1]^{d}}\left(\prod_{i\in\boldsymbol{u}}\widetilde{P}_{k_{i}}^{(\alpha)}(x_{i})\right)\left(\frac{1}{h_{0}^{\alpha}}\right)^{\frac{d-|\boldsymbol{u}|}{2}}\left(\prod_{j\in\boldsymbol{v}}\widetilde{P}_{l_{j}}^{(\alpha)}(x_{j})\left(\frac{1}{h_{0}^{\alpha}}\right)^{\frac{d-|\boldsymbol{v}|}{2}}\right)\prod_{s=1}^{d}w_{\alpha}(x_{s})dx_{s}$
$\displaystyle=\left(\int_{-1}^{1}\widetilde{P}_{k_{i_{0}}}^{(\alpha)}(x_{i_{0}})\frac{w_{\alpha}(x_{i_{0}})}{\sqrt{h_{0}^{(\alpha)}}}dx_{i_{0}}\right)\int_{[-1,1]^{d-1}}\frac{\left(\prod_{i\in\boldsymbol{u}\backslash
i_{0};j\in\boldsymbol{v}}\widetilde{P}_{k_{i}}^{(\alpha)}(x_{i})\widetilde{P}_{l_{j}}^{(\alpha)}(x_{j})\right)}{\sqrt{h_{0}^{(\alpha)}}^{2d-|\boldsymbol{u}|-|\boldsymbol{v}|-1}}\prod_{s\in[[1,d]]\backslash
i_{0}}w_{\alpha}(x_{s})dx_{s}=0$

Similarly to (17), we have in the case where $|\boldsymbol{v}|=0$
$<\Phi_{\boldsymbol{u,k}}^{(\alpha)},\Phi_{\boldsymbol{0}}^{(\alpha)}>_{\alpha}=0$.

∎

Next, we consider the following reduced size polynomial space, defined for
$m\in[[1,N]]$ on which we will construct our estimator.

$\mathcal{P}_{N,m,d}=Span\left\\{\prod_{i\in\boldsymbol{u}}\widetilde{P_{k_{i}}}^{(\alpha)}(x_{i})\
;\ \boldsymbol{u}\subset\mathcal{D}\ ;\ \ |\boldsymbol{u}|\leq m;\
\boldsymbol{k}\in F_{\boldsymbol{u}},\ \ ||\boldsymbol{k}||_{1}\leq
N\right\\}\,.$ (18)

We will call this space ANOVA type polynomial space. The dimension of the
space $\mathcal{P}_{N,m,d}$ as well as an estimate of its upper bound are
given by the following proposition.

######  Proposition 2.

For any positive integers $1\leq m\leq\min(d,N),$ we have

$M_{N,m,d}=\dim\mathcal{P}_{N,m,d}=\sum_{k=0}^{m}{d\choose k}{N\choose k}.$
(19)

Moreover, we have

$M_{N,m,d}\leq\left\\{\begin{array}[]{ll}\frac{\pi}{e}\Big{(}1+\frac{m}{\min(d,N)-m+\frac{1}{2}}\Big{)}^{2\big{(}\max(d,N)-m+\frac{1}{2}\big{)}}\cdot\left(\frac{\max(d,N)+\frac{1}{2}}{m+\frac{1}{2}}\right)^{2m}&\mbox{
if }m\leq\frac{1}{2}\min(d,N),\\\ &\\\
2^{\min(d,N)}\Big{(}1+\frac{m}{\max(d,N)-m}\Big{)}^{\max(d,N)-m}\Big{(}\frac{\max(d,N)}{m}\Big{)}^{m}&\mbox{
if }\frac{1}{2}\min(d,N)\leq m<\min(d,N)\end{array}\right.$ (20)

and, if $m=\min(d,N)$,

$M_{N,m,d}={N+d\choose
m}\leq\sqrt{\frac{\pi}{(2d+2N+1)e}}\Big{(}1+\frac{N}{d+\frac{1}{2}}\Big{)}^{d+\frac{1}{2}}\Big{(}1+\frac{d}{N+\frac{1}{2}}\Big{)}^{N+\frac{1}{2}}$
(21)

###### Proof.

We first check (19). For this purpose, we first consider the two special cases
$m=2$ and $m=3,$. Then, we check the previous identity for any
$m\leq\min(d,N).$ For $m=2,$ the space $\mathcal{P}_{N,2,d}$ can be rewritten
as

$\mathcal{P}_{N,2,d}=H_{1}^{N}\oplus H_{2}^{N}\,,$

with

$H_{1}^{N}=Span\\{\widetilde{P}_{l}^{(\alpha)}(x_{i}),\ 0\leq l\leq N,\ \
1\leq i\leq d\\}\,$

and

$H_{2}^{N}=Span\\{\widetilde{P}_{k_{1}}^{(\alpha)}(x_{i})\widetilde{P}_{k_{2}}^{(\alpha)}(x_{j}),\
(k_{1},k_{2})\in[[1,N-1]]^{2},\ 2\leq k_{1}+k_{2}\leq N,\ \ 1\leq i<j\leq
d\\}\,.$

It is not difficult to check that the dimensions of $H_{1}^{N}$ and
$H_{2}^{N}$ are given by

$\dim H_{1}^{N}=1+Nd=\sum_{k=0}^{1}{d\choose k}{N\choose k},\qquad\dim
H_{2}^{N}=\frac{d(d-1)N(N-1)}{4}={d\choose 2}{N\choose 2}.$

Hence,

$\dim\mathcal{P}_{N,2,d}=M_{N,2,d}=\sum_{k=0}^{2}{d\choose k}{N\choose k}.$

In a similar manner; for $m=3,$ we have
$\mathcal{P}_{N,3,d}=\mathcal{P}_{N,2,d}\oplus H_{3}^{N}\,,$ with

$H_{3}^{N}=Span\Big{\\{}\prod_{i=1}^{3}\widetilde{P}_{k_{i}}^{(\alpha)}(x_{j_{i}}),\
(k_{1},k_{2},k_{3})\in[[1,N-2]]^{3},\ 3\leq k_{1}+k_{2}+k_{3}\leq N,\ \ 1\leq
j_{1}<j_{2}<j_{3}\leq d\Big{\\}}\,.$

Note that there exist ${d\choose 3}$ different $3-$tuples
$(x_{j_{1}},x_{j_{2}},x_{j_{3}})$ in $H_{3}^{N}.$ Moreover, for each such a
tuple, there correspond
${\displaystyle\frac{1}{2}\sum_{k=2}^{N-1}k(k-1)={N\choose 3}}$ different
polynomials products
${\displaystyle\prod_{i=1}^{3}\widetilde{P}_{k_{i}},\,3\leq\sum_{i=1}^{3}k_{i}\leq
N.}$ Consequently, we have ${\displaystyle\dim H_{3}^{N}={d\choose 3}{N\choose
3}}$ and

$\dim\mathcal{P}_{N,3,d}=\dim\mathcal{P}_{N,2,d}+\dim
H_{3}^{N}=\sum_{k=0}^{3}{d\choose k}{N\choose k}.$

Continuing in this manner, one gets the recurrence formula

$\dim\mathcal{P}_{N,m,d}=\dim\mathcal{P}_{N,m-1,d}+\dim
H_{m}^{N}=\sum_{k=0}^{m-1}{d\choose k}{N\choose k}+{d\choose m}{N\choose
m}=\sum_{k=0}^{m}{d\choose k}{N\choose k}.$

Next, to prove (20), we proceed as follows. By using (8), one gets for any
integers $n\geq k\geq 0$

$\displaystyle{n\choose k}$ $\displaystyle=$
$\displaystyle\frac{\Gamma(n+1)}{\Gamma(k+1)\Gamma(n-k+1)}\leq\sqrt{\frac{\pi}{2e^{2}}}\frac{\left(\frac{n+\frac{1}{2}}{e}\right)^{n+\frac{1}{2}}}{\left(\frac{k+\frac{1}{2}}{e}\right)^{k+\frac{1}{2}}\left(\frac{n-k+\frac{1}{2}}{e}\right)^{n-k+\frac{1}{2}}}$
(22) $\displaystyle\leq$
$\displaystyle\sqrt{\frac{\pi}{2e}}\frac{1}{\sqrt{k+\frac{1}{2}}}\left(1+\frac{k}{n-k+\frac{1}{2}}\right)^{n-k+\frac{1}{2}}\left(\frac{n+\frac{1}{2}}{k+\frac{1}{2}}\right)^{k}\,.$

Since for $1\leq m\leq\frac{1}{2}\min(d,N),$ the finite sequences
$\Big{\\{}{d\choose i}\Big{\\}}_{0\leq i\leq m},\Big{\\{}{N\choose
i}\Big{\\}}_{0\leq i\leq m}$ are increasing, then by using (22), one gets

$\displaystyle M_{N,m,d}$ $\displaystyle\leq$ $\displaystyle(m+1){d\choose
m}{N\choose
m}=(m+1)\,\frac{\Gamma(d+1)\cdot\Gamma(N+1)}{\Gamma(m+1)\Gamma\left(d-m+\frac{1}{2}\right)\cdot\Gamma(m+1)\Gamma\left(N-m+\frac{1}{2}\right)}$
$\displaystyle\leq$
$\displaystyle\frac{\pi}{e}\Big{(}1+\frac{m}{\min(d,N)-m+\frac{1}{2}}\Big{)}^{2\big{(}\max(d,N)-m+\frac{1}{2}\big{)}}\cdot\left(\frac{\max(d,N)+\frac{1}{2}}{m+\frac{1}{2}}\right)^{2m}\,.$

In a similar manner, for $\frac{1}{2}\min(d,N)\leq m<\min(d,N),$ we have

$\displaystyle M_{N,m,d}$ $\displaystyle=$
$\displaystyle\sum_{k=0}^{m}{d\choose k}{N\choose
k}\leq\sum_{k=0}^{\min(d,N)}{\min(d,N)\choose
k}\cdot\sum_{k=0}^{m}{\max(d,N)\choose k}$ $\displaystyle\leq$ $\displaystyle
2^{\min(d,N)}\frac{\max(d,N)^{\max(d,N)}}{m^{m}((\max(d,N)-m)^{\max(d,N)-m}}.$

This last inequality is a consequence of the inequality,

$\sum_{k=0}^{m}{n\choose
k}\leq\frac{n^{n}}{m^{m}(n-m)^{n-m}},\qquad\forall\,\,0\leq m<n.$

The previous inequality is a direct consequence of the following upper bound
for the binomial coefficient, see [[13], p.353]

${n\choose k}\leq{\displaystyle 2^{nH\big{(}k/nbig)}},$

where for $0<p<1,$ $H(p)$ is the binary entropy function, given by
${\displaystyle H(p)=-p\log_{2}(p)-(1-p)\log_{2}(1-p).}$ For the last case
where $m=\min(d,N)$, using Chu-Vandermonde’s identity, see for example [31],
we get

$\sum_{k=0}^{m}{d\choose k}{N\choose
k}=\sum_{k=0}^{\min(d,N)}{\min(d,N)\choose k}{\max(d,N)\choose
k}=\sum_{k=0}^{\min(d,N)}{\min(d,N)\choose\min(d,N)-k}{\max(d,N)\choose
k}={N+d\choose m}.$

Using 8, we get

${N+d\choose
m}=\frac{\Gamma(N+d+1)}{\Gamma(d+1)\Gamma(N+1)}\leq\sqrt{\frac{\pi}{2e}}\frac{1}{\sqrt{N+d+\frac{1}{2}}}\left(\frac{N+d+\frac{1}{2}}{d+\frac{1}{2}}\right)^{d+\frac{1}{2}}\left(\frac{N+d+\frac{1}{2}}{N+\frac{1}{2}}\right)^{N+\frac{1}{2}}\,.$
(23)

######  Remark 1.

For the particular case $m=d$, the space $\mathcal{P}_{N,d,d}$ is the usual
total degree polynomial of order $N.$ From Proposition 2, we have
$M_{N,m,d}={N+d\choose d}$ which recovers the well known dimension of the
total degree polynomial space.

##  4 Stability of the NP estimator

In this section, we first describe our proposed Least squares NP regression
estimator based on the use of the orthonormal $d-$variate polynomials
$\Phi_{\boldsymbol{u,k}}^{(\alpha)}.$ Then, we prove the stability of the
proposed estimator under the condition that the random sampling covariates
follow a $d-$dimensional Beta distribution.

Recall that the regression system at hand is given by

$Y_{i}=f(X_{i})+\varepsilon_{i},\ i=1,\ldots,n\,.$ (24)

Here, $f$ is the regression function to be approximated, the
$X_{i}\in\mathbb{R}^{d}$ are the covariates which we assume to be i.i.d.
random vectors and the $(\varepsilon_{i})_{1\leq i\leq n}$ are centered i.i.d.
random variables with a finite variance
$\sigma^{2}=\mathbb{E}[\varepsilon_{1}^{2}]$. Also we assume that the $X_{i}$
are independent from the $\varepsilon_{j}.$ In the sequel, we assume that for
a real $\alpha\geq-\frac{1}{2},$ the set $\\{X_{i},\,i=1,\ldots,n\\}$ is a
random sampling set with the $X_{i}$ following a $d-$variates
$\mbox{Beta}(\alpha+1,\alpha+1)-$distribution. For two positive integers
$m\leq d$ and $N$, we build an estimator $\widehat{f}_{N,n,m}^{(\alpha)}$ of
the regression function $f$ which is obtained using the approximation of $f$
by its projection over $\mathcal{P}_{N,m,d}$. For
$\boldsymbol{u}\subset\mathcal{D}$ with $|u|\geq 1$ and $\boldsymbol{k}\in
F_{\boldsymbol{u}}$, let
$\Phi_{k_{\boldsymbol{u}}}^{(\alpha)}:=\Phi_{\boldsymbol{u,k}}^{(\alpha)}$.
Let $K_{N,0}:=\left\\{(0,\ldots,0)\in\mathbb{R}^{d}\right\\}$ and for
$p\in[[1,m]]$, let $K_{N,p}$ be the subsets defined as follows

$K_{N,p}:=\left\\{k_{\boldsymbol{u}}\in F_{\boldsymbol{u}}:|\boldsymbol{u}|=p\
;\ ||k_{\boldsymbol{u}}||_{1}\leq N\right\\}\,\qquad
K_{N,m,d}:=\bigcup_{p=0}^{m}K_{N,p}.$

Let $g:K_{N,m,d}\mapsto[[1,M_{N,m,d}]]$ be a correspondence (order) defined on
the indexes $(\boldsymbol{k},\boldsymbol{u})$ of our basis :
$g(\boldsymbol{k},\boldsymbol{u})\in[[1,M_{N,m,d}]]$. Then, we can introduce
the notation

$\Psi_{j}^{(\alpha)}:=\Phi_{g^{-1}(j)}^{(\alpha)},\ \mbox{with}\
i\in[[1,M_{N,m,d}]]\,.$

Using these definitions, our NP regression estimator is given by

$\widehat{f}_{N,n,m}^{(\alpha)}(x)=\sum_{k_{\boldsymbol{u}}\in
K_{N,m,d}}\widehat{C}_{k_{\boldsymbol{u}}}(f)\Phi_{k_{\boldsymbol{u}}}(x)=\sum_{j=1}^{M_{N,m,d}}\widehat{C}_{j}(f)\Psi_{j}(x),\quad
x\in I^{d}.$ (25)

Assuming that $\widehat{f}_{N,n,m}^{(\alpha)}$ satisfies
$\widehat{f}_{N,n,m}^{(\alpha)}(X_{i})=Y_{i}$ and multiplying the previous
equation by
$\frac{\left[2^{2\alpha+1}\mbox{B}(\alpha+1,\alpha+1)\right]^{\frac{d}{2}}}{n^{\frac{1}{2}}}$,
we obtain

$\sum_{j=1}^{M_{N,m,d}}\widehat{C}_{j}(f)\frac{\left[2^{2\alpha+1}\mbox{B}(\alpha+1,\alpha+1)\right]^{\frac{d}{2}}}{n^{\frac{1}{2}}}\Psi_{j}(X_{i})=\frac{\left[2^{2\alpha+1}\mbox{B}(\alpha+1,\alpha+1)\right]^{\frac{d}{2}}}{n^{\frac{1}{2}}}Y_{i}.$
(26)

This system can be rewritten as a system of linear equations where the unknown
is the expansion coefficients vector
$\widehat{C}_{N,n,m}^{(\alpha)}=\left(\widehat{C}_{j}\right)_{1\leq j\leq
M_{N,m,d}}^{T}$:

$F_{N,n,m}^{(\alpha)}.\widehat{C}_{N,n,m}^{(\alpha)}=\frac{\left[2^{2\alpha+1}\mbox{B}(\alpha+1,\alpha+1)\right]^{\frac{d}{2}}}{n^{\frac{1}{2}}}Y.$

Consider the positive definite random matrix (to be checked later on),

$G_{N,n,m}^{(\alpha)}=\left(F_{N,n,m}^{(\alpha)}\right)^{T}\cdot
F_{N,n,m}^{(\alpha)}.$ (27)

Then, we have

$\widehat{C}_{N,n,m}^{(\alpha)}=\left(G_{N,n,m}^{(\alpha)}\right)^{-1}\left(F_{N,n,m}^{(\alpha)}\right)^{T}.\frac{\left[2^{2\alpha+1}\mbox{B}(\alpha+1,\alpha+1)\right]^{\frac{d}{2}}}{n^{\frac{1}{2}}}Y,$
(28)

with

$F_{N,n,m}^{(\alpha)}(i,j)=\frac{\left[2^{2\alpha+1}\mbox{B}(\alpha+1,\alpha+1)\right]^{\frac{d}{2}}}{n^{\frac{1}{2}}}.\Psi_{j}^{(\alpha)}(X_{i}).$
(29)

Next, we show that our proposed NP regression estimator is stable in the sense
that the random matrix $G_{N,n,m}^{(\alpha)}$ is well conditioned with respect
to the $2-$norm. As the random matrix $G_{N,n,m}^{(\alpha)}$ is positive
definite, its condition number denoted by
$\kappa_{2}\left(G_{N,n,m}^{(\alpha)}\right)$ is given by

$\kappa_{2}\left(G_{N,n,m}^{(\alpha)}\right)=\frac{\lambda_{\max}\left(G_{N,n,m}^{(\alpha)}\right)}{\lambda_{\min}\left(G_{N,n,m}^{(\alpha)}\right)}.$
(30)

Recall that the i.i.d. random samples $(X_{i})_{1\leq i\leq n}$ follow the
$d-$variate Beta distribution on $I^{d}$ with the density function
$\boldsymbol{g}_{\alpha}$ given by

$\boldsymbol{g}_{\alpha}(\boldsymbol{x})=\frac{1}{\left[2^{2\alpha+1}\mbox{B}(\alpha+1,\alpha+1)\right]^{d}}\boldsymbol{w}_{\alpha}(x){\mathds{1}}_{I^{d}}(\boldsymbol{x})=\frac{1}{\left(h_{0}^{(\alpha)}\right)^{d}}\boldsymbol{w}_{\alpha}(\boldsymbol{x}){\mathds{1}}_{I^{d}}(\boldsymbol{x})\,.$
(31)

Before stating the theorem about the condition number upper bound (Theorem 1),
we need the following technical lemma.

######  Lemma 4.

For $\alpha\geq-\frac{1}{2}$, $N\in\mathbb{N}$ and a positive integer $m\leq
d,$ let

$D(N,\alpha):=\begin{cases}\pi^{\frac{3}{4}}N^{(\alpha+\frac{1}{2})}e^{(\alpha+\frac{3}{4})}\
\ \mbox{if}\ \ \alpha>-\frac{1}{2}\\\ 2\ \ \mbox{if}\ \
\alpha=-\frac{1}{2}\,.\end{cases}$ (32)

Then, we have

$\left\|\Phi_{\boldsymbol{0}}^{(\alpha)}\right\|_{\infty}=\left(\frac{1}{h_{0}^{(\alpha)}}\right)^{\frac{d}{2}}\,.$
(33)

Moreover, for $1\leq|\boldsymbol{u}|\leq m$ and
$||\boldsymbol{k}_{\boldsymbol{u}}||_{1}\leq N$, we have

$\left\|\Phi_{k_{\boldsymbol{u}}}^{(\alpha)}\right\|_{\infty}\leq\left(\frac{1}{h_{0}^{(\alpha)}}\right)^{\frac{d}{2}}\cdot[D(N,\alpha)]^{m}\,.$
(34)

###### Proof.

Equality (33) is immediate since
$\Phi_{0}^{(\alpha)}=\left[\widetilde{P}_{0}^{(\alpha)}\right]^{d}=\left(\frac{1}{h_{0}^{(\alpha)}}\right)^{\frac{d}{2}}$.
Also from the definition of $\Phi_{k_{\boldsymbol{u}}}^{(\alpha)}$, we have

$\|\Phi_{k_{\boldsymbol{u}}}^{(\alpha)}\|_{\infty}\leq\|\widetilde{P}_{0}^{(\alpha)}\|_{\infty}^{d-|\boldsymbol{u}|}\prod_{i\in\boldsymbol{u}}\|\widetilde{P}_{k_{i}}^{(\alpha)}\|_{\infty}=\left(\frac{1}{h_{0}^{(\alpha)}}\right)^{\frac{d-|\boldsymbol{u}|}{2}}\prod_{i\in\boldsymbol{u}}\|\widetilde{P}_{k_{i}}^{(\alpha)}\|_{\infty}\,.$

Next, for the case where $\alpha>-\frac{1}{2}$ and by using Proposition 1 and
Lemma 2, one gets :

$\displaystyle\|\Phi_{k_{\boldsymbol{u}}}^{(\alpha)}\|_{\infty}$
$\displaystyle\leq\left(\frac{1}{h_{0}^{(\alpha)}}\right)^{\frac{d-|\boldsymbol{u}|}{2}}\prod_{i\in\boldsymbol{u}}\left[\frac{\pi^{\frac{1}{4}}N^{\alpha+\frac{1}{2}}e^{\alpha+\frac{3}{4}}}{\sqrt{2}}\right]$
(35)
$\displaystyle=\left(\frac{1}{h_{0}^{(\alpha)}}\right)^{\frac{d}{2}}\left(h_{0}^{(\alpha)}\right)^{\frac{|\boldsymbol{u}|}{2}}\left[\frac{\pi^{\frac{|\boldsymbol{u}|}{4}}N^{(\alpha+\frac{1}{2})|\boldsymbol{u}|}e^{(\alpha+\frac{3}{4})|\boldsymbol{u}|}}{\sqrt{2}^{|\boldsymbol{u}|}}\right]$
$\displaystyle\leq\left(\frac{1}{h_{0}^{(\alpha)}}\right)^{\frac{d}{2}}\left[\left(\pi^{3}\right)^{\frac{|\boldsymbol{u}|}{4}}N^{(\alpha+\frac{1}{2})|\boldsymbol{u}|}e^{(\alpha+\frac{3}{4})|\boldsymbol{u}|}\right]\,.$
$\displaystyle\leq\left(\frac{1}{h_{0}^{(\alpha)}}\right)^{\frac{d}{2}}\left[\pi^{\frac{3}{4}}N^{(\alpha+\frac{1}{2})}e^{(\alpha+\frac{3}{4})}\right]^{m}\,.$

Finally, for the case $\alpha=-\frac{1}{2}$ and by using again Proposition 1
and Lemma 2, one gets

$\displaystyle\|\Phi_{k_{\boldsymbol{u}}}^{(\alpha)}\|_{\infty}$
$\displaystyle\leq\left(\frac{1}{h_{0}^{(\alpha)}}\right)^{\frac{d}{2}}\left(h_{0}^{(-\frac{1}{2})}\right)^{\frac{|\boldsymbol{u}|}{2}}\prod_{i\in\boldsymbol{u}}\left[\frac{2}{\sqrt{\pi}}\right]$
(36)
$\displaystyle\leq\left(\frac{1}{h_{0}^{(\alpha)}}\right)^{\frac{d}{2}}\pi^{\frac{|\boldsymbol{u}|}{2}}\left[\frac{2}{\sqrt{\pi}}\right]^{|\boldsymbol{u}|}\leq\left(\frac{1}{h_{0}^{(\alpha)}}\right)^{\frac{d}{2}}2^{m}\,.$

∎

In the following theorem, we show that with high probability, the $2-$norm
condition number of the positive definite random matrix $G_{N,n,m}^{(\alpha)}$
is bounded by a convenient constant depending on a parameter $0<\delta<1.$ The
results of this theorem can be considered as a generalization of a similar
result given in [6] for tensor product multivariate Jacobi polynomials basis.

######  Theorem 1.

Under the previous notation and hypotheses on $N,m,\alpha,$ let $0<\delta<1$
and let $G_{N,n,m}^{(\alpha)}$ be the random matrix given by (27). Then, we
have

$\mathbb{P}\left(\kappa_{2}\left(G_{N,n,m}^{(\alpha)}\right)\leq\frac{1+\delta}{1-\delta}\right)\geq
1-2M_{N,m,d}.\exp{\left(-\delta^{2}\frac{n}{3D^{2m}(N,\alpha)M_{N,m,d}}\right)}.$
(37)

Here, the quantity $D(N,\alpha)$ is as defined by Lemma 4 and $M_{N,m,d}$ is
as given by Proposition 2.

###### Proof.

From (29), we have
$F_{N,n,m}^{(\alpha)}F(p,i)=\frac{\left[2^{2\alpha+1}\mbox{B}(\alpha+1,\alpha+1)\right]^{\frac{d}{2}}}{n^{1/2}}.\Psi_{i}^{(\alpha)}(X_{p})$
and $G_{N,n,m}^{(\alpha)}=\left(F_{N,n,m}^{(\alpha)}\right)^{T}\cdot
F_{N,n,m}^{(\alpha)}\,,$ one gets

$G_{N,n,m}^{(\alpha)}(i,j)=\sum_{p=1}^{n}\frac{\left[2^{2\alpha+1}\mbox{B}(\alpha+1,\alpha+1)\right]^{d}}{n}.\Psi_{i}^{(\alpha)}(X_{p})\Psi_{j}^{(\alpha)}(X_{p})=\sum_{p=1}^{n}\frac{\left(h_{0}^{(\alpha)}\right)^{d}}{n}.\Psi_{i}^{(\alpha)}(X_{p})\Psi_{j}^{(\alpha)}(X_{p})\,.$

Consequently, one gets

$\mathbb{E}[G_{N,n,m}^{(\alpha)}(i,j)]=\frac{\left(h_{0}^{(\alpha)}\right)^{d}}{n}\sum_{p=1}^{n}\mathbb{E}\left[\Psi_{i}^{(\alpha)}(X_{p})\Psi_{j}^{(\alpha)}(X_{p})\right]\,.$

Now to compute
$\mathbb{E}\left[\Psi_{i}^{(\alpha)}(X_{p})\Psi_{j}^{(\alpha)}(X_{p})\right]$,
we let $i=g(\boldsymbol{k},\boldsymbol{u})$ and
$j=g(\boldsymbol{l},\boldsymbol{v}),$ then by using Lemma 3, we obtain

$\displaystyle\mathbb{E}\left[\Psi_{i}^{(\alpha)}(X_{p})\Psi_{j}^{(\alpha)}(X_{p})\right]$
$\displaystyle=\mathbb{E}\left[\Phi_{\boldsymbol{k},\boldsymbol{u}}^{(\alpha)}(X_{p})\Phi_{\boldsymbol{l},\boldsymbol{v}}^{(\alpha)}(X_{p})\right]$
(38)
$\displaystyle=\int_{[-1,1]^{d}}\Phi_{\boldsymbol{k},\boldsymbol{u}}^{(\alpha)}(\boldsymbol{x})\Phi_{\boldsymbol{l},\boldsymbol{v}}^{(\alpha)}(\boldsymbol{x})\frac{\prod_{s=1}^{d}w_{\alpha}(\boldsymbol{x}_{s})d\boldsymbol{x}_{s}}{\left(h_{0}^{(\alpha)}\right)^{d}}$
$\displaystyle=\frac{1}{\left(h_{0}^{(\alpha)}\right)^{d}}<\Phi_{\boldsymbol{k},\boldsymbol{u}}^{(\alpha)},\Phi_{\boldsymbol{l},\boldsymbol{v}}^{(\alpha)}>_{\alpha}=\frac{1}{\left(h_{0}^{(\alpha)}\right)^{d}}\delta_{(\boldsymbol{k},\boldsymbol{u}),(\boldsymbol{l},\boldsymbol{v})}=\frac{1}{\left(h_{0}^{(\alpha)}\right)^{d}}\delta_{i,j}\,.$

Hence, we have
$\mathbb{E}\left[\Psi_{i}^{(\alpha)}(X_{p})\Psi_{j}^{(\alpha)}(X_{p})\right]=\frac{1}{\left(h_{0}^{(\alpha)}\right)^{d}}\delta_{i,j}$.
Therefore $\mathbb{E}\big{[}G_{N,n,m}^{(\alpha)}\big{]}$ is the identity
matrix of dimension $M_{N,m,d}$. Also, note that

$G_{N,n,m}^{(\alpha)}=\sum_{p=1}^{n}H_{p}\,,$

where
$H_{p}(i,j)=\frac{\left(h_{0}^{(\alpha)}\right)^{d}}{n}.\Psi_{i}^{(\alpha)}(X_{p})\Psi_{j}^{(\alpha)}(X_{p})$.
We show that, for all $p\in[[1,n]]$, all eigenvalues of $H_{p}$ are non-
negative. Moreover, we give an estimate for an upper bound for the eigenvalues
of the $H_{p}$. The matrix $H_{p}$ can be written as $H_{p}=A^{T}_{p}.A_{p}$
where
$A_{p}:=\frac{\left(h_{0}^{(\alpha)}\right)^{\frac{d}{2}}}{\sqrt{n}}.\left(\Psi_{1}^{(\alpha)}(X_{p}),\ldots,\Psi_{M_{N,m,d}}^{(\alpha)}(X_{p})\right)$.
The relation $H_{p}=A^{T}_{p}.A_{p}$ implies that the eigenvalues of all the
matrices $H_{p}$ are non-negative. Applying Gershgorin Theorem to $H_{p}$, one
gets

$\lambda_{\max}(H_{p})\leq\max_{1\leq i\leq
M_{n,m,d}}\frac{\left(h_{0}^{(\alpha)}\right)^{d}}{n}\left(\big{|}\Psi_{i}^{(\alpha)}(X_{p})\big{|}^{2}+\sum_{j\neq
i}\big{|}\Psi_{i}^{(\alpha)}(X_{p})\Psi_{j}^{(\alpha)}(X_{p})\big{|}\right).$

Therefore
${\displaystyle\lambda_{\max}(H_{p})\leq\frac{D(N,\alpha)^{2m}}{n}\cdot
M_{N,m,d}}$ for all $l\in[[1,n]]$. Next, by applying the matrix Chernoff
Theorem to the matrix $G_{N,n,m}^{(\alpha)}$ as a sum of positive semi
definite random matrices $G_{N,n,m}^{(\alpha)}$ and satisfying $0\preceq
H_{p}\preceq L\cdot I_{M_{N,m,d}},$
$L:=\frac{\left(D(N,\alpha)\right)^{2m}\cdot M_{N,m,d}}{n}$ and as
$\mu_{{\min}}=\lambda_{{\min}}\mathbb{E}\left(G_{N,n,m}^{(\alpha)}\right)=1$,
$\mu_{{\max}}=\lambda_{{\max}}\mathbb{E}\left(G_{N,n,m}^{(\alpha)}\right)=1$,
we get :

$\mathbb{P}\left(\lambda_{{\min}}\left(G_{N,n,m}^{(\alpha)}\right)\geq
1-\delta\right)\geq
1-M_{N,m,d}\cdot\exp\left(-\frac{\delta^{2}n}{2[D(N,\alpha)]^{2m}\cdot
M_{N,m,d}}\right)$ (39)

and

$\mathbb{P}\left(\lambda_{{\max}}\left(G_{N,n,m}^{(\alpha)}\right)\leq
1+\delta\right)\geq
1-M_{N,m,d}\cdot\exp\left(-\frac{\delta^{2}n}{3[D(N,\alpha)]^{2m}\cdot
M_{N,m,d}}\right)$ (40)

for $\delta\in(0,1)$. Finally, consider the events
$A_{1}:=\left(\kappa_{2}\left(G_{N,n,m}^{(\alpha)}\right)\leq\frac{1+\delta}{1-\delta}\right)$,
$A_{2}=\left(\lambda_{min}\left(G_{N,n,m}^{(\alpha)}\right)\geq
1-\delta\right)$ and
$A_{3}:=\left(\lambda_{\max}\left(G_{N,n,m}^{(\alpha)}\right)\leq
1+\delta\right)$. Using the fact that
$\mathbb{P}(A_{1}^{c})\leq\mathbb{P}(A_{2}^{c})+\mathbb{P}(A_{3}^{c})$, we get

$\mathbb{P}(A_{1})\geq
1-2M_{n,m,d}\cdot\exp\left(-\frac{\delta^{2}n}{3[D(N,\alpha)]^{2m}.M_{N,m,d}}\right)\,.$

∎

######  Remark 2.

By using (32) and (37), one concludes that a convenient choice for the value
of the parameter $\alpha$ is given by $\alpha=-\frac{1}{2}.$ For this value, a
given upper bound for $2-$norm condition number of the random matrix
$G_{N,n,m}^{(\alpha)}$ is obtained with fewer number of $n,$ the number of
random sampling points $X_{i}.$ This behaviour is illustrated by the numerical
simulations given by Table 1 of the numerical examples section.

##  5 $L^{2}-$ Risk error of the NP regression estimator

In this section, we give an estimate for the $L^{2}-$risk error of our
proposed NP regression estimator. For this purpose and it is done in [12], we
assume that there exists a constant $K_{f}$ such that

$|f(\boldsymbol{x})|\leq K_{f},\ \forall\ \boldsymbol{x}\ \in\ I^{d}\,.$ (41)

We let $\widehat{F}_{N,n,m}^{(\alpha)}$ denote the truncated version of the
estimator $\widehat{f}_{N,n,m}^{(\alpha)}$, given by

$\widehat{F}_{N,n,m}^{(\alpha)}(\boldsymbol{x}):=\mbox{Sign}\left(\widehat{f}_{N,n,m}^{(\alpha)}(\boldsymbol{x})\right)\min\left\\{K_{f},|\widehat{f}_{N,n,m}^{(\alpha)}(\boldsymbol{x})|\right\\}\,.$
(42)

######  Theorem 2.

Let $f$ be a function satisfying the hypothesis (41) and let
$\widehat{F}_{N,n,m}^{(\alpha)}$ be the truncated version of the estimator
$\widehat{f}_{N,n,m}^{(\alpha)}$, given by (42). For $\alpha\geq-\frac{1}{2}$
and $0<\delta<1$, we have

$\mathbb{E}\left[\|f-\widehat{F}_{N,n,m}^{(\alpha)}\|_{\alpha}^{2}\right]\leq
4K_{f}^{2}\left(h_{0}^{(\alpha)}\right)^{d}M_{N,m,d}\cdot\exp\left(-\frac{\delta^{2}n}{2[D(N,\alpha)]^{2m}\cdot
M_{N,m,d}}\right)\\\
+\frac{M_{N,m,d}}{n(1-\delta)^{2}}\bigg{(}[D(N,\alpha)]^{2m}\|f-\Pi_{N,m}f\|_{\alpha}^{2}+\sigma^{2}\left(h_{0}^{(\alpha)}\right)^{d}\bigg{)}+\|f-\Pi_{N,m}f\|_{\alpha}^{2}\,,$
(43)

where the quantity $D(N,\alpha)$ is as defined by Lemma 4.

###### Proof.

As it is done in [12], We write $\Omega$ as $\Omega_{+}\cup\Omega_{-},$ where

$\Omega_{-}:=\\{(X_{1},\ldots,X_{n}):\lambda_{{\min}}(G)<1-\delta\\}\ \
\mbox{and}\ \ \Omega_{+}:=\\{(X_{1},\ldots,X_{n}):\lambda_{{\min}}(G)\geq
1-\delta\\}\,.$

Then

$\mathbb{E}\left[||f-\widehat{F}_{N,n,m}^{(\alpha)}||_{\alpha}^{2}\right]=\int_{\Omega_{-}}||f-\widehat{F}_{N,n,m}^{(\alpha)}||_{\alpha}^{2}d\rho_{n}+\int_{\Omega_{+}}||f-\widehat{F}_{N,n,m}^{(\alpha)}||_{\alpha}^{2}d\rho_{n}\,,$

where $d\rho_{n}$ is the probability measure on $(I^{d})^{n}$ given by the
tensor product

$d\rho_{n}(\boldsymbol{u_{1}},\ldots,\boldsymbol{u_{n}})=\prod_{k=1}^{n}dg_{\alpha}(\boldsymbol{u_{k}})\,,$

with $g_{\alpha}$ given in (31). To get an upper bound for
$\int_{\Omega_{-}}\|f-\widehat{F}_{N,n,m}^{(\alpha)}\|_{\alpha}^{2}d\rho_{n}$,
we proceed as follows. From (39), we have

$\mathbb{P}\left(\lambda_{{\min}}\left(G_{N,n,m}^{(\alpha)}\right)<1-\delta\right)\leq
M_{N,m,d}\cdot\exp\left(-\frac{\delta^{2}n}{2[D(N,\alpha)]^{2m}\cdot
M_{N,m,d}}\right)\,.$

Then

$\displaystyle\int_{\Omega_{-}}\|f-\widehat{F}_{N,n,m}^{(\alpha)}\|_{\alpha}^{2}d\rho_{n}$
$\displaystyle\leq
4K_{f}^{2}\int_{\Omega_{-}}d\rho_{n}=4K_{f}^{2}\left(h_{0}^{(\alpha)}\right)^{d}\mathbb{P}\left(\lambda_{{\min}}\left(G_{N,n,m}^{(\alpha)}\right)<1-\delta\right)$
(44) $\displaystyle\leq
4K_{f}^{2}\left(h_{0}^{(\alpha)}\right)^{d}M_{N,m,d}\cdot\exp\left(-\frac{\delta^{2}n}{2[D(N,\alpha)]^{2m}\cdot
M_{N,m,d}}\right)\,.$

For an estimate of an Upper bound of
$\int_{\Omega_{+}}||f-\widehat{F}_{N,n,m}^{(\alpha)}||_{\alpha}^{2}d\rho_{n}$,
we use the fact that $|f-\widehat{F}_{N,n,m}^{(\alpha)}|$ is upper bounded by
$|f-\widehat{f}_{N,n,m}^{(\alpha)}|$ and the fact that $(f-\Pi_{N,m}f)$ and
$(\Pi_{N,m}f-\widehat{f}_{N,n,m}^{(\alpha)})$ are orthogonal. Hence, we get

$\displaystyle\int_{\Omega_{+}}\|f-\widehat{F}_{N,n,m}^{(\alpha)}\|_{\alpha}^{2}d\rho_{n}$
$\displaystyle\leq\int_{\Omega_{+}}\|f-\widehat{f}_{N,n,m}^{(\alpha)}\|_{\alpha}^{2}d\rho_{n}$
(45)
$\displaystyle\leq\int_{\Omega_{+}}\|f-\Pi_{N,m}f\|_{\alpha}^{2}d\rho_{n}+\int_{\Omega_{+}}\|\Pi_{N,m}f-\widehat{f}_{N,n,m}^{(\alpha)}||_{\alpha}^{2}d\rho_{n}\,.$

On the other hand, let $C_{N,n,m}^{(\alpha)}$ the vector containing the
coefficients of $\Pi_{N,m}f$ on the basis
$\\{\Psi_{1}^{(\alpha)},\ldots,\Psi_{M_{N,m,d}}^{(\alpha)}\\}$,i.e.,
$\Pi_{N,m}f(X_{i})=\sum_{j=1}^{M_{N,m,d}}C_{j}\Psi_{j}^{(\alpha)}(X_{i})$ for
$i=1,\ldots,n$. This leads to

$C_{N,n,m}^{(\alpha)}=\left(G_{n,N,m}^{(\alpha)}\right)^{-1}\left(F_{N,n,m}^{(\alpha)}\right)^{T}.\sqrt{\frac{\left(h_{0}^{(\alpha)}\right)^{d}}{n}}\left(\Pi_{N,m}(f)(X_{i})\right)_{1\leq
i\leq n}\,.$

The identity (28) used for the computation of $\widehat{f}_{N,n,m}^{(\alpha)}$
can be rewritten as:

$\widehat{C}_{N,n,m}^{(\alpha)}=\left(G_{N,n,m}^{(\alpha)}\right)^{-1}\left(F_{N,n,m}^{(\alpha)}\right)^{T}\cdot\sqrt{\frac{\left(h_{0}^{(\alpha)}\right)^{d}}{n}}\left(f(X_{i})+\epsilon_{i})\right)_{1\leq
i\leq n}.$

Combining the previous two equations, we get

$\widehat{C}_{N,n,m}^{(\alpha)}-C_{N,n,m}^{(\alpha)}=\left(G_{N,n,m}^{(\alpha)}\right)^{-1}(F_{N,n,m}^{(\alpha)})^{T}\cdot\sqrt{\frac{\left(h_{0}^{(\alpha)}\right)^{d}}{n}}\cdot\left(g(X_{i})+\epsilon_{i}\right)_{1\leq
i\leq n}\,.$

where $g(X_{i}):=\left(f-\Pi_{N,m}(f)\right)(X_{i})$. From Parseval’s
equality, we have on $\Omega_{+}$

$\displaystyle\|\Pi_{N,m}(f)-\widehat{f}_{N,n,m}^{(\alpha)}\|_{\alpha}^{2}$
$\displaystyle=||\widehat{C}_{N,n,m}^{(\alpha)}-C_{N,n,m}^{(\alpha)}||_{l_{2}}^{2}$
(46)
$\displaystyle\leq\frac{\left(h_{0}^{(\alpha)}\right)^{d}}{n}\left\|\left(G_{N,n,m}^{(\alpha)}\right)^{-1}\right\|_{2}^{2}\cdot\left\|\left(F_{N,n,m}^{(\alpha)}\right)^{T}\cdot\left(g(X_{i})+\epsilon_{i}\right)_{1\leq
i\leq n}\right\|_{l_{2}}^{2}$
$\displaystyle\leq\frac{\left(h_{0}^{(\alpha)}\right)^{d}}{n(1-\delta)^{2}}\left\|(F_{N,n,m}^{(\alpha)})^{T}\cdot\left(g(X_{i})+\epsilon_{i}\right)_{1\leq
i\leq n}\right\|_{l_{2}}^{2}$
$\displaystyle=\frac{\left(h_{0}^{(\alpha)}\right)^{2d}}{n^{2}(1-\delta)^{2}}\sum_{j=1}^{M_{N,m,d}}\sum_{k,l=1}^{n}\left(\Psi_{j}(X_{k})(g(X_{k})+\epsilon_{k})\right)\cdot\left(\Psi_{j}(X_{l})(g(X_{l})+\epsilon_{l})\right)\,.$

Considering the expectations of both sides of the previous inequality and
taking into account that
$<\Psi_{j}^{(\alpha)},g>_{\alpha}=0$ and that the $\varepsilon_{k}$ are
independent from the $X_{k}$ with $\mathbb{E}(\varepsilon_{k})=0$ and
$\mathbb{E}(\epsilon_{k}^{2})=\sigma^{2}$, we get

$\displaystyle\mathbb{E}\left[\|\Pi_{N,m}(f)-\widehat{f}_{N,n,m}^{(\alpha)}\|_{\alpha}^{2}\right]$
$\displaystyle\leq\frac{\left(h_{0}^{(\alpha)}\right)^{2d}}{n^{2}(1-\delta)^{2}}\sum_{j=1}^{M_{N,m,d}}\sum_{k=1}^{n}\left(E\left[\Psi_{j}^{2}(X_{k})g^{2}(X_{k})\right]+\sigma^{2}\cdot\mathbb{E}\left[\Psi_{j}^{2}(X_{k})\right]\right)$
(47)
$\displaystyle=\frac{\left(h_{0}^{(\alpha)}\right)^{2d}}{n^{2}(1-\delta)^{2}}\sum_{j=1}^{M_{N,m,d}}\sum_{k=1}^{n}\mathbb{E}\left[\Psi_{j}^{2}(X_{k})g^{2}(X_{k})\right]+\frac{\left(h_{0}^{(\alpha)}\right)^{d}M_{N,m,d}\cdot\sigma^{2}}{n(1-\delta)^{2}}.$

Using Lemma 4, we have :

$\mathbb{E}\left[\Psi_{j}^{2}(X_{k})g^{2}(X_{k})\right]\leq\|\Psi_{j}||_{\infty}^{2}\cdot\mathbb{E}\left[g^{2}(X_{k})\right]\leq\left(D(N,\alpha)\right)^{2m}\cdot\frac{\|f-\Pi_{N,m}f\|_{\alpha}^{2}}{\left(h_{0}^{(\alpha)}\right)^{2d}}.$

This implies that

$\displaystyle\int_{\Omega_{+}}\|\Pi_{N,m}(f)-\widehat{f}_{N,n,m}^{(\alpha)}\|_{\alpha}^{2}d\rho_{n}$
$\displaystyle\leq\mathbb{E}\left[\|\Pi_{N,m}(f)-\widehat{f}_{N,n,m}^{(\alpha)}\|_{\alpha}^{2}\right]$
(48)
$\displaystyle\leq\frac{M_{N,m,d}}{n(1-\delta)^{2}}\bigg{(}[D(N,\alpha)]^{2m}\cdot\|f-\Pi_{N,m}f\|_{\alpha}^{2}+\sigma^{2}\left(h_{0}^{(\alpha)}\right)^{d}\bigg{)}\,.$

∎

##  6 Quality of the estimation in a weighted Sobolev space

In this paragraph, we give a precise rate of convergence of our estimator in
the case where the $d-$variate regression functions belongs to a weighted
Sobolev space. More precisely, we give an estimate for the term
$\|f-\Pi_{N,m}f\|_{\alpha}^{2}$ in the $L^{2}-$risk error given by Theorem 2.
For this purpose, we recall some definitions and results mainly borrowed from
[30]. Let $\mathbb{T}$ be a unit torus and let $f\in L^{2}(\mathbb{T}^{d}).$
The Fourier series expansion of $f$ is given by

$f(\boldsymbol{x})=\sum_{\boldsymbol{p}\in\mathbb{\mathbb{Z}}^{d}}a_{\boldsymbol{p}}(f)e^{2i\pi\boldsymbol{p}\cdot\boldsymbol{x}},\quad\boldsymbol{x}\in\mathbb{T}^{d},\quad(\mbox{or
}\mathbb{R}^{d}).$ (49)

We have the following partition of $\mathbb{Z}^{d}$ :
$\mathbb{Z}^{d}=\bigcup_{\boldsymbol{u}\in\mathcal{D}}F_{\boldsymbol{u}}$.
This gives the analysis of variance (ANOVA) decomposition of $f$ :

$f(x)=f_{\emptyset}+\sum_{i=1}^{d}f_{i}(x_{i})+\sum_{i=1}^{d-1}\sum_{j=i+1}^{d}f_{i,j}(x_{i},x_{j})+\ldots+f_{\mathcal{D}}(\boldsymbol{x})=\sum_{\boldsymbol{u}\in\mathcal{D}}f_{\boldsymbol{u}}(x_{\boldsymbol{u}})\,,$

where the functions $f_{\boldsymbol{u}}$ are called ANOVA terms, see [30]. Let
$\boldsymbol{U}\subset\mathcal{P}(\mathcal{D})$. The truncated ANOVA
decomposition over $\boldsymbol{U}$ is defined as:

$T_{\boldsymbol{U}}(f):=\sum_{\boldsymbol{u}\in\boldsymbol{U}}f_{\boldsymbol{u}}\,.$

In particular, for $1\leq s\leq d$, we define

$T_{s}(f):=\sum_{|\boldsymbol{u}|\leq s}f_{\boldsymbol{u}}\,.$

######  Definition 1.

Let $s>0$. Let $w^{(s)}:\mathbb{Z}^{d}\mapsto[1,\infty)$ be the weight
function defined by

$w^{(s)}(\boldsymbol{p}):=\prod_{j=1}^{d}\left(1+|\boldsymbol{p}_{j}|\right)^{s}\,\,\
,\ \forall\ \boldsymbol{p}\in\mathbb{Z}^{d}\,.$

We associate to this weight function the Sobolev space

$H^{s}(I^{d}):=\left\\{f\in
L^{2}(I^{d});\,\|f\|_{H^{s}(I^{d})}:=\sum_{\boldsymbol{p}\in\mathbb{Z}^{d}}(w^{(s)}(\boldsymbol{p}))^{2}\cdot|a_{\boldsymbol{p}}(f)|^{2}<+\infty\right\\}\,,$
(50)

and the weighted Wiener algebra

$\mathcal{A}^{s}(I^{d}):=\left\\{f\in
L^{1}(I^{d});\,\|f\|_{\mathcal{A}^{s}(I^{d})}=\sum_{\boldsymbol{p}\in\mathbb{Z}^{d}}w^{(s)}(\boldsymbol{p})\cdot|a_{\boldsymbol{p}}(f)|<+\infty\right\\}\,.$

The following lemma provides us with an estimate for the decay rate of the
expansion series expansion coefficients of the $f_{\boldsymbol{u}}$ when $f\in
H^{s+\frac{d}{2}}(I^{d})$ and with respect to the basis functions
$\Phi_{\boldsymbol{k},\boldsymbol{v}}^{(\alpha)}.$

######  Lemma 5.

Let $0<\xi<\frac{1}{2}$, $N\in\mathbb{N}$ and $\boldsymbol{k}\in
F_{\boldsymbol{u}}$ such that $||\boldsymbol{k}||_{1}\geq N+1$ with

$\frac{\frac{N+1}{|\boldsymbol{u}|}}{\ln\left(\frac{N+1}{|\boldsymbol{u}|}\right)}\geq-\ln\left(\xi\right)\left(1+s+\frac{d}{2}\right)\,.$

For any $f\in H^{s+\frac{d}{2}}(I^{d})$

$|C_{\boldsymbol{k},\boldsymbol{u}}(f_{\boldsymbol{u}})|=|<f_{\boldsymbol{u}},\Phi_{\boldsymbol{k},\boldsymbol{v}}^{(\alpha)}>_{\alpha}|\lesssim_{\alpha,d,s}||\boldsymbol{k}||_{\infty}^{-s-\frac{d}{2}}\left(||f_{\boldsymbol{u}}||_{2}+||f_{\boldsymbol{u}}||_{H^{s+\frac{d}{2}}(I^{d})}\right)\,.$
(51)

###### Proof.

We have:

$f_{\boldsymbol{u}}(\boldsymbol{x})=\sum_{\boldsymbol{p}\in\mathbb{\mathbb{Z}}^{d}}a_{\boldsymbol{p}}\left(f_{\boldsymbol{u}}\right)e^{2i\pi\boldsymbol{p}\cdot\boldsymbol{x}},\quad\boldsymbol{x}\in
I^{d}.$ (52)

For $\boldsymbol{v}\in\mathbb{Z}^{d}$ and
$\boldsymbol{k}\in\mathbb{Z}^{|\boldsymbol{v}|}$, we have :

$C_{\boldsymbol{k},\boldsymbol{v}}(f_{\boldsymbol{u}})=<f_{\boldsymbol{u}},\Phi_{\boldsymbol{k},\boldsymbol{v}}^{(\alpha)}>_{\alpha}=\sum_{\boldsymbol{p}\in\mathbb{\mathbb{Z}}^{d}}a_{\boldsymbol{p}}(f_{\boldsymbol{u}})<e^{2i\pi\boldsymbol{p}\cdot\boldsymbol{x}},\Phi_{\boldsymbol{k},\boldsymbol{v}}^{(\alpha)}(\boldsymbol{x})>_{\alpha}=\sum_{\boldsymbol{p}\in\mathbb{\mathbb{Z}}^{d}}a_{\boldsymbol{p}}(f_{\boldsymbol{u}})d_{\boldsymbol{k},\boldsymbol{v},\boldsymbol{p}}\,,$

where
$d_{\boldsymbol{k},\boldsymbol{v},\boldsymbol{p}}=<e^{2i\pi\boldsymbol{p}\cdot\boldsymbol{x}},\Phi_{\boldsymbol{k},\boldsymbol{v}}^{(\alpha)}(\boldsymbol{x})>_{\alpha}$.
Note that if $\boldsymbol{v}\neq\boldsymbol{u}$, then

$C_{\boldsymbol{k},\boldsymbol{v}}\left(f_{\boldsymbol{u}}\right)=<f_{\boldsymbol{u}},\Phi_{\boldsymbol{k},\boldsymbol{v}}^{(\alpha)}>_{\alpha}=0\,.$

Consequently, we need only to estimate
$C_{\boldsymbol{k},\boldsymbol{u}}\left(f_{\boldsymbol{u}}\right)$. In the
special case $|\boldsymbol{u}|=0$, it is easy to show that
$|d_{p}|:=|<e^{2i\boldsymbol{p},\boldsymbol{x}},\Phi_{\boldsymbol{0}}^{(\alpha)}(\boldsymbol{x})>_{\alpha}|\leq
1$. Next, for $|\boldsymbol{u}|>0$, we have

$\displaystyle\big{|}d_{\boldsymbol{k},\boldsymbol{u},\boldsymbol{p}}\big{|}$
$\displaystyle=\Big{|}<e^{2i\boldsymbol{p},\boldsymbol{x}},\left(\frac{1}{\sqrt{h_{0}^{(\alpha)}}}\right)^{d-|\boldsymbol{u}|}\prod_{j\in\boldsymbol{u}}\widetilde{P}_{k_{j}}^{(\alpha)}(x_{j})>_{\alpha}\Big{|}$
(53) $\displaystyle=\Big{|}\int_{[-1,1]^{d}}\prod_{l=1}^{d}e^{2i\pi
p_{l}x_{l}}\left(\frac{1}{\sqrt{h_{0}^{(\alpha)}}}\right)^{d-|\boldsymbol{u}|}\prod_{j\in\boldsymbol{u}}\widetilde{P}_{k_{j}}^{(\alpha)}(x_{j})\boldsymbol{w}(\boldsymbol{x})d\boldsymbol{x}\Big{|}$
$\displaystyle=\left[\prod_{l\in\mathcal{D}\backslash\boldsymbol{u}}\Bigg{|}\int_{-1}^{1}\frac{e^{2i\pi
p_{l}x_{l}}}{\sqrt{h_{0}^{(\alpha)}}}w_{\alpha}(x_{l})dx_{l}\Bigg{|}\right]\times\left[\prod_{j\in\boldsymbol{u}}\Bigg{|}\int_{-1}^{1}e^{2i\pi
p_{j}x_{j}}\widetilde{P}_{k_{j}}^{(\alpha)}(x_{j})w_{\alpha}(x_{j})dx_{j}\Bigg{|}\right]$
$\displaystyle\leq\prod_{j\in\boldsymbol{u}}\Bigg{|}\int_{-1}^{1}e^{2i\pi
p_{j}y}\widetilde{P}_{k_{j}}^{(\alpha)}(y)w_{\alpha}(y)dy\Bigg{|}=\prod_{j\in\boldsymbol{u}}\Bigg{|}d_{p_{j},k_{j}}\Bigg{|}\,,$

where, for $l\in\mathbb{Z}$ and $r\in\mathbb{N}$,
$d_{l,r}:=\int_{-1}^{1}e^{2i\pi
ly}\widetilde{P}_{r}^{(\alpha)}(y)w_{\alpha}(y)dy$. Using (6), we get

$d_{l,r}=i^{r}\sqrt{\pi}\sqrt{2r+2\alpha+1}\sqrt{\frac{\Gamma(r+2\alpha+1)}{\Gamma(r+1)}}\frac{J_{r+\alpha+\frac{1}{2}}(2\pi
l)}{(2\pi l)^{\alpha+\frac{1}{2}}}\,,$

where $J_{a}$ is the Bessel function of the first kind and order $a>-1$. Note
that since $d_{l,r}=(-1)^{r}d_{-l,r}$ then $|d_{l,r}|=|d_{-l,r}|$ which allow
us to consider only the case where $l\geq 0$. Now, we apply the inequality (7)
to the Bessel function $J_{a}$ of the previous equation. We obtain :

$|d_{l,r}|\leq\sqrt{\pi}\sqrt{2r+2\alpha+1}\sqrt{\frac{\Gamma(r+2\alpha+1)}{\Gamma(r+1)}}\frac{1}{\Gamma(r+\alpha+\frac{3}{2})}\frac{(\pi|l|)^{r}}{2^{\alpha+\frac{1}{2}}}\,.$

Using (8), we get

$\displaystyle|d_{l,r}|$
$\displaystyle\leq\sqrt{r}(e\pi|l|)^{r}\frac{\pi^{\frac{3}{4}}e^{\frac{1}{4}}}{2^{\alpha+\frac{1}{2}}}\sqrt{\alpha+\frac{3}{2}}\frac{(r+2\alpha+\frac{1}{2})^{\frac{r}{2}+\alpha+\frac{1}{4}}}{(r+\frac{1}{2})^{\frac{r}{2}+\frac{1}{4}}\cdot(r+\alpha+1)^{r+\alpha+1}}$
(54) $\displaystyle\leq
K(\alpha)\sqrt{r}\left(\frac{e\pi|l|}{r+\alpha}\right)^{r}\,,$

with
$K(\alpha):=\frac{\pi^{\frac{3}{4}}e^{\alpha+\frac{1}{4}}}{\sqrt{2}}\sqrt{\alpha+\frac{3}{2}}(\alpha+1)^{\alpha}$.
We also have, from Cauchy-Schwartz inequality that $|d_{p_{j},k_{j}}|\leq 1$
for all $k_{j}\in\mathbb{N}$ and $p_{j}\in\mathbb{Z}$. Injecting this in (53),
we get

$|d_{\boldsymbol{k},\boldsymbol{u},\boldsymbol{p}}|\leq|d_{p_{j_{0}},k_{j_{0}}}|\,$

where $j_{0}\in[[1,d]]$ is such that
$||\boldsymbol{k}||_{\infty}=|k_{j_{0}}|$. Let $0<\xi<1$ such that
$||\boldsymbol{p}||_{\infty}\leq\xi\frac{||\boldsymbol{k}||_{\infty}}{e\pi}$.
This implies that

$|d_{\boldsymbol{k},\boldsymbol{u},\boldsymbol{p}}|\lesssim_{\alpha}\sqrt{||\boldsymbol{k}||_{\infty}}\xi^{||\boldsymbol{k}||_{\infty}}\
,\ \mbox{if}\
||\boldsymbol{p}||_{\infty}\leq\xi\frac{||\boldsymbol{k}||_{\infty}}{e\pi}\,,$

where the notation $\lesssim_{\gamma}$ means in general that the inequality is
true up to constant depending only on a variable $\gamma$. Let us now re-write
the expression of $C_{\boldsymbol{k},\boldsymbol{u}}(f_{\boldsymbol{u}})$ in
the following manner :

$C_{\boldsymbol{k},\boldsymbol{u}}(f_{\boldsymbol{u}})=\underbrace{\sum_{e||\pi||_{\infty}\leq\xi||\boldsymbol{k}||_{\infty}}a_{\boldsymbol{p}}(f_{\boldsymbol{u}})d_{\boldsymbol{k},\boldsymbol{u},\boldsymbol{p}}}_{S_{1}}+\underbrace{\sum_{e||\pi||_{\infty}>\xi||\boldsymbol{k}||_{\infty}}a_{\boldsymbol{p}}(f_{\boldsymbol{u}})d_{\boldsymbol{k},\boldsymbol{u},\boldsymbol{p}}}_{S_{2}}.$

To get an upper bound for $|S_{1}|,$ we let
$\mathcal{A}:=\\{\boldsymbol{p}\in\mathbb{Z}^{d}:||\boldsymbol{p}||_{\infty}\leq\xi\frac{||\boldsymbol{k}||_{\infty}}{e\pi}\\}$.
This set contains at most
$\left(2\left[\frac{\xi}{e\pi}||\boldsymbol{k}||_{\infty}\right]+1\right)^{d}$
elements where $[x]$ denotes the integer part of $x$. Hence, by using Bessel’s
and Cauchy-Schwartz inequalities, we obtain

$\displaystyle|S_{1}|=\Bigg{|}\sum_{\boldsymbol{p}\in\mathcal{A}}a_{\boldsymbol{p}}(f_{\boldsymbol{u}})d_{\boldsymbol{k},\boldsymbol{u},\boldsymbol{p}}\Bigg{|}$
$\displaystyle\leq\left(\left(\sum_{\boldsymbol{p}\in\mathcal{A}}|a_{\boldsymbol{p}}(f_{\boldsymbol{u}})|^{2}\right)\cdot\left(\sum_{\boldsymbol{p}\in\mathcal{A}}|d_{\boldsymbol{k},\boldsymbol{u},\boldsymbol{p}}|^{2}\right)\right)^{\frac{1}{2}}$
(55)
$\displaystyle\lesssim_{\alpha}\left(\sum_{\boldsymbol{p}\in\mathcal{A}}||\boldsymbol{k}||_{\infty}\xi^{2||\boldsymbol{k}||_{\infty}}\right)^{\frac{1}{2}}\cdot||f_{\boldsymbol{u}}||_{2}$
$\displaystyle\lesssim_{\alpha}\sqrt{||\boldsymbol{k}||_{\infty}}\xi^{||\boldsymbol{k}||_{\infty}}\left(2\left[\frac{\xi}{e\pi}||\boldsymbol{k}||_{\infty}\right]+1\right)^{\frac{d}{2}}||f_{\boldsymbol{u}}||_{2}$
$\displaystyle\lesssim_{\alpha}||\boldsymbol{k}||_{\infty}^{\frac{d+1}{2}}\xi^{||\boldsymbol{k}||_{\infty}}||f_{\boldsymbol{u}}||_{2}\,.$

Next, we get an upper bound for $|S_{2}|$. For thus purpose, we suppose that
$f\in H^{s+\frac{d}{2}}(I^{d})$. It follows, using Lemma3.9 of [30], that
$f_{\boldsymbol{u}}\in H^{s+\frac{d}{2}}(I^{d})$
$\forall\,|\boldsymbol{u}|\leq d$. Applying Cauchy-Schwartz inequality to
$|S_{2}|^{2}$ followed by Bessel’s inequality, we obtain

$|S_{2}|^{2}\leq\left(\sum_{\boldsymbol{p}\in\mathcal{A}^{c}}|a_{\boldsymbol{p}}(f_{\boldsymbol{u}})|^{2}\right).\left(\sum_{\boldsymbol{p}\in\mathcal{A}^{c}}|d_{\boldsymbol{k},\boldsymbol{u},\boldsymbol{p}}|^{2}\right)\\\
\leq\left(\sum_{\boldsymbol{p}\in\mathcal{A}^{c}}|a_{\boldsymbol{p}}(f_{\boldsymbol{u}})|^{2}\right)\|\Phi_{\boldsymbol{u},\boldsymbol{k}}^{(\alpha)}\boldsymbol{w}^{(\alpha)}\|_{2}^{2}\leq\left(\sum_{\boldsymbol{p}\in\mathcal{A}^{c}}|a_{\boldsymbol{p}}(f_{\boldsymbol{u}})|^{2}\right).||\Phi_{\boldsymbol{u},\boldsymbol{k}}^{(\alpha)}\|_{\alpha}^{2}\cdot\|\boldsymbol{w}^{(\alpha)}\|_{\infty}^{2}\leq\left(\sum_{\boldsymbol{p}\in\mathcal{A}^{c}}|a_{\boldsymbol{p}}(f_{\boldsymbol{u}})|^{2}\right)\,.$
(56)

Note that for $\boldsymbol{p}\in\mathcal{A}^{c}$, we have
$\left[\prod_{j=1}^{d}\left(1+|p_{j}|\right)^{s+\frac{d}{2}}\right]^{2}\geq||\boldsymbol{p}||_{\infty}^{2s+d}\geq\left[\xi\frac{||\boldsymbol{k}||_{\infty}}{(e\pi)}\right]^{2s+d}$.
Consequently, for $f_{u}\in H^{s+\frac{d}{2}}(I^{d})$, we have :

$|S_{2}|^{2}\leq\sum_{\boldsymbol{p}\in\mathcal{A}^{c}}|a_{\boldsymbol{p}}(f_{\boldsymbol{u}})|^{2}\leq\left(\frac{e\pi}{\xi||\boldsymbol{k}||_{\infty}}\right)^{2s+d}\sum_{\boldsymbol{p}\in\mathcal{A}^{c}}\left(1+\sum_{j=1}^{d}|p_{j}|^{2}\right)^{s+\frac{d}{2}}|a_{\boldsymbol{p}}(f_{\boldsymbol{u}})|^{2}\leq\left(\frac{e\pi}{\xi||\boldsymbol{k}||_{\infty}}\right)^{2s+d}||f_{\boldsymbol{u}}||_{H^{s+\frac{d}{2}}(I^{d})}^{2}\,.$

Hence,

$|S_{2}|\lesssim_{\alpha,d,s}\|\boldsymbol{k}\|_{\infty}^{-s-\frac{d}{2}}\|f_{\boldsymbol{u}}\|_{H^{s+\frac{d}{2}}(I^{d})}\,.$
(57)

Combining (55) and (57), we get

$\displaystyle|C_{\boldsymbol{k},\boldsymbol{u}}(f_{\boldsymbol{u}})|$
$\displaystyle\lesssim_{\alpha,d,s}\|\boldsymbol{k}\|_{\infty}^{\frac{d+1}{2}}\xi^{\|\boldsymbol{k}||_{\infty}}\|f_{\boldsymbol{u}}\|_{2}+\|\boldsymbol{k}\|_{\infty}^{-s-\frac{d}{2}}\|f_{\boldsymbol{u}}\|_{H^{s+\frac{d}{2}}(I^{d})}$
(58)
$\displaystyle\leq\left(\|\boldsymbol{k}\|_{\infty}^{\frac{d+1}{2}}\xi^{\|\boldsymbol{k}\|_{\infty}}+\|\boldsymbol{k}\|_{\infty}^{-s-\frac{d}{2}}\right)\left(\|f_{\boldsymbol{u}}\|_{2}+\|f_{\boldsymbol{u}}\|_{H^{s+\frac{d}{2}}(I^{d})}\right)$

Finally, if $\|\boldsymbol{k}\|_{1}\geq(N+1)$ and $N\in\mathbb{N}$ is such
that
$\frac{N+1}{|\boldsymbol{u}|.(\ln(N+1)-\ln(|\boldsymbol{u}|))}\geq\frac{1}{\ln\left(\frac{1}{\xi}\right)}\left(1+s+\frac{d}{2}\right),$
then
$\|\boldsymbol{k}\|_{\infty}^{\frac{d+1}{2}}\xi^{\|\boldsymbol{k}\|_{\infty}}\leq\|\boldsymbol{k}\|_{\infty}^{-s-\frac{d}{2}}$.
Consequently

$|C_{\boldsymbol{k},\boldsymbol{u}}(f_{\boldsymbol{u}})|\lesssim_{\alpha,d,s}\|\boldsymbol{k}\|_{\infty}^{-s-\frac{d}{2}}\left(\|f_{\boldsymbol{u}}\|_{2}+\|f_{\boldsymbol{u}}\|_{H^{s+\frac{d}{2}}(I^{d})}\right)\,.$

∎

######  Theorem 3.

Let $f\in H^{s+\frac{d}{2}}(I^{d})$ and let $1\leq m\leq\min\\{N,d\\}$ with
$N>e.m-1$ and such that

$\frac{\frac{N+1}{m}}{\ln\left(\frac{N+1}{m}\right)}\geq-\ln\left(\xi\right)\left(1+s+\frac{d}{2}\right)\,,$

where $0<\xi<\frac{1}{2}$. Suppose that $s+\frac{d}{2}>m+1,$ then

$\|f-\Pi_{N,m}(f)\|_{\alpha}\lesssim_{\alpha,d,s}\frac{1}{2^{(s+\frac{d}{2})(m+1)}}||f||_{H^{s+\frac{d}{2}}(I^{d})}+\frac{m\|f\|}{s+\frac{d}{2}-m}.\left(2+\frac{m}{N+1}\right)^{d}\left(\frac{m}{N+1}\right)^{s+\frac{d}{2}-m}\,.$
(59)

and

$\|f-\Pi_{N,m}(f)\|_{\infty}\lesssim_{\alpha,d,s}\frac{1}{2^{(s+\frac{d}{2})(m+1)}}\left\|f\right\|_{\mathcal{A}^{s}(I^{d})}+\frac{\|f\|D^{m}(N,\alpha)m}{s+\frac{d}{2}-m}\left(2+\frac{m}{N+1}\right)^{d}\left(\frac{m}{N+1}\right)^{s+\frac{d}{2}-m}\,.$
(60)

Here, $\|f\|=||f||_{2}+||f||_{H^{s+\frac{d}{2}}}$ and $D(N,\alpha)$ is as
given by Lemma 4.

###### Proof.

The orthogonal projection of $f$ on $\mathcal{P}_{N,m,d}$, $\Pi_{N,m}f$,
verifies

$\left\|f-\Pi_{N,m}f\right\|_{\alpha}=\Big{\|}\big{[}f-T_{m}(f)\big{]}+\big{[}T_{m}(f)-\Pi_{N,m}(T_{m}(f))\big{]}+\big{[}\Pi_{N,m}(T_{m}(f))-\Pi_{N,m}(f)\big{]}\Big{\|}_{\alpha}$

Note that for $N\geq m$, $\Pi_{N,m}(T_{m}(f))=\Pi_{N,m}(f)$ and consequently,
we have

$\left\|f-\Pi_{N,m}f\right\|_{\alpha}\leq\left\|f-T_{m}(f)\right\|_{\alpha}+\left\|T_{m}(f)-\Pi_{N.m}(T_{m}(f))\right\|_{\alpha}.$

From [30], we have for $f\in H^{s+\frac{d}{2}}(I^{d})$

$\|f-T_{m}f\|_{L^{2}(I^{d})}\leq\frac{1}{2^{(s+\frac{d}{2})(m+1)}}\|f\|_{H^{s+\frac{d}{2}}(I^{d})}\,.$

To get an upper bound for
$\left\|T_{m}(f)-\Pi_{N,m}(T_{m}(f))\right\|_{\alpha}$, we proceed as follows.
The function $\Pi_{N,m}f$ writes as $\Pi_{N,m}f=\sum_{|\boldsymbol{v}|\leq
m;||\boldsymbol{k}||_{1}\leq
N}C_{\boldsymbol{k},\boldsymbol{v}}(f)\Phi_{\boldsymbol{k},\boldsymbol{v}}(x)\,.$
hence,

$\displaystyle\left\|T_{m}(f)-\Pi_{N,m}(T_{m}(f))\right\|_{\alpha}$
$\displaystyle=\Big{\|}\sum_{|\boldsymbol{u}|\leq
m}f_{\boldsymbol{u}}-\Pi_{N,m}\Big{(}\sum_{|\boldsymbol{u}|\leq
m}f_{\boldsymbol{u}}\Big{)}\Big{\|}_{\alpha}$ (61)
$\displaystyle=\left\|\sum_{|\boldsymbol{u}|\leq
m}\left[f_{\boldsymbol{u}}-\Pi_{N,m}(f_{\boldsymbol{u}})\right]\right\|_{\alpha}=\left\|\sum_{|\boldsymbol{u}|\leq
m;|\boldsymbol{v}|\leq m;\|\boldsymbol{k}\|_{1}\geq
N+1}C_{\boldsymbol{k},\boldsymbol{v}}\left(f_{\boldsymbol{u}}\right)\Phi_{\boldsymbol{k},\boldsymbol{v}}^{(\alpha)}\right\|_{\alpha}$
$\displaystyle=\left\|\sum_{|\boldsymbol{u}|\leq m;\|\boldsymbol{k}\|_{1}\geq
N+1}C_{\boldsymbol{k},\boldsymbol{u}}\left(f_{\boldsymbol{u}}\right)\Phi_{\boldsymbol{k},\boldsymbol{u}}^{(\alpha)}\right\|_{\alpha}\,.$

Consequently, we get

$\displaystyle\left\|T_{m}(f)-\Pi_{N,m}(T_{m}(f))\right\|_{\alpha}$
$\displaystyle\leq\sum_{|\boldsymbol{u}|\leq m;\|\boldsymbol{k}\|_{1}\geq
N+1}|C_{\boldsymbol{k},\boldsymbol{u}}\left(f_{\boldsymbol{u}}\right)|\cdot\left\|\Phi_{\boldsymbol{k},\boldsymbol{u}}^{(\alpha)}\right\|_{\alpha}=\sum_{|\boldsymbol{u}|\leq
m;\|\boldsymbol{k}_{\boldsymbol{u}}\|_{1}\geq
N+1}|C_{\boldsymbol{k},\boldsymbol{u}}\left(f_{\boldsymbol{u}}\right)|$ (62)
$\displaystyle=\sum_{|\boldsymbol{u}|=1}^{m}\sum_{\|\boldsymbol{k}\|_{1}\geq
N+1}|C_{\boldsymbol{k},\boldsymbol{u}}\left(f_{\boldsymbol{u}}\right)|$
$\displaystyle\leq\sum_{|\boldsymbol{u}|=1}^{m}\sum_{\|\boldsymbol{k}\|_{\infty}\geq\frac{N+1}{|\boldsymbol{u}|}}|C_{\boldsymbol{k},\boldsymbol{u}}\left(f_{\boldsymbol{u}}\right)|\,.$

Let $1\leq|\boldsymbol{u}|\leq m$ and $\boldsymbol{k}\in F_{\boldsymbol{u}}.$
Then for $1\leq j\leq m$ and $l\in\mathbb{N}$, the number of elements
$(\boldsymbol{u},\boldsymbol{k})$ such that $|\boldsymbol{u}|=j$ and
$||\boldsymbol{k}||_{\infty}=l$ is ${d\choose j}j(l+1)^{j-1}\,.$ Let
$\|f_{\boldsymbol{u}}\|:=||f_{\boldsymbol{u}}||_{2}+||f_{\boldsymbol{u}}||_{H^{s+\frac{d}{2}}}$.
Using the result of Lemma3.9 in [30] and adapting its proof for the
$||.||_{H^{s}}-$norm, we conclude that for all
$\boldsymbol{u}\subset\mathcal{D}$ with $|\boldsymbol{u}|\geq 1$
$\|f_{\boldsymbol{u}}\|\leq\|f\|.$ Moreover, by using the result of Lemma 5,
we get

$\displaystyle\left\|T_{m}(f)-\Pi_{N,m}(T_{m}(f))\right\|_{\alpha}$
$\displaystyle\lesssim_{\alpha,d,s}\|f\|\cdot\sum_{j=1}^{m}{d\choose
j}\,j\,\sum_{l\geq\frac{N+1}{j}}(l+1)^{j-1}\frac{1}{l^{s+\frac{d}{2}}}$ (63)
$\displaystyle\leq\|f\|\cdot\sum_{j=1}^{m}{d\choose
j}\,j\,\left(1+\frac{m}{N+1}\right)^{j-1}\sum_{l\geq\frac{N+1}{j}}l^{j-1-s-\frac{d}{2}}$
$\displaystyle\leq\frac{m\|f\|}{s+\frac{d}{2}-m}.\left(2+\frac{m}{N+1}\right)^{d}\left(\frac{m}{N+1}\right)^{s+\frac{d}{2}-m}\,.$

In a similar manner, we have

$\left\|f-\Pi_{N,m}f\right\|_{\infty}\leq\left\|f-T_{m}(f)\right\|_{\infty}+\left\|T_{m}(f)-\Pi_{N.m}(T_{m}(f))\right\|_{\infty}\,.$

In [30], it has been shown that

$\left\|f-T_{m}(f)\right\|_{\infty}\leq\frac{1}{2^{(s+\frac{d}{2})(m+1)}}\left\|f\right\|_{\mathcal{A}^{s}(I^{d})}.$

On the other hand and by using Lemma 4 and Lemma 5, one gets

$\displaystyle\left\|T_{m}(f)-\Pi_{N.m}(T_{m}(f))\right\|_{\infty}$
$\displaystyle\leq$
$\displaystyle\sum_{|\boldsymbol{u}|=1}^{m}\sum_{\|\boldsymbol{k}\|_{\infty}\geq\frac{N+1}{|\boldsymbol{u}|}}|C_{\boldsymbol{k},\boldsymbol{u}}\left(f_{\boldsymbol{u}}\right)|.\left\|\Phi_{\boldsymbol{k},\boldsymbol{u}}^{(\alpha)}\right\|_{\infty}$
$\displaystyle\leq$
$\displaystyle\sum_{|\boldsymbol{u}|=1}^{m}\sum_{\|\boldsymbol{k}\|_{\infty}\geq\frac{N+1}{|\boldsymbol{u}|}}|C_{\boldsymbol{k},\boldsymbol{u}}\left(f_{\boldsymbol{u}}\right)|.\cdot\frac{D^{m}(N,\alpha)}{\left(h_{0}^{(\alpha)}\right)^{\frac{d}{2}}}$
$\displaystyle\lesssim_{\alpha,d,s}$
$\displaystyle\frac{D^{m}(N,\alpha)}{\left(h_{0}^{(\alpha)}\right)^{\frac{d}{2}}}\cdot\|f\|\sum_{|\boldsymbol{u}|=1}^{m}\sum_{\|\boldsymbol{k}\|_{\infty}\geq\frac{N+1}{|\boldsymbol{u}|}}||\boldsymbol{k}||^{-s-\frac{d}{2}}_{\infty}$
$\displaystyle=$
$\displaystyle\frac{D^{m}(N,\alpha)}{\left(h_{0}^{(\alpha)}\right)^{\frac{d}{2}}}\|f\|\sum_{j=1}^{m}{d\choose
j}\,j\,\sum_{l\geq\frac{N+1}{j}}l^{j-1-s-\frac{d}{2}}\left(1+\frac{1}{l}\right)^{j-1}$
$\displaystyle\lesssim_{\alpha,d,s}$
$\displaystyle\|f\|D^{m}(N,\alpha)\sum_{j=1}^{m}{d\choose
j}\,j\,\left(1+\frac{j}{N+1}\right)^{j-1}\sum_{l\geq\frac{N+1}{j}}l^{m-1-s-\frac{d}{2}}$
$\displaystyle\lesssim_{\alpha,d,s}$
$\displaystyle\frac{m\|f\|D^{m}(N,\alpha)}{s+\frac{d}{2}-m}\left(2+\frac{m}{N+1}\right)^{d}\left(\frac{m}{N+1}\right)^{s+\frac{d}{2}-m}\,.$

∎

######  Remark 3.

From (59), one concludes that under the conditions that
${\displaystyle\gamma_{m,N}=\frac{m}{N+1}<1}$ and $s+\frac{d}{2}-m>0$ with
$2^{(s+\frac{d}{2})(m+1)}>3^{d}\gamma_{m,N}^{m-\frac{d}{2}-s},$ we have

$\|f-\Pi_{N,m}(f)\|_{\alpha}=O\Big{(}3^{d}\gamma_{m,N}^{s+\frac{d}{2}-m}\Big{)}.$

The previous estimate together with the estimate given by Theorem 2, provide
us with a precise estimate for the $L^{2}-$risk error of our estimator
$\widehat{F}^{(\alpha)}_{N,n,m}$ when the regression function belongs to the
weighted Sobolev space (50).

##  7 Numerical Examples

In this paragraph, we give three numerical examples that illustrate the
different results of this work.

Example 1: In this first example, we illustrate the results of Proposition 2
and Theorem 1. For this purpose, we have considered the following parameter
values: $d=4$ and $d=6,$ for the dimension with the values of $2\leq N\leq 5,$
for the parameter relative to the total degree $d-$variate Jacobi polynomials
space. These polynomials are associated with the two special values of
$\alpha=-0.5$ and $\alpha=0.5.$ Moreover, we have considered the value of
$m=2,$ that we restrict ourselves to the ANOVA decomposition with $m=2$
interactions between the covariables. Also, for the case of $d=4,$
(respectively $d=6$), we have considered a random sampling set with size
$n=900$ (respectively $n=1600$) and following a multivariate
$\mbox{Beta}(\alpha+1,\alpha+1)$ distribution. Then, we have computed the
$\kappa_{2}\left(G_{N,n,m}^{(\alpha)}\right),$ the $2-$condition number of the
random projection matrix $G_{N,n,m}^{(\alpha)},$ given by (29). Also, for each
values of the couple $(d,N),$ we have provided the dimension $M_{N,2,d}$ of
the considered $d-$variate polynomial space. The obtained numerical results
are given by the following Table 1.

$d=4$ | $M_{N,2,d}$ | $\kappa_{2}\left(G_{N,n,2}^{(-0.5)}\right)$ | $\kappa_{2}(G_{N,n,2}^{(0.5)})$ | $d=6$ | $M_{N,2,d}$ | $\kappa_{2}\left(G_{N,n,2}^{(-0.5)}\right)$ | $\kappa_{2}\left(G_{N,n,2}^{(0.5)}\right)$
---|---|---|---|---|---|---|---
$N$ |  |  |  | $N$ |  |  |
$2$ | $15$ | $8.80$ | $6.70$ | $2$ | $28$ | $16.90$ | $11.75$
$3$ | $31$ | $20.32$ | $32.50$ | $3$ | $64$ | $24.35$ | $47.80$
$4$ | $53$ | $41.16$ | $48.25$ | $4$ | $115$ | $54.20$ | $87.75$
$5$ | $81$ | $65.19$ | $135.45$ | $5$ | $181$ | $223.22$ | $312.77$
Table 1: The Jacobi polynomial space dimension and the $2-$condition number of
the random projection matrix for $m=2,$ $(d,n)=(4,900),\,(6,1600),$
$\alpha=\frac{1}{2},-\frac{1}{2}$ and $2\leq N\leq 5.$

Also, in Figure 1, we give the plots of the spectrum of $G_{N,n,2}^{(-0.5)},$
for $(d,n)=(4,900),(6,1600)$ and the different values of $2\leq N\leq 5.$ Note
that these plots are fairly coherent with the predicted theoretical behaviour
of the spectrum of the random matrix $G_{N,n,2}^{(\alpha)},$ given by Theorem
1 and in particular by the lower and upper bounds (39) and (40).

Figure 1: Plots of the spectrum of $G_{N,n,2}^{(-0.5)},$ for (a)
$(d,n)=(4,900)$ (b) $(d,n)=(6,1600)$ and different values $N=2,3,4,5,$ (from
left to right).

Example 2: In this second example, we illustrate the performance of
$\widehat{f}_{N,n,m}^{(\alpha)},$ our proposed stable NP regression estimator,
that is based on least squares by means of multivariate Jacobi polynomials.
For this purpose, we have considered the NP regression problem (24) for the
special case of the dimension $d=4$ with a synthetic test true regression
function $f$, given by

$\displaystyle f(x,y,z,t)$ $\displaystyle=$ $\displaystyle
x+(2y-1)^{2}+\frac{\sin(2\pi z)}{2-\sin(2\pi z)}+0.1\,\sin(2\pi
t)+0.2\,\cos(2\pi t)$ (64) $\displaystyle\hskip 142.26378pt+0.3\,(\sin(2\pi
t))^{2}+0.4\,(\cos(2\pi t))^{3}+0.5\,(\sin(2\pi t))^{3}.$

Note that this test regression function corresponds to an additive
multidimensional regression model. Hence, $m=1$ is the appropriate value of
this parameter. Then, we have constructed our estimator
$\widehat{f}_{N,n,1}^{(\alpha)}$ with $\alpha=-\frac{1}{2},$ $n=900$ i.i.d.
random sampling points following a $4-$D $\mbox{Beta}(\alpha+1,\alpha+1)$
distribution. Also, we have considered the different values of $N=4,6,8,10$
together with a noise free model as well as noised models associated to two
different values of $\sigma=0.1,0.5.$ We have computed the empirical mean
squared error over a test random set of size $n$ with i.i.d. random points
$X_{i}$ following also a $4-$D $\mbox{Beta}(\alpha+1,\alpha+1)$ distribution.
This empirical mean squared error is given by

$MSE=\frac{1}{n}\sum_{i=1}^{n}\Big{(}\widehat{f}^{(\alpha)}_{N,n,1}(X_{i})-f(X_{i})\Big{)}^{2}.$

The obtained numerical results are given by the following Table 2.

| $\sigma=0$ | $\sigma=0.1$ | $\sigma=0.5$
---|---|---|---
$N$ | $MSE$ | $MSE$ | $MSE$
$4$ | $2.69e-2$ | $3.99e-2$ | $4.42e-2$
$6$ | $1.60e-2$ | $2.11e-2$ | $2.53e-2$
$8$ | $5.89e-3$ | $1.05e-2$ | $1.79e-2$
$10$ | $9.29e-4$ | $2.63e-3$ | $6.20e-2$
Table 2: Numerical simulations for test function (64).

Note that the numerical results given by Table 2 are coherent with the
theoretical $L^{2}-$risk and the approximation error of the proposed estimator
$\widehat{f}_{N,n,1}^{(\alpha)},$ given by Theorems 2 and 3. Also, note that
the loss of accuracy we have observed for the special values of $N=10$ and
$\sigma=0.5$ is due to the fact that for the given couple $(N,n)=(10,900),$
the $2-$condition number of the random matrix $G_{N,n,m}^{(\alpha)}$ is
relatively large to handle noised data with relatively large $\sigma=0.5$
Nonetheless, for $n=1600$ and the same values of the parameters, we have
obtained an $MSE\approx 1.16e-2.$

Example 3: In this last example, we consider the Kriging model test function
borrowed from [10] and given, for $x_{1},x_{2},x_{3},x_{4}\in[0,1]$, by

$f(x_{1},x_{2},x_{3},x_{4})=1+\exp\left[-2\big{(}(x_{1}-1)^{2}+x_{2}^{2}\big{)}-0.5\big{(}x_{3}^{2}+x_{4}^{2}\big{)}\right]+\exp\left[-2\big{(}x_{1}^{2}+(x_{2}-1)^{2}\big{)}-0.5\big{(}x_{3}^{2}+x_{4}^{2}\big{)}\right]\,.$
(65)

Then, we have considered the values of the parameters $m=2,$
$\alpha=-\frac{1}{2},$ $n=1600$ and constructed our estimator
$\widehat{f}_{N,n,2}^{(\alpha)},$ with $N=4,5,6.$ As in the previous example,
we have computed the different $MSE:$ the empirical mean squared errors for
the different values of the Gaussian noise standard deviation
$\sigma=0,\,0.1,\,0.5.$ These $MSE$ are computed by the use of a new set of
$n_{1}=400$ i.i.d. random sampling points following a multivariate
$\mbox{Beta}(\alpha+1,\alpha+1)$ distribution. The obtained numerical results
are given by Table 3.

| $\sigma=0$ | $\sigma=0.1$ | $\sigma=0.5$
---|---|---|---
$N$ | $MSE$ | $MSE$ | $MSE$
$4$ | $5.35e-3$ | $5.94e-3$ | $1.38e-2$
$5$ | $1.57e-3$ | $1.52e-2$ | $2.57e-2$
$6$ | $1.06e-3$ | $1.07e-2$ | $2.69e-2$
Table 3: Numerical simulations for test function (65) .

Note that these numerical results are also coherent with the theoretical
$L^{2}-$risk and the approximation error given by Theorems 2 and 3. For the
noise free model, that is $\sigma=0,$ the larger $N,$ the smallest is the
empirical mean squared error. For the noised versions of the model, that is
for $\sigma=0.1$ or $\sigma=0.5,$ the situation is slightly reversed. This is
due to the contribution of the variance term. According to Theorem 2, this
last quantity is affected by larger values of the parameter $N.$

## References

  * [1] B. Adcock and A. C. Hansen, A Generalized Sampling Theorem for Stable Reconstructions in Arbitrary Bases, J. Four. Anal. Appl., 18, (2012), 685–716.
  * [2] A. Alexanderian, On spectral methods for variance based sensitivity analysis, Probab. Surv., 10 (2013), 51–68.
  * [3] G. E. Andrews, R. Askey and R. Roy, Special Functions, Cambridge University Press , Cambridge, New York, 1999.
  * [4] N. Batir, Inequalities for the Gamma function, Arch. Math.,  91 (2008), 554–563.
  * [5] B. Bauer M. and Kohler, On deep learning as a remedy for the curse of dimensionality in nonparametric Regression. Ann. Statist., 47 (4) (2019), 2261-–2285.
  * [6] A. Ben Saber, S. Dabo and and A. Karoui, Multivariate nonparametric regression by least squares Jacobi polynomials approximations, submitted for publication (2022), available at arXiv:2202.01283
  * [7] G. Blatman and B. Sudret, Adaptive sparse polynomial chaos expansion based on Least Angle Regression, J. Comput. Phys., 230 (2011), 2345–2367.
  * [8] G. Blatman and B. Sudret, Efficient computation of global sensitivity indices using sparse polynomial chaos expansions, Reliab. Eng. Syst. Saf., 95 (11), (2010), 1216–1229.
  * [9] Y. De Castro, F. Gamboa, D. Henrion, R. Hess and J.B. Lasserre, Approximate optimal designs for multivariate polynomial regression Ann. Statist., 47(1) (2019), 127–155.
  * [10] W. Chen, R. Jin, A. Sudjianto, Analytical Variance-Based Global Sensitivity Analysis in Simulation-Based Design Under Uncertainty, J. Mech. Des. 127 (5), (2004), 875–886.
  * [11] A. Cohen and G. Migliorati, Optimal weighted least-squares methods, SMAI J. Comput. Math, 3 (2017), 181–203.
  * [12] A. Cohen, M.A. Davenport and D.Leviatan, On the stability and accuracy of least square approximations, Found. Comput. Math., 13 (5) (2013), 819–834.
  * [13] T. M. Cover and J. A. Thomas, Elements of Information Theory, Hoboken, New Jersey, Wiley, 2006.
  * [14] L. Guo, A. Narayan and T. Zhou, Constructing Least-Squares Polynomial Approximations, SIAM Review, 62 (2) (2020), 483–508.
  * [15] L. Gyorfi, M. Kohler, A. Krzyzak,and H. Walk, A Distribution-Free Theory of Nonparametric Regression, Springer, 2002.
  * [16] M. Hadigol and A. Doostan, Least Squares Polynomial Chaos Expansion: A Review of Sampling Strategies, Comput. Methods Appl. Mech. Eng., 332, (2018), 382–407.
  * [17] J. Hampton and A. Doostan, Coherence motivated sampling and convergence analysis of least squares polynomial chaos regression, Comput. Methods Appl. Mech. Eng., 290 (2015), 73–97.
  * [18] J. Hampton and A. Doostan, Compressive sampling of polynomial chaos expansions: Convergence analysis and sampling strategies, J. Comput. Phys., 280, (2015), 363– 386.
  * [19] R. A. Horn and C. R. Johnson, Matrix Analysis, second edition, Cambridge University Press, 2013.
  * [20] J. Z. Huang, Projection estimation in multiple regression with application to functional ANOVA models, Ann. Statist., 26 (1), (1998), 242–272.
  * [21] J. D. Jakeman, M. S. Eldred, K. Sargsyan, Enhancing $\ell_{1}$-minimization estimates of polynomial chaos expansions using basis selection, J. Comput. Phys., 289, (2015), 18–34.
  * [22] J. D. Jakeman, F. Franzelin, A. Narayan, M. Eldred and D. Plfüger, Polynomial chaos expansions for dependent random variables, Comput. Methods Appl. Mech. Eng.,  351, (2019), 643–666.
  * [23] A. Karoui and A. Souabni, Generalized Prolate Spheroidal Wave Functions: Spectral Analysis and Approximation of Almost Band-Limited Functions, J. Four. Anal. Appl., 22, (2016), 383-–412.
  * [24] Y. Lin and H. H. Zhang, Component Selection and Smoothing in Multivariate Nonparametric Regression, Ann. Stat., 26 (5) (2006), 2272–2297.
  * [25] Y. Lin, Tensor Product Space ANOVA Models, Ann. Stat.,  28 (3), (2000), 734–755.
  * [26] D. Loukrezis, A. Galetzka and H. De Gersem, Robust adaptive least squares polynomial chaos expansions in high-frequency applications, Int. J. Numer. Model. El., 33 (6), (2020), 15 pages.
  * [27] G. Migliorati, F. Nobile, E. Von Schwerin and R. Tempone, Analysis of Discrete $L^{2}$ Projection on Polynomial Spaces with Random Evaluations, Found. Comput. Math., 14, (2014), 419–456.
  * [28] A. Narayan, J. D. Jakeman and T. Zhou, A Christoffel function weighted least squares algorithm for collocation approximations, Math. Comp., 86, (2017), 1913–1947.
  * [29] Q. Pan, Q. Xingru, L. Leilei and D. Dias, A sequential sparse polynomial chaos expansion using Bayesian regression for geotechnical reliability estimations, Int. J. Numer. Anal. Methods Geomech., 44 (6), (2020), 874–889.
  * [30] D. Potts and M. Schmischke, Approximation of High-Dimensional Periodic Functions with Fourier-Based Methods, SIAM J. Num. Anal., 59 (5), (2021).
  * [31] R. Roy (1987) Binomial Identities and Hypergeometric Series, Am. Math. Mon., 94 (1), (1987), 36–46.
  * [32] A. Saltelli, P. Annoni, I. Azzini, F. Campolongo, M. Ratto, and S. Tarantola, Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index, Comput. Phys. Commun., 181, (2010), 259–270.
  * [33] Y. Shin and D. Xiu, On a near optimal sampling strategy for least squares polynomial regression, J. Comput. Phys., 326 (2016), 931–946.
  * [34] Y. Shin and D. Xiu, Nonadaptive quasi-optimal points selection for least squares linear regression, SIAM J. Sci. Comput., 38, (2016), 385–411.
  * [35] S. Smale and D. X. Zhou, Learning Theory Estimates via Integral Operators and Their Approximations, Constructive Approximation 26 (2) (2007), 153–172.
  * [36] I.M. Sobol, Sensitivity estimates for non-linear mathematical models, Mathematical Modelling and Computational Experiment., 1 (1993), 407–414;
  * [37] J. C. Stone, Optimal global rates of convergence for nonparametric regression. Ann. Statist., 10 (4) (1982), 1040-–1053.
  * [38] C. J. Stone, The use of polynomial splines and their tensor products in multivariate function estimation, Ann. Statist., 22 (1), (1994), 118–171.
  * [39] A. Tarakanov and A. H. Elsheikh, Regression-based sparse polynomial chaos for uncertainty quantification of subsurface flow models, J. Comput. Phys., 399, (2019), Article 108909.
  * [40] E. Torre, S. Marelli, P. Embrechts and B. Sudret, Data-driven polynomial chaos expansion for machine learning regression, J. Comput. Phys., 388 (2019), 601–623.
  * [41] J. A. Tropp, Matrix Concentration & Computational Linear Algebra, Caltech CMS Lecture Notes 2019-01, Pasadena, July 2019.
  * [42] N. Wiener, The homogeneous chaos, Amer. J. Math., 60, (1938), 897–936.
  * [43] D. Xiu and G. Karniadakis, The Wiener–Askey Polynomial Chaos for Stochastic Differential Equation, SIAM J. Sci. Comput., 24 (2), (2002), 619–644.
  * [44] T. Zhou, A. Narayan and D. Xiu, Weighted discrete least-squares polynomial approximation using randomized quadratures, J. Comput. Phys., 298 (2015), 787–800.