# Adaptive warped kernel estimation for nonparametric regression with circular
responses

Tien Dat Nguyen Laboratoire de Mathématiques, UMR 8628, Université Paris Sud,
91405 Orsay Cedex France; E-mail<EMAIL_ADDRESS>Thanh Mai Pham
Ngoc Laboratoire de Mathématiques, UMR 8628, Université Paris Sud, 91405 Orsay
Cedex France; E-mail<EMAIL_ADDRESS>Vincent Rivoirard
Ceremade, CNRS, UMR 7534, Université Paris-Dauphine, PSL Research University,
75016 Paris, France; E-mail<EMAIL_ADDRESS>
###### Abstract

In this paper, we deal with nonparametric regression for circular data,
meaning that observations are represented by points lying on the unit circle.
We propose a kernel estimation procedure with data-driven selection of the
bandwidth parameter. For this purpose, we use a warping strategy combined with
a Goldenshluger-Lepski type estimator. To study optimality of our methodology,
we consider the minimax setting and prove, by establishing upper and lower
bounds, that our procedure is nearly optimal on anisotropic Hölder classes of
functions for pointwise estimation. The obtained rates also reveal the
specific nature of regression for circular responses. Finally, a numerical
study is conducted, illustrating the good performances of our approach.

Keywords— circular data, nonparametric regression, warping method, kernel
rule, adaptive minimax estimation, Goldenshluger-Lepski procedure.

##  1 Introduction

Directional statistics is the branch of statistics which deals with
observations that are directions. In this paper, we will consider more
specifically circular data which arises whenever using a periodic scale to
measure observations. These data are represented by points lying on the unit
circle of $\mathbb{R}^{2}$ denoted in the sequel by $\mathbb{S}^{1}$. Circular
data are collected in many research fields, for example in ecology (animal
orientations), earth sciences (wind, ocean current directions, cross-bed
orientations to name a few), medicine (circadian rhythm), forensics (crime
incidence) or social science (clock or calendar effects). Various
comprehensive surveys on statistical methods for circular data can be found in
Mardia and Jupp [15], Jammalamadaka and SenGupta [11], Ley and Verdebout [14]
and recent advances are collected in Pewsey and García-Portugués [18]. Note
that the term circular data is also used to distinguish them from data
supported on the real line $\mathbb{R}$ (or some subset of it), which
henceforth are referred to as linear data.

In the present work, we focus on a nonparametric regression model with a
circular response and linear predictor. Assume that we have an independent
identically distributed (i.i.d in the sequel) sample
$\left\\{(X_{i},\Theta_{i})\right\\}_{i=1}^{n}$ distributed as $(X,\Theta)$,
where $\Theta$ is a circular random variable, and $X$ is a random variable
with density $f_{X}$ supported on $\mathbb{R}$ such that the cumulative
distribution function of $X$ is invertible on $\mathbb{R}$. It means that
$f_{X}$ is positive on $\mathbb{R}$ and $F_{X}(\mathbb{R})=(0,1)$. We aim at
estimating a function $m$ which contains the dependence structure between the
predictors $X_{i}$ and the observations $\Theta_{i}$.

Regression with circular response and linear covariates has been first and
mostly explored from a parametric point of view. Pioneered contributions are
due to Gould [10], Johnson and Wehlry [12] or Fisher and Lee [7]. The latter
proposed the most popular link-based function (namely the function $2\arctan$)
to model the conditional mean. Major difficulties, among others of such link-
based models involve computational drawbacks to estimate parameters as
identified by Presnell et al. [21]. Presnell et al. [21] in turn suggested
alternatively a spherically projected multivariate linear model. Since then,
numerous parametric approaches have been proposed, we refer the reader to all
the references in Pewsey and García-Portugués [18]. In order to get a more
flexible approach, nonparametric paradigm has been considered, first in the
pioneering work by Di Marzio et al. [16] and more recently in Meilán-Vila et
al. [17] for the multivariate setting. Surprisingly enough, the nonparametric
point of view has only been considered in very few papers. Note that contrary
to all works aforementioned which classically focus on the condition mean
(which is our goal as well) Alonso-Pena and Crujeiras [1] proposed a
nonparametric multimodal regression method for estimating the conditional
density when for instance the latter is highly skewed or multimodal.
Estimation procedures developed in [16] or [17] consist in estimating the
arctangent function of the ratio of the trigonometric moments of $\Theta$
(more details about this approach are given in the next section as it is the
starting point of our procedure). More precisely, in the case of pointwise
estimation and covariates supported on $[0,1]$, Di Marzio et al. [16]
investigated the performances of a Nadaraya-Watson and a local linear
polynomial estimators. Theoretically, for regression functions being twice
continuously differentiable, they obtained expressions for asymptotic bias and
variance. Their proofs are based on linearization of the function arctangent
by using Taylor expansions, but no sharp controls of the remainder terms in
the expansions are obtained. Actually obtaining such controls would be very
tedious with such an approach based on Taylor expansions. As for the more
recent work of Meilan Vila et al. [17], they studied the multivariate setting
$[0,1]^{d}$ with the same estimators and proofs technics. In both papers,
neither rates of convergence nor adaptation are obtained and cross-validation
is used to select the kernel bandwidth in practice. By adaptation, we mean
that the estimators do not require the specification of the regularity of the
regression function which is crucial from a practical point of view. In view
of this, we were motivated to fill the gap in the literature. Our goal is
twofold: obtaining optimal rates of convergence for predictors supported on
$\mathbb{R}$ and adaptation for estimating $m$ the regression function. To
achieve this, we propose a new strategy based on concentration inequalities
along with warping methods.

Our contributions. Assuming that the cumulative distribution function (c.d.f.)
of the design $X$ is known and invertible, warping methods used in this paper
consists in introducing the auxiliary function $g:=m\circ F_{X}^{<-1>}$, with
$F_{X}^{<-1>}$ the inverse of $F_{X}$. We then use classical kernel rules to
estimate the function $g$ in the specific framework of circular data. Our
procedure needs to select two bandwidths. Fully data-driven selection of
bandwidths is performed by using a Goldenshluger-Lepski type procedure [9].
Then, theoretical performances are studied. We consider the minimax setting
and prove by establishing upper and lower bounds that our procedure is nearly
optimal on anisotropic Hölder classes of functions for pointwise estimation.
These results are stated in Theorems 3.5 and 3.10 respectively. Then, we
conduct a numerical study whose goal is twofold. We first investigate the best
tuning parameters of our procedure. Once tuned, our estimates are used on
artificial data and compared to other classical methods. The numerical study
reveals the good performances of our methodology.

Plan. In section 2, we explain how to take into account the circular nature of
the response and then propose our data-driven kernel estimator of the
regression function $m$ based on warping strategy and the Goldenshluger-Lepski
bandwidth selection rule. Section 3 contains the theoretical results. Section
4 presents numerical results including simulations. Finally, all the proofs
are deferred to Section 5.

Notations. It is necessary to equip the reader with some notations. In the
sequel, a point on $\mathbb{S}^{1}$ will not be represented as a two-
dimensional vector $\mathbb{w}=(w_{2},w_{1})^{\top}$ with unit Euclidean norm
but as an angle $\theta=atan2(w_{1},w_{2})$ defined as follows:

######  Definition 1.1.

The function $\mbox{atan2}:\mathbb{R}^{2}\setminus(0,0)\mapsto[-\pi;\pi]$ is
defined for any $(w_{1},w_{2})\in\mathbb{R}^{2}\setminus(0,0)$ by __

$\mbox{atan2}(w_{1},w_{2}):=\left\\{\begin{array}[]{ll}\arctan\big{(}\frac{w_{1}}{w_{2}}\big{)}&\quad\mbox{if
}\quad w_{2}\geq 0,w_{1}\neq 0\\\ {0}&\quad{\mbox{if }\quad
w_{2}>0,w_{1}=0}\\\ \arctan\big{(}\frac{w_{1}}{w_{2}}\big{)}+\pi&\quad\mbox{if
}\quad w_{2}<0,w_{1}>0\\\
\arctan\big{(}\frac{w_{1}}{w_{2}}\big{)}-\pi&\quad\mbox{if }\quad
w_{2}<0,w_{1}\leq 0,\end{array}\right.$

_with $\arctan$ taking values in $[-\pi/2,\pi/2]$. In particular
$\arctan(+\infty)=\pi/2$ and $\arctan(-\infty)=-\pi/2$._

In this definition, one has arbitrarily fixed the origin of $\mathbb{S}^{1}$
at $(1,0)^{\top}$ and uses the anti-clockwise direction as positive. Thus, a
circular random variable can be represented as angle over $[-\pi,\pi)$.
Observe that $\mbox{atan2}(0,0)$ is not defined. Hereafter,
$\left\|\cdot\right\|_{\mathbb{L}^{1}(\mathbb{R})}$ and
$\left\|\cdot\right\|_{\mathbb{L}^{2}(\mathbb{R})}$ respectively denote the
$\mathbb{L}^{1}$ and $\mathbb{L}^{2}$ norm on $\mathbb{R}$ with respect to the
Lebesgue measure:

$\left\|f\right\|_{\mathbb{L}^{1}(\mathbb{R})}=\int_{\mathbb{R}}|f(y)|dy,\quad\left\|f\right\|_{\mathbb{L}^{2}(\mathbb{R})}=\Big{(}\int_{\mathbb{R}}|f(y)|^{2}dy\Big{)}^{1/2}.$

The $\mathbb{L}^{\infty}$ norm is defined by
$\left\|f\right\|_{\infty}=\sup_{y\in\mathbb{R}}|f(y)|$. Moreover, we denote
$*$ the classical convolution defined for functions $f,g$ by
$f*g(x):=\int_{\mathbb{R}}f(x-y).g(y)dy$, for $x\in\mathbb{R}$. Finally, for
$\alpha\in\mathbb{R}$, $[\alpha]_{+}:=\max\left\\{\alpha;0\right\\}$ and for
$\beta>0$, $\lfloor\beta\rfloor$ denotes the largest integer strictly smaller
than $\beta$.

##  2 The estimation procedure

We denote $F_{X}:y\in\mathbb{R}\mapsto\mathbb{P}(X\leq y)$ the cumulative
distribution function (c.d.f.) of $X$ and in the sequel, we assume that
$F_{X}$ is known and is invertible, so that $F_{X}(\mathbb{R})=(0,1)$.

After recalling the framework of circular data in Section 2.1, Section 2.2 is
devoted to the construction of an estimator for $m(x)$, at a given point
$x\in\mathbb{R}$ which will be fixed along this paper, using warped kernel
methods. Then, Section 2.3 presents a data-driven procedure for bandwidth
selection by using the Goldenshluger-Lepski methodology.

###  2.1 The framework of circular data

There is no doubt that, due to their periodic nature, circular data are
fundamentally different from linear ones, and thus need specific tools. To
measure the closeness between two angles, we use the angular distance

$d_{c}(\theta_{1},\theta_{2}):=\big{(}1-\cos(\theta_{1}-\theta_{2})\big{)},\quad\theta_{1},\theta_{2}\in[-\pi,\pi).$

Notice that this angular distance corresponds to the usual Euclidean norm in
$\mathbb{R}^{2}$. Indeed, the angles $\theta_{1}$ and $\theta_{2}$ determine
the corresponding points $(\cos\theta_{1},\sin\theta_{1})$ and
$(\cos\theta_{2},\sin\theta_{2})$ respectively on the unit circle
$\mathbb{S}^{1}$. Then, the usual squared Euclidean norm in $\mathbb{R}^{2}$
reads

$\displaystyle\big{(}\cos\theta_{1}-\cos\theta_{2}\big{)}^{2}+\big{(}\sin\theta_{1}-\sin\theta_{2}\big{)}^{2}=2.\big{[}1-\cos(\theta_{1}-\theta_{2})\big{]}=2.d_{c}(\theta_{1},\theta_{2}).$

Hence, using the angular distance, we look for a function $m$ minimizing the
following risk:

$\displaystyle L(m(X)):=\mathbb{E}\big{[}1-\cos(\Theta-m(X))\big{|}X\big{]}.$

Minimizing $\mathbb{E}\big{[}1-\cos(\Theta-m(X))|X\big{]}$ with respect to
$m(X)\in[-\pi,\pi)$ is equivalent to maximizing
$\mathbb{E}\big{[}\cos(\Theta-m(X))|X\big{]}$ with respect to
$m(X)\in[-\pi,\pi)$. Now for $x\in\mathbb{R}$, let

$m_{1}(x):=\mathbb{E}\big{(}\sin(\Theta)|X=x\big{)}\quad and\quad
m_{2}(x):=\mathbb{E}\big{(}\cos(\Theta)|X=x\big{)}.$ (1)

Moreover, write

$\displaystyle\mathbb{E}\big{[}\cos(\Theta-m(X))|X\big{]}$
$\displaystyle=\cos(m(X)).m_{2}(X)+\sin(m(X)).m_{1}(X)$
$\displaystyle=\sqrt{(m_{2}(X))^{2}+(m_{1}(X))^{2}}.\cos(m(X)-\gamma),$

where $\gamma\in[-\pi,\pi)$ is defined by

$\cos(\gamma):=\dfrac{m_{2}(X)}{\sqrt{(m_{2}(X))^{2}+(m_{1}(X))^{2}}}\quad\textrm{
and
}\quad\sin(\gamma):=\dfrac{m_{1}(X)}{\sqrt{(m_{2}(X))^{2}+(m_{1}(X))^{2}}}.$

Thus, we have

$\underset{m(X)\in[-\pi,\pi)}{\textrm{argmin}}L(m(X))=\underset{m(X)\in[-\pi,\pi)}{\textrm{argmax}}\cos(m(X)-\gamma)=\gamma=\mbox{atan2}\big{(}m_{1}(X),m_{2}(X)\big{)}.$
(2)

In conclusion, the circular nature of the response is taken into account by
the arctangent of the ratio of the conditional expectation of sine and cosine
components of $\Theta$ given $X$ and we tackle the problem by estimating the
function

$m(x)=\mbox{atan2}(m_{1}(x),m_{2}(x)),\quad x\in\mathbb{R},$

with $m_{1}$ and $m_{2}$ defined in (1). For this purpose, we use warped
estimators.

###  2.2 Warping strategy

The popular Nadaraya-Watson (NW) methodology provides a natural estimator of
$m$ of the form

$\widehat{m}^{NW}_{h}:x\longmapsto\dfrac{\frac{1}{n}\sum_{j=1}^{n}\Theta_{j}.K_{h}(x-X_{j})}{\frac{1}{n}\sum_{j=1}^{n}K_{h}(x-X_{j})},$

with $K:\mathbb{R}\rightarrow\mathbb{R}$ such that $\int_{\mathbb{R}}K(y)dy=1$
and $K_{h}(\cdot):=\frac{1}{h}K(\frac{\cdot}{h})$, for some bandwidth $h>0$.
However, on the one hand, the denominator which can be small may lead to some
instability. On the other hand, as adaptive estimation requires the data-
driven selection of the bandwidth, the ratio form of the NW estimate indicates
that we should select two bandwidths: one for the numerator and one for the
denominator. Consequently, considering NW estimators for $m_{1}$ and $m_{2}$
involve four bandwidths. This makes the study of these estimators quite
intricate.

Warping methods consist in introducing the auxiliary function $g:=m\circ
F_{X}^{<-1>}$, with $F_{X}^{<-1>}$ the inverse of $F_{X}$. The strategy then
boils down to first estimating $g$ by say $\widehat{g}$ and then estimating
the regression function of interest $m$ by $\widehat{g}\,\circ F_{X}$. To deal
with regression with random design, the warping strategy has been applied for
instance by Kerkyacharian and Picard [13], Pham Ngoc [19], Chagny [3] and
Chagny et al. [5]. Among the advantages of this method, let us mention that a
warped kernel estimator does not involve a ratio, which strengthens its
stability whatever the design distribution, even when the design is
inhomogeneous. Our fully data-driven approach is based on the selection of two
bandwidths that adapt automatically to the unknown smoothness of functions
$g_{1}$ and $g_{2}$.
We propose to adapt the strategy developed in the linear case by Chagny et al.
in [5]. The warping device is based on the transformation $F_{X}(X_{i})$ of
the data $X_{i}$, $i=1,\ldots,n$. We first define kernels considered in our
framework as follows.

######  Definition 2.1.

_Let $K:\mathbb{R}\rightarrow\mathbb{R}$ be an integrable function such that
$K$ is compactly supported,
$K\in\mathbb{L}^{\infty}(\mathbb{R})\cap\mathbb{L}^{1}(\mathbb{R})\cap\mathbb{L}^{2}(\mathbb{R})$.
We say that $K$ is a kernel if it satisfies $\int_{\mathbb{R}}K(y)dy=1$. _

Now, let $g_{1},g_{2}:(0,1)\longmapsto\mathbb{R}$ be defined by

$g_{1}:=m_{1}\circ F_{X}^{<-1>},\quad\textrm{and}\quad g_{2}:=m_{2}\circ
F_{X}^{<-1>},$

so that $m_{1}=g_{1}\circ F_{X}$ and $m_{2}=g_{2}\circ F_{X}$. Furthermore,
setting for $u\in(0,1)$,

$\displaystyle g(u):=\mbox{atan2}\big{(}g_{1}(u),g_{2}(u)\big{)},$

we observe that $m=g\circ F_{X}$. Now, for $u\in(0,1)$, we estimate $g_{1}(u)$
and $g_{2}(u)$ by

$\widehat{g}_{1,h_{1}}(u):=\dfrac{1}{n}\sum_{i=1}^{n}\sin(\Theta_{i}).K_{h_{1}}(u-F_{X}(X_{i}))\quad\textrm{and}\quad\widehat{g}_{2,h_{2}}(u):=\dfrac{1}{n}\sum_{i=1}^{n}\cos(\Theta_{i}).K_{h_{2}}(u-F_{X}(X_{i}))$
(3)

respectively, where $h_{1},h_{2}>0$ are bandwidths of kernels
$K_{h_{1}}(\cdot)$ and $K_{h_{2}}(\cdot)$ respectively.
Now, we estime $g$ by

$\widehat{g}_{h}(u):=\mbox{atan2}\big{(}\widehat{g}_{1,h_{1}}(u),\widehat{g}_{2,h_{2}}(u)\big{)},\quad
u\in(0,1),$ (4)

where we denote $h:=(h_{1},h_{2})$. Moreover, as a consequence, for
$x\in\mathbb{R}$, the estimators for $m_{1}$ and $m_{2}$ are

$\widehat{m}_{1,h_{1}}(x):=\widehat{g}_{1,h_{1}}\big{(}F_{X}(x)\big{)}=\dfrac{1}{n}\sum_{i=1}^{n}\sin(\Theta_{i}).K_{h_{1}}\big{(}F_{X}(x)-F_{X}(X_{i})\big{)},$
(5)

and

$\widehat{m}_{2,h_{2}}(x):=\widehat{g}_{2,h_{2}}\big{(}F_{X}(x)\big{)}=\dfrac{1}{n}\sum_{i=1}^{n}\cos(\Theta_{i}).K_{h_{2}}\big{(}F_{X}(x)-F_{X}(X_{i})\big{)}.$
(6)

We then obtain an estimator of $m(x)$ at $x\in\mathbb{R}$ by setting

$\widehat{m}_{h}(x):=\mbox{atan2}\big{(}\widehat{m}_{1,h_{1}}(x),\widehat{m}_{2,h_{2}}(x)\big{)}=\widehat{g}_{h}\big{(}F_{X}(x)\big{)}.$

###  2.3 Bandwidth selection

To study the pointwise quadratic-risk of the estimator $\widehat{m}_{h}(x)$
whose expression is

$\mathbb{E}\Big{[}\big{(}\widehat{m}_{h}(x)-m(x)\big{)}^{2}\Big{]}=\mathbb{E}\Big{[}\big{(}\widehat{g}_{h}(F_{X}(x))-g(F_{X}(x))\big{)}^{2}\Big{]},$

we first focus on the estimator $\widehat{g}_{h}$ of $g$ by studying the
adaptive choice of bandwidths belonging to a convenient grid
$\mathcal{H}_{n}$. To define the latter, we assume that the kernel $K$
satisfies $\mbox{supp}(K)\subseteq[-A,A]$ for some $A>0$ and we take
$h_{\max}$ a constant such that $F_{X}(x)-A.h_{\max}>0$ and
$F_{X}(x)+A.h_{\max}<1$. Then, we set

$\mathcal{H}_{n}:=\left\\{h=k^{-1}:\ k\in{\mathbb{N}}^{*},\ h\leq h_{\max},\
n.h>\max\Big{(}\dfrac{\left\|K\right\|_{\mathbb{L}^{2}(\mathbb{R})}^{2}}{\left\|K\right\|_{\infty}^{2}};1\Big{)}.\log(n)\right\\}.$
(7)

We have $\mbox{Card}\big{(}\mathcal{H}_{n}\big{)}\approx n/\log n$. In the
following, we apply the method proposed by Goldenshluger and Lepski in [9] to
select an optimal value for bandwidths $h_{1}$ and $h_{2}$ automatically. Let
$j\in\\{1,2\\}$. For $h_{j}\in\mathcal{H}_{n}$ and $v\in(0,1)$ we set

$A_{j}(h_{j},v):=\underset{h_{j}^{\prime}\in\mathcal{H}_{n}}{\sup}\Big{\\{}\big{|}\widehat{g}_{j,h_{j},h_{j}^{\prime}}(v)-\widehat{g}_{j,h_{j}^{\prime}}(v)\big{|}-\sqrt{\widetilde{V}_{j}(n,h_{j}^{\prime})}\Big{\\}}_{+},$
(8)

with
$\widetilde{V}_{j}(n,h_{j}^{\prime}):=c_{0,j}.\dfrac{\log(n).\left\|K\right\|_{\mathbb{L}^{2}(\mathbb{R})}^{2}}{n.h_{j}^{\prime}}$,
$c_{0,j}>0$ a tuning parameter and

$\widehat{g}_{j,h_{j},h_{j}^{\prime}}(v):=\big{(}K_{h_{j}^{\prime}}*\widehat{g}_{j,h_{j}}\big{)}(v),$

so that
$\widehat{g}_{j,h_{j},h_{j}^{\prime}}(v)=\widehat{g}_{j,h_{j}^{\prime},h_{j}}(v)$.
Then, a data-driven choice of bandwidth $h_{j}$ is performed as follows:

$\widehat{h}_{j}=\underset{h_{j}\in\mathcal{H}_{n}}{\textrm{argmin}}\Big{\\{}A_{j}(h_{j},v)+\sqrt{\widetilde{V}_{j}(n,h_{j})}\Big{\\}}.$
(9)

The criterion (9) is inspired from [9], in order to mimic the optimal ”bias-
variance” trade-off in the decomposition

$\mathbb{E}[|\widehat{g}_{j,h_{j}}(v)-g_{j}(v)|^{2}]=|\mathbb{E}[\widehat{g}_{j,h_{j}}(v)]-g_{j}(v)|^{2}+\mathbb{E}[|\widehat{g}_{j,h_{1}}(v)-\mathbb{E}[\widehat{g}_{j,h_{j}}(v)]|^{2}]=:b^{2}(h_{j},v)+V(h_{j},v).$

It is common to use $\widetilde{V}_{j}(n,h_{j}^{\prime})$ to provide an upper
bound for the variance term $V(h_{j},v)$ (see Section 5.2), whereas the more
involved task of the Goldenshluger-Lepski method is to provide an estimate for
the bias term by comparing pair-by-pair several estimators. In our framework,
the bias term corresponds to

$b(h_{j},v)=|\mathbb{E}[\widehat{g}_{j,h_{j}}(v)]-g_{j}(v)|=\big{|}\big{(}K_{h_{j}}*g_{j}\big{)}(v)-g_{j}(v)\big{|},$

(see (22)), so it is natural to estimate it by an estimator of the form
$\big{|}\big{(}K_{h_{j}}*\widehat{g}_{j,h_{j}^{\prime}}\big{)}(v)-\widehat{g}_{j,h_{j}^{\prime}}(v)\big{|}$.
Thus, the estimator of the bias term is $A_{j}(h_{j},v)$, defined in (8),
where the second term $\sqrt{\widetilde{V}_{j}(n,h_{j}^{\prime})}$ controls
the fluctuations of the first term.
Now, we define the kernel estimator of $g(v)$ with data-driven bandwidths as
follows:

$\widehat{g}_{\widehat{h}}(v):=\mbox{atan2}\big{(}\widehat{g}_{1,\widehat{h}_{1}}(v),\widehat{g}_{2,\widehat{h}_{2}}(v)\big{)},$
(10)

where we denote $\widehat{h}:=(\widehat{h}_{1},\widehat{h}_{2})$, and

$\widehat{g}_{1,\widehat{h}_{1}}(v):=\dfrac{1}{n}\sum_{i=1}^{n}\sin(\Theta_{i}).K_{\widehat{h}_{1}}\big{(}v-F_{X}(X_{i})\big{)},\hskip
5.69046pt\textrm{ and }\hskip
5.69046pt\widehat{g}_{2,\widehat{h}_{2}}(v):=\dfrac{1}{n}\sum_{i=1}^{n}\cos(\Theta_{i}).K_{\widehat{h}_{2}}\big{(}v-F_{X}(X_{i})\big{)}.$
(11)

As a consequence, at $x\in\mathbb{R}$, the adaptive estimators for $m_{1}$ and
$m_{2}$ are

$\widehat{m}_{1,\widehat{h}_{1}}(x):=\big{(}\widehat{g}_{1,\widehat{h}_{1}}\circ
F_{X}\big{)}(x)=\dfrac{1}{n}\sum_{i=1}^{n}\sin(\Theta_{i}).K_{\widehat{h}_{1}}\big{(}F_{X}(x)-F_{X}(X_{i})\big{)},$
(12)

and

$\widehat{m}_{2,\widehat{h}_{2}}(x):=\big{(}\widehat{g}_{2,\widehat{h}_{2}}\circ
F_{X}\big{)}(x)=\dfrac{1}{n}\sum_{i=1}^{n}\cos(\Theta_{i}).K_{\widehat{h}_{2}}\big{(}F_{X}(x)-F_{X}(X_{i})\big{)}.$
(13)

We finally define the adaptive estimator for $m(x)$ at $x\in\mathbb{R}$ by

$\widehat{m}_{\widehat{h}}(x):=\mbox{atan2}\big{(}\widehat{m}_{1,\widehat{h}_{1}}(x),\widehat{m}_{2,\widehat{h}_{2}}(x)\big{)}.$
(14)

##  3 Theoretical results

###  3.1 Minimax rates of convergence

The minimax approach is a framework that shows the optimality of an estimate
among all possible estimates. The minimax pointwise quadratic risk for the
estimator
$\widehat{g}_{\widehat{h}}:=\mbox{atan2}\big{(}\widehat{g}_{1,\widehat{h}_{1}},\widehat{g}_{2,\widehat{h}_{2}}\big{)}$
will be derived from the following control of the pointwise quadratic risks of
$\widehat{g}_{1,\widehat{h}_{1}}$ and $\widehat{g}_{2,\widehat{h}_{2}}$.

######  Proposition 3.1.

Consider the collection of bandwidths $\mathcal{H}_{n}$ defined in (7). Let
$j\in\\{1,2\\}$ and $q\geq 1$ and assume that
$\min\left\\{c_{0,1};c_{0,2}\right\\}\geq
16\big{(}2+q\big{)}^{2}.\big{(}1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}\big{)}^{2}$.
Then, for $n$ sufficiently large, with a probability larger than $1-4.n^{-q}$,

$\big{|}\widehat{g}_{j,\widehat{h}_{j}}(F_{X}(x)-g_{j}(F_{X}(x))\big{|}\leq\inf_{h_{j}\in\mathcal{H}_{n}}\Big{\\{}\big{(}1+2.\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}\big{)}.\left\|g_{j}-K_{h_{j}}*g_{j}\right\|_{\infty}+3.\sqrt{\widetilde{V}_{j}(n,h_{j})}\Big{\\}}.$

The proof of Proposition 3.1 is given in Section 5.4.1. Roughly speaking, in
view of results of Section 5.1, the right hand side of the inequality stated
in Proposition 3.1 may be viewed as the bias-variance decomposition of the
pointwise quadratic-risk of the best warped-kernel estimate, up to a
logarithmic term.

Since the function $\mbox{atan2}(w_{1},w_{2})$ is undefined when
$w_{1}=w_{2}=0$, it is reasonable to consider the following assumption:

######  Assumption 3.2.

_Assume that_

$|m_{1}(x)|=\big{|}g_{1}(F_{X}(x))\big{|}\geq\delta_{1}>0,\quad\textrm{ and
}\quad|m_{2}(x)|=\big{|}g_{2}(F_{X}(x))\big{|}\geq\delta_{2}>0.$

In the minimax setting, we need some assumptions on the regularity of $g_{1}$
and $g_{2}$. Thus, we introduce the following Hölder classes that are adapted
to local estimation.

######  Definition 3.3.

_Let $\beta>0$ and $L>0$. The Hölder class $\mathcal{H}(\beta,L)$ is the set
of functions $f:(0,1)\longmapsto\mathbb{R}$, such that $f$ admits derivatives
up to the order $\lfloor\beta\rfloor$, and for any
$(y,\widetilde{y})\in(0,1)^{2}$,_

$\displaystyle\left|\dfrac{d^{\lfloor\beta\rfloor}f}{(dy)^{\lfloor\beta\rfloor}}(\widetilde{y})-\dfrac{d^{\lfloor\beta\rfloor}f}{(dy)^{\lfloor\beta\rfloor}}(y)\right|\leq
L.\big{|}\widetilde{y}-y\big{|}^{\beta-\lfloor\beta\rfloor}.$

We also consider the following assumption on the kernel $K$:

######  Assumption 3.4.

_The kernel $K$ is of order $\mathcal{L}\in\mathbb{R}_{+}$, i.e._

  1. (i)

$C_{K,\mathcal{L}}:=\int_{\mathbb{R}}(1+|y|)^{\mathcal{L}}.|K(y)|dy<\infty$_;_

  2. (ii)

$\forall k\in\left\\{1,...,\lfloor\mathcal{L}\rfloor\right\\}$_,_
$\int_{\mathbb{R}}y^{k}.K(y)dy=0$.

Now, we obtain an upper bound for the pointwise quadratic risk of our final
fully data-driven estimator $\widehat{m}_{\widehat{h}}$ at $x$ defined in
(14):

######  Theorem 3.5.

Let $\beta_{1},\beta_{2},L_{1},L_{2}>0$. Suppose that $g_{1}$ belongs to
$\mathcal{H}(\beta_{1},L_{1})$, $g_{2}$ belongs to
$\mathcal{H}(\beta_{2},L_{2})$, the kernel K satisfies Assumption 3.4 with an
index $\mathcal{L}\in\mathbb{R}_{+}$ such that
$\mathcal{L}\geq\max(\beta_{1},\beta_{2})$. Let $q\geq 1$, and suppose that
$\min\left\\{c_{0,1};c_{0,2}\right\\}\geq
16\big{(}2+q\big{)}^{2}.\big{(}1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}\big{)}^{2}$.
Then, under Assumption 3.2, for $n$ sufficiently large,

$\mathbb{E}\Big{[}\big{|}\widehat{m}_{\widehat{h}}(x)-m(x)\big{|}^{2}\Big{]}\leq\Big{(}\dfrac{1}{\delta_{2}^{2}}+\dfrac{1}{\delta_{1}^{2}}\Big{)}.(C_{1}^{2}+C_{2}^{2}).\max\left\\{\psi_{n}^{2}(\beta_{1}),\psi_{n}^{2}(\beta_{2})\right\\},$

where

$\psi_{n}(\beta_{1})=\big{(}\log(n)/n\big{)}^{\frac{\beta_{1}}{2\beta_{1}+1}},\quad\psi_{n}(\beta_{2})=\big{(}\log(n)/n\big{)}^{\frac{\beta_{2}}{2\beta_{2}+1}},$

$C_{1}$ is a constant depending on $\beta_{1},L_{1},c_{0,1},K$ and $C_{2}$ is
a constant depending on $\beta_{2},L_{2},c_{0,2},K$.

A proof of Theorem 3.5 is given in Section 5.4.2. Observe that if
$\beta_{1}=\beta_{2}=\beta$, then we obtain the rate $\psi_{n}(\beta)=(\log
n/n)^{\beta/(2\beta+1)}$, which is the optimal rate for adaptive univariate
regression function estimation and pointwise risk (see e.g. Section 2 in [2]).
In this case, the obtained rate is much faster than the rate $(\log
n/n)^{\beta/(2\beta+2)}$ obtained for bivariate regression function estimation
in the isotropic case. This justifies the introduction of specific
methodologies for circular regression.

######  Remark 3.6.

_Eventually, to obtain the fully computable estimator, we replace the c.d.f.
$F_{X}$ by its natural estimate
$\widehat{F}_{n}(y):=n^{-1}\sum_{i=1}^{n}1_{X_{i}\leq y}$. Following arguments
of [4], this replacement should not change the final rate of convergence of
our nonparametric estimator. _

###  3.2 Minimax lower bounds

To establish minimax lower bounds, we need to consider a statistical model
which relates the $\Theta_{i}$’s and $X_{i}$’s, namely we assume:

$\Theta_{i}=m(X_{i})+\zeta_{i}\quad(\textrm{mod }2\pi),\quad i=1,\ldots,n$
(15)

where $m$ is a circular regression function and the $\zeta_{i}$’s are centered
i.i.d. random angles, independent of the $X_{i}$’s. We will assume that Model
(15) satisfies the following assumption.

######  Assumption 3.7.

The design points $X_{i}$’s are i.i.d. random variables with density $\mu(.)$
on $[0,1]$ such that there exists $\mu_{0}<\infty$ and $\mu(x)\leq\mu_{0}$
$\forall x\in[0,1]$ and the errors $\zeta_{i}$ have common density
$p_{\zeta}(.)$ on $\mathbb{S}^{1}$ with respect to the Lebesgue measure on
$\mathbb{S}^{1}$, verifying

$\exists\,p_{*}>0,\exists\,y_{0}>0:\int
p_{\zeta}(u)\log\frac{p_{\zeta}(u)}{p_{\zeta}(u+y)}du\leq
p_{*}y^{2},\;\forall|y|\leq y_{0}.$ (16)

In the sequel, the function $m$ belongs to the class $\tilde{\Sigma}(\beta,L)$
defined as the set of functions $f:[0,1]\longmapsto\mathbb{S}^{1}$ such that
the derivative $f^{(l)}$, $l=\lfloor\beta\rfloor$ exists and verifies

$|f^{(l)}(x)-f^{(l)}(y)|\leq L|x-y|^{\beta-l},\quad\forall x,y\in[0,1].$

######  Remark 3.8.

For two classes of functions $\mathcal{D}$ and $\mathcal{D}^{\prime}$ such
that $\mathcal{D}\subset\mathcal{D}^{\prime}$, a lower bound for the minimax
rate of convergence for $\mathcal{D}$ will also be a lower bound for the
minimax rate for $\mathcal{D}^{\prime}$. Hence, this justifies the restriction
of the study of the lower bound to circular functions $m$ defined on $[0,1]$.

######  Remark 3.9.

The classical von Mises density satisfies condition (16). See Equation (20)
where we recall the definition of the von Mises density.

We obtain the following lower bound:

######  Theorem 3.10.

Let $\beta>0$ and $L>0$. Under Assumptions 3.7, we have

$\liminf_{n\rightarrow\infty}\inf_{T_{n}}\sup_{m\in\tilde{\Sigma}(\beta,L)}\mathbb{E}\big{[}n^{\frac{2\beta}{2\beta+1}}(T_{n}(x)-m(x))^{2}\big{]}\geq
c,$

where $c$ depends only on $\beta,L,p_{*}$ and $\mu_{0}$ and the infimum is
taken over all possible estimates based on observations
$(\Theta_{i},X_{i})_{i=1,\ldots,n}$.

According to Remark 3.8, Theorem 3.10 entails that the lower bound for the
minimax risk for functions $m:\mathbb{R}\longmapsto\mathbb{S}^{1}$ such that
$m\in\mathcal{H}(\beta,L)$ is $n^{-\frac{2\beta}{2\beta+1}}$. Now let us
connect this result to the upper bound obtained in Theorem 3.5. As the
function $atan2$ is infinitely differentiable on
$\mathbb{R}^{*}\times\mathbb{R}^{*}$, and if $F_{X}$ is smoother than $g_{1}$
and $g_{2}$, then if one writes $m(x)=atan2(g_{1}(F_{X}(x)),g_{2}(F_{X}(x))$,
the smoothness $\beta$ of $m$ will be the minimum of the smoothness of $g_{1}$
and the smoothness of $g_{2}$. This shows that the rate achieved in Theorem
3.5 by our estimator is optimal up to a $\log$ factor. We conjecture that the
logarithmic term is unavoidable in minimax rates even if it remains an open
problem.

##  4 Numerical simulations

In this section, we implement some simulations to study the numerical
performances of our procedure. We consider three different regression models:

$\displaystyle\textrm{ M1}.\quad\Theta$
$\displaystyle=\mbox{atan2}\big{(}2X-1,X^{2}+2\big{)}+\zeta\quad\textrm{(mod
$2\pi$)},$ (17) $\displaystyle\textrm{ M2}.\quad\Theta$
$\displaystyle=\mbox{atan2}\big{(}-2X+1,X^{2}-1\big{)}+\zeta\quad\textrm{(mod
$2\pi$)},$ (18) $\displaystyle\textrm{ M3}.\quad\Theta$
$\displaystyle=\arccos\big{(}X^{5}-1\big{)}+3.\arcsin\big{(}X^{3}-X+1\big{)}+\zeta\quad\textrm{(mod
$2\pi$)},$ (19)

where the circular error, $\zeta$, is distributed according to a von Mises
distribution $vM(0,10)$ and is independent from $X$. Recall that the von Mises
distribution $vM(\widetilde{\mu},\widetilde{\kappa})$ has density

$\theta\in[-\pi,\pi)\longmapsto\dfrac{1}{2\pi.I_{0}(\widetilde{\kappa})}.\exp\big{(}\widetilde{\kappa}.\cos(\theta-\widetilde{\mu})\big{)},$
(20)

with $I_{0}(\widetilde{\kappa})$ the modified Bessel function of the first
kind and of order $0$111The modified Bessel function of the first kind and of
order $\alpha\geq 0$, $I_{\alpha}(z)$ for $z>0$, admits the integral
representation  $\displaystyle
I_{\alpha}(z)=\frac{1}{\pi}\int_{0}^{\pi}\exp\big{(}z.\cos(\theta)\big{)}.\cos(\alpha\theta)d\theta-\frac{\sin(\alpha\pi)}{\pi}\int_{0}^{\infty}\exp\big{(}-z.\cosh(y)-\alpha
y\big{)}dy.$ , a location parameter $\widetilde{\mu}\in[-\pi,\pi)$ and
concentration $\widetilde{\kappa}>0$. See e.g [14, Section 2.2.2]. In the
sequel, for Models M1 and M2, we consider two cases: $X\sim U([-5,5])$ and
$X\sim\mathcal{N}(0,1.5)$. For model M3, we consider $X\sim U([0,1])$. Then,
for different values of $n$, we draw a sample
$\big{(}\Theta_{i},X_{i}\big{)}_{i=1,\ldots,n}$ with the same distribution as
$(\Theta,X)$. To implement the Goldenshluger-Lepski methodology, we shall
consider either the Gaussian kernel defined by $y\longmapsto
K(y)=\dfrac{1}{\sqrt{2\pi}}.e^{-\frac{y^{2}}{2}},$ or the Epanechnikov kernel
$K$ defined by $y\longmapsto K(y)=\dfrac{3}{4}.(1-y^{2}).1_{|y|\leq 1}.$
Moreover, we consider the following collection of bandwidths $\mathcal{H}_{n}$
defined as

$\mathcal{H}_{n}:=\left\\{k^{-1}:k\in\mathbb{N},1\leq
k\leq\dfrac{n}{\log(n)}\right\\}.$

Finally, we make simulations for the general case of unknown design
distribution, i.e. the final estimators are computed using
$y\in\mathbb{R}\longmapsto\widehat{F}_{n}(y):=\dfrac{1}{n}\sum_{j=1}^{n}1_{X_{j}\leq
y}$ instead of $F_{X}$.

###  4.1 Practical calibration of tuning parameters

In the bandwidth selection procedure described in Section 2.3, we need to tune
two parameters $c_{0,1}$ and $c_{0,2}$ in order to find an optimal value of
the quadratic pointwise risk

$\mathcal{R}:=\big{|}\widehat{m}_{\widehat{h}}(x)-m(x)\big{|}^{2}$ (21)

with
$\widehat{m}_{\widehat{h}}(x)=\mbox{atan2}\big{(}\widehat{g}_{1,\widehat{h}_{1}}(\widehat{F}_{n}(x)),\widehat{g}_{2,\widehat{h}_{2}}(\widehat{F}_{n}(x))\big{)}$.
To do this, we implement preliminary simulations to calibrate $c_{0,1}$ and
$c_{0,2}$ by only considering model M1. Figure 1 displays an illustration of
our setting.

(a)  Model M1 : $X_{i}\sim U([-5,5])$

(b)  Model M1 : $X_{i}\sim\mathcal{N}(0,1.5)$

Figure 1: Illustration of model M1 with $n=200$ for two different density
functions of the design. Simulated data $(\Theta_{i})_{i=1}^{n}$ are displayed
in green points. The red curve represents the regression function $m$, while
the blue vertical line displays the point $x=-2$ and the orange vertical line
displays the point $x=1.25$ where we aim at estimating $m(x)$.

####  4.1.1 The case $c_{0,1}=c_{0,2}$

To select $\widehat{h}_{1}$ and $\widehat{h}_{2}$, we first consider the case
$c_{0,1}=c_{0,2}=c_{0}$. For different sample sizes
$n\in\left\\{100;200;500;1000\right\\}$, we compute the risk $\mathcal{R}$
defined in (21)) as a function of $c_{0}$ on the following discretization grid

$G_{c_{0}}:=\left\\{0.001;\hskip 2.84544pt0.0025;\hskip 2.84544pt0.005;\hskip
2.84544pt0.0075;\hskip 2.84544pt0.01;\hskip 2.84544pt0.025;\hskip
2.84544pt0.05;\hskip 2.84544pt0.075;\hskip 2.84544pt0.1;\hskip
2.84544pt0.2;\hskip 2.84544pt0.3;\hskip 2.84544pt0.4\right\\}.$

We denote $\mathcal{R}\equiv\mathcal{R}(c_{0})$. The numerical illustrations
are displayed in Figure 2 for $x=-2$ and in Figure 3 for $x=1.25$,
respectively.

(a) $X_{i}\sim\mathcal{N}(0,1.5)$

(b) $X_{i}\sim U([-5,5])$

Figure 2: Plot of the Monte Carlo estimation of the function $c_{0}\in
G_{c_{0}}\longmapsto\mathcal{R}(c_{0})$, based on 50 runs, for $x=-2$, the
Gaussian and the uniform designs and by using the Epanechnikov kernel for
$n\in\left\\{100;200;500;1000\right\\}$. The red vertical line displays the
point $c_{0}=0.04$.

(a) $X_{i}\sim\mathcal{N}(0,1.5)$

(b) $X_{i}\sim U([-5,5])$

Figure 3: Plot of the Monte Carlo estimation of the function $c_{0}\in
G_{c_{0}}\longmapsto\mathcal{R}(c_{0})$, based on 50 runs, for $x=1.25$, the
Gaussian and the uniform designs and by using the Epanechnikov kernel for
$n\in\left\\{100;200;500;1000\right\\}$. The red vertical line displays the
point $c_{0}=0.04$.

To further study an influence of the kernel rule, we consider the Gaussian
kernel. The associated numerical illustrations are provided in Figure 4 for
$X\sim U([-5,5])$ and for $X\sim\mathcal{N}(0,1.5)$.

(a)  Model M1 : $X_{i}\sim U([-5,5])$.

(b)  Model M1 : $X_{i}\sim\mathcal{N}(0,1.5)$.

Figure 4: Plot of the Monte Carlo estimation of the function $c_{0}\in
G_{c_{0}}\longmapsto\mathcal{R}(c_{0})$, based on 50 runs, for $x=-2$, the
Gaussian and the uniform designs and by using the Gaussian kernel for
$n\in\left\\{100;200;500;1000;2000\right\\}$.

This brief numerical study shows that the choice $c_{0,1}=c_{0,2}=0.04$ is
convenient for each numerical scheme.

####  4.1.2 The case $c_{0,1}\not=c_{0,2}$

We do no longer assume that $c_{0,1}=c_{0,2}$. For $n=200$, we compute the
risk $\mathcal{R}$ defined in (21) as a function of $(c_{0,1},c_{0,2})$ on the
following discretization grid

$G_{c_{0}}:=\left\\{0.001;\hskip 2.84544pt0.005;\hskip 2.84544pt0.01;\hskip
2.84544pt0.025;\hskip 2.84544pt0.05;\hskip 2.84544pt0.075;\hskip
2.84544pt0.1;\hskip 2.84544pt0.2;\hskip 2.84544pt0.3;\hskip
2.84544pt0.4\right\\}.$

We denote $\mathcal{R}\equiv\mathcal{R}(c_{0,1},c_{0,2})$. The associated
numerical illustrations are provided in Figure 5 and Figure 6 for the case
$X\sim\mathcal{N}(0,1.5)$ and $X\sim U([-5,5])$, respectively.

(a) estimation at $x=-2$

(b) estimation at $x=1.25$

Figure 5: 2D-representation of the Monte Carlo estimation of the function
$(c_{0,1},c_{0,1})\in G_{c_{0}}\times
G_{c_{0}}\longmapsto\mathcal{R}(c_{0,1},c_{0,2})$, based on 50 runs, for Model
M1 with $X_{i}\sim\mathcal{N}(0,1.5)$, $n=200$ at $x=-2$ and $x=1.25$ by using
the Epanechnikov kernel. We display the specific point $c_{0,1}=c_{0,2}=0.04$.

(a) estimation at $x=-2$

(b) estimation at $x=1.25$

Figure 6: 2D-representation of the Monte Carlo estimation of the function
$(c_{0,1},c_{0,1})\in G_{c_{0}}\times
G_{c_{0}}\longmapsto\mathcal{R}(c_{0,1},c_{0,2})$, based on 50 runs, for Model
M1 with $X_{i}\sim U([-5,5])$, $n=200$ at $x=-2$ and $x=1.25$ by using the
Epanechnikov kernel. We display the specific point $c_{0,1}=c_{0,2}=0.04$.

Even if it is not the best one, the choice of $c_{0,1}=c_{0,2}=0.04$ is
reasonable. For sake of simplicity, we fix $c_{0,1}=c_{0,2}=0.04$ for
subsequent numerical simulations.

###  4.2 Numerical results

We now illustrate the numerical performances obtained by our methodology for
models M1, M2 and M3 by using the Epanechnikov kernel. They are also compared
to other approaches. A similar scheme is conducted by using the Gaussian
kernel. Remember that in the following numerical experiments, our estimate is
tuned with $c_{0,1}=c_{0,2}=0.04$. We first display several graphs to
illustrate numerical performances obtained by our methodology, denoted GL, by
using the Epanechnikov kernel. More precisely, we display boxplots in Figures
7 and 8 summarizing our numerical results for Model M1 in the case $X\sim
U([-5,5])$ and in the case $X\sim\mathcal{N}(0,1.5)$, respectively. In both
cases, for model M1, we estimate $m(x)$ at $x=-2$ and at $x=1.25$. Figure 9
shows simulations for model M2 with $X\sim\mathcal{N}(0,1.5)$ and we estimate
$m(x)$ at $x=1.05$. Figure 10 shows simulations for model M3 with $X\sim
U([0,1])$ and we estimate $m(x)$ at $x=0.95$.

(a) Model M1: estimation at $x=-2$

(b) Model M1: estimation at $x=1.25$

Figure 7: Boxplots of the estimated risk with $50$ runs for the GL, NW and LL
methodologies for $n=200$ and $X\sim U([-5,5])$ by using the Epanechnikov
kernel.

(a) Model M1: estimation at $x=-2$

(b) Model M1: estimation at $x=1.25$

Figure 8: Boxplots of the estimated risk with $50$ runs for the GL, NW and LL
methodologies for $n=200$ and $X\sim\mathcal{N}(0,1.5)$ by using the
Epanechnikov kernel

(a) Model M2: $X_{i}\sim\mathcal{N}(0,1.5)$

(b) Boxplots for GL, NW and LL methodologies

Figure 9: (a): Simulated data $(\Theta_{i})_{i=1}^{n}$ of Model M2 (green
points) with $n=200$ and $X_{i}\sim\mathcal{N}(0,1.5)$. The red curve
represents the true regression function $m$, while the blue vertical line
displays the point $x=1.05$; (b): Boxplots of the estimated risk with $50$
runs for the GL, NW and LL methodologies by using the Epanechnikov kernel.

(a) Model M3: $X_{i}\sim U([0,1])$

(b) Boxplots for GL, NW and LL methodologies

Figure 10: (a): Simulated data $(\Theta_{i})_{i=1}^{n}$ of Model M3 (green
points) with $n=200$ and $X_{i}\sim U([0,1])$. The red curve represents the
true regression function $m$, while the blue vertical line displays the point
$x=0.95$; (b): Boxplots of the estimated risk with $50$ runs for the GL, NW
and LL methodologies by using the Epanechnikov kernel.

Moreover, to make a comparison with our adaptive estimator, as proposed in
[16], we also compute the Nadaraya-Watson (NW) estimator
$\widehat{m}_{h}^{NW}$ and the version of the local linear (LL) estimator
proposed by [16, Section 4.2]) denoted by $\widehat{m}_{h}^{LL}$. Cross-
Validation is used to select the bandwidth parameter for
$\widehat{m}_{h}^{NW}$ and $\widehat{m}_{h}^{LL}$. Boxplots in Figures 7, 8, 9
and 10 show that the performances of our estimator are quite satisfying.

We finally repeat the previous numerical experiments but with the use of the
Gaussian kernel: Figures 11 and 12 are the analogs of Figures 7 and 8. Figure
13 shows the numerical simulation for model M2 with $X\sim\mathcal{N}(0,1.5)$
and we estimate $m(x)$ at $x=1.05$. Figure 13 shows the numerical simulation
for model M3 with $X\sim U([0,1])$ and we estimate $m(x)$ at $x=0.95$. These
graphs show that the performances of our adaptive estimator associated with
the Gaussian kernel are quite satisfying as well.

(a) Model M1: estimation at $x=-2$

(b) Model M1: estimation at $x=1.25$

Figure 11: Boxplots of the estimated risk with $50$ runs for the GL, NW and LL
methodologies for $n=200$ and $X\sim U([-5,5])$ by using the Gaussian kernel.

(a) Model M1: estimation at $x=-2$

(b) Model M1: estimation at $x=1.25$

Figure 12: Boxplots of the estimated risk with $50$ runs for the GL, NW and LL
methodologies for $n=200$ and $X\sim\mathcal{N}(0,1.5)$ by using the Gaussian
kernel.

(a) Model M2 with $X\sim\mathcal{N}(0,1.5)$: estimation at $x=1.05$

(b) Model M3 with $X\sim U([0,1])$: estimation at $x=0.95$

Figure 13: Boxplots of the estimated risk with $50$ runs for the GL, NW and LL
methodologies for $n=200$ by using the Gaussian kernel.

##  5 Proofs

Along this section, we fix $x$ in $\mathbb{R}$ and we set $u_{x}:=F_{X}(x)$.

###  5.1 Preliminary results

In this section, we study several preliminary results for
$\widehat{g}_{1,h_{1}}$ and $\widehat{g}_{2,h_{2}}$ defined in (3). First of
all, via the warping method, we observe

$\displaystyle\mathbb{E}\big{(}\widehat{g}_{1,h_{1}}(u_{x})\big{)}=\mathbb{E}\Bigg{[}\dfrac{1}{n}\sum_{k=1}^{n}\sin(\Theta_{k}).K_{h_{1}}\big{(}u_{x}-F_{X}(X_{k})\big{)}\Bigg{]}$
$\displaystyle=\mathbb{E}\Big{[}\mathbb{E}\big{[}\sin(\Theta)|X\big{]}.K_{h_{1}}\big{(}u_{x}-F_{X}(X)\big{)}\Big{]}$
$\displaystyle=\int_{\mathbb{R}}\mathbb{E}[\sin(\Theta)|X=y].K_{h_{1}}\big{(}u_{x}-F_{X}(y)\big{)}.f_{X}(y)dy$
$\displaystyle=\int_{\mathbb{R}}m_{1}(y).K_{h_{1}}\big{(}u_{x}-F_{X}(y)\big{)}.f_{X}(y)dy$
$\displaystyle=\int_{F_{X}(\mathbb{R})}g_{1}(w).K_{h_{1}}\big{(}u_{x}-w\big{)}dw.$

Then, using the choice of $h_{\max}$, since
$u_{x}=F_{X}(x)\in(0,1)=F_{X}(\mathbb{R})$ is fixed and
$K_{h_{1}}\big{(}u_{x}-w\big{)}=0$ for $w\in(0,1)$ such that
$\big{|}u_{x}-w\big{|}>A.h_{\max}$, we have

$\displaystyle\int_{F_{X}(\mathbb{R})}g_{1}(w).K_{h_{1}}\big{(}u_{x}-w\big{)}dw=\int_{u_{x}-A.h_{\max}}^{u_{x}+A.h_{\max}}g_{1}(w).K_{h_{1}}\big{(}u_{x}-w\big{)}dw$
$\displaystyle=\big{(}K_{h_{1}}\ast g_{1}\big{)}(u_{x}).$ (22)

Thus, we obtain

$\mathbb{E}\big{(}\widehat{g}_{1,h_{1}}(u_{x})\big{)}=\big{(}K_{h_{1}}\ast
g_{1}\big{)}(u_{x})$

and similarly,

$\mathbb{E}\big{(}\widehat{g}_{2,h_{2}}(u_{x})\big{)}=\big{(}K_{h_{2}}\ast
g_{2}\big{)}(u_{x}).$ (23)

We obtain upper bounds for the bias and variance terms.

######  Lemma 5.1.

Suppose that $g_{1}$ belongs to $\mathcal{H}(\beta_{1},L_{1})$ and $g_{2}$
belongs to $\mathcal{H}(\beta_{2},L_{2})$, with
$L_{1},L_{2},\beta_{1},\beta_{2}\in\mathbb{R}^{*}_{+}$. Assume that the kernel
$K$ satisfies Assumption 3.4 with an index $\mathcal{L}\in\mathbb{R}_{+}$ such
that $\mathcal{L}\geq\max(\beta_{1},\beta_{2})$. Then, for any
$(h_{1},h_{2})\in{\mathcal{H}}_{n}^{2}$,

$\displaystyle\Big{|}\mathbb{E}\big{(}\widehat{g}_{1,h_{1}}(u_{x})\big{)}-g_{1}(u_{x})\Big{|}\leq
C_{K,\mathcal{L}}.L_{1}.h_{1}^{\beta_{1}},\quad$ $\displaystyle\textrm{ and
}\quad\mbox{Var}\big{(}\widehat{g}_{1,h_{1}}(u_{x})\big{)}\leq\dfrac{\left\|K\right\|_{\mathbb{L}^{2}(\mathbb{R})}^{2}}{n.h_{1}},$
$\displaystyle\Big{|}\mathbb{E}\big{(}\widehat{g}_{2,h_{2}}(u_{x})\big{)}-g_{2}(u_{x})\Big{|}\leq
C_{K,\mathcal{L}}.L_{2}.h_{2}^{\beta_{2}},\quad$ $\displaystyle\textrm{ and
}\quad\mbox{Var}\big{(}\widehat{g}_{2,h_{2}}(u_{x})\big{)}\leq\dfrac{\left\|K\right\|_{\mathbb{L}^{2}(\mathbb{R})}^{2}}{n.h_{2}},$

with $C_{K,\mathcal{L}}$ the constant defined in Assumption 3.4.

The proof of Lemma 5.1 is given in Section 5.2.
We introduce in the sequel several events on which we will establish some
concentration results for $\widehat{g}_{1,h_{1}}$ and $\widehat{g}_{2,h_{2}}$.

######  Definition 5.2.

_For $n\in\mathbb{N}^{*}$, $p\geq 1$, and $h_{1},h_{2}>0$, we define for an
arbitrary $v\in F_{X}(\mathbb{R})$ the following two events_

$\Omega_{1,n}(v,h_{1}):=\left\\{\big{|}\widehat{g}_{1,h_{1}}(v)-\mathbb{E}[\widehat{g}_{1,h_{1}}(v)]\big{|}\leq
c_{1}(p).\sqrt{\widetilde{V}_{1}(n,h_{1})}\right\\}$

_and_

$\Omega_{2,n}(v,h_{2}):=\left\\{\big{|}\widehat{g}_{2,h_{2}}(v)-\mathbb{E}[\widehat{g}_{2,h_{2}}(v)]\big{|}\leq
c_{1}(p).\sqrt{\widetilde{V}_{2}(n,h_{2})}\right\\},$

_with $c_{1}(p)$ satisfying
$c_{1}(p).\sqrt{\min\left\\{c_{0,1};c_{0,2}\right\\}}\geq 4p$.
Furthermore, for $L_{1},L_{2},\beta_{1},\beta_{2}>0$, we also introduce_

$E_{1,n}(v,h_{1}):=\left\\{\big{|}\widehat{g}_{1,h_{1}}(v)-g_{1}(v)\big{|}\leq\Phi_{1}(n,h_{1})\right\\}\textrm{
and
}E_{2,n}(v,h_{2}):=\left\\{\big{|}\widehat{g}_{2,h_{2}}(v)-g_{2}(v)\big{|}\leq\Phi_{2}(n,h_{2})\right\\}$

_where_

$\Phi_{1}(n,h_{1}):=c_{1}(p).\sqrt{\widetilde{V}_{1}(n,h_{1})}+C_{K,\mathcal{L}}.L_{1}.h_{1}^{\beta_{1}}$

_and_

$\Phi_{2}(n,h_{2}):=c_{1}(p).\sqrt{\widetilde{V}_{2}(n,h_{2})}+C_{K,\mathcal{L}}.L_{2}.h_{2}^{\beta_{2}}.$

Then, the following proposition gives a concentration inequality for
$\widehat{g}_{1,h_{1}}(u_{x})$ and $\widehat{g}_{2,h_{2}}(u_{x})$.

######  Proposition 5.3.

For $p\geq 1$ and bandwidths $h_{1},h_{2}\in\mathcal{H}_{n}$ (defined in (7)),
we have:

$\mathbb{P}\left(\Big{(}\Omega_{1,n}(u_{x},h_{1})\Big{)}^{c}\right)\leq
2.n^{-p},\quad\textrm{ and
}\quad\mathbb{P}\left(\Big{(}\Omega_{2,n}(u_{x},h_{2})\Big{)}^{c}\right)\leq
2.n^{-p}.$

Consequently, suppose further that $g_{1}$ belongs to
$\mathcal{H}(\beta_{1},L_{1})$, and $g_{2}$ belongs to
$\mathcal{H}(\beta_{2},L_{2})$, with
$L_{1},L_{2},\beta_{1},\beta_{2}\in\mathbb{R}^{*}_{+}$ and the kernel K
satisfies Assumption 3.4 with an index $\mathcal{L}\in\mathbb{R}_{+}$ such
that $\mathcal{L}\geq\max(\beta_{1},\beta_{2})$, then we get:

$\mathbb{P}\left(\Big{(}E_{1,n}(u_{x},h_{1})\Big{)}^{c}\right)\leq
2.n^{-p}\quad\textrm{ and
}\quad\mathbb{P}\left(\Big{(}E_{2,n}(u_{x},h_{2})\Big{)}^{c}\right)\leq
2.n^{-p}.$

The proof of Proposition 5.3 is given in Section 5.3.

###  5.2 Proof of Lemma 5.1

###### Proof.

First, for the bias of $\widehat{g}_{1,h}(u_{x})$ at $u_{x}=F_{X}(x)$, using
(22), we can write

$\displaystyle\mathbb{E}\big{(}\widehat{g}_{1,h_{1}}(u_{x})\big{)}-g_{1}(u_{x})$
$\displaystyle=\dfrac{1}{h_{1}}.\int_{u_{x}-A.h_{\max}}^{u_{x}+A.h_{\max}}K\Big{(}\dfrac{u_{x}-z}{h_{1}}\Big{)}.g_{1}(z)dz-
g_{1}(u_{x})$
$\displaystyle=\int_{-A}^{A}K(w).\big{(}g_{1}(u_{x}-h_{1}.w)-g_{1}(u_{x})\big{)}dw.$
(24)

Since $g_{1}$ belongs to $\mathcal{H}(\beta_{1},L_{1})$, using a Taylor
expansion for $g_{1}$, we get for $w\in[-A,A]$,

$\displaystyle
g_{1}(u_{x}-h_{1}.w)=g_{1}(u_{x})+g_{1}^{\prime}(u_{x}).(-h_{1}).w+...+\dfrac{(-h_{1}.w)^{\lfloor\beta_{1}\rfloor}}{(\lfloor\beta_{1}\rfloor)!}.g_{1}^{(\lfloor\beta_{1}\rfloor)}(u_{x}-\tau.h_{1}.w),\quad
0\leq\tau\leq 1.$

Then, under Assumption 3.4 with an index $\mathcal{L}\in\mathbb{R}_{+}$
satisfying $\mathcal{L}\geq\max(\beta_{1},\beta_{2})$, from (5.2) one gets

$\displaystyle\int_{-A}^{A}K(w).\big{(}g_{1}(u_{x}-h_{1}.w)-g_{1}(u_{x})\big{)}dw$
$\displaystyle=\int_{-A}^{A}K(w).\dfrac{(-h_{1}.w)^{\lfloor\beta_{1}\rfloor}}{(\lfloor\beta_{1}\rfloor)!}.g_{1}^{(\lfloor\beta_{1}\rfloor)}(u_{x}-\tau.h_{1}.w)dw$
$\displaystyle=\int_{-A}^{A}K(w).\dfrac{(-h_{1}.w)^{\lfloor\beta_{1}\rfloor}}{(\lfloor\beta_{1}\rfloor)!}.\Big{(}g_{1}^{(\lfloor\beta_{1}\rfloor)}(u_{x}-\tau.h_{1}.w)-g_{1}^{(\lfloor\beta_{1}\rfloor)}(u_{x})\Big{)}dw.$

This implies that with $0\leq\tau\leq 1$, with $C_{K,\mathcal{L}}$ the
constant defined in Assumption 3.4,

$\displaystyle\Big{|}\mathbb{E}\big{(}\widehat{g}_{1,h_{1}}(u_{x})\big{)}-g_{1}(u_{x})\Big{|}$
$\displaystyle\leq\int_{-A}^{A}|K(w)|.\dfrac{|h_{1}.w|^{\lfloor\beta_{1}\rfloor}}{(\lfloor\beta_{1}\rfloor)!}.\Big{|}g_{1}^{(\lfloor\beta_{1}\rfloor)}(u_{x}-\tau.h_{1}.w)-g_{1}^{(\lfloor\beta_{1}\rfloor)}(u_{x})\Big{|}dw$
$\displaystyle\leq\int_{-A}^{A}|K(w)|.\dfrac{|h_{1}.w|^{\lfloor\beta_{1}\rfloor}}{(\lfloor\beta_{1}\rfloor)!}.L_{1}.\big{|}\tau.h_{1}.w\big{|}^{\beta_{1}-\lfloor\beta_{1}\rfloor}dw\quad(\textrm{since
}g_{1}\in\mathcal{H}(\beta_{1},L_{1}))$ $\displaystyle\leq
L_{1}.h_{1}^{\beta_{1}}.\int_{-A}^{A}|K(w)|.|w|^{\beta_{1}}dw$
$\displaystyle\leq
L_{1}.h_{1}^{\beta_{1}}.\int_{-A}^{A}|K(w)|.(1+|w|)^{\beta_{1}}dw\leq
L_{1}.h_{1}^{\beta_{1}}.C_{K,\mathcal{L}}.$

Similarly, we obtain for $\widehat{g}_{2,h_{2}}$:

$\Big{|}\mathbb{E}\big{(}\widehat{g}_{2,h_{2}}(u_{x})\big{)}-g_{2}(u_{x})\Big{|}\leq
L_{2}.h_{2}^{\beta_{2}}.C_{K,\mathcal{L}}.$

For the variance of $\widehat{g}_{1,h_{1}}(u_{x})$, one gets:

$\displaystyle\mbox{Var}\big{(}\widehat{g}_{1,h_{1}}(u_{x})\big{)}=\mathbb{E}\Big{[}\Big{(}\widehat{g}_{1,h_{1}}(u_{x})-\mathbb{E}\big{(}\widehat{g}_{1,h_{1}}(u_{x})\big{)}\Big{)}^{2}\Big{]}$
$\displaystyle=\mbox{Var}\left(\dfrac{1}{n}.\sum_{k=1}^{n}\sin(\Theta_{k}).K_{h_{1}}\big{(}u_{x}-F_{X}(X_{k})\big{)}\right)$
$\displaystyle\leq\dfrac{1}{n}.\mathbb{E}\left[\big{(}\sin(\Theta)\big{)}^{2}.\big{[}K_{h_{1}}\big{(}u_{x}-F_{X}(X)\big{)}\big{]}^{2}\right]$
$\displaystyle\leq\dfrac{1}{n}.\mathbb{E}\left(\big{[}K_{h_{1}}\big{(}u_{x}-F_{X}(X)\big{)}\big{]}^{2}\right)\leq\dfrac{\left\|K\right\|_{\mathbb{L}^{2}(\mathbb{R})}^{2}}{n.h_{1}},$

and similarly, for $\widehat{g}_{2,h_{2}}(u_{x})$. This concludes the proof of
Lemma 5.1.

∎

###  5.3 Proof of Proposition 5.3

We shall use the following version of Bernstein inequality (see [6, Lemma 2]).

######  Lemma 5.4 (Bernstein inequality).

Let $T_{1},\ldots,T_{n}$ be i.i.d. random variables and
$S_{n}=\sum_{j=1}^{n}\big{[}T_{j}-\mathbb{E}(T_{j})\big{]}$. Then, for any
$\eta>0$,

$\displaystyle\mathbb{P}\Big{(}\big{|}S_{n}\big{|}\geq n.\eta\Big{)}\leq
2.\max\left(\exp\Big{(}-\dfrac{n.\eta^{2}}{4.V}\Big{)},\exp\Big{(}-\dfrac{n.\eta}{4.b}\Big{)}\right),$

with $\mbox{Var}(T_{1})\leq V$ and $|T_{1}|\leq b$, where $V$ and $b$ are two
positive deterministic constants.

Now, we can start to prove Proposition 5.3.

###### Proof of Proposition 5.3.

We follow the procedure proposed in [6, Proposition $6$]. First of all, we
define random variables
$Z_{k}(u_{x}):=\cos(\Theta_{k}).K_{h_{2}}\big{(}u_{x}-F_{X}(X_{k})\big{)}$,
for $1\leq k\leq n$, and hence,
$\widehat{g}_{2,h_{2}}(u_{x})=\dfrac{1}{n}\sum_{k=1}^{n}Z_{k}(u_{x})$. Notice
that $\mathbb{E}(Z_{k}(u_{x}))=\big{(}K_{h_{2}}*g_{2}\big{)}(u_{x})$ (see
(23)). Since $\left\|K\right\|_{\infty}<+\infty$, we then have for any $v\in
F_{X}(\mathbb{R})$:

$\left|Z_{k}(v)\right|=\left|\cos(\Theta_{k}).K_{h_{2}}\big{(}v-F_{X}(X_{k})\big{)}\right|\leq\dfrac{\left\|K\right\|_{\infty}}{h_{2}}=:b(h_{2}),$
(25)

and
$\mbox{Var}\big{(}Z_{k}(v)\big{)}=n.\mbox{Var}\big{(}\widehat{g}_{2,h_{2}}(v)\big{)}\leq
n.\dfrac{\left\|K\right\|_{\mathbb{L}^{2}(\mathbb{R})}^{2}}{n.h_{2}}=:n.V_{0}(n,h_{2})$.
Now, applying Lemma 5.4 to the $Z_{k}(v)$’s, we obtain for $\eta(h_{2})>0$,

$\displaystyle\mathbb{P}\left(\Big{|}\widehat{g}_{2,h_{2}}(v)-\mathbb{E}\big{(}\widehat{g}_{2,h_{2}}(v)\big{)}\Big{|}\geq\eta(h_{2})\right)$
$\displaystyle=\mathbb{P}\left(\Big{|}\sum_{k=1}^{n}Z_{k}(v)-\mathbb{E}\Big{(}Z_{k}(v)\Big{)}\Big{|}\geq
n.\eta(h_{2})\right)$ $\displaystyle\leq
2.\max\left\\{\exp\left(-\dfrac{n.\big{(}\eta(h_{2})\big{)}^{2}}{4n.V_{0}(n,h_{2})}\right);\exp\left(-\dfrac{n.\eta(h_{2})}{4b(h_{2})}\right)\right\\}.$

For $p\geq 1$, choose
$\eta(h_{2})=c_{1}(p).\sqrt{\widetilde{V}_{2}(n,h_{2})}$, with

$\widetilde{V}_{2}(n,h_{2})=c_{0,2}.\log(n).V_{0}(n,h_{2}).$ (26)

Then,

$\displaystyle\mathbb{P}\left(\Big{|}\widehat{g}_{2,h_{2}}(v)-\mathbb{E}\big{(}\widehat{g}_{2,h_{2}}(v)\big{)}\Big{|}\geq
c_{1}(p).\sqrt{\widetilde{V}_{2}(n,h_{2})}\right)$ $\displaystyle\leq
2.\max\left\\{\exp\left(-\dfrac{n.c_{1}(p)^{2}.\widetilde{V}_{2}(n,h_{2})}{4n.V_{0}(n,h_{2})}\right);\exp\left(-\dfrac{n.c_{1}(p).\sqrt{\widetilde{V}_{2}(n,h_{2})}}{4b(h_{2})}\right)\right\\}.$
(27)

We then choose $c_{1}(p)$.
$+$ First, $c_{1}(p)$ is chosen such that

$\dfrac{n.c_{1}(p)^{2}.\widetilde{V}_{2}(n,h_{2})}{4n.V_{0}(n,h_{2})}=\dfrac{n.c_{1}(p)^{2}.c_{0,2}.\log(n).V_{0}(n,h_{2})}{4n.V_{0}(n,h_{2})}\geq
p.\log(n),$ (28)

that is $c_{1}(p)$ satisfies $c_{1}(p)^{2}.c_{0,2}\geq 4p$.
$+$ Secondly, we can write

$\dfrac{n.c_{1}(p).\sqrt{\widetilde{V}_{2}(n,h_{2})}}{4b(h_{2})}=\dfrac{c_{1}(p).\sqrt{c_{0,2}}}{4}.\sqrt{\log(n)}.\dfrac{n.\sqrt{V_{0}(n,h_{2})}}{b(h_{2})}$

and for $h_{2}\in\mathcal{H}_{n}$,

$n.\dfrac{\sqrt{V_{0}(n,h_{2})}}{b(h_{2})}=n.\dfrac{\left\|K\right\|_{\mathbb{L}^{2}(\mathbb{R})}}{\sqrt{n.h_{2}}}.\dfrac{h_{2}}{\left\|K\right\|_{\infty}}=\sqrt{n.h_{2}}.\dfrac{\left\|K\right\|_{\mathbb{L}^{2}(\mathbb{R})}}{\left\|K\right\|_{\infty}}>\sqrt{\log(n)},$

then, we have

$\dfrac{n.c_{1}(p).\sqrt{\widetilde{V}_{2}(n,h_{2})}}{4b(h_{2})}=\dfrac{c_{1}(p).\sqrt{c_{0,2}}}{4}.\sqrt{\log(n)}.\dfrac{n.\sqrt{V_{0}(n,h_{2})}}{b(h_{2})}\geq\dfrac{c_{1}(p).\sqrt{c_{0,2}}}{4}.\log(n)\geq
p.\log(n),$ (29)

provided that $c_{1}(p).\sqrt{c_{0,2}}\geq 4p$. Note that this condition also
ensures the constraint $c_{1}(p)^{2}.c_{0,2}\geq 4p$.

Now, combining (28) and (29), we get from (5.3) for any $p\geq 1$:

$\displaystyle\mathbb{P}\left(\Big{(}\Omega_{2,n}(u_{x},h_{2})\Big{)}^{c}\right):=\mathbb{P}\left(\Big{|}\widehat{g}_{2,h_{2}}(u_{x})-\mathbb{E}\big{(}\widehat{g}_{2,h_{2}}(u_{x})\big{)}\Big{|}>c_{1}(p).\sqrt{\widetilde{V}_{2}(n,h_{2})}\right)\leq
2.n^{-p}.$

This implies that with probability larger than $1-2.n^{-p}$, we have:

$\displaystyle\big{|}\widehat{g}_{2,h_{2}}(u_{x})-\mathbb{E}\big{(}\widehat{g}_{2,h_{2}}(u_{x})\big{)}\big{|}\leq
c_{1}(p).\sqrt{\widetilde{V}_{2}(n,h_{2})}.$

Then, with probability larger than $1-2.n^{-p}$, we obtain

$\displaystyle\big{|}\widehat{g}_{2,h_{2}}(u_{x})-g_{2}(u_{x})\big{|}$
$\displaystyle\leq\big{|}\widehat{g}_{2,h_{2}}(u_{x})-\mathbb{E}\big{(}\widehat{g}_{2,h_{2}}(u_{x})\big{)}\big{|}+\big{|}\mathbb{E}\big{(}\widehat{g}_{2,h_{2}}(u_{x})\big{)}-g_{2}(u_{x})\big{|}$
$\displaystyle\leq c_{1}(p).\sqrt{\widetilde{V}_{2}(n,h_{2})}\quad+\quad
L_{2}.h_{2}^{\beta_{2}}.C_{K,\mathcal{L}}.$

Recall that
$\Phi_{2}(n,h_{2})=c_{1}(p).\sqrt{\widetilde{V}_{2}(n,h_{2})}+L_{2}.h_{2}^{\beta_{2}}.C_{K,\mathcal{L}}$,
therefore, we finally obtain for $p\geq 1$ that

$\mathbb{P}\left(\Big{(}E_{2,n}(u_{x},h_{2})\Big{)}^{c}\right)=\mathbb{P}\left(\big{|}\widehat{g}_{2,h_{2}}(u_{x})-g_{2}(u_{x})\big{|}>\Phi_{2}(n,h_{2})\right)\leq
2.n^{-p}.$ (30)

By similar arguments we obtain the desired result for $\widehat{g}_{1,h_{1}}$.
This concludes the proof of Proposition 5.3.

∎

###  5.4 Proofs of main results

####  5.4.1 Proof of Proposition 3.1

We first have the following concentration result.

######  Corollary 5.5.

Under the Assumptions of Proposition 3.1, for all
$h_{1},h_{1}^{\prime},h_{2},h_{2}^{\prime}\in\mathcal{H}_{n}$, for all $p\geq
1$, for $u_{x}=F_{X}(x)$,

$\mathbb{P}\left(\big{|}\widehat{g}_{1,h_{1},h_{1}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{1,h_{1},h_{1}^{\prime}}(u_{x})\big{]}\big{|}>c_{1}(p).\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}.\sqrt{\widetilde{V}_{1}(n,h_{1}^{\prime})}\right)\leq
2.n^{-p},$

and

$\mathbb{P}\left(\big{|}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{]}\big{|}>c_{1}(p).\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}.\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}\right)\leq
2.n^{-p},$

with $c_{1}(p)$ satisfying
$c_{1}(p).\sqrt{\min\left\\{c_{0,1};c_{0,2}\right\\}}\geq 4p$.

###### Proof of Corollary 5.5.

We define random variables
$\widetilde{Z}_{k}(u_{x}):=\cos(\Theta_{k}).\big{(}K_{h_{2}^{\prime}}\ast
K_{h_{2}}\big{)}\big{(}u_{x}-F_{X}(X_{k})\big{)}$, for $1\leq k\leq n$, and
hence,
$\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})=\dfrac{1}{n}\sum_{k=1}^{n}\widetilde{Z}_{k}(u_{x})$.
Since $\left\|K\right\|_{\infty}<+\infty$, we then have:

$\left|\widetilde{Z}_{k}(u_{x})\right|=\left|\cos(\Theta_{k}).\big{(}K_{h_{2}^{\prime}}\ast
K_{h_{2}}\big{)}\big{(}u_{x}-F_{X}(X_{k})\big{)}\right|\leq\dfrac{\left\|K\right\|_{\infty}.\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}}{h_{2}}=:\widetilde{b}(h_{2}),$

and
$\mbox{Var}\big{(}\widetilde{Z}_{k}(u_{x})\big{)}=n.\mbox{Var}\big{(}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{)}\leq
n.\dfrac{\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}^{2}.\left\|K\right\|_{\mathbb{L}^{2}(\mathbb{R})}^{2}}{n.h_{2}}=:n.\widetilde{V}_{0}(n,h_{2})$.
Using similar arguments of the proof of Proposition 5.3, we obtain with a
probability greater than $1-2.n^{-p}$ that

$\displaystyle\big{|}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{]}\big{|}\leq\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}.c_{1}(p).\sqrt{\widetilde{V}_{2}(n,h_{2})}.$

Similarly, we get the desired concentration for
$\big{|}\widehat{g}_{1,h_{1},h_{1}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{1,h_{1},h_{1}^{\prime}}(u_{x})\big{]}\big{|}$.
This concludes the proof of Corollary 5.5.

∎

Now, we can start to prove Proposition 3.1.

###### Proof of Proposition 3.1.

We follow the strategy proposed in [6, Theorem 2]. The target is to find an
upper bound for
$\big{|}\widehat{g}_{2,\widehat{h}_{2}}(u_{x})-g_{2}(u_{x})\big{|}$. Let
$h_{2}\in\mathcal{H}_{n}$ be fixed. We consider the following decomposition:

$\displaystyle\big{|}\widehat{g}_{2,\widehat{h}_{2}}(u_{x})-g_{2}(u_{x})\big{|}\leq\underbrace{\big{|}\widehat{g}_{2,\widehat{h}_{2}}(u_{x})-\widehat{g}_{2,h_{2},\widehat{h}_{2}}(u_{x})\big{|}}_{=:I_{g_{2},1}}+\underbrace{\big{|}\widehat{g}_{2,h_{2},\widehat{h}_{2}}(u_{x})-\widehat{g}_{2,h_{2}}(u_{x})\big{|}}_{=:I_{g_{2},2}}+\big{|}\widehat{g}_{2,h_{2}}(u_{x})-g_{2}(u_{x})\big{|}.$

By the definition of $A_{2}(h_{2},u_{x})$, we have

$\displaystyle
I_{g_{2},1}=\big{|}\widehat{g}_{2,\widehat{h}_{2}}(u_{x})-\widehat{g}_{2,h_{2},\widehat{h}_{2}}(u_{x})\big{|}$
$\displaystyle=\big{|}\widehat{g}_{2,\widehat{h}_{2}}(u_{x})-\widehat{g}_{2,h_{2},\widehat{h}_{2}}(u_{x})\big{|}-\sqrt{\widetilde{V}_{2}(n,\widehat{h}_{2})}+\sqrt{\widetilde{V}_{2}(n,\widehat{h}_{2})}$
$\displaystyle\leq\sup_{h_{2}^{\prime}\in\mathcal{H}_{n}}\Big{\\{}\big{|}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})-\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{|}-\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}\Big{\\}}_{+}+\sqrt{\widetilde{V}_{2}(n,\widehat{h}_{2})}$
$\displaystyle=A_{2}(h_{2},u_{x})+\sqrt{\widetilde{V}_{2}(n,\widehat{h}_{2})}.$

And similarly, by the definition of $A_{2}(\widehat{h}_{2},u_{x})$,

$\displaystyle
I_{g_{2},2}=\big{|}\widehat{g}_{2,h_{2},\widehat{h}_{2}}(u_{x})-\widehat{g}_{2,h_{2}}(u_{x})\big{|}$
$\displaystyle\leq\sup_{h_{2}^{\prime}\in\mathcal{H}_{n}}\Big{\\{}\big{|}\widehat{g}_{2,h_{2}^{\prime},\widehat{h}_{2}}(u_{x})-\widehat{g}_{2,h_{2}^{\prime}}(u_{x})\big{|}-\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}\Big{\\}}_{+}+\sqrt{\widetilde{V}_{2}(n,h_{2})}$
$\displaystyle=A_{2}(\widehat{h}_{2},u_{x})+\sqrt{\widetilde{V}_{2}(n,h_{2})}.$

Therefore, by using the definition of $\widehat{h}_{2}$, we get

$\displaystyle I_{g_{2},1}+I_{g_{2},2}\leq
A_{2}(h_{2},u_{x})+\sqrt{\widetilde{V}_{2}(n,\widehat{h}_{2})}+A_{2}(\widehat{h}_{2},u_{x})+\sqrt{\widetilde{V}_{2}(n,h_{2})}\leq
2.\Big{[}A_{2}(h_{2},u_{x})+\sqrt{\widetilde{V}_{2}(n,h_{2})}\Big{]}.$

Hence, we obtain

$\big{|}\widehat{g}_{2,\widehat{h}_{2}}(u_{x})-g_{2}(u_{x})\big{|}\leq
2.A_{2}(h_{2},u_{x})+2.\sqrt{\widetilde{V}_{2}(n,h_{2})}+\big{|}\widehat{g}_{2,h_{2}}(u_{x})-g_{2}(u_{x})\big{|}.$
(31)

Now, to study $A_{2}(h_{2},u_{x})$, we can write:

$\displaystyle\widehat{g}_{2,h_{2}^{\prime}}(u_{x})-\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})=\widehat{g}_{2,h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})\big{]}$
$\displaystyle-\big{(}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{]}\big{)}$
$\displaystyle\hskip
5.69046pt+\mathbb{E}\big{[}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})\big{]}-\mathbb{E}\big{[}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{]},$

and, we have
$\mathbb{E}\big{[}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})\big{]}=\big{(}K_{h_{2}^{\prime}}*g_{2}\big{)}(u_{x})$
as well as
$\mathbb{E}\big{[}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{]}=\mathbb{E}\big{[}K_{h_{2}^{\prime}}*\widehat{g}_{2,h_{2}}(u_{x})\big{]}=\big{(}K_{h_{2}^{\prime}}*K_{h_{2}}*g_{2}\big{)}(u_{x})$.
Thus,

$\displaystyle\big{|}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})-\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{|}-\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}$
$\displaystyle\leq\big{|}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})\big{]}\big{|}-\dfrac{\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}}{(1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})})}$
$\displaystyle\hskip
2.84544pt+\big{|}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{]}\big{|}-\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}.\dfrac{\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}}{(1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})})}$
$\displaystyle\hskip
11.38092pt+\big{|}\mathbb{E}\big{[}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})\big{]}-\mathbb{E}\big{[}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{]}\big{|}.$

However, for any $h_{2}^{\prime}\in\mathcal{H}_{n}$,

$\big{|}\mathbb{E}\big{[}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})\big{]}-\mathbb{E}\big{[}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{]}\big{|}=\big{|}K_{h_{2}^{\prime}}*\big{(}g_{2}-K_{h_{2}}*g_{2}\big{)}(u_{x})\big{|}\leq\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}.\left\|g_{2}-K_{h_{2}}*g_{2}\right\|_{\infty}.$

Hence, incorporating this bound in the definition of $A_{2}(h_{2},u_{x})$, we
obtain

$\displaystyle A_{2}(h_{2},u_{x})$
$\displaystyle=\sup_{h_{2}^{\prime}\in\mathcal{H}_{n}}\Big{\\{}\big{|}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})-\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{|}-\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}\Big{\\}}_{+}$
$\displaystyle\leq\sup_{h_{2}^{\prime}\in\mathcal{H}_{n}}\left\\{\big{|}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})\big{]}\big{|}-\dfrac{\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}}{(1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})})}\right\\}_{+}$
(32)
$\displaystyle+\sup_{h_{2}^{\prime}\in\mathcal{H}_{n}}\left\\{\big{|}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{]}\big{|}-\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}.\dfrac{\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}}{(1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})})}\right\\}_{+}$
(33) $\displaystyle\hskip
199.16928pt+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}.\left\|g_{2}-K_{h_{2}}*g_{2}\right\|_{\infty}.$

From Corollary 5.5, for $h_{2},h_{2}^{\prime}\in\mathcal{H}_{n}$,

$\displaystyle\mathbb{P}\left(\big{|}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{]}\big{|}>c_{1}(p).\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}.\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}\right)\leq
2.n^{-p}.$

It implies that if we take
$c_{1}(p)=\dfrac{1}{1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}}$ and if
$c_{0,2}\geq
16p^{2}.\big{(}1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}\big{)}^{2}$,
then

$\displaystyle\mathbb{P}\left(\sup_{h_{2}^{\prime}\in\mathcal{H}_{n}}\left\\{\big{|}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})\big{]}\big{|}-\dfrac{\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}}{(1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})})}\right\\}_{+}>0\right)\leq
2.\sum_{h_{2}\in\mathcal{H}_{n}}n^{-p}\leq 2.n^{1-p},$

as $\mbox{Card}(\mathcal{H}_{n})\leq n$. In the same way for all
$h_{2}\in\mathcal{H}_{n}$,

$\displaystyle\mathbb{P}\left(\sup_{h_{2}^{\prime}\in\mathcal{H}_{n}}\left\\{\big{|}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{]}\big{|}-\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}.\dfrac{\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}}{(1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})})}\right\\}_{+}>0\right)\leq
2.n^{1-p}.$

Consequently, the following set

$\displaystyle\widetilde{A}_{2}:=$
$\displaystyle\left\\{\sup_{h_{2}^{\prime}\in\mathcal{H}_{n}}\left\\{\big{|}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2}^{\prime}}(u_{x})\big{]}\big{|}-\dfrac{\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}}{(1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})})}\right\\}_{+}=0\right\\}$
$\displaystyle\cap\left\\{\forall\,h_{2}\in\mathcal{H}_{n},\sup_{h_{2}^{\prime}\in\mathcal{H}_{n}}\left\\{\big{|}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2},h_{2}^{\prime}}(u_{x})\big{]}\big{|}-\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}.\dfrac{\sqrt{\widetilde{V}_{2}(n,h_{2}^{\prime})}}{(1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})})}\right\\}_{+}=0\right\\}$

has probability greater than $(1-4.n^{2-p})$. Now, choose $p=2+q$ and then
$c_{0,2}\geq
16\big{(}2+q\big{)}^{2}.\big{(}1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}\big{)}^{2}$.
Thus, we obtain that $\mathbb{P}\big{(}\widetilde{A}_{2}\big{)}>1-4.n^{-q}$.
Combining inequalities (31) and (5.4.1), we have on $\widetilde{A}_{2}$:

$\displaystyle\big{|}\widehat{g}_{2,\widehat{h}_{2}}(u_{x})-g_{2}(u_{x})\big{|}$
$\displaystyle\leq
2.A_{2}(h_{2},u_{x})+2.\sqrt{\widetilde{V}_{2}(n,h_{2})}+\big{|}\widehat{g}_{2,h_{2}}(u_{x})-g_{2}(u_{x})\big{|}$
$\displaystyle\leq
2.\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}.\left\|g_{2}-K_{h_{2}}*g_{2}\right\|_{\infty}+2.\sqrt{\widetilde{V}_{2}(n,h_{2})}+\big{|}\widehat{g}_{2,h_{2}}(u_{x})-g_{2}(u_{x})\big{|},$

but still on $\widetilde{A}_{2}$, one gets
$\big{|}\widehat{g}_{2,h_{2}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2}}(u_{x})\big{]}\big{|}-\dfrac{\sqrt{\widetilde{V}_{2}(n,h_{2})}}{(1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})})}\leq
0$, so

$\displaystyle\big{|}\widehat{g}_{2,h_{2}}(u_{x})-g_{2}(u_{x})\big{|}$
$\displaystyle\leq\big{|}\mathbb{E}\big{[}\widehat{g}_{2,h_{2}}(u_{x})\big{]}-g_{2}(u_{x})\big{|}+\big{|}\widehat{g}_{2,h_{2}}(u_{x})-\mathbb{E}\big{[}\widehat{g}_{2,h_{2}}(u_{x})\big{]}\big{|}-\dfrac{\sqrt{\widetilde{V}_{2}(n,h_{2})}}{(1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})})}$
$\displaystyle\hskip
199.16928pt+\dfrac{\sqrt{\widetilde{V}_{2}(n,h_{2})}}{(1+\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})})}$
$\displaystyle\leq\left\|g_{2}-K_{h_{2}}*g_{2}\right\|_{\infty}+\sqrt{\widetilde{V}_{2}(n,h_{2})}.$

Therefore, on $\widetilde{A}_{2}$, we finally obtain

$\displaystyle\big{|}\widehat{g}_{2,\widehat{h}_{2}}(u_{x})-g_{2}(u_{x})\big{|}$
$\displaystyle\leq(1+2\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}).\left\|g_{2}-K_{h_{2}}*g_{2}\right\|_{\infty}+3.\sqrt{\widetilde{V}_{2}(n,h_{2})}.$

By similar arguments, we obtain the desired result for
$\widehat{g}_{1,\widehat{h}_{1}}(u_{x})$. This concludes the proof of
Proposition 3.1.

∎

####  5.4.2 Proof of Theorem 3.5

First, we establish a concentration result for
$\widehat{g}_{1,\widehat{h}_{1}}(u_{x})$ and
$\widehat{g}_{2,\widehat{h}_{2}}(u_{x})$ as follows:

######  Corollary 5.6.

For $\beta_{1},\beta_{2},L_{1},L_{2}>0$, suppose that $g_{1}$ belongs to
$\mathcal{H}(\beta_{1},L_{1})$ and $g_{2}$ belongs to
$\mathcal{H}(\beta_{2},L_{2})$. Then, under the assumptions of Proposition
3.1, for $q\geq 1$,

$\displaystyle\mathbb{P}\Big{(}\big{(}\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})\big{)}^{c}\Big{)}$
$\displaystyle:=\mathbb{P}\Big{(}\big{|}\widehat{g}_{1,\widehat{h}_{1}}(u_{x})-g_{1}(u_{x})\big{|}>C_{1}.\psi_{n}(\beta_{1})\Big{)}\leq
4.n^{-q},$
$\displaystyle\mathbb{P}\Big{(}\big{(}\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\big{)}^{c}\Big{)}$
$\displaystyle:=\mathbb{P}\Big{(}\big{|}\widehat{g}_{2,\widehat{h}_{2}}(u_{x})-g_{2}(u_{x})\big{|}>C_{2}.\psi_{n}(\beta_{2})\Big{)}\leq
4.n^{-q},$

where $C_{1}$ is a constant (depending on $\beta_{1},L_{1},c_{0,1},K$) and
$C_{2}$ is a constant (depending on $\beta_{2},L_{2},c_{0,2},K$). In the
previous expressions,
$\psi_{n}(\beta_{1})=\big{(}\log(n)/n\big{)}^{\frac{\beta_{1}}{2\beta_{1}+1}}$
and
$\psi_{n}(\beta_{2})=\big{(}\log(n)/n\big{)}^{\frac{\beta_{2}}{2\beta_{2}+1}}$.

###### Proof of Corollary 5.6.

Since $g_{1}$ belongs to $\mathcal{H}(\beta_{1},L_{1})$, from Lemma 5.1, we
have

$\left\|g_{1}-K_{h_{1}}*g_{1}\right\|_{\infty}\leq
L_{1}.h_{1}^{\beta_{1}}.C_{K,\mathcal{L}}.$

From Proposition 3.1, this implies that with a probability greater than
$1-4.n^{-q}$, one gets for any $h_{1}\in\mathcal{H}_{n}$:

$\displaystyle\big{|}\widehat{g}_{1,\widehat{h}_{1}}(u_{x})-g_{1}(u_{x})\big{|}\leq\big{(}1+2\left\|K\right\|_{\mathbb{L}^{1}(\mathbb{R})}\big{)}.L_{1}.h_{1}^{\beta_{1}}.C_{K,\mathcal{L}}+3.\sqrt{\widetilde{V}_{1}(n,h_{1})}.$
(34)

In (34), we take $h_{1}$ so that $h_{1}^{-1}$ is an integer and $h_{1}$ of
order $\Big{(}\dfrac{\log(n)}{n}\Big{)}^{\frac{1}{2\beta_{1}+1}}$. Since
$1/(2\beta_{1}+1)<1$, $h_{1}\in\mathcal{H}_{n},$ for $n$ large enough. As a
result, we obtain with probability greater than $1-4.n^{-q}$, that

$\displaystyle\big{|}\widehat{g}_{1,\widehat{h}_{1}}(u_{x})-g_{1}(u_{x})\big{|}\leq
C_{1}.\psi_{n}(\beta_{1}),$

with a constant $C_{1}$ (depending on $\beta_{1},L_{1},c_{0,1},K$) and
$\psi_{n}(\beta_{1})=\big{(}\log(n)/n\big{)}^{\frac{\beta_{1}}{2\beta_{1}+1}}$.
By similar arguments, we obtain the desired result for
$\widehat{g}_{2,\widehat{h}_{2}}(u_{x})$. This concludes the proof of
Corollary 5.6.

∎

Now, we start to prove Theorem 3.5.

###### Proof of Theorem 3.5.

We have
$\mathbb{E}\Big{[}\big{|}\widehat{m}_{\widehat{h}}(x)-m(x)\big{|}^{2}\Big{]}=\mathbb{E}\Big{[}\big{|}\widehat{g}_{\widehat{h}}(u_{x})-g(u_{x})\big{|}^{2}\Big{]}$.
First, on the event
$\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})$,
for $n$ sufficiently large satisfying
$C_{2}.\psi_{n}(\beta_{2})\leq\delta_{2}/2$ and
$C_{1}.\psi_{n}(\beta_{1})\leq\delta_{1}/2$, we have
$\big{|}\widehat{g}_{2,\widehat{h}_{2}}(u_{x})-g_{2}(u_{x})\big{|}\leq
C_{2}.\psi_{n}(\beta_{2})<\dfrac{\big{|}g_{2}(u_{x})\big{|}}{2}$ and
$\big{|}\widehat{g}_{1,\widehat{h}_{1}}(u_{x})-g_{1}(u_{x})\big{|}\leq
C_{1}.\psi_{n}(\beta_{1})<\dfrac{\big{|}g_{1}(u_{x})\big{|}}{2}$. Thus, we get
$\widehat{g}_{2,\widehat{h}_{2}}(u_{x}).g_{2}(u_{x})>0$ and
$\widehat{g}_{1,\widehat{h}_{1}}(u_{x}).g_{1}(u_{x})>0$. Then, we have

$\displaystyle\mathbb{E}\Big{[}\big{|}\widehat{g}_{\widehat{h}}(u_{x})-g(u_{x})\big{|}^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}$
$\displaystyle=$ $\displaystyle\hskip
2.84544pt\mathbb{E}\Big{[}\Big{|}\mbox{atan2}\big{(}\widehat{g}_{1,\widehat{h}_{1}}(u_{x}),\widehat{g}_{2,\widehat{h}_{2}}(u_{x})\big{)}-\mbox{atan2}\big{(}g_{1}(u_{x}),g_{2}(u_{x})\big{)}\Big{|}^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}$
$\displaystyle=$ $\displaystyle\hskip
2.84544pt\mathbb{E}\Big{[}\Big{|}\arctan\Big{(}\dfrac{\widehat{g}_{1,\widehat{h}_{1}}(u_{x})}{\widehat{g}_{2,\widehat{h}_{2}}(u_{x})}\Big{)}-\arctan\Big{(}\dfrac{g_{1}(u_{x})}{g_{2}(u_{x})}\Big{)}\Big{|}^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}$
$\displaystyle\leq$ $\displaystyle\hskip
2.84544pt2.\mathbb{E}\Big{[}\Big{|}\arctan\Big{(}\dfrac{\widehat{g}_{1,\widehat{h}_{1}}(u_{x})}{\widehat{g}_{2,\widehat{h}_{2}}(u_{x})}\Big{)}-\arctan\Big{(}\dfrac{g_{1}(u_{x})}{\widehat{g}_{2,\widehat{h}_{2}}(u_{x})}\Big{)}\Big{|}^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}$
(35)
$\displaystyle+2.\mathbb{E}\Big{[}\Big{|}\arctan\Big{(}\dfrac{g_{1}(u_{x})}{\widehat{g}_{2,\widehat{h}_{2}}(u_{x})}\Big{)}-\arctan\Big{(}\dfrac{g_{1}(u_{x})}{g_{2}(u_{x})}\Big{)}\Big{|}^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}.$

For $n$ sufficiently large,
$\big{|}\widehat{g}_{2,\widehat{h}_{2}}(u_{x})\big{|}\geq\big{|}g_{2}(u_{x})\big{|}-\big{|}g_{2}(u_{x})-\widehat{g}_{2,h}(u_{x})\big{|}>\delta_{2}-C_{2}.\psi_{n}(\beta_{2})\geq\delta_{2}/2$
on the event $\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})$, and using the
$1$-Lipschitz continuity of $\arctan$, we get for the first term in (35),
since on $\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})$ one has
$\big{|}\widehat{g}_{1,\widehat{h}_{1}}(u_{x})-g_{1}(u_{x})\big{|}\leq
C_{1}.\psi_{n}(\beta_{1})$

$\displaystyle\mathbb{E}\Big{[}\Big{|}\arctan\Big{(}\dfrac{\widehat{g}_{1,\widehat{h}_{1}}(u_{x})}{\widehat{g}_{2,\widehat{h}_{2}}(u_{x})}\Big{)}-\arctan\Big{(}\dfrac{g_{1}(u_{x})}{\widehat{g}_{2,\widehat{h}_{2}}(u_{x})}\Big{)}\Big{|}^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}$
$\displaystyle\leq\dfrac{4}{\delta_{2}^{2}}.\mathbb{E}\Big{[}\Big{|}\widehat{g}_{1,\widehat{h}_{1}}(u_{x})-g_{1}(u_{x})\Big{|}^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}$
$\displaystyle\leq\dfrac{4}{\delta_{2}^{2}}.\mathbb{E}\Big{[}C_{1}^{2}.\psi_{n}(\beta_{1})^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}$
$\displaystyle\leq\dfrac{4}{\delta_{2}^{2}}.C_{1}^{2}.\psi_{n}(\beta_{1})^{2}.\mathbb{P}\big{(}\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})\big{)}\leq\dfrac{4}{\delta_{2}^{2}}.C_{1}^{2}.\psi_{n}(\beta_{1})^{2}.$

Moreover, for the second term in (35), since
$\dfrac{g_{1}(u_{x})}{\widehat{g}_{2,\widehat{h}_{2}}(u_{x})}.\dfrac{g_{1}(u_{x})}{g_{2}(u_{x})}>0$
on $\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})$, we have

$\displaystyle\mathbb{E}\Big{[}\Big{|}\arctan\Big{(}\dfrac{g_{1}(u_{x})}{\widehat{g}_{2,\widehat{h}_{2}}(u_{x})}\Big{)}-\arctan\Big{(}\dfrac{g_{1}(u_{x})}{g_{2}(u_{x})}\Big{)}\Big{|}^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}$
$\displaystyle=\mathbb{E}\Big{[}\Big{|}\arctan\Big{(}\dfrac{\widehat{g}_{2,\widehat{h}_{2}}(u_{x})}{g_{1}(u_{x})}\Big{)}-\arctan\Big{(}\dfrac{g_{2}(u_{x})}{g_{1}(u_{x})}\Big{)}\Big{|}^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}$
$\displaystyle\leq\dfrac{1}{\big{|}g_{1}(u_{x})\big{|}^{2}}.\mathbb{E}\Big{[}\Big{|}\widehat{g}_{2,\widehat{h}_{2}}(u_{x})-g_{2}(u_{x})\Big{|}^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}$
$\displaystyle\leq\dfrac{1}{\delta_{1}^{2}}.\mathbb{E}\Big{[}C_{2}^{2}.\psi_{n}(\beta_{2})^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}\leq\dfrac{1}{\delta_{1}^{2}}.C_{2}^{2}.\psi_{n}(\beta_{2})^{2},$

by using Corollary 5.6.
Therefore, on the event
$\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})$,
for $n$ sufficiently large such that
$C_{2}.\psi_{n}(\beta_{2})\leq\delta_{2}/2$ and
$C_{1}.\psi_{n}(\beta_{1})\leq\delta_{1}/2$, we obtain

$\displaystyle\mathbb{E}\Big{[}\big{|}\widehat{g}_{\widehat{h}}(u_{x})-g(u_{x})\big{|}^{2}.1_{\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\cap\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})}\Big{]}\leq
8.\Big{(}\dfrac{1}{\delta_{2}^{2}}+\dfrac{1}{\delta_{1}^{2}}\Big{)}.(C_{1}^{2}+C_{2}^{2}).\max\Big{(}\psi_{n}(\beta_{1})^{2},\psi_{n}(\beta_{2})^{2}\Big{)}.$

On the other hand, on the complementary
$\big{(}\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\big{)}^{c}\cup\big{(}\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})\big{)}^{c}$,
using the fact that $\big{|}\mbox{atan2}(w_{1},w_{2})\big{|}\leq\pi$,
$\forall(w_{1},w_{2})$, we can simply obtain an upper-bound as follows:

$\displaystyle\mathbb{E}\Big{[}\big{|}\widehat{g}_{\widehat{h}}(u_{x})-g(u_{x})\big{|}^{2}.1_{\big{(}\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\big{)}^{c}\cup\big{(}\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})\big{)}^{c}}\Big{]}$
$\displaystyle=\mathbb{E}\Big{[}\Big{|}\mbox{atan2}\big{(}\widehat{g}_{1,\widehat{h}_{1}}(u_{x}),\widehat{g}_{2,\widehat{h}_{2}}(u_{x})\big{)}-\mbox{atan2}\big{(}g_{1}(u_{x}),g_{2}(u_{x})\big{)}\Big{|}^{2}.1_{\big{(}\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\big{)}^{c}\cup\big{(}\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})\big{)}^{c}}\Big{]}$
$\displaystyle\leq
4\pi^{2}.\mathbb{P}\left(\big{(}\widetilde{E}_{2,n}(u_{x},\widehat{h}_{2})\big{)}^{c}\right)+4\pi^{2}.\mathbb{P}\left(\big{(}\widetilde{E}_{1,n}(u_{x},\widehat{h}_{1})\big{)}^{c}\right)\leq
2.4\pi^{2}.4.n^{-q},$

by Corollary 5.6. For $q\geq 1$, we get that $n^{-q}$ is negligible in
comparison with
$C_{1}^{2}.\psi_{n}(\beta_{1})^{2}=C_{1}^{2}.\Big{(}\dfrac{\log(n)}{n}\Big{)}^{\frac{2\beta_{1}}{2\beta_{1}+1}}$
and
$C_{2}^{2}.\psi_{n}(\beta_{2})^{2}=C_{2}^{2}.\Big{(}\dfrac{\log(n)}{n}\Big{)}^{\frac{2\beta_{2}}{2\beta_{2}+1}}$.
This concludes the proof of Theorem 3.5. ∎

####  5.4.3 Proof of Theorem 3.10

Before tackling the proof of Theorem 3.10, the next lemma shows that the von
Mises density with zero mean direction satisfies condition (16).

######  Lemma 5.7.

The von Mises density with mean direction $\mu$ and concentration parameter
$\kappa$ satisfies condition (16).

###### Proof of Lemma 5.7.

We recall the expression of the von Mises density with mean direction
$\mu\in[-\pi,\pi)$ and concentration parameter $\kappa>0$ :

$f_{vM}(x)=c(\kappa)e^{\kappa\cos(x-\mu)}1_{x\in[-\pi,\pi]},\quad
x\in\mathbb{R}$

with $c(\kappa)$ a normalizing constant. Let us prove that $f_{vM}$ satisfies:

$\exists
p_{*}>0:\int_{-\pi}^{\pi}f_{vM}(x)\log\frac{f_{vM}(x)}{f_{vM}(x+y)}dx\leq
p_{*}y^{2},$

for all $y\in\mathbb{R}$. We have that

$\displaystyle\int_{-\pi}^{\pi}f_{vM}(x)\log\frac{f_{vM}(x)}{f_{vM}(x+y)}dx$
$\displaystyle=$ $\displaystyle
c(\kappa)\kappa\int_{-\pi}^{\pi}e^{\kappa\cos(x-\mu)}(\cos(x-\mu)-\cos(x+y-\mu))dx$
$\displaystyle=$ $\displaystyle
2c(\kappa)\kappa\sin\frac{y}{2}\int_{-\pi}^{\pi}e^{\kappa\cos(x-\mu)}\sin(x-\mu+\frac{y}{2})dx$
$\displaystyle=$ $\displaystyle
2c(\kappa)\kappa\sin\frac{y}{2}\int_{-\pi}^{\pi}e^{\kappa\cos(x-\mu)}\Big{(}\sin(x-\mu)\cos\frac{y}{2}+\sin\frac{y}{2}\cos(x-\mu)\Big{)}dx$
$\displaystyle=$ $\displaystyle
2c(\kappa)\kappa\Big{(}\sin\frac{y}{2}\Big{)}^{2}\underbrace{\int_{-\pi}^{\pi}e^{\kappa\cos(x-\mu)}\cos(x-\mu)dx}_{=C(\kappa)>0}$
$\displaystyle\leq$ $\displaystyle 2c(\kappa)\kappa C(\kappa)\frac{y^{2}}{4}$

for all $y\in\mathbb{R}$. Then, with

$p_{*}=\frac{c(\kappa)\kappa C(\kappa)}{2},$

we have for any $y\in\mathbb{R}$,

$\int_{-\pi}^{\pi}f_{vM}(x)\log\frac{f_{vM}(x)}{f_{vM}(x+y)}dx\leq
p_{*}y^{2}.$

∎

###### Proof of Theorem 3.10.

To prove the lower bound stated in Theorem 3.10, we follow the lines of
Section 2.5 in [22] for the regression at a point. The differences with our
problem lie in the circular response and the randomness of the $X_{i}$’s.

We consider $m_{0}(x)=0$ and $m_{1}(x)=Lh_{n}^{\beta}K(\frac{x-x_{0}}{h_{n}})$
with $h_{n}=c_{0}n^{-\frac{1}{2\beta+1}}$ and
$K:\mathbb{R}\longmapsto\mathbb{S}^{1}$ satisfying:

$K\in\tilde{\Sigma}(\beta,1/2)\cap C^{\infty}(\mathbb{R}),\quad K(u)>0\iff
u\in]-1/2,1/2[.$

Such functions $K$ exists. For instance, for a sufficiently small $a>0$, one
can take

$K(x)=a\exp\left(-\frac{1}{1-4x^{2}}\right)1_{[-0.5;0.5]}(x).$

We have now to check three points which are developed in the sequel.

  1. 1.

Let us prove that $m_{j}\in\tilde{\Sigma}(\beta,L)$. For
$l=\lfloor\beta\rfloor$ we have

$m_{1}^{(l)}(x)=Lh_{n}^{\beta-l}K^{(l)}\left(\frac{x-x_{0}}{h_{n}}\right)$

then, with $u=\frac{x-x_{0}}{h_{n}}$ and
$u^{\prime}=\frac{x^{\prime}-x_{0}}{h_{n}}$,

$\displaystyle|m_{1}^{(l)}(x)-m_{1}^{(l)}(x^{\prime})|$ $\displaystyle=$
$\displaystyle Lh_{n}^{\beta-l}|K^{(l)}(u)-K^{(l)}(u^{\prime})|$
$\displaystyle\leq$ $\displaystyle
Lh_{n}^{\beta-l}|u-u^{\prime}|^{\beta-l}/2=L|x-x^{\prime}|^{\beta-l}/2.$

Then, $m_{1}\in\tilde{\Sigma}(\beta,L)$.

  2. 2.

Let us show that $|m_{0}(x_{0})-m_{1}(x_{0})|\geq 2s_{n}$. We have

$|m_{0}(x_{0})-m_{1}(x_{0})|=|m_{1}(x_{0})|=Lh_{n}^{\beta}K(0)=Lc_{0}^{\beta}K(0)n^{-\frac{\beta}{2\beta+1}},$

then the condition is fulfilled with
$s_{n}=\frac{1}{2}Lc_{0}^{\beta}K(0)n^{-\frac{\beta}{2\beta+1}}=:A\psi_{n}.$

  3. 3.

Using the classical reduction to a two test hypotheses problem for the
pointwise regression problem, we get for any estimator $T_{n}$:

$\displaystyle\sup_{m\in\tilde{\Sigma}(\beta,L)}\mathbb{E}_{m}[\psi_{n}^{-2}|T_{n}-m(x_{0})|^{2}]$
$\displaystyle\geq$ $\displaystyle
A^{2}\max_{m\in\\{m_{0},m_{1}\\}}\mathbb{P}_{m}(|T_{n}-m(x_{0})|\geq
A\psi_{n})$ (36) $\displaystyle\geq$
$\displaystyle\frac{A^{2}}{2}\mathbb{E}_{X_{1},\dots,X_{n}}\Big{[}\mathbb{P}_{m_{0}}(|T_{n}-m_{0}(x_{0})|\geq
A\psi_{n}|X_{1},\dots,X_{n})$ $\displaystyle\hskip
85.35826pt+\mathbb{P}_{m_{1}}(|T_{n}-m_{1}(x_{0})|\geq
A\psi_{n}|X_{1},\dots,X_{n})\Big{]}$ $\displaystyle\geq$
$\displaystyle\frac{A^{2}}{2}\mathbb{E}_{X_{1},\dots,X_{n}}\Big{[}\inf_{\psi}\Big{\\{}\mathbb{P}_{m_{0}}(\psi\neq
0|X_{1},\dots,X_{n})$ $\displaystyle\hskip
85.35826pt+\mathbb{P}_{m_{1}}(\psi\neq 1|X_{1},\dots,X_{n})\Big{\\}}\Big{]},$

where $\inf_{\psi}$ denotes the infimum over all tests $\psi$ taking values in
$\\{0,1\\}$.

Now let us fix the $X_{i}$’s. The minimum average probability
$\overline{p}_{e,1}$ is defined as (see page 116 in [22]):

$\overline{p}_{e,1}:=\frac{1}{2}\inf_{\psi}\Big{\\{}\mathbb{P}_{m_{0}}(\psi\neq
0|X_{1},\dots,X_{n})+\mathbb{P}_{m_{1}}(\psi\neq
1|X_{1},\dots,X_{n})\Big{\\}}.$

We have for the Kullback Leibler divergence (still with the $X_{i}$’s fixed)

$\mathbb{K}(\mathbb{P}_{m_{0}},\mathbb{P}_{m_{1}})=\int\log\left(\frac{d\mathbb{P}_{m_{0}}}{d\mathbb{P}_{m_{1}}}\right)d\mathbb{P}_{m_{0}}=\sum_{i=1}^{n}\int\log\frac{p_{\zeta}(y)}{p_{\zeta}(y-m_{1}(X_{i}))}p_{\zeta}(y)dy.$
(37)

There exists $n_{0}$ such that $\forall n>n_{0}$, we have $nh_{n}\geq 1$ and
$Lh_{n}^{\beta}K_{\max}\leq y_{0}$ where $K_{\max}=\max_{x}K(x)$. Using (37)
and (16), we have:

$\displaystyle\mathbb{K}(\mathbb{P}_{m_{0}},\mathbb{P}_{m_{1}})$
$\displaystyle\leq$ $\displaystyle p_{*}\sum_{i=1}^{n}m_{1}^{2}(X_{i})$
$\displaystyle\leq$ $\displaystyle
p_{*}L^{2}h_{n}^{2\beta}K^{2}_{\max}\sum_{i=1}^{n}1_{\left\\{\left|\frac{X_{i}-x_{0}}{h_{n}}\right|\leq\frac{1}{2}\right\\}}.$

Now taking the expectation and using that the density of the $X_{i}$’s is
bounded by $\mu_{0}$, we get:

$\mathbb{E}_{X_{1},\dots,X_{n}}\mathbb{K}(\mathbb{P}_{m_{0}},\mathbb{P}_{m_{1}})\leq
p_{*}L^{2}K^{2}_{\max}h_{n}^{2\beta}n\mathbb{P}\left(\left|\frac{X_{1}-x_{0}}{h_{n}}\right|\leq\frac{1}{2}\right)\leq
p_{*}L^{2}K^{2}_{\max}\mu_{0}h_{n}^{2\beta+1}n.$

For $\alpha<2\log(2)$, since $h_{n}=c_{0}n^{-\frac{1}{2\beta+1}}$, setting

$c_{0}=\left(\frac{\alpha}{p_{*}\mu_{0}L^{2}K^{2}_{\max}}\right)^{\frac{1}{2\beta+1}},$

we get that

$\mathbb{E}_{X_{1},\dots,X_{n}}\mathbb{K}(\mathbb{P}_{m_{0}},\mathbb{P}_{m_{1}})\leq\alpha.$

As in Lemma 2.10 of [22], we introduce the function
$\mathcal{H}(x)=-x\log(x)-(1-x)\log(1-x)$ for $x\in(0,1)$ and
$\mathcal{H}(0)=\mathcal{H}(1)=0$. Inequality (2.70) of [22] with $M=1$ gives

$\mathbb{E}_{X_{1},\dots,X_{n}}[\mathcal{H}(\overline{p}_{e,1})]\geq\log(2)-\frac{1}{2}\mathbb{E}_{X_{1},\dots,X_{n}}\mathbb{K}(\mathbb{P}_{m_{0}},\mathbb{P}_{m_{1}})\geq\log(2)-\frac{\alpha}{2},$

since
$\mathbb{E}_{X_{1},\dots,X_{n}}\mathbb{K}(\mathbb{P}_{m_{0}},\mathbb{P}_{m_{1}})\leq\alpha$.
Since $\mathcal{H}$ is concave,
$\mathcal{H}(\mathbb{E}_{X_{1},\dots,X_{n}}[\overline{p}_{e,1})])\geq\mathbb{E}_{X_{1},\dots,X_{n}}[\mathcal{H}(\overline{p}_{e,1})]$
and

$\mathbb{E}_{X_{1},\dots,X_{n}}[\overline{p}_{e,1}]\geq\mathcal{H}^{-1}\big{(}\log
2-\frac{\alpha}{2}\big{)}>0,$

with, for any $t>0$,
$\mathcal{H}^{-1}(t)=\min\\{p\in(0,\frac{1}{2}]:\mathcal{H}(p)\geq t\\}$.
Hence we deduce using (36)

$\displaystyle\sup_{m\in\tilde{\Sigma}(\beta,L)}\mathbb{E}_{m}\big{[}\psi_{n}^{-2}|T_{n}-m(x_{0})|^{2}\big{]}\geq
A^{2}\mathcal{H}^{-1}\big{(}\log 2-\frac{\alpha}{2}\big{)},$

where the right hand side is a positive constant. This concludes the proof of
Theorem 3.10.

∎

##  6 Conclusion

Considering nonparametric regression for circular data, we derive minimax
convergence rates and prove near optimal properties of our kernel estimate
combined with a warping strategy on anisotropic Hölder classes of functions
for pointwise estimation. The bandwidth parameter is selected by using a data-
driven Goldenshluger-Lepski type procedure. After tuning hyperparameters of
our estimate, we show that it remains very competitive with respect to
existing methods.

As a natural extension, it could be very challenging to investigate our
regression problem with a response on the sphere $\mathbb{S}^{2}$ or more
generally on the unit hypersphere $\mathbb{S}^{d-1}$. The case of predictors
$X\in\mathbb{S}^{d-1}$ and a response $\Theta\in\mathbb{S}^{d-1}$ has been
tackled in [8]. The spherical context is of course more complicated than the
circular one and the arctangent function approach used here is not easily
generalizable in the spherical setting. In [8], Di Marzio et al. proposed a
local constant estimator by smoothing on each component of the response. Once
again no rates of convergence were obtained. Hence, in a future work, a first
task would be to obtain convergence rates and then investigate adaptation
issue.

## References

  * [1] M. Alonso-Pena and R. Crujeiras.  Nonparametric multimodal regression for circular data.  preprint, 2020.
  * [2] N. Bochkina and T. Sapatinas.  Minimax rates of convergence and optimality of Bayes factor wavelet regression estimators under pointwise risks.  Statistica Sinica, 19: $1389$-$1406$, 2009.
  * [3] G. Chagny.  Penalization versus Goldenshluger-Lepski strategies in warped bases regression.  ESAIM Probab. Stat., 17: 328-358, 2013.
  * [4] G. Chagny.  Adaptive warped kernel estimators.  Scandinavian Journal of Statistics, 42(2): 336-360, 2015.
  * [5] G. Chagny, T. Laloë and R. Servien.  Multivariate adaptive warped kernel estimation.  Electron. J. Statist., 13(1): 1759-1789, 2019.
  * [6] F. Comte and C. Lacour.  Anisotropic adaptive kernel deconvolution.  Ann. Inst. H. Poincaré Probab. Statist., 49(2): 569-609, 2013.
  * [7] N.I. Fisher and A.J. Lee.  Regression models for angular responses.  Biometrics, 48: 665-677, 1992.
  * [8] M. Di Marzio, A. Panzera and C.C. Taylor.  Nonparametric Regression for Spherical Data.  Journal of the American Statistical Association, 109:748-763, 2014.
  * [9] A. Goldenshluger and O. Lepski.  Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality.  Ann. Statist, 39(3): 1608-1632, 2011.
  * [10] A.L. Gould.  A regression technique for angular variates.  Biometrics, 25: 683-700, 1969.
  * [11] S.R. Jammalamadaka and A. SenGupta.  Topics in Circular Statistics.  World Scientific, Singapore, 2001.
  * [12] R.A. Johnson and T.E. Wehlry.  Some angular-linear distributions and related regression models.  Journal of the American Statistical Association, 73: 602-606, 1978.
  * [13] G. Kerkyacharian and D. Picard.  Regression in random design and warped wavelets.  Bernoulli 10(6):1053-1105, 2004.
  * [14] C. Ley and T. Verdebout .  Modern Directional Statistics.  (1st ed.). Chapman and Hall/CRC, 2017.
  * [15] K.V. Mardia and P.E. Jupp.  Directional Statistics.  New York, NY: John Wiley, 2000.
  * [16] M. Di Marzio, A. Panzera and C.C. Taylor.  Non-parametric regression for circular responses.  Scandinavian Journal of Statistics, 40: 238-255, 2013.
  * [17] A. Meilán-Vila, M. Francisco-Fernández, R.M. Crujeiras and A. Panzera.  Nonparametric multiple regression estimation for circular response.  TEST, 30: 650-672, (2021).
  * [18] A. Pewsey and E. García-Portugués.  Recent advances in directional statistics.  TEST, 30: 1-58, 2021.
  * [19] T.M. Pham Ngoc.  Regression in random design and Bayesian warped wavelets estimators.  Electron. J. Stat, 3: 1084-1112, 2009.
  * [20] T.M. Pham Ngoc.  Adaptive optimal kernel density estimation for directional data.  J. Multivariate Anal., 173: 248-267, 2019.
  * [21] B. Presnell, S.P. Morrison and R.C. Littel.  Projected multivariate linear models for directional data.  Journal of the American Statistical Association, 93: 1068-1077, 1998.
  * [22] A.B. Tsybakov.  Introduction to nonparametric estimation.  Springer Series in Statistics. Springer, New York, 2009.