$\sup_{\eta\in\mathbb{T}}\|\Delta_{12}(\partial_{\theta}^{\ell}\mathcal{J})(\ast,\cdot,\centerdot,\eta+\centerdot)\|_{q,s}^{\gamma,\mathcal{O}}\lesssim\|\Delta_{12}i\|_{q,s+3+\ell}^{\gamma,\mathcal{O}}+\|\Delta_{12}i\|_{q,s_{0}+3}^{\gamma,\mathcal{O}}\max_{j\in\\{1,2\\}}\|\mathfrak{I}_{j}\|_{q,s+3+\ell}^{\gamma,\mathcal{O}}.$
(4.59)

where $*,\cdot,\centerdot,$ denote the variables $\alpha,\varphi,\theta$ and
$\displaystyle\mathfrak{I}_{\ell}(\varphi)\triangleq
i_{\ell}(\varphi)-(\varphi,0,0).$

Now we shall start the reduction procedure of the operator
$\mathscr{L}_{\perp}.$ The first step is the reduction of the transport part
of $\mathcal{L}_{\varepsilon r}$ in (4.54), which is done by conjugation with
quasi-periodic symplectic change of variables close to the identity similarly
to [5, 15, 30, 51]. Actually, we apply here [51, Prop. 6.2]. The result is the
following.

######  Proposition 4.3.

Let $(d,s_{0},S,\gamma,q,q_{0})$ satisfy (4.2), (4.3), (4.4) and (4.49). We
consider the following parameters

$\begin{array}[]{ll}\upsilon\triangleq\frac{1}{q_{0}+3},&\tau_{1}\triangleq
dq_{0}+1,\\\ s_{l}\triangleq
s_{0}+\tau_{1}q+\tau_{1}+2,&\overline{\mu}_{2}\triangleq
4\tau_{1}q+6\tau_{1}+3,\\\ \overline{s}_{l}\triangleq
s_{l}+\tau_{2}q+\tau_{2},&\overline{s}_{h}\triangleq\frac{3}{2}\overline{\mu}_{2}+s_{l}+1,\\\
\sigma_{1}\triangleq s_{0}+\tau_{1}q+2\tau_{1}+4,&\sigma_{2}\triangleq
s_{0}+\sigma_{1}+3.\end{array}$ (4.60)

For every choice of additional parameters $(\mu_{2},\mathtt{p},s_{h})$ with
the constraints

$\mu_{2}\geqslant\overline{\mu}_{2},\qquad\mathtt{p}\geqslant 0,\qquad
s_{h}\geqslant\max\left(\frac{3}{2}\mu_{2}+s_{l}+1,\overline{s}_{h}+\mathtt{p}\right),$
(4.61)

there exists $\varepsilon_{0}>0$ such that if the following smallness
condition holds

$\varepsilon\gamma^{-1}N_{0}^{\mu_{2}}\leqslant\varepsilon_{0},\qquad\|\mathfrak{I}_{0}\|_{q,s_{h}+\sigma_{2}}^{\gamma,\mathcal{O}}\leqslant
1,$ (4.62)

then there exist $\mathtt{m}_{i_{0}}^{\infty}\in
W^{q,\infty,\gamma}(\mathcal{O},\mathbb{C})$ $\varepsilon$-close to the
equilibrium one $V_{0}$ in (4.25), namely

$\|\mathtt{m}_{i_{0}}^{\infty}-V_{0}\|_{q}^{\gamma,\mathcal{O}}\lesssim\varepsilon$
(4.63)

and an invertible quasi-periodic symplectic change of variables $\mathscr{B}$
in the form

$\mathscr{B}\triangleq(1+\partial_{\theta}\beta)\mathcal{B},\qquad\mathcal{B}\rho(\mu,\varphi,\theta)\triangleq\rho\big{(}\mu,\varphi,\theta+\beta(\mu,\varphi,\theta)\big{)},\qquad\beta\in
W^{q,\infty,\gamma}(\mathcal{O},H^{S})$

with inverse $\mathscr{B}^{-1}$ in the form

$\mathscr{B}^{-1}=(1+\partial_{y}\widehat{\beta})\mathcal{B}^{-1},\qquad\mathcal{B}^{-1}\rho(\mu,\varphi,y)=\rho\big{(}\mu,\varphi,y+\widehat{\beta}(\mu,\varphi,y)\big{)}$

with

$y=\theta+\beta(\mu,\varphi,\theta)\quad\Leftrightarrow\quad\theta=y+\widehat{\beta}(\mu,\varphi,y)$

enjoying the symmetry properties

$\beta(\alpha,\omega,-\varphi,-\theta)=-\beta(\alpha,\omega,\varphi,\theta),\qquad\widehat{\beta}(\alpha,\omega,-\varphi,-\theta)=-\widehat{\beta}(\alpha,\omega,\varphi,\theta)$
(4.64)

and the following estimates for all $s\in[s_{0},S]$

$\|\mathscr{B}^{\pm 1}\rho\|_{q,s}^{\gamma,\mathcal{O}}+\|\mathcal{B}^{\pm
1}\rho\|_{q,s}^{\gamma,\mathcal{O}}\lesssim\|\rho\|_{q,s}^{\gamma,\mathcal{O}}+\varepsilon\gamma^{-1}\|\mathfrak{I}_{0}\|_{q,s+\sigma_{1}}^{\gamma,\mathcal{O}}\|\rho\|_{q,s_{0}}^{\gamma,\mathcal{O}}$
(4.65)

and

$\|\widehat{\beta}\|_{q,s}^{\gamma,\mathcal{O}}\lesssim\|\beta\|_{q,s}^{\gamma,\mathcal{O}}\lesssim\varepsilon\gamma^{-1}\left(1+\|\mathfrak{I}_{0}\|_{q,s+\sigma_{1}}^{\gamma,\mathcal{O}}\right)$
(4.66)

such that for any $n\in\mathbb{N}$, then in restriction to the Cantor-like set

$\mathscr{O}_{\textnormal{\tiny{transport}}}^{n,\gamma,\tau_{1}}(i_{0})\triangleq\bigcap_{(l,j)\in\mathbb{Z}^{d}\times\mathbb{Z}\setminus\\{(0,0)\\}\atop|l|\leqslant
N_{n}}\left\\{(\alpha,\omega)\in\mathcal{O}\quad\textnormal{s.t.}\quad\big{|}\omega\cdot
l+j\mathtt{m}_{i_{0}}^{\infty}(\alpha,\omega)\big{|}>\tfrac{4\gamma^{\upsilon}\langle
j\rangle}{\langle l\rangle^{\tau_{1}}}\right\\},$ (4.67)

the following reduction holds

$\mathfrak{L}_{\varepsilon
r}\triangleq\mathscr{B}^{-1}\mathcal{L}_{\varepsilon
r}\mathscr{B}=\omega\cdot\partial_{\varphi}+\mathtt{m}_{i_{0}}^{\infty}\partial_{\theta}+\partial_{\theta}\mathcal{K}_{\alpha}\ast\cdot+\partial_{\theta}\mathbf{R}_{\varepsilon
r}+\mathbf{E}_{n}^{0},$ (4.68)

with $\mathcal{K}_{\alpha}$ as in (4.25) and
$\mathbf{E}_{n}^{0}=\mathbf{E}_{n}^{0}(\alpha,\omega,i_{0})$ a linear operator
satisfying

$\|\mathbf{E}_{n}^{0}\rho\|_{q,s_{0}}^{\gamma,\mathcal{O}}\lesssim\varepsilon
N_{0}^{\mu_{2}}N_{n+1}^{-\mu_{2}}\|\rho\|_{q,s_{0}+2}^{\gamma,\mathcal{O}}.$
(4.69)

The operator $\mathbf{R}_{\varepsilon r}$ is a real and reversibility
preserving integral operator satisfying

$\displaystyle\forall
s\in[s_{0},S],\quad\max_{k\in\\{0,1,2\\}}\|\partial_{\theta}^{k}\mathbf{R}_{\varepsilon
r}\|_{\textnormal{\tiny{O-d}},q,s}^{\gamma,\mathcal{O}}$
$\displaystyle\lesssim\varepsilon\gamma^{-1}\left(1+\|\mathfrak{I}_{0}\|_{q,s+\sigma_{2}}^{\gamma,\mathcal{O}}\right).$
(4.70)

In addition, for any tori $i_{1}$ and $i_{2}$ both satisfying (4.62), we have

$\|\Delta_{12}\mathtt{m}_{i}^{\infty}\|_{q}^{\gamma,\mathcal{O}}\lesssim\varepsilon\|\Delta_{12}i\|_{q,\overline{s}_{h}+2}^{\gamma,\mathcal{O}},\qquad\|\Delta_{12}\beta\|_{q,\overline{s}_{h}+\mathtt{p}}^{\gamma,\mathcal{O}}+\|\Delta_{12}\widehat{\beta}\|_{q,\overline{s}_{h}+\mathtt{p}}^{\gamma,\mathcal{O}}\lesssim\varepsilon\gamma^{-1}\|\Delta_{12}i\|_{q,\overline{s}_{h}+\mathtt{p}+\sigma_{1}}^{\gamma,\mathcal{O}}$
(4.71)

and

$\displaystyle\max_{k\in\\{0,1\\}}\|\Delta_{12}(\partial_{\theta}^{k}\mathbf{R}_{\varepsilon
r})\|_{\textnormal{\tiny{O-d}},q,\overline{s}_{h}+\mathtt{p}}^{\gamma,\mathcal{O}}$
$\displaystyle\lesssim\varepsilon\gamma^{-1}\|\Delta_{12}i\|_{q,\overline{s}_{h}+\mathtt{p}+\sigma_{2}}^{\gamma,\mathcal{O}}.$
(4.72)

###### Proof.

Using the computations in [51, Prop. 6.2] and [42, Prop. 6.2] for the
$V_{\varepsilon r}^{\textnormal{\tiny{E}}}$ and $V_{\varepsilon
r}^{\textnormal{\tiny{SW}}}$ in (4.16)-(4.19), we get the following estimate
for $V_{\varepsilon r}$ in (4.22),

$\|V_{\varepsilon
r}-V_{0}\|_{q,s}^{\gamma,\mathcal{O}}\leqslant\|V_{\varepsilon
r}^{\textnormal{\tiny{E}}}+\tfrac{1}{2}\|_{q,s}^{\gamma,\mathcal{O}}+\|V_{\varepsilon
r}^{\textnormal{\tiny{SW}}}-I_{1}K_{1}\|_{q,s}^{\gamma,\mathcal{O}}\lesssim\varepsilon\left(1+\|\mathfrak{I}_{0}\|_{q,s+1}^{\gamma,\mathcal{O}}\right).$

This allows us to start a KAM reduction procedure as detailed in [51, prop
6.2], which provides
$\mathscr{O}_{\textnormal{\tiny{transport}}}^{n,\gamma,\tau_{1}}(i_{0}),$
$\mathtt{m}_{i_{0}}^{\infty}$, $\beta$ and $\mathbf{E}_{n}^{0}$ as in the
statement, with the corresponding estimates (4.65), (4.66), (4.69), (4.71) and
symmetry (4.64) such that in restriction to
$\mathscr{O}_{\textnormal{\tiny{transport}}}^{n,\gamma,\tau_{1}}(i_{0})$ we
have

$\mathscr{B}^{-1}\Big{(}\omega\cdot\partial_{\varphi}+\partial_{\theta}\big{(}V_{\varepsilon
r}\cdot\big{)}\Big{)}\mathscr{B}=\omega\cdot\partial_{\varphi}+\mathtt{m}_{i_{0}}^{\infty}\partial_{\theta}+\mathbf{E}_{n}^{0}.$

Now, we shall look at the conjugation effect on the nonlocal term. Refering to
[51, Lem. 5.1] and [42, Lem. 5.1], we have the following decompositions for
the operators in (4.17)-(4.20),

$\displaystyle\partial_{\theta}\mathbf{L}_{\varepsilon
r}^{\textnormal{\tiny{E}}}$
$\displaystyle=\partial_{\theta}\mathcal{K}\ast\cdot+\partial_{\theta}\mathbf{L}_{\varepsilon
r,1}^{\textnormal{\tiny{E}}},\qquad\mathbf{L}_{\varepsilon
r,1}^{\textnormal{\tiny{E}}}(\rho)(\varphi,\theta)\triangleq\int_{\mathbb{T}}\rho(\varphi,\eta)\mathbb{K}_{\varepsilon
r}^{\textnormal{\tiny{E}}}(\varphi,\theta,\eta)d\eta,$ (4.73)
$\displaystyle\partial_{\theta}\mathbf{L}_{\varepsilon
r}^{\textnormal{\tiny{SW}}}$
$\displaystyle=\partial_{\theta}\mathcal{Q}_{\alpha}\ast\cdot+\partial_{\theta}\mathbf{L}_{\varepsilon
r,1}^{\textnormal{\tiny{SW}}},\qquad\mathbf{L}_{\varepsilon
r,1}^{\textnormal{\tiny{SW}}}(\rho)(\alpha,\varphi,\theta)\triangleq\int_{\mathbb{T}}\rho(\varphi,\eta)\mathbb{K}_{\varepsilon
r}^{\textnormal{\tiny{SW}}}(\alpha,\varphi,\theta,\eta)d\eta,$ (4.74)

where the kernels $\mathbb{K}_{\varepsilon r}^{\textnormal{\tiny{E}}}$ and
$\mathbb{K}_{\varepsilon r}^{\textnormal{\tiny{SW}}}$ are respectively defined
by

$\displaystyle\mathbb{K}_{\varepsilon
r}^{\textnormal{\tiny{E}}}(\varphi,\theta,\eta)$
$\displaystyle\triangleq\log\big{(}v_{\varepsilon
r}(\varphi,\theta,\eta)\big{)},\qquad v_{\varepsilon
r}(\varphi,\theta,\eta)\triangleq\left(\left(\tfrac{R(\varphi,\theta)-R(\varphi,\eta)}{2\sin\left(\frac{\theta-\eta}{2}\right)}\right)^{2}+R(\varphi,\theta)R(\varphi,\eta)\right)^{\frac{1}{2}}$
$\displaystyle\mathbb{K}_{\varepsilon
r}^{\textnormal{\tiny{SW}}}(\varphi,\theta,\eta)$
$\displaystyle\triangleq\mathscr{K}(\theta-\eta)\mathscr{K}_{\varepsilon
r,1}(\lambda,\varphi,\theta,\eta)+\mathscr{K}_{\varepsilon
r,2}(\lambda,\varphi,\theta,\eta),\qquad\mathscr{K}(\theta)\triangleq\sin^{2}\left(\tfrac{\theta}{2}\right)\log\left(\left|\sin\left(\tfrac{\theta}{2}\right)\right|\right),$
$\displaystyle\mathscr{K}_{\varepsilon r,1}(\lambda,\varphi,\theta,\eta)$
$\displaystyle\triangleq\sum_{m=1}^{\infty}\tfrac{(2\lambda)^{2m}}{(m!)^{2}}\sin^{2m-2}\left(\tfrac{\theta-\eta}{2}\right)\big{(}1-v_{\varepsilon
r}(\varphi,\theta,\eta)\big{)},\qquad\lambda\triangleq\frac{1}{\alpha}$
$\displaystyle\mathscr{K}_{\varepsilon r,2}(\lambda,\varphi,\theta,\eta)$
$\displaystyle\triangleq\log(\lambda)\sin^{2}\left(\frac{\eta-\theta}{2}\right)\mathscr{K}_{\varepsilon
r,1}(\lambda,\varphi,\theta,\eta)-\mathbb{K}_{\varepsilon
r}^{\textnormal{\tiny{E}}}(\varphi,\theta,\eta)I_{0}\big{(}\lambda
A_{\varepsilon r}(\varphi,\theta,\eta)\big{)}$
$\displaystyle\quad+f\big{(}\lambda A_{\varepsilon
r}(\varphi,\theta,\eta)\big{)}-f\left(2\lambda\left|\sin\left(\tfrac{\eta-\theta}{2}\right)\right|\right),\qquad
f\textnormal{ analytic.}$

One can easily check that (4.55) implies

$\mathbb{K}_{\varepsilon
r}^{\textnormal{\tiny{E}}}(-\varphi,-\theta,-\eta)=\mathbb{K}_{\varepsilon
r}^{\textnormal{\tiny{E}}}(\varphi,\theta,\eta),\qquad\mathbb{K}_{\varepsilon
r}^{\textnormal{\tiny{SW}}}(\alpha,-\varphi,-\theta,-\eta)=\mathbb{K}_{\varepsilon
r}^{\textnormal{\tiny{SW}}}(\alpha,\varphi,\theta,\eta)$ (4.75)

Hence, in restriction to
$\mathscr{O}_{\textnormal{\tiny{transport}}}^{n,\gamma,\tau_{1}}(i_{0})$ we
have by (4.73)-(4.74),

$\displaystyle\mathscr{B}^{-1}\mathcal{L}_{\varepsilon r}\mathscr{B}$
$\displaystyle=\omega\cdot\partial_{\varphi}+\mathtt{m}_{i_{0}}^{\infty}\partial_{\theta}+\partial_{\theta}\mathcal{B}^{-1}\mathbf{L}_{\varepsilon
r}^{\textnormal{\tiny{E}}}\mathscr{B}+\partial_{\theta}\mathcal{B}^{-1}\mathbf{L}_{\varepsilon
r}^{\textnormal{\tiny{SW}}}\mathscr{B}+\mathbf{E}_{n}^{0}$
$\displaystyle=\omega\cdot\partial_{\varphi}+\mathtt{m}_{i_{0}}^{\infty}\partial_{\theta}+\partial_{\theta}\mathcal{K}_{\alpha}\ast\cdot+\partial_{\theta}\mathbf{R}_{\varepsilon
r}+\mathbf{E}_{n}^{0},$

with

$\mathbf{R}_{\varepsilon r}\triangleq\mathcal{B}^{-1}\mathbf{L}_{\varepsilon
r,1}^{\textnormal{\tiny{E}}}\mathscr{B}+\mathcal{B}^{-1}\mathbf{L}_{\varepsilon
r,1}^{\textnormal{\tiny{SW}}}\mathscr{B}+\Big{(}\mathcal{B}^{-1}\big{(}\mathcal{K}\ast\cdot\big{)}\mathscr{B}-\mathcal{K}\ast\cdot\Big{)}+\Big{(}\mathcal{B}^{-1}\big{(}\mathcal{Q}_{\alpha}\ast\cdot\big{)}\mathscr{B}-\mathcal{Q}_{\alpha}\ast\cdot\Big{)}.$

The estimates (4.70) and (4.72) are obtained similarly to [51, Prop. 6.2] and
[42, Prop. 6.2] by using Lemma 4.3, Lemma 4.1 and (4.66)-(4.71). In
particular, the reversibility property follows from (4.75)-(4.64). This
concludes the proof of Proposition 4.3. ∎

Then we study the action of the localization in the normal directions. For
that, we introduce the operator

$\mathscr{B}_{\perp}\triangleq\Pi_{\mathbb{S}_{0}}^{\perp}\mathscr{B}\Pi_{\mathbb{S}_{0}}^{\perp},$

which satisfies by virtue of (4.65) the following estimate

$\|\mathscr{B}_{\perp}^{\pm
1}\rho\|_{q,s}^{\gamma,\mathcal{O}}\lesssim\|\rho\|_{q,s}^{\gamma,\mathcal{O}}+\varepsilon\gamma^{-1}\|\mathfrak{I}_{0}\|_{q,s+\sigma_{1}}^{\gamma,\mathcal{O}}\|\rho\|_{q,s_{0}}^{\gamma,\mathcal{O}}.$
(4.76)

Hence, the result reads as follows.

######  Proposition 4.4.

Let
$(d,s_{0},S,\gamma,q,q_{0},\tau_{1},s_{h},\overline{s}_{h},\sigma_{2},\mathtt{p})$
satisfy (4.2), (4.3), (4.4), (4.49), (4.60) and (4.61). There exist
$\varepsilon_{0}>0$ and
$\sigma_{3}=\sigma_{3}(\tau_{1},q,d,s_{0})\geqslant\sigma_{2}$ such that if
the following smallness condition holds

$\varepsilon\gamma^{-1}N_{0}^{\mu_{2}}\leqslant\varepsilon_{0},\qquad\|\mathfrak{I}_{0}\|_{q,s_{h}+\sigma_{3}}^{\gamma,\mathcal{O}}\leqslant
1,$

then for any $n\in\mathbb{N}^{*},$ in restriction to the Cantor-like set
$\mathscr{O}_{\textnormal{\tiny{transport}}}^{n,\gamma,\tau_{1}}(i_{0})$
defined in (4.67), the following decomposition holds

$\displaystyle\mathscr{B}_{\perp}^{-1}\mathscr{L}_{\perp}\mathscr{B}_{\perp}$
$\displaystyle=\big{(}\omega\cdot\partial_{\varphi}+\mathtt{m}_{i_{0}}^{\infty}\partial_{\theta}+\partial_{\theta}\mathcal{K}_{\alpha}\ast\cdot\big{)}\Pi_{\mathbb{S}_{0}}^{\perp}+\mathscr{R}_{0}+\mathbf{E}_{n}^{1}$
$\displaystyle\triangleq\omega\cdot\partial_{\varphi}\Pi_{\mathbb{S}_{0}}^{\perp}+\mathscr{D}_{0}+\mathscr{R}_{0}+\mathbf{E}_{n}^{1}$
$\displaystyle\triangleq\mathscr{L}_{0}+\mathbf{E}_{n}^{1}.$ (4.77)

The operator
$\mathscr{D}_{0}=\Pi_{\mathbb{S}_{0}}^{\perp}\mathscr{D}_{0}\Pi_{\mathbb{S}_{0}}^{\perp}$
is a reversible diagonal operator described by

$\mathscr{D}_{0}\mathbf{e}_{l,j}={\rm
i}\mathrm{d}_{j}^{0}\,\mathbf{e}_{l,j},\qquad\mathrm{d}_{j}^{0}(\alpha,\omega,i_{0})\triangleq\Omega_{j}^{\textnormal{\tiny{E}}}(\alpha)+jr^{0}(\alpha,\omega,i_{0}),\qquad
r^{0}(\alpha,\omega,i_{0})\triangleq\mathtt{m}_{i_{0}}^{\infty}(\alpha,\omega)-V_{0}(\alpha)$
(4.78)

with

$\|r^{0}\|_{q}^{\gamma,\mathcal{O}}\lesssim\varepsilon,\qquad\|\Delta_{12}r^{0}\|_{q}^{\gamma,\mathcal{O}}\lesssim\varepsilon\|\Delta_{12}i\|_{q,\overline{s}_{h}+2}^{\gamma,\mathcal{O}}.$
(4.79)

The operator
$\mathscr{R}_{0}=\Pi_{\mathbb{S}_{0}}^{\perp}\mathscr{R}_{0}\Pi_{\mathbb{S}_{0}}^{\perp}$
is a real and reversible Toeplitz in time integral operator satisfying

$\forall
s\in[s_{0},S],\quad\max_{k\in\\{0,1\\}}\|\partial_{\theta}^{k}\mathscr{R}_{0}\|_{\textnormal{\tiny{O-d}},q,s}^{\gamma,\mathcal{O}}\lesssim\varepsilon\gamma^{-1}\left(1+\|\mathfrak{I}_{0}\|_{q,s+\sigma_{3}}^{\gamma,\mathcal{O}}\right)$
(4.80)

and

$\|\Delta_{12}\mathscr{R}_{0}\|_{\textnormal{\tiny{O-d}},q,\overline{s}_{h}+\mathtt{p}}^{\gamma,\mathcal{O}}\lesssim\varepsilon\gamma^{-1}\|\Delta_{12}i\|_{q,\overline{s}_{h}+\mathtt{p}+\sigma_{3}}^{\gamma,\mathcal{O}}.$
(4.81)

The operator $\mathbf{E}_{n}^{1}$ satisfies the following estimate

$\|\mathbf{E}_{n}^{1}\rho\|_{q,s_{0}}^{\gamma,\mathcal{O}}\lesssim\varepsilon
N_{0}^{\mu_{2}}N_{n+1}^{-\mu_{2}}\|\rho\|_{q,s_{0}+2}^{\gamma,\mathcal{O}}.$
(4.82)

In addition, the operator $\mathscr{L}_{0}$ satisfies

$\forall
s\in[s_{0},S],\quad\|\mathscr{L}_{0}\rho\|_{q,s}^{\gamma,\mathcal{O}}\lesssim\|\rho\|_{q,s+1}^{\gamma,\mathcal{O}}+\varepsilon\gamma^{-1}\|\mathfrak{I}_{0}\|_{q,s+\sigma_{3}}^{\gamma,\mathcal{O}}\|\rho\|_{q,s_{0}}^{\gamma,\mathcal{O}}.$
(4.83)

###### Proof.

The identities (4.53) and
$\textnormal{Id}=\Pi_{\mathbb{S}_{0}}+\Pi_{\mathbb{S}_{0}}^{\perp}$ imply

$\displaystyle\mathscr{B}_{\perp}^{-1}\mathscr{L}_{\perp}\mathscr{B}_{\perp}$
$\displaystyle=\mathscr{B}_{\perp}^{-1}\Pi_{\mathbb{S}_{0}}^{\perp}(\mathcal{L}_{\varepsilon
r}-\varepsilon\partial_{\theta}\mathcal{R})\mathscr{B}_{\perp}$
$\displaystyle=\mathscr{B}_{\perp}^{-1}\Pi_{\mathbb{S}_{0}}^{\perp}\mathcal{L}_{\varepsilon
r}\mathscr{B}\Pi_{\mathbb{S}_{0}}^{\perp}-\mathscr{B}_{\perp}^{-1}\Pi_{\mathbb{S}_{0}}^{\perp}\mathcal{L}_{\varepsilon
r}\Pi_{\mathbb{S}_{0}}\mathscr{B}\Pi_{\mathbb{S}_{0}}^{\perp}-\varepsilon\mathscr{B}_{\perp}^{-1}\Pi_{\mathbb{S}_{0}}^{\perp}\partial_{\theta}\mathcal{R}\mathscr{B}_{\perp}.$

Now using (4.68) and the fact that

$\mathscr{B}_{\perp}^{-1}\Pi_{\mathbb{S}_{0}}^{\perp}=\mathscr{B}_{\perp}^{-1},\qquad[\Pi_{\mathbb{S}_{0}}^{\perp},F]=0=[\Pi_{\mathbb{S}_{0}},F],\quad
F\textnormal{ Fourier multiplier},$

we obtain in restriction to the Cantor set
$\mathscr{O}_{\textnormal{\tiny{transport}}}^{n,\gamma,\tau_{1}}(i_{0})$ the
following decomposition

$\displaystyle\mathscr{B}_{\perp}^{-1}\Pi_{\mathbb{S}_{0}}^{\perp}\mathcal{L}_{\varepsilon
r}\mathscr{B}\Pi_{\mathbb{S}_{0}}^{\perp}$
$\displaystyle=\mathscr{B}_{\perp}^{-1}\Pi_{\mathbb{S}_{0}}^{\perp}\mathscr{B}\mathfrak{L}_{\varepsilon
r}\Pi_{\mathbb{S}_{0}}^{\perp}$
$\displaystyle=\big{(}\omega\cdot\partial_{\varphi}+\mathtt{m}_{i_{0}}^{\infty}\partial_{\theta}+\partial_{\theta}\mathcal{K}_{\alpha}\ast\cdot\big{)}\Pi_{\mathbb{S}_{0}}^{\perp}+\Pi_{\mathbb{S}_{0}}^{\perp}\partial_{\theta}\mathbf{R}_{\varepsilon
r}\Pi_{\mathbb{S}_{0}}^{\perp}+\mathscr{B}_{\perp}^{-1}\mathscr{B}\Pi_{\mathbb{S}_{0}}\partial_{\theta}\mathbf{R}_{\varepsilon
r}\Pi_{\mathbb{S}_{0}}^{\perp}$
$\displaystyle\quad+\mathscr{B}_{\perp}^{-1}\Pi_{\mathbb{S}_{0}}^{\perp}\mathscr{B}\mathbf{E}_{n}^{0}\Pi_{\mathbb{S}_{0}}^{\perp}.$

Hence, using also (4.73), (4.74), one gets that in the Cantor set
$\mathscr{O}_{\textnormal{\tiny{transport}}}^{n,\gamma,\tau_{1}}(i_{0})$, the
following identity holds

$\displaystyle\mathscr{B}_{\perp}^{-1}\widehat{\mathcal{L}}_{\omega}\mathscr{B}_{\perp}$
$\displaystyle=\big{(}\omega\cdot\partial_{\varphi}+\mathtt{m}_{i_{0}}^{\infty}\partial_{\theta}+\partial_{\theta}\mathcal{K}_{\alpha}\ast\cdot\big{)}\Pi_{\mathbb{S}_{0}}^{\perp}+\Pi_{\mathbb{S}_{0}}^{\perp}\partial_{\theta}\mathbf{R}_{\varepsilon
r}\Pi_{\mathbb{S}_{0}}^{\perp}+\mathscr{B}_{\perp}^{-1}\mathscr{B}\Pi_{\mathbb{S}_{0}}\partial_{\theta}\mathbf{R}_{\varepsilon
r}\Pi_{\mathbb{S}_{0}}^{\perp}$
$\displaystyle\quad-\mathscr{B}_{\perp}^{-1}\Pi_{\mathbb{S}_{0}}^{\perp}\left(\partial_{\theta}\left(V_{\varepsilon
r}\cdot\right)+\partial_{\theta}\mathbf{L}_{\varepsilon
r,1}^{\textnormal{\tiny{E}}}+\partial_{\theta}\mathbf{L}_{\varepsilon
r,1}^{\textnormal{\tiny{SW}}}\right)\Pi_{\mathbb{S}_{0}}\mathscr{B}\Pi_{\mathbb{S}_{0}}^{\perp}-\varepsilon\mathscr{B}_{\perp}^{-1}\partial_{\theta}\mathcal{R}\mathscr{B}_{\perp}+\mathscr{B}_{\perp}^{-1}\Pi_{\mathbb{S}_{0}}^{\perp}\mathscr{B}\mathbf{E}_{n}^{0}\Pi_{\mathbb{S}_{0}}^{\perp}$
$\displaystyle\triangleq\omega\cdot\partial_{\varphi}\Pi_{\mathbb{S}_{0}}^{\perp}+\mathscr{D}_{0}+\mathscr{R}_{0}+\mathbf{E}_{n}^{1},$

with

$\mathscr{D}_{0}\triangleq\big{(}\mathtt{m}_{i_{0}}^{\infty}\partial_{\theta}+\partial_{\theta}\mathcal{K}_{\alpha}\ast\cdot\big{)}\Pi_{\mathbb{S}_{0}}^{\perp},\qquad\mathbf{E}_{n}^{1}\triangleq\mathscr{B}_{\perp}^{-1}\Pi_{\mathbb{S}_{0}}^{\perp}\mathscr{B}\mathbf{E}_{n}^{0}\Pi_{\mathbb{S}_{0}}^{\perp}.$
(4.84)

The estimate (4.82) is obtained gathering (4.84), (4.65), (4.76), (4.69) and
Lemma 4.1-(iii). The expression (4.78) follows from the Fourier representation
(4.28). The estimates (4.79) correspond to (4.63)-(4.71). The estimates (4.80)
and (4.81) are obtained by the same method as Lemma [51, Prop 6.3 and Lem.
6.3] by using a nice duality representations of $\mathscr{B}_{\perp}^{\pm 1}$
together with (4.58), (4.59), (4.70), (4.71) and (4.72) In particular, the
reversibility property of $\mathscr{R}_{0}$ is a consequence of (4.23),
(4.57), (4.64), (4.75) and the reversibility property of
$\mathbf{R}_{\varepsilon r}$. To prove (4.83), we use Lemma 4.2-(ii) together
with (4.63), (4.80) and

$\forall\,0<\lambda_{0}<\lambda_{1},\quad\sup_{j\in\mathbb{Z}}\Big{(}|j|\max_{k\in\llbracket
0,q\rrbracket}\big{\|}(I_{j}K_{j})^{(k)}\big{\|}_{L^{\infty}([\lambda_{0},\lambda_{1}])}\Big{)}\leqslant
C(\lambda_{0},\lambda_{1}).$

The last estimate has been proved in [51, Lem. 5.1]. ∎

The next step is to get rid of the remainder term $\mathscr{R}_{0}.$ The
properties (4.80), (4.81), (4.83) and Lemma 4.7-(v) allows to start a KAM
iterative procedure similar to [51, Prop. 6.5]. We also refer to [42, Prop.
6.4] for a brief explaination of this method. The result reads as follows.

######  Proposition 4.5.

Let
$(d,s_{0},S,\gamma,q,q_{0},\tau_{1},s_{l},\overline{s}_{l},\overline{s}_{h},\overline{\mu}_{2},\sigma_{3})$
as in (4.2), (4.3), (4.4), (4.49), (4.60) and Proposition 4.4. Set

$\tau_{2}\triangleq\tau_{1}+dq_{0}+1.$ (4.85)

For every choice of additional parameters $(\mu_{2},s_{h})$ enjoying the
following constraints compatibles with (4.61) for the choice
$\mathtt{p}=4\tau_{2}q+4\tau_{2}$,

$\displaystyle\mu_{2}\geqslant\overline{\mu}_{2}+2\tau_{2}q+2\tau_{2},\qquad
s_{h}\geqslant\frac{3}{2}\mu_{2}+\overline{s}_{l}+1,$ (4.86)

there exist $\varepsilon_{0}\in(0,1)$ and
$\sigma_{4}=\sigma_{4}(\tau_{1},\tau_{2},q,d)\geqslant\sigma_{3}$ such that if
the following smallness condition holds

$\varepsilon\gamma^{-2-q}N_{0}^{\mu_{2}}\leqslant\varepsilon_{0},\qquad\|\mathfrak{I}_{0}\|_{q,s_{h}+\sigma_{4}}^{\gamma,\mathcal{O}}\leqslant
1,$ (4.87)

then there exists an operator
$\Phi_{\infty}:\mathcal{O}\to\mathcal{L}\big{(}H^{s}\cap L_{\perp}^{2}\big{)}$
satisfying

$\forall s\in[s_{0},S],\quad\mbox{ }\|\Phi_{\infty}^{\pm
1}\rho\|_{q,s}^{\gamma,\mathcal{O}}\lesssim\|\rho\|_{q,s}^{\gamma,\mathcal{O}}+\varepsilon\gamma^{-2}\|\mathfrak{I}_{0}\|_{q,s+\sigma_{4}}^{\gamma,\mathcal{O}}\|\rho\|_{q,s_{0}}^{\gamma,\mathcal{O}}$
(4.88)

and a diagonal operator
$\mathscr{L}_{\infty}=\mathscr{L}_{\infty}(\alpha,\omega,i_{0})$ in the form

$\mathscr{L}_{\infty}=\omega\cdot\partial_{\varphi}\Pi_{\mathbb{S}_{0}}^{\perp}+\mathscr{D}_{\infty}$
(4.89)

with
$\mathscr{D}_{\infty}=\Pi_{\mathbb{S}_{0}}^{\perp}\mathscr{D}_{\infty}\Pi_{\mathbb{S}_{0}}^{\perp}$
a reversible Fourier multiplier operator given by,

$\mathscr{D}_{\infty}\mathbf{e}_{l,j}={\rm
i}\,\mathrm{d}_{j}^{\infty}\,\mathbf{e}_{l,j},\qquad\mathrm{d}_{j}^{\infty}(\alpha,\omega,i_{0})\triangleq\mathrm{d}_{j}^{0}(\alpha,\omega,i_{0})+r_{j}^{\infty}(\alpha,\omega,i_{0}),\qquad\sup_{j\in\mathbb{S}_{0}^{c}}|j|\|r_{j}^{\infty}\|_{q}^{\gamma,\mathcal{O}}\lesssim\varepsilon\gamma^{-1}$
(4.90)

such that in restriction to the Cantor-like set

$\displaystyle\mathscr{O}_{\textnormal{\tiny{remainder}}}^{n,\gamma,\tau_{1},\tau_{2}}(i_{0})\triangleq\bigcap_{\underset{|l|\leqslant
N_{n}}{(l,j,j_{0})\in\mathbb{Z}^{d}\times(\mathbb{S}_{0}^{c})^{2}}\atop(l,j)\neq(0,j_{0})}\Big{\\{}$
$\displaystyle(\alpha,\omega)\in\mathscr{O}_{\textnormal{\tiny{transport}}}^{n,\gamma,\tau_{1}}(i_{0}),\big{|}\omega\cdot
l+\mathrm{d}_{j}^{\infty}(\alpha,\omega,i_{0})-\mathrm{d}_{j_{0}}^{\infty}(\alpha,\omega,i_{0})\big{|}>\tfrac{2\gamma\langle
j-j_{0}\rangle}{\langle l\rangle^{\tau_{2}}}\Big{\\}}$ (4.91)

we have

$\Phi_{\infty}^{-1}\mathscr{L}_{0}\Phi_{\infty}=\mathscr{L}_{\infty}+\mathbf{E}^{2}_{n},\qquad\|\mathbf{E}^{2}_{n}\rho\|_{q,s_{0}}^{\gamma,\mathcal{O}}\lesssim\varepsilon\gamma^{-2}N_{0}^{{\mu}_{2}}N_{n+1}^{-\mu_{2}}\|\rho\|_{q,s_{0}+1}^{\gamma,\mathcal{O}}.$
(4.92)

Recall that the Cantor set
$\mathscr{O}_{\textnormal{\tiny{transport}}}^{n,\gamma,\tau_{1}}(i_{0})$, the
operator $\mathscr{L}_{0}$ and the frequencies
$\big{(}\mathrm{d}_{j}^{0}(\alpha,\omega,i_{0})\big{)}_{j\in\mathbb{S}_{0}^{c}}$
are respectively given by (4.67), (4.77) and (4.78). Moreover, for two tori
$i_{1}$ and $i_{2}$ both satisfying (4.87), we have

$\forall
j\in\mathbb{S}_{0}^{c},\quad\|\Delta_{12}r_{j}^{\infty}\|_{q}^{\gamma,\mathcal{O}}\lesssim\varepsilon\gamma^{-1}\|\Delta_{12}i\|_{q,\overline{s}_{h}+\sigma_{4}}^{\gamma,\mathcal{O}},\qquad\|\Delta_{12}\mathrm{d}_{j}^{\infty}\|_{q}^{\gamma,\mathcal{O}}\lesssim\varepsilon\gamma^{-1}|j|\|\Delta_{12}i\|_{q,\overline{s}_{h}+\sigma_{4}}^{\gamma,\mathcal{O}}.$
(4.93)

Now, that we have completely diagonalized the operator $\mathscr{L}_{\perp}$
in (4.53) up to error terms, we can find an almost approximate right inverse
for it. This is done by almost inverting the operator $\mathscr{L}_{\infty}$
in (4.89). The result is stated below and its proof follows word by word [51,
Prop. 6.6] or [42, Prop. 6.5]. Consequently, we omit the proof and only
mention that the estimates mainly follows from
(4.76)-(4.88)-(4.69)-(4.82)-(4.92) and Lemma 4.1-(ii).

######  Proposition 4.6.

Let $(d,s_{0},S,\gamma,q,\tau_{1},\tau_{2},s_{h},\mu_{2},\sigma_{4})$ as in
(4.2), (4.3), (4.4), (4.49), (4.60), (4.85), (4.86) and Proposition 4.5. There
exists $\sigma\triangleq\sigma(\tau_{1},\tau_{2},q,d)\geqslant\sigma_{4}$ such
that if the following smallness condition holds

$\varepsilon\gamma^{-2-q}N_{0}^{\mu_{2}}\leqslant\varepsilon_{0},\qquad\|\mathfrak{I}_{0}\|_{q,s_{h}+\sigma}^{\gamma,\mathcal{O}}\leqslant
1,$ (4.94)

  1. (i)

There exists a family of operators
$\big{(}\mathtt{T}_{n}\big{)}_{n\in\mathbb{N}}$ defined on $\mathcal{O}$
satisfying

$\forall
s\in[s_{0},S],\quad\sup_{n\in\mathbb{N}}\|\mathtt{T}_{n}\rho\|_{q,s}^{\gamma,\mathcal{O}}\lesssim\gamma^{-1}\|\rho\|_{q,s+\tau_{1}q+\tau_{1}}^{\gamma,\mathcal{O}}$

and such that for any $n\in\mathbb{N}$, in restriction to the Cantor set

$\mathscr{O}_{\textnormal{\tiny{inversion}}}^{n,\gamma,\tau_{1}}(i_{0})\triangleq\bigcap_{(l,j)\in\mathbb{Z}^{d}\times\mathbb{S}_{0}^{c}\atop|l|\leqslant
N_{n}}\left\\{(\alpha,\omega)\in\mathcal{O}\quad\textnormal{s.t.}\quad\left|\omega\cdot
l+\mathrm{d}_{j}^{\infty}(\alpha,\omega,i_{0})\right|>\tfrac{\gamma\langle
j\rangle}{\langle l\rangle^{\tau_{1}}}\right\\},$ (4.95)

the following identity holds

$\mathscr{L}_{\infty}\mathtt{T}_{n}=\textnormal{Id}+\mathbf{E}^{3}_{n},\qquad\forall
s_{0}\leqslant s\leqslant\overline{s}\leqslant
S,\quad\|\mathbf{E}^{3}_{n}\rho\|_{q,s}^{\gamma,\mathcal{O}}\lesssim
N_{n}^{s-\overline{s}}\gamma^{-1}\|\rho\|_{q,\overline{s}+1+\tau_{1}q+\tau_{1}}^{\gamma,\mathcal{O}}.$

This means that we have an almost approximate right inverse for the operator
$\mathscr{L}_{\infty}$ in (4.89).

  2. (ii)

There exists a family of operators
$\big{(}\mathtt{T}_{\perp,n}\big{)}_{n\in\mathbb{N}}$ defined on $\mathcal{O}$
satisfying

$\forall\,s\in\,[s_{0},S],\quad\sup_{n\in\mathbb{N}}\|\mathtt{T}_{\perp,n}\rho\|_{q,s}^{\gamma,\mathcal{O}}\lesssim\gamma^{-1}\left(\|\rho\|_{q,s+\sigma}^{\gamma,\mathcal{O}}+\|\mathfrak{I}_{0}\|_{q,s+\sigma}^{\gamma,\mathcal{O}}\|\rho\|_{q,s_{0}+\sigma}^{\gamma,\mathcal{O}}\right)$

and such that in restriction to the Cantor set

$\mathscr{G}_{n}(\gamma,\tau_{1},\tau_{2},i_{0})\triangleq\mathscr{O}_{\textnormal{\tiny{transport}}}^{n,\gamma,\tau_{1}}(i_{0})\cap\mathscr{O}_{\textnormal{\tiny{remainder}}}^{n,\gamma,\tau_{1},\tau_{2}}(i_{0})\cap\mathscr{O}_{\textnormal{\tiny{inversion}}}^{n,\gamma,\tau_{1}}(i_{0}),$
(4.96)

the following identity holds

$\mathscr{L}_{\perp}\mathtt{T}_{\perp,n}=\textnormal{Id}+\mathbf{E}_{n},$

with $\mathbf{E}_{n}$ satisfying

$\displaystyle\forall\,s\in[s_{0},S],\quad$
$\displaystyle\|\mathbf{E}_{n}\rho\|_{q,s_{0}}^{\gamma,\mathcal{O}}\lesssim
N_{n}^{s_{0}-s}\gamma^{-1}\Big{(}\|\rho\|_{q,s+\sigma}^{\gamma,\mathcal{O}}+\varepsilon\gamma^{-2}\|\mathfrak{I}_{0}\|_{q,s+\sigma}^{\gamma,\mathcal{O}}\|\rho\|_{q,s_{0}}^{\gamma,\mathcal{O}}\Big{)}$
$\displaystyle\qquad\qquad\quad+\varepsilon\gamma^{-3}N_{0}^{{\mu}_{2}}N_{n+1}^{-\mu_{2}}\|\rho\|_{q,s_{0}+\sigma}^{\gamma,\mathcal{O}}.$

Recall that $\mathscr{L}_{\perp},$
$\mathscr{O}_{\textnormal{\tiny{transport}}}^{n,\gamma,\tau_{1}}(i_{0})$ and
$\mathscr{O}_{\textnormal{\tiny{remainder}}}^{n,\gamma,\tau_{1},\tau_{2}}(i_{0})$
are respectively defined in (4.53), (4.67) and (4.91).

  3. (iii)

When restricted to the Cantor set
$\mathscr{G}_{n}(\gamma,\tau_{1},\tau_{2},i_{0})$, we have also have the
following splitting required to apply the Berti-Bolle theory

$\mathscr{L}_{\perp}=\mathtt{L}_{\perp,n}+\mathtt{R}_{\perp,n},\qquad\mathtt{L}_{\perp,n}\mathtt{T}_{\perp,n}=\textnormal{Id},\qquad\mathtt{R}_{\perp,n}=\mathbf{E}_{n}\mathtt{L}_{\perp,n},$

with the operators $\mathtt{L}_{\perp,n}$ and $\mathtt{R}_{\perp,n}$ defined
in $\mathcal{O}$ and satisfying

$\displaystyle\forall s\in[s_{0},S],\quad$
$\displaystyle\sup_{n\in\mathbb{N}}\|\mathtt{L}_{\perp,n}\rho\|_{q,s}^{\gamma,\mathcal{O}}\lesssim\|\rho\|_{q,s+1}^{\gamma,\mathcal{O}}+\varepsilon\gamma^{-2}\|\mathfrak{I}_{0}\|_{q,s+\sigma}^{\gamma,\mathcal{O}}\|\rho\|_{q,s_{0}+1}^{\gamma,\mathcal{O}},$
$\displaystyle\forall s\in[s_{0},S],\quad$
$\displaystyle\|\mathtt{R}_{\perp,n}\rho\|_{q,s_{0}}^{\gamma,\mathcal{O}}\lesssim
N_{n}^{s_{0}-s}\gamma^{-1}\left(\|\rho\|_{q,s+\sigma}^{\gamma,\mathcal{O}}+\varepsilon\gamma^{-2}\|\mathfrak{I}_{0}\|_{q,s+\sigma}^{\gamma,\mathcal{O}}\|\rho\|_{q,s_{0}+\sigma}^{\gamma,\mathcal{O}}\right)$
$\displaystyle\qquad\qquad\quad+\varepsilon\gamma^{-3}N_{0}^{\mu_{2}}N_{n+1}^{-\mu_{2}}\|\rho\|_{q,s_{0}+\sigma}^{\gamma,\mathcal{O}}.$

Finally, as mentioned at the beginning of this subsection, we can apply the
Berti-Bolle theory [11] and [40, Sec. 6] to construct an approximate right
inverse for the full linearized operator
$d_{i,\kappa}\mathscr{F}(i_{0},\kappa_{0}).$ For a complete proof of the
result, the reader is refered to [40, Thm. 6.1].

######  Theorem 4.1.

(Almost approximate right inverse)
Let $(d,s_{0},S,\gamma,q,\tau_{1},\tau_{2},s_{h},\mu_{2})$ satisfy (4.2),
(4.3), (4.4), (4.49), (4.60), (4.85) and (4.86). Then there exists
$\overline{\sigma}=\overline{\sigma}(\tau_{1},\tau_{2},d,q)>0$ and a family of
reversible operators $\big{(}\mathrm{T}_{0,n}\big{)}_{n\in\mathbb{N}}$ such
that if the smallness condition (4.94) holds, then for all
$g=(g_{1},g_{2},g_{3})$, satisfying

$g_{1}(\varphi)=g_{1}(\varphi),\qquad g_{2}(-\varphi)=-g_{2}(\varphi)\qquad
g_{3}(-\varphi)=(\mathscr{S}g_{3})(\varphi),$

the function $\mathrm{T}_{0,n}g$ satisfies

$\forall
s\in[s_{0},S],\quad\|\mathrm{T}_{0,n}g\|_{q,s}^{\gamma,\mathcal{O}}\lesssim\gamma^{-1}\left(\|g\|_{q,s+\overline{\sigma}}^{\gamma,\mathcal{O}}+\|\mathfrak{I}_{0}\|_{q,s+\overline{\sigma}}^{\gamma,\mathcal{O}}\|g\|_{q,s_{0}+\overline{\sigma}}^{\gamma,\mathcal{O}}\right).$

Moreover $\mathrm{T}_{0,n}$ is an almost-approximate right inverse of
$d_{i,\kappa}\mathscr{F}(i_{0},\kappa_{0})$ in the Cantor set
$\mathscr{G}_{n}(\gamma,\tau_{1},\tau_{2},i_{0})$. More precisely, for all
$(\alpha,\omega)\in\mathscr{G}_{n}(\gamma,\tau_{1},\tau_{2},i_{0})$ we can
write

$d_{i,\kappa}\mathscr{F}(i_{0},\kappa_{0})\circ\mathrm{T}_{0,n}-\textnormal{Id}=\mathscr{E}^{(n)}_{1}+\mathscr{E}^{(n)}_{2}+\mathscr{E}^{(n)}_{3},$

where the operators $\mathscr{E}^{(n)}_{1}$, $\mathscr{E}^{(n)}_{2}$ and
$\mathscr{E}^{(n)}_{3}$ are defined in the whole set $\mathcal{O}$ with the
estimates

$\displaystyle\|\mathscr{E}^{(n)}_{1}g\|_{q,s_{0}}^{\gamma,\mathcal{O}}$
$\displaystyle\lesssim\gamma^{-1}\|\mathscr{F}(i_{0},\kappa_{0})\|_{q,s_{0}+\overline{\sigma}}^{\gamma,\mathcal{O}}\|g\|_{q,s_{0}+\overline{\sigma}}^{\gamma,\mathcal{O}},$
$\displaystyle\forall\,\mathtt{b}\geqslant
0,\quad\|\mathscr{E}^{(n)}_{2}g\|_{q,s_{0}}^{\gamma,\mathcal{O}}$
$\displaystyle\lesssim\gamma^{-1}N_{n}^{-\mathtt{b}}\Big{(}\|g\|_{q,s_{0}+\mathtt{b}+\overline{\sigma}}^{\gamma,\mathcal{O}}+\varepsilon\|\mathfrak{I}_{0}\|_{q,s_{0}+\mathtt{b}+\overline{\sigma}}^{\gamma,\mathcal{O}}\big{\|}g\|_{q,s_{0}+\overline{\sigma}}^{\gamma,\mathcal{O}}\Big{)}\,,$
$\displaystyle\forall\,\mathtt{b}\in[0,S-s_{0}],\quad\|\mathscr{E}^{(n)}_{3}g\|_{q,s_{0}}^{\gamma,\mathcal{O}}$
$\displaystyle\lesssim
N_{n}^{-\mathtt{b}}\gamma^{-2}\Big{(}\|g\|_{q,s_{0}+\mathtt{b}+\overline{\sigma}}^{\gamma,\mathcal{O}}+{\varepsilon\gamma^{-2}}\|\mathfrak{I}_{0}\|_{q,s_{0}+\mathtt{b}+\overline{\sigma}}^{\gamma,\mathcal{O}}\|g\|_{q,s_{0}+\overline{\sigma}}^{\gamma,\mathcal{O}}\Big{)}$
$\displaystyle\quad+\varepsilon\gamma^{-4}N_{0}^{{\mu}_{2}}{N_{n}^{-\mu_{2}}}\|g\|_{q,s_{0}+\overline{\sigma}}^{\gamma,\mathcal{O}}.$

###  4.6 Construction of a non-trivial quasi-periodic solution

In this final section, we construct a non-trivial zero for the functional
$\mathscr{F}$ defined by (4.48). It is obtained by a Nash-Moser iteration
procedure. At each step, we can find an approximate right inverse of the
linearized operator following the construction explained in the previous
subsection. The proof is technical and already detailed in the previous works
[16, 40, 51]. Here, we may use the version exposed in [51, Prop. 7.1 and Cor.
7.1].

######  Proposition 4.7.

(Nash-Moser)

  1. (i)

Let $(d,s_{0},S,q,\tau_{1},\tau_{2},\overline{\sigma})$ as in (4.2), (4.3),
(4.49), (4.60), (4.85) and Theorem 4.1. Consider the parameters fixed by

$\begin{array}[]{c}\mu_{1}\triangleq
3q(\tau_{2}+2)+6\overline{\sigma}+6,\qquad a_{1}\triangleq
6q(\tau_{2}+2)+12\overline{\sigma}+15,\\\ \mu_{2}\triangleq
2q(\tau_{2}+2)+5\overline{\sigma}+7,\qquad a_{2}\triangleq
3q(\tau_{2}+2)+6\overline{\sigma}+9,\\\ s_{h}\triangleq
s_{0}+4q(\tau_{2}+2)+9\overline{\sigma}+11,\qquad b_{1}\triangleq
2s_{h}-s_{0},\qquad\overline{a}\triangleq\tau_{2}+2\end{array}$ (4.97)

and

$0<a<\tfrac{1}{\mu_{2}+q+2},\qquad\gamma\triangleq\varepsilon^{a},\qquad
N_{0}\triangleq\gamma^{-1}.$ (4.98)

We consider the finite dimensional subspaces

$\displaystyle\mathtt{E}_{n}\triangleq$
$\displaystyle\Big{\\{}\mathfrak{I}=(\Theta,I,z)\quad\textnormal{s.t.}\quad\Theta=P_{N_{n}}\Theta,\quad
I=P_{N_{n}}I\quad\textnormal{and}\quad z=P_{N_{n}}z\Big{\\}},$

where $P_{N}$ is the projector defined by

$f(\varphi,\theta)=\sum_{(l,j)\in\mathbb{Z}^{d}\times\mathbb{Z}}f_{l,j}e^{{\rm
i}(l\cdot\varphi+j\theta)}\quad\Rightarrow\quad
P_{N}f(\varphi,\theta)=\sum_{\langle l,j\rangle\leqslant N}f_{l,j}e^{{\rm
i}(l\cdot\varphi+j\theta)}.$

There exist $C_{\ast}>0$ and $\varepsilon_{0}>0$ such that for any
$\varepsilon\in[0,\varepsilon_{0}]$ we get for all $n\in\mathbb{N}$ the
following properties,

    1. (a)

There exists a $q$-times differentiable function

$\mathtt{W}_{n}:\begin{array}[t]{rcl}\mathcal{O}&\rightarrow&\mathtt{E}_{n-1}\times\mathbb{R}^{d}\times\mathbb{R}^{d+1}\\\
(\alpha,\omega)&\mapsto&\big{(}\mathfrak{I}_{n}(\alpha,\omega),\kappa_{n}(\alpha,\omega)-\omega,0\big{)}\end{array}$

satisfying $\mathtt{W}_{0}=0$ and if $n\in\mathbb{N}^{*},$

$\|\mathtt{W}_{n}\|_{q,s_{0}+\overline{\sigma}}^{\gamma,\mathcal{O}}\leqslant
C_{\ast}\varepsilon\gamma^{-1}N_{0}^{q\overline{a}},\qquad\|\mathtt{W}_{n}\|_{q,b_{1}+\overline{\sigma}}^{\gamma,\mathcal{O}}\leqslant
C_{\ast}\varepsilon\gamma^{-1}N_{n-1}^{\mu_{1}}.$

We set

$\mathtt{U}_{0}\triangleq\Big{(}(\varphi,0,0),\omega,(\alpha,\omega)\Big{)}\quad\textnormal{and}\quad\hbox{if}\quad
n\in\mathbb{N}^{*},\quad\mathtt{U}_{n}\triangleq\mathtt{U}_{0}+\mathtt{W}_{n},\qquad\mathtt{H}_{n}\triangleq\mathtt{U}_{n}-\mathtt{U}_{n-1}.$

Then

$\forall
s\in[s_{0},S],\,\|\mathtt{H}_{1}\|_{q,s}^{\gamma,\mathcal{O}}\leqslant\frac{1}{2}C_{\ast}\varepsilon\gamma^{-1}N_{0}^{q\overline{a}},\qquad\forall\,2\leqslant
k\leqslant
n,\,\|\mathtt{H}_{k}\|_{q,s_{0}+\overline{\sigma}}^{\gamma,\mathcal{O}}\leqslant
C_{\ast}\varepsilon\gamma^{-1}N_{k-1}^{-a_{2}}.$

We also have for $n\geqslant 2,$

$\|\mathtt{H}_{n}\|_{q,\overline{s}_{h}+\sigma_{4}}^{\gamma,\mathcal{O}}\leqslant
C_{\ast}\varepsilon\gamma^{-1}N_{n-1}^{-a_{2}}.$ (4.99)

    2. (b)

Define

$i_{n}\triangleq(\varphi,0,0)+\mathfrak{I}_{n},\qquad\gamma_{n}\triangleq\gamma(1+2^{-n})\in[\gamma,2\gamma].$
(4.100)

The torus $i_{n}$ is reversible. Define also

$\mathscr{A}_{0}^{\gamma}\triangleq\mathcal{O},\qquad\mathscr{A}_{n+1}^{\gamma}\triangleq\mathscr{A}_{n}^{\gamma}\cap\mathscr{G}_{n}(\gamma_{n+1},\tau_{1},\tau_{2},i_{n}),$
(4.101)

with $\mathscr{G}_{n}(\gamma_{n+1},\tau_{1},\tau_{2},i_{n})$ as in (4.96).
Consider the open sets

$\mathrm{O}_{n}^{\gamma}\triangleq\Big{\\{}(\alpha,\omega)\in\mathcal{O}\quad\textnormal{s.t.}\quad{\mathtt{dist}}\big{(}(\alpha,\omega),\mathscr{A}_{n}^{\gamma}\big{)}<\gamma
N_{n}^{-\overline{a}}\Big{\\}},\qquad\mathtt{dist}(x,A)\triangleq\inf_{y\in
A}\|x-y\|.$

Then we have

$\|\mathscr{F}(\mathtt{U}_{n})\|_{q,s_{0}}^{\gamma,\mathrm{O}_{n}^{\gamma}}\leqslant
C_{\ast}\varepsilon N_{n-1}^{-a_{1}}.$

  2. (ii)

There exists $\varepsilon_{0}>0$ such that, for any
$\varepsilon\in(0,\varepsilon_{0}),$ the following hold true. We consider the
Cantor set $\mathscr{G}_{\infty}^{\gamma}$ (related to $\varepsilon$ through
$\gamma$) and given by

$\mathscr{G}_{\infty}^{\gamma}\triangleq\bigcap_{n\in\mathbb{N}}\mathscr{A}_{n}^{\gamma}.$
(4.102)

There exists a $q$-times differentiable function

$\mathtt{U}_{\infty}:\begin{array}[t]{rcl}\mathcal{O}&\rightarrow&\left(\mathbb{T}^{d}\times\mathbb{R}^{d}\times
H^{s_{0}}\cap
L_{\perp}^{2}\right)\times\mathbb{R}^{d}\times\mathbb{R}^{d+1}\\\
(\alpha,\omega)&\mapsto&\big{(}i_{\infty}(\alpha,\omega),\kappa_{\infty}(\alpha,\omega),(\alpha,\omega)\big{)}\end{array}$

such that

$\forall(\alpha,\omega)\in\mathscr{G}_{\infty}^{\gamma},\quad\mathscr{F}(\mathtt{U}_{\infty}(\alpha,\omega))=0.$

The torus $i_{\infty}$ is reversible and $\kappa_{\infty}\in
W^{q,\infty,\gamma}(\mathcal{O},\mathbb{R}^{d}).$ Furthermore, there exists a
$q$-times differentiable function
$\alpha\in(\alpha_{0},\alpha_{1})\mapsto\omega(\alpha,\varepsilon)$ such that

$\kappa_{\infty}\big{(}\alpha,\omega(\alpha,\varepsilon)\big{)}=-\omega_{\textnormal{Eq}}(\alpha),\qquad\omega(\alpha,\varepsilon)=-\omega_{\textnormal{Eq}}(\alpha)+\bar{r}_{\varepsilon}(\alpha),\qquad\|\bar{r}_{\varepsilon}\|_{q}^{\gamma,\mathcal{O}}\lesssim\varepsilon\gamma^{-1}N_{0}^{q\overline{a}},$
(4.103)

and

$\forall\alpha\in\mathscr{C}_{\infty}^{\varepsilon},\quad\mathscr{F}\big{(}\mathtt{U}_{\infty}(\alpha,\omega(\alpha,\varepsilon))\big{)}=0,$

where the Cantor set $\mathscr{C}_{\infty}^{\varepsilon}$ is defined by

$\mathscr{C}_{\infty}^{\varepsilon}\triangleq\Big{\\{}\alpha\in(\alpha_{0},\alpha_{1})\quad\textnormal{s.t.}\quad\big{(}\alpha,\omega(\alpha,\varepsilon)\big{)}\in\mathscr{G}_{\infty}^{\gamma}\Big{\\}}.$
(4.104)

Now to conclude the proof of Theorem 1.2, it remains to prove that the Cantor
set $\mathscr{C}_{\infty}^{\varepsilon}$ in (4.104) is not empty. Actually, we
can prove a lower bound measure for $\mathscr{C}_{\infty}^{\varepsilon}$ which
shows that when the magnitude of the perturbation tends to zero, then the set
of admissible parameters tends to be of full Lebesgue measure in
$(\alpha_{0},\alpha_{1}).$

######  Proposition 4.8.

Let $(q_{0},\upsilon,a)$ as in Lemma 4.9, (4.60) and (4.98). Then there exists
$C>0$ such that

$\big{|}\mathscr{C}_{\infty}^{\varepsilon}\big{|}\geqslant(\alpha_{1}-\alpha_{0})-C\varepsilon^{\frac{a\upsilon}{q_{0}}},\qquad\textnormal{implying
in turn}\qquad\lim_{\varepsilon\to
0}\big{|}\mathscr{C}_{\infty}^{\varepsilon}\big{|}=\alpha_{1}-\alpha_{0}.$

###### Proof.

In view of (4.104) and (4.102), we can write

$\mathscr{C}_{\infty}^{\varepsilon}=\bigcap_{n\in\mathbb{N}}\mathscr{C}_{n}^{\varepsilon},\qquad\mathscr{C}_{n}^{\varepsilon}\triangleq\Big{\\{}\alpha\in(\alpha_{0},\alpha_{1})\quad\hbox{s.t}\quad\big{(}\alpha,{\omega}(\alpha,\varepsilon)\big{)}\in\mathscr{A}_{n}^{\gamma}\Big{\\}}.$
(4.105)

Hence,

$(\alpha_{0},\alpha_{1})\setminus\mathscr{C}_{\infty}^{\varepsilon}=\big{(}(\alpha_{0},\alpha_{1})\setminus\mathscr{C}_{0}^{\varepsilon}\big{)}\sqcup\bigsqcup_{n=0}^{\infty}\big{(}\mathscr{C}_{n}^{\varepsilon}\setminus\mathscr{C}_{n+1}^{\varepsilon}\big{)}.$
(4.106)

Observe that (4.103), (4.97) and (4.98) imply

$\sup_{\alpha\in(\alpha_{0},\alpha_{1})}\left|\omega(\alpha,\varepsilon)+\omega_{\textnormal{Eq}}(\alpha)\right|\leqslant\|\bar{\mathrm{r}}_{\varepsilon}\|_{q}^{\gamma,\mathcal{O}}\leqslant
C\varepsilon\gamma^{-1}N_{0}^{q\overline{a}}=C\varepsilon^{1-a(1+q\overline{a})},\qquad
0<a<\frac{1}{1+q\overline{a}}.$

Now by construction (4.50), the previous estimate implies for $\varepsilon$
small enough

$\forall\alpha\in(\alpha_{0},\alpha_{1}),\quad\omega(\alpha,\varepsilon)\in
B(0,R_{0}).$

We immediately deduce that

$\mathscr{C}_{0}^{\varepsilon}=(\alpha_{0},\alpha_{1}),\qquad\Big{|}(\alpha_{0},\alpha_{1})\setminus\mathscr{C}_{\infty}^{\varepsilon}\Big{|}\leqslant\sum_{n=0}^{\infty}\Big{|}\mathscr{C}_{n}^{\varepsilon}\setminus\mathscr{C}_{n+1}^{\varepsilon}\Big{|}.$
(4.107)

According to the notations (4.90) and (4.78), we denote the perturbed
frequencies associated with the reduced linearized operator at state $i_{n}$
in the following way

$\displaystyle\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)$
$\displaystyle\triangleq\mathrm{d}_{j}^{\infty}\big{(}\alpha,{\omega}(\alpha,\varepsilon),i_{n}\big{)}=\Omega_{j}^{\textnormal{\tiny{E}}}(\alpha)+jr^{0,n}(\alpha,\varepsilon)+r_{j}^{\infty,n}(\alpha,\varepsilon),$
(4.108)

with

$r^{0,n}(\alpha,\varepsilon)\triangleq\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon)-V_{0}(\alpha),\qquad\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon)\triangleq\mathtt{m}_{i_{n}}^{\infty}(\alpha,{\omega}(\alpha,\varepsilon)),\qquad
r_{j}^{\infty,n}(\alpha,\varepsilon)\triangleq
r_{j}^{\infty}\big{(}\alpha,{\omega}(\alpha,\varepsilon),i_{n}\big{)}.$
(4.109)

Now, by construction, one can write in view of (4.105), (4.101), (4.96),
(4.67), (4.91) and (4.95) for any $n\in\mathbb{N}$,

$\mathscr{C}_{n}^{\varepsilon}\setminus\mathscr{C}_{n+1}^{\varepsilon}=\bigcup_{(l,j)\in\mathbb{Z}^{d}\times\mathbb{Z}\setminus\\{(0,0)\\}\atop|l|\leqslant
N_{n}}\mathscr{R}_{l,j}^{(0)}(i_{n})\bigcup_{(l,j,j_{0})\in\mathbb{Z}^{d}\times(\mathbb{S}_{0}^{c})^{2}\atop|l|\leqslant
N_{n}}\mathscr{R}_{l,j,j_{0}}^{(2)}(i_{n})\bigcup_{(l,j)\in\mathbb{Z}^{d}\times\mathbb{S}_{0}^{c}\atop|l|\leqslant
N_{n}}\mathscr{R}_{l,j}^{(1)}(i_{n}),$ (4.110)

where

$\displaystyle\mathscr{R}_{l,j}^{(0)}(i_{n})$
$\displaystyle\triangleq\left\\{\alpha\in\mathscr{C}_{n}^{\varepsilon}\quad\textnormal{s.t.}\quad\Big{|}{\omega}(\alpha,\varepsilon)\cdot
l+j\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon)\Big{|}\leqslant\tfrac{4\gamma_{n+1}^{\upsilon}\langle
j\rangle}{\langle l\rangle^{\tau_{1}}}\right\\},$
$\displaystyle\mathscr{R}_{l,j,j_{0}}^{(2)}(i_{n})$
$\displaystyle\triangleq\left\\{\alpha\in\mathscr{C}_{n}^{\varepsilon}\quad\textnormal{s.t.}\quad\Big{|}{\omega}(\alpha,\varepsilon)\cdot
l+\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)-\mathrm{d}_{j_{0}}^{\infty,n}(\alpha,\varepsilon)\Big{|}\leqslant\tfrac{2\gamma_{n+1}\langle
j-j_{0}\rangle}{\langle l\rangle^{\tau_{2}}}\right\\},$
$\displaystyle\mathscr{R}_{l,j}^{(1)}(i_{n})$
$\displaystyle\triangleq\left\\{\alpha\in\mathscr{C}_{n}^{\varepsilon}\quad\textnormal{s.t.}\quad\Big{|}{\omega}(\alpha,\varepsilon)\cdot
l+\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)\Big{|}\leqslant\tfrac{\gamma_{n+1}\langle
j\rangle}{\langle l\rangle^{\tau_{1}}}\right\\}.$

Copying the proof of [51, Lem. 7.1] we can show, using in particular
(4.99)-(4.79)-(4.93), that for any $n\in\mathbb{N}\setminus\\{0,1\\}$ and any
$l\in\mathbb{Z}^{d}$ such that $|l|\leqslant N_{n-1},$ the following
properties hold.

  1. •

For $j\in\mathbb{Z}$ with $(l,j)\neq(0,0)$, we get
$\,\,\mathscr{R}_{l,j}^{(0)}(i_{n})=\varnothing.$

  2. •

For $(j,j_{0})\in(\mathbb{S}_{0}^{c})^{2}$ with $(l,j)\neq(0,j_{0}),$ we get
$\,\,\mathscr{R}_{l,j,j_{0}}^{(2)}(i_{n})=\varnothing.$

  3. •

For $j\in\mathbb{S}_{0}^{c}$, we get
$\,\,\mathscr{R}_{l,j}^{(1)}(i_{n})=\varnothing.$

This leads to write for any $n\in\mathbb{N}\setminus\\{0,1\\},$

$\mathscr{C}_{n}^{\varepsilon}\setminus\mathscr{C}_{n+1}^{\varepsilon}=\bigcup_{\underset{N_{n-1}<|l|\leqslant
N_{n}}{(l,j)\in\mathbb{Z}^{d}\times\mathbb{Z}\setminus\\{(0,0)\\}}}\mathscr{R}_{l,j}^{(0)}(i_{n})\cup\bigcup_{\underset{N_{n-1}<|l|\leqslant
N_{n}}{(l,j,j_{0})\in\mathbb{Z}^{d}\times(\mathbb{S}_{0}^{c})^{2}}}\mathscr{R}_{l,j,j_{0}}^{(2)}(i_{n})\cup\bigcup_{\underset{N_{n-1}<|l|\leqslant
N_{n}}{(l,j)\in\mathbb{Z}^{d}\times\mathbb{S}_{0}^{c}}}\mathscr{R}_{l,j}^{(1)}(i_{n}).$
(4.111)

To estimate, we shall make use of Rüssmann’s Lemma 4.6. First notice that

$W^{q,\infty,\gamma}(\mathcal{O},\mathbb{C})\hookrightarrow
C^{q-1}(\mathcal{O},\mathbb{C}),\qquad q=q_{0}+1,$

imply that for any $n\in\mathbb{N}$, the $C^{q_{0}}$ regularity for the curves

$\begin{array}[]{ll}\displaystyle\alpha\mapsto\omega(\alpha,\varepsilon)\cdot
l+j\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon),&\quad(l,j)\in\mathbb{Z}^{d}\times\mathbb{Z}\backslash\\{(0,0)\\},\vspace{0.1cm}\\\
\displaystyle\alpha\mapsto\omega(\alpha,\varepsilon)\cdot
l+\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)-\mathrm{d}_{j_{0}}^{\infty,n}(\alpha,\varepsilon),&\quad(l,j,j_{0})\in\mathbb{Z}^{d}\times(\mathbb{S}_{0}^{c})^{2},\vspace{0.1cm}\\\
\displaystyle\alpha\mapsto\omega(\alpha,\varepsilon)\cdot
l+\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon),&\quad(l,j)\in\mathbb{Z}^{d}\times\mathbb{S}_{0}^{c}.\end{array}$

In addition, the following perturbed Rüssmann conditions hold. They are
obtained by a perturbative argument from the equilibrium transversality
conditions in Lemma 4.9 similarly to [51, Lem. 7.3] : for $q_{0}$, $C_{0}$ and
$\rho_{0}$ as in Lemma 4.9, there exist $\varepsilon_{0}>0$ small enough such
that for any $\varepsilon\in[0,\varepsilon_{0}]$ we have

  1. •

For all $(l,j)\in\mathbb{Z}^{d+1}\setminus\\{(0,0)\\}$ such that $|j|\leqslant
C_{0}\langle l\rangle,$ we have

$\forall
n\in\mathbb{N},\quad\inf_{\alpha\in[\alpha_{0},\alpha_{1}]}\max_{k\in\llbracket
0,q_{0}\rrbracket}|\partial_{\alpha}^{k}\big{(}{\omega}(\alpha,\varepsilon)\cdot
l+j\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon)\big{)}|\geqslant\tfrac{\rho_{0}\langle
l\rangle}{2}.$

  2. •

For all $(l,j)\in\mathbb{Z}^{d}\times\mathbb{S}_{0}^{c}$ such that
$|j|\leqslant C_{0}\langle l\rangle,$ we have

$\forall
n\in\mathbb{N},\quad\inf_{\alpha\in[\alpha_{0},\alpha_{1}]}\max_{k\in\llbracket
0,q_{0}\rrbracket}\big{|}\partial_{\alpha}^{k}\big{(}{\omega}(\alpha,\varepsilon)\cdot
l+\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)\big{)}\big{|}\geqslant\tfrac{\rho_{0}\langle
l\rangle}{2}.$

  3. •

For all $(l,j,j_{0})\in\mathbb{Z}^{d}\times(\mathbb{S}_{0}^{c})^{2}$ such that
$|j-j_{0}|\leqslant C_{0}\langle l\rangle,$ we have

$\forall
n\in\mathbb{N},\quad\inf_{\alpha\in[\alpha_{0},\alpha_{1}]}\max_{k\in\llbracket
0,q_{0}\rrbracket}\big{|}\partial_{\alpha}^{k}\big{(}{\omega}(\alpha,\varepsilon)\cdot
l+\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)-\mathrm{d}_{j_{0}}^{\infty,n}(\alpha,\varepsilon)\big{)}\big{|}\geqslant\tfrac{\rho_{0}\langle
l\rangle}{2}.$

Notice that the proof of the previous transversality conditions requires in
particular the estimates (4.6) and (4.6). Thus, applying Lemma 4.6, we get for
all $n\in\mathbb{N}$,

$\displaystyle\Big{|}\mathscr{R}_{l,j}^{(0)}(i_{n})\Big{|}$
$\displaystyle\lesssim\gamma^{\frac{\upsilon}{q_{0}}}\langle
j\rangle^{\frac{1}{q_{0}}}\langle l\rangle^{-1-\frac{\tau_{1}+1}{q_{0}}},$
$\displaystyle\Big{|}\mathscr{R}_{l,j}^{(1)}(i_{n})\Big{|}$
$\displaystyle\lesssim\gamma^{\frac{1}{q_{0}}}\langle
j\rangle^{\frac{1}{q_{0}}}\langle l\rangle^{-1-\frac{\tau_{1}+1}{q_{0}}},$
(4.112) $\displaystyle\Big{|}\mathscr{R}_{l,j,j_{0}}^{(2)}(i_{n})\Big{|}$
$\displaystyle\lesssim\gamma^{\frac{1}{q_{0}}}\langle
j-j_{0}\rangle^{\frac{1}{q_{0}}}\langle
l\rangle^{-1-\frac{\tau_{2}+1}{q_{0}}}.$

Since (4.111) is valid for $n\in\mathbb{N}\setminus\\{0,1\\}$, we first need
to estimate the first two terms in the right hand-side of (4.107). Fix
$k\in\\{0,1\\},$ then we have by Lemma 4.12

$\displaystyle\Big{|}\mathscr{C}_{k}\setminus\mathscr{C}_{k+1}\Big{|}\lesssim\sum_{\underset{|j|\leqslant
C_{0}\langle l\rangle,|l|\leqslant
N_{k}}{(l,j)\in\mathbb{Z}^{d}\times\mathbb{Z}\setminus\\{(0,0)\\}}}\Big{|}\mathscr{R}_{l,j}^{(0)}(i_{k})\Big{|}+\sum_{\underset{\underset{\min(|j|,|j_{0}|)\leqslant
c_{2}\gamma_{k+1}^{-\upsilon}\langle l\rangle^{\tau_{1}}}{|j-j_{0}|\leqslant
C_{0}\langle l\rangle,|l|\leqslant
N_{k}}}{(l,j,j_{0})\in\mathbb{Z}^{d}\times(\mathbb{S}_{0}^{c})^{2}}}\Big{|}\mathscr{R}_{l,j,j_{0}}^{(2)}(i_{k})\Big{|}+\sum_{\underset{|j|\leqslant
C_{0}\langle l\rangle,|l|\leqslant
N_{k}}{(l,j)\in\mathbb{Z}^{d}\times\mathbb{S}_{0}^{c}}}\Big{|}\mathscr{R}_{l,j}^{(1)}(i_{k})\Big{|}.$
(4.113)

Observe that for $|j-j_{0}|\leqslant C_{0}\langle l\rangle$ and
$\min(|j|,|j_{0}|)\leqslant c_{2}\gamma_{k+1}^{-\upsilon}\langle
l\rangle^{\tau_{1}}$, then

$\max(|j|,|j_{0}|)=\min(|j|,|j_{0}|)+|j-j_{0}|\leqslant
c_{2}\gamma_{k+1}^{-\upsilon}\langle l\rangle^{\tau_{1}}+C_{0}\langle
l\rangle\lesssim\gamma^{-\upsilon}\langle l\rangle^{\tau_{1}}.$ (4.114)

Combining (4.6), (4.113) and (4.114), we get for $k\in\\{0,1\\}$,

$\displaystyle\Big{|}\mathscr{C}_{k}\setminus\mathscr{C}_{k+1}\Big{|}$
$\displaystyle\lesssim\gamma^{\frac{1}{q_{0}}}\sum_{(l,j)\in\mathbb{Z}^{d}\times(\mathbb{S}_{0})^{c}\atop|j|\leqslant
C_{0}\langle l\rangle}\langle j\rangle^{\frac{1}{q_{0}}}\langle
l\rangle^{-1-\frac{\tau_{1}+1}{q_{0}}}+\gamma^{\frac{1}{q_{0}}}\sum_{\underset{\underset{\min(|j|,|j_{0}|)\leqslant
c_{2}\gamma_{k+1}^{-\upsilon}\langle l\rangle^{\tau_{1}}}{|j-j_{0}|\leqslant
C_{0}\langle l\rangle,|l|\leqslant
N_{k}}}{(l,j,j_{0})\in\mathbb{Z}^{d}\times(\mathbb{S}_{0}^{c})^{2}}}\langle
j-j_{0}\rangle^{\frac{1}{q_{0}}}\langle
l\rangle^{-1-\frac{\tau_{2}+1}{q_{0}}}$
$\displaystyle\quad+\gamma^{\frac{\upsilon}{q_{0}}}\sum_{(l,j)\in\mathbb{Z}^{d}\times\mathbb{Z}\setminus\\{(0,0)\\}\atop|j|\leqslant
C_{0}\langle l\rangle}\langle j\rangle^{\frac{1}{q_{0}}}\langle
l\rangle^{-1-\frac{\tau_{1}+1}{q_{0}}}.$

Thus, by using (4.60) and (4.85), we deduce

$\displaystyle\max_{k\in\\{0,1\\}}\Big{|}\mathscr{C}_{k}\setminus\mathscr{C}_{k+1}\Big{|}$
$\displaystyle\lesssim\gamma^{\frac{1}{q_{0}}}\Bigg{(}\sum_{l\in\mathbb{Z}^{d}}\langle
l\rangle^{-\frac{\tau_{1}}{q_{0}}}+\gamma^{-\upsilon}\sum_{l\in\mathbb{Z}^{d}}\langle
l\rangle^{\tau_{1}-1-\frac{\tau_{2}}{q_{0}}}\Bigg{)}+\gamma^{\frac{\upsilon}{q_{0}}}\sum_{l\in\mathbb{Z}^{d}}\langle
l\rangle^{-\frac{\tau_{1}}{q_{0}}}$ (4.115)
$\displaystyle\lesssim\gamma^{\min\left(\frac{\upsilon}{q_{0}},\frac{1}{q_{0}}-\upsilon\right)}=\gamma^{\frac{\upsilon}{q_{0}}}.$

Now fix some $n\geqslant 2.$ Using (4.111) and Lemma 4.12, we get

$\displaystyle\Big{|}\mathscr{C}_{n}\setminus\mathscr{C}_{n+1}\Big{|}\leqslant\sum_{\underset{|j|\leqslant
C_{0}\langle l\rangle,N_{n-1}<|l|\leqslant
N_{n}}{(l,j)\in\mathbb{Z}^{d}\times\mathbb{Z}\setminus\\{(0,0)\\}}}\Big{|}\mathscr{R}_{l,j}^{(0)}(i_{n})\Big{|}+\sum_{\underset{\underset{\min(|j|,|j_{0}|)\leqslant
c_{2}\gamma_{n+1}^{-\upsilon}\langle l\rangle^{\tau_{1}}}{|j-j_{0}|\leqslant
C_{0}\langle l\rangle,N_{n-1}<|l|\leqslant
N_{n}}}{(l,j,j_{0})\in\mathbb{Z}^{d}\times(\mathbb{S}_{0}^{c})^{2}}}\Big{|}\mathscr{R}_{l,j,j_{0}}^{(2)}(i_{n})\Big{|}+\sum_{\underset{|j|\leqslant
C_{0}\langle l\rangle,N_{n-1}<|l|\leqslant
N_{n}}{(l,j)\in\mathbb{Z}^{d}\times\mathbb{S}_{0}^{c}}}\Big{|}\mathscr{R}_{l,j}^{(1)}(i_{n})\Big{|}.$

Therefore, we obtain from (4.6) and (4.114),

$\displaystyle\Big{|}\mathscr{C}_{n}\setminus\mathscr{C}_{n+1}\Big{|}\lesssim\gamma^{\frac{1}{q_{0}}}\Bigg{(}\sum_{l\in\mathbb{Z}^{d}\atop|l|>N_{n-1}}\langle
l\rangle^{-\frac{\tau_{1}}{q_{0}}}+\gamma^{-\upsilon}\sum_{l\in\mathbb{Z}^{d}\atop|l|>N_{n-1}}\langle
l\rangle^{\tau_{1}-1-\frac{\tau_{2}}{q_{0}}}\Bigg{)}+\gamma^{\frac{\upsilon}{q_{0}}}\sum_{l\in\mathbb{Z}^{d}\atop|l|>N_{n-1}}\langle
l\rangle^{-\frac{\tau_{1}}{q_{0}}}.$

Hence, we infer

$\displaystyle\sum_{n=2}^{\infty}\Big{|}\mathscr{C}_{n}\setminus\mathscr{C}_{n+1}\Big{|}$
$\displaystyle\lesssim\gamma^{\frac{\upsilon}{q_{0}}}.$ (4.116)

Plugging (4.116) and (4.115) into (4.107) and using (4.98) yields

$\Big{|}(\alpha_{0},\alpha_{1})\setminus\mathscr{C}_{\infty}^{\varepsilon}\Big{|}\lesssim\gamma^{\frac{\upsilon}{q_{0}}}=\varepsilon^{\frac{a\upsilon}{q_{0}}}.$

This achieves the proof of Proposition 4.8. ∎

To conclude, it remains to prove the following lemma providing necessary
constraints between time and space Fourier modes so that the sets in (4.110)
are not void.

######  Lemma 4.12.

There exists $\varepsilon_{0}$ such that for any
$\varepsilon\in[0,\varepsilon_{0}]$ and $n\in\mathbb{N}$ the following
assertions hold true.

  1. (i)

Let $(l,j)\in\mathbb{Z}^{d}\times\mathbb{Z}\setminus\\{(0,0)\\}.$ If
$\,\displaystyle\mathscr{R}_{l,j}^{(0)}(i_{n})\neq\varnothing,$ then
$|j|\leqslant C_{0}\langle l\rangle.$

  2. (ii)

Let $(l,j,j_{0})\in\mathbb{Z}^{d}\times(\mathbb{S}_{0}^{c})^{2}.$ If
$\,\displaystyle\mathscr{R}_{l,j,j_{0}}^{(2)}(i_{n})\neq\varnothing,$ then
$|j-j_{0}|\leqslant C_{0}\langle l\rangle.$

  3. (iii)

Let $(l,j)\in\mathbb{Z}^{d}\times\mathbb{S}_{0}^{c}.$ If
$\,\displaystyle\mathscr{R}_{l,j}^{(1)}(i_{n})\neq\varnothing,$ then
$|j|\leqslant C_{0}\langle l\rangle.$

  4. (iv)

Let $(l,j,j_{0})\in\mathbb{Z}^{d}\times(\mathbb{S}_{0}^{c})^{2}.$ There exists
$c_{2}>0$ such that if $\displaystyle\min(|j|,|j_{0}|)\geqslant
c_{2}\gamma_{n+1}^{-\upsilon}\langle l\rangle^{\tau_{1}},$ then

$\mathscr{R}_{l,j,j_{0}}^{(2)}(i_{n})\subset\mathscr{R}_{l,j-j_{0}}^{(0)}(i_{n}).$

###### Proof.

(i) Assume that $\mathscr{R}_{l,j}^{(0)}(i_{n})\neq\varnothing.$ Then, we can
find $\alpha\in(\alpha_{0},\alpha_{1})$ such that

$|\omega(\alpha,\varepsilon)\cdot
l+j\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon)|\leqslant\tfrac{4\gamma_{n+1}^{\upsilon}\langle
j\rangle}{\langle l\rangle^{\tau_{1}}}.$

By using the triangle and Cauchy-Schwarz inequalities together with (4.100),
(4.98) and the fact that
$(\alpha,\varepsilon)\mapsto\omega(\alpha,\varepsilon)$ is bounded, we infer

$\displaystyle|\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon)||j|$
$\displaystyle\leqslant 4|j|\gamma_{n+1}^{\upsilon}\langle
l\rangle^{-\tau_{1}}+|{\omega}(\alpha,\varepsilon)\cdot l|$
$\displaystyle\leqslant 4|j|\gamma_{n+1}^{\upsilon}+C\langle l\rangle$
$\displaystyle\leqslant 8\varepsilon^{a\upsilon}|j|+C\langle l\rangle.$

Now, by (4.109), we can write

$\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon)=V_{0}(\alpha)+r^{0,n}(\alpha,\varepsilon).$

Hence, applying (4.63), (4.98) and Proposition 4.7-(i)-(a) we find

$\displaystyle\forall k\in\llbracket
0,q\rrbracket,\quad\sup_{n\in\mathbb{N}}\sup_{\alpha\in(\alpha_{0},\alpha_{1})}|\partial_{\alpha}^{k}r^{0,n}(\alpha,\varepsilon)|$
$\displaystyle\leqslant\gamma^{-k}\sup_{n\in\mathbb{N}}\|r^{0,n}\|_{q}^{\gamma,\mathcal{O}}$
$\displaystyle\lesssim\varepsilon\gamma^{-k}$
$\displaystyle\lesssim\varepsilon^{1-ak}.$ (4.117)

Thus, for $\varepsilon$ small enough, we obtain by (4.25) and the decay
property of $I_{1}K_{1}$ on $(0,\infty),$

$\inf_{n\in\mathbb{N}}\inf_{\alpha\in(\alpha_{0},\alpha_{1})}|\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon)|\geqslant\tfrac{1}{2}V_{0}\left(\tfrac{1}{\alpha_{0}}\right).$

Hence, a suitable choice of $\varepsilon$ small enough provides the constraint
$|j|\leqslant C_{0}\langle l\rangle$ for some $C_{0}>0.$

(ii) Observe that
$\mathscr{R}_{l,j_{0},j_{0}}^{(2)}(i_{n})=\mathscr{R}^{(0)}_{l,0}(i_{n})$, so
the case $j=j_{0}$ can be included in the previous point. Now we assume $j\neq
j_{0}$ and $\mathscr{R}_{l,j,j_{0}}^{(2)}(i_{n})\neq\varnothing.$ There exists
$\alpha\in(\alpha_{0},\alpha_{1})$ such that

$|\omega(\alpha,\varepsilon)\cdot
l+\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)-\mathrm{d}_{j_{0}}^{\infty,n}(\alpha,\varepsilon)|\leqslant\tfrac{2\gamma_{n+1}|j-j_{0}|}{\langle
l\rangle^{\tau_{2}}}.$

Applying the triangle and Cauchy-Schwarz inequalities, (4.100) and (4.98), we
infer

$\displaystyle|\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)-\mathrm{d}_{j_{0}}^{\infty,n}(\alpha,\varepsilon)|$
$\displaystyle\leqslant 2\gamma_{n+1}|j-j_{0}|\langle
l\rangle^{-\tau_{2}}+|{\omega}(\alpha,\varepsilon)\cdot l|$
$\displaystyle\leqslant 2\gamma_{n+1}|j-j_{0}|+C\langle l\rangle$
$\displaystyle\leqslant 4\varepsilon^{a}|j-j_{0}|+C\langle l\rangle.$

Now, similarly to (4.6), we may obtain from (4.90)-(4.98),

$\displaystyle\forall k\in\llbracket
0,q\rrbracket,\quad\sup_{n\in\mathbb{N}}\sup_{j\in\mathbb{S}_{0}^{c}}\sup_{\alpha\in(\alpha_{0},\alpha_{1})}|j||\partial_{\alpha}^{k}r_{j}^{\infty,n}(\alpha,\varepsilon)|$
$\displaystyle\leqslant\gamma^{-k}\sup_{n\in\mathbb{N}}\sup_{j\in\mathbb{S}_{0}^{c}}|j|\|r_{j}^{\infty,n}\|_{q}^{\gamma,\mathcal{O}}$
$\displaystyle\lesssim\varepsilon\gamma^{-1-k}$
$\displaystyle\lesssim\varepsilon^{1-a(1+k)}.$ (4.118)

One obtains from the triangle inequality, Lemma 4.7-(iv), (4.6) and (4.6), up
to taking $\varepsilon$ sufficiently small

$\displaystyle|\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)-\mathrm{d}_{j_{0}}^{\infty,n}(\alpha,\varepsilon)|$
$\displaystyle\geqslant|\Omega_{j}^{\textnormal{\tiny{E}}}(\alpha)-\Omega_{j_{0}}^{\textnormal{\tiny{E}}}(\alpha)|-|r^{0,n}(\alpha,\varepsilon)||j-j_{0}|-|r_{j}^{\infty,n}(\alpha,\varepsilon)|-|r_{j_{0}}^{\infty,n}(\alpha,\varepsilon)|$
$\displaystyle\geqslant\big{(}\Omega-C\varepsilon^{1-a}\big{)}|j-j_{0}|$
$\displaystyle\geqslant\tfrac{\Omega}{2}|j-j_{0}|.$

The foregoing inequalities together give for $\varepsilon$ small enough, the
constraint $|j-j_{0}|\leqslant C_{0}\langle l\rangle,$ for some $C_{0}>0.$

(iii) As previously, we can forget the case $j=0.$ Assume that $j\neq 0$ and
$\mathscr{R}_{l,j}^{(1)}(i_{n})\neq\varnothing.$ There exists
$\alpha\in(\alpha_{0},\alpha_{1})$ such that

$|\omega(\alpha,\varepsilon)\cdot
l+\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)|\leqslant\tfrac{\gamma_{n+1}|j|}{\langle
l\rangle^{\tau_{1}}}.$

By the same techniques as in the other cases, we find

$\displaystyle|\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)|$
$\displaystyle\leqslant\gamma_{n+1}|j|\langle
l\rangle^{-\tau_{1}}+|{\omega}(\alpha,\varepsilon)\cdot l|$
$\displaystyle\leqslant 2\varepsilon^{a}|j|+C\langle l\rangle.$

Now (4.108), the triangle inequality, Lemma 4.7-(iii), (4.6) and (4.6) imply

$\displaystyle|\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)|$
$\displaystyle\geqslant\Omega|j|-|j||r^{0,n}(\alpha,\varepsilon)|-|r_{j}^{\infty,n}(\alpha,\varepsilon)|$
$\displaystyle\geqslant\Omega|j|-C\varepsilon^{1-a}|j|.$

Gathering the foregoing inequalities gives

$\displaystyle\big{(}\Omega-C\varepsilon^{1-a}-2\varepsilon^{a}\big{)}|j|$
$\displaystyle\leqslant C\langle l\rangle.$

Thus, for $\varepsilon$ small enough we deduce the constraint $|j|\leqslant
C_{0}\langle l\rangle,$ for some $C_{0}>0.$

(iv) We can forget the case $j=j_{0}.$ Assume $j\neq j_{0}.$ The symmetry
property $\mathrm{d}_{-j}^{\infty,n}=-\mathrm{d}_{j}^{\infty,n}$ implies that
without loss of generality we can assume that $0\leqslant j<j_{0}.$ Take
$\alpha\in\mathscr{R}_{l,j,j_{0}}^{(2)}(i_{n}),$ i.e.

$\big{|}{\omega}(\alpha,\varepsilon)\cdot
l+\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)\pm\mathrm{d}_{j_{0}}^{\infty,n}(\alpha,\varepsilon)\big{|}\leqslant\tfrac{2\gamma_{n+1}\langle
j\pm j_{0}\rangle}{\langle l\rangle^{\tau_{2}}}.$

Putting together (4.108), (4.27) and the triangle inequality, we find

$\displaystyle\big{|}{\omega}(\alpha,\varepsilon)\cdot l+(j\pm
j_{0})\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon)\big{|}$
$\displaystyle\leqslant\big{|}{\omega}(\alpha,\varepsilon)\cdot
l+\mathrm{d}_{j}^{\infty,n}(\alpha,\varepsilon)\pm\mathrm{d}_{j_{0}}^{\infty,n}(\alpha,\varepsilon)\big{|}+\tfrac{1}{2}(1\pm
1)$
$\displaystyle\quad+|jI_{j}\left(\tfrac{1}{\alpha}\right)K_{j}\left(\tfrac{1}{\alpha}\right)\pm
j_{0}I_{j_{0}}\left(\tfrac{1}{\alpha}\right)K_{j_{0}}\left(\tfrac{1}{\alpha}\right)|+\big{|}r_{j}^{\infty,n}(\alpha,\varepsilon)\pm
r_{j_{0}}^{\infty,n}(\alpha,\varepsilon)\big{|}.$

Hence, we deduce

$\displaystyle\big{|}{\omega}(\alpha,\varepsilon)\cdot l+(j\pm
j_{0})\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon)\big{|}\leqslant$
$\displaystyle\tfrac{2\gamma_{n+1}\langle j\pm j_{0}\rangle}{\langle
l\rangle^{\tau_{2}}}+|jI_{j}\left(\tfrac{1}{\alpha}\right)K_{j}\left(\tfrac{1}{\alpha}\right)\pm
j_{0}I_{j_{0}}\left(\tfrac{1}{\alpha}\right)K_{j_{0}}\left(\tfrac{1}{\alpha}\right)|$
(4.119) $\displaystyle\quad+\tfrac{1}{2}(1\pm
1)+\big{|}r_{j}^{\infty,n}(\alpha,\varepsilon)-r_{j_{0}}^{\infty,n}(\alpha,\varepsilon)\big{|}.$

It has been proved in [51, Lem. 7.2-(iv)] that

$\displaystyle\forall x>0,\quad|jI_{j}(x)K_{j}(x)\pm
j_{0}I_{j_{0}}(x)K_{j_{0}}(x)|\leqslant\tfrac{\langle j\pm
j_{0}\rangle}{\min(|j|,|j_{0}|)}\cdot$

Additionally, (4.90) gives

$\displaystyle\big{|}r_{j}^{\infty,n}(\alpha,\varepsilon)\pm
r_{j_{0}}^{\infty,n}(\alpha,\varepsilon)\big{|}\leqslant$ $\displaystyle
C\varepsilon^{1-a}\big{(}|j|^{-1}+|j_{0}|^{-1}\big{)}$
$\displaystyle\leqslant$ $\displaystyle C\varepsilon^{1-a}\tfrac{\langle j\pm
j_{0}\rangle}{\min(|j|,|j_{0}|)}\cdot$

Also one has the trivial bound

$\tfrac{1}{2}(1\pm 1)\leqslant\tfrac{\langle j\pm
j_{0}\rangle}{\min(|j|,|j_{0}|)}.$

Inserting the foregoing estimates into (4.119) yields

$\displaystyle\big{|}{\omega}(\alpha,\varepsilon)\cdot l+(j\pm
j_{0})\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon)\big{|}\leqslant$
$\displaystyle\tfrac{2\gamma_{n+1}\langle j\pm j_{0}\rangle}{\langle
l\rangle^{\tau_{2}}}+C\tfrac{\langle j\pm
j_{0}\rangle}{\min(|j|,|j_{0}|)}\cdot$

Thus, for
$\displaystyle\min(|j|,|j_{0}|)\geqslant\tfrac{1}{2}C\gamma_{n+1}^{-\upsilon}\langle
l\rangle^{\tau_{1}},$ then using (4.85), we get

$\big{|}{\omega}(\alpha,\varepsilon)\cdot l+(j\pm
j_{0})\mathtt{m}_{n}^{\infty}(\alpha,\varepsilon)\big{|}\leqslant\tfrac{4\gamma_{n+1}^{\upsilon}\langle
j\pm j_{0}\rangle}{\langle l\rangle^{\tau_{1}}}\cdot$

This proves Lemma 4.12, taking $c_{2}\triangleq\frac{C}{2}.$ ∎

##  Appendix A Properties of modified Bessel functions

We collect here the definitions and useful properties of modified Bessel
functions. The literature is huge as regards these special functions and we
may refer the reader to [1, 70] for a nice introduction. The modified Bessel
functions of first and second kind are defined as follows

$I_{\nu}(z)\triangleq\sum_{m=0}^{+\infty}\frac{\left(\frac{z}{2}\right)^{\nu+2m}}{m!\Gamma(\nu+m+1)},\mbox{
}\quad|\mbox{arg}(z)|<\pi$

and

$\forall\nu\in\mathbb{C}\backslash\mathbb{Z},\quad
K_{\nu}(z)\triangleq\frac{\pi}{2}\frac{I_{-\nu}(z)-I_{\nu}(z)}{\sin(\nu\pi)},\qquad\forall
n\in\mathbb{Z},\quad K_{n}(z)\triangleq\displaystyle\lim_{\nu\rightarrow
n}K_{\nu}(z),\qquad|\mbox{arg}(z)|<\pi.$

Symmetry and positivity properties (see [1, p. 375]) :

$\forall n\in\mathbb{N},\quad\forall x>0,\quad
I_{-n}(x)=I_{n}(x)\in\mathbb{R}_{+}^{*}\quad\mbox{ and }\quad
K_{-n}(x)=K_{n}(x)\in\mathbb{R}_{+}^{*}.$ (A.1)

Derivatives and Anti-derivatives :
If we set $\mathcal{Z}_{\nu}(z)\triangleq I_{\nu}(z)$ or
$e^{i\nu\pi}K_{\nu}(z)$, then for all $\nu\in\mathbb{R}$, we have

$\mathcal{Z}_{\nu}^{\prime}(z)=\mathcal{Z}_{\nu-1}(z)-\frac{\nu}{z}\mathcal{Z}_{\nu}(z)=\mathcal{Z}_{\nu+1}(z)+\frac{\nu}{z}\mathcal{Z}_{\nu}(z)$
(A.2)

and

$\int z^{\nu+1}\mathcal{Z}_{\nu}(z)dz=z^{\nu+1}\mathcal{Z}_{\nu+1}(z).$ (A.3)

Power series extension for $K_{n}$ (see [1, p. 375]) :

$\displaystyle K_{n}(z)=$
$\displaystyle\frac{1}{2}\left(\frac{z}{2}\right)^{-n}\sum_{k=0}^{n-1}\frac{(n-k-1)!}{k!}\left(\frac{-z}{4}\right)^{k}+(-1)^{n+1}\ln\left(\frac{z}{2}\right)I_{n}(z)$
$\displaystyle+\frac{1}{2}\left(\frac{-z}{2}\right)^{n}\sum_{k=0}^{+\infty}\left(\psi(k+1)+\psi(n+k+1)\right)\frac{\left(\frac{z^{2}}{4}\right)^{k}}{k!(n+k)!},$

where

$\psi(1)\triangleq-\boldsymbol{\gamma}\textnormal{ (Euler's
constant)}\qquad\qquad\forall
m\in\mathbb{N}^{*},\quad\psi(m+1)\triangleq\sum_{k=1}^{m}\frac{1}{k}-\boldsymbol{\gamma}.$

In particular

$K_{0}(z)=-\log\left(\frac{z}{2}\right)I_{0}(z)+\sum_{m=0}^{+\infty}\frac{\left(\frac{z}{2}\right)^{2m}}{(m!)^{2}}\psi(m+1),$
(A.4)

so $K_{0}$ behaves like a logarithm at $0.$

Decay property for the product $I_{\nu}K_{\nu}$ (see [8] and [26]) :
The application $(x,\nu)\mapsto I_{\nu}(x)K_{\nu}(x)$ is strictly decreasing
in each variable $x,\nu>0.$

Wronskian (see [1, p. 375]) :

$I_{\nu}^{\prime}(z)K_{\nu}(z)-I_{\nu}(z)K_{\nu}^{\prime}(z)=I_{\nu}(z)K_{\nu+1}(z)+I_{\nu+1}(z)K_{\nu}(z)=\frac{1}{z}.$
(A.5)

Ratio bounds (see [9]) :
For all $n\in\mathbb{N}$ and $x>0,$ we have

$\left\\{\begin{array}[]{l}\displaystyle\frac{xI_{n}^{\prime}(x)}{I_{n}(x)}<\sqrt{x^{2}+n^{2}}\\\
\\\
\displaystyle\frac{xK_{n}^{\prime}(x)}{K_{n}(x)}<-\sqrt{x^{2}+n^{2}}.\end{array}\right.$
(A.6)

Asymptotic expension of small argument (see [1, p. 375]) :

$\forall n\in\mathbb{N}^{*},\quad I_{n}(x)\underset{x\rightarrow
0}{\sim}\frac{\left(\frac{1}{2}x\right)^{n}}{\Gamma(n+1)}\quad\mbox{ and
}\quad K_{n}(x)\underset{x\rightarrow
0}{\sim}\frac{\Gamma(n)}{2\left(\frac{1}{2}x\right)^{n}}.$ (A.7)

Asymptotic expension of large argument (see [1, p. 375]) :

$\forall n\in\mathbb{N}^{*},\quad
I_{n}(x)\underset{x\rightarrow+\infty}{\sim}\frac{e^{x}}{\sqrt{2\pi
x}}\quad\mbox{ and }\quad
K_{n}(x)\underset{x\rightarrow+\infty}{\sim}\sqrt{\frac{\pi}{2x}}e^{-x}.$
(A.8)

Asymptotic of high order (see [1, p. 377]) :

$\forall x>0,\quad\mbox{
}I_{\nu}(x)\underset{\nu\rightarrow+\infty}{\sim}\frac{1}{\sqrt{2\pi\nu}}\left(\frac{ex}{2\nu}\right)^{\nu}\quad\mbox{
and }\quad
K_{\nu}(x)\underset{\nu\rightarrow+\infty}{\sim}\sqrt{\frac{\pi}{2\nu}}\left(\frac{ex}{2\nu}\right)^{-\nu}.$
(A.9)

##  Appendix B Local bifurcation theorem and singular integrals

We recall here the Crandall-Rabinowitz’s Theorem of local bifurcation theory
which was used to find the periodic solutions in Section 3. Its proof can be
found in [24] and [54, p.15].

######  Theorem B.1 (Crandall-Rabinowitz).

Let $X$ and $Y$ be two Banach spaces. Let $V$ be a neighborhood of $0$ in $X$
and let

$F:\begin{array}[t]{rcl}\mathbb{R}\times V&\rightarrow&Y\\\
(\Omega,x)&\mapsto&F(\Omega,x)\end{array}$

be a function of classe $C^{1}$ with the following properties

  1. (i)

(Trivial solution) $\forall\,\Omega\in\mathbb{R},\,\,F(\Omega,0)=0.$

  2. (ii)

(Regularity) $\partial_{\Omega}F$, $d_{x}F$ and $\partial_{\Omega}d_{x}F$
exist and are continuous.

  3. (iii)

(Fredholm property) $d_{x}F(0,0)$ is a Fredholm operator with index $0$ and
$\ker\left(d_{x}F(0,0)\right)=\langle x_{0}\rangle.$

  4. (iv)

(Transversality assumption) $\partial_{\Omega}d_{x}F(0,0)[x_{0}]\not\in
R\left(d_{x}F(0,0)\right).$

If $\chi$ denotes any complement of $\ker\left(d_{x}F(0,0)\right)$ in $X$,
then there exist

  1. •

$U$ a neighborhood of $(0,0)$ in $\mathbb{R}\times V,$

  2. •

an interval $(-a,a)$ for some $a>0$,

  3. •

continuous functions $\psi:(-a,a)\rightarrow\mathbb{R}$ and
$\phi:(-a,a)\rightarrow\chi$ satisfying $\psi(0)=0$ and $\phi(0)=0$

such that the set of the zeros of $F$ in $U$ can be described as the following
two curves intersecting at $(0,0)$

$\Big{\\{}(\Omega,x)\in U\quad\textnormal{s.t.}\quad
F(\Omega,x)=0\Big{\\}}=\Big{\\{}\big{(}\psi(s),sx_{0}+s\phi(s)\big{)}\quad\textnormal{s.t.}\quad|s|<a\Big{\\}}\cup\Big{\\{}(\Omega,0)\in
U\Big{\\}}.$

Now, we also state some continuity properties of singular integral operators.
We may refer to [36, 44, 56, 58] for a proof of the following result.

######  Lemma B.1.

Consider a function
$\mathtt{K}:\mathbb{T}\times\mathbb{T}\rightarrow\mathbb{C}$ such that for
some $C_{0}>0$ we have

  1. •

$\mathtt{K}$ is measurable on
$\mathbb{T}\times\mathbb{T}\setminus\\{(w,w),\,w\in\mathbb{T}\\}$ and

$\forall(w,\tau)\in\mathbb{T}^{2},\qquad
w\neq\tau\quad\Rightarrow\quad|\mathtt{K}(w,\tau)|\leqslant C_{0}.$

  2. •

For any $\tau\in\mathbb{T}$, the application $w\mapsto\mathtt{K}(w,\tau)$ is
differentiable in $\mathbb{T}\setminus\\{\tau\\}$ and

$\forall(w,\tau)\in\mathbb{T}^{2},\qquad
w\neq\tau\quad\Rightarrow\quad|\partial_{w}\mathtt{K}(w,\tau)|\leqslant\frac{C_{0}}{|w-\tau|}.$

Consider the operator $\mathcal{T}_{\mathtt{K}}$ defined by

$\mathcal{T}_{\mathtt{K}}(f)(w)=\fint_{\mathbb{T}}\mathtt{K}(w,\tau)f(\tau)\,d\tau.$

Then, the operator $\mathcal{T}_{\mathtt{K}}$ is continuous from
$L^{\infty}(\mathbb{T})$ to $C^{\delta}(\mathbb{T})$ for any $0<\delta<1$ and
there exists $C_{\delta}>0$ such that

$\|\mathcal{T}_{\mathtt{K}}(f)\|_{C^{\delta}(\mathbb{T})}\leqslant
C_{\delta}C_{0}\|f\|_{L^{\infty}(\mathbb{T})}.$

## References

  * [1] M. Abramowitz, I. A. Stegun, Handbook of mathematical functions with formulas, graphs, and mathematical tables, volume 55 of National Bureau of Standards Applied Mathematics Series, (1964).
  * [2] V. I. Arnold, Small denominators and problems of stability of motion in classical mechanics and celestial mechanics, Uspekhi Mat. Nauk 18 (1963), 91–192.
  * [3] P. Baldi, M. Berti, E. Haus, R. Montalto, Time quasi-periodic gravity water waves in finite depth, Invent. Math. 214 (2018), no. 2, 739–911.
  * [4] P. Baldi, M. Berti, R. Montalto, KAM for quasi-linear and fully nonlinear forced perturbations of Airy equation, Math. Ann. 359 (2014), no. 1-2, 471–536.
  * [5] P. Baldi, M. Berti, R. Montalto, KAM for autonomous quasi-linear perturbations of KdV, Ann. Inst. H. Poincaré Analyse Non. Lin. 33 (2016), no. 6, 1589–1638.
  * [6] P. Baldi, M. Berti, R. Montalto, KAM for autonomous quasi-linear perturbations of mKdV, Boll. Unione Mat. Ital. 9 (2016), 143–188.
  * [7] C. Bardos, J. S. Linshiz, E. S. Titi, Global regularity for a Birkhoff-Rott-$\alpha$ approximation of the dynamics of vortex sheets of the 2D Euler equations, Physica D: Nonlinear Phenomena 237 (2008), 1905–1911.
  * [8] A. Baricz, On a product of modified Bessel functions Proc. Amer. Math. Soc. 137 (2009), no. 1, 189–193.
  * [9] A. Baricz, S. Ponnusamy, On Turán type inequalities for modified Bessel functions, Proc. Amer. Math. Soc. 141 (2013), no. 2, 523–532.
  * [10] M. Berti, KAM theory for partial differential equations, Analysis in Theory and Applications, 35 (2019), no. 3, 235–267.
  * [11] M. Berti, P. Bolle, A Nash-Moser approach to KAM theory, Fields Institute Communications, special volume “Hamiltonian PDEs and Applications”, (2015), 255–284.
  * [12] M. Berti, L. Franzoi, A. Maspero, Traveling quasi-periodic water waves with constant vorticity, Archive for Rational Mechanics and Analysis, 240 (2021), 99–202.
  * [13] M. Berti, L. Franzoi, A. Maspero, Pure gravity traveling quasi-periodic water waves with constant vorticity, arXiv:2101.12006.
  * [14] M. Berti, Z. Hassainia, N. Masmoudi, Time quasi-periodic vortex patches, arXiv:2202.06215.
  * [15] M. Berti, T. Kappeler, R. Montalto, _Large KAM tori for quasi-linear perturbations of KdV,_ Archive for Rational Mechanics, 239 (2021), 1395–1500.
  * [16] M. Berti, R. Montalto, Quasi-periodic standing wave solutions of gravity-capillary water waves, MEMO, Volume 263, 1273, Memoires AMS, ISSN 0065-9266, (2020).
  * [17] J. Burbea, Motions of vortex patches, Lett. Math. Phys. 6 (1982), no. 1, 1–16.
  * [18] A. V. Busuioc, On second grade fluids with vanishing viscosity, Portugaliae Mathematica Nova Série, 59 (2002), no. 1, 47–65.
  * [19] A. V. Busuioc, D. Iftimie, Weak solutions for the $\alpha$-Euler equations and convergence to Euler, Nonlinearity 30 (2017), no. 12, 4534–4557.
  * [20] A. V. Busuioc, D. Iftimie, M. C. Lopes Filho, H. J. Nussenzveig Lopes, The limit $\alpha\to 0$ of the $\alpha$-Euler equations in the half plane with no-slip boundary conditions and vortex sheet initial data, SIAM Journal on Mathematical Analysis 52 (2020), no 5, 5257–5286.
  * [21] A. V. Busuioc, D. Iftimie, M. C. Lopes Filho, H. J. Nussenzveig Lopes, The incompressible $\alpha$-Euler equations in the exterior of a vanishing disk, arXiv:2203.14690.
  * [22] A. Castro, D. Córdoba, J. Gómez-Serrano, Uniformly rotating analytic global patch solutions for active scalars, Ann. PDE, 2 (2016), no. 1, 1–34.
  * [23] H. Chen, Four ways to evaluate a Poisson integral, Mathematics Magazine 75 (2002), no. 4, 290–294.
  * [24] M. G. Crandall, P. H. Rabinowitz, Bifurcation from simple eigenvalues. J. of Func. Analysis, 8 (1971), 321–340.
  * [25] G. S. Deem, N. J. Zabusky, Vortex waves : Stationary ”V-states”, Interactions, Recurrence, and Breaking, Phys. Rev. Lett. 40 (1978), no. 13, 859–862.
  * [26] D. G. Dritschel, T. Hmidi, C. Renault, Imperfect bifurcation for the shallow-water quasi-geostrophic equations, Arch. Ration. Mech. Anal. 231 (2019), no. 3, 1853–1915.
  * [27] D. G. Dritschel, M. M. Jalali, Stability and evolution of two opposite-signed quasi-geostrophic shallow-water vortex patches, Geophys. Astrophys. Fluid Dyn. 114 (2020), no. 4-5, 561–587.
  * [28] D. G. Dritschel, H. Plotka, Shallow-water vortex equilibria and their stability, Journal of Physics: Conference Series, 318 (2011), no. 6.
  * [29] R. Feola, F. Giuliani, Quasi-periodic traveling waves on an infinitely deep perfect fluid under gravity, Memoirs of the American Mathematical Society, (2021).
  * [30] R. Feola, F. Giuliani, R. Montalto, M. Procesi, Reducibility of first order linear operators on tori via Moser’s theorem, J. Funct. Anal. 276 (2019), no. 3, 932–970.
  * [31] R. Feola, F. Giuliani, M. Procesi, Reducible KAM tori for Degasperis-Procesi equation, Communications in Mathematical Physics 377 (2020), 1681–1759.
  * [32] C. García, Kármán vortex street in incompressible fluid models, Nonlinearity, 33 (2020), no. 4, 1625–1676.
  * [33] C. García, Vortex patches choreography for active scalar equations, Journal of Nonlinear Science, 31 (2021), no. 75, 1432–1467.
  * [34] J. Gómez-Serrano, On the existence of stationary patches Advances in Mathematics 343 (2019), 110–140.
  * [35] J. Gómez-Serrano, J. Park, J. Shi, Y. Yao, Symmetry in stationary and uniformly-rotating solutions of active scalar equations, Duke Math. J. 170 (2021), no. 13, 2957–3038.
  * [36] Z. Hassainia, T. Hmidi, On the V-States for the generalized quasi-geostrophic equations, Comm. Math. Phys. 337 (2015), no. 1, 321–377.
  * [37] Z. Hassainia, T. Hmidi, Steady asymmetric vortex pairs for Euler equations, American Institut of Mathematical Science, 41 (2021), no. 4, 1939–1969.
  * [38] Z. Hassainia, T. Hmidi, F. de la Hoz, Doubly connected V-states for the generalized surface quasi-geostrophic equations, Cambridge University Press, 439 (2018) 90–117.
  * [39] Z. Hassainia, T. Hmidi, F. de la Hoz, J. Mateu, An analytical and numerical study of steady patches in the disc, Analysis and PDE, 9 (2015), no. 10.
  * [40] Z. Hassainia, T. Hmidi, N. Masmoudi, KAM theory for active scalar equations, arXiv:2110.08615.
  * [41] Z. Hassainia, N. Masmoudi, M. H. Wheeler, Global bifurcation of rotating vortex patches, Comm. Pure Appl. Math. Vol. LXXIII (2020), 1933–1980
  * [42] Z. Hassainia, E. Roulley, Boundary effects on the existence of quasi-periodic solutions for Euler equations, arXiv:2202.10053.
  * [43] Z. Hassainia, M. Wheeler, Multipole vortex patch equilibria for active scalar equations, arXiv:2103.06839.
  * [44] L. L. Helms, Potential theory, Springer-Verlag London (2009).
  * [45] T. Hmidi, F. de la Hoz, J. Mateu, J. Verdera, Doubly connected V-states for the planar Euler equations, SIAM J. Math. Anal. 48 (2016), no. 3, 1892–1928.
  * [46] T. Hmidi, J. Mateu, Bifurcation of rotating patches from Kirchhoff vortices, Discrete Contin. Dyn. Syst. 36 (2016), no. 10, 5401–5422.
  * [47] T. Hmidi, J. Mateu, Degenerate bifurcation of the rotating patches, Adv. Math. 302 (2016), 799–850.
  * [48] T. Hmidi, J. Mateu, Existence of corotating and counter-rotating vortex pairs for active scalar equations. Comm. Math. Phys. 350 (2017), no. 2, 699–747.
  * [49] T. Hmidi, J. Mateu, J. Verdera, Boundary Regularity of Rotating Vortex Patches. Arch. Ration. Mech. Anal. 209 (2013), no. 1, 171–208.
  * [50] T. Hmidi, J. Mateu, J. Verdera, On rotating doubly connected vortices. J. Differential Equations 258 (2015), no. 4, 1395–1429.
  * [51] T. Hmidi, E. Roulley, Time quasi-periodic vortex patches for quasi-geostrophic shallow-water equations, arXiv:2110.13751.
  * [52] D. D. Holm, J. E. Marsden, T. S. Ratiu, Euler-Poincaré models of ideal fluids with nonlinear dispersion, Phys. Rev. Lett. 80 (1998), 4173–4176.
  * [53] D. D. Holm, J. E. Marsden, T. S. Ratiu, The Euler-Poincaré equations and semidirect products with applications to continuum theories, Adv. Math. 137 (1998), 1–81.
  * [54] H. Kielhöfer, Bifurcation Theory: An Introduction With Applications to Partial Differential Equations, Springer (2011)
  * [55] G. Kirchhoff, Vorlesungen uber mathematische Physik, Leipzig, (1874).
  * [56] R. Kress, Linear Integral Equations, Springer–New York–Heidelberg Dordrecht–London (2014).
  * [57] A. N. Kolmogorov, On the persistence of conditionally periodic motions under a small change of the hamiltonian function, Doklady Akad. Nauk SSSR 98 (1954), 527–530.
  * [58] E. Lieb, M. Loss, Analysis, American Mathematical Society (1997).
  * [59] J. S. Linshiz, E. S. Titi, On the convergence rate of the Euler-$\alpha$, an inviscid second-grade complex fluid, model to Euler equations, J. Stat. Phys. 138 (2010), 305–332.
  * [60] M. C. Lopes Filho, H. J. Nussenzveig Lopes, E. S. Titi, A. Zang, Convergence of the 2D Euler-$\alpha$ to Euler equations in the Dirichlet case: Indifference to boundary layers. Physica D: Nonlinear Phenomena (2015), 292–293:51–61.
  * [61] J. Moser, On invariant curves of area-preserving mappings of an annulus, Nachr. Akad. Wiss., Göttingen, Math. Phys. Kl. (1962), 1–20.
  * [62] J. Nash, C1-isometric imbeddings, Annals of Mathematics 60 (1954), no. 3, 383–396.
  * [63] M. Oliver, S. Shkoller, The vortex blob method as a second-grade non-Newtonian fluid, Communications in Partial Differential Equations 26 (1999), no. 1, 295–314.
  * [64] C. Pommerenke, Boundary behaviour of conformal maps, Springer-Verlag (1992).
  * [65] C. Renault, Relative equilibria with holes for the surface quasi-geostrophic equations, Journal of Differential Equations 263 (2017), no. 1, 567–614.
  * [66] E. Roulley, Vortex rigid motion in quasi-geostrophic shallow-water equations, arXiv:2202.00404.
  * [67] H. Rüssmann, Invariant tori in non-degenerate nearly integrable Hamiltonian systems, Regul. Chaotic Dyn. 6 (2001), no. 2, 119–204.
  * [68] G. K. Vallis, Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-Scale Circulation, Cambridge University Press, 2nd edition, (2017).
  * [69] S. E. Warschawski, On the higher derivatives at the boundary in conformal mapping, Trans. Amer. Math. Soc. 38 (1935), no. 2, 310–340.
  * [70] G. N. Watson, A Treatise on the Theory of Bessel Functions, Cambridge University Press, (1922).
  * [71] Y. Yudovich, Nonstationary flow of an ideal incompressible liquid, Zh. Vych. Mat. 3 (1963), 1032–1066.
  * [72] Xiaoguang You, Aibin Zang, Global Well-Posedness of 2D Euler-$\alpha$ Equations in Exterior Domain, arXiv:2109.00915.


# A Study on the Impact of Data Augmentation for Training Convolutional Neural
Networks in the Presence of Noisy Labels
††thanks:

Emeson Pereira  PPGIA, Department of Computing
Universidade Federal Rural
de Pernambuco, Recife, Brazil
<EMAIL_ADDRESS>Gustavo Carneiro  School of Computer Science
Australian Institute of Machine Learning
University of Adelaide, Adelaide, Australia
<EMAIL_ADDRESS>Filipe R. Cordeiro  Visual Computing Lab
Universidade Federal Rural
de Pernambuco, Recife, Brazil
<EMAIL_ADDRESS>
###### Abstract

Label noise is common in large real-world datasets, and its presence harms the
training process of deep neural networks. Although several works have focused
on the training strategies to address this problem, there are few studies that
evaluate the impact of data augmentation as a design choice for training deep
neural networks. In this work, we analyse the model robustness when using
different data augmentations and their improvement on the training with the
presence of noisy labels. We evaluate state-of-the-art and classical data
augmentation strategies with different levels of synthetic noise for the
datasets MNist, CIFAR-10, CIFAR-100, and the real-world dataset Clothing1M. We
evaluate the methods using the accuracy metric. Results show that the
appropriate selection of data augmentation can drastically improve the model
robustness to label noise, increasing up to 177.84% of relative best test
accuracy compared to the baseline with no augmentation, and an increase of up
to 6% in absolute value with the state-of-the-art DivideMix training strategy.

###### Index Terms:

label noise, deep learning, classification

##  I Introduction

Convolutional Neural Networks (CNNs) have been the state-of-the-art for
computer vision tasks in the last decade [li2021survey]. The training of these
models requires large amounts of data with precise annotations in order to
provide good generalization capabilities [luo2018does]. However, human or
automatic annotations are susceptible to mislabeling due to human failure,
challenging tasks, quality of data, etc [algan2021image]. Therefore, the
presence of label noise is common and often not considered for most of the
proposed CNNs and training strategies in literature [khan2020survey] .
Moreover, most real-world large datasets available for training CNNs for
computer vision tasks, such as ImageNet [imagenet], have the presence of noisy
labels. Zhang et al. [zhang2021understanding] show that when the dataset is
contaminated with label noise, the performance of the model is decreased. For
this purpose, different strategies for dealing with labels have been proposed
in the last years [cordeiro2020survey].

Recent works have shown the importance of using data augmentation along with
training strategies to deal with noisy labels. Zhang et al.
[zhang2021understanding] show that the use of regularisation, such as data
augmentation, dropout and weight decay helps to improve the generalization
performance. They show that the use of simple data augmentation, such as
random cropping and random perturbation of brightness helps to improve the
results. Nishi et al. [nishi2021augmentation] show that the use of weak and
strong data augmentation can improve training, showing that not only the
choice of the data augmentation matters, but also the intensity of the
augmentation. Zhang et al. propose the Mixup [mixup] augmentation, originally
proposed for general model training, but which has shown to improve the
robustness of the model in the presence of label noise [dividemix].

Most of the recent methods to deal with noisy labels use classical data
augmentation methods, such as random cropping, perturbation of brightness,
saturation, hue, and contrast, as present in AutoAugment (AutoAug)
[autoaugment]. Although the recent advances in training strategies and data
augmentation methods, there is no consensus on which is the best state-of-the-
art data augmentation for dealing with label noise. Moreover, to the best of
our knowledge, there is no work analysing the impact of data augmentation
design choice in the training of CNNs with noisy labels in the training set.

This work proposes the analysis of 13 classical and six state-of-the-art
(SOTA) data augmentation methods in order to (1) identify the impact on model
training when using different data augmentation strategies, and (2) evaluate
the best data augmentation method or combination for different levels of noise
rate. We perform the analysis for the symmetric, asymmetric and semantic noise
over different noise rates. The results are evaluated using a vanilla
ResNet-18 [resnet] with standard training and also with the SOTA training
strategy DivideMix [dividemix].

##  II Related Work

It is a common practice in modern image classification models to use data
augmentation to improve the diversity of the training set and increase model
generalization capabilities. Several training strategies have been proposed in
the last decade to address the noisy label problem [algan2021image]. The main
strategies can be classified as robust loss function [wang2019imae], sample
selection [han2018co], label cleansing [jaehwan2019photometric], meta-learning
[han2018pumpout] and semi-supervised learning [dividemix, propmix]. Although
the different proposals to train the model, only a few existing data
augmentation methods have been explored to address this problem. Figure 1
shows a comparison of the most used data augmentation methods present in
literature for training with noisy labels. We evaluated 61 papers related to
noisy labels from the last four years and the corresponding data augmentations
used. Each paper can use more than one data augmentation method, and we listed
the main approaches found in the literature for papers proposed to tackle the
label noise problem.

Figure 1: Number of papers related to training with noisy labels that use each
type of data augmentation. Sixty-one papers from the last years were evaluated
for this analysis.

For the recent strategies to deal with noisy labels, the standard approach is
to use classical image processing, such as random cropping and flip, as shown
in Figure 1. A few recent approaches [dividemix, propmix, zhang2020distilling]
have started to use SOTA data augmentation methods such as Mixup [mixup],
CutMix [cutmix] and RandAugment (RandAug) [randaugment].

The Mixup [mixup] data augmentation proposes a linear combination between
samples in order to reduce the empirical vicinal risk. Their approach showed
to improve the robustness of the model to label noise and it has been used in
training approaches such as DivideMix [dividemix] and PropMix [propmix]. The
DivideMix approach uses random crop and flipping along with Mixup, whereas
PropMix also includes RandAug approach.

Chen et al. [chen2021robustness] evaluate the training when using no
augmentation, standard augmentations (cropping and flip) and strong
augmentation (using AutoAug). Their results show an improvement when using
strong data augmentation, however their analysis is restricted to one strategy
of SOTA data augmentation and one combination of standard augmentation.

Although several recent data augmentation methods have been proposed in the
literature, they have not been evaluated in the context of noisy labels.
Furthermore, there is no study addressing the analysis of the impact of
different data augmentations strategies and the corresponding results for
training the model with noisy labels. This work performs this analysis,
comparing classical, SOTA and combinations of the data augmentations
approaches, evaluating their impact on different datasets and noise rates. In
a similar work, Cordeiro et al. [cordeiro2020survey] show the main strategies
to deal with noisy labels in the last years. Although they show the different
techniques, there is no experiments related to data augmentation or analysis
of data augmentation performance in the label noise context. Our work aims to
show how the choice of data augmentation impacts the training with noisy
labels and which strategies can make the model more robust under different
noise scenarios.

##  III Methods

###  III-A Problem Definition

Lets consider $D=\\{(x_{1},y_{1}),...,(x_{n},y_{n})\\}$ as the training set of
a classification problem, where $x_{i}\in\mathcal{X}$ is the $i^{th}$ image
and $y_{i}\in Y$ is a one-hot vector representing the label over $c$ classes.
We denote $y\in Y$ as the observed labels (which may or not contain noise). We
denote $y_{i}^{*}$ as the true label of $x_{i}$. The distribution of different
labels for sample $x$ is denoted by $p(c|x)$, and $\sum_{c=1}^{C}p(c|x)=1$.

The overall noise rate is defined by $\eta\in[0,1]$ and $\eta_{jc}$ represents
the probability of a class $j$ be flipped to class $c$, as
$p(y_{i}=c|y_{i}^{*}=j)$. The main types of noise evaluated in this work are
described as follows: symmetric, asymmetric and semantic.

The symmetric noise represents a noise process when a label has equal
probability to flip to another class. In the symmetric noise, the true label
is also included in the label flipping options, which means that in
$\eta_{jc}=\frac{\eta}{C-1},\forall j\in Y$. Although the symmetric noise is
unlikely to represent a realistic scenario for noisy labels, it is the main
baseline for noisy label experiments.

The asymmetric noise, as described in [patrini2017making], is closer to a
real-world label noise based on flipping labels between similar classes. For
asymmetric noise, $\eta_{jc}$ is class conditional. The semantic noise depends
on both the classes $j,c\in\mathcal{Y}$ and the image $\mathbf{x}_{i}$.

###  III-B Data Augmentations

The data augmentation methods evaluated in this work are divided into two
categories: basic and SOTA methods. For the basic augmentations, we evaluated
random crop, horizontal flip, rotation, translation, shear, autocontrast,
invert, equalize, posterize, contrast, brightness, sharpness, solarize and the
combination of them. For the SOTA methods we evaluated AutoAug [autoaugment],
RandAug [randaugment], Cutout [cutout], Mixup [mixup], CutMix [cutmix], AugMix
[augmix] and their combination with the previous approaches.

For the basic augmentations, we used spatial-level and pixel-level
transformations on the image, which usually are defined as weak augmentations.
We added Table III-B describing the effects of each basic augmentation method.
Figure LABEL:fig:dataug shows the different basic data augmentations and their
effects on the original image.

TABLE I: Description of the basic augmentations evaluated in this work,
separated as spatial-level and pixel-level.