# On exponential moments of the homogeneous Boltzmann equation for hard
potentials without cutoff

Nicolas Fournier  Sorbonne Université, LPSM-UMR 8001, Case courrier 158,75252
Paris Cedex 05, France<EMAIL_ADDRESS>
###### Abstract.

We consider the spatially homogeneous Boltzmann equation for hard potentials
without cutoff. We prove that an exponential moment of order
$\rho=\min\\{2\gamma/(2-\nu),2\\}$, with the usual notation, is immediately
created. This is stronger than what happens in the case with cutoff. We also
show that exponential moments of order $\rho\in(0,2]$ are propagated.

###### Key words and phrases:

Kinetic equations, Exponential moments, Strong localization.

###### 2010 Mathematics Subject Classification:

82C40

##  1\. Introduction and results

###  1.1. The Boltzmann equation

We consider a spatially homogeneous gas modeled by the Boltzmann equation: the
density $f_{t}(v)$ of particles with velocity $v\in{\mathbb{R}}^{3}$ at time
$t\geq 0$ solves

(1) $\displaystyle\partial_{t}f_{t}(v)=\int_{{\mathbb{R}}^{3}}{\rm
d}{v_{*}}\int_{{\mathbb{S}^{2}}}{\rm d}\sigma
B(|v-{v_{*}}|,\cos\theta)\big{[}f_{t}(v^{\prime})f_{t}(v^{\prime}_{*})-f_{t}(v)f_{t}({v_{*}})\big{]},$

where

(2) $\displaystyle
v^{\prime}=\frac{v+{v_{*}}}{2}+\frac{|v-{v_{*}}|}{2}\sigma,\quad
v^{\prime}_{*}=\frac{v+{v_{*}}}{2}-\frac{|v-{v_{*}}|}{2}\sigma\quad\hbox{and}\quad\cos\theta=\frac{v-{v_{*}}}{|v-{v_{*}}|}\cdot\sigma.$

We refer to the book of Cercignani [6] and to the long review papers of
Villani [15] and Alexandre [1] for some detailed and complete accounts of what
is known, both from the physical and mathematical points of view, about this
equation. One may assume without loss of generality that the initial condition
satisfies

$\int_{{\mathbb{R}}^{3}}f_{0}(v){\rm
d}v=1,\quad\int_{{\mathbb{R}}^{3}}vf_{0}(v){\rm
d}v=0\quad\hbox{and}\quad\int_{{\mathbb{R}}^{3}}|v|^{2}f_{0}(v){\rm d}v=1,$

and these quantities, namely the mass, momentum and kinetic energy, are
constant, at least informally, as time evolves.

###  1.2. Assumptions

We will suppose that for some $\gamma\in(0,1]$ and some $\nu\in(0,2)$,

$\displaystyle(H_{1}(\gamma))\qquad
B(|v-{v_{*}}|,\cos\theta)\sin\theta=|v-{v_{*}}|^{\gamma}\beta(\theta)\quad\hbox{for
some measurable $\beta:(0,\pi]\to{\mathbb{R}}_{+}$,}$
$\displaystyle(H_{2}(\nu))\qquad\exists\;\kappa_{1},\kappa_{2}\in(0,\infty),\;\forall\;\theta\in(0,\pi],\;\;\kappa_{1}\theta^{-\nu-1}\leq\beta(\theta)\leq\kappa_{2}\theta^{-\nu-1}.$

As explained in [6, 15, 1], when particles interact through a repulsive force
in $1/r^{s}$ for some $s>2$, we have $(H_{1}(\gamma))$ and $(H_{2}(\nu))$ with
$\gamma=(s-5)/(s-1)$ and $\nu=2/(s-1)$. When $\gamma\in(0,1)$ (i.e. $s>5$),
one speaks of hard potentials.

One speaks of hard potentials with cutoff when we have $(H_{1}(\gamma))$ for
some $\gamma\in(0,1]$ and when $(H_{2}(\nu))$ is replaced by the condition
$\int_{0}^{\pi}\beta(\theta){\rm d}\theta\in(0,\infty)$, which more or less
corresponds to the case where $\nu=0$.

###  1.3. Weak solutions

First, we parameterize (2) as in [8]. For every
$X\in{\mathbb{R}}^{3}\setminus\\{0\\}$, we introduce
$I(X),J(X)\in{\mathbb{R}}^{3}$ such that
$(\frac{X}{|X|},\frac{I(X)}{|X|},\frac{J(X)}{|X|})$ is an orthonormal basis of
${\mathbb{R}}^{3}$. We also put $I(0)=J(0)=0$. For
$X,v,{v_{*}}\in{\mathbb{R}}^{3}$, $\theta\in(0,\pi]$ and $\varphi\in[0,2\pi)$,
we set

(6)
$\displaystyle\left\\{\begin{array}[]{l}\Gamma(X,\varphi)=(\cos\varphi)I(X)+(\sin\varphi)J(X),\\\\[5.69054pt]
\displaystyle
v^{\prime}=v-\frac{1-\cos\theta}{2}(v-{v_{*}})+\frac{\sin\theta}{2}\Gamma(v-{v_{*}},\varphi),\\\\[5.69054pt]
\displaystyle
v^{\prime}_{*}={v_{*}}+\frac{1-\cos\theta}{2}(v-{v_{*}})-\frac{\sin\theta}{2}\Gamma(v-{v_{*}},\varphi).\end{array}\right.$

We denote by ${\mathcal{P}}({{\mathbb{R}}^{3}})$ the set of probability
measures on ${{\mathbb{R}}^{3}}$. For $p\in{\mathbb{R}}_{+}$ and
$f\in{\mathcal{P}}({{\mathbb{R}}^{3}})$, we introduce the moment of order $p$
of $f$:

$m_{p}(f)=\int_{{\mathbb{R}}^{3}}|v|^{p}f({\rm d}v).$

We use the following classical notion of weak solutions.

######  Definition 1.

Assume $(H_{1}(\gamma))$ and $(H_{2}(\nu))$ for some $\gamma\in(0,1]$ and some
$\nu\in(0,2)$. A weakly continuous family $(f_{t})_{t\geq 0}$ of probability
measures on ${{\mathbb{R}}^{3}}$ is a weak solution to (1) if for all $t\geq
0$,

$\displaystyle\int_{{\mathbb{R}}^{3}}vf_{t}({\rm d}v)=0\quad\hbox{and}\quad
m_{2}(f_{t})=1$

and if for any $\phi\in C^{2}_{b}({{\mathbb{R}}^{3}})$ and any $t\geq 0$,
using the parameterization (6),

(7) $\displaystyle\frac{{\rm d}}{{\rm
d}t}\\!\int_{{\mathbb{R}}^{3}}\\!\\!\phi(v)f_{t}({\rm
d}v)\\!=\\!\\!\int_{{\mathbb{R}}^{3}}\\!\int_{{\mathbb{R}}^{3}}\\!\int_{0}^{\pi}\\!\\!\\!\int_{0}^{2\pi}\\![\phi(v^{\prime})\\!+\phi({v_{*}}^{\prime})\\!-\phi(v)\\!-\phi({v_{*}})]|v\\!-\\!{v_{*}}|^{\gamma}{\rm
d}\varphi\beta(\theta){\rm d}\theta f_{t}({\rm d}{v_{*}})f_{t}({\rm d}v).$

As shown by Lu-Mouhot in [11], see also Villani [15], weak solutions exist
starting from any given initial condition
$f_{0}\in{\mathcal{P}}({{\mathbb{R}}^{3}})$ such that
$\int_{{\mathbb{R}}^{3}}vf_{0}({\rm d}v)=0$ and $m_{2}(f_{0})=1$, and they
satisfy

(8) $\displaystyle\textstyle\hbox{for all $p\geq 0$, all
$t_{0}>0$,}\quad\sup_{t\geq t_{0}}m_{p}(f_{t})<\infty.$

Let us mention the recent uniqueness result of Heydecker [10], which concerns
the case where $\nu\in(0,1)$, assuming only that $m_{p}(f_{0})<\infty$ for
some sufficiently large $p$.

###  1.4. Main result

Here is our main result.

######  Theorem 2.

Assume $(H_{1}(\gamma))$ and $(H_{2}(\nu))$ for some $\gamma\in(0,1]$ and some
$\nu\in(0,2)$. Consider any weak solution $(f_{t})_{t\geq 0}$ to (1). (i) Put
$\rho=\min\\{2\gamma/(2-\nu),2\\}$. There are some constants $T>0$ and
$\sigma>0$, depending only on $\gamma,\nu,\kappa_{1},\kappa_{2}$, such that

$\sup_{t\in[0,T]}\int_{{\mathbb{R}}^{3}}\exp[\sigma
t^{\rho/\gamma}|v|^{\rho}]f_{t}({\rm d}v)\leq 4.$

(ii) For any $A>0$, any $\sigma_{0}>0$, any $\rho\in(0,2]$, there is a
constant $\sigma>0$, depending only on
$\gamma,\nu,\kappa_{1},\kappa_{2},\rho,\sigma_{0},A$, such that

$\int_{{\mathbb{R}}^{3}}\exp[\sigma_{0}|v|^{\rho}]f_{0}({\rm d}v)\leq
A\quad\Longrightarrow\quad\sup_{t\geq
0}\int_{{\mathbb{R}}^{3}}\exp[\sigma|v|^{\rho}]f_{t}({\rm d}v)\leq 6.$

Since $\min\\{2\gamma/(2-\nu),2\\}>\gamma$, point (i) is stronger than in the
cutoff case where, as we will see in the next subsection, only exponential
moments of order $\rho=\gamma$ are created. In (ii), we have a possible
deterioration of the constant $\sigma$, as in all the references below.

By (ii), (i) can be extended to: there is $\sigma>0$ such that, with
$\rho=\min\\{2\gamma/(2-\nu),2\\}$,

$\sup_{t\geq
0}\int_{{\mathbb{R}}^{3}}\exp[\sigma\min\\{t^{\rho/\gamma},1\\}|v|^{\rho}]f_{t}({\rm
d}v)\leq 6.$

###  1.5. References

There is a large literature on the subject, because exponential moments can be
used for different purposes, such as estimating the rate of convergence to
equilibrium, see Mouhot [12], or uniqueness, see [9].

Using the famous Povzner inequality [13], Wennberg [16] discovered that
polynomial moments are immediately created by the homogeneous Boltzmann
equation for cutoff hard potentials (CHP in short), i.e. when the angular
cross section $\beta$ is assumed to be integrable on $[0,\pi]$, which roughly
corresponds to the case $\nu=0$. This really requires that $\gamma>0$ and the
main intuition is that particles with large velocities are quickly slowed
down, because they collide at large rate (since $\gamma>0$) with slow
particles.

In his seminal paper [4], Bobylev proved that Gaussian moments ($\rho=2$) are
propagated, still for CHP. This relies on very tight computations involving a
recursive ODE argument to estimate the polynomial moments, which are then
summed to estimate Gaussian moments. Let us also cite Bobylev-Gamba-Panferov
[5] who studied inelastic collisions. Following the ideas of [4], Mouhot [12]
managed to create exponential moments of order $\rho=\gamma/2$, still for CHP.
Following the same approach, Lu-Mouhot [11] were able to create exponential
moments of order $\rho=\gamma$ for CHP and non-cutoff hard potentials (NCHP).
Alonso-Cañizo-Gamba-Mouhot [2] found a much simpler method to create
exponential moments of order $\rho=\gamma$ and propagate exponential moments
of order $\rho\in(0,2]$, for CHP. Let us finally quote Alonso-Gamba-Tasković
[3], who studied some much stronger Lebesgue and Sobolev norms of $f_{t}$ with
exponential weights for CHP.

Concerning NCHP, there is the work of Lu-Mouhot [11] already mentioned. The
proof of Lemma 4.1 in Fournier-Mouhot [9], which concerns exponential moments
for NCHP, is unfortunately false, there is a major gap (the function
$\delta(\eta)$ in (4.6) actually depends on $p$). What was required there for
the uniqueness criterion for NCHP to imply a well-posedness result, was the
propagation of exponential moments of order $\rho=\gamma$. Tasković-Alonso-
Gamba-Pavlović [14] have shown, for NCHP, creation of exponential moments of
order $\rho=\gamma$ and propagation of exponential moments of order
$\rho\in(0,4/(2+\nu)]$ (which contains $\rho=\gamma\in(0,1]$ and thus fixes
the issue in [9]).

It thus seems that concerning NCHP, [14] contains the best available results,
and Theorem 2 is stronger both for creation and propagation. In particular, we
show that NCHP create more exponential moments than CHP.

The homogeneous Landau equation for hard potentials, which often behaves in a
similar way as the Boltzmann equation, but which is considerably simpler in
many points, immediately creates Gaussian moments ($\rho=2$) for any value of
$\gamma\in(0,1]$, see [7]. Once this is observed, it is natural to wonder if
NCHP create more exponential moments than CHP. The answer is not intuitively
clear, because the (considerably many) additional collisions caused by the
singularity of $\beta$ near $0$ involve some (considerably) small values of
$\theta$ and do not much slow down particles with high velocity. Actually, the
effect is strong enough to modify the behavior of the solutions: exponential
moments of order $\rho=\min\\{2\gamma/(2-\nu),2\\}>\gamma$ are created by
NCHP, while only exponential moments of order $\rho=\gamma$ are created by
CHP.

By the way, Theorem 2-(i) implies that for any $\gamma\in(0,1]$, Gaussian
moments ($\rho=2$) are created by NCHP if the angular cross section is
singular enough, namely if $\nu\geq 2-\gamma$.

Our strategy is the same as that of [14]: we adapt the ideas of [2] to NCHP,
taking advantage of the simplicity of the method. The present paper resembles
[14] in several points. In particular, some Mittag-Leffler moments of the form

$\sum_{n\geq 0}\frac{a^{n}m_{2n}(f)}{\Gamma(\alpha n+1)},$

with $a>0$ and $\alpha\geq 1$, are used in [14], while we are led to use some
series of the form

$\sum_{n\geq 0}\frac{a^{n}m_{2n}(f)}{(n!)^{\alpha}},$

with $a>0$ and $\alpha\geq 1$. This is almost the same thing and in both
cases, this is enough to control some exponential moments of the form
$\int_{{\mathbb{R}}^{3}}\exp(b|v|^{\rho})f({\rm d}v)$, with $\rho=2/\alpha$,
see Lemma 10. The main advantage of using such series is that it then suffices
to study integer moments $m_{2n}(f)$, which leads to much more explicit
computations than if using non-integer moments, as is done e.g. in [2], where
$\int_{{\mathbb{R}}^{3}}\exp(b|v|^{\rho})f({\rm d}v)$ is more naturally
studied through $\sum_{n\geq 0}(n!)^{-1}b^{n}m_{\rho n}(f)$.

However, we try to really take advantage of the singularity of the cross
section to establish a stronger Povzner inequality than in the cutoff case,
see Lemma 3 and the paragraph below. We then have to adapt suitably the proof
of [2], on the one hand because we can only deal with integer moments, and on
the other hand because we have to exploit the new Povzner inequality.

###  1.6. About optimality

Since the solutions to (1) converge to some Maxwell (Gaussian) distributions,
which are stationary solutions, we cannot expect to create or propagate
exponential moments of order $\rho>2$. The propagation result thus seems
optimal. Concerning creation, one may get convinced, following the proofs of
Lemmas 3 and 6, that for some constant $c>0$,

$\forall\,n\geq 2,\quad m_{2n}^{\prime}(f_{t})\geq-
cn^{\nu/2}m_{2n+\gamma}(f_{t}).$

Admitting, and this is not so clear, that the Hölder inequality is sharp
enough so that we have
$m_{2n+\gamma}(f_{t})\simeq[m_{2n}(f_{t})]^{1+\gamma/2n}$, we end with
$m_{2n}^{\prime}(f_{t})\gtrsim-n^{\nu/2}[m_{2n}(f_{t})]^{1+\gamma/2n}$, from
which we easily conclude, if $m_{2n}(0)=\infty$, that
$m_{2n}(f_{t})\gtrsim[n^{1-\nu/2}/t]^{2n/\gamma}$. Still informally, this
should tell us that $m_{\rho n}(f_{t})\gtrsim[n^{1-\nu/2}/t]^{\rho n/\gamma}$,
so that

$\int_{{\mathbb{R}}^{3}}\exp[\sigma|v|^{\rho}]f_{t}({\rm d}v)=\sum_{n\geq
0}\frac{\sigma^{n}m_{\rho n}(f_{t})}{n!}\gtrsim\sum_{n\geq
0}\frac{\sigma^{n}n^{\rho(1-\nu/2)n/\gamma}}{t^{\rho n/\gamma}n!}.$

By Stirling’s formula $n!\sim\sqrt{2\pi n}(n/e)^{n}$, this series is
divergent, for any value of $\sigma>0$, when $\rho(1-\nu/2)/\gamma>1$, i.e.
when $\rho>2\gamma/(2-\nu)$, which is coherent with Theorem 2-(i).

###  1.7. About uniqueness

Assume $(H_{1}(\gamma))$ and $(H_{2}(\nu))$ for some $\gamma\in(0,1]$ and some
$\nu\in(0,1)$. Using Theorem 2 and [9, Theorem 2.2], it seems possible to
prove, in a few pages, the well-posedness of (1) assuming that the initial
condition satisfies $\int_{{\mathbb{R}}^{3}}\exp{(|v|^{\delta})}f_{0}({\rm
d}v)<\infty$ for some $\delta>0$. This is stronger than [9], where we assumed
that $\int_{{\mathbb{R}}^{3}}\exp{(|v|^{\gamma})}f_{0}({\rm d}v)<\infty$, but
weaker than the recent result of Heydecker [10], who only assumes that
$m_{p}(f_{0})<\infty$ for some large explicitable $p$.

###  1.8. Plan

The paper is technical and we are guided by computations rather than
intuition.

In Section 2, we establish a Povzner lemma, which is stronger than what is
known in the cutoff case. We handle the whole computation as explicitly as
possible, not relying on any previous Povzner estimate, because this is
required if we really want to show that the singular part of the cross section
accelerates the slowing down of particles.

In Section 3, we derive some differential inequalities for the even integer
moment from the Povzner inequality, and we prove some first rough estimates
about these moments.

In Section 4, we quickly study how to control exponential moments by even
integer moments and vice-versa.

Finally, we adapt the proofs of [2] to show Theorem 2-(i) in Section 5 and
Theorem 2-(ii) in Section 6. This requires some work, because we can only use
integer moments, and because we start from a different Povzner estimate.

###  1.9. Notation

We use the convention that ${\mathbb{N}}=\\{0,1,...\\}$. For
$a,b\in{\mathbb{N}}$ with $a\leq b$, we set
$[\\![a,b]\\!]=\\{a,a+1,\dots,b\\}$. In the whole paper, $(f_{t})_{t\geq 0}$
is a given weak solution satisfying $m_{0}(f_{t})=m_{2}(f_{t})=1$ and
$\int_{{\mathbb{R}}^{3}}vf_{t}({\rm d}v)=0$ for all $t\geq 0$. For
$p\in{\mathbb{R}}_{+}$ and $t\geq 0$, we set $m_{p}(t)=m_{p}(f_{t})$.

##  2\. A non-cutoff Povzner lemma

The goal of this section is to establish the following Povzner inequality.

######  Lemma 3.

Assume $(H_{2}(\nu))$ for some $\nu\in(0,2)$. There are some constants
$\lambda_{1},\lambda_{2}\in(0,\infty)$, depending only on
$\nu,\kappa_{1},\kappa_{2}$, such that for all integer $n\geq 2$, all
$v,{v_{*}}\in{{\mathbb{R}}^{3}}$,

$\displaystyle D_{n}(v,{v_{*}}):=$
$\displaystyle\int_{0}^{\pi}\int_{0}^{2\pi}[|v^{\prime}|^{2n}+|v^{\prime}_{*}|^{2n}-|v|^{2n}-|{v_{*}}|^{2n}]{\rm
d}\varphi\beta(\theta){\rm d}\theta$ $\displaystyle\leq$
$\displaystyle-\lambda_{1}n^{\nu/2}(|v|^{2n}+|{v_{*}}|^{2n})$
$\displaystyle+\lambda_{2}\sum_{a=1}^{n-1}\binom{n}{a}\Big{(}\frac{n^{\nu/2}}{(n-a)^{\nu/2+1}}+\frac{1}{a}\Big{)}[|v|^{2a}|{v_{*}}|^{2(n-a)}+|v|^{2(n-a)}|{v_{*}}|^{2a}].$

In the case with cutoff, see e.g. [2], one gets (roughly) something like

$D_{n}(v,{v_{*}})\leq-(|v|^{2n}+|{v_{*}}|^{2n})+\epsilon_{n}\sum_{a=1}^{n-1}\binom{n}{a}[|v|^{2a}|{v_{*}}|^{2(n-a)}+|v|^{2(n-a)}|{v_{*}}|^{2a}],$

with $\epsilon_{n}\to 0$ as $n\to\infty$. Here the negative term is reinforced
by the factor $n^{\nu/2}$, and this is the main fact we will have to exploit.
We will also have to play tightly with the positive term, showing that despite
the fact it is not clearly multiplied by a small factor, it can be absorbed,
in some sense, by the negative term.

We start with an explicit computation of the $\varphi$-average.

######  Lemma 4.

For any integer $n\geq 2$, any $v,{v_{*}}\in{{\mathbb{R}}^{3}}$, any
$\theta\in(0,\pi]$, we have

$\displaystyle\Theta_{n}(v,{v_{*}},\theta):=\frac{1}{2\pi}\int_{0}^{2\pi}|v^{\prime}|^{2n}{\rm
d}\varphi=\Big{(}\frac{1+\cos\theta}{2}\Big{)}^{n}|v|^{2n}+\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{n}|{v_{*}}|^{2n}+\Lambda_{n}(v,{v_{*}},\theta),$

where, setting
${\mathcal{A}}_{n}=\\{(i,j,k)\in{\mathbb{N}}^{2}:i+j+k=n,\;i\leq n-1,\;j\leq
n-1,\;k\in 2{\mathbb{N}}\\}$,

$\Lambda_{n}(v,{v_{*}},\theta)\\!=\\!\\!\\!\\!\\!\\!\sum_{(i,j,k)\in{\mathcal{A}}_{n}}\\!\\!\\!\frac{n!}{i!j![(k/2)!]^{2}}\Big{(}\frac{1+\cos\theta}{2}\Big{)}^{i}\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{j}\Big{(}\frac{\sin\theta}{2}\Big{)}^{k}|v|^{2i}|{v_{*}}|^{2j}\Big{(}|v|^{2}|{v_{*}}|^{2}-(v\cdot{v_{*}})^{2}\Big{)}^{k/2}.$

###### Proof.

We fix $n\geq 2$ and divide the proof into 3 steps.

Step 1. Recalling from (6) that
$v^{\prime}=v-\frac{1-\cos\theta}{2}(v-{v_{*}})+\frac{\sin\theta}{2}\Gamma(v-{v_{*}},\varphi)$,
that $|\Gamma(v-{v_{*}},\varphi)|=|v-{v_{*}}|$ and that
$(v-{v_{*}})\cdot\Gamma(v-{v_{*}},\varphi)=0$, we find

$\displaystyle|v^{\prime}|^{2}=$
$\displaystyle|v|^{2}\\!+\\!\Big{(}\frac{1\\!-\\!\cos\theta}{2}\Big{)}^{2}|v-{v_{*}}|^{2}\\!+\\!\Big{(}\frac{\sin\theta}{2}\Big{)}^{2}|v-{v_{*}}|^{2}\\!-\\!(1-\cos\theta)v\cdot(v-{v_{*}})\\!+\\!(\sin\theta)v\cdot\Gamma(v-{v_{*}},\varphi)$
$\displaystyle=$
$\displaystyle\frac{1+\cos\theta}{2}|v|^{2}+\frac{1-\cos\theta}{2}|{v_{*}}|^{2}+(\sin\theta)v\cdot\Gamma(v-{v_{*}},\varphi).$

Hence, by Newton’s trinomial expansion, setting
${\mathcal{B}}_{n}=\\{(i,j,k)\in{\mathbb{N}}^{3}:i+j+k=n\\}$,

$\displaystyle|v^{\prime}|^{2n}=$
$\displaystyle\sum_{(i,j,k)\in{\mathcal{B}}_{n}}\frac{n!}{i!j!k!}\Big{(}\frac{1+\cos\theta}{2}\Big{)}^{i}\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{j}(\sin\theta)^{k}|v|^{2i}|{v_{*}}|^{2j}(v\cdot\Gamma(v-{v_{*}},\varphi))^{k}.$

Step 2. We now prove that for $k\in{\mathbb{N}}$,

$\frac{1}{2\pi}\int_{0}^{2\pi}(v\cdot\Gamma(v-{v_{*}},\varphi))^{k}{\rm
d}\varphi=\hbox{\rm 1}{\hskip-2.8pt}\hbox{\rm I}_{\\{k\in
2{\mathbb{N}}\\}}\frac{k!}{2^{k}[(k/2)!]^{2}}(|v|^{2}|{v_{*}}|^{2}-(v\cdot{v_{*}})^{2})^{k/2}.$

We have $v\cdot\Gamma(v-{v_{*}},\varphi)=a\cos\varphi+b\sin\varphi$, where
$a=v\cdot I(v-{v_{*}})$ and $b=v\cdot J(v-{v_{*}})$, whence

$v\cdot\Gamma(v-{v_{*}},\varphi)=\sqrt{a^{2}+b^{2}}\sin(\varphi+\varphi_{0}),$

for $\varphi_{0}$ such that $\frac{a}{\sqrt{a^{2}+b^{2}}}=\sin\varphi_{0}$ and
$\frac{b}{\sqrt{a^{2}+b^{2}}}=\cos\varphi_{0}$. We thus recognize a Wallis
integral:

$\frac{1}{2\pi}\int_{0}^{2\pi}(v\cdot\Gamma(v-{v_{*}},\varphi))^{k}{\rm
d}\varphi=\frac{2}{\pi}(a^{2}+b^{2})^{k/2}\hbox{\rm 1}{\hskip-2.8pt}\hbox{\rm
I}_{\\{k\in 2{\mathbb{N}}\\}}\int_{0}^{\pi/2}\sin^{k}\varphi{\rm
d}\varphi=\hbox{\rm 1}{\hskip-2.8pt}\hbox{\rm I}_{\\{k\in
2{\mathbb{N}}\\}}\frac{k!}{2^{k}[(k/2)!]^{2}}(a^{2}+b^{2})^{k/2}.$

To complete the step, we recall that
$(\frac{v-{v_{*}}}{|v-{v_{*}}|},\frac{I(v-{v_{*}})}{|v-{v_{*}}|},\frac{J(v-{v_{*}})}{|v-{v_{*}}|})$
is an orthonormal basis, whence

$|v|^{2}=\frac{[v\cdot(v-{v_{*}})]^{2}+[v\cdot I(v-{v_{*}})]^{2}+[v\cdot
J(v-{v_{*}})]^{2}}{|v-{v_{*}}|^{2}}=\frac{[v\cdot(v-{v_{*}})]^{2}+a^{2}+b^{2}}{|v-{v_{*}}|^{2}}$

and thus
$a^{2}+b^{2}=|v|^{2}|v-{v_{*}}|^{2}-[v\cdot(v-{v_{*}})]^{2}=|v|^{2}|{v_{*}}|^{2}-(v\cdot{v_{*}})^{2}$.

Step 3. Gathering Steps 1 and 2 and setting
${\mathcal{C}}_{n}=\\{(i,j,k)\in{\mathbb{N}}^{3}:i+j+k=n,\;k\in
2{\mathbb{N}}\\}$,

$\displaystyle\Theta_{n}(v,{v_{*}},\theta)\\!=\\!\\!\\!\\!\\!\\!\sum_{(i,j,k)\in{\mathcal{C}}_{n}}\\!\\!\\!\frac{n!}{i!j![(k/2)!]^{2}}\Big{(}\frac{1\\!+\\!\cos\theta}{2}\Big{)}^{i}\Big{(}\frac{1\\!-\\!\cos\theta}{2}\Big{)}^{j}\Big{(}\frac{\sin\theta}{2}\Big{)}^{k}|v|^{2i}|{v_{*}}|^{2j}\Big{(}|v|^{2}|{v_{*}}|^{2}\\!-(v\cdot{v_{*}})^{2}\Big{)}^{k/2}.$

It then suffices to isolate the two extreme terms $(i,j,k)=(n,0,0)$ and
$(i,j,k)=(0,n,0)$. ∎

We next estimate, sharply, some integrals in $\theta$.

######  Lemma 5.

Assume $(H_{2}(\nu))$. There are $\zeta_{1},\zeta_{2}\in(0,\infty)$, depending
only on $\nu,\kappa_{1},\kappa_{2}$, such that: (i) for all integer $n\geq 2$,

$a_{n}:=\int_{0}^{\pi}\Big{(}1-\Big{[}\frac{1+\cos\theta}{2}\Big{]}^{n}-\Big{[}\frac{1-\cos\theta}{2}\Big{]}^{n}\Big{)}\beta(\theta){\rm
d}\theta\geq\zeta_{1}n^{\nu/2};$

(ii) for all integers $a,n$ such that $1\leq a\leq n-1$, setting

$J_{n,a}:=\int_{0}^{\pi}\Big{(}\frac{1+\cos\theta}{2}\Big{)}^{a}\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{n-a}\beta(\theta){\rm
d}\theta,\quad\hbox{we
have}\quad\binom{n}{a}J_{n,a}\leq\zeta_{2}\Big{[}\frac{n^{\nu/2}}{(n-a)^{\nu/2+1}}+\frac{1}{a}\Big{]}.$

###### Proof.

We start with (i): the integrand in $a_{n}$ is nonnegative, because for
$x=(1+\cos\theta)/2\in[0,1]$, we have $x^{n}+(1-x)^{n}\leq 1$. Hence recalling
$(H_{2}(\nu))$,

$a_{n}\geq\kappa_{1}\int_{0}^{n^{-1/2}}\Big{(}1-\Big{[}\frac{1+\cos\theta}{2}\Big{]}^{n}-\Big{[}\frac{1-\cos\theta}{2}\Big{]}^{n}\Big{)}\theta^{-\nu-1}{\rm
d}\theta.$

For all $\theta\in[0,n^{-1/2}]$, we have $(1+\cos\theta)/2\leq
1-\theta^{2}/5$, whence
$[(1+\cos\theta)/2]^{n}\leq(1-\theta^{2}/5)^{n}\leq\exp(-n\theta^{2}/5)$ and
thus $1-[(1+\cos\theta)/2]^{n}\geq n\theta^{2}/10$. Next, still for
$\theta\in[0,n^{-1/2}]$, we have $(1-\cos\theta)/2\leq\theta^{2}/4$, whence,
since $n\geq 2$,

$1-\Big{[}\frac{1+\cos\theta}{2}\Big{]}^{n}-\Big{[}\frac{1-\cos\theta}{2}\Big{]}^{n}\geq\frac{n\theta^{2}}{10}-\frac{\theta^{2n}}{4^{n}}\geq\Big{(}\frac{n}{10}-\frac{1}{16}\Big{)}\theta^{2}\geq\Big{(}\frac{n}{10}-\frac{n}{32}\Big{)}\theta^{2}\geq\frac{n}{20}\theta^{2}.$

Consequently,

$a_{n}\geq\kappa_{1}\frac{n}{20}\int_{0}^{n^{-1/2}}\theta^{1-\nu}{\rm
d}\theta=\frac{\kappa_{1}}{20(2-\nu)}n^{\nu/2}.$

For (ii), we first use $(H_{2}(\nu))$ to write
$J_{n,a}\leq\kappa_{2}K_{\nu,n,a}+\kappa_{2}L_{\nu,n,a}$, where

$\displaystyle K_{\nu,n,a}=$
$\displaystyle\int_{0}^{\pi/2}\Big{(}\frac{1+\cos\theta}{2}\Big{)}^{a}\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{n-a}\theta^{-\nu-1}{\rm
d}\theta,$ $\displaystyle L_{\nu,n,a}=$
$\displaystyle\Big{(}\frac{2}{\pi}\Big{)}^{\nu+1}\int_{\pi/2}^{\pi}\Big{(}\frac{1+\cos\theta}{2}\Big{)}^{a}\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{n-a}{\rm
d}\theta.$

Using the substitution $\theta\to\pi-\theta$, we see that

$\displaystyle L_{\nu,n,a}=$
$\displaystyle\Big{(}\frac{2}{\pi}\Big{)}^{\nu+1}\int_{0}^{\pi/2}\Big{(}\frac{1+\cos\theta}{2}\Big{)}^{n-a}\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{a}{\rm
d}\theta$ $\displaystyle\leq$
$\displaystyle\Big{(}\frac{2}{\pi}\Big{)}^{\nu}\int_{0}^{\pi/2}\Big{(}\frac{1+\cos\theta}{2}\Big{)}^{n-a}\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{a}\theta^{-1}{\rm
d}\theta=\Big{(}\frac{2}{\pi}\Big{)}^{\nu}K_{0,n,n-a}.$

We will prove that for $\nu\in[0,2)$, there is a constant
$A_{\nu}\in(0,\infty)$ such that for all $1\leq a\leq n-1$,

(9) $\displaystyle\binom{n}{a}K_{\nu,n,a}\leq
A_{\nu}\frac{n^{\nu/2}}{(n-a)^{1+\nu/2}}.$

We will deduce that
$\binom{n}{a}L_{\nu,n,a}\leq[2/\pi]^{\nu}\binom{n}{a}K_{0,n,n-a}\leq[2/\pi]^{\nu}A_{0}a^{-1}$
and this will end the proof. For $\theta\in(0,\pi/2]$, we have $\theta\leq
2\sin\theta$ and $\theta^{-1}\leq[(1-\cos\theta)/2]^{-1/2}$, so that

$\theta^{-\nu-1}\leq 2\theta^{-\nu-2}\sin\theta\leq
2\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{-\nu/2-1}\sin\theta$

and thus

$K_{\nu,n,a}\leq
2\int_{0}^{\pi/2}\Big{(}\frac{1+\cos\theta}{2}\Big{)}^{a}\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{n-a-\nu/2-1}\sin\theta{\rm
d}\theta=4\int_{1/2}^{1}x^{a}(1-x)^{n-a-\nu/2-1}{\rm d}x,$

using the change of variables $x=(1+\cos\theta)/2$. Hence

$K_{\nu,n,a}\leq 4\int_{0}^{1}x^{a}(1-x)^{n-a-\nu/2-1}{\rm
d}x=4\frac{\Gamma(a+1)\Gamma(n-a-\nu/2)}{\Gamma(n+1-\nu/2)},$

where $\Gamma$ is Euler’s Gamma function. Using that $\Gamma(k+1)=k!$, that
$(n-a-\nu/2)\Gamma(n-a-\nu/2)=\Gamma(n-a+1-\nu/2)$ and setting
$u_{\nu,k}=\Gamma(k+1)/\Gamma(k+1-\nu/2)$, we realize that

$\binom{n}{a}K_{\nu,n,a}\leq
4\frac{n!\Gamma(n-a-\nu/2)}{\Gamma(n+1-\nu/2)(n-a)!}=4\frac{u_{\nu,n}}{(n-a-\nu/2)u_{\nu,n-a}}.$

But using Stirling’s formula $\Gamma(x+1)\sim\sqrt{2\pi x}(x/e)^{x}$ as
$x\to\infty$, one can verify that $u_{\nu,k}\sim k^{\nu/2}$ as $k\to\infty$ so
that there is a constant $A_{\nu}\in(1,\infty)$ such that for all $k\geq 1$,

$A_{\nu}^{-1}k^{\nu/2}\leq u_{\nu,k}\leq A_{\nu}k^{\nu/2}.$

We conclude that for all $1\leq a\leq n-1$,

$\binom{n}{a}K_{\nu,n,a}\leq
4A_{\nu}^{2}\frac{n^{\nu/2}}{(n-a)^{\nu/2}(n-a-\nu/2)}\leq\frac{4A_{\nu}^{2}}{1-\nu/2}\times\frac{n^{\nu/2}}{(n-a)^{\nu/2+1}}$

because $n-a\geq 1$ implies that $n-a-\nu/2\geq(n-a)(1-\nu/2)$. We have
checked (9) and the proof is complete. ∎

We can now handle the

###### Proof of Lemma 3.

We fix $n\geq 2$. Using (6), we realize that, with the notation of Lemma 4,

$\displaystyle\frac{1}{2\pi}\int_{0}^{2\pi}[|v^{\prime}|^{2n}+|v^{\prime}_{*}|^{2n}-|v|^{2n}-|{v_{*}}|^{2n}]{\rm
d}\varphi=$
$\displaystyle\Theta_{n}(v,{v_{*}},\theta)+\Theta_{n}({v_{*}},v,-\theta)-|v|^{2n}-|{v_{*}}|^{2n}.$

We deduce from Lemma 4 that

$\displaystyle\frac{1}{2\pi}\int_{0}^{2\pi}[|v^{\prime}|^{2n}+|v^{\prime}_{*}|^{2n}-|v|^{2n}-|{v_{*}}|^{2n}]{\rm
d}\varphi$ $\displaystyle=$
$\displaystyle-\Big{(}1-\Big{[}\frac{1+\cos\theta}{2}\Big{]}^{n}-\Big{[}\frac{1-\cos\theta}{2}\Big{]}^{n}\Big{)}\times(|v|^{2n}+|{v_{*}}|^{2n})+\Lambda_{n}(v,{v_{*}},\theta)+\Lambda_{n}({v_{*}},v,-\theta).$

Hence $D_{n}(v,{v_{*}})=-D_{n,1}(v,{v_{*}})+D_{n,2}(v,{v_{*}})$, where

$\displaystyle D_{n,1}(v,{v_{*}})=$ $\displaystyle
2\pi\int_{0}^{\pi}\Big{(}1-\Big{[}\frac{1+\cos\theta}{2}\Big{]}^{n}-\Big{[}\frac{1-\cos\theta}{2}\Big{]}^{n}\Big{)}\beta(\theta){\rm
d}\theta\times(|v|^{2n}+|{v_{*}}|^{2n}),$ $\displaystyle D_{n,2}(v,{v_{*}})=$
$\displaystyle
2\pi\int_{0}^{\pi}[\Lambda_{n}(v,{v_{*}},\theta)+\Lambda_{n}({v_{*}},v,-\theta)]\beta(\theta){\rm
d}\theta.$

We now divide the proof into 5 steps.

Step 1. By Lemma 5-(i), we have $D_{n,1}(v,{v_{*}})\geq
2\pi\zeta_{1}n^{\nu/2}(|v|^{2n}+|{v_{*}}|^{2n})$.

Step 2. We next roughly bound $|v|^{2}|{v_{*}}|^{2}-(v\cdot{v_{*}})^{2}$ by
$|v|^{2}|{v_{*}}|^{2}$ in the expression of $\Lambda_{n}$ to find

$D_{n,2}(v,{v_{*}})\leq
2\pi\sum_{(i,j,k)\in{\mathcal{A}}_{n}}\\!\\!\\!\frac{n!}{i!j![(k/2)!]^{2}}I_{i,j,k}[|v|^{2i+k}|{v_{*}}|^{2j+k}+|v|^{2j+k}|{v_{*}}|^{2i+k}],$

where

$I_{i,j,k}=\int_{0}^{\pi}\Big{(}\frac{1+\cos\theta}{2}\Big{)}^{i}\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{j}\Big{(}\frac{\sin\theta}{2}\Big{)}^{k}\beta(\theta){\rm
d}\theta.$

This can be rewritten as

$\displaystyle D_{n,2}(v,{v_{*}})\leq
2\pi\sum_{a=0}^{n}K_{n,a}[|v|^{2a}|{v_{*}}|^{2(n-a)}+|v|^{2(n-a)}|{v_{*}}|^{2a}]$

where, setting ${\mathcal{A}}_{n,a}=\\{(i,j,k)\in{\mathcal{A}}_{n}:i+k/2=a$
(whence $j+k/2=n-a)\\}$,

$K_{n,a}=\sum_{(i,j,k)\in{\mathcal{A}}_{n,a}}\frac{n!}{i!j![(k/2)!]^{2}}I_{i,j,k}.$

Step 3. We have $K_{n,n}=K_{n,0}=0$. Indeed, we e.g. have
${\mathcal{A}}_{n,n}=\emptyset$ because for $(i,j,k)\in{\mathcal{A}}_{n,n}$,
we have $i+j+k=n$ and $j+k/2=0$, whence $i=n$, which is forbidden since
$(i,j,k)\in{\mathcal{A}}_{n}$.

Step 4. We now fix $a\in[\\![1,n-1]\\!]$. Then $(i,j,k)\in{\mathcal{A}}_{n,a}$
if and only if there is $\ell\in{\mathbb{N}}$ such that
$k=2\ell\in[\\![0,n]\\!]$, $i=a-\ell\in[\\![0,n-1]\\!]$ and
$j=(n-a)-\ell\in[\\![0,n-1]\\!]$, so that

$K_{n,a}=\sum_{\ell=0}^{a\land(n-a)}\frac{n!}{(a-\ell)!(n-a-\ell)!(\ell!)^{2}}I_{a-\ell,n-a-\ell,2\ell}.$

But for any $\ell\in[\\![0,a\land(n-a)]\\!]$, it holds that

$I_{a-\ell,n-a-\ell,2\ell}=\int_{0}^{\pi}\Big{(}\frac{1+\cos\theta}{2}\Big{)}^{a-\ell}\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{n-a-\ell}\Big{(}\frac{\sin\theta}{2}\Big{)}^{2\ell}\beta(\theta){\rm
d}\theta=J_{n,a},$

where

$J_{n,a}=\int_{0}^{\pi}\Big{(}\frac{1+\cos\theta}{2}\Big{)}^{a}\Big{(}\frac{1-\cos\theta}{2}\Big{)}^{n-a}\beta(\theta){\rm
d}\theta,$

because $(1+\cos\theta)(1-\cos\theta)=\sin^{2}\theta$. Thus

$K_{n,a}=J_{n,a}\sum_{\ell=0}^{a\land(n-a)}\frac{n!}{(a-\ell)!(n-a-\ell)!(\ell!)^{2}}=J_{n,a}\binom{n}{a}\sum_{\ell=0}^{a\land(n-a)}\binom{a}{\ell}\binom{n-a}{\ell}=J_{n,a}\Big{[}\binom{n}{a}\Big{]}^{2}.$

We finally used the well-known identity $\sum_{\ell=0}^{a\land
b}\binom{a}{\ell}\binom{b}{\ell}=\binom{a+b}{a}$, which can be shown noting
that $\binom{a+b}{a}$ is the coefficient in front of $X^{a}$ of $(1+X)^{a+b}$,
while $\sum_{\ell=0}^{a\land
b}\binom{a}{\ell}\binom{b}{\ell}=\sum_{\ell=0}^{a\land
b}\binom{a}{\ell}\binom{b}{b-\ell}$ is the coefficient in front of $X^{a}$ of
$(1+X)^{a}(1+X)^{b}$.

Step 5. Gathering Steps 2-3-4, we have checked that

$\displaystyle D_{n,2}(v,{v_{*}})\leq$ $\displaystyle
2\pi\sum_{a=1}^{n-1}J_{n,a}\Big{[}\binom{n}{a}\Big{]}^{2}[|v|^{2a}|{v_{*}}|^{2(n-a)}+|v|^{2(n-a)}|{v_{*}}|^{2a}].$

The conclusion then follows from Lemma 5-(ii), from which
$\binom{n}{a}J_{n,a}\leq\zeta_{2}[\frac{n^{\nu/2}}{(n-a)^{\nu/2+1}}+\frac{1}{a}]$.
∎

##  3\. Even integer moments

Using the previous Povzner inequality, we can derive the even integer moments.

######  Lemma 6.

Assume $(H_{1}(\gamma))$ and $(H_{2}(\nu))$ for some $\gamma\in(0,1]$ and some
$\nu\in(0,2)$. For any integer $n\geq 2$, any $t>0$,

$m_{2n}^{\prime}(t)\leq-
c_{1}n^{\nu/2}m_{2n+\gamma}(t)+c_{2}S_{n}(t)+2c_{1}n^{\nu/2}2^{2n/\gamma},$

where $c_{1}=\lambda_{1}$ and $c_{2}=16\lambda_{2}$ (see Lemma 3) depend only
on $\nu,\kappa_{1},\kappa_{2}$ and where

$S_{n}(t)=\sum_{a=1}^{\lfloor
n/2\rfloor}\binom{n}{a}\frac{n^{\nu/2}}{a^{\nu/2+1}}m_{2a}(t)m_{2(n-a)+\gamma}(t).$

###### Proof.

We fix $n\geq 2$ and use the weak formulation (7) with $\phi(v)=|v|^{2n}$,
which is licit thanks to (8), to find, for any $t>0$ and with the notation of
Lemma 3,

$m_{2n}^{\prime}(t)=\int_{{\mathbb{R}}^{3}}\int_{{\mathbb{R}}^{3}}D_{n}(v,{v_{*}})|v-{v_{*}}|^{\gamma}f_{t}({\rm
d}{v_{*}})f_{t}({\rm d}v).$

Hence using Lemma 3, $m_{2n}^{\prime}(t)\leq-A_{n}(t)+B_{n}(t)$, where

$\displaystyle A_{n}(t)=$
$\displaystyle\lambda_{1}n^{\nu/2}\int_{{\mathbb{R}}^{3}}\int_{{\mathbb{R}}^{3}}(|v|^{2n}+|{v_{*}}|^{2n})|v-{v_{*}}|^{\gamma}f_{t}({\rm
d}{v_{*}})f_{t}({\rm d}v),$ $\displaystyle B_{n}(t)=$
$\displaystyle\lambda_{2}\sum_{a=1}^{n-1}\binom{n}{a}\Big{(}\frac{n^{\nu/2}}{(n-a)^{\nu/2+1}}+\frac{1}{a}\Big{)}\int_{{\mathbb{R}}^{3}}\int_{{\mathbb{R}}^{3}}[|v|^{2a}|{v_{*}}|^{2(n-a)}+|v|^{2(n-a)}|{v_{*}}|^{2a}]$
$\displaystyle\hskip 256.0748pt|v-{v_{*}}|^{\gamma}f_{t}({\rm
d}{v_{*}})f_{t}({\rm d}v).$

We now divide the proof into 2 steps.

Step 1. Using a symmetry argument and that
$|v-{v_{*}}|^{\gamma}\geq|v|^{\gamma}-|{v_{*}}|^{\gamma}$, we see that

$A_{n}(t)=2\lambda_{1}n^{\nu/2}\int_{{\mathbb{R}}^{3}}\int_{{\mathbb{R}}^{3}}|v|^{2n}|v-{v_{*}}|^{\gamma}f_{t}({\rm
d}{v_{*}})f_{t}({\rm d}v)\geq
2\lambda_{1}n^{\nu/2}(m_{2n+\gamma}(t)-m_{2n}(t)m_{\gamma}(t)).$

By Hölder’s inequality, $m_{\gamma}(t)\leq[m_{2}(t)]^{\gamma/2}=1$. Using
moreover that $x^{2n}\leq\frac{1}{2}x^{2n+\gamma}+2^{2n/\gamma}$ for all
$x\geq 0$ (separate the cases $x\leq 2^{1/\gamma}$ and $x\geq 2^{1/\gamma}$),

$m_{\gamma}(t)m_{2n}(t)\leq
m_{2n}(t)=\int_{{\mathbb{R}}^{3}}|v|^{2n}f_{t}({\rm
d}v)\leq\int_{{\mathbb{R}}^{3}}\Big{(}\frac{1}{2}|v|^{2n+\gamma}+2^{2n/\gamma}\Big{)}f_{t}({\rm
d}v)=\frac{1}{2}m_{2n+\gamma}(t)+2^{2n/\gamma}.$

All in all,

$A_{n}(t)\geq\lambda_{1}n^{\nu/2}m_{2n+\gamma}(t)-2\lambda_{1}n^{\nu/2}2^{2n/\gamma}.$

Step 2. Using a symmetry argument and that
$|v-{v_{*}}|^{\gamma}\leq|v|^{\gamma}+|{v_{*}}|^{\gamma}$, we find

$\displaystyle B_{n}(t)\leq$ $\displaystyle
2\lambda_{2}\sum_{a=1}^{n-1}\binom{n}{a}\Big{(}\frac{n^{\nu/2}}{(n-a)^{\nu/2+1}}+\frac{1}{a}\Big{)}[m_{2a+\gamma}(t)m_{2(n-a)}(t)+m_{2a}(t)m_{2(n-a)+\gamma}(t)]$
$\displaystyle=$ $\displaystyle
2\lambda_{2}\sum_{a=1}^{n-1}\binom{n}{a}\Big{(}\frac{n^{\nu/2}}{a^{\nu/2+1}}+\frac{1}{a}\Big{)}[m_{2a+\gamma}(t)m_{2(n-a)}(t)+m_{2a}(t)m_{2(n-a)+\gamma}(t)]$

by symmetry again. Since now $1/a\leq n^{\nu/2}/a^{\nu/2+1}$ and since $a\to
n^{\nu/2}/a^{\nu/2+1}$ is decreasing,

$B_{n}(t)\leq 8\lambda_{2}\sum_{a=1}^{\lfloor
n/2\rfloor}\binom{n}{a}\frac{n^{\nu/2}}{a^{\nu/2+1}}[m_{2a+\gamma}(t)m_{2(n-a)}(t)+m_{2a}(t)m_{2(n-a)+\gamma}(t)].$

But for $a\in[\\![1,\lfloor n/2\rfloor]\\!]$, we have $a\leq n-a$ whence
$m_{2a+\gamma}(t)m_{2(n-a)}(t)\leq m_{2a}(t)m_{2(n-a)+\gamma}(t)$ by Lemma 7
below, so that

$B_{n}(t)\leq 16\lambda_{2}\sum_{a=1}^{\lfloor
n/2\rfloor}\binom{n}{a}\frac{n^{\nu/2}}{a^{\nu/2+1}}m_{2a}(t)m_{2(n-a)+\gamma}(t)=16\lambda_{2}S_{n}(t).$

Recalling that $m_{2n}^{\prime}(t)=-A_{n}(t)+B_{n}(t)$ and Step 1, we have
reached our goal. ∎

We now prove a classical lemma that we used a few lines above.

######  Lemma 7.

For any $b\geq a\geq 0$, any $\alpha>0$, any
$f\in{\mathcal{P}}({{\mathbb{R}}^{3}})$,

$m_{a+\alpha}(f)m_{b}(f)\leq m_{b+\alpha}(f)m_{a}(f).$

###### Proof.

We fix $\alpha>0$ and $f\in{\mathcal{P}}({{\mathbb{R}}^{3}})$ and have to
prove that the function $u(a)=m_{a+\alpha}(f)/m_{a}(f)$ is nondecreasing on
${\mathbb{R}}_{+}$. Observing that $\frac{{\rm d}}{{\rm
d}a}m_{a}(f)=\int_{{\mathbb{R}}^{3}}(\log|v|)|v|^{a}f({\rm d}v)$, we find

$u^{\prime}(a)=\frac{1}{m_{a}(f)}\int_{{\mathbb{R}}^{3}}(\log|v|)|v|^{a+\alpha}f({\rm
d}v)-\frac{1}{m_{a}^{2}(f)}\Big{(}\int_{{\mathbb{R}}^{3}}(\log|v|)|v|^{a}f({\rm
d}v)\Big{)}\Big{(}\int_{{\mathbb{R}}^{3}}|v|^{a+\alpha}f({\rm d}v)\Big{)}.$

Setting $g_{a}({\rm d}v)=|v|^{a}f({\rm d}v)/m_{a}(f)$, which is a probability
measure,

$u^{\prime}(a)=\frac{1}{\alpha}\int_{{\mathbb{R}}^{3}}(\log|v|^{\alpha})|v|^{\alpha}g_{a}({\rm
d}v)-\frac{1}{\alpha}\Big{(}\int_{{\mathbb{R}}^{3}}(\log|v|^{\alpha})g_{a}({\rm
d}v)\Big{)}\Big{(}\int_{{\mathbb{R}}^{3}}|v|^{\alpha}g_{a}({\rm d}v)\Big{)}.$

Hence $u^{\prime}(a)\geq 0$ by the Jensen inequality. ∎

We will also use the following remark.

######  Remark 8.

For $f\in{\mathcal{P}}({{\mathbb{R}}^{3}})$ such that $m_{2}(f)=1$, for all
$r\geq s\geq 2$, by Hölder’s inequality,

$\displaystyle m_{s}(f)=\int_{{\mathbb{R}}^{3}}|v|^{s-2}|v|^{2}f({\rm
d}v)\leq\Big{(}\int_{{\mathbb{R}}^{3}}|v|^{r-2}|v|^{2}f({\rm
d}v)\Big{)}^{(s-2)/(r-2)}=[m_{r}(f)]^{(s-2)/(r-2)}.$

We finally quickly prove some moment estimates that are more or less well-
known.

######  Lemma 9.

Assume $(H_{1}(\gamma))$ and $(H_{2}(\nu))$ for some $\gamma\in(0,1]$ and some
$\nu\in(0,2)$. (i) For all $r>0$, there is $K_{r}\in(0,\infty)$, depending
only on $\gamma,\nu,\kappa_{1},\kappa_{2},r$, such that for all $t>0$,

$m_{r}(t)\leq K_{r}(1+t^{-(r-2)/\gamma}).$

(ii) For all integer $n\geq 2$, all $A\geq 1$, there is
$K_{2n,A}\in(0,\infty)$, depending only on
$\gamma,\nu,\kappa_{1},\kappa_{2},n,A$, such that

$m_{2n}(0)\leq A\quad\Longrightarrow\quad\sup_{t\geq 0}m_{2n}(t)\leq
K_{2n,A}.$

###### Proof.

We first prove that for any fixed integer $n\geq 2$, there is a constant
$A_{n}$, depending only on $\gamma,\nu,\kappa_{1},\kappa_{2},n$, allowed to
vary from line to line, such that for all $t>0$,

(10)
$m_{2n}^{\prime}(t)\leq-\frac{c_{1}}{2}[m_{2n}(t)]^{1+\gamma/(2n-2)}+A_{n}.$

Using Lemma 6, a rough upperbound, and then Lemma 7, we get

$m_{2n}^{\prime}(t)\\!\leq\\!-c_{1}m_{2n+\gamma}(t)+A_{n}\\!\sum_{a=1}^{\lfloor
n/2\rfloor}\\!m_{2a}(t)m_{2(n-a)+\gamma}(t)+A_{n}\\!\leq\\!-c_{1}m_{2n+\gamma}(t)+A_{n}m_{2(n-1)+\gamma}m_{2}(t)+A_{n}.$

Using now Remark 8 and that $m_{2}(t)=1$, we find

$\displaystyle m_{2n}^{\prime}(t)\leq-
c_{1}[m_{2n}(t)]^{1+\gamma/(2n-2)}+A_{n}[m_{2n}(t)]^{1-(2-\gamma)/(2n-2)}+A_{n},$

from which (10) follows. Point (ii) clearly follows from (6). When $r=2n\geq
4$ is an even integer, point (i) also follows from (6). For a general $r>2$,
we consider an integer $n_{r}\geq 2$ such that $2n_{r}\geq r$ and we conclude
from Remark 8 that

$m_{r}(t)\leq[m_{2n_{r}}(t)]^{(r-2)/(2n_{r}-2)}\leq(K_{2n_{r}}(1+t^{-(2n_{r}-2)/\gamma}))^{(r-2)/(2n_{r}-2)},$

whence the result. Finally, for $r\in(0,2]$, we obviously have
$m_{r}(t)\leq[m_{2}(t)]^{r/2}\leq 1$. ∎

##  4\. Control of exponential moments by even integer moments

We explain how to control exponential moments from even integer moments and
vice versa.

######  Lemma 10.

(i) For $f\in{\mathcal{P}}({{\mathbb{R}}^{3}})$, for
$\sigma_{0}\in(0,\infty)$, $\alpha\in[1,\infty)$ and $K\in[1,\infty)$,

$\sup_{n\geq 0}\frac{\sigma_{0}^{n}m_{2n}(f)}{(n!)^{\alpha}}\leq
K\quad\Longrightarrow\quad\int_{{\mathbb{R}}^{3}}\exp(\sigma_{0}^{1/\alpha}|v|^{2/\alpha}/2)f({\rm
d}v)\leq 2K^{1/\alpha}.$

(ii) For $\rho\in(0,2]$, $\sigma_{0}\in(0,1]$ and $K\in(1,\infty)$, there is
$\sigma_{1}\in(0,\infty)$, depending only on $\rho,\sigma_{0},K$, such that
for all $f\in{\mathcal{P}}({{\mathbb{R}}^{3}})$,

$\int_{{\mathbb{R}}^{3}}\exp(\sigma_{0}|v|^{\rho})f({\rm d}v)\leq
K\quad\Longrightarrow\quad\sup_{n\geq
0}\frac{\sigma_{1}^{n}m_{2n}(f)}{(n!)^{2/\rho}}\leq 1.$

###### Proof.

We start with (i). By Hölder’s inequality, since $\alpha\geq 1$,

$m_{2n/\alpha}(f)\leq[m_{2n}(f)]^{1/\alpha}\leq
K^{1/\alpha}\sigma_{0}^{-n/\alpha}n!,$

whence

$\int_{{\mathbb{R}}^{3}}\exp[\sigma_{0}^{1/\alpha}|v|^{2/\alpha}/2]f({\rm
d}v)=\sum_{n\geq 0}\frac{\sigma_{0}^{n/\alpha}m_{2n/\alpha}(f)}{2^{n}n!}\leq
K^{1/\alpha}\sum_{n\geq 0}2^{-n}=2K^{1/\alpha}.$

We now check (ii). We know that

$\sup_{n\geq 0}\frac{\sigma_{0}^{n}m_{\rho n}(f)}{n!}\leq\sum_{n\geq
0}\frac{\sigma_{0}^{n}m_{\rho
n}(f)}{n!}=\int_{{\mathbb{R}}^{3}}\exp(\sigma_{0}|v|^{\rho})f({\rm d}v)\leq
K.$

For $n\geq 1$, we set $k_{n}=\lceil 2n/\rho\rceil\in[2n/\rho,2n/\rho+1)$ and
write, using that $\rho k_{n}\geq 2n$,

$m_{2n}(f)\leq 1+m_{\rho k_{n}}(f)\leq
1+\frac{Kk_{n}!}{\sigma_{0}^{k_{n}}},\quad\hbox{so
that}\quad\frac{\sigma_{1}^{n}m_{2n}(f)}{(n!)^{2/\rho}}\leq\frac{\sigma_{1}^{n}}{(n!)^{2/\rho}}+\frac{\sigma_{1}^{n}Kk_{n}!}{\sigma_{0}^{k_{n}}(n!)^{2/\rho}}.$

By Stirling’s formula $n!\sim\sqrt{2\pi n}(n/e)^{n}$ and since $k_{n}=\lceil
2n/\rho\rceil$, we find that, for some constant $A\in(0,\infty)$, depending
only on $\rho$ and allowed to vary, for all $n\geq 1$,

$\frac{k_{n}!}{(n!)^{2/\rho}}\\!\leq\\!A\frac{n^{1/2}[(2n/\rho+1)/e]^{2n/\rho+1}}{n^{1/\rho}[n/e]^{2n/\rho}}\\!\leq\\!An^{3/2-1/\rho}\frac{[(3n)/(e\rho)]^{2n/\rho}}{[n/e]^{2n/\rho}}\\!\leq\\!An^{3/2}\Big{(}\frac{3}{\rho}\Big{)}^{2n/\rho}\\!\\!\\!\leq
A\Big{(}\frac{4}{\rho}\Big{)}^{2n/\rho}.$

Observing next that $\sigma_{0}^{k_{n}}\geq\sigma_{0}^{2n/\rho+1}$ (since
$\sigma_{0}\in(0,1]$), we end with

$\hbox{for all $n\geq
1$,}\quad\frac{\sigma_{1}^{n}m_{2n}(f)}{(n!)^{2/\rho}}\leq\frac{\sigma_{1}^{n}}{(n!)^{2/\rho}}+\frac{KA}{\sigma_{0}}\frac{\sigma_{1}^{n}}{\sigma_{0}^{2n/\rho}}\Big{(}\frac{4}{\rho}\Big{)}^{2n/\rho}.$

This last quantity is bounded by $1$ if $\sigma_{1}>0$ is small enough
(depending on $\rho$, $K$, and $\sigma_{0}$). ∎

##  5\. Creation of exponential moments

The following estimate will allow us to prove the creation result by Lemma 10.

######  Lemma 11.

Assume $(H_{1}(\gamma))$ and $(H_{2}(\nu))$ for some $\gamma\in(0,1]$ and some
$\nu\in(0,2)$ and set $\alpha=\max\\{1,(2-\nu)/\gamma\\}$. There are
$\sigma\in(0,1]$ and $T>0$, depending only on
$\gamma,\nu,\kappa_{1},\kappa_{2}$, such that

$\sup_{t\in[0,T]}\sum_{n=0}^{\infty}\frac{(\sigma
t)^{2n/\gamma}m_{2n}(t)}{(n!)^{\alpha}}\leq 2.$

###### Proof.

We recall that by Lemma 6, for any integer $n\geq 2$, any $t>0$,

(11) $m_{2n}^{\prime}(t)\leq-
c_{1}n^{\nu/2}m_{2n+\gamma}(t)+c_{2}S_{n}(t)+2c_{1}n^{\nu/2}2^{2n/\gamma},$

where $S_{n}(t)=\sum_{a=1}^{\lfloor
n/2\rfloor}\binom{n}{a}\frac{n^{\nu/2}}{a^{\nu/2+1}}m_{2a}(t)m_{2(n-a)+\gamma}(t)$.

Step 1. We introduce, for $\sigma\in(0,1]$ to be chosen later, for $p\geq 2$
and $t\geq 0$,

$E_{p}(t)=\sum_{n=0}^{p}\frac{(\sigma
t)^{2n/\gamma}m_{2n}(t)}{(n!)^{\alpha}}.$

By Lemma 9-(i) and since $m_{0}(t)=1$, it holds that for some constant
$C_{p}\in(0,\infty)$,

$1\leq E_{p}(t)\leq 1+C_{p}\sum_{n=1}^{p}t^{2n/\gamma}m_{2n}(t)\leq
1+C_{p}\sum_{n=1}^{p}t^{2n/\gamma}[1+t^{-(2n-2)/\gamma}]=1+C_{p}\sum_{n=1}^{p}[t^{2n/\gamma}+t^{2/\gamma}],$

whence $\lim_{t\to 0}E_{p}(t)=1$.

Step 2. Since $m_{0}^{\prime}(t)=m_{2}^{\prime}(t)=0$, we deduce from (11)
that for all $p\geq 2$, all $t\in[0,1]$,

$E_{p}^{\prime}(t)\leq-
c_{1}F_{p}(t)+c_{2}G_{p}(t)+\frac{2\sigma}{\gamma}H_{p}(t)+C,$

where

$\displaystyle F_{p}(t)=\sum_{n=2}^{p}n^{\nu/2}\frac{(\sigma
t)^{2n/\gamma}m_{2n+\gamma}(t)}{(n!)^{\alpha}},\quad
G_{p}(t)=\sum_{n=2}^{p}\frac{(\sigma
t)^{2n/\gamma}S_{n}(t)}{(n!)^{\alpha}},\quad
H_{p}(t)=\sum_{n=1}^{p}\frac{n(\sigma
t)^{2n/\gamma-1}m_{2n}(t)}{(n!)^{\alpha}}$

and where, since $\sigma t\leq 1$,

$C=2c_{1}\sum_{n=2}^{\infty}\frac{n^{\nu/2}2^{2n/\gamma}}{(n!)^{\alpha}}<\infty.$

Step 3. We first prove that for all $\epsilon\in(0,\infty)$, there is
$A_{\epsilon}\in(0,\infty)$, depending only on
$\gamma,\nu,\kappa_{1},\kappa_{2},\epsilon$, such that for any choice of
$\sigma\in(0,1]$, for all $p\geq 2$, all $t\in[0,1]$,

$G_{p}(t)\leq\epsilon
E_{p}(t)F_{p}(t)+\sigma^{2/\gamma}A_{\epsilon}(F_{p}(t)+1).$

We start from

$\displaystyle G_{p}(t)=$ $\displaystyle\sum_{n=2}^{p}\sum_{a=1}^{\lfloor
n/2\rfloor}\frac{(\sigma
t)^{2n/\gamma}}{(n!)^{\alpha}}\frac{n^{\nu/2}}{a^{\nu/2+1}}\binom{n}{a}m_{2a}(t)m_{2(n-a)+\gamma}(t).$

Since $\alpha\geq 1$ and since $a\leq\lfloor n/2\rfloor$ implies $n\leq
2(n-a)$ and thus $n^{\nu/2}\leq 2^{\nu/2}(n-a)^{\nu/2}$,

$\displaystyle G_{p}(t)\leq$ $\displaystyle
2^{\nu/2}\sum_{n=2}^{p}\sum_{a=1}^{\lfloor n/2\rfloor}\frac{(\sigma
t)^{2n/\gamma}}{(n!)^{\alpha}}\frac{(n-a)^{\nu/2}}{a^{\nu/2+1}}\Big{[}\binom{n}{a}\Big{]}^{\alpha}m_{2a}(t)m_{2(n-a)+\gamma}(t)$
$\displaystyle=$ $\displaystyle 2^{\nu/2}\sum_{n=2}^{p}\sum_{a=1}^{\lfloor
n/2\rfloor}(n-a)^{\nu/2}\frac{(\sigma
t)^{2(n-a)/\gamma}m_{2(n-a)+\gamma}(t)}{((n-a)!)^{\alpha}}\frac{(\sigma
t)^{2a/\gamma}m_{2a}(t)}{(a!)^{\alpha}a^{\nu/2+1}}$ $\displaystyle=$
$\displaystyle 2^{\nu/2}\sum_{a=1}^{\lfloor
p/2\rfloor}\sum_{n=2a}^{p}(n-a)^{\nu/2}\frac{(\sigma
t)^{2(n-a)/\gamma}m_{2(n-a)+\gamma}(t)}{((n-a)!)^{\alpha}}\frac{(\sigma
t)^{2a/\gamma}m_{2a}(t)}{(a!)^{\alpha}a^{\nu/2+1}}.$

Using the change of indices $n\to\ell=n-a$

$\displaystyle G_{p}(t)\leq$ $\displaystyle 2^{\nu/2}\sum_{a=1}^{\lfloor
p/2\rfloor}\sum_{\ell=a}^{p-a}\ell^{\nu/2}\frac{(\sigma
t)^{2\ell/\gamma}m_{2\ell+\gamma}(t)}{(\ell!)^{\alpha}}\frac{(\sigma
t)^{2a/\gamma}m_{2a}(t)}{(a!)^{\alpha}a^{\nu/2+1}}$ $\displaystyle\leq$
$\displaystyle\Big{(}\sum_{\ell=1}^{p}\ell^{\nu/2}\frac{(\sigma
t)^{2\ell/\gamma}m_{2\ell+\gamma}(t)}{(\ell!)^{\alpha}}\Big{)}\times
2^{\nu/2}\Big{(}\sum_{a=1}^{p}\frac{(\sigma
t)^{2a/\gamma}m_{2a}(t)}{(a!)^{\alpha}a^{\nu/2+1}}\Big{)}$ $\displaystyle=$
$\displaystyle\Big{(}F_{p}(t)+(\sigma
t)^{2/\gamma}m_{2+\gamma}(t)\Big{)}I_{p}(t),$

where

$\displaystyle I_{p}(t)=2^{\nu/2}\sum_{a=1}^{p}\frac{(\sigma
t)^{2a/\gamma}m_{2a}(t)}{(a!)^{\alpha}a^{\nu/2+1}}.$

But setting $N_{\epsilon}=\lceil 2^{\nu/(\nu+2)}\epsilon^{-2/(\nu+2)}\rceil$,
it holds that $a\geq N_{\epsilon}$ implies
$2^{\nu/2}a^{-\nu/2-1}\leq\epsilon$, whence

$I_{p}(t)\leq\epsilon E_{p}(t)+J_{\epsilon}(t),\quad\hbox{where}\quad
J_{\epsilon}(t)=2^{\nu/2}\sum_{a=1}^{N_{\epsilon}}\frac{(\sigma
t)^{2a/\gamma}m_{2a}(t)}{(a!)^{\alpha}}.$

By Lemma 9-(i), we see that for some constants $A,A_{\epsilon}\in(0,\infty)$,
depending only on $\gamma,\nu,\kappa_{1},\kappa_{2},\epsilon$ and allowed to
vary, for any choice of $\sigma\in(0,1]$, for all $t\in[0,1]$,

$(\sigma t)^{2/\gamma}m_{2+\gamma}(t)\leq A(\sigma t)^{2/\gamma}(1+t^{-1})\leq
A\sigma^{2/\gamma}\;\;\hbox{and}\;\;J_{\epsilon}(t)\leq
A_{\epsilon}\sum_{a=1}^{N_{\epsilon}}(\sigma
t)^{2a/\gamma}[1+t^{-(2a-2)/\gamma}]\leq A_{\epsilon}\sigma^{2/\gamma}.$

All in all, we have proved that for any choice of $\sigma\in(0,1]$, for all
$t\in[0,1]$,

$G_{p}(t)\leq(F_{p}(t)+A\sigma^{2/\gamma})(\epsilon
E_{p}(t)+A_{\epsilon}\sigma^{2/\gamma})\leq\epsilon
E_{p}(t)F_{p}(t)+\sigma^{2/\gamma}[\epsilon
AE_{p}(t)+A_{\epsilon}F_{p}(t)+AA_{\epsilon}].$

Since $E_{p}(t)\leq m_{0}(t)+\sigma tm_{2}(t)+F_{p}(t)\leq 2+F_{p}(t)$, we
conclude that, as desired,

$G_{p}(t)\leq\epsilon E_{p}(t)F_{p}(t)+\sigma^{2/\gamma}[(\epsilon
A+A_{\epsilon})F_{p}(t)+AA_{\epsilon}+2\epsilon A].$

Step 4. We now verify that there are some constants $\kappa,B\in(0,\infty)$,
depending only on $\gamma,\nu,\kappa_{1},\kappa_{2}$, such that for any choice
of $\sigma\in(0,1]$, for all $p\geq 2$, all $t\in[0,1]$,

$H_{p}(t)\leq\kappa F_{p}(t)+B.$

We first observe that, for $\kappa>0$ to be chosen later,

$\frac{nm_{2n}(t)}{\sigma t}\leq\kappa
n^{\nu/2}[m_{2n}(t)]^{1+\gamma/(2n-2)}+\frac{n}{\sigma
t}\Big{(}\frac{n^{1-{\nu/2}}}{\kappa\sigma t}\Big{)}^{(2n-2)/\gamma}.$

Indeed, $nm_{2n}(t)/(\sigma t)$ is bounded by the second term if
$m_{2n}(t)\leq[n^{1-{\nu/2}}/(\kappa\sigma t)]^{(2n-2)/\gamma}$ and by the
first term else. Since $[m_{2n}(t)]^{1+\gamma/(2n-2)}\leq m_{2n+\gamma}(t)$ by
Remark 8, we conclude that

$\displaystyle{}H_{p}(t)=$
$\displaystyle\sum_{n=1}^{p}\frac{nm_{2n}(t)}{\sigma t}\frac{(\sigma
t)^{2n/\gamma}}{(n!)^{\alpha}}$ $\displaystyle\leq$
$\displaystyle\kappa\sum_{n=1}^{p}n^{\nu/2}\frac{(\sigma
t)^{2n/\gamma}m_{2n+\gamma}(t)}{(n!)^{\alpha}}+\sum_{n=1}^{p}\frac{n}{\sigma
t}\Big{(}\frac{n^{1-{\nu/2}}}{\kappa\sigma
t}\Big{)}^{(2n-2)/\gamma}\frac{(\sigma t)^{2n/\gamma}}{(n!)^{\alpha}}$
$\displaystyle=$ $\displaystyle\kappa F_{p}(t)+\kappa(\sigma
t)^{2/\gamma}m_{2+\gamma}(t)+(\sigma
t)^{2/\gamma-1}\sum_{n=1}^{p}\frac{n^{(2-\nu)n/\gamma-(2-\nu)/\gamma+1}}{(n!)^{\alpha}\kappa^{(2n-2)/\gamma}}.$

But by Lemma 9-(i), there is $A\in(0,\infty)$, depending only on
$\gamma,\nu,\kappa_{1},\kappa_{2}$, such that for all $t\in[0,1]$,

$t^{2/\gamma}m_{2+\gamma}(t)\leq At^{2/\gamma}[1+t^{-1}]\leq 2A.$

Moreover, since $\alpha=\max\\{1,(2-\nu)/\gamma\\}\geq(2-\nu)/\gamma$, the
series

$S:=\sum_{n=1}^{\infty}\frac{n^{(2-\nu)n/\gamma-(2-\nu)/\gamma+1}}{(n!)^{\alpha}\kappa^{(2n-2)/\gamma}}\leq\kappa^{2/\gamma}\sum_{n=1}^{\infty}\frac{n^{\alpha
n+1}}{(n!)^{\alpha}\kappa^{2n/\gamma}}$

is convergent if $\kappa=2e^{\alpha\gamma/2}$, by Stirling’s formula
$n!\sim\sqrt{2\pi n}(n/e)^{n}$. Hence for any choice of $\sigma\in(0,1]$, for
all $t\in[0,1]$,

$H_{p}(t)\leq\kappa F_{p}(t)+2\kappa A+S.$

Step 5. By Steps 2-3-4, for any choice of $\epsilon\in(0,\infty)$ and
$\sigma\in(0,1]$, for all $p\geq 2$, all $t\in(0,1]$,

$\displaystyle E_{p}^{\prime}(t)\leq$ $\displaystyle-
c_{1}F_{p}(t)+c_{2}G_{p}(t)+\frac{2\sigma}{\gamma}H_{p}(t)+C$
$\displaystyle\leq$ $\displaystyle-c_{1}F_{p}(t)+c_{2}[\epsilon
E_{p}(t)F_{p}(t)+\sigma^{2/\gamma}A_{\epsilon}(F_{p}(t)+1)]+\frac{2\sigma}{\gamma}[\kappa
F_{p}(t)+B]+C.$

Choosing $\epsilon=c_{1}/(4c_{2})$ and $\sigma\in(0,1]$ small enough so that
$c_{2}\sigma^{2/\gamma}A_{\epsilon}+2\kappa\sigma/\gamma\leq c_{1}/2$, we find

$E_{p}^{\prime}(t)\leq-\frac{c_{1}}{2}F_{p}(t)+\frac{c_{1}}{4}E_{p}(t)F_{p}(t)+D,$

for some constant $D\in(0,\infty)$. By Step 1, we know that
$E_{p}(0)=\lim_{t\to 0}E_{p}(t)=1$, whence

$T_{p}=\sup\\{t>0:E_{p}(t)\leq 2\\}>0.$

For $p\geq 2$ and $t\in[0,T_{p}\land 1)$, we have $E_{p}^{\prime}(t)\leq D$.
Recalling that $E_{p}(0)=1$, we deduce that $T_{p}\geq\min\\{1,1/D\\}=:T.$ In
other words, with our choice of $\sigma$, all $t\in[0,T]$, $\sup_{p\geq
2}E_{p}(t)\leq 2$. ∎

We can finally give the

###### Proof of Theorem 2-(i).

We assume $(H_{1}(\gamma))$ and $(H_{2}(\nu))$ for some $\gamma\in(0,1]$ and
some $\nu\in(0,2)$. We fix $\rho=\min\\{2\gamma/(2-\nu),2\\}$ and observe that
$\alpha:=2/\rho=\max\\{1,(2-\nu)/\gamma\\}$. Thus by Lemma 11, there are
$\sigma\in(0,1]$ and $T>0$, depending only on
$\gamma,\nu,\kappa_{1},\kappa_{2}$, such that

$\sup_{[0,T]}\sum_{n=0}^{\infty}\frac{(\sigma
t)^{2n/\gamma}m_{2n}(t)}{(n!)^{\alpha}}\leq 2.$

We deduce from Lemma 10-(i) (with $\sigma_{0}=(\sigma t)^{2/\gamma}$) that

$\sup_{[0,T]}\int_{{\mathbb{R}}^{3}}\exp[(\sigma
t)^{\rho/\gamma}|v|^{\rho}/2]f_{t}({\rm
d}v)=\sup_{[0,T]}\int_{{\mathbb{R}}^{3}}\exp[(\sigma
t)^{2/(\gamma\alpha)}|v|^{2/\alpha}/2]f_{t}({\rm d}v)\leq 2^{1+1/\alpha}\leq
4.$

as desired. ∎

##  6\. Propagation of exponential moments

We proceed as in the previous section.

######  Lemma 12.

Assume $(H_{1}(\gamma))$ and $(H_{2}(\gamma))$ for some $\gamma\in(0,1]$ and
some $\nu\in(0,2)$. For any $\sigma_{0}\in(0,\infty)$ and any $\alpha\geq 1$,
there is $\sigma\in(0,\sigma_{0}]$, depending only on
$\gamma,\nu,\kappa_{1},\kappa_{2},\sigma_{0}$, such that

$\displaystyle\sup_{n\geq 0}\frac{\sigma_{0}^{n}m_{2n}(0)}{(n!)^{\alpha}}\leq
1\quad\Longrightarrow\quad\sup_{t\geq
0}\sum_{n=0}^{\infty}\frac{\sigma^{n}m_{2n}(t)}{(n!)^{\alpha}}\leq 3.$

###### Proof.

We fix $\alpha\geq 1$ and $\sigma_{0}\in(0,\infty)$ and assume that
$\sup_{n\geq 0}(n!)^{-\alpha}\sigma_{0}^{n}m_{2n}(0)\leq 1$.

Step 1. We introduce, for $\sigma\in(0,1]$ to be chosen later, for $p\geq 2$
and $t\geq 0$,

$E_{p}(t)=\sum_{n=0}^{p}\frac{\sigma^{n}m_{2n}(t)}{(n!)^{\alpha}}.$

If $\sigma\in(0,\sigma_{0}/2]$, $E_{p}(0)\leq 2$ for all $p\geq 2$, because

$E_{p}(0)\leq 1+\sum_{n\geq
1}\frac{(\sigma_{0}/2)^{n}m_{2n}(0)}{(n!)^{\alpha}}\leq 1+\sum_{n\geq
1}2^{-n}=2.$

Step 2. By Lemma 6, since $m_{0}^{\prime}(t)=m_{2}^{\prime}(t)=0$ and since
$\sigma\in(0,1]$,

$E_{p}^{\prime}(t)\leq-c_{1}F_{p}(t)+c_{2}G_{p}(t)+C,$

where

$\displaystyle
F_{p}(t)=\sum_{n=2}^{p}n^{\nu/2}\frac{\sigma^{n}m_{2n+\gamma}(t)}{(n!)^{\alpha}},\qquad
G_{p}(t)=\sum_{n=2}^{p}\frac{\sigma^{n}S_{n}(t)}{(n!)^{\alpha}},\qquad
C=2c_{1}\sum_{n=2}^{\infty}\frac{n^{\nu/2}2^{2n/\gamma}}{(n!)^{\alpha}}.$

Step 3. We prove that for all $\epsilon\in(0,\infty)$, there is a constant
$A_{\epsilon}\in(0,\infty)$, depending only on
$\epsilon,\gamma,\nu,\kappa_{1},\kappa_{2},\alpha,\sigma_{0}$, such that for
any choice of $\sigma\in(0,1]$, for all $p\geq 2$, all $t\geq 0$,

$G_{p}(t)\leq\epsilon E_{p}(t)F_{p}(t)+\sigma A_{\epsilon}(F_{p}(t)+1).$

Exactly as in the previous section, Step 3, we have

$\displaystyle G_{p}(t)\leq$ $\displaystyle\Big{(}F_{p}(t)+\sigma
m_{2+\gamma}(t)\Big{)}\Big{(}\epsilon E_{p}(t)+J_{\epsilon}(t)\Big{)},$

where, setting $N_{\epsilon}=\lceil
2^{\nu/(\nu+2)}\epsilon^{-2/(\nu+2)}\rceil$,

$\displaystyle
J_{\epsilon}(t)=2^{\nu/2}\sum_{a=1}^{N_{\epsilon}}\frac{\sigma^{a}m_{2a}(t)}{(a!)^{\alpha}},$

For all $n\geq 2$, $m_{2n}(0)\leq\sigma_{0}^{-n}(n!)^{\alpha}$, so that by
Lemma 9-(ii), there are some constants $A,A_{\epsilon}\in(0,\infty)$,
depending only on
$\epsilon,\gamma,\nu,\kappa_{1},\kappa_{2},\alpha,\sigma_{0}$ and allowed to
vary, such that for all $t\geq 0$,

$m_{2+\gamma}(t)\leq[m_{4}(t)]^{(2+\gamma)/4}\leq A\quad\hbox{and}\quad
J_{\epsilon}(t)\leq A_{\epsilon}\sum_{a=1}^{N_{\epsilon}}\sigma^{a}\leq
A_{\epsilon}\sigma.$

All in all, we have proved that for any choice of $\sigma\in(0,1]$, for all
$t\geq 0$,

$G_{p}(t)\leq(F_{p}(t)+A\sigma)(\epsilon
E_{p}(t)+A_{\epsilon}\sigma)\leq\epsilon E_{p}(t)F_{p}(t)+\sigma[\epsilon
AE_{p}(t)+A_{\epsilon}F_{p}(t)+AA_{\epsilon}].$

The conclusion follows since $E_{p}(t)\leq m_{0}(t)+\sigma
m_{2}(t)+F_{p}(t)\leq 2+F_{p}(t)$.

Step 4. We now prove that for all $p\geq 2$, all $t\geq 0$,

$F_{p}(t)\geq\sigma^{-\gamma/2}(E_{p}(t)-e).$

We write

$\displaystyle F_{p}(t)=$
$\displaystyle\sum_{n=2}^{p}n^{\nu/2}\frac{\sigma^{n}m_{2n+\gamma}(t)}{(n!)^{\alpha}}\geq\frac{1}{\sigma^{\gamma/2}}\sum_{n=2}^{p}\frac{\sigma^{n+\gamma/2}m_{2n+\gamma}(t)}{(n!)^{\alpha}}\geq\frac{1}{\sigma^{\gamma/2}}\sum_{n=2}^{p}\frac{\sigma^{n}m_{2n}(t)-1}{(n!)^{\alpha}},$

because

$\sigma^{n+\gamma/2}m_{2n+\gamma}(t)=\int_{{\mathbb{R}}^{3}}(\sigma|v|^{2})^{n+\gamma/2}f_{t}({\rm
d}v)\geq\int_{{\mathbb{R}}^{3}}[(\sigma|v|^{2})^{n}-1]f_{t}({\rm
d}v)=\sigma^{n}m_{2n}(t)-1.$

Hence, since $m_{0}(t)=m_{2}(t)=1$, since $\sigma\in(0,1]$ and since
$\alpha\geq 1$,

$F_{p}(t)\geq\frac{1}{\sigma^{\gamma/2}}\Big{[}E_{p}(t)-m_{0}(t)-\sigma
m_{2}(t)-\sum_{n=2}^{p}\frac{1}{(n!)^{\alpha}}\Big{]}\geq\frac{1}{\sigma^{\gamma/2}}\Big{[}E_{p}(t)-\sum_{n=0}^{p}\frac{1}{(n!)^{\alpha}}\Big{]}\geq\frac{1}{\sigma^{\gamma/2}}[E_{p}(t)-e].$

Step 5. By Steps 2-3-4, for any choice of $\epsilon\in(0,\infty)$ and
$\sigma\in(0,1]$, for all $p\geq 2$, all $t\geq 0$,

$\displaystyle E_{p}^{\prime}(t)\leq$ $\displaystyle-
c_{1}F_{p}(t)+c_{2}G_{p}(t)+C\leq-c_{1}F_{p}(t)+c_{2}[\epsilon
E_{p}(t)F_{p}(t)+\sigma A_{\epsilon}(F_{p}(t)+1)]+C.$

Choosing $\epsilon=c_{1}/(16c_{2})$ and then $\sigma_{1}\in(0,1]$ small enough
so that $c_{2}\sigma_{1}A_{\epsilon}\leq c_{1}/2$, we conclude that for some
constant $D\in(0,\infty)$, for any choice of $\sigma\in(0,\sigma_{1}]$, any
$t\geq 0$,

$E_{p}^{\prime}(t)\leq-\frac{c_{1}}{2}F_{p}(t)+\frac{c_{1}}{16}E_{p}(t)F_{p}(t)+D.$

We now recall from Step 1 that for $\sigma\in(0,\sigma_{0}/2]$, we have
$E_{p}(0)\leq 2$ and by continuity,

$T_{p}=\sup\\{t\geq 0:E_{p}(t)\leq 4\\}>0.$

But for all $t\in[0,T_{p})$, if $\sigma\in(0,(\sigma_{0}/2)\land\sigma_{1}]$,

$E_{p}^{\prime}(t)\leq-\frac{c_{1}}{4}F_{p}(t)+D\leq-\frac{c_{1}}{4\sigma^{\gamma/2}}[E_{p}(t)-e]+D$

by Step 4. Since $E_{p}(0)\leq 2$, this implies that if
$\sigma\in(0,(\sigma_{0}/2)\land\sigma_{1}]$, for all $p\geq 2$ and all
$t\in[0,T_{p})$,

$E_{p}(t)\leq e+\frac{4\sigma^{\gamma/2}D}{c_{1}}.$

Choosing $\sigma\in(0,(\sigma_{0}/2)\land\sigma_{1}]$ small enough so that
${4\sigma^{\gamma/2}D}/{c_{1}}\leq 3-e$, we conclude that for all $p\geq 2$,
$E_{p}(t)\leq 3$ for all $t\in[0,T_{p})$, whence $T_{p}=\infty$ by continuity.
In other words, for all $p\geq 2$, all $t\geq 0$, $E_{p}(t)\leq 3$, which was
our goal. ∎

We finally can give the

###### Proof of Theorem 2-(ii).

We assume $(H_{1}(\gamma))$ and $(H_{2}(\nu))$ for some $\gamma\in(0,1]$ and
some $\nu\in(0,2)$. We fix $A>1$, $\rho\in(0,2]$ and $\sigma_{0}>0$ and assume
that $\int_{{\mathbb{R}}^{3}}\exp(\sigma_{0}|v|^{\rho})f_{0}({\rm d}v)\leq A$.

We set $\alpha=2/\rho\geq 1$. By Lemma 10-(ii), for some
$\sigma_{1}\in(0,\infty)$, depending only on $\rho,\sigma_{0},A$,

$\displaystyle\sup_{n\geq 0}\frac{\sigma_{1}^{n}m_{2n}(0)}{(n!)^{\alpha}}\leq
1.$

We thus may apply Lemma 12: there is $\sigma_{2}\in(0,\sigma_{1}]$, depending
only on $\gamma,\nu,\kappa_{1},\kappa_{2},\sigma_{1}$, such that

$\sup_{t\geq
0}\sum_{n=0}^{\infty}\frac{\sigma_{2}^{n}m_{2n}(t)}{(n!)^{\alpha}}\leq 3.$

We deduce from Lemma 10-(i) that, setting
$\sigma_{3}=\sigma_{2}^{1/\alpha}/2$,

$\sup_{t\geq 0}\int_{{\mathbb{R}}^{3}}\exp[\sigma_{3}|v|^{\rho}]f_{t}({\rm
d}v)=\sup_{t\geq
0}\int_{{\mathbb{R}}^{3}}\exp[\sigma_{2}^{1/\alpha}|v|^{2/\alpha}/2]f_{t}({\rm
d}v)\leq 2.3^{1/\alpha}\leq 6$

as desired. ∎

## References

  * [1] R. Alexandre, A review of Boltzmann equation with singular kernels, Kinet. Relat. Models 2 (2009), 551–646.
  * [2] R. Alonso, J.A. Cañizo, I. Gamba, C. Mouhot, A new approach to the creation and propagation of exponential moments in the Boltzmann equation, Comm. Partial Differential Equations 38 (2013), 155–169.
  * [3] R.J. Alonso, I.M. Gamba, M. Tasković, Exponentially-tailed regularity and time asymptotic for the homogeneous Boltzmann equation. arXiv:1711.06596
  * [4] A.V. Bobylev, Moment inequalities for the Boltzmann equation and applications to spatially homogeneous problems, J. Statist. Phys. 88 (1997), 1183–1214.
  * [5] A.V. Bobylev, I.M. Gamba, V.A. Panferov, Moment inequalities and high-energy tails for Boltzmann equations with inelastic interactions, J. Statist. Phys. 116 (2004), 1651–1682.
  * [6] C. Cercignani, The Boltzmann equation and its applications. Applied Mathematical Sciences, 67. Springer-Verlag, New York, 1988.
  * [7] N. Fournier, D. Heydecker, Stability, well-posedness and regularity of the homogeneous Landau equation for hard potentials, arXiv:2009.12940.
  * [8] N. Fournier, S. Méléard, A stochastic particle numerical method for 3D Boltzmann equations without cutoff, Math. Comp. 71 (2002), 583–604.
  * [9] N. Fournier, C. Mouhot, On the well-posedness of the spatially homogeneous Boltzmann equation with a moderate angular singularity, Comm. Math. Phys. 289 (2009), 803–824.
  * [10] D. Heydecker, Kac’s Process with Hard Potentials and a Moderate Angular Singularity, arXiv:2008.12943.
  * [11] X. Lu, C. Mouhot, On measure solutions of the Boltzmann equation, part I: moment production and stability estimates, J. Differential Equations 252 (2012), 3305–3363.
  * [12] C. Mouhot, Rate of convergence to equilibrium for the spatially homogeneous Boltzmann equation with hard potentials, Comm. Math. Phys. 261 (2006), 629–672.
  * [13] A.J. Povzner, About the Boltzmann equation in kinetic gas theory, Mat. Sborn, 58 (1962), 65–86.
  * [14] M. Tasković, R.J. Alonso, I.M. Gamba, N. Pavlović, On Mittag-Leffler moments for the Boltzmann equation for hard potentials without cutoff, SIAM J. Math. Anal. 50 (2018), 834–869.
  * [15] C. Villani, A review of mathematical topics in collisional kinetic theory, Handbook of mathematical fluid dynamics, Vol. I, 71–305, North-Holland, Amsterdam, 2002\.
  * [16] B. Wennberg, Entropy dissipation and moment production for the Boltzmann equation, J. Statist. Phys. 86 (1997), 1053–1066.


11institutetext: University of Chieti-Pescara, Italy 22institutetext: ISTI-
CNR, Pisa, Italy 33institutetext: University of Parma, Italy

# GloNets: Globally Connected Neural Networks

Antonio Di Cecco 11 0000-0002-9070-4663 Carlo Metta 22 0000-0002-9325-8232
Marco Fantozzi 33 0000-0002-0708-5495 Francesco Morandin 33
0000-0002-2022-2300 Maurizio Parton 11 0000-0003-4905-3544

###### Abstract

Deep learning architectures suffer from depth-related performance degradation,
limiting the effective depth of neural networks. Approaches like ResNet are
able to mitigate this, but they do not completely eliminate the problem. We
introduce Globally Connected Neural Networks (GloNet), a novel architecture
overcoming depth-related issues, designed to be superimposed on any model,
enhancing its depth without increasing complexity or reducing performance.
With GloNet, the network’s head uniformly receives information from all parts
of the network, regardless of their level of abstraction. This enables GloNet
to self-regulate information flow during training, reducing the influence of
less effective deeper layers, and allowing for stable training irrespective of
network depth. This paper details GloNet’s design, its theoretical basis, and
a comparison with existing similar architectures. Experiments show GloNet’s
self-regulation ability and resilience to depth-related learning challenges,
like performance degradation. Our findings suggest GloNet as a strong
alternative to traditional architectures like ResNets.

###### Keywords:

Neural Networks Deep Learning Skip Connections

##  1 Introduction

Deep learning’s success in AI is largely due to its hierarchical
representation of data, with initial layers learning simple features and
deeper ones learning more complex, nonlinear transformations of these features
[3]. Increasing depth should enhance learning, but sometimes it leads to
performance issues [4, 9]. Techniques like normalized initialization [15, 9,
21, 10] and normalization layers [14, 1] enable up to 30-layer deep networks,
but performance degradation persists at greater depths without skip
connections. This issue, detailed in the original ResNet paper [11], stems
from the fact that learning identity maps is not easy for a deeply nonlinear
layer. ResNet idea is to focus on learning nonlinear “residual” information,
with a backbone carrying the identity map. This brilliant solution has been
key in training extremely deep networks that, when weight-sharing and batch
normalization are used, can scale up to thousands of layers.

Deeper neural networks should not experience performance degradation.
Theoretically, a deeper network could match the performance of an $n$-layer
network by similarly learning features $\mathcal{G}_{1},\dots,\mathcal{G}_{n}$
in its initial layers, then minimizing the impact of additional layers. With
this ability to _self-regulate_ , such a network could effectively be
“infinitely deep”. However, even with ResNet architectures, performance
degradation persists beyond a certain depth, see Figure 5(b) or [7]. This
issue can be due to various factors, see for instance [9, 23, 17], and may
partly arise from the inability of modern architectures to self-regulate their
depth. Our paper introduces a novel technique to enable self-regulation in
neural network architectures, overcoming these depth-related performance
challenges.

Novel Contributions. The main contribution of this paper is introducing and
testing GloNet, an explainable-by-design layer that can be superimposed on any
neural network architecture, see Section 2. GloNet’s key feature is its
capacity to self-regulate information flow during training. It achieves this
by reducing the influence of the deepest layers to a negligible level, thereby
making the training more stable, preventing issues like vanishing gradients,
and making the network trainable irrespective of its depth, see Section 5.

This self-regulation capabilities of GloNet lead to several significant
benefits:

  1. 1.

Faster training: GloNet trains in half the ResNet time while achieving
comparable performance. Beyond the depth threshold where ResNet begins to
degrade, GloNet trains in less than half the time and outperforms ResNet.

  2. 2.

ResNet alternative: The inability of ResNet-based architectures to self-
regulate depth makes GloNet a preferable option, particularly for applications
requiring very deep architectures.

  3. 3.

No NAS needed: GloNet networks inherently find their effective depth,
eliminating the need for computationally expensive Network Architecture Search
methods to determine optimal network depth.

  4. 4.

More controllable efficiency/performance trade-off: Layers can be selectively
discarded to boost efficiency, allowing a controlled trade-off between
efficiency and performance, optimizing the network for specific requirements.

##  2 Notation and Model Definition

A feedforward neural network is described iteratively by a sequence of $L$
blocks:

$\textbf{x}_{\ell+1}=\mathcal{G}_{\ell}(\textbf{x}_{\ell}),\quad\ell=0,\dots,L-1,$
(1)

where $\textbf{x}_{0}$ denotes the input vector, and $\textbf{x}_{\ell+1}$ is
the output from the $\ell$-th block. In this context, a “block” is a modular
network unit, representing a broader concept than a traditional “layer”. Each
block function $\mathcal{G}_{\ell}$ typically merges a non-linearity, such as
ReLU, with an affine transformation, and may embody more complex structures,
like the residual blocks in ResNet.

At the end of the sequence (1), a classification or regression head
$\mathcal{H}$ is applied to $\textbf{x}_{L}$. For instance, a convolutional
architecture could use a head with average pooling and a fully connected
classifier. The fundamental principle in deep learning is that
$\mathcal{G}_{0},\dots,\mathcal{G}_{\ell-1}$ hierarchically extract meaningful
features from the input $\textbf{x}_{0}$, that can then be leveraged by
computing the output of the network:

$\text{output}=\mathcal{H}(\textbf{x}_{L}).$

When the blocks in (1) are simple layers like an affine map followed by a non-
linearity (this description comprises, for instance, fully connected and
convolutional neural networks), all features extracted at different depths are
exposed to the head by a single feature vector $\textbf{x}_{L}$ _that has gone
through several non-linearities_. This fact leads to several well-known
drawbacks, like vanishing gradients or difficulty in learning when the task
requires more direct access to low-level features. When using ReLU and shared
biases, some low-level information could actually be destroyed. Several
excellent solutions have been proposed to these drawbacks, like for instance
residual networks [11, 12], DenseNets [13], and preactivated units with non-
shared biases [17].

We propose an alternative solution: a modification to (1), consisting of a
simple layer between the feature-extraction sequence and the head, computing
the sum of every feature vector. The architecture is designed to receive
information uniformly from all parts of the network, regardless of their level
of abstraction:

$\begin{cases}\textbf{x}_{\ell+1}=\mathcal{G}_{\ell}(\textbf{x}_{\ell}),\quad\ell=0,\dots,L-1\\\
\textbf{x}_{L+1}=\sum_{\ell=1}^{L}\textbf{x}_{\ell}=\sum_{\ell=0}^{L-1}\mathcal{G}_{\ell}(\textbf{x}_{\ell})\\\
\text{output}=\mathcal{H}(\textbf{x}_{L+1})\end{cases}$ (2)

When feature vectors have different dimensions, adaptation to a common
dimension is required before the sum, as happens in ResNet. If
$\textbf{x}_{\ell}\in\mathbb{R}^{n_{\ell}}$, one can use embeddings in
$\mathbb{R}^{\max\\{n_{\ell}\\}}$ to maximally preserve information, and
embeddings or projections to $\mathbb{R}^{n_{L}}$ to maintain the same
parameters for the head. We refer to the additional layer in (2) as a _GloNet
layer_ , because all the features $\textbf{x}_{\ell}$ that without GloNet
would be preserved only “locally” up to the next
$\textbf{x}_{\ell+1}=\mathcal{G}_{\ell}(\textbf{x}_{\ell})$, appear now in the
“global” feature vector $\textbf{x}_{L+1}=\sum_{\ell=1}^{L}\textbf{x}_{\ell}$
as summands.

######  Remark 1 (Theoretical support for GloNet self-regulation)

GloNet provides skip connections solely to the head, and intermediate blocks
are not required to learn a residual map, see (2) and Figure 1. This ensures
direct _and simultaneous_ backpropagation pathways from each block, enabling
uniform information distribution across the network to the head. Due to SGD-
like training’s preference for shorter paths [25, 26, 16], _GloNet is expected
to accumulate information mostly in the initial blocks rather than the latter
ones_ , by reducing the influence of the deepest layers to a negligible level.
Consequently, GloNet self-regulates its depth during training, rendering it
akin to an “infinitely deep” architecture. Empirical evidence supporting this
claim is presented in Section 5.

Figure 1: On the left, a 3-blocks neural network followed by a classification
head. On the right, the same architecture with GloNet modification in red.

######  Remark 2 (Explainability-by-design)

Given the linearity of the GloNet layer, the global feature vector
$\textbf{x}_{L+1}$ provides the contribution that each layer makes to the
neural network’s prediction. In a feedforward neural network, GloNet enables
the analysis of an ensemble of networks represented by the outputs of each
network block. This ensemble, comprising blocks
$(\mathcal{G}_{\ell})_{\ell=0,\dots,k}$, functions as individual neural
networks, with GloNet integrating their outputs through a linear (currently
unweighted) combination. This architecture allows each sub-network to
specialize in learning features at varying levels of granularity, from low-
level in early blocks to more complex, large-scale features in later blocks.
The linear nature of the GloNet layer facilitates the attribution of
importance scores to these features, effectively creating an ’explainable-by-
design’ tool [8].

##  3 Related Work

In a ResNet with an activation-free backbone, also known as ResNetv2 [12],
blocks $\mathcal{G}_{\ell}$ are defined as $\text{Id}+\mathcal{F}_{\ell}$,
where Id is the skip connection and $\mathcal{F}_{\ell}$ is the “residual
block”, computing two times $\text{Conv}\circ\text{ReLU}\circ\text{BN}$, where
BN is batch normalization. Unrolling the ResNetv2 equation
$\textbf{x}_{\ell+1}=\textbf{x}_{\ell}+\mathcal{F}_{\ell}(\textbf{x}_{\ell})$
from any block output $\textbf{x}_{\ell}$ (see [12, Equation 4]) gives:

$\textbf{x}_{L}=\textbf{x}_{\ell}+\sum_{i=\ell}^{L-1}\mathcal{F}_{i}(\textbf{x}_{i})$
(3)

This equation shows that ResNet, much like GloNet, passes the output
$\textbf{x}_{\ell}$ of each block directly to the head. However, a key
distinction lies in how the head accesses these outputs: ResNet requires
distinct pathways for simultaneous access to different outputs, whereas
GloNet’s head achieves simultaneous access to each output via the GloNet
layer. This unique capability of GloNet may contribute to its additional
features compared to ResNetv2, as explored in Section 5.

Moreover, GloNet is faster than ResNet (not requiring batch normalization),
and can be seen as an ensemble computing the sum of models of increasing
complexity, giving an explainable-by-design model (differently from ResNet).

Unlike DenseNet [13], aggregating each block with all subsequent blocks
through concatenation, GloNet connects only to the last block, with summation
for aggregation. This approach avoids the parameter explosion given by
concatenation in DenseNet, and maintains the original complexity of the model.

Finally, GloNet can be viewed as a network with early exits at every block,
adapted and aggregated before the head. See [24, 18, 19] for the early exit
idea.

##  4 Implementing GloNet

Implementing GloNet within a certain architecture may not always be as
straightforward as described in Section 2. In this Section we describe how
GloNet can be implemented in common scenarios.

GloNet and Skip Connections. If the original architecture includes skip
connections (such as ResNet or DenseNet), these should be removed and replaced
with the GloNet connection. Otherwise, putting GloNet on top of skip
connections, training would not converge as we would be adding the identity to
the output multiple times. Note that GloNet provides only skip connections to
the GloNet layer, and does not ask the blocks to learn a residual map.

GloNet and Batch Normalization. Batch normalization plays a major role in
enhancing and stabilizing neural network training by normalizing the inputs of
each block. Its positive impact is widely acknowledged, though the specific
mechanisms of its benefits are still debated [14, 20]. Despite these
advantages, batch normalization poses challenges, particularly in its
interaction with GloNet. GloNet is designed to dynamically regulate the
outputs of different blocks, based on their contribution to the task. It makes
negligible the outputs of deeper blocks, a strategy that conflicts with the
objectives of batch normalization, which strives to maintain a consistent mean
and variance for block inputs. Consequently, batch normalization, and
similarly layer normalization, should be removed prior to the GloNet layer’s
aggregation. GloNet introduces an alternative form of regularization, which,
as we demonstrate in Section 5, is capable of achieving comparable
performances without the need for batch normalization.

GloNet into Residual Networks. Architectures using residual blocks feature
both skip connections and normalization. Once skip connections and
normalizations are removed from a ResNetv2 block computing
$\text{Id}+\text{affine map}\circ\text{ReLU}\circ\text{BN}\circ\text{affine
map}\circ\text{ReLU}\circ\text{BN}$, one is left with two simpler blocks
$\text{affine map}\circ\text{ReLU}$, and each of those blocks can potentially
be aggregated into the GloNet layer. In this case, one ResNetv2 block
corresponds to two simpler blocks. This is what we do in this paper, and for
this reason when GloNet has $n$ blocks, its equivalent ResNetv2 architecture
has $\nicefrac{{n}}{{2}}$ blocks.

GloNet into Vision Transformers. When using more complex architectures like
transformers, several different choices can be made, each one potentially
affecting the final performance of the GloNet-enhanced model. In this initial
exploration of GloNet, we propose a straightforward integration with a Vision
Transformer (ViT) [6] adapted to CIFAR-10. The image is segmented into 4x4
patches, its class encoded, and then concatenated with the patch encoding and
a positional embedding. This series is then fed into a cascade of $n$ encoders
with 4 attention heads each, which are accumulated into a GloNet layer, and
passed to a classification head. In our experiment, we compared $n=4,5$ and
$6$.

##  5 Experiments

In this section we provide experiments supporting the core claims of our
paper, as stated in the Introduction. In particular, we focus on showing that
GloNet trains much faster than ResNet, that GloNet performances are on par
with ResNet’s ones, that GloNet can self-regulate its depth, that GloNet does
not need batch normalization, and that GloNet is virtually immune to depth-
related problems. All the experiments can be reproduced using the source code
provided at [5].

SGEMM Fully Connected Regression. We experimented with a regression task from
the UCI repository [2, 22], focused on predicting the execution time of matrix
multiplication on an SGEMM GPU kernel. See [2, 22] for details on this task
and SGEMM dataset.

Since GloNet has skip connections, to obtain a fair comparison we used a
ResNetv2-like baseline. For comparison, we used also a vanilla baseline,
identical to the ResNetv2 baseline but without skip connections. Moreover,
since GloNet does not use batch normalization (in fact, GloNet self-regulation
capabilities can be tampered by normalization, see Section 4), we also
experimented with a vanilla and a ResNetv2 baseline with the BN layer removed.
GloNet and the corresponding baselines (denoted by vanilla, ResNetv2, vanilla-
no-BN, and ResNetv2-no-BN in figures) have a similar amount of parameters, the
only difference being given by the trainable BN parameters. All blocks have 16
units. All models starts with a linear layer mapping the 14-dimensional input
to $\mathbb{R}^{16}$, and ends with the head, a linear layer with 1 unit.
GloNet models have an additional GloNet layer before the head _with no
additional parameters_. The number of blocks ranges in [10, 24, 50, 100, 200],
respectively (halved for the ResNets because every block is twice the layers
of the corresponding non-ResNet model).

For training, we used MSE loss, L2-regularization with a coefficient of
$10^{-5}$ (we also tried $10^{-4}$ without improvements), Adam optimizer with
a batch size of 1024, learning rate set to $0.01$, He normal initializer for
weights, and zero initializer for biases. We trained all models for 200
epochs, the point at which baselines plateaued, potentially favoring them over
GloNet. The first thing to notice is that with GloNet _training takes almost
half or less than half the time of ResNet_ , see Table 1. This is because
GloNet, differently from ResNet, does not need batch normalization.

Table 1: Average epoch’s training time in seconds at different depths, for GloNet and its equivalent ResNetv2 baseline, on SGEMM regression task. architecture | depth
---|---
| 10 | 24 | 50 | 100 | 200 | 600 | 1000
ResNetv2 | 42 | 65 | 96 | 147 | 231 | 410 | 675
Glonet | 27 | 39 | 55 | 86 | 121 | 175 | 289

After 200 training epochs, we compared the best test errors and learning
curves across different block configurations, see Figure 2 for learning curves
and Table 2 for best test errors. At 200 blocks, GloNet surpassed ResNet in
both best test error and learning curve shape. See the caption of Figure 2 for
details on the results of this experiment.

Figure 2: Test errors while training for 200 epochs GloNet and four
corresponding baselines at different depths on SGEMM. Starting at 24 blocks,
GloNet and ResNetv2 outperform vanilla, as expected due to skip connections.
At 50 and 100 blocks, both ResNetv2 and GloNet show depth resilience, with
ResNetv2 slightly leading in best test error. ResNetv2-no-BN exhibits
significantly higher error, highlighting ResNetv2’s reliance on batch
normalization. At 100 blocks GloNet curve shows a markedly higher curvature
than ResNetv2. At 200 blocks, GloNet outperforms ResNetv2 in best test error
and curve shape.

The training shapes in Figure 2 suggested a unique aspect of GloNet not
present in the ResNetv2 baseline. GloNet’s training was unaffected by the
increasing depth, as shown by the shape of the learning curve that remained
consistent whether the network had 10, 24, 50, 100, or 200 blocks. On the
contrary, ResNet learning curve became flatter when depth is increasing.

To further explore this feature, GloNet was tested with even deeper models
(600 and 1000 blocks), and compared against the corresponding ResNetv2
baseline. Even at these substantial depths, GloNet’s learning curve maintained
its shape, as shown in Figure 3. Moreover, GloNet’s performance remained
stable across these varying depths, maintaining a best test error of around
$0.02$ regardless of the number of blocks ($10,24,50,100,200,600$, or 1000),
see Table 2.

Table 2: Best test errors across different network depths. architecture | depth
---|---
| 10 | 24 | 50 | 100 | 200 | 600 | 1000
vanilla | 0.023 | 0.030 | 1.997 | 2.750 | 2.985 | - | -
vanilla-no-BN | 0.018 | 0.021 | 2.624 | 2.624 | 2.624 | - | -
ResNetv2 | 0.018 | 0.019 | 0.020 | 0.019 | 0.027 | 0.040 | 0.048
ResNetv2-no-BN | 0.024 | 0.026 | 0.029 | 0.033 | 0.189 | - | -
Glonet | 0.021 | 0.021 | 0.020 | 0.022 | 0.021 | 0.022 | 0.022
Figure 3: Test MSE learning curves of GloNet (orange) with 600 blocks (left)
and 1000 blocks (right), against the equivalent ResNetv2 baseline (blue).
Comparing with Figure 2 shows these substantial depths severely degrade ResNet
performance, but do not affect at all GloNet performance.

In contrast, ResNet’s showed a clear decline as the network depth increased.
While its best test error remained around $0.02$ up to 200 blocks, this error
increased to $0.04$ and almost $0.05$ at 600 and 1000 blocks, respectively,
see again Table 2. As happened with 100 and 200 blocks, the ResNet learning
curve was flatter, diverging from GloNet’s more consistent curve shape as
depth increased.

GloNet accumulates information in the first few blocks and uses only the
required capacity for a specific task and architecture, leading to minimal
output from subsequent blocks, see Remark 1. This is not observed in baseline
models with or without batch normalization, see Figure 4, and likely
contributes to GloNet’s stable performance as network depth increases, in
contrast to the degradation observed in the baseline models under similar
depth conditions.

(a) vanilla-no-BN

(b) ResNetv2-no-BN

(c) ResNetv2

(d) GloNet

Figure 4: Block’s outputs L1 norm, 100 blocks, 200 epochs, SGEMM. Mean and
standard deviation from 5000 samples. GloNet (d) uses only the first 12
blocks, and achieves the least variance, indicating a more consistent learning
of features. In ResNetv2 (c) all blocks give a similar contribution, as
expected because of BN. ResNetv2-no-BN (b) uses all outputs in different ways,
because of the residual maps mixed with the skip connections. The baseline
vanilla-no-BN (a) shows the opposite behavior to GloNet, emphasizing late
outputs and diminishing earlier ones, indicating shorter gradient paths to the
last blocks.

MNIST Fully Connected Classification. To confirm that GloNet automatic choice
of optimal depth and GloNet training resilience to depth were not associated
to the particular SGEMM regression task, we performed a series of experiments
with identical architecture on a completely different task: image
classification with MNIST. Although using fully connected architectures for
image classification is generally not the best approach, with this task we
have been able to significantly increase the number of input features, which
theoretically could pose a greater challenge to models that are not very deep.

The only difference from the architecture used in SGEMM is the head, which in
this case is a fully connected layer followed by a SoftMax layer on 10
classes. We tested architectures with $6,10,24,50,60,80,100$, and $200$ blocks
for GloNet and a convolutional baseline, halved for the corresponding ResNet
baseline. Figure 5(a) shows that GloNet automatically chooses the optimal
number of blocks. However, notice that in this case, differently from Figure
4c, ResNetv2 outputs show a decreasing shape. This is probably due to the
trainable parameters of the batch normalization, that in this case are able to
force a small mean and variance on the last blocks. This indicates that also
with batch normalization the network struggles to self-regulate its depth.
Figure 5(b) confirms GloNet resilience to an increasing depth, and shows a
severe performance degradation of ResNetv2 when depth goes above 50 blocks.

(a)

(b)

Figure 5: (a) Block’s outputs L1 norm, GloNet (orange), and ResNetv2 (blue),
50 blocks, 200 epochs, MNIST. Mean and standard deviation from 5000 samples.
GloNet uses only the first 4 blocks. In ResNetv2, all blocks give a
significant contribution, as expected because of BN. (b) Accuracies of GloNet
(orange), ResNetv2 (blue), and vanilla (green), 200 epochs, MNIST, against
$6,10,24,50,60,80,100$, and $200$ blocks. GloNet is consistent across depths,
while vanilla and ResNetv2 show a rapid decline with increased depth.

CIFAR10 Convolutional Classification. We further experimented with a ResNet20
on CIFAR10. ResNet20 is a ResNetv2 with 3 stages of 3 residual blocks each,
described in [11]. We compared a ResNet20 architecture with its GloNet
version, obtained by removing the backbone and adding a GloNet layer before
the classification head, as detailed in Section 4. We trained for 200 epochs.
Learning curves are completely overlapping, with best test errors $91.12\%$
and $91.08\%$ for ResNet and GloNet respectively. This experiment shows that
also with convolutional architectures, GloNet performs on par with the
traditional ResNet architecture, despite taking half the time for training.

GloNet for Vision Transformer Classification. A Visual Transformer (ViT) is a
transformer applied to sequences of feature vectors extracted from image
patches [6]. The $n$ encoders outputs can be accumulated into a GloNet layer
before going to the classification head, see Section 4 for details. In this
experiment we compare ViT, with and without GloNet, on CIFAR-10, with $4,5$
and $6$ encoders. Training plateaus at around 500 epochs, and final accuracies
align with those from literature. With 4 and 6 encoders, accuracies overlap
for ViT and GloNet-ViT. With 5 encoders, GloNet-ViT appears to improve over
ViT, see Figure 6(b). ViT best accuracies are $0.707$, $0.709$, and $0.725$,
and GloNet-ViT best accuracies are $0.709$, $0.727$, and $0.729$, for 4, 5,
and 6 encoders, respectively. This is a proof-of-concept experiment showcasing
the robustness and versatility of GloNet for complex architectures like
transformers.

Controllable Efficiency/Performance Trade-Off. In an experiment to demonstrate
how GloNet can be used to choose an optimal efficiency/performance trade-off,
we trained a 50-block GloNet fully connected architecture on MNIST for 200
epochs. After training, we progressively removed the last block, adjusted
accordingly the GloNet layer to sum fewer blocks, and evaluated the shallower
model without retraining. As shown in Figure 6(a), removing up to 42 blocks
did not significantly impact accuracy, illustrating GloNet’s ability to
balance efficiency and performance.

(a)

(b)

Figure 6: (a) Block’s outputs L1 norm of GloNet (red), smaller networks
performance (blue), 50 blocks, 200 epochs, MNIST. Performance plateaus after
block eight. (b) Accuracy of ViT (blue), and GloNet-ViT (orange) with $5$
encoders. Best accuracies $0.709$, $0.727$ for ViT, GloNet-ViT, respectively.

##  6 Conclusions

We introduce GloNet, a method designed to augment existing architectures
without adding complexity or reducing performance. It effectively renders the
architecture resilient to depth-related learning issues. As an alternative to
ResNet, GloNet offers advantages, without any disadvantage: it achieves
similar training outcomes in nearly half the time at depths where ResNet
remains stable, and maintains consistent performance at greater depths where
ResNet falters.

## References

  * [1] Ba, L.J., Kiros, J.R., Hinton, G.E.: Layer normalization. CoRR abs/1607.06450 (2016), http://arxiv.org/abs/1607.06450
  * [2] Ballester-Ripoll, R., Paredes, E.G., Pajarola, R.: Sobol tensor trains for global sensitivity analysis. Reliability Engineering & System Safety 183, 311–322 (2019), https://www.sciencedirect.com/science/article/pii/S0951832018303132
  * [3] Bengio, Y., Courville, A., Vincent, P.: Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013). https://doi.org/10.1109/TPAMI.2013.50
  * [4] Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks 5(2), 157–166 (1994)
  * [5] Di Cecco, A.: GloNet, https://github.com/AntonioDiCecco/GloNet
  * [6] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: ICLR (2021), https://openreview.net/forum?id=YicbFdNTTy
  * [7] Ebrahimi, M.S., Abadi, H.K.: Study of residual networks for image recognition. Lecture Notes in Networks and Systems, vol. 284, pp. 754–763. Springer (2021). https://doi.org/10.1007/978-3-030-80126-7_53
  * [8] Gianfagna, L., Di Cecco, A.: Explainable AI with python. Springer (2021)
  * [9] Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: AISTATS. JMLR, vol. 9, pp. 249–256 (2010)
  * [10] He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: ICCV. pp. 1026–1034 (2015). https://doi.org/10.1109/ICCV.2015.123
  * [11] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE CVPR. pp. 770–778 (2016)
  * [12] He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Computer Vision - ECCV 2016 - 14th European Conference Proceedings, Part IV. Lecture Notes in Computer Science, vol. 9908, pp. 630–645. Springer (2016). https://doi.org/10.1007/978-3-319-46493-0_38
  * [13] Huang, G., Liu, Z., Maaten, L.V.D., Weinberger, K.Q.: Densely connected convolutional networks. In: CVPR. pp. 2261–2269 (2017). https://doi.org/10.1109/CVPR.2017.243
  * [14] Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: ICML. vol. 37, p. 448–456 (2015)
  * [15] LeCun, Y., Bottou, L., Orr, G.B., Müller, K.: Efficient backprop. In: Neural Networks: Tricks of the Trade - Second Edition, Lecture Notes in Computer Science, vol. 7700, pp. 9–48 (2012). https://doi.org/10.1007/978-3-642-35289-8_3
  * [16] Martin, C.H., Mahoney, M.W.: Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning. J. Mach. Learn. Res. 22, 165:1–165:73 (2021), http://jmlr.org/papers/v22/20-410.html
  * [17] Metta, C., Fantozzi, M., Papini, A., Amato, G., Bergamaschi, M., Galfrè, S.G., Marchetti, A., Vegliò, M., Parton, M., Morandin, F.: Increasing biases can be more efficient than increasing weights - extended version. In: IEEE/CVF, WACV (2024), https://arxiv.org/abs/2301.00924
  * [18] Panda, P., Sengupta, A., Roy, K.: Conditional deep learning for energy-efficient and enhanced pattern recognition. In: Proceedings of DATE. p. 475–480 (2016)
  * [19] Rajesh, G.: A benchmark repository of Early Exit Neural Networks in .onnx format. https://github.com/gorakraj/earlyexit_onnx (2021)
  * [20] Santurkar, S., Tsipras, D., Ilyas, A., Madry, A.: How does batch normalization help optimization? In: NeurIPS 31. pp. 2488–2498 (2018), https://proceedings.neurips.cc/paper/2018/hash/905056c1ac1dad141560467e0a99e1cf-Abstract.html
  * [21] Saxe, A.M., McClelland, J.L., Ganguli, S.: Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. In: ICLR (2014), http://arxiv.org/abs/1312.6120
  * [22] SGEMM: SGEMM GPU Kernel Performance. https://archive.ics.uci.edu/ml/datasets/SGEMM+GPU+kernel+performance (2018), accessed: 2023-05-03
  * [23] Srivastava, R.K., Greff, K., Schmidhuber, J.: Training very deep networks. In: NeurIPS. pp. 2377–2385 (2015), https://proceedings.neurips.cc/paper/2015/hash/215a71a12769b056c3c32e7299f1c5ed-Abstract.html
  * [24] Teerapittayanon, S., McDanel, B., Kung, H.T.: Branchynet: Fast inference via early exiting from deep neural networks. In: ICPR. pp. 2464–2469 (2016). https://doi.org/10.1109/ICPR.2016.7900006
  * [25] Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning requires rethinking generalization. In: ICLR (2017), https://openreview.net/forum?id=Sy8gdB9xx
  * [26] Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning (still) requires rethinking generalization. Commun. ACM 64(3), 107–115 (2021). https://doi.org/10.1145/3446776