# Higher moments of distribution of zeta zeros

Farzad Aryan

###### Abstract.

We develop a method for mean-value estimation of long Dirichlet polynomials.
For an application, we use our method to study properties of the logarithmic
derivative of the Riemann zeta function.

##  1\. Introduction.

In this paper we introduce a method to bound the moments of a long Dirichlet
polynomial. Let

$\mathcal{B}(s)=\sum\frac{b(n)}{n^{s}},$

be of length larger than $T^{1+\tau},$ for $\tau>0$. We are generally
interested to estimate

$\int\omega(\tfrac{1}{2}+it)|\mathcal{B}(\tfrac{1}{2}+it)|^{k},$ (1.1)

where $\omega$ is a smooth function that is supported on $t\sim T$. Throughout
the paper we take

$\omega(\tfrac{1}{2}+it)=\frac{\log T}{\sqrt{\pi}T}\hskip
2.84526pt\displaystyle{e^{-\frac{(t-T)^{2}\log^{2}T}{T^{2}}}}.$ (1.2)

Many important problems in analytic number theory, can be transform into
estimating mean-values like (1.1). In case that we do not have a clear
understanding of

$\sum_{n<x}b(n)b(n+h),$

evaluating (1.1) turns into a difficult task.

We use decomposition of $\mathcal{B}(s)$ and some ideas from the probability
theory to get lower bounds on some families of interesting sequences. Our
first corollary is an example of mean-value estimates that our method can
handle. Up to the author’s knowledge, this is the first result that provides a
lower bound on mean-value of Dirichlet polynomials involving Liouvill’s
function.

######  Corollary 1.1.

Let $d(n)$ be the divisor function and $\lambda(n)$ be Liouvill’s function.
Define

$b(n):=-\frac{2}{\log T}\sum_{\begin{subarray}{c}n=mp^{j}\\\ m<T\log^{-2}T\\\
p^{j}<T\end{subarray}}\log p\big{(}1-\frac{\log p^{j}}{\log
T}\big{)}\lambda(m)d(m).$

If all of the divisor of $n$ are smaller than $T,$ then $b(n)$ approximately
equals to $\lambda(n)d(n)\frac{\log n}{\log T}.$ We have that

$\int\omega(\tfrac{1}{2}+it)\bigg{|}\displaystyle{\sum_{n\geq\frac{T}{\log^{2}T}}}\hskip
5.69054pt\displaystyle{\frac{b(n)}{n^{\tfrac{{}_{1}}{2}+it}}}\bigg{|}^{2}dt>0.009A_{2}\log^{4}T,$

where $A_{2}$ is a constant.

Finding an asymptotic for the above requires [9] assuming very strong bound on
the shifted convolution sum of $\lambda(n)d(n)$ which falls more into the
premises of Sarnak conjecture [15] with power savings. Our lower bound is of
the correct order of magnitude, with what we get if we assume an strong
version of Sarnak conjecture. Therefore our result, can be viewed as an
evidence to support the conjecture.

Matomäki and Radziwiłł [13] proved an upper bound for a similar mean-values
with $b(n)=\lambda(n).$ Their work resulted in many interesting applications
including answering the Erdos discrepancy problem by Tao.

The paper is organized as follows:

First we explain the mollifier method that is often used to study properties
of $L$-functions. Next we state our corollaries to show the improvements we
obtain over this method. In section 2 we state our main theorem. In section 3
we explain how the method can be generally apply to obtain lower bound on
(1.1). In section 4 we explain the connection of our result to the Landau-
Siegel zero problem. In Section 5 we give example of random variables that
their moments are related to moments we are seeking to estimate. In the last
section we give proofs of our results.

###  1.1. The mollifier method

Using mollifiers to study analytic properties of $L$-functions is a common
tool in number theory. Generally speaking if we would like to show that an
$L$-function has a certain property, we can use the following recipe:

  1. (1)

First, using a Dirichlet polynomial $\displaystyle{A(s)=\sum a(n)n^{-s}}$,
that highlights that property, we build a probability measure in the following
way. Define,

$\mu_{A}((a,b]):=\frac{\int_{a}^{b}\omega(\frac{{}_{1}}{2}+it)\big{|}A(\frac{{}_{1}}{2}+it)\big{|}^{2}}{\int\omega(\frac{{}_{1}}{2}+it)\big{|}A(\frac{{}_{1}}{2}+it)\big{|}^{2}}.$
(1.3)

The polynomial $A$ is commonly referred to as a mollifier. If we set $A(s)=1,$
then $\mu_{1}$ gives the probability measure we get out of Lebesgue
integration.

  2. (2)

Next we consider a series expansion of the $L$-function, denoted by
$\mathcal{L}$, and estimate the first moment with respect to $\mu_{A}$:

$\int\mathcal{L}d\mu_{A}.$ (1.4)

The result is an indicator of the degree that the $L$-function satisfy the
property. For example proving (1.4) is bigger than $\overline{\mathcal{L}}$
shows that there exist $t$ such that
$|L(\frac{{}_{1}}{2}+it)|>\overline{\mathcal{L}}.$ Going forward, we refer to
the above as the mollifier method.

This method has been applied in [14, 16] to the problem of large values of the
Riemann zeta function111In the context of large values of Riemann zeta
function the method is called the resonance method. and in [2] to the problem
of gaps between zeta zeros.

In this paper we show how to extract more information out of the mollifier
method, by engaging in higher moments estimation. Our results are an indicator
of improvement we can obtain. Let us consider an smooth series expansion of
the logarithmic derivative of zeta
($\displaystyle{\tfrac{\zeta^{\prime}}{\zeta}}(s))$:

$\mathcal{Z}_{\alpha}(s):=\frac{-2}{\alpha\log
T}\sum_{n<T^{\alpha}}\frac{\Lambda(n)}{n^{s}}\Big{(}1-\frac{\log n}{\alpha\log
T}\Big{)},$ (1.5)

we write

$\mathcal{Z}_{\alpha}(\tfrac{1}{2}+it)=\emph{\emph{C}}_{\alpha}(t)+i\emph{\emph{Im}}_{\alpha}(t).$
(1.6)

Understanding the behaviour of $\mathcal{Z}_{\alpha}$ has significant
implications. For example, showing that there exist a $t\in\mathbb{R}$ such
that

$\emph{\emph{C}}_{1}(t)>1\text{ or }\emph{\emph{C}}_{2}(t)>0.5,$

proves that there are no Landau-Siegel zeros.222For a full discussion on the
connection with Landau-Siegel zeros see the introduction and Lemma 8.1 in [2].
Assuming the Riemann hypothesis (RH) $\emph{\emph{C}}_{\alpha}(t)$ is large
when $t$ is close to a zero cluster and it is approximately
$-\tfrac{1}{\alpha}$ on large gaps (see (4.1)).

Let $\lambda$ be Liouvill’s function. In [2], using the Mollifer method we
proved that

$\frac{1}{\log T}\int\omega(\tfrac{1}{2}+it)\hskip
1.42262pt\emph{C}_{1}(t)\hskip 1.42262pt\bigg{|}\sum_{n<T\log^{-2}T}\hskip
5.69054pt\frac{\lambda(n)}{n^{\frac{{}_{1}}{2}+it}}\bigg{|}^{2}dt=\frac{2}{3}+O(\tfrac{1}{\log
T}).$ (1.7)

For simplicity, and by using the notation in (1.3) we put
$\mu_{\lambda}$333This can be viewed as an approximation of
$\zeta(2s)/\zeta(s)$. Bulk of $\mu_{\lambda}$ is distributed in the close
vicinity of zeta zeros. See section 7 in [2]. in place of the mollifier

$\omega(\tfrac{1}{2}+it)\frac{1}{\log T}|\sum\lambda(n)n^{-{1}/{2}-it}|^{2}.$

Using this we can write (1.7) in the following way:

$\int\emph{\emph{C}}_{1}(t)d\mu_{\lambda}=\tfrac{2}{3}+O(\tfrac{1}{\log T}).$
(1.8)

We proceed by applying our method to study the higher moments of
$\emph{\emph{C}}_{\alpha}$ and $\emph{\emph{Im}}_{\alpha}$. A heuristic given
in [6] suggest that $\mathcal{Z}_{\alpha}$ should behave like a sum of
independent random variables since the primes are multiplicatively
independent. Therefore it is expected that value distribution of objects like
$\mathcal{Z}_{\alpha}$ behave like normal distribution. When $\alpha<1/k$ we
can estimate $k$-th moment with respect to the Lebesgue measures. For
$\alpha\geq 1,$ estimating higher moments with respect to the Lebesgue measure
is out of reach, let alone twisted higher moments with $\mu_{A}$. As our
second corollary we show:

######  Corollary 1.2.

By the same notation as (1.6) and letting $\mu_{\lambda}$ be as (1.3) with
$a(n)=\lambda(n)$, we have that

$\displaystyle\int{\emph{C}}^{2}_{1}(t)d\mu_{\lambda}\geq 0.46666,$ (1.9)
$\displaystyle\int{\emph{C}}^{2}_{2}(t)d\mu_{\lambda}\geq 0.174998.$ (1.10)

To see the improvement upon the mollifier method, note that using the
corollary we can show that there exist $t\in\text{ Supp}(\mu_{\lambda})$ such
that

$|\emph{\emph{C}}_{1}(t)|>0.6831,$

which obviously is an improvement over $2/3\simeq 0.666$ that we get from
(1.8). Only applying (1.8) and the fact that the variance of a real random
variable is greater than its mean square, gives us the lower bound $4/9\simeq
0.444,$ for the RHS of (1.9).

The imaginary part of $\mathcal{Z}_{\alpha}$ is more mysterious than its real
part due to its oscillatory nature. Therefore, proving results similar to
(1.7) or Corollary 1.2 is much trickier. For example, we have
$\int\omega(\tfrac{1}{2}+it)\emph{\emph{Im}}_{\alpha}(t)d\mu_{\lambda}\sim 0$,
which gives nothing but the trivial bound on the variance of
$\emph{\emph{Im}}_{\alpha}$. In terms of large values of the imaginary part we
can apply Landau-Gonek formula [11]. For $\alpha<1$, and letting $N(T)$ denote
the number of zeta zeros with height between $0$ and $T,$ we obtain

$\frac{1}{N(T)}\sum_{0<\gamma<T}\displaystyle{\emph{\emph{Im}}_{1-\epsilon}}(\gamma+\tfrac{2\pi
d}{\log T})\sim\int_{0}^{1}2u(1-u)\sin(2\pi ud)du.$ (1.11)

The integral has the maximum $\simeq 0.27$ at the shift $d\simeq 0.4147.$
Therefore, average value of ${\emph{\emph{Im}}_{1-\epsilon}},$ shifted
slightly to the right of zeros, is about $0.27.$

This observation suggest that the $\emph{\emph{Im}}_{\alpha}$ has large values
when we shift slightly to the right or left of a zero. In the next section we
discuss the importance of $\emph{\emph{Im}}_{\alpha}$ and we state our results
on the large values of this function.

###  1.2.  The imaginary part of the logarithmic derivative of $\zeta$

Studying the imaginary part of $\mathcal{Z}_{\alpha}$ is an important subject
in part, due to its connection to the error term in the formula for the number
of zeros of the Riemann zeta function.

Let $S(t)$ be the deviation from the expected value in the zeta zeros counting
function, which is formally defined as

$S(t)=\frac{1}{\pi}\arg\zeta(\tfrac{1}{2}+it).$

A famous theorem of Selberg states that on average the value of $S(t)$ is
related to the imaginary part of the logarithmic derivative of zeta:

######  Theorem 1.3 (Selberg).

For $k\geq 1,$ we have

$\int_{0}^{T}\Big{|}S(t)+\frac{1}{\pi}\hskip
0.7113pt\displaystyle{\Im\sum_{n<T^{1/k}}\hskip
5.69054pt\frac{\Lambda(n)}{n^{\tfrac{{}_{1}}{2}+it}\small{\log
n}}}\Big{|}^{k}\ll_{k}T.$ (1.12)

Using the above, he went on to show that all even moments of $S(t)$ match the
moments of the Gaussian distribution, which confirm the speculation that
$S(t)$ should behave like a sum of independent RV.

Similar to Corollary 1.2 we give a lower bounds on variance of
$\emph{\emph{Im}}_{\alpha}$ with respect to $\mu_{\lambda}.$ Also, we will
introduce a surprisingly interesting measure that significantly improve the
bounds we get from $\mu_{\lambda}$ on the large values of
$\emph{\emph{Im}}_{\alpha}$.

######  Corollary 1.4.

By the same notation as (1.6), we have that

$\displaystyle\int{\emph{Im}_{1}^{2}}(t)\hskip 1.42262ptd\mu_{\lambda}\geq
0.1,$ (1.13)
$\displaystyle\int{\emph{Im}_{1}^{2}}(t)\emph{C}_{1}^{2}(t)d\mu_{\lambda}\geq
0.02763\cdots,$ (1.14)

######  Remark 1.

An important feature of Corollaries 1.2 and 1.4 is that asymptotically
estimating (1.13) and (1.14) requires off-diagonal treatments that is far from
the reach of our current techniques. At minimum it requires a strong version
of the Chowla conjecture plus a power saving in shifted convolution of von
Mangoldt and Liouville’s function.

In section 4, we will show that the imaginary part will also play a roll in
the problem of gaps between zeta zeros.

###  1.3. An interesting arithmetic function & large values of
$\emph{\emph{Im}}_{\alpha}$

We define a generalization of the Liouville’s function as follows:

Let $n=p^{\alpha_{1}}_{1}\cdots p^{\alpha_{m}}_{m},$ and define

$\lambda_{k}(n)=(-1)^{[\frac{\Omega(n)}{k}]},$ (1.15)

where $\Omega(n)=\alpha_{1}+\cdots+\alpha_{k},$ and $[\cdot]$ is the floor
function.

For $k=1$, $\lambda_{1}$ is the Liouville’s $\lambda$-function.
For $k=2$, we have $\lambda_{2}(1)=\lambda_{2}(p_{1})=1,\hskip
5.69054pt\lambda_{2}(p_{1}p_{2})=\lambda_{2}(p_{1}p_{2}p_{3})=-1\hskip
5.69054pt$ …

To our knowledge no connection has been made between

$L(s,\lambda_{2}):=\sum_{n=1}^{\infty}\frac{\lambda_{2}(n)}{n^{s}},\hskip
14.22636pt\Re(s)>1,$ (1.16)

and the zeros of the Riemann zeta function. Here we make an interesting link
between $L(s,\lambda_{2}),$ and large values of $\emph{\emph{Im}}_{\alpha}$
and gaps between zeta zeros. We will show

  1. (1)

The mollifier $\mu_{\lambda_{2}}$ that we get by truncating $L(s,\lambda_{2})$
for $n<T\log^{-2}T$ correlates highly with large values of
$\emph{\emph{Im}}_{\alpha}.$

  2. (2)

Support of $\mu_{\lambda_{2}}$ is expected to be on mid size gaps, and away
from close vicinity of zeros.

To establish these facts more precisely, first we put forward this corollary:

######  Corollary 1.5.

Let $\mu_{\lambda}$ be as (1.3) with $a(n)=\lambda(n)$ and define
$\mu_{\zeta}$ similarly by setting $a(n)=1,$ for $n<n<T\log^{-2}T$. We have
that

$\displaystyle\int\big{(}\emph{\emph{C}}^{2}_{1}(t)-{\emph{Im}}^{2}_{1}(t)\big{)}d\mu_{\lambda}=\int\big{(}\emph{\emph{C}}^{2}_{1}(t)-{\emph{Im}}^{2}_{1}(t)\big{)}d\mu_{\zeta}\sim
0.36666\cdots$ (1.17)

This corollary shows that $\emph{\emph{C}}_{1}^{2}$ is larger than
$\emph{\emph{Im}}_{1}^{2}$ on supports of both $\mu_{\lambda}$ and
$\mu_{\zeta}$. In [2] we showed that bulk of $\mu_{\lambda}$ is centred close
to zeros, and bulk of $\mu_{\zeta}$ is centred away from zeros and on large
gaps.444 A similar result for $\emph{\emph{C}}_{2}$ and $\emph{\emph{Im}}_{2}$
holds with the RHS of (1.17) replaced with $0.12708\cdots$.

Now we state the main corollary of this section:

######  Corollary 1.6.

Let $\lambda_{2}$ be as (1.15) and $\mu_{\lambda_{2}}$ be a measure we build
with $\lambda_{2}$ as in (1.3). We have that

$\int\emph{Im}^{2}_{1}(t)d\mu_{\lambda_{2}}\geq 0.46666.$

Moreover,

$\int\big{(}{\emph{Im}}^{2}_{1}(t)-\emph{\emph{C}}^{2}_{1}(t)\big{)}d\mu_{\lambda_{2}}=0.36666\cdots$

Similar to corollary 1.4 this shows that there exist a $t$ in the support of
$\mu_{\lambda_{2}}$ such that

$|\emph{\emph{Im}}_{1}(t)|>0.6831.$

Also, we have that the imaginary part is larger than the real part on the
support of $\mu_{\lambda_{2}}.$ This means that this measure is mainly
supported on region where $\mu_{\lambda}$ and $\mu_{\zeta}$ are almost void.
Therefore we expect that $\mu_{\lambda_{2}}$ be supported on mid size gaps,
and away from close vicinity of zeros.

As we mentioned before $\mu_{\lambda}$ is an approximation of the behaviour of
$|\tfrac{\zeta(2s)}{\zeta(s)}|^{2}$, and consequently it make sense that it is
mainly centred in a close vicinity of zeros. It is not clear how we can have a
similar explanation for $\mu_{\lambda_{2}}$. The alteration that occurs in the
definition of $\lambda_{2}$ is an arithmetic one. It would be interesting to
have an explanation as to how the analytic shift happens in the support of
$\mu_{\lambda_{2}}$.

##  2\. The Main Theorem

Our main theorem gives all of the pseudo moments of $\mathcal{Z}_{\alpha}$
with respect to variety of measures built in (1.3) with

$a(n)=\lambda_{\beta_{1},\beta_{2}}(n)d_{r}(n),$ (2.1)

where $d_{r}$ is the generalized divisor function and
$\lambda_{\beta_{1},\beta_{2}}$ is defined to be a completely multiplicative
function with $\lambda_{\beta_{1},\beta_{2}}(p)=-1$ for $T^{\beta_{1}}<p\leq
T^{\beta_{2}}$ and $\lambda_{\beta_{1},\beta_{2}}(p)=1$ otherwise.

######  Theorem 2.1.

Let $\mathcal{Z}_{\alpha}$ be as in (1.5) and
$\mu_{A_{\beta_{1},\beta_{2},r}}$ be the measure we built with
$a(n)=\lambda_{\beta_{1},\beta_{2}}(n)d_{r}(n)$ as in (1.3). We have

$\displaystyle\int$
$\displaystyle\mathcal{Z}^{k}_{\alpha}(\tfrac{1}{2}+it)d\mu_{A_{\beta_{1},\beta_{2},r}}+O(\frac{1}{\log
T})$ (2.2)
$\displaystyle=\big{(}\frac{-2r}{\alpha}\big{)}^{k}\frac{\small\displaystyle{\int_{0}^{1}\ldots\int_{0}^{\small
1-v_{2}\cdots-
v_{k}}\Lambda_{\beta_{1},\beta_{2}}(v_{1})\ldots\Lambda_{\beta_{1},\beta_{2}}(v_{k})(1-v_{1}\cdots-
v_{k})^{r^{2}}\small{d_{v_{1}}\ldots
d_{v_{k}}}}}{\int_{0}^{1}v^{r^{2}-1}(1-v)^{2\eta}dv}$

where

$\Lambda_{\beta_{1},\beta_{2}}(v)=\begin{cases}-\big{(}1-\tfrac{v}{\alpha}\big{)}\text{
for }\beta_{1}<u\leq\beta_{2}\\\ 1-\tfrac{v}{\alpha},\text{ otherwise.}\\\
\end{cases}$

For the special case $a(n)=\lambda(n)$ and $\alpha\geq 1$ we have

$\int\mathcal{Z}^{k}_{\alpha}(\tfrac{1}{2}+it)d\mu_{\lambda}(t)=\Big{(}\frac{2}{\alpha}\Big{)}^{k}\sum_{i=0}^{k}{k\choose
i}\big{(}\frac{-1}{\alpha}\big{)}^{i}\frac{1}{(k+i+1)!}+O(\frac{1}{\log T}).$
(2.3)

Moreover for $a(n)=\lambda_{2}(n)$, and even moments we have that

$\int\mathcal{Z}^{2k}_{\alpha}(\tfrac{1}{2}+it)d\mu_{\lambda_{2}}(t)=(-1)^{k}\int\mathcal{Z}^{2k}_{\alpha}(\tfrac{1}{2}+it)d\mu_{\lambda}(t).$
(2.4)

######  Remark 2.

An interesting feature of the theorem is that the $k$-th moment decays almost
like $1/(k+1)!$ times $(2/\alpha)^{k}.$ This is the property that helps us to
get the mentioned results on the imaginary part. It would be interesting to
categorize random variables whose moments satisfy this property. We will
discuss an example of this in Section 5.

Dirichlet polynomial $\mathcal{Z}_{\alpha}$ is a smooth approximation of
$\frac{\zeta^{\prime}}{\zeta}$ using the the triangular function
$(1-\tfrac{\cdot}{{\alpha\log T}})$ as a weight. Using this smooth weight is
not necessary for our results, but it is very useful for investigating the
distribution of zeros of the zeta function.

Let $\sigma=\tfrac{1}{2}+\tfrac{\beta}{\log T}$ where $\beta\gg 1.$ Assuming
the Riemann hypothesis Selberg proved

$\frac{1}{T}\int_{0}^{T}\Big{|}\frac{1}{\log
T}\frac{\zeta^{\prime}}{\zeta}(\sigma+it)\Big{|}^{2}dt\ll_{\beta}1.$ (2.5)

Later Goldston, Gonek and Montgomery[10] proved that getting an asymptotic for
(2.5) is equivalent to Montgomery’s pair correlation conjecture on the
distribution of the spacing between zeta zeros.

In the proof they use the simple fact that

$\Big{|}\frac{\zeta^{\prime}}{\zeta}(s)\Big{|}^{2}=2\Re\Big{(}\frac{\zeta^{\prime}}{\zeta}(s)\Big{)}^{2}-\Re\Big{(}\big{(}\frac{\zeta^{\prime}}{\zeta}(s)\big{)}^{2}\Big{)}.$
(2.6)

Then they integrate the LHS of (2.6) with respect to the Lebesgue measure.
Consequently the last term in RHS of the (2.6) disappears and only

$\Re\Big{(}\frac{\zeta^{\prime}}{\zeta}(s)\Big{)}^{2}$

plays a role. As we mentioned the real part is closely tied to the
distribution of zeros. Therefore, they used Montgomery’s study of the pair
correlation to get their results.

In our work by switching the measure from the Lebesgue, we take most advantage
of

$\Re\Big{(}\big{(}\frac{\zeta^{\prime}}{\zeta}(s)\big{)}^{2}\Big{)},$

in the RHS of (2.6), which does not contribute when we use the Lebesgue
measure. In addition, we show that the imaginary part also play a role in the
problem of gaps between zeta zeros.

We conclude this section with stating our result related to the mean-square of
$\frac{\zeta^{\prime}}{\zeta}$. In the next section we explain how the method
of this paper can generally be applied to obtain a lower bound on the mean-
square of general Dirichlet polynomials.

######  Corollary 2.2.

For $\alpha\geq 1,$ we have that

$\int\big{|}\mathcal{Z}_{\alpha}(\tfrac{1}{2}+it)\big{|}^{2}d\mu_{\lambda}\geq\frac{4}{3\alpha^{2}}-\frac{1}{\alpha^{3}}+\frac{7}{30\alpha^{4}}.$

##  3\. General Way of Applying our Method

Let $\mathcal{Z}=\mathcal{C}+iIm,$ be a complex random variable. If we can
determine all the moments of $|\mathcal{Z}|$ with respect to a probability
measure, i.e.

$\int|\mathcal{Z}|^{k}d\mu,$ (3.1)

then we can determine the distribution. The question is that how much
information we can get from pseudo moments of $\mathcal{Z},$ i.e.

$\mathcal{M}_{k}:=\int\mathcal{Z}^{k}d\mu?$ (3.2)

The mollifier method that we discussed in the introduction, in essence, is
looking at the above for $k=1.$

When we work with $L$-functions, $\mathcal{Z}$ can be an approximation of our
function and $\mu$ can be the appropriate probability measure built by a
Dirichlet polynomial. Usually if the first pseudo moment is calculable, then
the higher moments are as well. Our method works the best, and yield
improvements over the mollifer method when the $k$-th pseudo moment is
different than the first moment to the power of $k$.

To demonstrate this, suppose that all pseudo moments are real. Moreover,
assume

$E[\mathcal{Z}^{k}]=\mathcal{M}_{k}=(\mathcal{M}_{1})^{k}+\delta_{k},\textrm{
with }\hskip 2.84526pt|\delta_{k}|>0.$ (3.3)

Since we considered $\mu$ to be a probability measure, using the fact that the
variance of a real random variable is greater than its mean square, we have

$E[\Re(\mathcal{Z})^{2}]>(\mathcal{M}_{1})^{2}.$ (3.4)

Also, by using the assumption that moments are real and (3.3), we have

$\displaystyle
E[\Re(\mathcal{Z}^{2})]=E[(\Re\mathcal{Z})^{2}]-E[(\Im\mathcal{Z})^{2}]=(\mathcal{M}_{1})^{2}+\delta_{2}.$
(3.5)

If $\delta_{2}>0,$ then (3.5) immediately gives
$E[(\Re\mathcal{Z})^{2}]>(\mathcal{M}_{1})^{2}+\delta_{2},$ which obviously is
better than (3.4). If $\delta_{2}<0,$ then

$\displaystyle
E[(\Im\mathcal{Z})^{2}]=E[(\Re\mathcal{Z})^{2}]-(\mathcal{M}_{1})^{2}-\delta_{2}>|\delta_{2}|,$

Which gives a lower bound on the square of the imaginary part of $\mathcal{Z}$
and hence, a lower bound on $|\mathcal{Z}|^{2}:$

$\int|\mathcal{Z}|^{2}d\mu>\mathcal{M}^{2}_{1}+|\delta_{2}|.$

###  3.1. Mean-value of long Dirichlet polynomials

For application to number theory, imagine that we have a Dirichlet polynomials
of length larger than $T^{1+\tau},$ for $\tau>0$:

$\mathcal{B}(s)=\sum\frac{b(n)}{n^{s}},$

and we are interested in estimating

$\int\omega(\tfrac{1}{2}+it)|\mathcal{B}(\tfrac{1}{2}+it)|^{2}.$ (3.6)

Recall that the support $\omega$ is on an interval of length almost $T$ and
therefore to estimate (3.6), normally we need to engage in off-diagonal
estimation that requires some knowledge on the shifted convolution sums of
$b(n)$ 555See [9] for results on mean-value theorems for long Dirichlet
polynomials.. In many cases knowing about this turns to a harder problem than
our original one in estimating (3.6).

Now assume that we can decompose $\mathcal{B}$ as a product of two Dirichlet
polynomial plus another polynomial with a negligible contribution:

$\mathcal{B}(s)=\mathcal{G}(s)\mathcal{F}(s)+\mathcal{E}(s).$

Using the triangle inequity we have that
$|\mathcal{B}|^{2}\geq|\mathcal{G}\mathcal{F}|^{2}-|\mathcal{E}|^{2}.$

In this decomposition the length of $\mathcal{F}$ should be smaller than
$T^{1-\epsilon}$. Next we turn $\mathcal{F}$ to a probability measure
$\mu_{\mathcal{F}}$ by dividing with

$\int\omega(\tfrac{1}{2}+it)|\mathcal{F}(\tfrac{1}{2}+it)|^{2},$

which is calculable using Montgomery and Vaughan method [7].

Again by using the fact that length of $\mathcal{F}$ is smaller than
$T^{1-\epsilon}$ we are able to estimate pseudo moments of $\mathcal{G}$:

$\varrho_{k}=\int\mathcal{G}^{k}d\mu_{\mathcal{F}}.$

By using the mollifier method we can have that

$\int\omega(\tfrac{1}{2}+it)|\mathcal{B}(\tfrac{1}{2}+it)|^{2}\geq\varrho^{2}_{1}\int\omega(\tfrac{1}{2}+it)|\mathcal{F}(\tfrac{1}{2}+it)|^{2},$
(3.7)

assuming that $\int|\mathcal{E}|^{2}d\mu_{\mathcal{F}}$ has a negligible
contribution.

Now as we explained in (3.3), (3.4), and (3.5) if for $\sigma\neq 0$ we have
that

$\varrho_{2}-\varrho^{2}_{1}=\sigma,$

then we can get a better lower bound than (3.7):

$\int\omega(\tfrac{1}{2}+it)|\mathcal{B}(\tfrac{1}{2}+it)|^{2}\geq(\varrho^{2}_{1}+|\sigma|)\int\omega(\tfrac{1}{2}+it)|\mathcal{F}(\tfrac{1}{2}+it)|^{2}.$

Next we move to the problem of gaps between the zeros of the Riemann zeta
function.

##  4\. Landau-Siegel Zeros and the Distribution of Zeta Zeros

Distribution of zeros of the Riemann the zeta function on the critical line,
is an important problem, in part, due to its connection to Landau-Siegel zeros
and their effect on the class number formula. As we explained extensively in
the introduction of [2], in order to reject the possibility of Landau-Siegel
zeros we need to rule out a certain distribution of zeta zeros known as the
alternative hypothesis (AH).

Let $\bar{\gamma}=\frac{1}{2\pi}\gamma\log(\frac{\gamma}{2\pi})$ be the
normalized equivalent of an imaginary part of a zeta zero. By $\gamma^{+}$
and$\gamma^{-}$ we mean the zero after and before $\gamma$ respectively.

######  The Alternative Hypothesis (AH).

There exists a real number $T_{0}$ such that if $\gamma>T_{0}$ , then

$\tilde{\gamma}^{+}-\tilde{\gamma}\in\tfrac{1}{2}\mathbb{Z}.$

That is, almost all the normalized neighbour spacing’s are an integer or half-
integer.

This hypothesis is believed by many to be “absurd,” however no one yet knows
how to reject it. Commonly believed conjecture regarding the distribution of
zeta zeros is the Montgomery’s pair correlation conjecture (PCC). This
conjecture suggest that we will see a certain randomness in the distribution
of zeta zeros as oppose to the rigidity that we see in AH. For example under
PCC and RH we expect that $\emph{\emph{C}}_{\alpha}(t)$ must get arbitrary
large. To justify this note that we have a neat description of
$\emph{\emph{C}}_{\alpha}(t)$ that connects it to zeros in the vicinity of
$t$:

For $\alpha\leq 4$ and $t\sim T,$ define 666We need to consider the complex
definition of sine function in case that RH does not hold.:

$\mathcal{C}_{\alpha}(t)=\sum_{\zeta(\rho)=0}\bigg{(}\frac{\sin(\tfrac{\alpha}{2i}(\rho-(\tfrac{1}{2}+it))\log
T)}{\tfrac{\alpha}{2i}(\rho-(\tfrac{1}{2}+it))\log
T}\bigg{)}^{2}-\alpha^{-1}.$ (4.1)

Using a formula for expansion of $\mathcal{C}_{\alpha}$ (see [11, proof of
Lemma 1]) we have that

$\mathcal{C}_{\alpha}(t)=\emph{\emph{C}}_{\alpha}(t)+O(\tfrac{1}{\log T}).$

Assuming RH, (4.1) turns into

$\mathcal{C}_{\alpha}(t)=\sum_{\zeta(\rho)=0}\bigg{(}\frac{\sin(\tfrac{\alpha}{2}(\gamma-t)\log
T)}{\tfrac{\alpha}{2}(\gamma-t)\log T)}\bigg{)}^{2}-\alpha^{-1},$ (4.2)

and if there are many zeros very close to $t,$ they each add a positive amount
to the sum above, which increase the value of $\emph{\emph{C}}_{\alpha}(t)$.

In order to look more closely into AH we will take $\alpha=2$ in (4.2). The
reason behind it, is that $\mathcal{Z}_{2}$ captures the essence of AH very
well. To see this note, under RH, proving $\Re\mathcal{Z}_{2}(t)>1/2,$ for
some $t$, would precisely reject AH by showing that there must be gaps of size
not equal to half integers. For example, consider (4.2) for
$\rho=\tfrac{1}{2}+i\gamma$ and $t=\gamma.$ Moreover assume that
$\tilde{\gamma}^{+}-\tilde{\gamma}=0.75$, which immediately gives
$\emph{\emph{C}}_{\alpha}(\gamma)>0.545$. Thus, if, for instance, one can show
that

$\frac{2\pi}{T\log
T}\sum_{0<\gamma<T}\Re\mathcal{Z}_{2}(\tfrac{1}{2}+i\gamma)>1/2,$ (4.3)

then it would contradict AH. Evaluating the above for for
$\mathcal{Z}_{\alpha}$ with $\alpha<1$ is possible using the Landau-Gonek
formula. For $\alpha>1$ it remains an open problem. Now let us connect the
imaginary part of $\mathcal{Z}_{2}$ to the problem of gaps between zeros. Form
our theorem we get that

$\int\big{(}\Re\mathcal{Z}^{2}_{2}(t)-\Im\mathcal{Z}^{2}_{2}(t)\big{)}d\mu_{\lambda}=\int\big{(}\emph{\emph{C}}^{2}_{2}(t)-\emph{\emph{Im}}^{2}_{2}(t)\big{)}d\mu_{\lambda}=0.1270\cdots,$
(4.4)

Therefore a lower bound on

$\int\emph{\emph{Im}}^{2}_{2}(t)d\mu_{\lambda},$

would result on a lower bond on $\Re\mathcal{Z}^{2}_{2}.$

So far from (1.11) and Corollary 1.6 we have established that
$\emph{\emph{Im}}^{2}_{2}(t)$ must be small when $t$ is close to a zeta zero.

Now let us continue with assuming the pair correlation conjecture. From the
last paragraph we see that the imaginary part and the real part of
$\mathcal{Z}_{2}$ demonstrate some degree of independence. Therefore, we
expect that $\emph{\emph{ Im}}_{2}$ to be smaller than $\emph{\emph{C}}_{2}$
inside a cluster of zeros. Having these two facts in mind, we consider our
theorem for $\alpha=2$ in the following form

$0<\int\sum_{n=0}^{[k/2]}{k\choose
2n}(-1)^{n}\emph{\emph{C}}_{2}^{k-2n}(t)\emph{\emph{Im}}_{2}^{2n}(t)d\mu_{\lambda}(t)\approx\frac{1}{(k+1)!}.$
(4.5)

The higher moments in the above basically takes the problem to where the
action exist, meaning that

$\mu_{\lambda}\big{(}\\{t:|\emph{\emph{C}}_{2}(t)|<1\text{ and
}|\emph{\emph{Im}}_{2}(t)|<1\\}\big{)}$

does not contribute much. The main contribution comes from $t$ such that $\
|\emph{\emph{C}}_{2}(t)|\geq 1\text{ or }|\emph{\emph{Im}}_{2}(t)|\geq 1$.

The pair correlation conjecture suggest that $|\emph{\emph{C}}_{2}(t)|$ gets
arbitrary large. Considering this, $1/(k+1)!$ in the RHS of (4.5) suggests a
complicated cancellation inside the sum in (4.5). Moreover since the
expression involve higher moments of $\emph{\emph{C}}_{2}$ and $\emph{\emph{
Im}}_{2}$ the main contribution shifts mainly from average size gaps to the
zero clusters. Explaining (4.5), assuming PCC, can be a step forward in
understating the problem in a better way.

##  5\. Examples of Similar Random Variables

Motivated by our discussion in the last section and the exponential decay of
moments in (4.5) we ask the following question: If pseudo moments of a complex
random variable decrease very fast, would it imply that the random variable is
bounded?

We present two random variables that are examples of negative answers to the
above question. An important property of the first one is that, our method on
estimating pseudo moments does not reveal any useful information. In other
words, it is an example of where our method fails. Moments of the second RV
matches the moments of $\mathcal{Z}_{2}$ as in (4.5).

Consider

$\mathcal{X}(t)=\sum_{n=1}^{\infty}a_{n}e^{int}+\tfrac{1}{2},$

where $t\sim\mathcal{U}(0,2\pi).$ Then $E[\mathcal{X}^{k}]=\tfrac{1}{2^{k}}.$
This example shows that not much can be said about $\mathcal{X},$ since the
non-constant part that determines the behaviour of $\mathcal{X,}$ has no role
in determining the psudo moments. In our case with $\mathcal{Z}_{\alpha},$ we
saw that the $k$-th pseudo moment decrease approximately like $1/k!.$ A RV
like $\mathcal{Z}_{\alpha}$ with this property is a bit more complicated. Here
is one example:777This example is provided by Mateusz Wasilewski on
mathoverflow. The exposition given here, is by Iosif Pinelis [12].

Let $Z:=XU$, where $X$ and $U$ are independent with $X>0$ almost surely.
Assume that $X$ is unbounded with $E[X^{k}]<\infty$. Let $U=e^{iT}$ where $T$
is a RV with values in the interval $[0,2\pi)$ and its probability density
function is given by

$p(t)=\frac{1}{2\pi}\,\Big{(}1+2\sum_{n=1}^{\infty}a_{n}\cos nt\Big{)}.$ (5.1)

We need to assume that

$2\sum_{n=1}^{\infty}a_{n}<1,$

so that $p$ is well defined. Now for $k\in\mathbb{N}$ we have

$E[U^{k}]=E[e^{ikT}]=\int_{0}^{2\pi}e^{ikt}p(t)dt=a_{k},$

which by assuming

$0<a_{k}\sim\frac{1}{(k+1)!\,E[X]^{k}},$

and by using the independence of $X,U$ we have that

$E[Z^{k}]\sim\frac{1}{(k+1)!},$

as we desired. An important feature of this example is that the unboundedness
of $Z$ comes from $X$ which is independent from $U.$ Therefore the event that
$X$ attain large values should be independent of the event that $U$ gets
large.

It is not clear how we can connect this example to our result on pseudo
moments of $\mathcal{Z}_{\alpha}.$ To some extent it is true that when $t$ is
close to an isolated zero, $\mathcal{Z}_{\alpha}(t),$ behaves similarly to
$e^{it},$ and $\mu_{\lambda}$ behaves in a similar way to $p(t).$ However, if
$t$ is inside a zero cluster we must have

$\mathcal{Z}_{2}\sim X(t)e^{it,}$

where $X$ can be viewed as accumulation factor. The problem with connecting
this example to the moments of $\mathcal{Z}_{\alpha}$ is the independence
condition. It not clear how $X(t)$ can be independent of $\mu_{\lambda}$
unless it behave like a constant.

##  6\. Proof of the Theorem and its Corollaries

In this section we are mainly seeking to estimate

$\displaystyle\int\mathcal{Z}^{k}_{\alpha}(\tfrac{1}{2}+it)d\mu_{A}=\frac{\int\omega(\tfrac{1}{2}+it)\mathcal{Z}^{k}_{\alpha}(\tfrac{1}{2}+it)\Big{|}\sum_{n<T_{0}}\frac{a(n)}{n^{\tfrac{{}_{1}}{2}+it}}\Big{|}^{2}dt}{\int\omega(\tfrac{1}{2}+it)\Big{|}\sum_{n<T_{0}}\frac{a(n)}{n^{\tfrac{{}_{1}}{2}+it}}\Big{|}^{2}dt}$

and

$\displaystyle\int|\mathcal{Z}_{\alpha}(\tfrac{1}{2}+it)|^{2}d\mu_{A}=\frac{\int\omega(\tfrac{1}{2}+it)|\mathcal{Z}_{\alpha}(\tfrac{1}{2}+it)|^{2}\Big{|}\sum_{n<T_{0}}\frac{a(n)}{n^{\tfrac{{}_{1}}{2}+it}}\Big{|}^{2}dt}{\int\omega(\tfrac{1}{2}+it)\Big{|}\sum_{n<T_{0}}\frac{a(n)}{n^{\tfrac{{}_{1}}{2}+it}}\Big{|}^{2}dt},$

where we set $T_{0}=T\log^{-2}T$ for abbreviation. Let us recall that

$\mathcal{Z}_{\alpha}(s):=\frac{-2}{\alpha\log
T}\sum_{n<T^{\alpha}}\frac{\Lambda(n)}{n^{s}}\Big{(}1-\frac{\log n}{\alpha\log
T}\Big{)},$

with

$\mathcal{Z}_{\alpha}(\tfrac{1}{2}+it)=\emph{\emph{C}}_{\alpha}(t)+i\emph{\emph{Im}}_{\alpha}(t).$

Also the definition of the smooth weight is given by

$\omega(\tfrac{1}{2}+it)=\frac{\log T}{\sqrt{\pi}T}\hskip
2.84526pt\displaystyle{e^{\frac{((t-T)\log T)^{2}}{T^{2}}}},$ (6.1)

which basically is the Gaussian weight. For the Fourier transform of $\omega$
we have

$\frac{1}{2\pi
i}\int_{(c)}\omega(s)x^{s}ds=\frac{1}{2\pi}x^{\frac{1}{2}+iT}e^{-\frac{T^{2}\log^{2}x}{4\log^{2}T}}.$
(6.2)

We will use the above equation in the following form: Let $m$ or $n$ be
smaller than $T\log^{-2}T,$ then

$\int_{-\infty}^{\infty}\omega(\tfrac{1}{2}+it)\big{(}\frac{m}{n}\big{)}^{it}dt=\begin{cases}1\text{
if }m=n,\\\ 0\hskip 5.69054pt\mbox{otherwise.}\end{cases}$ (6.3)

We proceed with giving a proof of Theorem 2.1. Define
$\lambda_{\beta_{1},\beta_{2}}$ to be a completely multiplicative function
with

$\lambda_{\beta_{1},\beta_{2}}(p)=-1,\hskip 2.84526pt\text{ for
}T^{\beta_{1}}<p\leq T^{\beta_{2}}$

and $\lambda_{\beta_{1},\beta_{2}}(p)=1$ otherwise. Furthermore, let
$\mu_{A_{\beta_{1},\beta_{2},r,\eta}}$ be the measure we build, as in (1.3),
using

$a(n)=\lambda_{\beta_{1},\beta_{2}}(n)d_{r}(n)\big{(}1-\frac{\log n}{\log
T}\big{)}^{\eta},$ (6.4)

and $d_{r}$ is the generalized divisor function. Note that we considered a
more general definition for the arithmetic function in (6.4). This obviously
covers the arithmetic function in the statement of the theorem.

###### Proof of Theorem 2.1.

We have that

$\displaystyle\mathcal{Z}^{k}_{\alpha}(\tfrac{1}{2}+it)=\Big{(}\frac{-2}{\alpha\log
T}\Big{)}^{k}$
$\displaystyle\sum_{n_{j}<T^{\alpha}}\frac{\Lambda(n_{1})\cdots\Lambda(n_{k})}{(n_{1}\cdots
n_{k})^{\tfrac{1}{2}+it}}$ $\displaystyle\times\Big{(}1-\frac{\log
n_{1}}{\alpha\log T}\Big{)}\cdots\Big{(}1-\frac{\log n_{k}}{\alpha\log
T}\Big{)}.$

Therefore, by setting $\hat{\Lambda}(n)=\Lambda(n)(1-\tfrac{\log n}{\alpha\log
T})$ we get that

$\displaystyle\int$
$\displaystyle\omega(\tfrac{1}{2}+it)\mathcal{Z}^{k}_{\alpha}(\tfrac{1}{2}+it)\Big{|}\sum_{n<T_{0}}\frac{a(n)}{n^{\tfrac{1}{2}+it}}\Big{|}^{2}dt$
(6.5) $\displaystyle=\Big{(}\frac{-2}{\alpha\log
T}\Big{)}^{k}\sum_{\begin{subarray}{c}n_{j}<T^{\alpha}\\\
m,n<T_{0}\end{subarray}}\frac{\hat{\Lambda}(n_{1})\cdots\hat{\Lambda}(n_{k})a(m)a(n)}{\sqrt{mnn_{1}\cdots
n_{k}}}\int\omega(\tfrac{1}{2}+it)\big{(}\frac{m}{nn_{1}\cdots
n_{k}}\big{)}^{it}dt$ $\displaystyle=\Big{(}\frac{-2}{\alpha\log
T}\Big{)}^{k}\sum_{\begin{subarray}{c}n_{j}<T^{\alpha}\\\ nn_{1}\cdots
n_{k}<T_{0}\end{subarray}}\frac{\hat{\Lambda}(n_{1})\cdots\hat{\Lambda}(n_{k})a(n)a(nn_{1}\cdots
n_{k})}{nn_{1}\cdots n_{k}}.$

This is obtained using the fact that $a(m)=0$ for $n>T_{0}$ and by applying
(6.3) to the off-diagonal terms $m\neq nn_{1}\cdots n_{k}$ in the expansion of
the LHS of (6.5). By inserting the definition of $a(\cdot)$ given in (6.4),
the above would simplify to

$\Big{(}\frac{-2r}{\alpha\log
T}\Big{)}^{k}\sum_{\begin{subarray}{c}n_{j}<T^{\alpha}\end{subarray}}\frac{\lambda_{\beta_{1},\beta_{2}}(n_{1})\hat{\Lambda}(n_{1})\cdots\lambda_{\beta_{1},\beta_{2}}(n_{k)}\hat{\Lambda}(n_{k})}{n_{1}\cdots
n_{k}}\sum_{n<\tfrac{T_{0}}{n_{1}\cdots
n_{k}}}\frac{|d_{r}(n)|^{2}}{n}W_{n_{1},\cdots,n_{k}}(n),$ (6.6)

with

$W_{n_{1},\cdots,n_{k}}(n)=(1-\tfrac{\log n}{\log
T})^{\eta}\big{(}1-\tfrac{\log n}{\log T}-\tfrac{\log n_{1}}{\log
T}\cdots-\tfrac{\log n_{k}}{\log T}\big{)}^{\eta}.$

We use the following estimate for $d_{r}(\cdot)$ from [4]:

$\sum_{m<x}\frac{d^{2}_{r}(m)}{m}=A_{r}(\log x)^{r^{2}}+O\big{(}(\log
T)^{r^{2}-1}\big{)},$ (6.7)

Therefore, by Stieltjes integration, the innermost sum equals to

$\int_{1}^{T_{0}/n_{1}\cdots n_{k}}\frac{A_{r}r^{2}(\log
x)^{r^{2}-1}}{x}W_{n_{1},\cdots,n_{k}}(x)dx+O(\log^{r^{2}-1}T).$

Using the prime number theorem we have the (6.6) equals to

$\int_{\begin{subarray}{c}x_{i}<T^{\alpha}\\\ x_{1}\cdots
x_{k}<T\end{subarray}}\tfrac{\lambda_{\beta_{1},\beta_{2}}(x_{1})(1-\tfrac{\log
x_{1}}{\alpha\log T})\cdots\lambda_{\beta_{1},\beta_{2}}(x)(1-\tfrac{\log
x_{k}}{\alpha\log T})}{x_{1}\cdots x_{k}}\int_{1}^{T_{0}/x_{1}\cdots
x_{k}}\tfrac{(\log x)^{r^{2}-1}W_{x_{1},\cdots,x_{k}}(x)}{x}dx,$

times $A_{r}r^{2}\Big{(}\tfrac{-2r}{\alpha\log T}\Big{)}^{k}$ plus a
negligible error term. We use variables changes $u=\tfrac{\log x}{\log T}$ and
$v_{i}=\tfrac{\log x_{i}}{\log T}$ and we divide the whole thing by

$\displaystyle\int$
$\displaystyle\omega(\tfrac{{}_{1}}{2}+it)\Big{|}\sum\frac{a(n)}{n^{\tfrac{{}_{1}}{2}+it}}\Big{|}^{2}dt=\sum_{n<T\log^{-2}T}\frac{|a(n)|^{2}}{n}$
$\displaystyle=A_{r}r^{2}\log^{r^{2}}T\int_{0}^{1}v^{r^{2}-1}(1-v)^{2\eta}dv+O\big{(}(\log
T)^{r^{2}-1}\big{)}.$ (6.8)

Therefore we have that $\int\mathcal{Z}^{k}_{\alpha}(\tfrac{1}{2}+it)d\mu_{A}$
equals to

$\displaystyle\big{(}\frac{-2r}{\alpha}\big{)}^{k}\frac{\int_{0}^{1}\int_{0}^{1-v_{1}}\cdots\int_{0}^{\small
1-v_{2}\cdots-
v_{k}}Q(v_{1},\cdots,v_{k})u^{r^{2}-1}W_{v_{1},\cdots,v_{k}}(u)du\hskip
2.84526pt\small{d_{v_{1}}\ldots
d_{v_{k}}}}{\int_{0}^{1}v^{r^{2}-1}(1-v)^{2\eta}dv},$

plus an error term of order $\log^{-1}T.$ Note that we take

$Q(v_{1},\cdots,v_{k})=\lambda_{\beta_{1},\beta_{2}}(v_{1})(1-\tfrac{v_{1}}{\alpha})\cdots\lambda_{\beta_{1},\beta_{2}}(v_{k})(1-\tfrac{v_{k}}{\alpha}).$

Now it is easy to see that (2.2) is a special case (with $\eta=0$ which is
equivalent to $W_{v_{1},\cdots,v_{k}}(u)=1$) of what we showed in the above.

For $\alpha\geq 1$ and simple cases of $a(n)=\lambda(n),1,$ or
$\lambda_{2}(n)$ we can get a much cleaner estimate than (2.2):

If $a(n)=\lambda(n)$ we have that $a(n)a(np_{1}\cdots p_{k})$ in (6.5) is
$(-1)^{k}.$

If $a(n)=\lambda_{2}(n)$ and for even moments ($\mathcal{Z}_{\alpha}^{2k}$),
we use

$\lambda_{2}(n)\lambda_{2}(np_{1}\cdots p_{2k})=(-1)^{k}.$ (6.9)

For $a(n)=\lambda_{2}(n)$ and odd moments we divide into two cases and we get

$\lambda_{2}(n)\lambda_{2}(np_{1}\cdots
p_{k})=\begin{cases}\lambda(n)&\mbox{if }k=4r+1\\\ -\lambda(n).&\mbox{if
}k=4r+3\end{cases}$ (6.10)

If we insert (6.9) into (6.5) we get:

$\displaystyle(-1)^{k}\sum_{n_{1}\cdots
n_{2k}<T^{\alpha}}\tfrac{\Lambda(n_{1})\cdots\Lambda(n_{2k})}{n_{1}\cdots
n_{2k}}\big{(}1-\tfrac{\log n_{1}}{\alpha\log
T}\big{)}\cdots\big{(}1-\tfrac{\log n_{2k}}{\alpha\log
T}\big{)}\log\big{(}\tfrac{T_{0}}{n_{1}\cdots n_{2k}}\big{)},$ (6.11)

which justify $(-1)^{k}$ in (2.4) in the statement of the theorem.

For $a(n)=\lambda(n)$ we end up with

$\displaystyle(-1)^{k}\sum_{n_{1}\cdots
n_{k}<T^{\alpha}}\tfrac{\Lambda(n_{1})\cdots\Lambda(n_{k})}{n_{1}\cdots
n_{k}}\big{(}1-\tfrac{\log n_{1}}{\alpha\log
T}\big{)}\cdots\big{(}1-\tfrac{\log n_{k}}{\alpha\log
T}\big{)}\log\big{(}\tfrac{T_{0}}{n_{1}\cdots n_{k}}\big{)}.$ (6.12)

Therefore, we will proceed to estimate (6.12) since it covers both cases.
Using Perron’s formula we have

$\displaystyle\log\frac{T}{n_{1}\cdots n_{k}}\mathds{1}_{n_{1}\cdots
n_{k}<T}=\frac{1}{2\pi i}\int_{(c)}\frac{T^{s}}{(n_{1}\cdots
n_{k})^{s}}\frac{ds}{s^{2}}.$

Hence (6.11) comes to

$\displaystyle\frac{1}{2\pi i}\int_{(c)}\sum_{n_{j}}$
$\displaystyle\frac{\Lambda(n_{1})\cdots\Lambda(n_{k})}{(n_{1}\cdots
n_{k})^{s}}\Big{(}1-\tfrac{\log n_{1}}{\alpha\log
T}\Big{)}\cdots\Big{(}1-\tfrac{\log n_{k}}{\alpha\log
T}\Big{)}\frac{ds}{s^{2}}$ $\displaystyle=\frac{1}{2\pi
i}\int_{(c)}T^{s}\Big{(}\sum\frac{\Lambda(n)}{n^{s+1}}-\frac{1}{\alpha\log
T}\frac{\Lambda(n)\log n}{n^{s+1}}\Big{)}^{k}\frac{ds}{s^{2}}=$
$\displaystyle\frac{1}{2\pi
i}\int_{(c)}T^{s}\Big{(}-\frac{\zeta^{\prime}}{\zeta}(s+1)+\frac{1}{\alpha\log
T}\Big{(}\big{(}\frac{\zeta^{\prime}}{\zeta}(s+1)\big{)}^{\prime}\Big{)}^{k}\frac{ds}{s^{2}}.$

We move the line of integration to $c=1-\log^{-3/4}T$ and use the zero free
region of the zeta function. We just need to consider the residue at $s=0$ in
the above. We end up with

$\displaystyle\frac{1}{2\pi i}$ $\displaystyle\int
T^{s}\Big{(}\frac{1}{s}-\frac{1}{\alpha\log
Ts^{2}}\Big{)}^{k}\frac{ds}{s^{2}}$ $\displaystyle=\frac{1}{2\pi i}\int
T^{s}\sum_{i=0}^{k}{k\choose i}\big{(}\frac{-1}{\alpha\log
T}\big{)}^{i}\frac{1}{s^{k+i+2}}ds$ $\displaystyle=\frac{1}{2\pi
i}\sum_{i=0}^{k}{k\choose i}\big{(}\frac{-1}{\alpha\log
T}\big{)}^{i}\int\frac{T^{s}}{s^{k+i+2}}$ $\displaystyle=(\log
T)^{k+1}\sum_{i=0}^{k}{k\choose
i}\big{(}\frac{1}{\alpha}\big{)}^{i}\frac{(-1)^{i}}{(k+i+1)!}.$

Therefore we have

$\displaystyle\int\omega(\tfrac{1}{2}+it)\mathcal{Z}^{k}_{\alpha}(\tfrac{1}{2}+it)\Big{|}\sum_{n<T_{0}}\frac{\lambda(n)}{n^{\tfrac{1}{2}+it}}\Big{|}^{2}dt=\Big{(}\frac{2}{\alpha}\Big{)}^{k}\sum_{i=0}^{k}{k\choose
i}\frac{(-1)^{i}}{\alpha^{i}}\frac{\log T}{(k+i+1)!}.$

Finally we divide by $\log T$, to get to $d\mu_{\lambda}.$ For $\lambda_{2}$
we have the same calculation, with only difference that we must considered it
for the $2k$-moment. This finishes the proof. ∎

Next we give a corollary of the theorem that we will use later.

######  Corollary 6.1.

We have that

$\displaystyle\int\Big{(}{\emph{C}}_{1}^{2}(t)-\textrm{
\emph{Im}}_{1}^{2}(t)\Big{)}d\mu_{\lambda}=0.36666\cdots+O\big{(}\tfrac{1}{\log
T}\big{)},$ (6.13)
$\displaystyle\int\Big{(}{\emph{C}}_{1}^{3}(t)-3{\emph{C}}_{1}(t)\textrm{
\emph{Im}}^{2}(t)\Big{)}d\mu_{\lambda}=0.16504\cdots+O\big{(}\tfrac{1}{\log
T}\big{)},$ (6.14)
$\displaystyle\int\Big{(}{\emph{C}}_{1}^{4}(t)-6{\emph{C}}_{1}^{2}(t)\textrm{
\emph{Im}}^{2}(t)+\textrm{
\emph{Im}}^{4}(t)\Big{)}d\mu_{\lambda}=0.06194\cdots+O\big{(}\tfrac{1}{\log
T}\big{)}.$ (6.15)

###### Proof.

We have that

$\Re(\mathcal{Z}^{2}_{\alpha}(\tfrac{1}{2}+it))=C_{\alpha}(t)^{2}-\emph{\emph{Im}}^{2}_{\alpha}(t),$

and

$\Re(\mathcal{Z}^{3}_{\alpha}(\tfrac{1}{2}+it))=C_{\alpha}(t)^{3}-3C_{\alpha}(t)\text{Im}^{2}_{\alpha}(t),$

also a similar equation for the real part of the $4$-th moment

$\Re(\mathcal{Z}^{4}_{\alpha}(\tfrac{1}{2}+it))=C_{\alpha}(t)^{4}-6C^{2}_{\alpha}(t)\text{Im}^{2}_{\alpha}(t)+\text{Im}^{4}_{\alpha}(t).$

We consider the above equations and then we apply Theorem 2.1 to get the
corollary.∎

We continue with giving a proof of Corollaries 1.2 and 1.4. Using (1.5) we
have

$\emph{\emph{C}}_{\alpha}(t)=\Re\mathcal{Z}_{\alpha}=\frac{-2\Re}{\alpha\log
T}\sum_{n<T^{\alpha}}\frac{\Lambda(n)}{n^{\tfrac{{}_{1}}{2}+it}}\Big{(}1-\frac{\log
n}{\alpha\log T}\Big{)}.$

Therefore,

$\int\emph{\emph{C}}^{2}_{\alpha}(t)d\mu_{\lambda}=\int\frac{(\mathcal{Z}_{\alpha}+\overline{\mathcal{Z}}_{\alpha})^{2}}{4}d\mu_{\lambda}=\int\frac{\mathcal{Z}^{2}_{\alpha}}{4}+\frac{\overline{\mathcal{Z}^{2}_{\alpha}}}{4}+\frac{|\mathcal{Z}_{\alpha}|^{2}}{2}d\mu_{\lambda}.$
(6.16)

Note that

$\int\frac{\mathcal{Z}^{2}_{\alpha}}{4}+\frac{\overline{\mathcal{Z}^{2}_{\alpha}}}{4}=\int\frac{\emph{\emph{C}}_{1}(t)^{2}-\textrm{
\emph{\emph{Im}}}_{1}^{2}(t)}{2}d\mu_{\lambda}=0.1833333\cdots,$ (6.17)

using Corollary 6.1. For estimating

$\int|\mathcal{Z}_{\alpha}|^{2}d\mu_{A},$

we use the following lemma.

######  Lemma 6.2.

Let $a(n)=\lambda(n)d_{r}(n)\big{(}1-\frac{\log n}{\log T}\big{)}^{\eta},$ and
set

$\mathcal{Z}_{\alpha}(s)\big{(}\displaystyle{\sum_{n<T\log^{-2}T}}a(n)n^{-s}\big{)}=\sum_{m<T^{\alpha+1}}\frac{b(m)}{m^{s}}$
(6.18)

Then we have

$\displaystyle\int|\mathcal{Z}_{\alpha}($
$\displaystyle\tfrac{1}{2}+it)|^{2}d\mu_{A_{r,\eta}}=O\big{(}\frac{1}{\log
T}\big{)}$
$\displaystyle+\frac{\sum_{m<T\log^{-2}T}\frac{|b(m)|^{2}}{m}+\int\omega(\tfrac{1}{2}+it)\bigg{|}\displaystyle{\sum_{{T}{\small{\log^{-2}T}}\leq
n}}\frac{b(n)}{n^{\tfrac{{}_{1}}{2}+it}}\bigg{|}^{2}}{A_{r}\log^{r^{2}}T\int_{0}^{1}v^{r^{2}-1}(1-v)^{2\eta}dv}.$
(6.19)

Moreover, we have

$\displaystyle\sum_{m<T\log^{-2}T}$
$\displaystyle\frac{|b(m)|^{2}}{m}=A_{r}\frac{4r^{2}}{\alpha^{2}}\log^{r^{2}}T{\int_{0}^{1}\int_{0}^{1-v}vQ(u,v,v)dudv}$
(6.20)
$\displaystyle+A_{r}\frac{4r^{4}}{\alpha^{2}}\log^{r^{2}}T{\displaystyle{\int_{0}^{1}}Q(u,v_{1},v_{2})dudv_{2}dv_{1}}+O\big{(}\log^{r^{2}-1}T\big{)},$

with

$Q(u,v_{1},v_{2})=u^{r^{2}-1}\big{(}1-\frac{v_{1}}{\alpha}\big{)}\small{\big{(}1-\frac{v_{2}}{\alpha}\big{)}}\small{(1-u-v_{1})^{\eta}(1-u-v_{2})^{\eta}}.$

###### Proof.

First note that the denominator in (6.2) comes from the fact that we
considered $d\mu_{A_{r,\eta}}$ to be a probability measure, hence we need to
divide by $\sum|a(n)|^{2}/n$, which using (6.8) is

$A_{r}\log^{r^{2}}T\int_{0}^{1}v^{r^{2}-1}(1-v)^{2\eta}dv+O(\log^{r^{2}-1}T).$

For the sum with $b(n)$ in (6.18), we split the sum

$\displaystyle\sum_{m<T^{\alpha+1}}\frac{b(n)}{n^{s}}=\sum_{m<T\log^{-2}T}\frac{b(m)}{m^{s}}+\sum_{T\log^{-2}T\leq
n<T^{\alpha+1}}\frac{b(n)}{n^{s}}.$

Again, for simplicity we set $T_{0}=T\log^{-2}T.$ Hence

$\displaystyle\int\omega(\tfrac{1}{2}+it)\bigg{|}\sum_{m<T^{\alpha+1}}\frac{b(m)}{m^{\tfrac{1}{2}+it}}\bigg{|}^{2}=\int\omega(\tfrac{1}{2}+it)\bigg{|}\sum_{m<T_{0}}\frac{b(m)}{m^{\tfrac{1}{2}+it}}\bigg{|}^{2}$
$\displaystyle+\int\omega(\tfrac{1}{2}+it)\bigg{|}\sum_{T_{0}\leq
n<T^{\alpha+1}}\frac{b(n)}{n^{\tfrac{1}{2}+it}}\bigg{|}^{2}+2\Re\sum_{\begin{subarray}{c}m<T_{0}\\\
T^{\alpha}>n\geq
T_{0}\end{subarray}}\frac{b(n)b(m)}{\sqrt{mn}}\int\omega(\tfrac{1}{2}+it)\big{(}\frac{m}{n}\big{)}^{it}$

The last integral using the Fourier transform of the Gaussian in (6.2) is

$\ll e^{-\frac{T^{2}\log^{2}(m/n)}{4\log^{2}T}}.$ (6.21)

Using the fact that $m<T_{0}$ and $n\geq T_{0}$ we have

$|\log\big{(}\frac{m}{n}\big{)}|=|\log\big{(}1+\frac{n-m}{m}\big{)}|>\frac{1}{m}>\frac{\log^{2}T}{T}.$

Therefore (6.21) is smaller than $e^{-\log^{2}T},$ and therefore, using
$b(m)<m^{\epsilon},$ we get

$\frac{b(n)b(m)}{\sqrt{mn}}\int\omega(\tfrac{1}{2}+it)\big{(}\frac{m}{n}\big{)}^{it}\ll
T^{\alpha/2+\epsilon}e^{-\log^{2}T}\ll T^{-B},$

for every $B>0.$ Therefore we established that

$\int|\mathcal{Z}_{\alpha}|^{2}d\mu_{A_{r,\eta}}=\frac{\sum_{n<T_{0}}\frac{|b(m)|^{2}}{m}+\int\omega(\tfrac{1}{2}+it)\bigg{|}\sum_{T_{0}\leq\hskip
2.84526ptn<T^{\alpha+1}}\frac{b(n)}{n^{\tfrac{{}_{1}}{2}+it}}\bigg{|}^{2}dt}{\sum_{n<T_{0}}\frac{|a(n)|^{2}}{n}}+O(\tfrac{1}{T^{B}}),$

which justifies (6.2). To show (6.20), for $n<T_{0},$ we have

$\displaystyle b(n)=$ $\displaystyle-\frac{2}{\alpha\log
T}\sum_{d|n}\Lambda(d)\big{(}1-\frac{\log(d)}{\alpha\log
T}\big{)}\lambda(n/d)d_{r}(n/d)\big{(}1-\frac{\log n}{\log T}+\frac{\log
d}{\log T}\big{)}^{\eta}$ $\displaystyle=-\frac{2}{\alpha\log
T}d_{r}(n)\lambda(n)\sum_{p^{h}||n}s_{h}(p)\log p,$ (6.22)

where $p^{h}||n$ means that $p^{h}|n$ and $p^{h+1}\nmid n$ and

$s_{h}(p)=\sum_{j=1}^{h}(-1)^{j}\Big{(}\frac{\Gamma(h-j+r)h!}{\Gamma(h+r)(h-i)!}\Big{)}\big{(}1-\frac{j\log
p}{\alpha\log T}\big{)}\big{(}1-\frac{\log n}{\log T}+\frac{j\log p}{\log
T}\big{)}^{\eta}.$

We used the following formula

$d_{r}(p^{k})=\frac{\Gamma(k+r)}{\Gamma(r)k!},$

in the above. Consequently, we have

$\displaystyle\sum_{n<T_{0}}\frac{|b(n)|^{2}}{n}=\frac{4}{(\alpha\log
T)^{2}}\sum_{n<T_{0}}\frac{\big{(}d_{r}(n)\sum_{p^{h}||n}s_{h}(p)\log
p\big{)}^{2}}{n}$ $\displaystyle=\frac{4}{(\alpha\log
T)^{2}}\sum_{n<T_{0}}\frac{d^{2}_{r}(n)\sum_{p^{h}||n}s_{h}(p)\log
p\sum_{q^{h^{\prime}}||n}s_{h^{\prime}}(q)\log q}{n}.$ (6.23)

We show that the contribution of $h>1$ or $h^{\prime}>1$ is negligible. We
have that (6) is smaller than

$\frac{1}{(\alpha\log T)^{2}}\sum_{p^{h},q^{h}{{}^{\prime}}<T_{0}}\frac{\log
p\log q}{r^{2}p^{h}q^{h^{\prime}}}\sum_{\begin{subarray}{c}p\nmid n,q\nmid
n\\\ n<T/(p^{r}q^{r}{{}^{\prime}})\end{subarray}}\frac{d^{2}_{r}(n)}{n}$

We use (6.7) and we obtain that the above is

$\ll\log^{r^{2}-1}T,$

if either of $h,h^{\prime}$ are $>1.$ Therefore considering the fact that the
total mass of $\mu_{A_{r,\eta}}$ in the denominator in (6.20) is of size
$\log^{r^{2}}T$, proves the contribution of $h$ or $h^{\prime}$ bigger than
$1$ is negligible. Hence, we continue with $h=h^{\prime}=1.$ We set $n=mpq$ in
(6), when $p\neq q$, and by changing the order of the summation we are seeking
to estimate

$\displaystyle\frac{4}{(\alpha\log T)^{2}}$
$\displaystyle\sum_{p,q<T_{0}}\frac{\log p\big{(}1-\tfrac{\log p}{\alpha\log
T}\big{)}\log q\big{(}1-\tfrac{\log q}{\alpha\log T}\big{)}}{r^{2}pq}$
$\displaystyle\times\sum_{\begin{subarray}{c}p\nmid m,q\nmid m\\\
m<T/pq\end{subarray}}\frac{d^{2}_{r}(mpq)\big{(}1-\frac{\log m}{\log
T}+\frac{\log p}{\log T}\big{)}^{\eta}\big{(}1-\frac{\log m}{\log
T}-\frac{\log q}{\log T}\big{)}^{\eta}}{m}.$ (6.24)

Therefore, for $p\neq q,$ we have

$\displaystyle\frac{4r^{2}}{(\alpha\log T)^{2}}\sum_{pq<T_{0}}$
$\displaystyle\frac{\log p\big{(}1-\tfrac{\log p}{\alpha\log T}\big{)}\log
q\big{(}1-\tfrac{\log q}{\alpha\log T}\big{)}}{pq}$ (6.25)
$\displaystyle\times\sum_{\begin{subarray}{c}p\nmid m,q\nmid m\\\
m<T/pq\end{subarray}}\frac{d^{2}_{r}(m)\big{(}1-\frac{\log m}{\log
T}-\frac{\log p}{\log T}\big{)}^{\eta}\big{(}1-\frac{\log m}{\log
T}-\frac{\log q}{\log T}\big{)}^{\eta}}{m}.$

For $p=q,$ we set $m=np$ in (6) and we get

$\frac{4}{(\alpha\log T)^{2}}\sum_{p<T_{0}}\frac{\log^{2}p\big{(}1-\tfrac{\log
p}{\alpha\log T}\big{)}^{2}}{p}\sum_{\begin{subarray}{c}p\nmid m\\\
m<T/p\end{subarray}}\frac{d^{2}_{r}(m)}{m}\big{(}1-\frac{\log m}{\log
T}\big{)}^{2\eta}.$ (6.26)

Conditions $p\nmid m$ and $p\nmid m,q\nmid m$ in (6.25) and (6.26) can be
dropped with a negligible error term, since we have

$\sum_{m<T,\hskip 2.84526ptp|n}\frac{1}{m}=\frac{1}{p}\log(T/p)+O(1/T).$

Applying the prime number theorem and (6.7) imply that (6.25) equals to

$A_{r}\frac{4r^{4}}{\alpha^{2}}\log^{r^{2}}T\int_{0}^{1}\int_{0}^{1-v_{1}}\int_{0}^{1-v_{1}-v_{2}}Q(u,v_{1},v_{2})dudv_{1}dv_{2}+O(\log^{r^{2}-1}T),$
(6.27)

Recall that

$Q(u,v_{1},v_{2})=u^{r^{2}-1}\big{(}1-\frac{v_{1}}{\alpha}\big{)}\small{\big{(}1-\frac{v_{2}}{\alpha}\big{)}}\small{(1-u-v_{1})^{\eta}(1-u-v_{2})^{\eta}}.$

Similarly, (6.26) equals to

$A_{r}\frac{4r^{2}}{\alpha^{2}}\log^{r^{2}}T\int_{0}^{1}\int_{0}^{1-v}u^{r^{2}-1}\small{(1-u-v)^{2\eta}v\big{(}1-\frac{v}{\alpha}\big{)}^{2}}dudv+O(\log^{r^{2}-1}T).$

This completes the proof of Lemma 6.2. ∎

###### Proof of Corollaries 1.2 and 1.4.

By Lemma 6.2 for $r=1$ and $\eta=0,$ we have that

$\displaystyle\int|\mathcal{Z}_{\alpha}|^{2}d\mu_{\lambda}\geq$ $\displaystyle
4\alpha^{-2}\int_{0}^{1}\int_{0}^{1-u}(1-u-v)\big{(}1-\frac{u}{\alpha}\big{)}\big{(}1-\frac{v}{\alpha}\big{)}dvdu$
(6.28)
$\displaystyle+4\alpha^{-2}\int_{0}^{1}u(1-u)\big{(}1-\frac{u}{\alpha}\big{)}^{2}du+O(\log^{-1}T)$
$\displaystyle=\frac{4}{3\alpha^{2}}-\frac{1}{\alpha^{3}}+\frac{7}{30\alpha^{4}}.$
(6.29)

For $\alpha=1,$ we get

$\int|\mathcal{Z}_{1}|^{2}d\mu_{\lambda}\geq 0.56664.$ (6.30)

Recall that by (6.16) we have

$\int\emph{\emph{C}}^{2}_{1}(t)d\mu_{\lambda}=\int\Big{(}\frac{\mathcal{Z}^{2}_{1}}{4}+\frac{\overline{\mathcal{Z}^{2}_{1}}}{4}+\frac{|\mathcal{Z}_{1}|^{2}}{2}\Big{)}d\mu_{\lambda}.$

By combining (6.17) and (6.30) we have

$\int\emph{\emph{C}}^{2}_{1}(t)d\mu_{\lambda}\geq 0.46666.$ (6.31)

Using

$\int\Big{(}\emph{\emph{C}}_{1}(t)^{2}-\emph{\emph{Im}}_{1}^{2}(t)\Big{)}d\mu_{\lambda}=0.36666\cdots,$

we get a lower bound for $\emph{\emph{Im}}_{1}^{2}.$ For the $4$-th moment
note that

$\int\big{(}\emph{\emph{C}}_{1}^{2}(t)-0.46666)^{2}d\mu_{\lambda}(t)>0,$

which gives

$\int\emph{\emph{C}}_{1}^{4}(t)d\mu_{\lambda}(t)>2\times
0.46666\int\emph{\emph{C}}_{1}^{2}(t)d\mu_{\lambda}(t)-(0.46666)^{2}>(0.46666)^{2}.$
(6.32)

Similarly we can get a lower bound for the expectation of
$\emph{\emph{Im}}_{1}^{4}.$

For proving (1.14) we use (6.32) and the lower bound we get for the $4$-th
power of $\emph{\emph{Im}}_{1},$ and insert them in

$\int\Big{(}\emph{\emph{C}}_{1}^{4}(t)-6\emph{\emph{C}}_{1}^{2}(t)\emph{
\emph{Im}}_{1}^{2}(t)+\emph{
\emph{Im}}_{1}^{4}(t)\Big{)}d\mu_{\lambda}=0.06194\cdots+O\big{(}\tfrac{1}{\log
T}\big{)},$

which we obtained in Corollary 6.1. This finishes the proof. ∎

###### Proof of Corollary 1.1.

For $r=2$ and $\eta=0,$ using the theorem we have that

$\int\big{(}\emph{\emph{C}}^{2}_{1}(t)-\emph{\emph{Im}}^{2}_{1}(t)\big{)}d\mu_{A_{0,1,2}}=0.39047\cdots,$
(6.33)

where $\mu_{A_{0,1,2}}$ correspond to the measure we have with
$a(n)=\lambda(n)d_{2}(n).$ On the other hand we have that

$\int\emph{\emph{C}}_{1}(t)d\mu_{A_{0,1,2}}=2/3,$

which gives that

$\int\emph{\emph{C}}^{2}_{1}(t)d\mu_{A_{0,1,2}}>4/9.$

Employing (6.33) we have that

$\int\emph{\emph{Im}}^{2}_{1}(t)d\mu_{A_{0,1,2}}>0.0539,$

and therefore

$\int|\mathcal{Z}_{1}|^{2}(t)d\mu_{A_{0,1,2}}>0.498.$ (6.34)

Using Lemma 6.2 we have that

$\int|\mathcal{Z}_{1}|^{2}d\mu_{A_{0,1,2}}=0.4619+\frac{\int\omega(\tfrac{1}{2}+it)\bigg{|}\displaystyle{\sum_{{T}{\small{\log^{-2}T}}\leq
n}}\frac{b(n)}{n^{\tfrac{{}_{1}}{2}+it}}\bigg{|}^{2}dt}{A_{2}\log^{4}T\int_{0}^{1}v^{3}dv},$

where $b(n)=-\frac{2}{\log T}\sum_{d|n}\Lambda(d)\big{(}1-\frac{\log(d)}{\log
T}\big{)}\lambda(n/d)d_{2}(n/d)\asymp\lambda(n)d_{2}(n)\frac{\log n}{\log T}.$
Now by using (6.34), we have that

$\int\omega(\tfrac{1}{2}+it)\bigg{|}\displaystyle{\sum_{{T}{\log^{-2}T}\leq
n}}\hskip
5.69054pt\displaystyle{\frac{b(n)}{n^{\tfrac{{}_{1}}{2}+it}}}\bigg{|}^{2}dt>0.009A_{2}\log^{4}T.$

∎

######  Corollary 1.6.

The proof goes similarly to the proof of Corollary 1.4, therefore we only
comment on parts that are different. We have

$\int\emph{\emph{Im}}^{2}_{\alpha}(t)d\mu_{\lambda_{2}}=\int\frac{|\mathcal{Z}_{\alpha}|^{2}}{2}-\Big{(}\frac{\mathcal{Z}^{2}_{\alpha}}{4}+\frac{\overline{\mathcal{Z}^{2}_{\alpha}}}{4}\Big{)}\hskip
2.84526ptd\mu_{\lambda_{2}}.$ (6.35)

Using (2.4) with $k=1$ we get that

$\int\Big{(}\frac{\mathcal{Z}^{2}_{\alpha}}{4}+\frac{\overline{\mathcal{Z}^{2}_{\alpha}}}{4}\Big{)}d\mu_{\lambda_{2}}=-0.18333\cdots.$

For $\int{|\mathcal{Z}_{\alpha}|^{2}}d\mu_{\lambda_{2}},$ we use Lemma 6.2
with

$\displaystyle b(n)=-\frac{2}{\alpha\log
T}\sum_{d|n}\Lambda(d)\big{(}1-\frac{\log(d)}{\alpha\log
T}\big{)}\lambda_{2}(n/d)$ (6.36)

As we seen with the proof of Corollary 1.4, in estimating

$\sum_{n<T\log^{-2}T}\frac{|b(m)|^{2}}{m}$

main contributions comes form $d=p$ and therefore $\lambda_{2}(n/d)$ can be
pulled out of the sum in (6.36) with a factor of

$(-1)^{\big{[}(\Omega(n)-1)/2\big{]}}.$

Since we have in the sum of $|b(m)|^{2},$ we can ignore the contribution of
$-1,$ and the rest of calculation is the same as the calculation for $\lambda$
in (6). ∎

###### Proof of Corollary 2.2.

Equation (6.28) basically gives the result:

$\int|\mathcal{Z}_{\alpha}|^{2}d\mu_{\lambda}\geq\frac{4}{3\alpha^{2}}-\frac{1}{\alpha^{3}}+\frac{7}{30\alpha^{4}}.$

Now let us compare this with what we get from only applying the Mollifier
method. Using the theorem we have that

$\int\Re(\mathcal{Z}^{2}_{\alpha}(\tfrac{1}{2}+it))d\mu_{\lambda}=\int\big{(}C_{\alpha}(t)^{2}-\emph{\emph{Im}}^{2}_{\alpha}(t)\big{)}d\mu_{\lambda}=\tfrac{4}{6}\alpha^{-2}-\tfrac{1}{3}\alpha^{-3}+\tfrac{1}{30}\alpha^{-4}.$

On the other hand for $\alpha>1,$ using the mollifier method, we have that

$\int
C_{\alpha}(t)^{2}d\mu_{\lambda}>\big{(}\frac{1}{\alpha}-\frac{1}{3\alpha^{2}}\big{)}^{2},$

therefore

$\int\emph{\emph{Im}}^{2}_{\alpha}(t)d\mu_{\lambda}>\frac{1}{3\alpha^{2}}-\frac{1}{3\alpha^{3}}-\frac{17}{90\alpha^{4}},$

which give

$\int\big{(}C_{\alpha}(t)^{2}+\emph{\emph{Im}}^{2}(t)\big{)}d\mu_{\lambda}\geq\frac{4}{3}\alpha^{-2}-\alpha^{-3}+\tfrac{17}{90}\alpha^{-4}.$

∎

## Acknowledgement

This work was supported by Institute for Research in Fundamental Sciences
(IPM) in Iran. I would like to thank them for their support and hospitality
during my visit.

## References

  * [1] F. Aryan On an extension of the Landau-Gonek formula, preprint, available at arXiv:1902.05473.
  * [2] F. Aryan A new approach to gaps between zeta zeros, preprint, available at arXiv:1910.02408.
  * [3] F. Aryan N.C. Ng, Discrete Mean Values of Dirichlet Polynomials, In preparation.
  * [4] H.M. Bui, M.B. Milinovich, and N.C. Ng, A note on the gaps between consecutive zeros of the Riemann zeta-function, Proc. Amer. Math. Soc. 138 (2010), no. 12, 4167-4175.
  * [5] D. W. Farmer, S. M. Gonek, Y. Lee, Pair correlation of the zeros of the derivative of the Riemann $\xi$‐function, Journal of the London Mathematical Society, 90 (2014), 241–269.
  * [6] A. J. Harper The Riemann zeta function in short intervals [after Najnudel, and Arguin, Belius, Bourgade, Radziwill, and Soundararajan]. arXiv:1904.08204.
  * [7] H. L. Montgomery and R. C. Vaughan The large sieve, Mathematika, 1973 20, 119–134
  * [8] H. L. Montgomery, The pair correlation of the zeros of the zeta function, Proc. Symp. Pure Math. 24, A.M.S., Providence 1973, 181–193.
  * [9] D.A. Goldston and S. M. Gonek,  Mean value theorems for long Dirichlet polynomials and tails of Dirichlet series, Acta Arith. 84 (1998), no. 2, 155–192.
  * [10] D. A. Goldston, S. M. Gonek, and H. L. Montgomery, Mean values of the logarithmic derivative of the Riemann zeta-function with applications to primes in short intervals, J. Reine Angew. Math. 537 (2001), 105-126.
  * [11] S. M. Gonek, An explicit formula of Landau and its applications to the theory of the zeta function, Contemp. Math 143 (1993), 395–413.
  * [12] Iosif Pinelis (https://mathoverflow.net/users/36721/iosif-pinelis), Moments of complex random variables, URL (version: 2020-07-09): https://mathoverflow.net/q/365197.
  * [13] K. Matomäki and M. Radziwiłł. Multiplicative Functions in Short Intervals. Ann. of Math 183, 2016, pp. 1015-1056.
  * [14] K. Soundararajan, Extreme values of zeta and L-functions, Math. Ann. 342 (2008), 467–486.
  * [15] P. Sarnak.  Three lectures on the Mobius function randomness and dynamics https://www.math.ias.edu/files/wam/2011/PSMobius.pdf, 2011.
  * [16] Bondarenko, Andriy; Seip, Kristian. Large greatest common divisor sums and extreme values of the Riemann zeta function. Duke Math. J. 166 (2017), no. 9, 1685–1701.


# Semi-supervised cross-lingual speech emotion recognition

Mirko Agarla, Simone Bianco1, Luigi Celona1, Paolo Napoletano1, Alexey
Petrovsky2, Flavio Piccoli1, Raimondo Schettini1, Ivan Shanin2 1 University of
Milano-Bicocca, Milano, Italy
2 Moscow Research Center, Huawei Technologies Co. Ltd, Russia

###### Abstract

Speech emotion recognition (SER) on a single language has achieved remarkable
results through deep learning approaches over the last decade. However, cross-
lingual SER remains a challenge in real-world applications due to (i) a large
difference between the source and target domain distributions, (ii) the
availability of few labeled and many unlabeled utterances for the new
language. Taking into account previous aspects, we propose a Semi-Supervised
Learning (SSL) method for cross-lingual emotion recognition when a few labels
from the new language are available. Based on a Convolutional Neural Network
(CNN), our method adapts to a new language by exploiting a pseudo-labeling
strategy for the unlabeled utterances. In particular, the use of a hard and
soft pseudo-labels approach is investigated. We thoroughly evaluate the
performance of the method in a speaker-independent setup on both the source
and the new language and show its robustness across five languages belonging
to different linguistic strains.

###### keywords:

semi-supervised domain adaptation, speech emotion recognition, cross-lingual,
semi-supervised learning

††journal: Journal of XXX

##  1 Introduction

Speech emotion recognition (SER) is a fundamental aspect of computational
paralinguistics as it concerns the analysis of the non-verbal elements of
speech [1]. SER, which aims to infer the emotional state of a speaker [2],
could support a wide range of applications, including human-computer
interaction [1], healthcare [3], and public safety [4]. Existing SER models
have achieved satisfactory level results for valence/arousal estimation [5]
and emotion classification [6] when the training and test data are from the
same corpus. However, the performance of the previous models degrades when
applied to new corpora of same/different languages due to the domain shift
[7].

Numerous approaches have been proposed to reduce the domain shift problem for
cross-corpus or cross-lingual SER which consists in the elimination or
reduction of the difference between the source and target data distribution
[8, 9]. Supervised Domain Adaptation (SDA) methods for SER exploit labeled
utterances of the target corpus to adapt the recognition model to work
properly on the new set of data [10, 11, 9]. The previous methods require the
new language utterances to be labeled, which may not be possible as their
collection is expensive. Therefore, a more practical solution is Unsupervised
Domain Adaptation (UDA) which only demands unlabeled utterances from the new
language. Many UDA methods try to reduce the distribution shift between the
source and target languages [12, 8, 13, 14].

In this paper we formulate the cross-lingual SER as a Semi-Supervised Learning
(SSL) problem. This scenario assumes that there are few labeled and many
unlabeled utterances for the new language. We first train a deep learning
based SER model on the source language dataset in which all utterances are
annotated with the emotion label (see Figure 1(a)). The SER model is then
adapted to a new language for which the emotion of most training utterances is
unknown. The labeled data of the first language are available (see Figure
1(b)). Pseudo-labeling is adopted to generate labels for the unlabeled
utterances and guide the learning process. Unlike most cross-lingual SER
methods which focus on the binary classification of valence, our approach
deals with the prediction of five emotion categories. In our experiments we
consider English as the source language since it is the most widespread
language in the world [15]. To summarize, the contributions of this paper are:

  * 1.

A cross-lingual Speech Emotion Recognition (SER) framework spanning five
languages;

  * 2.

A Semi-Supervised Learning (SSL) based cross-lingual SER method for emotion
categorization;

  * 3.

Two different approaches for generating pseudo-labels are investigated.

The rest of the paper is organized as follows. Section 2 introduces some
previous works on cross-lingual emotion recognition. In Section 3 we formalize
the semi-supervised learning based cross-lingual SER. The proposed method is
described in Section 4. The experimental setup and results analysis are
presented in Section 5 and finally we conclude in Section 7.

(a) Single-language SER

(b) Train on a new language

Figure 1: Pipeline of the proposed cross-lingual SER method. (a) The model is
first trained on the source language and (b) it is then adapted to the new
language for which a few labeled utterances are available.

##  2 Related work

Cross-lingual speech emotion recognition methods involve the use of two
languages, the source language for which the emotion information is available
for all the samples and the target language, for which only few labeled
samples are available. The aim of cross-lingual SER is then to learn from the
source language and extend the learned knowledge on the unlabelled samples of
the target language. This propagation is an active process that involves the
use of the few samples available and, if present, of auxiliary information
available for both languages. At each learning episode, unlabeled samples of
the target language which are believed to belong to an emotional class, are
labeled. This process is called pseudo-labelling and in turn will support
future learning episodes for evaluating the remaining samples. The choice of
the acoustic features is quite important and must be kept into account [9].

The primary taxonomy of cross-lingual SER methods is given by the strategy
used to convey knowledge on the new language. Two are the main schemas used in
the state of art. The first one is the use of auxiliary information that is
available on both languages to learn a shared feature space. In this context,
the emotion recognition task and the side task are performed simultaneously
through a joint-training. Cai et al. [8] proposed a neural network with two
branches, leading respectively to emotion and language classification. During
training, the first branch is trained only with the source language corpus
with emotion labels while the second by both languages. This training schema
allows to exploit all existing information of the first and the second
language to create a shared feature space. Gradient Reversal Layer (GRL) is
adopted to force the features to be meaningful for the primary task (emotion
classification) and at the same time to be indistinguishable for the auxiliary
task (language classification). Performance are measured in terms of valence
and arousal on Urdu, Estonian, IEMOCAP, Persian, and German (4 emotions). Li
et al. [13] proposed Neural Network with Pseudo Multilabel (NNPM), an SSDA
memory-based system called Neural Network with Pseudo Multilabel (NMPM). In
first place, they use a siamese network with self attention for projecting
source and target utterances in a learned feature space. Then, the source-
domain features are dynamically stored in the dynamic external memory. Emotion
similarity is gained through cosine distance between features in the memory
and of the target utterance. Pseudo-labels are given to the target domain
utterances based on the similarity score. Hard negative sample mining strategy
is used to improve the learning whereas the features result less
representative. Performance measured with weighted and unweighted accuracy.
Ocquaye et al. [14] exploited joint training to perform SSDA. A neural network
with one common branch and a set of task-specific branches is proposed. Two
branches perform emotion recognition respectively on the source samples and
the pseudo-labeled target samples. The evaluation has been conduced on SAVEE,
IEMOCAP, EMO‐DB, FAU‐AIBO (German), and EMOVO for valence classification.

The second strategy widely used in the context of cross-lingual SER is the use
of adversarial training. This technology showed great ability in domain
transfer and thus is very effective in this scenario. Latif et al. [12]
proposed a method for learning a language-independent emotion recognition
feature vector in the context of UDA. This system is based on generative
adversarial networks (GANs) as this technology has shown great potential in
learning the underlying data distribution. Specifically, the proposed method
has two generators to project respectively the source and the target utterance
in a common feature space that is later evaluated through a critic. The
feature space is then constrained to carry emotion information through a
classification loss. In other words, the adversarial loss makes the feature
space homogeneous both for the source and the target languages while the
classification loss makes the features meaningful for the task of emotion
recognition. Performance assessed on EMOVO, SAVEE, Urdu, and EMO-DB for
valence classification. Domain Adversarial Neural Network (DANN) is also a
method by generating a domain invariant feature representation that reduces
the gap between features in the source and target domain [16]. However, the
effectiveness of domain adversarial training is highly correlated with the
distribution of two databases so that adversarial attacks and instabilities
may occur during training if the data points are sharply different from each
other. Aggregate multi-task Learning (AL) methodology for unsupervised cross-
lingual is another approach to improve the generalization ability of the model
by incorporating information on gender and naturalness [17]. An alternative
approach is proposed by training to learn the emotion similarities from source
domain samples through few-shot learning adapted to the target domain.
Extending the work of Sung et al. [18], Ahn et al. [19] presented Few-shot
Learning and Unsupervised Domain Adaptation (FLUDA), which aims to train an
embedding and a metric module that respectively project the utterances in a
meaningful shared feature space and learn the differences among classes. The
embedding and metric module are optimized to predict class similarity for each
episode by exploiting few samples composing the support set and pseudo-labels
assigned in the previous episode. During training, an auxiliary module is used
to determine weather the labeled sample is real or pseudo-labeled. The
proposed method estimates four caterogical emotions (neutral, happy, sad and
angry) and uses IEMOCAP and CREMA-D as source corpora while MSP-IMPROV, EMO-DB
or KME were used as target corpus. However, samples in few-shot learning
significantly depend on the choice of support set that might make the
challenging to apply to practical settings. Moreover, because of the strong
assumption of the support set sampled uniformly from a single distribution, it
leads to the unstable number of samples selected for each class during the
training process. In an attempt to provide a solution to the previous
challenges. Zhou et al. [11] used adversarial network to perform SSDA.
Specifically, a GAN is modified such that the generator projects the utterance
in a feature space carring emotion information and the critic has to determine
which emotion class belongs the input feature and the used language. This
stage is trained with the source language for which the labels are known. In
second place the critic is frozen and the encoder is adapted to the new
language. This second step forces the encoder to adapt and generate compatible
features with the ones obtained in the first step. The method has been
benchmarked on EMO-DB and Aibo in terms of positive and negative emotions.

##  3 Problem formulation

We formulate our cross-lingual SER as the following domain adaptation task. We
have a source language corpus with $N_{s}$ labeled utterances as source
domain, $\mathcal{D}_{s}=\\{(x^{s}_{i},y^{s}_{i})\\}_{i=1}^{N_{s}}$, and a new
language corpus, $\mathcal{D}_{t}$, as target domain. The new language corpus,
$\mathcal{D}_{t}=\\{\mathcal{U}_{t}\cup\mathcal{K}_{t}\\}$, consists of a set
$N_{u}$ of unlabeled utterances
$\mathcal{U}_{t}=\\{x^{t}_{i}\\}^{N_{u}}_{i=1}$, and a set $N_{k}$ of labeled
utterances $\mathcal{K}_{t}=\\{x^{t}_{i},y^{t}_{i}\\}^{N_{k}}_{i=1}$.
Utterances $x^{s}_{i}$ and $x^{t}_{i}$ are elements of $\mathbb{R}^{F\times
T}$, where $F$ and $T$ are the number of frequency bins and the number of time
frames, respectively. The utterance labels $y^{s}_{i}$ and $y^{t}_{i}$ are
$C$-dimensional one-hot encoding vectors where $C$ is the number of emotion
categories within the corpora. We consider that the source and target corpora
contain the same number $C$ of emotion categories.

Our goal is to learn a reliable emotion classifier on $\mathcal{D}_{s}$,
$\mathcal{U}_{t}$, and $\mathcal{K}_{t}$, which preserves performance on
source language $\mathcal{D}_{s}$ and generalizes well on $\mathcal{D}_{t}$.

##  4 Method

The proposed cross-language SER method is based on two modules: a recognition
and adaptation module. The SER recognition backbone is composed of an
utterance embedding model followed by an emotion speech classifier. The
adaptation module is based on a pseudo-labeling strategy that allows to infer
labels for unlabeled utterances.

###  4.1 SER backbone

The utterance embedding model is based on BYOL-S [6], that is a variant of the
BYOL-A model presented by Niizumi et al. in 2021 [20]. BYOL-A is a CNN model
for audio representation inspired by the Bootstrapping Your Own Latent (BYOL)
model initially proposed for self-supervised image classification [21]. It is
trained on the AudioSet [22] dataset and it achieved state-of-the-art results
on a range of audio classification tasks. BYOL-S (S denoting speech) is an
adaptation of the BYOL-A to the case of only speech utterances. BYOL-S is
currently the state of the art in SER [6]. The model accepts inputs of
variable length and returns a single 1024-dimensional embedding per input
utterance.

In [6], BYOL-S has been used as a pre-trained feature extractor and several
classifiers (e.g. logistic regression, support vector machine, and random
forests) have been exploited to map the embedding into an emotion category. In
this paper, instead, the emotion speech classifier consists of a linear layer
that maps the 1024-dimensional embedding in $C$ emotion categories. The SER
backbone is then finetuned to be adapted to the new languages.

###  4.2 Pseudo-labeling

The proposed model $h_{\theta}(x)$ is trained end-to-end using a set of $N$
training utterances belonging to the joint domain
$\mathcal{D}=\\{\mathcal{D}_{s}\cup\mathcal{D}_{t}\\}$. $\mathcal{D}$ contains
a set of $N_{s}+N_{k}$ labeled utterances
$\\{\mathcal{D}_{s},\mathcal{K}_{t}\\}$ coming from both the source and the
target language and a set of $N_{u}$ unlabeled utterances $\mathcal{U}_{t}$
originating from the target language only, for which we want to estimate a
label. To this purpose, an SSL approach is used to generate pseudo-labels
$\tilde{y}$. This operation is repeated at each step to take advantage of the
knowledge learned in previous steps and results in the creation of a hybrid
dataset $\tilde{\mathcal{D}}$ composed by real samples $(x_{i},y_{i})$ and
samples with generated ground truth $(x_{i},\tilde{y}_{i})$.

In our method the parameters $\theta$ of the model are optimized using
categorical cross-entropy:

$\mathcal{L}_{CE}=-\sum_{i=1}^{N}y_{i}\mathrm{log}(h_{\theta}(x_{i})),$ (1)

where $h_{\theta}(x)$ are the softmax probabilities predicted by the model,
$\mathrm{log(\cdot)}$ is applied element-wise and $y_{i}$ can be a real or
pseudo label. A key decision is how to generate the pseudo-labels $\tilde{y}$
for the $N_{u}$ unlabeled utterances. In this paper, we experiment with the
use of hard pseudo-labels (i.e. one-hot vectors) and soft pseudo-labels.

####  4.2.1 Hard pseudo-labels

In this approach hard pseudo-labels are directly obtained from network
predictions. Let $p_{i}$ be the probability outputs of our trained
$h_{\theta}$ model for the utterance. Using the probability vector, the
pseudo-label for the utterance $x_{i}$ corresponds to
$\tilde{y}=\mathrm{arg\;max}(p_{i})$. We select the subset of pseudo-labels
which are less noisy to limit the confirmation bias, i.e. the overfitting of
incorrect pseudo-labels predicted by the model. In particular, we select only
the pseudo-labels corresponding to the high-confidence predictions. Let
$g=\\{g_{b}\\}_{b=1}^{B}$ be a binary vector representing the selected pseudo-
labels in a mini-batch $B$. This vector is obtained as follows:

$g_{b}=\mathbbm{1}\left[\mathrm{max}(p_{b})\geq\tau\right],$ (2)

where $\tau\in(0,1)$ is a confidence threshold.

####  4.2.2 Soft pseudo-labels

We investigate the use of soft pseudo-label inspired by [23] since it has
demonstrated in some cases to perform better than hard pseudo-labels [24].

Let $p_{i}$ be the probability outputs of our trained $h_{\theta}$ model for
the utterance $x_{i}$. Two regularization terms are used to improve
convergence. The first regularization term discourages the model to assign all
samples to a single class by adding:

$R_{A}=\sum_{c=1}^{C}p_{c}\mathrm{log}\left(\frac{p_{c}}{\bar{h}_{c}}\right),$
(3)

where $p_{c}$ is the prior probability distribution for class $c$ assumed as a
uniform distribution $p_{c}=1/C$ and $\bar{h}_{c}$ denotes the mean softmax
probability of the model for class $c$ across batch samples.

The second regularization is the average per-sample entropy ($R_{H}$ stands
for entropy regularization) that forces the probability distribution to peak
on a single class:

$R_{H}=-\sum_{i=1}^{B}\sum_{c=1}^{C}h_{\theta}^{c}(x_{i})\mathrm{log}(h_{\theta}^{c}(x_{i})),$
(4)

where $h_{\theta}^{c}(x_{i})$ denotes the $c$ class value of the softmax
output $h_{\theta}(x_{i})$ and it is estimated on a mini-batch $B$. The total
loss is the following:

$\mathcal{L}=\mathcal{L}_{CE}+\lambda_{A}R_{A}+\lambda_{H}R_{H},$ (5)

where $\lambda_{A}$ and $\lambda_{H}$ control the contribution of each
regularization term.

To limit the confirmation bias problem, we exploit the mixup data augmentation
technique proposed in [25]. It combines data augmentation with label smoothing
to reduce the confidence of the model on its predictions. Mixup trains on
convex combinations of sample pairs ($x_{p}$ and $x_{q}$) and corresponding
labels ($y_{p}$ and $y_{q}$):

$\displaystyle x=\alpha x_{p}+(1-\alpha)x_{q},$ (6) $\displaystyle y=\alpha
y_{p}+(1-\alpha)y_{q},$ (7)

where $\alpha$ is randomly sampled in the range $(0,1)$.

##  5 Experiments

###  5.1 Datasets

A summary of the datasets used for our experiments is presented in Table 1. We
consider seven speech emotion classification datasets in five languages: three
in English (RAVDESS, SAVEE and TESS), one in French (CaFE), German (EmoDB),
Italian (EMOVO), and Persian (ShEMO). In each dataset, speech samples have
three attributes: audio data (i.e., the raw waveform, in mono), speaker
identifier, and emotion label (e.g., angry, happy, sad). The datasets comprise
scripted and acted utterances and vary in size (i.e., number of utterances),
number of speakers, sample rate, and number of classes. All of them comprise
utterances in which the speaker acts a specific emotion.

Table 1: List of considered datasets for speaker emotion recognition.

Name | Spkrs | Emot. | SR (Hz) | Utter. | Lang. |  Avg. dur. (s) |  Tot. dur. (h)
---|---|---|---|---|---|---|---
CaFE [26] | 12 | 7 | 48,000 | 864 | French | 4.5 | 1.1
EMO-DB [27] | 10 | 7 | 16,000 | 535 | German | 2.8 | 0.4
EMOVO [28] | 6 | 7 | 48,000 | 588 | Italian | 3.1 | 0.5
RAVDESS [29] | 24 | 8 | 48,000 | 1,440 | English | 3.7 | 1.5
SAVEE [30] | 4 | 7 | 44,100 | 480 | English | 3.8 | 0.5
ShEMO [31] | 87 | 6 | 44,100 | 3,000 | Persian | 4.0 | 3.3
TESS [32] | 2 | 7 | 24,414 | 2,800 | English | 2.0 | 1.6

The considered datasets share the same five primary emotions, which are anger
(A), fear (F), happiness (H), neutral (N), and sadness (S). Following [9], in
this study we consider only utterances annotated with one of the previous five
emotions and discard the remaining utterances. Thus, the number of emotion
categories is $C=5$. TESS, SAVEE, and RAVDESS datasets are merged to obtain a
large English language dataset. A summary of the distributions of utterances
by emotion for each language is shown in the Table 2.

Table 2: Utterance distribution of the selected datasets across emotion classes. | A | F | H | N | S | Total
---|---|---|---|---|---|---
English (RAVDESS, SAVEE, TESS) | 652 | 652 | 652 | 652 | 616 | 3224
French (CaFE) | 144 | 144 | 144 | 144 | 72 | 648
German (EMO-DB) | 127 | 69 | 71 | 79 | 62 | 408
Italian (EMOVO) | 84 | 84 | 84 | 84 | 84 | 420
Persian (ShEMO) | 1059 | 38 | 201 | 1028 | 449 | 2775

###  5.2 Experimental setup

Each dataset was split into training, validation, and testing sets to
respectively train, optimize and evaluate task-specific emotion speech
classifiers. Following SERAB [6], each dataset is split into 60% training, 20%
validation, and 20% testing sets. Each data partition is speaker-independent,
i.e., the sets of speakers included in each part is mutually disjoint.

Importantly, the utterances of the various datasets have different sampling
rates, for this reason they were all resampled to 16kHz before any processing.
During training a sequence of 2 seconds is randomly sampled from the whole
utterance for augmentation, while the whole utterance is used at testing time.
Voice Activity Detection (VAD) is not used in training and testing. We
converted all utterances to a log-scaled mel spectrogram with a window size of
64ms, hop size of 10ms, and mel-spaced frequency bins $F=64$ in the range
60–7,800 Hz. Each spectrogram is normalized by subtracting the mean and
dividing by the estimated standard deviation for the frames of the
spectrogram. We initialize the backbone BYOL-S with the parameters learned on
the AudioSet dataset [22], while the linear layer for emotion speech
classification is randomly initialized.

All experiments are run three times with different random seeds, and the
unweighted accuracy is chosen as our evaluation criterion.

####  5.2.1 Hyperparameters

In our experiments, we train the model for 100 epochs using the Adam optimizer
with an initial learning rate equal to $10^{-3}$ which decays by a factor of
0.95 every 10 epochs, a batch of 100 utterances, and exponential decay rates
$\beta_{1}$ and $\beta_{2}$ equal to 0.9 and 0.999. We experiment with
different values of $\tau$ (see 6.3.1 for a study of this hyperparameter)
which lead to the choice of $\tau=0.50$, but do not attempt a careful
adjustment of the regularization weights $\lambda_{A}$ and $\lambda_{H}$ and
simply set them to 0.8 and 0.4 as done in [24].

##  6 Results

In this section we describe the results achieved for different configurations.
In all the experiments we consider English as the first language while the
other languages were chosen one at a time as the second language.

###  6.1 Cross-lingual results

In this section we measure the performance obtainable by our method on a
totally unknown new language. These results give an idea of the worst
accuracy, defined as the lower bound, achievable for cross-lingual SER. To
this end, we perform experiments using only the training set of the source
language (i.e. English) and testing on the new language data. Table 3 reports
accuracy results for the four new languages. As it is possible to see the
highest accuracy equal to 60.56% is achieved for the German language while the
lowest accuracy of 18.28% is obtained for Persian. This results depends on the
fact that as stated by several linguistic distances [33, 34, 35] Persian
belongs to a linguistic strain very different from the English one. Therefore,
it is conceivable that learning on utterance in Persian will produce an
important gain in performance.

Table 3: SER accuracy on new languages by training only on the source language (English). |  Accuracy on the new language
---|---
French | 54.32
German | 60.56
Italian | 49.28
Persian | 18.28

###  6.2 Multi-lingual results

In this section, multi-lingual SER experiments are performed following a
supervised training using all the data of both the source and new language.
The results obtained give the upper bound for cross-lingual SER, i.e. the best
accuracy obtainable for each language having all the labels for the new
language available. Since we want to evaluate whether the SER classifier
generalizes on the new language but also if it preserves the performance on
the source language, in the Table 4 we show the accuracy on the source
language (i.e. first column of the table) and the accuracy on the new language
(i.e. second column of the table).

From the results obtained it is possible to make various considerations.
First, as expected, there is an increase in performance by using the training
data of both the source language and the new one. This result is particularly
evident for the Persian, where the performance increases from 18.28% to
86.43%. Second, the performance on English, which is the source language, does
not considerably degrade by adding the training data of the new language.
Starting from the 79.13% of accuracy obtained by training only on English, we
obtain a loss of about 19% with French, of the 4% with German, of about 5%
adding Italian, and about of the 8% adding Persian.

Table 4: SER accuracy on new languages thanks to training on the combination of source and new language training sets. |  Accuracy on the source language |  Accuracy on the new language
---|---|---
English | 79.13 | –
English & French | 60.08 | 58.02
English & German | 75.04 | 73.24
English & Italian | 73.78 | 59.28
English & Persian | 71.26 | 86.43

###  6.3 Pseudo-labeling results

In this subsection, we report the results for cross-lingual SER by varying the
number of labeled utterances for the new language from 0 to 100. In these
experiments all the utterances for the source language are labeled. Figure 2
shows the performance achieved by using hard pseudo-labels (see Fig. 2(a)) and
soft pseudo-labels (see Fig. 2(b)). Each colored bar represents the accuracy
of a given pseudo-label approach with a given number of available labels for
the new language. Black narrow bars represent the accuracy on the source
language.

(a) Results for hard pseudo-labels

(b) Results for soft pseudo-labels

Figure 2: SER accuracy by varying the number of labeled utterances for the new
language and using (a) hard pseudo-labels and (b) soft pseudo-labels for the
unlabeled utterances. Black bars indicate the accuracy on the source language
(i.e. English).

Several considerations can be made. First, overall hard pseudo-labels obtain
better performance than soft pseudo-labels. Second, soft pseudo-labels adapt
less to the new language while better preserving performance on the source
language. Third, for both pseudo-label approaches and for all languages,
having more available labels for the new language allows to achieve higher
accuracy. Pseudo-label approaches can not permit to reach the upper bound in
any language but for German and Persian these approaches allow to achieve
better performance than the lower bound with an improvement of about 15% and
57% respectively. Unfortunately, in the case of Italian and French languages
there are no improvements with a decrease of about 1% and 9% respectively.

####  6.3.1 Effect of tau

Among all the hyperparameters, the confidence threshold $\tau$ used for hard
pseudo-labels (see Section 4.2.1) is the one that needs to be carefully tuned.
This subsection studies the effect $\tau$ on our hard pseudo-labeling
approach. Table 5 reports the cross-lingual SER results by considering 100
labeled utterance for the new language and varying $\tau$. As can be seen from
the results, the choice of the best $\tau$ does not generalize to all
languages. However, for 0.50 the best performance is obtained in most cases
(gap compared to the other values very evident for German and Italian) and it
was chosen for this reason.

Table 5: SER accuracy by using 100 labeled utterances for the new language and varying $\tau$ values for hard pseudo-labeling. Best result for each language is in bold. | $\tau=0.50$ | $\tau=0.65$ | $\tau=0.85$
---|---|---|---
English | 51.99 | 49.76 | 55.25
French | 45.68 | 49.59 | 48.35
English | 59.82 | 61.29 | 60.60
German | 75.59 | 69.48 | 69.01
English | 61.05 | 55.80 | 62.18
Italian | 48.10 | 41.67 | 46.90
English | 58.77 | 50.16 | 56.69
Persian | 75.99 | 74.15 | 73.13

###  6.4 Discussion

Figure 3 summarizes the results achieved in the different configurations
evaluated for the recognition of emotions on new languages. The aim is to
highlight the gap between training the method by having all labeled data for
the new language and the use of pseudo-labels. In the chart we compare the
performance for (i) the cross-lingual experiment (results collected from the
Table 3), (ii) the multi-lingual experiment (accuracy reported in the Table
4), and (iii) the SSL cross-lingual experiment based on the use of hard
pseudo-labels having 100 labeled utterances for the new language (see Figure
2(a)). Multi-lingual SER with all labeled utterances, as presumable, results
in a noticeable increase in performance compared to the cross-lingual SER.
This increase is particularly significant for Persian (+60%), a language that
has very different linguistic traits from those of English and for this reason
the adaptation of the model is very important. Cross-lingual SSL based on hard
pseudo-labels allows to improve the performance of the cross-lingual
configuration for all the languages considered with the exception of French
and Italian. This result depends on the linguistic distance between the source
and target languages. Specifically, the shorter the linguistic distance
between the source language and the target language, the lower the gain
obtained with cross-language SSL. According to [33], French and Italian have a
lower linguistic distance from English than both German and Persian. This
aspect is reflected in the performances obtained for the estimation of
emotions in which the cross-lingual result on French and Italian is very
similar to the intra-corpus performance, respectively equal to 61.28% and
56.12%. Consequently we can claim that cross-lingual SSL is not bad on these
two languages, but the cross-lingual model on the source language is very
good.

Figure 3: Comparison of the results on the new languages between the cross-
lingual, multi-lingual and the best SSL (hard pseudo-labels considering 100
labeled utterances).

###  6.5 Comparison with state-of-the-art

In this section, we compare the performance of the proposed method with four
recent state-of-the-art methods, Aggregate multi-task Learning (AL) [17],
Domain Adversarial Neural Network (DANN) [16], Few-shot Learning and
Unsupervised Domain Adaptation (FLUDA) [19], and Neural Network with Pseudo
Multilabel (NNPM) [13]. The above methods are trained for discriminating 4
emotion categories namely anger, happiness, neutral, and sadness.
Implementations of the methods are not publicly available, we do not implement
these systems but instead cite their performance on the EMO-DB dataset from
the original papers or re-implementations. Therefore, we exclude from the EMO-
DB dataset the utterances labeled with emotion of fear and retrain our method.
In particular, we retrain our method without using labeled utterances of the
new language and taking advantage of hard pseudo-labels. The resulting
accuracy on English corresponds to 74.02% and the accuracy on German is equal
to 66.7%. Table 6 summarizes the comparison with the other methods on German.

The results show that the proposed method achieves better performance than
recent state-of-the-art methods. More specifically, our method outperforms the
second best method also based on pseudo-labeling, namely NNPM, with an
improvement in relative accuracy of 16%. We get better performance than the
multi-task learning method, i.e. AL, with 14% better accuracy, and the
unsupervised cross-corpus SER model based on few-shot learning (i.e., FLUDA)
of an increase of 32%.

Table 6: Comparison with other state-of-the-art methods considering the EMO-DB corpus as the new language. Method | EMO-DB
---|---
DANN [19] | 28.5
FLUDA [19] | 34.9
AL [19] | 42.5
NNPM [13] | 50.6
Our | 66.7

##  7 Conclusions

In the cross-lingual speech emotion recognition domain it is a common scenario
where there are many labeled utterances for the English language, instead, the
availability of a few labeled utterances and many unlabeled for other
languages. Based on the previous consideration, in this article we proposed a
SSL approach for cross-lingual speech emotion recognition. It consists of a
CNN to classify an utterance into an emotion category. For SSL, we
experimented with the use of hard and soft pseudo-labels for unlabeled
utterances. The proposed method is evaluated using English as source language
and four different languages (French, German, Italian and Persian) as new
languages. It is revealed that the use of hard pseudo-labels over soft allows
for better results on the new language but at the expense of performance on
the source language.

## References

  * [1] B. Schuller, A. Batliner, Computational paralinguistics: emotion, affect and personality in speech and language processing, John Wiley & Sons, 2013.
  * [2] M. El Ayadi, M. S. Kamel, F. Karray, Survey on speech emotion recognition: Features, classification schemes, and databases, Elsevier Pattern recognition 44 (3) (2011) 572–587.
  * [3] V. Tumanova, C. Woods, Q. Wang, Effects of physiological arousal on speech motor control and speech motor practice in preschool-age children who do and do not stutter, Journal of Speech, Language, and Hearing Research 63 (10) (2020) 3364–3379.
  * [4] I. Lefter, C. M. Jonker, Aggression recognition using overlapping speech, in: International Conference on Affective Computing and Intelligent Interaction (ACII), IEEE, 2017, pp. 299–304.
  * [5] Z. Xiao, D. Wu, X. Zhang, Z. Tao, Speech emotion recognition cross language families: Mandarin vs. western languages, in: International Conference on Progress in Informatics and Computing (PIC), IEEE, 2016, pp. 253–257.
  * [6] N. Scheidwasser-Clow, M. Kegler, P. Beckmann, M. Cernak, Serab: A multi-lingual benchmark for speech emotion recognition, in: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2022, pp. 7697–7701.
  * [7] S. M. Feraru, D. Schuller, et al., Cross-language acoustic emotion recognition: An overview and some tendencies, in: International Conference on Affective Computing and Intelligent Interaction (ACII), IEEE, 2015, pp. 125–131.
  * [8] X. Cai, Z. Wu, K. Zhong, B. Su, D. Dai, H. Meng, Unsupervised cross-lingual speech emotion recognition using domain adversarial neural network, in: International Symposium on Chinese Spoken Language Processing (ISCSLP), IEEE, 2021, pp. 1–5.
  * [9] G. Tamulevičius, G. Korvel, A. B. Yayak, P. Treigys, J. Bernatavičienė, B. Kostek, A study of cross-linguistic speech emotion recognition based on 2d feature spaces, MDPI Electronics 9 (10) (2020) 1725.
  * [10] M. Neumann, et al., Cross-lingual and multilingual speech emotion recognition on english and french, in: International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2018, pp. 5769–5773.
  * [11] H. Zhou, K. Chen, Transferable positive/negative speech emotion recognition via class-wise adversarial domain adaptation, in: International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2019, pp. 3732–3736.
  * [12] S. Latif, J. Qadir, M. Bilal, Unsupervised adversarial domain adaptation for cross-lingual speech emotion recognition, in: International Conference on Affective Computing and Intelligent Interaction (ACII), IEEE, 2019, pp. 732–737.
  * [13] J. Li, N. Yan, L. Wang, Unsupervised cross-lingual speech emotion recognition using pseudo multilabel, in: Automatic Speech Recognition and Understanding Workshop (ASRU), IEEE, 2021, pp. 366–373.
  * [14] E. N. Ocquaye, Q. Mao, Y. Xue, H. Song, Cross lingual speech emotion recognition via triple attentive asymmetric convolutional neural network, International Journal of Intelligent Systems 36 (1) (2021) 53–71.
  * [15] Berlitz, The most spoken languages in the world, accessed on 29-jun-2022 (2021).
URL https://www.berlitz.com/blog/most-spoken-languages-world

  * [16] M. Abdelwahab, C. Busso, Domain adversarial for acoustic emotion recognition, IEEE/ACM Transactions on Audio, Speech, and Language Processing 26 (12) (2018) 2423–2435.
  * [17] J. Kim, G. Englebienne, K. P. Truong, V. Evers, Towards speech emotion recognition “in the wild” using aggregated corpora and deep multi-task learning, in: Interspeech, 2017, pp. 1113–1117.
  * [18] F. Sung, Y. Yang, L. Zhang, T. Xiang, P. H. Torr, T. M. Hospedales, Learning to compare: Relation network for few-shot learning, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1199–1208.
  * [19] Y. Ahn, S. J. Lee, J. W. Shin, Cross-corpus speech emotion recognition based on few-shot learning and domain adaptation, IEEE Signal Processing Letters 28 (2021) 1190–1194.
  * [20] D. Niizumi, D. Takeuchi, Y. Ohishi, N. Harada, K. Kashino, Byol for audio: Self-supervised learning for general-purpose audio representation, in: International Joint Conference on Neural Networks (IJCNN), IEEE, 2021, pp. 1–8.
  * [21] J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Gheshlaghi Azar, et al., Bootstrap your own latent-a new approach to self-supervised learning, Advances in Neural Information Processing Systems 33 (2020) 21271–21284.
  * [22] J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, M. Ritter, Audio set: An ontology and human-labeled dataset for audio events, in: International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2017, pp. 776–780.
  * [23] E. Arazo, D. Ortego, P. Albert, N. E. O’Connor, K. McGuinness, Pseudo-labeling and confirmation bias in deep semi-supervised learning, in: International Joint Conference on Neural Networks (IJCNN), IEEE, 2020, pp. 1–8.
  * [24] D. Tanaka, D. Ikami, T. Yamasaki, K. Aizawa, Joint optimization framework for learning with noisy labels, in: Conference on Computer Vision and Pattern Recognition, IEEE, 2018, pp. 5552–5560.
  * [25] H. Zhang, M. Cisse, Y. N. Dauphin, D. Lopez-Paz, Mixup: Beyond empirical risk minimization, in: International Conference on Learning Representations, 2018.
  * [26] P. Gournay, O. Lahaie, R. Lefebvre, A canadian french emotional speech dataset, in: Multimedia Systems, ACM, 2018, pp. 399–402.
  * [27] F. Burkhardt, A. Paeschke, M. Rolfes, W. F. Sendlmeier, B. Weiss, A database of german emotional speech, in: Interspeech, 2005, pp. 1517–1520.
  * [28] G. Costantini, I. Iaderola, A. Paoloni, M. Todisco, Emovo corpus: an italian emotional speech database, in: International Conference on Language Resources and Evaluation (LREC 2014), European Language Resources Association (ELRA), 2014, pp. 3501–3504.
  * [29] S. R. Livingstone, F. A. Russo, The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english, PloS one 13 (5) (2018) e0196391.
  * [30] W. Wang, Machine Audition: Principles, Algorithms and Systems: Principles, Algorithms and Systems, IGI Global, 2010.
  * [31] O. M. Nezami, P. J. Lou, M. Karami, Shemo: a large-scale validated database for persian speech emotion detection, Language Resources and Evaluation 53 (1) (2019) 1–16.
  * [32] M. K. Pichora-Fuller, K. Dupuis, Toronto emotional speech set (TESS) (2020).  doi:10.5683/SP2/E8H2MF.
URL https://doi.org/10.5683/SP2/E8H2MF

  * [33] B. R. Chiswick, P. W. Miller, Linguistic distance: A quantitative measure of the distance between english and other languages, Journal of Multilingual and Multicultural Development 26 (1) (2005) 1–11.
  * [34] F. Petroni, M. Serva, Language distance and tree reconstruction, Journal of Statistical Mechanics: Theory and Experiment 2008 (08) (2008) P08012.
  * [35] P. Gamallo, J. R. Pichel, I. Alegria, From language identification to language distance, Physica A: Statistical Mechanics and its Applications 484 (2017) 152–162.