[19], where Ivić supposed the condition $a(n)\ll M^{\varepsilon}$ for $M<n\leq
2M.$ However we see easily from Ivić’s argument that this condition can be
relaxed to the condition (8.36).

######  Lemma 12.

Let $0\leq\theta\leq 1/3$ be a fixed real number and let $\mathfrak{L}(s)$ be
an element of ${\cal S}_{real}^{\theta}$ with degree 2. Then we have

(8.37) $\int_{0}^{T}|\mathfrak{L}(\sigma+it)|^{1/(1-\sigma)}dt\ll
T^{1+\varepsilon}$

for $1/2<\sigma<2/3$. Especially we have

(8.38) $\int_{0}^{T}|\mathfrak{L}(5/8+it)|^{8/3}dt\ll T^{1+\varepsilon}.$

Furthermore if we assume

(8.39) $\int_{0}^{T}|\mathfrak{L}(1/2+it)|^{6}dt\ll T^{2+\varepsilon},$

we have

(8.40) $\int_{0}^{T}|\mathfrak{L}(\sigma+it)|^{2/(3-4\sigma)}dt\ll
T^{1+\varepsilon}$

for $1/2<\sigma<5/8$. Especially we have

(8.41) $\int_{0}^{T}|\mathfrak{L}(7/12+it)|^{3}dt\ll T^{1+\varepsilon}.$

###### Proof.

We follow the approach of Ivić [22]. Suppose $T<t_{1}<t_{2}<\cdots<t_{R}<2T$
such that

(8.42) $|\mathfrak{L}(\sigma+it_{r})|\geq V\gg T^{\varepsilon}$

and

$|t_{r}-t_{s}|\gg\log^{4}T\ (1\leq r\not=s\leq R).$

Suppose that

$\mathfrak{L}(s)=\sum_{n\geq 1}a(n)n^{-s},\ \ {\rm Re\,}s>1.$

From the formula (see (A7) of Ivić [19])

$e^{-v}=(2\pi i)^{-1}\int_{b-i\infty}^{b+i\infty}\Gamma(w)v^{-w}dw\ (b,v>0),$

we have

(8.43) $\sum_{n\geq 1}a(n)e^{-n/Y}n^{-s}=\frac{1}{2\pi
i}\int_{2-i\infty}^{2+i\infty}\Gamma(w)Y^{w}\mathfrak{L}(s+w)dw,$

where $1\ll Y\ll T^{A}$ is a parameter to be chosen later and $A>0$ is an
arbitrary constant. Let $s$ be any one of $\sigma+it_{r}$ with $\sigma>1/2$ in
(8.43). Moving the line of integration to $1/2-\sigma$ in (8.43) we get by the
residue theorem that

$\displaystyle\mathfrak{L}(s)=\sum_{n\geq 1}a(n)e^{-n/Y}n^{-s}-\frac{1}{2\pi
i}\int_{1/2-\sigma-i\infty}^{1/2-\sigma+i\infty}\Gamma(w)Y^{w}\mathfrak{L}(s+w)dw+O(T^{-B}),$

where $B>1$ is a large fixed constant. So we get

$\displaystyle\mathfrak{L}(\sigma+it_{r})\ll\left|\sum_{n\leq
Y\log^{2}T}a(n)e^{-n/Y}n^{-\sigma-
it_{r}}\right|+Y^{1/2-\sigma}\int_{-\log^{2}T}^{\log^{2}T}|\mathfrak{L}(1/2+it_{r}+iv)|dv+1,$

which implies that if (8.42) holds, we must have

(8.44) $\displaystyle V$ $\displaystyle\ll\left|\sum_{n\leq
Y\log^{2}T}a(n)e^{-n/Y}n^{-\sigma-it_{r}}\right|$ $\displaystyle\ll\log
T\max_{N_{1}\leq\frac{Y\log^{2}T}{2}}\left|\sum_{N_{1}<n\leq
2N_{1}}a(n)e^{-n/Y}n^{-\sigma-it_{r}}\right|$ or (8.45) $\displaystyle V$
$\displaystyle\ll
Y^{1/2-\sigma}\int_{-\log^{2}T}^{\log^{2}T}|\mathfrak{L}(1/2+it_{r}+iv)|dv.$

Let $\mathcal{A}_{1}$ be the set of points $t_{r}$ which satisfy (8.44) and
$\mathcal{A}_{2}$ the set of points $t_{r}$ which satisfy (8.45). We also let
$R_{1}=|\mathcal{A}_{1}|$ and $R_{2}=|\mathcal{A}_{2}|$, whence
$R=R_{1}+R_{2}$.

For $R_{1}$ we have

(8.46) $R_{1}\ll T^{\varepsilon}(Y^{2-2\sigma}V^{-2}+TV^{-2/(3-4\sigma)})$

by Lemma 11.

Now we are going to estimate $R_{2}$. Since $t_{r}\in\mathcal{A}_{2}$
satisfies (8.45) we have by Cauchy’s inequality

$\displaystyle V$ $\displaystyle\ll Y^{1/2-\sigma}\log
T\left(\int_{-\log^{2}T}^{\log^{2}T}|\mathfrak{L}(1/2+it_{r}+iv)|^{2}dv\right)^{1/2}.$

Squaring both sides and summing up over $t_{r}\in\mathcal{A}_{2}$, we get

(8.47) $\displaystyle R_{2}$ $\displaystyle\ll Y^{1-2\sigma}V^{-2}(\log
T)^{2}\sum_{t_{r}\in\mathcal{A}_{2}}\int_{-\log^{2}T}^{\log^{2}T}|\mathfrak{L}(1/2+it_{r}+iv)|^{2}dv$
$\displaystyle\ll Y^{1-2\sigma}V^{-2}(\log
T)^{2}\int_{T-\log^{2}T}^{2T+\log^{2}T}|\mathfrak{L}(1/2+iv)|^{2}dv$
$\displaystyle\ll Y^{1-2\sigma}V^{-2}T^{1+\varepsilon},$

where in the last inequality we used the mean square estimate (8.33) for
$\mathfrak{L}(1/2+it)$ of degree 2. Hence as a first estimate we get, from
(8.46) and (8.47),

(8.48) $R\ll
T^{\varepsilon}\left(Y^{2-2\sigma}V^{-2}+TV^{-2/(3-4\sigma)}+Y^{1-2\sigma}V^{-2}T\right).$

We shall derive another estimate for $R_{2}$. It is convenient to consider the
conditional case first, so we assume (8.39). This time we use Hölder’s
inequality in (8.45). Thus we have

$\displaystyle V\ll Y^{1/2-\sigma}(\log
T)^{5/3}\left(\int_{-\log^{2}T}^{\log^{2}T}|\mathfrak{L}(1/2+it_{r}+iv)|^{6}dv\right)^{1/6},$

and

(8.49) $\displaystyle R_{2}$ $\displaystyle\ll Y^{3-6\sigma}V^{-6}(\log
T)^{10}\sum_{t_{r}\in\mathcal{A}_{2}}\int_{-\log^{2}T}^{\log^{2}T}|\mathfrak{L}(1/2+it_{r}+iv)|^{6}dv$
$\displaystyle\ll Y^{3-6\sigma}V^{-6}(\log
T)^{10}\int_{T-\log^{2}T}^{2T+\log^{2}T}|\mathfrak{L}(1/2+iv)|^{6}dv$
$\displaystyle\ll Y^{3-6\sigma}V^{-6}T^{2+\varepsilon}.$

Hence (8.46) and (8.49) give the second estimate

(8.50) $R\ll
T^{\varepsilon}\left(Y^{2-2\sigma}V^{-2}+TV^{-2/(3-4\sigma)}+Y^{3-6\sigma}V^{-6}T^{2}\right).$

If $V\leq T^{(3-4\sigma)/4}$, we apply (8.48) with $Y=T$. Then

$\displaystyle R$
$\displaystyle\ll(Y^{2-2\sigma}V^{-2}+TV^{-\frac{2}{3-4\sigma}}+TY^{1-2\sigma}V^{-2})T^{\varepsilon}$
$\displaystyle\ll(T^{2-2\sigma}V^{-2}+TV^{-\frac{2}{3-4\sigma}})T^{\varepsilon}$
$\displaystyle\ll T^{1+\varepsilon}V^{-\frac{2}{3-4\sigma}}$

by recalling the condition $V\leq T^{(3-4\sigma)/4}.$ Next if $V\geq
T^{(3-4\sigma)/4}$, we apply (8.50) with
$Y=T^{2/(4\sigma-1)}V^{-4/(4\sigma-1)}+1$ and get

$\displaystyle R$
$\displaystyle\ll(Y^{2-2\sigma}V^{-2}+TV^{-\frac{2}{3-4\sigma}}+T^{2}Y^{3-6\sigma}V^{-6})T^{\varepsilon}$
$\displaystyle\ll(TV^{-\frac{2}{3-4\sigma}}+T^{\frac{4-4\sigma}{4\sigma-1}}V^{-\frac{6}{4\sigma-1}})T^{\varepsilon}$
$\displaystyle\ll T^{1+\varepsilon}V^{-\frac{2}{3-4\sigma}}$

if $T^{\frac{5-8\sigma}{4\sigma-1}}\leq
V^{\frac{4(5-8\sigma)}{(4\sigma-1)(3-4\sigma)}},$ which is true when $V\geq
T^{(3-4\sigma)/4}$ and $1/2<\sigma\leq 5/8.$ Summing up, if $1/2<\sigma\leq
5/8$,

$R\ll TV^{-2/(3-4\sigma)}$

holds true for all $V$, completing the proof of (8.40).

Next we shall prove (8.37). We use the fourth-power mean value theorem:

(8.51) $\int_{0}^{T}|\mathfrak{L}(1/2+it)|^{4}dt\ll T^{2+\varepsilon},$

which is known to be true unconditionally (see e.g. Kanemitsu,
Sankaranarayanan and Tanigawa [29]). By the similar way we have

(8.52) $R\ll
T^{\varepsilon}\left(Y^{2-2\sigma}V^{-2}+TV^{-2/(3-4\sigma)}+Y^{2-4\sigma}V^{-4}T^{2}\right)$

from (8.51). If $V\leq T^{1-\sigma}$, we take $Y=T$ in (8.48) and get

$R\ll T^{\varepsilon}\left(T^{2-2\sigma}V^{-2}+TV^{-2/(3-4\sigma)}\right)\ll
T^{1+\varepsilon}V^{-1/(1-\sigma)}.$

On the other hand if $V\geq T^{1-\sigma}$, we take $V=(TV^{-1})^{1/\sigma}$ in
(8.52) and get

$R\ll T^{2-2\sigma)/\sigma}V^{-2/\sigma}+TV^{-2/(3-4\sigma)}\ll
TV^{-1/(1-\sigma)}.$

Therefore the estimate

$R\ll T^{1+\varepsilon}V^{-1/(1-\sigma)}$

holds true for all $V$ for $1/2<\sigma<2/3$. This completes the proof of
(8.37).

Now we prove (8.34) and (8.35). Recall that ${\cal L}(s)={\cal L}_{1}(s){\cal
L}_{2}(s)$, where ${\cal L}_{1}(s)\in{\cal S}_{real}^{\theta}$ is a function
of degree $1$ and ${\cal L}_{2}(s)\in{\cal S}_{real}^{\theta}$ is a function
of degree $2.$ By the work of Kaczorowski and Perelli [27], it is known that
the functions of degree one in the Selberg class are the Riemann zeta-function
$\zeta(s)$ and the shifts $L(s+i\vartheta,\chi)$ of Dirichlet $L$-functions
attached to primitive characters $\chi$ with $\vartheta\in{\mathbb{R}}.$ So we
have

$\int_{0}^{T}|{\cal L}_{1}(5/8+it)|^{8}dt\ll T\log^{4}T$

and

$\int_{0}^{T}|{\cal L}_{2}(5/8+it)|^{8/3}dt\ll T^{1+\varepsilon}.$

Hence by Hölder’s inequality and (8.38)

$\displaystyle\int_{0}^{T}|{\cal L}(5/8+it)|^{2}dt$
$\displaystyle=\int_{0}^{T}|{\cal L}_{1}(5/8+it){\cal L}_{2}(5/8+it)|^{2}dt$
$\displaystyle\ll\left(\int_{0}^{T}|{\cal L}_{1}(5/8+it)|^{2\cdot
4}dt\right)^{1/4}\left(\int_{0}^{T}|{\cal L}_{2}(5/8+it)|^{2\cdot
4/3}dt\right)^{3/4}$ $\displaystyle\ll T^{1+\varepsilon}.$

Assume that (1.10) holds. Then by

$\int_{0}^{T}|{\cal L}_{1}(7/12+it)|^{6}dt\ll T^{1+\varepsilon}$

and

$\int_{0}^{T}|{\cal L}_{2}(7/12+it)|^{3}dt\ll T^{1+\varepsilon}$

(this comes from (8.41)), we have

$\int_{0}^{T}|{\cal L}(7/12+it)|^{2}dt\ll T^{1+\varepsilon}$

similarly. This completes the proof of Corollary 2.

##  9 Proofs of Theorem 2 and Theorem 3

We shall follow Heath-Brown’s argument [15] to prove Theorem 2 and Theorem 3.
We first quote some results from [15]. The following Hypothesis (H), Lemma 13
and Lemma 14 are Hypothesis (H), Theorem 5 and Theorem 6 of [15],
respectively.

Hypothesis (H): Let ${\cal M}(t)$ be a real valued function,
$a_{1}(t),a_{2}(t),\cdots,$ be continuous real valued functions of period 1,
and suppose there are non-zero constants $\gamma_{1},\gamma_{2},\cdots$ such
that

$\lim_{N\rightarrow\infty}\limsup_{T\rightarrow\infty}\frac{1}{T}\int_{0}^{T}\min\left(1,\left|{\cal
M}(t)-\sum_{n\leq N}a_{n}(\gamma_{n}t)\right|\right)dt=0.$

######  Lemma 13.

Suppose ${\cal M}(t)$ satisfies (H) and suppose that the constants
$\gamma_{i}$ are linearly independent over ${\mathbb{Q}}$. Suppose further

$\displaystyle\int_{0}^{1}a_{n}(t)dt=0\ \ \ (n\in{\mathbb{N}}),$
$\displaystyle\sum_{n=1}^{\infty}\int_{0}^{1}a_{n}^{2}(t)dt<\infty,$

and there is a constant $\mu>1$ such that

$\displaystyle\max_{t\in[0,1]}|a_{n}(t)|\ll n^{1-\mu},$
$\displaystyle\lim_{n\rightarrow\infty}n^{\mu}\int_{0}^{1}a_{n}^{2}(t)dt=\infty.$

Then ${\cal M}(t)$ has a distribution function $f(\alpha)$ with the properties
described in Theorem 2.

######  Lemma 14.

Suppose ${\cal M}(t)$ satisfies (H) and that

$\displaystyle\int_{0}^{T}|{\cal M}(t)|^{K}dt\ll T$

holds for some positive number $K$. Then for any real number $k\in[0,K),$ the
limit

$\displaystyle\lim_{T\rightarrow\infty}\frac{1}{T}\int_{0}^{T}|{\cal
M}(t)|^{K}dt$

exists.

Suppose $T\leq y\leq(1+\delta)T,$ define

$\displaystyle{\cal M}(y)$
$\displaystyle=\kappa_{0}^{-1}(2\pi/h)^{-\frac{d-1}{2}}y^{-\frac{d-1}{2}}E((2\pi)^{d}h^{-d}y^{d}),$
$\displaystyle a_{n}(y)$
$\displaystyle=\frac{q_{d}(n)}{n^{\frac{d+1}{2d}}}\sum_{m=1}^{\infty}\frac{a(nm^{d})}{m^{\frac{d+1}{2}}}\cos(2\pi
ym+c_{0}\pi),$ and $\displaystyle\gamma_{n}$ $\displaystyle=n^{1/d},$

where $q_{d}(n)$ is $1$ if $n$ is $d$-free and $0$ otherwise.

It is easy to see that $a_{n}(y)$ satisfies all conditions of Lemma 13 for any
fixed constant $1+1/d<\mu<3/2+1/2d.$

Now we suppose $M_{0}\asymp T^{\frac{d}{2d(1-\sigma^{*})-1}}$ such that
$E(M_{0})\ll T^{d\theta+\varepsilon}$ and $N_{0}=T^{d(2d-1)-\varepsilon}$. By
(8.4) we have

$\displaystyle{\cal M}(y)={\cal M}_{1}(y)+{\cal M}_{2}(y),$
$\displaystyle{\cal M}_{1}(y)=\sum_{1\leq n\leq
M_{0}}\frac{a(n)}{n^{\frac{d+1}{2d}}}\cos\left(2\pi yn^{1/d}+c_{0}\pi\right),$
$\displaystyle{\cal
M}_{2}(y)=\kappa_{0}^{-1}(2\pi/h)^{-\frac{d-1}{2}}y^{-\frac{d-1}{2}}\sum_{j=2}^{7}R_{j}((2\pi)^{d}h^{-d}y^{d}),$

where $R_{j}(y)$ is defined in Subsection 8.1.

It is easy to see that for any integer $N_{1}\leq M_{0}$ we have

$\displaystyle|{\cal M}(y)-\sum_{n\leq N_{1}}a_{n}(\gamma_{n}y)|$
$\displaystyle\ll\left|\sum_{n\leq
M_{0}}{\\!\\!}^{{}^{*}}\,\frac{a(n)}{n^{\frac{d+1}{2d}}}\cos(2\pi
yn^{1/d}+c_{0}\pi)\right|+\sum_{n\leq
N_{1}}\frac{1}{n^{\frac{d+1}{2d}}}\sum_{m>(M_{0}n^{-1})^{1/d}}\frac{|a(nm^{d})|}{m^{(d+1)/2}}+|{\cal
M}_{2}(y)|$ $\displaystyle\ll\left|\sum_{n\leq
M_{0}}{\\!\\!}^{{}^{\ast}}\,\frac{a(n)}{n^{\frac{d+1}{2d}}}\cos(2\pi
yn^{1/d}+c_{0}\pi)\right|+N_{1}^{1-1/d}M_{0}^{\theta+1/2d-1/2+\varepsilon}+|{\cal
M}_{2}(y)|,$

where $\sum^{*}$ means that $n$ has a $d$-free kernel great that $N_{1},$ and
for the sum involving $a(nm^{d})$ in the above formula we used the first
estimate of (1.4) and the condition $\theta<1/2-1/2d.$

Similarly to the approach of (8.9) we have

$\displaystyle\int_{T}^{(1+\delta)T}\left|\sum_{n\leq
M_{0}}{\\!\\!}^{{}^{*}}\,\frac{a(n)}{n^{\frac{d+1}{2d}}}\cos(2\pi
yn^{1/d}+c_{0}\pi)\right|^{2}dy$ $\displaystyle\ll T\sum_{n\leq
M_{0}}{\\!\\!}^{{}^{*}}\,\frac{a^{2}(n)}{n^{\frac{d+1}{d}}}+\sum_{\stackrel{{\scriptstyle
n\not=m}}{{n,m\leq
M_{0}}}}\frac{|a(n)a(m)|}{(nm)^{\frac{d+1}{2d}}|n^{1/d}-m^{1/d}|}$
$\displaystyle\ll
T\sum_{n>N_{1}}\frac{a^{2}(n)}{n^{\frac{d+1}{d}}}+M_{0}^{1-2/d}\log^{\eta}M_{0}$
$\displaystyle\ll TN_{1}^{-1/d+\varepsilon}+M_{0}^{1-2/d+\varepsilon}.$

Let ${\cal M}_{2}^{*}(y)=\sum_{j=2}^{7}R_{j}(y),$ then ${\cal M}_{2}(y)={\cal
M}_{2}^{*}((2\pi)^{d}h^{-d}y^{d}).$ From (8.10), (8.13)–(8.19) we get

$\int_{T}^{(1+\delta)T}{\cal
M}_{2}^{*2}(y)dy\ll\sum_{j=2}^{7}\int_{T}^{(1+\delta)T}R_{j}^{2}(y)dy\ll
T^{1-\frac{d+1-2d\sigma^{*}}{2d(1-\sigma^{*})-1}+\varepsilon},$

which combining a change of variable implies that

$\displaystyle\int_{T}^{(1+\delta)T}{\cal M}_{2}^{2}(y)dy\ll
T^{1/d-\frac{d+1-2d\sigma^{*}}{2d(1-\sigma^{*})-1}+\varepsilon}\ll T^{-c}$

for some positive constant $c>0.$

From the above estimates and Cauchy’s inequality we get

$\displaystyle\limsup_{T\rightarrow\infty}\frac{1}{T}\int_{T}^{2T}\Bigl{|}{\cal
M}(y)-\sum_{n\leq N_{1}}a_{n}(\gamma_{n}y)\Bigr{|}^{2}dy\ll
N_{1}^{-1/d+\varepsilon}$

and whence Hypothesis (H) follows. From Lemma 13 with $\mu=5/4+3/4d$ we get
Theorem 2.

From Theorem 1 and integration by parts we get easily that

$\int_{0}^{T}\left(y^{-\frac{d-1}{2d}}E(y)\right)^{2}dy\ll T,$

hence Theorem 3 follows from Lemma 14 by taking ${\cal
M}(y)=y^{-\frac{d-1}{2d}}E(y).$

##  10 Some applications of our main results

In this section we consider some applications of our results.

###  10.1 Some functions of degree $2$

Example 1: The Dirichlet divisor problem. The first example is
$\zeta^{2}(s)\in{\cal S}^{0}_{real}.$ Since

$\zeta^{2}(s)=\sum_{n=1}^{\infty}d(n)n^{-s}\qquad{\rm Re\,}s>1,$

where $d(n)$ is the divisor function, this is the Dirichlet divisor problem.
This problem is fundamental for all theory and hence has a long history. Let
us review some of them. Dirichlet first proved that the error term

$\Delta(x):=\sideset{}{{}^{\prime}}{\sum}_{n\leq x}d(n)-x\log
x-(2\gamma-1)x,\hskip 8.53581ptx\geq 2$

satisfies $\Delta(x)=O(x^{1/2}).$ The latest result is due to Huxley [17], who
showed that

$\Delta(x)\ll x^{131/416}(\log x)^{26947/8320}.$

Cramér [6] proved the classical mean-square result

$\int_{1}^{T}\Delta^{2}(x)dx\;=\;\frac{(\zeta(3/2))^{4}}{6\pi^{2}\zeta(3)}T^{3/2}+F(T)$

with $F(T)=O(T^{5/4+\varepsilon}).$ The estimate $F(T)=O(T^{5/4+\varepsilon})$
was improved to $O(T\log^{5}T)$ by Tong [49], from which the present work
originates. Tong’s result was improved to $O(T\log^{4}T)$ by Preissmann [40],
and recently to $O(T\log^{3}T\log\log T)$ by Lau and Tsang [34]. For the
asymptotic formulae of higher power moments of $\Delta(x)$ see, for example,
the papers of Ivić and Sargos [24], Tsang [51] and Zhai [55]. For a survey of
the Dirichlet divisor problem, see for example, Krätzel [31], Ivić [19] or
Tsang [50].

Corollary 1 gives $F(T)=O(T^{1+\varepsilon})$. Since our setting of the
problem is of general nature, it is weaker than the estimate of the form
$F(T)\ll T\log^{c}T,$ but is much stronger than Cramér’s result
$F(T)=O(T^{5/4+\varepsilon}).$

Example 2: The automorphic $L$-function attached to holomorphic cusp forms
over $SL_{2}({\mathbb{Z}})$. Let $f$ be a primitive holomorphic cusp form of
weight $k\geq 1$ for the full modular group $SL_{2}({\mathbb{Z}}).$ (For the
sake of simplicity we only consider the form for $SL_{2}(\mathbb{Z})$.) Let

(10.1) $f(z)=\sum_{n=1}^{\infty}\lambda_{f}(n)n^{(k-1)/2}e(nz),$

where $e(z)=e^{2\pi iz}$, be its normalized Fourier expansion at the cusp
$\infty$. Then the automorphic $L$-function

${\cal L}(f,s)=\sum_{n=1}^{\infty}\lambda_{f}(n)n^{-s}$

is an $L$-function of degree $2$ satisfying the functional equation

$(2\pi)^{-s}\Delta(s){\cal L}(f,s)=(-1)^{k/2}(2\pi)^{-(1-s)}\Delta(1-s){\cal
L}(f,1-s)$

with the gamma factor $\Delta(s)=\Gamma\left(s+\frac{k-1}{2}\right)$. Deligne
[7] proved that $|\lambda_{f}(n)|\leq d(n)$, so we have ${\cal L}(f,s)\in{\cal
S}_{real}^{0}$. It is well-known that

$A(y)=\sum_{n\leq y}\lambda_{f}(n)\ll y^{1/3}$

and (see Rankin [41] or Selberg [45])

$\sum_{n\leq y}|\lambda_{f}(n)|^{2}=c_{f}y+O(y^{3/5})$

for some positive constant $c_{f}.$ Walfisz [53] proved that

$\int_{1}^{T}A^{2}(y)dy=C_{A}T^{3/2}+O(T\log^{2}T)$

holds for some positive constant $C_{A}>0$. From Corollary 1 we have a weaker
result

$\int_{1}^{T}A^{2}(y)dy=C_{A}T^{3/2}+O(T^{1+\varepsilon}).$

Example 3: The automorphic $L$-function attached to Maass forms for
$SL_{2}({\mathbb{Z}})$.
Let $\varphi$ be a primitive Maass form for $SL_{2}({\mathbb{Z}})$, which is
an eigenfunction of the Laplace operator with an eigenvalue
$\lambda=1/4+r^{2},$ where $r\in\mathbb{R}.$ Write its Fourier expansion at
infinity in the form

$\varphi(z)=\sqrt{v}\sum_{n\in{\mathbb{Z}}\setminus\\{0\\}}\rho(n)K_{ir}(2\pi|n|v)e(nu)\quad(\
z=u+iv,u\in{\mathbb{R}},v>0),$

where $K_{ir}$ is the modified Bessel function of the third kind. The
corresponding automorphic $L$-function is

${\cal L}(\varphi,s)=\sum_{n=1}^{\infty}\rho(n)n^{-s},$

which is a function of degree $2$, entire on ${\mathbb{C}},$ with the
functional equation

$\pi^{-s}\Delta(s){\cal
L}(\varphi,s)=(-1)^{\delta}\pi^{-(1-s)}\Delta(1-s){\cal L}(\varphi,1-s),$

where $\Delta(s)=\Gamma\left((s+\delta+ir)/2\right)\Gamma\left((s+\delta-
ir)/2\right),$ and $\delta$ is the parity of $\varphi$ defined by $\delta=0$
if $\varphi$ is even and $\delta=1$ if $\varphi$ is odd. Kim and Sarnak [30]
proved that

(10.2) $|\rho(n)|\leq d(n)n^{7/64}.$

The Rankin-Selberg method implies that

(10.3) $\sum_{n\leq y}|\rho(n)|^{2}\ll y.$

So we have ${\cal L}(\varphi,s)\in{\cal S}_{real}^{\theta}$ with
$\theta=7/64.$

Hafner and Ivić [14] proved that

$A_{r}(y)=\sum_{n\leq y}\rho(n)\ll y^{2/5}.$

Meurman [37] proved that

$A_{r}(y)\ll y^{\frac{1+\mu}{3}+\varepsilon},$

where $\mu\geq 0$ is any real number such that $\rho(n)\ll n^{\mu}.$

From Corollary 1 we get

$\int_{1}^{T}A_{r}^{2}(y)dy=C_{A_{r}}T^{3/2}+O(T^{1+\varepsilon}),$

where $C_{A_{r}}>0$ is a positive constant.

###  10.2 Some functions of degree $3$

Example 1: Let ${\cal D}_{r}$ denote the set of all Dirichlet $L$-functions
corresponding to real primitive Dirichlet characters, ${\cal D}_{c}$ denote
the set of all Dirichlet $L$-functions corresponding to complex primitive
Dirichlet characters, $f$ denote a primitive holomorphic cusp form and
$\varphi$ denote a primitive Maass form for $SL_{2}({\mathbb{Z}})$ for some
fixed eigenvalue $\lambda=1/4+r^{2}(r\in\mathbb{R}),$ respectively. Define

$\displaystyle{\cal D}_{1f}=\\{{\cal L}(f,s)L(s,\chi):\ L(s,\chi)\in{\cal
D}_{r}\\},$ $\displaystyle{\cal D}_{1\varphi}=\\{{\cal
L}(\varphi,s)L(s,\chi):\ L(s,\chi)\in{\cal D}_{r}\\},$ $\displaystyle{\cal
D}_{r}^{3}=\\{L(s,\chi_{1})L(s,\chi_{2})L(s,\chi_{3}):\ L(s,\chi_{j})\in{\cal
D}_{r},j=1,2,3\\},$ $\displaystyle{\cal D}_{r}{\cal D}_{c}\overline{{\cal
D}_{c}}=\\{L(s,\chi_{1})L(s,\chi_{2})L(s,\overline{\chi_{2}}):\
L(s,\chi_{1})\in{\cal D}_{r},L(s,\chi_{2})\in{\cal D}_{c}\\}.$

Let ${\cal L}(s)$ denote any function in the set ${\cal D}_{1f}\cup{\cal
D}_{1\varphi}\cup{\cal D}^{3}\cup{\cal D}_{r}{\cal D}_{c}\overline{{\cal
D}_{c}}.$ It is easy to see that ${\cal L}(s)$ is a function of degree $3$,
which satisfies the conditions of Corollary 2. We consider only the case
${\cal L}(s)=L(s,\chi){\cal L}(\varphi,s),$ where $\chi$ is a real primitive
Dirichlet character modulo some positive integer. Write

${\cal L}(s)=\sum_{n\geq 1}a(n)n^{-s},\ {\rm Re\,}s>1.$

Then $a(n)=\sum_{d|n}\rho(d)\chi(n/d).$ From the estimate (10.2) we have
$a(n)\ll d^{2}(n)n^{7/64}.$ By Cauchy’s inequality we have

$a^{2}(n)\leq d(n)\sum_{d|n}\rho^{2}(d),$

which combining (10.3) gives

$\displaystyle\sum_{n\leq y}a^{2}(n)$ $\displaystyle\ll
y^{\varepsilon/2}\sum_{dn\leq y}\rho^{2}(d)\ll y^{\varepsilon/2}\sum_{n\leq
y}\sum_{d\leq y/n}\rho^{2}(d)\ll y^{1+\varepsilon},$

where we used the bound $d(n)\ll n^{\varepsilon/2}.$ Whence (1.4) holds.

As an analogue of the sixth moment of the automorphic $L$-function attached to
a cusp form [25], we have also

$\int_{1}^{T}|{\cal L}(\varphi,1/2+it)|^{6}dt\ll T^{2+\varepsilon}.$

This estimate can be proved by Jutila’s method analogously. So from Lemma 12
we have

$\int_{1}^{T}|{\cal L}(7/12+it)|^{2}dt\ll T^{1+\varepsilon},$

namely, (1.5) holds for $\sigma^{*}=7/12,$ which obviously satisfies (1.6). So
from Corollary 2 we have

$\int_{1}^{T}E^{2}(y)dy=C_{E}T^{5/3}+O(T^{14/9+\varepsilon}),$

where $C_{E}$ is a positive constant.

Example 2: Let $K/{\mathbb{Q}}$ be a number field of degree $\kappa$ with
discriminant $d_{K}$. The Dedekind zeta-function defined by

$\zeta_{K}(s)=\sum_{\mathfrak{a}\in\mathfrak{L}\setminus\\{0\\}}\frac{1}{(N\mathfrak{a})^{s}}$

is an $L$-function of degree $\kappa$ with the functional equation

$(\pi^{-\kappa}|d_{K}|)^{s/2}\Delta(s)\zeta_{K}(s)=(\pi^{-\kappa}|d_{K}|)^{(1-s)/2}\Delta(1-s)\zeta_{K}(1-s)$

with the gamma factor
$\Delta(s)=\Gamma^{r_{1}+r_{2}}\left(\frac{s}{2}\right)\Gamma^{r_{2}}\left(\frac{s+1}{2}\right)$,
where $r_{1}$ is the number of real embeddings of $K$ and $r_{2}$ the number
of pairs of complex embeddings so that $\kappa=r_{1}+2r_{2}.$ The Dedekind
zeta-function $\zeta_{K}(s)$ has a simple at $s=1.$

From now we suppose that $\kappa=3.$ Let $E_{K}(y):=\sum_{n\leq y}a(n)-\lambda
y,$ where $a(n)$ denotes the number of ideals in $\mathfrak{L}$ of norm $n$
and $\lambda$ is the residue of $\zeta_{K}(s)$ at $s=1.$ Weber and Landau
showed that

$E_{K}(y)=O\left(y^{1/2}\right).$

In 1964, Chandrasekharan and Narasimhan [3] studied the mean square of
$E_{K}(y)$ and proved

$\int_{1}^{T}E_{K}^{2}(y)dy=O\left(T^{5/3}\log^{3}T\right).$

In 1999, Y.-K. Lau [33] improved the above result to

$\int_{1}^{T}E_{K}^{2}(y)dy=O\left(T^{5/3}\log^{2}T\right).$

Fomenko [10] followed Tong’s approach and proved that the asymptotic formula

$\displaystyle\int_{1}^{T}E_{K}^{2}(y)dy=C_{K}T^{5/3}+O\left(T^{8/5+\varepsilon}\right)$

holds for some positive constants $C_{K}>0$ with the help of the estimate

(10.4) $\int_{1}^{T}|\zeta_{K}(\sigma+it)|^{2}\ll T^{1+\varepsilon}$

with $\sigma=5/8.$

When $\kappa=3$, we can show that the Dedekind zeta-function $\zeta_{K}(s)$
can be written as $\zeta_{K}(s)=\zeta(s){\cal L}_{2}(s)$ such that where
${\cal L}_{2}(s)\in{\cal S}_{real}^{0}$ is a function of degree $2$ satisfying
the condition (1.10). When $K/{\mathbb{Q}}$ is a normal extension, from Lemma
1 of W. Müller [39] we know that
$\zeta_{K}(s)=\zeta(s)L(s,\chi_{1})L(s,\overline{\chi_{1}})$ for some
primitive Dirichlet character $\chi_{1}.$ So we have
$\zeta_{K}(s)=\zeta(s){\cal L}_{2}(s),$ where ${\cal
L}_{2}(s)=L(s,\chi_{1})L(s,\overline{\chi_{1}})$ satisfies (1.10) via Meurman
[38].

When $K/{\mathbb{Q}}$ is a not normal extension, from the formula (1) of
Fomenko [10] we have $\zeta_{K}(s)=\zeta(s)L(s,F),$ where $F$ is a holomorphic
primitive cusp form of weight $1$ with respect to $\Gamma_{0}(|d_{K}|)$ and
$L(s,F)$ is its corresponding automorphic $L$-function of degree $2.$ We have
the estimate

$\int_{1}^{T}|L(1/2+it,F)|^{6}dt\ll T^{2+\varepsilon}.$

The proof is also due to Jutila [25], who proved it in the case of a
holomorphic cusp form $F$ of even weight with respect to
$SL_{2}({\mathbb{Z}}).$ His proof can be applied to $L(s,F),$ too.

From Lemma 12 we see that (10.4) holds for $\sigma=7/12.$ So from Corollary 2
we get

$\displaystyle\int_{1}^{T}E_{K}^{2}(y)dy=C_{K}T^{5/3}+O\left(T^{14/9+\varepsilon}\right),$

which is a slight improvement to Fomenko’s result.

###  10.3 Some examples of degree $4$

Example 1. Let ${\cal D}_{r}$ denote the set of all Dirichlet $L$-functions
corresponding to real primitive Dirichlet characters, ${\cal D}_{c}$ denote
the set of all Dirichlet $L$-functions corresponding to complex primitive
Dirichlet characters, $f$ denote a primitive holomorphic cusp form and
$\varphi$ denote a primitive Maass form for $SL_{2}({\mathbb{Z}})$ for some
fixed an eigenvalue $\lambda=1/4+r^{2}(r\in\mathbb{R}),$ respectively. Define

$\displaystyle{\cal D}_{2f}=\\{{\cal L}(f,s)L(s,\chi_{1})L(s,\chi_{2}):\
L(s,\chi_{j})\in{\cal D}_{r},\quad j=1,2\\},$ $\displaystyle{\cal
D}_{2\varphi}=\\{{\cal L}(\varphi,s)L(s,\chi_{1})L(s,\chi_{2}):\
L(s,\chi_{j})\in{\cal D}_{r},\quad j=1,2\\},$ $\displaystyle{\cal
D}_{r}^{4}=\\{\prod_{j=1}^{4}L(s,\chi_{j}):\ L(s,\chi_{j})\in{\cal
D}_{r},1\leq j\leq 4\\},$ $\displaystyle{\cal D}_{r}^{2}{\cal
D}_{c}\overline{{\cal
D}_{c}}=\\{L(s,\chi_{1})L(s,\chi_{2})L(s,\chi_{3})L(s,\overline{\chi_{3}}):\
L(s,\chi_{1}),L(s,\chi_{2})\in{\cal D}_{r},L(s,\chi_{3})\in{\cal D}_{c}\\},$
$\displaystyle({\cal D}_{c}\overline{{\cal
D}_{c}})^{2}=\\{\prod_{j=1}^{2}L(s,\chi_{j})L(s,\overline{\chi_{j}}):\
L(s,\chi_{j})\in{\cal D}_{c},j=1,2\\}.$

Suppose ${\cal L}(s)$ is any function in the set

${\cal D}_{2f}\cup{\cal D}_{2\varphi}\cup{\cal D}^{4}\cup{\cal D}_{r}^{2}{\cal
D}_{c}\overline{{\cal D}_{c}}\cup({\cal D}_{c}\overline{{\cal
D}_{c}})^{2}\cup\\{{\cal L}^{2}(f,s),{\cal L}^{2}(\varphi,s),{\cal
L}(f,s){\cal L}(\varphi,s)\\}.$

Obviously ${\cal L}(s)$ is a function of degree $4.$ From Lemma 12 it is easy
to see that

(10.5) $\int_{1}^{T}|{\cal L}(5/8+it)|^{2}dt\ll T^{1+\varepsilon}.$

For the mean square of $E(y)$ corresponding to ${\cal L}(s)$ we have

(10.6) $T^{7/4}\ll\int_{1}^{T}E^{2}(y)dy\ll T^{7/4+\varepsilon}.$

The left inequality in (10.6) follows from Theorem 5 directly. The proof of
the right inequality in (10.6) is the same as the case $k=4$ in Theorem 13.4
of Ivić [19] with the help of (10.5). So we omit the details. We note that the
upper bound in (10.6) is already proved in Ivić [22] when ${\cal L}(s)={\cal
L}^{2}(f,s).$

If we could find a number $1/2\leq\sigma_{0}<5/8$ such that

$\int_{1}^{T}|{\cal L}(\sigma_{0}+it)|^{2}dt\ll T^{1+\varepsilon},$

then from Theorem 1 the asymptotic formula

(10.7) $\int_{1}^{T}E^{2}(y)dy=C_{4}T^{7/4}+O(T^{7/4-\delta_{\cal L}})$

would hold, where $C_{4}>0$ and $\delta_{\cal L}>0$ are positive constants.
The formula (10.7) implies that the estimate (10.6) is best possible.

Example 2: The error term in the Rankin-Selberg problem.

Let $f$ be a primitive holomorphic cusp form of weight $k$ for the full
modular group $SL_{2}({\mathbb{Z}})$ and its normalized Fourier expansion at
the cusp $\infty$ can be written as (10.1). The corresponding Rankin-Selberg
zeta function defined by

${\cal L}(f\otimes f,s)=\sum_{n=1}^{\infty}c(n)n^{-s},\
c(n)=\sum_{m^{2}|n}|\lambda_{f}(n/m^{2})|^{2},\ ({\rm Re\,}s>1)$

is a function of degree $4$ satisfying the functional equation

$(2\pi)^{-2s}\Delta(s){\cal L}(f\otimes f,s)=(2\pi)^{-2(1-s)}\Delta(1-s){\cal
L}(f\otimes f,1-s)$

with the gamma factor $\Delta(s)=\Gamma(s)\Gamma(s+k-1)$.

Rankin [41] and Selberg [45] proved independently that

$\displaystyle\sum_{n\leq y}c(n)=C_{f}y+E(y),\ E(y)=O(y^{3/5}),$

where $C_{f}>0$ is a positive constant. From Deligne’s estimate
$|\lambda_{f}(n)|\leq d(n)$ we have

(10.8) $c(n)\leq d^{2}(n)d(1,2;n)\leq d^{3}(n)\ll n^{\varepsilon},$

where $d(1,2;n)=\sum_{n=m^{2}d}1\leq d(n).$ We write
$c(n)=\sum_{n=m^{2}d}|\lambda_{f}(d)|^{2}.$ By Cauchy’s inequality we have

$c^{2}(n)\leq d(1,2;n)\sum_{n=m^{2}d}|\lambda_{f}(d)|^{4}\ll
n^{\varepsilon}\sum_{n=m^{2}d}|\lambda_{f}(d)|^{4},$

where we used $d(1,2;n)\ll n^{\varepsilon}.$ So we have

(10.9) $\displaystyle\sum_{n\leq y}c^{2}(n)$ $\displaystyle\ll
y^{\varepsilon}\sum_{m^{2}d\leq y}|\lambda_{f}(d)|^{4}$ $\displaystyle\ll
y^{\varepsilon}\sum_{m\leq y^{1/2}}\sum_{d\leq y/m^{2}}|\lambda_{f}(d)|^{4}$
$\displaystyle\ll y^{\varepsilon}\sum_{m\leq y^{1/2}}y/m^{2}\ll
y^{1+\varepsilon},$

where we used the estimate in Lü [35]

$\sum_{n\leq y}|\lambda_{f}(n)|^{4}\ll y.$

The estimates (10.8) and (10.9) show that (1.4) holds.

Ivić [21] proved that

$\displaystyle\int_{1}^{T}E^{2}(y)dy\ll T^{1+2\beta+\varepsilon},$

where $\beta=2/(5-2\mu(1/2))$ with $\mu(1/2)$ satisfying
$\zeta(1/2+it)\ll(|t|+1)^{\mu(1/2)+\varepsilon}.$ Huxley’s best bound[18]
$\mu(1/2)\leq 32/205$ implies $\beta\leq 410/961=0.4266\cdots.$ The Lindelöf
Hypothesis implies $\beta\leq 0.4.$ In [20] Ivić even conjectured

(10.10) $\displaystyle\int_{1}^{T}E^{2}(y)dy\ll T^{\frac{7}{4}+\varepsilon}.$

The estimate (10.10), if it is true, is best possible. Since from Theorem 5 we
have

$\displaystyle\int_{1}^{T}E^{2}(y)dy\gg T^{\frac{7}{4}}.$

If we could find a $\sigma_{0}<\frac{5}{8}$ such that

$\int_{1}^{T}|{\cal L}(f\otimes f,\sigma_{0}+it)|^{2}dt\ll T^{1+\varepsilon},$

then from Theorem 1 we would have the asymptotic formula

$\displaystyle\int_{1}^{T}E^{2}(y)dy=C_{f}T^{\frac{7}{4}}+O\left(T^{\frac{7}{4}-\delta_{f}}\right)$

for some positive constants $C_{f}>0,\delta_{f}>0.$

## References

  * [1] X. Cao, The general divisor problem(in Chinese), Acta Math. Sinica, 36 (5) (1993), 644–653.
  * [2] X. Cao, Y. Tanigawa and W. Zhai, Mean square of the error term in the asymmetric many dimensional divisor problem, submitted.
  * [3] K. Chandrasekharan and R. Narasimhan, On the mean value of the error terms for a class of arithmetical functions, Acta Math. 112 (1964), 41–67.
  * [4] K. Chandrasekharan and R. Narasimhan, Functional equations with multiple gamma factors and the everage order of arithmetical functions, Ann. Math. 76 (1962), 93–136.
  * [5] K. Chandrasekharan and R. Narasimhan, Approximate functional equations for a class of zeta-functions, Math. Ann. 152 (1963), 30–64.
  * [6] H. Cramér, Über zwei Sätze von Herrn G. H. Hardy, Math. Z. 15 (1922), 201–210.
  * [7] P. Deligne, La conjecture de Weil I, Inst. Hautes Etudes Sci. Publ. Math. 43 (1974), 273–307.
  * [8] A. Erdélyi, W. Magnus, F. Oberhettinger and F.G. Tricomi, Higher Transcendental Functions. Volume I, McGraw-Hill, 1953.
  * [9] J.B. Friedlander and H. Iwaniec, Summation formulae for coefficients of $L$-functions, Canada. J. Math. 57 (3) (2005), 494–505.
  * [10] O.M. Fomenko, Mean values connected with the Dedekind zeta function. (Russian) Zap. Nauchn. Sem. S.-Peterburg. Otdel. Mat. Inst. Steklov. (POMI) 350 (2007), Analitcheskaya Teoriya Chisel i Teoriya Funktsii. 22, 187–198, 203; translation in J. Math. Sci. (N. Y.) 150 (2008), no. 3, 2115–2122.
  * [11] O.M. Fomenko, Mean value theorems for automorphic $L$-functions. (Russian) Algebra i Analiz 19 (2007), no. 5, 246–264; translation in St. Petersburg Math. J. 19 (2008), no. 5, 853–866.
  * [12] J. L. Hafner, On the representation of summatory functions of a class of arithmetical functions, in Analytic Number Theory, LNM 899, Springer-Verlag, Berlin-Heidelberg-New York 1981, 148–165.
  * [13] J. L. Hafner, On the average order of a class of arithmetical functions, J. Number Theory, 15 (1982), 36–76.
  * [14] J. L. Hafner and A. Ivić, On sums of Fourier coefficients of cusp forms, L’Enseignement Mathématique, t. 35 (1989), 375–382.
  * [15] D.R. Heath-Brown, The distribution and moments of the error term in the Dirichlet divisor problem, Acta Arith. 60 (1992), 389–415.
  * [16] D.R. Heath-Brown, The twelth power moment of the Riemann zeta-function, Quart. J. Math. (Oxford) 29 (1978), 443-462.
  * [17] M. N. Huxley, Exponential sums and lattice points III, Proc. London Math. Soc., 87 (2003), 591–609.
  * [18] M. N. Huxley, Exponential sums and the Riemann zeta-function V, Proc. London Math. Soc. (3) 90(2005), 1-41.
  * [19] A. Ivić, The Riemann Zeta-Function, John Wiley and Sons, 1985.
  * [20] A. Ivić, Large values of certain number-theoretic error terms, Acta Arith. 56(1990), 135–159.
  * [21] A. Ivić, On some mean square estimates in the Rankin-Selberg problem, Applicable Analysis and Discrete Math. 1 (2007), 111–121.
  * [22] A. Ivić, On zeta-functions associated with Fourier coefficients of cusp forms, Proc. Amalfi Conference on Analytic Number Theory (Mainoi, 1989), Salerno (1992), 231–246.
  * [23] A. Ivić, E. Krätzel, M. Kühleitner and W. G. Nowak, Lattice points in large regions and related arithmetic functions: recent developments in a very classic topic. (English summary) Elementare und analytische Zahlentheorie, 89-128, Schr.Wiss. Ges. Johann Wolfgang Goethe Univ.Frankfurt am Main, 20, Franz Steiner Verlag Stuttgart, Stuttgart, 2006.
  * [24] A. Ivić and P. Sargos, On the higher power moments of the error term in the divisor problem, Illinois J. of Math. 81 (2007), 353–377.
  * [25] M. Jutila, A Method in the Theory of Exponential Sums, Tata Inst. of Fundamental Research, Bombay Lectures No. 80, Springer-Verlag, Berlin-Heidelberg-New York, 1987.
  * [26] J. Kaczorowski, Axiomatic theory of $L$-functions: the Selberg class, in ”Analytic Number Theory” eds. A. Perelli and C. Viola, Springer Verlag, Berlin-Heidelberg, 2006, pp. 133–209.
  * [27] J. Kaczorowski and A. Perelli, On the structure of the Selberg class, $0\leq d\leq 1$, Acta Math. 182 (1999), 207–241.
  * [28] J. Kaczorowski and A. Perelli, On the structure of the Selberg class, $1<d<5/3$, Invent. Math. 150 (2002), 485–516.
  * [29] S. Kanemitsu, A. Sankaranarayanan and Y. Tanigawa, A mean value theorem for Dirichlet series and a general divisor problem, Monatsh. Math. 136 (2002), 17–34.
  * [30] H. Kim and P. Sarnak, Refined estimates towards the Ramanujan and Selberg conjectures, J. Amer. Math. Soc. 16 (2003), 175–181.
  * [31] E. Krätzel, Lattice Points, Kluwer Academic Publishers, Dordrecht 1988.
  * [32] E. Landau, Über die Anzahl der Gitterpunkte in gewissen Bereichen, Gẗt. Nachr. (1912), 687–771.
  * [33] Y.-K. Lau, On the mean value formula of the error term for a class of arithmetical functions, Mh. Math. 128 (1999),111–129.
  * [34] Y. K. Lau and K. M. Tsang, On the mean square formula of the error term in the Dirichlet divisor problem, Math. Proc. Camb. Phil. Soc. 146 (2009), 277–287.
  * [35] G. S. Lü, On an open problem of Sankaranarayanan, Sci. China Ser. A. 39 (2009), 1023–1028.
  * [36] T. Meurman, The mean square of the error term in a generalization of Dirichlet’s divisor problem, Acta Arith. 74 (1996), 351–364.
  * [37] T. Meurman, On exponential sums involving the Fourier coefficients of Maass wave forms, J. Reine Angew. Math. 384 (1988), 192–207.
  * [38] T. Meurman, The mean twelfth power of Dirichlet $L$-functions on the critical line, Ann. Acad. Sci. Fenn. Ser. A I Math. Dissertationes No. 52 (1984), 44pp.
  * [39] W. Müller, On the distribution of ideals in cubic number fields, Mh. Math., 106 (1988), 211–219.
  * [40] E. Preissman, Sur la moyenne de la fonction zêta. Nagasaka, Kenji (ed.), Analytic number theory and related topics. Proceedings of the symposium, Tokyo, Japan, November 11-13, 1991. Singapore: World Scientific, 119-125 (1993).
  * [41] R. A. Rankin, Contributions to the theory of Ramanujan’s function $\tau(n)$ and similar arithmetical functions II. The order of the fourier coefficients of integral modular forms, Proc. Cambridge Phil. Soc. 35 (1939), 357–372.
  * [42] D. Redmond, Mean value theorems for a class of Diriclet series, Pacific J. Math. 78 (1978), 191–231.
  * [43] A. de Roton, On the mean square of the error term for an extended Selberg class, Acta Arith. 126 (1) (2007), 27–55.
  * [44] A. Selberg, Old and new conjectures and results about a class of Dirichlet series, in ”Proc. Amalfi Conf. Analytic Number Theory 1989 (E. Bombieri et al. eds.)”, University of Salerno, Salerno, 1992, pp. 367–385.
  * [45] A. Selberg, Bemerkungen über eine Dirichletsche Reihe, die mit der Theorie der Modulformen nahe verbunden ist, Arch. Math. Naturvid. 43 (1940), 47–50.
  * [46] J. Steuding, Value-distribution of $L$-functions, LNM 1877, Springer: Berlin, 2007.
  * [47] J. Steuding, Voronoï’s contribution to modern number theory, S̆iauliai Math. Semin., 10(2007), 67-106.
  * [48] K.-C. Tong, On divisor problems I, Acta Math. Sinica 5 (3),(1955), 313–323.
  * [49] K.-C. Tong, On divisor problems III, Acta Math. Sinica 6 (1956), 515–541.
  * [50] K-M Tsang, Recent progress on the Dirichlet divisor problem and the mean square of the Riemann zetafunction, Science China Mathematics, 53 (2010), 2561–2572.
  * [51] K.-M. Tsang, Higher-power moments of $\Delta(x),E(t)$ and $P(x)$, Proc. London Math. Soc.(3) 65 (1992), 65–84.
  * [52] G. Voronoï, Sur une fonction transcendante et ses applications à la sommation de quelques séries, Ann. École Normale 21 (1904), 207–267, 459–533.
  * [53] A. Walfisz, Über die Koeffizientensummen einiger Modulformen. Math. Ann. 108 (1933), 75–90.
  * [54] Z. X. Wang and D. R. Guo, ”Special Functions” (in Chinese), Science Press, Beijing, 1979.
  * [55] W. Zhai, On higher-power moments of $\Delta(x)$ (II), Acta Arith. 114 (2004), 35–54.

Xiaodong Cao

Department of Mathematics and Physics,

Beijing Institute of Petro-Chemical Technology,

Beijing, 102617, P. R. China

e-mail<EMAIL_ADDRESS>

Yoshio Tanigawa

Graduate School of Mathematics,

Nagoya University,

Nagoya, 464-8602, Japan

e-mail<EMAIL_ADDRESS>

Wenguang Zhai

Department of Mathematics,

China University of Mining and Technology,

Beijing 100083, P. R. China

e-mail<EMAIL_ADDRESS>


# Explaining Math Word Problem Solvers

Abby Newcomb<EMAIL_ADDRESS>St. Olaf College1500 St. Olaf
AveNorthfieldMinnesotaUSA550571 and  Jugal Kalita<EMAIL_ADDRESS>University of Colorado, Colorado Springs1420 Austin Bluffs PkwyColorado
Springs, COUSA

(2018; 2022)

###### Abstract.

Automated math word problem solvers based on neural networks have successfully
managed to obtain 70-80% accuracy in solving arithmetic word problems.
However, it has been shown that these solvers may rely on superficial patterns
to obtain their equations. In order to determine what information math word
problem solvers use to generate solutions, we remove parts of the input and
measure the model’s performance on the perturbed dataset. Our results show
that the model is not sensitive to the removal of many words from the input
and can still manage to find a correct answer when given a nonsense question.
This indicates that automatic solvers do not follow the semantic logic of math
word problems, and may be overfitting to the presence of specific words.

math word problems, neural networks, explainability

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††conference: NLP ; December; Bangkok, Thailand††ccs: Computing
methodologies Neural networks††journalyear: 2022††copyright:
acmlicensed††conference: 2022 6th International Conference on Natural Language
Processing and Information Retrieval; December 16–18, 2022; Bangkok,
Thailand††booktitle: 2022 6th International Conference on Natural Language
Processing and Information Retrieval (NLPIR 2022), December 16–18, 2022,
Bangkok, Thailand††price: 15.00††doi: 10.1145/3582768.3582777††isbn:
978-1-4503-9762-9/22/12

##  1\. Introduction

Math word problem (MWP) solving is an area of natural language processing
(NLP) that uses machine learning to solve simple arithmetic problems. MWPs
consist of a few sentences of text including a few numbers and an unknown
quantity, similar to problems humans are presented with in grade school.
Neural networks are trained to generate the correct equation which computes
the unknown quantity. Little is known about how neural networks manage to
solve math word problems. In this paper, we remove parts of math word problems
and measure the model’s performance on the changed data in order to ascertain
which words the model is using to choose the correct equation.

Various parts of speech work together to construct the full meaning of a
sentence, so even when a certain part of speech is removed, other words may
still indicate the desired operation. In order to more specifically gauge
which words are important to the model’s prediction, we employ input
reduction, a strategy that iteratively removes the least important word from
the input until the model produces an incorrect result. This method allows us
to see how removing specific words affects the model.

We also perform analysis of which words appear most frequently in the datasets
used to train the model. We also look at the most common words for each type
of problem (+, -, *, /, multiple) to see whether certain words appear to
indicate specific operations.

In order to determine which parts of speech are most important to MWP solvers,
we remove specific words from MWP test datasets and test a Seq2seq MWP solver
on its ability to determine the correct answer on these reduced problems. The
contributions of this paper are as follows:

  * •

We show that the lexical diversity of MaWPS is low.

  * •

We show that the RNN Seq2seq solver performs little semantic reasoning, since
it can produce correct answers with significantly reduced input.

We begin by explaining related work, then cover the methods and results of
each experiment in turn, followed by the conclusion.

Figure 1. Percentage of MWPs in MaWPS dataset of each operation type, on
average across all CV folds.

##  2\. Related Work

Various neural network MWP solvers have been created and benchmarked on well-
known datasets. Few explainability techniques have yet been applied to MWP
solving.

###  2.1. Math Word Problems

The current most commonly used datasets for Math Word Problem solving are
MaWPS (Koncel-Kedziorski et al., 2016) and ASDiv-A (Miao et al., 2021). These
datasets are currently the largest ones available, though they are quite small
for machine learning datasets. MaWPS has 2373 MWPs while ASDiv-A has only 1218
problems.

Figure 2. The average accuracy of the Seq2seq model trained on the MaWPS
dataset, when evaluated on various perturbed datasets. The model’s average
accuracy on the original test dataset is indicated by the red dashed line.

Various types of neural networks for solving MWPs have been developed. Wang et
al. ((2017)) use a GRU encoder and an LSTM decoder in a sequence to sequence
approach. Another model is a graph to tree model proposed by Zhang et al.
((2020)), which uses a graph transformer and tree structured decoder to
generate the MWP solution expression tree. Griffith and Kalita ((2019)) use a
transformer-based model. Xie and Sun ((2019)) use a model called GTS in a
process they call goal decomposition to find relationships between quantities.
Their approach uses feed-forward networks and an RNN model at different steps
in the algorithm.

Though these models obtain high accuracy, their success was called into
question when MWP solvers were shown to obtain similar accuracy when the
actual question was removed, leaving only the descriptive body of text at the
beginning of the problem (Patel et al., 2021). MWP solvers also perform poorly
on the SVAMP challenge dataset, which was specifically generated to require
attention to the question itself (Patel et al., 2021). This implies that the
solvers are relying on superficial patterns in the initial text rather than
actually answering the question posed in the problem. However, it was later
shown that performance on the SVAMP dataset could be improved simply by
generating more data to increase the size of MWP training datasets (Kumar et
al., 2022).

Figure 3. The average accuracy of the Seq2seq model trained on the ASDiv-A
dataset, when evaluated on various perturbed datasets. The model’s average
accuracy on the original test dataset is indicated by the red dashed line.

###  2.2. Explainability Techniques

The strategy of removing parts of the input to an NLP model is often used to
explain a model’s decisions. Importance scores have been assigned to words in
the input by looking at the effects of removing those words (Li et al., 2017).
Similarly, the process of input reduction involves successively removing the
word that affects the model’s confidence score the least, until we are left
with the smallest possible input with which the model can still make a correct
prediction (Feng et al., 2018). This process shows us which words in the input
are most important to the model’s prediction. These methods, among others,
have been implemented by Wallace et al. ((2019)) in their AllenNLP framework
for NLP explainability techniques. However, applications of these methods
often focus on large models such as BERT and tasks such as to sentiment
analysis, reading comprehension, or textual entailment. This method has not
yet been applied to MWP solving.

Another method of understanding NLP model predictions is adversarial attacks,
in which various changes are made to the input of a model, and the performance
of the model is measured in order to determine how sensitive the model is to
the changes in the perturbed dataset. Adversarial attacks are different from
the aforementioned methods because the new inputs to the model are meant to be
semantically equivalent to the previous inputs and should still be
grammatically correct (Lee et al., 2019). Adversarial examples have been used
for interpretability of reading comprehension systems (Jia and Liang, 2017)
and question answering systems (Lee et al., 2019) in the past. Adversarial
attacks involving question reordering and sentence paraphrasing were also used
by Kumar et al. ((2021)) to show that MWP solvers are not robust to these
seemingly irrelevant perturbations.

|  |  |  |
---|---|---|---|---

| Count | Pct
---|---|---
book | 68 | 0.16
will | 62 | 0.14
were | 61 | 0.14
box | 52 | 0.12
tree | 52 | 0.12
total | 51 | 0.12
at | 49 | 0.11
is | 49 | 0.11
pick | 48 | 0.11
from | 46 | 0.11
park | 45 | 0.1
(a) Addition

| Count | Pct
---|---|---
dollar | 63 | 0.19
total | 52 | 0.16
game | 44 | 0.13
balloon | 43 | 0.13
book | 41 | 0.12
will | 40 | 0.12
at | 39 | 0.12
were | 39 | 0.12
pick | 37 | 0.11
is | 37 | 0.11
all | 36 | 0.11
(b) Subtraction

| Count | Pct
---|---|---
card | 30 | 0.18
were | 26 | 0.16
box | 25 | 0.15
will | 25 | 0.15
now | 23 | 0.14
total | 23 | 0.14
book | 22 | 0.13
from | 22 | 0.13
pick | 21 | 0.13
one | 21 | 0.13
all | 20 | 0.12
(c) Multiplication

| Count | Pct
---|---|---
piece | 23 | 0.16
his | 23 | 0.16
dollar | 21 | 0.15
box | 21 | 0.15
from | 19 | 0.14
make | 18 | 0.13
hour | 18 | 0.13
at | 17 | 0.12
now | 16 | 0.11
balloon | 16 | 0.11
game | 15 | 0.11
(d) Division

| Count | Pct
---|---|---
dollar | 89 | 0.16
box | 77 | 0.14
piece | 73 | 0.13
book | 69 | 0.13
total | 69 | 0.13
at | 65 | 0.12
will | 63 | 0.11
all | 61 | 0.11
game | 61 | 0.11
from | 61 | 0.11
would | 61 | 0.11
(e) Multiple

Table 1. The top words for each operation in MaWPS CV Fold 1, excluding words
that appeared in all 5 lists, by count of MWPs it appears in. Percentage of
MWPs of that operation that the word appears in is also provided for
comparison’s sake.

##  3\. Problem Statement

The question remains to what degree MWP solvers perform semantic reasoning,
and what information they use to generate an equation for a solution to a
given problem. We apply various methods to search for trigger words and other
superficial patterns that the model may be relying on instead of semantic
reasoning.

##  4\. Experiment 1: Removing Parts of Speech

We removed various parts of speech from the MWPs and tested an MWP solver’s
performance on the perturbed datasets in order to see how important different
types of words are to the model. A large decrease in accuracy due to the
removal of a part of speech indicates that that part of speech is important to
the model’s prediction, since the model cannot perform as well without it.

###  4.1. Methods

We generate perturbed MWPs by identifying parts of speech using the Natural
Language Toolkit (NLTK) part-of-speech tagger (Bird et al., 2009) and then
removing the targeted words. We use the Seq2seq model created by Patel et al.
((2021)) for all experiments. We also use Patel et al. ((2021))’s optimized
parameters for training. Two models were trained on either MaWPS or AsDIV-A
with 5 fold cross-validation, and then each was evaluated on perturbed
examples from its respective dataset. Accuracy is measured on the model’s
success in generating the correct answer, rather than by the proximity of the
generated equation to the true equation.

As a bit of preliminary analysis, we looked at the relative concentration of
different types of MWPs in MaWPS, as seen in Figure 1. The first four
categories are characterized by having a single operation of the specified
type, while the problems in the “multi” category have multiple operations of
different types in them. The majority of problems in MaWPS (73%) have only one
operation. The dataset appears to represent addition and subtraction the best,
and have a much smaller number of multiplication and division problems. This
may contribute to the slightly decreased accuracy on multiplication and
division problems visible on Figure 4.

Figure 4. The RNN Seq2seq model’s accuracy on each type of problem for the
perturbed datasets with only one part of speech and number tokens remaining.

###  4.2. Results

Perturbation |  Original Question |  Perturbed Question |  Correct Equation
---|---|---|---
Verbs Removed |  Tommy had some balloons . His mom gave him number0 more balloons for his birthday . Then , Tommy had number1 balloons . How many balloons did Tommy have to start with ? |  Tommy some balloons . His mom him number0 more balloons for his birthday . Then , Tommy number1 balloons . How many balloons Tommy to with ? |  \- number1 number0
Nouns Removed |  The first minute of a telephone call costs number0 cents and each additional minute number1 cents . What is the cost of a number2 minute telephone call ? |  The first of a number0 and each additional number1 . What is the of a number2 ? |  \+ number0 * number1 number2
Nouns and Verbs Removed |  Virginia starts with number0 eggs . Amy takes number1 away . How many eggs does Virginia end with ? |  with number0 . number1 away . How many with ? |  \- number0 number1
Prepositions and Verbs Removed |  In March it rained number0 inches . It rained number1 inches less in April than in March . How much did it rain in April ? |  March it number0 inches . It number1 inches less April March . How much it April ? |  \- number0 number1
Table 2. Examples of perturbed MWPs from the MaWPS dataset. In this dataset,
the actual numbers are removed and replaced with number tokens (“number0”,
“number1”, etc.) in order for the model to process them more easily.

Some examples of perturbed MWPs obtained from the MaWPS dataset are shown in
Table 2. The models’ accuracy on each perturbed dataset is listed in Figures 2
and 3, and Table 3 shows all percent accuracies and decreases in accuracy. On
the original dataset, the model trained on ASDiv-A had 72.4% accuracy while
the MaWPS model had 86.5% accuracy. Removal of common adjectives such as
“more” resulted in accuracy decreases of 5.4% and 2.4% respectively, while
removing question adjectives such as “how” decreased accuracy by only 2.1% and
1.3%, and removal of all adjectives decreased accuracy by 5.1% and 2.9%.
Removal of named entities such as “Jim” was only conducted with MaWPS because
of the formatting of the data. MaWPS model accuracy decreased by only 2.8%
with no named entities. Removal of all nouns, including named entities and all
common nouns, decreased accuracy by 9.4% on ASDiv-A and 16.6% on MaWPS.
Removing prepositions decreased accuracy by 4.1% and 3.3% respectively.
Removing verbs decreased accuracy by 11.1% in the ASDiv-A model and by 5.9% on
the MaWPS model.

Perturbation |  MaWPS CV Accuracy |  MaWPS Decrease in Accuracy |  ASDiv-A CV Accuracy |  ASDiv-A Decrease in Accuracy
---|---|---|---|---
original dataset |  0.857 |  - |  0.716 |  -
common adjectives removed |  0.841 |  0.017 |  0.67 |  0.046
wh-adjectives removed |  0.852 |  0.004 |  0.703 |  0.013
all adjectives removed |  0.836 |  0.021 |  0.673 |  0.043
named entities removed |  0.837 |  0.02 |  - |  -
nouns removed |  0.699 |  0.158 |  0.63 |  0.086
prepositions removed |  0.832 |  0.025 |  0.683 |  0.033
verbs removed |  0.806 |  0.051 |  0.613 |  0.103
nouns and verbs removed |  0.553 |  0.304 |  0.519 |  0.197
prepositions and verbs removed |  0.726 |  0.13 |  0.582 |  0.134
only nouns and number tokens remaining |  0.232 |  0.625 |  0.217 |  0.499
only prepositions and number tokens remaining |  0.125 |  0.732 |  0.193 |  0.523
only verbs and number tokens remaining |  0.197 |  0.66 |  0.253 |  0.463
all words except number tokens removed |  0.122 |  0.735 |  0.171 |  0.545
Table 3. Seq2seq model CV accuracy and decrease in CV accuracy on each
perturbed dataset.

We also tested the models on datasets with two different parts of speech
missing. On a dataset with all nouns and verbs missing, the ASDiv-A model
accuracy decreased by 20.5% and MaWPS by 31.2%. With all prepositions and
verbs removed, the models’ accuracy decreased by 14.2% and 13.9% respectively.

The model was also tested on datasets where only a specific part of speech and
the number tokens were left in the MWP, with all other words removed from the
input. The results on these datasets tended to somewhat mirror the model’s
performance on the datasets with that part of speech removed.

On a dataset with all words except for the number tokens removed, the model
achieved 12.2% accuracy for MaWPS and 17.1% accuracy on AsDIV-A. It is
difficult to calculate what a completely random accuracy would be and how
close these are to random guesses because of the complexity of multiple
operations, but the AsDIV-A model does manage a significantly higher accuracy,
which indicates that it may not rely on the word content as much as the MaWPS
model.

###  4.3. Discussion

The model’s overall higher accuracies on the MaWPS dataset can likely be
attributed to its size, since with 2373 MWPs it is nearly twice as large as
ASDiv-A’s 1218 problems. The MaWPS model was also less affected by the removal
of any single part of speech compared to the ASDiv-A model (average accuracy
difference of 5.0% to ASDiv-A’s 6.2%), and thus seems to be less sensitive to
this type of perturbation overall. The MaWPS model was also more affected by
the removal of multiple parts of speech, as the decrease in performance on the
twice perturbed datasets was larger than the sum of the decrease in
performance on either of the once perturbed datasets, which was not the case
for the ASDiv-A model.

The removal of any single part of speech does not appear to significantly
affect either model. Overall, the MaWPS model was most affected by the removal
of nouns at a 16.6% decrease in accuracy, and the ASDiv-A model was most
affected by the removal of verbs at an 11.1% decrease. As hypothesized,
certain operations are more affected by the removal of some parts of speech
more than others, as seen in Figures 4 and 5. The models’ decent performance
on these reduced datasets indicates that no single part of speech is
incredibly important to its decision.

Figure 5. The RNN Seq2seq model’s accuracy on each type of problem for the
perturbed datasets with one or two parts of speech missing.

However, both models were still achieving an accuracy above 50% with no nouns
or verbs in the MWPs. This relatively high accuracy indicates that these
models are likely not performing semantic reasoning about the events described
in the MWP, since there is not enough information in the problem with no verbs
or nouns for the model to truly be reasoning about the quantities present.
Instead, the solver may be relying on the presence of trigger words. For
example, the words “more” and “together” are likely to signal addition even if
the model is given no additional context, while “each” may signal
multiplication or division.

For the datasets with only one part of speech and the number tokens remaining,
no extremely large jumps in accuracy were observed that would suggest that the
model relies entirely on one part of speech. However, accuracy was nearly
doubled from 12% to 23% with only nouns in the MaWPS model, which does suggest
at least some reliance on the presence of certain nouns in this model since
clearly with only nouns to go draw its conclusions, no logical reasoning of
events is possible.

##  5\. Experiment 2: MaWPS Word Frequency

Figure 6. A histogram of the percentage of words in a given MWP were removed
before the model produced an incorrect solution to the problem. This histogram
does not include MWPs that the model gave an incorrect prediction with the
original text.

In this experiment, we examine the diversity, or lack thereof, of words in the
MaWPS’ dataset’s vocabulary. Our work is intended to reveal possible trigger
words that may frequently appear in some types of problems but not others.

###  5.1. Methods

We looked at the word frequency of words in the first cross-validation fold of
the MaWPS dataset, both the training and testing datasets. The problem texts
were first set to all lowercase letters, then stemmed and lemmatized in order
to count all occurrences of the words.

We counted the number of MWPs that each word appeared in rather than the total
number of appearances of each word. We found the top 50 words, by number of
MWPs the word appeared in, for every operation type (+, -, *, /, multiple),
then filtered out any words that appeared in every list. In this way we can
see which words are uniquely frequent in specific operations, and are not just
frequent in the corpus overall.

###  5.2. Results

The results are shown in Table 1. We can see that these words often appear in
10-20% of all problems of a given type, though the majority of the words do
not appear to have any correlation to the type of operation that they most
often appear in.

###  5.3. Discussion

None of the most popular words appeared to be relevant to the category of
problem that they most frequently appeared in. The fact that these words are
appearing so frequently indicates a low lexical diversity in the MaWPS
dataset, which may encourage the model to rely on the occurrence of these
words to classify problems into different operations.

##  6\. Experiment 3: Input Reduction

We used input reduction to uncover how many words can be removed from an MWP
before the model will produce an incorrect answer. If very few words remain
and have little to do with the correct equation, it suggests that the model is
not performing much semantic reasoning between quantities in order to find the
correct equation.

###  6.1. Methods

Our approach is based on the work of Feng et al. ((2018)), but does not follow
their exact methodology. We implemented confidence scores using the posterior
probability of each label, summed those probabilities and divided by the
number of outputs, since we were using an RNN model. For the input reduction
process, we iteratively removed the word which reduced the model’s confidence
score the least.

We used only the RNN Seq2seq model created from the first CV fold of MaWPS for
our input reduction predictions.

###  6.2. Results

A histogram of the percentage of words removed when the model gave an
incorrect prediction is shown in Figure 6. The histogram does not include MWPs
that the model got wrong with the original text. The mean percentage of words
removed is 62.3%, while the median is 68.1%. This means that the model
produces the correct prediction with less than 68.1% of the words for half of
the problems it is able to solve.

An example of the input reduction process is shown in Figure 4. In this
example, 22 words are removed before the model produces an incorrect equation.
The most reduced input to receive a correct equation is “his number0 each his
number1 many,” which arguably contains little to no information about what the
correct equation is, and yet the model still solves the problem with high
confidence (99%).

| Score | Model Confidence | Removed Word |  Question
---|---|---|---|---
0 | Correct | 0.999997 | nan |  Emily collects number0 cards . Emily ’s father gives Emily number1 more . Bruce has number2 apples . How many cards does Emily have ?
1 | Correct | 0.999999 | does |  Emily collects number0 cards . Emily ’s father gives Emily number1 more . Bruce has number2 apples . How many cards Emily have ?
2 | Correct | 0.999999 | has |  Emily collects number0 cards . Emily ’s father gives Emily number1 more . Bruce number2 apples . How many cards Emily have ?
3 | Correct | 0.999999 | how |  Emily collects number0 cards . Emily ’s father gives Emily number1 more . Bruce number2 apples . many cards Emily have ?
4 | Correct | 0.999998 | collects |  Emily number0 cards . Emily ’s father gives Emily number1 more . Bruce number2 apples . many cards Emily have ?
5 | Correct | 0.999998 | ’s |  Emily number0 cards . Emily father gives Emily number1 more . Bruce number2 apples . many cards Emily have ?
6 | Correct | 0.999998 | emily |  number0 cards . father gives number1 more . Bruce number2 apples . many cards have ?
7 | Correct | 0.999997 | gives |  number0 cards . father number1 more . Bruce number2 apples . many cards have ?
8 | Correct | 0.999995 | more |  number0 cards . father number1 . Bruce number2 apples . many cards have ?
9 | Correct | 0.999995 | father |  number0 cards . number1 . Bruce number2 apples . many cards have ?
10 | Correct | 0.999994 | have |  number0 cards . number1 . Bruce number2 apples . many cards ?
11 | Correct | 0.999993 | many |  number0 cards . number1 . Bruce number2 apples . cards ?
12 | Correct | 0.999968 | ? |  number0 cards . number1 . Bruce number2 apples . cards
13 | Correct | 0.995461 | cards |  number0 . number1 . Bruce number2 apples .
14 | Incorrect | 0.944366 | bruce |  number0 . number1 . number2 apples .
Table 4. An example of the input reduction process.

###  6.3. Discussion

The results of the input reduction experiment show that in most cases more
than half of the total words can be removed from the MWP before the model
produces an incorrect answer. With over half of the words removed, these
problems are nonsensical to humans, as in Table 4. This indicates that the
model is not truly performing reasoning about the sequence of events explained
in the problem, since it can still produce a correct equation with over half
of the information removed from the input.

##  7\. Future Work

We would like to implement the gradient-based method used by Feng et al.
((2018)) in order to obtain a more objective idea of how much removing a given
word affects the model. The current confidence score approach produces very
high confidence on almost every input, even when it is wrong, which reduces
the credibility of our input reduction results.

We would also like to implement the high-entropy output fine-tuning suggested
by Feng et al. ((2018)) to possibly improve the interpretability and accuracy
of the RNN Seq2seq MWP solver.

Another possible avenue of word would be to increase the lexical diversity of
MaWPS by writing code to change words to synonyms before the MWPs are fed into
the model for training. This way, the model would not be able to rely on the
high frequency of certain words to make its predictions.

##  8\. Conclusion

The results of Experiment 1, parts of speech removal, indicated a small
reliance on some parts of speech, especially nouns and verbs. The AsDIV-A
model was also shown to be more reliant on specific parts of speech than
MaWPS, perhaps indicating some overfitting to those words. Experiment 2, word
frequency in MaWPS, shows that the lexical diversity of MaWPS is low.
Experiment 3, input reduction, shows that well over half of the words in a
given MWP can be removed before the model gives an incorrect prediction. This
shows that the model is not using all of the information in the question to
make its prediction, and may be relying on occurrences of some of the words
from Experiment 2, or some other superficial patterns, to make its
predictions.

###### Acknowledgements.

The work reported in this paper is supported by the National Science
Foundation under Grant No. 2050919. Any opinions, findings, and conclusions or
recommendations expressed in this work are those of the author(s) and do not
necessarily reflect the views of the National Science Foundation.

## References

  * (1)
  * Bird et al. (2009) Steven Bird, Ewan Klein, and Edward Loper. 2009.  _Natural language processing with Python: analyzing text with the natural language toolkit_.  O’Reilly Media, Inc.  https://www.nltk.org/book/
  * Feng et al. (2018) Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, and Jordan Boyd-Graber. 2018.  Pathologies of Neural Models Make Interpretations Difficult.  https://doi.org/10.48550/arXiv.1804.07781 arXiv:1804.07781.
  * Griffith and Kalita (2019) Kaden Griffith and Jugal Kalita. 2019.  Solving Arithmetic Word Problems Automatically Using Transformer and Unambiguous Representations. In _2019 International Conference on Computational Science and Computational Intelligence (CSCI)_. 526–532.  https://doi.org/10.1109/CSCI49370.2019.00101
  * Jia and Liang (2017) Robin Jia and Percy Liang. 2017.  Adversarial Examples for Evaluating Reading Comprehension Systems.  https://doi.org/10.48550/arXiv.1707.07328 arXiv:1707.07328.
  * Koncel-Kedziorski et al. (2016) Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. 2016\.  MAWPS: A Math Word Problem Repository. In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. Association for Computational Linguistics, San Diego, California, 1152–1157.  https://doi.org/10.18653/v1/N16-1136
  * Kumar et al. (2021) Vivek Kumar, Rishabh Maheshwary, and Vikram Pudi. 2021\.  _Adversarial Examples for Evaluating Math Word Problem Solvers_.  Technical Report arXiv:2109.05925. arXiv.  https://doi.org/10.48550/arXiv.2109.05925 arXiv:2109.05925.
  * Kumar et al. (2022) Vivek Kumar, Rishabh Maheshwary, and Vikram Pudi. 2022\.  _Practice Makes a Solver Perfect: Data Augmentation for Math Word Problem Solvers_.  Technical Report arXiv:2205.00177. arXiv.  https://doi.org/10.48550/arXiv.2205.00177
  * Lee et al. (2019) Gyeongbok Lee, Sungdong Kim, and Seung-won Hwang. 2019\.  QADiver: Interactive Framework for Diagnosing QA Models.  _Proceedings of the AAAI Conference on Artificial Intelligence_ 33, 01 (July 2019), 9861–9862.  https://doi.org/10.1609/aaai.v33i01.33019861
  * Li et al. (2017) Jiwei Li, Will Monroe, and Dan Jurafsky. 2017.  Understanding Neural Networks through Representation Erasure.  https://doi.org/10.48550/arXiv.1612.08220 arXiv:1612.08220.
  * Miao et al. (2021) Shen-Yun Miao, Chao-Chun Liang, and Keh-Yih Su. 2021.  _A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers_.  Technical Report arXiv:2106.15772. arXiv.  http://arxiv.org/abs/2106.15772
  * Patel et al. (2021) Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021\.  Are NLP Models really able to Solve Simple Math Word Problems?. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. 2080–2094.  https://doi.org/10.18653/v1/2021.naacl-main.168
  * Wallace et al. (2019) Eric Wallace, Jens Tuyls, Junlin Wang, Sanjay Subramanian, Matt Gardner, and Sameer Singh. 2019\.  AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models.  https://doi.org/10.48550/arXiv.1909.09251
  * Wang et al. (2017) Yan Wang, Xiaojiang Liu, and Shuming Shi. 2017.  Deep Neural Solver for Math Word Problems. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Copenhagen, Denmark, 845–854.  https://doi.org/10.18653/v1/D17-1088
  * Xie and Sun (2019) Zhipeng Xie and Shichao Sun. 2019.  A Goal-Driven Tree-Structured Neural Model for Math Word Problems. In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence_. Macao, China, 5299–5305.  https://doi.org/10.24963/ijcai.2019/736
  * Zhang et al. (2020) Jipeng Zhang, Lei Wang, Roy Ka-Wei Lee, Yi Bin, Yan Wang, Jie Shao, and Ee-peng LIM. 2020.  Graph-to-tree learning for solving math word problems.  _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ (July 2020), 3928–3937.  https://doi.org/10.18653/v1/2020.acl-main.362