k^{1/2-2\beta}\,\sum\limits_{d\ll
k}d^{\alpha-2\beta-\epsilon}\det(Y_{d})^{\beta}\sum\limits_{TY_{d}\in\mathcal{X}}1,$

where $\mathcal{X}$ is the set of diagonalizable matrices whose eigenvalues
are both of size $k/(4\pi)+O(\sqrt{k}\log(k))$. Then from [Blo15, Lemma 4], we
know that $\sum_{TY_{d}\in\mathcal{X}}1\ll k^{3/2+\epsilon}\det(Y)^{-3/4}$.

(7.24) $\ll k^{1/2-2\beta+3/2}\,\sum\limits_{d\ll
k}d^{\alpha-2\beta-\epsilon}\det(Y)^{\beta-3/4}.\qed$

Plugging in the values of $(\alpha,\beta)$ from (7.21) into Lemma 7.1 we get

(7.25) $\displaystyle q_{*}(Y)\ll
k^{5/4+\epsilon}+k^{5/4+1/12}\det(Y)^{-1/4}.$

So here we need to focus on the region $\det(Y)\ll k^{1/3}$ from methods
pertaining to the Bergman kernel for a suitable space. Further, note that
since $\det Y\ll k^{1/3}$ and $v\leq v^{\prime}$ and $vv^{\prime}\asymp\det
Y$, we also have

(7.26) $v\ll k^{1/6},\quad v^{\prime}\ll k^{1/3}.$

####  7.3.1. The remaining region

In this subsection, we assume that $v,v^{\prime}$ satisfy (7.26), which
defines a region $\mathcal{R}$. In this region $\mathcal{R}$ we would use the
information on the Bergman kernel on the space of Jacobi forms. To do this, we
embed the space spanned by the $V_{m}$-images of $J_{k,1}^{cusp}$ into
$J^{cusp}_{k,m}$ and use the results from section 4.2 on
$\mathbb{B}_{k,m}(\tau,z)$; it turns out to be sufficient for the region under
consideration. First, let us recall the bound for $\mathbb{B}_{k,m}(\tau,z)$:

(7.27) $\displaystyle\mathbb{B}_{k,m}(\tau,z)\ll
km\left(1+vk^{-1/2+\epsilon}+vk^{-A}\right)\left(1+v^{1/2}m^{-1/2}\right).$

The first quantity in braces above is bounded in $\mathcal{R}$. Moreover, we
can rewrite the above bound as

(7.28) $\mathbb{B}_{k,m}(\tau,z)\ll
km^{1/2}v^{1/2}\left(1+vk^{-1/2+\epsilon}+vk^{-A}\right)\left(1+m^{1/2}v^{-1/2}\right).$

Therefore in $\mathcal{R}$,

(7.29)
$\displaystyle\mathbb{B}_{k,m}(\tau,z)\ll\begin{cases}km^{1/2}v^{1/2}&\text{
if }m\leq v;\\\ km&\text{ if }v\leq m.\end{cases}$

First note that

$\displaystyle\sum\limits_{F}\det(Y)^{k}|F(Z)|^{2}$ $\displaystyle\leq
t^{k}\sum\limits_{F}\left(\sum\limits_{m\geq 1}v^{k/2}\exp(-2\pi
my^{2}/v)|\phi_{m,F}|\exp(-2\pi mt)\right)^{2}$ (7.30) $\displaystyle\leq
t^{k}\sum\limits_{m,m^{\prime}}\,\sum\limits_{F}\widetilde{\phi_{m,F}}\widetilde{\phi_{m^{\prime},F}}\,\exp(-2\pi(m+m^{\prime})t)$
(7.31) $\displaystyle\leq t^{k}(\sum\limits_{m}p(m)^{1/2}\exp(-2\pi mt))^{2},$

where we have put

(7.32) $\displaystyle
p(m):=\sum\limits_{F}\widetilde{\phi_{m,F}(\tau,z)}^{2},\quad
t=v^{\prime}-y^{2}/v.$

Next, we want to check that if $F,G\in\mathrm{SK}_{k}$ are orthogonal, then so
are $\phi_{m,F},\phi_{m,G}$.

######  Lemma 7.2.

Let $F,G\in\mathrm{SK}_{k}$ be Hecke eigenforms such that $\langle
F,G\rangle=0$. Then for all $m\geq 1$,
$\langle\phi_{m,F},\phi_{m,G}\rangle=0$.

###### Proof.

Let $F,G$ be SK lifts of $f,g\in S_{2k-2}$ respectively. We have (cf. [KS89,
p. 549-550])

$\displaystyle\langle\phi_{m,F},\phi_{m,G}\rangle$ $\displaystyle=\langle
V_{m}\phi_{1,F},V_{m}\phi_{1,G}\rangle=\langle
V_{m}^{*}V_{m}\phi_{1,F},\phi_{1,G}\rangle$ (7.33)
$\displaystyle=\langle\sum\limits_{d|m}\psi(d)d^{k-2}T^{J}(m/d)\phi_{1,F},\phi_{1,G}\rangle=\sum\limits_{d|m}\psi(d)d^{k-2}\lambda_{f}(m/d)\langle\phi_{1,F},\phi_{1,G}\rangle,$

where $\psi(t)=t\prod_{p|t}(1+1/p)$. Here we should keep in mind the
convention made before (1.2). Moreover, $\langle F,G\rangle=0$ implies that
$\langle\phi_{1,F},\phi_{1,G}\rangle=0$ (see [KS89, Theorem 2]). The lemma
follows.∎

The above proof also shows that

(7.34) $\displaystyle\langle\phi_{m,F},\phi_{m,F}\rangle\ll
m^{k-1+\epsilon}\cdot\langle\phi_{1},\phi_{1}\rangle.$

Note however that $\phi_{m,F}$ could be $0$ for some $m$. Therefore, when
$m\geq v$,

(7.35) $\displaystyle
p(m)=\sideset{}{{}^{*}}{\sum\limits}_{F}\frac{\widetilde{\phi_{m,F}}^{2}}{\langle\phi_{m,F},\phi_{m,F}\rangle}\cdot\langle\phi_{m,F},\phi_{m,F}\rangle\ll
km\cdot m^{k-1+\epsilon}\pi^{k}c_{k}\ll\frac{(4\pi m)^{k}}{\Gamma(k-1)},$

where $\sum^{*}$ denotes the sum over non-zero modular forms. Thus this part
of the Fourier series gives the contribution $k^{5/2+\epsilon}$ because

(7.36) $\displaystyle\sum\limits_{m\geq v}\cdots\leq\frac{(4\pi
t)^{k}}{\Gamma(k-1)}(\sum\limits_{m\geq 1}m^{k/2}\exp(-2\pi mt))^{2}\ll
2^{k}\frac{\Gamma(k/2+1)^{2}}{\Gamma(k-1)}\ll k^{5/2+\epsilon}.$

For the terms where $m\leq v$ we use the first inequality in (7.29) to get

(7.37) $p(m)\ll k^{13/12}m^{1/2}(4\pi)^{k}\,\Gamma(k)^{-1},$

and this gives the bound for $\sum_{m\leq v}$

(7.38) $\displaystyle\ll k^{13/12}\frac{(4\pi t)^{k}}{\Gamma(k)}(\sum_{m\leq
v}m^{k/2-1/4}\exp(-2\pi mt))^{2}\ll
2^{k}k^{13/12}\frac{t^{1/2}}{\Gamma(k)}\cdot\Gamma(k/2+3/4)^{2}\ll
k^{9/4+\epsilon}.$

Thus combining (7.38) and (7.36) we get from (7.31) that

(7.39) $\sum\nolimits_{F\in\mathcal{B}^{*}_{k}}\det(Y)^{k}|F(Z)|^{2}\ll
k^{5/2+\epsilon}.$

###  7.4. Upper bound – second method

In this section, we provide an alternate proof of Theorem 1.1, that is
independent of the bounds for the geometric side of the Bergman kernel of
Jacobi forms. In this direction, we start by considering the sum

(7.40) $\displaystyle A(Z):=\sum\limits_{F\in
B_{k}^{*}}\frac{\det(Y)^{k}|F(Z)|^{2}}{\langle F,F\rangle},$

where $B^{*}_{k}$ denotes a Hecke basis for $\mathrm{SK}_{k}$ (cf. the
convention before (1.2)).

Let $\tau=u+iv$, $z=x+iy$ and $\tau^{\prime}=u^{\prime}+iv^{\prime}$ and
$\phi\in J_{k,1}^{cusp}$ denote the first Fourier–Jacobi coefficient of $F$.
Consider the expression

(7.41) $\displaystyle\det(Y)^{-k}A(Z)$
$\displaystyle=\sum\limits_{F}\frac{|F(Z)|^{2}}{\langle
F,F\rangle}=\pi^{k}c_{k}\sum\limits_{F}\frac{|F(Z)|^{2}}{\langle\phi,\phi\rangle}\frac{1}{|L(k,f)|}$
(7.42)
$\displaystyle\ll\pi^{k}c_{k}\sum\limits_{\phi}\frac{1}{\langle\phi,\phi\rangle}\sum\limits_{r,s}\phi_{r}(\tau,z)\overline{\phi_{s}(\tau,z)}e(r\tau^{\prime}-s\tau^{\prime}).$

where in (7.41) we have used (7.13) and bounded the quantity $L(k,f)^{-1}$ by
an absolute constant, since $k$ falls in the region of absolute convergence.

We have $\phi_{r}=\phi|V_{r}$ with, $\phi\in J_{k,1}$. Thus, we only have to
bound the quantity:

(7.43)
$A^{*}(Z):=\pi^{k}c_{k}\sum\limits_{\phi}\frac{1}{\langle\phi,\phi\rangle}\sum\limits_{r,s}\phi(\tau,z)|V_{r}\cdot\overline{\phi(\tau,z)|V_{s}}\,e(r\tau^{\prime}-s\tau^{\prime}).$

Let us now replace $\phi/\left\lVert\phi\right\rVert_{2}$ by $\phi$ – which
therefore henceforth will run over an orthonormalized basis of
$J_{k,1}^{cusp}$. With this arrangement, and using the Fourier expansion of
$\phi|V_{r}$, $A^{*}(Z)$ can be written as

(7.44)
$\sum_{\phi}\sum_{r,s}\sum_{n_{1},n_{2},t_{1},t_{2}}\sum_{d_{1}|(n_{1},t_{1},r)}\sum_{d_{2}|(n_{2},t_{2},s)}(d_{1}d_{2})^{k-1}c_{\phi}(\tfrac{n_{1}r}{d_{1}^{2}},\tfrac{t_{1}}{d_{1}})\overline{c_{\phi}(\tfrac{n_{2}s}{d_{2}^{2}},\tfrac{t_{2}}{d_{2}})}e(\cdots).$

For $i=1,2$, write $D_{i}=4n_{i}r-t_{i}^{2}$ and put
$T_{i}=\left(\begin{smallmatrix}(D_{i}+t_{i}^{2})/4r&t_{i}/2\\\ t_{i}/2&r\\\
\end{smallmatrix}\right)$. Also, put

(7.45)
$S(D_{1},D_{2}):=\sum\limits_{\phi}c_{\phi}(D_{1},t_{1})\overline{c_{\phi}(D_{2},t_{2})}.$

Thus we get

(7.46)
$\begin{split}A^{*}(Z)=&\pi^{k}c_{k}\sum_{r,s}\sum_{D_{1},D_{2},t_{1},t_{2}}\sum_{d_{1}|c(T_{1})}\sum_{d_{2}|c(T_{2})}(d_{1}d_{2})^{k-1}S(D_{1}d_{1}^{-2},D_{2}d_{2}^{-2})e(\cdots)\\\
&=\pi^{k}c_{k}\sum_{T_{1},T_{2}>0}\sum_{d_{1}|c(T_{1})}\sum_{d_{2}|c(T_{2})}(d_{1}d_{2})^{k-1}S(D_{1}d_{1}^{-2},D_{2}d_{2}^{-2})e(Tr(T_{1}Z+T_{2}Z)).\end{split}$

Now we use the Poincaré series to estimate the sum $S(D_{1},D_{2})$. Since
$\\{\phi\\}$ form an orthonormal basis, we can write

(7.47)
$2\pi^{k-\frac{3}{2}}D_{2}^{k-\frac{3}{2}}P_{D_{2}}=\Gamma(k-3/2)\sum\limits_{\phi}\overline{c_{\phi}(D_{2})}\phi.$

Comparing the Fourier coefficients at $D_{1}$ on both sides, we see that

(7.48)
$\Gamma(k-3/2)S(D_{1},D_{2})=2\pi^{k-\frac{3}{2}}D_{2}^{k-\frac{3}{2}}p_{D_{2}}(D_{1}).$

Now we estimate proceed to estimate the coefficients $p_{D_{2}}(D_{1})$.
First, we observe here that the Fourier coefficients of the Jacobi Poincaré
series of index $1$ are the same as that of the half–integral weight Poincaré
series in the Kohnen-plus space (see [Das10, Proposition 6.2]). This allows us
to get better bounds for the Fourier coefficients $p_{D_{2}}(D_{1})$ as
compared to the bounds in Proposition 4.1.

######  Proposition 7.3.

For $k$ large, we have

(7.49) $\displaystyle
p_{D_{2}}(D_{1})\ll\delta_{D_{1},D_{2}}+\left(\tfrac{D_{1}}{D_{2}}\right)^{k/2-3/4}\left(\tfrac{(D_{1}D_{2})^{1/4+\epsilon}}{k}+\tfrac{(D_{1}D_{2})^{\epsilon}}{k^{1/3}}\right).$

Thus we get the following bound from (7.48);

######  Proposition 7.4.

(7.50) $\displaystyle
S(D_{1},D_{2})\ll\tfrac{\pi^{k}}{\Gamma(k-3/2)}\cdot\left(\delta_{D_{1},D_{2}}D_{2}^{k-3/2}+\tfrac{(D_{1}D_{2})^{k/2-1/2}}{k}+\tfrac{(D_{1}D_{2})^{k/2-3/4+\epsilon}}{k^{1/3}}\right).$

Now we proceed to bound $A^{*}(Z)$. We write $A^{*}(Z)\ll
A_{1}(Z)+A_{2}(Z)+A_{3}(Z)$ corresponding to three terms in (7.50) and we
bound each $A_{i}(Z)$ as below.

####  7.4.1. Bounding $A_{i}(Z)$:

(7.51) $\displaystyle
A_{1}(Z)\ll\frac{(4\pi)^{k}(4\pi)^{k-\frac{3}{2}}}{\Gamma(k)\Gamma(k-\frac{3}{2})}\sum\limits_{T>0}\sum\limits_{d\mid\mathbf{c}(T)}d^{k-1}\mathrm{det}(d^{-1}T)^{k-3/2}e^{-4\pi
Tr(TY)}\ll\frac{k^{3/2}Q(1/4,3/4;Y)^{2}}{\det(Y)^{k}},$

where $Q(\alpha,\beta;Y)$ is as in (7.22). Therefore, from Lemma (7.1) we see
that,

(7.52) $\displaystyle A_{1}(Z)$ $\displaystyle\ll k^{3/2}\cdot
k^{1+\epsilon}\det(Y)^{-k}\ll k^{5/2+\epsilon}\det(Y)^{-k}$

and similarly

(7.53) $\displaystyle A_{2}(Z)$ $\displaystyle\ll
k^{1/2}Q(\epsilon,1/2;Y)^{2}\det(Y)^{-k}\ll k^{5/2}\det(Y)^{-k-1/2},$

from Lemma 7.1 with $\alpha=\epsilon$ and $\beta=1/2$.

(7.54) $\displaystyle A_{3}(Z)$ $\displaystyle\ll
k^{7/6}Q(1/2,3/4;Y)^{2}\det(Y)^{-k}\ll k^{13/6+\epsilon}\det(Y)^{-k},$

by Lemma 7.1 with $\alpha=1/2$ and $\beta=3/4$. This gives us $A(Z)\ll
k^{5/2+\epsilon}$, which finishes the proof.

##  8\. Higher moments and scope for improvement via subconvexity

In this section, we investigate the relation between the subconvexity bounds
(for example, see [You17]) on the twisted central $L$-values
$L(1/2,f\otimes\chi_{D})$ of a $\operatorname{GL}(2)$ Hecke eigenform $f$ and
the higher moments of $\det(Y)^{k/2}|F(Z)|$ along an orthonormal Hecke basis.
Put, for any $r>0$,

(8.1)
$\mathscr{F}_{2r}:=\sup\nolimits_{Z\in\mathcal{F}_{2}}\sum\limits_{F}\det(Y)^{kr}|F(Z)|^{2r}.$

For any $r\in\mathbb{R}$, $r>1$, and $Z\in\mathbb{H}_{2}$ consider the
inequality

(8.2)
$\displaystyle\sum\limits_{F}\det(Y)^{kr}|F(Z)|^{2r}\ll\left(\max_{F}\det(Y)^{k}|F(Z)|^{2}\right)^{r-1}\sum\limits_{F}\det(Y)^{k}|F(Z)|^{2};$

which shows immediately that $\mathscr{F}_{2r}\ll k^{5r/2+\epsilon}$ if we use
Theorem 1.1.

Next, consider the following bound for the third moment of twisted central
$L$-values $L(1/2,f\otimes\chi_{D})$ from [You17], where $D$ is an odd
fundamental discriminant (we assume it here for every fundamental
discriminant): for any $\epsilon>0$, one has

(8.3) $\sum\limits_{f\in
B_{k}}L(1/2,f\otimes\chi_{D})^{3}\ll_{\epsilon}k^{1+\epsilon}D^{1-\delta+\epsilon}$

for some absolute constant $\delta$ such that $0\leq\delta\leq 1$. Thus, we
assume an improvement the $D$ aspect of Young’s result in [You17]. Our goal in
the rest of this section is to show that this results in an improvement of the
upper bound of $k^{15/2+\epsilon}$ of $\mathscr{F}_{6}$ from (8.2). It seems
to us that this is the only viable avenue to improve the individual bounds for
an SK lift. In [Blo15], assuming Generalized Lindelöf Hypothesis for
$L(1/2,f\otimes\chi_{D})$ for all negative fundamental discriminants, it was
shown that $\left\lVert F\right\rVert_{\infty}\ll k^{3/4+\epsilon}$, which is
the best possible, as there exist eigenforms with sup-norm $\gg k^{3/4}$.

We start with the inequality:

$\displaystyle\sum\limits_{F}\big{(}\frac{\det(Y)^{k/2}|F(Z)|}{\langle
F,F\rangle^{\frac{1}{2}}}\big{)}^{6}=\sum\limits_{F}\frac{\det(Y)^{3k}|F(Z)|^{6}}{\langle
F,F\rangle^{3}}\ll(\sum\limits_{T}p_{6}(T)^{\frac{1}{6}}\det(Y)^{\frac{k}{2}}\exp(-TY))^{6},$

where we have put $p_{6}(T)=\sum_{F}|a_{F}(T)|^{6}/\langle F,F\rangle^{3}$.

For any $T$, write $4\det(T)=D=f^{2}D_{0}$ where $D_{0}$ is fundamental. This
expression is unique. This $f$, we believe has no chance of being confused
with the lifted modular form $f\in S_{2k-2}$. Looking at the expression for
$a_{F}(T)$ from (2.15), which involves the terms $\det(2T)/a^{2}$, we have to
determine $(D/a^{2})_{0}$, for a given $T$ and $a$ as above, to relate them to
fundamental discriminants. Clearly, if $n_{o}$ denotes the ‘odd’ part of an
integer $n$, then $\mathbf{c}(T)_{o}|f_{o}$. Thus $(D/a^{2})_{0}=D_{0}$ if $a$
is odd. So is the case if $D_{0}$ is odd.

Thus we only have to look at $p=2$. Let $n_{2}$ be the $2$-part of $n$. If
$2^{\beta}\|a$, then $2^{2\beta}|f^{2}D_{0}$. But we know that
$\nu_{2}(D_{0})\leq 3$, so $2^{2\beta-3}|f^{2}$ which means
$2^{2\beta-2}|f^{2}$. Therefore, writing $D_{0}=4m$ ($m\equiv 2,3\bmod 4$ and
square-free)

(8.4)
$\displaystyle\frac{D}{a^{2}}=\frac{f^{2}D_{0}}{a^{2}}=\frac{(f_{o}/a_{o})^{2}f_{2}^{2}D_{0}}{a_{2}^{2}}=\frac{(f_{o}/a_{o})^{2}(f_{2}/2^{\beta-1})^{2}D_{0}}{4}=A^{2}\cdot
m,$

for some $A\in\mathbb{N}$. Since $m\equiv 2,3\bmod 4$ and square-free, the
only way to write $A^{2}\cdot m$ as $B^{2}\cdot\mathcal{D}$ ($\mathcal{D}$
fundamental discriminant)–is $\mathcal{D}=4m$ and $B=A/2$. This will show that
$(D/a^{2})_{0}=D_{0}$. In effect, we have also proved above that $a|f$ (in
particular $\mathbf{c}(T)|f$). Therefore

$\displaystyle\sum\limits_{a|\mathbf{c}(T)}a^{k-1}c_{\phi}(4\det(T)/a^{2})=c_{\phi}(|D_{0}|)\sum\limits_{a|\mathbf{c}(T)}a^{k-1}\sum\limits_{b|\frac{f}{a}}\mu(b)\left(\frac{D}{b}\right)b^{k-3/2}a_{f}(\frac{f}{ab})$
(8.5) $\displaystyle\leq
c_{\phi}(|D_{0}|)\sum\limits_{a|\mathbf{c}(T)}a^{k-1}(f/a)^{k-3/2+\epsilon}\ll
c_{\phi}(|D_{0}|)f^{k-3/2+\epsilon}\mathbf{c}(T)^{1/2}.$

Keeping in mind (8.3), this gives us

$\displaystyle
p_{6}(T)\leq\sum\limits_{F}\frac{c_{\phi}(|D_{0}|)^{6}f^{6(k-\frac{3}{2})+\epsilon}\mathbf{c}(T)^{3}}{\langle
F,F\rangle^{3}}\ll\frac{\det(T)^{3(k-\frac{3}{2})+\epsilon}\mathbf{c}(T)^{3}}{k^{3}\Gamma(2k-2)^{3}}\sum\limits_{f}\frac{L(k-1,f\otimes\chi_{D_{0}})^{3}}{L(k,f)^{3}L(k,\mathrm{sym}^{2}f)^{3}},$

where we have used the relation between Petersson norms from (7.13). Thus,

(8.6) $\displaystyle p_{6}(T)$
$\displaystyle\ll\frac{\det(T)^{3k-9/2+\epsilon}\mathbf{c}(T)^{3}}{k^{3}\Gamma(2k-2)^{3}}k^{1+\epsilon}D_{0}^{1-\delta+\epsilon}\ll\frac{k^{5/2+\epsilon}}{\Gamma(k)^{6}}\det(T)^{3k-7/2-\delta+\epsilon}\mathbf{c}(T)^{1-\epsilon}.$

In conclusion therefore, we get (with the function $Q(\cdot,\cdot;Y)$ from
(7.22))

(8.7) $\displaystyle\sum\limits_{F}\det(Y)^{3k}|F(Z)|^{6}/\langle
F,F\rangle^{3}\ll k^{5/2+\epsilon}Q(1/6,7/12+\delta/6-\epsilon;Y)^{6}\ll
k^{15/2-2\delta+\epsilon}\det(Y)^{\delta-1},$

which is bounded by $k^{15/2-2\delta+\epsilon}$ since $\delta\leq 1$. For an
individual $F$, we would have

(8.8) $\det(Y)^{k/2}|F(Z)|\ll k^{5/4-\delta/3+\epsilon}.$

##  9\. $L^{\infty}$ mass of pullback of SK lifts

Here we would first restrict $F_{f}\in\mathrm{SK}_{k}$ ($f\in S_{2k-2}$) to
the diagonal. The resulting object (pullback) can be written as

(9.1) $\displaystyle
F_{f}^{o}(\tau,\tau^{\prime}):=F_{f}|_{z=0}=\sum\limits_{g\in
B_{k}}c_{g,f}g(\tau)g(\tau^{\prime}),$

and by Ichino (see [Ich05]), the quantities $c_{g,f}$ are given by $L$-values.
More precisely,

(9.2) $\displaystyle|c_{g,f}|^{2}=\frac{|\langle F_{f}|_{z=0},g\times
g\rangle|^{2}}{\langle
g,g\rangle^{4}}=2^{-k-1}\frac{\Lambda(1/2,\mathrm{sym}^{2}g\times f),\langle
h,h\rangle}{\langle f,f\rangle\,\langle g,g\rangle^{2}}.$

Our goal in this section is to understand the size of the space
$\mathrm{SK}_{k}^{\circ}$ spanned by the pullbacks of SK lifts; the
$L^{2}$-size of the same was studied in [LY14], [BKY13]. In order to quantify
the concentration of $F_{f}$ along the diagonally embedded
$\mathbb{H}\times\mathbb{H}\subset\mathbb{H}_{2}$, we introduce the following
three quantities (for $f\neq 0$)

(9.3) $\displaystyle
M_{1}(F_{f})=\frac{\sup(F_{f}^{\circ})}{\sup(F_{f})};\quad
M_{2}(F_{f})=\frac{\sup(F_{f}^{o})}{\left\lVert F_{f}\right\rVert};\quad
M_{3}(f):=\frac{\sup(F_{f}^{\circ})}{\left\lVert F_{f}^{\circ}\right\rVert}.$

We present a precise result on the size of the relevant object of this
subsection – the mass of pullbacks of SK lifts measured along
$\mathrm{SK}_{k}$. Recall that $B_{2k-2}$ denotes the set of newforms of level
one on $\operatorname{SL}_{2}(\mathbb{Z})$. Put

(9.4)
$\displaystyle\sup(\mathrm{SK}_{k}^{p}):=\sup\nolimits_{(\tau,\tau^{\prime})\in\mathbb{H}\times\mathbb{H}}\sum\limits_{f\in
B_{2k-2}}(vv^{\prime})^{k}|F_{f}^{o}(\tau,\tau^{\prime})|^{2}/\langle
F_{f},F_{f}\rangle\,\,(=\sup(\mathrm{SK}_{k}|_{z=0}))$

and to quantify the density of the pullbacks, we introduce the quantity

(9.5)
$d:=d(\mathrm{SK}_{k}^{p}):=\sup(\mathrm{SK}_{k}^{p})\big{/}\sup(\mathrm{SK}_{k}).$

Then $d\leq 1$ and the value of $d$ may be interpreted as the density of the
pullbacks inside the space of SK lifts in terms of $L^{\infty}$ mass – it
measures the proportion of the contribution of the pullbacks to the BK coming
from SK lifts. Note however that the summation in (9.4) runs over a spanning
set of $\mathrm{SK}_{k}^{\circ}$ but the forms $F_{f}^{\circ}$ may not be
orthogonal.

The bound on $d$ immediately gives us that $\sup(\mathrm{SK}_{k}^{p})\ll
k^{5/2+\epsilon}$. Next, we prove the lower bound
$\sup(\mathrm{SK}_{k}^{p})\gg k^{5/2}$ (cf. Proposition 9.1 below), which
shows that almost all of the "mass" of $\mathrm{SK}_{k}$ is concentrated along
$z=0$. A natural way to do this would be to appeal to (9.1), and via the
method of section 7.2; however, this would lead us to averages
$\sum_{f}(\sum_{g}c_{g,f})^{2}$ – as $\sum_{g}c_{g,f}$ is the $(1,1)$-th
Fourier coefficient of the RHS of (9.1). Since we have at present not much
control on this quantity, we take a different path.

We first consider the Fourier expansion of $F_{f}^{\circ}$ borrowed from that
of $F_{f}$ (for convenience, we use the notation $a_{F}(T)=a_{F}(n,r,m)$ when
$T=\left(\begin{smallmatrix}n&r/2\\\ r/2&m\\\ \end{smallmatrix}\right)$):

(9.6) $\displaystyle
F_{f}^{\circ}=\sum\limits_{n,m}(\sum\limits_{r}a_{F_{f}}(n,r,m))e(n\tau+m\tau^{\prime}).$

Let $v_{0}$, $v_{0}^{\prime}\in\mathbb{H}$. Then we have
$\sup(\mathrm{SK}_{k}^{p})$ is

$\displaystyle\gg(v_{0}v_{0}^{\prime})^{k}e^{-4\pi(v_{0}+v_{0}^{\prime})}\sum\limits_{f}\frac{|a_{F_{f}}(I_{2})+a_{F_{f}}(1,1,1)|^{2}}{\langle
F_{f},F_{f}\rangle}$ (9.7)
$\displaystyle\gg(v_{0}v_{0}^{\prime})^{k}e^{-4\pi(v_{0}+v_{0}^{\prime})}\Big{(}\sum\limits_{f}\frac{|a_{F_{f}}(I_{2})|^{2}}{\langle
F_{f},F_{f}\rangle}+\sum\limits_{f}\frac{|a_{F_{f}}(1,1,1)|^{2}}{\langle
F_{f},F_{f}\rangle}-2\Big{|}\sum\limits_{f}\frac{a_{F}(I_{2})\overline{a_{F}(1,1,1)}}{\langle
F_{f},F_{f}\rangle}\Big{|}\Big{)}.$

Now consider the individual terms on the LHS. Using the same arguments as in
(7.16),

(9.8)
$(v_{0}v_{0}^{\prime})^{k}e^{-4\pi(v_{0}+v_{0}^{\prime})}\sum\limits_{f}\frac{|a_{F_{f}}(I_{2})|^{2}}{\langle
F_{f},F_{f}\rangle}\gg k^{5/2}.$

Here we have made the choice $v_{0}=v_{0}^{\prime}=k/4\pi$. Similarly, since
$D=-3$ in the second term, using the same arguments as in (7.16), we get

(9.9)
$(v_{0}v_{0}^{\prime})^{k}e^{-4\pi(v_{0}+v_{0}^{\prime})}\sum\limits_{f}\frac{|a_{F_{f}}(1,1,1)|^{2}}{\langle
F_{f},F_{f}\rangle}\gg k^{5/2}(3\big{/}4)^{k}.$

Now to estimate the off diagonal term, we use Poincaré series. The off
diagonal term is given by

(9.10)
$(v_{0}v_{0}^{\prime})^{k}e^{-4\pi(v_{0}+v_{0}^{\prime})}\sum\limits_{f}\frac{a_{F}(I_{2})\overline{a_{F}(1,1,1)}}{\langle
F_{f},F_{f}\rangle}.$

Using the relation between Petersson norms from (7.13) and Proposition 7.4,
(9.10) is

(9.11)
$\displaystyle\ll\frac{(4\pi)^{k}}{\Gamma(k)}(v_{0}v_{0}^{\prime})^{k}e^{-4\pi(v_{0}+v_{0}^{\prime})}S(4,3)\ll\frac{\pi^{k}}{\Gamma(k-\frac{3}{2})}\frac{(4\pi)^{k}}{\Gamma(k)}\frac{k^{2k}}{(4\pi)^{2k}}e^{-2k}\frac{12^{k/2}}{k}\ll
k^{3/2}(3\big{/}4)^{k/2}.$

where $S(D_{1},D_{2})$ is as in (7.45). Thus we have proved

######  Proposition 9.1.

Let $\sup(\mathrm{SK}_{k}^{p})$ be as above. Then

(9.12) $k^{5/2}\ll\sup(\mathrm{SK}_{k}^{p})\ll k^{5/2+\epsilon}.$

Thus the density $d\asymp 1$ – it will be interesting to calculate it
explicitly.

######  Corollary 9.2.

For $f\in S_{k}$ normalized Hecke eigenform, one has the ‘trivial’ bound on
its size: $\sup_{\mathbb{H}}|F^{\circ}_{f}|/\left\lVert F_{f}\right\rVert\ll
k^{5/4+\epsilon}$.

###  9.1. The intrinsic BK

Recall the space $\mathrm{SK}_{k}^{\circ}$ spanned by the pullbacks of all
$F\in\mathrm{SK}_{k}$ and let $B_{k}^{\circ}\subset\mathrm{SK}_{k}^{\circ}$ be
an orthogonal basis. Further, put $(S_{k}\otimes S_{k})^{\circ}$ to be the
$\mathbb{C}$ span of $g\otimes g$ which is identified with
$g(\tau)g(\tau^{\prime}),\,g\in S_{k},\,$ a normalized Hecke eigenform. Call
this set $B_{k}^{\circ\circ}$. The following lemma gives a description of the
space of "pullbacks" of SK lifts.

######  Lemma 9.3.

With the above notation, $\mathrm{SK}_{k}^{\circ}=(S_{k}\otimes
S_{k})^{\circ}$.

###### Proof.

From (9.1), clearly $\mathrm{SK}_{k}^{\circ}\subset(S_{k}\otimes
S_{k})^{\circ}$. For the other direction, we compare dimensions.
$\dim(S_{k}\otimes S_{k})^{\circ}=\dim S_{k}$. Let us put
$\mathrm{SK}_{k}(0):=\\{F_{f}\mid F^{\circ}_{f}=0\\}$, so that
$\mathrm{SK}_{k}(0)$ is the kernel of the Witt map $W$ ($F\mapsto
F^{\circ}=F|_{z=0}$) restricted to $\mathrm{SK}_{k}$.

Then from [LY14, Proof of Thm. 1.9], we know that $\dim\mathrm{SK}_{k}(0)=\dim
M_{k-10}=\dim S_{k+2}$. Therefore, the dimension of the image
$W(\mathrm{SK}_{k})$ of the Witt map is $\dim S_{2k-2}-\dim S_{k+2}=\dim
S_{k}$. Since $W(\mathrm{SK}_{k})$ is simply $\mathrm{SK}_{k}^{\circ}$, we are
done. ∎

The lemma has a curious corollary about linear independence of (square-root)
central $L$-values. Recall the quantity $c_{g,f}$. Up to constants, its square
equals $L(1/2,\mathrm{sym}^{2}g\times f)$. Let $d_{1}=\dim S_{2k-2}$ and
$d_{2}=\dim S_{k}$. Consider $\mathcal{L}:=\mathcal{L}(d_{1},d_{2})$, the
$d_{1}\times d_{2}$ matrix $(c_{g,f})_{f,g}$ where $f,g$ run over normalized
Hecke eigenforms in $S_{2k-2},S_{k}$ respectively. Writing the elements
$F^{\circ}$ ($F$ Hecke eigenform in $\mathrm{SK}_{k}$) as a column, say
$C_{k}$, one can recast (9.1) into a matrix equality: $C_{k}=\mathcal{L}\cdot
B_{k}^{\circ\circ}$. If we let $\mathbf{L}$ to be the linear operator
corresponding to $\mathcal{L}$ mapping the span $(S_{k}\otimes S_{k})^{\circ}$
of $B_{k}^{\circ\circ}$ to the span $\mathrm{SK}_{k}^{\circ}$ of $C_{k}$.
Clearly $\mathbf{L}$ is onto. This implies that the rank of $\mathcal{L}$
equals $d_{2}$. We summarize this in the corollary below.

######  Corollary 9.4.

Let $\mathcal{L}$ be the $d_{1}\times d_{2}$ matrix as above. Then
$\operatorname{rank}(\mathcal{L})$ is maximal.

Recall that $B_{k}^{\circ}\subset\mathrm{SK}_{k}^{\circ}$ denotes an
orthogonal basis. Let us define

(9.13)
$\displaystyle\sup(\mathrm{SK}_{k}^{\circ}):=\sup\nolimits_{(\tau,\tau^{\prime})\in\mathbb{H}\times\mathbb{H}}\sum\nolimits_{G\in
B_{k}^{\circ}}(vv^{\prime})^{k}|G(\tau,\tau^{\prime})|^{2}/\langle
G,G\rangle.$

In view of Lemma 9.3, since the spaces $\mathrm{SK}_{k}^{\circ}$ and
$(S_{k}\otimes S_{k})^{\circ}$ are the same, we can also write (9.13) as

(9.14) $\sup(\mathrm{SK}_{k}^{\circ})=\sup((S_{k}\otimes
S_{k})^{\circ})=\sup\nolimits_{(\tau,\tau^{\prime})\in\mathbb{H}\times\mathbb{H}}\sum\nolimits_{g\in
B_{k}}(vv^{\prime})^{k}|g(\tau)g(\tau^{\prime})|^{2}/\langle g,g\rangle^{2}.$

######  Proposition 9.5.

With the above notation, $k^{2}\ll\sup(\mathrm{SK}_{k}^{\circ})\ll
k^{2+\epsilon}$.

###### Proof.

In view of the Lemma (9.3), for the basis $B_{k}^{\circ}$ we can take the set
$\\{g(\tau)\times g(\tau^{\prime}),\,g=h\left\lVert h\right\rVert_{2}^{-1}\in
S_{k}$, $h$ a normalized Hecke eigenform}. Then $a_{g\times
g}(m,n)=a_{g}(m)a_{g}(n)$. Therefore as in section 3.2, with $m=n=1$ and by
using the Cauchy–Schwarz and basic integral inequalities, we have
$\sup(\mathrm{SK}_{k}^{\circ})$ is

(9.15)
$\displaystyle\geq(vv^{\prime})^{k}\exp(-4\pi(v+v^{\prime}))\sum\limits_{h}|a_{h\times
h}(1,1)|^{2}\langle h,h\rangle^{-2}$ (9.16)
$\displaystyle\gg\frac{(vv^{\prime})^{k}\exp(-4\pi(v+v^{\prime}))}{k}\big{(}\sum\limits_{h}\frac{|a_{h}(1)|^{2}}{\langle
h,h\rangle}\big{)}^{2}\gg\frac{(vv^{\prime})^{k}\exp(-4\pi(v+v^{\prime}))}{k}\frac{(4\pi)^{2k}}{\Gamma(k-1)^{2}}\gg
k^{2},$

where we have chosen $v=v^{\prime}=k/4\pi$ and used the fact that the sum over
$h$ is $\asymp(4\pi)^{k-1}/\Gamma(k-1)$ (see [IK04, Proposition 14.5]).

For the upper bound we use individual sup-norm bounds for $h$: from the result
of Xia [Xia07], we know that $\left\lVert g\right\rVert_{\infty}\ll
k^{1/4+\epsilon}$ for $g$ as above. Thus, from (9.14), we see that

(9.17)
$\sum\nolimits_{g}(vv^{\prime})^{k}\frac{|g(\tau)g(\tau^{\prime})|^{2}}{\langle
g,g\rangle^{2}}\ll(k^{1/2+\epsilon})^{2}\cdot k\ll k^{2+\epsilon}.\qed$

######  Corollary 9.6.

For $F_{f}\in\mathrm{SK}_{k}$ a normalized Hecke eigenform, one has the
‘trivial’ bound on its size: $\sup_{\mathbb{H}}|F^{\circ}_{f}|/\left\lVert
F^{\circ}_{f}\right\rVert\ll k^{1+\epsilon}$, provided $F^{\circ}_{f}\neq 0$.

Note that we of course have the bound
$\sup_{\mathbb{H}}|F^{\circ}_{f}|/\left\lVert F_{f}\right\rVert\ll k^{5/4}$
from Corollary 9.2 or from Theorem 1.1 by restriction of domain. If we assume
the conjectural size $N(F_{f})\sim 2$ as $k\to\infty$ (cf. [LY14], [BKY13]),
then it is immediate from Corollary 9.6 that
$\sup_{\mathbb{H}}|F^{\circ}_{f}|/\left\lVert F_{f}\right\rVert\ll
k^{1+\epsilon}$, and $k^{1/2+\epsilon}$ if we expect that Corollary 9.6 can be
improved to $k^{1/2+\epsilon}$. To see this, we write and note
$|F^{\circ}_{f}|/\left\lVert F_{f}\right\rVert=|F^{\circ}_{f}|/\left\lVert
F^{\circ}_{f}\right\rVert\cdot\left\lVert
F^{\circ}_{f}\right\rVert/\left\lVert
F_{f}\right\rVert\asymp|F^{\circ}_{f}|/\left\lVert
F^{\circ}_{f}\right\rVert\cdot N(F_{f})$. However, the current record for the
size of $N(F_{f})$ is $N(F_{f})\ll k^{1-\delta}$ (cf. [AD18]) for a very small
$\delta\approx 0.0047$ and this does not help (unconditionally) towards the
betterment of Corollary 9.2 that one would like to have.

###  9.2. Appendix to section 9

In this section, we calculate the $L^{\infty}$ size of the image of the Witt
operator on $S^{2}_{k}$ which we define by (so $W(F)=F^{\circ}$ from the
previous section)

(9.18) $\displaystyle W(F)(\tau,\tau^{\prime}):=F_{z=0}.$

Since the arguments here are closely related to our ideas in this paper, and
especially to this section, we feel it is apt to include this here as an
appendix. Assume that $k$ is even. Recall that our point of interest is

(9.19)
$\displaystyle\sup(W(S^{2}_{k}))=\sup\nolimits_{\tau_{1},\tau_{2}\in\mathbb{H}}\sum\limits_{F\in\mathcal{BW}(S^{2}_{k})}\frac{(v_{1}v_{2})^{k/2}|W(F)|^{2}}{\langle
W(F),W(F)\rangle},$

where $\mathcal{BW}(S^{2}_{k})$ is an orthonormal basis for $W(S^{2}_{k})$.
The main idea is to find a convenient basis for $W(S^{2}_{k})$ and work with
it, owing to the invariance of the Bergman kernel with respect to the choice
of a basis.

First, note that the size of $S_{k}\otimes S_{k}$ is clearly $k^{3}$ as its BK
is the product of that of $S_{k}$ with itself. Secondly, note that since
$F(\left(\begin{smallmatrix}\tau&z\\\ z&\tau^{\prime}\\\
\end{smallmatrix}\right))=F(\left(\begin{smallmatrix}\tau^{\prime}&z\\\
z&\tau\\\ \end{smallmatrix}\right))$ for all $F\in S^{2}_{k}$, one has
$W(F)\in(S_{k}\otimes S_{k})^{\mathrm{sym}}$, where $(S_{k}\otimes
S_{k})^{\mathrm{sym}}$ be the space of all symmetric tensors in $S_{k}\otimes
S_{k}$. Its dimension is $s_{k}(s_{k}+1)/2$, where we have put $s_{k}=\dim
S_{k}$. From [Fre13, 1.5 Hilfssatz, Folgerung, p.150] we know that when $k$ is
even, $W$ is surjective. Therefore, if we want to find the size of the space
of pullbacks $W(S^{2}_{k})$ for $S^{2}_{k}$ – we merely have to choose an
orthonormal basis of $(S_{k}\otimes S_{k})^{\mathrm{sym}}$. Since $2$ is
invertible for us, it is clear that $(S_{k}\otimes
S_{k})^{\mathrm{sym}}=(S_{k}\otimes S_{k})^{\mathrm{diag}}$ where
$(S_{k}\otimes S_{k})^{\mathrm{diag}}=\\{\sum_{i=1}^{s}c_{i}g_{i}\otimes
g_{i},\,g_{i}\in S_{k},\,s\geq 1\\}$. This will indicate how much of the
$L^{\infty}$ mass of $S^{2}_{k}$ is supported by the diagonal.

We will prove that its size is $k^{3}$. To see this, first put
$s(f,g)=f\otimes g+g\otimes f$. Further, let $\\{g_{j}\\}_{j}$ be the
normalized Hecke basis for $S_{k}$. We note that $s(g_{i},g_{j})$, $i\leq j$,
$1\leq j\leq s_{k}$ is a basis for $(S_{k}\otimes S_{k})^{\mathrm{sym}}$.
Further, the elements of this basis are pairwise orthogonal. To see this, note
that

(9.20) $\displaystyle\langle s(g_{1},g_{2}),s(g_{3},g_{4})\rangle=2\langle
g_{1},g_{3}\rangle\langle g_{2},g_{4}\rangle+2\langle
g_{1},g_{4}\rangle\langle g_{2},g_{3}\rangle.$

Suppose that (without loss of generality) $g_{1},g_{3}$ are unequal, hence
orthogonal. Then (9.20) reduces to $2\langle g_{1},g_{4}\rangle\langle
g_{2},g_{3}\rangle$ which can be non-zero if and only if
$g_{1}=g_{4},g_{2}=g_{3}$.

As before, the quantity which measures the size is the BK: we sum over the
modulus square of the elements in an orthonormal basis. Since the BK does not
depend on the choice of the orthonormal basis, we can write

(9.21) $\displaystyle\sup((S_{k}\otimes
S_{k})^{\mathrm{sym}})=\sum\limits_{i\leq
j}\frac{(vv^{\prime})^{k}|s(g_{i},g_{j})|^{2}}{\langle
s(g_{i},g_{j}),s(g_{i},g_{j})\rangle}.$

Let $P_{n}$ be the $n$-th Poincaré series in $S_{k}$. We can write

(9.22) $\displaystyle s(P_{m},P_{n})=\frac{\Gamma(k-1)^{2}}{(4\pi
m)^{k-1}(4\pi n)^{k-1}}\sum\limits_{i\leq
j}\frac{(\overline{a_{i}(m)}\overline{a_{j}(n)}+\overline{a_{i}(n)}\overline{a_{j}(m)})}{\langle
s(g_{i},g_{j}),s(g_{i},g_{j})\rangle}s(g_{i},g_{j});$

from which we get a Petersson formula:

(9.23) $\displaystyle\frac{\Gamma(k-1)^{2}}{(4\pi m)^{k-1}(4\pi
n)^{k-1}}\sum\limits_{i\leq
j}|a_{i}(m)a_{j}(n)+a_{i}(m)a_{j}(n)|^{2}=p_{m}(m)p_{n}(n).$

Now putting $m=n=1$ and following the method in section 7.2, the lower bound
$k^{3}$ is immediate.

For the upper bound, let $h_{i}$ denote the orthonormalized $g_{i}$. Then the
RHS of (9.21) can be bounded (up to absolute constants) by

(9.24) $\displaystyle\sum\limits_{i\leq j}(v^{k}|h_{i}(\tau)|^{2}\,v^{\prime
k}|h_{j}(\tau^{\prime})|^{2}+v^{k}|h_{j}(\tau)|^{2}\,v^{\prime
k}|h_{i}(\tau^{\prime})|^{2}),$

which is bounded by
$(\sum_{i}v^{k}|h_{i}(\tau)|^{2})(\sum_{j}v^{k}|h_{j}(\tau^{\prime})|^{2})\ll
k^{3}$ by [DK, Section 7.2].

Summarizing the above calculations, we can now state the following.

######  Theorem 9.7.

With the above notation and setting, if $k$ is even, $\sup(W(S^{2}_{k}))\asymp
k^{3}.$

## References

  * [AB15] M. Aggarwal and J. Brown  “Saito–Kurokawa lifts of square-free level”  In _Kyoto Journal of Mathematics_ 55.3 Kyoto University, 2015, pp. 641–662
  * [AD18] Pramath Anamby and Soumya Das  “Bounds for the Petersson norms of the pullbacks of Saito–Kurokawa lifts”  In _Journal of Number Theory_ 191 Elsevier, 2018, pp. 289–304
  * [Ara92] Tsuneo Arakawa  “Selberg zeta functions and Jacobi forms”  In _Zeta Functions in Geometry_ Mathematical Society of Japan, 1992, pp. 181–218
  * [Ara99] Tsuneo Arakawa  “Saito-Kurokawa lifting of odd number weights by the Duke-Imamoḡlu method. (Japanese) Automorphic forms and L-functions (Kyoto, 1999)”  In _Zeta Functions in Geometry_ Sūrikaisekikenkyūsho Kōkyūroku, 1999, pp. 187–199
  * [BH08] V. Blomer and G. Harcos  “Hybrid bounds for twisted $L$-functions”  In _J. für die reine und angewandte Mathematik_ 621 De Gruyter, 2008, pp. 53–79
  * [BK93] Siegfried Boecherer and Winfried Kohnen  “Estimates for Fourier coefficients of Siegel cusp forms”  In _Mathematische Annalen_ 297.1 Springer, 1993, pp. 499–517
  * [BKY13] Valentin Blomer, Rizwanur Khan and Matthew Young  “Distribution of mass of holomorphic cusp forms”  In _Duke Mathematical Journal_ 162.14 Duke University Press, 2013, pp. 2609–2644
  * [Blo15] Valentin Blomer  “On the size of Ikeda lifts”  In _Manuscripta Mathematica_ 148.3 Springer, 2015, pp. 341–349
  * [BM07] Ehud Moshe Baruch and Zhengyu Mao  “Central value of automorphic L-functions”  In _GAFA Geometric And Functional Analysis_ 17.2 Springer, 2007, pp. 333–384
  * [BM20] Thanasis Bouganis and Jolanta Marzec  “Algebraicity of special L-values attached to Siegel–Jacobi modular forms”  In _manuscripta mathematica_ Springer, 2020, pp. 1–44
  * [BP16] V. Blomer and A. Pohl  “The sup-norm problem on the Siegel modular space of rank two”  In _Amer. J. Math._ 138.4, 2016, pp. 999–1027
  * [Das10] Soumya Das  “Nonvanishing of Jacobi Poincaré series”  In _Journal of the Australian Mathematical Society_ 89.2 Cambridge University Press, 2010, pp. 165–179
  * [DK] Soumya Das and Hariram Krishna  “Bounds for Bergman kernel and the supnorm of holomorphic Siegel cusp forms” Preprint
  * [DK15] Soumya Das and Winfried Kohnen  “Nonvanishing of Koecher–Maass series attached to Siegel cusp forms”  In _Advances in Mathematics_ 281 Elsevier, 2015, pp. 624–669
  * [DS] Soumya Das and Jyoti Sengupta  “On the sup-norm of Maass lifts”  In _RMS Lecture Notes 2020_
  * [EZ85] Martin Eichler and Don Zagier  “The theory of Jacobi forms”  Springer, 1985
  * [Fre13] Eberhard Freitag  “Siegelsche modulfunktionen”  Springer-Verlag, 2013
  * [Got59] Erhard Gottschling  “Explizite bestimmung der randflächen des fundamentalbereiches der modulgruppe zweiten grades”  In _Mathematische Annalen_ 138.2 Springer, 1959, pp. 103–124
  * [GR14] Izrail Solomonovich Gradshteyn and Iosif Moiseevich Ryzhik  “Table of integrals, series, and products”  Academic press, 2014
  * [Ibu12] Tomoyoshi Ibukiyama  “Saito–Kurokawa liftings of level N and practical construction of Jacobi forms”  In _Kyoto journal of mathematics_ 52.1 Duke University Press, 2012, pp. 141–178
  * [Ich05] Atsushi Ichino  “Pullbacks of Saito–Kurokawa lifts”  In _Inventiones mathematicae_ 162.3 Springer, 2005, pp. 551–647
  * [IK04] Henryk Iwaniec and Emmanuel Kowalski  “Analytic Number Theory”  American Mathematical Soc., 2004
  * [ILS00] Henryk Iwaniec, Wenzhi Luo and Peter Sarnak  “Low lying zeros of families of $L$-functions”  In _Publications Mathématiques de l’IHÉS_ 91, 2000, pp. 55–131
  * [Jab17] Carine Jaber  “Algorithmic approaches to Siegel’s fundamental domain”  In _General Mathematics [math.GM]. Université Bourgogne Franche-Comté_ HAL open science, 2017, pp. 1–161
  * [Kli90] Helmut Klingen  “Introductory lectures on Siegel modular forms”  In _Cambridge Stud. Adv. Math._ 20 Cambridge Univ. Press, 1990
  * [Kli96] H. Klingen  “Automorphic forms and functions with respect to the Jacobi group”  In _Math. Ann._ 306, 1996, pp. 675–690
  * [Koh82] Winfried Kohnen  “Newforms of half-integral weight”  In _J. für die reine und angewandte Mathematik_ 333.lala lala, 1982, pp. 32–72
  * [Koh85] Winfried Kohnen  “Fourier coefficients of modular forms of half-integral weight”  In _Math. Ann._ 271 Springer, 1985, pp. 237–268
  * [KS89] Wilfried Kohnen and N-P Skoruppa  “A certain Dirichlet series attached to Siegel modular forms of degree two”  In _Inventiones mathematicae_ 95.3 Springer, 1989, pp. 541–558
  * [KST11] Emmanuel Kowalski, Abhishek Saha and Jacob Tsimerman  “A note on Fourier coefficients of Poincaré series”  In _Mathematika_ 57.1 London Mathematical Society, 2011, pp. 31–40
  * [LY14] Sheng-Chi Liu and Matthew P Young  “Growth and nonvanishing of restricted Siegel modular forms arising as Saito-Kurokawa lifts”  In _American Journal of Mathematics_ 136.1 Johns Hopkins University Press, 2014, pp. 165–201
  * [Maa79] Hans Maass  “Über eine Spezialschar von Modulformen zweiten Grades (II)”  In _Inventiones mathematicae_ 53.3 Springer, 1979, pp. 249–253
  * [MR00] M Manickam and B Ramakrishnan  “On Shimura, Shintani and Eichler-Zagier correspondences”  In _Transactions of the American Mathematical Society_ 352.6, 2000, pp. 2601–2617
  * [PY19] Ian Petrow and Matthew P Young  “A generalized cubic moment and the Petersson formula for newforms”  In _Mathematische Annalen_ 373.1 Springer, 2019, pp. 287–353
  * [Sko85] Nils-Peter Skoruppa  “Über den Zusammenhang zwischen Jacobiformen und Modulformen halbganzen Gewichts.”  In _Bonner Mathematische Schriften [Bonn Mathematical Publications],_ 159, 1985
  * [Ste16] Raphael S Steiner  “Uniform bounds on sup-norms of holomorphic forms of real weight”  In _International Journal of Number Theory_ 12.05 World Scientific, 2016, pp. 1163–1185
  * [SZ88] Nils-Peter Skoruppa and Don Zagier  “Jacobi forms and a certain space of modular forms.”  In _Inventiones mathematicae_ 94.1 Springer-Verlag, Berlin, 1988, pp. 113–146
  * [SZ89] N-P Skoruppa and Don Zagier  “A trace formula for Jacobi forms.”  In _Journal für die reine und angewandte Mathematik_ Walter de Gruyter, Berlin/New York Berlin, New York, 1989, pp. 168–198
  * [Wal80] J-L Waldspurger  “Correspondance de Shimura”  In _J. Math. Pure et Appl._ 59, 1980, pp. 1–133
  * [Wal81] J-L Waldspurger  “Sur les coefficients de Fourier des formes modulaires de poids demi-entier”  In _J. Math. Pures Appl._ 60, 1981, pp. 375–484
  * [Xia07] Honggang Xia  “On $L^{\infty}$ norms of holomorphic cusp forms”  In _Journal of Number Theory_ 124.2 Elsevier, 2007, pp. 325–327
  * [You17] Matthew P Young  “Weyl-type hybrid subconvexity bounds for twisted $L$-functions and Heegner points on shrinking sets”  In _Journal of the European Mathematical Society_ 19.5, 2017, pp. 1545–1576
  * [Zie89] Claus Ziegler  “Jacobi forms of higher degree”  In _Abhandlungen aus dem Mathematischen Seminar der Universität Hamburg_ 59.1, 1989, pp. 191–224  Springer
  * [ZWL10] Haigang Zhou, Tianze Wang and Hongwen Lu  “On dimension formulas for Jacobi forms”  In _Mathematische Zeitschrift_ 264.3 Springer, 2010, pp. 709–725


# Multi-layer Representation Learning for Robust OOD Image Classification

Aristotelis Ballas  0000-0003-1683-8433 Department of Informatics and
TelematicsHarokopio UniversityOmirou 9, TavrosAthensGreece<EMAIL_ADDRESS>and
Christos Diou  0000-0002-2461-1928 Department of Informatics and
TelematicsHarokopio UniversityOmirou 9, TavrosAthensGreece<EMAIL_ADDRESS>
(2022)

###### Abstract.

Convolutional Neural Networks have become the norm in image classification.
Nevertheless, their difficulty to maintain high accuracy across datasets has
become apparent in the past few years. In order to utilize such models in
real-world scenarios and applications, they must be able to provide
trustworthy predictions on unseen data. In this paper, we argue that
extracting features from a CNN’s intermediate layers can assist in the model’s
final prediction. Specifically, we adapt the Hypercolumns method to a
ResNet-18 and find a significant increase in the model’s accuracy, when
evaluating on the NICO dataset.

deep learning, domain generalization, out of distribution, image
classification

††journalyear: 2022††copyright: acmcopyright††conference: 12th Hellenic
Conference on Artificial Intelligence; September 7–9, 2022; Corfu,
Greece††booktitle: 12th Hellenic Conference on Artificial Intelligence (SETN
2022), September 7–9, 2022, Corfu, Greece††price: 15.00††doi:
10.1145/3549737.3549780††isbn: 978-1-4503-9597-7/22/09

##  1\. Introduction

In the past few years, Deep Learning has established itself in academia and
industry. In particular, Convolutional Neural Networks have dominated image
classification (Krizhevsky et al., 2012), achieving near-human, if not
superhuman (He et al., 2015), accuracy. Despite their outstanding results in
IID (independent and identically distributed) datasets, most models today fail
to generalize well on unseen or out of distribution (OOD) settings (Recht et
al., 2019), since they tend to incorporate statistical correlations present in
the training data (Arjovsky et al., 2020). In real-world scenarios, data
hardly ever originate from the same distribution, creating a need for
approaches that are able to distinguish between biases and trivial features
and make decisions based on invariant factors. This has been a long standing
issue in the Machine Learning community and, as a result, in 2011 the Domain
Generalization (DG) problem (Blanchard et al., 2011) was formally introduced.
In the DG setting, the data used to evaluate the trained model originate from
a different distribution than the training data. By making predictions on
unseen data distributions, we can appropriately assess the model’s ability to
generalize.

Figure 1. Visualization of the proposed model. The backbone of the
architecture is a vanilla ResNet-18. By extracting feature maps from
intermediate outputs of convolutional layers in the backbone model, our method
takes advantage of the CNN’s ability to detect edge and bar-like figures in
the early layers and combines them with the semantic features extracted from
layers further down the network. We argue that by incorporating outputs from
different intermediate levels of the network, we enable the model to
disentangle the invariant qualities of an object. We extract features from the
layers and residual connections marked in light blue.

Visualization of the proposed model.

In this work, we propose a method to tackle DG by adapting Hypercolumns
(Hariharan et al., 2015) to extract local attributes of an image in earlier
layers and semantics in layers further down the architecture. We hypothesize
that by incorporating features from across the network, a classifier can be
trained to ignore spurious correlations in the dataset and make predictions
based on invariant features. We evaluate our method by experimenting on NICO
(He et al., 2021), a dataset specifically designed for OOD image
classification and are able to achieve promising results. Furthermore, to
confirm our assumptions and intuition, we provide visual evidence of our
model’s ability to distinguish between spurious and invariant characteristics
present in an image.

###  1.1. Domain Generalization

In this section, we formally introduce the notations and definitions of DG.
Let $X$ be an input (feature) space and $Y$ an output (label) space. A domain
is defined as a joint distribution $\mathcal{P(X,Y)}$ $\sim$
$\mathcal{P_{XY}}$ on $\mathcal{X\times Y}$ .

In DG, the training and test distributions are OOD, in the sense that we are
given S source (training) domains and T target (test) domains, where
$\mathcal{P}^{i}_{\mathcal{XY}}$ $\neq$ $\mathcal{P}^{j}_{\mathcal{XY}}$, 1
$\leq$ i, j $\leq$ S, T. Given labeled source domains S, the goal is to learn
a model $\mathcal{F}$ , trained on data from S, which can adequately
generalize to an unseen domain T.

##  2\. Related Work

Domain Generalization is arguably one of the most challenging problems in
Machine Learning. To this end, a plethora of approaches have been proposed in
the past few years. The most closely related fields to DG are:

  * •

Domain Adaptation (Wang and Deng, 2018) and Transfer Learning (Zhuang et al.,
2021) methods , which are perhaps the most common, focus on boosting their
accuracy on unseen data by fine-tuning pretrained models on the target
domain(s).

  * •

Meta-Learning (Huisman et al., 2021), aims to learn-to-learn and select the
best method for solving the issue at hand.

  * •

Continual Learning (Mai et al., 2022) algorithms are used to overcome the
issue of catastrophic forgetting by remembering the knowledge acquired over
time and domains.

  * •

Zero-Shot Learning (Xian et al., 2017), like DG deals with unseen
distributions but in the label space.

With regard to learning stable or invariant features across domains, besides
the baseline CNN proposed in the original paper (He et al., 2021), several
other methods have been suggested. To address the issues of complex, non-
linear correlations between data in DG, the authors of (Zhang et al., 2021)
propose StableNet, a model which utilizes Random Fourier Features for sample
weighting. In (Li et al., 2021), the authors introduce the LIRR algorithm in
the Semi-Supervised Domain Adaptation setting, for learning invariant
representations and risks. Another approach is to use gradient-based semantic
augmentation (Bai et al., 2020) to improve the generalizability of a model.
Finally, the causal structure of the data can also be utilized while a model
is trained, as shown in (Sun et al., 2021).

##  3\. Methodology

###  3.1. Hypercolumns

Hypercolumns were first introduced in Neuroscience by Hubel and Wiesel (Hubel
and Wiesel, 1962), in order to describe a vertical set of V1 neurons that
behave similarly to optical stimuli of the retina. The authors of (Hariharan
et al., 2015) borrowed this term and applied its fundamental attributes to a
Convolutional Neural Network, in an attempt to leverage the different levels
of information passed on the network’s intermediate layers. Namely, a
Hypercolumn at a certain location is a stacked vector of the layer outputs of
the CNN’s units above said location. In order to classify pixels using
Hypercolumns, it is assumed that bounding boxes of the points of interest have
been provided from an object detection system. For each bounding box, a
${50\times 50}$ heatmap (locations) is predicted, which is projected onto the
initial image and then passed into a CNN. Selected intermediate outputs of the
CNN are then concatenated into a vector and each location is classified via
${1\times 1}$ convolutional and fully connected layers. In the same paper, the
Efficient Hypercolumn method is also described, where the ${1\times 1}$
convolutions are replaced by ${n\times n}$ convolutions and upsampling. In a
subsequent work (Hariharan et al., 2017), the authors were able to
significantly speed up their Hypercolumn pipeline by passing the whole image
through the CNN and cropping the bounding box locations afterwards.

Hypercolumns have been used for semantic segmentation (Hariharan et al.,
2017), object detection (Bochkovskiy et al., 2020), visual correspondence (Min
et al., 2019) and in some cases for abnormality detection in the Biomedical
domain (Toğaçar et al., 2021). However, all above implementations construct
their hypercolumns by concatenating the upsampled images into a hyper vector,
possibly without taking full advantage of the already extracted features of
the original image. Thus, we propose a novel implementation of the original
Hypercolumn method and adapt it for robust image classification in the DG
setting.

###  3.2. Adapting hypercolumns to robust image classification

Due to their convolutional nature, earlier layers in a CNN are prominent in
detecting edges and bars, but cannot distinguish between edges that belong to
a vehicle or an animal, per se. The extracted information is then passed down
the network and generalized in the final layer, where inference occurs. An
object or class consists of features which remain invariant (Arjovsky et al.,
2020) across domains. For example, a horse still has legs and a mane, whether
it is standing aside a person, lying in sand or galloping through snow.
Therefore, by disentangling an input image into distinct features, a causal
decision can be made based upon the features present in the image. Following
the above example, the presence of ‘legs’ in an image lead us to believe that
an animal is most likely depicted. The presence of a ‘mane’, ‘long tail’ and
‘oval-shaped’ hoove make us confident that the animal is a horse. We argue
that by taking advantage of the early and intermediate features of a CNN, we
can ‘push’ a model to learn these invariant features (i.e edges and bars which
correspond to parts of the depicted objects). For our model, we follow the
original Hypercolumns implementation and select the outputs from intermediate
conv layers, pass them through a ${1\times 1}$ conv layer and then upsample
the outputs via bilinear interpolation. However, before concatenating the
upsampled images to a hyper vector, we pass them through a ${56\times 56}$
MaxPool2D layer, with ${26\times 26}$ strides, in an attempt to capture the
features of the depicted class. After the pooling layer, the outputs are
concatenated into a hyper vector and passed through a classification head,
which consists of a fully connected layer followed by a softmax activation.
Our model architecture is depicted in Figure 1.

##  4\. Experiments

###  4.1. Datasets

In our experiments we adopt the NICO (He et al., 2021) dataset. The NICO
dataset was created for OOD image classification and is therefore a good
starting point for the evaluation of the robustness of classification
algorithms. NICO contains 2 super-classes of Animal and Vehicle. The Animal
superclass contains 10 classes and the Vehicle contains 9. Each class contains
9 or 10 contexts, which try to simulate real world scenarios, such as
‘airplane aside mountain’ and ‘airplane on grass’. The total images in the
dataset are 25.000.

###  4.2. Experimental Setup

To evaluate our model we follow the _leave-one-domain-out_ protocol as
described in (Li et al., 2017), where in our case a domain is a context of a
class. In order to demonstrate the robustness of our model, during training we
select to hold out 3, 5 and 7 contexts from each class. For the backbone of
our model, we select a ResNet-18, pre-trained on ImageNet. The selected
intermediate layers include all residual half block layers (i.e., those
leading to reduction of the output size), as well as selected convolutional
layers. All later layers of the network are included. We train the model with
SGD for 30 epochs and with a batch size of 32 images. The learning rate is
initially set at 0.001 and decays with a rate of of 0.1 at epoch 24. For a
baseline, we used a vanilla pre-trained ResNet-18 with the same
hyperparameters as above. Both models were implemented with PyTorch on one
NVIDIA RTX A5000 GPU.

Table 1. Top-1% Accuracy Results on the NICO Dataset when leaving out N contexts. If an image class has C contexts, we create a training split with the $\mathbf{C-N}$ contexts and evaluate our model on the remaining N. The presented results are averaged over 3 runs. Model | N=3 | N=5 | N=7
---|---|---|---
ResNet-18 | 79.6 | 78.1 | 78.9
Our Model | 82.8 | 81.3 | 83.0
Figure 2. Visualization of saliency maps, produced by the baseline vanilla
ResNet-18 model and our method, from images in the NICO dataset. The brighter
the pixel, the more it contributes to the model’s prediction. To a certain
degree, our approach disregards the pixels corresponding to the contexts and
spurious correlations (i.e water, snow, grass and road) in each image, and
focuses on the depicted object.

###  4.3. Results

Table 1 depicts the averaged results on NICO after 3 runs. We can observe that
our model outperforms the baseline by approximately 3.2% when 3 contexts are
left out, 3.2% in the case where 5 contexts are left out and 2.4% when we
leave out 7.

To validate our initial assumptions, we also visualize our model’s prediction
with saliency maps. To be more specific, we adopted the _Image-Specific Class
Saliency_ method, as proposed in (Simonyan et al., 2014). By computing and
visualizing the gradient of the loss function for the predicted class, with
respect to the input pixels, one is able to produce a map of the pixels
affecting the model’s prediction. As shown in Fig. 2, the brightness in the
saliency maps indicate the pixels which the model pays more ‘attention’ to.
Due to spurious correlations present in the training data, the vanilla
ResNet-18 tends to make assumptions based on unimportant features of the image
(e.g. water, snow, grass and road - indicated by the bright pixels around the
object in the saliency maps), while our method seems to infer based on
features of the object itself and to some extent overlook the context
features.

##  5\. Conclusion

In this paper we attempt to tackle the Domain Generalization problem by
adopting the Hypercolumns method and adapting it for robust image
classification. We argue that by utilizing the extracted features from a CNN’s
intermediate layers, the model can be forced to focus on the invariant
features in an image. This claim is supported by the results of our
experiments on NICO, a dataset dominated by spurious correlations, where we
demonstrate our model’s ability to perform well on unseen data. Through visual
examples, we show that our method is capable of emphasizing on the causal
characteristics of an object and not on the inconsequential features of the
input image. As future work, we aim to advance our method’s extraction
mechanism and conduct further experiments on additional datasets.

###### Acknowledgements.

The work leading to these results has received funding from the European
Union’s Horizon 2020 research and innovation programme under Grant Agreement
No. 965231 project REBECCA (REsearch on BrEast Cancer induced chronic
conditions supported by Causal Analysis of multi-source data).

## References

  * (1)
  * Arjovsky et al. (2020) Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. 2020.  Invariant Risk Minimization.  _arXiv:1907.02893 [cs, stat]_ (2020).
  * Bai et al. (2020) Haoyue Bai, Rui Sun, Lanqing Hong, Fengwei Zhou, Nanyang Ye, Han-Jia Ye, S. H. Gary Chan, and Zhenguo Li. 2020\.  DecAug: Out-of-Distribution Generalization via Decomposed Feature Representation and Semantic Augmentation.  arXiv:2012.09382 [cs.LG]
  * Blanchard et al. (2011) Gilles Blanchard, Gyemin Lee, and Clayton Scott. 2011\.  Generalizing from Several Related Classification Tasks to a New Unlabeled Sample. In _Advances in Neural Information Processing Systems_ , J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Q. Weinberger (Eds.), Vol. 24. Curran Associates, Inc.  https://proceedings.neurips.cc/paper/2011/file/b571ecea16a9824023ee1af16897a582-Paper.pdf
  * Bochkovskiy et al. (2020) Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. 2020.  YOLOv4: Optimal Speed and Accuracy of Object Detection.  _arXiv:2004.10934 [cs, eess]_ (2020).
  * Hariharan et al. (2015) Bharath Hariharan, Pablo Arbelaez, Ross Girshick, and Jitendra Malik. 2015.  Hypercolumns for object segmentation and fine-grained localization. In _2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, Boston, MA, USA, 447–456.  https://doi.org/10.1109/CVPR.2015.7298642
  * Hariharan et al. (2017) Bharath Hariharan, Pablo Arbeláez, Ross Girshick, and Jitendra Malik. 2017.  Object Instance Segmentation and Fine-Grained Localization Using Hypercolumns.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ (2017).
  * He et al. (2015) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015\.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification.  _CoRR_ (2015).
  * He et al. (2021) Yue He, Zheyan Shen, and Peng Cui. 2021.  Towards non-iid image classification: A dataset and baselines.  _Pattern Recognition_ 110 (2021), 107383.
  * Hubel and Wiesel (1962) D. H. Hubel and T. N. Wiesel. 1962.  Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex.  _The Journal of Physiology_ 160, 1 (Jan. 1962), 106–154.2.  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523/
  * Huisman et al. (2021) Mike Huisman, Jan N. van Rijn, and Aske Plaat. 2021.  A survey of deep meta-learning.  _Artif Intell Rev_ 54, 6 (Aug. 2021), 4483–4541.  https://doi.org/10.1007/s10462-021-10004-4
  * Krizhevsky et al. (2012) Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012\.  ImageNet Classification with Deep Convolutional Neural Networks. In _Advances in Neural Information Processing Systems_ , F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Eds.), Vol. 25. Curran Associates, Inc.  https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
  * Li et al. (2021) Bo Li, Yezhen Wang, Shanghang Zhang, Dongsheng Li, Kurt Keutzer, Trevor Darrell, and Han Zhao. 2021.  Learning Invariant Representations and Risks for Semi-Supervised Domain Adaptation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 1104–1113.
  * Li et al. (2017) Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. 2017.  Deeper, Broader and Artier Domain Generalization. In _Proceedings of the IEEE International Conference on Computer Vision (ICCV)_.
  * Mai et al. (2022) Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. 2022\.  Online continual learning in image classification: An empirical survey.  _Neurocomputing_ 469 (Jan. 2022), 28–51.  https://doi.org/10.1016/j.neucom.2021.10.021
  * Min et al. (2019) Juhong Min, Jongmin Lee, Jean Ponce, and Minsu Cho. 2019\.  Hyperpixel Flow: Semantic Correspondence With Multi-Layer Neural Features. In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_.
  * Recht et al. (2019) Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. 2019.  Do ImageNet Classifiers Generalize to ImageNet?. In _Proceedings of the 36th International Conference on Machine Learning_ _(Proceedings of Machine Learning Research)_ , Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR.
  * Simonyan et al. (2014) Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014\.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps.  arXiv:1312.6034 [cs.CV]
  * Sun et al. (2021) Xinwei Sun, Botong Wu, Xiangyu Zheng, Chang Liu, Wei Chen, Tao Qin, and Tie yan Liu. 2021.  Latent Causal Invariant Model.  arXiv:2011.02203 [cs.LG]
  * Toğaçar et al. (2021) Mesut Toğaçar, Zafer Cömert, and Burhan Ergen. 2021\.  Enhancing of dataset using DeepDream, fuzzy color image enhancement and hypercolumn techniques to detection of the Alzheimer’s disease stages by deep learning model.  _Neural Computing and Applications_ (2021).
  * Wang and Deng (2018) Mei Wang and Weihong Deng. 2018.  Deep Visual Domain Adaptation: A Survey.  _Neurocomputing_ 312 (Oct. 2018), 135–153.  https://doi.org/10.1016/j.neucom.2018.05.083 arXiv: 1802.03601.
  * Xian et al. (2017) Yongqin Xian, Bernt Schiele, and Zeynep Akata. 2017\.  Zero-Shot Learning - the Good, the Bad and the Ugly. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_.
  * Zhang et al. (2021) Xingxuan Zhang, Peng Cui, Renzhe Xu, Linjun Zhou, Yue He, and Zheyan Shen. 2021\.  Deep stable learning for out-of-distribution generalization. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 5372–5382.
  * Zhuang et al. (2021) Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. 2021\.  A Comprehensive Survey on Transfer Learning.  _Proc. IEEE_ 109, 1 (Jan. 2021), 43–76.  https://doi.org/10.1109/JPROC.2020.3004555