# On the Pseudonullity of Fine Selmer groups over function fields.

Sohan Ghosh  Harish Chandra Research Institute, Prayagraj 211019, India
<EMAIL_ADDRESS>
###### Abstract.

The $p^{\infty}$-fine Selmer group of an elliptic curve $E$ over a global
field is a subgroup of the classical $p^{\infty}$-Selmer group. Coates and
Sujatha discovered that the structure of the fine Selmer group of $E$ over
certain $p$-adic Lie extensions of a number field is intricately related to
some deep questions in classical Iwasawa theory. Inspired by a conjecture of
Greenberg, [CS] made prediction about the structure of the fine Selmer group
over certain $p$-adic Lie extensions of a number field, which they called
Conjecture B. In this article, we discuss some new cases of Conjecture B and
its analogues over some $p$-adic Lie extensions of function fields of
characteristic $p$.

††footnotetext: AMS Subject Classification: 11R23, 11G05, 11S25, 11R60
Keywords and phrases: Iwasawa theory, fine Selmer groups, function fields.

Let $p$ be an odd prime and let $E$ be an elliptic curve defined over a number
field $F$. The $p^{\infty}$-fine Selmer group is a subgroup of the classical
$p^{\infty}$-Selmer group obtained by imposing strict local conditions on the
primes dividing $p$.

The fine Selmer group has been studied in Iwasawa theory throughout, under
different names, like${\mbox{\textcyr{Sh}}}_{1},\text{Sel}_{0}$ by Billot,
Greenberg, Kurihara [Ku], Perrin-Riou [Pr] and others over the cyclotomic
$\mathbb{Z}_{p}$ extensions of number fields. In 2005, Coates-Sujatha [CS]
found an important connection between the structure of the $p^{\infty}$-fine
Selmer group over $F_{\mathrm{cyc}}$, $R(E/F_{\mathrm{cyc}})$, and Iwasawa’s
classical $\mu=0$ conjecture about the growth of the $p$-part of the ideal
class group in the cyclotomic tower for $F_{\mathrm{cyc}}$. In particular,
they showed that

######  Theorem 0.1.

[CS] Let $E/F$ be an elliptic curve such that $F(E_{p^{\infty}})/F$ is a
pro-$p$ extension. Then $R(E/F_{\mathrm{cyc}})^{\vee}$ is a finitely generated
$\mathbb{Z}_{p}$ module if and only if Iwasawa’s $\mu=0$ conjecture holds for
$F_{\mathrm{cyc}}$.

Motivated by this, [CS] formulated the following conjecture:

######  Conjecture A.

For all elliptic curves $E$ over $F$, $R(E/F_{\mathrm{cyc}})^{\vee}$ is a
finitely generated $\mathbb{Z}_{p}$ module.

Following Conjecture A, the fine Selmer group over the cyclotomic
$\mathbb{Z}_{p}$ extensions of a number field have been studied by various
authors including [Wu, Lim2, LV, JS] and others. Coates-Sujatha also studied
the structure of the fine Selmer over more general $p$-adic Lie extensions of
a number field in the framework of non-commutative Iwasawa theory [CFKSV].
Motivated by a conjecture of Greenberg [Gr, Conjecture 3.5], [CS] made the
following prediction regarding the structure of the fine Selmer group over
certain $p$-adic Lie extensions of a number field:

######  Conjecture B.

[CS] Assume that the Conjecture A holds for $E$ over $F_{\mathrm{cyc}}$. Let
$F_{\infty}$ be an admissible $p$-adic Lie extension of $F$ such that
$G=\mathrm{Gal}(F_{\infty}/F)$ has dimension at least 2 as a p-adic Lie group.
Then $R(E/F_{\infty})^{\vee}$ is a pseudonull $\mathbb{Z}_{p}[[G]]$-module.

Following Conjecture B, various authors[LP, Jh, Sh, Oc] have investigated the
properties of the fine Selmer group over $p$-adic Lie extensions of a number
field.

In [GJS], the authors initiated the study of the fine Selmer groups over
function fields (both $\ell\neq p$ and $\ell=p$ cases). More precisely, they
investigated analogues of Conjecture A and Conjecture B over function fields.
While the answer to an analogue of Conjecture A was affirmative, it was shown
that the analogue of Conjecture B fails in the $\ell\neq p$ case. For the
$\ell=p$ case, evidences were given in support of Conjecture B [GJS,
Proposition 3.8, Theorem 3.14]. More precisely, in [GJS, Theorem 3.14], the
authors show that under suitable hypotheses, Conjecture B holds for certain
$\mathbb{Z}_{p}^{2}$ extensions of a function field. In another main result of
this article, we generalise [GJS, Theorem 3.14] to give a new class of
examples over two different certain $\mathbb{Z}_{p}^{d}$ extensions of
$K=\mathbb{F}(t)$, where Conjecture $B$ holds. More precisely, we prove the
following results:

######  Theorem 0.2 (Theorem 1.7).

Let $K_{\infty}$ be the arithmetic $\mathbb{Z}_{p}$ extension of $K$ and
$K_{d}^{\mathfrak{P}}$ be the geometric $\mathbb{Z}_{p}^{d}$ extension of $K$,
where $d\geq 1$ (see §1). Let $F_{\infty}=K_{\infty}K_{d}^{\mathfrak{P}}$. Let
$\nu_{ram}$ be the unique prime of $K$ that ramifies in $F_{\infty}$.

Assume $E/K$ be an ordinary elliptic curve that has good reduction outside
$\nu_{ram}$. Then $R^{S}(E/F_{\infty})^{\vee}$ is a pseudonull
$\mathbb{Z}_{p}[[G]]$ module, where
$G=\mathrm{Gal}(F_{\infty}/F)\cong\mathbb{Z}_{p}^{d+1}$ and $S$ is any set of
primes of $K$ containing the prime $\nu_{ram}$.

######  Proposition 0.3 (Theorem 1.8).

Let $K_{d}^{\mathfrak{P}}$ be the geometric $\mathbb{Z}_{p}^{d}$ extension of
$K$, (see §1). Let $v_{r}$ be the unique prime of $K$ that ramifies in
$K_{d}^{\mathfrak{P}}$ and set $S^{\prime}=\\{v_{r}\\}$. Suppose that $E/K$ be
an ordinary elliptic curve that has good reduction outside the set
$S^{\prime}$.

Then $R^{S}(E/K_{d}^{\mathfrak{P}})^{\vee}$ is a pseudonull
$\mathbb{Z}_{p}[[\mathcal{G}]]$ module, where
$\mathcal{G}=\mathrm{Gal}(K_{d}^{\mathfrak{P}}/F)$ and $S$ is any set of
primes of $K$ with $S^{\prime}\subset S$.

###### Remark 0.4

  1. (1)

In §1.1, we will see that there are infinitely many $\mathbb{Z}_{p}^{d}$
extensions of function fields unlike the number field case.

  2. (2)

The condition that the elliptic curve $E$ has good reduction at all places
outside $\nu_{ram}$ is trivially satisfied by all the elliptic curves defined
over the finite field $\mathbb{F}$.

  3. (3)

Other than $\nu_{ram}$, every prime of $K$ splits infinitely over
$F_{\infty}$. Infact, this is the case with every $\mathbb{Z}_{p}^{d}$
extension that are constructed from Carlitz module, where $d\geq 1$, see [Ro,
Theorem 12.10, Theorem 12.14 and discussion on Page 213]). Therefore, using
techniques as in the proof of [GJS, Theorem 3.14], the Theorem 1.7 cannot be
generalised to include elliptic curves with bad reduction outside the ramified
prime $v_{r}$.

∎

##  1\. Fine Selmer Groups over function fields

Fix an integer prime $p$. Let $K$ be a function field in one variable over a
finite field $\mathbb{F}$ of characteristic $p$. Let $E$ be an elliptic curve
defined over $K$. Consider an open dense subset $U$ of
$C_{K}=\mathbb{P}^{1}_{\mathbb{F}}$ such that $E/K$ has good reductions at
every place of $U$. Let $\mathcal{E}$ denote the Néron model of $E$ over
$C_{K}$. Let $\Sigma_{K}$ be the set of all the primes of $K$ and $S$ denote
the set of primes of $K$ outside $U$ i.e., the places of $C_{K}\setminus U$.
Therefore, $S$ is a finite set of primes of $K$ that contains the set of all
primes of bad reduction of $E/K$. Let $K_{S}$ denote the maximal algebraic
extension of $K$ unramified outside $S$. Consider a finite extension of $K$,
$L\subset K_{S}$. Let $v$ be any prime of $K$ and $w$ denote a prime of $L$.
Define

$\displaystyle J_{v}^{1}(E/L):=\underset{w\mid
v}{\prod}\frac{H^{1}_{fl}(L_{w},E_{p^{\infty}})}{im(\kappa_{w})}\ \text{and
}K_{v}^{1}(E/L):=\underset{w\mid v}{\prod}H^{1}_{fl}(L_{w},E_{p^{\infty}}).$
(1.1)

Here $H^{i}_{\text{fl}}(-,-)$ denotes the flat cohomology [M2, Chapters II,
III] and
$\kappa_{w}:E(L_{w})\otimes{\mbox{$\mathbb{Q}$}_{p}}/{\mathbb{Z}_{p}}\hookrightarrow
H^{1}_{fl}(L_{w},E_{p^{\infty}})$ is induced by the Kummer map.

######  Definition 1.1.

[KT, Prop. 2.4] Let $\Sigma_{K},S\text{ and }K\subset L\subset K_{S}$ be as
above. Then the Selmer group $S(E/L)$ is defined as:

$S(E/L):=\ker\big{(}H^{1}_{\text{fl}}(L,{E}_{p^{\infty}})\longrightarrow\underset{v\in\Sigma_{K}}{\bigoplus}J_{v}^{1}(E/L)\big{)}.$
(1.2)

We define the $S$-fine Selmer group as:

$\displaystyle
R^{S}(E/L):={}\ker\big{(}H^{1}_{\text{fl}}(L,E_{p^{\infty}})\longrightarrow\underset{v\in
S}{\bigoplus}K_{v}^{1}(E/L)\underset{v\in\Sigma_{K}\setminus
S}{\bigoplus}J_{v}^{1}(E/L)\big{)}$ (1.3)
$\displaystyle\cong\ker\big{(}S(E/L)\longrightarrow\underset{w\mid v,v\in
S}{\bigoplus}E(L_{w})\otimes\mbox{$\mathbb{Q}$}_{p}/\mathbb{Z}_{p}\big{)}.$

For an infinite algebraic extension $\mathcal{L}$ of $K$, the above
definitions extend, as usual, by taking inductive limit over finite
subextensions of $\mathcal{L}$ over $K$.

###  1.1. $\mathbb{Z}_{p}^{d}$ extensions

Let $\mathbb{F}^{(p)}$ be the unique subfield of $\bar{\mathbb{F}}$ such that
$\mathrm{Gal}(\mathbb{F}^{(p)}/\mathbb{F})\cong\mathbb{Z}_{p}$. Set
$K_{\infty}:=K\mathbb{F}^{(p)}$. Note that $K_{\infty}/K$ is unramified
everywhere.

The second type of $\mathbb{Z}_{p}$ extensions that bears a close analogy with
the cyclotomic $\mathbb{Z}_{p}$ extension of a number field, is the
“cyclotomic extension at the prime ideal $\mathfrak{P}$”. Consider a field
extension $F$ of $K$. Then $F$ can be thought of as a $\mathbb{F}[t]$ module,
where the action of $\mathbb{F}[t]$ is given by the Carlitz polynomials.
Choose $\mathfrak{P}$ to be a prime of $\mathbb{F}[t]$. For $n>0$, let

$\Lambda_{\mathfrak{P}^{n}}:=\\{\lambda\in\overline{\mathbb{F}(t)}|[\mathfrak{P}^{n}](\lambda)=0\\}.$

Here $K(\Lambda_{\mathfrak{P}^{n}})/K$ is Galois with
$\mathrm{Gal}(K(\Lambda_{\mathfrak{P}^{n}})/K)\cong(\mathbb{F}[t]/\mathfrak{P}^{n})^{\times}$.
Put $\widetilde{K}:=\underset{n\geq 1}{\bigcup}K(\Lambda_{\mathfrak{P}^{n}})$,
then
$\mathrm{Gal}(\widetilde{K}/K)\cong\mathbb{Z}_{p}^{\mathbb{N}}\times(\mathbb{F}[t]/\mathfrak{P})^{\times}$.
The $\mathbb{Z}_{p}^{d}$ extension obtained from $\widetilde{K}$, for $d\geq
1$, denoted by $K_{d}^{\mathfrak{P}}$, is ramified only at one prime and it is
totally ramified at that prime [Ro, Proposition 12.7]. Set $K^{\mathfrak{P}}$
as the $\mathbb{Z}_{p}$ extension $K_{1}^{\mathfrak{P}}$ .

Let $F_{\infty}=K_{d}^{\mathfrak{P}}K_{\infty}$. Therefore, $F_{\infty}$ is a
$\mathbb{Z}_{p}^{d+1}$ extension of $K$, that is ramified at exactly one
prime, which splits finitely over $F_{\infty}$.

###  1.2. Pseudonullity

Let $Cl(L)$ denote the divisor class group of $L$, where $L$ is an algebraic
extension of $K$.

From now on, we will assume that $E/K$ is an ordinary elliptic curve..

######  Definition 1.2.

Let $E,K,S$ be as before and $L\subset K_{S}$ be a finite extension of $K$.
For $\bullet\in\\{\mu_{p^{\infty}},\ \mbox{$\mathbb{Q}$}_{p}/\mathbb{Z}_{p},\
\mu_{p},\ \mathbb{Z}/p\mathbb{Z}\\}$, set $K_{v}^{1}(\bullet/L):$=
$\underset{w\mid v}{\bigoplus}H^{1}_{fl}(L_{w},\bullet)$ and
$J_{v}^{1}(\bullet/L):=\underset{w\mid
v}{\bigoplus}H^{1}_{fl}(L_{w},\bullet)/H^{1}_{fl}(O_{w},\bullet)$. We define
the groups $S^{\prime}(\bullet/L)$ and $R^{S}(\bullet/L)$ as follows:

$S^{\prime}(\bullet/L):=\ker(H^{1}_{\text{fl}}(L,\bullet)\longrightarrow\underset{v\in\Sigma_{K}}{\bigoplus}J_{v}^{1}(\bullet/L)).$
(1.4)
$R^{S}(\bullet/L):=\ker(H^{1}_{\text{fl}}(L,\bullet)\longrightarrow\underset{v\in
S}{\bigoplus}K_{v}^{1}(\bullet/L))\underset{v\in\Sigma_{K}\setminus
S}{\bigoplus}J_{v}^{1}(\bullet/L)),$ (1.5)

Similarly, the group $R^{S}(E[p]/L)$ of $E[p]$ over $L$ is defined by:

$R^{S}(E[p]/L):=\ker(H^{1}_{\text{fl}}(L,E[p])\longrightarrow\underset{v\in
S}{\bigoplus}K_{v}^{1}(E[p]/L))\underset{v\in\Sigma_{K}\setminus
S}{\bigoplus}J_{v}^{1}(E[p]/L)),$ (1.6)

where $K_{v}^{1}(E[p]/L):=\underset{w\mid v}{\bigoplus}H^{1}_{fl}(L_{w},E[p])$
and $J_{v}^{1}(E[p]/L):=\underset{w\mid
v}{\bigoplus}H^{1}_{fl}(L_{w},E[p])/H^{1}_{fl}(O_{w},\mathcal{E}[p])$.

Before proving our main theorems, let us discuss some preliminary results that
are needed to prove Theorem 1.7.

######  Theorem 1.3.

Let $E/K$ be an ordinary elliptic curve and $S$ be as defined in §1. Then,
$R^{S}(E/K_{\infty})^{\vee}$ is a finitely generated $\mathbb{Z}_{p}$ module.
∎

The proof of the above theorem can be found in [GJS, Theorem 3.7]. Here, we
give a brief sketch of it.

###### Proof.

It is easy to see that the kernel and the cokernel of the natural map
$R^{S}(E[p]/K_{\infty})\rightarrow R^{S}(E/K_{\infty})[p]$ are finite. Hence,
we obtain that $\mu(R^{S}(E/K_{\infty})^{\vee})=0$ if and only if
$R^{S}(E[p]/K_{\infty})$ is finite.

Now, let $K_{\infty}^{p}:=K\bar{\mathbb{F}}_{p}$. We claim that
$R^{S}(E[p]/K_{\infty}^{p})$ is finite. Consider the connected-$\acute{e}$tale
sequence (see, for example, [LLSTT, §3.2])

$0\longrightarrow E[p]^{0}\longrightarrow
E[p]\longrightarrow\pi_{0}(E[p])\longrightarrow 0$ (1.7)

where $E[p]^{0}$ and $\pi_{0}(E[p])$ are Cartier dual to each other. As $E/K$
is ordinary, we know that $\pi_{0}(E[p])\cong\mathbb{Z}/p\mathbb{Z}$ and
$E[p]^{0}\cong\mu_{p}$, where $\mu_{p}$ is the Cartier dual to
$\mathbb{Z}/p\mathbb{Z}$.

Now, using the definitions in (1.6) and equation (1.7), we get the following
commutative diagram of complexes, which is not necessarily exact:

${0}$${R^{S}\big{(}(\mathbb{Z}/p\mathbb{Z})/K_{\infty}^{p}\big{)}}$${H^{1}_{fl}(K_{\infty}^{p},\mathbb{Z}/pZ)}$${\prod\limits_{\begin{subarray}{c}w\mid
v\\\ v\in
S\end{subarray}}H^{1}_{\text{fl}}(K_{\infty,w}^{p},\mathbb{Z}/p\mathbb{Z})\prod\limits_{\begin{subarray}{c}w\mid
v\\\ v\notin
S\end{subarray}}\frac{H^{1}_{\text{fl}}(K_{\infty,w}^{p},\mathbb{Z}/p\mathbb{Z})}{H^{1}_{\text{fl}}(O_{\infty,w},\mathbb{Z}/p\mathbb{Z})}}$${0}$${R^{S}(E[p]/K_{\infty}^{p})}$${H^{1}_{fl}(K_{\infty}^{p},E[p])}$${\prod\limits_{\begin{subarray}{c}w\mid
v\\\ v\in
S\end{subarray}}H^{1}_{\text{fl}}(K_{\infty,w}^{p},E[p])\prod\limits_{\begin{subarray}{c}w\mid
v\\\ v\notin
S\end{subarray}}\frac{H^{1}_{\text{fl}}(K_{\infty,w}^{p},E[p])}{H^{1}_{\text{fl}}(O_{\infty,w},\mathcal{E}[p])}}$${0}$${R^{S}(\mu_{p}/K_{\infty}^{p})}$${H^{1}_{\text{fl}}(K_{\infty}^{p},\mu_{p})}$${\prod\limits_{\begin{subarray}{c}w\mid
v\\\ v\in
S\end{subarray}}H^{1}_{\text{fl}}(K_{\infty,w}^{p},\mu_{p})\prod\limits_{\begin{subarray}{c}w\mid
v\\\ v\in
S\end{subarray}}\frac{H^{1}_{\text{fl}}(K_{\infty,w}^{p},\mu_{p})}{H^{1}_{\text{fl}}(O_{\infty,w},\mu_{p})}}$$\scriptstyle{\alpha}$$\scriptstyle{\beta}$$\scriptstyle{\gamma=\underset{w}{\prod}\gamma_{w}}$
(1.8)

Using the proof of [OT, Lemma 3.4] and the fact that $Cl(K_{\infty}^{p})[p]$,
the $p$-part of the divisor class group, is finite [Ro, Proposition 11.16], we
deduce that $R^{S}(\mu_{p},K_{\infty}^{p})$ and
$R^{S}(\mathbb{Z}/p\mathbb{Z},K_{\infty}^{p})$ are finite. From [M1, §III.7],
we know that for $w\mid v,v\notin S$, $\ker(\gamma_{w})=0$, which implies that
$\ker(\gamma)$ is finite. By applying a snake lemma to the lower complex, we
deduce that $R^{S}(E[p]/K_{\infty}^{p})$ is finite. As
$G=\mathrm{Gal}(K_{\infty}^{p}/K_{\infty})\cong\underset{l\neq
p}{\prod}\mathbb{Z}_{l}$, we get $R^{S}(E[p]/K_{\infty}^{p})^{G}\cong
R^{S}(E[p]/K_{\infty})$. This completes our proof. ∎

######  Proposition 1.4.

Let $F_{\infty}=K_{\infty}K^{\mathfrak{P}}_{d}$ be a $\mathbb{Z}_{p}^{d+1}$
extension, where $d\geq 1$. Let
$\mathcal{H}=\mathrm{Gal}(F_{\infty}/K_{\infty}K^{\mathfrak{P}}_{d-1})\cong\mathbb{Z}_{p}^{d}$.
Then, $Cl(F_{\infty})[p^{\infty}]^{\vee}$ is a finitely generated torsion
$\mathbb{Z}_{p}[[\mathcal{H}]]$ module.

###### Proof.

By a result of [BH, Page-229], it suffices to show that
$(Cl(F_{\infty})[p^{\infty}])^{\mathcal{H}}$ is finite.

Using the commutative diagram in [OT, Page-38], we obtain the following
commutative diagram:

${0}$${\left(\underset{m}{\varinjlim}\
\dfrac{(\mathbb{F}^{(p)})^{\times}}{((\mathbb{F}^{(p)})^{\times})^{p^{m}}}\right)^{\mathcal{H}}}$${S^{\prime}(\mu_{p^{\infty}}/F_{\infty})^{\mathcal{H}}}$${(Cl(F_{\infty})[p^{\infty}])^{\mathcal{H}}}$${0}$${\underset{m}{\varinjlim}\
\dfrac{\mathbb{F}^{\times}}{(\mathbb{F}^{\times})^{p^{m}}}}$${S^{\prime}(\mu_{p^{\infty}}/K^{\mathfrak{P}})}$${Cl(K^{\mathfrak{P}})[p^{\infty}]}$${0}$$\scriptstyle{\beta}$$\scriptstyle{\alpha}$$\scriptstyle{\gamma}$
(1.9)

Note that
$\dfrac{(\mathbb{F}^{(p)})^{\times}}{((\mathbb{F}^{(p)})^{\times})^{p^{m}}}=0$
for all $m$. Therefore, $\beta$ is an isomorphism. Further, by [Aa,
Proposition 2], $Cl(K^{\mathfrak{P}})[p^{\infty}]$ is finite. We claim that
$\text{coker}(\alpha)$ is finite. Then the lemma follows by using a snake
lemma in diagram 1.9. We now establish the claim. Consider the commutative
diagram:

${0}$${S^{\prime}(\mu_{p^{\infty}}/F_{\infty})^{\mathcal{H}}}$${H^{1}_{\text{fl}}(F_{\infty},\mu_{p^{\infty}})^{\mathcal{H}}}$${\left(\underset{w\mid
v,v\in\Sigma_{K}}{\oplus}\frac{H^{1}_{\text{fl}}(F_{\infty,w},\mu_{p^{\infty}})}{H^{1}_{\text{fl}}(O_{F_{\infty,w}},\mu_{p^{\infty}})}\right)^{\mathcal{H}}}$${0}$${S^{\prime}(\mu_{p^{\infty}}/K^{\mathfrak{P}})}$${H^{1}_{\text{fl}}(K^{\mathfrak{P}},\mu_{p^{\infty}})}$${\left(\underset{w\mid
v,v\in\Sigma_{K}}{\oplus}\frac{H^{1}_{\text{fl}}(K^{\mathfrak{P}}_{\infty,w},\mu_{p^{\infty}})}{H^{1}_{\text{fl}}(O_{K^{\mathfrak{P}}_{\infty,w}},\mu_{p^{\infty}})}\right)}$$\scriptstyle{\alpha}$$\scriptstyle{\delta}$$\scriptstyle{\eta}$
(1.10)

Note that
$H^{1}_{\text{fl}}(\text{Spec}(R),\mu_{p^{n}})=R^{\times}/(R^{\times})^{p^{n}}$,
for a local ring $R$ [OT, Page-37]. Hence, we get that
$\ker(\eta)=\mathrm{coker}(\delta)=0$. Then $\text{coker}(\alpha)=0$ follows
from the diagram (1.10).∎

######  Lemma 1.5.

$R^{S}\big{(}(\mathbb{Z}/p\mathbb{Z})/F_{\infty}\big{)}$ is finite.

###### Proof.

We have
$F_{\infty}=\underset{n_{1},\cdots,n_{d}}{\bigcup}F_{n_{1},\cdots,n_{d+1}}$,
where
$\mathrm{Gal}(F_{n_{1},\cdots,n_{d+1}}/F)\cong\mathbb{Z}/p^{n_{1}}\mathbb{Z}\times\cdots\times\mathbb{Z}/p^{n_{d+1}}\mathbb{Z}$.
Using [St, Theorem 27.6], we observe that
$R^{S}\big{(}(\mathbb{Z}/p\mathbb{Z})/F_{n_{1},\cdots,n_{d+1}}\big{)}\cong\mathrm{Hom}(G_{S}^{\text{ab}}(F_{n_{1},\cdots,n_{d+1}})(p),\mathbb{Z}/p\mathbb{Z})\hookrightarrow\mathrm{Hom}(G_{\phi}^{\text{ab}}(F_{n_{1},\cdots,n_{d+1}})(p),\mathbb{Z}/p\mathbb{Z})$,
where $G_{\phi}(F_{n_{1},\cdots,n_{d+1}})^{\text{ab}}(p)$ is the Galois group
of the maximal abelian everywhere unramified pro-$p$ extension of
$F_{n_{1},\cdots,n_{d+1}}$. Also,
$\mathrm{Hom}(G_{\phi}^{\text{ab}}(F_{n_{1},\cdots,n_{d+1}})(p),\mathbb{Z}/p\mathbb{Z})\cong\mathrm{Hom}(Cl(F_{n_{1},\cdots,n_{d+1}})\otimes\mathbb{Z}_{p},\mathbb{Z}/p\mathbb{Z})$
$\cong\mathrm{Hom}(Cl(F_{n_{1},\cdots,n_{d+1}})/p,\mathbb{Z}/p\mathbb{Z})$. By
applying a control theorem, similar to the proof of Proposition 1.4, we get
that the kernel and the cokernel of the map
$Cl(F_{n_{1},\cdots,n_{d+1}})[p]\longrightarrow
Cl(F_{\infty})[p]^{\mathrm{Gal}(F_{\infty}/F_{n_{1},\cdots,n_{d+1}})}$ are
finite and bounded independently of $n_{i}$’s. As a result,
$Cl(F_{n_{1},\cdots,n_{d+1}})[p]$ is finite and bounded independently of
$n_{i}$’s. Moreover, as $Cl(F_{n_{1},\cdots,n_{d+1}})(p)$ is finite, the same
is true for $Cl(F_{n_{1},\cdots,n_{d+1}})/(p)$. Thus,
$R^{S}\big{(}(\mathbb{Z}/p\mathbb{Z})/F_{n_{1},\cdots,n_{d+1}}\big{)}$ is
finite and bounded independently of $n_{i}^{\prime}s$. Hence, we conclude that
$R^{S}\big{(}(\mathbb{Z}/p\mathbb{Z})/F_{\infty}\big{)}$ is finite. ∎

######  Lemma 1.6.

$S^{\prime}\big{(}\mu_{p}/F_{\infty}\big{)}$ is finite. It follows that
$R^{S}(\mu_{p}/F_{\infty})$ is also finite.

###### Proof.

We have an exact sequence:

$0\longrightarrow\dfrac{(\mathbb{F}^{(p)})^{\times}}{((\mathbb{F}^{(p)})^{\times})^{p}}\longrightarrow
S^{\prime}(\mu_{p}/F_{\infty})\longrightarrow Cl(F_{\infty})[p]\longrightarrow
0.$

Note that
$\frac{(\mathbb{F}^{(p)})^{\times}}{((\mathbb{F}^{(p)})^{\times})^{p^{m}}}=0$,
hence $S^{\prime}\big{(}\mu_{p}/F_{\infty}\big{)}\cong Cl(F_{\infty})[p]$. We
have $K_{d}^{\mathfrak{P}}=\underset{n_{1},\cdots
n_{d}}{\bigcup}K_{n_{1}\cdots n_{d}}$, where $K\subset K_{n_{1}\cdots
n_{d}}\subset K^{\mathfrak{P}}_{d}$ with $\mathrm{Gal}(K_{n_{1}\cdots
n_{d}}/K)\cong\mathbb{Z}/p^{n}_{1}\mathbb{Z}\times\cdots\times\mathbb{Z}/p^{n}_{d}\mathbb{Z}$.
Set $K_{n_{1}\cdots n_{d}}^{\infty}=K_{n_{1}\cdots n_{d}}K_{\infty}$ and
$\mathcal{G}_{n}=\mathrm{Gal}(K_{n_{1}\cdots
n_{d}}^{\infty,{p}}/K_{n_{1}\cdots n_{d}}^{\infty})$ with $K_{n_{1}\cdots
n_{d}}^{\infty,{p}}=K_{n_{1}\cdots n_{d}}^{\infty}\bar{\mathbb{F}}_{p}(t)$. As
profinite order of $\mathcal{G}_{n}$ is prime to $p$, following a standard
diagram chase using the definition of $S^{\prime}(\mu_{p^{\infty}}/-)$, we get
$S^{\prime}(\mu_{p^{\infty}}/K_{n_{1}\cdots
n_{d}}^{\infty,{p}})^{\mathcal{G}_{n}}\cong
S^{\prime}(\mu_{p^{\infty}}/K_{n_{1}\cdots n_{d}}^{\infty})$. Similarly, we
obtain that $Cl(K_{n_{1}\cdots n_{d}}^{\infty})[p^{\infty}]\cong
Cl({K_{n_{1}\cdots n_{d}}^{\infty,{p}})[p^{\infty}]}^{\mathcal{G}_{n}}$. Let
$C_{K_{n_{1}\cdots n_{d}}^{\infty,{p}}}$ be a proper smooth geometrically
connected curve which is the model of the function field $K_{n_{1}\cdots
n_{d}}^{\infty,{p}}$. On the other hand, [NSW, Proposition 10.1.1], we observe
that $Cl(K_{n_{1}\cdots
n_{d},\infty}^{{p}})[p^{\infty}]\cong(\mbox{$\mathbb{Q}$}_{p}/\mathbb{Z}_{p})^{r_{n_{1}\cdots
n_{d}}}$, where $0\leq r_{n_{1}\cdots n_{d}}\leq\text{genus}(C_{K_{n_{1}\cdots
n_{d}}^{\infty,{p}}})$. Therefore
$Cl(F_{\infty})[p^{\infty}]\cong\underset{n_{1},\cdots,n_{d}}{\varinjlim}\
Cl(K_{n_{1}\cdots n_{d}}^{\infty,p})[p^{\infty}]$ is $p$-divisible. By
Proposition 1.4, $Cl(F_{\infty})[p^{\infty}]^{\vee}$ is also a finitely
generated torsion $\mathbb{Z}_{p}[[\mathcal{H}]]$ module. Consequently, by
[Lim1, Lemma 2.4.1], we easily get that
$S^{\prime}\big{(}\mu_{p}/F_{\infty}\big{)}$ is finite. As
$R^{S}(\mu_{p}/F_{\infty})\hookrightarrow
S^{\prime}\big{(}\mu_{p}/F_{\infty}\big{)}$ (see Definition 1.2),
$R^{S}(\mu_{p}/F_{\infty})$ is finite as well. ∎

######  Theorem 1.7.

Let $K_{\infty}$ be the arithmetic $\mathbb{Z}_{p}$ extension of $K$ and
$K_{d}^{\mathfrak{P}}$ be the geometric $\mathbb{Z}_{p}^{d}$ extension of $K$,
where $d\geq 1$ (see §1). Let $F_{\infty}=K_{\infty}K_{d}^{\mathfrak{P}}$. Let
$\nu_{ram}$ be the unique prime of $K$ that ramifies in $F_{\infty}$.

Assume $E/K$ be an ordinary elliptic curve that has good reduction outside
$\nu_{ram}$. Then $R^{S}(E/F_{\infty})^{\vee}$ is a pseudonull
$\mathbb{Z}_{p}[[G]]$ module, where
$G=\mathrm{Gal}(F_{\infty}/F)\cong\mathbb{Z}_{p}^{d+1}$ and $S$ is any set of
primes of $K$ containing the prime $\nu_{ram}$.

###### Proof.

Let $H=\mathrm{Gal}(F_{\infty}/K_{\infty})$. By Theorem 1.3,
$R^{S}(E/K_{\infty})^{\vee}$ is a finitely generated $\mathbb{Z}_{p}$ module.
It is easy to see that the kernel and the cokernel of the map
$(R^{S}(E/F_{\infty})^{\vee})_{H}\longrightarrow R^{S}(E/K_{\infty})^{\vee}$
are finitely generated $\mathbb{Z}_{p}$-modules. Hence by Nakayama lemma,
$R^{S}(E/F_{\infty})^{\vee}$ is a finitely generated $\mathbb{Z}_{p}[[H]]$
module.

Let $S^{\prime}=\\{\nu_{ram}\\}$. Using ordinarity of $E/K$, from (1.7), we
obtain a complex (not necessarily exact):

$R^{S^{\prime}}(\mu_{p}/F_{\infty})\longrightarrow
R^{S^{\prime}}(E[p]/F_{\infty})\longrightarrow
R^{S^{\prime}}\big{(}(\mathbb{Z}/p\mathbb{Z})/F_{\infty}\big{)}.$ (1.11)

Then, from the definition of $R^{S^{\prime}}(\textunderscore/F_{\infty})$,
there is a commutative diagram (not necessarily exact):

${0}$${R^{S^{\prime}}\big{(}(\mathbb{Z}/p\mathbb{Z})/F_{\infty}\big{)}}$${H^{1}_{fl}(F_{\infty},\mathbb{Z}/p\mathbb{Z})}$${\underset{w\mid
v_{r}}{\prod}H^{1}_{\text{fl}}(F_{\infty,w},\mathbb{Z}/p\mathbb{Z})\underset{w\nmid
v_{r}}{\prod}\frac{H^{1}_{\text{fl}}(F_{\infty,w},\mathbb{Z}/p\mathbb{Z})}{H^{1}_{\text{fl}}(O_{F_{\infty,w}},\mathbb{Z}/p\mathbb{Z})}}$${0}$${R^{S^{\prime}}(E[p]/F_{\infty})}$${H^{1}_{fl}(F_{\infty},E[p])}$${\underset{w\mid
v_{r}}{\prod}H^{1}_{\text{fl}}(F_{\infty,w},E[p])\underset{w\nmid
v_{r}}{\prod}\frac{H^{1}_{\text{fl}}(F_{\infty,w},E[p])}{H^{1}_{\text{fl}}(O_{F_{\infty,w}},\mathcal{E}[p])}}$${0}$${R^{S^{\prime}}(\mu_{p}/F_{\infty})}$${H^{1}_{\text{fl}}(F_{\infty},\mu_{p})}$${\underset{w\mid
v_{r}}{\prod}H^{1}_{\text{fl}}(F_{\infty,w},\mu_{p})\underset{w\nmid
v_{r}}{\prod}\frac{H^{1}_{\text{fl}}(F_{\infty,w},\mu_{p})}{H^{1}_{\text{fl}}(O_{F_{\infty,w}},\mu_{p})}}$$\scriptstyle{\alpha}$$\scriptstyle{\beta}$$\scriptstyle{\gamma=\underset{w}{\prod}\gamma_{w}}$
(1.12)

Here $w$ denote a place of $F_{\infty}$. Note that
$R^{S^{\prime}}(\mu_{p}/F_{\infty})$ and
$R^{S^{\prime}}\big{(}(\mathbb{Z}/p\mathbb{Z})/F_{\infty}\big{)}$ are finite
by Lemmas 1.6 and 1.5, respectively. Note that there are only finitely many
primes of $F_{\infty}$ lying above $v_{r}$ in $F_{\infty}$.Now, from the proof
of Proposition 1.3, we get that $\ker(\gamma)$ is finite. Again using a
diagram chase, as in the proof of Proposition 1.3, we get that
$R^{S^{\prime}}(E[p]/F_{\infty})$ is finite.

Next, it is easy to see that the kernel and the cokernel of the natural map
$R^{S^{\prime}}(E[p]/F_{\infty})\rightarrow R^{S^{\prime}}(E/F_{\infty})[p]$
are finite. Thus
$R^{S^{\prime}}(E/F_{\infty})^{\vee}/(pR^{S^{\prime}}(E/F_{\infty})^{\vee})$
is finite. Now, it follows that $R^{S^{\prime}}(E/F_{\infty})^{\vee}$ is a
finitely generated torsion $\mathbb{Z}_{p}[[H]]$ module by [Lim1, Lemma
2.4.1]. Therefore, $R^{S^{\prime}}(E/F_{\infty})^{\vee}$ is a pseudonull
$\mathbb{Z}_{p}[[G]]$ module[Ve]. Finally, for any finite set $S$ of primes of
$K$ containing $S^{\prime}$=$\\{v_{r}\\}$, such that $E/K$ has good reduction
outside $v_{r}$, $R^{S}(E/L_{\infty})^{\vee}$ is a pseudonull
$\mathbb{Z}_{p}[[G]]$ module. ∎

Next, we give another new class of $\mathbb{Z}_{p}^{d}$ extensions which
satisfies Conjecture B.

######  Proposition 1.8.

Let $K_{d}^{\mathfrak{P}}$ be the geometric $\mathbb{Z}_{p}^{d}$ extension of
$K$, (see §1). Let $v_{r}$ be the unique prime of $K$ that ramifies in
$K_{d}^{\mathfrak{P}}$ and set $S^{\prime}=\\{v_{r}\\}$. Suppose that $E/K$ be
an ordinary elliptic curve that has good reduction outside the set
$S^{\prime}$.

Then $R^{S}(E/K_{d}^{\mathfrak{P}})^{\vee}$ is a pseudonull
$\mathbb{Z}_{p}[[\mathcal{G}]]$ module, where
$\mathcal{G}=\mathrm{Gal}(K_{d}^{\mathfrak{P}}/F)$ and $S$ is any set of
primes of $K$ with $S^{\prime}\subset S$.

###### Proof.

By [Aa, Proposition 2], we know that $Cl(K^{\mathfrak{P}})[p^{\infty}]$ is
finite. The proof now follows from arguments similar to the proof of Theorem
1.7. ∎

######  Example 1.

Let $p=2$ and $K=\mathbb{F}_{2}(t)$. Let $K_{d}^{\mathfrak{P}}$ be the
$\mathbb{Z}_{p}^{d}$ extension, where $d\geq 1$, constructed using Carlitz
module that is ramified only at the prime $\mathfrak{P}=(t)$. Let $E/K$ be
given by the Weierstrass equation:

$y^{2}+xy=x^{3}+(1/t)x^{2}+1,$ (1.13)

Then $E$ has bad reduction only at the prime $(t)$ of $K$ and is an ordinary
elliptic curve. Hence, this example of $E/K$ satisfies the assumptions of
Theorem 1.7 and Proposition 1.8. This in turn shows that $R^{S}(E/K_{\infty})$
is a pseudonull $\mathbb{Z}_{p}^{d+1}$ module and
$R^{S}(E/K^{\mathfrak{P}}_{d})$ is a pseudonull $\mathbb{Z}_{p}^{d}$ module.

Acknowledgements We would like to thank Aprameyo Pal, Somnath Jha and
Sudhanshu Shekhar for many helpful discussions and comments. The author
gratefully acknowledges the support from HRI postdoctoral fellowship.

## References

  * [Aa] A.Aiba, On the vanishing of Iwasawa invariants of geometric cyclotomic $\mathbb{Z}_{p}$-extensions, Acta Arith., 108 (2) (2003), 113-122.
  * [BH] P.N.Balister, S. Howson, Note on Nakayama’s lemma for compact $\Lambda$-modules, Asian Journal of Mathematics, 1 (2) (1997), 224-229
  * [CS] J.Coates, R.Sujatha, Fine Selmer group of Elliptic Curves over p-adic Lie extension, Math.Ann. 331 (4) (2005), 809-839.
  * [CFKSV] J.Coates, T.Fukaya, K.Kato, R.Sujatha, O.Venjakob, The $\rm GL_{2}$ main conjecture for elliptic curves without complex multiplication, Publ. Math. IHES 101 (2005), 163-208.
  * [Gr] R.Greenberg, Iwasawa theory-past and present, class field theory-its centenary and prospect, Adv. Stud. Pure Math. Math. Soc. Japan, Tokyo 30 (2001), 335-385.
  * [GJS] S.Ghosh, S.Jha, S.Shekhar, Iwasawa theory of fine Selmer groups over global fields, https://arxiv.org/abs/2201.01751.
  * [Jh] S.Jha, Fine Selmer group of Hida deformations overnon-commutative $p$-adic Lie extensions, Asian J. Math., 16(2)(2012), 353-365.
  * [JS] S.Jha, R.Sujatha, On the Hida deformations of fine Selmer groups, J. Algebra 338 (2011), 180-196.
  * [KT] K.Kato, F.Trihan, On the conjectures of Birch and Swinnerton-Dyer in characteristic $p>0$, Invent. Math. 153 (3) (2003), 537-592.
  * [Ku] M.Kurihara, On the Tate Shafarevich groups over cyclotomic fields of an elliptic curve with supersingular reduction. I, Inventiones Mathematicae 149(1) (2002), 195-224.
  * [LP] A.Lei, B.Palvannan, Codimension two cycles in Iwasawa theory and elliptic curves with supersingular reduction, Forum of Math. Sigma 7 (2019), Paper No. e25, 81.
  * [Lim1] M.F.Lim, Comparing the $\pi$-primary submodules of the dual Selmer groups, Asian J. Math. 21 (2017), 1153-1181.
  * [Lim2] M.F.Lim, On the control theorem for fine Selmer Groups and the growth of fine Tate-Shafarevich groups in $\mathbb{Z}_{p}$ extensions, Doc. Math. 25 (2020), 2445-2471.
  * [LLSTT] K.Lai, I.Longhi, T.Suzuki, K.Tan, F.Trihan, On the $\mu$-invariants of abelian varieties over function fields of positive characteristic, Algebra Number Theory 15 (4), 2021, 863-907.
  * [LV] M.F.Lim, V.Murty, The growth of fine Selmer groups, J. Ramanujan Math. Soc.  31(1) (2016), 79-94.
  * [M1] J.Milne, Arithmetic duality theorems, BookSurge, LLC, Charleston, SC 2 (2006).
  * [M2] J.Milne, Étale cohomology, Princeton University Press, Princeton, N.J. 33 (1986).
  * [NSW] J.Neukirch, A.Schmidt, K.Wingberg, Cohomology of number fields, Springer-Verlag, Berlin 323 (2) (2008), xvi+825.
  * [Oc] Y.Ochi, A remark on the pseudo-nullity conjecture for fine Selmer groups of elliptic curves, Comment. Math. Univ. St. Pauli 58 (1) (2009), 1-7.
  * [OT] T.Ochiai, F.Trihan, On the Selmer groups of abelian varieties over function fields of characteristic $p>0$, Math. Proc. Cambridge Philos. Soc. 146(1) (2009), 23-43.
  * [Pr] B.Perrin-Riou, Fonctions $L$ $p$-adiques des représentations $p$-adiques, Astérisque, 229 (1995).
  * [Ro] M.Rosen, Number theory in function fields, Graduate Texts in Mathematics, Springer-Verlag, New York,210 (2002), xii+358.
  * [St] W.Stein, A short course on Galois cohomology (2010) https://wstein.org/edu/2010/582e/lectures/all/galois_cohomology.pdf
  * [Sh] S.Shekhar, Comparing the corank of fine Selmer group and Selmer group of elliptic curves, J. Ramanujan Math. Soc., 33 (2) (2018), 205-217.
  * [Ve] O.Venjakob, A non-commutative Weierstrass preparation theorem and applications to Iwasawa theory, J. Reine Angew. Math. 559 (2003), 153-191.
  * [Wu] C.Wuthrich, Iwasawa theory of the fine Selmer group, J. Algebraic Geom.,16(1) (2007), 83-108.


# An Attention Free Long Short-Term Memory for Time Series Forecasting

Hugo Inzirillo  CREST - Institut Polytechnique de Paris
Simons
Email<EMAIL_ADDRESS>Ludovic De Villelongue
Université Paris Dauphine-PSL
Simons
Email<EMAIL_ADDRESS>
###### Abstract

Deep learning is playing an increasingly important role in time series
analysis. We focused on time series forecasting using attention free
mechanism, a more efficient framework, and proposed a new architecture for
time series prediction for which linear models seem to be unable to capture
the time dependence. We proposed an architecture built using attention free
LSTM layers that overcome linear models for conditional variance prediction.
Our findings confirm the validity of our model, which also allowed to improve
the prediction capacity of a LSTM, while improving the efficiency of the
learning task.

##  I Introduction

Recently we have seen a strong interest in the development of models based on
attention mechanisms, for encoder decoder based models. More recently, some
researchers have focused on the development of more efficient mechanisms, and
developed some alternatives such as the attention free mechanism [1] an
efficient variant of Transformers [2]. The initial problem was to improve the
performance of the encoder-decoder model. The one behind the attention
mechanism was to allow the decoder to use the most relevant parts. However,
the complex structure of these transformers does not seem to significantly
improve the prediction when dealing with time series [3]. In this paper we
have retained the usefulness of the attention mechanism and more particularly
its capacity to use the most relevant parts of the input sequence. We have
combined this strength with the power of a LSTM [4] to design a new framework
for time series prediction for assets for which linear models are no longer
sufficient to model the conditional volatility.

Digital assets became popular for their high volatility, typically where
linear models may have difficulties on predictiont task. In this paper we
focus on the modeling of volatility by feeding the outputs of a GARCH model
and adjust the prediction using an Attention Free Long Short-Term Memory (AF-
LSTM). The results are promising on various assets and make the AF-LSTM Layer
a more efficient predicator for time series.

##  II Deep Volatility

The assumption made about the errors of classical linear models is that they
are homoscedastic, they follow a centered distribution with a standard
deviation $\sigma$. This assumption is not realistic in financial
applications. In fact, in practice, we find that the errors are
heteroscedastic and this is more or less strong depending on the asset class.
In other words, the variance is time dependent. When we look at certain asset
classes, we observe volatility clusters and this is even stronger when we talk
about digital assets. This characteristic is typical of conditional
heteroscedasticity[5].

###  II-A Autoregressive conditional hererocesdastic models

ARCH is a univariate and nonlinear model in which volatility is estimated with
the square of past returns. Despite its appealing features, such as
simplicity, nonlinearity, easiness, and adjustment for forecast, the ARCH
model has certain drawbacks:

  1. 1.

equal response to positive and negative shocks

  2. 2.

strong assumptions such as restrictions on parameters

  3. 3.

possible misprediction due to slow adjustments to large movements

###  II-B GARCH Model

GARCH model [5] is also an approach to attempt to measure the volatility of
certain financial assets. GARCH models has been seen for decades as one of the
leading tools in terms of estimating the volatility of the financial returns.
Three main reasons have made GARCH models so popular:

  1. 1.

the variance of residuals is auto-correlated

  2. 2.

it allows to manage volatility clusters

  3. 3.

it represents the conditional variance for each step of time

A GARCH(P,Q) is defined as:

$\sigma_{t}^{2}=\omega+\sum_{p=1}^{P}\alpha_{p}\epsilon_{t-p}^{2}+\sum_{q=1}^{Q}\beta_{q}\sigma_{t-q}^{2}$
(1)

where $\epsilon_{t}=\sigma_{t}\mu_{t}$, and
$\mu_{t}\overset{i.i.d}{\sim}\mathcal{N}(0,1)$. If $\alpha+\beta<1$ the
volatility fluctuates around the unconditional variance and is given by:

$\sigma^{2}:=Var(r_{t})=\frac{\omega}{1-\alpha-\beta}$

###  II-C GJR-GARCH Model

The Glosten-Jagannathan-Runkle GARCH (GJR-GARCH) [6] is a variant of a GARCH
model, the underlying assumption is that negative shocks in the previous
period have a stronger impact on the variance than positive shocks. The GJR-
GARCH(P,Q) is defined as:

$\sigma_{t}^{2}=\omega+\sum_{p=1}^{P}\alpha_{p}\epsilon_{t-p}^{2}\gamma_{p}\epsilon_{t-p}^{2}\mathbf{1}_{\\{\epsilon_{t-p}<0\\}}+\sum_{q=1}^{Q}\beta_{q}\sigma_{t-q}^{2}$
(2)

where $\epsilon_{t}=\sigma_{t}\mu_{t}$, and
$\mu_{t}\overset{i.i.d}{\sim}\mathcal{N}(0,1)$. $\gamma_{p}$ equals to 0
implies that there is no asymmetric shock on the time series. If
$\alpha+\frac{\gamma}{2}+\beta<1$ the volatility fluctuates around the
unconditional variance, given by:

$\sigma_{GJR}^{2}:=Var(r_{t})=\frac{\omega}{1-\alpha-\frac{\gamma}{2}-\beta}$

Deep learning models are proposed to estimate the conditional volatility of
cryptocurrencies. Recurring neural networks (RNNs) are built to improve GARCH
predictions. For this purpose, a LSTM using sequences of past values of
volatility is taken as starting point and updated with embeddings
corresponding to the GARCH predicted volatility.

###  II-D LSTM

The LSTM framework [4] are RNNs with gated mechanisms designed to avoid
vanishing gradient. The blocks of LSTM contain 3 non-linear gates that make it
smarter than a classical neuron, as well as a memory for sequences. The 3
types of non-linear gates include :

  * •

Input Gate ($i_{t}$) : decides which values from the input are used to update
the memory

  * •

Forget Gate ($f_{t}$) : handles what information to throw away from the block

  * •

Output Gate ($o_{t}$): handles what will be in output based on input and
memory gate.

During the training step, each iteration provides an update of the model
weights proportional to the partial derivative and in some cases the gradient
may be vanishingly small and weights may not be updated. The LSTM networks is
defined in Figure 1 as:

Figure 1: LSTM Cell

$\begin{split}f_{t}&=\sigma(W_{f}[h_{t-1},z_{t-1}]+b_{f}),\\\
i_{t}&=\sigma(W_{i}[h_{t-1},z_{t-1}]+b_{i}),\\\
{\tilde{c}}_{t}&={\tanh(W_{c}[h_{t-1},z_{t-1}]+b_{c})},\\\ c_{t}&=f_{t}\odot
c_{t-1}+i_{t}\odot{\tilde{c}}_{t},\\\
o_{t}&=\sigma(W_{o}[h_{t-1},z_{t-1}]+b_{o}),\\\
h_{t}&=o_{t}\odot{\tanh(c_{t}).}\end{split}$ (3)

where $\Theta\triangleq\\{W_{f},W_{i},W_{c},W_{o}\\}$ are the weights of the
model and $z_{t-1}$ is the input vector at time t. At each time step, the
memory cell takes the current input $z_{t-1}$, the previous hidden state
$h_{t-1}$ and the previous memory state $c_{t-1}$.

The goal is to estimate $\hat{y_{t}}\triangleq$ $f(h_{t};\theta)$. In this
case it is equivalent to estimate:

$\mathbb{E}[\sigma_{t}^{2}|\mathcal{F}_{t-1}]=\mathbb{E}[\sigma_{t}^{2}|(\sigma_{t-1}^{2},...,\sigma_{t-q}^{2}),(\epsilon_{t-1}^{2},...,\epsilon_{t-p}^{2})].$

The loss function given by $L:Y\times Y\rightarrow{\mathbb{R}_{+}}$ measures
the distance between two outputs. In this case, the $l_{2}-loss$ is used. It
is given by:

$\begin{split}L(y,f(z;\theta))&=(y-f(z_{0:T-1};\theta))^{2},\\\
&=\frac{1}{T}\sum_{t=1}^{T}(y_{t}-f(z_{t-1};\theta))^{2}.\end{split}$ (4)

The number of parameters of each layer in the model should be carefully
selected to handle overfitting or underfitting situations.

###  II-E Deep GARCH

With the expanding need to improve volatility modeling precision, GARCH models
have been a redundant concern in finance and economic literature. However, the
relation between the conditional variances being non linear, these models have
found their limits. Some researcher works focusing on introducing non
linearity in these models rely on neural networks to introduce non-linearity
into classical models [7, 8]. Kim and Won [9] mixed the LSTM with multiple
GARCH-type models to introduce non linearity [10]. In this paper we seek to
improve the predictive ability of GARCH models. We do not question the
estimation of the model but we try to correct the prediction made at each time
step. To do this, we recover the conditional variances obtained after a first
estimation made by the GARCH model and then we apply filters and linear
transformations made possible by the use of several AF-LSTM layers.

###  II-F Attention Free Block

The transformer architecture has made it possible to develop new models
capable of being trained on large dataset while being much better than
recurrent neural networks such as LSTM. The Attention Free Transformer [1]
introduces the attention free block, an alternative to attention block [2],
which eliminates the need for dot product self attention. The transformer is
an efficient tool to capture the long term dependency. Just like the
transformer, the AF Block includes interactions between queries, keys and
values. The difference is that the AF block firstly combines key and value
together with a set of learned position biases described in [1]. Then the
query is combined with the context vector. Let Q, K, V denote the query, key
and value, respectively.

$\begin{split}Y_{t}&=\sigma_{q}(Q_{t})\odot\frac{\sum_{t^{{}^{\prime}}=1}^{T}exp(K_{t^{{}^{\prime}}}+w_{t,t^{{}^{\prime}}})\odot
V_{t^{{}^{\prime}}}}{\sum_{t^{{}^{\prime}}=1}^{T}exp(K_{t^{{}^{\prime}}}+w_{t,t^{{}^{\prime}}}))},\\\
&=\sigma_{q}(Q_{t})\odot\sum_{t^{{}^{\prime}}=1}^{T}(\text{Sofmax}(K)\odot
V)_{t{{}^{\prime}}}.\end{split}$ (5)

where $Q=ZW_{q}$, $K=ZW_{k}$ and $V=ZW_{v}$. The activation function
$\sigma_{q}$ is the sigmoid function and $w_{t}\in\mathbb{R}^{T\times T}$ is a
learned matrix of pair-wise position biases.

###  II-G Attention-free LSTM

As explained in the models described above, the expectation of volatility is
conditioned by past values. However some of these variables may at some point
not be relevant for the prediction. To reproduce this observable phenomenon,
we added an Attention Free Block to build an Attention-free LSTM layer (AF-
LSTM). The Attention-free LSTM layer processing step can be split in several
parts. The input at each time step will firstly be processed by an Attention
Free layer defined as:

$\tilde{Z}_{t-1}^{(l)}=AF^{(l)}_{1}(W_{1}^{(l)};z_{t-1}),$ (6)

where $Z_{t}\in\mathbb{R}^{q}$, $q$ the sequence length of our input and
$AF^{(l)}_{1}$ is the Attention Free of the $l-th$ layer:

$\begin{split}x_{t-1}^{(l)}&=\text{Concat}(W_{x}^{(l)}Z_{t-1}^{(l)}+b_{x}^{(l)}),\\\
Q_{t-1}^{(l)}&=W_{q}^{(l)}x_{t-1}^{(l)},\\\
K_{t-1}^{(l)}&=W_{k}^{(l)}x_{t-1}^{(l)},\\\
V_{t-1}^{(l)}&=W_{v}^{(l)}x_{t-1}^{(l)},\\\
\eta_{t-1}^{(l)}&=\sum_{t=1}^{T}\text{Softmax}(K_{t-1}^{(l)})\odot
V_{t-1}^{(l)},\\\
\tilde{Z}_{t-1}^{(l)}&=\sigma(Q_{t-1}^{(l)})\odot\eta_{t-1}^{(l)}.\\\
\end{split}$ (7)

we apply the Layer Normalization (LN) function [11]

$LN(x;\psi;\phi)=\psi\frac{(x-\mu_{x})}{\sigma_{x}}+\phi,$

and filter our $\tilde{Z}_{t-1}^{(l)}$ using the ReLU activation function,
$ReLU(x)=max(0,x)$.

Hence we have:

$\tilde{z}_{t-1}^{(l)}=ReLU(LayerNorm(\tilde{Z}_{t-1}^{(l)})),$ (8)

On the right side of 2 the input is the same as the left size. We denote
$\bar{Z}_{t-1}$ the output of the attention-free mechanism $AF_{2}^{(l)}$.
$\bar{Z}_{t-1}$ will resume the information of the input tensor hence it will
be multiplied by the ouput of the left side to only select the observations
that have a positive impact on the prediction.

$\begin{split}\bar{Z}_{t-1}^{(l)}&=AF_{2}^{(l)}(W_{2}^{(l)};Z_{t-1}),\\\
\bar{z}_{t-1}^{(l)}&=LN(\bar{Z}_{t-1};\psi^{(l)};\phi^{(l)}).\end{split}$ (9)

The output from the two channels will be multiplied to apply the filter on our
input sequence.

$\begin{split}\eta_{t-1}^{(l)}&=(\tilde{z}_{t-1}^{(l)}\odot\bar{z}_{t-1}^{(l)}),\\\
\zeta_{t-1}^{(l)}&=LN(\eta^{(l)}_{t-1};\psi^{(l)};\phi^{(l)}).\end{split}$
(10)

$\zeta_{t-1}^{(l)}$ will be the input of a parametrized function $g_{w}^{(l)}$
such as:

$h_{t}^{(l)}=g_{w}^{(l)}(\zeta_{t-1}^{(l)}),$ (11)

in Fig 2 $g_{w}^{(l)}(.)$ is a LSTM defined in Fig 1. Using the output of
$g_{w}^{(l)}(.)$ we can estimate the output of the $l-th$ layer
$\sigma_{t}^{(l)}$ given by:

$\sigma_{t-1}^{(l_{final})}=W_{y}^{(l_{final})}h_{t-1}^{(l_{final})}+b_{y}^{(l_{final})},\quad
l:=\\{1,...,l_{final}\\}.$ (12)

The ouput of our network is defined as $\sigma_{t}$ which is a proxy of the 5d
rolling volatility.

Figure 2: Attention-Free LSTM Layer

##  III Methodology

###  III-A Empirical Risk Minimization

Considering the training data
$\mathcal{D}\triangleq\\{Z^{(i)}\\}_{i=1}^{m}\in\mathcal{Z}^{m}$ and a
function $f:\mathcal{Z}->\mathcal{Y}$, the ERM is defined by:

$\hat{\mathcal{R}}_{\mathcal{D}}(f):=\mathcal{L}(f,Z^{(i)}).$ (13)

Lets define $\mathcal{A}_{erm}$ as an empirical risk minimization algorithm.
Hence, given a set of $\mathcal{F}$, the objective function is:

$A_{erm}(\mathcal{D})=\underset{f\in\mathcal{F}}{\text{arg
min}}\quad\hat{\mathcal{R}}_{\mathcal{D}}(f),$ (14)

where f is the residual mean squared error (RMSE):

$RMSE=\sqrt{\frac{\sum_{t=1}^{n}\left(Y_{t}-\hat{Y}_{t}\right)^{2}}{n}}.$ (15)

###  III-B Data

The data retrieved are the close prices of a single cryptocurrency asset from
inception. The log returns of this time series are calculated and fed to the
GARCH model. The distribution chosen for the standardized residuals is a
normal one. The lag parameters are defined to perform a first order GARCH
model. Once the model is fitted, the forecasted log returns are given by the
following formula:

$\begin{split}&r_{t}=\sigma_{t}\epsilon_{t},\\\ &\sigma_{t}^{2}=\omega+\alpha
r^{2}_{t-1}+\beta\sigma^{2}_{t-1}.\end{split}$ (16)

Then, the volatility calculated on a five period rolling window is compared
with the forecasted volatility obtained by dividing the GARCH forecasted log
returns with $\epsilon_{t}$. The volatility at time step t is calculated by
using the moving average of the previous 4 time steps’ returns (from time step
t-5 to time step t-1).The train and test sets are composed of real
volatilities and volatilities forecasted by the GARCH model. They are split
according to the 80/20 rule on a five period window and plugged to the LSTM
and AF-LSTM. Once the train and test sets are created, two scalers are
initialized on $x$ and $y$ to scale and translate them to a (0,1) range.
Afterwards, the train set for $x$ and $y$ are fitted and transformed, and the
test set for $x$ and $y$ only transformed. The fit method is calculating the
mean and variance of each of the features present in the data. The transform
method is transforming all the features using the respective mean and
variance. The parameters learned by the model using the training data will
help to transform the test data. Once it is done, the train and test sets
arrays are converted to tensors and inserted into dictionaries with the
scalers.

The LSTM network is fed with the following parameters :

  * •

Input size: 2 (number of expected features in the input)

  * •

Hidden size: 64 (number of features in hidden state h)

The AF-LSTM network is fed with the following parameters:

  * •

Input size: 2 (number of expected features in the input)

  * •

Hidden size: 64 (number of features in hidden state h)

  * •

Maximum sequence length: 1000 (maximum number of timesteps to be fed in)

  * •

Dim: 2 (embedding dimension of the keys, queries and values)

  * •

Hidden size : 64 (hidden dimension used inside the Attention Free Transformer)

After creation of the LSTM and AF-LSTM networks with an Adam optimizer, the
mean squared error loss function is computed on 1000 epochs with a learning
rate of 0.001. Once the predictions are obtained, the scalings of the
predicted and actual volatilities are inversed to get back to true volatility
values. For comparison purposes, the results are transformed to single arrays.
The root mean squared errors (RMSE) for the train and test sets can then be
calculated. Finally, the actual volatilities and predicted volatilities for
the train and test sets are plotted using the two models.

###  III-C Estimation

Figure 3: Actual Volatility vs Forecasted Volatility by GARCH(1,1)

A scalability matter is encountered when comparing the GARCH forecasted
volatility with the actual rolling volatility in Figure 3. The GARCH (1,1)
model is thus a weak learner for such a time series.

Figure 4:  LSTM and AF-LSTM Loss Functions for Train and Test Sets Figure 5:
LSTM and AF-LSTM Loss Functions Zoomed for Train and Test Sets

The loss function in Figure 4 is enlightning the better performance of the AF-
LSTM in taking information into account. On the one hand, regarding the train
set, the loss of the AF-LSTM is lower than the LSTM’s loss on all epochs. On
the other hand, regarding the test set, both losses are bottommost and are
decreasing at a truly slight rate. On a smaller interval, it is clearly
noticeable that the AF-LSTM has a lower loss than the LSTM on the test set, as
enlightened in Figure 5. It is the case for the integrality of the epochs.

Figure 6: Actual Volatility vs LSTM Forecasted Volatility for Train Set Figure
7: Actual Volatility vs LSTM Forecasted Volatility for Test Set Figure 8:
Actual Volatility vs AF-LSTM Forecasted Volatility for Train Set Figure 9:
Actual Volatility vs AF LSTM Forecasted Volatility for Test Set

According to the predictions in Figure 6 and Figure 8, the AF-LSTM is able to
catch more patterns than the LSTM. In fact, the lack of amplitude perceived in
the LSTM is partially corrected by the AF-LSTM on the train set. When we look
at the predictions on the test set in Figure 7 and Figure 9, the main
difference to notice is the reduction of the offset between the LSTM and AF-
LSTM results compared to the train set. However, the superiority of the AF-
LSTM in approximating the rolling volatility is still demonstrated.

The RMSE confirm those results:

Dataset | LSTM RMSE | AF LSTM RMSE
---|---|---
Train Set | 0.061 | 0.051
Test Set | 0.044 | 0.042

##  IV Results

By reducing the time and space complexity of the prediction algorithm, the AF-
LSTM Figure 9 has a real advantage over the LSTM in terms of sequential
computation time on a large data set. In fact, the Attention Free Mechanism
has a linear memory complexity with respect to both the context size and the
dimension of features, making it compatible to both large input and model
sizes. Moreover, the loss function and predictions are improved as additional
information are captured by this model. The AF-LSTM thus demonstrates
competitive performance while providing increased efficiency at the same time.

##  V Conclusion

The GARCH(1,1) derives its value based solely on the most recent updates of
$\epsilon$ and $\sigma$, which is helpful if the price changes are relatively
similar. However, this model’s weakness emerges when there are more sudden
jumps in log returns. To consider the repercution of theses shocks on the
volatility, the LSTM model should be used, as it tries to mimick the real data
with lags. To boost the performance and increase the power of prediction,
using an Attention Free LSTM layer can be very useful. It eliminates the
quadratic complexity of the self attention mechanism while selecting the past
observation that will have a positive impact on the prediction. In fact,
instead of carrying out the dot product for the creation of the attention
matrix, a weighted average of the values is carried out for each target
position. It maintains all the advantages of the dot product without the
computation cost.

## References

  * [1] S. Zhai, W. Talbott, N. Srivastava, C. Huang, H. Goh, R. Zhang, and J. M. Susskind, “An attention free transformer,” _CoRR_ , vol. abs/2105.14103, 2021.
  * [2] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” _CoRR_ , vol. abs/1706.03762, 2017.
  * [3] A. Zeng, M. Chen, L. Zhang, and Q. Xu, “Are transformers effective for time series forecasting?” 2022.
  * [4] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” _Neural computation_ , vol. 9, pp. 1735–80, 12 1997.
  * [5] T. Bollerslev, “Generalized autoregressive conditional heteroskedasticity,” _Journal of Econometrics_ , vol. 31, no. 3, pp. 307–327, 1986.
  * [6] L. R. Glosten, R. Jagannathan, and D. E. Runkle, “On the relation between the expected value and the volatility of the nominal excess return on stocks,” _The Journal of Finance_ , vol. 48, no. 5, pp. 1779–1801, 1993.
  * [7] N. Nikolaev, P. Tino, and E. Smirnov, “Time-dependent series variance learning with recurrent mixture density networks,” _Neurocomputing_ , vol. 122, pp. 501–512, 2013, advances in cognitive and ubiquitous computing.
  * [8] W. Kristjanpoller and M. C. Minutolo, “Gold price volatility: A forecasting approach using the artificial neural network–garch model,” _Expert Systems with Applications_ , vol. 42, no. 20, pp. 7245–7251, 2015.
  * [9] H. Y. Kim and C. H. Won, “Forecasting the volatility of stock price index: A hybrid model integrating lstm with multiple garch-type models,” _Expert Systems with Applications_ , vol. 103, pp. 25–37, 2018.
  * [10] W. K. Liu and M. K. P. So, “A garch model with artificial neural networks,” _Information_ , vol. 11, no. 10, 2020.
  * [11] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” 2016.
  * [12] A. Charles and O. Darné, “The accuracy of asymmetric garch model estimation,” _International Economics_ , vol. 157, pp. 179–202, 2019.
  * [13] H. Nessrine and B. Regaieg, “The glosten-jagannathan-runkle-generalized autoregressive conditional heteroscedastic approach to investigating the foreign exchange forward premium volatility,” _International Journal of Economics and Financial Issues_ , vol. 6, pp. 1608–1615, 09 2016.
  * [14] I. Bello, B. Zoph, A. Vaswani, J. Shlens, and Q. V. Le, “Attention augmented convolutional networks,” 2019.
  * [15] M. Buczyński and M. Chlebus, “Garchnet -value-at-risk forecasting with novel approach to garch models based on neural networks,” 06 2021.