# On the curvature of the blowup of a point

Gunnar Þór Magnússon  Hafnarfjörður, Iceland<EMAIL_ADDRESS>
###### Abstract.

We consider the blowup of a point of a compact Kähler manifold and a metric of
the form $\mu^{*}h+tb$ on it, where $h$ is a Kähler metric on the original
manifold and $b$ is Hermitian form that looks like the Fubini–Study metric
near the exceptional divisor. We calculate the curvature tensor of this metric
on the exceptional divisor and show that its holomorphic sectional curvature
is negative in some directions for all small enough $t$, which torpedos a
natural approach to showing that blowups of manifolds of positive holomorphic
sectional curvature have positive curvature.

## Introduction

Let $X$ be a compact Kähler manifold of dimension $n$. Let $h$ be a Kähler
metric on $X$. We write $D$ for its Chern connection and
$R(\alpha,\overline{\beta},\gamma,\overline{\delta})=h(\frac{i}{2}D^{2}_{\smash{\alpha,\overline{\beta}}}\gamma,\overline{\delta})$
for its curvature tensor. The holomorphic sectional curvature of $h$ is

$H(\xi)=R(\xi,\overline{\xi},\xi,\overline{\xi})/|\xi|^{4},$

where $\xi$ is a nonzero tangent field. We say that $h$ has positive
holomorphic sectional curvature if $H>0$ for all tangent fields. Examples of
manifolds that carry such metrics are the complex projective space with its
Fubini–Study metric, the standard metric on a Grassmannian manifold, and
projectivized bundles over bases of positive holomorphic sectional curvature
[2]. Tsukamoto [6] showed that manifolds that admit such metrics are simply
connected, and recently Xiaokui Yang [9] proved that such manifolds are
projective and rationally connected, answering a question of Yau [10, Problem
67].

A related question of Yau (again Problem 67) is whether the blowup of a
compact Kähler manifold of positive holomorphic sectional curvature along a
smooth submanifold again has positive holomorphic sectional curvature. This is
open; even for complex surfaces it is not known whether del Pezzo surfaces
admit positively curved metrics (aside from the del Pezzo surfaces of degrees
8 and 9, where the nontrivial one admits such a metric because it’s also a
Hirzebruch surface and thus a projectivized bundle). In this note we explain
that the brute force approach to this problem does not work.

Prior art in this direction has focused on showing that projective bundles
over manifolds with positive holomorphic sectional curvature are also
positive, along with more general fibrations with positively curved fibers and
base. The common thread in the papers of Hitchin [4], Sung [5], Álvarez [1],
Álvarez, Heier and Zheng [2] or Chaturvedi and Heier [3] is to consider
metrics of the form $\mu^{*}h+tb$, where $h$ is a metric on the base and $b$ a
fiberwise metric on the fibers and show they have positive curvature for small
enough $t$. In this note we try this approach for the blowup of a point. We
calculate the full curvature tensor of $h_{t}$ on a point on the exceptional
divisor and show that the metric does not have positive holomorphic sectional
curvature there, no matter what metric $h$ we started with. This of course
doesn’t mean the blowup doesn’t have positive holomorphic sectional curvature,
only that the most straightforward method of constructing a metric on it fails
to show that.

##  1\. Curvature

Let $X$ be a compact Kähler manifold of dimension $\dim_{\mathbf{C}}X=n$ and
let $h$ be a Kähler metric on it.

######  Proposition 1.

Let $\mu:\widehat{X}\to X$ be the blowup of $X$ at a point $p$. There exists a
closed $(1,1)$-form $\beta$ on $\widehat{X}$ whose restriction to the
exceptional divisor is the Fubini–Study metric.

###### Proof.

This is proved for blowups of smooth submanifolds in Voisin’s textbook [7]. We
can simplify that proof a little since we’re only blowing up points. In that
case the statement goes back at least to Kodaira’s proof of the embedding
theorem.

Locally around $p$, which may assume is the origin in a complex vector space
$V$, the blowup is

$\widehat{V}=\\{(v,[w])\in V\times\mathbf{P}(V)\mid v\in\mathbf{C}w\\}$

and $\mu$ is the projection onto the first factor. Pick an inner product on
$V$ and let $B(r)\subset V$ be a ball of radius $r$ centered at $0$. We pick
$r$ so that $B(r)$ fits into the coordinate chart that implicitly lurks in the
background. Let $\psi$ be a bump function supported on that chart that is
identically $1$ on $B(r)$. The $(1,1)$-form
$\frac{i}{2}\partial\bar{\partial}\log|v|^{2}$ on $V\setminus\\{0\\}$ descends
to $\mathbf{P}(V)$ and defines the Fubini–Study metric. If $p_{j}:V\times
V\setminus\\{0\\}\to V$ for $j=1,2$ are the projections onto the first and
second factors then
$\frac{i}{2}\partial\bar{\partial}(p_{1}^{*}\psi\log|p_{2}^{*}v|^{2})$ defines
a closed $(1,1)$-form on $V\times\mathbf{P}(V)$ that restricts to the pullback
of the Fubini–Study metric by $p_{2}$ on $B(r)\times\mathbf{P}(V)$. It also
extends to the rest of $X$ by zero. Its restriction to $\widehat{V}$ is the
form we want. ∎

Let $p\in X$ be a point and blow it up to obtain $\mu:\widehat{X}\to X$. Let
$b$ be the Hermitian form associated to the $(1,1)$-form $\beta$ on
$\widehat{X}$ we constructed in Proposition 1. Then $h_{t}=\mu^{*}h+tb$ is a
Kähler metric on $\widehat{X}$ for all $t$ small enough. We’re going to
calculate its curvature tensor at a point on the exceptional divisor.

Recall that locally around $p$ the blowup is

$\widehat{V}=\\{(v,[w])\in V\times\mathbf{P}(V)\mid v\in\mathbf{C}w\\}.$

If $f\in\operatorname{GL}V$ then $f$ acts on the blowup by
$f(v,[w])=(f(v),[f(w)])$. This is an isomorphism that maps the exceptional
divisor to itself, and we can map any point on the divisor to any other point
on it. Let $(0,[w])$ be a point on $E$. Let’s choose normal coordinates
$(x_{1},\ldots,x_{n})$ centered at $p$. There exists $f\in U(n)$ so that
$f(w)=(0\ldots,0,1)$, and the coordinates obtained by applying $f$ to the old
ones are still centered at $p$ and are normal there because $f\in U(n)$.
Picking the chart $\\{(y_{1},\ldots,y_{n})\in\mathbf{C}^{n}\mid
y_{n}\not=0\\}$ for $\mathbf{P}(V)$ we realize the blowup as

$\widehat{X}=\\{(x,y)\in\mathbf{C}^{n}\times\mathbf{C}^{n-1}\mid
x_{j}y_{k}=x_{k}y_{j}\text{ for $j,k=1,\ldots,n$, where $y_{n}=1$}\\}.$

In these coordinates the point we want to calculate the holomorphic sectional
curvature at is $(0,0)$ and we have $D_{h,\xi}\partial/\partial x_{j}=0$ at
$0$ for $j=1,\ldots,n$.

We also note that close to the exceptional divisor, $h_{t}=\mu^{*}h+tb$ is
just the restriction of the product metric $p_{1}^{*}h\oplus tp_{2}^{*}b$ on
$V\times\mathbf{P}(V)$ to $\widehat{X}$, where $p_{j}$ are the projections
onto the factors and $b$ is the Fubini–Study metric.

The only equations that give us any information at $(0,0)$ are

$x_{j}-y_{j}x_{n}=0,\quad j=1,\ldots,n-1.$

Their differentials are

$dx_{j}-y_{j}dx_{n}-x_{n}dy_{j}=0,\quad j=1,\ldots,n-1$

and so the tangent fields

$\xi_{j}=x_{n}e_{j}+f_{j},\quad
j=1,\ldots,n-1,\quad\text{and}\quad\xi_{n}=\sum_{j=1}^{n-1}y_{j}e_{j}+e_{n}$

are a basis for the intersection of all the kernels of the differentials, that
is, of $T_{\widehat{X}}$. Here $e_{j}$ is the tangent field corresponding to
the coordinate $x_{j}$ and $f_{k}$ the one corresponding to $y_{k}$. Note that
the orthogonal complement of $T_{\smash{\widehat{X}}}$ at $(0,0)$ is spanned
by $(e_{1},\ldots,e_{n-1})$. Any holomorphic tangent field $\xi$ on
$\widehat{X}$ near $(0,0)$ can be written as

$\xi=\sum_{j=1}^{n}\alpha_{j}\xi_{j}=\sum_{j=1}^{n-1}(\alpha_{j}x_{n}+\alpha_{n}y_{j})e_{j}+\sum_{j=1}^{n-1}\alpha_{j}f_{j}+\alpha_{n}e_{n},$

where the $\alpha_{j}$ are holomorphic functions.

While writing out the curvature tensor it will be useful to have the
semipositive Hermitian form
$\tau(\alpha,\bar{\beta})=\langle\alpha,\bar{\beta}\rangle-\alpha_{n}\bar{\beta}_{n}$
at hand. This is the pullback of the standard inner product by the projection
onto the subspace spanned by the first $n-1$ basis elements.

######  Proposition 2.

Let $\alpha=\sum_{j}\alpha_{j}\xi_{j}$, $\beta=\sum_{k}\beta_{k}\xi_{k}$,
$\gamma=\sum_{l}\gamma_{l}\xi_{l}$ and $\delta=\sum_{m}\delta_{m}\xi_{m}$ be
holomorphic tangent fields close to $(0,0)$. The curvature of $h_{t}$ at the
origin is

$\halign{\hbox to\displaywidth{$\hfil\displaystyle#\hfil$}\cr
0.0pt{\hfil$\displaystyle
R_{h_{t}}(\alpha,\overline{\beta},\gamma,\overline{\delta})=H_{h}(e_{n})\alpha_{n}\bar{\beta}_{n}\gamma_{n}\bar{\delta}_{n}+t\bigl{(}\tau(\alpha,\bar{\beta})\tau(\gamma,\bar{\delta})+\tau(\alpha,\bar{\delta})\tau(\gamma,\bar{\beta})\bigr{)}\hfill\cr
0.0pt{\hfil$\displaystyle\hfill{}-\alpha_{n}\bar{\beta}_{n}\tau(\gamma,\bar{\delta})-\alpha_{n}\bar{\delta}_{n}\tau(\gamma,\bar{\beta})-\gamma_{n}\bar{\beta}_{n}\tau(\alpha,\bar{\delta})-\gamma_{n}\bar{\delta}_{n}\tau(\alpha,\bar{\beta}).\crcr}}}$

###### Proof.

Recall that the curvature of $h_{t}$ is

$R_{h_{t}}(\alpha,\overline{\beta},\gamma,\overline{\delta})=p_{1}^{*}R_{h}(\alpha,\overline{\beta},\gamma,\overline{\delta})+tp_{2}^{*}R_{b}(\alpha,\overline{\beta},\gamma,\overline{\delta})-\langle\sigma(\alpha,\gamma),\overline{\sigma(\beta,\delta)}\rangle,$

where $\sigma(\alpha,\beta)=\pi_{N}(D_{h\oplus tb,\alpha}\beta)$ is the second
fundamental form, and the inner product is on the orthogonal complement of the
tangent bundle.

First,

$p_{1}^{*}R_{h}(\alpha,\overline{\beta},\gamma,\overline{\delta})=H_{h}(e_{n})\alpha_{n}\bar{\beta}_{n}\gamma_{n}\bar{\delta}_{n}$

at the origin. Second,

$\displaystyle
p_{2}^{*}R_{b}(\alpha,\overline{\beta},\gamma,\overline{\delta})$
$\displaystyle=\langle p_{2}(\alpha),\overline{p_{2}(\beta)}\rangle_{b}\langle
p_{2}(\gamma),\overline{p_{2}(\delta)}\rangle_{b}+\langle
p_{2}(\alpha),\overline{p_{2}(\delta)}\rangle_{b}\langle
p_{2}(\gamma),\overline{p_{2}(\beta)}\rangle_{b}$
$\displaystyle=\tau(\alpha,\bar{\beta})\tau(\gamma,\bar{\delta})+\tau(\alpha,\bar{\delta})\tau(\gamma,\bar{\beta})$

at the origin because the Fubini–Study metric has constant holomorphic
sectional curvature 2 and $\tau$ is just the inner product on
$\mathbf{C}^{n-1}$ extended to $\mathbf{C}^{n}$.

Let’s now write $\pi_{N}$ for the projection onto the orthogonal complement of
$T_{\smash{\widehat{X}}}$ in
$T_{\smash{\mathbf{C}^{n}\times\mathbf{C}^{n-1}|\widehat{X}}}$. At the origin
it is just the projection onto the first $n-1$ coordinates. Third, then,

$\pi_{N}(D_{h_{t},\alpha}\gamma)=\pi_{N}\biggl{(}\sum_{j=1}^{n}\delta_{\alpha}\gamma_{j}\,\xi_{j}+\sum_{j=1}^{n}\gamma_{j}D_{\alpha}\xi_{j}\biggr{)}.$

Note that $\pi_{N}(\xi_{j})=0$ for $j=1,\ldots,n$ so the first term from that
sum is zero. The second term is

$\pi_{N}(D_{\alpha}\xi_{j})=\pi_{N}(D_{\alpha}(x_{n}e_{j}+f_{j}))=\alpha_{n}e_{j}+x_{n}D_{\alpha}e_{j}=\alpha_{n}e_{j}$

for $j=1,\ldots,n-1$ at the origin because $D_{\alpha}e_{j}=0$ there. We also
have

$\pi_{N}(D_{\alpha}\xi_{n})=\pi_{N}\biggl{(}\sum_{k=1}^{n-1}\delta_{\alpha}y_{k}\,e_{k}+y_{k}D_{\alpha}e_{k}+D_{\alpha}e_{n}\biggr{)}=\sum_{k=1}^{n-1}\alpha_{k}e_{k}$

at the origin. Together we get

$\sum_{j=1}^{n}\gamma_{j}\pi_{N}(D_{\alpha}\xi_{j})=\sum_{j=1}^{n-1}\gamma_{j}\alpha_{n}e_{j}+\gamma_{n}\sum_{j=1}^{n-1}e_{j}=\sum_{j=1}^{n-1}(\alpha_{n}\gamma_{j}+\gamma_{n}\alpha_{j})e_{j}.$

And fourth,

$\pi_{N}(p_{2}^{*}D_{g,\alpha}\gamma)=\pi_{N}\biggl{(}\sum_{j=1}^{n-1}\delta_{\alpha}\gamma_{j}\,f_{j}+\gamma_{j}p_{2}^{*}D_{g,\alpha}f_{j}\biggr{)}=0$

at the origin because there the argument to $\pi_{N}$ takes values in the
bundle spanned by the $f_{j}$. The second fundamental form then contributes

$\displaystyle\langle\sigma(\alpha,\gamma),\overline{\sigma(\beta,\delta)}\rangle$
$\displaystyle=\sum_{j=1}^{n-1}(\alpha_{n}\gamma_{j}+\gamma_{n}\alpha_{j})(\bar{\beta}_{n}\bar{\delta}_{j}+\bar{\delta}_{n}\bar{\beta}_{j})$
$\displaystyle=\alpha_{n}\bar{\beta}_{n}\tau(\gamma,\bar{\delta})+\alpha_{n}\bar{\delta}_{n}\tau(\gamma,\bar{\beta})+\gamma_{n}\bar{\beta}_{n}\tau(\alpha,\bar{\delta})+\gamma_{n}\bar{\delta}_{n}\tau(\alpha,\bar{\beta})$

to the curvature. Putting all of this together we get the result. ∎

A fairly natural strategy to proving that the blowup of a point of a manifold
that has a metric with positive holomorphic sectional curvature is also
positively curved is to show that $h_{t}$ has positive holomorphic sectional
curvature on a neighborhood around the exceptional divisor for all small
enough $t$. Then we could conclude that the blowup has positive holomorphic
sectional curvature by using Wu’s [8] characterization of the holomorphic
sectional curvature to show positivity outside of that neighborhood for $t$
small enough. The following corollary shows this strategy cannot work.

######  Corollary 3.

The holomorphic sectional curvature of $h_{t}$ is negative in some tangent
directions at the origin for all small enough $t$.

###### Proof.

Let $\xi=\sum_{j}\alpha_{j}\xi_{j}$ be a holomorphic tangent field. Note that
$|\xi_{j}|_{h_{t}}^{2}=t$ for $j=1,\ldots,n-1$ and $|\xi_{n}|_{h_{t}}^{2}=1$
at the origin. We have

$H_{h_{t}}(\xi)=\frac{|\alpha_{n}|^{4}H_{h}(e_{n})+2t\tau(\alpha,\bar{\alpha})^{2}-4|\alpha_{n}|^{2}\tau(\alpha,\bar{\alpha})}{(t\tau(\alpha,\bar{\alpha})+|\alpha_{n}|^{2})^{2}}.$

First suppose that $H_{h}(e_{n})<0$. Picking $\alpha$ with $|\alpha_{n}|=1$ we
have $\tau(\alpha,\bar{\alpha})=0$ so $H_{h_{t}}(\xi)=H_{h}(e_{n})<0$.

Then suppose that $H_{h}(e_{n})\geq 0$. We may assume that $|\alpha|=1$ when
checking for positivity and focus only on the numerator. We’re then left
considering the positivity of the polynomial

$\displaystyle p_{t}(x)$ $\displaystyle=H_{h}(e_{n})x^{2}+2t(1-x)^{2}-4x(1-x)$
$\displaystyle=(H_{h}(e_{n})+2t+4)x^{2}-4(1-t)x+2t$

for $0\leq x\leq 1$. Its critical point is

$x_{t}=\frac{2(1-t)}{H_{h}(e_{n})+2t+4}$

where its value is

$p_{t}(x_{t})=-\frac{4(1-t)^{2}}{H_{h}(e_{n})+2t+4}+2t,$

which is negative for all small $t$. ∎

This corollary shows that _no_ metric of the form $\mu^{*}h+tb$ can have
positive holomorphic sectional curvature when $h$ does, so we can’t prove that
the blowup carries such a metric in this way. Note however that the blowup of
the projective plane at a single point is a Hirzebruch surface, thus a
projective bundle, and thus carries a metric of positive holomorphic sectional
curvature.

######  Corollary 4.

Let $\alpha=\sum_{j}\alpha_{j}\xi_{j}$, $\beta=\sum_{k}\beta_{k}\xi_{k}$ be
holomorphic tangent fields close to $(0,0)$. The Ricci tensor of $h_{t}$ is

$r_{t}(\alpha,\overline{\beta})=H_{h}(e_{n})\alpha_{n}\bar{\beta}_{n}+(n-1)\tau(\alpha,\bar{\beta})-(n-1)\alpha_{n}\bar{\beta}_{n}/t$

at the origin.

###### Proof.

The tangent fields that correspond to
$v_{j}=(\delta_{1j}/\sqrt{t},\ldots,\delta_{n-1,j}/\sqrt{t},\delta_{nj})$ are
orthonormal at the origin. If $\xi$ and $\eta$ are the fields corresponding to
$(\alpha_{j})$ and $(\beta_{j})$ we first have

$\sum_{j=1}^{n}H_{h}(e_{n})\alpha_{n}\bar{\beta}_{n}\delta_{jn}=H_{h}(e_{n})\alpha_{n}\bar{\beta}_{n},$

and then

$\sum_{j=1}^{n}\tau(\alpha,\bar{\beta})\tau(v_{j},\bar{v}_{j})+\tau(\alpha,\bar{v}_{j})\tau(v_{j},\bar{\beta})=n\tau(\alpha,\bar{\beta})/t.$

The second fundamental form contributes

$\halign{\hbox to\displaywidth{$\hfil\displaystyle#\hfil$}\cr
0.0pt{\hfil$\displaystyle\sum_{j=1}^{n}\alpha_{n}\bar{\beta}_{n}\tau(v_{j},\bar{v}_{j})+\alpha_{n}\delta_{jn}\tau(v_{j},\bar{\beta})+\delta_{jn}\bar{\beta}_{n}\tau(\alpha,\bar{v}_{j})+\delta_{jn}^{2}\tau(\alpha,\bar{\beta})\hfill\cr\vskip-10.0pt\cr
0.0pt{\hfil$\displaystyle\hfill{}=(n-1)\alpha_{n}\bar{\beta}_{n}/t+\tau(\alpha,\bar{\beta})\crcr}}}$

because $\tau(\alpha,\bar{v}_{n})=0$. Together we get

$r_{t}(\alpha,\bar{\beta})=H_{h}(e_{n})\alpha_{n}\bar{\beta}_{n}+(n-1)\tau(\alpha,\bar{\beta})-(n-1)\alpha_{n}\bar{\beta}_{n}/t.\qed$

We see that the Ricci tensor can also be negative in some directions for $t$
large enough, but is positive in other directions for all $t$.

######  Corollary 5.

The scalar curvature of $h_{t}$ is

$s_{t}=H_{h}(e_{n})+(n-1)(n-2)/t$

at the origin.

This is the same as Hitchin gets for the scalar curvature if we take his
expressions for $R_{0}$ and $R_{1}$ in [4, Lemmas 5.15 and 5.16] and evaluate
them at $r=0$, which is a nice sanity check. With more work I think we could
recover Hitchin’s full results. We’ve made some attempts in that direction to
see if we could extend his result on positive scalar curvature to Hermitian
metrics, but seem to run into obstructions due to the nonvanishing of the
connection at the origin.

## References

  * [1] Angelynn Reario Álvarez.  On the positive holomorphic sectional curvature of projectivized vector bundles over compact complex manifolds.  PhD thesis, University of Houston, 2016.  https://hdl.handle.net/10657/3250.
  * [2] Angelynn Reario Álvarez, Gordon Heier, and Fangyang Zheng.  On projectivized vector bundles and positive holomorphic sectional curvature.  Proc. AMS, 146(7):2877–2882, 2018.  URL: http://dx.doi.org/10.1090/proc/13868.
  * [3] Ananya Chaturvedi and Gordon Heier.  Hermitian metrics of positive holomorphic sectional curvature on fibrations.  Mathematische Zeitschrift, 295(1):349–364, 2020.  URL: https://doi.org/10.1007/s00209-019-02341-6.
  * [4] Nigel Hitchin.  On the curvature of rational surfaces.  In Proc. Symp. Pure Math, volume 27, pages 65–80, 1975.
  * [5] Myong-Hee Sung.  Kähler surfaces of positive scalar curvature.  Annals of Global Analysis and Geometry, 15(6):509–518, 1997.  URL: https://doi.org/10.1023/A:1006543703319.
  * [6] Yôtarô Tsukamoto.  On Kählerian manifolds with positive holomorphic sectional curvature.  Proceedings of the Japan Academy, 33(6):333–335, 1957.  URL: https://doi.org/10.3792/pja/1195525029.
  * [7] Claire Voisin.  Théorie de Hodge et géométrie algébrique complexe.  Number 10 in Cours Spécialisés. Soc. Math. France, 2002.
  * [8] H Wu and SS Chern.  A remark on holomorphic sectional curvature.  Indiana University Mathematics Journal, 22(11):1103–1108, 1973\.  URL: https://www.jstor.org/stable/24890553.
  * [9] Xiaokui Yang.  RC-positivity, rational connectedness and Yau’s conjecture.  Camb. J. Math., 6(2):183–212, 2018.  URL: https://dx.doi.org/10.4310/CJM.2018.v6.n2.a2.
  * [10] Shing-Tung Yau.  Open problems in geometry.  In Proc. Symp. Pure Math, volume 54, pages 1–28, 1993.


[a,b]German F. R. Sborlini

#  September 2023

Tackling Feynman integrals with quantum minimization algorithms

###### Abstract

One of the most severe bottlenecks to reach high-precision predictions in QFT
is the calculation of multiloop multileg Feynman integrals. Several new
strategies have been proposed in the last years, allowing impressive results
with deep implications in particle physics. Still, the efficiency of such
techniques starts to drastically decrease when including many loops and legs.
In this talk, we explore the implementation of quantum algorithms to optimize
the integrands of scattering amplitudes. We rely on the manifestly causal
loop-tree duality, which translates the loop into phase-space integrals and
avoids the spurious singularities due to non-causal effects. Then, we built a
Hamiltonian codifying causal-compatible contributions and minimize it using a
Variational Quantum Eigensolver. Our very promising results point towards a
potential speed-up for achieving a more numerically-stable representation of
Feynman integrals by using quantum computers.

##  1 Motivation

In the context of high-energy physics at colliders, we are usually interested
in computing cross-sections. Since the energy of the collisions is very high
compared to the mass of the particles, we can safely rely on perturbative QFT.
Then, the precision of the prediction is increased by adding higher orders to
the perturbative expansion. However, this is not an easy task, because we need
to deal with Feynman loop integrals and real emission processes. Both
contributions contain singularities which must be regulated in order to cancel
them. This increases their complexity, leading to severe bottlenecks to go
beyond the current state-of-the-art, namely next-to-next-to-leading order
(NNLO) in QCD for most of the processes of interest at colliders.

In recent years, several methods were developed to extract very precise
predictions from the Standard Model (SM) and other quantum field theories
(QFT) for collider phenomenology [1]. Most of these methods rely on a
cancellation of singularities _after_ integration although local techniques
(i.e. for cancelling the singularities _before_ integration) are starting to
appear. One of them is based on the Loop-Tree Duality (LTD) theorem [2, 3, 4,
5]. It allows to open the loops and recast the virtual contributions in terms
of tree-level-like amplitudes integrated over a real radiation phase-space.
When adding a mapping, both real and virtual contributions can be merged
within a single _finite_ integral: this is the Four-Dimensional Unsubtraction
(FDU) framework [6, 7, 8].

Still, the LTD has shown fascinating properties that go beyond the local
cancellation of singularities. In particular, when adding all the dual
contributions, the result is manifestly causal [9, 10, 11, 12, 13]. This
causal behaviour can be recovered from geometrical rules [14], and is deeply
connected to the identification of directed acyclic graphs (DAG) [15, 16].

Besides the development of new methods to calculate cross-sections, overcoming
the precision frontier might require to explore novel technologies. One
candidate is quantum computing, since it has shown a potential speed-up in
database searching [17, 18], factorization [19] and solving
minimization/optimization problems [20]. In the context of high-energy
physics, it has been applied to scattering amplitude calculations [21, 22,
23], jet clustering [24] and, very recently, to perform the causal
reconstruction of scattering amplitudes within LTD [15, 16]. In this article,
we will briefly describe how to solve DAG detection with a quantum computer
and exploit this information to bootstrap the causal LTD representation of
multiloop multileg scattering amplitudes.

##  2 Causal Loop-Tree Duality from Geometry

The LTD theorem relies on the iterative application of Cauchy’s residue (CR)
theorem to remove one degree-of-freedom per loop. Even if the formalism is
general, we eliminate the energy component of each loop in order to change the
integration domain from a Minkowski to an Euclidean one, thus closely
resembling a phase-space integral. In this way, computing the residue is
equivalent to put on-shell internal lines of the corresponding loop diagram.

The iterative application of the CR theorem has several subtleties [12], among
them the cancellation of non-physical contributions originated from the so-
called _displaced poles_. In the end, each combination of poles contributing
to the dual amplitude (i.e. the LTD representation of the amplitude) can be
mapped into a tree-level-like diagram obtained by cutting $L$ lines for a
$L$-loop Feynman amplitude.

Furthermore, when all the dual contributions are added together, a noticeable
simplification takes place [9, 10, 12]. In fact, any $L$-loop scattering
amplitude with $V$ vertices can be expressed as

$A^{(L)}=\sum_{\sigma\in\Sigma}\int_{\vec{\ell}_{1},\ldots,\vec{\ell}_{1}}(-1)^{k}\,\frac{{\cal
N}_{\sigma}(\\{q_{r,0}^{(+)}\\},\\{p_{j,0}\\})}{x_{L+k}}\,\prod_{i=1}^{k}\frac{1}{\lambda_{\sigma(i)}}\,+\,(\sigma\longleftrightarrow\bar{\sigma})\,,$
(1)

which is known as a _causal representation_. In this expression, $k=V-1$ is
the order of the topology associated to the _reduced_ Feynman diagram,
$q^{(+)}_{i,0}$ is the on-shell positive energy of the $i$-th internal line
and $x_{L+k}=\prod_{j=1}^{L+k}2q^{(+)}_{j,0}$ is the analogous to the
integration measure of the phase-space111More information about this formula
is available in Refs. [9, 25, 14].. $\\{\lambda_{i}\\}$ are associated to
physical threshold singularities of the diagram and $1/\lambda_{i}$ are known
as _causal propagators_. $\sigma$ represents a combination of $k$ _entangled
thresholds_ , and the compatibility conditions that determine $\Sigma$ can be
fixed by following geometrical rules [14]. One of these rules establishes that
the binary partitions associated to each $\lambda_{\sigma(i)}$ for
$\sigma\in\Sigma$ must have a compatible momentum flux. It turns out that such
momentum flux compatibility is possible if and only if the associated directed
graph (DG) for the reduced Feynman diagram has no loops [15, 26]. Thus, the
geometrical formulation of causal LTD implies that the causal representation
in Eq. (1) can be bootstrapped from DAGs.

##  3 Quantum algorithms for LTD

At this point, the problem of finding a causal representation has been
partially reduced to the identification of DAGs. Classical algorithms for this
purpose rely on construction strategies, as well as the direct evaluation of
acyclicity conditions. In both cases, for dense graphs (i.e. corresponding to
multiloop multileg Feynman diagrams with multi-particle interactions), the
scaling is almost exponential in the number of edges. Being able to
efficiently parallelize the evaluation of the acyclicity condition could lead
to an important speed-up: for this reason, we decided to explore quantum
algorithms.

Figure 1: Execution of the multi-run VQE algorithm on a representative
topology with 5 edges. In four steps, we manage to collect exactly the 9
acyclic configurations, with 0 error rate.

Let us start considering a directed graph $G$ composed by the vertices $V$
linked by the set of edges $E$. Each element in $E$ is denoted as
$e_{i}=(v_{i,0},v_{i,1})$. To define a quantum algorithm, we promote the
vertices and edges to a Hilbert space ${\cal H}=E\times V$. From graph theory,
we know that all the geometric information of $G$ is codified within the
adjacency matrix $A$, so we define the associated operator

${\cal A}=\sum_{e_{i}=(v_{i,0},v_{i,1})\in
E}\left[\sigma_{v_{0}}^{-}\pi^{0}_{e}\sigma_{v_{1}}^{+}+\sigma_{v_{1}}^{-}\pi^{1}_{e}\sigma_{v_{0}}^{+}\right]\,,$
(2)

with $\sigma^{\pm}_{v}$ the Pauli matrices in the vertex space and
$\pi_{e}^{0,1}$ a projector w.r.t. a reference orientation of the edges [16].
The powers of the classical adjacency matrix contain information about cycles:
if ${\rm tr}(A^{n})=0$ then there are not $n$-cycles. So, we define the
Hamiltonian

${\cal H}=\sum_{n=1}^{\\#(V)}{\rm tr}_{V}\left({\cal A}^{n}\right)\,,$ (3)

that will have, by construction, a degenerated ground state with zero energy
composed by all the possible acyclic configurations of $G$. This Hamiltonian
is an operator acting on the space of edges, since we trace over the space of
vertices.

In Ref. [16], we built Hamiltonians for different topologies and minimized it
using a Variational Quantum Eigensolver (VQE) [20], which can offer advantages
w.r.t. classical minimization algorithms. However, standard VQE is not
intended to minimize a multi-degenerated problem and, in our case, there are
roughly $2^{\\#E-1}$ solutions for some topologies. For this reason, we
proposed a _multi-run_ VQE algorithm: we run a first VQE, impose a threshold
to retain potential solutions, add a penalization term to ${\cal H}$ and then
re-run the VQE. We use the energy of the approximated ground state to control
the converge and penalization terms to avoid finding repeated solutions. In
Fig. 1 we present a representative example in which our algorithm took 4 steps
to collect all the acyclic graphs associated to a sunrise-like diagram.

##  4 Conclusions and outlook

In this article, we review potential advantages of the LTD theorem and the
corresponding causal representations for computing multiloop multileg
scattering amplitudes. The causal representation contains only physical
singularities at integrand level, and it is defined in an Euclidean space. As
we shown in Ref. [14], it can be obtained through geometrical selection rules
that provide the full set of causal-compatible entangled thresholds.
Furthermore, in Refs. [15, 16], we show that causal representations can be
bootstrapped from the set of DAGs associated to the Feynman diagrams.

It turns out that relating causality and DAGs allows to recast the problem of
obtaining causal representations in such a way that it can be handled by
quantum computers. Thus, we codified the possible directions of the edges into
qubits and we define a Hamiltonian whose ground state contains all the
possible acyclic graphs. By using a multi-run VQE algorithm, we manage to
identify the acyclic graphs with a high success rate. These developments can
be considered as a first step towards computing Feynman integrals through
quantum minimization algorithms. Besides allowing to calculate complicated
multiloop multileg scattering amplitudes, these quantum algorithms might
revolutionize the way in which we obtain high-precision predictions from QFT.

### Acknowledgments

We thank D. Suarez Fontanella, D. Rentería-Estrada and A. Rentería-Olivo for
their fruitful comments. This work is supported by the Spanish Government
(Agencia Estatal de Investigación MCIN /AEI/10.13039/501100011033) Grants No.
PID2020-114473GB-I00, PID2022-141910NB-I00, Generalitat Valenciana Grant No.
PROMETEO/2021/071 and H2020-MSCA-COFUND USAL4EXCELLENCE-PROOPI-391 project
under Grant Agreement No 101034371.

## References

  * [1] G. Heinrich, _Collider Physics at the Precision Frontier_ , _Phys. Rept._ 922 (2021) 1–69, [2009.00516].
  * [2] S. Catani, T. Gleisberg, F. Krauss, G. Rodrigo and J.-C. Winter, _From loops to trees by-passing Feynman’s theorem_ , _JHEP_ 09 (2008) 065, [0804.3170].
  * [3] I. Bierenbaum, S. Catani, P. Draggiotis and G. Rodrigo, _A Tree-Loop Duality Relation at Two Loops and Beyond_ , _JHEP_ 10 (2010) 073, [1007.0194].
  * [4] I. Bierenbaum, S. Buchta, P. Draggiotis, I. Malamos and G. Rodrigo, _Tree-Loop Duality Relation beyond simple poles_ , _JHEP_ 03 (2013) 025, [1211.5048].
  * [5] S. Buchta, G. Chachamis, P. Draggiotis, I. Malamos and G. Rodrigo, _On the singular behaviour of scattering amplitudes in quantum field theory_ , _JHEP_ 11 (2014) 014, [1405.7850].
  * [6] R. J. Hernandez-Pinto, G. F. R. Sborlini and G. Rodrigo, _Towards gauge theories in four dimensions_ , _JHEP_ 02 (2016) 044, [1506.04617].
  * [7] G. F. R. Sborlini, F. Driencourt-Mangin, R. Hernandez-Pinto and G. Rodrigo, _Four-dimensional unsubtraction from the loop-tree duality_ , _JHEP_ 08 (2016) 160, [1604.06699].
  * [8] G. F. R. Sborlini, F. Driencourt-Mangin and G. Rodrigo, _Four-dimensional unsubtraction with massive particles_ , _JHEP_ 10 (2016) 162, [1608.01584].
  * [9] J. J. Aguilera-Verdugo, F. Driencourt-Mangin, R. J. Hernández-Pinto, J. Plenter, S. Ramirez-Uribe, A. E. Renteria Olivo et al., _Open Loop Amplitudes and Causality to All Orders and Powers from the Loop-Tree Duality_ , _Phys. Rev. Lett._ 124 (2020) 211602, [2001.03564].
  * [10] J. J. Aguilera-Verdugo, R. J. Hernandez-Pinto, G. Rodrigo, G. F. R. Sborlini and W. J. Torres Bobadilla, _Causal representation of multi-loop Feynman integrands within the loop-tree duality_ , _JHEP_ 01 (2021) 069, [2006.11217].
  * [11] S. Ramírez-Uribe, R. J. Hernández-Pinto, G. Rodrigo, G. F. R. Sborlini and W. J. Torres Bobadilla, _Universal opening of four-loop scattering amplitudes to trees_ , _JHEP_ 04 (2021) 129, [2006.13818].
  * [12] J. Aguilera-Verdugo, R. J. Hernández-Pinto, G. Rodrigo, G. F. R. Sborlini and W. J. Torres Bobadilla, _Mathematical properties of nested residues and their application to multi-loop scattering amplitudes_ , _JHEP_ 02 (2021) 112, [2010.12971].
  * [13] J. Aguilera-Verdugo et al., _A Stroll through the Loop-Tree Duality_ , _Symmetry_ 13 (2021) 1029, [2104.14621].
  * [14] G. F. R. Sborlini, _Geometrical approach to causality in multiloop amplitudes_ , _Phys. Rev. D_ 104 (2021) 036014, [2102.05062].
  * [15] S. Ramírez-Uribe, A. E. Rentería-Olivo, G. Rodrigo, G. F. R. Sborlini and L. Vale Silva, _Quantum algorithm for Feynman loop integrals_ , _JHEP_ 05 (2022) 100, [2105.08703].
  * [16] G. Clemente, A. Crippa, K. Jansen, S. Ramírez-Uribe, A. E. Rentería-Olivo, G. Rodrigo et al., _Variational quantum eigensolver for causal loop Feynman diagrams and acyclic directed graphs_ , 2210.13240.
  * [17] L. K. Grover, _Quantum computers can search rapidly by using almost any transformation_ , _Phys. Rev. Lett._ 80 (1998) 4329–4332, [quant-ph/9712011].
  * [18] L. K. Grover, _Quantum mechanics helps in searching for a needle in a haystack_ , _Phys. Rev. Lett._ 79 (1997) 325–328, [quant-ph/9706033].
  * [19] P. W. Shor, _Polynomial time algorithms for prime factorization and discrete logarithms on a quantum computer_ , _SIAM J. Sci. Statist. Comput._ 26 (1997) 1484, [quant-ph/9508027].
  * [20] J. Tilly et al., _The Variational Quantum Eigensolver: A review of methods and best practices_ , _Phys. Rept._ 986 (2022) 1–128, [2111.05176].
  * [21] G. Agliardi, M. Grossi, M. Pellen and E. Prati, _Quantum integration of elementary particle processes_ , _Phys. Lett. B_ 832 (2022) 137228, [2201.01547].
  * [22] H. A. Chawdhry and M. Pellen, _Quantum simulation of colour in perturbative quantum chromodynamics_ , 2303.04818.
  * [23] J. J. M. de Lejarza, M. Grossi, L. Cieri and G. Rodrigo, _Quantum Fourier Iterative Amplitude Estimation_ , 2305.01686.
  * [24] J. J. M. de Lejarza, L. Cieri and G. Rodrigo, _Quantum clustering and jet reconstruction at the LHC_ , _Phys. Rev. D_ 106 (2022) 036021, [2204.06496].
  * [25] W. J. Torres Bobadilla, _Loop-tree duality from vertices and edges_ , _JHEP_ 04 (2021) 183, [2102.05048].
  * [26] G. F. R. Sborlini, _Geometry and causal flux in multi-loop Feynman diagrams_ , _Rev. Mex. Fis. Suppl._ 3 (2022) 020703, [2109.14476].


# Multistream Graph Attention Networks for Wind Speed Forecasting

Dogan Aykas  Department of Knowledge Engineering
Maastricht University
Maastricht, The Netherlands
<EMAIL_ADDRESS>Siamak Mehrkanoon *corresponding
author. Department of Knowledge Engineering
Maastricht University
Maastricht, The Netherlands
<EMAIL_ADDRESS>
###### Abstract

Reliable and accurate wind speed prediction has significant impact in many
industrial sectors such as economic, business and management among others.
This paper presents a new model for wind speed prediction based on Graph
Attention Networks (GAT). In particular, the proposed model extends GAT
architecture by equipping it with a learnable adjacency matrix as well as
incorporating a new attention mechanism with the aim of obtaining attention
scores per weather variable. The output of the GAT based model is combined
with the LSTM layer in order to exploit both the spatial and temporal
characteristics of the multivariate multidimensional historical weather data.
Real weather data collected from several cities in Denmark and Netherlands are
used to conduct the experiments and evaluate the performance of the proposed
model. We show that in comparison to previous architectures used for wind
speed prediction, the proposed model is able to better learn the complex
input-output relationships of the weather data. Furthermore, thanks to the
learned attention weights, the model provides an additional insights on the
most important weather variables and cities for the studied prediction task.

###### Index Terms:

Wind speed prediction, graph attention networks, attention per weather
variable, learnable adjacency matrix, attention visualisation.

##  I Introduction

Wind speed forecasting has gained a lot of attention in recent years since
wind speed plays a significant role in human life. Sectors like agriculture,
construction and energy can greatly benefit from an accurate estimation of
several time steps ahead wind speed values. The field of weather forecasting
generally utilizes Numerical Weather Prediction (NWP) [1] methods which are
based on mathematical formulations using several atmospheric features. With
this method, the predictions are made by using equations that are derived with
the aid of laws of physics and thermodynamics. Although NWP is a powerful
method to such forecasting tasks, it also has its drawbacks such as requiring
too much computational power [2]. Moreover, due to the assumptions of weather
variables used in this method, forecasting with NWP might be sensitive to
noise that may be present in the weather variable measurements [3].

The availability of large training data allows data-driven models to learn the
existing underlying complex pattern of the data. In addition, thanks to the
advances in computational power, one is able to train deep machine learning
models. Therefore, deep learning based models have recently been increasingly
used in weather forecasting tasks as an alternative to NWP [4, 5, 6, 7, 8, 9,
10, 11, 12]. In particular, the literature of weather forecasting models has
witnessed the use of Convolutional Neural Network based approaches [3, 13, 8,
14], Graph Convolutional Neural Networks [15, 16], Recurrent Convolutional
Neural Networks [17, 18] and Restricted Boltzmann Machines [19, 20].

This paper uses Graph Attention Networks (GAT) [21] as the backbone of the
proposed model. Graph Attention Networks are a subclass of Graph Neural
Networks designed for analyzing graph data. GAT learns the spatial
relationship between different vertices of a graph and how the features of a
vertex may affect features of other vertices. Here, we propose an extension of
GAT architecture such that features of a vertex can be represented as a
2-dimensional matrix. The introduced model consists of GAT architecture
followed by a Long Short Term Memory (LSTM) layer in order to utilize both the
spatial and temporal aspect of the data, respectively. In contrast to vanilla
GATs, which assume each vertex of a graph has one dimensional features, our
proposed model includes GAT layers that compute the attention scores by
allowing the vertices to have two dimensional features. This enhancement to
GAT allows calculating attention of the vertices per weather variable,
resulting in a more detailed and interpretable GAT structure. The authors in
[15] proposed an deep Graph Convolutional Networks model for wind speed
prediction tasks. In particular, they treated the weather stations as nodes of
a graph whose associated adjacency matrix is learnable. Here, following the
lines of [15], the GAT layer in our proposed model is also equipped with
learnable adjacency matrix. This helps the model to learn the connection
between weather stations in an end-to-end fashion, This type of information is
not typically available for such forecasting tasks.

In our experiments, two real life historical weather data from Dutch and
Danish cities are used to predict the wind speed values in multiple weather
stations for several hours ahead ranging from 2 up to 24 hours. The paper is
organized as follows. A brief overview of the related work is given in section
II. The proposed model is described in section III. The numerical experiments,
discussion of the obtained results and comparison with other models are given
in Section IV. The conclusion is drawn in section V.

##  II Related Work

With the rise of impressive results obtained by data driven based models, the
literature has witnessed the design of various deep learning architectures for
weather forecasting tasks [22, 6, 23, 24, 3, 13]. The weather data can be
considered as time series data where observations occur as a sequence.
Recurrent neural networks have been often used for dealing with time series
forecasting tasks [25]. Specifically, Long Short Term Memory (LSTM) models are
preferred for such tasks due to their success in handling vanishing/exploding
gradients [26] which can possibly hinder the performance during training
phase. However, vanilla LSTMs do not explicitly exploit the spatial features
of data and thus, they are not able to capture the influences of the features
observed in one city to the other cities and vice versa.

On the other hand, Convolutional Neural Networks (CNN) are widely used,
especially for computer vision tasks such as face recognition, image
classification [27]. While vanilla CNNs are able to process the spatial aspect
of the data, it is unable to efficiently make use of the temporal
characteristics which are important for such weather forecasting applications.
The authors in [3], introduced a 3D-CNN based model for weather elements
forecasting which exploit the spatio-temporal multivariate weather data for
learning shared representations. In [28], Temporal Convolutional Networks
(TCN), a type of CNN which uses one dimensional filters in order to focus on
the temporal aspect of data, are used in combination with vanilla CNNs for
weather forecasting tasks . CNNs have the ability to process the spatio-
temporal aspect of the data with the addition of TCNs.

Due the nature of the fixed size filters, CNNs have the tendency to treat
observations from multiple stations as regular or Euclidean data [29].
Considering the spatial dimension of our data which corresponds to different
weather stations, it can be highlighted that the underlying graph structure
cannot be efficiently expressed under the the regular data setting. Therefore,
the captured spatial relationship between observations are restricted to the
filter size in CNNs. In the context of weather data, the spatial aspect of
historical observations from multiple weather stations can be expressed more
efficiently as a graph, with stations as vertices and connections between
stations as edges.

Graph Neural Networks (GNN) [30], has recently gained popularity for analyzing
graph data. A variation of GNN, i.e. Graph Convolutional Networks (GCNs) [31],
combines the basics of GNNs by working with graph data rather than assuming
the data to be Euclidean, while maintaining a convolutional architecture.
Although most variants of GCNs mainly deal with node classification tasks
which require the vertex and edge information of the graph data to be known,
there has been GCN adaptations in [32, 15] with a learnable adjacency matrix.
The learnable adjacency matrix, which is also used in our proposed model, is
significant for data with spatial features that are not represented explicitly
in a graph format with a known adjacency matrix. This on one hand, allows
regular data to be represented with an adjacency matrix usually required in
GNN implementations. On the other hand, the adjacency matrix is learned during
the training phase, which is beneficial for the spatial information
extraction. Attention mechanisms have recently gained a lot of popularity,
thanks to the transformer architecture introduced in [33]. The attention
mechanism has also been incorporated in GNN frameworks. For instance, GAT [21]
uses self attention mechanism similar to the one in transformer model. In
particular, GAT uses a self attention mechanism in order to calculate the
attention coefficient between vertices of a graph. Furthermore, there have
been implementations of GATs used for spatio-temporal data, specifically for
traffic flow forecasting [34].

In what follows, we present a novel model which consists of an extension of
GATs architecture followed by LSTM layer for wind speed prediction tasks. The
proposed modifications of GATs offer an improved performance for weather
element forecasting and further interpretability using the visualization of
the learned attention weights of cities per weather variable.

##  III Proposed Method

The proposed model consists of two components, i.e. two stream GAT layers
followed by LSTM layer, in order to extract the spatial and temporal
information of weather data respectively (see Fig. 2). The spatial component
of our proposed model is inspired by classical GAT architecture [21]. More
precisely, our proposed two stream GAT model exploits the graph related
spatial information, while the LSTM layer captures the temporal dependencies
between weather variables observed at different timesteps. The weather data is
first cast to a graph where the weather stations form the vertices of a graph
and each vertex contains two dimensions, i.e. weather variables and timesteps.
Here, weather variables includes wind speed, wind direction, temperature, dew
point, air pressure and rain amount. Timestep dimension contains values from
historical observations of these weather variables. In the subsequent
subsections, we first extend GAT model by equipping it with a learnable
adjacency matrix as well as incorporating 2-dimensional node features. The
later results in obtaining attention scores per weather variable. The proposed
model architecture will then be discussed.

###  III-A GAT with Learnable Adjacency Matrix

Often a graph is represented by an adjacency matrix. However, in case that the
graph structure is not known, one may learn the edge weights of the graph.
Previous studies [16], [35] and [15] made the adjacency matrix learnable which
has shown to improve the predictive performance of the model. In addition, it
can potentially provides an insights into how the features of a particular
node is influenced by the features of the other nodes.

The classical GAT model [21], assumes that the adjacency matrix of the graph
is known. Here, in contrast to the model in [21], we make the adjacency matrix
learnable. Therefore, we let the network to learn the graph spatial
connections between the nodes in an end-to-end fashion. This learned
information is later used in GAT formulation for obtaining the attention
scores. Similar to [15], the following operations are applied to the adjacency
matrix during the learning phase:

$\left\\{\begin{array}[]{l}\hat{A}=A+I,\\\
\hat{A}=\frac{\hat{A}-\hat{A}_{min}}{\hat{A}_{max}-\hat{A}_{\min}}.\end{array}\right.$
(1)

Here, the identity matrix is added to the adjacency matrix $A$ in order to
consider the possible self-loop connection to each vertex. Subsequently, min-
max normalization is applied to the updated adjacency matrix $\hat{A}$.

Figure 1: The two stream GAT layer used in our proposed model.

###  III-B 2-Dimensional GAT

In classical GAT formulation [21], the feature values of vertices are
considered to be one dimensional. Given the vertices and their one dimensional
features, GAT computes attention scores which are then used for obtaining a
new representation of each node in the graph. Following the lines of [21], we
assume that the set of node features are expressed as
${h}=\left\\{{h}_{1},{h}_{2},\ldots,{h}_{N}\right\\}$, where $N$ is the number
of nodes. ${h}_{i}\in\mathbb{R}^{d}$, where $d$ is the number of features in
each node [21]. The feature values are first linearly transformed to a pre-
selected $d^{\prime}$ number of features by means of a shared weight matrix.
Then using a self attention mechanism, attention scores of each vertex with
respect to others are calculated and included in the transformation of feature
values as a dot product. More precisely, given the $i$-th and $j$-th vertex
pair, the attention score $\alpha_{i,j}$ is computed as follows [21]:

$\alpha_{ij}=\frac{\exp\left(\text{ LeakyReLU
}\left({a}^{T}\left[{W}{h}_{i}\|{W}{h}_{j}\right]\right)\right)}{\sum_{k\in\mathcal{N}}\exp\left(\operatorname{LeakyReLU}\left({a}^{T}\left[{W}{h}_{i}\|{W}{h}_{k}\right]\right)\right)}.$
(2)

Equation (2) shows how features of the $i$-th and $j$-th vertex pair are
transformed into $d^{\prime}$ dimensions and concatenated, i.e.
$\left[{W}{h}_{i}\|{W}{h}_{j}\right]\in\mathbb{R}^{2d^{\prime}}$. Here,
${h}_{i}\in\mathbb{R}^{d}$, and ${h}_{j}\in\mathbb{R}^{d}$ refer to the
feature vectors of the $i$-th and $j$-th vertex respectively.
${W}\in\mathbb{R}^{d^{\prime}\times d}$ is the shared weight matrix which
linearly transforms features of each vertex. In GAT, the self attention
mechanism is modeled by a single layer feed-forward neural network
parameterized by ${a}\in\mathbb{R}^{2d^{\prime}}$. The attention mechanism
outputs a scalar. Next the LeakyRELU nonlinearity is applied with negative
slope of 0.2. followed by a softmax to normalize the attention score. In GAT
[21], the softmax is applied with respect to all vertices within the
neighborhood of the $j$-th vertex which is pre-determined by the adjacency
matrix of the graph. However, in our case, adjacency matrix is learned during
the training phase without imposing any constraint on the connectivity between
vertices of the graph. Therefore, in our case all nodes are connected and the
strength between the nodes are learnt through the elements of the adjacency
matrix. The linearly transformed features of vertices are multiplied with the
attention scores and then the sum is taken with respect to neighboring
vertices [21].

As previously stated, in our proposed model the nodes can have 2-dimensional
features, i.e. weather variables and timesteps. Here, we extend GAT model so
it can operate on 2-dimensional features and compute the attention scores per
weather variable. To this end, we use Eq. (2), with a $W$ of size
$\mathbb{R}^{t^{\prime}\times t}$ and ${h_{i}}$ of size $\mathbb{R}^{t\times
d}$. Here, the weight matrix $W$ is used to transform the timestep dimension.
The concatenation of ${W}{h}_{i}$ and ${W}{h}_{j}$, i.e.
$\left[{W}{h}_{i}\|{W}{h}_{j}\right]$, will result in a matrix of size
$2t^{\prime}\times d$. In contract to classical GAT formulation, here the
product between the self attention coefficients
${a}\in\mathbb{R}^{2t^{\prime}}$ and the concatenated nodes features
$\left[{W}{h}_{i}\|{W}{h}_{j}\right]$ will result in a vector of size $d$,
i.e. number of weather variables, instead of a scalar which is the case for
GAT [21]. Next, the new representation of the $i$-th vertex is computed as
follows:

$\hat{h}_{i}=\sigma\left(\sum_{j\in\mathcal{N}}{W}{h}_{j}\textrm{diag}({\alpha_{ij}^{1},...,\alpha_{ij}^{d}})\right),$
(3)

where, $\alpha_{ij}^{p}$ refers to the attention scores between $i$-th and
$j$-th nodes for the $p$-th weather variable. Similar to [21], we apply $K$
independent attention mechanisms and concatenate their results to obtain the
final representation of the nodes as follows:

$\hat{h}_{i}=\|_{k=1}^{K}\sigma\left(\sum_{j\in\mathcal{N}}{W}^{k}{h}_{j}\left[\textrm{diag}({\alpha_{ij}^{1},...,\alpha_{ij}^{d}})\right]^{k}\right).$
(4)

After obtaining $\hat{h}_{i}$, the final representation of the $i$-th node is
computed as follows [15]:

$\left\\{\begin{array}[]{l}\hat{D}_{ii}=\sum_{j}\hat{A}_{ij},\\\
\tilde{h}_{i}=\hat{h}_{i}\left(\hat{D}^{-\frac{1}{2}}\hat{A}\hat{D}^{-\frac{1}{2}}\right).\end{array}\right.$
(5)

Here, diagonal matrix $D$ is calculated based on the obtained $\hat{A}$ in Eq.
(1). Following this step, the symmetric normalization, i.e.
$\hat{D}^{-\frac{1}{2}}\hat{A}\hat{D}^{-\frac{1}{2}}$, is applied . Finally,
the features of a $i$-th node shown as $\hat{h}_{i}$ is multiplied with the
resulting transformed matrix in order to calculate the new feature
representation $\tilde{h}_{i}$. The explained GAT layers extracts spatial
information of weather data. However, one can further enhance the model
capability to capture temporal dependencies in the data and that how different
observations at different timesteps affect one another. To this end, we use
Long Short Term Memory networks [26]. After the spatial layer consisting of
two stream GAT layers, the output tensor is reshaped and fed into a LSTM
network consisting of a single number of recurrent layer.

###  III-C Proposed Architecture

The proposed model consists of two streams of GAT layers in the spatial part,
see Fig. 1. First stream functions as the default GAT formulation [21], where
each vertex has a one dimensional feature. Since the input data is originally
3 dimensional, i.e. weather variables $\times$ timestep $\times$ cities, the
timestep and weather variables dimensions are stacked as a single dimensional
vector when fed into this layer. The second stream is our proposed GAT
architecture for obtaining attention per weather variable. In particular, in
the second stream each vertex has two dimensional feature, therefore, the
structural information of the weather data is kept unchanged. The output of
each stream is a tensor after reshaping and in the end the output tensors to
GATs are merged along feature dimension before being fed into the LSTM layer.
The proposed model architecture is shown in Fig. 2.

Figure 2: Overview of the proposed Multistream GAT model.

##  IV Numerical Experiments

Two distinct weather datasets have been used for the evaluation of the
proposed model: Netherlands and Denmark datasets. Both datasets contain hourly
observations of several weather variables, collected simultaneously from
different cities. Min-max normalization is applied to both dataset before the
training phase. The inputs are first reshaped into a tensorial format which
consist of 3 dimensions: weather variables, timesteps and cities.

###  IV-A Netherlands Dataset

The Netherlands dataset contains hourly measurements of $6$ weather variables
including wind speed, wind direction, temperature, dew point, air pressure and
rain amount. For the Netherlands, the observations are collected from
Schiphol, De Bilt, Leeuwarden, Eelde, Rotterdam, Eindhoven and Maastricht
weather stations. Each training instance has $7\times 30\times 6$ shape,
corresponding to weather variables $\times$ timesteps $\times$ cities. In our
experiments, the number of timesteps is set to 30 as it is empirically found
to be the optimal value. For this dataset, the target wind speed variable is
predicted for 2, 4, 6, 8 and 10 hours ahead. The data contains observations
from January 1, 2011 to March 29, 2020. The samples observed before January
2019 are used for training and validation, while samples from January 2019 to
March 2020 are used for testing.

###  IV-B Denmark Dataset

The Denmark data has the same 3 dimensions as the Netherlands dataset, with
different sizes for weather variables and cities. Correspondingly, a total of
four different weather variables i.e. temperature, pressure, wind speed and
wind direction are observed from five different weather stations: Aalborg,
Aarhus, Esbjerg, Odense and Roskilde. Each training instance has $5\times
30\times 4$ shape corresponding to weather variables $\times$ timesteps
$\times$ cities. For this dataset, we aim at forecasting the wind speed of
each city for 6, 12, 18 and 24 hours ahead as individual forecasting tasks.
The measurement data from 2000 to 2010 is used. The samples before 2010 are
used for training and validation, and the data in 2010 is used for testing.

###  IV-C Results and Discussion

The mean absolute error and mean squared error metrics are used to evaluate
the performance of the tested models. The formulation of the used metrics are
as follows: MAE$=\frac{\sum_{i=1}^{n}\left|y_{i}-\hat{y}_{i}\right|}{n}$ and
MSE$=\frac{\sum_{i=1}^{n}\left(y_{i}-\hat{y}_{i}\right)^{2}}{n}$. Here, the
real measurements and model predictions are denoted by $\hat{y}_{i}$ and
$\hat{y}_{i}$ respectively and $n$ is the number of instances. The performance
of the proposed model is compared with other models in [15, 13, 3]. In order
to have a fair comparison, the training/validation/test sets are identical for
all of the models. It can be observed from Tables LABEL:Tab:T1 and
LABEL:Tab:T2 that the proposed Multistream GAT model outperforms the other
tested baseline models in terms of MAE and MSE scores for both Danish and
Netherlands datasets at different timesteps. Furthermore, thanks to the
learned attention coefficients per weather variable, the Multistream GAT model
provides more detailed insights to the predictions. Although the used weather
data has spatial characteristics, it does not necessarily contain underlying
graph related edge information regarding how different cities are connected to
each other. The hidden graph related knowledge is discovered and utilized, by
means of the incorporation of learnable adjacency matrix in each of the GAT
stream layer.

(a)

(b)

Figure 3: The average of the models performance across different time steps
for each city. (a): Danish cities, (b): Dutch cities.

(a)

(b)

(c)

(d)

Figure 4: The wind speed predictions for Eelde and Roskilde. (a) and (b): 2
and 6 hours ahead prediction for Eelde. (c) and (d): 6 and 12 hours a head
prediction for Roskilde.

The classical GAT architecture offers some form of interpretability in terms
of the attention weights between different nodes. Our proposed Multistream
GAT, makes it possible to localize the attentions in a more detailed way for
every weather variable. The representation of the attention weights of four
Dutch cities can be observed in Fig. LABEL:fig3. The colormaps represent how
much wind speed of the target city is affected by the weather variables of
other cities. For instance, Fig. 3 (c) shows that the wind direction of Eelde
causes a significant influence on the wind speed of Rotterdam, while the wind
speed of Rotterdam seems to be mostly affected by the rain amount of
Eindhoven. Fig. 3 compares the performance of different models by their
average MAE scores per city. It can be observed that the proposed Multistream
GAT model achieves the best result in each city. Moreover, the comparison of
real and predicted wind speed values of Eelde and Roskilde cities for multiple
timesteps ahead are shown in Fig. 4.

The proposed multistream GAT model extracts both the spatial and temporal
information of the data, thanks to the combination of GAT and LSTM layers. In
particular, the adjusted weather variables of the cities are fed into the LSTM
network so that the temporal related information can be analyzed and taken
into consideration by the model.

It should be noted that, the graph convolutional architecture [15], which is
shown to be successful for modeling the weather data, utilizes GCNs and
temporal convolutions to deal with the spatio-temporal characteristic of the
data. However, here the combination of GAT with LSTM outperforms the previous
convolution based attempts, since the GAT allows more in depth analysis with
the help of self attention mechanisms related to the spatial information.
Furthermore, the temporal information is possibly utilized more efficiently
with the LSTM layer than the one dimensional filters present in Temporal
Convolutional Networks.

##  V Conclusion

In this paper, a novel multistream Graph Attention Network is proposed for
wind speed forecasting. We show that combining a graph based architecture with
a recurrent model gives rise to efficient information extraction with respect
to the spatio-temporal data. Thanks to the incorporation of the learnable
adjacency matrix as well as attention per weather variable, the model can
better exploit the underlying complex spatio-temporal characteristic of the
weather data. Furthermore, the visualization of the learned attention weights
per weather variable, provides a better insight into the understanding of the
data. The applicability of the proposed model is shown on two real life
historical weather data and it improved the wind speed forecasting in multiple
weather stations. The implementation of our proposed model including the
trained models are available on Github
https://github.com/doganaykas/Multistream-Graph-Attention-Networks.

## References

  * [1] P. Bauer, A. Thorpe, and G. Brunet, “The quiet revolution of numerical weather prediction,” _Nature_ , vol. 525, no. 7567, pp. 47–55, 2015.
  * [2] A. C. Lorenc, “Analysis methods for numerical weather prediction,” _Quarterly Journal of the Royal Meteorological Society_ , vol. 112, no. 474, pp. 1177–1194, 1986.
  * [3] S. Mehrkanoon, “Deep shared representation learning for weather elements forecasting,” _Knowledge-Based Systems_ , vol. 179, pp. 120–128, 2019.
  * [4] M. A. Mohandes, S. Rehman, and T. O. Halawani, “A neural networks approach for wind speed prediction,” _Renewable Energy_ , vol. 13, no. 3, pp. 345–354, 1998.
  * [5] A. M. Foley, P. G. Leahy, A. Marvuglia, and E. J. McKeogh, “Current methods and advances in forecasting of wind power generation,” _Renewable energy_ , vol. 37, no. 1, pp. 1–8, 2012.
  * [6] S. Scher and G. Messori, “Predicting weather forecast uncertainty with machine learning,” _Quarterly Journal of the Royal Meteorological Society_ , vol. 144, no. 717, pp. 2830–2841, 2018.
  * [7] D. Tukymbekov, A. Saymbetov, M. Nurgaliyev, N. Kuttybay, G. Dosymbetova, and Y. Svanbayev, “Intelligent autonomous street lighting system based on weather forecast using lstm,” _Energy_ , vol. 231, p. 120902, 2021.
  * [8] I. A. Abdellaoui and S. Mehrkanoon, “Symbolic regression for scientific discovery: an application to wind speed forecasting,” in _Accepted for publication in the Proc. of IEEE Symposium Series on Computational Intelligence (IEEE-SSCI)_ , 2021.
  * [9] J. G. Fernández, I. A. Abdellaoui, and S. Mehrkanoon, “Deep coastal sea elements forecasting using U-Net based models,” _arXiv preprint arXiv:2011.03303_ , 2020.
  * [10] J. G. Fernandez and S. Mehrkanoon, “Broad-UNet: Multi-scale feature learning for nowcasting tasks,” _Neural Networks_ , vol. 144, pp. 419–427, 2021.
  * [11] O. Bilgin, P. Mąka, T. Vergutz, and S. Mehrkanoon, “TENT: Tensorized encoder transformer for temperature forecasting,” _arXiv preprint arXiv:2106.14742_ , 2021.
  * [12] K. Trebing, T. Stańczyk, and S. Mehrkanoon, “SmaAT-UNet: Precipitation nowcasting using a small attention-unet architecture,” _Pattern Recognition Letters_ , vol. 145, pp. 178–186, 2021.
  * [13] K. Trebing and S. Mehrkanoon, “Wind speed prediction using multidimensional convolutional neural networks,” in _IEEE Symposium Series on Computational Intelligence (IEEE-SSCI)_ , 2020, pp. 713–720.
  * [14] R. Chen, X. Wang, W. Zhang, X. Zhu, A. Li, and C. Yang, “A hybrid cnn-lstm model for typhoon formation forecasting,” _Geoinformatica_ , vol. 23, no. 3, pp. 375–396, 2019.
  * [15] T. Stanczyk and S. Mehrkanoon, “Deep graph convolutional networks for wind speed prediction,” in _European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN)_ , 2021, pp. 147–152.
  * [16] T. Wilson, P.-N. Tan, and L. Luo, “A low rank weighted graph convolutional approach to weather prediction,” in _2018 IEEE International Conference on Data Mining (ICDM)_. IEEE, 2018, pp. 627–636.
  * [17] I. A. Abdellaoui and S. Mehrkanoon, “Deep multi-stations weather forecasting: explainable recurrent convolutional neural networks,” _arXiv preprint arXiv:2009.11239_ , 2020.
  * [18] S. Xingjian, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-c. Woo, “Convolutional lstm network: A machine learning approach for precipitation nowcasting,” in _Advances in neural information processing systems_ , 2015, pp. 802–810.
  * [19] R. Hrasko, A. G. Pacheco, and R. A. Krohling, “Time series prediction using restricted boltzmann machines and backpropagation,” _Procedia Computer Science_ , vol. 55, pp. 990–999, 2015.
  * [20] T. Sogabe, H. Ichikawa, K. Sakamoto, K. Yamaguchi, M. Sogabe, T. Sato, and Y. Suwa, “Optimization of decentralized renewable energy system by weather forecasting and deep machine learning techniques,” in _2016 IEEE Innovative Smart Grid Technologies-Asia (ISGT-Asia)_. IEEE, 2016, pp. 1014–1018.
  * [21] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, “Graph attention networks,” _arXiv preprint arXiv:1710.10903_ , 2017.
  * [22] P. Hewage, M. Trovati, E. Pereira, and A. Behera, “Deep learning-based effective fine-grained weather forecasting model,” _Pattern Analysis and Applications_ , vol. 24, no. 1, pp. 343–366, 2021.
  * [23] K. Zhou, Y. Zheng, B. Li, W. Dong, and X. Zhang, “Forecasting different types of convective weather: A deep learning approach,” _Journal of Meteorological Research_ , vol. 33, no. 5, pp. 797–809, 2019.
  * [24] E. R. Rodrigues, I. Oliveira, R. Cunha, and M. Netto, “Deepdownscale: a deep learning strategy for high-resolution weather forecast,” in _2018 IEEE 14th International Conference on e-Science (e-Science)_. IEEE, 2018, pp. 415–422.
  * [25] M. A. Zaytar and C. El Amrani, “Sequence to sequence weather forecasting with long short-term memory recurrent neural networks,” _International Journal of Computer Applications_ , vol. 143, no. 11, pp. 7–11, 2016.
  * [26] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” _Neural computation_ , vol. 9, no. 8, pp. 1735–1780, 1997.
  * [27] S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, “Face recognition: A convolutional neural-network approach,” _IEEE transactions on neural networks_ , vol. 8, no. 1, pp. 98–113, 1997.
  * [28] P. Hewage, A. Behera, M. Trovati, E. Pereira, M. Ghahremani, F. Palmieri, and Y. Liu, “Temporal convolutional neural (tcn) network for an effective weather forecasting using time-series data from the local weather station,” _Soft Computing_ , vol. 24, no. 21, pp. 16 453–16 482, 2020.
  * [29] M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst, “Geometric deep learning: going beyond euclidean data,” _IEEE Signal Processing Magazine_ , vol. 34, no. 4, pp. 18–42, 2017.
  * [30] K. Xu, W. Hu, J. Leskovec, and S. Jegelka, “How powerful are graph neural networks?” _arXiv preprint arXiv:1810.00826_ , 2018.
  * [31] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” _arXiv preprint arXiv:1609.02907_ , 2016.
  * [32] H. Gao, Z. Wang, and S. Ji, “Large-scale learnable graph convolutional networks,” in _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, 2018, pp. 1416–1424.
  * [33] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in _Advances in neural information processing systems_ , 2017, pp. 5998–6008.
  * [34] T. Wu, F. Chen, and Y. Wan, “Graph attention lstm network: A new model for traffic flow forecasting,” in _2018 5th International Conference on Information Science and Control Engineering (ICISCE)_. IEEE, 2018, pp. 241–245.
  * [35] L. Shi, Y. Zhang, J. Cheng, and H. Lu, “Two-stream adaptive graph convolutional networks for skeleton-based action recognition,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 12 026–12 035.