# The World as a Graph:
Improving El Niño Forecasts with Graph Neural Networks

Salva Rühling Cachay1, Emma Erickson∗2,
Arthur Fender C. Bucker∗3, 4, Ernest Pokropek∗5, Willa Potosnak∗111Contributed
equally as second authors.6,
Suyash Bire8, Salomey Osei7, and Björn Lütjens8
Manuscript received March 12, 2021. This work has been submitted to the IEEE
for possible publication. Copyright may be transferred without notice, after
which this version may no longer be accessible. _(Emma Erickson, Arthur Fender
C. Bucker, Ernest Pokropek, and Willa Potosnak contributed equally.)
Corresponding author: S. R. Cachay (email: salvaruehling@gmail.com)._
1Technical University of Darmstadt, 2University of Illinois at Urbana-
Champaign,  3University of São Paulo, 4 Technical University of Munich,
5Warsaw University of Technology,  6Duquesne University, 7African Institute
for Mathematical Sciences, 8Massachusetts Institute of Technology

###### Abstract

Deep learning-based models have recently outperformed state-of-the-art
seasonal forecasting models, such as for predicting El Niño-Southern
Oscillation (ENSO). However, current deep learning models are based on
convolutional neural networks which are difficult to interpret and can fail to
model large-scale atmospheric patterns. In comparison, graph neural networks
(GNNs) are capable of modeling large-scale spatial dependencies and are more
interpretable due to the explicit modeling of information flow through edge
connections. We propose the first application of graph neural networks to
seasonal forecasting. We design a novel graph connectivity learning module
that enables our GNN model to learn large-scale spatial interactions jointly
with the actual ENSO forecasting task. Our model, Graphiño, outperforms state-
of-the-art deep learning-based models for forecasts up to six months ahead.
Additionally, we show that our model is more interpretable as it learns
sensible connectivity structures that correlate with the ENSO anomaly pattern.

###### Index Terms:

Graph Neural Networks, Deep Learning, Seasonal Forecasting, Atmospheric
Sciences.

##  I Introduction

Figure 1: We propose Graph Neural Networks (GNNs) to forecast El Niño–Southern
Oscillation (ENSO). GNNs can extract patterns at a global scale indicative of
ENSO, contrary to CNNs, which are based on spatially local feature extractors,
i.e. grid convolutions, and assume translational equivariance. The right part
of the figure visualizes this key difference in a toy example. In this work,
the goal is forecast the ONI, which is the averaged sea surface temperature
anomalies over the ONI region (5∘N-5∘S, 120∘-170∘W) over three months. We
jointly learn a global graph connectivity structure, represented by an
adjacency matrix, with our proposed structure learning module.

El Niño–Southern Oscillation (ENSO) has a large influence on climate
variability as it causes disasters such as floods [1], droughts [2], and heavy
rains [3, 4] in various regions of the world. It also has severe implications
on public health [5, 6]. ENSO forecasts have remained at traditionally low
skill due to the high variability of ENSO manifestations and the difficulty in
capturing the global scale and complexity of the ocean-atmosphere interactions
that cause it [7]. Data-driven forecasting systems are additionally confronted
by the limited availability of observational data. Various indices exist to
measure the presence and strength of ENSO events. As in the most related work
[8], we here focus on forecasting the commonly used Oceanic Niño Index (ONI).
In a recent work, a deep learning system based on a convolutional neural
network (CNN) was successfully applied [8] to forecasting ENSO by exploiting
vast amounts of simulation data from climate models [9]. The CNN model was
indeed able to outperform state-of-the-art dynamical systems, and provide
skillful forecasts of the ONI for up to 17 months ahead. However, some of the
fundamental assumptions behind CNNs [10] are not well suited for seasonal and
long range forecasting:

  * •

Parameter-shared convolutions lead to _translational equivariance_ , meaning
that if the input is moved, its output representation will move by the same
amount. In earth science applications however, the location of a certain
pattern is very important. For example, sea surface temperature anomalies
occurring in the tropical Pacific should be treated differently from those
occurring in the north Atlantic.

  * •

CNNs build representations from spatially close regions of the input, leading
to a _spatial locality bias_. Many climate phenomena however, are driven by
global interactions. CNNs account for large-scale patterns only through deep
layers, which misses the importance of modeling predominantly large-scale
patterns.

  * •

CNNs need to use all grid cells of the input. This makes them inflexible in
cases where some regions of the input are known to not be needed and must be
masked out for the CNN (e.g., as in this work, all terrestrial locations could
be discarded when only using oceanic variables).

Therefore, we advocate for formulating the _ONI forecasting problem as a graph
regression problem, and model it with Graph Neural Networks_ (GNN) [11]. GNNs
generalize convolutions to non-Euclidean data, and thus allow us to model
large-scale global connections as edges of a graph. We visualize this key
modeling difference in Fig. 1. Furthermore, GNNs can enhance model
interpretability, given that domain knowledge can be encoded into the graph
connection structure, or, if using an adaptive graph structure, we can analyze
the learned edges. Our proposed model also requires 12 times less models than
a state-of-the-art deep learning-based approach [8], which requires a separate
model for each target season (and each number of lead months). Lastly, we note
that GNNs are more efficient than recurrent neural networks and LSTMs [12],
which are often used in ENSO forecasting models [13, 14], as well as
significantly more efficient than dynamical models, which are compute- and
resource-intensive.

Motivated by these modeling advantages that GNNs enjoy over other deep
learning architectures, we can summarize our key contributions as follows:

  * •

We propose the first application of GNNs to long range and seasonal
forecasting.

  * •

Building upon established previous research we develop and open-source
Graphiño222Code is available at https://github.com/salvaRC/Graphino: a
flexible graph convolutional network architecture for long range forecasting
applications in the climate and earth sciences.

  * •

We introduce a novel graph structure learning module, which makes our model
applicable even without a pre-defined connectivity structure.

  * •

We show that our model is competitive to state-of-the-art statistical and
dynamical ENSO forecasting systems, and _outperforms them for forecasts of up
to six months_.

  * •

We exploit our model’s interpretability, to show how it learns sensible
connections that are consistent with existing theories on ENSO dynamics
predictability.

(a) 1 lead month

(b) 3 lead months

(c) 6 lead months

(d) 9 lead months

(e) 12 lead months

(f) 23 lead months

Figure 2:  The learned world connectivity structure makes our proposed model
Graphiño more interpretable than other black-box statistical models, while
retaining a high predictive skill. To visualize the learned connectivity, we
plot the eigenvector centrality of each node as a heatmap. It measures the
influence of a node on the learned graph. Nodes with the highest importance
can be seen in or near the ONI region for 1 lead month, while becoming more
global with more lead months, as expected. Interestingly, our 23-lead model in
Fig. 2(f) achieves a high correlation skill of $0.408$, which is considerably
better than our ensemble and the main ensemble model from [8]. Please refer to
Section V-C for a more detailed discussion of the learned connectivity and its
benefits.

##  II Background

Methods to forecast ENSO can be broadly classified into dynamical and
statistical systems [15, 16, 17]. The former are based on physical
processes/climate models (e.g. atmosphere–ocean coupled models) [18, 19, 20],
while the latter are data-driven, including machine learning (ML)-based
approaches.

The presence of an ENSO event is commonly measured via the running mean over
$k$ months of sea surface temperature anomalies (SSTA) over the Oceanic Niño
Index (ONI, $k=3$) region (5N-5S, 120-170W), also known as the Niño3.4 index
region ($k=5$).

##  III Related Work

###  III-A Machine Learning for ENSO forecasting

Recently, deep learning was successfully used to forecast ENSO
1$\,\mathrm{yr}$ ahead [21] as well as with a lead time of up to
$1.5\,\mathrm{yrs}$ [8], thus out-performing state-of-the-art dynamical
methods. Both project the Oceanic Niño Index (ONI) for various lead times. The
former only use the ONI index time series as input of a temporal Convolutional
Neural Network (CNN), while the latter feed sea surface temperature (SST) and
heat content anomaly maps data to a CNN. We note that the predictive skill of
the model in [21] can be mostly attributed to the use of a denoising method
(EEMD [22]), which is contentious since the smoothing process may be
transferring information from the future (i.e. test set) to the past [23].

Most statistical methods can only predict the single-valued index, an averaged
metric over SST anomalies that does not convey zonal information. A notable
exception, makes use of an encoder-decoder approach [13]. An overview over
other machine learning methods used to project ENSO is given in [24].

###  III-B Climate networks

In climate networks [25], which stem from the field of complex networks, each
grid cell of a climate dataset is considered a network node and edges between
pairs of nodes are set up using a similarity measure. They have been used to
detect and characterize SST teleconnections [26] and to successfully forecast
the presence of ENSO $1\,\mathrm{yr}$ prior [27]. The latter exploits the
observation that, a year before an ENSO event, a large-scale cooperative mode
seems to link the equatorial Pacific corridor (“El Niño basin”) and the rest
of the Pacific ocean [27]. Our GNN approach for ENSO forecasting builds on the
climate network’s precedent of describing climate as a network of nodes
related by non-local connections.

###  III-C Graph neural networks

In the past years, GNNs have surged as a popular sub-area of research within
machine learning [28]. Interestingly, they have scarcely been used in earth
and atmospheric sciences. A few applications use them for earthquake source
detection [29], power outage prediction [30] and wind-farm power estimation
[31]. The representation of data as a graph, however, makes GNNs a very
promising candidate to learn distant relationships in ENSO forecasting. This
work is the first to explore the performance of GNNs for seasonal forecasting.
As in the CNN-based work [8], we do not explicitly model temporal
relationships in the present paper, but instead build upon the standard graph
convolutional network architecture [32]. A natural extension would therefore
be to explicitly model the temporal patterns, e.g. with spatiotemporal GNNs
that have already been extensively applied to traffic forecasting [12, 33, 34,
35, 36].

TABLE I: Our proposed GNN model, Graphiño, outperforms the deep learning-based model, CNN [8], and the dynamical model, SINTEX-F [37], in all-season correlation skill for forecasts up to six months. Our model also outperforms other dynamical models from the North-American Multi-Model Ensemble project which were shown to perform at best as good as the proposed CNN in [8]. The correlation skill was measured on the held-out GODAS test set (1984-2017) with the same setup as [8]. The skill for SINTEX-F was generously estimated from Fig. 2a in [8].  Model | $n=1$ | $n=2$ | $n=3$ | $n=4$ | $n=5$ | $n=6$ | $n=9$ | $n=12$ | $n=23$
---|---|---|---|---|---|---|---|---|---
SINTEX-F [37] | 0.895 | 0.89 | 0.84 | 0.805 | 0.78 | 0.74 | 0.62 | 0.51 | 0.315
CNN [8] | 0.9423 | 0.9158 | 0.8761 | 0.8320 | 0.7983 | 0.7616 | 0.7133 | 0.6515 | 0.2870
Graphiño | 0.9747 | 0.9461 | 0.9170 | 0.8742 | 0.8226 | 0.7800 | 0.6313 | 0.5755 | 0.3363

##  IV Methods

To see how we can map general climate datasets into a problem appropriate for
a GNN, we note that these datasets are usually gridded. Hence, the grid cells
(i.e. geographical locations) can be naturally mapped to the nodes of a GNN.
The graph’s edges, which model the flow of information between nodes, are the
main argument in favor of a GNN approach. Edges can be chosen based on domain
expertise or on edges analyzed in climate networks research, or they can be
jointly learned with the target forecasting task. The explicit modeling of
interdependencies based on domain expertise, or the GNN’s choice of meaningful
edges (e.g. well known patterns or teleconnections), greatly enhances the
model’s interpretability. In this work, we propose a novel graph structure
learning module to jointly learn the connectivity structure, and forecast the
ONI. In the following, we present the formal setup of our approach.

###  IV-A Problem formulation

Let $\mathcal{V}=\\{V_{1},V_{2},\dots,V_{N}\\}$ be the nodes of the graph
$G=(\mathcal{V},\mathcal{E})$, where each node $V_{i}$ is a grid cell of a
(flattened) climate dataset, defined by its latitude and longitude. We will
later define the set of edges, $\mathcal{E}$. For each time step,
$t\in\\{1,\dots,T\\}$, we associate with each such location a feature vector
$\mathbf{V}_{i}^{(t)}\in\operatorname{\mathbb{R}}^{D}$ of $D$ climate
variables. For time step $t$ let
$\mathbf{X}_{t}=\\{\mathbf{V}_{1}^{(t)},\dots,\mathbf{V}_{N}^{(t)}\\}\in\operatorname{\mathbb{R}}^{N\times
D}$ be a snapshot measurement over all locations. Given a temporal sequence of
such climate measurements
$\mathbf{X}=\\{\mathbf{X}_{t_{1}},\dots,\mathbf{X}_{t_{w}}\\}\in\mathcal{X}$,
our goal is to forecast the ONI index
$Y=Y_{t_{w+h}}\in\operatorname{\mathbb{R}}$ with $h$ months of anticipation
and the window size, $w$, i.e. the number of timesteps used for prediction. In
this paper, we do not explicitly model the temporal component, and instead
simply concatenate the $w$ temporal sequence elements
$\mathbf{X}_{t_{i}}\in\operatorname{\mathbb{R}}^{N\times D}$ to a single
representation matrix $\mathbf{X}\in\operatorname{\mathbb{R}}^{N\times wD}$.

Our dataset then consists of a timeseries of such pairs $(\mathbf{X},Y)$ and
our goal is to learn a neural network model
$f_{\phi}:\mathcal{X}\rightarrow\operatorname{\mathbb{R}}$ , parameterized by
$\phi$. We do so by minimizing an appropriate loss function, here the mean
squared error between the predicted and true ONI index. We note that simple,
but promising, extensions to this basic setup include multi-step forecasting,
and forecasting of multiple values (e.g. multiple zonal SSTAs).

To finalize our graph’s definition, we also need to define the set of edges,
$\mathcal{E}$, that encodes the connectivity structure between geographical
locations. This can be done through an adjacency matrix
$\mathbf{A}\in\\{0,1\\}^{N\times N}$, where $\mathbf{A}_{i,j}$ equals $1$ when
there exists an edge from node $V_{i}$ to node $V_{j}$, and 0 otherwise. To
mirror grid-convolutions from CNNs we could choose them according to
geographical neighborhood, i.e. by connecting adjacent grid cells of the
climate dataset. This would, however, seriously limit the predictive power of
our model, as we show in Section V-D, since ENSO is inherently a large-scale
phenomenon. In this paper, we choose to view the edge structure as learnable,
jointly with the model’s parameters $\phi$. This makes it possible to inspect
the learned adjacency matrix during, and after model training, to validate
whether it is sensible. We now introduce our structure learning component in
detail.

###  IV-B Graph structure learning module

We propose a directed, end-to-end learnable edge structure represented by a
continuous adjacency matrix, $\mathbf{A}\in[0,1]^{N\times N}$, where each
entry, $\mathbf{A}_{i,j}$, can be now interpreted as a weighted connection
from node $V_{i}$ to node $V_{j}$. In the following we use tildes for
variables only occurring in our structure learning module (e.g.
$\tilde{\mathbf{X}}$), that should not be confused with similar, but unrelated
counterparts without tilde used by the GCN module (e.g. $\mathbf{X}$).
Concretely, we make use of static node representations,
$\tilde{\mathbf{X}}\in\operatorname{\mathbb{R}}^{N\times\tilde{d}_{1}}$, to
compute

$\displaystyle\mathbf{M}_{1}$
$\displaystyle=\text{tanh}\left(\alpha_{1}\tilde{\mathbf{X}}\tilde{\mathbf{W}}_{1}\right)\in\operatorname{\mathbb{R}}^{N\times\tilde{d}_{2}},$
(1) $\displaystyle\mathbf{M}_{2}$
$\displaystyle=\text{tanh}\left(\alpha_{1}\tilde{\mathbf{X}}\tilde{\mathbf{W}}_{2}\right)\in\operatorname{\mathbb{R}}^{N\times\tilde{d}_{2}},$
(2) $\displaystyle\mathbf{A}$
$\displaystyle=\text{sigmoid}\left(\alpha_{2}\mathbf{M}_{1}\mathbf{M}_{2}^{T}\right)\in[0,1]^{N\times
N},$ (3)

where
$\tilde{\mathbf{W}}_{1},\tilde{\mathbf{W}}_{2}\in\operatorname{\mathbb{R}}^{\tilde{d}_{1}\times\tilde{d}_{2}}$
are learnable parameters, and $\alpha_{1}$ and $\alpha_{2}$ are
hyperparameters. The smaller $\alpha_{1}$ is, the more distinct values are
generated, while a high $\alpha_{2}>1$ leads to more confident scores on
whether there is an edge or not (i.e. it discourages values close to $0.5$).
In a final step, we set all but the largest $e$ edge values $\mathbf{A}_{i,j}$
to $0$ in order to enforce a sparse connectivity structure. This module is
inspired by the unidirectional module from [33]. The key differences are 1) We
allow for bi-directional edges, while uni-directional edges are enforced in
[33]; 2) We set an upper limit, $e$, on the _total_ number of edges, instead
of fixing a maximum number of neighbors for each node. Note that while these
differences are subtle, they are absolutely key for a better performance, and
it lends itself for a better interpretation of the learned connectivity
structure. We discuss these advantages in more detail in Section V-D. As is
standard practice, we add self-loops to the graph, so as to preserve node
information, by letting the diagonal of $\mathbf{A}$ be non-zero.

###  IV-C Graph Neural Network

The problem of forecasting the ONI can be framed as a graph regression
problem. As such, any GNN can be used to model this task. For this work, we
build upon the popular graph convolutional neural network (GCN) architecture
[32]. At each GCN layer $l$, node embeddings $\mathbf{Z}^{(l)}_{i}$ are
generated for each node $V_{i}$. The node embeddings $\mathbf{Z}^{(l)}_{i}$
for node $V_{i}$ are aggregated from the previous-layer embeddings of its
neighbors: $\\{\mathbf{Z}^{(l-1)}_{j}:A_{ij}\neq 0\\}$. Thus, in the deeper
layers, information from more distant nodes propagates to each node embedding.
This process is therefore also called message-passing. Mathematically the node
embeddings $\mathbf{Z}^{(l)}$ of the $l$-th graph convolutional layer can be
written as:

$\mathbf{Z}^{(l)}=\sigma\left(\mathbf{A}\mathbf{Z}^{(l-1)}\mathbf{W}^{(l)}\right)\in\operatorname{\mathbb{R}}^{N\times
D_{l}},$ (4)

where $\mathbf{Z}^{(l-1)}\in\operatorname{\mathbb{R}}^{N\times D_{l-1}}$ are
the node embeddings of the previous layer (with the first layer,
$\mathbf{Z}^{(0)}=\mathbf{X})$, and the activation function, $\sigma$.

In the standard case with a discrete $A_{i,j}\in\\{0,1\\}$ the aggregation
$\mathbf{A}\mathbf{Z}^{(l-1)}$ is a simple sum, while with our continuous
graph structure learning formulation, $A_{i,j}\in[0,1]$, it becomes a weighted
sum.

We then use the output of the last layer $L$ to pool a graph embedding
$\mathbf{g}\in\operatorname{\mathbb{R}}^{D_{L}}$ by aggregating the node
embeddings:

$\mathbf{g}=\text{Aggregate}\left(\mathbf{Z}^{(L)}_{1},\dots,\mathbf{Z}^{(L)}_{N}\right),$
(5)

where the aggregation function can be, e.g., a mean or sum over the node
embeddings. In a final step, the graph representation $\mathbf{g}$ is used as
input to a fully connected multi-layer perceptron, to forecast our estimate of
the ONI index: $\hat{Y}=\text{MLP}\left(\mathbf{g}\right)$.

In practice, we use a more complex formulation with jumping knowledge and
residual connections, which have been shown to increase performance in a wide
variety of tasks [38]. Jumping knowledge connections, means that the node
embeddings from the intermediate GCN layers are concatenated to the final one
from layer $L$ [38]. Please refer to the Appendix for the full mathematical
formulation. Further, while the original graph convolution normalizes the
input node embeddings by the in-degree, we have found that replacing it with
batch normalization over the feature dimension gives better results.

(a) 1 lead months, RMSE $=0.222$

(b) 3 lead months, RMSE $=0.375$

(c) 6 lead months, RMSE $=0.576$

Figure 3: Our proposed graph neural network (orange) accurately predicts the
Oceanic Niño index (ONI) timeseries (blue). The high correlation skill of
$0.97,0.92$ and $0.78$ for the 1-, 3- and 6-month lead times confirms the
accurate prediction of the trend. While the predictions are strong for
standard ENSO events, the GNN underpredicts the strength of the record-year el
Niño events in 1998 and 2016.

Figure 4: Graphiño (red) outperforms all statistical models (top) and most
dynamical models (bottom) in the prediction of the extreme El Niño year, 2015.
The plume plot shows the ONI forecasts issued on January 2015. The entries in
legends are sorted by the correlation coefficient $\rho$ with the first record
being the best one (highest $\rho$). The lines correspond to the predicted ONI
for various seasons, e.g., JFM being the running average of SST anomalies in
the ONI region for January, February and March. Dynamical and statistical
model predictions were provided by the International Research Institute for
Climate and Society (IRI) [39], and the observations by the National Oceanic
and Atmospheric Administration (NOAA) [40].

##  V Experiments

###  V-A Experimental setup

To validate the predictive skill of our GNN Graphiño, we benchmark it on the
same setup and in a fair comparison against the CNN and dynamical models from
[8]. That is, as training set we use the same SODA reanalysis dataset
(1871-1973), and climate model simulations from CMIP5, that were used in [8].
The augmentation of the dataset with potentially noisy simulations is
important in order to have a sample size suitable for deep learning methods
[8] (in total $30k$ samples, while SODA only has $1200$ samples). The GODAS
dataset for the period of 1984 to 2017 serves as the held-out test set. The
datasets are used in a resolution of $5$ degrees, and only the geographical
locations that lie within $55S-60N$ and $0-360W$ are used. This results in
$N=1345$ nodes (after filtering out all terrestrial ones). The features for
each node are the sea surface temperature (SST) and heat content anomalies
over $w=3$ months (i.e., $D=2,D_{0}=2\times 3=6$). The prediction target is
the ONI index for $h$ months ahead. This is the exact same setup from [8].

Differently than the CNN in [8] however, we

  * •

only run a single model for a given number of target lead months, whereas they
run a separate model for each target season (i.e. one for DJF, another one for
JFM, etc.). This makes our approach require 12 times fewer models.

  * •

do not use the transfer learning technique, but instead train in a single
training process both on CMIP5 simulations and the SODA dataset, since we
observed massive overfitting to the small SODA dataset otherwise.

Also, note that our GNN is a more natural representation for the task, since
we can simply filter out all terrestrial locations, whereas the inflexible CNN
grid structure requires them to be present (with all features equal to zero).
Just as in [8], we report the performance of an ensemble of size four. For
more details, please refer to Section V-E.

###  V-B Results

####  V-B1 Average prediction

We find that our proposed GNN model outperforms the state-of-the-art CNN of
[8] for up to 6 lead months, as well as the competitive dynamical model
SINTEX-F [37] for all lead times, see Table I. Our model is able to
significantly advance the state-of-the-art of machine learning for ENSO models
in this seasonal forecasting range of up to six months. We hypothesize that
the more rapid decrease in performance of our model for more than six lead
months compared to the CNN model, can be attributed to the fact that our model
also needs to learn the connectivity structure. This potentially leads to a
higher sample complexity and makes the model more prone to overfitting. The
ONI time series forecasted by our model, for $n=1,3,6$ lead months, are
plotted in Fig. 3.

Interestingly, we found that one of our ensemble members achieves a
correlation coefficient of $0.408$ for $23$ lead months, which is very high
given that the skill of both, our ensemble and the one from [8], only achieve
$0.34$ and $0.29$ respectively. This indicates, that further research could
potentially achieve skillful multi-year forecasts. We therefore include the
learned connectivity structure of this model in Fig. 2(f), as we believe that
it may be of interest to the community.

####  V-B2 Rare event prediction

Fig. 4 compares Graphiño to established selected statistical and dynamical
models predicting ENSO events for year 2015, during which occurred one of the
strongest El Niño in past decades [41]. Graphiño considerably outperforms all
of the statistical, and a vast majority of the dynamical models, being notably
better than the average of both of the groups, achieving a correlation
coefficient of 0.948 across the year.

###  V-C Analysis of the connectivity structure learned by the GNN

Recall that in our experiments, we do not pre-define any edges between the
nodes. Thus, the GNN model needs to learn suitable edges in addition to the
actual forecasting task. Note that while we associate each node with its
geographical location defined by its latitude and longitude, the GCN model has
no notion of spatiality. That being said, we now analyze the learned
connections and demonstrate their reasonableness, which emphasizes our
argument that we can encode powerful and interpretable inductive biases into
GNNs for seasonal forecasting.

####  V-C1 Eigenvector centrality

Since the number of learned edges and nodes is too large to explicitly plot
every learned connection in an informative way, we instead choose to analyze
the assigned importance of each node, as measured by their eigenvector
centrality. This quantity measures the influence of a node on the graph – a
node with a high centrality score means that it is connected to many other
nodes with high scores, and therefore plays a central role in the graph. In
mathematical terms, the centrality score of a node, $i\in\\{1,\dots,N\\}$, is
the $i$-th element of the eigenvector $\mathbf{v}\in R_{+}^{N}$ that
corresponds to the largest eigenvalue $\lambda_{\text{max}}$ of the adjacency
matrix. That is, it satisfies $\mathbf{Av}=\lambda_{\text{max}}\mathbf{v}$
[42].

####  V-C2 Interpretation of learned edges in terms of ENSO models

In Fig. 2 we plot heatmaps of the computed eigenvector centrality scores of
each node of our best performing model for lead times
$h\in\\{1,3,6,9,12,23\\}$. These show which nodes (in darker color) play a
central role in the graph and are connected to other central nodes. High
eigenvector centrality scores translates to having a high influence in the GCN
since the node’s information will spread more during message passing.

In order to interpret the spatial features of the edges in terms of physical
processes we first need to understand the basic physics behind ENSO. El Niño
or La Niña events are characterized by the Bjerknes feedback, which is a
positive feedback between SST and wind anomalies [43]. Wind tends to rush
towards (away from) regions of warm (cold) SST which pushes even more warm
water towards (away from) that region. This further intensifies the wind,
which further intensifies the SST anomaly. During EL Niño (La Niña) events
warm (cold) SST anomalies form over the eastern Pacific thereby intensifying
the wind towards the eastern Pacific. If the Bjerknes feedback is not
interrupted, the eastern Pacific would keep on getting warm (cold). The
theories of ENSO attempt to explain the processes that interrupt the feedback
and cause transition from one phase to another. The most popular theories
imagine the tropical Pacific as a self-sustaining oscillator. The Rossby and
Kelvin waves [44, 45], two standard modes of variability observed in the
ocean, play a crucial role in these models. The Rossby (Kelvin) wave
propagates westward (eastward). Moreover both can either deepen or shoal the
thermocline, which can be considered to be the boundary between the warm
surface water and cold deeper water, thereby increasing or lowering the heat
content in the ocean. These are called downwelling and upwelling waves,
respectively.

The delayed oscillator mechanism [46] suggests that during a neutral phase of
ENSO an anomalous eastward wind burst could set off an eastward downwelling
Kelvin wave and a westward upwelling Rossby wave. The eastward Kelvin wave,
the speed of which is well-known, carries the warm water from the western
Pacific warm pool towards the east thereby heralding the onset of El Niño.
Similarly, an anomalous westward windburst triggers upwelling Kelvin and
downwelling Rossby waves leading to a La Niña in the winter. The Kelvin waves
are generally set off in the boreal spring and reach the eastern Pacific by
the winter of the same year. Thus, most of our short term predictability, for
example Fig. 2(b) and Fig. 2(c), can be explained by the propagation of these
waves. Once these waves are set off in the spring it becomes highly likely
that an El Niño or a La Niña event would occur in the winter. Dynamical models
also do a good job of predicting the El Niño/La Niña once these waves are
triggered. However, they struggle to predict whether these waves would be
triggered in any given year leading to the spring predictability barrier [47]
(Fig. 4).

Yet another theory, the western Pacific oscillator [48], predicts that the SST
and thermocline anomalies in the western Pacific warm pool trigger anomalous
winds in the western Pacific setting off Kelvin waves towards the east. The
SST and sea level pressure anomalies preceding the spring in the Niño5
($120^{\circ}–140^{\circ}$E, $5^{\circ}S–5^{\circ}$N) and Niño6
($140^{\circ}–160^{\circ}$E, $8^{\circ}–16^{\circ}$N) region trigger eastward
wind anomalies in the western Pacific. These wind anomalies are deemed
important according to this theory. Significant 9-month lead predictability of
our model seems to originate from this region (Fig. 2(d)) which shows that
this mechanism provides a source of predictability for our model even before
the spring.

The recharge-discharge oscillator [49] is yet another theory which suggests
that during the warm phase anomalous eastward wind drives poleward transport
of warm water thereby shoaling the thermocline and reducing the equatorial
ocean heat content in the tropical Pacific. This leads to a transition to the
cold phase, which is associated with anomalous westward winds. The westward
winds then drive warm water towards the tropical Pacific performing what is
known as the recharge phase. The signature of the discharge (recharge) process
where warm water is transported away from (towards) the equator is evident in
figures Fig. 2(d) and Fig. 2(e).

For two year lead in figure Fig. 2(f) we see hotspots of connectivity in the
tropical Indian and Atlantic oceans. Warm SST over Indian and Atlantic ocean
strengthens the trade winds from the Paicific thereby leading to weak El Niños
or prolonged La Niña-like conditions [50, 51, 52]. More recently, a unified
oscillator theory has been proposed [53, 54] which merges the effects of all
the above mentioned oscillators and suggests that all of them play a role in
ENSO dynamics to varying extents. It is then encouraging to see that our GNN
model is able to decipher patterns of variability that can be linked to
multiple known theories of ENSO.

###  V-D Connectivity structure ablation

To validate the advantage of modeling distant interdependencies in our GNN
with help of our proposed connectivity structure learning module, we run the
same main experiments but with a fixed adjacency matrix based on geographical
neighborhood. Concretely, each node is connected to all nodes within a radius
of $5$ degrees, i.e. a center node has $8$ neighbors. We then run the same GNN
as in the main experiments, but without the structure learning module. As
expected, we find that incorporating information from distant parts of the
world is critical. Our flexible GNN that can learn an arbitrary connectivity
structure considerably outperforms the same GNN provided with a fixed, local
connectivity structure only, see Table II.

Furthermore, we find that our proposed structure learning module significantly
outperforms the structure learning module proposed in [33]. This can be
attributed to the more appropriate inductive biases imposed by our proposed
method. Concretely, [33] enforces uni-directional connections, whereas we give
the module the freedom to learn arbitrary connections. Importantly, they
enforce each node of the $N$ nodes to have $k$ neighbors/connections, whereas
we only impose an upper limit on the total number of connections (e.g. $kN$).
This is a better inductive bias for seasonal forecasting since we expect that
some nodes (e.g. around the ONI region) will be considerably more important
than others. Therefore, such nodes should be more connected than other nodes
(i.e. should play a more central role in the GNN message passing) whose
information is less/not important for the downstream ONI forecasting task.

TABLE II: Incorporating geographical distant information is key for a strong performance. We report the correlation skill for $n$ lead months of the same GNN with 1) our structure learning module, 2) the structure learning module from [33], and 3) a fixed, local connectivity structure with edges based on spatial proximity (local). Edge structure | $n=1$ | $n=3$ | $n=6$ | $n=9$
---|---|---|---|---
Local | 0.9063 | 0.7752 | 0.5946 | 0.4586
[33] | 0.9117 | 0.8503 | 0.6439 | 0.4190
Graphiño | 0.9747 | 0.9170 | 0.7800 | 0.6313

###  V-E Implementation details

As in [8], we report the predictive skill of an ensemble of four models. Two
of them are 2-layer GCNs with layer sizes of $250\times 100$ and $250\times
250$ . The other two are 3-layer GCNs with mean and sum pooling concatenated
as the output of the graph representation. The dimensions are $200\times
200\times 200$, and $250\times 250\times 250$ respectively. To avoid
overfitting with the larger, more complex 3-layer GCNs, we apply a L2-weight
decay of $10^{-4}$, and $10^{-3}$ respectively, while the 2-layer GCNs are
trained with a L2 weight decay of $10^{-6}$ only. Note that graph networks
often perform best with few layers, differently than, e.g., CNNs. All of the
GCNs are followed by a two-layer MLP. For both networks we use the ELU
activation function [55]. The batch size is $64$, and we use SGD with a
learning rate of $0.005$ and Nesterov momentum of $0.9$ as the optimizer. We
do not use neither a learning rate scheduler nor dropout. We report the held-
out test performance on GODAS of the last checkpoint after 50 epochs of
training. As indicated before, we found that batch normalization over the
feature/embedding dimension gave better results than the standard in-degree
normalization proposed in [32] for the GCN. The MLP uses batch normalization
too. We set the static node representations, $\tilde{\mathbf{X}}$ to be equal
to the SODA timeseries of SST and heat content anomalies, plus the latitude
and longitudes of the nodes. Note that when no static representations are
available, $\tilde{\mathbf{X}}$ can be learnable embeddings as in [33]. To
mimic an average number of neighbors equal to the number of adjacent cells
used in a 3x3 CNN filter, we choose a maximum number of edges $e=8N$. We set
$\alpha_{1}=0.1$ and $\alpha_{2}=2$. Since Graphiño is flexible enough to
support additional nodes, we add an ONI node represented by the averaged out
SST and heat content anomalies over the ONI region for each time step. In each
optimization step (i.e. for each batch), we retrieve the adjacency matrix
$\mathbf{A}$ from our structure learning module. It is then used in the
following GNN forward pass. We then jointly optimize for the parameters $\phi$
of our main model $f_{\phi}$, and the structure learning parameters
$\tilde{\mathbf{W}}_{1},\tilde{\mathbf{W}}_{2}$, based on the mean squared
error between the predicted $\hat{Y}=f_{\phi}(\mathbf{A},\mathbf{X})$ and true
ONI $Y$. Our 2-layer models run at around $40s/$epoch, while the 3-layer
models require around $60s/$epoch on a 4xGeForce GTX 1080Ti GPU. The total
training time is $33$ and $50$ min respectively. After the training,
inferences are very fast, as is usual with neural networks.

##  VI Discussion

Our proposed GNN approach for forecasting ENSO outperforms dynamical models
like the SINTEX-F [37] (in [8], the proposed CNN is also shown to outperform
the North American Multi-Model Ensemble members), and is better or comparable
to state-of-the-art machine learning models [8]. Furthermore, our methodology
outperforms the aforementioned study for seasonal forecasts, indicating the
potential for improved model performance for longer leads multi-year forecasts
with the inclusion of more variables, or an extensive hyperparameter search.

Our proposed approach is easily applicable to other important complex weather
and climate forecasting problems. Besides an increased predictive skill with
our model, we expect that the connectivity structure encoded into our model,
that is nicely interpretable in earth sciences applications, will be just as,
if not more, valuable to the community. Lastly, we believe that a very
promising direction is to further improve the graph structure learning module,
and use the learned connectivity structure to advance our current
understanding of the predictability of ENSO. Application of this ENSO model
could have a significant impact on weather prediction and human preparation if
leveraged as a tool by climate researchers to provide better as well as longer
lead-time predictions. This would also allow global populations to better
prepare for the predicted climate and its effects on industry, agriculture,
safety, and human quality of life.

A limitation of our model is the underestimation of the extreme ENSO events,
as can be seen in Figure 3. By definition, these events are rare, which makes
it a hard task for an ML model to correctly predict them from the limited
sample size. A promising research direction may therefore focus on skillfully
forecasting these extreme ENSOs, e.g supported by a custom loss function.

##  VII Conclusion

Our proposed Graphiño model, based on a GNN architecture and a graph
connectivity learning module, outperforms state-of-the-art ENSO forecasting
methods for up to 6 months lead time. Our work shows promising results for the
use of a GNN architecture for ENSO forecasting and other atmospheric modeling
purposes, while also enhancing the ML model interpretability. Future work
using other relevant climate variables with GNNs and better architectures,
e.g. that explicitly model the temporal axis, could further improve
forecasting results as well as provide novel information regarding the
relationship between global regions as represented by the learned connectivity
structure.

March 12, 2021

##  Appendix A Full mathematical formulation

In this section we formally define the graph convolutional network (GCN) model
in the full form that we use, with jumping knowledge and residual connections.
Both of these methods aim at increasing the quality of the final node
representations, and are motivated by the _over-smoothing_ issue to which GNNs
are sensitive to. Over-smoothing refers to node representations becoming
increasingly similar with the number of message passing iterations (i.e.
layers in our GCN) [11].

Recall that the generated node embeddings $\mathbf{Z}^{(l)}$ of the $l$-th
graph convolutional layer can be written as:

$\mathbf{Z}^{(l)}=\sigma\left(\mathbf{A}\mathbf{Z}^{(l-1)}\mathbf{W}^{(l)}\right)\in\operatorname{\mathbb{R}}^{N\times
D_{l}},$ (6)

where $\mathbf{Z}^{(l-1)}\in\operatorname{\mathbb{R}}^{N\times D_{l-1}}$ are
the node embeddings of the previous layer, $\mathbf{Z}^{(0)}=\mathbf{X}$, and
$\sigma$ is an activation function. If we add a _residual connection_ to layer
$l$, provided that $D_{l}=D_{l-1}$, this becomes:

$\mathbf{Z}^{(l)}=\sigma\left(\mathbf{A}\mathbf{Z}^{(l-1)}\mathbf{W}^{(l)}\right)+\mathbf{Z}^{(l-1)}.$
(7)

Recall that in the standard GCN setting, the final representation
$\mathbf{Z}_{i}$ of each node $i$ is simply its node embedding
$\mathbf{Z}^{(L)}_{i}\in\operatorname{\mathbb{R}}^{D_{L}}$ at the last layer
$L$. _Jumping knowledge_ connections additionally incorporate the embeddings
from intermediate layers [38]. That is, the final node embedding becomes the
concatenation of the outputs of all layers:

$\mathbf{Z}_{i}=[{\mathbf{Z}_{i}^{(1)}}^{T},\dots,{\mathbf{Z}^{(L)}_{i}}^{T}]\in\operatorname{\mathbb{R}}^{d},$
(8)

where $d=\sum_{l=1}^{L}D_{l}$. We then use this final node representations to
pool a graph embedding $\mathbf{g}\in\operatorname{\mathbb{R}}^{d}$ by
aggregating them:

$\mathbf{g}=\text{Aggregate}\left(\mathbf{Z}_{1},\dots,\mathbf{Z}_{N}\right).$
(9)

The aggregation function can be a simple average, that is:

$\mathbf{g}=\frac{1}{N}\sum_{i=1}^{N}\mathbf{Z}_{i}.$ (10)

We indeed use this approach for two of our ensemble members, while for the
other two we additionally concatenate the sum over the node representations:

$\mathbf{g}=\left[\sum_{i=1}^{N}\mathbf{Z}_{i},\frac{1}{N}\sum_{i=1}^{N}\mathbf{Z}_{i}\right]\in\operatorname{\mathbb{R}}^{2d}.$
(11)

Other possible aggregation functions can be a $\max(\cdot)$, or an attention
mechanism. The best such aggregation function often varies across the specific
applications and datasets. Indeed, while we found that the simple mean gives
consistently solid results, a more extensive study on the most appropriate
graph pooling approaches for ENSO and seasonal forecasting is required.

## Acknowledgment

We would like to thank the ProjectX organizing committee for motivating this
work. We gratefully acknowledge the computational support by the Microsoft AI
for Earth Grant. We would also like to thank Captain John Radovan for sharing
his expertise regarding the current ENSO models and their global applications,
Chris Hill for his guidance on machine learning for oceans, as well as Chen
Wang for his guidance on GNN architecture.

The research was partially sponsored by the United States Air Force Research
Laboratory and the United States Air Force Artificial Intelligence Accelerator
and was accomplished under Cooperative Agreement Number FA8750-19-2-1000. The
views and conclusions contained in this document are those of the authors and
should not be interpreted as representing the official policies, either
expressed or implied, of the United States Air Force or the U.S. Government.
The U.S. Government is authorized to reproduce and distribute reprints for
Government purposes notwithstanding any copyright notation herein.

## References

  * [1] P. J. Ward, B. Jongman, M. Kummu, M. D. Dettinger, F. C. S. Weiland, and H. C. Winsemius, “Strong influence of el niño southern oscillation on flood risk around the world,” _Proceedings of the National Academy of Sciences_ , vol. 111, no. 44, pp. 15 659–15 664, 2014.
  * [2] F. Siegert, G. Ruecker, A. Hinrichs, and A. Hoffmann, “Increased damage from fires in logged forests during droughts caused by el nino,” _Nature_ , vol. 414, no. 6862, pp. 437–440, 2001.
  * [3] J. P. Donnelly and J. D. Woodruff, “Intense hurricane activity over the past 5,000 years controlled by el niño and the west african monsoon,” _Nature_ , vol. 447, no. 7143, pp. 465–468, 2007.
  * [4] C. F. Ropelewski and M. S. Halpert, “Global and regional scale precipitation patterns associated with the el niño/southern oscillation,” _Monthly weather review_ , vol. 115, no. 8, pp. 1606–1626, 1987.
  * [5] R. S. Kovats, M. J. Bouma, S. Hajat, E. Worrall, and A. Haines, “El niño and health,” _The Lancet_ , vol. 362, no. 9394, pp. 1481–1489, 2003.
  * [6] J. A. Patz, D. Campbell-Lendrum, T. Holloway, and J. A. Foley, “Impact of regional climate change on human health,” _Nature_ , vol. 438, no. 7066, pp. 310–317, 2005.
  * [7] A. G. Barnston, M. K. Tippett, M. L. L’Heureux, S. Li, and D. G. DeWitt, “Skill of Real-Time Seasonal ENSO Model Predictions during 2002–11: Is Our Capability Increasing?” _Bulletin of the American Meteorological Society_ , vol. 93, no. 5, pp. 631–651, 05 2012.
  * [8] Y.-G. Ham, J.-H. Kim, and J.-J. Luo, “Deep learning for multi-year enso forecasts,” _Nature_ , vol. 573, pp. 568–572, 9 2019.
  * [9] K. E. Taylor, R. J. Stouffer, and G. A. Meehl, “An overview of cmip5 and the experiment design,” _Bulletin of the American meteorological Society_ , vol. 93, no. 4, pp. 485–498, 2012.
  * [10] I. Goodfellow, Y. Bengio, and A. Courville, _Deep learning_ , 2016.
  * [11] W. L. Hamilton, “Graph representation learning,” _Synthesis Lectures on Artifical Intelligence and Machine Learning_ , vol. 14, no. 3, pp. 1–159, 2020\.
  * [12] B. Yu, H. Yin, and Z. Zhu, “Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting,” in _Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18_ , 7 2018, pp. 3634–3640. [Online]. Available: https://doi.org/10.24963/ijcai.2018/505
  * [13] D. He, P. Lin, H. Liu, L. Ding, and J. Jiang, “Dlenso: A deep learning enso forecasting model,” in _PRICAI_ , 2019.
  * [14] C. Broni-Bedaiko, F. A. Katsriku, T. Unemi, M. Atsumi, J.-D. Abdulai, N. Shinomiya, and E. Owusu, “El niño-southern oscillation forecasting using complex networks analysis of lstm neural networks,” _Artificial Life and Robotics_ , vol. 24, no. 4, pp. 445–451, 2019.
  * [15] C. Wang, C. Deser, J.-Y. Yu, P. DiNezio, and A. Clement, _El Niño and Southern Oscillation (ENSO): A Review_. Dordrecht: Springer Netherlands, 2017, pp. 85–106.
  * [16] D. Chen, M. A. Cane, A. Kaplan, S. E. Zebiak, and D. Huang, “Predictability of el niño over the past 148 years,” _Nature_ , vol. 428, no. 6984, p. 733—736, April 2004.
  * [17] A. V. Fedorov, S. L. Harper, S. G. Philander, B. Winter, and A. Wittenberg, “How Predictable is El Niño?” _Bulletin of the American Meteorological Society_ , vol. 84, no. 7, pp. 911–920, 07 2003. [Online]. Available: https://doi.org/10.1175/BAMS-84-7-911
  * [18] S. E. Zebiak and M. A. Cane, “A model el nino–southern oscillation,” _Monthly Weather Review_ , vol. 115, no. 10, pp. 2262 – 2278, 01 Oct. 1987\.
  * [19] T. N. Palmer, A. Alessandri, U. Andersen, P. Cantelaube, M. Davey, P. Delécluse, M. Déqué, E. Diez, F. J. Doblas-Reyes, H. Feddersen _et al._ , “Development of a european multimodel ensemble system for seasonal-to-interannual prediction (demeter),” _Bulletin of the American Meteorological Society_ , vol. 85, no. 6, pp. 853–872, 2004.
  * [20] S. Saha, S. Moorthi, H.-L. Pan, X. Wu, J. Wang, S. Nadiga, P. Tripp, R. Kistler, J. Woollen, D. Behringer _et al._ , “The ncep climate forecast system reanalysis,” _Bulletin of the American Meteorological Society_ , vol. 91, no. 8, pp. 1015–1058, 2010.
  * [21] J. Yan, L. Mu, L. Wang, R. Ranjan, and A. Y. Zomaya, “temporal convolutional networks for the advance prediction of enso,” _Scientific Reports_ , vol. 10, no. 1, pp. 1–15, 2020.
  * [22] Z. Wu and N. E. Huang, “Ensemble empirical mode decomposition: a noise-assisted data analysis method,” _Advances in adaptive data analysis_ , vol. 1, no. 01, pp. 1–41, 2009.
  * [23] Y. Guo, X. Cao, B. Liu, and K. Peng, “El niño index prediction using deep learning with ensemble empirical mode decomposition,” _Symmetry_ , vol. 12, no. 6, p. 893, 2020.
  * [24] H. Dijkstra, E. Hernandez-Garcia, C. Lopez _et al._ , “The application of machine learning techniques to improve el nino prediction skill,” _Frontiers in Physics_ , vol. 7, p. 153, 2019.
  * [25] A. A. Tsonis, K. L. Swanson, and P. J. Roebber, “What Do Networks Have to Do with Climate?” _Bulletin of the American Meteorological Society_ , vol. 87, no. 5, pp. 585–596, 05 2006. [Online]. Available: https://doi.org/10.1175/BAMS-87-5-585
  * [26] A. Agarwal, L. Caesar, N. Marwan, R. Maheswaran, B. Merz, and J. Kurths, “Network-based identification and characterization of teleconnections on different scales,” _Scientific Reports_ , vol. 9, no. 1, pp. 1–12, 2019\.
  * [27] J. Ludescher, A. Gozolchiani, M. I. Bogachev, A. Bunde, S. Havlin, and H. J. Schellnhuber, “Very early warning of next el niño,” _Proceedings of the National Academy of Sciences_ , vol. 111, no. 6, pp. 2064–2066, 2014. [Online]. Available: https://www.pnas.org/content/111/6/2064
  * [28] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, “A comprehensive survey on graph neural networks,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2020.
  * [29] M. P. van den Ende and J.-P. Ampuero, “Automated seismic source characterisation using deep graph neural networks,” _Geophysical Research Letters_ , p. e2020GL088690, 2020.
  * [30] D. Owerko, F. Gama, and A. Ribeiro, “Predicting power outages using graph neural networks,” in _2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP)_. IEEE, 2018, pp. 743–747.
  * [31] J. Park and J. Park, “Physics-induced graph neural network: An application to wind-farm power estimation,” _Energy_ , vol. 187, p. 115883, 2019.
  * [32] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in _International Conference on Learning Representations (ICLR)_ , 2017.
  * [33] Z. Wu, S. Pan, G. Long, J. Jiang, X. Chang, and C. Zhang, “Connecting the dots: Multivariate time series forecasting with graph neural networks,” in _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , ser. KDD ’20. New York, NY, USA: Association for Computing Machinery, 2020, p. 753–763. [Online]. Available: https://doi.org/10.1145/3394486.3403118
  * [34] C. Song, Y. Lin, S. Guo, and H. Wan, “Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, no. 01, 2020, pp. 914–921.
  * [35] Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang, “Graph wavenet for deep spatial-temporal graph modeling,” in _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence_ , S. Kraus, Ed., 2019, pp. 1907–1913, international Joint Conference on Artificial Intelligence 2019, IJCAI-19. [Online]. Available: https://ijcai19.org/
  * [36] L. Bai, L. Yao, C. Li, X. Wang, and C. Wang, “Adaptive graph convolutional recurrent network for traffic forecasting,” _Advances in Neural Information Processing Systems_ , vol. 33, 2020.
  * [37] J.-J. Luo, S. Masson, S. K. Behera, and T. Yamagata, “Extended enso predictions using a fully coupled ocean–atmosphere model,” _Journal of Climate_ , vol. 21, no. 1, pp. 84 – 93, 01 Jan. 2008.
  * [38] K. Xu, C. Li, Y. Tian, T. Sonobe, K.-i. Kawarabayashi, and S. Jegelka, “Representation learning on graphs with jumping knowledge networks,” in _Proceedings of the 35th International Conference on Machine Learning_ , vol. 80. PMLR, 10–15 Jul 2018, pp. 5453–5462.
  * [39] International Research Institute for Climate and Society, “Iri enso forecast iri/cpc enso predictions plume,” Jan 2015, accessed 03-13-2021. [Online]. Available: https://iri.columbia.edu/our-expertise/climate/forecasts/enso/2015-January-quick-look/?enso_tab=enso-sst_table
  * [40] NOAA, National Weather Service, Climate Prediction Center, “Cold and warm episodes by season,” 2021, accessed: 03-13-2021. [Online]. Available: https://origin.cpc.ncep.noaa.gov/products/analysis_monitoring/ensostuff/ONI_v5.php
  * [41] I. Iskandar, D. Lestari, P. Utari, Q. Sari, D. Setiabudidaya, W. Mardiansyah, Supardi, and Rozirwan, “How strong was the 2015/2016 el niño event?” vol. 1011, p. 012030, publisher: IOP Publishing. [Online]. Available: https://doi.org/10.1088/1742-6596/1011/1/012030
  * [42] M. Newman, _Networks_. Oxford university press, 2018.
  * [43] J. Bjerknes, “Atmospheric teleconnections from the equatorial Pacific,” _Monthly Weather Review_ , vol. 97, no. 3, p. 163–172, Mar 1969. [Online]. Available: http://dx.doi.org/10.1175/1520-0493(1969)097<0163:ATFTEP>2.3.CO;2
  * [44] T. Matsuno, “Quasi-geostrophic motions in the equatorial area,” _Journal of the Meteorological Society of Japan_ , vol. 44, no. 1, p. 25–43, 1966. [Online]. Available: http://dx.doi.org/10.2151/jmsj1965.44.1_25
  * [45] A. E. Gill, “Some simple solutions for heat-induced tropical circulation,” _Quarterly Journal of the Royal Meteorological Society_ , vol. 106, no. 449, p. 447–462, Jul 1980. [Online]. Available: http://dx.doi.org/10.1002/qj.49710644905
  * [46] M. J. Suarez and P. S. Schopf, “A delayed action oscillator for ENSO,” _Journal of the Atmospheric Sciences_ , vol. 45, no. 21, p. 3283–3287, Nov 1988. [Online]. Available: http://dx.doi.org/10.1175/1520-0469(1988)045<3283:ADAOFE>2.0.CO;2
  * [47] M. A. Balmaseda, M. K. Davey, and D. L. T. Anderson, “Decadal and seasonal dependence of ENSO prediction skill,” _Journal of Climate_ , vol. 8, no. 11, p. 2705–2715, Nov 1995. [Online]. Available: http://dx.doi.org/10.1175/1520-0442(1995)008<2705:DASDOE>2.0.CO;2
  * [48] R. H. Weisberg and C. Wang, “A western Pacific oscillator paradigm for the El Niño-Southern Oscillation,” _Geophysical Research Letters_ , vol. 24, no. 7, p. 779–782, Apr 1997. [Online]. Available: http://dx.doi.org/10.1029/97GL00689
  * [49] F.-F. Jin, “An equatorial ocean recharge paradigm for ENSO. Part II: A stripped-down coupled model,” _Journal of the Atmospheric Sciences_ , vol. 54, no. 7, p. 830–847, Apr 1997. [Online]. Available: http://dx.doi.org/10.1175/1520-0469(1997)054<0830:AEORPF>2.0.CO;2
  * [50] L. Dong and M. J. McPhaden, “Unusually warm Indian Ocean sea surface temperatures help to arrest development of El Niño in 2014,” _Scientific Reports_ , vol. 8, no. 1, Feb 2018. [Online]. Available: http://dx.doi.org/10.1038/s41598-018-20294-4
  * [51] X. Li, S.-P. Xie, S. T. Gille, and C. Yoo, “Atlantic-induced pan-tropical climate change over the past three decades,” _Nature Climate Change_ , vol. 6, no. 3, p. 275–279, Nov 2015. [Online]. Available: http://dx.doi.org/10.1038/nclimate2840
  * [52] J.-J. Luo, W. Sasaki, and Y. Masumoto, “Indian Ocean warming modulates Pacific climate change,” vol. 109, no. 46, p. 18701–18706, Oct 2012. [Online]. Available: http://dx.doi.org/10.1073/pnas.1210239109
  * [53] C. Wang, “A unified oscillator model for the El Niño–Southern Oscillation,” _Journal of Climate_ , vol. 14, no. 1, p. 98–115, Jan 2001\. [Online]. Available: http://dx.doi.org/10.1175/1520-0442(2001)014<0098:AUOMFT>2.0.CO;2
  * [54] ——, “A review of enso theories,” _National Science Review_ , vol. 5, no. 6, p. 813–825, Oct 2018. [Online]. Available: http://dx.doi.org/10.1093/nsr/nwy104
  * [55] D. Clevert, T. Unterthiner, and S. Hochreiter, “Fast and accurate deep network learning by exponential linear units (elus),” in _4th International Conference on Learning Representations, (ICLR)_ , 2016.

|  Salva Rühling Cachay is a computer science undergraduate student at the
Technical University of Darmstadt, Germany. He has interned at Carnegie
Mellon’s University Auton Lab with Prof. Artur Dubrawski and Benedikt
Boecking. He is currently pursuing a research internship with Prof. David
Rolnick at Mila, Montreal. His research interests include weakly- and self-
supervised learning as well impactful problems at the intersection of machine
learning with the climate and earth sciences.
---|---
|  Emma Erickson is an undergraduate student studying electrical engineering
at the University of Illinois at Urbana-Champaign. Previously she interned
under Prof. Artur Dubrawski and Robert Edman at Carnegie Mellon University,
using machine learning to extract clinically relevant information from medical
videos. Currently she researches active learning with the Computational
Imaging Group at her home university under Prof. Minh Do and Corey Snyder. Her
primary research interests lie at the intersection of signal processing,
machine learning, and healthcare.
---|---
|  Ernest Pokropek is a undergraduate student of computer science at Faculty
of Electronics and Information Technology of Warsaw University of Technology,
Poland. Co-founder of Polish student research group ”FiberTeam”, he is
interested in applications of machine learning for various scientific domains,
especially medicine and sensors, signal processing and feature extraction
methodologies.
---|---
|  Arthur Fender Coelho Bucker is a mechatronics engineering undergraduate
student at the University of São Paulo, Brazil, and is currently enrolled in a
double degree program for master’s at the Technical University of Munich,
Germany. His previous works at Carnegie Mellon’s University AirLab address the
fields of multi-robot motion planning and semantic control for autonomous
aerial cinematography. Nowadays, his main research interests are in machine
intelligence, AI applied to bio-robotics, and Brain-computer Interfaces.
---|---
|  Willa Potosnak is a student in her 3rd year in the Biomedical Engineering
Department at Duquesne University. She plans to continue her studies by
pursuing a graduate degree in machine learning for medical applications and is
interested in using advanced machine learning algorithms to improve predictive
analytics. Currently, she is researching the use of machine learning to
predict post-cardiac surgery renal failure using continuous intraoperative
data.
---|---
Suyash Bire is a postdoctoral research associate at Earth, Atmospheric, and
Planetary Sciences, MIT. He holds a Ph.D. in marine science from Stony Brook
University with specializations in boundary current dynamics and interaction
of large scale mean flow with turbulent eddies. He is currently working with
Prof. John Marshall to explore the likely ocean circulation on icy moons in
the solar system. His research interests include large-scale ocean
circulation, eddy-mean flow interaction, boundary currents, and hydrothermal
convection. He is also interested in exploring the applicability of machine
learning techniques to climate and earth sciences.
---
|  Salomey Osei holds a Master of Philosophy in Applied Mathematics from the
Kwame Nkrumah University of Science and Technology. She is the team lead of
unsupervised methods for Ghana NLP and a co-organizer for the Women in Machine
Learning and Data Science (WiMLDS) Accra chapter. Her research interest
includes applications of machine learning, especially to finance and NLP. She
is also passionate about mentoring students, especially females in STEM and
her long term goal is to share her knowledge with others by lecturing.
---|---
|  Björn Lütjens Björn Lütjens is a PhD candidate at the Human Systems
Laboratory, Department of Aeronautics and Astronautics, MIT. Together with
Prof. Dava Newman, Dr. Cait Crawford, and Prof. Youssef Marzouk, he adapts
physics-informed neural networks to quantify the uncertainty in localized
climate projections. This research positions him at the intersection of the
physical/ecological sciences and machine learning with physics-informed neural
networks, Bayesian deep learning, and robust optimization. He is also
monitoring forest carbon from aerial imagery, which is supported by Microsoft,
NASA, WWF, MIT PkG, MIT Legatum, and MIT Sandbox. He has previously obtained
his Master’s in Autonomous Systems, pioneering with Prof. Jon How safe and
robust deep reinforcement learning techniques and holds a B.Sc. in Engineering
Science from Technical University of Munich.
---|---