AA YYYY

# Machine Learning for Climate Physics and Simulations

Ching-Yao Lai  1 Pedram Hassanzadeh  2 Aditi Sheshadri3 Maike Sonnewald4
Raffaele Ferrari5 Venkatramani Balaji6 1Department of Geophysics, Stanford
University, Stanford, CA 94305, USA; email<EMAIL_ADDRESS>2Department
of Geophysical Sciences and Commitee on Computational and Applied Mathematics,
University of Chicago, Chicago, IL 60637, USA  3Department of Earth System
Sciences, Stanford University, Stanford, CA 94305, USA  4Department of
Computer Science, University of California Davis, Davis, CA 95616, USA
5Department of Earth, Atmospheric, and Planetary Sciences, Massachusetts
Institute of Technology, Cambridge, MA 02139, USA  6Schmidt Sciences, USA

###### Abstract

We discuss the emerging advances and opportunities at the intersection of
machine learning (ML) and climate physics, highlighting the use of ML
techniques, including supervised, unsupervised, and equation discovery, to
accelerate climate knowledge discoveries and simulations. We delineate two
distinct yet complementary aspects: (1) ML for climate physics and (2) ML for
climate simulations. While physics-free ML-based models, such as ML-based
weather forecasting, have demonstrated success when data is abundant and
stationary, the physics knowledge and interpretability of ML models become
crucial in the small-data/non-stationary regime to ensure generalizability.
Given the absence of observations, the long-term future climate falls into the
small-data regime. Therefore, ML for climate physics holds a critical role in
addressing the challenges of ML for climate simulations. We emphasize the need
for collaboration among climate physics, ML theory, and numerical analysis to
achieve reliable ML-based models for climate applications.

###### doi:

10.1146/((please add article doi))

###### keywords:

climate, machine learning, physics-informed machine learning, equation
discovery, parameterization, emulator

††journal: Xxxx. Xxx. Xxx. Xxx.

##  1 Introduction

Figure 1: Diagram of machine learning (ML) applications in climate sciences
with respect to the availability of existing mathematical models, availability
of data and interpretability. [**Note to Annual Reviews: We created this
figure for this article; it is not based on any previously published image.**]

Machine learning (ML) has led to breakthroughs in various areas, from playing
Go to text generated with large language models (LLM), and more recently, to
weather forecasting [1, 2, 3, 4, 5]. Different from Go and LLM, the language
scientists use to understand and simulate weather and climate has been
equations rooted in fundamental physics. Physics-based equations, often
differential equations, are essential to simulate systems where direct
observations are limited and noisy. Even more so for projections of future
climates where data are altogether unavailable. (Fig. 1). Recently, ML has
emerged as an alternative tool for predictive modeling as well as improving
the understanding of climate physics (Fig. 2). For instance, due to the
expressiveness of some ML techniques such as neural networks (NNs), which are
universal approximators of functions [6] and operators [7], physics-free ML
models trained on limited data, from observations or physics-based
simulations, have demonstrated remarkable ability to perform accurate
nowcasting [8] with lead time of a few hours, weather prediction [9] with lead
times of several days, and El Nino forecast with lead times of a year [10]. It
remains an open question if some of the strategies and success in these short-
time predictions can be applied to improve climate projections, i.e., to
estimate changes in the statistics of weather events (e.g. return periods of
heat waves or tropical cyclones) in the next decades, centuries, and beyond.

[] ModelA representation of data to make predictions. Models can be ML-based,
physics-based, or coupled. SimulationPhysics-based models solved numerically.
For example the General Circulation Model is a climate simulation that solves
the Navier-Stokes equation…etc. EmulatorIn this article emulator refers to a
subset of models that fit the data, bypassing solving physics-based equations.
ML-based emulatorEmulators that fit the data with ML models (e.g., deep neural
networks), bypassing solving physics-based equations. Neural Network
(NN)Function approximators parameterized by function operations and parameters
$\gamma$, that are optimized to minimize specified cost functions
$\mathcal{L}$ Non-stationaritySystems with time-evolving statistical
properties, so that a limited time series is not representative of the past or
the future.

##### Weather versus climate: non-stationarity

The success story of ML for prediction (Section 6.2) has been primarily
showcased in weather forecasting [11]. Followed by initial attempts started in
2019 (e.g., [12, 13, 14, 15, 16, 17, 18]), by 2022-2023 ML weather models
(often called “emulators”; Section 6.2) achieve, at a fraction of the
computational cost, similar or better forecast skills than state-of-the-art
physics-based weather prediction models (e.g., [1, 2, 3, 4, 5, 19]). This
success has generated excitement about using ML to improve climate projections
as well. Yet, climate projections involve major additional challenges [20,
21]. In weather forecasting, we have a constant stream of real-time and
historical data for training ML-based emulators (lower-right of Fig. 1a) to
make predictions for a few weeks, where the statistics can be assumed to be
stationary. In climate, we are often interested in predicting the climate’s
forced response to changes in greenhouse gases in the atmosphere, leading to
“non-stationarity” [22], e.g. climate with different mean and variability. ML
models are not suited to predict the behavior of a system substantially
different from the one they have been trained on. Yet we simply do not have
observational data for the future (i.e. lower-left corner in Fig. 1a) to train
and validate future predictions; this is an issue for both ML and physics-
based models. We summarize the non-stationary challenge and potential
solutions, such as incorporating physics constraints, in Section 9.2.
Furthermore, the long-term predictions of future climate involves the
interactions between the atmosphere, ocean, cryosphere, land, and biosphere,
which make the problem more challenging than short-term weather forecast.

[] Subgrid-scale (SGS) parameterizationA model to parametrize the relationship
between the resolved states as a function of the unresolved (subgrid) state.
ML-based parametrizationSGS parametrization represented by a ML model rather
than physics-based equations.

##### Challenges in understanding and simulating climate

The climate system consists of interacting processes that span orders of
magnitude in spatial (from microns to planetary) and temporal (from seconds to
centuries) scales. Simulating the climate system to resolve all these scales
is computationally challenging. Due to its multi-scale nature, representing
physics in the under-resolved scales (e.g., cloud microphysics, turbulence) in
low-resolution climate simulations—referred to as subgrid-scale (SGS)
parameterization—has been a central goal for climate scientists. ML has
emerged as a promising alternative to SGS parameterization due to its ability
to perform equation discovery and its desirable properties as universal
function approximators, which do not require prior assumptions about the
functional forms of the parameterization. We summarize recent advances of ML
in SGS parameterization in Section 6.1, as well as its major challenges, such
as interpretability in Section 9.1 (lower-right corner of Fig. 1b) and
uncertainty quantification (UQ) in Section 9.3. Without understanding what the
ML model actually learns and the reasoning behind it, we cannot deduce when
the ML model will generalize well to future climates. One approach to improve
interpretability is equation discovery (Section 4; upper side of Fig. 1b).
Equations have long been the language for physicists to develop understandings
of the systems they govern. With the emergence of ML, ways to comprehend and
extract knowledge from ML-based models are new areas of research. Several
methods developed to understand what the ML model itself learns are detailed
in Section 9.3.

We organize this paper focusing on two distinct goals through which ML is
reshaping climate science: (1) ML for climate physics and (2) ML for climate
simulations. The former focuses on utilizing the increasing availability of
data from the Earth system to extract understandings, including knowledge
discovery (Section 2.1) and data-driven model discovery (Section 2.2). The
latter discusses recent advances in accelerating simulations, including data-
driven parameterization (Section 6.1) and climate emulators (Section 6.2).
Section 8.1 discusses methods used to add physical constraints to ML models.
For inverse problems with sparse data (upper left of Fig. 1a) in the “small-
data regime”, including physical constraints are necessary for generalizable
prediction where data does not exist. Finally, the future climate also falls
in the small-data regime, as no future observations exist. Along with an
incomplete understanding of the physics of future climate, it lies at the
bottom left of Fig. 1a, making it the most challenging among others in Fig.
1a. Thus, predicting climate is not merely about accelerating simulation but
essentially requires generating more physical knowledge than currently
available, moving towards the tractable upper left regime of Fig. 1a. While
training ML models to make accurate predictions faster than physics-based
simulations, making previously challenging tasks computationally tractable, is
an achievement, the ability to simulate does not equate to improved physical
understanding [23]. We stress that accurate and fast simulations or
predictions are not sufficient; deeper physical understandings of the climate
are necessary to address the climate modeling challenges, which are essential
for mitigation and adaptation efforts

Figure 2: An overview of the areas where ML has played a role in uncovering
climate physics and advancing climate simulations. Panels c, f, i, j, h
adapted with permission from Sonnewald et al. 2019 [24] (CC BY 4.0), Xiao et
al. 2023 [25] (CC BY 4.0), Lam et al. 2023 [4] (copyright 2023 AAAS), Yuval
and O’Gorman 2020 [26] (CC BY 4.0), Wang et al. 2021 [27] (CC BY 4.0),
respectively.

##  2 Machine Learning for Climate Physics

The increase in availability of data, from both observations and high-fidelity
simulations, is a key driver for new physical insights. Below we introduce two
emerging trends of research using ML to improve our understanding of the
climate physics using data: (1) Data-informed knowledge discovery, e.g.
identifying patterns and dynamical regimes in high-dimensional, complex
observations and simulations and (2) discovering data-informed predictive
models.

###  2.1 Data-informed knowledge discovery

Knowledge discovery (e.g. Fig. 2a-c), such as identifying coherent patterns of
dynamical significance in spatio-temporal data, has long been a fundamental
process for making discoveries in climate science. A classical example in
atmospheric science is the determination of what we now call the El Niño
Southern Oscillation (ENSO) by Walker in 1928 [28]. Stationed in India during
the British occupation, Walker employed an army of Indian clerks to conduct
principal component analysis (PCA) by hand on all available data, decomposing
it into orthogonal modes that revealed coherent structures associated with
ENSO. Today, we have access to vast amounts of data, both observational and
computational. Advanced ML techniques emerge as powerful pattern recognition
tools that computationally scale well with increasing volumes of data. Off-
the-shelf tools widely adopted in climate sciences include supervised learning
methods (e.g., random forest, Gaussian process regression (GPR), and NN) and
unsupervised learning methods such as autoencoder, which can be used for
dimensionality reduction, and clustering algorithms (e.g., k-means, self-
organizing maps). Several of the above tools have been used in climate science
for decades, long before deep learning took off.

[] Supervised learningAlgorithms that learn a mapping from input data
(features) to output labels based on the training examples provided
Unsupervised learningAlgorithms that discover patterns, structures, or
relationships within the input data (features) without explicit guidance from
labeled examples.

##### Dimensionality reduction

Climate data often involve high-dimensional, nonlinearly correlated, spatial
and temporal variables, such as temperature, pressure, and precipitation,
across large geographical regions and long time periods. Dimensionality
reduction has long been used in for transforming high-dimensional climate data
into a lower-dimensional space, which might be more amenable to physical
interpretation and to develop reduced-order predictive models. PCA, also known
as Empirical Orthogonal Function (EOF), is one of the commonly used linear
techniques for dimensionality reduction (Fig. 2a). Reducing decades of
observational data into a few modes of variability, such as ENSO, has
facilitated understanding of the underlying dynamics and even the robust
detection and interpretation of the anthropogenic climate-change footprint
[29, 30]. However, traditional techniques such as PCA/EOF have major
limitations, such as the lack of a dynamical meaning of the discovered “modes”
and the linearity [31]. (See Section “Beyond EOFs: Principal Oscillation
Patterns and ML” in the Supplemental Material).

ML techniques have the potential to address the linearity limitation. For
example, autoencoders, a type of NN with high-dimensional input and output
layers and a lower-dimension latent space, are powerful tools for
dimensionality reduction. A single-layer autoencoder with linear activation
functions is mathematically equivalent to PCA. In contrast, deep autoencoders
with nonlinear activation functions have more expressive power in capturing
the low-dimensional representation of the high-dimensional input [32, 33], and
their applications to climate science have started to emerge. Shamekh et al.
[34], interpreting the latent space of an autoencoder, developend of a new
metric for cloud and precipitation organization, enabling the development of a
better parameterization for moist convection.

##### Finding patterns in climate data

The task of finding patterns in climate data extends far beyond dimensionality
reduction and is a fruitful area that still has much to be explored. For
example, unsupervised methods like clustering (Fig. 2b) have been used to
identify the balance between terms of the equations governing simulation data
and to discover global ocean dynamical regions as parsimonious representations
of the governing equations [24] (Fig. 2c).

Data from real observations can often inspire new knowledge about climate
systems [35]. Supervised methods facilitate the utilization of vast amounts of
satellite observations, such as reconstructing a pan-Arctic dataset of sea-ice
thickness during periods when data are unavailable [36], revealing the strong
nonlinear interactions of ocean eddies [37], reconstructing ocean surface
kinematics with sea-surface height measurements [25], and detecting icebergs
to understand their contribution to the freshwater budget [38]. Extracting
information from remote-sensing data can fill missing gaps required to inform
physics-based models. For example, the identification of ice fractures (under-
resolved in simulations) is needed to constrain parameters for modeling ice
dynamics [39, 40]. Some of the above tasks have long been done manually and
often “subjectively” by scientists. ML offers an efficient alternative that
can be easily scaled up to all available data that may be intractable
otherwise, and can be made easily accessible, reproducible, and transparent
via open-source software. That said, the design of the loss functions is still
subjective.

###  2.2 Data-informed model discovery

Apart from distilling knowledge from data, physicists have been developing
predictive models to describe observations for centuries. The utility of a
model, if it accurately represents the observations, lies in its ability to
make predictions when the data is unavailable, such as projections about the
future. The crux of climate physics is creating trustworthy future
predictions, and determining how to construct a model that faithfully
describes the data is essential. It’s worth noting that in traditional
physical sciences, a “model” often takes the form of mathematical equations.
In modern ML literature, a model can refer to functional operations (e.g.,
NNs) that parameterize relationships between specified input and output
variables. For clarity, in this section by “model discovery” we meant the
discovery of mathematical equations.

Here, we broadly classify three different ML approaches that have been used
for finding models to describe climate data: parametric, state, and structural
estimations (Fig. 2). We use a dynamical-system example of the following form
to illustrate the differences:

$\frac{d}{dt}\bm{z}(t)=\bm{f}(\bm{z}(t),\bm{\theta}),$ (1)

where $\bm{z}(t)\in\mathbb{R}^{n}$ represents the states vector of the system
that evolves with time $t\in\mathbb{R}$, and its evolution is dictated by the
mathematical expression of the dynamics $\bm{f}$ and the model parameters
$\bm{\theta}$. See sidebar titled The Lorenz 63 system for an example. [h]

##  3 The Lorenz 63 system

The Lorenz 63 system [41], a simplified mathematical model for atmospheric
convection, is described by the following set of ordinary differential
equations:

$\frac{d}{dt}{x}=a(y-x),\quad\frac{d}{dt}{y}=x(b-z)-y,\quad\frac{d}{dt}{z}=xy-
cz,$ (2)

where the model parameters $\bm{\theta}=[a,b,c]$ are constants, and
$\bm{z}(t)=[x(t),y(t),z(t)]$ are the time-evolving states.

##### Parametric estimation $\bm{\theta}$

Given a discrete dataset of states measured at discrete times $t_{i}$, i.e.
$\\{x(t_{i}),y(t_{i}),z(t_{i})\\}_{i=1}^{N}$, parametric estimation refers to
predicting the free parameters $\bm{\theta}=[a,b,c]$ when the functional form
of the mathematical model $\bm{f}(\bm{z}(t),\bm{\theta})$ is known (Fig. 2d).

##### State estimation $\bm{z}$

State estimation involves predicting the state variables $\bm{z}(t)$ given the
mathematical model $\bm{f}(\bm{z}(t),\bm{\theta})$, model parameters
$\bm{\theta}$ and data $\\{x(t_{i}),y(t_{i}),z(t_{i})\\}_{i=1}^{N}$.
Estimating states is particularly useful for data interpolation when the
available data is sparse in time or space, for data denoising, or for
inversion when the predicted state is not measurable (e.g. Fig. 2f) and thus
completely unavailable in the data library, e.g., predicting $z(t)$ with data
of $\\{x(t_{i}),y(t_{i})\\}_{i=1}^{N}$.

##### Structural estimation $\bm{f}$

Also referred to as equation discovery. Prediction of the complete
mathematical expression of $\bm{f}(\bm{z}(t),\bm{\theta})$ (e.g. equation
(2)), including the free parameters $\bm{\theta}=[a,b,c]$ given only the
discrete data $\\{x(t_{i}),y(t_{i}),z(t_{i})\\}_{i=1}^{N}$. The determination
of the model $\bm{f}(\bm{z}(t),\bm{\theta})$ fully relies on the data;
therefore, dense data is often needed to guarantee the success of the
algorithms.

Beyond these three categories, an emerging data-driven approach involves
replacing $\bm{z}(t)$ with a ML-based emulator. In this approach, instead of
using mathematical equations, the dynamics $\bm{f}$ are represented by black-
box ML models, as detailed in Section 6.2.

####  3.0.1 Algorithms and examples

In this section, we list a few examples that demonstrate how ML has influenced
data-driven model discovery within the three categories described above;
ensemble Kalman inversion (EKI) for parameter $\bm{\theta}$ estimation,
physics-informed ML for state $\bm{z}$ estimation, and equation discovery for
structural $\bm{f}(\bm{z},\bm{\theta})$ estimation.

[h]

##  4 Inverse problem and data assimilation

The problems of estimating parameters $\bm{\theta}$ and states $\bm{z}$ of a
model $\bm{f}(\bm{z},\bm{\theta})$ falls under the umbrella of inverse
problems [42]. The importance of parametric and state estimations lies in the
fact that direct observations of model parameters and states are often
unavailable, yet they are crucial for simulating and predicting both the
weather and climate accurately. Various inverse problems arise in weather and
climate, such as estimating parameters in climate models or determining
initial conditions for improved weather forecasting. Data assimilation [43]
refers to the process of combining observational data with numerical models.

The synergies between data assimilation and ML has been increasingly
recognized [44, 45, 46], including using ML to correct model error in data
assimilation [47] and emulation of a dynamical system [48].

##### Ensemble Kalman inversion (EKI)

Here we focus on one family of methods, EKI, as an example, as it is
increasingly used in climate science. EKI [49] is a well-developed parameter
estimation technique in the climate modeling community, having been used in
various contexts such as convection, turbulence, and clouds [50, 51, 52],
gravity waves [53], and ocean convection [54]. EKI is a derivative-free [49]
optimization method for parametric estimation $\theta$, based on Ensemble
Kalman filtering (EnKF) [55], which is used for estimating states $\bm{z}(t)$
in numerical weather prediction (NWP) [56] given noisy observations. EKI [57]
attempts to find a distribution of model parameters $\bm{\theta}$ that can
describe time-averaged statistics of a “truth”, which could be from
observational data or simulations, removing dependence on state variables by
utilizing long integrations. EKI optimizes for macrophysical climate
statistics (e.g., derived by averaging over many occurrences of the event of
interest).

Quantification of parametric uncertainty is important as it illustrates how
perturbations of parameters $\theta$ that we want to estimate would translate
to the predictions of $z$. As shown in Fig. 2e, while an optimal value of
$\theta^{*}$ minimizing the cost function only captures one prediction
$z(\theta^{*})$, a range of $\theta$ could yield wide-ranging predictions
(e.g., covering extreme events). Running ensembles of forward physics-based
simulations $\bm{f}(\theta)$ with a range of $\theta$ for climate models to
quantify these uncertainties propagated by parameter uncertainty is currently
computationally infeasible. To address this challenge, the calibrate-emulate-
sample (CES) approach [50, 51] trained a Gaussian process regression (GPR) as
a cheap method to emulate the prediction of interest $z$ as a function of
$\theta$. Sampling the GPR emulator with Markov chain Monte Carlo enables
substantially faster uncertainty quantification of the predictions $z$
resulting from the plausible range of $\theta$. GPR has also been used for
calibrating parameters with UQ in Earth system models [58] using non-EKI
methods.

##### Physics-informed neural networks (PINN)

The use of PINNs [59, 60] for planetary-scale geophysical flow problems has
started to emerge in the past few years. Introduced by Raissi et al. (2019)
[59], PINN is a differentiable solver for partial differential equations
(PDEs) that is particularly useful for inverse problems involving sparse-data
inference, super-resolution, data denoising, and state estimation $\bm{z}$ in
data assimilation (see sidebar titled Inverse problem and data assimilation).
Unlike classical ML where the cost function typically only involves data, PINN
encodes physics-based equations directly in the cost function (Fig. 3a).

Figure 3:  (a) The physics-informed ML algorithm and its applications in data
assimilation (b-c). Panels b and c adapted with permission from Eusebi et al.
2024 [61] (CC BY 4.0) and Wang et al. 2022 [62] (CC BY 4.0), respectively.

Throughout the training iterations, the optimizer identifies the best ML-
parameterized states $\bm{z}=NN(\bm{x},t,\gamma)$ that are consistent with
both the data and the governing equations. In the small-data regime, without
evaluating the NN-parametrized $\bm{z}$ against known physical laws (such as
conservation of mass, momentum, and energy), the ML predictions can be
physically inconsistent and non-extrapolatable beyond the available
observational data (e.g., deviation from truth in Fig. 2g). In contrast, by
incorporating PDEs, PINN can achieve both physics-informed data interpolation
and extrapolation, as demonstrated by the examples in Fig. 3, which cannot be
achieved by ML models trained with observational data alone.

Fig. 3b-c demonstrate the applications of PINN on real observations, ranging
from estimating the initial conditions $\bm{z}(\bm{x},t=0)$ of hurricanes for
subsequent forecasts [61] to inferring the non-measurable viscosity structure
$z(\bm{x})$ of Antarctic ice shelves [62]. Both examples fall within the
“small-data regime” in the upper left corner of Fig. 1a, where incorporating
knowledge of PDEs becomes crucial for solving the inverse problems; the PINN-
reconstructed wind field $\bm{z}(\bm{x},t)$ (Fig. 3b) involves only sparse
observations of wind velocity itself as training data, obtained from
measurements by hurricane hunter planes and dropsondes. The PINN prediction of
ice viscosity is achieved without any observations of viscosity in the
training data (Fig. 3c); it relies solely on equations and other observable
states (velocity and thickness fields) as training data. Thus, both examples
involve substantial extrapolation beyond the sparse observational data, i.e.,
limited velocity data and no viscosity data.

As long as the same data and physics-based equations are used to solve the
inverse problem, the predictions generated by properly trained PINNs are as
trustworthy as those produced by established data assimilation methods. Due to
PINN’s leverage of GPU and differentiable modeling to infer accurate initial
conditions without ensembles of forward modeling, as used in ensemble-based
data assimilation methods [61] (see Supplemental Material for a brief
comparison), PINNs require orders of magnitude fewer computational resources
to construct hurricane initial conditions with similar accuracy as the
ensemble-based data assimilation methods [63]. That being said, established
data assimilation methods are supported by several mature theories, which are
relatively lacking for PINN methods. Although several models used in climate
predictions are not easily differentiable without substantial engineering
efforts, the development of differentiable solvers for atmospheric dynamics
[64] demonstrates promises. Differentiable ice-flow solver and emulator have
also recently emerged as a new tool for forward and inverse modeling [65].

##### Equation discovery

Existing equations $\bm{f}(\bm{z},\bm{\theta})$ describing the numerous
processes in the climate system, particularly the SGS processes, are far from
complete. Equation discovery, which outputs equations that are most consistent
with data, has been used to tackle this problem. Inspired by earlier symbolic
regression algorithms for distilling physical laws from data [66], Sparse
Identification of Nonlinear Dynamics (SINDy) [67] has emerged as a widely used
method for discovering $\bm{f}(\bm{z},\bm{\theta})$ from data of the states
$\bm{z}(t)$. It demonstrates the power of sparse regression for learning the
most relevant terms in the prescribed function library that describes the
data. To learn the correct $\bm{f}(\bm{z},\bm{\theta})$, SINDy requires dense
time series data of both $\bm{z}(t)$ and $d/dt({\bm{z}}(t))$. For many climate
problems these state measurements are sparse and noisy, or entirely
unavailable. Schneider et al. [68] showed that time-averaged statistics of the
states $\bm{z}(t)$, which are available for the climate system, can be
sufficient to recover both the functional form of $\bm{f}(\bm{z},\bm{\theta})$
as well as the noise level of the data using sparse regression combined with
EKI. Sparse EKI is robust to noisy data and was successfully implemented to
recover the Lorenz 96 equations [68]. Other approaches for equation discovery
from data assimilation increments [69, 70] and from partial observations [71],
motivated by climate problems, have been proposed too.

Arguably the most successful example of the application of equation discovery
for climate physics so far has been the learning of an ocean mesoscale SGS
parameterizations [72]. Trained on high-resolution simulation data, Zanna and
Bolton [72] showed that Bayesian linear sparse regression with relevance
vector machines identifies relevant terms in the prescribed function library
to discover the closed-form equations of $\Pi(\bar{z})$ (defined in Fig. 2)
for eddy momentum and temperature forcing. The closed-form equation is
consistent with an analytically drivable physics-based model [73, 74]. As
discussed later in Section 6.1, SGS parameterizations (Fig. 2) are essential
for improving the accuracy of computationally feasible low-resolution climate
simulations. While black-box NNs have shown promises for developing data-
driven SGS parameterizations (Section 6.1), the significant interest in
equation discovery stems from their better generalization to future climates
and their interpretability.

Inspired by early work on symbolic regression [66, 75], the symbolic genetic
algorithm [76] was developed to discover PDEs without the need to pre-
determine a function library. It uses a binary tree to parameterize common
mathematical operations (e.g., addition, multiplication, derivative, division)
and finds the correct operations such that the discovered equation matches the
data. In climate applications, genetic algorithms have been used for finding
equations for cloud cover parameterization [77] and ocean parameterization
[78].

##  5 Machine Learning for Climate Simulations

We discuss two major directions leveraging ML methods to improve the accuracy
of climate simulations: (a) SGS parameterization, aimed at developing more
accurate climate models via better representation of small-scale (expensive to
resolve) physical processes, and (b) Emulators, aimed at generating large
ensembles of simulations (or directly, the statistics) at a fraction of the
computational cost of a physics-based climate model. These two approaches are
briefly discussed below.

[h]

##  6 Uncertainties in climate projections

Climate projections are affected by three sources of uncertainty [79]: (a)
model uncertainty (also known as structural error), (b) internal variability
uncertainty (e.g., the signal-to-noise ratio problem), and (c) scenario
uncertainty (related to how much greenhouse gases will be released in the
future). Reducing model uncertainty requires developing more accurate climate
models (e.g., improving parameterization or increasing resolutions), while
reducing the internal variability and scenario uncertainties requires
computationally efficient climate models that can generate long, large
ensembles of simulations and explore different scenarios.

###  6.1 Subgrid-scale parameterization (SGS)

There are two reasons computationally feasible climate models require SGS
parameterization: (a) the process of interest varies on length or time scales
smaller than a climate model’s resolution, and (b) equations to describe the
process are not known. SGS parameterizations estimate the effect of these
unresolved processes on the resolved scales. Developing SGS parameterizations
for climate modeling, but also for high resolution simulations in limited
domain, has been an active area of research since the pioneering work of
Smagorinsky [80] on the first climate models in the early 1960s. Still, today,
the approximations made in formulating these parameterizations are a leading
cause of model uncertainty in climate models (see the sidebar titled Climate
projection uncertainties). ML presents a potentially exciting path forward in
improving these SGS parameterizations or developing new ones. The general idea
is to use observations or high-resolution simulations to learn a data-driven
representation or closed-form equation of the SGS term $\Pi$ (defined in Fig.
2). Examples of the latter approach were discussed in Section 3.0.1; below, we
mainly focus on approaches based on NNs (Fig. 2j). [] Model
uncertaintyDeviation of the physics-based model from the data, could be due to
inaccurate parameters $\theta$ or the physics equations $f$ itself. Structural
errorA type of model uncertainty arising from inaccuracies in the equations
used to represent the data or processes of interest.

Studies have demonstrated the ability of ML algorithms such as NNs to learn
parameterizations as a supervised learning task
$NN(\bar{\bm{z}},\gamma)=\Pi(\bar{\bm{z}})$ for prototypes of geophysical
turbulence (e.g., [72, 81]), ocean turbulence (e.g., [82, 83]), moist
convection and clouds (e.g., [84, 85, 86, 87, 88, 89, 90]), and atmospheric
gravity waves (e.g., [91, 92, 93]). Some of these examples have achieved
stable simulations that are more accurate than those with common physics-based
parameterizations [87, 72, 81, 89]. However, while promising, this approach
faces a number of challenges. Some are common among other ML applications to
climate (e.g., interpretability, extrapolation) and are discussed in Section
9. Challenges specific to SGS parameterization via supervised learning are
availability of high-resolution data and instabilities arising once these ML-
based SGS parameterizations are coupled to the climate models [87, 72, 94].

This discussion of SGS parameterization for climate models would not be
complete if we did not emphasize the distinct challenges compared to its use
in weather forecasting. In weather forecasting, the objective is to predict a
specific trajectory based on initial conditions, necessitating precise
prediction of SGS physics. Conversely, climate studies aim to predict changes
in the system’s average behavior over decades. Thus, it is sufficient to
predict the SGS statistics rather than all its specific features. This shift
represents a novel challenge for ML applications, which are typically
optimized to capture detailed information from training data in supervised
learning. Despite these challenges, a few studies have made progress in
producing stable and accurate simulations of simple climate prototypes using
NNs trained with differentiable modeling [95] and EKI [94]. Recently, Google
Research has made strides in this direction by developing a climate model’s
dynamical core that learns SGS physics statistics directly from observation-
derived data [64]. Yet, ensuring model stability remains problematic, likely
due to the intricate requirement of capturing only statistical information
about SGS physics while preserving the numerical stability of the equations
that describe the climate’s mean evolution.

###  6.2 Climate emulators

“Emulator” refers to several types of tools in the climate science literature.
In general, an emulator is trained to mimic the data, from physics-based
simulations or observations, to substantially reduce the computational cost of
climate simulations.produce new predictions, e.g. for other climate conditions
within the distribution of the training data

Emulators can be used to interpolate the projections from expensive climate
simulations, making their projections among different emission scenarios
accessible without re-running the simulations. Earlier use of ML for emulators
followed the successful approach of traditional pattern-scaling emulators [96,
97], which, for example, predict the change in statistics of variables of
interest (e.g., regional annual-mean surface temperature or the return period
of extreme events at a later time) given a small set of inputs (e.g., year,
greenhouse gas forcing, global mean surface temperature). Using ML techniques
(e.g., GPR, NN), emulators such as ClimateBench [98] have been employed to
estimate the climate impacts of anthropogenic emissions annually up to 2100.
However, the skill of these ML-based emulators are not superior to that of
pattern-scaling emulators, i.e. emulators that regress regional temperature on
global mean temperature or cumulative emissions.

##### Spatio-temporal emulators

While the aforementioned emulators can predict aggregated statistics within an
often large window of length and time scales, another type of emulator has
emerged in recent years with the aim of predicting the evolution of the
climate system at fine spatio-temporal scales. These “spatio-temporal”
emulators leverage the success of ML-based weather forecast models, which are
physics-free and trained solely on reanalysis data [99] (spanning
1979-present; see sidebar titled Reanalysis). Recent ML-based weather forecast
models (e.g., FourCastNet [2], Pangu [3], GraphCast [4]) are time-stepping
algorithms that solve the initial-value problem of predicting the state
$\bm{z}(t)$ of the global atmosphere forward in time (from $t_{i}$ to
$t_{i+1}$, then from $t_{i+1}$ to $t_{i+2}$, and so on; Fig. 2i). They exhibit
comparable or even better skill than the best physics-based weather prediction
models for lead times of up to around 10 days [4]. However, weather and
climate predictions are different problems. The former is an initial value
problem while the latter is more akin to a boundary value problem in the sense
that the focus is on how external boundary conditions impact the system over
longer periods of time. [] Spatio-temporal emulatorAn emulator with
predictions that evolve with space and time, e.g. weather forecasting, sea ice
seasonal forecasting. Initial value problemA forward model $\bm{f}(\bm{z})$
with predictions $\bm{z}(t)$ subject to the initial conditions $\bm{z}(t_{0})$
specified at time $t_{0}$. Boundary value problemA forward model
$\bm{f}(\bm{z})$ with predictions $\bm{z}(t)$ subject to the boundary
conditions $\bm{z}(x_{0},t)$, that can be time-evolving, at the boundaries
$x_{0}$.

[h]

##  7 Reanalysis

Reanalysis, sometimes referred to as “maps without gaps”, refers to a method
of using a physical model to assimilate disparate observational data streams
into a combined multivariate dataset uniform in space and time. The model
fills in the data-poor regions and ensures physical consistency between
variables. The output is often referred to as _reanalysis data_ but it is
important to keep in mind that these data are not observations, but outputs of
a forecast model. In fact reanalysis products from different weather centers
will differ amongst themselves, and this spread can be taken as a measure of
uncertainty in observations and understanding.

For climate predictions, atmospheric spatio-temporal emulators are built to
solve boundary-value problems that integrate the global atmospheric state
given external forcings (e.g. radiative forcing) and time-evolving boundary
conditions (e.g., sea-surface and land temperature) for decades or centuries.
The AI2 Climate Emulator (ACE) [100, 101] is a promising example of such a
spatio-temporal emulator trained on physics-based simulations. Similar work on
oceanic spatio-temporal emulators [102, 103] suggests that coupled climate
emulators might start to emerge as well.

ML spatio-temporal emulators have shown even more promise in simulating
components of the climate system whose physics are less well understood. For
the cryosphere, deep learning-based emulators for seasonal sea-ice prediction
have been found to outperform state-of-the-art physics-based dynamical models
in terms of forecast accuracy [104, 105, 106], with a lead time of a few
months. Some of these sea-ice emulators capture atmospheric-ice-ocean
interactions by training with appropriate climate variables [106, 104].
Because these emulators were trained directly on sea ice observational data,
they learn the atmospheric-ice-ocean interactions that are incompletely
parameterized in the physics-based dynamical models, thereby correcting the
model’s structural error).

[h]

##  8 Challenges and opportunities of ML-based emulators

Emulators can address the climate response to a particular emission scenario
and internal variability uncertainty. However, emulators are at best as
accurate as the data they are trained on, which may still contain model
uncertainty; this may be partially overcome by training with data from very
high-resolution yet very expensive simulations, such as the emerging global
1-km runs. Nonetheless, major questions about the stability and physical
consistency of the trained spatio-temporal emulators need to be addressed. For
example, ML weather forecast models have been shown to produce unstable or
unphysical atmospheric circulations beyond 10 days, poorly represent small
scales processes [107, 9], and do not reproduce the chaotic behavior of
weather [108]. Addressing these challenges will likely require incorporating
physical constraints into ML models (Section 8.1) and developing a deeper
understanding of the different sources of error in these models (Section
9.1-9.2).

###  8.1 Physics-informed machine learning (PIML)

Despite ML’s ability to emulate weather (Section 6.2) and parameterize SGS
processes when trained on high-resolution simulations or observations (Section
6.1), there is no guarantee that its predictions are physically sound (e.g.,
conserve mass, energy). This physical inconsistency is problematic and makes
long-term climate projections using ML-based emulators and ML-based
parameterizations not trustworthy (Fig. 2g, left panel). Incorporating physics
constraints such as conservation laws, symmetries, and more broadly,
equivariances (defined below), has been shown to alleviate a number of
challenges such as instabilities and learning in the small-data
regime–Kashinath et al. [109] review earlier work in physics-informed ML for
weather and climate modeling.

##### Conservation laws

Various methods exist for incorporating conservation laws into ML models, such
as embedding them in the loss function (e.g., physics-informed NNs [59];
Section 4) or other components of the ML architecture. For instance, Beucler
et al. [110] demonstrated that conserving quantities like mass and energy can
be enforced as hard constraints within the NN architecture. Their
architecture-constrained NN, trained as a SGS parameterization of moist
convection, significantly improved simulated climate.

##### Symmetries and equivariances

Incorporating symmetries and equivariances has also shown advantages,
particularly in the small-data regime. For a variable $x$, the nonlinear
function $g$ is equivariant under transformation $\mathbf{A}$ if
$\mathbf{A}g(x)=g(\mathbf{A}x)$. For example, by incorporating various
symmetries (e.g., scale equivariance, rotational equivariance) into
convolutional neural networks (CNNs) trained on turbulence data from previous
time steps, the CNNs generalize well to future time steps [27] (Fig. 2h).
Enforcing rotational equivariance through Capsule NN, CNNs, or customized
latent spaces has improved ML-based predictions of large-scale weather
patterns [111] and turbulent flows [112, 27].

##### Spectrum information

Including information about the Fourier spectrum of geophysical turbulence in
the loss function has been shown to aid in learning small scales and reducing
spectral bias, thereby improving the stability and physical consistency of ML-
based emulators [107] (see Section 9.1 for further discussions).

##  9 Challenges and Promises

###  9.1 Quantifying uncertainties of ML models

Broadly speaking, there are two sources of error in ML-based models: errors in
the training data and the epistemic uncertainty for the ML model. The errors
in data can stem from sparsity and measurement noise, which are particularly
relevant for observations. Shortcomings of physics-based models are the source
of errors in simulation data. The epistemic uncertainty of the ML model arises
from different sources, such as model architecture and hyperparameters [113].
Some ML techniques, such as GPR, provide rigorous estimates of uncertainty
(e.g., [20, 98] for climate applications). However, for deep learning, UQ is
more complicated and the subject of extensive research (see recent review
papers in the context of scientific applications [113, 114]). [] Epistemic
uncertaintyDeviation of the ML model from the data, could be due to
approximation, optimization, and generalization errors.

Understanding the sources of errors in ML models can improve their stability,
physical consistency, and reliability. For example, in simulations coupled
with ML-based parameterization, errors from the ML model are propagated into
the simulations and vice versa, potentially leading to instabilities and
nonphysical behavior. Similarly, errors in a spatio-temporal emulator can
accumulate and destabilize the emulation. Since we cannot directly estimate
”accuracy” during inference (as we do not have access to the ground truth),
the best approach is to estimate the ”uncertainty” of the ML model’s output,
as this uncertainty may be indicative of its accuracy. Here, we provide
examples from climate science for uncertainty quantification (UQ) of NNs, and
we also discuss two impactful sources of epistemic uncertainties related to
representation error (e.g., spectral bias) and data imbalance (e.g., rare
extreme events).

##### Quantifying epistemic uncertainty

A variety of techniques from the ML literature have been employed for UQ of
NNs in climate applications. For instance, deep ensembles [115, 116, 117, 18]
and Bayesian NNs [18, 118] are used to assess the mean and spread of
predictions. In [119] these two methodologies were combined to reveal the
consequences of architecture choice, as determined by UQ and the ability to
approximate the physical system. Other techniques, such as variational
autoencoders, drop out, and abstension, have also been explored [120, 121,
118, 122]. See [123, 118] for detailed discussions.

Figure 4: The spectral bias of neural networks [124, 125] can be widely
observed in climate applications and can cause major challenges such as
intsabilities. (a-b) State-of-the-art ML-based weather emulator predictions
after one time step and 10 days (based on results from [107], courtesy of
Qiang Sun). (c-d) Quasi-geostrophic (QG) turbulence prediction after one time
step ([107], courtesy of Ashesh Chattopadhyay) or reconstruction (based on
results from [70], courtesy of Rambod Mojgani). (e) Atmospheric boundary layer
turbulence reconstruction [126]. (f) Simple 1D function reconstruction
(Courtesy of Yongji Wang). Panels a-c, d, e adapted with permission from
Chattopadhyay and Hassanzadeh 2023 [107] (CC BY 4.0), Mojgani et al. 2024 [70]
(CC BY 4.0), Rybchuk et al. 2023 [126] (CC BY 4.0), respectively.

##### Spectral bias

Another example of epistemic error in ML models affecting climate applications
is “spectral bias” [124] (or frequency principle [125]). Namely, NNs learn to
represent the large scales much more easily than small scales, which can pose
challenges for multi-scale climate problems. Fig 4 a shows the spectrum of the
one-time-step ($\sim$ 6 or 12h) prediction of upper-level wind from a few
state-of-the-art ML weather emulators. While the predictions exhibit the
correct spectrum for up to zonal Fourier wave numbers of $\sim 30$ (scales of
40000 km to 500 km), smaller scales (from 500 km to 25 km) are poorly learned.
It’s noteworthy that these predictions all boast around $\sim 99\%$ accuracy
based on anomaly pattern correlation. Fig 4b illustrates how these errors in
small scales grow to larger scales after 10 days. Eventually, these
predictions either blow up or become unphysical [107]. The same behavior is
observed in simpler tasks such as time-stepping prediction for quasi-
geostrophic (QG) turbulence (Fig 4c) and its re-construction (Fig 4d),
atmospheric boundary layer turbulence (Fig 4e), or even a simple 1D function
(Fig 4f). Solutions based on Fourier regularization of the loss function (Fig
4 c from [107]), and random Fourier features [127, 128] (Fig 4 d from [70])
have shown promising results. Superposing small NNs [129] (Fig 4 f) also
improves spectral bias compared with vanilla NN. [] Spectral biasNNs’ tendency
to preferentially capture certain frequencies of the training data.

##### Rare extreme events

Another example of epistemic error relates to rare events (e.g., heat waves,
hurricanes, ice-shelf collapse, ocean circulation collapse). Predicting these
rarer events is crucial, but they are often underrepresented or entirely
absent from the training set, leading to significant “data imbalance”.
Addressing data imbalance and improving the learning of rare events in an
active area of research. Common approaches such as re-sampling [130, 131],
using weighted loss function [118, 132], and learning the equations that cause
the rare behaviour [119] have shown promise. Innovative approaches, such as
combining ML-based emulators with mathematical tools for rare events [133,
134], may enable the learning of the rarest events.

###  9.2 Non-stationarity: out-of-distribution error of ML models

Climate change is inherently non-stationary [22]: the mean state and its
variability change over time. This poses a major challenge for applications of
ML models to climate-change projections. For instance, ML-based models trained
on data from the current climate may not perform well for a warmer future
climate with higher greenhouse gas concentrations. Studies have already
demonstrated unstable or unphysical simulations resulting from NN’s inability
to extrapolate beyond its training data [87, 135, 136]. The non-stationary
problem raises new questions: How can we ensure that the prediction task is
within the distribution of the training data? How do we ensure that the ML
model leverages information that is climate-invariant? Would a hybrid
approach, coupling physics- and ML-based models, improve long-term climate
simulations?

Examples of strategies for dealing with non-stationarity and out-of-
distribution generalization include: (a) incorporating physical knowledge
[136] and (b) transfer learning [135, 81]. As an example of (a), the recently
proposed “climate-invariant” ML [136] learns the mapping between variables of
interests that is universal across climates. This study showed promising
offline results of data-driven parameterization of moist convection across a
range of cold to warm climates once temperature, relative humidity, and latent
heating were properly transformed. This approach leverages physical insight
and available theories of climate. The main challenge is finding the
appropriate transformations, which are easier to find for thermodynamics but
harder to find for dynamics (e.g., wind). [] Climate-invariant MLML that
utilizes relationships that stay the same across different climates to improve
generalization.

Transfer learning, a common framework in ML for addressing out-of-distribution
generalization, involves training an ML model for a given system (e.g., the
current climate) and then re-training it with a much smaller amount of data
from a new system (e.g., a warmer climate). The re-trained ML model can then
perform well for the new system. Several studies have demonstrated the
potential of transfer learning to address significant changes in parameters
$\theta$ (e.g., a 100x increase in Reynolds number in geophysical turbulence)
or forcing [135, 81]. The key challenge with transfer learning is obtaining
reliable data for re-training. In climate change prediction, we must rely on
simulations, as observations from the future are unavailable. Libraries of
high-resolution global and regional simulations that strategically sample from
a range of climates are emerging, providing a valuable source of training and
re-training data [137, 138, 139].

Figure 5: Understanding what ML learns. Panel (a, b) is a cartoon illustration
of ensuring that the input data necessary for the ML model to demonstrate its
learning of physics is present from the THOR method (Tracking global Heating
with Ocean Regimes) [115]. (a, top) Training a NN to predict sections of the
ocean dominated by different balances in equation terms describing the flow
(colors in output) using related surface fields (e.g., wind). (a, bottom)
Looking backwards using XAI to see where in the input the NN saw as relevant
(blue: not relevant, red: relevant). (b) For the pink section in the North
Atlantic, only two equation terms are relevant (red boxes), and relevances
show conformance in two maps below, e.g., where the mountain range (close
black lines) in depth (H) gives negative relevance. (c) The two leftmost
panels show examples of the state $\bm{z}$ and SGS term $\Pi$ (defined in Fig.
2) from two setups of geophysical turbulence that differ in forcing scale and
dynamics. The other panels show examples of the Fourier spectra of
convolutional kernels of NNs trained as ML-based SGS parameterizations
$NN(\bar{\bm{z}},\gamma)=\Pi$. The Fourier analysis shows the emergence of
low-pass, high-pass, and band-pass Gabor filters [135]. Panels a-b and c
adapted with permission from Sonnewald and Lguensat 2021 [115] (CC BY 4.0) and
Subel et al. 2023 [135] (CC BY 4.0), respectively.

###  9.3 Understanding what ML is learning

Explainable AI (XAI) is gaining popularity (e.g., [115, 140, 141, 142, 143,
144, 145]) and is actively being developed within the ML community. Examining
which inputs in a NN are most useful for predicting outputs is being
increasingly used by the geoscience community [146]. The core motivation
behind XAI is that ML, by design, can detect spurious patterns and lead to
inaccurate predictions. Sonnewald et al. [115] used XAI within a dense
ensemble to determine their ML model’s accuracy (Fig. 5 top panel). The ML
model’s task was to predict physical ‘regimes’, i.e., dominant balances
between terms in the equation governing the flow (Fig. 5b). XAI determined
which model input fields were most relevant (Fig. 5b bottom panel). Despite
the individual ensemble members being in close agreement for the prediction,
there was some spread in the individual assessments of relevance. This
reflects the complex optimization problem the ML models were solving,
resulting in similar predictions but using different ’reasoning’ from the
input fields. As the ML model was trained to determine what equation terms
dominate at a given location, the relevance maps could then be used to
determine if the predictions were made for reasons consistent with the
governing equations. Illustrated in Fig. 9.3, for example, the correct terms
are emphasized (Fig. 5b). Similar equation determining frameworks are in [119,
116, 18].

More standard mathematical techniques can also be applied to understand NNs.
For example, spectral analysis of data and/or equations, e.g., via Fourier
transform, is a powerful tool for understanding physical systems. Fourier
analysis of deep NNs has provided insight into their learning process [124,
125]. In the context of geophysical turbulence, Fourier analyses of CNNs [135]
have shown to reveal what the NN learns and explain changes in the weights
during transfer learning from one flow to another (e.g., as Reynolds number
increases). The convolution kernels (with over 1 million learnable parameters)
used for parameterization were shown to fall into just a few classes: low-pass
and high-pass filters, and Gabor wavelets (Fig. 5c). These findings align well
with prior work on using wavelets for turbulence modeling [147], and even more
so with theoretical ML studies on the need for such spectral filters for
learning multi-scale, localized data [148, 149]. More recent work has found
this approach useful in interpreting deep NNs in climate applications, where
concepts from physics and ML are examined together [94, 107].

##  10 Summary and Outlook

Numerous scientific discoveries and rigorous understandings have been driven
by empirical relationships. We summarized in Section 2.1 how ML can facilitate
this, for example, by accelerating the search for patterns in climate data
that can be used to derive physical understandings. We also summarized in
Section 2.2 the promises ML has brought to find closed-form equations for
poorly understood climate processes. In many aspects of the climate system, we
do not have process-level models to describe the system (e.g., sea ice
rheology inconsistent with observations). The increasing amount of
observational data offers exciting opportunities for both equation and
knowledge discovery to improve the fundamental understanding of climate
physics.

On the other hand, ML can be used as tools to improve simulations. ML models
can be coupled with traditional physics-based models and used to parameterize
processes for which closed-form equations are not yet available (Section 6.1).
ML has led to breakthroughs in weather forecasting, a task not widely expected
to be possible a couple of years ago. We discussed the challenges to overcome
when moving forward from weather forecasting to climate prediction (Sections
6.2 and 9).

Progress towards the above goals can benefit from collaborations among
scientists from various disciplines. For example, the numerical analysis of
differential equations and the advent of digital computers played a key role
in starting the field of numerical weather and climate prediction [150].
Developing similar rigorous tools, by closely combining methods from climate
physics, ML theory, and numerical analysis, can potentially help with building
stable, accurate, and trustworthy ML-based models.

## DISCLOSURE STATEMENT

The authors are not aware of any affiliations, memberships, funding, or
financial holdings that might be perceived as affecting the objectivity of
this review.

## ACKNOWLEDGMENTS

We thank Mingjing Tong, Oliver Dunbar, and Jinlong Wu for their helpful
discussions regarding data assimilation methods, GPR with EKI, and sparse
learning, respectively. We also thank Qiang Sun, Ashesh Chattopadhyay, Rambod
Mojgani, and Yongji Wang for helping to remake the spectral bias figure.
C.Y.L., P.H., and A.S. acknowledge the National Science Foundation for funding
via Grants No. DMS-2245228, OAC-2005123, and OAC-2004492, respectively. P.H.
and A.S. also acknowledge funding from Schmidt Sciences through the Virtual
Earth System Research Institute.

## References

  * [1] R. Keisler, “Forecasting global weather with graph neural networks,” arXiv preprint arXiv:2202.07575, 2022.
  * [2] J. Pathak, S. Subramanian, P. Harrington, S. Raja, A. Chattopadhyay, M. Mardani, T. Kurth, D. Hall, Z. Li, K. Azizzadenesheli, P. Hassanzadeg, K. Kashinath, and A. Anandkumar, “Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators,” arXiv preprint arXiv:2202.11214, 2022.
  * [3] K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, and Q. Tian, “Accurate medium-range global weather forecasting with 3d neural networks,” Nature, vol. 619, no. 7970, pp. 533–538, 2023.
  * [4] R. Lam, A. Sanchez-Gonzalez, M. Willson, P. Wirnsberger, M. Fortunato, F. Alet, S. Ravuri, T. Ewalds, Z. Eaton-Rosen, W. Hu, et al., “Learning skillful medium-range global weather forecasting,” Science, vol. 382, no. 6677, pp. 1416–1421, 2023.
  * [5] L. Chen, X. Zhong, F. Zhang, Y. Cheng, Y. Xu, Y. Qi, and H. Li, “Fuxi: a cascade machine learning forecasting system for 15-day global weather forecast,” npj Climate and Atmospheric Science, vol. 6, no. 1, p. 190, 2023.
  * [6] K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural networks, vol. 2, no. 5, pp. 359–366, 1989.
  * [7] T. Chen and H. Chen, “Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems,” IEEE transactions on neural networks, vol. 6, no. 4, pp. 911–917, 1995.
  * [8] S. Ravuri, K. Lenc, M. Willson, D. Kangin, R. Lam, P. Mirowski, M. Fitzsimons, M. Athanassiadou, S. Kashem, S. Madge, et al., “Skilful precipitation nowcasting using deep generative models of radar,” Nature, vol. 597, no. 7878, pp. 672–677, 2021.
  * [9] Z. Ben-Bouallegue, M. C. Clare, L. Magnusson, E. Gascon, M. Maier-Gerber, M. Janousek, M. Rodwell, F. Pinault, J. S. Dramsch, S. T. Lang, et al., “The rise of data-driven weather forecasting,” arXiv preprint arXiv:2307.10128, 2023.
  * [10] Y.-G. Ham, J.-H. Kim, and J.-J. Luo, “Deep learning for multi-year enso forecasts,” Nature, vol. 573, no. 7775, pp. 568–572, 2019.
  * [11] S. Rasp, S. Hoyer, A. Merose, I. Langmore, P. Battaglia, T. Russel, A. Sanchez-Gonzalez, V. Yang, R. Carver, S. Agrawal, et al., “Weatherbench 2: A benchmark for the next generation of data-driven global weather models,” arXiv preprint arXiv:2308.15560, 2023.
  * [12] J. Pathak, B. Hunt, M. Girvan, Z. Lu, and E. Ott, “Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach,” Physical review letters, vol. 120, no. 2, p. 024102, 2018.
  * [13] P. D. Dueben and P. Bauer, “Challenges and design choices for global weather and climate models based on machine learning,” Geoscientific Model Development, vol. 11, no. 10, pp. 3999–4009, 2018.
  * [14] J. A. Weyn, D. R. Durran, and R. Caruana, “Can machines learn to predict weather? using deep learning to predict gridded 500-hpa geopotential height from historical weather data,” Journal of Advances in Modeling Earth Systems, vol. 11, no. 8, pp. 2680–2693, 2019.
  * [15] A. Chattopadhyay, E. Nabizadeh, and P. Hassanzadeh, “Analog forecasting of extreme-causing weather patterns using deep learning,” Journal of Advances in Modeling Earth Systems, vol. 12, no. 2, p. e2019MS001958, 2020.
  * [16] S. Rasp, P. D. Dueben, S. Scher, J. A. Weyn, S. Mouatadid, and N. Thuerey, “Weatherbench: a benchmark data set for data-driven weather forecasting,” Journal of Advances in Modeling Earth Systems, vol. 12, no. 11, p. e2020MS002203, 2020.
  * [17] S. Rasp and N. Thuerey, “Data-driven medium-range weather prediction with a resnet pretrained on climate simulations: A new model for weatherbench,” Journal of Advances in Modeling Earth Systems, vol. 13, no. 2, p. e2020MS002405, 2021.
  * [18] M. C. A. Clare, O. Jamil, and C. J. Morcrette, “Combining distribution-based neural networks to predict weather forecast probabilities,” Quarterly Journal of the Royal Meteorological Society, vol. 147, pp. 4337–4357, Oct. 2021.
  * [19] I. Price, A. Sanchez-Gonzalez, F. Alet, T. Ewalds, A. El-Kadi, J. Stott, S. Mohamed, P. Battaglia, R. Lam, and M. Willson, “Gencast: Diffusion-based ensemble forecasting for medium-range weather,” arXiv preprint arXiv:2312.15796, 2023.
  * [20] D. Watson-Parris, “Machine learning for weather and climate are worlds apart,” Philosophical Transactions of the Royal Society A, vol. 379, no. 2194, p. 20200098, 2021.
  * [21] T. Schneider, S. Behera, G. Boccaletti, C. Deser, K. Emanuel, R. Ferrari, L. R. Leung, N. Lin, T. Müller, A. Navarra, et al., “Harnessing ai and computing to advance climate modelling and prediction,” nature climate change, vol. 13, no. 9, pp. 887–889, 2023.
  * [22] T. N. Palmer, “A nonlinear dynamical perspective on climate prediction,” Journal of Climate, vol. 12, no. 2, pp. 575–591, 1999.
  * [23] I. M. Held, “The gap between simulation and understanding in climate modeling,” Bulletin of the American Meteorological Society, vol. 86, no. 11, pp. 1609–1614, 2005.
  * [24] M. Sonnewald, C. Wunsch, and P. Heimbach, “Unsupervised learning reveals geography of global ocean dynamical regions,” Earth and Space Science, vol. 6, no. 5, pp. 784–794, 2019.
  * [25] Q. Xiao, D. Balwada, C. S. Jones, M. Herrero-González, K. S. Smith, and R. Abernathey, “Reconstruction of surface kinematics from sea surface height using neural networks,” Journal of Advances in Modeling Earth Systems, vol. 15, no. 10, p. e2023MS003709, 2023.
  * [26] J. Yuval and P. A. O’Gorman, “Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions,” Nature communications, vol. 11, no. 1, p. 3295, 2020.
  * [27] R. Wang, R. Walters, and R. Yu, “Incorporating symmetry into deep dynamics models for improved generalization,” ICLR, 2021.
  * [28] G. Walker, “World weather. vol. 54,” QJR Meteorol Soc, pp. 79–87, 1928.
  * [29] S. Corti, F. Molteni, and T. Palmer, “Signature of recent climate change in frequencies of natural atmospheric circulation regimes,” Nature, vol. 398, no. 6730, pp. 799–802, 1999.
  * [30] D. W. Thompson and S. Solomon, “Interpretation of recent southern hemisphere climate change,” Science, vol. 296, no. 5569, pp. 895–899, 2002.
  * [31] A. H. Monahan, J. C. Fyfe, M. H. Ambaum, D. B. Stephenson, and G. R. North, “Empirical orthogonal functions: The medium is the message,” Journal of Climate, vol. 22, no. 24, pp. 6501–6514, 2009.
  * [32] J. Page, M. P. Brenner, and R. R. Kerswell, “Revealing the state space of turbulence using machine learning,” Physical Review Fluids, vol. 6, no. 3, p. 034402, 2021.
  * [33] B. Lusch, J. N. Kutz, and S. L. Brunton, “Deep learning for universal linear embeddings of nonlinear dynamics,” Nature communications, vol. 9, no. 1, p. 4950, 2018.
  * [34] S. Shamekh, K. D. Lamb, Y. Huang, and P. Gentine, “Implicit learning of convective organization explains precipitation stochasticity,” Proceedings of the National Academy of Sciences, vol. 120, no. 20, p. e2216158120, 2023.
  * [35] M. Reichstein, G. Camps-Valls, B. Stevens, M. Jung, J. Denzler, N. Carvalhais, and f. Prabhat, “Deep learning and process understanding for data-driven earth system science,” Nature, vol. 566, no. 7743, pp. 195–204, 2019.
  * [36] J. C. Landy, G. J. Dawson, M. Tsamados, M. Bushuk, J. C. Stroeve, S. E. Howell, T. Krumpen, D. G. Babb, A. S. Komarov, H. D. Heorton, et al., “A year-round satellite sea-ice thickness record from cryosat-2,” Nature, vol. 609, no. 7927, pp. 517–522, 2022.
  * [37] S. A. Martin, G. Manucharyan, and P. Klein, “Deep learning improves global satellite observations of ocean eddy dynamics,” 2024.
  * [38] S. Rezvanbehbahani, L. A. Stearns, R. Keramati, S. Shankar, and C. van der Veen, “Significant contribution of small icebergs to the freshwater budget in greenland fjords,” Communications Earth & Environment, vol. 1, no. 1, p. 31, 2020.
  * [39] T. Surawy-Stepney, A. E. Hogg, S. L. Cornford, and B. J. Davison, “Episodic dynamic change linked to damage on the thwaites glacier ice tongue,” Nature Geoscience, vol. 16, no. 1, pp. 37–43, 2023.
  * [40] C.-Y. Lai, J. Kingslake, M. G. Wearing, P.-H. C. Chen, P. Gentine, H. Li, J. J. Spergel, and J. M. van Wessem, “Vulnerability of antarctica’s ice shelves to meltwater-driven fracture,” Nature, vol. 584, pp. 574–578, 2020.
  * [41] E. N. Lorenz, “Deterministic nonperiodic flow,” Journal of atmospheric sciences, vol. 20, no. 2, pp. 130–141, 1963.
  * [42] A. Tarantola, Inverse problem theory and methods for model parameter estimation.  SIAM, 2005.
  * [43] E. Kalnay, Atmospheric modeling, data assimilation and predictability.  Cambridge university press, 2003.
  * [44] A. J. Geer, “Learning earth system models from observations: machine learning or data assimilation?,” Philosophical Transactions of the Royal Society A, vol. 379, no. 2194, p. 20200089, 2021.
  * [45] J. Brajard, A. Carrassi, M. Bocquet, and L. Bertino, “Combining data assimilation and machine learning to infer unresolved scale parametrization,” Philosophical Transactions of the Royal Society A, vol. 379, no. 2194, p. 20200086, 2021.
  * [46] S. Cheng, C. Quilodrán-Casas, S. Ouala, A. Farchi, C. Liu, P. Tandeo, R. Fablet, D. Lucor, B. Iooss, J. Brajard, et al., “Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review,” IEEE/CAA Journal of Automatica Sinica, vol. 10, no. 6, pp. 1361–1387, 2023.
  * [47] A. Farchi, P. Laloyaux, M. Bonavita, and M. Bocquet, “Using machine learning to correct model error in data assimilation and forecast applications,” Quarterly Journal of the Royal Meteorological Society, vol. 147, no. 739, pp. 3067–3084, 2021.
  * [48] J. Brajard, A. Carrassi, M. Bocquet, and L. Bertino, “Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: A case study with the lorenz 96 model,” Journal of computational science, vol. 44, p. 101171, 2020.
  * [49] M. A. Iglesias, K. J. Law, and A. M. Stuart, “Ensemble kalman methods for inverse problems,” Inverse Problems, vol. 29, no. 4, p. 045001, 2013.
  * [50] E. Cleary, A. Garbuno-Inigo, S. Lan, T. Schneider, and A. M. Stuart, “Calibrate, emulate, sample,” Journal of Computational Physics, vol. 424, p. 109716, 2021.
  * [51] O. R. Dunbar, A. Garbuno-Inigo, T. Schneider, and A. M. Stuart, “Calibration and uncertainty quantification of convective parameters in an idealized gcm,” Journal of Advances in Modeling Earth Systems, vol. 13, no. 9, p. e2020MS002454, 2021.
  * [52] I. Lopez-Gomez, C. Christopoulos, H. L. Langeland Ervik, O. R. Dunbar, Y. Cohen, and T. Schneider, “Training physics-based machine-learning parameterizations with gradient-free ensemble kalman methods,” Journal of Advances in Modeling Earth Systems, vol. 14, no. 8, p. e2022MS003105, 2022.
  * [53] L. Mansfield and A. Sheshadri, “Calibration and uncertainty quantification of a gravity wave parameterization: A case study of the quasi-biennial oscillation in an intermediate complexity climate model,” Journal of Advances in Modeling Earth Systems, vol. 14, no. 11, p. e2022MS003245, 2022.
  * [54] A. N. Souza, G. Wagner, A. Ramadhan, B. Allen, V. Churavy, J. Schloss, J. Campin, C. Hill, A. Edelman, J. Marshall, et al., “Uncertainty quantification of ocean parameterizations: Application to the k-profile-parameterization for penetrative convection,” Journal of Advances in Modeling Earth Systems, vol. 12, no. 12, p. e2020MS002108, 2020.
  * [55] G. Evensen, “Sequential data assimilation with a nonlinear quasi-geostrophic model using monte carlo methods to forecast error statistics,” Journal of Geophysical Research: Oceans, vol. 99, no. C5, pp. 10143–10162, 1994.
  * [56] P. L. Houtekamer and F. Zhang, “Review of the ensemble kalman filter for atmospheric data assimilation,” Monthly Weather Review, vol. 144, no. 12, pp. 4489–4532, 2016.
  * [57] N. B. Kovachki and A. M. Stuart, “Ensemble kalman inversion: a derivative-free technique for machine learning tasks,” Inverse Problems, vol. 35, no. 9, p. 095005, 2019.
  * [58] D. Watson-Parris, A. Williams, L. Deaconu, and P. Stier, “Model calibration using esem v1. 1.0–an open, scalable earth system emulator,” Geoscientific Model Development, vol. 14, no. 12, pp. 7659–7672, 2021.
  * [59] M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” Journal of Computational physics, vol. 378, pp. 686–707, 2019.
  * [60] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, “Physics-informed machine learning,” Nature Reviews Physics, vol. 3, no. 6, pp. 422–440, 2021.
  * [61] R. Eusebi, G. A. Vecchi, C.-Y. Lai, and M. Tong, “Realistic tropical cyclone wind and pressure fields can be reconstructed from sparse data using deep learning,” Communications Earth & Environment, vol. 5, no. 1, p. 8, 2024.
  * [62] Y. Wang, C.-Y. Lai, and C. Cowen-Breen, “Discovering the rheology of antarctic ice shelves via physics-informed deep learning,” 2022.
  * [63] X. Lu, X. Wang, M. Tong, and V. Tallapragada, “Gsi-based, continuously cycled, dual-resolution hybrid ensemble–variational data assimilation system for hwrf: System description and experiments with edouard (2014),” Monthly Weather Review, vol. 145, no. 12, pp. 4877–4898, 2017.
  * [64] D. Kochkov, J. Yuval, I. Langmore, P. Norgaard, J. Smith, G. Mooers, J. Lottes, S. Rasp, P. Düben, M. Klöwer, S. Hatfield, P. Battaglia, A. Sanchez-Gonzalez, M. Willson, M. P. Brenner, and S. Hoyer, “Neural general circulation models,” 2023.
  * [65] G. Jouvet and G. Cordonnier, “Ice-flow model emulator based on physics-informed deep learning,” Journal of Glaciology, pp. 1–15, 2023.
  * [66] M. Schmidt and H. Lipson, “Distilling free-form natural laws from experimental data,” science, vol. 324, no. 5923, pp. 81–85, 2009.
  * [67] S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Discovering governing equations from data by sparse identification of nonlinear dynamical systems,” Proceedings of the national academy of sciences, vol. 113, no. 15, pp. 3932–3937, 2016.
  * [68] T. Schneider, A. M. Stuart, and J.-L. Wu, “Ensemble kalman inversion for sparse learning of dynamical systems from time-averaged data,” Journal of Computational Physics, vol. 470, p. 111559, 2022.
  * [69] M. Lang, P. Jan Van Leeuwen, and P. Browne, “A systematic method of parameterisation estimation using data assimilation,” Tellus A: Dynamic Meteorology and Oceanography, vol. 68, no. 1, p. 29012, 2016.
  * [70] R. Mojgani, A. Chattopadhyay, and P. Hassanzadeh, “Interpretable structural model error discovery from sparse assimilation increments using spectral bias-reduced neural networks: A quasi-geostrophic turbulence test case,” Journal of Advances in Modeling Earth Systems, 2024.
  * [71] N. Chen and Y. Zhang, “A causality-based learning approach for discovering the underlying dynamics of complex systems from partial observations with stochastic parameterization,” Physica D: Nonlinear Phenomena, vol. 449, p. 133743, 2023.
  * [72] L. Zanna and T. Bolton, “Data-driven equation discovery of ocean mesoscale closures,” Geophysical Research Letters, vol. 47, no. 17, p. e2020GL088376, 2020.
  * [73] J. A. Anstey and L. Zanna, “A deformation-based parametrization of ocean mesoscale eddy reynolds stresses,” Ocean Modelling, vol. 112, pp. 99–111, 2017.
  * [74] K. Jakhar, Y. Guan, R. Mojgani, A. Chattopadhyay, P. Hassanzadeh, and L. Zanna, “Learning closed-form equations for subgrid-scale closures from high-fidelity data: Promises and challenges,” arXiv preprint arXiv:2306.05014, 2023.
  * [75] J. R. Koza, “Genetic programming as a means for programming computers by natural selection,” Statistics and computing, vol. 4, pp. 87–112, 1994.
  * [76] Y. Chen, Y. Luo, Q. Liu, H. Xu, and D. Zhang, “Symbolic genetic algorithm for discovering open-form partial differential equations (sga-pde),” Physical Review Research, vol. 4, no. 2, p. 023174, 2022.
  * [77] A. Grundner, T. Beucler, P. Gentine, and V. Eyring, “Data-driven equation discovery of a cloud cover parameterization,” Journal of Advances in Modeling Earth Systems, vol. 16, no. 3, p. e2023MS003763, 2024.
  * [78] A. Ross, Z. Li, P. Perezhogin, C. Fernandez-Granda, and L. Zanna, “Benchmarking of machine learning ocean subgrid parameterizations in an idealized model,” Journal of Advances in Modeling Earth Systems, vol. 15, no. 1, 2023.
  * [79] E. Hawkins and R. Sutton, “The potential to narrow uncertainty in regional climate predictions,” Bulletin of the American Meteorological Society, vol. 90, no. 8, pp. 1095–1108, 2009.
  * [80] J. Smagorinsky, “General circulation experiments with the primitive equations: I. the basic experiment,” Monthly weather review, vol. 91, no. 3, pp. 99–164, 1963.
  * [81] Y. Guan, A. Chattopadhyay, A. Subel, and P. Hassanzadeh, “Stable a posteriori les of 2d turbulence using convolutional neural networks: Backscattering analysis and generalization to higher re via transfer learning,” Journal of Computational Physics, vol. 458, p. 111090, 2022.
  * [82] T. Bolton and L. Zanna, “Applications of deep learning to ocean data inference and subgrid parameterization,” Journal of Advances in Modeling Earth Systems, vol. 11, no. 1, pp. 376–399, 2019.
  * [83] A. Sane, B. G. Reichl, A. Adcroft, and L. Zanna, “Parameterizing vertical mixing coefficients in the ocean surface boundary layer using neural networks,” Journal of Advances in Modeling Earth Systems, vol. 15, no. 10, p. e2023MS003890, 2023.
  * [84] P. Gentine, M. Pritchard, S. Rasp, G. Reinaudi, and G. Yacalis, “Could machine learning break the convection parameterization deadlock?,” Geophysical Research Letters, vol. 45, no. 11, pp. 5742–5751, 2018.
  * [85] P. Gentine, V. Eyring, and T. Beucler, “Deep learning for the parametrization of subgrid processes in climate models,” Deep Learning for the Earth Sciences: A Comprehensive Approach to Remote Sensing, Climate Science, and Geosciences, pp. 307–314, 2021.
  * [86] J. Yuval and P. A. O’Gorman, “Neural-network parameterization of subgrid momentum transport in the atmosphere,” Journal of Advances in Modeling Earth Systems, vol. 15, no. 4, p. e2023MS003606, 2023.
  * [87] S. Rasp, M. S. Pritchard, and P. Gentine, “Deep learning to represent subgrid processes in climate models,” Proceedings of the National Academy of Sciences, vol. 115, no. 39, pp. 9684–9689, 2018.
  * [88] A. Grundner, T. Beucler, P. Gentine, F. Iglesias-Suarez, M. A. Giorgetta, and V. Eyring, “Deep learning based cloud cover parameterization for icon,” Journal of Advances in Modeling Earth Systems, vol. 14, no. 12, p. e2021MS002959, 2022.
  * [89] T. Arcomano, I. Szunyogh, A. Wikner, B. R. Hunt, and E. Ott, “A hybrid atmospheric model incorporating machine learning can capture dynamical processes not captured by its physics-based component,” Geophysical Research Letters, vol. 50, no. 8, p. e2022GL102649, 2023.
  * [90] O. Watt-Meyer, N. D. Brenowitz, S. K. Clark, B. Henn, A. Kwa, J. McGibbon, W. A. Perkins, L. Harris, and C. S. Bretherton, “Neural network parameterization of subgrid-scale physics from a realistic geography global storm-resolving simulation,” Journal of Advances in Modeling Earth Systems, vol. 16, no. 2, p. e2023MS003668, 2024.
  * [91] D. Matsuoka, S. Watanabe, K. Sato, S. Kawazoe, W. Yu, and S. Easterbrook, “Application of deep learning to estimate atmospheric gravity wave parameters in reanalysis data sets,” Geophysical Research Letters, vol. 47, no. 19, p. e2020GL089436, 2020.
  * [92] Z. I. Espinosa, A. Sheshadri, G. R. Cain, E. P. Gerber, and K. J. DallaSanta, “Machine learning gravity wave parameterization generalizes to capture the qbo and response to increased co2,” Geophysical Research Letters, vol. 49, no. 8, p. e2022GL098174, 2022.
  * [93] S. C. Hardiman, A. A. Scaife, A. van Niekerk, R. Prudden, A. Owen, S. V. Adams, T. Dunstan, N. J. Dunstone, and S. Madge, “Machine learning for nonorographic gravity waves in a climate model,” Artificial intelligence for the Earth systems, vol. 2, no. 4, p. e220081, 2023.
  * [94] H. A. Pahlavan, P. Hassanzadeh, and M. J. Alexander, “Explainable offline-online training of neural networks for parameterizations: A 1d gravity wave-qbo testbed in the small-data regime,” Geophysical Research Letters, vol. 51, no. 2, p. e2023GL106324, 2024.
  * [95] H. Frezat, J. Le Sommer, R. Fablet, G. Balarac, and R. Lguensat, “A posteriori learning for quasi-geostrophic turbulence parametrization,” Journal of Advances in Modeling Earth Systems, vol. 14, no. 11, p. e2022MS003124, 2022.
  * [96] L. Beusch, L. Gudmundsson, and S. I. Seneviratne, “Emulating earth system model temperatures with mesmer: from global mean temperature trajectories to grid-point-level realizations on land,” Earth System Dynamics, vol. 11, no. 1, pp. 139–159, 2020.
  * [97] C. Tebaldi, A. Snyder, and K. Dorheim, “Stitches: creating new scenarios of climate model output by stitching together pieces of existing simulations,” Earth System Dynamics, vol. 13, no. 4, pp. 1557–1609, 2022.
  * [98] D. Watson-Parris, Y. Rao, D. Olivié, Ø. Seland, P. Nowack, G. Camps-Valls, P. Stier, S. Bouabid, M. Dewey, E. Fons, et al., “Climatebench v1. 0: A benchmark for data-driven climate projections,” Journal of Advances in Modeling Earth Systems, vol. 14, no. 10, p. e2021MS002954, 2022.
  * [99] H. Hersbach, B. Bell, P. Berrisford, S. Hirahara, A. Horányi, J. Muñoz-Sabater, J. Nicolas, C. Peubey, R. Radu, D. Schepers, et al., “The era5 global reanalysis,” Quarterly Journal of the Royal Meteorological Society, vol. 146, no. 730, pp. 1999–2049, 2020.
  * [100] O. Watt-Meyer, G. Dresdner, J. McGibbon, S. K. Clark, B. Henn, J. Duncan, N. D. Brenowitz, K. Kashinath, M. S. Pritchard, B. Bonev, et al., “Ace: A fast, skillful learned global atmospheric model for climate prediction,” arXiv preprint arXiv:2310.02074, 2023.
  * [101] J. P. Duncan, E. Wu, J.-C. Golaz, P. M. Caldwell, O. Watt-Meyer, S. K. Clark, J. McGibbon, G. Dresdner, K. Kashinath, B. Bonev, et al., “Application of the ai2 climate emulator to e3smv2’s global atmosphere model, with a focus on precipitation fidelity,” Authorea Preprints, 2024.
  * [102] S. Bire, B. Lütjens, K. Azizzadenesheli, A. Anandkumar, and C. N. Hill, “Ocean emulation with fourier neural operators: Double gyre,” Authorea Preprints, 2023.
  * [103] A. Subel and L. Zanna, “Building ocean climate emulators,” arXiv preprint arXiv:2402.04342, 2024.
  * [104] T. R. Andersson, J. S. Hosking, M. Pérez-Ortiz, B. Paige, A. Elliott, C. Russell, S. Law, D. C. Jones, J. Wilkinson, T. Phillips, et al., “Seasonal arctic sea ice forecasting with probabilistic deep learning,” Nature communications, vol. 12, no. 1, p. 5124, 2021.
  * [105] Y. Wang, X. Yuan, Y. Ren, M. Bushuk, Q. Shu, C. Li, and X. Li, “Subseasonal prediction of regional antarctic sea ice by a deep learning model,” Geophysical Research Letters, vol. 50, no. 17, p. e2023GL104347, 2023.
  * [106] Y. Zhu, M. Qin, P. Dai, S. Wu, Z. Fu, Z. Chen, L. Zhang, Y. Wang, and Z. Du, “Deep learning-based seasonal forecast of sea ice considering atmospheric conditions,” Journal of Geophysical Research: Atmospheres, vol. 128, no. 24, p. e2023JD039521, 2023.
  * [107] A. Chattopadhyay and P. Hassanzadeh, “Long-term instabilities of deep learning-based digital twins of the climate system: The cause and a solution,” arXiv preprint arXiv:2304.07029, 2023.
  * [108] T. Selz and G. C. Craig, “Can artificial intelligence-based weather prediction models simulate the butterfly effect?,” Geophysical Research Letters, vol. 50, no. 20, p. e2023GL105747, 2023.
  * [109] K. Kashinath, M. Mustafa, J. Wu, C. Jiang, R. Wang, A. Chattopadhyay, A. Singh, et al., “Physics-informed machine learning: case studies for weather and climate modelling,” Philosophical Transactions of the Royal Society A, vol. 379, no. 2194, p. 20200093, 2021.
  * [110] T. Beucler, M. Pritchard, S. Rasp, J. Ott, P. Baldi, and P. Gentine, “Enforcing analytic constraints in neural networks emulating physical systems,” Physical Review Letters, vol. 126, no. 9, p. 098302, 2021.
  * [111] A. Chattopadhyay, M. Mustafa, P. Hassanzadeh, E. Bach, and K. Kashinath, “Towards physics-inspired data-driven weather forecasting: integrating data assimilation with a deep spatial-transformer-based u-net in a case study with era5,” Geoscientific Model Development, vol. 15, no. 5, pp. 2221–2237, 2022.
  * [112] Y. Guan, A. Subel, A. Chattopadhyay, and P. Hassanzadeh, “Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate les,” Physica D: Nonlinear Phenomena, vol. 443, p. 133568, 2023.
  * [113] A. F. Psaros, X. Meng, Z. Zou, L. Guo, and G. E. Karniadakis, “Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons,” Journal of Computational Physics, vol. 477, p. 111902, 2023.
  * [114] M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, P. Fieguth, X. Cao, A. Khosravi, U. R. Acharya, et al., “A review of uncertainty quantification in deep learning: Techniques, applications and challenges,” Information fusion, vol. 76, pp. 243–297, 2021.
  * [115] M. Sonnewald and R. Lguensat, “Revealing the impact of global heating on north atlantic circulation using transparent machine learning,” Journal of Advances in Modeling Earth Systems, vol. 13, no. 8, p. e2021MS002496, 2021.
  * [116] W. Yik, M. Sonnewald, M. C. A. Clare, and R. Lguensat, “Southern Ocean Dynamics Under Climate Change: New Knowledge Through Physics-Guided Machine Learning,” arXiv e-prints, p. arXiv:2310.13916, Oct. 2023.
  * [117] L. A. Mansfield and A. Sheshadri, “Uncertainty Quantification of a Machine Learning Subgrid-Scale Parameterization for Atmospheric Gravity Waves,” ESS Open Archive, Feb. 2024.
  * [118] Y. Q. Sun, H. A. Pahlavan, A. Chattopadhyay, P. Hassanzadeh, S. W. Lubis, M. J. Alexander, E. Gerber, A. Sheshadri, and Y. Guan, “Data imbalance, uncertainty quantification, and generalization via transfer learning in data-driven parameterizations: Lessons from the emulation of gravity wave momentum transport in waccm,” arXiv preprint arXiv:2311.17078, 2023.
  * [119] S. Dräger and M. Sonnewald, “The Importance of Architecture Choice in Deep Learning for Climate Applications,” arXiv e-prints, p. arXiv:2402.13979, Feb. 2024.
  * [120] A. P. Guillaumin and L. Zanna, “Stochastic-deep learning parameterization of ocean momentum forcing,” Journal of Advances in Modeling Earth Systems, vol. 13, no. 9, p. e2021MS002534, 2021.
  * [121] D. Foster, D. J. Gagne, and D. B. Whitt, “Probabilistic machine learning estimation of ocean mixed layer depth from dense satellite and sparse in situ observations,” Journal of Advances in Modeling Earth Systems, vol. 13, no. 12, p. e2021MS002474, 2021.
  * [122] E. A. Barnes and R. J. Barnes, “Controlled abstention neural networks for identifying skillful predictions for regression problems,” Journal of Advances in Modeling Earth Systems, vol. 13, no. 12, p. e2021MS002575, 2021.
  * [123] K. Haynes, R. Lagerquist, M. McGraw, K. Musgrave, and I. Ebert-Uphoff, “Creating and evaluating uncertainty estimates with neural networks for environmental-science applications,” Artificial Intelligence for the Earth Systems, vol. 2, no. 2, p. 220061, 2023.
  * [124] N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, and A. Courville, “On the spectral bias of neural networks,” in International conference on machine learning, pp. 5301–5310, PMLR, 2019.
  * [125] Z.-Q. J. Xu, Y. Zhang, T. Luo, Y. Xiao, and Z. Ma, “Frequency principle: Fourier analysis sheds light on deep neural networks,” arXiv preprint arXiv:1901.06523, 2019.
  * [126] A. Rybchuk, M. Hassanaly, N. Hamilton, P. Doubrawa, M. J. Fulton, and L. A. Martínez-Tossas, “Ensemble flow reconstruction in the atmospheric boundary layer from spatially limited measurements through latent diffusion models,” Physics of Fluids, vol. 35, no. 12, 2023.
  * [127] M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng, “Fourier features let networks learn high frequency functions in low dimensional domains,” Advances in neural information processing systems, vol. 33, pp. 7537–7547, 2020.
  * [128] S. Wang, H. Wang, and P. Perdikaris, “On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks,” Computer Methods in Applied Mechanics and Engineering, vol. 384, p. 113938, 2021.
  * [129] Y. Wang and C.-Y. Lai, “Multi-stage neural networks: Function approximator of machine precision,” Journal of Computational Physics, p. 112865, 2024.
  * [130] G. Miloshevich, B. Cozian, P. Abry, P. Borgnat, and F. Bouchet, “Probabilistic forecasts of extreme heatwaves using convolutional neural networks in a regime of lack of data,” Physical Review Fluids, vol. 8, no. 4, p. 040501, 2023.
  * [131] I. Lopez-Gomez, A. McGovern, S. Agrawal, and J. Hickey, “Global extreme heat forecasting using neural weather models,” Artificial Intelligence for the Earth Systems, no. 1, p. e220035, 2023.
  * [132] S. H. Rudy and T. P. Sapsis, “Output-weighted and relative entropy loss functions for deep learning precursors of extreme events,” Physica D: Nonlinear Phenomena, vol. 443, p. 133570, 2023.
  * [133] F. Ragone, J. Wouters, and F. Bouchet, “Computation of extreme heat waves in climate models using a large deviation algorithm,” Proceedings of the National Academy of Sciences, vol. 115, no. 1, pp. 24–29, 2018.
  * [134] J. Finkel, R. J. Webber, E. P. Gerber, D. S. Abbot, and J. Weare, “Learning forecasts of rare stratospheric transitions from short simulations,” Monthly Weather Review, vol. 149, no. 11, pp. 3647–3669, 2021.
  * [135] A. Subel, Y. Guan, A. Chattopadhyay, and P. Hassanzadeh, “Explaining the physics of transfer learning in data-driven turbulence modeling,” PNAS nexus, vol. 2, no. 3, p. pgad015, 2023.
  * [136] T. Beucler, P. Gentine, J. Yuval, A. Gupta, L. Peng, J. Lin, S. Yu, S. Rasp, F. Ahmed, P. A. O’Gorman, et al., “Climate-invariant machine learning,” Science Advances, vol. 10, no. 6, p. eadj7250, 2024.
  * [137] Z. Shen, A. Sridhar, Z. Tan, A. Jaruga, and T. Schneider, “A library of large-eddy simulations forced by global climate models,” Journal of Advances in Modeling Earth Systems, vol. 14, no. 3, p. e2021MS002631, 2022.
  * [138] Y. Q. Sun, P. Hassanzadeh, M. J. Alexander, and C. G. Kruse, “Quantifying 3d gravity wave drag in a library of tropical convection-permitting simulations for data-driven parameterizations,” Journal of Advances in Modeling Earth Systems, vol. 15, no. 5, p. e2022MS003585, 2023.
  * [139] M. Satoh, B. Stevens, F. Judt, M. Khairoutdinov, S.-J. Lin, W. M. Putman, and P. Düben, “Global cloud-resolving models,” Current Climate Change Reports, vol. 5, pp. 172–184, 2019.
  * [140] C. Irrgang, N. Boers, M. Sonnewald, E. A. Barnes, C. Kadow, J. Staneva, and J. Saynisch‐Wagner, “Towards neural earth system modelling by integrating artificial intelligence in earth system science,” Nature Machine Intelligence, vol. 3, pp. 667 – 674, 2021.
  * [141] M. Sonnewald, R. Lguensat, D. C. Jones, P. D. Dueben, J. Brajard, and V. Balaji, “Bridging observations, theory and numerical simulation of the ocean using machine learning,” Environmental Research Letters, vol. 16, p. 073008, jul 2021.
  * [142] P. Bommer, M. Kretschmer, A. Hedström, D. Bareeva, and M. M. C. Höhne, “Finding the right xai method – a guide for the evaluation and ranking of explainable ai methods in climate science,” 2024.
  * [143] M. Flora, C. Potvin, A. Mcgovern, and S. Handler, “A machine learning explainability tutorial for atmospheric sciences,” Artificial Intelligence for the Earth Systems, vol. 3, 11 2023.
  * [144] A. Mamalakis, E. A. Barnes, and I. Ebert-Uphoff, “Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience,” Artificial Intelligence for the Earth Systems.
  * [145] K. J. Mayer and E. A. Barnes, “Subseasonal forecasts of opportunity identified by an explainable neural network,” Geophysical Research Letters, vol. 48, no. 10, p. e2020GL092092, 2021.
  * [146] B. A. Toms, E. A. Barnes, and I. Ebert-Uphoff, “Physically interpretable neural networks for the geosciences: Applications to earth system variability,” Journal of Advances in Modeling Earth Systems, vol. 12, no. 9, p. e2019MS002002, 2020.
  * [147] M. Farge, “Wavelet transforms and their applications to turbulence,” Annual review of fluid mechanics, vol. 24, no. 1, pp. 395–458, 1992.
  * [148] S. Mallat, “Understanding deep convolutional networks,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 374, no. 2065, p. 20150203, 2016.
  * [149] B. A. Olshausen and D. J. Field, “Emergence of simple-cell receptive field properties by learning a sparse code for natural images,” Nature, vol. 381, no. 6583, pp. 607–609, 1996.
  * [150] V. Balaji, “Climbing down charney’s ladder: machine learning and the post-dennard era of computational climate science,” Philosophical Transactions of the Royal Society A, vol. 379, no. 2194, p. 20200085, 2021.