# Knowledge-guided Machine Learning: Current Trends and Future Prospects

Anuj Karpatne  Xiaowei Jia  Vipin Kumar<EMAIL_ADDRESS>Department of
Computer Science, Virginia Tech<EMAIL_ADDRESS>Department of Computer
Science, University of Pittsburgh<EMAIL_ADDRESS>Department of Computer
Science and Engineering, University of Minnesota,

###### Abstract

This paper presents an overview of scientific modeling and discusses the
complementary strengths and weaknesses of ML methods for scientific modeling
in comparison to process-based models. It also provides an introduction to the
current state of research in the emerging field of scientific knowledge-guided
machine learning (KGML) that aims to use both scientific knowledge and data in
ML frameworks to achieve better generalizability, scientific consistency, and
explainability of results. We discuss different facets of KGML research in
terms of the type of scientific knowledge used, the form of knowledge-ML
integration explored, and the method for incorporating scientific knowledge in
ML. We also discuss some of the common categories of use cases in
environmental sciences where KGML methods are being developed, using
illustrative examples in each category.

##  1 Introduction

As advances in artificial intelligence (AI) and machine learning (ML) continue
to revolutionize mainstream applications in commercial problems, there is a
growing excitement in the scientific community to harness the power of ML for
accelerating scientific discovery [151, 181, 86, 1]. This is especially true
in environmental sciences that are rapidly transitioning from being data-poor
to data-rich, e.g., with the ever-increasing volumes of environmental data
being collected by Earth observing satellites, in-situ sensors, and those
generated by model simulations (e.g., climate model runs [113]). Similar to
how recent developments in ML has transformed how we interact with the
information on the Internet, it is befitting to ask how ML advances can enable
Earth system scientists to transform a fundamental goal in science, which is
to build better models of physical, biological, and environmental systems.

The conventional approach for modeling relationships between input drivers and
response variables is to use process-based models rooted in scientific
equations. Despite their ability to leverage the mechanistic understanding of
scientific phenomena, process-based models suffer from several shortcomings
limiting their adoption in complex real-world settings, e.g., due to
imperfections in model formulations (or modeling bias), incorrect choices of
parameter values in equations, and high computational costs in running high-
fidelity simulations. In response to these challenges, ML methods offer a
promising alternative to capture statistical relationships between inputs and
outputs directly from data. However, “black-box” ML models, that solely rely
on the supervision contained in data, show limited generalizability in
scientific problems, especially when applied to out-of-distribution data. One
of the reasons for this lack of generalizability is the limited scale of data
in scientific disciplines in contrast to mainstream applications of AI and ML
where large-scale datasets in computer vision and natural language modeling
have been instrumental in the success of state-of-the-art AI/ML models.
Another fundamental deficiency in black-box ML models is their tendency to
produce results that are inconsistent with existing scientific theories and
their inability to provide a mechanistic understanding of discovered patterns
and relationships from data, limiting their usefulness in science.

In contrast to black-box ML, a new generation of ML methods is being developed
that leverage the information contained in data without ignoring the wealth of
knowledge available in scientific disciplines. This emerging field of research
in ML is referred to as scientific knowledge-guided machine learning (KGML)
[77] where scientific theories (e.g., the principle of mass and energy
conservation) are used to guide the construction and training of ML models for
improved generalizability and scientific consistency. Even though the idea of
incorporating scientific knowledge in ML is relatively recent [37, 77, 129],
this area has seen a surge in interest that is reflected in the large volume
of papers published just in the last few years on topics related to KGML in a
wide range of physical, biological, and environmental applications (see a
recent book [78] and survey articles [163, 76, 148]).

This paper aims to accomplish the following goals. First, it provides the ML
community with a gentle introduction to the field of scientific modeling and
discusses complementary strengths and weaknesses of process-based and ML
models (Section 2). Second, this paper provides an easy entry point to the
field of KGML for a broad research audience, by summarizing prior research in
the field in a simple and accessible format using a three-dimensional view,
namely the type of scientific knowledge used, the form of knowledge-ML
integration deployed, and the method for incorporating scientific knowledge in
ML (Section 3). Third, it provides a detailed view of three major research
methods for integrating scientific knowledge in ML frameworks (Section 4).
Fourth, it describes research in KGML from the context of four primary use-
cases in scientific disciplines: forward modeling, inverse modeling,
generative modeling, and downscaling (Section 5). Finally, it highlights
emerging opportunities for KGML research in building foundation models in
environmental science (Section 6), and provides concluding remarks and future
directions of research in this field (Section 7).

Figure 1: Generic framework for scientific modeling using process-based models

##  2 Background on Scientific Modeling: Opportunities and Challenges

One of the fundamental pursuits in science is to understand the nature of the
physical world by building computational models that are capable of predicting
(or simulating) the response of physical/environmental systems. Figure 1
provides a brief overview of the generic framework of scientific modeling,
where the goal is to model relationships between input drivers and response
variables. For example, consider the problem of modeling lake water quality.
Here, we are interested in capturing relationships between input drivers such
as the temperature of air above the lake and the net flow of nutrients to the
lake (e.g., through agricultural discharge), and response variables of the
lake such as the temperature of the water and concentrations of chemical and
biological constituents such as dissolved oxygen and chlorophyll content. Note
that both the input drivers and response variables can have spatial and
temporal structures and can be represented as spatio-temporal rasters
(available on regular grids), time-varying graphs (available on irregular
grids), or time series. Environmental data for both driver and response
variables are generally collected through ground-based sensors, satellites, or
simulation runs of other process-based models. Further, while we expect the
input drivers to be relatively well-observed, the response variables are
generally hard to measure directly and thus have a limited number of
observations at desired spatial and temporal scales. By modeling relationships
between input drivers and response variables, an important end-goal of
scientific modeling is to improve our understanding of scientific systems in
novel (or unseen) testing scenarios, where the distributions of input and
response variables are different than those observed while building the model.

### Process-based Models

The conventional approach for scientific modeling is to use process-based
models (also referred to as physics-based or science-based models), where the
solution structure of relationships between inputs and response variables is
rooted in scientific equations (e.g., laws of energy and mass conservation).
In particular, process-based models make use of scientific equations to infer
the evolution of time-varying latent or hidden variables of the system, also
referred to as system states (see Figure 1). Additionally, process-based
models often involve model parameters that need to be specified or calibrated
(often with the help of observational data) for approximating real-world
phenomena. For example, state-of-the-art models in lake modeling such as the
general lake model (GLM) [62] involve parameters such as water clarity that
can dramatically change the nature of relationships between inputs and outputs
and have to be tuned for every lake based on observations of input-output
pairs. Taken together, the conventional paradigm of scientific
modeling—developed over centuries of systematic research—forms the foundation
of our present-day understanding of scientific systems across a wide spectrum
of problems in environmental sciences.

A key feature of process-based models is their ability to provide a
mechanistic understanding of the cause-effect mechanisms between input and
output variables that can be used as a building block for advancing scientific
knowledge. As a result, process-based models are continually updated and
improved by the scientific community to fill knowledge gaps in current
modeling standards and discover new theories and formulations of scientific
equations that better match with observations and are scientifically
meaningful and explainable. Since process-based models are rooted in
scientific equations that are assumed to hold true in any testing scenario,
they are also expected to easily generalize even outside the data used during
model building. For example, process-based models can be made to extrapolate
in space (e.g., over different geographic regions), in time (e.g., forecasting
the future of a system under varying forcings of input drivers), or in scale
(e.g., discovering emergent properties of larger-scale systems using models of
smaller-scale system components).

### Limitations of Process-based Models

Despite the widespread use of process-based models in almost every domain of
science, they suffer from certain research gaps limiting their effectiveness
in many real-world scientific applications. First, process-based models often
involve several parameters that have to be calibrated for every new system
using observations specific to the system. For example, in the problem of
modeling the quality of water in lakes, there are several hundreds of
parameters in process-based models such as GLM-Aquatic EcoDynamics (GLM-AED)
[61] capturing different aspects of lake thermodynamics and
chemical/biological processes that need to be calibrated for every lake using
observational data. While it is still possible to calibrate model parameters
in highly-observed systems with a sufficient number of data samples, process-
based models are difficult to generalize in sparsely-observed systems where
the underlying model parameters can be difficult to estimate using limited (or
sometimes non-existent) observations. For example, we have a sufficiently good
number of observations in a few highly-monitored lakes but for the vast
population of lakes in the U.S. and across the world, we do not have dedicated
programs for high-resolution monitoring of lake water quality, making it
challenging to calibrate process-based models with little to no data.

Second, process-based models frequently suffer from several imperfections (or
bias) in their solution structure (e.g., due to incomplete understanding of
the system or approximations introduced in the model to speed up model runs),
leading to poor generalizability of model outputs. For example, while state-
of-the-art models in lake modeling such as the GLM [62] adequately capture the
thermodynamic processes responsible for mass and energy balance in lakes, the
processes governing the biogeochemistry of lake nutrients such as dissolved
oxygen, organic matter, and phytoplankton counts are less well-understood.
Such processes are generally approximated using over-parameterized forms of
weakly-informed equations, introducing large modeling errors and
uncertainties. Furthermore, the introduced over-parametrizations are often
difficult to understand and relate to physically meaningful quantities, making
them fall short in serving the goal of explainability.

Third, process-based models are often computationally expensive to be run at
the required scales in operational settings. In particular, in many scientific
problems, learning the ‘forward’ mapping from input drivers to response
variables (also referred to as the problem of forward modeling) requires
finding solutions to complicated scientific equations often involving complex
numerical operations. On the other hand, learning the ‘inverse mapping’ from
observed inputs and outputs to model parameters (referred to as the problem of
inverse modeling) is also a computationally demanding process as it requires
multiple runs of the forward model for every round of model inversion [142,
106, 10]. For example, in the problem of lake modeling, custom-calibrating the
GLM model is both labor- and computation-intensive, and hence there is a
trade-off between increasing the accuracy of GLM estimates and expanding the
feasibility of study to a large number of lakes.

### Opportunities for ML in Scientific Modeling

To address the research gaps of process-based models in scientific modeling,
ML methods offer a promising alternative to capture relationships between
input drivers and output response variables directly from data. ML models are
similar to process-based models in many ways. For example, both ML and
process-based models pursue the common goal of mapping inputs to outputs. Just
like the parameters of process-based models, many ML models also involve
several learnable parameters (e.g., the weights and biases of deep neural
networks) that have to be inferred from data often using computationally
efficient optimization techniques such as first-order gradient descent.

However, ML models also show fundamental differences from process-based models
endowing them with complementary advantages. First, while the functional forms
of process-based models may have imperfections that are difficult to fix using
data, the solution structure of ML models (e.g., deep learning models) use
generic architectures that are highly expressive and capable of capturing
arbitrarily complex relationships between inputs and outputs provided they are
fed with adequate training data. Second, ML methods (e.g., deep learning
methods) are very fast at making forward inferences using simple matrix
operations that are amenable to modern GPU architectures. As a result, ML
models, once trained, can offer significant computational gains in modeling
complicated scientific systems in comparison to process-based models employing
complex numerical operations. Third, a unique advantage of many ML frameworks
(e.g., deep learning) is that the parameters of ML models trained over a
certain system can be used as a starting point to initialize the learning of
parameters in a new system. A number of research strategies have been
developed over the years in ML in the fields of transfer learning and meta
learning to enable the transfer of learning from source tasks with a
sufficient number of data samples to target tasks with little to no data,
possibly even with shifts in distributions of input and output variables.
Building upon the idea of transfer learning in mainstream ML applications of
computer vision and natural language understanding, there is a growing trend
to create Foundation models [13] such as chatGPT [121] that have been pre-
trained on a wide variety of tasks to learn universal feature representations
of data, which can be fine-tuned on a new task with very few data. This
inherent transferability of learning in ML models provides a novel opportunity
in environmental sciences to build broad-scale models that leverage knowledge
from a large number of well-observed systems and can be fine-tuned on new
less-observed systems with very few (or even no) observations.

### Challenges for Black-box ML

Despite their promise, black-box ML models—that are designed, trained, and
deployed agnostic to scientific theories—have led to spectacular failures in
scientific problems [16, 92, 116](e.g., the rise and fall of Meta AI’s
Galactica [57] and Google Flu Trends [92]). This is primarily because black-
box ML models solely rely on the supervision contained in data and hence are
only as good as the data they are trained with. In problems where we have
limited coverage of data distributions in the training set, black-box ML
models can fail to generalize on out-of-distribution samples and produce
results that are not only inaccurate but also nonsensical and incoherent with
existing scientific theories while being highly overconfident. This is in
contrast to process-based models that are designed to produce robust and
scientifically consistent predictions even if they are inaccurate due to poor
calibration of parameters or modeling imperfections. One strategy to improve
the generalization capability of black-box ML models is to scale the size and
diversity of training data along with increasing the size of models, which is
the approach pursued in the creation of Foundation models [13]. While this is
possible in mainstream applications of ML where we have Internet-scale data
with millions [32] or even trillions [120] of labeled samples, the nature of
scientific data is very different that makes it infeasible to match the scales
of labeled data possible in commercial domains. In particular, scientific
systems exhibit rich heterogeneity in the distributions of input and output
variables over space and time, with complex interactions of processes
operating at multiple scales with varying memory footprints, often involving
unknown confounding factors. This makes it difficult to cover the full
spectrum of possible data distributions that one may encounter when deploying
ML models in scientific applications.

Another fundamental limitation of black-box ML models is their inability to
generate explainable hypotheses of the relationships between input and output
variables in a manner consistent with existing scientific theories. Since one
of the primary goals in science is to explain the cause-effect mechanisms of
phenomena observed in the physical world, a black-box model that achieves
somewhat good predictive performance but fails to deliver a mechanistic
understanding of the underlying processes is unfit for being used as a basis
for subsequent scientific developments. There is a growing interest in machine
learning to incorporate the tools of causal inference [125, 124] to move
beyond learning associations in data (which is what most predictive models do)
to designing interventions and eventually reasoning with counterfactuals.
Scientific problems provide a unique opportunity to study causal reasoning
rooted in existing theories of cause-effect mechanisms available as process-
based models.

##  3 Knowledge-guided Machine Learning (KGML): An Overview

Despite the challenges in using black-box ML, modeling of environmental
systems also presents novel opportunities for developing new advances in ML
that leverage the unique characteristics of data encountered in scientific
domains. For example, data collected in environmental sciences capture
valuable information about a variety of Earth system processes operating at
different spatial and temporal scales. While a given data set may only provide
a partial view of the complex nexus of physical processes evolving over space
and time, they represent real-world phenomena grounded in core scientific
theories (e.g., the principle of mass and energy conservation). As a result,
there is a growing interest in scientific and ML communities to leverage
scientific knowledge in ML frameworks to produce explainable and
scientifically grounded solutions that are likely to generalize even on out-
of-distribution samples with limited training data. Research in this emerging
field is referred to as scientific knowledge-guided machine learning (KGML)
[78, 77] where both scientific knowledge and data are used at an equal footing
as complementary sources of supervision in ML frameworks, in contrast to
first-principle process-based models and data-only (black-box) models (see
Figure 2(a)). KGML is a rapidly growing research community that has seen
several notable examples of promising research in recent years. One of the
first major efforts to present the roadmap for research in KGML includes a
2017 perspective article [77] that laid the foundation of several research
methods for combining scientific knowledge in ML framework. This consequently
sparked ample interest in the ML and scientific communities to pursue research
in KGML, as evidenced by the large volume of literature published in the last
few years including survey articles [163, 76, 148], perspective papers [134,
172], and a recent book on KGML [78].

Figure 2: (a) Pictorial depiction of how KGML models are different from data-
only (black-box) ML models and science-only models. (b) A three-dimensional
view of KGML research where different settings of these dimensions correspond
to various research directions in KGML

### A Multi-dimensional View of KGML Research

KGML has quickly grown to become a thriving area of research that is reflected
in the diversity of research communities that have emerged to leverage KGML in
the context of varied scientific problems. As a result, research in KGML is
also referred to by many names including ‘theory-guided data science’,
‘physics-guided machine learning’, ‘science-guided machine learning’, and
‘physics-informed machine learning’. While all of these terms refer to the
common goal of integrating scientific knowledge with ML, it is important to
understand the differentiating aspects of KGML research that lead to its rich
diversity. We present these differentiating aspects in the form of a three-
dimensional view of prior research in KGML: type of scientific knowledge
(ranging from perfect and complete to imperfect and partial), form of
knowledge-ML integration (ranging from process-centric to ML-centric), and
method for incorporating scientific knowledge in ML (with choices including
knowledge-guided learning, architecture, or pretraining). See Figure 2(b) for
a schematic depiction of these three dimensions, which represent different
steps or decisions one must take to take to select a KGML formulation best-
suited for a scientific problem. They also help in categorizing previous
research in KGML and exposing commonalities between research directions
explored in diverse scientific communities. We briefly describe each of these
three dimensions in the following.

#### Type of Scientific Knowledge

The first aspect that one must decide when working on a KGML problem is the
type of scientific knowledge available in the problem. Scientific knowledge is
available in diverse formats in various disciplines, including first-principle
equations, domain-specific laws, rules, heuristics, or ontological
relationships. In many scientific applications, we only have partial knowledge
of the system (i.e., we may not have complete knowledge of all system
processes) or we have to deal with approximate terms in scientific equations.
We refer to such systems as having partial and/or imperfect scientific
knowledge. A seminal line of work in incorporating partial and imperfect
scientific knowledge in the learning of ML models is the framework of physics-
guided neural networks (PGNNs) [79] that was originally developed in 2017 for
modeling the temperature of water in lakes using monotonic constraints of
density and depth. This has led to a growing body of research in KGML for
incorporating diverse forms of scientific knowledge including monotonic
constraints [118], eigenvalue equations [35, 46], energy balance [71, 72,
133], and scaling laws [54]. For example, Jia et al. developed the framework
of physics-guided recurrent neural networks (PGRNNs) [71] that accounted for
the conservation of energy across time for predicting lake temperatures, while
allowing for error tolerance in the equation for energy balance involving
approximate terms.

In other applications, scientific knowledge can be assumed to be perfect and
complete, i.e., all of the underlying processes of a scientific system can be
fully described by a set of scientific equations that are exactly known with
no incorrect or missing terms. For example, in many applications, scientific
knowledge is expressed in the form of idealized partial differential equations
(PDEs), where the goal is to solve these equations in a computationally
efficient manner. Many techniques in KGML have been developed for problems
where scientific knowledge can be assumed to be perfect and complete. In such
problems, ML methods can serve as a promising alternative to numerical methods
for solving scientific equations with significant computational gains. One of
the seminal works in this direction is the framework of Physics-Informed
Neural Networks (PINNs) [129, 130, 128] that were developed in 2017 to solve
PDEs using feed-forward neural networks. This has opened an entirely new line
of research on solving PDEs using advanced neural architectures such as neural
operators [155, 97, 50]. There is thus a continuum of research directions in
KGML based on the type of scientific knowledge, which can range from perfect
and complete to imperfect and partial.

#### Form of Knowledge-ML Integration

There is also a continuum of research in KGML depending on the exact form of
knowledge-ML integration, ranging from ML-centric approaches to process-
centric approaches. In ML-centric approaches, the mapping of inputs to
response variables is primarily driven by ML methods while scientific
knowledge plays the role of guiding ML algorithms to scientifically consistent
solutions. We describe the different strategies for incorporating scientific
knowledge in ML frameworks in detail in Section 4. On the other hand, in
process-centric approaches, the entire modeling exercise is primarily driven
by process knowledge whereas ML methods are only used to inform or augment
certain components of process-based models. In these approaches, ML is
embedded inside the process-based model and the final predictions of response
variables are made by the process-based model. For example, the role of ML can
be limited to identifying (or calibrating) latent parameters in process-based
models from data (e.g., in a recent line of work on differential parameter
learning for hydrology [140]), which results in better match between process-
based model outputs with observations of response variables. Another example
of process-centric KGML formulations is the area of subgrid parameterization
in climate science [12], where ML methods are used to learn corrective terms
in science-based equations running at coarse resolutions of space (e.g., at
1km grids), to capture the effect of subgrid processes at finer resolutions
not accounted by the equations. This is related to the general direction of
using ML to augment certain components in process-based models that are known
to be imperfect, e.g., in the field of turbulence modeling to close the gap
between high-fidelity and low-fidelity simulations [34] and in geosciences
where ML-augmented equations are used to model hydrological processes [40].
There is also a range of approaches in KGML to create hybrid combinations of
ML-based and process-based components, referred to as the field of hybrid
modeling [77, 78, 163]. Approaches in this field include residual modeling
strategies [145], where ML is simply used to learn the residuals of process-
based models and the final predictions are equally contributed by both ML and
process-based model outputs. Another strategy is to use the outputs of
process-based models as additional inputs in ML models along with drivers to
predict response variables, referred to as the framework of hybrid-physics-
data (HPD) models [79]. This style of hybrid modeling is closer to being ML-
centric as the final predictions of response variables are determined by the
ML model, and the process-based simulations are only used as inputs that serve
as initial estimates.

Figure 3: Research methods for incorporating scientific knowledge in ML
frameworks

#### Method for Incorporating Scientific Knowledge in ML

When it comes to using ML components in KGML approaches (either process-
centric or ML-centric), an important aspect is the method used for
incorporating scientific knowledge in ML frameworks. One of the primary ways
of doing this is by modifying the learning algorithm of ML models (e.g., deep
learning models) to favor the selection of models that are consistent with
scientific equations (e.g., using knowledge-guided loss functions), so as to
steer the training trajectory toward generalizable solutions. This method is
adopted in the frameworks of PGNN and PGRNN for lake temperature modeling and
the framework of PINNs for solving PDEs. Another approach is to “hard-code” or
“bake-in” scientific knowledge directly in the solution structure of deep
learning models, resulting in knowledge-guided neural network architectures.
Some examples of research in this area include the framework of physics-guided
architecture (PGA) of LSTM models (PGA-LSTM) [31] and approaches for capturing
symmetries and invariances of dynamical systems in the architecture of neural
networks [152]. The basic goal of these methods is to improve the scientific
explainability of features learned at the hidden layers of neural networks,
grounded in scientific theories. A third approach is to initialize neural
network parameters using weights informed by scientific knowledge, e.g., by
pretraining the model on simulations of science-based models before fine-
tuning on gold-standard observations [71, 72]. See Figure 3 for a pictorial
depiction of these three methods. Note that these three methods are also not
mutually exclusive; they can be combined together in various ways depending on
the context of application.

| KGML
---
Research
| Type of
---
Knowledge
| Knowledge-ML
---
Integration
| KGML
---
Method
| Additional
---
Comments
PINN [128] |  | Perfect &
---
Complete
ML-centric | Learning |  | Solves a single PDE
---
at any arbitrary point
using feedforward networks
| Neural Operators
---
(FNO [96],
DeepONet [109])
| Perfect &
---
Complete
ML-centric |  | Learning
---
(optional)
| Solves a family of PDEs
---
at arbitrary resolutions
using feedforward networks
PGNN [30] |  | Imperfect &
---
Partial
ML-centric | Learning |  | Uses monotonic constraints
---
(partial) and process
simulations (imperfect)
in feedforward networks
PGRNN [70] |  | Imperfect &
---
Partial
ML-centric |  | Learning &
---
Pretraining
| Uses energy conservation
---
(imperfect and partial)
in RNN with process
simulations (imperfect)
PGA-LSTM [31] |  | Perfect &
---
Partial
ML-centric | Architecture |  | Uses monotonic constraints
---
(perfect and partial)
in LSTM architecture
| Equivariant-Nets
---
[152]
| Perfect &
---
Partial
ML-centric | Architecture |  | Uses symmetries (perfect
---
and partial) in CNN
dPL [140] |  | Perfect &
---
Complete
Process-centric | - |  | Learns parameters in
---
differentiable process model
| Subgrid
---
Parameterization
[12]
| Imperfect &
---
Complete
Process-centric | - |  | Uses ML to correct
---
effects of subgrid processes
Table 1: Categorization of previous research in KGML in terms of the three-
dimensional view of KGML research

Table 1 provides a categorization of a few example research works in KGML from
the perspective of the three dimensions. They are described in more detail in
later sections.

##  4 Research Methods in KGML

In this section, we expand upon the three primary ways of integrating
scientific knowledge in ML that we introduced in the previous section, namely
knowledge-guided learning algorithms (e.g., using loss functions), knowledge-
guided architecture of ML models, and knowledge-guided pretraining or
initialization of ML models.

###  4.1 Knowledge-guided Learning

Directly building ML models from limited observational data often leads to the
learning of spurious patterns that cannot be generalized to new scenarios,
which necessitates the awareness of underlying physical knowledge in the
learning of ML models. The predominant method for integrating scientific
knowledge into ML is to directly introduce physical principles into the
training objective of ML models [78, 79, 20, 71, 128]. For example, the
framework of physics-guided neural network (PGNN) [79] was one of the first
works to add physical constraints (in particular, the density-depth
relationship) as loss functions to guide the training of neural networks to
physically consistent solutions in the context of predicting lake water
temperature. The PGRNN model [71] further extended this idea to capture the
heat transfer process as physics-guided loss functions in the prediction of
water temperature dynamics in lakes. This method represents a pioneering
effort in integrating the energy conservation law into ML training. In
particular, this method estimates intermediate heat fluxes in the heat
transfer process and then defines the energy conservation over these fluxes
and the predicted water temperature over the water column. Read et al. [133],
for the first time, systematically evaluated the generalizability of the PGRNN
approach on out-of-distribution samples. They demonstrated that the PGRNN
model significantly outperforms both standard ML models and process-based
models when tested on years or seasons with very different weather conditions
compared to training data.

The idea of knowledge-guided learning can be utilized to integrate scientific
knowledge in different forms as loss functions in the training of ML models.
For example, scientific theories are commonly represented as equations,
particularly differential equations, which offer a direct way to embed known
scientific laws into models [111] such as the approach of physics-informed
neural networks (PINN) [128] and the physics-informed neural operator (PINO)
[97]. Integrating scientific equations into the loss functions of ML models
can be especially useful when the goal is to solve a known target equation,
e.g., in the problem of solving PDEs [4, 176, 14]. Similarly, heuristics,
rules, and structured relationships in ontologies and knowledge graphs can
also be encoded and added to the loss function [166, 167, 165, 42, 19]. By
penalizing deviations from these established knowledge bases, the training
algorithm has a higher chance of learning physically consistent data patterns
and thus achieving improved generalizability.

However, there are challenges in defining and utilizing knowledge-guided loss
functions when dealing with imperfect knowledge, e.g., equations not perfectly
aligned with true observations. In such cases, we can adjust the weight of
regularization terms in the loss functions, enabling a balance between
minimizing training errors and satisfying domain constraints [76, 128, 97].
Moreover, incorporating knowledge-guided loss functions can introduce
potential conflict between data-driven and domain-guided loss functions, which
can lead to the challenge of competing objectives during ML training [35].
Additionally, the optimization landscape could suffer from the presence of
poor local minima due to the added complexity of knowledge constraints, making
convergence to an optimal solution more elusive [154, 29, 156, 153]. These
issues can be potentially addressed by multi-objective optimization strategies
[52, 78, 88], which offer a solution by exploiting trade-offs between
conflicting goals.

###  4.2 Knowledge-guided Architecture

Figure 4: The hierarchical physical variables simulated in the hydrological
systems. The variables shown on each module is the output of the module.

ML models are essentially parameterized architectures that need to be trained
using observational data. The internal structures of ML models often involve
common components such as convolutional layers, recurrent layers, and graph
layers for capturing complex spatio-temporal data dependencies. However, these
data-driven components are prone to overfitting when the available
observations are limited or not representative of the data distribution in
larger regions or different time periods. There is a general interest in
integrating scientific knowledge into specific components in ML architectures
[71, 3, 73, 7, 33, 100, 2]. These methods can be broadly grouped into three
categories: (1) incorporating knowledge to modify components in existing deep
learning models (e.g., convolutional and recurrent layers, graph structures),
(2) decomposing ML models guided by the interactions of different processes,
and (3) encoding physical properties (e.g., invariance or equivariance) into
the architecture.

Prior work has explored different ways to enhance the deep learning
architecture by integrating scientific knowledge into certain modeling
components [100, 6, 7, 71, 73, 31, 2]. In the context of water temperature
prediction in freshwater ecosystems, prior works have incorporated the energy
conservation law in modeling the hidden information being transferred across
time in the RNN structure [71] and across stream segments in the graph
convolutional process [73]. Other works have explored directly using known
PDEs to inform certain model components. For example, Bao et al. [7] built
dynamic graph structures across stream segments based on the heat transfer PDE
[33]. Lienen et al. [100] developed a graph neural network by incorporating
physics knowledge on the unknown PDE to improve sea surface temperature and
gas flow predictions. Airphynet [58] also utilized multiple graphs to capture
the diffusion and advection effect in the air pollutant concentration. In the
work by Daw et al. [31], a physics-guided architecture of LSTM models (PGA-
LSTM) was developed to explicitly encode the density-depth physics in the
connections among LSTM nodes for the problem of lake water temperature
modeling. PGA-LSTM was shown to produce meaningful uncertainty estimates of
water temperature profiles across depth in lakes, where every realization of
temperature profile generated by the model was guaranteed to respect density-
depth physics.

Another thread of research is in decomposing the model architecture into
multiple modules responsible for unique physical processes, with interactions
among processes captured by module dependencies. For example, the
agroecosystem consists of diverse types of physical processes, e.g., weather,
soil conditions, plant, and respiration, which jointly form the cycling of
energy, water, and carbon. These processes are simulated by existing physics-
based models, which are often built with a modular structure. Fig. 4 shows a
general modular structure of many hydrology models for simulating streamflow.
Here each module is responsible for converting input variables to specific
output variables based on known or modeled physical processes, and these
variables can be either observable or unobservable. Each module is implemented
as a series of mathematical computations, which can be either explicit (e.g.,
following mathematical equations) or implicit (e.g., solving a differential
equation). Conventional ML largely ignores the modular structure and try to
directly create a mapping between input drivers (e.g., weather) and the
response variable (e.g., streamflow). To fully capture the dynamics of
variables and their interactions in scientific systems, previous methods
developed modularized NN architectures, which are consistent with the
processes defined in physics-based models [103, 84, 104, 56]. This entails
ascribing physical meanings to the output of each NN module while also
maintaining the interrelationships amongst these modules as defined in
existing physics-based models. This architecture decomposes the modeling of
different variables and thus offers the flexibility for selecting different NN
structures (e.g., recurrent and convolutional networks) for each module that
best fit the nature of each variable that may evolve and interact at different
scales. For example, Liu et al. [103] created a hierarchical NN structure
using multiple GRUs for predicting different intermediate physical variables
in agroecosystems, such as CO2 fluxes and soil NH${}_{4}^{+}$. Feng and Shen
et al. [40, 140] further extended this idea by replacing certain process-based
modules by NN structures while implementing other time-discrete processes
directly on PyTorch to allow automatic differentiation. They also model the
spatial variation of physical parameters (in the process-based model) using a
separate NN model (e.g., LSTM) [147], which takes the forcing data and
attributes to predict the static parameters or dynamic parameters of the
process-based model, and feeds them to the main predictive model.

In addition, the ML architecture can be designed to explicitly capture desired
physical properties, e.g., the equivariance property in a dynamical particle
system. While the equivariance can also be partially achieved through data
augmentation, directly encoding it in model architecture can reduce the number
of model parameters, and thus mitigate overfitting and improve the
generalizability. Most existing equivariance models (e.g., G-CNN [26],
Steerable CNN [27]) are developed by extending the translation equivariance of
the standard CNN model to group equivariance. The EGNN model [137] further
extends this to higher dimensional space on a graph structure. It is also
noteworthy that the development of many commonly used ML architectures was
initially inspired by important physical properties. For example, the
convolutional layers in the standard CNN are designed to preserve
translational equivariance in image recognition.

###  4.3 Knowledge-guided Pre-training

Overparametrized ML models are prone to overfitting due to the large parameter
space and the instability inherent in their optimization process. Scientific
knowledge can be leveraged to initialize the model parameters so less data is
needed to refine the model parameters to learn generalizable data patterns.
Existing KGML pre-training methods can be conducted by utilizing simulated
data or self-defined proxy learning tasks (self-supervised learning), as
discussed in the following.

Use of Simulated data: Simulated data generated by process-based models (even
generic uncalibrated models) can be used to pre-train ML models. Such
simulated data encode fundamental physical principles governing the complex
systems, and thus can provide valuable information in training ML models,
especially when observations are sparse. Use of such pre-training is one of
the easiest ways of leveraging the knowledge about scientific principles
governing complex environmental systems into ML [173, 71, 72]. Jia et al. [71]
first utilized simulated data from a generic process-based model to pre-train
the ML model and demonstrated the improvement in the performance under data-
sparse and out-of-sample scenarios.

One limitation of this approach is that the simulated data are often biased
due to the parameterization used in physics-based models, which makes the pre-
training less helpful when we adapt (or fine-tune) the model to the target
system. In particular, physics-based models simulate different physical
processes with preset parameterizations, e.g., the setting of soil moisture
and watershed characteristics when simulating streamflow and water
temperature, but these preset parameter values can be very different in the
target ecosystems. Several approaches have been developed to address this
issue [69, 23, 67, 21]. For example, Jia et al. [69] proposed a meta-learning
algorithm to initialize an ML model using multiple sets of simulated data
created by varying the parameterizations of the physics-based model. This
process results in a diverse dataset that closely emulates real-world
hydrological data, allowing the ML model to learn and generalize effectively
to real-world scenarios. McCabe et al. [117] also explored embedding data from
multiple physical systems into a shared representation space so as to create a
generalizable pre-trained model. In another work, Chen et al. [23] utilized
multiple physical equations to simulate baseflow in river basins, and
selectively used simulated values to regularize the ML model in its
initialization and adaptation phases.

Another issue with simulated data is that they are often generated
independently by physics-based models without considering the effect of other
real-world factors, e.g., human infrastructures. For example, most physics-
based stream models do not explicitly capture the effect of dams, reservoirs,
and transportation systems, which introduces bias to the simulated streamflow
and water quality measures. Such biased simulated data can be less helpful in
initializing ML models as they present only physical processes that are in
ideal scenarios but could behave very differently from true observations. One
interesting direction is to combine simulated datasets independently created
over different systems and use data-driven approaches to model their
interactions. Recently, there was a data release from the U.S. Geological
Survey on stream temperature simulations that combines two types of physics-
based models, the SNTemp model [8] for streams and the GLM model for
reservoirs [61]. Such a combination is through a linear function between each
stream and its upstream reservoir, and the weight of the reservoir is
exponentially decayed over a longer distance. Jia et al. [68] further extended
this idea by using a graph neural network to combine the simulations on
different types of nodes, and then use the obtained composite simulations to
pre-train the graph model for predicting stream water temperature.

Self-supervised learning:  In the field of self-supervised learning (SSL),
deep neural networks are trained to learn useful feature representations using
pseudo labels created from pre-defined pretext tasks, e.g., colorization [91]
and inpainting [123] in computer vision. These pretext tasks are designed in a
way such that solving these pretext tasks requires the extraction of complex
data patterns needed for the target prediction task. For the self-supervised
methods to be effective in environmental science problems, one promising
direction is to create pretext tasks that can reflect knowledge about
underlying physical processes. In general, the self-supervised learning
paradigm can be broadly categorized as sample-wise similarity learning and
predictive learning, described in the following.

The core concept of sample-wise similarity learning involves learning
representations for individual data samples, aiming to precisely reflect the
inherent similarities between them [174, 149]. This approach primarily
concentrates on deciphering the inherent relationships within the data,
considering each data point in relation to every other point in the sample
space. Specifically, a similarity is estimated between each pair of samples
based on their characteristics in scientific systems, e.g., depth and surface
area in lake systems [164, 23, 24]. Next, ML models are pre-trained to map
data samples to a representation space, in which similar samples are pulled
closer and dissimilar ones are pushed further apart [47, 161]. This can be
commonly achieved by contrastive learning [93]. For example, Ghosh et al. [47]
adopted this in streamflow prediction. They developed a representation
learning method to embed basin-specific characteristics, which is trained via
contrastive learning using positive (similar) sample pairs from the same basin
and negative sample pairs from different basins.

As an alternative method, predictive SSL learning aims to pre-train ML models
by predicting certain state variables in the target system. The idea is to
enforce the learning of key state variables related to the target variables.
For example, an ML model for crop yield prediction can be pre-trained to
predict variables related to its carbon and water cycles [99]. The training
can be conducted using simulated values or proxy values for these state
variables. Prior works further extended this idea to pre-training a knowledge-
guided ML architecture, which contains multiple ML modules to represent
different processes [103]. They leveraged simulated data for intermediate
modular outputs to pre-train different ML modules.

The Challenge of Catastrophic forgetting:  The pre-training enables the ML
model to learn the representation of physics from simulated data. Next, the
model can be fine-tuned with true observations from the target system [139].
One potential issue is that the fine-tuning could distort the representation
learned from pre-training, leading to degraded generalizability [87]. There is
an opportunity to explore different fine-tuning techniques, e.g., tuning a
subset of parameters at a time while fixing other parameters [69]. The
objective is to preserve the generalizable patterns extracted from simulated
data when adapting the pre-trained model to the target system.

##  5 Use Cases in Environmental Sciences

Figure 5: Schematic diagrams of four primary use-cases of KGML in scientific
disciplines

Figure 5 illustrates many prominent use-cases for using KGML methodologies in
scientific problems. First, KGML methods can be used to learn the forward
mapping from input drivers to response variables better than state-of-the-art
(or sometimes, non-existent) science-based forward models in terms of accuracy
and/or computational efficiency. A second use-case is to learn the inverse
mapping from observations of inputs and outputs to the parameters of science-
based models using ML algorithms, which can help provide useful information of
the system states and aid in calibrating physics-based models. A third use-
case of KGML in scientific domains is to use generative modeling approaches to
create digital twins of scientific systems, where ML methods are used to
generate synthetic distributions of variables of interest (e.g., spatio-
temporal variations of Earth’s surface temperature), optionally conditioned on
a few parameters. This is useful for the purpose of data generation, which can
then be fed into downstream tasks of forward modeling. A fourth use-case of
KGML in scientific domains is to convert coarse-scale information of physical
variables to higher resolutions in space and/or time, referred to as the
problem of downscaling (also referred to as the problem of super-resolution in
the field of computer vision). This can help reduce the expensive
computational cost of creating simulations at fine scales.

###  5.1 Forward Modeling

Process-based models consist of a series of mathematical equations to simulate
the forward mapping of underlying processes from input drivers to response or
target variables. Running these models often requires substantial
computational costs due to the need to solve internal equations (e.g., complex
PDEs). This poses a major impediment for simulating scientific processes at
fine spatial resolutions and high time frequency required in many real-world
settings. Moreover, existing process-based models often use parameterizations
and approximations due to incomplete knowledge or excessive complexity in
modeling certain processes, leading to degraded predictive performance. ML
approaches are increasingly being utilized to build forward models, but
standard ML approaches are not designed for capturing complex dynamics and
interactions of scientific processes, especially when training data are
limited.

KGML methods can be used in the problem of forward modeling in two scenarios:
(1) surrogate modeling: when target variables can be assumed to be perfectly
described by process-based models, e.g., simulated data generated by solving
PDEs, KGML methods can be utilized to build computationally efficient
surrogate models, and (2) improved forward modeling: when we have
observational data for target variables and process-based models are known to
be imperfect (or approximate) and incomplete representations of reality, KGML
methods can be used to improve both computational efficiency and predictive
accuracy of forward modeling. We describe both these scenarios in the
following.

####  5.1.1 Surrogate Modeling

Solving PDEs:  Traditional numerical solvers for PDEs often use the Finite
Elements Method or the Finite Difference Method [41], and can be prohibitively
expensive for many simulation tasks, e.g., simulating turbulent flow with a
large Reynolds number. Recently, neural network (NN) models have been used to
approximate the solution of PDEs through the neural network (NN) forward
process, which significantly reduces the computational cost compared to
conventional numerical solvers. NN-based PDE solvers have shown promise in
approximating PDE solutions with reasonable accuracy in a number of
applications. Some popular approaches include physics-informed neural networks
(PINN) [76] that are designed to solve a single instance of a PDE, and neural
operator learning methods that can learn a family of PDEs including the
Fourier Neural Operator (FNO) [96] and DeepONet [108]. Operator learning
methods such as FNO and DeepONet can be further enhanced by adding physical
constraints in the training process [97, 155].

One area of the recent focus in the problem of solving PDE is in using FNO to
approximate the PDE solutions through a transformation in a Fourier space. The
intuition is to approximate the Green functions by kernels, which are
parameterized by neural networks in the Fourier space. Specifically, the FNO
approach initially transforms the input function, representing the initial or
boundary condition of a PDE, from the spatial domain to the frequency domain
using the Fourier transformation, which allows the representation of the
function to capture the global information effectively. A neural network then
learns the mapping between the Fourier coefficients of the input and output
representation in the frequency domain. After this learning process, the
inverse Fourier transform is applied to convert the learned representation
back to the spatial domain, yielding the approximated solution to the PDE.
Based on such network design, FNO is able to combine the global information of
the entire field embedded through the Fourier transformation and the
expressive power of neural networks, enabling the learning and approximation
of high-dimensional and complex PDE operators directly from data. An
additional benefit of FNO is its ability to create the PDE solution at
arbitrary spatial resolutions. This has the potential to achieve zero-shot or
few-shot downscaling in complex dynamical systems. Several variants of FNO
have already emerged [83, 146, 182, 112] and have also been used to address
the scientific challenges in environmental science [95, 74, 176]. The PICL
approach [107] was introduced to further extend FNO for simultaneously
modeling multiple governing PDEs by using contrastive learning to enforce the
distance amongst samples from different PDE systems. Furthermore, Lippe et al.
[101] introduced the diffusion model [162] as PDE-Refiner into FNO for more
accurate modeling of PDE dynamics via a multi-step refinement process.

Surrogate Modeling for Other Systems: Existing process-based models rely on
intensive computations to simulate complex physical processes. Surrogate
modeling addresses this challenge by creating data-driven models that
approximate the behavior of physical models and predict the outputs of these
complex systems with much less computational effort. This makes them extremely
valuable in scenarios where rapid decision-making is crucial, such as
emergency response planning, environmental impact assessments, or policy
development [132, 38, 102, 180, 11, 184, 60, 179]. Surrogate models are
typically developed using advanced machine learning tools [11, 184, 60, 179,
144, 158] (e.g., Koopman operator [15]) or statistical methods [43], and are
trained on simulated data generated by complex physical models. To fully
capture the essential patterns and relationships inherent in environmental
processes, scientific knowledge can also be leveraged to enhance the ML-based
surrogate models.

####  5.1.2 Improved Forward Modeling

Utilizing scientific knowledge can further enhance the ability of process-
based models to better align with ground-truth observations. Building upon the
traditional approach of residual modeling where ML or statistical methods are
used to learn the residual of a process-based model w.r.t. ground-truth data
[145], researchers have investigated the combination of simulated and
observational data through a pretrain-finetune pipeline [103, 53, 98]. This
method leverages the knowledge embedded in comprehensive simulated data while
further improving the prediction on observational data through parameter
refinement. Another KGML approach for improved forward modeling is to
incorporate domain knowledge into the ML model’s architecture [103]. This
integration ensures that the model adheres to established scientific
principles, while the model parameters can be further adjusted to match true
observations. Advancements also be made through feature selection, which
utilizes domain knowledge to select or extract the most relevant features for
the target task [171]. This enhances the model’s ability to focus on the most
informative aspects of the data, thereby improving its predictive performance.
Moreover, the ML model can be constrained by known physical relationships.
This can be achieved by either adding physical laws as constraints during the
model training process [175, 80] or adjusting model outputs in the post-
processing phase to maintain physical consistency [19].

###  5.2 Inverse Modeling

Standard forward models aim to create a mapping $f$ from input drivers $x_{t}$
to target variables $y_{t}$. The mapping $x_{t}\rightarrow y_{t}$ can be
different for every system (e.g., different basins or farmlands) due to the
variation of physical characteristics (e.g., groundwater, hydraulic
conductivity, soil, and vegetation conditions) [48]. Hence, forward models
often need to include physical characteristics $c$ as additional input, i.e.,
$y_{t}=f(x_{t},c)$. Most physical characteristics $c$ are considered static or
evolving slowly. Due to the technical difficulties or prohibitively expensive
cost in measuring these characteristics, they are often modeled as physical
parameters and need to be calibrated using real observations $\\{x,y\\}$.
Traditional calibration of physics-based models commonly uses grid search or
Bayesian approaches to find combinations of parameter values leading to the
best match with observations or measurements, which can be highly time-
consuming and require extensive domain expertise to select the range for each
variable [59, 115, 39]. This process can also degrade the predictive
performance using the obtained parameters due to the limited computational
resources available to explore all the possible parameter combinations,
especially in scenarios involving complex interactions among variables and
parameters.

Advances in ML and deep learning have brought opportunities to estimate
physical characteristics directly using data-driven approaches. Given some
available data samples $\\{x_{t},y_{t}\\}$, the characteristics (or
parameters) $c$ can be estimated by learning the inverse mapping
$c=g(x_{t},y_{t})$. A key motivation behind inferring the parameters of
process-based models is to recover critical information about the state of
scientific processes that are difficult to observe directly. Another
motivation is to identify useful settings of model parameters that when fed
inside process-based forward models result in simulated outputs that best
match with observations, referred to as the problem of model calibration.

Building upon traditional inverse modeling approaches based on regularized
regression [36], researchers are now building ML models for the inverse
modeling of characteristics in hydrology [45], photonics [126], land surface
temperature [159], among many others. For example, in the problem of seismic
imaging, ML methods have been developed to learn the inverse mapping from
observations of waveform amplitudes to the velocity profile of wave
propagation through different layers of the Earth’s subsurface [75]. Some
existing works on discovering PDEs can also be viewed as special cases of
data-driven inverse modeling through the estimation of PDE coefficients from
available simulations [127, 135]. Note that inverse modeling can also be
performed with partial knowledge of the underlying system without explicitly
using a process-based forward model, as explored in a recent work in the
domain of hydrology [47]. Another recent line of work in KGML for inverse
modeling includes the framework of differentiable parameter learning (dPL)
[147] for geoscience applications, where deep learning methods are used to
infer the parameters of differentiable physics-based models using gradient
descent algorithms leveraging automatic differentiation of the models.

Due to the intrinsic complexity of the inverse mapping, training inverse
models can be challenging for some systems with sparse observations. Prior
work has applied physical constraints to NN-based inverse models by
regularizing the predicted latent state variables from observations [65].
Another promising direction is to build an inverse loop for the forward model
(e.g., using an invertible NN structure [143]), which allows natural
incorporation of scientific knowledge in the forward process.

###  5.3 Generative Modeling

Generative models have found great success in computer vision [157, 28] and
natural language processing [64]. Some popular models include generative
adversarial networks [49], variational autoencoder [81], normalizing flow-
based modes [82], and diffusion models [63]. These models aim to capture the
underlying data distribution and then create synthetic samples that look
similar to the real observations. Given their success in generating images,
speech, and text data, the expectation is rising for using these models to
generate virtual physical simulations under specified conditions.

Despite the power of these models, they are often found to generate spurious
data patterns or artifacts when directly applied to complex applications in
fluid dynamics and environmental sciences, especially when training data are
scarce (e.g., simulations at high resolutions). To address this challenge,
prior work has explored regularizing the generative models by additional
physical constraints, which have shown improvement in the quality of generated
data in fluid dynamics [177, 168, 22], material science [17], and climate
science [44]. For example, conservation laws have been utilized to regularize
the GAN-based for generating the simulation of turbulent flows [22].
Similarly, Gao et al. also include the anticipated precipitation intensity for
regularizing the denoising process of the diffusion model, and show the
promise of this method for precipitation nowcasting and the detection of
extreme cases such as rainstorms and droughts [44]. The generative modeling
approaches have also been used for refining the initial predictions. For
example, Lipple et al. [101] found that data-driven PDE solvers (e.g., FNOs)
often neglect components of the spatial frequency spectrum that have low
amplitude. To overcome this issue, they developed PDE-Refiner, which employs
diffusion models to refine the prediction by considering information from all
frequency components at different amplitude levels.

###  5.4 Downscaling

In environmental science problems, complex physics-based models are used to
capture the details of physical reality by incorporating diverse components
that account for numerous processes at fine spatial or temporal scales. These
models are often restricted by computational expenses and model complexity.
Consequently, many models are operated at a coarser resolution than necessary
to accurately represent the underlying physical phenomena. For instance,
cloud-resolving models (CRMs) effectively capture boundary layer eddies and
low clouds with sub-kilometer horizontal resolution [131]. However, running
global climate models at such fine resolutions, even with anticipated
advancements in computing power, remains unfeasible.

Handling downscaling problems in environmental science using machine learning
involves deriving fine-scale information from coarser-scale data. Note that
the coarse-scale inputs can either be obtained through observations or through
simulations of low-fidelity models operating at coarser scales. In particular,
machine learning models have been widely used for automatically projecting the
coarser-scale or lower-resolution environmental data into fine-scale or
higher-resolution ones, which is especially popular in the domains of climate
science, hydrology, and ecology [66, 178, 138, 105, 51, 136, 163, 9, 80, 122].
For example, Wang et al. [150] adopted super-resolution methods to generate
higher-resolution predictions (e.g. temperature, precipitation, etc) in
different locations and times at the local scale from coarse spatial
resolutions. The authors further extended this work by transferring the
trained model in one region to downscale the precipitation in another region
under a different environment.

In many environmental science problems, building data-driven downscaling
methods is challenging due to the often limited availability of fine-scale
observational data. Few-shot and zero-shot learning could be potential
solutions to such problems. Few-shot learning [160] and zero-short learning
[74, 176, 55] can leverage models trained from data-sufficient regions and
adapt them to regions that are poorly observed or completely unobserved after
moderate fine-tuning. Despite the promise of these approaches, using these
approaches for improving scientific data downscaling remains largely under-
explored. Recently, FNO has shown encouraging results in performing zero-shot
downscaling of PDE solutions to arbitrary specified resolutions [96]. Another
limitation of existing data-driven downscaling approaches is their ability to
harness irregularly structured data. Existing downscaling techniques in
environmental science are often applied to gridded data, such as remote
sensing imagery. There are plenty of machine learning models suitable for
these kinds of data, e.g., Convolutional Neural Networks (CNNs) and Generative
Adversarial Networks (GANs) are commonly used in super-resolution (SR) methods
for refining coarse spatial datasets [90, 5]. More recently, researchers have
started exploring downscaling approaches for irregularly structured data, such
as river networks [18]. In prior work, a graph network model is used to
represent and predict intricate network dynamics, while a long short-term
memory (LSTM) model is used to capture flow sequences along river paths [141].
These methodologies hold promise for enhancing fine-scale environmental
predictions from coarser datasets.

##  6 Foundation models in environmental science

Existing process-based and data-driven models are often built for specific
target problems in environmental science. The coupling of multiple models is
often non-trivial, making it difficult to capture the relationships or share
the information across different problems. For example, the prediction of
water quality variables (e.g., water temperature, concentration of nutrients)
and water quantity variables (e.g., streamflow) are often separately modeled
even though they involve some common processes. Foundation models, which aim
to establish a general paradigm to tackle a diverse set of tasks, have found
immense success in natural language processing (NLP) [13, 183]. Utilizing the
textual understanding capabilities acquired from extensive text data in pre-
training, these models have been shown to provide reasonable predictive
performance on different tasks under zero-shot or few-shot settings. The
intricate structure of the large-scale model also allows the information to be
forwarded through numerous branches of routes in the model, facilitating the
encoding of both shared and distinct information essential for various tasks.
This enables learning simultaneously over diverse tasks and then generating
predictions for any task specified by a user.

Recently, researchers started exploring using foundation models (e.g., LLMs)
in environmental applications [170, 114, 1]. When used in environmental
science applications, foundation models provide new opportunities for
performing predictions for different physical variables and at different
scales, or over different sets of locations and different time periods. The
expectation is also rising for using foundation models to understand the
complex nature of the environment from limited observation data. Additionally,
prior works have shown that LLMs are effective in harnessing missing or
inconsistent input data (e.g., weather data, soil properties, and other
modeled variables) across space and assimilating new observations in
environmental applications [110].

Despite the promise, several major challenges could prevent existing
foundation models from reaching the same level of success as they did in the
text and vision domains. Environmental data are often not included in training
existing foundation models. This omission can lead to a knowledge gap when the
model is directly deployed for query tasks involving complex environmental
processes. Although some open-sourced foundation models can be fine-tuned
toward the downstream task, the size of observation data in many environmental
applications is often far smaller than what is needed for tuning large-scale
models. Moreover, existing foundation models remain limited in interpreting
model predictions from a scientific perspective, e.g., the simulation of
intermediate processes that lead to the dynamics of final predicted variables.
Third, running large-scale foundation models requires high computational cost
and financial cost, which makes them unsuitable for prediction over long
periods and large regions. To address these challenges, one promising
direction is to better leverage scientific knowledge for enhancing the
foundation models, e.g., through prompting, reasoning, and tuning, to improve
their generalizability and interpretability. New research is also urgently
needed for selectively running foundation models and existing data-driven or
KGML models in a hybrid manner to reduce the cost.

Alternatively, researchers also investigated building foundation models
directly on environmental applications without referring to existing
foundation models in vision and text domains. For example, the ClimaX model
[119] has been built using climate datasets and a Transformer-based
architecture. It was reported to achieve good predictive performance and
generalizability on different climate and weather tasks. Several previous
studies in scientific modeling can be considered as foundation models. For
example, FNO [95] has been shown to be able to perform PDE solving tasks for
PDEs with different settings (e.g., different initial conditions). FNO and
implicit neural representation [25] can be used to downsample data to an
arbitrary resolution after they are trained on a specific spatial resolution.
Existing works on building heterogeneity-aware models over a large number of
different locations can also be viewed as simplified foundation models that
are specifically designed for spatial generalization [169, 94, 85]. For
example, Xie et al. [169] create a general pipeline for transforming deep
learning models to predict land covers for locations with statistically
different distributions. These models could be further extended to address the
problems of different spatial scales.

Another thread in building foundation models is based on the concept of
transfer learning [185]. By using this approach, a model is first trained on a
broad dataset that encompasses multiple scientific topics [89]. This initial
training allows the model to learn general trends and behaviors. After this
broad training, the same model can be fine-tuned or adjusted on a more
specific dataset related to a particular environmental task [119]. Consider an
example in hydrological problems. A foundation model trained using broad
environmental features like climate data, satellite imagery, and general
topography could be fine-tuned using river basin-specific data. This
specialized model can predict river flow dynamics or water quality, optimizing
its performance by leveraging both the broad knowledge from the global pre-
training and the specific nuances from the fine-tuning phase. However, several
challenges arise in this approach. Data in different scientific problems often
vary in scale, resolution, and quality, making the integration a complex
procedure. Moreover, the intricate interactions in environmental systems, like
hydrology with its dependencies on soil, plants, and weather, make it
challenging to capture all nuances.

##  7 Concluding Remarks and Future Directions

This paper provided a summary of prior research in KGML and contextualized it
using a three-dimensional view. KGML is indeed a rapidly growing field of
research, as illustrated by the diversity of research methods and scientific
use-cases that are being explored in a variety of application domains. There
are also several directions for future work in KGML. First, along with serving
the goal of better predictive accuracy, novel advances in KGML are needed to
solve the end-goal of discovering new scientific knowledge from data. While
there is a growing focus in mainstream ML on explainability, trustworthiness,
and fairness of ML results, scientific problems may require entirely new
definitions of explainability and related concepts. This is because in
scientific applications, we need to go beyond learning input attributions to
make sure that the patterns, insights, or rules discovered from data are
consistent with existing scientific theories and are meaningful to domain
scientists. We also need to go beyond modeling associations to understand the
causality of scientific systems, e.g., using counterfactuals, in conjunction
with existing scientific knowledge. Another advance that is needed in the
field of KGML is to design better tools for uncertainty quantification in
scientific applications of ML. For example, we need to improve our
understanding of the limits of large-scale ML models (e.g., Foundation models)
and predict in advance when ML models will fail and by how much given the
characteristics of a new dataset.

##  8 Acknowledgements

We thank Shengyu Chen for his valuable contributions to summarizing the
existing literature on KGML research methods. We also thank Runlong Yu and Yue
Wan for their very helpful review on an earlier version of the manuscript.
This work was supported in part by National Science Foundation (NSF) awards
IIS-2239328, IIS-2313174, IIS-2239175, IIS-2147195, IIS-2316305, OAC-1934721,
OAC-2203581, and DEB-2213550.

## References

  * [1] Microsoft Research AI4Science and Microsoft Azure Quantum.  The impact of large language models on scientific discovery: a preliminary study using gpt-4.  arXiv preprint arXiv:2311.07361, 2023.
  * [2] Anonymous.  Airphynet: Harnessing physics-guided neural networks for air quality prediction.  In Submitted to The Twelfth International Conference on Learning Representations, 2023.  under review.
  * [3] Anonymous.  ClimODE: Climate forecasting with physics-informed neural ODEs.  In Submitted to The Twelfth International Conference on Learning Representations, 2023.  under review.
  * [4] Muhammad Shoaib Arif, Kamaleldin Abodayeh, and Yasir Nawaz.  A computational approach to a mathematical model of climate change using heat sources and diffusion.  Civil Engineering Journal, 8(7):1358–1368, 2022.
  * [5] Tristan Ballard and Gopal Erinjippurath.  Contrastive learning for climate model bias correction and super-resolution.  arXiv preprint arXiv:2211.07555, 2022.
  * [6] Tianshu Bao, Shengyu Chen, Taylor T Johnson, Peyman Givi, Shervin Sammak, and Xiaowei Jia.  Physics guided neural networks for spatio-temporal super-resolution of turbulent flows.  In Uncertainty in Artificial Intelligence, pages 118–128. PMLR, 2022.
  * [7] Tianshu Bao, Xiaowei Jia, Jacob Zwart, Jeffrey Sadler, Alison Appling, Samantha Oliver, and Taylor T Johnson.  Partial differential equation driven dynamic graph networks for predicting stream water temperature.  In 2021 IEEE International Conference on Data Mining (ICDM), pages 11–20. IEEE, 2021.
  * [8] John Bartholow.  Stream network and stream segment temperature models software.  Technical report, US Geological Survey, 2010.
  * [9] Katrina E Bennett, Satish Karra, and Velimir V Vesselinov.  Characterization of extreme hydroclimate events in earth system models using ml/ai.  Technical report, Artificial Intelligence for Earth System Predictability (AI4ESP …, 2021.
  * [10] Keith Beven and Andrew Binley.  The future of distributed models: model calibration and uncertainty prediction.  Hydrological processes, 6(3):279–298, 1992.
  * [11] Marc Bocquet.  Surrogate modeling for the climate sciences dynamics with machine learning and data assimilation.  Frontiers in Applied Mathematics and Statistics, 9:1133226, 2023.
  * [12] Thomas Bolton and Laure Zanna.  Applications of deep learning to ocean data inference and subgrid parameterization.  Journal of Advances in Modeling Earth Systems, 11(1):376–399, 2019.
  * [13] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al.  On the opportunities and risks of foundation models.  arXiv preprint arXiv:2108.07258, 2021.
  * [14] Oussama Boussif, Yoshua Bengio, Loubna Benabbou, and Dan Assouline.  Magnet: Mesh agnostic neural pde solver.  Advances in Neural Information Processing Systems, 35:31972–31985, 2022.
  * [15] Steven Brunton, Eurika Kaiser, and Nathan Kutz.  Koopman operator theory: Past, present, and future.  In APS Division of Fluid Dynamics Meeting Abstracts, pages L27–004, 2017.
  * [16] P. Caldwell et al.  Statistical significance of climate sensitivity predictors obtained by data mining.  Geophysical Research Letters, 41(5):1803–1808, 2014.
  * [17] Ruijin Cang, Hechao Li, Hope Yao, Yang Jiao, and Yi Ren.  Improving direct physical properties prediction of heterogeneous materials from imaging data via convolutional neural network and a morphology-aware generative model.  Computational Materials Science, 150:212–221, 2018.
  * [18] Shengyu Chen, Alison Appling, Samantha Oliver, Hayley Corson-Dosch, Jordan Read, Jeffrey Sadler, Jacob Zwart, and Xiaowei Jia.  Heterogeneous stream-reservoir graph networks with data assimilation.  In 2021 IEEE International Conference on Data Mining (ICDM), pages 1024–1029. IEEE, 2021.
  * [19] Shengyu Chen, Tianshu Bao, Peyman Givi, Can Zheng, and Xiaowei Jia.  Reconstructing turbulent flows using spatio-temporal physical dynamics.  ACM Transactions on Intelligent Systems and Technology, 15(1):1–18, 2024.
  * [20] Shengyu Chen, Shihang Feng, Yao Huang, Zhou Lei, Xiaowei Jia, Youzuo Lin, and Estaben Rougier.  Hossnet: An efficient physics-guided neural network for simulating micro-crack propagation.  Computational Materials Science, 236:112846, 2024.
  * [21] Shengyu Chen, Nasrin Kalanat, Yiqun Xie, Sheng Li, Jacob A Zwart, Jeffrey M Sadler, Alison P Appling, Samantha K Oliver, Jordan S Read, and Xiaowei Jia.  Physics-guided machine learning from simulated data with different physical parameters.  Knowledge and Information Systems, 65(8):3223–3250, 2023.
  * [22] Shengyu Chen, Shervin Sammak, Peyman Givi, Joseph P Yurko, and Xiaowei Jia.  Reconstructing high-resolution turbulent flows using physics-guided neural networks.  In 2021 IEEE International Conference on Big Data (Big Data), pages 1369–1379. IEEE, 2021.
  * [23] Shengyu Chen, Yiqun Xie, Xiang Li, Xu Liang, and Xiaowei Jia.  Physics-guided meta-learning method in baseflow prediction over large regions.  In Proceedings of the 2023 SIAM International Conference on Data Mining (SDM), pages 217–225. SIAM, 2023.
  * [24] Shengyu Chen, Jacob A Zwart, and Xiaowei Jia.  Physics-guided graph meta learning for predicting water temperature and streamflow in stream networks.  In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2752–2761, 2022.
  * [25] Zeyuan Chen, Yinbo Chen, Jingwen Liu, Xingqian Xu, Vidit Goel, Zhangyang Wang, Humphrey Shi, and Xiaolong Wang.  Videoinr: Learning video implicit neural representation for continuous space-time super-resolution.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2047–2057, 2022.
  * [26] Taco Cohen and Max Welling.  Group equivariant convolutional networks.  In International conference on machine learning, pages 2990–2999. PMLR, 2016.
  * [27] Taco S Cohen and Max Welling.  Steerable cnns.  arXiv preprint arXiv:1612.08498, 2016.
  * [28] Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah.  Diffusion models in vision: A survey.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  * [29] Arka Daw, Jie Bu, Sifan Wang, Paris Perdikaris, and Anuj Karpatne.  Mitigating propagation failures in physics-informed neural networks using retain-resample-release (r3) sampling.  In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org, 2023.
  * [30] Arka Daw, Anuj Karpatne, William D Watkins, Jordan S Read, and Vipin Kumar.  Physics-guided neural networks (pgnn): An application in lake temperature modeling.  In Knowledge Guided Machine Learning, pages 353–372. Chapman and Hall/CRC, 2022.
  * [31] Arka Daw, R Quinn Thomas, Cayelan C Carey, Jordan S Read, Alison P Appling, and Anuj Karpatne.  Physics-guided architecture (pga) of neural networks for quantifying uncertainty in lake temperature modeling.  In Proceedings of the 2020 SIAM International Conference on Data Mining, pages 532–540. SIAM, 2020.
  * [32] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei.  ImageNet: A Large-Scale Hierarchical Image Database.  In CVPR09, 2009.
  * [33] Stephen J Dugdale, David M Hannah, and Iain A Malcolm.  River temperature modelling: A review of process-based approaches and future directions.  Earth-Science Reviews, 175:97–113, 2017.
  * [34] Karthik Duraisamy, Gianluca Iaccarino, and Heng Xiao.  Turbulence modeling in the age of data.  Annual review of fluid mechanics, 51:357–377, 2019.
  * [35] Mohannad Elhamod, Jie Bu, Christopher Singh, Matthew Redell, Abantika Ghosh, Viktor Podolskiy, Wei-Cheng Lee, and Anuj Karpatne.  Cophy-pgnn: Learning physics-guided neural networks with competing loss functions for solving eigenvalue problems.  ACM Transactions on Intelligent Systems and Technology, 13(6):1–23, 2022.
  * [36] Heinz Werner Engl, Martin Hanke, and Andreas Neubauer.  Regularization of inverse problems, volume 375.  Springer Science & Business Media, 1996.
  * [37] J. H. Faghmous, A. Banerjee, S. Shekhar, M. Steinbach, V. Kumar, A. R. Ganguly, and N. Samatova.  Theory-guided data science for climate change.  Computer, 47(11):74–78, 2014.
  * [38] Yue Fan, Wenxi Lu, Tiansheng Miao, Yongkai An, Jiuhui Li, and Jiannan Luo.  Optimal design of groundwater pollution monitoring network based on the svr surrogate model under uncertainty.  Environmental Science and Pollution Research, 27:24090–24102, 2020.
  * [39] Simone Fatichi, Enrique R Vivoni, Fred L Ogden, Valeriy Y Ivanov, Benjamin Mirus, David Gochis, Charles W Downer, Matteo Camporese, Jason H Davison, Brian Ebel, et al.  An overview of current applications, challenges, and future trends in distributed process-based models in hydrology.  Journal of Hydrology, 537:45–60, 2016.
  * [40] Dapeng Feng, Jiangtao Liu, Kathryn Lawson, and Chaopeng Shen.  Differentiable, learnable, regionalized process-based models with multiphysical outputs can approach state-of-the-art hydrologic prediction accuracy.  Water Resources Research, 58(10):e2022WR032404, 2022.
  * [41] J Fish and T Belytschko.  A first course in finite elements.  Wiley, 2007.
  * [42] Susannah Fisher.  Opening up new geographical ontologies around adapting to climate change.  Tijdschrift voor economische en sociale geografie, 114(2):79–85, 2023.
  * [43] JK Furtney, C Thielsen, W Fu, and Romain Le Goc.  Surrogate models in rock and soil mechanics: Integrating numerical modeling and machine learning.  Rock Mechanics and Rock Engineering, pages 1–15, 2022.
  * [44] Zhihan Gao, Xingjian Shi, Boran Han, Hao Wang, Xiaoyong Jin, Danielle Maddix, Yi Zhu, Mu Li, and Yuyang Wang.  Prediff: Precipitation nowcasting with latent diffusion models.  arXiv preprint arXiv:2307.10422, 2023.
  * [45] Hojat Ghorbanidehno, Amalia Kokkinaki, Jonghyun Lee, and Eric Darve.  Recent developments in fast and scalable inverse modeling and data assimilation methods in hydrology.  Journal of Hydrology, 591:125266, 2020.
  * [46] Abantika Ghosh, Mohannad Elhamod, Jie Bu, Wei-Cheng Lee, Anuj Karpatne, and Viktor A Podolskiy.  Physics-informed machine learning for optical modes in composites.  Advanced Photonics Research, 3(11):2200073, 2022.
  * [47] Rahul Ghosh, Arvind Renganathan, Kshitij Tayal, Xiang Li, Ankush Khandelwal, Xiaowei Jia, Christopher Duffy, John Nieber, and Vipin Kumar.  Robust inverse framework using knowledge-guided self-supervised learning: An application to hydrology.  In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 465–474, 2022.
  * [48] Rahul Ghosh, Haoyu Yang, Ankush Khandelwal, Erhu He, Arvind Renganathan, Somya Sharma, Xiaowei Jia, and Vipin Kumar.  Entity aware modelling: A survey.  arXiv preprint arXiv:2302.08406, 2023.
  * [49] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In NIPS, pages 2672–2680, 2014.
  * [50] Somdatta Goswami, Aniruddha Bora, Yue Yu, and George Em Karniadakis.  Physics-informed neural operators.  arXiv preprint arXiv:2207.05748, 2022.
  * [51] Mario Guevara and Rodrigo Vargas.  Downscaling satellite soil moisture using geomorphometry and machine learning.  PloS One, 14(9):e0219639, 2019.
  * [52] Yinan Guo, Guoyu Chen, Min Jiang, Dunwei Gong, and Jing Liang.  A knowledge guided transfer strategy for evolutionary dynamic multiobjective optimization.  IEEE Transactions on Evolutionary Computation, 2022.
  * [53] Jingye Han, Liangsheng Shi, Christos Pylianidis, Qi Yang, and Ioannis N Athanasiadis.  Deeporyza: A knowledge guided machine learning model for rice growth simulation.  In 2nd AAAI Workshop on AI for Agriculture and Food Systems, 2023.
  * [54] Paul C Hanson, Aviah B Stillman, Xiaowei Jia, Anuj Karpatne, Hilary A Dugan, Cayelan C Carey, Joseph Stachelek, Nicole K Ward, Yu Zhang, Jordan S Read, et al.  Predicting lake surface water phosphorus dynamics using process-guided machine learning.  Ecological Modelling, 430:109136, 2020.
  * [55] Nidhin Harilal, Mayank Singh, and Udit Bhatia.  Augmented convolutional lstms for generation of high-resolution climate change projections.  IEEE Access, 9:25208–25218, 2021.
  * [56] Erhu He, Yiqun Xie, Licheng Liu, Weiye Chen, Zhenong Jin, and Xiaowei Jia.  Physics guided neural networks for time-aware fairness: an application in crop yield prediction.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 14223–14231, 2023.
  * [57] Will Douglas Heaven.  Why Meta’s latest large language model survived only three days online.  MIT Technology Review, 2022.
  * [58] Kethmi Hirushini Hettige, Jiahao Ji, Shili Xiang, Cheng Long, Gao Cong, and Jingyuan Wang.  Airphynet: Harnessing physics-guided neural networks for air quality prediction.  arXiv preprint arXiv:2402.03784, 2024.
  * [59] Mary C Hill and Claire R Tiedeman.  Effective groundwater model calibration: with analysis of data, sensitivities, predictions, and uncertainty.  John Wiley & Sons, 2006.
  * [60] Michael D Himes, Joseph Harrington, Adam D Cobb, Atılım Güneş Baydin, Frank Soboczenski, Molly D O’Beirne, Simone Zorzan, David C Wright, Zacchaeus Scheffer, Shawn D Domagal-Goldman, et al.  Accurate machine-learning atmospheric retrieval via a neural-network surrogate model for radiative transfer.  The Planetary Science Journal, 3(4):91, 2022.
  * [61] Matthew R Hipsey, Louise C Bruce, Casper Boon, Brendan Busch, Cayelan C Carey, David P Hamilton, Paul C Hanson, Jordan S Read, Eduardo de Sousa, Michael Weber, et al.  A general lake model (glm 3.0) for linking with high-frequency sensor data from the global lake ecological observatory network (gleon).  Geoscientific Model Development, 12(1):473–523, 2019.
  * [62] MR Hipsey, LC Bruce, and DP Hamilton.  Glm—general lake model: Model overview and user information.  Perth (Australia): University of Western Australia Technical Manual, 2014.
  * [63] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  Advances in neural information processing systems, 33:6840–6851, 2020.
  * [64] Touseef Iqbal and Shaima Qureshi.  The survey: Text generation models in deep learning.  Journal of King Saud University-Computer and Information Sciences, 34(6):2515–2528, 2022.
  * [65] Kazuya Ishitsuka and Weiren Lin.  Physics-informed neural network for inverse modeling of natural-state geothermal systems.  Applied Energy, 337:120855, 2023.
  * [66] Julie Jebeile, Vincent Lam, and Tim Räz.  Understanding climate change with statistical downscaling and machine learning.  Synthese, 199:1877–1897, 2021.
  * [67] Xiaowei Jia, Shengyu Chen, Yiqun Xie, Haoyu Yang, Alison Appling, Samantha Oliver, and Zhe Jiang.  Modeling reservoir release using pseudo-prospective learning and physical simulations to predict water temperature.  In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), pages 91–99. SIAM, 2022.
  * [68] Xiaowei Jia, Shengyu Chen, Can Zheng, Yiqun Xie, Zhe Jiang, and Nasrin Kalanat.  Physics-guided graph diffusion network for combining heterogeneous simulated data: An application in predicting stream water temperature.  In Proceedings of the 2023 SIAM International Conference on Data Mining (SDM), pages 361–369. SIAM, 2023.
  * [69] Xiaowei Jia et al.  Physics-guided machine learning from simulation data: an application in modeling lake and river systems.  In ICDM, 2021.
  * [70] Xiaowei Jia, Mengdie Wang, Ankush Khandelwal, Anuj Karpatne, and Vipin Kumar.  Recurrent generative networks for multi-resolution satellite data: An application in cropland monitoring.  In IJCAI, 2019.
  * [71] Xiaowei Jia, Jared Willard, Anuj Karpatne, Jordan Read, Jacob Zwart, Michael Steinbach, and Vipin Kumar.  Physics guided rnns for modeling dynamical systems: A case study in simulating lake temperature profiles.  In Proceedings of the 2019 SIAM International Conference on Data Mining, pages 558–566. SIAM, 2019.
  * [72] Xiaowei Jia, Jared Willard, Anuj Karpatne, Jordan S Read, Jacob A Zwart, Michael Steinbach, and Vipin Kumar.  Physics-guided machine learning for scientific discovery: An application in simulating lake temperature profiles.  ACM/IMS Transactions on Data Science, 2(3):1–26, 2021.
  * [73] Xiaowei Jia, Jacob Zwart, Jeffrey Sadler, Alison Appling, Samantha Oliver, Steven Markstrom, Jared Willard, Shaoming Xu, Michael Steinbach, Jordan Read, et al.  Physics-guided recurrent graph model for predicting flow and temperature in river networks.  In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pages 612–620. SIAM, 2021.
  * [74] Peishi Jiang, Zhao Yang, Jiali Wang, Chenfu Huang, Pengfei Xue, TC Chakraborty, Xingyuan Chen, and Yun Qian.  Efficient super-resolution of near-surface climate modeling using the fourier neural operator.  Journal of Advances in Modeling Earth Systems, 15(7):e2023MS003800, 2023.
  * [75] Peng Jin, Xitong Zhang, Yinpeng Chen, Sharon Xiaolei Huang, Zicheng Liu, and Youzuo Lin.  Unsupervised learning of full-waveform inversion: Connecting cnn and partial differential equation in a loop.  arXiv preprint arXiv:2110.07584, 2021.
  * [76] George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang.  Physics-informed machine learning.  Nature Reviews Physics, 3(6):422–440, 2021.
  * [77] Anuj Karpatne, Gowtham Atluri, James H Faghmous, Michael Steinbach, Arindam Banerjee, Auroop Ganguly, Shashi Shekhar, Nagiza Samatova, and Vipin Kumar.  Theory-guided data science: A new paradigm for scientific discovery from data.  IEEE Transactions on Knowledge and Data Engineering, 29(10):2318–2331, 2017.
  * [78] Anuj Karpatne, Ramakrishnan Kannan, and Vipin Kumar.  Knowledge Guided Machine Learning: Accelerating Discovery Using Scientific Knowledge and Data.  CRC Press, 2022.
  * [79] Anuj Karpatne, William Watkins, Jordan Read, and Vipin Kumar.  Physics-guided neural networks (pgnn): An application in lake temperature modeling.  arXiv preprint arXiv:1710.11431, 2017.
  * [80] K Kashinath, M Mustafa, A Albert, JL Wu, C Jiang, S Esmaeilzadeh, K Azizzadenesheli, R Wang, A Chattopadhyay, A Singh, et al.  Physics-informed machine learning: case studies for weather and climate modelling.  Philosophical Transactions of the Royal Society A, 379(2194):20200093, 2021.
  * [81] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  arXiv preprint arXiv:1312.6114, 2013.
  * [82] Ivan Kobyzev, Simon JD Prince, and Marcus A Brubaker.  Normalizing flows: An introduction and review of current methods.  IEEE transactions on pattern analysis and machine intelligence, 43(11):3964–3979, 2020.
  * [83] Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar.  Neural operator: Learning maps between function spaces.  arXiv preprint arXiv:2108.08481, 2021.
  * [84] Basil Kraft, Martin Jung, Marco Körner, Sujan Koirala, and Markus Reichstein.  Towards hybrid modeling of the global hydrological cycle.  Hydrology and Earth System Sciences, 26(6):1579–1614, 2022.
  * [85] Frederik Kratzert, Daniel Klotz, Guy Shalev, Günter Klambauer, Sepp Hochreiter, and Grey Nearing.  Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets.  Hydrology and Earth System Sciences, 23(12):5089–5110, 2019.
  * [86] Mario Krenn, Robert Pollice, Si Yue Guo, Matteo Aldeghi, Alba Cervera-Lierta, Pascal Friederich, Gabriel dos Passos Gomes, Florian Häse, Adrian Jinich, AkshatKumar Nigam, et al.  On scientific understanding with artificial intelligence.  Nature Reviews Physics, 4(12):761–769, 2022.
  * [87] Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang.  Fine-tuning can distort pretrained features and underperform out-of-distribution.  arXiv preprint arXiv:2202.10054, 2022.
  * [88] Vijendra Kumar, Naresh Kedam, Kul Vaibhav Sharma, Darshan J Mehta, and Tommaso Caloiero.  Advanced machine learning techniques to improve hydrological prediction: A comparative analysis of streamflow prediction models.  Water, 15(14):2572, 2023.
  * [89] Alexandre Lacoste, Nils Lehmann, Pau Rodriguez, Evan David Sherwin, Hannah Kerner, Björn Lütjens, Jeremy Andrew Irvin, David Dao, Hamed Alemohammad, Alexandre Drouin, et al.  Geo-bench: Toward foundation models for earth monitoring.  arXiv preprint arXiv:2306.03831, 2023.
  * [90] Devyani Lambhate and Deepak N Subramani.  Super-resolution of sea surface temperature satellite images.  In Global Oceans 2020: Singapore–US Gulf Coast, pages 1–7. IEEE, 2020.
  * [91] Gustav Larsson, Michael Maire, and Gregory Shakhnarovich.  Colorization as a proxy task for visual understanding.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6874–6883, 2017.
  * [92] David Lazer, Ryan Kennedy, Gary King, and Alessandro Vespignani.  The Parable of Google Flu: Traps in Big Data Analysis.  Science (New York, N.Y.), 343(6176):1203–5, March 2014.
  * [93] Phuc H Le-Khac, Graham Healy, and Alan F Smeaton.  Contrastive representation learning: A framework and review.  Ieee Access, 8:193907–193934, 2020.
  * [94] Xiang Li, Ankush Khandelwal, Xiaowei Jia, Kelly Cutler, Rahul Ghosh, Arvind Renganathan, Shaoming Xu, Kshitij Tayal, John Nieber, Christopher Duffy, et al.  Regionalization in a global hydrologic deep learning model: from physical descriptors to random vectors.  Water Resources Research, 58(8):e2021WR031794, 2022.
  * [95] Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar.  Fourier neural operator with learned deformations for pdes on general geometries.  arXiv preprint arXiv:2207.05209, 2022.
  * [96] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar.  Fourier neural operator for parametric partial differential equations.  arXiv preprint arXiv:2010.08895, 2020.
  * [97] Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar.  Physics-informed neural operator for learning partial differential equations.  arXiv preprint arXiv:2111.03794, 2021.
  * [98] LIU Licheng, Wang Zhou, Kaiyu Guan, Bin Peng, Chongya Jiang, Jinyun Tang, Sheng Wang, Robert Grant, Symon Mezbahuddin, Xiaowei Jia, et al.  Knowledge-based artificial intelligence for agroecosystem carbon budget and crop yield estimation.  Authorea Preprints, 2023.
  * [99] LIU Licheng, Wang Zhou, Zhenong Jin, Jinyun Tang, Xiaowei Jia, Chongya Jiang, Kaiyu Guan, Bin Peng, Shaoming Xu, Yufeng Yang, et al.  Estimating the autotrophic and heterotrophic respiration in the us crop fields using knowledge guided machine learning.  Authorea Preprints, 2022.
  * [100] Marten Lienen and Stephan Günnemann.  Learning the dynamics of physical systems from sparse observations with finite element networks.  arXiv preprint arXiv:2203.08852, 2022.
  * [101] Phillip Lippe, Bastiaan S Veeling, Paris Perdikaris, Richard E Turner, and Johannes Brandstetter.  Pde-refiner: Achieving accurate long rollouts with neural pde solvers.  arXiv preprint arXiv:2308.05732, 2023.
  * [102] Lei Liu and Xue-yi You.  Study of water quality response to water transfer patterns in the receiving basin and surrogate model.  Environmental Science and Pollution Research, pages 1–19, 2021.
  * [103] Licheng Liu, Shaoming Xu, Jinyun Tang, Kaiyu Guan, Timothy J Griffis, Matthew D Erickson, Alexander L Frie, Xiaowei Jia, Taegon Kim, Lee T Miller, et al.  Kgml-ag: a modeling framework of knowledge-guided machine learning to simulate agroecosystems: a case study of estimating n 2 o emission using data from mesocosm experiments.  Geoscientific Model Development, 15(7):2839–2858, 2022.
  * [104] Licheng Liu, Wang Zhou, Kaiyu Guan, Bin Peng, Shaoming Xu, Jinyun Tang, Qing Zhu, Jessica Till, Xiaowei Jia, Chongya Jiang, et al.  Knowledge-guided machine learning can improve carbon cycle quantification in agroecosystems.  Nature Communications, 15(1):357, 2024.
  * [105] Yangxiaoyue Liu, Xiaolin Xia, Ling Yao, Wenlong Jing, Chenghu Zhou, Wumeng Huang, Yong Li, and Ji Yang.  Downscaling satellite retrieved soil moisture using regression tree-based machine learning algorithms over southwest france.  Earth and Space Science, 7(10):e2020EA001267, 2020.
  * [106] Lennart Ljung.  System identification.  In Signal analysis and prediction, pages 163–173. Springer, 1998.
  * [107] Cooper Lorsung and Amir Barati Farimani.  Picl: Physics informed contrastive learning for partial differential equations.  arXiv preprint arXiv:2401.16327, 2024.
  * [108] Lu Lu, Pengzhan Jin, and George Em Karniadakis.  Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators.  arXiv preprint arXiv:1910.03193, 2019.
  * [109] Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis.  Learning nonlinear operators via deeponet based on the universal approximation theorem of operators.  Nature machine intelligence, 3(3):218–229, 2021.
  * [110] Shiyuan Luo, Juntong Ni, Shengyu Chen, Runlong Yu, Yiqun Xie, Licheng Liu, Zhenong Jin, Huaxiu Yao, and Xiaowei Jia.  Free: The foundational semantic recognition for modeling environmental ecosystems.  arXiv preprint arXiv:2311.10255, 2023.
  * [111] Yingtao Luo, Qiang Liu, Yuntian Chen, Wenbo Hu, Tian Tian, and Jun Zhu.  Physics-guided discovery of highly nonlinear parametric partial differential equations.  In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1595–1607, 2023.
  * [112] Björn Lütjens, Catherine H Crawford, Campbell D Watson, Christopher Hill, and Dava Newman.  Multiscale neural operator: Learning fast and grid-independent pde solvers.  arXiv preprint arXiv:2207.11417, 2022.
  * [113] Peter Lynch.  The origins of computer weather prediction and climate modeling.  Journal of computational physics, 227(7):3431–3444, 2008.
  * [114] Gengchen Mai, Weiming Huang, Jin Sun, Suhang Song, Deepak Mishra, Ninghao Liu, Song Gao, Tianming Liu, Gao Cong, Yingjie Hu, et al.  On the opportunities and challenges of foundation models for geospatial artificial intelligence.  arXiv preprint arXiv:2304.06798, 2023.
  * [115] Annikki Mäkelä, Joe Landsberg, Alan R Ek, Thomas E Burk, Michael Ter-Mikaelian, Göran I Ågren, Chadwick D Oliver, and Pasi Puttonen.  Process-based models for forest ecosystem management: current state of the art and challenges for practical implementation.  Tree physiology, 20(5-6):289–298, 2000.
  * [116] Gary Marcus and Ernest Davis.  Eight (no, nine!) problems with big data.  The New York Times, 6(04):2014, 2014.
  * [117] Michael McCabe, Bruno Régaldo-Saint Blancard, Liam Holden Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Siavash Golkar, Geraud Krawezik, Francois Lanusse, et al.  Multiple physics pretraining for physical surrogate models.  arXiv preprint arXiv:2310.02994, 2023.
  * [118] Nikhil Muralidhar, Mohammad Raihanul Islam, Manish Marwah, Anuj Karpatne, and Naren Ramakrishnan.  Incorporating prior domain knowledge into deep neural networks.  In 2018 IEEE International Conference on Big Data (Big Data), pages 36–45. IEEE, 2018.
  * [119] Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K Gupta, and Aditya Grover.  Climax: A foundation model for weather and climate.  arXiv preprint arXiv:2301.10343, 2023.
  * [120] Peter Norvig.  Natural language corpus data.  Beautiful data, pages 219–242, 2009.
  * [121] R OpenAI.  Gpt-4 technical report. arxiv 2303.08774.  View in Article, 2, 2023.
  * [122] Sungwon Park, Karandeep Singh, Arjun Nellikkattil, Elke Zeller, Tung Duong Mai, and Meeyoung Cha.  Downscaling earth system models with deep learning.  In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3733–3742, 2022.
  * [123] Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros.  Context encoders: Feature learning by inpainting.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2536–2544, 2016.
  * [124] Judea Pearl.  The seven tools of causal inference, with reflections on machine learning.  Communications of the ACM, 62(3):54–60, 2019.
  * [125] Judea Pearl and Dana Mackenzie.  The book of why: the new science of cause and effect.  Basic books, 2018.
  * [126] Laura Pilozzi, Francis A Farrelly, Giulia Marcucci, and Claudio Conti.  Machine learning inverse problem for topological photonics.  Communications Physics, 1(1):1–7, 2018.
  * [127] Maziar Raissi.  Deep hidden physics models: Deep learning of nonlinear partial differential equations.  The Journal of Machine Learning Research, 19(1):932–955, 2018.
  * [128] Maziar Raissi, Paris Perdikaris, and George E Karniadakis.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.  Journal of Computational Physics, 378:686–707, 2019.
  * [129] Maziar Raissi, Paris Perdikaris, and George Em Karniadakis.  Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations.  arXiv preprint arXiv:1711.10561, 2017.
  * [130] Maziar Raissi, Paris Perdikaris, and George Em Karniadakis.  Physics informed deep learning (part ii): Data-driven discovery of nonlinear partial differential equations.  arXiv preprint arXiv:1711.10566, 2017.
  * [131] Stephan Rasp, Michael S Pritchard, and Pierre Gentine.  Deep learning to represent subgrid processes in climate models.  Proceedings of the National Academy of Sciences, 115(39):9684–9689, 2018.
  * [132] Saman Razavi, Bryan A Tolson, and Donald H Burn.  Review of surrogate modeling in water resources.  Water Resources Research, 48(7), 2012.
  * [133] Jordan S Read, Xiaowei Jia, Jared Willard, Alison P Appling, Jacob A Zwart, Samantha K Oliver, Anuj Karpatne, Gretchen JA Hansen, Paul C Hanson, William Watkins, et al.  Process-guided deep learning predictions of lake water temperature.  Water Resources Research, 55(11):9173–9190, 2019.
  * [134] Markus Reichstein, Gustau Camps-Valls, Bjorn Stevens, Martin Jung, Joachim Denzler, Nuno Carvalhais, et al.  Deep learning and process understanding for data-driven earth system science.  Nature, 566(7743):195–204, 2019.
  * [135] Samuel H Rudy, Steven L Brunton, Joshua L Proctor, and J Nathan Kutz.  Data-driven discovery of partial differential equations.  Science advances, 3(4):e1602614, 2017.
  * [136] F Sabzehee, AR Amiri-Simkooei, S Iran-Pour, BD Vishwakarma, and R Kerachian.  Enhancing spatial resolution of grace-derived groundwater storage anomalies in urmia catchment using machine learning downscaling methods.  Journal of Environmental Management, 330:117180, 2023.
  * [137] Vıctor Garcia Satorras, Emiel Hoogeboom, and Max Welling.  E (n) equivariant graph neural networks.  In International conference on machine learning, pages 9323–9332. PMLR, 2021.
  * [138] Raphael Schneider, Julian Koch, Lars Troldborg, Hans Jørgen Henriksen, and Simon Stisen.  Machine-learning-based downscaling of modelled climate change impacts on groundwater table depth.  Hydrology and Earth System Sciences, 26(22):5859–5877, 2022.
  * [139] Somya Sharma, Swati Sharma, Licheng Liu, Rishabh Tushir, Andy Neal, Robert Ness, John Crawford, Emre Kiciman, and Ranveer Chandra.  Knowledge guided representation learning and causal structure learning in soil science.  arXiv preprint arXiv:2306.09302, 2023.
  * [140] Chaopeng Shen, Alison P Appling, Pierre Gentine, Toshiyuki Bandai, Hoshin Gupta, Alexandre Tartakovsky, Marco Baity-Jesi, Fabrizio Fenicia, Daniel Kifer, Li Li, et al.  Differentiable modelling to unify machine learning and physical models for geosciences.  Nature Reviews Earth & Environment, 4(8):552–567, 2023.
  * [141] Muhammed Sit, Bekir Zahit Demiray, and Ibrahim Demir.  Spatial downscaling of streamflow data with attention based spatio-temporal graph convolutional networks.  2023\.
  * [142] Albert Tarantola.  Inverse problem theory and methods for model parameter estimation.  SIAM, 2005.
  * [143] Kshitij Tayal, Xiaowei Jia, Rahul Ghosh, Jared Willard, Jordan Read, and Vipin Kumar.  Invertibility aware integration of static and time-series data: An application to lake temperature modeling.  In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), pages 702–710. SIAM, 2022.
  * [144] Kshitij Tayal, Arvind Renganathan, Rahul Ghosh, Xiaowei Jia, and Vipin Kumar.  Koopman invertible autoencoder: Leveraging forward and backward dynamics for temporal modeling.  In 2023 IEEE International Conference on Data Mining (ICDM), pages 588–597. IEEE, 2023.
  * [145] Michael L Thompson and Mark A Kramer.  Modeling chemical processes using prior knowledge and neural networks.  AIChE Journal, 40(8):1328–1340, 1994.
  * [146] Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong.  Factorized fourier neural operators.  arXiv preprint arXiv:2111.13802, 2021.
  * [147] Wen-Ping Tsai, Dapeng Feng, Ming Pan, Hylke Beck, Kathryn Lawson, Yuan Yang, Jiangtao Liu, and Chaopeng Shen.  From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modeling.  Nature communications, 12(1):5988, 2021.
  * [148] Laura von Rueden, Sebastian Mayer, Katharina Beckh, Bogdan Georgiev, Sven Giesselbach, Raoul Heese, Birgit Kirsch, Julius Pfrommer, Annika Pick, Rajkumar Ramamurthy, et al.  Informed machine learning–a taxonomy and survey of integrating knowledge into learning systems.  arXiv preprint arXiv:1903.12394, 2019.
  * [149] Yue Wan, Jialu Wu, Tingjun Hou, Chang-Yu Hsieh, and Xiaowei Jia.  From molecules to scaffolds to functional groups: building context-dependent molecular representation via multi-channel learning.  arXiv preprint arXiv:2311.02798, 2023.
  * [150] Fang Wang, Di Tian, Lisa Lowe, Latif Kalin, and John Lehrter.  Deep learning for daily precipitation and temperature downscaling.  Water Resources Research, 57(4):e2020WR029308, 2021.
  * [151] Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, et al.  Scientific discovery in the age of artificial intelligence.  Nature, 620(7972):47–60, 2023.
  * [152] Rui Wang, Robin Walters, and Rose Yu.  Incorporating symmetry into deep dynamics models for improved generalization.  arXiv preprint arXiv:2002.03061, 2020.
  * [153] Sifan Wang, Shyam Sankaran, Hanwen Wang, and Paris Perdikaris.  An expert’s guide to training physics-informed neural networks.  arXiv preprint arXiv:2308.08468, 2023.
  * [154] Sifan Wang, Yujun Teng, and Paris Perdikaris.  Understanding and mitigating gradient flow pathologies in physics-informed neural networks.  SIAM Journal on Scientific Computing, 43(5):A3055–A3081, 2021.
  * [155] Sifan Wang, Hanwen Wang, and Paris Perdikaris.  Learning the solution operator of parametric partial differential equations with physics-informed deeponets.  Science advances, 7(40):eabi8605, 2021.
  * [156] Sifan Wang, Hanwen Wang, and Paris Perdikaris.  Improved architectures and training algorithms for deep operator networks.  Journal of Scientific Computing, 92(2):35, 2022.
  * [157] Zhengwei Wang, Qi She, and Tomas E Ward.  Generative adversarial networks in computer vision: A survey and taxonomy.  ACM Computing Surveys (CSUR), 54(2):1–38, 2021.
  * [158] Zhihao Wang, Yiqun Xie, Xiaowei Jia, Lei Ma, and George Hurtt.  High-fidelity deep approximation of ecosystem simulation over long-term at large scale.  In Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems, pages 1–10, 2023.
  * [159] Zhihao Wang, Yiqun Xie, Zhili Li, Xiaowei Jia, Zhe Jiang, Aolin Jia, and Shuo Xu.  Simfair: Physics-guided fairness-aware learning with simulation models.  arXiv preprint arXiv:2401.15270, 2024.
  * [160] Zhihao Wei, Kebin Jia, Xiaowei Jia, Pengyu Liu, Ying Ma, Ting Chen, and Guilian Feng.  Mapping large-scale plateau forest in sanjiangyuan using high-resolution satellite imagery and few-shot learning.  Remote Sensing, 14(2):388, 2022.
  * [161] Zhihao Wei, Kebin Jia, Pengyu Liu, Xiaowei Jia, Yiqun Xie, and Zhe Jiang.  Large-scale river mapping using contrastive learning and multi-source satellite imagery.  Remote Sensing, 13(15):2893, 2021.
  * [162] Johannes G Wijmans and Richard W Baker.  The solution-diffusion model: a review.  Journal of membrane science, 107(1-2):1–21, 1995.
  * [163] Jared Willard, Xiaowei Jia, Shaoming Xu, Michael Steinbach, and Vipin Kumar.  Integrating scientific knowledge with machine learning for engineering and environmental systems.  ACM Computing Surveys, 55(4):1–37, 2022.
  * [164] Jared D Willard, Jordan S Read, Alison P Appling, Samantha K Oliver, Xiaowei Jia, and Vipin Kumar.  Predicting water temperature dynamics of unmonitored lakes with meta-transfer learning.  Water Resources Research, 57(7):e2021WR029579, 2021.
  * [165] Jiantao Wu, Fabrizio Orlandi, Declan O’Sullivan, and Soumyabrata Dev.  An ontology model for climatic data analysis.  In 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pages 5739–5742. IEEE, 2021.
  * [166] Jiantao Wu, Fabrizio Orlandi, Declan O’Sullivan, and Soumyabrata Dev.  Linkclimate: An interoperable knowledge graph platform for climate data.  Computers & Geosciences, 169:105215, 2022.
  * [167] Jiantao Wu, Jarrett Pierse, Fabrizio Orlandi, Declan O’Sullivan, and Soumyabrata Dev.  Improving tourism analytics from climate data using knowledge graphs.  IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 16:2402–2412, 2023.
  * [168] Jin-Long Wu, Karthik Kashinath, Adrian Albert, Dragos Chirila, Heng Xiao, et al.  Enforcing statistical constraints in generative adversarial networks for modeling chaotic dynamical systems.  Journal of Computational Physics, 406:109209, 2020.
  * [169] Yiqun Xie, Shashi Shekhar, and Yan Li.  Statistically-robust clustering techniques for mapping spatial hotspots: A survey.  ACM Computing Surveys (CSUR), 55(2):1–38, 2022.
  * [170] Yiqun Xie, Zhaonan Wang, Gengchen Mai, Yanhua Li, Xiaowei Jia, Song Gao, and Shaowen Wang.  Geo-foundation models: Reality, gaps and opportunities.  In Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems, pages 1–4, 2023.
  * [171] Bo Xu, Haofei Yu, Zongbo Shi, Jinxing Liu, Yuting Wei, Zhongcheng Zhang, Yanqi Huangfu, Han Xu, Yue Li, Linlin Zhang, et al.  Knowledge-guided machine learning reveals pivotal drivers for gas-to-particle conversion of atmospheric nitrate.  Environmental Science and Ecotechnology, 19:100333, 2024.
  * [172] Qingsong Xu, Yilei Shi, Jonathan Bamber, Ye Tuo, Ralf Ludwig, and Xiao Xiang Zhu.  Physics-aware machine learning revolutionizes scientific paradigm for machine learning and process-based hydrology.  arXiv preprint arXiv:2310.05227, 2023.
  * [173] Yuanhao Xu, Kairong Lin, Caihong Hu, Shuli Wang, Qiang Wu, Li Zhang, and Guang Ran.  Deep transfer learning based on transformer for flood forecasting in data-sparse basins.  Journal of Hydrology, 625:129956, 2023.
  * [174] Jiexi Yan, Lei Luo, Cheng Deng, and Heng Huang.  Adaptive hierarchical similarity metric learning with noisy labels.  IEEE Transactions on Image Processing, 32:1245–1256, 2023.
  * [175] Qi Yang, Licheng Liu, Junxiong Zhou, Rahul Ghosh, Bin Peng, Kaiyu Guan, Jinyun Tang, Wang Zhou, Vipin Kumar, and Zhenong Jin.  A flexible and efficient knowledge-guided machine learning data assimilation (kgml-da) framework for agroecosystem prediction in the us midwest.  Remote Sensing of Environment, 299:113880, 2023.
  * [176] Qidong Yang, Alex Hernandez-Garcia, Paula Harder, Venkatesh Ramesh, Prasanna Sattegeri, Daniela Szwarcman, Campbell D Watson, and David Rolnick.  Fourier neural operators for arbitrary resolution climate data downscaling.  arXiv preprint arXiv:2305.14452, 2023.
  * [177] Zeng Yang, Jin-Long Wu, and Heng Xiao.  Enforcing deterministic constraints on generative adversarial networks for emulating physical systems.  arXiv preprint arXiv:1911.06671, 2019.
  * [178] Abbas Yeganeh-Bakhtiary, Hossein EyvazOghli, Naser Shabakhty, Bahareh Kamranzad, and Soroush Abolfathi.  Machine learning as a downscaling approach for prediction of wind characteristics under future climate change scenarios.  Complexity, 2022, 2022.
  * [179] Faria T Zahura and Jonathan L Goodall.  Predicting combined tidal and pluvial flood inundation using a machine learning surrogate model.  Journal of Hydrology: Regional Studies, 41:101087, 2022.
  * [180] Di Zhang, Dongsheng Wang, Qidong Peng, Junqiang Lin, Tiantian Jin, Tiantian Yang, Soroosh Sorooshian, and Yi Liu.  Prediction of the outflow temperature of large-scale hydropower using theory-guided machine learning surrogate models of a high-fidelity hydrodynamics model.  Journal of Hydrology, 606:127427, 2022.
  * [181] Xuan Zhang, Limei Wang, Jacob Helwig, Youzhi Luo, Cong Fu, Yaochen Xie, Meng Liu, Yuchao Lin, Zhao Xu, Keqiang Yan, et al.  Artificial intelligence for science in quantum, atomistic, and continuum systems.  arXiv preprint arXiv:2307.08423, 2023.
  * [182] Jiawei Zhao, Robert Joseph George, Yifei Zhang, Zongyi Li, and Anima Anandkumar.  Incremental fourier neural operator.  arXiv preprint arXiv:2211.15188, 2022.
  * [183] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al.  A survey of large language models.  arXiv preprint arXiv:2303.18223, 2023.
  * [184] Qing Zhu, Fa Li, William J Riley, Li Xu, Lei Zhao, Kunxiaojia Yuan, Huayi Wu, Jianya Gong, and James Randerson.  Building a machine learning surrogate model for wildfire activities within a global earth system model.  Geoscientific Model Development, 15(5):1899–1911, 2022.
  * [185] Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He.  A comprehensive survey on transfer learning.  Proceedings of the IEEE, 109(1):43–76, 2020.