# Spatio-Temporal Fluid Dynamics Modeling via Physical-Awareness and Parameter
Diffusion Guidance

Hao Wu†, Fan Xu†, Yifan Duan, Ziwei Niu, Weiyan Wang, Gaofeng Lu, Kun
Wang${}^{\textrm{{\char 0\relax}}}$, Yuxuan Liang, and Yang
Wang${}^{\textrm{{\char 0\relax}}}$ H. Wu, F. Xu, Y F. Duan, G. Lu, K Wang and
Y. Wang are all with University of Science and Technology of China, Hefei,
Anhui, P.R.China. H. Wu, and W Y. Wang are all with Tencent. Machine Learning
Platform Department, Beijing, China. W Y. Wang is with The Hong Kong
University of Science and Technology. Ziwei Niu is with Zhejiang University,
Hangzhou, Zhejiang, P.R.China. Yuxuan Liang is with INTR Thrust & DSA Thrust,
The Hong Kong University of Science and Technology (Guangzhou). ✉ Yang Wang
and Kun Wang are the corresponding authors. Email<EMAIL_ADDRESS>&
<EMAIL_ADDRESS>
###### Abstract

This paper proposes a two-stage framework named ST-PAD for spatio-temporal
fluid dynamics modeling in the field of earth sciences, aiming to achieve
high-precision simulation and prediction of fluid dynamics through spatio-
temporal physics awareness and parameter diffusion guidance. In the upstream
stage, we design a vector quantization reconstruction module with temporal
evolution characteristics, ensuring balanced and resilient parameter
distribution by introducing general physical constraints. In the downstream
stage, a diffusion probability network involving parameters is utilized to
generate high-quality future states of fluids, while enhancing the model’s
generalization ability by perceiving parameters in various physical setups.
Extensive experiments on multiple benchmark datasets have verified the
effectiveness and robustness of the ST-PAD framework, which showcase that ST-
PAD outperforms current mainstream models in fluid dynamics modeling and
prediction, especially in effectively capturing local representations and
maintaining significant advantages in OOD generations. The source code is
available at https://github.com/easylearningscores/STFD.

###### Index Terms:

Spatio-temporal Data Mining, Fluid Dynamics, Out-of-distribution Generation.

##  1 Introduction

Fluid dynamics [1, 2, 3] is a significant issue in earth science research. A
deep understanding of fluid flows can greatly aid in explaining a vast array
of earth phenomena, such as combustion dynamics [4, 5], ocean dynamics [6, 7],
and molecular dynamics [8, 9], to name just a few. Typically, modeling fluid
dynamics involves incorporating historical temporal information and entails
the interaction between various spatial event states [10, 11, 12], which can
be further understood as a spatio-temporal modeling issue [13, 14].

Figure 1: We observe that a model trained under scenario 1 does not perform
well when directly transferred to scenario 2.

Modeling fluids presents a complex challenge due to their molecular structure,
which lacks resistance to external shear forces. As a result, fluids are
susceptible to deformation even under minimal forces, leading to highly
nonlinear and chaotic behavior [3]. From a theoretical standpoint, the
dynamics of fluids are governed by underlying partial differential equations
(PDEs) [15, 16]. Traditional approaches typically employ continuous Navier-
Stokes equations [17] to model fluid dynamics. Unfortunately, the high
dimensionality of fluids introduces notorious inefficiency challenges to the
solving process [18, 19]. Worse still, when modeling fluids, the training data
encompasses both the dynamics themselves and the parameters of the dynamical
system’s PDEs. Training within a singular scenario struggles to generalize
across systems with different parameter regimes [20, 21, 22].

TABLE I: Summary of existing high-performance numerical simulation models, PINNs, and pure data-driven models. The ST-PAD is distinguished by its unique ability to simultaneously offer the combined benefits of efficiency, PDE involvement, and generalizability. Traditional & Physical-Informed NN | High-performances Methods
---|---
FDS [23] | FLUENT [24] | PINN [25] | MAD [26] | PPINN [27] |  CAN-PINN [28]
Numerical calculation | PDE-involve | ✓ | ✓ |  |  |  |
Efficiency | × | × |  |  |  |
Generalizability | × | × |  |  |  |
PINN | PDE-involve |  |  | ✓ | ✓ | ✓ | ✓
Generalizability |  |  | × | × | × | ×
Data Driven | SOTA Methods
TAU [29] | PastNet [30] | Earthfarseer [13] | FNO [1] | DeepoNet [2] | ST-PAD (Ours)
Purely Deep NN | Efficiency | ✓ | ✓ | ✓ | ✓ | ✓ | ✓
PDE-involve | × | × | × | ✓ | × | ✓
Generalizability | × | × | × | × | × | ✓

To this end, an emerging trend in deep fluid prediction involves training deep
models to solve governing PDEs [2, 1, 31]. To address this, an emerging trend
in deep fluid prediction focuses on training deep models to resolve governing
partial differential equations (PDEs). A notable branch in this area is the
advent of Physics-Informed Neural Networks (PINNs) [32, 33, 34, 35], which
integrate deep learning principles with physics to tackle challenges in
scientific computing, especially within the realm of fluid dynamics. PINNs
enhance traditional neural network models by including a term in the loss
function that accounts for the physical laws governing fluid dynamics, such as
the Navier-Stokes equations [25, 36, 37]. This inclusion guarantees that the
network’s predictions not only align with empirical data but also adhere to
the fundamental principles of fluid dynamics. However, off-the-shelf PINNs
often lack sufficient generalization capabilities, primarily due to customized
loss function designs and the overlooking of network-specific parameter
contexts [20, 38].

Although some studies attempt to learn fluid dynamics through data-driven
approaches, these efforts [30, 13, 6, 1, 3, 39, 2, 1, 40, 41] predominantly
employ a purely data-driven fashion for prediction, which does not permit
generalization to PDE parameters in test data that differ from those in the
training data. As illustrated in Fig 1, we train fluid dynamics based on the
motion state of cars under the parameter setting $\gamma$. We observe that it
is challenging to transfer to the fluid motion of airplanes under the $\delta$
setting, denoted as $\gamma\not\Rightarrow\delta$. Though few models consider
PDE parameters, they are customized for specific neural networks [42, 43], and
such efforts are susceptible to perturbing the stable features, which can lead
to a loss of control over the fluid’s evolutionary characteristics. The above
dilemmas make it challenging for the deep fluid research community to design
models with high generalization capabilities that cater not only to the
initial conditions but also to different types of PDEs and their parameters.
With this consideration, in our study, we explore whether it is possible to
employ a universal fluid method that, while enhancing the network’s
generalization ability regarding PDE parameters, can also stably model the
fluid dynamics change process, maintaining the model’s stable understanding of
the entire dynamics.

To achieve this objective, we meticulously crafted a two-stage framework that
incorporates spatio-temporal physical-awareness and parameter diffusion
guidance, termed ST-PAD. Specifically, in the upstream, we design a Vector
Quantized (VQ) reconstruction module with time-evolution characteristics. This
module secures a reasonable parameter distribution by introducing a universal
physical loss, such as thermal conduction [44] and convective diffusion loss
[45]. To better capture the characteristics of the data, we incorporate Vector
Quantized technology. Notably, since traditional VQ [46, 47, 48] techniques
struggle with temporal tasks, we refine the entire VQ process with a Fourier
module. This ensures observations can be made globally in the Fourier domain
[30, 49], enabling the discovery of critical features throughout the temporal
process, rather than allowing the model to focus on trivial parts.

Going beyond this process, we ensure the generation of high-quality image
content while achieving parametric awareness of multiple physical settings
through a parameter-involved diffusion probabilistic network [50, 51, 52, 53]
in the downstream. Our approach provides a guiding framework for generating
high-quality fluid dynamics parameter data for the first time, without relying
on complex mechanisms, i.e., channel attention mechanism [20]. The
contributions of this paper can be summarized as follow:

  * ❶

We propose a two-stage framework, ST-PAD, that systematically locks in the
underlying task rules through “pre-train” in the upstream stage, and controls
the model’s generalization ability with downstream PDE parameter-involved
diffusion “fine-tuning”. To our knowledge, this is the first framework aimed
at modeling the generalization capability for fluid dynamics.

  * ❷

Our parameter-involved diffusion design, by integrating parameters into the
diffusion process, enables the rapid generation of reliable data related to
environmental settings, ensuring the capability for OOD generalization. This
offers a dependable and quick sample generation solution for future fluid
modeling. More importantly, as a plug-and-play and architecture-agnostic
framework, this approach can be transferred to any Earth science task.

  * ❸

Extensive experiments and iterative ablation studies demonstrate the strong
generalization capabilities of ST-PAD. We compare its performance and
generalizability against 12 of the current best models, and the results
indicate that it can quickly adapt to different physical parameters.

##  2 Related work

###  2.1 Spatiotemporal Predictive Learning

In recent years, many novel architectures have emerged in the field of spatio-
temporal (ST) predictive learning, various neural network architectures use
different inductive biases to process data [54]. They can be classified into
three main categories: (I) Convolutional Neural Network (CNN)-based
architectures. This research line focuses on extracting spatial features by
employing CNN-based architecture [55, 56, 57, 58]; (II) Recurrent Neural
Network (RNN)-based architectures [59, 60, 61, 62]. Within this category,
endeavors focus on optimizing the process of time-series data with some RNN
and LSTM-based designs. (III) Transformer-based architectures. In a parallel
vein, many models adopt Transformer-based frameworks to handle ST data [63,
64]. Going beyond these research lines, The hybrid model integrates the
frameworks of multiple research lines, aiming to tap into the potential of
each direction, while also inevitably introducing some additional parameters
[30, 29].

###  2.2 Fluid Dynamics Modeling

Early methods for fluid dynamics modeling primarily focused on utilizing tools
like FDS [23, 65], FLUENT [24, 66], and Comsol [67, 68] for solving PDEs. Due
to the inefficiency of these methods and their difficulty in scaling to large
datasets, many Physical Informed Neural Networks (PINNs) have subsequently
attempted to combine physical loss and deep models to map historical data to
future scenarios, which has become the mainstream paradigm in fluid modeling
[32, 33, 34, 35].

Though promising, PINNs heavily rely on domain-specific knowledge to introduce
targeted PDE equations, making it challenging to enhance the generalization
capability of fluid modeling. Then, many endeavors attempt to use purely data-
driven deep models to model fluid dynamics. A resilient branch try to use
neural operators to map historical data to future scenarios has become the
mainstream paradigm in fluid modeling. This trend started with the
introduction of DeepONet by Lu et al. [2], benefiting from the approximation
theory of neural networks. Soon after, Li et al [1]. proposed the FNO, which
approximates the temporal integration process in fluid prediction by learning
in the frequency domain. Building on this, Geo-FNO [40] is specially optimized
for irregular grids. The Galerkin Transformer [41], with enhanced Galerkin
attention, exploits the excellent ability of attention mechanisms to handle
long sequence data, calculating the correlation between features with linear
complexity. GNOT [69] learns an efficient transformer by integrating grid
information, equation parameters, and observation data. FactFormer [70]
accelerates the computation process by decomposing the attention mechanism
into a low-rank form. NMO [71] discovers finite subspaces from infinite-
dimensional spaces through manifold learning, significantly improving the
efficiency of PDE solutions and achieving state-of-the-art performance.
However, the above method relies only on historical observation data, leading
to poor generalization performance outside the distribution and lacks
consideration of physical parameters (See Tab I for comparisons).

###  2.3 Out-of-distribution Generalization

The goal of Out-of-Distribution (OOD) generalization research line [72, 73,
74] is to improve the model’s adaptability and robustness to new domains or
data distributions not seen during training. This capability is especially
critical for the practical implementation and deployment of applications, as
it allows models to maintain high performance when facing unknown environments
or tasks. OOD generalization is applied across multiple fields, including
computer vision [75], natural language processing [76], and ST data mining
[77], helping models make better decisions and predictions in the face of the
real world’s complexity and variability. Many methods achieve OOD
generalization through invariant learning [78, 79, 80], aiming to construct
invariant features across domains in latent space to minimize the misleading
correlations caused by distribution changes. Moreover, techniques such as
causal inference [81, 82], model selection [83, 84], and active learning [85,
86] are used to enhance models’ OOD generalization capabilities in real-world
situations. We make our models more robust during the evolution of dynamical
systems by combining ST physical awareness and guided parameter diffusion.

##  3 Preliminary

Background of fluid dynamics. In this article, we discuss partial differential
equations (PDEs) in fluid dynamics defined over a time dimension $t\in[0,T]$
and a spatial dimension
$\mathbf{x}=[x,y]\in\mathcal{X}\subseteq\mathbb{R}^{2}$. The spatial dimension
forms a $h\times w$ grid, with each point $(x,y)$ representing a specific
location in space. This setup allows for numerical approximations of fluid
dynamics equations and the representation of fluid behavior at different
locations on the grid. The fluid dynamics equations are:

$\displaystyle\frac{\partial u}{\partial t}=F(t,x,y,u,\nabla
u,\nabla^{2}u,\ldots)\quad$ (1)
$\displaystyle\text{s.t.}\quad(t,x,y)\in[0,T]\times\mathcal{X},$

where $u(t,x,y)$ represents the state of the fluid, which can be a function of
physical quantities like velocity, pressure, or density. The left side of the
equation, $\frac{\partial u}{\partial t}$, shows the rate of change of the
fluid state over time. The function $F$ on the right side describes how the
fluid state changes based on its current state and spatial variations ($u$ and
its spatial derivatives $\nabla u,\nabla^{2}u,\ldots$). This function can
incorporate various physical processes, including the effects of fluid
viscosity, pressure gradients, and external forces. The initial condition
$u(0,x,y)=u_{0}(x,y)$ specifies the fluid state at each spatial location at
$t=0$, and the boundary condition $B[u](t,x,y)=0$ sets rules for fluid
behavior at the boundaries. These rules can include fixed values (Dirichlet
boundary conditions) or fluid flow rates (Neumann boundary conditions) [87,
88].

Problem definition: ST-PAD addresses solving PDEs in fluid dynamics by mapping
them into sequences of spatial-temporal fields,
$\left\\{u_{k}\right\\}_{k=0}^{N}$. Each $u_{k}$ reflects the fluid’s
condition at time $t_{k}$ and coordinates $x,y$. With the timeline discretized
into intervals $t=T/N$ and covering all $x,y$ combinations, we precisely
depict fluid motion. Concretely, we aim to emulate numerical simulations
through a function $M:\mathcal{X}\rightarrow\mathcal{Y}$, which transitions
each state $u_{k}$ to $u_{k+1}$ on a $h\times w$ grid, using auto-regressive
fashion, which predicts future states from a series of fluid field data
tensors of length $\lambda$, generating a sequence
$\left\\{\bar{u}_{k}\right\\}_{k=\lambda}^{N}$. Model performance is assessed
by comparing these forecasts against ground-truth, using mean squared error
during the training phase:

$\mathcal{L}(\theta)=\sum_{k=\lambda}^{N}{\rm{MSE}}\;(u_{k},\bar{u}_{k}),$ (2)

By optimizing model parameters $\theta$, we aim to reduce the difference
between predictions and actual observations, improving prediction accuracy.
This method considers both time autoregressive properties and spatial
complexity, including all combinations of $x,y$ at $h\times w$ resolution,
assuming $\lambda>1$, enabling the model to start generating ST sequences with
an initial input sequence, capturing fluid details over time in complex
spaces.

##  4 Methodology

Figure 2: Overview of the ST-PAD framework: The upper half demonstrates the
upstream model, which locks in a reasonable range of model parameter
distributions through self-supervision. The lower half of the model utilizes a
parameter-involved diffusion model to enhance the model’s generalization
capabilities across different environmental settings.

In this section, we will systematically describe the technical details of the
ST-PAD framework. The pipeline of our ST-PAD framework is presented in Fig. 2,
which takes historical fluid observations as input, that is, a spatio-temporal
sequence. Specifically, we first present the entire process of upstream
parameter locking in section 4.1. Subsequently, we detail the design of our
parameter-involved diffusion in the downstream (Section 4.2). Finally, we
demonstrate the theoretical guarantees of our model under causal theory
(Section 4.3).

###  4.1 Physical Constraint Pre-trained Model

Within the ST-PAD framework, we employs pre-training technique with physical
laws to guarantee that the upstream outputs are consistent with empirical data
and adhere to established physical principles. This method improves
generalization and prediction in varying or missing data scenarios by
introducing a physics-driven loss function. In essence, it combines data-
driven and physics-driven learning for accurate predictions in complex
environments. Unlike PINNs, which heavily relies on explicit physical
equations, ST-PAD capture physical correlations with only partially known
equations (Showcased later in Section 5.2). Specifically, we utilize a self-
supervised reconstruction mechanism guided by physical constraints to pinpoint
a set of parameter weights that are relatively reasonable. This approach
serves to initialize the parameters within a "better" attraction basin,
positioning them near a local minimum that is conducive to enhanced
generalization [89, 90]. We then reuse the encoder and decoder parameters for
downstream tasks:

Encoder Block. The encoder consists of $L_{e}$ ConvNormReLU blocks, engineered
for efficient spatial signal acquisition. Initiated with
$\mathcal{Z}^{(0)}=\mathcal{X}$ and utilizing an activation function $\sigma$.
Mathematically, the transformation for each stage $i$, where $1\leq i\leq
L_{e}$, is formalized as:

$\begin{gathered}\mathcal{Z}^{(i)}=\sigma\left(\operatorname{GroupNorm}\left(\operatorname{ReLU}\left(\operatorname{Conv2d}\left(\mathcal{Z}^{(i-1)}\right)\right)\right)\right),\\\
\end{gathered}$ (3)

where $\sigma$ represents the activation function, and the
$\operatorname{Conv2d}$, $\operatorname{ReLU}$, and $\operatorname{GroupNorm}$
operations denote 2-dimensional convolution, rectified linear unit activation,
and group normalization, respectively, and $\mathcal{Z}^{(i-1)}$ and
$\mathcal{Z}^{(i)}$ denote the input and output of the $i$-th block with the
shapes $(t,c,h,w)$ and $(t,\hat{c},\hat{h},\hat{w})$, respectively.

Vector Quantization Discrete Encoding Module. To optimize the embedding space
and reduce computational costs while maintaining the accuracy of encoder
output features, we use a vector quantization (VQ) discretization module.
Noting that the output of encoder is $\mathcal{Z}^{e}$, then, it shape
transforms from the form $(t,\hat{c},\hat{h},\hat{w})$ to $(L,D)$, where
$L=h\times w$ and $D=t\times c$. In this process, each embedding vector
$\bm{z}$ from $\mathcal{Z}^{e}$ is meticulously mapped to the closest point in
the memory bank, optimizing for precision in the embedding space. The memory
bank is empirically structured to contain $K$ embeddings, denoted as
$\bm{\mathcal{M}}^{(K_{e})}$ = $\\{\bm{m}_{1},\cdots,\bm{m}_{K}\\}$.
Consequently, we establish a Vector Quantization mapping, $VQ$, to facilitate
this process:

$VQ(\bm{z})=\bm{m}_{\hat{k}},\quad\text{ where
}\quad\hat{k}=\operatorname{argmin}_{k}\left\|\bm{z}(\bm{x})-\bm{m}_{k}\right\|_{2},$
(4)

where each embedding $\bm{m}$ is concatenated to generate matrix
$\bar{\mathcal{Z}}=VQ(\mathcal{Z}^{e})$. The mapping connects continuous
vectors with given vectors in the memory bank to save the computational cost.
Finally, the dimensions are reduced to $(t,\hat{c},\hat{h},\hat{w})$ as inputs
to the dynamical evolution module.

Dynamical Evolution Block. To capture the dynamic evolution of data and better
understand data flows through complex transformations, we use dynamic
evolution blocks. The dynamical evolution block consists of down-sampling,
spectral domain processing and up-sampling three parts:

$\begin{gathered}\mathcal{Z}^{(i)}=\sigma\left(\phi\left(\text{Downsampling}\left(\mathcal{Y}^{(i-1)},\theta\right)\right)\right),\\\
\text{where};\mathcal{Y}^{(i-1)}=\int_{-\infty}^{+\infty}\mathcal{Z}^{(i-1)}(x),dx,;1\leq
i\leq L_{t}\end{gathered}$ (5)

$\begin{gathered}\mathcal{Z}^{(i)}=\sigma\left(\text{Fourier}\left(\text{Transform}\left(\mathcal{Z}^{(L_{t})},\theta^{\prime}\right),\theta\right)\right)\end{gathered}$
(6)

$\begin{gathered}\mathcal{Z}^{(i)}=\sigma\left(\text{Upsampling}\left(\Psi\left(\mathcal{Z}^{(i-1)},\text{skip}^{(i)},\lambda\right),\theta\right)\right),\\\
\text{where};\Psi\left(a,b,\lambda\right)=\lambda a+(1-\lambda)b,;L_{t}\geq
i>1\end{gathered}$ (7)

where the feature representation $\mathcal{Z}^{(i)}$ at each layer is obtained
through a series of transformations applied to the features of the previous
layer. These transformations include applying activation functions $\sigma$
and $\phi$ to increase the model’s nonlinear capabilities. The transformation
operation (Transform) processes data based on parameters $\theta^{\prime}$,
while a specific fusion function $\Psi$ combines the output of the previous
layer with information from skip connections, adjusting their contributions
through parameter $\lambda$. The merging or fusion operation ($\oplus$)
combines information from different sources to enrich the model’s expressive
capacity. Additionally, the introduced intermediate variables
$\mathcal{Y}^{(i-1)}$ and $\mathcal{X}$ represent the transformed feature
representations. Finally, $\theta$ and $\theta^{\prime}$ are sets of
parameters in different operations, adjusting the specific behaviors of their
respective transformations.

Decoder Block. To reconstruct the final output from features after dynamic
evolution and restore the original form of the data, we use decoder blocks,
our decoder contains $L_{d}$ unConvNormReLU blocks to output the final
reconstructions $\mathcal{V}_{rec}=\mathcal{Z}^{(L_{e}+L_{t}+L_{d})}$, where
$L_{e}+L_{t}+1\leq i\leq L_{e}+L_{t}+L_{d}$, In formulation, we can obtain:

$\displaystyle\mathcal{Z}^{(i)}=$
$\displaystyle\sigma\left(\operatorname{GroupNorm}\left(\operatorname{ReLU}\left(\operatorname{unConv2d}\left(\mathcal{Z}^{(i-1)}\right)\right)\right)\right),$
(8)

where
$\mathcal{Z}_{rec}=\bm{\mathcal{Z}}_{t+1:t+t^{\prime}}=\left\\{\mathcal{Z}_{t+1},\cdots,\mathcal{Z}_{t+t}^{\prime}\right\\}$,
and the shape is ($t,c,h,w$).

Physical Constraint Loss Function.To enhance the realism of model outputs and
adherence to basic physical principles, we introduce a new method markedly
different from traditional PINN. Unlike PINNs that rely on fully known
physical equations, our approach works with partially known physical equations
and emphasizes the application of Partial Differential Equations (PDE) by
integrating physical constraint losses. This flexibility allows our model to
make accurate predictions without complete understanding of the underlying
physical processes, thus ensuring predictions dynamically conform to
fundamental physical laws while broadening the application scope. The specific
form of the loss function is:

$L=\sum_{t,x,y}\left(\frac{\partial u}{\partial t}+u\cdot\nabla
u-\alpha\nabla^{2}u\right)^{2},$ (9)

where $u$ represents the velocity field and $\alpha$ is a model parameter that
represents the fluid’s viscosity coefficient. This loss function constrains
not only the temporal derivative of the predicted velocity field
$\frac{\partial u}{\partial t}$ but also includes constraints on the
convection term $u\cdot\nabla u$ and the diffusion term $\alpha\nabla^{2}u$.
By minimizing this loss function, we can guide the model to learn dynamic
fields that evolve within the framework of physical laws.

###  4.2 Parameter-involved Diffusion for OOD Generation

In downstream tasks, we reuse the encoder and decoder parameters trained in
the upstream stage. This strategy significantly saves resources needed for
training and greatly improves the model’s learning efficiency on specific
downstream tasks. Thanks to this parameter reuse, the model can transfer
knowledge more efficiently between different tasks. This section introduces a
novel parameter participation diffusion model design within the ST-PAD
framework. It aims to enhance the model’s generalization ability in different
environmental settings, especially in OOD scenarios. Through an environment
encoder and time embedding mechanism, the model captures environmental
conditions (such as temperature and concentration) affecting fluid behavior
and the dynamics of fluid changes over time. Moreover, the Diffusion Dynamic
System Decoder (DDSD) predicts future dynamics of fluid states by combining
environmental features and time-embedded data. The model trains by minimizing
a comprehensive loss function to ensure physical consistency in predicted
states and adherence to fundamental fluid dynamics laws, achieving accurate
and reliable fluid dynamic simulation and prediction in complex environmental
scenarios. Finally, we detail the component specifics in turn.

Environmental Encoder and Time Embedding. The environmental encoder extracts
environmental feature vectors $\mathcal{E}$ from the input observation data
sequence $\mathcal{X}$, capturing the environmental conditions that affect
fluid behavior, such as temperature and concentration, The time embedding
module encodes time information $\tau$ into a feature vector $\mathcal{T}$, to
capture the dynamics of fluid changes over time. The mathematical definition
is shown below:

$\mathcal{E}=\text{EnvEncoder}(\mathcal{X}),\mathcal{T}=\text{TimeEmbedding}(\tau),$
(10)

where $\mathcal{E}$ represents the environmental features encoded from the
observation data $\mathcal{X}$. Diffusion Dynamic System Decoder (DDSD). The
DDSD predicts future fluid states’ dynamics by combining environmental
features $\mathcal{E}$ and time embeddings $\mathcal{T}$. This module
simulates a conditional reverse Markov chain to denoise and restore fluid
states:

$\mathcal{Z}_{\theta}^{(t)}=f{\theta}\left(\mathcal{Z}^{(t+1)},\mathcal{E},\mathcal{T}_{t}\right)+\epsilon^{(t)}\sqrt{1-\beta_{t}},\quad
t=T,\ldots,1,$ (11)

where $f_{\theta}$ is a parameterized neural network function,
$\epsilon^{(t)}$ is the noise term, and $\beta_{t}$ is the noise scheduling
parameter at time step $t$. The training objective is achieved by minimizing
the following loss function:

$\displaystyle\mathcal{L}_{\text{DDSD}}(\theta)$
$\displaystyle=\mathbb{E}_{q(\mathcal{Z}_{0:T}|\mathcal{X})}\Bigg{[}\log
p_{\theta}(\mathcal{V}|\mathcal{Z}_{0})$ (12)
$\displaystyle\quad-\sum_{t=1}^{T}\text{KL}\big{(}q(\mathcal{Z}_{t-1}|\mathcal{Z}_{t},\mathcal{X})\,||\,p_{\theta}(\mathcal{Z}_{t-1}|\mathcal{Z}_{t})\big{)}\Bigg{]},$

To reinforce physical consistency and ensure the predicted states follow the
laws of fluid dynamics, a physical law regularization term
$\mathcal{L}_{\text{phys}}$ is further integrated:

$\mathcal{L}{\text{phys}}(\mathcal{Z},\mathcal{P})=\left|\nabla\cdot(\mathcal{Z}\otimes\mathcal{P})-\nabla\cdot(\mathcal{Z}{\text{pred}}\otimes\mathcal{P}_{\text{pred}})\right|_{2}^{2},$
(13)

Here, $\mathcal{Z}$ and $\mathcal{Z}_{\text{pred}}$ represent the velocity
fields of the real and predicted states, respectively, while $\mathcal{P}$ and
$\mathcal{P}_{\text{pred}}$ correspond to their respective pressure fields.

###  4.3 Theoretical Guarantee

In this section, we examine and assess the rationality of the model from a
causal perspective. Intuitively, the downstream diffusion process can be
viewed as an introduction of environmental variables into the diffusion
process to enhance the model’s generalizability. Specifically, our framework
can be interpreted as the process of front-door adjustment [79, 91, 92, 93],
as illustrated in the right half of Fig 3:

Figure 3: Diagram illustrating the front-door adjustment strategy in causal
inference, with $\mathcal{X}$ representing the input,
$\tilde{\mathcal{X}}_{\mathcal{C}}^{*}$, ${\mathcal{X}}_{\mathcal{C}}$ act as
the surrogate variable of ${{\mathcal{X}}_{\mathcal{C}}}$ and the causal part
of $\mathcal{X}$, ${\mathcal{D}}$ denoting as a confounder.

  * •

$\mathcal{D}$ operates as a confounder, establishing an illusory correlation
between $\mathcal{X}_{\mathcal{C}}$ and $\mathcal{Y}$. Here,
$\mathcal{X}_{\mathcal{C}}$ signifies the causal component incorporated within
$\mathcal{X}$.

  * •

In the causal chain, $\tilde{\mathcal{X}}_{\mathcal{C}}^{*}$ functions as the
surrogate for the causal variable $\mathcal{X}_{\mathcal{C}}$, enhancing it to
fit the data distribution. Initially deriving from and encompassing
$\mathcal{X}_{\mathcal{C}}$, it represents the complete observations that
should be apparent when observing the subpart $\mathcal{X}_{\mathcal{C}}$.
Additionally, $\mathcal{X}_{\mathcal{C}}^{*}$ complies with the data
distribution and maintains the essential knowledge of graph properties,
thereby disconnecting any association between $\mathcal{D}$ and
$\tilde{\mathcal{X}}_{\mathcal{C}}^{*}$. Hence,
$\tilde{\mathcal{X}}_{\mathcal{C}}^{*}$ aptly serves as the intermediary,
which in turn influences the model’s predictions for $\mathcal{Y}$.

In our front-door adjustment framework, we apply do-calculus to the variable
${\mathcal{X}}_{\mathcal{C}}$ to counteract the spurious correlations prompted
by ${\mathcal{D}}\to{\mathcal{Y}}$. This elimination is effected by summing
over the potential surrogate observations $\tilde{X}_{\mathcal{C}}^{*}$. This
strategy enables the connection of two distinguishable partial effects:
${\mathcal{X}}_{\mathcal{C}}\to\tilde{\mathcal{X}}_{\mathcal{C}}^{*}$ and
$\tilde{\mathcal{X}}_{\mathcal{C}}^{*}\to{\mathcal{Y}}$:

The process with
${\mathcal{X}}_{\mathcal{C}}\to\tilde{\mathcal{X}}_{\mathcal{C}}^{*}$ employs
a parameter-involved diffusion approach to incorporate potential parameter
settings. This methodology can be construed as a front-door adjustment
designed to keenly capture and generate feasible scenario graphs based on
physical parameter settings. By implementing the aforementioned strategy, we
substantially augment the diffusion process’s responsiveness to environmental
variations, thus bolstering the model’s generalization capacity.

###  4.4 Model Summary

In summary, the ST-PAD framework employs a physical constraint pre-trained
model and a physics-driven loss function to ensure model outputs align with
empirical data and adhere to physical principles, thus enhancing
generalization and prediction accuracy. Unlike methods fully relying on
explicit physical equations, we use self-supervised learning and partially
known equations for modeling physical constraints. Moreover, the environmental
encoder and time embedding mechanisms improve the model’s generalization
ability in various environmental settings, especially in OOD scenarios. The
DDSD combines environmental features and time information to predict fluid
dynamics, ensuring physical consistency in predictions through minimizing a
comprehensive loss function, achieving accurate fluid dynamics simulations in
complex environments. We summarize this as Algorithm 1.

Algorithm 1 ST-PAD Framework Process

1:Input: Spatio-temporal sequence $\mathcal{X}$

2:Output: Predicted dynamic fields $\mathcal{V}_{rec}$ $\triangleright$
Upstream Parameter Locking

3:Initialize Encoder with ConvNormReLU blocks

4:for $i=1$ to $L_{e}$ do

5: $\mathcal{Z}^{(i)}\leftarrow$ Apply ConvNormReLU block on
$\mathcal{Z}^{(i-1)}$

6:end for

7:$\mathcal{Z}^{e}\leftarrow$ Encoder’s final layer output

8:Perform Vector Quantization on $\mathcal{Z}^{e}$

9:Initialize Dynamical Evolution Block

10:for $i=1$ to $L_{t}$ do

11: Apply down-sampling, Fourier transform, up-sampling

12:end for

13:Initialize Decoder

14:for $i=L_{e}+L_{t}+1$ to $L_{e}+L_{t}+L_{d}$ do

15: $\mathcal{Z}^{(i)}\leftarrow$ Apply unConvNormReLU block on
$\mathcal{Z}^{(i-1)}$

16:end for

17:Compute Physical Constraint Loss $L$ $\triangleright$ Parameter-involved
Diffusion

18:Extract environmental features $\mathcal{E}$ using EnvEncoder

19:Encode time information $\mathcal{T}$ using TimeEmbedding

20:Initialize DDSD

21:for $t=T$ to $1$ do

22: Predict $\mathcal{Z}_{\theta}^{(t)}$ using DDSD

23:end for

24:Compute DDSD loss $\mathcal{L}_{\text{DDSD}}$

25:Compute Physical Law Regularization term $\mathcal{L}_{\text{phys}}$

26:return $\mathcal{Z}_{pred}$

##  5 Experiments

In this section, we showcase empirical evidence validating the efficacy of
fluid methodologies, ST-PAD. The experimental investigations are designed to
address the subsequent research inquiries:

  * •

RQ1: How does the performance of the ST-PAD model compare to that of the
current SOTA models?

  * •

RQ2: Is the ST-PAD model capable of achieving precise predictions
structurally, as measured by metrics such as PSNR and SSIM?

  * •

RQ3: Can the design of our various components significantly enhance the
capabilities of the ST-PAD model?

  * •

RQ4: Is the ST-PAD model capable of effectively ensuring the local fidelity of
its predictions?

###  5.1 Experimental Settings

Dataset. We split the dataset into video, simulated and real-world datasets.
The specific details are shown below. Video Data: We chose KTH [94] and
TaxiBJ+ [13] as video datasets. Simulated Data: We select representative
Partial Differential Equation (PDE) datasets showcasing turbulence phenomena,
including the Navier-Stokes (NS) equations [1] and the Shallow-Water (SW)
equations [95]. Additionally, we choose iconic Computational Fluid Dynamics
(CFD) datasets [96], covering combustion dynamics (CD) datasets simulated with
FDS and cylinder flow datasets. Real-world data: We select the ERA5 [97] and
Weatherbench [98] datasets to assess the model’s capability in weather
forecasting. Additionally, we choose the SEVIR [99] dataset to analyze the
model’s ability to predict extreme events. Table II presents the detailed
statistical information of the dataset.

TABLE II: Dataset statistics. $N\\_tr$ and $N\\_te$ denote the number of instances in the training and test sets. The lengths of the input and prediction sequences are $I_{l}$ and $O_{l}$, respectively. Dataset | $N\\_tr$ | $N\\_te$ | ($C,H,W$) | $I_{l}$ | $O_{l}$
---|---|---|---|---|---
TaxiBJ+ | 3555 | 445 | (2, 128, 128) | 12 | 12
KTH | 108717 | 4086 | (1, 128, 128) | 10 | 20
NS | 9000 | 1000 | (1, 64, 64) | 10 | 10
SW | 3555 | 445 | (2, 128, 128) | 12 | 12
CD | 8000 | 1000 | (3, 128, 256) | 10 | 10
SEVIR | 4158 | 500 | (1, 384, 384) | 10 | 10
ERA5 | 6000 | 1500 | (1, 208, 333) | 10 | 10
WeatherBench | 2000 | 500 | (3, 128, 128) | 6 | 6

Baselines. To comprehensively assess the effectiveness of our proposed method,
we select leading models in the fields of Physics-Informed Neural Network,
Spatio-Temporal Forecasting and Neural Operators for comparison. For Physics-
Informed Neural Network, we choose SOTA models as PINN [33], MAD [26]. For
Spatio-Temporal Forecasting, we choose models such as ConvLSTM [100], SimVP
[101], Earthformer [102], TAU [29], PastNet [30], and Earthfarseer [13]. In
the field of Neural Operators, we evaluate architectures like FNO [1], LSM
[3], UNO [103], and F-FNO [104]. The details are as follows:

  * •

PINN is a deep learning model that embeds physical laws, such as conservation
laws and dynamic equations, into the training process of neural networks to
enhance their simulation and prediction capabilities for complex physical
processes.

  * •

MAD is a novel approach for solving parametric partial differential equations
(PDEs) that leverages meta-learning to efficiently adapt to various
parameters, significantly enhancing solution accuracy and computational
efficiency.

  * •

ConvLSTM is a neural network model for spatiotemporal sequence prediction,
effectively capturing spatial and temporal dependencies in data by combining
features of Convolutional Neural Networks (CNN) and Long Short-Term Memory
networks (LSTM).

  * •

SimVP is a model designed for video prediction, effectively enhancing
prediction accuracy and efficiency by simplifying the structured
representation of the video prediction.

  * •

Earthformer is an exploratory approach that uses space-time Transformer models
for Earth system forecasting, predicting dynamic changes in the Earth system
by processing large-scale spatial-temporal data.

  * •

TAU is a model designed for video prediction, effectively enhancing prediction
accuracy and efficiency by simplifying the structured representation of the
video prediction.

  * •

PastNet is a spatiotemporal prediction model that integrates physical prior
knowledge, designed to analyze and forecast spatiotemporal patterns in
historical data for predicting future states.

  * •

Earthfarseer is specifically designed for spatiotemporal predictions in the
field of Earth sciences, capable of processing large-scale geospatial data and
predicting environmental and climate changes.

  * •

FNO is an operator learning framework that uses Fourier transforms to capture
complex mappings between functions, aimed at solving partial differential
equations and other continuous domain problems.

  * •

LSM is a method for learning and approximating spectral mappings, designed to
enhance the efficiency and accuracy of learning operators by capturing the
intrinsic spectral relationships between input and output data.

  * •

UNO is an operator learning model designed to generalize solutions for various
partial differential equations, enhancing the model’s generalization
capability by learning representations of underlying physical rules.

  * •

F-FNO is an improved version of FNO that enhances computational efficiency
through factorization techniques, aimed at efficiently and accurately solving
complex continuous domain problems.

TABLE III: Model comparison with the state-of-the-arts over different evaluation metrics. We report the mean results from five runs. TaxiBJ+ metrics take MAE, all others are MSE metrics. Backbone | Video Data | Simulated Data | Real-world data | Ranking
---|---|---|---|---
TaxiBJ+ (MAE) | KTH (MSE) | SW (MSE) | NS (MSE) | CD (MSE) | WeatherBench (MSE) | ERA5 (MSE) | SEVIR (MSE)
Physics-Informed Neural Network |
PINN [33] | - | - | 0.6542 | 0.6857 | 2.8763 | - | - | - | 13.00
MAD [26] | - | - | 0.5834 | 0.4467 | 2.6532 | - | - | - | 11.68
Spatio-temporal Predictive Models |
ConvLSTM [100] | 5.5432 | 126.2322 | 0.3731 | 0.3334 | 0.9742 | 0.2983 | 4.1351 | 3.8124 | 7.88
SimVP [101] | 3.0212 | 40.9432 | 0.2121 | 0.1845 | 0.1932 | 0.1271 | 2.9371 | 3.4134 | 6.87
Earthformer [102] | 3.1223 | 48.2643 | 0.1932 | 0.6543 | 0.2124 | 0.2124 | 3.5332 | 3.7159 | 8.63
TAU [29] | 2.9812 | 39.4312 | 0.2023 | 0.1723 | 0.1873 | 0.1345 | 2.9835 | 3.3397 | 6.50
PastNet [30] | 2.7143 | 33.8433 | 0.2011 | 0.2144 | 0.1743 | 0.1312 | 3.0941 | 3.2322 | 6.50
Earthfarseer [13] | 2.1109 | 31.8233 | 0.1921 | 0.1732 | 0.1384 | 0.098 | 2.8943 | 2.8314 | 4.37
Neural Operator Models |
FNO [1] | 5.4312 | 175.4432 | 0.1233 | 0.1546 | 1.5432 | 1.2432 | 2.8534 | 4.4312 | 8.25
LSM [3] | 6.7532 | 174.3231 | 0.1543 | 0.1677 | 1.6873 | 1.0982 | 2.9763 | 5.4983 | 9.38
UNO [103] | 3.2143 | 134.2131 | 0.1235 | 0.2012 | 1.4323 | 1.3321 | 2.5842 | 3.0941 | 7.62
F-FNO [104] | 5.3222 | 112.4312 | 0.1437 | 0.1924 | 1.8521 | 0.9874 | 8.9853 | 3.9984 | 9.38
ST-PAD | 2.0729 | 29.9784 | 0.1129 | 0.1372 | 0.1231 | 0.0875 | 1.9872 | 2.7562 | 2.25
Figure 4: Visualization on TaxiBJ+. For simplicity, we display the results of
the last 10 frames.

Evaluation Metrics. We adopt several metrics to assess the quality of our
predictions, and each has its unique significance:

  * •

Mean Squared Error (MSE): This metric provides the average of the squares of
the differences between the actual and predicted values. A lower MSE indicates
a closer fit of the predictions to the true values. It’s given by the
equation:

$\text{MSE}=\frac{1}{N}\sum_{i=1}^{N}(V_{\text{true},i}-V_{\text{fut},i})^{2}$
(14)

where $V_{\text{true},i}$ represents the true value, $V_{\text{fut},i}$
denotes the predicted value, and $N$ is the number of observations.

  * •

Mean Absolute Error (MAE): MAE provides the average of the absolute
differences between the predicted and actual values. This metric is less
sensitive to outliers than MSE, making it suitable for applications where
extreme errors should not disproportionately affect the overall error measure.

$\text{MAE}=\frac{1}{N}\sum_{i=1}^{N}|V_{\text{true},i}-V_{\text{fut},i}|$
(15)

  * •

Multi-Scale Structural Similarity (MS-SSIM): SSIM is designed to provide an
assessment of the structural integrity and similarity between two images, $x$
and $y$. Higher SSIM values suggest that the structures of the two images
being compared are more similar.

$\text{SSIM}(x,y)=\frac{(2\mu_{x}\mu_{y}+c_{1})(2\sigma_{xy}+c_{2})}{(\mu_{x}^{2}+\mu_{y}^{2}+c_{1})(\sigma_{x}^{2}+\sigma_{y}^{2}+c_{2})}$
(16)

where $\mu$ is the mean, $\sigma$ represents variance, and $c_{1}$ and $c_{2}$
are constants to avoid instability.

  * •

Peak Signal-to-Noise Ratio (PSNR): PSNR gauges the quality of a reconstructed
image compared to its original by measuring the ratio between the maximum
possible power of the signal and the power of corrupting noise. A higher PSNR
indicates a better reconstruction quality.

$\text{PSNR}=10\times\log_{10}\left(\frac{\text{MAX}_{I}^{2}}{\text{MSE}}\right)$
(17)

where $\text{MAX}_{I}$ is the maximum possible pixel value of the image.

Implementation details. In our experiment setup, we use the PyTorch framework
and a 40GB A100 GPU for model training. We set the training period to 500
epochs, with an initial learning rate of 0.001. To control the training
process more finely, we introduce a learning rate scheduler. It adjusts the
learning rate by a decay rate of 0.5 every 100 epochs, adapting to different
stages of model training. Also, we set our batch size to 10. On this basis,
all our prediction models use an autoregressive prediction architecture.

###  5.2 Main results

Figure 5: The figure shows input frames from 8 to 24 hours, along with their
actual conditions and future frame predictions. The left side compares the
performance of three models: ST-PAD, Earthforeseer, and SimVP. Red boxes
highlight the local details that both models and the actual conditions
successfully predicted. The bottom displays the error distribution of
predictions made by the ST-PAD and TAU models.

We first focus on verifying the effectiveness of the ST-PAD. To systematically
assess its capabilities, we select multiple datasets, and present the results
in Table III, Figure 4 and Figure 5. From these results, we can list
observations:

Obs.❶ The ST-PAD consistently outperforms the current mainstream models,
including PINNs, spatio-temporal forecasting models, and neural operator
models. As demonstrated in Tab III, ST-PAD maintains robust performance under
the metrics MAE and MSE, achieving the top ranking across all frameworks.
Specifically, on the video data set TaxiBJ+, ST-PAD attains a lower MAE
(1.0494) than the best model, Earthfarseer. On the CFD simulation data set, it
achieves an MSE reduction of 0.0153 compared to the best-performing
Earthfarseer. Moreover, on the ERA5 weather data, it secures an MSE advantage
of nearly 0.5970 over the SOTA model, UNO. These results validate the
effectiveness of ST-PAD as a purely data-driven approach in fluid modeling.

Obs.❷ ST-PAD showcases stronger robustness and versatility than PINNs. Since
PINNs require knowledge of explicit and complete PDEs, in our experiments with
real-world and video data, we are unable to determine the specific evolution
parameters, thus preventing computation within PINNs. However, the design of
ST-PAD only necessitates knowledge of general or partial physical equations at
the upstream level. This underscores the robustness and versatility of ST-PAD
(In our settings, we use conservation equations for KTH and TaxiBJ+, Wave
Equation for SW, Navier-Stokes for NS and real-world meteorological datasets,
Reaction-Convection-Diffusion Equation for CD benchmark.).

Obs.❸ ST-PAD can effectively achieve higher prediction fidelity. From Fig 4
and 5, it is clear that ST-PAD excels at capturing predictive details.
Compared to traditional video prediction frameworks, such as PastNet and
others, ST-PAD is adept at preserving the details in its predictions.

Figure 6: The figure shows the performance of the ST-PAD in simulating
combustion dynamics data. The top three sets of images compare the true values
(Truth) and predicted values (Pred) of U component, V component, and
temperature over 15 time steps. The middle two graphs depict the decline in
the loss function during training and validation across epochs. The bottom
three graphs present the comparison between the true and predicted values of U
component, V component, and temperature over time. Figure 7: The top shows
real and ST-PAD model predictions for 10 to 60-minute intervals, next to
Earthformer model predictions. A red box highlights extreme events captured by
both models and reality. Below are real temperature data and TAU and ST-PAD
model predictions. An enlargement in the right-side white box shows details
and SSIM scores, showing ST-PAD predictions closer to reality.

###  5.3 Structured prediction results (RQ2)

In this section, we further test whether ST-PAD can achieve more accurate
structured prediction results compared to mainstream models. We employ PSNR
and SSIM as prediction metrics and select MAD, Earthfarseer, FNO, and ST-PAD
for comparison. We adopt two prediction approaches: autoregressive and
parallel [105]. The autoregressive approach refers to recursive prediction,
while parallel denotes simultaneous prediction. Employing these methods allows
for a more comprehensive assessment of model stability across various
scenarios. As shown in Tab IV, we can easily list the observations:

Obs.❹ ST-PAD outperforms both PINNs and data-driven models on the TaxiBJ+, CD,
and Weatherbench datasets. Notably, it achieves a PSNR increase of $1.33\sim
5.25$ on the TaxiBJ+ dataset and a PSNR improvement of $2.54\sim 3.04$ on the
CD dataset. More importantly, in terms of the SSIM metric, ST-PAD perfectly
restores the predictive effect, achieving excellent performance with scores
over 0.95 on all three datasets. These results repeatedly validate ST-PAD’s
capability for structural prediction restoration and demonstrate its
robustness.

TABLE IV: The PSNR and SSIM across different benchmarks and models. | Models | PSNR/SSIM
---|---|---
| Autoregressive | Parallel
| TaxiBJ+
---
| MAD
---
Earthfarseer
FNO
ST-PAD
| $23.58/0.68$
---
$38.94/0.96$
$28.33/0.72$
$40.27/0.98$
| $21.32/0.67$
---
$34.94/0.93$
$24.46/0.69$
$40.19/0.98$
CD |  | MAD
---
Earthfarseer
FNO
ST-PAD
| $26.87/0.77$
---
$40.21/0.92$
$41.83/0.95$
$44.37/0.99$
| $26.37/0.76$
---
$40.14/0.91$
$40.42/0.94$
$43.46/0.98$
| Weatherbench
---
| MAD
---
Earthfarseer
FNO
ST-PAD
| $17.34/0.54$
---
$37.43/0.93$
$18.24/0.55$
$38.34/0.95$
| $16.34/0.51$
---
$33.44/0.91$
$19.23/0.57$
$38.47/0.96$

###  5.4 Ablation Study (RQ3)

We conduct extensive ablation experiments to demonstrate the performance of
the key components in ST-PAD, therefore, we propose the following model
variants: (1) remove the upstream physical equation loss function (ST-PAD w/o
Physical); (2) remove the evolution module (ST-PAD w/o Evolution); (3) remove
the downstream parameter module (ST-PAD w/o Parameter). As shown in Table V,
The experimental results show that the complete ST-PAD model outperforms its
variants on two benchmarks. Specifically, the full model reduces MAE to 2.0729
and increases SSIM to 0.9821 on the TaxiBJ+ benchmark, lowers MSE to 0.0875,
and increases SSIM to 0.9643 on the Weatherbench benchmark. These findings
highlight the crucial role of the physical equation loss function, evolution
module, and parameter module in enhancing model performance.

TABLE V: Ablation Studies on two dataset.
Variants | Benchmarks
---|---
TaxiBJ+ | Weatherbench
MAE | SSIM | MSE | SSIM
ST-PAD w/o Physical | 2.3214 | 0.9521 | 0.0983 | 0.9312
ST-PAD w/o Evolution | 2.2313 | 0.9623 | 0.0882 | 0.9533
ST-PAD w/o Parameter | 2.4312 | 0.9321 | 0.0893 | 0.9442
ST-PAD | 2.0729 | 0.9821 | 0.0875 | 0.9643

###  5.5 Local fidelity (RQ4)

In the final subsection, we examine the local fidelity capabilities of ST-PAD,
as local fidelity is crucial for fluid dynamics modeling. In Figure 6, we
observe the performance of the ST-PAD model in predicting the U and V
components as well as temperature values compared to the true values over time
steps $1\sim 15$. We notice that the predicted curves of the U component
closely align with the true U component for most time steps, despite some
deviations at certain points, indicating that ST-PAD performs well in
simulating the overall dynamics of the U component. Figure 7 highlights ST-
PAD’s exceptional performance in terms of local fidelity. Specifically, in
predicting local extreme events, the ST-PAD model demonstrates superior
capturing ability. For instance, at the 60-minute mark, the area of extreme
events predicted by ST-PAD nearly perfectly matches the ground truth, while
another compared model (Earthformer) underpredicts in the same area. In terms
of TAU predictions, the comparison of temperature predictions by both models
with the true data (Truth) shows that ST-PAD also excels in capturing local
temperature peaks, with the SSIM score improving from 0.63-0.94. This
indicates ST-PAD’s significant advantage in preserving local details.

##  6 Conclusion & Future work

In this section, we develop a sophisticated two-stage framework named ST-PAD,
which seamlessly integrates spatio-temporal physical-awareness and parameter
diffusion guidance. Through our meticulous design, we introduce a Vector
Quantized reconstruction module with time-evolution characteristics in the
upstream phase, ensuring a balanced parameter distribution via universal
physical constraint. In the downstream phase, we leverage a parameter-involved
diffusion probabilistic network to generate high-quality image content while
achieving parametric awareness of multiple physical settings. We conduct
extensive experiments across multiple benchmarks in various realms, and the
experimental results repeatedly demonstrate the effectiveness and robustness
of our algorithm.

Modeling fluid dynamics showcases great importance for Earth sciences. In this
paper, we systematically consider diffusion with the introduction of
parameters to enhance the generalization capability of fluid modeling. In the
future, we plan to further incorporate generalization theories, such as
invariant learning [106, 107, 108, 109, 110, 111], Mixture-of-experts [112,
113], and adapter techniques [114, 115], to improve generalization
capabilities.

##  7 Acknowledgment

This paper is partially supported by the National Natural Science Foundation
of China (No.62072427, No.12227901), the Project of Stable Support for Youth
Team in Basic Research Field, CAS (No.YSBR-005), Academic Leaders Cultivation
Program, USTC.

## References

  * [1] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, “Fourier neural operator for parametric partial differential equations,” _arXiv preprint arXiv:2010.08895_ , 2020.
  * [2] L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis, “Learning nonlinear operators via deeponet based on the universal approximation theorem of operators,” _Nature machine intelligence_ , vol. 3, no. 3, pp. 218–229, 2021.
  * [3] H. Wu, T. Hu, H. Luo, J. Wang, and M. Long, “Solving high-dimensional pdes with latent spectral models,” _arXiv preprint arXiv:2301.12664_ , 2023.
  * [4] S. Candel, “Combustion dynamics and control: Progress and challenges,” _Proceedings of the combustion institute_ , vol. 29, no. 1, pp. 1–28, 2002.
  * [5] C. Külsheimer and H. Büchner, “Combustion dynamics of turbulent swirling flames,” _Combustion and flame_ , vol. 131, no. 1-2, pp. 70–84, 2002.
  * [6] W. Xiong, Y. Xiang, H. Wu, S. Zhou, Y. Sun, M. Ma, and X. Huang, “Ai-goms: Large ai-driven global ocean modeling system,” _arXiv preprint arXiv:2308.03152_ , 2023.
  * [7] X. Wang, R. Wang, N. Hu, P. Wang, P. Huo, G. Wang, H. Wang, S. Wang, J. Zhu, J. Xu _et al._ , “Xihe: A data-driven model for global ocean eddy-resolving forecasting,” _arXiv preprint arXiv:2402.02995_ , 2024.
  * [8] M. Xu, J. Han, A. Lou, J. Kossaifi, A. Ramanathan, K. Azizzadenesheli, J. Leskovec, S. Ermon, and A. Anandkumar, “Equivariant graph neural operator for modeling 3d dynamics,” _arXiv preprint arXiv:2401.11037_ , 2024.
  * [9] S. A. Hollingsworth and R. O. Dror, “Molecular dynamics simulation for all,” _Neuron_ , vol. 99, no. 6, pp. 1129–1143, 2018.
  * [10] T. Pfaff, M. Fortunato, A. Sanchez-Gonzalez, and P. W. Battaglia, “Learning mesh-based simulation with graph networks,” 2021.
  * [11] M. Lienen and S. Günnemann, “Learning the dynamics of physical systems from sparse observations with finite element networks,” 2022.
  * [12] Y. Shao, C. C. Loy, and B. Dai, “Transformer with implicit edges for particle-based physics simulation,” 2022, pp. 549–564.
  * [13] H. Wu, S. Wang, Y. Liang, Z. Zhou, W. Huang, W. Xiong, and K. Wang, “Earthfarseer: Versatile spatio-temporal dynamical systems modeling in one model,” _arXiv preprint arXiv:2312.08403_ , 2023.
  * [14] K. Wang, H. Wu, G. Zhang, J. Fang, Y. Liang, Y. Wu, R. Zimmermann, and Y. Wang, “Modeling spatio-temporal dynamical systems with neural discrete learning and levels-of-experts,” _IEEE Transactions on Knowledge and Data Engineering_ , 2024.
  * [15] Z. Long, Y. Lu, X. Ma, and B. Dong, “Pde-net: Learning pdes from data,” in _International conference on machine learning_. PMLR, 2018, pp. 3208–3216.
  * [16] T. F. Chan, J. Shen, and L. Vese, “Variational pde models in image processing,” _Notices AMS_ , vol. 50, no. 1, pp. 14–26, 2003.
  * [17] P. Constantin and C. Foiaş, _Navier-stokes equations_. University of Chicago press, 1988.
  * [18] J. Rabault, F. Ren, W. Zhang, H. Tang, and H. Xu, “Deep reinforcement learning in fluid mechanics: A promising method for both active flow control and shape optimization,” _Journal of Hydrodynamics_ , vol. 32, pp. 234–246, 2020.
  * [19] K. Carlberg, C. Farhat, J. Cortial, and D. Amsallem, “The gnat method for nonlinear model reduction: effective implementation and application to computational fluid dynamics and turbulent flows,” _Journal of Computational Physics_ , vol. 242, pp. 623–647, 2013.
  * [20] M. Takamoto, F. Alesiani, and M. Niepert, “Cape: Channel-attention-based pde parameter embeddings for sciml,” 2022.
  * [21] S. Fotiadis, M. L. Valencia, S. Hu, S. Garasto, C. D. Cantwell, and A. A. Bharath, “Disentangled generative models for robust prediction of system dynamics,” 2023.
  * [22] S. Janny, M. Nadri, J. Digne, and C. Wolf, “Space and time continuous physics simulation from partial observations,” in _International Conference on Learning Representation_ , 2024.
  * [23] V. Verda, R. Borchiellini, S. Cosentino, E. Guelpa, and J. M. Tuni, “Expanding the fds simulation capabilities to fire tunnel scenarios through a novel multi-scale model,” _Fire Technology_ , vol. 57, no. 5, pp. 2491–2514, 2021.
  * [24] T. Rasinski, “Creating fluent readers,” _Educational leadership_ , vol. 61, no. 6, pp. 46–51, 2004.
  * [25] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, “Physics-informed machine learning,” _Nature Reviews Physics_ , vol. 3, no. 6, pp. 422–440, 2021.
  * [26] X. Huang, Z. Ye, H. Liu, S. Ji, Z. Wang, K. Yang, Y. Li, M. Wang, H. Chu, F. Yu _et al._ , “Meta-auto-decoder for solving parametric partial differential equations,” _Advances in Neural Information Processing Systems_ , vol. 35, pp. 23 426–23 438, 2022.
  * [27] X. Meng, Z. Li, D. Zhang, and G. E. Karniadakis, “Ppinn: Parareal physics-informed neural network for time-dependent pdes,” _Computer Methods in Applied Mechanics and Engineering_ , vol. 370, p. 113250, 2020.
  * [28] P.-H. Chiu, J. C. Wong, C. Ooi, M. H. Dao, and Y.-S. Ong, “Can-pinn: A fast physics-informed neural network based on coupled-automatic–numerical differentiation method,” _Computer Methods in Applied Mechanics and Engineering_ , vol. 395, p. 114909, 2022.
  * [29] C. Tan, Z. Gao, L. Wu, Y. Xu, J. Xia, S. Li, and S. Z. Li, “Temporal attention unit: Towards efficient spatiotemporal predictive learning,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 18 770–18 782.
  * [30] H. Wu, W. Xion, F. Xu, X. Luo, C. Chen, X.-S. Hua, and H. Wang, “Pastnet: Introducing physical inductive biases for spatio-temporal video prediction,” _arXiv preprint arXiv:2305.11421_ , 2023.
  * [31] A. Krishnapriyan, A. Gholami, S. Zhe, R. Kirby, and M. W. Mahoney, “Characterizing possible failure modes in physics-informed neural networks,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 26 548–26 560, 2021.
  * [32] Y. Chen and L. Dal Negro, “Physics-informed neural networks for imaging and parameter retrieval of photonic nanostructures from near-field data,” _APL Photonics_ , vol. 7, no. 1, 2022.
  * [33] M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” _Journal of Computational physics_ , vol. 378, pp. 686–707, 2019.
  * [34] J. Yu, L. Lu, X. Meng, and G. E. Karniadakis, “Gradient-enhanced physics-informed neural networks for forward and inverse pde problems,” _Computer Methods in Applied Mechanics and Engineering_ , vol. 393, p. 114823, 2022.
  * [35] Y. Xu, S. Kohtz, J. Boakye, P. Gardoni, and P. Wang, “Physics-informed machine learning for reliability and systems safety applications: State of the art and challenges,” _Reliability Engineering & System Safety_, vol. 230, p. 108900, 2023.
  * [36] H. Wang, T. Fu, Y. Du, W. Gao, K. Huang, Z. Liu, P. Chandak, S. Liu, P. Van Katwyk, A. Deac _et al._ , “Scientific discovery in the age of artificial intelligence,” _Nature_ , vol. 620, no. 7972, pp. 47–60, 2023.
  * [37] V. Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein, “Implicit neural representations with periodic activation functions,” _Advances in neural information processing systems_ , vol. 33, pp. 7462–7473, 2020.
  * [38] N. Göring, F. Hess, M. Brenner, Z. Monfared, and D. Durstewitz, “Out-of-domain generalization in dynamical systems reconstruction,” _arXiv preprint arXiv:2402.18377_ , 2024.
  * [39] H. Wu, K. Wang, F. Xu, Y. Li, X. Wang, W. Wang, H. Wang, and X. Luo, “Spatio-temporal twins with a cache for modeling long-term system dynamics,” 2024. [Online]. Available: https://openreview.net/forum?id=aE6HazMgRz
  * [40] Z. Li, D. Z. Huang, B. Liu, and A. Anandkumar, “Fourier neural operator with learned deformations for pdes on general geometries,” _Journal of Machine Learning Research_ , vol. 24, no. 388, pp. 1–26, 2023.
  * [41] S. Cao, “Choose a transformer: Fourier or galerkin,” _Advances in neural information processing systems_ , vol. 34, pp. 24 924–24 940, 2021.
  * [42] Y. Yin, I. Ayed, E. de Bézenac, N. Baskiotis, and P. Gallinari, “Leads: Learning dynamical systems that generalize across environments,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 7561–7573, 2021.
  * [43] L. Bereska and E. Gavves, “Continual learning of dynamical systems with competitive federated reservoir computing,” in _Conference on Lifelong Learning Agents_. PMLR, 2022, pp. 335–350.
  * [44] S. Lepri, R. Livi, and A. Politi, “Thermal conduction in classical low-dimensional lattices,” _Physics reports_ , vol. 377, no. 1, pp. 1–80, 2003.
  * [45] W. Gill and R. Sankarasubramanian, “Exact analysis of unsteady convective diffusion,” _Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences_ , vol. 316, no. 1526, pp. 341–350, 1970.
  * [46] A. Van Den Oord, O. Vinyals _et al._ , “Neural discrete representation learning,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [47] A. H. Liu, S. Jin, C.-I. J. Lai, A. Rouditchenko, A. Oliva, and J. Glass, “Cross-modal discrete representation learning,” _arXiv preprint arXiv:2106.05438_ , 2021.
  * [48] V. Fortuin, M. Hüser, F. Locatello, H. Strathmann, and G. Rätsch, “Som-vae: Interpretable discrete representation learning on time series,” _arXiv preprint arXiv:1806.02199_ , 2018.
  * [49] J. Guibas, M. Mardani, Z. Li, A. Tao, A. Anandkumar, and B. Catanzaro, “Adaptive fourier neural operators: Efficient token mixers for transformers,” _arXiv preprint arXiv:2111.13587_ , 2021.
  * [50] S. Bond-Taylor, P. Hessey, H. Sasaki, T. P. Breckon, and C. G. Willcocks, “Unleashing transformers: Parallel token prediction with discrete absorbing diffusion for fast high-resolution image generation from vector-quantized codes,” in _European Conference on Computer Vision_. Springer, 2022, pp. 170–188.
  * [51] Y. Benny and L. Wolf, “Dynamic dual-output diffusion models,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 11 482–11 491.
  * [52] J. Choi, J. Lee, C. Shin, S. Kim, H. Kim, and S. Yoon, “Perception prioritized training of diffusion models,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 11 472–11 481.
  * [53] F.-A. Croitoru, V. Hondru, R. T. Ionescu, and M. Shah, “Diffusion models in vision: A survey,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2023.
  * [54] S. Oprea, P. Martinez-Gonzalez, A. Garcia-Garcia, J. A. Castro-Vargas, S. Orts-Escolano, J. Garcia-Rodriguez, and A. Argyros, “A review on deep learning techniques for video prediction,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 44, no. 6, pp. 2806–2826, 2020.
  * [55] J. Oh, X. Guo, H. Lee, R. L. Lewis, and S. Singh, “Action-conditional video prediction using deep networks in atari games,” in _Advances in Neural Information Processing Systems_ , 2015, pp. 2863–2871.
  * [56] M. Mathieu, C. Couprie, and Y. LeCun, “Deep multi-scale video prediction beyond mean square error,” in _International Conference on Learning Representations_ , 2016.
  * [57] C.-F. R. Chen, R. Panda, K. Ramakrishnan, R. Feris, J. Cohn, A. Oliva, and Q. Fan, “Deep analysis of cnn-based spatio-temporal representations for action recognition,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 6165–6175.
  * [58] S. Tulyakov, M.-Y. Liu, X. Yang, and J. Kautz, “Mocogan: Decomposing motion and content for video generation,” in _Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 1526–1535.
  * [59] N. Srivastava, E. Mansimov, and R. Salakhudinov, “Unsupervised learning of video representations using lstms,” in _International Conference on Machine Learning_ , 2015, pp. 843–852.
  * [60] M. Babaeizadeh, C. Finn, D. Erhan, R. H. Campbell, and S. Levine, “Stochastic variational video prediction,” in _International Conference on Learning Representations_ , 2018.
  * [61] D. Seng, Q. Zhang, X. Zhang, G. Chen, and X. Chen, “Spatiotemporal prediction of air quality based on lstm neural network,” _Alexandria Engineering Journal_ , vol. 60, no. 2, 2021.
  * [62] Y. Wang, H. Wu, J. Zhang, Z. Gao, J. Wang, S. Y. Philip, and M. Long, “Predrnn: A recurrent neural network for spatiotemporal predictive learning,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 45, no. 2, pp. 2208–2225, 2022.
  * [63] D. Weissenborn, O. Täckström, and J. Uszkoreit, “Scaling autoregressive video models,” in _International Conference on Learning Representations_ , 2019.
  * [64] M. Kumar, M. Babaeizadeh, D. Erhan, C. Finn, S. Levine, L. Dinh, and D. Kingma, “Videoflow: A flow-based generative model for video,” in _International Conference on Learning Representations_ , 2020.
  * [65] X. Wang, C. Fleischmann, and M. Spearpoint, “Applying the fds pyrolysis model to predict heat release rate in small-scale forced ventilation tunnel experiments,” _Fire safety journal_ , vol. 112, p. 102946, 2020.
  * [66] A. Baddeley, R. Logie, I. Nimmo-Smith, and N. Brereton, “Components of fluent reading,” _Journal of memory and language_ , vol. 24, no. 1, pp. 119–131, 1985.
  * [67] R. W. Pryor, _Multiphysics modeling using COMSOL®: a first principles approach_. Jones & Bartlett Publishers, 2009.
  * [68] D. W. Pepper and J. C. Heinrich, _The finite element method: basic concepts and applications with MATLAB, MAPLE, and COMSOL_. CRC press, 2017.
  * [69] Z. Hao, Z. Wang, H. Su, C. Ying, Y. Dong, S. Liu, Z. Cheng, J. Song, and J. Zhu, “Gnot: A general neural operator transformer for operator learning,” in _International Conference on Machine Learning_. PMLR, 2023, pp. 12 556–12 569.
  * [70] Z. Li, D. Shu, and A. Barati Farimani, “Scalable transformer for pde surrogate modeling,” _Advances in Neural Information Processing Systems_ , vol. 36, 2024.
  * [71] Anonymous, “Neural manifold operators for learning the evolution of physical dynamics,” 2023. [Online]. Available: https://openreview.net/forum?id=SQnOmOzqAM
  * [72] R. Volpi, H. Namkoong, O. Sener, J. C. Duchi, V. Murino, and S. Savarese, “Generalizing to unseen domains via adversarial data augmentation,” 2018.
  * [73] L. Mansilla, R. Echeveste, D. H. Milone, and E. Ferrante, “Domain generalization via gradient surgery,” 2021, pp. 6630–6638.
  * [74] Q. Wu, H. Zhang, J. Yan, and D. Wipf, “Handling distribution shifts on graphs: An invariance perspective,” _arXiv preprint arXiv:2202.02466_ , 2022.
  * [75] X. Li, Y. Dai, Y. Ge, J. Liu, Y. Shan, and L.-Y. Duan, “Uncertainty modeling for out-of-distribution generalization,” _arXiv preprint arXiv:2202.03958_ , 2022.
  * [76] S. Gui, M. Liu, X. Li, Y. Luo, and S. Ji, “Joint learning of label and environment causal independence for graph out-of-distribution generalization,” _arXiv preprint arXiv:2306.01103_ , 2023.
  * [77] W. Lu, J. Wang, X. Sun, Y. Chen, X. Ji, Q. Yang, and X. Xie, “Diversify: A general framework for time series out-of-distribution detection and generalization,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2024.
  * [78] H. Wang, H. Wu, J. Sun, S. Zhang, C. Chen, X.-S. Hua, and X. Luo, “IDEA: An invariant perspective for efficient domain adaptive image retrieval,” in _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023. [Online]. Available: https://openreview.net/forum?id=77i6itptQW
  * [79] Y.-X. Wu, X. Wang, A. Zhang, X. He, and T.-S. Chua, “Discovering invariant rationales for graph neural networks,” _arXiv preprint arXiv:2201.12872_ , 2022.
  * [80] S. Wu, M. Yuksekgonul, L. Zhang, and J. Zou, “Discover and cure: Concept-aware mitigation of spurious correlation,” in _International Conference on Machine Learning_. PMLR, 2023, pp. 37 765–37 786.
  * [81] R. Wang, M. Yi, Z. Chen, and S. Zhu, “Out-of-distribution generalization with causal invariant transformations,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 375–385.
  * [82] K. Wang, H. Wu, Y. Duan, G. Zhang, K. Wang, X. Peng, Y. Zheng, Y. Liang, and Y. Wang, “Nuwadynamics: Discovering and updating in causal spatio-temporal modeling,” in _The Twelfth International Conference on Learning Representations_ , 2024. [Online]. Available: https://openreview.net/forum?id=sLdVl0q68X
  * [83] H. Ye, C. Xie, T. Cai, R. Li, Z. Li, and L. Wang, “Towards a theoretical framework of out-of-distribution generalization,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 23 519–23 531, 2021.
  * [84] H. Yao, Y. Wang, S. Li, L. Zhang, W. Liang, J. Zou, and C. Finn, “Improving out-of-distribution robustness via selective augmentation,” in _International Conference on Machine Learning_. PMLR, 2022, pp. 25 407–25 437.
  * [85] X. Deng, W. Wang, F. Feng, H. Zhang, X. He, and Y. Liao, “Counterfactual active learning for out-of-distribution generalization,” in _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 2023, pp. 11 362–11 377.
  * [86] Y. Yang, Y. Zhang, X. Song, and Y. Xu, “Not all out-of-distribution data are harmful to open-set active learning,” _Advances in Neural Information Processing Systems_ , vol. 36, 2024.
  * [87] J. Srebric, V. Vukovic, G. He, and X. Yang, “Cfd boundary conditions for contaminant dispersion, heat transfer and airflow simulations around human occupants in indoor environments,” _Building and Environment_ , vol. 43, no. 3, pp. 294–303, 2008.
  * [88] B. Blocken, T. Stathopoulos, and J. Carmeliet, “Cfd simulation of the atmospheric boundary layer: wall function problems,” _Atmospheric environment_ , vol. 41, no. 2, pp. 238–252, 2007.
  * [89] Y. You, T. Chen, Y. Shen, and Z. Wang, “Graph contrastive learning automated,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 12 121–12 132.
  * [90] Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang, and Y. Shen, “Graph contrastive learning with augmentations,” _Advances in neural information processing systems_ , vol. 33, pp. 5812–5823, 2020.
  * [91] S. Li, X. Wang, A. Zhang, Y. Wu, X. He, and T.-S. Chua, “Let invariant rationale discovery inspire graph contrastive learning,” in _International conference on machine learning_. PMLR, 2022, pp. 13 052–13 065.
  * [92] J. Fang, X. Li, Y. Sui, Y. Gao, G. Zhang, K. Wang, X. Wang, and X. He, “Exgc: Bridging efficiency and explainability in graph condensation,” _arXiv preprint arXiv:2402.05962_ , 2024.
  * [93] J. Fang, S. Zhang, C. Wu, Z. Liu, S. Li, K. Wang, W. Du, X. Wang, and X. He, “Moltc: Towards molecular relational modeling in language models,” _arXiv preprint arXiv:2402.03781_ , 2024.
  * [94] C. Schuldt, I. Laptev, and B. Caputo, “Recognizing human actions: a local svm approach,” in _Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004._ , vol. 3. IEEE, 2004, pp. 32–36.
  * [95] M. Takamoto, T. Praditia, R. Leiteritz, D. MacKinlay, F. Alesiani, D. Pflüger, and M. Niepert, “Pdebench: An extensive benchmark for scientific machine learning,” _Advances in Neural Information Processing Systems_ , vol. 35, pp. 1596–1611, 2022.
  * [96] H. Wu, K. Wang, F. Xu, Y. Li, X. Wang, W. Wang, H. Wang, and X. Luo, “Spatio-temporal twins with a cache for modeling long-term system dynamics,” 2023.
  * [97] K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, and Q. Tian, “Pangu-weather: A 3d high-resolution model for fast and accurate global weather forecast,” _arXiv preprint arXiv:2211.02556_ , 2022.
  * [98] S. Rasp, S. Hoyer, A. Merose, I. Langmore, P. Battaglia, T. Russel, A. Sanchez-Gonzalez, V. Yang, R. Carver, S. Agrawal _et al._ , “Weatherbench 2: A benchmark for the next generation of data-driven global weather models,” _arXiv preprint arXiv:2308.15560_ , 2023.
  * [99] M. Veillette, S. Samsi, and C. Mattioli, “Sevir: A storm event imagery dataset for deep learning applications in radar and satellite meteorology,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 22 009–22 019, 2020.
  * [100] X. Shi, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-c. Woo, “Convolutional lstm network: A machine learning approach for precipitation nowcasting,” _Advances in neural information processing systems_ , vol. 28, 2015.
  * [101] Z. Gao, C. Tan, L. Wu, and S. Z. Li, “Simvp: Simpler yet better video prediction,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 3170–3180.
  * [102] Z. Gao, X. Shi, H. Wang, Y. Zhu, Y. B. Wang, M. Li, and D.-Y. Yeung, “Earthformer: Exploring space-time transformers for earth system forecasting,” _Advances in Neural Information Processing Systems_ , vol. 35, pp. 25 390–25 403, 2022.
  * [103] M. A. Rahman, Z. E. Ross, and K. Azizzadenesheli, “U-no: U-shaped neural operators,” _arXiv preprint arXiv:2204.11127_ , 2022.
  * [104] A. Tran, A. Mathews, L. Xie, and C. S. Ong, “Factorized fourier neural operators,” _arXiv preprint arXiv:2111.13802_ , 2021.
  * [105] C. Tan, J. Wang, Z. Gao, S. Li, L. Wu, J. Xia, and S. Z. Li, “Revisiting the temporal modeling in spatio-temporal predictive learning under a unified view,” _arXiv preprint arXiv:2310.05829_ , 2023.
  * [106] J. Fang, W. Liu, A. Zhang, X. Wang, X. He, K. Wang, and T.-S. Chua, “On regularization for explaining graph neural networks: An information theory perspective,” 2023. [Online]. Available: https://openreview.net/forum?id=5rX7M4wa2R_
  * [107] K. Wang, Y. Liang, X. Li, G. Li, B. Ghanem, R. Zimmermann, H. Yi, Y. Zhang, Y. Wang _et al._ , “Brave the wind and the waves: Discovering robust and generalizable graph lottery tickets,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2023.
  * [108] J. Fang, W. Liu, Y. Gao, Z. Liu, A. Zhang, X. Wang, and X. He, “Evaluating post-hoc explanations for graph neural networks via robustness analysis,” in _Thirty-seventh Conference on Neural Information Processing Systems_ , 2023.
  * [109] J. Fang, X. Li, Y. Sui, Y. Gao, G. Zhang, K. Wang, X. Wang, and X. He, “Exgc: Bridging efficiency and explainability in graph condensation,” 2024.
  * [110] Y. Xia, Y. Liang, H. Wen, X. Liu, K. Wang, Z. Zhou, and R. Zimmermann, “Deciphering spatio-temporal graph forecasting: A causal lens and treatment,” _Advances in Neural Information Processing Systems_ , vol. 36, 2024.
  * [111] Z. Zhou, Q. Huang, K. Yang, K. Wang, X. Wang, Y. Zhang, Y. Liang, and Y. Wang, “Maintaining the status quo: Capturing invariant relations for ood spatiotemporal learning,” in _Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_ , 2023, pp. 3603–3614.
  * [112] Y. Zhou, T. Lei, H. Liu, N. Du, Y. Huang, V. Zhao, A. M. Dai, Q. V. Le, J. Laudon _et al._ , “Mixture-of-experts with expert choice routing,” _Advances in Neural Information Processing Systems_ , vol. 35, pp. 7103–7114, 2022.
  * [113] N. Du, Y. Huang, A. M. Dai, S. Tong, D. Lepikhin, Y. Xu, M. Krikun, Y. Zhou, A. W. Yu, O. Firat _et al._ , “Glam: Efficient scaling of language models with mixture-of-experts,” in _International Conference on Machine Learning_. PMLR, 2022, pp. 5547–5569.
  * [114] R. Wang, D. Tang, N. Duan, Z. Wei, X. Huang, G. Cao, D. Jiang, M. Zhou _et al._ , “K-adapter: Infusing knowledge into pre-trained models with adapters,” _arXiv preprint arXiv:2002.01808_ , 2020.
  * [115] Z. Chen, Y. Duan, W. Wang, J. He, T. Lu, J. Dai, and Y. Qiao, “Vision transformer adapter for dense predictions,” _arXiv preprint arXiv:2205.08534_ , 2022.

|  Hao Wu is a master’s student who is currently undergoing joint training by
the Department of Computer Science and Technology at the University of Science
and Technology of China (USTC) and the Machine Learning Platform Department at
Tencent TEG. His research interests encompass various areas, including spatio-
temporal data mining, modeling of physical dynamical systems, and meta-
learning, among others. His work has been published in top-tier conferences
and journals like ICLR, NeurIPS, AAAI and TKDE.
---|---
|  Fan Xu received his Bachelor’s degree from Dalian University of Technology
in 2022. He is currently working towards his M.S. degree at University of
Science and Technology of China (USTC). His research interests include graph
representation learning, spatio-temporal data mining, AI4Science and anomaly
detection.
---|---
|  Yifan Duan is currently a master’s student at the School of Software
Engineering, University of Science and Technology of China (USTC). With a
profound interest in the realm of spatio-temporal data mining and diffusion
models.
---|---
|  Ziwei Niu was born in 2000, received his B.S. degree from the College of
Computer Science and Technology at Anhui Agricultural University in 2021. He
is currently pursuing the Ph.D. degree with the College of Computer Science
and Technology at Zhejiang University since 2021. His research interests
include domain adaptation, domain generalization and multimodal learning.
---|---
|  Weiyan Wang currently work for Tencent as a intern. And he has just gotten
his Ph.D degree from Computer Science and Engineering Department, Hong Kong
University of Science and Technology. Before that, he worked as an engineer in
the headquater of Bank of China. He has received the M.Phil. degree on
Computer Software and Theory from Institution of Software, Chinese Academy of
Sciences. His B.Eng. degree in Computer Science and Technology is from
Huazhong University of Science and Technology.
---|---
|  Gaofeng Lu is currently pursuing his master’s degree in the University of
Science and Technology of China. He is interning at Tencent. His research
interests include recommender system, AutoML and AI for Science.
---|---
|  Kun Wang is currently a Ph.D. candidate of University of Science and
Technology of China (USTC). Kun Wang research primarily focuses on graph-
structured applications, delving into areas such as sparsification (including
pruning, compression, and condensation), data mining (with an emphasis on
spatio-temporal graph forecasting and Earth Science), and AI applications in
biochemistry. His work has been published in top-tier conferences and journals
like ICLR, NeurIPS, KDD, AAAI, WWW, TKDE, and TPAMI.
---|---
|  Yuxuan Liang (Member, IEEE) is currently an Assistant Professor at
Intelligent Transportation Thrust, also affiliated with Data Science and
Analytics Thrust, Hong Kong University of Science and Technology (Guangzhou).
He is working on the research, development, and innovation of spatio-temporal
data mining and AI, with a broad range of applications in smart cities. Prior
to that, he completed his PhD study at NUS. He published over 40 peer-reviewed
papers in refereed journals and conferences, such as TKDE, AI Journal, TMC,
KDD, WWW, NeurIPS, and ICLR. He was recognized as 1 out of 10 most innovative
and impactful PhD students focusing on data science in Singapore by Singapore
Data Science Consortium.
---|---
|  Yang Wang is now an associate professor at School of Computer Science and
Technology, School of Software Engineering, and School of Data Science in
USTC. He got his Ph.D. degree at University of Science and Technology of China
in 2007. Since then, he keeps working at USTC till now as a postdoc and an
associate professor successively. Meanwhile, he also serves as the vice dean
of school of software engineering of USTC. His research interest mainly
includes wireless (sensor) networks, distribute systems, data mining, and
machine learning, and he is also interested in all kinds of applications of AI
and data mining technologies especially in urban computing and AI4Science.
---|---