# Ensemble Flow Reconstruction in the Atmospheric Boundary Layer from
Spatially Limited Measurements through Latent Diffusion Models

Alex Rybchuk<EMAIL_ADDRESS>Malik Hassanaly  Nicholas Hamilton  Paula
Doubrawa  National Renewable Energy Laboratory, Golden, Colorado, USA
Mitchell J. Fulton  Department of Mechanical Engineering, University of
Colorado Boulder, Boulder, Colorado, USA  Luis A. Martínez-Tossas  National
Renewable Energy Laboratory, Golden, Colorado, USA

###### Abstract

Due to costs and practical constraints, field campaigns in the atmospheric
boundary layer typically only measure a fraction of the atmospheric volume of
interest. Machine learning techniques have previously successfully
reconstructed unobserved regions of flow in canonical fluid mechanics problems
and two-dimensional geophysical flows, but these techniques have not yet been
demonstrated in the three-dimensional atmospheric boundary layer. Here, we
conduct a numerical analogue of a field campaign with spatially limited
measurements using large-eddy simulation. We pose flow reconstruction as an
inpainting problem, and reconstruct realistic samples of turbulent, three-
dimensional flow with the use of a latent diffusion model. The diffusion model
generates physically plausible turbulent structures on larger spatial scales,
even when input observations cover less than 1% of the volume. Through a
combination of qualitative visualization and quantitative assessment, we
demonstrate that the diffusion model generates meaningfully diverse samples
when conditioned on just one observation. These samples successfully serve as
initial conditions for a large-eddy simulation code. We find that diffusion
models show promise and potential for other applications for other turbulent
flow reconstruction problems.

##  I Introduction

Atmospheric field campaigns aim to characterize the complicated state of the
atmosphere by deploying several measurement systems, often supplementing
observations with modeling. Field campaigns in the atmospheric boundary layer
(ABL, approximately the lowest 1 km), are critical for different research
areas, such as wind energy Moriarty  _et al._ (2020); Hamilton  _et al._
(2022), air quality barad1958project; Allwine and Flaherty (2006), and
wildland fires Warneke  _et al._ (2023). While field campaigns in the ABL
typically strive to measure as much of the lower atmosphere as possible, they
measure only a small fraction of the atmosphere due to practical constraints
and the costs associated with observation systems. Models can be used to fill
the unmeasured areas between the spatially limited measurements, so that
atmospheric dynamics can be characterized across a wide range of scales. This
combination of measurements and models enables what is referred to as flow
reconstruction: turning spatially and temporally sparse information into a
highly resolved field.

Machine learning has emerged as a powerful technique for turbulent flow
reconstructionBuzzicotti (2023); Yu, Bi, and Fan (2023), joining the ranks of
more traditional approaches such as data assimilation Meldi and Poux (2017);
Chandramouli, Memin, and Heitz (2020); Mons, Du, and Zaki (2021); Wang and
Zaki (2021); Zaki and Wang (2021) and nudging Di Leoni, Mazzino, and Biferale
(2020); Zauner  _et al._ (2022). The flow reconstruction problem can be
formulated in a variety of approaches, depending on the available data and
choice of machine learning architecture. Perhaps the most common framing is
that of a “super-resolution” problem (see Fukami et al. (2023) Fukami,
Fukagata, and Taira (2023) for a recent review). In super-resolution, low-
resolution data is available, and data is upsampled to higher resolution
through the use of an algorithm. Turbulence super-resolution has been carried
out through the use of convolutional neural network architectures Fukami,
Fukagata, and Taira (2019); Liu  _et al._ (2020); Fukami, Fukagata, and Taira
(2021) as well as generative adversarial network (GAN) architectures Xie  _et
al._ (2018); Deng  _et al._ (2019); Stengel  _et al._ (2020); Yousif, Yu, and
Lim (2021); Hassanaly  _et al._ (2022); Wang  _et al._ (2022). In an
alternative to super-resolution, turbulent flow reconstruction has been posed
as an “inpainting” problem Buzzicotti  _et al._ (2021). In this scenario, part
of an image is masked, and an algorithm plausibly reconstructs the missing
data. Others have specifically examined the problem of reconstruction given
sparse measurements, often through the use of convolutional neural networks
fukamiGlobalFieldReconstruction2021; Gundersen  _et al._ (2021). Finally,
instead of using machine learning architectures rooted in the field of
computer vision, others have drawn inspiration from the intersection of
partial differential equations and machine learning. Flow reconstruction
approaches in these categories employ architectures such as “physics-informed
neural networks” (PINNs)Cai  _et al._ (2021); Clark Di Leoni  _et al._
(2023a); Du, Wang, and Zaki (2022) and Deep Operator NetworksClark Di Leoni
_et al._ (2023b). All in all, the aforementioned flow reconstruction
techniques have been applied to a variety of turbulent flow environments,
ranging from two-dimensionalDeng  _et al._ (2019) and three-dimensionalYousif
_et al._ (2023) canonical fluid mechanics problems (e.g., flow behind a
cylinder) to two-dimensional geophysical
problemsfukamiGlobalFieldReconstruction2021 (e.g., the state of the sea
surface).

While flow reconstruction based on machine learning has been shown to be
powerful, these techniques have not yet been applied to real-world, three-
dimensional geophysical flows, such as an ABL. Real-world, ABL flow
reconstruction comes with important challenges that must be accounted for. For
example, many observation systems (e.g., Doppler lidar) measure line-of-sight
velocity, which often serves as an accurate proxy for just one velocity
component (e.g., streamwise velocity $u$) instead of measuring all three
velocity components ($u,v,w$). So far, only PINNs have demonstrated the
ability to reconstruct all velocity components from measurements of just a
scalar fieldCai  _et al._ (2021). While PINNs show great potential, they often
struggle for multi-scale problemsLi  _et al._ (2023) such as high $Re$ three-
dimensional turbulence, and their success has not yet been demonstrated in an
ABL. Another issue in reconstructing real-world flows is the sparseness of ABL
measurements, as mentioned earlier. The atmosphere is highly chaotic and flow
reconstruction from sparse measurements is typically an ill-posed problem, as
many non-unique states could correspond to the same observation. As such,
geoscientists often characterize its state probabilistically (e.g., in
ensemble-based weather forecastingWilks (2019)). However, most flow
reconstruction studies have been practically deterministic (e.g., using GANs,
which suffer from the “mode collapse” problem Salimans  _et al._ (2016)). Only
two studies thus far, Gundersen  _et al._ (2021) using variational
autoencoders and Hassanaly  _et al._ (2022) using modified GANs, have
reconstructed turbulent flow in a probabilistic manner, though it is not clear
if their two-dimensional architectures can computationally scale to a three-
dimensional, highly turbulent ABL.

Recently, a new neural architecture known as a diffusion model (DM) Sohl-
Dickstein  _et al._ (2015); Song and Ermon (2020) has achieved state-of-the-
art status for generating high-resolution imagery Dhariwal and Nichol (2021),
showing promise for turbulent flow reconstruction in the ABL. A specific
category of DM, known as a latent diffusion model (LDM) Rombach  _et al._
(2022) is computationally efficient, and LDMs have been used to generate high-
resolution two-dimensional imagery Rombach  _et al._ (2022) and three-
dimensional medical imagery Pinaya  _et al._ (2022). DMs are inherently
stochastic, and they have been shown to generate diverse, photorealistic
imagery when a random seed is changed. Finally, while DMs have shown strong
performance for the super-resolution problem Rombach  _et al._ (2022), they
have also excelled at inpainting Lugmayr  _et al._ (2022); Saharia  _et al._
(2022). We posit that the inpainting perspective is a natural approach to pose
the problem of flow reconstruction from spatially limited measurements.

In this paper, we ask: given measurements in one small region of the
atmosphere, can we estimate the instantaneous, unmeasured state of the ABL
nearby through inpainting with an LDM? Here, we study this problem in the
context of a synthetic field campaign that is conducted through large-eddy
simulation (LES). In this work, our flow reconstruction strategy is to
recreate instantaneous, volumetric flow fields. This strategy shares
similarities with data assimilation approaches that are commonly applied for
mesoscale and synoptic scale atmospheric reconstruction, namely 4DVARCourtier,
Thépaut, and Hollingsworth (1994) and the ensemble Kalman filter Evensen
(1994), which produce plausible initial conditions for a dynamical solver
given a time-history of observations. Our investigation here addresses three
key ABL reconstruction challenges in a synthetic environment: (1)
probabilistic reconstruction, (2) dealing with spatially limited measurements,
and (3) translating from scalar measurements to all three velocity components.
For the time being, we omit two additional challenges of ABL reconstruction,
namely dealing with (1) noisy measurements and (2) a time-history of
measurements. An optimal ABL reconstruction strategy would likely be able to
account for all these challenges.

We find that LDMs can successfully reconstruct ABL flow given limited
measurements, and we believe that LDMs can be applied for other turbulent flow
reconstruction problems as well. In this paper, we demonstrate the following
contributions:

  * •

Through the use of LDMs, we can generate diverse three-dimensional turbulent
flow fields. We characterize the quality of LDM samples through the use of
qualitative visualization and quantitative assessment.

  * •

We show that LDMs can reconstruct all three velocity components ($u$, $v$,
$w$), even when given observations of almost exclusively $u$.

  * •

While LDM studies in the field of computer vision have masked upwards of 75%
of an image, we show that LDM reconstructions can be conditioned, even when
minimal observations ($<$1% of the volume) are provided.

  * •

We demonstrate that a LDM sample can successfully be used as an initial
condition for an LES. To our knowledge, this is the first time that a machine
learning sample has demonstrated this kind of compatibility with an LES code.

The rest of this manuscript is structured as follows. In Section II, we
describe our synthetic field campaign, the configuration of the flow
reconstruction problem, and the LES dataset that is used to train and test the
LDM. In Section III, we provide details of the LDM architecture. In Section
IV, we study the performance of the LDMs. Finally, in Section V, we conclude
the paper and discuss potential future lines of inquiry.

##  II Data

In this work, we explore flow reconstruction in the context of the real-world
Rotor Aerodynamics Aeroelastics and Wakes (RAAW) field campaign (Hamilton  _et
al._ , 2022), which seeks to thoroughly characterize the behavior of a single,
utility-scale wind turbine with respect to the incoming atmospheric inflow,
particularly on shorter timescales (1 second to 10 minutes). Here, we conduct
a synthetic version of the field campaign, which is sometimes referred to as
an “observing system simulation experiment.”

We simulate a thermally neutral atmospheric boundary layer using the LES code
AMR-Wind (Sprague, 2021). AMR-Wind solves the incompressible Navier Stokes
equations using a finite-volume approach with second-order accuracy in space
and time. Details of the spatial and temporal discretization can be found in
Almgren et al. (1998) (Almgren  _et al._ , 1998). The simulation is run with a
Smagorinsky subgrid-scale model Smagorinsky (1963). The domain is forced by
$U=$ 10 m s-1 geostrophic winds in the $x$-direction, as well as Coriolis
forcing at a latitude of 90$\degree$. The LES domain is sized
$(x,y,z)=(1920,1920,960)$ m with $(nx,ny,nz)=(128,128,64)$ grid points,
leading to 15-m resolution in all three directions. The simulation uses a
fixed 0.5-s timestep. The LES is initially run for 6 hours to allow turbulence
to spin up and become stationary. Afterward, we generate data for the training
set by running the LES for 3.5 days of simulated time and saving the $(u,v,w)$
fields everywhere in the domain every minute, totaling 5,040 samples. After
the training set is generated, we generate a test dataset by simulating
another 3.5 days after the end of the training dataset period. By using test
data from after the training period, we ensure there is no cross-contamination
between datasets while also achieving good statistical agreement between the
two datasets in our heights of interest, rotor span heights of 56.5–183.5 m
(Section IV).

Figure 1: (a) The masked region and observed region associated with the box
mask. (b) The observations associated with the Field Campaign (FC) mask, with
each instrument highlighted. The regions without observations are masked.

Synthetic measurements are generated by masking regions of the three-
dimensional LES output, and in the context of this paper, we refer to unmasked
regions as “observations.” We test the LDM on two sets of masks (Fig. 1). The
“box mask” is commonly used in two-dimensional image inpainting problems
(Saharia  _et al._ , 2022; Lugmayr  _et al._ , 2022). Here, we mask a cube at
the center of the domain spanning $(64,64,32)$ grid points, in order to
qualitatively demonstrate flow reconstruction in a scenario with many
observations. For the box mask, we use observations of all three components
velocity. The “field campaign (FC) mask” hides data at locations specific to
the RAAW field campaign instrumentation layout, observing data at 1,525 of
1,048,576 LES grid cells. This mask could be modified for any other layout of
field measurements. All instruments are aligned to point into the incoming
wind, as they would be in the field. The turbine can be thought of as sitting
at $(x,y)=(1800,960)$ m. The horizontal scanning lidar reaches 1,000 m upwind
of the turbine, covering an azimuthal range of 18$\degree$ at a height of 120
m. The vertical spinner lidar scan covers the rotor area and sits 120 m upwind
of the turbine. The meteorological mast sits 360 m upwind of the turbine and
measures in a vertical column between 0 m and 180 m.

In this study, we take steps towards developing an algorithm that could be
applied to real-world measurements, but we still make a number of simplifying
assumptions that will be relaxed in future work.

  * •

We assume that measurements are noise-free.

  * •

We assume that both lidars directly measure the $u$-component of velocity and
only this component of velocity. In practice, lidars measure line-of-sight
velocity, which is then projected to other directions to obtain specific
components of the velocity vector. As our lidars measure directly upstream
into the oncoming flow, we believe this assumption is warranted. The
meteorological mast measures all three components of velocity.

  * •

We assume that lidars instantaneously scan their 2D $x-y$ plane of interest.
In practice, both lidars scan across their 2D planes in approximately 2–5
seconds.

##  III Methods

###  III.1 Background

Figure 2: A schematic depicting the major components of an LDM and their
function. (a) First, the encoder and decoder of an autoencoder are trained.
(b) Next, a diffusion model is trained with the help of the encoder from the
autoencoder. The conditioning is optional and is not included for
unconditional networks. (c) The trained encoder, diffusion model, and decoder
are combined into an LDM and used to generate samples, possibly given optional
conditioning. Additional details can be found in Fig. 3 and Table 1.

In order to generate synthetic atmospheric states, we employ an LDM (Rombach
_et al._ , 2022, Figs. 2, 3, Table 1), an architecture based on DMs. In recent
years, DMs have emerged as a new category of deep generative model. They were
originally developed through the perspective of nonequilibrium statistical
physics Sohl-Dickstein  _et al._ (2015), but they can be derived through the
lens of score-based modeling Song and Ermon (2020) or Markovian hierarchical
variational autoencoders Luo (2022). From one perspective Song  _et al._
(2021), DMs are trained with the assistance of a prescribed degradation
process, in which Gaussian noise is repeatedly added to a sample $\bm{x}$ over
an interval $t\in[0,T]$ until the final sample is indistinguishable from pure
Gaussian noise. This process can be thought of as a forward stochastic
differential equation

$\text{d}\bm{x}=\bm{f}(\bm{x},t)\text{d}t+g(t)\text{d}\bm{w}$ (1)

where $\bm{f}$ is the drift coefficient, $g$ is the diffusion coefficient, and
$\bm{w}$ is the standard Wiener process. This process can be undone such that
the final Gaussian noise state can be reverted to the initial sample using the
reverse stochastic differential equation

$\text{d}\bm{x}=[\bm{f}(\bm{x},t)\text{d}t-g(t)^{2}\nabla_{\bm{x}}\log
p_{t}(\bm{x})]\text{d}t+g(t)\text{d}\bm{\bar{w}}$ (2)

where $\bm{\bar{w}}$ is the standard Wiener process for the reverse equation
and $\nabla\log p_{t}(\bm{x})$ is the score function, which is challenging to
estimate. DMs learn the score function, after which they can be used to draw
samples from the probability distribution function $p_{0}(x)$ (which we will
simply refer to as $p(x)$ from here on) that characterizes the training
dataset by initially starting with Gaussian noise. Given supplemental
information, such as observations or class labels, DMs can draw samples from a
conditional probability density function $p(x|y)$ through a number of
approaches—for example, training on paired data to directly learn to sample
from $p(x|y)$ as is done by Song and Ermon (2020), or by first learning to
sample from the unconditional $p(x)$ and then learning an additional
conditioning mechanism (e.g., Kawar  _et al._ , 2022). We employ the first
approach in this study.

While traditional DMs have been shown to generate high-quality images, they
can be prohibitively expensive to train for large samples, and three-
dimensional samples are often large. LDMs address this issue by compressing
raw, pixel-space samples into a latent space with the use of an autoencoder
(AE) (Fig. 2a). The AE is traditionally trained with a mixture of $L_{1}$ or
$L_{2}$ losses, Kullback–Leibler divergence losses as in a variational AE
Kingma and Welling (2013), perceptual losses based on the VGG network Zhang
_et al._ (2018), and patch-based adversarial losses Isola  _et al._ (2018).
After the AE is trained, a diffusion model is trained to generate samples in
latent space (Fig. 2b). After the components of the autoencoder and diffusion
model are trained, they can be combined to generate samples in pixel-space
(Fig. 2c).

###  III.2 Modifications to the original LDM

We modify the original LDM architecture so that it can be applied to generate
synthetic LES data. The exact implementation of the architecture can be found
at

https://github.com/rybchuk/latent-diffusion-3d-atmospheric-boundary-layer, and
we provide a summary of the modifications to the LDM code here.

  * •

The original LDM was developed for 2D images, so we modify the architecture to
work on 3D data through the use of operations like 3D convolutions and 3D
normalization.

  * •

While the original LDM uses several attention blocks Brock, Donahue, and
Simonyan (2019) throughout the AE, we omit their use there due to the
computational demands of a 3D attention block.

  * •

We replace group normalization Wu and He (2018) with instance normalization
Ulyanov, Vedaldi, and Lempitsky (2017) everywhere except within attention
blocks, as we found the latter performed better.

  * •

In the process of selecting the finalized architecture for this paper, we
experimented with different weights for each of the loss components in the AE.
We also added the option for a physics-based loss that assesses mass
conservation by calculating the divergence of the velocity field. In the end,
we found the best performance by using an $L_{1}$ term, Kullback–Leibler
divergence term, and whole-sample adversarial term in the loss function. We
omit the VGG-based adversarial loss as well as the mass conservation loss.

  * •

In the original LDM architecture Rombach  _et al._ (2022), image inpainting is
accomplished by first drawing a sample from $p(x|y)$ using unmasked regions as
conditioning information. In this scenario, there is no guarantee that the
generated sample exactly agrees with the conditioning information $y$, so
inpainting is achieved by overlaying $y$ onto the sample in a postprocessing
step. In effect, this treats conditioning information as a hard constraint,
meaning the observation is exactly matched in the reconstruction, which makes
sense when inpainting images. However, we found this hard constraint would
lead to artifacts at the boundaries of FC mask observations, likely because
these observations are only one pixel wide in certain dimensions. As such, we
do not postprocess LDM output to exactly match the observations, thereby
treating observations as soft constraints, meaning observations are not
exactly matched in reconstructions. In other common atmospheric reconstruction
techniques, namely data assimilation Carrassi  _et al._ (2018), observations
are also treated as soft constraints. Though, we note that in data
assimilation, the relative influence of observations on reconstructions can be
controlled by prescribing a certain measurement noise magnitude, whereas in
our LDM technique here, we cannot modulate the influence of observations.

###  III.3 Network configurations

Figure 3: A schematic showing the internal components of the LDM in greater detail, including (a) in the autoencoder and (b) in the UNet of the diffusion model. Table 1:  Additional parameters used during training of the autoencoder and the diffusion model. Parameter | Value
---|---
Batch size, AE | 2
Batch size, DM | 2
Base learning rate, AE | 4.5e-6
Base learning rate, DM | 5e-5
Noise schedule | Linear
$N$ noise steps | 1000
Start variance | 0.0015
End variance | 0.0155
$N$ attention heads | 8

The LDM requires an AE network for compression to the latent space and
decompression (Fig. 3a). Here, the AE has three input, latent space, and
output channels that correspond to the velocity variables $(u,v,w)$. Future
work could expand this to learn other quantities like temperature, though we
prioritize the three velocity components in this study as these have the
greatest relevance to wind turbine dynamics. The encoder and decoder of the AE
have three internal levels, each with three residual blocks He  _et al._
(2016), compressing data with spatial dimensions of $(128,128,64)$ to
$(32,32,16)$. The first internal level uses 16 channels, followed by 32
channels, and then 64 channels. We use a batch size of 2 due to the high
memory use of 3D convolutions. This network is much smaller than the original
LDM used in Rombach  _et al._ (2022, https://github.com/CompVis/latent-
diffusion), which, for example, had six internal layers with 128 base
channels. Our network is smaller due to computational constraints (we train on
two 16-GB VRAM V100s) as well as our large three-dimensional samples.
Following common practice, we rescale the input data so that the values in
each channel fall in the range of -1 to 1.

While our primary goal is to generate candidate velocity fields given FC-style
observations, we train both a conditional DM and an unconditional DM to
provide more context on network behavior. Both DMs uses the same AE for
conversion between pixel space and latent space. The DMs use a UNet
architecture (Ronneberger, Fischer, and Brox, 2015) with three internal
layers, respectively using $(192,384,768)$ channels, and two residual blocks
per layer. Attention blocks are used in the 384 channel layers, 768 channel
layers, and the center of the UNet. The DM noise schedule uses 1,000 diffusion
steps and is linear. The only difference between the unconditional and
conditional DM is the input layer—the unconditional network uses three
channels, and the conditional network has an additional four channels, three
of which correspond to the compressed observation in latent space and the last
of which corresponds to a compressed binary mask corresponding to observed
pixels. The conditional DM is exposed to several different masks during
training, including box-style and the FC-style observations, in order to
increase robustness and potentially accuracy (Saharia  _et al._ , 2022). The
same conditional DM is then used to produce both box-style and FC-style
samples.

##  IV Results

Below, we assess the ability of the LDM to generate plausible atmospheric
states. We begin by visualizing unconditional and conditional LDM samples and
qualitatively assessing them, as is common in both computer vision manuscripts
(Song  _et al._ , 2021; Rombach  _et al._ , 2022) as well as turbulent flow
manuscripts (Fukami, Fukagata, and Taira, 2019, 2021; Leinonen  _et al._ ,
2023). Next, we quantitatively assess the LDM samples by calculating mean
profiles, statistics, spectra, and divergence. Finally, we quantitatively
assess the ability of the LDM to generate diverse samples from a single
observation.

###  IV.1 Qualitative assessment of LDM samples

Figure 4: (a)–(c) Streamwise cross sections of $u$ at $y$ = $960$ m from
ground truth, test data. (d)–(f) Same as above, except for samples from an
unconditional LDM.

First, in order to broadly demonstrate that LDMs can generate samples that
look like an ABL, we compare cross sections of $u$ from the unconditional LDM
to those from the ground-truth ABL training data (Fig. 4). Additional
qualitative visualizations of $v$ and $w$ as well as three-dimensional
isocontours are available in Appendix A. Similar to the training data, the LDM
samples have winds that are generally slow near the ground and stronger aloft
until they hit the capping inversion near 480 m, above which the winds are
less turbulent. The LDM turbulent structures look like turbulent structures in
the ground truth data. The horizontal boundaries of the samples are periodic,
just as they are in the testing data. Crucially, however, closer inspection
reveals that the LDM samples appear slightly smoother than the ground truth
data. This discrepancy on smaller scales is further characterized in Section
IV.2.

Figure 5: Streamwise cross sections of $u$ at $y$ = $960$ m for (a) a ground
truth sample, (b) the box observation created from the ground truth sample,
and (c) one sample from the conditional LDM, given the observation. (d) The
mean from $n=100$ conditional LDM samples. (e) The standard deviation from
$n=100$ conditional LDM samples. All subpanels display a white dashed line as
a reference to the mask.

We next show that conditional LDMs can inpaint well when provided abundant
data, as is the scenario with the box max (Fig. 5). We visualize a $u$ cross
section of a ground truth sample, the associated box mask observation, one
conditionally generated sample, and the mean and standard deviation from
$n=100$ conditionally generated samples. The conditional samples look similar
to the unconditional samples, except the perimeter of the conditional sample
and the averaged field is relatively unchanged. The transition between the
unmasked and the masked region is smooth and does not display any artifacts.
This smooth transition is in part achieved by treating observations as a soft
constraint instead of a traditional inpainting hard constraint, in which the
exact values of the observations would be superimposed over the unmasked
region. The statistical behavior of the conditional predictions qualitatively
matches expected behavior. The mean prediction clearly displays the capping
inversion, above which the mean flow structures inside the masked region agree
well with the structures in the unmasked region. Beneath the capping
inversion, the mean prediction is smoothed out when compared to individual
turbulent realizations. Inside the unmasked region, the standard deviation of
LDM predictions is largest at the bottom of the mask and smaller aloft. This
aligns with physical behavior and is expected, as mean shear gradients at the
surface encourage mechanical generation of turbulent kinetic energy and
therefore wind speed variance. The LDM also displays small, nonzero standard
deviation in the unmasked regions, which would not be the case if we treated
observations as a hard constraint.

Figure 6: (a)–(e) Same as Fig. 5, except for the FC mask. (f) An external
estimate of the conditional standard deviation for the observation in panel
(b), which is discussed in greater detail in Section IV.3. (g)–(l)
Corresponding top-down views at the height of the scanning lidar for the data
in panels (a)–(f).

Finally, we find that conditional LDMs can generate samples that look
realistic even when provided minimal observations, as is the case for the FC
mask (Fig. 6). We include additional visualizations, including visualizations
of $v$ and $w$ which are only conditioned on meteorological mast measurements,
in Appendix B. For the particular observation shown in Fig. 6, there is a
region of slow $u$-wind in the vicinity of the meteorological mast.
Correspondingly, each of the samples also has a patch of slow wind in the same
area. This slow flow structure is especially clear in the mean prediction
(Fig. 6d, j), and qualitatively, the mean prediction aligns well with the
ground truth in the vicinity of the observations. This behavior illustrates
that the LDM includes the conditional information of the measurements when
generating samples. Away from the observations, the mean prediction becomes
much smoother, showing that the conditioning only has local effects, as would
be expected in a turbulent flow. The standard deviation of the predictions for
the FC mask (Fig. 6e, k) also show a smaller standard deviation in the
vicinity of the observations, especially near the horizontal lidar
measurement. Thus, the conditioning is reducing the sample-to-sample spread
near the measurements, as desired. The FC samples exhibit much higher variance
than the box mask samples (Fig. 5e), as would be expected because the FC
samples are conditioned on a smaller region. As we discuss in Section IV.3 in
greater detail, the sample standard deviation qualitatively matches up with a
different estimate of the standard deviation that comes from another tool
(Fig. 5f).

After assessing the visual quality of samples, we further demonstrate the
quality of LDM samples by showing that they can successfully act initial
conditions for our LES code. We generate a FC mask sample using the
observation in Fig. 6 and then use it as the initial condition in a 1-hour LES
simulation. We create a video that visualizes a time history of a streamwise
cross section and a top-down view of the simulation at
https://github.com/rybchuk/latent-diffusion-3d-atmospheric-boundary-layer. The
simulation successfully runs and does not display any obvious numerical
artifacts such as discontinuities in the flow field around the masked region
or ringing in the form of waves or instabilities.

###  IV.2 Quantitative assessment of LDM samples

We next quantitatively assess the performance of the LDMs against physical
aspects of the LES: vertical profiles of velocity components and kinematic
fluxes, probability density functions of velocity components, turbulence
spectra, and mass conservation. For completeness, we compare LDM sample
statistics to those from both the training dataset and testing dataset. The
training and testing datasets have similar but distinct statistics, as they
come from distinct simulation periods. However, as we demonstrate below, their
statistics match within our region of interest (rotor disk heights), thereby
improving the probability of success for our flow reconstruction problem. To
facilitate these quantitative comparisons, we examine characteristics between
heights of 0 m to 480 m, namely below the capping inversion.

Figure 7: (a)–(c) Vertical profiles of horizontally averaged velocity
components from training data, testing data, unconditional LDM samples, and
FC-mask conditional LDM samples. Averages are shown as solid lines, and $\pm$1
standard deviations are shown as faded lines. (d) Kinematic fluxes of the same
data.

The LDM accurately captures average velocity profiles (Fig. 7a–c), while
slightly underperforming on kinematic velocity profiles (Fig. 7d). We compare
horizontally averaged profiles from the training dataset, testing dataset,
$n=100$ unconditional samples, and $n=100$ conditional samples from the FC
mask. The mean and standard deviation of LDM velocity profiles agree well with
training data at all heights, and with testing data beneath the capping
inversion. Thus, the LDM performs well on first-order statistics. However, the
LDMs do show some deviations when comparing kinematic fluxes, a second-order
statistic that is more challenging to capture than the mean. In particular,
the unconditional LDM underestimates the downward flux
$\overline{u^{\prime}w^{\prime}}$ by as much as 20% between heights of 15 m
and 150 m. The conditional LDM performs better in this region with only an 8%
discrepancy, possibly because of the near-surface conditioning information.

Figure 8: The probability distribution function for training data, testing
data, unconditional LDM samples, and conditional LDM samples for (a) $u$, (b)
$v$, and (c) $w$.

Similarly, the probability distribution functions (pdfs) of velocity
components show broad agreement, while missing fine-scale details (Fig. 8). We
calculate velocity distributions using the same three-dimensional data that
was used to calculate average vertical profiles. The pdfs from LDM samples
match the pdfs from training data well in all bins, only showing minor
discrepancies.

Figure 9: The turbulence spectra of $u$ at $z=90$ m for training data, testing
data, unconditional LDM samples, and conditional LDM samples.

While the power spectra of LDM samples match ground truth at large spatial
scales, the LDM spectra have too much energy at the smallest spatial scales
(Fig. 9). We calculate one-dimensional spectra of $u$ at a height of 90 m. The
LDM and ground truth spectra have similar values for wavenumbers smaller than
0.01 m-1, and the LDM accurately captures the inertial scale of turbulence, as
visualized by the $-5/3$ slope. However, for wavenumbers larger than 0.01 m-1,
the LDM has too much energy. This behavior has been observed in other machine
learning reconstruction studiesFukami, Fukagata, and Taira (2019, 2021) and is
consistent with the “spectral bias” problem Rahaman  _et al._ (2019) inherent
to many neural networks—the autoencoder part of our network seeks to minimize
the $L_{1}$ of the samples, thereby prioritizing reconstructing fields on the
largest spatial scales. As such, the smallest scales have a relatively small
impact on the loss function.

Figure 10: An assessment of mass conservation, as visualized by distributions
of velocity divergence for training data, testing data, unconditional LDM
samples, and conditional LDM samples. Training and testing distributions sit
on top of one another.

Finally, we assess the LDM’s ability to satisfy mass conservation by examining
distributions of velocity divergence $\frac{\partial u_{i}}{\partial x_{i}}$
(Fig. 10). If continuity were exactly satisfied, the distribution of
divergence would be shaped like a Dirichlet function at $\frac{\partial
u_{i}}{\partial x_{i}}=0$. The testing and training data approximately satisfy
this behavior, showing small deviations from 0 due to numerical artifacts.
While distributions for both the unconditional and conditional LDM data show a
spike centered on 0, their spikes are not nearly as sharp. Thus, the LDMs
appear to have some awareness of mass conservation, but they do not satisfy it
here as well as the ground truth data.

###  IV.3 Assessment of ensembles

####  IV.3.1 Rank histogram analysis

For ensemble-based flow reconstruction, it is important to have a well-
calibrated ensemble Wilks (2019). Ensemble members are drawn from some pdf,
and in an ideal scenario, the ground truth would be seen as just another draw
from that pdf, such that the ground truth is statistically indistinguishable
from the ensemble members, a condition termed “ensemble consistency.” Ideally,
the simulated ensemble would show zero bias relative to the ground truth, and
the ensemble should have an appropriate but not excessive amount of spread.

One common approach to assess ensemble consistency in the geosciences is the
rank histogram (Hamill, 2001; Leinonen  _et al._ , 2023). When given a group
of predictions, the rank of the ground truth is calculated by sorting the
ensemble members from smallest to largest by some scalar quantity (e.g., $u$
at a particular grid cell), and then identifying the position of the ground
truth relative to the ensemble members. When repeated for several
reconstructions, rank calculations can be collected into a rank histogram, a
diagram that can diagnose ensemble bias as well as overconfidence or
underconfidence.

Figure 11: Rank histogram for $u$ at the turbine nacelle, given 10-member LDM
FC ensembles for 360 observations.

We assess ensemble quality by compiling a rank histogram using the first 6
hours (360 observations) from the test set and generating 10 ensemble members
per observation (Fig. 11). We calculate the rank of each ground truth by
examining $u$ at the unobserved grid cell immediately downwind of the first
scanning lidar observation, a cell that can be thought of as the turbine
nacelle. An ideal rank histogram would look like a uniform distribution.
However, the rank histogram here looks like a combination of a U-shape and a
downward linear slope. The U-shape shows that LDM samples are underly
diverse—the ground truth disproportionately falls at one extreme of the
ensemble. The downward slope indicates that the ground truth tends to be
disproportionately smaller than the rest of the ensemble, or in other words,
the ensemble often overpredicts. We verify this by calculating the bias of the
LDM samples relative to their respective ground truths, finding that $u$ is on
average 0.02 m s-1 higher. Thus, while the downward slope suggests a tendency
to overpredict $u$ at turbine hub height, the near-zero bias is balanced out
by the right arm of the U-shaped distribution. In the end, the LDM ensembles
deviate from ideal behavior, which is in practice the case even for skillful
ensemble forecasts (e.g., Monache  _et al._ , 2013; Ravuri  _et al._ , 2021).
These deviations may be small in the end, and future work will use these
ensembles in a data assimilation process and assess if the unoptimal behavior
is problematic in practice.

####  IV.3.2 External estimate of conditional standard deviation

While rank histograms assess ensemble consistency and are used primarily in
the ensemble weather forecasting field, we can alternatively assess the
ensemble quality by examining “sample diversity”. This concept is commonly
examined in the deep generative modeling field, and when applied to computer
vision problems, metrics such as Inception Score (Salimans  _et al._ , 2016)
are commonly used. Here, we assess sample diversity by simply examining
standard deviation, which is a valuable measure for physics-based problems. In
Sec. IV.1, we assessed LDM sample diversity by taking a single observation,
e.g., the one in Fig 6a, and calculating the standard deviation of $u$ across
100 conditionally generated samples (Fig 6e). This conditional standard
deviation was calculated using samples from the LDM for a given observation.
Alternatively, by applying tools such as fully connected neural networks or
stochastic estimation (Hassanaly  _et al._ , 2022) to our training dataset, it
is possible to estimate what the conditional standard deviation should be for
a given observation, without the need to generate any samples. This function
has no awareness of the LDM or the samples it generates, and as such, it
serves as an external check on the sample-calculated conditional standard
deviation.

Following Hassanaly  _et al._ (2022), we obtain an external estimate (Fig 6f,
l) of the conditional standard deviation for the observation in Fig 6a, g by
training a fully connected neural network. We provide more details on the
external estimate methodology in Appendix C. From the side (Fig 6f), this
external estimate agrees well with the LDM sample standard deviation (Fig 6e),
but it fills in weaknesses that arise from calculating the sample standard
deviation with a finite number of samples. From the side, the two standard
deviations agree well in magnitude everywhere: the lowest grid cells, the
observation network, and above the capping inversion. Both estimates also show
larger variance just upwind of the meteorological mast and above the scanning
lidar, a behavior that arises due to the presence of a coherent structure.
When viewed from the top however (Fig 6l), the external estimate disagrees
with the sample standard deviation. This discrepancy arises because the LDM
here treats observations as a soft constraint, whereas the external estimate
treats observations more like a hard constraint. As such, the external
estimate shows a near-zero standard deviation in pixels where observation are
available. From both the side and the top, the external estimate clearly shows
that the sensing network has an impact on the flow reconstructions close to
the observations, up to a maximum reach of roughly 150 m in any direction. By
comparing the two standard deviations, we build confidence that the LDM
doesn’t show major deficiencies in terms of sample diversity. This result
complements the rank histogram analysis, suggesting that indeed the LDM is
only slightly underly diverse.

####  IV.3.3 Influence of FC observations on reconstruction accuracy

Figure 12: Vertical profiles of root mean squared error for (a) $u$, (b) $v$,
and (c) $w$ along the centerline of the domain at the meteorological mast
($x=1440$ m), the upstream edge of scanning lidar measurements ($x=795$ m),
and a location upstream of all observations ($x=480$ m).

Finally, we quantify the impact of FC observations on the reconstruction
accuracy at three locations (Fig. 12). We calculate vertical profiles of root
mean squared error (RMSE) using the dataset that was used in the rank
histogram analysis: 360 ground truth samples from which observations are
sampled, and 10 corresponding reconstructions for each ground truth. We
calculate RMSE along the centerline of observations ($y=960$ m) at the
location of the meteorological mast ($x=1440$ m), the upstream edge of
scanning lidar measurements ($x=795$ m), and an upstream location where
conditional standard deviation analysis revealed that observations no longer
had an influence on reconstructions ($x=480$ m).

The RMSE profiles show that observations improve reconstruction accuracy
predominantly as would be expected. Far away from observations at $x=480$ m,
the RMSE values for all three velocity components are generally largest when
compared to the other locations. The shape of these RMSE profiles correlates
well with velocity standard deviations (Fig. 7)—heights with larger standard
deviations are also associated with larger RMSE for the uncorrelated location.
Within the perimeter of the synthetic field campaign at $x=795$ m and $x=1440$
m, observations have their strongest impacts at rotor disk heights, with
relatively small impacts above and below the rotor disk. Relative to the RMSE
outside of the measurement perimeter, the RMSE of $u$ is decreased most
substantially (approximately 0.40 m s-1) at the height of the scanning lidar
measurements, as observations of $u$ are abundant at this elevation. While the
field campaign measurements predominantly measure $u$, the observations still
marginally impact RMSE profiles of $v$ and $w$ at rotor disk heights,
decreasing $v$ RMSE by approximately 0.05–0.01 m s-1 at most and $w$ RMSE by
approximately 0.1 m s-1. The meteorological mast at $x=1440$ m has the only
observations of $v$ and $w$, and correspondingly, RMSE for these variables
slightly outperforms RMSE at $x=795$ m.

##  V Conclusion

In this paper, we investigate turbulent flow reconstruction in the context of
a synthetic field campaign in the atmospheric boundary layer in which only
spatially limited observations are available. We demonstrate that latent
diffusion models (LDMs) create diverse, three-dimensional turbulent fields
that are convincingly similar to true LES fields. These reconstructions match
many physical characteristics of LES, such vertical profiles and the largest
spatial scales of the spectra. However, LDMs struggle at the smallest spatial
scales, diverging from LES spectra in this region and failing to preserve
continuity.

Our LDM work extends machine learning turbulent flow reconstruction literature
in several key manners. In contrast to many of the deterministic approaches,
LDMs generate diverse samples, which is important for turbulent environments.
Our LDM reconstructed samples with minimal observations ($<1\%$ of the domain
is observed), though the algorithm also works well with abundant observations.
And notably, LDM samples can be used as initial conditions for computationally
stable LES simulations.

This study suggests several lines of possible future inquiry for turbulent
flow reconstruction. In upcoming work, we will explore applying LDMs to noisy,
real-world measurements in the RAAW field campaign. The LDM architecture could
be modified to improve performance at the smallest spatial scales, perhaps
through the use of physics-informed lossesRaissi, Perdikaris, and Karniadakis
(2019). Finally, diffusion model researchers are investigating methods for
quicker sampling (e.g., Meng  _et al._ , 2022), which could potentially open
the road to real-time flow reconstruction. In summary, the results indicate
that diffusion models are a powerful class of machine learning algorithm that
is worthy of further exploration in turbulence research.

###### Acknowledgements.

This work was authored in part by the National Renewable Energy Laboratory,
operated by Alliance for Sustainable Energy, LLC, for the U.S. Department of
Energy (DOE) under Contract No. DE-AC36-08GO28308. Funding provided by the
U.S. Department of Energy Office of Energy Efficiency and Renewable Energy
Wind Energy Technologies Office. The views expressed in the article do not
necessarily represent the views of the DOE or the U.S. Government. The U.S.
Government retains and the publisher, by accepting the article for
publication, acknowledges that the U.S. Government retains a nonexclusive,
paid-up, irrevocable, worldwide license to publish or reproduce the published
form of this work, or allow others to do so, for U.S. Government purposes. The
research was performed using computational resources sponsored by the
Department of Energy’s Office of Energy Efficiency and Renewable Energy and
located at the National Renewable Energy Laboratory. This work would not have
been possible without several open-source diffusion modeling resources,
including those by HuggingFace (https://huggingface.co/blog/annotated-
diffusion), Janspiry (https://github.com/Janspiry/Palette-Image-to-Image-
Diffusion-Models), Walter H. L. Pinaya
(https://huggingface.co/spaces/Warvito/diffusion_brain), and CompVis
(https://github.com/CompVis/latent-diffusion). The analysis was performed
using a number of open-source Python libraries, including numpy
(harrisArrayProgrammingNumPy2020)), Xarray (hoyerXarrayNDLabeled2017), and
matplotlib (Hunter, 2007). Thank you to Hristo Chipilski for helpful
discussions regarding data assimilation, and Michael Kuhn for AMR-Wind
support. Additionally, thank you to the editor and the reviewers in helping
craft this manuscript.

##  Appendix A Additional visualizations of test data and unconditional
samples

Figure 13: (a)–(f) Same as Figure 4, but for $v$. Figure 14: (a)–(f) Same as
Figure 4, but for $w$. Figure 15: (a)–(f) Isometric views of vortices
identified as contours of the Q-criterion (Jeong and Hussain, 1995) with a
value of 0.0003 m s-1.

blabla

blabla

##  Appendix B Additional visualizations of box and FC conditional samples

Figure 16: A distinct (a) ground truth and (b) box observation from the one
shown in Fig. 5. The (c) mean prediction and (d) standard deviation of
prediction from 100 LDM samples are shown. (e)–(h) Four distinct LDM samples
that are conditioned on the observation. Figure 17: (a)–(h) Same as Fig. 16,
except for a distinct observation. Figure 18: A distinct (a) ground truth and
(b) FC observation of $u$ from the one shown in Fig. 6. The (c) mean
prediction and (d) standard deviation of prediction from 100 LDM samples are
shown. (e)–(h) Four distinct LDM samples that are conditioned on the
observation. Figure 19: (a)–(p) The same as Fig. 18, except for $v$. Figure
20: (a)–(p) The same as Fig. 18, except for $w$.

blabla

blabla

blabla

blabla

##  Appendix C Technical details for external estimates of the conditional
standard deviation

The a priori moment estimation method is detailed here. Consider an
observation $y\in R^{d}$ where $d$ is the length of the observation vector,
and $x\in R^{m}$ where $m$ is the length of the state vector. The function $f$
that best approximates $x$ given $y$, i.e., that minimizes $||f(y)-x||_{2}$,
is called the optimal estimator of $x$ given $y$. In addition, the optimal
estimator is the conditional expectation of $x$ given $y$ (Papoulis and
Unnikrishna Pillai, 2002). Therefore, by seeking the optimal estimator of $x$
given $y$, one can approximate the conditional moments of $x$ given $y$.
Likewise, by seeking the conditional moments of $x^{2}$ given $y$, one can
approximate the diagonal of the covariance matrix of the distribution sampled
(i.e., the pointwise variance).

We obtain a priori estimates of conditional standard deviation by using a
neural network. We construct the neural network to approximate the optimal
estimator of the atmospheric state (for the first moment) and the square of
the atmospheric state (for the second moment), given the observed field as the
input. We use the architecture of Ledig  _et al._ (2017), a fully
convolutional network with residual blocks and skip connections.

To ensure that a reasonable approximation of the optimal estimator has been
reached, we train multiple neural nets with increasing complexity, until the
mean-square-error (MSE) loss stops improving. Similar to Hassanaly  _et al._
(2022), we adjust two model architecture hyperparameters to converge to the
optimal estimator: the number of filters per convolutional layers varied in
the set $\\{1,2,4,8\\}$ and the number of residual blocks varied in the set
$\\{2,4,8,16\\}$. Unlike in Hassanaly  _et al._ (2022), the input and output
of the network have the same dimension. The input is altered by setting the
masked pixels to a zero value. The loss function is calculated by computing
the MSE with the masked pixels only, thereby enforcing that unmasked pixels do
not contribute to evaluating the conditional estimator. Since 3D convolutions
are used and to accommodate memory requirements, the batch size is typically
set to $8$ but changed to $4$ for the biggest network. A fixed learning rate
of $1e-3$ was used. Similar to Hassanaly  _et al._ (2022), the second moment
is estimated by outputting the square of the state substracted with the first
moment. All networks considered here were sufficiently trained to reach
convergence (Figure 21).

Figure 21: MSE training loss versus epoch number for all the model
architectures for FC mask for the first moment estimate (left) and the second
moment estimate (right).

Once fully trained, the final training loss is examined to decide if further
increase of network complexity is warranted to appropriately approximate the
optimal estimator (Figure 22). We find that as the number of trainable
parameters increases beyond approximately 9,000 to approximately 60,000, the
MSE decreases until a point where adding additional trainable parameters only
marginally improves the loss. As such we stop increasing the complexity of the
estimator network and produce the figure shown in Fig. 6f using the model with
the highest number of parameters.

Figure 22: Final MSE training loss versus number of trainable parameters for
the FC mask for the first moment estimate (left) and the second moment
estimate (right).

## References

  * Moriarty  _et al._ (2020) P. Moriarty, N. Hamilton, M. Debnath, T. Herges, B. Isom, J. K. Lundquist, D. Maniaci, B. Naughton, R. Pauly, J. Roadman, W. Shaw, J. van Dam, and S. Wharton, “American WAKE experimeNt (AWAKEN),” Tech. Rep. NREL/TP-5000-75789 (National Renewable Energy Lab. (NREL), Golden, CO (United States), 2020).
  * Hamilton  _et al._ (2022) N. Hamilton, P. Doubrawa, J. Naughton, and C. Kelley, “Rotor aerodynamics, aeroelastics, and wake (RAAW) campaign overview,” Bulletin of the American Physical Society (2022).
  * Allwine and Flaherty (2006) K. J. Allwine and J. E. Flaherty, “Joint Urban 2003: Study Overview And Instrument Locations,” Tech. Rep. PNNL-15967 (Pacific Northwest National Lab. (PNNL), Richland, WA (United States), 2006).
  * Warneke  _et al._ (2023) C. Warneke, J. P. Schwarz, J. Dibb, O. Kalashnikova, G. Frost, J. Al-Saad, S. S. Brown, W. A. Brewer, A. Soja, F. C. Seidel, R. A. Washenfelder, E. B. Wiggins, R. H. Moore, B. E. Anderson, C. Jordan, T. I. Yacovitch, S. C. Herndon, S. Liu, T. Kuwayama, D. Jaffe, N. Johnston, V. Selimovic, R. Yokelson, D. M. Giles, B. N. Holben, P. Goloub, I. Popovici, M. Trainer, A. Kumar, R. B. Pierce, D. Fahey, J. Roberts, E. M. Gargulinski, D. A. Peterson, X. Ye, L. H. Thapa, P. E. Saide, C. H. Fite, C. D. Holmes, S. Wang, M. M. Coggon, Z. C. J. Decker, C. E. Stockwell, L. Xu, G. Gkatzelis, K. Aikin, B. Lefer, J. Kaspari, D. Griffin, L. Zeng, R. Weber, M. Hastings, J. Chai, G. M. Wolfe, T. F. Hanisco, J. Liao, P. Campuzano Jost, H. Guo, J. L. Jimenez, J. Crawford, and T. F.-A. S. Team, “Fire Influence on Regional to Global Environments and Air Quality (FIREX-AQ),” Journal of Geophysical Research: Atmospheres 128, e2022JD037758 (2023).
  * Buzzicotti (2023) M. Buzzicotti, “Data reconstruction for complex flows using AI: Recent progress, obstacles, and perspectives,” Europhysics Letters 142, 23001 (2023).
  * Yu, Bi, and Fan (2023) C. Yu, X. Bi, and Y. Fan, “Deep learning for fluid velocity field estimation: A review,” Ocean Engineering 271, 113693 (2023).
  * Meldi and Poux (2017) M. Meldi and A. Poux, “A reduced order model based on Kalman filtering for sequential data assimilation of turbulent flows,” Journal of Computational Physics 347, 207–234 (2017).
  * Chandramouli, Memin, and Heitz (2020) P. Chandramouli, E. Memin, and D. Heitz, “4D large scale variational data assimilation of a turbulent flow with a dynamics error model,” Journal of Computational Physics 412, 109446 (2020).
  * Mons, Du, and Zaki (2021) V. Mons, Y. Du, and T. A. Zaki, “Ensemble-variational assimilation of statistical data in large-eddy simulation,” Physical Review Fluids 6, 104607 (2021).
  * Wang and Zaki (2021) M. Wang and T. A. Zaki, “State estimation in turbulent channel flow from limited observations,” Journal of Fluid Mechanics 917, A9 (2021).
  * Zaki and Wang (2021) T. A. Zaki and M. Wang, “From limited observations to the state of turbulence: Fundamental difficulties of flow reconstruction,” Physical Review Fluids 6, 100501 (2021).
  * Di Leoni, Mazzino, and Biferale (2020) P. C. Di Leoni, A. Mazzino, and L. Biferale, “Synchronization to Big Data: Nudging the Navier-Stokes Equations for Data Assimilation of Turbulent Flows,” Physical Review X 10, 011023 (2020).
  * Zauner  _et al._ (2022) M. Zauner, V. Mons, O. Marquet, and B. Leclaire, “Nudging-based data assimilation of the turbulent flow around a square cylinder,” Journal of Fluid Mechanics 937 (2022), 10.1017/jfm.2022.133.
  * Fukami, Fukagata, and Taira (2023) K. Fukami, K. Fukagata, and K. Taira, “Super-resolution analysis via machine learning: A survey for fluid flows,” Theoretical and Computational Fluid Dynamics (2023), 10.1007/s00162-023-00663-0.
  * Fukami, Fukagata, and Taira (2019) K. Fukami, K. Fukagata, and K. Taira, “Super-resolution reconstruction of turbulent flows with machine learning,” Journal of Fluid Mechanics 870, 106–120 (2019).
  * Liu  _et al._ (2020) B. Liu, J. Tang, H. Huang, and X.-Y. Lu, “Deep learning methods for super-resolution reconstruction of turbulent flows,” Physics of Fluids 32, 025105 (2020).
  * Fukami, Fukagata, and Taira (2021) K. Fukami, K. Fukagata, and K. Taira, “Machine-learning-based spatio-temporal super resolution reconstruction of turbulent flows,” Journal of Fluid Mechanics 909 (2021), 10.1017/jfm.2020.948.
  * Xie  _et al._ (2018) Y. Xie, E. Franz, M. Chu, and N. Thuerey, “tempoGAN: A temporally coherent, volumetric GAN for super-resolution fluid flow,” ACM Transactions on Graphics 37, 95:1–95:15 (2018).
  * Deng  _et al._ (2019) Z. Deng, C. He, Y. Liu, and K. C. Kim, “Super-resolution reconstruction of turbulent velocity fields using a generative adversarial network-based artificial intelligence framework,” Physics of Fluids 31, 125111 (2019).
  * Stengel  _et al._ (2020) K. Stengel, A. Glaws, D. Hettinger, and R. N. King, “Adversarial super-resolution of climatological wind and solar data,” Proceedings of the National Academy of Sciences 117, 16805–16815 (2020).
  * Yousif, Yu, and Lim (2021) M. Z. Yousif, L. Yu, and H.-C. Lim, “High-fidelity reconstruction of turbulent flow from spatially limited data using enhanced super-resolution generative adversarial network,” Physics of Fluids 33, 125119 (2021).
  * Hassanaly  _et al._ (2022) M. Hassanaly, A. Glaws, K. Stengel, and R. N. King, “Adversarial sampling of unknown and high-dimensional conditional distributions,” Journal of Computational Physics 450, 110853 (2022).
  * Wang  _et al._ (2022) Z. Wang, X. Li, L. Liu, X. Wu, P. Hao, X. Zhang, and F. He, “Deep-learning-based super-resolution reconstruction of high-speed imaging in fluids,” Physics of Fluids 34, 037107 (2022).
  * Buzzicotti  _et al._ (2021) M. Buzzicotti, F. Bonaccorso, P. C. Di Leoni, and L. Biferale, “Reconstruction of turbulent data with deep generative models for semantic inpainting from TURB-Rot database,” Physical Review Fluids 6, 050503 (2021).
  * Gundersen  _et al._ (2021) K. Gundersen, A. Oleynik, N. Blaser, and G. Alendal, “Semi-conditional variational auto-encoder for flow reconstruction and uncertainty quantification from limited observations,” Physics of Fluids 33, 017119 (2021).
  * Cai  _et al._ (2021) S. Cai, Z. Wang, F. Fuest, Y. J. Jeon, C. Gray, and G. E. Karniadakis, “Flow over an espresso cup: Inferring 3-D velocity and pressure fields from tomographic background oriented Schlieren via physics-informed neural networks,” Journal of Fluid Mechanics 915, A102 (2021).
  * Clark Di Leoni  _et al._ (2023a) P. Clark Di Leoni, K. Agarwal, T. A. Zaki, C. Meneveau, and J. Katz, “Reconstructing turbulent velocity and pressure fields from under-resolved noisy particle tracks using physics-informed neural networks,” Experiments in Fluids 64, 95 (2023a).
  * Du, Wang, and Zaki (2022) Y. Du, M. Wang, and T. A. Zaki, “State estimation in minimal turbulent channel flow: A comparative study of 4DVar and PINN,” (2022), arxiv:2210.09424 [physics] .
  * Clark Di Leoni  _et al._ (2023b) P. Clark Di Leoni, L. Lu, C. Meneveau, G. E. Karniadakis, and T. A. Zaki, “Neural operator prediction of linear instability waves in high-speed boundary layers,” Journal of Computational Physics 474, 111793 (2023b).
  * Yousif  _et al._ (2023) M. Z. Yousif, L. Yu, S. Hoyas, R. Vinuesa, and H. Lim, “A deep-learning approach for reconstructing 3D turbulent flows from 2D observation data,” Scientific Reports 13, 2529 (2023).
  * Li  _et al._ (2023) Z. Li, H. Zheng, N. Kovachki, D. Jin, H. Chen, B. Liu, K. Azizzadenesheli, and A. Anandkumar, “Physics-Informed Neural Operator for Learning Partial Differential Equations,” (2023), arxiv:2111.03794 [cs, math] .
  * Wilks (2019) D. S. Wilks, “Statistical Methods in the Atmospheric Sciences,” in  _Statistical Methods in the Atmospheric Sciences (Fourth Edition)_ (Elsevier, 2019) pp. i–ii.
  * Salimans  _et al._ (2016) T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, X. Chen, and X. Chen, “Improved Techniques for Training GANs,” in  _Advances in Neural Information Processing Systems_ , Vol. 29 (Curran Associates, Inc., 2016).
  * Sohl-Dickstein  _et al._ (2015) J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep Unsupervised Learning using Nonequilibrium Thermodynamics,” in  _Proceedings of the 32nd International Conference on Machine Learning_ (PMLR, 2015) pp. 2256–2265.
  * Song and Ermon (2020) Y. Song and S. Ermon, “Generative Modeling by Estimating Gradients of the Data Distribution,”  (2020), arxiv:1907.05600 [cs, stat] .
  * Dhariwal and Nichol (2021) P. Dhariwal and A. Nichol, “Diffusion Models Beat GANs on Image Synthesis,” in  _Advances in Neural Information Processing Systems_ , Vol. 34 (Curran Associates, Inc., 2021) pp. 8780–8794.
  * Rombach  _et al._ (2022) R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,”  (2022), arxiv:2112.10752 [cs] .
  * Pinaya  _et al._ (2022) W. H. L. Pinaya, P.-D. Tudosiu, J. Dafflon, P. F. da Costa, V. Fernandez, P. Nachev, S. Ourselin, and M. J. Cardoso, “Brain Imaging Generation with Latent Diffusion Models,”  (2022), arxiv:2209.07162 [cs, eess, q-bio] .
  * Lugmayr  _et al._ (2022) A. Lugmayr, M. Danelljan, A. Romero, F. Yu, R. Timofte, and L. Van Gool, “RePaint: Inpainting Using Denoising Diffusion Probabilistic Models,” in  _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ (2022) pp. 11461–11471.
  * Saharia  _et al._ (2022) C. Saharia, W. Chan, H. Chang, C. A. Lee, J. Ho, T. Salimans, D. J. Fleet, and M. Norouzi, “Palette: Image-to-Image Diffusion Models,” (2022), arxiv:2111.05826 [cs] .
  * Courtier, Thépaut, and Hollingsworth (1994) P. Courtier, J.-N. Thépaut, and A. Hollingsworth, “A strategy for operational implementation of 4D-Var, using an incremental approach,” Quarterly Journal of the Royal Meteorological Society 120, 1367–1387 (1994).
  * Evensen (1994) G. Evensen, “Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics,” Journal of Geophysical Research: Oceans 99, 10143–10162 (1994).
  * Sprague (2021) M. A. Sprague, “ExaWind: Predictive wind energy simulations,” Tech. Rep. (National Renewable Energy Lab.(NREL), Golden, CO (United States), 2021).
  * Almgren  _et al._ (1998) A. S. Almgren, J. B. Bell, P. Colella, L. H. Howell, and M. L. Welcome, “A Conservative Adaptive Projection Method for the Variable Density Incompressible Navier–Stokes Equations,” Journal of Computational Physics 142, 1–46 (1998).
  * Smagorinsky (1963) J. Smagorinsky, “GENERAL CIRCULATION EXPERIMENTS WITH THE PRIMITIVE EQUATIONS: I. THE BASIC EXPERIMENT,” Monthly Weather Review 91, 99–164 (1963).
  * Luo (2022) C. Luo, “Understanding Diffusion Models: A Unified Perspective,”  (2022), arxiv:2208.11970 [cs] .
  * Song  _et al._ (2021) Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-Based Generative Modeling through Stochastic Differential Equations,”  (2021), arxiv:2011.13456 [cs, stat] .
  * Kawar  _et al._ (2022) B. Kawar, M. Elad, S. Ermon, and J. Song, “Denoising Diffusion Restoration Models,”  (2022), 10.48550/arXiv.2201.11793.
  * Kingma and Welling (2013) D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,”  (2013), 10.48550/arXiv.1312.6114.
  * Zhang  _et al._ (2018) R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,”  (2018), arxiv:1801.03924 [cs] .
  * Isola  _et al._ (2018) P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks,”  (2018), arxiv:1611.07004 [cs] .
  * Brock, Donahue, and Simonyan (2019) A. Brock, J. Donahue, and K. Simonyan, “Large Scale GAN Training for High Fidelity Natural Image Synthesis,”  (2019), arxiv:1809.11096 [cs, stat] .
  * Wu and He (2018) Y. Wu and K. He, “Group Normalization,”  (2018), arxiv:1803.08494 [cs] .
  * Ulyanov, Vedaldi, and Lempitsky (2017) D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Instance Normalization: The Missing Ingredient for Fast Stylization,”  (2017), arxiv:1607.08022 [cs] .
  * Carrassi  _et al._ (2018) A. Carrassi, M. Bocquet, L. Bertino, and G. Evensen, “Data assimilation in the geosciences: An overview of methods, issues, and perspectives,” WIREs Climate Change 9, e535 (2018).
  * He  _et al._ (2016) K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in  _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ (2016) pp. 770–778.
  * Ronneberger, Fischer, and Brox (2015) O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,”  (2015), arxiv:1505.04597 [cs] .
  * Leinonen  _et al._ (2023) J. Leinonen, U. Hamann, D. Nerini, U. Germann, and G. Franch, “Latent diffusion models for generative precipitation nowcasting with accurate uncertainty quantification,”  (2023), arxiv:2304.12891 [physics] .
  * Rahaman  _et al._ (2019) N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, and A. Courville, “On the Spectral Bias of Neural Networks,” in  _Proceedings of the 36th International Conference on Machine Learning_ (PMLR, 2019) pp. 5301–5310.
  * Hamill (2001) T. M. Hamill, “Interpretation of Rank Histograms for Verifying Ensemble Forecasts,” Monthly Weather Review 129, 550–560 (2001).
  * Monache  _et al._ (2013) L. D. Monache, F. A. Eckel, D. L. Rife, B. Nagarajan, and K. Searight, “Probabilistic Weather Prediction with an Analog Ensemble,” Monthly Weather Review 141, 3498–3516 (2013).
  * Ravuri  _et al._ (2021) S. Ravuri, K. Lenc, M. Willson, D. Kangin, R. Lam, P. Mirowski, M. Fitzsimons, M. Athanassiadou, S. Kashem, S. Madge, R. Prudden, A. Mandhane, A. Clark, A. Brock, K. Simonyan, R. Hadsell, N. Robinson, E. Clancy, A. Arribas, and S. Mohamed, “Skilful precipitation nowcasting using deep generative models of radar,” Nature 597, 672–677 (2021).
  * Raissi, Perdikaris, and Karniadakis (2019) M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” Journal of Computational Physics 378, 686–707 (2019).
  * Meng  _et al._ (2022) C. Meng, R. Rombach, R. Gao, D. P. Kingma, S. Ermon, J. Ho, and T. Salimans, “On Distillation of Guided Diffusion Models,”  (2022), arxiv:2210.03142 [cs] .
  * Hunter (2007) J. D. Hunter, “Matplotlib: A 2D Graphics Environment,” Computing in Science Engineering 9, 90–95 (2007).
  * Jeong and Hussain (1995) J. Jeong and F. Hussain, “On the identification of a vortex,” Journal of Fluid Mechanics 285, 69–94 (1995).
  * Papoulis and Unnikrishna Pillai (2002) A. Papoulis and S. Unnikrishna Pillai,  _Probability, Random Variables and Stochastic Processes_ (2002).
  * Ledig  _et al._ (2017) C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, “Photo-realistic single image super-resolution using a generative adversarial network,” in  _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ (2017).