††thanks:

# Artificial neural network-based reduced-order modeling for turbulent wake of
a finite wall-mounted square cylinder

Mustafa Z. Yousif  School of Mechanical Engineering, Pusan National
University, 2, Busandaehak-ro 63beon-gil, Geumjeong-gu, Busan, 46241, Rep. of
KOREA  Hee Chang Lim  Corresponding author<EMAIL_ADDRESS>School of
Mechanical Engineering, Pusan National University, 2, Busandaehak-ro 63beon-
gil, Geumjeong-gu, Busan, 46241, Rep. of KOREA

###### Abstract

This study presents an artificial neural network and proper orthogonal
decomposition (POD)-based reduced-order model (ROM) of turbulent flow around a
finite wall-mounted square cylinder. The proposed model is suitable for
turbulent wake control applications because it can predict the dynamics of the
main features of the flow field without computing Navier-Stokes equations.
Long short-term memory neural network (LSTM NN) and bidirectional long short-
term memory neural network (BLSTM NN) are used to predict the temporal
evolution of the POD time coefficients at different planes along the height of
the obstacle. The improved delayed detached-eddy simulation (IDDES) is
performed to generate the training datasets. Transfer learning (TL) approach
is utilized in the training process by using the weights of the LSTM/BLSTM NN
that are used to predict the POD time coefficients of the planes at lower
elevations to initialize the weights of the networks at higher elevations
along the height of the obstacle. The use of TL results in a remarkable
improvement in the capability of the LSTM/BLSTM NN prediction compared with
the one when the network is initialized with random weights. BLSTM NN shows
better results compared with LSTM NN in terms of training and prediction
error, indicating that the BLSTM-POD model is more suitable to be used as a
ROM for predicting the turbulent wake. Furthermore, the temporal behavior of
the time coefficients is carefully examined using the phase space plots and
Poincar$\acute{e}$ sections. The results of using different lengths of the
prediction time window showed that the prediction error of the POD time
coefficients increases as the prediction time window increases and the error
increasing rate decreases with the ranking of the POD time coefficients.

††preprint: AIP/123-QED

##  I Introduction

The study of wake dynamics behind finite wall-mounted cylinders (FWMCs) has
received considerable attention over the past few decades. From skyscrapers,
chimneys, and submarine appendages to electronic chips, many engineering
structures can be approximated as FWMCs. The study of the wakes behind the
finite wall-mounted obstacles helps understand and control the flows around
the engineering structures. However, this type of flows has a complex behavior
as it is three-dimensional, this is different from the the case of infinite-
length cylinder where the flow is mostly two-dimensional [1, 2, 3]. For the
FWMC, the flow is resulted from the effect of the downwash from the free-end,
the von K$\acute{a}$rm$\acute{a}$n spanwise vortex shedding, and the base
upwash[4, 55].

This complex nature of the flow requires an appropriate experimental setup and
a numerical model that can accurately capture most of the flow-field features
and provide high-fidelity data. The recent advances in experimental and
computational techniques have helped in visualizing and analyzing the flow
structure around the obstacles and helped obtain enormous amounts of data for
instance, from practical image velocimetry (PIV) and direct numerical
simulation (DNS) studies. Nevertheless, understanding the main features of the
flow field based on such big data is difficult due to the complexity of the
flow behavior as well as the existence of redundant data. Furthermore,
building an optimum model that has a low computational cost is an important
factor in turbulent flow-control applications [5]. For the finite wall-mounted
obstacles, building an analytical low-order model that can describe the
shedding process is often not applicable because the shedding process changes
with the obstacle length and the flow becomes three-dimensional as it
approaches the free-end due to the downwash effect [6, 7].

Data-driven methods such as proper orthogonal decomposition (POD) [8] and
dynamic mode decomposition (DMD) [9] have been widely used in fluid dynamics
for ROM. In these methods, the flow fields are mapped to a low-dimensional
space that is represented by an optimal set of modes. In POD, these modes are
based on the energy content of the flow, while in DMD, they are related to the
associated frequencies. The traditional way of modeling the dynamics of the
flow in these methods is to perform the Galerkin projection approach [10, 11,
12]. The Galerkin projection method uses the spatio-temporal dynamics that are
captured from POD or DMD to build a low-order model that can evolve over time
instead of the full-order Navier-Stokes equations. This can result in a
cheaper computation because the low-order model contains considerably fewer
degrees of freedom [13]. However, the Galerkin projection approach exhibits an
unstable behavior, even for simple canonical cases [14, 15]. Furthermore, this
approach still requires effort in terms of the computational model setup.
These issues led to the development of other methods [16]. However, for
complex turbulent flow problems, an accurate approach is required to capture
the non-linear dynamics of the flow.

machine learning learning has shown considerable promise in modeling complex
and non-linear systems in diverse fields like image processing, robotics,
weather forecasting, etc. In recent years, machine learning has been used
extensively in fluid dynamics for experimental data processing, turbulence
closure modeling, data-driven control as well as reduced-order modeling [18,
19]. For reduced-order modeling, promising results have been obtained for the
prediction of turbulent flows using various machine learning algorithms that
based on artificial neural networks such as multi-layer perceptrons (MLPs)
(also known as fully-connected-layer neural networks), convolution neural
networks (CNNs), and recurrent neural networks (RNNs) including the long-
short-term memory neural networks (LSTM NNs).

It has been established that the results obtained from a fully connected-layer
neural network with a single hidden layer and linear activation function are
equivalent to that obtained from a POD [20, 21]. This shows that the neural
networks are based on data-driven training and can be compared and used with
other traditional data-driven methods. Milano and Koumoutsakos [22] showed
that by using POD-based neural networks with non-linear activation functions,
the prediction of the near-wall velocity field is improved significantly,
indicating better predictive capabilities of the neural networks.

G$\ddot{u}$emes et al. [23] used CNN to predict the POD time coefficients from
streamwise wall-shear stress measurements that were used to build a reduced-
order reconstruction of the velocity fields. They recorded that CNN
significantly outperformed the extended POD (EPOD), which is considered a
linear method. Murata et al. [24] used a CNN auto-encoder and LSTM NN to
obtain a low-order representation of flow around bluff bodies of various
shapes. Their instantaneous results, as well as statistical calculations,
showed that the flow fields conformed to be well reproduced. Motivated by the
capabilities of LSTM NNs in modeling sequential data, Z. Wang et al. [25] used
LSTM NN to build a POD-based reduced-order model of an ocean gyre and flow
past a cylinder by modeling the POD time coefficients. They obtained a
significant reduction in the computational cost with reasonable predictive
accuracy. Mohan and Gaitonde [26] applied LSTM and bidirectional long-short
term (BLSTM) NNs to predict the POD time coefficients that were used to build
a reduced-order model of canonical DNS datasets. In their study, LSTM NN
outperformed BLSTM NN, suggesting that BLSTM NN is more suitable for the flow
with periodic or quasi-periodic temporal behavior such as two-dimensional
bluff body wake shedding.

In this study, a deep neural network-based approach to build a reduced-order
model using LSTM and BLSTM NNs combined with POD is applied to datasets
obtained from an improved delayed detached-eddy simulation (IDDES) of flow
past a finite wall-mounted square cylinder with an aspect ratio of four at
Reynolds number $Re_{D}=12000$. The objective of this approach is to
investigate the applicability of the LSTM-POD and BLSTM-POD models for
predicting the instantaneous coherent structure of the flow field along the
height of the obstacle.

The remainder of this paper is organized as follows: In Section II, the case
geometry and the numerical model used to generate the training data are
explained, and the simulation results are validated with the available
experimental and DNS data. POD is briefly explained in Section III. Section IV
provides a brief introduction to LSTM and BLSTM NNs while Section V explains
TL approach used in this study. The methodology of building LSTM-POD/BLSTM-POD
ROM is explained in Section VI. The results of this study are thoroughly
discussed in Section VII. Finally, the conclusions of this study are presented
in Section VIII.

##  II Preparing the training data

###  II.1 Case geometry and boundary conditions

The choice of a finite-wall mounted square cylinder with an aspect ratio of
four in this study is because it is well studied experimentally [1] and
numerically [4]. Furthermore, the sharp leading edges of the square cylinder
ensure that the shedding process is not affected by the change in $Re_{D}$, at
least over the range tested by Sattari et al. [3]. The schematic diagram of
the numerical domain for the finite wall-mounted square cylinder is shown in
Fig. 1.

Figure 1: Schematic of the computational domain.

The square cylinder has a height $H=4D$, where $D$ is its width, and
$Re_{D}=12,000$ ( based on the cylinder width and free stream velocity
$U_{\infty}$). The size of the domain is equal to $29D\times 18D\times 15D$.
Therefore, the blockage ratio is found to be approximately 1.5%, which is
lower than the maximum value of 3% recommended by Franke et al. (Franke et
al., 2004). The obstacle is located at a distance of $8D$ downstream from the
inlet and $20D$ upstream from the outlet. The distance of the obstacle from
each side of the domain is set to $8.5D$ while the distance from the end of
the obstacle to the top surface of the domain is set to $11D$. In turbulent
flows simulations, applying turbulent inflow conditions is crucial for the
simulation reliability. Thus, the vortex method [27, 28] is implemented in
this study to generate unsteady turbulent inflow conditions. Pressure outlet
boundary condition is assigned for the outlet plane of the domain, and
periodic boundary condition is applied to the spanwise direction. No-slip
condition is applied to the obstacle walls and the ground, while a symmetry
plane is applied to the top surface of the computational domain.

###  II.2 Governing equations and turbulence modeling

The momentum equation for an incompressible viscous fluid is:

$\left(\frac{\partial\bf{u}}{\partial
t}+\bf{u}\cdot\nabla\bf{u}\right)=-\frac{1}{\rho}\nabla
p+\nu\nabla^{2}\bf{u},$ (1)

where ${\bf u}$ is the velocity of the fluid, $\rho$ is the density, $p$ is
the pressure, and $\nu$ is the kinematic viscosity. The incompressibility of
the fluid is expressed by the continuity equation:

$\nabla\cdot\bf{u}=0.$ (2)

Different numerical approaches have been used to solve and model the flow
parameters in Eqs. (1) and (2). DNS resolves all scales of turbulent flow
structures, from the large energy-containing scales to the smallest Kolmogorov
length scale, in the dissipation range [29]. To ensure this, an extremely fine
grid size is required that results in a high computational cost, thus making
DNS impractical for several applications, especially those with a high $Re$
value and complex geometries [30]. The Reynolds-Averaged Navier-Stokes (RANS)
equations solve only the time-averaged flow field, i.e., the effects of the
fluctuating components are considered through the modeled Reynolds stresses.
Thus, the RANS model is not suitable to describe the details of the flow field
such as the instantaneous effect of multi-scale eddies [9, 31].

In LES, Kolmogorov’s first hypothesis is considered, i.e., the small eddies in
the turbulent flow tend to exhibit universal behavior. This property allows
the modeling of small eddies and solving for the large eddies [32, 33]. This
process can be performed using spatial filters to separate the large scales
from the small ones. The small-scale structures (small eddies) are modeled and
introduced as a subgrid-scale (SGS) term in the filtered Navier–Stokes
equations. Thus, LES is considerably more accurate as compared to RANS for
describing the flow characteristics. Nevertheless, it requires high
computational effort, which makes it impractical to simulate flow around
engineering structures at high $Re$ values [34]. Hybrid RANS-LES models, such
as the DES, have been introduced by Spalart and co-authors [35, 36, 37] to
bridge the gap between LES and RANS. DES has been designed to simulate the
flow at a high $Re$ value and to flow around bodies with massive flow
separation.

By applying RANS near the solid walls within the attached boundary layer and
using LES outside the boundary layer in the separated flow region, the grid
size for numerical simulations can be reduced significantly in the near-wall
region, compared to that required by LES. DES is a promising turbulence
modeling tool that can be applied for simulations including wall-bounded flows
at high $Re$ values. However, some undesirable phenomena such as modeled
stress depletion and grid-induced separation may occur and affect the accuracy
of the model [38]. This could be attributed to the mesh resolution inside the
boundary layer that may result in DES attempting LES rather than RANS in the
near-wall region. To overcome these limitations of DES, variant versions
including delayed DES (DDES) [39] and IDDES [40] have been introduced that are
suitable for different grid spacing inside the boundary layer regardless of
the boundary layer thickness. Therefore, considering the relatively high
Reynolds number used in this study and the complex flow that is expected to be
obtained from the simulation, the Spalart–Allmaras IDDES (S-A IDDES) is used
in this study to simulate the flow.

###  II.3 Numerical framework

The computational fluid dynamics (CFD) finite-volume code OpenFOAM-5.0x is
used to perform the simulation. A structured hexahedral mesh is used for
spatial discretization. Local mesh refinement is applied using the stretching
mesh technique in regions where high-velocity gradients are expected. Grid-
independence tests are performed using three different grid sizes: $132\times
122\times 90$, $152\times 142\times 110$, and $172\times 162\times 130$. The
overall time-averaged drag coefficient and Strouhal number are chosen as the
parameters for the test. As shown in Table 1, less than 1% variation is
observed in the drag coefficient while there is no variation in Strouhal
number with the progressive refinement of the mesh, ultimately converging with
the grid-independence test. Considering the maximum dimensionless first cell
spacing $y^{+}$, the finest mesh size of $172\times 162\times 130$ is chosen
for this study.

No of grid elements |  $N_{x}\times N_{y}\times N_{z}$ 111$N_{x},N_{y}$ , and $N_{z}$ represent the number of grid points distributed on the walls of the obstacale. | Overall drag, $C_{D}$ | Strouhal No., $St$
---|---|---|---
$132\times 122\times 90$ | $22\times 22\times 40$ | 1.212 | 0.101
$152\times 142\times 110$ | $22\times 22\times 50$ | 1.238 | 0.104
$172\times 162\times 130$ | $22\times 22\times 60$ | 1.241 | 0.104
Table 1: Grid independence test.

Figure 2 shows the distribution of $y^{+}$ along the ground surfaces and
around the obstacle surfaces in the central $x-z$ plane ($y/D=0$). The maximum
value of $y^{+}$ is less than 0.5 on the ground while it is approximately 15
on the obstacle. These values of $y^{+}$ are within the applicable range for
IDDES [38]. Advancement is done through a time-step of $6\times 10^{-6}$s
while maintaining the mean Courant number (CFL) at 0.52 and the maximum CFL at
0.95 to ensure a stable simulation. The approximate time required for every
vortex shedding cycle is $8.5\times 10^{-3}$s. To ensure that the flow reached
a statistically stationary state, the simulation is first extended over 30
shedding cycles. Subsequently, the flow statistics are collected through
approximately 60 shedding cycles. The PIMPLE algorithm, which is a combination
of the PISO algorithm (pressure implicit split operator) and the SIMPLE
algorithm (semi-implicit method for pressure-linked equations), is employed to
solve the coupled pressure momentum system. The convective fluxes are
discretized with a second-order-accurate linear upwind scheme, and all other
discretization schemes used in the simulation have second-order accuracy. To
perform parallel computing, the computational domain is divided into 80 sub-
domains and Scotch method is used to decompose the domains. Correspondingly,
80 processors were used to solve the flow field at each time-step. All the
computations are performed using a computer cluster with Intel Xeon Skylake
(Gold 6148) 2.4 GHz processor.

Figure 2: Distribution of $y^{+}$ in the central $x-z$ plane ($y/D=0$): (a)
along the ground surface, and (b) around the obstacle surfaces.

###  II.4 Validation of the numerical results

To ensure that the simulation produced reliable training data, the first and
second-order flow statistics obtained from the simulation are compared with
the available experimental measurements of Bourgeois et al. [1] and the DNS
data of Saeedi et al [4]. Figure 3 shows the recovery of the mean streamwise
velocity, $U$ in the central $x-z$ plane ($y/D=0$) for two different
elevations behind the obstacle, $z/D=$ 1 and 3. For both elevations, the
velocity has negative values near the wall due to the reverse flow as a result
of recirculation. For $x/D\approx$ 3, the velocity value becomes positive and
the flow recovers to the free stream velocity with a higher rate at elevation
$z/D$ = 3. Both Figs. 3(a) and (b) correlate with the experimental and DNS
results.

Figure 3: Streamwise profile of the mean streamwise velocity at two different
elevations in the central $x-z$ plane ($y/D=0$): (a) $z/D=1$, and (b) $z/D=3$.

Figure 4 shows the streamwise profile of the turbulence intensity, $u_{rms}$
obtained from the simulation, experimental measurements, and DNS data in the
central $x-z$ plane ($y/D$ = 0) for two different elevations $z/D$ = 1 and 3.
The values of turbulent intensity have been well reproduced at the near-wall
peak region for both elevations. However, under-prediction of the values in
the far downstream of the obstacle at an elevation $z/D=3$ can be observed in
Fig. 4(b). This is attributed to the turbulent level attenuation in this
region, which demands more computational effort.

Figure 4: Streamwise profile of the turbulence intensity at two different
elevations in the central $x-z$ plane ($y/D=0$): (a) $z/D=1$, and (b) $z/D=3$.

The mean streamwise velocity profile along the lateral direction at elevation
$z/D=3$ and streamwise location, $x/D=2$ is shown in Fig. 5 (a). The
streamwise velocity values become negative at $y/D$ between -0.5 and 0.5
indicating the effect of recirculation inside the wake. The velocity profile
obtained from the numerical simulation complies with the experimental
measurements and DNS results. However, the streamwise velocity values are
slightly over-predicted in the recirculation region. Figure 5 (b) compares the
numerical results with the experimental measurements and DNS results of the
mean Reynolds stress component $\overline{u^{\prime}\upsilon^{\prime}}$
($\upsilon^{\prime}$ is the fluctuating part of the spanwise velocity
component) at an elevation $z/D=3$ along the lateral direction and at the
streamwise location $x/D=2$. As shown in the figure, the mean Reynolds stress
profile has maximum negative and positive values at $y/D=\pm 1$ that decrease
rapidly in the lateral direction indicating that the shear stress accrues due
to the combination of the vortices induced behind the obstacle. A favorable
agreement with the experimental and DNS results can be observed from the
figure. Here, the S-A IDDES model successfully reproduces the flow field
around and behind the obstacle with commendable accuracy. This is attributed
to the turbulent inflow conditions obtained from the vortex method and the
small blockage ratio used in this study.

Figure 5: Spanwise profile of (a) the mean streamwise velocity, and (b) mean
Reynolds stress component $\overline{u^{\prime}\upsilon^{\prime}}$ at
elevation $z/D=3$ and streamwise location $x/D=2$.

##  III Proper orthogonal decomposition

POD [8] is widely used in fluid dynamics to extract the main features of the
flow field, which is known in literature as the coherent structure [41]. POD
decomposes the fluctuating parts of the velocity components into spatial
orthogonal modes $\psi_{m}$ and corresponding time coefficients $a_{m}(t)$.
The extracted modes represent the most coherent structure of the flow field
according to the energy content of the modes. In this study, POD was applied
to different $x-y$ planes along the obstacle height.

$u^{\prime}(x,y,t)=\Sigma_{m=1}^{M}a_{m}(t)\phi_{m}(x,y),$ (3)

where $u^{\prime}$ here represents the fluctuating part of the streamwise
velocity component and $M$ is the number of the POD modes. The snapshot POD
method [42] was used in this study, which is based on computing the auto-
correlation from numerous snapshots of the instantaneous flow field. The
temporal modes are the eigenvectors that can be obtained by solving the
eigenvalue problem of the auto-correlation matrix,

$C=VV^{T},$ (4)

where $V$ is a matrix containing the fluctuating part of the velocity:

$V=\begin{bmatrix}{u^{\prime}}_{1}^{1}&\cdots&{u^{\prime}}_{1}^{N}\\\
\vdots&\ddots&\vdots\\\ {u^{\prime}}_{M}^{1}&\cdots&{u^{\prime}}_{M}^{N}\\\
\end{bmatrix},$ (5)

where $N$ represents the position in each snapshot (instantaneous) data of the
flow. The corresponding sets of $M$ eigenvalues $\lambda_{m}$ represent the
contribution to the turbulent kinetic energy captured by the respective POD
modes. The eigenvalues are arranged in decreasing order such that:

$\lambda_{1}\geq\lambda_{2}\cdots\geq\lambda_{M}\geq 0.$ (6)

##  IV Long short-term memory neural networks

LSTM NN [43] has been widely used in different fields such as speech
recognition, language translation, as well as time-series prediction. LSTM NN
is a special variant of RNNs [44], which is designed to overcome the stability
issues and limitations of traditional RNNs such as blowing up or vanishing
gradient [45]. Because of its architecture, LSTM NN has a great ability to
learn the temporal dependence from the data. These unique properties make LSTM
NN suitable for time-series problems. The architecture of LSTM NN is shown in
Fig. 6. The hidden layer in LSTM NN receives an input vector that contains the
sequential data x and generates an output vector $h$, where $h$ represents the
hidden state and $t$ is the step of the sequential data (a particular time-
step of the time series data). The hidden layer maintains the hidden state
with every iteration in the LSTM cell and updates it based on the cell input
and the information from the previous time steps. This information is
represented by the previous hidden state and the cell state ($C$). The cell
state plays a crucial role in reducing the effect of short-term memory by
carrying the relevant information throughout the processing of the sequence.

Figure 6: LSTM NN Architecture.

Figure 7 shows the basic architecture of the LSTM cell. LSTM cell contains
three gates namely, input gate ($i$), output gate ($o$), and forget gate
($f$). The LSTM cell changes the flow of training information according to
these gates. The input gate controls the flow of the input information into
the LSTM cell. The forget gate filters the information from the previous time
steps by deciding which information should be discarded or retained. The
output gate controls the flow of the LSTM cell output. The equations that
compute the output of the gates can be represented as follows:

Figure 7: LSTM cell Architecture.

$i_{t}=\sigma\left(W_{i}x_{t}+U_{i}h_{t-1}+b_{i}\right),$ (7)

$f_{t}=\sigma\left(W_{f}x_{t}+U_{f}h_{t-1}+b_{f}\right),$ (8)

$\tilde{C}_{t}=tanh\left(W_{C}x_{t}+U_{C}h_{t-1}+b_{C}\right),$ (9)

$C_{t}=i_{t}\otimes\tilde{C}_{t}+f_{t}\otimes C_{t-1},$ (10)

$o_{t}=\sigma\left(W_{o}x_{t}+U_{o}h_{t-1}+b_{o}\right),$ (11)

$h_{t}=o_{t}\otimes tanh({C}_{t}),$ (12)

where $W_{i}$, $W_{f}$, $W_{C}$ and $W_{o}$ represent the weights that map the
input to each of the gates and $U_{i}$, $U_{f}$, $U_{C}$ and $U_{o}$ are the
weights related to the hidden state of the cell in the previous time step.
Here, $\tilde{C}_{t}$ here represents the updated cell state. BLSTM NN [46,
47] is a variant of LSTM NN that processes sequential data using two separate
hidden layers in both forward and backward directions and connect both of them
to the same output layer, as shown in Fig. 8 . This architecture of BLSTM NN
enables the two-way flow of the information so that the sequence can be
trained in both forward and backward directions. BLSTM NN has shown an
improved accuracy comparing with LSTM NN in speech recognition and language
modeling [48, 49]. To compare the accuracy of these two networks, both LSTM
and BLSTM NNs are used in this study to predict the time coefficients of POD
modes for different $x-y$ planes along the height of the obstacle.

Figure 8: BLSTM NN Architecture.

##  V Transfer learning

The availability of sufficient training data and the time required for the
training process are factors that should be considered for training neural
networks. TL approach [50] has been extensively used in different fields and
showed promising results in terms of reducing the data and the time required
for the training process. This method enables the reuse of pretrained network
parameters to train another neural network that shares some features with the
pretrained one even if it is designed for a different purpose than the
pretrained network is designed for. Hence, TL helps to achieve optimum use of
the available training data as well as the reduction of the training time
compared with the network if it is trained from scratch, i.e., with randomly
initialized parameters. In this study, TL is used to train LSTM/BLSTM NNs by
transferring the weights from the networks that trained by using the data of
the lower elevation flow field to the higher elevation one. LSTM/BLSTM NNs are
trained using the data of the lower elevation flow field, $z/D=0.02$ and used
to initialize the weights in the training process of the network that predicts
the POD time coefficients at the next $x-y$ plane with higher elevation. The
weights are transferred to the network that is used for the subsequent
elevation, and so on.

##  VI Methodology

The main objective of this study is to build ROM based on POD modes and the
temporal evolution of the time coefficients that can be obtained using LSTM or
BLSTM NN. To provide a dataset that is adequate for performing the POD and
training the LSTM and BLSTM NNs, four $x-y$ planes along the obstacle height
are selected according to the expected behavior of the turbulent wake and 50
000 snapshots that are obtained from the IDDES, which are equivalent to 36
shedding cycles. The time coefficients obtained from the POD analysis are then
used to train the LSTM and BLSTM NNs. The procedure that followed in this
study is outlined in Fig. 9 and is expressed as follows,

Figure 9: Schematic of the LSTM/BLSTM-POD ROM.

  1. 1.

POD analysis is performed on the datasets to extract the dominant POD modes
and the corresponding time coefficients. The time coefficients are then used
as datasets for LSTM and BLSTM NNs with 80% of them as training data and 20%
as test data. Here the test datasets are used to validate the predicted time
coefficients that are obtained from LSTM and BLSTM NNs.

  2. 2.

LSTM and BLSTM NNs are trained using the training datasets and validated by
comparing the predicted time coefficients of the future time-steps with the
test datasets obtained from the POD analysis at the same time-steps.

  3. 3.

The predicted flow field is then reconstructed using POD modes and the
predicted time coefficients obtained from LSTM or BLSTM NN by using Eq. (3).

The training datasets are normalized using the min-max normalization,

$x_{m}(t)=\frac{a_{m}(t)-min(a_{m}(t))}{max(a_{m}(t))-min(a_{m}(t))},$ (13)

and decomposed into a series of short-time data samples and each sample is
advanced by one time-step from the previous one. Each sample is used to
predict the next time-steps that have the same time window length as the input
sample. For instance, if a sample has a length of 10 time-steps, the model is
trained to predict the time window with the length of the next 10 time-steps.

One LSTM/BLSTM layer with 500 units is used with one fully connected layer
that has the number of units equal to the number of time coefficients in the
prediction time window to predict the final output of the NN. As mentioned
before, the TL approach is used to train the LSTM/BLSTM NN using the weights
of the network that are used to predict the time coefficients at the previous
$x-y$ plane starting from the lower elevation, i.e., at $z/D=0.02$. Here, the
learning rates for all the training processes are reduced by a factor of 20%
to prevent the optimizer from diverging too quickly from the weights that
transferred from the previous LSTM/BLSTM NN. Four LSTM/BLSTM-POD models are
used for every $x-y$ plane. Each model is trained with five sets of time
coefficients, so the total number of time coefficients sets is twenty, which
corresponds to the first twenty POD modes. All the weights are updated by
back-propagation through the iteration time. 1,000 epochs are used for the
training process and the adaptive moment estimation (Adam) optimizer [51] is
used for both LSTM and BLSTM NNs.

##  VII Results and discussion

###  VII.1 POD analysis

POD analysis was applied to four different $x-y$ planes along the height of
the obstacle, $z/D$ = 0.02, 1, 2, and 3 respectively. A total of 50,000
snapshots across $6\times 10^{-6}$s are used to perform POD at each elevation.
Here, the same small time-steps used in the simulation are used in the POD
analysis to capture most of the flow-field details that can enable better
prediction. Figure 10 shows the cumulative energy percentage of the
eigenvalues and the normalized eigenvalues of the first 20 POD modes at
different elevations along the obstacle length. Here, the eigenvalues
represent the contribution of the corresponding POD modes to the total
turbulent kinetic energy (TKE). As shown in Fig. 10(a), the maximum cumulative
energy obtained from the first 20 modes is at $z/D$ = 1 (where the flow shows
a quasi-periodic behavior) that is approximately 92% of the total turbulent
kinetic energy. However, as the periodic behavior becomes less at higher
elevations and near the ground, the cumulative energy becomes less and reaches
about 85% at $z/D$ = 3. Similarly, in Fig. 10(b), the fractional contribution
to the turbulent kinetic energy of the first two modes reaches 62% at $z/D$ =
1 and 38% at $z/D$ = 3. The normalized eigenvalues decrease smoothly for the
rest of the modes with much lower values compared to the eigenvalues that
corresponded to the first two modes.

Figure 10: Cumulative energy percentage of the eigenvalues (a), and the
normalized eigenvalues of the first 20 POD modes (b).

Figure 11 shows the contour plots for different POD modes of the streamwise
velocity field at three different elevations. The contour plot of the POD
modes at $z/D=2$ is not shown here because the behavior is the same as at
$z/D=1$. It can be seen clearly that from all the elevations along the
obstacle height, the first and second modes contain the most coherent large-
scale structure of the flow with clear alternative shedding behavior. However,
the other modes exhibit less periodic behavior with an increase in the random
behavior with the obstacle height. This change in behavior of the modes is
attributed to the downwash effect from the free end of the obstacle, which
tends to weaken the alternative shedding [7].

Figure 11: Contour plots for different POD modes of the streamwise velocity
field at three different elevations: (a) $z/D=0.02$, (b) $z/D=1$, and (c)
$z/D=3$.

Figure 12 shows the power spectrum density of different POD time coefficients
obtained from the POD analysis at different elevations along the obstacle
height. It can be seen from the figure that the time coefficients with the
same POD mode ranking have similar frequency content along the obstacle
height. This encourages us to perform TL according to the height of the
obstacle with the aim that the time coefficients could share some similar
features that can be transferred through the LSTM/BLSTM NN weights.

Figure 12: Power spectrum density of different POD time coefficients obtained
from the POD analysis at different elevations: (a) $a(t)_{1}$, (b) $a(t)_{2}$,
and (c) $a(t)_{3}$, (d) $a(t)_{20}$.

###  VII.2 TL and training error

In this section, the effect of TL on the training accuracy and the
capabilities of LSTM and BLSTM NNs are investigated. The LSTM/BLSTM NN is
trained using samples with different lengths. We consider for the illustration
here, a sample with a length of 10 time-steps. Table II shows the root-mean-
square error of the LSTM/BLSTM NN output. The root-mean-square error is
defined as:

$\text{RMSE}=\sqrt{\frac{\Sigma_{i=1}^{k}(\hat{y}_{i}-y_{i})^{2}}{k}},$ (14)

where $\hat{y}$ is the predicted value and $y$ is the actual value of the
normalized time coefficient obtained from the test dataset. $k$ is the number
of the time-steps.

| POD mode | $z/D=0.02$ | $z/d=1$ | $z/D=2$ | $z/D=3$
---|---|---|---|---|---
LSTM without | 1 | 0.0628 | 0.0599 | 0.0758 | 0.0666
TL | 2 | 0.0948 | 0.0693 | 0.0916 | 0.0721
| 3 | 0.0963 | 0.0825 | 0.1162 | 0.0802
| 20 | 0.1285 | 0.1100 | 0.1731 | 0.1467
LSTM with | 1 |  | 0.0468 | 0.0532 | 0.0562
TL | 2 |  | 0.0562 | 0.0868 | 0.0713
| 3 |  | 0.0588 | 0.1051 | 0.0795
| 20 |  | 0.0836 | 0.1710 | 0.1383
BLSTM without | 1 | 0.0599 | 0.0589 | 0.0529 | 0.0503
TL | 2 | 0.0726 | 0.0596 | 0.0621 | 0.0711
| 3 | 0.0903 | 0.0761 | 0.0874 | 0.0782
| 20 | 0.1170 | 0.1006 | 0.1205 | 0.1183
BLSTM with | 1 |  | 0.0522 | 0.0476 | 0.0548
TL | 2 |  | 0.0413 | 0.0492 | 0.0683
| 3 |  | 0.0451 | 0.0567 | 0.0696
| 20 |  | 0.0729 | 0.0593 | 0.1037
Table 2: RMSE of LSTM/BLSTM NN without and with TL.

The results show that the effect of TL on the accuracy of the prediction is
remarkable as the RMSE has lower values for all the elevations along the
obstacle height when the TL is used, compared to the cases when the models are
trained without TL, i.e., when the weights are initialized randomly in the
training process. It is also shown that when the BLSTM NN is used, the RMSE
values are lower than when LSTM NN is used. This can be observed for all the
elevations and both of the cases, i.e., without and with TL.

Figure 13 shows a comparison of the loss function value of the last epoch in
the training process of LSTM and BLSTM NNs at different elevations along the
obstacle. Here the loss function is measured using mean-square error (MSE). It
can be seen that the BLSTM NN outperformed the LSTM NN as it keeps the loss
function with approximately fixed values even for higher elevations. However,
for LSTM NN, the loss function values are affected by the increasing of
elevation as the time coefficients exhibit more complex behavior because of
the three-dimensional complex nature associated with the shedding process.

Figure 13: Comparison of the loss function value of the last epoch at
different elevations: (a) $z/D=0.02$, (b) $z/D=1$, (c) $z/D=2$, and (d)
$z/D=3$.

###  VII.3 Prediction of time coefficients and flow field reconstruction

One of the essential requirements in this study is to obtain an accurate
prediction of the POD time coefficients that can be used along with the
corresponding POD modes to reconstruct the flow field. Figure 14 shows a plot
of the real and predicted time coefficients for different POD modes along the
obstacle height. The predicted time coefficients using LSTM and BLSTM NNs show
accuracy similar to the real-time coefficients. Nevertheless, BLSTM NN showes
a better prediction for the modes that exhibit complex non-linear behavior.
This demonstrates that BLSTM NN has a great potential for addressing complex
non-linear time-series problems. After the time coefficients are predicted,
the instantaneous reduced-order flow field can be reconstructed using Eq.(3).

Figure 14: Plots of the real and predicted time coefficients for different POD
modes at different elevations: (a) $z/D=0.02$, (b) $z/D=1$, and (c) $z/D=3$.

Figure 15 shows the contour plots of a random instantaneous streamwise
velocity field from the test data calculated by the IDDES and the
corresponding reconstructed reduced-order stream-wise velocity field
calculated using the POD with real-time coefficients, LSTM-POD model, and
BLSTM-POD model, respectively. It is observed from the figure that the first
20 POD modes with the real-time coefficients could successfully reconstruct
the large-scale flow structure. However, a slight difference could be seen
between the real and the POD reduced-order flow fields in terms of the
reconstruction of small-scale eddies. This difference occurs due to the
energy-loss because the POD reconstruction considers only 20 modes
corresponding to 82%–92% of the total turbulent kinetic energy. The velocity
field that is reconstructed using BLSTM-POD models demonstrates noticeably
better results thanthe one obtained from the LSTM-POD model. As can be obseved
from the figure , a slight over-prediction can be recognized from the LSTM-POD
contour plots with the increasing elevation along the obstacle. To quantify
the reconstruction error of the LSTM-POD and BLSTM-POD models, the absolute
reconstruction error was applied as follow:

Figure 15: Contour plots of a random instantaneous streamwise velocity at
different elevations: (a) $z/D=0.02$, (b) $z/D=1$, and (c) $z/D=3$.

$\varepsilon=\frac{\left|u_{pred}-u_{POD}\right|}{u_{\infty}},$ (15)

where $u_{POD}$ and $u_{pred}$ are the velocity reconstructed using real POD
time coefficients and the time coefficients predicted using LSTM-POD and
BLSTM-POD models. As shown in Fig. 16, the maximum reconstruction error occurs
in the vicinity of the large-scale structures in the flow fields, i.e., near
the edge of the obstacle and where the shedding happens directly behind the
obstacle. The error obtained from the BLSTM-POD model is generally lower than
that from the LSM-POD model, which indicates that the BLSTM model outperformed
the LSTM model. These results are consistent with the results obtained from
Table II and Fig. 13.

Figure 16: Contour plots of the absolute reconstruction error of LSTM-POD and
BLSTM-POD models at different elevations: (a) $z/D=0.02$, (b) $z/D=1$, and (c)
$z/D=3$.

The two models are used to predict the instantaneous velocity field for 50 000
timesteps that are enough to reproduce the mean velocity profiles as shown in
Fig. 17 . The results from LSTM and BLSTM-POD models show an excellent
agreement with the POD and IDDES results indicating that the LSTM and BLSTM-
POD models could successfully capture the main flow fields features and for a
sufficiently large number of time steps that can produce a statistically
stationary data.

Figure 17: Streamwise profile of the mean streamwise velocity at elevation
$z/D=3$ in the central $x-z$ plane ($y/D=0$) (a) and spanwise profile of the
mean streamwise velocity at elevation $z/D=3$ and streamwise location $x=2$
(b).

###  VII.4 Effect of the prediction time window

LSTM NN is designed to model sequential data using correlations between
subsequent realizations. Hence, LSTM NN assume that the realizations of the
data are originally correlated with each other. In other words, they share
specific features that can together represent the sequential system using the
network parameters. This assumption could also be considered as a memory that
the network can obtain from the behavior of the sequential data. This memory
relies on the characteristics of the sequential data, which can have a
periodic or repetitive behavior that can be considered as a long-time
correlated event or as chaotic dynamic behavior. Hence, it is important to
study the temporal behavior of the POD time coefficients before investigating
the effect of the prediction time window length.

Fig. 18 shows the phase space plots of the time coefficients, where $\tau$ is
the time lag, which is considered as 10 time-steps. The yellow plane in each
plot illustrates the Poincar$\acute{e}$ section [52]. Poincar$\acute{e}$
section shows the intersection points of the trajectories of the discrete time
interval with an infinite plane. The set of the intersection points on this
plane indicates if the dynamical system is periodic, quasi-periodic, or has a
chaotic behavior. For instance, for a pure periodic system, two distinct
points will appear on the section, while for a quasi-periodic system, the
points will be scattered in a certain deterministic order. For systems with
chaotic behavior, the points will be scattered with irregular patterns. As can
be seen from the figure, at elevations $z/D$ = 0.02, 1 and 2, the phase space
plots of the first two time coefficients show a quasi-periodic behavior with a
slight decrease in the periodicity as the elevation increases. However, at
$z/D$ = 3, this behavior is barely recognized due to the aforementioned
downwash effect of the obstacle-free end.

For the remaining time coefficients, chaotic behavior can be recognized,
indicating that these coefficients exhibit lesser long-term correlations
compared to the first two time coefficients. Considering these observations,
LSTM and BLSTM NNs are trained using two different sample lengths of 50 and
100 time steps to predict the time coefficients with time windows of 50 and
100 time steps, respectively.

Table III shows the RMSE of the predicted time coefficients using time window
lengths of 50 and 100 time steps with the percentage increase of the RMSE
compared to a predicted time window of 10 time steps. In general, the RMSE
shows higher values as the prediction time window lengths increase. This
reduction in the accuracy can be explained by considering Lyapunov exponent
theory [53] , which states that as the trajectories of a dynamic system move
farther in time from the initial state, a continuous divergence starts
emerging between these trajectories. As expected, the time coefficients with a
higher POD ranking show higher RMSE values compared to the time coefficients
that correspond to the first two POD modes. This indicates that the long-term
memory is reduced as the POD ranking of the time coefficients increases, which
is consistent with the results obtained from Fig. 18. Interestingly, the
percentage rate of increase of RMSE reduces as the POD ranking increases
indicating that even though they have a long-term correlation, the time
coefficients with low POD ranking tend to demonstrate a faster reduction in
the prediction accuracy as the prediction time window increases.

Prediction time window (time steps) |  | POD mode | $z/D=0.02$ | $z/d=1$ | $z/D=2$ | $z/D=3$
---|---|---|---|---|---|---
50 | LSTM | 1 | 0.1955 (211.3%) | 0.1714 (186.1%) | 0.1865 (146.0%) | 0.2623 (293.8%)
|  | 2 | 0.2627 (177.1%) | 0.1761 (154.1%) | 0.2663 (190.7%) | 0.2805 (289.0%)
|  | 3 | 0.2560 (165.8%) | 0.1943 (135.5%) | 0.3174 (173.1%) | 0.3059 (281.4%)
|  | 20 | 0.2473 (92.4%) | 0.2198 (99.8%) | 0.4283 (147.4%) | 0.4271 (191.1%)
| BLSTM | 1 | 0.1543(157.5%) | 0.1692 (187.2%) | 0.1581 (198.8%) | 0.2014 (300.3%)
|  | 2 | 0.1712 (135.8%) | 0.1693 (184.0%) | 0.1851 (198.0%) | 0.2544 (257.8%)
|  | 3 | 0.2115 (134.2%) | 0.1906 (131.0%) | 0.2512 (187.4%) | 0.2804 (258.5%)
|  | 20 | 0.2352 (83.0%) | 0.2088 (107.5%) | 0.3451 (186.3%) | 0.3808 (221.8%)
100 | LSTM | 1 | 0.2153 (208.2%) | 0.2251 (275.8%) | 0.3898 (414.2%) | 0.4629 (595.0%)
|  | 2 | 0.2922 (203.4%) | 0.2276 (228.4%) | 0.4651 (407.7%) | 0.4972 (589.6%)
|  | 3 | 0.3101 (222.0%) | 0.2322 (181.4%) | 0.4932 (324.4%) | 0.5516 (587.7%)
|  | 20 | 0.3928 (205.7%) | 0.2819 (156.2%) | 0.5243 (202.8%) | 0.6316 (330.5%)
| BLSTM | 1 | 0.1974 (229.5) | 0.2168 (268.0%) | 0.3214 (507.5%) | 0.4015 (698.2%)
|  | 2 | 0.2361 (225.2%) | 0.1645 (176.0%) | 0.3631 (484.7%) | 0.4692 (559.9%)
|  | 3 | 0.2675 (1962.2%) | 0.2032 (146.0%) | 0.4758 (444.3%) | 0.5074 (548.8%)
|  | 20 | 0.3135 (143.9%) | 0.2512 (149.7%) | 0.5001 (315.0%) | 0.6173 (421.8%)
Table 3:  RMSE of LSTM/BLSTM NN and percentage rate (in parentheses) of RMSE
increasing with the use of two different prediction time windows. Figure 18:
Phase space plots and Poincar$\acute{e}$ sections of different normalized POD
time coefficients at different elevations: (a) z/D = 0.02, (b) z/D = 1, (c)
z/D = 2, (d) z/D = 3..

##  VIII Conclusion

This study presented a ROM based on LSTM/BLSTM NN and POD that can predict the
instantaneous turbulent wake behavior of a finite wall-mounted square
cylinder. The main idea of the ROM in this study is to accurately predict the
POD time coefficients for several future time steps instantaneously using the
LSTM/BLSTM NN and reconstruct the flow field using a few selected POD modes.
The following summarizes the conclusions and findings of this study:

  1. 1.

Because the first 20 POD modes for all the x-y planes along the obstacle
height contribute to 82%–92% of the total turbulent kinetic energy, they can
successfully reconstruct the main features of the flow field with the
corresponding time coefficients. Nevertheless, an inevitable difference is
recognized between the reduced-order flow fields and the flow obtained from
the IDDES due to the energy loss of the POD reconstruction.

  2. 2.

The use of the TL approach in this study shows a noticeable improvement in the
accuracy of the prediction.

  3. 3.

A comparative study of LSTM and BLSTM NNs revealed that BLSTM NN outperforms
LSTM NN in both training and prediction accuracy, suggesting that BLSTM NN has
more potential to predict time series data with more complex nonlinear
behavior comparing with LSTM NN.

  4. 4.

Both LSTM-POD and BLSTM-POD ROMs demonstrate good reconstruction results for
most of the regions in the flow fields. However, the reconstruction error
analysis showed that the BLSTM-POD ROM outperforms the LSTM-POD ROM at the
regions near the obstacle edges and right behind the obstacle, which is
consistent with the results obtained from the training and prediction error
analysis.

  5. 5.

The flow predicted by LSTM-POD and BLSTM-POD ROMs showed a statistically good
agreement with the reference IDDES data, suggesting that the ROMs could
predict the main features of the flow and for a long time that is sufficient
to produce statistically stationary data.

  6. 6.

The phase plots and Poincar$\acute{e}$ sections of the POD time coefficients
showed that the periodicity of the flow is mainly represented by the first two
POD modes and it decreases with increasing obstacle elevation, indicating that
the downwash effect from the obstacle-free end becomes more dominant. For the
rest of the POD time coefficients, a chaotic behavior can be seen.

  7. 7.

The results of training the LSTM/BLSTM NN with longer lengths of the
prediction time window revealed that the time coefficients corresponding to
the first two POD modes have more potential for prediction even with longer
lengths of the prediction time window because they have long time-correlated
behavior. However, as they showed a higher increasing rate of the RMSE as the
prediction time window length increases, the time coefficients with lower POD
ranking tend to have a faster decrease in the prediction accuracy comparing
with the time coefficients of higher POD ranking.

The previous observations indicate that the ROM presented in this study could
predict the reduced-order flow field of the turbulent wake with commendable
accuracy when the BLSTM-POD model is used. As the prediction accuracy of the
model is affected by the increase in the prediction time window length,
further studies focusing on the optimization of the model will be conducted to
improve the prediction accuracy for longer prediction time windows. Moreover,
the prediction capability of the proposed ROM will be further examined using a
range of $Re$ and different obstacles shapes.

###### Acknowledgements.

This work was supported by ’Human Resources Program in Energy Technology’ of
the Korea Institute of Energy Technology Evaluation and Planning (KETEP),
granted financial resource from the Ministry of Trade, Industry & Energy,
Republic of Korea (no. 20184030202200). In addition, this work was supported
by the National Research Foundation of Korea (NRF) grant funded by the Korea
government (MSIP) (no. 2019R1I1A3A01058576). This work was also supported by
the National Supercomputing Center with supercomputing resources including
technical support (KSC-2020-INO-0025).

## Data Availability

The data that supports the findings of this study are available within this
article.

## References

  * Bourgeois, Sattari, and Martinuzzi [2011] J. A. Bourgeois, P. Sattari, and R. J. Martinuzzi, “Alternating half-loop shedding in the turbulent wake of a finite surface-mounted square cylinder with a thin boundary layer,” Physics of Fluids 23(9), 095101 (2011).
  * Park and Lee [2000] C.-W. Park and S.-J. Lee, “Free end effects on the near wake flow structure behind a finite circular cylinder,” Journal of Wind Engineering and Industrial Aerodynamics 88(2–3), 231–246 (2000).
  * Sattari, Bourgeois, and Martinuzzi [2012] P. Sattari, J. A. Bourgeois, and R. J. Martinuzzi, “On the vortex dynamics in the wake of a finite surface-mounted square cylinder,” Experiments in Fluids 52(5), 1149–1167 (2012).
  * Saeedi, LePoudre, and Wang [2014] M. Saeedi, P. P. LePoudre, and B. C. Wang, “Direct numerical simulation of turbulent wake behind a surface-mounted square cylinder,” Journal of Fluids and Structures 50, 20–39 (2014).
  * Tang, Graham, and Peraire [1996] K. Tang, W. Graham, and J. Peraire, “Active flow control using a reduced order model and optimum control,” https://doi.org/10.2514/6.1996-1946 (1996).
  * Wang and Zhou [2009] H. F. Wang and Y. Zhou, “The finite-length square cylinder near wake,” Journal of Fluid Mechanics 638, 453–490 (2009).
  * Yousif and Lim [2020] M. Z. G. Yousif and H. Lim, “On the characteristics of the turbulent wake behind a wall-mounted square cylinder,” ArXiv:2012.11263 [Physics]. (2020).
  * Lumley [1967] J. L. Lumley, “The structure of inhomogeneous turbulent flows,” Atmospheric Turbulence and Radio Wave Propagation (1967).
  * Schmid [2010] P. J. Schmid, “Dynamic mode decomposition of numerical and experimental data,” Journal of Fluid Mechanics 656, 5–28 (2010).
  * Burkardt, Gunzburger, and Lee [2006] J. Burkardt, M. Gunzburger, and H.-C. Lee, “Pod and cvt-based reduced-order modeling of navier–stokes flows,” Computer Methods in Applied Mechanics and Engineering 196(1), 337–355 (2006).
  * Carlberg, Bou-Mosleh, and Farhat [2011] K. Carlberg, C. Bou-Mosleh, and C. Farhat, “Efficient non-linear model reduction via a least-squares petrov–galerkin projection and compressive tensor approximations,” International Journal for Numerical Methods in Engineering 86(2), 155–181 (2011).
  * Rap$\acute{u}$n and Vega [2010] M.-L. Rap$\acute{u}$n and J. M. Vega, “Reduced order models based on local pod plus galerkin projection,” Journal of Computational Physics 229(8), 3046–3063 (2010).
  * Rowley, Colonius, and Murray [2000] C. W. Rowley, T. Colonius, and R. M. Murray, “Model reduction for compressible flows using pod and galerkin projection,” Physica D: Nonlinear Phenomena 189(1), 115–129 (2000).
  * Akhtar, Nayfeh, and Ribbens [2009] I. Akhtar, A. H. Nayfeh, and C. J. Ribbens, “On the stability and extension of reduced-order galerkin models in incompressible flows.” Theoretical and Computational Fluid Dynamics 23(3), 213–237 (2009).
  * Rempfer [2000] D. Rempfer, “On low-dimensional galerkin models for fluid flow,” Theoretical and Computational Fluid Dynamics 14(2), 75–88 (2000).
  * Shinde  _et al._ [2016] V. Shinde, E. Longatte, F. Baj, Y. Hoarau, and M. Braza, “A galerkin-free model reduction approach for the navier–stokes equations,” Journal of Computational Physics 309, 148–163 (2016).
  * LeCun, Bengio, and Hinton [2015] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015).
  * Brunton, Noack, and Koumoutsakos [2020] S. L. Brunton, B. R. Noack, and P. Koumoutsakos, “Machine learning for fluid mechanics,” Annual Review of Fluid Mechanics 52(1), 477–508 (2020).
  * Kutz [2017] J. N. Kutz, “Deep learning in fluid dynamics,” Journal of Fluid Mechanics 814, 1–4 (2017).
  * Baldi and Hornik [1989] P. Baldi and K. Hornik, “Neural networks and principal component analysis: Learning from examples without local minima,” Neural Networks 2(1), 53–58 (1989).
  * Bourlard and Kamp [1988] H. Bourlard and Y. Kamp, “Auto-association by multilayer perceptrons and singular value decomposition,” Biological Cybernetics 54(4), 291–294 (1988).
  * Milano and Koumoutsakos [2002] M. Milano and P. Koumoutsakos, “Neural network modeling for near wall turbulent flow,” Journal of Computational Physics 182(1), 1–26 (2002).
  * G$\ddot{u}$emes, Discetti, and Ianiro [2019] A. G$\ddot{u}$emes, S. Discetti, and A. Ianiro, “Sensing the turbulent large-scale motions with their wall signature,” Physics of Fluid 31(12), 125112 (2019).
  * Murata, Fukami, and Fukagata [2020] T. Murata, K. Fukami, and K. Fukagata, “Nonlinear mode decomposition with convolutional neural networks for fluid dynamics,” Journal of Fluid Mechanics 882 (2020).
  * Wang  _et al._ [2018] Z. Wang, D. Xiao, F. Fang, R. Govindan, C. C. Pain, and Y. Guo, “Model identification of reduced order fluid dynamics systems using deep learning,” International Journal for Numerical Methods in Fluids 86(4), 255–268 (2018).
  * Mohan and Gaitonde [2002] A. T. Mohan and D. V. Gaitonde, “A deep learning based approach to reduced order modeling for turbulent flow control using lstm neural networks,” ArXiv Preprint ArXiv:1804.09269 (2002).
  * Mathey  _et al._ [2006] F. Mathey, D. Cokljat, J. P. Bertoglio, and E. Sergent, “Assessment of the vortex method for large eddy simulation inlet conditions,” Progress in Computational Fluid Dynamics, an International Journal 6(1–3), 58–67 (2006).
  * Sergent [2002] E. Sergent, “Vers une méthodologie de couplage entre la simulation des grandes échelles et les modèles statistiques [these de doctorat],” Ecully, Ecole centrale de Lyon (2002).
  * Kolmogorov [1941] A. N. Kolmogorov, “The local structure of turbulence in incompressible viscous fluid for very large reynolds numbers,” C. R. Acad. Sci. URSS 30, 301–305 (1941).
  * Moin and Mahesh [1998] P. Moin and K. Mahesh, “Direct numerical simulation: A tool in turbulence research,” Annual Review of Fluid Mechanics 30(1), 539–578 (1998).
  * Wilcox [1993] D. C. Wilcox, “Turbulence modeling for cfd,” DCW Industries, Inc. (1993).
  * Smagorinsky [1987] J. Smagorinsky, “General circulation experiments with the primitive equationsi. the basic experiment.” Monthly Weather Review 91(3), 99–164 (1987).
  * Stolz, Adams, and Kleiser [2001] S. Stolz, N. A. Adams, and L. Kleiser, “An approximate deconvolution model for large-eddy simulation with application to incompressible wall-bounded flows,” Physics of Fluids 13(4), 997–1015 (2001).
  * Wang, Thompson, and Hu [2019] Y. Wang, D. Thompson, and Z. Hu, “Effect of wall proximity on the flow over a cube and the implications for the noise emitted,” Physics of Fluids 31(7), 077101 (2019).
  * Spalart  _et al._ [1997] P. Spalart, W. Jou, M. Strelets, and S. R. Allmaras, “Comments on the feasibility of les for wings, and on a hybrid rans/les approach.” Proceedings of First AFOSR International Conference on DNS/LES (1997).
  * Spalart [1997] P. R. Spalart, “Strategies for turbulence modelling and simulations.” International Journal of Heat and Fluid Flow 21(3), 252–263 (1997).
  * Travin  _et al._ [2000] A. Travin, M. Shur, M. Strelets, and P. Spalart, “Detached-eddy simulations past a circular cylinder,” Flow, Turbulence and Combustion 63(1), 293–313 (2000).
  * Spalart [2009] R. P. Spalart, “Detached-eddy simulation.” Annual Review of Fluid Mechanics 41(1), 181–202 (2009).
  * Spalart  _et al._ [2006] P. Spalart, S. Deck, M. L. Shur, K. D. Squires, M. K. Strelets, and A. Travin, “A new version of detached-eddy simulation, resistant to ambiguous grid densities.” Theoretical and Computational Fluid Dynamics 20(3), 181 (2006).
  * Shur  _et al._ [2008] M. L. Shur, P. R. Spalart, M. K. Strelets, and A. K. Travin, “A hybrid rans-les approach with delayed-des and wall-modelled les capabilities,” International Journal of Heat and Fluid Flow 29(6), 1638–1649 (2008).
  * Berkooz, Holmes, and Lumley [1993] G. Berkooz, P. Holmes, and J. L. Lumley, “The proper orthogonal decomposition in the analysis of turbulent flows,” Annual Review of Fluid Mechanics 25(1), 539–575 (1993).
  * Sirovich [1987] L. Sirovich, “Turbulence and the dynamics of coherent structures. i. coherent structures,” Quarterly of Applied Mathematics 45(3), 561–571 (1987).
  * Hochreiter and Schmidhuber [1997] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation 9(8), 1735–1780 (1997).
  * Rumelhart, Hinton, and Williams [1986] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature 323(6088), 533–536 (1986).
  * Graves [2012] A. Graves, “Long short-term memory.” In A. Graves (Ed.), Supervised Sequence Labelling with Recurrent Neural Networks , 37––45 (2012).
  * Graves, Fernández, and Schmidhuber [2005] A. Graves, S. Fernández, and J. Schmidhuber, “Bidirectional lstm networks for improved phoneme classification and recognition,” In W. Duch, J. Kacprzyk, E. Oja, & S. Zadrożny (Eds.), Artificial Neural Networks: Formal Models and Their Applications – ICANN 2005 , 799––804 (2005).
  * Graves and Schmidhuber [2005] A. Graves and J. Schmidhuber, “Framewise phoneme classification with bidirectional lstm and other neural network architectures,” Neural Networks 18(5), 602–610 (2005).
  * Huang, Xu, and Yu [2015] Z. Huang, W. Xu, and K. Yu, “Bidirectional lstm-crf models for sequence tagging,” ArXiv:1508.01991 [Cs] (2015).
  * Marchi  _et al._ [2014] E. Marchi, G. Ferroni, F. Eyben, L. Gabrielli, S. Squartini, and B. Schuller, “Multi-resolution linear prediction based features for audio onset detection with bidirectional lstm neural networks,” 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2164–2168 (2014).
  * Weiss, Khoshgoftaar, and Wang [2016] K. Weiss, T. M. Khoshgoftaar, and D. Wang, “A survey of transfer learning,” Journal of Big Data 3(1), 9 (2016).
  * Kingma and Ba [2014] B. P. Kingma and J. Ba, “Adam: A method for stochastic optimization.” https://arxiv.org/abs/1412.6980v9 (2014).
  * Cavers [2017] J. Cavers, “Chaos and time series analysis: Optimization of the poincaré section and distinguishing between deterministic and stochastic time series,” Electronic Thesis and Dissertations. (2017).
  * Wolf  _et al._ [1985] A. Wolf, J. B. Swift, H. Swinney, and J. A. Vastano, “Determining lyapunov exponents from a time series,” Physica D: Nonlinear Phenomena 16(3), 285–317 (1985).
  * Franke  _et al._ [2004] J. Franke, C. Hirsch, A. Jensen, H. Krüs, M. Schatzmann, P. Westbury, S. Miles, J. Wisse, and N. G. Wright, “Recommendations on the use of cfd in wind engineering, cost action c14,” European Science Foundation COST Office 14, c1 (2004).
  * Saha [2013] A. K. Saha, “Unsteady flow past a finite square cylinder mounted on a wall at low reynolds number,” Computers & Fluids 88, 599–615 (2013).
  * Schmitt [2007] F. G. Schmitt, “About boussinesq’s turbulent viscosity hypothesis: Historical remarks and a direct evaluation of its validity,” Comptes Rendus Mécanique 335(9), 617–627 (2007).
  * Weiss [2019] J. Weiss, “A tutorial on the proper orthogonal decomposition,” In AIAA Aviation 2019 Forum (Vol. 1–0). American Institute of Aeronautics and Astronautics (2019).