# A Multi-Fidelity Methodology for Reduced Order Models with High-Dimensional
Inputs

Bilal Mufti111Graduate Researcher, ASDL, Daniel Guggenheim School of Aerospace
Engineering Georgia Institute of Technology, Atlanta, Georgia, 30332
Christian Perron222Research Engineer II, ASDL, Daniel Guggenheim School of
Aerospace Engineering Georgia Institute of Technology, Atlanta, Georgia, 30332
Dimitri N. Mavris333S.P. Langley Distinguished Regents Professor and Director
of ASDL, Daniel Guggenheim School of Aerospace Engineering, AIAA Fellow
Georgia Institute of Technology, Atlanta, Georgia, 30332

###### Abstract

In the early stages of aerospace design, reduced order models (ROMs) are
crucial for minimizing computational costs associated with using physics-rich
field information in many-query scenarios requiring multiple evaluations. The
intricacy of aerospace design demands the use of high-dimensional design
spaces to capture detailed features and design variability accurately.
However, these spaces introduce significant challenges, including the curse of
dimensionality, which stems from both high-dimensional inputs and outputs
necessitating substantial training data and computational effort. To address
these complexities, this study introduces a novel multi-fidelity, parametric,
and non-intrusive ROM framework designed for high-dimensional contexts. It
integrates machine learning techniques for manifold alignment and dimension
reduction—employing Proper Orthogonal Decomposition (POD) and Model-based
Active Subspace—with multi-fidelity regression for ROM construction. Our
approach is validated through two test cases: the 2D RAE 2822 airfoil and the
3D NASA CRM wing, assessing combinations of various fidelity levels, training
data ratios, and sample sizes. Compared to the single-fidelity PCAS method,
our multi-fidelity solution offers improved cost-accuracy benefits and
achieves better predictive accuracy with reduced computational demands.
Moreover, our methodology outperforms the manifold-aligned ROM (MA-ROM) method
by 50% in handling scenarios with large input dimensions, underscoring its
efficacy in addressing the complex challenges of aerospace design.

## Nomenclature

$\mathbf{Y},\mathbf{Z}$ = | high- and low-fidelity training fields
---|---
$\mathcal{X}$ = | input parameter space
$Re_{\infty}$ = | free stream Reynolds number
$M_{\infty}$ = | free stream Mach number
$C_{l},C_{d}$ = | lift and drag coefficients
$C_{p}$ = | coefficient of pressure
$m_{1},m_{2}$ = | number of high- and low-fidelity samples
$p,q$ = | dimensionality of high- and low-fidelity fields
$d$ = | input space dimensionality
$n_{v}$ = | number of test samples
$\tau$ = | multi-fidelity ratio parameter
$\tilde{E},\tilde{E}_{RC},\tilde{E}_{RG}$ = | field prediction, reconstruction, and regression errors

_Procrustes Manifold Alignment_

$\mathbf{H},\mathbf{S}$ = | high- and low-fidelity field latent coordinates
---|---
$k$ = | latent output space dimensionality
$\mathbf{\Theta}_{k},\mathbf{\Xi}_{k}$ = | high- and low-fidelity POD modes
$\mathbf{U},\mathbf{V}$ = | squares matrices containing left and right singular vectors
$\boldsymbol{\Sigma}$ = | diagonal matrix of singular values
$t$ = | scaling factor
$\mathbf{G}$ = | low-fidelity latent variables aligned with the high-fidelity variables

_Model-Based Active Subspace_

$\delta$ = | correction term for the difference between high- and low-fidelity latent variables
---|---
$a,\mathbf{b}$ = | linear regression coefficients
$w$ = | weights of surrogate model
$\phi,\epsilon$ = | basis function and polynomial trend
$\mathbf{C}_{MF}$ = | multi-fidelity covariance matrix of gradients
$\mathbf{\Lambda}$ = | diagonal matrix of eigenvalues
$\mathbf{W}$ = | orthogonal matrix containing eigenvectors
$\mathbf{W}_{AS}$ = | active subspace matrix

_Subscripts and Superscripts_

$L,U$ = | linked and unlinked data
---|---
$\tilde{}$ = | model prediction
$*$ = | out-of-sample design
$AS$ = | active subspace
′ = | gradient of the function

##  1 Introduction

The last few decades have witnessed an exponential growth in global
computational power. However, this increase has not significantly offset the
computational cost of certain high-fidelity numerical simulations,
particularly those employing discretization methods like finite element or
finite volume [1]. These simulations become resource-intensive in _many-query_
scenarios such as design space exploration, design optimization, uncertainty
quantification, and propagation [2]. In aerospace design, the early design
phase frequently uses these high-fidelity simulations in many-query contexts
to assess multiple design configurations and to conduct in-depth multi-
disciplinary analyses. Yet, the high computational expense of these
simulations presents a substantial challenge to their practical application in
the initial stages of aerospace systems design. This situation has driven
designers to seek and adopt more efficient methods and techniques that can
alleviate the computational load without compromising the quality and depth of
the analyses.

Reduced order modeling is emerging as a powerful tool in the engineering and
computational science fields due to its ability to significantly reduce the
computational cost of obtaining high-fidelity numerical simulation results.
Reduced order models (ROMs) serve a purpose similar to traditional surrogate
models, i.e., they offer a cheap approximation of an expensive numerical
simulation. However, unlike traditional data-fit surrogate models that are
used to predict integrated scalar quantities, ROMs can be used to represent
high-dimensional field solutions having both spatial and temporal correlations
[3]. Once trained, they can be used to replace high-fidelity simulations in
many query applications without incurring additional costs. ROMs require high
upfront offline training costs to allow rapid evaluations with minimal online
costs.

The main objective of ROM is to find a low-dimensional representation, called
a latent space or subspace, that best describes the original high-dimensional
solution space. This often takes the form of a linear subspace such that the
transformation from the high- to the low-dimensional field is given by a
projection map.

Non-intrusive ROM methods are frequently used in engineering applications due
to their versatility and ease of implementation [4]. This category of methods
treats the original simulation model as a black-box function and does not
require access to the underlying simulation source code. Proper orthogonal
decomposition (POD) is a simple and efficient dimensionality reduction method
that is often combined with a regression model to form a non-intrusive ROM [5,
6, 7, 8, 9]. This POD-regressor combination can be trained to predict new
solutions in latent space. The linearity of the POD technique can then be used
to map the predicted solution from latent space to output dimension space.
Recently, deep learning (DL)-based ROM techniques have emerged as popular
tools due to their prowess in capturing complex nonlinear relationships
between inputs and outputs [10, 11, 12]. However, it’s important to note that
DL-based ROMs require a substantial amount of training data to effectively
learn these relationships. A recent study by Mufti et al. [13] demonstrated
that linear POD-based ROMs hold an advantage over DL-based ROMs when the
number of training samples is limited.

###  1.1 ROMs with High-Dimensional Input

To effectively capture the variability within the design space, engineering
models that are represented by ROMs and scalar-based data-fit surrogate models
are often parameterized by a large number of design variables. As a result,
they tend to suffer from the _curse of dimensionality_ [14], a common
expression used to illustrate the challenges associated with high-dimensional
problems. This results in a rapid increase in the volume of the design space
with increasing design variables, requiring a disproportionately large number
of samples for a dense coverage of the design space. High-dimensional input
spaces also make it harder to reason on distances when defining input-output
relationships, a phenomenon sometimes referred to as the _concentration of
distances_ [15]. Input dimensionality reduction (DR) techniques can be used to
effectively alleviate this curse of dimensionality. Unsupervised dimension
reduction techniques such as principal component analysis (PCA) [16],
diffusion maps [17], etc., work decently when the input variables are
correlated. However, in most ROMs or surrogate modeling applications, the
input space is usually sampled uniformly with a quasi-random design of
experiment (DOE). For such problems, no structure in the inputs alone can be
exploited, and a supervised DR technique that leverages instead the explicit
input-output relationship is best suited. Sensitivity analysis [18], automatic
relevance determination [19], and partial least squares [20] are some of the
supervised DR techniques that have been used to carry out input space
dimension reduction with limited success.

Active subspace methodology (ASM) is a tool that has recently gained success
due to its ability to effectively reduce input dimensionality. The methodology
identifies a low dimensional linear subspace, referred to as the active
subspace, that best describes the variability in the output due to changes in
input variables [21, 22, 23, 24]. The classic ASM methodology applies
eigenvalue decomposition on gradient information to find this subspace. It has
been successfully applied to solve aerodynamic shape optimization [25, 26],
hull design [27], cavity design to stabilize supersonic flames [28], hyper-
sonic engine uncertainty quantification [29], materials design [30] and
sensitivity analysis for lithium-ion battery [31]. A multi-fidelity version of
ASM can be used to reduce the cost of input dimension reduction using active
subspace [32, 33, 34]. However, obtaining gradient information through
intrusive methods from a simulation is not always straightforward, especially
for closed-source tools that do not readily provide this capability. This
limitation can restrict the application of the active subspace methodology. To
address this challenge, gradient-free methods have been proposed, which
identify the active subspace using only direct function evaluations. Some of
these gradient-free methods use probabilistic techniques, such as Gaussian
process regressor (GPR) with built-in dimensionality reduction [35, 36, 37].
The active subspace projection matrix is treated as an additional hyper-
parameter to be estimated from the given data.

The input DR methods discussed previously have been effectively used to create
scalar-based data-fit surrogate models. However, constructing ROMs with high-
dimensional input space poses additional challenges: (1) we need to find an
active subspace in which the low-dimensional representation of the output
field varies the most, and (2) a separate active subspace has to be computed
for each of the projected coordinates in the low-dimensional output space. The
PCAS technique has been successfully used to create ROMs for engineering
problems with high-dimensional input space. Fig. 1 gives a graphical
representation of steps involved in the application of the PCAS model to
construct ROMs. The methodology first carries out output space reduction using
POD. A linear regression model is then created linking POD modes and the input
space. Gradient information is found using this regression model. The ASM is
then used to find the active subspace in the input domain. PCAS has been used
in additive manufacturing [38], aircraft fuselage panel analysis [39], beam
structural analysis [40] and ship hull design [27]. A modified version of this
method called PCAS-AK combines PCAS with active Kriging to adaptively select
training samples to improve computational efficiency [41]. Recently, a non-
intrusive POD-based ROM which uses a GPR with an in-built input space
reduction [42], and derivative-informed projected neural networks (DIPNets)
[43] have also been used to develop a prediction model for problems with high-
dimensional input and output space.

###  1.2 Multi-Fidelity ROM Techniques

A major drawback of existing ROM methods for high dimensional input space is
that they require substantial training data to achieve reasonable accuracy
[41]. For most engineering problems, generating a large quantity of high-
fidelity simulation data can become cost-prohibitive. Multi-fidelity methods
alleviate the high training cost of creating such ROMs by leveraging cheaper
to evaluate low-fidelity data [44, 45]. Multi-fidelity ROM techniques use
dense low-fidelity data to improve their prediction accuracy with a modest
increase in computational budget. Most multi-fidelity ROM techniques
necessitate consistent representations between field solutions of various
fidelities, meaning they require a similar discretization of the output field.
Such methods use different numerical methods for different fidelity levels
[46]. Manifold alignment [47] was recently introduced as a technique that can
be used to align underlying latent spaces of various fidelity levels having
inconsistent representation [48]. This technique was combined with a POD-
regressor combination to construct a multi-fidelity manifold aligned ROM (MA-
ROM) and was shown to offer a computational cost and accuracy advantage over
its single-fidelity counterpart in predicting pressure and stress fields [49,
50]. Most of the multi-fidelity ROM techniques are limited by the number of
input variables that can be used. They have been developed for problems with 1
to 10 design variables [51, 50]. The need for developing multi-fidelity ROMs
for high-dimensional design spaces has been recognized by several authors [52,
1].

Figure 1: A graphical representation of the PCAS method to develop ROMs for
high-dimensional input space.

###  1.3 Contributions of Current Work

This study proposes a method that combines the machine learning concept of
manifold alignment with dimension reduction via POD and ASM to address the
limitations of existing methods. This method, applied in a multi-fidelity
setting, creates a non-intrusive ROM for problems with high dimensional input
spaces. The main contributions of this study are: (1) it develops a multi-
fidelity, computationally efficient, and simple approach for constructing ROMs
for high dimensional input space, and (2) it uses a multi-fidelity model-based
active subspace approach to carry out input dimensional reduction. The
sensitivity of the proposed methodology is assessed by: (a) varying the number
of training samples, (b) changing the ratio of high- and low-fidelity data in
training samples, and (c) varying the total number of input dimensions. The
proposed method is also benchmarked against existing ROM techniques.

The rest of the paper is organized as follows. In Sec. 2, a detailed
explanation of the proposed approach is given followed by the formalization of
the method. Sec. 3 introduces the performance metrics and test cases that will
be used to evaluate the performance of the proposed method. Sec. 4 discusses
the results of the experiments and establishes the computational efficiency of
the method by comparing its performance with the single-fidelity PCAS and MA-
ROM methods.

##  2 Methodology

This section provides details of the numerical method developed in this study.
An overview of the Procrustes manifold alignment technique, which is used for
output dimension reduction is given. Then an explanation of a multi-fidelity
model-based active subspace technique used to reduce input dimensionality is
presented. Finally, the two methods are integrated to create a multi-fidelity
ROM for high-dimensional input space.

###  2.1 Output Dimension Reduction using Procrustes Manifold Alignment

Manifold alignment (MA) is a machine learning technique that is used to find a
unifying representation of multiple datasets that share a common low-
dimensional latent space. Manifold alignment works on the principle of
transfer learning. The technique has been used to solve problems in fields of
bio-informatics [53], medical imaging [54], computer vision [55], etc.
Procrustes analysis is an approach used for manifold alignment. The analysis
aligns the latent space of multiple datasets which can be found using a
dimensionality reduction technique [56]. In the proposed method, Procrustes
manifold alignment is combined with POD to carry out dimension reduction and
alignment of different fidelity output fields as demonstrated by Perron et al.
[50].

####  2.1.1 Mathematical Formulation

Let us consider two datasets $\mathbf{Y}\in\mathbb{R}^{p\times m_{1}}$ and
$\mathbf{Z}\in\mathbb{R}^{q\times m_{2}}$, with $m_{2}>m_{1}$. Here, $m_{1}$,
and $m_{2}$ represent the number of samples, and $p$ and $q$ represent the
dimensionality of datasets. The datasets $\mathbf{Y}$ and $\mathbf{Z}$
represent simulation results from a high- and low-fidelity analysis
respectively with the columns of the matrices holding field solutions. It is
assumed that these datasets are related in some way, e.g., they share similar
governing equations but have different field discretization. We can consider
both $\mathbf{Y}$ and $\mathbf{Z}$ to be simulation results for an input
parameter space $\mathcal{X}\in\mathbb{R}^{d}$, where d is the dimension of
the input space. Additionally, we can assume that the first $m_{1}$ solutions
in both datasets were generated using the same input parameter values. In
other words, these $m_{1}$ data points are considered _linked_ while the
remainder is assumed to be _unlinked_.

The first step of Procrustes manifold alignment is to find the low dimensional
latent space for both the high- and low-fidelity datasets individually. In the
current case, this is done using the conventional POD method and produces the
following results

$\displaystyle\mathbf{H}$
$\displaystyle=\boldsymbol{\Theta}_{k}^{T}\mathbf{Y}$ (1)
$\displaystyle\mathbf{S}$ $\displaystyle=\boldsymbol{\Xi}_{k}^{T}\mathbf{Z}$
(2)

where $\mathbf{H}\in\mathbb{R}^{k_{1}\times m_{1}}$ and
$\mathbf{S}\in\mathbb{R}^{k_{2}\times m_{2}}$ contains latent variables. The
matrices $\boldsymbol{\Theta}_{k}\in\mathbb{R}^{p\times k_{1}}$ and
$\boldsymbol{\Xi}_{k}\in\mathbb{R}^{q\times k_{2}}$ represent the orthonormal
bases corresponding to the largest eigenvalues of $\mathbf{Y}$ and
$\mathbf{Z}$ respectively, where $(k_{1},k_{2})\ll(p,q)$. The dimensionality
($k_{1},k_{2}$) of both high- and low-fidelity latent space can be found using
relative information criterion (RIC) [57], defined as :

$\text{RIC}=\frac{\sum_{i=1}^{k}\lambda_{i}}{\sum_{j=1}^{p|q}\lambda_{j}}$ (3)

In this formulation, we will assume $k_{1}=k_{2}=k$, but in some cases, due to
a large number of low-fidelity samples, $k_{2}$ can be larger than $k_{1}$.
For such cases, the Procrustes analysis is still applicable, but the alignment
problem turns into a projection Procrustes problem [58].

We can partition the low-fidelity latent variables into linked and unlinked
datasets, i.e. $\mathbf{S}=[\mathbf{S}_{L},\mathbf{S}_{U}]$ where
$\mathbf{S}_{L}\in\mathbb{R}^{k\times m_{1}}$ and
$\mathbf{S}_{U}\in\mathbb{R}^{k\times(m_{2}-m_{1})}$. Once both datasets have
been projected to a reduced dimensional latent space, the next step is to find
an affine transformation (i.e., translation, rotation, reflection, scaling) to
align latent variables sets having the same input parameter values, i.e.,
align $\mathbf{S_{L}}$ with $\mathbf{H}$. This can be achieved with the
following steps:

  1. 1.

Translate $\mathbf{S_{L}}$ and $\mathbf{H}$ such that their respective mean is
zero, i.e., centroids of the datasets should be at the origin.

  2. 2.

Compute the SVD of $\mathbf{S_{L}}\mathbf{H}$ as

$\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{T}=\mathbf{S}_{L}\mathbf{H}^{T}$ (4)

Where,

$\mathbf{\Sigma}\in\mathbb{R}^{k\times k}$ is a diagonal matrix containing the
singular values.

$\mathbf{U}\in\mathbb{R}^{k\times k}$ and $\mathbf{V}\in\mathbb{R}^{k\times
k}$ are square matrices containing the left and right singular vectors
respectively.

  3. 3.

Using the left and right singular vectors we can find the optimal
transformation matrix as:

$\displaystyle\mathbf{P}$ $\displaystyle=\mathbf{V}\mathbf{U}^{T}$ (5)

  4. 4.

The scaling factor can be calculated using the eigenvalue matrix

$\displaystyle
t=\frac{\text{trace}(\boldsymbol{\Sigma})}{\text{trace}\left(\mathbf{S}_{L}\mathbf{S}_{L}^{T}\right)}$
(6)

  5. 5.

As a final step, the low-fidelity latent variables that are aligned with high-
fidelity variables can be computed as:

$\mathbf{G}=t\mathbf{P}\mathbf{S}$ (7)

The main steps of the Procrustes manifold alignment algorithm are outlined in
Algorithm 1. At the end of the output dimension reduction phase, we have two
datasets $\mathbf{H}$ and $\mathbf{G}$ containing latent variables of the
high- and low-fidelity data respectively. The dimensionality of both these
data sets is the same, i.e., $\mathbf{H}\in\mathbb{R}^{k\times m_{1}}$ and
$\mathbf{G}\in\mathbb{R}^{k\times m_{2}}$, and they lie on the same manifold.
We only need to store $\mathbf{H}$, $\mathbf{G}$, and
$\boldsymbol{\Theta}_{k}$, at the end of this phase. $\mathbf{H}$ and
$\mathbf{G}$ will be used to construct an active subspace for each latent
variable, and $\boldsymbol{\Theta}_{k}$ will be used to reconstruct the output
field of interest.

Algorithm 1 Procrustes Manifold Alignment

Input: High-fidelity data $\mathbf{Y}\in\mathbb{R}^{p\times m_{1}}$, low-
fidelity data $\mathbf{Z}\in\mathbb{R}^{q\times m_{2}}$
Output: Aligned latent variables $\mathbf{H}$, $\mathbf{G}$, and high-fidelity
POD modes $\boldsymbol{\Theta}_{k}$

1: Find orthonormal bases $\boldsymbol{\Theta}_{k}\in\mathbb{R}^{p\times k}$
and $\boldsymbol{\Xi}_{k}\in\mathbb{R}^{q\times k}$ using the largest
eigenvalues of $\mathbf{Y}$ and $\mathbf{Z}$, respectively

2: Compute low-dimensional latent variables
$\mathbf{H}=\boldsymbol{\Theta}_{k}^{T}\mathbf{Y}$ and
$\mathbf{S}=\boldsymbol{\Xi}_{k}^{T}\mathbf{Z}$

3: Partition $\mathbf{S}$ into $\mathbf{S}=[\mathbf{S}_{L},\mathbf{S}_{U}]$,
where $\mathbf{S}_{L}\in\mathbb{R}^{k\times m_{1}}$ and
$\mathbf{S}_{U}\in\mathbb{R}^{k\times(m_{2}-m_{1})}$

4: Translate $\mathbf{S}_{L}$ and $\mathbf{H}$ such that their respective mean
is zero

5: Compute the SVD of $\mathbf{S}_{L}\mathbf{H}^{T}$:
$\mathbf{U}\boldsymbol{\Sigma}\mathbf{V}^{T}=\mathbf{S}_{L}\mathbf{H}^{T}$

6: Compute optimal transformation matrix:
$\mathbf{P}=\mathbf{V}\mathbf{U}^{T}$

7: Compute scaling factor:
$t=\frac{\text{trace}(\boldsymbol{\Sigma})}{\text{trace}\left(\mathbf{S}_{L}\mathbf{S}_{L}^{T}\right)}$

8: Compute low-fidelity latent variables aligned with high-fidelity variables:
$\mathbf{G}=t\mathbf{P}\mathbf{S}$

9: return $\mathbf{H}$, $\mathbf{G}$,$\boldsymbol{\Theta}_{k}$

####  2.1.2 Multi-Fidelity Reduced Order Model

We can create a multi-fidelity reduced-order model that uses both high- and
low-fidelity latent variables $\mathbf{H}$ and $\mathbf{G}$, to predict the
new high-fidelity field solutions. This manifold-aligned ROM (MA-ROM) can be
constructed using any multi-fidelity regression technique [59, 60, 61] and
will be computationally efficient as compared to the corresponding single-
fidelity ROM. However, this ROM would have an input domain $\mathcal{X}$,
which is still high-dimensional and hence would suffer from the curse of
dimensionality.

###  2.2 Input Dimension Reduction using Multi-Fidelity Model-Based Active
Subspace

To reduce input dimensionality, we want to find a low dimensional subspace
that effectively captures the variability in output latent space due to change
in $\mathcal{X}$. This active subspace is spanned by the dominant eigenvectors
of the estimator for the covariance (second moment) matrix. To find the multi-
fidelity approximation of this covariance matrix we need to find gradient
information of latent variables in $\mathbf{H}$ and $\mathbf{G}$ with
$\mathcal{X}$. This gradient information can be obtained using adjoint
methods, automatic differentiation, or finite differencing. However,
implementing adjoint formulation or automatic differentiation becomes
unfeasible for proprietary or closed-source simulation tools unless these
methods are already in place. On the other hand, finite difference
approximations, although more broadly applicable, tend to scale poorly as the
number of input variables increases. The application of these methods is also
computationally expensive as we need to find $2k$ gradient computations
corresponding to each latent variable in $\mathbf{H}$ and $\mathbf{G}$.
Instead, we will use a multi-fidelity model-based active subspace method, a
simple but effective technique to find the active subspace. This technique has
previously been used to construct scalar-based surrogate models for high-
dimensional inputs and is modified here for applications in ROMs [62].

####  2.2.1 Mathematical Formulation

It is important to remember that we are interested in predicting high-fidelity
field solutions by estimating their latent space coordinates using multi-
fidelity information. We know that each row in the latent variable matrices
$\mathbf{H}$ and $\mathbf{G}$ corresponds to variability in a feature due to a
change in inputs in $\mathcal{X}$. We can consider each feature
$\mathbf{h}_{i}$ and $\mathbf{g}_{i}$ as a scalar-valued function of the set
of inputs. An active subspace in the context of this problem is a low-
dimensional subspace in which most of the variations in $\mathbf{h}_{i}$ and
$\mathbf{g}_{i}$ due to variability in $\mathcal{X}$ are concentrated.

We can predict our quantity of interest using a multi-fidelity surrogate
modeling framework that uses additive correction [59].

$\tilde{h}_{i}(\mathbf{x})=g_{i}(\mathbf{x})+\delta(\mathbf{x})$ (8)

where $\delta(\mathbf{x})$ is the correction term that can be determined by
finding the difference between high- and low-fidelity latent variables at
$m_{1}$ linked data points. We can find $\tilde{h}_{i}(\mathbf{x})$ at unknown
points by replacing $g_{i}(\mathbf{x})$ and $\delta(\mathbf{x})$ with a
surrogate model i.e. $\tilde{g}_{i}(\mathbf{x})$ and
$\tilde{\delta}(\mathbf{x})$. The availability of high-fidelity points is
limited due to computational cost considerations. Hence, we will model
$\tilde{\delta}(\mathbf{x})$ using a linear model

$\tilde{\delta}(\mathbf{x})=a+\mathbf{b}^{T}\mathbf{x}$ (9)

where $a$ and $\mathbf{b}$ can be found by solving the linear least squares
problem

$\underset{a\in\mathbb{R},\mathbf{b}\in\mathbb{R}^{d}}{\text{min}}\sum_{j=1}^{m_{1}}\left[(h_{ij}-g_{ij})-(a+\mathbf{b}^{T}\mathbf{x}_{j})\right]^{2}$
(10)

We would like to highlight at this point that the multi-fidelity model
$\tilde{h}$ from Eq. (8), will be used to find the gradient information needed
to develop the covariance matrix to find an active subspace. We will not use
$\tilde{h}$ to directly predict the latent coordinates of our output field of
interest. We just require $\tilde{h}$ to be accurate enough to find variations
in output latent space coordinates with changes in inputs.

We can generate a relatively large number of low-fidelity data points.
Depending on the cost of obtaining low-fidelity samples, the low-fidelity
solution in Eq. (8) can be replaced with a linear model, a higher-order
polynomial or a radial basis function (RBF) surrogate model. We will use an
RBF surrogate with a non-parametric basis function in this study to represent
the low-fidelity data. Non-parametric basis function does not contain hyper-
parameters. Hence, the process of finding these hyper-parameters which scale
poorly with the increasing number of input variables can be avoided. The
mathematical expression for an RBF surrogate model is given as

$\tilde{g}(\mathbf{x})=\sum_{i=1}^{m_{2}}w_{i}\phi(\lVert\mathbf{x}-\mathbf{x}_{i}\rVert)+\epsilon(\mathbf{x})$
(11)

where $w_{i}$ are the weights of the surrogate model, and $\phi$ represents
the basis function. The polynomial trend is represented by
$\epsilon(\mathbf{x})$. We can now find the multi-fidelity gradient
information

$\nabla\tilde{h}(\mathbf{x})=\nabla\tilde{g}(\mathbf{x})+\nabla\tilde{\delta}(\mathbf{x})=\mathbf{b}+(\epsilon^{\prime}(\mathbf{x})+\mathbf{\Phi^{\prime}}\mathbf{w})$
(12)

Here, $\epsilon^{\prime}(\mathbf{x}$) is the gradient of the polynomial trend,
and $\mathbf{\Phi^{\prime}}$ represents the gradient of the kernel matrix. For
a non-parametric cubic kernel,
$\phi(\mathbf{x},\mathbf{x}_{i})=\lVert\mathbf{x}-\mathbf{x}_{i}\rVert^{3}$,
gradient of kernel matrix can be found out using

$\mathbf{\Phi^{\prime}}=3\lVert\mathbf{x}-\mathbf{x}_{i}\rVert(\mathbf{x}-\mathbf{x}_{i})$
(13)

The covariance matrix can then be found using the outer product of the
gradient with itself.

$\mathbf{C}_{MF}=\int\nabla\tilde{h}(\mathbf{x})\nabla\tilde{h}(\mathbf{x})^{T}\rho(\mathbf{x})\,dx$
(14)
$\mathbf{{C}}_{MF}=\int\left(\mathbf{b}+(\epsilon^{\prime}(\mathbf{x})+\mathbf{\Phi^{\prime}}\mathbf{w}\right))\left(\mathbf{b}+(\epsilon^{\prime}(\mathbf{x})+\mathbf{\Phi^{\prime}}\mathbf{w}\right))^{T}\rho(\mathbf{x})\,dx$
(15)

where $\rho(\mathbf{x})$ is the probability density on $\mathcal{X}$. We can
solve the integration in Eq. (15) numerically to obtain $\mathbf{{C}}_{MF}$.
The covariance estimator is multi-fidelity in nature as the gradient
information used has been approximated using a multi-fidelity framework. Since
matrix $\mathbf{{C}}_{MF}$ is symmetric, it can be diagonalized as

$\mathbf{{C}}_{MF}=\mathbf{{W}{\Lambda}}{\mathbf{W}}^{T}$ (16)

In the above equation,
$\mathbf{{\Lambda}}=diag(\lambda_{1},\lambda_{1},\dots,\lambda_{d})$ is a
diagonal matrix whose elements are the eigenvalues of $\mathbf{{C}_{MF}}$
organized in descending order, i.e., $\lambda_{1}\geq\cdots\geq\lambda_{d}\geq
0$. Also, $\mathbf{{W}}=[\mathbf{w}_{1},\dots,\mathbf{w}_{d}]$ is a orthogonal
matrix where the $i$-th column $\mathbf{w}_{i}$ is the eigenvector
corresponding to $\lambda_{i}$. A higher eigenvalue here means a greater
variation in response gradient, we can examine the eigenvalue decay or use a
pre-defined criterion to separate the largest $l$ eigenvalues with their
corresponding eigenvectors such that

$\mathbf{{\Lambda}}=\begin{bmatrix}\boldsymbol{{\Gamma}}_{AS}&0\\\
0&\boldsymbol{{\Gamma}}_{\perp}\\\
\end{bmatrix},\;\mathbf{{W}}=\begin{bmatrix}\mathbf{{W}}_{AS}&\mathbf{W}_{\perp}\\\
\end{bmatrix}$ (17)

The selected $l$ eigenvectors
$\mathbf{{W}}_{AS}=\left\\{\mathbf{w}_{i}\,|\,i=1,\dots,l\right\\}$ form the
active subspace while the remaining $d-l$ eigenvectors are defined as the
inactive subspace.

The multi-fidelity model-based active subspace approach is expected to be
computationally advantageous as (1) it effectively reduces the input dimension
mitigating the effects of the curse of dimensionality, and (2) it uses cheap-
to-evaluate surrogate models to predict gradient information. The methodology
is expected to work best in cases where high-fidelity simulation output is
expensive to evaluate and low-fidelity simulation is a good approximation of
it. The approach also assumes the existence of an active subspace, for most
engineering problems this is a valid assumption as they generally exhibit low-
dimensional structure in their input-output relationships. The main steps of
the multi-fidelity model-based active subspace algorithm are outlined in the
Algorithm 2.

Algorithm 2 Multi-Fidelity Model-Based Active Subspace

Input: $m_{1}$ samples of $h(\mathbf{x})$, $m_{2}$ samples of $g(\mathbf{x})$
Output: Set of eigenvectors $\mathbf{{W}}_{AS}$ forming an active subspace

1: Represent high-fidelity quantity of interest as
$\tilde{h}(\mathbf{x})=\tilde{g}(\mathbf{x})+\tilde{\delta}(\mathbf{x})$

2: Compute correction term
$\tilde{\delta}(\mathbf{x})=a+\mathbf{b}^{T}\mathbf{x}$, where $a$,
$\mathbf{b}$ are minimizers of:
$\underset{a\in\mathbb{R},\mathbf{b}\in\mathbb{R}^{d}}{\text{min}}\sum_{j=1}^{m_{1}}\left[(h_{j}-g_{j})-(a+\mathbf{b}^{T}\mathbf{x}_{j})\right]^{2}$

3: Approximate $\tilde{g}(\mathbf{x})$ as:
$\tilde{g}(\mathbf{x})=\sum_{i=1}^{m_{2}}w_{i}\phi(||\mathbf{x}-\mathbf{x}_{i}||)+\epsilon(\mathbf{x})$

4: Compute gradient
$\nabla\tilde{h}(\mathbf{x})=\mathbf{b}+(\epsilon^{\prime}(\mathbf{x})+\mathbf{\Phi^{\prime}}\mathbf{w})$

5: Compute the eigenpairs of covariance matrix $\mathbf{C}_{MF}$ using:
$\mathbf{C}_{MF}=\int\nabla\tilde{h}(\mathbf{x})\nabla\tilde{h}(\mathbf{x})^{T}\rho(\mathbf{x})\,dx=\mathbf{{W}{\Lambda}}{\mathbf{W}}^{T}$

6: Select $l$ eigenvectors
$\mathbf{{W}}_{AS}=\left\\{\mathbf{w}_{i}\,|\,i=1,\dots,l\right\\}$ using the
eigenvalue spectrum.

7: return $\mathbf{{W}}_{AS}$

###  2.3 Multi-Fidelity Reduced Order Model in Active Subspace

After the application of multi-fidelity model-based active subspace, the
dimension reduction in input space is $\mathbb{R}^{d}\mapsto\mathbb{R}^{l}$,
where $l<d$. We can now construct a multi-fidelity regression model mapping
low-dimensional input space to high and low-fidelity latent variables, i.e.
$\mathbf{h}_{i}$ and $\mathbf{g}_{i}$. In this study, we will use Hierarchical
Kriging (HK) [63], a relatively simple and efficient multi-fidelity regression
technique. The formulation for HK is given by

$\tilde{h}_{i}(\mathbf{\xi})=\alpha\tilde{g}_{i}(\mathbf{\xi})+\mathbf{\omega^{T}}\mathbf{r(\mathbf{\xi})}$
(18)

where, $\mathbf{\xi}=\mathbf{{W}}_{AS_{i}}\mathbf{x}$ is the reduced dimension
input space, $\tilde{h}_{i}$ and $\tilde{g}_{i}$ are Kriging predictors of
high and low-fidelity latent variables respectively, $\alpha$ is a scaling
factor, $\mathbf{\omega}$ is a weight vector, and $\mathbf{r(\mathbf{\xi})}$
is a kernel function.

A separate multi-fidelity active subspace is computed for each latent variable
in $\mathbf{H}$ and a corresponding surrogate model fit $\tilde{h}_{i}$ is
carried out i.e. a total of $k$ different multi-fidelity active subspaces and
multi-fidelity regression models are constructed. This multi-fidelity
regression model will suffer less from the curse of dimensionality as it has
been built over a reduced dimensional input space. New high-fidelity field
solutions can now be predicted by estimating their latent variables using Eq.
(18) and mapping them back to output space using high-fidelity POD modes
$\boldsymbol{\Theta}_{k}$.

$\mathbf{\tilde{y}}=\sum_{i=1}^{k}\boldsymbol{\theta}_{i}\tilde{h}_{i}(\mathbf{\xi})$
(19)

###  2.4 Proposed Multi-Fidelity ROM Method for High-Dimensional Input Space

The main contribution of this work is the formulation of a computationally
efficient, multi-fidelity, non-intrusive ROM for problems having high-
dimensional design space. The methodology uses Procrustes manifold alignment
to carry out output dimension reduction. The curse of dimensionality is then
alleviated by using a multi-fidelity model-based active subspace approach. The
proposed methodology is henceforth referred to as MF-PCAS. The overall
procedure for constructing an MF-PCAS model is illustrated in Algorithm 3 and
in Fig. 2, and is summarized below:

  1. 1.

Generate Data: Sample $m_{1}$ designs from high-dimensional input space
$\mathcal{X}$. Run high- and low-fidelity models to generate matrices
$\mathbf{Y}$ and $\mathbf{Z}_{L}$ containing field solutions in their columns.
Sample an additional $m_{2}-m_{1}$ design from $\mathcal{X}$, and run only a
low-fidelity model to generate matrix $\mathbf{Z}_{U}$.

  2. 2.

Reduce Output Dimensionality Using Procrustes Manifold Alignment: Apply
Procrustes manifold alignment to align the low-dimensional latent spaces of
matrices $\mathbf{Y}$ and $\mathbf{Z}=[\mathbf{Z}_{L},\mathbf{Z}_{U}]$. This
can be achieved with the following sub-steps:

    1. (a)

Apply POD to obtain the low-dimensional latent variables $\mathbf{H}$ and
$\mathbf{S}=[\mathbf{S}_{L},\mathbf{S}_{U}]$, as well as the orthonormal bases
$\boldsymbol{\Theta}_{k}$ and $\boldsymbol{\Xi}_{k}$ for high- and low-
fidelity fields respectively.

    2. (b)

Find an optimal translation vector, transformation matrix, and scaling factor
by performing the Procrustes analysis of $\mathbf{H}$ and $\mathbf{S}_{L}$.

    3. (c)

Get the low-fidelity latent variables $\mathbf{G}$ by using the calculated
optimal transformation, ensuring that they have the same number of dimensions
and are located on the same manifold as $\mathbf{H}$.

  3. 3.

Reduce Input Dimensionality Using Multi-Fidelity Model-Based Active Subspace:
Find a separate active subspace for each latent variable in $\mathbf{H}$ using
multi-fidelity model-based heuristics. The process is described in the
following sub-steps:

    1. (a)

Develop a multi-fidelity surrogate modeling framework to predict the high-
fidelity latent variable $\tilde{h}_{i}(\mathbf{x})$, using the low-fidelity
latent variable $\tilde{g}_{i}(\mathbf{x})$ and a discrepancy term
$\tilde{\delta}(\mathbf{x})$.

    2. (b)

Find the covariance matrix $\mathbf{{C}}_{MF}$ using the outer product of
gradient $\nabla\tilde{h}_{i}(\mathbf{x})$ with itself, and apply eigenvalue
decomposition to it.

    3. (c)

Analyze the eigenvalue spectrum to find the first $l$ eigenvectors that define
the active subspace $\mathbf{{W}}_{AS}$.

    4. (d)

Repeat the process $k$ times to find a different active subspace for each
latent variable in a $k$-dimensional latent variable matrix $\mathbf{H}$.

  4. 4.

Train Regression Model: Train a set of $k$ different hierarchical Kriging
models
$\mathbf{\tilde{h}}(\mathbf{\xi})=[\tilde{h}_{1}(\mathbf{\xi}_{1}),\dots,\tilde{h}_{k}(\mathbf{\xi}_{k})]$.
These models are built over reduced dimensional input space
$\mathbf{\xi}_{i}=\mathbf{{W}}_{AS_{i}}\mathbf{x}$, and combine information
from both $\mathbf{H}$ and $\mathbf{G}$.

  5. 5.

Predict Output Field of Interest: For an out-of-sample design point
$\mathbf{x}^{*}$, we can predict output field using following steps:

    1. (a)

Predict the corresponding latent variable using
$\mathbf{\tilde{h}^{*}}(\mathbf{\xi^{*}})=[\tilde{h}^{*}_{1}(\mathbf{\xi^{*}}_{1}),\dots,\tilde{h}^{*}_{k}(\mathbf{\xi}_{k})]$,
where $\mathbf{\xi}^{*}_{i}=\mathbf{{W}}_{AS_{i}}\mathbf{x}^{*}$.

    2. (b)

Reconstruct the output field using $\boldsymbol{\Theta}_{k}$ and
$\mathbf{\tilde{h}^{*}}$.

Algorithm 3 Proposed MF-PCAS method

Input: High-fidelity model $\textit{f}_{y}:\mathcal{X}\mapsto\mathbf{Y}$, low-
fidelity model $\textit{f}_{z}:\mathcal{X}\mapsto\mathbf{Z}$
Output: Predicted high-fidelity output field $\mathbf{\tilde{y}}$

1: Reduce output dimensionality using Procrustes manifold alignment
$\mathbf{H},\mathbf{G},\boldsymbol{\Theta}_{k}=Algorithm1(\mathbf{Y},\mathbf{Z})$

2: for $i=1$ to $k$ do

3: Find active subspace
$\mathbf{{W}}_{AS_{i}}=Algorithm2(\mathbf{H}[i,:],\mathbf{G}[i,:])$

4: Train HK model for $\tilde{h}_{i}(\mathbf{{W}}_{AS_{i}}\mathbf{x})$

5: end for

6: For a new design point $\mathbf{x}^{*}$, predict
$\mathbf{\tilde{y}}=\sum_{i=1}^{k}\boldsymbol{\theta}_{i}\tilde{h}^{*}_{i}(\mathbf{{W}}_{AS_{i}}\mathbf{x}^{*})$

7: return $\mathbf{\tilde{y}}$

Figure 2: A graphical representation of the MF-PCAS method to develop ROMs for
high-dimensional design space.

##  3 Application Problems

The benefits of our proposed multi-fidelity ROM approach are evaluated using
two application problems. The first application problem is the aerodynamic
analysis of the RAE 2822 transonic airfoil. For this application problem, the
effectiveness of our proposed approach is evaluated using different high and
low-fidelity combinations, and parametrizations. The second test problem
involves the more complex 3D geometry of the CRM wing. For both test problems,
the quantity of interest predicted by the models is the pressure coefficient
($C_{p}$) of the flow field. The motivation for selecting the $C_{p}$ field
stems from its important role in aircraft design, as it critically influences
aerodynamic performance and is necessary for aero-structural design. Both test
cases are analyzed with CFD simulations that are performed using the open-
source SU2 code [64]. In the latter parts of this section, performance metrics
used to quantify the accuracy of the proposed approach are also introduced.

###  3.1 Transonic Flow over RAE 2822 Airfoil

Two-dimensional compressible CFD simulations of the RAE 2822 airfoil are
performed to evaluate the flow field around an airfoil. RAE 2822 airfoil is
the starting geometry for the second benchmark problem for the AIAA
aerodynamic design optimization discussion group (ADODG) as it has been used
to evaluate many optimization methods [65, 66]. Reynolds-averaged-Navier-
Stokes (RANS) based simulations are carried out at a free stream flight Mach
and Reynolds number of $M_{\infty}=0.73$ and $Re_{\infty}=6.5\times 10^{6}$.
The Spalart-Allmaras model is used for turbulence modeling [67, 68]. A steady
flow solution is obtained using a backward Euler Scheme. High-fidelity flow
field results are obtained using an unstructured grid of 96,913 nodes (L3) as
shown in Fig. 3. Two coarser low-fidelity grids with 19,062 (L2) and 7,485
(L1) nodes are also generated, respectively. The L1 grid is solved using an
inviscid solver, which requires a much smaller grid size due to the absence of
near-wall refinements normally needed to model boundary layers. Table 5
summarizes the three fidelity levels. The reader is referred to [34] for the
grid convergence study for this test problem.

Figure 3: Close-up view of the grids used for RAE 2822 test case: (a) L1 grid
(7,485) nodes, (b) L2 grid (19,062 nodes), and (c) L3 grid (96,913 nodes).
Table 5: Summary of RAE 2822 CFD simulations for different fidelity levels.

Level | Num. of Nodes | CFD Solver | Cost/Sample [CPU-hr]
---|---|---|---
L1 | 7,485 | Inviscid | 0.008
L2 | 19,062 | RANS | 0.05
L3 | 96,913 | RANS | 0.329

∗CPU times are obtained using Intel Xeon Gold 6226 CPUs

###  3.2 Transonic Flow Over NASA CRM Wing

A second test case involving the aerodynamic analysis of NASA’s common
research model (CRM) wing is considered to evaluate our proposed methodology
on a problem more representative of an industrial application. The CRM
aircraft geometry is also one of the benchmark problems for the ADODG and has
been designed to test and develop aerodynamic prediction [48], and shape
optimization methods [69, 65]. A CFD setup, similar to RAE 2822 is used. Free
stream conditions of $M_{\infty}=0.85$ and $Re_{\infty}=5\times 10^{6}$,
similar to one defined by the ADODG are selected. A structured grid of $3.7$
million nodes used for high-fidelity simulations (L3) is shown in Fig. 4. This
grid was also used previously in a multi-level optimization method [69] and
has been made public as part of the ADODG benchmark problem. Two coarser grids
having $0.46$ million (L2) and $0.26$ million (L1) nodes are generated for
low-fidelity solutions. The L1 grid is unstructured and solved using an
inviscid solver and hence does not include near-wall refinement. Table. 6
gives details for different fidelity grids and their computational time.

Figure 4: Close-up view of grids used for CRM wing test case: (a) L1
unstructured grid (0.26M nodes), (b) L2 structured grid (0.46M nodes), and (c)
L3 structured grid (3.7M nodes). Only surface grids are shown in the figure
for visualization purposes. Table 6: Summary of CRM wing CFD simulations for
different fidelity levels.

Level | Num. of Nodes | CFD Solver | Cost/Sample [CPU-hr]
---|---|---|---
L1 | 251,821 | Inviscid | 1.162
L2 | 464,325 | RANS | 3.384
L3 | 3,659,337 | RANS | 91.20

∗CPU times are obtained using Intel Xeon Gold 6226 CPUs

###  3.3 Geometric Parametrization

We are interested in predicting field solutions for problems having high-
dimensional inputs. Predicting pressure fields for highly parameterized shapes
represents a problem having both high-dimensional inputs and outputs. Such
problems are frequently encountered in aerodynamic shape optimization studies,
where a large number of design variables parameterize aerodynamic geometries.
Varying these variables produces complex aerodynamic shapes whose performance
is then evaluated and optimized to meet a certain performance target.

Different shape parametrization methods exist in the literature such as the
class shape transformation (CST) [70], splines [71], and Hick Hennes bump
function [72]. A detailed discussion on different methods used for aerodynamic
shape parametrization has been given by Masters et al. [73]. In this study,
the free-form deformation (FFD) method is used. The geometry of the RAE 2822
airfoil and CRM wing is enclosed in a rectangular FFD box. The nodes of the
FFD box are then displaced causing a change in the shape of the enclosed
geometry. The deformation of the FFD node is then propagated into the rest of
the grid using a linear elasticity approach. The work of Kenway et al. [74]
can be referred to for a detailed description of the FFD methodology.

For the airfoil test case, four different FFD boxes having a dimension of
$7\times 2$, $12\times 2$, $22\times 2$, and $32\times 2$, are generated as
shown in Fig. 5. The four corner FFD nodes are held fixed to prevent the
deformation from altering the airfoil angle of attack. The remainder of the
nodes is allowed to move only in the vertical direction with a displacement
limit of $\pm 0.02$ chord length. Overall, the current setup provides four
different scenarios having an input dimension of 10, 20, 40, and 60,
respectively. Similarly, an FFD box of $15\times 8\times 2$ is generated for
the CRM wing case having an input dimension of $240$ as shown in Fig. 5. All
the nodes are allowed to move only in the vertical direction with a higher
displacement limit of $\pm 0.03$ chord length.

Figure 5: FFD box used for test cases

###  3.4 Performance Metrics

We can quantify the prediction accuracy of our proposed approach by finding
the error between the predicted field solution and the actual field solution.
For $n_{\text{v}}$ fields computed for out-of-sample designs, that is, designs
not involved in the model’s training phase, we calculate the root-mean-squared
(RMS) prediction error $\tilde{E}(\mathbf{y})$ as

$\tilde{E}(\mathbf{y})=\sqrt{\frac{1}{n_{v}}\sum_{i=1}^{n_{v}}\lVert\mathbf{y}^{*}_{i}-\mathbf{\tilde{y}}^{*}_{i}\rVert^{2}}$
(20)

where $\mathbf{y}_{i}^{*}$ represents the $i$-th out-of-sample solution and
$\mathbf{\tilde{y}}_{i}$ denotes the prediction made by the ROM.

The framework introduced in this study integrates input and output
dimensionality reduction with a regression model. This enables us to split the
total prediction error into components of reconstruction and regression
errors. The error due to reconstruction, $\tilde{E}_{RC}(\mathbf{y})$,
pertains to the error arising from the mapping between the reduced and the
original spaces, or in the context of POD, it is the error attributable to the
accuracy of the POD modes. The reconstruction error
$\tilde{E}_{RC}(\mathbf{y})$ can be expressed as

$\tilde{E}_{RC}(\mathbf{y})=\sqrt{\frac{1}{n_{v}}\sum_{i=1}^{n_{\text{v}}}\lVert(\mathbf{I}-\boldsymbol{\Theta}_{k}\boldsymbol{\Theta}_{k}^{T})\mathbf{y}_{i}^{*}\rVert^{2}}$
(21)

The regression error is defined by the discrepancies within the actual and
predicted latent space coordinates due to imperfections in the regression
models, whether from inaccuracies in determining the active subspace
$\mathbf{\hat{W}}_{AS}$ or the fit of the HK regression model. The regression
error $\tilde{E}_{RG}(\mathbf{y})$ is given as

$\tilde{E}_{RG}(\mathbf{y})=\sqrt{\sum_{i=1}^{n_{\text{v}}}\lVert\boldsymbol{\Theta}_{k}(\boldsymbol{\Theta}_{k}^{T}\mathbf{y}_{i}^{*}-\mathbf{\tilde{\mathbf{H}}}_{i})\rVert^{2}}$
(22)

where $\mathbf{\tilde{\mathbf{H}}}_{i}$ represents the estimated latent space
coordinates for $\mathbf{y}_{i}^{*}$. Given that the reconstruction and
regression errors act as orthogonal components of the total error, we can
combine $\tilde{E}_{RC}(\mathbf{y})$ and $\tilde{E}_{RG}(\mathbf{y})$ into the
total error $\tilde{E}$, as demonstrated in [48], in the following manner:

$\tilde{E}(\mathbf{y})=\sqrt{\tilde{E}_{RC}(\mathbf{y})^{2}+\tilde{E}_{RG}(\mathbf{y})^{2}}$
(23)

##  4 Results and Discussion

In this section, we present the results of evaluating the predictive
performance of the proposed MF-PCAS method for the RAE 2822 airfoil and NASA
CRM wing test cases. We compare the accuracy and computational cost of the
multi-fidelity MF-PCAS method to the single-fidelity PCAS method to assess the
benefits of considering multiple levels of fidelity in creating ROMs for
problems with high-dimensional input space. We also examine the error caused
by the model-based active subspace and present visual comparisons of the
predicted $\tilde{C}_{p}$ field errors. Finally, we evaluate the performance
of the proposed MF-PCAS method as the input dimensionality changes and compare
it to the MA-ROM method. To obtain our results, we use datasets containing
$2200$ CFD simulations for the RAE 2822 airfoil and $800$ CFD simulations for
the CRM wing test case, These datasets are created for each fidelity level
described in Tables 5 and 6. A subset of $m_{1}$ training samples is randomly
selected from both high- and low-fidelity datasets. A multi-fidelity ratio
parameter $\tau=m_{2}/m_{1}$ is used to select an additional $m_{2}-m_{1}$
low-fidelity samples. The MF-PCAS and MA-ROM models are then trained using the
selected high- and low-fidelity samples, while the PCAS model is trained using
only high-fidelity samples. This process is repeated 20 times or more for each
setup, and the mean prediction error is documented for each combination of the
size of the training samples and the level of fidelity. In this section, all
the ROMs use the fidelity level L3 as the high-fidelity output field of
interest and combine it with one of the other fidelity levels to form a multi-
fidelity model.

###  4.1 Cost vs Accuracy Trade-off

The objective of the proposed multi-fidelity model is to predict the pressure
coefficient ($C_{p}$) of the high-fidelity flow field L3. The cost vs accuracy
trade-off section demonstrates the benefit of combining the expensive-to-
obtain L3 data with cheaper data sources. Figures 6 and 7 present the average
prediction errors ($\tilde{E}(C_{p})$) of the pressure field for both the
single-fidelity PCAS and the multi-fidelity MF-PCAS methods as a function of
the number of high-fidelity training samples ($m_{1}$) and total training cost
[CPU-hr]. Figure 6 shows the results of combining the fidelity levels L3 + L2,
while Fig. 7 shows the results of combining L3 + L1 data. Both figures display
the errors in predicting the $C_{p}$ field around the RAE 2822 airfoil with a
$d=40$ parametrization.

From the left side of both figures, it is clear that $\tilde{E}(C_{p})$
decreases nearly exponentially as the number of training samples increases for
all models. The results show that combining L3 with either L2 or L1 data using
MF-PCAS consistently reduces $\tilde{E}(C_{p})$ for a given $m_{1}$. The
reduction in $\tilde{E}(C_{p})$ is higher when using L2 data compared to L1.
This is because the L1 data uses a simpler inviscid model and a coarser grid,
resulting in a weaker correlation with the high-fidelity data and reduced
prediction accuracy of MF-PCAS. Increasing $\tau$ also improves the accuracy
of the multi-fidelity method, although with diminishing return as $\tau$
increases to a large value. The MF-PCAS method with both combinations
outperforms the single-fidelity PCAS method when $m_{1}$ is relatively low. As
$m_{1}$ increases to large values, the predictive accuracy of PCAS catches up
with MF-PCAS. This suggests that with a large enough training dataset, the
single-fidelity model can adequately capture the relationship between inputs
and outputs for this test case, rendering the multi-fidelity model
superfluous.

The variation of $\tilde{E}(C_{p})$ with computational cost to construct the
single- and multi-fidelity ROM is shown on the right-hand side of Figs. 6 and
7. The computational cost only includes the cost of generating training data
as the cost of constructing the ROM is negligible in comparison. It is worth
noting that in practical scenarios, the computational cost of generating
training data is often limited by a computational budget. The MF-PCAS method
provides better prediction accuracy for a fixed computational budget than
single-fidelity PCAS. The prediction error decreases when $\tau$ is increased
from 2 to 4 for the same training cost. However, beyond a threshold value of
$\tau$, adding more low-fidelity data no longer improves the model’s training.
When the available computational budget is sufficiently large, PCAS and MF-
PCAS give similar prediction errors for all values of $\tau$. For such
scenarios, the cost of high-fidelity simulations dominates the training cost,
and low-fidelity data has minimal impact on both cost and accuracy. Comparing
the right-hand sides of Fig. 6 and 7, we observe that although the combination
of L3 + L2 provides a lower prediction error than the combination of L3 + L1
for the same $m_{1}$ and $\tau$, the cost of generating an L2 sample is $6.25$
times higher than that of an L1 sample. As a result, both combinations, L3 +
L2 and L3 + L1 offer a similar cost-accuracy trade-off.

Figure 6: Variation of prediction error $\tilde{E}(C_{p})$ with number of
high-fidelity samples $m_{1}$ and training cost for L3 and L2 datasets.
Results for RAE 2822 test case with $d=40$. Figure 7: Variation of prediction
error $\tilde{E}(C_{p})$ with number of high-fidelity samples $m_{1}$ and
training cost for L3 and $L1$ datasets. Results for RAE 2822 test case with
$d=40$

Figures 8 and 9 show the average prediction errors $\tilde{E}(C_{p})$ of the
pressure field for PCAS and MF-PCAS methods as a function of $m_{1}$ and total
training cost for CRM wing test case with $d=240$. Unlike the airfoil test
case, we only consider the pressure distribution on the wing surface as
generally most design applications are focused on predicting aerodynamic and
structural loads on the wing surface. We can observe that MF-PCAS using both
L3 + L2 and L3 + L1 combinations outperforms single-fidelity PCAS for all
values of $m_{1}$ and total training cost. Even for relatively large values of
$m_{1}$ and training cost, the accuracy of the single-fidelity model fails to
match the prediction accuracy of the MF-PCAS method. The 3D wing geometry
typically has a more complex flow field and a higher degree of parametrization
than the 2D airfoil, making it more difficult to model accurately with single-
fidelity PCAS. The use of MF-PCAS with different levels of accuracy allows for
a more comprehensive representation of the system. From Figs. 8 and 9 we can
observe that increasing $\tau$, for a fixed $m_{1}$ or computational budget
results in a continuous reduction in $\tilde{E}(C_{p})$. This shows that
adding more low-fidelity data allows the model to learn the complexities of
the flow field better.

Similar, to the airfoil test case we see that L3 + L2 and L3 + L1 combinations
offer a similar cost vs accuracy trade-off for the CRM wing test case. For the
sake of brevity, in the remaining part of this section, we will only show and
discuss results for the L3 + L2 fidelity combination for both test cases.

Figure 8: Variation of prediction error $\tilde{E}(C_{p})$ with number of
high-fidelity samples $m_{1}$ and training cost for L3 and L2 datasets.
Results for CRM wing test case with $d=240$ Figure 9: Variation of prediction
error $\tilde{E}(C_{p})$ with number of high-fidelity samples $m_{1}$ and
training cost for L3 and L1 datasets. Results for CRM wing test case with
$d=240$

###  4.2 Discussion of Prediction Errors

A detailed breakdown of the prediction errors for the RAE 2822 airfoil and CRM
wing test cases is shown in Tables 7 and 8, respectively. Increasing $m_{1}$
allows for the computation of more accurate and additional POD modes, which
results in a quick decrease in the reconstruction error
$\tilde{E}_{RC}(C_{p})$ for the PCAS model. If an MF-PCAS model is trained
with a similar amount of high-fidelity data $m_{1}$, it will use the same
number of POD modes ($k$) to represent the high-fidelity field as discussed in
Sec. 2. As a result, the MF-PCAS will have a comparable
$\tilde{E}_{RC}(C_{p})$ value to a single-fidelity PCAS model, irrespective of
the value of $\tau$ selected. However, using a multi-fidelity approach allows
for the accurate computation of the active subspace and the enrichment of the
low-dimensional latent space with low-fidelity data. This results in a better
fit for the HK regression model and a reduction of $\tilde{E}_{RG}(C_{p})$.

Table 7: Breakdown of the prediction error of the MF-PCAS and PCAS method. Results are for the RAE 2822 test case with $d=40$ and $\text{RIC}=99\%$. . [HTML]FFFFFF Method | Levels | $m_{1}$ | $\tau$ | $m_{2}$ | k | $\tilde{E}(C_{p})$ | $\tilde{E}_{RG}(C_{p})$ | $\tilde{E}_{RC}(C_{p})$
---|---|---|---|---|---|---|---|---
|  | 50 |  |  | 27 | 0.0839 | 0.0787 | 0.0291
|  | 100 |  |  | 36 | 0.0686 | 0.0653 | 0.02104
PCAS | L3 | 500 | - | - | 49 | 0.0560 | 0.0541 | 0.0143
|  |  | 2 | 200 | 36 | 0.0598 | 0.0560 | 0.02104
|  |  | 4 | 400 | 36 | 0.0567 | 0.0527 | 0.02104
MF-PCAS | L3 + L2 | 100 | 8 | 800 | 36 | 0.0556 | 0.0515 | 0.02104

From Table 8 we can observe that as $\tau$ increases, the regression error
decreases and eventually reaches a magnitude similar to that of the
reconstruction error. For such cases, we can further reduce
$\tilde{E}_{RC}(C_{p})$ by increasing the RIC threshold value. This allows for
the inclusion of additional POD modes for high-fidelity field reconstruction.
However, the additional POD modes tend to capture more fine-grained details in
the high-fidelity field, which can be noise or insignificant variations.
Finding a model-based active subspace and fitting a regression model to these
modes become challenging and result in an increase in $\tilde{E}_{RG}(C_{p})$.
Table 9 shows the breakdown of prediction error for both test cases with a RIC
= $99.99\%$. Comparing the errors in Table 9 with those in Tables 7 and 8, we
can observe that for the same $m_{1}$ and $\tau$, increasing RIC results in an
increase in selected POD modes $k$. As a result we see a decrease in
$\tilde{E}_{RC}(C_{p})$ and an increase in $\tilde{E}_{RG}(C_{p})$. The
overall prediction error only changes by a relatively small amount.

Table 8: Breakdown of the prediction error of the MF-PCAS and PCAS method. Results are for the CRM wing test case with $d=240$ and $\text{RIC}=99\%$. [HTML]FFFFFF Method | Levels | $m_{1}$ | $\tau$ | $m_{2}$ | k | $\tilde{E}(C_{p})$ | $\tilde{E}_{RG}(C_{p})$ | $\tilde{E}_{RC}(C_{p})$
---|---|---|---|---|---|---|---|---
|  | 25 |  |  | 22 | 0.116 | 0.0995 | 0.0605
|  | 75 |  |  | 54 | 0.105 | 0.0990 | 0.0372
PCAS | L3 | 500 | - | - | 118 | 0.0679 | 0.0648 | 0.0203
|  |  | 2 | 150 | 54 | 0.0883 | 0.0801 | 0.0372
|  |  | 4 | 300 | 54 | 0.0640 | 0.0521 | 0.0372
MF-PCAS | L3 + L2 | 75 | 8 | 600 | 54 | 0.05321 | 0.0380 | 0.0372
Table 9: Breakdown of the prediction error of the MF-PCAS method with $\text{RIC}=99.99\%$ . Results are for the RAE 2822 test case with $d=40$ and the CRM Wing Test case with $d=240$. [HTML]FFFFFF Test Case | Levels | $m_{1}$ | $\tau$ | $m_{2}$ | k | $\tilde{E}(C_{p})$ | $\tilde{E}_{RG}(C_{p})$ | $\tilde{E}_{RC}(C_{p})$
---|---|---|---|---|---|---|---|---
|  |  | 2 | 200 | 91 | 0.0590 | 0.0584 | 0.0130
|  |  | 4 | 400 | 91 | 0.0564 | 0.0548 | 0.0130
RAE 2822 | L3 + L2 | 100 | 8 | 800 | 91 | 0.05521 | 0.05364 | 0.0130
|  |  | 2 | 150 | 75 | 0.0884 | 0.0814 | 0.0343
|  |  | 4 | 300 | 75 | 0.0634 | 0.0532 | 0.0343
CRM Wing | L3 + L2 | 75 | 8 | 600 | 75 | 0.05212 | 0.03918 | 0.0343

The error due to the multi-fidelity model-based active subspace is included in
the regression error of the model. We can write the total regression error in
Eq. (22) as the root sum of squares of the errors in predicting each POD mode

$\tilde{E}_{RG}(C_{p})=\sqrt{\sum_{j=1}^{k}\sum_{i=1}^{n_{\text{v}}}\lVert\boldsymbol{\theta}_{j}(\boldsymbol{\theta}_{j}^{T}\mathbf{y}_{i}^{*}-\mathbf{\tilde{h}}_{j})\rVert^{2}}$
(24)
$\tilde{E}_{RG}(C_{p})=\sqrt{\tilde{e}^{2}_{1}+\tilde{e}^{2}_{2}+\dots+\tilde{e}^{2}_{k-1}+\tilde{e}^{2}_{k}}$
(25)

where
$\tilde{e}=\sum_{i=1}^{n_{\text{v}}}\lVert\boldsymbol{\theta}(\boldsymbol{\theta}^{T}\mathbf{y}_{i}^{*}-\mathbf{\tilde{h}})\rVert$
is the error in predicting each POD mode coefficient. From Fig. 10 we can see
that for both test cases, the MF-PCAS model that uses a multi-fidelity active
subspace to reduce input space dimensionality has a lower prediction error for
the first few dominant POD. For the latter noisy modes, both the single-
fidelity and multi-fidelity methods fail to discover a good active subspace
and produce an inaccurate regression model. However, the contribution of these
latter modes is less significant than the first few dominant modes in overall
prediction error, which lessens their impact on the ROM.

Figure 10: Regression error in predicting each POD mode with
$\text{RIC}=99\%$.(a) RAE 2822 test case with $m_{1}=100$ and $d=40$ (b) CRM
wing test case with $m_{1}=75$ and $d=240$

###  4.3 Visualization of Error in Flow Field Prediction

The figures presented in this subsection provide a visual representation of
flow field errors and complement the previous discussions and explanations of
errors observed in the experiments. Figure 11 illustrates the CFD pressure
coefficient ($C_{p}$) field and prediction error ($C_{p}-\tilde{C}_{p}$) field
for the RAE 2822 airfoil test case. The selected airfoil design exhibits
complex flow features on the top surface of the airfoil, including multiple
shocks and separation bubbles. The MF-PCAS method predicts the flow field
accurately, and the prediction error is almost zero for most of the flow
domain. However, spurious oscillations on the top surface of the airfoil are
observed, although the magnitude of the error is lower than that of the
single-fidelity prediction. Additionally, Fig. 11 confirms the previous
observation that beyond a certain threshold, adding further low-fidelity data
has no significant effect on the prediction accuracy of the MF-PCAS method for
the airfoil test case. We can observe that increasing the value of $\tau$ from
4 to 8 does not significantly impact the error contours.

Figure 11: Prediction error ($C_{p}-\tilde{C}_{p}$) in the flow field. Results
for RAE 2822 airfoil test case with $m_{1}=100$ and $d=40$

In Fig. 12, we examine the pressure and predicted error fields for the top
surface of the CRM wing test case, where abrupt pressure changes and complex
flow phenomena typically occur. The comparison of the fields in Fig. 12
indicates that most of the prediction error for all models is concentrated
near a strong shock. Both the single- and multi-fidelity methods utilize POD
for output dimensionality reduction. POD is a linear method and assumes that a
linear combination of variables can capture the dominant patterns in the
field. However, shockwaves represent a highly non-linear phenomenon, and POD
cannot capture their sudden and rapid changes in the field. Nevertheless, the
MF-PCAS method performs better in capturing the shockwave when compared to its
single-fidelity counterpart. Moreover, adding more low-fidelity data by
increasing $\tau$ results in a further reduction in prediction error near the
shock region, which is consistent with the previously observed trend.

Figure 12: Prediction error ($C_{p}-\tilde{C}_{p}$) in the flow field. Results
for CRM wing test case with $m_{1}=75$ and $d=240$

###  4.4 Effect of Input Dimensionality

To evaluate the effect of input dimensionality on the predictive performance
of the MF-PCAS method, we analyze four different airfoil test cases with input
dimensions of 10, 20, 40, and 60, respectively. The test cases are generated
using different FFD parameterizations, as described in Sec. 3. We compare the
results with those of the manifold-aligned ROM (MA-ROM) method, which carries
out output dimension reduction using Procrustes manifold alignment but
develops the ROM over the original high-dimensional input space. For both the
MF-PCAS and MA-ROM methods, we use an HK model with automatic relevance
determination squared-exponential kernel for the prediction of latent POD
modes.

Figure 13 shows the variation of $\tilde{E}(C_{p})$ with $m_{1}$ and $\tau$
for different input dimensions, $d$. As the number of input dimensions
increases, the total prediction error increases for both the MA-ROM and MF-
PCAS methods. An increasing number of input dimensions results in airfoil
shapes with complex flow fields that are difficult to predict. We observe that
when $d=10$, the MA-ROM method outperformed the MF-PCAS method for all values
of $m_{1}$ and $\tau$. For the same number of high-fidelity training samples,
MA-ROM and MF-PCAS are expected to have similar reconstruction errors. When
input dimensionality is low, the MA-ROM method is not expected to suffer from
the curse of dimensionality and would have a low regression error. On the
contrary, finding a multi-fidelity model-based active subspace to reduce input
dimensionality actually introduces additional errors and increases the
regression error for MF-PCAS. When the input dimensionality is increased to
20, the MF-PCAS method accuracy catches up with that of the MA-ROM method for
low $m_{1}$ values. As $m_{1}$ increases to a higher value, the MA-ROM
method’s accuracy improves significantly as sufficient training data becomes
available for the model to overcome dimensionality issues.

Figure 13: Effect of Input Dimensionality on total prediction error
$\tilde{E}(C_{p})$

When the input dimensionality is increased to a sufficiently large value, we
see a clear advantage of using MF-PCAS over MA-ROM. Fig. 13 shows that for
$d=40$ and 60, the MF-PCAS method consistently performs better than the MA-ROM
method. For high-dimensional input problems, a large number of training
samples are needed to fit an accurate prediction model due to the curse of
dimensionality. Reducing input dimensionality by finding a multi-fidelity
model-based active subspace helps to alleviate the effects of the curse of
dimensionality and reduces the regression error and total prediction error
$\tilde{E}(C_{p})$. The performance of the MA-ROM model only catches up with
MF-PCAS for very large $m_{1}$. However, generating such a large amount of
high-fidelity data often becomes computationally infeasible.

Table 10 shows the total prediction error for the MA-ROM and MF-PCAS models
trained using the same number of high- and low-fidelity samples when the input
dimension is 40. The computational cost of training both models is the same as
differences in the model fitting cost were negligible compared to the cost of
generating training data. From the table, we can observe that, for the same
training sample size and cost, MF-PCAS offers around 50% better prediction
accuracy than MA-ROM. This prediction accuracy advantage slightly decreases to
46% when we add more low-fidelity training data by increasing $\tau$ from 2 to
8. Despite this decrease, this experiment clearly shows the advantage of using
the MF-PCAS method over the MA-ROM when the input dimensionality is large.

Table 10: Comparison of total prediction error $\tilde{E}(C_{p})$ for MF-PCAS and MA-ROM. Results are for the RAE 2822 test case with $d=40$. |  |  |  |  |  |  | $\tilde{E}(C_{p})$
---|---|---|---|---|---|---|---
Test Case | Levels | $m_{1}$ | T | $m_{1}$ | k | Cost [CPU-hr] |  [HTML]FFFFFFMF-PCAS |  [HTML]FFFFFFMA-ROM |  [HTML]FFFFFF$\%$ Change
|  |  | 2 | 200 | 36 | 42.6 | 0.0598 | 0.124 | 51.77%
|  |  | 4 | 400 | 36 | 52.3 | 0.0567 | 0.1179 | 51.90%
RAE 2822 | L3+L2 | 100 | 8 | 800 | 36 | 71.7 | 0.0556 | 0.10475 | 46.92%

##  5 Conclusion

This work has successfully demonstrated the efficacy of a multi-fidelity,
parametric, and non-intrusive ROM framework for high-dimensional input spaces.
Our approach integrates machine learning concepts of output field dimension
reduction via POD, Procrustes manifold alignment, input space reduction using
a gradient-free model-based active subspace, and multi-fidelity regression. We
evaluated our approach using a 2D RAE 2822 airfoil and a 3D NASA CRM wing.
These evaluations have shown our methodology to be superior to the single-
fidelity PCAS method, particularly in achieving a favorable cost and accuracy
trade-off. The results indicate that our approach offers comparable predictive
accuracy at a reduced computational cost and outperforms the manifold-aligned
ROM (MA-ROM) by 50% in scenarios with large input dimensions.

While the current work has focused on aerodynamic design cases, the framework
is versatile enough to be applied to any surrogate modeling problem that
involves high-dimensional inputs and outputs and multiple data sources. This
universality underscores the potential for broad application across various
domains of engineering design.

A notable limitation identified in this study is the use of a linear POD
technique for output dimension reduction and linear Procrustes manifold
alignment, which can restrict the model’s ability to accurately predict non-
linear phenomena such as shock waves, as observed in our results. To address
this limitation, future work will concentrate on incorporating non-linear deep
learning-based dimensionality reduction and manifold alignment techniques.
This advancement is expected to enhance the predictive performance of our
model, especially in scenarios involving complex non-linearities like shock
waves.

## References

  * Benner et al. [2015] Benner, P., Gugercin, S., and Willcox, K., “A survey of projection-based model reduction methods for parametric dynamical systems,” _SIAM review_ , Vol. 57, No. 4, 2015, pp. 483–531.
  * Yondo et al. [2018] Yondo, R., Andrés, E., and Valero, E., “A review on design of experiments and surrogate models in aircraft real-time and many-query aerodynamic analyses,” _Progress in aerospace sciences_ , Vol. 96, 2018, pp. 23–61.
  * Forrester and Keane [2009] Forrester, A. I., and Keane, A. J., “Recent advances in surrogate-based optimization,” _Progress in Aerospace Sciences_ , Vol. 45, No. 1-3, 2009, pp. 50–79.  10.1016/j.paerosci.2008.11.001.
  * Xiao et al. [2015] Xiao, D., Fang, F., Buchan, A. G., Pain, C. C., Navon, I. M., and Muggeridge, A., “Non-intrusive reduced order modelling of the Navier-Stokes equations,” _Computer Methods in Applied Mechanics and Engineering_ , Vol. 293, 2015, pp. 522–541.  10.1016/j.cma.2015.05.015.
  * Lu et al. [2019] Lu, K., Jin, Y., Chen, Y., Yang, Y., Hou, L., Zhang, Z., Li, Z., and Fu, C., “Review for order reduction based on proper orthogonal decomposition and outlooks of applications in mechanical systems,” _Mechanical Systems and Signal Processing_ , Vol. 123, 2019, pp. 264–297.
  * Fossati [2015] Fossati, M., “Evaluation of aerodynamic loads via reduced-order methodology,” _AIAA Journal_ , Vol. 53, No. 8, 2015, pp. 2389–2405.  10.2514/1.J053755.
  * Decker et al. [2020] Decker, K., Schwartz, H. D., and Mavris, D., “Dimensionality reduction techniques applied to the design of hypersonic aerial systems,” _AIAA Aviation 2020 Forum_ , 2020, p. 3003.
  * Rajaram et al. [2020a] Rajaram, D., Perron, C., Puranik, T. G., and Mavris, D. N., “Randomized algorithms for non-intrusive parametric reduced order modeling,” _AIAA Journal_ , Vol. 58, No. 12, 2020a, pp. 5389–5407.
  * Behere et al. [2021] Behere, A., Rajaram, D., Puranik, T. G., Kirby, M., and Mavris, D. N., “Reduced order modeling methods for aviation noise estimation,” _Sustainability (Switzerland)_ , Vol. 13, No. 3, 2021, pp. 1–19.  10.3390/su13031120.
  * Chen and Thuerey [2023] Chen, L. W., and Thuerey, N., “Towards high-accuracy deep learning inference of compressible flows over aerofoils,” _Computers & Fluids_, Vol. 250, 2023, p. 105707.  10.1016/J.COMPFLUID.2022.105707.
  * Kashefi et al. [2021] Kashefi, A., Rempe, D., and Guibas, L. J., “A point-cloud deep learning framework for prediction of fluid flow fields on irregular geometries,” _Physics of Fluids_ , Vol. 33, No. 2, 2021.  10.1063/5.0033376.
  * Deng et al. [2023] Deng, Z., Wang, J., Liu, H., Xie, H., Li, B. K., Zhang, M., Jia, T., Zhang, Y., Wang, Z., and Dong, B., “Prediction of transonic flow over supercritical airfoils using geometric-encoding and deep-learning strategies,” _Physics of Fluids_ , Vol. 35, No. 7, 2023.  10.1063/5.0155383.
  * Mufti et al. [2024] Mufti, B., Bhaduri, A., Ghosh, S., Wang, L., and Mavris, D. N., “Shock wave prediction in transonic flow fields using domain-informed probabilistic deep learning,” _Physics of Fluids_ , Vol. 36, No. 1, 2024.
  * Koch et al. [1999] Koch, P. N., Simpson, T. W., Allen, J. K., and Mistree, F., “Statistical Approximations for Multidisciplinary Design Optimization: The Problem of Size,” _Journal of Aircraft_ , Vol. 36, No. 1, 1999, pp. 275–286.  10.2514/2.2435, URL https://arc.aiaa.org/doi/10.2514/2.2435.
  * Kumari and Jayaram [2017] Kumari, S., and Jayaram, B., “Measuring Concentration of Distances—An Effective and Efficient Empirical Index,” _IEEE Transactions on Knowledge and Data Engineering_ , Vol. 29, No. 2, 2017, pp. 373–386.  10.1109/TKDE.2016.2622270.
  * Brunton et al. [2019] Brunton, S. L., Noack, B. R., and Koumoutsakos, P., “Annual Review of Fluid Mechanics Machine Learning for Fluid Mechanics,” _Annu. Rev. Fluid Mech. 2020_ , Vol. 52, 2019, pp. 477–508.  10.1146/annurev-fluid-010719, URL https://doi.org/10.1146/annurev-fluid-010719-.
  * Dietrich et al. [2018] Dietrich, F., Künzner, F., Neckel, T., Köster, G., and Bungartz, H.-J., “Fast and flexible uncertainty quantification through a data-driven surrogate model,” _International Journal for Uncertainty Quantification_ , Vol. 8, No. 2, 2018.
  * Saltelli et al. [2008] Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., and Tarantola, S., _Global sensitivity analysis: the primer_ , John Wiley & Sons, 2008.
  * Neal [1998] Neal, R. M., “Assessing relevance determination methods using DELVE,” _Nato Asi Series F Computer And Systems Sciences_ , Vol. 168, 1998, pp. 97–132.
  * Bouhlel et al. [2016] Bouhlel, M. A., Bartoli, N., Otsmane, A., and Morlier, J., “An Improved Approach for Estimating the Hyperparameters of the Kriging Model for High-Dimensional Problems through the Partial Least Squares Method,” _Mathematical Problems in Engineering_ , Vol. 2016, 2016.  10.1155/2016/6723410.
  * Constantine et al. [2014] Constantine, P. G., Dow, E., and Wang, Q., “Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces,” _SIAM Journal on Scientific Computing_ , Vol. 36, No. 4, 2014, pp. A1500–A1524.  10.1137/130916138, URL http://arxiv.org/abs/1304.2070http://dx.doi.org/10.1137/130916138http://epubs.siam.org/doi/10.1137/130916138.
  * Berguin and Mavris [2015] Berguin, S. H., and Mavris, D. N., “Dimensionality reduction using principal component analysis applied to the gradient,” _AIAA Journal_ , Vol. 53, American Institute of Aeronautics and Astronautics Inc., 2015, pp. 1078–1090.  10.2514/1.J053372.
  * Berguin et al. [2015] Berguin, S. H., Rancourt, D., and Mavris, D. N., “Method to facilitate high-dimensional design space exploration using computationally expensive analyses,” _AIAA Journal_ , Vol. 53, American Institute of Aeronautics and Astronautics Inc., 2015, pp. 3752–3765.  10.2514/1.J054035.
  * Jiang et al. [2020] Jiang, X., Hu, X., Liu, G., Liang, X., and Wang, R., “A generalized active subspace for dimension reduction in mixed aleatory-epistemic uncertainty quantification,” _Computer Methods in Applied Mechanics and Engineering_ , Vol. 370, 2020.  10.1016/j.cma.2020.113240.
  * Lukaczyk et al. [2014] Lukaczyk, T. W., Constantine, P., Palacios, F., and Alonso, J. J., “Active subspaces for shape optimization,” _10th AIAA multidisciplinary design optimization conference_ , 2014, p. 1171.
  * Li et al. [2019] Li, J., Cai, J., and Qu, K., “Surrogate-based aerodynamic shape optimization with the active subspace method,” _Structural and Multidisciplinary Optimization_ , Vol. 59, No. 2, 2019, pp. 403–419.  10.1007/s00158-018-2073-5.
  * Tezzele et al. [2019] Tezzele, M., Demo, N., and Rozza, G., “Shape optimization through proper orthogonal decomposition with interpolation and dynamic mode decomposition enhanced by active subspaces,” _arXiv preprint arXiv:1905.05483_ , 2019.
  * Wei et al. [2023] Wei, J., An, J., Zhang, Q., Zhou, H., and Ren, Z., “Exploiting Active Subspaces for Geometric Optimization of Cavity-Stabilized Supersonic Flames,” _AIAA Journal_ , 2023, pp. 1–12.
  * Constantine et al. [2015] Constantine, P. G., Emory, M., Larsson, J., and Iaccarino, G., “Exploiting active subspaces to quantify uncertainty in the numerical simulation of the HyShot II scramjet,” _Journal of Computational Physics_ , Vol. 302, 2015, pp. 1–20.  https://doi.org/10.1016/j.jcp.2015.09.001, URL https://www.sciencedirect.com/science/article/pii/S002199911500580X.
  * Khatamsaz et al. [2021] Khatamsaz, D., Molkeri, A., Couperthwaite, R., James, J., Arróyave, R., Srivastava, A., and Allaire, D., “Adaptive active subspace-based efficient multifidelity materials design,” _Materials and Design_ , Vol. 209, 2021.  10.1016/j.matdes.2021.110001.
  * Constantine and Doostan [2017] Constantine, P. G., and Doostan, A., “Time-dependent global sensitivity analysis with active subspaces for a lithium ion battery model,” _Statistical Analysis and Data Mining_ , Vol. 10, No. 5, 2017, pp. 243–262.  10.1002/sam.11347.
  * Lam et al. [2020] Lam, R. R., Zahm, O., Marzouk, Y. M., and Willcox, K. E., “Multifidelity dimension reduction via active subspaces,” _SIAM Journal on Scientific Computing_ , Vol. 42, No. 2, 2020, pp. A929–A956.  10.1137/18M1214123.
  * Romor et al. [2023] Romor, F., Tezzele, M., Mrosek, M., Othmer, C., and Rozza, G., “Multi-fidelity data fusion through parameter space reduction with applications to automotive engineering,” _International Journal for Numerical Methods in Engineering_ , Vol. 124, No. 23, 2023, pp. 5293–5311.
  * Mufti et al. [2022] Mufti, B., Chen, M., Perron, C., and Mavris, D. N., “A Multi-Fidelity Approximation of the Active Subspace Method for Surrogate Models with High-Dimensional Inputs,” _AIAA AVIATION 2022 Forum_ , American Institute of Aeronautics and Astronautics Inc, AIAA, 2022.  10.2514/6.2022-3488.
  * Tripathy et al. [2016] Tripathy, R., Bilionis, I., and Gonzalez, M., “Gaussian processes with built-in dimensionality reduction: Applications to high-dimensional uncertainty propagation,” _Journal of Computational Physics_ , Vol. 321, 2016, pp. 191–223.  10.1016/j.jcp.2016.05.039.
  * Tsilifis et al. [2021] Tsilifis, P., Pandita, P., Ghosh, S., Andreoli, V., Vandeputte, T., and Wang, L., “Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes,” _Computer Methods in Applied Mechanics and Engineering_ , Vol. 386, 2021.  10.1016/j.cma.2021.114147.
  * Gautier et al. [2022] Gautier, R., Pandita, P., Ghosh, S., and Mavris, D., “A Fully Bayesian Gradient-Free Supervised Dimension Reduction Method using Gaussian Processes,” _International Journal for Uncertainty Quantification_ , Vol. 12, No. 2, 2022.
  * Vohra et al. [2020] Vohra, M., Nath, P., Mahadevan, S., and Tina Lee, Y. T., “Fast surrogate modeling using dimensionality reduction in model inputs and field output: Application to additive manufacturing,” _Reliability Engineering and System Safety_ , Vol. 201, 2020.  10.1016/j.ress.2020.106986.
  * Guo et al. [2023] Guo, Y., Mahadevan, S., Matsumoto, S., Taba, S., and Watanabe, D., “Investigation of Surrogate Modeling Options with High-Dimensional Input and Output,” _AIAA Journal_ , Vol. 61, No. 3, 2023, pp. 1334–1348.
  * Demo et al. [2019] Demo, N., Tezzele, M., and Rozza, G., “A non-intrusive approach for the reconstruction of POD modal coefficients through active subspaces,” _Comptes Rendus Mécanique_ , Vol. 347, No. 11, 2019, pp. 873–881.
  * Ji et al. [2022] Ji, Y., Xiao, N. C., and Zhan, H., “High dimensional reliability analysis based on combinations of adaptive Kriging and dimension reduction technique,” _Quality and Reliability Engineering International_ , Vol. 38, No. 5, 2022, pp. 2566–2585.  10.1002/qre.3091.
  * Rajaram et al. [2020b] Rajaram, D., Gautier, R. H., Perron, C., Pinon-Fischer, O. J., and Mavris, D., “Non-intrusive parametric reduced order models with high-dimensional inputs via gradient-free active subspace,” _AIAA AVIATION 2020 FORUM_ , 2020b, p. 3184.
  * O’Leary-Roseberry et al. [2022] O’Leary-Roseberry, T., Villa, U., Chen, P., and Ghattas, O., “Derivative-informed projected neural networks for high-dimensional parametric maps governed by PDEs,” _Computer Methods in Applied Mechanics and Engineering_ , Vol. 388, 2022.  10.1016/j.cma.2021.114199.
  * Forrester et al. [2007] Forrester, A. I., Sóbester, A., and Keane, A. J., “Multi-fidelity optimization via surrogate modelling,” _Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences_ , Vol. 463, No. 2088, 2007.  10.1098/rspa.2007.1900.
  * Peherstorfer et al. [2018] Peherstorfer, B., Willcox, K., and Gunzburger, M., “Survey of multifidelity methods in uncertainty propagation, inference, and optimization,” _Siam Review_ , Vol. 60, No. 3, 2018, pp. 550–591.
  * Bertram et al. [2018] Bertram, A., Othmery, C., and Zimmermannz, R., “Towards real-time vehicle aerodynamic design via multi-fidelity data-driven reduced order modeling,” _AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, 2018_ , Vol. 0, American Institute of Aeronautics and Astronautics Inc, AIAA, 2018.  10.2514/6.2018-0916.
  * Wang et al. [2011] Wang, C., Krafft, P., Mahadevan, S., Ma, Y., and Fu, Y., “Manifold alignment,” _Manifold Learning: Theory and Applications_ , Vol. 510, 2011.
  * Perron [2020] Perron, C., “Multi-fidelity reduced-order modeling applied to fields with inconsistent representations,” Ph.D. thesis, Atlanta: Georgia Institute of Technology, 2020.
  * Perron et al. [2021] Perron, C., Rajaram, D., and Mavris, D. N., “Multi-fidelity non-intrusive reduced-order modelling based on manifold alignment,” _Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences_ , Vol. 477, No. 2253, 2021.  10.1098/rspa.2021.0495.
  * Perron et al. [2022] Perron, C., Sarojini, D., Rajaram, D., Corman, J., and Mavris, D., “Manifold alignment-based multi-fidelity reduced-order modeling applied to structural analysis,” _Structural and Multidisciplinary Optimization_ , Vol. 65, No. 8, 2022.  10.1007/s00158-022-03274-1.
  * Decker et al. [2023] Decker, K., Iyengar, N., Rajaram, D., Perron, C., and Mavris, D., “Manifold Alignment-Based Nonintrusive and Nonlinear Multifidelity Reduced-Order Modeling,” _AIAA Journal_ , Vol. 61, No. 1, 2023, pp. 454–474.  10.2514/1.J061720.
  * Li et al. [2022] Li, J., Du, X., and Martins, J. R., “Machine learning in aerodynamic shape optimization,” _Progress in Aerospace Sciences_ , Vol. 134, 2022, p. 100849.
  * Wang and Mahadevan [2009] Wang, C., and Mahadevan, S., “A general framework for manifold alignment,” _2009 AAAI Fall Symposium Series_ , 2009.
  * Guerrero et al. [2014] Guerrero, R., Ledig, C., and Rueckert, D., “Manifold alignment and transfer learning for classification of Alzheimer’s disease,” _International Workshop on Machine Learning in Medical Imaging_ , Springer, 2014, pp. 77–84.
  * Bousmalis et al. [2016] Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., and Erhan, D., “Domain separation networks,” _Advances in neural information processing systems_ , Vol. 29, 2016.
  * Wang and Mahadevan [2008] Wang, C., and Mahadevan, S., “Manifold alignment using procrustes analysis,” _Proceedings of the 25th international conference on Machine learning_ , 2008, pp. 1120–1127.
  * Pinnau [2008] Pinnau, R., “Model reduction via proper orthogonal decomposition,” _Model order reduction: theory, research aspects and applications_ , Springer, 2008, pp. 95–109.
  * Gower [2010] Gower, J. C., “Procrustes methods,” _Wiley Interdisciplinary Reviews: Computational Statistics_ , Vol. 2, No. 4, 2010, pp. 503–508.
  * Choi et al. [2009] Choi, S., Alonso, J. J., and Kroo, H. M., “Two-level multifidelity design optimization studies for supersonic jets,” _Journal of Aircraft_ , Vol. 46, No. 3, 2009, pp. 776–790.  10.2514/1.34362.
  * Han et al. [2012] Han, Z. H., Zimmermann, and Görtz, S., “Alternative cokriging model for variable-fidelity surrogate modeling,” _AIAA Journal_ , Vol. 50, No. 5, 2012, pp. 1205–1210.  10.2514/1.J051243.
  * Han and Görtz [2012] Han, Z. H., and Görtz, S., “Hierarchical kriging model for variable-fidelity surrogate modeling,” _AIAA Journal_ , Vol. 50, 2012, pp. 1885–1896.  10.2514/1.J051354.
  * Mufti et al. [2023] Mufti, B., Perron, C., Gautier, R., and Mavris, D. N., “Design Space Reduction using Multi-Fidelity Model-Based Active Subspaces,” _AIAA AVIATION 2023 Forum_ , 2023, p. 3592.
  * Han and Görtz [2012] Han, Z. H., and Görtz, S., “Hierarchical kriging model for variable-fidelity surrogate modeling,” _AIAA Journal_ , Vol. 50, No. 9, 2012, pp. 1885–1896.  10.2514/1.J051354.
  * Economon et al. [2016] Economon, T. D., Palacios, F., Copeland, S. R., Lukaczyk, T. W., and Alonso, J. J., “SU2: An open-source suite for multiphysics simulation and design,” _AIAA Journal_ , Vol. 54, No. 3, 2016, pp. 828–846.
  * Lee et al. [2015] Lee, C., Koo, D., Telidetzki, K., Buckley, H., Gagnon, H., and Zingg, D. W., “Aerodynamic shape optimization of benchmark problems using jetstream,” _53rd AIAA Aerospace Sciences Meeting_ , 2015, p. 0262.
  * Poole et al. [2015] Poole, D. J., Allen, C. B., and Rendall, T., “Control point-based aerodynamic shape optimization applied to AIAA ADODG test cases,” _53rd AIAA Aerospace Sciences Meeting_ , 2015, p. 1947.
  * Mufti et al. [2019] Mufti, B., Khan, T., Masud, J., and Toor, Z., “Flow field analysis of a diverterless supersonic inlet using embedded les methodology,” _2019 IEEE 10th International Conference on Mechanical and Aerospace Engineering (ICMAE)_ , IEEE, 2019, pp. 79–84.
  * Toor et al. [2020] Toor, Z., Masud, J., Irfan, T., Mufti, B., and Khan, O., “Comparative Analysis of Aerodynamic Characteristics of a Transport Aircraft and its AWACS Variant,” _AIAA Scitech 2020 Forum_ , 2020, p. 2227.
  * Lyu et al. [2015] Lyu, Z., Kenway, G. K. W., and Martins, J. R. R. A., “Aerodynamic Shape Optimization Investigations of the Common Research Model Wing Benchmark,” _AIAA Journal_ , Vol. 53, No. 4, 2015, pp. 968–985.  10.2514/1.J053318, URL https://arc.aiaa.org/doi/10.2514/1.J053318.
  * Kulfan [2008] Kulfan, B. M., “Universal Parametric Geometry Representation Method,” _Journal of Aircraft_ , Vol. 45, No. 1, 2008, pp. 142–158.  10.2514/1.29958.
  * Masters et al. [2016] Masters, D. A., Poole, D. J., Taylor, N. J., Rendall, T., and Allen, C. B., “Impact of shape parameterisation on aerodynamic optimisation of benchmark problem,” _54th AIAA Aerospace Sciences Meeting_ , 2016, p. 1544.
  * Hicks and Henne [1978] Hicks, R. M., and Henne, P. A., “Wing Design By Numerical Optimization,” _Journal of Aircraft_ , Vol. 15, No. 7, 1978, pp. 407–412.  10.2514/3.58379.
  * Masters et al. [2017] Masters, D. A., Taylor, N. J., Rendall, T. C., Allen, C. B., and Poole, D. J., “Geometric comparison of aerofoil shape parameterization methods,” _AIAA Journal_ , Vol. 55, No. 5, 2017, pp. 1575–1589.  10.2514/1.J054943.
  * Kenway et al. [2010] Kenway, G., Kennedy, G., and Martins, J., “A CAD-Free Approach to High-Fidelity Aerostructural Optimization,” _13th AIAA/ISSMO Multidisciplinary Analysis Optimization Conference_ , American Institute of Aeronautics and Astronautics, Reston, Virigina, 2010, pp. 1–18.  10.2514/6.2010-9231, URL http://arc.aiaa.org/doi/10.2514/6.2010-9231.