\(m\)total mass of all species in the reactor
\(\dot{m}_{\text{in}}\)mass inflow of an inlet
\(\dot{m}_{\text{out}}\)mass outflow of an outlet
\(\dot{m}_{k,\text{gen}}\)the rate of creation of species $k$
\(W_k\)molecular weight of species $k$
\(\dot{\omega}_{k}\)net reaction rate of species $k$
\(Y_k\)mass fraction of species $k$
\(X_k\)molar concentration of species $k$
\(Y_{k,\text{in}}\)mass fraction of species $k$ in mass inflow
\(Y_{k,\text{out}}\)mass fraction of species $k$ in mass outflow
\(\mathcal{M}_k\)symbol for species $k$
\(\nu'_{kj}, \nu''_{kj}\) molar stoichiometric coefficients of species $k$ in reaction $j$
\(\mathcal{Q}_j\)rate of progress of reaction $j$
\(K_{fj}\)forward rates of reaction $j$
\(K_{rj}\)reverse rates of reaction $j$
\(A_{fj}\)pre-exponential factor of reaction $j$
\(\beta_j\)temperature exponent of reaction $j$
\(\Delta S^0_j\)entropy changes of reaction $j$
\(\Delta H^0_j\)enthalpy changes of reaction $j$
\(E_j\)activation energy of reaction $j$
\(\dot{Q}\)heat source
\(h\)total enthalpy per unit mass
\(h_{\text{in}}\)enthalpy of the mass inflow
\(U\)total internal energy
\(u_k\)internal energy of species $k$
\(T\)Temperature

§ ABSTRACT

In this work, we present a novel physics-based data-driven framework for reduced-order modeling of laser ignition in a model rocket combustor based on parameterized neural ordinary differential equations (PNODE). Deep neural networks are embedded as functions of high-dimensional parameters of laser ignition to predict various terms in a 0D flow model including the heat source function, pre-exponential factors, and activation energy. Using the governing equations of a 0D flow model, our PNODE needs only a limited number of training samples and predicts trajectories of various quantities such as temperature, pressure, and mass fractions of species while satisfying physical constraints. We validate our physics-based PNODE on solution snapshots of high-fidelity Computational Fluid Dynamics (CFD) simulations of laser-induced ignition in a prototype rocket combustor. We compare the performance of our physics-based PNODE with that of kernel ridge regression and fully connected neural networks. Our results show that our physics-based PNODE provides solutions with lower mean absolute errors of average temperature over time, thus improving the prediction of successful laser ignition with high-dimensional parameters.

§ INTRODUCTION

§.§ Background and Related Work.

Computational Fluid Dynamics (CFD) simulations are a critical tool in the design of combustion systems, as they provide predictions of crucial quantities that are often not accessible in experiments. However, such high-fidelity simulations are computationally expensive because they require the solution of partial differential equations with complex combustion chemistry with a wide range of spatiotemporal scales. This becomes an obstacle to the optimization, control, and design of combustion systems, including scramjets, rocket combustors, and gas turbines, where repeated experiments or simulations of ignition are often needed to evaluate the sensitivity to ignition and uncertainty under various operational conditions. Therefore, many studies in the past have focused on building predictive and scalable surrogate models for combustion that can help improve the development of efficient engine systems.

One possible approach is the use of thermodynamical models without spatial information, which are also known as zero-dimensional models <cit.>. Predicting average combustion variables that are crucial to understanding the combustion dynamics, zero-dimensional single-zone models provide real-time simulations with significantly lower computational cost while also maintaining reasonable accuracy for model-based control of combustion engines. First introduced as a mean value model by , a zero-dimensional single-zone model was constructed based on energy conservation laws and fitted to spark ignition engines that showed 2% error over the entire operating range <cit.>. The use of a reaction-based 0D diesel combustion model for the turbocharged engine was considered by  to predict the combustion rate, the temperature and the pressure of the engine <cit.>.  applied a 0D model with eight reactions to homogeneous charge compression ignition (HCCI) engines for control analysis <cit.>. However, the combustion parameters of the 0D model often need to be re-calibrated under different operational conditions, and the calibration of chemical reaction parameters, such as reaction rate and activation energy, requires a deep understanding of chemical kinetics and combustion dynamics <cit.>.

Data-driven reduced-order modeling (ROM), where fast solvers are constructed based on data from high-fidelity simulations, has become another attractive alternative for the optimization and quantification of uncertainties in combustion systems. In particular, machine learning-based surrogate models, which are known for their ability to learn highly complex and non-linear mappings in physical or chemical processes, have been widely applied to build reduced-order models for Direct Numerical Simulations (DNS) of fluid turbulence in recent years <cit.>.

Many attempts have been made to use machine learning-based models as surrogate models for chemical reactions in combustion processes, which, when coupled with high-fidelity CFD simulations, require significant computational resources that are challenging to satisfy. To predict the evolution of reactive species and physical quantities as a function of species composition,  trained an artificial neural network (ANN) on simulation data generated from a reduced-order model of the methane/air combustion system. The performance of ANNs demonstrated encouraging savings in computational time and memory <cit.>.  predicted the evolution of reacting species in a hydrogen-oxidation reaction with an ANN and integrated it into a real-world reacting flow CFD <cit.>.    used deep neural networks, which take species mass fractions and temperature as input, to predict the chemical source term from DNS of a turbulent non-premixed syngas oxy-flame interacting with a cooled wall <cit.>. Their results show that ANNs coupled with numerical simulations can achieve a computational speedup of 25 compared to DNS with a detailed chemistry mechanism <cit.>.

Furthermore, recently many works have also started applying machine learning-based models to high-dimensional data from either real experiments or full-scaled high-fidelity simulations.  replaced a conventional numerical solver with a deep learning-based solver consisting of convolutional neural networks to predict flow fields from hydrogen-fueled turbulent combustion simulations <cit.>.  replaced a numerical integration with artificial neural networks (ANN) in a laminar flamelet model to predict mean reactive scalars in a non-premixed methane/hydrogen/nitrogen flame. Neural networks were shown to provide accurate predictions for the concentration and temperature of mean species while having a computational cost approximately one order of magnitude lower than traditional integration schemes <cit.>. However, the lack of physical constraints in data-driven models means that their solutions will not typically satisfy physical constraints, and thus can result in non-physical solutions. Furthermore, in those previous studies, deep learning models are trained based on paired samples with information on the current and next states. Therefore, when coupled with numerical integration or other iterative schemes, these deep learning models often diverge from the true solutions due to the accumulation of errors in the early stages of the algorithm.

§.§ Neural Ordinary Differential Equations

Deep neural networks have been known for their capability to learn highly complex and non-linear functions with a relatively low computational cost. With sufficient training data and a suitable model structure, neural networks can approximate arbitrary functions with arbitrary precision <cit.>. For dynamical systems, a novel class of deep learning models known as neural ordinary differential equations has become popular recently, where deep neural networks are used to predict the derivative of quantities of interest given the current state variables <cit.>. More precisely, a neural ODE is a parameterized ODE system of the following form:

\begin{equation}
\frac{du}{dt} = F(u(t), t;\theta),
\label{eq:neuralode}
\end{equation}

where $u$ is the state variable of the system, $t$ is time, $F$ is a deep neural network, and $\theta$ are model parameters such as weights and biases. Gradients of the loss function with respect to model parameters are computed through automatic differentiation or the adjoint sensitivity method, which allows the neural ODE to be trained with high computational efficiency.

Instead of computing the loss function based on paired samples of consecutive states, neural ODEs calculate the loss function directly based on solutions from the ODE, thus avoiding accumulating errors when performing numerical integration separately. Previous studies in combustion systems have attempted to use neural ODEs to learn chemical reactions from homogeneous reactors <cit.>.  proposed a deep learning framework based on neural ODEs to learn the homogeneous auto-ignition of hydrogen-air mixture with varying initial temperature and species composition <cit.>.  combined an auto-encoder neural network with a neural ODE to compute the solution in the latent space instead of the original stiff system of chemical kinetics <cit.>. However, since a neural ODE can learn a single ODE from a given training data set <cit.>, only initial conditions are varied during the neural ODE training phase in the aforementioned studies. For instance, when a neural ODE learns the following simple equation:

\begin{equation}
\frac{dy}{dx} = \alpha x
\end{equation}

with $y(0) = \beta$, only the value of of $\beta$ is varied in training data. When the value of $\alpha$ changes, a new neural ODE needs to be trained on a different data set. To model a wide range of ODE systems simultaneously,  changed the original NODE framework and introduced parameterized neural normal differential equations (PNODE) that take additional parameters as input to deep neural networks <cit.>. Parameterized Neural Ordinary Differential Equations now describe the following system:

\begin{equation}
\frac{du}{dt} = F(u(t,\eta), t, \eta;\theta),
\label{eq:pnode}
\end{equation}

where $\eta$ represents problem-dependent parameters for the dynamical systems. This allows us to learn a parameterized family of ODEs instead of a single ODE. Using a similar optimization procedure (i.e., automatic differentiation) and the same set of model parameters, PNODE is capable of predicting solutions of dynamical systems under different conditions depending on the input parameter $\eta$.

In previous data-driven approaches based on neural ODEs, the source function $F$ in <ref> is a deep neural network $F_{\theta}$ with weights $\theta$ that only learns to predict the time derivative of the state variables from input and output data. To learn combustion processes and chemical reactions with a wider range of parameters, large amounts of combustion data are often required for such purely data-driven neural ODEs, which can be challenging if training data are obtained from expensive high-fidelity simulations or physical experiments. Therefore, previous applications of neural ODEs in combustion studies have focused on simulations of simplified chemical kinetics in homogeneous or 0D reactors, with data sets that vary only two or three parameters, such as mass inflow, initial temperature, and species composition.

In this work, we consider the use of neural ODE for combustion studies from a different perspective. Instead of learning the source function directly with deep neural networks in NODE, we consider using existing 0D models as source functions and learn important parameters such as the heat source and activation energy in 0D models with neural networks. More precisely, to incorporate physical knowledge into our PNODE, we construct $F$ based on a 0D flow model $F_{0D}$ and embed deep neural networks into our 0D flow model to model terms such as the heat source and activation energy. Let $\dot{Q}(t, \eta; \theta)$ be the heat source function, $A_f(t,\eta; \theta)$ be the pre-exponential factor, and $E(\eta; \theta)$ be the activation energy function. These are represented by deep neural networks and are used as inputs to the 0D flow model $F$. Then we have

\[\frac{du}{dt} = F\left(u(t,\eta), t, \eta;\theta\right) = F_{0D}\left(u(t, \eta), \dot{Q}(t, \eta; \theta), A_f(t, \eta; \theta), E(\eta; \theta)\right)\]

The prediction $\hat{u}(t_i)$ can be obtained by solving the ODE system using a numerical solver:

\[\hat{u} = (\hat{u}(t_0), \dots, \hat{u}(t_n)) = \textrm{ODESolve}(u(t_0), F_{0D}, t_1,...,t_n)\]

To optimize the parameters in the neural networks that model the heat source, pre-exponential factor, and activation energy, we minimize the root mean squared loss between observations and predictions. This is accomplished using reverse mode automatic differentiation applied to the numerical ODE solver. The exact formulation of each component in $F_{0D}$ is described in <ref>. To the best of our knowledge, our work is the first application to consider the use of 0D models with neural ODEs for combustion systems. With additional information from the 0D model, our physics-based PNODE differs from previous neural ODE approaches in that it learns parameters from a higher dimensional space and directly from high-fidelity simulations.

§ OUR CONTRIBUTION

In this work, we propose a novel physics-based PNODE framework that directly learns from high-fidelity CFD instead of 0D simulations with simple chemical kinetics. We parameterize our neural ODEs with additional parameters to account for CFD simulations with different experimental conditions such as mass inflow, initial species composition, and geometry of the combustor chamber. Physical knowledge is incorporated into our neural ODEs, as we only use deep neural networks to predict the heat source term and chemical kinetics in our 0D flow model, instead of the entire source term $F$ in <ref>. In particular, our physics-based PNODE framework allows us to build a reduced-order surrogate model for laser ignition in a rocket combustor that

* provides solutions satisfying physical constraints,

* learns combustion with high-dimensional input parameters,

* needs less training samples than purely data-driven approaches,

* matches the accuracy of high-fidelity simulations.

The general workflow is illustrated in <ref>. We validate our approach on high-fidelity simulation data describing laser-induced ignition of a prototype rocket combustor. We compare the performance of our PNODE-based 0D model with conventional interpolation methods such as kernel ridge regression and neural networks.

The remainder of this paper is organized as follows. <Ref> describes our overall methodology for physics-based PNODEs. <Ref> describes the overall framework of our PNODE-based 0D model. <Ref> reviews the structure of deep neural networks, the choice of hyper-parameters, and optimization pipelines. <Ref> shows the numerical benchmarks of the PNODE-based models on high-fidelity simulations of laser-induced ignition of a rocket combustor.

[node distance=2cm]
(laser_parameter) [values, align=center] Parameters of
Laser Ignition;
(experiments) [simulation, right of=laser_parameter, xshift = 2.5cm, align=center] Experiments or High-
fidelity Simulations;
(pnode) [simulation, below of=experiments] Physics-based
(output) [values, right of=experiments, xshift = 2.5cm, align=center] Average Temperature,
Pressure, and Mass
Fractions of Species;
[arrow] (laser_parameter) – (experiments);
[arrow] (laser_parameter) |- (pnode);
[arrow] (pnode) -| node[anchor = south east, align = center, xshift = -0.4cm]Lower Cost (output);
[arrow] (experiments) – (output);
[arrow] (output) – (9.0, 2.0) – node[anchor=south]Improved Design (0.0, 2.0) – (laser_parameter);

Motivation for constructing a physics-based PNODE model. Time traces of volume-averaged temperature, pressure, and mass fractions of species provide crucial information that indicates ignition success or failure. Instead of computing volume-averaged quantities from high-fidelity simulations or physical experiments, we construct a PNODE-based 0D flow model that provides volume-averaged time traces directly with a lower computational cost.

§ METHODOLOGY

§.§ 0D flow model

Given a parameter $\eta$, we wish to use a 0D model to describe the ignition process of the target combustion system by predicting the evolution of average quantities such as the temperature, pressure, and mass fractions of species. We formulate our 0D flow model based on the Continuously Stirred Tanked Reactor (CSTR) model with fixed volume from the Cantera library <cit.>. CSTR is a simplified reactor with several inlets, several outlets, and a constant volume, and the reacting flow in the chamber is considered spatially homogeneous.

every picture/.style=line width=0.75pt

(229,35.67) – (229,80) ;

(170.48,80) – (207.5,80) – (229,80) ;

(229,188.95) – (229,237.86) ;

(229,99) – (169.75,99) ;

(229,36) – (460,36) ;

(460,36.09) – (460,189) ;

(460,237) – (229,237) ;

(460,210) – (460,237) ;

(460,189) – (519.43,189) ;

(460,210.13) – (519.43,210) ;

(229,189) – (229,99) ;

(290,20.25) – (400.5,20.25) ;

(345.25,20.25) – (345.25,170) ;

(320.5,170) – (370.5,170) ;

[fill=rgb, 255:red, 255; green, 255; blue, 255   ,fill opacity=1 ] (288,80.5) .. controls (288,66.69) and (310.39,55.5) .. (338,55.5) – (338,70.5) .. controls (310.39,70.5) and (288,81.69) .. (288,95.5) ;[fill=rgb, 255:red, 255; green, 255; blue, 255   ,fill opacity=1 ] (288,95.5) .. controls (288,105.75) and (300.34,114.56) .. (318,118.42) – (318,123.42) – (338,113) – (318,98.42) – (318,103.42) .. controls (300.34,99.56) and (288,90.75) .. (288,80.5)(288,95.5) – (288,80.5) ;

[fill=rgb, 255:red, 255; green, 255; blue, 255   ,fill opacity=1 ] (402.85,95.07) .. controls (403.36,108.87) and (381.4,120.88) .. (353.81,121.9) – (353.25,106.91) .. controls (380.85,105.89) and (402.81,93.88) .. (402.3,80.08) ;[fill=rgb, 255:red, 255; green, 255; blue, 255   ,fill opacity=1 ] (402.3,80.08) .. controls (401.92,69.84) and (389.26,61.49) .. (371.48,58.28) – (371.29,53.29) – (351.69,64.44) – (372.21,78.27) – (372.03,73.27) .. controls (389.82,76.48) and (402.47,84.83) .. (402.85,95.07)(402.3,80.08) – (402.85,95.07) ;

(250.5,170.86) .. controls (250.44,163.61) and (266.06,157.6) .. (285.39,157.44) .. controls (304.72,157.27) and (320.44,163) .. (320.5,170.25) .. controls (320.56,177.5) and (304.95,183.51) .. (285.62,183.68) .. controls (266.29,183.84) and (250.57,178.11) .. (250.5,170.86) – cycle ;

(370.5,170.25) .. controls (370.5,163) and (386.17,157.13) .. (405.5,157.13) .. controls (424.83,157.13) and (440.5,163) .. (440.5,170.25) .. controls (440.5,177.5) and (424.83,183.37) .. (405.5,183.37) .. controls (386.17,183.37) and (370.5,177.5) .. (370.5,170.25) – cycle ;

[arrow]    (109.43,88) – node[anchor=south]$\dot{m}_{\text{in}}$ (156,88) ;

[arrow]   (534.43,200) – node[anchor=south]$\dot{m}_{\text{out}}$ (581,200);

(290,217.31) node   [align=left] [lt]68pt

Illustration of a Continuously Stirred Tanked Reactor (CSTR) with one inlet and one outlet. $P$ is volume-averaged pressure, $V$ is volume, $T$ is volume-averaged temperature, and $Y_i$ is the mass fraction of species $i$ in the reactor. Gases inside the reactor are perfectly mixed. This is a simplified model for high-fidelity simulations, where fuels are injected into the chamber through the inlet at rate $\dot{m}_{\text{in}}$ and gases inside the reactor leave through the outlet at rate $\dot{m}_{\text{out}}$.

§.§.§ Mass conservation

In a CSTR reactor, the total mass of the system is conserved. Therefore, we have

\begin{equation}
\frac{dm}{dt} = \sum_{in} \dot{m}_{in} - \sum_{out} \dot{m}_{out},
\label{eq:mass_conservation}
\end{equation}

where $m$ is the total mass of all species in the reactor, $\dot{m}_{in}$ is the mass inflow of an inlet and $\dot{m}_{out}$ is the mass outflow of an outlet.

§.§.§ Species conservation

In reacting flows, species are created and destroyed in time from chemical reactions. The rate of species creation $k$ is given as:

\begin{equation}
\dot{m}_{k,\text{gen}} = V\dot{\omega}_{k}W_{k},
\label{eq:creationrate}
\end{equation}

where $W_k$ is the molecular weight of species $k$ and $\dot{\omega}_k$ is the net reaction rate. The rate of change of a species' mass is given by

\begin{equation}
\frac{d(mY_k)}{dt} = \sum_{in} \dot{m}_{in}Y_{k, \text{in}} - \sum_\text{out} \dot{m}_\text{out} Y_{k,\text{out}} + \dot{m}_{k,\text{gen}},
\label{eq:speciesmass}
\end{equation}

where $Y_k$ is the mass fraction of species $k$ in the reactor, $Y_{k, in}$ is the mass fraction of species $k$ in the inflow, $Y_{k, out}$ is the mass fraction of species $k$ in the outflow. Combining <ref>, we obtain

\begin{equation}
m\frac{dY_k}{dt} = \sum_{in} \dot{m_{in}}(Y_{k,in}-Y_k) - \sum_{out} \dot{m}_{out}(Y_k - Y_{k, out}) +  V \dot{\omega}_{k} W_k.
\label{eq:species}
\end{equation}

§.§.§ Chemical reactions

Suppose that for reaction $j$ we have

\begin{equation}
\sum_{k=1}^N \nu'_{kj} \mathcal{M}_k \leftrightharpoons \sum_{k=1}^N \nu''_{kj}\mathcal{M}_k,
\label{eq:reactions}
\end{equation}

where $\mathcal{M}_k$ is a symbol for species $k$, $\nu'_{kj}$ and $\nu''_{kj}$ are the stoichiometric coefficients of species $k$ in reaction $j$. Let $\nu_{kj} = \nu''_{kj} - \nu'_{kj}$. Then the source term $\dot{\omega}_{k}$ can be written as the sum of the source terms $\dot{\omega}_{k,j}$ from reaction $j$:

\begin{equation}
\dot{\omega}_k = \sum_{j} \dot{\omega}_{k, j}
= W_k \sum_{j=1}^M \nu_{kj}\mathcal{Q}_j,
\label{eq:omegasum}
\end{equation}

\begin{equation}
\mathcal{Q}_j = K_{fj}\prod_{k=1}^N [X_k]^{\nu'_{kj}}-K_{rj}\prod_{k=1}^{N}[X_k]^{\nu''_{kj}},
\label{eq:qj}
\end{equation}

\begin{equation}
K_{fj} = A_{fj}T^{\beta_j}\exp{\left(-\frac{E_{j}}{RT}\right)}, K_{rj} = \frac{K_{fj}}{\left(\frac{p_a}{RT}\right)^{\sum_{k=1}^N\nu_{kj}}\exp{\left( \frac{\Delta S^0_j}{R} - \frac{\Delta H^0_j}{RT} \right)}},
\label{eq:forwardconstant}
\end{equation}

where $\mathcal{Q}_j$ is the rate of progress of reaction $j$, $X_k$ is the molar concentration of species $k$, $K_{fj}$ and $K_{rj}$ are the forward and reverse rates of reaction $j$, $A_{fj}$, $\beta_j$ and $E_j$ are the pre-exponential factor, temperature exponent and activation energy for reaction $j$, respectively. $\Delta S^0_j$ and $\Delta H^0_j$ are entropy and enthalpy changes, respectively, for reaction $j$ <cit.>.

§.§.§ Energy conservation

For a CSTR reactor with fixed volume, the internal energy can be expressed by writing the first law of thermodynamics for an open system:

\begin{equation}
\frac{dU}{dt} = - \dot{Q} + \sum_{in} \dot{m}_{in}h_{in} - \sum_{out} h\dot{m}_{out},
\label{eq:thermo}
\end{equation}

where $\dot{Q}$ is the heat source, $h$ is the enthalpy per unit mass of the homogeneous gas in the reactor and $h_{in}$ is the enthalpy of the mass inflow. We can describe the evolution of the average temperature by expressing the internal energy $U$ in terms of the species mass fractions $Y_k$ and temperature $T$:

\begin{equation}
U = m\sum_{k}Y_ku_k(T).
\label{eq:enthalpy}
\end{equation}

So that

\begin{equation}
\frac{dU}{dt} = u\frac{dm}{dt} + mc_v\frac{dT}{dt} + m \sum_k u_k \frac{dY_k}{dt}.
\label{eq:dudt}
\end{equation}

From <ref> we have

\begin{align*}
mc_v \frac{dT}{dt} & = \frac{dU}{dt} - u \frac{dm}{dt} - m\sum_{k}u_k \frac{dY_k}{dt}, \\
& = -\dot{Q} + \sum_{int} \dot{m}_{in}h_{in} - \sum_{out} h\dot{m}_{out} - u \frac{dm}{dt} - m\sum_{k}u_k \frac{dY_k}{dt}.
\end{align*}

Next, using <ref> we have

\[
mc_v \frac{dT}{dt} = -\dot{Q} + \sum_{in} \dot{m}_{in}\left(h_{in} - \sum_{k} u_k Y_{k,in}\right) - \sum_{out} \dot{m}_{out}\left(h - \sum_{k} u_k Y_k - (Y_k - Y_{k, out})\right) - \sum_k V\dot{\omega}_{k}W_ku_k
\]

Finally, using the identity $hm = U + pV$ and <ref>, we have

\begin{equation}
mc_v \frac{dT}{dt} = -\dot{Q} + \sum_{in} \dot{m}_{in}\left(h_{in} - \sum_{k} u_k Y_{k,in}\right) - \sum_{out} \dot{m}_{out}\left(\frac{pV}{m} - (Y_k - Y_{k, out})\right) - \sum_k V\dot{\omega}_{k}W_ku_k.
\label{eq:energy}
\end{equation}

§.§ Physics-based parameterized neural ordinary differential equations

Using information from the parameters $\eta = (\eta_1, ..., \eta_p)$ of the laser ignition as input, we compute the heat source $\dot{Q}$ in <ref>, the pre-exponential factor $A_{f,j}$, and the activation energy $E_j$ in <ref> with fully connected feed-forward neural networks $F_{\theta}(t, \eta) = (\dot{Q}, E_{j},A_{fj})$ as shown in <ref>, where $\theta$ are model parameters.

^^JNeural network, shifted
T, P

^^J  Layer

ı[evaluate==̧int(ı==); =/2-ı-*̧;
=; =; =int(ı-1); =int(); =̆(ı<?int(ı-2):"[]")] in 1,...,

[node ] (N-ı) at (,) $\Zstr[\i]$;
[[node ] (N-ı) at (,) $\Cstr[\lay]_{\u}$;
[[node ] (N-ı) at (,) $\Cstr[\lay]_{\index}$;
[node ] (N-ı) at (,) $t$;
[[node ] (N-ı) at (,) $\Cstr[\z]_{\index}$;

ȷin 1,...,[]
[connect,white,line width=1.2] (N-ȷ) – (N-ı);
[connect] (N-ȷ) – (N-ı);

(N-) –++ (0,1+) node[midway,scale=1.5] $\vdots$;

[above=5,align=center,mygreen!60!black] at (N1-1.90) input
[above=2,align=center,myblue!60!black] at (N3-1.90) hidden layers;
[above=10,align=center,myred!60!black] at (N-1.90) output

Structure of a fully connected neural network $F_{\theta}$ with three hidden layers. Here $a_{j}^{(i)}$ refers to the $j$th neuron in the $i$th hidden layer of our deep neural network. In each forward pass, our deep neural network takes laser parameters $\eta_1,...,\eta_p$, time $t$ as input and predicts the volume-averaged temperature $T$, pressure $P$, and mass fractions $Y_i$ at time $t$ as output.

To ensure the training stability of our physics-based PNODE, we also normalize the total heat deposited by our heat source function $\dot{Q}$ and multiply it by another neural network $C_{\xi}$ that learns the total amount of heat injected into the system separately. That is, we define

\begin{equation}
\tilde{\dot{Q}}(t, \eta) = C_{\xi}(\eta)
\frac{\dot{Q}(t,\eta)}{\int_{t_\text{start}}^{t_\text{end}}
\dot{Q}(t,\eta) dt},
\label{eq:normalization}
\end{equation}

and we replace <ref> with the following equation:

\begin{equation}
mc_v \frac{dT}{dt} = -\dot{\tilde{Q}}(t, \eta)
+ \sum_{in} \dot{m}_{in}
\Big(h_{in} - \sum_{k} u_k Y_{k,in}\Big)
- \sum_{out} \dot{m}_{out}\Big(\frac{pV}{m} - (Y_k - Y_{k, out})\Big)
- \sum_k V\dot{\omega}_{k}W_ku_k,
\label{eq:parametrized_energy}
\end{equation}

where $C_{\xi}(\eta)$ is another deep neural network with model parameter $\xi$.

We recall the advantage of this parameterization of the 0D flow model using neural networks. While only scalars parameters are calibrated in conventional 0D models, our physics-based PNODE framework uses deep neural networks that are optimized automatically based on arbitrary loss functions. Additionally, deep neural networks have been shown as a powerful tool for learning complex and nonlinear chemical reactions in combustion studies<cit.>. The temperature $T(i,j)$ and the mass fraction of oxygen $Y_{\ce{O2}}(i,j)$ over time are used to compute the mean squared error as our loss function. That is, our loss function is defined as

\begin{equation}
L = \sum_{i = 1}^{n} \sum_{j = 1}^{m} (T(i, j) - T_\text{true}(i,j))^2 + \alpha (Y_{\ce{O2}}(i,j) - Y_{\ce{O2}, \text{true}}(i, j))^2,
\label{eq:loss}
\end{equation}

where $\alpha$ is a hyper-parameter that determines the weighted mean squared errors of the temperature and mass fraction of oxygen, $T(i,j)$ and $Y_{\ce{O2}}(i,j)$ are the predictions of the temperature and mass fraction of oxygen, respectively, at observation time $t_j$ for sample $i$. Note that since only a 1 step chemical mechanism (which is described in <ref>) is used in our 0D flow model and high-fidelity simulations, it is sufficient to minimize over the mass fraction of oxygen in the loss function.

The gradient of the loss function $L$ with respect to the weights $\theta$ are computed via automatic differentiation using ADCME <cit.>. In our physics-based PNODE, we use a Runge-Kutta fourth-order method to perform numerical integration. We use L-BFGS-B as the optimizer for the weights of the neural networks in ADCME. The overall optimization pipeline is illustrated in <ref>.

[node distance=2cm]

(laser_parameter) [values, align=center] Parameters of
Laser Ignition;
(in1) [model, below of=laser_parameter, align=center] Deep
Neural Networks;
(pnode) [values, right of=in1, xshift = 2.5cm, yshift = -1cm] Chemical Kinetics;
(experiments) [values, right of=laser_parameter, xshift = 2.5cm, yshift = -1cm, align=center] Heat Source;
(output) [model, right of=experiments, xshift = 2.5cm, yshift = -1cm, align=center] 0D Flow Model;
(condition) [values, above of=output, align=center] Initial and
Conditions ;
(prediction) [values, below of=output] Prediction ;
(observation) [values, below of=prediction] Observation ;
(loss_function) [loss, left of=observation, xshift = -2.5cm,  yshift = 1cm] Loss Function ;
[arrow] (in1) – (experiments);
[arrow] (laser_parameter) – (in1);
[arrow] (in1) – (pnode);
[arrow] (pnode) – (output);
[arrow] (experiments) – (output);
[arrow] (condition) – (output);
[arrow] (output) – node[anchor = west]Numerical Integration (prediction);
[arrow] (observation) – (loss_function);
[arrow] (prediction) – (loss_function);
[arrow] (loss_function) -| node[xshift = 1.5cm, yshift = 0.5cm, align = center] Automatic

Optimization pipeline of the physics-based PNODE framework. Parameters of laser ignition are passed into deep neural networks to predict the heat source and chemical kinetics for our 0D flow model. Predictions of volume-average temperature, pressure, and mass fractions of species over time are obtained through numerical integration. We update the weights in our deep neural networks by leveraging reverse mode automatic differentiation.

§ NUMERICAL EXPERIMENTS

We validate our approach on data generated from high-fidelity simulations of a planar jet diffusion based on the Hypersonics Task-Based Research (HTR) solver developed by <cit.>. In the rocket combustor, a gaseous O2 jet, along with CH4 coflow, is injected into the chamber filled with gaseous CH4. The jet is ignited by intense, short-time heating at a specific location. <Ref> illustrates the setup of the rocket combustor in our 2D high-fidelity simulations. In this work, we consider six parameters in total for laser-induced ignition, which are described in <ref>. We use a 1 step chemical mechanism, which has 5 species and 1 global reaction, as the chemical scheme in our 0D flow model <cit.>. To model ignition in high-fidelity simulations, we adjusted our initial values of the Arrhenius reaction parameters before training for our 0D flow model. In particular, for initial values before optimization, we set our pre-exponential factors to be 121.75 and set the activation energies equal to $1.68 \times 10^7$. Note that to fit simulation data from high-fidelity 2D simulations, we are setting both the pre-exponential factor and the activation energy to be smaller than default values. This is because the volume-averaged temperature during ignition is significantly lower than that of default 0D reactions, and also it increases at a much slower rate after successful ignition. We also choose $\alpha$ in <ref> as $1 \times 10^7$.

§.§ Simulation setup

To validate our approach, we used a direct numerical simulation of a two-dimensional planar jet diffusion flame as a high-fidelity reference.  Although simplified compared to full-scale rocket combustors, for the present objectives, this configuration bears a sufficient physical resemblance to such a system while requiring relatively low computational resources, and it is taken as a high-fidelity model compared to the 0D model (<ref>). A schematic of the simulation domain is shown in <ref>. The combustion chamber initially contains 100% methane at 350K and 50662.5Pa, and pure oxygen is injected at 350K and with Reynolds number $\Rey\equiv Ud/\nu_{\ce{O2}}=400$ and Mach number $\Ma\equiv U_{\ce{O2}}/a_{\ce{O2}}=0.1$, where $U_{\ce{O2}}$ is the injection velocity, $d$ is the jet diameter, $\nu_{\ce{O2}}$ is the kinematic viscosity of the injected oxygen and $a_{\ce{O2}}$ is its speed of sound. A coflow of methane at a lower velocity $U_{\ce{CH4}}/U_{\ce{O2}}=0.001$ accompanies the oxygen jet.

The compressible Navier–Stokes equations for a multicomponent ideal gas are solved with four chemical species (CH4, O2, H2O, CO2) and a single irreversible reaction for methane–oxygen combustion <cit.>:

\[
\ce{CH4 + 2 O2 -> 2 H2O + CO2}.
\]

Characteristic boundary conditions are used at inflow and outflow boundaries, and an isothermal no-slip condition is used on the walls. Standard methods for calculating transport and thermodynamic properties are used. The simulations are conducted using HTR <cit.>.

Shortly after the injection of methane and oxygen begins, a focused deposition of energy is deployed near the leading tip of the oxygen jet, as shown in <ref>. This is modeled as an energy source $\dot{Q}_L$ in the governing equation for total energy,

\[
\dot{Q}_L = B \frac{1}{2\pi\sigma_r^2}
\exp\Big[-\frac{(x-x_0)^2+(y-y_0)^2}{2\sigma_r^2}\Big]
\frac{2}{\sigma_t\sqrt{2\pi}}
\exp\Big[-\frac{4(t-t_0)^2}{2\sigma_t^2}\Big]\,,
\]

where $B$ is the amplitude, $\sigma_r$ is the radius of the energy kernel, $\sigma_t$ is the duration of the energy pulse, $(x_0,y_0)$ is the focal location, and $t_0$ is the time of energy deposition.  This produces a kernel of hot gas, seen in the <ref> inset.  Successful ignition depends on the parameters of the energy deposition as well as the local composition and flow conditions as the hot kernel cools and advects with the flow.

(at) at (-3, 3) ;
(et) at (5, 3) ;
[draw=red, dashed, minimum width=1.1cm, minimum height=1.2cm, text centered] (bt) at (3,3) ;
[yshift = -0.3cm, xshift = -0.1cm] (ct) at (at.north) ;
[yshift = 0.5cm, xshift = -0.1cm] (dt) at (at.south) ;
[arrow, dashed, red] (bt.north) – (ct);
[arrow, dashed, red] (bt.south) – (dt);
[decorate, decoration = calligraphic brace] (7.96,1) – node[anchor=north]60.0 (jet thickness) (1.5,1);
[decorate, decoration = calligraphic brace] (9,4.75) – node[anchor=south, rotate = 270]20.0 (jet thickness) (9,1.25);
[arrow] (9.75, 4) – node[anchor=south] Outflow (11, 4);
[arrow] (9.75, 2) – node[anchor=south] Outflow (11, 2);
[arrow] (-0.75, 4) – node[anchor=south] CH4 co-flow (1.25, 4);
[arrow] (-0.75, 3) – node[anchor=south] O2 jet (1.25, 3);
[arrow] (-0.75, 2) – node[anchor=south] CH4 co-flow (1.25, 2);
[text = white] at (4.75,4.65) Wall;
[text = white] at (4.75,1.5) Wall;
at (4.75,5.1) Mass Fraction of Oxygen;
at (-3.25,5.1) Temperature (F);
[draw = red, circle] (rc) at (-3.8, 2.6) ;

(rc) node[anchor=south, text = white, above=5pt]Ignition;
[dashed, red] (rc) – node[anchor = south, text = black, below = 15pt] $x$(-3.8, 1.3);
[dashed, red] (rc) – node[anchor = east, text = black, left = 8pt] $y$ (-4.8, 2.6);

Setup of a 2D laser-induced ignition in a rocket combustor. The snapshot was taken during laser deployment. Pure oxygen is injected at 350K into a two-dimensional chamber, along with a methane co-flow. Here $x$ and $y$ are the two-dimensional coordinates of laser-induced ignition inside the chamber.

$Y_{\text{Oxygen}}$, t=0.0e+00s

$Y_{\text{Oxygen}}$, t=2.0e-04s

Temperature, t=0.0e+00s

Temperature, t=2.0e-04s

$Y_{\text{Oxygen}}$, t=4.0e-05s

$Y_{\text{Oxygen}}$, t=2.4e-04s

Temperature, t=4.0e-05s

Temperature, t=2.4e-04s

$Y_{\text{Oxygen}}$, t=8.0e-05s

$Y_{\text{Oxygen}}$, t=2.8e-04s

Temperature, t=8.0e-05s

Temperature, t=2.8e-04s

$Y_{\text{Oxygen}}$, t=1.2e-04s

$Y_{\text{Oxygen}}$, t=3.2e-04s

Temperature, t=1.2e-04s

Temperature, t=3.2e-04s

$Y_{\text{Oxygen}}$, t=1.6e-04s

$Y_{\text{Oxygen}}$, t=3.6e-04s

Temperature, t=1.6e-04s

Temperature, t=3.6e-04s

Evolution of the temperature and mass fraction of oxygen in the 2D chamber with successful ignition. Snapshots of temperature and mass fraction of oxygen are taken every 40 microseconds. Oxygen and methane are constantly injected into the reactor from the left side. A laser is deployed inside the chamber to induce ignition around 120 microseconds after the simulation starts. This triggers combustion mechanisms, gradually increasing the average temperature of the temperature as more and more methane is ignited in the engine.

lParameter     lDefinition     cRange

x     x coordinate of the center of the heat kernel     [0,   7.0]

y      y coordinate of the center of the heat kernel     [0,   1.0]

amplitude     amount of energy deposited by the heat kernel     [0,   0.08]

radius     spatial radius of the heat kernel     [0,   0.5]

duration     duration of the heating     [0,   1.0]

MaF     Mach number of the co-flowing CH4 jet     [0,   0.02]

Description of combustion parameters

§.§ Planar jet diffusion simulation with fixed combustion parameters

We first show the performance of our physics-based PNODE model in learning a single trajectory from a planar jet diffusion simulation with successful ignition, which is shown in <ref>. <Ref> show the evolution of temperature and the mass fraction of oxygen over time. In this case, we choose $2 \times 10^{-5} \ \rm s$ as the time step of our numerical integration for our physics-based PNODE and choose a neural network with 2 hidden layers and 50 neurons in each layer as $F_{\theta}$. With 20
observations (i.e., points in simulation chosen to compute the mean squared errors) of the temperature and the mass fraction of oxygen, our physics-based PNODE can predict the evolution of both the average temperature and the mass fraction of oxygen with high accuracy. <Ref> shows the prediction of the total mass of the system over time. Note that with knowledge of the mass inflow and mass outflow, our physics-based PNODE can recover the change in total mass over time. <Ref> shows the predicted heat source from the neural network as a function of time. We observe that the heat source in our 0D flow model represents the laser energy deposited. The predicted temperature from the PNODE continues to rise due to the energy released from chemical reactions after the peak of the heat source function, showing that our physics-based PNODE is able to learn not only the impact of the laser beam but also the chemical reactions from the combustion system.

Prediction of average temperature by physics-based PNODE. The temperature initially stays at 350K and then rises gradually to 3500K after successful ignition. The evolution of volume-averaged temperature from PNODE matches closely with observations from the simulation.

Prediction of mass fraction of O2 by physics-based PNODE. The mass fraction of oxygen gradually increases from 0 to 12%, with a temporary decrease during laser ignition that triggers combustion mechanisms. Similarly, the prediction from PNODE matches closely with observations.

Prediction of total mass by physics-based PNODE. The overall mass inside the reactor constantly decreases due to mass outflow from the outlet. Since our physics-based PNODE preserves physical laws, the total mass matches perfectly with observations.

Heat source $\dot{Q}$ predicted by deep neural networks to represent the deposition of laser energy. Our physics-based PNODE calibrates the amount of heat deposited into the reactor so that the 0D model is able to simulate successful ignition from high-fidelity simulations.

§.§ Planar jet diffusion simulations with varying y coordinate and amplitude of heat kernel

In this section, we consider data generated from planar jet diffusion simulations with only two varying parameters: the $y$ coordinate of the location of the laser in the chamber and the amplitude of the laser beam. 143 data points are generated uniformly at random from selected intervals for the y location and amplitude of the laser. The generated final data points are shown in <ref>, which show the distribution of the temperature and the mass fraction of O2, respectively, at the end of the simulation with different $y$ locations and amplitudes. <Ref> show the evolution of the average temperature and the mass fraction of O2, respectively, over time for all simulations in the training data. We observe that there are sharp transitions of temperature near the boundary of ignition success (i.e., the boundary of the subset of laser parameters where the final temperature is above 1,000K) in the distribution when we vary two laser parameters.

We randomly selected 100 data points out of 143 samples to train our PNODE-based 0D model with neural networks with the hyperparameters shown in <ref>. Since we are using a simple feed-forward neural network, we manually optimize the structure of our PNODE based on its performance on 20 samples from a validation dataset randomly chosen from 100 training samples. The performance of our PNODE on seen parameters (training data) is shown in <ref>, which show the mean relative error of temperature and mass fraction of oxygen over time. We observe that the prediction of our neural  ODE model has high accuracy on all training data points with mean relative error of temperature lower than 1.6% and mean relative error of mass fraction of oxygen lower than 13.3%. We also test the performance of our PNODE-based 0D model with the remaining 43 unseen samples from our data set. The results are shown in <ref>. Our PNODE-based model is able to predict ignition success or failure accurately for all points far away from the boundary of the ignition area. Since there are sharp transitions from temperature of 350K to temperature higher than 1000K near the boundary of the area of successful ignition, false prediction of either ignition or non-ignition will both lead to large errors in mean relative error of temperature and mass fraction of oxygen. Hence, we observe that there are some errors by our physics-based PNODE for some of the test data points close to the boundary of the ignition area. Nevertheless, our PNODE-based model is still able to capture the sharp transition from non-ignition region to ignition region, which is often difficult to learn with conventional interpolation methods.

Hyperparameter     Value
Number of hidden layers      2

Number of neurons in each layer       300

Activation function     tanh

Optimization algorithm     L-BFGS-B

Hyperparameters for PNODE

Final average temperature with different $y$ and amplitude of laser beam. We observe that there are sharp transitions near the boundary of successful ignition. The final temperature is equal to 350K in the non-ignition area but increases drastically to around 1400K once the parameters cross the S-shaped boundary as shown in the figure.

Final mass fraction of O2 with different $y$ and amplitude of laser beam. Similarly, we observe that the average mass fraction displays a binary nature. The final values are either 0.04 or 0.06 depending on the success of laser ignition. The parameters that lead to ignition can be separated from other parameters by an S-shaped curve.

Evolution of average temperature in training data. The average temperature is initially at 350K. With successful ignition, the temperature gradually increases to values above 1000k. We observe that most ignitions happen between 100 and 150 microseconds after the simulation starts.

Evolution of mass fraction of O2 in training data. The mass fraction of O2 increases linearly first due to the injection of oxygen into the chamber. Successful ignition triggers chemical reactions that consume the oxygen, leading to a decrease in mass fraction of O2

Mean relative error of temperature on training data. We observe that errors for data points in the non-ignition region are around 0.1%, while errors for the ignition region are around 1.6%. The observed difference is most likely attributed to the wider range of final temperatures ranging from 1000K to 1400K for successful ignition cases, in contrast to the final temperature of 350K across all training data points for non-ignition cases.

Mean relative error of mass fraction of O2 on training data. We observe that the error ranges from 8.9% to 13.3% across all the training data points. The evolution of the mass fraction of oxygen is more difficult for PNODE to learn, as it is continuously increasing due to the injection of the fuel. The mean relative errors for successful ignition cases are around 11.2%, which is slightly lower than the values for non-ignition cases.

Mean relative error of temperature on test data. Most of the errors from our physics-based PNODE are less than 5%. Due to sharp transitions of final temperature from 350K to 1400K, failure in predicting ignition success leads to large relative errors. Hence, we observe that there are mean relative errors around 31% occur near the boundary of the ignition area.

Mean relative error of mass fraction of O2 on test data. Most of the errors are less than 15%. Similar to errors in final temperature, there are some errors larger than 20% near the boundary of the ignition region because of failure in predicting ignition success. The overall errors are also higher than errors in temperature, as it is more difficult for PNODE to predict the mass fraction of oxygen.

§.§ Planar jet diffusion simulations with six varying combustion parameters

We now consider planar jet diffusion simulations with all six varying combustion parameters. We generate training data by sampling within selected intervals for each of the six parameters uniformly at random as specified in <ref>. To illustrate both the expressiveness and robustness of our PNODE, we use the same hyper-parameters in <ref> and train our PNODE with data sets of two different sizes: one with 100 samples and one with 4,000 samples. Both data sets are generated by random uniform sampling from selected intervals for each of the six parameters in <ref>.

We test the performance of our physics-based PNODE on test data that represent cross sections of selected two dimensions of our six-dimensional parameter space. More precisely, after choosing two parameters out of six parameters for validation, we collect test data on a 30 by 30 grid for those two parameters while fixing the value of the remaining four parameters. In particular, we choose the following three pairs of parameters for test data:

* radius and amplitude

* duration and amplitude

* y coordinate and amplitude

We also compare the results of our PNODE with kernel ridge regression and neural networks. 20% of the samples in each data set are used as validation data. To illustrate the improvement from our physics-based PNODE, we deliberately choose the same network structure for the neural network as the ones embedded in our 0D flow model. For kernel ridge regression, we use a radial basis function with $\alpha = 2$ and $\gamma = 1$ from the scikit-learn package, which is manually optimized based on its performance on the validation set. The kernel ridge regression and classical neural network interpolate the temperature at the end of simulations in the six-dimensional parameter space. That is, they take the combustion parameter $\eta$ as input and predict the final temperature $T$ as output, which is a significantly simpler task than predicting the evolution of the system, including temperature, pressure, and mass fractions of species. We tested the performance of the three models on additional 100 test samples based on the final temperature. The mean absolute errors of the PNODE, kernel ridge regression, and neural network are shown in <ref>. The physics-based PNODE provides the most accurate predictions with both 100 training samples and 4,000 training samples.

cOne hundred samples     cFour thousand samples

Physics-based PNODE      200.35      149.17

Neural Network      501.33      397.65

Kernel Ridge Regression      273.59      225.48

Mean absolute error of prediction on the final average temperature by three models in the test data

The prediction of the three models for the cross-section data with 100 training samples is shown in <ref>. The prediction with 4,000 training samples is shown in <ref>. With only 100 training data points to interpolate in six-dimensional parameter space, both neural networks and kernel ridge regression predict solutions that transition smoothly from non-ignition to ignition in all three cross sections, while our PNODE can capture the sharp transitions near the boundary of the ignition area and also provides an accurate prediction for final temperature for most cases with successful ignition. With the advantage of physics-based PNODE, our 0D flow model can capture the non-linearity and complexity of the ignition map for planar jet diffusion simulations, despite the limited size of training data. With 4,000 training samples, all three models improve their predictions of the ignition area as shown in three cross sections. However, the neural network and kernel ridge regression still predict linear transitions across the boundary of successful ignition, while physics-based PNODE can predict an almost vertical jump from non-ignition to ignition, which is consistent with our ground-truth data. This further shows that physics-based PNODE, with sufficient training data, is capable of matching the accuracy of high-fidelity simulations in predicting the region of successful ignition, even when learning combustion from a high-dimensional parameter space.

In this study, we aim to distinguish the performance of physics-based PNODE, neural networks, and kernel ridge regression on predicting final temperatures in combustion systems. To achieve this, we applied Density-based Spatial Clustering of Applications with Noise (DBSCAN) on 300 additional test data points and analyzed the error distribution of the three methods. Our test data was separated into two data sets: ignited and non-ignited. We calculate the distances between data points based on six combustion parameters and divided the points into three categories: core, boundary, and noise points.

Core points have many close neighbors in the same data set, making them easier to predict than other points. Boundary points, on the other hand, are located at the margin of two clusters, most of which are often close to both the ignition and non-ignition data sets. Lastly, we have noise data points that are far away from both data sets. The error distribution of the three models on different types of data points for the ignited case and non-ignited case are shown in <ref>. Our results indicate that physics-based PNODE provides consistently accurate predictions of final temperature for both core and boundary points, with most absolute errors less than 200K.

On the contrary, the performance of neural networks and kernel ridge regression was significantly worse on boundary points, as it is more challenging to distinguish ignited cases from non-ignited cases near the boundary. All three methods have low accuracy for noise points, as they are difficult to predict due to a lack of nearby data points. Our analysis showed that a large number of errors from neural networks and kernel ridge regression fall within the 400 to 1000 range, indicating a smooth transition from ignition to non-ignition regions in terms of final temperature. This is inconsistent with the actual ignition process, as final temperatures from high-fidelity simulations or experiments typically display a binary nature with sharp transitions from non-ignition to ignition regions.

Prediction of the final temperature by the three models with 100 training samples on simulations with varying amplitude and duration of the laser beam. Top left: ground truth by Planar Jet Diffusion Simulations. Top right: prediction by kernel ridge regression. Bottom left: prediction by neural networks. Bottom right: prediction by physics-based PNODE. We observe that, with only 100 training data points, our physics-based PNODE provides the most accurate prediction of final temperature among all three methods, with a sharp boundary between ignition and non-ignition regions. On the contrary, kernel ridge regression and neural networks predict the final temperature of many data points to be between 400K and 1000K, which is inconsistent with our high-fidelity simulations.

Prediction of the final temperature by the three models with 4,000 training samples on simulations with varying amplitude and duration of the laser beam. Top left: ground truth by Planar Jet Diffusion Simulations. Top right: prediction by kernel ridge regression. Bottom left: prediction by neural networks. Bottom right: prediction by physics-based PNODE. We observe that, with only 100 training data points, our physics-based PNODE provides the most accurate prediction of final temperature among all three methods, with a sharp boundary between ignition and non-ignition regions. On the contrary, kernel ridge regression and neural networks predict the final temperature of many data points to be between 400K and 1000K, which is inconsistent with our high-fidelity simulations.

Prediction of the final temperature by the three models with 100 training samples on simulations with varying amplitude and radius of the laser beam. Top left: ground truth by Planar Jet Diffusion Simulations. Top right: prediction by kernel ridge regression. Bottom left: prediction by neural networks. Bottom right: prediction by physics-based PNODE. We observe that, with only 100 training data points, our physics-based PNODE provides the most accurate prediction of final temperature among all three methods, with a sharp boundary between ignition and non-ignition regions. On the contrary, kernel ridge regression and neural networks predict the final temperature of many data points to be between 400K and 1000K, which is inconsistent with our high-fidelity simulations.

Prediction of the final temperature by the three models with 4,000 training samples on simulations with varying amplitude and radius of the laser beam. Top left: ground truth by Planar Jet Diffusion Simulations. Top right: prediction by kernel ridge regression. Bottom left: prediction by neural networks. Bottom right: prediction by physics-based PNODE. We observe that, with only 100 training data points, our physics-based PNODE provides the most accurate prediction of final temperature among all three methods, with a sharp boundary between ignition and non-ignition regions. On the contrary, kernel ridge regression and neural networks predict the final temperature of many data points to be between 400K and 1000K, which is inconsistent with our high-fidelity simulations.

Prediction of the final temperature by the three models with 100 training samples on simulations with varying amplitude and y coordinate of the laser beam. Top left: ground truth by Planar Jet Diffusion Simulations. Top right: prediction by kernel ridge regression. Bottom left: prediction by neural networks. Bottom right: prediction by physics-based PNODE. With 4000 data points, all three methods improve their predictions on volume-average final temperature across all data points. We observe that neural networks and kernel ridge regression still predict linear transitions across the boundary of successful ignition, while physics-based PNODE can predict an almost vertical jump from non-ignition to ignition, which is consistent with our high-fidelity simulations.

Prediction of the final temperature by the three models with 4,000 training samples on simulations with varying amplitude and y coordinate of the laser beam. Top left: ground truth by Planar Jet Diffusion Simulations. Top right: prediction by kernel ridge regression. Bottom left: prediction by neural networks. Bottom right: prediction by physics-based PNODE. With 4000 data points, all three methods improve their predictions on volume-average final temperature across all data points. We observe that neural networks and kernel ridge regression still predict linear transitions across the boundary of successful ignition, while physics-based PNODE can predict an almost vertical jump from non-ignition to ignition, which is consistent with our high-fidelity simulations.

Error distribution of physics-based PNODE, neural networks, and kernel ridge regression on ignited data sets. Our results indicate that the neural networks and kernel ridge regression provide less accurate predictions than physics-based PNODE, especially on boundary points, as it is more challenging to distinguish ignited cases from non-ignited cases near the boundary of successful ignition.

Error distribution of physics-based PNODE, neural networks and kernel ridge regression on non-ignited data sets. Our results indicate that neural networks and kernel ridge regression provide less accurate predictions than physics-based PNODE, especially on boundary points, as it is more challenging to distinguish ignited cases from non-ignited cases near the boundary of successful ignition.

§ CONCLUSION

In this work, we proposed a novel hybrid model that combines a 0D reacting flow model and deep neural networks based on physics-based PNODEs. By embedding deep neural networks into the 0D model as the heat source function and Arrhenius reaction parameters, we were able to build a reduced-order surrogate model for high-fidelity Computational Fluid Dynamics simulations of combustion processes. Furthermore, our PNODE-based model is capable of providing, with a limited number of training samples, physically constrained solutions that describe highly complex parameter space for combustion systems. We validated our approach with high-fidelity Planar Jet Diffusion simulations using the HTR solver with six varying combustion parameters. The performance of our PNODE-based 0D model is compared with that of two widely used approaches, namely kernel ridge regression and classical neural networks. Our results show that our PNODE-based 0D model is able to predict, with only a limited number of data points, sharp transitions near the boundary of successful ignition, as well as the evolving chemistry of the combustion system.

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

SAGE Publications Sage UK: London, England

journaltitleInternational Journal of Engine Research
titleReview of engine control-oriented combustion models

journaltitleSAE transactions
titleMean value modelling of spark ignition engines

PhD Thesis, Michigan State University, 2019. Google Scholar

titleReaction-based modeling and control of an electrically boosted diesel engine

booktitle2004 43rd IEEE conference on decision and control (CDC)(IEEE Cat. No. 04CH37601)
titleModeling of HCCI engine combustion for control analysis

booktitleProceedings of the 2005, American Control Conference, 2005.
titleModeling operation of HCCI engines fueled with ethanol

journaltitleEnergy and AI
titleMachine learning for combustion

journaltitleProgress in Energy and Combustion Science
titleCombustion machine learning: Principles, progress and prospects

journaltitleCombustion and Flame
titleModelling the temporal evolution of a reduced combustion chemical system with an artificial neural network

booktitleAIAA scitech 2020 forum
titleDeep learning for scalable chemical kinetics

journaltitleCombustion and Flame
titleChemistry reduction using machine learning trained from non-premixed micro-mixing modeling: Application to DNS of a syngas turbulent oxy-flame with side-wall effects

journaltitleInternational Journal of Hydrogen Energy
titleA deep learning framework for hydrogen-fueled turbulent combustion simulation

journaltitleApplied Mathematical Modelling
titleLaminar flamelet modeling of a turbulent CH4/H2/N2 jet diffusion flame using artificial neural networks

journaltitleNeural networks
titleMultilayer feedforward networks are universal approximators

journaltitleAdvances in neural information processing systems
titleNeural ordinary differential equations

journaltitleEnergy and AI
titleChemNODE: A neural ordinary differential equations framework for efficient chemical kinetic solvers

booktitleAIAA SCITECH 2022 Forum
titleStiffness-Reduced Neural ODE Models for Data-Driven Reduced-Order Modeling of Combustion Chemical Kinetics

journaltitleAdvances in Neural Information Processing Systems
titleAugmented neural odes

journaltitlearXiv preprint arXiv:2006.09545
titleNeural optimal control for representation learning

The Royal Society

journaltitleProceedings of the Royal Society A
titleParameterized neural ordinary differential equations: Applications to computational physics problems

noteVersion 2.5.1
titleCantera: An Object-oriented Software Toolkit for Chemical Kinetics, Thermodynamics, and Transport Processes

RT Edwards, Inc.

titleTheoretical and numerical combustion

booktitleSymposium (International) on Combustion
titleAn integrated PDF/neural network approach for simulating turbulent reacting systems

journaltitleCombustion and Flame
titleArtificial neural network implementation of chemistry with pdf simulation of H2/CO2 flames

journaltitlearXiv preprint arXiv:2011.11955
titleADCME: Learning spatially-varying physical fields using deep neural networks

journaltitleComputer Physics Communications
titleHTR solver: An open-source exascale-oriented task-based multi-GPU high-order code for hypersonic aerothermodynamics

journaltitleCombustion and flame
titleLarge eddy simulation of combustion instabilities in a lean partially premixed swirled flame