# Towards Solving Industry-Grade Surrogate Modeling Problems using Physics
Informed Machine Learning

Saakaar Bhatnagara
<EMAIL_ADDRESS>Corresponding Author Andrew Comerforda
<EMAIL_ADDRESS>Araz Banaeizadeha
<EMAIL_ADDRESS>
(aAltair Engineering Inc., 100 Mathilda Place, Sunnyvale, CA, USA)

###### Abstract

Deep learning combined with physics-based modeling represents an attractive
and efficient approach for producing accurate and robust surrogate modeling.
In this paper, a new framework that utilizes Physics Informed Neural Networks
(PINN) to solve PDE-based problems for the creation of surrogate models for
steady-state flow-thermal engineering design applications is introduced. The
surrogate models developed through this framework are demonstrated on several
use cases from electronics cooling to biomechanics. Additionally, it is
demonstrated how these trained surrogate models can be combined with design
optimization methods to improve the efficiency and reduced the cost of the
design process. The former is shown through several realistic 3D examples and
the latter via a detailed cost-benefit trade off. Overall, the findings of
this paper demonstrate that hybrid data-PINN surrogate models combined with
optimization algorithms can solve realistic design optimization and have
potential in a wide variety of application areas.

Keywords: Surrogate Modeling; Physics-Based Machine Learning; Physics Informed
Neural Networks; Design of Experiments; Design Optimization; Electronics
Thermal Design

##  1 Introduction

Over the last few years, there has been significant growth in the popularity
of machine learning algorithms to solve partial differential equations (PDE)
or assist PDE solvers, such as a computational fluid dynamics(CFD) solver [7,
10]. The attraction of ML algorithms in these scenarios is the ability to find
solutions in domains that are challenges in conventional CFD, such as large
design space explorations [37], turbulence model closure [73] or solving
incomplete/ill-posed problems [9].

A particular application where CFD solvers struggle, due to the computational
cost, is iterative design optimization. This is the process of continually
updating a design (e.g. an electronics assembly layout) and computing the
solution (e.g. flow or thermal fields) to optimize the performance (e.g.
constrain the temperatures or reduce the pressure drop). The challenge for CFD
is the input-output relationship is one-to-one. Therefore, any changes to the
input vector (e.g. geometric variations) need to be re-simulated, leading to
high costs when iterating on different design scenarios [31]. Overall high-
fidelity iterative design requires a prohibitive level of resources, both
computationally and monetarily and often leads to a sub-optimal outcome.

To enhance the iterative design process and speed up CFD simulations surrogate
models are usually utilized. Surrogate modeling represents an essential
simulation technology for many computationally expensive applications. In
literature, there has been a significant amount of research on the
construction of surrogate or reduced order models using simulation data [32],
[20]; these methods include Proper Orthogonal Decomposition(POD) [32], Gappy
POD [42], Manifold Learning [40]. These approaches serve as good surrogates
that give near real-time predictions for input parameters within a certain
range. However, these methods have remained largely academic in their
application due to limitations, such as training data requirements and only
working in simple scenarios. Recently, increased attention has been given to
statistical methods like Gaussian processes and neural networks that
incorporate Machine Learning (ML) to create the same surrogates. Bhatnagar et
al. [4] used a CNN architecture to predict aerodynamic flow fields over
airfoils, and created a surrogate model that generalized between flow
conditions and airfoil geometries. Guo et al. [22] also used a Convolutional
Neural Network (CNN) architecture to predict steady flows over automotive
vehicles . Lee and You [34] used Generative Adversarial Networks(GANs) coupled
with physical laws to predict unsteady flow around a cylinder, demonstrating
the benefits of using embedded physics. Raissi and Karniadakis [50] uses
Gaussian processes to model and identify several complex PDEs.

Machine learning, and more specifically Artificial Neural Networks (ANNs),
have seen success in areas such as computer vision, natural language
processing and healthcare [14, 51, 63]. Much of this success can be attributed
to the development of new model architectures, programming frameworks (e.g.
Tensorflow and PyTorch), and most importantly large training datasets. The
ability of ANNs to capture complex nonlinear relationships between inputs and
outputs have also made them very well-suited for regression problems. For
design engineering applications, the ANN-based surrogate model must be
consistent with the underlying physics; however, to achieve this, a number of
hurdles must be overcome:

  1. 1.

Data is Expensive: Unlike many other commercial applications, solution data to
create surrogate models is generally expensive to obtain. Typically, the data
is obtained through repetitive simulation. Since ANNs work best with large
amounts of data to train them, this makes the cost of conventional data-driven
surrogate modeling via ANNs expensive.

  2. 2.

Spectral Bias: It is a well-documented feature of ANNs [48, 62, 11] that they
possess a bias towards learning the lower frequencies of a signal they are
being trained on first. This leads to a more challenging training process for
ANNs for engineering problems, where it is important to capture these high-
frequency components of the solution in an economical manner.

  3. 3.

Multiphysics and Multiscale problems: Many engineering applications,
especially in the flow-thermal realm, are multiscale in nature with complex
flow physics that need to be captured by surrogate models. This is not
straightforward and requires special treatment to ensure the generalizability
of the model [28, 13], as well as the correctness of the solution across any
interfaces.

  4. 4.

Unphysical Results: Purely data-driven models may not end up learning the
underlying physics from the data. This leads to unphysical results when trying
to generalize to unseen test cases with these models.

In order to build scalable, robust surrogate models using ANNs for general
engineering design problems, the above challenges need to be addressed. The
ideal surrogate model must satisfy the following criteria: 1) limited input
data; 2) generalizable; 3) easy and economical to create; and 4)
parallelizable.

In this paper, the Altair framework for flow-thermal surrogate model creation
and automatic design optimization is presented. The aforementioned
difficulties are individually addressed with a number of different proposed
solutions while addressing the lack of research applying physics-informed ML
to problems closer to the complexity of industrial design problems. The
effectiveness of adding very sparse solution data in the domain to aid Physics
Informed Neural Network convergence in 3D is demonstrated for a complex
geometry at a high Reynolds Number. Additionally, flow through a stenosis is
solved at a Reynolds Number of 150 in 3 dimensions, done with realistic
physical parameters in a purely physics-informed manner. Finally, a novel
method of coupling physics-informed ANN surrogates with traditional design
optimization algorithms (e.g. Particle Swarm Optimization [29]) is presented
for several industrial electronics assemblies, which are complex 3D setups
with realistic physical parameters, to enable automatic and more efficient
design optimization using surrogates compared to brute force search.

##  2 Physics Informed Neural Networks (PINNs)

###  2.1 Physics informed Neural Networks

Proposed by Raissi et al. [49], physics-informed neural networks (PINNs)
leverage automatic differentiation to obtain an analytical representation of
an output variable and its derivatives, given a parametrization using the
trainable weights of the network. By employing the underlying static graph it
is possible to construct the differential equations that govern physical
phenomena. Using gradient-based optimization the residual is converted into a
loss function and driven to zero in order to satisfy the PDE. Similarly, the
same methods can be used for the boundary conditions (Dirichlet/Neumann/Robin)
and initial conditions to completely construct a well-posed PDE for the neural
network to solve.

Once the static graph for PINN training has been created, any iterative
gradient-based optimization algorithm can be used to minimize the loss
function. Popular choices in literature, include Adam [30] and L-BFGS. Both
these optimizers come with advantages and drawbacks. For smaller problems, a
combination of Adam followed by L-BFGS to fine-tune the optimization works
well. However, for larger problems, the full batch size requirement of the
L-BFGS second-order optimizer limits its use to small/simplified problems.

Theoretically, PINNs require no data to train since the loss function will
contain all the terms necessary to completely describe the PDE problem.
However, depending on the implementation, they may require training data. If
the objective is to do a forward solve (i.e solving for the solution of the
PDE), no data should be required. If, however, one is solving an inverse
problem (i.e given some/all of the solution and part of the PDE infer the rest
of the PDE), they will need some training data and the known part of the PDE.
This diversity in potential applications is where the true potential of PINNs
lies; PINNs represent the intersection of PDE-based and data-driven solution
methods for systems that can be described by partial differential equations,
in a very simple, efficient manner. They allow for the creation of models
that, if trained correctly, increase the likelihood that the models will obey
the PDE-based laws of physics as applicable to the processes they are
modeling. Even in cases where PINNs cannot be trained correctly in a data-free
manner (described in Section 3), including the physics as an inductive bias
has been shown to improve the surrogate models by reducing overfitting and
making the results more physical [34, 65].

###  2.2 Mathematical Formulation

A PDE problem in the general form reads:

$\mathcal{N}_{\textbf{x}}[u]=0,\textbf{x}\in\Omega,$

$u(\textbf{x})=g(\textbf{x}),\textbf{x}\in\partial\Omega.$

In order to solve the PDE using the PINN method, the residual of the governing
PDE is minimized, which is defined by

$r_{\theta}(\textbf{x})=\mathcal{N}_{\textbf{x}}[f_{\theta}(\textbf{x})],$

where $f_{\theta}$ is the predicted value by the network. The residual value,
along with the deviation of the prediction from boundary/initial conditions,
is used to construct the loss, which takes the form:

$L(\theta)=L_{r}(\theta)+\sum^{M}_{i=1}\lambda_{i}L_{i}(\theta),$

where the index i refers to different components of the loss function,
relating to initial conditions, boundary conditions and measurement/simulation
data. $\lambda_{i}$ refer to the weight coefficient of each loss term. The
individual loss terms are constituted as follows:

$L_{r}=\frac{1}{N_{r}}\sum_{i}^{N_{r}}[r(\textbf{x}_{r}^{i})]^{2},$

$L_{b}=\frac{1}{N_{b}}\sum_{i}^{N_{b}}[u(\textbf{x}_{b}^{i})-g_{b}^{i}]^{2},$

$L_{d}=\frac{1}{N_{d}}\sum_{i}^{N_{d}}[u(\textbf{x}_{d}^{i})-\hat{u}(x_{d}^{i},t_{d}^{i})]^{2},$

where the subscripts r,b,d refer to collocation, boundary, initial and data
points, respectively.

###  2.3 Current Challenges with PINNs

Although the PINN method shows great promise, it still has a number of
unresolved issues. The biggest challenges with PINNs currently lie in the
scalability of the algorithms to large 3D problems as well as problems with
complex nonlinearities, and unsteady problems.

####  2.3.1 Weak imposition of Boundary Conditions

The solution of a PDE problem must obey all initial and boundary conditions
imposed on it while minimizing the residual of the governing equation.
However, for neural network based solvers it is difficult to impose boundary
and initial conditions in an exact manner. This is because the standard way to
impose B.C in PINNs is to create a linear combination of loss functions (as
described mathematically in the previous section). Each loss either describes
the deviation of the network output from a specific boundary condition, or the
magnitude of the residual of the governing equations. Therefore, boundary
conditions are only satisfied in a weak manner. There has been research
demonstrating the utility of exact imposition of boundary conditions [60, 59,
38] or creative multi-network approaches [52], such implementations are mostly
problem-specific and do not generalize well.

Weak imposition of boundary conditions also creates another issue, one that is
fairly common in multi-task learning and multi-objective optimization:
choosing the values of loss term coefficients that make up the linear
combination. Choosing these weights is a nontrivial exercise that would
require calibration via hyper-parameter search, which is not feasible. Wang et
al. [64] introduced a heuristic dynamic weighting algorithm to update and
select these weights automatically and continuously during the training, to
enable convergence to the correct answer. Additionally, there have been
several other algorithms proposed to choose the correct scheme for weighting
the losses [68, 67, 5]. This continues to be an active area of research in the
PINNs community. Finally, methods have been proposed to impose the boundary
conditions in a strong manner by manipulating the output formulations [60] or
by utilizing operator networks [53].

####  2.3.2 Difficult Optimization Problem

A second problem is the nature of the loss landscape itself, in which a
reasonable local minimum is required to be found. As seen in Krishnapriyan et
al. [33], Gopakumar et al. [21],Subramanian et al. [58] and Basir and Senocak
[2], as well as the author’s own experiments, different non-dimensional
quantities (e.g. Reynolds number) in the governing equations, the number of
dimensions of the problem, the point cloud/discretization, the boundary
conditions and the complexity of the solution to be predicted can adversely
affect the loss landscape of the neural network training. This makes the
optimization challenging and can fail to find an adequate local minimum via a
gradient descent-based algorithm. Recently, methods borrowing concepts from
optimization theory have shown alternate formulations (e.g. augmented
lagrangian method for the loss functions) can aid the convergence properties
of the training problem [2, 57]. There have also been efforts towards imposing
physical constraints in an integral form [23].

####  2.3.3 Cost of training

Constructing the PDE loss functions involves several backward passes through
the network, which is a costly operation. PINNs on average takes longer to
train than their data-driven counterparts for exactly this reason; the
computation graph of a PINN training is much more complex. Moreover, for the
Navier-Stokes equations, it has been seen that although the stream function
formulation provides better results (due to exact enforcement of continuity),
it is costlier in terms of training time. As seen in NVIDIA’s experiments
[25], it can take several million iterations for the more complex problems to
be solved via PINNs. To reduce the cost of training approaches such as
automatic differentiation for finite difference formulations [24], or using
first-order formulations [18], have been proposed. However, these solutions
tend to be mostly problem-specific and do not necessarily generalize well to
increased problem complexity and grid definitions. Meta-learning algorithms
[16] have also recently gained significance as an effective way to reduce the
cost of training neural networks on new tasks, and some of this work has been
extended to PINNs [46] as well.

####  2.3.4 Unsteady problems

As discussed above, in a standard implementation, solving a time-dependent
problem via PINNs is a challenging task. This is primarily because the
standard implementation treats time as another axis just like space; what this
means is that in creating the spatial point discretization, the space mesh has
to be repeated for every time step. This can be a very expensive process for
any problem except simple 1D and 2D problems. Moreover, another very important
issue with this method is the lack of respect for causality. It was
demonstrated by Wang et al. [66] that networks have a tendency to learn the
solution of the PDE at later time steps first, and if the solution does not
correctly solve for earlier time steps, the solution at later time steps will
most likely be incorrect. In literature, a number of methods have been
proposed to get around these issues [6, 33]. Krishnapriyan et al. [33]
suggests using a sequential model approach, breaking down the total set of
time steps into blocks of consecutive steps. Each block of time steps has its
own dedicated trained model, using either the initial condition provided (with
the PDE to solve) or the solution of the previous block of time steps as the
initial condition for the next block’s neural network solve. This approach has
been shown to lend stability to the training, as occasionally trying to solve
for all time steps simultaneously would fail. However, the bigger benefit for
simulating industry-grade problems, which have larger meshes/point clouds, is
that with this breakdown of the problem, the training for each network that
represents a sub-block can be fit on a GPU. These networks can then be trained
in a sequential manner, much faster on a GPU as compared to a CPU. To address
the problem of causality, Wang et al. [66] suggested a dynamic weighting
approach for the loss terms corresponding to each time step that would favor
the network learning the solution in a causal manner.

Another way of solving time-dependent problems using PINNs, is to use layers
that were designed to deal with sequential data in the first place. In
conventional data-driven deep learning, for applications like time-series
analysis and natural language processing, RNN and LSTM layers were designed to
deal with sequences of data instead. For applications in scientific machine
learning, these layers can be adapted to deal with time sequences. Work by Wu
et al. [69] and Cheng et al. [12] has already demonstrated promising
applications of these layers to scientific ML.

##  3 Important Features

In this section, some of the important features used in the framework are
outlined for the creation of ANN-based surrogate models.

###  3.1 Fourier Feature Embeddings

As described in Tancik et al. [62], Artificial Neural Networks suffer from a
spectral bias problem. To overcome this, they introduced a Fourier feature
embedding that allows models to capture high-frequency components of the
solution effectively. In addition, there have been other proposed solutions
for the spectral bias problem for applications to PDE problems, such as the
Siren activation [56], Fourier Neural Operators [35], and weighting schemes
derived from the theory of Neural Tangent Kernels (NTK) [68].

###  3.2 Embedding Physics as an Inductive Bias

It is clear that the current state of the art in PINNs has improved the method
significantly from what was proposed in the first paper [49], and researchers
have been able to gradually solve PDE-based problems involving more complex
nonlinearities and in more dimensions. However, the PINNs method is currently
not suited for solving complex engineering problems often encountered in
industry in a data-free manner. The optimization issues and cost of model
training outlined above make the method, presently, unsuitable for use as a
forward solver. To get the best of both worlds, the PINNs method can be
augmented with data. Figure 1 depicts the tradeoff between using only data or
only physics, and that the sweet spot lies in using both. There have been
several examples showing that the inclusion of solution data significantly
improves the convergence capabilities of the method [54, 9, 8]. This has a
number of distinct advantages:

  1. 1.

The combined data-physics approach can be used to solve ill-posed problems
(for example, problems with missing B.C that have some solution data
available)

  2. 2.

Create surrogate models using lesser solution data that are more consistent
with physics and hence more accurate [34, 65, 1]

  3. 3.

Inverse problems for system identification [70, 27]

The simplicity and non-intrusiveness of the method allow a user to quickly set
up a problem on top of a standard solver.

Figure 1: The spectrum of data-driven versus physics-informed models.
Incorporating governing physics information into the models during creation
serves as an effective form of regularization and often helps reduce the
amount of data required to achieve the same accuracy levels

####  3.2.1 Demonstration

To demonstrate the effect of adding physics-based regularizers to the training
and their effect on the final converged solution, a comparison of data versus
data plus physics was undertaken. In this experiment, a coarse selection (1%
of the total nodes) of points was selected randomly to use for the data term;
all nodes were used for the physics. Since the mesh had approximated 230,000
node points, this meant 2,300 data points. The experiment is divided into two
parts, first, a model is trained on this 1 % data without any physics, and
then a new model is trained starting with 1 % data and then adding the physics
at all nodal points (using a warm start approach described in Section 3.4). To
the author’s knowledge, this represents a novel attempt at using a very coarse
solution to aid convergence of the network to physically correct solutions in
3D for an external flow at a high Reynolds Number. This class of problem is
conventionally challenging for PINNs due to the nonlinearity and gradients
involved.

(a)

(b)

(c)

Figure 2: ANN prediction with and without physics, for very coarse data
supplied. (2(a)) Trained on 1% solution data from solver (2(b)) Trained on 1%
data and physics (2(c)) True Solution from CFD solver

Figure 2 shows the ANN predictions for different case results. It is evident
that by using sparse data, the network is able to converge to the right answer
using the physics-based regularizer. It should be noted that the eddy
viscosities are provided at each node point in this example, to the physics
residual. However, this opens exciting possibilities about using physics-based
regularizers in the future. Data generation costs to create surrogate models
can be greatly reduced by providing solution data on a very coarse grid and
solving the physics on a much finer grid. This has been demonstrated by
Gopakumar et al. [21] for 2D flow over a block. Ghosh et al. [17] also uses
sparse solution data in the domain to create surrogate models for flow over a
cylinder at high Reynolds number in 2D.

It should be noted that the above result was demonstrated using a small
fraction of overall solution data to show the effect of adding physics. In the
rest of the paper, wherever a data term was used for training, all available
training data for a set of input parameters was used.

###  3.3 Domain Decomposition

The domain decomposition process is defined as breaking the overall solution
domain into several subdomains and defining a model for each subdomain. As
discussed by Jagtap et al. [28]and Hu et al. [26], breaking down the overall
solution domain into subdomains can allow for better generalization capability
for complex multiphysics and multiscale problems since predictions for each
subdomain are performed on the sub-network of that domain. Furthermore, the
training of sub-networks obtained via domain decomposition is inherently
parallelizable and the sub-domain models can be placed on separate GPUs with
an associated speedup. Care must be taken as mentioned by Hu et al. [26] to
avoid overfitting the model.

###  3.4 Warm Start for Physics

One of the issues with using standard initialization schemes like Xavier [19]
for training a PINN in a data-free manner, is that the resulting outputs of
the networks have no physical meaning at the start of the training. Minimizing
residual-based losses, in this case, is very ineffective since the gradients
generated by backpropagation are likely to be very noisy and increase the
likelihood of the training converging to a bad local minimum. One solution is
to have a ”warm start” by first training on some solution data only. This has
two benefits:

  1. 1.

It brings the network weights closer to their ”converged” values before using
the PDE residuals to improve the quality of the solution generated by the
network. This allows the PDE residuals to be much more effective in the
training process.

  2. 2.

It reduces the cost of training. As discussed in Section 2.3.3, computing
gradients of PDE residuals via backpropagation is an expensive operation. By
using a warm start via a pretraining approach, one can mitigate this cost by
avoiding training iterations where the physics would have little to no benefit
on the training process.

##  4 Implementation

###  4.1 Model Setup

A simple fully connected architecture with 3 hidden layers of 128 nodes each
for fluid domains, and 2 hidden layers of 64 nodes each for solid domains were
used. This distinction was made because of multiple reasons:

  * •

The solution in the fluid subdomains is likely to be much more non-linear,
particularly in regions of fine sampling like the boundary layer. Hence a
smaller network can be used for the thermal subdomains as solutions in these
subdomains are not likely to require as much expressive power.

  * •

A flow-thermal domain is likely to have more thermal subdomains than fluid
subdomains. This can be seen in the example on the electronics box (Section
5.2.3) where there is 1 flow subdomain and 27 thermal subdomains. In the
interest of cost-effectiveness from a computing perspective, smaller networks
are used to express the solutions in solid domains where only the energy
equation is being solved.

In addition, the space coordinates are encoded in the frequency domain as
described in [62]. The number of outputs of the network depends on the
governing equation. For the Navier Stokes Equations, there are four outputs
(u,v,w,p), and for the energy equation, there is one output (T). The number of
inputs to the network depends on the number of axes of parametrization for the
given problem. For example, if the parametrization is along one axis (say Re)
then the total number of inputs would be 4 (3 spatial + one parameter). This
network architecture is not the result of a parameter search. The model is
trained using an ADAM optimizer. The framework also supports using L-BFGS,
however, it is only recommended to use L-BFGS with full batch size; this is
problematic from a memory perspective for 3D nonlinear problems.

(a)

(b)

Figure 3: Training graphs during different training phases. (3(a)) Training
graph for warm start (3(b)) Training graph when physics is included

The training process for a surrogate model can be divided into two phases,
depicted in Figures 3(a) and 3(b). The input vector to any of the _N_ sub-
networks is the coordinates and the input parameter set of _m_ parameters (if
creating a surrogate model). For the training warm-up (described in Section
3.4), only the solution data loss is included in the total loss, referenced in
Figure 3(a). Once the warm-up is complete (based on a fixed number of
iterations or convergence criteria), include the physics-based losses to
enhance the training (shown in Figure 3(b)). For the case with multiple sub-
domains, the PDE losses are scaled relative to one another at the start of the
training as follows:

$\beta_{i}=\min(\frac{\max_{i=1....N}(\text{PDE Loss})_{i,iter=0}}{(\text{PDE
Loss})_{i,iter=0}},10^{5}),$

The outer min is added to avoid exploding gradients due to very high
coefficients. The inter loss coefficients $\lambda_{i}$ are determined using
Learning Rate Annealing [64].

###  4.2 Design Optimizer Setup

To optimize physical designs with trained models that predict flow-thermal
results, the Particle Swarm Optimization (PSO) [29] algorithm is used. It is a
zero-order optimization method that uses an initial distribution of particles
in the search space and based on the ”fitness” of each particle computes new
positions and velocities of particles in the search space. Eventually, the
particles converge towards the optima.

The current categories of problems being solved in this paper ( i.e
constrained flow-thermal design optimization problems) take the general form

$\min_{\textbf{u}}\;f(\textbf{u}),$

s.t

$g_{i}(\textbf{u})\leq X_{i}\;\;\;\;\;i=1.....N,$

$u_{j}^{min}\leq u_{j}\leq u_{j}^{max}\;\;\;\;j=1..M,$

where $f$ represents an objective function, $g_{i}$ represents the ith
constraint and $X_{i}$ represents constraint values. u represents the input
vector of design parameters (of length M), and each component of u has a
$u_{j}^{min}$ and $u_{j}^{max}$ that they can take. The objective and
constraint values would be a derivative of flow thermal variables like
pressure, temperature, or velocities, or design variables like geometry
lengths, inflow rates, or source terms. The objective and constraints can be
placed on bodies, individual surfaces, or the internal volumes of components
that are a part of the simulation.

In order to solve this problem via the PSO algorithm, the constrained
optimization problem is converted to an unconstrained problem via the penalty
method to get the objective function:

$f(\textbf{u})+\sum_{i=1}^{N}\lambda_{i}\cdot(g_{i}(\textbf{u})-X_{i})^{2}\cdot\mathbb{1}(g_{i}(\textbf{u})>X_{i})+\beta\sum_{i=1}^{N}\mathbb{1}(g_{i}(\textbf{u})>X_{i}).$

The first term is the constrained objective function. The second term
represents the degree of deviation of the constraint from the boundary if the
constraint is violated. The third term adds cost based on the number of
constraints violated, and $\lambda_{i}$ and $\beta$ are constants.

Once the objective function is set up, the ith particle positions (u) and
velocities are updated according to the equations

$\textbf{u}^{i}=\textbf{u}^{i}+\textbf{v}^{i},$

$\textbf{v}^{i}=w\textbf{v}^{i}+c_{1}r_{1}(\textbf{u}^{i}_{best}-\textbf{u}^{i})+c_{2}r_{2}(\textbf{u}_{best}-\textbf{u}^{i}),$

where $c_{1}$,$c_{2}$,$r_{1}$,$r_{2}$ and $w$ are constants.

There are several reasons why PSO is chosen for design optimization:

  1. 1.

It is more economical to use compared to brute force grid search of Design of
Experiment (DoE) space via querying solutions from the ANN. While this may not
seem intuitive, in cases where there is pre-processing required (of say the
point cloud) before the ANN can be queried for the result, It helps to
minimize the amount of computation required. An example of this is when
parametrizing geometry, in which case to query a new u a new points cloud/mesh
would have to be generated.

  2. 2.

The distribution of particles at convergence gives a smaller subspace to do
high-fidelity modeling, rather than returning a single particle as the best
solution. This is important because the modeling process via ANNs is not exact
or high fidelity, and it is better to have a subset of the DoE space returned
rather than a single point, as for example, a gradient-based optimization
method may do. Moreover, there may be multiple regions of the DoE space which
satisfy the constraints and minimize the objective, and the PSO method can
return both regions

Figure 4 depicts the design cycle using the PSO algorithm. In traditional
design optimization, Step 1 is done using CFD solvers, but this can get very
expensive. NN-based surrogates are an ideal tool to replace CFD solvers for
early-stage design cycles.

Figure 4: A design iteration using a design optimization algorithm (like PSO)

##  5 Applications

###  5.1 Forward Solve of 3D Stenosis Problem

As a first step to demonstrate the solver/surrogate modeling framework, flow
through an idealized 3D stenosis geometry at a physiologically relevant
Reynolds number is demonstrated, see Figure 5 for details about the geometry.
To the author’s knowledge, flow through a stenosis has been solved using PINNs
only at a low Reynolds number of approximately 6 (based on inlet diameter)
[60]. Flow through irregular geometries has been solved at a higher Re (500),
but in 2D [44]. In this paper, the stenosis problem is solved at Re 150, and
in 3 dimensions. As pointed out by Krishnapriyan et al. [33], at higher
Reynolds numbers the standard PINN implementation struggles to achieve a good
local minimum. This was confirmed using a standard PINN implementation. To
alleviate this issue, there are several approaches that can be taken. First,
sporadic solution data can be added throughout the domain of interest
(depicted in Figure 6(b)). Tests confirmed that this significantly reduces
convergence time and allows us to arrive at the correct solution in complex
domains. Second, as proposed by NVIDIA continuity planes can be used to
enforce mass/volume flow rate conservation across the cross-section of
internal flows [25] (depicted in Figure 6(a)). This was implemented in the
framework, using basic Euler integration to solve for the mass flow through a
cross-section based on the inlet mass flow rate.

####  5.1.1 Problem Setup

Figure 5: Visual description of stenosis problem

The flow problem through the stenosis is solved by solving the steady-state
Navier-Stokes equations:

$\nabla\cdot\textbf{u}=0,$

$(\textbf{u}\cdot\nabla)\textbf{u}=-\frac{1}{\rho}\nabla\textbf{p}+\nu\nabla\cdot(\nabla\textbf{u}),$

subject to

$\textbf{u}(x_{b1})=0,x_{b1}\in\partial\Omega_{2},$

$\nabla u_{i}(x_{b2})\cdot\textbf{n}=0,x_{b2}\in\partial\Omega_{3},i=1,2,3,$

$\textbf{u}(x_{b3})=g(x_{b3}),x_{b3}\in\partial\Omega_{1},$

where $g(x_{b3})$ represents a profiled input to the stenosis.

In the present problem, a parabolic profiled input is provided with a peak
value inflow of 0.15 m/s. The ratio of areas of the throat to the inlet is
0.36.

The output of the network is approximated as $G_{\theta}$, which is a
4-component output, and use the network output to estimate all 4 components of
the output:

$u=G_{\theta,1},$

$v=G_{\theta,2},$

$w=G_{\theta,3},$

$p=G_{\theta,4}.$

(a)

(b)

Figure 6: Slices of stenosis showing several ways of aiding PINN convergence
(6(a)) Positions of continuity planes (6(b)) Position of solution data
provided

####  5.1.2 Results

(a)

(b)

Figure 7: Solution Comparison. (7(a)) Altair AcuSolve® Solution to stenosis
problem (7(b)) PINN forward solve to stenosis problem.

(a)

(b)

Figure 8: Centerline solution comparisons : PINN versus Altair AcuSolve®
(8(a)) Total Velocity Comparison (8(b)) Pressure Comparison

Figure 7 compares the velocity magnitude returned by the trained PINN model
and Altair AcuSolve® through a 2D slice of the stenosis. As can be seen, the
essential features of the flow are captured. Figure 8(a) and 8(b) compare the
velocity and pressure profile through the center of the stenosis. The
differences between the line plots are attributed to differences in mesh
fineness between the two cases. It should also be noted that making minor
adjustments to the fineness of sampling of the continuity plane affects the
solution from the PINN.

###  5.2 Surrogate Modeling and Design Optimization

In the following subsections, the PINNs surrogate modeling technique is
demonstrated for rapid design iteration in the electronics cooling space.
Three key ideas are combined in this approach: data-driven modeling; physics-
informed methods; and optimization algorithms (see section 4.2). For the data
enhancement, sampling of FE data on a coarse DoE space is undertaken. The
physics-informed nature comes about through the inclusion of a PDE-based
regularizer as previously discussed. The combination of these two methods
allows both accuracy and robustness (as discussed in Section 3.2).

####  5.2.1 Heat Sink Design Optimization

The electronics assembly utilizes a chip with a fin-type heatsink on top to
dissipate heat into the surrounding fluid. The chip-heatsink assembly is
cooled by forced convection of air. The geometry and setup are shown in Figure
9. The governing equations solved for this conjugate heat transfer problem
are, the Navier-Stokes for the flow domain surrounding the chip-heatsink
assembly:

$\nabla\cdot\textbf{u}=0,$
$(\textbf{u}\cdot\nabla)\textbf{u}=-\frac{1}{\rho}\nabla\textbf{p}+\nu\nabla\cdot(\nabla\textbf{u}),$

subject to no-slip boundary conditions on the surrounding box and on the chip-
heatsink assembly with a variable velocity at the inlet. The energy equation
in both fluid and solid reads:

$\nabla^{2}T+\dot{q}_{src}-\textbf{u}\cdot\nabla T=0,$

where the velocity is zero in the solid. At the interface between the fluid
and solid domain (fluid-sink, sink-chip, and fluid-chip) the interface
condition is applied by minimizing the following loss terms as shown in [28]:

$L_{flux}=\frac{1}{N_{int}}\sum_{i=1}^{N_{int}}(f_{d_{1}}(\textbf{u}(x_{i}))\cdot\textbf{n}_{d_{1}}+f_{d_{2}}(\textbf{u}(x_{i}))\cdot\textbf{n}_{d_{2}})^{2},$

$L_{val}=\frac{1}{N_{int}}\sum_{i=1}^{N_{int}}(\textbf{u}_{d_{j}}(x_{i})-\overline{\textbf{u}_{d_{j}}(x_{i})})^{2},$

where $\textbf{n}_{d1}=-\textbf{n}_{d2}$ and j=1,2. The average is taken over
j. $d_{1}$ and $d_{2}$ refer to the domains on both sides of the interface,
and $N_{int}$ is the number of node points on the interface.

The goal of the heatsink optimization is to find the maximum power the chip
can generate subject to certain constraints.

The design variables that can be altered for this present optimization are:

  * •

Inflow Velocity

  * •

Fin height

  * •

Source term in the chip (has to be maximized)

The upper and lower limits of each of the design variables mentioned above are
summarized in Table 1. The inlet velocity is set based on typical values found
in literature [36] and corresponds to a Reynolds number range of Re 10,300 to
Re 24,000.

Parameter | Lower Value | Upper Value
---|---|---
Inflow Velocity ($m/s$) | 3 | 7
Fin Height ($mm$) | 15 | 23
Source Term ($W$) | 30 | 60
Table 1: Design of Experiments space axes ranges for the heat sink design
optimization

The domain is divided into three parts (Fluid, chip and sink) and each part
has its own prediction network. The material properties were as follows: Chip
is made of FR-4; Heatsink is made of Aluminium; and the cooling fluid is air.
The material properties are shown in Table 2

Domain | Density ($kg/m^{3}$) |  Dynamic Viscosity($kg/ms$) | Conductivity($W/mK$) | Specific Heat($J/KgK$)
---|---|---|---|---
Chip | 7190.0 | - | 50.208 | 460.0
Sink | 2770.0 | - | 175.0 | 986.0
Air | 1.225 | 1.78 e-5 | 0.0251 | 1006.0
Table 2: Physical properties of different subdomains in the heat sink design
optimization Figure 9: Basic problem geometry and flow depiction

##### The Model Creation Process

The sampling of the above Design of DoE space is done via an efficient space-
sampling method to optimally fill the DoE space [41]. Sampling strategies are
important to minimize the number of samples while keeping the design
exploration process accurate and simultaneously cost-efficient. The model was
trained on a single Titan V GPU card, on which the overall training time was
around 130 mins.

##### The Optimization Process

In order to optimize designs based on user-defined constraints, the created
surrogate models are interfaced with an optimizer that solves a generic
constrained optimization problem, described in section 4.2. The same is
demonstrated in the example case:

Pressure drop across the heat sink channel $<=$ 11 Pa

Maximum temperature anywhere on the chip $<=$ 350 K

(a)

(b)

(c)

Figure 10: Design optimization iterations of the heat sink problem (10(a))
Iteration 0 (10(b)) Iteration 5 (10(c)) Iteration 10

Each snapshot in Figure 10 represents a design iteration, and each particle
represents a point in the DoE space. Each axis of a plot represents a
parameter axis.

For the given constraints, the particles converge to a much smaller region of
the DoE space. Due to imperfections in the overall modeling process, the aim
should not be to return a single design point to the user but a massively
truncated DoE space in which they can then run high-fidelity CFD to optimize
their design. For the sake of demonstration of results, however, a single
point in this truncated DoE space is chosen. The design point returned by the
optimizer in this case is:

Inflow Velocity: 6 m/s

Chip Power: 50W

Fin Height: 17mm

To test the result, high-fidelity CFD of the problem is run at the above
design point and compared to the result from the framework. As shown in
Figures 11 and 12, not only are the differences in solution minimal, but the
given design point nearly satisfies the design constraints. The time taken for
the overall design optimization after model training was 10 minutes, and this
approach shows how effective surrogate modeling-based optimization approaches
can be in reducing turnaround times for design problems. The resultant best
parameters can hence be tweaked a little by using high-fidelity CFD to obtain
a satisfactory design.

Figure 11: Temperature plot through a slice at the rear of the sink (from
bottom to top). The comparison between the high-fidelity solution on the fine
mesh and the PINN prediction on a coarser mesh shows good agreement Figure 12:
Pressure plot through a slice through the middle of the flow channel (from
left to right). The comparison between the high-fidelity solution on the fine
mesh and the PINN prediction on a coarser mesh shows good agreement

####  5.2.2 Printed Circuit Board Thermal Analysis

In this example, thermal analysis of a PCB board and its components is done.
The setup is shown in Figure 13, with the material properties of the
components described in Table 3. The value ranges for each axis of the DoE is
shown in Table 4 The goal is to compare the solution from the ANN surrogate to
the solution from the CFD solver and test the ability of the model to
generalize to new test points. After that, a similar optimization problem to
the previous section is solved. The physics of the problem is identical to
that of the heat sink design problem and is not described again for brevity.

Figure 13: Basic problem geometry and flow depiction Domain | Density ($kg/m^{3}$) |  Dynamic Viscosity($kg/ms$) |  Conductivity ($W/mK$) |  Specific Heat ($J/KgK$)
---|---|---|---|---
Air | 1.225 | 1.78 e-5 | 0.0251 | 1006.0
Capacitor | 700.0 | - | 82 | 2330.0
Chip | 700.0 | - | 82 | 2330.0
Sink | 2770.0 | - | 175 | 986.0
PCB | 1850 | - | 0.294 | 1900.0
Table 3: Material Properties for PCB analysis problem Parameter | Lower Value | Upper Value
---|---|---
Inflow Velocity ($m/s$) | 3 | 9
Source Term(Chip) ($W$) | 30 | 100
Source Term(Capacitor) ($W$) | 3 | 10
Table 4: DoE parameter ranges for PCB analysis problem

##### The Model Creation Process

The sampling of the above DoE space is done via an efficient space-sampling
method, to minimize the number of samples required to fill in the DoE space in
an efficient manner. 8 data points are used for the model creation process.
The model was trained on a single Titan V GPU card, on which the overall
training time was 100 mins. The total time taken to generate the model was
around 120 minutes (including data generation time, done on a highly efficient
32-core cluster).

##### Results

Prediction versus truths for 2 line plots for two randomly selected test
points (far enough from any training points) within the DoE space as defined
above. The two lines were chosen to get a quantitative comparison between
results while capturing the relevant physics. Though only results for energy
equation solves were shown, the model trained was a flow-thermal one.

Parameter set 1

  * •

Inflow Velocity: 5 m/s

  * •

Thermal Power in Chip: 90 W

  * •

Thermal Power in Capacitor: 10 W

Figure 14: Line plot results for Parameter Set 1: CFD prediction versus
Network Prediction

Figure 14 compares the Altair AcuSolve® and ANN solution through two lines.
The left figure captures the solution in a fin of the sink, in a region that
is the hottest during the operation of the chip. The right figure captures the
temperature in the streamwise direction. The solutions for this test case show
good agreement, except for the wake area behind the sink. This is attributed
to imperfections in the flow solution in the wake of the sink.

Parameter set 2

  * •

Inflow Velocity: 8 m/s

  * •

Thermal Power in Chip: 50 W

  * •

Thermal Power in Capacitor: 5 W

Figure 15: Line plot results for Parameter Set 2: CFD prediction versus
Network Prediction

Figure 15 again compares the Altair AcuSolve® and ANN solution through two
lines. The solutions for this test case also show good agreement, with some
deviation in the wake of the sink.

##### Design Optimization

The optimization feature of the framework is used to solve a similar design
optimization problem as before. One example problem is as follows:

Maximize $Q_{chip}$ s.t

Pressure drop across channel $<=$ 20 Pa

Maximum Temperature in Chip $<=$ 340 K

Maximum Temperature in Capacitor $<=$ 340 K

This makes for an interesting optimization problem as there is a slight
interaction between the wake of the chip and the capacitor

(a)

(b)

(c)

Figure 16: Design optimization iterations of the PCB design problem (16(a))
Iteration 0 (16(b)) Iteration 5 (16(c)) Iteration 10

Figure 16 shows the PSO algorithm converging to the best region of the DoE
space, which satisfies the constraints and solves the optimization problem.
Although the purpose of the optimizer is not to return a single point but to
return a truncated DoE space, for argument’s sake a single returned point,
which has the parameters:

Inflow Velocity: 5.5 m/s

Chip Source Term: 5.5 $e6W/m^{3}$

Capacitor Source Term: 3.87 $e5W/m^{3}$

The returned answer is compared to the CFD result at this optimal point.
Looking at the constraints defined above, it is visible in Figure 17 that the
surrogate-optimizer combination has very efficiently returned an optimal
operating point that maximizes the utilization of the chip (and the thermal
power dissipated).

(a)

(b)

(c)

Figure 17: Checking values of physical variables in solution domain to see if
they satisfy constraints. (17(a)) Start of the plot indicates the temperature
in the PCB followed by chip (17(b)) Pressure across flow channel. Left end
shows pressure at the inlet (17(c)) Temperature through the length of the
capacitor

####  5.2.3 Electronics Box Analysis

Finally, the ability of the framework to model the flow thermal
characteristics of an entire electronics box is presented. It represents the
most challenging problem of the three due to having many components (28 bodies
in total), being made up of several different materials, and having many
different heat generation sources.

##### Model Creation Process

Figure 18: Basic problem geometry and flow depiction

The sampling of the DoE space is done via the same efficient space sampling
method of the DoE space. The ranges of parameters for each axis are given in
Table 5. 15 data points were used for the model creation process. The model
training time was around 85 minutes (done on a Titan V GPU), and the total
time taken to generate the model was around 200 minutes (which includes data
generation time, created on an 8-core CPU machine).

Parameter | Lower Value | Upper Value
---|---|---
Inflow Velocity ($m/s$) | 0.5 | 5
Source Term(BGA) ($W/m^{3}$) | $1e^{6}$ | $1e^{7}$
Source Term(D25 Die) ($W/m^{3}$) | $5e^{6}$ | $5e^{7}$
Source Term(D26 Die) ($W/m^{3}$) | $5e^{6}$ | $5e^{7}$
Source Term(Q11 Die) ($W/m^{3}$) | $1e^{7}$ | $1e^{8}$
Table 5: DoE parameter ranges for the electronics box problem

##### Results

Figure 19 shows the temperature and pressure contours for a selected randomly
drawn test point. Figures 19(a) and 19(b) show the temperature comparison, and
Figures 19(c) and 19(d) show the pressure prediction. This qualitative
accuracy of the test results versus CFD demonstrates the ability of the model
to generalize well and predict flow-thermal results within the DoE design
space, for very nontrivial geometries like the electronics box.

(a)

(b)

(c)

(d)

Figure 19: Comparisons between a commercial solver (Altair AcuSolve®) and the
surrogate model. (19(a)) CFD temperature prediction on the test case (19(b))
Model temperature prediction on the test case (19(c)) CFD pressure prediction
on the test case (19(d)) Model pressure prediction on the test case

####  5.2.4 Cost Benefit Analysis Compared to Standard CFD

In this section, a quantitative comparison of PINNs bases surrogate modeling
versus CFD for DoE problems is presented. For the PINNs model training was
performed on a single Titan V GPU card.

##### Heat Sink Design Optimization

Table 6 shows the cost-benefit analysis of the hybrid PINNs-CFD surrogate
model versus CFD for the heat sink design optimization problem shown in
Section 5.2.1. The comparison, visualized in Figure 20 shows the total time
taken to optimize the design using the PSO method and compares the time taken
versus the number of iterations. The ”model training time” encapsulates the
time taken to create the data and train the model, so that it is a fair
comparison.

Solve Type | Model Training Time |  Time for a Design Iteration |  Time for 10 Design Iterations
---|---|---|---
CFD (4 cores) | - | 160 min | 1600 min
PINN (1 GPU + 1 core) | 286 mins | 55 seconds | 300 mins
Table 6: Comparing design iteration times using CFD versus ANN surrogate for
the heatsink problem

Figure 20: Time comparison of doing PSO-based design iterations using the
surrogate versus CFD. Once the surrogate returns a truncated DoE space, the
designer can perform high-fidelity CFD to fine-tune the design

##### PCB Thermal Analysis

Table 7 and Figure 21 shows a similar comparison for the PCB electronics
assembly (see section 5.2.2). It is evident from both results that using
accurate surrogate models can cap the cost of thoroughly solving DoE problems
and reduce costs and increase efficiency.

Solve Type | Model Training Time | Time for each prediction
---|---|---
CFD (32 cores) | - | 120 s
PINN (1 GPU + 1 core) | 116 mins | 3 seconds
Table 7: Comparing design iteration times using CFD versus ANN surrogate for
the PCB design problem

Figure 21: Comparison of time cost of using a surrogate to query versus CFD

####  5.2.5 Accuracy Comparisons

The created network models have several features that allow the creation of
surrogates that are accurate, train quickly, and show good generalization
capabilities. To demonstrate this, a few comparisons of predictions (for a
test case) are shown, done with a standard fully connected network with none
of the features that are described in Section 3, with predictions done using a
model from the framework. These are example cases where using a standard data-
driven network can lead to regions with unphysical results in the domain. Both
models in both comparisons are created with the same dataset, the same
hyperparameters (besides those applicable to only Physics-ML models) and
trained on the same hardware for the same number of iterations.

##### Example 1: Temperature on PCB

(a)

(b)

(c)

(d)

Figure 22: Temperature comparison on the PCB between predictions from standard
data-driven NN versus PINN-based networks. (22(a)) Prediction from framework
network (22(b)) Prediction from standard fully connected NN (22(c)) True
Solution (22(d)) Insets from the prediction of standard NN showing unphysical
results, especially near the boundaries

Figure 22 shows an example qualitative comparison of using models created by
the framework versus standard data-driven models. The result from the standard
data-driven model has regions of unphysical results, shown in Figure 22(d).

##### Example 2: Pressure around Electronics Box

(a)

(b)

(c)

Figure 23: Pressure comparisons for flow over the electronics box (23(a))
Prediction from framework network (23(b)) Prediction from standard fully
connected NN (23(c)) True Solution

Figure 23 shows a similar difference in the accuracy of the result, in that
the standard data-driven NN ends up with regions of non-physical results, all
other applicable model creation parameters remaining the same.

####  5.2.6 Active Research Areas

Given that PINNs is such a promising yet young field, there are several areas
of active research on applying them to different problems. A few pertinent
ones are addressed here:

##### Turbulence Modeling

One unexplored area in PINNs is incorporating turbulence modeling into the
learning process. Currently in the Altair framework, for creating surrogate
models, the eddy viscosities from Altair AcuSolve® are used. Only very simple
turbulence models have been implemented in 3D [25] for PINNs. Ghosh et al.
[17] implemented the $\kappa-\epsilon$ 2 equation model for 2D flows, but this
has yet to be extended to 3D. The lack of research on turbulence models for
PINNs means that only the simplest turbulence models can be used currently,
though Eivazi et al. [15] used PINNs to simulate simpler turbulent flow cases.

##### Geometry Parametrization

One of the most pressing problems that Neural Network based surrogate models
have promised to solve is that of shape optimization. Design exploration by
changing geometry is a time-consuming and complicated process in traditional
CFD due to having to re-mesh for every small change, followed by running the
solver. Yet, topology and shape optimization has become a key feature of most
commercial solvers, highlighting the demand for the feature. The quick
prediction capability of ANNs, coupled with the ability to predict on point
clouds instead of meshes makes this method very promising in this application.

A recently popular approach is to use graph neural networks to read in point
clouds or meshes directly [47, 39, 45], opening the door to exciting
possibilities of parametrizing complex geometry relatively easily. There are
also works using generative design and neural operators to solve problems of
topology optimization. There have been several instances of solving these for
structural problems but more recently there has been work solving flow-thermal
topology optimization problems as well[43, 61, 55].

##### Uncertainty Quantification

One of the key metrics in being able to trust the results of ANN-based
surrogates is to have a quantitative idea about the uncertainty of the
prediction of the model. There has been a recent increase in the volume of
works related to this topic [72, 3, 71], as practitioners and researchers have
realized the importance of providing uncertainty measurements.

##  6 Conclusions and Future Work

In this paper, Physics Informed Neural Networks were introduced and some of
their current limitations were discussed. Recent research on how to enhance
the convergence properties of PINNs was discussed and a novel example in 3D
was demonstrated, showing the benefit of adding physics to sparsely provided
solution data. A demo problem solve was also demonstrated showing that PINNs
can be used to solve realistic problems in a data-free manner for the 3D
Navier Stokes equations. Their applications were also demonstrated for
industry-grade flow-thermal surrogate modeling problems with the inclusion of
some solution data and showed how combined with optimization algorithm models
can arrive at optimal designs automatically based on user-defined constraints,
in near real-time. It was also demonstrated that models from the framework
perform better than basic data-driven ANNs for the exact same hyperparameters
while showing the cost-benefit analysis for creating and using these models
(including data creation and training time). The results shown in this paper
can be built on to reduce surrogate cost creation, improve accuracy and reduce
the black-box nature of ANN-based surrogate models.

There are multiple avenues through which the work shown in this paper can be
improved. Research has to be done to improve convergence and offer guarantees
of PINNs training toward the correct local minimums. This will further reduce
data requirements for the creation of ANN-based surrogates. Further research
also needs to be done for unsteady problems, which form an important class of
problems that require surrogate models to model. More work needs to be done to
incorporate and use other turbulence models with PINNs. Finally, one of the
core issues with using physics-based regularizers is the additional cost they
impose on the training process, and an important yet unexplored area of
research is being able to use the physics regularization terms in a more
memory and compute-efficient manner.

Acknowledgements
This research did not receive any specific grant from funding agencies in the
public or not-for-profit sectors, or from any external commercial entities.
The authors gratefully acknowledge the use of Altair Engineering Inc.’s
computing facilities for running experiments.

CRediT authorship contribution statement

Saakaar Bhatnagar: Formal Analysis, Investigation, Methodology, Software,
Validation, Writing-original draft. Andrew Comerford: Conceptualization,
Investigation, Project Administration, Supervision, Writing- review and
editing Araz Banaeizadeh: Conceptualization, Project Administration,
Supervision, Writing- review and editing

## References

  * Antonello et al. [2023] Federico Antonello, Jacopo Buongiorno, and Enrico Zio.  Physics informed neural networks for surrogate modeling of accidental scenarios in nuclear power plants.  _Nuclear Engineering and Technology_ , 2023.
  * Basir and Senocak [2022] Shamsulhaq Basir and Inanc Senocak.  Critical investigation of failure modes in physics-informed neural networks.  In _AIAA SCITECH 2022 Forum_. American Institute of Aeronautics and Astronautics, 2022.  doi: 10.2514/6.2022-2353.  URL https://doi.org/10.2514%2F6.2022-2353.
  * Bharadwaja et al. [2022] BVSS Bharadwaja, Mohammad Amin Nabian, Bharatkumar Sharma, Sanjay Choudhry, and Alankar Alankar.  Physics-informed machine learning and uncertainty quantification for mechanics of heterogeneous materials.  _Integrating Materials and Manufacturing Innovation_ , 11(4):607–627, 2022.
  * Bhatnagar et al. [2019] Saakaar Bhatnagar, Yaser Afshar, Shaowu Pan, Karthik Duraisamy, and Shailendra Kaushik.  Prediction of aerodynamic flow fields using convolutional neural networks.  _Computational Mechanics_ , 64:525–545, 2019.
  * Bischof and Kraus [2021] Rafael Bischof and Michael Kraus.  Multi-objective loss balancing for physics-informed deep learning.  _arXiv preprint arXiv:2110.09813_ , 2021.
  * Bora et al. [2021] Aniruddha Bora, Weizhong Dai, Joshua P Wilson, and Jacob C Boyt.  Neural network method for solving parabolic two-temperature microscale heat conduction in double-layered thin films exposed to ultrashort-pulsed lasers.  _International Journal of Heat and Mass Transfer_ , 178:121616, 2021.
  * Brunton et al. [2020] Steven L Brunton, Bernd R Noack, and Petros Koumoutsakos.  Machine learning for fluid mechanics.  _Annual review of fluid mechanics_ , 52:477–508, 2020.
  * Cai et al. [2021a] Shengze Cai, Zhiping Mao, Zhicheng Wang, Minglang Yin, and George Em Karniadakis.  Physics-informed neural networks (pinns) for fluid mechanics: A review.  _Acta Mechanica Sinica_ , 37(12):1727–1738, 2021a.
  * Cai et al. [2021b] Shengze Cai, Zhicheng Wang, Sifan Wang, Paris Perdikaris, and George Em Karniadakis.  Physics-Informed Neural Networks for Heat Transfer Problems.  _Journal of Heat Transfer_ , 143(6):060801, 2021b.  doi: 10.1115/1.4050542.
  * Calzolari and Liu [2021] Giovanni Calzolari and Wei Liu.  Deep learning to replace, improve, or aid cfd analysis in built environment applications: A review.  _Building and Environment_ , 206:108315, 2021.
  * Cao et al. [2019] Yuan Cao, Zhiying Fang, Yue Wu, Ding-Xuan Zhou, and Quanquan Gu.  Towards understanding the spectral bias of deep learning.  _arXiv preprint arXiv:1912.01198_ , 2019.
  * Cheng et al. [2021] Chen Cheng, Hao Meng, Yong-Zheng Li, and Guang-Tao Zhang.  Deep learning based on pinn for solving 2 dof vortex induced vibration of cylinder.  _Ocean Engineering_ , 240:109932, 2021.
  * D. Jagtap and Em Karniadakis [2020] Ameya D. Jagtap and George Em Karniadakis.  Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations.  _Communications in Computational Physics_ , 10, 2020.  doi: https://doi.org/10.4208/cicp.OA-2020-0164.
  * Devlin et al. [2018] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ , 2018.
  * Eivazi et al. [2022] Hamidreza Eivazi, Mojtaba Tahani, Philipp Schlatter, and Ricardo Vinuesa.  Physics-informed neural networks for solving reynolds-averaged navier–stokes equations.  _Physics of Fluids_ , 34(7), 2022.  doi: 10.1063/5.0095270.  URL https://doi.org/10.1063%2F5.0095270.
  * Finn et al. [2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine.  Model-agnostic meta-learning for fast adaptation of deep networks.  In _International conference on machine learning_ , pages 1126–1135. PMLR, 2017.
  * Ghosh et al. [2023] Shinjan Ghosh, Amit Chakraborty, Georgia Olympia Brikis, and Biswadip Dey.  Rans-pinn based simulation surrogates for predicting turbulent flows.  _arXiv preprint arXiv:2306.06034_ , 2023.
  * Gladstone et al. [2022] Rini J Gladstone, Mohammad A Nabian, and Hadi Meidani.  Fo-pinns: A first-order formulation for physics informed neural networks.  _arXiv preprint arXiv:2210.14320_ , 2022.
  * Glorot and Bengio [2010] Xavier Glorot and Yoshua Bengio.  Understanding the difficulty of training deep feedforward neural networks.  In _Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics_ , Proceedings of Machine Learning Research, pages 249–256. PMLR, 2010.
  * Gonçalves et al. [2020] Gabriel FN Gonçalves, Assen Batchvarov, Yuyi Liu, Yuxin Liu, Lachlan R Mason, Indranil Pan, and Omar K Matar.  Data-driven surrogate modeling and benchmarking for process equipment.  _Data-Centric Engineering_ , 1:e7, 2020.
  * Gopakumar et al. [2023] Vignesh Gopakumar, Stanislas Pamela, and Debasmita Samaddar.  Loss landscape engineering via data regulation on pinns.  _Machine Learning with Applications_ , 12:100464, 2023.
  * Guo et al. [2016] Xiaoxiao Guo, Wei Li, and Francesco Iorio.  Convolutional neural networks for steady flow approximation.  pages 481–490. Association for Computing Machinery, 2016.  doi: 10.1145/2939672.2939738.
  * Hansen et al. [2023] Derek Hansen, Danielle Maddix Robinson, Shima Alizadeh, Gaurav Gupta, and Michael Mahoney.  Learning physical models that can respect conservation laws.  In _ICML 2023_ , 2023.
  * He and Pathak [2020] Haiyang He and Jay Pathak.  An unsupervised learning approach to solving heat equations on chip based on auto-encoder and image gradient.  _arXiv preprint arXiv:2007.09684_ , 2020.
  * Hennigh et al. [2021] Oliver Hennigh, Susheela Narasimhan, Mohammad Amin Nabian, Akshay Subramaniam, Kaustubh Tangsali, Zhiwei Fang, Max Rietmann, Wonmin Byeon, and Sanjay Choudhry.  Nvidia simnet™: An ai-accelerated multi-physics simulation framework.  In _International conference on computational science_ , pages 447–461. Springer, 2021.
  * Hu et al. [2022] Zheyuan Hu, Ameya D. Jagtap, George Em Karniadakis, and Kenji Kawaguchi.  When do extended physics-informed neural networks (XPINNs) improve generalization?  _SIAM Journal on Scientific Computing_ , 44(5):A3158–A3182, 2022.  doi: 10.1137/21m1447039.  URL https://doi.org/10.1137%2F21m1447039.
  * Ishitsuka and Lin [2023] Kazuya Ishitsuka and Weiren Lin.  Physics-informed neural network for inverse modeling of natural-state geothermal systems.  _Applied Energy_ , 337:120855, 2023.  ISSN 0306-2619.  doi: https://doi.org/10.1016/j.apenergy.2023.120855.  URL https://www.sciencedirect.com/science/article/pii/S0306261923002192.
  * Jagtap et al. [2020] Ameya D. Jagtap, Ehsan Kharazmi, and George Em Karniadakis.  Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems.  _Computer Methods in Applied Mechanics and Engineering_ , 365:113028, 2020.  doi: https://doi.org/10.1016/j.cma.2020.113028.
  * Kennedy and Eberhart [1995] J. Kennedy and R. Eberhart.  Particle swarm optimization.  In _Proceedings of ICNN’95 - International Conference on Neural Networks_ , volume 4, pages 1942–1948, 1995.  doi: 10.1109/ICNN.1995.488968.
  * Kingma and Ba [2014] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Knight [2001] Doyle Knight.  _Design optimization in computational fluid dynamics_.  Springer US, Boston, MA, 2001.  doi: 10.1007/0-306-48332-7˙90.  URL https://doi.org/10.1007/0-306-48332-7_90.
  * Krath et al. [2021] Elizabeth H. Krath, Forrest L. Carpenter, Paul G.A. Cizmas, and David A. Johnston.  An efficient proper orthogonal decomposition based reduced-order model for compressible flows.  _Journal of Computational Physics_ , 426:109959, 2021.  doi: 10.1016/j.jcp.2020.109959.
  * Krishnapriyan et al. [2021] Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney.  Characterizing possible failure modes in physics-informed neural networks.  _Advances in Neural Information Processing Systems_ , 34:26548–26560, 2021.
  * Lee and You [2019] Sangseung Lee and Donghyun You.  Data-driven prediction of unsteady flow over a circular cylinder using deep learning.  _Journal of Fluid Mechanics_ , 879:217–254, 2019.  doi: 10.1017/jfm.2019.700.
  * Li et al. [2020] Zongyi Li, Nikola B. Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar.  Fourier neural operator for parametric partial differential equations.  _CoRR_ , 2020.
  * Lindstedt and Karvinen [2012] Matti Lindstedt and Reijo Karvinen.  Optimization of plate fin arrays with laminar and turbulent forced convection.  In _Journal of Physics: Conference Series_ , volume 395, page 012059\. IOP Publishing, 2012.
  * Liu et al. [2022] Ruo-Lin Liu, Yue Hua, Zhi-Fu Zhou, Yubai Li, Wei-Tao Wu, and Nadine Aubry.  Prediction and optimization of airfoil aerodynamic performance using deep neural network coupled bayesian method.  _Physics of Fluids_ , 34(11), 2022.
  * Liu et al. [2019] Zeyu Liu, Yantao Yang, and Qing-Dong Cai.  Solving differential equation with constrained multilayer feedforward network.  _arXiv preprint arXiv:1904.06619_ , 2019.
  * Mayr et al. [2023] Andreas Mayr, Sebastian Lehner, Arno Mayrhofer, Christoph Kloss, Sepp Hochreiter, and Johannes Brandstetter.  Boundary graph neural networks for 3d simulations.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 37, pages 9099–9107, 2023.
  * Melas-Kyriazi [2020] Luke Melas-Kyriazi.  The mathematical foundations of manifold learning.  _arXiv preprint arXiv:2011.01307_ , 2020.
  * Morris and Mitchell [1995] Max D. Morris and Toby J. Mitchell.  Exploratory designs for computational experiments.  _Journal of Statistical Planning and Inference_ , 43(3):381–402, 1995.  ISSN 0378-3758.  doi: https://doi.org/10.1016/0378-3758(94)00035-T.
  * Nekkanti and Schmidt [2023] Akhil Nekkanti and Oliver T. Schmidt.  Gappy spectral proper orthogonal decomposition.  _Journal of Computational Physics_ , 478:111950, 2023.  doi: 10.1016/j.jcp.2023.111950.
  * Oh et al. [2019] Sangeun Oh, Yongsu Jung, Seongsin Kim, Ikjin Lee, and Namwoo Kang.  Deep Generative Design: Integration of Topology Optimization and Generative Models.  _Journal of Mechanical Design_ , 141(11):111405, 2019.  doi: 10.1115/1.4044229.
  * Oldenburg et al. [2022] Jan Oldenburg, Finja Borowski, Alper Öner, Klaus-Peter Schmitz, and Michael Stiehm.  Geometry aware physics informed neural network surrogate for solving navier–stokes equation (gapinn).  _Advanced Modeling and Simulation in Engineering Sciences_ , 9(1):8, 2022.  doi: https://doi.org/10.1186/s40323-022-00221-z.
  * Pegolotti et al. [2023] Luca Pegolotti, Martin R Pfaller, Natalia L Rubio, Ke Ding, Rita Brugarolas Brufau, Eric Darve, and Alison L Marsden.  Learning reduced-order models for cardiovascular simulations with graph neural networks.  _arXiv preprint arXiv:2303.07310_ , 2023.
  * Penwarden et al. [2023] Michael Penwarden, Shandian Zhe, Akil Narayan, and Robert M Kirby.  A metalearning approach for physics-informed neural networks (pinns): Application to parameterized pdes.  _Journal of Computational Physics_ , 477:111912, 2023.
  * Pfaff et al. [2020] Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W Battaglia.  Learning mesh-based simulation with graph networks.  _arXiv preprint arXiv:2010.03409_ , 2020.
  * Rahaman et al. [2019] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville.  On the spectral bias of neural networks.  In _International Conference on Machine Learning_ , pages 5301–5310. PMLR, 2019.
  * Raissi et al. [2019] M. Raissi, P. Perdikaris, and G.E. Karniadakis.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.  _Journal of Computational Physics_ , 378:686–707, 2019\.  doi: https://doi.org/10.1016/j.jcp.2018.10.045.
  * Raissi and Karniadakis [2018] Maziar Raissi and George Em Karniadakis.  Hidden physics models: Machine learning of nonlinear partial differential equations.  _Journal of Computational Physics_ , 357:125–141, 2018\.  doi: 10.1016/j.jcp.2017.11.039.
  * Ramesh et al. [2021] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever.  Zero-shot text-to-image generation.  In _International Conference on Machine Learning_ , pages 8821–8831. PMLR, 2021.
  * Rao et al. [2021] Chengping Rao, Hao Sun, and Yang Liu.  Physics-informed deep learning for computational elastodynamics without labeled data.  _Journal of Engineering Mechanics_ , 147(8):04021043, 2021.
  * Saad et al. [2023] Nadim Saad, Gaurav Gupta, Shima Alizadeh, and Danielle Maddix Robinson.  Guiding continuous operator learning through physics-based boundary constraints.  In _ICLR 2023_ , 2023.
  * Sharma et al. [2023] Rahul Sharma, Maziar Raissi, and Yuebin Guo.  Physics-informed deep learning of gas flow-melt pool multi-physical dynamics during powder bed fusion.  _CIRP Annals_ , 2023.  doi: https://doi.org/10.1016/j.cirp.2023.04.005.
  * Shukla et al. [2023] Khemraj Shukla, Vivek Oommen, Ahmad Peyvan, Michael Penwarden, Luis Bravo, Anindya Ghoshal, Robert M Kirby, and George Em Karniadakis.  Deep neural operators can serve as accurate surrogates for shape optimization: a case study for airfoils.  _arXiv preprint arXiv:2302.00807_ , 2023.
  * Sitzmann et al. [2020] Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein.  Implicit neural representations with periodic activation functions.  _Advances in neural information processing systems_ , 33:7462–7473, 2020.
  * Son et al. [2023] Hwijae Son, Sung Woong Cho, and Hyung Ju Hwang.  Enhanced physics-informed neural networks with augmented lagrangian relaxation method (al-pinns).  _Neurocomputing_ , page 126424, 2023.
  * Subramanian et al. [2022] Shashank Subramanian, Robert M Kirby, Michael W Mahoney, and Amir Gholami.  Adaptive self-supervision algorithms for physics-informed neural networks.  _arXiv preprint arXiv:2207.04084_ , 2022.
  * Sukumar and Srivastava [2022] N. Sukumar and Ankit Srivastava.  Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks.  _Computer Methods in Applied Mechanics and Engineering_ , 389:114333, 2022.  doi: 10.1016/j.cma.2021.114333.
  * Sun et al. [2020] Luning Sun, Han Gao, Shaowu Pan, and Jian-Xun Wang.  Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data.  _Computer Methods in Applied Mechanics and Engineering_ , 361:112732, 2020.  doi: 10.1016/j.cma.2019.112732.
  * Sun et al. [2023] Yubiao Sun, Ushnish Sengupta, and Matthew Juniper.  Physics-informed deep learning for simultaneous surrogate modeling and pde-constrained optimization of an airfoil geometry.  _Computer Methods in Applied Mechanics and Engineering_ , 411:116042, 2023.  doi: https://doi.org/10.1016/j.cma.2023.116042.
  * Tancik et al. [2020] Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng.  Fourier features let networks learn high frequency functions in low dimensional domains.  _Advances in Neural Information Processing Systems_ , 33:7537–7547, 2020.
  * Varoquaux and Cheplygina [2022] Gaël Varoquaux and Veronika Cheplygina.  Machine learning for medical imaging: methodological failures and recommendations for the future.  _NPJ digital medicine_ , 5(1):48, 2022.
  * Wang et al. [2021a] Sifan Wang, Yujun Teng, and Paris Perdikaris.  Understanding and mitigating gradient flow pathologies in physics-informed neural networks.  _SIAM Journal on Scientific Computing_ , 43(5):A3055–A3081, 2021a.  doi: 10.1137/20M1318043.
  * Wang et al. [2021b] Sifan Wang, Hanwen Wang, and Paris Perdikaris.  Learning the solution operator of parametric partial differential equations with physics-informed deeponets.  _Science Advances_ , 7(40):eabi8605, 2021b.  doi: 10.1126/sciadv.abi8605.
  * Wang et al. [2022a] Sifan Wang, Shyam Sankaran, and Paris Perdikaris.  Respecting causality is all you need for training physics-informed neural networks.  _arXiv preprint arXiv:2203.07404_ , 2022a.
  * Wang et al. [2022b] Sifan Wang, Hanwen Wang, and Paris Perdikaris.  Improved architectures and training algorithms for deep operator networks.  _Journal of Scientific Computing_ , 2022b.
  * Wang et al. [2022c] Sifan Wang, Xinling Yu, and Paris Perdikaris.  When and why pinns fail to train: A neural tangent kernel perspective.  _Journal of Computational Physics_ , 449:110768, 2022c.
  * Wu et al. [2022] Benjamin Wu, Oliver Hennigh, Jan Kautz, Sanjay Choudhry, and Wonmin Byeon.  Physics informed rnn-dct networks for time-dependent partial differential equations.  In _International Conference on Computational Science_ , pages 372–379. Springer, 2022.
  * Xu and Darve [2022] Kailai Xu and Eric Darve.  Physics constrained learning for data-driven inverse modeling from sparse observations.  _Journal of Computational Physics_ , 453:110938, 2022.  ISSN 0021-9991.  doi: https://doi.org/10.1016/j.jcp.2021.110938.  URL https://www.sciencedirect.com/science/article/pii/S0021999121008330.
  * Yang et al. [2021] Liu Yang, Xuhui Meng, and George Em Karniadakis.  B-pinns: Bayesian physics-informed neural networks for forward and inverse pde problems with noisy data.  _Journal of Computational Physics_ , 425:109913, 2021.
  * Zhang et al. [2019] Dongkun Zhang, Lu Lu, Ling Guo, and George Em Karniadakis.  Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems.  _Journal of Computational Physics_ , 397:108850, 2019.  doi: 10.1016/j.jcp.2019.07.048.  URL https://doi.org/10.1016%2Fj.jcp.2019.07.048.
  * Zhao et al. [2020] Yaomin Zhao, Harshal D Akolekar, Jack Weatheritt, Vittorio Michelassi, and Richard D Sandberg.  Rans turbulence model development using cfd-driven machine learning.  _Journal of Computational Physics_ , 411:109413, 2020.


# Theoretical possibilities of head transplant

Santanu Acharjee
Department of Mathematics, Gauhati University, Assam, India
e-mail<EMAIL_ADDRESS>
## Abstract:

Recently, Zielinski and Sokal [ Zielinski, P., Sokal, P. (2016). Full spinal
cord regeneration after total transection is not possible due to entropy
change. Medical Hypotheses, 94, 63-65] proposed one hypothesis by considering
three assumptions against the spinal cord regeneration after total
transection. Thus, their claims are concluding that head transplant is not
possible. But, using theoretical justifications, we show that head transplant
is possible without any information loss. We design a spinal cord logic
circuit bridge (SCLCB), which is a reversible logic circuit and thus, we show
that there is no information loss.

2020 AMS Classifications: 92C50, 94C11.

Keywords: Spinal cord regeneration, entropy, laws of thermodynamics, head
transplant, logic gates, SCLGB.

##  1 Background

In 2013, Canavero [1] announced that full head (or body) transplant of a human
is possible. Since then, Canavero and Ren [2] have been facing criticisms.
Often, ideas of Canavero [1] or related procedural developments by Canavero
and Ren [3] are considered to be medically impossible tasks and unethical. One
may find many articles on medical impossibilities as well as ethical issues
related to head transplantation in human [4, 5, 6], but only a few groups are
supporting possibilities of head transplantation in human [7, 8]. In short,
Canavero’s HEAVEN (The head anastomosis venture project) with spinal linkage
(project GEMINI) [1] is a matter of discussion from various sides, in spite of
Ren and Canavero’s hope [7].

Amidst all the debates regarding possibilities and impossibilities related to
head transplantation in human, an article [9] attracted our attention. In
2016, Zielinski and Sokal [9] considered a hypothesis and proved that full
spinal cord regeneration after total transection is not possible. Their
hypothesis concludes that head transplant with cent percent success rate is
not possible. Their hypothesis is cited below.

“The hypothesis is that full spinal cord restoration after transection is not
possible due to irreversible loss of information about its structure, needed
to restore the connections.”

To prove their hypothesis, Zielinski and Sokal [9] considered following three
assumptions:

  * •

There are two million of axons in the pyramidal tract in the cervical region
and these axons are important for spinal cord regeneration and restoration of
adequate quality of life of a patient.

  * •

The second assumption is that the regeneration of damaged spinal cord should
lead to axonal growth through the lesion site from the proximal end of the
cord to the distal end and their re-connections with adequate target cells
with loss of distal parts of axons, below the level of transection and there
is an equal number of targets.

  * •

The axonal growth of the severed spinal cord is made fully possible

Zielinski and Sokal [9] provided some basic mathematical justifications (using
permutations) behind their claims, which cannot be neglected at first glance.
They also claimed that there is a lack of mathematical background in this area
of research and thus unnecessary expenditures of high research funds. But, in
next section, we provide mathematical justifications and prove that head
transplant with cent percent success rate is possible in theoretical sense.

##  2 Hypothesis

We consider the following hypothesis:

Head transplant is possible without any loss of information. Moreover, a logic
circuit (SCLCB) can be used to transmit nerve signals in case of transection
of spinal cord due to spinal cord injury.

##  3  Shannon entropy, uncertainty and information loss during head
transplant

In [9], Zielinski and Sokal considered that there are two millions pyramidal
axons in the spinal cord. They considered spinal cord as an open system whose
entropy would be lost after injury. After transection due to injury, the
proximal part and the distal part would be formed. Both of these parts would
have new entropy. Now, while reconnecting proximal part with distal part, if
one makes incorrect re-connections, the brain may have some problems to
reorganise, refining its own connectivity due to the brain plasticity [10].
Here, we develop related mathematics to show that head transplant is not
possible in case of maximum uncertainty. Let us rename two millions pyramidal
axons as $A_{1},A_{2},...,A_{2,000,000}$. If $p_{i}$ be the probability of
establishing correct interconnection of a pyramidal axons $A_{i}$ from
proximal part to its counter distal part, then $p_{i}=\frac{1}{2,000,000}$;
$\forall i=1,2,...,2,000,000$. In this case, if $H_{random}$ be the Shannon
entropy [11], then
$H_{random}=-\sum_{i=1}^{2,000,000}p_{i}log_{2}(p_{i})=log_{2}(2000000)$.
Thus, maximum uncertainty is obtained when one wants to establish
interconnections randomly. We consider an ideal hypothetical condition where
interconnection after total transection will be done as same as before total
transection. In this case, if $p_{i}^{\prime}$ be the probability of
establishing correct interconnection of a pyramidal axons $A_{i}$ from
proximal part to its counter distal part, then $p_{i}^{\prime}=1$; $\forall
i=1,2,...,2,000,000$. Let $H_{ideal}$ be the Shannon entropy [11] for this
ideal case, then
$H_{ideal}=-\sum_{i=1}^{2,000,000}p_{i}^{\prime}log_{2}(p_{i}^{\prime})=0$.
Thus, the entropy is zero i.e. there is no uncertainty only in case of ideal
hypothetical case. Zielinski and Sokal [9] assumed random re-connections and
thus obtained 2,000,000! permutations. But, why do we assume that
neurosurgeons will establish random interconnections of pyramidal axons? In
[13], Ren et al. discussed various medical methods along with experimental
evidences regarding possibilities of spinal cord fusion, axons regeneration,
etc. Moreover, they quoted the following paragraph from Busy et al. [14].

“The pyramidal tract is not essential to useful control of the skeletal
musculature. In the absence of the corticospinal fibers, other fiber systems,
particularly some multi‑neuronal mechanism passing through the mesencephalic
tegmentum, are capable of producing useful, well‑coordinated, strong, and
delicate movements of the extremities.”

Thus, the above paragraph encourages us to develop mathematical possibilities
with functionally equivalent mechanism. Let us subdivide 2,000,000 pyramidal
axons into some classes based on their functional equivalence and non-
functional equivalence mechanisms. Let $F_{1}$, $F_{2}$,…, $F_{K}$ be $k$
classes having functional equivalence mechanism and $F_{K+1}$ is a class of
pyramidal axons based on non-functional equivalence mechanism. We assume
$|F_{1}|+|F_{2}|+...+|F_{K}|=2,000,000-n$ and $|F_{K+1}|=n$, where
$n<<2,000,000$. Due to functional equivalence mechanism, Shannon entropy for
class $F_{i}$ (symbolically $H_{F_{i}}$) is zero, for $i=1,2,3,...,k$ and
$H_{F_{k+1}}$ = $log_{2}(n)$ due to non-functional equivalence. If
$H_{functional}$ be the entropy based on functional equivalence and non-
functional equivalence mechanisms, then due to Shannon [9], we obtain
$H_{functional}$=$\sum\limits_{i=1}^{k+1}H_{F_{i}}$=$log_{2}(n)$. Since
$n<<2,000,000$, thus $H_{ideal}\leq H_{functional}<H_{random}$. Thus, it can
be concluded mathematically that functions will be restored due to re-
connections as long as the mismatch is not extreme [9]. Since, entropy of
information theory is connected to the second law of thermodynamics [15], thus
we urged against the claim of Zielinski and Sokal [9]. Hence, we conclude that
spinal cord regeneration is theoretically possible if neurosurgeons identify
classes $F_{i}$s with functional equivalence mechanisms at most matching.

##  4 Spinal Cord Logic Circuit Bridge (SCLCB)

Figure 1:  Spinal Cord Logic Circuit Bridge (SCLCB) Figure 2: (a) Spinal cord
injured (SCI) and transected (b) SCLCB between transected spinal cord

Recently, Nie et al.[16] showed that nerve signal transmission after spinal
cord injury is possible in quantum realm and thus, they proposed spinal cord
quantum bridge (SCQB). This inspires us to built a logic circuit using only
three logical operators AND, OR, and NOT. Figure 1 shows a logic circuit which
takes input X and input Y in right and left side respectively. Again, it gives
output Y and output X in right and left side respectively. We call this logic
circuit as Spinal Cord Logic Circuit Bridge (SCLCB).

In (a) of figure 2, an injured spinal cord is shown, which is transected.
Here, X represents transmission of nerve signals in upward direction and Y
represents transmission of nerve signals in downward direction. More
elaborately, X represents transmission of sensory nerve signals from
peripheral nerves to central nerves and Y represents transmission of motor
nerve signals from central nerves to outer peripheral nerves. But, due to
transection of spinal cord, transmission of X and Y are stopped. In (b) of
figure 2, we implant SCLCB between two breakpoints Part A and Part B. While
implanting SCLCB, we are to connect X(input) and Y(output) in Part B and
X(output) and Y(input) in Part A of transected spinal cord. SCLCB has the
property that it takes both the signals X and Y simultaneously as inputs at
time $t_{1}$ and transmit signals X and Y as outputs at time $t_{2}$. For
example, if we consider X=1 and Y=1 as inputs at time $t_{1}$ i.e. signals
transmit in both directions at time $t_{1}$, then we can easily check that
SCLCB gives outputs X=1 and Y=1 at time $t_{2}$. One may check for other three
cases of inputs viz. (i) X=1 and Y=0 (ii) X=0 and Y=1, and (iii) X=0 and Y=0.
Thus, we propose that SCLCB can be implant between two parts of spinal cord
after transection due to spinal cord injury. Thus, we propose that
transmissions of nerve signals are possible in SCLCB. Since, both SCLCB and
SCQB [16] show the possibilities to transmit nerve signals during spinal cord
injury, thus we theoretically predict that head transplant is possible.

##  5 Information loss in digital circuit and SCLCB

Recently, Hänninen et al. [17] established methodologies to quantify
irreversible information loss in digital circuits. The property of mapping a
set of input bits onto a set of outputs in a logical operation is called
logical reversibility [17]. In a logically reversible computation, the inputs
can be known from the outputs. Physical reversibility occurs in an isolated
system. According to Hänninen et al. [17], a logically irreversible
computation can be obtained in an isolated system. One may refer to Keyes and
Landauer [20] for energy dissipation in logic gates. We cite the following
paragraph from Hänninen et al. [17].

“One bit can be in two states, so the associated entropy is $S={k_{\beta}}ln2$
Erasing an unknown bit, changing either 0 or 1 to a NULL state, means there is
a transfer of this entropy to the environment with associated free energy:

$\Delta E=TS={k_{\beta}}Tln2$.

Thus a physical implementation of a bit ensure or any logical operation that
loses 1 bit of information must necessarily dissipate an amount of heat
greater than or equal to $\Delta E$.”

In case of logical reversible computation, if the computation can be made
sufficiently slow, then dissipation of arbitrary small amount of energy is
occurred [21]. This claim is quite similar to the claim made by Ren et al.
[13] in favour of extremely sharp cut. Moreover, Bennett [21] suggested that
minimisation of the energy dissipation can be obtained near thermodynamic
equilibrium. Thus, our ideas of functionally equivalent mechanism and
$H_{functional}$ are important for neurosurgeons for head transplantation.
Bennett [21] showed that a logically irreversible computation can be made
logically reversible in each step. But, it is important to note that SCLCB is
reversible logical circuit because one can determine both the inputs from the
outputs [17]. Thus, there is no case of information loss. Hence, we can
conclude that there is no dissipation of energy in SCLCB [20, 21]. More
suitable chemical and electronic technologies viz. molecular logic gates [23],
electronic circuit design [22], etc. may be used to make SCLCB more effective.
Recently, clinical trails on partial restoration of spinal cord neural
continuity were done [18, 19]. Thus, we should remain optimistic about head
transplant. So, we can justify our hypothesis.

##  6 Conclusion

Zielinski and Sokal [9] predicted that full spinal cord regeneration after
total transection is not possible due to entropy change, and thus it concluded
indirectly that head transplant is impossible. To predict, they considered
three assumptions. In this paper, considering the same assumptions we proved
that head transplant is possible. We showed that functional equivalence
mechanisms yield less information loss. Moreover, we design the spinal cord
logic circuit bridge (SCLCB) as a technical measure for transmission of nerve
signals through spinal cords after transection due to spinal cord injury. Both
SCLCB and SCQB of [16] show the possibilities of head transplantation in
theoretical sense. Since, the procedure of head transplantation requires
sophistication in medical and engineering technologies, thus we assume that
the domain of head transplant attract attention of several interdisciplinary
fields.

Competing interests: The author has no conflict of interest.

Funding statement: No funding received.

Ethical approval/Ethical approval: Not required.

## References

  * [1] Canavero, S. (2013). HEAVEN: The head anastomosis venture project outline for the first human head transplantation with spinal linkage (GEMINI). Surgical Neurology International, 4(Suppl 1), S335.
  * [2] Ren, X., Canavero, S. (2017). HEAVEN in the making: Between the rock (the academe) and a hard case (a head transplant). AJOB Neuroscience, 8(4), 200-205.
  * [3] Canavero, S., Ren, X. (2016). Houston, GEMINI has landed: Spinal cord fusion achieved. Surgical neurology international, 7(Suppl 24), S626.
  * [4] Furr, A., Hardy, M. A., Barret, J. P., Barker, J. H. (2017). Surgical, ethical, and psychosocial considerations in human head transplantation. International Journal of Surgery, 41, 190-195.
  * [5] Wolpe, P. R. (2017). Ahead of our time: Why head transplantation is ethically unsupportable. AJOB Neuroscience, 8(4), 206-210.
  * [6] Suskin, Z. D., Giordano, J. J. (2018). Body–to-head transplant; a” caputal” crime? Examining the corpus of ethical and legal issues. Philosophy, Ethics, and Humanities in Medicine, 13(1), 1-6.
  * [7] Ren, X., Canavero, S. (2017). From hysteria to hope: The rise of head transplantation. International Journal of Surgery, 41(5), 203-4.
  * [8] Iamsakul, K., Pavlovcik, A. V., Calderon, J. I., Sanderson, L. M. (2017). PROJECT HEAVEN: Preoperative training in virtual reality. Surgical Neurology International, 8:59.
  * [9] Zielinski, P., Sokal, P. (2016). Full spinal cord regeneration after total transection is not possible due to entropy change. Medical Hypotheses, 94, 63-65.
  * [10] Lundborg, G. (2003). Nerve injury and repair–a challenge to the plastic brain. Journal of the Peripheral Nervous System, 8(4), 209-226.
  * [11] Shannon, C. E. (1948). A mathematical theory of communication. The Bell system technical journal, 27(3), 379-423.
  * [12] Rényi, A. (1961). On measures of entropy and information. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics (Vol. 4, pp. 547-562). University of California Press.
  * [13] Ren, X., Kim, C. Y., Canavero, S. (2019). Bridging the gap: spinal cord fusion as a treatment of chronic spinal cord injury. Surgical neurology international, 10.
  * [14] Bucy, P. C., Keplinger, J. E., Siqueira, E. B. (1964). Destruction of the “pyramidal tract” in man. Journal of neurosurgery, 21(5), 385-398.
  * [15] Duncan, T. L., Semura, J. S. (2007). Information loss as a foundational principle for the second law of thermodynamics. Foundations of Physics, 37(12), 1767-1773.
  * [16] Nie, M., Zhang, L., Yang, G. (2013). The bidirectional relay transmission of nerve signal after spinal cord injury based on quantum entanglement. In Proceedings of 2013 3rd International Conference on Computer Science and Network Technology, 1174-1177, IEEE.
  * [17] Haenninen, I. K., Lent, C. S., Snider, G. L. (2014). Quantifying irreversible information loss in digital circuits. ACM Journal on Emerging Technologies in Computing Systems, 11(2), 1-17.
  * [18] Ren, X., Zhang, W., Mo, J., Qin, J., Chen, Y., Han, J.,Feng, S., Liang, H., Ren, S. (2022). Partial Restoration of Spinal Cord Neural Continuity via Sural Nerve Transplantation Using a Technique of Spinal Cord Fusion. Frontiers in neuroscience, 16.
  * [19] Ren, X., Zhang, W., Qin, J., Mo, J., Chen, Y., Han, J., … Ren, S. (2022). Partial restoration of spinal cord neural continuity via vascular pedicle hemisected spinal cord transplantation using spinal cord fusion technique. CNS Neuroscience $\&$ Therapeutics.00,1–13.
  * [20] Keyes, R. W., Landauer, R. (1970). Minimal energy dissipation in logic. IBM Journal of Research and Development, 14(2), 152-157.
  * [21] Bennett, C. H. (1973). Logical reversibility of computation. IBM journal of Research and Development, 17(6), 525-532.
  * [22] Gift, S. J., Maundy, B. (2020). Electronic Circuit Design and Application. Springer Nature.
  * [23] Liu, L., Liu, P., Ga, L., Ai, J. (2021). Advances in Applications of Molecular Logic Gates. ACS omega, 6(45), 30189-30204.