# Initialization-enhanced physics-informed neural network with domain
decomposition (IDPINN)

Chenhao Si
School of Data Science
The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)
Shenzhen, 518172
<EMAIL_ADDRESS>
&Ming Yan*
School of Data Science
The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)
Shenzhen, 518172
<EMAIL_ADDRESS>

###### Abstract

We propose a new physics-informed neural network framework, IDPINN, based on
the enhancement of initialization and domain decomposition to improve
prediction accuracy. We train a PINN using a small dataset to obtain an
initial network structure, including the weighted matrix and bias, which
initializes the PINN for each subdomain. Moreover, we leverage the smoothness
condition on the interface to enhance the prediction performance. We
numerically evaluated it on several forward problems and demonstrated the
benefits of IDPINN in terms of accuracy.

##  1 Introduction

Physics-Informed Neural Network (PINN) has emerged as a promising approach to
tackle both forward and inverse problems associated with Partial Differential
Equations (PDEs) [1]. PINNs incorporate the governing equation into the neural
network’s architecture at its core, augmenting the loss function with a
residual term derived from the equation. This structural adjustment penalizes
deviations from the equation, narrowing the range of viable solutions.
Consequently, it transforms the process of determining PDE solutions into an
optimization task focused on minimizing the loss function.

PINNs have demonstrated adaptability and effectiveness across various domains
and interdisciplinary fields [2, 3, 4, 5]. Moreover, it terms out as a rapid
expansion of research that applies across various scientific fields, such as
fluid flows [6, 7, 8], physics modeling [9, 10], solid mechanics [11, 12],
etc. Some recent works also used variation structures of PINNs in solving
certain types of PDE [13], including wave modeling [14], multi-scale modeling
in thermal conductivity [15], and hydrodynamic simulations [16].

As practical applications proliferate, interest in understanding the
theoretical foundations of the PINN framework has surged. Researchers have
embarked on meticulous studies to explore the convergence properties and error
bounds of PINNs [17] to establish a robust theoretical foundation. These
investigations seek to unravel the intricate mechanisms that drive PINN
performance, anticipating that such understanding will contribute to
developing more efficient and accurate models.

However, several challenges remain to be solved in PINNs. Many aspects of
PINNs need to be improved, and these have attracted the attention of
researchers in the field. One of the main limitations of PINN is its
computationally intensive training process. Therefore, a key goal is
accelerating model training while enhancing accuracy and efficiency without
compromising the model’s capabilities.

Numerous adapted variants of PINNs have shown promising progress in mitigating
inaccuracies. For instance, Chakraborty et al. (2021) implemented a model of
shared knowledge within PINNs, incorporating multiple levels of detail to
enhance accuracy [18]. Additional research conducted by Desai et al. (2020),
Goswami et al. (2020), Tang et al. (2022), and Xue et al. (2022) also
demonstrated the successful application of shared knowledge in this context
[19, 20, 21, 22]. Furthermore, new constructions of PINNs have been developed,
such as PINNs with enhanced gradients (gPINNs) [23, 24, 25]. These gPINNs have
exhibited improved accuracy, particularly in training points for complex PDEs.

Notably, recent research, as exemplified by Lin et al. (2022) and Yang et al.
(2021), has combined PINNs with Bayesian knowledge, revealing the limitations
of the original structure of PINNs and the necessity of developing new
frameworks [26, 27]. Collectively, these works underscore the evolving
landscape of PINNs, where innovation and adaptation are key to overcoming
existing challenges and advancing the field.

To improve the accuracy and enhance the efficiency of solving both forward and
inverse problems, researchers have adopted the domain decomposition approach
[28]. Recent works showcase its effectiveness and practicality [14, 29, 30,
31]. This approach involves partitioning the entire domain into regular and
irregular subdomains. Therefore, we can train each subdomain using different
neural networks simultaneously and independently.

The benefits and advantages of domain decomposition can be summarized as
follows [28, 32, 33]:

1\. Distinct behaviors in varied domains: Systems show distinct behaviors in
different domains in many real-world scenarios. Therefore, the specific
physical characteristics may significantly diverge for different subdomains.
For instance, this phenomenon is often observed in problems involving
disruptive elements like shock waves.

2\. Simplified neural network structures: Segmenting a vast domain into
smaller subdomains helps reduce the complexity of the neural network structure
required for each subdomain. Domain decomposition strategy simplifies the
training process and makes it more efficient.

One significant challenge in domain decomposition for PINNs is that when some
subdomains lack the necessary boundary conditions or initial conditions, PINNs
(for the corresponding subdomains) will result in constant-zero predictions
for certain PDEs. In this case, the interface conditions will be used as the
boundary conditions to the corresponding subdomains enforced on the networks.
Therefore, the interface conditions in the loss function become crucial in
domain decomposition strategies.

One notable solution to this challenge is the conservative PINN (cPINN)
framework, proposed by Japtapel et al. (2020) [32]. This framework divides the
problem domain into subdomains and implements separate, localized high-
capacity networks in these regions. The interface condition is established
through flux continuity within the network. As a result, a set of individual
local PINNs is generated, and each is adapted to the specific prior knowledge
about the solution within its respective subdomain. The flux continuity
condition effectively governs the behavior of predictions.

Expanding upon the cPINNs concept, Japtapel et al. further introduced the
broadened PINNs (XPINNs) framework [33], which extends the previous approach
to address general PDEs. XPINNs modify the loss function associated with the
interface conditions. However, it should be noted that effectively managing
the loss at the interfaces between subdomains remains a challenging task [34].
Consequently, the research landscape has witnessed the emergence of an adapted
framework known as APINNs [35], which builds upon the foundations of both
PINNs and XPINNs. APINNs aim to address the challenges associated with
interface conditions in domain decomposition strategies.

Despite significant advancements, the persistent challenges of high training
costs and errors near the interface remain obstacles in domain decomposition
approaches for PINNs. These challenges impact the efficiency and accuracy of
the methodology, calling for innovative solutions. Recent studies conducted by
Chuang et al. (2023) [36], Ben et al. (2023) [37], and Alena et al. (2023)
[38] have shed light on the potential of domain decomposition to alleviate
training cost problems and effectively handle errors occurring at the
interface. However, these challenges persist, motivating researchers to
explore new methods for improvement.

We proposed a new framework called Initialization-enhanced Physics-Informed
Neural Network with Domain Decomposition (IDPINN) to address the above
challenges. We train a PINN using a smaller dataset and fewer training steps.
The trained model structures are then used to initialize the networks for each
subdomain, demonstrating greater efficiency compared to the standard PINN
model. Additionally, we adjust the loss functions and enforce a smoothness
condition at the interface, significantly reducing the interface error.

The contribution and novelty of this paper can be summarized as follows:

  1. •

Initialization enhancement: Instead of using a random initialization for our
network, we use the network construction from a previously trained PINN. This
initialization strategy proves to be highly effective in enhancing the
performance of the PINN, reducing both training costs and the relative error
significantly.

  2. •

Improvement for the domain decomposition: We present a novel model that
employs domain decomposition and new loss functions. Our empirical
investigations reveal significant enhancements when used for network training
with the domain decomposition approach.

The organization of this paper is as follows: In Section 2, we provide a brief
review of the PINN framework. The framework is introduced in Section 3, and
the experimental results are shown in Section 4. Specifically, we apply our
framework to the Helmholtz equation, 2D Poisson equation, heat equation. and
viscous Burgers equation We compare our results with those obtained using PINN
and XPINN.

##  2 Problem setup

###  2.1 Neural networks

We denote $\mathcal{NN}:\mathbb{R}^{n}\rightarrow\mathbb{R}^{m}$ as a feed-
forward neural network consisting of $L$ hidden layers, where its $l^{th}$
hidden layer has $N_{l}$ neurons ($1\leq l\leq L$). In particular, the input
and output layers have $n$ and $m$ neurons, respectively. The weight matrix
and bias vector in the $l^{th}$ hidden layer are denoted as $\mathbf{W}^{l}$
and $\mathbf{b}^{l}$, respectively. Let $h^{l}$ be the nonlinear activation
function in the $l^{th}$ hidden layer, then the output vector of the $l^{th}$
hidden layer $\mathbf{z}^{l}$ is obtained by:

$\displaystyle\mathbf{z}^{l}=h^{l}(\mathbf{W}^{l}\mathbf{z}^{l-1}+\mathbf{b}^{l}).$
(1)

The input vector of the neural network is its input layer $\mathbf{z}^{0}$.
The output of the neural network is

$\displaystyle
u_{\theta}(\mathbf{z}^{0})=\mathcal{NN}(\mathbf{z}^{0};\theta),$ (2)

where $\theta=\\{\mathbf{W}^{l},\mathbf{b}^{l}\\}_{l=1}^{L}$ is the collection
set of all weight matrix and bias vectors of the neural network. For
convenience, we may write $\mathcal{NN}(\mathbf{z})$ instead of
$\mathcal{NN}(\mathbf{z};\theta)$ in the remaining sections when there is no
confusion.

###  2.2 Partial differential equations

A partial differential equation (PDE) with its solution $u(\mathbf{x},t)$
takes the general form as follows:

$\displaystyle\mathcal{F}(u,\mathbf{x},t)$
$\displaystyle=0,\quad\qquad\mathbf{x}\in\Omega\subseteq\mathbb{R}^{n},~{}t\in[0,T]$
(3) $\displaystyle u(\mathbf{x},t)$
$\displaystyle=f(\mathbf{x},t),\quad\mathbf{x}\in\partial\Omega,~{}t\in[0,T]$
(4) $\displaystyle u(\mathbf{x},0)$ $\displaystyle=g(\mathbf{x}),$ (5)

where $\Omega$ denotes the spatial domain with its boundary $\partial\Omega$.
We also denote $\overline{\Omega}=\Omega\cup\partial\Omega$. Besides,
$\mathbf{x}$ stands for the spatial coordinates such that
$\mathbf{x}=(x_{1},x_{2},x_{3},\ldots,x_{n})\in\mathbb{R}^{n}$, $t$ stands for
the time coordinate, and $\mathcal{F}$ is the differential operator. Equations
(4) and (5) ensure the boundary and initial conditions for the solution $u$,
respectively, where $f(\mathbf{x},t)$ and $g(\mathbf{x})$ are functions on the
boundary with time and the spatial domain without time, respectively. Here is
an example of a PDE system, which is so-called the 1D viscous Burgers
equation:

$\displaystyle u_{t}+uu_{x}=\frac{0.01}{\pi}u_{xx},\quad
x\in[-1,1],~{}t\in[0,1],$ (6) $\displaystyle u(-1,t)=u(1,t)=0,$ (7)
$\displaystyle u(x,0)=-\sin(\pi x),~{}\quad x\in(-1,1).$ (8)

The differential operator is
$\mathcal{F}(u,\mathbf{x},t)=u_{t}+uu_{x}-\frac{0.01}{\pi}u_{xx}$ with spatial
domain $\Omega=[-1,1]$ and its boundary $\partial\Omega=\\{-1,1\\}$.

###  2.3 Physics-informed neural networks (PINNs)

The fundamental concept of PINN is to utilize a neural network
$\mathcal{NN}(\mathbf{x},t;\theta)$ with parameters $\theta$ to represent the
unidentified solution $u(\mathbf{x},t)$. Let
$\\{(\mathbf{x}_{k}^{R},t_{k}^{R})\\}_{k=1}^{N_{R}}$ be the set of $N_{R}$
training residual points in the domain $\Omega\times(0,T)$,
$\\{(\mathbf{x}_{k}^{B},t_{k}^{B})\\}_{k=1}^{N_{B}}$ be the set of $N_{B}$
boundary points on the boundary $\partial\Omega\times(0,T)$, and
$\\{(\mathbf{x}_{k}^{I},0)\\}_{k=1}^{N_{I}}$ be the set of $N_{I}$ initial
points. The predicted solution from the neural network is denoted as
$\hat{u}(\mathbf{x},t)$. We construct the loss function with PDE residues, the
boundary conditions, and initial conditions as:

$\displaystyle\mathcal{L}_{R}(\theta)$
$\displaystyle=\frac{1}{N_{R}}\sum_{k=1}^{N_{R}}|\mathcal{F}(\hat{u},\mathbf{x}_{k}^{R},t_{k}^{R})|^{2},$
(9) $\displaystyle\mathcal{L}_{B}(\theta)$
$\displaystyle=\frac{1}{N_{B}}\sum_{k=1}^{N_{B}}|\hat{u}(\mathbf{x}_{k}^{B},t_{k}^{B})-f(\mathbf{x}_{k}^{B},t_{k}^{B})|^{2},$
(10) $\displaystyle\mathcal{L}_{I}(\theta)$
$\displaystyle=\frac{1}{N_{I}}\sum_{k=1}^{N_{I}}|\hat{u}(\mathbf{x}_{k}^{I},0)-g(\mathbf{x}_{k}^{I})|^{2}.$
(11)

Here $\mathcal{L}_{R}(\theta)$, $\mathcal{L}_{B}(\theta)$, and
$\mathcal{L}_{I}(\theta)$ are called the residue loss of the PDE, boundary
loss, and initial loss, respectively. The loss function for the standard PINN
is defined as:

$\displaystyle\mathcal{L(\theta)}=\lambda_{1}\mathcal{L}_{R}(\theta)+\lambda_{2}\mathcal{L}_{B}(\theta)+\lambda_{3}\mathcal{L}_{I}(\theta),$
(12)

where $\lambda_{1}$, $\lambda_{2}$ and $\lambda_{3}$ are weights corresponding
to the three losses.

##  3 Our proposed framework for domain decomposition

We briefly introduce domain decomposition and the motivation for a new
framework in Section 3.1. Section 3.2 introduces a new framework named
Initialization-enhanced Physics-Informed Neural Network with Domain
Decomposition (IDPINN), whose overall structure is shown in Fig. 2. By
decomposing the entire domain into several disjoint subdomains, we build an
individual neural network on each subdomain and train the neural networks
together. As shown in Fig. 2, the loss function comprises six parts, whose
details will be explained later.

###  3.1 Domain decomposition: issues and motivation

The main idea of domain decomposition is dividing a domain into several
subdomains, and functions on different subdomains are represented by different
functions. These subdomains could be disjoint or overlapped. In this paper, we
focus on the disjoint case. Using domain decomposition in PINNs can utilize
parallel computing to tackle larger-scale problems. Moreover, learning
functions on a relatively small domain enhances the accuracy and stability of
PINN [28, 32, 33].

Suppose that the entire domain $\Omega$ is decomposed into $M$ non-overlapping
subdomains $\\{\Omega_{m}\\}_{m=1}^{M}$. We denote the interface between two
adjacent subdomains $\Omega_{i}$ and $\Omega_{j}$ as $\partial\Omega_{ij}$
(here we let $i<j$ for the uniqueness). We denote
$\\{(\mathbf{x}^{i}_{k},t^{i}_{k})\\}_{k=1}^{N_{f_{i}}}$ as the set of data
points within the subdomain $i$, i.e.,
$(\mathbf{x}_{k}^{i},t_{k}^{i})\in\Omega_{i}\times(0,T)$. Moreover, to impose
the interface condition, we denote
$\\{(\mathbf{x}^{ij}_{k},t^{ij}_{k})\\}_{k=1}^{N_{ij}}$ as the set of $N_{ij}$
data points randomly generated on the interface $\partial\Omega_{ij}$. The
total data points generated are denoted as $\\{(\mathbf{x}^{*},t^{*})\\}$.

It is not surprising to use the difference of
$\hat{u}_{i}(\mathbf{x}^{ij}_{k},t^{ij}_{k})$ and
$\hat{u}_{j}(\mathbf{x}^{ij}_{k},t^{ij}_{k})$ as an additional penalty term,
where $\hat{u}_{i}(\mathbf{x}^{ij}_{k},t^{ij}_{k})$ and
$\hat{u}_{j}(\mathbf{x}^{ij}_{k},t^{ij}_{k})$ stand for the predictions on the
interface $\partial\Omega_{ij}$ from the PINNs to the $\Omega_{i}$ and
$\Omega_{j}$, respectively. This penalty term is as follows:

$\displaystyle\mathcal{L}_{inter}(\theta)=$
$\displaystyle\sum_{ij}\frac{1}{N_{ij}}\sum_{k=1}^{N_{ij}}|\hat{u}_{i}(\mathbf{x}^{ij}_{k},t^{ij}_{k})-\hat{u}_{j}(\mathbf{x}^{ij}_{k},t^{ij}_{k})|^{2}.$
(13)

Here, we add the loss functions for all interfaces together. The notation
$\sum_{ij}$ means that the summation is over all interfaces. There are other
types of loss functions to enforce the consistency between two adjacent
subdomains. For instance, the remarkable decomposition model XPINN [33] uses

$\displaystyle\mathcal{L}_{inter}(\theta)=\lambda_{residual}\mathcal{L}_{residual}(\theta)+\lambda_{avg}\mathcal{L}_{avg}(\theta),$

where

$\displaystyle\mathcal{L}_{residual}(\theta)=$
$\displaystyle\sum_{ij}\frac{1}{N_{ij}}\sum_{k=1}^{N_{ij}}|\mathcal{F}(\hat{u}_{i},\mathbf{x}^{ij}_{k},t^{ij}_{k})-\mathcal{F}(\hat{u}_{j},\mathbf{x}^{ij}_{k},t^{ij}_{k})|^{2},$
(14) $\displaystyle\mathcal{L}_{avg}(\theta)=$
$\displaystyle\sum_{ij}\frac{1}{N_{ij}}\sum_{k=1}^{N_{ij}}\left|\hat{u}_{i}(\mathbf{x}^{ij}_{k},t^{ij}_{k})-\frac{\hat{u}_{i}(\mathbf{x}^{ij}_{k},t^{ij}_{k})+\hat{u}_{j}(\mathbf{x}^{ij}_{k},t^{ij}_{k})}{2}\right|^{2}.$
(15)

In the original XPINN [33], the authors use $\theta_{i}$ as the parameters of
the PINN for the subdomain $i$ and construct the loss function subdomain-wise
assuming that the parameters for other PINNs are known. Here, we denote
$\theta$ as the parameters for all PINNs in the domain for convenience. The
loss functions introduced above are equivalent to the functions in XPINN. Note
that the term $\mathcal{L}_{avg}(\theta)=\mathcal{L}_{inter}(\theta)/4$, where
$\mathcal{L}_{inter}(\theta)$ is defined in equation (13).

We apply XPINN on the Helmholtz equation by dividing the domain
$\Omega=[-1,1]\times[-1,1]$ into two subdomains with the line $x=0$ being the
interface (more details on this experiment can be found in Section 4.1) and
plot some results in the first two rows of Fig. 1. In the first row, we plot
the output function of XPINN for four different $y$ values. We can see the
changes in the slop near the interface between both subdomains. This figure
shows that only enforcing consistency in the function value may cause a non-
smooth connection across the interface. To be fair enough, we test it under
several choices of the weights $\lambda$ and plot one of the best results.

Figure 1: Slices of the exact (blue) and predicted (red) solution ($u(x,y)$)
of the Helmholtz equation for given $y$ values, and their corresponding PDE
residuals ($f(x,y,u)$). First and second row: We use the loss functions on the
interface: $\mathcal{L}_{residual}(\theta)$ (14) and
$\mathcal{L}_{avg}(\theta)$ (15). Third and fourth row: We use IDPINN-3.
($\mathcal{L}_{\nabla}(\theta)$ (17) and $\mathcal{L}_{PDE_{g}}(\theta)$
(18)).

Also, we plot the residual functions of PDE
$f(x,y,u)=\mathcal{F}(u,\mathbf{x})$ for the same $y$ values in the second row
of Fig. 1. Though the true residuals are zero in the whole domain, the
residuals from XPINN are not smooth around the interface. This observation
shows that the consistency in the function and residual across the interface
may not be sufficient, which motivates us to use high-order continuity.

###  3.2 Our proposed IDPINN

Motivated by the above experiment, which shows that XPINN introduces non-
smoothness across the interface, we propose to penalize the difference between
the gradients of the function and the residual at the interface to ensure
smoothness. We introduce the new loss function as follows:

$\displaystyle\mathcal{L}(\theta)=\lambda_{1}\mathcal{L}_{r}(\theta)+\lambda_{2}\mathcal{L}_{b}(\theta)+\lambda_{3}\mathcal{L}_{I}(\theta)+\lambda_{4}\mathcal{L}_{inter}(\theta)+\lambda_{5}\mathcal{L}_{\nabla}(\theta)+\lambda_{6}\mathcal{L}_{PDE_{g}}(\theta),$
(16)

where $\mathcal{L}_{inter}(\theta)$ is defined in equation (13), and

$\displaystyle\mathcal{L}_{\nabla}(\theta)=$
$\displaystyle\sum_{ij}\frac{1}{N_{ij}}\sum_{k=1}^{N_{ij}}\|\nabla\hat{u}_{i}(\mathbf{x}_{k}^{ij},t_{k}^{ij})-\nabla\hat{u}_{j}(\mathbf{x}_{k}^{ij},t_{k}^{ij})\|^{2}_{2},$
(17) $\displaystyle\mathcal{L}_{PDE_{g}}(\theta)=$
$\displaystyle\sum_{ij}\frac{1}{N_{ij}}\sum_{k=1}^{N_{ij}}\|\nabla\mathcal{F}(\hat{u}_{i},\mathbf{x}_{k}^{ij},t_{k}^{ij})-\nabla\mathcal{F}(\hat{u}_{j},\mathbf{x}_{k}^{ij},t_{k}^{ij})\|_{2}^{2}.$
(18)

The loss function $\mathcal{L}_{inter}(\theta)$ ensures continuity across the
interface, and we add the smoothness term $\mathcal{L}_{\nabla}(\theta)$ at
the interface, which plays a crucial role in our model. However, for the
residuals, we replace the consistency term at the surface (14) with the
consistency in its gradient (17). We do not keep the consistency term (14)
since we already have the residual loss term (9) within each subdomain.

Besides the changes in the loss function, we introduce an initialization
stage, referred to as the IDPINN-init stage, to reduce the overall training
time. During this stage, We randomly select a subset of data points
$\\{(\mathbf{x}^{init},t^{init})\\}\subset\\{(\mathbf{x}^{*},t^{*})\\}$, which
includes both boundary and initial data points, and train a single PINN using
this small dataset for a few iterations. The trained parameters are denoted as
$\theta^{0}$. Then, we use $\theta^{0}$ as the initial parameters for PINNs
across all subdomains during the IDPINN-main stage.

Figure 2: Structure of IDPINN, where each subdomain uses a neural network to
approximate the function on the subdomain, and these neural networks are
integrated through interface conditions. The PINN to the subdomains
$\Omega_{i}$ and $\Omega_{j}$ have different parameters $\theta_{i}$ and
$\theta_{j}$, respectively. Here, due to the simplicity, we use $\theta$.

Fig. 2 shows the overall structure of our model. For simplicity, we use
$\theta_{k}$ to represent the PINN parameters on subdomain $k$, and $\theta$
encompasses all parameters.

We apply IDPINN to the same Helmholtz equation problem discussed in the
previous subsection, shown in the third and fourth rows of Fig. 1. In the
third row, the red curve (prediction) closely aligns with the entire blue
curve (exact), exhibiting no discernible discontinuity or non-smoothness, even
near the interface. In the fourth row, the influence of adding
$\mathcal{L}_{PDE_{g}}(\theta)$ is further depicted. The curve exhibits
overall smoother behavior, with no discontinuous shock occurring at the
interface. The residual functions are also closer to zero compared with the
second row. This improvement is attributed to the smoothness at the interface,
thereby enabling each subdomain’s PINN to benefit from a more stable interface
condition.

##  4 Numerical experiments

There are six terms in IDPINN: the first three terms come from PINN (PDE
residual, boundary condition, and initial condition); the fourth and fifth
terms are used to promote continuity and smoothness of the function across
interfaces, respectively; the last term promotes the smoothness of the
function and the PDE residual across interface using higher-order information.
We keep the first four terms in the following experiments and demonstrate the
benefit of adding the smoothness terms (17) and (18). We denote IDPINN-1 as
the model with the smoothness term (17) only, IDPINN-2 as the model with the
smoothness term (18) only, and IDPINN-3 as the model with both (17) and (18).
The differences of these three models are summarized in Table 1.

Table 1: Loss term information for IDPINN-1, IDPINN-2, and IDPINN-3. Model | IDPINN-1 | IDPINN-2 | IDPINN-3
---|---|---|---
Smoothness term $\mathcal{L}_{\nabla}(\theta)$ | ✓ | ✗ | ✓
Smoothness term $\mathcal{L}_{PDE_{g}}(\theta)$ | ✗ | ✓ | ✓

We use the following relative L2 error (referred to as L2 error) to measure
the accuracy of the models:

$\displaystyle\text{L2-error}=\frac{\sqrt{\sum_{k=1}^{N}|\hat{u}(\mathbf{x}_{k},t_{k})-u(\mathbf{x}_{k},t_{k})|^{2}}}{\sqrt{\sum_{k=1}^{N}|u(\mathbf{x}_{k},t_{k})|^{2}}},$
(19)

where $\hat{u}$ and $u$ are predicted and exact solutions, respectively. The
set of points $\\{(\mathbf{x}_{k},t_{k})\\}_{k=1}^{N}$ used to compute the L2
error will be explained in the subsequent experiments.

###  4.1 Helmholtz equation

The Helmholtz partial differential equation (PDE) is frequently utilized to
describe wave and diffusion phenomena. It is used to depict the development of
wave propagation and diffusion processes within spatial or spatiotemporal
domains. In this investigation, we focus on a particular Helmholtz PDE that
exists exclusively within the spatial (x, y) domain by:

$\displaystyle u_{xx}+u_{yy}+u-q(x,y)=0,\quad x\in[-1,1],y\in[-1,1],$ (20)
$\displaystyle u(-1,y)=u(1,y)=0,$ (21) $\displaystyle u(x,-1)=u(x,1)=0,$ (22)

where

$\displaystyle q(x,y)$ $\displaystyle=-\pi^{2}\sin(\pi x)\sin(4\pi
y)-(4\pi)^{2}\sin(\pi x)\sin(4\pi y)+\sin(\pi x)\sin(4\pi y),$ (23)

with an exact solution

$\displaystyle u(x,y)=\sin(\pi x)\sin(4\pi y).$ (24)

We partition the domain along the line $x=0$ and evaluate the performance of
IDPINN-3 and XPINN using a 5-layer network with 55 neurons per layer. We
generate a uniform grid of size $300\times 300$ across the domain
$[-1,1]\times[-1,1]$ to calculate the L2 error. Additionally, we uniformly
sample 5000 points along the interface $x=0$. We select training points from
the total of 95000 generated points as follows: Initially, 200 boundary points
and 3000 residual points are randomly chosen from the grid for each subdomain.
Subsequently, 200 points are randomly selected from the interface $x=0$.

The weights for the loss function in IDPINN-3 are specified as follows:
$\lambda_{1}=1$, $\lambda_{2}=10$, $\lambda_{3}=0$, $\lambda_{4}=20$,
$\lambda_{5}=2$, and $\lambda_{6}=5$. Among the 6400 points selected on the
grid, we chose 80 boundary points and 500 residual points during the IDPINN-
init stage. Subsequently, the model is trained using these 6600 points for
98,000 iterations via the Adam optimizer, employing a learning rate of
$8\times 10^{-5}$. The resulting prediction L2 error amounts to $1.1\times
10^{-2}$. Please refer to Table 2 and Fig. 3 for detailed arrangement
information.

Additionally, we explore various weight configurations for XPINN, where
$\lambda_{1}=1$, $\lambda_{2}=20$, $\lambda_{3}=0$, $\lambda_{residual}=20$,
and $\lambda_{avg}=80$ yield a comparatively lower L2 error ($1.41\times
10^{-1}$). These configurations are utilized to produce Fig. 1. Fig. 4
presents the visualization of the predictions. In our initial experiments, we
adopted the same learning rate as that utilized for IDPINN-3. However, we
observed that the L2 error plateaued after approximately 20k iterations.
Consequently, we increased the learning rate to $1\times 10^{-2}$, which
resulted in enhanced performance. Fig. 5 illustrates the L2 error history of
these models under both a lower learning rate ($8\times 10^{-5}$) and a higher
learning rate ($1\times 10^{-2}$).

Figure 3: Point selection for the Helmholtz equation with two subdomains. The selected points in each subdomain are represented by red (Subdomain 1) and blue (Subdomain 2), respectively. The boundary points in two subdomains and interface points are represented by green, orange, and purple, respectively. Table 2: Number of points and iterations in each stage (Helmholtz equation). The points for the IDPINN-init stage are collected from the training set of IDPINN-main. stage | region | residual | boundary | initial | interface | iterations
---|---|---|---|---|---|---
IDPINN-init | all-region | 500 | 80 | NA | NA | 2000
IDPINN-main | sub-domain 1 (red) | 3000 | 200 | 200 | 98000
sub-domain 2 (blue) | 3000 | 200

Figure 4: First row: exact solution of the Helmholtz equation. Second row:
predicted solution and point-wise error for XPINN after 100k total iterations,
resulting in a $1.41\times 10^{-1}$ L2 error. Third row: predicted solution
and point-wise error for IDPINN-3 after 100k total iterations, resulting in a
$1.1\times 10^{-2}$ L2 error. Last row: predicted solution and point-wise
error for IDPINN-3 without initialization after 100k total iterations,
resulting in a $4.82\times 10^{-2}$ L2 error.

####  4.1.1 Effect of the Initialization

In this section, we discuss the importance of initialization. With the same
setting as above, we train IDPINN-3 without initialization for $100k$
iterations and compare it to IDPINN-3 with initialization.

The last row in Fig. 4 shows the performance of the IDPINN without the
initialization. The overall performance is better than XPINN while there are
still a small error occurring around the interface, comparing with IDPINN-3.
Further, the L2 error history for IDPINN-3 without initialization, using a
lower learning rate ($8\times 10^{-5}$), is presented in Fig. 5 (purple
curve). We realized that the L2 error decreased slowly in the initial
iterations. Consequently, we increase the learning rate to $1\times 10^{-2}$
(yellow curve). Notably, IDPINN-3 outperforms other models under both low
($8\times 10^{-5}$) and high ($1\times 10^{-2}$) learning rates. Furthermore,
the performance of IDPINN-3 remains relatively better and more stable at a
lower learning rate.

Figure 5: The L2 error history in 100,000 iterations for the Helmholtz
equation for XPINN, IDPINN-3, and IDPINN-3 without initialization, under two
different learning rates.

Figure 6: First: Loss history for IDPINN without initialization. Second: Loss
history for IDPINN-3 after the initialization. Third: Loss history of the
IDPINN-init stage.

We also compare the training loss history over two subdomains for IDPINN-3
without and with initialization under the higher learning rate ($1\times
10^{-2}$) in Fig. 6. With initialization, the initial training loss drops from
around $1\times 10^{4}$ to $1\times 10^{3}$, and the training loss can
decrease to below $1\times 10^{-1}$. However, without initialization, the
training loss struggles to reach around 20. This highlights the importance of
initialization. Understanding its importance prompts the question: how many
iterations are sufficient for the initialization stage? Too many iterations
may prolong the training time unnecessarily, while too few iterations may not
adequately prepare the model. Achieving the right balance is crucial for
efficient training. The next subsection will examine this.

####  4.1.2 Different number of iterations in the IDPINN-init stage

The previous subsection demonstrates the importance of initialization, while
this subsection focuses on determining the number of iterations in the IDPINN-
init stage. Since the IDPINN-init stage initializes the parameter for the
IDPINN-main part, finding a good initialization is crucial. We vary the number
of iterations in the IDPINN-init stage and compare their performance in the
IDPINN-main part in Fig. 7. This figure shows that 100 iterations do not
produce a satisfactory initialization, and when the number of iterations
exceeds 500, the performance becomes very similar. This suggests that we do
not need too many iterations in the initialization stage. Therefore, we only
require a certain number of iterations in the IDPINN-init stage to achieve
satisfactory performance.

Figure 7: The L2 error history for IDPINN with different numbers of iterations
in the IDPINN-init stage. Only 100 iterations (green curve) do not produce a
satisfactory initialization; when the number of iterations exceeds 500, the
performance becomes very similar.

In addition, we examine the solution constructed by initialization and its
point-wise error in Fig. 8. After 100 iterations, the solution remains very
close to zero. However, after 500 iterations, discernible patterns begin to
emerge. Comparing the solutions for 500 and 2000 iterations, we observe that
the range of the function values becomes closer to the true range $[-1,1]$ as
the number of iterations increases. The larger function value also justifies
why the initialization with 500 iterations has the highest initial L2 error in
Fig. 7.

This and the preceding subsections demonstrate that a certain number of
iterations in the initialization stage is adequate to enhance the performance
of IDPINN. Without precise knowledge of the exact number of iterations
required, opting for a relatively large number is advisable, considering that
the initialization step is fast.

Figure 8: The prediction and the point-wise error for the IDPINN-init, when
the number of training steps is different. (100 iterations for the first row,
500 for the second, and 2000 for the last row.)

###  4.2 2D Poisson Equation

This experiment uses IDPINN-3, which includes smoothness terms (17) and (18),
to solve the 2D Poisson equation under the same domain setup as [33]:

$\displaystyle\Delta u=f(x,y)\quad(x,y)\in\Omega\subset\mathbb{R}^{2}.$

The boundary is
$\partial\Omega:=\big{\\{}(0.02+r(\theta)\cos\theta,r(\theta)\sin\theta):\theta\in[0,2\pi)\big{\\}}$,
where

$\displaystyle
r(\theta)=1.5+0.14\sin(4\theta)+0.12\cos(6\theta)+0.09\cos(5\theta).$

The two interface curves are defined by:
$\Gamma_{1}:=\big{\\{}(-0.4+r_{1}(\theta)\cos\theta,-0.4+r_{1}(\theta)\sin\theta):\theta\in[0,2\pi)\big{\\}}$
and
$\Gamma_{2}:=\big{\\{}(0.5+r_{2}(\theta)\cos\theta,0.6+r_{2}(\theta)\sin\theta):\theta\in[0,2\pi)\big{\\}}$,
where

$\displaystyle r_{1}(\theta)=$ $\displaystyle
0.50+0.18\sin(3\theta)+0.08\cos(2\theta)+0.2\cos(5\theta),$ $\displaystyle
r_{2}(\theta)=$ $\displaystyle
0.34+0.04\sin(5\theta)+0.18\cos(3\theta)+0.1\cos(6\theta).$

Assume the true solution is $u(x,y)=e^{x}+e^{y}$, and we have
$f(x,y)=e^{x}+e^{y}$.

The data points shared with [33] consist of 18211, 2855, and 1291 residual
points in subdomains 1, 2, and 3, respectively. Additionally, there are 6284
points each for both interfaces and the boundary. All these data points are
used to compute the L2 error. The training points, randomly collected from
these points, are depicted in Fig. 9, whose summary information is in Table 3.
In summary, the IDPINN-main stage uses 1300 residual points, 100 interface
points, and 100 boundary points, from which the points in the IDPINN-init
stage are selected.

Figure 9: Training points for the 2D Poisson equation. There are three subdomains. The training points in each subdomain are represented by blue (subdomain 1), red (subdomain 2), and green (subdomain 3). Points on the interface and boundary are represented by orange and black, respectively. Table 3: Number of training points for the 2D Poisson equation. The training points in the IDPINN-init stage are chosen from those in the IDPINN-main stage. Stage | Region | Residual | Interface | Boundary | Initial | Iterations
---|---|---|---|---|---|---
IDPINN-init | all-region | 800 | NA | 50 | NA | 500
IDPINN-main XPINN | sub-domain 1 (blue) | 1000 | 100 (50 per interface) | 100 | 3500
sub-domain 2 (red) | 180 | 0
sub-domain 3 (green) | 120

For XPINN, we initially adopt the layer structure as defined in [33], which is
[2, 20, 20, 20, 20, 1] for sub-domain 1, [2, 25, 25, 25, 1] for sub-domain 2,
and [2, 30, 30, 1] for sub-domain 3. However, utilizing different networks for
distinct sub-domains poses challenges in initialization for IDPINN. Thus, we
conduct further experiments with XPINN by employing uniform layer
configurations for all sub-domains, set as [2, 20, 20, 20, 20, 1]. For
simplicity, we denote this model as XPINN-U20. The weighted parameters for the
loss functions in both models are set as $\lambda_{1}=1$, $\lambda_{2}=20$,
$\lambda_{3}=0$, $\lambda_{residual}=\lambda_{avg}=20$. For IDPINN, the
network structure is defined as [2, 20, 20, 20, 20, 1]. The weighted
parameters assigned to the loss terms are $\lambda_{1}=1$, $\lambda_{2}=20$,
$\lambda_{3}=0$, $\lambda_{4}=2$, $\lambda_{5}=5$, and $\lambda_{6}=5$. The
network structures are summarized in Table 4.

Table 4: Network structures of IDPINN and XPINN for the 2D Poisson equation. Model | Region | Network structure
---|---|---
IDPINNs | all sub-domains | [2, 20, 20, 20, 20, 1]
XPINN-U20 | all sub-domains | [2, 20, 20, 20, 20, 1]
XPINN | sub-domain 1 (blue) | [2, 20, 20, 20, 20, 1]
sub-domain 2 (red) | [2, 25, 25, 25, 1]
sub-domain 3 (green) | [2, 30, 30, 1]

The performance of different methods is shown in Fig. 10. Notably, IDPINN
demonstrates better performance than XPINN, particularly at the interface. The
second row shows the benefit of incorporating both smoothness terms.
Specifically, IDPINN-3 with both terms performances better than the other two
models with only one smooth term. In the next subsection, we will compare the
performance with different $\lambda_{6}$ values.

A (Exact solution)

B (L2 error: $2.47\times 10^{-3}$)

C (L2 error: $3.53\times 10^{-3}$)

D (L2 error: $1.07\times 10^{-3}$)

E (L2 error: $9.72\times 10^{-4}$)

F (L2 error: $6.48\times 10^{-4}$)

Figure 10: Exact solution of the 2D Poisson equation and the point-wise
relative error for different methods after 4k XPINN or 4k IDPINN-main
iterations. A: Exact solution. B and C: Relative error of XPINN and XPINN-U20.
D-F: Relative error of IDPINN-1, IDPINN-2, and IDPINN-3, respectively. The L2
errors are shown in the captions as well.

####  4.2.1 Effect of choosing different $\lambda_{6}$

In Fig. 10, we set $\lambda_{6}=2$ for IDPINN-3, and while the model IDPINN-1
corresponds to $\lambda_{6}=0$. We observed that IDPINN-3 performs better at
the interface. Therefore, in this subsection, we test the performance at the
interfaces with different $\lambda_{6}$ values. We choose
$\lambda_{6}\in\\{0.01,0.1,1,2,20\\}$ while fixing the other parameters. Fig.
11 shows the L2 error history specifically at the interfaces. When
$\lambda_{6}$ is set to 1 (red) or 2 (blue), the overall performance of the
history curve is better than smaller or larger $\lambda_{6}$ values.

Figure 11: The L2 error history of the interfaces for different $\lambda_{6}$
values in IDPINN-3.

####  4.2.2 Effect of the layer structure

The layer structure significantly influences the neural network’s performance.
In this section, we conduct additional experiments using various layer
architectures (different numbers of layers and number of neurons in the
layer). The L2 error results are summarized in Table 5. For each structure, we
train the model five times with different random seeds and select the best-
performing result. While there is no obvious pattern in the results, we
observe that IDPINN-3 performs better when the width of the network is set to
40 or 60. Additionally, increasing the number of hidden layers does not
necessarily lead to improved prediction performance.

Table 5: The L2 error of different layer structures on the 2D Poisson equation. neurons layers | 2 | 3 | 4 | 6 | 8
---|---|---|---|---|---
20 | $0.8778\times 10^{-3}$ | $0.7858\times 10^{-3}$ | $0.6824\times 10^{-3}$ | $1.7923\times 10^{-3}$ | $1.1480\times 10^{-3}$
40 | $1.2391\times 10^{-3}$ | $0.9278\times 10^{-3}$ | $0.3857\times 10^{-3}$ | $0.8020\times 10^{-3}$ | $0.7896\times 10^{-3}$
60 | $0.4433\times 10^{-3}$ | $0.4403\times 10^{-3}$ | $0.7442\times 10^{-3}$ | $1.6240\times 10^{-3}$ | $1.3766\times 10^{-3}$
80 | $1.0639\times 10^{-3}$ | $1.0544\times 10^{-3}$ | $1.4418\times 10^{-3}$ | $1.1049\times 10^{-3}$ | $4.1409\times 10^{-3}$

###  4.3 Heat equation

This subsection explores the heat equation, where XPINN performed worse than
PINN [34]. We consider the same heat equation:

$\displaystyle u_{t}-u_{xx}$ $\displaystyle=0,\quad x\in[-1,1],~{}t\in[0,1],$
(25)

with the boundary and initial conditions

$\displaystyle u(1,t)$
$\displaystyle=-e^{-\pi^{2}t}+0.6e^{-4\pi^{2}t}+0.3e^{4t-4}\cosh(2)+0.1e^{t-1}\sinh(1),$
(26) $\displaystyle u(-1,t)$
$\displaystyle=-e^{-\pi^{2}t}+0.6e^{-4\pi^{2}t}+0.3e^{4t-4}\cosh(-2)+0.1e^{t-1}\sinh(-1),$
(27) $\displaystyle u(x,0)$ $\displaystyle=\cos(\pi x)+0.6\cos(2\pi
x)+0.3e^{-4}\cosh(2x)+0.1e^{-1}\sinh(x).$ (28)

The exact solution for the heat equation is

$\displaystyle u(x,t)=e^{-\pi^{2}t}\cos{\pi x}+0.6e^{-4\pi^{2}t}\cos{2\pi
x}+0.3e^{4t-4}\cosh{2x}+0.3e^{t-1}\sinh{x}.$ (29)

Here, we follow a similar domain decomposition as the Ref. [34], where the
interface is chosen to be $t=0.5$. The region for $t\leq 0.5$ will be denoted
as subdomain 1, and the rest will be subdomain 2 in our experiment.

Table 6 and Fig. 12 give information about the training data points. We
generate a $300\times 300$ grid uniformly over the domain $[-1,1]\times[0,1]$
to compute the L2 error. Additionally, 5000 points are uniformly generated on
the interface $t=0.5$. We select training points from the total of 95000
generated points as follows: initially, 100 boundary points and 2000 residual
points are randomly chosen for each subdomain. Moreover, 200 initial points
are randomly selected in subdomain 1. Within the 4400 selected points, we
further select 50 boundary points, 50 initial points, and 1000 residual points
in the IDPINN-init stage. For the IDPINN-main stage, 200 points are randomly
chosen on the interface. Subsequently, the model is trained using these 4600
points for $14$k iterations by the Adam optimizer with a learning rate of
$5.5\times 10^{-6}$.

For the numerical experiment, we use a 9-layer neural network with 20 neurons
per layer. The weights for IDPINN-3 are set as $\lambda_{1}=2$,
$\lambda_{2}=10$, $\lambda_{3}=20$, $\lambda_{4}=20$, $\lambda_{5}=5$, and
$\lambda_{6}=5$. The XPINN is configured with
$\lambda_{residual}=\lambda_{avg}=20$, while other parameters remain
consistent with IDPINN-3. Also, we train a PINN using 4600 training points
within the defined grid, employing the same network structure and learning
rate. Visualizations of PINN, XPINN, and IDPINN-3 predictions are shown in
Fig. 13.

Table 6: Number of points and iterations in each stage (Heat equation). The training data in the IDPINN-init stage are selected from the training points in the IDPINN-main stage. stage | region | residual | boundary | initial | interface | iterations
---|---|---|---|---|---|---
IDPINN-init | all-region | 1000 | 50 | 50 | NA | 1000
IDPINN-Main | sub-domain 1 (red) | 2000 | 100 | 200 | 200 | 14000
sub-domain 2 (blue) | 2000 | 100 | NA

Figure 12: Exact solution and the data selection of the Heat equation.

Figure 13: From up to down, left to right: Predicted solution and point-wise
error of IDPINN-3 (L2 error: $3.51\times 10^{-3}$), PINN (L2 error:
$6.22\times 10^{-3}$), and XPINN (L2 error: $9.30\times 10^{-3}$), to the Heat
equation.

As shown in Fig. 13, XPINN exhibits significant point-wise errors near the
interface and boundary. After $15$k iterations, XPINN yields a relative L2
error of $9.30\times 10^{-3}$, while PINN achieves a slightly lower error of
$6.22\times 10^{-3}$. The experiments with different settings in [34] also
show that PINN outperforms XPINN for the heat equation, even with an increased
number of training points. However, IDPINN-3 achieves an even smaller L2 error
of $3.51\times 10^{-3}$ compared to PINN, showcasing its superiority in
accuracy. Moreover, IDPINN-3 exhibits reduced point-wise errors near the
interface and boundary, demonstrating its effectiveness in handling boundary-
dominated problems.

###  4.4 Viscous Burgers equation

The viscous Burgers equation, expressed as (6)-(7), serves as a fundamental
PDE in fluid dynamics and nonlinear physics. It provides a simplified model
for a wide range of complex phenomena, including turbulence and shock waves.
Combining the influences of convection and diffusion, the equation
encapsulates the nonlinear advection and viscous dissipation characteristics
of fluid behavior.

We consider the whole domain as $(x,t)\in[-1,1]\times[0,1]$ and divide it with
a circle of radius $0.3$ centered at $(0.5,0.5)$. The region outside the
circle is designated as subdomain 1, while the region inside is labeled as
subdomain 2. For testing, we utilize the same dataset as [1], which provides
ground truth for the testing points.

Table 7 provides details about the training data points. A $300\times 300$
grid is uniformly generated over the domain $[-1,1]\times[0,1]$. Additionally,
5000 points are uniformly generated on the circle
$\\{(x,t):(x-0.5)^{2}+(t-0.5)^{2}=0.3^{2}\\}$. Training points are selected
from the 95000 generated points. Initially, 120 boundary points and 2450
residual points are randomly chosen for subdomain 1, along with 450 residual
points for subdomain 2. Furthermore, 60 initial points are randomly selected
in subdomain 1, along with 100 points on the interface.

Table 7: Number of points and iterations in each stage (Burgers equation). The training data in the IDPINN-init stage are collected from the training sample in the IDPINN-main stage. stage | region | residual | interface | boundary | initial | iterations
---|---|---|---|---|---|---
IDPINN-init | all-region | 1000 | NA | 60 | 30 | 1000
IDPINN-Main | sub-domain (inside) | 450 | 100 | 0 | 0 | 14000
sub-domain (outside) | 2450 | 120 | 60
PINN | all-region | 3000 | NA | 120 | 60 | 15000

We compare IDPINN-1, IDPINN-3, and PINN for this experiment. The network
architecture has 7 hidden layers with 20 neurons in each hidden layer. The
parameters for IDPINN-1 are set as $\lambda_{1}=1$, $\lambda_{2}=20$,
$\lambda_{3}=5$, $\lambda_{4}=20$, $\lambda_{5}=2$ and $\lambda_{6}=0$.
Letting $\lambda_{6}=2$ yields IDPINN-3. In addition, we use the same
parameters ($\lambda_{1}$, $\lambda_{2}$, and $\lambda_{3}$) for PINN. We
train the models using the Adam optimizer with a learning rate of $7\times
10^{-4}$. The numerical results are shown in Fig. 14.

The improvement of IDPINN over PINN in this experiment may not be substantial,
and the majority of the error occurs at $x=0$, where the shock wave appears.
Since IDPINN assumes smoothness across the interface, we avoid placing the
interface over the shock. For such problems, adaptive sampling strategies may
be beneficial [39][40][41]. These strategies dynamically adjust the sampling
density based on the solution behavior, potentially improving accuracy and
efficiency in regions of interest.

Figure 14: The numerical results for the Burgers equation. Top row: the exact
solution. Second row: the predicted solution and point-wise error for PINN (L2
error: $4.03\times 10^{-2}$). Third row: the predicted solution and point-wise
error for IDPINN-1 (L2 error: $2.92\times 10^{-2}$). Last row: the predicted
solution and point-wise error for IDPINN-3 (L2 error: $3.35\times 10^{-2}$).

##  5 Conclusion

This paper introduces IDPINN, a novel framework that combines advanced
initialization and domain decomposition techniques to enhance physics-informed
neural networks (PINNs). By formulating special loss functions for interface
conditions, IDPINN effectively recovers smooth solutions, while its
initialization from a small dataset enhances training efficiency, saving time
and computing resources. Experimental results demonstrate the superiority of
IDPINN over XPINN and PINN. Additionally, we explore IDPINN’s applicability to
various domain decomposition strategies, including straight lines and
irregular closed curves. It demonstrates the versatility of our approach,
making it applicable across a spectrum of domain decomposition scenarios.

## Acknowledgments

This work was partially supported by the Guangdong Provincial Key Laboratory
of Mathematical Foundations for Artificial Intelligence (2023B1212010001),
Shenzhen Stability Science Program, and the Shenzhen Science and Technology
Program under grant no. ZDSYS20211021111415025.

## References

  * [1] M. Raissi, P. Perdikaris, and G.E. Karniadakis.  Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.  J. Comput. Phys., 378:686–707, 2019.
  * [2] K. Shukla, P.C. Di Leoni, J. Blackshire, D. Sparkman, and G.E. Karniadakis.  Physics-informed neural network for ultrasound nondestructive quantification of surface breaking cracks.  JJ. Nondestruct. Eval., 39(3):1–20, 2020.
  * [3] M. Yin, X. Zheng, J.D. Humphrey, and G.E. Karniadakis.  Non-invasive inference of thrombus material properties with physics-informed neural networks.  Comput. Methods Appl. Mech. Eng., 375:113603, 2021.
  * [4] S. Cai, Z. Wang, P. Perdikaris, and G.E. Karniadakis.  Physics-informed neural networks for heat transfer problems.  J. Heat Transf, 143, 2021.
  * [5] A. Henkes, H. Wessels, and R. Mahnken.  Physics informed neural networks for continuum micromechanics.  Comput. Methods Appl. Mech. Eng., 393:114790, 2022.
  * [6] Maziar Raissi, Alireza Yazdani, and George Em Karniadakis.  Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations.  Science, 367(6481):1026–1030, 2020.
  * [7] Alvin Wei Ze Chew, Renfei He, and Limao Zhang.  Modelling clogging dynamics in groundwater systems using multiscale homogenized physics informed neural network (mhpinn).  Engineering Science and Technology, an International Journal, 49:101605, 2024.
  * [8] Chengping Rao, Hao Sun, and Yang Liu.  Physics-informed deep learning for incompressible laminar flows.  Theoretical and Applied Mechanics Letters, 10(3):207–212, 2020.
  * [9] Shengze Cai, Zhicheng Wang, Lu Lu, Tamer A Zaki, and George Em Karniadakis.  Deepm&mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks.  Journal of Computational Physics, 436:110296, 2021.
  * [10] Zhiping Mao, Lu Lu, Olaf Marxen, Tamer A Zaki, and George Em Karniadakis.  Deepm&mnet for hypersonics: Predicting the coupled flow and finite-rate chemistry behind a normal shock using neural-network approximation of operators.  Journal of computational physics, 447:110698, 2021.
  * [11] Ruiyang Zhang, Yang Liu, and Hao Sun.  Physics-informed multi-lstm networks for metamodeling of nonlinear structures.  Computer Methods in Applied Mechanics and Engineering, 369:113226, 2020.
  * [12] Chun-Teh Chen and Grace X Gu.  Learning hidden elasticity with deep neural networks.  Proceedings of the National Academy of Sciences, 118(31):e2102721118, 2021.
  * [13] Yifan Wang and Linlin Zhong.  Nas-pinn: neural architecture search-guided physics-informed neural network for solving pdes.  Journal of Computational Physics, 496:112603, 2024.
  * [14] Pu Ren, Chengping Rao, Su Chen, Jian-Xun Wang, Hao Sun, and Yang Liu.  Seismicnet: Physics-informed neural networks for seismic wave modeling in semi-infinite domain.  Computer Physics Communications, 295:109010, 2024.
  * [15] Bokai Liu, Yizheng Wang, Timon Rabczuk, Thomas Olofsson, and Weizhuo Lu.  Multi-scale modeling in thermal conductivity of polyurethane incorporated with phase change materials using physics-informed neural networks.  Renewable Energy, 220:119565, 2024.
  * [16] Zhenyu Dai, Ben Moews, Ricardo Vilalta, and Romeel Davé.  Physics-informed neural networks in the recreation of hydrodynamic simulations from dark matter.  Monthly Notices of the Royal Astronomical Society, 527(2):3381–3394, 2024.
  * [17] M. Siddhartha and M. Roberto.  Estimates on the generalization error of physics-informed neural networks for approximating pdes.  IMA J. Numer. Anal., 43:1–43, 2023.
  * [18] S. Chakraborty.  Transfer learning based multi-fidelity physics informed deep neural network.  J. Comput. Phys., 426:109942, 2021.
  * [19] S. Desai, M. Mattheakis, H. Joy, P. Protopapas, and S. Roberts.  One-shot transfer learning of physics-informed neural networks.  arXiv preprint arXiv:2110.11286, 2022.
  * [20] S. Goswami, C. Anitescu, S. Chakraborty, and T. Rabczuk.  Transfer learning enhanced physics informed neural network for phase-field modeling of fracture.  Theor. Appl. Fract. Mech., 106:102447, 2020.
  * [21] H. Tang, Y. Liao, H. Yang, and L. Xie.  A transfer learning-physics informed neural network (tl-pinn) for vortex-induced vibration.  Ocean Eng., 266:113101, 2022.
  * [22] C. Xu, B.T. Cao, Y. Yuan, and G. Meschke.  Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios.  Comput. Methods Appl. Mech. Eng., 405:115852, 2023.
  * [23] J. Yu, L. Lu, X. Meng, and G. E. Karniadakis.  Gradient-enhanced physics-informed neural networks for forward and inverse pde problems.  Comput. Methods Appl. Mech. Eng., 393:114823, 2022.
  * [24] M. Mohammadian, K. Baker, and F. Fioretto.  Gradient-enhanced physics-informed neural networks for power systems operational support.  Electr. Power Syst. Res., 223:109551, 2023.
  * [25] S. Lin and Y. Chen.  Gradient-enhanced physics-informed neural networks based on transfer learning for inverse problems of the variable coefficient differential equations.  arXiv preprint arXiv:2305.08310, 2023.
  * [26] K. Linka, A. Schafer, X. Meng, Z. Zou, G. E. Karniadakis, and E. Kuhl.  Bayesian physics informed neural networks for real-world nonlinear dynamical systems.  Comput. Methods Appl. Mech. Eng., 402:115346, 2022.
  * [27] L. Yang, X. Meng, and G.E. Karniadakis.  B-pinns: Bayesian physics-informed neural networks for forward and inverse pde problems with noisy data.  J. Comput. Phys., 425:109913, 2021.
  * [28] E. Kharazmi, Z. Zhang, and G.E. Karniadakis.  hp-vpinns: Variational physics-informed neural networks with domain decomposition.  arXiv, 2003.05385, 2020.
  * [29] Jianchuan Yang, Xuanqi Liu, Yu Diao, Xi Chen, and Haikuo Hu.  Adaptive task decomposition physics-informed neural networks.  Computer Methods in Applied Mechanics and Engineering, 418:116561, 2024.
  * [30] Mario De Florio, Enrico Schiassi, Francesco Calabrò, and Roberto Furfaro.  Physics-informed neural networks for 2nd order odes with sharp gradients.  Journal of Computational and Applied Mathematics, 436:115396, 2024.
  * [31] Benrong Zhang, Fajie Wang, and Lin Qiu.  Multi-domain physics-informed neural networks for solving transient heat conduction problems in multilayer materials.  Journal of Applied Physics, 133(24), 2023.
  * [32] A.D. Japtap and G.E. Karniadakis.  Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problem.  Comput. Methods Appl. Mech. Eng., 365:113028, 2020.
  * [33] A.D. Japtap and G.E. Karniadakis.  Extended physics-informed neural networks (xpinns) : A generalized spacetime domain decomposition based deep learning framework for nonlinear partial differential equations.  Commun. Comput. Phys., 28:2002–2041, 2020.
  * [34] Z. Hu, A.D. Japtap, G.E. Karniadakis, and K. Kawaguchi.  When do extended physics-informed neural networks(xpinns) improve generalization?  SIAM Journal on Scientific Computing, 44(5):A3158–A3182, 2022.
  * [35] Z. Hu, A. D. Jagtap, G.E. Karniadakis, and K. Kawaguchi.  Augmented physics-informed neural networks (apinns): A gating network-based soft domain decomposition methodology.  arXiv preprint arXiv:2211.08939, 2022.
  * [36] L. Chuang and W. Hengan.  cv-pinn: Efficient learning of variational physics-informed neural network with domain decomposition.  Extreme Mech. Lett., 63:102051, 2023.
  * [37] B. Moseley, A. Markham, and T. Nissen-Meyer.  Finite basis physics-informed neural networks (fbpinns): a scalable domain decomposition approach for solving differential equations.  Adv. Comput. Math, 49:62, 2023.
  * [38] A. Kopaničáková, H. Kothari, G. E. Karniadakis, and R. Krause.  Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies.  arXiv preprint arXiv:2306.17648, 2023.
  * [39] W. Chenxi, Z. Min, T. Qinyang, K. Yadhu, and L. Lu.  A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks.  Comput. Methods Appl. Mech. Eng., 403:115671, 2023.
  * [40] A. Daw, J. Bu, S. Wang, P. Perdikaris, and A. Karpatne.  Rethinking the importance of sampling in physics-informed neural networks.  arXiv:2207.02338, 2022.
  * [41] Z. Gao, L. Yan, and T. Zhou.  Failure-informed adaptive sampling for pinns.  SIAM Journal on Scientific Computing, 45(4):A1971–A1994, 2023.