# Determining conductivity and embedded obstacles from partial boundary
measurements

Jiaqing Yang School of Mathematics and Statistics, Xi’an Jiaotong University,
Xi’an, Shaanxi, 710049, China<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

In this paper, we consider an inverse conductivity problem on a bounded domain
$\Omega\subset\mathbb{R}^{n}$, $n\geq 2$, also known as Electrical Impedance
Tomography (EIT), for the case where unknown impenetrable obstacles are
embedded into $\Omega$. We show that a piecewise-constant conductivity
function and embedded obstacles can be simultaneously recovered in terms of
the local Dirichlet-to-Neumann map defined on an arbitrary small open subset
of the boundary of the domain $\Omega$. The method depends on the well-
posedness of a coupled PDE-system constructed for the conductivity equations
in the $H^{1}$-space and some elementary a priori estimates for Harmonic
functions.

Keywords: Inverse conductivity problem, Dirichlet-to-Neumann map, partial
data, embedded obstacle.

##  1 Introduction

This paper is concerned with an inverse boundary value problem for the
conductivity equation. We aim to recover the electrical conductivity of a
bounded body in ${\mathbb{R}}^{n}$, $n\geq 2$, as well as the possibly
embedded obstacles by taking local boundary measurements. Precisely, let
$\Omega$ and $D$ be two bounded domains in ${\mathbb{R}}^{n}$ with Lipschitz
continuous boundaries such that $\overline{D}\subseteq\Omega$. In the physical
situation, $D$ corresponds to the impenetrable obstacle inside $\Omega$ which
consists of possibly several physical components satisfying different boundary
conditions. Then the conductivity problem can be formulated in finding a weak
solution $u\in H^{1}(\Omega\setminus\overline{D})$ such that

$\displaystyle\left\\{\begin{array}[]{ll}{\rm div\,}(\gamma\nabla
u)=0,&\text{in}\;\Omega\setminus\overline{D}\\\
\;u=f,&\text{on}\;\partial\Omega\\\ \;\mathcal{B}u=0,&\text{on}\;\partial
D\end{array}\right.$ (1.4)

for any given voltage potential $f\in H^{1/2}(\partial\Omega)$. In (1.4), the
function $\gamma\in L^{\infty}(\Omega\setminus\overline{D})$ denotes the
electric conductivity satisfying the ellipticity condition
$0<c_{0}\leq\gamma\leq c_{0}^{-1}$ a.e. in the domain
$\Omega\setminus\overline{D}$ for some positive constant
$c_{0}\in{\mathbb{R}}_{+}$. The operator $\mathcal{B}$ indicates the boundary
condition, described by $\mathcal{B}u:=u$ if $D$ is a sound-soft obstacle and
$\mathcal{B}u:=\partial_{\nu}u+{\rm i}\lambda u$ with ${\rm Re}(\lambda)\geq
0$ if $D$ is an imperfect obstacle. Here, $\nu$ denotes the exterior unit
normal on the boundary $\partial D$. The impedance boundary condition can be
reduced to a Neumann boundary condition if $\lambda=0$.

Under the conditions, (1.4) can be proved to be well-posed in
$H^{1}(\Omega\setminus\overline{D})$. Therefore, we can define the well-
defined Dirichlet-to-Neumann (D-N) map by

$\displaystyle\Lambda_{\gamma,D,\mathcal{B}}f=\gamma\frac{\partial
u}{\partial\nu}\Big{|}_{\partial\Omega}\in H^{-1/2}(\partial\Omega).$ (1.5)

If measurements are taken only on an open subset of $\partial\Omega$, denoted
by $\Gamma$, then the relevant map in (1.5) is called the local D-N map. In
this case, we write the map for $\Lambda^{\Gamma}_{\gamma,D,\mathcal{B}}$
which indicates the dependence on the measurement boundary $\Gamma$. Then, the
inverse conductivity problem is to determine all unknown $\gamma,D$ and
$\mathcal{B}$ by taking full measurements $\Lambda_{\gamma,D,\mathcal{B}}$ or
partial measurements $\Lambda^{\Gamma}_{\gamma,D,\mathcal{B}}$. This kind of
problems can find important applications in diverse fields such as geophysical
exploration, nondestructive testing and medical imaging. The first
mathematical formulation on the inverse problem can date back to A. P.
Calderón [9], where the uniqueness was considered in recovering $\gamma$ by
the knowledge of $\Lambda_{\gamma,D,\mathcal{B}}$ for the case when no
obstacles are embedded in the domain $\Omega$, i.e., $D=\emptyset$.
Substantial progress has been made in this direction since then; see
fundamental papers like [18], [19], [20], [22] in both the two-dimensional and
three-dimensional cases. It was shown in [22] that the global uniqueness
theorem was proved in determining a sufficiently smooth conductivity or
potential for dimension $n\geq 3$, by taking measurements over the boundary.
This result was later extended in [5], [6], [12], [21] and related references
quoted there to the cases of less regularity conductivity or potential.
Moreover, in dimension $n=2$, the global uniqueness theorem was also proved in
[4], [20] for a smooth conductivity and [2] for the relaxed regularity
conductivity, by taking full boundary data. Compared to the full data case,
the inverse conductivity problem by the local D-N map has been dealt with more
recently. We refer to [1] [3], [14], [15], [16] and related references quoted
there in both three dimensions and two dimensions.

The works mentioned above are mainly based on constructing suitable complex
geometrical optics solutions associated with the conductivity equation or
Schrödinger equation. With a different version, a local uniqueness result was
proved [1] on the conductivity, by assuming $\gamma$ is a piecewise-constant
function in a known finite partition of the domain; see also [8] for the
direct geophysical setting of the problem.

We remark that in all above works except [15], the inverse conductivity
problems were dealt with only for the case without embedded obstacles in the
body. In the current work, we shall study the inverse problem with
impenetrable obstacles embedded in $\Omega$, which makes the problem become
more challenging since unknown embedded obstacles need to be also recovered by
full or partial measurements over the boundary. If $D$ is known to be a sound-
soft obstacle in advance, it has been shown in [15] that the conductivity
$\gamma$ can be uniquely recovered in the domain $\Omega\setminus\overline{D}$
by assuming the partial data. We also mention a related work [23, 24] from
Yang et al. in the field of the inverse scattering, in which the unique
recovery of embedded obstacles and the surrounding inhomogeneous medium
described by a piecewise-constant refractive index were considered. A novel
version was proposed in [23, 24] relying on the $L^{2}$-well-posedness of the
interior transmission problem or its modified version for the Helmholtz
equations in a sufficiently small domain, which were recently extended to
other cases such as the inverse fluid-solid interaction problem [11] and the
inverse cavity problem [10]. In this paper, motivated by the previous works of
the author, we shall study the inverse conductivity problem by a priori
assuming that the conductivity is a piecewise-constant function in an unknown
finite partition of the domain. Under this assumption, we are able to prove
that the local D-N map defined on an arbitrary open subset of the boundary can
uniquely determine all unknown $\gamma$, $D$ and $\mathcal{B}$. The approach
proposed in this work depends only on the well-posedness of a coupled PDE-
system associated with the conductivity equations in the $H^{1}$ space, which
is essentially simple and elementary. Moreover, our approach always works
whenever in the two dimensional case or a more higher dimensional case.

Next let us outline the underlying main results of this work. We first present
some assumptions pertaining to the domain partition. Assume that the bounded
domain $\Omega$ consists of a finite number of disjoint Lipschitz subdomains
$\Omega_{\ell}$, $\ell=1,2,\cdots,N$, such that
$\overline{\Omega}=\bigcup_{1\leq\ell\leq N}\overline{\Omega_{\ell}}$. And the
electrical conductivity $\gamma$ is considered to be constant in
$\Omega_{\ell}$, $\ell=1,2,\cdots,N$ satisfying a natural condition that
$c_{\ell_{1}}\not=c_{\ell_{2}}$ if the interaction
$\partial\Omega_{\ell_{1}}\cap\partial\Omega_{\ell_{2}}$ contains a non-empty
open subset in the ($n-1$)-dimensional manifold of ${\mathbb{R}}^{n}$.
Moreover, we also assume that a smooth obstacle $D$ is embedded inside some
subdomain denoted by $\Omega_{\ell_{0}}$ $(1\leq\ell_{0}\leq N)$, i.e.,
$\overline{D}\subset\Omega_{\ell_{0}}$. That is, the material surrounding the
obstacle $D$ is assumed to be homogeneous. Under the above considerations, the
conductivity $\gamma$ can be described as the following function

$\displaystyle\gamma(x)=\sum_{\ell=1}^{N}c_{\ell}\cdot\chi_{\Omega_{\ell}}(x),\quad{\rm
for\;almost\;every\;}x\in\Omega\setminus\overline{D},$ (1.6)

where $c_{\ell}$, $\ell=1,2,\cdots,N$, are unknown real positive constants,
and $\chi_{\Omega_{\ell}}$ denotes the characteristic function in the
subdomain $\Omega_{\ell}$, which is defined by $1$ in $\Omega_{\ell}$ and $0$
otherwise. For the non-empty open subset $\Gamma$ of the boundary
$\partial\Omega$, we define the following partial Cauchy data

$\displaystyle\mathcal{C}_{\gamma,D,\mathcal{B}}:=\\{(f|_{\Gamma},(\Lambda_{\gamma,D,\mathcal{B}}f)|_{\Gamma}):f\in
H^{\frac{1}{2}}(\partial\Omega)\;{\rm with}\;f=0\;{\rm
on}\;\partial\Omega\setminus\overline{\Gamma}\\}$ (1.7)

which will be used to recover internal properties of $\Omega$ in the inverse
conductivity problem. For convenience, we use symbols
$\mathcal{C}_{\gamma_{j},D_{j},\mathcal{B}_{j}}$, $j=1,2$, indicating the
dependence on two classes of different conductivities and embedded obstacles.
Based on (1.6)-(1.7), we now formulate our main result in the following
theorem.

######  Theorem 1.1.

If
$\mathcal{C}_{\gamma_{1},D_{1},\mathcal{B}_{1}}=\mathcal{C}_{\gamma_{2},D_{2},\mathcal{B}_{2}}$,
then $\gamma_{1}=\gamma_{2}$, $D_{1}=D_{2}$ and
$\mathcal{B}_{1}=\mathcal{B}_{2}$.

The remaining part of the paper is organized as follows. In Sections 2 we
introduce an auxiliary boundary value problem associated with the modified
conductivity equation and prove its well-posedness in a suitable Sobolev space
by employing the variational method. In Section 3 we propose a novel technique
for the inverse conductivity problem, also known as Calderón problem, of
determining the conductivity of a piecewise-constant function from Cauchy data
measured on an arbitrary open subset of the boundary.

##  2 A related PDE-model

In this section, we introduce a related PDE-system in $D_{0}$ described by two
modified conductivity equations which are coupled by a pair of Cauchy
conditions on the boundary $\partial D_{0}$:

$\displaystyle\left\\{\begin{array}[]{ll}{\rm div\,}(a_{1}(x)\nabla
u_{1})-b_{1}u_{1}=\rho_{1},&\text{in}\;D_{0},\\\ {\rm div\,}(a_{2}(x)\nabla
u_{2})-b_{2}u_{2}=\rho_{2},&\text{in}\;D_{0},\\\
u_{1}-u_{2}=f_{1},&\text{on}\;\partial D_{0},\\\
a_{1}(x)\partial_{\nu}u_{1}-a_{2}(x)\partial_{\nu}u_{2}=f_{2},&\text{on}\;\partial
D_{0},\\\ \end{array}\right.$ (2.5)

where $D_{0}$ is a simply connected, bounded domain with the Lipschitz
boundary $\partial D_{0}$, and $\rho_{1},\rho_{2}\in L^{2}(D_{0})$, $f_{1}\in
H^{\frac{1}{2}}(\partial D_{0})$ and $f_{2}\in H^{-\frac{1}{2}}(\partial
D_{0})$. In (2.5), $a_{j}$, $j=1,2$, are assumed to two bounded measured
functions satisfying $\infty>c_{2}\geq a_{j}\geq c_{1}>0$ for almost every
$x\in D_{0}$, and $b_{j}$, $j=1,2$, are also assumed to be two positive
constants.

Under some appropriate assumptions on $a_{j},b_{j}$, we shall next prove that
(2.5) is well-posed in the classical $H^{1}(D_{0})$ space for the given data
$\rho_{1},\rho_{2},f_{1}$ and $f_{2}$. We remark that the well-posedness of
(2.5) will play an important role in the inverse problem of determining a
piecewise-constant conductivity $\gamma$. Indeed, as shown in the proof of
Theorem 1.1 in next section, the regularity of the solutions of (1.4) with a
family of special boundary data $f$ having a singularity like $1/|x-z|$ in the
three-dimensional case or $1/\ln|x-z|$ in the two-dimensional case can be
exactly improved uniformly by analyzing the solvability of (2.5). This will
lead to a contradiction. The detailed discussions can be found in next
section.

In order to study the existence of a solution to (2.5), we shall adopt the
suggestion from [7] with employing a technical variational approach for the
coupled Helmholtz equations. To this end, we introdue the Hilbert space
$X_{a_{2}}$ defined by

$\displaystyle X_{a_{2}}:=\\{\bm{v}\in L^{2}(D_{0})^{3}:{\rm
div\,}(a_{2}(x)\bm{v})\in L^{2}(D_{0}),\;{\rm
curl\,}\bm{v}=0\;\;\text{in\;}D_{0}\\}$ (2.6)

under the product
$(\bm{v}_{1},\bm{v}_{2})_{X_{a_{2}}}:=(\bm{v}_{1},\bm{v}_{2})_{L^{2}(D_{0})^{3}}+({\rm
div\,}(a_{2}\bm{v}_{1}),{\rm div\,}(a_{2}\bm{v}_{2}))_{L^{2}(D_{0})}$ for
$\bm{v}_{1},\bm{v}_{2}\in X_{a_{2}}$. Obviously, it makes sense due to the
assumption on $a_{2}$. The norm on $X_{a_{2}}$ is thus given by

$\displaystyle\|\bm{v}\|^{2}_{X_{a_{2}}}:=\|\bm{v}\|^{2}_{L^{2}(D_{0})^{2}}+\|{\rm
div\,}(a_{2}\bm{v})\|^{2}_{L^{2}(D_{0})}.$

If $u_{2}\in H^{1}(D_{0})$ is the solution of the second equation in (2.5), we
let $\bm{u}_{2}:=\nabla u_{2}$ in $D_{0}$. Then, one has $\bm{u}_{2}\in
L^{2}(D_{0})^{3}$ satisfying

$\displaystyle{\rm div\,}(a_{2}\bm{u}_{2})=b_{2}u_{2}+\rho_{2}\in
L^{2}(D_{0})\quad\text{and}\quad{\rm curl\,}\bm{u}_{2}=0\;\;\text{in\;}D_{0},$
(2.7)

which yields that $\bm{u}_{2}\in X_{a_{2}}$. This fact motivates us to find
the solution $(u_{1},\bm{u}_{2})$ of (2.5) with the second equation replaced
by (2.7) on a new product space $H^{1}(D_{0})\times X_{a_{2}}(D_{0})$. By the
Green’s theorems, it first follows that

$\displaystyle\int_{D_{0}}{\rm div\,}(a_{2}\bm{u}_{2})\cdot{\rm
div\,}(a_{2}\bm{v}){\rm d}x=\int_{D_{0}}(b_{2}u_{2}+\rho_{2}){\rm
div\,}(a_{2}\bm{v}){\rm d}x$
$\displaystyle\qquad\qquad\qquad\quad=\int_{\partial
D_{0}}a_{2}b_{2}(\nu\cdot\bm{v})u_{2}{\rm
d}s(x)-\int_{D_{0}}a_{2}b_{2}(\bm{v}\cdot\bm{u}_{2}){\rm
d}x+\int_{D_{0}}\rho_{2}{\rm div\,}(a_{2}\bm{v}){\rm d}x$
$\displaystyle\qquad\qquad\qquad\quad=\int_{\partial
D_{0}}a_{2}b_{2}(\nu\cdot\bm{v})(u_{1}-f_{1}){\rm
d}s(x)-\int_{D_{0}}a_{2}b_{2}(\bm{v}\cdot\bm{u}_{2}){\rm
d}x+\int_{D_{0}}\rho_{2}{\rm div\,}(a_{2}\bm{v}){\rm d}x$

and

$\displaystyle\int_{D_{0}}({\rm div\,}(a_{1}\nabla
u_{1})-b_{1}u_{1})\varphi{\rm d}x=\int_{D_{0}}\rho_{1}\varphi{\rm d}x$
$\displaystyle\qquad\qquad\qquad\qquad\quad=\int_{\partial
D_{0}}a_{1}(\partial_{\nu}u_{1})\varphi{\rm d}s(x)-\int_{D_{0}}a_{1}\nabla
u_{1}\cdot\nabla\varphi{\rm d}x-\int_{D_{0}}b_{1}u_{1}\varphi{\rm d}x$
$\displaystyle\qquad\qquad\qquad\qquad\quad=\int_{\partial
D_{0}}a_{2}(\nu\cdot\bm{u}_{2}+f_{2})\varphi{\rm
d}s(x)-\int_{D_{0}}a_{1}\nabla u_{1}\cdot\nabla\varphi{\rm
dx}-\int_{D_{0}}b_{1}u_{1}\varphi{\rm d}x$

for each $\bm{v}\in X_{a_{2}}$ and $\varphi\in H^{1}(D_{0})$, where the
integral on $\partial D_{0}$ is understood in the dual sense of
$\langle\cdot,\cdot\rangle_{H^{-1/2}\times H^{1/2}}$ due to the term
$\nu\cdot\bm{v}\in H^{-1/2}(\partial D_{0})$ for any $\bm{v}\in X_{a_{2}}$.

Combining above equalities shows that the solution $(u_{1},u_{2})$ to (2.5)
satisfies following variation formulation

$\displaystyle\mathbb{A}(u_{1},\bm{u}_{2};\varphi,\bm{v})=\mathbb{F}(\varphi;\bm{v}),\qquad\text{for}\;(\varphi,\bm{v})\in
H^{1}(D_{0})\times X_{a_{2}},$ (2.8)

where the sesquilinear form $\mathbb{A}(\cdot;\cdot)$ and the functional
$\mathbb{F}(\cdot)$ on $H^{1}(D_{0})\times X_{a_{2}}(D_{0})$ are defined by

$\displaystyle\mathbb{A}(\varphi,\bm{v}):$ $\displaystyle=$
$\displaystyle\int_{D}({\rm div\,}(a_{2}\bm{u}_{2})\cdot{\rm
div\,}(a_{2}\overline{\bm{v}})+a_{2}b_{2}\bm{u}_{2}\cdot\overline{\bm{v}}){\rm
d}x$ $\displaystyle+\int_{D}(a_{1}\nabla
u_{1}\cdot\nabla\overline{\varphi}+b_{1}u_{1}\overline{\varphi}){\rm
d}x-\int_{\partial D}a_{2}b_{2}(\nu\cdot\overline{\bm{v}})u_{1}{\rm
d}s-\int_{\partial D}a_{2}(\nu\cdot\bm{u}_{2})\overline{\varphi}{\rm d}s$

and

$\displaystyle\mathbb{F}(\varphi,\bm{v}):$ $\displaystyle=$
$\displaystyle\int_{D}\rho_{2}{\rm div\,}(a_{2}\overline{\bm{v}}){\rm
d}x-\int_{D}\rho_{1}\overline{\varphi}{\rm d}x+\int_{\partial
D}a_{2}f_{2}\overline{\varphi}{\rm d}s-\int_{\partial
D}a_{2}b_{2}(\nu\cdot\overline{\bm{v}})f_{1}{\rm d}s,$

respectively.

Conversely, on the other hand, we suppose that $(u_{1},\bm{u}_{2})$ is one
solution to (2.8). It follows from the condition ${\rm curl\,}\bm{u}_{2}=0$
that there exists some function denoted by $u_{2}$ such that $u_{2}\in
H^{1}(D_{0})$ with $\bm{u}_{2}=\nabla(u_{2}+c)$ for $c\in{\mathbb{C}}$, since
$D_{0}$ is a simply connected, bounded domain. Following similar arguments to
[7, Theorem 6.5], we can choose some constant $c_{0}\in{\mathbb{C}}$ so that
$u_{2}+c_{0}$ is the solution to (2.5). Thus, the solvability of (2.5) is
equivalent to that of the variational formulation (2.8).

######  Theorem 2.1.

Assume $a_{j}(x),b_{j}>c_{0}>0$ for $j=1,2$. Furthermore, if there exists some
positive constant $\varepsilon_{0}>0$ with $0<\varepsilon_{0}<1$ such that

$\displaystyle\varepsilon_{0}b_{1}>\left(\frac{1+b_{2}}{2}\right)^{2}\qquad{\rm
and}\qquad\varepsilon_{0}\cdot\left(\inf_{x\in
D_{0}}\frac{a_{1}(x)}{a_{2}(x)}\right)>\left(\frac{1+b_{2}}{2}\right)^{2},$

then the variational formulation $(\ref{2.4})$ has a unique solution in the
sense that

$\displaystyle\|u_{1}\|_{H^{1}(D_{0})}+\|\textbf{u}_{2}\|_{X_{a_{2}}}\leq
C(\|\rho_{1}\|_{L^{2}(D_{0})}+\|\rho_{2}\|_{L^{2}(D_{0})}+\|f_{1}\|_{H^{1/2}(\partial
D_{0})}+\|f_{2}\|_{H^{-1/2}(\partial D_{0})}).$ (2.9)

###### Proof.

Using Schwarz’s inequality with the trace theorem, it is first concluded that

$\displaystyle|\mathbb{F}(\varphi,\textbf{u}_{2})|$ $\displaystyle\leq$
$\displaystyle\|\rho_{1}\|_{L^{2}(D_{0})}\|\varphi\|_{L^{2}(D_{0})}+\|\rho_{2}\|_{L^{2}(D_{0})}\|{\rm
div\,}(a_{2}\textbf{v})\|_{L^{2}(D_{0})}$
$\displaystyle\quad+C\|\varphi\|_{H^{1}(D_{0})}\|f_{2}\|_{H^{-1/2}(\partial
D_{0})}+C(\|a_{2}\textbf{v}\|_{L^{2}(D_{0})}+\|{\rm
div\,}(a_{2}\textbf{v})\|_{L^{2}(D_{0})})\|f_{1}\|_{H^{1/2}(\partial D_{0})}$
$\displaystyle\leq$ $\displaystyle
C(\|\rho_{1}\|_{L^{2}(D_{0})}+\|\rho_{2}\|_{L^{2}(D_{0})}+\|f_{1}\|_{H^{1/2}(\partial
D_{0})}+\|f_{2}\|_{H^{-1/2}(\partial
D_{0})})(\|\varphi\|_{H^{1}(D_{0})}+\|\textbf{u}_{2}\|_{X_{a_{2}}}),$

which yields that $\mathbb{F}(\cdot,\cdot)$ defines a bounded linear
functional on the product space $H^{1}(D_{0})\times X_{a_{2}}$.

Let $(\varphi;\bm{v}):=(u_{1},\bm{u}_{2})$. Then, it holds that

$\displaystyle|\mathbb{A}(u_{1},\bm{u}_{2};u_{1},\bm{u}_{2})|$
$\displaystyle=$ $\displaystyle\int_{D_{0}}|({\rm
div\,}(a_{2}\bm{u}_{2})|^{2}+a_{2}b_{2}|\bm{u}_{2}|^{2}){\rm
d}x+\int_{D_{0}}(a_{1}|\nabla u_{1}|^{2}+b_{1}|u_{1}|^{2}){\rm d}x$ (2.10)
$\displaystyle-\int_{\partial
D_{0}}a_{2}(\nu\cdot\bm{u}_{2})\overline{u_{1}}{\rm d}s(x)-\int_{\partial
D_{0}}a_{2}b_{2}(\nu\cdot\overline{\bm{u}_{2}})u_{1}{\rm d}s(x).$

In (2.10), for the integrals on $\partial D$, we apply the divergence theorem
to obtain

$\displaystyle\int_{\partial
D_{0}}a_{2}(\nu\cdot\bm{u}_{2})\overline{u_{1}}{\rm d}s(x)$ $\displaystyle=$
$\displaystyle\int_{D_{0}}{\rm div\,}(a_{2}\bm{u}_{2})\overline{u_{1}}{\rm
d}x+\int_{D_{0}}a_{2}\bm{u}_{2}\cdot\nabla\overline{u}_{1}{\rm d}x.$ (2.11)

Taking the equality (2.11) into (2.10) deduces

$\displaystyle|\mathbb{A}(u_{1},\bm{u}_{2};u_{1},\bm{u}_{2})|$
$\displaystyle\geq$ $\displaystyle\int_{D_{0}}(|{\rm
div\,}(a_{2}\bm{u}_{2})|^{2}+b_{1}|u_{1}|^{2}-(1+b_{2})|{\rm
div\,}(a_{2}\bm{u}_{2})||u_{1}|){\rm d}x$ (2.12)
$\displaystyle+\int_{D_{0}}a_{2}\left(|\bm{u}_{2}|^{2}+\left(\frac{a_{1}}{a_{2}}\right)|\nabla
u_{1}|^{2}-(1+b_{2})|\bm{u}_{2}||\nabla u_{1}|\right){\rm d}x.$

For $0<\varepsilon_{0}<1$, by the mean value inequality, it holds

$\displaystyle(1+b_{2})|\bm{u}_{2}||\nabla
u_{1}|\leq\varepsilon_{0}|\bm{u}_{2}|^{2}+\frac{(1+b_{2})^{2}}{4\varepsilon_{0}}|\nabla
u_{1}|^{2},$ (2.13) $\displaystyle(1+b_{2})|{\rm
div\,}(a_{2}\bm{u}_{2})||u_{1}|\leq\varepsilon_{0}|{\rm
div\,}(a_{2}\bm{u}_{2})|^{2}+\frac{(1+b_{2})^{2}}{4\varepsilon_{0}}|u_{1}|^{2}.$
(2.14)

Define three constants $c_{4},c_{5}$ and $c_{6}$ by
$c_{3}:=b_{1}-(1+b_{2})^{2}/(4\varepsilon_{0})$, $c_{4}:=\inf_{x\in
D_{0}}(a_{1}/a_{2})-(1+b_{2})^{2}/(4\varepsilon_{0})$ and
$c_{5}:=\min\\{(1-\varepsilon_{0}),c_{0}(1-\varepsilon_{0}),c_{3},c_{0}c_{4}\\}$,
respectively. It follows from the assumptions on $a_{j},b_{j}$ for $j=1,2$,
that $c_{3},c_{4}$ and $c_{5}$ are all positive. Thus, we conclude from
(2.12)-(2.14) that

$\displaystyle|\mathbb{A}(u_{1},\bm{u}_{2};u_{1},\bm{u}_{2})|$
$\displaystyle\geq$ $\displaystyle c_{5}\int_{D_{0}}(|{\rm
div\,}(a_{2}\bm{u}_{2})|^{2}+|u_{1}|^{2}+|\bm{u}_{2}|^{2}+|\nabla
u_{1}|^{2}){\rm d}x,$

which means that $\mathbb{A}(\cdot;\cdot)$ is coercive on the space
$H^{1}(D_{0})\times X_{a_{2}}$. Therefore, the existence of a solution to
(2.8) with the stable estimate (2.9) follows from Lax-Milgram theorem. ∎

######  Corollary 2.2.

Let $a_{j},b_{j}$ satisfy the conditions in Theorem 2.1. Then, the boundary
value problem $(\ref{2.1})$ has a unique solution $(u_{1},u_{2})\in
H^{1}(D_{0})\times H^{1}(D_{0})$ such that

$\displaystyle\|u_{1}\|_{H^{1}(D_{0})}+\|u_{2}\|_{H^{1}(D_{0})}\leq
C(\|\rho_{1}\|_{L^{2}(D_{0})}+\|\rho_{2}\|_{L^{2}(D_{0})}+\|f_{1}\|_{H^{\frac{1}{2}}(\partial
D_{0})}+\|f_{2}\|_{H^{-\frac{1}{2}}(\partial D_{0})}).$ (2.15)

###### Proof.

(2.15) is a direct consequence of Theorem 2.1 by using the equality
$u_{2}={\rm div\,}(\gamma_{2}\bm{u}_{2})-\rho_{2}\in L^{2}(D_{0})$ in (2.5). ∎

##  3 Proofs of Theorem 1.1

This section is devoted to the proof of Theorem 1.1 when the conductivity
$\gamma$ is described as a piecewise constant function (1.6). In this case, we
can show that both the embedded obstacle $D$ and coefficients $c_{j}$,
$j=1,2,\cdots,N$, in an unknown finite partition of
$\Omega\setminus\overline{D}$ can be uniquely recovered by local boundary
measurements $\mathcal{C}_{\gamma,D,\mathcal{B}}$ on an non-empty open subset
$\Gamma$ of $\partial\Omega$. A novel and elementary proof will be presented
which depends only on the well-posedness of the boundary value problem (2.5).

Proofs of Theorem 1.1. For $j=1,2$, let
$S_{j}:=\\{\Omega_{j,\ell}:\ell=1,2,\cdots,N_{j}\\}$ denote the sets
consisting of a finite number of subdomains of $\Omega$ with respect to two
classes of different parameters $(\gamma_{1},D_{1},\mathcal{B}_{1})$ and
$(\gamma_{2},D_{2},\mathcal{B}_{2})$. Without of loss of generality, we may
assume that the boundary of $\Omega_{1,1}$ contains a non-empty open subset of
the measurement boundary $\Gamma$. By the assumption on the obstacle $D_{j}$,
$j=1,2$, there exists two real number $1\leq\kappa_{1}\leq N_{1}$ and
$1\leq\kappa_{2}\leq N_{2}$ such that $D_{1}\subseteq\Omega_{1,\kappa_{1}}$
and $D_{2}\subseteq\Omega_{2,\kappa_{2}}$, respectively. For convenience, we
could still assume $\kappa_{1},\kappa_{2}\geq 2$.

In the following, we shall first prove $\gamma_{2}=\gamma_{1}$ in
$\Omega_{1,1}$ by assuming
$\mathcal{C}_{\gamma_{1},D_{1},\mathcal{B}_{1}}=\mathcal{C}_{\gamma_{2},D_{2},\mathcal{B}_{2}}$.
Since $\Gamma\cap\partial\Omega_{1,1}\not=\emptyset$, we can choose a point
$x_{*}\in(\Gamma\cap\partial\Omega_{1,1})$, and define the domain
$\mathcal{O}_{\varepsilon}:=B_{\varepsilon}(x_{*})\cap\Omega_{1,1}$ with
sufficiently small $\varepsilon>0$ so that both $\gamma_{1}$ and $\gamma_{2}$
are all constants in $\mathcal{O}_{\varepsilon}$, due to the assumptions on
$\gamma_{j}$, $j=1,2$. Here, $B_{\varepsilon}(x_{*})$ denotes a ball centered
at $x_{*}$ with radii $\varepsilon>0$. We will first prove that
$\gamma_{2}=\gamma_{1}$ in $\mathcal{O}_{\varepsilon}$ by contradiction.
Without loss of generality, assume that $\gamma_{1}>\gamma_{2}$ in
$\mathcal{O}_{\varepsilon}$.

For $\ell=1,2$, let $u^{(\ell)}_{j}$, $j\in{\mathbb{N}}_{+}$, be the solution
to (1.4) with the Dirichlet data
$f(x):=f_{j}(x)=\chi_{\delta}(x)\Phi_{n}(x,x_{j})$, corresponding to the case
$(\gamma,D,\mathcal{B}):=(\gamma_{\ell},D_{\ell},\mathcal{B}_{\ell})$. Here,
$\Phi_{n}(\cdot,x_{j})$ denotes the fundamental solution of the Laplace
equation $-\triangle\Phi_{n}(x,x_{j})=\delta_{x_{j}}(x)$ in
${\mathbb{R}}^{n}$, given by

$\displaystyle\Phi_{n}(x,x_{j})=\left\\{\begin{array}[]{ll}\displaystyle\;\;\frac{1}{4\pi}\frac{1}{|x-x_{j}|},&\text{for}\;n=3,\\\\[14.22636pt]
\displaystyle 2\pi\ln|x-x_{j}|,&\text{for}\;n=2,\end{array}\right.$

$\chi_{\delta}$ is a smooth cut-off function which is supported in
$B_{\delta}(x_{*})$ and satisfies $|\chi_{\delta}(x)|\leq 1$ for
$x\in{\mathbb{R}}^{n}$ and $\chi_{\delta}(x)=1$ for $x\in
B_{\delta/2}(x_{*})$, and the point sequence $x_{j}$ are defined by

$\displaystyle x_{j}:=x_{*}+\frac{1}{j}\nu(x_{*}),\qquad
j=J_{0},J_{0}+1,J_{0}+2,\cdots\cdots,$

where $J_{0}>0$ is chosen such that
$x_{j}\in{\mathbb{R}}^{n}\setminus\overline{\Omega}$ for all
$J_{0}+{\mathbb{N}}_{+}$. Obviously, one has $f_{j}=0$ on
$\partial\Omega\setminus\overline{\Gamma}$, if $\delta>0$ is sufficiently
small. This means that
$(f_{j}|_{\Gamma},u_{j}^{(1)}|_{\Gamma})\in\mathcal{C}_{\gamma_{1},D_{1},\mathcal{B}_{1}}$
and
$(f_{j}|_{\Gamma},u_{j}^{(2)}|_{\Gamma})\in\mathcal{C}_{\gamma_{2},D_{2},\mathcal{B}_{2}}$
for all large $j\geq J_{0}$. Thus, we conclude by
$\mathcal{C}_{\gamma_{1},D_{1},\mathcal{B}_{1}}=\mathcal{C}_{\gamma_{2},D_{2},\mathcal{B}_{2}}$
that
$\Lambda_{\gamma_{1},D_{1},\mathcal{B}_{1}}f_{j}=\Lambda_{\gamma_{2},D_{2},\mathcal{B}_{2}}f_{j}$
on $\Gamma$ for all large $j\geq J_{0}$.. Then, we consider the following
boundary value problem

$\displaystyle\left\\{\begin{array}[]{ll}{\rm
div\,}(\gamma_{1}\nabla\Psi_{1})-2\Psi_{1}=\zeta_{1},&\text{in}\;\mathcal{O}_{\varepsilon}\\\
{\rm
div\,}(\gamma_{2}\nabla\Psi_{2})-\Psi_{2}=\zeta_{2},&\text{in}\;\mathcal{O}_{\varepsilon}\\\
\Psi_{1}-\Psi_{2}=\varrho_{1},&\text{on}\;\partial\mathcal{O}_{\varepsilon}\\\
\gamma_{1}\partial_{\nu}\Psi_{1}-\gamma_{2}\partial_{\nu}\Psi_{2}=\varrho_{2},&\text{on}\;\partial\mathcal{O}_{\varepsilon}\\\
\end{array}\right.$ (3.6)

where $\zeta_{1}:=-2u^{(1)}_{j}$, $\zeta_{2}:=-u^{(2)}_{j}$,
$\varrho_{1}:=u^{(1)}_{j}-u^{(2)}_{j}$ and
$\varrho_{2}:=\gamma_{1}u^{(1)}_{j}-\gamma_{2}u^{(2)}_{j}$. We intend to prove
the well-posedness of (3.6) by utilizing Corollary 2.2, if
$\zeta_{1},\zeta_{2},\varrho_{1}$ and $\varrho_{2}$ can be shown to be bounded
uniformly for $j\geq J_{0}$ in the corresponding function spaces.

$\bullet$ The boundedness of $\zeta_{1}$ and $\zeta_{2}$. Since $\gamma_{1}$
is constant in $\mathcal{O}_{\delta}$, we can thus consider the following
Dirichlet problem

$\displaystyle\Delta\Theta_{j}=0\quad{\rm
in\;}\Omega\setminus\overline{D_{1}},\qquad\Theta_{j}=f_{j}\quad{\rm
in\;}\partial\Omega\quad{\rm and}\quad\mathcal{B}_{1}\Theta_{j}=0\quad{\rm
on\;}\partial D_{1}.$ (3.7)

It is noticed that $f_{j}(\cdot)=\Phi_{n}(\cdot,x_{j})$ on
$(B_{\delta/2}(x_{*})\cap\partial\Omega)$, which yields that the difference
$\Pi_{j}(\cdot):=\Theta_{j}(\cdot)-\Phi_{n}(\cdot,x_{j})$ in
$\Omega\setminus\overline{D_{1}}$ is bounded uniformly for all $j\geq J_{0}$
under the classical $H^{1}$-norm. This is because $\Pi_{j}(\cdot)$ solves the
harmonic equation $\Delta\Pi_{j}=0$ in $\Omega\setminus\overline{D_{1}}$ with
the boundary data $\Pi_{j}(\cdot)=\Theta_{j}(\cdot)-\Phi_{n}(\cdot;x_{j})$
vanishing for $x\in(B_{\delta/2}(x_{*})\cap\partial\Omega)$. Thus, the
variational method can be applied to prove the above assertion. By the
definition of $\Phi_{n}(\cdot,x_{j})$, one has

$\displaystyle\|\Theta_{j}\|_{L^{2}(\Omega\setminus\overline{D_{1}})}+\|\Theta_{j}\|_{H^{1}(\Omega\setminus\overline{D_{1}\cup
B_{\varepsilon}(x_{*})})}\leq C,\quad{\rm for\;sufficiently\;}\varepsilon>0,$
(3.8)

where $C>0$ is independent of the choice of $j\geq J_{0}$. The estimate (3.8)
leads in a similar way to that the functions $u_{j}^{(1)}$ have the same
property

$\displaystyle\|u_{j}^{(1)}\|_{L^{2}(\Omega\setminus\overline{D_{1}})}+\|u_{j}^{(1)}\|_{H^{1}(\Omega\setminus\overline{D_{1}\cup
B_{\varepsilon}(x_{*})})}\leq C$ (3.9)

uniformly for all $j\geq J_{0}$ by considering the difference
$u^{(1)}_{j}(\cdot)-\Theta_{j}(\cdot)$ which satisfies the equation ${\rm
div\,}(\gamma_{1}\nabla w)=-{\rm
div\,}((\gamma_{1}-\gamma_{1}|_{\Omega_{1,1}})\nabla\Theta_{j})$ in the domain
$\Omega\setminus\overline{D_{1}}$ in the distribution sense with the
homogeneous boundary data $w=0$ on $\partial\Omega$ and $\mathcal{B}_{1}w=0$
on $\partial D_{1}$. Hence, one has from (3.9) that $\zeta_{1}$ is bounded in
$L^{2}(\mathcal{O}_{\delta})$ uniformly all $j\geq J_{0}$. The same assertion
also holds for $\zeta_{2}$ by a similar discussion.

$\bullet$ The boundedness of $\varrho_{1}$ and $\varrho_{2}$. First, by
$\Lambda_{\gamma_{1},D_{1},\mathcal{B}_{1}}f_{j}=\Lambda_{\gamma_{2},D_{2},\mathcal{B}_{2}}f_{j}$
on $\Gamma$, it is known that $\gamma_{1}u_{j}^{(1)}=\gamma_{2}u_{j}^{(1)}$ on
$\Gamma$. Consequently, $\varrho_{1}=0$ and $\varrho_{2}=0$ on $\Gamma$. We
then construct a smooth function $\eta\in C^{\infty}({\mathbb{R}}^{n})$
satisfying ${\rm supp}\eta\subseteq B_{\delta}(x_{*})$, $|\eta|\leq 1$ in
${\mathbb{R}}^{n}$ and $\eta=1$ in $B_{\delta/2}(x_{*})$, where $\delta>0$ is
sufficiently small such that $(B_{\delta}(x_{*})\cap\Gamma)\subseteq\Gamma$.
Define

$\displaystyle\mathbb{H}_{1}(x):=(1-\eta(x))(u_{j}^{(1)}(x)-u_{j}^{(2)}(x)),\quad{\rm
in\;}\mathcal{O}_{\varepsilon},$
$\displaystyle\mathbb{H}_{2}(x):=(1-\eta(x))(\gamma_{1}u_{j}^{(1)}(x)-\gamma_{2}u_{j}^{(2)}(x)),\quad{\rm
in\;}\mathcal{O}_{\varepsilon}.$

It is easy to check that $\varrho_{1}={\rm Tr}_{0}\mathbb{H}_{1}$ and

$\displaystyle\varrho_{2}={\rm
Tr}_{1}\mathbb{H}_{2}-(\partial_{\nu}(1-\eta))(\gamma_{1}u_{j}^{(1)}(x)-\gamma_{2}u_{j}^{(2)}(x)),$
(3.10)

where ${\rm Tr}_{\ell}$, $\ell=1,2$, denote the trace operators defined by
${\rm Tr}_{0}v=v|_{\partial\mathcal{O}_{\varepsilon}}$ and ${\rm
Tr}_{1}v=(\partial_{\nu}v)|_{\partial\mathcal{O}_{\varepsilon}}$ for a smooth
function $v$. Due to $1-\eta=0$ in $B_{\delta/2}(x_{*})$, it follows from
(3.9) that

$\displaystyle\|{\rm
Tr}_{1}\mathbb{H}_{2}-\varrho_{2}\|_{H^{-1/2}(\partial\mathcal{O}_{\varepsilon})}+\|\mathbb{H}_{1}\|_{H^{1}(\mathcal{O}_{\varepsilon})}+\|\mathbb{H}_{2}\|_{H^{1}(\mathcal{O}_{\varepsilon})}\leq
C_{\eta}$ (3.11)

for all $j\geq J_{0}$. Furthermore, using the equations $\Delta
u^{(\ell)}_{j}=0$, one has
$\Delta\mathbb{H}_{2}=\Delta(1-\eta)(\gamma_{1}u_{j}^{(1)}-\gamma_{2}u_{j}^{(2)})+2\nabla(1-\eta)\cdot\nabla(\gamma_{1}u_{j}^{(1)}-\gamma_{2}u_{j}^{(2)})$
which leads to that

$\displaystyle\|\Delta\mathbb{H}_{2}\|_{L^{2}(\mathcal{O}_{\varepsilon})}\leq
C_{\eta}.$ (3.12)

With the aid of the trace theorems, we arrive from (3.11)-(3.12) at that

$\displaystyle\|\varrho_{1}\|_{H^{1/2}(\partial\mathcal{O}_{\varepsilon})}+\|\varrho_{2}\|_{H^{-1/2}(\partial\mathcal{O}_{\varepsilon})}\leq
C_{\eta},$ (3.13)

which finishes the second assertion.

In (3.6), let $a_{1}:=\gamma_{1}$, $a_{2}:=\gamma_{2}$, $b_{1}:=2$ and
$b_{2}:=1$. Obviously, $a_{1},a_{2},b_{1},b_{2}$ satisfy the conditions in
Theorem 1.1, due to $\gamma_{1}>\gamma_{2}$ in $\mathcal{O}_{\varepsilon}$. It
is then shown by Corollary 2.2 that

$\displaystyle\sum_{\ell=1}^{2}\|\Psi_{\ell}\|_{H^{1}(\mathcal{O}_{\varepsilon})}\leq
C(\|\zeta_{1}\|_{L^{2}(\mathcal{O}_{\varepsilon})}+\|\zeta_{2}\|_{L^{2}(\mathcal{O}_{\varepsilon})}+\|\varrho_{1}\|_{H^{1/2}(\partial\mathcal{O}_{\varepsilon})}+\|\varrho_{2}\|_{H^{-1/2}(\partial\mathcal{O}_{\varepsilon})})\leq
C$ (3.14)

uniformly for $j\geq J_{0}$. Notice that, for boundary data defined in (3.6),
$(\Psi_{1},\Psi_{2}):=(u_{j}^{(1)},u_{j}^{(1)})$ is the unique solution to
(3.6). Using the trace theorem again, it follows from the above equality that

$\displaystyle+\infty\longleftarrow\|f_{j}\|_{H^{1/2}(\partial\mathcal{O}_{\varepsilon}\cap\Gamma)}\leq\|u_{j}^{(1)}\|_{H^{1}(\mathcal{O}_{\varepsilon})}\leq
C,\quad{\rm as\;}j\to\infty,$

which clearly leads to a contradiction. Hence, $\gamma_{1}=\gamma_{2}$ in
$\mathcal{O}_{\varepsilon}$.

If $\Omega_{1,1}\subseteq\Omega_{2,\ell_{0}}$ for some $j_{0}\leq N_{2}$, then
one immediately has $\gamma_{2}=\gamma_{1}$ in $\Omega_{1,1}$. Otherwise, we
shall claim that it must hold

$\displaystyle\overline{\Omega_{1,1}}$ $\displaystyle=$
$\displaystyle\bigcup_{1\leq j\leq M}\overline{\Omega_{2,\ell_{j}}}\quad{\rm
for\;some\;}M\leq N_{2},$ (3.15)

which means that $\Omega_{1,1}$ consists of a finite number of elements in
$S_{2}$. We shall prove (3.15) by contradiction. Assume that it does not hold
true for (3.15). Then, there exists some subdomain denoted by
$\Omega_{2,j_{1}}$ in $S_{2}$ such that
$(\Omega_{1,1}\cap\Omega_{2,j_{1}})\not=\emptyset$,
$\Omega_{1,1}\nsubseteq\Omega_{2,j_{1}}$ and
$\Omega_{2,j_{1}}\nsubseteq\Omega_{1,1}$. So we can choose two subsets $A_{1}$
and $A_{2}$ of ${\mathbb{R}}^{n}$ satisfying the following conditions: ${\rm
i})$ $A_{1}\subseteq(\Omega_{1,1}\cap\Omega_{2,j_{1}})$; ${\rm ii})$
$A_{2}\subseteq(\Omega_{1,2}\cap\Omega_{2,j_{1}})$ with
$\emptyset\not=\partial
A_{2}\cap(\partial\Omega_{1,1}\cap\partial\Omega_{1,2})$ having a non-empty
open subset, denoted by $\Sigma$, in the low-dimensional manifold of
${\mathbb{R}}^{n}$. Here, $\Omega_{1,2}$ denotes the element in $S_{1}$ which
shares a non-empty common partial boundary with $\Omega_{1,1}$. We now take a
point $p$ at the boundary $\partial A_{1}\cap\partial\Omega_{2,j_{1}}$.
Recalling the assumption on the domain $\Omega\setminus\overline{D}_{2}$, we
can always find a bounded curve $\ell_{x_{*},p}\subseteq\Omega_{1,1}$ with two
endpoints $x_{*}$ and $p$ such that $\ell_{x_{*},p}\cap A$ ($A\in S_{2}$)
contains at most one point. Therefore, there exists a finite number of small
balls $B_{\varepsilon_{1}}(p_{j})$ ($1\leq j\leq M_{1}<\infty$) in
$\Omega_{1,1}$ with $p_{j}\in\ell_{x_{*},p}$, $p_{1}=x_{*}$, $p_{M_{1}}=p$ and
sufficiently small $\varepsilon_{1}>0$ such that
$B_{\varepsilon_{1}}:=\bigcup_{1\leq j\leq
M_{1}}B_{\varepsilon_{1}}(p_{j})\supset\ell_{x_{*},p}$.

Since $\gamma_{1}$ is constant in $\Omega_{1,1}$, we let
$\gamma_{1}(x):=c_{1,1}\in{\mathbb{R}}_{+}$ in $\Omega_{1,1}$. We next claim
that $\gamma_{2}(x)=c_{1,1}$ in $A^{\prime}_{1}:=A_{1}\cap
B_{\varepsilon_{1}}(p)$. Without loss of generality, it can be assumed that
$\gamma_{2}(x)=c_{1,1}$ in
$B_{\varepsilon_{1}}\setminus\overline{\Omega_{2,j_{1}}}$. For $\ell=1,2$, we
now consider the Dirichlet-Green functions $G_{\ell}(\cdot,y)$,
$y\in(\Omega\setminus\overline{D_{\ell}})$, which solves the following
boundary value problem

$\displaystyle\left\\{\begin{array}[]{ll}{\rm div\,}(\gamma_{\ell}\nabla
G_{\ell}(\cdot,y))=-\delta_{y}(\cdot)&\text{in}\;\Omega\setminus\overline{D_{\ell}},\\\\[4.2679pt]
G_{\ell}(\cdot,y)=0&\text{on}\;\partial\Omega,\\\\[4.2679pt]
\mathcal{B}_{\ell}G_{\ell}(\cdot,y)=0&\text{on}\;\partial
D_{\ell}.\end{array}\right.$

It is easily checked that $G_{\ell}(\cdot,y)$ has the same singularity with
the fundamental solution $\Phi_{n}(\cdot,y)$ at $x=y$. For any $f\in
H^{1/2}(\partial\Omega)$ with ${\rm
supp}f\subseteq\Gamma_{1}:=(\Gamma\cap\partial\Omega_{1,1})$, let
$u^{(\ell)}$, $\ell=1,2$, be the solutions to (1.4) with the boundary data
$f$, corresponding to two classes of different parameters
$(\gamma_{1},D_{1},\mathcal{B}_{1})$ and $(\gamma_{1},D_{2},\mathcal{B}_{2})$.
Using the representation formulation for $u^{(\ell)}$, $\ell=1,2$, yields

$\displaystyle u^{(\ell)}(x)$ $\displaystyle=$
$\displaystyle\int_{\partial\Omega}\gamma_{\ell}(y)\left(u^{(\ell)}(y)\frac{\partial
G_{\ell}(x,y)}{\partial\nu(y)}-\frac{\partial
u^{(\ell)}(y)}{\partial\nu(y)}G_{\ell}(x,y)\right){\rm d}s(y)$ (3.17)
$\displaystyle=$ $\displaystyle\int_{\Gamma_{1}}\frac{\partial
G_{\ell}(x,y)}{\partial\nu(y)}\gamma_{\ell}(y)f(y){\rm d}s(y)\qquad\quad{\rm
for\;}x\in B_{\varepsilon_{1}}(p)\setminus\overline{\Omega_{2,j_{1}}}.$

Recalling
$\Lambda_{\gamma_{1},D_{1},\mathcal{B}_{1}}f=\Lambda_{\gamma_{2},D_{2},\mathcal{B}_{2}}f$,
the unique continuation principle can be thus applied for the Laplace equation
to deduce $u^{(1)}(x)=u^{(2)}(x)$ in $\mathcal{O}_{\varepsilon}\cap
B_{\varepsilon_{1}}(x_{*})$. Then, it holds by turns that
$u^{(1)}(x)=u^{(2)}(x)$ in
$B_{\varepsilon_{1}}\setminus\overline{\Omega_{2,j_{1}}}$. Thus, it is shown
by (3.17) in combination with the fact $\gamma_{1}(x)=\gamma_{2}(x)$ in
$\mathcal{O}_{\varepsilon}$ that

$\displaystyle\int_{\Gamma_{1}}\frac{\partial
G_{1}(x,y)}{\partial\nu(y)}f(y){\rm d}s(y)=\int_{\Gamma_{1}}\frac{\partial
G_{2}(x,y)}{\partial\nu(y)}f(y){\rm ds}(y)$ (3.18)

for $x\in B_{\varepsilon_{1}}(p)\setminus\overline{\Omega_{2,j_{1}}}$ and all
$f\in H^{1/2}(\partial\Omega)$ with the support in $\Gamma_{1}$. This means
that $\partial_{\nu}G_{1}(x,\cdot)=\partial_{\nu}G_{2}(x,\cdot)$ on
$\Gamma_{1}$. Moreover, one can also concalud from the similar discussion
above (3.18) that

$\displaystyle G_{1}(x,y)=G_{2}(x,y)\qquad{\rm for\;}x,y\in
B_{\varepsilon_{1}}\setminus\overline{\Omega_{2,j_{1}}}\;{\rm and\;}\;x\neq
y.$ (3.19)

We are at a position to show that $\gamma_{2}(x)=c_{1,1}$ in $A^{\prime}_{1}$,
which will be proved by contradiction. Assume that $\gamma_{2}(x)\neq c_{1,1}$
in $A^{\prime}_{1}$. Without loss of generality, we may assume
$\gamma_{1}>\gamma_{2}$ in $A^{\prime}_{1}$. Similar to (3.6), we instead
consider the following boundary value problem

$\displaystyle\left\\{\begin{array}[]{ll}{\rm
div\,}(\gamma_{1}\nabla\Psi_{1})-2\Psi_{1}=\zeta_{1},&\text{in}\;A^{\prime}_{1}\\\
{\rm
div\,}(\gamma_{2}\nabla\Psi_{2})-\Psi_{2}=\zeta_{2},&\text{in}\;A^{\prime}_{1}\\\
\Psi_{1}-\Psi_{2}=\varrho_{1},&\text{on}\;\partial A^{\prime}_{1}\\\
\gamma_{1}\partial_{\nu}\Psi_{1}-\gamma_{2}\partial_{\nu}\Psi_{2}=\varrho_{2},&\text{on}\;\partial
A^{\prime}_{1}\\\ \end{array}\right.$ (3.24)

with $\zeta_{1},\zeta_{2},\varrho_{1}$ and $\varrho_{2}$ defined by

$\displaystyle\zeta_{1}:=-2G_{1}(\cdot,x_{j}),\quad\zeta^{\prime}_{2}:=-G_{2}(\cdot,x_{j}),$
$\displaystyle\varrho_{1}:=G_{1}(\cdot,x_{j})-G_{2}(\cdot,x_{j}),\quad\varrho_{2}:=\gamma_{1}\partial_{\nu}G_{1}(\cdot,x_{j})-\gamma_{2}\partial_{\nu}G_{2}(\cdot,x_{j}),$

respectively, where $x_{j}$ are defined by $x_{j}:=p+(1/j)\nu(p)\in
B_{\varepsilon_{1}}(p)\setminus\overline{\Omega_{2,j_{1}}}$ for all large
$j>0$.

For $\zeta_{j},j=1,2,$ it follows from the properties of the Dirichlet-Green’s
function (cf.[13]) that there exists a fixed constant $K>0$, depending only on
the coefficients of the conductivity equation, such that
$K^{-1}\Phi_{n}(x,y)\leq G_{1}(x,y),G_{2}(x,y)\leq K\Phi_{n}(x,y)$ for all
$x,y\in A^{\prime}_{1}$. This leads to that $\zeta_{1}$ and $\zeta_{2}$ are
bounded in $L^{2}(A^{\prime}_{1})$ uniformly for sufficiently large $j>0$.

Furthermore, by the equality (3.19) and transmission conditions on $\partial
A^{\prime}_{1}\cap\partial\Omega_{2,j_{1}}$, we observe that $\varrho_{1}=0$
and $\varrho_{2}=0$ on $\partial A^{\prime}_{1}\cap\partial\Omega_{2,j_{1}}$.
Thus, in order to obtain the uniform boundedness of $\varrho_{1}$ and
$\varrho_{2}$ in $H^{1/2}(\partial A^{\prime}_{1})$ and $H^{-1/2}(\partial
A^{\prime}_{1})$, respectively, it is enough to estimate $G_{1}(\cdot,x_{j})$
and $G_{2}(\cdot,x_{j})$ in the sense of
$H^{1}(A^{\prime}_{1}\setminus\overline{B_{\varepsilon^{\prime}_{1}}(p)})$
with $\varepsilon^{\prime}_{1}=\varepsilon_{1}/2$ (see the analysis between
(3.10) and (3.13)). Obviously, this holds true due to the properties of the
Dirichlet-Green’s functions. Corollary 2.2 can be now applied to obtain the
estimate

$\displaystyle\|\Psi_{1}\|_{H^{1}(A^{\prime}_{1})}+\|\Psi_{2}\|_{H^{1}(A^{\prime}_{1})}\leq
C$ (3.25)

uniformly for sufficiently large $j>0$. Noticing that for each $j>0$,
$(\Psi_{1},\Psi_{2}):=(G_{1}(\cdot,x_{j}),G_{2}(\cdot,x_{j}))$ is the unique
solution to (3.24), one has from (3.25) that $G_{1}(\cdot,x_{j})$ is bounded
in $H^{1}(A^{\prime}_{1})$ uniformly for all sufficiently $j>0$. However, this
is contradicted with the fact

$\displaystyle\|G_{1}(\cdot,x_{j})\|_{H^{1}(A^{\prime}_{1})}\to\infty\qquad{\rm
as\;}j\to\infty.$

Therefore, one has $\gamma_{2}(x)=c_{1,1}$ in $A^{\prime}_{1}$, which further
yields $\gamma_{2}(x)=c_{1,1}$ in $\Omega_{2,j_{1}}$, since
$A^{\prime}_{1}\subset A_{1}\subset(\Omega_{1,1}\cap\Omega_{2,j_{1}})$ and
$\gamma_{2}(x)$ is constant in $\Omega_{2,j_{1}}$.

Based on the above assertion, we further claim that
$\gamma_{1}(x)=\gamma_{2}(x)$ in $A_{2}$, a previously chosen subdomain in
condition ${\rm ii})$ after (3.15), which is sill proved by contradiction.
Suppose that $\gamma_{1}\not=\gamma_{2}$ in $A_{2}$. Without loss of
generality, we can assume that $\gamma_{1}>\gamma_{2}$ in $A_{2}$. We now take
a new point $p_{*}\in\Sigma$ (see the definition in condition ${\rm ii})$),
and define the point sequence $y_{j}:=p_{*}+(1/j)\nu(p_{*})$ for sufficiently
large $j>0$ such that $y_{j}\in(\Omega_{1,1}\cap B_{\varepsilon_{2}}(p_{*}))$
for sufficiently small $\varepsilon_{2}>0$. Following similar discussions as
in deriving (3.19), we can first prove that

$\displaystyle G_{1}(x,y_{j})=G_{2}(x,y_{j})\qquad{\rm
for\;}x_{j}\not=x\in\Omega_{1,1}\cap B_{\varepsilon_{2}}(p_{*}).$ (3.26)

for each large $j>0$, This motivates us to consider another new boundary value
problem in the subdomain $A^{\prime}_{2}:=A_{2}\cap
B_{\varepsilon_{2}}(p_{*})$:

$\displaystyle\left\\{\begin{array}[]{ll}{\rm
div\,}(\gamma_{1}\nabla\Psi_{1}^{\prime})-2\Psi_{1}^{\prime}=\zeta_{1}^{\prime},&\text{in}\;A^{\prime}_{2}\\\
{\rm
div\,}(\gamma_{2}\nabla\Psi_{2}^{\prime})-\Psi_{2}^{\prime}=\zeta_{2}^{\prime},&\text{in}\;A^{\prime}_{2}\\\
\Psi_{1}^{\prime}-\Psi_{2}^{\prime}=\varrho_{1}^{\prime},&\text{on}\;\partial
A^{\prime}_{2}\\\
\gamma_{1}\partial_{\nu}\Psi_{1}^{\prime}-\gamma_{2}\partial_{\nu}\Psi_{2}^{\prime}=\varrho_{2}^{\prime},&\text{on}\;\partial
A^{\prime}_{2}\\\ \end{array}\right.$ (3.31)

which is similar to (3.24) with $A_{1}^{\prime}$ and
$\zeta_{1},\zeta_{2},\varrho_{1},\varrho_{2}$ replaced by $A_{2}^{\prime}$ and

$\displaystyle\zeta^{\prime}_{1}:=-2G_{1}(\cdot,y_{j}),\quad\zeta^{\prime}_{2}:=-G_{2}(\cdot,y_{j}),$
$\displaystyle\varrho^{\prime}_{1}:=G_{1}(\cdot,y_{j})-G_{2}(\cdot,y_{j}),\quad\varrho^{\prime}_{2}:=\gamma_{1}\partial_{\nu}G_{1}(\cdot,y_{j})-\gamma_{2}\partial_{\nu}G_{2}(\cdot,y_{j}),$

respectively.

Using the properties on the Dirichlet-Green’s functions again yields that both
$\zeta_{1}^{\prime}$ and $\zeta_{2}^{\prime}$ are bounded in
$L^{2}(A_{2}^{\prime})$ uniformly for all large $j>0$. Moreover, (3.26) in
combination with the transmission conditions on $\Sigma$ leads to
$\varrho_{1}^{\prime}=\varrho_{2}=0$ on $\partial A_{2}^{\prime}\cap\Sigma$.
This makes that $\varrho_{1}^{\prime}$ and $\varrho_{2}^{\prime}$ are bounded
in $H^{1/2}(\partial A^{\prime}_{2})$ and $H^{-1/2}(\partial A^{\prime}_{2})$,
respectively, uniformly for all large $j>0$; see the analysis between (3.24)
and (3.25). Corollary 2.2 can be again applied to obtain the estimate

$\displaystyle\|\Psi_{1}^{\prime}\|_{H^{1}(A^{\prime}_{2})}+\|\Psi_{2}^{\prime}\|_{H^{1}(A^{\prime}_{2})}\leq
C$ (3.32)

uniformly for sufficiently large $j>0$. However, this is a contradiction since
$(\Psi_{1}^{\prime},\Psi_{2}^{\prime}):=(G_{1}(\cdot,y_{j}),G_{2}(\cdot,y_{j}))$
is the unique solution to (3.31) and

$\displaystyle\|G_{1}(\cdot,y_{j})\|_{H^{1}(A^{\prime}_{2})}\to\infty,\qquad{\rm
as\;}j\to\infty.$

Therefore, we arrive at that $\gamma_{1}=\gamma_{2}$ in $A_{2}$. Notice that
this is impossible since $\gamma_{2}$ has been proved to be $c_{1,1}$ in
$A_{2}$ and $\gamma_{1}:=c_{1,2}$, a new constant, is assumed to be not
$c_{1,1}$ (see the conditions between (1.6) and (1.7)). Hence, (3.15) holds
true.

It still needs to be shown that $\gamma_{2}=c_{1,1}$ in $\Omega_{1,1}$. By
(3.15), we can easily choose some subdomain denoted by $\Omega_{2,l_{1}}$ such
that $\Omega_{2,l_{1}}\subseteq\Omega_{1,1}$ and
$\Gamma\cap\partial\Omega_{2,l_{1}}\not=\emptyset$. That is,
$\partial\Omega_{2,l_{1}}$ and $\Gamma$ share a non-empty open subset in a
($n-1$)-dimensional manifold of ${\mathbb{R}}^{n}$. By repeating the above
process, one can conclude $\gamma_{2}=c_{1,1}$ in $\Omega_{2,l_{1}}$. If
$\Omega_{1,1}=\Omega_{2,l_{1}}$, then the proof is ended. If not, we can
choose another subdomain denoted by $\Omega_{2,l_{2}}$ such that
$\Omega_{2,l_{2}}\subseteq\Omega_{1,1}$ and
$\emptyset\not=\partial\Omega_{2,l_{1}}\cap\partial\Omega_{2,l_{2}}$
containing a non-empty open subset in ($n-1$)-dimensional manifold of
${\mathbb{R}}^{n}$. Again repeating the previous analysis leads to
$\gamma_{2}=c_{1,1}$ in $\Omega_{2,l_{2}}$. If
$\Omega_{1,1}=(\Omega_{2,l_{1}}\cup\Omega_{2,l_{2}})$, then the proof is
ended. If not, we continue to choose a new subdomain denoted by
$\Omega_{2,l_{3}}$ such that $\Omega_{2,l_{3}}\subseteq\Omega_{1,1}$, and
$\emptyset\not=\partial\Omega_{2,l_{3}}\cap\partial\Omega_{2,l_{2}}$ or
$\emptyset\not=\partial\Omega_{2,l_{3}}\cap\partial\Omega_{2,l_{1}}$ contains
at least a non-empty open subset in ($n-1$)-dimensional manifold of
${\mathbb{R}}^{n}$; see Figure 2 for a geometry setting. The similar arguments
leads to $\gamma_{2}=c_{1,1}$ in $\Omega_{2,l_{3}}$ once again. Thus, we can
arrive in turn at $\gamma_{2}=c_{1,1}$ in $\Omega_{1,1}$ since $M$ in (3.15)
is just a bounded positive integer.

Finally, the inverse conductivity problem can be reduced in a similar way to
the unique determination of an unknown obstacle embedded into a known
background medium. The standard arguments as in [17] implies that
$D_{1}=D_{2}$ and $\mathcal{B}_{1}=\mathcal{B}_{2}$. The proof is thus
complete.

## Acknowledgements

This work is supported by the NNSF of China under Grant No. 11771349.

## References

  * [1] G. Alessandrini and S. Vessella, Lipschitz stability for the inverse conductivity problem, Advances in Applied Mathematics 35 (2005), 207-241.
  * [2] K. Astala and L. Päivärinta, Calderón inverse conductivity problem in the plane, Ann. of Math. 163 (2006), 265-299.
  * [3] A.L. Bukhgeim, G. Uhlmann, Recovering a potential from partial Cauchy data, Comm. PDE 27 (2002), 653-668.
  * [4] A.L. Bukhgeim, Recovering a potential from Cauchy data in the two- dimensional case, J. Inverse Ill-posed Probl. 16 (2008), 19-34.
  * [5] R.M. Brown, Global Uniqueness in the Impedance-Imaging Problem for Less Regular Conductivities, SIAM J. Math. Anal. 27 (1996), 1049-1056.
  * [6] R.M. Brown and R.H. Torres, Uniqueness in the Inverse Conductivity Problem for Conductivities with 3/2 Derivatives in $L^{p}$, $p>2n$, Journal of Fourier Analysis and Applications 9 (2003), 563-574.
  * [7] F. Cakoni and D. Colton, Qualitative Methods in Inverse Scattering Theory, Springer, Berlin, 2006.
  * [8] V. Druskin, On the uniqueness of inverse problems from incomplete boundary data, SIAM J. Appl. Math., 58 (1998), 1591-1603.
  * [9] A. P. Calderón, On an inverse boundary value problem, Seminar on Numerical Analysis and its Applications to Continuum Physics (Rio de Janeiro, 1980), 65-73, Soc. Brasil. Mat., Rio de Janeiro, 1980. Reprinted in: Comput. Appl. Math. 25 (2-3) (2006), 133?138.
  * [10] F. Qu and J. Yang, On recovery of an inhomogeneous cavity in inverse acoustic scattering, Inverse Probl. Imaging, Accepted for publication, 2017.
  * [11] F. Qu, J. Yang and B. Zhang, Recovering an elastic body containing embedded obstacles by the acoustic far-field measurements, Inverse Problems, in appear, 2017.
  * [12] B. Haberman and D. Tataru, Uniqueness in Calderón’s problem with Lipschitz conductivities, Duke Math. J. 162 (2013), 497-516.
  * [13] V. Isakov, On uniqueness in the inverse transmission scattering problem, Comm. PDE 15 (1990), 1565-1587.
  * [14] V. Isakov, On uniqueness in the inverse conductivity problem with local data, Inverse Probl. Imaging 1 (2007), 95-105.
  * [15] O. Imanuvilov, G. Uhlmann and M. Yamamoto,The Calderón problem with partial data in two dimensions, J. Amer. Math. Soc. 23 (2010), 655-691.
  * [16] C. E. Kenig, J. Sjóstrand and G.Uhlmann,The Calderón problem with partial data, Ann. of Math. 165 (2007), 567-591.
  * [17] A. Kirsch and L. Päivärinta, On recovering obstacles inside inhomogeneities, Math. Methods Appl. Sci. 21 (1998), 619-651.
  * [18] R. Kohn and M. Vogelius, Determining conductivity by boundary measurements, Comm. Pure. Appl. Math. 37 (1984), 289-298.
  * [19] R. Kohn and M. Vogelius, Determining conductivity by boundary mea- surements II. Interior Results, Comm. Pure. Appl. Math. 38 (1985), 643-667.
  * [20] A. Nachman, Global uniqueness for a two-dimensional inverse boundary value problem, Ann. of Math. 143 (1996), 71-96.
  * [21] L. Päivärinta, A. Panchenko, and G. Uhlmann, Complex geometrical optics solutions for Lipschitz conductivities, Revista Matematica Iberoamericana 19 (2003), 57-72.
  * [22] J. Sylvester and G. Uhlmann, A global uniqueness theorem for an inverse boundary value problem, Ann. of Math. 125 (1987), 153-169.
  * [23] J. Yang and B. Zhang, Direct and inverse scattering by embedded obstacles in an unknown inhomogeneous background medium, Preprint.
  * [24] J. Yang, B. Zhang and H. Zhang, Uniqueness in inverse acoustic and electromagnetic scattering by penetrable obstacles, arXiv:1305.0917.


11institutetext: Department of Materials Science and Engineering, Delft
University of Technology, Mekelweg 2, Delft, 2628 CD, The Netherlands
22institutetext: Pattern Recognition and Bioinformatics Laboratory, Delft
University of Technology, Van Mourik Broekmanweg 6, Delft, 2628 XE, The
Netherlands 33institutetext: School of Engineering, Brown University, 184 Hope
St., Providence, RI 02912, USA
33email<EMAIL_ADDRESS>
# iPINNs: Incremental learning for Physics-informed neural networks

Aleksandr Dekhovich 11 Marcel H.F. Sluiter 11 David M.J. Tax 22 Miguel A.
Bessa 33

###### Abstract

Physics-informed neural networks (PINNs) have recently become a powerful tool
for solving partial differential equations (PDEs). However, finding a set of
neural network parameters that lead to fulfilling a PDE can be challenging and
non-unique due to the complexity of the loss landscape that needs to be
traversed. Although a variety of multi-task learning and transfer learning
approaches have been proposed to overcome these issues, there is no
incremental training procedure for PINNs that can effectively mitigate such
training challenges. We propose incremental PINNs (iPINNs) that can learn
multiple tasks (equations) sequentially without additional parameters for new
tasks and improve performance for every equation in the sequence. Our approach
learns multiple PDEs starting from the simplest one by creating its own
subnetwork for each PDE and allowing each subnetwork to overlap with
previously learned subnetworks. We demonstrate that previous subnetworks are a
good initialization for a new equation if PDEs share similarities. We also
show that iPINNs achieve lower prediction error than regular PINNs for two
different scenarios: (1) learning a family of equations (e.g., 1-D convection
PDE); and (2) learning PDEs resulting from a combination of processes (e.g.,
1-D reaction-diffusion PDE). The ability to learn all problems with a single
network together with learning more complex PDEs with better generalization
than regular PINNs will open new avenues in this field.

###### Keywords:

Physic-informed neural networks (PINNs) Scientific machine learning (SciML)
Incremental learning Sparsity

##  1 Introduction

Deep neural networks (DNNs) play a central role in scientific machine learning
(SciML). Recent advances in neural networks find applications in real-life
problems in physics [38, 6, 50, 28], medicine [41, 51, 49], finance [22, 66,
17, 58], and engineering [5, 55, 11, 26]. In particular, they are also applied
to solve Ordinary Differential Equations and Partial Differential Equations
(ODEs/PDEs) [34, 42, 65, 46]. Consider the following PDE,

$\displaystyle\mathcal{F}[u(\mathbf{x},t)]$
$\displaystyle=f(\mathbf{x}),\quad\mathbf{x}\in\mathrm{\Omega},\
t\in[t_{0},T],$ (1) $\displaystyle\mathcal{B}[u(\mathbf{x},t)]$
$\displaystyle=b(\mathbf{x}),\quad\mathbf{x}\in\mathrm{\partial\Omega},$ (2)
$\displaystyle u(\mathbf{x},t_{0})$
$\displaystyle=h(\mathbf{x}),\quad\mathbf{x}\in\mathrm{\Omega},$ (3)

where $\mathcal{F}$ is a differential operator, $\mathcal{B}$ is a boundary
condition operator, $h(\mathbf{x})$ is an initial condition, and
$\mathrm{\Omega}$ is a bounded domain.

The first neural network-based approaches incorporated a form of the equation
into the loss function with initial and boundary conditions included as hard
constraints [31, 32]. However, these works used relatively small neural
networks with one or two hidden layers. On the contrary, PINNs [46] encode
initial and boundary conditions as soft constraints into the loss function of
a DNN. Subsequently, PINNs and their extensions found applications in fluid
mechanics [40, 60, 7], inverse problems [13, 37, 61] and finance [46, 2].
Later, the generalized version of PINNs, called XPINNs [24], was proposed by
decomposing the domain into multiple subdomains. However, this method uses as
many networks as the number of subdomains, increasing the algorithm’s
complexity. Multi-head PINNs (MH-PINNs) [67] is a multi-task learning approach
for PINNs that is employed to learn stochastic processes, synergistic learning
of PDEs and uncertainty quantification. MH-PINNs have a shared part of the
network and task-specific output heads for prediction. Therefore, it uses
additional parameters for every head, increasing the model’s size with respect
to the number of tasks.

Despite the popularity of DNNs, and PINNs in particular, there are few
incremental learning algorithms available in SciML literature. Yet,
incremental learning and continual learning algorithms [8, 9, 27] are capable
of handling tasks sequentially, instead of altogether as in multi-task
learning and other strategies. Moreover, they are still capable of not
forgetting how to solve all of the previously learned tasks. If tasks have
some similarities with each other, new tasks have the potential of being
learned better (i.e., faster or with lower testing error) with the help of
previously learned ones. The goal of this work is to propose an incremental
learning algorithm for PINNs such that similar symbiotic effects can be
obtained.

#### Background and main challenges.

PINNs formulate the PDE solution problem by including initial and boundary
conditions into the loss function of a neural network as soft constraints. Let
us denote the output of the network $\mathcal{N}$ with learnable parameters
$\theta$ as $\hat{u}(\theta,\mathbf{x},t)=\mathcal{N}(\theta;\mathbf{x},t)$.
Then sampling the set of collocation points, i.e. a set of points in the
domain, $\mathcal{CP}=\\{(x^{i},t^{i}):x^{i}\in\text{int }\mathrm{\Omega},\
t^{i}\in(t_{0},T],\ i=1,2,\ldots N_{\mathcal{F}}\\}$, the set of initial
points $\mathcal{IP}=\\{(x^{j},t_{0}):x^{j}\in\partial\mathrm{\Omega},\
j=1,2,\ldots,N_{u_{0}}\\}$ and the set of boundary points
$\mathcal{BP}=\\{(x^{k},t^{k}):x^{k}\in\partial\mathrm{\Omega},\
t^{k}\in(t_{0},T],\ k=1,2,\ldots,N_{b}\\}$ one can write the optimization
problem and loss function arising from PINNs as follows:

$\displaystyle\mathcal{L}(\theta)$
$\displaystyle=\mathcal{L}_{\mathcal{F}}(\theta)+\mathcal{L}_{u_{0}}(\theta)+\mathcal{L}_{b}(\theta)\to\min_{\theta},$
(4) $\displaystyle\mathcal{L}_{\mathcal{F}}(\theta)$
$\displaystyle=\frac{1}{N_{\mathcal{F}}}\sum_{i=1}^{N_{\mathcal{F}}}\big{\lvert}\big{\lvert}\mathcal{F}[\hat{u}(\theta,x^{i},t^{i})]-f(x^{i})\big{\rvert}\big{\rvert}^{2},\quad(x^{i},t^{i})\in\mathcal{CP},$
(5) $\displaystyle\mathcal{L}_{u_{0}}(\theta)$
$\displaystyle=\frac{1}{N_{u_{0}}}\sum_{j=1}^{N_{u_{0}}}\big{\lvert}\big{\lvert}\hat{u}(\theta,x^{j},t_{0})-h(x^{j})\big{\rvert}\big{\rvert}^{2},\quad(x^{j},t_{0})\in\mathcal{IP},$
(6) $\displaystyle\mathcal{L}_{b}(\theta)$
$\displaystyle=\frac{1}{N_{b}}\sum_{k=1}^{N_{b}}\big{\lvert}\big{\lvert}\mathcal{B}[\hat{u}(\theta,x^{k},t^{k})]-b(x^{k})\big{\rvert}\big{\rvert}^{2},\quad(x^{k},t^{k})\in\mathcal{BP}.$
(7)

However, sometimes PINNs struggle to learn the ODE/PDE dynamics [59, 30, 47,
43] (see Figure 1). Wight & Zhao [62] proposed several techniques to improve
the optimization process compared to the original formulation: mini-batch
optimization and adaptive sampling of collocation points. Adaptive sampling in
time, splits the time interval $[t_{0},T]=\cup_{k=0}^{K}[t_{k-1},t_{k}],\
t_{K}=T$, and solves an equation on the first interval $[t_{0},t_{1}]$, then
on $[t_{0},t_{2}]$, and so on up to $[t_{0},T]$. Thus, if a solution can be
found on a domain $\mathrm{\Omega}\times[t_{0},t_{k-1}]$, then the network is
pretrained well for the extended domain $\mathrm{\Omega}\times[t_{0},t_{k}]$.
Krishnapriyan et al. [30] proposed the seq2seq approach that splits the domain
into smaller subdomains in time and learns the solution on each of the
subdomains with a separate network. Thus, both adaptive sampling in time and
seq2seq are based on the idea of splitting the domain into multiple
subdomains, on which solutions can be learned easier.

As explained in [47], improving PINN’s solutions by considering small
subdomains is possible because the loss residuals ($\mathcal{L}_{\mathcal{F}}$
term) can be trivially minimized in the vicinity of fixed points, despite
corresponding to nonphysical system dynamics that do not satisfy the initial
conditions. Therefore, the reduction of the domain improves the convergence of
the optimization problem (4) and helps to escape nonphysical solutions.

(a) Exact solution.

(b) PINNs solution.

Figure 1: 1-D reaction equation with parameter $\rho=5$ (see P1.2).

Another strategy is to consider transfer learning. Transfer learning is
commonly used in computer vision and natural language processing [3, 57, 48,
23]. It tries to improve the optimization process by starting with better
weight initialization. In PINNs, transfer learning is also successfully used
to accelerate the loss convergence [18, 45, 10, 63]. For instance, Chen et al.
[12] apply transfer learning to learn faster different PDEs creating tasks by
changing coefficients or source terms in equations. Analogously, curriculum
regularization (similar to curriculum learning [4]) is proposed in [30] to
find good initial weights.

#### Our contribution.

We propose incremental PINNs (iPINNs) and implement this strategy by creating
one subnetwork per task such that a complete neural network can learn multiple
tasks. Each subnetwork $\mathcal{N}_{i}$ has its own set of parameters
$\theta_{i}\subset\theta$, and the model is trained sequentially on different
tasks. A subnetwork for a new task can overlap with all previous subnetworks,
which helps to assimilate the new task. As a result, the network consists of
overlapping subnetworks, while the free parameters can be used for future
tasks. To illustrate the benefits of the algorithm we consider two problem
formulations (Section 3). Firstly, we learn a family of equations (e.g.,
convection) starting from a simple one and incrementally learning new
equations from that family. Secondly, we learn a dynamical system that
consists of two processes (e.g., reaction-diffusion) by first learning the
individual components of the process. Both scenarios demonstrate that the
incremental approach enables an iPINN network to learn for cases where regular
PINNs fail. To the best of our knowledge, this is the first example where one
network can sequentially learn multiple equations without extending its
architecture, with the added benefit that performance is significantly
improved.

##  2 Related work

Our methodology is based on creating sparse network representations and,
similarly to other PINN research, is sensitive to the choice of activation
functions. We briefly highlight key related work herein.

#### Sparse network representation.

Sparse architectures are often advantageous compared to dense ones [19, 1, 64,
35]. According to the lottery ticket hypothesis (LTH) [16], every randomly
initialized network contains a subnetwork that can be trained in isolation to
achieve comparable performance as the original network. Based on this
observation, the idea of using subnetworks has been adopted in continual
learning [39, 53, 54]. In this paradigm, every subnetwork created is
associated with a particular task and used only for this task to make a
prediction. One of the approaches to find these tasks-related subnetworks is
connections’ pruning [33, 21, 20, 15, 14] that removes unimportant parameters
while exhibiting similar performance.

#### Choice of the activation function.

There are several studies that investigate how different activation functions
affect the performance of neural networks in classification and regression
tasks [56, 25]. It was shown that ReLU [44] activation function which can be
powerful in classification tasks, in the case of physics-informed machine
learning (PIML) regression, may not be the optimal choice. Meanwhile,
hyperbolic tangent (tanh) or sine (sin) perform well for PIML. Sinusoidal
representation networks (SIRENs) [52] tackle the problem of modeling the
signal with fine details. Special weights initialization scheme combined with
sin activation function allows SIREN to learn complex natural signals. Hence,
we use sin activation function in our experiments. In Section 6.1, we provide
the comparison in results between the discussed activation functions.

##  3 Problem formulation

We focus on two scenarios: (1) incremental PINNs learning, where the network
sequentially learns several equations from the same family; and (2) learning a
combination of multiple equations that create another physical process. To
illustrate these cases, we consider one-dimensional convection, reaction and
reaction-diffusion problems with periodic boundary conditions.

###  3.1 Scenario 1: Equation incremental learning

We consider the problem of learning the sequence of equations that belong to
one family:

$\mathcal{F}_{k}(u(x,t))=0,\quad x\in\mathrm{\Omega},\ t\in[t_{0},T],\
k=1,2,\ldots,K,$ (P1)

where $\\{\mathcal{F}_{k}\\}_{k=1}^{K}$ are differential operators from the
same family of equations.

1-D convection equation $\displaystyle\frac{\partial u}{\partial
t}+\beta_{k}\frac{\partial u}{\partial x}$ $\displaystyle=0,$ (P1.1)
$\displaystyle u(x,0)$ $\displaystyle=h_{1}(x),$ $\displaystyle u(0,t)$
$\displaystyle=u(2\pi,t),$ where $t\in[0,1],\ x\in[0,2\pi],\
\beta_{k}\in\mathcal{B}\subset\mathbb{N}$. 1-D reaction equation
$\displaystyle\frac{\partial u}{\partial t}-\rho_{k}u(1-u)$ $\displaystyle=0,$
(P1.2) $\displaystyle u(x,0)$ $\displaystyle=h_{2}(x),$ $\displaystyle u(0,t)$
$\displaystyle=u(2\pi,t),$ where $t\in[0,1],\ x\in[0,2\pi],\
\rho_{k}\in\mathcal{R}\subset\mathbb{N}$.

In this case, every task $k$ is associated with $\mathcal{D}_{k}=\\{(x,t,k):\
x\in[0,2\pi],\ t\in[t_{0},T],\ k\in\mathbb{N}\\}$. Following [30], we take
$h_{1}(x)=\sin{x}$ and $h_{2}(x)=e^{-\frac{(x-\pi)^{2}}{2(\pi/4)^{2}}}$.

###  3.2 Scenario 2: Combination of multiple equations

We also consider the case when a dynamic process consists of multiple
components. Let us consider the reaction-diffusion equation:

$\displaystyle\frac{\partial u}{\partial t}-\nu\frac{\partial^{2}u}{\partial
x^{2}}-\rho u(1-u)$ $\displaystyle=0,$ (P2) $\displaystyle u(x,0)$
$\displaystyle=h_{2}(x),$ $\displaystyle u(0,t)$ $\displaystyle=u(2\pi,t),$

where $t\in[0,1],\ x\in[0,2\pi],\ \nu,\rho>0$. This process consists of two
parts: reaction term $(\nu=0)$: $-\rho u(1-u)$ and diffusion term $(\rho=0)$:
$-\nu\frac{\partial^{2}u}{\partial x^{2}}$. Therefore, we construct one task
as the reaction, another one as the diffusion, and the final one as the
reaction-diffusion. We can change the order of the reaction tasks and
diffusion tasks to show the robustness of incremental learning. The reaction-
diffusion task should be the last one since our goal is first to learn the
components of the system and only then the full system.

Considering these two problems, we want to show that better generalization can
be achieved by pretraining the network with simpler related problems rather
than by dividing the domain into smaller subdomains. In the following section,
we show how one network can incrementally learn different equations without
catastrophic forgetting.

##  4 Methodology

Figure 2: An example of iPINNs with two PDEs: every subnetwork corresponds to
only one task (PDE).

The proposed method needs to be applicable to both types of problems P1 and
P2. However, these problems cannot be solved by one network with the same
output head for all $K$ tasks, since
$\mathcal{F}_{i}(u(x,t))\neq\mathcal{F}_{j}(u(x,t))$ for $i\neq j$ and
$x\in\mathrm{\Omega},\ t\in[t_{0},T]$. Therefore, we propose iPINNs – an
incremental learning algorithm that focuses on learning task-specific
subnetworks $\mathcal{N}_{1},\mathcal{N}_{2},...,\mathcal{N}_{K}$ for each
task $k=1,2,\ldots,K$. To create these subnetworks, we use an iterative
pruning algorithm NNrelief [14]. This pruning approach uses input data to
estimate the contribution of every connection to the neuron in the pretrained
network and delete the least important ones. However, in principle, any
connections pruning algorithm or any other approach that is able to find and
train sparse network representations is suitable.

iPINN trains task-related subnetworks with pruning, allowing the subnetworks
to overlap on some connections. This way the method provides knowledge sharing
between the subnetworks. These overlaps are updated with respect to all tasks
that are assigned to a particular connection. Let us denote the loss of each
task $\mathcal{D}_{j}$ as
$\mathcal{L}_{j}=\mathcal{L}(\theta_{j};\mathcal{D}_{j})$, where $\theta_{j}$
is the parameter vector for task $\mathcal{D}_{j}$, $1\leq j\leq k$. Then the
total loss and its gradient with respect to a parameter $w$ can be written as:

$\displaystyle\mathcal{L}$ $\displaystyle=\sum_{j=1}^{k}\mathcal{L}_{j},$ (8)
$\displaystyle\frac{\partial\mathcal{L}}{\partial w}$
$\displaystyle=\sum_{j=1}^{k}\frac{\partial\mathcal{L}_{j}}{\partial
w}=\sum_{j:\ w\in\mathcal{N}_{j}}\frac{\partial\mathcal{L}_{j}}{\partial w},$
(9)

because if $w\not\in\mathcal{N}_{j}$ , then
$\frac{\partial\mathcal{L}_{j}}{\partial w}=0$. The pseudocode of the
algorithm is shown as follows:

1:neural network $\mathcal{N}$, training datasets $\mathcal{D}_{k}\
(k=1,2,\dots,K)$, training hyperparameters, pruning hyperparameters
($num\\_iters$).

2:for $k=1,2,\ldots,K$ do

3: $\mathcal{N}_{k}\leftarrow\mathcal{N}$ $\triangleright$ set full network as
a subnetwork

4: Train $\mathcal{N}_{1},\mathcal{N}_{2},\ldots,\mathcal{N}_{k}$ on tasks
$\mathcal{D}_{1},\mathcal{D}_{2},\ldots,\mathcal{D}_{k}$ using Eq. 9.

5: for $it=1,2,\ldots,num\\_iters$ do $\triangleright$ repeat pruning

6: $\mathcal{N}_{k}\leftarrow Pruning(\mathcal{N}_{k},\mathcal{D}_{k})$
$\triangleright$ reduce unimportant connections

7: Retrain subnetworks
$\mathcal{N}_{1},\mathcal{N}_{2},\ldots,\mathcal{N}_{k}$ on tasks
$\mathcal{D}_{1},\mathcal{D}_{2},\ldots,\mathcal{D}_{k}$ using Eq. 9.

8: end for

9:end for

Algorithm 1 PINN incremental learning

The main advantage of the proposed approach is that a neural network learns
all tasks (subdomains or equations) that were given during training and not
only the last one. This is achieved by constantly replaying old data. In the
next section, we experimentally show that pretrained parts of the network help
to improve the convergence process.

##  5 Numerical experiments

Our findings illustrate the advantage of the Algorithm over regular PINNs
[46]. The Algorithm allows the network to learn multiple equations (P1) from
the same family. Furthermore, by starting with simpler tasks, the network can
learn more complex ones that cannot be learned separately.

#### Experiments setup.

Let us start by examining the proposed algorithms on the convection and
reaction equations with periodic boundary conditions (P1). Following the setup
in [30], we use a four-layer neural network with 50 neurons per layer. We use
1000 randomly selected collocation points on every time interval between 0 and
1 for $\mathcal{L}_{\mathcal{F}}$. The Adam optimizer [29] is used to train
the model.

To evaluate the performance of the algorithms we compare the final error after
the last task. In addition, following continual learning literature [36], we
compare backward and forward transfer metrics. Let us denote the test set as
$\mathcal{D}^{test}=\\{(x^{i},t^{i},l):x^{i}\in[0,2\pi],\ t^{i}\in[0,1],\
l\text{ is the task-ID}\\}$, the solution of the equation at the point
$(x^{i},t^{i},l)$ as $\mathbf{u}_{l,k}^{i}=u_{l,k}^{i}(x^{i},t^{i})$, and
$\mathbf{\hat{u}}_{l,k}^{i}$ is a prediction of the model at point
$(x^{i},t^{i},l)$ after task $\mathcal{D}_{k}$ is learned. Relative and
absolute errors are denoted as $r_{l,k}$ and $\varepsilon_{l,k}$,
respectively, as they are calculated for task $l$ after task $k$ is learned
($l\leq k$).

$\displaystyle\text{Relative error:}\quad r_{l,k}$
$\displaystyle=\frac{1}{N}\frac{\lvert\lvert\mathbf{u}_{l}-\mathbf{\hat{u}}_{l,k}\rvert\rvert_{2}}{\lvert\lvert\mathbf{u}_{l}\rvert\rvert_{2}}\times
100\%,$ (10) $\displaystyle\text{Absolute error:}\quad\varepsilon_{l,k}$
$\displaystyle=\frac{1}{N}\sum_{i=1}^{N}\lvert\mathbf{u}_{l}^{i}-\mathbf{\hat{u}}_{l,k}^{i}\rvert,$
(11) Backward Transfer:BWT
$\displaystyle=\frac{1}{k-1}\sum_{l=1}^{k-1}\varepsilon_{l,k}-\varepsilon_{l,l}\
\text{ or }$ (12) BWT
$\displaystyle=\frac{1}{k-1}\sum_{l=1}^{k-1}r_{l,k}-r_{l,l}$ (13)

###  5.1 Results

Table 1 presents the results after all reaction equations are learned varying
$\rho$ from 1 to 5. Figure 3 shows the error history for every equation after
incremental steps. The Table summarizes the performance improvement of iPINNs
compared to regular PINNs, exhibiting negligible error for all values of
$\rho$, which is especially relevant for cases when $\rho$ is larger.
Moreover, iPINNs provide negative BWT which means that previous subnetworks
help to learn the following ones.

Table 1: Final error and forgetting after all reaction equations are learned. |  | regular PINN | iPINN
---|---|---|---
$\rho=1$ | abs. err | $1.09\times 10^{-3}$ | $\mathbf{1.5\times 10^{-4}}$
rel. err | 0.263% |  $\mathbf{0.039}$%
$\rho=2$ | abs. err | $1.97\times 10^{-3}$ | $\mathbf{2.5\times 10^{-4}}$
rel. err | 0.479% | $\mathbf{0.070\%}$
$\rho=3$ | abs. err | $6.72\times 10^{-3}$ | $\mathbf{6.1\times 10^{-4}}$
rel. err | 2.05% | $\mathbf{0.210}\%$
$\rho=4$ | abs. err | $1.13\times 10^{-2}$ | $\mathbf{1.18\times 10^{-3}}$
rel. err | 3.68% | $\mathbf{0.458\%}$
$\rho=5$ | abs. err | $5.04\times 10^{-2}$ | $\mathbf{1.91\times 10^{-3}}$
rel. err | 12.19% | $\mathbf{0.763\%}$
BWT | abs. err | N/A | -$3.8\times 10^{-4}$
rel. err | N/A | -0.112%

Figure 3: Relative error history for reaction eqations ($\alpha=0.99$). Every
row shows the error after a new task is learned.

Similarly, we observe for the convection equation the same learning behaviour.
By learning incrementally the sequence of convection equations, we achieve
much lower absolute and relative errors for the equations that are more
difficult to learn ($\beta=30,40$). In Table 2 we show final errors at the end
of training, and Figure 4 shows the absolute error history for each equation.

Table 2: Final error and forgetting after all convection equations are learned. |  | regular PINN | iPINN
---|---|---|---
$\beta=1$ | abs. err | $\mathbf{2.3\times 10^{-4}}$ | $4.2\times 10^{-4}$
rel. err | $\mathbf{0.042\%}$ | 0.074%
$\beta=10$ | abs. err | $1.3\times 10^{-3}$ | $\mathbf{5.0\times 10^{-4}}$
rel. err | 0.222% | $\mathbf{0.087\%}$
$\beta=20$ | abs. err | $1.9\times 10^{-3}$ | $\mathbf{1.66\times 10^{-3}}$
rel. err | 0.339% | $\mathbf{0.288\%}$
$\beta=30$ | abs. err | $2.2\times 10^{-1}$ | $\mathbf{1.44\times 10^{-3}}$
rel. err | 3.957% | $\mathbf{0.246\%}$
$\beta=40$ | abs. err | $2.3\times 10^{-1}$ | $\mathbf{6.02\times 10^{-3}}$
rel. err | 37.4% | $\mathbf{1.139\%}$
BWT | abs. err | N/A | $1.8\times 10^{-4}$
rel. err | N/A | 0.0280%

Figure 4: Relative error history for convection eqations ($\alpha=0.95$).
Every row shows the error after a new task is learned.

In Figures 5 and 6, we illustrate the error of iPINNs on convection and
reaction equations and the exact solutions for every value of parameter
$\beta$ or $\rho$ that were considered. Overall, we see that the neural
network learns more complicated tasks more accurately if parts of the network
are pretrained with easier tasks. At the same time, iPINNs replay the training
data for previous PDEs during training for the new one. There are no
additional costs to store or generate input points $(x,t)$ for previous tasks
since they can be easily sampled when necessary.

(a) 1-D reaction equation (absolute error).

(b) 1-D reaction equation (exact solution).

Figure 5: iPINNs on 1-D reaction equation.

(a) Absolute error.

(b) Exact solution.

Figure 6: iPINNs on 1-D convection equation.

Another illustration of the method is learning the problem P2. We consider the
values of $\rho$ and $\nu$ for which PINN does not have difficulties with
learning each component separately. Results obtained when first learning the
reaction part (diffusion part) are shown in Table 3 (Table 4). The main
finding is that the network can learn almost every equation at least as well
as when it is learned independently. In fact, for the reaction equation, the
neural network improves significantly the prediction error. Another
interesting observation is that the model learns the reaction-diffusion
equation with almost the same error, regardless of the order of the tasks.

Table 3: Final error and forgetting for reaction $\to$ diffusion $\to$ reaction-diffusion. parameters | equation |  | regular PINN | iPINN
---|---|---|---|---
$\rho=3,\ \nu=5$ | reaction | abs. err | $6.72\times 10^{-3}$ | $\mathbf{9.41\times 10^{-4}}$
rel. err | 2.05% | $\mathbf{0.31\%}$
diffusion | abs. err | $\mathbf{1.38\times 10^{-4}}$ | $1.85\times 10^{-4}$
rel. err | $\mathbf{0.05\%}$ | 0.06%
reaction-diffusion | abs. err | $4.89\times 10^{-3}$ | $\mathbf{4.10\times 10^{-3}}$
rel. err | 0.80% | $\mathbf{0.68\%}$
$\rho=4,\ \nu=4$ | reaction | abs. err | $1.13\times 10^{-2}$ | $\mathbf{7.88\times 10^{-3}}$
rel. err | 3.68% | $\mathbf{2.99\%}$
diffusion | abs. err | $\mathbf{4.35\times 10^{-4}}$ | $5.84\times 10^{-4}$
rel. err | $\mathbf{0.16\%}$ | 0.19%
reaction-diffusion | abs. err | $4.58\times 10^{-3}$ | $\mathbf{4.42\times 10^{-3}}$
rel. err | 0.70% | $\mathbf{0.67\%}$
$\rho=4,\ \nu=5$ | reaction | abs. err | $5.04\times 10^{-2}$ | $\mathbf{4.20\times 10^{-3}}$
rel. err | 12.19% | $\mathbf{1.71\%}$
diffusion | abs. err | $5.18\times 10^{-4}$ | $\mathbf{2.30\times 10^{-4}}$
rel. err | 0.18% | $\mathbf{0.08\%}$
reaction-diffusion | abs. err | $4.61\times 10^{-3}$ | $\mathbf{4.58\times 10^{-3}}$
rel. err | 0.69% | $\mathbf{0.68\%}$
Table 4: Final error and forgetting for diffusion $\to$ reaction $\to$ reaction-diffusion. parameters | equation |  | regular PINN | iPINN
---|---|---|---|---
$\rho=3,\ \nu=5$ | diffusion | abs. err | $\mathbf{1.38\times 10^{-4}}$ | $8.64\times 10^{-4}$
rel. err | $\mathbf{0.05\%}$ | 0.28%
reaction | abs. err | $6.72\times 10^{-3}$ | $\mathbf{2.11\times 10^{-3}}$
rel. err | 2.05% | $\mathbf{0.68\%}$
reaction-diffusion | abs. err | $4.89\times 10^{-3}$ | $\mathbf{4.07\times 10^{-3}}$
rel. err | 0.80% | $\mathbf{0.67\%}$
$\rho=4,\ \nu=4$ | diffusion | abs. err | $4.35\times 10^{-4}$ | $\mathbf{3.45\times 10^{-4}}$
rel. err | 0.16% | $\mathbf{0.12\%}$
reaction | abs. err | $1.13\times 10^{-2}$ | $\mathbf{4.91\times 10^{-3}}$
rel. err | 3.68% | $\mathbf{1.97\%}$
reaction-diffusion | abs. err | $4.58\times 10^{-3}$ | $\mathbf{4.42\times 10^{-3}}$
rel. err | 0.70% | $\mathbf{0.67\%}$
$\rho=4,\ \nu=5$ | diffusion | abs. err | $\mathbf{5.18\times 10^{-4}}$ | $1.05\times 10^{-3}$
rel. err | $\mathbf{0.18\%}$ | 0.33%
reaction | abs. err | $5.04\times 10^{-2}$ | $\mathbf{9.15\times 10^{-3}}$
rel. err | 12.19% | $\mathbf{3.39\%}$
reaction-diffusion | abs. err | $4.61\times 10^{-3}$ | $\mathbf{4.30\times 10^{-3}}$
rel. err | 0.69% | $\mathbf{0.65\%}$

##  6 Additional study

In this section, we provide additional information about the learning
procedure of iPINNs . We highlight some important training details such as the
presence of regularization and the choice of activation functions. Also, we
explore the subnetworks that our approach produces showing the proportion of
parameters allocated to each task.

###  6.1 Sensitivity to hyperparameters

Here we illustrate the influence of different training hyperparameters on the
performance of iPINNs. First, we compare the results with and without
regularization parameter (weight decay). In Figure 7, it can be observed that
the presence of weight decay worsens the prediction error. However, looking at
the result it is clear that iPINNs still work if weight decay is present. We
can explain the lack of need for weight decay with the fact that many
parameters are assigned to multiple tasks and cannot overfit to a particular
one. Each subnetwork is also less parameterized than the original network and
therefore does not tend to overfit. Thus, weight decay is not necessary and
its presence only worsens the result due to the complication of the
optimization procedure.

Figure 7: Influence of weight decay on the results for reaction (left) and
convection (right) equations after all tasks are learned.

Furthermore, we compare the performance when using sin and tanh activation
functions for two task orderings in Figure 8. We observe that sin works
significantly better in both cases. Also, we test ReLU activation but it
demonstrates poor performance in both PDEs orderings. If the reaction is
learned first, the absolute errors are $0.4959,0.2369$ and $0.1493$. If we
start with the diffusion equation and then learn reaction and reaction-
diffusion PDEs, the errors are $0.2399,0.2977$ and $0.3003$.

Figure 8: Influence of activation function on the results when the reaction
learned first (left) and diffusion learned first (right).

In addition, we present how different values of pruning parameter $\alpha$
affect the results. The higher the value of $\alpha$ is, the less the network
is pruned. Therefore, if $\alpha=0.95$ the task-specific subnetworks are
sparser than with $\alpha=0.99$ but less sparse if $\alpha=0.9$. In Figure 9,
we observe that for the reaction equation, we can prune less and achieve
better performance which can be explained by the fact that PDEs in the
reaction family are quite similar. Therefore, we can allow the network to have
more overlaps to share knowledge between subnetworks. For the case of learning
within the same family of convection PDEs, the value of $\alpha=0.95$ was
revealed to be a better option for constructing a sufficiently expressive
task-specific subnetwork and frees space for future tasks. Notwithstanding,
the performance is good with any reasonable choice of pruning parameter.

Figure 9: iPINNs with different values of pruning parameter $\alpha$.

###  6.2 Subnetworks analysis

In Figure 10, we present the portions of the subnetworks that are occupied by
each task. We will illustrate this by considering both orders – when the model
learns the reaction equation first (Figure 10(a)), and when diffusion comes
first (Figure 10(b)). These results are averaged over 3 different runs for
each of the orderings. It is noteworthy that the percentage of parameters
occupied by all tasks is very similar for both orderings (31.8% and 31.5%
respectively of all network parameters). On the other hand, the percentages of
used parameters for both cases are 79.5% and 79.3%. This means that the total
number of trained parameters for the two incremental procedures is the same
for both cases, which shows the robustness of the method. Moreover, the
network has about 20% of free connections to learn new tasks.

(a) reaction $\to$ diffusion $\to$ reaction-diffusion.

(b) diffusion $\to$ reaction $\to$ reaction-diffusion.

Figure 10: Percentage of parameters used for every equation with $\rho=4,\
\nu=4$.

##  7 Conclusion

In this work, we propose an incremental learning approach for PINNs where
every task is presented as a new PDE. Our algorithm is based on task-related
subnetworks for every task obtained by iterative pruning. To illustrate our
idea, we consider two cases when incremental learning is applicable to a
sequence of PDEs. In the first case, we consider the family of
convection/reaction PDEs, learning them sequentially. In the second example,
we consider the reaction-diffusion equation and learn firstly the components
of the process, namely reaction and diffusion, and only then the reaction-
diffusion equation. Our main goal is to show the possibility of incremental
learning for PINNs without significantly forgetting previous tasks. From our
numerical experiments, the proposed algorithm can learn all the given tasks,
which is not possible with standard PINNs. Importantly, we also show that
future tasks are learned better because they can share connections trained
from previous tasks, leading to significantly better performance than if these
tasks were learned independently. We demonstrate that this stems from the
transfer of knowledge occurring between subnetworks that are associated with
each task. Interestingly, the model’s performance on previous tasks is also
improved by learning the following tasks. In essence, iPINNs demonstrate
symbiotic training effects between past and future tasks by learning them with
a single network composed of dedicated subnetworks for each task that share
relevant neuronal connections.

## References

  * [1] Ahmad, S., Scheinkman, L.: How can we be so dense? the benefits of using highly sparse representations. arXiv preprint arXiv:1903.11257 (2019)
  * [2] Bai, Y., Chaolu, T., Bilige, S.: The application of improved physics-informed neural network (ipinn) method in finance. Nonlinear Dynamics 107(4), 3655–3667 (2022)
  * [3] Bengio, Y.: Deep learning of representations for unsupervised and transfer learning. In: Proceedings of ICML workshop on unsupervised and transfer learning. pp. 17–36. JMLR Workshop and Conference Proceedings (2012)
  * [4] Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: Proceedings of the 26th annual international conference on machine learning. pp. 41–48 (2009)
  * [5] Bessa, M.A., Bostanabad, R., Liu, Z., Hu, A., Apley, D.W., Brinson, C., Chen, W., Liu, W.K.: A framework for data-driven analysis of materials under uncertainty: Countering the curse of dimensionality. Computer Methods in Applied Mechanics and Engineering 320, 633–667 (2017)
  * [6] Bogatskiy, A., Anderson, B., Offermann, J., Roussi, M., Miller, D., Kondor, R.: Lorentz group equivariant neural network for particle physics. In: International Conference on Machine Learning. pp. 992–1002. PMLR (2020)
  * [7] Cai, S., Mao, Z., Wang, Z., Yin, M., Karniadakis, G.E.: Physics-informed neural networks (pinns) for fluid mechanics: A review. Acta Mechanica Sinica pp. 1–12 (2022)
  * [8] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European conference on computer vision (ECCV). pp. 233–248 (2018)
  * [9] Cermelli, F., Geraci, A., Fontanel, D., Caputo, B.: Modeling missing annotations for incremental learning in object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3700–3710 (2022)
  * [10] Chakraborty, S.: Transfer learning based multi-fidelity physics informed deep neural network. Journal of Computational Physics 426, 109942 (2021)
  * [11] Chandrasekhar, A., Suresh, K.: Tounn: topology optimization using neural networks. Structural and Multidisciplinary Optimization 63(3), 1135–1149 (2021)
  * [12] Chen, X., Gong, C., Wan, Q., Deng, L., Wan, Y., Liu, Y., Chen, B., Liu, J.: Transfer learning for deep neural network-based partial differential equations solving. Advances in Aerodynamics 3(1), 1–14 (2021)
  * [13] Chen, Y., Lu, L., Karniadakis, G.E., Dal Negro, L.: Physics-informed neural networks for inverse problems in nano-optics and metamaterials. Optics express 28(8), 11618–11633 (2020)
  * [14] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021)
  * [15] Dong, X., Chen, S., Pan, S.: Learning to prune deep neural networks via layer-wise optimal brain surgeon. Advances in Neural Information Processing Systems 30 (2017)
  * [16] Frankle, J., Carbin, M.: The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635 (2018)
  * [17] Gogas, P., Papadimitriou, T.: Machine learning in economics and finance. Computational Economics 57(1), 1–4 (2021)
  * [18] Goswami, S., Anitescu, C., Chakraborty, S., Rabczuk, T.: Transfer learning enhanced physics informed neural network for phase-field modeling of fracture. Theoretical and Applied Fracture Mechanics 106, 102447 (2020)
  * [19] Guo, Y., Zhang, C., Zhang, C., Chen, Y.: Sparse dnns with improved adversarial robustness. Advances in neural information processing systems 31 (2018)
  * [20] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015)
  * [21] Hassibi, B., Stork, D.: Second order derivatives for network pruning: Optimal brain surgeon. Advances in neural information processing systems 5 (1992)
  * [22] Hosaka, T.: Bankruptcy prediction using imaged financial ratios and convolutional neural networks. Expert systems with applications 117, 287–299 (2019)
  * [23] Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., Gelly, S.: Parameter-efficient transfer learning for nlp. In: International Conference on Machine Learning. pp. 2790–2799. PMLR (2019)
  * [24] Jagtap, A.D., Karniadakis, G.E.: Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. In: AAAI Spring Symposium: MLPS (2021)
  * [25] Jagtap, A.D., Karniadakis, G.E.: How important are activation functions in regression and classification? a survey, performance comparison, and future directions. arXiv preprint arXiv:2209.02681 (2022)
  * [26] Juan, N.P., Valdecantos, V.N.: Review of the application of artificial neural networks in ocean engineering. Ocean Engineering 259, 111947 (2022)
  * [27] Kang, M., Park, J., Han, B.: Class-incremental learning by knowledge distillation with adaptive feature consolidation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 16071–16080 (2022)
  * [28] Khatib, O., Ren, S., Malof, J., Padilla, W.J.: Learning the physics of all-dielectric metamaterials with deep lorentz neural networks. Advanced Optical Materials p. 2200097 (2022)
  * [29] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  * [30] Krishnapriyan, A., Gholami, A., Zhe, S., Kirby, R., Mahoney, M.W.: Characterizing possible failure modes in physics-informed neural networks. Advances in Neural Information Processing Systems 34, 26548–26560 (2021)
  * [31] Lagaris, I.E., Likas, A., Fotiadis, D.I.: Artificial neural network methods in quantum mechanics. Computer Physics Communications 104(1-3), 1–14 (1997)
  * [32] Lagaris, I.E., Likas, A., Fotiadis, D.I.: Artificial neural networks for solving ordinary and partial differential equations. IEEE transactions on neural networks 9(5), 987–1000 (1998)
  * [33] LeCun, Y., Denker, J., Solla, S.: Optimal brain damage. Advances in neural information processing systems 2 (1989)
  * [34] Lee, H., Kang, I.S.: Neural algorithm for solving differential equations. Journal of Computational Physics 91(1), 110–131 (1990)
  * [35] Liao, N., Wang, S., Xiang, L., Ye, N., Shao, S., Chu, P.: Achieving adversarial robustness via sparsity. Machine Learning 111(2), 685–711 (2022)
  * [36] Lopez-Paz, D., Ranzato, M.: Gradient episodic memory for continual learning. Advances in neural information processing systems 30 (2017)
  * [37] Lu, L., Pestourie, R., Yao, W., Wang, Z., Verdugo, F., Johnson, S.G.: Physics-informed neural networks with hard constraints for inverse design. SIAM Journal on Scientific Computing 43(6), B1105–B1132 (2021)
  * [38] Madrazo, C.F., Heredia, I., Lloret, L., de Lucas, J.M.: Application of a convolutional neural network for image classification for the analysis of collisions in high energy physics. In: EPJ Web of Conferences. vol. 214, p. 06017\. EDP Sciences (2019)
  * [39] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. pp. 7765–7773 (2018)
  * [40] Mao, Z., Jagtap, A.D., Karniadakis, G.E.: Physics-informed neural networks for high-speed flows. Computer Methods in Applied Mechanics and Engineering 360, 112789 (2020)
  * [41] Marques, G., Agarwal, D., de la Torre Díez, I.: Automated medical diagnosis of covid-19 through efficientnet convolutional neural network. Applied soft computing 96, 106691 (2020)
  * [42] Meade Jr, A.J., Fernandez, A.A.: Solution of nonlinear ordinary differential equations by feedforward neural networks. Mathematical and Computer Modelling 20(9), 19–44 (1994)
  * [43] Mojgani, R., Balajewicz, M., Hassanzadeh, P.: Lagrangian pinns: A causality-conforming solution to failure modes of physics-informed neural networks. arXiv preprint arXiv:2205.02902 (2022)
  * [44] Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: International Conference on Machine Learning (2010)
  * [45] Niaki, S.A., Haghighat, E., Campbell, T., Poursartip, A., Vaziri, R.: Physics-informed neural network for modelling the thermochemical curing process of composite-tool systems during manufacture. Computer Methods in Applied Mechanics and Engineering 384, 113959 (2021)
  * [46] Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics 378, 686–707 (2019)
  * [47] Rohrhofer, F.M., Posch, S., Gößnitzer, C., Geiger, B.C.: Understanding the difficulty of training physics-informed neural networks on dynamical systems. arXiv preprint arXiv:2203.13648 (2022)
  * [48] Ruder, S., Peters, M.E., Swayamdipta, S., Wolf, T.: Transfer learning in natural language processing. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: Tutorials. pp. 15–18 (2019)
  * [49] Sarvamangala, D., Kulkarni, R.V.: Convolutional neural networks in medical image understanding: a survey. Evolutionary intelligence 15(1), 1–22 (2022)
  * [50] Shlomi, J., Battaglia, P., Vlimant, J.R.: Graph neural networks in particle physics. Machine Learning: Science and Technology 2(2), 021001 (2020)
  * [51] Si, T., Bagchi, J., Miranda, P.B.: Artificial neural network training using metaheuristics for medical data classification: an experimental study. Expert Systems with Applications 193, 116423 (2022)
  * [52] Sitzmann, V., Martel, J., Bergman, A., Lindell, D., Wetzstein, G.: Implicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems 33, 7462–7473 (2020)
  * [53] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Spacenet: Make free space for continual learning. Neurocomputing 439, 1–11 (2021)
  * [54] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European conference on machine learning and knowledge discovery in databases. Springer (2022)
  * [55] Sosnovik, I., Oseledets, I.: Neural networks for topology optimization. Russian Journal of Numerical Analysis and Mathematical Modelling 34(4), 215–223 (2019)
  * [56] Szandała, T.: Review and comparison of commonly used activation functions for deep neural networks. In: Bio-inspired neurocomputing, pp. 203–224. Springer (2021)
  * [57] Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., Liu, C.: A survey on deep transfer learning. In: International conference on artificial neural networks. pp. 270–279. Springer (2018)
  * [58] Wang, J., Gan, X.: Neurodynamics-driven portfolio optimization with targeted performance criteria. Neural Networks 157, 404–421 (2023)
  * [59] Wang, S., Teng, Y., Perdikaris, P.: Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing 43(5), A3055–A3081 (2021)
  * [60] Wessels, H., Weißenfels, C., Wriggers, P.: The neural particle method–an updated lagrangian physics informed neural network for computational fluid dynamics. Computer Methods in Applied Mechanics and Engineering 368, 113127 (2020)
  * [61] Wiecha, P.R., Arbouet, A., Girard, C., Muskens, O.L.: Deep learning in nano-photonics: inverse design and beyond. Photonics Research 9(5), B182–B200 (2021)
  * [62] Wight, C.L., Zhao, J.: Solving allen-cahn and cahn-hilliard equations using the adaptive physics informed neural networks. arXiv preprint arXiv:2007.04542 (2020)
  * [63] Xu, C., Cao, B.T., Yuan, Y., Meschke, G.: Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios. Computer Methods in Applied Mechanics and Engineering 405, 115852 (2023)
  * [64] Ye, S., Xu, K., Liu, S., Cheng, H., Lambrechts, J.H., Zhang, H., Zhou, A., Ma, K., Wang, Y., Lin, X.: Adversarial robustness vs. model compression, or both? In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 111–120 (2019)
  * [65] Yentis, R., Zaghloul, M.: Vlsi implementation of locally connected neural network for solving partial differential equations. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications 43(8), 687–690 (1996)
  * [66] Yu, P., Yan, X.: Stock price prediction based on deep neural networks. Neural Computing and Applications 32(6), 1609–1628 (2020)
  * [67] Zou, Z., Karniadakis, G.E.: L-hydra: Multi-head physics-informed neural networks. arXiv preprint arXiv:2301.02152 (2023)