# Multitwists in big mapping class groups

George Domat<EMAIL_ADDRESS>Rice University, Department of
Mathematics, 6100 Main St.–MS 136, Houston, TX 77005, USA  ,  Federica Fanoni
<EMAIL_ADDRESS>CNRS, Univ Paris Est Creteil, Univ Gustave Eiffel,
LAMAR8050, F-94010 Creteil, France  and  Sebastian Hensel<EMAIL_ADDRESS>Mathematisches Institut der Universität München, Theresienstr. 39, 80333
München, Germany

###### Abstract.

We show that the group generated by multitwists (i.e. products of powers of
twists about disjoint non-accumulating curves) doesn’t contain the Torelli
group of an infinite-type surface. As a consequence, multitwists don’t
generate the closure of the compactly supported mapping class group of a
surface of infinite type.

##  1\. Introduction

The mapping class group of a surface of finite type has been thoroughly
studied for decades. In particular, multiple _simple_ sets of generators are
known. The Dehn–Lickorish theorem ([Deh38], [Lic64]), in combination with the
Birman exact sequence ([Bir69]), shows that the pure mapping class group of a
finite-type surface can be generated by finitely many Dehn twists about
nonseparating curves, and we need to add finitely many half-twists to generate
the full mapping class group. Humphries [Hum79] proved that, if the surface is
closed and of genus $g\geq 2$, $2g+1$ Dehn twists about nonseparating curves
suffice to generate the mapping class group, and moreover this number is
optimal: fewer than $2g+1$ Dehn twists cannot generate. Other results show
that mapping class groups can be generated by two elements (see e.g. [Waj96]),
by finitely many involutions or by finitely many torsion elements (see e.g.
[BF04]).

In the case of surfaces of infinite type, the (pure) mapping class group is
uncountable, so in particular it is not finitely (nor countably) generated.
For some of these surfaces the mapping class group is generated by torsion
elements, or even by involutions (see [MT21] and [CC22]), while for other
surfaces they aren’t (see [Dom22], [MT21], [CC22]). To the best of our
knowledge, no other generating set is known.

Note that the (pure) mapping class group of a surface of infinite type is
endowed with an interesting topology, induced by the compact-open topology on
the group of homeomorphisms of the surface. So it is interesting to talk about
_topological_ generating sets (sets whose _closure_ of the group they generate
is the (pure) mapping class group). It follows from the finite-type results
that Dehn twists topologically generate the closure of the compactly supported
mapping class group. Moreover, Patel and Vlamis [PV18] proved that the pure
mapping class group of a surface is topologically generated by Dehn twists if
the surface has at most one nonplanar end, and by Dehn twists and maps called
_handle shifts_ otherwise.

The goal of this note is to investigate a natural candidate for a set of
generators of the closure of the compactly supported mapping class group of a
surface: the collection of _multitwists_. A multitwist is a (possibly
infinite) product of powers of Dehn twists about a collection of simple closed
curves that do not accumulate anywhere in the surface. Our main result is a
negative one, and it follows from a non-generation result for the Torelli
group:

######  Theorem A.

Let $S$ be an infinite-type surface. Then the subgroup of the mapping class
group of $S$ generated by multitwists doesn’t contain the Torelli group. In
particular, multitwists don’t generate the closure of the compactly supported
mapping class group.

The idea of the proof is to produce an explicit element in the Torelli group
that is not in the subgroup generated by multitwists. This element is built by
taking an infinite product of increasing powers of partial pseudo-Anosov
homeomorphisms supported on disjoint finite-type subsurfaces. We use work of
Bestvina, Bromberg and Fujiwara [BBF16] to certify that the mapping class we
construct is not in the subgroup generated by multitwists.

Theorem A also begs the following question:

######  Question B.

What is the subgroup generated by the collection of multitwists? Is there an
alternative, more explicit description of its elements?

Furthermore, our theorem shows that the subgroup generated by the collection
of multitwists is not a closed subgroup of the mapping class group. Therefore,
it does not immediately inherit a Polish topology from the topology on the
mapping class group.

######  Question C.

Is the subgroup generated by the collection of multitwists a Polish group?

### Acknowledgements

The authors would like to thank Mladen Bestvina for his suggestion of how to
remove an unnecessary assumption in the main theorem. They are also grateful
to the organizers of the _Big Mapping Class Groups and Diffeomorphism Groups_
conference, during which most of the work was done. They thank the referees
for their useful comments. The second author thanks Peter Feller for useful
conversations.

The first author was supported in part by the Fields Institute for Research in
Mathematical Sciences and NSF RTG–1745670.

##  2\. Preliminaries

In this note, a surface is a connected, orientable, Hausdorff, second
countable two-dimensional manifold, without boundary unless otherwise stated.
One notable exception is any subsurface, which will always have compact
boundary. Boundary components of subsurfaces are assumed to be homotopically
nontrivial, but are allowed to be homotopic to a puncture.

Surfaces are _of finite type_ if their fundamental groups are finitely
generated and _of infinite type_ otherwise. A surface $S$ is _exceptional_ if
it has genus zero and at most four punctures or genus one and at most one
puncture, otherwise it is _nonexceptional_.

The _mapping class group_ of a surface $S$ is the group
$\operatorname{MCG}(S)$ of orientation preserving homeomorphisms of $S$ up to
homotopy. The _pure mapping class group_ $\operatorname{PMCG}(S)$ is the
subgroup of $\operatorname{MCG}(S)$ fixing all ends and —if there are any —
boundary components, and $\overline{\operatorname{MCG}_{c}(S)}$ denotes the
closure of the subgroup generated by compactly supported mapping classes. The
_Torelli group_ $\operatorname{\mathcal{I}}(S)$ is the subgroup of the mapping
class group given by elements acting trivially on the first homology group of
the surface.

A pseudo-Anosov mapping class $f$ of a finite-type surface is _chiral_ if
$f^{k}$ is not conjugate to $f^{-k}$ for every $k\neq 0$. Note that it follows
from [BBF16] that every pseudo-Anosov element in the Torelli group is chiral:
indeed, by [BBF16, Theorem 5.6], every nontrivial element in the Torelli group
has positive stable commutator length, and by [BBF16, Theorem 4.2] any pseudo-
Anosov element with positive stable commutator length is chiral.

A _curve_ on a surface is the homotopy class of an essential (i.e. not
homotopic to a point, a puncture or a boundary component) simple closed curve.
Given a curve $\alpha$, we denote by $\tau_{\alpha}$ the Dehn twist about
$\alpha$.

An _integral weighted multicurve_ $\mu$ is a formal sum $\sum_{i\in
I}n_{i}\alpha_{i}$, where the $\alpha_{i}$ are pairwise disjoint curves not
accumulating anywhere and the $n_{i}$ are integers. Given an integral weighted
multicurve $\mu$, we define $\tau_{\mu}$ to be the mapping class

$\tau_{\mu}=\prod_{i\in I}\tau_{\alpha_{i}}^{n_{i}}.$

Such a mapping class is called a _multitwist_.

We say that an integral weighted multicurve is _finite_ if $I$ is finite (i.e.
it contains finitely many curves). An integral weighted multicurve $\nu$ is a
_submulticurve_ of an integral weighted multicurve $\mu=\sum_{i\in
I}n_{i}\alpha_{i}$ if $\nu=\sum_{i\in J}n_{i}\alpha_{i}$, where $J\subset I$.

Given a surface with boundary, an _arc_ is the homotopy class (relative to the
boundary) of a simple arc that cannot be homotoped into the boundary. We
denote by $\mathcal{C}(S)$ the _curve and arc graph_ of a surface $S$, where
vertices are curves and, if $\partial S\neq\emptyset$, arcs, and two vertices
are adjacent if they have disjoint representatives.

For any two subsurfaces $A$ and $B$ of $S$ that have an essential
intersection, the _subsurface projection_ of $B$ to $A$ is the subset
$\partial B\cap A\subset\mathcal{C}(A)$. This projection is denoted by
$\pi_{A}(B)$. For any $\beta\in\mathcal{C}(B)$ we also define
$\pi_{A}(\beta):=\pi_{A}(B)$. These projections always have bounded diameter
[MM00] and given any intersecting subsurfaces $A,B,C\subset S$ we define the
_projection distance_ as:

$\displaystyle
d_{A}(B,C):=\operatorname{diam}_{\mathcal{C}(A)}(\pi_{A}(B)\cup\pi_{A}(C)).$

For a subgroup $G<\operatorname{MCG}(S)$, we say that a subsurface, $K\subset
S$, is _$G$ -nondisplaceable_ if $gK$ and $K$ cannot be homotoped to be
disjoint for any $g\in G$. Note that if a subsurface $K$ is
$G$-nondisplaceable, subsurface projections are always defined between
$G$-translates of $K$.

##  3\. Proof of Theorem A

Fix an infinite-type surface $S$, different from the Loch Ness monster.

Figure 1 shows some examples of
$\overline{\operatorname{MCG}_{c}(S)}$-nondisplaceable subsurfaces that will
be used in the following lemma.

######  Lemma 1.

If $S$ is an infinite-type surface, different from the Loch Ness monster, it
contains infinitely many finite-type
$\overline{\operatorname{MCG}_{c}(S)}$-nondisplaceable subsurfaces, which are
pairwise disjoint, pairwise homeomorphic and non-accumulating. Moreover, the
subsurfaces can be chosen to be nonexceptional.

###### Proof.

The proof is a case-by-case analysis. In each case we describe a finite-type
$\overline{\operatorname{MCG}_{c}(S)}$-nondisplaceable subsurface such that we
can clearly find infinitely many copies with the required properties.

Case 1: $S$ is the once-punctured Loch Ness monster. Then note that any
separating curve $\alpha$ which separates the two ends cannot be mapped
disjointly from itself by any mapping class (because it bounds a
nondisplaceable subsurface). As a consequence, for any $g\geq 1$, any
genus-$g$ subsurface with two boundary components separating the two ends is
nondisplaceable.

Case 2: $S$ has at least two nonplanar ends. Note that by the argument in
[PV18, Proposition 6.3], any separating curve such that both complementary
components have infinite genus is
$\overline{\operatorname{MCG}_{c}(S)}$-nondisplaceable.

  * •

If $S$ has at least one nonplanar end — denoted $e$ — which is isolated in
$\operatorname{Ends}(S)$, for any $g\geq 1$, any genus-$g$ subsurface with two
separating boundary components, each of which cuts off a surface containing
only the end $e$, is $\overline{\operatorname{MCG}_{c}(S)}$-nondisplaceable.

  * •

If $S$ has at least one nonplanar end — denoted $e$ — which is isolated in
$\operatorname{Ends}_{g}(S)$ but not in $\operatorname{Ends}(S)$, for any
$g\geq 1$, any genus-$g$ subsurface with three separating boundary components,
two of which cut off a subsurface whose only nonplanar end is $e$ and the
third one cuts off a planar surface, is
$\overline{\operatorname{MCG}_{c}(S)}$-nondisplaceable.

  * •

If no nonplanar end is isolated in $\operatorname{Ends}_{g}(S)$,
$\operatorname{Ends}_{g}(S)$ is a Cantor set. If it contains an end $e$ that
is not accumulated by planar ends, we choose a genus-$g$ subsurface with two
separating boundary components and no planar ends, so that each complementary
component has infinite genus, for $g\geq 1$. Otherwise, we choose a genus-$g$
subsurface with three separating boundary components, so that two
complementary components have infinite genus and one is a planar subsurface,
for $g\geq 1$.

Case 3: $S$ has no nonplanar ends. We can then choose any $n$-holed sphere
whose boundary curves are separating, so that there is at least one end in
each complementary component, for $n\geq 5$. ∎

Figure 1. The subsurfaces of Lemma 1

Fix a finite-type $\overline{\operatorname{MCG}_{c}(S)}$–nondisplaceable
subsurface $\Sigma\subset S$ and let $\mathcal{Y}$ be the
$\overline{\operatorname{MCG}_{c}(S)}$-orbit of $\Sigma$. As $\Sigma$ is
$\overline{\operatorname{MCG}_{c}(S)}$-nondisplaceable, any two surfaces in
$\mathcal{Y}$ have intersecting boundaries — in particular, subsurface
projections $\pi_{A}$ between surfaces in $\mathcal{Y}$ are always defined.
Moreover, by [Beh06] and [MM00], there is some constant $\mu>0$ so that for
every $A,B,C\in\mathcal{Y}$:

  * •

at most one of $d_{A}(B,C)$, $d_{B}(A,C)$ and $d_{C}(A,B)$ is bigger than
$\mu$, and

  * •

$|\\{D\in\mathcal{Y}\;|\;d_{D}(A,B)>\mu\\}|$ is finite.

See [Dom22, Lemma 3.8] for details on checking these in the infinite-type
case. We can therefore run the projection complex machinery to deduce (see
[BBF16, Proposition 2.7]):

######  Proposition 2.

$\overline{\operatorname{MCG}_{c}(S)}$ acts by isometries on a hyperbolic
graph $\mathcal{C}(\mathcal{Y})$ so that for every $A,B\in\mathcal{Y}$, $A\neq
B$:

  1. (1)

$\mathcal{C}(A)$ is isometrically embedded as a convex set in
$\mathcal{C}(\mathcal{Y})$ and the images of $\mathcal{C}(A)$ and
$\mathcal{C}(B)$ are disjoint;

  2. (2)

the inclusion

$\bigsqcup_{C\in\mathcal{Y}}\mathcal{C}(Y)\hookrightarrow\mathcal{C}(\mathcal{Y})$

is $\overline{\operatorname{MCG}_{c}(S)}$-equivariant;

  3. (3)

the nearest point projection to $\mathcal{C}(A)$ sends $\mathcal{C}(B)$ to a
bounded set, which is at uniformly bounded distance from $\pi_{A}(B)$;

  4. (4)

if $g\in\overline{\operatorname{MCG}_{c}(S)}$ is supported on $A$ and the
restriction is pseudo-Anosov, and $\Gamma$ is the subgroup of
$\overline{\operatorname{MCG}_{c}(S)}$ given by elements leaving $A$ invariant
and preserving the stable and unstable foliations of $g$, then
$(\overline{\operatorname{MCG}_{c}(S)},\mathcal{C}(\mathcal{Y}),g,\Gamma)$
satisfies WWPD.

Furthermore, the same proof as [BBF16, Lemma 2.8] yields:

######  Lemma 3.

Let $\tau$ be a multitwist about a finite multicurve $\mu$. Then for every
$A\in\mathcal{Y}$ there is a vertex $v_{A}$ of $\mathcal{C}(\mathcal{Y})$ such
that the nearest point projection to $\mathcal{C}(A)$ of the $\tau$ orbit of
$v_{A}$ is uniformly bounded. In particular, if $\tau$ is hyperbolic, its
virtual quasi-axis can intersect $\mathcal{C}(A)$ only in a bounded length
segment.

###### Proof.

If $\mu\cap A=\emptyset$, $\tau$ fixes any element of $\mathcal{C}(A)$, so
it’s elliptic. Otherwise, let $v_{A}$ be an element of
$A\cap\mu\neq\emptyset$. Then the nearest point projection of
$\tau^{n}(v_{A})$ to $\mathcal{C}(A)$ is a uniformly bounded distance from
$\pi_{A}(\tau^{n}(v_{A}))$, which is defined to be
$\pi_{A}(\partial(\tau^{n}(A))$. But this is at bounded distance from
$\tau^{n}(\mu)\cap A=\mu\cap A$, so the projection of $\tau^{n}(v_{A})$ is at
uniformly bounded distance from $A\cap\mu$ for every $n$. This proves the
first statement of the lemma. The second statement follows as in the proof of
[BBF16, Lemma 2.8]. ∎

As a consequence, we can apply [BBF16, Proposition 3.1] to deduce:

######  Proposition 4.

Let $\Sigma$ be a $\overline{\operatorname{MCG}_{c}(S)}$-nondisplaceable
subsurface of finite type and $f$ a mapping class that is a pure chiral
pseudo-Anosov mapping class of $\Sigma$ of sufficiently large translation
length and the identity on the complement. Then there is a homogeneous
quasimorphism $\varphi:\overline{\operatorname{MCG}_{c}(S)}\to\mathbb{R}$ of
defect $\Delta$ such that $|\varphi(f^{n})|\to\infty$ and
$|\varphi(\tau)|\leq\Delta$ for every multitwist $\tau$.

###### Proof.

The only thing we need to check is that $|\varphi(\tau)|\leq\Delta$ for every
multitwist $\tau$. But a multitwist $\tau$ associated to an integral weighted
multicurve $\mu=\sum_{i\in I}n_{i}\alpha_{i}$ can be written as a product of
two multitwists, $\tau_{1}$ and $\tau_{2}$, where $\tau_{1}$ is associated to
the integral weighted multicurve

$\mu_{1}=\sum_{i:\alpha_{i}\cap\Sigma\neq\emptyset}n_{i}\alpha_{i}$

and $\tau_{2}$ to the integral weighted multicurve

$\mu_{2}=\sum_{i:\alpha_{i}\cap\Sigma=\emptyset}n_{i}\alpha_{i}.$

Then $\tau_{2}$ acts elliptically on $\mathcal{C}(\mathcal{Y})$, for
$\mathcal{Y}=\overline{\operatorname{MCG}_{c}(S)}\cdot\Sigma$, so
$\varphi(\tau_{2})=0$. By Lemma 3, if $\tau_{1}$ doesn’t act elliptically on
$\mathcal{C}(\mathcal{Y})$, its virtual quasi-axis has small projections.
Thus, by how the quasimorphism $\phi$ is constructed in [BBF16, Proposition
3.1], if the translation length of $f$ is larger than the projection bound
from Lemma 3, we have $\varphi(\tau_{1})=0$. As a consequence
$|\varphi(\tau)|\leq\Delta$. ∎

###### Proof of Theorem A.

Suppose first that $S$ is not the Loch Ness monster. We will construct an
element $F$ of $\operatorname{\mathcal{I}}(S)$ which is not a finite product
of multitwists. Since $F$ is in $\overline{\operatorname{MCG}_{c}(S)}$ (by
construction, or by the fact that
$\operatorname{\mathcal{I}}(S)\subset\overline{\operatorname{MCG}_{c}(S)}$ by
[AGK+19]), this will also show that multitwists don’t generate the closure of
the compactly supported mapping class group. By Lemma 1, we can find pairwise
disjoint and nonaccumulating finite-type subsurfaces $\Sigma_{n}$, all
pairwise homeomorphic and
$\overline{\operatorname{MCG}_{c}(S)}$-nondisplaceable. Fix $\Sigma$ a surface
homeomorphic to the $\Sigma_{n}$ and $\theta_{n}:\Sigma\to\Sigma_{n}$ a
homeomorphism. Choose a pure chiral pseudo-Anosov mapping class $f$ in the
Torelli group of $\Sigma$, and let $F_{n}$ be the mapping class of $S$ with
support on $\Sigma_{n}$ and so that $F_{n}|_{\Sigma_{n}}=\theta_{n}\circ
f\circ\theta_{n}^{-1}$. Then for any $n$, by Proposition 4 (after potentially
passing to a power in order to increase the translation length), we can find a
homogeneous quasimorphism
$\varphi_{n}:\overline{\operatorname{MCG}_{c}(S)}\to\mathbb{R}$ with defect
$\Delta$ (independent of $n$) that is unbounded on powers of $F_{n}$ and
bounded by $\Delta$ on all multitwists or elements acting elliptically on
$\mathcal{C}(\overline{\operatorname{MCG}_{c}(S)}\cdot\Sigma_{n})$. Choose
powers $k_{n}$ so that

$|\varphi_{n}(F_{n}^{k_{n}})|\to\infty,$

which exist because by assumption $|\varphi_{n}(F_{n})|>1$ for every $n$.
Define

$F=\prod_{n\in\mathbb{N}}F_{n}^{k_{n}}.$

For every $n$, $\prod_{m\neq n}F_{m}^{k_{m}}$ acts elliptically on
$\mathcal{C}(\overline{\operatorname{MCG}_{c}(S)}\cdot\Sigma_{n})$, so

$|\varphi_{n}(F)|=\left|\varphi_{n}\left(F_{n}^{k_{n}}\circ\prod_{m\neq
n}F_{m}^{k_{m}}\right)\right|\geq\left|\varphi_{n}\left(F_{n}^{k_{n}}\right)\right|-\Delta\to\infty.$

If $F$ were a product of $k$ multitwists $\tau_{1},\dots,\tau_{k}$, then for
any $n$,

$|\varphi_{n}(F)|=|\varphi_{n}(\tau_{k}\circ\dots\circ\tau_{1})|\leq
2k\Delta,$

which gives a contradiction.

Suppose now that $S$ is the Loch Ness monster and fix a point $x\in S$. By the
Birman exact sequence [Dom22, Appendix], the kernel of the surjection

$\operatorname{\mathcal{I}}(S\smallsetminus\\{x\\})\to\operatorname{\mathcal{I}}(S)$

is the fundamental group of $S$ and is therefore generated by twists. In
particular, if $\operatorname{\mathcal{I}}(S)$ is contained in the subgroup
generated by multitwists, so is the Torelli group of the once-punctured Loch
Ness monster, a contradiction. By [PV18],
$\operatorname{MCG}(S)=\overline{\operatorname{MCG}_{c}(S)}$ and
$\operatorname{MCG}(S\smallsetminus\\{x\\})=\overline{\operatorname{MCG}_{c}(S\smallsetminus\\{x\\})}$,
so the same argument applied to $\operatorname{MCG}(S)$ and
$\operatorname{MCG}(S\smallsetminus\\{x\\})$ proves the result for the closure
of the compactly supported mapping class group.

∎

## References

  * [AGK+19] Javier Aramayona, Tyrone Ghaswala, Autumn E. Kent, Alan McLeay, Jing Tao, and Rebecca R. Winarski.  Big Torelli groups: generation and commensuration.  Groups Geom. Dyn., 13(4):1373–1399, 2019.
  * [BBF16] Mladen Bestvina, Ken Bromberg, and Koji Fujiwara.  Stable commutator length on mapping class groups.  Ann. Inst. Fourier (Grenoble), 66(3):871–898, 2016.
  * [Beh06] Jason A. Behrstock.  Asymptotic geometry of the mapping class group and Teichmüller space.  Geom. Topol., 10:1523–1578, 2006.
  * [BF04] Tara E. Brendle and Benson Farb.  Every mapping class group is generated by 6 involutions.  J. Algebra, 278(1):187–198, 2004.
  * [Bir69] Joan S. Birman.  Mapping class groups and their relationship to braid groups.  Comm. Pure Appl. Math., 22:213–238, 1969.
  * [CC22] Danny Calegari and Lvzhou Chen.  Normal subgroups of big mapping class groups.  Trans. Amer. Math. Soc. Ser. B, 9:957–976, 2022.
  * [Deh38] M. Dehn.  Die Gruppe der Abbildungsklassen.  Acta Math., 69(1):135–206, 1938.  Das arithmetische Feld auf Flächen.
  * [Dom22] George Domat.  Big pure mapping class groups are never perfect.  MRL, 29(3):691–726, 2022.  Appendix with Ryan Dickmann.
  * [Hum79] Stephen P. Humphries.  Generators for the mapping class group.  In Topology of low-dimensional manifolds (Proc. Second Sussex Conf., Chelwood Gate, 1977), volume 722 of Lecture Notes in Math., pages 44–47. Springer, Berlin, 1979.
  * [Lic64] W. B. R. Lickorish.  A finite set of generators for the homeotopy group of a $2$-manifold.  Proc. Cambridge Philos. Soc., 60:769–778, 1964.
  * [MM00] H. A. Masur and Y. N. Minsky.  Geometry of the complex of curves. II. Hierarchical structure.  Geom. Funct. Anal., 10(4):902–974, 2000.
  * [MT21] Justin Malestein and Jing Tao.  Self-similar surfaces: involutions and perfection.  arXiv e-prints, page arXiv:2106.03681, June 2021.
  * [PV18] Priyam Patel and Nicholas G. Vlamis.  Algebraic and topological properties of big mapping class groups.  Algebr. Geom. Topol., 18(7):4109–4142, 2018.
  * [Waj96] Bronislaw Wajnryb.  Mapping class group of a surface is generated by two elements.  Topology, 35(2):377–383, 1996.


# Meta Learning of Interface Conditions for Multi-Domain Physics-Informed
Neural Networks

Shibo Li∗<EMAIL_ADDRESS>
School of Computing
University of Utah Michael Penwarden∗<EMAIL_ADDRESS>
Scientific Computing and Imaging Institute
University of Utah Robert M. Kirby<EMAIL_ADDRESS>
School of Computing, Scientific Computing and Imaging Institute
University of Utah Shandian Zhe<EMAIL_ADDRESS>
School of Computing
University of Utah

###### Abstract

Physics-informed neural networks (PINNs) are emerging as popular mesh-free
solvers for partial differential equations (PDEs). Recent extensions decompose
the domain, applying different PINNs to solve the equation in each subdomain
and aligning the solution at the interface of the subdomains. Hence, they can
further alleviate the problem complexity, reduce the computational cost, and
allow parallelization. However, the performance of the multi-domain PINNs is
sensitive to the choice of the interface conditions for solution alignment.
While quite a few conditions have been proposed, there is no suggestion about
how to select the conditions according to specific problems. To address this
gap, we propose META Learning of Interface Conditions (METALIC), a simple,
efficient yet powerful approach to dynamically determine the optimal interface
conditions for solving a family of parametric PDEs. Specifically, we develop
two contextual multi-arm bandit models. The first one applies to the entire
training procedure, and online updates a Gaussian process (GP) reward
surrogate that given the PDE parameters and interface conditions predicts the
solution error. The second one partitions the training into two stages, one is
the stochastic phase and the other deterministic phase; we update a GP
surrogate for each phase to enable different condition selections at the two
stages so as to further bolster the flexibility and performance. We have shown
the advantage of METALIC on four bench-mark PDE families.

††∗ Equal Contribution.

##  1 Introduction

Physics-informed neural networks (PINNs) (Raissi et al., 2019a) are emerging
as a popular mesh-free approach for solving partial differential equations
(PDEs). They have shown success in many scientific and engineering problems,
such as in bio-engineering (Sahli Costabal et al., 2020; Kissas et al., 2020),
fluids mechanics (Raissi et al., 2019b; Sun et al., 2020; Raissi et al.,
2020), and material design (Fang and Zhan, 2019; Liu and Wang, 2019). The
recent multi-domain extensions, e.g., (Jagtap et al., 2020; Jagtap and
Karniadakis, 2021), further extend PINNs with a divide-and-conquer strategy,
and have attracted considerable attention. Specifically, they decompose the
domain of interest into several subdomains, place a separate PINN to solve the
PDE at each subdomain, and then align the solutions at the interface of the
subdomains. In this way, the multi-domain PINNs can alleviate problem
complexity, adopt simpler architectures, reduce the training cost, and enable
parallel computation (Shukla et al., 2021).

However, the performance of multi-domain PINNs is sensitive to the choice of
interface conditions (or regularizers) that align the PINN solutions at
different subdomains. Quite a few interface conditions have been proposed,
such as those that encourage solution and residual continuity (Jagtap and
Karniadakis, 2021), flux conservation (Jagtap et al., 2020), gradient
continuity (De Ryck et al., 2022) and residual gradient continuity (Yu et al.,
2022). On one hand, naively combining all possible interface conditions does
not necessarily give the optimal performance; instead, it can complicate the
loss landscape (Krishnapriyan et al., 2021), making the optimization more
costly and challenging. On the other hand, the best interface conditions can
vary across problems, which are up to the problem properties. Currently, there
are no suggestions about how to select interface conditions according to
specific problems, bringing inconvenience and difficulties to the multi-domain
practice.

To address this issue, in this paper we propose METALIC, a simple, efficient
and powerful meta learning method that can dynamically determine the best
interface conditions for solving a family of parametric PDEs. Specifically, we
develop two contextual multi-arm bandit (MAB) models. We view the PDE
parameters as the context, sets of interface conditions as the arms, and the
solution accuracy as the reward. Our first model applies to the whole training
procedure, and online estimates a Gaussian process (GP) reward surrogate that
predicts the solution accuracy given the context and binary encoding of the
arm. We use Upper Confidence Bound (UCB) and Thompson sampling (TS) to fulfill
effective exploration and exploitation tradeoffs. Due to the online nature of
MAB, we can use the GP surrogate to select the optimal interface conditions
given new PDE parameters while continuously improving the surrogate. Our
second model partitions the training into two phases, the stochastic (e.g.,
ADAM) and deterministic (e.g., L-BFGS) phase. These two phases are often
sequentially combined in PINNs to achieve reliable performance. We learn a
separate contextual bandit, namely, a GP reward surrogate for each phase. In
this way, we can select the optimal interface conditions for different
training stages to enhance flexibility and to further improve performance. To
coordinate the two bandits to optimize the final accuracy, we append the loss
after the first phase into the context of the second bandit to predict the
best conditions for training continuation; we add a discounted reward of the
second bandit to the first one so as to integrate the influence of the first
bandit on the subsequent training into its reward model updates.

We examined METALIC in four equation families: the Poisson, advection,
reaction and Burger’s equations, with parameters in the equation or boundary
conditions. The evaluation of the online performance shows that the
accumulated solution error of METALIC grows sublinearly, which is much slower
than randomly selecting the arm at each play. The latter exhibits a linear
growth. The results demonstrate that with more plays, our method keeps
improving the reward surrogate while utilizing it to achieve promising
solution accuracy. We then conducted a static test on a set of new PDEs after
the online playing. With the interface conditions determined by our learned
surrogate, the multi-domain PINNs achieve solution errors one order of
magnitude smaller than using randomly selected conditions, which also
outperform the standard single-domain PINNs.

##  2 Background

Physics-Informed Neural Networks (PINNs) estimate PDE solutions with (deep)
neural networks, in light of the their universal approximation ability.
Consider a PDE of the following general form,

$\displaystyle{\mathcal{F}}[u]({\bf x})$ $\displaystyle=f({\bf x}),\;\;{\bf
x}\in\Omega,$ $\displaystyle u({\bf x})$ $\displaystyle=g({\bf x}),\;\;{\bf
x}\in\partial\Omega$ (1)

where ${\mathcal{F}}$ is the differential operator for the PDE, $\Omega$ is
the domain, $\partial\Omega$ is the boundary of the domain,
$f:\Omega\rightarrow\mathbb{R}$, and $g:\partial\Omega\rightarrow\mathbb{R}$
are the given source and boundary functions, respectively. To solve the PDE,
the PINN uses a deep neural network ${\widehat{u}}_{\bm{\theta}}({\bf x})$ to
represent the solution $u$, samples $N$ collocation points $\\{{\bf
x}_{c}^{i}\\}_{i=1}^{N}$ from $\Omega$ and $M$ points $\\{{\bf
x}_{b}^{i}\\}_{i=1}^{M}$ from $\partial\Omega$, and minimizes the loss,

$\displaystyle\bm{\theta}^{*}={\operatornamewithlimits{argmin}}_{\bm{\theta}}\;\;\lambda_{b}L_{b}(\bm{\theta})+L_{r}(\bm{\theta})$
(2)

where
$L_{b}(\bm{\theta})=\frac{1}{M}\sum_{i=1}^{M}\left({\widehat{u}}_{\bm{\theta}}({\bf
x}_{b}^{i})-g({\bf x}_{b}^{i})\right)^{2}$ is the boundary term to fit the
boundary condition,
$L_{r}(\bm{\theta})=\frac{1}{N}\sum_{i=1}^{N}\left({\mathcal{F}}[{\widehat{u}}_{\bm{\theta}}]({\bf
x}_{c}^{i})-f({\bf x}_{c}^{i})\right)^{2}$ is the residual term to fit the
equation, and $\lambda_{b}>0$ is the weight of the boundary term. One can also
add an initial condition term in the loss function to fit the initial
conditions (when needed).

Multi-Domain PINNs decompose the domain $\Omega$ into several subdomains
$\Omega_{1},\ldots,\Omega_{K}$, and assign a separate PINN
${\widehat{u}}_{\bm{\theta}_{k}}$ to solve the PDE in each subdomain
$\Omega_{k}$. The loss for each PINN includes a boundary term
$L_{b}^{k}(\bm{\theta}_{k})$ and residual term $L_{r}^{k}(\bm{\theta}_{k})$
similar to those in (2), based on the boundary and collocation points sampled
from $\partial\Omega_{k}$ and $\Omega_{k}$, respectively. In addition, to
stitch the subdomains to obtain an overall solution over $\Omega$, we
introduce interface conditions into the loss to align the solutions of
different PINNs at the intersection of the subdomains. There have been quite a
few interface conditions. Suppose
$\Omega_{k}\cap\Omega_{k^{\prime}}\neq\emptyset$. We sample a set of interface
points $\\{{\bf
x}_{k,k^{\prime}}^{i}\\}_{i=1}^{J_{k,k^{\prime}}}\in\Omega_{k}\cap\Omega_{k^{\prime}}$.
One commonly used interface condition is to encourage the solution continuity
(Jagtap and Karniadakis, 2021),

$\displaystyle
I_{1}(\bm{\theta}_{k},\bm{\theta}_{k^{\prime}})=\frac{1}{J_{k,k^{\prime}}}\sum_{i=1}^{J_{k,k^{\prime}}}\left({\widehat{u}}_{\bm{\theta}_{k}}({\bf
x}_{k,k^{\prime}}^{i})-{\widehat{u}}^{\text{avg}}_{k,k^{\prime}}({\bf
x}_{k,k^{\prime}}^{i})\right)^{2}$

where ${\widehat{u}}^{\text{avg}}_{k,k^{\prime}}({\bf
x}_{k,k^{\prime}}^{i})=\frac{1}{2}\left({\widehat{u}}_{\bm{\theta}_{k}}({\bf
x}_{k,k^{\prime}}^{i})+{\widehat{u}}_{\bm{\theta}_{k^{\prime}}}({\bf
x}_{k,k^{\prime}}^{i})\right)$. A second one is the residual continuity
(Jagtap and Karniadakis, 2021),

$\displaystyle
I_{2}(\bm{\theta}_{k},\bm{\theta}_{k^{\prime}})=\frac{1}{J_{k,k^{\prime}}}\sum_{i=1}^{J_{k,k^{\prime}}}\big{(}\left({\mathcal{F}}[{\widehat{u}}_{\bm{\theta}_{k}}]({\bf
x}_{k,k^{\prime}}^{i})-f({\bf x}_{k,k^{\prime}}^{i})\right)$
$\displaystyle-\left({\mathcal{F}}[{\widehat{u}}_{\bm{\theta}_{k^{\prime}}}]({\bf
x}_{k,k^{\prime}}^{i})-f({\bf x}_{k,k^{\prime}}^{i})\right)\big{)}^{2}.$ (3)

Other choices include the gradient continuity (De Ryck et al., 2022) that
encourages $\partial{\widehat{u}}_{\bm{\theta}_{k}}({\bf
x}_{k,k^{\prime}}^{i})/\partial{\bf x}_{k,k^{\prime}}^{i}$ and
$\partial{\widehat{u}}_{\bm{\theta}_{k^{\prime}}}({\bf
x}_{k,k^{\prime}}^{i})/\partial{\bf x}_{k,k^{\prime}}^{i}$ to be identical,
the residual gradient continuity (Yu et al., 2022), flux conservation (Jagtap
et al., 2020), etc. In general, the loss for each subdomain $k$ has the
following form

$\displaystyle{\mathcal{L}}^{k}=\lambda_{b}L_{b}^{k}(\bm{\theta}_{k})+L_{r}^{k}(\bm{\theta}_{k})+\lambda_{I}\sum_{k^{\prime}:\Omega_{k^{\prime}}\cap\Omega_{k}\neq\emptyset}\sum_{n\in{\mathcal{S}}}I_{n}(\bm{\theta}_{k},\bm{\theta}_{k^{\prime}})$

where ${\mathcal{S}}$ is the set of interface conditions, and $\lambda_{I}>0$
is the weight of the interface term. The training is to minimize
${\mathcal{L}}=\sum_{k=1}^{K}{\mathcal{L}}^{k}$. The final solution inside
each sub-domain $k$ is given by the associated PINN
${\widehat{u}}_{\bm{\theta}_{k}}$, while on the interface, by the average of
the PINNs that share the interface.

##  3 Meta Learning of Interface Conditions

While multi-domain PINNs have shown successes, the selection of the interface
conditions remains an open and difficult problem. On one hand, naively adding
all possible conditions together will complicate the loss landscape, making
the optimization challenging and expensive, yet not necessarily giving the
best performance. On the other hand, different problems can demand a different
set of the interface conditions as the best choice, which is up to the
properties of the problem itself. Currently, there is a lack of methodologies
to identify conditions for different problems. To address this issue, we
propose METALIC, a meta learning approach to select problem-specific interface
conditions. Specifically, we consider a parametric PDE family ${\mathcal{A}}$,
where each PDE $\zeta\in{\mathcal{A}}$ is parameterized by
$\bm{\beta}\in{\mathcal{X}}\subset\mathbb{R}^{d}$. The parameters can come
from the operator ${\mathcal{F}}$, the source term $f$ and/or the boundary
function $g$ (see (1)). Denote by ${\mathcal{S}}=\\{I_{1},\ldots,I_{q}\\}$ the
full set of interface conditions. Our goal is, given a PDE parameterized by
arbitrary $\bm{\beta}\in{\mathcal{X}}$, to determine
$I(\bm{\beta})\subseteq{\mathcal{S}}$ — the best set of interface conditions —
for multi-domain PINNs to solve that PDE.

###  3.1 Multi-Arm Bandit for Entire Training

We first propose a multi-arm bandit (MAB) model (Slivkins et al., 2019) to
select the interface conditions for the entire training procedure of multi-
domain PINNs. The MAB is a classical reinforcement learning problem. Consider
a gambler playing $q$ slot machines (i.e., arms). Pulling the lever of each
machine will return a random reward from a machine-specific probabilistic
distribution, which is unknown apriori. The gambler aims to maximize the total
reward earned from a series of lever-pulls across the $q$ machines. For each
play, the gambler needs to decide the tradeoff between exploiting the machine
that has observed the largest expected payoff so far and exploring the payoffs
of other machines. To determine PDE-specific interface conditions, we build a
contextual MAB model. We consider the PDE parameters
$\bm{\beta}\in{\mathcal{X}}$ as the state (context) of the system, all
possible combinations of the interface conditions (i.e., the power set of
${\mathcal{S}}$) as the arms, and the negative solution error as the reward.
The problem space can be represented by a triplet
$({\mathcal{X}},{\mathcal{P}},r(\cdot,\cdot))$, where ${\mathcal{X}}$ is the
state space, ${\mathcal{P}}$ is the action space (the power set of
${\mathcal{S}}$), and
$r:{\mathcal{X}}\times{\mathcal{P}}\rightarrow\mathbb{R}$ is the reward
function. We represent each action by a $q$-dimensional binary vector ${\bf
a}$, where each element corresponds to a particular interface condition in
${\mathcal{S}}$. The $i$-th element $a_{i}=1$ means the interface condition
$i$ is selected in the action.

To estimate the unknown reward function $r(\cdot,\cdot)$, we assign a Gaussian
process (GP) prior,

$\displaystyle r\sim\mathcal{GP}\left(0,\kappa\left([\bm{\beta},{\bf
a}],[\bm{\beta}^{\prime},{\bf a}^{\prime}]\right)\right)$ (4)

where $\kappa(\cdot,\cdot)$ is a kernel (covariance) function. Considering the
categorical nature of the action input, we design a product kernel,

$\displaystyle\kappa\left([\bm{\beta},{\bf a}],[\bm{\beta}^{\prime},{\bf
a}^{\prime}]\right)=\kappa_{1}(\bm{\beta},\bm{\beta}^{\prime})\kappa_{2}({\bf
a},{\bf a}^{\prime})$ (5)

where
$\kappa_{1}(\bm{\beta},\bm{\beta}^{\prime})=\exp(-\tau_{1}\|\bm{\beta}-\bm{\beta}^{\prime}\|^{2})$
is the square exponential (SE) kernel for continuous PDE parameters, and

$\displaystyle\kappa_{2}({\bf a},{\bf
a}^{\prime})=\exp\left(\tau_{2}\cdot\frac{1}{q}\sum_{i=1}^{q}\mathds{1}(a_{i}=a^{\prime}_{i})\right)$
(6)

where $\mathds{1}(\cdot)$ is the indicator function. Hence, the similarity
between actions is based on the overlap ratio of the selected interface
conditions, which is natural and intuitive.

To learn the MAB, each step we randomly sample a state $\bm{\beta}$ from
${\mathcal{X}}$, and then select an action ${\bf a}$, i.e., a set of interface
conditions, according to the current GP reward surrogate model. We then run
the multi-domain PINNs with the interface conditions to solve the PDE
parameterized by $\bm{\beta}$. We evaluate the negative solution error $s$ as
the received reward. We add the new data point $([\bm{\beta},{\bf a}],-s)$
into the current training set, and retrain (update) the GP reward model. We
repeat this procedure until a given maximum number of trials (plays) is
finished. To fulfill a good exploration-exploitation tradeoff, we use the
Upper Confidence Bound (UCB) (Auer, 2002; Srinivas et al., 2010) or Thompson
sampling (TS) (Thompson, 1933; Chapelle and Li, 2011) to select the action at
each step. Specifically, denote the current predictive distribution of the GP
surrogate by

$p(\widehat{r}|\mathcal{D},\bm{\beta},{\bf
a})=\mathcal{N}\left(\widehat{r}|\mu({\bf a},\bm{\beta}),\sigma^{2}({\bf
a},\bm{\beta})\right)$

where $\mathcal{D}$ is the accumulated training set so far. The UCB score is

$\text{UCB}({\bf a})=\mu({\bf a},\bm{\beta})+c\cdot\sigma({\bf a},\bm{\beta})$

where $c>0$, and the TS score is sampled from the predictive distribution,

$\text{TS}({\bf a})\sim p(\widehat{r}|\mathcal{D},\bm{\beta},{\bf a}).$

We can see that both scores integrate the predictive mean (which reflects the
exploitation part) and the variance information (exploration part). We
evaluate the score for each action ${\bf a}\in{\mathcal{P}}$ (UCB or TS), and
then select the one with the highest score (corresponding to the best
tradeoff).

In the online scenario, we can keep using UCB or TS to select the action for
incoming new PDEs while improving the reward surrogate model according to the
solution error. When the online playing is done and we no longer conduct
exploration to update our model, given a new PDE (say, indexed by
$\bm{\beta}^{*}$), we evaluate the predictive mean $\mu$ given
$\bm{\beta}^{*}$ and every ${\bf a}\in{\mathcal{P}}$. We then select the one
with the largest predictive mean (reward). We use the corresponding interface
conditions to run the multi-domain PINNs to solve the PDE. Our MAB learning is
summarized in Algorithm 1.

Figure 1: The illustration of the sequential MAB model.

###  3.2 Sequential Multi-Arm Bandits

In practice, to achieve good and reliable performance, the training of PINNs
is often divided into two stages. The first phase is stochastic training,
typically with ADAM optimizer (Kingma and Ba, 2014), to find a nice valley of
the loss landscape. The second phase is deterministic optimization, typically
with L-BFGS, to ensure convergence to the (local) minimum. Due to the
different nature of the two phases, the best interface conditions can vary as
well. To enable more flexible choices so as to further improve the
performance, we propose a sequential MAB model, as illustrated in Fig. 1.
Specifically, for each training phase, we introduce a MAB similar to Sec 3.1,
which updates a separate GP reward model. To coordinate the two MABs to
optimize the final accuracy, the reward of the first MAB, denoted by $r_{1}$,
includes not only the negative solution error after the stochastic training
phase, but also a discounted error after the second phase,

$\displaystyle r_{1}=-s_{1}+\gamma\cdot(-s_{2})$ (7)

where $\gamma$ is the discount factor, and $s_{1}$ and $s_{2}$ are the
solution errors after the stochastic and deterministic training phases,
respectively. In this way, the influence of the interface conditions at the
first training phase on the final solution accuracy is also integrated into
the learning of the GP reward model. Next, we expand the context of the second
MAB with the training loss after the first phase. The system state includes
both the PDE parameters $\bm{\beta}$ and the loss value $\ell$, which together
with the action ${\bf a}$ constitute the input to the GP reward model of the
second MAB. In this way, the training status of the first phase is also used
to determine the interface conditions for the second phase.

To learn our sequential MAB model, each step we randomly sample the PDE
parameters $\bm{\beta}\in{\mathcal{X}}$. Then based on the GP reward model of
the first MAB, we use UCB (or TS) to select an action (i.e., the first set of
interface conditions), with which we conduct the stochastic training of the
multi-domain PINNs. We then feed the training loss $\ell$, PDE parameters
$\bm{\beta}$ and every possible action into the GP reward model of the second
MAB. We select the second set of interface conditions, with which to continue
the training of the multi-domain PINNs using deterministic optimization. We
evaluate the solution error after each phase, and obtain the reward and new
examples for the two GP reward models. We update these models accordingly. The
learning of the sequential MABs is summarized in Algorithm 2.

Algorithm 1 METALIC-single($T$)

1: Initialize the GP reward surrogate model, and
$\mathcal{D}\leftarrow\emptyset$.

2: repeat

3: Randomly sample the PDE parameters $\bm{\beta}\in{\mathcal{X}}$

4: For each action ${\bf a}\in{\mathcal{P}}$, compute the predictive
distribution of the GP reward model, $\mathcal{N}\left(\mu({\bf
a},\bm{\beta}),\sigma^{2}({\bf a},\bm{\beta})\right)$

5: Compute the UCB score: $\text{UCB}({\bf a})=\mu+c\cdot\sigma$ or TS score:
$\text{TS}({\bf a})\sim\mathcal{N}(\mu,\sigma^{2})$

6: ${\bf a}^{*}=\operatornamewithlimits{argmax}_{{\bf
a}\in{\mathcal{P}}}\text{UCB}({\bf a})$ or ${\bf
a}^{*}=\operatornamewithlimits{argmax}_{{\bf a}\in{\mathcal{P}}}\text{TS}({\bf
a})$

7: Use the interface conditions of ${\bf a}^{*}$ to train the multi-domain
PINNs to solve the PDE parameterized by $\bm{\beta}$, and evaluate the
solution error $s$

8: $\mathcal{D}\leftarrow\mathcal{D}\cup\\{\left([{\bf
a}^{*},\bm{\beta}],-s\right)\\}$

9: Retrain the GP reward model on $\mathcal{D}$

10: until $T$ iterations are done

Algorithm 2 METALIC-seq($\gamma$, $T$)

1: Initialize two GP reward models. Set their training sets $\mathcal{D}_{1}$
and $\mathcal{D}_{2}$ to empty.

2: repeat

3: Randomly sample $\bm{\beta}\in{\mathcal{X}}$

4: Based on the predictive distribution of the first GP model, use UCB or TS
to select the best action ${\bf a}^{*}_{1}$.

5: Use the interface conditions of ${\bf a}^{*}_{1}$ to train the multi-domain
PINNs with ADAM. Evaluate the error $s_{1}$ for solving the PDE parameterized
by $\bm{\beta}$.

6: Given the current training loss $\ell$ and $\bm{\beta}$, compute the
predictive distribution of the second GP reward model for each action, and use
UCB or TS to select the best action ${\bf a}^{*}_{2}$.

7: Use the interface conditions of ${\bf a}^{*}_{2}$ to continue the training
with L-BFGS. Evaluate the solution error $s_{2}$.

8: $\mathcal{D}_{1}\leftarrow\mathcal{D}_{1}\cup\\{\left([{\bf
a}^{*},\bm{\beta}],-s_{1}-\gamma
s_{2}\right)\\},\mathcal{D}_{2}\leftarrow\mathcal{D}_{2}\cup\\{([{\bf
a}^{*},\bm{\beta},\ell],-s_{2})\\}$

9: Retrain the two GP models on $\mathcal{D}_{1}$ and $\mathcal{D}_{2}$,
respectively.

10: until $T$ iterations are done

##  4 Related Work

As an alternative to mesh-based numerical methods, PINNs have achieved many
success stories, e.g., (Raissi et al., 2020; Chen et al., 2020; Sirignano and
Spiliopoulos, 2018; Zhu et al., 2019; Geneva and Zabaras, 2020; Sahli Costabal
et al., 2020). Multi-domain PINNs, e.g., XPINNs (Jagtap and Karniadakis, 2021)
and cPINNs (Jagtap et al., 2020), extend PINNs based on domain decomposition
and use a set of PINNs to solve the PDE in different subdomains. To align the
solutions across the subdomains, XPINNs used solution continuity and residual
continuity as the interface conditions. Other conditions are also available,
such as the flux conservation in cPINNs (Jagtap et al., 2020), the gradient
continuity (De Ryck et al., 2022), and the residual gradient continuity in
gPINNs (Yu et al., 2022). Recently, Hu et al. (2021) developed a theoretical
understanding on the convergence and generalization properties of XPINNs, and
empirically examined when XPINNs outperform PINNs.

Meta-learning (Schmidhuber, 1987; Naik and Mammone, 1992; Thrun and Pratt,
2012) is an important topic in machine learning. The existing works can be
roughly attributed to three categories: (1) metric-learning that learns a
metric space with which the tasks can make predictions via matching the
training points, e.g., nonparametric nearest neighbors (Koch et al., 2015;
Vinyals et al., 2016; Snell et al., 2017; Oreshkin et al., 2018; Allen et al.,
2019), (2) learning black-box models (e.g., neural networks) that map the task
dataset and hyperparameters to the optimal model parameters or parameter
updating rules, e.g., (Hochreiter et al., 2001; Andrychowicz et al., 2016; Li
and Malik, 2016; Ravi and Larochelle, 2017; Santoro et al., 2016; Duan et al.,
2016; Wang et al., 2016; Munkhdalai and Yu, 2017; Mishra et al., 2017), and
(3) bi-level optimization where the outer level optimizes the hyperparameters
and the inner level optimizes the model parameters given the hyperparameters
(Finn et al., 2017; Finn, 2018; Bertinetto et al., 2018; Lee et al., 2019;
Zintgraf et al., 2019; Li et al., 2017; Finn et al., 2018; Zhou et al., 2018;
Harrison et al., 2018). A recent influential work is the model-agnostic meta
learning (MAML) (Finn et al., 2017), which uses the bi-level optimization
framework to learn a good model initialization for a family of tasks. There
have been a large number of subsequent works, such as (Grant et al., 2018;
Yoon et al., 2018; Finn et al., 2018; Song et al., 2020; Liu et al., 2019;
Penwarden et al., 2021). In our work, the task family consists of parametric
PDEs, and the learning model is the multi-domain PINNs. The hyperparameters
are the set of interface conditions, not the model initialization. Another
difference is that our method predicts task-specific hyperparameters, rather
than assumes one single set of conditions apply to all the tasks (PDEs).

Multi-arm bandit is a classical online decision making framework (Lai et al.,
1985; Auer et al., 2002a, b; Mahajan and Teneketzis, 2008; Bubeck et al.,
2012), and have numerous applications, such as online advertising (Avadhanula
et al., 2021), collaborative filtering (Li et al., 2016), clinical trials
(Aziz et al., 2021) and robot control (Laskey et al., 2015). To our knowledge,
our work is the first to use MAB for meta learning of task-specific
hyperparameters , which is advantageous in its simplicity and efficiency. MAB
can be viewed as an instance of reinforcement learning (RL) (Sutton and Barto,
2018). But it only needs to online estimates a reward function. While one can
design more expressive RL models to meanwhile learn a Markov decision process
(in MAB, we simply use UCB or TS), it often demands we run a massive number of
PINN training trajectories, which is much more expensive. The model estimation
is also much more challenging.

##  5 Experiment

To evaluate METALIC, we considered four benchmark equation families. We list
the equations and domain decomposition settings in the following.

The Poisson Equation. First, we considered a 2D Poisson equation with a
parameterized source function,

$\displaystyle u_{xx}+u_{yy}=\widetilde{f}(x,y;s)$ (8)

where $(x,y)\in[0,1]\times[0,1]$,
$\widetilde{f}(x,y;s)=f(x,y;s)/\max_{x,y}f(x,y;s)$, and

$\displaystyle
f(x,y;s)=\left[\text{erf}((x-0.25)s)-\text{erf}((x-0.75)s)\right]$
$\displaystyle\cdot\left[\text{erf}((y-0.25)s)-\text{erf}((y-0.75)s)\right],$
(9)

where $\text{erf}(z)=\frac{2}{\sqrt{\pi}}\int_{0}^{z}e^{-t^{2}}{\rm d}t$, and
$s\in[0,50]$ is called the sharpness parameter that controls the sharpness of
the interior square in the source. We used Dirichlet boundary conditions, and
ran a finite difference solver to obtain an accurate “gold-standard” solution.
To run multi-domain PINNs, we split the domain into two subdomains, where the
interface is a line at $y=0.5$. We visualize an exemplar solution and the
subdomains, including the sampled boundary and collocation points in Fig. 2.

|
---|---

(a) Solution at $s=20$

(b) Subdomains

Figure 2: The Poisson equation. The interface is the green line at $y=0.5$.
Blue and black dots show the sampled boundary points in each subdomain, and
the internal dots (red and cyan) are the sampled collocation points inside
each subdomain.

Advection Equation. We next considered a 1D advection (one-way wave) equation,

$u_{t}+\beta u_{x}=0$

where $x\in[0,2\pi]$, $t\in[0,1]$, and $\beta$ is the PDE parameter denoting
the wave speed. We used Dirichlet boundary conditions, and the solution has an
analytical form, $u(x,t)=h(x-\beta t)$ were $h(x)$ is the initial condition
(which we select as $h(x)=\sin(x)$). For domain decomposition, we split the
domain at $t=0.5$ to obtain two subdomains. Fig. 3 shows an exemplar solution
and the subdomains with the interface.

|
---|---

(a) Solution at $\beta=30$

(b) Subdomains

Figure 3: Advection equation. The interface is the green line at $t=0.5$.

Reaction Equation. Third, we evaluate a 1D reaction equation,

$u_{t}-\rho u(1-u)=0$

where $\rho$ is the reaction coefficient (ODE parameter), $x\in[0,2\pi]$,
$t\in[0,1]$ and $u(x,0)=e^{-\frac{(x-\pi)^{2}}{2(\pi/4)^{2}}}$. The exact
solution is $u(x,t)=u(x,0)\cdot[e^{\rho t}/\left(u(x,0)e^{\rho
t}+1-u(x,0)\right)]$. We split the domain at $t=0.5$ to obtain two subdomains.
Although not required for well-posedness of the ODE system, because we are
solving for the PINN space-time field $u(x,t)$, we use the exact solution to
define a boundary loss term. This enhances training without compromising the
time partitioning we wish to highlight. We show a solution example and the
subdomains in Fig. 4.

Burger’s Equation. Fourth, we considered the viscous Burger’s equation,

$u_{t}+uu_{x}=\nu u_{xx}$

where $\nu\in[0.001,0.05]$ is the viscosity (PDE parameter), $x\in[-1,1]$,
$t\in[0,1]$, and $u(x,0)=-\sin(\pi x)$. We ran a numerical solver to obtain an
accurate “gold-standard” solution. To decompose the domain, we take the middle
portion that includes the shock waves as one subdomain, namely,
$\Omega_{1}:x\in[-0.1,0.1],t\in[0,1]$, and the remaining as the other
subdomain, $\Omega_{2}:x\in[-1,-0.1]\cap[0.1,1],t\in[0,1]$. Hence, the
interface consists of two lines. See Fig. 5 for the illustration and solution
example.

|
---|---

(a) Solution at $\rho=5.0$

(b) Subdomains

Figure 4: Reaction equation. The interface is at $t=0.5$.

|
---|---

(a) Solution at $\nu=0.001$

(b) Subdomains

Figure 5: Burger’s equation. The interfaces are at $x=-0.1$ and $x=0.1$. The
middle portion (filled with cyan dots) is the first subdomain, and the
remaining parts constitute the second subdomain.

|  |  |
---|---|---|---

(a) Poisson equation

(b) Advection equation

(c) Reaction equation

(d) Burgers’ equation

Figure 6: Online performance of METALIC-single.

|  |  |
---|---|---|---

(a) Poisson equation

(b) Advection equation

(c) Reaction equation

(d) Burgers’ equation

Figure 7: Online performance of METALIC-seq.

---

(a) Poisson equation ($s=46.1$)

(b) Advection equation ($\beta=24.02$)

(c) Reaction equation ($\rho=4.1$)

(d) Burgers’ equation ($\nu=0.0036$)

Figure 8: Point-wise solution error.

To evaluate METALIC, we used 9 interface conditions, which are listed in the
supplementary material. For the PINN in each subdomain, we used two layers,
with 20 neurons per layer and tanh activation function. We randomly sampled
1,000 collocation points and 100 boundary points for each PINN. To inject the
interface conditions, we then randomly sampled 101 interface points for the
Poisson, advection and reaction equations, and 802 interface points for
Burger’s equation. We set $\lambda_{b}=20$ and $\lambda_{I}=5$, which follows
the insight of (Wang et al., 2021, 2022) to adopt large weights for the
boundary and interface terms so as to prevent the training of PINNs from being
dominated by the residual term. We denote our single MAB by METALIC-single,
and sequential MABs by METALIC-seq. For the latter, we set the discount factor
$\gamma=0.9$ (see (7)). For better numerical stability, we used the relative
$L_{2}$ error in the log domain to obtain the reward for updating the GP
surrogate models. The running of the multi-domain PINNs consists of 10K ADAM
epochs (with learning rate $10^{-3}$) and then 50K L-BFGS iterations (the
first order optimality and parameter change tolerances set to $10^{-6}$ and
$10^{-9}$ respectively). We set $c=1$ to compute the UCB score. We ran 200
plays (trials) for our method. For static (offline) test, we randomly sampled
100 PDEs (which do not overlap with the PDEs sampled during the online
playing). We then used the learned reward model to determine the best
interface conditions for each particular PDE (according to the predictive
mean), with which we ran the multi-domain PINNs to solve the PDE, and computed
the relative $L_{2}$ error.

Method | Poisson | Advection | Reaction | Burger’s
---|---|---|---|---
Random-Single | 0.3992 $\pm$ 0.0037 | 0.07042 $\pm$ 0.01575 | 0.00612 $\pm$ 0.000278 | 0.04021 $\pm$ 0.00057
Random-Seq | 0.3004 $\pm$ 0.0032 | 0.03922 $\pm$ 0.00919 | 0.01284$\pm$ 0.000707 | 0.03486 $\pm$ 0.00066
PINN-Sub | 0.03078 $\pm$ 0.00177 | 0.00130 $\pm$ 8.108e-5 | 0.00213 $\pm$ 0.00021 | 0.00738 $\pm$ 0.00313
PINN-Merge-H | 0.02398 $\pm$ 0.00144 | 0.00098 $\pm$ 5.038e-5 | 0.00223 $\pm$ 0.00026 | 0.00951 $\pm$ 0.00390
PINN-Merge-V | 0.02184 $\pm$ 0.00211 | 0.00079 $\pm$ 3.391e-5 |  0.00099 $\pm$ 0.00013 | 0.00276 $\pm$ 0.00041
METALIC-Single-TS | 0.02503 $\pm$ 0.0002 | 0.00079 $\pm$ 4.5897e-5 | 0.00204 $\pm$ 1.013e-4 |  0.00109 $\pm$ 1.306e-5
METALIC-Single-UCB | 0.0245$\pm$ 0.0002 | 0.00078 $\pm$ 3.6771e-5 | 0.00102 $\pm$ 8.945e-6 | 0.00161 $\pm$ 2.939e-5
METALIC-Seq-TS | 0.01639$\pm$ 9.5384e-5 | 0.00078 $\pm$ 3.6473e-5 |  0.00099 $\pm$ 8.4704e-6 | 0.00152 $\pm$ 5.571e-5
METALIC-Seq-UCB |  0.01406 $\pm$ 9.1099e-5 |  0.00070 $\pm$ 3.2790e-5 |  0.00099 $\pm$ 5.999e-6 | 0.00139 $\pm$ 3.948e-5

Table 1: The average $L_{2}$ relative error of single-domain PINNs and multi-
domain PINNs for solving 100 test PDEs. The interface conditions of the multi-
domain PINNs are provided by METALIC and random selection. {Single, Seq}
indicate using a single set or two sequential sets of interface conditions for
the running of the multi-domain PINNs. {TS, UCB} corresponds to our method
using TS or UCB score to determine the interface conditions at each play.

First, to examine the online performance of METALIC, we looked into the
accumulated solution error along with the number of plays. We compared with
randomly selecting the arm at each play. In the case of running METALIC-seq,
this baseline correspondingly randomly selects the arm twice, one at the
stochastic training phase, and the other at the deterministic phase. The
results are shown in Fig. 6 and 7. As we can see, the accumulated error of
METALIC with both UCB and TS grows much slower, i.e., sublinearly, than the
random selection approach (note that the reward of the optimal action is
unknown due to the randomness in the running of PINNs, and we cannot compute
the regret). This has shown that our method achieves a much better
exploration-exploitation tradeoff in the online interface condition decision
and model updating, which is consist with many other MAB applications (see Sec
4). The results demonstrate the advantage of our MAB-based approach. First,
via effective exploration, METALIC can collect valuable training examples
(rewards at new actions and context) to improve the learning efficiency and
performance of the GP reward surrogate model. Second, the online decision also
takes advantage of the predictive abaility of the current reward model, i.e.,
exploitation, to select effective interface conditions, which results in
increasingly better solution accuracy of the multi-domain PINNs. The online
nature of METALIC enables us to keep improving the reward model while
utilizing it to solve new equations with promising accuracy.

Next, we conducted an offline test, namely, without online exploration and
model updates any more after 200 plays. We compared with (1) Random-Single,
which, for each PDE, randomly selects a set of interface conditions applied to
the entire training of the multi-domain PINNs, and (2) Random-Seq, which for
each PDE, randomly selects two sets of interface conditions, one for the
stochastic training and the other for the deterministic training phase. We
also tested single-domain PINNs that do not incorporate interface conditions.
Specifically, we compared with (3) PINN-Sub, which used the same architecture
as the PINN in each subdomain, but applied to the entire domain, (4) PINN-
Merge-H, which horizontally pieced all the PINNs in the subdomains, i.e.,
doubling the layer width yet fixing the depth, (5) PINN-Merge-V, which
vertically stacked the PINNs, i.e., doubling the depth while fixing the width.
Note that while PINN-Merge-H and PINN-Merge-V merge the PINNs in each
subdomain, the total number of neurons actually increases (for connecting
these PINNs). Hence, the merged PINN is more expressive. Each single-domain
PINN used the union of the boundary points and collocation points from every
subdomain. We used the same weight for the boundary term, i.e.,
$\lambda_{b}=20$. The running of each single-domain PINN follows exactly the
same setting of the multi-domain PINNs (i.e., 10K ADAM epochs and 50K L-BFGS
iterations).

We report the average relative $L_{2}$ solution error and the standard
deviation in Table 1. As we can see, randomly selecting interface conditions,
no matter for the whole training procedure or two training phases, result in
much worse solution accuracy of multi-domain PINNs. The solution error is one
order of magnitude bigger than METALIC in all the settings. It confirms that
the success of the multi-domain PINNs is up to appropriate interface
conditions. Next, we can observe that while the performance of METALIC-Single
is similar to METALIC-Seq, the best solution accuracy is in most cases
obtained by interface conditions selected by METALIC-Seq (except in solving
the Burger’s equation). It demonstrates that our sequential MAB model that can
employ different conditions for the two training phases is more flexible and
brings additional improvement. We also observe that in most cases using the
UCB for online playing can lead to better performance for both METALIC-Single
and METALIC-Seq. This is consistent with the online performance evaluation
(see Fig. 6 and 7). Third, among the single-domain PINN methods, PINN-Merge-V
outperforms PINN-sub in all the equation families and PINN-Merge-H outperforms
PINN-sub in the Poisson and advection equations, showing that deeper or wider
architectures can help further improve the solution accuracy. However, their
performance is still second to the best setting of METALIC, which uses simpler
subdomain PINN architectures and fewer total learnable parameters. These
multi-domain PINNs can be further parallelized to accelerate training. By
contrast, if the interface conditions are inferior, such as those selected by
Random-Single and Random-Seq, the solution error becomes much worse (orders of
magnitude bigger) than single-domain PINNs. Together these results have shown
the importance of the interface conditions for multi-domain PINNs and the
advantage of our method.

For a fine-grained comparison, we visualize the point-wise solution error of
PINN-Sub, Random-Single, Random-Seq, and our method in solving four random
instances of the equations. As shown in Fig. 8, the point-wise error of both
METALIC-Single and METALIC-Seq is quite uniform across the domain and close to
zero (dark blue). By contrast, the competing methods often exhibit relative
large errors in a few local regions, e.g., those in the middle (where the
shock waves appear) of the domain of the Burger’s equation (PINN-Sub, Random-
Single, Random-Seq), and the central part of the domain of the Poisson and
advection equation (Random-Single). It shows that our method not only can give
a superior global accuracy, but locally also better recovers individual
solution values.

Finally, we analyzed the selected interface conditions in each equation family
by METALIC. We found that those conditions are interesting, physically
meaningful, and consistent with the properties of the equations. Due to the
space limit, we provide the detailed analysis and discussion in the
supplementary material.

##  6 Conclusion

We have presented METALIC, a simple, efficient and powerful meta learning
approach to select PDE-specific interface conditions for general multi-domain
PINNs. The results at four bench-mark equation families are encouraging. In
the future, we will extend our method to also meta learn the interface
locations as well as the conditions, as a function of not only accuracy but
training time so as to improve both the solution accuracy and training
efficiency of the multi-domain PINNs.

##  A Interface Conditions

We used a total number of 9 interface conditions throughout all the
experiments, which are listed in Table 1. Note that $I_{z}$ and $I_{zz}$
correspond to the first and second-order derivatives w.r.t an input to the PDE
solution function. Since all the test PDE problems consist of two spatial or
spatiotemporal dimensions, $I_{z}$ and $I_{zz}$ give four interface
conditions. There are no mixed derivatives across different input dimensions.
In the case that one $I_{z}$ is the same as $I_{c}$, such as in Poisson
equation, the de-duplication gives 9 different conditions. In the case that
all $I_{z}$’s are different from $I_{c}$, such as in Burger’s equation, we
used $I_{c}$ and removed one $I_{z}$ ($z=y$ or $z=t$), so that we still
maintain 9 interface conditions to be consistent with other experiments.

Table 2: Interface Conditions of Multi-domain PINNs

$I_{u}$ | Solution continuity (10)
---|---
$I_{u_{avg}}$ | Average solution continuity (11)
$I_{r}$ | Residual (12)
$I_{rc}$ | Residual continuity (13)
$I_{gr}$ | Gradient-enhanced residual (14)
$I_{c}$ | Flux continuity (15)
$I_{z}$ | First-order spatial/temporal derivative continuity (16)
$I_{zz}$ | Second-order spatial/temporal derivative continuity (17)

$\displaystyle
I_{u}(\bm{\theta}_{k},\bm{\theta}_{k^{\prime}})=\frac{1}{J_{k,k^{\prime}}}\sum_{i=1}^{J_{k,k^{\prime}}}\left({\widehat{u}}_{\bm{\theta}_{k}}({\bf
x}_{k,k^{\prime}}^{i})-{\widehat{u}}_{\bm{\theta}_{k^{\prime}}}({\bf
x}_{k,k^{\prime}}^{i})\right)^{2}$ (10)

$\displaystyle
I_{u_{avg}}(\bm{\theta}_{k},\bm{\theta}_{k^{\prime}})=\frac{1}{J_{k,k^{\prime}}}\sum_{i=1}^{J_{k,k^{\prime}}}\left({\widehat{u}}_{\bm{\theta}_{k}}({\bf
x}_{k,k^{\prime}}^{i})-{\widehat{u}}^{\text{avg}}_{k,k^{\prime}}({\bf
x}_{k,k^{\prime}}^{i})\right)^{2}$ (11)
$\displaystyle\text{where}\;{\widehat{u}}^{\text{avg}}_{k,k^{\prime}}({\bf
x}_{k,k^{\prime}}^{i})=\frac{1}{2}\left({\widehat{u}}_{\bm{\theta}_{k}}({\bf
x}_{k,k^{\prime}}^{i})+{\widehat{u}}_{\bm{\theta}_{k^{\prime}}}({\bf
x}_{k,k^{\prime}}^{i})\right)$

$\displaystyle
I_{r}(\bm{\theta}_{k},\bm{\theta}_{k^{\prime}})=\frac{1}{J_{k,k^{\prime}}}\sum_{i=1}^{J_{k,k^{\prime}}}\left(\left({\mathcal{F}}[{\widehat{u}}_{\bm{\theta}_{k}}]({\bf
x}_{k,k^{\prime}}^{i})-f({\bf
x}_{k,k^{\prime}}^{i})\right)^{2}+\left({\mathcal{F}}[{\widehat{u}}_{\bm{\theta}_{k^{\prime}}}]({\bf
x}_{k,k^{\prime}}^{i})-f({\bf x}_{k,k^{\prime}}^{i})\right)^{2}\right)$ (12)

$\displaystyle
I_{rc}(\bm{\theta}_{k},\bm{\theta}_{k^{\prime}})=\frac{1}{J_{k,k^{\prime}}}\sum_{i=1}^{J_{k,k^{\prime}}}\left(\left({\mathcal{F}}[{\widehat{u}}_{\bm{\theta}_{k}}]({\bf
x}_{k,k^{\prime}}^{i})-f({\bf
x}_{k,k^{\prime}}^{i})\right)-\left({\mathcal{F}}[{\widehat{u}}_{\bm{\theta}_{k^{\prime}}}]({\bf
x}_{k,k^{\prime}}^{i})-f({\bf x}_{k,k^{\prime}}^{i})\right)\right)^{2}$ (13)

$\displaystyle I_{gr}(\bm{\theta}_{k},\bm{\theta}_{k^{\prime}})$
$\displaystyle=\frac{1}{J_{k,k^{\prime}}}\sum_{i=1}^{J_{k,k^{\prime}}}\sum_{j=1}^{2}\left(\left|\frac{\partial}{\partial{\bf
x}_{k,k^{\prime}}^{i}[j]}\left({\mathcal{F}}[{\widehat{u}}_{\bm{\theta}_{k}}]({\bf
x}_{k,k^{\prime}}^{i})-f({\bf
x}_{k,k^{\prime}}^{i})\right)\right|^{2}+\left|\frac{\partial}{\partial{\bf
x}_{k,k^{\prime}}^{i}[j]}\left({\mathcal{F}}[{\widehat{u}}_{\bm{\theta}_{k^{\prime}}}]({\bf
x}_{k,k^{\prime}}^{i})-f({\bf x}_{k,k^{\prime}}^{i})\right)\right|^{2}\right)$
(14)

$\displaystyle
I_{c}(\bm{\theta}_{k},\bm{\theta}_{k^{\prime}})=\frac{1}{J_{k,k^{\prime}}}\sum_{i=1}^{J_{k,k^{\prime}}}\left(\phi\left({\widehat{u}}_{\bm{\theta}_{k}}({\bf
x}_{k,k^{\prime}}^{i})\right)\cdot{\bf
n}-\phi\left({\widehat{u}}_{\bm{\theta}_{k^{\prime}}}({\bf
x}_{k,k^{\prime}}^{i})\right)\cdot{\bf n}\right)^{2}$ (15)
$\displaystyle\text{where}\;\phi({\widehat{u}}_{\bm{\theta}})\cdot{\bf
n}\;\text{are fluxes normal at the interface}$

$\displaystyle
I_{z}(\bm{\theta}_{k},\bm{\theta}_{k^{\prime}})=\frac{1}{J_{k,k^{\prime}}}\sum_{i=1}^{J_{k,k^{\prime}}}\left(\frac{\partial}{\partial
z^{i}}{\widehat{u}}_{\bm{\theta}_{k}}({\bf
x}_{k,k^{\prime}}^{i})-\frac{\partial}{\partial
z^{i}}{\widehat{u}}_{\bm{\theta}_{k^{\prime}}}({\bf
x}_{k,k^{\prime}}^{i})\right)^{2}$ (16) $\displaystyle\text{where}\;z^{i}={\bf
x}^{i}_{k,k^{\prime}}[1]\;\text{or}\;z^{i}={\bf x}^{i}_{k,k^{\prime}}[2].$

$\displaystyle
I_{zz}(\bm{\theta}_{k},\bm{\theta}_{k^{\prime}})=\frac{1}{J_{k,k^{\prime}}}\sum_{i=1}^{J_{k,k^{\prime}}}\left(\frac{\partial^{2}}{\partial{z^{i}}^{2}}{\widehat{u}}_{\bm{\theta}_{k}}({\bf
x}_{k,k^{\prime}}^{i})-\frac{\partial^{2}}{\partial{z^{i}}^{2}}{\widehat{u}}_{\bm{\theta}_{k^{\prime}}}({\bf
x}_{k,k^{\prime}}^{i})\right)^{2}$ (17) $\displaystyle\text{where}\;z^{i}={\bf
x}^{i}_{k,k^{\prime}}[1]\;\text{or}\;z^{i}={\bf x}^{i}_{k,k^{\prime}}[2].$

##  B Preliminary Study of the Interface Conditions

We conducted a preliminary study on a 2D Poisson equation $u_{xx}+u_{yy}=1$
with the solution shown in Figure 9.

Figure 9: Poisson solution.

Given this PDE problem, we compared three types of boundary and collocation
point sampling methods: random, grid, and Poisson disc sampling, as shown in
Figure 10. The comparison was done between a standard PINN and XPINN, where
the number of collocation points in each XPINN subdomain is the same as the
total number of collocation points used by the PINN. We trained the two models
with the boundary loss term weight $\lambda_{b}$ set to 1 and 20. We also
varied the interface loss term weight $\lambda_{I}$ from {1, 20}. The
interface loss term is computed from (11) and (13). Table 3 shows the $L_{2}$
relative error averaged over 10 runs to minimize the variance in network
initialization and optimization. We can see that the XPINN performance is
relevantly less variant to differences in sampling and weights, but for PINNs
these differences result in order of magnitude changes in error. For this
reason, we have conducted all the evaluations fairly by using random sampling
and larger boundary weights for PINN’s which was the best overall setting. We
also make the insight that random sampling allows a PINN to see higher
frequencies according to the Nyquist-Shannon sampling theorem which may be the
reason for increased performance over the other sampling methods. But the
XPINN has the additional complexity of subdomains and interface conditions
which may dominate the training resulting and give less varying performance
across all the sampling methods.

|  |
---|---|---

(a) Random

(b) Grid

(c) Poisson Disc

Figure 10: Random, grid, and Poisson disc sampling for the Poisson equation problem. The interface is the green line at $y=0.5$. Blue and black dots show the sampled boundary points in each subdomain, and the internal dots (red and cyan) the sampled points inside each subdomain. Model | Random | Grid | Poisson Disc
---|---|---|---
$\lambda_{b,I}$ = 1 |  $\lambda_{b,I}$ = 20 |  $\lambda_{b,I}$ = 1 |  $\lambda_{b,I}$ = 20 |  $\lambda_{b,I}$ = 1 |  $\lambda_{b,I}$ = 20
PINN | 9.0325e-4 | 4.3223e-4 | 6.0278e-3 | 2.3699e-3 | 5.3902e-3 | 2.2375e-3
XPINN | 5.0884e-3 | 5.3205e-3 | 6.0764e-3 | 4.7829e-3 | 6.5061e-3 | 4.4888e-3
Table 3: Average $L_{2}$ relative error over 10 runs for different sampling
techniques and loss term weights.

###  B.1 Interface Condition Combination

For the same PDE problem with random sampling, we ran multi-domain PINNs with
different sets of conditions. We used the generalized interface condition
notations for multi-domain PINNs as described in Table A. For example, an
XPINN can be described as $I_{u_{avg}}+I_{rc}$. The weights on all terms are
unity. As seen by the results in Table 4, the multi-domain PINNs with
interfaces $I_{u_{avg}}+I_{c}+I_{yy}$ outperforms the rest as well as the
PINN. We can see that with the correct interface conditions, the multi-domain
PINN can greatly improve upon the standard XPINN. In fact, the additional
residual continuity term, a trait of XPINNs, performs infinitesimally better
than only using the average solution continuity. These results are the
foundation of the METALIC method as we have shown that different combinations
of conditions result in drastically different performances. We can also see
that multi-domain PINNs are more general and flexible than the existing PINN
decomposition models such as XPINN and CPINN, and hence have more potentials.
Having only used XPINNs in Table 3, one might conclude decomposing this
problem is inferior to a standard PINN. However, we have shown that CPINN
outperformed XPINN by an order of magnitude and that adding the additional
term $I_{yy}$ improved the CPINN even further. Furthermore, naively adding all
possible terms such as the final row, does not necessarily give the best
results either. This leaves three options for multi-domain PINNs: manually
tuning the interface conditions, running all possible permutations such as we
have done here, or devise a method to learn the appropriate interfaces such as
METALIC.

Model |  $L_{2}$ Relative Error
---|---
PINN | 1.05e-3 ± 4.38e-4
$I_{u_{avg}}$ | 4.28e-3 ± 2.63e-3
$I_{u_{avg}}+I_{rc}$ | 3.92e-3 ± 2.25e-3
$I_{u_{avg}}+I_{c}$ | 9.45e-4 ± 2.85e-4
$I_{u_{avg}}+I_{rc}+I_{c}$ | 9.77e-4 ± 3.45e-4
$I_{u_{avg}}+I_{rc}+I_{gr}$ | 4.57e-3 ± 3.18e-3
$I_{u_{avg}}+I_{c}+I_{yy}$ | 5.26e-4 ± 1.97e-4
$I_{u_{avg}}+I_{rc}+I_{gr}+I_{c}+I_{yy}$ | 9.34e-4 ± 3.18e-4
Table 4: Average $L_{2}$ relative error over 10 runs for different interface
combinations. Note: $I_{c}=I_{y}$ for this problem.

##  C Meta Learning Result Analysis

There are $2^{9}=512$ possible combinations of the interface conditions. For
convenience, we use an integer to index each configuration (combination),
$\text{index}=\sum_{i=0}^{n}2^{i}{\bf c}[i]$ where ${\bf c}$ is a list of
binaries, and ${\bf c}[i]=1$ means $i$-th interface condition is turned on. We
therefore can show how different sets of interface conditions are selected
along with the equation parameters (see Figures 11, 14, 17, and 20).

###  C.1 Poisson Equation

For each PDE test case, we provide three analysis plots to better understand
the METALIC results. For the Poisson problem, Figure 11 provides an overview
of the interface configuration groupings as the equation parameter $s$ varies.
As opposed to Random-Single, the various METALIC methods predict interface
configurations in groupings based on parameter $s$. This indicates that for
these ranges, the PDE solution behaves similarly across the interfaces. It can
also be seen that the configurations chosen between METALIC-Single and
METALIC-Seq are different, indicating that the optimization is an important
factor. This is logical since at the beginning of training, the PINN must
first propagate information from the initial and boundary conditions inward to
the entire domain. Therefore, interface conditions during this phase may in
fact make learning more difficult in terms of the loss landscape as the
network is trying to enforce continuity at a location which has no information
but is simply a set of random predictions given the initialization of weights
and bias of the network.

In Figure 12, we can see the number of times the interface conditions are
selected over the 100 test cases. Random-Single serves as a baseline with each
interface being chosen roughly half of the time. The two most noticeable
trends are that the gradient-enhanced residual term is almost never chosen and
the flux continuity which is equivalent to $u_{y}$ for this case is always
chosen by METALIC. This is interesting as the gradient term in gPINNs was
shown to be beneficial in PINN training but appears to be a poor chose on a
set of interface points, possibly because with all the other terms, it simply
make the loss landscape more complex and does not provide a significant
accuracy benefit compared to the other more theoretically sound terms such as
flux. This is a novel result as it proves the robustness of METALIC in being
able to distinguish between valid and invalid terms, something that would take
a user doing manual tuning of these terms much trail and error to determine.
We also note that the METALIC choices align with our results in Section B.1
that the flux conditions from CPINN greatly outperforms the residual
continuity conditions in XPINN for Poisson’s equation. Flux continuity is a
well studied conservation term rooted in traditional methods whereas residual
continuity is a term devised with the convince of PINNs and automatic-
differentiation(AD) in mind. We also note that when comparing the METALIC-Seq-
UCB ADAM and L-BFGS choices, L-BFGS uses more terms on average than ADAM. This
confirms our hypothesis from Figure 11 that more interface terms at the start
of training may in-fact make it more difficult. This validates the result that
not only is a sequential interface prediction more accurate, but also faster
as it adds in terms as needed which would reduce computational cost. We also
not that including the residual points in the overall set of collocation
points is rarely chosen, likely due to the fact that the interface point set
is an order of magnitude smaller than the collocation point set so assuming it
is well sampling its contribution is negligible. All these insights further
confirm the method is working well and is consistent with our intuition and
the properties of the equation.

Finally, in Figure 13, we show the $L^{2}$ relative error as a function of
$s$. This is a more detailed version of the error table in the manuscript
which tells us how the problem difficulty changes over the parameterization of
the problem. For this Poisson problem, it is quite consistent other than the
lower bound around $s=1$ in which the forcing term is very smooth and as we
expect the problem is quite simple, as reflected by the lower errors there.

Figure 11: Scatter plot of interface configuration vs. the equation parameter
$s$. Figure 12: Horizontal bar plot of the quantity of interfaces chosen
throughout testing over 100 randomly drawn equation parameters. Figure 13:
Scatter plot of the relative $L_{2}$ error vs. the equation parameter $s$.

###  C.2 Advection Equation

For the Advection problem, Figure 15 shows us that very few terms where needed
in this case. So much so that the ADAM step of METALIC-Seq-UCB has only one
term, $I_{u_{avg}}$, the weaker form of the solution continuity. We again
point out the benefit of METALIC in being able to sub select few terms out of
many while still resulting in the best accuracy as seen in Figures 14 & 16
which show that METALIC-Seq-UCB uses the fewest number of terms but has the
best error. This emphasizes that more terms are not always better as the loss
landscape in terms of optimization can become more complex. Another
interesting feature is that the first derivative in space ($u_{x}$) is chosen
more than the first derivative in time ($u_{t}$) despite the subdomain split
being in time. This is opposite of the Poisson results in which the derivative
normal to the interface ($u_{y}$), representing the flux, was chosen in all
cases. Both terms, $u_{t}$ and $u_{x}$ are part of the PDE with flux simply
being $u$, but the tangential derivative $u_{x}$ appears to be a much more
meaningful term when it comes to propagating the wave through the interface.
There are also no second order terms which was the case with Poisson.

Figure 14: Scatter plot of interface configuration vs. the equation parameter
$\beta$. Figure 15: Horizontal bar plot of the quantity of interfaces chosen
throughout testing over 100 randomly drawn equation parameters. Figure 16:
Scatter plot the relative $L_{2}$ error vs. the equation parameter $\beta$.

###  C.3 Reaction Equation

For the Reaction problem, it is interesting to note that whole this is an ODE,
with no spatial derivatives, they were counter-intuitively chosen as
interfaces. This emphasizes the fact that PINNs, and machine learning
techniques in general, do not work the same as traditional methods as these
terms are not necessary for well-posedness. Given this, Figure 19 shows that
METALIC outperformed the PINN while using these conditions. This is an
interesting line of investigation in the future as it shows counter-intuitive
terms can provide a training benefit to PINNs even in contract to the previous
Advection problem where almost no terms where chosen. It is not clear why in
some cases only the most basic of terms are used while in others terms which
do not make physical sense are chosen, but in both, the accuracy is quite
good. This shows that METALIC learns something about training PINNs that is
not evidently clear to the human user.

Figure 17: Scatter plot of interface configuration vs. the equation parameter
$\rho$. Figure 18: Horizontal bar plot of the quantity of interfaces chosen
throughout testing over 100 randomly drawn equation parameters. Figure 19:
Scatter plot of the relative $L_{2}$ error vs. the equation parameter $\rho$.

###  C.4 Burger’s Equation

For the Burger’s problem, we see more of what one might expect from interface
terms. Figure 21 shows that the flux term is predominately chosen, just as in
Poisson’s equation. Although here we see the flux is not equivalent to the
first order derivative, enforcing the idea that it is in fact the flux
providing the training benefit and not a coincidence of the first-order
derivative and flux being the same for Poisson. We also see the largest
improvement in error of METALIC over PINNs as seen in Figure 22. The trend is
also consistent with our physical understanding, as viscosity ($\nu$)
increases the problem becomes more simple. This is because at lower
viscosities a shock forms and creates a discontinuity in the solution which is
difficult for PINNs to resolve. The decomposition of this problem is the most
sound, in that we allow one network to handle the sharp discontinuity in the
center, and another to handle the relatively simple solution around it. This
allows for the network in the center to learn a higher frequency basis with
which to approximate the discontinuity instead of also having to fit the lower
frequencies around it which is delegated to the second network. Given this, it
makes sense that a multi-domain PINN and METALIC greatly outperform PINNs
here. It also emphasizes that in the less physically motivated decompositions
for Poission, Advection, and Reaction, we still see improvement using multi-
domain PINNs and METALIC.

Figure 20: Scatter plot of interface configuration vs. the equation parameter
$\nu$. Figure 21: Horizontal bar plot of the quantity of interfaces chosen
throughout testing over 100 randomly drawn equation parameters. Figure 22:
Scatter plot of the relative $L_{2}$ error vs. the equation parameter $\nu$.

## References

  * Allen et al. (2019) Kelsey Allen, Evan Shelhamer, Hanul Shin, and Joshua Tenenbaum.  Infinite mixture prototypes for few-shot learning.  In _International Conference on Machine Learning_ , pages 232–241. PMLR, 2019.
  * Andrychowicz et al. (2016) Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas.  Learning to learn by gradient descent by gradient descent.  _arXiv preprint arXiv:1606.04474_ , 2016.
  * Auer (2002) Peter Auer.  Using confidence bounds for exploitation-exploration trade-offs.  _Journal of Machine Learning Research_ , 3(Nov):397–422, 2002.
  * Auer et al. (2002a) Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer.  Finite-time analysis of the multiarmed bandit problem.  _Machine learning_ , 47(2):235–256, 2002a.
  * Auer et al. (2002b) Peter Auer, Nicolo Cesa-Bianchi, Yoav Freund, and Robert E Schapire.  The nonstochastic multiarmed bandit problem.  _SIAM journal on computing_ , 32(1):48–77, 2002b.
  * Avadhanula et al. (2021) Vashist Avadhanula, Riccardo Colini Baldeschi, Stefano Leonardi, Karthik Abinav Sankararaman, and Okke Schrijvers.  Stochastic bandits for multi-platform budget optimization in online advertising.  In _Proceedings of the Web Conference 2021_ , pages 2805–2817, 2021\.
  * Aziz et al. (2021) Maryam Aziz, Emilie Kaufmann, and Marie-Karelle Riviere.  On multi-armed bandit designs for dose-finding clinical trials.  _Journal of Machine Learning Research_ , 22(1-38):4, 2021.
  * Bertinetto et al. (2018) Luca Bertinetto, Joao F Henriques, Philip HS Torr, and Andrea Vedaldi.  Meta-learning with differentiable closed-form solvers.  _arXiv preprint arXiv:1805.08136_ , 2018.
  * Bubeck et al. (2012) Sébastien Bubeck, Nicolo Cesa-Bianchi, et al.  Regret analysis of stochastic and nonstochastic multi-armed bandit problems.  _Foundations and Trends® in Machine Learning_ , 5(1):1–122, 2012.
  * Chapelle and Li (2011) Olivier Chapelle and Lihong Li.  An empirical evaluation of thompson sampling.  In _Advances in neural information processing systems_ , pages 2249–2257, 2011.
  * Chen et al. (2020) Yuyao Chen, Lu Lu, George Em Karniadakis, and Luca Dal Negro.  Physics-informed neural networks for inverse problems in nano-optics and metamaterials.  _Optics express_ , 28(8):11618–11633, 2020.
  * De Ryck et al. (2022) Tim De Ryck, Ameya D Jagtap, and Siddhartha Mishra.  Error estimates for physics informed neural networks approximating the navier-stokes equations.  _arXiv preprint arXiv:2203.09346_ , 2022.
  * Duan et al. (2016) Yan Duan, John Schulman, Xi Chen, Peter L Bartlett, Ilya Sutskever, and Pieter Abbeel.  Rl2: Fast reinforcement learning via slow reinforcement learning.  _arXiv preprint arXiv:1611.02779_ , 2016.
  * Fang and Zhan (2019) Zhiwei Fang and Justin Zhan.  Deep physical informed neural networks for metamaterial design.  _IEEE Access_ , 8:24506–24513, 2019.
  * Finn (2018) Chelsea Finn.  _Learning to learn with gradients_.  PhD thesis, UC Berkeley, 2018.
  * Finn et al. (2017) Chelsea Finn, Pieter Abbeel, and Sergey Levine.  Model-agnostic meta-learning for fast adaptation of deep networks.  In _International Conference on Machine Learning_ , pages 1126–1135. PMLR, 2017.
  * Finn et al. (2018) Chelsea Finn, Kelvin Xu, and Sergey Levine.  Probabilistic model-agnostic meta-learning.  _arXiv preprint arXiv:1806.02817_ , 2018.
  * Geneva and Zabaras (2020) Nicholas Geneva and Nicholas Zabaras.  Modeling the dynamics of pde systems with physics-constrained deep auto-regressive networks.  _Journal of Computational Physics_ , 403:109056, 2020.
  * Grant et al. (2018) Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths.  Recasting gradient-based meta-learning as hierarchical Bayes.  In _6th International Conference on Learning Representations, ICLR 2018_ , 2018.
  * Harrison et al. (2018) James Harrison, Apoorva Sharma, and Marco Pavone.  Meta-learning priors for efficient online bayesian regression.  In _International Workshop on the Algorithmic Foundations of Robotics_ , pages 318–337. Springer, 2018.
  * Hochreiter et al. (2001) Sepp Hochreiter, A Steven Younger, and Peter R Conwell.  Learning to learn using gradient descent.  In _International Conference on Artificial Neural Networks_ , pages 87–94. Springer, 2001.
  * Hu et al. (2021) Zheyuan Hu, Ameya D Jagtap, George Em Karniadakis, and Kenji Kawaguchi.  When do extended physics-informed neural networks (xpinns) improve generalization?  _arXiv preprint arXiv:2109.09444_ , 2021.
  * Jagtap and Karniadakis (2021) Ameya D Jagtap and George E Karniadakis.  Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations.  In _AAAI Spring Symposium: MLPS_ , 2021.
  * Jagtap et al. (2020) Ameya D Jagtap, Ehsan Kharazmi, and George Em Karniadakis.  Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems.  _Computer Methods in Applied Mechanics and Engineering_ , 365:113028, 2020.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Kissas et al. (2020) Georgios Kissas, Yibo Yang, Eileen Hwuang, Walter R Witschey, John A Detre, and Paris Perdikaris.  Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4d flow mri data using physics-informed neural networks.  _Computer Methods in Applied Mechanics and Engineering_ , 358:112623, 2020.
  * Koch et al. (2015) Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov.  Siamese neural networks for one-shot image recognition.  In _ICML deep learning workshop_ , volume 2. Lille, 2015.
  * Krishnapriyan et al. (2021) Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney.  Characterizing possible failure modes in physics-informed neural networks.  _Advances in Neural Information Processing Systems_ , 34:26548–26560, 2021.
  * Lai et al. (1985) Tze Leung Lai, Herbert Robbins, et al.  Asymptotically efficient adaptive allocation rules.  _Advances in applied mathematics_ , 6(1):4–22, 1985.
  * Laskey et al. (2015) Michael Laskey, Jeff Mahler, Zoe McCarthy, Florian T. Pokorny, Sachin Patil, Jur van den Berg, Danica Kragic, Pieter Abbeel, and Ken Goldberg.  Multi-armed bandit models for 2d grasp planning with uncertainty.  In _2015 IEEE International Conference on Automation Science and Engineering (CASE)_ , pages 572–579, 2015.  doi: 10.1109/CoASE.2015.7294140.
  * Lee et al. (2019) Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto.  Meta-learning with differentiable convex optimization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10657–10665, 2019.
  * Li and Malik (2016) Ke Li and Jitendra Malik.  Learning to optimize.  _arXiv preprint arXiv:1606.01885_ , 2016.
  * Li et al. (2016) Shuai Li, Alexandros Karatzoglou, and Claudio Gentile.  Collaborative filtering bandits.  In _Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval_ , pages 539–548, 2016.
  * Li et al. (2017) Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li.  Meta-sgd: Learning to learn quickly for few-shot learning.  _arXiv preprint arXiv:1707.09835_ , 2017.
  * Liu and Wang (2019) Dehao Liu and Yan Wang.  Multi-fidelity physics-constrained neural network and its application in materials modeling.  _Journal of Mechanical Design_ , 141(12), 2019.
  * Liu et al. (2019) Hao Liu, Richard Socher, and Caiming Xiong.  Taming maml: Efficient unbiased meta-reinforcement learning.  In _International Conference on Machine Learning_ , pages 4061–4071. PMLR, 2019.
  * Mahajan and Teneketzis (2008) Aditya Mahajan and Demosthenis Teneketzis.  Multi-armed bandit problems.  In _Foundations and applications of sensor management_ , pages 121–151. Springer, 2008.
  * Mishra et al. (2017) Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel.  A simple neural attentive meta-learner.  _arXiv preprint arXiv:1707.03141_ , 2017.
  * Munkhdalai and Yu (2017) Tsendsuren Munkhdalai and Hong Yu.  Meta networks.  In _International Conference on Machine Learning_ , pages 2554–2563. PMLR, 2017.
  * Naik and Mammone (1992) Devang K Naik and Richard J Mammone.  Meta-neural networks that learn by learning.  In _[Proceedings 1992] IJCNN International Joint Conference on Neural Networks_ , volume 1, pages 437–442. IEEE, 1992.
  * Oreshkin et al. (2018) Boris N Oreshkin, Pau Rodriguez, and Alexandre Lacoste.  Tadam: Task dependent adaptive metric for improved few-shot learning.  _arXiv preprint arXiv:1805.10123_ , 2018.
  * Penwarden et al. (2021) Michael Penwarden, Shandian Zhe, Akil Narayan, and Robert M Kirby.  Physics-informed neural networks (pinns) for parameterized pdes: A metalearning approach.  _arXiv preprint arXiv:2110.13361_ , 2021.
  * Raissi et al. (2019a) Maziar Raissi, Paris Perdikaris, and George E Karniadakis.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.  _Journal of Computational Physics_ , 378:686–707, 2019a.
  * Raissi et al. (2019b) Maziar Raissi, Zhicheng Wang, Michael S Triantafyllou, and George Em Karniadakis.  Deep learning of vortex-induced vibrations.  _Journal of Fluid Mechanics_ , 861:119–137, 2019b.
  * Raissi et al. (2020) Maziar Raissi, Alireza Yazdani, and George Em Karniadakis.  Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations.  _Science_ , 367(6481):1026–1030, 2020.
  * Ravi and Larochelle (2017) Sachin Ravi and Hugo Larochelle.  Optimization as a model for few-shot learning.  In _In International Conference on Learning Representations (ICLR)_ , 2017.
  * Sahli Costabal et al. (2020) Francisco Sahli Costabal, Yibo Yang, Paris Perdikaris, Daniel E Hurtado, and Ellen Kuhl.  Physics-informed neural networks for cardiac activation mapping.  _Frontiers in Physics_ , 8:42, 2020.
  * Santoro et al. (2016) Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap.  Meta-learning with memory-augmented neural networks.  In _International conference on machine learning_ , pages 1842–1850. PMLR, 2016.
  * Schmidhuber (1987) Jürgen Schmidhuber.  _Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-… hook_.  PhD thesis, Technische Universität München, 1987.
  * Shukla et al. (2021) Khemraj Shukla, Ameya D Jagtap, and George Em Karniadakis.  Parallel physics-informed neural networks via domain decomposition.  _Journal of Computational Physics_ , 447:110683, 2021.
  * Sirignano and Spiliopoulos (2018) Justin Sirignano and Konstantinos Spiliopoulos.  Dgm: A deep learning algorithm for solving partial differential equations.  _Journal of computational physics_ , 375:1339–1364, 2018\.
  * Slivkins et al. (2019) Aleksandrs Slivkins et al.  Introduction to multi-armed bandits.  _Foundations and Trends® in Machine Learning_ , 12(1-2):1–286, 2019.
  * Snell et al. (2017) Jake Snell, Kevin Swersky, and Richard Zemel.  Prototypical networks for few-shot learning.  _Advances in neural information processing systems_ , 30, 2017.
  * Song et al. (2020) Xingyou Song, Wenbo Gao, Yuxiang Yang, Krzysztof Choromanski, Aldo Pacchiano, and Yunhao Tang.  Es-maml: Simple hessian-free meta learning.  In _ICLR_ , 2020.
  * Srinivas et al. (2010) Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger.  Gaussian process optimization in the bandit setting: no regret and experimental design.  In _Proceedings of the 27th International Conference on International Conference on Machine Learning_ , pages 1015–1022, 2010.
  * Sun et al. (2020) Luning Sun, Han Gao, Shaowu Pan, and Jian-Xun Wang.  Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data.  _Computer Methods in Applied Mechanics and Engineering_ , 361:112732, 2020.
  * Sutton and Barto (2018) Richard S Sutton and Andrew G Barto.  _Reinforcement learning: An introduction_.  MIT press, 2018.
  * Thompson (1933) William R Thompson.  On the likelihood that one unknown probability exceeds another in view of the evidence of two samples.  _Biometrika_ , 25(3/4):285–294, 1933.
  * Thrun and Pratt (2012) Sebastian Thrun and Lorien Pratt.  _Learning to learn_.  Springer Science & Business Media, 2012.
  * Vinyals et al. (2016) Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra.  Matching networks for one shot learning.  _arXiv preprint arXiv:1606.04080_ , 2016.
  * Wang et al. (2016) Jane X Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z Leibo, Remi Munos, Charles Blundell, Dharshan Kumaran, and Matt Botvinick.  Learning to reinforcement learn.  _arXiv preprint arXiv:1611.05763_ , 2016.
  * Wang et al. (2021) Sifan Wang, Yujun Teng, and Paris Perdikaris.  Understanding and mitigating gradient flow pathologies in physics-informed neural networks.  _SIAM Journal on Scientific Computing_ , 43(5):A3055–A3081, 2021.
  * Wang et al. (2022) Sifan Wang, Xinling Yu, and Paris Perdikaris.  When and why pinns fail to train: A neural tangent kernel perspective.  _Journal of Computational Physics_ , 449:110768, 2022.
  * Yoon et al. (2018) Jaesik Yoon, Taesup Kim, Ousmane Dia, Sungwoong Kim, Yoshua Bengio, and Sungjin Ahn.  Bayesian model-agnostic meta-learning.  In _Proceedings of the 32nd International Conference on Neural Information Processing Systems_ , pages 7343–7353, 2018.
  * Yu et al. (2022) Jeremy Yu, Lu Lu, Xuhui Meng, and George Em Karniadakis.  Gradient-enhanced physics-informed neural networks for forward and inverse pde problems.  _Computer Methods in Applied Mechanics and Engineering_ , 393:114823, 2022.
  * Zhou et al. (2018) Fengwei Zhou, Bin Wu, and Zhenguo Li.  Deep meta-learning: Learning to learn in the concept space.  _arXiv preprint arXiv:1802.03596_ , 2018.
  * Zhu et al. (2019) Yinhao Zhu, Nicholas Zabaras, Phaedon-Stelios Koutsourelakis, and Paris Perdikaris.  Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data.  _Journal of Computational Physics_ , 394:56–81, 2019.
  * Zintgraf et al. (2019) Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson.  Fast context adaptation via meta-learning.  In _International Conference on Machine Learning_ , pages 7693–7702. PMLR, 2019.