# Well-definedness of Physical Law Learning:
the Uniqueness Problem

Philipp Scholl
Ludwig-Maximilian University of Munich
Munich
Germany
<EMAIL_ADDRESS>
&Aras Bacho
Ludwig-Maximilian University of Munich
Munich
Germany
<EMAIL_ADDRESS>
&Holger Boche
Technical University of Munich
Munich Center for Quantum Science and Technology (MCQST)
Munich Quantum Valley (MQV)
Munich
Germany
<EMAIL_ADDRESS>
&Gitta Kutyniok
Ludwig-Maximilian University of Munich
University of Tromsø
Munich Center for Machine Learning (MCML)
Munich
Germany
<EMAIL_ADDRESS>

###### Abstract

Physical law learning is the ambiguous attempt at automating the derivation of
governing equations with the use of machine learning techniques. The current
literature focuses however solely on the development of methods to achieve
this goal, and a theoretical foundation is at present missing. This paper
shall thus serve as a first step to build a comprehensive theoretical
framework for learning physical laws, aiming to provide reliability to
according algorithms. One key problem consists in the fact that the governing
equations might not be uniquely determined by the given data. We will study
this problem in the common situation that a physical law is described by an
ordinary or partial differential equation. For various different classes of
differential equations, we provide both necessary and sufficient conditions
for a function to uniquely determine the differential equation which is
governing the phenomenon. We then use our results to devise numerical
algorithms to determine whether a function solves a differential equation
uniquely. Finally, we provide extensive numerical experiments showing that our
algorithms in combination with common approaches for learning physical laws
indeed allow to guarantee that a unique governing differential equation is
learnt, without assuming any knowledge about the function, thereby ensuring
reliability.

_K_ eywords physical law learning $\cdot$ learning differential equations
$\cdot$ machine learning.

##  1 Introduction

For most of human history, scientists had to derive physical laws by hand. As
the availability of data in physics is growing and artificial intelligence is
becoming more powerful, engineers and physicists are aiming to apply machine
learning to infer the governing laws from experimental data. This has the
potential to advance our modeling and understanding of complex dynamics across
disciplines. Schmidt and Lipson [1] accurately compute dynamics used in
mechanics and biology in experiments. Xu et al. [2] display the ability to
compute equations used in environmental science, fluid dynamics, high-energy
physics, and electronic science. Moreover, Martius and Lampert [3] were able
to learn dynamics from atomic physics and robotics.

The common assumption is always that the underlying physical law is described
by a function, an ordinary differential equation (ODE), or a partial
differential equation (PDE). In this paper we concentrate on methods, which
take data from a solution $(u(t_{i},x_{j}))_{ij}\subset\mathbb{R}$ of some ODE
or PDE as input and output the corresponding differential equation.

Pioneering work in this field has been done, for instance, by Bongard and
Lipson [4], Schmidt and Lipson [1] and Schmidt and Lipson [5], who applied
genetic programming to compute the governing equations. In recent years,
neural networks were incorporated into evolutionary algorithms to benefit from
their expressiveness and differentiability and the ease to train them. Xu et
al. [2], for example, start with approximating the solution of some PDE with a
neural network to generate more training data and stable derivatives for the
genetic programming algorithm which then computes the corresponding PDE.
Udrescu and Tegmark [6] and Udrescu et al. [7] approximate the governing
equation first by a neural network to gain more insights about the function to
be learned, which is then incorporated into the evolutionary algorithms.

A different approach has been taken by Brunton et al. [8] who introduced SINDy
(Sparse Identification of Non-linear Dynamics). SINDy learns an ODE using
sparse linear regression with a huge amount of features to retain the high
expressivity of genetic programming but improve on its trainability. Rudy et
al. [9] extended this method to PDEs and in Champion et al. [10] an
autoencoder is first trained to learn the feature library. Hasan et al. [11]
and Chen et al. [12] train again first a neural network to approximate the
solution. Afterwards they utilize the neural network to deduce more training
data and numerical derivatives for the linear regression.

Instead of applying neural networks in auxiliary ways as in the above
mentioned approaches, there have been many attempts in recent years to model
the governing equation as a neural network [3, 13, 14, 15]. The central idea
is to increase the interpretability of the network by applying sparsity
constraints on the weights and reducing the size of the network, sacrificing
some of the expressivity of deep neural networks.

The main difference between standard machine learning and physical law
learning is that physical law learning aims to compute the governing equations
exactly instead of approximately. This has two reasons. Firstly, following
Occam’s razor [16] it is commonly assumed that physical laws are "simple"
formulas. Secondly, most classical machine learning algorithms allow little
interpretability while compact physical laws can be understood by humans.

Contrary to most of the literature to date which concentrates on methods, in
this paper we intend to build the foundation to analyze physical law learning
from a theoretical point of view. For this, we focus on the problem of well-
definedness of the ODE and PDE learning problem, which is a central issue of
ODE and PDE learning and the necessary first step for further theoretical
considerations: Given the input $u$ and some $n\in\mathbb{N}\setminus\\{0\\}$,
a learning algorithm should return the function $F$ describing the unique PDE
such that

$F\left(u,\frac{\partial u}{\partial x_{1}},...,\frac{\partial u}{\partial
x_{m}},\frac{\partial^{2}u}{\partial^{2}x_{1}},\frac{\partial^{2}u}{\partial
x_{1}\partial x_{2}},...\right)=\frac{\partial^{n}u}{\partial^{n}t}.$ (1)

Hence, the learning algorithm aims to approximate the operator $O$ which maps
a solution $u$ to its corresponding PDE described by the function $F$, i.e.,
$O(u)=F$. This operator $O$, however, is only well-defined if, for all $u$,
there exists exactly one function $F$ describing a PDE, which is solved by
$u$. If there is more than one function $F$ describing a PDE solved by $u$ the
learning problem is not well-defined and an algorithm cannot decide based on
the data which PDE to select. In this paper we assume that there exists at
least one function $F$ such that (1) holds and focus exclusively on uniqueness
of $F$.

To establish that uniqueness is indeed not given in general but is highly
desirable, we provide the following example, see Rudy et al. [9].

######  Example 1.

The Korteweg–De Vries (KdV) equation is defined as

$u_{t}=6uu_{x}-u_{xxx}$ (2)

and is solved by the 1-soliton (= self-reinforcing wave) solution

$u(t,x)=\frac{c}{2}\operatorname{sech}^{2}\left(\frac{\sqrt{c}}{2}(x-ct-a)\right).$
(3)

Here, $\operatorname{sech}(x)=\frac{1}{\cosh(x)}$ is the hyperbolic secant.
The problem for physical law learning algorithms is that $u$ also solves the
one-way wave equation $u_{t}=-cu_{x}$. Thus, the learning algorithm cannot
know which PDE to select and might output the wrong one [9].

To the best of our knowledge, Example 1 in [9] is the only time this
uniqueness problem is mentioned in the physical law learning literature and,
furthermore, it is never tackled theoretically. Thus, this phenomenon poses a
potential danger for any scientific insights inferred using the existing
methods. Additionally, to ensure reliability of algorithms, which aim to learn
physical laws, it is essential to investigate various properties such as the
robustness or complexity of the learning problem, i.e., asking how strong the
effect of noise on the measurements on the computed PDE is or whether there
exists a Turing machine which can perform this task. However, one can only
approach these questions formally if the operator $O$ is well-defined and
uniqueness of the PDE is the first step towards this. Thus, this paper should
be seen as the foundation for a mathematical understanding of physical law
learning.

###  1.1 Our contributions

In this paper we want to raise awareness of the uniqueness problem, as it is
vital when conducting physical law learning to keep in mind what could go
wrong. Furthermore, developing a thorough understanding of the uniqueness
problem is essential to build the foundations for further theoretical
investigations of physical law learning. To understand the uniqueness problem,
we provide necessary and sufficient conditions for uniqueness of PDEs.
Evidently, we have to assume specific function spaces for $F$ in Equation (1)
as otherwise uniqueness becomes almost impossible to achieve. Our choices are
linear, polynomial, algebraic, analytic, smooth and continuous functions, as
these cover large parts of important PDEs and their structure enables a
systematic analysis of uniqueness. After proving the necessary and sufficient
conditions for uniqueness of PDEs, we apply these to specific classes of ODEs.
In the end, we introduce numerical algorithms to check uniqueness of PDEs and
show their correctness in numerical experiments.

Before we summarize our uniqueness results for the different classes of PDEs,
we start with defining uniqueness exactly. Note that we write $F(g)=f$, with
functions $g:U\rightarrow\mathbb{R}^{k}$, $f:U\rightarrow\mathbb{R}$,
$F:\mathbb{R}^{k}\rightarrow\mathbb{R}$, and $U\subset\mathbb{R}^{m+1}$ open,
if $F(g(t,x))=f(t,x)$ for all $(t,x)\in U$. If $F(g(t,x))=0$ for all $(t,x)\in
U$, we write $F(g)=0$.

######  Definition 1 (Uniqueness).

Let $u:U\rightarrow\mathbb{R}$ be a differentiable function on the open set
$U\subset\mathbb{R}^{m+1}$. Let each $g_{1},...,g_{k}:U\rightarrow\mathbb{R}$
be either a projection on one of the coordinates, any derivative of $u$ that
exists, or the function $u$. Denote
$g=(g_{1},...,g_{k}):U\rightarrow\mathbb{R}^{k}$. Let
$n\in\mathbb{N}\setminus\\{0\\}$ such that the $n^{th}$ time derivative of $u$
exists. Let $V$ be a set of functions which map from $\mathbb{R}^{k}$ to
$\mathbb{R}$ and $F\in V$ such that

$\frac{\partial^{n}u}{\partial^{n}t}=F(g).$ (4)

We say that the function $u$ _solves a unique PDE described by functions in
$V$ for $g=(g_{1},...,g_{k})$ with time derivative of $n^{th}$-order_ if $F$
is the unique function in $V$ such that Equation (4) holds. If $n=1$, we often
omit the last part. If none of the $g_{i}$ are spatial derivatives, we say ODE
instead of PDE.

With this definition we can express that $u$ as in Example 1 does not solve a
unique PDE described by polynomials for $g=(u,u_{x},u_{xx},u_{xxx})$ with time
derivative of $1^{st}$-order. Definition 1 captures a broader class of PDEs
than those described in Equation (1). For example, by setting
$g:U\rightarrow\mathbb{R}^{m+l+1},\;g(t,x)=(t,x_{1},...,x_{m},u_{\alpha^{1}}(t,x),...,u_{\alpha^{l}}(t,x))$,
Definition 1 covers PDEs which directly depend on $t$ and $x$:

$\frac{\partial^{n}u}{\partial^{n}t}=F(t,x_{1},...,x_{m},u_{\alpha^{1}}(t,x),...,u_{\alpha^{l}}(t,x)).$
(5)

Here $\alpha^{1},...,\alpha^{l}\in\mathbb{N}^{m}$ are some _multi-indices_.
For a multi-index $\alpha\in\mathbb{N}^{m}$, we write
$u_{\alpha}=\frac{\partial^{|\alpha|}u}{\partial^{\alpha_{1}}x_{1}...\partial^{\alpha_{m}}x_{m}}$
with $|\alpha|=\sum_{i=1}^{m}\alpha_{i}$. Furthermore, Definition 1 covers
ODEs, e.g., an autonomous second-order ODE

$u_{tt}=F(u,u_{t})$ (6)

can be expressed by setting $n=2$ and $g=(g_{1},g_{2})=(u,u_{t})$. We will use
this flexibility for various classes of ODEs in Section 3.

Throughout this paper, let $u$, $U$, $g_{1},...,g_{k}$ and $n$ be as described
in Definition 1. We now summarize the results from Section 2 which show under
which conditions $u$ solves a unique PDE described by functions in various
function classes for $g=(g_{1},...,g_{k})$ with time derivative of
$n^{th}$-order.

Linear PDEs: For linear PDEs we prove Corollary 1, which yields that
$F(g)=\frac{\partial^{n}u}{\partial^{n}t}$ is unique among linear functions
$F$ if and only if $g_{1},...,g_{k}$ are linearly independent.

Polynomial PDEs: We call a PDE _polynomial_ if $F$ in Equation (4) is a
polynomial. Let $\mathcal{D}=g(U)$ denote the image of $g$. For polynomial
PDEs we deduce that uniqueness is equivalent to each of the following
conditions:

  * (1)

There is no algebraic set $X\neq\mathbb{R}^{k}$ such that $\mathcal{D}\subset
X$ (see Definition 4).

  * (2)

The correspondence $I$ of $\mathcal{D}$ is trivial, i.e., $I(\mathcal{D})$
contains no function which is not the zero function (see Definition 4).

  * (3)

The functions $g_{1},...,g_{k}$ are algebraically independent over
$\mathbb{R}$ (see Definition 5).

If we furthermore assume that $u$ is an algebraic function, Theorem 1 yields
that uniqueness is only possible if there are less derivatives involved than
variables, i.e., $k\leq m+1$. Additionally, Theorem 2 states in that case that
uniqueness is equivalent to the Jacobi $J_{g}(t,x)$ being full rank in at
least one point $(t,x)\in U$.

Algebraic PDEs: We call a PDE _algebraic_ if $F$ in Equation (4) is an
algebraic function, contrary to the convention in the differential-algebraic
functions literature [17]. Proposition 2 yields the surprising result that the
conditions for uniqueness coincide for polynomial PDEs and for algebraic PDEs.

Analytic PDEs: We call a PDE _analytic_ if $F$ in Equation (4) is an analytic
function. For analytic PDEs an equivalent condition to
$F(g)=\frac{\partial^{n}u}{\partial^{n}t}$ being unique is that the image
$\mathcal{D}=g(U)$ of $g$ is not a subset of a C-analytic set
$A\neq\mathbb{R}^{k}$ (see Definition 8). Each of the following conditions is
sufficient for uniqueness of analytic PDEs:

  * (1)

There is no analytic set $X\neq\mathbb{R}^{k}$ such that $\mathcal{D}\subset
X$ (see Definition 7).

  * (2)

The $k$-dimensional Lebesgue measure of the image is non-zero, i.e.,
$\lambda^{k}(\mathcal{D})>0$ (see Proposition 4).

  * (3)

The Jacobian $J_{g}(t,x)$ has full rank in at least one point $(t,x)\in U$
(see Theorem 3).

As analytic functions include polynomials, all these conditions are also
sufficient for uniqueness of polynomial and, therefore, also algebraic PDEs.

Smooth PDEs: We call a PDE _smooth_ if $F$ in Equation (4) is a smooth
function. Theorem 4 shows that uniqueness for smooth PDEs is particularly hard
to achieve as it is equivalent to the image $\mathcal{D}$ of $g$ being dense
in $\mathbb{R}^{k}$.

Continuous PDEs: We call a PDE _continuous_ if $F$ in Equation (4) is a
continuous function. Theorem 4 also shows that the uniqueness condition for
smooth PDEs holds for continuous PDEs. Therefore, for all $C^{p}$ function
classes, independent of $0\leq p\leq\infty$, the PDE is unique if and only if
the image $\mathcal{D}$ of $g$ is dense in $\mathbb{R}^{k}$.

All presented results are summarized in Table 1.

Table 1: Uniqueness for $F(g_{1},...,g_{k})=\frac{\partial^{n}u}{\partial^{n}t}$. The second column shows conditions which are equivalent or sufficient to uniqueness of a PDE in the class of the first column. PDE class | Equivalent (E) or sufficient (S) for uniqueness
---|---
Linear PDE | (E) $g_{1},...,g_{k}$ are linear independent
Polynomial or algebraic PDE |  (E) There is no hypersurface/algebraic set $A\neq\mathbb{R}^{k}$ such that $\mathcal{D}\subset A$ (E) $\mathcal{D}$ has a non trivial "correspondence $I$" (E) Assuming $u$ is algebraic: $k\leq m+1$ and Jacobi $J_{g}(t,x)$ has full rank for some $t,x\in U$
Analytic PDE |  (E) There is no C-analytic set $A\neq\mathbb{R}^{k}$ such that $\mathcal{D}\subset A$ (S) There is no analytic set $A\neq\mathbb{R}^{k}$ such that $\mathcal{D}\subset A$ (S) $\lambda^{k}(\mathcal{D})>0$ (S) The Jacobi $J_{g}(t,x)$ is full-rank for some $t,x\in U$
$C^{p}$, $0\leq p\leq\infty$ | (E) $\overline{\mathcal{D}}=\mathbb{R}^{k}$

###  1.2 Outline

The outline of the paper is as follows. We start with proving necessary and
sufficient conditions for uniqueness in the sense of Definition 1 of various
classes of PDEs in Section 2. In Section 3 we apply these results to specific
classes of ODEs, where we utilize the additional structure to deduce stronger
statements. Based on the previous sections, Section 4 is devoted to deriving
numerical algorithms which allow to determine whether a function solves a
unique PDE. We validate these algorithms in Section 5 by extensive numerical
experiments.

##  2 Uniqueness of PDEs

In this section, we aim to derive conditions which guarantee uniqueness/non-
uniqueness of PDEs in the sense of Definition 1. Throughout this section we
let $u:U\rightarrow\mathbb{R}$ be a differentiable function on the open set
$U\subset\mathbb{R}^{m+1}$ and each $g_{1},...,g_{k}:U\rightarrow\mathbb{R}$
either a projection on one of the coordinates, e.g., $g_{i}(t,x)=x_{k}$ for
$(t,x)\in U$, any derivative of $u$ that exists, or the function $u$.
Furthermore, we let $n\in\mathbb{N}\setminus\\{0\\}$ be such that
$\frac{\partial^{n}u}{\partial^{n}t}$ exists.

We continue by defining a property over function spaces which turns out to be
equivalent to uniqueness of the PDE in the given function space.

######  Definition 2.

Let $g:U\rightarrow\mathbb{R}^{k}$, $U\subset\mathbb{R}^{m+1}$ open, be some
function and $V$ any set of functions mapping from $\mathbb{R}^{k}$ to
$\mathbb{R}$ including the constant zero-function. Then we say that $g$ is
_non-trivially annihilated in_ $V$, if there exists some $H\in
V\setminus\\{0\\}$ such that $H(g)=0$. Otherwise we say that $g$ is _only
trivially annihilated in_ $V$.

We can easily prove that the function $g=(g_{1},...,g_{k})$ is only trivially
annihilated in a function class closed under addition and subtraction if and
only if $u$ solves a unique PDE described by the given function class for
$g_{1},...,g_{k}$ with time derivatives of $n^{th}$-order.

######  Proposition 1.

Define the mapping $g=(g_{1},...,g_{k}):U\rightarrow\mathbb{R}^{k}$, with
$U\subset\mathbb{R}^{m+1}$ open and $g_{1},...,g_{k}$ as in Definition 1. Let
$V$ be any class of functions mapping from $\mathbb{R}^{k}$ to $\mathbb{R}$
which is closed under addition and subtraction. Assume there exists $F\in V$
such that $F(g)=\frac{\partial^{n}u}{\partial^{n}t}$. Then, $F$ is the unique
function in $V$ such that $F(g)=\frac{\partial^{n}u}{\partial^{n}t}$ if and
only if $g$ is only trivially annihilated in $V$.

###### Proof.

Let $F,G\in V$ be functions such that
$F(g)=\frac{\partial^{n}u}{\partial^{n}t}=G(g)$. Then, $(F-G)(g)=F(g)-G(g)=0$
which proves both directions. ∎

In the following subsections we use Proposition 1 to deduce uniqueness
criteria for several function classes, all summarized in Section 1.1 and Table
1.

###  2.1 Linear PDEs

The smallest set of PDEs we investigate is the class of linear PDEs, which is
well understood theoretically and covers important equations such as the heat
and wave equation. In the following we will prove that a function is non-
trivially annihilated in the set of linear functions if and only if its
coordinates are linearly dependent.

######  Corollary 1 (Uniqueness for linear PDEs).

Define $g=(g_{1},...,g_{k}):U\rightarrow\mathbb{R}^{k}$, with
$U\subset\mathbb{R}^{m+1}$ open and $g_{1},...,g_{k}:U\rightarrow\mathbb{R}$
as in Definition 1. Assume there exists at least one linear function
$F:\mathbb{R}^{k}\rightarrow\mathbb{R}$ with
$F(g)=\frac{\partial^{n}u}{\partial^{n}t}$. Then there exists a unique linear
function $F$ such that $F(g)=\frac{\partial^{n}u}{\partial^{n}t}$ if and only
if $g_{1},...,g_{k}$ are linear independent.

###### Proof.

The corollary follows immediately from Proposition 1, since there exists a
non-zero linear function $H:\mathbb{R}^{k}\rightarrow\mathbb{R}$ with $H(g)=0$
if and only if $g_{1},...,g_{k}$ are linearly dependent. ∎

###  2.2 Polynomial and Algebraic PDEs

The next larger class are polynomial PDEs. However, Proposition 2 shows that a
function is non-trivially annihilated in the set of polynomial functions if
and only if it is non-trivially annihilated in the set of algebraic functions.
Thus, all the results in this subsection apply to both classes. We start with
defining algebraic functions.

######  Definition 3.

We call a function an _algebraic function_ if it solves an equation defined by
an irreducible non-zero polynomial with real coefficients, i.e., any function
$f:U\rightarrow\mathbb{R}$, for an open set $U\subset\mathbb{R}^{m}$, such
that there exists an irreducible non-zero polynomial
$p:\mathbb{R}^{m+1}\rightarrow\mathbb{R}$ with $p(x,f(x))=0$ for all $x\in U$.

Algebraic functions encompass polynomial functions, rational functions and
also non-elementary functions, as can be seen by the Abel-Ruffini Theorem [18,
19]. Furthermore, the root and the inverse of an algebraic function, if it
exists, is algebraic. Thus, this class of PDEs is very general, including the
KdV-equation from Example 1, Helmholtz equation, Burgers’ equation, minimal
surface equation, and many more.

Interestingly, polynomials are strongly investigated in the fields of algebra
and algebraic geometry and we can, therefore, reformulate the problem in
algebraic languages. In Appendix A we introduce all concepts from algebra
which are necessary to follow the proofs in this section. Let us start with
algebraic geometry [20].

######  Definition 4.

Let $K$ be a field. We call a subset $X\subset K^{n}$ an _algebraic set_ if
there exists an ideal $A\subset K[x_{1},...,x_{n}]$ such that $X=\\{x\in
K^{n}|f(x)=0,\;\forall f\in A\\}$. If $A$ is a principal ideal, i.e., there
exists a polynomial $f\in K[x_{1},...,x_{n}]$ such that $A=(f)$, then $X$ is
called a _hypersurface_ in $K^{n}$.

Given $X\subset K^{n}$ we define the _correspondence $I$_ as

$I(X)\coloneqq\\{f\in K[x_{1},...,x_{n}]|f(x)=0,\;\forall x\in X\\}.$ (7)

By definition, these quantities provide two equivalent characterisations of
the image $\mathcal{D}=g(U)$ such that $g=(g_{1},...,g_{k})$ is only trivially
annihilated in the set of polynomial functions and in the set of algebraic
functions:

  * (1)

There exists no algebraic set $X\neq\mathbb{R}^{k}$ such that
$\mathcal{D}\subset X$. In particular, if $g$ is non-trivially annihilated in
the set of polynomial functions, then there exists a hypersurface $X$ such
that $\mathcal{D}\subset X$.

  * (2)

The correspondence $I$ of $\mathcal{D}$ is trivial, i.e.,
$I(\mathcal{D})=\\{0\\}$.

Additionally, classical algebra provides us with an important definition we
will use in the remainder of this section.

######  Definition 5.

We call functions $f_{1},...,f_{q}:K^{p}\rightarrow K$ _algebraically
dependent over a field $K$_, if there exists a polynomial $P\in
K[x_{1},...,x_{q}]\backslash\\{0\\}$ such that
$P(f_{1}(x_{1},..,x_{p}),...,f_{q}(x_{1},..,x_{p}))=0$. If no such $P$ exists,
we call $f_{1},...,f_{q}$ _algebraically independent_.

This yields a third characterization for $g$ being only trivially annihilated
in the set of polynomials:

  * (3)

The functions $g_{1},...,g_{k}$ are algebraically independent over
$\mathbb{R}$.

We now use this characterization to prove that a function is non-trivially
annihilated in the set of polynomial functions if and only if it is non-
trivially annihilated in the set of algebraic functions. Thus, all the result
in this subsection apply to polynomial and algebraic PDEs.

######  Proposition 2.

Let $f_{1},...,f_{k}:U\rightarrow\mathbb{R}$ be functions such that there
exists a non-zero algebraic function $F:\mathcal{D}\rightarrow\mathbb{R}$ with
$F(f_{1}(x),...,f_{k}(x))=0$ for each $x\in U$, with $U\subset\mathbb{R}^{m}$
open and $\mathcal{D}$ the image of
$(f_{1},...,f_{k}):U\rightarrow\mathbb{R}^{k}$. Then, there exists a non-zero
polynomial $P:\mathbb{R}^{k}\rightarrow\mathbb{R}$ such that
$P(f_{1}(x),...,f_{k}(x))=0$.

###### Proof.

As $F$ is an algebraic function, there exists an irreducible non-zero
polynomial $p:\mathbb{R}^{k}\times\mathbb{R}\rightarrow\mathbb{R}$ such that
$p(y,F(y))=0$ holds for all $y\in\mathcal{D}$. We start with proving that
$p(y,0)\neq 0$ for some $y\in\mathbb{R}^{k}$.

Towards a contradiction, we assume that $p(y,0)=0$ for all
$y\in\mathbb{R}^{k}$. Then there exists some polynomial
$q:\mathbb{R}^{k}\times\mathbb{R}\rightarrow\mathbb{R}$ such that
$p(y,t)=q(y,t)t$ for all $y\in\mathbb{R}^{k}$ and $t\in\mathbb{R}$. As $p$ is
irreducible and non-zero, $q$ must be constant and non-zero. Thus,
$q(y,F(y))F(y)=0$ implies that $F(x)=0$ for all $y\in\mathcal{D}$, which is a
contradiction to the assumption. This yields that there exist
$y\in\mathbb{R}^{k}$ such that $p(y,0)\neq 0$.

Obviously, also $p(f_{1}(x),...,f_{k}(x),F(f_{1}(x),...,f_{k}(x)))=0$ holds
for all $x\in U$. Furthermore, $F(f_{1}(x),...,f_{k}(x))=0$ yields
$p(f_{1}(x),...,f_{k}(x),0)=0$. By setting $P(x)=p(x,0)$, we obtain a non-zero
polynomial $P:\mathbb{R}^{k}\rightarrow\mathbb{R}$ with
$P(f_{1}(x),...,f_{k}(x))=0$. ∎

Having established Proposition 2, we are interested in understanding when a
function is non-trivially annihilated in the set of polynomial functions and
in the set of algebraic functions in more detail. For continuous
differentiable functions $u$, we develop sufficient conditions for uniqueness
of analytic PDEs in the next subsection. These can also be used for the
smaller class of polynomial PDEs and, thus, also for algebraic PDEs. In the
remainder of Subsection 2.2 we restrict $g_{1},...,g_{k}$ to be algebraic
functions, as one can prove additional results for polynomial and algebraic
PDEs then.

We start with citing the result that there exist at most $m$ algebraically
independent algebraic functions for $m$ unknowns [21].

######  Theorem 1.

Let $f_{1},...,f_{m+1}:U\rightarrow\mathbb{R}$, $U\subset\mathbb{R}^{m}$ open,
be $m+1$ algebraic functions in $m$ variables. Then, $f_{1},...,f_{m+1}$ are
algebraically dependent.

Contrary to us, Ehrenborg and Rota [21] state Theorem 1 for algebraic
independence over $\mathbb{C}$. However, for real-valued functions
$f_{i}:\mathbb{R}\rightarrow\mathbb{R}$ algebraic independence over
$\mathbb{R}$ is equivalent to algebraic independence over $\mathbb{C}$, as we
prove in Lemma 2 in Appendix A.

Theorem 1 shows that if $g_{1},..,g_{k}$ are algebraic functions, we have to
ensure that $k\leq m+1$, as otherwise it is certain that $u$ does not solve a
unique polynomial PDE for $g=(g_{1},...,g_{k})$. In the case that we have at
least as many unknown variables as functions, it is possible to use the Jacobi
criterion [21], similar to Theorem 3 in Section 2.3. This provides an
equivalent condition for a function to be non-trivially annihilated in the set
of polynomial function and in the set of algebraic functions. Again we have to
apply Lemma 2 from Appendix A to obtain independence over $\mathbb{R}$.

######  Theorem 2 (Jacobi criterion for algebraic independence).

Define $g=(g_{1},...,g_{k}):U\rightarrow\mathbb{R}^{k}$, with
$U\subset\mathbb{R}^{m+1}$ open and $g_{1},...,g_{k}:U\rightarrow\mathbb{R}$
as in Definition 1. Furthermore, assume that $g_{1},..,g_{k}$ are algebraic
functions and $k\leq m+1$. Then, $g_{1},...,g_{k}$ are algebraically
independent over $\mathbb{R}$ and, thus, $g$ is only trivially annihilated in
the set of polynomial function and in the set of algebraic functions, if and
only if there exists one point $(t,x)\in U$ with $rank(J_{g}(t,x))=k$.

Theorem 2 is seemingly similar to Theorem 3. However, it shows that for
algebraic functions $g_{1},...,g_{k}$, $rank(J_{g}(t,x))=k$ is an equivalent
condition to $g$ being only trivially annihilated in the set of polynomial
function and in the set of algebraic functions. Theorem 3, on the other hand,
holds for arbitrary continuously differentiable functions $g_{1},..,g_{k}$,
but only yields a sufficient condition for $g$ not being annihilated.

An interesting fact about algebraic functions is that the derivative of an
algebraic function is again algebraic. This means that the condition
$g_{1},...,g_{k}$ being algebraic from Theorem 1 and 2 follows from $u$ being
algebraic.

######  Proposition 3.

Let $u:\mathbb{R}^{m}\rightarrow\mathbb{R}$ be an algebraic function and
$\alpha\in\mathbb{N}^{m}$ be a multi index such that $u_{\alpha}$ exists and
is continuous. Then, $u_{\alpha}$ is an algebraic function.

###### Proof.

We only show the statement for $|\alpha|=1$, as the case $|\alpha|>1$ follows
by induction.

In the case $|\alpha|=1$, there exists $1\leq i\leq m$ such that
$u_{\alpha}=u_{i}$. Let $x=(x_{1},...,x_{m})$. As $u$ is an algebraic
function, we know that there exists a non-zero polynomial
$p\in\mathbb{R}[x,t]$ such that

$p(x,u(x))=0.$ (8)

Let $p$ be a polynomial with minimal degree such that Equation (8) holds.
Differentiating Equation (8) with respect to $x_{i}$ yields

$p_{x_{i}}(x,u(x))+p_{t}(x,u(x))u_{i}(x)=0.$ (9)

Now, define $f(x,t)=p_{x_{i}}(x,u(x))+p_{t}(x,u(x))t$. In Appendix A we
introduce the concept of algebraic functions as the algebraic closure of a
rational function field and, thus, we know that the powers and sums of
algebraic functions are again algebraic functions. It follows that $f$ is an
algebraic function.

Since we chose $p$ as the non-zero polynomial with minimal degree such that
$p(x,u(x))=0$, we know that $p_{x_{i}}(x,u(x))$ and $p_{t}(x,u(x))$ are both
non-zero and, therefore, $f$ is non-zero. Furthermore, Equation (9) yields
that $f(x,u_{i}(x))=0$. From Proposition 2 it then follows that $u_{i}$ is an
algebraic function. ∎

###  2.3 Analytic PDEs

The logical extension of polynomials are analytic functions. This class is
particularly large and encompasses most of the relevant PDEs, including also
Liouville’s equation, Zeldovich–Frank-Kamenetskii equation,
Calogero–Degasperis– Fokas equation, and Josephson equations.

######  Definition 6 (Analytic function [22]).

A function $F\in C^{\infty}(X)$ with $X\subset\mathbb{R}^{k}$ open is called
an _analytic function_ if at each point $x\in X$ there exists an open set
$V\subset X$ with $x\in V$ such that the Taylor series of $F$ converges
pointwise to $F$ on $V$. We denote the set of analytic functions on $X$ as
$C^{\omega}(X)$.

Obviously, $C^{\omega}$ includes polynomials but also exponential and
trigonometric functions. Furthermore, sums, products and compositions of
analytic functions are as well analytic functions, as are reciprocals of non-
zero analytic functions and the inverse of an analytic function with non-zero
derivative. We remark that there are functions which are algebraic but not
analytic such as $x^{1/3}$. However, there also exist functions which are
analytic but not algebraic such as $\exp(x)$. More information on this broad
set of functions can be found in Krantz and Parks [22].

The analog of algebraic sets in algebraic geometry are analytic sets in
analytic geometry [23].

######  Definition 7 (Analytic set).

A set $A\subset\mathbb{R}^{n}$ is called an _analytic set_ if for each $a\in
A$ there exists a neighbourhood $V$ such that $A\cap
V=\\{x\in\mathbb{R}^{n}|f_{1}(x)=...=f_{m}(x)=0\\}$ for some analytic
functions $f_{1},...,f_{m}$.

Hence, we can conclude that the image $\mathcal{D}$ of $g$ is a subset of an
analytic set $A\neq\mathbb{R}^{k}$, if $g$ is non-trivially annihilated in
$C^{\omega}$. However, we cannot conclude that $g$ is non-trivially
annihilated in $C^{\omega}$, if its image $\mathcal{D}$ is a subset of an
analytic set $A\neq\mathbb{R}^{k}$, because of the locality in the definition
of analytic sets. Thus, the following definition from [24] is more fitting for
our purposes.

######  Definition 8 (C-analytic set).

A set $A\subset\mathbb{R}^{n}$ is called a _C-analytic set_ , if
$A=\\{x\in\mathbb{R}^{n}|f_{1}(x)=...=f_{m}(x)=0\\}$ for some analytic
functions $f_{1},...,f_{m}$.

For this class of sets, we obtain equivalence: A function $g$ is non-trivially
annihilated in $C^{\omega}$ if and only if its image $\mathcal{D}$ is a subset
of a C-analytic set $A\neq\mathbb{R}^{k}$.

We consider now a sufficient condition for uniqueness of analytic functions
which is proven in Mityagin [25]:

######  Proposition 4.

Let $F:\mathbb{R}^{k}\rightarrow\mathbb{R}$ be an analytic function and
$\mathcal{D}\subset\mathbb{R}^{k}$ a set with $\lambda^{k}(\mathcal{D})>0$,
where $\lambda^{k}$ is the $k$-dimensional Lebesgue-measure. Then
$F|_{\mathcal{D}}=0$ implies $F=0$.

This implies immediately the following corollary.

######  Corollary 2 (Measure criterion for analytic functions).

Define the function $g=(g_{1},...,g_{k}):U\rightarrow\mathbb{R}^{k}$, with
$U\subset\mathbb{R}^{m+1}$ open, $g_{1},...,g_{k}:U\rightarrow\mathbb{R}$ as
in Definition 1 and set $\mathcal{D}\coloneqq g(U)$. If
$\lambda^{k}(\mathcal{D})>0$, then $g$ is only trivially annihilated in
$C^{\omega}$.

Corollary 2 shows uniqueness of the analytic PDE provided that the image of
$g$ is a set with positive measure. The image of a differentiable function,
which maps from a lower-dimensional to a higher-dimensional space, is always a
null set [26]. Therefore, $\lambda^{k}(\mathcal{D})>0$ can only be true if
$m+1\geq k$. In this case, uniqueness of the PDE can be assessed by
investigating the Jacobi matrix of $g$, as can be seen in the following
theorem. This criterion is of high interest since it can be checked
numerically, see Section 4.

######  Theorem 3 (Jacobi criterion for analytic functions).

Define the function $g=(g_{1},...,g_{k}):U\rightarrow\mathbb{R}^{k}$, with
$U\subset\mathbb{R}^{m+1}$ open and $g_{1},...,g_{k}:U\rightarrow\mathbb{R}$
as in Definition 1. Furthermore, assume that $g_{1},...,g_{k}$ are
continuously differentiable. If there exists one point $(t,x)\in U$ with
$rank(J_{g}(t,x))=k$, then $g$ is only trivially annihilated in $C^{\omega}$.

###### Proof.

Let $(y^{0})\in U$ such that $rank(J_{g}(y^{0}))=k$. Then there exist $k$
independent columns of $J_{g}(y^{0})$ and, without loss of generality, we can
achieve
$\det\left(\left(\frac{dg(y^{0})}{dy_{i}}\right)_{i=1,...,k}\right)\neq 0$ by
reordering the components of $y^{0}$.

Let $V\coloneqq U\cap\mathbb{R}^{k}$. Define the function
$\tilde{g}:V\rightarrow\mathbb{R}^{k}$ by
$\tilde{g}(y_{1},...,y_{k})=g(y_{1},...,y_{k},y_{k+1}^{0},...,y_{m+1}^{0})$.
By construction, we obtain
$\det\left(J_{\tilde{g}}((y_{1}^{0},...,y_{k}^{0}))\right)\neq 0$ and, thus,
the inverse function theorem yields that there exist open sets $A\subset V$
and $B\subset\mathbb{R}^{k}$ such that $\tilde{g}(A)=B$. This means that $B$
is in the image of $\tilde{g}$ and, therefore, also of $g$, i.e.,
$B\subset\mathcal{D}$. As $B$ is non-empty and open, this yields
$0<\lambda^{k}(B)\leq\lambda^{k}(\mathcal{D})$. ∎

The question now is whether $\lambda(\mathcal{D})>0$ is also necessary for
uniqueness of the analytic PDE. It turns out that this is not the case, at
least if we disregard the structure of $g$ so far. This follows from Lemma 1
which is proven in Neelon [27]. Notice, that Neelon [27] proves this result
only for functions $f$ with $f(0)=0$. This can be circumvented in Lemma 1
below by applying the original statement to $\tilde{f}(x)=f(x)-c$ and
$\tilde{g}(x,y)=g(x,y+c)$ to include general functions $f$ with $f(0)=c$. This
approach succeeds since $f$ is an analytic function if and only if $\tilde{f}$
is one.

######  Lemma 1.

Let $f:\mathbb{R}^{m}\rightarrow\mathbb{R}$ be in $C^{\infty}\setminus
C^{\omega}$. Then $g=(x,f(x))$ is only trivially annihilated in $C^{\omega}$.

Let $g:\mathbb{R}^{m}\rightarrow\mathbb{R}^{m+1}$ be defined as in Lemma 1. As
$g$ is differentiable, we know that $\lambda^{m+1}(g(\mathbb{R}^{m}))=0$ [26].
Therefore, Lemma 1 yields examples for which $\lambda^{k}(\mathcal{D})=0$, but
the function $g$ is only trivially annihilated in $C^{\omega}$. This shows
that $\lambda^{k}(\mathcal{D})>0$ is only a sufficient but not a necessary
condition, for $g$ being only trivially annihilated by $C^{\omega}$.

In Appendix B we discuss the problem that occurs when applying the matroid
framework [21] to analytic functions and analytic PDEs. This and Lemma 1
reveal the severe difficulty of achieving stronger uniqueness results for
analytic functions.

###  2.4 Continuous and Smooth PDEs

As the last function class we consider the class of $C^{p}$ functions, for any
$0\leq p\leq\infty$. It will turn out that for this class we can almost never
achieve uniqueness. For this, we first recall Proposition 2.3.4 from Krantz
and Parks [22].

######  Proposition 5.

Let $E\subset\mathbb{R}^{k}$ be any closed set. Then there exists a function
$H\in C^{\infty}(\mathbb{R}^{k})$ such that $H^{-1}(\\{0\\})=E$.

This allows to derive the following statement:

######  Theorem 4 (Uniqueness for $C^{p}$ functions).

Define $g=(u_{\alpha^{1}},...,u_{\alpha^{k}}):U\rightarrow\mathbb{R}^{k}$,
with $U\subset\mathbb{R}^{m+1}$ open, $g_{1},...,g_{k}:U\rightarrow\mathbb{R}$
as in Definition 1 and set $\mathcal{D}=g(U)$. Then the function $g$ is only
trivially annihilated in $C^{p}$, for any $0\leq p\leq\infty$, if and only if
the closure of $\mathcal{D}$ is equal to $\mathbb{R}^{k}$.

###### Proof.

"$\Rightarrow$" Assume that the closure $\overline{\mathcal{D}}$ does not
equal $\mathbb{R}^{k}$. Then, Proposition 5 yields that there exists $H\in
C^{\infty}$ such that $H^{-1}(\\{0\\})=\overline{\mathcal{D}}$. Therefore,
$H(g)=0$ holds, but $H\neq 0$ on
$\mathbb{R}^{k}\setminus\overline{\mathcal{D}}$. Thus, $g$ is non-trivially
annihilated in $C^{\infty}$ and, therefore, also in any $C^{p}$ with $0\leq
p\leq\infty$.

"$\Leftarrow$" Assume now that $\overline{\mathcal{D}}=\mathbb{R}^{k}$. Let
$H\in C^{0}$ be such that $H|_{\mathcal{D}}=0$. Since $H$ is continuous, we
obtain that $H$ is 0 on the entire space $\mathbb{R}^{k}$. Thus, $g$ is only
trivially annihilated in $C^{0}$ and, consequently, also in $C^{p}$, for any
$0\leq p\leq\infty$. ∎

This shows that for $C^{p}$, for any $0\leq p\leq\infty$, we can never achieve
uniqueness unless the image of $g$ is dense in $\mathbb{R}^{k}$. This shows
the significance of the ambiguity of PDEs for physical law learning and also
motivates the focus on smaller function classes.

Note that, if we allow $F$ to be discontinuous, surjectivity of $g$ is
necessary and sufficient for ensuring uniqueness in the sense of Definition 1.

##  3 Uniqueness of ODEs

In the previous section we investigated uniqueness of PDEs of the form
$\frac{\partial^{n}u}{\partial^{n}t}=F(g_{1},...,g_{k})$, $1\leq i\leq k$, and
allowed $g_{1},...,g_{k}$ to be either a projection on a single coordinate,
any derivative of $u$ that exists or the function $u$. This enables us to
apply our developed theoretical framework in this section to ODEs as special
cases. Indeed, ODEs are of significant importance for applications, for
example, for electrical circuits, PID-controller, and any system derived from
Euler-Langrage equations [28].

In each of the following subsections, we investigate a specific type of ODEs.
Our analysis will show that additional structures will lead to more refined
necessary and sufficient conditions than merely restricting the previous case
to general ODEs. Throughout this section $u:U\rightarrow\mathbb{R}$ will be a
differentiable function on $U\subset\mathbb{R}$ open. We denote from now on
$u_{t}=\frac{\partial u}{\partial t}$,
$u_{tt}=\frac{\partial^{2}u}{\partial^{2}t}$ and
$u_{ttt}=\frac{\partial^{3}u}{\partial^{3}t}$.

###  3.1 Autonomous ODEs

We start with autonomous ODEs, i.e., ODEs which do not depend directly on $t$,
but only indirectly through $u$ and its derivatives.

####  3.1.1 First order ODEs $u_{t}=F(u)$

As long as $u\in C^{1}(U)$, with $U\subset\mathbb{R}$ open, is not a constant
function, we obtain that $\lambda(\mathcal{D})>0$, where $\mathcal{D}=u(U)$.
Therefore, $u$ is only trivially annihilated in the sets of linear functions,
polynomials or analytic functions. Thus, $u$ solves a unique ODE described by
these function classes for $g=(u)$ with first order time derivative by
Proposition 4.

Uniqueness in the sense of Definition 1 for $C^{p}$, $0\leq p\leq\infty$
follows from Theorem 4 if and only if the image of $u$ is dense in
$\mathbb{R}$. Density in $\mathbb{R}$, however, is equivalent to $u$ being
surjective, since we assume $u$ to be differentiable and, thus, continuous.

####  3.1.2 Second order ODEs $u_{tt}=F(u,u_{t})$

By Corollary 1, the function $g=(u,u_{t}):U\rightarrow\mathbb{R}^{2}$,
$U\subset\mathbb{R}$ open, is non-trivially annihilated in the set of linear
functions if and only if $u$ and $u_{t}$ are linearly independent. This yields
the following proposition:

######  Proposition 6.

Let $u:U\rightarrow\mathbb{R}$, $U\subset\mathbb{R}$ open, and
$F:\mathbb{R}^{2}\rightarrow\mathbb{R}$ linear such that $u_{tt}=F(u,u_{t})$.
Then $F$ is the unique linear function with $u_{tt}=F(u,u_{t})$ if and only if
$u$ is neither a constant nor an exponential function.

###### Proof.

"$\Rightarrow$" If $u$ is constant or an exponential function, then $u$ and
$u_{t}$ are linearly dependent and, thus, $F$ is not unique.

"$\Leftarrow$" We prove that if $F$ is not unique, then $u$ is a constant or
exponential function. For this, assume that $F$ is not unique. The functions
$u$ and $u_{t}$ are then linearly dependent by Corollary 1. Assuming $u$ is
not constant, there exists $\lambda\in\mathbb{R}\setminus\\{0\\}$ such that
$u=\lambda u_{t}$. This ODE is uniquely solved by the exponential function,
which concludes our proof. ∎

If $u$ is an algebraic function, Theorem 1 implies that $F$ does not describe
the unique polynomial PDE for $g=(u,u_{x})$. As $u$ is differentiable, we
obtain that $\lambda^{2}(\mathcal{D})=0$, where $\mathcal{D}$ is the image of
$g=(u,u_{t})$. This allows to conclude that Proposition 4 and, consequently,
Theorem 3 will not yield any information. This also implies that $\mathcal{D}$
is not dense in the image space and, thus, $g$ is non-trivially annihilated in
$C^{p}$.

####  3.1.3 Third order ODEs $u_{ttt}=F(u,u_{t},u_{tt})$

Uniqueness for linear ODEs in $u$, $u_{t}$ and $u_{tt}$ is equivalent to the
independence of $u$, $u_{t}$ and $u_{tt}$, which results in the following
proposition.

######  Proposition 7.

Let $u:U\rightarrow\mathbb{R}$, with $U\subset\mathbb{R}$ open, and let $F$ be
linear such that $u_{tt}=F(u,u_{t})$. Then $F$ is the unique linear function
with $u_{ttt}=F(u,u_{t},u_{tt})$ if and only if $u$ is none of the following
functions:

  * (1)

a constant function,

  * (2)

an exponential function,

  * (3)

a linear combination of two exponential functions,

  * (4)

a linear combination of $t\mapsto\exp(\mu t)$ and $t\mapsto t\exp(\mu t)$ for
any $\mu\in\mathbb{R}$ and

  * (5)

a linear combination of $t\mapsto\exp(\mu t)\cos(\omega t)$ and
$t\mapsto\exp(\mu t)\sin(\omega t)$, for any $\mu,\omega\in\mathbb{R}$.

###### Proof.

$F$ is the unique linear function with $u_{ttt}=F(u,u_{t},u_{tt})$ if and only
if $u$, $u_{t}$ and $u_{tt}$ are linear independent. Thus, $F$ is not unique
if and only if there exists a non-zero $\lambda\in\mathbb{R}^{3}$ such that
$\lambda_{1}u+\lambda_{2}u_{t}+\lambda_{3}u_{tt}=0$. The case $\lambda_{3}=0$
is equivalent to the setting in Proposition 6 and results in the first two
options. The case $\lambda_{3}\neq 0$ yields a second-order linear ODE with
constant coefficients and it is known that those can only be solved by the
functions of items (3) to (5) [29]. ∎

For third order polynomial, algebraic, analytic, smooth and continuous ODEs
the uniqueness conditions coincide with those in the previous case
$u_{tt}=F(u,u_{t})$.

###  3.2 Non-autonomous ODEs

We are now considering non-autonomous ODEs. The theory developed in Section 2
is still applicable as we allowed $g_{1},...,g_{k}$ to be a projection on a
single coordinate, i.e., we can now choose $g_{1}(t)=t$ for all $t\in U$. It
is important to note, however, that it is not meaningful to consider functions
$F$ which are, e.g., linear in all coordinates as usually ODEs are only linear
in $u$ and its derivatives, but not in $t$.

####  3.2.1 First order ODEs $u_{t}=F(t,u)$

We start by assuming that $F$ is linear in $u:U\rightarrow\mathbb{R}$ and
continuous in $t\in U$, with $U\subset\mathbb{R}$ open. This means that we can
write $F(t,u(t))=\lambda(t)u(t)$ for some continuous function
$\lambda:U\rightarrow\mathbb{R}$. For ODEs of this type we can prove the
following proposition.

######  Proposition 8.

Let $u:U\rightarrow\mathbb{R}$, $U\subset\mathbb{R}$ open, be any
differentiable function and let
$F:\mathbb{R}\times\mathbb{R}\rightarrow\mathbb{R}$ be continuous in the first
component and linear in the second component with $F(t,u)=u_{t}$. Then, $F$ is
the only function which is continuous in the first and linear in the second
component if and only if $u$ has only isolated zeros.

###### Proof.

Let $F(t,u(t))=\lambda(t)u(t)$, for $\lambda:U\rightarrow\mathbb{R}$
continuous and $t\in U$. As the class of functions which are continuous in the
first component and linear in the second is closed under addition and
subtraction, we can apply Proposition 1 and, thus, only have to check if
$g(t)=(t,u(t))$ is non-trivially annihilated in this function class. This
means that we have to investigate whether there exists a nonzero function $H$
in this function class satisfying $H(t,u(t))=0$ for all $t\in U$.

"$\Rightarrow$" We show that if $u$ has a zero which is not isolated, then $F$
is not unique. For this, let $\epsilon>0$ be such that
$u(B_{\epsilon}(t_{0}))=0$ holds, with
$B_{\epsilon}(t_{0})\coloneqq\\{t\in\mathbb{R}:|t-t_{0}|<\epsilon\\}\subset
U$. Defining $\phi:\mathbb{R}\rightarrow\mathbb{R}$ as a bump function with
the closure of $B_{\epsilon}(t_{0})$ as support and
$H:\mathbb{R}^{2}\rightarrow\mathbb{R}$ by $H(t,s)=\phi(t)s$ yields
$H(t,u(t))=\phi(t)u(t)=0$ for all $t\in U$. Since $H$ is nonzero, continuous
in $t$, and linear in $s$, the function $g(t)=(t,u(t))$ is non-trivially
annihilated in the set of functions which are continuous in the first and
linear in the second component.

"$\Leftarrow$" Let $H(t,s)\coloneqq\phi(t)s$ be such that

$H(t,u(t))=\phi(t)u(t)=0$ (10)

for all $t\in U$. Assume that $H$ is not the zero function. Then there exists
$t_{0}\in U$ such that $\phi(t_{0})\neq 0$. Equation (10) yields that
$\phi(t_{0})\neq 0$ implies $u(t_{0})=0$. As the zeros of $u$ are isolated,
there exists a ball $B_{\epsilon}(t_{0})\subset U$ with radius $\epsilon>0$
around $t_{0}$ such that $u(t)\neq 0$ for all $t\in
B_{\epsilon}(t_{0})\setminus\\{t_{0}\\}$. By Equation (10), $\phi(t)=0$ holds
for all $t\in B_{\epsilon}(t_{0})\setminus\\{0\\}$ and, by continuity, also at
$t_{0}$. This is a contradiction to $\phi(t_{0})\neq 0$. Thus, $H$ must be the
zero function and, by Proposition 1, $F$ is unique in its class. ∎

Now, consider the polynomial ODE case, i.e., assume that $F(t,u_{t})$ is
continuous in $t$ and polynomial in $u_{t}$. If we assume no additional
constraints on this class of PDEs, we immediately obtain that $g(t)=(t,u(t))$
is non-trivially annihilated, since $H(t,s)=u(t)-s$ is non zero, continuous in
$t$, polynomial in $s$ and $H(t,u(t))=0$ for all $t\in U$. A possible
constraint is that $F(t,s)$ has to be polynomial in $t$ and $s$. By
definition, $g$ is then non-trivially annihilated if and only if $u(t)$ is an
algebraic function. Applying Proposition 2 extends this to the case of
algebraic functions $F$.

For $F(t,s)$ continuous in $t$ and analytic in $s$, uniqueness in the sense of
Definition 1 is impossible as well. Thus, let us now assume that $F$ is
analytic in both $t$ and $s$. In the following, we prove that for this class
$g(t)=(t,u(t))$ is non-trivially annihilated if and only if $u$ is analytic.

######  Proposition 9.

The function $g(t)=(t,u(t))$ is non-trivially annihilated in $C^{\omega}$ if
and only if $u$ is analytic.

###### Proof.

"$\Leftarrow$" Let $u$ be an analytic function. Then, $H(t,s)=u(t)-s$ is a
non-zero analytic function fulfilling $H(t,u(t))=0$ for all $t\in U$. Thus,
$g(t)=(t,u(t))$ is non-trivially annihilated in $C^{\omega}$.

"$\Rightarrow$" This direction follows directly from Lemma 1. ∎

####  3.2.2 Second order ODEs $u_{tt}=F(t,u,u_{t})$

Similar to the last section we start with assuming that
$F(t,u,u_{t})=\lambda(t)u(t)+\mu(t)u_{t}(t)$ is linear in $u$ and $u_{t}$ and
continuous in $t$, i.e., $\lambda$ and $\mu$ are arbitrary continuous
functions. Unfortunately, uniqueness in the sense of Definition 1 of $F$ can
never be achieved in this case.

######  Proposition 10.

Let $u:U\rightarrow\mathbb{R}$, $U\subset\mathbb{R}$ open, be any
differentiable function and let $F:\mathbb{R}^{3}\rightarrow\mathbb{R}$ be
continuous in the first component and linear in the second and third component
with $F(t,u,u_{t})=u_{tt}$. Then $F$ is not the unique function which is
continuous in the first and linear in the second and third component and
fulfills $F(t,u,u_{t})=u_{tt}$.

###### Proof.

Set $\tilde{\lambda}(t)=u_{t}(t)$ and $\tilde{\mu}(t)=-u(t)$ for all $t\in U$
to define
$H:U\times\mathbb{R}^{2}\rightarrow\mathbb{R}^{3},H(t,y,z)\coloneqq\tilde{\lambda}(t)y+\tilde{\mu}(t)z=u_{t}(t)y-u(t)y$.
Notice that $H$ is continuous in the first component since $u$ is twice
differentiable. Furthermore, $H$ is linear in the second and third and
fulfills $H(t,u,u_{t})=0$ for any $t$. Thus,
$g:U\times\mathbb{R}^{3},g(t)\coloneqq(t,u(t),u_{t}(t))$ is non-trivially
annihilated in the class of functions, which are continuous in the first and
linear in the second and third component. ∎

Consequently, uniqueness of $F$ can also never achieved for functions with
polynomial, algebraic, analytic, smooth or continuous second and third
components.

##  4 Algorithms

In the last two sections we established a deeper understanding of the
uniqueness problem of ODE and PDE learning. One immediate application is to
deduce algorithms from the theory to enable practitioners to check numerically
if they are facing ambiguity of the ODE or PDE. These algorithms will
determine for given multi-indices $\alpha^{1},...,\alpha^{k}$, whether there
exists more than one function $F$ in a specific function class for which
$F(u_{\alpha^{1}},...,u_{\alpha^{k}})=u_{t}$ holds.

###  4.1 Linear PDEs

Corollary 1 shows that we have to check for linear dependency of functions in
the case of linear PDEs. Linear dependence can be determined by applying SVD
and comparing the least singular value with some threshold [30]. This is
displayed in Algorithm 1 as Feature Rank Computation (FRanCo).

Input:
$(g(t^{i},x^{j}))_{ij}=((u_{\alpha^{1}}(t^{i},x^{j}),...,u_{\alpha^{k}}(t^{i},x^{j}))_{ij}$,
threshold $\delta>0$

Output: Boolean value indicating if $g$ is non-trivially annihilated in the
set of linear functions

1 $least\\_sv\leftarrow$ smallest singular value of $(g(t^{i},x^{j}))_{ij})$

2 if  _$least\\_sv <\delta$_ then

return $False$

// a small singular value indicates non-uniqueness

3

4else

return $True$

// large singular values indicate uniqueness

5

6 end if

Algorithm 1 Feature Rank Computation (FRanCo)

In general, one has to deal carefully with the errors introduced through
numerical derivatives, as we show in the following.

Assume we have equispaced data for $x\in\mathbb{R}$, i.e.,
$h=x_{1}-x_{0}=x_{2}-x_{1}=...$ and we can bound the measurement error by
$\epsilon>0$. Assume furthermore that the third derivative of $u$ is bounded
by $M(t,x)>0$ on each interval $[x-h,x+h]$. Then, we obtain

$\left|u_{x}(t,x)-\tilde{u}_{x}(t,x)\right|\leq\frac{\epsilon}{h}+\frac{h^{2}}{6}M(t,x),$
(11)

where

$\tilde{u}_{x}(t,x)=\frac{\tilde{u}(x+h)-\tilde{u}(t,x)}{2h}$ (12)

is the three-point finite differences approximation of the derivative of
$\tilde{u}$ [31].

We now compute the bound in Equation (11) for $u(t,x)=\exp(x-at)$, which is
used as the first example in Section 5.1.2. The measurement error is the
machine accuracy, i.e., $\epsilon=10^{-16}$ and we have 300 samples of $x$ on
the interval $[0,10]$, so $h=\frac{1}{30}$. The third derivative in $x$ is
$u_{xxx}(t,x)=\exp(x-at)$ and, thus, a bound is given by
$M(t,x)=\exp(x+h-at)\lesssim\exp(10)$. Therefore, Equation (11) yields

$\left|u_{x}(t,x)-\tilde{u}_{x}(t,x)\right|\leq 3\cdot
10^{-15}+\frac{1}{6\cdot 900}M(t,x)\lesssim 3\cdot
10^{-15}+\frac{1}{4500}e^{10}\approx 5,$ (13)

which is of the same order as the error we witness in Section 5.1.2. Clearly,
this error introduces linear independence of the matrix
$(u(t_{i},x_{j}),\tilde{u}_{x}(t_{i},x_{j}))\in\mathbb{R}^{nm\times 2}$ and,
thus, the experiments in Section 5.1.2 show that its smallest singular value
is 10. Thus, the naive algorithm FRanCo wrongly yields that a PDE is unique.

To deal with the numerical instabilities we increase the order of the finite
differences and check if the lowest singular values converges to 0
exponentially fast. This is expected to happen if the exact matrix does not
have full rank, since higher order approximations have higher order residual
terms [31]. Therefore, we propose applying Stable Feature Rank Computation
(S-FRanCo), described in Algorithm 2, and assess if the least singular value
decreases exponentially for higher finite differences orders. The advantage of
this approach becomes clear in Section 5. S-FRanCo takes a method $features$
as input. The method $features$ outputs the desired feature matrix for given
derivatives and is needed as an input to S-FRanCo, to determine the feature
matrix of which the rank is supposed to be computed. Its explicit application
becomes apparent in the next two subsections. For linear PDEs it should simply
concatenate the derivatives to a linear feature matrix, i.e., for the inputs
$u_{\alpha^{1}}(t^{i},x^{j})_{ij},...,u_{\alpha^{k}}(t^{i},x^{j})_{ij}$
$features$ should return

$U=\left(\begin{array}[]{ccc}|&\ldots&|\\\
u_{\alpha^{1}}(t^{i},x^{j})_{ij}&\ldots&u_{\alpha^{k}}(t^{i},x^{j})_{ij}\\\
|&\ldots&|\end{array}\right).$ (14)

Input: $(u(t^{i},x^{j}))_{ij}$, $\alpha^{1},...,\alpha^{k}$ multi-indices,
$features$ method which takes the derivatives as input and outputs the desired
feature matrix

Output: List of the lowest singular value for each finite-difference order of
the matrix consisting of monomials of the vectors
$(u_{\alpha^{1}}(t^{i},x^{j})_{ij},...,u_{\alpha^{k}}(t^{i},x^{j})_{ij})$

1 Let $ls\\_sv$ be an empty list

2 for  _$l=2$ to $d$_ do

3 for  _$m=1$ to $k$_ do

4 Let $(u_{\alpha^{m}}(t^{i},x^{j})_{ij})$ be the derivative computed by
finite differences of $l^{th}$ order for multi-index $\alpha^{m}$

5 end for

$(g(t^{i},x^{j}))_{ij}\leftarrow
features((u_{\alpha^{1}}(t^{i},x^{j})_{ij},...,u_{\alpha^{k}}(t^{i},x^{j})_{ij}))$

// construct the feature library

6 $least\\_sv\leftarrow$ smallest singular value of $(g(t^{i},x^{j}))_{ij})$

7 $ls\\_sv.append(least\\_sv)$

8 end for

return $ls\\_sv$

Algorithm 2 Stable Feature Rank Computation (S-FRanCo)

###  4.2 Polynomial and Algebraic PDEs

If we assume that $u$ is an algebraic functions, Theorem 2 yields that the
Jacobi has full rank if and only if $g$ is only trivially annihilated in the
sets of polynomials or algebraic functions. We can check the Jacobi rank for a
selected number of points with Jacobi Rank Computation (JRC), as described in
Algorithm 3. As numerical errors also influence the Jacobian, we propose to
use two different finite-difference orders, usually a small one and a large
one. Afterwards, one has to determine if there exists at least one point for
which the least singular value decreased significantly. This can be visualized
by a heat map as shown in Figure 5 in Section 5.2.2.

Input: $(u(t^{i},x^{j}))_{ij}$, $\alpha^{1},...,\alpha^{k}$ multi-indices,
$d_{1}$ small finite-differences order, $d_{2}$ large finite-differences
order, $A$ index set determining the data points for which the Jacobian is
computed

Output: For both finite-difference orders a list of the lowest singular values
of the Jacobian at selected points of the matrix
$((u_{\alpha^{1}}(t^{i},x^{j}),...,u_{\alpha^{k}}(t^{i},x^{j}))_{ij}$

1

2Let $ls\\_sv_{1}$ and $ls\\_sv_{2}$ be empty lists

3

4for  _$p=1$ to 2_ do

// Compute the function $g$

5 for  _$m=1$ to $k$_ do

6 Let $(u_{\alpha^{m}}(t^{i},x^{j})_{ij})$ be the derivative computed by
finite differences of $d_{p}^{th}$ order for multi-index $\alpha^{m}$

7

8 end for

9
$(g(t^{i},x^{j}))_{ij}\leftarrow((u_{\alpha^{1}}(t^{i},x^{j}),...,u_{\alpha^{k}}(t^{i},x^{j}))_{ij}$

// Compute the least singular value of the Jacobian of $g$ at selected points

10 for  _$(i,j)\in A$_ do

11 $jacobian\leftarrow$ the Jacobian of $((g(t^{i},x^{j}))_{ij}$ computed
using finite differences of $d_{p}^{th}$ order

12 $least\\_sv\leftarrow$ smallest singular value of $(g(t^{i},x^{j}))_{ij})$

13 $ls\\_sv_{p}.append(least\\_sv)$

14

15 end for

16

17 end for

18

19return $ls\\_sv$

Algorithm 3 Jacobi Rank Computation (JRC)

In case one can not assume that $u$ is algebraic, we can still use the Jacobi
criterion for analytic functions from Theorem 3, since uniqueness for analytic
PDEs implies uniqueness for polynomial PDEs. Therefore, a full rank Jacobian
in at least one data point yields uniqueness of the PDE. However, the Jacobi
criterion for analytic functions cannot be used to prove non-uniqueness and,
thus, we need a different algorithm for the case that the Jacobian has nowhere
full rank.

Thus, if one does not want to assume that $u$ is algebraic, we propose the
following the approach:

  * (1)

Check for uniqueness in the larger space of analytic PDEs using the Jacobi
criterion from Theorem 3 by applying JRC. If the Jacobian has full rank for at
least one point $(t,x)\in\mathbb{R}^{m+1}$, we are done. Note that Theorem 3
is a consequence of Proposition 4. Thus, it can only be meaningfully used if
there are less derivatives $u_{\alpha^{1}},...,u_{\alpha^{k}}$ than input
variables $(t,x_{1},...,x_{m})$, i.e., if $k\leq m+1$, as mentioned in Section
2.3. Therefore, JRC is only useful in this case.

  * (2)

If Theorem 3 cannot be applied, check uniqueness for polynomials with degree
at most $p\in\mathbb{N}$ by constructing a feature matrix from the monomials
of the derivatives and determining the rank of this matrix. If this matrix has
full rank, the PDE is unique among polynomials with degree $p$. Otherwise we
know that it is not unique. This can be determined in a stable manner by
S-FranCo again, by defining the method $features$ such that it returns for a
list of vectors all monomials of the vectors with degree at most $p$. We can
then assess the rank of the matrices by checking for exponential decay of the
lowest singular value.

###  4.3 Analytic PDEs

Following Theorem 3, JRC can also be applied to show that an analytic PDE is
unique. Unfortunately, Theorem 3 yields only a sufficient condition for
uniqueness and, thus, we gain no information if the rank of the Jacobi matrix
is below $k$.

Similarly to the polynomial case, we then have to directly rely on Proposition
1 and search for an analytic function $H$ such that $H(g)=0$. Thus, we start
by defining the $features$ method which returns a matrix consisting of
features expected in the PDE, e.g., $u_{\alpha^{i}}$, $u_{\alpha^{i}}^{2}$,
$\exp(u_{\alpha^{i}})$ to build the feature matrix $U$. Using this as an input
to S-FRanCo shows whether there exists a unique PDE which can be described as
a linear combination of the chosen features.

###  4.4 Continuous and Smooth PDEs

Theorem 4 shows that for uniqueness of the PDE in $C^{p}$ for any $0\leq
p\leq\infty$, we need to check if the image of $g:U\rightarrow\mathbb{R}^{k}$,
with $U\subset\mathbb{R}^{m+1}$ open, is dense in $\mathbb{R}^{k}$. Given
finitely many evaluations of $g$ this is not rigorously possible. However,
this is not a severe problem as most PDEs can be described by algebraic and
analytic functions.

##  5 Numerical Experiments

In this section we consider PDEs and their solution and determine whether
there exists another PDE which is solved by the given function by using the
theory developed in the last sections. Afterwards, we apply the algorithms
introduced in Section 4 to determine uniqueness numerically without assuming
any prior knowledge. The source code of our experiments is publicly available
in the github repository https://github.com/Philipp238/physical-law-learning-
uniqueness.

###  5.1 Linear PDEs

We start again with linear PDEs. In Section 5.1.1 we consider a function which
solves more than one linear PDE. We observe that the naive algorithm FRanCo
cannot show the ambiguity of the PDE, while the improved version S-FRanCo does
succeed. For this reason, we then solely apply S-FRanCo to the function in
Section 5.1.2 to prove uniqueness of a linear PDE.

####  5.1.1 Non-uniqueness

As a first experiment we consider the function
$u:\mathbb{R}^{2}\rightarrow\mathbb{R},u(t,x)=\exp(x-at)$, with $a=3$. This
function solves the linear PDE $u_{t}=-au_{x}$. The goal is now to determine
if this is the unique linear PDE in the form $u_{t}=F(u,u_{x})$ which is
solved by $u$. For this, we investigate if the function $g\coloneqq(u,u_{x})$
is non-trivially annihilated in the set of linear functions. By Corollary 1,
we know that this is equivalent to $u$ and $u_{x}$ being linearly independent.
Since $u=u_{x}$ holds, the function $g=(u,u_{x})$ is non-trivially annihilated
in the set of linear functions and we easily see that $u$ solves also
$u_{t}=-au$.

We now show the non uniqueness numerically. For this, we sample $u(t,x)$ on
the square $[0,10]^{2}$ with 200 measurements for $t$ and 300 for $x$ without
using noise. The derivatives are computed numerically from this data using
second order finite differences.

FRanCo should reveal that multiple linear PDEs $u_{t}=F(u,u_{x})$ exist, which
are solved by $u$. However, computing the derivative using the second order
finite differences introduces numerical errors. This causes linear
independence among $u$ and $u_{x}$, as the lowest singular value is 10, see
also Section 4.1.

Following the ideas from the last section, we apply S-FRanCo with linear
features and compute the derivatives for different orders for the finite-
differences. As described in the last section, S-FRanCo computes the lowest
singular values for all orders and the user has to check if those converge to
0 exponentially fast. This is expected to happen if the exact matrix does not
have full rank, since higher order approximations have higher order residual
terms [30]. Indeed Figure 1 shows precisely this behaviour. Thus, we can be
certain that the matrix
$(u(t_{i},x_{j}),u_{x}(t_{i},x_{j}))_{i,j}\in\mathbb{R}^{60,000\times 2}$ is
singular and, therefore, $u$ and $u_{x}$ are linearly dependent. This allows
to conclude that there do exist multiple linear PDEs $u_{t}=F(u,u_{x})$ solved
by $u$.

Figure 1: Plot of the lowest singular value of
$(u(t_{i},x_{j}),u_{x}(t_{i},x_{j}))_{i,j}\in\mathbb{R}^{60,000\times 2}$,
where $u_{x}$ was computed using finite differences of different orders for
$u(t,x)=\exp(x-at)$.

####  5.1.2 Uniqueness

We consider the linear PDE $u_{t}=au+bu_{x}$, with $a=1$ and $b=2$,
$u(0,x)=x$, $u\in C^{\infty}(\mathbb{R}^{2})$, which is solved by

$u(t,x)=(x+bt)\exp(at),\;t,x\in\mathbb{R}.$ (15)

The question we address in this subsection is, whether there exists another
linear PDE $u_{t}=F(u,u_{x})$ which is solved by $u$. As the functions
$u:\mathbb{R}^{2}\rightarrow\mathbb{R},u(t,x)=(x+bt)\exp(at)$ and
$u_{x}:\mathbb{R}^{2}\rightarrow\mathbb{R},u_{x}(t,x)=\exp(at)$ are linearly
independent, Corollary 1 yields that $u_{t}=au+bu_{x}$ is the unique linear
PDE of the form $u_{t}=F(u,u_{x})$ solved by $u$.

As before, we now sample $u(t,x)$ on the square $[0,10]^{2}$ with 200
measurements for $t$ and 300 for $x$ without using noise. We then assess
numerically uniqueness, i.e., the linear dependence of $u$ and $u_{x}$ by
using the plot of the singular values in Figure 2 computed by S-FRanCo with
linear features. Since Figure 2 indicates no exponential convergence to 0 of
the lowest singular value for increasing accuracy, we showed numerically that
$u$ and $u_{x}$ are linearly independent.

Figure 2: Plot of the lowest singular value of
$(u(t_{i},x_{j}),u_{x}(t_{i},x_{j}))_{i,j}\in\mathbb{R}^{60,000\times 2}$,
where $u_{x}$ was computed using finite differences of different orders for
$u(t,x)=(x+bt)\exp(at)$.

Note that it was important to specify the derivatives used, i.e., that we
addressed uniqueness for PDEs of the form $u_{t}=F(u,u_{x})$ for
$F:\mathbb{R}^{2}\rightarrow\mathbb{R}$ linear. We intend to check now if
$u_{t}=au+bu_{x}$ is the unique PDE solved by $u$, when allowing PDEs of the
form $u_{t}=G(u,u_{x},u_{xx})$, for $G:\mathbb{R}^{3}\rightarrow\mathbb{R}$
linear. However, since $u_{xx}=0$ holds, the functions $u$, $u_{x}$ and
$u_{xx}$ are linear dependent. This implies that $u_{t}=au+bu_{x}$ is not
unique anymore when we allow $u_{xx}$ to be used. An additional PDE solved by
$u$ is

$u_{t}=au+bu_{x}+u_{xx}.$ (16)

Thus, it becomes apparent that there indeed do exist infinitely many linear
PDEs solved by $u$, but only one of the form $F(u,u_{x})=u_{t}$.

###  5.2 Polynomial and Algebraic PDEs

For the experiments with polynomial PDEs, we start by revisiting the
Korteweg–De Vries equation in Section 5.2.1 and consider afterwards a PDE with
algebraic solution in Section 5.2.2.

####  5.2.1 Korteweg–De Vries equation

In this section we consider Example 1 again and aim to verify that our
algorithms are able to show the encountered ambiguity displayed there. For our
experiments we set $a=0$ and $c=1$ and sample $u(t,x)$ on the square
$[0,10]^{2}$ with 200 measurements for $t$ and 300 for $x$. First we check
uniqueness for $u_{t}=F(u,u_{x},u_{xx},u_{xxx})$ for linear functions $F$. For
this, we apply S-FRanCo with linear features and visualize the results in
Figure 3. The clear absence of exponential decay shows that the one-way wave
equation is indeed the unique linear PDE.

Figure 3: Plot of the lowest singular value of
$(u(t_{i},x_{j}),u_{x}(t_{i},x_{j}),u_{xx}(t_{i},x_{j}),u_{xxx}(t_{i},x_{j}))_{i,j}\in\mathbb{R}^{60,000\times
4}$, where the derivatives were computed using finite differences of different
orders for $u(t,x)=\frac{c}{2}sech^{2}(\frac{\sqrt{c}}{2}(x-ct-a))$.

Next, we aim to show numerically that the one-way wave equation is not the
unique polynomial PDE. As $u$ is not algebraic, we cannot use Theorem 1 and 2.
Following the plan from Section 4.2, we would usually start by applying JRC,
i.e., by trying to apply the Jacobi criterion for analytic functions. However,
there are more functions than variables involved and, therefore, the Jacobi
criterion is not helpful as the rank of the Jacobian is at most $m+1=2<4=k$.

Hence, we directly jump to the second step, see Section 4.2, namely using
S-FRanCo with monomial features. This means we construct a library of
monomials of the derivatives and check them for linear independence.
Naturally, we start with monomials up to order 2, i.e., we construct the
feature matrix

$U=\left(\begin{array}[]{cccccccc}|&|&|&|&|&|&\ldots&|\\\
u&u_{x}&u_{xx}&u_{xxx}&u^{2}&uu_{x}&\ldots&u_{xxx}^{2}\\\
|&|&|&|&|&|&\ldots&|\end{array}\right).$ (17)

The rank computation of $U$ can be done using S-FRanCo and is shown in Figure
4. We clearly see that $U$ is indeed not full rank, which means that there
exists a polynomial $p$ of degree 2 such that $p(u,u_{x},u_{xx},u_{xxx})=0$.
This polynomial is, as we already know, given by the difference of the right
hand side of the one-way wave equation and of the KDV equation.

Figure 4: Plot of the lowest singular value of the feature matrix consisting
out of all monomials up to degree 2 of $u$, $u_{x}$, $u_{xx}$ and $u_{xxx}$,
where the derivatives were computed using finite differences of different
orders for $u(t,x)=\frac{c}{2}sech^{2}(\frac{\sqrt{c}}{2}(x-ct-a))$.

####  5.2.2 Algebraic solution

In this section we consider the algebraic function
$u:\mathbb{R}_{>0}^{2}\rightarrow\mathbb{R},u(t,x)=1/(t+x)$ which solves the
linear PDE $u_{t}=u_{x}$ and the polynomial PDE $u_{t}=-u^{2}$. For our
experiments we sample $u(t,x)$ on the square $[1,5]^{2}$ with 200 measurements
for $t$ and 300 for $x$. We assume that we know that $u$ is an algebraic
function. We then aim to show non-uniqueness of polynomial PDEs of the form
$u_{t}=F(u,u_{x})$ by applying the Jacobi criterion. Figure 5 shows the
smallest singular value of the Jacobian at different data points
$(t_{i},x_{j})$, as computed by JRC. The upper image was created by computing
the derivatives using $2^{nd}$ order finite differences is used and the lower
image using $7^{th}$ order finite differences. We observe a clear trend from
singular values around $10^{-5}$ to $10^{-12}$ and, thus, deduce that the
Jacobian is at no point $(t_{i},x_{j})$ full rank. As $u$ is algebraic,
Theorem 2 yields that $u$ solves multiple polynomial PDEs.

Figure 5: Smallest singular value of the Jacobian at different points
$(t_{i},x_{j})$ of $g=(u,u_{x})$ for $u(t,x)=1/(x+t)$. For the upper image,
the derivatives were computed using 2nd order finite differences and, for the
lower image, 7th order finite differences were used.

###  5.3 Analytic PDEs

In this last section we investigate the usefulness of the Jacobi criterion for
analytic PDEs. For this, we first consider a case where the function solves
multiple analytic PDEs and reveal that its Jacobian is indeed never full rank.
The second case then considers functions which solve only one analytic PDE. We
then prove uniqueness numerically by showing that the Jacobian has full rank
at all points.

####  5.3.1 Non-uniqueness

Consider the PDE

$u_{t}=u_{x},\;u(0,x)=sin(x)$ (18)

which is solved by $u:\mathbb{R}^{2}\rightarrow\mathbb{R},u(t,x)=sin(x+t)$. We
sample $u(t,x)$ on the square $[0,5]^{2}$ with 200 equispaced measurements for
$t$ and 300 for $x$. We now aim to check if $u_{t}=u_{x}$ is the only analytic
PDE of the form $u_{t}=F(u,u_{x})$, which is solved by $u$. Thus, we
investigate the function

$g(t,x)\coloneqq(u(t,x),u_{x}(t,x))=(sin(t+x),cos(t+x)).$ (19)

Obviously, $u$ and $u_{x}$ are linearly independent and, therefore,
$u_{t}=u_{x}$ is the unique linear PDE only using $u$ and $u_{x}$ which is
solved by $u$.

Since $\mathcal{D}=\\{(x,y)\in\mathbb{R}^{2}:x^{2}+y^{2}=1\\}$ we obtain
$\lambda^{2}(\mathcal{D})=0$. Thus, the Jacobian has nowhere full rank, so
Proposition 3 and 4 are not applicable. We now intend to verify this also
numerically. Figure 6 shows the smallest singular value of the Jacobian at
different data points $(t_{i},x_{j})$ as computed by JRC. We clearly see the
trend to a small singular value in every data point, indicating that the
Jacobi matrix has never full rank. Therefore, we cannot deduce that the PDE
was the unique analytic PDE.

Figure 6: Smallest singular value of the Jacobian at different points
$(t_{i},x_{j})$ of $g=(u,u_{x})$ for $u(t,x)=sin(x+t)$. For the upper image,
the derivatives were computed using 2nd order finite differences and, for the
lower image, 7th order finite differences were used.

Thus, we continue with S-FRanCo and check the singular value plot of a feature
library using monomials. This is displayed in Figure 7 for monomials up to
degree 2, which shows that $u_{t}=u_{x}$ is not the unique polynomial PDE
solved by $u$. Indeed one can verify that the function $u$ also solves
$u_{t}=u_{x}+u^{2}+u_{x}^{2}-1$.

Figure 7: Plot of the lowest singular value of the feature matrix consisting
out of all monomials up to degree 2 of $u$ and $u_{x}$, where the derivatives
were computed using finite differences of different orders for
$u(t,x)=sin(x+t)$.

####  5.3.2 Uniqueness

We now continue the investigation of the function
$u:\mathbb{R}^{2}\rightarrow\mathbb{R},u(t,x)=(x+bt)\exp(at)$, with $a=2$ and
$b=3$, which we know from Section 5.1.2 solves only the linear PDE
$u_{t}=au+bu_{x}$. We again sample $u(t,x)$ on the square $[0,10]^{2}$ with
200 equispaced measurements for $t$ and 300 for $x$. This time we focus on the
question, whether it is also the unique analytic PDE solved by $u$. Therefore,
we apply JRC to check if there exists at least one data point for which the
Jacobi matrix has full rank. Figure 8 shows that the Jacobian has full rank at
every point and, thus, $u_{t}=au+bu_{x}$ is the unique analytic PDE solved by
$u$. This can also be seen theoretically, since the image of $(u,u_{x})$ is
$\mathbb{R}\times\mathbb{R}_{>0}$ and has therefore non-zero measure.

Figure 8: Smallest singular value of the Jacobian at different points
$(t_{i},x_{j})$ of $g=(u,u_{x})$ for $u(t,x)=(x+bt)\exp(at)$, with $a=1$ and
$b=2$. For the upper image, the derivatives were computed using 2nd order
finite differences and, for the lower image, 7th order finite differences were
used.

At last, we consider a PDE which is non-algebraic but analytic:

$u_{t}=u_{x}-\frac{u}{u_{x}}\sin(u_{x}).$ (20)

We first show, that Equation (20) is solved by
$u:\mathbb{R}_{>0}\times\mathbb{R}\rightarrow\mathbb{R},u(t,x)=(x+t)v(t)$, for
$t>0$, where
$v:\mathbb{R}_{>0}\rightarrow\mathbb{R},v(t)=\arcsin(\operatorname{sech}(t))$.
We start with computing

$v_{t}(t)=-\frac{\tanh(t)\operatorname{sech}(t)}{\sqrt{1-\operatorname{sech}^{2}(t)}}=-\operatorname{sech}(t)=-sin(v(t)),$
(21)

for $t>0$ and, as $u_{x}=v$, we obtain for all $t>0$ and $x\in\mathbb{R}$

$u_{t}=v(t)-(x+t)v_{t}(t)=u_{x}-\frac{u}{u_{x}}sin(u_{x}).$ (22)

This is well-defined, since $u_{x}(t,x)\neq 0$ holds for $t>0$. This follows
from the fact that $\operatorname{sech}(t)=\frac{2e^{t}}{e^{2t}+1}\in(0,1)$,
for $t>0$, and $\arcsin((0,1))=(0,\pi/2)$. We sample $u(t,x)$ on the square
$[1,5]^{2}$ with 200 equispaced measurements for $t$ and 300 for $x$. We now
ask whether Equation (20) is the unique analytic PDE of the form
$u_{t}=F(u,u_{x})$ which is solved by $u$. For this, we apply the Jacobian
criterion using JRC and check the singular values in Figure 9. Again, we see
no trend towards 0 for the least singular values and, therefore, we deduce
that Equation (20) is the unique analytic PDE solved by $u$.

Figure 9: Smallest singular value of the Jacobian at different points
$(t_{i},x_{j})$ of $u(t,x)=(x+t)\arccos(\operatorname{sech}(-t))$. For the
upper image, the derivatives were computed using 2nd order finite differences
and, for the lower image, 7th order finite differences were used.

This follows also theoretically by Proposition 4, as the image of $(u,u_{x})$
is $\mathbb{R}\times v(\mathbb{R})$ which has non-zero measure.

##  6 Conclusion

Even though the non-uniqueness of PDEs is a vital issue for physical law
learning applications,—to the best of our knowledge—it has never been adressed
in the literature, apart from Rudy et al. [9], see Example 1. Furthermore,
Rudy et al. [9] were only able to resolve the non-uniqueness problem for one
specific equation because they knew in advance which equation they wanted to
learn. That motivated us to theoretically address the question: When does a
continuously differentiable function $u$ solve a unique PDE of the form
$F(u_{\alpha^{1}},...,u_{\alpha^{k}})=\frac{\partial^{n}u}{\partial^{n}t}$,
where $F:\mathbb{R}^{k}\rightarrow\mathbb{R}$ belongs to a specific function
class?

Contrary to our question, standard PDE literature asks if a function is the
unique solution of a PDE. Thus, by the nature of the problem we are
apprehending a completely new field, which implies that we require completely
different branches of mathematics than the ones commonly used in standard PDE
literature. To tackle the uniqueness question we exploit techniques from other
mathematical fields such as algebra, algebraic geometry and analytic geometry.
These tools enabled us to prove uniqueness statements for the most important
classes of PDEs.

As we kept the theory for PDEs as general as possible, we could directly apply
it to different classes of ODEs in Section 3. This section can also be seen as
a guideline on how to obtain uniqueness statements for special cases of
PDEs/ODEs. The considerations in Section 3.2.2 show how complicated achieving
uniqueness for non-autonomous ODEs and PDEs becomes. This implies that one has
to restrict the class of functions $F$ that explicitly depend on $t$ and $x$
in a physically meaningful way to achieve uniqueness.

Besides developing a theoretical understanding of the uniqueness issue, one of
the goals is to make the theory useful for applications. Thus, we aimed for
proving criterions which are numerically possible to verify. Section 4
provides an overview of algorithms available for practitioners who want to
ensure that they learned the unique PDE. These algorithms were then verified
in Section 5, which also supports our theoretical results from the previous
sections.

With this paper we built the foundation for theoretical research in physical
law learning, a field which has so far only been driven by practical
applications. In future work we aim to extend these uniqueness results to
larger sets of PDEs. Furthermore, we intend to add also existence results to
build a proper base for any theoretical study of physical law learning, such
as its robustness and computability. Hopefully, these insights not only foster
our understanding of physical law learning but also help to derive more
powerful and more reliable physical law learning algorithms in the future.

## Acknowledgments

This work of P. Scholl, G. Kutyniok, and H. Boche was supported in part by the
ONE Munich Strategy Forum (LMU Munich, TU Munich, and the Bavarian Ministery
for Science and Art).

G. Kutyniok acknowledges support from the Konrad Zuse School of Excellence in
Reliable AI (DAAD), the Munich Center for Machine Learning (BMBF) as well as
the German Research Foundation under Grants DFG-SPP-2298, KU 1446/31-1 and KU
1446/32-1 and under Grant DFG-SFB/TR 109, Project C09 and the German Federal
Ministry of Education and Research under Grant MaGriDo.

This work of H. Boche was supported in part by the German Federal Ministry of
Education and Research (BMBF) under Grant 16ME0442.

## References

  * [1] Michael Schmidt and Hod Lipson.  Distilling free-form natural laws from experimental data.  Science, 324(5923):81–85, 2009.
  * [2] Hao Xu, Haibin Chang, and Dongxiao Zhang.  Dlga-pde: Discovery of pdes with incomplete candidate library via combination of deep learning and genetic algorithm.  Journal of Computational Physics, 418:109584, 2020.
  * [3] Georg Martius and Christoph Lampert.  Extrapolation and learning equations.  5th International Conference on Learning Representations, ICLR 2017 - Workshop Track Proceedings, 10 2017.
  * [4] Josh Bongard and Hod Lipson.  Automated reverse engineering of nonlinear dynamical systems.  Proceedings of the National Academy of Sciences, 104(24):9943–9948, 2007.
  * [5] Michael D. Schmidt and Hod Lipson.  Age-fitness pareto optimization.  GECCO ’10, page 543–544, New York, NY, USA, 2010. Association for Computing Machinery.
  * [6] Silviu-Marian Udrescu and Max Tegmark.  Ai feynman: A physics-inspired method for symbolic regression.  Science Advances, 6(16):eaay2631, 2020.
  * [7] Silviu-Marian Udrescu, Andrew Tan, Jiahai Feng, Orisvaldo Neto, Tailin Wu, and Max Tegmark.  Ai feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity.  In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 4860–4871. Curran Associates, Inc., 2020.
  * [8] Steven L. Brunton, Joshua L. Proctor, and J. Nathan Kutz.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems.  Proceedings of the National Academy of Sciences, 113(15):3932–3937, 2016.
  * [9] Samuel H. Rudy, Steven L. Brunton, Joshua L. Proctor, and J. Nathan Kutz.  Data-driven discovery of partial differential equations.  Science Advances, 3, 2017.
  * [10] Kathleen Champion, Bethany Lusch, J. Nathan Kutz, and Steven L. Brunton.  Data-driven discovery of coordinates and governing equations.  Proceedings of the National Academy of Sciences, 116(45):22445–22451, 2019.
  * [11] Ali Hasan, João M. Pereira, Robert J. Ravier, Sina Farsiu, and Vahid Tarokh.  Learning partial differential equations from data using neural networks.  ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3962–3966, 2020.
  * [12] Zhao Chen, Yang Liu, and Hao Sun.  Physics-informed learning of governing equations from scarce data.  Nature communications, 12(1):6136, October 2021.
  * [13] Subham S. Sahoo, Christoph H. Lampert, and Georg Martius.  Learning equations for extrapolation and control.  In Proc. \35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden, 2018, volume 80, pages 4442–4450. PMLR, 2018.
  * [14] Zichao Long, Yiping Lu, and Bin Dong.  Pde-net 2.0: Learning pdes from data with a numeric-symbolic hybrid deep network.  Journal of Computational Physics, 399:108925, 2019.
  * [15] Saaketh Desai and Alejandro Strachan.  Parsimonious neural networks learn interpretable physical laws.  2021\.
  * [16] W. M. Thorburn.  Occam’s razor.  Mind, 24(2):287–288, 1915.
  * [17] Linda R. Petzold.  Differential/algebraic equations are not ode’s.  Siam Journal on Scientific and Statistical Computing, 3:367–384, 1982.
  * [18] Paolo Ruffini.  Riflessioni intorno alla soluzione delle equazioni algebraiche generali.  Presso La Società, Modena, 1813.
  * [19] Niels H. Abel.  Mémoire sur les équations algébriques, ou l’on démontre l’impossibilité de la résolution de l’équation générale du cinquième degré.  In Œuvres Complètes de Niels Henrik Abel, Norway, 1881. Grondahl and Son.
  * [20] Azniv Kasparian.  Lectures on Curves, Surfaces and Projective Varieties (A Classical View of Algebraic Geometry) by Mauro C. Beltrametti, Ettore Carletti, Dionisio Gallarati and Giacomo M. Bragadin.  Journal of Geometry and Symmetry in Physics, 18(none):87 – 92, 2010\.
  * [21] Richard Ehrenborg and Gian-Carlo Rota.  Apolarity and canonical forms for homogeneous polynomials.  Eur. J. Comb., 14(3):157–181, may 1993.
  * [22] Steven G Krantz and Harold R Parks.  A primer of real analytic functions.  Springer Science & Business Media, 2002.
  * [23] E.M. Chirka.  Complex Analytic Sets.  Springer Netherland, Netherland, 1 edition, 1989.
  * [24] F. Acquistapace, F. Broglia, and J. F. Fernando.  Some results on global real analytic geometry.  Contemporary Mathematics, 697, 2017.
  * [25] Boris Mityagin.  The zero set of a real analytic function.  Mathematical Notes, 107, 12 2015.
  * [26] Walter Rudin.  Real and Complex Analysis, 3rd Ed. McGraw-Hill, Inc., USA, 1987.
  * [27] Tejinder S. Neelon.  On solutions of real analytic equations.  Proceedings of the American Mathematical Society, 125(9):2531–2535, 1997.
  * [28] Carmen Chicone.  Ordinary Differential Equations with Applications.  Texts in Applied Mathematics. Springer Science+Business Media, USA, 2 edition, 2006.
  * [29] R. Thompson and W. Walter.  Ordinary Differential Equations.  Graduate Texts in Mathematics. Springer New York, 2013.
  * [30] William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery.  Numerical Recipes 3rd Edition: The Art of Scientific Computing.  Cambridge University Press, USA, 3 edition, 2007.
  * [31] Richard L. Burden and Annette M. Burden.  Numerical Analysis.  Youngstown State University, Youngstown, Ohio, USA, 10 edition, 2015.
  * [32] Patrick Morandi.  Field and Galois Theory.  Springer New York, NY, USA, 1 edition, 1996.
  * [33] Serge Lang.  Algebra, volume 211.  Springer Science & Business Media, 2012.
  * [34] S. S. Abhyankar and T. T. Moh.  On analytic independence.  Transactions of the American Mathematical Society, 219:77–87, 1976\.

##  Appendix A Fundamentals of Algebra

In this section we introduce all definitions and results from algebra which
are needed to understand the proofs in Section 2.2. A more thorough treatment
can be found in Morandi [32] and Lang [33].

We start with some basic notation. We denote the _ring of polynomials_ over
$x=(x_{1},...,x_{m})$ with coefficients in the field $K$ by $K[x]$. A
polynomial $p\in K[x]$ is called _irreducible_ over $K$ if there exist no non-
constant polynomials $p_{1},p_{2}\in K[x]$ such that $p(x)=p_{1}(x)p_{2}(x)$.
The _rational function field_ $K(x)\coloneqq\\{p(x)/q(x):p,q\in K[x],\;q\neq
0\\}$ is the field of rational functions with variables $x$ and coefficients
in $K$.

The next definitions we need concern field extensions.

######  Definition 9.

Let $F$ and $K$ be fields with $F\subset K$. Then, we call $K$ a _field
extension_ of $F$, denoted by $K/F$. We call an element $\alpha\in K$
_algebraic_ over $F$ if there exists a nonzero polynomial $P\in F[x]$ with
$P(\alpha)=0$. If every element of $K$ is algebraic over $F$, we say that $K$
is algebraic over $F$ and $K/F$ is called an _algebraic extension_.

We will use this these definitions to define algebraic functions as algebraic
elements over $\mathbb{C}(x)$.

######  Definition 10.

A field $F$ is called _algebraically closed_ if every polynomial $p\in F[x]$
with degree at least one has a root in $F$. An algebraic field extension
$F\subset K$ is called the _algebraic closure_ of $F$ if it is algebraically
closed. We denote $\overline{F}=K$. Now we can define an _algebraic function_
$f$ over $\mathbb{C}$ as an element of the algebraic closure of the rational
function field $\mathbb{C}(x)$, i.e., $f\in\overline{\mathbb{C}(x)}$.

A short note on polynomials and algebraic functions: Formally, polynomials and
polynomial function are two different quantities. A polynomial $p\in K[x]$,
for $K$ some field, is a purely symbolic element from a polynomial ring which
can be considered a function via the _evaluation homorphism_ , which maps a
polynomial to the corresponding polynomial function $p_{K}:K\rightarrow K$.
This differentiation is especially important if the evaluation homomorphism is
not injective. This happens, for example, for $K=\mathbb{F}_{2}=\\{0,1\\}$ as
$p(x)=x^{2}+x$ is unequal to the zero polynomial, even though
$p_{K}(\alpha)=0$ for all $\alpha\in\mathbb{F}_{2}$. For $K=\mathbb{R}$ or
$K=\mathbb{C}$, however, the evaluation homorphism is injective and,
therefore, we do not have to differentiate between polynomials and polynomial
functions. This way we can connect the two definitions of algebraic functions,
Definition 3 and Definition 10.

Many of the theorems on algebraic dependence of algebraic functions are about
algebraic dependence over $\mathbb{C}$. However, we are interested in real-
valued functions and algebraic depedence over $\mathbb{R}$. That we can still
use those results follows from the next lemma.

######  Lemma 2.

Let $f_{i}:\mathbb{R}\rightarrow\mathbb{R}$, $1\leq n$ be real-valued
functions. Then the functions $f_{i}$ are algebraically independent over
$\mathbb{R}$ if and only if $f_{i}$ are algebraically independent over
$\mathbb{C}$.

###### Proof.

Let $p\in\mathbb{C}[x]$ and decompose it as $p=p_{1}+ip_{2}$ with
$p_{1},p_{2}\in\mathbb{R}[x]$. Then, we observe that $p(f_{1},...,f_{n})=0$ is
equivalent to $p_{1}(f_{1},...,f_{n})=0=p_{2}(f_{1},...,´f_{n})$. This shows
that $0\neq p\in\mathbb{C}[x]$ with $p(f_{1},...,f_{n})=0$ exists if and only
if there exists $0\neq q\in\mathbb{R}[x]$ with $q(f_{1},...,f_{n})=0$. ∎

##  Appendix B Analytic Independence

A logical consideration is to try if Theorem 1 and 2 extend to analytic
functions and analytic independence. This would be a powerful extension as it
might be a plausible assumption in many cases that $u$ is an analytic
functions. Also, the proof in Ehrenborg and Rota [21] shows that it suffices
to prove that analytic dependence defines a matroid similar to how algebraic
dependence defines a matroid. Analytic analogues of Theorem 1 and 2 would
follow similar as for algebraic functions and algebraic PDEs as shown in
Section 2.2 and in Ehrenborg and Rota [21]. Let us now define analytic
dependence [34], before we discuss why a proof as for algebraic function and
polynomial/algebraic PDEs cannot exist for analytic functions and analytic
PDEs.

######  Definition 11.

We call functions $f_{1},...,f_{q}:\mathbb{C}^{p}\rightarrow\mathbb{C}$
_analytically dependent over $\mathbb{C}$_ if there exists a non-zero analytic
function $P\in C^{\omega}(\mathbb{C}^{q})\backslash\\{0\\}$ such that
$P(f_{1}(x_{1},..,x_{p}),...,f_{q}(x_{1},..,x_{p}))=0$. If no such $P$ exists,
we call $f_{1},...,f_{q}$ _analytically independent over $\mathbb{C}$_.

Following the proof in Ehrenborg and Rota [21] the analytic extension of
Theorem 2 would rely on the extension of Theorem 1. However, this approach is
not feasible as the following example shows the existence of two analytic
functions in one variable which are analytically independent.

######  Example 2.

Let $a,b\in\mathbb{R}$ be linearly independent over $\mathbb{Q}$. Then there
does not exist a non-zero analytic function $F:U\rightarrow\mathbb{C}$, for
$U\subset\mathbb{C}^{2}$ open and $B_{1}(0)\subset U$, which fulfills
$F(e^{iat},e^{ibt})=0$ for all $t\in\mathbb{R}$. Thus, the functions $t\mapsto
e^{iat}$ and $t\mapsto e^{ibt}$ are analytically independent.

###### Proof.

Let $F:U\rightarrow\mathbb{C}$ be any analytic function on some open set
$U\subset\mathbb{C}^{2}$ with $B_{1}(0)\subset U$. Furthermore, let
$F(x,y)=\sum_{k,l=1}^{\infty}c_{kl}x^{k}y^{l}$ be its Taylor series around
$(x,y)=0$ which we know converges on the entire set $U\subset\mathbb{R}^{2}$.
Let $f(t)=e^{iat}$ and $g(t)=e^{ibt}$ for $t\in\mathbb{R}$. Then,
$F(f(t),g(t))=\sum_{k,l=1}^{\infty}c_{kl}e^{i(ak+bl)t}=0$ for any
$t\in\mathbb{R}$. This implies

$0=||\sum_{k,l=1}^{\infty}c_{kl}e^{i(ak+bl)t}||_{2}^{2}=\langle\sum_{k,l=1}^{\infty}c_{kl}e^{i(ak+bl)t},\sum_{k,l=1}^{\infty}c_{kl}e^{i(ak+bl)t}\rangle=\sum_{k,l=1}^{\infty}|c_{kl}|^{2}.$
(23)

The last step follows from the fact that $a$ and $b$ are linearly independent
over $\mathbb{Q}$. Thus, $ak+bl$ are different for each $k,l\in\mathbb{Z}$
and, therefore, $(e^{i(ak+bl)x})_{k,l}$ is an orthonormal set. This yields
that $c_{kl}=0$ for all $k,l\in\mathbb{N}$ and, thus, $F=0$. This finishes the
proof. ∎