# Ensemble forecasts in reproducing kernel Hilbert space family: dynamical
systems in Wonderland

Benjamin Dufée  Bérenger Hug  Étienne Mémin  Gilles Tissot  INRIA/IRMAR Campus
universitaire de Beaulieu, Rennes, 35042 Cedex, France

###### Abstract

A methodological framework for ensemble-based estimation and simulation of
high dimensional dynamical systems such as the oceanic or atmospheric flows is
proposed. To that end, the dynamical system is embedded in a family of
reproducing kernel Hilbert spaces with kernel functions driven by the
dynamics. This family is nicknamed Wonderland for its appealing properties. In
Wonderland the Koopman and Perron-Frobenius operators are unitary and
uniformly continuous. This property warrants they can be expressed in
exponential series of diagonalizable bounded infinitesimal generators. Access
to Lyapunov exponents and to exact ensemble based expressions of the tangent
linear dynamics are directly available as well. Wonderland enables us the
devise of strikingly simple ensemble data assimilation methods for trajectory
reconstructions in terms of constant-in-time linear combinations of trajectory
samples. Such an embarrassingly simple strategy is made possible through a
fully justified superposition principle ensuing from several fundamental
theorems.

##  1 Introduction

Forecasting the state of geophysical fluids has become of paramount importance
for our daily life either at short time scale, for weather prediction, or at
long time scale, for climate study. There is in particular a strong need to
provide reliable likely scenarios for probabilistic forecasts and uncertainty
quantification. To that end, methods based on linear combinations of an
ensemble of numerical simulations are becoming more and more ubiquitous.
Ensemble forecasting and data assimilation techniques, where the linear
coefficients are further constrained through partial observations of the
system and Gaussian stochastic filtering strategies, are good examples
developed in meteorological centers for weather forecast operational routine
use. However, although efficient in practice, this linear superposition
principle of solutions remains theoretically questionable for nonlinear
dynamics. Very recently, such techniques have been further coupled with
machine learning to characterize surrogate dynamical models from long time-
series of observations. This includes neural networks, analog forecasting,
reservoir computing, kernel methods to name only a few of the works of this
wide emerging research effort [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]. To characterize
a surrogate model of the dynamics, these techniques are built upon principles
such as delay coordinates embedding, empirical basis functions, or sparse
linear representations. The difficulty here is to get sufficiently long data
series to represent the manifold associated to a given observable of the
system as well as to exhibit an intrinsic representation of the whole system
that does not depend on parameters such as the discrete time-stepping, the
integration time window, or the initial conditions. Looking for such intrinsic
representation often leads to focus (at least theoretically) on reversible
ergodic systems.

Spectral representation of the Koopman operator [11] or of its infinitesimal
generator constitutes one of the most versatile approaches to achieve this
objective as they enable in theory to extract intrinsic eigenfunctions of the
system dynamics. Koopman operator or its dual, the Perron-Frobenious operator,
have attracted a great deal of attention in the operator-theoretic branch of
ergodic theory [12] as well as in data science for the study of data driven
modeling of dynamical systems. Several techniques, starting from [13] and
[14], have been proposed to estimate such spectral representation. Most of
them rely either on longtime harmonic averages [14, 15, 16, 17] or on finite-
dimensional approximations of the Koopman operator such as the dynamic modes
representation and its extensions [18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28]
or on Galerkin approximation and delay embedding [29, 30, 31]. These methods
enable performing spectral projection from long time series of measured
observables of the system. The practical linear nature of the compositional
Koopman operator, defined for any observable, $f$, of a dynamical system
$\Phi_{t}(x)$ with initial condition $x$, as $(U_{t}f)(x):=f\circ\Phi_{t}(x)$,
is nevertheless hindered by its infinite dimension. Due to this, it often
exhibits a spectrum with a continuous component [32] and is not amenable to
diagonalization nor to any sparsity assumption. Finite dimensional
approximations and the direct use of matrix algebra numerical methods raises
immediate questions on the pertinence of the associated estimation. To tackle
these difficulties several techniques combining regularization,
compactification of the generators, and reproducing kernel Hilbert space
(RKHS) to work in spaces of smooth functions have been recently proposed [16,
17, 33]. They are possibly associated with forecast strategies [34, 35, 36]
and transfer operator estimations [37, 38]. For data driven specification of
dynamical systems RKHS or kernel methods have been also explored in a more
direct way in terms of advanced Gaussian process regression techniques [39, 5,
40]. Kernel methods have been found particularly efficient in this context
when coupled with a learning strategy for the kernel [6].

Working with RKHS brings convenient regular basis functions together with
approximation convergence guaranties. However, the dynamical systems need
still to be ergodic, and invertible to ensure that the Koopman operator has a
simple point spectrum and an associated system of orthogonal basis
eigenfunctions [12]. Measure invariance and invertibility warrants that the
associated Koopman/Perron-Frobenius operators are dual unitary operators [12].
Note nevertheless that state-of-the-art estimation techniques based on Dynamic
Mode Decomposition or Hankel matrix [41, 42] require very long time series to
converge to actual Koopman eigenvalues [43]. This is all the more detrimental
for high-dimensional state-spaces. Recent reviews on modern Koopman analysis
applications and their related algorithmic developments from measurements time
series can be found in [44, 45, 46]. Another limitation is related to the
definition of a kernel (and hence of a RKHS) independent of the dynamics to
describe the system’s observable, which de facto boils down to the strong
assumption of an invariance of the RKHS under the Koopman groups. This
condition requires to define the RKHS from the subspace spanned by the Koopman
operator eigenfunctions, which excludes in general classical parametric
kernels. Note that in particular biais in the estimation of the Kopman
eigenvalues have been observed for kernel-based methods based on finite
dimensional approximation and sparsity assumptions [47, 48].

In this study we will explore the problem from a different angle. Instead of
working with long time series of data, we will work with a set of realizations
of a given dynamical system. The objective will be to learn this dynamics
locally in the phase space to synthesize in a fast way new trajectories of the
system through a theoretically well justified superposition principle. This
superposition principle will in particular enable us to fully justify the
linear combination assumption employed in many ensemble data assimilation
filters even though the dynamics is nonlinear. To perform an efficient
estimation inherently bound to the dynamics, we propose to embed the ensemble
members in a set of time-evolving RKHS defining a family of spaces. This
setting is designed to deal with large-scale systems such as oceanic or
meteorological flows, for which it is out of the question to explore the whole
attractor (if any), neither to run very long time simulations calibrated on
real-world data of a likely transient system due to climate change. The
present work relies on the fact that the kernel functions, usually called
feature maps, between the native space and the RKHS family are transported by
the dynamical system. This creates, at any time, an isometry between the
evolving RKHS and the RKHS at a given initial time. Instead of being attached
to a system state, the kernel is now fully associated with the dynamics and
new ensemble members embedded in the RKHS family at a given time can be very
simply forecast at a further time. This framework displays striking properties
– which affectionately makes us give to the RKHS family the nickname of
Wonderland. For invertible dynamical systems the Koopman and Perron-Frobenius
operators in Wonderland are unitary, like in $L^{2}$. However, they are in
addition uniformly continuous, with bounded generators, and diagonalizable. As
such they can be rigorously expanded in exponential form and their
eigenfunctions provide an intrinsic orthonormal basis system related to the
dynamics. This set of analytical properties immediately brings practical
techniques to estimate Koopman eigenfunctions or essential features of the
dynamics. Evaluations of these Koopman eigenfunctions at the ensemble members
are directly available. Finite-time Lyapunov exponents associated with each
Koopman eigenfunction are easily accessible on the RKHS family as well and can
be used to get practical predictability time of the system. Expressions
relating Koopman eigenfunctions to the tangent linear dynamics and its adjoint
are also immediately available for data assimilation. These expressions,
ensuing from the fundamental results listed previously, correspond in their
simpler forms to the ensemble-based finite difference approximations
intensively used in ensemble methods. Beyond that, Wonderland leads finally to
a theoretically well grounded superposition principle (for nonlinear dynamics)
enabling the devise of embarrassingly simple methods for trajectory
reconstructions in terms of constant-in-time linear combinations of trajectory
samples.

The embedding of the trajectories in a time-evolving RKHS which follows the
dynamical system can be related to concepts of local kernels [49], embedding
in dynamical coordinates [50] or embedding of changing data over the parameter
space as described in Coifman and Hirn [51]. In our case, we further exploit
the relations with the Koopman operator together with an associated isometry,
which allows us to define simple practical schemes for the setting of the
time-evolving kernels and the modal information that can be extracted from
them.

This paper is organized as follows. We start briefly recalling the definition
and some properties of RKHS in the next section. These definitions are
complemented by A which details and adapts several known results to our
functional setting. The embedding of the Koopman and Perron-Frobenius
operators in Wonderand are then presented in sections 3 and 4. This is
followed by the enunciation of our main results. The main theorems and
properties are demonstrated in section 4 and in the appendixes. Numerical
results for a quasi-geostrophic barotropic ocean model in an idealistic north
Atlantic configuration will be presented in section 5 to support the
pertinence of the proposed approach.

##  2 Reproducing kernel Hilbert spaces

A RKHS, $\mathcal{H}$, is a Hilbert space of smooth complex functions
$f:E\mapsto\mathbb{C}$ defined over a non empty abstract set $E$ on which a
positive definite kernel and a kernel-based inner product,
$\langle\bm{\cdot}\,,\,\bm{\cdot}\rangle_{{\cal H}}$ can be defined.
Throughout this work, we will consider the set $E$ to be a locally convex
topological compact set as we will work with an integral compact operator from
which a convenient functional description can be set and as we will deal with
a RKHS of functions that are Gateaux differentiable – through an assumption of
smooth enough kernels. RKHS possess remarkable properties, which make their
use very appealing in statistical machine learning applications and
interpolation problems [52, 53]. The kernels from which they are defined have
a so called “reproducing property”.

######  Definition 1 (Reproducing kernel)

Let ${\cal H}$ be a Hilbert space of $\mathbb{C}$-valued functions defined on
a non-empty compact locally convex topological space $E$. A map $k:E\times
E\mapsto\mathbb{C}$ is called a reproducing kernel of ${\cal H}$ if it
satisfies the following principal properties: $\forall x\in E,$

  * $\bullet$

membership of the evaluation function $\;k(\bm{\cdot},x)\in{\cal H}$,

  * $\bullet$

reproducing property $\forall f\in{\cal
H},\;\bigl{\langle}f,k(\bm{\cdot},x)\bigr{\rangle}_{{\cal H}}=f(x)$.

□

The last property, provides an expression of the kernel as
$k(x,y)=\overline{k(y,x)}=\bigl{\langle}k(\bm{\cdot},y),k(\bm{\cdot},x)\bigr{\rangle}_{{\cal
H}}$, which is Hermitian – with $\overline{\bullet}$ denoting complex
conjugate –, positive definite and associated with a continuous evaluation
function $\delta_{x}f=\bigl{\langle}f,k(\bm{\cdot},x)\bigr{\rangle}_{{\cal
H}}=f(x)$. The continuity of the Dirac evaluation operator is indeed sometimes
taken as a definition of RKHS. By the Moore-Aronszajn theorem [54], the kernel
$k$ defines uniquely the RKHS, ${\cal H}$, and vice versa. The set spanned by
the feature maps ${\rm{Span}}\\{k(\bm{\cdot},x),x\in E\\}$, is dense in
$({\cal H},\|\bm{\cdot}\|_{{\cal H}})$. We note also that useful kernel
closure properties enable to define kernels through operations such as
addition, Schur product, and function composition [52]. Besides, RKHS can be
meaningfully characterized through integral operators, leading to an isometry
with $L^{2}_{\mathbb{C}}(E,\nu)$ the space of square integrable functions
defined on a compact metric space $E$ with finite measure $\nu$ [53].

### Integral kernel operators

Let $k:E\times E\mapsto\mathbb{C}$ be $C^{(1,1)}(E\times E)$, Hermitian, and
positive definite, with continuous derivative on both arguments, and let the
map ${\cal L}_{k}:L_{\mathbb{C}}^{2}(E,\nu)\mapsto L_{\mathbb{C}}^{2}(E,\nu)$
be defined as:

$\bigl{(}{\cal L}_{k}f\bigr{)}(x)=\int_{E}k(x,y)f(y)\,\nu({\mathrm{d}}y).$ (1)

This operator, which must be understood within the composition with the
continuous inclusion $i:C^{0}(E,\mathbb{C})\hookrightarrow
L_{\mathbb{C}}^{2}(E,\nu)$, can be shown to be well defined, positive, compact
and self-adjoint [53]. The range of this operator is assumed to be dense in
$L_{\mathbb{C}}^{2}(E,\nu)$. From Mercer’s theorem [55], the feature maps
$k(\bm{\cdot},x)$ span a RKHS defined through the eigenpairs
$(\mu_{i},\varphi_{i})_{i\in\mathbb{N}}$ of the kernel $k$ :

${\cal H}:=\bigl{\\{}f\in
L_{\mathbb{C}}^{2}(E,\nu),\;f=\displaystyle\sum_{i=0}^{\infty}a_{i}\varphi_{i}\;:\;\displaystyle\sum_{i}\dfrac{\,|a_{i}|^{2}}{\mu_{i}}<\infty\bigr{\\}},$
(2)

with no null eigenvalues since we have assumed that the kernel range is dense
in $L_{\mathbb{C}}^{2}(E,\nu)$. The rank of the kernel (number of – non-zero –
eigenvalues) corresponds to the dimension of ${\cal H}$, and will potentially
be infinite in this work, as $E$ will be a subspace of functions. The RKHS
${\cal H}$ is a space of smooth functions with decreasing high frequency
coefficients. In fact, for all $u\in E$, there exists a constant $C>0$ for
which $\|\partial_{u}f\|_{{L_{\mathbb{C}}^{2}(E,\nu)}}\leq
C\,\left\|f\right\|_{{{\cal H}}}$ (Theorem 7 – A.2), where the derivative
denotes the Gateau directional derivative of function $f$ in the direction $u$
defined as

$\forall x,u\in E,\partial_{u}f{(x)}=\lim_{\epsilon\to
0}\frac{1}{\epsilon}\bigl{(}f(x+\epsilon u)-f(x)\bigr{)}.$

We may also define uniquely a square-root symmetric isometric bijective
operator ${\cal L}_{k}^{\scriptscriptstyle{1/2}}$ between
$L_{\mathbb{C}}^{2}(E,\nu)$ and ${\cal H}$ (A.3). This operator enables to go
from $L_{\mathbb{C}}^{2}(E,\nu)$ to ${\cal H}$ by increasing the functions
regularity while its inverse lowers the function regularity by bringing them
back to $L_{\mathbb{C}}^{2}(E,\nu)$. Both operators are bounded. The injection
$j:{\cal H}\mapsto L^{2}_{\mathbb{C}}(E,\nu)$ is continuous and compact
(Theorem 8 – A.3) and $j({\cal H})$ is assumed to be dense in
$L_{\mathbb{C}}^{2}(E,\nu)$ (Prop. 11 – A.3, see Remark 14 – A.3 to relax this
hypothesis). These statements are precisely recalled in A.

##  3 Dynamical systems on a RKHS family

We consider an invertible nonsingular dynamical system $X(t)=\Phi_{t}(X_{0})$,
defined from a continuous flow, $\Phi_{t}$ on a compact invariant metric phase
space manifold, $\Omega$, (i.e. $\Phi^{-1}_{t}(\Omega)=\Omega$, $\forall
t\in\mathbb{R}^{+}$) of time evolving vector functions over a spatial support
$\Omega_{x}$. The functions
$X:\mathbb{R}^{+}\times\Omega_{x}\mapsto\mathbb{R}^{d}$ with $X\in C^{p}$,
$p\geq 1$, are solutions of the following $d$-dimensional differential system:

$\left\\{\begin{aligned} \partial_{t}X(t,x)&={\cal
M}\bigl{(}X(t,x)\bigr{)},\text{ with }X(t)\in\Omega,\forall t>0,\\\
X(0,x)&=X_{0}(x).\end{aligned}\right.$ (3)

The nonlinear differential operator ${\cal M}{:\Omega\to\Omega}$ is assumed
$C^{1}{(\Omega)}$, and in particular its linear tangent expression defined as
the Gateaux derivative:

$\partial_{X}{\cal M}(X)\delta X=\lim_{\beta\to
0}\frac{1}{\beta}\biggl{(}{\cal
M}\bigl{(}X(t,x)+\beta{\delta}X(t,x)\bigr{)}-{\cal
M}\bigl{(}X(t,x)\bigr{)}\biggr{)}$

is such that $\sup_{X\in\Omega}\partial_{X}{\cal M}(X)<\infty$ (since $\Omega$
is compact). We consider the measure space $(\Omega_{x},\mathrm{d})$ where
$\mathrm{d}$ is the Lebesgue measure on $\Omega_{x}$. We note as
$L^{2}(\Omega_{x},\mathbb{R}^{d})$ the space of the square-integrable
functions on $(\Omega_{x},\mathrm{d})$ and
$L^{2}(\Omega_{x},\mathbb{R}^{d}):=\\{f=(f_{1},..,f_{d}):\Omega_{x}\mapsto\mathbb{R}^{d}\;:\;\text{for
all}\;1\leq i\leq d\quad f_{i}\in L^{2}(\Omega_{x},\mathbb{R})\\}$ the vector
space of square integrable functions on $\Omega_{x}$. We note
$\|\bm{\cdot}\|_{{L^{2}}}$ the norm associated with
$L^{2}(\Omega_{x},\mathbb{R}^{d})$ and given by
$\|f\|^{2}_{{L^{2}}}:=\sum_{i=1}^{d}\|f_{i}\|^{2}_{L^{2}(\Omega_{x},\mathbb{R})}$
for all $f\in L^{2}(\Omega_{x},\mathbb{R}^{d})$. The set $\Omega$ is included
in $L^{2}(\Omega_{x},\mathbb{R}^{d})$. The system (3) is assumed to admit a
finite invariant measure $\nu$ on $\Omega$ (note that from invertibility the
measure is also nonsingular with $\nu(\Phi^{-1}_{t})(A)=0$, $\forall
A{\subset}\Omega$ such that $\nu(A)=0$). The system’s observables are square
integrable measurable complex functions with respect to measure $\nu$. They
belong to the Hilbert space $L^{2}_{\mathbb{C}}(\Omega,\nu)$ with the inner
product
$\langle\bm{\cdot}\,,\,\bm{\cdot}\rangle_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}$
given for $f$ and $g\in L^{2}_{\mathbb{C}}(\Omega,\nu)$ by

$\langle
f\,,\,g\rangle_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}:=\int_{\Omega}f(y)\,\overline{\,g(y)\,}\;\nu(\mathrm{d}y).$

Depending on the context $X(t)$ or $X_{t}$ will denote either an element of
$\Omega$ or the function $X(t)\colon\Omega_{x}\to\mathbb{R}^{d}$ such that
$x\mapsto X(t,x)$.

In this work we place ourselves in a situation where $n$ realizations (called
ensemble members) of this dynamical system, $\\{X^{(i)}_{t},i=1,\ldots,n\\}$,
generated from a compact set of different initial conditions
$\Omega_{0}=\\{X^{(i)}_{0},i=1,\ldots,n\\}$, are available up to time $t$. For
all time $t\geq 0$, we denote by $\Omega_{t}\subset\Omega\subset
L^{2}(\Omega_{x},\mathbb{R}^{d})$ the space defined as
$\Omega_{t}:=\Phi_{t}(\Omega_{0})$. Furthermore, the set $\Omega_{0}$ of
initial conditions will be assumed to be composed of points that uniquely
characterize all the dynamical system trajectories and are rich enough so that
$\bigcup_{t\geq 0}(\Omega_{t})=\Omega$. Hence, by this, each point of the
manifold, $\Omega$, is hence assumed to be uniquely characterized by an
initial condition and the integration of the dynamical system over a given
time $t$. In other words, for any $X\in\Omega$, there exists a unique initial
condition $X_{0}\in\Omega_{0}$ and a unique time $t\in\mathbb{R}_{+}$ such
that $X=\Phi_{t}(X_{0})$. All the sets $\Omega_{t}$ will be assumed to be
locally convex and compact.

Defining at each time $t$, from ensemble $\Omega_{t}$, a positive Hermitian
kernel $k_{\Omega_{t}}:\Omega_{t}\times\Omega_{t}$, there exists a unique
associated RKHS ${\cal H}_{t}$. In the following for the sake of concision the
kernels $k_{\Omega_{t}}$, will be denoted by $k_{t}$ to refer to the
dependence on the set $\Omega_{t}$. For all $t\geq 0$, we will use the
notation $X_{t}=\Phi_{t}(X_{0})$ and $\left({\cal
H}_{t},\langle\,\bm{\cdot}\,,\,\bm{\cdot}\,\rangle_{{\cal
H}_{t}},\|\bm{\cdot}\|_{{}_{{\cal H}_{t}}}\right)$ for the RKHS associated
with the kernel $k_{t}$ defined on $\Omega_{t}\times\Omega_{t}$. The kernels
are assumed to be $C^{(2,2)}(\Omega_{t}\times\Omega_{t})$ and as a consequence
as shown in appendix A, the associated feature maps have derivatives in ${\cal
H}_{t}$. The RKHS ${\cal H}_{t}$ for all time $t\geq 0$ forms a family of
Hilbert spaces of complex functions, each of them equipped with their own
inner product $k_{t}(Y_{t},X_{t})$, for all functions $X_{t},Y_{t}\in$
$\Omega_{t}$. At any time, a measurable function of the system state, usually
referred to as an observable, $f$, belonging to the RKHS ${\cal H}_{t}$ can be
described as a linear combination of the feature maps
$\\{k_{t}(\bm{\cdot},X^{(i)}_{t}),i=1,\ldots,n,\;t\in[0,T]\\}$ and any
observable trajectory $\bigl{(}f(X_{t})\bigr{)}_{t\geq 0}$ can be locally
described as the linear combinations of time varying features maps of
$\bigl{(}{\cal H}_{t}\bigr{)}_{t\geq 0}$. As it will be shown, the features
maps of this RKHS family can be expressed on a time evolving orthonormal
systems of basis functions, connected to each other through an exponential
form and given by the eigenfunctions of the infinitesimal generator of a
“Koopman-like” operator defined on the RKHS family. The RKHS family
${\mathcal{W}}=({\cal H}_{t})_{t\geq 0}$ is here nicknamed Wonderland because
of its very appealing properties. In the following, we shall present a summary
of the mathematical results associated to the Koopman operator in Wonderland.

##  4 Koopman operator in Wonderland

So far, we did not give any precise definition of the kernels associated to
the RKHS family ${\cal H}_{t}$ yet. These kernels are defined from a known a
priori initial kernel, $k_{0}:\Omega_{0}\times\Omega_{0}$, as:

######  Definition 1 (${\cal H}_{t}$ kernel)

The kernel $k_{t}$ associated to the RKHS ${\cal H}_{t}$ are defined as

$\forall X_{t},Y_{t}\in\Omega_{t},\quad
k_{t}\bigl{(}Y_{t}\,,\,X_{t}\bigr{)}=k_{0}\bigl{(}\Phi^{-1}_{t}(Y_{t})\,,\,\Phi^{-1}_{t}(X_{t})\bigr{)},$
(4)

where $k_{0}:\Omega_{0}\times\Omega_{0}$ is a given kernel. □

These kernels can also be equivalently defined introducing the operators,
${\cal U}_{t}$, operating on the feature maps. An isometric property of this
operator on the RKHS family, enables us to fully define the kernels along time
in the same way as the previous definition. The operator ${\cal U}_{t}:{\cal
H}_{0}\mapsto{\cal H}_{t}$ defined such that

${\cal
U}_{t}k_{0}(\bm{\cdot},X_{0})=k_{t}\bigl{(}\bm{\cdot},\Phi_{t}(X_{0})\bigr{)},$
(5)

transports the kernel feature maps on the RKHS family by composition with the
system’s dynamics. This operator, and more specifically its infinitesimal
operator, will enable us to define the feature maps of ${\cal H}_{t}$ from an
initial set of feature maps on ${\cal H}_{0}$. As will be fully detailed in
section 4.1, we will see that the operator ${\cal U}_{t}$ is indeed directly
related to the restriction on ${\cal H}_{t}$ of the adjoint of the Koopman
operator $U_{t}$ on a bigger RKHS space, ${\cal H}$, associated to a fixed
kernel defined on the whole phase space, $\Omega$. As ${\cal U}_{t}$
propagates forward the second argument of the feature maps, it is referred in
the following to as the Koopman kernel operator in Wonderland. Indeed, it will
be pointed out that, for any $f\in{\cal H}$, and any $X_{0}\in\Omega_{0}$,

$\displaystyle U_{t}f(X_{0})=f(X_{t})$
$\displaystyle=\bigl{\langle}R_{t}f\,,\,k_{t}(\bm{\cdot}\,,\,X_{t})\bigr{\rangle}_{{\cal
H}_{t}}$ $\displaystyle=\bigl{\langle}R_{t}f\,,\,{\cal
U}_{t}k_{0}(\bm{\cdot}\,,\,X_{0})\bigr{\rangle}_{{\cal H}_{t}},$

where $U_{t}$ denotes the Koopman operator operating on
$L^{2}_{\mathbb{C}}(\Omega,\nu)$, and $R_{t}f$ denotes the restriction of $f$
to $\Omega_{t}\subset\Omega$. This expression exhibits a kernel expression of
the Koopman operator definition and formally, at this point, the operator
${\cal U}_{t}$ can be thought as a kernel expression of the Koopman operator.

The global kernel $k$ (respectively the associated RKHS) is tightly bound to
the time evolving kernels $k_{t}$ (respectively $({\cal H}_{t})_{t\geq 0}$).
The restriction on ${\cal H}$ of the Koopman operator $U_{t}$ and its adjoint
the Perron-Frobenius $P_{t}$ exhibit some remarkable properties. As
classically, the operators $U_{t}$ and $P_{t}$ are unitary in
$L^{2}_{\mathbb{C}}(\Omega,\nu)$ (Prop. 1), but they have the striking
property to be uniformly continuous in $L^{2}_{\mathbb{C}}(\Omega,\nu)$ (i.e.
with bounded generators – Theorem 2). As such, they can be expanded in an
uniformly converging exponential series. Nevertheless, it must be outlined
that the fixed kernel $k(x,y)$ and consequently ${\cal H}$ are in practice
only partially accessible as they are defined on the whole manifold of the
dynamics and such expansion cannot be directly used. A local representation of
the RKHS family ${\mathcal{W}}$ is on the other hand much easier to infer in
practice through the time evolution of ensembles $\Omega_{t}$ and kernels
$k_{t}$. As we shall see, the operator ${\cal U}_{t}$ on ${\mathcal{W}}$
inherits many properties from $U_{t}$ and in particular, a related form of
exponential series expansion (Theorem 1).

The Koopman kernel operator in Wonderland, ${\cal U}_{t}$, defines an isometry
from ${\cal H}_{0}$ to ${\cal H}_{t}$ (Theorem 3)

$\displaystyle\biggl{\langle}k_{t}\bigl{(}\bm{\cdot},\\!\Phi_{t}(Y_{0})\bigr{)},k_{t}\bigl{(}\bm{\cdot},\\!\Phi_{t}(X_{0})\bigr{)}\biggr{\rangle}_{\\!\\!{\cal
H}_{t}}\\!\\!$
$\displaystyle=\\!\biggl{\langle}k_{0}\bigl{(}\bm{\cdot},\\!Y_{0}\bigr{)},k_{0}\bigl{(}\bm{\cdot},\\!X_{0}\bigr{)}\biggr{\rangle}_{\\!\\!{\cal
H}_{0}},$ (6) $\displaystyle
k_{t}\bigl{(}\Phi_{t}(X_{0}),\Phi_{t}(Y_{0})\bigr{)}\\!$
$\displaystyle=\\!k_{0}(X_{0},Y_{0}).$

This isometry ensues obviously directly from definition 1. But, it can be also
guessed from definition (5) and the unitarity of ${\cal U}_{t}$ (Theorem 3,
Prop.6), inherited from the unitarity in $L^{2}_{\mathbb{C}}(\Omega,\nu)$ of
the Koopman operator and of its adjoint, the Perron-Frobenious operator. This
property is of major practical interest as it allows us to define the kernels
of the Wonderland RKHS family from a given initial kernel fixed by the user.
The kernels remain constant along the system trajectories. Alternatively, an
explicit form of the feature maps can be obtained from an adjoint transport
equation associated to the infinitesimal generator of the Koopman operator in
Wonderland. Nevertheless, the isometry is far more straightforward to use to
set the kernel evolution. This isometry will reveal also very useful for the
data assimilation and trajectories reconstuction problems investigated in
section 5. Strikingly, we have even more than this kernel isometry. An
evolution operator $A_{U\\!,\,t}:{\cal H}_{t}\to
L_{\mathbb{C}}^{2}(\Omega_{t},\nu)$, associated to the infinitesimal generator
of Koopman operator $U_{t}$ can also be defined as

$A_{U\\!,\,t}\;k_{t}(\bm{\cdot},X_{t}):=\partial_{{\cal
M}(\bm{\cdot})}\,k_{t}(\bm{\cdot},X_{t}),$ (7)

where $\partial_{u}k_{t}(\bm{\cdot},X_{t})$ stands for the Gateaux directional
derivative along $u\in\Omega$ of function $k(\bm{\cdot},X_{t})$. This operator
that will be shown to be bounded (Prop.7) and skew-symmetric (Prop.8) for the
inner product of $L^{2}_{\mathbb{C}}(\Omega,\nu)$ plays the role of an
infinitesimal generator on ${\cal W}$ and enables us expressing an exponential
expansion of ${\cal U}_{t}$.

######  Theorem 1 (Wonderland spectral representation)

For a measure preserving invertible dynamical system of the form (3), assuming
a $C^{{(}1,1{)}}$ initial kernel, the evolution operator $A_{U\\!,\,t}:{\cal
H}_{t}\mapsto L_{\mathbb{C}}^{2}(\Omega_{t},\nu)$ defined in (7), and which
corresponds to the infinitesimal generator of the Koopman operator in the
Wonderland RKHS family $\bigl{(}{\cal H}_{t}\bigr{)}_{t\geq 0}$ (5) can be
diagonalized, at any time $t\geq 0$, by an orthonormal basis
$(\psi^{t}_{\ell})_{\ell}$ of ${\cal H}_{t}$ such that

$A_{U\\!,\,t}\psi^{t}_{\ell}(X_{t})=\lambda_{\ell}{{j\circ}}\psi^{t}_{\ell}(X_{t}),$

where $j$ is the injection $j:{\cal H}_{t}\mapsto
L^{2}_{\mathbb{C}}(\Omega,\nu)$. We have in addition the following relation
between the orthonormal basis systems along time:

$\forall t\geq
0\;\;\psi^{t}_{\ell}(X_{t})=\exp(t\;\lambda_{\ell})\psi^{0}_{\ell}(X_{0}),$
(8)

with $X_{t}=\Phi_{t}(X_{0})$. Furthermore, the purely imaginary eigenvalues
$(\lambda_{\ell})_{\ell}$ do not depend on time. □

This theorem, which constitutes our main result, provides us a time-evolving
system of orthonormal bases of the RKHS family. It brings us the capability to
express any observable of the system in terms of bases that are intrinsically
linked to the dynamics and related to each other by an exponential relation.
The eigenvalues of $A_{U\\!,\,t}$ are purely imaginary since this operator is
skew-symmetric in $L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$. Remarkably, the
eigenvalues of each $A_{U\\!,\,t}$ do not depend on time and are connected
with the same covariant eigenfunctions (in the sense of (8)). These
eigenfunctions correspond to restrictions of eigenfunctions of the
infinitesimal generator of Koopman/Perron-Frobenius operators defined on
${\cal H}$. The proofs of these results are fully detailed in the next
section.

The frozen in time spectrum, also referred to as iso-spectral property in the
litterature, connects directly Wonderland representation with Lax pair theory
of integrable system [56]. Here the Lax pair is provided by the operator
$A_{U\\!,\,t}$ at a given time and the evolution equation (8). The expression
of the dynamical system in Wonderland (i.e. through the feature maps) provides
a compatility condition for the Lax pair. As the existence of a Lax pair is
directly related to integrable systems, it means that the Wonderland
representation formally retains only the integrable part of a (non-necessary
integrable) dynamical system. This relation between Koopman operator and
integrable systems has been already put forward in several works [44, 57].
Here an ensemble characterization of such a relation is exhibited.

The kernel isometry (6) (Theorem 3) and the Koopman spectral representation in
Wonderland (Theorem 1) constitute fundamental results enabling us to build
very simple ensemble-based trajectory reconstructions for new initial
conditions without the requirement of resimulating the dynamical system.
Amazingly, the Wonderland family of RKHS together with the Koopman isometry
allows to define a system’s trajectory as a constant-in-time linear
combination of the time varying RKHS feature maps. Several of such
reconstruction techniques, based on this fully justified superposition
principle, will be shown in section 5. We will in particular demonstrate the
potential of this framework with a simple and very cheap data assimilation
technique in the context of very scarce data in space and time. In the next
section we demonstrate the different properties related to Wonderland.

###  4.1 Proofs on the properties of the Koopman operator in Wonderland and of
the Wonderland spectral theorem

As explained in the previous section the RKHS family, $\bigl{(}{\cal
H}_{t}\bigr{)}_{t\geq 0}$, does not have a good topology to work with. We need
first to start defining a “big” set with a better topology and which
encompasses all the RKHS ${\cal H}_{t}$. On this big encompassing set, we
shall then define a Koopman operator enabling us to study properly the Koopman
operator in Wonderland.

####  4.1.1 Construction of the “big” encompassing set ${\cal H}$

The phase space $\Omega$ corresponds to the set generated by the value of the
dynamical system at a given time $t$. We note hence that $\Omega_{t}$ is a
subset of $\Omega$. Each point of $\Omega$ designates a phase-space point
$X=\Phi_{t}(X_{0})$ uniquely defined from time $t$ and initial condition
$X_{0}\in\Omega_{0}$. In order to define the RKHS ${\cal H}$, let us define
from the kernel $k_{t}:\Omega_{t}\times\Omega_{t}$ a symmetric positive
definite map $k:\Omega\times\Omega$.

######  Definition 2 (${\cal H}$ kernel)

For all $X=\Phi_{r}(X_{0}),\,Y=\Phi_{s}(Y_{0})\in\Omega$, with
$X_{0},\,Y_{0}\in\Omega_{0}$ we define

$k(X\,,\,Y)=k_{0}\bigl{(}X_{0}\,,\,Y_{0}\bigr{)}\ell(r,s)=k_{t}\bigl{(}\Phi_{t}\bigl{(}X_{0})\bigr{)}\,,\,\Phi_{t}\bigl{(}Y_{0})\bigr{)}\bigr{)}\ell(r,s)\quad\forall
t\geq 0,$ (9)

where $\ell:\mathbb{R}_{+}\times\mathbb{R}_{+}\to\mathbb{R}$ is a symmetric
kernel defined, for all $r,s\in\mathbb{R}{{}^{+}}$ by

$\ell(r,s)=\varphi(r-s),$ (10)

where $\varphi$ is a twice-derivable even function such that $\varphi(0)=1$. □

The positivity and symmetry of kernel $k$ ensues from the property of kernels
$k_{0}$ and $\ell$, which are assumed to be valid kernels. Kernel $k$ inherits
the regularity conditions of $k_{0}$ and $\ell$ and is
$C^{(2,2)}(\Omega\times\Omega)$ as well. In the trivial case where
$\varphi=1$, then comparing any pair of points on two trajectories would be
equivalent to compare the initial conditions, which would result in a quite
poor kernel, and degeneracy issues. In order to enrich the kernel structure,
one can think of $\varphi$ as a regularized Dirac distribution, or a time
Gaussian distribution, that will discriminate the points of the phase space
that are reached at different times.

By the Moore-Aronszajn theorem, there exists a unique RKHS called $\left({\cal
H},\langle\,\bm{\cdot}\,,\,\bm{\cdot}\,\rangle_{{\cal
H}},\|\bm{\cdot}\|_{{}_{{\cal H}}}\right)$ with kernel $k$. We note that in
practice the full knowledge of the phase-space is completely unreachable.
Again, we therefore stress the fact that the setting of this encompassing RKHS
${\cal H}$ has only a theoretical purpose. The RKHS ${\cal H}$ can be
connected to each RKHS of the family ${\mathcal{W}}$ through extension and
restriction operators denoted $E_{t}$ and $R_{t}$ respectively, and defined as
follows. For all $t\geq 0$, let

$\begin{array}[]{ccccc}E_{t}:&&{\cal H}_{t}&\mapsto&{\cal H}\\\
&&k_{t}(\,\bm{\cdot}\,,X_{t})&\mapsto&\;k(\,\bm{\cdot}\,,\,X_{t})\end{array}$
(11)

and extend this definition by linearity on ${\rm
Span}\\{k_{t}(\,\bm{\cdot}\,,X_{t})\,:\,X_{t}\in\Omega_{t}\\}$. Then, by
density the function $E_{t}(f)$ is defined for all $f\in{\cal H}_{t}$. The
restriction,

$\begin{array}[]{ccccc}R_{t}:&&{\cal H}&\mapsto&{\cal H}_{t}\\\
&&k(\,\bm{\cdot}\,,\,X)=k(\,\bm{\cdot}\,,\,\Phi_{r}(X_{0}))&\mapsto&\;k(\,\bm{\cdot}\,,\,X)\restriction_{{}_{\Omega_{t}}}=k_{t}\bigl{(}\,\bm{\cdot}\,,\,\Phi_{t}(X_{0})\bigr{)}\ell(t,r)\\\
\end{array}$ (12)

is defined similarly for $g\in{\cal H}^{0}={\rm
Span}\\{k\bigl{(}\,\bm{\cdot}\,,X\bigr{)}\;:\;X\in\Omega\\}$ and extended by
density in ${\cal H}$ by the Moore-Aronszajn theorem [54]. The expression of
$R_{t}(f)$ for $f\in{\cal H}$ is further specified in remark 17 (B). The
extension map is built in such a way that each RKHS ${\cal H}_{t}$ of the
family is included in the “big” encompassing RKHS ${\cal H}$.

In B, we list several useful properties of the restriction and extension
operators. Namely, $E_{t}$ and $R_{t}$ are both isometries (Prop.12); they
form an adjoint pair (Prop.13); the restriction is continuous in
$L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$ (Prop.14). We define now the Koopman
operator on the encompassing RKHS ${\cal H}$.

####  4.1.2 Koopman operators on ${\cal H}$

For all $t\geq 0$, we consider the Koopman operator on ${\cal H}$ defined by

${U}_{t}(f)(X):=f\circ{\Phi}_{t}({X})=f\bigl{(}\Phi_{t}(X)\bigr{)},\text{ for
all }f\in{\cal H}.$ (13)

Since ${\cal H}$ is dense in
$(L^{2}_{\mathbb{C}}(\Omega,\nu),\|\bm{\cdot}\|_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}})$
(by proposition 11 (A.3), the operator $U_{t}$ can be continuously extended on
$L^{2}_{\mathbb{C}}(\Omega,\nu)$ and to avoid notations inflation we keep
noting $U_{t}$ this extension. We first study
$U_{t}:L^{2}_{\mathbb{C}}(\Omega,\nu)\mapsto L^{2}_{\mathbb{C}}(\Omega,\nu)$
with the $L^{2}_{\mathbb{C}}(\Omega,\nu)$ topology. The family $(U_{t})_{t\geq
0}$ is a strongly continuous semi-group on
$(L^{2}_{\mathbb{C}}(\Omega,\nu),\|\bm{\cdot}\|_{L^{2}_{\mathbb{C}}(\Omega,\nu)})$
since $t\mapsto\Phi_{t}(\bm{\cdot})$ is continuous on $\mathbb{R}^{+}$. As the
feature maps are functions of ${\cal H}$, it can be noticed that for all
$X_{r}\in\Omega$

$U_{t}\,[\,k(\,\bm{\cdot}\,,\,{X}_{r})\,]\;=\;k\bigl{(}\,{\Phi}_{t}(\bm{\cdot})\,,\,{X}_{r}\bigr{)}.$
(14)

This corresponds to a natural expression of the Koopman operator for any
function $g\in{\cal H}^{0}$, extended then by density to ${\cal H}$. Yet
another useful equivalent expression of the Koopman operator is available for
the feature maps. We have, for any points
$X=\Phi_{r}(X_{0}),Y=\Phi_{s}(Y_{0})\in\Omega$

$U_{t}\left[\,k(\,\bm{\cdot}\,,\,X)\,\right](Y)=k\big{(}{\Phi}_{t}(Y)\,,\,X\big{)}=k\big{(}\Phi_{t+s}\bigl{(}Y_{0})\bigr{)}\,,\,\Phi_{r}\bigl{(}X_{0})\bigr{)}\big{)}=k_{0}(Y_{0},X_{0})\ell(t+s,r).$
(15)

From the properties of the time kernel $\ell$, we get

$\ell(t+s,r)=\varphi(t+s-r)=\varphi(s-(r-t))=\ell(s,r-t),$ (16)

which leads to

$k\big{(}{\Phi}_{t}(Y)\,,\,X\big{)}=k_{0}(Y_{0},X_{0})\ell(t+s,r)=k_{0}(Y_{0},X_{0})\ell(s,r-t)=k\big{(}\Phi_{s}\bigl{(}Y_{0})\bigr{)}\,,\,\Phi_{r-t}\bigl{(}X_{0})\bigr{)}\big{)}=k\big{(}Y,\Phi_{t}^{-1}(X)\big{)}.$
(17)

and hence

$U_{t}\left[\,k(\,\bm{\cdot}\,,\,X)\,\right](Y)=k\big{(}Y,\Phi_{t}^{-1}(X)\big{)}.$
(18)

As the previous equality is true for all $Y\in\Omega$, this implies that for
all $X\in\Omega$

$U_{t}\,[\,k(\,\bm{\cdot}\,,\,X)\,]\;=\;k\bigl{(}\,\bm{\cdot}\,,\,{\Phi}_{t}^{-1}({X})\bigr{)}.$
(19)

This dual formulation of the kernel expression of the Koopman operator is
intrinsically linked to the definition of the kernel $k$. This dual expression
will be of central interest in the following as it enables us to formulate the
time evolution of the feature maps in terms of the Koopman operator $U_{t}$
and its adjoint at any time $t\geq 0$.

######  Remark 1 (Transport of the kernel $k$)

For all $X=\Phi_{r}(X_{0})\in\Omega$ and $t\geq 0$, by definition of the
Koopman operator on the feature maps and (19),
$U_{t}\left[\,k(\,\bm{\cdot}\,,\,{X})\right]$ has two expressions and we
obtain

$U_{t}\left[\,k(\,\bm{\cdot}\,,\,{X})\right]=k\bigl{(}\,{\Phi}_{t}(\bm{\cdot})\,,\,{X}\,\bigr{)}\;=\;k\bigl{(}\,\bm{\cdot}\,,\,{\Phi}_{t}^{-1}({X})\bigr{)}\;.$

□

The next remark provides a useful commutation property between $U_{t}$ and the
kernel integral operator $\mathcal{L}_{k}$ defined in equation (1) or of its
unique symmetric square-root $\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}$
defined from the square-root of the kernel eigenvalues (see A.3 for a precise
definition in the general case). Note that in the case of ${\cal H}$, the
kernel integral operator is indeed a complex object which hides a time
dependency.

######  Remark 2 (Commutation between
$\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}$ (or $\mathcal{L}_{k}$) and $U_{t}$
)

For all ${X}\in\Omega$, we have

$\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\circ
U_{t}\,[k(\,\bm{\cdot}\,,\,{X})]\,=\,U_{t}\circ\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\,[k(\,\bm{\cdot}\,,\,{X})].$

□

###### Proof

By (14), we have $\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\circ
U_{t}\,[k(\,\bm{\cdot}\,,\,{X})]\,=\,\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\,[k(\,{\Phi}_{t}(\bm{\cdot})\,,\,{X})]$.
Then we obtain, for all ${Y}\in\Omega$

$\left(\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\circ
U_{t}\,[k(\,\bm{\cdot}\,,\,{X})]\,\right)({Y})=\left(\,\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\,[k(\,{\Phi}_{t}(\bm{\cdot})\,,\,{X})]\right)(Y)=[\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\,k(\,\bm{\cdot}\,,\,{X})]\bigl{(}{\Phi}_{t}({Y})\bigr{)}\,=\,U_{t}\,[\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\,k(\,\bm{\cdot}\,,\,{X})]({Y})\,,$

by definition of the Koopman operator since
$\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\,k(\,\bm{\cdot}\,,\,{X})\in{\cal
H}$. $\Box$■

This commutation property that ensues directly from the compositional nature
of the Koopman operator allows us to write immediately the equality

$\mathcal{L}_{k}\circ
U_{t}\,[k(\,\bm{\cdot}\,,\,\bm{X})]\,=\,U_{t}\circ\mathcal{L}_{k}\,[k(\,\bm{\cdot}\,,\,\bm{X})].$
(20)

By linearity these properties extend to all functions of ${\cal H}$. Let us
note that the function
$\mathcal{L}_{k}^{-\scriptscriptstyle{1/2}}\,k(\,\bm{\cdot}\,\,,{X})$, for all
${X}\in\Omega$, does not necessarily belong to ${\cal H}$, therefore this
proof cannot be applied to $\mathcal{L}_{k}^{-\scriptscriptstyle{1/2}}$.

The next proposition shows the Koopman operator defined on RKHS ${\cal H}$ is
unitary in $L^{2}_{\mathbb{C}}(\Omega,\nu)$, which is a classical property of
the Koopman operator in $L^{2}$ for measure preserving invertible system.

######  Proposition 1 (Unitarity of the Koopman operator in
$L^{2}_{\mathbb{C}}(\Omega,\nu)$)

The map $U_{t}$ :
$(L^{2}_{\mathbb{C}}(\Omega,\nu),\|\bm{\cdot}\|_{L^{2}_{\mathbb{C}}(\Omega,\nu)})$
$\mapsto$
$(L^{2}_{\mathbb{C}}(\Omega,\nu),\|\bm{\cdot}\|_{L^{2}_{\mathbb{C}}(\Omega,\nu)})$
is unitary for all $t\geq 0$. □

###### Proof

Since the dynamical system $(\Phi_{t})_{t\geq 0}$ is measure preserving, (i.e.
$\forall A\in\Omega,\nu(A)=\nu\bigl{(}\Phi_{t}^{-1}(A)\bigr{)}$, with
$\Phi_{t}^{-1}(A)$ denoting the pre-image of $\Phi_{t}$) and left invariant
(i.e. $\Phi_{t}(\Omega)=\Omega$, with $Ran(\Phi_{t})=\Omega$), the Koopman
operator $U_{t}$ is an isometry: for all $f$ and $g\in
L^{2}_{\mathbb{C}}(\Omega,\nu)$,

$\langle
U_{t}(f)\,,\,U_{t}(g)\rangle_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}=\langle
f\,,\,g\rangle_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}\,.$

Besides, it can be noticed that $U_{t}({\cal H}^{0})$, with ${\cal H}^{0}={\rm
Span}\\{\,k(\,\bm{\cdot}\,,\,X)\,:\,{X}\in\Omega\\}$, is a subset of the range
of $U_{t}:L^{2}_{\mathbb{C}}(\Omega,\nu)\mapsto
L^{2}_{\mathbb{C}}(\Omega,\nu)$, $U_{t}({\cal H}^{0})={\rm
Span}\\{\,k\bigl{(}\,\bm{\cdot}\,,\,{\Phi}_{t}({X})\bigr{)}\,:\,{{X}\in\Omega\\}}$
and we have the following inclusions

$U_{t}({\cal H}^{0})\subset Ran(U_{t})\subset
L^{2}_{\mathbb{C}}(\Omega,\nu)\,.$

We show now that $U_{t}({\cal H}^{0})$ is dense in
$\bigl{(}L^{2}_{\mathbb{C}}(\Omega,\nu),\|\bm{\cdot}\|_{L^{2}_{\mathbb{C}}(\Omega,\nu)}\bigr{)}$.
Let $f\in L^{2}_{\mathbb{C}}(\Omega,\nu)$ be such that
$\bigl{\langle}f,k\bigl{(}\,\bm{\cdot},{\Phi}_{t}({X})\bigr{)}\bigr{\rangle}_{L^{2}_{\mathbb{C}}(\Omega,\nu)}\\!=0$
for all ${X}\in\Omega$. Then, from the definition of $\mathcal{L}_{k}$, it
means that $({\cal L}_{k}f)[{\Phi}_{t}({X})]=0$. The injectivity of
$\mathcal{L}_{k}$ ensures that $f\circ\Phi_{t}=0$, so $U_{t}f=0$. As $U_{t}$
is an isometry, it is in particular injective, which eventually leads to
$f=0$. By a consequence of Hahn-Banach theorem this result shows that the
range of $U_{t}$ is dense in $L^{2}_{\mathbb{C}}(\Omega,\nu)$ and Proposition
1 is proved. $\Box$■

Proposition 1, shows that the Koopman operator $U_{t}$ is invertible and that
its inverse in $L^{2}_{\mathbb{C}}(\Omega,\nu)$ is $U^{*}_{t}$. Denoting by
$P_{t}$ the operator defined by $P_{t}:=U^{*}_{t}$, for all $f$ and $g\in
L^{2}_{\mathbb{C}}(\Omega,\nu)$ we have
$\langle\,U_{t}(f)\,,\,g\rangle_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}=\langle
f\,,\,P_{t}(g)\,\rangle_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}$. The family
$(P_{t})_{t}$ is a strongly continuous semi-group on
$\bigl{(}L^{2}_{\mathbb{C}}(\Omega,\nu),\|\bm{\cdot}\|_{L^{2}_{\mathbb{C}}(\Omega,\nu)}\bigr{)}$.
This operator is referred to as the Perron-Frobenius operator. For all $t\geq
0$, the Perron-Frobenius operator $P_{t}$ verifies for all $X\in\Omega$

$P_{t}[k\bigl{(}\Phi_{t}(\bm{\cdot}),X)\bigr{]}=k(\,\bm{\cdot}\,,X).$ (21)

From Remark 1, we can write a more explicit expression of $P_{t}$ on the
featutre maps: for all $X\in\Omega$,

$P_{t}\big{[}k(\bm{\cdot},X)\big{]}=k\big{(}\bm{\cdot},\Phi_{t}(X)\big{)}.$
(22)

Informally, if we see the function $k(\bm{\cdot},X)$ as an atom of the
measure, then its expression at a future time is provided by (22), which
corresponds well to the idea that the Perron-Frobenius operator advances in
time the density.

As previously stated the Koopman operator $U_{t}$ is an isometry in
$L^{2}_{\mathbb{C}}(\Omega,\nu)$. The next result asserts that $U_{t}$ is also
an isometry in ${\cal H}$.

######  Proposition 2 (Isometric relation of the kernel)

For all $X$ and $Y\in\Omega$, we have
$k\big{(}{\Phi}_{t}(X)\,,{\Phi}_{t}(Y)\big{)}=k(X\,,\,Y)$. □

###### Proof

This results follows immediately from the kernel definition: for all
$X=\Phi_{r}(X_{0}),Y=\Phi_{s}(Y_{0})\in\Omega$, we have

$k\big{(}{\Phi}_{t}(X)\,,{\Phi}_{t}(Y)\big{)}=k\big{(}\Phi_{t+r}(X_{0}),\Phi_{t+s}(Y_{0})\big{)}=k_{0}(X_{0},Y_{0})\ell(t+r,t+s)=k_{0}(X_{0},Y_{0})\ell(r,s)=k(X,Y).$
(23)

■

Remark 1 on the application of the operator $U_{t}$ to
$k(\,\bm{\cdot}\,,\,X)$, with $X\in\Omega$, shows that applying the flow,
$\Phi_{t}$, on the first variable of $k(\,\bm{\cdot}\,,\,\bm{\cdot}\,)$. is
equivalent to applying $\Phi_{t}^{-1}$ to the second variable and vice-versa.
Consequently, for all $f\in L^{2}_{\mathbb{C}}(\Omega,\nu)$ and $X\in\Omega$,
we have $\langle
f\,,\,k\bigl{(}\,\bm{\cdot}\,,\,\Phi_{t}(X)\bigr{)}\,\rangle_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}\;=\langle
f\,,\,k\bigl{(}\,\Phi_{t}^{-1}(\bm{\cdot})\,,\,X\bigr{)}\,\rangle_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}\;=\;\langle
f\;,\;P_{t}\,k(\,\bm{\cdot}\,,\,X)\rangle_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}$
and by definition of the adjoint, we obtain

$\bigl{\langle}f\,,\,k\bigl{(}\,\bm{\cdot}\,,\,\Phi_{t}(X)\bigr{)}\,\bigr{\rangle}_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}=\;\bigl{\langle}U_{t}(f)\;,\;\,k(\,\bm{\cdot}\,,\,X)\bigr{\rangle}_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}.$
(24)

We already knew that this equality was right for $f\in\mathcal{H}$ and for the
inner product in $\mathcal{H}$, namely

$\big{\langle}f,k\big{(}\bm{\cdot},\Phi_{t}(X)\big{)}\big{\rangle}_{\mathcal{H}}=f\big{(}\Phi_{t}(X)\big{)}=U_{t}f(X)=\big{\langle}U_{t}(f),k(\bm{\cdot},X)\big{\rangle}_{\mathcal{H}}.$
(25)

Equation (24) provides a weak (in the sense that the
$L^{2}_{\mathbb{C}}(\Omega,\nu)$-inner product against a feature map is no
longer the evaluation function) formulation of the transport of any observable
in $L^{2}_{\mathbb{C}}(\Omega,\nu)$ by the flow.

In order to further exhibit several analytical results on the Koopman operator
$(U_{t})_{t\geq 0}$ in $L^{2}_{\mathbb{C}}(\Omega,\nu)$, we introduce in the
following its infinitesimal generator.

####  4.1.3 Koopman infinitesimal generator

We will note as $\bigl{(}A_{{}_{U}},\mathcal{D}(A_{{}_{U}})\bigr{)}$ the
infinitesimal generator of the strongly continuous semigroup $(U_{t})_{t\geq
0}$ on $L^{2}_{\mathbb{C}}(\Omega,\nu)$ and its domain. As the Koopman and
Perron-Frobenius operators are adjoint in $L^{2}_{\mathbb{C}}(\Omega,\nu)$
their infinitesimal generators are also adjoint of each other with possibly
their own domain. The following lemma characterizes first the Perron-Frobenius
infinitesimal generator $A_{{}_{P}}$ and its domain
$\mathcal{D}\left(A_{{}_{P}}\right)$.

######  Lemma 1 (Perron-Frobenius infinitesimal generator)

The Perron-Frobenius infinitesimal generator is the unbounded operator,
$\left(A_{{}_{P}},\mathcal{D}\left(A_{{}_{P}}\right)\right)$ defined by

$\mathcal{D}\left(A_{{}_{P}}\right)=\left\\{f\in
L^{2}_{\mathbb{C}}(\Omega,\nu)\;:\;x\mapsto\partial_{{\cal M}(x)}f(x)\;\in
L^{2}_{\mathbb{C}}(\Omega,\nu)\right\\}\;\text{and
}\;A_{{}_{P}}f:=\,-\partial_{{\cal
M}(\bm{\cdot})}\,f(\bm{\cdot})\quad\text{for}\quad
f\in\mathcal{D}\left(A_{{}_{P}}\right),$

where ${\cal M}$ denotes the differential tendency of the system dynamics (3)
and $\partial_{u}f$ stands for the directional derivative of $f$ along
$u\in\Omega$. □

###### Proof

See C. ■

For all $t\geq 0$, $P_{t}$ is the adjoint of $U_{t}$ in
$L^{2}_{\mathbb{C}}(\Omega,\nu)$. The Koopman infinitesimal generator in
$L^{2}_{\mathbb{C}}(\Omega,\nu)$ is consequently given by

$\mathcal{D}(A_{{}_{U}})=\mathcal{D}(A^{*}_{{}_{P}})\quad\text{and}\quad
A_{{}_{U}}=A_{{}_{P}}^{*}.$ (26)

The Koopman and Frobenius-Perron operators are unitary in
$L^{2}_{\mathbb{C}}(\Omega,\nu)$, as $\mathcal{D}\left(A_{{}_{U}}\right)$ is
dense in $L^{2}_{\mathbb{C}}(\Omega,\nu)$, by Stone’s theorem, their
infinitesimal generators are therefore skew symmetric for
$\langle\,\bm{\cdot}\,,\,\bm{\cdot}\,\rangle_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}$
and we have

$\mathcal{D}\left(A^{*}_{{}_{P}}\right)=\mathcal{D}\left(A_{{}_{P}}\right)\quad\text{and}\quad
A_{{}_{P}}^{*}=-A_{{}_{P}},$ (27)
$\mathcal{D}\left(A^{*}_{{}_{U}}\right)=\mathcal{D}\left(A_{{}_{U}}\right)\quad\text{and}\quad
A_{{}_{U}}^{*}=-A_{{}_{U}}.$ (28)

The adjoint has to be understood in the topology of
$L^{2}_{\mathbb{C}}(\Omega,\nu)$. The two next propositions – well known in
$L^{2}_{\mathbb{C}}(\Omega,\nu)$ for invertible measure preserving systems –,
summarize (26), (27) and (28).

######  Proposition 3 (Koopman infinitesimal generator)

The Koopman infinitesimal generator of $(U_{t})_{t}$ in
$L^{2}_{\mathbb{C}}(\Omega,\nu)$ is the unbounded operator,
$\bigl{(}A_{{}_{U}},\mathcal{D}(A_{{}_{U}})\bigr{)}$, defined by

$\mathcal{D}\left(A_{{}_{U}}\right)\,=\,\left\\{f\in
L^{2}_{\mathbb{C}}(\Omega,\nu)\;:\;x\mapsto\partial_{{\cal M}(x)}f(x)\;\in
L^{2}_{\mathbb{C}}(\Omega,\nu)\,\right\\}\;\text{and
}\;A_{{}_{U}}f=\partial_{{\cal
M}(\bm{\cdot})}f(\bm{\cdot})\quad\text{for}\quad
f\in\mathcal{D}\left(A_{{}_{U}}\right).$

□

######  Proposition 4 (Skew symmetry of the generators)

The Koopman infinitesimal generator $A_{U}$ is skew-symmetric in
$L^{2}_{\mathbb{C}}(\Omega,\nu)$ . □

We set now a continuity property of the Koopman infinitesimal generator on a
subspace of $\mathcal{D}(A_{{}_{U}})$. Let $f\in{\cal H}$, by theorem 6 (A.2),
${x\mapsto\partial_{{\cal M}(x)}f(x)}\in L^{2}_{\mathbb{C}}(\Omega,\nu)$, and
through Cauchy-Schwarz inequality we get that $f\in\mathcal{D}(A_{{}_{U}})$
since the real valued function ${\cal M}$ belongs to
$L^{2}_{\mathbb{C}}(\Omega,\nu)$ and consequently ${\cal
H}\subset\mathcal{D}(A_{{}_{U}})$.

######  Theorem 2 (Continuity of the Koopman generator on ${\cal H}$)

The restriction of the Koopman infinitesimal generator

$A_{{}_{U}}:\left({\cal H}\,,\,\|\bm{\cdot}\|_{{}_{\cal
H}}\right)\mapsto\left(L^{2}_{\mathbb{C}}(\Omega,\nu)\,,\,\|\bm{\cdot}\|_{L^{2}_{\mathbb{C}}(\Omega,\nu)}\right)$

is continuous. □

###### Proof

For all $f\in{\cal H}$ using the Cauchy-Schwarz inequality as above and
theorem 7 (A.2) we have

${\|A_{{}_{U}}f\|_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}=\|\partial_{{\cal
M}(\bm{\cdot})}\,f(\bm{\cdot})\|_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}\,\leq
C\;\|f\|_{{}_{{\cal H}}}.}$

$\Box$■

Through the dual expression of the Koopman operator (remark 1) the
infinitesimal generator $A_{{}_{U}}$ provides a dynamical system specifying
the time evolution of the feature maps as, for all $X\in\Omega$, the feature
map $k(\,\bm{\cdot}\,,\Phi_{t}(X))$ verifies

$\partial_{t}\,U_{t}\,k(\,\bm{\cdot}\,,\,X)={\partial_{t}\,k\bigl{(}\,\Phi_{t}(\bm{\cdot})\,,\,X\bigr{)}=A_{{}_{U}}\,k\bigl{(}\,\Phi_{t}(\bm{\cdot})\,,\,X\bigr{)}\,}.$
(29)

We have the following useful differentiation formulae.

######  Proposition 5 (Differentiation formulae)

For all $f\in{\cal H}$ (with $f=E_{t}g,\,g\in{\cal H}_{t}$) and
$X_{t}\in\Omega_{t}$ we have

$\langle{\cal
L}_{k}\circ\partial_{t}f\;,\;k(\,\bm{\cdot}\,,\,X_{t})\rangle_{{}_{\cal
H}}=-\langle f\;,\;{\cal
L}_{k}\circ\partial_{t}\,k(\,\bm{\cdot}\,,\,X_{t})\rangle_{{}_{\cal H}}.$

□

###### Proof

As ${\cal L}_{k}\circ A_{{}_{U}}f$ belongs to ${\cal H}$, we have ${\cal
L}_{k}\circ A_{{}_{U}}f(X_{t})={\cal L}_{k}\circ\partial_{t}f(X_{t})$ and

$\langle{\cal
L}_{k}\circ\partial_{t}f\;,\;k(\,\bm{\cdot}\,,\,X_{t})\rangle_{{}_{\cal
H}}=\langle{\cal L}_{k}\circ
A_{{}_{U}}f\;,\;k(\,\bm{\cdot}\,,\,X_{t})\rangle_{{}_{\cal H}}=\;\langle{\cal
L}^{\scriptscriptstyle{1/2}}_{k}\circ A_{{}_{U}}f\;,\;{\cal
L}^{-\scriptscriptstyle{1/2}}_{k}\,k(\,\bm{\cdot}\,,\,X_{t})\rangle_{L^{2}_{\mathbb{C}}(\Omega,\nu)}.$

As ${\cal L}^{\scriptscriptstyle{1/2}}_{k}$ is self-adjoint in
$L^{2}_{\mathbb{C}}(\Omega,\nu)$ and $A_{{}_{U}}$ is skew-symmetric in
$L^{2}_{\mathbb{C}}(\Omega,\nu)$, we obtain

$\langle{\cal
L}_{k}\circ\partial_{t}f\;,\;k(\,\bm{\cdot}\,,\,X_{t})\rangle_{{}_{\cal
H}}=\langle
A_{{}_{U}}f\,,\,k(\,\bm{\cdot}\,,\,X_{t})\rangle_{L^{2}_{\mathbb{C}}(\Omega,\nu)}=-\langle
f\;,\;A_{{}_{U}}\,k(\,\bm{\cdot}\,,\,X_{t})\rangle_{L^{2}_{\mathbb{C}}(\Omega,\nu)}=-\langle
f\;,\;\partial_{t}\,k(\,\bm{\cdot}\,,\,X_{t})\rangle_{L^{2}_{\mathbb{C}}(\Omega,\nu)}.$

As $\langle
f\;,\;\partial_{t}\,k(\,\bm{\cdot}\,,\,X_{t})\rangle_{L^{2}_{\mathbb{C}}(\Omega,\nu)}=\langle
f\;,\;\mathcal{L}_{k}\circ\partial_{t}\,k(\,\bm{\cdot}\,,X_{t})\rangle_{{}_{\cal
H}}$, this complete the proof. $\Box$■

As already outlined, the whole phase space, $\Omega$, and the global embedding
RKHS, ${\cal H}$, defined on it are both completely inaccessible for high
dimensional state spaces. Instead of seeking to reconstruct this global RKHS,
we will work in the following with time-varying “local” RKHS spaces built from
a small (w.r.t. the phase space dimension) ensemble of initial conditions. The
set of these time-varying spaces forms the Wonderland family. To express the
time evolution of the features maps associated to the Wonderland RKHS family
we now define an appropriate expression of the Koopman operator on this
family.

####  4.1.4 Derivation of Koopman operator expression in ${\mathcal{W}}$

To fully specify the Koopman operators in Wonderland, we rely on the family of
extension, restriction mappings $(E_{t})_{t\geq 0}$ and $(R_{t})_{t\geq 0}$
(11, 12) relating the “big” encompassing RKHS ${\cal H}$ to the family of
time-evolving RKHS ${\cal H}_{t}$ and use the Koopman operators
$U_{t}:L^{2}_{\mathbb{C}}(\Omega,\nu)\mapsto L^{2}_{\mathbb{C}}(\Omega,\nu)$
for $t\geq 0$. The adjoints $P_{t}$ will also be very helpful as well as
Remark 1 on the dual relation of the application of the flow on the global
kernel $k$.

From now on, we note $X_{t}:=\Phi_{t}(X_{0})$ for all $X_{0}\in\Omega_{0}$
with $X_{t}$ belonging to $\Omega_{t}$. We define the Koopman operator in
Wonderland by ${\cal U}_{t}:=R_{t}\circ P_{t}\circ E_{0}$ for all $t\geq 0$.
For all $X_{0}\in\Omega_{0}$, we have

$\mathcal{U}_{t}\left[\,k_{0}(\,\bm{\cdot}\,,\,X_{0})\,\right]\,=\,R_{t}\,\circ
P_{t}\left[\,k(\,\bm{\cdot}\,,X_{0})\,\right]\,=\,R_{t}\,k(\,\bm{\cdot}\,,X_{t})\,\,=\,k_{t}(\bm{\cdot}\,,\,X_{t})\,,$
(30)

where the second equality is due to (22), and the third equality holds true
from the definition of $R_{t}$ and the fact that $\ell(t,t)=1$. The Koopman
operator in Wonderland, $\mathcal{U}_{t}:{\cal H}_{0}\mapsto{\cal H}_{t}$,
transports the kernel feature maps on the RKHS family by composition with the
system’s dynamics. It inherits some of the nice properties of the Perron-
Frobenius operator defined on the encompassing global RKHS. As shown by the
following theorem, proposition 2 remains valid for the family of kernels
$(k_{t})_{t\geq 0}$ and $\mathcal{U}_{t}$ is still unitary in the sense of the
following theorem.

######  Theorem 3 (Koopman RKHS isometry)

The Koopman operator on the RKHS family defines an isometry from ${\cal
H}_{0}$ to ${\cal H}_{t}$: for all $X_{0}$ and $Y_{0}\in\Omega_{0}$

$\\!\\!\\!\bigl{\langle}\mathcal{U}_{t}\,k_{0}(\,\bm{\cdot}\,,\,\\!Y_{0})\;,\;\mathcal{U}_{t}\,k_{0}(\,\bm{\cdot}\,,\,\\!X_{0})\bigr{\rangle}_{{\cal
H}_{t}}\\!\\!\;=\;\\!\bigl{\langle}k_{0}(\,\bm{\cdot}\,,\,\\!Y_{0})\;,\;k_{0}(\,\bm{\cdot}\,,\,\\!X_{0})\bigr{\rangle}_{{\cal
H}_{0}}.$

The range of $\mathcal{U}_{t}:{\cal H}_{0}\mapsto{\cal H}_{t}$ is dense in
${\cal H}_{t}$. □

###### Proof

Upon applying equations (67 of Prop. 12, B), which states that
$R_{t}:\mathcal{H}\rightarrow\mathcal{H}_{t}$ is an isometry, we have

$\bigl{\langle}\mathcal{U}_{t}\,k_{0}(\,\bm{\cdot}\,,\,Y_{0})\;,\;\mathcal{U}_{t}\,k_{0}(\,\bm{\cdot}\,,\,X_{0})\bigr{\rangle}_{{\cal
H}_{t}}=\bigl{\langle}P_{t}k(\,\bm{\cdot}\,,\,Y_{0})\;,\;P_{t}\,k(\,\bm{\cdot}\,,\,X_{0})\bigr{\rangle}_{{}_{\cal
H}}\;=\;\bigl{\langle}k\big{(}\,\bm{\cdot}\,,\,\Phi_{t}(Y_{0})\big{)}\;,\;k\big{(}\,\bm{\cdot}\,,\,\Phi_{t}(X_{0})\big{)}\bigr{\rangle}_{{}_{\cal
H}}=k(\Phi_{t}(X_{0})\,,\,\Phi_{t}(Y_{0})).$

With proposition 2 we obtain

$\bigl{\langle}\mathcal{U}_{t}\,k_{0}(\,\bm{\cdot}\,,\,Y_{0})\;,\;\mathcal{U}_{t}\,k_{0}(\,\bm{\cdot}\,,\,X_{0})\bigr{\rangle}_{{\cal
H}_{t}}=k(X_{0}\,,\,Y_{0})=k_{0}(X_{0},Y_{0}){=}\langle
k_{0}(\,\bm{\cdot}\,,\,Y_{0})\,,\,k_{0}(\,\bm{\cdot}\,,\,X_{0})\rangle_{{\cal
H}_{0}}.$

Besides, the image of ${\rm
Span}\\{k_{0}(\,\bm{\cdot}\,,\,Y_{0})\,:\,Y_{0}\in\Omega_{0}\\}$ under
$\mathcal{U}_{t}$ is the set ${\rm
Span}\\{k_{t}(\,\bm{\cdot}\,,\,Y_{t})\,:\,Y_{t}\in\Omega_{t}\\}$ which is
dense in $({\cal H}_{t},\|\bm{\cdot}\|_{{}_{{\cal H}_{t}}})$. The range of
$\mathcal{U}_{t}:{\cal H}_{0}\mapsto{\cal H}_{t}$ is thus dense in ${\cal
H}_{t}$ and the operator $\mathcal{U}_{t}$ is, in that sense, unitary. $\Box$■

Let us now determine the adjoint of the Koopman operator in Wonderland. Let
$\mathcal{P}_{t}:=\;R_{0}\circ U_{t}\circ E_{t}$ for all $t\geq 0$ and let
$X_{t}=\Phi_{t}(X_{0})\in\Omega_{t}$. With the same arguments as for
$\mathcal{U}_{t}$, we have

$\mathcal{P}_{t}\,\left[\,k_{t}(\,\bm{\cdot}\,,X_{t})\,\right]\,=\,k_{0}(\bm{\cdot}\,,\,X_{0})\;.$
(31)

The mapping $\mathcal{P}_{t}:{\cal H}_{t}\mapsto{\cal H}_{0}$ with $t\geq 0$
constitutes the Perron-Frobenius family of operators in Wonderland. The
mapping $\mathcal{P}_{t}:{\cal H}_{t}\mapsto{\cal H}_{0}$ is unitary for the
RKHS family topology (isometry from ${\cal H}_{t}$ to ${\cal H}_{0}$ and the
range of $\mathcal{P}_{t}$ is dense in ${\cal H}_{0}$). The next proposition
justifies that $\mathcal{U}_{t}$ and $\mathcal{P}_{t}$ have inverse action on
the feature maps.

######  Proposition 6 (Koopman Perron-Frobenius duality)

For all $X_{0}\in\Omega_{0}$ and $Y_{t}=\Phi_{t}(Y_{0})\in\Omega_{t}$, we have

$\left\langle\,\mathcal{U}_{t}\,k_{0}(\,\bm{\cdot}\,,\,X_{0})\;,\;k_{t}(\,\bm{\cdot}\,,Y_{t})\,\right\rangle_{{\cal
H}_{t}}\;=\;\left\langle\,k_{0}(\,\bm{\cdot}\,,\,X_{0})\;,\;\mathcal{P}_{t}\left[k_{t}(\,\bm{\cdot}\,,Y_{t})\right]\,\right\rangle_{{\cal
H}_{0}}\,.$

□

###### Proof

By the equations (30) and (31), we have

$\left\langle\,\mathcal{U}_{t}\,k_{0}(\,\bm{\cdot}\,,\,X_{0})\;,\;k_{t}(\,\bm{\cdot}\,,Y_{t})\,\right\rangle_{{\cal
H}_{t}}={k_{t}\big{(}Y_{t},\Phi_{t}(X_{0})\big{)}}\;\text{and
}\;\left\langle\,k_{0}(\,\bm{\cdot}\,,\,X_{0})\;,\;\mathcal{P}_{t}\left[k_{t}(\,\bm{\cdot}\,,Y_{t})\right]\,\right\rangle_{{\cal
H}_{0}}=k_{0}(Y_{0},X_{0})\,,$

and both terms are equal by construction because of Definition 1. $\Box$■

In order to derive the Koopman and Perron-Frobenius operators’ spectral
representation in Wonderland, we exhibit now two propagation operators that
will allow us to express the evolution of the Wonderland’s feature maps.

####  4.1.5 Wonderland spectral representation

We specify hereafter a family of operator $A_{U\\!,\,t}$, related to the
Koopman infinitesimal generator $A_{{}_{U}}$, and that give rise to an
evolution equation on $\Omega_{t}$ akin to (29). They will play, in that
sense, the role of infinitesimal generators on the RKHS family.

For all $t\geq 0$, let $A_{U\\!,\,t}$ be defined by $A_{U\\!,\,t}:=R_{t}\circ
A_{{}_{U}}\circ E_{t}$ with $E_{t}:{\cal H}_{t}\mapsto{\cal H}$ and
$R_{t}:L^{2}_{\mathbb{C}}(\Omega,\nu)\mapsto
L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$ (69 B).

######  Proposition 7 (Continuity of $A_{U\\!,\,t}$)

The mapping $A_{U\\!,\,t}:({\cal H}_{t},\|\bm{\cdot}\|_{{}_{{\cal
H}_{t}}})\mapsto(L^{2}_{\mathbb{C}}(\Omega_{t},\nu),\|\bm{\cdot}\|_{{L^{2}_{\mathbb{C}}(\Omega_{t},\nu)}})$
is well-defined and continuous. □

###### Proof

Upon applying (66 of Prop. 12, B), the continuity of $A_{{}_{U}}$ (Theorem 2)
and of $R_{t}$ (Proposition 14, B) we obtain the proof of the proposition.
$\Box$■

In a very similar way as the infinitesimal generator on ${\cal H}$, the
operator $A_{U\\!,\,t}$ can be understood as an evolution equation of the
feature maps defined on $\Omega_{t}$ and associated to ${\cal H}_{t}$. For
$X_{t}\in\Omega_{t}$, by proposition 3 it can be noticed that

$A_{U\\!,\,t}\;k_{t}(\,\bm{\cdot}\,,\,X_{t})=R_{t}\circ
A_{{}_{U}}\left[k(\,\bm{\cdot}\,,\,X_{t})\right]=R_{t}\left[\,{\partial_{{\cal
M}(\bm{\cdot})}}\,k(\,\bm{\cdot}\,,\,X_{t})\,\right]$

and in particular on $\Omega_{t}$

$A_{U\\!,\,t}\;k_{t}(\,\bm{\cdot}\,,\,X_{t})={\partial_{{\cal
M}(\bm{\cdot})}}\,k_{t}(\,\bm{\cdot}\,,\,X_{t}).$ (32)

###### Remark 3

Operator $A_{U\\!,\,t}$ is alike the Koopman infinitesimal generator
$A_{{}_{U}}$. As a matter of fact by proposition 3 and theorem 2 we have for
all $X\in\Omega$

$A_{{}_{U}}k(\,\bm{\cdot}\,,\,X)={\partial_{{\cal
M}(\bm{\cdot})}}\,k(\,\bm{\cdot}\,,\,X),$

and for all $X_{t}\in\Omega_{t}$

$A_{U\\!,\,t}\;k_{t}(\,\bm{\cdot}\,,\,X_{t})={\partial_{{\cal
M}(\bm{\cdot})}}\,k_{t}(\,\bm{\cdot}\,,\,X_{t}).$

The domains of both operators are nevertheless different. □

###### Remark 4

Operators $A_{U\\!,\,t}$ can be understood as an evolution operator for the
feature maps. As a matter of fact through (29) and the definition of
$A_{U\\!,\,t}$, we have, for all $X\in\Omega_{t_{0}}$

$\partial_{t}\,k\bigl{(}\,\bm{\cdot}\,,\,\Phi_{t}(X)\bigr{)}_{{|_{\Omega_{t}}}}=A_{U\\!,\,t}\,k_{t}\bigl{(}\,\bm{\cdot}\,,\,\Phi_{t}(X)\bigr{)}\,.$
(33)

□

Through the above remark the operator $A_{U\\!,\,t}$ inherits the properties
of operator $A_{{}_{U}}$ defined on the global encompassing RKHS ${\cal H}$.
As shown in the next proposition it remains in particular skew-symmetric.

######  Proposition 8 (Skew-symmetry of $A_{U\\!,\,t}$)

The operator $A_{U\\!,\,t}$ is skew-symmetric in
$L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$: for all $f$ and $g\in{\cal H}_{t}$

$\langle
A_{U\\!,\,t}\,f\;,\;g\rangle_{{L^{2}_{\mathbb{C}}(\Omega_{t},\nu)}}\;=\;-\,\langle
f\;,\;A_{U\\!,\,t}\,g\rangle_{{L^{2}_{\mathbb{C}}(\Omega_{t},\nu)}}.$

□

For the need of this proof only, we introduce the following notation. For a
function $h$ defined on $\Omega_{t}$, we note $\widetilde{h}$ the extension on
$\Omega$ defined by

$\widetilde{h}:=\left\\{\begin{array}[]{ll}h&\mbox{on }\Omega_{t}\\\
0&\mbox{else.}\end{array}\right.$

###### Proof

As $g=E_{t}(g)$ holds on $\Omega_{t}$, we have

$\langle
A_{U\\!,\,t}\,f\;,\,g\rangle_{{L^{2}_{\mathbb{C}}(\Omega_{t},\nu)}}\;=\;\langle
A_{{}_{U}}\circ
E_{t}(f)\restriction_{{}_{\Omega_{t}}}\,,\,E_{t}(g)\,\rangle_{{L^{2}_{\mathbb{C}}(\Omega_{t},\nu)}}\;=\;\langle\widetilde{A_{{}_{U}}\circ
E_{t}(f)}\,,\,E_{t}(g)\,\rangle_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}{.}$

By proposition 4 and since $E_{t}(f)\restriction{{}_{\Omega_{t}}}=f$ on
$\Omega_{t}$ we infer that

$\langle
A_{U\\!,\,t}\,f\;,\,g\,\rangle_{{L^{2}_{\mathbb{C}}(\Omega_{t},\nu)}}\;=\;-\,\langle
E_{t}(f)\,,\,\widetilde{A_{{}_{U}}\circ
E_{t}(g)}\,\rangle_{{L^{2}_{\mathbb{C}}(\Omega,\nu)}}\;=\;-\langle
f\,,\,A_{{}_{U}}\circ
E_{t}(g)\,\rangle_{{L^{2}_{\mathbb{C}}(\Omega_{t},\nu)}}\;=\;-\,\langle
f\;,\;A_{U\\!,\,t}(g)\,\rangle_{{L^{2}_{\mathbb{C}}(\Omega_{t},\nu)}}.$

$\Box$■

We are now ready to prove the Wonderland spectral representation theorem,
which states that the bounded operator $A_{U\\!,\,t}:{\cal H}_{t}\mapsto
L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$ is diagonalizable for all $t\geq 0$.

####  4.1.6 Proof of Theorem 1 [Wonderland spectral representation]

The full proof, organized in two main steps, is thoroughly detailed in D. For
this proof, we consider the infinitesimal generator $A_{{}_{U}}:{\cal
H}\mapsto L^{2}_{\mathbb{C}}(\Omega,\nu)$, which is connected to each
$A_{U\\!,\,t}$ for $t\geq 0$ through the restriction operator $R_{t}$. In the
first step, the diagonalization of operator $A_{{}_{U}}:{\cal H}\mapsto
L^{2}_{\mathbb{C}}(\Omega,\nu)$ in $L^{2}_{\mathbb{C}}(\Omega,\nu)$ is first
performed. To that end we introduce an intermediate (approximating) operator
denoted $\widetilde{A}_{{}_{U}}$, directly related to $A_{{}_{U}}$ and whose
inverse is shown to be compact and self-adjoint. The second step of the proof
consists in deducing the diagonalization of each $A_{U\\!,\,t}$ from the
diagonalization of $A_{{}_{U}}$ obtained at step 1. These two steps are
thoroughly presented in D.

###  4.2 Tangent linear dynamics

A result of practical interest concerns the establishment of a rigorous
ensemble expression of the tangent linear dynamics operator. Recall that we
note $X_{t}=\Phi_{t}(X_{0})$ for all $t\geq 0$. We define $\delta
X(t,x):=\Phi_{t}[X_{0}(x)+\delta X_{0}(x)]-\Phi_{t}[X_{0}(x)]$ for all $t\geq
0$ and almost $x\in\Omega_{x}$, and where $\delta X_{0}(x)$ is a perturbation
of the initial condition at point $x$. The function $\delta X(t,\bm{\cdot})\in
L^{2}(\Omega_{x},\mathbb{R}^{d})$ is the perturbation of the flow at time $t$
with respect to the initial condition $X_{0}$. We have for almost any
$x\in\Omega_{x}$

$\Phi_{t}(X_{0}+\delta X_{0})-\Phi_{t}(X_{0})=\int_{0}^{t}{\cal
M}[\Phi_{s}(X_{0}+\delta X_{0})]-{\cal
M}[\Phi_{s}(X_{0})]\,\mathrm{d}s+o(\|\delta
X_{0}\|)=\int_{0}^{t}{\mathrm{d}}({\cal M}\circ\Phi_{s}(X_{0}))\delta
X_{0}\;\mathrm{d}s+o(\|\delta X_{0}\|)$

and we obtain also that

$\Phi_{t}(X_{0}+\delta X_{0})-\Phi_{t}(X_{0})=\int_{0}^{t}{\mathrm{d}}{\cal
M}[\Phi_{s}(X_{0})]\,\,[{\mathrm{d}}\Phi_{s}(X_{0})\delta
X_{0}]\,\mathrm{d}s+o(\|\delta X_{0}\|)=\int_{0}^{t}\partial_{X}{\cal
M}[\Phi_{s}(X_{0})]\,\,\delta X(s,\bm{\cdot})\;\mathrm{d}s+o(\|\delta
X_{0}\|).$

The variation of the flow verifies $\delta
X(t,\bm{\cdot})=\int_{0}^{t}\partial_{X}{\cal M}[X_{s}]\delta
X(s,\bm{\cdot})\;\mathrm{d}s$ almost everywhere on $\Omega_{x}$. Recall that
${\cal M}$ is assumed $C^{1}$, in particular
$\sup_{X\in\Omega}\partial_{X}{\cal M}(X)<\infty$ (since $\Omega$ is compact).
The function $\delta X(t,\bm{\cdot})$ belongs to
$L^{2}(\Omega_{x},\mathbb{R}^{d})$ and verifies for almost $x\in\Omega_{x}$
and all $t\geq 0$

$\partial_{t}\,\delta X(t,x)\;=\;\partial_{X}{\cal M}[X_{t}]\delta X(t,x).$
(34)

Each component $[\delta X(t,\bm{\cdot})]_{i}$ of $\delta X(t,\bm{\cdot})$
belongs to $L^{2}(\Omega_{x},\mathbb{R})$. Let $[g^{\epsilon}_{x}(\delta
X_{t})]_{i}:\Omega\mapsto\mathbb{C}$ be the representation of $[\delta
X(t,\bm{\cdot})]_{i}$ in $L^{2}_{\mathbb{C}}(\Omega,\nu)$ defined for all
$x\in\Omega_{x}$ by

$[g^{\epsilon}_{x}(\delta X_{t})]_{i}:=[\delta X_{t}(x)]_{i}.$

For any $x\in\Omega_{x}$, the family of functions $g^{\epsilon}_{x}(X_{t})$
correspond to vector observables of the dynamical system and we may
immediately write for all $t\geq 0$

$\partial_{t}\,g^{\epsilon}_{x}(\delta X_{t})\;=\;\partial_{X}{\cal
M}(X_{t})g^{\epsilon}_{x}(\delta X_{t}).$ (35)

We are now ready to exhibit a kernel expression of the tangent linear operator
$\partial_{X}{\cal M}(X_{t})$. For all $1\leq i\leq d$, the function
$\mathcal{L}_{k}\,[g^{\epsilon}_{x}]_{i}\in{\cal H}$ verifies for all $\delta
X_{t}\in\Omega$

$\langle{\cal L}_{k}\circ\partial_{t}\,{\cal
L}_{k}\,[g^{\epsilon}_{x}]_{i}\;,\,k(\,\bm{\cdot}\,,\,\delta
X_{t})\rangle_{{}_{\cal H}}={\cal L}_{k}\;(\partial_{t}\,{\cal
L}_{k}\,[g^{\epsilon}_{x}]_{i})(\delta X_{t}).$ (36)

Upon applying the differenciation formulae of proposition 5 and the evolution
equation (29), we have

$\displaystyle\langle{\cal L}_{k}\circ\partial_{t}\,{\cal
L}_{k}\,[g^{\epsilon}_{x}]_{i}\,,\,k(\,\bm{\cdot}\,,\,\delta
X_{t})\,\rangle_{{}_{\cal H}}$ $\displaystyle=-\langle{\cal
L}_{k}[g^{\epsilon}_{x}]_{i}\,,\,{\cal
L}_{k}\circ\partial_{t}\,k(\,\bm{\cdot}\,,\,\delta X_{t})\rangle_{{}_{\cal
H}}$ $\displaystyle=-\langle{\cal L}_{k}[g^{\epsilon}_{x}]_{i}\,,\,{\cal
L}_{k}\,[A_{{}_{U}}\,k(\,\bm{\cdot}\,,\,\delta X_{t}]\,\rangle_{{}_{\cal H}}$
$\displaystyle=-\langle\,{\cal
L}^{\scriptscriptstyle{1/2}}_{k}\,[g^{\epsilon}_{x}]_{i}\;,\;{\cal
L}^{\scriptscriptstyle{1/2}}_{k}\,[A_{{}_{U}}\,k(\,\bm{\cdot}\,,\,\delta
X_{t}]\,\rangle_{L^{2}_{\mathbb{C}}(\Omega,\nu)}.$

The operator $\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}$ being self-adjoint in
$L^{2}_{\mathbb{C}}(\Omega,\nu)$ and $A_{{}_{U}}$ skew-symmetric for the inner
product of $L^{2}_{\mathbb{C}}(\Omega,\nu)$, we have

$\displaystyle\langle{\cal L}_{k}\circ\partial_{t}\,{\cal
L}_{k}\,[g^{\epsilon}_{x}]_{i}\;,\;k(\,\bm{\cdot}\,,\,\delta
X_{t})\,\rangle_{{}_{\cal H}}$ $\displaystyle=\langle A_{{}_{U}}\,{\cal
L}_{k}[g^{\epsilon}_{x}]_{i}\;,\;k(\,\bm{\cdot}\,,\,\delta
X_{t})\,\rangle_{L^{2}_{\mathbb{C}}(\Omega,\nu)}$ $\displaystyle=\langle{\cal
L}^{\scriptscriptstyle{1/2}}_{k}\,A_{{}_{U}}\,{\cal
L}_{k}[g^{\epsilon}_{x}]_{i}\;,\;{\cal
L}^{-\scriptscriptstyle{1/2}}_{k}\,k(\,\bm{\cdot}\,,\,\delta
X_{t})\,\rangle_{L^{2}_{\mathbb{C}}(\Omega,\nu)}$ $\displaystyle=\langle{\cal
L}_{k}\,A_{{}_{U}}\,{\cal
L}_{k}[g^{\epsilon}_{x}]_{i}\;,\;k(\,\bm{\cdot}\,,\,\delta
X_{t})\,\rangle_{{}_{\cal H}}$ $\displaystyle={\cal L}_{k}\,A_{{}_{U}}\,{\cal
L}_{k}\,[g^{\epsilon}_{x}]_{i}(\delta X_{t}).$

Combining the right-hand side of the above expression with (36), we have

${\cal L}_{k}(\,\partial_{t}\,{\cal L}_{k}[g^{\epsilon}_{x}]_{i}\,)(\delta
X_{t})={\cal L}_{k}\,(A_{{}_{U}}\,{\cal L}_{k}[g^{\epsilon}_{x}]_{i}\,)(\delta
X_{t}).$ (37)

As ${\cal L}_{k}$ is injective, the kernel of the tangent linear operator in
the RKHS ${\cal H}$ reads:

$\left(\partial_{t}\,{\cal L}_{k}[g^{\epsilon}_{x}]_{i}\right)(\delta
X_{t})=A_{{}_{U}}\,{\cal L}_{k}\,[g^{\epsilon}_{x}]_{i}(\delta X_{t}).$

As $\Omega$ is bounded, the function $t\mapsto\delta X(t,\bm{\cdot})$ belongs
to $L^{\infty}([0,T])$ and therefore $\delta X_{t}\mapsto
g^{\epsilon}_{x}(\delta X_{t})$ belongs to
$L^{\infty}\bigl{(}[0,T],L^{2}(\Omega,\nu)\bigr{)}$. By (35), we have also
that $\delta X_{t}\mapsto\partial_{t}\,g^{\epsilon}_{x}(\delta X_{t})$ belongs
to $L^{\infty}\bigl{(}[0,T],L^{2}(\Omega,\nu)\bigr{)}$ since
$\sup_{Y\in\Omega}\partial_{X}{\cal M}(Y)<\infty$. We have therefore on
$\Omega$

$\partial_{t}\,\mathcal{L}_{k}[g^{\epsilon}_{x}]_{i}(\delta
X_{t})\;=\;\int_{\Omega}k(\delta
X_{t}\,,\,z)\;\partial_{t}[g^{\epsilon}_{x}]_{i}(z)\,\nu(\mathrm{d}z)\;=\mathcal{L}_{k}\,(\partial_{t}\,[g^{\epsilon}_{x}]_{i}\,)(\delta
X_{t}),$

and thus,

$\partial_{t}\,[g^{\epsilon}_{x}]_{i}(\delta X_{t})\;=\;{\cal
L}^{-1}_{k}\,A_{{}_{U}}\,{\cal L}_{k}\,[g^{\epsilon}_{x}]_{i}(\delta X_{t}).$

By the commutation property of remark 2, together with (35) and as ${\cal
L}^{\scriptscriptstyle{1/2}}_{k}\,[g^{\epsilon}_{x}]_{i}\in{\cal H}$, the
following equalities hold for all $\delta X_{t}\in\Omega$

$\partial_{t}\,[g^{\epsilon}_{x}]_{i}(\delta X_{t})\;=\;{\cal
L}^{-\scriptscriptstyle{1/2}}_{k}\,A_{{}_{U}}\,{\cal
L}^{\scriptscriptstyle{1/2}}_{k}\,[g^{\epsilon}_{x}]_{i}(\delta X_{t}),$
$\partial_{X}{\cal M}(X_{t})\,g^{\epsilon}_{x}\,(\delta X_{t})={\cal
L}^{-\scriptscriptstyle{1/2}}_{k}\,A_{{}_{U}}\,{\cal
L}^{\scriptscriptstyle{1/2}}_{k}\,g^{\epsilon}_{x}(\delta X_{t}).$ (38)

Let’s specify now the kernel expression of the tangent linear dynamics in
$\Omega_{s}$ for all $s\geq 0$. The function
$\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\,[g^{\epsilon}_{x}]\restriction_{\Omega_{s}}$
belongs to ${\cal H}_{s}$. We have as well
$\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}(\partial_{t}g^{\epsilon}_{x})=\mathcal{L}_{k_{s}}^{\scriptscriptstyle{1/2}}(\partial_{t}g^{\epsilon}_{x})$
on $\Omega_{s}$. For all $Y_{s}$ and $X^{\prime}_{s}\in\Omega_{s}$ we have
that
$A_{{}_{U}}\,k(\,\bm{\cdot}\,,\,X^{\prime}_{s})(Y_{s})=A_{U,s}\,k_{s}(\,\bm{\cdot}\,,\,X^{\prime}_{s})(Y_{s})$.
In particular, we get
$A_{{}_{U}}\,\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\,[g^{\epsilon}_{x}](Y_{s})=A_{U,s}\,\mathcal{L}_{k_{s}}^{\scriptscriptstyle{1/2}}\,[g^{\epsilon}_{x}](Y_{s})$.
We obtain hence the kernel expression for all $Y_{s}\in\Omega_{s}$

$\partial_{X}{\cal M}(X_{s})\,g^{\epsilon}_{x}\,(Y_{s})={\cal
L}^{-\scriptscriptstyle{1/2}}_{k_{s}}A_{U\\!,\,s}\;{\cal
L}^{\scriptscriptstyle{1/2}}_{k_{s}}\,g^{\epsilon}_{x}\,(Y_{s}).$ (39)

Note that the domains of the kernel expressions of the tangent linear (38) and
(39) are different. The right-hand side of (39) provides a convenient kernel
expression of the tangent linear operator, enabling us to evaluate the tangent
linear dynamics from an ensemble of feature maps. The adjoint of the tangent
linear dynamics is straightforwardly given by:

$\partial_{X}{\cal M}^{*}(X_{s})g^{\epsilon}_{x}\,(Y_{s})=-{\cal
L}^{-\scriptscriptstyle{1/2}}_{k_{s}}A_{U\\!,\,s}\;{\cal
L}^{\scriptscriptstyle{1/2}}_{k_{s}}g^{\epsilon}_{x}\,(Y_{s}).$ (40)

######  Remark 5 (Projection observables)

The point observable functions $g_{x}^{\epsilon}$ used above can be extended
to other functions defined from basis $(\Psi_{j})_{j\geq 0}$ of
$L^{2}(\Omega_{x},\mathbb{R}^{d})$, with

$\displaystyle g^{\psi}_{x}X_{t}:=\sum_{j}^{\infty}\langle
X_{t}\,,\Psi_{j}\rangle_{L^{2}(\Omega_{x})}\Psi_{j}(x),$ $\displaystyle
g^{\psi}_{x}:\Omega\mapsto\sum_{j}^{\infty}\langle\bm{\cdot}\,,\Psi_{j}\rangle_{L^{2}(\Omega_{x})}\Psi_{j}(x)\in\mathbb{C}.$

□

######  Remark 6 (Dependence on $X_{t}$)

It should be noted that in (39) and (40) the right-hand side does not depend
on $X_{s}$ whereas the tangent linear operator or its adjoint on the left-hand
side does. Function $X_{s}$ indicates around which function of $\Omega_{s}$
the nonlinear system is linearized. The infinitesimal generator of the Koopman
operator, representing the dynamics’ linear tangent operator on the RKHS
family, depends necessarily also on this function. This dependence is here
implicit and induced by the considered sampled functions used to define the
RKHS ${\cal H}_{t}$. If the set of members $(X_{t}^{(i)})_{i\geq 1}$ are
centered around a particular function $X_{s}$, the infinitesimal generators of
the Koopman operator can be interpreted as a representation of the tangent
linear operator around function $X_{s}$. In ensemble methods, $X_{s}$, is in
general taken as the ensemble mean, and $(X^{(i)}_{s})_{i\geq 1}$ is an
ensemble of of time-dependent perturbations around this mean. □

######  Remark 7 (More regularity on $\delta X_{t}$)

For all $1\leq i\leq d$, if we suppose that the function
$[g^{\epsilon}_{x}]_{i}$ belongs to ${\cal H}\subset
L^{2}_{\mathbb{C}}(\Omega,\nu)$, the proof can be simplified and (39) is
replaced by

$\partial_{X}{\cal M}(X_{s})\,g^{\epsilon}_{x}\,(\delta
X_{s})=A_{U\\!,\,s}\,g^{\epsilon}_{x}\,(\delta X_{s}).$ (41)

□

It can be pointed out that the expression above corresponds to the
approximation of the tangent linear dynamics used in ensemble method if we
work in a finite dimensional space such that $\Omega\subset\mathbb{R}^{n}$ and
assume that $k_{t}(\bm{\cdot},X_{t})$ is defined as
$(p-1)^{-\scriptscriptstyle{1/2}}\langle(X_{t}-\overline{X}_{t}),\bm{\cdot}\,-\overline{X}_{t}\rangle_{\mathbb{R}^{n}}$,
with $\overline{X}_{t}$ the empirical ensemble mean. With that definition, we
have
$A_{U\\!,\,t}k_{t}(\bm{\cdot},X^{(i)})=\sum_{j}{\partial_{X_{j}}}k_{t}(\bm{\cdot},X_{t}^{(i)}){\cal
M}(\bm{\cdot})_{j}=\langle(\delta_{X^{(i)}_{t}}-\frac{1}{p}\sum_{\ell}\delta_{X^{(\ell)}_{t}}){\cal
M}(\bm{\cdot}),\bm{\cdot}\,-\overline{X}_{t}\rangle_{\mathbb{R}^{n}}$ that
reads $(p-1)^{-\scriptscriptstyle{1/2}}\langle{\cal
M}(X_{t}^{(i)})-\frac{1}{p}\sum_{j}{\cal
M}(X_{t}^{(j)}),\bm{\cdot}\,-\overline{X}_{t}\rangle_{\mathbb{R}^{n}}$, for
$i=1,\ldots,p$ and for which, when associated to the $\mathbb{R}^{n}$
Euclidean inner product on a resolution grid of size $n$, the left-hand side
of this latter expression corresponds to the so-called $(p\times n)$ anomaly
matrix built from $p$ ensemble members of the dynamical system. The tangent
linear approximation provided by ensemble methods can be thus immediately
interpreted as a particular instance of feature maps together with a given
choice of specific inner product to define the reproducing kernel. Keeping a
finite dimensional approximation but working without assuming that the
functions $[g^{\epsilon}_{x}]$ belong to ${\cal H}$, and thus with now
expression (38) for the ensemble tangent linear expression, corresponds to the
case in which a localization procedure identified to the square root operator
${\cal L}^{\scriptscriptstyle{1/2}}_{k_{t}}$ has been considered. These two
choices embed the problem within a particular RKHS family of functions. The
relation between the tangent linear dynamics and the anomaly matrix is in our
case exact and does not correspond to a finite difference approximation as
classically presented in ensemble methods. Wonderland family can be seen as a
way of linearizing locally a nonlinear system in a convenient sequence of
spaces of smooth functions.

###  4.3 Finite time Lyapunov exponents

The kernel of the Koopman operator provides also a direct access to the finite
time Lyapunov exponents. Recalling from (35) that for any punctual observable
$g_{x}$, as defined previously, we have

$g_{x}(\delta X_{t})\;=\int_{0}^{t}\partial_{X}{\cal M}(X_{s})g_{x}(\delta
X_{s}){\mathrm{d}}s+o(\|\delta X_{0}\|).$ (42)

With the expression of the tangent linear operator in terms of the Koopman
infinitesimal generator (38) on $\Omega_{s}$, we have then

$g_{x}(\delta X_{t})=\int_{0}^{t}\\!\\!\\!\;{\cal
L}^{-\scriptscriptstyle{1/2}}_{k_{s}}A_{U,s}\;{\cal
L}^{\scriptscriptstyle{1/2}}_{k_{s}}g_{x}(\delta X_{s})\;\mathrm{d}s.$ (43)

For all $s\in[0,t]$, we consider a perturbation $g_{x}(\delta X_{s})={\cal
L}^{-\scriptscriptstyle{1/2}}_{k_{s}}\psi_{\ell}^{s}(\delta X_{s})$ along a
Koopman generator eigenfunction associated to the eigenvalue of maximal
modulus $|\lambda_{\ell}|$. By Theorem 1, we have

$\partial_{t}g_{x}(\delta X_{s})=\lambda_{\ell}\,{\cal
L}^{-\scriptscriptstyle{1/2}}_{k_{s}}\psi^{s}_{\ell}\\\
=\lambda_{\ell}\,g_{x}(\delta X_{s}).$

Therefore we get

$|g_{x}(\delta X_{t})|=e^{|\lambda_{\ell}|t}\;|g_{x}(\delta X_{0})|,$ (44)

and the finite time Lyapunov exponent is consequently defined as

$\sigma=|\lambda_{\ell}|.$ (45)

For regular perturbations $g_{x}\in{\cal H}_{s}$ with unitary perturbation
$g_{x}(\delta X_{s})=\psi_{\ell}^{s}(\delta X_{s})$, the derivation is even
simpler as we obtain from remark 7

$\partial_{t}g_{x}(\delta X_{s})\;=A_{U\\!,\,t}\;g_{x}(\delta X_{s}),$

which yields directly to expression (44) and to the same expression for the
Lyapunov exponent.
The modulus of the larger Koopman eigenvalue in Wonderland provides thus an
estimate of the Lyapunov exponent. It can be outlined that the computation of
Lyapunov exponents for large scale systems is computationally very demanding
as it requires the construction of the linear tangent dynamics operator and
the solution of an eigenvalue problem of very big dimension. The construction
of the exact numerical tangent linear operator is in general a tedious task
when expressed in $L^{2}(\Omega_{x})$, as in equation (34). The ensemble based
method provided by our formalism is on the contrary very simple by expressing
the tangent linear operator in $L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$, as in
equation (35). It can be noticed that, by this change of norm in the
definition, the computed values are not the same.

Three distinct values can then be defined for practical computations. First,
the Lyapunov spectrum expressed in $L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$ can be
determined by computing the singular values of ${\cal
L}^{-\scriptscriptstyle{1/2}}_{k_{t}}A_{U,t}\;{\cal
L}^{\scriptscriptstyle{1/2}}_{k_{t}}$. The time integral is dropped since the
evaluations are constant along trajectories. It can be viewed as an advantage
of working in $L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$ instead of
$L^{2}(\Omega_{x})$ as performed classically. The time independence is due to
the fact that the Koopman operator is intrinsic to the dynamical system.
However, from a numerical point of view, as the computation is performed in
practice through an ensemble with a limited number of members, the learned
spectrum is representative only of the local dynamics at the time ($t=t_{0}$)
at which the kernel has been evaluated. As an alternative, modal Lyapunov
exponents can be defined by the square root of the first singular values of
${\cal
L}^{-\scriptscriptstyle{1/2}}_{k_{t}}\psi_{\ell}|\lambda_{\ell}|^{2}\psi_{\ell}^{*}K_{t}^{-1}{\cal
L}^{\scriptscriptstyle{1/2}}_{k_{t}}={\cal
L}^{-\scriptscriptstyle{1/2}}_{k_{t}}\psi_{\ell}|\lambda_{\ell}|^{2}\psi_{\ell}^{*}{\cal
L}^{-\scriptscriptstyle{1/2}}_{k_{t}}$ with
$K_{t}(i,j)=k_{t}(X_{t}^{(i)}\,,\,X_{t}^{(j)})$. We call these singular values
the _Koopman modal Lyapunov exponents_ (KMLE). Finally, as a third option,
equation (45) can simply be considered to evaluate modal exponents. We can
notice that the two modal Lyapunov exponents definitions are very similar; the
former being expressed in $L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$ and the latter
in $\mathcal{H}_{t}$. An example of the estimation of the two types of modal
exponents will be presented for a quasi-geostrophic dynamics in the numerical
section.

###  4.4 Practical considerations

Let us stress again that in practice, we only have access to the mappings
$k_{t}$ and $A_{U\\!,\,t}$ with $t\geq 0$. The mapping $k$ and $A_{{}_{U}}$
are completely inaccessible for high-dimensional systems as they require the
complete knowledge of the phase space or at least of a long enough orbit with
a density assumption in the whole phase space. This last assumption is
associated to strong requirements of the dynamical system and is not
necessarily valid for a given time series of a particular observable. Instead
of working with an infinite (dense) trajectory, Theorem 1 enables us to
estimate the eigenvalues and eigenfunctions of the Koopman operator locally in
the “Wonderland” RKHS family, which can locally conveniently be accessed from
an ensemble of finite time trajectories. As it will be described in the
following, operators $A_{U\\!,\,t}$, can be discretized as an ensemble matrix
– itself related, as we saw it previously, to the tangent linear dynamics
operator. This matrix is then diagonalized to get access to Koopman
eigenvalues and their associated eigenfunctions. In theory, the
diagonalization of $A_{U\\!,\,t}$ needs to be performed only once, at a given
time, to access the Koopman eigenpairs $(\psi_{\ell}^{t})_{\ell}$ and
$(\lambda_{\ell})_{\ell}$. However, the exponential relation between distinct
instants allows us also to consider averaging strategies to eventually
robustify the estimation in practice. This capability will be exploited in the
numerical section for the data assimilation of time series.

#### Diagonalization in practice

For all $t\geq 0$, let $m_{t}$ be the kernel expression of the operator
$A_{U\\!,\,t}$ given by
$m_{t}(X_{t}\,,\,Y_{t}):=A_{U\\!,\,t}^{*}\bigl{[}\,k_{t}(\,\bm{\cdot}\,,\,Y_{t})\,\bigr{]}(X_{t})$
for all $X_{t}$ and $Y_{t}\in\Omega_{t}$. By Proposition 8, we have
$m_{t}(X_{t}\,,\,Y_{t})=-A_{U\\!,\,t}\bigl{[}\,k_{t}(\,\bm{\cdot}\,,\,Y_{t})\,\bigr{]}(X_{t})$.

Let us denote by $\\{X_{t}^{(i)}\,:\,1\leq i\leq n\\}$ an ensemble of members
generated by the dynamical system and by
$\\{k_{t}(\,\bm{\cdot}\,,\,X^{(i)}_{t})\;:\;1\leq i\leq n\\}$ the $n$
associated feature maps. For all time $t\geq 0$, these $n$ feature maps enable
us to build a kernel expression of operator $A_{{U},t}$ as the $n\times n$
matrix
${\mathbb{M}}_{t}=\left(\,{m_{t}}(X^{(i)}_{t}\,,\,X^{(j)}_{t})\,\right)_{1\leq
i,j\leq n}$ with:

${\left(\mathbb{M}_{t}\right)_{ij}}\;:=\;-A_{{U}\\!,\,t}\left[\,k_{t}(\bm{\cdot}\,,\,X^{(j)}_{t})\,\right](X_{t}^{(i)})=\left[-\partial_{{\cal
M}(\bm{\cdot})}\,k_{t}(\,\bm{\cdot}\,,\,X^{(j)}_{t})\right]\,(X^{(i)}_{t}).$

As shown in the following, this matrix enables us to access to the Koopman
generator eigenvalues and to the evaluation of the eigenfunctions at the
ensemble members.

By definition of $A_{U\\!,\,t}$, we have
$A_{U\\!,\,t}[\,k_{t}(\bm{\cdot}\,,\,X^{(j)}_{t})\,](X_{t}^{(i)})=A_{{}_{U}}[\,k(\bm{\cdot}\,,\,X^{(j)}_{t})\,](X_{t}^{(i)})$.
Otherwise, as $-\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\circ
A_{{}_{U}}[\,k(\,\bm{\cdot}\,,\,X^{(j)}_{t})\,]$ belongs to ${\cal H}$, by the
proof (Step 1) (D) of Theorem 1 and denoting $(\psi_{\ell})_{\ell}$ an
orthonormal basis of ${\cal H}$ sets from the eigenfunctions of $A_{{}_{U}}$,
we get

$\displaystyle-\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\circ
A_{{}_{U}}\bigl{[}\,k(\bm{\cdot}\,,\,X^{(j)}_{t})\,\bigr{]}(X_{t}^{(i)})$
$\displaystyle=-\sum_{\ell=0}^{\infty}\langle\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\circ
A_{{}_{U}}\,\bigl{[}\,k(\bm{\cdot}\,,\,X^{(j)}_{t})\,\bigr{]}\,,\,\psi_{\ell}\,\rangle_{{}_{{\cal
H}}}\;\psi_{\ell}(X_{t}^{(i)}),$ $\displaystyle=-\sum_{\ell=0}^{\infty}\langle
A_{{}_{U}}\,k(\,\bm{\cdot}\,,\,X^{(j)}_{t})\;,\;\mathcal{L}_{k}^{-\scriptscriptstyle{1/2}}\,\psi_{\ell}\,\rangle_{L^{2}_{\mathbb{C}}(\Omega,\nu)}\;\;\psi_{\ell}(X_{t}^{(i)}).$

Noting $\beta_{\ell}$ the eigenvalues of $\mathcal{L}_{k}$, we have also (step
1 D theorem 1)
$\mathcal{L}_{k}\psi_{\ell}(X_{t}^{(j)})=\beta_{\ell}\,\psi_{\ell}(X_{t}^{(j)})$
and

$-\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\circ
A_{{}_{U}}\bigl{[}\,k(\bm{\cdot}\,,\,X^{(j)}_{t})\,\bigr{]}(X_{t}^{(i)})=-\sum_{\ell=0}^{\infty}\,\beta_{\ell}^{-\scriptscriptstyle{1/2}}\,\langle
A_{{}_{U}}\,k(\,\bm{\cdot}\,,\,X^{(j)}_{t})\;,\;\psi_{\ell}\,\rangle_{L^{2}_{\mathbb{C}}(\Omega,\nu)}\;\;\psi_{\ell}(X_{t}^{(i)}).$

By the skew symmetry of the generators (proposition 4) we have that

$-\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\circ
A_{{}_{U}}\bigl{[}\,k(\bm{\cdot}\,,\,X^{(j)}_{t})\,\bigr{]}(X_{t}^{(i)})=\\\
\sum_{\ell=0}^{\infty}\,\lambda_{\ell}\;\beta_{\ell}^{-\scriptscriptstyle{1/2}}\;\langle
k(\bm{\cdot}\,,\,X^{(j)}_{t})\,,\,\psi_{\ell}\,\rangle_{L^{2}_{\mathbb{C}}(\Omega,\nu)}\;\;\psi_{\ell}(X_{t}^{(i)})=\sum_{\ell=0}^{\infty}\,\lambda_{\ell}\;\beta_{\ell}^{-\scriptscriptstyle{1/2}}\;\overline{\mathcal{L}_{k}\psi_{\ell}(X_{t}^{(j)})}\;\psi_{\ell}(X_{t}^{(i)}),$

which leads to

$-\mathcal{L}_{k}^{\scriptscriptstyle{1/2}}\circ
A_{{}_{U}}\bigl{[}\,k(\bm{\cdot}\,,\,X^{(j)}_{t})\,\bigr{]}(X_{t}^{(i)})=\sum_{\ell=0}^{\infty}\,\lambda_{\ell}\;\beta_{\ell}^{\scriptscriptstyle{1/2}}\overline{\psi_{\ell}(X_{t}^{(j)})}\;\psi_{\ell}(X_{t}^{(i)}),$

and by continuity of $\mathcal{L}_{k}^{-\scriptscriptstyle{1/2}}$

$-A_{{}_{U}}\bigl{[}\,k(\bm{\cdot}\,,\,X^{(j)}_{t})\,\bigr{]}(X_{t}^{(i)})=\sum_{\ell=0}^{\infty}\,\lambda_{\ell}\;\beta_{\ell}^{\scriptscriptstyle{1/2}}\;\overline{\psi_{\ell}(X_{t}^{(j)})}\;\;\mathcal{L}_{k}^{-\scriptscriptstyle{1/2}}\psi_{\ell}(X_{t}^{(i)})=\sum_{\ell=0}^{\infty}\,\lambda_{\ell}\;\overline{\psi_{\ell}(X_{t}^{(j)})}\;\psi_{\ell}(X_{t}^{(i)}).$

By the restriction expression (remark 17, B), we finally obtain the following
equality for all $t\geq 0$

${m_{t}}(X^{(i)}_{t}\,,\,X^{(j)}_{t})=\sum_{\ell=0}^{\infty}\,\lambda_{\ell}\;\psi^{t}_{\ell}(X_{t}^{(i)})\;\overline{\psi^{t}_{\ell}(X_{t}^{(j)})},$

which shows that the diagonalization of ${\mathbb{M}}_{t}$ provides a set of
the Koopman generator eigenpairs in Wonderland.

In practice, the skew-symmetric matrix
$\mathbb{M}_{t}:=\bigl{(}{m_{t}}(X^{(i)}_{t}\,,\,X^{(j)}_{t})\,\bigr{)}_{{}_{1\leq
i,j\leq n}}$ is assembled from the definition of $A_{U\\!,\,t}$ and a given
choice of the kernel. As explained in the previous section this matrix
corresponds to a kernel expression of $A_{U\\!,\,t}$ with

${\mathbb{M}}_{t}(X^{(i)}_{t}\,,\,X^{(j)}_{t})\;:=\;-A_{U\\!,\,t}\bigl{[}\,k_{t}(\bm{\cdot}\,,\,X^{(j)}_{t})\,\bigr{]}(X_{t}^{(i)})=[-{\partial_{{\cal
M}(\bm{\cdot})}}\,k_{t}(\,\bm{\cdot}\,,\,X^{(j)}_{t})]\,(X^{(i)}_{t}).$

This matrix can be interpreted in the RKHS setting as resulting from the
matrix multiplication:

${\mathbb{M}}_{t}(X^{(i)}_{t}\,,\,X^{(j)}_{t})\;:=\;\sum_{\ell}{\mathbb{F}}_{t}(j,\ell)K_{t}(\ell,i{)},$

with
${\mathbb{F}}_{t}(i,\ell)=-A_{U\\!,\,t}\bigl{[}\,k_{t}(Z^{(\ell)}\,,\,X^{(j)}_{t})\bigr{]}$.
This indeed corresponds to a discretization of the kernel expression of
operator $A_{U\\!,\,t}$ through the empirical Dirac measure. Numerically,
instead of working with matrix ${\mathbb{M}}_{t}$ (i.e. the evaluation of
$A_{U\\!,\,t}k_{t}(\bm{\cdot},X)$ at several discrete points), we will work
directly with matrix ${\mathbb{F}}_{t}={\mathbb{M}}_{t}K_{t}^{-1}$. This has
the advantage of directly working with an implicit discretization of operator
$A_{U\\!,\,t}$, and to relax somewhat its dependency on the kernel choice. The
skew-symetric matrix ${\mathbb{M}}_{t}K_{{t}}^{-1}$ is then diagonalized
through a direct numerical procedure (using LAPACK library and working
numerically on the anti-symetric part of ${\mathbb{M}}_{t}K_{{t}}^{-1}$) and
can be written $\mathbb{M}_{t}K_{{t}}^{-1}=V_{t}\Lambda V^{T}_{t}$, with
$V_{t}$ a unitary matrix and $\Lambda$ a diagonal matrix. The matrix $V_{t}$
gathers eigenvectors $V_{j}^{t}$ of ${\mathbb{F}}$, which is a discretization
of $A_{U\\!,\,t}k_{t}(\bm{\cdot},X)$ giving access to the values of the
Koopman eigenfunctions $\psi^{t}_{j}={\cal
L}_{k}^{\scriptscriptstyle{1/2}}V_{j}^{t}$ at the $n$ ensemble members points
$X^{(i)}_{t}$. The matrix $\Lambda$ is composed of Koopman eigenvalues with
conjugate pairs of pure imaginary eigenvalues.

As previously mentioned this diagonalization can be performed at a single time
or at several instants accompanied with an averaging procedure. Theorem 1 and
(8), give access to the eigenvectors evaluation along a trajectory for all
time instants.

We provide below as examples, expressions of the evaluation of
$\mathbb{M}_{t}(X_{t}^{(i)},X_{t}^{(j)})=\left(-\partial_{{\cal
M}(\bm{\cdot})}\,k_{t}(\bm{\cdot},X_{t}^{(j)})\right)\left(X_{t}^{(i)}\right)$
for the empirical covariance kernel ($k_{E}$) and the Gaussian kernel
($k_{G}$). The empirical covariance kernel is defined through the kernel
isometry property as

${k_{t}}(X_{t}^{(i)},X_{t}^{(j)})={k_{E}(X_{0}^{(i)},X_{0}^{(j)})}=\left\langle
X_{0}^{(i)},X_{0}^{(j)}\right\rangle_{\Omega_{x}},$

with $(\bm{\cdot},\bm{\cdot})_{\Omega_{x}}$ the inner product of
$L^{2}(\Omega_{x},\mathbb{R}^{d})$ and where $\Omega_{x}$ denotes the physical
domain of the considered dynamics. We obtain

$\mathbb{M}_{t}(X_{t}^{(i)},X_{t}^{(j)})=\left\langle\frac{\partial}{\partial
t}X_{0}^{(j)},X_{0}^{(i)}\right\rangle_{\Omega_{x}}.$ (46)

In this expression, we see that the time derivative of the ensemble members at
the initial time is required. Similarly, the Gaussian kernel is defined as

$k_{t}(X_{t}^{(i)},X_{t}^{(j)})={k_{G}(X_{0}^{(i)},X_{0}^{(j)})}=\exp{\left(-\frac{1}{{\ell_{G}}^{2}}\left\|X_{0}^{(i)}-X_{0}^{(j)}\right\|_{\Omega_{x}}^{2}\right)},$

This leads to

$\mathbb{M}_{t}(X_{t}^{(i)},X_{t}^{(j)})=-\frac{2}{{\ell_{G}}^{2}}\left\langle\frac{\partial}{\partial
t}X_{0}^{(j)},\left(X_{0}^{(i)}-X_{0}^{(j)}\right)\right\rangle_{\Omega_{x}}\exp{\left(-\frac{1}{{\ell_{G}}^{2}}\left\|X_{0}^{(i)}-X_{0}^{(j)}\right\|_{\Omega_{x}}^{2}\right)}.$
(47)

Thanks to the isometry property, this matrix needs to be evaluated only at a
single time.

##  5 Application to ensemble methods

###  5.1 Quasi-geostrophic model

Based on a set of $p$ ensemble members, the ability of the proposed method to
predict a trajectory associated with a new initial condition is demonstrated
on a quasi-geostrophic barotropic model of a double-gyre [58] in a
“rectangular-box ocean”. This idealistic model of North-Atlantic large-scale
circulation considers the transport of the ageostrophic state variable
components by a velocity in geostrophic balance. The barotropic vorticity
equation written for the potential vorticity $q$ and the stream function
$\psi$ in $\Omega_{x}=\left([0:L]\times[-L:L]\right)$, with $L=1$, is

$\left\\{\begin{aligned} &\frac{\partial q}{\partial
t}+J(\psi,q)=f-\left(\delta/L\right)^{5}\nabla^{4}\omega\\\
&\Delta\psi=\omega\quad;\quad q=R_{o}\omega+y_{c},\end{aligned}\right.$ (48)

where $\omega$ is the vorticity and $\boldsymbol{x}_{c}=(x_{c},y_{c})$ are the
latitude and the longitude increasing eastward and northward, respectively,
while $J(f,g)=\partial_{x}f\partial_{y}g-\partial_{x}g\partial_{y}f$ stands
for the Jacobian (with $x$ and $y$ denoting horizontal and vertical
coordinates of $\mathbb{R}^{2}$. The dimensionless Rossby number
$R_{o}=0.0036$ measures the ratio between inertial forces and the Coriolis
force, and a steady wind-forcing is defined as $f=\sin(\pi y_{c})$. The
parameter $\delta/L=0.032$ controls the hyper-viscosity model, with $\delta$
the Munk boundary layer width. Scales smaller than $\delta$ will not be
resolved. Numerical details and links with dimensional variables associated
with a mid-latitude ocean basins, such as North Atlantic, are given in E.

###  5.2 Reconstructions using RKHS

In this study we focus only on two specific kernels: the empirical covariance
kernel ($k_{E}$), and the Gaussian Kernel ($k_{G}$). They are defined
respectively by

$k_{E}(X_{t}^{(i)},\,X_{t}^{(j)})=\left\langle{{q}}^{(i)}(\bm{\cdot},t_{0})\,,\,{{q}}^{(j)}(\bm{\cdot},t_{0})\right\rangle_{\Omega_{x}},$

and

$k_{G}(X_{t}^{(i)},\,X_{t}^{(j)})=\exp{\frac{1}{{\ell_{G}}^{2}}\left\|{q}^{(i)}(\bm{\cdot},t_{0})-{q}^{(j)}(\bm{\cdot},t_{0})\right\|^{2}_{\Omega_{x}}},$

where $\left\langle\bm{\cdot},\bm{\cdot}\right\rangle_{\Omega_{x}}$ stands for
the standard $L^{2}$ inner product in $\Omega_{x}$ and $q^{(i)}(x,t)$ is the
solution of equation (48) of the $i^{\text{th}}$ ensemble member at time $t$
at the space location $x$. The former has the advantage to be simple, and to
be directly related to standard ensemble methods employed in data
assimilation. However, it is strongly rank deficient and introduces spurious
correlations in the physical domain $\Omega_{x}$ between two far apart
locations. Localization techniques [59] have been introduced to tackle this
problem and may be interpreted in our setting as embeddings in specific RKHS.
This interpretation may open new computationally efficient generalisations
related for instance to localization techniques in the model space as
explained after remark 7. The parallel done between operators ${\cal
L}^{\scriptscriptstyle{1/2}}_{k_{t}}$ and localization procedure would in
particular immediately enables to set up time-varying and flow dependant
localization techniques, which are difficult to built. Gaussian kernels are
widely used in machine learning for its high regularity and non-compact
support in $L^{2}(\Omega,\nu)$ [60]. It can be also interpreted as a way to
perform localization. In this study, compared to other tested – but not
presented – kernels (polynomial and exponential for instance), it led to
better performances.

Once the kernel matrix $K_{ij}=k_{t}(X_{t}^{(i)}\,,\,X_{t}^{(j)})$ for
$(i,j)\in[1\dots p]\times[1\dots p]$ (equal for all $t$ – kernels isometry,
Theorem 3) is evaluated, a very simple reconstruction of the whole trajectory
can be performed. Denoting $X^{\prime}_{0}$ a new ensemble member initial
condition, the linear combination vector $\bm{\beta}$ associated to the
reconstructed trajectory
$\hat{X}^{\prime}_{t}=\sum_{i=1}^{p}\beta_{i}X_{t}^{(i)}$ can be estimated by
kernel regression from the evaluation $k(X^{\prime}_{0},X_{0}^{j})$, for
$j=[1\dots p]$, and the reproducing property as

$\bm{\beta}=K^{-1}k(X^{\prime}_{0}\,,\,X_{0}^{[1\dots p]}).$ (49)

Strikingly, Koopman isometry implies that these coefficients are constant
along time, and brings hence a direct reconstruction of the whole trajectory.

###  5.3 Evaluation of the Koopman eigenfunctions

Performing the $\mathcal{H}_{t}$ inner product of equation (7) with the
ensemble members’ feature maps leads to the infinitesimal generator matrix

$\mathbb{M}(X_{t}^{(i)},\,X_{t}^{(j)})=\left(-{\partial_{{\cal
M}(\bm{\cdot})}}\,k_{t}(\bm{\cdot},X_{t}^{(j)})\right)\left(X_{t}^{(i)}\right).$
(50)

As explained previously eigen-elements of $\mathbb{M}K^{-1}$, noted
$(\lambda_{\ell},V_{\ell}(X_{t}^{(i)}))$, give access to the exact evaluation
of Koopman eigenfunctions on the RKHS family at the ensemble members
$\psi_{\ell}^{t}(X_{t}^{(i)})={\cal
L}_{k}^{\scriptscriptstyle{1/2}}V_{\ell}(X_{t}^{(i)})$, and to the associated
eigenvalues $\lambda_{\ell}$. We note that $\partial_{{\cal
M}(\bm{\cdot})}k_{t}(\bm{\cdot}\,,\,X_{t}^{(j)})$ is obtained by
differentiating the kernel expression (see equations 46, 47 for the two
selected kernels). Skew-symmetry of $\mathbb{M}K^{-1}$, inherited from the
skew-symmetry of $A_{U\\!,\,t}$, is only approached numerically. Evaluations
are thus enhanced by considering only the skew-symmetric part of the matrix
$\mathbb{M}K^{-1}$, which is dominant in the numerical evaluation of (50).

###  5.4 Lyapunov times

The Lyapunov times evaluated using the two kernels $k_{E}$ and $k_{G}$ are
compared. Three methods to compute the Lyapunov exponents $\sigma$ are
detailed in section 4.3. The associated Lyapunov times are defined by
$T=\sigma^{-1}\ln C$, with, $C=10^{3}$.

The Lyapunov times associated with exponents expressed in
$L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$ are shown in figure 1(a). The spectrum
associated with all singular values, truncated such that $\sigma>10^{-5}$, is
displayed. Due to this truncation, very slow modes are not well captured. We
can see that the Gaussian kernel $k_{G}$ estimates in average larger Lyapunov
times.

(a) Lyapunov times spectrum expressed in $L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$
for the Gaussian kernel $k_{G}$ and the empirical covariance $k_{E}$.

(b) Modal Lyapunov times associated with individual Koopman modes. Comparison
with expression in $L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$ and in
$\mathcal{H}_{t}$.

Figure 1: Lyapunov times computed using the Gaussian kernel $k_{G}$ and the
empirical covariance kernel $k_{E}$.

In figure 1(b), the Koopman modal Lyapunov times are shown as a function of
the absolute value of the Koopman eigenvalue. The Lyapunov times expressed
with the RKHS norm equation (45) is directly $T_{\ell}=1/|\lambda_{\ell}|\ln
C$, which explains the perfect decaying slope in logarithmic scale. We observe
that the computation of the modal exponents in
$L^{2}_{\mathbb{C}}(\Omega_{t},\nu)$ leads simply to a vertical shift of the
curves due to the scaling factor induced by the change of norm. We can notice
that with the Gaussian kernel $k_{G}$, slower Koopman eigenfunctions are
evaluated compared to $k_{E}$, which leads to longer Lyapunov times. This
explains longer predictions, which will be presented in section 5.5. As it
will be seen, the modal exponents and their associated Lyapunov times will
reveal very useful, as they will allow us to filter out sequentially the
contributions of the Koopman eigenfunctions beyond their predictability time.

###  5.5 Reconstructions

The ability of trajectories reconstruction is illustrated in figure 2, where a
full trajectory is estimated from the embedding of a new initial condition in
the RKHS family. We compare the anomaly (figure 2(a)) of a given new ensemble
member at an advanced time (3.7 days), with reconstructions using the Gaussian
kernel $k_{G}$ (figure 2(c)). Figure 2(b) shows the best possible
reconstruction as a linear combination of ensemble members, with regards to
the $L^{2}(\Omega_{x})$-norm of the projection error.

(a) Test ensemble member

(b) $L^{2}$norm projection

(c) Gaussian kernel

Figure 2: Comparison between the reference potential vorticity of a test
ensemble member at $t=0.07$ (corresponding to $3.7$ days of prediction) (a)
and the predicted reconstruction using the Gaussian kernel (${\ell_{G}}=1$)
(c). The best possible reconstruction using a linear combination of ensemble
members with respect to the $L^{2}$ norm is shown in (b).

Reconstruction errors are quantified by averaging the square of the
$L^{2}(\Omega_{x})$-norm over the whole test ensemble composed of $100$ new
members (i.e. not used in the training phase). The standard deviations of the
error are also displayed on this figure. The time evolution of this error is
presented in figure 3. It is compared to the projection error defined as the
minimal linear combinations of ensemble members w.r.t.
$L^{2}(\Omega_{x})$-norm. The reconstruction error performed by the ensemble
average is also shown. This error is obviously high for small times, but
estimations are difficult to be more precise than the ensemble average
estimation once the predictability time of the system has passed. The
reconstructions performed with $k_{E}$, grow and overtake the ensemble average
reconstruction around $t=0.2$ (i.e. $18.4$ days). On this basis, the use of
$k_{G}$ (${\ell_{G}}=1.0$) leads to better results. Indeed, despite a slight
reconstruction penalty at the initial time due to a stronger smoothing (see
figure 2), the reconstruction errors become quickly significantly lower, and
the error never reaches the ensemble anomaly error. This small loss of
performance in $L^{2}(\Omega_{x})$-norm at the initial time allows to have a
gain in robustness in terms of long-time prediction.

Figure 3: Mean estimation error using an empirical covariance kernel (short
red dashes), a Gaussian kernel (green dash-dots), the reconstructed Gaussian
kernel using all Koopman eigenfunctions (orange squares) and the Lyapunov
filtered kernel (blue dot-dashes). “Ensemble average” (long black dashes) is
the estimation error of the ensemble average. The projection error (purple
solid line) corresponds to the optimal time-varying linear combination of
ensemble members w.r.t. the $L^{2}(\Omega_{x})$-norm. The vertical width of
the stripes regions over the mean error is equal to the standard deviation of
the error. One dimensionless time $t$ corresponds to $92$ days. Errors are
averaged over the $100$ members of the test ensemble.

The reconstructions based on (49) are valid for all times. Koopman isometry
warrants the use of the same linear combination for the whole trajectory.
Nevertheless, in practice the ensemble are of limited (small) size, and the
eigenpairs of the infinitesimal generator close to zero (representing almost
stationary functions) are difficult to estimate accurately. Modal
predictability time associated to Lyapunov exponents (44) can be used to
robustify the forecast of new initial conditions. The kernel can be expanded
in the Koopman eigenfunctions basis as

$k(X_{t}^{(i)},\,X_{t}^{(j)})=\sum_{j=1}^{\infty}\psi_{\ell}^{t}(X_{t}^{(i)})\overline{\psi_{\ell}^{t}}(X_{t}^{(j)})=\sum_{j=1}^{\infty}\psi_{\ell}^{0}(X_{0}^{(i)})\overline{\psi_{\ell}^{0}}(X_{0}^{(j)}).$
(51)

With the numerical procedure proposed here, the $p$ evaluated Koopman
eigenfunctions allow to perform an exact reconstruction of the kernel. An
appealing potential of equation (51) for faithful predictions consists in
filtering the kernel by switching-off components that have passed their
predictability time. We set this time from the finite-time Lyapunov exponents
(44) as: $T_{\ell}=|\lambda_{\ell}|^{-1}\ln C$, with, $C=10^{3}$. The filtered
kernel $\widetilde{k}(X_{t}^{(i)},\,X_{t}^{(j)})$ and its associated $p\times
p$ kernel matrix $\widetilde{K}$ allows us to define reconstructions, with the
modified linear coefficients
$\widetilde{\bm{\beta}}=\widetilde{K}^{+}\widetilde{K}\bm{\beta}$. The
superscript $\bullet^{+}$ stands for the Moore–Penrose pseudo-inverse. It can
be remarked that the operator $\widetilde{K}^{+}\widetilde{K}$ is a projector
performing filtering directly on the vector of coefficients $\bm{\beta}$.

In figure 3, it is assessed that the kernel is exactly reconstructed by the
Koopman eigenfunctions (superimposition of the green dash-dots curve and the
orange squares). When the kernel is Lyapunov filtered, the reconstruction
almost fits at short time the previous predictions (superimposition of the
blue dot-dashes curve, the green dash-dots curve and the orange squares). For
longer time it significantly reduces the error and clearly separates the
spread of the ensemble from the ensemble average estimation. For very long
times (not shown in the figure), filtering acts on all modes, which reduces to
the ensemble average estimation. This strategy takes advantage of the modal
predictability time. Slow eigenfunctions are used for long time predictions,
and fast ones only for short time forecasts. As a final remark, we note that
very fast modes (with large Koopman eigenvalues modulus) does not seem to
contribute significantly to the reconstruction, since at short times the
filtered and full reconstructions match.

###  5.6 Data assimilation

A possible application of the proposed procedure is ensemble-based data
assimilation. In the context of satellite data, ensemble optimal interpolation
(EnOI) is widely used to reconstruct spatial fields [61, 62]. It is for
instance employed in operational gridded altimetry reconstructions [63]. EnOI
consists in expressing at a given time the solution as a linear combination of
ensemble members, which minimises an objective cost function. As shown in [64,
65], incorporating information from other times together with an adequate
dynamical model significantly improves the estimations. This is obviously at
the price of forecasting the whole ensemble by the dynamics.

Koopman isometry provided by Wonderland allows us to incorporate time-series
of observations without any new simulation of the dynamics. The spatial
reconstruction is embarrassingly simply given as the constant-in-time best
linear combination of ensemble members trajectories.

Based on the same ensemble as in the previous sections, we propose two
demonstration tests considering i) a time-series of extremely sparse (two
satellite swaths like) observations ii) an even sparser case with the same
swaths but at a single time only. To that end, we consider a synthetic
observation mimicking two along tracks measurements of an orbiting satellite,
idealized by two vertical lines. In the first case these lines are assumed to
be known every $0.9$ days $\Delta t=0.01$, while in the second case only day
$5.5$ is available $(t=0.06)$. The observations are perturbed by a centered
Gaussian white noise with the value of the selected standard deviation equal
to 10% of the signal root-mean-square. These 2 benchmarks are carried out for
each of the 100 members of the test ensemble. As previously, the members of
the test ensemble have not been used in the learning phase.

In this numerical experiment, we search for the constant in time coefficients
$\bm{\beta}$ (thanks to isometry), which minimises

$\mathcal{J}=\frac{1}{2T}\int_{0}^{T}\|\mathbb{H}(\hat{X}_{t})-\mathcal{Y}\|_{R^{-1}}^{2}\,\mathrm{d}t+\frac{\alpha^{2}}{2}\|\bm{\beta}{-\bar{\bm{\beta}}}\|^{2}\text{
s.t. }\mathbb{H}(\hat{X}_{t})=\sum_{i=1}^{p}\beta_{i}\mathbb{H}(X_{t}^{(i)}),$
(52)

where $\mathbb{H}$ is the observation operator associated with the satellite
measurements $\mathcal{Y}$, $\hat{X}_{t}$ is the estimated state, and
$\bar{\beta}=\frac{1}{p}(1,\dots,1)^{T}$. The observation operator is assumed
to have its range in the RKHS family and hence provides observables belonging
to the RKHS family, which can be interpolated using the Wonderland kernels. It
can be noticed that the expression of the constraint is due to kernel
interpolation, and not to some linear property of $\mathbb{H}$. This treatment
is similar to the one performed for instance in discrete empirical
interpolation (DEIM) [66]. In the case of a single observation, the time
integral is dropped, and the cost function corresponds to a standard EnOI cost
function. The matrix $R$ is the observation covariance matrix, defined by the
time average of the empirical covariance, localized by performing a Schur
product with matrix $\exp{(-r_{ij}^{2}/\ell_{loc}^{2})}$, where $r_{ij}$ is
the distance between measurements $i$ and $j$, and $\ell_{loc}=0.05$. The
regularization parameter has been set to $\alpha=10$. For this problem we
selected the simple empirical kernel, defined from a non localized potential
vorticity matrix as feature maps of the ensemble. Working with the isometry
property justifies the estimation of constant in time coefficients for the
linear combination. This simplistic method could be easily extended by
introducing more elaborated kernels, Koopman eigenfunctions representation to
project any system’s observable and forecast them, as well as the Lyapunov
filtering schme presented in section (5.5).

Figure 4 shows the potential vorticity trajectory (panel a), the single-time
observation (panel b), the trajectory reconstruction associated to this single
observation (panel c) and the one obtained from the swaths time-series (panel
d). Three consecutive times spaced of $0.02$ (1.8 days) are displayed.

Figure 4: Estimation of potential vorticity at times $t=0.04$ (top), $t=0.06$
(middle) and $t=0.08$ (bottom). Panel (a): True anomaly. Panel (b):
Observation associated with the single-time swath observation. Panel (c):
Estimation knowing the two satellite swaths only at day 5.5 $(t=0.06)$. Panel
(d): Estimation knowing the two swaths every 0.9 days $(\Delta t=0.01)$.

We can see that despite extremely sparse measurements, a single observation
already provides a fairly good estimation, together with forward and backward
in time forecasts based only on the estimated linear combination of the
ensemble members (i.e. without any simulation). As expected, a time-series of
swaths improves further the performances.

In figure 5, the average of the errors obtained over all $100$ members of the
test ensemble are plotted.

Figure 5: Estimation errors averaged over the whole $100$ members of the test
ensemble. The best reconstruction error with constant coefficients (red),
estimation by the ensemble average (black), estimation with the time-series of
satellite swaths observations (yellow) and only at day $5.5$ $(t=0.06)$
(blue).

As a reference, the best possible reconstruction with respect to the $L^{2}$
norm with constant in time linear coefficients $\bm{\beta}$ is shown, as well
as the estimation by the ensemble average. The error performed with the time-
series of satellite observations is almost optimal in terms of average and
standard deviation. We note that for the second test-case, near the
observation time, the error is lower than the one obtained for the whole time-
series. Compared to the time series of observations it may be interpreted as
an over-fitting at the observation time since only a single observation is
used to infer the ensemble members’ coefficients. As expected, this error
grows, but reasonably, as the estimation is propagated forward and backward in
time.

##  6 Concluding remarks

We proposed a theoretical setting to estimate the Koopman eigen-pairs of
compact dynamical systems in embedding the system in a RKHS family, called
Wonderland. Beyond the Koopman operator spectral representation, we showed
that Wonderland provides access to finite time Lyapunov exponents, as well as
to the tangent linear dynamics. These features enabled us to propose very
simple strategies for data assimilation of very sparse observations in time
and space. Obviously more efficient data assimilation techniques could be used
instead of the simple regression strategy explored here. As Wonderland boils
down to a linear setting it should be perfectly adapted to ensemble Kalman
filters. Extension of classical ensemble data assimilation filters on
Wonderland are currently in development and will be the subject of future
publications. The technique lies on the sequence of time-evolving RKHS to
represent the ensemble dynamics and on the superposition principle attached to
Wonderland. This study provides a theorem-based setting to estimate the
Koopman eigenvalues and eigenfunctions, as well as exact ensemble-based
representation of the tangent linear dynamics and of the flow Lyapunov
exponents. This work relies on the assumption of an invertible deterministic
dynamical system defined on a compact manifold. In future studies, we will
focus on the generalisation to stochastic fluid flow dynamics with transport
noise models [67, 68, 69] and the introduction of more sophisticated ensemble
data assimilation techniques [70, 71]. Let’s conclude by playing around with
Lewis Carroll’s novel: “Who are you?” said the Caterpillar to Alice. “Well, I
barely don’t know, being so many different sizes in a day is very confusing”
answered Alice. “It isn’t! Duplicate yourself! Shrink and expend repeatedly!
And gather all your differences to reveal the stable harmonic parts of
yourself.” ended the Caterpillar.

## Acknowledgments

The authors acknowledge the support of the ERC project 856408-STUOD and the
French National program LEFE (Les Enveloppes Fluides et l’Environnement)

##  Appendix A Reproducing kernel Hilbert spaces

In the following appendices we provide complementary elements for the detailed
proofs of the results listed in the paper. We first recall some classical
theorems on RKHS and present their adaptations to our functional setting. In
particular we present useful results on differentiability of RKHS functions
and reproducing kernels. In the following $L^{2}(\Omega_{x})$ stands for the
Hilbert space of Lebesgue measurable $\mathbb{R}^{d}$-valued of square-
integrable functions on the set $\Omega_{x}\subset\mathbb{R}^{p}$ $p=2,3$
endowed at a fixed time, $t$, with the inner product

$\langle
f\,,g\rangle_{L^{2}(\Omega_{x})}=\int_{\Omega_{x}}f(y)\,\,g(y)\;\mathrm{d}y.$

Let $E\subset L^{2}(\Omega_{x})$ denote a locally convex compact set equipped
with the norm of $L^{2}(\Omega_{x})$, by the Aronszajn Theorem, [54], the
reproducing kernel Hilbert spaces (RKHS) of $\mathbb{C}$-valued functions
defined on $E$ are uniquely characterized through a reproducing kernel
$k:E\times E\mapsto\mathbb{C}$. This kernel is a Hermitian, positive definite
function. In the following we will assume furthermore it is
$C^{{(}1,1{)}}(E\times E)$, which means here it is twice differentiable with
respect to its two variables. The (Gateaux) derivative with respect to the
first and second variable in the direction $v\in E$ are denoted
$\partial_{v}^{\tiny{(1)}}k$, and $\partial_{v}^{\tiny{(2)}}k$, respectively.
The reproducing kernel defines an evaluation function $k(\bm{\cdot},x)\in{\cal
H}$ with a reproducing property (Def. 1):

$\forall x\in E,\forall f\in{\cal
H},\;\bigl{\langle}f,k(\,\bm{\cdot}\,,\,x)\bigr{\rangle}_{{\cal H}}=f(x).$

The RKHS ${\cal H}$ corresponds to a space of smooth functions with a strong
uniform convergence property. The elements of ${\cal H}$ are in addition
continuous functions and we have for all $f\in{\cal H}$ that

$\forall\,x\in E\qquad\left|f(x)\right|\,\leq\,\sup_{y\in
E}k(y,y)^{1/2}\;\|f\|_{{}_{\cal H}}.$ (53)

###  A.1 Functional description of ${\cal H}$

We recall in the following the Mercer’s theorem which gives a functional
description of the RKHS ${\cal H}$ and of its associated reproducing kernel
$k$.
Let $L^{2}_{\mathbb{C}}(E,\nu)$ be the space of measurable complex valued
functions defined on the compact set E for which the square modulus is
integrable with respect to measure $\nu$, $L^{2}_{\mathbb{C}}(E,\nu)$ is a
Hilbert space equipped with the inner product
$\langle\bm{\cdot}\,,\,\bm{\cdot}\rangle_{{L^{2}_{\mathbb{C}}(E,\nu)}}$ given
for $f$ and $g\in L^{2}_{\mathbb{C}}(E,\nu)$ by

$\langle
f\,,\,g\rangle_{{L^{2}_{\mathbb{C}}(E,\nu)}}:=\int_{E}f(y)\,\overline{\,g(y)\,}\;\nu(\mathrm{d}y),$

so that it is linear in its first argument and antilinear in its second. Let
us first define ${\cal L}_{k}:L_{\mathbb{C}}^{2}(E,\nu)\mapsto
L_{\mathbb{C}}^{2}(E,\nu)$ the integral operator with kernel
${k\in{C^{{(}1,1{)}}}(E\times E)}$ as:

$\bigl{(}{\cal L}_{k}f\bigr{)}(x)=\int_{E}k(x,y)f(y)\,\nu(\mathrm{d}y).$ (54)

We give below a useful remark relating the integral operator $\mathcal{L}_{k}$
and the inner product
$\langle\bm{\cdot}\,,\,\bm{\cdot}\rangle_{L^{2}_{\mathbb{C}}(E,\nu)}$.

######  Remark 8 (Links between $\mathcal{L}_{k}$ and
$\langle\,\bm{\cdot}\,,\,\bm{\cdot}\,\rangle_{L^{2}_{\mathbb{C}}(E,\nu)}$)

As $k$ is Hermitian, it can be noticed that for all $x\in E$, $\bigl{(}{\cal
L}_{k}f\bigr{)}(x)=\int_{E}f(y)\,\overline{k(y,x)}\,\nu(\mathrm{d}y)$, and we
have thus $\bigl{(}{\cal L}_{k}f\bigr{)}(x)=\langle
f\,,\,k(\,\bm{\cdot}\,,\,x)\rangle_{L^{2}_{\mathbb{C}}(E,\nu)}$. □

This kernel integral operator can be shown to be well-defined and compact (by
Arzelà Ascoli theorem). It is immediate to see that as $k$ is positive
definite and Hermitian, ${\cal L}_{k}$ is positive and self-adjoint on
$L_{\mathbb{C}}^{2}(E,\nu)$. By the spectral theorem of compact self-adjoint
operators, there exists an orthonormal basis $(\varphi_{i})_{i}$ of
$L_{\mathbb{C}}^{2}(E,\nu)$ consisting of eigenvectors of ${\cal L}_{k}$ with
corresponding decreasing real eigenvalues $(\mu_{i})_{i}$ :
$\mu_{i}\geq\mu_{i+1}\geq 0$ for all integers $i$. These results are proved in
[53]. We consider in the following that $\mu_{i}\neq 0$ for all $i$ and
$\mathcal{L}_{k}$ is injective. This denseness assumption could be relaxed
(see remark 14) but simplifies significantly the presentation. The eigenpairs
$(\mu_{i},\varphi_{i})_{i\in\mathbb{N}}$ enables to formulate the following
theorems (with proofs available for instance in [53] and [55]). Elaborating on
remark 8 and the kernel eigenfunctions (often referred to as Mercer’s
eigenfunctions due to theorem 4) we get immediately relations between the
eigenfunction evaluation and the $L^{2}(E,\nu)$ inner product.

######  Remark 9 (Links between $(\varphi_{i})_{i\geq 0}$ and
$\langle\bm{\cdot}\,,\,\bm{\cdot}\rangle_{L^{2}_{\mathbb{C}}(E,\nu)}$)

For all $x\in E$, by Remark 8, we note that

$\displaystyle\langle\varphi_{i}\,,\,k(\,\bm{\cdot}\,,\,x)\rangle_{L^{2}_{\mathbb{C}}(E,\nu)}\;=\;\mathcal{L}_{k}(\varphi_{i})(x)\;=\;\mu_{i}\,\varphi_{i}(x)\qquad\text{and}$
$\displaystyle\langle
k(\,\bm{\cdot}\,,\,x)\,,\,\varphi_{i}\rangle_{L^{2}_{\mathbb{C}}(E,\nu)}\;=\;\overline{\langle\varphi_{i}\,,\,k(\,\bm{\cdot}\,,\,x)\rangle}_{L^{2}_{\mathbb{C}}(E,\nu)}\;=\;\mu_{i}\,\overline{\varphi_{i}(x)}.$

□

###### Remark 10

As $k\in{C^{{(}1,1{)}}}(E\times E)$, the range of $\mathcal{L}_{k}$ is
included in $C^{1}(E)$ and the eigenfunctions of $\mathcal{L}_{k}$ are
$C^{1}(E)$. □

######  Theorem 4 (Mercer decomposition)

For all $x$ and $y\in E$, $\displaystyle
k(x,y)=\sum_{i=0}^{\infty}\mu_{i}\,\varphi_{i}(x)\;\overline{\varphi_{i}(y)}$
where the convergence is absolute (for each $x,y\in E\times E$) and uniform on
$E\times E$. □

In particular, the series $\sum\mu_{i}$ is convergent as

$\displaystyle\int_{E}k(y,y)\nu({\mathrm{d}}y)=\int_{E}\sum_{i=0}^{\infty}\mu_{i}\varphi_{i}(y)\;\overline{\varphi_{i}(y)}\nu({\mathrm{d}}y)=\sum_{i=0}^{\infty}\mu_{i}<\infty.$
(55)

######  Theorem 5 (Mercer representation of RKHS)

The RKHS $\left({\cal H},\langle\,\bm{\cdot}\,,\,\bm{\cdot}\,\rangle_{{}_{\cal
H}},\|\bm{\cdot}\|_{{\cal H}}\right)$ is the Hilbert space characterized by

$\displaystyle{\cal H}=\bigl{\\{}f\in
L_{\mathbb{C}}^{2}(E,\nu),\;f=\sum_{i=0}^{\infty}a_{i}\varphi_{i}\;:\;\sum_{i}\dfrac{\,|a_{i}|^{2}}{\mu_{i}}<\infty\bigr{\\}},$
$\bigl{\langle}f,g\bigr{\rangle}_{{\cal
H}}=\sum_{i=0}^{\infty}\frac{1}{\mu_{i}}\,\langle
f\,,\,\varphi_{i}\rangle_{L^{2}_{\mathbb{C}}(E,\nu)}\,\overline{\langle
g\,,\,\varphi_{i}\rangle}_{L^{2}_{\mathbb{C}}(E,\nu)}\quad$
$\text{and}\quad\|f\|^{2}_{{}_{\cal H}}=\sum_{i=0}^{\infty}\frac{|\langle
f\,,\,\varphi_{i}\rangle_{L^{2}_{\mathbb{C}}(E,\nu)}|^{2}}{\mu_{i}}.$

□

The inner product $\langle\,\bm{\cdot}\,,\,\bm{\cdot}\,\rangle_{{}_{\cal H}}$
is sesqui-linear (linear in its first argument and antilinear in its second).
The proof of Mercer’s theorem warrants that the linear space spanned by
$\\{k(\,\bm{\cdot}\,,x):x\in E\\}$ is dense in $L^{2}_{\mathbb{C}}(E,\nu)$. If
$f=\sum_{i=0}^{\infty}a_{i}\varphi_{i}\in{\cal H}$ then the series converges
absolutely and uniformly to $f$.

The following remark shows that the feature map’s conjugate as well as its
real and imaginary parts belong to ${\cal H}$.

######  Remark 11 (Consequence of the Mercer representation)

For all $y\in E$, the functions $\overline{\,k(\,\bm{\cdot}\,,\,y)}$,
$Re[k(\,\bm{\cdot}\,,\,y)]$ and $Im[k(\,\bm{\cdot}\,,\,y)]$ belong to ${\cal
H}$. By Theorems 5 and 4, we have indeed that

$\sum_{i=0}^{\infty}\dfrac{1}{\mu_{i}}\,|\langle\overline{\,k(\,\bm{\cdot}\,,\,y)\,}\,,\,\varphi_{i}\rangle_{L^{2}_{\mathbb{C}}(E,\nu)}|^{2}\;=\;\\\
\sum_{i=0}^{\infty}\dfrac{1}{\mu_{i}}\,|\,\mathcal{L}_{k}\varphi_{i}(y)|^{2}=\sum_{i=0}^{\infty}\mu_{i}\,|\varphi_{i}(y)|^{2}<\infty,$
(56)

and $\overline{\,k(\,\bm{\cdot}\,,\,y)}\in{\cal H}$. By operation,
$Re[k(\,\bm{\cdot}\,,\,y)]$ and $Im[k(\,\bm{\cdot}\,,\,y)]$ belong thus to
${\cal H}$. □

######  Remark 12 (Comparison of the norms ${\cal H}$ and
$L_{\mathbb{C}}^{2}(E,\nu)$)

For all $f\in{\cal H}$, by Theorem 5 we have for all $x\in E$, the inequality

$\displaystyle
f(x)\leq|f(x)|=\left|\sum_{i=0}^{\infty}a_{i}\varphi_{i}(x)\right|\leq\left(\sum_{i=0}^{\infty}\dfrac{\,|a_{i}|^{2}\,}{\mu_{i}}\right)^{\scriptscriptstyle{1/2}}\;\left(\sum_{i=0}^{\infty}\mu_{i}|\varphi_{i}(x)|^{2}\right)^{\scriptscriptstyle{1/2}},$

with theorem 4 we obtain

$\displaystyle\|f\|_{{L_{\mathbb{C}}^{2}(E,\nu)}}\;\leq\;\left(\int_{E}|k(x,x)|^{2}\,\nu(\mathrm{d}x)\right)^{\scriptscriptstyle{1/2}}\,\|f\|_{{}_{\cal
H}}.$ (57)

□

###### Remark 13

For all $f\in L^{2}_{\mathbb{C}}(E,\nu)$ , $\mathcal{L}_{k}(f)$ belongs to
${\cal H}$, in fact we have

$\sum_{i=0}^{\infty}\frac{1}{\mu_{i}}\left|\langle\mathcal{L}_{k}(f)\,,\,\varphi_{i}\rangle_{L^{2}_{\mathbb{C}}(E,\nu)}\right|^{2}=\sum_{i=0}^{\infty}\mu_{i}\left|\langle
f\,,\,\varphi_{i}\rangle_{L^{2}_{\mathbb{C}}(E,\nu)}\right|^{2}\leq\left(\sum_{i=0}^{\infty}\mu_{i}\,\right)\,\|f\|^{2}_{L^{2}_{\mathbb{C}}(E,\nu)}<\infty.$

Moreover, by Remark 12 the mapping $\mathcal{L}_{k}:{\cal H}\to{\cal H}$ is
continuous : for all $f\in{\cal H}$ we have

$\|\mathcal{L}_{k}(f)\|^{2}_{{\cal
H}}\leq\left(\sum_{i=0}^{\infty}\mu_{i}\,\right)\;\|f\|^{2}_{L^{2}_{\mathbb{C}}(E,\nu)}\leq
C\,\|f\|^{2}_{{\cal H}}.$

□

With a $C^{{{(}2,2{)}}}(E\times E)$ regularity condition on the kernel, we
will prove that the RKHS ${\cal H}$ is a subspace of smoother functions than
$C^{0}(E,\mathbb{C})$ (theorem 6). In the following appendix we discuss
differentiability property of the feature maps.

###  A.2 Differentiability of feature maps and RKHS functions ${\cal H}$

In this appendix we extend results given in [72, 73] for
$E\subset\mathbb{R}^{d}$ to the case where $E$ is a locally convex compact
subset of a function space. In the following $E$ designates a compact set
included in $L^{2}(\Omega_{x},\mathbb{R}^{d})$, the space of measurable
functions that are square integrable on $\Omega_{x}$. Denoting by
$\partial^{(2)}_{u}\,k(\,\bm{\cdot}\,,y)$ the kernel functional derivative in
the direction $u$ with respect to the second variable $y$ – while the upper-
script ${\partial_{u}^{(1)}}$ indicates a directional derivative in the first
variable –, we have first the following lemma on the feature maps derivative.

######  Lemma 2 (Feature maps differentiability)

The kernel derivatives belong to ${\cal H}$. More precisely, let
$k\in{C^{(1,1)}}(E\times E)$, then for all $y\in E$ and ${u}\in E$, we have
$\partial^{(2)}_{{u}}\,k(\,\bm{\cdot}\,,y)\in{\cal H}$. □

###### Proof

Let $y$ and ${u}\in E$. Let $(h_{n})_{n\geq 0}$ be a sequence of
$\mathbb{C}\backslash\\{{0}\\}$ which converges to $0$ and such that for all
$n$, $y+h_{n}{u}\in E$. For all integers $n$, let

$d_{n}(\,\bm{\cdot}\,,\,y):=k(\,\bm{\cdot}\,,\,y+h_{n}{u})-k(\,\bm{\cdot}\,,\,y)\text{
and
}\;t_{n}(\,\bm{\cdot}\,,\,y\,):=\dfrac{\,d_{n}(\,\bm{\cdot}\,,\,y)\,}{h_{n}}.$

We first notice that $(t_{n}(\bm{\cdot}\,,\,y))_{n\geq 0}$ converges pointwise
to ${\partial^{(2)}_{{u}}}\,k(\,\bm{\cdot}\,,\,y)$ since $k$ is
${C^{(1,1)}}(E\times E)$. Then, we now show that
$(t_{n}(\,\bm{\cdot}\,,\,y))_{n}$ is a Cauchy sequence of $({\cal
H},\|\bm{\cdot}\|_{{}_{\cal H}})$. For all integers $p$ and $q$, we have

$\|t_{p}(\,\bm{\cdot}\,,\,y)-t_{q}(\,\bm{\cdot}\,,\,y)\|^{2}_{{}_{\cal
H}}=\|t_{p}(\,\bm{\cdot}\,,\,y)\|^{2}_{{}_{\cal
H}}+\,\|t_{q}(\,\bm{\cdot}\,,\,y)\|^{2}_{{}_{\cal
H}}-2Re[\langle\,t_{p}(\,\bm{\cdot}\,,\,y)\;,\;t_{q}(\,\bm{\cdot}\,,\,y)\,\rangle_{{}_{\cal
H}}].$ (58)

By the reproducing property of ${\cal H}$ and as $k$ is ${C^{(1,1)}}(E\times
E)$, we have for all $x\in E$ and for some $p$ and $q$ large enough,

$\displaystyle\langle\,t_{p}(\,\bm{\cdot}\,,\,y)\;,\;t_{q}(\,\bm{\cdot}\,,\,x)\,\rangle_{{}_{\cal
H}}$
$\displaystyle=\dfrac{1}{h_{p}\,h_{q}}\langle\,k(\,\bm{\cdot}\,,\,y+h_{p}u)-k(\,\bm{\cdot}\,,\,y)\,\;,\;k_{q}(\,\bm{\cdot}\,,\,x+h_{q}u)-k(\,\bm{\cdot}\,,x)\,\rangle_{{}_{\cal
H}}$
$\displaystyle=\dfrac{1}{h_{p}\,h_{q}}\,\left(\,k(x+h_{q}u\,,\,y+h_{p}u)-k(x\,,\,y+h_{p}u)-k(x+h_{q}u\,,\,y)+k(x,y)\,\right)$
$\displaystyle=\dfrac{1}{h_{p}\,h_{q}}\,\left(\,\partial^{(1)}_{u}k(x\,,\,y+h_{p}u)\,\times\,h_{q}-\partial^{(1)}_{u}k(x\,,\,y)\times
h_{q}+o(h_{q})\,\right)$
$\displaystyle=\partial^{(2)}_{u}\,\partial^{(1)}_{u}k(x\,,\,y)+o_{p,q}(1)$

We have also for some $p$ and $q$ large enough (with $y=x$) and
$\varepsilon>0$,

$\left|\bigl{\langle}\,t_{p}(\,\bm{\cdot}\,,\,y)\;,\;t_{q}(\,\bm{\cdot}\,,\,y)\,\bigr{\rangle}_{{}_{\cal
H}}-\partial^{(2)}_{{u}}\partial^{(1)}_{{u}}k(y\,,\,y)\right|\leq\varepsilon,$

and by (58) we obtain hence
$\|t_{p}(\,\bm{\cdot}\,,\,y)-t_{q}(\,\bm{\cdot}\,,\,y)\|^{2}_{{}_{\cal H}}\leq
4\varepsilon$, which means $\bigl{(}t_{n}(\,\bm{\cdot}\,,\,y)\bigr{)}_{\\!n}$
is a Cauchy sequence in ${\cal H}$. The sequence
$\bigl{(}t_{n}(\,\bm{\cdot}\,,\,y)\bigr{)}_{\\!n}$ converges to a function
$t(\,\bm{\cdot}\,,\,y)$ in ${\cal H}$ and by (53) the convergence is
pointwise. Consequently, we obtain also
$\partial^{(2)}_{{u}}\,k(\,\bm{\cdot}\,,\,y)=t(\,\bm{\cdot}\,,\,y)\in{\cal
H}$. $\Box$■

As the sequence $\bigl{(}t_{n}(\cdot\,,x)\bigr{)}_{n}$ converges to
${\partial^{(2)}_{{u}}}\,k(\,\cdot\,,x)$ in ${\cal H}$, we obtain also

$\left\langle
f\,,\,{\partial^{(2)}_{{u}}}\,k(\,\cdot\,,\,x)\right\rangle_{{\cal
H}}=\lim_{p\to\infty}\left\langle
f\,,\,\dfrac{k(\,\cdot\,,\,x+h_{p})-k(\,\cdot\,,\,x)}{h_{p}}\,\right\rangle_{{\cal
H}}=\lim_{p\to\infty}\dfrac{f(x+h_{p}u)-f(x)}{h_{p}}=\partial_{u}f(x).$ (59)

We provide in theorem 6 an expression of the directional derivative along
direction $u\in E$ of an RKHS function $f$. To that end we first introduce
useful remarks together with an intermediate lemma.

As $k$ is ${C^{(1,1)}}(E\times E)$, the Mercer eigenfunctions $\varphi_{i}$
are $C^{1}(E,\mathbb{C})$. Similarly to remark 9, we get a computational rule
relating the kernel operator derivatives and the inner product
$\langle\bm{\cdot}\,,\,\bm{\cdot}\rangle_{L^{2}_{\mathbb{C}}(E,\nu)}$. For all
$y\in E$ and ${u}\in E$, we note that

${\langle\partial^{(2)}_{{u}}\,k(\cdot\,,\,y),\varphi_{i}\rangle_{{L_{\mathbb{C}}^{2}(E,\nu)}}=\int_{E}\partial^{(2)}_{{u}}\,k(x\,,\,y)\,\overline{\varphi_{i}(x)}\,\,\nu(\mathrm{d}x)=\partial_{u}\int_{E}\overline{k(y\,,\,x)}\,\,\overline{\varphi_{i}(x)}\,\,\nu(\mathrm{d}x)=\partial_{u}\overline{{\cal
L}_{k}(\varphi_{i})(y)}=\mu_{i}\,\partial_{u}\overline{\varphi_{i}(y)}}{.}$

We obtain for all $y\in E$ and ${u}\in E$ that