# Bayesian inference of covariate-parameter relationships for population
modelling

Han Cheng Lie<EMAIL_ADDRESS>Institut für Mathematik, Universität
Potsdam, Campus Golm, Potsdam OT Golm 14476, Germany

###### Abstract

We consider population modelling using parametrised ordinary differential
equation initial value problems (ODE-IVPs), in the setting where the parameter
vector for each individual is not known, but a vector of covariates is
available for each individual and follows an unknown distribution. For each
individual, only one component of the solution to the ODE-IVP can be observed
at only finitely many observation times. An important problem is to identify a
covariate-parameter relationship that maps covariate vectors to parameter
vectors. Such settings and problems arise in pharmacology, where the
observations are blood drug concentrations, the covariates are clinically
observable quantities, and the covariate-parameter relationship is used for
personalised drug dosing. We use the framework presented in (Nickl, 2023) to
analyse a nonparametric, nonlinear Bayesian inverse problem for the unknown
covariate-parameter relationship, for a fixed finite time design, random
covariate design, and a family of rescaled Gaussian priors. We provide
sufficient conditions for posterior contraction, local asymptotic normality,
and a Bernstein–von Mises theorem for linear functionals, and show
theoretically that these conditions are met on a two-compartment model from
the pharmacokinetics literature.

Keywords: ordinary differential equation, initial value problem, population
modelling, covariate parameter relationship, statistical inverse problem,
nonparametric Bayes

##  1 Introduction

Many mathematical models for deterministic continuous dynamical systems are
expressed in terms of ordinary differential equation initial value problems
(ODE-IVPs) where the vector field or the initial condition depend on a vector
of parameters. An important task when applying such parametrised ODE-IVPs is
to identify suitable parameter values so that the models better fit
experimental data, or have better predictive power. This task becomes more
challenging when the task is to model individuals in a population using the
same ODE but different parameters, and when the number of observations of the
ODE solution for each individual is limited, so that estimates of the
parameter vector for each individual may not be available or sufficiently
accurate. Such population modelling and statistical estimation problems arise
frequently in pharmacokinetics, in the context of ‘compartment models’; see
e.g. [17, 7]. For example, a ‘two-compartment model’ involves representing a
patient as a two-dimensional dynamical system $s(t)=(s_{1}(t),s_{2}(t))$,
where $s_{1}(t)$ describes the concentration of a drug in the so-called
‘central compartment’ of the patient and $s_{2}(t)$ describes the
concentration of the drug in the so-called ‘peripheral compartment’. The
‘volume’ of the central and peripheral compartments is assumed to be $V_{1}$
and $V_{2}$ respectively. In many cases, the central compartment includes the
circulatory system together with the target region of the body, and the
peripheral compartment is the complement of the central compartment. The rate
of flow between compartments is assumed to be $Q$. Suppose the drug dosage is
chosen to be proportional to the patient’s weight $w$ and is injected to the
central compartment at time $0$, and suppose that the drug is removed from the
patient’s body via the same compartment at some ‘elimination clearance’ rate
$CL$. Then we obtain the following ODE-IVP on some time interval $[0,T]$:

$\displaystyle V_{1}\frac{\mathrm{d}}{\mathrm{d}t}s_{1}(t)=$ $\displaystyle
Q(s_{2}(t)-s_{1}(t))-CL\ s_{1}(t),$ $\displaystyle V_{1}s_{1}(0)$
$\displaystyle=D_{0}w,$ (1.1) $\displaystyle
V_{2}\frac{\mathrm{d}}{\mathrm{d}t}s_{2}(t)=$ $\displaystyle
Q(s_{1}(t)-s_{2}(t)),$ $\displaystyle s_{2}(0)$ $\displaystyle=0.$

For the ODE-IVP above, the vector $(CL,V_{1},Q,V_{2})$ of parameters cannot be
observed or measured in a clinical setting. The task of estimating the vector
$(CL,V_{1},Q,V_{2})$ of parameters for each patient using observations of the
concentration $s_{1}(t)$ at discrete times $t_{1},\ldots,t_{d_{o}}$ may be
complicated by the fact that for some patients — e.g. extremely young or ill
patients — as few as one blood sample per patient may be collected; see e.g.
[17, Section 3] and [12, p. 136].

One approach to tackling these challenges is to exploit the information that
is available in the form of so-called ‘covariates’, such as the gender, age,
or weight of the patient. Unlike the parameters of the ODE-IVP, the covariates
can be measured in clinical settings. Given a choice of covariates and a
parametrised ODE-IVP, one needs to find a function that maps the vector of
covariates to the vector of parameters. In the pharmacokinetics literature,
this function is sometimes referred to as a ‘parameter-covariate relationship’
or a ‘covariate-parameter relationship’. We shall refer to this function as
the ‘covariate-to-parameter map’ (CPM).

The importance of the CPM in pharmacology is that it can be used to determine
dosing regimes that are tailored to each patient, based on the patient’s
covariate vector. Tailored dosing regimes can help to improve the efficacy of
the drug treatment and reduce the risk of adverse side-effects. However, in
many cases, the CPM is chosen to be a simple function of a covariate.
Piecewise linear functions, power functions, and exponential functions are
frequently used; see the examples in [8, p. 1465], [7, pp. 121-122], and [15,
Equation (2.11)], for example. The choice of functional forms of the
covariate-parameter relationship is often motivated by convenience [9, p. 51].
This raises the question of whether one can infer the true CPM under minimal
assumptions.

#### Outline

The goal of this paper is to establish some mathematical foundations for a
Bayesian statistical approach for inferring the CPM from partial observations
of solutions to parametrised linear homogeneous ODE-IVPs, in the setting where
only finitely many observations of one component of the solution are
available.

In Section 2, we consider the collection of parametrised linear homogeneous
ODE-IVPs with diagonalisable right-hand sides. We state and prove some
properties of the evaluation map that maps a parameter vector to the
corresponding vector of observations of one component of the solution, taken
at a fixed time grid.

In Section 3, we use the results from Section 2 to analyse forward operators
associated to a fixed, finite time design and random covariate design. The
main results in this section consist of local boundedness, local Lipschitz
continuity, and stability estimates of the forward model.

In Section 4, we define a Bayesian inverse problem in which the object to be
inferred is the true CPM that is valid for the entire population. We apply the
framework of [13], and define a family of posteriors associated to a family of
rescaled Gaussian priors. For these posteriors, we establish posterior
contraction, local asymptotic normality, and a Bernstein–von Mises theorem.

In Section 5, we apply a posterior contraction result from Section 4 to a two-
compartment model from the pharmacokinetics literature [6, 14]. We conclude in
Section 6.

#### Contributions

Our analysis uses the theoretical framework presented in [13] for nonlinear
statistical Bayesian inverse problems with rescaled Gaussian process priors.
In this framework, the key challenge lies in verifying that the forward model
satisfies certain regularity and stability estimates. We are not aware of any
other work in the literature where this framework is applied to infer CPMs in
parametrised ODE-IVPs. We therefore believe that the estimates in Section 3
and Section 4 are novel in this context and in the context of statistical
inference for pharmacokinetics. In addition, while we expect that some of the
results in Section 2 are known in the theory of ODEs, we believe that Lemma
2.6 — which states the local Lipschitz continuity of the inverse of the
parameter to observable map — is new.

Our strategy is to prove estimates on the forward model that hold uniformly
with respect to the covariate. In this way, we can prove estimates of the form
given in [13, Chapter 2] without imposing any regularity assumptions on the
unknown covariate distribution, such as admitting a Lebesgue density. This is
advantageous for applications in pharmacokinetics, where in general one has
very little knowledge about the true distribution of patient covariates, and
where the covariates may take discrete values.

###  1.1 Related work

For deterministic differential equations, the work [5] considers the
approximation of solutions of deterministic differential equations by
numerical integration methods, and shows the consistency of the family of
posteriors indexed by the resolution parameter of the numerical method. We
consider standard posterior contraction in the ‘classical’ limit of infinite
data.

The papers [2, 3, 15] develop a two-step approach to the task of parameter
inference for ODEs, where the data are obtained by discrete observations of
the solutions to the parametrised ODE and the ODE does not admit a closed-form
solution, but the ODE vector field is known. In the first step, the parameter
inference problem is expressed as a problem of regressing the data against the
unknown parameters, where the regression function is modelled
nonparametrically, e.g. using B-splines. The second step uses the ‘derivative
matching’ idea in that a minimisation problem is solved to find the parameter
values that best match the observed values of the ODE vector field. The recent
paper [4] develops this approach to the case where the constraint is expressed
by a PDE instead of an ODE. For the above-mentioned papers, Bayesian methods
are used to establish properties such as posterior consistency and
Bernstein–von Mises theorems in the random design setting, for a finite-
dimensional vector of unknowns. In contrast, we aim to infer an infinite-
dimensional object, namely the CPM that maps every covariate vector to the
corresponding vector of parameters to be used in the ODE-IVP, without
considering a specific basis.

The review [7] presents some ideas of covariate pharmacokinetic model
building, and [17] focuses on the Bayesian approach to population
pharmacokinetics. The work [10] uses Bayesian regression to estimate
pharmacokinetic parameters from observed drug concentrations. In [11],
generalised additive models and splines were used in a parametric approach to
describe the CPM. This approach was further analysed and modified in [8]. In
contrast to the parametric or finite-dimensional approaches described above,
the paper [12] used the nonparametric maximum likelihood estimation to
estimate the joint distribution of parameters and covariates, but not the CPM
itself. The work [9] uses regression splines or neural networks to estimate
the CPM, but does not prove posterior contraction. [6] tests the goodness-of-
fit of certain parametric classes of covariate-parameter relationships against
nonparametric alternatives, using kernel-based Tikhonov regularisation and
ideas from statistical learning, but does not consider a Bayesian approach.
The recent work [1] studies the problem of predicting changes in drug
concentrations for pharmacokinetic models, but uses regularisation schemes and
statistical learning instead of a Bayesian approach.

###  1.2 Notation

For $n\in\mathbb{N}$, $[n]\coloneqq\\{1,\ldots,n\\}$. For
$x\in\mathbb{R}^{n}$, $A\in\mathbb{R}^{m\times n}$, and $0<q\leq\infty$,
$\|x\|_{q}\coloneqq(\sum_{i=1}^{n}|x_{i}|^{q})^{1/q}$,
$\|A\|_{q}\coloneqq\sup_{\|x\|_{q}\leq 1}\|Ax\|_{q}$, and
$B_{2}(x,r)\coloneqq\\{z:\|z-x\|_{2}<r\\}$. We denote the set of invertible
elements of $\mathbb{R}^{n\times n}$ by $GL(n,\mathbb{R})$. For
$a,b\in\mathbb{R}$, $a\wedge b\coloneqq\min\\{a,b\\}$ and $a\vee
b\coloneqq\max\\{a,b\\}$. We denote the closure and cardinality of a subset
$A$ of some Euclidean space by $\overline{A}$ and $\\#A$ respectively.

If $(V,\|\cdot\|_{V})$ is a Banach space and $W\subset V$ is a linear
subspace, then we denote by $\overline{W}|_{V}$ the closure of $W$ with
respect to the norm $\|\cdot\|_{V}$, and $B_{V}(x,r)\coloneqq\\{v\in V\ :\
\|v-x\|_{V}<r\\}$. For $d_{1},d_{2}\in\mathbb{N}$, nonempty Borel sets
$D_{1}\subseteq\mathbb{R}^{d_{1}}$, $D_{2}\subseteq\mathbb{R}^{d_{2}}$, a
measure $\mu$ on $D_{1}$, and $0<q\leq\infty$, let
$\|f\|_{L^{q}_{\mu}}\coloneqq(\int_{D_{1}}|f(x)|_{2}^{q}\mu(\mathrm{d}x))^{1/q}$
and $L^{q}_{\mu}(D_{1},D_{2})$ denote the corresponding Banach space. Let
$\|f\|_{\infty}\coloneqq\sup_{x\in D_{1}}\|f(x)\|$. For $\beta\geq 0$, denote
by $H^{\beta}_{\mu}(D_{1},D_{2})$ the Sobolev space of functions from $D_{1}$
to $D_{2}$ with weak derivatives of up to order $\beta$ that are
$L^{2}_{\mu}$-integrable, with
$H^{0}_{\mu}(D_{1},D_{2})=L^{2}_{\mu}(D_{1},D_{2})$. Denote the space of
bounded, continuous functions from $D_{1}$ to $D_{2}$ by $C_{b}(D_{1},D_{2})$.

The notation $a\leftarrow b$ means that $a$ is replaced with $b$, while
$a\lesssim b$ means that $a\leq Cb$ for some positive scalar $C$ that does not
depend on $a$ or $b$.

We fix a common underlying probability space. Given a random variable $Z$
defined on this probability space, $\mu_{Z}$ denotes the law of $Z$.

We denote the set of admissible covariates by
$\mathcal{X}\subset\mathbb{R}^{d_{x}}$, the set of probability measures on
$\mathcal{X}$ by $\mathcal{M}_{1}(\mathcal{X})$, the true unknown population
distribution of covariates by $\mu_{X}$, the open set of admissible parameters
by $\mathcal{P}\subset\mathbb{R}^{d_{p}}$, and the set of admissible CPMs by
$\Theta$. Both $\mathcal{X}$ and $\mathcal{P}$ have nonempty interior. We
denote an arbitrary element of $\mathcal{X}$, $\mathcal{P}$, and $\Theta$ by
$x$, $p$, and $\theta$ respectively. In Section 3, we use $\theta_{0}$ to
denote a reference CPM, and in Section 4 the reference CPM $\theta_{0}$ is
taken to be the true data-generating CPM.

##  2 Properties of solutions to linear ODE-IVPs

In this section, we specify the type of ODE-IVPs that we shall consider and
state some results about their solutions. These results are essential for
proving desirable properties — such as posterior contraction — of the
inference problem that we study in Section 4.

Fix a bounded time interval $[0,T]$ for some $0<T<\infty$. Consider a time-
homogeneous, linear ODE-IVP in $\mathbb{R}^{d_{s}}$, $d_{s}\in\mathbb{N}$,
where both the matrix $A:\mathbb{R}^{d_{p}}\to\mathbb{R}^{d_{s}\times d_{s}}$
and the initial condition $s_{0}:\mathbb{R}^{d_{p}}\to\mathbb{R}^{d_{s}}$
depend on a parameter vector $p\in\mathbb{R}^{d_{p}}$:

$s(0,p)=s_{0}(p)\in\mathbb{R}^{d_{s}},\quad\frac{\mathrm{d}}{\mathrm{d}t}s(t,p)=A(p)s(t,p),\quad
t\in[0,T].$ (2.1)

We refer to $(s(t,p))_{t\in[0,T]}$ as the ‘solution’ of (2.1) for the
parameter $p$, and $s(t,p)$ as the ‘state’ at time $t$ of this solution.

The following assumption states that both the matrix $A(p)$ and initial
condition $s_{0}(p)$ in (2.1) are locally bounded and locally Lipschitz
continuous functions of $p$.

######  Assumption 2.1 (Locally bounded and locally Lipschitz continuous ODE-
IVP data).

For every $M>0$ there exists some $C_{1}(M)>0$ such that

$\displaystyle\sup\\{\|A(p)\|_{2}\vee\|s_{0}(p)\|_{2}:p\in B_{2}(0,M)\\}\leq$
$\displaystyle C_{1}(M),$ (2.2a)
$\displaystyle\sup\left\\{\|A(p)-A(q)\|_{2}\vee\|s_{0}(p)-s_{0}(q)\|_{2}:p,q\in
B_{2}(0,M)\right\\}\leq$ $\displaystyle C_{1}(M)\|p-q\|_{2}.$ (2.2b)

The next result shows that local boundedness and local Lipschitz continuity of
the data of the ODE-IVP (2.1) implies the corresponding properties for the
solution of (2.1). We expect that this result is known in the theory of
parametrised ODE-IVPs, but state and prove it for the sake of completeness.

######  Lemma 2.2.

Suppose 2.1 holds with constant $C_{1}(M)$ for every $M$. Then for every $M>0$
and $t\in[0,T]$,

$\forall(t,p)\in[0,T]\times B_{2}(0,M),\quad\|s(t,p)\|_{2}\leq
e^{C_{1}(M)t}C_{1}(M),$ (2.3)

and there exists $L=L(C_{1}(M),T)>0$ such that

$\forall p,q\in B_{2}(0,M),\quad\sup_{t\in[0,T]}\|s(t,p)-s(t,q)\|_{2}\leq
L\|p-q\|_{2}.$ (2.4)

###### Proof of Lemma 2.2.

Let $t\in[0,T]$ be arbitrary. Since $\|Ax\|_{q}\leq\|A\|_{q}\|x\|_{q}$ for any
$A\in\mathbb{R}^{n\times n}$ and $x\in\mathbb{R}^{n}$, we obtain

$\|s(t,p)\|_{2}=\left\|e^{A(p)t}s_{0}(p)\right\|_{2}\leq\left\|e^{A(p)t}\right\|_{2}\|s_{0}(p)\|_{2}\leq
e^{\|A(p)\|_{2}t}\|s_{0}(p)\|\leq e^{C_{1}(M)t}C_{1}(M),$

where the final inequality follows from (2.2a). This proves (2.3). For the
second statement, we obtain

$\displaystyle\|s(t,p)-s(t,q)\|_{2}$ $\displaystyle=$
$\displaystyle\left\|s_{0}(p)+\int_{0}^{t}A(p)s(r,p)\mathrm{d}r-s_{0}(q)-\int_{0}^{t}A(q)s(r,q)\mathrm{d}r\right\|_{2}$
$\displaystyle\leq$
$\displaystyle\|s_{0}(p)-s_{0}(q)\|_{2}+\left\|\int_{0}^{t}A(p)s(r,p)-A(p)s(r,q)+A(p)s(r,q)-A(q)s(r,q)\mathrm{d}r\right\|_{2}$
$\displaystyle\leq$
$\displaystyle\|s_{0}(p)-s_{0}(q)\|_{2}+\left\|\int_{0}^{t}A(p)\left(s(r,p)-s(r,q)\right)\mathrm{d}r\right\|_{2}+\left\|\int_{0}^{t}\left(A(p)-A(q)\right)s(r,q)\mathrm{d}r\right\|_{2}$
$\displaystyle\leq$
$\displaystyle\|s_{0}(p)-s_{0}(q)\|_{2}+\|A(p)\|_{2}\int_{0}^{t}\left\|s(r,p)-s(r,q)\right\|_{2}\mathrm{d}r+\left\|A(p)-A(q)\right\|_{2}\int_{0}^{t}\|s(r,q)\|_{2}\mathrm{d}r$
$\displaystyle\leq$ $\displaystyle
C_{1}(M)\|p-q\|_{2}+C_{1}(M)\int_{0}^{t}\left\|s(r,p)-s(r,q)\right\|_{2}\mathrm{d}r+C_{1}(M)\|p-q\|_{2}~{}e^{C_{1}(M)t}C_{1}(M)t$
$\displaystyle\leq$ $\displaystyle
C_{1}(M)\left(1+e^{C_{1}(M)t}C_{1}(M)t\right)\|p-q\|_{2}+C_{1}(M)\int_{0}^{t}\left\|s(r,p)-s(r,q)\right\|_{2}\mathrm{d}r,$

where the final two inequalities use (2.2a) and (2.2b). Gronwall’s inequality
yields (2.4) with $L(C_{1}(M),T)\coloneqq
e^{C_{1}(M)T}C_{1}(M)(1+e^{C_{1}(M)T}C_{1}(M)T)$. ∎

Next, we make the following assumption.

######  Assumption 2.3 (Diagonalisable ODE matrix).

For every $p\in\mathcal{P}$, there exists $\Lambda(p),V(p)\in
GL(d_{s},\mathbb{R})$ with diagonal $\Lambda(p)$, such that $A(p)$ in (2.1)
satisfies $A(p)=V(p)\Lambda(p)V^{-1}(p)$.

There are linear homogeneous ODEs that do not satisfy 2.3, e.g. if $A(p)$ has
complex eigenvalues. The importance of 2.3 is to keep the analysis in this
section and in subsequent sections simple. In addition, as we show in Section
5, there are examples in the literature that satisfy 2.3.

Given (2.1), it is known that $s(t,p)=e^{A(p)t}s_{0}(p)$. Thus 2.3 implies
that $s(t,p)=V(p)\exp(\Lambda(p)t)V(p)^{-1}s_{0}(p)$, and every component
$s_{i}$, $i\in[d_{s}]$, of the solution of (2.1) is a linear combination of
exponential functions. We fix an arbitrary choice and consider the first
component of the solution from now on. For every $p\in\mathcal{P}$ there exist
$(\widehat{a}_{i}(p))_{i\in[d_{s}]},(\widehat{\lambda}_{i}(p))_{i\in[d_{s}]}\in\mathbb{R}^{d_{s}}$
such that

$\forall t\geq 0,\quad
s_{1}(t,p)=\sum_{i\in[d_{s}]}\widehat{a}_{i}(p)e^{\widehat{\lambda}_{i}(p)t},$

where $\widehat{a}(p)\in\mathbb{R}^{d_{s}}$ depends only on the eigenvector
matrix $V(p)$ and initial condition $s_{0}(p)$ and satisfies
$\sum_{i\in[d_{s}]}\widehat{a}_{i}(p)=s_{1}(0,p)$, and the
$(\widehat{\lambda}_{i}(p))_{i\in[d_{s}]}$ are the diagonal entries of
$\Lambda(p)$, i.e. the eigenvalues of $A(p)$. Consider the functions
$(\widehat{\lambda}_{i})_{i\in[d_{s}]}$. If there exist distinct
$k,\ell\in[d_{s}]$ with $\widehat{\lambda}_{k}=\widehat{\lambda}_{\ell}$ on
$\mathcal{P}$, then
$\widehat{a}_{k}(p)e^{\widehat{\lambda}_{k}(p)t}+\widehat{a}_{\ell}(p)e^{\widehat{\lambda}_{\ell}(p)t}=(\widehat{a}_{k}(p)+\widehat{a}_{\ell}(p))e^{\widehat{\lambda}_{k}(p)t}$
for every $t$. This motivates the following definition.

######  Definition 2.4.

Suppose 2.3 holds, and let $\\{\lambda_{j}:j\in[\mathfrak{d}]\\}$ be the
distinct elements of the set $\\{\widehat{\lambda}_{i}:i\in[d_{s}]\\}$. Then
the _intrinsic dimension_ of $t\mapsto s_{1}(t,p)$ is $2\mathfrak{d}$, and the
_coefficient map_ of $t\mapsto s_{1}(t,p)$ is the map
$p\mapsto(a(p),\lambda(p))\in\mathbb{R}^{2\mathfrak{d}}$, where

$\forall t\geq 0,\quad
s_{1}(t,p)=\sum_{i\in[\mathfrak{d}]}a_{i}(p)e^{\lambda_{i}(p)t}.$ (2.5)

In what follows, we shall assume that our information about the dynamical
system is obtained solely from measurements of the first component $s_{1}$.
The choice of the first component is arbitrary. We shall also make the
following important assumption.

######  Assumption 2.5.

The coefficient map $\mathcal{P}\ni
p\mapsto(a(p),\lambda(p))\in(\mathbb{R}\setminus\\{0\\})^{\mathfrak{d}}\times\mathbb{R}^{\mathfrak{d}}$
is $C^{1}$, and for every $q\in\mathcal{P}$,

$\mathcal{J}(q)\coloneqq\begin{bmatrix}\nabla_{p}a_{1}(p)|\cdots|\nabla_{p}a_{\mathfrak{d}}(p)|\nabla_{p}\lambda_{1}(p)|\cdots|\nabla_{p}\lambda_{\mathfrak{d}}(p)\end{bmatrix}^{\top}\bigr{|}_{p=q}\in\mathbb{R}^{2\mathfrak{d}\times
d_{p}}$ (2.6)

has full rank.

Next, we fix an arbitrary finite time design, i.e. an arbitrary collection
$(t_{j})_{j\in[d_{o}]}\subset[0,T]$ of $d_{o}\in\mathbb{N}$ distinct
observation times. Define the map that evaluates $s_{1}$ at
$(t_{j})_{j\in[d_{o}]}$:

$\mathbb{R}^{d_{p}}\supseteq\mathcal{P}\ni
p\mapsto\mathscr{E}(p)\equiv\mathscr{E}(p;(t_{j})_{j\in[d_{o}]})\coloneqq(s_{1}(t_{j},p))_{j\in[d_{o}]}\in\mathbb{R}^{d_{o}}.$
(2.7)

The result below provides a sufficient condition for the restriction of
$\mathscr{E}$ to an arbitrary ball to have an inverse, and for this inverse to
be Lipschitz. It exploits (2.5), i.e. the fact that the observed component
$s_{1}$ of the solution of the ODE-IVP (2.1) is a sum of exponential
functions, and uses 2.5 to obtain a lower bound on the size $d_{o}$ of the
fixed, finite time design $(t_{j})_{j\in[d_{o}]}$.

######  Lemma 2.6 (Locally Lipschitz continuity of inverse evaluation map).

Suppose that 2.3 and 2.5 hold. If $d_{o}\geq d_{p}=2\mathfrak{d}$, then for
every $M>0$ such that $B_{2}(0,M)\subset\mathcal{P}$, there exists
$L(M,(t_{j})_{j\in[d_{o}]})>0$ such that

$\forall p,q\in B_{2}(0,M),\quad\|p-q\|_{2}\leq
L(M,(t_{j})_{j\in[d_{o}]})\|(s_{1}(t_{j},p))_{j\in[d_{o}]}-(s_{1}(t_{j},q))_{j\in[d_{o}]}\|_{2}.$
(2.8)

###### Proof of Lemma 2.6.

Note that if $d_{o}>d_{p}$, then

$\|(s_{1}(t_{j},p))_{j\in[d_{p}]}-(s_{1}(t_{j},q))_{j\in[d_{p}]}\|_{2}\leq\|(s_{1}(t_{j},p))_{j\in[d_{o}]}-(s_{1}(t_{j},q))_{j\in[d_{o}]}\|_{2}.$

Thus, it suffices to consider the case where $d_{o}=d_{p}$.

Using the product rule and the hypothesis in 2.5 that $\mathcal{P}\ni
p\mapsto(a(p),\lambda(p)$ is $C^{1}$, it follows from (2.5) that

$\forall t\geq
0,\quad\nabla_{p}s_{1}(t,p)=\sum_{i\in[\mathfrak{d}]}e^{\lambda_{i}(p)t}\left(\nabla_{p}a_{i}(p)+a_{i}(p)\nabla_{p}\lambda_{i}(p)t\right)\in\mathbb{R}^{d_{p}}.$
(2.9)

Thus, the map $p\mapsto\mathscr{E}(p)=(s_{1}(t_{j},p))_{j\in[d_{o}]}$ is
$C^{1}$ on $\mathcal{P}$, and its Jacobian is given by

$\mathscr{J}(q)\coloneqq\begin{bmatrix}\nabla_{p}s_{1}(t_{1},p)|\cdots|\nabla_{p}s_{1}(t_{d_{o}},p)\end{bmatrix}^{\top}\bigr{|}_{p=q}\in\mathbb{R}^{d_{o}\times
d_{p}}.$ (2.10)

Let $0\neq\alpha\in\mathbb{R}^{d_{p}}$ and $q\in\mathcal{P}$ be arbitrary. We
shall show that $\mathscr{J}(q)\alpha$ is nonzero. By the hypothesis in 2.5
that $\mathcal{J}(q)$ defined in (2.6) has full rank, it follows that if
$2\mathfrak{d}\geq d_{p}$, then

$\mathbb{R}^{2\mathfrak{d}}\ni\left(\left\langle\nabla_{p}a_{1}(p),\alpha\right\rangle,\ldots,\left\langle\nabla_{p}a_{\mathfrak{d}}(p),\alpha\right\rangle,\left\langle\nabla_{p}\lambda_{1}(p),\alpha\right\rangle,\ldots,\left\langle\nabla_{p}\lambda_{\mathfrak{d}}(p),\alpha\right\rangle\right)\bigr{|}_{p=q}=\mathcal{J}(q)\alpha\neq
0.$

Combining this with the hypothesis in 2.5 that
$a(p)\in(\mathbb{R}\setminus\\{0\\})^{\mathfrak{d}}$ for every
$p\in\mathcal{P}$, we may apply Lemma A.1 with $n\leftarrow\mathfrak{d}$,
$\beta_{k}\leftarrow\lambda_{k}(p)$, $\gamma_{k}\leftarrow
a_{k}(q)\left\langle\nabla_{p}\lambda_{k}(p),\alpha\right\rangle$, and
$\xi_{k}\leftarrow\left\langle\nabla_{p}a_{k}(p),\alpha\right\rangle$, to
conclude that

$t\mapsto\left\langle\nabla_{p}s_{1}(t,p),\alpha\right\rangle|_{p=q}=\biggr{[}\sum_{i\in[\mathfrak{d}]}e^{\lambda_{i}(p)t}\bigr{(}\left\langle\nabla_{p}a_{i}(p),\alpha\right\rangle+a_{i}(p)\left\langle\nabla_{p}\lambda_{i}(p),\alpha\right\rangle
t\bigr{)}\biggr{]}\biggr{|}_{p=q}$

has at most $2\mathfrak{d}-1$ roots in $\mathbb{R}$. Thus, if $d_{o}\geq
d_{p}$ and if $d_{p}\geq 2\mathfrak{d}$, then

$\mathscr{J}(q)\alpha=\begin{bmatrix}\left\langle\nabla_{p}s_{1}(t_{1},p),\alpha\right\rangle|\cdots|\left\langle\nabla_{p}s_{1}(t_{d_{o}},p),\alpha\right\rangle\end{bmatrix}^{\top}\bigr{|}_{p=q}\neq
0.$ (2.11)

Since $\alpha\in\mathbb{R}^{d_{p}}\setminus\\{0\\}$ was arbitrary, it follows
that $\mathscr{J}(q)$ has full rank. If $d_{o}=d_{p}$, then $\mathscr{J}(q)$
is invertible, and $\mathscr{E}$ satisfies the hypotheses of the inverse
function theorem.

Let $M>0$ be such that $B_{2}(0,M)\subset\mathcal{P}$. Recalling from Section
1.2 that $\mathcal{P}$ is assumed to be open, it follows that
$\overline{B_{2}(0,M)}\subset\mathcal{P}$. Since $\mathscr{E}$ is $C^{1}$ on
$\mathcal{P}$ and its Jacobian is invertible on $\overline{B_{2}(0,M)}$, we
may apply the inverse function theorem to conclude that $\mathscr{E}$ is
injective in a neighbourhood $U$ of $\overline{B_{2}(0,M)}$, and its inverse
$\mathscr{E}^{-1}$ is a $C^{1}$ map from $\mathscr{E}(U)$ to $U$. Since
$C^{1}$ maps are locally Lipschitz maps, and since $\overline{B_{2}(0,M)}$ is
a convex, compact subset of $U$, it follows that $\mathscr{E}^{-1}$ is
Lipschitz on $\overline{B_{2}(0,M)}$ with Lipschitz constant

$\left\|\mathscr{E}^{-1}|_{\overline{B_{2}(0,M)}}\right\|_{\textup{Lip}}\coloneqq\sup\left\\{\frac{\|p-q\|_{2}}{\|\mathscr{E}(p)-\mathscr{E}(q)\|_{2}}:p,q\in\overline{B_{2}(0,M)},p\neq
q\right\\}<\infty.$

This proves the desired inequality (2.8), with
$L(M,(t_{j})_{j\in[d_{o}]})\coloneqq\|\mathscr{E}^{-1}|_{\overline{B_{2}(0,M)}}\|_{\textup{Lip}}$.
∎

##  3 Forward operator for fixed time and random covariate design

Recall the coefficient map defined in Definition 2.4.

######  Assumption 3.1.

The coefficient map $p\mapsto(a(p),\lambda(p))\in\mathbb{R}^{2\mathfrak{d}}$
is such that for every $(t,p)\in[0,T]\times\mathcal{P}$, $s_{1}(t,p)>0$.

A sufficient condition for 3.1 is that for every $p\in\mathcal{P}$,
$a(p)\in\mathbb{R}_{>0}^{\mathfrak{d}}$, for example. In Section 5 we describe
one instance of (2.1) for which this sufficient condition holds.

Let $\Theta\subset L^{2}_{\mu_{X}}(\mathcal{X},\mathcal{P})$ be nonempty. For
$(t_{j})_{j\in[d_{o}]}\subset[0,T]$ as in (2.7), define the forward operator

$\mathcal{G}:\Theta\to
L^{2}_{\mu_{X}}(\mathcal{X},\mathbb{R}^{d_{o}}),\quad\theta(\cdot)\mapsto\mathcal{G}(\theta)(\cdot)\coloneqq(\log
s_{1}(t_{j},\theta(\cdot)))_{j\in[d_{o}]}.$ (3.1)

The forward operator is obtained by applying the logarithm to every component
of the evaluation map $\mathscr{E}$ from (2.7). 3.1 ensures that the forward
operator maps CPMs to $\mathbb{R}^{d_{o}}$-valued functions on $\mathcal{X}$.
The choice of the logarithm is motivated by (2.5): for every
$p\in\mathcal{P}$, $t\mapsto s_{1}(t,p)$ is an exponentially growing or
decaying function.

Fix a ‘regularisation space’ $(\mathcal{R},\|\cdot\|_{\mathcal{R}})$, i.e. a
normed subspace of $\Theta$, where

$\mathcal{R}\subseteq
L^{\infty}(\mathcal{X},\mathcal{P})\cap\Theta,\quad\|f\|_{\mathcal{R}}\geq\|f\|_{\infty},\quad
B_{\mathcal{R}}(M)\coloneqq\\{\phi\in\mathcal{R}\ :\
\|\phi\|_{\mathcal{R}}<M\\}.$ (3.2)

We will specify $\mathcal{R}$ further in Section 4. For the results in this
section, the properties in (3.2) will suffice.

######  Proposition 3.2 (Local boundedness and Lipschitz continuity of forward
operator).

Suppose 2.1, 2.3 and 3.1 hold. Then for every $M>0$, there exists some
$C_{2}(M,T)$ such that

$\sup_{\theta\in\Theta\cap
B_{\mathcal{R}}(M)}\sup_{x\in\mathcal{X}}\|\mathcal{G}(\theta)(x)\|_{2}\leq\sqrt{d_{o}}C_{2}(M,T).$
(3.3)

In addition, for the constant $L(M,T)$ in (2.4), it holds for every
$x\in\mathcal{X}$ and $\theta^{(i)}\in B_{\mathcal{R}}(M)$, $i=1,2$ that

$\|\mathcal{G}(\theta^{(1)})(x)-\mathcal{G}(\theta^{(2)})(x)\|_{2}\leq\sqrt{d_{o}}L(M,T)e^{C_{2}(M,T)}\|\theta^{(1)}(x)-\theta^{(2)}(x)\|_{2}.$
(3.4)

Thus,

$\forall\mu\in\mathcal{M}_{1}(\mathcal{X}),\
q\in[0,\infty],\quad\|\mathcal{G}(\theta^{(1)})-\mathcal{G}(\theta^{(2)})\|_{L^{q}_{\mu}}\leq\sqrt{d_{o}}L(M,T)e^{C_{2}(M,T)}\|\theta^{(1)}-\theta^{(2)}\|_{L^{q}_{\mu}}.$
(3.5)

The conclusion (3.5) implies that $\mathcal{G}$ satisfies [13, Condition
2.1.1], with the choices

$\mathcal{Z}\leftarrow\mathcal{X},\ W\leftarrow\mathcal{P},\
V\leftarrow\mathbb{R}^{d_{o}},\ \zeta\leftarrow\mu,\ \lambda\leftarrow\mu,\
\kappa\leftarrow 0,\ U\leftarrow 1\vee\sqrt{d_{o}}L(M,T)e^{C_{2}(M,T)}.$ (3.6)

###### Proof of Proposition 3.2.

Since 2.1 holds, we may apply the conclusions (2.3) and (2.4) of Lemma 2.2.
Fix an arbitrary $(t,p)\in[0,T]\times B_{2}(0,M)$. Then

$(s_{0})_{1}(p)e^{-\|A(p)\|_{2}t}\leq s_{1}(t,p)\leq e^{C_{1}(M)t}C_{1}(M),$

where the upper bound follows from (2.3). For the lower bound, we use 2.3 and
the definition of the spectral norm $\|\cdot\|_{2}$ to bound
$\lambda_{i}(p)\geq-\|A(p)\|_{2}$ for every $i\in[\mathfrak{d}]$, and then
apply (2.5) together with the relation that
$\sum_{i\in[\mathfrak{d}]}a_{i}(p)=(s_{0})_{1}(p)$. By 3.1,
$(s_{0})_{1}(p)=s_{1}(0,p)>0$. Using $a\vee b\leq a+b$ for $a,b\geq 0$, we
obtain

$\sup_{p\in B_{2}(0,M)}|\log s_{1}(t,p)|\leq C_{1}(M)t+|\log
C_{1}(M)|+\sup_{p\in B_{2}(0,M)}|\log(s_{0})_{1}(p)|~{}\|A(p)\|_{2}t.$

By (2.2a), $\|A(p)\|_{2}\leq C_{1}(M)$. By (2.2a) and (2.2b), $p\mapsto
s_{0}(p)$ is locally bounded and Lipschitz, hence so is
$p\mapsto(s_{0})_{1}(p)$. By 3.1 and the definition of the
$\|\cdot\|_{2}$-norm, $0<(s_{0})_{1}(p)\leq\|s_{0}(p)\|_{2}$ for every
$p\in\mathcal{P}$. Thus, for every $M>0$ there exists some $C^{\prime}(M)>0$
such that $\sup_{p\in B_{2}(0,M)}|\log(s_{0})_{1}(p)|\leq C^{\prime}(M)$, and

$\sup_{t\in[0,T]}\sup_{p\in B_{2}(0,M)}|\log s_{1}(t,p)|\leq C_{1}(M)T+|\log
C_{1}(M)|+C^{\prime}(M)C_{1}(M)T\eqqcolon C_{2}(M,T).$ (3.7)

If $\theta\in B_{\mathcal{R}}(M)$, then since
$\|\cdot\|_{\mathcal{R}}\geq\|\cdot\|_{\infty}$ by (3.2), it follows that
$\theta(x)\in B_{2}(0,M)$. Thus, by the definition (3.1) of $\mathcal{G}$, we
obtain (3.3).

To prove (3.4), we use local Lipschitz continuity of the logarithm (A.1a) and
the conclusion (2.4) from Lemma 2.2: for arbitrary $p,q\in B_{2}(0,M)$ and
$t\in[0,T]$,

$|\log s_{1}(t,p)-\log
s_{1}(t,q)|\leq\frac{|s_{1}(t,p)-s_{1}(t,q)|}{s_{1}(t,p)\wedge
s_{1}(t,q)}\leq\frac{L(M,T)\|p-q\|_{2}}{s_{1}(t,p)\wedge s_{1}(t,q)}.$

By (3.7), $\inf_{t\in[0,T]}\inf_{p\in
B_{2}(0,M)}s_{1}(t,p)\geq\exp(-C_{2}(M,T))$. Thus

$|\log s_{1}(t,p)-\log s_{1}(t,q)|\leq L(M,T)e^{C_{2}(M,T)}\|p-q\|_{2}$

and using the definition (3.1) of $\mathcal{G}$ then yields

$\forall
x\in\mathcal{X},\quad\|\mathcal{G}(\theta^{(1)})(x)-\mathcal{G}(\theta^{(2)})(x)\|_{2}\leq\sqrt{d_{o}}L(M,T)e^{C_{2}(M,T)}\|\theta^{(1)}(x)-\theta^{(2)}(x)\|_{2},$

which proves (3.4). The inequality (3.5) follows directly from (3.4). ∎

######  Proposition 3.3 (Stability estimate for forward operator).

Suppose 2.1, 2.3, 2.5, and 3.1 hold, and suppose $d_{o}\geq d_{p}$. Then for
every $M>0$, there exists $C_{3}(M,T,(t_{j})_{j\in[d_{o}]})>0$ such that for
any $\theta^{(i)}\in B_{\mathcal{R}}(M)$, $i=1,2$,

$\forall
x\in\mathcal{X},\quad\left\|\theta^{(1)}(x)-\theta^{(2)}(x)\right\|_{2}\leq
C_{3}(M,T,(t_{j})_{j\in[d_{o}]})\left\|\mathcal{G}(\theta^{(1)})(x)-\mathcal{G}(\theta^{(2)})(x)\right\|_{2}.$

Thus,

$\forall\mu\in\mathcal{M}_{1}(\mathcal{X}),\
q\in[0,\infty],\quad\left\|\theta^{(1)}-\theta^{(2)}\right\|_{L^{q}_{\mu}}\leq
C_{3}(M,T,(t_{j})_{j\in[d_{o}]})\left\|\mathcal{G}(\theta^{(1)})-\mathcal{G}(\theta^{(2)})\right\|_{L^{q}_{\mu}}.$

The conclusion of Proposition 3.3 imply that $\mathcal{G}$ satisfies [13,
Condition 2.1.4] for $L^{2}_{\zeta}$ and $L^{2}_{\lambda}$ as in (3.6),
arbitrary $\theta_{0}$ and $\delta$, and

$L^{\prime}=L^{\prime}_{\mathcal{G}}\leftarrow
C_{3}(M,T,(t_{j})_{j\in[d_{o}]}),\quad\eta\leftarrow 1.$ (3.8)

###### Proof of Proposition 3.3.

Fix $M>0$. If $\theta^{(i)}\in B_{\mathcal{R}}(M)$, $i=1,2$, then for every
$x\in\mathcal{X}$, $\theta^{(i)}(x)\in B_{2}(0,M)$. Since 2.3 and 2.5 hold and
since $d_{o}\geq d_{p}$, we may apply Lemma 2.6: there exists
$L(M,(t_{j})_{j\in[d_{o}]})$ such that for every $x\in\mathcal{X}$,

$\left\|\theta^{(1)}(x)-\theta^{(2)}(x)\right\|_{2}\leq
L(M,(t_{j})_{j\in[d_{o}]})\left\|(s_{1}(t_{j},\theta^{(1)}(x)))_{j\in[d_{o}]}-(s_{1}(t_{j},\theta^{(2)}(x)))_{j\in[d_{o}]}\right\|_{2}.$

Since 2.1 holds, we can apply the bound (2.3) from Lemma 2.2 and the local
Lipschitz continuity of the exponential function (A.1b):

$\displaystyle\left|s_{1}(t,p)-s_{1}(t,q)\right|\leq$ $\displaystyle e^{\log
s_{1}(t,p)\vee\log s_{1}(t,q)}|\log s_{1}(t,p)-\log s_{1}(t,q)|$
$\displaystyle\leq$ $\displaystyle e^{C_{1}(M)T}C_{1}(M)|\log s_{1}(t,p)-\log
s_{1}(t,q)|.$

Thus, by definition (3.1) of $\mathcal{G}$, it holds for every
$x\in\mathcal{X}$ that

$\left\|\theta^{(1)}(x)-\theta^{(2)}(x)\right\|_{2}\leq
L(M,(t_{j})_{j\in[d_{o}]})e^{C_{1}(M)T}C_{1}(M)\left\|\mathcal{G}(\theta^{(1)})(x)-\mathcal{G}(\theta^{(2)})(x)\right\|_{2},$

which yields the first conclusion, with
$C_{3}(M,T,(t_{j})_{j\in[d_{o}]})\coloneqq
L(M,(t_{j})_{j\in[d_{o}]})e^{C_{1}(M)T}C_{1}(M)$. By integration, the second
conclusion follows. ∎

For arbitrary $\theta_{0}\in\Theta$, define the transpose of the Jacobian of
$\mathcal{G}$:

$\mathcal{P}\ni q\mapsto\mathfrak{J}(q)\coloneqq\begin{bmatrix}\nabla_{p}\log
s_{1}(t_{1},p)~{}|\cdots|~{}\nabla_{p}\log
s_{1}(t_{d_{o}},p)\end{bmatrix}^{\top}\bigr{|}_{p=q}\in\mathbb{R}^{d_{o}\times
d_{p}}.$ (3.9)

######  Lemma 3.4.

Suppose 2.3, 2.5, and 3.1 hold. If $d_{o}\geq d_{p}=2\mathfrak{d}$, then for
every $q\in\mathcal{P}$, $\mathfrak{J}(q)$ has full rank.

###### Proof of Lemma 3.4.

Since 2.3, 2.5, and 3.1 hold, we may apply statements in the proof of Lemma
2.6.

By the chain rule, $\nabla_{p}\log
s_{1}(t,p)=(s_{1}(t,p))^{-1}\nabla_{p}s_{1}(t,p)$, for every
$(t,p)\in[0,T]\times\mathcal{P}$. Thus, the Jacobian $\mathscr{J}(q)$ defined
in (2.10) and the Jacobian $\mathfrak{J}(q)$ in (3.9) satisfy

$\forall
q\in\mathcal{P},\quad\mathfrak{J}(q)=\begin{bmatrix}s_{1}(t_{1},q)&0&\cdots&0\\\
0&s_{1}(t_{2},q)&\ddots&\vdots\\\ \vdots&\ddots&\ddots&0\\\
0&\cdots&0&s_{1}(t_{o},q)\end{bmatrix}^{-1}\mathscr{J}(q),$

where the existence of the inverse matrix on the right-hand side is guaranteed
by 3.1. By (2.11), $\mathscr{J}(q)\in\mathbb{R}^{d_{o}\times d_{p}}$ has full
rank. The equation above implies that $\mathfrak{J}(q)$ also has full rank. ∎

The following assumption strengthens the $C^{1}$ condition of the coefficient
map in 2.5.

######  Assumption 3.5.

The coefficient map $\mathcal{P}\ni p\mapsto(a(p),\lambda(p))$ is $C^{2}$.

The following assumption constrains a reference CPM $\theta_{0}$. The
assumption holds if $\|\theta_{0}\|_{\infty}$ is finite, for example.

######  Assumption 3.6.

There exists a compact subset $K$ of $\mathcal{P}$ such that
$\\{\theta_{0}(x):x\in\mathcal{X}\\}\subset K$.

The next result describes the derivative of the forward operator.

######  Proposition 3.7.

Suppose 2.3, 2.5, and 3.1 hold. Let $\theta_{0}\in\Theta$ and let $H\subset
L^{\infty}(\mathcal{X},\mathcal{P})$ be a linear space such that for some
$\epsilon>0$, $\\{\theta_{0}+h:h\in H,\
\|h\|_{\infty}<\epsilon\\}\subset\Theta$. Define a linear operator
$\mathbb{I}_{\theta_{0}}$ acting on $H$ according to

$H\ni
h(\cdot)\mapsto\mathbb{I}_{\theta_{0}}[h](\cdot)\coloneqq\mathfrak{J}(\theta_{0}(\cdot))h(\cdot).$
(3.10)

  1. 1.

For any $\mu\in\mathcal{M}_{1}(\mathcal{X})$,

$\rho_{\theta_{0}}[h]\coloneqq\left\|\mathcal{G}(\theta_{0}+h)-\mathcal{G}(\theta_{0})-\mathbb{I}_{\theta_{0}}[h]\right\|_{L^{2}_{\mu}}=o(\|h\|_{\infty}),\quad\|h\|_{\infty}\to
0.$ (3.11)

  2. 2.

If $d_{o}\geq d_{p}=2\mathfrak{d}$, then $\mathbb{I}_{\theta_{0}}$ is
injective.

  3. 3.

If $\sum_{j\in[d_{o}]}\|\nabla_{p}\log
s_{1}(t_{j},p)|_{p=\theta_{0}}\|^{2}_{L^{2}_{\mu}}$ is finite, then
$\mathbb{I}_{\theta_{0}}:(H,\left\langle\cdot,\cdot\right\rangle_{L^{2}_{\mu}(\mathcal{X},\mathcal{P})})\to
L^{2}_{\mu}(\mathcal{X},\mathbb{R}^{d_{o}})$ is continuous.

  4. 4.

If in addition 3.5 and 3.6 hold, then
$\rho_{\theta_{0}}[h]=O(\|h\|_{\infty}^{2})$.

Statements 1–3 of Proposition 3.7 imply that $\mathcal{G}$ satisfies [13,
Condition 3.1.1], for the spaces $L^{2}_{\zeta}(\mathcal{Z},W)$ and
$L^{2}_{\lambda}(\mathcal{X},V)$ defined by (3.6). Statement 4 of Proposition
3.7 implies that [13, Condition 4.1.4] holds under simpler conditions; see the
discussion after equation (4.6) on [13, p.70]. We will use this fact in the
proof of Theorem 4.6.

###### Proof of Proposition 3.7.

Let $\theta_{0}$ and $H$ be as given, and fix an arbitrary $h\in H$.

By 2.5, $\mathcal{P}\ni p\mapsto(a(p),\lambda(p))$ is $C^{1}$ smooth. By
applying Taylor’s theorem to the map $p\mapsto\log s_{1}(t,p)$, we have for
every $(t,x)\in[0,T]\times\mathcal{X}$ that

$\log s_{1}(t,\theta_{0}(x)+h(x))=\log
s_{1}(t,\theta_{0}(x))+\left\langle\nabla_{p}\log
s_{1}(t,p)|_{p=\theta_{0}(x)},h(x)\right\rangle+o(\|h(x)\|_{2}).$

By the definitions (3.1) and (3.10) of $\mathcal{G}$ and
$\mathbb{I}_{\theta_{0}}$ respectively, this implies that

$\forall
x\in\mathcal{X},\quad\|\mathcal{G}(\theta_{0}+h)(x)-\mathcal{G}(\theta_{0})(x)-\mathbb{I}_{\theta_{0}}[h](x)\|_{2}=o(\|h(x)\|_{2}).$

Fix an arbitrary $\mu\in\mathcal{M}_{1}(\mathcal{X})$. By the preceding fact,
the dominated convergence theorem, and the fact that
$\|h(x)\|_{2}\leq\|h\|_{\infty}$,

$\lim_{\|h\|_{\infty}\to
0}\frac{\rho_{\theta_{0}}[h]^{2}}{\|h\|_{\infty}^{2}}=\lim_{\|h\|_{\infty}\to
0}\frac{\left\|\mathcal{G}(\theta_{0}+h)-\mathcal{G}(\theta_{0})-\mathbb{I}_{\theta_{0}}[h]\right\|^{2}_{L^{2}\mu}}{\|h\|^{2}_{\infty}}=\int_{\mathcal{X}}\lim_{\|h\|^{2}_{\infty}\to
0}\frac{o(\|h(x)\|^{2}_{2})}{\|h\|^{2}_{\infty}}\mu(\mathrm{d}x)=0.$

This proves statement 1.

By Lemma 3.4, $\mathfrak{J}(\theta_{0}(x))$ has full rank for every
$x\in\mathcal{X}$. Thus, by (3.10), $\mathbb{I}_{\theta_{0}}[h_{1}-h_{2}]=0$
if and only if $h_{1}-h_{2}=0$. This proves statement 2.

By (3.10), (3.9), and the Cauchy–Schwarz inequality,

$\displaystyle\left\|\mathbb{I}_{\theta_{0}}[h]\right\|^{2}_{L^{2}_{\mu}}=$
$\displaystyle\sum_{j\in[d_{o}]}\left\|\left\langle\nabla_{p}\log
s_{1}(t_{j},p)|_{p=\theta_{0}},h\right\rangle\right\|^{2}_{L^{2}_{\mu}}\leq\sum_{j\in[d_{o}]}\left\|\nabla_{p}\log
s_{1}(t_{j},p)|_{p=\theta_{0}}\right\|^{2}_{L^{2}_{\mu}}\left\|h\right\|^{2}_{L^{2}_{\mu}},$

which proves statement 3.

If 3.5 holds, then by Taylor’s theorem, it holds for any
$(t,x)\in[0,T]\times\mathcal{X}$ that

$|\log s_{1}(t,(\theta_{0}+h)(x))-\log
s_{1}(t,\theta_{0}(x))-\left\langle\nabla_{p}\log
s_{1}(t,p)|_{p=\theta_{0}(x)},h(x)\right\rangle|\leq
C(\theta_{0},t,x)\|h(x)\|_{2}^{2},$

where $C(\theta_{0},t,x)\lesssim\|\nabla_{p}^{2}\log
s_{1}(t,p)|_{p=\theta_{0}(x)}\|_{2}$. Let $K$ be the compact set described in
3.6. Define $C(K)>0$ such that

$\max_{j\in[d_{o}]}\sup_{x\in\mathcal{X}}\left\|\nabla_{p}^{2}\log
s_{1}(t_{j},p)|_{p=\theta_{0}(x)}\right\|_{2}\leq\max_{j\in[d_{o}]}\sup_{q\in
K}\left\|\nabla_{p}^{2}\log s_{1}(t_{j},p)|_{p=q}\right\|_{2}\eqqcolon C(K).$

By applying the extreme value theorem to the continuous maps
$p\mapsto\|\nabla^{2}_{p}\log s_{1}(t_{j},p)\|_{2}$, $j\in[d_{o}]$, it follows
that $C(K)$ is finite. Thus,

$\forall
x\in\mathcal{X},\quad\|\mathcal{G}(\theta_{0}+h)(x)-\mathcal{G}(\theta_{0})(x)-\mathbb{I}_{\theta_{0}}[h](x)\|_{2}\lesssim
C(K)\|h(x)\|_{2}\leq C(K)\|h\|_{\infty}^{2},$

and by the definition (3.11) of $\rho_{\theta_{0}}[h]$, statement 4 follows. ∎

######  Proposition 3.8.

Suppose that 2.1, 2.3, 2.5, and 3.1 hold. Let
$\Theta=H=L^{\infty}(\mathcal{X},\mathcal{P})$ and $\theta_{0}\in\Theta$. Then
for every $M>0$ there exists $L>0$ such that for every $\theta^{(i)}$ with
$\|\theta^{(i)}\|_{\mathcal{R}}\leq M$ for $i=1,2$,

$\left\|\mathcal{G}(\theta^{(1)})-\mathcal{G}(\theta^{(2)})\right\|_{\infty}\leq
L\left\|\theta^{(1)}-\theta^{(2)}\right\|_{\infty}.$

In addition, $\mathbb{I}_{\theta_{0}}:(H,\|\cdot\|_{\infty})\to
L^{\infty}(\mathcal{X},\mathbb{R}^{d_{o}})$ is continuous.

The conclusions of Proposition 3.8 imply that [13, Condition 4.1.1] holds for
$H\leftarrow L^{\infty}(\mathcal{X},\mathcal{P})$ and
$V\leftarrow\mathbb{R}^{d_{o}}$.

###### Proof of Proposition 3.8.

Since 2.1, 2.3, and 3.1 hold, we may apply Proposition 3.2. In particular, the
bound (3.4) implies the desired bound, with
$L\leftarrow\sqrt{d_{o}}L(M,T)e^{C_{2}(M,T)}$.

Given the stated hypotheses, we may apply Proposition 3.7. By the definition
(3.10) of $\mathbb{I}_{\theta_{0}}$, we have for every $x\in\mathcal{X}$ that

$\displaystyle\|\mathbb{I}_{\theta_{0}}[h](x)\|_{2}^{2}=$
$\displaystyle\sum_{j\in[d_{o}]}\left|\left\langle\nabla_{p}\log
s_{1}(t_{j},p)|_{p=\theta_{0}(x)},h(x)\right\rangle\right|^{2}$
$\displaystyle\leq$ $\displaystyle\sum_{j\in[d_{o}]}\left\|\nabla_{p}\log
s_{1}(t_{j},p)|_{p=\theta_{0}(x)}\right\|^{2}_{2}\left\|h(x)\right\|^{2}_{2}$
$\displaystyle\leq$
$\displaystyle\sup_{y\in\mathcal{X}}\sum_{j\in[d_{o}]}\left\|\nabla_{p}\log
s_{1}(t_{j},p)|_{p=\theta_{0}(y)}\right\|^{2}_{2}\left\|h(x)\right\|^{2}_{2}.$

Since $\theta_{0}\in\Theta=H=L^{\infty}(\mathcal{X},\mathcal{P})$, the
Heine–Borel theorem implies that
$K\coloneqq\overline{\\{\theta_{0}(x):x\in\mathcal{X}\\}}$ is a compact subset
of $\mathcal{P}$. Thus

$\sup_{y\in\mathcal{X}}\sum_{j\in[d_{o}]}\left\|\nabla_{p}\log
s_{1}(t_{j},p)|_{p=\theta_{0}(y)}\right\|^{2}_{2}\leq\sup_{q\in
K}\sum_{j\in[d_{o}]}\|\nabla_{p}\log s_{1}(t_{j},p)|_{p=q}\|^{2}_{2}.$

By 2.5 and the chain rule, $q\mapsto\nabla_{p}\log s_{1}(t_{j},p)|_{p=q}$ is
continuous. Thus, by the extreme value theorem, the supremum on the right-hand
side of the inequality above is finite,

$\sup_{x\in\mathcal{X}}\|\mathbb{I}_{\theta_{0}}[h](x)\|_{2}\leq\biggr{(}\sup_{q\in
K}\sum_{j\in[d_{o}]}\|\nabla_{p}\log
s_{1}(t_{j},p)|_{p=q}\|^{2}_{2}\biggr{)}^{1/2}\sup_{x\in\mathcal{X}}\|h(x)\|_{2},$

and thus $\mathbb{I}_{\theta_{0}}:(H,\|\cdot\|_{\infty})\to
L^{\infty}(\mathcal{X},\mathbb{R}^{d_{o}})$ is continuous. ∎

Let $\mu\in\mathcal{M}_{1}(\mathcal{X})$. Given
$\mathfrak{J}:\mathcal{X}\to\mathbb{R}^{d_{p}\times d_{o}}$ and
$\mathbb{I}_{\theta_{0}}[h](\cdot)=\mathfrak{J}(\theta_{0}(\cdot))h(\cdot)$
from (3.9) and (3.10), the operator

$\mathbb{I}_{\theta_{0}}^{\ast}:L^{2}_{\mu}(\mathcal{X},\mathbb{R}^{d_{o}})\to\overline{(H,\langle\cdot,\cdot\rangle_{L^{2}_{\mu}(\mathcal{X},\mathcal{P})})}|_{L^{2}_{\mu}},\qquad
g(\cdot)\mapsto\mathbb{I}_{\theta_{0}}^{\ast}[g](\cdot)\coloneqq(\mathfrak{J}(\theta_{0}(\cdot))^{\top}g(\cdot)$
(3.12)

is the Hilbert space adjoint of $\mathbb{I}_{\theta_{0}}$, because

$\displaystyle\left\langle\mathbb{I}_{\theta_{0}}[h],g\right\rangle_{L^{2}_{\mu}(\mathcal{X},\mathbb{R}^{d_{o}})}=$
$\displaystyle\int_{\mathcal{X}}\left\langle\mathfrak{J}(\theta_{0}(x))h(x),g(x)\right\rangle_{\mathbb{R}^{d_{o}}}\mu(\mathrm{d}x)$
$\displaystyle=$ $\displaystyle\int_{\mathcal{X}}\left\langle
h(x),\mathfrak{J}(\theta_{0}(x))^{\top}\right\rangle_{\mathbb{R}^{d_{p}}}\mu(\mathrm{d}x)=\left\langle
h,\mathbb{I}_{\theta_{0}}^{\ast}[g]\right\rangle_{L^{2}_{\mu}(\mathcal{X},\mathcal{P})}.$

Following [13, Definition 3.1.2], we define the information operator
corresponding to $\mathcal{G}$ at $\theta_{0}$ and a measure
$\mu\in\mathcal{M}_{1}(\mathcal{X})$ by

$\mathbb{I}_{\theta_{0}}^{\ast}\mathbb{I}_{\theta_{0}}:(H,\langle\cdot,\cdot\rangle_{L^{2}_{\mu}(\mathcal{X},\mathcal{P})})\to\overline{(H,\langle\cdot,\cdot\rangle_{L^{2}_{\mu}(\mathcal{X},\mathcal{P})})}|_{L^{2}_{\mu}(\mathcal{X},\mathcal{P})},\quad\mathbb{I}_{\theta_{0}}^{\ast}\mathbb{I}_{\theta_{0}}[h](x)\coloneqq\mathfrak{J}^{\top}\mathfrak{J}(\theta_{0}(x))h(x).$
(3.13)

######  Proposition 3.9 (Solvability of information equation).

Suppose the hypotheses of Proposition 3.8 hold. Then for every $\psi\in
L^{\infty}(\mathcal{X},\mathcal{P})$, there exists a unique
$\overline{\psi}_{\theta_{0}}\in L^{\infty}(\mathcal{X},\mathcal{P})$ such
that
$\mathbb{I}_{\theta_{0}}^{\ast}\mathbb{I}_{\theta_{0}}[\overline{\psi}_{\theta_{0}}]=\psi$
on $\mathcal{X}$. In particular, for all $h\in
L^{\infty}(\mathcal{X},\mathcal{P})$,
$\left\langle\mathbb{I}_{\theta_{0}}^{\ast}\mathbb{I}_{\theta_{0}}[\overline{\psi}]-\psi,h\right\rangle_{L^{2}_{\mu}(\mathcal{X},\mathcal{P})}=0$.

The conclusion of Proposition 3.9 implies that [13, Condition 4.1.2] holds.

###### Proof of Proposition 3.9.

The hypotheses of Proposition 3.7 include those of Lemma 3.4. Thus, for every
$x\in\mathcal{X}$, $\mathfrak{J}(\theta_{0}(x))\in\mathbb{R}^{d_{o}\times
d_{p}}$ has full rank, and $\mathfrak{J}^{\top}\mathfrak{J}(\theta_{0}(x))\in
GL(d_{p},\mathbb{R})$. Given $\theta_{0},\psi\in\Theta$, define
$\overline{\psi}_{\theta_{0}}$ by

$\forall
x\in\mathcal{X},\quad\overline{\psi}_{\theta_{0}}(x)\coloneqq\left(\mathfrak{J}^{\top}\mathfrak{J}(\theta_{0}(x))\right)^{-1}\psi(x).$

Then (3.13) implies that
$\mathbb{I}_{\theta_{0}}^{\ast}\mathbb{I}_{\theta_{0}}[\overline{\psi}_{\theta_{0}}]=\psi$
on $\mathcal{X}$. We prove
$\overline{\psi}_{\theta_{0}}\in\Theta=L^{\infty}(\mathcal{X},\mathcal{P})$,
using the same strategy as the proof of Proposition 3.8: since $\theta_{0}\in
L^{\infty}(\mathcal{X},\mathcal{P})$, it follows that
$K\coloneqq\overline{\\{\theta_{0}(x):x\in\mathcal{X}\\}}$ is compact, and

$\sup_{x\in\mathcal{X}}\left\|\bigr{(}\mathfrak{J}^{\top}\mathfrak{J}(\theta_{0}(x))\bigr{)}^{-1}\right\|_{2}\leq\sup_{q\in
K}\left\|\bigr{(}\mathfrak{J}^{\top}\mathfrak{J}(q)\bigr{)}^{-1}\right\|_{2}.$

By 2.5 and (3.9), $q\mapsto\mathfrak{J}^{\top}\mathfrak{J}(q)$ is continuous.
Thus, by the extreme value theorem, the right-hand side of the inequality
above is finite. By the definition of $\overline{\psi}_{\theta_{0}}$, we have
$\|\overline{\psi}_{\theta_{0}}\|_{\infty}\leq\|\mathfrak{J}^{\top}_{\theta_{0}}\mathfrak{J}(\theta_{0})^{-1}\|_{\infty}\|\psi\|_{\infty}$,
and $\psi\in L^{\infty}(\mathcal{X},\mathcal{P})$ implies
$\overline{\psi}_{\theta_{0}}\in L^{\infty}(\mathcal{X},\mathcal{P})$. ∎

##  4 The Bayesian inverse problem and properties of its solution

Recall the definition (3.1) of $\mathcal{G}$, and consider the following
observation model: for $k\in\mathbb{N}$, the $k$-th observation is a random
variable $Y^{(k)}$ given by

$\mathbb{R}^{d_{o}}\ni
Y^{(k)}=\mathcal{G}(\theta_{0})(X^{(k)})+\varepsilon^{(k)},\quad\varepsilon^{(k)}\stackrel{{\scriptstyle\textup{iid}}}{{\sim}}\mathcal{N}\left(0,\varsigma^{2}I\right),\quad
X^{(k)}\stackrel{{\scriptstyle\textup{iid}}}{{\sim}}\mu_{X},$ (4.1)

where $\varsigma>0$ is presumed to be known, and
$\mu_{X}\in\mathcal{M}_{1}(\mathcal{X})$ is not known. In this section, we
consider the following statistical inverse problem: given pairs
$(Y^{(k)},X^{(k)})$, $k\in\mathbb{N}$, infer the true data-generating CPM
$\theta_{0}$. We shall take the Bayesian approach to solving this inverse
problem, by specifying a family of Gaussian priors and analysing the
properties of the associated family of posterior laws.

We emphasise that while the ODE-IVP (2.1) that defines the forward model
$\mathcal{G}$ is linear, the dependence of the solution of (2.1) — in
particular, of the observed component (2.5) of the solution — on the parameter
$p$ is nonlinear. Thus, the forward model $\mathcal{G}$ and the resulting
inverse problem are nonlinear.

Recall that $d_{x}=\operatorname*{dim}(\mathcal{X})$. For $r\geq 0$, let
$H^{r}(\mathcal{X},\mathcal{P})$ denote the Sobolev space of regularity $r$.

######  Assumption 4.1 ($\alpha$-smooth Gaussian base prior).

Let $\Pi^{\prime}$ be a centred Gaussian Borel probability measure on
$\Theta\subseteq L^{2}_{\mu_{X}}(\mathcal{X},\mathcal{P})$ with reproducing
kernel Hilbert space (RKHS) $\mathcal{H}$ that is continuously embedded in
$H^{\alpha}(\mathcal{X},\mathcal{P})$ for some $\alpha>0$, and
$\Pi^{\prime}(\mathcal{R})=1$ for a separable normed linear subspace
$(\mathcal{R},\|\cdot\|_{\mathcal{R}})$ of $\Theta$.

The scalar $\alpha>0$ describes the ‘smoothness’ of the base prior
$\Pi^{\prime}$. 4.1 implies that [13, Condition 2.1.1] holds. The condition
that $\mathcal{H}$ is continuously embedded in $H^{\alpha}$ is from [13,
Theorem 2.2.2], and is due to the fact that our value of $\kappa=0$ is
strictly less than $1/2$; see (3.6).

Define a sequence $(\Pi_{N})_{N\in\mathbb{N}}$ of Gaussian priors by rescaling
the Gaussian base prior $\Pi^{\prime}$ according to the sample size $N$:

$\theta^{(N)}\coloneqq
N^{-d_{x}/(4\alpha+2d_{x})}\theta^{\prime},\quad\theta^{\prime}\sim\Pi^{\prime},\quad\Pi_{N}\coloneqq\mu_{\theta^{(N)}}.$
(4.2)

The definitions (4.2) follow from [13, Equation (2.18)] with the replacement
$\kappa\leftarrow 0$ in (3.6). For $N\in\mathbb{N}$,
$\Pi_{N}(\cdot~{}|(Y^{(k)},X^{(k)})_{k\in[N]})$ denotes the posterior
corresponding to $(Y^{(k)},X^{(k)})_{k\in[N]}$ and the $N$-th prior $\Pi_{N}$.

Next, define a sequence $(\delta_{N})_{N\in\mathbb{N}}$ of positive numbers
and a sequence $(\Theta_{N})_{N\in\mathbb{N}}$ of regularisation sets in
$\mathcal{R}$, following [13, Equations (2.19) and (2.20)]:

$\displaystyle\delta_{N}\coloneqq$ $\displaystyle
N^{-\alpha/(2\alpha+d_{x})},$ (4.3a) $\displaystyle\Theta_{N}\coloneqq$
$\displaystyle\left\\{\theta_{1}+\theta_{2}\in\mathcal{R}\ :\
\|\theta_{1}\|_{L^{2}_{\mu_{X}}}\leq M\delta_{N},\
\|\theta_{2}\|_{\mathcal{H}}\leq M,\
\|\theta_{1}+\theta_{2}\|_{\mathcal{R}}\leq M\right\\}.$ (4.3b)

We now state sufficient conditions for the posteriors
$(\Pi_{N}(\cdot~{}|(Y^{(k)},X^{(k)})_{k\in[N]}))_{N\in\mathbb{N}}$ to contract
to the truth $\theta_{0}$ with rate $(\delta_{N})_{n\in\mathbb{N}}$.

######  Theorem 4.2 (Posterior contraction in $L^{2}_{\mu_{X}}$ norm).

Suppose that $\mathcal{X}$ is a bounded domain of $\mathbb{R}^{d_{x}}$ with
smooth boundary, and that the hypotheses of Proposition 3.3 hold. Suppose 4.1
holds for $\mathcal{R}$ given in (3.2) and some $\alpha>0$. If
$\theta_{0}\in\mathcal{H}\cap\mathcal{R}$, then for all $b>0$, we can choose
$m>0$ large enough such that

$P^{N}_{\theta_{0}}\left(\Pi_{N}(\theta\in\Theta_{N}:\|\mathcal{G}(\theta)-\mathcal{G}(\theta_{0})\|_{L^{2}_{\mu_{X}}}\leq
m\delta_{N}|(Y^{(k)},X^{(k)})_{k\in[N]})\leq
1-e^{bN\delta_{N}^{2}}\right)\xrightarrow[N\to\infty]{}0.$

In addition, for $L^{\prime}$ in (3.8),

$P^{N}_{\theta_{0}}\left(\Pi_{N}(\theta\in\Theta_{N}:\|\theta-\theta_{0}\|_{L^{2}_{\mu_{X}}}\leq
L^{\prime}m\delta_{N}|(Y^{(k)},X^{(k)})_{k\in[N]})\leq
1-e^{bN\delta_{N}^{2}}\right)\xrightarrow[N\to\infty]{}0,$

and the sequence of posterior means
$(E_{\Pi_{N}}[\theta|(Y^{(k)},X^{(k)})_{k\in[N]}])_{N\in\mathbb{N}}$ satisfies

$\left\|E_{\Pi_{N}}[\theta|(Y^{(k)},X^{(k)})_{k\in[N]}]-\theta_{0}\right\|_{L^{2}_{\mu_{X}}}=O_{P^{N}_{\theta_{0}}}(\delta_{N}).$

###### Proof of Theorem 4.2.

The hypotheses of Proposition 3.3 include those of Proposition 3.2. Thus,
$\mathcal{G}$ satisfies [13, Condition 2.1.1] for the choices in (3.6). Since
$\mathcal{X}$ is a bounded domain with smooth boundary, $\kappa\leftarrow 0$
in (3.6), and 4.1 holds, the hypotheses of [13, Theorem 2.2.2] are satisfied.
Since the hypotheses of Proposition 3.3 hold, we conclude that $\mathcal{G}$
satisfies [13, Condition 2.1.4] for the choices in (3.6) and (3.8). The
conclusion then follows from [13, Theorem 2.3.1]. The stochastic bound on the
error of the posterior mean follows from [13, Theorem 2.3.2]. ∎

We write ‘$\xrightarrow[]{d}$’ to denote convergence in distribution under
$P^{\mathbb{N}}_{\theta_{0}}$ of $\mathbb{R}$-valued random variables. We now
state sufficient conditions for the observation model (4.1) to satisfy a
locally asymptotically normal (LAN) approximation.

######  Theorem 4.3 (LAN approximation).

Suppose the hypotheses of Proposition 3.8 hold, so that
$\Theta=H=L^{\infty}(\mathcal{X},\mathcal{P})$. Fix an arbitrary $h\in H$. If
$\theta_{0}\in\Theta$, then the log-likelihood ratio process in the model
(4.1) satisfies the LAN-approximation

$\log\frac{\mathrm{d}P^{N}_{\theta_{0}+h/\sqrt{N}}}{\mathrm{d}P^{N}_{\theta_{0}}}((Y^{(k)},X^{(k)})_{k\in[N]})\xrightarrow[N\to\infty]{d}\mathcal{N}\left(-\tfrac{1}{2}\|\mathbb{I}_{\theta_{0}}[h]\|^{2}_{L^{2}_{\mu_{X}}(\mathcal{X},\mathbb{R}^{d_{o}})},\|\mathbb{I}_{\theta_{0}}[h]\|^{2}_{L^{2}_{\mu_{X}}(\mathcal{X},\mathbb{R}^{d_{o}})}\right).$

###### Proof of Theorem 4.3.

The hypotheses of Proposition 3.8 include those of Proposition 3.7, which
implies that [13, Condition 3.1.1] is satisfied. Next, fix an arbitrary
$\hat{s}>0$. Given the stated hypotheses, we can apply Proposition 3.8 with
$\theta^{(1)}\leftarrow\theta_{0}$,
$\theta^{(2)}\leftarrow\theta_{0}+\hat{s}h$, and
$M\leftarrow\max\\{\|\theta_{0}+sh\|_{\infty}:-\hat{s}\leq s\leq\hat{s}\\}$ to
conclude that there exists $L=L(M)>0$ such that

$\forall
s\in[-\hat{s},\hat{s}],\quad\|\mathcal{G}(\theta_{0}+sh)-\mathcal{G}(\theta_{0})\|_{\infty}\leq
L\|\theta_{0}+sh-\theta_{0}\|_{\infty}=L|s|\|h\|_{\infty}\leq
L\hat{s}\|h\|_{\infty}.$

Thus, we may apply [13, Theorem 3.1.3] to obtain the desired conclusion. ∎

For the following theorem, recall the definition (3.12) of
$\mathbb{I}_{\theta_{0}}^{\ast}$, and fix an arbitrary
$\widehat{\psi}_{\theta_{0}}\in
L^{2}_{\mu_{X}}(\mathcal{X},\mathbb{R}^{d_{o}})$. If
$\Theta=H=L^{\infty}(\mathcal{X},\mathcal{P})$, define

$\Psi:H\to\mathbb{R},\quad
h\mapsto\Psi(h)\coloneqq\left\langle\mathbb{I}_{\theta}[h],\widehat{\psi}_{\theta_{0}}\right\rangle_{L^{2}_{\mu_{X}}(\mathcal{X},\mathcal{P})}.$

######  Theorem 4.4 (Local asymptotic minimax bound for linear functionals).

Suppose that $d_{o}\geq d_{p}=2\mathfrak{d}$ and that the hypotheses of
Theorem 4.3 hold. Then the local asymptotic minimax risk for estimating $\Psi$
at $\theta_{0}$ satisfies

$\liminf_{N\to\infty}~{}\inf_{\overline{\psi}_{N}:(\mathbb{R}^{d_{o}}\times\mathcal{X})^{N}\to\mathbb{R}}~{}\sup_{h\in
H:\|h\|_{L^{2}_{\mu_{X}}}\leq
1/\sqrt{N}}N\mathbb{E}_{P^{N}_{\theta_{0}+h}}[(\overline{\psi}_{N}-\Psi(\theta_{0}+h))^{2}]\geq\|\widehat{\psi}_{\theta_{0}}\|^{2}_{L^{2}_{\mu_{X}}}.$

###### Proof of Theorem 4.4.

In the proof of Theorem 4.3, we showed the hypotheses of Theorem 4.3 include
the hypotheses of [13, Theorem 3.1.3]. The hypotheses of Theorem 4.3 also
include those of Proposition 3.7. Since $d_{o}\geq d_{p}=2\mathfrak{d}$, we
conclude from statement 2 of Proposition 3.7 that
$\mathbb{I}_{\theta_{0}}:H\to L^{2}_{\mu_{X}}$ is injective. Thus, we may
apply [13, Theorem 3.1.4] to obtain the desired conclusion. ∎

In preparation for a Bernstein–von Mises type result, we state sufficient
conditions for posterior contraction with respect to the supremum norm.

######  Proposition 4.5 (Posterior contraction in $L^{\infty}$).

Suppose the hypotheses of Theorem 4.2 hold for a regularisation space
$\mathcal{R}$ that is continuously embedded into $H^{\beta}(\mathcal{X})$ for
$\beta>d_{x}/2$. For fixed $M>0$, define

$\displaystyle\overline{\delta}_{N}(\beta^{\prime})\coloneqq$
$\displaystyle\delta_{N}^{(\beta-\beta^{\prime})/\beta},\quad
d_{x}/2<\beta^{\prime}<\beta,$ (4.4a)
$\displaystyle\Theta_{N,M,\infty}(\beta^{\prime})\coloneqq$
$\displaystyle\\{\theta\in\Theta\ :\ \|\theta\|_{\mathcal{R}}\leq M,\
\|\theta-\theta_{0}\|_{\infty}\leq M\overline{\delta}_{N}(\beta^{\prime})\\}.$
(4.4b)

Then for all $b>0$ we can choose $M$ large enough such that

$P^{N}_{\theta_{0}}\left(\Pi_{N}\left(\theta\in\Theta_{N,M,\infty}(\beta^{\prime})\middle|(Y^{(k)},X^{(k)})_{k\in[N]}\right)\leq
1-e^{-bN\delta_{N}^{2}}\right)\xrightarrow[N\to\infty]{}0.$

###### Proof of Proposition 4.5.

In the proof of Theorem 4.2, we showed that the hypotheses of Theorem 4.2
imply that the hypotheses of [13, Theorem 2.2.2] and [13, Condition 2.1.4] are
satisfied. Therefore, given the hypotheses on $\mathcal{R}$ and $\beta$, the
desired conclusion follows from [13, Proposition 4.1.3]. ∎

The last result of this section is a Bernstein–von Mises theorem for a class
of linear functionals. Recall that Proposition 3.9 gives conditions such that
for every $\psi\in\Theta$, there exists
$\overline{\psi}_{\theta_{0}}\in\Theta$ such that
$\mathbb{I}_{\theta_{0}}^{\ast}\mathbb{I}_{\theta_{0}}[\overline{\psi}_{\theta_{0}}]=\psi$
on $\mathcal{X}$. For every $N\in\mathbb{N}$ and
$(Y^{(k)},X^{(k)})_{k\in[N]}$, define

$\widehat{\Psi}_{N}\coloneqq\left\langle\psi,\theta_{0}\right\rangle_{L^{2}_{\mu_{X}}}+\frac{1}{N}\sum_{k=1}^{N}\left\langle\mathbb{I}_{\theta_{0}}[\overline{\psi}_{\theta_{0}}](X^{(k)}),\varepsilon^{(k)}\right\rangle_{\mathbb{R}^{d_{o}}}.$

######  Theorem 4.6 (Bernstein–von Mises for linear functionals).

Suppose that the hypotheses of Theorem 4.3 and 3.5 hold, and suppose the
hypotheses of Proposition 4.5 hold with $\beta>2d_{x}$ and some

$\frac{d_{x}}{2}<\beta^{\prime}<\frac{\beta(\alpha-d_{x})}{3\alpha}.$ (4.5)

Then for a random variable $\theta$ drawn from the posterior
$\Pi_{N}(\cdot~{}|(Y^{(k)},X^{(k)})_{k\in[N]})$,

$\sqrt{N}\bigr{(}\langle\theta,\psi\rangle_{L^{2}_{\mu_{X}}}-\widehat{\Psi}_{N}\bigr{)}\bigr{|}(Y^{(k)},X^{(k)})_{k\in[N]}\xrightarrow[N\to\infty]{d}\mathcal{N}\left(0,\|\mathbb{I}_{\theta_{0}}[\overline{\psi}_{\theta_{0}}]\|^{2}_{L^{2}_{\mu_{X}}}\right).$

The conditions (4.5) are feasible if and only if
$d_{x}<\tfrac{2\alpha\beta}{3\alpha+2\beta}$. Given $d_{x}\in\mathbb{N}$,
setting $(\alpha,\beta)\leftarrow(4d_{x},3d_{x})$ ensures that this inequality
is satisfied, for example.

###### Proof of Theorem 4.6.

Theorem 4.6 follows from [13, Theorem 4.1.5]. We verify the hypotheses of this
theorem on the forward model $\mathcal{G}$ and on $\psi$.

The hypotheses of Proposition 4.5 include the hypotheses of Theorem 4.2. In
the proof of Theorem 4.2, we showed that the forward model $\mathcal{G}$
satisfies [13, Conditions 2.1.1, 2.1.4]. In the proof of Theorem 4.3, we
showed that [13, Condition 3.1.1] holds with
$H=\Theta=L^{\infty}(\mathcal{X},\mathcal{P})$. Since Theorem 4.3 assumes that
the hypotheses of Proposition 3.8 holds, it follows that [13, Condition 4.1.1]
holds. Thus the hypotheses of [13, Theorem 4.1.5] on $\mathcal{G}$ are
satisfied.

The hypotheses of Theorem 4.3 include the hypotheses of Proposition 3.8, which
coincide with those of Proposition 3.9. Thus, $\psi$ satisfies [13, Condition
4.1.2]. We must show that the unique $\overline{\psi}_{\theta_{0}}$ given in
Proposition 3.9 satisfies [13, Condition 4.1.4]. Recall the hypothesis that
3.5 holds. Since $\theta_{0}\in\Theta=L^{\infty}(\mathcal{X},\mathcal{P})$,
the set $K\coloneqq\overline{\\{\theta_{0}(x):x\in\mathcal{X}\\}}$ satisfies
3.6. Since the hypotheses of Proposition 3.8 include those of Proposition 3.7,
it follows from statement 4 of Proposition 3.7 that the approximation in [13,
Condition 3.1.1] is quadratic, i.e.
$\rho_{\theta_{0}}[h]=O(\|h\|_{\infty}^{2})$. By (4.3a) and (4.4a),

$N\overline{\delta}_{N}^{3}=N^{\tfrac{\beta(2\alpha+d_{x})-3\alpha(\beta-\beta^{\prime})}{\beta(2\alpha+d_{x})}}.$

Now $\beta(2\alpha+d_{x})-3\alpha(\beta-\beta^{\prime})<0$ if and only if
$\beta^{\prime}<\tfrac{\beta(\alpha-d_{x})}{3\alpha}$, so (4.5) implies that
$N\overline{\delta}_{N}^{3}\to 0$ as $N\to\infty$. Given the hypothesis
$\beta>2d_{x}$, it follows from the discussion after equation (4.6) on [13, p.
70] that [13, Condition 4.1.4] holds.

Since all the hypotheses of [13, Theorem 4.1.5] hold, the desired conclusion
follows. ∎

##  5 Application to two-compartment model

In this section, we illustrate the Bayesian approach to inferring covariate-
parameter maps on the two-compartment model (1.1),

$\displaystyle V_{1}\frac{\mathrm{d}}{\mathrm{d}t}s_{1}(t)=$ $\displaystyle
Q(s_{2}(t)-s_{1}(t))-CL\ s_{1}(t),$ $\displaystyle V_{1}s_{1}(0)$
$\displaystyle=D_{0}w,$ $\displaystyle
V_{2}\frac{\mathrm{d}}{\mathrm{d}t}s_{2}(t)=$ $\displaystyle
Q(s_{1}(t)-s_{2}(t)),$ $\displaystyle s_{2}(0)$ $\displaystyle=0.$

where $D_{0}$ is a reference constant and $w$ is the weight of an individual.
The parameters of the two-compartment model above are $(CL,V_{1},Q,V_{2})$. On
[6, p. 569], the ODE-IVP is converted to another ODE-IVP, using a weight-
dependent rescaling procedure described in [14]. We summarise this procedure
below and refer interested readers to [14] for the pharmacological motivation
for this procedure.

Given a reference weight $w_{0}$, define a rescaled weight
$\mathfrak{w}\coloneqq w/w_{0}$, and define the weight-normalised parameters
$(CL^{\ast},V_{1}^{\ast},Q^{\ast},V_{2}^{\ast})$ according to

$CL^{\ast}\coloneqq CL\mathfrak{w}^{-3/4},\quad V_{1}^{\ast}\coloneqq
V_{1}\mathfrak{w}^{-1},\quad Q^{\ast}=Q\mathfrak{w}^{-3/4},\quad
V_{2}^{\ast}=V_{2}\mathfrak{w}^{-1}.$

Since $(CL,V_{1},Q,V_{2})\in\mathbb{R}^{4}_{>0}$ and $w>0$ for every
individual, dividing the first and second row of (1.1) by $V_{1}$ and $V_{2}$
respectively and using the definitions of $CL^{\ast}$, $V_{1}^{\ast}$,
$Q^{\ast}$, and $V_{2}^{\ast}$ yields

$\displaystyle\frac{\mathrm{d}}{\mathrm{d}t}s_{1}(t)=$
$\displaystyle\mathfrak{w}^{-1/4}\left(\frac{Q^{\ast}}{V_{1}^{\ast}}(s_{2}(t)-s_{1}(t))-\frac{CL^{\ast}}{V_{1}^{\ast}}s_{1}(t)\right),$
$\displaystyle s_{1}(0)$ $\displaystyle=\frac{D_{0}w_{0}}{V_{1}^{\ast}},$
(5.1) $\displaystyle\frac{\mathrm{d}}{\mathrm{d}t}s_{2}(t)=$
$\displaystyle\mathfrak{w}^{-1/4}\frac{Q^{\ast}}{V_{2}^{\ast}}(s_{1}(t)-s_{2}(t)),$
$\displaystyle s_{2}(0)$ $\displaystyle=0.$

We now choose the exponential function as a link function and define

$p_{1}\coloneqq\log(\mathfrak{w}^{-1/4}CL^{\ast}),\quad p_{2}\coloneqq\log
V_{1}^{\ast},\quad p_{3}\coloneqq\log(\mathfrak{w}^{-1/4}Q^{\ast}),\quad
p_{4}\coloneqq\log V_{2}^{\ast}.$

Then (5.1) becomes

$\frac{\mathrm{d}}{\mathrm{d}t}\begin{bmatrix}s_{1}(t,p)\\\
s_{2}(t,p)\end{bmatrix}=\begin{bmatrix}-e^{p_{1}-p_{2}}-e^{p_{3}-p_{2}}&e^{p_{3}-p_{2}}\\\
e^{p_{3}-p_{4}}&-e^{p_{3}-p_{4}}\end{bmatrix}\begin{bmatrix}s_{1}(t,p)\\\
s_{2}(t,p)\end{bmatrix},\quad\begin{bmatrix}s_{1}(0,p)\\\
s_{2}(0,p)\end{bmatrix}=\begin{bmatrix}D_{0}w_{0}e^{-p_{2}}\\\
0\end{bmatrix},$ (5.2)

which is of the form (2.1). We define the covariate to consist of the rescaled
weight and the age, i.e. $x=(\mathfrak{w},a)$, and we define $\mathcal{X}$ to
be a bounded subset of $\mathbb{R}^{2}_{>0}$ with smooth boundary, such that
$\inf\\{x_{i}:x\in\mathcal{X}\\}>0$ for $i=1,2$. This assumption ensures that
$\log\mathfrak{w}^{-1/4}$ is bounded from above and from below, and is
reasonable given that medical treatments can be given only within prescribed
age and weight ranges.

######  Theorem 5.1.

The ODE-IVP (5.2) satisfies 2.1, 2.3, 2.5, 3.1, and 3.5 for
$\mathcal{P}\leftarrow\mathbb{R}^{4}$. If $\Theta\coloneqq
L^{\infty}(\mathcal{X},\mathcal{P})$, then every $\theta_{0}\in\Theta$
satisfies 3.6.

###### Proof.

In Lemma B.1, we show that the ODE-IVP (5.2) satisfies 2.1. In (B.4), we
recall the diagonalisation given in the supplemental material of [6]; this
shows that (5.2) satisfies 2.3. In Lemma B.2, we show that 2.5, 3.1, and 3.5
hold. In the proof of Proposition 3.8, we showed that if $\theta_{0}\in
L^{\infty}(\mathcal{X},\mathcal{P})$, then the set
$K\coloneqq\overline{\\{\theta_{0}(x):x\in\mathcal{X}\\}}$ satisfies 3.6. ∎

We end this section with the following result.

######  Corollary 5.2 (Posterior contraction for two-compartment model).

Let $d_{o}\geq 4$ and $\Theta=H=L^{\infty}(\mathcal{X},\mathcal{P})$. Suppose
that $\Pi^{\prime}$ satisfies 4.1 for $\alpha>0$ and $\mathcal{R}$ given in
Proposition 4.5 for $\beta>1$, and suppose
$\theta_{0}\in\mathcal{H}\cap\mathcal{R}$. Then for
$\overline{\delta}_{N}(\beta^{\prime})$ and
$\Theta_{N,M,\infty}(\beta^{\prime})$ given in (4.4a) and (4.4b) respectively,
the conclusion of Proposition 4.5 holds. If in addition $\beta>4$, then the
conclusion of Theorem 4.6 holds.

###### Proof of Corollary 5.2.

By Theorem 5.1 and the hypotheses that $d_{o}\geq 4=d_{p}$, the hypotheses of
Proposition 3.3 and Proposition 3.8 hold. Since $\beta>1=d_{x}/2$, the
hypotheses of Theorem 4.2, Theorem 4.3, and Proposition 4.5 also hold. The
first conclusion then follows from Proposition 4.5. If in addition $\beta>4$,
then the hypotheses of Theorem 4.6 holds, and the second conclusion follows. ∎

##  6 Conclusion

We considered parametrised linear homogeneous ODE-IVPs of the form
$\tfrac{\mathrm{d}}{\mathrm{d}t}s(t,p)=A(p)s(t,p)$ on $[0,T]$ with initial
condition $s(0,p)=s_{0}(p)$, with a given set $\mathcal{P}$ of admissible
parameters. Given observations of only one component of the solution at a
fixed finite collection of observation times $(t_{j})_{j\in[d_{o}]}$, and
given an a priori chosen set $\mathcal{X}$ of admissible covariates, the task
was to infer a covariate-to-parameter map $\theta_{0}$ that maps the covariate
vector $x$ for every individual in a population to the corresponding parameter
vector $p=\theta_{0}(x)$ that defines the ODE-IVP model for that individual.
The constraint that only one component of the solution is observed is
motivated by pharmacological settings, where only drug concentrations in the
patient’s blood can be measured, and it is difficult to measure drug
concentrations in specific tissues or organs.

In this work, we presented a framework for proving posterior contraction,
local asymptotic normality, and Bernstein–von Mises theorems for the Bayesian
inverse problem of inferring the covariate-to-parameter map $\theta_{0}$ using
pairs $(X^{(k)},Y^{(k)})_{k\in\mathbb{N}}$ of covariates and observation
vectors, where each $Y^{(k)}$ is related to $X^{(k)}$ by the fixed finite time
design and random covariate design model

$Y^{(k)}=(\log
s_{1}(t_{j},\theta_{0}(X^{(k)})))_{j\in[d_{o}]}+\varepsilon^{(k)},\quad\varepsilon^{(k)}\stackrel{{\scriptstyle\textup{iid}}}{{\sim}}\mathcal{N}\left(0,\varsigma^{2}I\right),\
X^{(k)}\stackrel{{\scriptstyle\textup{iid}}}{{\sim}}\mu_{X},$

and the true population distribution $\mu_{X}$ of the
$(X^{(k)})_{k\in\mathbb{N}}$ is unknown.

Our framework involves some key assumptions. First, in 2.1, the ODE right-hand
side $A(p)$ and initial condition $s_{0}(p)$ are assumed to be locally bounded
and locally Lipschitz continuous functions of the parameter $p$. We then
further assume in 2.3 that $A(p)$ is diagonalisable in $\mathbb{R}$. The
purpose of 2.3 is to ensure that the observed component of the solution can be
expressed as a sum of exponential functions (2.5). As a sum of exponential
functions, the observed component is now completely determined by the so-
called coefficient map introduced in Definition 2.4 and the intrinsic
dimension $2\mathfrak{d}$. In addition, being a sum of exponentials
facilitates the analysis of the Jacobian of the evaluation map
$p\mapsto(s_{1}(t_{j},p))_{j\in[d_{o}]}$ in terms of the Jacobian of the
coefficient map. If 2.5 holds, i.e. if the Jacobian of the coefficient map
exists and satisfies a full rank condition, then we can exploit the structure
of being a sum of exponentials to show that the evaluation map has a locally
Lipschitz inverse; see Lemma 2.6. Using the strategy of proving estimates that
are uniform in the covariate, our framework avoids imposing any constraints on
the unknown covariate law $\mu_{X}$.

Our framework is designed to be applied together with the framework of Nickl
for nonlinear Bayesian inverse problems [13]. In our framework, Lemma 2.6 is
the most important result, because it leads directly to the stability estimate
that is essential in [13] for obtaining posterior contraction rates for the
unknown covariate-to-parameter map $\theta_{0}$ from contraction rates for
$\mathcal{G}(\theta_{0})$. In addition, Lemma 2.6 shows that in the fixed
finite time design setting, the lower bound $d_{o}\geq 2\mathfrak{d}$ on the
size $d_{o}$ of the time design $(t_{j})_{j\in[d_{o}]}$ is a sufficient
condition for stability, and thereby for posterior contraction for
$\theta_{0}$. The condition $d_{o}\geq 2\mathfrak{d}$ is also important for
showing that the derivative of the forward operator is injective and for
showing that the information equation is solvable; see statement 2 of
Proposition 3.7 and Proposition 3.9 respectively. In the context of Bayesian
inference, the key results of this work are the results in Section 4, e.g. the
posterior contraction and Bernstein–von Mises theorems Theorem 4.2 and Theorem
4.6.

For future work, it would be interesting to investigate whether one can reduce
further the number $d_{o}$ of observations taken for each covariate drawn from
the population. This would be relevant for applications in pharmacology, where
there are some groups of individuals for which clinicians may only be able to
draw one blood sample per individual. One may also investigate whether the
diagonalisability property in 2.3 can be weakened, and to consider more
general families of ODEs.

## Acknowledgements

The research of HCL has been partially funded by the Deutsche
Forschungsgemeinschaft (DFG) — Project-ID 318763901 — SFB1294. The author
would like to thank Giuseppe Carere, Niklas Hartung, and Wilhelm Huisinga for
helpful discussions.

##  Appendix A Auxiliary results

We use the following local Lipschitz continuity bounds:

$\displaystyle\forall t,s>0,\quad|\log t-\log s|\leq$
$\displaystyle\frac{|t-s|}{s\wedge t},$ (A.1a) $\displaystyle\forall
t,s\in\mathbb{R},\quad|\exp t-\exp s|\leq$ $\displaystyle\exp(s\vee t)|t-s|.$
(A.1b)

We state and prove a result that is inspired but not implied by the main
result of [16]. We use this result in Lemma 2.6.

######  Lemma A.1.

Let $n\in\mathbb{N}$ and $(\beta_{k})_{k\in[n]}$, $(\gamma_{k})_{k\in[n]}$,
$(\xi_{k})_{k\in[n]}$ be such that
$(\gamma_{k},\xi_{k})_{k\in[n]}\in\mathbb{R}^{2n}\setminus\\{0\\}$ and the
$(\beta_{k})_{k\in[n]}\in\mathbb{R}^{n}$ are distinct. Then the function

$\mathbb{R}\ni t\mapsto\sum_{k\in[n]}e^{\beta_{k}t}(\xi_{k}+\gamma_{k}t)$
(A.2)

has at most $2n-1$ roots.

###### Proof of Lemma A.1.

We prove the claim by induction. Let $n=1$. Consider the function
$\mathbb{R}\ni t\mapsto e^{\beta t}(\xi+\gamma t)$. If $\gamma=0$, then
$\xi\neq 0$, and the function has no roots. If $\gamma\neq 0$, then the
function has exactly one root at $t=-\xi/\gamma$. This proves the statement in
the base case.

Now suppose that the claim is true for some $n\leq 1$. Since

$h(t)\coloneqq\sum_{k\in[n+1]}e^{\beta_{k}t}(\xi_{k}+\gamma_{k}t)=e^{\beta_{1}t}\left[(\xi_{1}+\gamma_{1}t)+\sum_{k=2}^{n+1}e^{(\beta_{k}-\beta_{1})t}(\xi_{k}+\gamma_{k}t)\right]\eqqcolon
e^{\beta_{1}t}\widehat{h}(t),$

it follows that $h(t)=0$ if and only if $\widehat{h}(t)=0$. Note that for
arbitrary $\beta,\gamma,\xi\in\mathbb{R}$,

$\frac{\mathrm{d}^{2}}{\mathrm{d}t^{2}}e^{\beta t}(\xi+\gamma
t)=\frac{\mathrm{d}}{\mathrm{d}t}\left[e^{\beta t}\left(\beta(\xi+\gamma
t)+\gamma\right)\right]=e^{\beta t}\left[\beta^{2}\xi+2\beta\gamma+\beta\gamma
t\right].$

Hence,

$\displaystyle\frac{\mathrm{d}^{2}}{\mathrm{d}t^{2}}\widehat{h}(t)=\frac{\mathrm{d}^{2}}{\mathrm{d}t^{2}}\sum_{k=2}^{n+1}e^{(\beta_{k}-\beta_{1})t}(\xi_{k}+\gamma_{k}t)$
$\displaystyle=$
$\displaystyle\sum_{k=2}^{n+1}e^{(\beta_{k}-\beta_{1})t}\left[(\beta_{k}-\beta_{1})^{2}\xi_{k}+2(\beta_{k}-\beta_{1})\gamma_{k}+(\beta_{k}-\beta_{1})\gamma_{k}t\right].$

By the hypotheses, the $(\beta_{k}-\beta_{1})_{k=2}^{n+1}$ are distinct, and
$((\beta_{k}-\beta_{1})\gamma_{k},(\beta_{k}-\beta_{1})^{2}\xi_{k}+2(\beta_{k}-\beta_{1})\gamma_{k})_{k=2}^{n+1}\in\mathbb{R}^{2n}\setminus\\{0\\}$.
Thus we may apply the base case to conclude that
$\tfrac{\mathrm{d}^{2}}{\mathrm{d}t^{2}}\widehat{h}(t)$ has at most $2n-1$
roots. By the mean value theorem,
$\tfrac{\mathrm{d}}{\mathrm{d}t}\widehat{h}(t)$ has at most $2n$ roots, and
thus $\widehat{h}(t)$ has at most $2n+1$ roots. Since
$h(t)=e^{\beta_{1}t}\widehat{h}(t)$, it follows that $h$ has at most
$2n+1=2(n+1)-1$ roots. This proves the inductive step and completes the proof
of Equation A.2. ∎

##  Appendix B Properties of the two-compartment model

Let $\kappa\coloneqq D_{0}w_{0}>0$. Then (5.2) is of the form (2.1), for $A$
and $s_{0}$ given in (B.1a) below:

$\displaystyle A(p)\coloneqq$
$\displaystyle\begin{bmatrix}-e^{p_{3}-p_{2}}-e^{p_{1}-p_{2}}&e^{p_{3}-p_{2}}\\\
e^{p_{3}-p_{4}}&-e^{p_{3}-p_{4}}\end{bmatrix},$ $\displaystyle
s_{0}(p)\coloneqq\begin{bmatrix}\kappa e^{-p_{2}}\\\ 0\end{bmatrix},$ (B.1a)
$\displaystyle\sigma(p)\coloneqq$ $\displaystyle
e^{p_{3}-p_{2}}+e^{p_{1}-p_{2}}+e^{p_{3}-p_{4}},$
$\displaystyle\delta(p)\coloneqq\sqrt{\sigma(p)^{2}-4e^{p_{1}-p_{2}}e^{p_{3}-p_{4}}},$
(B.1b) $\displaystyle\lambda^{\pm}(p)\coloneqq$
$\displaystyle\frac{\pm\delta(p)-\sigma(p)}{2},$ $\displaystyle
v^{\pm}(p)\coloneqq\lambda^{\pm}(p)e^{p_{4}-p_{3}}+1,$ (B.1c) $\displaystyle
v^{+}(p)-v^{-}(p)=$ $\displaystyle\delta(p)e^{p_{4}-p_{3}},$
$\displaystyle\eta(p)\coloneqq\frac{\kappa}{e^{p_{2}}(v^{+}(p)-v^{-}(p))}.$
(B.1d)

Since $A(p)\in\mathbb{R}^{2\times 2}$, we have $d_{s}=2$.

We now show that

$\delta(p)>0,\quad\lambda^{-}(p)<\lambda^{+}(p)<0,\quad v^{-}(p)<0<v^{+}(p).$
(B.2)

For $\alpha,\beta,\gamma\in\mathbb{R}$,

$(\beta+\gamma)^{2}-4\beta\gamma=\beta^{2}-2\beta\gamma+\gamma^{2}\geq 0$

and

$\displaystyle(\alpha+\beta+\gamma)^{2}-4\beta\gamma-(\alpha+\beta-\gamma)^{2}$
$\displaystyle=$
$\displaystyle(\alpha+\beta+\gamma+\alpha+\beta-\gamma)(\alpha+\beta+\gamma-(\alpha+\beta-\gamma))-4\beta\gamma$
$\displaystyle=$ $\displaystyle
4(\alpha+\beta)\gamma-4\beta\gamma=4\alpha\gamma.$

Hence, if $\alpha,\beta,\gamma>0$,

$(\alpha+\beta+\gamma)^{2}>(\beta+\gamma)^{2}\geq
4\beta\gamma,\quad\sqrt{(\alpha+\beta+\gamma)^{2}-4\beta\gamma}>|\alpha+\beta-\gamma|.$

Set $\alpha\leftarrow e^{p_{3}-p_{3}}$, $\beta\leftarrow e^{p_{1}-p_{2}}$, and
$\gamma\leftarrow e^{p_{3}-p_{4}}$. Then by (B.1b),
$\sigma(p)=\alpha+\beta+\gamma$ and
$\delta(p)=\sqrt{(\alpha+\beta+\gamma)^{2}-4\beta\gamma}$. Thus, the
inequality $(\alpha+\beta+\gamma)^{2}>4\beta\gamma$ is equivalent to
$\delta>0$, while $\delta<\sigma$ holds by definition of $\delta(p)$. By the
definitions (B.1c) of $\lambda^{\pm}(p)$ and $v^{\pm}(p)$, it follows that
$\lambda^{-}(p)<\lambda^{+}(p)<0$. Since

$v^{\pm}(p)=\frac{\pm\sqrt{(\alpha+\beta+\gamma)^{2}-4\beta\gamma}-(\alpha+\beta+\gamma)+2\gamma}{2\gamma}=\frac{\pm\sqrt{(\alpha+\beta+\gamma)^{2}-4\beta\gamma}-(\alpha+\beta-\gamma)}{2\gamma}$

and $\sqrt{(\alpha+\beta+\gamma)^{2}-4\beta\gamma}>|\alpha+\beta-\gamma|$, it
follows that $v^{+}(p)>0>v^{-}(p)$.

We now use the eigenvalue inequalities in (B.2) to prove local boundedness and
local Lipschitz continuity of solutions to (2.1) for $A(p)$ and $s_{0}(p)$
given in (B.1a).

######  Lemma B.1.

Let $A(p)$ and $s_{0}(p)$ be as in (B.1a). For every $M>0$,

$\displaystyle\sup\\{\|A(p)\|_{2}\vee\|s_{0}(p)\|_{2}:p\in B_{2}(0,M)\\}\leq$
$\displaystyle(3\vee\kappa)e^{2M},$ (B.3a)
$\displaystyle\sup\left\\{\|A(p)-A(q)\|_{2}\vee\|s_{0}(p)-s_{0}(q)\|_{2}:p,q\in
B_{2}(0,M)\right\\}\lesssim$ $\displaystyle(6\vee\kappa)e^{2M}\|p-q\|_{2}.$
(B.3b)

In particular, 2.1 holds, with $C_{1}(M)\lesssim(6\vee\kappa)e^{2M}$.

###### Proof of Lemma B.1.

Since $\|A(p)\|_{2}$ denotes the spectral norm of $A(p)$, we have

$\|A(p)\|=|\lambda^{-}(p)|=\tfrac{1}{2}|\delta(p)+\sigma(p)|\leq\tfrac{1}{2}|\sqrt{\sigma(p)^{2}}+\sigma(p)|=\sigma(p)\leq
3e^{2M},$

where the first equation follows from the eigenvalue inequalities in (B.2),
the second equation follows from the definition of $\lambda^{+}(p)$ in (B.1c),
the first inequality follows from the definition of $\delta(p)$ in (B.1b), and
the last inequality follows from the definition of $\sigma(p)$ in (B.1b)
combined with the fact that $p\in B_{2}(0,M)$ implies that $\|p\|_{\infty}<M$.
By definition of $s_{0}(p)$ in (B.1a), $\|s_{0}(p)\|_{2}\leq\kappa e^{M}$, for
$p\in B_{2}(0,M)$. This proves (B.3a).

Since $A\in\mathbb{R}^{d_{s}\times d_{s}}$, we have
$\|A(p)\|_{2}\lesssim\|A(p)\|_{\infty}$, where the hidden constant depends
only on $d_{s}$. Recalling that $\|A(p)\|_{\infty}$ is the maximum absolute
row sum norm of $A(p)$ and using the definition (B.1a) of $A(p)$, we obtain

$\displaystyle\|A(p)-A(q)\|_{\infty}\leq\max\left\\{2\left|e^{p_{3}-p_{2}}-e^{q_{3}-q_{2}}\right|+\left|e^{p_{1}-p_{2}}-e^{q_{1}-q_{2}}\right|,2\left|e^{p_{3}-p_{4}}-e^{q_{3}-q_{4}}\right|\right\\}.$

By the local Lipschitz continuity (A.1b) of the exponential function, the
triangle inequality, and the fact that $p,q\in B_{2}(0,M)$, we have for every
$i,j\in[d_{p}]$ with $i\neq j$ that

$\left|e^{q_{i}-q_{j}}-e^{p_{i}-p_{j}}\right|\leq
e^{\max\left\\{q_{i}-q_{j},p_{i}-p_{j}\right\\}}\left|(q_{i}-q_{j})-(p_{i}-p_{j})\right|\leq
2e^{2M}\|q-p\|_{\infty}.$

Combining the preceding inequalities yields

$\|A(p)-A(q)\|_{\infty}\leq 6e^{2M}\|p-q\|_{\infty}\leq 6e^{2M}\|p-q\|_{2}.$

By (B.1a), (A.1b), and $p,q\in B_{2}(0,M)$,

$\|s_{0}(p)-s_{0}(q)\|_{2}=\kappa|e^{-p_{2}}-e^{-q_{2}}|\leq\kappa
e^{\max\\{-p_{2},-q_{2}\\}}|p_{2}-q_{2}|\leq\kappa e^{M}\|p-q\|_{2}.$

This proves (B.3b). ∎

One can show by direct calculations — see e.g. the supplemental material of
[6] — that

$A(p)V(p)=V(p)\Lambda(p),\quad
V(p)\coloneqq\begin{bmatrix}v^{+}(p)&v^{-}(p)\\\
1&1\end{bmatrix},\quad\Lambda(p)\coloneqq\begin{bmatrix}\lambda^{+}(p)&0\\\
0&\lambda^{-}(p)\end{bmatrix}.$ (B.4)

Thus, the matrix $A(p)$ in (B.1a) satisfies 2.3. Since
$s(t,p)=e^{A(p)t}s_{0}(p)$ and $AV=V\Lambda$, one can show that the solution
to (2.1) with $A(p)$ and $s_{0}(p)$ defined in (B.1a) satisfies

$\begin{bmatrix}s_{1}(t,p)\\\
s_{2}(t,p)\end{bmatrix}=\eta(p)\begin{bmatrix}v^{+}(p)e^{\lambda^{+}(p)t}-v^{-}(p)e^{\lambda^{-}(p)t}\\\
e^{\lambda^{+}(p)t}-e^{\lambda^{-}(p)t}.\end{bmatrix}$

In particular, (2.5) is valid:

$s_{1}(t,p)=\eta v^{+}(p)e^{\lambda^{+}(p)t}-\eta
v^{-}(p)e^{\lambda^{-}(p)t},\quad\forall t\geq 0.$ (B.5)

######  Lemma B.2.

The coefficient map $p\mapsto(\eta v^{+},-\eta
v^{-},\lambda^{+},\lambda^{-})(p)\in\mathbb{R}^{2}_{>0}\times\mathbb{R}^{2}_{<0}$
of (B.5) is $C^{2}$ smooth. In addition,

$\forall
q\in\mathcal{P},\quad\mathcal{J}(q)\coloneqq\begin{bmatrix}\nabla_{p}\eta
v^{+}(p)|\nabla_{p}\eta
v^{-}(p)|\nabla_{p}\lambda^{+}(p)|\nabla_{p}\lambda^{-}(p)\end{bmatrix}^{\top}|_{p=q}\in
GL(4,\mathbb{R}).$ (B.6)

Thus, 2.5, 3.1, and 3.5 hold.

###### Proof of Lemma B.2.

It follows from (B.1c) and (B.1d) that $\eta$, $v^{\pm}$, and $\lambda^{\pm}$
are $C^{\infty}$ functions of $p$. Thus, the coefficient map $p\mapsto(\eta
v^{+},-\eta v^{-},\lambda^{+},\lambda^{-})(p)$ for $s_{1}$ in (B.5) is
$C^{2}$, which shows that 3.5 holds. By (B.2), $p\mapsto(\eta v^{+},\eta
v^{-},\lambda^{+},\lambda^{-})\in\mathbb{R}_{>0}^{2}\times\mathbb{R}^{2}_{<0}$.
By the eigenvalue inequalities $\lambda^{+}>\lambda^{-}$ in (B.2), the sum of
exponentials (B.5), and the definition of $\eta$ in (B.1d), we have

$s_{1}(t,p)\geq\left(\eta v^{+}(p)-\eta
v^{-}(p)\right)e^{\lambda^{-}(p)t}=\kappa e^{-p_{2}+\lambda^{-}(p)t}>0,$

so 3.1 holds.

It remains to show that 2.5 holds. For matrices $M$ and $N$ of the same size,
let $M\sim N$ denote the property that $M$ and $N$ are column equivalent, i.e.
one can change $M$ to $N$ and vice versa by elementary column operations, and
let $M\propto N$ denote the property that there exists some scalar
$\lambda\in\mathbb{R}$ such that $M=\lambda N$. Since
$\lambda^{+}-\lambda^{-}=\delta$ and $\lambda^{+}+\lambda^{-}=-\sigma$ by
(B.1c), the matrix $\mathcal{J}(q)$ in (B.6) satisfies

$\mathcal{J}(q)\sim\begin{bmatrix}\nabla_{p}\eta v^{+}(p)|\nabla_{p}\eta
v^{-}(p)|\nabla_{p}\sigma(p)|\nabla_{p}\delta(p)\end{bmatrix}^{\top}|_{p=q}.$
(B.7)

Now note that

$\displaystyle\nabla_{p}\sigma(p)=$
$\displaystyle(e^{p_{1}-p_{2}},-e^{p_{3}-p_{2}}-e^{p_{1}-p_{2}},e^{p_{3}-p_{2}}+e^{p_{3}-p_{4}},-e^{p_{3}-p_{4}})$
$\displaystyle\nabla_{p}\delta(p)=$
$\displaystyle\sigma\delta^{-1}\nabla_{p}\sigma(p)+2\delta^{-1}(p)e^{p_{1}-p_{2}+p_{3}-p_{4}}\underbrace{(-1,+1,-1,+1)}_{\eqqcolon
u},$

and
$\nabla_{p}\delta,\nabla_{p}\lambda^{\pm}\in\textup{span}\\{\nabla_{p}\sigma,u\\}$.
Since the second component of

$-e^{-p_{1}+p_{2}}\nabla_{p}\sigma(p)=(-1,e^{p_{3}-p_{2}-p_{1}+p_{2}}+1,e^{p_{3}-p_{2}-p_{1}+p_{2}}-e^{p_{3}-p_{4}-p_{1}+p_{2}},e^{p_{3}-p_{4}-p_{1}+p_{2}})$

is strictly larger than $+1$, which is the second component of
$u=(-1,+1,-1,+1)$, it follows that $\nabla_{p}\sigma(p)$ and $u$ are linearly
independent. Thus by (B.7),

$\mathcal{J}(q)\sim\begin{bmatrix}\nabla_{p}\eta v^{+}(p)|\nabla_{p}\eta
v^{-}(p)|\nabla_{p}\sigma(p)|~{}u~{}\end{bmatrix}^{\top}|_{p=q}.$

By (B.1d),

$\eta
v^{\pm}(p)=\frac{\kappa}{e^{p_{2}+p_{4}-p_{3}}\delta(p)}\left(\lambda^{\pm}e^{p_{4}-p_{3}}+1\right)=\frac{\kappa}{\delta}\left(\lambda^{\pm}e^{-p_{2}}+e^{-p_{2}-p_{4}+p_{3}}\right).$

By the equation above and the product rule,

$\nabla_{p}(\eta
v^{\pm})=\frac{\kappa}{\delta}\left(\frac{e^{-p_{2}}\lambda^{\pm}+e^{-p_{2}+p_{3}-p_{4}}}{\delta}(-\nabla_{p}\delta)+\lambda^{\pm}\nabla_{p}e^{-p_{2}}+e^{p_{2}}\nabla\lambda^{\pm}+\nabla_{p}e^{-p_{2}+p_{3}-p_{4}}\right).$

Note that $\nabla_{p}e^{-p_{2}}\propto(0,-1,0,0)\eqqcolon v$ and
$\nabla_{p}e^{-p_{2}+p_{3}-p_{4}}\propto(0,-1,+1,-1)\eqqcolon w$. Recalling
that
$\nabla_{p}\delta,\nabla_{p}\lambda^{\pm}\in\textup{span}\\{\nabla_{p}\sigma,u\\}$
implies that $\nabla_{p}\eta v^{\pm}$ is a linear combination of
$\nabla_{p}\sigma$, $u$, $v$, and $w$, which implies
$\mathcal{J}(q)\sim[\nabla_{p}\sigma|~{}u~{}|~{}v~{}|~{}w~{}]^{\top}|_{p=q}$.
Let $(e_{i})_{i\in[4]}$ denote the canonical orthonormal basis of
$\mathbb{R}^{4}$. Since

$\begin{bmatrix}~{}w~{}|~{}v~{}|~{}u~{}\end{bmatrix}=\begin{bmatrix}0&0&-1\\\
-1&-1&+1\\\ +1&0&-1\\\ -1&0&+1\end{bmatrix}\sim\begin{bmatrix}0&0&+1\\\
0&+1&0\\\ +1&0&0\\\
-1&0&0\end{bmatrix}=\begin{bmatrix}~{}e_{3}-e_{4}~{}|~{}e_{2}~{}|~{}e_{1}~{}\end{bmatrix},$

it follows that $\mathcal{J}(q)$ has full rank if and only if
$\nabla_{p}\sigma(p)\notin\textup{span}\\{e_{3}-e_{4},e_{2},e_{1}\\}$. Now

$\nabla_{p}\sigma(p)=\begin{bmatrix}e^{p_{1}-p_{2}}\\\
-e^{p_{3}-p_{2}}-e^{p_{1}-p_{2}}\\\ e^{p_{3}-p_{2}}+e^{p_{3}-p_{4}}\\\
-e^{p_{3}-p_{4}}\end{bmatrix}\in\textup{span}\left\\{\begin{bmatrix}0\\\ 0\\\
+1\\\ -1\end{bmatrix},\begin{bmatrix}0\\\ +1\\\ 0\\\
0\end{bmatrix},\begin{bmatrix}+1\\\ 0\\\ 0\\\
0\end{bmatrix}\right\\}\Longleftrightarrow
e^{p_{3}-p_{2}}+e^{p_{3}-p_{4}}=e^{p_{3}-p_{4}},$

where the equivalence follows by comparing the third and fourth components of
$\nabla_{p}\sigma(p)$, and the third and fourth components of any vector in
$\textup{span}\\{e_{3}-e_{4},e_{2},e_{1}\\}$. Since $e^{p_{3}-p_{2}}>0$ for
every $p_{3},p_{2}\in\mathbb{R}$, it follows that
$\nabla_{p}\sigma(p)\notin\textup{span}\\{e_{3}-e_{4},e_{2},e_{1}\\}$, and
thus $\mathcal{J}(q)$ in (B.6) has full rank. Thus, 2.5 holds. ∎

## References

  * [1] Abhishake, Tapio Helin, and Nicole Mücke, _Statistical inverse learning problems with random observations_ , 2023, arXiv:2312.15341.
  * [2] Prithwish Bhaumik and Subhashis Ghosal, _Bayesian two-step estimation in differential equation models_ , Electron. J. Stat. 9 (2015), no. 2, 3124 – 3154.
  * [3] , _Efficient Bayesian estimation and uncertainty quantification in ordinary differential equation models_ , Bernoulli 23 (2017), no. 4B, 3537 – 3570.
  * [4] Prithwish Bhaumik, Wenli Shi, and Subhashis Ghosal, _Two-step Bayesian methods for generalized regression driven by partial differential equations_ , Bernoulli 28 (2022), no. 3, 1625 – 1647.
  * [5] Oksana A. Chkrebtii, David A. Campbell, Ben Calderhead, and Mark A. Girolami, _Bayesian Solution Uncertainty Quantification for Differential Equations_ , Bayes. Anal. 11 (2016), no. 4, 1239 – 1267.
  * [6] Niklas Hartung, Martin Wahl, Abhishake Rastogi, and Wilhelm Huisinga, _Nonparametric goodness-of-fit testing for parametric covariate models in pharmacometric analyses_ , CPT Pharmacometrics Syst. Pharmacol. 10 (2021), no. 6, 564–576.
  * [7] Markus Joerger, _Covariate pharmacokinetic model building in oncology and its potential clinical relevance_ , AAPS J. 14 (2012), no. 1, 119–132.
  * [8] E. N. Jonsson and M. O. Karlsson, _Automated covariate model building within NONMEM_ , Pharm. Res. 15 (1998), 1463–1468.
  * [9] Tze Leung Lai, Mei-Chiung Shih, and Samuel P. Wong, _A new approach to modeling covariate effects and individualization in population pharmacokinetics-pharmacodynamics_ , J. Pharmacokinet. Pharmacodyn. 33 (2006), no. 1, 49–74.
  * [10] P. O. Maitre, M. Bührer, D. Thomson, and D. R. Stanski, _A three-step approach combining Bayesian regression and NONMEM population analysis: Application to midazolam_ , J. Pharmacokinet. Biop. 19 (1991), no. 4, 377 – 384.
  * [11] Jaap W. Mandema, Davide Verotta, and Lewis B. Sheiner, _Building population pharmacokinetic pharmacodynamic models. I. Models for covariate effects_ , J. Pharmacokinet. Biop. 20 (1992), no. 5, 511–528.
  * [12] Florence Mesnil, France Mentré, Catherine Dubruc, Jean-Paul Thẽnot, and Alain Mallet, _Population pharmacokinetic analysis of mizolastine and validation from sparse data on patients using the nonparametric maximum likelihood method_ , J. Pharmacokinet. Biop. 26 (1998), no. 2, 133–161.
  * [13] Richard Nickl, _Bayesian non-linear statistical inverse problems_ , Zurich Lectures in Advanced Mathematics, European Mathematical Society Press, Berlin, Germany, 2023.
  * [14] Gabriel J. Robbie, Liang Zhao, John Mondick, Genevieve Losonsky, and Lorin K. Roskos, _Population pharmacokinetics of palivizumab, a humanized anti-respiratory syncytial virus monoclonal antibody, in adults and children_ , Antimicrob. Agents Ch. 56 (2012), no. 10, 5431–5431.
  * [15] Qianwen Tan and Subhashis Ghosal, _Bayesian analysis of mixed-effect regression models driven by ordinary differential equations_ , Sankhya B 83 (2021), no. 1, 3–29.
  * [16] Timo Tossavainen, _The Lost Cousin of the Fundamental Theorem of Algebra_ , Math. Mag. 80 (2007), no. 4, 290–294.
  * [17] Jon Wakefield, Leon Aarons, and Amy Racine-Poon, _The Bayesian approach to population pharmacokinetic/pharmacodynamic modeling_ , Case Studies in Bayesian Statistics (New York, NY) (Constantine Gatsonis, Robert E. Kass, Bradley Carlin, Alicia Carriquiry, Andrew Gelman, Isabella Verdinelli, and Mike West, eds.), Springer New York, 1999, pp. 205–265.