Table 5.2.2: Relative errors, $\ell$ and speed-up in approximating the MPC-FE
solution.

As one can notice, the risk is that $\ell$ increases at each update, so that
the MPC-POD method becomes slower. Furthermore, as Algorithm 5.2.4 proceeds,
$\ell$ can potentially become greater than the number of degrees of freedom of
the full-order model. To avoid this, a more elaborated basis update strategy
can be developed [Mec19].

The computed feedback controls are shown in Figure 5.2.3 together with the
MPC-POD-NU and MPC-POD error approximation of the MPC-FE control. Clearly, the
MPC-POD-NU method can not reconstruct the full-order feedback control without
updating the POD basis, due to the fact that the involved dynamics change
according to the time-dependent parameters. A change of these parameters after
the training horizon used for the POD method will cause an important lack in
approximation. This lack in approximation can even be present since the begin,
making necessary an immediate correction, as in this case ($\ell=21$). The
subsequent basis updates happening respectively at time steps
$n=\left\\{4,17,42\right\\}$ ($\ell=\left\\{33,48,64\right\\}$) show anyway
that the update of the POD model is important also during the MPC algorithm.
$\blacklozenge$

(a) MPC feedback controls

(b) $\delta:=\|u^{\text{MPC-FE}}-u\|_{\mathscr{U}}$

Figure 5.2.3: Feedback control and POD approximation error up to $t_{199}=5$.

###  5.3 Multiobjective PDE-constrained optimization

In real applications, optimization problems are often described by introducing
several objective functions conflicting with each other. This leads to
multiobjective or multicriterial optimization problems; cf. [Ehr05, Mie98,
Sta88]. One prominent example is given by an energy efficient heating,
ventilation and air-conditioning (HVAC) operation of a building with
conflicting objectives such as minimal energy consumption and maximal comfort;
cf. [FHC06, KTX11]. Finding the optimal control that represents a good
compromise is the main issue in these problems. For that reason the concept of
Pareto optimal or efficient points is developed. In contrast to scalar-valued
optimization problems, the computation of a set of Pareto optimal points is
required. Consequently, many scalar-valued constrained optimization problems
have to be solved.

In this section we follow [Ban17, BBV16, BBV17] and apply the reference point
method [Wie80, Wie86, RBW+09] in order to transform a bicriterial optimal
control problem into a sequence of scalar-valued optimal control problems and
solve them using well-known optimal control techniques; see [Trö10]. Let us
mention that preliminary results combining reduced-order modeling and
multiobjective PDE-constrained optimization have recently been derived; cf.
[ITV16, IUV17, POBD16].

In this section we will utilize the following notation: If
$x^{1},x^{2}\in\mathbb{R}^{2}$ are two vectors, we write $x^{1}\leq x^{2}$ if
$x^{1}_{i}\leq x^{2}_{i}$ for $i=1,2$, and analogously for $x^{1}\geq x^{2}$.
For $x\in\mathbb{R}^{2}$ we set $\mathbb{R}^{2}_{\geq
x}=\\{y\in\mathbb{R}^{2}\,|\,y\geq x\\}$ and $\mathbb{R}^{2}_{\leq
x}=\\{y\in\mathbb{R}^{2}\,|\,y\leq x\\}$. For brevity, we define
$\mathbb{R}^{2}_{\leq}:=\mathbb{R}^{2}_{\leq 0}$ and
$\mathbb{R}^{2}_{\geq}:=\mathbb{R}^{2}_{\geq 0}$.

####  5.3.1 Problem formulation

The Hilbert space ${\mathscr{U}}$ is either $\mathbb{R}^{\mathsf{m}}$ or
$L^{2}(\mathscr{D};\mathbb{R}^{\mathsf{m}})$ with an open subset
$\mathscr{D}\subset\mathbb{R}^{\mathsf{m}_{\circ}}$. Let Assumption 4.1.8
hold. Suppose that ${\mathscr{H}}$ is a real Hilbert space and
$\mathcal{C}:{\mathscr{Y}}\to{\mathscr{H}}$ a linear, bounded operator. Then
we consider the following bicriterial optimal control problem:

$\min J(y,u)=\left(\begin{array}[]{c}J_{1}(y,u)\\\
J_{2}(y,u)\end{array}\right)=\frac{1}{2}\left(\begin{array}[]{c}{\|\mathcal{C}y-{y^{\mathsf{d}}}\|}_{\mathscr{H}}^{2}\\\\[2.84526pt]
{\|u-{u^{\mathsf{n}}}\|}_{\mathscr{U}}^{2}\end{array}\right),$ (5.3.50e)
subject to $(y,u)\in{\mathscr{X}}={\mathscr{Y}}\times{\mathscr{U}}$ solves
$\displaystyle\frac{\mathrm{d}}{\mathrm{d}t}{\langle
y(t),\varphi\rangle}_{H}+a(t;y(t),\varphi)$
$\displaystyle={\langle(f+\mathcal{B}u)(t),\varphi\rangle}_{V^{\prime},V}$
$\displaystyle\text{for all }\varphi\in V\text{ a.e. in
}({t_{\circ}},{t_{\mathsf{e}}}],$ (5.3.50f) $\displaystyle{\langle
y({t_{\circ}}),\varphi\rangle}_{H}$ $\displaystyle={\langle
y_{\circ},\varphi\rangle}_{H}$ $\displaystyle\text{for all }\varphi\in H,$ and
$u\in{{\mathscr{U}}_{\mathsf{ad}}}=\big{\\{}\tilde{u}\in{\mathscr{U}}\,\big{|}\,u_{a}\leq\tilde{u}\leq
u_{b}\text{ in }{\mathscr{U}}\big{\\}},$ (5.3.50g)

where we additionally assume $u_{a},u_{b}\in
L^{\infty}(\mathscr{D};\mathbb{R}^{\mathsf{m}})$ in the case of
$U=L^{2}(\mathscr{D};\mathbb{R}^{\mathsf{m}})$. Recall that the solution to
(5.3.50f) can be expressed as $y=\hat{y}+\mathcal{S}u$. We set
$\mathcal{G}=\mathcal{C}\mathcal{S}$, which implies that
$\mathcal{G}\in\mathscr{L}({\mathscr{U}},{\mathscr{H}})$ holds.

Then we define the reduced objective $\hat{J}:{\mathscr{U}}\to\mathbb{R}^{2}$
by

$\displaystyle\hat{J}(u)=\left(\begin{array}[]{c}\hat{J}_{1}(u)\\\
\hat{J}_{2}(u)\end{array}\right)=J(\hat{y}+\mathcal{S}u,u)\quad\text{for
}u\in{\mathscr{U}}$

and consider the reduced bicriterial optimal control problem

$\min\hat{J}(u)\quad\text{s.t.}\quad u\in{{\mathscr{U}}_{\mathsf{ad}}}.$
($\mathbf{\widehat{BP}}$)

Problem ($\mathbf{\widehat{BP}}$) involves the minimization of a vector-valued
objective. This is done by using the concept of order relation and Pareto
optimality; cf. [Ehr05], for instance.

######  Definition 5.3.14.

The point $\bar{u}\in{{\mathscr{U}}_{\mathsf{ad}}}$ is called Pareto optimal
for ($\mathbf{\widehat{BP}}$) provided there is no other control
$u\in{{\mathscr{U}}_{\mathsf{ad}}}\setminus\\{\bar{u}\\}$ with
$\hat{J}_{i}(u)\leq\hat{J}_{i}(\bar{u})$, $i=1,2$, and
$J_{j}(u)<\hat{J}_{j}(\bar{u})$ for at least one index $j\in\\{1,2\\}$.

The goal of multiobjective optimization is to compute (a numerical
approximation of) the Pareto set

$\displaystyle{\mathscr{P}_{\mathsf{s}}}=\big{\\{}u\in{{\mathscr{U}}_{\mathsf{ad}}}\,\big{|}\,u\text{
is Pareto optimal}\big{\\}}\subset{\mathscr{U}}$

and the Pareto front

$\displaystyle{\mathscr{P}_{\mathsf{f}}}=\big{\\{}\hat{J}(u)\,\big{|}\,u\in{\mathscr{P}_{\mathsf{s}}}\big{\\}}\subset\mathbb{R}^{2}.$

In the next section we will introduce the Euclidean reference point method as
a tool to achieve this goal. For the theoretical investigation we need the
following properties of the functions $\hat{J}_{1}$ and $\hat{J}_{2}$, which
are proved in [BBV17].

######  Lemma 5.3.15.

  1. 1)

Both $\hat{J}_{1}$ and $\hat{J}_{2}$ are bounded from below.

  2. 2)

Both $\hat{J}_{1}$ and $\hat{J}_{2}$ are twice continuously Fréchet-
differentiable.

  3. 3)

$\hat{J}_{1}$ is convex and $\hat{J}_{2}$ is strictly convex.

  4. 4)

$\hat{J}_{1}$ is even strictly convex provided
$\mathcal{G}=\mathcal{C}\mathcal{S}$ is injective.

######  Remark 5.3.16.

Since the functions $\hat{J}_{1}$ and $\hat{J}_{2}$ are bounded from below,
convex and continuous, and ${{\mathscr{U}}_{\mathsf{ad}}}$ is bounded, the
ideal vector
${y^{\mathsf{id}}}=(y^{\mathsf{id}}_{1},y^{\mathsf{id}}_{2})\in\mathbb{R}^{2}$
given as

$\displaystyle
y^{\mathsf{id}}_{i}=\min\big{\\{}\hat{J}_{i}(u)\,\big{|}\,u\in{{\mathscr{U}}_{\mathsf{ad}}}\big{\\}}$

is well-defined. $\blacksquare$

####  5.3.2 The Euclidean reference point method

One popular methodology to solve multiobjective optimization problems is the
so-called scalarization method, where the multiobjective problem is
transformed into a series of scalar optimization problems. These can then be
solved by using well-known techniques for scalar optimization. Here we follow
[BBV17, POBD16, RBW+09]: Given a reference point

$\displaystyle
z=(z_{1},z_{2})\in{\mathscr{P}_{\mathsf{f}}}+\mathbb{R}^{2}_{\leq}=\big{\\{}z+x\,\big{|}\,z\in{\mathscr{P}_{\mathsf{f}}}\text{
and }x\in\mathbb{R}^{2}_{\leq}\big{\\}}$

we introduce the (Euclidean) distance function
$F_{z}:{\mathscr{U}}\to\mathbb{R}^{2}$ by

$\displaystyle
F_{z}(u)=\frac{1}{2}\,{\|\hat{J}(u)-z\|}_{2}^{2}=\frac{1}{2}\,\big{(}\hat{J}_{1}(u)-z_{1}\big{)}^{2}+\frac{1}{2}\,\big{(}\hat{J}_{2}(u)-z_{2}\big{)}^{2}\quad\text{for
}u\in{\mathscr{U}}.$

######  Lemma 5.3.17.

If $z\leq{y^{\mathsf{id}}}$ holds, $F_{z}$ is strictly convex.

Proof. The distance functions has the form

$\displaystyle F_{z}(u)=G_{1}(\hat{J}_{1}(u))+G_{2}(\hat{J}_{2}(u))$

with $G_{i}(\xi)=\tfrac{1}{2}(\xi-z_{i})^{2}$ being strictly monotonically
increasing on $[z_{i},\infty)$. Since $\hat{J}_{1}$ is convex and
$\hat{J}_{2}$ strictly convex by Lemma 5.3.15, $F_{z}$ itself is strictly
convex because $\hat{J}(u)\geq y^{\mathsf{id}}\geq z$ for all
$u\in{{\mathscr{U}}_{\mathsf{ad}}}$. $\Box$

To find a point that approximates $z$ as good as possible we consider

$\min F_{z}(u)\quad\text{s.t.}\quad u\in{{\mathscr{U}}_{\mathsf{ad}}}.$
($\mathbf{\widehat{BP}}_{z}$)

Notice that ($\mathbf{\widehat{BP}}_{z}$) is a scalar-valued optimization
problem which is convex provided $z\leq{y^{\mathsf{id}}}$ holds true. The next
result is proved in [Ban17].

######  Proposition 5.3.18.

Let $z\in{\mathscr{P}_{\mathsf{f}}}+\mathbb{R}^{2}_{\leq}$. Then
($\mathbf{\widehat{BP}}_{z}$) has a unique solution
$\bar{u}_{z}=(\bar{u}_{zj})_{1\leq j\leq\mathsf{m}}$, which is Pareto optimal
for ($\mathbf{\widehat{BP}}$).

The solution $\bar{u}_{z}$ can be characterized by the following optimality
conditions; cf. [BBV17].

######  Proposition 5.3.19.

Let $z\in{\mathscr{P}_{\mathsf{f}}}+\mathbb{R}^{2}_{\leq}$ and $\mathcal{G}$
be injective. If the point $\bar{u}_{z}$ is optimal for
($\mathbf{\widehat{BP}}_{z}$), then the first-order necessary optimality
condition

$\big{\langle}(\hat{J}_{1}(\bar{u}_{z})-z_{1})\mathcal{G}^{\star}(\mathcal{G}\bar{u}_{z}+\hat{y}-{y^{\mathsf{d}}})+(\hat{J}_{2}(\bar{u}_{z})-z_{2})(\bar{u}_{z}-{u^{\mathsf{n}}}),u-\bar{u}_{z}\big{\rangle}_{\mathscr{U}}\geq
0$ (5.3.51)

holds for all $u\in{{\mathscr{U}}_{\mathsf{ad}}}$. Moreover, if
$z_{1}\leq\hat{J}_{1}(\bar{u}_{z})$ and $z_{2}<\hat{J}_{2}(\bar{u}_{z})$ we
have that

$\displaystyle{\langle
F_{z}^{\prime\prime}(\bar{u}_{z})u^{\delta},u^{\delta}\rangle}_{{\mathscr{U}}^{\prime},{\mathscr{U}}}\geq\kappa_{z}\,{\|u^{\delta}\|}_{\mathscr{U}}^{2}\quad\forall
u^{\delta}\in{\mathscr{U}},$

where the hessian of
$F_{z}^{\prime\prime}(u):{\mathscr{U}}\to{\mathscr{U}}^{\prime}$ at a point
$u\in{\mathscr{U}}$ is given as

$\displaystyle
F_{z}^{\prime\prime}(u)u^{\delta}=\left\langle\big{(}(\hat{J}_{1}(u)-z_{1})\mathcal{G}^{\star}\mathcal{G}+(\hat{J}_{2}(u)-z_{2})\mathcal{I}_{\mathscr{U}}\big{)}u^{\delta},\cdot\right\rangle_{\mathscr{U}}\quad\text{for
}u^{\delta}\in{\mathscr{U}}$

and $\kappa_{z}=\hat{J}_{2}(\bar{u}_{z})-z_{2}>0$. If even
$z\leq{y^{\mathsf{id}}}$ is valid, (5.3.51) is a sufficient optimality
condition for ($\mathbf{\widehat{BP}}_{z}$).

The idea is to solve ($\mathbf{\widehat{BP}}_{z}$) for different appropriately
chosen values of $z$ to get a uniform approximation of the Pareto front. To be
more precise, we want to construct a sequence
$\\{z^{(k)}\\}_{k\in\mathbb{N}}\subset\mathbb{R}^{2}$ of reference points
along with optimal controls
$\\{u^{(k)}\\}_{k\in\mathbb{N}}\subset{{\mathscr{U}}_{\mathsf{ad}}}$ that
solve ($\mathbf{\widehat{BP}}_{z}$) with $z=z^{(k)}$ as well as
$\\{\hat{J}^{(k)}\\}_{k\in\mathbb{N}}\subset\mathbb{R}^{2}$ with
$\hat{J}^{(k)}=\hat{J}(u^{(k)})$, such that a predefined quality criteria for
the approximation of the Pareto front is fulfilled. That this is possible in
theory is shown in the next proposition, which was proved in [Ban17, Theorem
3.43, Theorem 3.47].

######  Proposition 5.3.20.

  1. 1)

For any $u\in{\mathscr{P}_{\mathsf{s}}}$ there is
$z\in{\mathscr{P}_{\mathsf{f}}}+\mathbb{R}^{2}_{\leq}$ such that $u$ is the
unique solution of ($\mathbf{\widehat{BP}}_{z}$).

  2. 2)

Let $z^{(1)},z^{(2)}\in{\mathscr{P}_{\mathsf{f}}}+\mathbb{R}^{2}_{\leq}$ be
arbitrary and denote by
$\bar{u}_{z^{(1)}},\bar{u}_{z^{(2)}}\in{\mathscr{P}_{\mathsf{s}}}$ the unique
solutions of the respective Euclidean reference point problems
($\mathbf{\widehat{BP}}_{z}$). Then it holds

$\left\|J(\bar{u}_{z^{(1)}})-J(\bar{u}_{z^{(2)}})\right\|\leq\left\|z^{(1)}-z^{(2)}\right\|.$
(5.3.52)

The first statement of the previous proposition tells us that we can indeed
compute every Pareto optimal control $u\in{\mathscr{P}_{\mathsf{s}}}$ by
solving a Euclidean reference point problem ($\mathbf{\widehat{BP}}_{z}$).
Moreover, the second statement implies that the distance between two reference
points $z^{(1)},z^{(2)}$ is an upper bound of the distance between the
corresponding Pareto optimal points
$J(\bar{u}_{z^{(1)}}),J(\bar{u}_{z^{(2)}})\in{\mathscr{P}_{\mathsf{f}}}$. This
enables us to define an iterative sequence of reference points with a
guaranteed upper bound on the maximal distance between two neighbouring Pareto
optimal points. The exact procedure is the following: We first compute the
minimizers of $\hat{J_{1}}$ and $\hat{J}_{2}$, which are denoted by
$\bar{u}^{(1)}$ and $\bar{u}^{(\text{end})}$. Then the corresponding Pareto
optimal points $\hat{J}^{(1)}:=\hat{J}(\bar{u}^{(1)})$ and
$\hat{J}^{(\text{end})}:=\hat{J}(\bar{u}^{(\text{end})})$ mark the start and
the end point of our iteration. Note that in our example computing the
minimizer of $\hat{J}_{1}$ numerically is quite challenging. Therefore,
instead of the minimizer of $\hat{J}_{1}$, we compute the minimizer of the
weighted-sum function $\hat{J}_{1}+\alpha\hat{J}_{2}$ for a weight
$0<\alpha\ll 1$, which is on the one hand a good approximation of the
minimizer of $\hat{J}_{1}$ and on the other hand regular enough to solve it
numerically. Starting at the point $\hat{J}^{(1)}$, we generate the first
reference point by the formula

$z^{(1)}=\hat{J}^{(1)}+h^{||}\cdot\frac{(\alpha,-1)^{T}}{\left\|(\alpha,-1)^{\top}\right\|}+h^{\perp}\cdot\frac{(-1,-\alpha)^{\top}}{\left\|(-1,-\alpha)^{T}\right\|}.$
(5.3.53)

Here, we make use of the fact that the Pareto front at the point
$\hat{J}^{(1)}$ is tangential to the vector $\left(\alpha,-1\right)^{T}$.
Thus, the parameter $h^{||}>0$ determines how far along the Pareto front the
reference point will be chosen. By $h^{\perp}\geq 0$ we can choose how far the
new reference point should be located from the Pareto front. Analogously, one
can show that after solving a Euclidean reference point problem
($\mathbf{\widehat{BP}}_{z}$) to a reference point $z$, the vectors
$\varphi^{\perp}=z-\hat{J}(\bar{u}^{z})$ and
$\varphi^{||}=(-\varphi^{\perp}_{2},\varphi^{\perp}_{1})^{\top}$ are
perpendicular and tangential to the Pareto front at the point
$\hat{J}(\bar{u}^{z})$, respectively. This motivates the iterative formula for
the reference points

$z^{(i+1)}=z^{(i)}+h^{||}\frac{\varphi^{||}}{\left\|\varphi^{||}\right\|}+h^{\perp}\frac{\varphi^{\perp}}{\left\|\varphi^{\perp}\right\|}$
(5.3.54)

for $i=1,\ldots$, see Figure 5.3.4.

Figure 5.3.4:  Graphical visualization of the iterative reference point
generation (left) and a Pareto front which was computed by Algorithm 5.3.2
(right).

Using this formula it is guaranteed that

$\left\|J^{(i+1)}-J^{(i)}\right\|\leq h^{||}$

holds for all $i=1,\ldots$, which is why we can control the approximation
quality by adapting the parameter $h^{||}$. We stop this iteration when the
first component of the new reference point is larger than the first component
of our end point $\hat{J}^{(\text{end})}$. The complete methodology is
summarized in Algorithm 5.3.2.

0: Parameters $h^{||}>0$, $h^{\perp}\geq 0$, weighted-sum parameter
$\alpha>0$.

1: Minimize $\hat{J}_{1}+\alpha\hat{J}_{2}$ and $\hat{J}_{2}$ and save the
solutions as $\bar{u}^{1}$ and $\bar{u}^{(\text{end})}$. Set
$\hat{J}^{(1)}:=\hat{J}(\bar{u}^{1})$ and
$\hat{J}^{(\text{end})}:=\hat{J}(\bar{u}^{(\text{end})})$.

2: Compute $z^{(1)}$ by (5.3.53).

3: for $n=1,\ldots$ do

4: Solve ($\mathbf{\widehat{BP}}_{z}$) with reference point $z^{(n)}$ and save
the solution as $\bar{u}^{(n+1)}$ and
$\hat{J}^{(n+1)}:=\hat{J}(\bar{u}^{(n+1)})$.

5: Compute $z^{(n+1)}$ by (5.3.54).

6: if $z^{(n+1)}_{1}>\hat{J}^{(\text{end})}_{1}$ then

7: return $(\bar{u}^{(1)},\ldots,\bar{u}^{(n+1)},\bar{u}^{(\text{end})})$ and
$(\hat{J}^{(1)},\ldots,\hat{J}^{(n+1)},\hat{J}^{(\text{end})})$

8: end if

9: end for

For more details we refer to [BBV17, Section 6.1] and [Ban17, Section 3.4].

####  5.3.3 The POD approximation

In this section we want to show how the POD method can be successfully applied
to the bicriterial optimal control problem (5.3.50). In particular, we will
follow the outline of Section 4.2 and show a-priori convergence results and
a-posteriori estimates. The analysis is only done for the continuous POD
method, but can also be transferred to the POD method for the semidiscrete and
the fully discrete approximation.

Suppose that we have computed a POD basis $\\{\psi_{i}\\}_{i=1}^{\ell}$ of
rank $\ell$. Then the reduced-order cost function is given as

$\displaystyle\hat{J}^{\ell}(u)=J(\hat{y}^{\ell}+\mathcal{S}^{\ell}u,u)\quad\text{for
}u\in{{\mathscr{U}}_{\mathsf{ad}}}.$

######  Remark 5.3.21.

From the definition it follows that $\hat{J}_{2}=\hat{J}_{2}^{\ell}$. However,
to avoid misunderstandings we will still write $\hat{J}_{2}^{\ell}$ if we want
to refer to the second cost function in the POD case. $\blacksquare$

The reduced-order bicriterial optimal control problem then reads

$\min\hat{J}^{\ell}(u)\quad\text{s.t.}\quad
u\in{{\mathscr{U}}_{\mathsf{ad}}}.$ ($\mathbf{\widehat{BP}}^{\ell}$)

Its Pareto set and Paret front are denoted by
${\mathscr{P}_{\mathsf{s}}^{\ell}}$ and ${\mathscr{P}_{\mathsf{f}}^{\ell}}$,
respectively. For a given reference point
$z\in{\mathscr{P}_{\mathsf{f}}^{\ell}}+\mathbb{R}^{2}_{\leq}$ the Euclidean
reference point problem ($\mathbf{\widehat{BP}}_{z}$) is replaced by its
reduced-order equivalent

$\min F^{\ell}_{z}(u)\quad\text{s.t.}\quad u\in{{\mathscr{U}}_{\mathsf{ad}}},$
($\mathbf{\widehat{BP}}^{\ell}_{z}$)

where the reduced-order distance function
$F^{\ell}_{z}:{\mathscr{U}}\to\mathbb{R}^{2}$ is given by

$\displaystyle
F^{\ell}_{z}(u)=\frac{1}{2}\,{\|\hat{J}^{\ell}(u)-z\|}_{2}^{2}=\frac{1}{2}\,\big{(}\hat{J}^{\ell}_{1}(u)-z_{1}\big{)}^{2}+\frac{1}{2}\,\big{(}\hat{J}^{\ell}_{2}(u)-z_{2}\big{)}^{2}\quad\text{for
}u\in{\mathscr{U}}.$

######  Remark 5.3.22.

The results of the Lemmas 5.3.15 and 5.3.17, and the Propositions 5.3.18 and
5.3.20 still hold true for the reduced-order problem. Moreover, the first-
order necessary optimality condition for the optimal solution
$\bar{u}_{z}^{\ell}$ of ($\mathbf{\widehat{BP}}^{\ell}_{z}$) is given by

$\big{\langle}(\hat{J}_{1}^{\ell}(\bar{u}_{z}^{\ell})-z_{1})(\mathcal{G}^{\ell})^{\star}(\mathcal{G}^{\ell}\bar{u}_{z}^{\ell}+\hat{y}^{\ell}-{y^{\mathsf{d}}})+(\hat{J}_{2}^{\ell}(\bar{u}_{z}^{\ell})-z_{2})(\bar{u}_{z}^{\ell}-{u^{\mathsf{n}}}),u-\bar{u}_{z}^{\ell}\big{\rangle}_{\mathscr{U}}\geq
0$ (5.3.55)

for all $u\in{{\mathscr{U}}_{\mathsf{ad}}}$, where the operator
$\mathcal{G}^{\ell}$ is defined as
$\mathcal{G}^{\ell}=\mathcal{C}\mathcal{S}^{\ell}$. $\blacksquare$

##### POD a-priori analysis

In this section we deal with the a-priori analysis for the optimality system
of the Euclidean reference point problem ($\mathbf{\widehat{BP}}_{z}$), which
is no linear-quadratic optimal control problem anymore due to the
concatenation of the linear-quadratic cost functions with the Euclidean norm,
for the continuous POD method. However, the idea and the procedure of our
analysis strongly follows Section 4.2.3. A-priori estimates for both the state
and adjoint equation for the continuous POD method can be found in Section
3.3.2 and Section 4.2.2. In particular, for a given reference point $z$ we are
interested in a-priori convergence results for the error between the solutions
$\bar{u}$ and $\bar{u}^{\ell}$ to ($\mathbf{\widehat{BP}}_{z}$) and
($\mathbf{\widehat{BP}}^{\ell}_{z}$), respectively. To be able to prove an
a-priori convergence result for the optimal controls of
($\mathbf{\widehat{BP}}^{\ell}_{z}$) to the one of
($\mathbf{\widehat{BP}}_{z}$), we first need to show the following result
involving the first cost function $\hat{J}_{1}$ and its gradient
$\nabla\hat{J}_{1}$.

######  Lemma 5.3.23.

Assume that Assumptions 4.1.1, 4.1.8 and 4.2.2 for a $u\in{\mathscr{U}}$ are
satisfied. Then it holds

$\left|\hat{J}_{1}(u)-\hat{J}_{1}^{\ell}(u)\right|^{2}\leq
C\cdot\left\\{\begin{aligned}
&\sum_{i>\ell}\lambda_{i}^{H}\big{(}{\|\psi_{i}^{H}\|}_{V}^{2}+1\big{)}&&\text{for
}X=H,~{}\mathcal{P}^{\ell}=\mathcal{P}^{\ell}_{H},\\\
&\sum_{i>\ell}\lambda_{i}^{H}\big{\|}\psi_{i}^{H}-\mathcal{P}^{\ell}_{V}\psi_{i}^{H}\big{\|}_{V}^{2}&&\text{for
}X=H,~{}\mathcal{P}^{\ell}=\mathcal{P}^{\ell}_{V},\\\
&\sum_{i>\ell}\lambda_{i}^{V}{\|\psi_{i}^{V}-\mathcal{Q}_{H}^{\ell}\psi_{i}^{V}\|}_{V}^{2}&&\text{for
}X=V,~{}\mathcal{P}^{\ell}=\mathcal{Q}^{\ell}_{H},\\\
&\sum_{i>\ell}\lambda_{i}^{V}&&\text{for
}X=V,~{}\mathcal{P}^{\ell}=\mathcal{Q}^{\ell}_{V},\end{aligned}\right.$
(5.3.56)

for a constant $C>0$ independent of $\ell$ and

$\left\|\nabla\hat{J}_{1}(u)-\nabla\hat{J}_{1}^{\ell}(u)\right\|_{\mathscr{U}}^{2}\leq
C\cdot\left\\{\begin{aligned}
&\sum_{i>\ell}\lambda_{i}^{H}\,\big{\|}\psi_{i}^{H}\big{\|}_{V}^{2},&&X=H,~{}\mathcal{P}^{\ell}=\mathcal{P}^{\ell}_{H},\\\
&\sum_{i>\ell}\lambda_{i}^{V}\,\big{\|}\psi_{i}^{V}-\mathcal{P}^{\ell}\psi_{i}^{V}\big{\|}_{V}^{2},&&X=V,~{}\mathcal{P}^{\ell}=\mathcal{Q}^{\ell}_{H},\end{aligned}\right.$
(5.3.57)

for a constant $C>0$ independent of $\ell$. In particular, it holds

$\lim_{\ell\to\infty}\left|\hat{J}_{1}(u)-\hat{J}_{1}^{\ell}(u)\right|=0,\quad\lim_{\ell\to\infty}\left\|\nabla\hat{J}_{1}(u)-\nabla\hat{J}_{1}^{\ell}(u)\right\|_{\mathscr{U}}=0.$
(5.3.58)

Proof. To prove (5.3.56) we note that it is possible to derive the following
estimate for an arbitrary control $u\in{\mathscr{U}}$, (cf. [Ban17, Proof of
Theorem 5.31], where a similar estimate was shown for the case that
$\hat{y}^{\ell}=\hat{y}$),

$\displaystyle\left|\hat{J}_{1}(u)-\hat{J}_{1}^{\ell}(u)\right|$ (5.3.59)
$\displaystyle\quad\leq\frac{1}{2}\left|\big{\|}\mathcal{S}u+\hat{y}\big{\|}_{L^{2}(0,T;H)}+\big{\|}\mathcal{S}^{\ell}u+\hat{y}^{\ell}\big{\|}_{L^{2}(0,T;H)}\right|\big{\|}\mathcal{S}u+\hat{y}-(\mathcal{S}^{\ell}u+\hat{y}^{\ell})\big{\|}_{L^{2}(0,T;H)}$
$\displaystyle\qquad+\big{\|}\mathcal{S}u+\hat{y}-(\mathcal{S}^{\ell}u+\hat{y}^{\ell})\big{\|}_{L^{2}(0,T;H)}\big{\|}y_{d}\big{\|}_{L^{2}(0,T;H)}.$

Now (5.3.56) follows from Theorem 3.3.14 and Corollary 3.3.9. For (5.3.57) we
can use the formula for the gradient to conclude

$\displaystyle\big{\|}\nabla\hat{J}_{1}(u)-\nabla\hat{J}_{1}^{\ell}(u)\big{\|}_{\mathscr{U}}\leq\|\mathcal{B}^{\prime}\|_{\mathscr{L}(L^{2}(0,T;V),{\mathscr{U}})}\big{\|}\mathcal{A}u+\hat{p}-(\mathcal{A}^{\ell}u+\hat{p}^{\ell})\big{\|}_{L^{2}(0,T;V)}.$
(5.3.60)

Together with Theorem 4.2.8 this implies (5.3.57). $\Box$

Now it is possible to show a-priori convergence with a convergence rate.

######  Theorem 5.3.24.

Let
$z\in\bigcap_{\ell\in\mathbb{N}}({\mathscr{P}_{\mathsf{f}}^{\ell}}+\mathbb{R}_{\leq}^{2})\bigcap({\mathscr{P}_{\mathsf{f}}}+\mathbb{R}_{\leq}^{2})$
be a reference point and denote by $\bar{u}$ and $\bar{u}^{\ell}$ the
solutions to ($\mathbf{\widehat{BP}}_{z}$) and
($\mathbf{\widehat{BP}}^{\ell}_{z}$), respectively. Assume that Assumptions
4.1.1, 4.1.8 and 4.2.2 with $u=\bar{u}$ hold. Let
$\|\mathcal{Q_{H}^{\ell}}\|_{\mathcal{L}(V)}$ be bounded independently of
$\ell$. If it holds $\hat{J}_{2}(\bar{u})>z_{2}$, then

$\big{\|}\hat{J}(\bar{u})-\hat{J}^{\ell}(\bar{u}^{\ell})\big{\|}^{2}+\big{\|}\bar{u}-\bar{u}^{\ell}\big{\|}_{\mathscr{U}}^{2}\leq
C\cdot\left\\{\begin{aligned}
&\sum_{i>\ell}\lambda_{i}^{H}\,\big{\|}\psi_{i}^{H}\big{\|}_{V}^{2},&&X=H,~{}\mathcal{P}^{\ell}=\mathcal{P}^{\ell}_{H},\\\
&\sum_{i>\ell}\lambda_{i}^{V}\,\big{\|}\psi_{i}^{V}-\mathcal{P}^{\ell}\psi_{i}^{V}\big{\|}_{V}^{2},&&X=V,~{}\mathcal{P}^{\ell}=\mathcal{Q}^{\ell}_{H}.\end{aligned}\right.$
(5.3.61)

In particular, it holds

$\lim_{\ell\to\infty}\big{\|}\hat{J}(\bar{u})-\hat{J}^{\ell}(\bar{u}^{\ell})\big{\|}+\big{\|}\bar{u}-\bar{u}^{\ell}\big{\|}_{{\mathscr{U}}}=0.$
(5.3.62)

Proof. We follow the lines of the proof of [Ban17, Theorem 5.41] and get the
estimate

$\displaystyle\big{\|}\hat{J}^{\ell}(\bar{u}^{\ell})-\hat{J}(\bar{u})\big{\|}_{\mathbb{R}^{2}}^{2}+\left(\frac{\hat{J}_{1}^{\ell}(\bar{u}^{\ell})+\hat{J}_{1}(\bar{u})}{2}-z_{1}\right)\big{\|}\mathcal{S}^{\ell}(\bar{u}^{\ell}-\bar{u})\big{\|}_{L^{2}(0,T;H)}^{2}$
$\displaystyle+\left(\frac{\hat{J}_{2}(\bar{u})+\hat{J}_{2}^{\ell}(\bar{u}^{\ell})}{2}-z_{2}\right)\big{\|}\bar{u}^{\ell}-\bar{u}\big{\|}_{\mathscr{U}}^{2}$
$\displaystyle\leq\left(\hat{J}_{1}(\bar{u})-z_{1}\right)\langle\nabla\hat{J}_{1}(\bar{u})-\nabla\hat{J}_{1}^{\ell}(\bar{u}),\bar{u}^{\ell}-\bar{u}\rangle_{U}+\left(\hat{J}_{1}(\bar{u})-\hat{J}_{1}^{\ell}(\bar{u}^{\ell})\right)\left(\hat{J}_{1}(\bar{u})-\hat{J}_{1}^{\ell}(\bar{u})\right).$

Using $\hat{J}_{1}^{\ell}(\bar{u}^{\ell}),\hat{J}_{1}(\bar{u})\geq z_{1}$,
$\hat{J}_{2}^{\ell}(\bar{u}^{\ell})\geq z_{2}$ (which follow from [Ban17,
Corollary 3.41]), the Cauchy-Schwarz inequality and Young’s inequality we get

$\displaystyle\big{\|}\hat{J}^{\ell}(\bar{u}^{\ell})-\hat{J}(\bar{u})\big{\|}_{\mathbb{R}^{2}}^{2}+\frac{\hat{J}_{2}(\bar{u})-z_{2}}{2}\big{\|}\bar{u}^{\ell}-\bar{u}\big{\|}_{\mathscr{U}}^{2}$
$\displaystyle\leq
C(\epsilon)\left(\hat{J}_{1}(\bar{u})-z_{1}\right)^{2}\big{\|}\nabla\hat{J}_{1}(\bar{u})-\nabla\hat{J}_{1}^{\ell}(\bar{u})\big{\|}_{\mathscr{U}}^{2}+\epsilon\,\big{\|}\bar{u}^{\ell}-\bar{u}\big{\|}_{\mathscr{U}}^{2}$
$\displaystyle\quad+\frac{1}{2}\left(\hat{J}_{1}(\bar{u})-\hat{J}_{1}^{\ell}(\bar{u}^{\ell})\right)^{2}+\frac{1}{2}\left(\hat{J}_{1}(\bar{u})-\hat{J}_{1}^{\ell}(\bar{u})\right)^{2},$

which yields

$\displaystyle\big{\|}\hat{J}^{\ell}(\bar{u}^{\ell})-\hat{J}(\bar{u})\big{\|}_{\mathbb{R}^{2}}^{2}+\big{\|}\bar{u}^{\ell}-\bar{u}\big{\|}_{\mathscr{U}}^{2}\leq
C\left(\big{\|}\nabla\hat{J}_{1}(\bar{u})-\nabla\hat{J}_{1}^{\ell}(\bar{u})\big{\|}_{\mathscr{U}}^{2}+\left(\hat{J}_{1}(\bar{u})-\hat{J}_{1}^{\ell}(\bar{u})\right)^{2}\right)$

for a constant $C>0$ independent of $\ell$. Now the claim follows from Lemma
5.3.23.

Theorem 5.3.24 together with the continuity of $\hat{J}_{1}$ and
$\nabla\hat{J}_{1}$ and a-priori convergence statements for the state and
adjoint equation also yields a-priori convergence results for the cost
functions and its gradients, cf. [Ban17, Corollary 5.42].

######  Theorem 5.3.25.

Let the assumptions of Theorem 5.3.24 be satisfied. Then it holds

  1. 1)

$\hat{J}_{1}(\bar{u}^{\ell})\to\hat{J}_{1}(\bar{u})$ as $\ell\to\infty$.

  2. 2)

$\nabla\hat{J}_{1}(\bar{u}^{\ell})\to\nabla\hat{J}_{1}(\bar{u})$ and
$\nabla\hat{J}_{1}^{\ell}(\bar{u}^{\ell})\to\nabla\hat{J}_{1}(\bar{u})$ as
$\ell\to\infty$.

  3. 3)

$F_{z}(\bar{u}^{\ell})\to F_{z}(\bar{u})$ and $F_{z}^{\ell}(\bar{u}^{\ell})\to
F_{z}(\bar{u})$ as $\ell\to\infty$.

  4. 4)

$\nabla F_{z}(\bar{u}^{\ell})\to\nabla F_{z}(\bar{u})$ and $\nabla
F_{z}^{\ell}(\bar{u}^{\ell})\to\nabla F_{z}(\bar{u})$ as $\ell\to\infty$.

##### POD a-posteriori analysis

The focus of this section is to discuss an a-posteriori estimate for the error
$\|\bar{u}-\bar{u}^{\ell}\|_{{\mathscr{U}}}$ between the solution $\bar{u}$ of
($\mathbf{\widehat{BP}}_{z}$) and a suboptimal control $\bar{u}^{\ell}$, which
is possibly – but not necessarily – the solution of
($\mathbf{\widehat{BP}}^{\ell}_{z}$). As in Section 4.2.4 we make use of a
perturbation method (cf. [DHPY95]) and the fact that the cost functions
$\hat{J}_{1}$ and $\hat{J}_{2}$ are still linear-quadratic. Thus, for details
of the idea of the perturbation method we directly refer to Section 4.2.4.
Here, we only state the following theorem, which was proved in [Ban17,
Corollary 5.43 and Theorem 5.45].

######  Theorem 5.3.26.

Suppose that Assumption 4.1.8 holds. Let
$z\in{\mathscr{P}_{\mathsf{f}}}+\mathbb{R}_{\leq}^{2}$ be a reference point
and $\\{\psi_{i}\\}_{i=1}^{\ell}$ be a POD basis of rank $\ell$. Define the
function $\zeta^{\ell}\in{\mathscr{U}}$ by

$\displaystyle\zeta_{i}^{\ell}(t)=\left\\{\begin{aligned} &-\min(0,(\nabla
F_{z}^{\ell}(\bar{u}^{\ell}))_{i}(s))&&\text{a.e. in
}\mathscr{A}^{\ell}_{\mathsf{a}i}=\big{\\{}s\in\mathscr{D}\,|\bar{u}^{\ell}_{i}(s)=u_{\mathsf{a}i}(s)\big{\\}},\\\
&-\max(0,(\nabla F_{z}^{\ell}(\bar{u}^{\ell}))_{i}(s))&&\text{a.e. in
}\mathscr{A}^{\ell}_{\mathsf{b}i}=\big{\\{}s\in\mathscr{D}\,|\bar{u}^{\ell}_{i}(s)=u_{\mathsf{b}i}(s)\big{\\}},\\\
&-(\nabla F_{z}^{\ell}(\bar{u}^{\ell}))_{i}(t)&&\text{a.e. in
}\mathscr{D}\setminus\big{(}\mathscr{A}^{\ell}_{\mathsf{a}i}\cup\mathscr{A}_{\mathsf{b}i}^{\ell}\big{)}.\end{aligned}\right.$

If $\hat{J}(\bar{u}^{\ell})\geq z$ and $\hat{J}_{2}(\bar{u}^{\ell})>z_{2}$ are
satisfied, then the a-posteriori error estimates

${\|\bar{u}-\bar{u}^{\ell}\|}_{\mathscr{U}}\leq\left(\frac{\hat{J}_{2}(\bar{u}^{\ell})-z_{2}}{2}\right)^{-1}\,{\|\zeta^{\ell}\|}_{\mathscr{U}},\quad{\|\hat{J}(\bar{u})-\hat{J}(\bar{u}^{\ell})\|}_{\mathscr{U}}\leq\frac{1}{2}\left(\frac{\hat{J}_{2}(\bar{u}^{\ell})-z_{2}}{2}\right)^{-1/2}\,{\|\zeta^{\ell}\|}_{\mathscr{U}}$
(5.3.63)

hold. In particular, it also holds
$\lim\limits_{\ell\to\infty}\|\zeta^{\ell}\|_{\mathscr{U}}=0$.

######  Remark 5.3.27.

In contrast to the a-posteriori estimate in Theorem 4.2.14, we also get an
estimate for the error between the optimal and the suboptimal cost function
values $\hat{J}(\bar{u})$ and $\hat{J}(\bar{u}^{\ell})$. Also for the a-priori
convergence statement in Theorem 5.3.24 we get a convergence rate for the
error $\left\|\hat{J}(\bar{u})-\hat{J}^{\ell}(\bar{u}^{\ell})\right\|$ – again
in contrast to the a-priori convergence result in Theorem 4.2.12. Together
with the statement from Proposition 5.3.20 2), this is the main reason to use
the Euclidean reference point method instead of the weighted-sum method (see
e.g. [Ehr05, Chapter 3]) to solve the bicriterial optimal control problem
($\mathbf{\widehat{BP}}$). By concatenating the cost functions with the
Euclidean norm the Euclidean reference point problem gives us more geometrical
information about the Pareto front, which can then be used to generate a
uniform approximation of it, even in the case when POD is applied. This makes
up for the fact that Euclidean reference point problems are not linear-
quadratic anymore and thus harder to solve than the weighted-sum problems in
general. $\blacksquare$

Using the a-posteriori estimate from Theorem 5.3.26 we set up an algorithm
using POD for model-order reduction and controlling the maximal error between
the POD and the finite element solutions; see Algorithm 5.3.3.

0: Recursive parameters $h^{||}>0$, $h^{\perp}\geq 0$, weighted-sum parameter
$\alpha>0$, initial length of POD basis $\ell_{0}$, number of incremented
basis elements $\ell_{\text{incr}}$, maximal error in control space
$\varepsilon_{\text{max}}$.

1: Minimize $\hat{J}_{1}+\alpha\hat{J}_{2}$ and $\hat{J}_{2}$ and save the
solutions as $\bar{u}^{(1),\ell}$ and $\bar{u}^{(\text{end}),\ell}$. Set
$\hat{J}^{(1),\ell}:=\hat{J}(\bar{u}^{(1),\ell})$ and
$\hat{J}^{(\text{end}),\ell}:=\hat{J}(\bar{u}^{(\text{end}),\ell})$.

2: Compute a POD basis of length $\ell_{0}$ using the snapshots
$\mathcal{S}\bar{u}^{(1),\ell}+\hat{y}$ and
$\mathcal{A}\bar{u}^{(1),\ell}+\hat{p}$.

3: Compute $z^{(1),\ell}$ by (5.3.53).

4: for $n=1,\ldots$ do

5: Set bool $\leftarrow$ FALSE.

6: while bool is FALSE do

7: Solve ($\mathbf{\widehat{BP}}^{\ell}_{z}$) with reference point
$z^{(n),\ell}$.

8: Compute the a-posteriori estimate $\eta$ from (5.3.63)

9: if $\eta>\varepsilon_{\text{max}}$ then

10: Set size $\ell$ of POD basis to $\ell\leftarrow\ell+\ell_{\text{incr}}$.

11: else

12: Save the solution to ($\mathbf{\widehat{BP}}^{\ell}_{z}$) as
$\bar{u}^{(n+1),\ell}$ and
$\hat{J}^{(n+1),\ell}:=\hat{J}(\bar{u}^{(n+1),\ell})$.

13: Set bool $\leftarrow$ TRUE.

14: end if

15: end while

16: Compute $z^{(n+1),\ell}$ by (5.3.54).

17: if $z^{(n+1),\ell}_{1}>\hat{J}^{(\text{end}),\ell}_{1}$ then

18: return
$(\bar{u}^{(1),\ell},\ldots,\bar{u}^{(n+1),\ell},\bar{u}^{(\text{end}),\ell})$
and
$(\hat{J}^{(1),\ell},\ldots,\hat{J}^{(n+1),\ell},\hat{J}^{(\text{end}),\ell})$.

19: end if

20: end for

Of course, this simple POD basis extension strategy might not work for complex
problems. More elaborate strategies to adapt the POD basis when necessary can
be found in [Mak18, Spu19, BMV19].

##  Chapter A Appendix

###  A.1 Basics

######  Definition A.1.28 (Complex numbers).

  1. 1)

For a complex number $\alpha\in\mathbb{C}$, let $\Re(\alpha)$ and
$\Im(\alpha)\in\mathbb{R}$ denote its real and imaginary part, so we have
$\alpha=\Re(\alpha)+\jmath\cdot\Im(\alpha)$ where $\jmath$ is the complex
unity. The _complex conjugate_ of $\alpha$ is given by
$\overline{\alpha}=\Re(\alpha)-\jmath\cdot\Im(\alpha)$.

  2. 2)

We write a vector $z\in\mathbb{C}^{n}$ as $z=(z_{i})_{i}$ with
$z_{1},\ldots,z_{n}\in\mathbb{C}$ and a matrix $Z\in\mathbb{C}^{m\times n}$ as
$Z=(Z_{ij})_{ij}$ with $Z_{ij}\in\mathbb{C}$ for $i=1,...,m$, $j=1,...,n$. The
vector $z$ can then also be understood as a matrix $z\in\mathbb{C}^{n\times
1}$. For the matrix, we define its _transpose_ as
$Z^{\top}=(Z_{ij})_{ji}\in\mathbb{C}^{n\times m}$ and its _Hermitian_ as
$Z^{\mathsf{H}}=(\overline{Z}_{ij})_{ji}\in\mathbb{C}^{n\times m}$..

  3. 3)

The canonical inner product in $\mathbb{C}^{n}$ is given by $\langle
y,z\rangle_{\mathbb{C}^{n}}=y^{\top}\overline{z}$ for $y,z\in\mathbb{C}^{n}$.
The induced norm is given by $\|z\|_{\mathbb{C}^{n}}=\sqrt{\langle
z,z\rangle_{\mathbb{C}^{n}}}$ for $z\in\mathbb{C}^{n}$.

  4. 4)

We write $|\cdot|_{2}$ for the Euclidean norm, and
$\langle\cdot\,,\cdot\rangle_{2}$ stands for the canonical inner product in
$\mathbb{R}^{m}$.

Recall that the Frobenius norm is given by

$\displaystyle{\|{\bm{A}}\|}_{F}=\sqrt{\sum_{i=1}^{m}\sum_{j=1}^{n}\big{|}A_{ij}\big{|}^{2}}\quad\text{for
}{\bm{A}}\in\mathbb{C}^{m\times n}.$

######  Definition A.1.29 (Tensor notation).

Let $X$ be a set, $n,m\in\mathbb{N}$ and $b_{i}\in X$ for $1\leq i\leq n$ as
well as $a_{ij}\in X$ for $1\leq i\leq n$ and $1\leq j\leq m$. Then we write
$(b_{i})$ for the vector $(b_{1},\ldots,b_{n})\in X^{n}$ and $((a_{ij}))$ for
the matrix $(a_{ij})_{1\leq i\leq n,\,1\leq j\leq m}\in X^{n\times m}$.

######  Lemma A.1.30 (Young’s inequality, [Alt92]).

For any real numbers $a,b,\varepsilon>0$ and for all $p,q\in(1,\infty)$ with
$\tfrac{1}{p}+\tfrac{1}{q}=1$, it holds

$\displaystyle ab\leq\frac{\varepsilon
a^{p}}{p}+\frac{b^{q}}{q\varepsilon^{q/p}}$

######  Definition A.1.31 (Gradient and Hessian).

  1. 1)

If $f:\mathbb{R}^{n}\to\mathbb{R}^{m}$ is a differentiable function with
$m>1$, we write its _Jacobian matrix_ as $\nabla f(x)=(\tfrac{\partial
f_{i}}{\partial x_{j}}(x))_{ij}\in\mathbb{R}^{m\times n}$ for
$x\in\mathbb{R}^{n}$. If $m=1$, we use this symbol for the _gradient_ $\nabla
f(x)=(\tfrac{\partial f}{\partial x_{1}}(x),\ldots,\tfrac{\partial f}{\partial
x_{n}}(x))^{\top}\in\mathbb{R}^{n}$.

  2. 2)

If $f$ is twice diffentiable with $m=1$, we denote the _Hessian matrix_ by

$\displaystyle\nabla^{2}f(x)=\nabla_{xx}f(x)=\big{(}\tfrac{\partial^{2}f}{\partial
x_{i}\partial x_{j}}(x)\big{)}_{ij}\in\mathbb{R}^{n\times n}\qquad\text{for
}x\in\mathbb{R}^{n}$

Recall that if $f$ is twice continuously differentiable, the Schwarz theorem
states that $(\nabla^{2}f(x))^{\top}=\nabla^{2}f(x)$ holds for all
$x\in\mathbb{R}^{n}$.

######  Proposition A.1.32 (Gronwall inequality, [Eva08, p. 708]).

Let $\eta:[{t_{\circ}},{t_{\mathsf{e}}}]\to\mathbb{R}$ be a nonnegative,
differentiable function satisfying

$\displaystyle\eta^{\prime}(t)\leq\varphi(t)\eta(t)+\chi(t)\quad\text{a.e. in
}({t_{\circ}},{t_{\mathsf{e}}}],$

where $\varphi$ and $\chi$ are real-valued, nonnegative, integrable functions
on $[{t_{\circ}},{t_{\mathsf{e}}}]$. Then,

$\displaystyle\eta(t)\leq\exp\bigg{(}\int_{t_{\circ}}^{t}\varphi(s)\,\mathrm{d}s\bigg{)}\bigg{(}\eta({t_{\circ}})+\int_{t_{\circ}}^{t}\chi(s)\,\mathrm{d}s\bigg{)}$

In particular, if $\chi=0$ almost everywhere in $[0,T]$ and $\eta(0)=0$ hold
true, we have $\eta=0$ almost everywhere in $[0,T]$.

Setting $\tilde{\eta}(t)=\eta({t_{\circ}}+{t_{\mathsf{e}}}-t)$ for $t\in[0,T]$
we easliy derive the following result from Proposition A.1.32.

######  Corollary A.1.33.

Let $\eta:[0,T]\to\mathbb{R}$ be a nonnegative, differentiable function
satisfying

$\displaystyle-\eta^{\prime}(t)\leq\varphi(t)\eta(t)+\chi(t)\quad\text{a.e. in
}(0,T],$

where $\varphi$ and $\chi$ are real-valued, nonnegative, integrable functions
on $[{t_{\circ}},{t_{\mathsf{e}}}]$. Then,

$\displaystyle\eta(t)\leq\exp\bigg{(}\int_{t_{\circ}}^{t}\varphi(s)\,\mathrm{d}s\bigg{)}\bigg{(}\eta({t_{\mathsf{e}}})+\int_{t_{\circ}}^{t}\chi(s)\,\mathrm{d}s\bigg{)}$

In particular, if $\chi=0$ almost everywhere in $[0,T]$ and
$\eta({t_{\mathsf{e}}})=0$ hold true, we have $\eta=0$ almost everywhere in
$[0,T]$.

Recall the following wel-known inequality.

######  Lemma A.1.34 (Bernoulli Inequality).

For $x\geq-1$ and $n\in\mathbb{N}$, we have

$\displaystyle(1+x)^{n}\geq 1+nx$ (A.1.1)

###  A.2 Linear operators in Banach spaces

######  Definition A.2.35 (Banach space).

A complex vector space $X$ together with a map $\|\cdot\|_{X}:X\to\mathbb{R}$
is called a _normed space_ if

  1. 1)

$\|\varphi\|_{X}\geq 0$ for all $\varphi\in X$ and $\|\varphi\|_{X}=0$ if and
only if $\varphi=0$.

  2. 2)

$\|\alpha\varphi\|_{X}=|\alpha|\cdot\|\varphi\|_{X}$ for all $\varphi\in X$,
$\alpha\in\mathbb{C}$.

  3. 3)

$\|\varphi+\phi\|_{X}\leq\|\varphi\|_{X}+\|\phi\|_{X}$ for all
$\varphi,\phi\in X$.

The space $X$ is called a _Banach space_ if it is additionally complete,
meaning that every Cauchy sequence converges in $X$.

######  Definition A.2.36 (Hilbert space).

A complex vector space $X$ together with a map
$\langle\cdot,\cdot\rangle_{X}:X\times X\to\mathbb{C}$ is called an _inner
product space_ if

  1. 1)

$\langle\varphi,\varphi\rangle_{X}\geq 0$ for all $\varphi\in X$ and
$\langle\varphi,\varphi\rangle_{X}=0$ if and only if $\varphi=0$.

  2. 2)

The map $\langle\cdot,\varphi\rangle_{X}:X\to\mathbb{C}$ is linear for all
$\varphi\in X$.

  3. 3)

$\langle\varphi,\phi\rangle_{X}=\overline{\langle\phi,\varphi\rangle}_{X}$ for
all $\varphi,\phi\in X$.

The space $X$ is then also a normed space with the induced norm
$\|\varphi\|_{X}=\sqrt{\langle\varphi,\varphi\rangle_{X}}$ for all $x\in X$.
$X$ is called a _Hilbert space_ if it is additionally complete. A subset
$\mathscr{S}\subset X$ is called _orthogonal_ if it holds
$\langle\phi,\psi\rangle_{X}=0$ for all $\phi,\psi\in\mathscr{S}$ with
$\phi\neq\psi$. It is called _orthonormal_ if we additionally have
$\|\phi\|_{X}=1$ for all $\phi\in\mathscr{S}$. Lastly, $\mathscr{S}$ is called
an _orthonormal basis_ of $X$ if it is orthonormal and its linear hull is
dense in $X$. The orthogonal complement of $\mathscr{S}$ is defined as

$\displaystyle\mathscr{S}^{\bot}=\big{\\{}\varphi\in
X\,\big{|}\,{\langle\varphi,\phi\rangle}_{X}=0\text{ for all
}\phi\in\mathscr{S}\big{\\}}.$

The space $X$ is called _separable_ if it has a countable orthonormal basis
$\\{\psi_{i}\\}_{i\in\mathbb{I}}\subset X$. Here we have
$\mathbb{I}=\\{1,...,n\\}$ (if $X$ has the finite dimension $n\in\mathbb{N}$)
or $\mathbb{I}=\mathbb{N}$ (if $X$ has infinite dimension).

######  Example A.2.37.

For any $n\in\mathbb{N}$ the spaces $\mathbb{R}^{n}$ and $\mathbb{C}^{n}$
endowed with the respective inner products

$\displaystyle{\langle x,y\rangle}_{\mathbb{R}^{n}}$
$\displaystyle=x^{\top}y=\sum_{i=1}^{n}x_{i}y_{i}\quad\text{ for all
}x,y\in\mathbb{R}^{n}$ $\displaystyle\langle x,y\rangle_{\mathbb{C}^{n}}$
$\displaystyle:=x^{\top}\bar{y}=\sum_{i=1}^{n}x_{i}\bar{y}_{i}\quad\text{ for
all }x,y\in\mathbb{C}^{n}$

are Hilbert spaces. In both cases the induced norm is just given by the
Euclidean norm. We refer to these inner products as the canonical inner
product in $\mathbb{R}^{n}$ and $\mathbb{C}^{n}$, respectively.
$\blacklozenge$

If $X$ is separable with orthonormal basis $\\{\psi_{i}\\}_{i\in\mathbb{I}}$,
it can be shown [RS80] that

$\displaystyle\varphi=\sum_{i\in\mathbb{I}}\langle\varphi,\psi_{i}\rangle_{X}\psi_{i}\quad\text{and}\quad{\|\varphi\|}_{X}^{2}=\sum_{i\in\mathbb{I}}|\langle\varphi,\psi_{i}\rangle_{X}|^{2}\qquad\text{for
all }\varphi\in X.$

The following result is proven in [RS80, p. 43].

######  Theorem A.2.38 (Riesz isomorphism).

Let $X$ be a Hilbert space with inner product $\langle\cdot,\cdot\rangle_{X}$.
Then the mapping

$\displaystyle\mathcal{J}_{X}:X\to
X^{\prime},\quad{\langle\mathcal{J}_{X}x_{1},x_{2}\rangle}_{X^{\prime},X}={\langle
x_{1},x_{2}\rangle}_{X}$ (A.2.2)

is an isometric isomorphism called the _Riesz isomorphism_.

######  Theorem A.2.39 (Projection theorem).

Let $X$ be a Hilbert space and $M\subset X$ a nonempty and closed subspace.
Then the space $X$ can be written as the direct sum $X=M\oplus M^{\perp}$,
i.e., for every $\varphi\in X$, there is a unique decomposition
$\varphi=\varphi_{M}+\varphi_{\perp}$ with $\varphi_{M}\in M$ and
$\varphi_{\perp}\in M^{\perp}$. Furthermore, $\varphi_{M}$ is the unique
solution to the approximation problem

$\displaystyle\min\limits_{\phi\in M}~{}~{}\|\varphi-\phi\|_{X}.$ (A.2.3)

Proof. The fact that $X=M\oplus M^{\perp}$ is a standard result and is proven,
e.g., in [RS80, Theorem II.3]. Let now $\varphi=\varphi_{M}+\varphi_{\perp}\in
X$ with $\varphi_{M}\in M$ and $\varphi_{\perp}\in M^{\perp}$ and $\phi\in M$
be arbitrarily given. It follows that

$\displaystyle\|\varphi-\phi\|_{X}^{2}=\|\varphi_{M}-\phi\|_{X}^{2}+\|\varphi_{\perp}\|_{X}^{2}\geq\|\varphi_{\perp}\|_{X}^{2}=\|\varphi-\varphi_{M}\|_{X}^{2}.$

Therefore, $\varphi_{M}$ solves (A.2.3); cf. Figure A.2.1. $\Box$

$M$$\varphi$$\varphi_{M}$$\varphi_{\perp}$ Figure A.2.1: Visualization of the
Theorem A.2.39.

######  Lemma A.2.40 (Cauchy-Schwarz inequality, [RS80]).

Let $X$ be a Hilbert space with inner product $\langle\cdot,\cdot\rangle_{X}$.
Then it holds

$\displaystyle\big{|}{\langle\varphi,\phi\rangle}_{X}\big{|}\leq{\|\varphi\|}_{X}{\|\phi\|}_{X}\qquad\text{for
all }\varphi,\phi\in X.$

######  Lemma A.2.41.

Let $X$ be an inner product space with inner product
$\langle\cdot,\cdot\rangle_{X}$. Then it holds

$\displaystyle{\langle\varphi-\phi,\phi\rangle}_{X}=-\frac{1}{2}\,{\|\varphi-\phi\|}_{X}^{2}+\frac{1}{2}\,{\|\varphi\|}_{X}^{2}-\frac{1}{2}\,{\|\phi\|}_{X}^{2}\quad\text{for
all }\varphi,\phi\in X$ (A.2.4)

Proof. We have

$\displaystyle{\langle\varphi-\phi,\phi\rangle}_{X}+\frac{1}{2}\,{\|\varphi-\phi\|}_{X}^{2}+\frac{1}{2}\,{\|\phi\|}_{X}^{2}$
$\displaystyle=\frac{1}{2}\,{\langle\varphi-\phi,\varphi+\phi\rangle}_{X}+\frac{1}{2}\,{\|\phi\|}_{X}^{2}=\frac{1}{2}\,{\|\varphi\|}_{X}^{2}$

where we have utilized the fact that $\langle\cdot\,,\cdot\rangle_{X}$ is
bilinear. $\Box$

######  Definition A.2.42 (Linear operators in Banach spaces).

Let $X$, $Y$ be two complex Banach spaces. The operator $\mathcal{T}:X\to X$
is called a linear, bounded operator if these conditions are satisfied:

  1. 1)

$\mathcal{T}(\alpha_{1}\varphi+\beta\psi)=\alpha\mathcal{T}\varphi+\beta\mathcal{T}\phi$
for all $\alpha,\beta\in\mathbb{C}$ and $\varphi,\,\phi\in X$.

  2. 2)

There exists a constant $c>0$ such that $\|\mathcal{T}\varphi\|_{Y}\leq
c\,\|\varphi\|_{X}$ for all $\varphi\in X$.

The set of all linear, bounded operators from $X$ to $Y$ is denoted by
$\mathscr{L}(X,Y)$ which is a Banach space equipped with the operator norm
[RS80, pp. 69-70]

$\displaystyle{\|\mathcal{T}\|}_{\mathscr{L}(X,Y)}=\sup_{\|\varphi\|_{X}=1}{\|\mathcal{T}\varphi\|}_{Y}\quad\text{for
}\mathcal{T}\in\mathscr{L}(X,Y).$

If $X=Y$ holds, we briefly write $\mathscr{L}(X)$ instead of
$\mathscr{L}(X,X)$. For $\mathcal{T}\in\mathscr{L}(X,Y)$ the subspace

$\displaystyle\mathrm{ran}\,\mathcal{T}=\mathcal{T}(X)\subset Y$

is called the range of $\mathcal{T}$. The subspace

$\displaystyle\mathrm{ker}\,\mathcal{T}=\\{\varphi\in
X\,|\,\mathcal{T}\varphi=0\\}\subset X$

is said to be the kernel or null space of $\mathcal{T}$.

######  Definition A.2.43 (Dual operators in Banach spaces).

Let $X,Y$ be two complex Banach spaces and $\mathcal{T}\in\mathscr{L}(X,Y)$.
The dual operator $\mathcal{T}^{\prime}:Y^{\prime}\to X^{\prime}$ of a linear
operator $\mathcal{T}\in\mathscr{L}(X,Y)$ is defined as

$\displaystyle{\langle\mathcal{T}^{\prime}f,\varphi\rangle}_{X^{\prime},X}={\langle
f,\mathcal{T}\varphi\rangle}_{Y^{\prime},Y}\quad\text{for all }(\varphi,f)\in
X\times Y^{\prime},$

where, for instance, $\langle\cdot\,,\cdot\rangle_{X^{\prime},X}$ denotes the
dual pairing of the space $X$ with its dual space
$X^{\prime}=\mathscr{L}(X,\mathbb{C})$.

######  Definition A.2.44 (Adjoint operators in Hilbert spaces).

Let $X$ and $Y$ denote two complex Hilbert spaces. For a given
$\mathcal{T}\in\mathscr{L}(X,Y)$ the adjoint operator
$\mathcal{T}^{\star}:Y\to X$ is uniquely defined by

$\displaystyle{\langle\mathcal{T}^{\star}\phi,\varphi\rangle}_{X}={\langle\phi,\mathcal{T}\varphi\rangle}_{Y}=\overline{\langle\mathcal{T}\varphi,\phi\rangle}_{Y}\quad\text{for
all }(\varphi,\phi)\in X\times Y.$

With the Riesz isomorphism, we can draw a connection between the dual operator
from Definition A.2.43 and the adjoint operator from Definition A.2.44. This
result is proven e.g. in [Trö10, p. 186].

######  Theorem A.2.45.

Let $X$ and $Y$ be two complex Hilbert spaces and $\mathcal{J}_{X}:X\to
X^{\prime}$, $\mathcal{J}_{Y}:Y\to Y^{\prime}$ denote the Riesz isomorphisms
from Theorem A.2.38. For an operator $\mathcal{T}\in\mathscr{L}(X,Y)$, it then
holds

$\displaystyle\mathcal{T}^{\star}=\mathcal{J}_{X}^{-1}\mathcal{T}^{\prime}\mathcal{J}_{Y}.$

Moreover, $(\mathcal{T}^{\star})^{\star}=\mathcal{T}$.

######  Definition A.2.46.

Let $X,Y$ be two complex Hilbert spaces and $\mathcal{T}\in\mathscr{L}(X,Y)$.

  1. 1)

If $X=Y$, the operator $\mathcal{T}$ is called _self-adjoint_ if
$\mathcal{T}^{\star}=\mathcal{T}$.

  2. 2)

If $\mathcal{T}$ is self-adjoint, we call it _nonnegative_ if
$\langle\mathcal{T}\varphi,\varphi\rangle_{X}\geq 0$ for all $\varphi\in X$.

  3. 3)

$\mathcal{T}$ is called _compact_ if for every bounded sequence
$\\{\varphi_{n}\\}_{n\in\mathbb{N}}\subset X$, the sequence
$\\{\mathcal{T}\varphi_{n}\\}_{n\in\mathbb{N}}\subset Y$ contains a convergent
subsequence.

######  Remark A.2.47.

Let $X$ be a complex Hilbert space. If $\mathcal{T}\in\mathscr{L}(X)$ is self-
adjoint it is easy to show that each eigenvalue $\lambda$ of $\mathcal{T}$
fulfils $\lambda\in\mathbb{R}$. Additionally, if $\mathcal{T}$ is even
nonnegative, it holds $\lambda\geq 0$ for each eigenvalue $\lambda$ of
$\mathcal{T}$. $\blacksquare$

###  A.3 Spectral theory of linear operators

######  Definition A.3.48 (Spectrum and resolvent).

Let $X$ be a complex Hilbert space and $\mathcal{T}\in\mathscr{L}(X)$.

  1. 1)

A complex number $\lambda\in\mathbb{C}$ belongs to the _resolvent set_
$\rho(\mathcal{T})$ if $\lambda\mathcal{I}-\mathcal{T}$ is a bijection with a
bounded inverse. Here, $\mathcal{I}\in\mathscr{L}(X)$ stands for the identity
operator in $X$. We define the _spectrum_ of $\mathcal{T}$ as
$\sigma(\mathcal{T})=\mathbb{C}\setminus\rho(\mathcal{T})$.

  2. 2)

For a spectral point $\lambda\in\mathbb{C}$, if
$\lambda\mathcal{I}-\mathcal{T}$ is not injective, $\lambda$ is said that to
belong to the _point spectrum_ of $\mathcal{T}$. In this case, a vector
$\varphi\in X\setminus\\{0\\}$ with $\mathcal{T}\varphi=\lambda\varphi$ is
called an _eigenvector_ of $\mathcal{T}$.

To summarize, the complex numbers are separated into
$\mathbb{C}=\sigma(\mathcal{T})\cup\rho(\mathcal{T})$ and part of the spectrum
$\sigma(\mathcal{T})$ is made up of the point spectrum.

The following two theorems are essential for the spectral analysis of compact
operators and can be found in [RS80, p. 203].

######  Theorem A.3.49 (Riesz-Schauder).

Let $X$ be a complex Hilbert space and $\mathcal{T}:X\to X$ a linear, compact
operator. Then the spectrum $\sigma(\mathcal{T})$ is a discrete set having no
limit points except perhaps $0$. Furthermore, every nonzero
$\lambda\in\sigma(\mathcal{T})$ belongs to the point spectrum of $\mathcal{T}$
and the space of eigenvectors to $\lambda$ is finite dimensional.

######  Theorem A.3.50 (Hilbert-Schmidt).

Let $X$ be a complex separable Hilbert space and
$\mathcal{T}\in\mathscr{L}(X)$ a compact and selfadjoint operator. Then there
is a sequence of real eigenvalues $\\{\lambda_{i}\\}_{i\in\mathbb{I}}$ and
corresponding eigenvectors $\\{\psi_{i}\\}_{i\in\mathbb{I}}\subset X$ that
form a complete orthonormal basis of $X$. If $X$ has infinite dimension, we
further get $\lambda_{i}\to 0$ as $i\to\infty$.

The following result comes from [Kat95, III-§6.4].

######  Theorem A.3.51.

Let $X$ be a Banach space, $\mathcal{T}\in\mathscr{L}(X)$ and
$\Sigma\subset\sigma(\mathcal{T})$ a part of the spectrum such that there
exists a closed curve $\Gamma\subset\rho(\mathcal{T})$ with $\Sigma$ on the
inside and $\Sigma^{\prime}:=\sigma(\mathbb{C})\setminus\Sigma$ on the
outside. Then there exists a decomposition $X=M\oplus M^{\prime}$ which
satisfies

  1. 1)

$\mathcal{T}(M)\subset M$ and $\mathcal{T}(M^{\prime})\subset M^{\prime}$.

  2. 2)

The spectra of the restricted operators $\mathcal{T}|_{M}\in\mathscr{L}(M)$
and $\mathcal{T}|_{M^{\prime}}\in\mathscr{L}(M^{\prime})$ coincide with
$\Sigma$ respectively $\Sigma^{\prime}$.

######  Corollary A.3.52.

Let $X$ be a Banach space, $\mathcal{T}\in\mathscr{L}(X)$ be compact and self-
adjoint and $\Sigma:=\\{\lambda\\}$ with $\lambda$ being a nonzero eigenvalue
of $\mathcal{T}$. Then in the decomposition $X=M\oplus M^{\prime}$ from
Theorem A.3.51, $M$ is given by the space of eigenvectors to $\lambda$.

Proof. First of all, the closed curve $\Gamma\subset\rho(\mathcal{T})$ may be
chosen as a circle around $\lambda$. Since $\lambda$ is nonzero, Theorem
A.3.49 tells us that it will be the only spectral point inside $\Gamma$ if the
radius is chosen small enough. Therefore, Theorem A.3.51 is applicable.
Furthermore, it was shown in Section III-§6.5 of [Kat95] that the operator
$D_{\lambda}:=\lambda\mathcal{I}_{M}-\mathcal{T}|_{M}$ is quasi-nilpotent.
Because $\mathcal{T}$ is compact, Theorem A.3.50 tells us that $M$ is finite-
dimensional, so $D_{\lambda}$ is even nilpotent. However, the only self-
adjoint and nilpotent operator is zero which yields
$\lambda\mathcal{I}_{M}=\mathcal{T}|_{M}$. In other words, every nonzero
vector in $M$ is an eigenvector to $\lambda$. On the other hand, if $x\in X$
is an arbitrary eigenvector of $\mathcal{T}$, it can be written as
$x=x_{1}+x_{2}$ with $x_{1}\in M$, $x_{2}\in M^{\prime}$. Since $M,M^{\prime}$
are $\mathcal{T}$-invariant by Theorem A.3.51, it follows that
$\mathcal{T}x_{i}=\lambda x_{i}$ for $i=1,2$. Because the spectrum of
$\mathcal{T}|_{M^{\prime}}$ does not contain $\lambda$, we infer $x_{2}=0$, so
$x=x_{1}\in M$. $\Box$

######  Remark A.3.53.

In [Kat95, IV-§3.5] it is stated that the previous claim also holds true for a
finite system of eigenvalues, i.e., if $\mathcal{T}\in\mathscr{L}(X)$ is
compact and self-adjoint and $\Sigma=\\{\lambda_{1},\ldots,\lambda_{n}\\}$ is
a set of $n$ distinct nonzero eigenvalues of $\mathcal{T}$, then in the
decomposition $X=M\oplus M^{\prime}$ from Theorem A.3.51, $M$ is given by the
space of eigenvectors to $\lambda_{1},\ldots,\lambda_{n}$. $\blacksquare$

The following result is taken from [Kat95, IV-§3.4].

######  Theorem A.3.54.

Let $X$ be a Banach space, $\mathcal{T}\in\mathscr{L}(X)$ and
$\Sigma,\Sigma^{\prime}\subset\sigma(\mathcal{T})$ be parts of the spectrum
separated by a closed curve $\Gamma\in\mathbb{C}\setminus\sigma(\mathcal{T})$
that encloses $\Sigma$. Let further $X=M(\mathcal{T})\oplus
M^{\prime}(\mathcal{T})$ be the decomposition from Theorem A.3.51. Then there
exists a $\delta>0$ depending on $\mathcal{T}$ and $\Gamma$ with the following
properties:

  1. 1)

Any $\mathcal{S}\in\mathscr{L}(X)$ with
$\|\mathcal{S}-\mathcal{T}\|_{\mathscr{L}(X)}<\delta$ has its spectrum
$\sigma(\mathcal{S})$ likewise split into two parts $\Sigma(\mathcal{S})$,
$\Sigma^{\prime}(\mathcal{S})$ separated by $\Gamma$. It holds
$\Gamma\subset\rho(\mathcal{S})$.

  2. 2)

In the associated decomposition $X=M(\mathcal{S})\oplus
M^{\prime}(\mathcal{S})$ from Theorem A.3.51, $M(\mathcal{S})$ and
$M^{\prime}(\mathcal{S})$ are isomorphic to $M(\mathcal{T})$ and
$M^{\prime}(\mathcal{T})$, respectively.

  3. 3)

Let $P(\mathcal{T})$, $P(\mathcal{S})\in\mathscr{L}(X)$ denote the projections
from $X$ to $M(\mathcal{T})$ and $M(\mathcal{S})$, respectively. Then it holds
$P(\mathcal{S})\to P(\mathcal{T})$ in $\mathscr{L}(X)$ as
$\mathcal{S}\to\mathcal{T}$ in $\mathscr{L}(X)$.

###  A.4 Integration and differentiation in Banach spaces

######  Definition A.4.55.

Let $X$ be a Banach space and $\mathcal{D}\subset\mathbb{R}^{\mathfrak{p}}$. A
function $f:\mathcal{D}\to X$ is called _measurable_ if there exists a
sequence $\\{f_{n}\\}_{n\in\mathbb{N}}$ of step functions
$f_{n}:\mathcal{D}\to X$ such that $f_{n}\to f$ almost everywhere. For
$p\in[1,\infty)$, the $p$-th _Bochner space_ is defined as

$\displaystyle L^{p}(\mathcal{D};X)=\big{\\{}f:\mathcal{D}\to
X~{}\big{|}~{}f\text{ is measurable and
}\int_{\mathcal{D}}\|f({\bm{x}})\|_{X}^{p}~{}\mathrm{d}{\bm{x}}<\infty\big{\\}}$

Endowed with the norm

$\displaystyle\|f\|_{p}:=\|f\|_{L^{p}(\mathcal{D};X)}:=\bigg{(}\int_{\mathcal{D}}\|f({\bm{x}})\|_{X}^{p}~{}\mathrm{d}{\bm{x}}\bigg{)}^{1/p}\quad\text{
for all }f\in L^{p}(\mathcal{D};X),$

The space $L^{p}(\mathcal{D};X)$ is a Banach space. If $X=\mathbb{R}$ holds,
we write briefly $L^{p}(\mathcal{D})$ instead of
$L^{p}(\mathcal{D};\mathbb{R})$. If $X$ together with the inner product
$\langle\cdot,\cdot\rangle_{X}$ is even a Hilbert space,
$L^{2}(\mathcal{D};X)$ is also a Hilbert space _[AE01]_ endowed with the inner
product

$\displaystyle{\langle
f,g\rangle}_{L^{2}(\mathcal{D};X)}=\int_{\mathcal{D}}{\langle
f({\bm{x}}),g({\bm{x}})\rangle}_{X}~{}\mathrm{d}{\bm{x}}\qquad\text{for all
}f,g\in L^{2}(\mathcal{D};X).$

Furthermore, the essential Bochner supremum is defined for a measurable
funnction $f:\mathcal{D}\to X$ as

$\displaystyle{\|f\|}_{\infty}={\|f\|}_{L^{\infty}(\mathcal{D};X)}:=\inf\big{\\{}C\geq
0~{}\big{|}~{}{\|f({\bm{x}})\|}_{X}\leq C\text{ for almost all
}{\bm{x}}\in\mathcal{D}\big{\\}}.$

Finally, the Bochner space of essentially bounded fuctions consists of all
measurable functions $f:\mathcal{D}\to X$ with $\|f\|_{\infty}<\infty$.

######  Theorem A.4.56 (Hölder inequality, [AE01]).

Let $X$ be a Banach space and $\mathcal{D}\subset\mathbb{R}^{\mathfrak{p}}$.
Then it holds

$\displaystyle\int_{\mathcal{D}}\|f({\bm{x}})\|_{X}\cdot\|g({\bm{x}})\|_{X}~{}\mathrm{d}{\bm{x}}~{}\leq~{}\|f\|_{L^{2}(\mathcal{D};X)}\cdot\|g\|_{L^{2}(\mathcal{D};X)}\quad\text{for
all }f,g\in L^{2}(\mathcal{D};X).$

In the following definition, we use the symbol
$\mathbb{N}_{0}=\mathbb{N}\cup\\{0\\}$. An element
$\alpha\in\mathbb{N}_{0}^{\mathfrak{p}}$ is called a _multi-index_ of order
$|\alpha|=\sum_{i=1}^{\mathfrak{p}}\alpha_{i}$. For a function
$\varphi:\mathcal{D}\to\mathbb{R}$ that is $|\alpha|$-times classically
differentiable, we use the short notation

$\displaystyle\partial^{\alpha}\varphi({\bm{x}})=\left(\frac{\partial}{\partial
x_{1}}\right)^{\alpha_{1}}\ldots\left(\frac{\partial}{\partial
x_{\mathfrak{p}}}\right)^{\alpha_{\mathfrak{p}}}\varphi({\bm{x}})\quad\text{for
}x\in\mathcal{D}.$

######  Definition A.4.57 (Sobolev space).

Let $X$ be a Banach space, $\mathcal{D}\subset\mathbb{R}^{\mathfrak{p}}$ open
and $f\in L^{2}(\mathcal{D};X)$. For a multi-index
$\alpha\in\mathbb{N}^{\mathfrak{p}}_{0}$, a function $f_{\alpha}\in
L^{2}(\mathcal{D};X)$ is called the _weak $\alpha$-derivative_ of $f$ if

$\displaystyle\int_{\mathcal{D}}f({\bm{x}})\cdot\partial^{\alpha}\varphi({\bm{x}})~{}\mathrm{d}{\bm{x}}=(-1)^{|\alpha|}\int_{\mathcal{D}}f_{\alpha}({\bm{x}})\cdot\varphi({\bm{x}})~{}\mathrm{d}{\bm{x}}\quad\text{for
all }\varphi\in C_{0}^{\infty}(\mathcal{D})$

Here, the space $C_{0}^{\infty}(\mathcal{D})$ is the space of all infinitely
differentiable functions $\varphi:\mathcal{D}\to\mathbb{R}$ such that the
support $\text{supp
}\varphi=\overline{\\{{\bm{x}}\in\mathcal{D}~{}|~{}\varphi({\bm{x}})\neq
0\\}}$ is compact in $\mathbb{R}^{\mathfrak{p}}$. If $f$ has such a weak
$\alpha$-derivative, we indicate this by writing $\partial^{\alpha}f\in
L^{2}(\mathcal{D};X)$ and identify $\partial^{\alpha}f$ with $f_{\alpha}$. For
$k\in\mathbb{N}$, we define the _Sobolev space_

$\displaystyle H^{k}(\mathcal{D};X)=\big{\\{}f\in
L^{2}(\mathcal{D};X)~{}\big{|}~{}\partial^{\alpha}f\in
L^{2}(\mathcal{D};X)\text{ for }|\alpha|\leq k\big{\\}}.$

If $X$ is is a Hilbert space, $H^{1}(\mathcal{D};X)$ is also a Hilbert space
[DL00, Eva08] endowed with the inner product

$\displaystyle{\langle f,g\rangle}_{H^{1}(\mathcal{D};X)}={\langle
f,g\rangle}_{L^{2}(\mathcal{D};X)}+\sum_{i=1}^{\mathfrak{p}}{\langle
f_{x_{i}},g_{x_{i}}\rangle}_{L^{2}(\mathcal{D};X)}\quad\text{for }f,g\in
H^{1}(\mathcal{D};X).$

Here, we have used the notion of weak partial derivatives
$f_{x_{i}}=\tfrac{\partial}{\partial x_{i}}f=\partial^{\alpha}f$ with
$\alpha_{i}=1$ and $\alpha_{j}=0$ for $j\neq i$.

######  Definition A.4.58 (Gelfand triple).

Let $V$ and $H$ be real, separable Hilbert spaces such that $V$ lies dense in
$H$ with compact embedding. We identify $H$ with its dual space $H^{\prime}$
so that we have the Gelfand triple

$\displaystyle V\hookrightarrow H\simeq H^{\prime}\hookrightarrow V^{\prime}$

where the last embedding is understood as follows: For every element
$h^{\prime}\in H^{\prime}$ and $v\in V$, we also have $v\in H$ by the first
embedding, so we can define $\langle
h^{\prime},v\rangle_{V^{\prime},V}=\langle
h^{\prime},v\rangle_{H^{\prime},H}$.

Let now $T>0$ and the Hilbert spaces $V$, $H$ form a Gelfand triple. Then we
define the space

$W(0,T)=L^{2}(0,T;V)\cap H^{1}(0,T;V^{\prime})=\big{\\{}\varphi\in
L^{2}(0,T;V)\,\big{|}\,\varphi_{t}\in L^{2}(0,T;V^{\prime})\big{\\}}$ (A.4.5)

The following properties of $W(0T)$ are proven e.g. in [DL00, pp. 472-479] and
[Trö10, pp. 146-148].

######  Lemma A.4.59.

Let $V,H$ form a Gelfand triple and $T>0$.

  1. 1)

$W(0,T)$ is a Hilbert space endowed with the inner product

$\displaystyle{\langle\varphi,\phi\rangle}_{W(0,T)}=\int_{0}^{T}{\langle\varphi(t),\phi(t)\rangle}_{V}+{\langle\varphi_{t}(t),\phi_{t}(t)\rangle}_{V^{\prime}}\,\mathrm{d}t\quad\text{for
}\varphi,\phi\in W(0,T)$

and the induced norm
$\|\varphi\|_{W(0,T)}=\langle\varphi,\varphi\rangle_{W(0,T)}^{1/2}$.

  2. 2)

The space $W(0,T)$ is continuously embedded into the space $C([0,T];H)$, the
space of all continuous mappings from $[0,T]$ to $H$. Thus, there exists a
constant $C_{W}>0$ such that

${\|\varphi\|}_{C([0,T];H)}\leq C_{W}\,{\|\varphi\|}_{W(0,T)}\quad\text{for
all }\varphi\in W(0,T)$ (A.4.6)

In particular, $\varphi(t)$, $t\in[0,T]$, is meaningful in $H$ for an element
$\varphi\in W(0,T)$.

  3. 3)

The integration by parts formula reads for all $\varphi,\phi\in W(0,T)$

$\displaystyle\int_{0}^{T}{\langle\varphi_{t}(t),\phi(t)\rangle}_{V^{\prime},V}\,\mathrm{d}t+\int_{0}^{T}{\langle\phi_{t}(t),\varphi(t)\rangle}_{V^{\prime},V}\,\mathrm{d}t$
(A.4.7)
$\displaystyle\quad=\int_{0}^{T}\frac{\mathrm{d}}{\mathrm{d}t}{\langle\varphi(t),\phi(t)\rangle}_{H}\,\mathrm{d}t={\langle\varphi({t_{\mathsf{e}}}),\phi({t_{\mathsf{e}}})\rangle}_{H}-{\langle\varphi({t_{\circ}}),\phi({t_{\circ}})\rangle}_{H}$

  4. 4)

We have the formula

$\displaystyle{\langle\varphi_{t}(t),\phi\rangle}_{V^{\prime},V}=\frac{\mathrm{d}}{\mathrm{d}t}\,{\langle\varphi(t),\phi\rangle}_{H}\quad\text{for
}(\varphi,\phi)\in W(0,T)\times V,\text{ f.a.a. }t\in[0,T].$

######  Definition A.4.60 (Continuous functions).

Let $X,Y$ be Banach spaces and $\mathcal{O}\subset X$ an open subset. Then we
denote by

$\displaystyle C(\mathcal{O};Y):=\\{f:\mathcal{O}\subset X\to Y\mid f\text{ is
continuous}\\}$

the set of all continuous functions from $\mathcal{O}$ to $Y$.

######  Definition A.4.61 (Gâteaux and Fréchet derivatives).

Let $X,Y$ be real Banach spaces, $\mathcal{O}\subset X$ an open subset and
$f:\mathcal{O}\to Y$ a given mapping.

  1. 1)

For a point $x\in\mathcal{O}$ and a given direction $x^{\delta}\in X$, the
_directional derivative_ of $x$ towards $x^{\delta}$ is defined by

$\displaystyle Df(x,x^{\delta}):=\lim_{\varepsilon\searrow
0}\frac{1}{\varepsilon}\big{(}f(x+\varepsilon x^{\delta})-f(x)\big{)}$

provided the limit exists in $Y$.

  2. 2)

If the directional derivative exists for every direction $x^{\delta}\in X$ and
we have $Df(x,\cdot)\in\mathscr{L}(X,Y)$, then $f$ is said to be _Gâteaux
differentiable_ at $x$ with _Gâteaux derivative_ $f^{\prime}(x)=Df(x,\cdot)$.
If $f$ is Gâteaux differentiable in every $x\in\mathcal{O}$, we say that $f$
is Gâteaux differentiable (in $\mathcal{O}$).

  3. 3)

If $f$ is Gâteaux differentiable in $x\in\mathcal{O}$ and it holds

$\displaystyle\frac{\|f(x+x^{\delta})-f(x)-f^{\prime}(x)x^{\delta}\|_{Y}}{\|x^{\delta}\|_{X}}\to
0\quad\text{as}\quad x^{\delta}\to 0\text{ in }X,$

then $f$ is said to be _Fréchet differentiable_ in $x$. If $f$ is Fréchet
differentiable in every $x\in\mathcal{O}$, we call $f$ Fréchet differentiable
(in $\mathcal{O}$). In that case, if the mapping

$\displaystyle f^{\prime}:\mathcal{O}\to L(X,Y),\quad x\mapsto f^{\prime}(x)$
(A.4.8)

is continuous, $f$ is called _continuously Fréchet differentiable_ (in
$\mathcal{O}$).

######  Definition A.4.62 (Second-order Fréchet derivatives).

Let $X,Y$ be real Banach spaces, $\mathcal{O}\subset X$ an open subset and
$f:\mathcal{O}\to Y$ Fréchet differentiable. Then we call $f$ _twice Fréchet
differentiable_ in $x\in\mathcal{O}$ if the function $f^{\prime}$ from (A.4.8)
is Fréchet differentiable in $x$, i.e. if a mapping $f^{\prime\prime}(x)\in
L(X,L(X,Y))$ exists with

$\displaystyle\frac{\|f^{\prime}(x+x^{\delta})-f^{\prime}(x)-f^{\prime\prime}(x)x^{\delta}\|_{L(X,Y)}}{\|x\|_{X}}\to
0\quad\text{as}\quad x^{\delta}\to 0\text{ in }X.$

In that case we call $f^{\prime\prime}(x)$ the _second Fréchet derivative_ of
$f$ in $x$. If $f$ is twice Fréchet differentiable in every $x\in\mathcal{O}$,
$f$ is called twice Fréchet differentiable (in $\mathcal{O}$). If the mapping

$\displaystyle f^{\prime\prime}:\mathcal{O}\to L(X,L(X,Y)),\quad x\mapsto
f^{\prime\prime}(x)$

is continuous, $f$ is said to be _twice continuously Fréchet differentiable_
(in $\mathcal{O}$).

######  Remark A.4.63.

For two real Banach spaces $X$ and $Y$, a function $A:X\times X\to Y$ is said
to be bilinear if for every $x\in X$, the functions $A(x,\cdot)$ and
$A(\cdot,x)$ are linear mappings from $X$ to $Y$. It is said to be bounded if
there exists a constant $c>0$ such that

$\displaystyle{\|A(x_{1},x_{2})\|}_{Y}\leq
c\,{\|x_{1}\|}_{X}{\|x_{2}\|}_{X}\quad\text{for all }x_{1},x_{2}\in X$

If we denote the space of bilinear and bounded functions with
$\mathscr{L}_{2}(X,Y)$ and endow it with the norm

$\displaystyle{\|A\|}_{\mathscr{L}_{2}(X,Y)}=\sup\big{\\{}\|A(x_{1},x_{2})\|_{Y}~{}\big{|}~{}{\|x_{1}\|}_{X}={\|x_{2}\|}_{X}=1\big{\\}},$

it can be shown (see, e.g., [Ger06]) that it is a Banach space and
isometrically isomorph to the space $\mathscr{L}(X,\mathscr{L}(X,Y))$.
Therefore, we can identify the second Fréchet derivative with a bilinear,
continuous function $f^{\prime\prime}(x)\in\mathscr{L}_{2}(X,Y)$. Schwarz’s
theorem [Ger06] states that $f^{\prime\prime}(x)$ is always symmetric, so we
have
$f^{\prime\prime}(x)(x^{\delta}_{1},x^{\delta}_{2})=f^{\prime\prime}(x)(x^{\delta}_{2},x^{\delta}_{1})$
for all directions $x^{\delta}_{1},x^{\delta}_{2}\in X$. $\blacksquare$

######  Definition A.4.64 (Gradient and Hessian).

Let $X$ be a real Hilbert space, $\mathcal{O}\subset X$ open and
$f:\mathcal{O}\to\mathbb{R}$ Fréchet differentiable in $x\in\mathcal{O}$. With
the Riesz isomorphism $\mathcal{J}_{X}:X\to X^{\prime}$, the _Gradient_ of $f$
at $x$ is defined as $\nabla f(x)=\mathcal{J}_{X}^{-1}f^{\prime}(x)\in X$ and
satisfies

$\displaystyle{\langle\nabla
f(x),x^{\delta}\rangle}_{X}=f^{\prime}(x)x^{\delta}\quad\text{for all
}x^{\delta}\in X.$

If $f$ is twice differentiable in $x$, the _Hessian_ of $f$ in $x$ is defined
as a mapping

$\displaystyle\nabla^{2}f(x):X\to
X,\quad\nabla^{2}f(x)x^{\delta}=\mathcal{J}_{X}^{-1}f^{\prime\prime}(x)x^{\delta}.$

If we understand $f^{\prime\prime}(x)$ as a bilinear function as mentioned in
Remark A.4.63, it holds by definition of the Riesz isomorphism

$\displaystyle{\langle\nabla^{2}f(x)x^{\delta}_{1},x^{\delta}_{2}\rangle}_{X}=f^{\prime\prime}(x)(x^{\delta}_{1},x^{\delta}_{2})\qquad\text{for
all }x^{\delta}_{1},x^{\delta}_{2}\in X.$

Since the Riesz isomorphism is an isometry, it holds $\|\nabla
f(x)\|_{X}=\|f^{\prime}(x)\|_{X^{\prime}}$ and
$\nabla^{2}f(x)\in\mathscr{L}(X)$ with
$\|\nabla^{2}f(x)\|_{\mathscr{L}(X)}=\|f^{\prime\prime}(x)\|_{\mathscr{L}_{2}(X)}$.

## Bibliography

  * [ABK01] J.A. Atwell, J.T. Borggaard, and B.B. King.  Reduced-order controllers for burgers’ equation with a nonlinear observer.  International Journal of Applied Mathematics and Computer Science, 11:1311–1330, 2001.
  * [AE01] H. Amman and J. Escher.  Analysis III.  Birkhäuser, Basel, 2001.
  * [AFS00] E. Arian, M. Fahl, and E.W. Sachs.  Trust-region proper orthogonal decomposition for flow control.  Technical report, Technical Report 2000-25, ICASE, 2000.
  * [AH02] K. Afanasiev and M. Hinze.  Adaptive control of a wake flow using proper orthogonal decomposition.  Lecture Notes in Pure and Applied Mathematics, 216:317–332, 2002\.
  * [Alt92] H.W. Alt.  Lineare Funktionalanalysis. Eine anwendungsorientierte Einführung.  Springer, Berin, 1992.
  * [Ant05] A.C. Antoulas.  Approximation of Large-Scale Dynamical Systems.  Advances in Design and Control. SIAM, Philadelphia, PA 19104, 2005.
  * [AV15] A. Alla and S. Volkwein.  Asymptotic stability of pod based model predictive control for a semilinear parabolic PDE.  Advances in Computational Mathematics, 41:1073–1102, 2015.
  * [AWWB08] A. Astrid, S. Weiland, K. Willcox, and T. Backx.  Missing point estimation in models described by proper orthogonal decomposition.  IEEE Transactions on Automatic Control, 53:2237–2251, 2008.
  * [Ban17] S. Banholzer.  POD-based bicriterial optimal control of convection-diffusion equations.  Master’s thesis, Department of Mathematics and Statistics, University of Konstanz, Germany, 2017.
  * [BBV16] S. Banholzer, D. Beermann, and S. Volkwein.  POD-based bicriterial optimal control by the reference point method.  IFAC-PapersOnLine, 49:210–215, 2016.
  * [BBV17] S. Banholzer, D. Beermann, and S. Volkwein.  POD-based error control for reduced-order bicriterial PDE-constrained optimization.  Annual Reviews in Control, 44:226–237, 2017.
  * [BJWW00] H.T. Banks, M.L. Joyner, B. Winchesky, and W.P. Winfree.  Nondestructive evaluation using a reduced-order computational methodology.  Inverse Problems, 16:1–17, 2000.
  * [BMV19] S. Banholzer, E. Makarov, and S. Volkwein.  POD-based multiobjective optimal control of time-variant heat phenomena.  Lecture Notes in Computational Science and Engineering, 126:881–888, 2019.
  * [BNP04] M. Barrault, M. Maday, N.C. Nguyen, and A.T. Patera.  An ’empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations.  Comptes Rendus Mathematique, 339:667–672, 2004.
  * [BS08] S.C. Brenner and L.R. Scott.  The Mathematical Theory of Finite Element Methods.  Texts in Applied Mathematics. Springer, New York, Berlin, Paris, 2008\.
  * [BS12] A. Borzì and V. Schulz.  Computational Optimization of Systems Governed by Partial Differential Equations.  Computational Sciences and Engineering. SIAM, Philadelphia, 2012.
  * [BSV14] P. Benner, E. Sachs, and S. Volkwein.  Model order reduction for pde constrained optimization.  International Series of Numerical Mathematics, 165:303–326, 2014\.
  * [CGSM12] D. Chapelle, A. Gariah, and J. Saint-Marie.  Galerkin approximation with proper orthogonal decomposition: new error estimates and illustrative examples.  ESAIM: Mathematical Modelling and Numerical Analysis, 46:731–757, 2012.
  * [Cha00] A. Chatterjee.  An introduction to the proper orthogonal decomposition.  Current Science, 78:539–575, 2000.
  * [Cha11] F. Chatelin.  Spectral Approximation of Linear Operators.  Classics in Applied Mathematics. SIAM, Philadelphia, 2011.
  * [CS10a] S. Chaturantabut and D.C. Sorensen.  Application of POD and DEIM on a dimension reduction of nonlinear miscible viscous fingering in porous media.  Mathematical and Computer Modelling of Dynamical Systems, 17:337–353, 2010.
  * [CS10b] S. Chaturantabut and D.C. Sorensen.  Nonlinear model reduction via discrete empirical interpolation.  SIAM Journal on Scientific Computing, 32:2737–2764, 2010.
  * [CS12] S. Chaturantabut and D.C. Sorensen.  A state space estimate for pod-deim nonlinear model reduction.  SIAM Journal on Numerical Analysis, 50:46–63, 2012.
  * [DH13] M. Dihlmann, , and B. Haasdonk.  Certified nonlinear parameter optimization with reduced basis surrogate models.  PAMM, 13:3–6, 2013.
  * [DHPY95] A.L. Dontchev, W.W. Hager, A.B. Poore, and B. Yang.  Optimality, stability, and convergence in nonlinear control.  Applied Mathematics & Optimization, 31:297–326, 1995.
  * [DL00] R. Dautray and J.-L. Lions.  Mathematical Analysis and Numerical Methods for Science and Technology. Volume 5: Evolution Problems I.  Springer, Berlin, 2000.
  * [DV01] F. Diwoky and S. Volkwein.  Nonlinear boundary control for the heat equation utilizing proper orthogonal decomposition.  International Series of Numerical Mathematics, 138:73–87, 2001\.
  * [Ehr05] M. Ehrgott.  Multicriteria Optimization.  Springer, Berlin, Heidelberg, 2005.
  * [Eva08] L.C. Evans.  Partial Differential Equations.  Graduate Studies in Mathematics. American Mathematical Society, Providence, Rhode Island, 2008.
  * [FHC06] K.F. Fong, V.I. Hanby, and T.-T. Chow.  HVAC system optimization for energy management by evolutionary programming.  Energy and Buildings, 38:220–231, 2006.
  * [Ger06] C. Gerhardt.  Analysis 2.  International Press of Boston, Inc., Somerville, Massachusetts, 2006.
  * [GGV15] E. Grimm, M. Gubisch, and S. Volkwein.  Numerical analysis of optimality-system POD for constrained optimal control.  Lecture Notes in Computational Science and Engineering, 105:297–317, 2015.
  * [GHV21] C. Gräßle, M. Hinze, and S. Volkwein.  Model order reduction by proper orthogonal decomposition, pages 47–96.  De Gruyter, Berlin, Boston, 2021.
  * [GMNP07] M. Grepl, Y. Maday, N.C. Nguyen, and A.T. Patera.  Efficient reduced-basis treatment of nonaffine and nonlinear partial differential equations.  ESAIM: Mathematical Modelling and Numerical Analysis, 41:575–605, 2007.
  * [GNV17] M. Gubisch, I. Neitzel, and S. Volkwein.  A-posteriori error estimation of discrete POD models for PDE-constrained optimal control, pages 213–234.  Springer International Publishing, Cham, 2017.
  * [GP16] L. Grüne and J. Pannek.  Nonlinear Model Predictive Control: Theory and Algorithms.  Communications and Control Engineering. Springer, London, 2 edition, 2016\.
  * [Grä19] C. Gräßle.  Adaptivity in Model Order Reduction with Proper Orthogonal Decomposition.  PhD thesis, University of Hamburg, 2019.
  * [Gre12] M. Grepl.  Certified reduced basis method for nonaffine time-varying and nonlinear parabolic partial differential equations.  Mathematical Models and Methods in Applied Science, 22:1150015, 2012\.
  * [Gri13] E. Grimm.  Optimality system POD and a-posteriori error analysis for linear-quadratic optimal control problems.  Master’s thesis, Department of Mathematics and Statistics, University of Konstanz, Germany, 2013.
  * [GT77] D. Gilbarg and N.S. Trudinger.  Elliptic Differential Equations of Second Order.  Springer, Berlin, 1977.
  * [GU14] J. Ghiglieri and S. Ulbrich.  Optimal flow control based on POD and MPC and an application to the cancellation of Tollmien-Schlichting waves.  Optimization and Software, 29:1042–1074, 2014.
  * [Gub16] M. Gubisch.  Model Order Reduction Techniques for the Optimal Control of Parabolic Partial Differential Equations with Control and State Constraints.  PhD thesis, University of Konstanz, Germany, 2016.
  * [GV17] M. Gubisch and S. Volkwein.  Chapter 1: Proper Orthogonal Decomposition for Linear-Quadratic Optimal Control, pages 3–63.  Society for Industrial and Applied Mathematics, Philadelphia, PA, 2017\.
  * [Haa12] B. Haasdonk.  Convergence rates of the POD-greedy method.  ESAIM: Mathematical Modelling and Numerical Analysis, 47:859–873, 2012.
  * [HDPV06] A. Hepberger, F. Diwoky, H. Priebsch, and S. Volkwein.  Impedance identification out of pressure data’s with a hybrid measurement-simulation methodology up to 1 khz.  In Proceedings of International Conference on Noise and Vibration Engineering, Leuven, Belgium, 2006.
  * [HIK03] M. Hintermüller, K. Ito, and K. Kunisch.  The primal-dual active set strategy as a semismooth Newton method.  SIAM Journal on Optimization, 13:865–888, 2003.
  * [HKV09] M. Hintermüller, I. Kopacka, and S. Volkwein.  Mesh-independence and preconditioning for solving control problems with mixed control-state constraints.  ESAIM: Control, Optimisation and Calculus of Variations, 15:626–652, 2009.
  * [HLBR12] P. Holmes, J.L. Lumley, G. Berkooz, and C.W. Rowley.  Turbulence, Coherent Structures, Dynamical Systems and Symmetry.  Cambridge Monographs on Mechanics, Cambridge University Press, 2 edition, 2012.
  * [HO08] B. Haasdonk and M. Ohlberger.  Reduced basis method for finite volume approximations of parametrized linear evolution equations.  ESAIM: Mathematical Modelling and Numerical Analysis, 42:277–302, 2008.
  * [HO14] C. Himpe and M. Ohlberger.  Cross-gramian-based combined state and parameter reduction for large-scale control systems.  Mathematical Problems in Engineering, 2014:1–13, 2014.
  * [HPUU09] M. Hinze, R. Pinnau, M. Ulbrich, and S. Ulbrich.  Optimization with PDE Constraints.  Mathematical Modelling: Theory and Applications. Springer, Dordrecht, 2009\.
  * [HRS16] J.S. Hesthaven, G. Rozza, and B. Stamm.  Certified Reduced Basis Methods for Parametrized Partial Differential Equations.  SpringerBriefs in Mathematics. Springer, Cham, 2016.
  * [HSS08] M. Heinkenschloss, D.C. Sorensen, and K. Sun.  Balanced truncation model reduction for a class of descriptor systems with application to the Oseen equations.  SIAM Journal on Scientific Computing, 30:1038–1063, 2008.
  * [HV05] M. Hinze and S. Volkwein.  Proper orthogonal decomposition surrogate models for nonlinear dynamical systems: error estimates and suboptimal control.  Lecture Notes in Computational Science and Engineering, 45:261–306, 2005.
  * [HV08] M. Hinze and S. Volkwein.  Error estimates for abstract linear-quadratic optimal control problems using proper orthogonal decomposition.  Computational Optimization and Applications, 39:319–345, 2008.
  * [IK08] K. Ito and K. Kunisch.  Lagrange Multiplier Approach to Variational Problems and Applications.  Advances in Design and Control. SIAM, Philadelphia, 2008.
  * [IR98] K. Ito and S.S. Ravindran.  A reduced basis method for control problems governed by PDEs.  In W. Desch, F. Kappel, and K. Kunisch, editors, Control and Estimation of Distributed Parameter Systems, pages 153–168, Basel, 1998. Birkhäuser Basel.
  * [ITV16] L. Iapichino, S. Trenz, and S. Volkwein.  Reduced-order multiobjective optimal control of semilinear parabolic problems.  Lecture Notes in Computational Science and Engineering, 112:389–397, 2016.
  * [IUV17] L. Iapichino, S. Ulbrich, and S. Volkwein.  Multiobjective pde-constrained optimization using the reduced-basis method.  Advances in Computational Mathematics, 43:945–972, 2017.
  * [Kat95] T. Kato.  Perturbation Theory for Linear Operators.  Classics in Mathematics. Springer, Berlin, Heidelberg, 2 edition, 1995\.
  * [KR09] K. Krumbiegel and A. Rösch.  A virtual control concept for state constrained optimal control problems.  Computational Optimization and Application, 43:213–233, 2009.
  * [KTV13] E. Kammann, F. Tröltsch, and S. Volkwein.  A posteriori error estimation for semilinear parabolic optimal control problems with application to model reduction by POD.  ESAIM: Mathematical Modelling and Numerical Analysis, 47:555–581, 2013.
  * [KTX11] A. Kusiak, F. Tang, and G. Xu.  Multi-objective optimization of HVAC system with an evolutionary computation algorithm.  Energy, 36:2440–2449, 2011.
  * [KV01] K. Kunisch and S. Volkwein.  Galerkin proper orthogonal decomposition methods for parabolic problems.  Numerische Mathematik, 90:117–148, 2001.
  * [KV02a] K. Kunisch and S. Volkwein.  Crank-nicolson galerkin proper orthogonal decomposition approximations for a general equation in fluid dynamics.  In Proceedings of the 18th GAMM Seminar on Multigrid and related methods for optimization problems, Leipzig, pages 97–114, 2002.
  * [KV02b] K. Kunisch and S. Volkwein.  Galerkin proper orthogonal decomposition methods for a general equation in fluid dynamics.  SIAM Journal on Numerical Analysis, 40:492–515, 2002.
  * [KV07] M. Kahlbacher and S. Volkwein.  Galerkin proper orthogonal decomposition methods for parameter dependent elliptic systems.  Discussiones Mathematicae: Differential Inclusions, Control and Optimization, 27:95–117, 2007.
  * [KV08] K. Kunisch and S. Volkwein.  Proper orthogonal decomposition for optimality systems.  ESAIM: Mathematical Modelling and Numerical Analysis, 42:1–23, 2008\.
  * [KV09] M. Kahlbacher and S. Volkwein.  Estimation of regularization parameters in elliptic optimal control problems by POD model reduction.  In A. Korytowski, K. Malanowski, W. Mitkowski, and M. Szymkat, editors, System Modeling and Optimization, pages 307–318, Berlin, Heidelberg, 2009. Springer.
  * [KV10] K. Kunisch and S. Volkwein.  Optimal snapshot location for computing pod basis functions.  ESAIM: Mathematical Modelling and Numerical Analysis, 44:509–529, 2010.
  * [KV12] M. Kahlbacher and S. Volkwein.  Pod a-posteriori error based inexact sqp method for bilinear elliptic optimal control problems.  ESAIM: Mathematical Modelling and Numerical Analysis, 46:491–511, 2012.
  * [KVX04] K. Kunisch, S. Volkwein, and L. Xie.  HJB-POD based feedback design for the optimal control of evolution problems.  SIAM Journal on Applied Dynamical Systems, 3:701–722, 2004.
  * [Lio71] J.L. Lions.  Optimal Control of Systems Governed by Partial Differential Equation.  Grundlehren der mathematischen Wissenschaften. Springer, Berlin, Heidelberg, 1971.
  * [LMG02] S. Lall, J.E. Marsden, and S. Glavaski.  A subspace approach to balanced truncation for model reduction of nonlinear control systems.  International Journal of Robust and Nonlinear Control, 12:519–535, 2002.
  * [LT01] H.V. Ly and H.T. Tran.  Modeling and control of physical processes using proper orthogonal decomposition.  Mathematical and Computer Modeling, 33:223–236, 2001.
  * [Lue69] D.G. Luenberger.  Optimization by Vector Space Methods.  John Wiley & Sons, New York, 1969.
  * [LV06] F. Leibfritz and S. Volkwein.  Reduced order output feedback control design for PDE systems using proper orthogonal decomposition and nonlinear semidefinite programming.  Linear Algebra and Its Applications, 415:542–575, 2006.
  * [LV14] O. Lass and S. Volkwein.  Adaptive pod basis computation for parametrized nonlinear systems using optimal snapshot location.  Computational Optimization and Applications, 58:645–677, 2014.
  * [Mak18] E. Makarov.  Pod-based bicriterial optimal control of time-dependent convection-diffusion equations with basis update.  Master’s thesis, Department of Mathematics and Statistics, University of Konstanz, Germany, 2018.
  * [Mec19] L. Mechelli.  POD-based state-constrained economic model predictive control of convection-diffusion phenomena.  PhD thesis, Department of Mathematics and Statistics, University of Konstanz, Germany, 2019.
  * [Mie98] K. Miettinen.  Nonlinear Multiobjective Optimization.  International Series in Operations Research & Management Science. Springer, New York, NY, 1998.
  * [MS05] V. Mehrmann and T. Stykel.  Balanced truncation model reduction for large-scale systems in descriptor form.  Lecture Notes in Computational Science and Engineering, 45:83–115, 2005.
  * [MV18] L. Mechelli and S. Volkwein.  POD-based economic optimal control of heat-convection phenomena, pages 63–87.  Springer International Publishing, Cham, 2018.
  * [MV19] L. Mechelli and S. Volkwein.  POD-based economic model predictive control for heat-convection phenomena.  In F.A. Radu, K. Kumar, I. Berre, J.M. Nordbotten, and I.S. Pop, editors, Numerical Mathematics and Advanced Applications ENUMATH 2017, pages 663–671, Cham, 2019. Springer International Publishing.
  * [MZ79] H. Maurer and J. Zowe.  First and second order necessary and sufficient optimality conditions for infinite-dimensional programming problems.  Mathematical Programming, 16:98–110, 1979.
  * [NAM+03] B. Noack, K. Afanasiev, M. Morzynski, G. Tadmor, and F. Thiele.  A hierarchy of low-dimensional models for the transient and post-transient cylinder wake.  Journal of Fluid Mechanics, 497:335–363, 2003.
  * [NRMQ13] F. Negri, G. Rozza, A. Manzoni, and A. Quateroni.  Reduced basis method for parametrized elliptic optimal control problems.  SIAM Journal on Scientific Computing, 35:A2316–A2340, 2013.
  * [NW06] J. Nocedal and S.J. Wright.  Numerical Optimization.  Springer Series in Operations Research and Financial Engineering. Springer, New York, NY, 2 edition, 2006.
  * [OS13] M. Ohlberger and M. Schaefer.  Error control based model reduction for parameter optimization of elliptic homogenization problems.  IFAC Proceedings Volumes, 46:251–256, 2013.
  * [Paz83] A Pazy.  Semigroups of Linear Operators and Applications to Partial Differential Equations.  Applied Mathematical Sciences. Springer, New York, NY, 1983.
  * [Pir09] O. Pironneau.  Calibration of options on a reduced basis.  Journal of Computational and Applied Mathematics, 232:139–147, 2009\.
  * [POBD16] S. Peitz, S. Ober-Blöbaum, and M. Dellnitz.  Reduced order model based multiobjective optimal control of fluids.  In Proceedings of International Congress of Theoretical and Applied Mechanics, 2016.
  * [PR05] A.T. Patera and G. Rozza.  Reduced basis approximation and a posteriori error estimation for parametrized partial differential equations.  Massachusetts Institute of Technology, Department of Mechanical Engineering, 2005.
  * [QMN16] A. Quarteroni, A. Manoni, and F. Negri.  Reduced Basis Methods for Partial Differential Equations.  UNITEXT. Springer, Cham, 2016.
  * [Rav00] S.S. Ravindran.  Reduced-order adaptive controllers for fluid flows using POD.  SIAM Journal on Scientific Computing, 15:457–478, 2000.
  * [RBW+09] C. Romaus, J. Bocker, K. Witting, A. Seifried, and O. Znamenshchykov.  Optimal energy management for a hybrid energy storage system combining batteries and double layer capacitors.  In 2009 IEEE Energy Conversion Congress and Exposition, pages 1640–1647, 2009.
  * [RM09] J.B. Rawlings and D.Q. Mayne.  Model Predictive Control: Theory and Design.  Nob Hill Publishing, Madison, Wisconsin, 2009.
  * [Row05] C.W. Rowley.  Model reduction for fluids, using balanced proper orthogonal decomposition.  International Journal on Bifurcation and Chaos, 15:997–1013, 2005\.
  * [RP02] M. Rathinam and L. Petzold.  Dynamic iteration using reduced order models: a method for simulation of large scale modular systems.  SIAM Journal on Numerical Analysis, 40:1446–1474, 2002.
  * [RS80] M. Reed and B. Simon.  Methods of Modern Mathematical Physics I: Functional Analysis.  Methods of Modern Mathematical Physics. Academic Press, New York, 1980\.
  * [RTV17] S. Rogg, S. Trenz, and S. Volkwein.  Trust-region POD using a-posteriori error estimation for semilinear parabolic optimal control problems.  Technical report, Technical Report, Konstanzer Schriften in Mathematik, No. 359, 2017.
  * [Sch13] M. Schu.  Adaptive Trust-Region POD Methods and their Application in Finance.  PhD thesis, University of Trier, Germany, 2013.
  * [Sin14] J.R. Singler.  New POD error expressions, error bounds, and asymptotic results for reduced order models of parabolic PDEs.  SIAM Journal on Numerical Analysis, 52:852–876, 2014.
  * [Sir87] L. Sirovich.  Turbulence and the dynamics of coherent structures. Parts I-II.  Quarterly of Applied Mathematics, XVL:561–590, 1987.
  * [Spu19] F. Spura.  Reduced-order bicriterial optimal control of evolution equations.  Master’s thesis, Department of Mathematics and Statistics, University of Konstanz, Germany, 2019.
  * [SS05] E.W. Sachs and M. Schu.  A-priori error estimates for reduced order models in finance.  Mathematical Modelling and Numerical Analysis, 47:449–469, 2005\.
  * [Sta88] W. Stadler.  Multicriteria Optimization in Engineering and in the Sciences.  Mathematical Concepts and Methods in Science and Engineering. Springer, New York, NY, 1988.
  * [Str04] J.C. Strickwerda.  Finite Difference Schemes and Partial Differential Equations.  SIAM, Philadelphia, PA, 2004.
  * [SV10] E.W. Sachs and S. Volkwein.  POD galerkin approximations in pde-constrained optimization.  GAMM-Mitteilungen, 33:194–208, 2010.
  * [SV13] A. Studinger and S. Volkwein.  Numerical analysis of pod a-posteriori error estimation for optimal control.  International Series of Numerical Mathematics, 164:137–158, 2013\.
  * [SvdVR08] W.H.A. Schilders, H.A. van der Vorst, and J. Rommes.  Model Order Reduction: Theory, Research Aspects and Applications.  Mathematics in Industry. Springer, Berlin, Heidelberg, 2008.
  * [TB22] L.N. Trefethen and D. Bau.  Numerical Linear Algebra.  American Mathematical Society, Philadelphia, PA, 2022.
  * [Tem79] R. Temam.  Navier-Stokes Equations.  Studies in Mathematics and its Applications. North-Holland, Amsterdam, London, 1979.
  * [Tem97] R. Temam.  Infinite-Dimensional Dynamical Systems in Mechanics and Physics.  Applied Mathematical Sciences. Springer, New York, NY, 1997.
  * [Tho06] V. Thomée.  Galerkin Finite Element Methods for Parabolic Problems.  Springer Series in Computational Mathematics. Springer, Berlin, Heidelberg, 2006.
  * [Tre17] S. Trenz.  POD-Based A-Posteriori Error Estimation for Control Problems Governed by Nonlinear PDEs.  PhD thesis, University of Konstanz, Germany, 2017.
  * [Trö10] F. Tröltzsch.  Optimal Control of Partial Differential Equations. Theory, Methods and Applications.  Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, 2010.
  * [TV09] F. Tröltzsch and S. Volkwein.  POD a-posteriori error estimates for linear-quadratic optimal control problems.  Computational Optimization and Applications, 44:83–115, 2009.
  * [VH08] S. Volkwein and A. Hepberger.  Impedance identification by pod model reduction techniques.  at-Automatisierungstechnik, 8:437–446, 2008.
  * [Vol01] S. Volkwein.  Optimal control of a phase-field model using proper orthogonal decomposition.  Zeitschrift für Angewandte Mathematik und Mechanik, 81:83–97, 2001.
  * [Vol10] S. Volkwein.  Admittance identification from point-wise sound pressure measurements using reduced-order modelling.  Journal of Optimization Theory and Applications, 147:169–193, 2010\.
  * [Vol11] S. Volkwein.  Optimality system POD and a-posteriori error analysis for linear-quadratic problems.  Control and Cybernetics, 40:1109–1125, 2011.
  * [Vol12] S. Volkwein.  Proper Orthogonal Decomposition: Theory and Reduced-Order Modelling.  Lecture Notes, University of Konstanz, 2012.
  * [Wie80] A.P. Wierzbicki.  The use of reference objectives in multiobjective optimization.  In G. Fandel and T. Gal, editors, Multiple Criteria Decision Making Theory and Application, pages 468–486, Berlin, Heidelberg, 1980. Springer.
  * [Wie86] A.P. Wierzbicki.  On the completeness and constructiveness of parametric characterizations to vector optimization problems.  OR Spectrum, 8:73–87, 1986.
  * [Wil65] J.H. Wilkinson.  The Algebraic Eigenvalue Problem.  Oxford University Press (Clarendon Press), London and New Yor, 1965.
  * [WP02] K. Willcox and J. Peraire.  Balanced model reduction via the proper orthogonal decomposition.  American Institute of Aeronautics and Astronautics, 40:2323–2330, 2002.
  * [Yos95] K. Yosida.  Functional Analysis.  Classics in Mathematics. Springer, Berlin, Heidelberg, 6 edition, 1995\.
  * [ZDG96] K. Zhou, J.C. Doyle, and K. Glover.  Robust and Optimal Control.  Prentice Hall, Upper Saddle River, New Jersey, 07458, 1996.

## Index

  * $\partial^{\alpha}$ §A.4
  * $\overline{\alpha}$ item 1)
  * a.e., almost everywhere item 2)
  * Bilinear form
    * parameter-dependent, $a({\bm{\mu}};\cdot\,,\cdot)$ §3.7.1
    * time-dependent, $a(t;\cdot\,,\cdot)$ item 3), item 3), Remark 4.1.3
    * time-independent, $a(\cdot\,,\cdot)$ item 1), §3.6.1
  * Control shape functions §1.2
  * Cost functional
    * desired states, ${y^{\mathsf{d}}_{1}}$, ${y^{\mathsf{d}}_{2}}$ §1.2
    * expected control, ${u^{\mathsf{n}}}$ §1.2
    * nominal control, ${u^{\mathsf{n}}}$ §1.2
    * quadratic §1.2, item 3), §4.1.1
    * reduced §1.2, §4.1.1
    * regularization parameter, $\gamma_{i}^{\mathsf{d}}$, $\gamma_{i}^{\mathsf{b}}$ §1.2
    * tracking weight, $\alpha_{Q}$, $\alpha_{\Omega}$ §1.2
    * tracking-type §1.2
  * Derivative
    * directional item 1)
    * Fréchet item 3)
    * Gâteaux item 2)
    * second Fréchet Definition A.4.62
  * Distance function §5.3.2
  * Domain
    * of a linear operator, $D(\mathcal{A})$ item 1)
  * Domain, spatial, $\Omega$ §1.2
  * Equation
    * heat §1.2
    * Helmholtz Example 2.1.8
      * angular frequency Example 2.1.8
      * impedance Example 2.1.8
      * sound pressure Example 2.1.8
      * wave number Example 2.1.8
    * Navier-Stokes Example 3.6.3
    * nonlinear heat §5.1.1
    * partial differential
      * diffusion-reaction Example 4.1.13
      * elliptic Example 3.7.7
      * parabolic §1.2, §3.1, Example 4.1.13
    * partial differential, PDE Chapter 1, §1.2
  * Error estimate
    * a-posteriori
      * control variable §4.3.7, Theorem 4.2.14
      * dual variable §4.3.6, Proposition 4.3.10
      * gradient §4.3.6
      * state variable §3.6.3, Theorem 3.5.12, Theorem 3.4.13, Proposition 3.6.9
    * a-priori
      * control variable Theorem 4.2.12, Theorem 4.3.8
      * dual variable Theorem 4.3.6, Theorem 4.2.8
      * state variable §3.4.5, §4.2.1, §4.3.3, Theorem 3.5.11, Theorem 3.4.12, Theorem 3.3.14, Theorem 3.6.14, Theorem 3.3.17, Theorem 3.5.3, Theorem 3.6.6, Theorem 3.4.7, Theorem 3.5.9
  * f.a.a., for almost all §2.5.3
  * Gelfand triple item 1), Remark 2.4.2, Remark 3.3.2, Definition A.4.58
  * Inequality
    * Bernoulli Lemma A.1.34, Lemma A.1.34
    * Cauchy-Schwarz Lemma A.2.40
    * embedding Remark 2.4.2
    * Gronwall Proposition A.1.32
    * Hölder Theorem A.4.56
    * Young Lemma A.1.30
  * Lagrange functional §2.1.8
  * Linear Operator Definition A.2.42
  * Linear operator
    * adjoint, $\mathcal{T}^{\star}$ Definition A.2.44
    * bounded Definition A.2.42
    * control-to-state Remark 3.3.10
    * control, $\mathcal{B}$ §3.1, item 4), item 4)
    * contrrol, $\mathcal{B}$ §3.6.1
    * domain, $D(\mathcal{A})$ item 1)
    * dual, $\mathcal{T}^{\prime}$ Definition A.2.43
    * graph norm item 1)
    * kernel, $\mathrm{ker}\,\mathcal{T}$ Definition A.2.42
    * norm Definition A.2.42
    * null space, $\mathrm{ker}\,\mathcal{T}$ Definition A.2.42
    * observation item 3)
    * POD solution, $\mathcal{S}^{\ell}$ item 2)
    * range, $\mathrm{ran}\,\mathcal{T}$ Definition A.2.42
    * Riesz isomorphism Theorem A.2.38
    * solution, $\mathcal{S}$ item 2)
  * Matrix
    * identity, ${\bm{I}}$ Proposition 2.3.6
    * mass
      * FE, ${\bm{A}}^{h}$ §3.4.1
      * FE, ${\bm{M}}$ item 2)
      * FE, ${\bm{M}}^{h}$ item 1)
      * POD, ${\bm{M}}^{\ell}$ §3.3.1
    * POD
      * mass, ${\bm{M}}^{h\ell}$ 3.4.25
      * stiffness, ${\bm{A}}^{h\ell}$ 3.4.25
    * stiffness
      * FE, ${\bm{S}}$ item 2)
      * FE, ${\bm{S}}^{h}$ item 2)
      * POD, ${\bm{A}}^{\ell}$ §3.3.1
    * weighting §3.4.3
    * weighting, ${\bm{W}}$ §2.1.9
  * Method
    * balanced truncation §1.1
    * Crank-Nicolson §2.1.10, §3.5.1
    * discrete empirical interpolation, DEIM §3.6.5
    * empirical interpolation, EIM §3.6.5
    * Euclidean reference point §5.3.2
    * explicit Euler §3.5.1
    * finite element, FE item 2), Example 2.3.2, Example 3.4.2
    * Gram-Schmidt item 2), item 2)
    * implicit Euler §3.5.1
    * optimality system POD, OS-POD §4.4
    * $\theta$-scheme §3.5.1
  * Model predictive control, MPC Remark 5.2.10
  * $\mathbb{N}_{0}$ §A.4
  * Optimal solution
    * global Definition 4.1.10
    * local Definition 4.1.10
  * Optimality conditions
    * first-irder
      * necessary Theorem 4.1.20
    * first-order
      * necessary §2.1.8
    * first-order necessary §2.1.8
    * second-order §2.1.8
      * necessary §2.1.8
      * sufficient §2.1.8, §4.1.5
  * Pareto front §5.3.1
  * Pareto set §5.3.1
  * POD method
    * continuous variant §2.4.1, §3.3, §3.7.2, Problem 2.2.1
      * methods of snapshots Remark 2.2.9
      * POD basis of rank $\ell$ Definition 2.2.2
      * snapshot space §2.2.1, §3.7.2
      * snapshots §2.2.1
      * SVD §2.4.1, Remark 2.2.9
    * discrete variant §2.4.2, §3.7.3, Problem 2.1.2
      * greedy §2.1.6
      * methods of snapshots item 1), item 1)
      * POD basis of rank $\ell$ Definition 2.1.3
      * relative total energy §2.1.5, §2.1.9
      * snapshot space §2.1.1, §2.1.6, §2.4.2
      * snapshots §2.1.1, §2.4.2
      * SVD §2.1.4, §2.1.7
    * error formula §2.1.1
    * optimality item 1)
    * optimality system, OS-POD §4.4
    * POD basis Definition 2.2.2
    * snapshot space item 2), §4.3.2
    * trust-region §4.3.6
  * POD space
    * discrete variant
      * snapshot space §3.5.3
      * snapshots §3.5.3
  * Problem
    * bicriterial optimization §5.3.1
    * multicriterial optimization §5.3
    * multiobjective optimization §5.3
    * nonlinear heat equation §5.1.1
    * nonlinear programming §5.1.3
    * optimal control §1.2
      * linear-quadratic §1.2, Remark 4.1.7
      * reduced §1.2, §4.1.1
    * quadratic programming §4.1.1
      * PDE-constrained §1.2
    * reference point §5.3.2
  * Reference point §5.3.2
  * Set
    * of admissible controls, ${{\mathscr{U}}_{\mathsf{ad}}}$ §1.2, Definition 4.1.6
    * Pareto §5.3.1
  * Singular value decomposition, SVD §2.1.4, §2.1.7, §2.4.1
  * Space
    * Banach Definition A.2.35
    * bidual §4.1.4
    * Hilbert Definition A.2.36
      * control, ${\mathscr{U}}$ §3.1, item 2), item 2)
      * inner product Definition A.2.36
      * orthogonal complement Definition A.2.36
      * orthonormal Definition A.2.36
      * orthonormal basis Definition A.2.36
      * orthoogonal Definition A.2.36
      * separable Definition A.2.36
    * normed Definition A.2.35
    * state, ${\mathscr{Y}}$ §3.2.1, §4.1.1
    * state,${\mathscr{Y}}$ §4.4
  * Theorem
    * Hilbert-Schmidt Theorem A.3.50
    * projection Theorem A.2.39
    * Riesz Theorem A.2.45
    * Riesz-Schauder Theorem A.3.49
    * trace, $\gamma_{\Gamma}$ §3.1
  * $(z_{i})_{i},~{}(Z_{ij})_{ij}$ item 2)