# Optimal control of SPDEs driven by time-space Brownian motion

Nacira Agram${}^{1},$ Bernt Øksendal2, Frank Proske2 and Olena Tymoshenko2,3

(Corrected version 31 December 2023)

###### Abstract

In this paper we study a Pontryagin type stochastic maximum principle for the
optimal control of a system, where the state dynamics satisfy a stochastic
partial differential equation (SPDE) driven by a two-parameter (time-space)
Brownian motion (also called Brownian sheet).

We first discuss some properties of a Brownian sheet driven linear SPDE which
models the growth of an ecosystem.

Further, applying time-space white noise calculus we derive sufficient
conditions and necessary conditions of optimality of the control.

Finally, we illustrate our results by solving a linear quadratic control
problem and an optimal harvesting problem in the plane. We also study possible
applications to machine learning.

11footnotetext: Department of Mathematics, KTH Royal Institute of Technology
100 44, Stockholm, Sweden.
Email<EMAIL_ADDRESS>Work supported by the Swedish Research Council grant
(2020-04697).22footnotetext: Department of Mathematics, University of Oslo,
Norway.
Emails<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Department of Mathematics, KTH Royal
Institute of Technology 100 44, Stockholm, Sweden.
Email<EMAIL_ADDRESS>Work supported by the Swedish Research Council grant
(2020-04697).22footnotetext: Department of Mathematics, University of Oslo,
Norway.
Emails<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Department of Mathematical Analysis
and Probability Theory, NTUU Igor Sikorsky Kyiv Polytechnic Institute, Kyiv,
Ukraine.

Dedicated to the memory of Yuri Kondratiev

##  1 Introduction

The purpose of this paper is to study optimal control of systems driven by the
Brownian sheet. Throughout this work, we denote by $\\{B(t,x):t\geq
0,x\in\mathbb{R}\\}$ a Brownian sheet and $(\Omega,\mathcal{F},P)$ a complete
probability space on which we define the (completed) $\sigma$-field
$\mathcal{F}_{t,x}$ generated by $B(s,a),s\leq t,a\leq x$.

Wong & Zakai [WZ] generalised the notion of stochastic integrals with respect
to 1-parameter Brownian motion to stochastic integrals driven by the two-
parameter Brownian sheet.

Let us denote by $\mathbb{R}^{2}_{+}$ the positive quadrant of the plane and
let $z\in\mathbb{R}^{2}_{+}$, We define a first type stochastic integral with
respect to the two-parameter Brownian motion in Cairoli [C72] denoted by:

$\displaystyle\int_{R_{z}}\phi(\zeta)B(d\zeta)$

and a second type [WZ74] stochastic integral denoted by

$\displaystyle\int_{R_{z}}\int_{R_{z}}\psi(\zeta,\zeta^{\prime})B(d\zeta)B(d\zeta^{\prime}),$

where $R_{z}=[0,t]\times[0,x]$ if $z=(t,x)$.

In Wong & Zakai [WZ], an Itô formula for stochastic integrals in the plane is
given.

###### Example 1.1

(An optimal harvesting problem)

A classical model for the growth of an ecosystem (e.g. a population or a
forest) with value $Y(t)$ at time $t$ in a random environment is the geometric
Brownian motion, defined by the Itô stochastic differential equation (SDE)

$dY(t)=\kappa Y(t)dt+\gamma Y(t)dB(t),\quad t\geq 0,\quad Y(0)>0,$

where $\kappa$ and $\gamma$ are given constants. Equivalently, in terms of
white noise $\overset{\bullet}{B}$ and Wick product $\diamond$, the equation
can be written

$\frac{d}{dt}Y(t)=\kappa Y(t)+\gamma Y(t)\diamond\overset{\bullet}{B}(t),\quad
Y(0)>0,$

where

$\overset{\bullet}{B}(t)=\frac{d}{dt}B(t)\text{ is (time) white noise.}$

A natural extension of this model to the case where the noise of the
environment depends on both time $t$ and position $x$, is the following SPDE
in the value $Y(t,x)$ of the ecosystem at time $t$ and position $x$:

$\frac{\partial^{2}}{\partial t\partial
x}Y(t,x)=\alpha_{0}(t,x)Y(t,x)+\beta_{0}(t,x)Y(t,x)\diamond\overset{\bullet}{B}(t,x),\quad
Y(0,0)>0,$ (1.1)

where $\alpha_{0}(t,x)$ and $\beta_{0}(t,x)$ are given bounded deterministic
functions, and

$\displaystyle\overset{\bullet}{B}(t,x)=\frac{\partial^{2}}{\partial t\partial
x}B(t,x)\text{ is time-space white noise.}$ (1.2)

Alternatively, if we use the following result:

$\int_{R_{z}}\varphi(s,a)B(ds,da)=\int_{R_{z}}\varphi(s,a)\diamond\overset{\bullet}{B}(s,a)dsda,\text{
}\quad\forall\varphi,t,x.$ (1.3)

(see e.g. Holden et al [HØUZ]), we can rewrite the equation (1.1) as:

$\displaystyle
Y(t,x)=Y(0,0)+\int_{R_{z}}\alpha_{0}(\zeta)Y(\zeta)d\zeta+\int_{R_{z}}\beta_{0}(\zeta)Y(\zeta)B(d\zeta).$
(1.4)

Assume for simplicity that $\alpha_{0}$ and $\beta_{0}$ are constants.
If we at $(t,x)$ harvest from $Y(t,x)$ at the rate $u(t,x)$, the dynamics
(1.1) becomes

$\frac{\partial^{2}}{\partial t\partial
x}Y_{u}(t,x)=\alpha_{0}Y_{u}(t,x)-u(t,x)+\beta_{0}Y_{u}(t,x)\diamond\overset{\bullet}{B}(t,x),$

or, in integral form,

$\displaystyle Y_{u}(t,x)$
$\displaystyle=Y(0,0)+\int_{0}^{t}\int_{0}^{x}\\{\alpha_{0}Y_{u}(s,a)-u(s,a)\\}dsda$
$\displaystyle+\int_{0}^{t}\int_{0}^{x}\beta_{0}Y_{u}(s,a)B(ds,da).$

For given utility functions $U_{1},U_{2}$ and given constants $T>0,X>0$ define
the combined utility of the harvesting and the terminal population by

$J(u)=E\left[\int_{0}^{T}\int_{0}^{X}U_{1}(u(s,a))dsda+U_{2}(Y_{u}(T,X))\right].$

We want to find the harvesting strategy $u^{\ast}(s,x)$ which maximizes the
utility of the harvest, i.e.

$J(u^{\ast})=\sup_{u\in\mathcal{A}}J(u).$

Here the set of admissible controls is denoted by $\mathcal{A}.$

###### Example 1.2

(A linear-quadratic (LQ) problem)
Consider the following linear-quadratic (LQ) control problem for time-space
random fields:
Suppose the state $Y(t,x)$ is given by

$\displaystyle Y(t,x)=Y(0,0)+\int_{0}^{t}\int_{0}^{x}u(s,a)dsda+\beta
B(t,x),\quad t\geq 0,x\in\mathbb{R}.$

We want to drive the state $Y$ to 0 at time-space $(T,X)$ with minimal use of
energy. Hence we put

$J(u)=-\tfrac{1}{2}E\Big{[}\int_{0}^{T}\int_{0}^{X}u^{2}(s,a)dsda+\theta
Y^{2}(T,X)\Big{]},$

where $\theta>0$ is a given constant.
The problem is to find $u^{\ast}\in\mathcal{A}$ such that

$\displaystyle J(u^{\ast})=\sup_{u\in\mathcal{A}}J(u).$ (1.5)

###### Example 1.3

(A machine learning problem)
Consider the following hyperbolic SPDE:

$Y(t,x)=y-\int_{0}^{t}\int_{0}^{x}u(s,a)\nabla f(Y(s,a))dsda+\sigma
B(t,x),y\in\mathbb{R}^{d},t,x\geq 0\text{,}$ (1.6)

where $B$ is a Brownian sheet in $\mathbb{R}^{d}$,
$\sigma\in\mathbb{R}^{d\times d}$ and $\nabla f$ is the gradient of a function
$f\in C^{1}(\mathbb{R}^{d};\mathbb{R})$. Further,
$u:\Omega\times\left[0,\infty\right)^{2}\longrightarrow\left[0,\infty\right)$
is a stochastic learning rate in time _and_ space, which is assumed to be an
adapted random field. By formally setting $u=\eta\delta_{x}$ in (1.6) for the
Dirac delta function $\delta_{x}$ in a fixed point $x$ and $\eta\geq 0$ we get
(as a special case of (1.6) the SDE

$dY_{t}=-\eta\nabla f(Y_{t})dt+\sigma dB_{t},Y_{0}=x\in\mathbb{R}^{d},t\geq
0\text{.}$ (1.7)

The latter type of SDE (1.7) is used in machine learning in connection with
the stochastic gradient descent method (SGD) to minimize or maximize the
objective or loss function $f$. Since the dynamics (1.6) is more general than
that of (1.7), one may replace for the sake of a deeper understanding of the
classical SGD approach (at the possible expense of numerical tractability) the
equation (1.7) by (1.6) and study an optimal control problem with respect to
the (time-space) stochastic learning rate $u$.

In order to illustrate this application in a simplified framework, we
approximate $\nabla f$ (for smooth $f$) by Taylor‘s expansion in the case of
$d=1$ and consider in this paper the controlled process

$\displaystyle Y_{u}(t,x)$
$\displaystyle=Y(0,0)-\int_{0}^{t}\int_{0}^{x}u(s,a)Y_{u}(s,a)dsda$
$\displaystyle+\beta_{0}B(t,x),Y(0,0),\beta_{0}\in\mathbb{R},t,x\geq 0$

with respect to (in this context) natural performance functional

$J(u)=-E\left[\int_{0}^{T}\int_{0}^{X}u^{2}(s,a)dsda+\theta Y^{2}(T,X)\right]$

for $\theta>0$.

##  2 A general formulation

The examples mentioned above are special cases of the following general
optimal stochastic control problem:We study optimal control of solutions
$Y(t,x),t\geq 0,x\in\mathbb{R}$ of SPDEs of the form

$Y_{u}(t,x)=Y(t_{0},x_{0})+\int_{R(t,x)}\alpha_{u}(Y_{u}(s,a))dsda+\int_{R(t,x)}\beta_{u}(Y_{u}(s,a))B(ds,da),$
(2.1)

where

$R(t,x)=R^{(t_{0},x_{0})}(t,x)=[t_{0},t]\times[x_{0},x],t\geq t_{0},x\geq
x_{0},$

and

$B(t,x)=\text{Brownian sheet.}$

The differential form of (2.1) is

$\frac{\partial^{2}}{\partial t\partial
x}Y_{u}(t,x)=\alpha_{u}(Y_{u}(t,x))+\beta_{u}(Y_{u}(t,x))\diamond\overset{\bullet}{B}(t,x).$
(2.2)

The identity of (2.1) and (2.2) comes from the fact that

$\int_{R(t,x)}\varphi(s,a)B(ds,da)=\int_{R(t,x)}\varphi(s,a)\diamond\overset{\bullet}{B}(s,a)dsda,\text{
}\quad\forall\varphi,t,x.$ (2.3)

See e.g. Holden et al [HØUZ].

###### Remark 2.1

Hyperbolic SPDEs of the type (1.1) have been studied over the years by several
authors. See e.g. Cairoli [C72] and Yeh [Y], who established strong existence
and pathwise uniqueness of solutions $Y$ of the equation

$\displaystyle Y(t,x)$ $\displaystyle=y_{0}+\int_{R(t,x)}b(s,a,Y(s,a))dsda$
$\displaystyle+\int_{R(t,x)}\sigma(s,a,Y(s,a))B(ds,da),y_{0}\in\mathbb{R}^{d},$
(2.4)

when $b$ and $\sigma$ are Lipschitz continuous vector fields of linear growth.

Further, smoothness of solutions to (2.4) in the sense of Malliavin
differentiability for sufficiently regular $b$ and $\sigma$ was analysed in
Nualart & Sanz [NS]. See also Bogso et al [BDMPP], where the authors construct
Malliavin differentiable unique solutions to (2.4), when the drift vector
field $b$ is merely bounded and measurable and $\sigma$ is given by the unit
matrix. As for other works in this direction (in the case of both weak and
strong solutions), we also refer to Yeh [Y87] and [Y85].
In addition we refer to the book of Nualart [N].
Finally, we want to point out the interesting link between hyperbolic SPDEs
and non-linear (random) wave equations, when $d=1$ and $b$,
$\sigma:\mathbb{R}\mapsto\mathbb{R}$:

By applying the orthogonal transformation $u=x+t,v=x-t$ to the SPDE (2.2) we
see that the corresponding differential version of equation (2.4) can be
transformed into the following non-linear stochastic wave equation:

$\frac{\partial^{2}}{\partial t^{2}}Y(t,x)-\frac{\partial^{2}}{\partial
x^{2}}Y(t,x)=\sigma(Y(t,x))\frac{\partial^{2}}{\partial t\partial
x}\widetilde{B}(t,x)+b(Y(t,x)),$

where $\widetilde{B}$ is another Brownian sheet, obtained by applying the
inverse orthogonal transformation to $B$. See e.g. Walsh [W] for further
details.

In the sequel we assume that the performance functional $J_{u}(t_{0},x_{0})$
has the form

$\small{J_{u}(t_{0},x_{0})=E^{(t_{0},x_{0})}\left[\int_{t_{0}}^{T}\int_{x_{0}}^{X}f_{u}(Y_{u}(s,a))dsda+g(Y_{u}(T,X))\right],}$
(2.5)

and consider the following problem:

###### Problem 2.2

Let $\mathcal{A}$ be a given family of admissible controls. Find an optimal
control $u^{\ast}\in\mathcal{A}$ and the value function $\Phi$ such that

$\Phi(t_{0},x_{0})=J_{u^{\ast}}(t_{0},x_{0})=\sup_{u\in\mathcal{A}}J_{u}(t_{0},x_{0}).$
(2.6)

The control $u^{\ast}$ is called an optimal control and the function $\Phi$ is
referred to as the value function of this problem.

We handle those problems by using a maximum principle approach. Therefore we
need to study the adjoint equation given by a BSPDE in the plane. The
existence and the uniqueness of solutions to such a BSPDE were proven in Zaidi
& Nualart [ZN] for a particular Lipschitz constant.

Here is an outline of the rest of the paper:

  * •

In the next section we check properties of the solution of SPDE.

  * •

In Section 3 we introduce some background about stochastic calculus of time-
space white noise.

  * •

In Section 4 we prove two types of maximum principles, a sufficient
(verification theorem) and a necessary maximum principle.

  * •

Finally, in Section 5 we apply the results of Section 4 to study the problems
mentioned above.

##  3 A discussion of the solution of the SPDE

It is natural to ask what are the properties the solution $Y(t,x)$ of the
equation

$\displaystyle
Y(t,x)=Y(0,0)+\int_{R_{z}}\alpha_{0}(\zeta)Y(\zeta)d\zeta+\int_{R_{z}}\beta_{0}(\zeta)Y(\zeta)B(d\zeta).$
(3.1)

For example, is $Y(t,x)$ always positive?

To study this question, let us suppose that the coefficients
$\alpha_{0},\beta_{0}$ are bounded deterministic functions. Then it follows
from (1.1) that

$\widetilde{Y(t,x)}(z)=Y(0,0)+\int_{R_{t,x}}\widetilde{K(s,a)}(z)\widetilde{Y(s,v)}(z)dsdv\text{,}$

where $\widetilde{Y(t,x)}(z)$ denotes the Hermite transform of $Y(t,x)$ for
$z\in\left(\mathbb{C}^{\mathbb{N}}\right)_{c}$ (set of all finite sequences in
$\mathbb{C}^{\mathbb{N}}$) and where
$R_{(t,x)}=\left[0,t\right]\times\left[0,x\right]$. Further,

$K(s,a)=\alpha_{0}(s,a)+\beta_{0}(s,a)\overset{\bullet}{B}(s,a),$

where

$\overset{\bullet}{B}(s,a)=\frac{\partial^{2}}{\partial s\partial
a}B(s,a)\text{ is }\text{white noise.}$ (3.2)

See [HØUZ] for the properties of the Hermite transform.

Then, using Picard iteration, we find with $y_{0}=Y(0,0)>0$, the semi-explicit
solution

$\displaystyle\widetilde{Y(t,x)}(z)$
$\displaystyle=y_{0}\sum_{n=0}^{\infty}\int_{R_{(t,x)}}\int_{R_{(s_{1},a_{1})}}...\int_{R_{(s_{n-1},a_{n-1})}}\prod_{j=1}^{n}\widetilde{K(s_{j},a_{j})}(z)ds_{n}da_{n}...ds_{1}da_{1},$
(3.3)

for $z\in\left(\mathbb{C}^{\mathbb{N}}\right)_{c}$, where $s_{1}>...>s_{n}$,
$a_{1}>...>a_{n}$.

By Theorem 2.11.4 in Holden et al [HØUZ] $Y(t,x)$ is positive if and only if
for all $m$ the function

$g(y):=\widetilde{Y(t,x)}(iy)e^{-\tfrac{1}{2}|y|^{2}};\quad
i=\sqrt{-1},y=(y_{1},y_{2},...,y_{m})\in\mathbb{R}^{m}$ (3.4)

is positive definite.

In this context, note that

$\displaystyle\widetilde{K(s,a)}(iy)$
$\displaystyle=\alpha_{0}(s,a)+\beta_{0}(s,a)\widetilde{\overset{\bullet}{B}(s,a)}(iy)$
$\displaystyle=\alpha_{0}(s,a)+\beta_{0}(s,a)\sum_{k=1}^{m}\mu_{k}(s,a)iy_{k}.$
(3.5)

Here $\mu_{k}(s,a)$ is the $k$’th element in an orthonormal basis of
$L^{2}(\mathbb{R}^{2})$ consisting of tensor products of Hermite functions.
See Holden et al [HØUZ], Section 2.2.1 and 2.2.2.

Combining the above we get

$\displaystyle\widetilde{Y(t,x)}(iy)$ (3.6) $\displaystyle=$ $\displaystyle
y_{0}\sum_{n=0}^{\infty}\int_{R(t,x)}\int_{R(s_{1},a_{1})}...\int_{R(s_{n-1},a_{n-1})}$
$\displaystyle\prod_{j=1}^{n}\left(\alpha_{0}(s_{j},a_{j})+i\beta_{0}(s_{j},a_{j})\sum_{k=1}^{m}\mu_{k}(s_{j},a_{j})y_{k}\right)ds_{n}da_{n}...ds_{1}da_{1}\text{.}$

Therefore the positivity question is equivalent to the following:

_Is for all $m=1,2.,...$ the function $g:\mathbb{R}^{m}\mapsto\mathbb{C}$
given by_

$\displaystyle
g(y):=y_{0}\sum_{n=0}^{\infty}\int_{R(t,x)}\int_{R(s_{1},a_{1})}...\int_{R(s_{n-1},a_{n-1})}$
$\displaystyle\prod_{j=1}^{n}\left(\alpha_{0}(s_{j},a_{j})+i\beta_{0}(s_{j},a_{j})\sum_{k=1}^{m}\mu_{k}(s_{j},a_{j})y_{k}\right)ds_{n}da_{n}...ds_{1}da_{1}e^{-\tfrac{1}{2}|y|^{2}}$

_positive definite_?__

It turns out that the latter is not true, in general. In what follows, we want
to give an explanation for this in the case of $\alpha_{0}=0$ and $\beta_{0}$
given by

$\beta_{0}(s,a)=\beta_{1}(s)\beta_{2}(a)\text{,}$

where $\beta_{1}$ and $\beta_{2}$ are bounded measurable functions. Assume
that $m=1$. We also note that we can write

$\mu_{1}(s,a)=\xi_{1}(s)\xi_{2}(a)$

with elements $\xi_{1}$ and $\xi_{2}$ of an orthonormal basis of
$L^{2}(\mathbb{R})$.

In this case, we obtain the representation

$\displaystyle\widetilde{Y(t,x)}(iy_{1})$ (3.7) $\displaystyle=$
$\displaystyle
y_{0}\sum_{n=0}^{\infty}\int_{R(t,x)}\int_{R(s_{1},a_{1})}...\int_{R(s_{n-1},a_{n-1})}\prod_{j=1}^{n}\left(i\beta_{0}(s_{j},a_{j})\mu_{1}(s_{j},a_{j})y_{1}\right)ds_{n}da_{n}...ds_{1}da_{1}$
$\displaystyle=$ $\displaystyle
y_{0}\sum_{n=0}^{\infty}\frac{1}{(n!)^{2}}\left(i\eta(t,x)y_{1}\right)^{n}\text{,}$

where $\eta(t,x):=\int_{R(t,x)}\beta_{0}(s,a)\mu_{1}(s,a)dsda$. Let us now
have a look at the expression

$i^{k}\left(\eta(t,x)y_{1}\right)^{k}\exp(-\frac{1}{2}\left|y_{1}\right|^{2}).$

Then for a standard normally distributed random variables $Z$, the inverse
Fourier transform of the latter is given by

$\displaystyle
i^{k}\frac{1}{(\sqrt{2\pi})^{1/2}}\int_{\mathbb{R}}\left(\eta(t,x)y_{1}\right)^{k}\exp(-\frac{1}{2}\left|y_{1}\right|^{2})\exp(iu_{1}y_{1})dy$
$\displaystyle=$ $\displaystyle
i^{k}E\left[\left(\eta(t,x)Z\right)^{k}\exp(iu_{1}Z)\right]$ $\displaystyle=$
$\displaystyle\left(\eta(t,x)\right)^{k}\frac{\partial^{k}}{\partial
u_{1}^{k}}\varphi_{Z}(u_{1})=\left(\eta(t,x)\right)^{k}\frac{\partial^{k}}{\partial
u_{1}^{k}}(\exp(-\frac{1}{2}u_{1}^{2}))$ $\displaystyle=$
$\displaystyle\left(\eta(t,x)\right)^{k}(-1)^{k}h_{k}(u_{1})\exp(-\frac{1}{2}u_{1}^{2})\text{,}$

where $\varphi_{Z}$ denotes the characteristic function of $Z$ and where
$h_{k}$ is the $k$’th Hermite polynomial.
So, using dominated convergence, the inverse Fourier transform of the function
$g$ (for $m=1$) is

$b(u_{1})\exp(-\frac{1}{2}u_{1}^{2})\text{,}$ (3.8)

where

$b(u_{1}):=y_{0}\sum_{n=0}^{\infty}\frac{1}{(n!)^{2}}\left(\eta(t,x)\right)^{n}(-1)^{n}h_{n}(u_{1})\text{.}$

Let us now show that the function $b$ cannot be non-negative. For this purpose
let us recall the following properties of Hermite polynomials:

$h_{n}(x+y)=\sum_{k=0}^{n}\frac{n!}{(n-k)!k!}x^{n-k}h_{k}(y)$

and

$h_{k}\left(\int_{\mathbb{R}^{2}}\phi(s,a)B(ds,da)\right)=\left(\int_{\mathbb{R}^{2}}\phi(s,a)B(ds,da)\right)^{\diamond
k}\text{,}$

where $\diamond$ denotes the Wick product and where $\phi\in
L^{2}(\mathbb{R}^{2})$ with $\left\|\phi\right\|_{L^{2}(\mathbb{R}^{2})}=1$
(see [HØUZ]).

So we obtain that

$\displaystyle h_{n}\left(c+\int_{\mathbb{R}^{2}}\phi(s,a)B(ds,da)\right)$
$\displaystyle=\sum_{k=0}^{n}\frac{n!}{(n-k)!k!}c^{n-k}h_{k}\left(\int_{\mathbb{R}^{2}}\phi(s,a)B(ds,da)\right)$
$\displaystyle=\left(c+\int_{\mathbb{R}^{2}}\phi(s,a)B(ds,da)\right)^{\diamond
n}$

for all $c\in\mathbb{R}$. Assume that the above function $b$ is non-negative.
Then the random variable

$\displaystyle b\left(c+\int_{\mathbb{R}^{2}}\phi(s,a)B(ds,da)\right)$
$\displaystyle=y_{0}\sum_{n=0}^{\infty}\frac{(-1)^{n}}{(n!)^{2}}\left(\eta(t,x)\right)^{n}h_{n}\left(c+\int_{\mathbb{R}^{2}}\phi(s,a)B(ds,da)\right)$
$\displaystyle=y_{0}\sum_{n=0}^{\infty}\frac{1}{(n!)^{2}}\left(-\eta(t,x)(c+\int_{\mathbb{R}^{2}}\phi(s,a)B(ds,da))\right)^{\diamond
n}\text{.}$

must be non-negative, too, for all $c\in\mathbb{R}$.

Let $\phi$ be the first basis element of an orthornormal basis of (tensored)
Hermite functions in $L^{2}(\mathbb{R}^{2})$. Then the Hermite transform of
$b(c+\int_{\mathbb{R}^{2}}\phi(s,a)B(ds,da))$ evaluated at $z=iy$,
$y\in\mathbb{R}$ is given by

$y_{0}\sum_{n=0}^{\infty}\frac{1}{(n!)^{2}}\left(-\eta(t,x)(c+iy)\right)^{n}\text{.}$

So using once again the positivity criterion in (3.4), the expression

$y_{0}\sum_{n=0}^{\infty}\frac{1}{(n!)^{2}}\left(-\eta(t,x)(c+iy)\right)^{n}e^{-\tfrac{1}{2}|y|^{2}}$

must be _in particular_ for $y=0$ non-negative (according to the definition of
a positive-definite function).

So for

$f_{0}(r):=\sum_{n=0}^{\infty}\frac{1}{(n!)^{2}}r^{n}$

the expression

$y_{0}f_{0}(-\eta(t,x)c)$

must be non-negative for all $c$. On the other hand, it is known that the
function $f_{0}$, which is related to the Bessel function of order zero, has
the following asymptotic property:

$f_{0}(y)\sim(\pi\sqrt{\left|y\right|})^{-1/2}\cos(2\sqrt{\left|y\right|}-\frac{\pi}{4})\text{
for }y\rightarrow-\infty\text{.}$

See e.g. Nualart [N87]. Hence, if $\eta(t,x)\neq 0$, the above expression can
become negative, which leads to a contradiction. So we conclude that the
function $b$ and therefore also the function in (3.8) can become negative. So
it follows from Bochner’s theorem that the function $g$ is not positive
definite.

We get the following conclusion:

###### Theorem 3.1

The solution $Y(t,x)$ of the equation (3) is not always positive

###### Remark 3.2

Using a different method, we mention that the latter result was obtained in
Nualart [N87] in the special case of $\alpha_{0}=0$ and $\beta_{0}$ a
constant. There Nualart proves that $Y(t,x)<0$ uniformly in $(t,x)$ on an open
set, with positive probability.

##  4 Background

###  4.1 The Itô formula

To study such optimal control problems, we will use a version of the Itô
formula for such systems. First we introduce some notation from Wang & Zakai
[WZ].

  * •

We put $\zeta=(\zeta_{1},\zeta_{2})=(s,a)\in\mathbb{R}\times\mathbb{R}$ and
$d\zeta=d\zeta_{1}d\zeta_{2}=dsda$,

  * •

$B(t,x)$ is a Brownian sheet; $t\geq 0,x\in\mathbb{R}$,

  * •

$z=(z_{1},z_{2})=(t,x),R_{z}=[0,z_{1}]\times[0,z_{2}]$,

  * •

$\int_{R_{z}}\varphi(\zeta)B(d\zeta)$ denotes the Itô integral with respect to
$B(\cdot)$ over $R_{z}$,

  * •

$\int_{R_{z}}\psi(\zeta)d\zeta$ is 2-dimensional Lebesgue integral of $\psi$,

  * •

if $a=(a_{1},a_{2}),b=(b_{1},b_{2})$, then $a\vee
b=(\max(a_{1},b_{1}),\max(a_{2},b_{2})).$

  * •

$I((a_{1},a_{2})\bar{\wedge}(b_{1},b_{2}))=1\newline \text{ if }a_{1}\leq
b_{1}\quad\&\quad a_{2}\geq b_{2},\quad 0$ otherwise.

######  Theorem 4.1 (Itô formula, Wang & Zakai [WZ])

Suppose

$Y(z)=Y_{0}+\int_{R_{z}}\alpha(\zeta)d\zeta+\int_{R_{z}}\beta(\zeta)B(d\zeta)+\iint\limits_{R_{z}\times
R_{z}}\psi(\zeta,\zeta^{\prime})B(d\zeta)B(d\zeta^{\prime}).$ (4.1)

Then, if $f:\mathbb{R\rightarrow R}$ is smooth, we have

$\displaystyle
f(Y(z))=f(Y_{0})+\int_{R_{z}}f^{\prime}(Y(\zeta))[\alpha(\zeta)d\zeta+\beta(\zeta)B(d\zeta)]$
$\displaystyle+\tfrac{1}{2}\int_{R_{z}}f^{\prime\prime}(Y(\zeta))\beta^{2}(\zeta)d\zeta$
$\displaystyle+\iint\limits_{R_{z}\times
R_{z}}\Big{\\{}f^{\prime\prime}(Y(\zeta\vee\zeta^{\prime}))u_{k}\widetilde{u}_{l}+f^{\prime}(Y(\zeta\vee\zeta^{\prime}))\psi(\zeta,\zeta^{\prime})\Big{\\}}B(d\zeta)B(d\zeta^{\prime})$
$\displaystyle+\iint\limits_{R_{z}\times
R_{z}}\Big{\\{}f^{\prime\prime}(Y(\zeta\vee\zeta^{\prime}))\Big{(}u_{k}\alpha(\zeta)+\psi(\zeta,\zeta^{\prime})\widetilde{u}_{l}\Big{)}$
$\displaystyle+\tfrac{1}{2}f^{(3)}(Y(\zeta\vee\zeta^{\prime}))u^{2}_{k}\widetilde{u}_{l}\Big{\\}}d\zeta
B(d\zeta^{\prime})$ $\displaystyle+\iint\limits_{R_{z}\times
R_{z}}\Big{\\{}f^{\prime\prime}(Y(\zeta\vee\zeta^{\prime}))\Big{(}\widetilde{u}_{l}\alpha(\zeta^{\prime})+\psi(\zeta,\zeta^{\prime})u_{k}\Big{)}$
$\displaystyle+\tfrac{1}{2}f^{(3)}(Y(\zeta\vee\zeta^{\prime}))u^{2}_{k}\widetilde{u}_{l}\Big{\\}}d\zeta
B(d\zeta^{\prime})$ $\displaystyle+\iint\limits_{R_{z}\times
R_{z}}I(\zeta\bar{\wedge}\zeta^{\prime})\Big{\\{}f^{\prime\prime}(Y(\zeta\vee\zeta^{\prime}))\Big{(}\alpha(\zeta^{\prime})\alpha(\zeta)+\tfrac{1}{2}\psi^{2}(\zeta,\zeta^{\prime})\Big{)}$
$\displaystyle+f^{(3)}(Y(\zeta\vee\zeta^{\prime}))u\widetilde{u}\psi(\zeta,\zeta^{\prime})+\tfrac{1}{2}f^{(3)}(Y(\zeta\vee\zeta^{\prime}))\left[\alpha(\zeta^{\prime})\widetilde{u}^{2}+\alpha(\zeta)u^{2}\right]$
$\displaystyle+\tfrac{1}{4}f^{(4)}(Y(\zeta\vee\zeta^{\prime}))u^{2}\widetilde{u}^{2}\Big{\\}}d\zeta
d\zeta^{\prime},$

where

$u=\beta(\zeta^{\prime})+\int_{R_{z}}I(\zeta\bar{\wedge}\zeta^{\prime})\psi(\zeta,\zeta^{\prime})B(d\zeta),$

$\widetilde{u}=\beta(\zeta)+\int_{R_{z}}I(\zeta\bar{\wedge}\zeta^{\prime})\psi(\zeta,\zeta^{\prime})B(d\zeta^{\prime}).$

It is proved in [WZ] that the double $B(d\zeta)B(d\zeta^{\prime})$-integrals,
and the mixed $d\zeta B(d\zeta^{\prime})$ and
$B(d\zeta)d\zeta^{\prime}$-integrals are all weak martingales and hence have
expectation $0$. Therefore, by the Itô formula above we get the following:

###### Theorem 4.2

(Dynkin formula)

$\displaystyle E[f(Y(z))]$
$\displaystyle=f(Y_{0})+E\Big{[}\int_{R_{z}}\Big{\\{}\alpha(\zeta)f^{\prime}(Y(\zeta))+\frac{1}{2}\beta^{2}(\zeta)f^{{}^{\prime\prime}}(Y(\zeta))\Big{\\}}d\zeta$
$\displaystyle+\iint\limits_{R_{z}\times
R_{z}}I(\zeta\bar{\wedge}\zeta^{\prime})\Big{\\{}f^{\prime\prime}(Y(\zeta\vee\zeta^{\prime}))\Big{(}\alpha(\zeta^{\prime})\alpha(\zeta)+\tfrac{1}{2}\psi^{2}(\zeta,\zeta^{\prime})\Big{)}$
$\displaystyle+f^{(3)}(Y(\zeta\vee\zeta^{\prime}))u_{k}\widetilde{u}_{l}\psi(\zeta,\zeta^{\prime})$
$\displaystyle+\tfrac{1}{2}f^{(3)}(Y(\zeta\vee\zeta^{\prime}))\left[\alpha(\zeta^{\prime})\widetilde{u}^{2}_{l}+\alpha(\zeta)u^{2}_{k}\right]$
$\displaystyle+\tfrac{1}{4}f^{(4)}(Y(\zeta\vee\zeta^{\prime}))u^{2}_{k}\widetilde{u}^{2}_{l}\Big{\\}}d\zeta
d\zeta^{\prime}\Big{]}.$

######  Lemma 4.3 (Integration by parts)

Suppose that

$\displaystyle
Y_{k}(z)=Y_{k}(0)+\int_{R_{z}}\alpha_{k}(\zeta)d\zeta+\int_{R_{z}}\beta_{k}(\zeta)B(d\zeta)+\iint\limits_{R_{z}\times
R_{z}}\psi_{k}(\zeta,\zeta^{\prime})B(d\zeta)B(d\zeta^{\prime}).\quad k=1,2.$

Then

$\displaystyle E[Y_{1}(z)Y_{2}(z)]=Y_{1}(0)Y_{2}(0)$
$\displaystyle+E\Big{[}\int_{R_{z}}\Big{\\{}Y_{1}(\zeta)\alpha_{2}(\zeta)+Y_{2}(\zeta)\alpha_{1}(\zeta)+\beta_{1}(\zeta)\beta_{2}(\zeta)\Big{\\}}d\zeta$
$\displaystyle+\iint\limits_{R_{z}\times
R_{z}}I(\zeta\bar{\wedge}\zeta^{\prime})\Big{\\{}\alpha_{1}(\zeta^{\prime})\alpha_{2}(\zeta)+\alpha_{1}(\zeta)\alpha_{2}(\zeta^{\prime})+\psi_{1}(\zeta,\zeta^{\prime})\psi_{2}(\zeta,\zeta^{\prime})\Big{\\}}d\zeta
d\zeta^{\prime}\Big{]}.$

Proof. The proof follows from the Itô formula, Proposition 5.1 in Wang & Zakai
[WZ]. $\square$

##  5 BSPDEs in the plane

To simplify the notation we sometimes put $z=(t,x),\zeta=(s,a)$ in the
following:
Let $\mathcal{P}$ be the predictable $\sigma$-algebra of subsets of
$\Omega\times R_{z_{0}}$ generated by the sets $(z,z^{\prime}]\times A$ where
$A\in\mathcal{F}_{z}$ and we denote by $\mathcal{D}$ the $\sigma$-algebra of
$\Omega\times R_{z_{0}}\times R_{z_{0}}$ generated by the sets
$(z_{1},z_{1}^{\prime}]\times(z_{2},z_{2}^{\prime}]\times A$ where
$(z_{1},z_{1}^{\prime}]\bar{\wedge}(z_{2},z_{2}^{\prime}]$ and
$A\in\mathcal{F}_{z_{1}\vee z_{2}}$.
The solution of the BSDE will live in the following spaces:

  * •

$L^{2}_{a,1}$ is the space of predictable processes {$\phi(z),z\in
R_{z_{0}}$}, such that
$E[\int_{R_{z}}\phi(z)^{2}dz<\infty]$,

  * •

$L^{2}_{a,2}$ is the space of processes {$\psi(z,z^{\prime}),(z,z^{\prime})\in
R_{z_{0}}\times R_{z_{0}}$}, such that

    * (a)

$\psi(z,z^{\prime})=0$ unless $z\bar{\wedge}z^{\prime}$,

    * (b)

$\psi$ is $\mathcal{D}$-measurable,

    * (c)

$E[\int_{R_{z}}\int_{R_{z}}\psi(z,z^{\prime})^{2}dzdz^{\prime}<\infty]$.

Let us recall now the representation of square integrable martingales.

######  Theorem 5.1 (Wong & Zakai [WZ])

If $M=\\{M(z),\mathcal{F}_{z},z\in\mathbb{R}_{+}^{2}\\}$ is square integrable
martingale, then for each $z\in\mathbb{R}_{+}^{2}$

$\displaystyle
M(z)=M(0)+\int_{R_{z}}\phi(\zeta)B(d\zeta)+\iint\limits_{R_{z}\times
R_{z}}\psi(\zeta,\zeta^{\prime})B(d\zeta)B(d\zeta^{\prime}),$ (5.1)

where $\phi,\psi$ are adapted processes.

Let $Z=(T,X)$ and if we fix a rectangle $R_{Z}=[0,T]\times[0,X]$, and let
$\xi$ be an $\mathcal{F}_{Z}$-measurable random variable and
$h(\omega,\zeta,p,q)$ is a
$\mathcal{P}\times\mathcal{B}_{\mathbb{R}}\times\mathcal{B}_{\mathbb{R}}$-measurable
function such that $\int_{R_{z}}|h(\zeta,p(\zeta),q(\zeta))|d\zeta<\infty$.
Then we can define a triple of processes $(p,q,r)\in L^{2}_{a,1}\times
L^{2}_{a,1}\times L^{2}_{a,2}$ solution of the BSPDE in the plane

$\displaystyle p(z)$ $\displaystyle=\xi-\int_{R_{Z}\setminus
R_{z}}h(\zeta,p(\zeta),q(\zeta))d\zeta-\int_{R_{Z}\setminus
R_{z}}q(\zeta)B(d\zeta)$ $\displaystyle-\int_{R_{Z}\setminus
R_{z}}\int_{R_{Z}\setminus
R_{z}}r(\zeta,\zeta^{\prime})B(d\zeta)B(d\zeta^{\prime}).$ (5.2)

Alternatively, let us introduce the notation

$\displaystyle M_{r}(z)=\iint\limits_{R_{z}\times
R_{z}}r(\zeta,\zeta^{\prime})B(d\zeta)B(d\zeta^{\prime}),\quad r\in L_{a,2}.$
(5.3)

Then $M_{r}(z)$ is a martingale, and we can write the equation for $(p,q,r)$
above in differential form as follows

$\displaystyle p(dz)$
$\displaystyle=h(z,p(z),q(z))dz+q(z)B(dz)+M_{r}(dz),\quad z\leq Z,$ (5.4)
$\displaystyle p(Z)$ $\displaystyle=\xi.$

Assumptions We impose the following set of assumptions:

  * (i)

$\xi\in L^{2}(\Omega,\mathcal{F}_{z_{0}},P)$,

  * (ii)

$h(\cdot,p,q)\in L^{2}_{a,1}$ for all $p,q\in\mathbb{R}$,

  * (ii)

$|h(\zeta,p,q)-h(\zeta^{\prime},p^{\prime},q^{\prime})|^{2}\leq
K_{1}|p-p^{\prime}|^{2}+K_{2}|q-q^{\prime}|^{2}$, for all
$p,q,p^{\prime},q^{\prime}\in\mathbb{R}$ and $\zeta\in R_{Z}.$

Let $f_{0}$ be the Bessel function of order zero and $r_{0}\approx 1.4458$ be
the first nonnegative zero of $J_{0}$:

$r_{0}=inf\left\\{t>0:f_{0}(2\sqrt{t})=\sum_{j=0}^{\infty}\frac{(-1)^{j}}{{j!}^{2}}t^{j}=0\right\\}.$

######  Theorem 5.2 (Existence and Uniqueness (Zaidi & Nualart) [ZN])

Under the above assumptions (i)-(iii) and if the Lipschitz constant satisfies
$K_{1}|z_{0}|<\sqrt{r_{0}}$ and $K_{2}|z_{0}|<\sqrt{r_{0}}$, there exists a
unique solution of the BSPDE (5).

##  6 Maximum principle approaches

We introduce the following notation:

###### Definition 6.1

For general functions $h,k:[0,T]\times[0,X]\mapsto\mathbb{R}$, we define

$\displaystyle(h\star
k)(t,x)=\int_{0}^{x}\int_{-t}^{T}\\{h(t,x)k(s,a)+h(s,a)k(t,x)\\}dsda$ (6.1)

Note that with this notation we have

$\displaystyle\iint\limits_{R_{z}\times
R_{z}}I(\zeta\bar{\wedge}\zeta^{\prime})\Big{\\{}\alpha_{1}(\zeta^{\prime})\alpha_{2}(\zeta)+\alpha_{1}(\zeta)\alpha_{2}(\zeta^{\prime})\Big{\\}}d\zeta
d\zeta^{\prime}=\int_{R_{z}}(\alpha_{1}\star\alpha_{2})(\zeta)d\zeta,$ (6.2)

and the integration by parts formula can be written

######  Lemma 6.2 (Integration by parts (2))

Suppose that

$\displaystyle
Y_{k}(z)=Y_{k}(0)+\int_{R_{z}}\alpha_{k}(\zeta)d\zeta+\int_{R_{z}}\beta_{k}(\zeta)B(d\zeta),\quad
k=1,2.$

Then

$\displaystyle E[Y_{1}(z)Y_{2}(z)]=Y_{1}(0)Y_{2}(0)$
$\displaystyle+E\Big{[}\int_{R_{z}}\Big{\\{}Y_{1}(\zeta)\alpha_{2}(\zeta)+Y_{2}(\zeta)\alpha_{1}(\zeta)+\beta_{1}(\zeta)\beta_{2}(\zeta)+(\alpha_{1}\star\alpha_{2})(\zeta)\Big{\\}}d\zeta\Big{]}.$
(6.3)

Given a subset $U$ of $\mathbb{R}$ we denote by $\mathcal{U}$ the set of all
$\mathcal{F}_{t,x}$-adapted control processes $u=\\{u(t,x),t<T,x<X\\}$ valued
in $U$. We then define the set of admissible control processes
$\mathcal{A}\subset\mathcal{U}$ to be the collection of all
$\mathcal{F}_{t,x}$-adapted processes with values in $U$.

Let $f$ and $g$ be given functions and consider the performance functional

$\displaystyle
J(u)=E\Big{[}\int_{R_{Z}}f(\zeta,Y(\zeta),u(\zeta))d\zeta+g(Y(Z))\Big{]},$

where

$R_{Z}=[0,T]\times[0,X],$

with $Z=(T,X)$ for some given $T>0,X>0$, and the state $Y$ of the system is
described by the equation

$\displaystyle
Y(z)=Y(t,x)=Y(0)+\int_{R_{z}}\alpha(\zeta,Y(\zeta),u(\zeta))d\zeta+\int_{R_{z}}\beta(\zeta,Y(\zeta),u(\zeta))B(d\zeta),\quad
z\leq Z,$ (6.4)

where $R_{z}=[0,t]\times[0,x]$ when $z=(t,x)$, and $u$ denotes a control
process.

###### Problem 6.3

We want to find $\widehat{u}\in\mathcal{A}$ such that

$J(\widehat{u})=\sup_{u\in\mathcal{A}}J(u).$ (6.5)

The maximum principle approach to this problem, adapted to the time-space
situation, is to introduce the following associated Hamiltonian

$H:R_{Z}\times R_{Z}\times\mathbb{R}\times
U\times\mathbb{R}\times\mathbb{R}\times\mathbb{R}$

given by

$\displaystyle
H(z,Y,u,p,q,L)=f(z,y,u)+p\alpha(z,y,u)+q\beta(z,y,u)+(L\star\alpha)(z),$ (6.6)

where we recall that

$(L\star\alpha)(z)=\int_{0}^{x}\int_{t}^{T}\Big{\\{}L(z)\alpha(\zeta^{\prime},y,u)+L(\zeta^{\prime})\alpha(z,y,u)\Big{\\}}d\zeta^{\prime};\quad
z=(t,x)$

and the adjoint processes $(p,q,r,L)=(p(t,x),q(t,x),r(t,x,\cdot),L(t,x))$ are
given by the following equations:

  * •

$L:R_{Z}\mapsto\mathbb{R}$ is defined implicitly by the integral equation

$\displaystyle L(z)=-\frac{\partial H}{\partial
y}(z,Y(z),u(z),p(z),q(z),r(z,\cdot),L(z));\quad z=(t,x)$ (6.7)

  * •

and $(p,q,r)$ is the solution of the BSPDE

$\displaystyle\begin{cases}p(dz)&=-\frac{\partial H}{\partial
y}(z,Y(z),p(z),q(z),r(z,\cdot),L(z))dz\\\ &+q(z)B(dz)+M_{r}(dz),\quad
0\leq(t,x)\leq(T,X),\\\ p(Z)&=\frac{\partial g}{\partial y}(Y(Z)),\end{cases}$
(6.8)

or, in integrated form,

$\displaystyle\begin{cases}p(z)&=p(0)-\int_{R_{z}}\frac{\partial H}{\partial
y}(\zeta,Y(\zeta),p(\zeta),q(\zeta),r(\zeta,\cdot),L(\zeta))d\zeta\\\
&+\int_{R_{z}}q(\zeta)B(d\zeta)+\iint\limits_{R_{z}\times
R_{z}}r(\zeta,\zeta^{\prime})B(d\zeta)B(d\zeta^{\prime}),\quad z\leq Z.\\\
p(Z)&=\frac{\partial g}{\partial y}(Y(Z))\end{cases}$ (6.9)

There are two versions of the maximum principle for this problem, namely the
so-called _sufficient maximum principle_ and the _necessary maximum
principle_. We present them both below.

###  6.1 The sufficient maximum principle

######  Theorem 6.4 (Sufficient maximum principle)

Suppose $\widehat{u}\in\mathcal{A}$ with corresponding solutions
$\widehat{Y},(\widehat{p},\widehat{q},\widehat{r},\widehat{L})$ of the
equations above. Moreover, suppose that $y\mapsto g(y)$ is concave and
$Y,u\mapsto H(z,Y,u,p,q,r,L)$ is concave for all $z,p,q,r,L$ and that

$\displaystyle\small\sup_{v\in A}$ $\displaystyle
H(z,\widehat{Y}(\cdot),v,\widehat{p}(z),\widehat{q}(z),\widehat{r}(z,\cdot),\widehat{L}(z))$
(6.10)
$\displaystyle=H(z,\widehat{Y}(\cdot),\widehat{u}(z),\widehat{p}.(z),\widehat{q}(z),\widehat{r}(z,\cdot),\widehat{L}(z)),$
(6.11)

Then $\widehat{u}$ is an optimal control for problem (6.5).

Proof. Suppose $\widehat{u}\in\mathcal{A}$ satisfies (6.10) with corresponding
$\widehat{Y}$. Choose another $u\in\mathcal{A}$. Then

$\displaystyle J(u)-J(\widehat{u})=I_{1}+I_{2},$ (6.12)

where

$\displaystyle I_{1}$
$\displaystyle=E\Big{[}\int_{R_{Z}}\Big{\\{}f(\zeta,Y(\zeta),u(\zeta))-f(\zeta,\widehat{Y}(\zeta),\widehat{u}(\zeta))\Big{\\}}d\zeta\Big{]}=E\Big{[}\int_{R_{Z}}\tilde{f}(\zeta)d\zeta\Big{]},$
$\displaystyle I_{2}$ $\displaystyle=E[g(Y(Z))-g(\widehat{Y}(Z))].$

Using the definition of $H$ we can write

$\displaystyle
I_{1}=E\Big{[}\int_{R_{Z}}\Big{\\{}H(\zeta)-\widehat{H}(\zeta)-\widehat{p}(\zeta)\widetilde{\alpha}(\zeta)-\widehat{q}(\zeta)\widetilde{\beta}(\zeta)-(\widehat{L}\star\tilde{\alpha})(\zeta)\Big{\\}}d\zeta\Big{]},$
(6.13)

where
$\tilde{\alpha}=\alpha-\widehat{\alpha},\widehat{\alpha}=\alpha(\zeta,\widehat{Y}(\zeta),\widehat{u}(\zeta))$
etc.

Using the concavity of $g$ and Lemma 6.2, and the fact that the
$B(dz)$-integrals and the $B(dz)B(dz^{\prime})$-integrals are orthogonal (see
[CW], Theorem 2.5), we get

$\displaystyle I_{2}$ $\displaystyle\leq E\Big{[}\frac{\partial g}{\partial
y}(\widehat{Y}(Z))\widetilde{Y}(Z)\Big{]}=E\Big{[}\widehat{p}(Z)\tilde{Y}(Z)\Big{]}$
$\displaystyle=E\Big{[}\int_{R_{Z}}\Big{\\{}\widehat{p}(\zeta)\widetilde{\alpha}(\zeta)-\frac{\partial\widehat{H}}{\partial
y}\tilde{Y}(\zeta)+\widehat{q}(\zeta)\tilde{\beta}(\zeta)-(\frac{\partial\widehat{H}}{\partial
y}\star\widetilde{\alpha})(\zeta)\Big{\\}}d\zeta\Big{]}.$ (6.14)

Adding (6.13) and (6.14) we get, using that
$\widehat{L}=-\frac{\partial\widehat{H}}{\partial y}$ and the concavity of
$H(y,u)$,

$\displaystyle J(u)-J(\widehat{u})$ $\displaystyle=I_{1}+I_{2}\leq
E\Big{[}\int_{R_{Z}}\Big{\\{}H(\zeta)-\widehat{H}(\zeta)-\frac{\partial\widehat{H}}{\partial
y}(\zeta)\tilde{Y}(\zeta)\Big{\\}}d\zeta\Big{]}$ (6.15) $\displaystyle\leq
E\Big{[}\int_{R_{Z}}\frac{\partial\widehat{H}}{\partial
u}(\zeta)\tilde{u}(\zeta)\rangle d\zeta\Big{]}\leq 0\text{ by condition
\eqref{max.cond}}.$ (6.16)

This proves that

$\displaystyle J(u)-J(\widehat{u})\leq 0\text{ for all }u\in\mathcal{A},$
(6.17)

and therefore $\widehat{u}$ is optimal. $\square$

###  6.2 The necessary maximum principle

It is a drawback of the sufficient maximum principle that we have to assume
that $y\longmapsto g(y)$ and $(y,u)\longmapsto H(z,y,u,p,q,L)$ are concave.
The following result does not need concavity, but we have to add conditions of
the set $\mathcal{A}$ of admissible controls instead, as follows:

  * (A1)

$\mathcal{A}$ is a convex set

  * (A2)

For all $z_{0}=(t_{0},x_{0})<Z=(T,X)$ and all bounded
$\mathcal{F}_{z_{0}}$-measurable random variables $\theta_{z_{0}}$, the
control

$u_{z_{0}}(\zeta)=\theta_{z_{0}}\mathbf{1}_{R_{z_{0}}}(\zeta)$

is admissible, where

$\displaystyle\mathbf{1}_{R_{z_{0}}}(\zeta)=\begin{cases}1\text{ if }\zeta\in
R_{z_{0}}\\\ 0\text{ if }\zeta\notin R_{z_{0}}\end{cases}$ (6.18)

is the indicator function of the rectangle
$R_{z_{0}}=[t_{0},T]\times[x_{0},X]$.

###### Lemma 6.5

For all $u,v\in\mathcal{A}$ the derivative process

$G(\zeta):=\underset{\epsilon\rightarrow 0}{\lim}\frac{Y^{u+\epsilon
v}(\zeta)-Y^{u}(\zeta)}{\epsilon}$

satisfies the equation

$\displaystyle G(z)$
$\displaystyle=G(0)+\int_{R_{z_{0}}}\Big{\\{}\frac{\partial\alpha}{\partial
y}(\zeta)G(\zeta)+\frac{\partial\alpha}{\partial
u}(\zeta)v(\zeta)\Big{\\}}d\zeta$
$\displaystyle+\int_{R_{z_{0}}}\Big{\\{}\frac{\partial\beta}{\partial
y}(\zeta)G(\zeta)+\frac{\partial\beta}{\partial
u}(\zeta)v(\zeta)\Big{\\}}B(d\zeta),$

where $\frac{\partial\alpha}{\partial y}(\zeta)=\frac{\partial\alpha}{\partial
y}(\zeta,Y^{u}(\zeta),u(\zeta))$ etc.

Proof. This follows by the chain rule. $\square$

###### Lemma 6.6

For all $u,v\in\mathcal{A}$, we have

$\frac{d}{d\epsilon}J(u+\epsilon
v)_{\epsilon=0}=E\Big{[}\int_{R_{Z}}\frac{\partial H}{\partial
u}(\zeta)v(\zeta)d\zeta\Big{]}.$

Proof.

$\displaystyle\frac{d}{d\epsilon}$ $\displaystyle J(u+\epsilon
v)_{\epsilon=0}$ $\displaystyle=\underset{\epsilon\rightarrow
0}{lim}\frac{1}{\epsilon}E\Big{[}\int_{R_{Z}}\Big{\\{}f(\zeta,Y^{u+\epsilon
v}(\zeta),u+\epsilon
v)(\zeta))-f(\zeta,Y^{u}(\zeta),u)(\zeta))\Big{\\}}d\zeta$
$\displaystyle+g(Y^{u+\epsilon v}(Z))-g(Y^{u}(Z))\Big{]}$
$\displaystyle=E\Big{[}\int_{R_{Z}}\Big{\\{}\frac{\partial f}{\partial
y}(\zeta,Y^{u}(\zeta),u(\zeta))G(\zeta)+\frac{\partial f}{\partial
u}(\zeta,Y^{u}(\zeta),u(\zeta))v(\zeta)\Big{\\}}d\zeta$
$\displaystyle+\frac{\partial g}{\partial
y}(Y^{u}(Z))G(Z)\Big{]}=I_{1}+I_{2},$

where

$\displaystyle I_{1}$ $\displaystyle=E\Big{[}\int_{R_{Z}}\Big{(}\frac{\partial
H}{\partial y}(\zeta)-\frac{\partial\alpha}{\partial
y}(\zeta)p(\zeta)-\frac{\partial\beta}{\partial
y}(\zeta)q(\zeta)-(L\star\frac{\partial\alpha}{\partial
y})(\zeta)\Big{)}G(\zeta)d\zeta$
$\displaystyle+\int_{R_{Z}}\Big{(}\frac{\partial H}{\partial
u}(\zeta)-\frac{\partial\alpha}{\partial
u}(\zeta)p(\zeta)-\frac{\partial\beta}{\partial
u}(\zeta)q(\zeta)-(L\star\frac{\partial\alpha}{\partial
u}(\zeta)\Big{)}v(\zeta)d\zeta\Big{]},$

and

$\displaystyle I_{2}$ $\displaystyle=E\Big{[}\frac{\partial g}{\partial
y}(Y^{u}(Z))G(Z)\Big{]}=E[p(Z)G(Z)]$
$\displaystyle=E\Big{[}\int_{R_{Z}}\Big{(}p(\zeta)\Big{\\{}\frac{\partial\alpha}{\partial
y}(\zeta)G(\zeta)+\frac{\partial\alpha}{\partial
u}(\zeta)v(\zeta)\Big{\\}}-\frac{\partial H}{\partial y}(\zeta)G(\zeta)$
$\displaystyle+q(\zeta)\Big{\\{}\frac{\partial\beta}{\partial
y}(\zeta)G(\zeta)+\frac{\partial\beta}{\partial
u}(\zeta)v(\zeta)\Big{\\}}-\big{\\{}(\frac{\partial H}{\partial
y}\star(\frac{\partial\alpha}{\partial y}G+\frac{\partial\alpha}{\partial
u}v)\big{\\}}(\zeta)\Big{\\}}d\zeta\Big{]}.$

Adding $I_{1}$ and $I_{2}$, we get, if we choose $L=-\frac{\partial
H}{\partial y}$,

$\frac{d}{d\epsilon}J(u+\epsilon
v)_{\epsilon=0}=E\Big{[}\int_{R_{Z}}\frac{\partial H}{\partial
u}(\zeta)v(\zeta)d\zeta\Big{]}.$

$\square$

From Lemma 6.4, we deduce the following:

######  Theorem 6.7 (Necessary maximum principle)

Suppose $\widehat{u}\in\mathcal{A}$ is optimal for Problem 2.5. Then

$\displaystyle\frac{\partial H}{\partial
u}(\zeta,\widehat{Y}(\zeta),\widehat{u}(\zeta),\widehat{p}(\zeta),\widehat{q}(\zeta),\widehat{L}(\zeta))=0\text{
for a.a. }\zeta.$

Proof. Since $J(\widehat{u}+\epsilon v)_{\epsilon=0}\leq J(\widehat{u})$ for
all $\epsilon,v,$ we get by Lemma 6.4 that

$E\Big{[}\int_{R_{Z}}\frac{\partial H}{\partial
u}(\zeta)v(\zeta)d\zeta\Big{]}\leq 0,\text{ for all }v\in\mathcal{A}.$

In particular, applying this to

$v(\zeta)=\theta_{z_{0}}\mathbf{1}_{R_{z_{0}}}(\zeta)$

as in A2, this gives

$E\Big{[}\int_{R_{z_{0}}}\frac{\partial H}{\partial
u}(\zeta)\theta_{z_{0}}d\zeta\Big{]}\leq 0.$

Since this holds for all $z_{0}$ we deduce that

$\frac{\partial^{2}}{\partial t_{0}\partial
x_{0}}\left(E\left[\int_{R_{z_{0}}}\frac{\partial H}{\partial
u}\theta_{z_{0}}d\zeta\right]\right)=\frac{\partial H}{\partial
u}(z_{0})\theta_{z_{0}}\leq 0.$

Since this holds for all $\theta_{z_{0}}\in\mathbb{R}$, we conclude that

$\frac{\partial H}{\partial u}(z_{0})=0.$

$\square$

##  7 Applications

###  7.1 Return to the optimal harvesting problem in the plane

Suppose that the growth of a population at time $t$ and position $x$ with
density $Y(t,x)$ satisfies

$\displaystyle Y_{u}(t,x)$
$\displaystyle=Y(0,0)+\int_{0}^{t}\int_{0}^{x}\\{\alpha_{0}Y_{u}(s,a)-u(s,a)\\}dsda$
$\displaystyle+\int_{0}^{t}\int_{0}^{x}\beta_{0}Y_{u}(s,a)B(ds,da),$

where $\alpha_{0},\beta_{0}$ are given constants and $Y(0,0)>0.$

For given constants $T>0,X>0$ such that $T>t,X>x$, define the combined utility
of the harvesting and the terminal population by

$J(u)=E\left[\int_{0}^{T}\int_{0}^{X}ln(u^{2}(s,a))dsda+\theta
Y_{u}(T,X)\right],$

where $\theta$ is a given bounded, $\mathcal{F}_{Z}$-measurable random
variable.

###### Problem 7.1

We want to find the harvesting strategy $u^{\ast}(s,x)$ which maximizes the
utility of the harvest, i.e. we want to find $u^{\ast}\in\mathcal{A}$ such
that

$\displaystyle J(u^{\ast})=\sup_{u\in\mathcal{A}}J(u).$

The associated Hamiltonian for this case is

$\displaystyle
H(t,x,y,u,p,q,L)=ln(u^{2})+(\alpha_{0}y-u)p+\beta_{0}yq+(L\star(\alpha_{0}y-u)).$

The Hamiltonian has a maximum at $u$ given by the equation

$\frac{\partial H}{\partial u}=\frac{2}{u(z)}-p-(L\star 1)(z)=0.$

Hence we must have

$u=\frac{2}{p+(L\star 1)},$

where

$\displaystyle(L\star
1)(z)=x(T-t)L(z)+\int_{0}^{x}\int_{t}^{T}L(\zeta^{\prime})d\zeta^{\prime}.$
(7.1)

Since

$\displaystyle\frac{\partial H}{\partial
y}=\alpha_{0}p+\beta_{0}q+(L\star\alpha_{0})$ (7.2)

the corresponding adjoint equation is

$\displaystyle\begin{cases}p(dz)&=-(\alpha_{0}p+\beta_{0}q+L\star\alpha_{0})(z)dz\\\
&+q(z)B(dz)+M_{r}(dz),\quad 0\leq(t,x)\leq(T,X),\\\ p(Z)&=\frac{\partial
g}{\partial y}(Y(Z)).\end{cases}$ (7.3)

The adjoint variable $L$ is defined by

$\displaystyle L(z)$ $\displaystyle=-\frac{\partial H}{\partial
y}(z)=-[\alpha_{0}p(z)+\beta_{0}q(z)+(L\star 1)(z)]$
$\displaystyle=-[\alpha_{0}p(z)+\beta_{0}q(z)+x(T-t)L(z)+\int_{0}^{x}\int_{t}^{T}L(\zeta^{\prime})d\zeta^{\prime},$
(7.4)

which is an integral equation for $L(z)$. We summarise this as follows:

###### Theorem 7.2

Let $z>0$ and assume that $p(z)+(L\star 1)(z)\neq 0$. Then the optimal
harvesting rate $u^{*}$ for Problem 7.1 is given by

$\displaystyle u^{*}(t,x)=u^{*}(z)=\frac{2}{p(z)+(L\star 1)(z)},$

with $p$ and $L$ given by (7.3) and (7.1), respectively.

###  7.2 Return to the linear-quadratic (LQ) problem in the plane

We now return to the linear-quadratic (LQ) control problem for time-space
random fields discussed in the introduction: Suppose the state $Y(t,x)$ is
given by

$\displaystyle Y(t,x)=Y(0,0)+\int_{0}^{t}\int_{0}^{x}u(s,a)dsda+\beta
B(t,x);\quad t\geq 0,x\in\mathbb{R}.$ (7.5)

We want to drive the state $Y(t,x)$ to 0 at time $T$ and point $X$ with
minimal use of energy. Hence we put

$J(u)=-\tfrac{1}{2}E\left[\int_{0}^{T}\int_{0}^{X}u^{2}(s,a)dsda+\theta
Y^{2}(T,X)\right],$ (7.6)

where $\theta>0$ is a given constant.

###### Problem 7.3

We want to find $u^{\ast}\in\mathcal{A}$ such that

$\displaystyle J(u^{\ast})=\sup_{u\in\mathcal{A}}J(u).$ (7.7)

In this case we have $\nabla_{y}H=0$, so we can simplify the Hamiltonian to

$\displaystyle H_{0}(t,x,y,u,p,q)=-\tfrac{1}{2}u^{2}+up+\beta q$ (7.8)

The maximum of $u\mapsto H_{0}(u)$ is obtained when $\frac{\partial
H_{0}}{\partial u}=-u+p=0$, i.e. when

$u=p.$ (7.9)

The adjoint equation is

$\displaystyle p(dz)$ $\displaystyle=q(z)B(dz);\quad z<Z=(T,X)$ $\displaystyle
p(T,X)$ $\displaystyle=\theta Y(T,X)$ (7.10)

Let us try to put

$\displaystyle p(t,x)=\lambda(t,x)Y(t,x)$ (7.11)

for some deterministic function $\lambda$. Then by the Itô formula

$\displaystyle\lambda(t,x)Y(t,x)$ $\displaystyle=\lambda(0,0)Y(0,0)$
$\displaystyle+\int_{0}^{t}\int_{0}^{x}\Big{\\{}Y(s,a)\frac{\partial^{2}\lambda}{\partial
s\partial x}(s,a)+\lambda(s,a)u(s,a)\Big{\\}}dsda$ (7.12)
$\displaystyle+\text{ terms containing }B(ds,da).$

Using the concept of quadratic variation of $2-$parameter martingales (see
e.g. Imkeller [I2]), one finds that the decomposition of a (continuous)
$2-$parameter ”semimartingale”, which is given by a sum of a $2-$parameter
process of bounded variation and a $2-$parameter martingale, is unique. So,
comparing the latter equation with the adjoint equation, we see that we must
have

$\displaystyle Y(t,x)\frac{\partial^{2}\lambda}{\partial t\partial
x}(t,x)+\lambda(t,x)u(t,x)=0\text{ for all }t,x.$ (7.13)

Combining this with (7.11) we get

$\displaystyle Y(t,x)\Big{[}\frac{\partial^{2}\lambda}{\partial t\partial
x}(t,x)+\lambda^{2}(t,x)\Big{]}=0,$ (7.14)

with terminal condition

$\displaystyle\lambda(T,X)=\theta.$ (7.15)

In addition we get from (7.11) the other boundary condition

$\displaystyle\lambda(0,0)=\frac{E[\theta Y(T,X)]}{Y(0,0)}.$ (7.16)

With this choice of $u,p,\lambda$ we see that all the conditions of the
sufficient maximum principle are satisfied, and we have proved the following:

###### Theorem 7.4

The optimal control $\widehat{u}$ for the LQ problem (7.3) is given in
feedback form by

$\displaystyle\widehat{u}(t,x)=\lambda(t,x)Y(t,x);\quad t\leq T,x\leq X,$
(7.17)

where $\lambda(t,x)$ solves the time-space Riccati equation

$\displaystyle\begin{cases}\frac{\partial^{2}\lambda}{\partial t\partial
x}(t,x)+\lambda^{2}(t,x)=0;\quad 0\leq t\leq T,0\leq x\leq X,\\\
\lambda(T,X)=\theta,\\\ \lambda(0,0)=\frac{E[\theta
Y(T,X)]}{Y(0,0)}.\end{cases}$ (7.18)

###### Remark 7.5

Let $\varphi_{1}$ be a solution to the Riccati equation

$\dot{\varphi}_{1}(t)=(\varphi_{1}(t))^{2},\varphi_{1}(0)=1,0\leq t\leq T$

and $\varphi_{2}$ be a solution to

$\dot{\varphi}_{2}(x)=-(\varphi_{2}(x))^{2},\varphi_{2}(0)=\theta,0\leq x\leq
X.$

Define $\lambda(t,x)=$ $\alpha_{1}(t)\alpha_{2}(x)$, where
$\alpha_{1}(t):=\varphi_{1}(T-t)$ and $\alpha_{2}(x):=\varphi_{2}(X-x)$. Then

$\frac{\partial^{2}\lambda(t,x)}{\partial t\partial
x}=\dot{\varphi}_{1}(T-t)\dot{\varphi}_{2}(X-x)=(\alpha_{1}(t))^{2}(-(\alpha_{2}(t))^{2})=-(\lambda(t,x))^{2}$

with $\lambda(T,X)=\varphi_{1}(0)\varphi_{2}(0)=\theta$.

By solving the Riccati equations, we find that $\lambda$ given by

$\lambda(t,x)=\frac{1}{(1-T+t)(\theta^{-1}+X-x)}$

is an explicit solution to the above hyperbolic PDE with boundary condition
$\lambda(T,X)=\theta$ for $0<T<1$.

Let us now have a look at the other condition

$\lambda(0,0)=\frac{\theta E\left[Y(T,X)\right]}{Y(0,0)}$

We observe that

$E\left[Y(t,x)\right]=Y(0,0)+\int_{0}^{t}\int_{0}^{x}\lambda(s,a)E\left[Y(s,a)\right]dsda\text{.}$

So, if we use Picard iteration combined with the fact that $\lambda$ can be
written as a product of a function in $t$ and another function in $x$, we see
that the solution of the latter equation has the representation

$E\left[Y(t,x)\right]=Y(0,0)f\left(\int_{0}^{t}\int_{0}^{x}\lambda(s,a)dsda\right),$

where the function $f:\mathbb{R}\mapsto\mathbb{R}$ is defined by

$f(y)=\sum_{n\geq 0}\frac{y^{n}}{(n!)^{2}}\text{.}$

On the other hand,

$\displaystyle\int_{0}^{T}\int_{0}^{X}\lambda(s,a)dsda$ $\displaystyle=$
$\displaystyle\int_{0}^{T}\int_{0}^{X}\frac{1}{(1-T+s)(\theta^{-1}+X-a)}dsda$
$\displaystyle=$ $\displaystyle-\log(1-T)\log(1+X\theta).$

So the condition

$\lambda(0,0)=\frac{\theta E\left[Y(T,X)\right]}{Y(0,0)}$

is equivalent to

$\frac{1}{(1-T)(\theta^{-1}+X)}=\theta f(-\log(1-T)\log(1+X\theta))$

or

$1=(1-T)(1+X\theta)f(-\log(1-T)\log(1+X\theta)).$ (7.19)

For given $T<1$ and $\theta>0$ the expression on the right hand side of the
latter equation converges to $(1-T$) for $X\rightarrow 0$. For
$X\rightarrow\infty$, this expression converges to $\infty$. Because of
continuity, we then see that there exists a $X=X(T,\theta)>0$ such that the
equation (7.19) is satisfied. Using such a time horizon $X$, gives the other
boundary condition.

###  7.3 Example related to machine learning

In machine learning the (continuous-time) stochastic gradient descent method
(see e.g. [MS] and the references therein) is used to minimize an objective
function $f:\mathbb{R}^{d}\longrightarrow\mathbb{R}$. Compared to the
classical gradient descent method without noise, this approach is especially
computationally efficient, when the dimension $d$ in practical optimization
problems is high. If the objective function is sufficiently smooth the
critical points of $f$ with respect to local or global minima may be found by
means of solutions to SDEs of the type

$dY_{t}=-\eta\nabla f(Y_{t})dt+\beta_{0}dB_{t},Y_{0}=x\in\mathbb{R}^{d},t\geq
0\text{,}$ (7.20)

where $\eta\geq 0$ is the learning rate (or step size),
$\beta_{0}\in\mathbb{R}^{d\times d}$, $B_{t},t\geq 0$ a Brownian motion and
where $\nabla$ denotes the gradient of a function. In general, the selection
of an ”optimal” learning rate $\eta$, which determines the optimal step size
towards a minimum in the sense of speed, is in general difficult. If $\eta$ is
chosen too small, the solution may converge too slowly to a critical point. On
the other hand, a too large $\eta$ could result in overshoot or divergence. In
order to gain a deeper understanding of the latter problem, one may consider
instead of the SDE (7.20) a more general framework (at the possible expense of
computational cost) in connection with the following type of hyperbolic SPDE:

$Y(t,x)=y-\int_{0}^{t}\int_{0}^{x}u(s,a)\nabla
f(Y(s,a))dsda+\beta_{0}B(t,x),y\in\mathbb{R}^{d},t,x\geq 0\text{,}$ (7.21)

where
$u:\Omega\times\left[0,\infty\right)^{2}\longrightarrow\left[0,\infty\right)$
is a stochastic learning rate in time and space given by an adapted random
field and where $B$ is a Brownian sheet in $\mathbb{R}^{d}$. Formally, by
choosing in (7.21) $u=\eta\delta_{x}$ for the Dirac delta function
$\delta_{x}$ in a fixed point $x$ and $\eta\geq 0$ we obtain an SDE of the
type (7.20). So the random field dynamics (7.21) provides a more general
framework than that in the one-parameter case (7.20) for finding the critical
points of $f$.

On the other hand, we may view the integral term

$\int_{0}^{x}u(s,a)\nabla f(Y(s,a))da$

in (7.21) for a fixed $x$ and a certain class of stochastic $2-$parameter
learning rate processes as an (weighted) average of $\nabla f(Y(s,a))$, $0\leq
a\leq x$ in (7.20). Here $Y(s,a),0\leq a\leq x$ can be interpreted as a group
of mountain hikers in the optimisation landscape who communicate with each
other with respect to (average) gradient information in order to find the
descent to the valley (i.e. minimum).

The latter, combined with the ”exploration ability” of the Brownian sheet with
respect to the spatial parameter direction in the optimisation landscape,
suggests a solution that converges to rather flat minima, while escaping from
sharp minima. The convergence to flat minima, however, is in many applications
a favorable feature from a machine learning point of view (see [HS]).

In order to construct optimal stochastic $2-$parameter learning rate processes
one may e.g study stochastic control problems based on the stochastic maximum
principle for SPDEs driven by a Brownian sheet with respect to certain
performance functionals as e.g.

$J(u)=-E\left[\int_{0}^{T}\int_{0}^{X}u^{2}(s,a)dsda+f(Y(T,X))\right]\text{,}$
(7.22)

where one mimimises the expected value of $f(Y(T,X))$, while the ”energy
invested” in $u$ is kept minimal.

Using the first order Taylor expansion, we can also approximate $\nabla f$ in
(7.21) by an affine function $g$ given by $g(x)=a+Ax$ for
$a\in\mathbb{R}^{d}$, $A\in\mathbb{R}^{d\times d}$ and obtain a more
simplified framework for our stochastic control problem with respect to $u$.
In this setting, let us now consider the case $d=1$ and the following
controlled process:

$\displaystyle Y_{u}(t,x)$
$\displaystyle=Y(0,0)-\int_{0}^{t}\int_{0}^{x}u(s,a)Y_{u}(s,a)dsda+\int_{0}^{t}\int_{0}^{x}\beta_{0}B(ds,da).$

###### Problem 7.6

We want to maximise the performance functional

$J(u)=-E\Big{[}\int_{0}^{T}\int_{0}^{X}u^{2}(s,a)dsda+\theta
Y^{2}(T,X)\Big{]}.$

In this case the associated Hamiltonian is

$\displaystyle H(t,x,y,u,p,q,L)=-u^{2}-yup+\beta_{0}q-(L\star(uy)),$

which gives

$\displaystyle\frac{\partial H}{\partial y}=-up-(L\star u)$ (7.23)

and hence the adjoint variable $L$ is given by

$\displaystyle L=-\frac{\partial H}{\partial y}=up+(L\star u)$ (7.24)

which is an integral equation for $L(z)$.

The adjoint BSDE is

$\displaystyle p(z)$ $\displaystyle=p(0)+\int_{R_{z}}(u(\zeta)p(\zeta)+(L\star
u))(\zeta)d\zeta$
$\displaystyle-\int_{R_{z}}q(\zeta)B(d\zeta)-\iint\limits_{R_{z}\times
R_{z}}r(\zeta,\zeta^{\prime})B(d\zeta)B(d\zeta^{\prime}),\quad z\leq(T,X)$
(7.25) $\displaystyle p(T,X)$ $\displaystyle=-2\theta Y(T,X).$ (7.26)

In order to maximize $H$ with respect to $u$, we first compute

$\frac{\partial H}{\partial u}=-2u-yp-(L\star y)$

which gives the optimal control

$u=u^{*}=\tfrac{1}{2}(yp+(L\star y))$

We have proved:

###### Theorem 7.7

The optimal control $u(z)=u^{*}(x)$ for Problem 7.6 is given as the 4th
component of the solution $(Y(z),L(z),p(z),u(z))$ of the coupled system of
equations, consisting of

$\displaystyle u(z)$ $\displaystyle=u^{*}(z)=\tfrac{1}{2}[Y(z)p(z)+(L\star
Y)(z)],$ (7.27) $\displaystyle Y(t,x)$
$\displaystyle=Y(0,0)-\int_{0}^{t}\int_{0}^{x}u(s,a)Y_{u}(s,a)dsda+\int_{0}^{t}\int_{0}^{x}\beta_{0}B(ds,da),$
(7.28) $\displaystyle L(z)$ $\displaystyle=u(z)p(z)+(L\star u)(z)$ (7.29)

and

$\displaystyle\begin{cases}p(z)&=p(0)+\int_{R_{z}}(u(\zeta)p(\zeta)+(L\star
u))(\zeta)d\zeta\\\ &-\int_{R_{z}}q(\zeta)B(d\zeta)-\iint\limits_{R_{z}\times
R_{z}}r(\zeta,\zeta^{\prime})B(d\zeta)B(d\zeta^{\prime}),\quad z\leq(T,X)\\\
p(T,X)&=-2\theta Y(T,X).\end{cases}$ (7.30)

###### Remark 7.8

In the more general case, when $\triangledown f(x)=Ax$ for
$A\in\mathbb{R}^{d\times d}$, one can show a similar result about the optimal
control $u^{\ast}$ with respect to the controlled process (7.21) and
performance functional (7.22)

## References

  * [BDMPP] Bogso, A. M., Dieye, M., Pamen, O. M., & Proske, F. (2022). Smoothness of solutions of hyperbolic stochastic partial differential equations with $L^{\infty}$-vector fields. arXiv preprint arXiv:2212.08466.
  * [C72] Cairoli, R. (1972). Sur une équation différentielle stochastique. CRAS Série AB, 274, A1739-A1742.
  * [CW] Cairoli, R., & Walsh, J. B. (1975). Stochastic integrals in the plane. Acta Math. 134, 111-183.
  * [DØP] Di Nunno, G., Øksendal, B., Proske, F. (2009). Malliavin Calculus for Lévy processes with Applications to Finance. Springer, Universitext.
  * [FN] Farré, M., Nualart, D. (1993). Nonlinear stochastic integral equations in the plane. Stochastic processes and their applications, 46(2), 219-239.
  * [HS] Hochreiter, S., & Schmidhuber, J. (1997). Flat minima. Neural computation, 9(1), 1-42.
  * [HØUZ] Holden, H., Øksendal, B., Ubøe, J., & Zhang, T. (2010). Stochastic Partial Differential Equations. Second Edition. Universitext, Springer.
  * [I1] Imkeller, P. (1985). A stochastic calculus for continuous N-parameter strong martingales. Stochastic processes and their applications, 20(1), 1-40.
  * [I2] Imkeller, P. (2006). Two-parameter martingales and their quadratic variation (Vol. 1308). Springer.
  * [ZN] Lanjri Zaidi, N., & Nualart, D. (2002). Backward stochastic differential equations in the plane. Potential Analysis, 16, 373-386.
  * [L] Liang, Z. (1999). Existence and pathwise uniqueness of solutions for stochastic differential equations with respect to martingales in the plane. Stochastic processes and their Applications, 83(2), 303-317.
  * [MS] Mertikopoulos, P., & Staudigl, M. (2018). On the convergence of gradient-like flows with noisy gradient input. SIAM Journal on Optimization, 28(1), 163-197.
  * [N87] Nualart, D. (1987). Some remarks on a linear stochastic differential equation. Statistics & probability letters, 5(3), 231-234.
  * [N] Nualart, D. (2006). The Malliavin calculus and related topics (Vol. 1995, p. 317). Berlin: Springer.
  * [NS] Nualart, D., Sanz, M. (1989). Stochastic differential equations on the plane: Smoothness of the solution. Journal of Multivariate Analysis 31, 1-29.
  * [Ø] Øksendal, B. (2023). Space-time stochastic calculus and white noise. In Morel, J.-M. & Teissier, B. (editors). Mathematics Going Forward. Springer Lecture Notes in Mathematics 2313.
  * [W] Walsh, J. B. (1986). An introduction to stochastic partial differential equations. In École d’Été de Probabilités de Saint Flour XIV-1984, pages 265–439. Springer.
  * [WZ74] Wong, E., & Zakai, M. (1974). Martingales and stochastic integrals for processes with a multi-dimensional parameter. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 29(2), 109-122.
  * [WZ] Wong, E., & Zakai, M. (1978). Differentiation formulas for stochastic integrals in the plane. Stochastic Processes and their Applications, 6(3), 339-349.
  * [Y] Yeh, J. (1981). Existence of strong solutions for stochastic differential equations in the plane. Pacific Journal of Mathematics, 97(1), 217-247.
  * [Y85] Yeh, J. (1985). Existence of weak solutions to stochastic differential equations in the plane with continuous coefficients. Transactions of the American Mathematical Society, 290(1), 345-361.
  * [Y87] Yeh, J. (1987). Uniqueness of strong solutions to stochastic differential equations in the plane with deterministic boundary process. Pacific journal of mathematics, 128(2), 391-400.