# Non-standard Stochastic Control with Nonlinear Feynman-Kac Costs

René Carmona & Mathieu Laurière & Pierre-Louis Lions

###### Abstract.

We consider the conditional control problem introduced by P.L. Lions in his
lectures at the Collège de France in November 2016. In his lectures, Lions
emphasized some of the major differences with the analysis of classical
stochastic optimal control problems, and in so doing, raised the question of
the possible differences between the value functions resulting from
optimization over the class of Markovian controls as opposed to the general
family of open loop controls. The goal of the paper is to elucidate this
quandary and provide elements of response to Lions’ original conjecture.
First, we justify the mathematical formulation of the conditional control
problem by the description of practical model from evolutionary biology. Next,
we relax the original formulation by the introduction of _soft_ as opposed to
hard killing, and using a _mimicking_ argument, we reduce the open loop
optimization problem to an optimization over a specific class of feedback
controls. After proving existence of optimal feedback control functions, we
prove a superposition principle allowing us to recast the original stochastic
control problems as deterministic control problems for dynamical systems of
probability Gibbs measures. Next, we characterize the solutions by forward-
backward systems of coupled non-linear Partial Differential Equations (PDEs)
very much in the spirit of the Mean Field Game (MFG) systems. From there, we
identify a common optimizer, proving the conjecture of equality of the value
functions. Finally we illustrate the results by convincing numerical
experiments.

##  1\. Introduction

In this paper, we consider the conditional control problem introduced by P.L.
Lions in his lectures at the Collège de France in November 2016. See [15]. As
originally stated, the problem does not fit in the usual categories of
stochastic control problems considered in the literature, so its solution
requires new ideas, if not new technology. In his lectures, Lions emphasized
some of the major differences with the analysis of classical stochastic
optimal control problems, and in so doing, raised the question of the possible
differences between the value functions resulting from optimization over the
class of Markovian controls as opposed to the general family of open loop
controls merely assumed to be adapted. The equality of the values of these
optimization problems is accepted as a _folk theorem_ in the classical theory
of stochastic control. However, optimizing an objective function whose values
strongly depend upon the past history of the controlled trajectories of the
system is a strong argument in favor of differences between the optimization
results over these two classes of control processes. The goal of this paper is
to elucidate this quandary and provide elements of response to Lions’ original
conjecture.

A standard stochastic control problem is concerned with the minimization of an
expected cost when the latter is incurred by a controller, and is aggregated
over a specific time horizon. In the problem considered in this paper, the
aggregation over time is done through the integral over the time horizon of
_conditional_ expectations of instantaneous and terminal costs. Instead of
accumulating expected costs incurred by controlling a single stochastic
process over time, the distribution of the process over which the expectation
is computed changes at each time because of the conditional nature of the
expectations. Intuitively, at each time $t$, the expected running cost at that
time could be interpreted as the limit of an expectation over a large particle
system of the Fleming-Viot type. See for example [9]. Still, there does not
seem to be a single particle system which could be used for different times.

This formulation of the optimization is highly unusual. For this reason, we
provide a simple model of evolutionary biology to motivate and justify the
mathematical formulation of the optimization problem. We argue that
egalitarian resource sharing leads naturally to the formulation of a fitness
criterion in terms of time aggregation of conditional expectations of
instantaneous rewards. Such a special form of egalitarian cooperation has been
observed in many species (see for example the study of social spiders in
[11]).

The purpose of the present paper is to provide a thorough analysis of an
instance of this new form of conditional control with complete proofs. Our
approach is to work with a relaxed version of the problem in which we replace
the hard conditioning of Lions original proposal, by a soft conditioning.
Still, the main thrust of the paper is to highlight and take advantage of the
role played by the distribution of the state in the evaluation of the costs.
In both cases (feedback Markovian and open loop controls), we reformulate the
problem as a standard control problem in infinite dimensions, the controlled
dynamics being given by the time evolution of a flow of probability measures
obtained by distorting and renormalizing the original distributions of the
state, pretty much in the same way Gibbs measures are introduced in
statistical physics. See for example [17].

Given the introduction of the problem in [15] and the work [1] on the large
time asymptotics of the Markovian case, the contributions of the present paper
are summarized in the following list: 1) justification of the mathematical
formulation of the conditional control problem by the description of practical
models from evolutionary biology (see Section 2.2); 2) relaxation of the
original exit problem formulation into the analysis of smoother (distorted)
Feynman-Kac semigroups (see Section 2.3); 3) reformulation in the case of
Markovian feedback controls, of the original stochastic control problems as a
deterministic control problems over a space of probability measures, the time
evolution of the states being given by controlled dynamical systems of Gibbs
measures; 4) proof of a non-local superposition principle in Subsection 3.3,
guaranteeing that the deterministic formulation is equivalent to the original
optimization problem; 5) existence of optimal feedback control functions,
derivation of a form of maximum principle; 6) characterization of optimality
by a forward-backward system of non-linear, non-local partial differential
equations (PDEs) very much in the spirit of the Mean Field Game (MFG) systems
[14], and analysis of the system (see Sections 3.1 and 4.4); 7) introduction
of mimicking arguments to reduce the open loop optimization problem to an
optimization over a class of feedback controls (see Theorem 3); 8)
characterization of the solutions by similar forward-backward systems of
coupled non-linear Partial Differential Equations (see Sections 3.6.6 and
4.4.3); 9) identification of the PDE systems and proof of the value equality
conjecture (see Theorem 4); and finally, 9) convincing numerical experiments
illustrating the validity of our result (see Section 5). We managed to prove
the results we were after without proving existence and uniqueness of
solutions of the forward-backward system of non-linear, non-local PDEs
characterizing optimality. So for the sake of completeness, we provide a
complete proof of existence and uniqueness of classical solutions for small
data, namely short time horizon and small terminal condition.

The rest of the paper is organized as follows. In Section 2, we introduce the
problem of optimal control with conditional exit, we provide a motivation, and
we propose an approximate problem with nonlinear Feynman-Kac semigroups, on
which we focus in the sequel. In Section 3, we present a detailed analysis of
the problem with feedback Markovian controls. Among other things, we prove
existence of optimal controls and a non-local superposition principle of
independent interest, see Theorem 1 in Subsection 3.3, and a form of maximum
principle tailored to the present non-local dynamical equations. There, we
also provide a viscosity analysis of the forward-backward PDE system
characterizing the optimum. In Section 4, we analyze the problem with open-
loop controls: we show the equivalence with feedback Markovian controls for an
extended state leading to our proof of the existence of a common optimal
control, and consequently of the equality of the value functions. We conclude
with Section 5 reporting on numerical experiments corroborating our
theoretical result. As mentioned above the appendix contains a detailed proof
of the well posedness (i.e. existence and uniqueness of a classical solution)
for the fundamental forward-backward PDE system in the case of small data
(i.e. time horizon and terminal condition).

_Acknowledgments:_ The first two named authors benefited from the support of
NSF grant DMS-1716673, ARO grant W911NF-17-1-0578, and AFOSR awards
FA9550-19-1-0291 and FA9550-23-1-0324. We would like to thank Dan Lacker for
pointing out to us the relevance of the superposition principle and Samuel
Daudin for providing us with the argument reproduced in Remark 9 on the
equality of the infima.

##  2\. The Conditional Exit Control Problem

For the sake of definiteness, we review the conditional control problem
originally introduced by P.L. Lions. Let $D$ be a bounded open domain in
$\mathbb{R}^{d}$ with a smooth boundary $\partial D$. Let us denote by
$C([0,\infty);\mathbb{R}^{d})$ the space of continuous functions of time
$t\in[0,\infty)$ with values in $\mathbb{R}^{d}$, and by
$C_{0}([0,\infty);\mathbb{R}^{d})$ the subspace of those functions $x\in
C([0,\infty);\mathbb{R}^{d})$ satisfying $x(0)=0$. If $x\in
C([0,\infty);\mathbb{R}^{d})$, we denote by $\tau^{x}_{D}$ the first exit time
of the path $x$ from $D$, namely the quantity:

(1) $\tau^{x}_{D}=\inf\\{t\geq 0;\,x(t)\notin D\\}$

with the convention that $\inf\emptyset=\infty$. We shall skip the superscript
$x$ and/or the subscript $D$ when their values are clear from the context.

###  2.1. The Optimization Problem

We consider an optimization problem which is underpinned by a controlled state
process $\mathbf{X}=(X_{t})_{t\geq 0}$ whose dynamics are given by:

(2) $dX_{t}=\alpha_{t}dt+\sigma dW_{t}$ (3)
$\mathbb{E}\int_{0}^{T}|\alpha_{t}|^{p}dt<\infty,$

for some $p\geq 1$, and we shall use $p=2$ most often.

The goal of the optimization problem is to minimize a cost
$J^{\tau}(\bm{\alpha})$ associated to the control process $\bm{\alpha}$. This
cost is derived from a running cost function $f:\mathbb{R}^{d}\times
A\mapsto\mathbb{R}$ and a terminal cost function
$g:\mathbb{R}^{d}\mapsto\mathbb{R}$ (whose regularity properties will be
specified later on) in the form:

(4)
$\begin{split}J^{\tau}(\bm{\alpha})&=\int_{0}^{T}\mathbb{E}[f(X_{t},\alpha_{t})|\tau^{X}_{D}\geq
t]\;dt+\mathbb{E}[g(X_{T})|\tau^{X}_{D}\geq T]\\\
&=\int_{0}^{T}\frac{\mathbb{E}\Bigl{[}f(X_{t},\alpha_{t})\textbf{1}_{\tau^{X}_{D}\geq
t}\Bigr{]}}{\mathbb{P}[\tau^{X}_{D}\geq
t]}dt+\frac{\mathbb{E}\Bigl{[}g(X_{T})\textbf{1}_{\tau^{X}_{D}\geq
T}\Bigr{]}}{\mathbb{P}[\tau^{X}_{D}\geq T]}.\end{split}$

We shall assume that for each $x\in\mathbb{R}^{d}$, the function
$A\ni\alpha\mapsto f(x,\alpha)$ is convex. Moreover, for illustration
purposes, we shall often restrict ourselves to the case of separable running
cost functions $f$ of the form:

(5) $f(x,\alpha)=\frac{1}{2}|\alpha|^{2}+\tilde{f}(x)$

for some bounded measurable function $\tilde{f}$ on $\mathbb{R}^{d}$.

###  2.2. Evolutionary Biology Motivation

The mathematical formulation of the above optimization problem is non-
standard, and except for the original lectures of P.L. Lions [15] and the
numerical experiments presented in [1], we do not know of any reported
mathematical analysis of such a model. However, we claim that it is very
natural from the perspective of the study of populations of altruistic
individuals in high resource environments practicing egalitarian resource
sharing. The analysis of the evolution of these populations could be based on
dynamical models of the following type. We consider the evolution of identical
individuals foraging for food independently of each other, in a safe
territory. We assume that the outcome of foraging is random, and that at the
end of each time period (one can think of a period as the weaning period for
an offspring generation) the food is shared among the surviving individuals in
an egalitarian manner which allots the same amount of food to each member
still alive.

Let us denote by $D$ the territory, and let us assume that the individuals
disappear or die when they leave the territory. If for $i=1,\cdots,N_{t}$ we
denote by $X^{i}_{t}$ the positions at time $t$ of the $N_{t}$ individuals
still alive at the beginning of period $t+1$, we assume that foraging will
take those who survive (i.e. do not exit the territory) to positions
$X^{i}_{t+1}$ at the end of the period, and that they will have accumulated
the amount $f(X_{t}^{i})$ of resources (say food for example). Resource
sharing takes place in the following form: all the resources are first
aggregated, and then redistributed in equal amounts to the surviving members
of the population. In other words, the resource allocated to each individual
still alive is:

$\frac{1}{N_{t+1}}\sum_{i=1}^{N_{t+1}}f(X^{i}_{t+1}).$

So an individual still alive at the end of the $T$-th period will have
benefitted from the resources:

$\sum_{t=0}^{T-1}\frac{1}{N_{t+1}}\sum_{i=1}^{N_{t+1}}f(X^{i}_{t+1}).$

Not surprisingly, in the limit of small foraging periods, the summation over
time will converge toward the integral between $0$ and $T$ of the resource
enjoyed at time $t$. What is more interesting is the form of the integrand
when the size, say $N$, of the population increases. Indeed, notice first that
the ratio $N_{t}/N$ converges toward the probability that a typical individual
is still alive at time $t$, in other words $\mathbb{P}[\tau>t]$ if we use the
notation $\tau$ for the time of death of the individual. Recall that the
latter is the first exit time of the domain $D$. Next, the quantity

$\frac{1}{N}\sum_{i=1}^{N_{t}}f(X^{i}_{t})=\frac{1}{N}\sum_{i=1}^{N}f(X^{i}_{t})\textbf{1}_{\tau(X^{i})>t}$

converges toward $\mathbb{E}[f(X_{t})\textbf{1}_{\tau>t}]$ and the integrand
at time $t$ is indeed given by the conditional expectation of the resource at
time $t$ given that the individual is still alive at that time. Optimization
of the fitness of the individuals still alive naturally leads to the
conditional control problem which we propose to study in this paper.

###  2.3. Control of Nonlinear Feynman-Kac Semigroups

We shall not solve the model with hard killing through the exit time
introduced above. Instead, we shall provide a complete analysis of a relaxed
version of the model based on _soft killing_. Indeed, it is natural to
consider the following generalization of the original model proposed by P.L.
Lions. Working with the same basic controlled state equation (2), we can
generalize the conditioning by considering a measurable function
$V:\mathbb{R}^{d}\mapsto[0,\infty]$ which we assume to be non-negative for the
sake of simplicity. $V$ could as well be bounded below, or even have some
negative singularities of a specific type, but we shall not worry about this
type of generality in this paper. The goal of the new formulation of the
control problem is still to minimize a cost $J^{V}(\bm{\alpha})$ associated to
a control process $\bm{\alpha}$, and this cost is still derived from a running
cost function $f:\mathbb{R}^{d}\times A\mapsto\mathbb{R}$ and a terminal cost
function $g:\mathbb{R}^{d}\mapsto\mathbb{R}$ in the form:

(6)
$J^{V}(\bm{\alpha})=\int_{0}^{T}\frac{\mathbb{E}\Bigl{[}f(X_{t},\alpha_{t})e^{-\int_{0}^{t}V(X_{s})ds}\Bigr{]}}{\mathbb{E}\Bigl{[}e^{-\int_{0}^{t}V(X_{s})ds}\Bigr{]}}dt+\frac{\mathbb{E}\Bigl{[}g(X_{T})e^{-\int_{0}^{T}V(X_{s})ds}\Bigr{]}}{\mathbb{E}\Bigl{[}e^{-\int_{0}^{T}V(X_{s})ds}\Bigr{]}}.$

Lions’ model based on conditioning the state to remain in a given domain $D$
is recovered by considering the function $V=V^{\infty}$ given by:

(7) $V^{\infty}(x)=\begin{cases}0&\text{if }x\in D\\\
\infty&\text{otherwise},\end{cases}$

in which case:

(8) $\int_{0}^{t}V^{\infty}(X_{s})ds=\begin{cases}0&\text{if
}X_{s}\in\overline{D},\;0\leq s\leq t\\\ \infty&\text{if
}X_{s}\notin\overline{D}\text{ for some }0\leq s\leq t,\end{cases}$

so that:

$e^{-\int_{0}^{t}V^{\infty}(X_{s})ds}=\textbf{1}_{[X_{s}\in\overline{D},\;0\leq
s\leq t]}=\textbf{1}_{[\tau_{D}\geq t]},$

where $\tau_{D}=\tau^{X}_{D}$ is the first exit time of the domain $D$ defined
in (1). Accordingly:

(9)
$J^{V^{\infty}}(\bm{\alpha})=\int_{0}^{T}\mathbb{E}\Bigl{[}f(X_{t},\alpha_{t})\big{|}\tau_{D}\geq
t\Bigr{]}dt+\mathbb{E}\Bigl{[}g(X_{T})\big{|}\tau_{D}\geq T\Bigr{]},$

which is indeed the case considered earlier in (4). In what follows, we
approximate $V^{\infty}$ by potential functions $V^{n}=nV^{1}$ where $V^{1}$
is a continuous approximation of the indicator function of the domain $D$. To
be specific, we choose $V^{1}(x)=\chi^{\epsilon}(d(x,D))$ where $d(x,D)$
denotes the distance from $x\in\mathbb{R}^{d}$ to the domain $D$, $\epsilon>0$
is an arbitrary fixed number whose specific value will not matter, and
$\chi^{\epsilon}$ is the continuous function:

(10) $\chi^{\epsilon}(d)=\begin{cases}0&\text{if }d\leq 0\\\ \text{linear
}&\text{if }0\leq d\leq\epsilon\\\ 1&\text{if }d\geq\epsilon.\end{cases}$

Notice that since we assume that the boundary $\partial D$ is smooth, we can
modify $\chi^{\epsilon}(d)$ when $0\leq d\leq\epsilon$ in such a way that
$\chi^{\epsilon}$ can also be assumed to be smooth. The choice of this family
of potential functions is justified by the following simple result.

######  Lemma 1.

If $\mathbf{X}=(X_{t})_{t\geq 0}$ satisfies
$X_{t}=x_{0}+\int_{0}^{t}\alpha_{s}ds+W_{t}$ for some $x_{0}\in D$ and
$\bm{\alpha}=(\alpha_{t})_{t\geq 0}$ is admissible, then for any bounded
function $g$

(11)
$\mathbb{E}[g(X_{T})\;|\;\tau_{D}>T]=\lim_{n\to\infty}\frac{\mathbb{E}[g(X_{T})e^{-n\int_{0}^{T}V^{1}(X_{s})ds}]}{\mathbb{E}[e^{-n\int_{0}^{T}V^{1}(X_{s})ds}]}.$

Similarly, if $\int_{0}^{T}\mathbb{E}[|f(X_{t},\alpha_{t})|]dt<\infty$, we
also have:

(12)
$\int_{0}^{T}\mathbb{E}[f(X_{t},\alpha_{t})\;|\;\tau_{D}>t]dt=\lim_{n\to\infty}\int_{0}^{T}\frac{\mathbb{E}[f(X_{t},\alpha_{t})e^{-n\int_{0}^{t}V^{1}(X_{s})ds}]}{\mathbb{E}[e^{-n\int_{0}^{t}V^{1}(X_{s})ds}]}.$

###### Proof.

Notice that if $\tau_{D}\geq T$, $X_{t}\in D$ for $0\leq t\leq T$ and
$\int_{0}^{T}V^{1}(X_{s})ds=0$. On the other hand, if $\tau_{D}<T$, the set of
times $t\leq T$ for which $X_{t}\notin D$ is of positive Lebesgue’s measure
which implies that $\int_{0}^{T}V^{1}(X_{s})ds>0$ since $V^{1}(x)>0$ if
$x\notin D$. Consequently,

$\lim_{n\to\infty}e^{-n\int_{0}^{T}V^{1}(X_{s})ds}=\textbf{1}_{\tau_{D}\geq
T}$

and since $g$ is bounded, Lebesgue’s dominated convergence theorem gives:

$\lim_{n\to\infty}\mathbb{E}[g(X_{T})e^{-n\int_{0}^{T}V^{1}(X_{s})ds}]=\mathbb{E}[g(X_{T})\textbf{1}_{\tau_{D}\geq
T}]$

and using the same result with $g\equiv 1$ for the denominator, we get the
desired limit (11). The argument needed for the proof of (12) involving the
running cost is exactly the same. ∎

######  Remark 1.

Notice that the above lemma does not say anything when $x_{0}\notin D$. Indeed
in this case, since the boundary is smooth, we have $\tau_{D}=0$ almost
surely, and the left hand sides of (11) and (12) are referring to conditional
expectations with respect to an event of probability zero.

The advantages of a soft killing given by a bounded continuous potential
function $V$ are twofold: 1) it sets all the equations in the whole space
$\mathbb{R}^{d}$ and avoids having to deal with boundary conditions on
$\partial D$; 2) it makes technical proofs easier as it provides continuity
with respect to the time variable $t$ which may not be available otherwise. To
be specific, we shall make the following assumptions.

###  2.4. Assumptions

Our analysis is predicated on the following assumptions which will be in force
throughout the remainder of the paper, even if many of the individual results
still hold under weaker conditions.

######  Assumption 1.

The running cost and the terminal cost functions satisfy (recall that the
action space $A$ is a closed convex subset of $\mathbb{R}^{d}$):

  * •

The function $g$ is Lipschitz continuous and bounded on $\mathbb{R}^{d}$;

  * •

For each $\alpha\in A$, the function $f(\cdot,\alpha)$ is continuous and
bounded on $\mathbb{R}^{d}$.

  * •

For each $x\in\mathbb{R}^{d}$, the function $f(x,\cdot)$ is convex on $A$.

  * •

There exist $C_{1}>0$ and $C_{2}>0$ such that $C_{1}(1+|\alpha|^{2})\leq
f(x,\alpha)\leq C_{2}(1+|\alpha|^{2}),\;\;x\in\mathbb{R}^{d},\alpha\in A$.

As stated earlier, we shall often concentrate on the case of a separable
running cost function of the form (5) for a bounded Lipschitz continuous
function $\tilde{f}$. As for the potential function $V$, in order to be
specific, we will make the following assumption.

######  Assumption 2.

The function $V$ is Lipschitz continuous on $\mathbb{R}^{d}$ and $0\leq V\leq
1$. In fact, without any loss of generality, we shall assume that it is
continuously differentiable with bounded derivatives when needed.

###  2.5. Approximation by Bounded Controls

In this subsection we work on a probability space
$(\Omega,\mathcal{F},\mathbb{P})$ equipped with a Wiener process
$\mathbf{W}=(W_{t})_{0\leq t\leq T}$ and for each process
$\bm{\alpha}=(\alpha_{t})_{0\leq t\leq T}$ adapted to the filtration of the
Brownian motion $\mathbf{W}$ satisfying the integrability condition (3) with
$p=2$ we consider the corresponding state process $\mathbf{X}=(X_{t})_{0\leq
t\leq T}$ satisfying the state dynamics (2). For the purpose of what we are
about to do, $\bm{\alpha}$ could be a general adapted process, or it could be
given in feedback form $\alpha_{t}=\phi_{t}(X_{t})$ for a measurable function
$\phi$. As we do throughout the paper, we denote by $\nu_{t}$ the distribution
of $X_{t}$ and by $\mu_{t}$ the conditioned probability measure defined on
$\mathbb{R}^{d}$ by:

(13)
$\mu_{t}(dx)=\frac{1}{\mathbb{E}[e^{-A_{t}}]}\mathbb{E}\bigl{[}\delta_{X_{t}}(dx)e^{-A_{t}}\bigr{]}$

with $A_{t}=\int_{0}^{T}V(X_{s})ds$.

For each $K>0$, we denote by $\bm{\alpha}^{K}=(\alpha^{K}_{t})_{0\leq t\leq
T}$ given by $\alpha_{t}^{K}=\alpha_{t}\mathbf{1}{|\alpha_{t}|\leq K}$, by
$\mathbf{X}^{K}=(X^{K}_{t})_{0\leq t\leq T}$ the corresponding state process
and by $A^{K}_{t}=\int_{0}^{T}V(X^{K}_{s})ds$ the corresponding additive
functional.

######  Lemma 2.

For any adapted control process $\bm{\alpha}=(\alpha_{t})_{0\leq t\leq T}$
satisfying

(14) $\mathbb{E}\int_{0}^{T}|\alpha_{t}|^{2}dt<\infty$

we have

(15) $|J^{V}(\bm{\alpha})-J^{V}(\bm{\alpha}^{K})|\leq
C\bigl{[}\varepsilon(K)^{1/2}+\varpi(K)\bigr{]}$

where we used the notations

(16)
$\varepsilon(K)=\mathbb{E}\int_{0}^{T}|\alpha_{t}|^{2}\mathbf{1}_{|\alpha_{t}|>K}dt,\quad\text{and}\quad\varpi(K)=\int_{0}^{T}\mathbb{E}\Bigl{[}|\alpha^{K}_{t}|^{2}|e^{-A_{t}}-e^{-A^{K}_{t}}|\Bigr{]}dt,$

and where the constant $C$ depends only upon the data (i.e. $T$, the sup-norms
of $\tilde{f}$ and $g$, and the Lipschitz constants of $\tilde{f}$, $g$ and
$V$). In particular, since both $\varepsilon(K)$ and $\varpi(K)$ converge to
$0$ when $K\nearrow\infty$, we have:

(17) $\lim_{K\nearrow\infty}J^{V}(\bm{\alpha}^{K})=J^{V}(\bm{\alpha}).$

###### Proof.

For the sake of simplicity, we only consider the case of separable running
costs given by (5).

(18)
$\begin{split}J^{V}(\bm{\alpha})-J^{V}(\bm{\alpha}^{K})&=\int_{0}^{T}\frac{1}{\mathbb{E}[e^{-A_{t}}]}\mathbb{E}\bigl{[}\bigl{(}f(X_{t},\alpha_{t})-f(X^{K}_{t},\alpha^{K}_{t})\bigr{)}e^{-A_{t}}\bigr{]}dt+\frac{\mathbb{E}\bigl{[}\bigl{(}g(X_{T})-g(X^{K}_{T})\bigr{)}e^{-A_{T}}\bigr{]}}{\mathbb{E}[e^{-A_{T}}]}\\\
&+\int_{0}^{T}\mathbb{E}\Bigl{[}f(X^{K}_{t},\alpha^{K}_{t})\Bigl{(}\frac{e^{-A_{t}}}{\mathbb{E}[e^{-A_{t}}]}-\frac{e^{-A^{K}_{t}}}{\mathbb{E}[e^{-A^{K}_{t}}]}\Bigr{)}\Bigr{]}dt+\mathbb{E}\Bigl{[}g(X^{K}_{T})\Bigl{(}\frac{e^{-A_{T}}}{\mathbb{E}[e^{-A_{T}}]}-\frac{e^{-A^{K}_{T}}}{\mathbb{E}[e^{-A^{K}_{T}}]}\Bigr{)}\Bigr{]}\\\
&=(i)+(ii)+(iii)+(iv).\end{split}$

Notice that

(19)
$\begin{split}\mathbb{E}\int_{0}^{T}|X_{t}-X^{K}_{t}|dt&\leq\mathbb{E}\int_{0}^{T}\int_{0}^{t}|\alpha_{s}-\alpha^{K}_{s}|ds\;dt\\\
&=\mathbb{E}\int_{0}^{T}\int_{0}^{t}|\alpha_{s}|\mathbf{1}_{|\alpha_{s}|>K}ds\;dt\\\
&\leq
T\mathbb{E}\int_{0}^{T}|\alpha_{t}|\mathbf{1}_{|\alpha_{t}|>K}dt,\end{split}$

and that

(20) $\begin{split}|e^{-A_{t}}-e^{-A^{K}_{t}}|&\leq|A_{t}-A^{K}_{t}|\\\
&=\bigl{|}\int_{0}^{t}[V(X_{s})-V(X^{K}_{s})]ds\bigr{|}\\\
&\leq\|V\|_{Lip1}\int_{0}^{t}|X_{s}-X^{K}_{s}|ds\\\ &\leq
T\|V\|_{Lip1}\int_{0}^{T}|\alpha_{t}|\mathbf{1}_{|\alpha_{t}|>K}dt.\end{split}$

For every $K>0$ and every $(t,\omega)\in[0,T]\times\Omega$ we have
$|\alpha^{K}_{t}|^{2}|e^{-A_{t}}-e^{-A^{K}_{t}}|\leq 2|\alpha_{t}|^{2}$ whose
integral with respect to $dt\mathbb{P}(d\omega)$ is finite. Moreover,
$\lim_{K\nearrow\infty}\int_{0}^{T}|\alpha_{t}|\mathbf{1}_{|\alpha_{t}|>K}dt=0$
$\mathbb{P}$-almost surely proving that

$\lim_{K\nearrow\infty}\varpi(K)=0.$

Now,

(21)
$\begin{split}|(i)|&\leq\int_{0}^{T}\frac{1}{\mathbb{E}[e^{-A_{t}}]}\mathbb{E}\bigl{[}\bigl{|}\tilde{f}(X_{t})-\tilde{f}(X^{K}_{t})\bigr{|}e^{-A_{t}}\bigr{]}dt+\frac{1}{2}\int_{0}^{T}\frac{1}{\mathbb{E}[e^{-A_{t}}]}\mathbb{E}\bigl{[}\bigl{(}|\alpha_{t}|^{2}-|\alpha^{K}_{t}|^{2}\bigr{)}e^{-A_{t}}\bigr{]}dt\\\
&\leq
e^{T}\|\tilde{f}\|_{Lip1}\mathbb{E}\int_{0}^{T}|X_{t}-X^{K}_{t}|dt+\frac{1}{2}\int_{0}^{T}\frac{1}{\mathbb{E}[e^{-A_{t}}]}\mathbb{E}\bigl{[}\bigl{|}|\alpha_{t}|^{2}\mathbf{1}_{|\alpha_{t}|>K}e^{-A_{t}}\bigr{]}dt\\\
&\leq
Te^{T}\|\tilde{f}\|_{Lip1}\mathbb{E}\int_{0}^{T}|\alpha_{t}|\mathbf{1}_{|\alpha_{t}|>K}dt+\frac{e^{T}}{2}\mathbb{E}\int_{0}^{T}|\alpha_{t}|^{2}\mathbf{1}_{|\alpha_{t}|>K}dt\\\
&\leq\frac{e^{T}}{2}\varepsilon(K)+T^{3/2}e^{T}\|\tilde{f}\|_{Lip1}\varepsilon(K)^{1/2}\end{split}$

if we use the notation introduced in (16) for $\varepsilon(K)$, and use
Hölder’s inequality. Similarly

(22) $\begin{split}|(ii)|&\leq
e^{T}\|g\|_{Lip1}\mathbb{E}[|X_{T}-X^{K}_{T}|]\\\ &\leq
e^{T}\|g\|_{Lip1}\mathbb{E}\int_{0}^{T}|\alpha_{t}|\mathbf{1}_{|\alpha_{t}|>K}dt\\\
&\leq e^{T}\|g\|_{Lip1}\varepsilon(K)^{1/2}.\end{split}$

Next

(23)
$\begin{split}|(iii)|&\leq\int_{0}^{T}\mathbb{E}\Bigl{[}|\tilde{f}(X^{K}_{t})|\Bigl{|}\frac{e^{-A_{t}}}{\mathbb{E}[e^{-A_{t}}]}-\frac{e^{-A^{K}_{t}}}{\mathbb{E}[e^{-A^{K}_{t}}]}\Bigr{|}\Bigr{]}dt+\frac{1}{2}\int_{0}^{T}\mathbb{E}\Bigl{[}|\alpha^{K}_{t}|^{2}\Bigl{|}\frac{e^{-A_{t}}}{\mathbb{E}[e^{-A_{t}}]}-\frac{e^{-A^{K}_{t}}}{\mathbb{E}[e^{-A^{K}_{t}}]}\Bigr{|}\Bigr{]}dt\\\
&\leq\|\tilde{f}\|_{\infty}\int_{0}^{T}\Bigl{(}\frac{1}{\mathbb{E}[e^{-A_{t}}]}\mathbb{E}[|e^{-A_{t}}-e^{-A^{K}_{t}}|]+\mathbb{E}[e^{-A^{K}_{t}}]\Bigl{(}\frac{1}{\mathbb{E}[e^{-A_{t}}]}-\frac{1}{\mathbb{E}[e^{-A^{K}_{t}}]}\Bigr{)}\Bigr{)}dt\\\
&\hskip
45.0pt+\frac{1}{2}\int_{0}^{T}\frac{1}{\mathbb{E}[e^{-A^{K}_{t}}]}\mathbb{E}\Bigl{[}|\alpha^{K}_{t}|^{2}\Bigl{|}e^{-A_{t}}-e^{-A^{K}_{t}}\Bigr{|}\Bigr{]}dt+\frac{1}{2}\int_{0}^{T}\mathbb{E}\Bigl{[}|\alpha^{K}_{t}|^{2}e^{-A_{t}}\Bigl{|}\frac{1}{\mathbb{E}[e^{-A_{t}}]}-\frac{1}{\mathbb{E}[e^{-A^{K}_{t}}]}\Bigr{|}\Bigr{]}dt\\\
&\leq
2e^{T}\|\tilde{f}\|_{\infty}\int_{0}^{T}\mathbb{E}[|e^{-A_{t}}-e^{-A^{K}_{t}}|]dt\\\
&\hskip
45.0pt+\frac{e^{T}}{2}\int_{0}^{T}\mathbb{E}\Bigl{[}|\alpha^{K}_{t}|^{2}|e^{-A_{t}}-e^{-A^{K}_{t}}|\Bigr{]}dt+\frac{e^{T}}{2}\int_{0}^{T}\mathbb{E}\Bigl{[}|\alpha^{K}_{t}|^{2}\frac{e^{-A_{t}}}{\mathbb{E}[e^{-A_{t}}]}\Bigr{]}\mathbb{E}[|e^{-A_{t}}-e^{-A^{K}_{t}}|]dt\\\
&\leq\bigl{(}2e^{T}\|\tilde{f}\|_{\infty}+\frac{T^{3/2}}{2}e^{2T}\|V\|_{Lip1}\varepsilon(K)^{1/2}+\frac{e^{T}}{2}\int_{0}^{T}\mathbb{E}\Bigl{[}|\alpha^{K}_{t}|^{2}|e^{-A_{t}}-e^{-A^{K}_{t}}|\Bigr{]}dt.\end{split}$

Finally

(24)
$\begin{split}|(iv)|&\leq\mathbb{E}\bigl{[}|g(X^{K}_{t})|\frac{1}{\mathbb{E}[e^{-A_{T}}]}|e^{-A_{T}}-e^{-A^{K}_{T}}|\bigr{]}+\mathbb{E}\bigl{[}|g(X^{K}_{t})|\;e^{-A^{K}_{T}}\bigl{|}\frac{1}{\mathbb{E}[e^{-A_{T}}]}-\frac{1}{\mathbb{E}[e^{-A^{K}_{T}}]}\bigr{|}\bigr{]}\\\
&\leq\|g\|_{\infty}e^{T}\mathbb{E}[|e^{-A_{T}}-e^{-A^{K}_{T}}|]+\|g\|_{\infty}e^{2T}\mathbb{E}[|e^{-A_{T}}-e^{-A^{K}_{T}}|]\\\
&\leq 2\|g\|_{\infty}e^{2T}T\|V\|_{Lip1}\varepsilon(K)^{1/2}.\end{split}$

Putting together (21), (22), (23), and (24) gives the desired estimate (15).
Clearly,

$\lim_{K\nearrow\infty}\varepsilon(K)=0,$

and we already argued that $\varpi(K)$ also converges to $0$. ∎

The first obvious consequence of the above result is that the search for the
infimum

(25) $J^{V*}=\inf_{\bm{\alpha}}J^{V}(\bm{\alpha})$

over all the adapted processes satisfying the integrability condition (3) with
$p=2$ can be restricted to bounded control processes $\bm{\alpha}$. However,
if and when we are able to prove that this infimum is attained, it will
require extra work to prove that the minimizer is in fact a bounded control.

##  3\. The Case of Markovian Feedback Controls

###  3.1. Formulation of the problem

In this section, we restrict the optimization problem to controls processes
$\bm{\alpha}=(\alpha_{t})_{0\leq t\leq T}$ of the form:

(26) $\alpha_{t}=\phi_{t}(X_{t})$

where $\phi:[0,T]\times\mathbb{R}^{d}\mapsto A$ is a (deterministic)
measurable function, and $\mathbf{X}=(X_{t})_{0\leq t\leq T}$ (which we shall
sometimes denote $\mathbf{X}^{\phi}$ to emphasize the dependence upon the
feedback function $\phi$) satisfies:

(27) $dX_{t}=\phi_{t}(X_{t})dt+dW_{t}.$

We will refer to such controls as _Markovian_ controls or _feedback_ controls.
For a given feedback control function $\phi$, the existence of the controlled
state process requires the solution of a stochastic differential equation. So
in order to be admissible, a feedback function should be at a minimum, a
measurable functions $\phi$ for which such a solution exists and satisfies

(28) $\mathbb{E}\Bigl{[}\int_{0}^{T}|\phi_{t}(X_{t})|^{p}\Bigr{]}<\infty,$

with $p=2$ in order for the objective function we plan to minimize to make
sense. We shall denote by $\Phi^{(p)}$ the set of admissible feedback control
functions, namely the set of $\mathbb{R}^{d}$-valued measurable functions
$\phi$ on $[0,T]\times\mathbb{R}^{d}$ for which the stochastic differential
equation (27) has a weak solution satisfying (28). In some cases, we shall
make a stronger assumption on the feedback controls. For this reason, we
introduce the space $\Phi^{(\infty)}$ of $\mathbb{R}^{d}$-valued bounded
measurable functions on $[0,T]\times\mathbb{R}^{d}$. In dimension $d=1$, all
the measurable bounded functions $\phi$ are in all the $\Phi^{(p)}$ because of
the classical result of Zvonkin [22] which guarantees existence of strong
solutions for (27). In dimension $d>1$ existence and uniqueness of a strong
solutions still hold for bounded drifts because the volatility is the identity
matrix. More general volatility terms could be accommodated by Veretennikov’s
extension [21] which provides a large class of bounded measurable functions
$\phi$ for which the result still holds in more general non-degenerate cases.
However, as we shall point out in several instances, our main requirement is
the existence of weak solutions, and this is guaranteed under much weaker
conditions on $\phi$ as long as the volatility is the identity matrix.

Throughout the section, we alternatively use the notation $\bm{\phi}$ or
$\bm{\alpha}$ for the control, even if $\phi$ is only the feedback function
determining the actual control process $\bm{\alpha}$ through the solution of
the equation (27).

###  3.2. A Non-local Fokker-Planck-Kolmogorov (FPK) Equation

Given the form of the running and terminal costs, the objective function (6)
can be rewritten in the form:

(29) $J(\phi)=\int_{0}^{T}\int f(x,\phi_{t}(x))\mu_{t}(dx)\;dt+\int
g(x)\mu_{T}(dx),$

where we use the notation $\mu_{t}$ for the probability measure:

(30)
$\mu_{t}(dx)=\frac{\mathbb{E}[\delta_{X_{t}}(dx)e^{-A_{t}}]}{\mathbb{E}[e^{-A_{t}}]},\qquad
0\leq t\leq T,$

where from now on, we use the notation $A_{t}=\int_{0}^{t}V(X_{s})ds$. Notice
that $t\mapsto\mu_{t}$ is continuous for the topology of weak convergence of
probability measures, so in the sequel, we shall restrict ourselves to
continuous flows of probability measures.

######  Lemma 3.

The measure valued function $t\mapsto\mu_{t}$ defined in (30) satisfies the
(non-local) forward Fokker-Planck-Kolmogorov (FPK) equation:

(31)
$\partial_{t}\mu_{t}=\frac{1}{2}\Delta\mu_{t}-\operatorname{div}(\phi_{t}\mu_{t})-(V-<\mu_{t},V>)\mu_{t},$

in the sense of Schwartz distributions.

The notation $<\mu,V>$ used in formula (31) stands for
$<\mu,V>=\int_{\mathbb{R}^{d}}V(x)\mu(dx)$.

###### Proof.

If $\varphi$ is a smooth function on $\mathbb{R}^{d}$ with compact support,
Itô’s formula gives:

$\begin{split}\frac{d}{dt}<\mu_{t},\varphi>&=\frac{d}{dt}\frac{\mathbb{E}[\varphi(X_{t})e^{-A_{t}}]}{\mathbb{E}[e^{-A_{t}}]}\\\
&=\frac{1}{\mathbb{E}[e^{-A_{t}}]}\mathbb{E}\Bigl{[}\Bigl{(}\frac{1}{2}\Delta_{x}\varphi(X_{t})+\phi_{t}(X_{t})\cdot\nabla_{x}\varphi(X_{t})\Bigr{)}e^{-A_{t}}\\\
&\hskip
75.0pt-\varphi(X_{t})V(X_{t})e^{-A_{t}}\Bigr{]}+\mathbb{E}[\varphi(X_{t})e^{-A_{t}}]\frac{\mathbb{E}[V(X_{t})e^{-A_{t}}]}{\mathbb{E}[e^{-A_{t}}]^{2}}\\\
&=<\mu_{t},\;\frac{1}{2}\Delta_{x}\varphi+\phi_{t}\cdot\nabla_{x}\varphi-(V-<\mu_{t},V>)\varphi>\\\
&=<\frac{1}{2}\Delta_{x}\mu_{t}-\operatorname{div}_{x}(\phi_{t}\mu_{t})-(V-<\mu_{t},V>)\mu_{t},\;\varphi>\end{split}$

where we used stochastic integration by parts and the fact that $\varphi$ has
compact support. ∎

For general existence and uniqueness results for classical, i.e. without the
third term in the right hand side of (31), FPK equations, together with
existence and regularity results for possible density for the solutions, we
refer the interested reader to [5, Chapter 6] and references therein. However,
these results are not general enough to cover the case of equation (31) of
interest to us. Indeed, as far as we can tell, the mean field nature of
equation (31) makes it escape the realm of those results. For the purpose of
our analysis, we proceed in the following way. Given a measurable feedback
function $\phi$ for which there exists a weak solution $X_{t}$ of the
stochastic differential equation

(32) $dX_{t}=\phi_{t}(X_{t})dt+dW_{t},$

Lemma 3 guarantees that the flow of probability measures $\mu_{t}$ defined by
(30) with $A_{t}=\int_{0}^{t}V(X_{s})ds$ is continuous and is a solution of
(31). This is mostly what we shall need for the existence of solutions of
(31). Still the following properties of such solutions will come handy in the
sequel.

######  Proposition 1.

Let us assume that $\bm{\phi}=\bigl{(}\phi_{t}(x)\bigr{)}_{0\leq t\leq
T,\;x\in\mathbb{R}^{d}}$ is an $\mathbb{R}^{d}$-valued measurable function on
$[0,T]\times\mathbb{R}^{d}$ and $\bm{\mu}=(\mu_{t})_{0\leq t\leq T}$ is a
measurable flow of probability measures satisfying

(33) $\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi_{t}(x)|^{p}\mu_{t}(dx)dt<\infty,$

for some $p\geq 1$. If $\bm{\mu}=(\mu_{t})_{0\leq t\leq T}$ solves the Fokker-
Planck-Kolmogorov equation (31) in the sense of distributions, then

  * (i)

$\bm{\mu}(dx,dt)=\mu_{t}(dx)dt=\rho_{t}(x)dxdt$ for some non-negative
measurable function $\rho\in L^{r}_{loc}([0,T]\times\mathbb{R}^{d})$ for every
$r\in[1,(d+2)^{\prime})$.

  * (ii)

If $p>d+2$, the density $\rho$ can be chosen to belong to the Sobolev spaces
$\mathcal{H}^{1,p}(I\times K)$ for every closed interval $I\subset(0,T)$ and
compact set $K\subset\mathbb{R}^{d}$.

  * (iii)

If $\phi$ is bounded, the continuous version of the density is strictly
positive, hence bounded below away from $0$ on every closed interval
$I\subset(0,T)$ and compact set $K\subset\mathbb{R}^{d}$.

###### Proof.

We use a prime $p^{\prime}$ to denote the conjugate exponent of
$p\in[0,\infty)$. (i) is a direct consequence of [5, Corollary 6.3.2] with
$c=V-<\mu_{t},V>$. (ii) is a direct consequence of [5, Corollary 6.4.3], given
the fact that we use the definition of the space $\mathcal{H}^{1,p}$ which can
be found on page 245 of this book. As for (iii), it follows directly from the
properties of $\nu_{t}=\mathcal{L}(X_{t})$. ∎

###  3.3. A Form of Superposition Principle

In this subsection, we start with a couple $(\bm{\phi},\bm{\mu})$ where
$\bm{\phi}=\bigl{(}\phi_{t}(x)\bigr{)}_{0\leq t\leq T,\;x\in\mathbb{R}^{d}}$
is a $\mathbb{R}^{d}$-valued measurable function on
$[0,T]\times\mathbb{R}^{d}$ and $\bm{\mu}=(\mu_{t})_{0\leq t\leq T}$ is a
measurable flow of probability measures satisfying

(34)
$\int_{0}^{T}\int_{\mathbb{R}^{d}}\;|\phi_{t}(x)|^{2}\mu_{t}(dx)dt<\infty$

and the non-local FPK equation (31), and we construct a stochastic process
$\mathbf{X}=(X_{t})_{0\leq t\leq T}$ solution of the state stochastic
differential equation (27) which is related to $\mu_{t}$ through formula (30).
This result should be viewed as a non-linear or non-local superposition
principle in the spirit of the superposition principle proved by Trevisan in
[20] in the classical case.

Notice that since the potential function $V$ is bounded, assumption (34) is
equivalent to our prior assumption (28).

######  Lemma 4.

If $(\mu_{t})_{0\leq t\leq T}$ solves the FPK equation (31), then the flow
$(\gamma_{t})_{0\leq t\leq T}$ of non-negative measures defined by

(35) $\gamma_{t}=e^{-\int_{0}^{t}<\mu_{s},V>ds}\mu_{t}$

is the unique solution of the linear PDE

(36)
$\partial_{t}\gamma_{t}=\frac{1}{2}\Delta\gamma_{t}-\operatorname{div}(\phi_{t}\gamma_{t})-V\gamma_{t}$

with initial condition $\gamma_{0}=\mu_{0}$.

###### Proof.

If $\gamma_{t}$ is defined by (35) in Lemma 4, and $\varphi$ is a test
function, we have

$\begin{split}\frac{d}{dt}<\gamma_{t},\varphi>&=<\mu_{t},V>e^{\int_{0}^{t}<\mu_{s},V>ds}<\mu_{t},\varphi>+e^{\int_{0}^{t}<\mu_{s},V>ds}\frac{d}{dt}<\mu_{t},\varphi>\\\
&=-<\mu_{t},V><\gamma_{t},\varphi>+e^{-\int_{0}^{t}<\mu_{s},V>ds}<\mu_{t},\frac{1}{2}\Delta\varphi+\phi_{t}\nabla\varphi-(V-<\mu_{t},V>)\varphi>\\\
&=e^{-\int_{0}^{t}<\mu_{s},V>ds}<\mu_{t},\frac{1}{2}\Delta\varphi+\phi_{t}\nabla\varphi-V\varphi>\\\
&=<\gamma_{t},\frac{1}{2}\Delta\varphi+\phi_{t}\nabla\varphi-V\varphi>.\end{split}$

Uniqueness is a consequence of [5, Theorem 9.3.6] and the fact that
$\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi_{t}(x)|\gamma_{t}(dx)dt<\infty$ which
is implied by our assumption (34) and the definition (35). ∎

######  Lemma 5.

There exists a weak solution $(X_{t})_{0\leq t\leq T}$ of the stochastic
differential equation (27) for which

(37)
$\gamma_{t}(dx)=\mathbb{E}\bigl{[}\delta_{X_{t}}(dx)e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}.$

###### Proof.

For each $\epsilon>0$, let $\rho_{\epsilon}$ be the $d$-dimensional Gaussian
density with mean $0$ and variance $\epsilon$ times the identity matrix in
dimension $d$, and for each $t\in[0,T]$, let

(38)
$\gamma_{t}^{\epsilon}=\gamma_{t}*\rho_{\epsilon},\qquad\text{and}\qquad\phi_{t}^{\epsilon}=\frac{d\bigl{(}(\phi_{t}\gamma_{t})*\rho_{\epsilon}\bigr{)}}{d(\gamma_{t}*\rho_{\epsilon})},\qquad\text{and}\qquad
V^{\epsilon}_{t}=\frac{d\bigl{(}(V\gamma_{t})*\rho_{\epsilon}\bigr{)}}{d(\gamma_{t}*\rho_{\epsilon})}.$

$\gamma^{\epsilon}_{t}$ is the unique solution of the linear FPK equation

(39)
$\partial\gamma_{t}^{\epsilon}=\frac{1}{2}\Delta\gamma_{t}^{\epsilon}-\operatorname{div}(\phi^{\epsilon}_{t}\gamma_{t}^{\epsilon})-V^{\epsilon}_{t}\gamma_{t}^{\epsilon}.$

Indeed, for each test function $\varphi\in C^{\infty}_{c}$ we have:

$\begin{split}\partial_{t}<\varphi,\gamma_{t}^{\epsilon}>&=\partial_{t}<\varphi*\rho_{\epsilon},\gamma_{t}>\\\
&=<\frac{1}{2}\Delta(\varphi*\rho_{\epsilon})+\phi_{t}\nabla(\varphi*\rho_{\epsilon})-V(\varphi*\rho_{\epsilon}),\gamma_{t}>\\\
&=<\frac{1}{2}\Delta\varphi,\gamma_{t}*\rho_{\epsilon}>+<\nabla\varphi,(\phi_{t}\gamma_{t})*\rho_{\epsilon}>-<\varphi,(V\gamma_{t})*\rho_{\epsilon}>\\\
&=<\frac{1}{2}\Delta\varphi+\frac{d\bigl{(}(\phi_{t}\gamma_{t})*\rho_{\epsilon}\bigr{)}}{d\gamma_{t}^{\epsilon}}\nabla\varphi-\frac{d\bigl{(}(V\gamma_{t})*\rho_{\epsilon}\bigr{)}}{d\gamma_{t}^{\epsilon}}\varphi,\gamma_{t}^{\epsilon}>\\\
&=<\varphi,\frac{1}{2}\Delta\gamma_{t}^{\epsilon}-\operatorname{div}\bigl{(}\phi^{\epsilon}_{t}\gamma_{t}^{\epsilon}\bigr{)}-V_{t}^{\epsilon}\gamma_{t}^{\epsilon}>.\end{split}$

Using [3, Lemma 8.1.10] we get

(40)
$\int_{0}^{T}\int_{\mathbb{R}^{d}}|\bm{\phi}_{t}^{\epsilon}|^{2}d\gamma_{t}^{\epsilon}dt\leq\int_{0}^{T}\int_{\mathbb{R}^{d}}|\bm{\phi}_{t}|^{2}d\gamma_{t}dt\leq\int_{0}^{T}\int_{\mathbb{R}^{d}}|\bm{\phi}_{t}|^{2}d\mu_{t}dt<\infty$

because of (34). Using once more [5, Theorem 9.3.6], we conclude that for each
$\epsilon>0$, $(\gamma_{t}^{\epsilon})_{0\leq t\leq T}$ is the unique solution
of (39).

For each $\epsilon>0$, [5, Theorem 6.6.2] gives existence of a narrowly
continuous flow $(\nu^{\epsilon}_{t})_{0\leq t\leq T}$ of probability measures
satisfying

$\partial_{t}\nu^{\epsilon}_{t}=\frac{1}{2}\Delta\nu^{\epsilon}_{t}-\operatorname{div}(\phi^{\epsilon}_{t}\nu^{\epsilon}_{t})$

and using the superposition theorem (see for example [20, Theorem 2.5]) we
deduce the existence of a probability measure $\mathbb{P}^{\epsilon}$ solving
the martingale problem in the sense that for every test function $\varphi\in
C^{\infty}_{c}(\mathbb{R}^{d})$,

(41)
$\varphi(X_{t})-\varphi(X_{0})-\int_{0}^{t}[\frac{1}{2}\Delta\varphi(X_{s})+\phi^{\epsilon}_{s}(X_{s})\nabla\varphi(X_{s})]ds$

where $(X_{t})_{0\leq t\leq T}$ is the coordinate process on
$C([0,T];\mathbb{R}^{d})$, is a $\mathbb{P}^{\epsilon}$-martingale for the
canonical filtration of $C([0,T];\mathbb{R}^{d})$, and for each $t\in[0,T]$,
$\nu^{\epsilon}_{t}$ is the law of $X_{t}$ under $\mathbb{P}^{\epsilon}$.

We conclude the proof assuming momentarily the result of Lemma 6 below.
$\mathbb{P}$ being a solution of the martingale problem for the drift
$\phi_{t}$, for each test function $\varphi\in
C^{\infty}_{c}(\mathbb{R}^{d})$, integration by parts implies that

(42)
$\varphi(X_{t})e^{-\int_{0}^{t}V(X_{s})ds}-\varphi(X_{0})-\int_{0}^{t}e^{-\int_{0}^{s}V(X_{r})dr}\bigl{[}\frac{1}{2}\Delta\varphi(X_{s})+\phi_{s}(X_{s})\nabla\varphi(X_{s})-V(X_{s})\varphi(X_{s})\bigr{]}ds$

is a zero-expectation $\mathbb{P}$-martingale. So if we define the flow of
measures $(\tilde{\gamma}_{t})_{0\leq t\leq T}$ by

$\tilde{\gamma}_{t}(dx)=\mathbb{E}^{\mathbb{P}}\bigl{[}\delta_{X_{t}}(dx)e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}$

then, taking $\mathbb{P}$-expectation of (42) we get

$<\varphi,\tilde{\gamma}_{t}>=<\varphi,\tilde{\gamma}_{0}>+\int_{0}^{t}<\frac{1}{2}\Delta\varphi+\phi_{t}\nabla\varphi-V\varphi,\tilde{\gamma}_{s}>ds$

which shows that $\tilde{\gamma}_{T}$ satisfies the PDE (36), and by
uniqueness, that $\tilde{\gamma}_{t}=\gamma_{t}$ and consequently, that
formula (37) holds. ∎

######  Lemma 6.

The family $(\mathbb{P}^{\epsilon})_{\epsilon>0}$ is tight and any limit point
$\mathbb{P}$ solves the martingale problem (41) with $(\phi_{t})_{0\leq t\leq
T}$ instead of $(\phi^{\epsilon}_{t})_{0\leq t\leq T}$

###### Proof.

_Step 1. Tightness._ For $\delta>0$, we denote by
$\omega_{x}(\delta)=\sup_{0\leq s,t\leq T,\;|t-s|\leq\delta}|x(t)-x(s)|$ the
$\delta$-modulus of continuity of any function $x\in C([0,T];\mathbb{R}^{d})$.
For $\delta^{\prime}>0$ we have

$\begin{split}\mathbb{P}^{\epsilon}[\omega_{\cdot}(\delta)>\delta^{\prime}]&\leq\mathbb{P}^{\epsilon}\Bigl{[}\sup_{|t-s|\leq\delta}|\int_{s}^{t}\phi^{\epsilon}_{r}(X_{r})dr|>\delta^{\prime}/2]+\mathbb{P}^{\epsilon}[\sup_{|t-s|\leq\delta}|W_{t}-W_{s}|>\delta^{\prime}/2\Bigr{]}\\\
&\leq\mathbb{P}^{\epsilon}\Bigl{[}\sup_{|t-s|\leq\delta}\delta^{1/2}\bigl{(}\int_{0}^{T}|\phi^{\epsilon}_{r}(X_{r})|^{2}dr\bigr{)}^{1/2}>\delta^{\prime}/2\Bigr{]}+\mathbb{P}^{\epsilon}\bigl{[}\sup_{0\leq
r\leq\delta}|W_{r}|>\delta^{\prime}/2\bigr{]}\\\
&\leq\frac{4\delta}{\delta^{\prime
2}}\mathbb{E}^{\mathbb{P}^{\epsilon}}\Bigl{[}\int_{0}^{T}|\phi^{\epsilon}_{r}(X_{r})|^{2}dr\Bigr{]}+o(\delta)\\\
&\leq\frac{4\delta}{\delta^{\prime
2}}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi_{t}^{\epsilon}(x)|^{2}\nu^{\epsilon}_{t}(dx)dt+o(\delta)\end{split}$

and we conclude that for each $\delta^{\prime}>0$, we have

$\lim_{\delta\searrow
0}\sup_{\epsilon>0}\mathbb{P}^{\epsilon}[\omega_{\cdot}(\delta)>\delta^{\prime}]=0$

because

$\sup_{\epsilon}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi_{t}^{\epsilon}(x)|^{2}\nu^{\epsilon}_{t}(dx)dt\leq\sup_{\epsilon}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi_{t}^{\epsilon}(x)|^{2}\gamma^{\epsilon}_{t}(dx)dt<\infty$

is implied by (40). Given the fact that the probability measures
$(\nu^{\epsilon}_{0})_{\epsilon}$ are tight because the
$(\gamma^{\epsilon}_{0})_{\epsilon}$ are, we conclude that the
$(\mathbb{P}^{\epsilon})_{\epsilon}$ are themselves tight on
$C([0,T];\mathbb{R}^{d})$.

_Step 2. Any limit point $\mathbb{P}$ solves the martingale problem for
$(\phi_{t})_{0\leq t\leq T}$. _ Let us fix $0\leq s<t\leq T$ and let $Y$ be a
bounded continuous function on $C([0,T];\mathbb{R}^{d})$ measurable with
respect to the past up to time $s$. For any test function $\varphi\in
C^{\infty}_{c}(\mathbb{R}^{d})$ and for aany $\epsilon>0$ we have:

$0=\mathbb{E}^{\mathbb{P}^{\epsilon}}\Bigl{[}Y\bigl{[}\varphi(X_{t})-\varphi(X_{s})-\int_{s}^{t}[\frac{1}{2}\Delta\varphi(X_{r})+\phi^{\epsilon}_{r}(X_{r})\nabla\varphi(X_{r})]dr\bigr{]}\Bigr{]}.$

In order to prove the same equality for $\mathbb{P}$ and $\phi$ instead of
$\mathbb{P}^{\epsilon}$ and $\phi^{\epsilon}$, it is enough to prove:

(43) $\lim_{\epsilon\searrow
0}\mathbb{E}^{\mathbb{P}^{\epsilon}}\Bigl{[}Y\int_{s}^{t}\phi^{\epsilon}_{r}(X_{r})\nabla\varphi(X_{r})]dr\Bigr{]}=\mathbb{E}^{\mathbb{P}}\Bigl{[}Y\int_{s}^{t}\phi_{r}(X_{r})\nabla\varphi(X_{r})]dr\Bigr{]}.$

Given a bounded Lipschitz continuous function $\tilde{\phi}\in
C([0,T]\times\mathbb{R}^{d})$ with compact support, we have:

(44) $\lim_{\epsilon\searrow
0}\mathbb{E}^{\mathbb{P}^{\epsilon}}\Bigl{[}Y\int_{s}^{t}\tilde{\phi}_{r}(X_{r})\nabla\varphi(X_{r})]dr\Bigr{]}=\mathbb{E}^{\mathbb{P}}\Bigl{[}Y\int_{s}^{t}\tilde{\phi}_{r}(X_{r})\nabla\varphi(X_{r})]dr\Bigr{]}$

since $\mathbb{P}^{\epsilon}$ converges weakly toward $\mathbb{P}$. So in
order to prove (43) it is be enough to prove

(45) $\lim_{\epsilon\searrow
0}\mathbb{E}^{\mathbb{P}^{\epsilon}}\Bigl{[}Y\int_{s}^{t}[\phi^{\epsilon}_{r}(X_{r})-\tilde{\phi}_{r}(X_{r})]\nabla\varphi(X_{r})]dr\Bigr{]}=0$

if one can choose $\tilde{\phi}$ so that the expectation

(46)
$\mathbb{E}^{\mathbb{P}}\Bigl{[}Y\int_{s}^{t}[\phi_{r}(X_{r})-\tilde{\phi}_{r}(X_{r})]\nabla\varphi(X_{r})]dr\Bigr{]}$

can be made as small as desired. Prompted by the definition of
$\phi^{\epsilon}_{t}$ given in (38), we define
$\tilde{\phi}^{\epsilon}_{r}(x)$ by

$\tilde{\phi}^{\epsilon}_{r}(x)=\frac{d\bigl{(}(\tilde{\phi}_{r}\gamma_{r})*\rho_{\epsilon}\bigr{)}}{d(\gamma_{r}*\rho_{\epsilon})}.$

We first prove (45). Since $Y$ and $\nabla\varphi$ are bounded, we have

$\begin{split}\mathbb{E}^{\mathbb{P}^{\epsilon}}\Bigl{[}Y\int_{s}^{t}[\phi^{\epsilon}_{r}(X_{r})-\tilde{\phi}_{r}(X_{r})]\nabla\varphi(X_{r})]dr\Bigr{]}&\leq\|Y\|_{\infty}\|\nabla\varphi\|_{\infty}\mathbb{E}^{\mathbb{P}^{\epsilon}}\Bigl{[}\int_{s}^{t}|\phi^{\epsilon}_{r}(X_{r})-\tilde{\phi}_{r}(X_{r})|dr\Bigr{]}\\\
&\leq\|Y\|_{\infty}\|\nabla\varphi\|_{\infty}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{\epsilon}_{r}(x)-\tilde{\phi}_{r}(x)|\nu^{\epsilon}_{r}(dx)dr\\\
&\leq\|Y\|_{\infty}\|\nabla\varphi\|_{\infty}e^{T}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{\epsilon}_{r}(x)-\tilde{\phi}_{r}(x)|\gamma^{\epsilon}_{r}(dx)dr.\end{split}$

We use the fact that

$\begin{split}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{\epsilon}_{r}(x)-\tilde{\phi}_{r}(x)|\gamma^{\epsilon}_{r}(dx)dr&\leq\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{\epsilon}_{r}(x)-\tilde{\phi}^{\epsilon}_{r}(x)|\gamma^{\epsilon}_{r}(dx)dr+\int_{0}^{T}\int_{\mathbb{R}^{d}}|\tilde{\phi}^{\epsilon}_{r}(x)-\tilde{\phi}_{r}(x)|\gamma^{\epsilon}_{r}(dx)dr\\\
&=(i)+(ii)\end{split}$

and we estimate separately $(i)$ and $(ii)$.

$\begin{split}(i)&=\int_{0}^{T}\int_{\mathbb{R}^{d}}\Bigl{|}\frac{d\bigl{(}(\phi_{t}\gamma_{t})*\rho_{\epsilon}\bigr{)}}{d(\gamma_{t}*\rho_{\epsilon})}-\frac{d\bigl{(}(\tilde{\phi}_{t}\gamma_{t})*\rho_{\epsilon}\bigr{)}}{d(\gamma_{t}*\rho_{\epsilon})}\Bigr{|}\;d(\gamma_{t}*\rho_{\epsilon})dt\\\
&\leq\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi_{t}(x)-\tilde{\phi}_{t}(x)|\;\gamma_{t}(dx)dt\end{split}$

where we used [3, Lemma 8.1.10], and this quantity can be made as small as
desired by density of the Lipschitz continuous functions with compact support
in $L^{1}\bigl{(}[0,T]\times\mathbb{R}^{d},\gamma_{t}(dx)dt\bigr{)}$. Next,

$\begin{split}(ii)&=\int_{0}^{T}\int_{\mathbb{R}^{d}}\Bigl{|}[(\tilde{\phi}_{t}(x)\gamma_{t})*\rho_{\epsilon}](x)-\tilde{\phi}_{t}(x)(\gamma_{t}*\rho_{\epsilon})(x)\Bigr{|}dxdt\\\
&=\int_{0}^{T}\int_{\mathbb{R}^{d}}\Bigl{|}\int_{\mathbb{R}^{d}}\rho_{\epsilon}(x-y)\tilde{\phi}_{t}(y)\gamma_{t}(dy)-\tilde{\phi}_{t}(x)\int_{\mathbb{R}^{d}}\rho_{\epsilon}(x-y)\gamma_{t}(dy)\Bigr{|}dxdt\\\
&\leq\int_{0}^{T}\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{d}}\rho_{\epsilon}(x-y)|\tilde{\phi}_{t}(y)-\tilde{\phi}_{t}(x)|\gamma_{t}(dy)dxdt\\\
&\leq
C\int_{0}^{T}\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{d}}\rho_{\epsilon}(x-y)(|x-y|\wedge
1)\;\gamma_{t}(dy)dxdt\\\
&=CT\int_{\mathbb{R}^{d}}\rho_{\epsilon}(z)\,(|z|\wedge 1)\;dz\end{split}$

where we used Fubini’s theorem, the Lipschitz property of $\tilde{\phi}$, and
the fact that $\gamma_{t}(\mathbb{R}^{d})\leq 1$, and this quantity converges
to $0$ as $\epsilon\searrow 0$. This completes the proof since the above
argument also proves (46). ∎

We now state as a theorem the main result of this subsection.

######  Theorem 1 (Non-local Superposition Principle).

Let us assume that the couple $(\bm{\phi},\bm{\mu})$ is such that
$\bm{\phi}=\bigl{(}\phi_{t}(x)\bigr{)}_{0\leq t\leq T,\;x\in\mathbb{R}^{d}}$
is a $\mathbb{R}^{d}$-valued measurable function on
$[0,T]\times\mathbb{R}^{d}$ and $\bm{\mu}=(\mu_{t})_{0\leq t\leq T}$ is a
measurable flow of probability measures satisfying (34) and the non-local FPK
equation (31) in the sense of distributions.

Then there exists a weak solution $\mathbf{X}=(X_{t})_{0\leq t\leq T}$ of the
stochastic differential equation (32) satisfying (34), and such that for each
$t\in[0,T]$, the probability measure $\mu_{t}$ is given by (30). Moreover,

(47) $\sup_{0\leq t\leq T}\mathbb{E}[|X_{t}|^{2}]<\infty.$

###### Proof.

The existence of $\mathbf{X}$ follows the set of lemmas proven above. We only
need to prove the bound (47). Itô’s formula gives:

$|X_{t}|^{2}=|X_{0}|^{2}+\int_{0}^{t}[2\phi_{s}(X_{s})\cdot
X_{s}+d]ds+2\int_{0}^{t}X_{s}\cdot dW_{s}$

and taking expectations we get

$\mathbb{E}[|X_{t}|^{2}]\leq\Bigl{(}\mathbb{E}[|X_{0}|^{2}]+d\,T+\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi_{t}(x)|^{2}d\nu_{t}(dx)dt\Bigr{)}+\int_{0}^{t}\mathbb{E}[|X_{s}|^{2}]ds$

and we conclude using Gronwall’s inequality. ∎

###  3.4. Reformulation of the Optimization Problem

In this subsection we reformulate the relaxed optimization problem over
stochastic state processes as a deterministic control problem on a space of
probability measures.

For $\mu_{0}\in\mathcal{P}_{2}(\mathbb{R}^{d})$, we now denote by
$\mathbb{A}=\mathbb{A}(\mu_{0})$ the set of couples $(\theta,\mu)$ where
$\mu\in\mathcal{M}_{+}([0,T]\times\mathbb{R}^{d})$ is a non-negative measure
on $[0,T]\times\mathbb{R}^{d}$ of the form $\mu(dt,dx)=\mu_{t}(dx)dt$ for a
measurable flow $\bm{\mu}=(\mu_{t})_{0\leq t\leq T}$ of probability measures
on $\mathbb{R}^{d}$, and
$\theta\in\mathcal{M}([0,T]\times\mathbb{R}^{d};\mathbb{R}^{d})$ is an
$\mathbb{R}^{d}$-valued measure on $[0,T]\times\mathbb{R}^{d}$ of the form
$\theta(dt,dx)=\theta_{t}(dx)dt$ for a measurable flow
$\bm{\theta}=(\theta_{t})_{0\leq t\leq T}$ of $\mathbb{R}^{d}$-valued measures
on $\mathbb{R}^{d}$, satisfying

(48)
$\partial_{t}\mu_{t}-\frac{\sigma^{2}}{2}\Delta\mu_{t}+\operatorname{div}(\theta_{t})+(V-<\mu_{t},V>)\mu_{t}=0$

in the sense of distributions, and with initial condition
$\mu_{|t=0}=\mu_{0}$.

We denote by $\mathbb{A}^{(2)}=\mathbb{A}^{(2)}(\mu_{0})$ the subset of
$\mathbb{A}(\mu_{0})$ of couples $(\bm{\theta},\bm{\mu})$ for which
$\theta_{t}$ is absolutely continuous with respect to $\mu_{t}$ for all
$t\in[0,T]$, and for which there exists a measurable function
$[0,T]\times\mathbb{R}^{d}\ni(t,x)\mapsto\phi_{t}(x)\in\mathbb{R}^{d}$ such
that

$\frac{d\theta_{t}}{d\mu_{t}}(x)=\phi_{t}(x)\qquad\text{and}\qquad\int_{0}^{T}\int_{\mathbb{R}^{d}}\phi_{t}(x)^{2}\mu_{t}(dx)dt<\infty.$

If $(\bm{\theta},\bm{\mu})\in\mathbb{A}^{(2)}(\mu_{0})$, Theorem 1 implies
that there exists a process $\mathbf{X}=(X_{t})_{0\leq t\leq T}$ satisfying
$dX_{t}=\phi_{t}(X_{t})dt+dW_{t}$, (34) and (47), and such that the
probability measures $\mu_{t}$ are given by

$\mu_{t}(dx)=\frac{\mathbb{E}\bigl{[}\delta_{X_{t}}(dx)e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}.$

Later on, we shall use the analog $\mathbb{A}^{(2)}_{[s,T]}$ of
$\mathbb{A}^{(2)}$ defined over the interval $[s,T]$ instead of $[0,T]$. We
now introduce the functional $J$ defined on $\mathbb{A}$ by

(49) $J(\bm{\theta},\bm{\mu})=\begin{cases}\displaystyle\int_{0}^{T}\int
f(x,\phi_{t}(x))\mu_{t}(dx)\;dt+\int g(x)\mu_{T}(dx),\quad&\text{if
}(\bm{\theta},\bm{\mu})\in\mathbb{A}^{(2)}\\\
\infty&\text{otherwise.}\end{cases}$

When $(\bm{\theta},\bm{\mu})\in\mathbb{A}^{(2)}$ we use the notations
$J(\bm{\theta},\bm{\mu})$ and $J(\bm{\phi},\bm{\mu})$ interchangeably.

###  3.5. Existence of an Optimal Control

Next, we state and prove the existence of an optimal Markovian control.

######  Proposition 2.

There exists a couple $(\bm{\theta},\bm{\mu})=(\phi_{t},\mu_{t})_{0\leq t\leq
T}\in\mathbb{A}^{(2)}$ minimizing $J(\bm{\phi},\bm{\mu})$ over $\mathbb{A}$.

###### Proof.

The idea is to consider a minimizing sequence, and to show that it converges
in a suitable sense to a minimizer of $J(\bm{\theta},\bm{\mu})$.
$\mathbb{A}^{(2)}$ being non-empty,

$J^{*}:=\inf_{(\bm{\theta},\bm{\mu})\in\mathbb{A}}J(\bm{\theta},\bm{\mu})<\infty$

and because of the definition (49), we can limit the search for a minimizer to
$\mathbb{A}^{(2)}$.

_Step 1_. Let $(\bm{\phi}^{n},\bm{\mu}^{n})_{n\geq 0}$ be a minimizing
sequence in $\mathbb{A}^{(2)}$. Since
$\sup_{n}J(\bm{\phi}^{n},\bm{\mu}^{n})<\infty$, we have

(50)
$C_{\phi}:=\sup_{n}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{n}_{t}(x)|^{2}\mu^{n}_{t}(dx)dt<\infty,$

from which we argue that, extracting a sub-sequence if necessary, the sequence
$(\theta^{n})_{n\geq 0}$ of $\mathbb{R}^{d}$-valued measures defined by

$\theta^{n}(dt,dx)=\phi^{n}_{t}(x)\mu^{n}_{t}(dx)dt$

converges weakly toward a measure
$\theta(dt,dx)\in\mathcal{M}([0,T]\times\mathbb{R}^{d};\mathbb{R}^{d})$.
Moreover, and for similar reasons, we may assume without any loss of
generality, that $\bigl{(}\mu^{n}_{t}(dx)dt\bigr{)}_{n\geq 0}$ converges
weakly toward a non-negative measure which is necessarily of the form
$\mu_{t}(dx)dt$. The fact that the sequences $(\theta^{n})_{n\geq 0}$ and
$(\mu^{n})_{n\geq 0}$ are tight despite the fact that the state space is not
compact is a simple consequence of

$\int_{0}^{T}\int_{\mathbb{R}^{d}}|x|\;|\phi^{n}_{t}(x)|\mu^{n}_{t}(dx)dt\leq\Bigl{[}\int_{0}^{T}\int_{\mathbb{R}^{d}}|x|^{2}\mu^{n}_{t}(dx)dt\Bigr{]}^{1/2}\Bigl{[}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{n}_{t}(x)|^{2}\mu^{n}_{t}(dx)dt\Bigr{]}^{1/2}<c$

for a finite constant $c$ independent of $n$. Indeed, using the test function
$\varphi(x)=|x|^{2}$ in (48) we get:

(51)
$\begin{split}\partial_{t}<|x|^{2},\mu^{n}_{t}>&=<\frac{1}{2}\Delta|x|^{2}+\phi^{n}_{t}(x)\nabla|x|^{2}-(V(x)-<V,\mu_{t}>)|x|^{2},\mu^{n}_{t}>\\\
&\leq d+2<x\cdot\phi^{n}_{t}(x),\mu^{n}_{t}>+<|x|^{2},\mu^{n}_{t}>\\\ &\leq
d+4<|\phi^{n}_{t}(x)|^{2},\mu^{n}_{t}>+5<|x|^{2},\mu^{n}_{t}>\\\ &\leq
d+4C_{\phi}+5<|x|^{2},\mu^{n}_{t}>\end{split}$

and Gronwall’s inequality gives

$\int_{\mathbb{R}^{d}}|x|^{2}\mu^{n}_{t}(dx)\leq
e^{5T}\int_{\mathbb{R}^{d}}|x|^{2}\mu_{0}(dx)+C^{\prime},$

for a positive constant $C^{\prime}$ independent of $n$.

_Step 2_. We now show that $\theta(dt,dx)$ is of the form
$\theta(dt,dx)=\phi_{t}(x)\mu_{t}(dx)dt$ for some
$\bm{\phi}=(\phi_{t}(x))_{t,x}\in L^{2}(\mu_{t}(dx)dt)$. If $h\in
C_{b}([0,T]\times\mathbb{R}^{d};\mathbb{R}^{d})$ is a bounded continuous
function on $[0,T]\times\mathbb{R}^{d}$ with values in $\mathbb{R}^{d}$, we
have:

(52) $\begin{split}|<\theta,h>|&=\lim_{n\to\infty}|<\theta^{n},h>|\\\
&=\lim_{n\to\infty}\Bigl{|}\int_{0}^{T}\int_{\mathbb{R}^{d}}h(t,x)\cdot\phi^{n}_{t}(x)\mu^{n}_{t}(dx)dt\Bigr{|}\\\
&\leq\limsup_{n\to\infty}\Bigl{(}\int_{0}^{T}\int_{\mathbb{R}^{d}}|h(t,x)|^{2}\mu^{n}_{t}(dx)dt\Bigr{)}^{1/2}\Bigl{(}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{n}_{t}(x)|^{2}\mu^{n}_{t}(dx)dt\Bigr{)}^{1/2}\;\\\
&\leq
C_{\phi}^{1/2}\limsup_{n\to\infty}\Bigl{(}\int_{0}^{T}\int_{\mathbb{R}^{d}}|h(t,x)|^{2}\mu^{n}_{t}(dx)dt\Bigr{)}^{1/2}\\\
&=C_{\phi}^{1/2}\Bigl{(}\int_{0}^{T}\int_{\mathbb{R}^{d}}|h(t,x)|^{2}\mu_{t}(dx)\;dt\Bigr{)}^{1/2}\end{split}$

which shows that $\theta$ is a bounded linear form on the Hilbert space
$L^{2}\bigl{(}\mu_{t}(dx)dt\bigr{)}$, proving the existence of $\bm{\phi}\in
L^{2}(\mu_{t}(dx)dt)$ such that $\theta(dt,dx)=\phi_{t}(x)\mu_{t}(dx)dt$.

_Step 3_. For each integer $n\geq 0$, since $(\phi^{n}_{t},\mu^{n}_{t})_{0\leq
t\leq T}\in\mathbb{A}^{(2)}$, for each test function
$(t,x)\mapsto\varphi(t,x)$ in $C^{1,2}_{b}([0,T]\times\mathbb{R}^{d})$, namely
a smooth function with enough bounded derivatives so we can use integration by
parts and push the derivatives from $\mu_{t}$ to $\varphi$, and hopefully not
have boundary terms to deal with, we have:

$\int_{0}^{T}\hskip-4.0pt\int_{\mathbb{R}^{d}}\Bigl{[}\partial_{t}\varphi(t,x)+\frac{\sigma^{2}}{2}\Delta\varphi(t,x)-(V(x)-<V,\mu^{n}_{t}>)\varphi(t,x)\Bigr{]}\mu^{n}_{t}(dx)dt=-\int_{0}^{T}\hskip-4.0pt\int_{\mathbb{R}^{d}}\partial_{x}\varphi(t,x)\cdot\phi^{n}_{t}(x)\,\mu^{n}_{t}(dx)dt.$

We can pass to the limit $n\to\infty$ using the convergence of
$\mu^{n}_{t}(dx)dt$ in the left hand side and the convergence of
$\phi^{n}_{t}(x)\mu^{n}_{t}(dx)dt$ in the right hand side to conclude that
$(\phi_{t},\mu_{t})_{0\leq t\leq T}\in\mathbb{A}^{(2)}$.

_Step 4_. For the sake of convenience, we shall use the notation

$\tilde{J}(\bm{\psi},\bm{\nu})=\frac{1}{2}\int_{0}^{T}\int_{\mathbb{R}^{d}}\;|\psi_{t}(x)|^{2}\nu_{t}(dx)dt.$

For each $\epsilon>0$ and for each $t\in[0,T]$, we define
$\theta^{\epsilon}_{t}=\theta_{t}*\rho_{\epsilon}$, and
$\mu_{t}^{\epsilon}=\mu_{t}*\rho_{\epsilon}$ where
$(\rho_{\epsilon})_{\epsilon>0}$ is an approximate identity (say a Gaussian
density in $\mathbb{R}^{d}$ with variance $\epsilon$,
$\rho_{\epsilon}(x)=(2\pi x)^{-d/2}\exp(-|x|^{2}/2\epsilon)$), and where the
operation of convolution is done component by component when appropriate. We
then define $\phi^{\epsilon}_{t}(x)$ as the density of $\theta^{\epsilon}_{t}$
with respect to $\mu_{t}^{\epsilon}$. Using [3, Lemma 8.1.10] we get

(53)
$\tilde{J}(\bm{\phi}^{\epsilon},\bm{\mu}^{\epsilon})\leq\tilde{J}(\bm{\phi},\bm{\mu})$

and since for each $t\in[0,T]$, $\mu^{\epsilon}_{t}$ and
$\theta^{\epsilon}_{t}$ converge weakly toward $\mu_{t}$ and $\theta_{t}$
respectively, using the fact that the functional

$(\theta,\mu)\mapsto\int_{\mathbb{R}^{d}}\Bigl{|}\frac{d\theta(x)}{d\mu(x)}\Bigr{|}^{2}\mu(dx)$

is lower semi continuous (see for instance Theorem 2.34 and Example 2.36 in
[2]), we conclude that:

(54) $\lim_{\epsilon\searrow
0}\tilde{J}(\bm{\phi}^{\epsilon},\bm{\mu}^{\epsilon})=\tilde{J}(\bm{\phi},\bm{\mu}).$

Similarly, for each integer $n\geq 1$ we define
$\theta^{n,\epsilon}_{t}=\theta^{n}_{t}*\rho_{\epsilon}$, and
$\mu_{t}^{n,\epsilon}=\mu^{n}_{t}*\rho_{\epsilon}$, and
$\phi^{n,\epsilon}_{t}$ as the density of $\theta^{n,\epsilon}_{t}$ with
respect to $\mu_{t}^{n,\epsilon}$. Notice that for each $\epsilon>0$,
$\bm{\mu}^{n,\epsilon}$ and $\bm{\theta}^{n,\epsilon}$ converge weakly toward
$\bm{\mu}^{\epsilon}$ and $\bm{\theta}^{\epsilon}$ respectively. For the sake
of notation we define
$\phi^{(\epsilon,K)}_{t}(x)=(-K)\vee\phi^{\epsilon}_{t}(x)\wedge K$ where the
operations of minimum and maximum are interpreted component by component. We
have:

(55)
$\begin{split}\tilde{J}(\bm{\phi}^{\epsilon},\bm{\mu}^{\epsilon})&=\lim_{K\nearrow\infty}\int_{0}^{T}\int_{\mathbb{R}^{d}}\phi^{(\epsilon,K)}_{t}(x)\cdot\phi^{\epsilon}_{t}(x)\mu^{\epsilon}_{t}(dx)dt\\\
&=\lim_{K\nearrow\infty}\lim_{n\nearrow\infty}\int_{0}^{T}\int_{\mathbb{R}^{d}}\phi^{(\epsilon,K)}_{t}(x)\cdot\phi^{n,\epsilon}_{t}(x)\mu^{n,\epsilon}_{t}(dx)dt\\\
&\leq\lim_{K\nearrow\infty}\lim_{n\nearrow\infty}\Bigl{[}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{(\epsilon,K)}_{t}(x)|^{2}\mu^{n,\epsilon}_{t}(dx)dt\Bigr{]}^{1/2}\Bigl{[}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{n,\epsilon}_{t}(x)|^{2}\mu^{n,\epsilon}_{t}(dx)dt\Bigr{]}^{1/2}\\\
&=\lim_{K\nearrow\infty}\Bigl{[}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{(\epsilon,K)}_{t}(x)|^{2}\mu^{\epsilon}_{t}(dx)dt\Bigr{]}^{1/2}\lim_{n\nearrow\infty}\Bigl{[}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{n,\epsilon}_{t}(x)|^{2}\mu^{n,\epsilon}_{t}(dx)dt\Bigr{]}^{1/2}\\\
&=\Bigl{[}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{\epsilon}_{t}(x)|^{2}\mu^{\epsilon}_{t}(dx)dt\Bigr{]}^{1/2}\lim_{n\nearrow\infty}\Bigl{[}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{n,\epsilon}_{t}(x)|^{2}\mu^{n,\epsilon}_{t}(dx)dt\Bigr{]}^{1/2}.\end{split}$

Notice that

(56)
$\begin{split}\lim_{n\nearrow\infty}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi^{n,\epsilon}_{t}(x)|^{2}\mu^{n,\epsilon}_{t}(dx)dt&=\lim_{n\nearrow\infty}J(\bm{\theta}^{n,\epsilon},\bm{\mu}^{n,\epsilon})-\lim_{n\nearrow\infty}\int_{0}^{T}\int_{\mathbb{R}^{d}}\tilde{f}(x)\mu^{n,\epsilon}_{t}(dx)dt-\lim_{n\nearrow\infty}\int_{\mathbb{R}^{d}}g(x)\mu^{n,\epsilon}_{T}(dx)\\\
&=\lim_{n\nearrow\infty}J(\bm{\theta}^{n,\epsilon},\bm{\mu}^{n,\epsilon})-\int_{0}^{T}\int_{\mathbb{R}^{d}}\tilde{f}(x)\mu^{\epsilon}_{t}(dx)dt-\int_{\mathbb{R}^{d}}g(x)\mu^{\epsilon}_{T}(dx)\\\
\end{split}$

because we assume that $\tilde{f}$ and $g$ are bounded and continuous. Notice
that (56) implies that

$\lim_{n\nearrow\infty}J(\bm{\theta}^{n,\epsilon},\bm{\mu}^{n,\epsilon})\geq\int_{0}^{T}\int_{\mathbb{R}^{d}}\tilde{f}(x)\mu^{\epsilon}_{t}(dx)dt+\int_{\mathbb{R}^{d}}g(x)\mu^{\epsilon}_{T}(dx)$

which in turn implies that, if
$\tilde{J}(\bm{\theta}^{\epsilon},\bm{\mu}^{\epsilon})=0$,

$\lim_{n\nearrow\infty}J(\bm{\theta}^{n,\epsilon},\bm{\mu}^{n,\epsilon})\geq
J(\bm{\phi}^{\epsilon},\bm{\mu}^{\epsilon}).$

On the other hand, if
$\tilde{J}(\bm{\theta}^{\epsilon},\bm{\mu}^{\epsilon})>0$, (55) implies

(57)
$\begin{split}\tilde{J}(\bm{\theta}^{\epsilon},\bm{\mu}^{\epsilon})&\leq\lim_{n\nearrow\infty}J(\bm{\theta}^{n,\epsilon},\bm{\mu}^{n,\epsilon})-\int_{0}^{T}\int_{\mathbb{R}^{d}}\tilde{f}(x)\mu^{\epsilon}_{t}(dx)dt-\int_{\mathbb{R}^{d}}g(x)\mu^{\epsilon}_{T}(dx)\\\
&=\lim_{n\nearrow\infty}J(\bm{\theta}^{n,\epsilon},\bm{\mu}^{n,\epsilon})+\tilde{J}(\bm{\theta}^{\epsilon},\bm{\mu}^{\epsilon})-J(\bm{\theta}^{\epsilon},\bm{\mu}^{\epsilon}),\end{split}$

implying that

$J(\bm{\theta}^{\epsilon},\bm{\mu}^{\epsilon})\leq\lim_{n\nearrow\infty}J(\bm{\theta}^{n,\epsilon},\bm{\mu}^{n,\epsilon})$

holds in all cases. Now

(58)
$\begin{split}J(\bm{\theta}^{n,\epsilon},\bm{\mu}^{n,\epsilon})&=\tilde{J}(\bm{\theta}^{n,\epsilon},\bm{\mu}^{n,\epsilon})+\int_{0}^{T}\int_{\mathbb{R}^{d}}\tilde{f}(x)\mu^{n,\epsilon}_{t}(dx)dt+\int_{\mathbb{R}^{d}}g(x)\mu^{n,\epsilon}_{T}(dx)\\\
&\leq\tilde{J}(\bm{\theta}^{n},\bm{\mu}^{n})+\int_{0}^{T}\int_{\mathbb{R}^{d}}\tilde{f}(x)\mu^{n,\epsilon}_{t}(dx)dt+\int_{\mathbb{R}^{d}}g(x)\mu^{n,\epsilon}_{T}(dx)\\\
\end{split}$

if we use once more (53) from [3, Lemma 8.1.10]. Consequently:

(59)
$\lim_{n\nearrow\infty}J(\bm{\theta}^{n,\epsilon},\bm{\mu}^{n,\epsilon})\leq\lim_{n\nearrow\infty}\tilde{J}(\bm{\theta}^{n},\bm{\mu}^{n})+\int_{0}^{T}\int_{\mathbb{R}^{d}}\tilde{f}(x)\mu^{\epsilon}_{t}(dx)dt+\int_{\mathbb{R}^{d}}g(x)\mu^{\epsilon}_{T}(dx)\\\
$

and

(60) $\begin{split}\lim_{\epsilon\searrow
0}\lim_{n\nearrow\infty}J(\bm{\theta}^{n,\epsilon},\bm{\mu}^{n,\epsilon})&\leq\lim_{n\nearrow\infty}\tilde{J}(\bm{\theta}^{n},\bm{\mu}^{n})+\int_{0}^{T}\int_{\mathbb{R}^{d}}\tilde{f}(x)\mu_{t}(dx)dt+\int_{\mathbb{R}^{d}}g(x)\mu_{T}(dx)\\\
&\leq\lim_{n\nearrow\infty}J(\bm{\theta}^{n},\bm{\mu}^{n})-\lim_{n\nearrow\infty}\int_{0}^{T}\int_{\mathbb{R}^{d}}\tilde{f}(x)\mu^{n}_{t}(dx)dt-\lim_{n\nearrow\infty}\int_{\mathbb{R}^{d}}g(x)\mu^{n}_{T}(dx)\\\
&\hskip
95.0pt+\int_{0}^{T}\int_{\mathbb{R}^{d}}\tilde{f}(x)\mu_{t}(dx)dt+\int_{\mathbb{R}^{d}}g(x)\mu_{T}(dx)\\\
&=J^{*},\end{split}$

from which we deduce, using (54):

$J(\bm{\theta},\bm{\mu})=\lim_{\epsilon\searrow
0}J(\bm{\theta}^{\epsilon},\bm{\mu}^{\epsilon})\leq\lim_{\epsilon\searrow
0}\lim_{n\nearrow\infty}J(\bm{\theta}^{n,\epsilon},\bm{\mu}^{n,\epsilon})\leq
J^{*}.$

This completes the proof. ∎

######  Remark 2.

The above argument shows that there is no loss of generality in limiting the
search for optima to the subset of admissible feedback control functions
satisfying

(61) $\mathbb{E}\int_{0}^{T}|\phi_{t}(X_{t})|^{2}\;dt\leq K$

for a large enough constant $K>0$. Recall (50) and the fact that the
expectation is over a process $(X_{t})_{0\leq t\leq T}$ satisfying the state
dynamics (27) driven by the control $\bm{\phi}$ and that

$\mathbb{E}\int_{0}^{T}|\phi_{t}(X_{t})|^{2}\;dt\leq
e^{T}\int_{0}^{T}\frac{\mathbb{E}\bigl{[}|\phi_{t}(X_{t})|^{2}e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}=e^{T}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\phi_{t}(x)|^{2}\;\mu_{t}(dx)dt.$

###  3.6. Solution of the Deterministic Control Problem

In line with the computations of, and the notations used in the previous
subsections, the running and terminal cost functions of the deterministic
infinite dimensional control problem are defined as:

(62) $F^{(1)}(\mu,\phi)=\int f(x,\phi(x))\;\mu(dx),\quad\text{and}\quad
G^{(1)}(\mu)=\int g(x)\;\mu(dx),$

for $\mu$ and $\phi$ as above. So at east formally, the definition of the
corresponding Hamiltonian $\mathbb{H}^{(1)}$ should be:

(63)
$\mathbb{H}^{(1)}(\mu,\varphi,\phi)=<\frac{1}{2}\Delta\mu-\operatorname{div}(\phi\mu)-(V-<\mu,V>)\mu,\;\varphi>+F^{(1)}(\mu,\phi)\\\
$

where the bracket $<\,\cdot\,,\,\cdot\,>$ stands for the duality between
measures and functions, and coincides with the inner product in
$L^{2}(\mathbb{R}^{d},dx)$. After integration by parts of the first term, the
definition of this Hamiltonian reads:

(64)
$\mathbb{H}^{(1)}(\mu,\varphi,\phi)=-\frac{1}{2}<\nabla\mu,\nabla\varphi>+<\mu,\phi\nabla\varphi>-<\mu,V\varphi>+<\mu,V><\mu,\varphi>+F^{(1)}(\mu,\phi)$

which is well defined for $\mu\in\mathcal{P}(\mathbb{R}^{d})$ as long as
$\nabla\mu$ in the sense of distributions belongs to
$L^{2}(\mathbb{R}^{d},dx;\mathbb{R}^{d})$, $\varphi\in
L^{2}(\mathbb{R}^{d},dx)$ with a gradient (in the sense of distributions)
belonging to $L^{2}(\mathbb{R}^{d},dx;\mathbb{R}^{d})$ and
$L^{2}(\mathbb{R}^{d},\mu;\mathbb{R}^{d})$, and $\phi$ a $A$-valued measurable
function on $\mathbb{R}^{d}$ satisfying $\int|\phi(x)|^{2}\mu(dx)<\infty$.

####  3.6.1. The Adjoint PDE

Let us assume that $(\bm{\phi},\bm{\mu})\in\mathbb{A}^{(2)}$. We say that the
function $u$ is an adjoint variable (or a co-state) if it satisfies the PDE
$\partial_{t}u=-(\delta\mathbb{H}^{(1)}/\delta\mu)(\mu,u,\phi)$ with terminal
condition $u_{T}(x)=(\delta G^{(1)}/\delta\mu)(x)=g(x)$ in the sense of
distributions. Here the notation $\delta/\delta\mu$ stands for the _flat
derivative_ which we now define. Referring to [8, Definition 5.43] the flat
derivative (also called the linear functional derivative) of a function
$F:\mathcal{M}(\mathbb{R}^{k})\to\mathbb{R}$ of measures on $\mathbb{R}^{k}$
for some integer $k$, is defined to satisfy:

(65) $F(\mu^{\prime})-F(\mu)=\int_{0}^{1}\int\frac{\delta
F}{\delta\mu}(\theta\mu^{\prime}+(1-\theta)\mu)(x)\bigl{[}\mu^{\prime}-\mu\bigr{]}(dx)d\theta.$

Note that this notion of flat derivative is only defined up to a constant, but
this will not matter in the present analysis. Accordingly, the adjoint
equation reads:

(66)
$\partial_{t}u=-\frac{1}{2}\Delta_{x}u-\phi_{t}\cdot\nabla_{x}u+(V-<\mu,V>)u-V<\mu,u>-f\bigl{(}\cdot,\phi_{t}(\cdot)\bigr{)},$

which can be rewritten as

(67)
$0=\partial_{t}u+\frac{1}{2}\Delta_{x}u+\phi_{t}\cdot\nabla_{x}u-(V-<\mu,V>)u+V<\mu,u>+\frac{1}{2}|\phi_{t}|^{2}+\tilde{f}$

in the case of separable running cost functions of the form (5).

The fact that from now on, we deal with Partial Differential Equations (PDEs)
requires a strengthening of the assumptions made so far. In particular,
assumption (28) will be replaced by

######  Assumption 3.

(68)
$K_{\phi}:=\sup_{(t,x)\in[0,T]\times\mathbb{R}^{d}}\mathbb{E}\int_{t}^{T}|\phi_{s}(X^{t,x}_{s})|^{2}ds<\infty$

where $\mathbf{X}^{t,x}=(X^{t,x}_{s})_{t\leq s\leq T}$ satisfies the state
equation $dX_{s}=\phi_{s}(X_{s})ds+dW_{s}$ over the interval $[t,T]$ with
initial condition $X_{t}=x$.

This assumption is obviously satisfied when $\phi$ is bounded. More generally
it is satisfied for larger classes of functions $\phi$. For example, [10]
proves that Assumption 3 is satisfied if $\phi\in
L^{q}\bigl{(}[0,T];L^{p}(\mathbb{R}^{d})\bigr{)}$ for some

(69) $p\geq 2,\quad q>2,\quad\frac{d}{p}+\frac{2}{q}<1.$

Note also that earlier works like [13] used a local version of assumption 3 in
the sense that it is only required to be satisfied for
$\phi\mathbf{1}_{|x|\leq n}\in
L^{q}\bigl{(}[0,T];L^{p}(\mathbb{R}^{d})\bigr{)}$ for every integer $n\geq 1$.

######  Proposition 3.

In the case of separable cost functions, for each continuous flow
$(\mu_{t})_{0\leq t\leq T}$ of probability measures on $\mathbb{R}^{d}$ and
each feedback function $\phi$ satisfying the strong assumption 3, the adjoint
PDE (67) admits a solution in the sense of viscosity.

###### Proof.

First notice that if $u$ is a classical solution of (66), the _variation of
the constant formula_ and Itô’s formula give a form of Feynman-Kac implicit
representation

(70)
$\begin{split}&u_{t}(x)=\mathbb{E}\Bigl{[}\int_{t}^{T}e^{-\int_{t}^{r}\bigl{(}V(X_{\tau})-<\mu_{\tau},V>\bigr{)}d\tau}\bigl{[}<\mu_{r},u_{r}>V(X_{r})+f\bigl{(}X_{r},\phi_{r}(X_{r})\bigr{)}\bigr{]}dr\\\
&\hskip
155.0pt+g(X_{T})e^{-\int_{t}^{T}\bigl{(}V(X_{\tau})-<\mu_{\tau},V>\bigr{)}d\tau}\;\Bigl{|}\;X_{t}=x\Bigr{]}\end{split}$

where the expectation $\mathbb{E}$ is over the process $(X_{s})_{t\leq s\leq
T}$ satisfying (27) starting from $X_{t}=x$ at time $t$. The first step of the
proof is to treat (70) as a fixed point equation for the function $u$, and
show that such an equation has a unique solution. This is identifying $u$ as a
_mild solution_ of equation (66). Next, we argue that the latter is in fact
the unique solution of an affine BSDE, and we conclude that it is a viscosity
solution of (66) using a classical result of the theory of BSDEs. _Step 1_.
For each $u\in C_{b}([0,T]\times\mathbb{R}^{d})$ where
$C_{b}([0,T]\times\mathbb{R}^{d})$ denotes the Banach space of bounded
continuous functions on $[0,T]\times\mathbb{R}^{d}$, we denote by $\Phi(u)$
the right hand side of (70) seen as a function of $(t,x)$. Using the fact that
$f(x,\alpha)=\frac{1}{2}|\alpha|^{2}+\tilde{f}(x)$ with $\tilde{f}$ bounded,
assumption 3 implies that
$\mathbb{E}[\int_{t}^{T}|\phi_{r}(X_{r})|^{2}]dr|X_{t}=x]$ is uniformly
bounded in $t$ and $x$, and it is straightforward to check that $\Phi(u)\in
C_{b}([0,T]\times\mathbb{R}^{d})$, which we equip with the norm

$\|u\|_{\alpha}=\sup_{(t,x)\in[0,T]\times\mathbb{R}^{d}}e^{\alpha
t}|u_{t}(x)|$

for a fixed number $\alpha>1$. If $u^{1}$ and $u^{2}$ are two functions in
this Banach space,

(71) $\begin{split}\|\Phi(u^{1})-\Phi(u^{2})\|_{\alpha}&=\sup_{t,x}e^{\alpha
t}\Bigl{|}\mathbb{E}\Bigl{[}\int_{t}^{T}e^{-\int_{t}^{r}\bigl{(}V(X_{\tau})-<\mu_{\tau},V>\bigr{)}d\tau}|<\mu_{r},u^{1}_{r}-u^{2}_{r}>|V(X_{r})dr\;\Bigl{|}\;X_{t}=x\Bigr{]}\Bigr{|}\\\
&\leq\sup_{t,x}e^{\alpha
t}\int_{t}^{T}e^{r-t}\sup_{y\in\mathbb{R}^{d}}|u^{1}_{r}(y)-u^{2}_{r}(y)|dr\\\
&\leq\sup_{t}e^{\alpha t}\int_{t}^{T}e^{(1-\alpha)r-t}e^{\alpha
r}\sup_{x}|u^{1}_{r}(x)-u^{2}_{r}(x)|dr\\\
&\leq\|u^{1}-u^{2}\|_{\alpha}\sup_{t}e^{(\alpha-1)t}\int_{t}^{T}e^{(1-\alpha)r}dr\\\
&=\frac{1-e^{(1-\alpha)T}}{\alpha-1}\|u^{1}-u^{2}\|_{\alpha},\end{split}$

and the fraction is smaller than $1$ when $\alpha$ is large enough, because
$e^{(1-\alpha)T}\to 0$ and $1/(\alpha-1)\to 0$ as $\alpha\to+\infty$ which
proves that $\Phi$ is a strict contraction since $\alpha>e^{T}$. Its unique
fixed point is what we shall use as solution of the adjoint equation (66).
_Step 2_. In order to match the notation of [18, Remark 3.3] we set
$c(t,x)=-[V(x)-<\mu_{t},V>]$ and
$h(t,x)=<\mu_{t},u_{t}>V(x)+f\bigl{(}x,\phi_{t}(x)\bigr{)}$. Notice that the
function $c$ is bounded and since the function $u$ constructed above as a
fixed point is bounded, for each $(t,x)\in[0,T]\times\mathbb{R}^{d}$ we have:

$\mathbb{E}\Bigl{[}\int_{t}^{T}|h(s,X^{t,x}_{s})|ds\Bigr{]}\leq
C+\mathbb{E}\Bigl{[}\int_{t}^{T}|\phi_{s}(X^{t,x}_{s})|^{2}dt\Bigr{]}\leq
C+K_{\phi}<\infty$

where $(X^{t,x}_{s})_{t\leq s\leq T}$ is the solution of
$dX_{s}=\phi_{s}(X_{s})ds+dW_{s}$ over the interval $[t,T]$ and initial
condition $X_{t}=x$. For each $(t,x)$ we denote by
$(Y^{t,x}_{s},Z^{t,x}_{s})_{t\leq s\leq T}$ the unique solution of the affine
BSDE:

$dY^{t,x}_{s}=-[c(s,X^{t,x}_{s})Y^{t,x}_{s}+h(s,X^{t,x}_{s})]ds+Z^{t,x}_{s}dW_{s}$

with terminal condition $Y^{t,x}_{T}=g(X^{t,x}_{T})$. Being _linear_ , this
BSDE has an explicit solution

(72)
$Y^{t,x}_{s}=g(X^{t,x}_{T})e^{\int_{s}^{T}c(\tau,X^{t,x}_{\tau})d\tau}+\int_{s}^{T}h(r,X^{t,x}_{r})e^{\int_{s}^{r}c(\tau,X^{t,x}_{\tau})d\tau}dr-\int_{s}^{T}e^{\int_{s}^{r}c(\tau,X^{t,x}_{\tau})d\tau}Z^{t,x}_{r}dW_{r}$

and taking $s=t$, one recovers the Feynman-Kac formula

$Y^{t,x}_{t}=\mathbb{E}\Bigl{[}g(X^{t,x}_{T})e^{\int_{t}^{T}c(\tau,X^{t,x}_{\tau})d\tau}+\int_{t}^{T}h(r,X^{t,x}_{r})e^{\int_{t}^{r}c(\tau,X^{t,x}_{\tau})d\tau}dr\\\
\Bigr{]}$

which is exactly our formula (70) if we set $u(t,x)=Y^{t,x}_{t}$. Strictly
speaking, since $f(x,\phi_{t}(x))=\tilde{f}(x)+\frac{1}{2}|\phi_{t}(x)|^{2}$,
the assumptions of [18] would require that $|\phi_{t}(x)|$ be of polynomial
growth. However, given the special (linear) nature of the BSDE involved, the
fact that $\int_{t}^{T}\mathbb{E}[\phi_{t}(X^{t,x}_{s})|^{2}ds<\infty$ for
every $t\in[0,T]$ and $x\in\mathbb{R}^{d}$ is enough for the argument of [18]
to go through. We conclude using [18, Theorem 3.2]. ∎

######  Remark 3 (A first a-priori bound.).

We claim that under the weaker assumption 61, if $u$ admits the Feynman-Kac
representation (70), then

(73) $|<\mu_{t},u_{t}>|\leq
e^{2(T-t)}\bigl{(}\|\tilde{f}\|_{\infty}+\|g\|_{\infty}\bigr{)}+\frac{e^{T}}{4}(e^{2(T-t)}+1).$

Recall that we denote by $\nu_{t}$ the marginal distribution of the state
$X_{t}$ controlled by the drift $\phi_{t}$. So by definition of the
probability measure $\mu_{t}$, for any non-negative random variable $Y$ which
depend only upon the future after time $t$, we have

$\int_{\mathbb{R}^{d}}\mathbb{E}[Y|X_{t}=x]\mu_{t}(dx)\leq
e^{T}\int_{\mathbb{R}^{d}}\mathbb{E}[Y|X_{t}=x]\nu_{t}(dx)=e^{T}\mathbb{E}[Y]$

where the last expectation is with respect to the state process with initial
distribution $\mu_{0}$. So integrating both sides of the Feynman-Kac
representation (70) with respect to $\mu_{t}$ we get:

(74)
$\begin{split}&<\mu_{t},u_{t}>=\int_{\mathbb{R}^{d}}\mathbb{E}\Bigl{[}\int_{t}^{T}e^{-\int_{t}^{r}\bigl{(}V(X_{\tau})-<\mu_{\tau},V>\bigr{)}d\tau}\bigl{[}<\mu_{r},u_{r}>V(X_{r})+f\bigl{(}X_{r},\phi_{r}(X_{r})\bigr{)}\bigr{]}dr\\\
&\hskip
155.0pt+g(X_{T})e^{-\int_{t}^{T}\bigl{(}V(X_{\tau})-<\mu_{\tau},V>\bigr{)}d\tau}\;\Bigl{|}\;X_{t}=x\Bigr{]}\mu_{t}(dx)\end{split}$

and if we limit ourselves to feedback functions satisfying (61), we get:

$\begin{split}|<\mu_{t},&u_{t}>|\\\
&\leq\int_{t}^{T}e^{r-t}|<\mu_{r},u_{r}>|\,dr+e^{T-t}\|\tilde{f}\|_{\infty}+\frac{e^{T}}{2}\int_{\mathbb{R}^{d}}\mathbb{E}[\int_{t}^{T}|\phi_{s}(X_{s})|^{2}]ds\,|X_{t}=x]\nu_{t}(dx)+e^{T-t}\|g\|_{\infty}\\\
&\leq
e^{T-t}\bigl{(}\|\tilde{f}\|_{\infty}+\|g\|_{\infty}\bigr{)}+\frac{e^{T}}{2}K_{\phi}+\int_{t}^{T}e^{r-t}|<\mu_{r},u_{r}>|dr\end{split}$

and using the form of Gronwall inequality in Lemma 7 we get:

$e^{t}|<\mu_{t},u_{t}>|\leq
e^{2T-t}\bigl{(}\|\tilde{f}\|_{\infty}+\|g\|_{\infty}\bigr{)}+\frac{e^{T}}{4}(e^{2T-t}+1).$

######  Remark 4 (A stronger a-priori bound.).

Under the assumptions of Proposition 3, we have:

(75)
$\|u_{t}\|_{\infty}\leq\frac{e^{t}}{2}\bigl{(}e^{2(T-t)}-1\bigr{)}\|\tilde{f}\|_{\infty}+\frac{e^{2T}}{2}K_{\phi}+e^{T}\|g\|_{\infty}.$

Indeed, computing the supremum of the left hand side of the Feynman-Kac
representation (70) instead of integrating with respect to $\mu_{t}$, we get:

$\begin{split}|u_{t}(x)|&\leq\int_{t}^{T}e^{r-t}\bigl{(}\|u_{r}\|_{\infty}+\|\tilde{f}\|_{\infty}\bigr{)}\;dr+\int_{t}^{T}\mathbb{E}\bigl{[}|\phi_{r}(X_{r})|^{2}\,|\,X_{t}=x\bigr{]}+e^{T-t}\|g\|_{\infty}\end{split}$

and

$e^{t}\|u_{t}\|_{\infty}\leq\int_{t}^{T}e^{r}\|u_{r}\|_{\infty}dr+(e^{T}-e^{t})\|\tilde{f}\|_{\infty}+e^{t}K_{\phi}+e^{T}\|g\|_{\infty}.$

Finally, we get (78) using Lemma 7 with
$\zeta(t)=e^{T}\|g\|_{\infty}+e^{t}K_{\phi}+(e^{T}-e^{t})\|\tilde{f}\|_{\infty}$,
$c=1$ and $\xi(t)=e^{t}\|u_{t}\|_{\infty}$.

####  3.6.2. Analysis of the Adjoint PDE

In this subsection, we assume that
$(\bm{\phi},\bm{\mu})\in\mathbb{A}^{(2)}(\mu_{0})$. In particular, $\bm{\mu}$
solves the Fokker-Planck-Kolmogorov equation driven by $\bm{\phi}$, and we
denote by $\bm{\nu}=(\nu_{t})_{0\leq t\leq T}$ the marginal distributions of
the state process $\mathbf{X}=(X_{t})_{0\leq t\leq T}$ whose existence is
guarateed by our form of the superposition principle proven in Theorem 1.

For the sake of later reference, we provide without proof, a simple version of
Gronwall’s lemma which we shall use repeatedly in the sequel.

######  Lemma 7.

Let us assume that $\xi(t)\leq\zeta(t)+c\int_{t}^{T}\xi(r)dr$, then

(76) $\xi(t)\leq\zeta(t)+ce^{c(T-t)}\int_{t}^{T}\zeta(r)e^{-c(T-r)}dr.$

######  Lemma 8.

If $\tilde{\bm{\phi}}=(\tilde{\phi}_{t})_{0\leq t\leq T}\in
C_{b}([0,T]\times\mathbb{R}^{d};\mathbb{R}^{d})$ and
$\tilde{F}=(\tilde{F}_{t})_{0\leq t\leq T}\in
C_{b}([0,T]\times\mathbb{R}^{d})$ are smooth enough, then the equation

(77) $0=\partial_{t}v_{t}+\frac{1}{2}\Delta v_{t}+\tilde{\phi}_{t}\cdot\nabla
v_{t}-(V-<\mu_{t},V>)v_{t}+V<\mu_{t},v_{t}>+\tilde{F}$

with bounded terminal condition $v_{T}=g$, has a unique classical solution
satisfying the upper bound

(78) $\|v_{t}\|_{\infty}\leq
e^{2T-t}\Bigl{(}\|g\|_{\infty}+\frac{1}{2}\|\tilde{F}\|_{\infty}\Bigr{)}.$

Moreover, when $g=0$, this solution satisfies:

(79) $\int_{0}^{T}\int_{\mathbb{R}^{d}}|\nabla v_{t}(x)|^{2}\mu_{t}(dx)dt\leq
4e^{4T}\|\tilde{F}\|_{\infty}^{2}\bigl{(}1+\frac{3}{2}T+\|\tilde{\bm{\phi}}-\bm{\phi}\|^{2}_{L^{2}(\bm{\mu})}\bigr{)}.$

###### Proof.

Equation (77) is not a standard PDE because of the presence of the non-local
term $V<\mu_{t},v_{t}>$ in the right hand side. The solution is obtained by
constructing a fixed point to the mapping
$\tilde{\mathbf{v}}\mapsto\Phi(\tilde{\mathbf{v}})=\mathbf{v}$ where
$\mathbf{v}$ is the unique classical solution of the regular linear PDE

$0=\partial_{t}v_{t}+\frac{1}{2}\Delta v_{t}+\tilde{\phi}_{t}\cdot\nabla
v_{t}-(V-<\mu_{t},V>)v_{t}+V<\mu_{t},\tilde{v}_{t}>+\tilde{F},$

with terminal condition $v_{T}=g$. From now on, we let $\mathbf{v}$ be a
solution to (77). The first a-priori bound (78) follows easily from the
Feynman-Kac representation in terms of the solution
$\tilde{\mathbf{X}}=(\tilde{X}_{t})_{0\leq t\leq T}$ of the state stochastic
differential equation
$d\tilde{X}_{t}=\tilde{\phi}_{t}(\tilde{X}_{t})dt+dW_{t}$. Indeed,
$\tilde{\phi}$ being bounded, it satisfies Assumption 3 and we can repeat the
argument of Remark 4: from (77) we get:

$\begin{split}v_{t}(x)&=\mathbb{E}\Bigl{[}\int_{t}^{T}e^{-\int_{t}^{r}\bigl{(}V(\tilde{X}_{\tau})-<\mu_{\tau},V>\bigr{)}d\tau}\bigl{[}<\mu_{r},v_{r}>V(\tilde{X}_{r})+\tilde{F}(r,\tilde{X}_{r})\bigr{]}dr\\\
&\hskip
155.0pt+g(\tilde{X}_{T})e^{-\int_{t}^{T}\bigl{(}V(\tilde{X}_{\tau})-<\mu_{\tau},V>\bigr{)}d\tau}\;\Bigl{|}\;\tilde{X}_{t}=x\Bigr{]}\end{split}$

from which we get

$\begin{split}|v_{t}(x)|&\leq\int_{t}^{T}e^{r-t}\bigl{(}\|v_{r}\|_{\infty}+\|\tilde{F}\|_{\infty}\bigr{)}\;dr+e^{T-t}\|g\|_{\infty}\end{split}$

and

$e^{t}\|v_{t}\|_{\infty}\leq\int_{t}^{T}e^{r}\|v_{r}\|_{\infty}dr+(e^{T}-e^{t})\|\tilde{F}\|_{\infty}+e^{T-t}\|g\|_{\infty}.$

Finally, we get (78) using Lemma 7 with
$\zeta(t)=e^{T-t}\|g\|_{\infty}+(e^{T}-e^{t})\|\tilde{F}\|_{\infty}$, $c=1$
and $\xi(t)=e^{t}\|v_{t}\|_{\infty}$. The derivation of the a-priori bound
(79) is more involved. We notice that since $\bm{\mu}$ solves the Fokker-
Planck-Kolmogorov equation driven by $\bm{\phi}$, a simple integration by
parts implies that if $\varphi$ is a smooth bounded function on
$[0,T]\times\mathbb{R}^{d}$ we must have:

(80)
$\int_{0}^{T}\int_{\mathbb{R}^{d}}\Bigl{[}\partial_{t}\varphi^{2}+\frac{1}{2}\Delta_{x}\varphi^{2}+\phi_{t}\cdot\nabla_{x}\varphi^{2}-(V-<\mu_{t},V>)\varphi^{2}\Bigr{]}\mu_{t}(dx)dt=<\mu_{T},\varphi^{2}_{T}>-<\mu_{0},\varphi^{2}_{0}>.$

as long as the above integrals exist. Using (77) we get:

$\begin{split}&\int_{0}^{T}\int_{\mathbb{R}^{d}}v_{t}(x)\bigl{(}\tilde{F}_{t}(x)+\tilde{\phi}_{t}\cdot\nabla_{x}v_{t}(x)\bigr{)}\mu_{t}(dx)dt\\\
&\hskip
15.0pt=\int_{0}^{T}\int_{\mathbb{R}^{d}}v_{t}(x)\Bigl{(}-\partial_{t}v_{t}(x)-\frac{1}{2}\Delta_{x}v_{t}(x)+(V(x)-<\mu_{t},V>)v_{t}(x)-V(x)<\mu_{t},v_{t}>\Bigr{)}\mu_{t}(dx)dt\\\
&\hskip
15.0pt=-\frac{1}{2}\int_{0}^{T}\int_{\mathbb{R}^{d}}\Bigl{(}\partial_{t}v^{2}_{t}(x)+\frac{1}{2}\Delta_{x}v^{2}_{t}(x)+\phi_{t}(x)\cdot\nabla_{x}v^{2}_{t}(x)-(V(x)-<\mu_{t},V>)v^{2}_{t}(x)\Bigr{)}\mu_{t}(dx)dt\\\
&\hskip
55.0pt+\int_{0}^{T}\int_{\mathbb{R}^{d}}\Bigl{(}\frac{1}{2}(V(x)-<\mu_{t},V>)v^{2}_{t}(x)-V(x)<\mu_{t},v_{t}>v_{t}(x)\\\
&\hskip
105.0pt+\frac{1}{2}|\nabla_{x}v_{t}(x)|^{2}+\frac{1}{2}\phi_{t}(x)\cdot\nabla_{x}v^{2}_{t}(x)\Bigr{)}\mu_{t}(dx)dt\\\
&\hskip
15.0pt=\frac{1}{2}<v^{2}_{0},\mu_{0}>+\int_{0}^{T}\int_{\mathbb{R}^{d}}\Bigl{(}\frac{1}{2}(V(x)-<\mu_{t},V>)v^{2}_{t}(x)-V(x)<\mu_{t},v_{t}>v_{t}(x)\\\
&\hskip
105.0pt+\frac{1}{2}|\nabla_{x}v_{t}(x)|^{2}+\frac{1}{2}\phi_{t}(x)\cdot\nabla_{x}v^{2}_{t}(x)\Bigr{)}\mu_{t}(dx)dt\end{split}$

where we use (89) with $\varphi=v$ and the fact that $v_{T}=0$. Consequently:

$\begin{split}&\int_{0}^{T}\int_{\mathbb{R}^{d}}|\nabla_{x}v_{t}(x)|^{2}\mu_{t}(dx)dt\\\
&\leq
2\int_{0}^{T}\int_{\mathbb{R}^{d}}v_{t}(x)\bigl{(}\tilde{F}_{t}(x)+\tilde{\phi}_{t}\cdot\nabla_{x}v_{t}(x)\bigr{)}\mu_{t}(dx)dt\\\
&\hskip
15.0pt-\int_{0}^{T}\int_{\mathbb{R}^{d}}\Bigl{(}(V(x)-<\mu_{t},V>)v^{2}_{t}(x)+2V(x)<\mu_{t},v_{t}>v_{t}(x)-\phi_{t}(x)\cdot\nabla_{x}v^{2}_{t}(x)\Bigr{)}\mu_{t}(dx)dt\\\
&=2\int_{0}^{T}\int_{\mathbb{R}^{d}}v_{t}(x)\tilde{F}_{t}(x)\mu_{t}(dx)dt+2\int_{0}^{T}\int_{\mathbb{R}^{d}}v_{t}(x)\bigl{(}\tilde{\phi}_{t}(x)-\phi_{t}(x)\bigr{)}\cdot\nabla_{x}v_{t}(x)\mu_{t}(dx)dt\\\
&\hskip
15.0pt-\int_{0}^{T}\int_{\mathbb{R}^{d}}\Bigl{(}(V(x)-<\mu_{t},V>)v^{2}_{t}(x)+2V(x)<\mu_{t},v_{t}>v_{t}(x)\Bigr{)}\mu_{t}(dx)dt\\\
&\leq
2\|v\|_{\infty}\|\tilde{F}\|_{L^{1}(\bm{\mu})}+2\int_{0}^{T}\int_{\mathbb{R}^{d}}v^{2}_{t}(x)\bigl{|}\tilde{\phi}_{t}(x)-\phi_{t}(x)\bigr{|}^{2}\mu_{t}(dx)dt+\frac{1}{2}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\nabla_{x}v_{t}(x)|^{2}\mu_{t}(dx)dt+3T\|v\|_{\infty}^{2},\end{split}$

where we used the inequality $ab\leq a^{2}+b^{2}/4$. Consequently

$\int_{0}^{T}\int_{\mathbb{R}^{d}}|\nabla_{x}v_{t}(x)|^{2}\mu_{t}(dx)dt\leq
4\|v\|_{\infty}\|\tilde{F}\|_{L^{1}(\bm{\mu})}+6T\|v\|^{2}_{\infty}+4\|v\|^{2}_{\infty}\|\tilde{\bm{\phi}}-\bm{\phi}\|^{2}_{L^{2}(\bm{\mu})}$

and we conclude using the a-priori bound (78) and the fact that integrals with
respect to $\bm{\nu}$ are controlled by integrals with respect to $\bm{\mu}$.
∎

######  Remark 5.

Remark 4 implies that the proofs of the above a-priori bounds do not really
need that $\tilde{\phi}$ is bounded, but merely that it satisfies assumption
3. We stated with the boundedness assumption for the sake of simplicity.

We now tackle the issue of existence, uniqueness and regularity of solutions
of the adjoint equation.

######  Theorem 2.

If $(\bm{\phi},\bm{\mu})\in\mathbb{A}^{(2)}(\mu_{0})$ is such that $\bm{\phi}$
is bounded, the viscosity solution of the adjoint equation is a bounded
continuous function on $\mathbb{R}^{d}$ whose first order derivatives in
$x\in\mathbb{R}^{d}$ in the sense of distributions are functions in
$L^{2}([0,T]\times\mathbb{R}^{d},\bm{\mu})$ and
$L_{loc}^{2}([0,T]\times\mathbb{R}^{d},dt\,dx)$.

###### Proof.

Let us introduce the family $(\bm{\phi}^{\epsilon})_{\epsilon>0}$ of bounded
smooth Markovian feedback functions defined by
$\phi_{t}^{\epsilon}(x)=[\phi_{t}*\rho^{\epsilon}](x)$ where the convolution
is performed component by component, and where
$(\rho^{\epsilon})_{0<\epsilon\leq 1}$ is an approximate identity, the support
of $\rho^{1}$ being included in the unit ball of $\mathbb{R}^{d}$. For every
$p\in[1,\infty)$, the $C^{\infty}$ bounded functions $\bm{\phi}^{\epsilon}$
converges to $\bm{\phi}$ in $L^{p}_{loc}([0,T]\times\mathbb{R}^{d},dx\,dt)$
when $\epsilon\searrow 0$, and for each $\epsilon>0$,
$\|\bm{\phi}^{\epsilon}\|_{\infty}\leq\|\bm{\phi}\|_{\infty}$. For each
$\epsilon>0$ we set
$F^{\epsilon}_{t}(x)=\frac{1}{2}|\phi^{\epsilon}_{t}(x)|^{2}+\tilde{f}(x)$, we
denote by $w^{\epsilon}$ the classical solution of the PDE (77) for
$\tilde{\phi}_{t}(x)=\phi^{\epsilon}_{t}(x)$ and
$\tilde{F}_{t}(x)=F^{\epsilon}_{t}(x)$, and we set
$v^{\epsilon}_{t}(x)=w^{\epsilon}_{t}(x)\chi^{\epsilon}(x)$ where
$\chi^{\epsilon}$ is $C^{\infty}$, $\chi^{\epsilon}(x)=1$ if $|x|\leq
1/\epsilon$ and $\chi^{\epsilon}(x)=0$ if $|x|\geq 1+1/\epsilon$. We have:

$0=\partial_{t}w^{\epsilon}_{t}(x)+\frac{1}{2}\Delta
w^{\epsilon}_{t}(x)+\phi^{\epsilon}_{t}(x)\cdot\nabla
w^{\epsilon}_{t}(x)-(V(x)-<\mu_{t},V>)w^{\epsilon}_{t}(x)+V(x)<\mu_{t},w^{\epsilon}_{t}>+F^{\epsilon}_{t}(x)$

and we can replace $w^{\epsilon}_{t}(x)$ by $v^{\epsilon}_{t}(x)$ whenever
$|x|\leq 1/\epsilon$.

Applying the a-priori estimates (78) to $w^{\epsilon}$ we get that for each
compact set $K\subset\mathbb{R}^{d}$ we have

$\limsup_{\epsilon\searrow 0}\|v^{\epsilon}\|_{L^{2}([0,T]\times
K,dt\,dx)}\leq\limsup_{\epsilon\searrow 0}\|w^{\epsilon}\|_{L^{2}([0,T]\times
K,dt\,dx)}<\infty$

giving the existence of $v\in L^{2}_{loc}([0,T]\times\mathbb{R}^{d},dt\,dx)$
for which, after extracting a sub-sequence if needed,

$v=\lim_{\epsilon\searrow 0}v^{\epsilon}$

locally (i.e. for each compact subset $K$) for the weak topology. Also, the
a-priori estimates (78) implies that the $v^{\epsilon}$’s form a bounded set
in $L^{2}(\bm{\mu})$, so extracting a further sub-sequence if necessary, one
can assume that $v^{\epsilon}$ converges weakly toward $v$ in
$L^{2}(\bm{\mu})$ implying that, extracting a further subsequence if needed,
for almost every $t\in[0,T]$:

(81) $\lim_{\epsilon\searrow
0}<\mu_{t},w^{\epsilon}_{t}>=\lim_{\epsilon\searrow
0}<\mu_{t},v^{\epsilon}_{t}>=\lim_{\epsilon\searrow
0}<1,v^{\epsilon}_{t}>_{L^{2}(\mu_{t})}=<1,v_{t}>_{L^{2}(\mu_{t})}=<\mu_{t},v_{t}>.$

Furthermore, the a-priori estimate (79) implies that one can extract a further
sub-sequence for which $\nabla_{x}v^{\epsilon}$ converges weakly in
$L^{2}([0,T]\times\mathbb{R}^{d},\bm{\mu};\mathbb{R}^{d})$ toward a
$\mathbb{R}^{d}$-valued function $\tilde{\varphi}$ in
$L^{2}([0,T]\times\mathbb{R}^{d},\bm{\mu};\mathbb{R}^{d})$. This measurable
vector field $\tilde{\varphi}$ can be identified with the gradient (in the
sense of distributions) of $v$. Indeed, if $\varphi$ is a smooth test function
with compact support in $(0,T)\times\mathbb{R}^{d}$

$\lim_{\epsilon\searrow
0}\int_{0}^{T}\int_{\mathbb{R}^{d}}\nabla_{x}v^{\epsilon}_{t}(x)\varphi_{t}(x)dxdt=-\lim_{\epsilon\searrow
0}\int_{0}^{T}\int_{\mathbb{R}^{d}}\nabla_{x}\varphi_{t}(x)v^{\epsilon}_{t}(x)dxdt=-\int_{0}^{T}\int_{\mathbb{R}^{d}}\nabla_{x}\varphi_{t}(x)v_{t}(x)\;dxdt$

and since $\bm{\phi}$ is assumed to be bounded

$\lim_{\epsilon\searrow
0}\int_{0}^{T}\int_{\mathbb{R}^{d}}\nabla_{x}v^{\epsilon}_{t}(x)\varphi_{t}(x)dx=\lim_{\epsilon\searrow
0}\int_{0}^{T}\int_{\mathbb{R}^{d}}\nabla_{x}v^{\epsilon}_{t}(x)\frac{\varphi_{t}(x)}{\rho_{t}(x)}\mu_{t}(dx)=\int_{0}^{T}\int_{\mathbb{R}^{d}}\tilde{\varphi}_{t}(x)\varphi_{t}(x)\;dx$

where we used part (iii) of Proposition 1 to benefit from the fact that the
continuous density $\rho$ is locally strictly positive and bounded below away
from $0$. This shows that the gradient in the sense of distributions of $v$ is
a function (namely the function $\tilde{\varphi}$), and that
$\nabla_{x}v^{\epsilon}$ converges weakly in
$L_{loc}^{2}([0,T]\times\mathbb{R}^{d},dx\,dt;\mathbb{R}^{d})$ toward
$\tilde{\varphi}=\nabla_{x}v$. This implies that

(82) $\lim_{\epsilon\searrow
0}\int_{0}^{T}\int_{\mathbb{R}^{d}}\varphi_{t}(x)\;\phi^{\epsilon}_{t}(x)\cdot\nabla
w^{\epsilon}_{t}(x)\;dxdt=\int_{0}^{T}\int_{\mathbb{R}^{d}}\varphi_{t}(x)\;\phi_{t}(x)\cdot\nabla
v_{t}(x)\;dxdt$

where $\nabla v_{t}(x)$ is the function $\tilde{\varphi}_{t}(x)$ identified as
the gradient in the sense of distributions of the function $v$ constructed
above. Indeed, if $\epsilon>0$ is small enough,

$\begin{split}&\int_{0}^{T}\int_{\mathbb{R}^{d}}\varphi_{t}(x)\;\phi^{\epsilon}_{t}(x)\cdot\nabla
w^{\epsilon}_{t}(x)\;dxdt-\int_{0}^{T}\int_{\mathbb{R}^{d}}\varphi_{t}(x)\;\phi_{t}(x)\cdot\nabla
v_{t}(x)\;dxdt\\\ &\hskip
45.0pt=\int_{0}^{T}\int_{\mathbb{R}^{d}}\varphi_{t}(x)\;\bigl{(}\phi^{\epsilon}_{t}(x)-\phi_{t}(x)\bigr{)}\cdot\nabla
v^{\epsilon}_{t}(x)\;dxdt+\int_{0}^{T}\int_{\mathbb{R}^{d}}\varphi_{t}(x)\;\phi_{t}(x)\cdot\bigl{(}\nabla
v^{\epsilon}_{t}(x)-\nabla v_{t}(x)\bigr{)}\;dxdt.\end{split}$

The first term goes to $0$ because

$\limsup_{\epsilon\searrow
0}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\varphi_{t}(x)|\;|\nabla
v^{\epsilon}_{t}(x)|^{2}\;dxdt<\infty$

since the $L^{2}(\bm{\mu})$-norm of $\nabla_{x}v^{\epsilon}$ is uniformly
bounded because of the a-priori estimate (79) and the fact that the density of
$\mu_{t}$ is bounded from below away from $0$ on the support of $\varphi$, and
the fact that $\phi^{\epsilon}$ converges toward $\phi$ in
$L^{2}_{loc}(dxdt)$. As for the second term, it also converges toward $0$
because $\nabla_{x}v^{\epsilon}$ converges toward
$\tilde{\varphi}=\nabla_{x}v$ weakly in $L^{2}_{loc}(dxdt)$. Again, if
$\varphi$ is a smooth test function with compact support in
$(0,T)\times\mathbb{R}^{d}$, we have

(83) $\begin{split}&\lim_{\epsilon\searrow
0}\int_{0}^{T}\int_{\mathbb{R}^{d}}\Bigl{[}-\partial_{t}\varphi_{t}(x)+\frac{1}{2}\Delta\varphi_{t}(x)-(V(x)-<\mu_{t},V>)\varphi_{t}(x)\Bigr{]}v^{\epsilon}_{t}(x)dxdt\\\
&\hskip
35.0pt=\int_{0}^{T}\int_{\mathbb{R}^{d}}\Bigl{[}-\partial_{t}\varphi_{t}(x)+\frac{1}{2}\Delta\varphi_{t}(x)-(V(x)-<\mu_{t},V>)\varphi_{t}(x)\Bigr{]}v_{t}(x)dxdt.\end{split}$

Moreover

(84) $\lim_{\epsilon\searrow
0}\int_{0}^{T}\int_{\mathbb{R}^{d}}\varphi_{t}(x)|\phi^{\epsilon}_{t}(x)|^{2}dxdt=\int_{0}^{T}\int_{\mathbb{R}^{d}}\varphi_{t}(x)|\phi_{t}(x)|^{2}dxdt.$

Putting together the limits (81), (3.6.2), (83) and (84) we find that

(85)
$\begin{split}&0=\int_{0}^{T}\int_{\mathbb{R}^{d}}\Bigl{(}\bigl{[}-\partial_{t}\varphi_{t}(x)+\frac{1}{2}\Delta\varphi_{t}(x)-(V(x)-<\mu_{t},V>)\varphi_{t}(x)\bigr{]}v_{t}(x)+\varphi_{t}(x)\phi_{t}(x)\cdot\nabla_{x}v_{t}(x)\\\
&\hskip
95.0pt+\bigl{[}<\mu_{t},v_{t}>V(x)+\frac{1}{2}|\phi_{t}(x)|^{2}+\tilde{f}(x)\bigr{]}\varphi_{t}(x)\Bigr{)}dxdt.\end{split}$

once more, given the test function $\varphi$, we use the fact that
$w^{\epsilon}$ and $v^{\epsilon}$ coincide on the support of $\varphi$ for
$\epsilon>0$ small enough. This proves that $v$ is a weak solution (in the
sense of distributions) of the adjoint equation. ∎

####  3.6.3. A Maximum Principle

This subsection is devoted to the statement and the proof of a version of the
Pontryagin maximum principle tailored to our needs. Throughout the subsection
we assume that $\bm{\phi}=(\phi_{t})_{0\leq t\leq T}$ is a bounded measurable
feedback control function, that $\bm{\mu}=(\mu_{t})_{0\leq t\leq T}$ is the
solution of the corresponding FPK equation (31), and that $u$ is a solution of
the associated adjoint equation (66). If $\bm{\beta}=(\beta_{t})_{0\leq t\leq
T}$ is another bounded measurable feedback control function, for each
$\epsilon>0$, we define $\bm{\phi}^{\epsilon}=(\phi^{\epsilon}_{t})_{0\leq
t\leq T}$ as $\phi^{\epsilon}_{t}=\phi_{t}+\epsilon\beta_{t}$, and we denote
by $\bm{\mu}^{\epsilon}=(\mu^{\epsilon}_{t})_{0\leq t\leq T}$ the solution of
the FPK equation driven by the control $\phi^{\epsilon}_{t}$.

######  Lemma 9.

For each $t\in[0,T]$ and each bounded measurable function $\varphi$ on
$\mathbb{R}^{d}$, the limit

$\lim_{\epsilon\searrow
0}<\varphi,\frac{\mu_{t}^{\epsilon}-\mu_{t}}{\epsilon}>$

exists and is given by the integral of $\varphi$ with respect to a finite
signed measure $\lambda_{t}$ such that $\lambda_{t}(\mathbb{R}^{d})=0$ and
satisfying the PDE

(86)
$\partial_{t}\lambda_{t}=\frac{1}{2}\Delta\lambda_{t}-\operatorname{div}(\phi_{t}\lambda_{t})-\operatorname{div}(\beta_{t}\mu_{t})+<\lambda_{t},V>\mu_{t}-(V-<\mu_{t},V>)\lambda_{t}$

in the sense of distributions with initial condition $\lambda_{0}=0$.

###### Proof.

If we denote by $X_{t}$ (resp. $X^{\epsilon}_{t}$) the state controlled by
$\phi_{t}$ (resp. $\phi^{\epsilon}_{t}$), i.e. the solution of the stochastic
differential equation $dX_{t}=\Phi_{t}(X_{t})dt+dW_{t}$ (resp.
$dX^{\epsilon}_{t}=\Phi^{\epsilon}_{t}(X^{\epsilon}_{t})dt+dW_{t}$),
Girsanov’s theorem gives:

$\begin{split}\frac{1}{\epsilon}(<\varphi,\mu_{t}^{\epsilon}>-<\varphi,\mu_{t}>)&=\frac{1}{\epsilon}\Bigl{(}\frac{\mathbb{E}\bigl{[}\varphi(X^{\epsilon}_{t})e^{-\int_{0}^{t}V(X^{\epsilon}_{s})ds}\bigr{]}}{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X^{\epsilon}_{s})ds}\bigr{]}}-\frac{\mathbb{E}\bigl{[}\varphi(X_{t})e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}\Bigr{)}\\\
&=\frac{1}{\epsilon}\Bigl{(}\frac{\mathbb{E}\bigl{[}\varphi(X^{\epsilon}_{t})e^{-\int_{0}^{t}V(X^{\epsilon}_{s})ds}\bigr{]}-\mathbb{E}\bigl{[}\varphi(X_{t})e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X^{\epsilon}_{s})ds}\bigr{]}}\\\
&\hskip
45.0pt-\mathbb{E}\bigl{[}\varphi(X_{t})e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}\Bigl{(}\frac{1}{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}-\frac{1}{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X^{\epsilon}_{s})ds}\bigr{]}}\Bigr{)}\Bigr{)}\\\
&=\frac{1}{\epsilon}\frac{\mathbb{E}\bigl{[}\varphi(X_{t})e^{-\int_{0}^{t}V(X_{s})ds}\bigl{(}e^{\epsilon\int_{0}^{t}\beta_{s}(X_{s})dW_{s}-\frac{\epsilon^{2}}{2}\int_{0}^{t}|\beta_{s}(X_{s})|^{2}ds}-1\bigr{)}\bigr{]}}{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X^{\epsilon}_{s})ds}\bigr{]}}\\\
&\hskip
45.0pt-\frac{\mathbb{E}\bigl{[}\varphi(X_{t})e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}\frac{1}{\epsilon}\frac{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X^{\epsilon}_{s})ds}\bigr{]}-\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X^{\epsilon}_{s})ds}\bigr{]}}\end{split}$

from which we conclude

(87) $\lim_{\epsilon\searrow
0}<\varphi,\frac{\mu_{t}^{\epsilon}-\mu_{t}}{\epsilon}>=\frac{\mathbb{E}\bigl{[}\varphi(X_{t})\bigl{(}\int_{0}^{t}\beta_{s}(X_{s})dW_{s}\bigr{)}e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}-<\varphi,\mu_{t}>\frac{\mathbb{E}\Bigl{[}\bigl{(}\int_{0}^{t}\beta_{s}(X_{s})dW_{s}\bigr{)}e^{-\int_{0}^{t}V(X_{s})ds}\Bigr{]}}{\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}}$

because all the moments of the random variable
$e^{-\epsilon\int_{0}^{t}\beta_{s}(X_{s})dW_{s}-\frac{\epsilon^{2}}{2}\int_{0}^{t}|\beta_{s}(X_{s})|^{2}ds}$
are finite, because

$\lim_{\epsilon\searrow
0}\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X^{\epsilon}_{s})ds}\bigr{]}=\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]},$

and because

$\begin{split}\lim_{\epsilon\searrow
0}\frac{1}{\epsilon}\Bigl{(}\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X^{\epsilon}_{s})ds}\bigr{]}-\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X_{s})ds}\bigr{]}\Bigr{)}&=\frac{1}{\epsilon}\Bigl{(}\mathbb{E}\bigl{[}e^{-\int_{0}^{t}V(X_{s})ds}\bigl{[}e^{\epsilon\int_{0}^{t}\beta_{s}(X_{s})dW_{s}-\frac{\epsilon^{2}}{2}\int_{0}^{t}|\beta_{s}(X_{s})|^{2}ds}-1\bigr{]}\Bigr{)}\\\
&=\mathbb{E}\Bigl{[}\bigl{(}\int_{0}^{t}\beta_{s}(X_{s})dW_{s}\bigr{)}e^{-\int_{0}^{t}V(X_{s})ds}\Bigr{]}.\end{split}$

Notice that the right hand side of (87) is a linear form in $\varphi$ which
defines a signed measure $\lambda_{t}$ with total mass $0$. In order to
identify the PDE satisfied by $\lambda_{t}$ one can use the rules of Itô
calculus to compute the time derivative of $<\varphi,\lambda_{t}>$ by
computing the derivative of the right hand side of (87). Since these
computations are long and tedious, we choose a more direct approach.
Subtracting one FPK equation from another we find that

$\partial_{t}\frac{\mu^{\epsilon}_{t}-\mu_{t}}{\epsilon}=\frac{1}{2}\Delta\frac{\mu^{\epsilon}_{t}-\mu_{t}}{\epsilon}-\operatorname{div}\bigl{(}\phi_{t}\frac{\mu^{\epsilon}_{t}-\mu_{t}}{\epsilon}\bigr{)}-\operatorname{div}\bigl{(}\frac{\phi^{\epsilon}_{t}-\phi_{t}}{\epsilon}\mu^{\epsilon}_{t}\bigr{)}+<\frac{\mu^{\epsilon}_{t}-\mu_{t}}{\epsilon},V>\mu_{t}-(V-<\mu_{t},V>)\frac{\mu^{\epsilon}_{t}-\mu_{t}}{\epsilon}$

so taking the limit $\epsilon\searrow 0$ we derive the PDE (86). ∎

######  Lemma 10.

$\nabla u$ is square integrable with respect to the measure $|\lambda|$ in the
sense that

(88) $\int_{0}^{T}\int_{\mathbb{R}^{d}}|\nabla
u_{t}(x)|^{2}|\lambda_{t}|(dx)dt<\infty.$

The strategy of the proof is similar to the proof of estimate (79) in Lemma 8.

###### Proof.

The PDE (86) implies that for any test function $\varphi$ we have

(89)
$\begin{split}&\int_{0}^{T}\int\Bigl{[}\partial_{t}\varphi^{2}+\frac{1}{2}\Delta\varphi^{2}+\phi_{t}\nabla\varphi^{2}-(V-<\mu_{t},V>)\varphi^{2}\Bigr{]}\lambda_{t}(dx)dt\\\
&\hskip
45.0pt+\int_{0}^{T}\int\Bigl{[}\beta_{t}\nabla\varphi^{2}+<V,\lambda_{t}>\varphi^{2}\Bigr{]}\mu_{t}(dx)dt=<\varphi^{2},\lambda_{T}>.\end{split}$

Using the fact that $u$ solves the adjoint equation (66) we can write

$\begin{split}&\int_{0}^{T}\int
u_{t}\bigl{[}f\bigl{(}.,\phi_{t}(.)\bigr{)}+\phi_{t}\nabla
u_{t}\bigr{]}\lambda_{t}(dx)dt\\\ &\hskip 35.0pt=\int_{0}^{T}\int
u_{t}\bigl{[}-\partial_{t}u_{t}-\frac{1}{2}\Delta
u_{t}+(V-<\mu_{t},V>)u_{t}-V<\mu_{t},u_{t}>\bigr{]}\lambda_{t}(dx)dt\\\
&\hskip
35.0pt=-\frac{1}{2}\int_{0}^{T}\int\bigl{[}\partial_{t}u_{t}^{2}+\frac{1}{2}\Delta
u_{t}^{2}+\phi_{t}\nabla
u_{t}^{2}-(V-<\mu_{t},V>)u_{t}^{2}\bigr{]}\lambda_{t}(dx)dt\\\ &\hskip
75.0pt+\int_{0}^{T}\int\bigl{[}\frac{1}{2}|\nabla
u_{t}|^{2}+\frac{1}{2}\phi_{t}\nabla
u_{t}^{2}+\frac{1}{2}(V-<\mu_{t},V>)u_{t}^{2}-V<\mu_{t},u_{t}>u_{t}\bigr{]}\lambda_{t}(dx)dt\\\
&\hskip 35.0pt=\frac{1}{2}\int_{0}^{T}\int\bigl{[}\beta_{t}\nabla
u_{t}^{2}+<V,\lambda_{t}>u_{t}^{2}\bigr{]}\mu_{t}(dx)dt+\frac{1}{2}<u_{T}^{2},\lambda_{T}>\\\
&\hskip 75.0pt+\int_{0}^{T}\int\bigl{[}\frac{1}{2}|\nabla
u_{t}|^{2}+\frac{1}{2}\phi_{t}\nabla
u_{t}^{2}+\frac{1}{2}(V-<\mu_{t},V>)u_{t}^{2}-V<\mu_{t},u_{t}>u_{t}\bigr{]}\lambda_{t}(dx)dt\end{split}$

from which we get

$\begin{split}\frac{1}{2}\int_{0}^{T}\int_{\mathbb{R}^{d}}|\nabla
u_{t}|^{2}\lambda_{t}(dx)dt&=\int_{0}^{T}\int
u_{t}\bigl{[}f\bigl{(}.,\phi_{t}(.)\bigr{)}+\phi_{t}\nabla
u_{t}\bigr{]}\lambda_{t}(dx)dt\\\ &\hskip
35.0pt-\frac{1}{2}\int_{0}^{T}\int\bigl{[}\beta_{t}\nabla
u_{t}^{2}+<V,\lambda_{t}>u_{t}^{2}\bigr{]}\mu_{t}(dx)dt-\frac{1}{2}<u_{T}^{2},\lambda_{T}>\\\
&\hskip 35.0pt-\int_{0}^{T}\int\bigl{[}\frac{1}{2}\phi_{t}\nabla
u_{t}^{2}+\frac{1}{2}(V-<\mu_{t},V>)u_{t}^{2}-V<\mu_{t},u_{t}>u_{t}\bigr{]}\lambda_{t}(dx)dt\\\
&\leq\|u\|_{\infty}\|f\|_{\infty}|\lambda|([0,T]\times\mathbb{R}^{d})+2\|u\|_{\infty}^{2}\|\phi\|_{\infty}^{2}|\lambda|([0,T]\times\mathbb{R}^{d})\\\
&\hskip 35.0pt+\frac{1}{8}\int_{0}^{T}|\int\nabla
u_{t}|^{2}\lambda_{t}(dx)dt+\frac{1}{2}\|\beta\|_{\infty}\Bigl{(}T\|u\|_{\infty}^{2}+\int_{0}^{T}\int|\nabla
u_{t}|^{2}\mu_{t}(dx)dt\Bigr{)}\\\ &\hskip
35.0pt+2(\|u\|_{\infty}^{2}+\|u\|_{\infty})|\lambda|([0,T]\times\mathbb{R}^{d})\\\
&\hskip
35.0pt+2T\|\phi\|_{\infty}^{2}\|u\|_{\infty}^{2}+\frac{1}{8}\int_{0}^{T}\int|\nabla
u_{t}|^{2}\lambda_{t}(dx)dt\end{split}$

from which we conclude. ∎

######  Proposition 4.

Let $\bm{\phi}=(\phi_{t})_{0\leq t\leq T}$ be a bounded measurable feedback
control function, let $\bm{\mu}=(\mu_{t})_{0\leq t\leq T}$ be the
corresponding solution of the FPK equation (31), and let $u$ the solution of
the corresponding adjoint equation (66). If $\bm{\beta}=(\beta_{t})_{0\leq
t\leq T}$ is another bounded measurable feedback control function, we have:

(90)
$\frac{d}{d\epsilon}J(\bm{\phi}+\epsilon\bm{\beta})\Bigr{|}_{\epsilon=0}=\int_{0}^{T}<\beta_{t}(\nabla
u_{t}+\phi_{t}),\mu_{t}>\,dt.$

As a result, if $\bm{\phi}$ is a critical point, then

(91) $\phi_{t}(x)=-\nabla
u_{t}(x),\qquad\qquad\mu_{t}-a.s.\;x\in\mathbb{R}^{d},\quad a.e.\;t\in[0,T].$

###### Proof.

We use freely the notations introduced at the beginning of the subsection and
in the statement of the previous lemma. Step 1. We derive a first expression
for the derivative (90).

$\begin{split}&\frac{1}{\epsilon}\bigl{[}J(\bm{\phi}+\epsilon\bm{\beta})-J(\bm{\phi})\bigr{]}\\\
&\hskip
25.0pt=\frac{1}{\epsilon}\int_{0}^{T}\Bigl{(}<\tilde{f},\mu^{\epsilon}_{t}>+\frac{1}{2}<|\phi^{\epsilon}_{t}|^{2},\mu^{\epsilon}_{t}>-<\tilde{f},\mu_{t}>-\frac{1}{2}<|\phi_{t}|^{2},\mu_{t}>\Bigr{)}\;dt+\frac{1}{\epsilon}<g,\mu^{\epsilon}_{T}-\mu_{T}>\\\
&\hskip
25.0pt=\int_{0}^{T}\Bigl{(}<\tilde{f},\frac{\mu^{\epsilon}_{t}-\mu_{t}}{\epsilon}>+\frac{1}{2}<|\phi_{t}|^{2},\frac{\mu^{\epsilon}_{t}-\mu_{t}}{\epsilon}>+\frac{1}{2}<\frac{|\phi^{\epsilon}_{t}|^{2}-|\phi_{t}|^{2}}{\epsilon},\mu^{\epsilon}_{t}>\Bigr{)}dt+<g,\frac{\mu^{\epsilon}_{T}-\mu_{T}}{\epsilon}>\end{split}$

so that

(92) $\lim_{\epsilon\searrow
0}\frac{1}{\epsilon}\bigl{[}J(\bm{\phi}+\epsilon\bm{\beta})-J(\bm{\phi})\bigr{]}=\int_{0}^{T}\Bigl{(}<\tilde{f},\lambda_{t}>+\frac{1}{2}<|\phi_{t}|^{2},\lambda_{t}>+<\beta_{t}\phi_{t},\mu_{t}>\Bigr{)}dt+<g,\lambda_{T}>.$

where we used the stronger form of convergence toward $\lambda_{t}$ proven in
Lemma 9 because $|\phi_{t}|^{2}$ is merely bounded measurable and not
necessarily continuous. Step 2. We now prove the formula

(93) $<u_{T},\lambda_{T}>=\int_{0}^{T}\Bigl{(}<\beta_{t}\nabla
u_{t},\mu_{t}>-\bigl{(}<\tilde{f},\lambda_{t}>+\frac{1}{2}<|\phi_{t}|^{2},\lambda_{t}>\bigr{)}\Bigr{)}\,dt$

where the function $u$ is the solution of the adjoint equation driven by the
control $\bm{\phi}$. Recall that Theorem 2 says that $u$ is bounded and
continuous and that its derivatives in the sense of distributions are square
integrable functions with respect to $\bm{\mu}$.

Let us define the functions $u_{t}^{n}(x)=\chi^{n}(x)u_{t}(x)$ and
$\lambda_{t}^{n}(dx)=\chi^{n}(x)\lambda_{t}(dx)$ where the sequence
$(\chi^{n})_{n\geq 1}$ of $C^{\infty}$ functions is such that
$0\leq\chi^{n}(x)\leq 1$, $\chi^{n}(x)=1$ if $|x|\leq n$, $\chi^{n}(x)=0$ if
$|x|>n+1$ and its derivatives of all orders are uniformly bounded. We first
notice that

(94) $<u_{T},\lambda_{T}>=\lim_{n\nearrow\infty}<u^{n}_{T},\lambda^{n}_{T}>$

because $u_{T}$ is bounded. Next, using the facts that
$\partial_{t}u^{n}_{t}=\chi^{n}\partial_{t}u_{t}$,
$\partial_{t}\lambda^{n}_{t}=\chi^{n}\partial_{t}\lambda_{t}$ and
$\lambda_{0}=0$ , we get:

$\begin{split}<u^{n}_{T},\lambda^{n}_{T}>&=<u^{n}_{0},\lambda^{n}_{0}>+\int_{0}^{T}<u^{n}_{t},\partial_{t}\lambda^{n}_{t}>+\int_{0}^{T}<\partial_{t}u^{n}_{t},\lambda^{n}_{t}>\\\
&=\int_{0}^{T}\Bigl{(}<\chi^{n}u_{t},\frac{1}{2}\chi^{n}\Delta\lambda_{t}>-<\chi^{n}u_{t},\chi^{n}\operatorname{div}(\phi_{t}\lambda_{t})>-<\chi^{n}u_{t},\chi^{n}\operatorname{div}(\beta_{t}\mu_{t})>\\\
&\hskip
95.0pt+<\lambda_{t},V><\chi^{n}u_{t},\chi^{n}\mu_{t}>-<\chi^{n}u_{t},\chi^{n}(V-<\mu_{t},V>)\lambda_{t}>\\\
&\hskip 25.0pt+<-\frac{1}{2}\chi^{n}\Delta
u_{t},\chi^{n}\lambda_{t}>-<\chi^{n}\phi_{t}\nabla
u_{t},\chi^{n}\lambda_{t}>+<\chi^{n}(V-<\mu_{t},V>)u_{t},\chi^{n}\lambda_{t}>\\\
&\hskip
95.0pt-<\mu_{t},u_{t}><\chi^{n}V,\chi^{n}\lambda_{t}>-\frac{1}{2}<\chi^{n}|\phi_{t}|^{2},\chi^{n}\lambda_{t}>-<\chi^{n}\tilde{f},\chi^{n}\lambda_{t}>\Bigr{)}\;dt.\end{split}$

Using integration by parts we get:

$\begin{split}<\chi^{n}u_{t},\chi^{n}\Delta\lambda_{t}>&=<(\chi^{n})^{2}u_{t},\Delta\lambda_{t}>\\\
&=-<(\chi^{n})^{2}\nabla
u_{t},\nabla\lambda_{t}>-<\nabla(\chi^{n})^{2}\;u_{t},\nabla\lambda_{t}>\\\
&=-<(\chi^{n})^{2}\nabla
u_{t},\nabla\lambda_{t}>+<\nabla\bigl{(}\nabla(\chi^{n})^{2}u_{t}\bigr{)},\lambda_{t}>,\end{split}$

and similarly

$\begin{split}<\chi^{n}\Delta u_{t},\chi^{n}\lambda_{t}>&=<\Delta
u_{t},(\chi^{n})^{2}\lambda_{t}>\\\ &=-<\nabla
u_{t},(\chi^{n})^{2}\nabla\lambda_{t}>-<\nabla(\chi^{n})^{2}\nabla
u_{t},\lambda_{t}>,\end{split}$

so that

$\lim_{n\to\infty}<\chi^{n}u_{t},\chi^{n}\Delta\lambda_{t}>-<\chi^{n}\Delta
u_{t},\chi^{n}\lambda_{t}>=\lim_{n\to\infty}<\nabla\bigl{(}\nabla(\chi^{n})^{2}u_{t}\bigr{)},\lambda_{t}>-\lim_{n\to\infty}<\nabla(\chi^{n})^{2}\nabla
u_{t},\lambda_{t}>=0$

because $u_{t}$ is bounded, and $\nabla u_{t}$ is square integrable for
$\lambda_{t}$ according to Lemma 10. Again, we use the fact that the
derivatives of $\chi^{n}$ are uniformly bounded and supported in the annulus
$n\leq|x|\leq n+1$ whose $\lambda_{t}$ measure tends to $0$. Next

$\lim_{n\to\infty}<\chi^{n}u_{t},\chi^{n}\operatorname{div}(\phi_{t}\lambda_{t})>+<\chi^{n}\phi_{t}\nabla
u_{t},\chi^{n}\lambda_{t}>=-\lim_{n\to\infty}<\nabla(\chi^{n})^{2}\phi_{t}u_{t},\lambda_{t}>=0$

because $\phi_{t}$ and $u_{t}$ are bounded. Similarly

$\lim_{n\to\infty}<\chi^{n}u_{t},\chi^{n}\operatorname{div}(\beta_{t}\mu_{t})>=-<\beta_{t}\nabla
u_{t},\mu_{t}>,$

as well as

$\lim_{n\to\infty}<\chi^{n}|\phi_{t}|^{2},\chi^{n}\lambda_{t})>=<|\phi_{t}|^{2},\lambda_{t}>,$

and

$\lim_{n\to\infty}<\chi^{n}\tilde{f},\chi^{n}\lambda_{t})>=<\tilde{f},\lambda_{t}>.$

Finally

$\lim_{n\to\infty}<\lambda_{t},V><\chi^{n}u_{t},\chi^{n}\mu_{t}>-<\mu_{t},u_{t}><\chi^{n}V,\chi^{n}\lambda_{t}>=0,$

since $V$ is bounded. This concludes the proof of Step 2.

Step 3. We conclude the proof of the proposition by plugging expression (93)
in (92), and obtain the expression of the derivative of the expected cost
stated in (90). From this we conclude that (91) holds whenever $\bm{\phi}$ is
a critical point of the expected cost $J$. ∎

####  3.6.4. Value Function.

We now introduce the value function:

(95) $U^{(1)}(t,\mu)=\inf_{(\phi_{s},\mu_{s})_{t\leq s\leq
T}}\int_{t}^{T}F^{(1)}(\mu_{s},\phi_{s})\,ds+G^{(1)}(\mu_{T}),$

where the infimum is over the couples
$(\bm{\phi},\bm{\mu})\in\mathbb{A}^{(2)}_{[t,T]}$ satisfying the _initial
condition_ $\mu_{t}=\mu$. Because of the dynamic programming principle, this
value function is expected to solve in some appropriate sense the HJB
equation:

(96) $\partial_{t}U^{(1)}(t,\mu)+\mathbb{H}^{(1)*}\left(\mu,\frac{\delta
U^{(1)}}{\delta\mu}(t,\mu)\right)=0$

with terminal condition $U^{(1)}(T,\mu)=G^{(1)}(\mu)$, where the minimized
Hamiltonian $\mathbb{H}^{(1)*}(\mu,\varphi)$ is defined as:

$\mathbb{H}^{(1)*}(\mu,\varphi)=\inf_{\phi}\mathbb{H}^{(1)}(\mu,\varphi,\phi),$

where the infimum is taken over the $A$-valued measurable functions $\phi$ on
$\mathbb{R}^{d}$ satisfying $\int|\phi(x)|^{2}\mu(dx)<\infty$.

####  3.6.5. Minimization of the Hamiltonian.

The following assumption is made in order to be able to use lighter notations.

######  Assumption 4.

We assume the existence of a function
$\hat{\phi}:(\mu,\varphi)\mapsto\hat{\phi}[\mu,\varphi]$ which is an
$A$-valued measurable function on $\mathbb{R}^{d}$ satisfying for every
$(\mu,\varphi)$ satisfying

$\int_{\mathbb{R}^{d}}|\hat{\phi}[\mu,\varphi]|^{2}\mu(dx)<\infty,\qquad\text{and}\qquad\hat{\phi}[\mu,\varphi]\in\operatorname*{arg\,inf}_{\phi}\mathbb{H}^{(1)}(\mu,\varphi,\phi).$

Under this assumption, $\mathbb{H}^{(1)*}(\mu,\varphi)$ can be written as:

$\mathbb{H}^{(1)*}(\mu,\varphi)=\mathbb{H}^{(1)}(\mu,\varphi,\hat{\phi}[\mu,\varphi]),$

and the HJB equation (96) can be conveniently rewritten as:

(97) $\partial_{t}U^{(1)}(t,\mu)+\mathbb{H}^{(1)}\left(\mu,\frac{\delta
U^{(1)}}{\delta\mu}(t,\mu),\hat{\phi}\Bigl{[}\mu,\frac{\delta
U^{(1)}}{\delta\mu}(t,\mu)\Bigr{]}\right)=0.$

Recall that $\frac{\delta U^{(1)}}{\delta\mu}(t,\mu)$ is understood as a
functional “flat” derivative, and as such, it is a function of
$x\in\mathbb{R}^{d}$.

######  Example.

In the particular case of the separable running cost function $f$ given by
(5), if we assume that the derivative of the function $\varphi$ in the sense
of distributions is a function, we have:

$\begin{split}\hat{\phi}[\mu,\varphi]&\in\operatorname*{arg\,inf}_{\phi}-\frac{1}{2}<\nabla\mu,\nabla\varphi>+<\mu,\phi\nabla\varphi>-<\mu,(V-<\mu,V>)\varphi>\\\
&\hskip
135.0pt+\frac{1}{2}\int|\phi(x)|^{2}\;\mu(dx)+\int\tilde{f}(x)\mu(dx)\\\
&=\operatorname*{arg\,inf}_{\phi}\int_{\mathbb{R}^{d}}[\phi(x)\cdot\nabla\varphi(x)+\frac{1}{2}|\phi(x)|^{2}]\;\mu(dx),\end{split}$

from which we get:

(98)
$\hat{\phi}[\mu,\varphi](x)=\Pi_{A}\bigl{(}-\nabla\varphi(x)\bigr{)},\qquad\qquad
x\in\mathrm{supp}(\mu)$

by minimizing under the integral sign for each fixed $x\in\mathrm{supp}(\mu)$,
where the notation $\text{supp}(\mu)$ stands for the topological support of
the measure $\mu$, and $\Pi_{A}$ for the projection of $\mathbb{R}^{d}$ onto
the closed convex set $A$, defined as:

(99) $\Pi_{A}(x)=\operatorname*{arg\,inf}_{y\in A}\|x-y\|$

which gives the element of $A$ at the shortest distance from $x$. For each
$\varphi\in C_{c}^{\infty}(\mathbb{R}^{d})$,
$x\hookrightarrow-\nabla\varphi(x)$ is a bounded measurable function on
$\mathbb{R}^{d}$ with values in $\mathbb{R}^{d}$. Unfortunately, we are not
guaranteed that it takes values in $A$ as the definition of admissibility
requires, hence the need for the projection $\Pi_{A}$. Still, this projection
remains bounded and the feedback function $\hat{\phi}(\mu,\varphi)$ is
admissible.

If, for the sake of simplicity, we take $A=\mathbb{R}^{d}$, we then have:

(100) $\hat{\phi}[\mu,\varphi](x)=-\nabla\varphi(x)$

and the minimized Hamiltonian takes the form

(101)
$\begin{split}\inf_{\phi}\mathbb{H}^{(1)}(\mu,\varphi,\phi)&=\int\mu(dx)\Bigl{[}\frac{1}{2}\Delta\varphi(x)-\frac{1}{2}|\nabla\varphi(x)|^{2}-(V(x)-<\mu,V>)\varphi(x)+\tilde{f}(x)\Bigr{]}.\end{split}$

Then, the HJB equation reads:

(102)
$\begin{split}&0=\partial_{t}U^{(1)}(t,\mu)+\int\mu(dx)\Bigl{[}\frac{1}{2}\Delta_{x}\frac{\delta
U^{(1)}}{\delta\mu}(t,\mu)(x)-\frac{1}{2}|\nabla_{x}\frac{\delta
U^{(1)}}{\delta\mu}(t,\mu)(x)|^{2}\\\ &\hskip
85.0pt-(V(x)-<\mu,V>)\frac{\delta
U^{(1)}}{\delta\mu}(t,\mu)(x)+\tilde{f}(x)\Bigr{]}.\end{split}$

####  3.6.6. The Forward-Backward PDE System

As before, we assume that the optimal control $\hat{\bm{\phi}}$ whose
existence we proved in Proposition 2 is bounded. Then, the form of the
Pontryagin maximum principle proved in Proposition 4 implies that
$\hat{\phi}_{t}(x)=-\nabla_{x}\hat{u}_{t}(x)$ $\hat{\mu}_{t}$ \- almost surely
where $\hat{\bm{\mu}}$ is the solution of the FPK equation driven by the
control $\hat{\bm{\phi}}$ and $\hat{u}_{t}$ is the solution of the
corresponding adjoint equation. So injecting this formula in the FKP and the
adjoint equations we see that the couple $(\hat{\mu},\hat{u})$ is a solution
of the forward backward PDE system:

(103a)
$\displaystyle\partial_{t}\mu=\frac{1}{2}\Delta_{x}\mu+\operatorname{div}_{x}(\nabla_{x}u\;\mu)-(V-<\mu,V>)\mu$
(103b) $\displaystyle
0=\partial_{t}u+\frac{1}{2}\Delta_{x}u-\frac{1}{2}|\nabla_{x}u|^{2}-(V-<\mu,V>)u+V<\mu,u>+\tilde{f}.$

on the support of $\bm{\mu}$.

######  Remark 6.

Both equations in the above system are strongly coupled, and the difficulty in
the second equation is twofold. The combination of the nonlinear term
$|\nabla_{x}u|^{2}$ and the non-local term $<\mu,u>$ prevents us from
appealing directly to standard results from PDEs or BSDEs to study the well-
posedness of the system.

The sense in which equation (103a) should be understood is clear when the term
$\nabla_{x}u$ is well defined as a function: it should be understood in the
sense of distributions. However, the situation with the second equation is not
as clear, in part because of the presence of non-local and non-linear terms.
While the notion of viscosity solution could be viewed as a natural option, we
shall refrain from using this interpretation for two independent reasons.
Firstly, the comparison constraints involved in the definition of a viscosity
solution would need to be modified to account for the non-local terms.
Secondly, viscosity solutions are not always differentiable, and our
derivation of equation (103b) was based on the substitution in the adjoint
equation, of the identification of the optimal control as the gradient of its
own adjoint function (i.e. co-state). In order to avoid this oxymoron created
by a seemingly circular argument, we choose to search for solutions by first
freezing the non-local terms, and then, work with _mild solutions_ in order to
avoid the possible lack of differentiability of the solution. The following
lemma highlights the fact that, absent the non-local nature of the equations,
viscosity and mild solutions are the same in our setting. We believe that such
a result is part of the folklore on the subject. See for example [4]. We state
it in the form we need for further references, and we give a proof for the
sake of completeness.

######  Lemma 11.

Let $G:\mathbb{R}^{d}\to\mathbb{R}$ be bounded and Lipschitz, and let
$F:[0,T]\times\mathbb{R}^{d}\times\mathbb{R}\ni(t,x,u)\mapsto
F(t,x,u)\in\mathbb{R}$ be bounded and Lipschitz in $(x,u)$ uniformly in
$t\in[0,T]$.

  * (i)

The semilinear PDE

(104) $\partial_{t}u+\frac{1}{2}\Delta u+F(t,x,u(t,x))=0,\qquad u(T,x)=G(x),$

has a unique viscosity solution.

  * (ii)

If for $(t,x)\in[0,T]\times\mathbb{R}^{d}$, we set $B^{t,x}_{r}=x+B_{r}-B_{t}$
for $t\leq r\leq T$ where $(B_{t})_{0\leq t\leq T}$ is some process of
Brownian motion, the equation

(105)
$v(t,x)=\mathbb{E}\Bigl{[}G(B^{t,x}_{T})+\int_{t}^{T}F\bigl{(}r,B^{t,x}_{r},v(r,B^{t,x}_{r})\bigr{)}dr\Bigr{]}\qquad(t,x)\in[0,T]\times\mathbb{R}^{d}$

has a unique bounded (jointly) continuous solution.

  * (iii)

$u=v$.

  * (iv)

Furthermore, this unique solution $u$ satisfies:

    * (iv)-1

$u$ is continuously differentiable and its gradient is uniformly bounded
whenever $F\in C^{0,1}_{b}([0,T]\times\mathbb{R}^{d}\times\mathbb{R})$, that
is when $F$ is continuously differentiable in $(x,u)$ with bounded
derivatives.

    * (iv)-2

$u$ is a classical solution of (104) in
$C^{1,2}_{b}([0,T]\times\mathbb{R}^{d})$ whenever $G$ is $3$-times
continuously differentiable with derivatives growing at most polynomially, and
for each fixed $t\in[0,T]$ $(x,u)\mapsto F(t,x,u)$ is $3$-times continuously
differentiable with bounded derivatives of orders $1$ and $2$ and polynomially
bounded derivatives of order $3$.

Instead of looking for the most general statement, we merely stated the
results under assumptions which will be satisfied in the situations in which
we need them, and for which we can use powerful already existing results. For
example, we could dispense with the boundedness assumptions for $G$ and $F$,
in which case the functions $u$ and $v$ in (i) and (ii) would be of at most
linear growth because of the global Lipschitz assumption on $F$ and $G$.

###### Proof.

(i) This is a particular case of [19, Theorem 4.3]. Indeed, for each
$(t,x)\in[0,T]\times\mathbb{R}^{d}$ we consider the BSDE

(106)
$Y^{t,x}_{s}=G(B^{t,x}_{T})+\int_{s}^{T}F\bigl{(}r,B^{t,x}_{r},Y^{t,x}_{r}\bigr{)}dr-\int_{s}^{T}Z^{t,x}_{r}dB_{r}$

where $B^{t,x}_{r}=x+B_{r}-B_{t}$ for $t\leq r\leq T$ is a process of Brownian
motion conditioned to be at $x$ at time $t$. Under our assumptions, this BSDE
has a unique solution for each fixed $(t,x)\in[0,T]\times\mathbb{R}^{d}$. Next
we define the function $u$ by setting $u(t,x)=Y^{t,x}_{t}$ (which is
deterministic), and [19, Theorem 4.3] says that $u$ is the unique viscosity
solution of (104).

(ii) Like in the proof of Proposition 3, in order to define a function $\Psi$
from $C_{b}([0,T]\times\mathbb{R}^{d})$ into itself we posit that $[\Psi
v](t,x)$ is given by the right hand side of (105), and exactly as in the proof
of Proposition 3, we check that $\Psi$ is a strict contraction for the norm
$\|\cdot\|_{\alpha}$ for $\alpha$ large enough.

(iii) By uniqueness of the solution of the BSDE (106), if $t\leq r\leq T$, we
have:

$Y^{t,x}_{r}=Y_{r}^{r,B_{r}^{t,x}}=u\bigl{(}r,B^{t,x}_{r}\bigr{)}$

by definition of the function $u$ in (i). Accordingly, the BSDE (106) can be
rewritten as

(107)
$Y^{t,x}_{s}=G(B^{t,x}_{T})+\int_{s}^{T}F\bigl{(}r,B^{t,x}_{r},u\bigl{(}r,B^{t,x}_{r}\bigr{)}\bigr{)}dr-\int_{s}^{T}Z^{t,x}_{r}dB_{r}$

and setting $s=t$ and taking expectations on both sides we get:

$u(t,x)=Y^{t,x}_{t}=\mathbb{E}\Bigl{[}G(B^{t,x}_{T})+\int_{t}^{T}F\bigl{(}r,B^{t,x}_{r},u\bigl{(}r,B^{t,x}_{r}\bigr{)}\bigr{)}dr\Bigr{]}$

showing that $u$ is a solution of (105), and hence that $u=v$ by uniqueness of
such a solution. Notice that the stochastic integral in (107) is indeed a
square integrable martingale so its expectation is $0$.

(iv)-1 is a consequence of [16, Theorem 3.1 p.1397] and its corollary [16,
Corollary 3.2], and (iv)-2 follows [19, Theorem 3.2]. ∎

######  Proposition 5.

In the case of separable cost functions, for each continuous flow
$\hat{\bm{\mu}}=(\hat{\mu}_{t})_{0\leq t\leq T}$ of probability measures on
$\mathbb{R}^{d}$, the PDE (103b) admits a solution in the sense of viscosity
which is continuously differentiable with uniformly bounded first derivatives.
Moreover, this solution is actually a classical solution when $\tilde{f}$ and
$g$ are three times differentiable with bounded derivatives.

###### Proof.

As suggested by the content of Remark 4, we limit the search for a solution to
bounded functions $u$. In fact, if we denote by $K$ the bound in that remark,
using $K_{T}=Ke^{2T}$, we look for $u$ in the class of functions satisfying
$\|u_{t}\|_{\infty}\leq K_{T}e^{-2t}$ for all $t\in[0,T]$.

_Step 1_. In fact, instead of looking directly for solutions $\tilde{u}$
satisfying $\|\tilde{u}_{t}\|_{\infty}\leq K_{T}e^{-2t}$, we look for the
exponentials of such functions. Indeed, if we set
$\tilde{v}_{t}=e^{-\tilde{u}_{t}}$ and define

$F^{\tilde{v}}(t,x)=(V(x)-<\hat{\mu}_{t},V>)\log\tilde{v}(t,x)-V(x)<\hat{\mu}_{t},\log\tilde{v}_{t}>+\tilde{f}(x),$

then since the function $\log\tilde{v}$ is bounded, the function
$F^{\tilde{v}}$ is also bounded, and we define $v=\Psi(\tilde{v})$ as the
solution of

(108) $0=\partial_{t}v+\frac{1}{2}\Delta v-F^{\tilde{v}}v,\qquad
v(T,\cdot)=G,$

which is given by

(109)
$v(t,x)=\mathbb{E}\Bigl{[}G(B^{t,x}_{T})\exp\bigl{[}-\int_{t}^{T}F^{\tilde{v}}(r,B^{t,x}_{r})dr\bigr{]}\Bigr{]}.$

Notice that

(110) $\|F^{\tilde{v}}_{t}\|_{\infty}\leq
2\|\log\tilde{v}_{t}\|_{\infty}+\|\tilde{f}\|_{\infty}\leq
2K_{T}e^{-2t}+\|\tilde{f}\|_{\infty},$

so if $u=-\log v$ we have

$\displaystyle\|u_{t}\|_{\infty}$
$\displaystyle=\sup_{x}\;\Bigl{|}\log\mathbb{E}\Bigl{[}G(B^{t,x}_{T})\exp\bigl{[}-\int_{t}^{T}F^{\tilde{v}}(r,B^{t,x}_{r})dr\bigr{]}\Bigr{]}\Bigr{|}$
$\displaystyle\leq\|g\|_{\infty}+2K_{T}\int_{t}^{T}e^{-2r}dr+(T-t)\|\tilde{f}\|_{\infty}$
(111) $\displaystyle\leq
K_{T}e^{-2t}\Bigl{(}\frac{\|g\|_{\infty}+(T-t)\|\tilde{f}\|_{\infty}}{K}+1-e^{-2(T-t)}\Bigr{)}$

Now increasing the value of the original $K$ if necessary, we can assume
without any loss of generality that

(112) $K>e^{2T}\bigl{(}\|g\|_{\infty}+e^{T}\|\tilde{f}\|_{\infty}\bigr{)}$

in which case:

$\frac{\|g\|_{\infty}+(T-t)\|\tilde{f}\|_{\infty}}{K}+1-e^{-2(T-t)}\leq 1.$

So, by (111), we have $\|u_{t}\|_{\infty}\leq K_{T}e^{-2t}$ for $0\leq t\leq
T$. So if $-K_{T}e^{-2t}\leq\log\tilde{v}_{t}\leq K_{T}e^{-2t}$, then
$v=\Psi(\tilde{v})$ satisfies $-K_{T}e^{-2t}\leq\log v_{t}\leq K_{T}e^{-2t}$
as well.

_Step 2._ Given the above preliminaries, we define the closed subset

$\mathcal{C}_{K}=\\{v\in
C_{b}([0,T]\times\mathbb{R}^{d};\mathbb{R}_{+});\;-K_{T}e^{-2t}\leq\log
v_{t}\leq K_{T}e^{-2t}\;\;0\leq t\leq T\\}$

of $C_{b}([0,T]\times\mathbb{R}^{d};\mathbb{R}_{+})$, which is left invariant
by the map $\Psi$ since we just proved that
$\Psi(\mathcal{C}_{K})\subseteq\mathcal{C}_{K}$. Next, we prove that $\Psi$
has a unique fixed point in $\mathcal{C}_{K}$, using a strict contraction
argument on $\mathcal{C}_{K}$ endowed with the norm
$\|v\|_{\alpha}=\sup_{t\in[0,T]}e^{\alpha t}\|v_{t}\|_{\infty}$. If
$\tilde{v}^{1}$ and $\tilde{v}^{2}$ are in $\mathcal{C}_{K}$, we have:

(113)
$\begin{split}\bigl{\|}\bigl{(}\Psi(\tilde{v}^{1})-\Psi(\tilde{v}^{2})\bigr{)}_{t}\bigr{\|}_{\infty}&=\sup_{x}\mathbb{E}\Bigl{[}\Bigl{|}G(B^{t,x}_{T})\Bigl{(}\exp\bigl{[}-\int_{t}^{T}F^{\tilde{v}^{1}}(r,B^{t,x}_{r})dr\bigr{]}-\exp\bigl{[}-\int_{t}^{T}F^{\tilde{v}^{2}}(r,B^{t,x}_{r})dr\bigr{]}\Bigr{)}\Bigr{|}\Bigr{]}\\\
&\leq e^{\|g\|_{\infty}}e^{T\sup_{0\leq t\leq
T}\|F_{t}^{\tilde{v}^{1}}\|_{\infty}\vee\|F_{t}^{\tilde{v}^{2}}\|_{\infty}}\mathbb{E}\Bigl{[}\int_{t}^{T}\bigl{|}F^{\tilde{v}^{1}}(r,B^{t,x}_{r})-F^{\tilde{v}^{2}}(r,B^{t,x}_{r})\bigr{|}\;dr\Bigr{]}\\\
&\leq
C^{\prime}\int_{t}^{T}\|\log\tilde{v}^{1}_{r}-\log\tilde{v}^{2}_{r}\|_{\infty}\;dr\\\
&\leq
C^{\prime}e^{K}\int_{t}^{T}\|\tilde{v}^{1}_{r}-\tilde{v}^{2}_{r}\|_{\infty}\;dr,\end{split}$

where we used the fact that $\tilde{v}^{1}$ and $\tilde{v}^{2}$ are in
$\mathcal{C}_{K}$ and
$C^{\prime}=2\exp[\|g\|_{\infty}+T(2K_{T}+\|\tilde{f}\|_{\infty})]$. Notice
that we used the fact that $\tilde{v}^{1}$ and $\tilde{v}^{2}$ are uniformly
bounded from abobe and from below away from $0$, so we only need to rely on
the local Lipschitz properties of the exponential and logarithm functions.
Moreover, we have

(114) $e^{\alpha
t}\bigl{\|}\bigl{(}\Psi(\tilde{v}^{1})-\Psi(\tilde{v}^{2})\bigr{)}_{t}\bigr{\|}_{\infty}\leq
C^{\prime}e^{K}e^{\alpha t}\int_{t}^{T}e^{-\alpha r}e^{\alpha
r}\|\tilde{v}^{1}_{r}-\tilde{v}^{2}_{r}\|_{\infty}\;dr\leq
C^{\prime}e^{K}\frac{1-e^{-\alpha(T-t)}}{\alpha}\sup_{0\leq r\leq T}e^{\alpha
r}\|\tilde{v}^{1}_{r}-\tilde{v}^{2}_{r}\|_{\infty}$

so that

$\bigl{\|}\Psi(\tilde{v}^{1})-\Psi(\tilde{v}^{2})\bigr{\|}_{\alpha}=\sup_{0\leq
t\leq T}e^{\alpha
t}\bigl{\|}\bigl{(}\Psi(\tilde{v}^{1})-\Psi(\tilde{v}^{2})\bigr{)}_{t}\bigr{\|}_{\infty}\leq\frac{C^{\prime}e^{K}}{\alpha}\|\tilde{v}^{1}-\tilde{v}^{2}\|_{\alpha}$

which shows that $\Psi$ is a strict contraction if $\alpha>C^{\prime}e^{K}$.
If $v$ is the unique fixed point of this strict contraction, then $v$
satisfies

$0=\partial_{t}v+\frac{1}{2}\Delta v-F^{v}v,\qquad v(T,\cdot)=G,$

in other words:

(115) $0=\partial_{t}v+\frac{1}{2}\Delta
v-\bigl{[}(V(x)-<\hat{\mu}_{t},V>)\log v(t,x)-V(x)<\hat{\mu}_{t},\log
v_{t}>+\tilde{f}(x)\bigr{]}v,\qquad v(T,\cdot)=G.$

_Step 3._ We now return to the function $u$ and conclude the proof of the
proposition. The function

$\displaystyle[0,T]\times\mathbb{R}^{d}\times[e^{-K_{T}},e^{K_{T}}]\ni(t,x,u)\mapsto
F(t,x,u)$ $\displaystyle=(V(x)-<\hat{\mu}_{t},V>)u\log u$
$\displaystyle\qquad+[-V(x)<\hat{\mu}_{t},\log v_{t}>+\tilde{f}(x)]u$

satisfies all the assumptions of Lemma 11 since it is bounded and Lipschitz
continuous in $(x,u)$ uniformly in $t\in[0,T]$. Since $G$ also satisfies the
required assumption, and since equation (115) coincides with (104), we can
conclude that the function $v$ we just constructed as a fixed point of $\Psi$
is continuously differentiable and its gradient is uniformly bounded.
Moreover, $v$ is a classical solution whenever $g$ is $3$-times continuously
differentiable with bounded derivatives.

Since the logarithm function is monotone, the function $u=-\log v$ is a
viscosity solution of

(116) $0=\partial_{t}u_{t}+\frac{1}{2}\Delta u_{t}-\frac{1}{2}|\nabla
u_{t}|^{2}+(V-<\hat{\mu}_{t},V>)u_{t}+V<\hat{\mu}_{t},u_{t}>+\tilde{f},\qquad
u_{T}=g,$

which is equation (103b) with $\bm{\mu}=\hat{\bm{\mu}}$. Also, since $v$ is
bounded away from $0$, $u$ is continuously differentiable and its gradient is
uniformly bounded. Moreover, $u$ is a classical solution of (116) whenever $g$
is $3$-times continuously differentiable with bounded derivatives. ∎

######  Remark 7.

While the derivation of the necessity of the existence of a solution for the
PDE system (103) at optimality was done under the assumption that the optimal
control was bounded, the above proposition actually implies that the optimal
control obtained from the gradient of the solution is necessarily bounded.

##  4\. Analysis of the Open Loop Conditional Control Problem

In this section, we consider the case of general open loop controls. Our first
goal is to show that the search for optimal controls can be restricted to a
subclass of controls of a specific feedback form, namely deterministic
functions of time, the controlled state $X_{t}$, and a specific function of
the history of its path prior to time $t$.

###  4.1. Precise Formulation of the Open Loop Conditional Control Problem

We now define in detail the open loop version of the control problem discussed
in the introduction. For each $t\in[0,T]$, we denote by $\Theta_{t}$ the set
of $\theta=(\Omega,\mathcal{F},(\mathcal{F}_{s})_{t\leq s\leq
T},\mathbb{P},\mathbf{W}=(W_{s})_{t\leq s\leq T})$ where
$(\Omega,\mathcal{F},\mathbb{P})$ is a probability space supporting a process
$\mathbf{W}$ which satisfies $W_{t}=0$ and is a Brownian motion for the
filtration $(\mathcal{F}_{s})_{t\leq s\leq T}$. When for some $t$,
$\theta\in\Theta_{t}$ is a generic element, we implicitly use the notation
$\mathbb{E}=\mathbb{E}^{\mathbb{P}}$ for the expectation under $\mathbb{P}$,
and by _almost surely_ , we mean $\mathbb{P}$-almost surely.

For each $\theta\in\Theta_{t}$, we define $\mathbb{A}_{t}^{\theta}$ as the set
of $A$-valued progressively measurable processes
$\bm{\alpha}=(\alpha_{s})_{t\leq s\leq T}$ satisfying the admissibility
conditions defined earlier, namely

$\mathbb{E}^{\mathbb{P}}\int_{t}^{T}|\alpha_{s}|^{2}\,ds\;<\infty.$

Now for each stochastic basis $\theta\in\Theta_{t}$ and each admissible
control $\bm{\alpha}\in\mathbb{A}_{t}^{\theta}$, the controlled state process
$\mathbf{X}^{\alpha}=(X^{\alpha}_{s})_{t\leq s\leq T}$ is given by:

(117) $dX^{\alpha}_{s}=\alpha_{s}ds+dW_{s},\qquad t\leq s\leq T.$

We shall specify the initial condition $X_{t}$ when needed, and we shall skip
the superscript $\alpha$ whenever convenient as long as no confusion is
possible. Also, we shall systematically drop the subscript $t$ in
$\mathbb{A}_{t}^{\theta}$ whenever $t=0$.

The goal of the control problem is to minimize the cost $J^{V}(\bm{\alpha})$
defined in (6). We denote by $U^{(2)}(t,\mu)$ the value function of the
problem. It is defined for $t\in[0,T]$ and $x\in\mathbb{R}^{d}$ as follows:

(118)
$U^{(2)}(t,\mu)=\inf_{\theta\in\Theta_{t},\;\bm{\alpha}\in\mathbb{A}_{t}^{\theta},\;X_{t}\sim\mu}\Bigl{(}\int_{t}^{T}\frac{\mathbb{E}\Bigl{[}f(X_{s},\alpha_{s})e^{-\int_{t}^{s}V(X_{u})du}\Bigr{]}}{\mathbb{E}\Bigl{[}e^{-\int_{t}^{s}V(X_{u})du}\Bigr{]}}ds+\frac{\mathbb{E}\Bigl{[}g(X_{T})e^{-\int_{t}^{T}V(X_{s})ds}\Bigr{]}}{\mathbb{E}\Bigl{[}e^{-\int_{t}^{T}V(X_{s})ds}\Bigr{]}}\Bigr{)}.$

The initial condition $X_{t}\sim\mu$ stipulating that the distribution of
$X_{t}$ is $\mu$, reduces to the classical case of a deterministic initial