# Stochastic Linear-Quadratic Optimal Control Problems with Random
Coefficients and Markovian Regime Switching System

Jiaqiang Wen , Xun Li , Jie Xiong , Xin Zhang Department of Mathematics and
SUSTech International Center for Mathematics, Southern University of Science
and Technology, Shenzhen, Guangdong, 518055, China (Email:
wenjq@sustech.edu.cn). This author is supported by National Natural Science
Foundation of China (grant No. 12101291) and Guangdong Basic and Applied Basic
Research Foundation (grant No. 2022A1515012017).Department of Applied
Mathematics, The Hong Kong Polytechnic University, Hong Kong, China (Email:
li.xun@polyu.edu.hk). This author is supported by RGC of Hong Kong (grant Nos.
15209614, 15213218 and 15215319) and partially from CAS AMSS-PolyU Joint
Laboratory of Applied Mathematics.Department of Mathematics and SUSTech
International center for Mathematics, Southern University of Science and
Technology, Shenzhen, Guangdong, 518055, China (Email: xiongj@sustech.edu.cn).
This author is supported by SUSTech Start up fund Y01286120 and National
Natural Science Foundation of China (grant No. 61873325).School of
Mathematics, Southeast University, Nanjing, Jiangsu, 211189, China (Email:
x.zhang.seu@gmail.com). This author is supported by National Natural Science
Foundation of China (grant No. 12171086) and Fundamental Research Funds for
the Central Universities (grant No. 2242021R41082).

Abstract. This paper thoroughly investigates stochastic linear-quadratic
optimal control problems with the Markovian regime switching system, where the
coefficients of the state equation and the weighting matrices of the cost
functional are random. We prove the solvability of stochastic Riccati equation
under the uniform convexity condition, and obtain the closed-loop
representation of the open-loop optimal control using the unique solvability
of the corresponding stochastic Riccati equation. Moreover, by applying Itô’s
formula with jumps, we get a representation of the cost functional on a
Hilbert space, characterized as the adapted solutions of some forward-backward
stochastic differential equations. We show that the necessary condition of the
open-loop optimal control is the convexity of the cost functional, and the
sufficient condition of the open-loop optimal control is the uniform convexity
of the cost functional. In addition, we study the properties of the stochastic
value flow of the stochastic linear-quadratic optimal control problem.
Finally, as an application, we present a continuous-time mean-variance
portfolio selection problem and prove its unique solvability.

Key words: Stochastic linear-quadratic optimal control, Markovian regime
switching, random coefficient, stochastic Riccati equation, mean-variance
portfolio selection.

AMS subject classifications. 49N10, 93E20.

##  1 Introduction

Linear-quadratic (LQ, for short) optimal control problem plays a fundamental
role in control theory, which appeared with the birth of stochastic analysis
and developed rapidly in recent decades due to its wide range of applications.
The study of stochastic linear-quadratic (SLQ, for short) optimal control
problems can be traced back to the work of Kushner [12] and Wonham [25]. In
the classical setting, the SLQ optimal control problem can be solved elegantly
via the Riccati equation under some mild conditions on the weighting
coefficients (see Yong–Zhou [26, Chapter 6]). Chen–Li–Zhou [3] investigated
SLQ optimal control problems with indefinite weighting control matrix and
their applications in solving continuous-time mean-variance portfolio
selection problems in 1990s. From then on, there has been increasing interest
in the so-called indefinite SLQ optimal control problems as well as addressing
their applications (see Chen–Yong [4], Ait Rami–Moore–Zhou [1], and
Li–Zhou–Lim [14]).

To tackle SLQ optimal control problems with random coefficients, Bismut [2]
initially derived the existence and uniqueness of the solution of the
stochastic Riccati equation (SRE, for short) using techniques of functional
analysis. Peng [18] posted it as an open problem in the general setting in his
collection of open problems for backward stochastic differential equations
(BSDEs, for short). Kohlmann–Tang [11], Tang [22] and Sun–Xiong–Yong [20]
established the existence and uniqueness of solutions to the related SRE under
different conditions using their methods, which partially addressed the open
problem introduced by Peng [18]. In detail, Kohlmann–Tang [11] studied the
multidimensional backward stochastic Riccati equations and gave an application
to the stochastic optimal control problem. Tang [22] studied general SLQ
optimal control problems with random coefficients and proved the existence and
uniqueness of related backward stochastic Riccati equations. Sun–Xiong–Yong
[20] studied the SLQ optimal control problems with random coefficients, proved
the solvability of the corresponding SRE, and obtained the closed-loop
representation of the open-loop optimal control. Li–Wu–Yu [13] analyzed a
special type of indefinite SLQ problem with random coefficients. For more
details about the efforts devoted to the stochastic Riccati equation and its
connection with SLQ optimal control problems, we refer the interested readers
to Kohlmann–Tang [10], Tang [23], etc.

Recently, there has been a dramatically increasing interest in the SLQ optimal
control problems with random jumps, such as Poisson jumps or the regime
switching jumps, which are of practical importance in various fields such as
economics, financial management, science, and engineering. In the past few
years, researchers have focused on models of financial markets whose key
parameters are described by Markov processes, such as stock returns, interest
rates, and volatility. In particular, one could face two market regimes in
financial markets, one of which stands for a bull market with price rises,
while the other for a bear market with price drops. We call such a formulation
the regime switching model, where the market parameters depend on market modes
that switch among a finite number of regimes. More recently, applications of
SLQ optimal control problems with regime switching models or Poisson jumps
have been extensively developed. For instance, Ji–Chizeck [9] formulated a
class of continuous-time LQ optimal control problems with Markovian jumps.
Zhou–Yin [29] studied a mean-variance portfolio selection with regime
switching. Liu–Yin–Zhou [16] studied the near-optimal controls of regime
switching LQ control problems. Hu–Oksendal [7] discussed SLQ optimal control
problems with Poisson jumps and partial information using the technique of
completing squares. Song–Tang–Wu [19] established the maximum principle for
progressive stochastic optimal control problems with random jumps.
Zhang–Li–Xiong [28] investigated open-loop and closed-loop solvabilities for
SLQ optimal control problems with a Markovian regime switching system.
Hu–Shi–Xu [8] applied the constrained SLQ control with regime switching to a
portfolio problem. For some other important works, we refer the readers to [5,
15, 24, 27], and the references therein.

In a real market, besides the Markov chain, it is more reasonable allowing the
market parameters to depend on the Brownian motion, due to the fact that the
interest rates, stock rates, and volatilities are affected by the
uncertainties caused by the Brownian motion. However, up to now, few results
have been obtained on this topic. In this paper, inspired by the continuous-
time mean-variance portfolio selection problems, we are interested in studying
this topic, i.e., the SLQ optimal control problems under the Markovian regime
switching system with random coefficients. Further, by developing some ideas
of Sun–Xiong–Yong [20] and Li–Zhou–Lim [14], we establish the existence of an
open-loop optimal control and prove the unique solvability of the associated
SRE. Also, we apply the theoretical results shown in this paper to treat a
continuous-time mean-variance portfolio selection problem. Next, we present
our main results and difficulties in detail.

  1. (i)

We first introduce the associated stochastic Riccati equation of Problem
(M-SLQ) (see (2.2)), and prove that the optimal state process of Problem
(M-SLQ) is invertible under the uniform convexity condition (3.3). Then, we
show that a bounded process $\hat{P}(\cdot,\alpha(\cdot))$, together with two
square-integrable processes $\hat{\Lambda}(\cdot)$ and $\hat{\zeta}(\cdot)$,
uniquely satisfies the associated stochastic Riccati equation (3.1), and thus
establishing the unique solvability of the associated SRE (see Theorem 3.1).
Moreover, we derive a closed-loop representation for the open-loop optimal
control of Problem (M-SLQ) using the unique solvability of the SRE (see
Theorem 3.2).

  2. (ii)

In order to prove the above theoretical results, we prove some auxiliary
results. We establish the equivalence between Problem (M-SLQ) and Problem
${(\hbox{M-SLQ})}_{0}$, i.e., a control $u^{*}(\cdot)\in{\cal U}[t,T]$ is
optimal for Problem (M-SLQ)0 if and only if it is optimal for Problem
${(\hbox{M-SLQ})}$ (see 3.6), and based on the equivalence, we analyze the
time-consistency of the optimal control (see 3.8). Then, we obtain a quadratic
representation of the stochastic value flow in terms of a bounded, left
continuous, and $\mathbb{S}^{n}$-valued process (see Theorem 3.11 and Theorem
3.12). In addition, by the technique of Itô’s formula with jumps, we represent
the cost functional of Problem (M-SLQ) as a bilinear form, in terms of the
adapted solutions of some forward-backward stochastic differential equations
(FBSDEs, for short) in a suitable Hilbert space (see 5.4).

  3. (iii)

As a financial application, we present an example of the continuous-time mean-
variance portfolio selection problem and prove the unique solvability of the
related mean-variance problem. Also, we derive the representation of the
unique optimal investment strategy (see Theorem 6.1), which further develops
the work of Li–Zhou–Lim [14] to the Markovian regime switching system with
random coefficients.

Compared with Sun–Xiong–Yong [20], the difficulties of this paper mainly come
from the solvability of the associated SRE, due to the presence of the
Markovian regime switching jumps.

  1. (iv)

Firstly, due to the presence of random coefficients, the Riccati equation
associated with Problem (M-SLQ) becomes a nonlinear BSDE, usually referred to
as the backward stochastic Riccati equation. Furthermore, the backward
stochastic Riccati equation is driven by both the Brownian motion and the
martingales $(\widetilde{N}_{kl}(\cdot))_{k,l\in{\cal S}}$ generated by the
Markov chain thanks to the occurrence of Markovian regime switching jumps in
the model. Thus, the solvability of this BSDE is more complicated than that of
the model with deterministic coefficients and without the Markovian regime
switching jumps.

  2. (v)

Secondly, let $(X_{j}(s),Y_{j}(s),Z_{j}(s),\Gamma^{j}(s))$ be the solution of
the FBSDE composed by the open-loop optimal state process and adjoint equation
corresponding to initial state $(t,e_{j},\vartheta)$. It is worth to mention
that the construction of the fourth component $\boldsymbol{\Gamma}(s)$ for the
solution of matrix-valued FBSDE (3.11) is not directly extending the
dimensional by combing $\Gamma^{j}(s)$ as
$(\Gamma^{1}(s),\cdots,\Gamma^{n}(s))$. Furthermore, the $\Gamma$-term of the
solution to equation (3.6) associated with the initial state
$(t,\xi,\vartheta)$ is represented as $\boldsymbol{\Gamma}(s)\circ\xi$, which
is different from that of $(Y(s),Z(s))=({\bf Y}(s)\xi,{\bf Z}(s)\xi)$. Please
see more details in 3.10.

  3. (vi)

Thirdly, the existence of process $P(t)$ appeared in the equation (42) of
Sun–Xiong–Yong [20] for the representation of the value function follows
directly by its definition $\mathbb{E}[{\bf M}(T)+\int_{t}^{T}{\bf
N}(s)ds|\mathscr{F}_{t}]$. In our case, we have a similar representation for
the value function but with ${\bf M}(T)$ and ${\bf N}(s)$ replaced by
functions depend on the Markov chain $\alpha(\cdot)$ and $\mathscr{F}_{t}$
replaced by the $\sigma$-field generated by the Brownian motion and the Markov
chain $\alpha(\cdot)$. To solve our problem, we need to prove that there
exists a process $P:[0,T]\times{\cal S}\times\Omega\rightarrow\mathbb{S}^{n}$,
which is $\mathbb{F}$-adapted, such that

$P(t,\alpha(t))=\mathbb{E}\Big{[}\mathbf{M}(T,\alpha(T))+\int_{t}^{T}\boldsymbol{N}(s,\alpha(s))ds\Big{|}\mathscr{F}_{t}\Big{]}.$

This cannot be obtained directly from the definition of conditional
expectation and we give proof for this (see the proof of Theorem 3.11). In our
model, although the evolution of ${\bf X}(s)$ and ${\bf Y}(s)$ depends on the
Markov chain $\alpha(\cdot)$, ${\bf Y}(s){\bf X}(s)^{-1}$ may not equal
$P(s,\alpha(s))$. We further prove the equality of $P(s,\alpha(s))={\bf
Y}(s){\bf X}(s)^{-1}$ (see the proof of Theorem 3.1), which plays an important
role in solving our problem.

The paper is organized as follows. In Section 2, we present some preliminaries
and formulate Problem (M-SLQ) with random coefficients and regime switching.
In Section 3, we state our main results, i.e., the solvability of the
corresponding stochastic Riccati equation, the closed-loop representation of
the open-loop optimal control, and some auxiliary results. In Section 4, we
prove the invertibility of the optimal state process, the solvability of the
corresponding SRE, and the closed-loop representation of the open-loop optimal
control. In Section 5, we strictly prove some auxiliary results in detail. In
Section 6, we present an example of the continuous-time mean-variance
portfolio selection problem under the Markovian regime switching system with
random coefficients. In Section 7, we conclude the results.

##  2 Preliminaries

Let $(\Omega,\mathscr{F},\mathbb{F},\mathbb{P})$ be a complete filtered
probability space on which a standard one-dimensional Brownian motion
$\\{W(t)\\}_{t\geqslant 0}$ and a continuous-time and finite-state Markov
chain $\\{\alpha(t)\\}_{t\geqslant 0}$ are defined, where the processes
$W(\cdot)$ and $\alpha(\cdot)$ are independent and
$\mathbb{F}=\\{\mathscr{F}_{t}\\}_{t\geqslant 0}$ is the natural filtration of
them with $\mathscr{F}_{0}$ containing all $\mathbb{P}$-null sets of
$\mathscr{F}$. Let $\mathbb{F}^{W}=\\{\mathscr{F}^{W}_{t}\\}_{t\geqslant 0}$
be the filtration generated by $W(\cdot)$ and
$\mathbb{F}^{\alpha}=\\{\mathscr{F}^{\alpha}_{t}\\}_{t\geqslant 0}$ be the
filtration generated by $\alpha(\cdot)$. We identify the state space of the
Markov chain $\alpha(\cdot)$ with a finite set ${\cal
S}\triangleq\\{1,2\dots,D\\}$, where $D\in\mathbb{N}$. Furthermore, the
generator of the Markov chain $\alpha(\cdot)$ under $\mathbb{P}$ is denoted by
$\lambda(t)\triangleq[\lambda_{kl}(t)]_{k,l\in{\cal S}}$, where
$\lambda_{kl}(t)$ is the constant transition intensity of the Markov chain
from state $k$ to state $l$ at time $t$. For each fixed $k,l\in{\cal S}$, we
let $N_{kl}(t)$ be the number of jumps from state $k$ into state $l$ up to
time $t$ and set
$\tilde{\lambda}_{kl}(t)\triangleq\int^{t}_{0}\lambda_{kl}(s)I_{\\{\alpha(s-)=k\\}}ds$.
Let $N(t)\triangleq(N_{kl}(t))_{k,l\in\cal{S}}$ and
$\widetilde{N}(t)\triangleq(\widetilde{N}_{kl}(t))_{k,l\in\cal{S}}$, where
$\widetilde{N}_{kk}(t)\equiv 0$ and
$\widetilde{N}_{kl}(t)=N_{kl}(t)-\tilde{\lambda}_{kl}(t)$ when $k\neq l$.

Let $T>0$ be a fixed terminal time. The trace of a square matrix $M$ is
denoted by $\hbox{\rm tr$\,$}[M]$, the set of all $n\times n$ symmetric
matrices is denoted by $\mathbb{S}^{n}$, the set of all $D\times D$ matrices
$M\triangleq(M_{kl})$ with $M_{kl}\in\mathbb{S}^{n}$ is denoted by
$\mathbb{M}_{D}(\mathbb{S}^{n})$, and the set of all $D\times D$ matrices
$M\triangleq(M_{kl})$ with $M_{kl}\in\mathbb{R}^{n\times m}$ is denoted by
$\mathbb{M}_{D}(\mathbb{R}^{n\times m})$. For $\mathbb{H}=\mathbb{R}^{n}$,
$\mathbb{R}^{n\times m}$ or $\mathbb{S}^{n}$, denote by
$L^{2}_{\mathscr{F}}(\Omega;\mathbb{H})$ (resp.,
$L^{\infty}_{\mathscr{F}}(\Omega;\mathbb{H})$) the set of all
$\mathscr{F}$-measurable, $\mathbb{H}$-valued, and square integrable (resp.,
bounded) random variables. Denote by $L_{\mathbb{F}}^{2}(t,T;\mathbb{H})$
(resp., $L_{\mathbb{F}}^{\infty}(t,T;\mathbb{H})$) the set of all
$\mathbb{H}$-valued, $\mathbb{F}$-progressive measurable stochastic processes
$\phi(s)$ with $\mathbb{E}\int^{T}_{t}|\phi(s)|^{2}ds<\infty$ (resp.,
$\mathop{\rm esssup}_{s\in[t,T]}|\phi(s)|<\infty$), and denote by
$L_{\mathbb{F}}^{2}(\Omega;C([t,T];\mathbb{H}))$ the set of all
$\mathbb{H}$-valued, $\mathbb{F}$-adapted and continuous processes $\phi(s)$
with $\mathbb{E}[\sup_{s\in[t,T]}|\phi(s)|^{2}]<\infty$. Moreover, for
$\mathbb{G}=\mathbb{M}_{D}(\mathbb{R}^{n\times m})$ or
$\mathbb{M}_{D}(\mathbb{S}^{n})$, we denote by
$L_{\mathbb{F}}^{2}(t,T;\mathbb{G})$ the set of all $\mathbb{G}$-valued
$\mathbb{F}$-progressively measurable process
$\phi(s)\triangleq(\phi_{kl}(s))\in\mathbb{G}$ with
$\mathbb{E}\int^{T}_{t}\sum_{k,l=1}^{D}|\phi_{kl}(s)|^{2}\lambda_{kl}(s)I_{\\{\alpha(s-)=k\\}}ds<\infty.$
For $\eta(s)\triangleq(\eta_{kl}(s))\in{\cal M}_{D}(\mathbb{R}^{n\times n})$,
we further define

$\eta(s)\bullet
d\widetilde{N}(s)\triangleq\sum_{k,l=1}^{D}\eta_{kl}(s)d\widetilde{N}_{kl}(s).$

We now introduce the following state equation, which is the controlled
Markovian regime switching linear stochastic differential equation (SDE, for
short) over a finite time horizon $[t,T]$:

$\left\\{\begin{aligned}
\displaystyle&dX(s)=\big{[}A(s,\alpha(s))X(s)+B(s,\alpha(s))u(s)\big{]}ds\\\
\vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&\qquad\qquad+\big{[}C(s,\alpha(s))X(s)+D(s,\alpha(s))u(s)\big{]}dW(s),\quad
s\in[t,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&\displaystyle
X(t)=\xi,\quad\alpha(t)=\vartheta,\end{aligned}\right.$ (2.1)

where $A(t,\omega,i)$, $B(t,\omega,i)$, $C(t,\omega,i)$ and $D(t,\omega,i)$
are given $\mathscr{F}^{W}_{t}$-measurable processes for each
$i\in\mathcal{S}$. We call $(t,\xi,\vartheta)$ an initial triple, which comes
from the following set:

$\mathcal{D}=\big{\\{}(t,\xi,\vartheta)\ |\ t\in[0,T],\ \xi\in
L^{2}_{\mathscr{F}_{t}}(\Omega;\mathbb{R}^{n}),\ \vartheta\in
L_{\mathscr{F}_{t}^{\alpha}}^{2}(\Omega;\mathcal{S})\big{\\}}.$

In the state equation (2.1), the process $u(\cdot)$ comes from the control
space ${\cal U}[t,T]\triangleq L^{2}_{\mathbb{F}}(t,T;\mathbb{R}^{m})$, called
the control process, and the solution $X(\cdot)$ of (2.1) is called the state
process with $(t,\xi,\vartheta)$ and $u(\cdot)$. Let us state our SLQ optimal
control problem.

Problem $(\hbox{M-SLQ})$. For any given initial triple
$(t,\xi,\vartheta)\in{\cal D}$, find a control
$u^{*}(\cdot)\in\mathcal{U}[t,T]$, such that

$\displaystyle{J}(t,\xi,\vartheta;u^{*}(\cdot))=\mathop{\rm
essinf}_{u(\cdot)\in{\cal
U}[t,T]}{J}(t,\xi,\vartheta;u(\cdot))\triangleq{V}(t,\xi,\vartheta),$ (2.2)

where the cost functional is given as the following quadratic form

$\begin{array}[]{ll}\displaystyle{J}(t,\xi,\vartheta;u(\cdot))\triangleq\mathbb{E}_{t}\left[\big{\langle}G(\alpha(T))X(T),X(T)\big{\rangle}+\int_{t}^{T}\left\langle\begin{pmatrix}Q(s,\alpha(s))&S(s,\alpha(s))^{\top}\\\
S(s,\alpha(s))&R(s,\alpha(s))\end{pmatrix}\begin{pmatrix}X(s)\\\
u(s)\end{pmatrix},\begin{pmatrix}X(s)\\\ u(s)\end{pmatrix}\right\rangle
ds\right].\end{array}$ (2.3)

Note that $\mathbb{E}_{t}[\cdot]\triangleq\mathbb{E}[\cdot|\mathscr{F}_{t}]$
represents the conditional expectation with respect to (w.r.t., for short)
$\mathscr{F}_{t}$. For the initial triple $(t,\xi,\vartheta)$, we call the
control process $u^{*}(\cdot)$ an open-loop optimal control of Problem (M-SLQ)
if it satisfies (2.2), call the corresponding state process
$X^{*}(\cdot)\equiv X\left(\cdot;t,\xi,\vartheta,u^{*}(\cdot)\right)$ an open-
loop optimal state process, and call the state-control pair
$\left(X^{*}(\cdot),u^{*}(\cdot)\right)$ an open-loop optimal pair. We call
$(t,\xi,\vartheta)\mapsto{V}(t,\xi,\vartheta)$ the stochastic value flow of
Problem ${(\hbox{M-SLQ})}$, due to that the space
$L_{\mathscr{F}_{t}}^{2}\left(\Omega;\mathbb{R}^{n}\right)$ becomes larger
when $t$ increases.

######  Remark 2.1.

In the cost functional (2.3), when the conditional expectation
$\mathbb{E}_{t}[\cdot]$ degenerates to the expectation $\mathbb{E}$, we denote
the related problem, cost functional, and value function by Problem (M-SLQ)0,
$J_{0}(t,\xi,\vartheta;u(\cdot))$, and $V_{0}(t,\xi,\vartheta)$, respectively.
Clearly, $V_{0}(t,\xi,\vartheta)=\mathbb{E}[V(t,\xi,\vartheta)]$.

######  Definition 2.2.

Problem (M-SLQ) is said to be (uniquely) open-loop solvable for the initial
triple $(t,\xi,\vartheta)\in{\cal D}$ if there exists a (unique)
$u^{*}(\cdot)=u^{*}(\cdot\ ;t,\xi,\vartheta)\in{\cal U}[t,T]$ (depending on
$(t,\xi,\vartheta)$) such that

$J(t,\xi,\vartheta;u^{*}(\cdot))\leqslant
J(t,\xi,\vartheta;u(\cdot)),\quad\hbox{a.s.},\ \forall u(\cdot)\in{\cal
U}[t,T],$

and said to be (uniquely) open-loop solvable if it is (uniquely) open-loop
solvable for any initial triple $(t,\xi,\vartheta)\in{\cal D}$.

For the coefficients in the state equation (2.1) and the weighting matrices in
the cost functional (2.3), we post the following assumption:

  * (H)

For any choice of $i\in{\cal S}$, $A(t,\omega,i)$, $C(t,\omega,i)\in
L_{\mathbb{F}^{W}}^{\infty}(0,T;\mathbb{R}^{n\times n})$, $B(t,\omega,i)$,
$D(t,\omega,i)\in L_{\mathbb{F}^{W}}^{\infty}(0,T;\mathbb{R}^{n\times m})$,
$G(\omega,i)\in L^{\infty}_{\mathscr{F}^{W}_{T}}(\Omega;\mathbb{S}^{n})$,
$Q(t,\omega,i)\in L^{\infty}_{\mathbb{F}^{W}}(0,T;\mathbb{S}^{n})$,
$S(t,\omega,i)\in L^{\infty}_{\mathbb{F}^{W}}(0,T;\mathbb{R}^{m\times n})$,
and $R(t,\omega,i)\in L^{\infty}_{\mathbb{F}^{W}}(0,T;\mathbb{S}^{m})$.

Under the condition (H), for any initial triple
$(t,\xi,\vartheta)\in\mathcal{D}$ and any control
$u(\cdot)\in\mathcal{U}[t,T]$, the classical theory of SDEs (see Lemma 2.1 of
Wen–Li–Xiong [24]) implies that the equation (2.1) has a unique solution
$X(\cdot)\equiv X(\cdot;t,\xi,\vartheta,u(\cdot))$, which is square-integrable
and whose path is continuous. Moreover, to simplify our further analysis, we
finally introduce the following BSDE:

$\left\\{\begin{aligned} \displaystyle
dM(s)=&-\big{[}M(s)A(s,\alpha(s))+A(s,\alpha(s))^{\top}M(s)+C(s,\alpha(s))^{\top}M(s)C(s,\alpha(s))+\Phi(s)C(s,\alpha(s))\\\
\vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&+C(s,\alpha(s))^{\top}\Phi(s)+Q(s,\alpha(s))\big{]}ds+\Phi(s)dW(s)+\eta(s)\bullet
d\widetilde{N}(s),\quad s\in[t,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle M(T)=&\
G(\alpha(T)),\quad\alpha(t)=\vartheta.\end{aligned}\right.$ (2.4)

The solution of the above BSDE is denoted by the triple
$(M(\cdot),\Phi(\cdot),\eta(\cdot))$, where
$\eta(\cdot)\triangleq(\eta_{kl}(\cdot))_{k,l\in\mathcal{S}}\in{\cal
M}_{D}(\mathbb{R}^{n\times n})$. Note that the terminal value $G(\cdot)$ is
bounded, so the classical theory of BSDEs combining Itô’s formula with jumps
deduces the following result, i.e., $M(\cdot)$ is bounded too.

######  Proposition 2.3.

Under the condition (H), the process $M(\cdot)$ is bounded, where
$(M(\cdot),\Phi(\cdot),\eta(\cdot))$ is the adapted solution of BSDE (2.4).

##  3 The main results

In this section, we state our main results, such as the solvability of the
corresponding SRE, the closed-loop representation of the open-loop optimal
control, and some auxiliary results.

###  3.1 Solvability of stochastic Riccati equation

As shown in Sun–Xiong–Yong [20], the solvability of the corresponding SRE
deduces the closed-loop representation of the open-loop optimal control, which
is important to deal with Problem (M-SLQ). Therefore, the SRE plays a crucial
role in studying the SLQ problem, and our core goal is to establish the
solvability of the corresponding SRE. For this, we introduce the following
SRE:

$\left\\{\begin{aligned} \displaystyle
d\hat{P}(s,\alpha(s))&=-\big{[}\hat{Q}(s,\alpha(s))+\hat{S}(s,\alpha(s))^{\top}\Theta(s,\alpha(s))\big{]}ds\\\
\vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&\quad+\hat{\Lambda}(s)dW(s)+\hat{\zeta}(s)\bullet
d\widetilde{N}(s),\quad s\in[0,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle\hat{P}(T,\alpha(T))&=G(\alpha(T)),\quad\alpha(0)=i_{0},\end{aligned}\right.$
(3.1)

where $i_{0}\in{\cal S}$ is the initial state of $\alpha(\cdot)$, and for any
$(s,\omega,i)\in[0,T]\times\Omega\times{\cal S}$,

$\displaystyle\hat{Q}(s,i)$
$\displaystyle\triangleq\hat{P}(s,i)A(s,i)+A(s,i)^{\top}\hat{P}(s,i)+C(s,i)^{\top}\hat{P}(s,i)C(s,i)$
(3.2) $\displaystyle\quad\
+\hat{\Lambda}(s)C(s,i)+C(s,i)^{\top}\hat{\Lambda}(s)+Q(s,i),$
$\displaystyle\hat{S}(s,i)$ $\displaystyle\triangleq
B(s,i)^{\top}\hat{P}(s,i)+D(s,i)^{\top}\hat{P}(s,i)C(s,i)+D(s,i)^{\top}\hat{\Lambda}(s)+S(s,i),$
$\displaystyle\hat{R}(s,i)$ $\displaystyle\triangleq
R(s,i)+D(s,i)^{\top}\hat{P}(s,i)D(s,i),\quad\
\Theta(s,i)\triangleq-\hat{R}(s,i)^{-1}\hat{S}(s,i).$

Suppose that the cost functional $J(0,0,i_{0};u(\cdot))$ is uniformly convex
in $u(\cdot)$, i.e., for any choice of $i_{0}\in{\cal S}$, there is a positive
constant $\varepsilon$ such that

$\displaystyle
J(0,0,i_{0};u(\cdot))\geqslant\varepsilon\mathbb{E}\int_{0}^{T}|u(s)|^{2}ds,\quad\forall
u(\cdot)\in\mathcal{U}[0,T].$ (3.3)

######  Theorem 3.1.

Under conditions (H) and (3.3), SRE (3.1) admits a unique adapted solution
$(\hat{P}(\cdot,\alpha(\cdot)),\hat{\Lambda}(\cdot),\hat{\zeta}(\cdot))\in$
$L_{\mathbb{F}}^{\infty}(0,T;\mathbb{S}^{n})\times
L_{\mathbb{F}}^{2}(0,T;\mathbb{S}^{n})\times
L_{\mathbb{F}}^{2}(0,T;\mathbb{M}_{D}(\mathbb{S}^{n}))$. In addition,

$\hat{R}(s,\alpha(s))=R(s,\alpha(s))+D(s,\alpha(s))^{\top}\hat{P}(s,\alpha(s))D(s,\alpha(s))\geqslant\varepsilon
I_{m},\quad\hbox{a.e. on $[0,T]$, a.s.}.$ (3.4)

The above theorem shows the solvability of SRE (3.1), based on which the
following closed-loop representation of the open-loop optimal control can be
derived.

######  Theorem 3.2.

Under conditions (H) and (3.3), Problem (M-SLQ) is uniquely open-loop solvable
and the unique open-loop optimal control
$\left\\{u^{*}(s)\right\\}_{s\in[t,T]}$ with respect to the initial triple
$(t,\xi,\vartheta)\in{\cal D}$ has the following linear state feedback:

$u^{*}(s)=\Theta(s,\alpha(s))X^{*}(s),\quad s\in[t,T],$ (3.5)

where $\Theta(\cdot)$ is defined in (3.2) and $\\{X^{*}(s)\\}_{s\in[t,T]}$ is
the solution of the following closed-loop system:

$\left\\{\begin{aligned} \displaystyle
dX^{*}(s)=&\big{[}A(s,\alpha(s))+B(s,\alpha(s))\Theta(s,\alpha(s))\big{]}X^{*}(s)ds\\\
\vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&+\big{[}C(s,\alpha(s))+D(s,\alpha(s))\Theta(s,\alpha(s))\big{]}X^{*}(s)dW(s),\quad
s\in[t,T],\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle X^{*}(t)=&\
\xi,\quad\alpha(t)=\vartheta.\end{aligned}\right.$

######  Remark 3.3.

Sun–Xiong–Yong [20] studied SLQ optimal control problems with random
coefficients, proved the solvability of the corresponding SRE, and obtained
the closed-loop representation of the open-loop optimal control. Compared with
[20], on one hand, the above theorems further extend the results in [20] to
the framework within the Markovian regime switching, which is important in
continuous-time mean-variance portfolio selection problems (see Section 6). On
the other hand, due to the presence of the Markovian regime switching jump in
SRE (3.1), it is difficult to directly prove the solvability of SRE (3.1). To
overcome it, we derive some auxiliary results first.

###  3.2 Some auxiliary results

For simplify the notations, from now on, for any
$(s,\omega,i)\in[0,T]\times\Omega\times{\cal S}$, $x,y,z\in\mathbb{R}^{n}$ and
$u\in\mathbb{R}^{m}$, let

$\displaystyle\displaystyle F(s,i,x,y,z,u)$ $\displaystyle\triangleq
B(s,i)^{\top}y+D(s,i)^{\top}z+S(s,i)x+R(s,i)u,$
$\displaystyle\displaystyle\widetilde{F}(s,i,x,y,z,u)$
$\displaystyle\triangleq
A(s,i)^{\top}y+C(s,i)^{\top}z+Q(s,i)x+S(s,i)^{\top}u,$
$\displaystyle\displaystyle F_{0}(s,i,x,y,z)\triangleq\ $ $\displaystyle
F(s,i,x,y,z,0),\quad\widetilde{F}_{0}(s,i,x,y,z)\triangleq\widetilde{F}(s,i,x,y,z,0).$

The following BSDE is called the associated adjoint equation of the state
equation (2.1):

$\left\\{\begin{array}[]{ll}\displaystyle
dY(s)=-\widetilde{F}(s,\alpha(s),X(s),Y(s),Z(s),u(s))ds+Z(s)dW(s)+\Gamma(s)\bullet
d\widetilde{N}(s),\quad s\in[t,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle
Y(T)=G(\alpha(T))X(T),\quad\alpha(t)=\vartheta,\end{array}\right.$ (3.6)

where $(X(\cdot),u(\cdot))$ comes from (2.1). In order to prove the
solvability of SRE (3.1), we present an alternative characterization of the
solvability of Problem (M-SLQ)0 in terms of the state equation (2.1) and the
adjoint equation (3.6), and then show that Problems (M-SLQ) and (M-SLQ)0 are
equivalent.

######  Theorem 3.4.

Let (H) hold and let the initial triple $(t,\xi,\vartheta)\in{\cal D}$ be
given. A process $u^{*}(\cdot)\in{\cal U}[t,T]$ is an open-loop optimal
control of Problem (M-SLQ)0 with respect to $(t,\xi,\vartheta)$ if and only if
the following two conditions hold:

  * (i)

The mapping $u(\cdot)\mapsto J_{0}(t,0,\vartheta;u(\cdot))$ is convex, or
equivalently,

$J_{0}(t,0,\vartheta;u(\cdot))\geqslant 0,\quad\forall u(\cdot)\in{\cal
U}[t,T].$ (3.7)

  * (ii)

The following stationarity condition hold:

$\displaystyle
F(s,\alpha(s),X^{*}(s),Y^{*}(s),Z^{*}(s),u^{*}(s))=0,\quad\hbox{a.e.
$s\in[t,T]$, a.s.,}$ (3.8)

where $X^{*}(\cdot)$ is the solution of SDE (2.1) and
$(Y^{*}(\cdot),Z^{*}(\cdot),\Gamma^{*}(\cdot))$ is the solution of BSDE (3.6)
with $u(\cdot)$ replaced by $u^{*}(\cdot)$.

######  Remark 3.5.

In the assertion (ii) of Theorem 3.4, the quadruple
$(X^{*}(\cdot),Y^{*}(\cdot),Z^{*}(\cdot),\Gamma^{*}(\cdot))$ is essentially
the solution of the following decoupled FBSDE:

$\left\\{\begin{array}[]{ll}\displaystyle
dX^{*}(s)=\big{[}A(s,\alpha(s))X^{*}(s)+B(s,\alpha(s))u^{*}(s)\big{]}ds+\big{[}C(s,\alpha(s))X^{*}(s)+D(s,\alpha(s))u^{*}(s)\big{]}dW(s),\\\
\vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
dY^{*}(s)=-\widetilde{F}(s,\alpha(s),X^{*}(s),Y^{*}(s),Z^{*}(s),u^{*}(s))ds+Z(s)dW(s)+\Gamma^{*}(s)\bullet
d\widetilde{N}(s),\quad s\in[t,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle X^{*}(t)=\xi,\quad\alpha(t)=\vartheta,\quad
Y^{*}(T)=G(\alpha(T))X^{*}(T).\end{array}\right.$ (3.9)

######  Proposition 3.6.

Let the condition (H) hold. For any given initial triple
$(t,\xi,\vartheta)\in{\cal D}$, a control $u^{*}(\cdot)\in{\cal U}[t,T]$ is
optimal for Problem (M-SLQ)0 if and only if it is optimal for Problem
${(\hbox{M-SLQ})}$.

Based on Theorem 3.4 and 3.6, we have the following results, which are useful
to prove Theorem 3.1 later. For this, denote by $\tau$ an
$\mathbb{F}$-stopping time with values in $[0,T]$ and denote by
$\mathcal{T}[a,b]$ the set of all $\mathbb{F}$-stopping times valued in the
interval $[a,b]$ with $a,b\in[0,T]$.

######  Corollary 3.7.

Let (H) hold and suppose that $u^{*}(\cdot)\in{\cal U}[t,T]$ is an open-loop
optimal control w.r.t. the initial triple $(t,\xi,\vartheta)\in{\cal D}$, then

${V}(t,\xi,\vartheta)={J}(t,\xi,\vartheta;u^{*}(\cdot))=\langle
Y^{*}(t),\xi\rangle,$

where the quadruple
$(X^{*}(\cdot),Y^{*}(\cdot),Z^{*}(\cdot),\Gamma^{*}(\cdot))$ is the adapted
solution of FBSDE (3.9) w.r.t. $u^{*}(\cdot)$.

######  Corollary 3.8.

Let (H) hold and suppose that $u^{*}(\cdot)\in{\cal U}[t,T]$ is an open-loop
optimal control w.r.t. the initial triple $(t,\xi,\vartheta)\in{\cal D}$, then
for any stopping time $\tau\in{\cal T}[t,T]$, the restriction

$u^{*}(\cdot)|_{[\tau,T]}\triangleq\\{u^{*}(s);s\in[\tau,T]\\}$

over the time horizon $[\tau,T]$ remains optimal w.r.t. the initial triple
$(\tau,X^{*}(\tau),\alpha(\tau))$, where $X^{*}(\cdot)$ is the solution of the
forward equation of (3.9).

######  Proposition 3.9.

Let (H) and (3.3) hold, then for any $\tau\in{\cal T}[0,T)$,

$J_{0}(\tau,0,\alpha(\tau);u(\cdot))\geqslant\varepsilon\mathbb{E}\int_{\tau}^{T}|u(s)|^{2}ds,\quad\forall
u(\cdot)\in\mathcal{U}[\tau,T].$

As a direct consequence, Problem (M-SLQ) is uniquely solvable.

Next, we state some properties for the stochastic value flow
${V}(t,\xi,\vartheta)$, which are important to prove Theorem 3.1 later. Recall
that $\\{e_{j}\\}_{j=1}^{n}$ is the standard basis of $\mathbb{R}^{n}$.

######  Proposition 3.10.

Let (H) hold. Suppose that Problem (M-SLQ) is solvable at the initial triple
$(t,e_{j},\vartheta)$ with $1\leqslant j\leqslant n$. Let the state-control
pair $\\{\left(X_{j}(s),u_{j}(s)\right)\\}_{s\in[t,T]}$ of SDE (2.1) be an
open-loop optimal pair w.r.t. $(t,e_{j},\vartheta)$, and let
$\\{\left(Y_{j}(s),Z_{j}(s),\Gamma^{j}(s)\right)\\}_{s\in[t,T]}$ be the
solution of the associated adjoint equation (3.6). Denote
$\boldsymbol{\Gamma}_{kl}(s)\triangleq(\Gamma_{kl}^{1}(s),\cdots,\Gamma_{kl}^{n}(s))$
and

$\displaystyle{\bf X}(s)$
$\displaystyle\triangleq\left(X_{1}(s),\cdots,X_{n}(s)\right),\quad{\bf
U}(s)\triangleq\left(u_{1}(s),\cdots,u_{n}(s)\right),\quad s\in[t,T],$ (3.10)
$\displaystyle{\bf Y}(s)$
$\displaystyle\triangleq\left(Y_{1}(s),\cdots,Y_{n}(s)\right),\quad{\bf
Z}(s)\triangleq\left(Z_{1}(s),\cdots,Z_{n}(s)\right),\quad\boldsymbol{\Gamma}(s)\triangleq\big{(}\boldsymbol{\Gamma}_{kl}(s)\big{)}_{{k,l}\in{\cal
S}}.$

Then the quintuple of matrix-valued processes
$\big{(}\boldsymbol{X}(\cdot),\boldsymbol{U}(\cdot),\boldsymbol{Y}(\cdot),\boldsymbol{Z}(\cdot),\boldsymbol{\Gamma}(\cdot)\big{)}$
satisfies the following FBSDE:

$\left\\{\begin{array}[]{ll}\displaystyle d{\bf
X}(s)=\big{[}A(s,\alpha(s)){\bf X}(s)+B(s,\alpha(s)){\bf
U}(s)\big{]}ds+\big{[}C(s,\alpha(s)){\bf X}(s)+D(s,\alpha(s)){\bf
U}(s)\big{]}dW(s),\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
d{\bf Y}(s)=-\widetilde{F}(s,\alpha(s),{\bf X}(s),{\bf Y}(s),{\bf Z}(s),{\bf
U}(s))ds+{\bf Z}(s)dW(s)+\boldsymbol{\Gamma}(s)\bullet d\widetilde{N}(s),\quad
s\in[t,T],\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle{\bf
X}(t)=I_{n},\quad\alpha(t)=\vartheta,\quad{\bf Y}(T)=G(\alpha(T)){\bf
X}(T),\end{array}\right.$ (3.11)

and

$\displaystyle F(s,\alpha(s),{\bf X}(s),{\bf Y}(s),{\bf Z}(s),{\bf
U}(s))=0,\quad\hbox{a.e. $s\in[t,T]$, a.s.}.$ (3.12)

In addition, for every $\xi\in
L_{\mathscr{F}_{t}}^{\infty}\left(\Omega;\mathbb{R}^{n}\right)$, the state-
control pair $({\bf X}\xi,{\bf U}\xi)=\\{({\bf X}(s)\xi,{\bf
U}(s)\xi)\\}_{s\in[t,T]}$ is optimal with respect to $(t,\xi,\vartheta)$ and
the triple $({\bf Y}\xi,{\bf Z}\xi,\boldsymbol{\Gamma}\circ\xi)=\\{({\bf
Y}(s)\xi,{\bf Z}(s)\xi,\boldsymbol{\Gamma}(s)\circ\xi)\\}_{s\in[t,T]}$ with
$\mathbf{\Gamma}(s)\circ\xi\triangleq(\Gamma_{kl}(s)\xi)_{k,l\in{\cal S}}$
solves the adjoint BSDE (3.6) associated with the state-control pair $({\bf
X}\xi,{\bf U}\xi)$.

Based on the above results, we have the following theorems, which present a
quadratic form of the stochastic value flow into a bounded and left-continuous
process.

######  Theorem 3.11.

Let (H) hold. For any given $t\in[0,T]$ and $\vartheta\in
L_{\mathscr{F}_{t}^{\alpha}}^{2}(\Omega;{\cal S})$, if Problem (M-SLQ) is
solvable at the initial time $t$, then there is a process $P:[0,T]\times{\cal
S}\times\Omega\rightarrow\mathbb{S}^{n}$, which is $\mathbb{F}$-adapted, such
that

${V}(t,\xi,\vartheta)=\langle P(t,\vartheta)\xi,\xi\rangle,\quad\forall\xi\in
L_{\mathscr{F}_{t}}^{\infty}(\Omega;\mathbb{R}^{n}).$ (3.13)

######  Theorem 3.12.

Let conditions (H) and (3.3) hold. Then the process
$P=\\{P(t,i);(t,i)\in[0,T]\times{\cal S}\\}$ appeared in (3.13) is bounded and
left-continuous.

Finally, based on Theorem 3.11 and Theorem 3.12, we introduce a stopped
stochastic linear-quadratic problem and present some results for it, which is
useful to the proof of Theorem 3.1 too. Recall that $\tau\in{\cal T}(0,T]$ is
an $\mathbb{F}$-stopping time, and set

$\mathcal{D}^{\tau}\triangleq\big{\\{}(\sigma,\xi,\vartheta)\ |\
\sigma\in\mathcal{T}[0,\tau),\ \xi\in
L^{2}_{\mathscr{F}_{\sigma}}(\Omega;\mathbb{R}^{n}),\ \vartheta\in
L_{\mathscr{F}_{\sigma}^{\alpha}}^{2}(\Omega;\mathcal{S})\big{\\}}.$

Problem (M-SLQ)τ: For any given initial triple
$(\sigma,\xi,\vartheta)\in\mathcal{D}^{\tau}$, find a control
$u^{*}(\cdot)\in\mathcal{U}[\sigma,\tau]$ such that

$J^{\tau}(\sigma,\xi,\vartheta;u^{*}(\cdot))=\mathop{\rm
essinf}_{u(\cdot)\in{\cal
U}[\sigma,\tau]}J^{\tau}(\sigma,\xi,\vartheta;u(\cdot))\triangleq
V^{\tau}(\sigma,\xi,\vartheta),$

where the cost functional

$\displaystyle{J}^{\tau}(\sigma,\xi,\vartheta;u(\cdot))\triangleq\mathbb{E}_{\sigma}\left[\big{\langle}P(\tau,\alpha(\tau))X(\tau),X(\tau)\big{\rangle}+\int_{\sigma}^{\tau}\left\langle\begin{pmatrix}Q(s,\alpha(s))&S(s,\alpha(s))^{\top}\\\
S(s,\alpha(s))&R(s,\alpha(s))\end{pmatrix}\begin{pmatrix}X(s)\\\
u(s)\end{pmatrix},\begin{pmatrix}X(s)\\\ u(s)\end{pmatrix}\right\rangle
ds\right],$

and $X(\cdot)$ is the solution of (2.1) w.r.t. the initial triple
$(\sigma,\xi,\vartheta)$ over the stochastic interval $[\sigma,\tau]$.

Similar to 2.1, when the above conditional expectation
$\mathbb{E}_{\sigma}[\cdot]$ degenerates to the expectation $\mathbb{E}$, we
denote the related problem, cost functional, and value function by Problem
(M-SLQ)${}^{\tau}_{0}$, $J^{\tau}_{0}(t,\xi,\vartheta;u(\cdot))$, and
$V^{\tau}_{0}(t,\xi,\vartheta)$, respectively.

######  Proposition 3.13.

Let conditions (H) and (3.3) hold. Then

  * (i)

Problem (M-SLQ)τ is uniquely solvable at every $\sigma\in\mathcal{T}[0,\tau)$.

  * (ii)

If $u^{*}(\cdot)\in\mathcal{U}[\sigma,T]$ is an open-loop optimal control of
Problem (M-SLQ) w.r.t. $(\sigma,\xi,\vartheta)\in\mathcal{T}[0,\tau)\times
L_{\mathscr{F}_{\sigma}}^{\infty}\left(\Omega;\mathbb{R}^{n}\right)\times
L_{\mathscr{F}_{\sigma}^{\alpha}}^{2}\left(\Omega;{\cal S}\right)$, then the
restriction $\left.u^{*}(\cdot)\right|_{[\sigma,\tau]}$ of $u^{*}(\cdot)$ on
the interval $[\sigma,\tau]$ is also an open-loop optimal control of Problem
(M-SLQ)τ w.r.t. the same initial triple $(\sigma,\xi,\vartheta)$.

  * (iii)

The stochastic value flow $V^{\tau}(\cdot)$ of Problem (M-SLQ)τ has the
following form:

$V^{\tau}(\sigma,\xi,\vartheta)=\langle
P(\sigma,\vartheta)\xi,\xi\rangle,\quad\forall(\sigma,\xi,\vartheta)\in\mathcal{T}[0,\tau)\times
L_{\mathscr{F}_{\sigma}}^{\infty}\left(\Omega;\mathbb{R}^{n}\right)\times
L_{\mathscr{F}_{\sigma}^{\alpha}}^{2}\left(\Omega;{\cal S}\right),$

where $P(\cdot,\alpha(\cdot))$ is the process appeared in (3.13).

##  4 Proof of the solvability of SRE

In this section, based on the auxiliary results of Subsection 3.2, we prove
the solvability of SRE (3.1). First, we prove that SRE (3.1) is uniquely
solvable, and then prove that the first component
$\hat{P}(\cdot,\alpha(\cdot))$ of SRE (3.1) is exactly the process
$P(\cdot,\alpha(\cdot))$ appeared in (3.13). Finally, as a by-product, the
open-loop optimal control is represented as linear feedback of the state.

We will use several lemmas to prove Theorem 3.1. Recall that SDE (2.1) is the
state equation, BSDE (3.6) is the associated adjoint equation, and
$\\{e_{j}\\}_{j=1}^{n}$ is the standard basis of $\mathbb{R}^{n}$.

######  Lemma 4.1.

Suppose the conditions (H) and (3.3) hold. Denote by
$\\{X_{j}(s)\\}_{s\in[0,T]}$ the (unique) open-loop optimal state process
corresponding to the initial triple $(t,\xi,\vartheta)=(0,e_{j},i_{0})$. Then
the $\mathbb{R}^{n\times n}$-valued process
$\\{\boldsymbol{X}(s)\\}_{s\in[0,T]}$ with
$\boldsymbol{X}(s)\triangleq\left(X_{1}(s),\ldots,X_{n}(s)\right)$ is
invertible.

###### Proof.

Let $u_{j}(\cdot)\in\mathcal{U}[0,T]$ be the unique open-loop optimal control
w.r.t. $(0,e_{j},i_{0})$ so that

$\left\\{\begin{aligned}
\displaystyle&dX_{j}(s)=\big{[}A(s,\alpha(s))X_{j}(s)+B(s,\alpha(s))u_{j}(s)\big{]}ds\\\
\vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&\qquad\qquad+\big{[}C(s,\alpha(s))X_{j}(s)+D(s,\alpha(s))u_{j}(s)\big{]}dW(s),\quad
s\in[0,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&\displaystyle
X(0)=e_{j},\quad\alpha(0)=i_{0}.\end{aligned}\right.$

Then, with $\boldsymbol{U}(s)=\left(u_{1}(s),\ldots,u_{n}(s)\right)$, one has

$\left\\{\begin{aligned} \displaystyle&d{\bf X}(s)=\big{[}A(s,\alpha(s)){\bf
X}(s)+B(s,\alpha(s)){\bf U}(s)\big{]}ds\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&\qquad\quad\ \ +\big{[}C(s,\alpha(s)){\bf
X}(s)+D(s,\alpha(s)){\bf U}(s)\big{]}dW(s),\quad s\in[0,T],\\\ \vskip 3.0pt
plus 1.0pt minus 1.0pt\cr\displaystyle&{\bf
X}(0)=I_{n},\quad\alpha(0)=i_{0}.\end{aligned}\right.$ (4.1)

We define the following stopping time (at which, for the first time, ${\bf
X}(\cdot)$ is not invertible)

$\theta(\omega)=\inf\big{\\{}s\in[0,T];\operatorname{det}(\boldsymbol{X}(s,\omega))=0\big{\\}},$

where we use the convention that the infimum of the empty set is infinity. It
should be pointed out that if we prove that $\mathbb{P}(\theta=\infty)=1$,
i.e., the set $\mathbb{O}\triangleq\\{\omega\in\Omega:\theta(\omega)\leqslant
T\\}$ has probability zero, then we can get that $X(\cdot)$ is invertible.

Suppose the contrary and set $\tau=\theta\wedge T$. Then $\tau$ is in ${\cal
T}(0,T]$ too. When $\tau=\theta$ on $\mathbb{O}$, we know that
$\boldsymbol{X}(\tau)$ is not invertible on $\mathbb{O}$ by the definition of
$\theta$. Thus, we can choose an $\mathscr{F}_{\tau}$-measurable,
$\mathbb{S}^{n}$-valued, positive semi-definite random matrix $H$ with $|H|=1$
on $\mathbb{O}$ such that $H(\omega)\boldsymbol{X}(\tau(\omega),\omega)=0$ for
any $\omega\in\Omega$.

Note that $P=\\{P(t,i);(t,i)\in[0,T]\times{\cal S}\\}$ is bounded, left-
continuous, and satisfies (3.13) according to Theorem 3.11 and Theorem 3.12.
For $\sigma\in{\cal T}[0,\tau]$ with $s\in[\sigma,\tau]$, we consider the
following equation

$\left\\{\begin{array}[]{ll}\displaystyle
dX(s)=\big{[}A(s,\alpha(s))X(s)+B(s,\alpha(s))u(s)\big{]}ds+\big{[}C(s,\alpha(s))X(s)+D(s,\alpha(s))u(s)\big{]}dW(s),\\\
\vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
X(\sigma)=\xi,\quad\alpha(\sigma)=\vartheta,\end{array}\right.$ (4.2)

and the following auxiliary cost functional

$J^{H}(\sigma,\xi,\vartheta;u(\cdot))\triangleq
J^{\tau}(\sigma,\xi,\vartheta;u(\cdot))+\mathbb{E}_{\sigma}\big{[}\langle
HX(\tau),X(\tau)\rangle\big{]}.$ (4.3)

Consider the problem of minimizing the cost functional (4.3) subject to the
state equation (4.2), called Problem ${(\hbox{M-SLQ})_{H}}$. Moreover, denote
by ${V_{H}}(\cdot)$ the associated stochastic value flow. Then the following
two assertions hold:

  * (i)

For any choice of $\sigma\in\mathcal{T}[0,\tau)$ and $\vartheta\in
L_{\mathscr{F}_{t}^{\alpha}}^{2}(\Omega;\mathcal{S})$ with
$\alpha(\sigma)=\vartheta$, the following inequality holds:

$\displaystyle\mathbb{E}\big{[}{J^{H}}(\sigma,0,\vartheta;u(\cdot))\big{]}\geqslant\mathbb{E}\big{[}J^{\tau}(\sigma,0,\vartheta;u(\cdot))\big{]}\geqslant\varepsilon\mathbb{E}\big{[}\int_{\sigma}^{\tau}|u(s)|^{2}ds\big{]},\quad\forall
u(\cdot)\in\mathcal{U}[\sigma,\tau].$

  * (ii)

The restriction $u_{j}^{\tau}(\cdot)=\left.u_{j}(\cdot)\right|_{[0,\tau]}$
over the time inverval $[0,\tau]$ is optimal for both Problem (M-SLQ)τ and
Problem ${(\hbox{M-SLQ})_{H}}$ w.r.t. the same initial triple
$(0,e_{j},i_{0})$.

For the above assertion (i), in fact, the first inequality is true since $H$
is positive semi-definite, and the second inequality comes from the proof of
the assertion (i) of 3.13. Then, as a result of 3.9, both Problems (M-SLQ)τ
and ${(\hbox{M-SLQ})_{H}}$ are uniquely solvable at any
$\sigma\in\mathcal{T}[0,\tau).$

For the above assertion (ii), on one hand, by the assertion (ii) of 3.13, we
see that $u_{j}^{\tau}(\cdot)$ is optimal for Problem (M-SLQ)τ w.r.t. the
initial triple $(0,e_{j},i_{0})$. On the other hand, according to Theorem 3.4,
in order to show that $u_{j}^{\tau}(\cdot)$ is an open-loop optimal control
for Problem ${(\hbox{M-SLQ})_{H}}$ w.r.t. $\left(0,e_{j},i_{0}\right)$, it
suffices to prove that the solution
$(X_{j}^{\tau}(\cdot),Y_{j}^{\tau}(\cdot),Z_{j}^{\tau}(\cdot),\Gamma^{j,\tau}(\cdot))$
of the following FBSDE:

$\left\\{\begin{array}[]{ll}\displaystyle
dX_{j}^{\tau}(s)=\big{[}A(s,\alpha(s))X_{j}^{\tau}(s)+B(s,\alpha(s))u_{j}^{\tau}(s)\big{]}ds+\big{[}C(s,\alpha(s))X_{j}^{\tau}(s)+D(s,\alpha(s))u_{j}^{\tau}(s)\big{]}dW(s),\\\
\vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
dY_{j}^{\tau}(s)=-\widetilde{F}(s,\alpha(s),X_{j}^{\tau}(s),Y_{j}^{\tau}(s),Z_{j}^{\tau}(s),u_{j}^{\tau}(s))ds+Z_{j}^{\tau}(s)dW(s)+\Gamma^{j,\tau}(s)\bullet
d\widetilde{N}(s),\ s\in[0,\tau],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle X_{j}^{\tau}(0)=e_{j},\quad\alpha(0)=i_{0},\quad
Y_{j}^{\tau}(\tau)=[P(\tau,\alpha(\tau))+H]X_{j}^{\tau}(\tau),\end{array}\right.$
(4.4)

satisfying the following stationarity condition:

$\displaystyle
F(s,\alpha(s),X_{j}^{\tau}(s),Y_{j}^{\tau}(s),Z_{j}^{\tau}(s),u_{j}^{\tau}(s))=0,\quad\hbox{a.e.
$s\in[0,\tau]$, a.s.}.$ (4.5)

Note that $X_{j}^{\tau}(s)=\boldsymbol{X}(s)e_{j}$ for $0\leqslant
s\leqslant\tau$. Thus, by the choice of $H$, we have

$HX_{j}^{\tau}(\tau)=H\boldsymbol{X}(\tau)e_{j}=0.$ (4.6)

It follows that the terminal value
$Y_{j}^{\tau}(\tau)=P(\tau,\alpha(\tau))X_{j}^{\tau}(\tau)$, which implies
that FBSDE (4.4) is exactly the FBSDE associated with Problem (M-SLQ)τ. Then
from Theorem 3.4 again, the stationarity condition (4.5) follows from the fact
that $u_{j}^{\tau}(\cdot)$ is an open-loop optimal control of Problem (M-SLQ)τ
with respect to $\left(0,e_{j},i_{0}\right)$.

Now, for Problem ${(\hbox{M-SLQ})_{H}}$, by the above assertion (i) and
Theorem 3.11, there is a bounded, left-continuous and $\mathbb{S}^{n}$-valued
process $P_{H}(\cdot,i)$ such that

${V_{H}}(\sigma,\xi,\vartheta)=\langle
P_{H}(\sigma,\vartheta)\xi,\xi\rangle,\quad\forall(\sigma,\xi,\vartheta)\in{\cal
T}[0,\tau)\times
L_{\mathscr{F}_{\sigma}}^{\infty}(\Omega;\mathbb{R}^{n})\times
L_{\mathscr{F}_{\sigma}^{\alpha}}^{2}(\Omega;{\cal S}).$ (4.7)

Next, we prove that the process $P(\cdot,\alpha(\cdot))$ appeared in (3.13)
equals to $P_{H}(\cdot,\alpha(\cdot))$ appeared in (4.7), i.e.,

$P(t,\alpha(t))={P}_{H}(t,\alpha(t)),\quad t\in[0,\tau).$ (4.8)

By the above assertion (ii), we see that
$\\{\left(X_{j}^{\tau}(s),u_{j}^{\tau}(s)\right)\\}_{s\in[0,\tau]}$ is the
optimal state-control pair for both Problem (M-SLQ)τ and Problem
${(\hbox{M-SLQ})_{H}}$ w.r.t. the initial triple $\left(0,e_{j},i_{0}\right)$.
Let

$\boldsymbol{X}^{\tau}(s)\triangleq\left(X_{1}^{\tau}(s),\ldots,X_{n}^{\tau}(s)\right),\quad\boldsymbol{U}^{\tau}(s)\triangleq\left(u_{1}^{\tau}(s),\ldots,u_{n}^{\tau}(s)\right),\quad
s\in[0,\tau],$

and take an arbitrary $x\in\mathbb{R}^{n}$. From 3.10, we have that the pair
$\left(\boldsymbol{X}^{\tau}x,\boldsymbol{U}^{\tau}x\right)$ is the optimal
state-control for both Problem (M-SLQ)τ and Problem ${(\hbox{M-SLQ})_{H}}$
w.r.t. the initial triple $(0,x,i_{0})$. Moreover, by 3.8, the following pair

$\big{(}\boldsymbol{X}^{\tau}(s)x,\boldsymbol{U}^{\tau}(s)x\big{)},\quad
s\in[t,\tau],$

remains optimal w.r.t. $\left(t,\boldsymbol{X}^{\tau}(t)x,\alpha(t)\right)$
for every $t\in[0,\tau)$. Thus, note that $H\boldsymbol{X}^{\tau}(\tau)=0$ by
(4.6),

$\displaystyle{V_{H}}\left(t,\boldsymbol{X}^{\tau}(t)x,\alpha(t)\right)$
$\displaystyle={J_{H}}\left(t,\boldsymbol{X}^{\tau}(t)x,\alpha(t);\boldsymbol{U}^{\tau}(t)x\right)=J\left(t,\boldsymbol{X}^{\tau}(t)x,\alpha(t);\boldsymbol{U}^{\tau}(t)x\right)+\mathbb{E}\left\langle
H\boldsymbol{X}^{\tau}(\tau)x,\boldsymbol{X}^{\tau}(\tau)x\right\rangle$
$\displaystyle=J\left(t,\boldsymbol{X}^{\tau}(t)x,\alpha(t);\boldsymbol{U}^{\tau}(t)x\right)={V}\left(t,\boldsymbol{X}^{\tau}(t)x,\alpha(t)\right).$

Noting that $\boldsymbol{X}^{\tau}(t)=\boldsymbol{X}(t)$ when $t\in[0,\tau)$,
we deduce from the above that

$\langle{P}_{H}(t,\alpha(t))\boldsymbol{X}(t)x,\boldsymbol{X}(t)x\rangle={V_{H}}\left(t,\boldsymbol{X}^{\tau}(t)x,\alpha(t)\right)={V}\left(t,\boldsymbol{X}^{\tau}(t)x,\alpha(t)\right)=\langle
P(t,\alpha(t))\boldsymbol{X}(t)x,\boldsymbol{X}(t)x\rangle.$

Since $x\in\mathbb{R}^{n}$ is arbitrary, it follows that

$\boldsymbol{X}(t)^{\top}P(t,\alpha(t))\boldsymbol{X}(t)=\boldsymbol{X}(t)^{\top}{P}_{H}(t,\alpha(t))\boldsymbol{X}(t),\quad
t\in[0,\tau).$

From the definition of $\tau,$ we see that $\boldsymbol{X}$ is invertible on
$[0,\tau)$, which implies that the relation (4.8) holds. Thus, for some choice
of $i\in{\cal S}$, one has

$P(t,i)=P_{H}(t,i),\quad t\in[0,\tau).$ (4.9)

However, on the other hand, ${P}_{H}(\tau,i)=P(\tau,i)+H$, and both
$P(\cdot,i)$ and ${P}_{H}(\cdot,i)$ are left-continuous. Finally, in (4.9),
letting $t\uparrow\tau$ yields a contradiction: $P(\tau,i)=P(\tau,i)+H$, since
$|H|=1$ on $\mathbb{O}$. ∎

Note that under the conditions of Theorem 3.1, Problem (M-SLQ) is uniquely
solvable by 3.9. Let $\\{\left(X_{j}(s),u_{j}(s)\right)\\}_{s\in[0,T]}$ be the
open-loop optimal pair and
$\left\\{\left(Y_{j}(s),Z_{j}(s),\Gamma^{j}(s)\right)\right\\}_{s\in[t,T]}$ be
the adapted solution of the adjoint BSDE corresponding to the initial triple
$(0,e_{j},i_{0})$ for each $j=1,2,...,n$, respectively. Now, with
$\boldsymbol{\Gamma}_{kl}(s)\triangleq(\Gamma_{kl}^{1}(s),\cdots,\Gamma_{kl}^{n}(s)),$
3.10 implies that the matrix-valued processes
$(\boldsymbol{X}(\cdot),\boldsymbol{U}(\cdot),\boldsymbol{Y}(\cdot),\boldsymbol{Z}(\cdot),\boldsymbol{\Gamma}(\cdot))$
defined by (3.10) satisfy FBSDE (3.11) with the initial triple
$(0,I_{n},i_{0})$ over $[0,T]$. Moreover, the following stationary holds:

$\displaystyle F(s,\alpha(s),{\bf X}(s),{\bf Y}(s),{\bf Z}(s),{\bf
U}(s))=0,\quad\hbox{a.e. $s\in[0,T]$, a.s.}.$ (4.10)

In addition, on one hand, from 4.1 we see that ${\bf X}(\cdot)$ is invertible.
On the other hand, from Theorem 3.11 and Theorem 3.12, there is a process
$P:[0,T]\times{\cal S}\times\Omega\rightarrow\mathbb{S}^{n}$, which is left-
continuous, bounded and $\mathbb{F}^{W}$-adapted, that satisfies (3.13).

######  Lemma 4.2.

Suppose that condition (H) holds, and there is a constant $\varepsilon>0$ such
that (3.3) holds. Then

$P(s,\alpha(s))=\boldsymbol{Y}(s)\boldsymbol{X}(s)^{-1},\quad s\in[0,T],$
(4.11)

where $P(\cdot,\alpha(\cdot))$ is the process appeared in (3.13) and the pair
$(\boldsymbol{X}(\cdot),\boldsymbol{Y}(\cdot))$ is defined in (3.10).

###### Proof.

For arbitrary $x\in\mathbb{R}^{n}$ and $s\in[0,T]$, set that

$\displaystyle X^{*}(s)=\boldsymbol{X}(s)x,\quad
u^{*}(s)=\boldsymbol{U}(s)x,\quad Y^{*}(s)=\boldsymbol{Y}(s)x,\quad
Z^{*}(s)=\boldsymbol{Z}(s)x,\quad\Gamma^{*}(s)=\boldsymbol{\Gamma}(s)\circ x.$

Then, on one hand, 3.10 implies that the pair
$\left(X^{*}(\cdot),u^{*}(\cdot)\right)$ is an open-loop optimal pair w.r.t.
the initial triple $(0,x,i_{0})$, and that the triple
$\left(Y^{*}(\cdot),Z^{*}(\cdot),\Gamma^{*}(\cdot)\right)$ is the adapted
solution to the adjoint BSDE associated with
$\left(X^{*}(\cdot),u^{*}(\cdot)\right)$. On the other hand, for every
$s\in[0,T]$, 3.8 implies that the restriction
$(X^{*}(\cdot)|_{[s,T]},u^{*}(\cdot)|_{[s,T]})$ of
$(X^{*}(\cdot),u^{*}(\cdot))$ over the interval $[s,T]$ remains optimal with
respect to $(s,X^{*}(s),\alpha(s))$. Thus, from 3.7, we have that

${V}\left(s,X^{*}(s),\alpha(s)\right)=\left\langle
Y^{*}(s),X^{*}(s)\right\rangle.$

Owing to the relation (3.13), the above equation implies that

$\displaystyle
x^{\top}\boldsymbol{X}(s)^{\top}P(s,\alpha(s))\boldsymbol{X}(s)x$
$\displaystyle=\langle
P(s,\alpha(s))\boldsymbol{X}(s)x,\boldsymbol{X}(s)x\rangle=\left\langle
P(s,\alpha(s))X^{*}(s),X^{*}(s)\right\rangle={V}\left(s,X^{*}(s),\alpha(s)\right)$
$\displaystyle=\left\langle
Y^{*}(s),X^{*}(s)\right\rangle=\langle\boldsymbol{Y}(s)x,\boldsymbol{X}(s)x\rangle=x^{\top}\boldsymbol{X}(s)^{\top}\boldsymbol{Y}(s)x.$

Since $x\in\mathbb{R}^{n}$ is arbitrary, we deduce that

$\boldsymbol{X}(s)^{\top}P(s,\alpha(s))\boldsymbol{X}(s)=\boldsymbol{X}(s)^{\top}\boldsymbol{Y}(s).$

The desired result follows from the fact that $\boldsymbol{X}(\cdot)$ is
invertible. ∎

In the following, please keep in mind that $P(\cdot,\alpha(\cdot))$ represents
the process appeared in (3.13) and
$(\boldsymbol{X}(\cdot),\boldsymbol{U}(\cdot),\boldsymbol{Y}(\cdot),\boldsymbol{Z}(\cdot),\boldsymbol{\Gamma}(\cdot))$
defined by (3.10) satisfy FBSDE (3.11) with $(0,I_{n},i_{0})$ over $[0,T]$.

######  Lemma 4.3.

Suppose (H) and (3.3) hold. Then, with relation (4.11) and notation,

$\displaystyle\ \Theta(s,\alpha(s))={\bf
U}(s)\boldsymbol{X}(s)^{-1},\quad\Lambda(s)=\Pi(s,\alpha(s))-P(s,\alpha(s))[C(s)+D(s)\Theta(s,\alpha(s))],$
(4.12) $\displaystyle\ \Pi(s,\alpha(s))={\bf
Z}(s)\boldsymbol{X}(s)^{-1},\quad\zeta(s)=(\zeta_{k,l}(s))_{k,l\in{\cal
S}}\mbox{ \rm with
}\zeta_{kl}(s)\triangleq\boldsymbol{\Gamma}_{kl}(s)\boldsymbol{X}^{-1}(s),\quad
s\in[0,T],$

we have that the triple $(P(\cdot,\alpha(\cdot)),\Lambda(\cdot),\zeta(\cdot))$
satisfies the following BSDE:

$\left\\{\begin{aligned} \displaystyle
dP(s,\alpha(s))=&-\big{[}\hat{Q}(s,\alpha(s))+\hat{S}(s,\alpha(s))^{\top}\Theta(s,\alpha(s))\big{]}ds+\Lambda(s)dW(s)+\zeta(s)\bullet
d\widetilde{N}(s),\quad s\in[0,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle
P(T,\alpha(T))=&G(\alpha(T)),\quad\alpha(0)=i_{0},\end{aligned}\right.$ (4.13)

where $\hat{Q}(\cdot)$ and $\hat{S}(\cdot)$ are defined in (3.2). Moreover,
$\Lambda(\cdot)$ is symmetric and the following relation holds:

$\displaystyle\hat{S}(s,\alpha(s))+\hat{R}(s,\alpha(s))\Theta(s,\alpha(s))=0,\quad\text{
a.e. }s\in[0,T],\text{ a.s.}.$ (4.14)

###### Proof.

First, by the relation (3.13), we have

$\langle G(\alpha(T))\xi,\xi\rangle={V}(T,\xi,\alpha(T))=\langle
P(T,\alpha(T))\xi,\xi\rangle,\quad\forall\xi\in
L_{\mathscr{F}_{T}}^{\infty}\left(\Omega;\mathbb{R}^{n}\right),$

which leads to that $P(T,\alpha(T))=G(\alpha(T))$. Note that
$\boldsymbol{X}(\cdot)$ satisfies SDE (4.1) and is invertible, so its
invertibility (denoted by $\boldsymbol{X}^{-1}(\cdot)$) exists and satisfies
the following SDE:

$d\boldsymbol{X}(s)^{-1}=\Xi(s,\alpha(s))ds+\Delta(s,\alpha(s))dW(s),\quad
s\in[0,T],$

where

$\displaystyle\Xi(s,\alpha(s))$
$\displaystyle=\boldsymbol{X}^{-1}(s)\big{\\{}\big{[}C(s,\alpha(s))+D(s,\alpha(s))\Theta(s,\alpha(s))\big{]}^{2}-A(s,\alpha(s))-B(s,\alpha(s))\Theta(s,\alpha(s))\big{\\}},$
$\displaystyle\Delta(s,\alpha(s))$
$\displaystyle=-\boldsymbol{X}^{-1}(s)\big{[}C(s,\alpha(s))+D(s,\alpha(s))\Theta(s,\alpha(s))\big{]}.$

Applying Itô’s formula to the right-hand side of (4.11), we have

$\displaystyle dP(s,\alpha(s))$ $\displaystyle=-F(s,\alpha(s),{\bf X}(s),{\bf
Y}(s),{\bf Z}(s),{\bf
U}(s))\boldsymbol{X}^{-1}(s)ds+Z(s)\Delta(s)ds+\boldsymbol{Z}(s)\boldsymbol{X}^{-1}(s)dW(s)$
$\displaystyle\quad+\sum_{k,l=1}^{D}\boldsymbol{\Gamma}_{kl}(s)\boldsymbol{X}^{-1}(s)d\widetilde{N}_{kl}(s)+\boldsymbol{Y}(s)\Xi(s,\alpha(s))ds+\boldsymbol{Y}(s)\Delta(s,\alpha(s))dW(s)$
$\displaystyle=\big{[}-A(s,\alpha(s))^{\top}P(s,\alpha(s))-C(s,\alpha(s))^{\top}\Pi(s,\alpha(s))-Q(s,\alpha(s))-S(s,\alpha(s))^{\top}\Theta(s,\alpha(s))$
$\displaystyle\quad+P(s,\alpha(s))\big{[}(C(s,\alpha(s))+D(s,\alpha(s))\Theta(s,\alpha(s)))^{2}-A(s,\alpha(s))-B(s,\alpha(s))\Theta(s,\alpha(s))\big{]}$
$\displaystyle\quad-\Pi(s,\alpha(s))(C(s,\alpha(s))+D(s,\alpha(s))\Theta(s,\alpha(s))\big{]}ds+\sum_{k,l=1}^{D}\zeta_{kl}(s)d\widetilde{N}_{kl}(s)$
$\displaystyle\quad+\big{[}\Pi(s,\alpha(s))-P(s,\alpha(s))[C(s,\alpha(s))+D(s,\alpha(s))\Theta(s,\alpha(s))]\big{]}dW(s)$
$\displaystyle=-\big{[}\hat{Q}(s,\alpha(s))+\hat{S}(s,\alpha(s))^{\top}\Theta(s,\alpha(s))\big{]}ds+\Lambda(s)dW(s)+\zeta(s)\bullet
d\widetilde{N}(s).$

Note that $P(s,\alpha(s))$ is symmetric, i.e.,
$P(s,\alpha(s))=P(s,\alpha(s))^{\top}$ for $s\in[0,T]$. Comparing the
diffusion coefficients of the above BSDEs satisfied by
$P(\cdot,\alpha(\cdot))$ and $P^{\top}(\cdot,\alpha(\cdot))$, we obtain
$\Lambda(s)=\Lambda(s)^{\top}$ for $s\in[0,T]$. Further combining (4.10) and
(4.12), we get

$\displaystyle\hat{S}(s,\alpha(s))+\hat{R}(s,\alpha(s))\Theta(s,\alpha(s))=F(s,\alpha(s),{\bf
X}(s),{\bf Y}(s),{\bf Z}(s),{\bf U}(s))\boldsymbol{X}^{-1}(s)=0.$

This completes the proof. ∎

######  Lemma 4.4.

Suppose (H) holds, and there exists a constant $\varepsilon>0$ such that (3.3)
holds. Then

$\displaystyle\hat{R}(s,\alpha(s))=R(s,\alpha(s))+D(s,\alpha(s))^{\top}P(s,\alpha(s))D(s,\alpha(s))\geqslant\varepsilon
I_{m},\quad\hbox{a.e. on $[0,T]$, a.s..}$ (4.15)

###### Proof.

The procedure of proof can be divided into three steps.

Step 1. Let us temporarily assume that processes
$\Theta(\cdot)=\\{\Theta(s,\alpha(s))\\}_{s\in[0,T]}$ and
$\Lambda(\cdot)=\\{\Lambda(s)\\}_{s\in[0,T]}$ defined by (4.12) satisfy the
following condition:

$\mathop{\rm
esssup}_{\omega\in\Omega}\int_{0}^{T}\big{[}|\Theta(s,\omega,\alpha(s))|^{2}+|\Lambda(s,\omega)|^{2}\big{]}ds<\infty.$
(4.16)

Choosing an arbitrary control $v(\cdot)\in\mathcal{U}[0,T]$, we consider the
following SDE

$\left\\{\begin{aligned} \displaystyle
dX^{v}(s)=&\big{\\{}\big{[}A(s,\alpha(s))+B(s,\alpha(s))\Theta(s,\alpha(s))\big{]}X^{v}(s)+B(s,\alpha(s))v(s)\big{\\}}ds\\\
\vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&+\big{\\{}\big{[}C(s,\alpha(s))+D(s,\alpha(s))\Theta(s,\alpha(s))\big{]}X^{v}(s)+D(s,\alpha(s))v(s)\big{\\}}dW(s),\quad
s\in[0,T],\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
X^{v}(0)=&0,\quad~{}\alpha(0)=i_{0}.\end{aligned}\right.$ (4.17)

By the standard SDE theory, we see that $X^{v}(\cdot)$, solution of SDE
(4.17), belongs to space $L_{\mathbb{F}}^{2}(\Omega;C([0,T];\mathbb{R}^{n}))$.
Hence,

$u(s)\triangleq\Theta(s,\alpha(s))X^{v}(s)+v(s)\in\mathcal{U}[0,T].$ (4.18)

According to $u(\cdot)$ defined in (4.18) and the uniqueness of SDEs,
$X(\cdot)$, solution of equation (2.1) w.r.t. the initial triple $(0,0,i_{0})$
over $[0,T]$, coincides with $X^{v}(\cdot)$, solution of (4.17), i.e.,

$X^{v}(s)=X(s),\quad s\in[0,T].$ (4.19)

Note that $P(s,\alpha(s))$ satisfies BSDE (4.13). By applying Itô’s formula to
$\big{\langle}P(s,\alpha(s))X^{v}(s),X^{v}(s)\big{\rangle}$ firstly and then
taking expectations on both sides, we can get

$\displaystyle\mathbb{E}\big{\langle}G(\alpha(T))X^{v}(T),X^{v}(T)\big{\rangle}$
$\displaystyle=\mathbb{E}\int_{0}^{T}\Big{\\{}-\big{\langle}\big{[}Q(s,\alpha(s))+\hat{S}(s,\alpha(s))^{\top}\Theta(s,\alpha(s))\big{]}X^{v}(s),X^{v}(s)\big{\rangle}$
(4.20) $\displaystyle\qquad\qquad\ \
+2\big{\langle}\big{[}\hat{S}(s,\alpha(s))-S(s,\alpha(s))\big{]}^{\top}u(s),X^{v}(s,\alpha(s))\big{\rangle}$
$\displaystyle\qquad\qquad\ \
+\big{\langle}D(s,\alpha(s))^{\top}P(s,\alpha(s))D(s,\alpha(s))u(s),u(s)\big{\rangle}\Big{\\}}ds.$

Substituting (4.20) into the cost functional (2.3) with the initial triple
$(0,0,i_{0})$ and noting that (4.19), we have

$\displaystyle J(0,0,i_{0};u(\cdot))$
$\displaystyle=\mathbb{E}\int_{0}^{T}\Big{\\{}-\big{\langle}\hat{S}(s,\alpha(s))^{\top}\Theta(s,\alpha(s))X^{v}(s),X^{v}(s)\big{\rangle}$
$\displaystyle\qquad\quad\quad\ \
+2\big{\langle}\hat{S}(s,\alpha(s))^{\top}u(s),X^{v}(s)\big{\rangle}+\big{\langle}\hat{R}(s,\alpha(s))u(s),u(s)\big{\rangle}\Big{\\}}ds.$

Combining (4.14) and (4.18), we further deduce that

$\displaystyle\displaystyle J(0,0,i_{0};u(\cdot))$
$\displaystyle=\mathbb{E}\int_{0}^{T}\big{\langle}\hat{R}(s,\alpha(s))\big{[}u(s)-\Theta(s,\alpha(s))X^{v}(s)\big{]},u(s)-\Theta(s,\alpha(s))X^{v}(s)\big{\rangle}ds$
$\displaystyle=\mathbb{E}\int_{0}^{T}\big{\langle}\hat{R}(s,\alpha(s))v(s),v(s)\big{\rangle}ds.$

Finally, (3.3) implies that

$J(0,0,i_{0};u(\cdot))\geqslant 0,\quad\forall u(\cdot)\in\mathcal{U}[0,T].$

Therefore, we conclude from the above equation that

$\hat{R}(s,\alpha(s))=R(s,\alpha(s))+D(s,\alpha(s))^{\top}P(s,\alpha(s))D(s,\alpha(s))\geqslant
0,\ \text{ a.e. on }[0,T],\text{ a.s.}.$ (4.21)

Step 2. Now we prove that without the additional condition (4.16), the above
result (4.21) still holds. The key method is to apply a localization technique
so that the preceding argument can be applied to a certain stopped SLQ
problem. In detail, for each $k\geqslant 1$, we define the following stopping
time (with the convention $\inf\varnothing=\infty)$:

$\tau_{k}=\inf\Big{\\{}t\in[0,T];\int_{0}^{t}\big{[}|\Theta(s,\alpha(s))|^{2}+|\Lambda(s)|^{2}\big{]}ds\geqslant
k\Big{\\}}\wedge T.$

Take an arbitrary control $v(\cdot)\in\mathcal{U}[0,T]$ and consider the state
equation (4.17) over the interval $\left[0,\tau_{k}\right]$. By the definition
of $\tau_{k}$, we have

$\int_{0}^{\tau_{k}}\left[|\Theta(s,\alpha(s))|^{2}+|\Lambda(s)|^{2}\right]ds\leqslant
k,$

which implies that $X^{v}(\cdot)$, solution of (4.17) over $[0,\tau_{k}]$,
belongs to
$L_{\mathbb{F}}^{2}\left(\Omega;C\left(\left[0,\tau_{k}\right];\mathbb{R}^{n}\right)\right)$.
Hence,

$u(s)\triangleq\Theta(s,\alpha(s))X^{v}(s)+v(s)\in\mathcal{U}[0,\tau_{k}].$

Then we can proceed as in Step 1 to get that

$\displaystyle\displaystyle
J(0,0,i_{0};u(\cdot))=\mathbb{E}\int_{0}^{\tau_{k}}\big{\langle}\big{[}R(s,\alpha(s))+D(s,\alpha(s))^{\top}P(s,\alpha(s))D(s,\alpha(s))\big{]}v(s),v(s)\big{\rangle}ds.$

By the proof of the assertion (i) of 3.13, we have that for any choosing
$i_{0}\in{\cal S}$,

$J(0,0,i_{0};u(\cdot))\geqslant 0,\quad\forall
u(\cdot)\in\mathcal{U}[0,\tau_{k}],$

and note that $v(\cdot)\in\mathcal{U}[0,T]$ is arbitrary, so we have

$\hat{R}(s,\alpha(s))=R(s,\alpha(s))+D(s,\alpha(s))^{\top}P(s,\alpha(s))D(s,\alpha(s))\geqslant
0,\quad\text{a.e. on }\left[0,\tau_{k}\right],\text{ a.s..}$ (4.22)

Due to that the process $\boldsymbol{X}^{-1}(\cdot)$ is continuous,
$\boldsymbol{U}(\cdot)$ and $\boldsymbol{Z}(\cdot)$ are square-integrable, and
the processes $P(\cdot,\alpha(\cdot))$, $C(\cdot)$ and $D(\cdot)$ are bounded,
from (4.12) we have

$\int_{0}^{T}\left[|\Theta(s,\alpha(s))|^{2}+|\Lambda(s)|^{2}\right]ds<\infty,\
\text{ a.s.}.$

This implies that $\lim_{k\rightarrow\infty}\tau_{k}=T$ almost surely. Then
result (4.21) still holds by letting $k\rightarrow\infty$ in (4.22).

Step 3. In order to obtain the stronger property (4.15), we take an arbitrary
but fixed $\epsilon\in(0,\varepsilon)$ and consider the following stochastic
LQ problem of minimizing

$\displaystyle J_{\epsilon}(t,\xi,\vartheta;u(\cdot))=$
$\displaystyle\mathbb{E}_{t}\bigg{[}\langle G(\alpha(T)X(T),X(T)\rangle$
$\displaystyle+\int_{t}^{T}\left\langle\left(\begin{array}[]{cc}Q(s,\alpha(s))&S(s,\alpha(s))^{\top}\\\
S(s,\alpha(s))&R(s,\alpha(s))-\epsilon
I_{m}\end{array}\right)\left(\begin{array}[]{c}X(s)\\\
u(s)\end{array}\right),\left(\begin{array}[]{c}X(s)\\\
u(s)\end{array}\right)\right\rangle ds\bigg{]},$

where $X(\cdot)$ is the solution of state equation (2.1). Clearly, with
$\varepsilon$ replaced by $\varepsilon-\epsilon$, the conditions of Theorem
3.1 still hold for the new cost functional
$J_{\epsilon}(t,\xi,\vartheta;u(\cdot))$. Thus, there is a process
$P_{\epsilon}(\cdot)$ such that

$V_{\epsilon}(t,\xi,\vartheta)\triangleq\inf_{u(\cdot)\in\mathcal{U}[t,T]}J_{{}_{\epsilon}}(t,\xi,\vartheta;u(\cdot))=\left\langle
P_{\epsilon}(t,\vartheta)\xi,\xi\right\rangle,\quad\forall(t,\xi,\vartheta)\in[0,T]\times
L_{\mathscr{F}_{t}}^{\infty}(\Omega;\mathbb{R}^{n})\times
L_{\mathscr{F}_{t}^{\alpha}}^{2}(\Omega;{\cal S}).$

Then, by the previous discussion, we have

$R(s,\alpha(s))-\epsilon
I_{m}+D(s,\alpha(s))^{\top}P_{\epsilon}(s,\alpha(s))D(s,\alpha(s))\geqslant
0,\quad\text{a.e. on }[0,T],\text{ a.s.}.$

Now, by the definition of $J_{\epsilon}(t,\xi,\vartheta;u(\cdot))$, we deduce
that

$\displaystyle\displaystyle V(t,\xi,\vartheta)$
$\displaystyle=\inf_{u(\cdot)\in\mathcal{U}[t,T]}J(t,\xi,\vartheta;u(\cdot))$
$\displaystyle\geqslant\inf_{u(\cdot)\in\mathcal{U}[t,T]}J_{\epsilon}(t,\xi,\vartheta;u(\cdot))=V_{\epsilon}(t,\xi,\vartheta),\quad\forall(t,\xi,\vartheta)\in[0,T]\times
L_{\mathscr{F}_{t}}^{\infty}\left(\Omega;\mathbb{R}^{n}\right)\times
L_{\mathscr{F}_{t}^{\alpha}}^{2}(\Omega;{\cal S}),$

from which we see that

$P(t,\vartheta)\geqslant P_{\epsilon}(t,\vartheta),\quad\forall t\in[0,T],\
\vartheta\in L_{\mathscr{F}_{t}^{\alpha}}^{2}(\Omega;{\cal S}),$

and therefore

$\displaystyle\hat{R}(s,\alpha(s))\geqslant
R(s,\alpha(s))+D(s,\alpha(s))^{\top}P_{\epsilon}(s,\alpha(s))D(s,\alpha(s))\geqslant\varepsilon
I_{m},\ \text{ a.e. on }[0,T],\text{ a.s.. }$

Finally, note that $\epsilon\in(0,\varepsilon)$ is arbitrary, so property
(4.15) holds. This completes the proof. ∎

Based on the above preparations, we now can prove Theorem 3.1 and Theorem 3.2.

###### Proof of Theorem 3.1.

On one hand, from 4.3, we see that the bounded process
$P(\cdot,\alpha(\cdot))$ in (3.13) and the processes $\Lambda(\cdot)$ and
$\zeta(\cdot)$ defined by (4.12) satisfy BSDE (4.13) and relation (4.14). On
the other hand, 4.4 implies that

$\hat{R}(s,\alpha(s))=R(s,\alpha(s))+D(s,\alpha(s))^{\top}P(s,\alpha(s))D(s,\alpha(s))\geqslant\varepsilon
I_{m},\quad\hbox{a.e. on $[0,T]$, a.s.,}$

which, together with relation (4.14), deduces that

$\displaystyle\Theta(s,\alpha(s))$
$\displaystyle=-\hat{R}(s,\alpha(s))^{-1}\hat{S}(s,\alpha(s)),\quad\text{a.e.
on }[0,T],\text{ a.s..}$ (4.23)

Substituting (4.23) into BSDE (4.13) yields that

$\left\\{\begin{aligned} \displaystyle dP(s,\alpha(s))=&\
[\hat{\Psi}(s,\alpha(s))-\hat{Q}(s,\alpha(s))]ds+\Lambda(s)dW(s)+\zeta(s)\bullet
d\widetilde{N}(s),\quad s\in[0,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle P(T,\alpha(T))=&\
G(\alpha(T)),\quad\alpha(0)=i_{0},\end{aligned}\right.$ (4.24)

where $\hat{Q}(\cdot)$ is defined by (3.2), and

$\hat{\Psi}(s,\alpha(s))\triangleq\hat{S}(s,\alpha(s))^{\top}\hat{R}(s,\alpha(s))^{-1}\hat{S}(s,\alpha(s)),\quad
s\in[0,T].$

Then SRE (3.1) follows easily from (4.24), and
$(\hat{P}(\cdot,\alpha(\cdot)),\hat{\Lambda}(\cdot),\hat{\zeta}(\cdot))$,
solution of SRE (3.1), coincide with
$(P(\cdot,\alpha(\cdot)),\Lambda(\cdot),\zeta(\cdot))$, solution of BSDE
(4.24), i.e.,

$\hat{P}(s,\alpha(s))=P(s,\alpha(s)),\quad\hat{\Lambda}(s)=\Lambda(s),\quad\hat{\zeta}(s)=\zeta(s),\quad
s\in[0,T].$ (4.25)

In the following, we still adopt
$(P(\cdot,\alpha(\cdot)),\Lambda(\cdot),\zeta(\cdot))$ in order to keep the
consistency of symbols.

It remains to prove that the processes $\Lambda(\cdot)$ and $\zeta(\cdot)$ are
square-integrable. Note that in BSDE (4.24), the matrix-valued processes
$A(\cdot)$, $C(\cdot)$, $Q(\cdot)$ and $P(\cdot,\alpha(\cdot))$ are all
bounded and the process $\hat{\Psi}(\cdot)$ is positive semi-definite, so we
can choose a positive constant $K$ such that

$\left\\{\begin{aligned}
\displaystyle&\operatorname{tr}[P(s,\alpha(s))]+|P(s,\alpha(s))|^{2}\leqslant
K,\qquad\operatorname{tr}[\hat{Q}(s,\alpha(s))]\leqslant K[1+|\Lambda(s)|],\\\
\vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&\operatorname{tr}[P(s,\alpha(s))\hat{Q}(s,\alpha(s))]\leqslant|P(s,\alpha(s))|\cdot|\hat{Q}(s,\alpha(s))|\leqslant
K[1+|\Lambda(s)|],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&\operatorname{tr}[-P(s,\alpha(s))\hat{\Psi}(s,\alpha(s))]\leqslant\lambda_{\max}[-P(s,\alpha(s))]\operatorname{tr}[\hat{\Psi}(s,\alpha(s))]\leqslant
K\operatorname{tr}[\hat{\Psi}(s,\alpha(s))],\end{aligned}\right.$ (4.26)

for Lebesgue-almost every $s,\mathbb{P}$-a.s.. In the last inequality, we
adopt Theorem 7.4.1.1 of Horn–Johnson [6]. In the following, we denote by the
same letter $K$ a generic positive constant whose value may be different from
line to line. Define for each $m\geqslant 1$ the stopping time (with the
convention $\inf\varnothing=\infty)$

$\lambda_{m}=\inf\Big{\\{}t\in[0,T];\int_{0}^{t}\big{[}|\Lambda(s)|^{2}+\sum_{k,l=1}^{D}|\zeta_{kl}(s)|^{2}\lambda_{kl}(s)I_{\\{\alpha(s-)=k\\}}\big{]}ds\geqslant
m\Big{\\}},$ (4.27)

which implies that $\lim_{m\rightarrow\infty}\lambda_{m}=\infty$ almost
surely. Then we have

$\displaystyle P\left(t\wedge\lambda_{m},\alpha(t\wedge\lambda_{m})\right)=$
$\displaystyle\
P(0,i_{0})+\int_{0}^{t\wedge\lambda_{m}}[\hat{\Psi}(s,\alpha(s))-\hat{Q}(s,\alpha(s))]ds$
(4.28)
$\displaystyle+\int_{0}^{t\wedge\lambda_{m}}\Lambda(s)dW(s)+\int_{0}^{t\wedge\lambda_{m}}\zeta(s)\bullet
d\widetilde{N}(s).$

From the definition of $\lambda_{m}$, it is easy to see that the following
processes

$\displaystyle\Big{\\{}\int_{0}^{t\wedge\lambda_{m}}\Lambda(s)dW(s)\Big{\\}}_{t\in[0,T]}$
$\displaystyle=\
\Big{\\{}\int_{0}^{t}\Lambda(s)\mathbf{1}_{\left\\{s\leqslant\lambda_{m}\right\\}}dW(s)\Big{\\}}_{t\in[0,T]},$
$\displaystyle\Big{\\{}\int_{0}^{t\wedge\lambda_{m}}\zeta(s)\bullet
d\widetilde{N}(s)\Big{\\}}_{t\in[0,T]}$ $\displaystyle=\
\Big{\\{}\int_{0}^{t}\zeta(s)\mathbf{1}_{\left\\{s\leqslant\lambda_{m}\right\\}}\bullet
d\widetilde{N}(s)\Big{\\}}_{t\in[0,T]},$

are matrix of square-integrable martingales w.r.t. the filtration
$\mathbb{F}=\\{\mathscr{F}_{t};0\leqslant t<\infty\\}$ and
$\mathbb{F}^{\alpha}=\\{\mathscr{F}^{\alpha}_{t};0\leqslant t<\infty\\}$,
respectively. Therefore, taking expectations on both side of (4.28), we have

$\displaystyle\displaystyle\mathbb{E}\big{[}P\left(t\wedge\lambda_{m},\alpha(t\wedge\lambda_{m})\right)\big{]}=P(0,i_{0})+\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}\big{[}\hat{\Psi}(s,\alpha(s))-\hat{Q}(s,\alpha(s))\big{]}ds.$

Thus, combining with (4.26), we see that

$\displaystyle\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}\operatorname{tr}[\hat{\Psi}(s,\alpha(s))]ds$
$\displaystyle=\mathbb{E}\operatorname{tr}\big{[}P\left(t\wedge\lambda_{m},\alpha(t\wedge\lambda_{m})\right)-P(0,i_{0})\big{]}+\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}\operatorname{tr}[\hat{Q}(s,\alpha(s))]ds$
$\displaystyle\leqslant
K\Big{[}1+\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}|\Lambda(s)|ds\Big{]}.$
(4.29)

On the other hand, for BSDE (4.24), applying Itô’s formula to
$P(s,\alpha(s))^{2}$ and denoting $\zeta^{2}(s)\triangleq(\zeta_{kl}(s)^{2})$,
we have

$\displaystyle d[P(s,\alpha(s))]^{2}=$
$\displaystyle\Big{\\{}P(s,\alpha(s))\big{[}\hat{\Psi}(s,\alpha(s))-\hat{Q}(s,\alpha(s))\big{]}+\big{[}\hat{\Psi}(s,\alpha(s))-\hat{Q}(s,\alpha(s))\big{]}P(s,\alpha(s))$
$\displaystyle+\Lambda(s)^{2}+\sum_{k,l=1}^{D}\zeta^{2}_{kl}(s)\lambda_{kl}(s)I_{\\{\alpha(s-)=k\\}}\Big{\\}}ds+\big{[}P(s,\alpha(s))\Lambda(s)+\Lambda(s)P(s,\alpha(s))\big{]}dW(t)$
$\displaystyle+\sum_{k,l=1}^{D}2P(s-)\zeta_{kl}(s)d\widetilde{N}_{kl}(s)+\zeta^{2}(s)\bullet
d\widetilde{N}(s).$

Now, note (4.27), a similar argument shows that

$\displaystyle\mathbb{E}[P(s,\alpha(s))]^{2}=$ $\displaystyle
P(0,i_{0})+\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}\Big{\\{}P(s,\alpha(s))\big{[}\hat{\Psi}(s,\alpha(s))-\hat{Q}(s,\alpha(s))\big{]}$
$\displaystyle+\big{[}\hat{\Psi}(s,\alpha(s))-\hat{Q}(s,\alpha(s))\big{]}P(s,\alpha(s))+\Lambda(s)^{2}+\sum_{k,l=1}^{D}\zeta_{kl}(s)^{2}\lambda_{kl}(s)I_{\\{\alpha(s-)=k\\}}\Big{\\}}ds.$

Combining with (4.26)-(4) and recalling the Frobenius norm, we have

$\displaystyle\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}|\Lambda(s)|^{2}ds+\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}\sum_{k,l=1}^{D}|\zeta_{kl}(s)|^{2}\lambda_{kl}(s)I_{\\{\alpha(s-)=k\\}}ds$
$\displaystyle=\,$
$\displaystyle\operatorname{tr}\bigg{[}\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}[\Lambda(s)]^{2}ds+\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}\sum_{k,l=1}^{D}[\zeta_{kl}(s)]^{2}\lambda_{kl}(s)I_{\\{\alpha(s-)=k\\}}ds\bigg{]}$
$\displaystyle=\,$
$\displaystyle\mathbb{E}\left|P\left(t\wedge\lambda_{m},\alpha(t\wedge\lambda_{m})\right)\right|^{2}-|P(0,i_{0})|^{2}+2\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}\operatorname{tr}[P(s,\alpha(s))\hat{Q}(s,\alpha(s))]ds$
$\displaystyle+2\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}\operatorname{tr}[-P(s,\alpha(s))\hat{\Psi}(s,\alpha(s))]ds$
$\displaystyle\leqslant\,$ $\displaystyle
K+K\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}[1+|\Lambda(s)|]ds+K\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}\operatorname{tr}[\hat{\Psi}(s,\alpha(s))]ds$
$\displaystyle\leqslant\,$ $\displaystyle
K+K\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}|\Lambda(s)|ds\leqslant
K+2K^{2}+\frac{1}{2}\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}|\Lambda(s)|^{2}ds,$

where in the last inequality we employ the Cauchy-Schwarz’s inequality. Hence,
we obtain that

$\displaystyle\frac{1}{2}\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}|\Lambda(s)|^{2}ds+\mathbb{E}\int_{0}^{t\wedge\lambda_{m}}\sum_{k,l=1}^{D}|\zeta_{kl}(s)|^{2}\lambda_{kl}(s)I_{\\{\alpha(s-)=k\\}}ds\leqslant
K+2K^{2}.$

Due to that $\lim_{m\rightarrow\infty}\lambda_{m}=\infty$ almost surely and
$K$ does not depend on $m$ and $t$, we conclude that the processes
$\Lambda(\cdot)$ and $\zeta(\cdot)$ are square-integrable by letting
$m\rightarrow\infty$ firstly and then $t\uparrow T$. ∎

###### Proof of Theorem 3.2.

Note that 3.9 and 3.10 imply that Problem (M-SLQ) is uniquely open-loop
solvable at any initial time $t<T$, and the open-loop optimal control
$u^{*}(\cdot)$ w.r.t. $(t,\xi,\vartheta)$ is given by

$u^{*}(s)=\left(u_{1}(s),\cdots,u_{n}(s)\right)\xi,\quad s\in[t,T].$

Therefore, for the sake of the optimal control w.r.t. any initial triple
$(t,\xi,\vartheta)\in\mathcal{D}$, it is sufficient to determine the open-loop
optimal control $u_{j}(\cdot)=\left\\{u_{j}(s);s\in[0,T]\right\\}$ w.r.t.
$\left(0,e_{j},i_{0}\right)$ for each $j=1,\ldots,n$. From 4.1, we see that
the process $\boldsymbol{X}(\cdot)=\\{\boldsymbol{X}(s);0\leqslant s\leqslant
T\\}$ is invertible. On the other hand, 4.3 tells us that finding the open-
loop optimal controls $u_{1},\cdots,u_{n}$ are equivalent to finding

$\Theta(s,\alpha(s))=\boldsymbol{U}(s)\boldsymbol{X}(s)^{-1},\quad s\in[0,T].$

The latter can be accomplished by solving SRE (3.1), whose solvability is
obtained by Theorem 3.1. In fact, from the proof of Theorem 3.1, we see that
$\Theta(\cdot)$ is actually determined by (4.23). Summarizing these
observations, we conclude the closed-loop representation (3.5). ∎

##  5 Proof of the auxiliary results

In this section, we prove the results of Subsection 3.2. Inspired by Mou–Yong
[17] and Chen–Yong [4], we present a representation of the cost functional,
which is characterized as a bilinear form on a Hilbert space in terms of the
adapted solutions of some forward-backward stochastic differential equations,
using the technique of Itô’s formula with jumps. Let us present a simple lemma
first.

######  Lemma 5.1.

Under (H), for any initial triple $(t,\xi,\vartheta)\in{\cal D}$ and a control
$u(\cdot)\in{\cal U}[t,T]$, we have

$\displaystyle{J}(t,\xi,\vartheta;u(\cdot))=$ $\displaystyle\langle
Y(t),\xi\rangle+\mathbb{E}\Big{[}\int_{t}^{T}\big{\langle}F(s,\alpha(s),X(s),Y(s),Z(s),u(s)),u(s)\big{\rangle}ds\Big{|}\mathscr{F}_{t}\Big{]},$
(5.1)

where the quadruple $(X(\cdot),Y(\cdot),Z(\cdot),\Gamma(\cdot))$ is the
solution of SDE (2.1) and BSDE (3.6).

###### Proof.

The proof is trivial by applying Itô’s formula to $\langle
Y(\cdot),X(\cdot)\rangle$ on $[t,T]$. In fact, using Itô’s formula to
$s\mapsto\langle Y(s),X(s)\rangle$ and then taking the conditional expectation
yields that

$\begin{array}[]{ll}\displaystyle\mathbb{E}\big{[}\langle
G(\alpha(T))X(T),X(T)\rangle|\mathscr{F}_{t}\big{]}=\langle
Y(t),\xi\rangle+\mathbb{E}\Big{[}\int_{t}^{T}\big{[}\big{\langle}B(s,\alpha(s))^{\top}Y(s)+D(s,\alpha(s))^{\top}Z(s)\\\
\vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle\qquad\qquad\qquad\hskip
79.6678pt-S(s,\alpha(s))X(s),u(s)\big{\rangle}-\big{\langle}Q(s,\alpha(s))X(s),X(s)\big{\rangle}\big{]}ds\Big{|}\mathscr{F}_{t}\Big{]}.\end{array}$
(5.2)

Then substituting (5.2) into the cost functional
${J}(t,\xi,\vartheta;u(\cdot))$ leads to (5.1). ∎

Next, we let
$(\tilde{X}(\cdot),\tilde{Y}(\cdot),\tilde{Z}(\cdot),\tilde{\Gamma}(\cdot))$
and $(\bar{X}(\cdot),\bar{Y}(\cdot),\bar{Z}(\cdot),\bar{\Gamma}(\cdot))$ be
the adapted solutions of the following decoupled FBSDEs, respectively,

$\displaystyle\left\\{\begin{array}[]{ll}\displaystyle
d\tilde{X}(s)=\big{[}A(s,\alpha(s))\tilde{X}(s)+B(s,\alpha(s))u(s)\big{]}ds+\big{[}C(s,\alpha(s))\tilde{X}(s)+D(s,\alpha(s))u(s)\big{]}dW(s),\\\
\vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
d\tilde{Y}(s)=-\widetilde{F}(s,\alpha(s),\tilde{X}(s),\tilde{Y}(s),\tilde{Z}(s),u(s))ds+\tilde{Z}(s)dW(s)+\tilde{\Gamma}(s)\bullet
d\widetilde{N}(s),\quad s\in[t,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle\tilde{X}(t)=0,\quad\alpha(t)=\vartheta,\quad\tilde{Y}(T)=G(\alpha(T))\tilde{X}(T),\end{array}\right.$
(5.6) $\displaystyle\left\\{\begin{array}[]{ll}\displaystyle
d\bar{X}(s)=A(s,\alpha(s))\bar{X}(s)ds+C(s,\alpha(s))\bar{X}(s)dW(s),\quad
s\in[t,T],\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
d\bar{Y}(s)=-\widetilde{F}_{0}(s,\alpha(s),\bar{X}(s),\bar{Y}(s),\bar{Z}(s))ds+\bar{Z}(s)dW(s)+\bar{\Gamma}(s)\bullet
d\widetilde{N}(s),\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle\bar{X}(t)=\xi,\quad\alpha(t)=\vartheta,\quad\bar{Y}(T)=G(\alpha(T))\bar{X}(T).\end{array}\right.$
(5.10)

Then the adapted solution $(X(\cdot),Y(\cdot),Z(\cdot),\Gamma(\cdot))$ of SDE
(2.1) and BSDE (3.6) could be written as the sum of
$(\tilde{X}(\cdot),\tilde{Y}(\cdot),\tilde{Z}(\cdot),\tilde{\Gamma}(\cdot))$
and $(\bar{X}(\cdot),\bar{Y}(\cdot),\bar{Z}(\cdot),\bar{\Gamma}(\cdot))$,
i.e.,

$X(s)=\tilde{X}(s)+\bar{X}(s),\quad~{}Y(s)=\tilde{Y}(s)+\bar{Y}(s),\quad~{}Z(s)=\tilde{Z}(s)+\bar{Z}(s),\quad~{}\Gamma(s)=\tilde{\Gamma}(s)+\bar{\Gamma}(s),\quad
s\in[t,T].$

In what follows, for any $u(\cdot),v(\cdot)\in{\cal U}[t,T]$, we set

$\displaystyle[[u,v]]_{t}=\mathbb{E}_{t}\int_{t}^{T}\langle u(s),v(s)\rangle
ds,\quad[[u,v]]=\mathbb{E}\int_{t}^{T}\langle u(s),v(s)\rangle ds.$

Now, for any $u(\cdot)\in{\cal U}[t,T]$ and $\xi\in{\cal X}_{t}\triangleq
L^{2}_{\mathscr{F}_{t}}(\Omega;\mathbb{R}^{n})$, we define two linear
operators ${\cal N}_{t}$ and ${\cal L}_{t}$ as follows: for $s\in[t,T]$,

$\displaystyle[{\cal
N}_{t}u](s,\alpha(s))=F(s,\alpha(s),\tilde{X}(s),\tilde{Y}(s),\tilde{Z}(s),u(s)),\quad[{\cal
L}_{t}\xi](s,\alpha(s))=F_{0}(s,\alpha(s),\bar{X}(s),\bar{Y}(s),\bar{Z}(s)).$
(5.11)

Then, for these two operators, we have the following result.

######  Lemma 5.2.

Under the condition (H), the following two assetions hold:

  * (i)

On the Hilbert space, the linear operator ${\cal N}_{t}$ defined in (5.11) is
a bounded self-adjoint operator.

  * (ii)

From the Hilbert space ${\cal X}_{t}$ into the Hilbert space ${\cal U}[t,T]$,
the linear operator ${\cal L}_{t}$ defined in (5.11) is a bounded operator.
Moreover, there exists a positive constant $K$, independent of
$(t,\xi,\vartheta)$, such that

$[[{\cal L}_{t}\xi,{\cal L}_{t}\xi]]\leqslant
K\mathbb{E}|\xi|^{2},\quad\forall\xi\in{\cal X}_{t}.$ (5.12)

###### Proof.

(i) Using the same argument as in Proposition 2.1 of Sun–Yong [21], it is easy
to obtain the boundedness of the operator ${\cal N}_{t}$. In order to show
that ${\cal N}_{t}$ is self-adjoint, it suffices to prove that for any
$u_{1}(\cdot),u_{2}(\cdot)\in{\cal U}[t,T]$, the following relation holds:

$\displaystyle\mathbb{E}\int_{t}^{T}\big{\langle}[{\cal
N}_{t}u_{1}](s,\alpha(s)),u_{2}(s)\big{\rangle}ds=\mathbb{E}\int_{t}^{T}\big{\langle}u_{1}(s),[{\cal
N}_{t}u_{2}](s,\alpha(s))\big{\rangle}ds.$ (5.13)

For this, we let
$(\tilde{X}_{j}(\cdot),\tilde{Y}_{j}(\cdot),\tilde{Z}_{j}(\cdot),\tilde{\Gamma}_{j}(\cdot))$
with $j=1$, $2$ be the adapted solution of FBSDE (5.6) with $u(\cdot)$ is
replaced by $u_{j}(\cdot)$. On one hand, applying Itô’s formula to
$\langle\tilde{Y}_{2}(s),\tilde{X}_{1}(s)\rangle$ first and then taking the
expectation, we have

$\displaystyle\mathbb{E}\big{\langle}G(\alpha(T))\tilde{X}_{2}(T),\tilde{X}_{1}(T)\big{\rangle}=$
$\displaystyle\
\mathbb{E}\int_{t}^{T}\big{[}\big{\langle}B(s,\alpha(s))^{\top}\tilde{Y}_{2}(s)+D(s,\alpha(s))^{\top}\tilde{Z}_{2}(s),u_{1}(s)\big{\rangle}$
$\displaystyle-\big{\langle}Q(s,\alpha(s))\tilde{X}_{2}(s),\tilde{X}_{1}(s)\big{\rangle}-\big{\langle}S(s,\alpha(s))\tilde{X}_{1}(s),u_{2}(s)\big{\rangle}\big{]}ds.$

On the other hand, applying Itô’s formula to
$\langle\tilde{Y}_{1}(s),\tilde{X}_{2}(s)\rangle$ first and then taking the
expectation lead to

$\displaystyle\mathbb{E}\big{\langle}G(\alpha(T))\tilde{X}_{1}(T),\tilde{X}_{2}(T)\big{\rangle}=$
$\displaystyle\
\mathbb{E}\int_{t}^{T}\big{[}\big{\langle}B(s,\alpha(s))^{\top}\tilde{Y}_{1}(s)+D(s,\alpha(s))^{\top}\tilde{Z}_{1}(s),u_{2}(s)\big{\rangle}$
$\displaystyle-\big{\langle}Q(s,\alpha(s))\tilde{X}_{1}(s),\tilde{X}_{2}(s)\big{\rangle}-\big{\langle}S(s,\alpha(s))\tilde{X}_{2}(s),u_{1}(s)\big{\rangle}\big{]}ds.$

Combining the above two equations and noting that $G(\cdot)$ and $Q(\cdot)$
are symmetric, we have

$\displaystyle\mathbb{E}\int_{t}^{T}\big{\langle}[{\cal
N}_{t}u_{1}](s,\alpha(s))-R(s,\alpha(s))u_{1}(s),u_{2}(s)\big{\rangle}ds=\mathbb{E}\int_{t}^{T}\big{\langle}u_{1}(s),[{\cal
N}_{t}u_{2}](s,\alpha(s))-R(s,\alpha(s))u_{2}(s)\big{\rangle}ds.$

Thus, from the fact that $R(\cdot)$ is also symmetric, we can easily obtain
(5.13).

(ii) It suffices to prove (5.12). Choose a positive constant $\rho_{1}$ such
that

$|G(\alpha(T))|^{2},|B(s,\alpha(s))|^{2},|D(s,\alpha(s))|^{2},|S(s,\alpha(s))|^{2},|Q(s,\alpha(s))|^{2}\leqslant\rho_{1},\quad\hbox{a.e.},s\in[0,T],\hbox{
a.s.}.$ (5.14)

Then using the vector inequality
$|\beta_{1}+\cdot\cdot\cdot+\beta_{n}|\leqslant
n(|\beta_{1}|^{2}+\cdot\cdot\cdot+\beta_{n}|^{2})$, we have

$\displaystyle\ [[{\cal L}_{t}\xi,{\cal
L}_{t}\xi]]=\mathbb{E}\int_{t}^{T}\big{|}F_{0}(s,\alpha(s),\bar{X}(s),\bar{Y}(s),\bar{Z}(s))\big{|}^{2}ds\leqslant\
3\rho_{1}\mathbb{E}\int_{t}^{T}\Big{[}|\bar{Y}(s)|^{2}+|\bar{Z}(s)|^{2}|+\bar{X}(s)|^{2}\Big{]}ds.$

By the classical theory of SDEs and BSDEs (see also Proposition 2.1 of
Sun–Yong [21]), we have that for the decoupled FBSDE (5.10), there exists a
positive constant $\rho_{2}$, independent of $(t,\xi,\vartheta)$, such that

$\displaystyle\mathbb{E}\int_{t}^{T}\big{[}|\bar{Y}(s)|^{2}+|\bar{Z}(s)|^{2}\big{]}ds$
$\displaystyle\leqslant\rho_{2}\mathbb{E}\Big{[}|G(\alpha(T))\bar{X}(T)|^{2}+\int_{t}^{T}|Q(s,\alpha(s))\bar{X}(s)|^{2}ds\Big{]},$
$\displaystyle\mathbb{E}|\bar{X}(T)|^{2}+\mathbb{E}\int_{t}^{T}|\bar{X}(s)|^{2}ds$
$\displaystyle\leqslant\rho_{2}\mathbb{E}|\xi|^{2}.$ (5.15)

Finally, combining (5.14) and (5.15), we deduce that $[[{\cal L}_{t}\xi,{\cal
L}_{t}\xi]]\leqslant
3\rho_{1}(\rho_{1}\rho_{2}^{2}+\rho_{2})\mathbb{E}|\xi|^{2}$ for all
$\xi\in{\cal X}_{t}$. ∎

######  Remark 5.3.

Denote by the quadruple $({\cal X}(\cdot),{\cal Y}(\cdot),{\cal
Z}(\cdot),\breve{\Gamma}(\cdot))$ the adapted solution of the FBSDE:

$\left\\{\begin{array}[]{ll}\displaystyle d{\cal X}(s)=A(s,\alpha(s)){\cal
X}(s)ds+C(s,\alpha(s)){\cal X}(s)dW(s),\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle d{\cal Y}(s)=-\widetilde{F}_{0}(s,\alpha(s),{\cal
X}(s),{\cal Y}(s),{\cal Z}(s))ds+{\cal Z}(s)dW(s)+\breve{\Gamma}(s)\bullet
d\widetilde{N}(s),\quad s\in[0,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle{\cal X}(0)=I_{n},\quad\alpha(0)=i_{0},\quad{\cal
Y}(T)=G(\alpha(T)){\cal X}(T).\end{array}\right.$

It is easy to check that the process ${\cal X}(\cdot)$ is invertible (denote
by ${\cal X}^{-1}(\cdot)$) with ${\cal X}^{-1}(\cdot)$ satisfying the
following equation:

$\left\\{\begin{array}[]{ll}\displaystyle d{\cal X}^{-1}(s)={\cal
X}^{-1}(s)\big{[}C(s,\alpha(s))^{2}-A(s,\alpha(s))\big{]}ds-{\cal
X}^{-1}(s)C(s,\alpha(s))dW(s),\quad s\in[0,T],\\\ \vskip 3.0pt plus 1.0pt
minus 1.0pt\cr\displaystyle{\cal
X}^{-1}(0)=I_{n},\quad\alpha(0)=i_{0}.\end{array}\right.$

For every $\xi\in L^{\infty}_{\mathscr{F}_{t}}(\Omega;\mathbb{R}^{n})$ and
$\eta(s)\triangleq(\eta_{kl}(s))\in{\cal M}_{D}(\mathbb{R}^{n\times n})$, we
set $\eta(s)\circ\xi\triangleq(\eta_{kl}(s)\xi)$. Now, on the other hand, the
following processes

$\displaystyle{\cal X}(s){\cal X}^{-1}(t)\xi,\quad~{}{\cal Y}(s){\cal
X}^{-1}(t)\xi,\quad~{}{\cal Z}(s){\cal
X}^{-1}(t)\xi,\quad~{}\breve{\Gamma}(s)\circ{\cal
X}^{-1}(t)\xi,\quad~{}s\in[t,T],$

are all square-integrable and satisfy FBSDE (5.10). Therefore, by the
uniqueness of the adapted solutions, one has that for $s\in[t,T]$,

$\displaystyle\bar{X}(s)={\cal X}(s){\cal X}^{-1}(t)\xi,\quad\bar{Y}(s)={\cal
Y}(s){\cal X}^{-1}(t)\xi,\quad\bar{Z}(s)={\cal Z}(s){\cal
X}^{-1}(t)\xi,\quad\bar{\Gamma}(s)=\breve{\Gamma}(s)\circ{\cal X}^{-1}(t)\xi.$

Hence, for every $\xi\in L^{\infty}_{\mathscr{F}_{t}}(\Omega;\mathbb{R}^{n})$,
the operator ${\cal L}_{t}\xi$ could be represented in terms of the quadruple
$({\cal X}(\cdot),{\cal Y}(\cdot),{\cal Z}(\cdot),\breve{\Gamma}(\cdot))$ as
follows,

$[{\cal L}_{t}\xi](s,\alpha(s))=F_{0}(s,\alpha(s),{\cal X}(s),{\cal
Y}(s),{\cal Z}(s)){\cal X}^{-1}(t)\xi,\quad s\in[t,T].$ (5.16)

This relation is useful in proving Theorem 3.12.

Now, we are ready to show the representation to the cost functional
$J_{0}(t,\xi,\vartheta;u(\cdot))$. Note that $J_{0}(t,\xi,\vartheta;u(\cdot))$
and ${J}(t,\xi,\vartheta;u(\cdot))$ have the relation
$J_{0}(t,\xi,\vartheta;u(\cdot))=\mathbb{E}{J}(t,\xi,\vartheta;u(\cdot))$, and
from 2.3, the first component $M(\cdot)$ of BSDE (2.4) is bounded.

######  Proposition 5.4.

Let ${\cal N}_{t}$ and ${\cal L}_{t}$ be defined in (5.11) and $M(\cdot)$ be
the first component of the solution of BSDE (2.4). Then, under (H), the cost
functional $J(t,\xi,\vartheta;u(\cdot))$ has the following representation:

$J(t,\xi,\vartheta;u(\cdot))=[[{\cal N}_{t}u,u]]_{t}+2[[{\cal
L}_{t}\xi,u]]_{t}+\langle M(t)\xi,\xi\rangle,\ \hbox{ for all
}(t,\xi,\vartheta)\in{\cal D}.$ (5.17)

Furthermore, the cost functional $J_{0}(t,\xi,\vartheta;u(\cdot))$ has the
following representation:

$J_{0}(t,\xi,\vartheta;u(\cdot))=[[{\cal N}_{t}u,u]]+2[[{\cal
L}_{t}\xi,u]]+\mathbb{E}\langle M(t)\xi,\xi\rangle,\ \hbox{ for all
}(t,\xi,\vartheta)\in{\cal D}.$ (5.18)

###### Proof.

Fix any $(t,\xi,\vartheta)\in{\cal D}$ and $u(\cdot)\in{\cal U}[t,T]$. Let
$(\tilde{X}(\cdot),\tilde{Y}(\cdot),\tilde{Z}(\cdot),\tilde{\Gamma}(\cdot))$,
$(\bar{X}(\cdot),\bar{Y}(\cdot),\bar{Z}(\cdot),\bar{\Gamma}(\cdot))$, and
$(X(\cdot),Y(\cdot),Z(\cdot),\Gamma(\cdot))$ be the adapted solutions of FBSDE
(5.6), FBSDE (5.10), and SDE (2.1) and BSDE (3.6), respectively. Then

$X(s)=\tilde{X}(s)+\bar{X}(s),\quad~{}Y(s)=\tilde{Y}(s)+\bar{Y}(s),\quad~{}Z(s)=\tilde{Z}(s)+\bar{Z}(s),\quad~{}\Gamma(s)=\tilde{\Gamma}(s)+\bar{\Gamma}(s),\quad
s\in[t,T].$

By 5.1, the definitions of ${\cal N}_{t}$ and ${\cal L}_{t}$, we have

$\begin{array}[]{ll}\displaystyle
J(t,\xi,\vartheta;u(\cdot))=\langle\tilde{Y}(t),\xi\rangle+\langle\bar{Y}(t),\xi\rangle+\mathbb{E}_{t}\int_{t}^{T}\big{\langle}[{\cal
N}_{t}u](s)+[{\cal L}_{t}\xi](s),u(s)\big{\rangle}ds.\end{array}$ (5.19)

On one hand, applying Itô’s formula to $\langle\tilde{Y}(s),\bar{X}(s)\rangle$
on $[t,T]$ implies that

$\displaystyle\mathbb{E}_{t}\langle
G(\alpha(T))\tilde{X}(T),\bar{X}(T)\rangle-\langle\tilde{Y}(t),\xi\rangle=-\mathbb{E}_{t}\int_{t}^{T}\Big{[}\langle
Q(s,\alpha(s))\tilde{X}(s),\bar{X}(s)\rangle+\langle
S(s,\alpha(s))\bar{X}(s),u(s)\rangle\Big{]}ds.$

On the other hand, applying Itô’s formula to
$\langle\bar{Y}(s),\tilde{X}(s)\rangle$ on $[t,T]$ gives that

$\displaystyle\mathbb{E}_{t}\langle
G(\alpha(T))\bar{X}(T),\tilde{X}(T)\rangle=\mathbb{E}_{t}\int_{t}^{T}\Big{[}\langle
B(s,\alpha(s))^{\top}\bar{Y}(s)+D(s,\alpha(s))^{\top}\bar{Z}(s),u(s)\rangle-\langle
Q(s,\alpha(s))\bar{X}(s),\tilde{X}(s)\rangle\Big{]}ds.$

So we have

$\displaystyle\langle\tilde{Y}(t),\xi\rangle=\mathbb{E}_{t}\int_{t}^{T}\Big{[}\langle
F_{0}(s,\alpha(s),\bar{X}(s),\bar{Y}(s),\bar{Z}(s)),u(s)\rangle\Big{]}ds=\mathbb{E}_{t}\int_{t}^{T}\langle[{\cal
L}_{t}\xi](s),u(s)\rangle ds.$

Moreover, applying Itô’s formula to $M(s)\bar{X}(s)$, we have

$\displaystyle d[M(s)\bar{X}(s)]$
$\displaystyle=-\Big{[}A(s,\alpha(s))^{\top}M(s)\bar{X}(s)+C(s,\alpha(s))^{\top}\big{(}M(s)C(s,\alpha(s))+\Phi(s)\big{)}\bar{X}(s)+Q(s,\alpha(s))\bar{X}(s)\Big{]}ds$
$\displaystyle\quad\
+\big{(}M(s)C(s,\alpha(s))+\Phi(s)\big{)}\bar{X}(s)dW(s)+(\eta(s)\circ\bar{X}(s))\bullet
d\widetilde{N}(s).$ (5.20)

Since $M(T)\bar{X}(T)=\bar{Y}(T)$, we see that the triple
$(M\bar{X},(MC+\Phi)\bar{X},\eta\circ\bar{X})$ satisfies the same BSDE as
$(\bar{Y}(\cdot),\bar{Z}(\cdot),\bar{\Gamma}(\cdot))$. Thus, by comparing
equations (5.10) and (5.20), we have

$\displaystyle\bar{Y}(s)=M(s)\bar{X}(s),\quad\bar{Z}(s)=\big{(}M(s)C(s,\alpha(s))+\Phi(s)\big{)}\bar{X}(s),\quad\bar{\Gamma}(s)=\eta(s)\circ\bar{X}(s),\quad
s\in[t,T].$

It follows that $\langle\bar{Y}(t),\xi\rangle=\langle M(t)\xi,\xi\rangle$.
Substituting this relation into (5.19) implies result (5.17). By taking
expectation to (5.17), we can easily obtain (5.18). ∎

######  Corollary 5.5.

Under the condition (H), the following two assertions hold:

  * (i)

A control $u^{*}(\cdot)\in{\cal U}[t,T]$ is optimal for Problem (M-SLQ)0
w.r.t. $(t,\xi,\vartheta)\in{\cal D}$ if and only if

$\displaystyle{\cal N}_{t}\geqslant 0,\quad\hbox{and}\quad{\cal
N}_{t}u^{*}+{\cal L}_{t}\xi=0.$ (5.21)

  * (ii)

If ${\cal N}_{t}$ is invertible and satisfies the positivity condition ${\cal
N}_{t}\geqslant 0$, then Problem (M-SLQ)0 is uniquely solvable at $t$, and the
unique open-loop optimal control $u^{*}(\cdot)$ w.r.t.
$(t,\xi,\vartheta)\in{\cal D}$ is given by

$\displaystyle u^{*}(s)=-[{\cal N}^{-1}_{t}{\cal L}_{t}\xi](s),\quad
s\in[t,T].$

###### Proof.

By 2.2, we see that $u^{*}(\cdot)\in{\cal U}[t,T]$ is optimal for Problem
(M-SLQ)0 w.r.t. the initial triple $(t,\xi,\vartheta)$ if and only if

$J_{0}(t,\xi,\vartheta;u^{*}(\cdot)+\lambda
v(\cdot))-J_{0}(t,\xi,\vartheta;u^{*}(\cdot))\geqslant 0,\quad\forall
v(\cdot)\in{\cal U}[t,T],\ \lambda\in\mathbb{R}.$ (5.22)

According to the representation (5.18), one has

$\displaystyle J_{0}(t,\xi,\vartheta;u^{*}(\cdot)+\lambda
v(\cdot))-J_{0}(t,\xi,\vartheta;u^{*}(\cdot))=$
$\displaystyle~{}\lambda^{2}[[{\cal N}_{t}v,v]]+2\lambda[[{\cal
N}_{t}u^{*},v]]+2\lambda[[{\cal L}_{t}\xi,v]].$

Hence, the equation (5.22) is equivalent to

$\displaystyle\lambda^{2}[[{\cal N}_{t}v,v]]+2\lambda[[{\cal N}_{t}u^{*}+{\cal
L}_{t}\xi,v]]\geqslant 0,\quad\forall v(\cdot)\in{\cal U}[t,T],\
\forall\lambda\in\mathbb{R}.$

Therefore, we must have

$\displaystyle[[{\cal N}_{t}v,v]]\geqslant 0\quad\hbox{and}\quad[[{\cal
N}_{t}u^{*}+{\cal L}_{t}\xi,v]]=0,\quad\forall v\in{\cal U}[t,T],$

which implies that (5.21) holds. The converse assertion is obvious. The
assertion (ii) is a direct consequence of the assertion (i). This completes
the proof. ∎

######  Remark 5.6.

It is noteworthy that in the assertion (ii) of 5.5, the assumption that ${\cal
N}_{t}\geqslant 0$ and ${\cal N}_{t}$ is invertible are equivalent to that
${\cal N}_{t}$ is uniformly positive, i.e., there exists a positive constant
$\varepsilon$ such that

$\displaystyle[[{\cal N}_{t}u,u]]\geqslant\varepsilon[[u,u]],\quad\forall
u(\cdot)\in{\cal U}[t,T].$ (5.23)

From the assertion (i) of 5.5, we see that the condition ${\cal
N}_{t}\geqslant 0$ (or equivalently, $[[{\cal N}_{t}u,u]]\geqslant 0$ for
every $u(\cdot)\in{\cal U}[t,T]$) is necessary for the existence of an open-
loop optimal control of Problem (M-SLQ)0, and from the assertion (ii) of 5.5,
we see that the condition (5.23), slightly stronger than ${\cal
N}_{t}\geqslant 0$, is sufficient for the existence of an open-loop optimal
control of Problem (M-SLQ)0. Moreover, by (5.18), we have $[[{\cal
N}_{t}u,u]]=J_{0}(t,0,\vartheta;u(\cdot)).$

Based on the above discussions, we are in a position to prove the auxiliary
results of Subsection 3.2.

###### Proof of Theorem 3.4.

By the assertion (i) of 5.5, we see that $u^{*}(\cdot)\in{\cal U}[t,T]$ is an
open-loop optimal control of Problem (M-SLQ)0 with respect to
$(t,\xi,\vartheta)$ if and only if (5.21) holds. According to (5.18), we see
that ${\cal N}_{t}\geqslant 0$ is equivalent to

$J_{0}(t,0,\vartheta;u(\cdot))=[[{\cal N}_{t}u,u]]\geqslant 0,\quad\forall
u(\cdot)\in{\cal U}[t,T],$

which is exactly condition (3.7). From the definitions of ${\cal N}_{t}$ and
${\cal L}_{t}$, it is easy to verify that

$\displaystyle[{\cal N}_{t}u^{*}+{\cal
L}_{t}\xi](s,\alpha(s))=F(s,\alpha(s),X^{*}(s),Y^{*}(s),Z^{*}(s),u^{*}(s)),\quad
s\in[t,T],$

where $(X^{*}(\cdot),Y^{*}(\cdot),Z^{*}(\cdot),\Gamma^{*}(\cdot))$ is the
adapted solution of FBSDE (3.9). Therefore, ${\cal N}_{t}u^{*}+{\cal
L}_{t}\xi=0$ is equivalent to stationarity condition (3.8). ∎

###### Proof of 3.6.

The sufficiency is obvious. Next we prove the necessity. Suppose that
$u^{*}(\cdot)\in{\cal U}[t,T]$ is optimal for Problem (M-SLQ)0, and let
$(X^{*}(\cdot),Y^{*}(\cdot),Z^{*}(\cdot),\Gamma^{*}(\cdot))$ be the adapted
solution of FBSDE (3.9). In order to prove that $u^{*}(\cdot)$ is also optimal
for Problem ${(\hbox{M-SLQ})}$, it suffices to show that for any set
$\Lambda\in\mathscr{F}_{t}$,

$\displaystyle\mathbb{E}[L(t,\xi,\vartheta;u^{*}(\cdot))\mathbf{1}_{\Lambda}]\leqslant\mathbb{E}[L(t,\xi,\vartheta;u(\cdot))\mathbf{1}_{\Lambda}],\quad\forall
u(\cdot)\in{\cal U}[t,T],$ (5.24)

where for any $(t,\xi,\vartheta)\in{\cal D}$ and $u(\cdot)\in{\cal U}[t,T]$,

$\begin{array}[]{ll}\displaystyle
L(t,\xi,\vartheta;u(\cdot))\triangleq\big{\langle}G(\alpha(T))X(T),X(T)\big{\rangle}+\int_{t}^{T}\left\langle\begin{pmatrix}Q(s,\alpha(s))&S(s,\alpha(s))^{\top}\\\
S(s,\alpha(s))&R(s,\alpha(s))\end{pmatrix}\begin{pmatrix}X(s)\\\
u(s)\end{pmatrix},\begin{pmatrix}X(s)\\\ u(s)\end{pmatrix}\right\rangle
ds.\end{array}$ (5.25)

Note that in (5.25), $X(\cdot)$ is the solution of the state equation (2.1)
with respect to $u(\cdot)$. For this, we would like to fix an arbitrary
control $u(\cdot)\in{\cal U}[t,T]$ and an arbitrary set
$\Lambda\in\mathscr{F}_{t}$. Define

$\displaystyle\widehat{\xi}(\omega)=\xi(\omega)\mathbf{1}_{\Lambda}(\omega),\quad\widehat{\vartheta}(\omega)=\vartheta(\omega)\mathbf{1}_{\vartheta}(\omega),\quad\widehat{u}(s)=u(s)\mathbf{1}_{\Lambda}(\omega),\quad\widehat{u}^{*}(s)=u^{*}(s)\mathbf{1}_{\Lambda}(\omega),$

and consider the following FBSDE:

$\left\\{\begin{array}[]{ll}d\widehat{X}^{*}(s)=[A(s,\alpha(s))\widehat{X}^{*}(s)+B(s,\alpha(s))\widehat{u}^{*}(s)]ds+[C(s,\alpha(s))\widehat{X}^{*}(s)+D(s,\alpha(s))\widehat{u}^{*}(s)]dW(s),\\\
\vskip 3.0pt plus 1.0pt minus 1.0pt\cr\vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle
d\widehat{Y}^{*}(s)=-\widetilde{F}(s,\alpha(s),\widehat{X}^{*}(s),\widehat{Y}^{*}(s),\widehat{Z}^{*}(s),\widehat{u}^{*}(s))ds+\widehat{Z}^{*}(s)dW(s)+\widehat{\Gamma}^{*}(s)\bullet
d\widetilde{N}(s),\quad s\in[t,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle\widehat{X}^{*}(t)=\widehat{\xi},\quad\alpha(t)=\widehat{\vartheta},\quad\widehat{Y}^{*}(T)=G(\alpha(T))\widehat{X}^{*}(T).\end{array}\right.$
(5.26)

It is straightforward to verify that the solution
$(\widehat{X}^{*}(\cdot),\widehat{Y}^{*}(\cdot),\widehat{Z}^{*}(\cdot),\widehat{\Gamma}^{*}(\cdot))$
of FBSDE (5.26) is given by

$\displaystyle\widehat{X}^{*}(s)=X^{*}(s)\mathbf{1}_{\Lambda}(\omega),\quad\widehat{Y}^{*}(s)=Y^{*}(s)\mathbf{1}_{\Lambda}(\omega),\quad\widehat{Z}^{*}(s)=Z^{*}(s)\mathbf{1}_{\Lambda}(\omega),\quad\widehat{\Gamma}^{*}(s)=\Gamma^{*}(s)\mathbf{1}_{\Lambda}(\omega).$

Now, from Theorem 3.4, the quadruple
$(X^{*}(\cdot),Y^{*}(\cdot),Z^{*}(\cdot),\Gamma^{*}(\cdot))$ satisfies
condition (3.8). Hence, multiplying on both sides of (3.8) by
$\mathbf{1}_{\Lambda}$, one has

$\displaystyle
F(s,\alpha(s),\widehat{X}^{*}(s),\widehat{Y}^{*}(s),\widehat{Z}^{*}(s),\widehat{u}^{*}(s))=0,\quad\hbox{a.e.
$s\in[t,T]$, a.s.}.$

Again, applying Theorem 3.4 to the initial triple
$(t,\widehat{\xi},\widehat{\vartheta})$, we conclude that
$\widehat{u}^{*}(\cdot)$ is an open-loop optimal control of Problem (M-SLQ)0
w.r.t. the initial triple $(t,\widehat{\xi},\widehat{\vartheta})$. Therefore,

$\mathbb{E}[L(t,\widehat{\xi},\widehat{\vartheta};\widehat{u}^{*}(\cdot))]\leqslant\mathbb{E}[L(t,\widehat{\xi},\widehat{\vartheta};\widehat{u}(\cdot))].$

Note that the state process
$X^{*}(\cdot)=X^{*}(\cdot;t,\xi,\vartheta,u^{*}(\cdot))$ and the state process
$\widehat{X}^{*}(\cdot)=X^{*}(\cdot;t,\widehat{\xi},\widehat{\vartheta},\widehat{u}^{*}(\cdot))$
are related by
$X^{*}(\cdot;t,\xi,\vartheta,u^{*}(\cdot))\mathbf{1}_{\Lambda}=X^{*}(\cdot;t,\widehat{\xi},\widehat{\vartheta},\widehat{u}^{*}(\cdot)).$
It follows that
$L(t,\xi,\vartheta;u^{*}(\cdot))\mathbf{1}_{\Lambda}=L(t,\widehat{\xi},\widehat{\vartheta};\widehat{u}^{*}(\cdot))$.
Similarly, we have
$L(t,\xi,\vartheta;u(\cdot))\mathbf{1}_{\Lambda}=L(t,\widehat{\xi},\widehat{\vartheta};\widehat{u}(\cdot))$.
Thus,

$\mathbb{E}[L(t,\xi,\vartheta;u^{*}(\cdot))\mathbf{1}_{\Lambda}]=\mathbb{E}[L(t,\widehat{\xi},\widehat{\vartheta};\widehat{u}^{*}(\cdot))]\leqslant\mathbb{E}[L(t,\widehat{\xi},\widehat{\vartheta};\widehat{u}(\cdot))]=\mathbb{E}[L(t,\xi,\vartheta;u(\cdot))\mathbf{1}_{\Lambda}],$

and from which we see that (5.24) holds. This completes the proof. ∎

###### Proof of 3.7.

Note that $(X^{*}(\cdot),u^{*}(\cdot))$ is an open-loop optimal pair with
respect to $(t,\xi,\vartheta)$, so Theorem 3.4 implies

$\displaystyle
F(s,\alpha(s),X^{*}(s),Y^{*}(s),Z^{*}(s),u^{*}(s))=0,\quad\hbox{a.e.
$s\in[t,T]$, a.s..}$

Then the result follows immediately from 5.1. ∎

###### Proof of 3.8.

Let $\tau\in{\cal T}[t,T]$. According to Theorem 3.4 and 3.6, it suffices to
show that $J_{0}(\tau,0,\alpha(\tau);u(\cdot))\geqslant 0$ for all
$u(\cdot)\in{\cal U}[t,T]$, and the adapted solution
$(X^{*}_{\tau}(\cdot),Y^{*}_{\tau}(\cdot),Z^{*}_{\tau}(\cdot),\Gamma^{*}_{\tau}(\cdot))$
to FBSDE (3.9) with $u^{*}(\cdot)$ replaced by $u^{*}(\cdot)|_{[\tau,T]}$
satisfies the following stationarity condition:

$B(s,\alpha(s))^{\top}Y^{*}_{\tau}(s)+D(s,\alpha(s))^{\top}Z^{*}_{\tau}(s)+S(s,\alpha(s))X^{*}_{\tau}(s)+R(s,\alpha(s))u^{*}(s)|_{[\tau,T]}=0,\
\ \hbox{a.e. $s\in[t,T]$, a.s..}$

For this, we let $u(\cdot)\in{\cal U}[\tau,T]$ be an arbitrary and define the
zero-extension of $u(\cdot)$ over the time interval $[t,T]$ as follows:

$u_{e}(s)=\left\\{\begin{array}[]{ll}\displaystyle 0,\qquad\ s\in[t,\tau),\\\
\vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle u(s),\quad
s\in[\tau,T].\end{array}\right.$ (5.27)

Clearly, $u_{e}(\cdot)$ belongs to the space ${\cal U}[t,T]$. Denote by
$X^{\tau}$ and $X^{t}$ the solutions to the following SDEs, respectively,

$\left\\{\begin{aligned} \displaystyle
dX^{\tau}(s)=&[A(s,\alpha(s))X^{\tau}(s)+B(s,\alpha(s))u(s)]ds\\\ \vskip 3.0pt
plus 1.0pt minus
1.0pt\cr\displaystyle&+[C(s,\alpha(s))X^{\tau}(s)+D(s,\alpha(s))u(s)]dW(s),\quad
s\in[\tau,T],\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
X^{\tau}(\tau)=&0,\quad\alpha(t)=\vartheta,\end{aligned}\right.$

and

$\left\\{\begin{aligned} \displaystyle
dX^{t}(s)=&[A(s,\alpha(s))X^{t}(s)+B(s,\alpha(s))u_{e}(s)]ds\\\ \vskip 3.0pt
plus 1.0pt minus
1.0pt\cr\displaystyle&+[C(s,\alpha(s))X^{t}(s)+D(s,\alpha(s))u_{e}(s)]dW(s),\quad
s\in[t,T],\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
X^{t}(t)=&0,\quad\alpha(t)=\vartheta.\end{aligned}\right.$

Since the initial states of the above two SDEs are 0, and note that $v_{e}=0$
on $[t,\tau)$, we have

$X^{t}(s)=\left\\{\begin{array}[]{ll}\displaystyle 0,\qquad\quad
s\in[t,\tau],\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
X^{\tau}(s),\quad s\in[\tau,T],\end{array}\right.$

from which one has that

$\displaystyle J_{0}(\tau,0,\alpha(\tau);u(\cdot))$
$\displaystyle=\mathbb{E}\bigg{[}\big{\langle}G(\alpha(T))X^{\tau}(T),X^{\tau}(T)\big{\rangle}+\int_{\tau}^{T}\left\langle\begin{pmatrix}Q(s,\alpha(s))&S(s,\alpha(s))^{\top}\\\
S(s,\alpha(s))&R(s,\alpha(s))\end{pmatrix}\begin{pmatrix}X^{\tau}(s)\\\
u(s)\end{pmatrix},\begin{pmatrix}X^{\tau}(s)\\\ u(s)\end{pmatrix}\right\rangle
ds\bigg{]}$
$\displaystyle=\mathbb{E}\bigg{[}\big{\langle}G(\alpha(T))X^{t}(T),X^{t}(T)\big{\rangle}+\int_{t}^{T}\left\langle\begin{pmatrix}Q(s,\alpha(s))&S(s,\alpha(s))^{\top}\\\
S(s,\alpha(s))&R(s,\alpha(s))\end{pmatrix}\begin{pmatrix}X^{t}(s)\\\
u_{e}(s)\end{pmatrix},\begin{pmatrix}X^{t}(s)\\\
u_{e}(s)\end{pmatrix}\right\rangle ds\bigg{]}$
$\displaystyle=J_{0}(t,0,\vartheta;u_{e}(\cdot)).$ (5.28)

Due to that under the conditions, Problem (M-SLQ) is solvable at
$(t,\xi,\vartheta)$, and from the assertion (i) of Theorem 3.4 and relation
(5.28), we obtain that

$J_{0}(\tau,0,\alpha(\tau);u(\cdot))=J_{0}(t,0,\vartheta;u_{e}(\cdot))\geqslant
0,\quad\forall u(\cdot)\in{\cal U}[\tau,T].$

Next, note that the quadruple
$(X^{*}(\cdot),Y^{*}(\cdot),Z^{*}(\cdot),\Gamma^{*}(\cdot))$ is the adapted
solution of FBSDE (3.9). Since $u^{*}(\cdot)\in{\cal U}[t,T]$ is an open-loop
optimal control w.r.t. $(t,\xi,\vartheta)$, by assertion (ii) of Theorem 3.4,

$B(s,\alpha(s))^{\top}Y^{*}(s)+D(s,\alpha(s))^{\top}Z^{*}(s)+S(s,\alpha(s))X^{*}(s)+R(s,\alpha(s))u^{*}(s)=0,\quad\hbox{a.e.
$s\in[t,T]$, a.s..}$

Then the results hold from the fact that

$(X^{*}_{\tau}(s),Y^{*}_{\tau}(s),Z^{*}_{\tau}(s),\Gamma^{*}_{\tau}(s))=(X^{*}(s),Y^{*}(s),Z^{*}(s),\Gamma^{*}(s)),\quad
s\in[\tau,T].$

This completes the proof. ∎

###### Proof of 3.9.

For arbitrary $u(\cdot)\in\mathcal{U}[\tau,T]$, we define $u_{e}(\cdot)$ as in
(5.27). Using the same argument as in the proof of 3.8 with $t=0$, one has

$J_{0}(\tau,0,\alpha(\tau);u(\cdot))=J_{0}\left(0,0,i_{0};u_{e}(\cdot)\right)\geqslant\varepsilon\mathbb{E}\int_{0}^{T}\left|u_{e}(s)\right|^{2}ds=\varepsilon\mathbb{E}\int_{\tau}^{T}|u(s)|^{2}ds.$

Hence, by setting $\tau\equiv t$ and from 5.6, the assertion (ii) of 5.5 and
3.6, we obtain that Problem (M-SLQ) is uniquely solvable. ∎

###### Proof of 3.10.

By Theorem 3.4, it is easy to see that the first assertion holds. Now, we
consider the second assertion. Note that $\xi\in
L_{\mathscr{F}_{t}}^{\infty}\left(\Omega;\mathbb{R}^{n}\right)$ is bounded, so
the pair
$(\mathbb{X}(s),\mathbb{U}(s))\triangleq(\boldsymbol{X}(s)\xi,\boldsymbol{U}(s)\xi)$
is square-integrable and satisfies the following state equation:

$\left\\{\begin{aligned} \displaystyle
d\mathbb{X}(s)&=\big{[}A(s,\alpha(s))\mathbb{X}(s)+B(s,\alpha(s))\mathbb{U}(s)\big{]}ds\\\
\vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&\quad+\big{[}C(s,\alpha(s))\mathbb{X}(s)+D(s,\alpha(s))\mathbb{U}(s)\big{]}dW(s),\quad
s\in[t,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle\mathbb{X}(t)&=\xi,\quad\alpha(t)=\vartheta.\end{aligned}\right.$
(5.29)

Similarly, we see that the triple
$\left(\mathbb{Y}(s),\mathbb{Z}(s),\breve{\mathbf{\Gamma}}(s)\right)\triangleq(\boldsymbol{Y}(s)\xi,{\bf
Z}(s)\xi,\boldsymbol{\Gamma}(s)\circ\xi)$ is the adapted solution to the
following adjoint BSDE associated with
$\left(\mathbb{X}(\cdot),\mathbb{U}(\cdot)\right)$:

$\left\\{\begin{array}[]{ll}\displaystyle
d\mathbb{Y}(s)=-\widetilde{F}(s,\alpha(s),\mathbb{X}(s),\mathbb{Y}(s),\mathbb{Z}(s),\mathbb{U}(s))ds+\mathbb{Z}(s)dW(s)+\breve{\mathbf{\Gamma}}(s)\bullet
d\widetilde{N}(s),\quad s\in[t,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle\mathbb{Y}(T)=G(\alpha(T))\mathbb{X}(T),\quad\alpha(t)=\vartheta.\end{array}\right.$

Furthermore, (3.12) deduces that

$\displaystyle
F(s,\alpha(s),\mathbb{X}(s),\mathbb{Y}(s),\mathbb{Z}(s),\mathbb{U}(s))=F(s,\alpha(s),{\bf
X}(s),{\bf Y}(s),{\bf Z}(s),{\bf U}(s))\xi=0,\quad\hbox{a.e. $s\in[t,T]$,
a.s.}.$

Thus, combining Theorem 3.4, the pair
$\left(\mathbb{X}(\cdot),\mathbb{U}(\cdot)\right)$ is optimal w.r.t. the
initial triple $(t,\xi,\vartheta)$. ∎

###### Proof of Theorem 3.11.

Let $\left\\{\left(X_{j}(s),u_{j}(s)\right)\right\\}_{s\in[t,T]}$ and
$\\{(\boldsymbol{X}(s),\boldsymbol{U}(s))\\}_{s\in[t,T]}$ be as in 3.10, then
the pair $\\{(\boldsymbol{X}(s)\xi,\boldsymbol{U}(s)\xi)\\}_{s\in[t,T]}$ is
optimal w.r.t. the initial triple $(t,\xi,\vartheta)$. For simplicity
presentation, we denote

$\mathbf{M}(T,\alpha(T))\triangleq\boldsymbol{X}(T)^{\top}G(\alpha(T))\boldsymbol{X}(T),\quad\boldsymbol{N}(s,\alpha(s))\triangleq\left(\begin{array}[]{c}\boldsymbol{X}(s)\\\
\boldsymbol{U}(s)\end{array}\right)^{\top}\left(\begin{array}[]{cc}Q(s,\alpha(s))&S(s,\alpha(s))^{\top}\\\
S(s,\alpha(s))&R(s,\alpha(s))\end{array}\right)\left(\begin{array}[]{c}\boldsymbol{X}(s)\\\
\boldsymbol{U}(s)\end{array}\right),$

and then we could rewrite

$L(t,\xi,\vartheta;\boldsymbol{U}\xi)=\langle\mathbf{M}(T,\alpha(T))\xi,\xi\rangle+\int_{t}^{T}\langle\boldsymbol{N}(s,\alpha(s))\xi,\xi\rangle
ds,$ (5.30)

where $L(\cdot)$ is defined in (5.25). Now, on one hand, since the pair
$\\{(\boldsymbol{X}(s)\xi,\boldsymbol{U}(s)\xi)\\}_{s\in[t,T]}$ is optimal
w.r.t. the initial triple $(t,\xi,\vartheta)$, from the definition (2.2) we
have

${V}(t,\xi,\vartheta)=\mathbb{E}[L(t,\xi,\vartheta;\boldsymbol{U}\xi)\mid\mathscr{F}_{t}].$
(5.31)

On the other hand, note that the state equation (5.29) works over the time
horizon $[t,T]$ with the initial value $\xi$ is $\mathscr{F}_{t}$-measurable
and $\alpha(t)=\vartheta$ is $\mathscr{F}_{t}^{\alpha}$-measurable, hence
there is a process $P:[0,T]\times{\cal
S}\times\Omega\rightarrow\mathbb{S}^{n}$, which is $\mathbb{F}$-adapted, such
that

$P(t,\alpha(t))=\mathbb{E}\Big{[}\mathbf{M}(T,\alpha(T))+\int_{t}^{T}\boldsymbol{N}(s,\alpha(s))ds\Big{|}\mathscr{F}_{t}\Big{]}.$
(5.32)

In fact, for the special case of $C=G=1$ and $B=D=Q=S=R=0$, one can calculate
the solution of SDE (5.29) to obtain that

$\mathbb{X}(s)=\xi\exp\left\\{\int_{t}^{s}\Big{[}A(r,\alpha(r))-\frac{1}{2}\Big{]}dr+W(s)-W(t)\right\\},\quad
s\in[t,T].$

Note that $\mathbb{X}(s)=\boldsymbol{X}(s)\xi$, and in this case
$\mathbf{M}(T,\alpha(T))=\boldsymbol{X}(T)^{\top}\boldsymbol{X}(T)$ and
$\boldsymbol{N}(s,\alpha(s))\equiv 0$, so

$\mathbb{E}[\mathbf{M}(T,\alpha(T))|\mathscr{F}_{t}]=\mathbb{E}\bigg{[}\exp\bigg{\\{}\int_{t}^{T}[2A(r,\alpha(r))-1]dr\bigg{\\}}\Big{|}\mathscr{F}_{t}\bigg{]}.$

Thus (5.32) holds due to that $\alpha(\cdot)$ is a Markov chain. Moreover, for
the general situation, one can still prove that (5.32) holds using a similar
argument. Finally, by combining (5.30)-(5.32) and noting that
$\alpha(t)=\vartheta$, we have

$\displaystyle{V}(t,\xi,\vartheta)=$ $\displaystyle\
\mathbb{E}[L(t,\xi,\vartheta;\boldsymbol{U}\xi)\mid\mathscr{F}_{t}]$
$\displaystyle=$ $\displaystyle\
\Big{\langle}\mathbb{E}\Big{[}\mathbf{M}(T,\alpha(T))+\int_{t}^{T}\boldsymbol{N}(s,\alpha(s))ds\Big{|}\mathscr{F}_{t}\Big{]}\xi,\xi\Big{\rangle}=\langle
P(t,\vartheta)\xi,\xi\rangle.$

This completes the proof. ∎

###### Proof of Theorem 3.12.

First, we prove the boundedness of the process
$P=\\{P(t,i);(t,i)\in[0,T]\times{\cal S}\\}$. From 3.9, we have that for every
$t\in[0,T)$, the operator $\mathcal{N}_{t}$ defined in (5.11) satisfies

$\displaystyle[[\mathcal{N}_{t}u,u]]=J_{0}(t,0,\vartheta;u(\cdot))\geqslant\varepsilon\mathbb{E}\int_{t}^{T}|u(s)|^{2}ds=\varepsilon[[u,u]],\quad\forall
u(\cdot)\in\mathcal{U}[t,T],$ (5.33)

which implies that the operator $\mathcal{N}_{t}$ is positive and invertible.
Moreover, on one hand, for any initial state $\xi\in
L_{\mathscr{F}_{t}}^{\infty}\left(\Omega;\mathbb{R}^{n}\right)$, by assertion
(ii) of 5.5, the related open-loop optimal control is given by

$\displaystyle u^{*}_{t,\xi}(s)=-[{\cal N}^{-1}_{t}{\cal L}_{t}\xi](s),\quad
s\in[t,T].$

Then, by substituting $u_{t,\xi}^{*}(\cdot)$ into (5.18), one has

$\mathbb{E}\langle
P(t,\vartheta)\xi,\xi\rangle=V_{0}(t,\xi,\vartheta)=\mathbb{E}\langle
M(t)\xi,\xi\rangle-[[\mathcal{N}_{t}^{-1}\mathcal{L}_{t}\xi,\mathcal{L}_{t}\xi]],$
(5.34)

where ${\cal N}_{t}$ and ${\cal L}_{t}$ are defined in (5.11) and $M(\cdot)$
is the first component of the solution of BSDE (2.4). Therefore,

$\mathbb{E}\langle P(t,\vartheta)\xi,\xi\rangle\leqslant\mathbb{E}\langle
M(t)\xi,\xi\rangle.$ (5.35)

On the other hand, combining (5.33) and (5.34) leads to

$\mathbb{E}\langle P(t,\vartheta)\xi,\xi\rangle\geqslant\mathbb{E}\langle
M(t)\xi,\xi\rangle-\varepsilon^{-1}[[\mathcal{L}_{t}\xi,\mathcal{L}_{t}\xi]].$
(5.36)

Thus, from assertion (ii) of 5.2, we have

$\mathbb{E}\langle P(t,\vartheta)\xi,\xi\rangle\geqslant\mathbb{E}\langle
M(t)\xi,\xi\rangle-\varepsilon^{-1}K\mathbb{E}|\xi|^{2}=\mathbb{E}\left\langle\left[M(t)-\varepsilon^{-1}KI_{n}\right]\xi,\xi\right\rangle,$
(5.37)

where $K$ is a positive constant comes from 5.2. Note that $\xi\in
L^{\infty}_{\mathscr{F}_{t}}(\Omega;\mathbb{R}^{n})$ is bounded and arbitrary,
which, together with (5.35) and (5.37), implies that

$M(t)-\varepsilon^{-1}KI_{n}\leqslant P(t,\vartheta)\leqslant
M(t),\quad(t,\vartheta)\in[0,T]\times
L_{\mathscr{F}_{t}^{\alpha}}^{2}(\Omega;\mathcal{S}).$

Therefore, the process $P=\\{P(t,i);(t,i)\in[0,T]\times{\cal S}\\}$ is bounded
follows from the above inequality and the fact that $M(\cdot)$ is bounded (see
2.3).

In the following, we prove that it is left-continuous. For simplicity but
without loss of generality, we only study the left-continuity at $t=T$, and
the situation of $t\in(0,T)$ could be proved similarly by considering the
related Problem (M-SLQ). Note that, owing to (5.35) and (5.36), for every
initial triple $(t,\xi,\vartheta)\in[0,T)\times
L_{\mathscr{F}_{t}}^{\infty}\left(\Omega;\mathbb{R}^{n}\right)\times
L_{\mathscr{F}_{t}^{\alpha}}^{2}(\Omega;\mathcal{S})$, we have

$\mathbb{E}\langle
M(t)\xi,\xi\rangle-\varepsilon^{-1}[[\mathcal{L}_{t}\xi,\mathcal{L}_{t}\xi]]\leqslant\mathbb{E}\langle
P(t,\vartheta)\xi,\xi\rangle\leqslant\mathbb{E}\langle M(t)\xi,\xi\rangle.$
(5.38)

Noting representation (5.16), we can rewrite $[[{\cal L}_{t}\xi,{\cal
L}_{t}\xi]]$ as follows,

$\displaystyle[[\mathcal{L}_{t}\xi,\mathcal{L}_{t}\xi]]=\mathbb{E}\int_{t}^{T}\Big{\langle}\big{[}F_{0}(s,\alpha(s),{\cal
X}(s),{\cal Y}(s),{\cal Z}(s)){\cal
X}^{-1}(t)\big{]}^{\top}\big{[}F_{0}(s,\alpha(s),{\cal X}(s),{\cal Y}(s),{\cal
Z}(s)){\cal X}^{-1}(t)\big{]}\xi,\xi\Big{\rangle}ds.$ (5.39)

Again, note that $\xi\in L^{\infty}_{\mathscr{F}_{t}}(\Omega;\mathbb{R}^{n})$
is bounded and arbitrary, which, together with (5.38) and (5.39), implies that
for any $(t,\vartheta)\in[0,T]\times
L_{\mathscr{F}_{t}^{\alpha}}^{2}(\Omega;\mathcal{S})$,

$\displaystyle M(t)-\varepsilon^{-1}\big{[}{\cal
X}^{-1}(t)\big{]}^{\top}\int_{t}^{T}F_{0}(s,\alpha(s),{\cal X}(s),{\cal
Y}(s),{\cal Z}(s))^{\top}F_{0}(s,\alpha(s),{\cal X}(s),{\cal Y}(s),{\cal
Z}(s))ds\ {\cal X}^{-1}(t)$ $\displaystyle\leqslant P(t,\vartheta)\leqslant
M(t).$

Note that ${\cal X}^{-1}(t)$ and $M(\cdot)$ are $\mathscr{F}_{t}$-measurable,
and $P(\cdot,\alpha(\cdot))\in L_{\mathbb{F}}^{\infty}(0,T;\mathbb{S}^{n})$.
Hence, by taking the conditional expectations w.r.t. $\mathscr{F}_{t}$ on both
side of the above inequality, one has

$\displaystyle M(t)-\varepsilon^{-1}\big{[}{\cal
X}^{-1}(t)\big{]}^{\top}\mathbb{E}\bigg{[}\int_{t}^{T}F_{0}(s,\alpha(s),{\cal
X}(s),{\cal Y}(s),{\cal Z}(s))^{\top}F_{0}(s,\alpha(s),{\cal X}(s),{\cal
Y}(s),{\cal Z}(s))ds\Big{|}\mathscr{F}_{t}\bigg{]}{\cal X}^{-1}(t)$
$\displaystyle\leqslant P(t,i)\leqslant M(t).$

Finally, letting $t\uparrow T$ and using the dominated convergence theorem, we
get that

$\lim_{t\uparrow T}P(t,i)=M(T)=G(i)=P(T,i).$

Hence, the second purpose is obtained. ∎

###### Proof of 3.13.

First, we prove the assertion (i). For any fix initial triple
$(\sigma,\xi,\vartheta)\in{\cal T}[0,\tau)\times
L_{\mathscr{F}_{\sigma}}^{\infty}(\Omega;\mathbb{R}^{n})\times
L_{\mathscr{F}_{\sigma}^{\alpha}}^{2}(\Omega;{\cal S})$ and
$u(\cdot)\in\mathcal{U}[\sigma,\tau]$, denote by
$\left\\{X_{1}(s)\right\\}_{s\in[\sigma,\tau]}$ the corresponding solution of
state equation (2.1) w.r.t. $(\sigma,\xi,\vartheta)$ over $[\sigma,\tau]$.
Consider Problem (M-SLQ) for the initial triple
$\left(\tau,X_{1}(\tau),\alpha(\tau)\right)$. By the assumptions, there exists
$\varepsilon>0$ such that (3.3) holds. Hence, from 3.9 and representation
(3.13), Problem (M-SLQ) is solvable at $\tau$ and

${V}\left(\tau,X_{1}(\tau),\alpha(\tau)\right)=\langle
P(\tau,\alpha(\tau))X_{1}(\tau),X_{1}(\tau)\rangle.$

Now for Problem (M-SLQ), let $v^{*}(\cdot)\in\mathcal{U}[\tau,T]$ be an open-
loop optimal control w.r.t. the initial triple
$\left(\tau,X_{1}(\tau),\alpha(\tau)\right)$ and let
$\\{X_{2}^{*}(s)\\}_{s\in[\tau,T]}$ be the corresponding open-loop optimal
state process. Define

$\left[u\oplus v^{*}\right](s)=\begin{cases}u(s),&s\in[\sigma,\tau),\\\
v^{*}(s),&s\in[\tau,T],\end{cases}\qquad\hbox{and}\qquad\widetilde{X}(s)=\begin{cases}X_{1}(s),&s\in[\sigma,\tau),\\\
X_{2}^{*}(s),&s\in[\tau,T].\end{cases}$

Then, it is easy to see that $[u\oplus v^{*}](\cdot)\in\mathcal{U}[\sigma,T]$
and $\widetilde{X}(\cdot)$ satisfies the following equation:

$\left\\{\begin{aligned}
\displaystyle&d\widetilde{X}(s)=\big{[}A(s,\alpha(s))\widetilde{X}(s)+B(s,\alpha(s))[u\oplus
v^{*}](s)\big{]}ds+\big{[}C(s,\alpha(s))\widetilde{X}(s)+D(s,\alpha(s))[u\oplus
v^{*}](s)\big{]}dW(s),\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&\widetilde{X}(\sigma)=\xi,\quad\alpha(\sigma)=\vartheta.\end{aligned}\right.$

Furthermore, we have

$\displaystyle J_{0}\left(\sigma,\xi,\vartheta;[u\oplus v^{*}](\cdot)\right)$
$\displaystyle=\mathbb{E}\bigg{[}\left\langle
G(\alpha(T))X_{2}^{*}(T),X_{2}^{*}(T)\right\rangle+\int_{\tau}^{T}\left\langle\left(\begin{array}[]{cc}Q(s,\alpha(s))&S(s,\alpha(s))^{\top}\\\
S(s,\alpha(s))&R(s,\alpha(s))\end{array}\right)\left(\begin{array}[]{c}X_{2}^{*}(s)\\\
v^{*}(s)\end{array}\right),\left(\begin{array}[]{c}X_{2}^{*}(s)\\\
v^{*}(s)\end{array}\right)\right\rangle ds\bigg{]}$ (5.46)
$\displaystyle\quad~{}+\mathbb{E}\bigg{[}\int_{\sigma}^{\tau}\left\langle\left(\begin{array}[]{cc}Q(s,\alpha(s))&S(s,\alpha(s))^{\top}\\\
S(s,\alpha(s))&R(s,\alpha(s))\end{array}\right)\left(\begin{array}[]{c}X_{1}(s)\\\
u(s)\end{array}\right),\left(\begin{array}[]{c}X_{1}(s)\\\
u(s)\end{array}\right)\right\rangle ds\bigg{]}$ (5.53)
$\displaystyle=\mathbb{E}\bigg{[}V\big{(}\tau,X_{1}(\tau),\alpha(\tau)\big{)}+\int_{\sigma}^{\tau}\left\langle\left(\begin{array}[]{cc}Q(s,\alpha(s))&S(s,\alpha(s))^{\top}\\\
S(s,\alpha(s))&R(s,\alpha(s))\end{array}\right)\left(\begin{array}[]{c}X_{1}(s)\\\
u(s)\end{array}\right),\left(\begin{array}[]{c}X_{1}(s)\\\
u(s)\end{array}\right)\right\rangle ds\bigg{]}$ (5.60)
$\displaystyle=J^{\tau}_{0}(\sigma,\xi,\vartheta;u(\cdot)),$ (5.61)

where in the last equation we employ relation (3.13) and the definition of
$J^{\tau}_{0}$. In particular, when $\xi=0$, from 3.9 we have

$J^{\tau}_{0}(\sigma,0,\vartheta;u(\cdot))=J_{0}(\sigma,0,\vartheta;[u\oplus
v^{*}](\cdot))\geqslant\varepsilon\mathbb{E}\int_{\sigma}^{T}\big{|}\left[u\oplus
v^{*}\right](s)\big{|}^{2}ds\geqslant\varepsilon\mathbb{E}\int_{\sigma}^{\tau}|u(s)|^{2}ds,$
(5.62)

which combining 5.6, assertion (ii) of 5.5, and 3.6 imply assertion (i) holds.

Now, we consider assertions (ii) and (iii). Note that (5.61) still holds when
the expectation $\mathbb{E}$ replaced by the conditional expectation
$\mathbb{E}_{\sigma}[\cdot]$. Therefore,

$J^{\tau}(\sigma,\xi,\vartheta;u(\cdot))\geqslant\big{\langle}P(\sigma,\vartheta)\xi,\xi\big{\rangle}.$
(5.63)

For Problem (M-SLQ), assume that $u^{*}(\cdot)\in\mathcal{U}[\sigma,T]$ is an
open-loop optimal control w.r.t. $(\sigma,\xi,\vartheta)$, and denote by
$X^{*}=\left\\{X^{*}(s);\sigma\leqslant s\leqslant T\right\\}$ the related
open-loop optimal state process, i.e., for $s\in[\sigma,T],$

$\left\\{\begin{aligned} \displaystyle
dX^{*}(s)&=\big{[}A(s,\alpha(s))X^{*}(s)+B(s,\alpha(s))u^{*}(s)\big{]}ds+\big{[}C(s,\alpha(s))X^{*}(s)+D(s,\alpha(s))u^{*}(s)\big{]}dW(s),\\\
\vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
X^{*}(\sigma)&=\xi,\quad\alpha(\sigma)=\vartheta.\end{aligned}\right.$

Then from 3.8, we see that for Problem (M-SLQ), the restriction
$\left.u^{*}(\cdot)\right|_{[\tau,T]}$ of $u^{*}(\cdot)$ on the interval
$[\tau,T]$ is optimal for the initial triple
$\left(\tau,X^{*}(\tau),\alpha(\tau)\right)$. Moreover, in (5.61), using
$\left.u^{*}(\cdot)\right|_{[\sigma,\tau]}$ and
$\left.u^{*}(\cdot)\right|_{[\tau,T]}$ to replace $u(\cdot)$ and
$v^{*}(\cdot)$, respectively, and note that
$u^{*}(\cdot)|_{[\sigma,\tau]}\oplus u^{*}(\cdot)|_{[\tau,T]}=u^{*}(\cdot),$
we deduce

$J^{\tau}\big{(}\sigma,\xi,\vartheta;\left.u^{*}(\cdot)\right|_{[\sigma,\tau]}\big{)}=J\left(\sigma,\xi,\vartheta;u^{*}(\cdot)\right)=\big{\langle}P(\sigma,\vartheta)\xi,\xi\big{\rangle}.$
(5.64)

Therefore, combining (5.63) and (5.64), assertions (ii) and (iii) hold. ∎

######  Remark 5.7.

Since many notations have been adopted, we finally summarize some frequently
used notations so as not to be confused. Recall that the quadruple
$(X(\cdot),Y(\cdot),Z(\cdot),\Gamma(\cdot))$ is the solution of SDE (2.1) and
BSDE (3.6), $(X^{*}(\cdot),Y^{*}(\cdot),Z^{*}(\cdot),\Gamma^{*}(\cdot))$ is
the solution of FBSDE (3.9), and
$(X^{*}_{\tau}(\cdot),Y^{*}_{\tau}(\cdot),Z^{*}_{\tau}(\cdot),\Gamma^{*}_{\tau}(\cdot))$
is the solution of FBSDE (3.9) with $u^{*}(\cdot)$ replaced by
$u^{*}(\cdot)|_{[\tau,T]}$. Moreover, the quadruple
$\big{(}\boldsymbol{X}(\cdot),\boldsymbol{Y}(\cdot),\boldsymbol{Z}(\cdot),\boldsymbol{\Gamma}(\cdot)\big{)}$
is the solution of FBSDE (3.11),
$(\tilde{X}(\cdot),\tilde{Y}(\cdot),\tilde{Z}(\cdot),\tilde{\Gamma}(\cdot))$
is the solution of FBSDE (5.6),
$(\bar{X}(\cdot),\bar{Y}(\cdot),\bar{Z}(\cdot),\bar{\Gamma}(\cdot))$ is the
solution of FBSDE (5.10), and
$(\widehat{X}^{*}(\cdot),\widehat{Y}^{*}(\cdot),\widehat{Z}^{*}(\cdot),\widehat{\Gamma}^{*}(\cdot))$
is the solution of FBSDE (5.26). Besides, the process
$\hat{P}(\cdot,\alpha(\cdot))$, together with $\hat{\Lambda}(\cdot)$ and
$\hat{\zeta}(\cdot)$, is the solution of SRE (3.1), and
$P(\cdot,\alpha(\cdot))$ is the process that satisfies the relation (3.13).

##  6 Example

As presented in the introduction, in a realistic market, it is better to allow
the market parameters to depend on both the Markov chain and Brownian motion,
due to the interest rates, stock rates, and volatilities being affected by the
uncertainties caused by the Brownian motion. In this section, as an
application of our main results, we give an example of the continuous-time
mean-variance portfolio selection problem, which partially develops the work
of Li–Zhou–Lim [14] to the Markovian regime switching system with random
coefficients. For simplicity, we would like to let $m=n=1$.

Suppose there is a market in which two assets are traded over a finite horizon
$[0,T]$. One of the assets is the bond whose asset price $S_{0}(\cdot)$ is
subject to the following ordinary differential equation:

$\left\\{\begin{array}[]{l}\displaystyle dS_{0}(s)=r(s)S_{0}(s)ds,\quad
s\in[0,T],\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
S_{0}(0)=S_{0}>0,\end{array}\right.$

where $r(\cdot)$ is a positive and bounded function, which represents the
bond’s interest rate. The other one of the assets is stock whose asset price
$S_{1}(\cdot)$ satisfies the following stochastic differential equation:

$\left\\{\begin{array}[]{l}\displaystyle
dS_{1}(s)=S_{1}(s)\Big{\\{}b(s,\alpha(s))dt+\sigma(s,\alpha(s))dW(s)\Big{\\}},\quad
s\in[0,T],\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle
S_{1}(0)=S_{1}>0,\quad\alpha(0)=i_{0},\end{array}\right.$

where $b(s,\alpha(s))$ is the appreciation rate and $\sigma(s,\alpha(s))$ is
the volatility or the dispersion of the stock. Assume that for any choice of
$i\in{\cal S}$, both $b(\cdot,i)$ and $\sigma(\cdot,i)$ are in the space
$L_{\mathbb{F}^{W}}^{\infty}(0,T;\mathbb{R})$, and there is a positive
constant $\delta$ such that

$\sigma(s,\alpha(s))^{2}\geqslant\delta,\quad\forall s\in[0,T].$

Assume that the trading of shares takes place continuously and transaction
costs and consumption are not considered. Then, a small investor’s self-
financing wealth process $X(\cdot)$ satisfies the following SDE:

$\left\\{\begin{aligned}
dX(s)=&\Big{\\{}r(s)X(s)+[b(s,\alpha(s))-r(s)]u(s)\Big{\\}}ds+\sigma(s,\alpha(s))u(s)dW(s),\quad
s\in[0,T],\\\ X(0)=&x_{0}>0,\quad\alpha(0)=i_{0},\end{aligned}\right.$ (6.1)

where $u(\cdot)\in L_{\mathbb{F}}^{2}(0,T;\mathbb{R})$ is a portfolio of the
investor, which may change over time $s\in[0,T]$. Note that $u(s)=0$ implies
that the investor invests his/her total wealth in the bond at time
$s\in[0,T]$.

As shown in Li–Zhou–Lim [14], the mean-variance portfolio selection refers to
the problem of finding an allowable investment policy (i.e., a dynamic
portfolio satisfying all the constraints) such that the expected terminal
wealth satisfies $\mathbb{E}[X(T)]=d$ while the risk measured by the variance
of the terminal wealth is

$\operatorname{Var}(X(T))=\mathbb{E}\big{[}X(T)-\mathbb{E}[X(T)]\big{]}^{2}=\mathbb{E}\big{[}(X(T)-d)^{2}\big{]}.$

Then we consider the following dynamic stochastic optimization

$\left\\{\begin{aligned} \min~{}&\mathbb{E}\big{[}(X(T)-d)^{2}\big{]},\\\
\mbox{s.t. }&\mathbb{E}[X(T)]=d,\\\ &(X(\cdot),u(\cdot))\mbox{ satisfies
(\ref{eq:wealth})}.\end{aligned}\right.$ (6.2)

Since (6.2) is a convex optimization problem, the equality constraint
$\mathbb{E}[X(T)]=d$ can be dealt with by introducing a Lagrange multiplier
$\mu\in\mathbb{R}$. Therefore, we have

$\mathbb{E}\big{[}(X(T)-d)^{2}\big{]}-2\mu(\mathbb{E}[X(T)]-d)=\mathbb{E}\big{[}(X(T)-(d+\mu))^{2}\big{]}-\mu^{2}=\mathbb{E}[X(T)-\gamma]^{2}-\mu^{2},$

where $\gamma=d+\mu$. Now, if we set

$\widetilde{X}(s)=X(s)-\gamma\exp\Big{\\{}-\int_{s}^{T}r(s)ds\Big{\\}},$

then (6.2) can be transferred into the following problem

$\left\\{\begin{aligned}
\displaystyle\min~{}&\mathbb{E}[\widetilde{X}(T)^{2}]\\\ \vskip 3.0pt plus
1.0pt minus 1.0pt\cr\displaystyle\mbox{s.t.
}&d\widetilde{X}(s)=\Big{\\{}r(s)\widetilde{X}(s)+[b(s,\alpha(s))-r(s)]u(s)\Big{\\}}ds+\sigma(s,\alpha(s))u(s)dW(s),\quad
s\in[0,T],\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr\displaystyle&\
\widetilde{X}(0)=x_{0}-\gamma\exp\Big{\\{}-\int_{0}^{T}r(s)ds\Big{\\}},\quad\alpha(0)=i_{0}.\end{aligned}\right.$
(6.3)

It is easy to see that Problem (6.3) is a special case of Problem (M-SLQ) with

$G(\alpha(T))=1,\quad A(s,\alpha(s))=r(s),\quad
B(s,\alpha(s))=b(s,\alpha(s))-r(s),\quad
D(s,\alpha(s))=\sigma(s,\alpha(s)),\quad s\in[0,T],$

and other coefficients are zero. Then the SRE associated to Problem (6.3) is:

$\left\\{\begin{aligned} \displaystyle
d\hat{P}(s,\alpha(s))&=-\big{[}\hat{Q}(s,\alpha(s))+\hat{S}(s,\alpha(s))^{\top}\Theta(s,\alpha(s))\big{]}ds\\\
\vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&\quad+\hat{\Lambda}(s)dW(s)+\hat{\zeta}(s)\bullet
d\widetilde{N}(s),\quad s\in[0,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle\hat{P}(T,\alpha(T))&=1,\quad\alpha(0)=i_{0},\end{aligned}\right.$
(6.4)

where $i_{0}\in{\cal S}$ and for $s\in[0,T]$,

$\displaystyle\hat{Q}(s,\alpha(s))$ $\displaystyle\triangleq
2r(s)\hat{P}(s,\alpha(s)),\qquad\hat{S}(s,\alpha(s))\triangleq[b(s,\alpha(s))-r(s)]\hat{P}(s,\alpha(s))+\sigma(s,\alpha(s))\hat{\Lambda}(s),$
(6.5) $\displaystyle\hat{R}(s,\alpha(s))$
$\displaystyle\triangleq\sigma(s,\alpha(s))^{2}\hat{P}(s,\alpha(s)),\qquad\Theta(s,\alpha(s))\triangleq-\hat{R}(s,\alpha(s))^{-1}\hat{S}(s,\alpha(s)).$

Now, we point out that conditions (H) and (3.3) hold for this case. In fact,
it is trivial to verify condition (H). As for condition (3.3), by solving the
linear SDE of (6.3), it is easy to verify that there is a positive constant
$\varepsilon$ such that

$\displaystyle\mathbb{E}[\widetilde{X}(T)^{2}]\geqslant\varepsilon\mathbb{E}\int_{0}^{T}|u(s)|^{2}ds,\quad\forall
u(\cdot)\in\mathcal{U}[0,T].$

Therefore, Theorem 3.1 and Theorem 3.2 deduce the following conclusion
concerning the mean-variance portfolio selection problem.

######  Theorem 6.1.

SRE (6.4) admits a unique adapted solution
$(\hat{P}(\cdot,,\alpha(\cdot)),\hat{\Lambda}(\cdot),\hat{\zeta}(\cdot))\in$
$L_{\mathbb{F}}^{\infty}(0,T;\mathbb{R})\times
L_{\mathbb{F}}^{2}(0,T;\mathbb{R})\times L_{\mathbb{F}}^{2}(0,T;\mathbb{R}))$
and the mean-variance problem (6.3) is uniquely solvable. Moreover, the unique
optimal investment strategy $u^{*}(\cdot)$ has the following representation:

$u^{*}(s)=\Theta(s,\alpha(s))X^{*}(s),\quad s\in[0,T],$ (6.6)

where $\Theta(\cdot)$ is defined in (6.5) and $X^{*}(\cdot)$ is the solution
of the following closed-loop system:

$\left\\{\begin{aligned}
\displaystyle&dX^{*}(s)=\Big{\\{}r(s)+[b(s,\alpha(s))-r(s)]\Theta(s,\alpha(s))\Big{\\}}X^{*}(s)ds+\sigma(s,\alpha(s))\Theta(s,\alpha(s))X^{*}(s)dW(s),\
\ s\in[0,T],\\\ \vskip 3.0pt plus 1.0pt minus
1.0pt\cr\displaystyle&X^{*}(0)=x(0)-\gamma\exp\Big{\\{}-\int_{0}^{T}r(s)ds\Big{\\}},\quad\alpha(0)=i_{0}.\end{aligned}\right.$

######  Remark 6.2.

The above result partially develops the mean-variance problems of Li–Zhou–Lim
[14] to the Markovian regime switching system with random coefficients, which
goes beyond the framework of Sun–Xiong–Yong [20]. Note that we consider the
one-dimensional state case of two assets (one bond and one stock) to be just
for simplicity of presentation, and the multi-dimensional case of $m+1$ assets
(one bond and $m$ stocks) can be proved using a similar argument.

##  7 Conclusion

This paper extends the work of Sun–Xiong–Yong [20] to the framework within the
Markovian regime switching system, obtains the solvability of SRE (3.1) with
jumps and random coefficients, gets the closed-loop representation of the
open-loop optimal control, and gives a financial application of the
continuous-time mean-variance portfolio selection problem, which develops the
work of Li–Zhou–Lim [14]. In addition, a new point of view for the uniform
convexity of the cost functional is presented, and the equivalence between
Problem (M-SLQ)0 and Problem ${(\hbox{M-SLQ})}$ is obtained. Note that it
remains open if someone could get the closed-loop solvability in this model.

## Acknowledgements

The authors would like to thank the associate editor and the anonymous
referees for their insightful comments that improve the quality of this paper.

## References

  * [1] M. Ait Rami, J. Moore, and X. Zhou, Indefinite stochastic linear quadratic control and generalized differential Riccati equation, SIAM J. Control Optim., 40 (2001), pp. 1296–1311.
  * [2] J. M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM J. Control Optim., 14 (1976), pp. 419–444.
  * [3] S. Chen, X. Li, and X. Zhou, Stochastic linear quadratic regulators with indefinite control weight costs, SIAM J. Control Optim., 36 (1998), pp. 1685–1702.
  * [4] S. Chen and J. Yong, Stochastic linear quadratic optimal control problems, Appl. Math. Optim., 43 (2001), pp. 21–45.
  * [5] C. Donnelly and A.J. Heunis, Quadratic risk minimization in a regime-switching model with portfolio constraints, SIAM J. Control Optim., 50 (2012), pp. 2431–2461.
  * [6] R. A. Horn and C. R. Johnson, Matrix Analysis, 2nd ed., Cambridge, New York, 2012.
  * [7] Y. Hu and B. Oksendal, Partial information linear quadratic control for jump diffusions, SIAM J. Control Optim., 47, (2008), pp. 1744–1761.
  * [8] Y. Hu, X. Shi, and Z. Xu, Constrained stochastic LQ control with regime switching and application to portfolio selection, Ann. App. Probab., 32 (2022), pp. 426–460.
  * [9] Y. Ji and H. Chizeck, Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control, IEEE Trans. Autom. Control, 35 (1990), pp. 777–788.
  * [10] M. Kohlmann and S. Tang, New developments in backward stochastic riccati equations and their applications, in Mathematical Finance, edited by M. Kohlmann, S. Tang. Birkhauser, Basel (2001), pp. 194–214.
  * [11] M. Kohlmann and S. Tang, Multidimensional backward stochastic Riccati equations and applications, SIAM J. Control Optim., 41 (2003), pp. 1696–1721.
  * [12] H. Kushner, Optimal stochastic control, IRE Trans. Autom. Control, 7 (1962), pp. 120–122.
  * [13] N. Li, Z. Wu, and Z. Yu, Indefinite stochastic linear-quadratic optimal control problems with random jumps and related stochastic Riccati equations, Sci. China Math., 61 (2018), pp. 563–576.
  * [14] X. Li, X. Zhou, and A.E.B. Lim, Dynamic mean-variance portfolio selection with no-shorting constraints, SIAM J. Control Optim., 40 (2002), pp. 1540–1555.
  * [15] X. Li, X. Zhou, and M.A. Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon, J. Global Optim., 27 (2003), pp. 149–175.
  * [16] Y. Liu, G. Yin, and X. Zhou, Near-optimal controls of random-switching LQ problems with indefinite control weight costs, Automatica, 41 (2005), pp. 1063–1070.