# Mean-Field Linear-Quadratic Stochastic Differential Games

Jingrui Sun Hanxiao Wang Zhen Wu Department of Mathematics, Southern
University of Science and Technology, Shenzhen 518055, China (Email:
sunjr@sustech.edu.cn). This author is supported by NSFC Grant 11901280,
Guangdong Basic and Applied Basic Research Foundation 2021A1515010031, and
SUSTech start-up funds Y01286128 and Y01286228.Corresponding author.
Department of Mathematics, National University of Singapore, Singapore 119076,
Singapore (Email: mathxw@nus.edu.sg). This author is supported by Singapore
MOE AcRF Grant R-146-000-271-112.School of Mathematics, Shandong University,
Jinan 250100, China (Email: wuzhen@sdu.edu.cn). This author is supported by
NSFC Grants 11831010, 61961160732 and Shandong Provincial Natural Science
Foundation ZR2019ZD42.

Abstract. The paper is concerned with two-person zero-sum mean-field linear-
quadratic stochastic differential games over finite horizons. By a Hilbert
space method, a necessary condition and a sufficient condition are derived for
the existence of an open-loop saddle point. It is shown that under the
sufficient condition, the associated two Riccati equations admit unique
strongly regular solutions, in terms of which the open-loop saddle point can
be represented as a linear feedback of the current state. When the game only
satisfies the necessary condition, an approximate sequence is constructed by
solving a family of Riccati equations and closed-loop systems. The convergence
of the approximate sequence turns out to be equivalent to the open-loop
solvability of the game, and the limit is exactly an open-loop saddle point,
provided that the game is open-loop solvable.

Keywords. linear-quadratic differential game, mean-field stochastic
differential equation, two-person, zero-sum, open-loop saddle point, Riccati
equation, closed-loop representation, perturbation approach.

AMS subject classifications. 91A15, 93E20, 49N10, 49N70.

##  1 Introduction

Let $(\Omega,{\cal F},\mathbb{P})$ be a complete probability space on which a
standard one-dimensional Brownian motion $W=\\{W(t);0\leqslant t<\infty\\}$ is
defined. The augmented natural filtration of $W$ is denoted by
$\mathbb{F}=\\{{\cal F}_{t}\\}_{t\geqslant 0}$. Consider the following
controlled linear mean-field stochastic differential equation (MF-SDE, for
short) on a finite horizon $[0,T]$:

$\left\\{\begin{aligned}
dX(s)&=\big{\\{}A(s)X(s)+\bar{A}(s)\mathbb{E}[X(s)]+B_{1}(s)u_{1}(s)+\bar{B}_{1}(s)\mathbb{E}[u_{1}(s)]\\\
&~{}\hphantom{=}+B_{2}(s)u_{2}(s)+\bar{B}_{2}(s)\mathbb{E}[u_{2}(s)]\big{\\}}ds+\big{\\{}C(s)X(s)+\bar{C}(s)\mathbb{E}[X(s)]\\\
&~{}\hphantom{=}+D_{1}(s)u_{1}(s)+\bar{D}_{1}(s)\mathbb{E}[u_{1}(s)]+D_{2}(s)u_{2}(s)+\bar{D}_{2}(s)\mathbb{E}[u_{2}(s)]\big{\\}}dW(s),\\\
X(0)&=x,\end{aligned}\right.$ (1.1)

where $A,\bar{A},\,C,\bar{C}:[0,T]\to\mathbb{R}^{n\times n}$,
$B_{i},\bar{B}_{i},\,D_{i},\bar{D}_{i}:[0,T]\to\mathbb{R}^{n\times m_{i}}$
($i=1,2$), called the coefficients of the state equation (1.1), are given
deterministic functions. The solution $X$ of (1.1) is called a state process,
and $u_{i}$ ($i=1,2$), belonging to the space

${\cal
U}_{i}=\bigg{\\{}\varphi:[0,T]\times\Omega\to\mathbb{R}^{m_{i}}\bigm{|}\varphi\hbox{~{}is
$\mathbb{F}$-progressively
measurable},~{}\mathbb{E}\int^{T}_{0}|\varphi(s)|^{2}ds<\infty\bigg{\\}},$

is called the control process of Player $i$. To measure the performance of the
controls $u_{1}$ and $u_{2}$, we introduce the following functional:

$\displaystyle J(x;u_{1},u_{2})$ $\displaystyle=\mathbb{E}\Big{\\{}\langle
GX(T),X(T)\rangle+\langle\bar{G}\mathbb{E}[X(T)],\mathbb{E}[X(T)]\rangle$
$\displaystyle~{}\hphantom{=}+\int_{0}^{T}\left\langle\\!\begin{pmatrix}Q&S_{1}^{\top}&S_{2}^{\top}\\\
S_{1}&R_{11}&R_{12}\\\ S_{2}&R_{21}&R_{22}\end{pmatrix}\begin{pmatrix}X\\\
u_{1}\\\ u_{2}\end{pmatrix},\begin{pmatrix}X\\\ u_{1}\\\
u_{2}\end{pmatrix}\\!\right\rangle ds$
$\displaystyle~{}\hphantom{=}+\int_{0}^{T}\left\langle\\!\begin{pmatrix}\bar{Q}&\bar{S}_{1}^{\top}&\bar{S}_{2}^{\top}\\\
\bar{S}_{1}&\bar{R}_{11}&\bar{R}_{12}\\\
\bar{S}_{2}&\bar{R}_{21}&\bar{R}_{22}\end{pmatrix}\begin{pmatrix}\mathbb{E}[X]\\\
\mathbb{E}[u_{1}]\\\
\mathbb{E}[u_{2}]\end{pmatrix},\begin{pmatrix}\mathbb{E}[X]\\\
\mathbb{E}[u_{1}]\\\ \mathbb{E}[u_{2}]\end{pmatrix}\\!\right\rangle
ds\Bigg{\\}},$ (1.2)

where $G$ and $\bar{G}$ are $n\times n$ symmetric matrices;
$Q,\bar{Q}:[0,T]\to\mathbb{R}^{n\times n}$,
$S_{i},\bar{S}_{i}:[0,T]\to\mathbb{R}^{m_{i}\times n}$, and
$R_{ij},\bar{R}_{ij}:[0,T]\to\mathbb{R}^{m_{i}\times m_{j}}$ ($i,j=1,2$) are
deterministic functions with $Q=Q^{\top}$, $\bar{Q}=\bar{Q}^{\top}$,
$R_{ji}=R^{\top}_{ij}$ and $\bar{R}_{ji}=\bar{R}_{ij}^{\top}$ ($i,j=1,2$). In
the Lebesgue integral on the right-hand side of (1), we have suppressed the
argument $s$, and we will do so in the sequel as long as no ambiguity arises.

The functional $J(x;u_{1},u_{2})$ represents the cost of Player $1$ and the
payoff of Player $2$ for using $u_{1}$ and $u_{2}$ to control the state
process that starts from $x$. Naturally, in this two-person zero-sum mean-
field linear-quadratic stochastic differential game (Problem (MF-SG), for
short), Player 1 wishes to minimize (1) by selecting his/her control from
${\cal U}_{1}$, and Player 2 wishes to maximize (1) by selecting his/her
control from ${\cal U}_{2}$. The control pair $(u_{1}^{*},u_{2}^{*})$
acceptable to both players is called an open-loop saddle point of Problem (MF-
SG), which is mathematically defined by the following inequalities:

$J(x;u^{*}_{1},u_{2})\leqslant J(x;u^{*}_{1},u^{*}_{2})\leqslant
J(x;u_{1},u^{*}_{2}),\quad\forall(u_{1},u_{2})\in{\cal U}_{1}\times{\cal
U}_{2}.$ (1.3)

From (1.3) one sees that if one of the players keeps his/her control
$u_{i}^{*}$ unchanged, the other cannot benefit by changing his/her control.
In this sense, an open-loop saddle point (if exists) will be the best choice
for both players.

When $\bar{A}$, $\bar{B}_{i}$, $\bar{C}$, $\bar{D}_{i}$, $\bar{G}$, $\bar{Q}$,
$\bar{S}_{i}$ and $\bar{R}_{ij}$ ($i,j=1,2$) all vanish, Problem (MF-SG)
reduces to the classical two-person zero-sum linear-quadratic (LQ, for short)
stochastic differential game (Problem (SG), for short), which has been studied
for a long history and is widely applied in engineering, economy, and biology,
etc. Since the purpose of the paper is not to make a lengthy survey on the
literature, we only list here some closely related works (see, e.g., [6, 38,
21, 9, 10, 28, 37]) and refer the reader to the book [29] by Sun–Yong for more
details and references cited therein. It is particularly worthy to mention
that in a recent paper [24] by Sun, the strongly regular solvability of the
Riccati equation associated with Problem (SG) is established under the so-
called uniform convexity-concavity condition. This result brings new insights
into the two-person zero-sum LQ stochastic differential game and serves as a
foundation for our study on Problem (MF-SG).

Mean-field stochastic optimal control problems, which can be regarded as
special cases of the mean-field stochastic differential game in the sense that
one is interested in a single decision maker, have also attracted a lot of
attention; see, for example, [1, 2, 7, 22, 19, 4]. The mean-field LQ
stochastic optimal control problem was initially studied by Yong [33] and was
later generalized by Huang–Li–Yong [13], Yong [34], Sun [23], Li–Sun–Xiong
[17], and Sun–Wang [26] to various cases. Let us briefly recall the motivation
for studying mean-field LQ stochastic optimal control problems proposed by
Yong [33]. In some cases, one hopes that the optimal state process and/or
control process could be not too sensitive with respect to the possible
variation of the random events. To achieve this, one needs to keep the
variances var$[X]$ and var$[u]$ small. Therefore, it is natural to take
var$[X]$ and var$[u]$ into account and consider cost functionals of the form

$\displaystyle J(x;u)$ $\displaystyle=\mathbb{E}\Big{\\{}\langle
GX(T),X(T)\rangle+g\hbox{var$\,$}[X(T)]+\int_{0}^{T}\big{[}\langle
Q(s)X(s),X(s)\rangle$
$\displaystyle~{}\hphantom{=}+q(s)\hbox{var$\,$}[X(s)]+\langle
R(s)u(s),u(s)\rangle+r(s)\hbox{var$\,$}[u(s)]\big{]}ds\Big{\\}}.$ (1.4)

In particular, in the mean-variance model (see [39], for example), the cost
functional is simply $\mu\hbox{var$\,$}[X(T)]-\mathbb{E}[X(T)]$. Note that

$\hbox{var$\,$}[X(s)]=\mathbb{E}[|X(s)|^{2}]-(\mathbb{E}[X(s)])^{2},\quad\hbox{var$\,$}[u(s)]=\mathbb{E}[|u(s)|^{2}]-(\mathbb{E}[u(s)])^{2}.$

The control problem with functional (1) is actually a mean-field LQ control
problem, due to the presence of $(\mathbb{E}[X(s)])^{2}$ and
$(\mathbb{E}[u(s)])^{2}$. Another motivation for studying such type of
problems is that the mean-field SDEs, also called McKean–Vlasov SDEs, can be
used to describe particle systems at the mesoscopic level. Recently, the mean-
field/McKean–Vlasov SDE has been wildly used in mean-field game theory. In
general, it is the mean-square limit of the system of interacting particles
(see [2, 7], for example). For more details of such type of motivations, we
refer the reader to Huang–Malhamé–Caines [14], Lasry–Lions [15],
Bensoussan–Frehse–Yam [4], Carmona–Delarue [8], and the references cited
therein. Along with the development of mean-field LQ optimal control problems,
the LQ differential games for mean-field SDEs have also attracted extensive
research, among which, we would like to mention Bensoussan–Sung–Yam–Yung [5],
Graber [11], Barreiro-Gomez–Duncan–Tembine [3], Li–Shi–Yong [16], Moon [20],
and Tian–Yu–Zhang [30].

For mean-field LQ control problems, two Riccati equations are derived by Yong
[33] to construct an open-loop optimal control. The solvability of these two
Riccati equations is established in [33] under certain positivity conditions
and is further shown to be equivalent to the uniform convexity of the cost
functional by Sun [23]. However, to our best knowledge, there are few
significant results on the general solvability of the Riccati equations
associated with Problem (MF-SG) so far. One of the main contributions of this
paper is to fill up this gap. Compared with Sun [24], the additional
difficulty mainly comes from the solvability of the second Riccati equation
associated with Problem (MF-SG). To overcome this difficulty, besides
establishing a technical 2.3, we also show that the solution of the first
Riccati equation satisfies a comparison property (i.e., inequality (4.7) in
Theorem 4.2). This property follows from the following observations: One
control in the saddle point of Problem (SG) is optimal for a backward
stochastic LQ control problem and the value function of this backward problem
is given exactly in terms of the solution to the first Riccati equation. This
observation is interesting in its own right and to our best knowledge, it is
completely new in the literature.

In the literature, forward-backward stochastic differential equations (FBSDEs,
for short) are usually used to characterize the open-loop solvability of
Problem (MF-SG) (see, for example, [28, 29]). This method is suitable for
deciding whether a control pair is an open-loop saddle point or not, but not
very effective in constructing open-loop saddle points, because the associated
Riccati equations might be not solvable and then the optimality system cannot
be decoupled. This paper provides an alternative characterization (see Theorem
5.1) for the open-loop solvability of Problem (MF-SG) by a perturbation
approach, which can be regarded as another important contribution. It is
worthy to point out that the characterization is new even for Problem (SG) (in
which there are no mean-field terms present).

The idea is to add two terms, $\varepsilon\|u_{1}\|^{2}$ and
$-\varepsilon\|u_{2}\|^{2}$, to the original functional so that the Problem
(MF-SG) with the new functional $J_{\varepsilon}(x;u_{1},u_{2})\triangleq
J(x;u_{1},u_{2})+\varepsilon\|u_{1}\|^{2}-\varepsilon\|u_{2}\|^{2}$ admits a
unique open-loop point $(u_{1}^{\varepsilon},u_{2}^{\varepsilon})$ that can be
represented explicitly in terms of the solutions to the associated Riccati
equations. Then using the boundedness/convergence of the family
$\\{(u_{1}^{\varepsilon},u_{2}^{\varepsilon})\\}_{\varepsilon>0}$ to justify
the open-loop solvability of the original game. The main difficulty here is
that the value function $V_{\varepsilon}(x)$ of the perturbed game is not
monotone in $\varepsilon$, due to which the technique used in the LQ control
problem (see [25]) cannot be applied directly. The significant difference
between the perturbation methods of controls and games is illustrated by
presenting an elaborate example (see 5.3). To overcome the difficulty, we
restudy the perturbation approach by a Hilbert space method, which helps us to
change the boundedness problem of
$\\{(u_{1}^{\varepsilon},u_{2}^{\varepsilon})\\}_{\varepsilon>0}$ into an
equivalent one: the norm estimate of some perturbed operators with special
structures (see 2.4). Furthermore, it is found that the explicit upper bound
estimate (2.5) in 2.4 also plays a crucial role in proving the strong
convergence of
$\\{(u_{1}^{\varepsilon},u_{2}^{\varepsilon})\\}_{\varepsilon>0}$, because in
Theorem 5.1 we hope to show that
$\\{(u_{1}^{\varepsilon},u_{2}^{\varepsilon})\\}_{\varepsilon>0}$ itself is
strongly convergent when Problem (MF-SG) is open-loop solvable.

To summarize, we list the main contributions of the paper as follows.

(1) The open-loop solvability of Problem (MF-SG) is studied by a Hilbert space
method. A necessary and sufficient condition for the existence of an open-loop
saddle point is derived (see 3.1).

(2) Under the uniform convexity-concavity condition, the strongly regular
solvability of the Riccati equations associated with Problem (MF-SG) is
established (see Theorem 4.2 and Theorem 4.4). Further, in terms of the
solutions to the Riccati equations, a closed-loop representation of the unique
open-loop saddle point is obtained (see Theorem 4.6).

(3) Under a necessary condition for the existence of an open-loop saddle
point, an equivalent characterization of the open-loop solvability is
established by a perturbation approach. This approach also provides an
explicit procedure for finding open-loop saddle points (see Theorem 5.1).

In other words, under the uniform convexity-concavity condition, we first
extend the results obtained in Sun [24] to the mean-field system. As explained
before, to prove the solvability of the associated Riccati equations, we need
to make some new observations and to overcome some new difficulties. Then
under the weaker convexity-concavity condition, we develop a perturbation
approach to characterize the open-loop solvability of Problem (MF-SG). This
approach is first established for the game problem and can be regarded as the
most technical part in the paper.

The rest of the paper is organized as follows. Section 2 collects some
preliminary results. Section 3 is devoted to the study of the performance
functional from a Hilbert space point of view. Section 4 establishes the
solvability of the associated Riccati equations and provides a closed-loop
representation of the open-loop saddle point. Section 5 investigates the open-
loop solvability of Problem (MF-SG) by a perturbation method. An example is
presented in Section 6 to illustrate the results obtained in previous
sections.

##  2 Preliminaries

Throughout this paper, let $\mathbb{R}^{n\times m}$ be the Euclidean space
consisting of $n\times m$ real matrices, endowed with the Frobenius inner
product $\langle M,N\rangle\triangleq\hbox{tr$\,$}[M^{\top}N]$, where
$M^{\top}$ and $\hbox{tr$\,$}(M)$ stand for the transpose and the trace of
$M$, respectively. The norm of a matrix $M$ induced by the Frobenius inner is
denoted by $|M|$ and the identity matrix of size $n$ is denoted by $I_{n}$.
Let $\mathbb{S}^{n}$ be the subspace of $\mathbb{R}^{n\times n}$ consisting of
symmetric matrices and $\mathbb{S}^{n}_{+}$ be the subset of $\mathbb{S}^{n}$
consisting of positive semidefinite matrices. For any Euclidean space
$\mathbb{H}$ (which could be $\mathbb{R}^{n}$, $\mathbb{R}^{n\times m}$,
$\mathbb{S}^{n}$, etc.), we introduce the following spaces:

$\displaystyle C([0,T];\mathbb{H}):$ $\displaystyle\hbox{~{}~{}the space of
$\mathbb{H}$-valued, continuous functions on $[0,T]$};$ $\displaystyle
L^{\infty}(0,T;\mathbb{H}):$ $\displaystyle\hbox{~{}~{}the space of
$\mathbb{H}$-valued, essentially bounded functions on $[0,T]$};$
$\displaystyle L^{2}_{\mathcal{F}_{T}}(\Omega;\mathbb{H}):$ the space of
$\mathcal{F}_{T}$-measurable, $\mathbb{H}$-valued random variables $\xi$
$\displaystyle\hbox{~{}~{}such that $\mathbb{E}|\xi|^{2}<\infty$};$
$\displaystyle L_{\mathbb{F}}^{2}(0,T;\mathbb{H}):$ the space of
$\mathbb{F}$-progressively measurable, $\mathbb{H}$-valued processes
$\displaystyle\hbox{~{}~{}$\varphi:[0,T]\times\Omega\to\mathbb{H}$ with
$\displaystyle\mathbb{E}\int_{0}^{T}|\varphi(s)|^{2}ds<\infty$};$
$\displaystyle L_{\mathbb{F}}^{2}(\Omega;C([0,T];\mathbb{H})):$ the space of
$\mathbb{F}$-adapted, continuous, $\mathbb{H}$-valued processes
$\displaystyle\hbox{~{}~{}$\varphi:[0,T]\times\Omega\to\mathbb{H}$ with
$\mathbb{E}\Big{[}\displaystyle\sup_{s\in[0,T]}|\varphi(s)|^{2}\Big{]}<\infty$}.$

We denote the norm of the Banach space ${\cal X}$ by $\|\cdot\|_{{\cal X}}$,
which is often simply written as $\|\cdot\|$ when no confusion occurs. For
$M,N\in\mathbb{S}^{n}$, we use the notation $M\geqslant N$ (respectively,
$M>N$) to indicate that $M-N$ is positive semidefinite (respectively, positive
definite). For any $\mathbb{S}^{n}$-valued measurable function $F$ on $[0,T]$,
we denote

$\left\\{\begin{aligned} F\geqslant 0\quad&\Longleftrightarrow\quad
F(s)\geqslant 0,\quad\hbox{a.e.}~{}s\in[0,T];\\\
F>0\quad&\Longleftrightarrow\quad F(s)>0,\quad\hbox{a.e.}~{}s\in[0,T];\\\ F\gg
0\quad&\Longleftrightarrow\quad F(s)\geqslant\delta
I_{n},\quad\hbox{a.e.}~{}s\in[0,T],~{}\hbox{for some
}\delta>0.\end{aligned}\right.$

For self-adjoint linear operators ${\cal M}$ and ${\cal N}$ defined on the
Hilbert space ${\cal H}$, we call ${\cal M}$ a positive operator if
$\langle{\cal M}x,x\rangle\geqslant 0;\,\forall x\in{\cal H}$ (see [36, page
317, Definition 2]), and we use ${\cal M}\geqslant{\cal N}$ to indicate that
${\cal M}-{\cal N}$ is a positive operator.

To guarantee that Problem (MF-SG) is well-posed, we impose the following
assumptions for the state equation (1.1) and the functional (1).

###### (H1)

The coefficients of state equation (1.1) satisfy

$A,\bar{A},C,\bar{C}\in L^{\infty}(0,T;\mathbb{R}^{n\times n});\quad
B_{i},\bar{B}_{i},D_{i},\bar{D}_{i}\in L^{\infty}(0,T;\mathbb{R}^{n\times
m_{i}}),~{}i=1,2.$

###### (H2)

The weighting matrices in the quadratic functional (1) satisfy:
$G,\bar{G}\in\mathbb{S}^{n}$, $Q,\bar{Q}\in L^{\infty}(0,T;\mathbb{S}^{n})$,
and for $i,j=1,2$,

$S_{i},\bar{S}_{i}\in L^{\infty}(0,T;\mathbb{R}^{m_{i}\times n}),\quad
R_{ij},\bar{R}_{ij}\in L^{\infty}(0,T;\mathbb{R}^{m_{i}\times m_{j}}),\quad
R_{ij}^{\top}=R_{ji},\quad\bar{R}_{ij}^{\top}=\bar{R}_{ji}.$

Under (H1), by [33, Proposition 2.1], state equation (1.1) admits a unique
solution $X\in L^{2}_{\mathbb{F}}(\Omega;C([0,T];\mathbb{R}^{n}))$. If the
assumption (H2) also holds, the random variables on the right-hand side of (1)
are integrable and Problem (MF-SG) is well-posed. Now we recall two important
notions of LQ game problems.

######  Definition 2.1.

A control pair $(u_{1}^{*},u_{2}^{*})\in{\cal U}_{1}\times{\cal U}_{2}$ is
called an open-loop saddle point of Problem (MF-SG) for the initial state
$x\in\mathbb{R}^{n}$ if

$J(x;u^{*}_{1},u_{2})\leqslant J(x;u^{*}_{1},u^{*}_{2})\leqslant
J(x;u_{1},u^{*}_{2}),\quad\forall(u_{1},u_{2})\in{\cal U}_{1}\times{\cal
U}_{2}.$ (2.1)

Problem (MF-SG) is said to be open-loop solvable at $x$, if it has an open-
loop saddle point for $x$.

######  Definition 2.2.

For any $x\in\mathbb{R}^{n}$, $V(x)$ is called a value of Problem (MF-SG) at
$x$ if

$V(x)=\inf_{u_{1}\in{\cal U}_{1}}\sup_{u_{2}\in{\cal
U}_{2}}J(x;u_{1},u_{2})=\sup_{u_{2}\in{\cal U}_{2}}\inf_{u_{1}\in{\cal
U}_{1}}J(x;u_{1},u_{2}).$ (2.2)

Note that the value function $V$ is well-defined at $x\in\mathbb{R}^{n}$ only
when the second equality in (2.2) holds. If $(u_{1}^{*},u_{2}^{*})\in{\cal
U}_{1}\times{\cal U}_{2}$ is an open-loop saddle point for $x$, then

$V(x)=J(x;u_{1}^{*},u_{2}^{*}).$ (2.3)

In the following, let us make some preparations for the subsequent analysis of
our main results. We first present a primary lemma, which seems to be new and
is crucial to proving the solvability of the Riccati equations. The proof is
sketched in Appendix for completeness.

######  Lemma 2.3.

Let $M\in\mathbb{S}^{n}_{+}$, $K\in\mathbb{R}^{n\times m}$ and
$L\in\mathbb{R}^{n\times n}$. Then for any $\delta>0$,

$L^{\top}ML-L^{\top}MK(K^{\top}MK+\delta I_{m})^{-1}K^{\top}ML\geqslant 0.$
(2.4)

The following result is concerned with bounded linear operators, by which we
shall develop a perturbation approach for the open-loop solvability of Problem
(MF-SG) in Section 5. Let ${\cal H}_{1}$ and ${\cal H}_{2}$ be two real
Hilbert spaces. Let

${\cal M}_{ij}:{\cal H}_{j}\to{\cal H}_{i},\quad i,j=1,2$

be linear bounded operators with ${\cal M}_{ji}={\cal M}_{ij}^{*}$, where
${\cal M}_{ij}^{*}$ denotes the adjoint operator of ${\cal M}_{ij}$. Then

${\cal M}\triangleq\begin{pmatrix}{\cal M}_{11}&{\cal M}_{12}\\\ {\cal
M}_{21}&{\cal M}_{22}\end{pmatrix}$

is a self-adjoint linear bounded operator on the product Hilbert space ${\cal
H}_{1}\times{\cal H}_{2}$ equipped with the inner product

$\left\langle\\!\begin{pmatrix}x_{1}\\\
y_{1}\end{pmatrix},\begin{pmatrix}x_{2}\\\
y_{2}\end{pmatrix}\\!\right\rangle\triangleq\langle x_{1},x_{2}\rangle+\langle
y_{1},y_{2}\rangle,\quad\forall x_{1},x_{2}\in{\cal
H}_{1},~{}y_{1},y_{2}\in{\cal H}_{2}.$

######  Proposition 2.4.

Suppose that ${\cal M}_{11}$ and $-{\cal M}_{22}$ are positive operators; that
is ${\cal M}_{11}\geqslant 0$ and $-{\cal M}_{22}\geqslant 0$. Then for any
$\varepsilon>0$,

${\cal M}_{\varepsilon}\triangleq\begin{pmatrix}{\cal M}_{11}+\varepsilon
I&{\cal M}_{12}\\\ {\cal M}_{21}&{\cal M}_{22}-\varepsilon I\end{pmatrix}$

is invertible. Moreover,

$\displaystyle\|{\cal M}_{\varepsilon}^{-1}{\cal M}\|\leqslant
1,\quad\forall\varepsilon>0.$ (2.5)

###### Proof.

Since ${\cal M}_{11}+\varepsilon I\geqslant\varepsilon I$, ${\cal
M}_{11,\varepsilon}\triangleq{\cal M}_{11}+\varepsilon I$ is invertible with
$\|{\cal M}_{11,\varepsilon}^{-1}\|\leqslant\varepsilon^{-1}$. Similarly,
${\cal M}_{22,\varepsilon}\triangleq{\cal M}_{22}-\varepsilon I$ is invertible
with $\|{\cal M}_{22,\varepsilon}^{-1}\|\leqslant\varepsilon^{-1}$, and the
self-adjoint operator

$\Phi_{\varepsilon}\triangleq{\cal M}_{22,\varepsilon}-{\cal M}_{21}{\cal
M}_{11,\varepsilon}^{-1}{\cal M}_{12}\equiv{\cal M}_{22,\varepsilon}-{\cal
M}_{12}^{*}{\cal M}_{11,\varepsilon}^{-1}{\cal M}_{12}$

is invertible with $\|\Phi_{\varepsilon}^{-1}\|\leqslant\varepsilon^{-1}$. Now
it is straightforward to verify that ${\cal M}_{\varepsilon}$ is invertible
with inverse

${\cal M}_{\varepsilon}^{-1}=\begin{pmatrix}{\cal
M}_{11,\varepsilon}^{-1}+({\cal M}_{11,\varepsilon}^{-1}{\cal
M}_{12})\Phi_{\varepsilon}^{-1}({\cal M}_{11,\varepsilon}^{-1}{\cal
M}_{12})^{*}&-({\cal M}_{11,\varepsilon}^{-1}{\cal
M}_{12})\Phi_{\varepsilon}^{-1}\\\\[2.84526pt] -\Phi_{\varepsilon}^{-1}({\cal
M}_{11,\varepsilon}^{-1}{\cal
M}_{12})^{*}&\Phi_{\varepsilon}^{-1}\end{pmatrix}.$

To prove (2.5), we write

${\cal M}_{\varepsilon}^{-1}{\cal M}=\begin{pmatrix}I&0\\\
0&I\end{pmatrix}-\varepsilon{\cal
M}_{\varepsilon}^{-1}\begin{pmatrix}[r]I&0\\\
0&-I\end{pmatrix}=\begin{pmatrix}I&0\\\
0&I\end{pmatrix}-\varepsilon\begin{pmatrix}{\cal M}_{11,\varepsilon}&{\cal
M}_{12}\\\ -{\cal M}_{12}^{*}&-{\cal M}_{22,\varepsilon}\end{pmatrix}^{-1}.$

Denote

$\hat{\cal M}_{\varepsilon}\triangleq\begin{pmatrix}{\cal
M}_{11,\varepsilon}&{\cal M}_{12}\\\ -{\cal M}_{12}^{*}&-{\cal
M}_{22,\varepsilon}\end{pmatrix}.$ (2.6)

Then

$\displaystyle{\cal M}_{\varepsilon}^{-1}{\cal M}=I-\varepsilon\hat{\cal
M}_{\varepsilon}^{-1},$ (2.7)

and thus

$\displaystyle({\cal M}_{\varepsilon}^{-1}{\cal M})^{*}{\cal
M}_{\varepsilon}^{-1}{\cal M}$ $\displaystyle=(I-\varepsilon\hat{\cal
M}_{\varepsilon}^{-1})^{*}(I-\varepsilon\hat{\cal M}_{\varepsilon}^{-1})$
$\displaystyle=I-\varepsilon(\hat{\cal
M}_{\varepsilon}^{-1})^{*}-\varepsilon\hat{\cal
M}_{\varepsilon}^{-1}+\varepsilon^{2}(\hat{\cal
M}_{\varepsilon}^{-1})^{*}\hat{\cal M}_{\varepsilon}^{-1}$
$\displaystyle=I-\varepsilon(\hat{\cal
M}_{\varepsilon}^{-1})^{*}\big{[}\hat{\cal M}_{\varepsilon}+\hat{\cal
M}_{\varepsilon}^{*}-\varepsilon I\big{]}\hat{\cal M}_{\varepsilon}^{-1}.$
(2.8)

Note that

$\displaystyle\hat{\cal M}_{\varepsilon}+\hat{\cal
M}_{\varepsilon}^{*}-\varepsilon I$ $\displaystyle=\begin{pmatrix}{\cal
M}_{11,\varepsilon}&{\cal M}_{12}\\\ -{\cal M}_{12}^{*}&-{\cal
M}_{22,\varepsilon}\end{pmatrix}+\begin{pmatrix}{\cal
M}_{11,\varepsilon}&-{\cal M}_{12}\\\ {\cal M}_{12}^{*}&-{\cal
M}_{22,\varepsilon}\end{pmatrix}-\varepsilon\begin{pmatrix}I&0\\\
0&I\end{pmatrix}$ $\displaystyle=\begin{pmatrix}2{\cal M}_{11}+\varepsilon
I&0\\\ 0&-2{\cal M}_{22}+\varepsilon I\end{pmatrix}.$

Using the fact that ${\cal M}_{11}$ and $-{\cal M}_{22}$ are positive
operators, we have

$\displaystyle\begin{pmatrix}2{\cal M}_{11}+\varepsilon I&0\\\ 0&-2{\cal
M}_{22}+\varepsilon I\end{pmatrix}\geqslant 0;$

that is

$\hat{\cal M}_{\varepsilon}+\hat{\cal M}_{\varepsilon}^{*}-\varepsilon
I\geqslant 0,$

which implies that

$\varepsilon(\hat{\cal M}_{\varepsilon}^{-1})^{*}\big{[}\hat{\cal
M}_{\varepsilon}+\hat{\cal M}_{\varepsilon}^{*}-\varepsilon I\big{]}\hat{\cal
M}_{\varepsilon}^{-1}\geqslant 0.$

Combining the above with (2.8) yields

$\displaystyle 0\leqslant({\cal M}_{\varepsilon}^{-1}{\cal M})^{*}{\cal
M}_{\varepsilon}^{-1}{\cal M}=I-\varepsilon(\hat{\cal
M}_{\varepsilon}^{-1})^{*}\big{[}\hat{\cal M}_{\varepsilon}+\hat{\cal
M}_{\varepsilon}^{*}-\varepsilon I\big{]}\hat{\cal
M}_{\varepsilon}^{-1}\leqslant I.$ (2.9)

Thus, we get

$\|{\cal M}_{\varepsilon}^{-1}{\cal M}\|^{2}=\|({\cal
M}_{\varepsilon}^{-1}{\cal M})^{*}{\cal M}_{\varepsilon}^{-1}{\cal
M}\|\leqslant 1.$

The proof is complete. ∎

##  3 Representation of the functional

In this section, we shall study the functional (1) from a Hilbert space point
of view and represent it as a quadratic functional of the controls
$(u_{1},u_{2})$, by which a necessary condition and a sufficient condition
will be derived for the open-loop solvability. For any $u_{i}\in{\cal U}_{i}$
($i=1,2$), consider the following MF-SDE:

$\left\\{\begin{aligned}
dX^{i,0}(s)&=\big{\\{}A(s)X^{i,0}(s)+\bar{A}(s)\mathbb{E}[X^{i,0}(s)]+B_{i}(s)u_{i}(s)+\bar{B}_{i}(s)\mathbb{E}[u_{i}(s)]\big{\\}}ds\\\
&\hphantom{=}~{}+\big{\\{}C(s)X^{i,0}(s)+\bar{C}(s)\mathbb{E}[X^{i,0}(s)]+D_{i}(s)u_{i}(s)\\\
&\hphantom{=}~{}+\bar{D}_{i}(s)\mathbb{E}[u_{i}(s)]\big{\\}}dW(s),\quad
s\in[0,T],\\\ X^{i,0}(0)&=0.\end{aligned}\right.$ (3.1)

Under (H1), the above MF-SDE admits a unique solution $X^{i,0}\in
L_{\mathbb{F}}^{2}(\Omega;C([0,T];\mathbb{R}^{n}))$ satisfying

$\mathbb{E}\Big{[}\sup_{s\in[0,T]}|X^{i,0}(s)|^{2}\Big{]}\leqslant
K\mathbb{E}\int_{0}^{T}|u_{i}(s)|^{2}ds,$ (3.2)

where the constant $K>0$ is independent of $u_{i}$. Thus we can define two
bounded linear operators ${\cal L}_{i}:{\cal U}_{i}\to
L_{\mathbb{F}}^{2}(\Omega;C([0,T];\mathbb{R}^{n}))$ and $\hat{\cal
L}_{i}:{\cal U}_{i}\to L_{{\cal F}_{T}}^{2}(\Omega;\mathbb{R}^{n})$ as
follows:

${\cal L}_{i}u_{i}=X^{i,0},\quad\hat{\cal L}_{i}u_{i}=X^{i,0}(T),\quad\forall
u_{i}\in{\cal U}_{i};\quad i=1,2.$ (3.3)

Also we can define the linear operators ${\cal N}:\mathbb{R}^{n}\to
L_{\mathbb{F}}^{2}(\Omega;C([0,T];\mathbb{R}^{n}))$ and $\hat{\cal
N}:\mathbb{R}^{n}\to L_{{\cal F}_{T}}^{2}(\Omega;\mathbb{R}^{n})$ as follows:

${\cal N}x=X^{0,x},\quad\hat{\cal N}x=X^{0,x}(T),\quad\forall
x\in\mathbb{R}^{n},$ (3.4)

with $X^{0,x}$ being the unique solution to the following MF-SDE:

$\left\\{\begin{aligned}
dX^{0,x}(s)&=\big{\\{}A(s)X^{0,x}(s)+\bar{A}(s)\mathbb{E}[X^{0,x}(s)]\big{\\}}ds\\\
&\hphantom{=}~{}+\big{\\{}C(s)X^{0,x}(s)+\bar{C}(s)\mathbb{E}[X^{0,x}(s)]\big{\\}}dW(s),\quad
s\in[0,T],\\\ X^{0,x}(0)&=x.\end{aligned}\right.$ (3.5)

For any given $(x,u_{1},u_{2})\in\mathbb{R}^{n}\times{\cal U}_{1}\times{\cal
U}_{2}$, it is easily checked that $X^{0,x}+X^{1,0}+X^{2,0}$ satisfies state
equation (1.1). Thus, by the uniqueness of the solution to MF-SDE (1.1), we
have

$X=X^{0,x}+X^{1,0}+X^{2,0}={\cal N}x+{\cal L}_{1}u_{1}+{\cal
L}_{2}u_{2},\quad\forall(x,u_{1},u_{2})\in\mathbb{R}^{n}\times{\cal
U}_{1}\times{\cal U}_{2}.$ (3.6)

In particular, the terminal value of $X$ can be represented by

$\displaystyle X(T)$ $\displaystyle=X^{0,x}(T)+X^{1,0}(T)+X^{2,0}(T)$
$\displaystyle=\hat{\cal N}x+\hat{\cal L}_{1}u_{1}+\hat{\cal
L}_{2}u_{2},\quad\forall(x,u_{1},u_{2})\in\mathbb{R}^{n}\times{\cal
U}_{1}\times{\cal U}_{2}.$ (3.7)

Then using (3.6)–(3.7), by the completion of squares technique it is
straightforward to obtain the following representation of the functional (1):

$\displaystyle J(x;u_{1},u_{2})=\langle{\cal M}u,u\rangle+2\langle{\cal
K}x,u\rangle+\langle{\cal O}x,x\rangle,$ $\displaystyle~{}\forall
x\in\mathbb{R}^{n},~{}u=(u_{1}^{\top},u^{\top}_{2})^{\top}\in{\cal
U}_{1}\times{\cal U}_{2},$ (3.8)

where

${\cal M}\triangleq\begin{pmatrix}{\cal M}_{11}&{\cal M}_{12}\\\ {\cal
M}_{21}&{\cal M}_{22}\end{pmatrix},~{}{\cal K}\triangleq\begin{pmatrix}{\cal
K}_{1}\\\ {\cal K}_{2}\end{pmatrix},~{}{\cal O}\triangleq\hat{\cal
N}^{*}(G+\mathbb{E}^{*}\bar{G}\mathbb{E})\hat{\cal N}+{\cal
N}^{*}(Q+\mathbb{E}^{*}\bar{Q}\mathbb{E}){\cal N},$ (3.9)

with

$\displaystyle{\cal M}_{ij}\triangleq\hat{\cal
L}_{j}^{*}(G+\mathbb{E}^{*}\bar{G}\mathbb{E})\hat{\cal L}_{i}+{\cal
L}_{j}^{*}(Q+\mathbb{E}^{*}\bar{Q}\mathbb{E}){\cal
L}_{i}+R_{ji}+\mathbb{E}^{*}\bar{R}_{ji}\mathbb{E}$
$\displaystyle\qquad\quad+(S_{j}+\mathbb{E}^{*}\bar{S}_{j}\mathbb{E}){\cal
L}_{i}+{\cal
L}_{j}^{*}(S_{i}^{\top}+\mathbb{E}^{*}\bar{S}_{i}^{\top}\mathbb{E}),\quad
i,j=1,2;$ (3.10) $\displaystyle{\cal K}_{i}\triangleq\hat{\cal
L}_{i}^{*}(G+\mathbb{E}^{*}\bar{G}\mathbb{E})\hat{\cal N}+{\cal
L}_{i}^{*}(Q+\mathbb{E}^{*}\bar{Q}\mathbb{E}){\cal
N}+(S_{i}+\mathbb{E}^{*}\bar{S}_{i}\mathbb{E}){\cal N},\quad i=1,2.$ (3.11)

By the above expression of ${\cal M}_{ij};i,j=1,2$, we have ${\cal
M}_{ji}={\cal M}_{ij}^{*};\,i,j=1,2$, which implies that ${\cal M}$ is a self-
adjoint operator. With the representation (3.8), we provide the following
characterization for the open-loop saddle points of Problem (MF-MG).

######  Proposition 3.1.

Let (H1)–(H2) hold. Let $x\in\mathbb{R}^{n}$ be any given initial state and
$u^{*}=(u_{1}^{*\top},u_{2}^{*\top})^{\top}\in{\cal U}_{1}\times{\cal U}_{2}$.
Then $u^{*}$ is an open-loop saddle point of Problem (MF-MG) for $x$ if and
only if

$(-1)^{i+1}{\cal M}_{ii}\geqslant 0;~{}i=1,2\quad\hbox{and}\quad{\cal
M}u^{*}+{\cal K}x=0.$ (3.12)

###### Proof.

By 2.1, $u^{*}=(u_{1}^{*\top},u_{2}^{*\top})^{\top}$ is an open-loop saddle
point of Problem (MF-MG) if and only if

$\displaystyle J(x;u_{1}^{*}+\lambda
u_{1},u_{2}^{*})-J(x;u_{1}^{*},u_{2}^{*})\geqslant 0,\quad\forall
u_{1}\in{\cal U}_{1},~{}\lambda\in\mathbb{R};$ (3.13) $\displaystyle
J(x;u_{1}^{*},u_{2}^{*}+\lambda u_{2})-J(x;u_{1}^{*},u_{2}^{*})\leqslant
0,\quad\forall u_{2}\in{\cal U}_{2},~{}\lambda\in\mathbb{R}.$ (3.14)

For any $u_{1}\in{\cal U}_{1}$ and $\lambda\in\mathbb{R}$, by (3.8) we have

$\displaystyle J(x;u_{1}^{*}+\lambda
u_{1},u_{2}^{*})-J(x;u_{1}^{*},u_{2}^{*})$
$\displaystyle\quad=\lambda^{2}\big{\langle}{\cal
M}_{11}u_{1},u_{1}\big{\rangle}+2\lambda\big{[}\big{\langle}{\cal
M}_{11}u^{*}_{1},u_{1}\big{\rangle}+\big{\langle}{\cal
M}_{12}u^{*}_{2},u_{1}\big{\rangle}+\big{\langle}{\cal
K}_{1}x,u_{1}\big{\rangle}\big{]}.$ (3.15)

Thus (3.13) holds if and only if

${\cal M}_{11}\geqslant 0\quad\hbox{and}\quad{\cal M}_{11}u^{*}_{1}+{\cal
M}_{12}u^{*}_{2}+{\cal K}_{1}x=0.$ (3.16)

By the same argument as the above, we can show that (3.14) holds if and only
if

${\cal M}_{22}\leqslant 0\quad\hbox{and}\quad{\cal M}_{22}u^{*}_{2}+{\cal
M}_{21}u^{*}_{1}+{\cal K}_{2}x=0.$ (3.17)

Note that (3.12) is equivalent to (3.16) and (3.17). The proof is thus
complete. ∎

From 3.1, we see that the following convexity-concavity condition,

$\displaystyle\langle{\cal M}_{11}u_{1},u_{1}\rangle$
$\displaystyle=J(0;u_{1},0)\geqslant 0,\quad\forall u_{1}\in{\cal U}_{1};$
(3.18) $\displaystyle\langle{\cal M}_{22}u_{2},u_{2}\rangle$
$\displaystyle=J(0;0,u_{2})\leqslant 0,\quad\forall u_{2}\in{\cal U}_{2},$

is necessary for the existence of an open-loop saddle point. Next we introduce
a condition slightly stronger than (3.18):

###### (H3)

There exists a constant $\alpha>0$ such that

$\displaystyle\langle{\cal M}_{11}u_{1},u_{1}\rangle$
$\displaystyle=J(0;u_{1},0)\geqslant\alpha\|u_{1}\|^{2},\quad~{}~{}\forall
u_{1}\in{\cal U}_{1};$ (3.19) $\displaystyle\langle{\cal
M}_{22}u_{2},u_{2}\rangle$
$\displaystyle=J(0;0,u_{2})\leqslant-\alpha\|u_{2}\|^{2},\quad\forall
u_{2}\in{\cal U}_{2}.$

If (H3) holds, for convenience we usually write (3.19) as follows:

$\displaystyle\langle{\cal M}_{11}u_{1},u_{1}\rangle$
$\displaystyle=J(0;u_{1},0)\gg 0,\quad\forall u_{1}\in{\cal U}_{1};$
$\displaystyle\langle{\cal M}_{22}u_{2},u_{2}\rangle$
$\displaystyle=J(0;0,u_{2})\ll 0,\quad\forall u_{2}\in{\cal U}_{2}.$

The following result shows that the uniform convexity-concavity condition (H3)
is sufficient for the open-loop solvability of Problem (MF-MG).

######  Proposition 3.2.

Let (H1)–(H3) hold, and the operators ${\cal M}$ and ${\cal K}$ be defined by
(3.9). Then ${\cal M}$ is invertible and for any $x\in\mathbb{R}^{n}$, Problem
(MF-MG) admits a unique open-loop saddle point
$u^{*}=(u_{1}^{*\top},u_{2}^{*\top})^{\top}$ given by

$u^{*}=-{\cal M}^{-1}{\cal K}x.$ (3.20)

###### Proof.

By (3.19), we obtain that the operators ${\cal M}_{11}$, ${\cal M}_{22}$ and
$\Phi\triangleq{\cal M}_{22}-{\cal M}_{21}{\cal M}_{11}^{-1}{\cal M}_{12}$ are
invertible. Then it is straightforward to verify that ${\cal M}$ is invertible
with inverse

${\cal M}^{-1}=\begin{pmatrix}{\cal M}_{11}^{-1}+({\cal M}_{11}^{-1}{\cal
M}_{12})\Phi^{-1}({\cal M}_{11}^{-1}{\cal M}_{12})^{*}&-({\cal
M}_{11}^{-1}{\cal M}_{12})\Phi^{-1}\\\\[2.84526pt] -\Phi^{-1}({\cal
M}_{11}^{-1}{\cal M}_{12})^{*}&\Phi^{-1}\end{pmatrix}.$ (3.21)

The remaining results follow from 3.1 directly. ∎

##  4 Open-loop saddle points and Riccati equations

According to 3.2, under the uniform convexity-concavity condition (H3), the
open-loop saddle point for the given $x\in\mathbb{R}^{n}$ can be uniquely
determined by (3.20). However, since ${\cal M}^{-1}{\cal K}$ is an abstract
operator and very complicated, it is usually difficult to find the open-loop
saddle point by computing (3.20) directly. Thus in this section, we shall give
a more explicit form of the open-loop saddle point by introducing two
associated Riccati equations. Furthermore, it will be shown that the unique
open-loop saddle point admits a closed-loop representation.

Recall from [24] that the Riccati equation associated with Problem (SG) reads

$\left\\{\begin{aligned} &\dot{P}+PA+A^{\top}P+C^{\top}PC+Q\\\
&\hphantom{\dot{P}}-(PB+C^{\top}PD+S^{\top})(R+D^{\top}PD)^{-1}(B^{\top}P+D^{\top}PC+S)=0,\\\
&P(T)=G,\end{aligned}\right.$ (4.1)

where

$\displaystyle B=(B_{1},B_{2}),\quad D=(D_{1},D_{2}),\quad
S=\begin{pmatrix}S_{1}\\\ S_{2}\end{pmatrix},\quad
R=\begin{pmatrix}R_{11}&R_{12}\\\ R_{21}&R_{22}\end{pmatrix},$ $\displaystyle
R+D^{\top}PD=\begin{pmatrix}R_{11}+D_{1}^{\top}PD_{1}&R_{12}+D_{1}^{\top}PD_{2}\\\
R_{21}+D_{2}^{\top}PD_{1}&R_{22}+D_{2}^{\top}PD_{2}\end{pmatrix},$
$\displaystyle
B^{\top}P+D^{\top}PC+S=\begin{pmatrix}B_{1}^{\top}P+D_{1}^{\top}PC+S_{1}\\\
B_{2}^{\top}P+D_{2}^{\top}PC+S_{2}\end{pmatrix}.$ (4.2)

######  Definition 4.1.

An absolutely continuous function $P:[0,T]\to\mathbb{S}^{n}$ is called a
strongly regular solution of Riccati equation (4.1) if

  1. (i)

For $i=1,2$, $(-1)^{i+1}[R_{ii}+D_{i}^{\top}PD_{i}]\gg 0$, and

  2. (ii)

$P$ satisfies (4.1) almost everywhere on $[0,T]$.

To establish the solvability of Riccati equation (4.1), we introduce the
following two optimal control problems: For $i=1,2$, consider the state
equation

$\left\\{\begin{aligned}
dX(s)&=\big{\\{}A(s)X(s)+\bar{A}(s)\mathbb{E}[X(s)]+B_{i}(s)u_{i}(s)+\bar{B}_{i}(s)\mathbb{E}[u_{i}(s)]\big{\\}}ds\\\
&\hphantom{=~{}}+\big{\\{}C(s)X(s)+\bar{C}(s)\mathbb{E}[X(s)]+D_{i}(s)u_{i}(s)+\bar{D}_{i}(s)\mathbb{E}[u_{i}(s)]\big{\\}}dW(s),\\\
X(0)&=x,\end{aligned}\right.$ (4.3)

and the cost functional

$\displaystyle J_{i}(x;u_{i})=(-1)^{i+1}\mathbb{E}\big{\\{}\langle
GX(T),X(T)\rangle+\langle\bar{G}\mathbb{E}[X(T)],\mathbb{E}[X(T)]\rangle$
$\displaystyle\qquad+\int_{0}^{T}\Bigg{[}\left\langle\\!\begin{pmatrix}Q&S_{i}^{\top}\\\
S_{i}&R_{ii}\end{pmatrix}\begin{pmatrix}X\\\
u_{i}\end{pmatrix},\begin{pmatrix}X\\\
u_{i}\end{pmatrix}\\!\right\rangle+\left\langle\\!\begin{pmatrix}\bar{Q}&\bar{S}_{i}^{\top}\\\
\bar{S}_{i}&\bar{R}_{ii}\end{pmatrix}\begin{pmatrix}\mathbb{E}[X]\\\
\mathbb{E}[u_{i}]\end{pmatrix},\begin{pmatrix}\mathbb{E}[X]\\\
\mathbb{E}[u_{i}]\end{pmatrix}\\!\right\rangle\Bigg{]}ds\Bigg{\\}}.$ (4.4)

If the mean-field terms in the above vanish, then (4.3) and (4) reduce to

$\left\\{\begin{aligned}
dX(s)&=\big{\\{}A(s)X(s)+B_{i}(s)u_{i}(s)\big{\\}}ds+\big{\\{}C(s)X(s)+D_{i}(s)u_{i}(s)\big{\\}}dW(s),\\\
X(0)&=x,\end{aligned}\right.$ (4.5)

and

$\displaystyle{\cal J}_{i}(x;u_{i})$
$\displaystyle=(-1)^{i+1}\mathbb{E}\Bigg{\\{}\langle
GX(T),X(T)\rangle+\int_{0}^{T}\left\langle\\!\begin{pmatrix}Q&S_{i}^{\top}\\\
S_{i}&R_{ii}\end{pmatrix}\begin{pmatrix}X\\\
u_{i}\end{pmatrix},\begin{pmatrix}X\\\ u_{i}\end{pmatrix}\\!\right\rangle
ds\Bigg{\\}}.$

The Riccati equations associated with the above LQ control problems read

$\left\\{\begin{aligned}
&\dot{P}_{i}+P_{i}A+A^{\top}P_{i}+C^{\top}P_{i}C+Q-(P_{i}B_{i}+C^{\top}P_{i}D_{i}+S_{i}^{\top})\\\
&\hphantom{\dot{P}}\quad\times(R_{ii}+D_{i}^{\top}P_{i}D_{i})^{-1}(B_{i}^{\top}P_{i}+D_{i}^{\top}P_{i}C+S_{i})=0,\quad
i=1,2,\\\ &P_{i}(T)=G,\quad i=1,2.\end{aligned}\right.$ (4.6)

######  Theorem 4.2.

Let (H1)–(H3) hold. Then Riccati equation (4.1) admits a unique strongly
regular solution $P\in C([0,T];\mathbb{S}^{n})$. Moreover, the strongly
regular solution $P$ satisfies

$P_{1}(t)\leqslant P(t)\leqslant P_{2}(t),\quad\forall t\in[0,T],$ (4.7)

and

$(-1)^{i+1}[R_{ii}+\bar{R}_{ii}+(D_{i}+\bar{D}_{i})^{\top}P(D_{i}+\bar{D}_{i})]\gg
0;\quad i=1,2,$ (4.8)

where $P_{i}\in C([0,T];\mathbb{S}^{n})$ ($i=1,2$) is the unique solution of
(4.6).

###### Proof.

Note that the assumption (H3) implies that for any $(u_{1},u_{2})\in{\cal
U}_{1}\times{\cal U}_{2}$,

$J_{1}(0;u_{1})=J(0;u_{1},0)\gg 0\quad\hbox{and}\quad
J_{2}(0;u_{2})=-J(0;0,u_{2})\gg 0.$

Thus for $i=1,2$, we obtain from [23, Theorems 4.2 and 4.4] that Riccati
equation (4.6) admits a unique solution $P_{i}\in C([0,T];\mathbb{S}^{n})$
satisfying

$(-1)^{i+1}[R_{ii}+D_{i}^{\top}P_{i}D_{i}]\gg 0,$ (4.9)

and

$(-1)^{i+1}[R_{ii}+\bar{R}_{ii}+(D_{i}+\bar{D}_{i})^{\top}P_{i}(D_{i}+\bar{D}_{i})]\gg
0.$ (4.10)

According to [23, Theorem 5.2], (4.9) implies that the mapping
$u_{i}\mapsto{\cal J}_{i}(0;u_{i})$ is uniformly convex. Thus,

$\displaystyle{\cal J}(0;u_{1},0)={\cal J}_{1}(0;u_{1})\gg
0,\quad~{}~{}\forall u_{1}\in{\cal U}_{1};$ $\displaystyle{\cal
J}(0;0,u_{2})=-{\cal J}_{2}(0;u_{2})\ll 0,\quad\forall u_{2}\in{\cal U}_{2},$
(4.11)

where ${\cal J}$ denotes the functional of Problem (SG); that is ${\cal
J}(x;u_{1},u_{2})\triangleq J(x;u_{1},u_{2})$ with $\bar{A}$, $\bar{B}_{i}$,
$\bar{C}$, $\bar{D}_{i}$, $\bar{G}$, $\bar{Q}$, $\bar{S}_{i}$ and
$\bar{R}_{ij}$ ($i,j=1,2$) all vanishing. Then from [24, Theorem 4.3], we
obtain that Riccati equation (4.1) has a strongly regular solution $P\in
C([0,T];\mathbb{S}^{n})$.

To prove the uniqueness, we suppose that $\bar{P},\tilde{P}\in
C([0,T];\mathbb{S}^{n})$ are two strongly regular solutions of (4.1). Then
$\bar{P}$ and $\tilde{P}$ satisfy:

$(-1)^{i+1}[R_{ii}+D_{i}^{\top}\tilde{P}D_{i}]\gg
0\quad\hbox{and}\quad(-1)^{i+1}[R_{ii}+D_{i}^{\top}\bar{P}D_{i}]\gg 0,\quad
i=1,2.$

Similar to (3.21), $R+D^{\top}\bar{P}D$ and $R+D^{\top}\tilde{P}D$ are
invertible with their inverses being bounded. Denote $\Delta
P=\bar{P}-\tilde{P}$. Then $\Delta P$ satisfies the following linear ordinary
differential equation:

$\left\\{\begin{aligned} &\Delta\dot{P}+\Delta PA+A^{\top}\Delta
P+C^{\top}\Delta PC-(\Delta PB+C^{\top}\Delta PD)(R+D^{\top}\bar{P}D)^{-1}\\\
&\hphantom{\dot{P}}\times(B^{\top}\bar{P}+D^{\top}\bar{P}C+S)-(\tilde{P}B^{\top}+C^{\top}\tilde{P}D+S^{\top})(R+D^{\top}\bar{P}D)^{-1}D^{\top}\Delta
PD\\\
&\hphantom{\dot{P}}\times(R+D^{\top}\tilde{P}D)^{-1}(B^{\top}\bar{P}+D^{\top}\bar{P}C+S)-(\tilde{P}B^{\top}+C^{\top}\tilde{P}D+S^{\top})\\\
&\hphantom{\dot{P}}\times(R+D^{\top}\tilde{P}D)^{-1}(B^{\top}\Delta
P+D^{\top}\Delta PC)=0,\\\ &\Delta P(T)=0.\end{aligned}\right.$ (4.12)

Note that $\bar{P}$, $\tilde{P}$, $(R+D^{\top}\bar{P}D)^{-1}$ and
$(R+D^{\top}\tilde{P}D)^{-1}$ are bounded. Then by a standard argument using
the Grönwall’s inequality, we get $\Delta P\equiv 0$, which yields the
uniqueness of the strongly regular solution to (4.1).

Next let us prove the unique strongly regular solution $P\in
C([0,T];\mathbb{S}^{n})$ of (4.1) satisfies (4.7). To this end, for any fixed
$u_{2}\in{\cal U}_{2}$, we introduce the following LQ control problem:
Consider the state equation

$\left\\{\begin{aligned}
dX(s)&=\big{\\{}A(s)X(s)+B_{1}(s)u_{1}(s)+B_{2}(s)u_{2}(s)\big{\\}}ds\\\
&~{}\hphantom{=}+\big{\\{}C(s)X(s)+D_{1}(s)u_{1}(s)+D_{2}(s)u_{2}(s)\big{\\}}dW(s),\\\
X(0)&=x,\end{aligned}\right.$ (4.13)

and the cost functional

$\displaystyle{\cal J}_{u_{2}}(x;u_{1})$ $\displaystyle\triangleq{\cal
J}(x;u_{1},u_{2})=\mathbb{E}\bigg{\\{}\langle
GX(T),X(T)\rangle+\int_{0}^{T}\Big{[}\langle QX,X\rangle+2\langle
S_{1}X,u_{1}\rangle$ $\displaystyle\qquad\quad+\langle
R_{11}u_{1},u_{1}\rangle+2\langle X,S_{2}^{\top}u_{2}\rangle+2\langle
u_{1},R_{12}u_{2}\rangle+\langle R_{22}u_{2},u_{2}\rangle\Big{]}ds\bigg{\\}}.$
(4.14)

Note that (4.11) implies the mapping $u_{1}\mapsto{\cal
J}_{u_{2}}(x;u_{1})={\cal J}(x;u_{1},u_{2})$ is uniformly convex, then by [25,
Theorem 4.3] the unique optimal control
$\bar{u}_{1}(\cdot)\equiv\bar{u}_{1}(\cdot;u_{2})$ of the above LQ control
problem admits the following closed-loop representation:

$\bar{u}_{1}(s)=\Theta(s)\bar{X}(s)+v(s)\equiv\Theta(s)\bar{X}(s;u_{2})+v(s;u_{2}),\quad
s\in[0,T],$ (4.15)

where

$\displaystyle\Theta$
$\displaystyle=-\big{(}R_{11}+D_{1}^{\top}P_{1}D_{1}\big{)}^{-1}\big{(}B_{1}^{\top}P_{1}+D_{1}^{\top}P_{1}C+S_{1}\big{)},$
$\displaystyle v$
$\displaystyle=-\big{(}R_{11}+D_{1}^{\top}P_{1}D_{1}\big{)}^{-1}\big{(}B_{1}^{\top}Y+D_{1}^{\top}Z+D_{1}^{\top}P_{1}D_{2}u_{2}+R_{12}u_{2}\big{)};$

$P_{1}$ is uniquely determined by Riccati equation (4.6);
$(Y(\cdot),Z(\cdot))\equiv(Y(\cdot;u_{2}),Z(\cdot;u_{2}))$ solves the backward
stochastic differential equation:

$\left\\{\begin{aligned}
dY(s)&=-\big{[}(A+B_{1}\Theta)^{\top}Y(s)+(C+D_{1}\Theta)^{\top}Z(s)+(C+D_{1}\Theta)^{\top}P_{1}D_{2}u_{2}\\\
&\hphantom{=-\big{[}}+\Theta^{\top}R_{12}u_{2}+P_{1}B_{2}u_{2}+S_{2}^{\top}u_{2}\big{]}ds+Z(s)dW(s),\quad
s\in[0,T],\\\ Y(T)&=0;\end{aligned}\right.$ (4.16)

and $\bar{X}(\cdot)\equiv\bar{X}(\cdot;u_{2})$ is the solution of the closed-
loop system:

$\left\\{\begin{aligned}
d\bar{X}(s)&=\big{\\{}A(s)\bar{X}(s)+B_{1}(s)[\Theta(s)\bar{X}(s)+v(s)]+B_{2}(s)u_{2}(s)\big{\\}}ds\\\
&~{}\hphantom{=}+\big{\\{}C(s)\bar{X}(s)+D_{1}(s)[\Theta(s)\bar{X}(s)+v(s)]+D_{2}(s)u_{2}(s)\big{\\}}dW(s),\\\
\bar{X}(0)&=x.\end{aligned}\right.$ (4.17)

Moreover,

$\displaystyle{\cal J}_{u_{2}}(x;\bar{u}_{1})={\cal
J}_{u_{2}}(x;\Theta\bar{X}+v)={\cal J}(x;\Theta\bar{X}+v,u_{2})$
$\displaystyle=\mathbb{E}\bigg{\\{}\langle P_{1}(0)x,x\rangle+2\langle
Y(0),x\rangle+\int_{0}^{T}\Big{[}\langle
P_{1}D_{2}u_{2},D_{2}u_{2}\rangle+2\langle Y,B_{2}u_{2}\rangle$
$\displaystyle\qquad+2\langle
Z,D_{2}u_{2}\rangle+\big{\langle}(R_{11}+D_{1}^{\top}P_{1}D_{1})^{-1}(B_{1}^{\top}Y+D_{1}^{\top}Z+D_{1}^{\top}P_{1}D_{2}u_{2}+R_{12}u_{2}),$
$\displaystyle\qquad\qquad(B_{1}^{\top}Y+D_{1}^{\top}Z+D_{1}^{\top}P_{1}D_{2}u_{2}+R_{12}u_{2})\big{\rangle}+\langle
R_{22}u_{2},u_{2}\rangle\Big{]}ds\bigg{\\}}.$ (4.18)

Since the condition (4.11) holds, by [24, Theorem 4.4] Problem (SG) admits a
unique open-loop saddle point $(u_{1}^{*},u_{2}^{*})$; that is

${\cal J}(x;u^{*}_{1},u_{2}^{*})=\sup_{u_{2}\in{\cal
U}_{2}}\inf_{u_{1}\in{\cal U}_{1}}{\cal J}(x;u_{1},u_{2})=\inf_{u_{1}\in{\cal
U}_{1}}\sup_{u_{2}\in{\cal U}_{2}}{\cal J}(x;u_{1},u_{2})=\langle
P(0)x,x\rangle,$

which implies that

$\inf_{u_{1}\in{\cal U}_{1}}{\cal J}_{u^{*}_{2}}(x;u_{1})={\cal
J}_{u^{*}_{2}}(x;u^{*}_{1})\quad\hbox{and}\quad{\cal
J}(x;u^{*}_{1},u_{2}^{*})=\sup_{u_{2}\in{\cal U}_{2}}{\cal
J}(x;u^{*}_{1},u_{2}).$ (4.19)

Recall that the mapping $u_{1}\mapsto{\cal J}(x;u_{1},u_{2}^{*})$ is uniformly
convex, which implies that $u_{1}^{*}$ is the unique control satisfying the
first equality in the above, thus we have

${\cal J}(x;u^{*}_{1},u_{2}^{*})={\cal J}_{u_{2}^{*}}(x;u^{*}_{1})={\cal
J}_{u^{*}_{2}}(x;\Theta\bar{X}^{*}+v^{*})={\cal
J}(x;\Theta\bar{X}^{*}+v^{*},u^{*}_{2}),$ (4.20)

with $\bar{X}^{*}(\cdot)=\bar{X}(\cdot;u_{2}^{*})$ and
$v^{*}(\cdot)=v(\cdot;u_{2}^{*})$. On the other hand, by the second equality
in (4.19) and the closed-loop representation (4.15) of the optimal control
$\bar{u}_{1}$, we get

${\cal J}(x;u^{*}_{1},u_{2}^{*})\geqslant{\cal J}(x;u_{1}^{*},u_{2})={\cal
J}_{u_{2}}(x;u_{1}^{*})\geqslant{\cal J}_{u_{2}}(x;\bar{u}_{1})={\cal
J}(x;\Theta\bar{X}+v,u_{2}),\quad\forall u_{2}\in{\cal U}_{2},$

with $\bar{X}(\cdot)=\bar{X}(\cdot;u_{2})$ and $v(\cdot)=v(\cdot;u_{2})$.
Combining the above with (4.20) yields that

${\cal J}(x;\Theta\bar{X}^{*}+v^{*},u^{*}_{2})\geqslant{\cal
J}(x;\Theta\bar{X}+v,u_{2})\equiv{\cal
J}(x;\Theta\bar{X}(\cdot;u_{2})+v(\cdot;u_{2}),u_{2}),\quad\forall
u_{2}\in{\cal U}_{2}.$ (4.21)

Taking $u_{2}=0$ in (4.21) and then making use of (4) and (4.16), we have

$\langle P(0)x,x\rangle={\cal
J}(x;\Theta\bar{X}^{*}+v^{*},u^{*}_{2})\geqslant{\cal
J}(x;\Theta\bar{X}+v,0)=\langle P_{1}(0)x,x\rangle.$

Using the above arguments to the Problem (SG) with the initial pair replaced
by $(t,x)\in[0,T]\times\mathbb{R}^{n}$, we obtain

$\langle P(t)x,x\rangle\geqslant\langle
P_{1}(t)x,x\rangle,\quad\forall(t,x)\in[0,T]\times\mathbb{R}^{n}.$

In a similar manner, we can also show that

$\langle P(t)x,x\rangle\leqslant\langle
P_{2}(t)x,x\rangle,\quad\forall(t,x)\in[0,T]\times\mathbb{R}^{n}.$

Thus, the unique strongly regular solution $P$ of Riccati equation (4.1)
satisfies (4.7). Finally, combining (4.7) with (4.10), we get (4.8)
immediately. The proof is thus complete. ∎

######  Remark 4.3.

If we define the state processes $(Y,Z)$ by (4.16) and the cost functional
$\hat{J}(x;u_{2})\triangleq-{\cal J}(x;\Theta\bar{X}+v,u_{2})$ by (4), then
the corresponding control problem is a backward LQ problem. From (4.21), we
see that under (H3), if $(u_{1}^{*},u_{2}^{*})$ is the unique open-loop saddle
point of Problem (SG), then $u_{2}^{*}$ is optimal for the above designed
backward problem. For more results of backward LQ control problems, we refer
the reader to [18, 17, 27] and the references cited therein.

With the strongly regular solution $P$ of (4.1), we now introduce the
following deterministic two-person zero-sum LQ differential game problem
(Problem (DG), for short): Consider the state equation

$\left\\{\begin{aligned}
\dot{y}(s)&=[A(s)+\bar{A}(s)]y(s)+[B_{1}(s)+\bar{B}_{1}(s)]v_{1}(s)+[B_{2}(s)+\bar{B}_{2}(s)]v_{2}(s),\\\
y(0)&=x,\end{aligned}\right.$ (4.22)

and the functional

$\displaystyle\bar{J}(x;v_{1},v_{2})=\big{\langle}(G+\bar{G})y(T),y(T)\big{\rangle}+\int_{0}^{T}\left\langle\\!\begin{pmatrix}\Upsilon&\Gamma_{1}^{\top}&\Gamma_{2}^{\top}\\\
\Gamma_{1}&\bar{\Sigma}_{11}&\bar{\Sigma}_{12}\\\
\Gamma_{2}&\bar{\Sigma}_{21}&\bar{\Sigma}_{22}\end{pmatrix}\begin{pmatrix}y\\\
v_{1}\\\ v_{2}\end{pmatrix},\begin{pmatrix}y\\\ v_{1}\\\
v_{2}\end{pmatrix}\\!\right\rangle ds,$ (4.23)

where

$\left\\{\begin{aligned}
&\Upsilon=Q+\bar{Q}+(C+\bar{C})^{\top}P(C+\bar{C}),\\\
&\Gamma_{i}=(D_{i}+\bar{D}_{i})^{\top}P(C+\bar{C})+(S_{i}+\bar{S}_{i}),\quad
i=1,2,\\\
&\bar{\Sigma}_{ij}=R_{ij}+\bar{R}_{ij}+(D_{i}+\bar{D}_{i})^{\top}P(D_{j}+\bar{D}_{j}),\quad
i,j=1,2.\end{aligned}\right.$ (4.24)

Since the strongly regular solution $P$ also satisfies (4.8), the matrices
$\Sigma$ and $\bar{\Sigma}$, defined by

$\Sigma\equiv R+D^{\top}PD,\quad\bar{\Sigma}\equiv
R+\bar{R}+(D+\bar{D})^{\top}P(D+\bar{D}),$ (4.25)

are invertible. The Riccati equation associated with Problem (DG) is

$\left\\{\begin{aligned}
&\dot{\Pi}+\Pi(A+\bar{A})+(A+\bar{A})^{\top}\Pi+Q+\bar{Q}+(C+\bar{C})^{\top}P(C+\bar{C})-\big{[}\Pi(B+\bar{B})\\\
&\hphantom{\dot{\Pi}}\qquad+(C+\bar{C})^{\top}P(D+\bar{D})+(S+\bar{S})^{\top}\big{]}\big{[}R+\bar{R}+(D+\bar{D})^{\top}P(D+\bar{D})\big{]}^{-1}\\\
&\hphantom{\dot{\Pi}}\quad\times\big{[}(B+\bar{B})^{\top}\Pi+(D+\bar{D})^{\top}P(C+\bar{C})+(S+\bar{S})\big{]}=0,\\\
&\Pi(T)=G+\bar{G},\end{aligned}\right.$ (4.26)

where $\bar{B}$, $\bar{D}$ and $\bar{R}$ are defined in a similar way to
(4.2). The following result shows that under (H3), Riccati equation (4.26) is
also solvable.

######  Theorem 4.4.

Let (H1)–(H3) hold. Then Riccati equation (4.26) is uniquely solvable.

###### Proof.

By [24, Theorem 4.3], to prove the solvability of Riccati equation (4.26), it
suffices to show that

$\bar{J}(0;v_{1},0)\gg 0\quad\hbox{and}\quad\bar{J}(0;0,v_{2})\ll 0.$ (4.27)

Denote

$\displaystyle\bar{J}_{1}(x;v_{1})=\langle(G+\bar{G})y(T),y(T)\rangle+\int_{0}^{T}\left\langle\\!\begin{pmatrix}\bar{\Upsilon}_{1}&\bar{\Gamma}_{1}^{\top}\\\
\bar{\Gamma}_{1}&\bar{\Sigma}_{11}&\end{pmatrix}\begin{pmatrix}y\\\
v_{1}\end{pmatrix},\begin{pmatrix}y\\\ v_{1}\end{pmatrix}\\!\right\rangle ds,$
(4.28)

where $y$ is the unique solution of (4.22) with $v_{2}\equiv 0$, and

$\left\\{\begin{aligned}
&\bar{\Upsilon}_{1}=Q+\bar{Q}+(C+\bar{C})^{\top}P_{1}(C+\bar{C}),\\\
&\bar{\Gamma}_{1}=(D_{1}+\bar{D}_{1})^{\top}P_{1}(C+\bar{C})+(S_{1}+\bar{S}_{1}),\\\
&\bar{\Sigma}_{11}=R_{11}+\bar{R}_{11}+(D_{1}+\bar{D}_{1})^{\top}P_{1}(D_{1}+\bar{D}_{1}),\end{aligned}\right.$
(4.29)

with $P_{1}$ being the unique solution of (4.6) for $i=1$. Recall the
definition (4) of $J_{1}$ and note that $J_{1}(0;u_{1})=J(0;u_{1},0)\gg 0$. By
[23, Theorem 4.4], we have

$\bar{J}_{1}(0;v_{1})\gg 0.$ (4.30)

On the other hand, for any $\delta>0$, by (4.23)–(4.28) we obtain

$\displaystyle\quad\bar{J}(0;v_{1},0)-\bar{J}_{1}(0;v_{1})+\delta\|v_{1}\|^{2}$
$\displaystyle\quad=\int_{0}^{T}\Big{\\{}\big{\langle}(C+\bar{C})^{\top}(P-P_{1})(C+\bar{C})y,y\big{\rangle}+2\big{\langle}(D_{1}+\bar{D}_{1})^{\top}(P-P_{1})(C+\bar{C})y,v_{1}\big{\rangle}$
$\displaystyle\qquad\qquad+\big{\langle}[(D_{1}+\bar{D}_{1})^{\top}(P-P_{1})(D_{1}+\bar{D}_{1})+\delta
I_{m_{1}}]v_{1},v_{1}\big{\rangle}\Big{\\}}ds$
$\displaystyle\quad\equiv\int_{0}^{T}\big{[}\langle{\cal
Q}y,y\rangle+2\langle{\cal S}y,x\rangle+\langle{\cal
R}v_{1},v_{1}\rangle\big{]}ds.$ (4.31)

Note that $P\geqslant P_{1}$ (recalling (4.7)), thus

${\cal R}\equiv(D_{1}+\bar{D}_{1})^{\top}(P-P_{1})(D_{1}+\bar{D}_{1})+\delta
I_{m_{1}}\geqslant\delta I_{m_{1}}\gg 0.$

Moreover, by 2.3 we have

$\displaystyle{\cal Q}-{\cal S}^{\top}{\cal R}^{-1}{\cal
S}=(C+\bar{C})^{\top}(P-P_{1})(C+\bar{C})-(C+\bar{C})^{\top}(P-P_{1})(D_{1}+\bar{D}_{1})$
$\displaystyle\qquad\times[(D_{1}+\bar{D}_{1})^{\top}(P-P_{1})(D_{1}+\bar{D}_{1})+\delta
I_{m_{1}}]^{-1}(D_{1}+\bar{D}_{1})^{\top}(P-P_{1})(C+\bar{C})\geqslant 0.$

Thus, the weighting matrices ${\cal Q}$, ${\cal S}$ and ${\cal R}$ satisfy the
so-called standard condition in the literature (see [35, Chapter 6], for
example) of LQ optimal control problems. Then

$\displaystyle\bar{J}(0;v_{1},0)-\bar{J}_{1}(0;v_{1})+\delta\|v_{1}\|^{2}$
$\displaystyle\quad=\int_{0}^{T}\Big{[}\big{\langle}({\cal Q}-{\cal
S}^{\top}{\cal R}^{-1}{\cal S})y,y\big{\rangle}+\big{\langle}{\cal
R}(v_{1}+{\cal R}^{-1}{\cal S}y),(v_{1}+{\cal R}^{-1}{\cal
S}y)\big{\rangle}\Big{]}ds$ $\displaystyle\quad\geqslant 0.$ (4.32)

Since $\delta>0$ is arbitrary, we get

$\bar{J}(0;v_{1},0)-\bar{J}_{1}(0;v_{1})\geqslant 0,$ (4.33)

which, together with (4.30), implies that

$\bar{J}(0;v_{1},0)\gg 0.$ (4.34)

Similarly, we can also prove that

$\bar{J}(0;0,v_{1})\ll 0.$ (4.35)

Combining (4.34) with (4.35), we get (4.27), which completes the proof of the
existence. Then by the same arguments as in the proof of Theorem 4.2, we
obtain the unique solvability of Riccati equation (4.26). ∎

######  Remark 4.5.

It is noteworthy that the comparison property (4.7) of $P$ and $P_{i};i=1,2$
serves as a crucial bridge to prove the solvability of Riccati equation (4.26)
in Theorem 4.4. The technical 2.3 is used to show the weighting matrices
${\cal Q}$, ${\cal S}$ and ${\cal R}$ defined by (4.31) exactly satisfy the
so-called standard condition.

With the strongly regular solvability of Riccati equations (4.1)–(4.26) having
been established, we present the closed-loop representation for the open-loop
saddle point of Problem (MF-SG).

######  Theorem 4.6.

Let (H1)–(H3) hold. Let $P\in C([0,T];\mathbb{S}^{n})$ be the strongly regular
solution to Riccati equation (4.1) satisfying (4.7)–(4.8) and $\Pi\in
C([0,T];\mathbb{S}^{n})$ be the solution to Riccati equation (4.26). Then with
the notations

$\displaystyle\Sigma=R+D^{\top}PD,\quad\bar{\Sigma}\equiv
R+\bar{R}+(D+\bar{D})^{\top}P(D+\bar{D}),$ (4.36)
$\displaystyle\Theta=-\Sigma^{-1}\big{(}B^{\top}P+D^{\top}PC+S\big{)},$ (4.37)
$\displaystyle\bar{\Theta}=-\bar{\Sigma}^{-1}[(B+\bar{B})^{\top}\Pi+(D+\bar{D})^{\top}P(C+\bar{C})+S+\bar{S}],$
(4.38)

the unique open-loop saddle point $u^{*}=(u_{1}^{*\top},u_{2}^{*\top})^{\top}$
for the initial state $x$ has the following closed-loop representation:

$u^{*}=\Theta\big{\\{}X^{*}-\mathbb{E}[X^{*}]\big{\\}}+\bar{\Theta}\mathbb{E}[X^{*}],$
(4.39)

where $X^{*}$ is the solution to the closed-loop system:

$\left\\{\begin{aligned}
dX^{*}(s)&=\big{\\{}(A+B\Theta)(X^{*}-\mathbb{E}[X^{*}])+[(A+\bar{A})+(B+\bar{B})\bar{\Theta}]\mathbb{E}[X^{*}]\big{\\}}ds\\\
&\hphantom{=\
}+\big{\\{}(C+D\Theta)(X^{*}-\mathbb{E}[X^{*}])+[(C+\bar{C})+(D+\bar{D})\bar{\Theta}]\mathbb{E}[X^{*}]\big{\\}}dW(s),\\\
X^{*}(0)&=x.\end{aligned}\right.$ (4.40)

Moreover, the value function of Problem (MF-SG) is given by
$V(x)=\langle\Pi(0)x,x\rangle$.

######  Remark 4.7.

By Theorem 4.6, we give an explicit representation for the unique open-loop
saddle point of Problem (MF-SG). Indeed, the LQ problems occupied the center
stage for research in control theory not only for its elegant solutions but
also for its ability to approximate more general nonlinear problems, as
pointed out by Wang–Zariphopoulou–Zhou in their recent work [32] of the
stochastic control approach in reinforcement learning.

In order to prove Theorem 4.6, we need the following lemma, whose proof is
standard and is similar to that of [28, Theorem 4.1].

######  Lemma 4.8.

A pair $(\bar{u}_{1},\bar{u}_{2})\in{\cal U}_{1}\times{\cal U}_{2}$ is an
open-loop saddle point of Problem (MF-SG) if and only if (3.18) holds and
$\bar{u}=(\bar{u}_{1}^{\top},\bar{u}_{2}^{\top})^{\top}$ satisfies

$\displaystyle
B(s)^{\top}\bar{Y}(s)+\bar{B}(s)^{\top}\mathbb{E}[\bar{Y}(s)]+D(s)^{\top}\bar{Z}(s)+\bar{D}(s)^{\top}\mathbb{E}[\bar{Z}(s)]+S(s)\bar{X}(s)$
$\displaystyle+\bar{S}(s)\mathbb{E}[\bar{X}(s)]+R(s)\bar{u}(s)+\bar{R}(s)\mathbb{E}[\bar{u}(s)]=0,\quad
s\in[0,T],\quad\hbox{a.s.},$ (4.41)

where $(\bar{X},\bar{Y},\bar{Z})$ is the unique solution to the following
mean-field forward-backward SDE (MF-FBSDE, for short):

$\left\\{\begin{aligned}
d\bar{X}(s)&=\big{\\{}A\bar{X}+\bar{A}\mathbb{E}[\bar{X}]+B\bar{u}+\bar{B}\mathbb{E}[\bar{u}]\big{\\}}ds\\\
&\hphantom{=}~{}+\big{\\{}C\bar{X}+\bar{C}\mathbb{E}[\bar{X}]+D\bar{u}+\bar{D}\mathbb{E}[\bar{u}]\big{\\}}dW(s),\quad
s\in[0,T],\\\
d\bar{Y}(s)&=-\big{\\{}A^{\top}\bar{Y}+\bar{A}^{\top}\mathbb{E}[\bar{Y}]+C^{\top}\bar{Z}+\bar{C}^{\top}\mathbb{E}[\bar{Z}]+Q\bar{X}+\bar{Q}\mathbb{E}[\bar{X}]\\\
&\qquad+S^{\top}\bar{u}+\bar{S}^{\top}\mathbb{E}[\bar{u}]\big{\\}}ds+\bar{Z}dW(s),\quad
s\in[0,T],\\\
\bar{X}(0)&=x,\quad\bar{Y}(T)=G\bar{X}(T)+\bar{G}\mathbb{E}[\bar{X}(T)].\end{aligned}\right.$
(4.42)

_Proof of Theorem 4.6._ We first prove that the control $u^{*}$ defined by
(4.39) is the unique open-loop saddle point. Denote

$\displaystyle Y^{*}$
$\displaystyle=P(X^{*}-\mathbb{E}[X^{*}])+\Pi\mathbb{E}[X^{*}],$
$\displaystyle Z^{*}$
$\displaystyle=P\big{\\{}C(X^{*}-\mathbb{E}[X^{*}])+(C+\bar{C})\mathbb{E}[X^{*}]+D(u^{*}-\mathbb{E}[u^{*}])+(D+\bar{D})\mathbb{E}[u^{*}]\big{\\}}.$
(4.43)

Note that

$\displaystyle\mathbb{E}[Y^{*}]=\Pi\mathbb{E}[X^{*}],\quad\mathbb{E}[u^{*}]=\bar{\Theta}\mathbb{E}[X^{*}],\quad\mathbb{E}[Z^{*}]=P\big{\\{}(C+\bar{C})\mathbb{E}[X^{*}]+(D+\bar{D})\mathbb{E}[u^{*}]\big{\\}}.$

By Itô’s formula, we have

$\displaystyle
dY^{*}(t)=\dot{P}(X^{*}-\mathbb{E}[X^{*}])dt+P\big{\\{}(A+B\Theta)(X^{*}-\mathbb{E}[X^{*}])\big{\\}}dt$
$\displaystyle\qquad+P\big{\\{}(C+D\Theta)(X^{*}-\mathbb{E}[X^{*}])+[(C+\bar{C})+(D+\bar{D})\bar{\Theta}]\mathbb{E}[X^{*}]\big{\\}}dW(t)$
$\displaystyle\qquad+\dot{\Pi}\mathbb{E}[X^{*}]dt+\Pi[(A+\bar{A})+(B+\bar{B})\bar{\Theta}]\mathbb{E}[X^{*}]dt$
$\displaystyle\quad=-\big{[}A^{\top}P+C^{\top}PC+Q+(C^{\top}PD+S^{\top})\Theta\big{]}(X^{*}-\mathbb{E}[X^{*}])dt-\big{\\{}(A+\bar{A})^{\top}\Pi+Q+\bar{Q}$
$\displaystyle\qquad\quad+(C+\bar{C})^{\top}P(C+\bar{C})+[(C+\bar{C})^{\top}P(D+\bar{D})+S^{\top}+\bar{S}^{\top}]\bar{\Theta}\big{\\}}\mathbb{E}[X^{*}]dt+Z^{*}dW(t)$
$\displaystyle\quad=-\Big{\\{}A^{\top}Y^{*}+\bar{A}^{\top}\mathbb{E}[Y^{*}]+C^{\top}P\big{\\{}C(X^{*}-\mathbb{E}[X^{*}])+(C+\bar{C})\mathbb{E}[X^{*}]+D(u^{*}-\mathbb{E}[u^{*}])$
$\displaystyle\qquad\qquad+(D+\bar{D})\mathbb{E}[u^{*}]\big{\\}}+\bar{C}^{\top}P\big{\\{}(C+\bar{C})\mathbb{E}[X^{*}]+(D+\bar{D})\mathbb{E}[u^{*}]\big{\\}}+QX^{*}+\bar{Q}\mathbb{E}[X^{*}]$
$\displaystyle\qquad\quad+S^{\top}u^{*}+\bar{S}^{\top}\mathbb{E}[u^{*}]\Big{\\}}dt+Z^{*}dW(t)$
$\displaystyle\quad=-\big{\\{}A^{\top}Y^{*}+\bar{A}^{\top}\mathbb{E}[Y^{*}]+C^{\top}Z^{*}+\bar{C}^{\top}\mathbb{E}[Z^{*}]+QX^{*}+\bar{Q}\mathbb{E}[X^{*}]+S^{\top}u^{*}$
$\displaystyle\qquad\quad+\bar{S}^{\top}\mathbb{E}[u^{*}]\big{\\}}dt+Z^{*}dW(t).$
(4.44)

Further, by the terminal values of $P$ and $\Pi$, we get

$\displaystyle Y^{*}(T)$
$\displaystyle=P(T)\\{X^{*}(T)-\mathbb{E}[X^{*}(T)]\\}+\Pi(T)\mathbb{E}[X^{*}(T)]$
$\displaystyle=GX^{*}(T)+\bar{G}\mathbb{E}[X^{*}(T)].$ (4.45)

Thus $(X^{*},Y^{*},Z^{*})$ satisfies MF-FBSDE (4.42) with the control
$\bar{u}\equiv u^{*}$. Moreover, note that

$\displaystyle
B^{\top}Y^{*}+\bar{B}^{\top}\mathbb{E}[Y^{*}]+D^{\top}Z^{*}+\bar{D}^{\top}\mathbb{E}[Z^{*}]+SX^{*}+\bar{S}\mathbb{E}[X^{*}]+Ru^{*}+\bar{R}\mathbb{E}[u^{*}]$
$\displaystyle\quad=B^{\top}\\{P(X^{*}-\mathbb{E}[X^{*}])+\Pi\mathbb{E}[X^{*}]\\}+\bar{B}^{\top}\Pi\mathbb{E}[X^{*}]+SX^{*}+\bar{S}\mathbb{E}[X^{*}]+\bar{R}\bar{\Theta}\mathbb{E}[X^{*}]$
$\displaystyle\qquad+R\big{\\{}\Theta(X^{*}-\mathbb{E}[X^{*}])+\bar{\Theta}\mathbb{E}[X^{*}]\big{\\}}+\bar{D}^{\top}P\big{\\{}(C+\bar{C})\mathbb{E}[X^{*}]+(D+\bar{D})\bar{\Theta}\mathbb{E}[X^{*}]\big{\\}}$
$\displaystyle\qquad+D^{\top}P\big{\\{}C(X^{*}-\mathbb{E}[X^{*}])+(C+\bar{C})\mathbb{E}[X^{*}]+D\Theta(X^{*}-\mathbb{E}[X^{*}])+(D+\bar{D})\bar{\Theta}\mathbb{E}[X^{*}]\big{\\}}$
$\displaystyle\quad=\big{\\{}B^{\top}P+S+D^{\top}PC+(D^{\top}PD+R)\Theta\big{\\}}(X^{*}-\mathbb{E}[X^{*}])+\big{\\{}(B+\bar{B})^{\top}\Pi$
$\displaystyle\qquad+S+\bar{S}+(D+\bar{D})^{\top}P(C+\bar{C})+[(D+\bar{D})^{\top}P(D+\bar{D})+R+\bar{R}]\bar{\Theta}\big{\\}}\mathbb{E}[X^{*}]$
$\displaystyle\quad=0.$ (4.46)

Then by 4.8, the control $u^{*}$ defined by (4.39) is an open-loop saddle
point. The uniqueness of open-loop saddle points follows from 3.2.

By integration by parts and (4.46), we get

$\displaystyle\mathbb{E}\langle
GX^{*}(T),X^{*}(T)\rangle+\langle\bar{G}\mathbb{E}[X^{*}(T)],\mathbb{E}[X^{*}(T)]\rangle=\mathbb{E}\langle
Y^{*}(T),X^{*}(T)\rangle$ $\displaystyle\quad=\mathbb{E}\langle
Y^{*}(0),X^{*}(0)\rangle+\mathbb{E}\int_{0}^{T}\Big{[}\big{\langle}B^{\top}Y^{*}+\bar{B}^{\top}\mathbb{E}[Y^{*}]+D^{\top}Z^{*}+\bar{D}^{\top}\mathbb{E}[Z^{*}]-SX^{*}$
$\displaystyle\qquad-\bar{S}\mathbb{E}[X^{*}],u^{*}\big{\rangle}-\langle
QX^{*},X^{*}\rangle-\langle\bar{Q}\mathbb{E}[X^{*}],X^{*}\rangle\Big{]}ds$
$\displaystyle\quad=\mathbb{E}\langle
Y^{*}(0),X^{*}(0)\rangle-\mathbb{E}\int_{0}^{T}\Big{[}\langle
Ru^{*},u^{*}\rangle+\langle\bar{R}\mathbb{E}[u^{*}],\mathbb{E}[u^{*}]\rangle+2\langle
SX^{*},u^{*}\rangle$
$\displaystyle\qquad+2\langle\bar{S}\mathbb{E}[X^{*}],\mathbb{E}[u^{*}]\rangle+\langle
QX^{*},X^{*}\rangle+\langle\bar{Q}\mathbb{E}[X^{*}],\mathbb{E}[X^{*}]\rangle\Big{]}ds.$

Noting that $Y^{*}(0)=\Pi(0)x$, we have the following representation of the
value function:

$\displaystyle V(x)$ $\displaystyle=J(x;u^{*})=\mathbb{E}\langle
GX^{*}(T),X^{*}(T)\rangle+\langle\bar{G}\mathbb{E}[X^{*}(T)],\mathbb{E}[X^{*}(T)]\rangle$
$\displaystyle\quad+\mathbb{E}\int_{0}^{T}\Big{[}\langle
Ru^{*},u^{*}\rangle+\langle\bar{R}\mathbb{E}[u^{*}],\mathbb{E}[u^{*}]\rangle+2\langle
SX^{*},u^{*}\rangle+2\langle\bar{S}\mathbb{E}[X^{*}],\mathbb{E}[u^{*}]\rangle$
$\displaystyle\quad+\langle
QX^{*},X^{*}\rangle+\langle\bar{Q}\mathbb{E}[X^{*}],\mathbb{E}[X^{*}]\rangle\Big{]}ds$
$\displaystyle=\mathbb{E}\langle
Y^{*}(0),X^{*}(0)\rangle=\langle\Pi(0)x,x\rangle,\quad
x\in\mathbb{R}^{n}.\qquad\qquad\qquad\qquad\qquad\qquad\hfill\qed$

##  5 Open-loop solvability: a perturbation approach

In Theorem 4.6, it is shown that under the uniform convexity-concavity
condition (H3), Problem (MF-SG) is uniquely open-loop solvable and the unique
open-loop saddle point admits the closed-loop representation (4.39). In this
section, we shall establish a characterization for the open-loop solvability
of Problem (MF-SG) without assumption (H3). Recall from 3.1 that the following
convexity-concavity condition is necessary for the open-loop solvability of
Problem (MF-SG):

$\displaystyle\langle{\cal M}_{11}u_{1},u_{1}\rangle$
$\displaystyle=J(0;u_{1},0)\geqslant 0,\quad\forall u_{1}\in{\cal U}_{1};$
(5.1) $\displaystyle\langle{\cal M}_{22}u_{2},u_{2}\rangle$
$\displaystyle=J(0;0,u_{2})\leqslant 0,\quad\forall u_{2}\in{\cal U}_{2}.$

Thus in this section, we always assume that (5.1) holds.

###  5.1 The perturbation approach

For each $\varepsilon>0$, we introduce the following perturbed functional:

$\displaystyle J_{\varepsilon}(x;u_{1},u_{2})\triangleq
J(x;u_{1},u_{2})+\varepsilon\mathbb{E}\int_{0}^{T}|u_{1}(s)|^{2}ds-\varepsilon\mathbb{E}\int_{0}^{T}|u_{2}(s)|^{2}ds$
$\displaystyle\quad=\mathbb{E}\big{\\{}\langle
GX(T),X(T)\rangle+\langle\bar{G}\mathbb{E}[X(T)],\mathbb{E}[X(T)]\rangle$
$\displaystyle\hphantom{=\mathbb{E}\big{\\{}~{}~{}}+\int_{0}^{T}\left\langle\\!\begin{pmatrix}Q&S_{1}^{\top}&S_{2}^{\top}\\\
S_{1}&R_{11}+\varepsilon I_{m_{1}}&R_{12}\\\ S_{2}&R_{21}&R_{22}-\varepsilon
I_{m_{2}}\end{pmatrix}\begin{pmatrix}X\\\ u_{1}\\\
u_{2}\end{pmatrix},\begin{pmatrix}X\\\ u_{1}\\\
u_{2}\end{pmatrix}\\!\right\rangle ds$
$\displaystyle\hphantom{=\mathbb{E}\big{\\{}~{}~{}}+\int_{0}^{T}\left\langle\\!\begin{pmatrix}\bar{Q}&\bar{S}_{1}^{\top}&\bar{S}_{2}^{\top}\\\
\bar{S}_{1}&\bar{R}_{11}&\bar{R}_{12}\\\
\bar{S}_{2}&\bar{R}_{21}&\bar{R}_{22}\end{pmatrix}\begin{pmatrix}\mathbb{E}[X]\\\
\mathbb{E}[u_{1}]\\\
\mathbb{E}[u_{2}]\end{pmatrix},\begin{pmatrix}\mathbb{E}[X]\\\
\mathbb{E}[u_{1}]\\\ \mathbb{E}[u_{2}]\end{pmatrix}\\!\right\rangle
ds\Bigg{\\}}.$ (5.2)

We denote the two-person zero-sum LQ stochastic differential game associated
with (1.1)–(5.1) by Problem (MF-SG)ε and the value function by
$V_{\varepsilon}$. Notice that

$\displaystyle J_{\varepsilon}(0;u_{1},0)$
$\displaystyle=J(0;u_{1},0)+\varepsilon\mathbb{E}\int_{0}^{T}|u_{1}(s)|^{2}ds\geqslant\varepsilon\|u_{1}\|^{2},\quad~{}~{}\forall
u_{1}\in{\cal U}_{1};$ (5.3) $\displaystyle J_{\varepsilon}(0;0,u_{2})$
$\displaystyle=J(0;0,u_{2})-\varepsilon\mathbb{E}\int_{0}^{T}|u_{2}(s)|^{2}ds\leqslant-\varepsilon\|u_{2}\|^{2},\quad\forall
u_{2}\in{\cal U}_{2}.$

Then with

$R_{\varepsilon}\triangleq\begin{pmatrix}R_{11}+\varepsilon
I_{m_{1}}&R_{12}\\\ R_{21}&R_{22}-\varepsilon I_{m_{2}}\end{pmatrix},$ (5.4)

it follows from Theorem 4.2 and Theorem 4.4 that the Riccati equations

$\left\\{\begin{aligned}
&\dot{P}_{\varepsilon}+P_{\varepsilon}A+A^{\top}P_{\varepsilon}+C^{\top}P_{\varepsilon}C+Q\\\
&\hphantom{\dot{P}}-(P_{\varepsilon}B+C^{\top}P_{\varepsilon}D+S^{\top})(R_{\varepsilon}+D^{\top}P_{\varepsilon}D)^{-1}(B^{\top}P_{\varepsilon}+D^{\top}P_{\varepsilon}C+S)=0,\\\
&P_{\varepsilon}(T)=G\end{aligned}\right.$ (5.5)

and

$\left\\{\begin{aligned}
&\dot{\Pi}_{\varepsilon}+\Pi_{\varepsilon}(A+\bar{A})+(A+\bar{A})^{\top}\Pi_{\varepsilon}+Q+\bar{Q}+(C+\bar{C})^{\top}P_{\varepsilon}(C+\bar{C})\\\
&\hphantom{\dot{\Pi}}-\big{[}\Pi_{\varepsilon}(B+\bar{B})+(C+\bar{C})^{\top}P_{\varepsilon}(D+\bar{D})+(S+\bar{S})^{\top}\big{]}\\\
&\hphantom{\dot{\Pi}}\times\big{[}R_{\varepsilon}+\bar{R}+(D+\bar{D})^{\top}P_{\varepsilon}(D+\bar{D})\big{]}^{-1}\\\
&\hphantom{\dot{\Pi}}\times\big{[}(B+\bar{B})^{\top}\Pi_{\varepsilon}+(D+\bar{D})^{\top}P_{\varepsilon}(C+\bar{C})+(S+\bar{S})\big{]}=0,\\\
&\Pi_{\varepsilon}(T)=G+\bar{G}\end{aligned}\right.$ (5.6)

admit unique solutions $P_{\varepsilon}\in C([0,T];\mathbb{S}^{n})$ and
$\Pi_{\varepsilon}\in C([0,T];\mathbb{S}^{n})$ satisfying

$\displaystyle(-1)^{i+1}[R_{ii}+(-1)^{i+1}\varepsilon
I_{m_{i}}+D_{i}^{\top}P_{\varepsilon}D_{i}]\gg 0,\quad i=1,2,$ (5.7)
$\displaystyle(-1)^{i+1}[R_{ii}+\bar{R}_{ii}+(-1)^{i+1}\varepsilon
I_{m_{i}}+(D_{i}+\bar{D}_{i})^{\top}P_{\varepsilon}(D_{i}+\bar{D}_{i})]\gg
0,\quad i=1,2.$ (5.8)

Denote

$\displaystyle\Sigma_{\varepsilon}=R_{\varepsilon}+D^{\top}P_{\varepsilon}D,\quad\bar{\Sigma}_{\varepsilon}=R_{\varepsilon}+\bar{R}+(D+\bar{D})^{\top}P_{\varepsilon}(D+\bar{D}),$
(5.9)
$\displaystyle\Theta_{\varepsilon}=-\Sigma_{\varepsilon}^{-1}\big{(}B^{\top}P_{\varepsilon}+D^{\top}P_{\varepsilon}C+S\big{)},$
(5.10)
$\displaystyle\bar{\Theta}_{\varepsilon}=-\bar{\Sigma}^{-1}_{\varepsilon}[(B+\bar{B})^{\top}\Pi_{\varepsilon}+(D+\bar{D})^{\top}P_{\varepsilon}(C+\bar{C})+S+\bar{S}].$
(5.11)

By Theorem 4.6, the unique open-loop saddle point
$u_{\varepsilon}=(u_{1}^{\varepsilon\top},u_{2}^{\varepsilon\top})^{\top}$ of
Problem (MF-SG)ε is given by:

$u_{\varepsilon}=\Theta_{\varepsilon}\big{\\{}X_{\varepsilon}-\mathbb{E}[X_{\varepsilon}]\big{\\}}+\bar{\Theta}_{\varepsilon}\mathbb{E}[X_{\varepsilon}],$
(5.12)

with $X_{\varepsilon}$ solving the closed-loop system:

$\left\\{\begin{aligned}
dX_{\varepsilon}(s)&=\big{\\{}(A+B\Theta_{\varepsilon})(X_{\varepsilon}-\mathbb{E}[X_{\varepsilon}])+[(A+\bar{A})+(B+\bar{B})\bar{\Theta}_{\varepsilon}]\mathbb{E}[X_{\varepsilon}]\big{\\}}ds\\\
&\hphantom{=\
}+\big{\\{}(C+D\Theta_{\varepsilon})(X_{\varepsilon}-\mathbb{E}[X_{\varepsilon}])+[(C+\bar{C})+(D+\bar{D})\bar{\Theta}_{\varepsilon}]\mathbb{E}[X_{\varepsilon}]\big{\\}}dW(s),\\\
X_{\varepsilon}(0)&=x.\end{aligned}\right.$

For any $\varepsilon>0$ and $x\in\mathbb{R}^{n}$, the value of Problem (MF-
SG)ε at $x$ is given by

$V_{\varepsilon}(x)=J_{\varepsilon}(x;u_{1}^{\varepsilon},u_{2}^{\varepsilon}),$

where $u_{\varepsilon}=(u_{1}^{\varepsilon},u_{2}^{\varepsilon})$ is defined
by (5.12). If the value $V(x)$ of Problem (MF-SG) exists for some
$x\in\mathbb{R}^{n}$, then by the same arguments as in the proof of [24,
Proposition 3.5] we have

$\lim_{\varepsilon\to 0}V_{\varepsilon}(x)=V(x).$

This means that the family $\\{u_{\varepsilon}\\}_{\varepsilon>0}$ defined by
(5.12) is an approximate sequence of Problem (MF-SG). But as pointed out in
[24], the existence of a value function does not imply that Problem (MF-SG)
has an open-loop saddle point. Thus in terms of the family
$\\{u_{\varepsilon}\\}_{\varepsilon>0}$, we present the following
characterization for the open-loop solvability of Problem (MF-SG), which is
the main result of this section.

######  Theorem 5.1.

Let (H1)–(H2) and (5.1) hold. Let $x\in\mathbb{R}^{n}$ be any given initial
state and $\\{u_{\varepsilon}\\}_{\varepsilon>0}$ be the sequence defined by
(5.12). Then the following statements are equivalent:

  1. (a)

Problem (MF-SG) has an open-loop saddle point at $x$;

  2. (b)

the family $\\{u_{\varepsilon}\\}_{\varepsilon>0}$ is bounded in the Hilbert
space $L^{2}_{\mathbb{F}}(0,T;\mathbb{R}^{m_{1}+m_{2}})\equiv{\cal
U}_{1}\times{\cal U}_{2}$, i.e.,

$\sup_{\varepsilon>0}\,\mathbb{E}\int_{0}^{T}|u_{\varepsilon}(s)|^{2}ds<\infty;$

  3. (c)

the family $\\{u_{\varepsilon}\\}_{\varepsilon>0}$ is strongly convergent in
$L^{2}_{\mathbb{F}}(0,T;\mathbb{R}^{m_{1}+m_{2}})$ as $\varepsilon\to 0$.

Whenever (a), (b), or (c) is satisfied, the strong limit of
$\\{u_{\varepsilon}\\}_{\varepsilon>0}$ is an open-loop saddle point of
Problem (MF-SG) for the initial state $x$.

######  Remark 5.2.

Since there is no coupled system in the above perturbation approach and all
the equations involved can be solved by iteration method, it will be much more
convenient for computational purposes.

The perturbation approach in stochastic LQ control problems was initially
introduced by Sun–Li–Yong [25], and further sharpened by Wang–Sun–Yong [31]
for finding the so-called weak closed-loop optimal strategies. However,
compared with the control problems [25, 31], in the game problem there are
some new difficulties, especially in proving the boundedness of
$\\{u_{\varepsilon}\\}_{\varepsilon>0}$. Before proving Theorem 5.1, we
present the following tailormade example, from which we can perceive some
essential differences between the perturbation approaches of LQ game and
control problems.

######  Example 5.3.

For any $x\in\mathbb{R}$, consider the one-dimensional state equation

$\left\\{\begin{aligned} \dot{X}(s)&=su_{1}(s)+u_{2}(s),\quad s\in[0,1],\\\
X(0)&=x,\end{aligned}\right.$ (5.13)

and the quadratic functional

$J(x;u_{1},u_{2})=-|X(1)|^{2}+\int_{0}^{1}s^{2}|u_{1}(s)|^{2}ds.$ (5.14)

In the example, we let ${\cal U}_{1}={\cal U}_{2}$ be the space of
$\mathbb{R}$-valued square-integrable functions on $[0,1]$. Note that

$\displaystyle J(1;0,u_{2})$
$\displaystyle=-|X(1)|^{2}=-\Big{|}1+\int_{0}^{1}u_{2}(s)ds\Big{|}^{2}\leqslant
0=J(1;0,-1),\qquad\forall u_{2}\in{\cal U}_{2};$ $\displaystyle J(1;u_{1},-1)$
$\displaystyle=-\Big{|}\int_{0}^{1}su_{1}(s)ds\Big{|}^{2}+\int_{0}^{1}|su_{1}(s)|^{2}ds\geqslant
0=J(1;0,-1),\quad\forall u_{1}\in{\cal U}_{1}.$

Thus the control pair $(0,-1)$ is an open-loop saddle point for $x=1$ and the
convexity-concavity condition (5.1) holds.

For any $\varepsilon>0$ and $(u_{1},u_{2})\in{\cal U}_{1}\times{\cal U}_{2}$,
denote

$J_{\varepsilon}(x;u_{1},u_{2})=J(x;u_{1},u_{2})-\varepsilon\int_{0}^{1}|u_{2}(s)|^{2}ds.$
(5.15)

Then

$\displaystyle
J_{\varepsilon}(0;0,u_{2})=J(0;0,u_{2})-\varepsilon\int_{0}^{1}|u_{2}(s)|^{2}ds\leqslant-\varepsilon\int_{0}^{1}|u_{2}(s)|^{2}ds,\quad\forall
u_{2}\in{\cal U}_{2},$ (5.16) $\displaystyle
J_{\varepsilon}(0;u_{1},0)=J(0;u_{1},0)\geqslant 0,\quad\forall u_{1}\in{\cal
U}_{1}.$ (5.17)

Roughly speaking, the above implies that the convexity of the mapping
$u_{1}\mapsto J_{\varepsilon}(x;u_{1},u_{2})$ equals that of $u_{1}\mapsto
J(x;u_{1},u_{2})$ and the concavity of $u_{2}\mapsto
J_{\varepsilon}(x;u_{1},u_{2})$ is stronger than that of $u_{2}\mapsto
J(x;u_{1},u_{2})$. However, we will show that the LQ game problem associated
with (5.13)–(5.15) has no open-loop saddle point for $x=1$.

We shall prove the above claim by contradiction. Suppose that the LQ game
problem associated with (5.13)–(5.15) has an open-loop saddle point
$(u_{1}^{*},u_{2}^{*})$. Then we must have

$\int_{0}^{1}u_{2}^{*}(s)ds=-1.$ (5.18)

Otherwise, $u^{*}_{1}$ is optimal for the LQ control problem associated with
the state equation

$\dot{X}(s)=su_{1}(s),\quad s\in[0,T];\quad X(0)=h\triangleq
1+\int_{0}^{1}u_{2}^{*}(s)ds\neq 0,$ (5.19)

and the cost functional $J(u_{1})=J_{\varepsilon}(1;u_{1},u_{2}^{*})$. The
corresponding optimal state is denoted by $X^{*}$. Then by [25, Corollary
3.3], $u_{1}^{*}$ satisfies the following stationary condition:

$sY^{*}(s)+s^{2}u_{1}^{*}(s)=0,\quad s\in[0,1],$ (5.20)

where $Y^{*}$ is the solution to the following adjoint equation:

$\dot{Y}^{*}(s)=0,\quad s\in[0,T];\quad Y^{*}(1)=-X^{*}(1).$ (5.21)

By solving (5.21), the stationary condition (5.20) can be rewritten as

$-sX^{*}(1)+s^{2}u_{1}^{*}(s)=0,\quad s\in[0,1].$ (5.22)

Recalling that $u_{1}^{*}\in{\cal U}_{1}$ is a square-integrable function, we
get

$X^{*}(1)=0,$ (5.23)

which implies that

$u^{*}_{1}(s)=0,\quad s\in[0,1].$ (5.24)

Thus,

$0=X^{*}(1)=h+\int_{0}^{1}su_{1}^{*}(s)ds=h.$ (5.25)

This contradicts (5.19) and thus (5.18) holds. Further, by (5.18), note that

$\displaystyle J_{\varepsilon}(1;u_{1},u^{*}_{2})$
$\displaystyle=-\Big{|}\int_{0}^{1}su_{1}(s)ds\Big{|}^{2}+\int_{0}^{1}|su_{1}(s)|^{2}ds-\varepsilon\int_{0}^{1}|u^{*}_{2}(s)|^{2}ds,$
$\displaystyle J_{\varepsilon}(1;u^{*}_{1},u^{*}_{2})$
$\displaystyle=\inf_{u_{1}\in{\cal
U}_{1}[0,1]}J_{\varepsilon}(1;u_{1},u^{*}_{2}),$ (5.26)

then we have

$u_{1}^{*}(s)=0,\quad s\in[0,1].$ (5.27)

It follows that $u^{*}_{2}$ is optimal for the control problem associated with
the state equation

$\dot{X}(s)=u_{2}(s),\quad s\in[0,1];\quad X(0)=1,$ (5.28)

and the cost functional $J(u_{2})=-J_{\varepsilon}(1;0,u_{2})$. Then,
$u_{2}^{*}$ satisfies the stationary condition:

$Y^{*}(s)+\varepsilon u_{2}^{*}(s)=0,\quad s\in[0,1],$ (5.29)

with $Y^{*}$ solving the adjoint equation:

$\dot{Y}^{*}(s)=0,\quad s\in[0,1];\quad Y^{*}(1)=X^{*}(1)=0.$ (5.30)

It follows that

$u_{2}^{*}(s)=-{Y^{*}(s)\over\varepsilon}=0,\quad s\in[0,1],$ (5.31)

which contradicts (5.18). Therefore, the claim is proved; that is the LQ game
problem associated with (5.13)–(5.15) has no open-loop saddle points for
$x=1$.

In the LQ control problems [25, 31], the perturbed cost functional
$J_{\varepsilon}(x;u)$ is defined by adding $\varepsilon\|u\|^{2}$ to the
original one $J(x;u)$, where $u$ is the control process. By the monotonicity
of the mapping $\varepsilon\mapsto V_{\varepsilon}(x)$, [25] showed that
$\|u_{\varepsilon}\|^{2}$ is bounded by $\|u^{*}\|^{2}$, provided the original
problem has an optimal control $u^{*}$.

From 5.3, we see that the new game associated with
$J(x;u_{1},u_{2})-\varepsilon\|u_{2}\|^{2}$ possibly has no saddle point even
if the original is open-loop solvable. Thus in the perturbation approach of
games, adding both $\varepsilon\|u_{1}\|^{2}$ and $-\varepsilon\|u_{2}\|^{2}$
to the original functional turns out to be necessary, due to which the value
function $V_{\varepsilon}(x)$ is not monotone in $\varepsilon$. Noticing this
key point, a seemingly feasible approach is to introduce two parameters
$\varepsilon_{1},\varepsilon_{2}>0$ and consider the game problem with the
functional:

$J_{\varepsilon_{1},\varepsilon_{2}}(x;u_{1},u_{2})=J(x;u_{1}.u_{2})+\varepsilon_{1}\|u_{1}\|^{2}-\varepsilon_{2}\|u_{2}\|^{2}.$

Since $J_{\varepsilon_{1},\varepsilon_{2}}$ is monotone in each
$\varepsilon_{i}$, it seems that the convergence of
$\\{u_{\varepsilon_{1},\varepsilon_{2}}\\}_{\varepsilon_{1},\varepsilon_{2}>0}$
can be obtained by letting $\varepsilon_{1}\to 0$ and $\varepsilon_{2}\to 0$
separately. But in fact, after letting $\varepsilon_{1}\to 0$, one cannot take
the limit by letting $\varepsilon_{2}\to 0$, because the game with the
functional $J_{0,\varepsilon_{2}}$ is possibly unsolvable (see 5.3). In
conclusion, the perturbation approaches of LQ games (i.e., Theorem 5.1) and
controls (i.e., [25, 31]) are essentially different.

###  5.2 Proof of Theorem 5.1

With the preparations in Sections 2, 3 and Subsection 5.1, now we are ready to
prove Theorem 5.1 by a Hilbert space method, in which Proposition 2.4 and the
Mazur’s theorem (see Yosida [36, p.120, Theorem 2]) play important roles.

_Proof of Theorem 5.1._ (i) We begin by proving the implication (a)
$\Rightarrow$ (b). Let $v^{*}$ be an open-loop saddle point of Problem (MF-SG)
for the initial state $x$. Then by 3.1, $v^{*}$ must satisfy

${\cal M}v^{*}+{\cal K}x=0,$ (5.32)

where the operators ${\cal M}$ and ${\cal K}$ are defined by (3.9). On the
other hand, 3.2 shows that for any $\varepsilon>0$, the unique open-loop
saddle point $u_{\varepsilon}$ of Problem (MF-SG)ε for $x$ can be also given
by

$u_{\varepsilon}=-{\cal M}^{-1}_{\varepsilon}{\cal K}x,$ (5.33)

where

$\displaystyle{\cal M}_{\varepsilon}\triangleq{\cal
M}+\begin{pmatrix}\varepsilon I_{m_{1}}&0\\\ 0&-\varepsilon
I_{m_{2}}\end{pmatrix}=\begin{pmatrix}{\cal M}_{11}+\varepsilon
I_{m_{1}}&{\cal M}_{12}\\\ {\cal M}_{21}&{\cal M}_{22}-\varepsilon
I_{m_{2}}\end{pmatrix}.$ (5.34)

Combining (5.32) with (5.33) yields that

$u_{\varepsilon}=-{\cal M}^{-1}_{\varepsilon}{\cal K}x={\cal
M}^{-1}_{\varepsilon}{\cal M}v^{*}.$ (5.35)

Then by 2.4, noting (5.1), we have

$\|u_{\varepsilon}\|^{2}=\|{\cal M}^{-1}_{\varepsilon}{\cal
M}v^{*}\|^{2}\leqslant\|{\cal M}^{-1}_{\varepsilon}{\cal
M}\|^{2}\|v^{*}\|^{2}\leqslant\|v^{*}\|^{2}.$ (5.36)

Thus, $\\{u_{\varepsilon}\\}_{\varepsilon>0}$ is bounded in the Hilbert space
$L^{2}_{\mathbb{F}}(0,T;\mathbb{R}^{m_{1}+m_{2}})$.

(ii) We show that (b) $\Rightarrow$ (a). For convenience, we let

$\sup_{\varepsilon>0}\|u_{\varepsilon}\|^{2}\leqslant\|\bar{u}\|^{2},\quad\hbox{for
some}~{}\bar{u}\in{\cal U}_{1}\times{\cal U}_{2}.$ (5.37)

Since $\\{u_{\varepsilon}\\}_{\varepsilon>0}$ is bounded in the Hilbert space
$L^{2}_{\mathbb{F}}(0,T;\mathbb{R}^{m_{1}+m_{2}})$, it admits a weakly
convergent subsequence. We denote this subsequence by
$\\{u_{\varepsilon_{j}}\\}_{j\geqslant
1}=\\{(u_{1}^{\varepsilon_{j}},u_{2}^{\varepsilon_{j}})\\}_{j\geqslant 1}$ and
its weak limit by $u^{*}=(u_{1}^{*},u_{2}^{*})$. Then by Mazur’s theorem there
exist $\lambda_{kj}\in[0,1]$, $j=1,2,...,N_{k}$ such that

$\sum_{j=1}^{N_{k}}\lambda_{kj}=1,\quad k=1,2,...,$ (5.38)

and

$\Big{\|}\sum_{j=1}^{N_{k}}\lambda_{kj}u_{\varepsilon_{k+j}}-u^{*}\Big{\|}\to
0,\quad\hbox{as}\quad k\to\infty.$ (5.39)

By the convexity of the mapping $u_{1}\mapsto J(x;u_{1},u_{2})$, we have

$\displaystyle
J\big{(}x;\sum_{j=1}^{N_{k}}\lambda_{kj}u_{1}^{\varepsilon_{k+j}},\sum_{j=1}^{N_{k}}\lambda_{kj}u_{2}^{\varepsilon_{k+j}}\big{)}\leqslant\sum_{j=1}^{N_{k}}\lambda_{kj}J\big{(}x;u_{1}^{\varepsilon_{k+j}},\sum_{j=1}^{N_{k}}\lambda_{kj}u_{2}^{\varepsilon_{k+j}}\big{)}$
$\displaystyle\quad=\sum_{j=1}^{N_{k}}\lambda_{kj}J_{\varepsilon_{k+j}}\big{(}x;u_{1}^{\varepsilon_{k+j}},\sum_{j=1}^{N_{k}}\lambda_{kj}u_{2}^{\varepsilon_{k+j}}\big{)}-\sum_{j=1}^{N_{k}}\lambda_{kj}\varepsilon_{k+j}\|u_{1}^{\varepsilon_{k+j}}\|^{2}$
$\displaystyle\qquad+\sum_{j=1}^{N_{k}}\lambda_{kj}\varepsilon_{k+j}\Big{\|}\sum_{j=1}^{N_{k}}\lambda_{kj}u_{2}^{\varepsilon_{k+j}}\Big{\|}^{2}$
$\displaystyle\quad\leqslant\sum_{j=1}^{N_{k}}\lambda_{kj}J_{\varepsilon_{k+j}}\big{(}x;u_{1}^{\varepsilon_{k+j}},\sum_{j=1}^{N_{k}}\lambda_{kj}u_{2}^{\varepsilon_{k+j}}\big{)}+\sum_{j=1}^{N_{k}}\lambda_{kj}\varepsilon_{k+j}\Big{\|}\sum_{j=1}^{N_{k}}\lambda_{kj}u_{2}^{\varepsilon_{k+j}}\Big{\|}^{2}.$

By (5.37), we get

$\Big{\|}\sum_{j=1}^{N_{k}}\lambda_{kj}u_{2}^{\varepsilon_{k+j}}\Big{\|}^{2}\leqslant\sum_{j=1}^{N_{k}}\lambda_{kj}\|u_{2}^{\varepsilon_{k+j}}\|^{2}\leqslant\|\bar{u}\|^{2},$

and thus

$\displaystyle
J\big{(}x;\sum_{j=1}^{N_{k}}\lambda_{kj}u_{1}^{\varepsilon_{k+j}},\sum_{j=1}^{N_{k}}\lambda_{kj}u_{2}^{\varepsilon_{k+j}}\big{)}$
$\displaystyle\quad\leqslant\sum_{j=1}^{N_{k}}\lambda_{kj}J_{\varepsilon_{k+j}}\big{(}x;u_{1}^{\varepsilon_{k+j}},\sum_{j=1}^{N_{k}}\lambda_{kj}u_{2}^{\varepsilon_{k+j}}\big{)}+\sum_{j=1}^{N_{k}}\lambda_{kj}\varepsilon_{k+j}\|\bar{u}\|^{2}.$
(5.40)

Moreover, note that $(u_{1}^{\varepsilon_{k+j}},u_{2}^{\varepsilon_{k+j}})$ is
an open-loop saddle of Problem (MF-SG)${}_{\varepsilon_{k+j}}$. Thus for any
$u_{1}\in{\cal U}_{1}$,

$J_{\varepsilon_{k+j}}\big{(}x;u_{1}^{\varepsilon_{k+j}},\sum_{j=1}^{N_{k}}\lambda_{kj}u_{2}^{\varepsilon_{k+j}}\big{)}\leqslant
J_{\varepsilon_{k+j}}\big{(}x;u_{1}^{\varepsilon_{k+j}},u_{2}^{\varepsilon_{k+j}}\big{)}\leqslant
J_{\varepsilon_{k+j}}\big{(}x;u_{1},u_{2}^{\varepsilon_{k+j}}\big{)}.$ (5.41)

Substituting the above into (5.40) and then by the concavity of the mapping
$u_{2}\mapsto J(x;u_{1},u_{2})$, we have

$\displaystyle
J\big{(}x;\sum_{j=1}^{N_{k}}\lambda_{kj}u_{1}^{\varepsilon_{k+j}},\sum_{j=1}^{N_{k}}\lambda_{kj}u_{2}^{\varepsilon_{k+j}}\big{)}\leqslant\sum_{j=1}^{N_{k}}\lambda_{kj}J_{\varepsilon_{k+j}}\big{(}x;u_{1},u_{2}^{\varepsilon_{k+j}}\big{)}+\sum_{j=1}^{N_{k}}\lambda_{kj}\varepsilon_{k+j}\|\bar{u}\|^{2}$
$\displaystyle\quad\leqslant\sum_{j=1}^{N_{k}}\lambda_{kj}J\big{(}x;u_{1},u_{2}^{\varepsilon_{k+j}}\big{)}+\sum_{j=1}^{N_{k}}\lambda_{kj}\varepsilon_{k+j}\|u_{1}\|^{2}+\sum_{j=1}^{N_{k}}\lambda_{kj}\varepsilon_{k+j}\|\bar{u}\|^{2}$
$\displaystyle\quad\leqslant
J\big{(}x;u_{1},\sum_{j=1}^{N_{k}}\lambda_{kj}u_{2}^{\varepsilon_{k+j}}\big{)}+\sum_{j=1}^{N_{k}}\lambda_{kj}\varepsilon_{k+j}\|u_{1}\|^{2}+\sum_{j=1}^{N_{k}}\lambda_{kj}\varepsilon_{k+j}\|\bar{u}\|^{2}.$
(5.42)

Thus by (5.38)–(5.39) and the (strong) continuity of the mapping
$(u_{1},u_{2})\mapsto J(x;u_{1},u_{2})$, letting $k\to\infty$ in (5.42) yields
that

$J(x;u_{1}^{*},u_{2}^{*})\leqslant J(x;u_{1},u_{2}^{*}),\quad\forall
u_{1}\in{\cal U}_{1}.$ (5.43)

By the same argument as the above, we also have

$J(x;u_{1}^{*},u_{2}^{*})\geqslant J(x;u^{*}_{1},u_{2}),\quad\forall
u_{2}\in{\cal U}_{2}.$ (5.44)

The result then concludes from (5.43)–(5.44).

(iii) The implication (c) $\Rightarrow$ (b) is trivially true. We next prove
(b) $\Rightarrow$ (c). We first claim: The family
$\\{u_{\varepsilon}\\}_{\varepsilon>0}$ is weakly convergent as
$\varepsilon\to 0$ and the weak limit is an open-loop saddle point of Problem
(MF-SG) for $x$.

If the above claim holds, then the family
$\\{u_{\varepsilon}\\}_{\varepsilon>0}$ converges weakly to an open-loop
saddle point $u^{*}$ of Problem (MF-SG) as $\varepsilon\to 0$. Thus by the
weakly lower semicontinuity of the mapping $u\mapsto\|u\|^{2}$, we have

$\mathbb{E}\int_{0}^{T}|u^{*}(s)|^{2}ds\leqslant\liminf_{\varepsilon\to
0}\mathbb{E}\int_{0}^{T}|u_{\varepsilon}(s)|^{2}ds.$ (5.45)

On the other hand, by (5.36), with $v^{*}$ replaced by $u^{*}$, we obtain

$\mathbb{E}\int_{0}^{T}|u_{\varepsilon}(s)|^{2}ds\leqslant\mathbb{E}\int_{0}^{T}|u^{*}(s)|^{2}ds,\quad\forall\varepsilon>0.$
(5.46)

Combining (5.45) with (5.46) yields that

$\lim_{\varepsilon\to
0}\mathbb{E}\int_{0}^{T}|u_{\varepsilon}(s)|^{2}ds=\mathbb{E}\int_{0}^{T}|u^{*}(s)|^{2}ds.$

Recall that $u^{*}$ is the weak limit of
$\\{u_{\varepsilon}\\}_{\varepsilon>0}$. Then the above implies that

$\lim_{\varepsilon\to
0}\mathbb{E}\int_{0}^{T}|u_{\varepsilon}(s)-u^{*}(s)|^{2}ds=0.$

It shows that $\\{u_{\varepsilon}\\}_{\varepsilon>0}$ converges strongly to
$u^{*}$ as $\varepsilon\to 0$. Thus to show (b) $\Rightarrow$ (c), we only
need to prove the claim.

Noting that $L_{\mathbb{F}}^{2}(0,T;\mathbb{R}^{m_{1}+m_{2}})$ is a Hilbert
space, to verify the claim, it suffices to show that every weakly convergent
subsequence of $\\{u_{\varepsilon}\\}_{\varepsilon>0}$ has the same weak
limit, which is an open-loop saddle point of Problem (MF-SG) for $x$. Let
$u^{*}$ and $\hat{u}$ be the weak limits of two different weakly convergent
subsequences $\\{u_{\varepsilon^{i}_{k}}\\}_{k=1}^{\infty}$ $(i=1,2)$ of
$\\{u_{\varepsilon}\\}_{\varepsilon>0}$. Then by the same argument as in the
proof of (b) $\Rightarrow$ (a), we can show that both $u^{*}$ and $\hat{u}$
are open-loop saddle points. By the convexity of the mapping $u_{1}\mapsto
J(x;u_{1},u_{2})$, we have

$\displaystyle J\left(x;{u_{1}^{*}+\hat{u}_{1}\over
2},{u_{2}^{*}+\hat{u}_{2}\over 2}\right)$ $\displaystyle\leqslant{1\over
2}J\left(x;u_{1}^{*},{u_{2}^{*}+\hat{u}_{2}\over 2}\right)+{1\over
2}J\left(x;\hat{u}_{1},{u_{2}^{*}+\hat{u}_{2}\over 2}\right).$ (5.47)

Noting that both $u^{*}$ and $\hat{u}$ are open-loop saddle points, and the
mapping $u_{2}\mapsto J(x;u_{1},u_{2})$ is concave, we have

$\displaystyle{1\over 2}J\left(x;u_{1}^{*},{u_{2}^{*}+\hat{u}_{2}\over
2}\right)+{1\over 2}J\left(x;\hat{u}_{1},{u_{2}^{*}+\hat{u}_{2}\over
2}\right)$ $\displaystyle\quad\leqslant{1\over
2}J\left(x;u_{1}^{*},u_{2}^{*}\right)+{1\over
2}J\left(x;\hat{u}_{1},\hat{u}_{2}\right)$ $\displaystyle\quad\leqslant{1\over
2}J\left(x;u_{1},u_{2}^{*}\right)+{1\over 2}J\left(x;u_{1},\hat{u}_{2}\right)$
$\displaystyle\quad\leqslant J\left(x;u_{1},{u_{2}^{*}+\hat{u}_{2}\over
2}\right),\quad\forall u_{1}\in{\cal U}_{1}.$ (5.48)

Thus,

$J\left(x;{u_{1}^{*}+\hat{u}_{1}\over 2},{u_{2}^{*}+\hat{u}_{2}\over
2}\right)\leqslant J\left(x;u_{1},{u_{2}^{*}+\hat{u}_{2}\over
2}\right),\quad\forall u_{1}\in{\cal U}_{1}.$ (5.49)

Similarly, we can prove

$J\left(x;{u_{1}^{*}+\hat{u}_{1}\over 2},{u_{2}^{*}+\hat{u}_{2}\over
2}\right)\geqslant J\left(x;{u_{1}^{*}+\hat{u}_{1}\over
2},u_{2}\right),\quad\forall u_{2}\in{\cal U}_{2}.$ (5.50)

Combining (5.49) with (5.50), we get that ${u^{*}+\hat{u}\over 2}$ is also an
open-loop saddle point of Problem (MF-SG) with respect to $x$. Thus by (5.36),
with $v^{*}$ replaced by ${u^{*}+\hat{u}\over 2}$, we obtain

$\mathbb{E}\int_{0}^{T}|u_{\varepsilon^{i}_{k}}(s)|^{2}ds\leqslant\mathbb{E}\int_{0}^{T}\left|{u^{*}(s)+\hat{u}(s)\over
2}\right|^{2}ds,\quad i=1,2.$ (5.51)

By the weakly lower semicontinuity of the mapping $u\mapsto\|u\|^{2}$ again,
we have

$\displaystyle\mathbb{E}\int_{0}^{T}|u^{*}(s)|^{2}ds\leqslant\liminf_{k\to\infty}\mathbb{E}\int_{0}^{T}|u_{\varepsilon^{1}_{k}}(s)|^{2}ds,$
$\displaystyle\mathbb{E}\int_{0}^{T}|\hat{u}(s)|^{2}ds\leqslant\liminf_{k\to\infty}\mathbb{E}\int_{0}^{T}|u_{\varepsilon^{2}_{k}}(s)|^{2}ds.$

Thus by taking inferior limits on the both sides of (5.51), we get

$\displaystyle\mathbb{E}\int_{0}^{T}|u^{*}(s)|^{2}ds\leqslant\mathbb{E}\int_{0}^{T}\left|{u^{*}(s)+\hat{u}(s)\over
2}\right|^{2}ds,$
$\displaystyle\mathbb{E}\int_{0}^{T}|\hat{u}(s)|^{2}ds\leqslant\mathbb{E}\int_{0}^{T}\left|{u^{*}(s)+\hat{u}(s)\over
2}\right|^{2}ds,$

which implies that

$\mathbb{E}\int_{0}^{T}|u^{*}(s)-\hat{u}(s)|^{2}ds\leqslant 0.$

The claim is established.$\hfill\qed$

######  Remark 5.4.

The boundedness of $\|{\cal M}_{\varepsilon}^{-1}{\cal M}\|^{2}$ is sufficient
for that of $\|u_{\varepsilon}\|^{2}$, which implies that
$\\{u_{\varepsilon}\\}_{\varepsilon>0}$ admits a weakly convergent
subsequence. With the help of Mazur’s theorem, the (strong) convergence of
$\\{u_{\varepsilon}\\}_{\varepsilon>0}$ follows from the explicit upper bound
(which is exactly $1$) of $\|{\cal M}_{\varepsilon}^{-1}{\cal M}\|^{2}$.

######  Remark 5.5.

If Problem (MF-SG) reduces to a (mean-field) LQ optimal control problem, then
${\cal M}={\cal M}_{1}$ and (5.35) becomes

$u_{\varepsilon}={\cal M}_{1\varepsilon}^{-1}{\cal M}_{1}v^{*}=({\cal
M}_{1}+\varepsilon I)^{-1}{\cal M}_{1}v^{*}.$

Note that ${\cal M}_{1}$ is a positive operator, while ${\cal M}$ is
indefinite in general. Then the Hilbert space method brings the following new
viewpoint: The perturbation approaches of LQ controls and games are the
outcomes of the explicit norm estimates for perturbed positive operators and
indefinite operators, respectively.

##  6 Example

In this section, we present a simple example to illustrate the procedure for
finding the open-loop saddle points by Theorem 4.6 under the sufficient
condition (3.19); and identifying the open-loop solvability by Theorem 5.1
under the necessary condition (3.18).

######  Example 6.1.

Consider the one-dimensional state equation

$\left\\{\begin{aligned} dX(s)&=u_{1}(s)ds+u_{2}(s)dW(s),\quad s\in[0,1],\\\
X(0)&=x,\end{aligned}\right.$ (6.1)

and the quadratic functional

$J(x;u_{1},u_{2})=\mathbb{E}\Big{\\{}-|X(1)|^{2}+\int_{0}^{1}\Big{(}|u_{1}(s)|^{2}-|\mathbb{E}[u_{2}(s)]|^{2}\Big{)}ds\Big{\\}}.$
(6.2)

It is straightforward to see that

$J(0;u_{1},0)\geqslant 0\quad\hbox{and}\quad J(0;0,u_{2})\leqslant
0,\quad\forall(u_{1},u_{2})\in{\cal U}_{1}\times{\cal U}_{2}.$ (6.3)

Suppose that $(u_{1}^{*},u_{2}^{*})$ is an open-loop saddle point, then by
4.8, $(u_{1}^{*},u_{2}^{*})$ must satisfy

$u_{1}^{*}=-Y,\quad\mathbb{E}[u_{2}^{*}]=-Z,$

with

$\left\\{\begin{aligned} &dX(s)=-Y(s)ds-Z(s)dW(s),\\\ &dY(s)=Z(s)dW(s),\\\
&X(0)=x,\quad Y(T)=-X(T).\end{aligned}\right.$ (6.4)

Note that the solution of the Riccati equation associated with (6.4) is
$-{1\over s}$, which is not integrable over $[0,1]$, thus the decoupling
technique is not applicable. Moreover, FBSDE (6.4) does not satisfy the so-
called monotone condition in Hu–Peng [12]. Thus we also cannot determine the
solvability of (6.4) by [12] directly.

In the following, let us apply Theorem 5.1 to determine the open-loop
solvability of the game. For any $\varepsilon>0$, we denote

$J_{\varepsilon}(x;u_{1},u_{2})=J(x;u_{1},u_{2})+\varepsilon\mathbb{E}\int_{0}^{1}|u_{1}(s)|^{2}ds-\varepsilon\mathbb{E}\int_{0}^{1}|u_{2}(s)|^{2}ds.$
(6.5)

By (6.3), we have

$\displaystyle
J_{\varepsilon}(0;u_{1},0)\geqslant\varepsilon\mathbb{E}\int_{0}^{1}|u_{1}(s)|^{2}ds,\quad~{}~{}\forall
u_{1}\in{\cal U}_{1},$ (6.6) $\displaystyle
J_{\varepsilon}(0;0,u_{2})\leqslant-\varepsilon\mathbb{E}\int_{0}^{1}|u_{2}(s)|^{2}ds,\quad\forall
u_{2}\in{\cal U}_{2}.$

Then we can apply Theorem 4.6 to find the unique open-loop saddle point of
Problem (MF-SG)ε with state equation (6.1) and functional (6.5). The
corresponding Riccati equations (5.5)–(5.6) in the example read:

$\left\\{\begin{aligned}
&\dot{P_{\varepsilon}}(s)-{P_{\varepsilon}(s)^{2}\over 1+\varepsilon}=0,\quad
s\in[0,1],\\\ &\dot{\Pi_{\varepsilon}}(s)-{\Pi_{\varepsilon}(s)^{2}\over
1+\varepsilon}=0,\quad s\in[0,1],\\\
&P_{\varepsilon}(1)=-1,\quad\Pi_{\varepsilon}(1)=-1.\end{aligned}\right.$
(6.7)

Solving (6.7) by separating variables, we get

$P_{\varepsilon}(s)=-{1+\varepsilon\over
s+\varepsilon},\quad\Pi_{\varepsilon}(s)=-{1+\varepsilon\over
s+\varepsilon},\quad s\in[0,1].$ (6.8)

Define the corresponding feedback operators $\Theta_{\varepsilon}$ (5.10) and
$\bar{\Theta}_{\varepsilon}$ (5.11) by

$\Theta_{\varepsilon}(s)=\begin{pmatrix}{1\over s+\varepsilon}\\\
0\end{pmatrix},\quad\bar{\Theta}_{\varepsilon}(s)=\begin{pmatrix}{1\over
s+\varepsilon}\\\ 0\end{pmatrix},\quad s\in[0,1].$ (6.9)

Then the unique open-loop saddle point of Problem (MF-SG)ε is given by

$u_{\varepsilon}=\begin{pmatrix}u_{1}^{\varepsilon}\\\
u^{\varepsilon}_{2}\end{pmatrix}=\Theta_{\varepsilon}\big{\\{}X_{\varepsilon}-\mathbb{E}[X_{\varepsilon}]\big{\\}}+\bar{\Theta}_{\varepsilon}\mathbb{E}[X_{\varepsilon}]=\begin{pmatrix}{X_{\varepsilon}\over
s+\varepsilon}\\\ 0\end{pmatrix},\quad s\in[0,1],$ (6.10)

with $X_{\varepsilon}$ being the unique solution to the following closed-loop
system:

$\left\\{\begin{aligned} dX_{\varepsilon}(s)&={X_{\varepsilon}(s)\over
s+\varepsilon}ds,\quad s\in[0,1],\\\
X_{\varepsilon}(0)&=x.\end{aligned}\right.$ (6.11)

By the variation of constants formula for ordinary differential equations, we
get

$X_{\varepsilon}(s)={s+\varepsilon\over\varepsilon}x,\quad s\in[0,1].$ (6.12)

Combining the above with (6.10), we obtain the following explicit
representation of $u_{\varepsilon}$:

$u_{\varepsilon}=\begin{pmatrix}u^{\varepsilon}_{1}\\\
u^{\varepsilon}_{2}\end{pmatrix}=\begin{pmatrix}{x\over\varepsilon}\\\
0\end{pmatrix},\quad s\in[0,1].$ (6.13)

Moreover, note that

$\sup_{\varepsilon>0}\mathbb{E}\int_{0}^{1}|u_{\varepsilon}(s)|^{2}=\infty,\quad\hbox{if}\quad
x\neq 0,$ (6.14)

and

$\sup_{\varepsilon>0}\mathbb{E}\int_{0}^{1}|u_{\varepsilon}(s)|^{2}=0,\quad\hbox{if}\quad
x=0.$ (6.15)

Thus according to Theorem 5.1, the problem has no open-loop saddle point for
$x\neq 0$ and has an open-loop saddle point $(0,0)$ for $x=0$.

##  7 Appendix

_Proof of 2.3_. The proof is divided into four cases.

Case 1.  If $K$ is invertible and $M$ is positive definite, then

$\displaystyle L^{\top}ML-L^{\top}MK(K^{\top}MK+\delta I_{m})^{-1}K^{\top}ML$
$\displaystyle\quad\geqslant
L^{\top}ML^{\top}-L^{\top}MK(K^{\top}MK)^{-1}K^{\top}ML=0.$ (7.1)

Case 2.  If $m=n$, then there exist a sequence of invertible matrices
$\\{K_{\varepsilon}\\}_{\varepsilon>0}$ and a sequence of positive definite
matrices $\\{M_{\varepsilon}\\}_{\varepsilon>0}$ such that

$\lim_{\varepsilon\to
0}K_{\varepsilon}=K\quad\hbox{and}\quad\lim_{\varepsilon\to
0}M_{\varepsilon}=M.$ (7.2)

Then from the facts $M\geqslant 0$, $\delta>0$ and the result obtained in Case
1, we have

$\displaystyle L^{\top}ML-L^{\top}MK(K^{\top}MK+\delta I_{m})^{-1}K^{\top}ML$
$\displaystyle\quad=\lim_{\varepsilon\to
0}\big{[}L^{\top}M_{\varepsilon}L-L^{\top}M_{\varepsilon}K_{\varepsilon}(K_{\varepsilon}^{\top}M_{\varepsilon}K_{\varepsilon}+\delta
I_{m})^{-1}K_{\varepsilon}^{\top}M_{\varepsilon}L\big{]}\geqslant 0.$ (7.3)

Case 3.  If $n>m$, set $\hat{K}=(K,\textbf{0})$ such that
$\hat{K}\in\mathbb{R}^{n\times n}$, where 0 is the zero matrix with an
appropriate dimension. Then by the results obtained in Case 2, we have

$\displaystyle 0$ $\displaystyle\leqslant
L^{\top}ML-L^{\top}M\hat{K}(\hat{K}^{\top}M\hat{K}+\delta
I_{n})^{-1}\hat{K}^{\top}ML$
$\displaystyle=L^{\top}ML-\begin{pmatrix}L^{\top}MK&\textbf{0}\end{pmatrix}\begin{pmatrix}K^{\top}MK+\delta
I_{m}&\textbf{0}\\\ \textbf{0}&\delta
I_{n-m}\end{pmatrix}^{-1}\begin{pmatrix}K^{\top}ML\\\ \textbf{0}\end{pmatrix}$
$\displaystyle=L^{\top}ML-L^{\top}MK(K^{\top}MK+\delta I_{m})^{-1}K^{\top}ML.$
(7.4)

Case 4.  If $n<m$, set

$\bar{L}=\begin{pmatrix}L&\textbf{0}\\\
\textbf{0}&\textbf{0}\end{pmatrix},~{}\bar{M}=\begin{pmatrix}M&\textbf{0}\\\
\textbf{0}&\textbf{0}\end{pmatrix},~{}\bar{K}=\begin{pmatrix}K\\\
\textbf{0}\end{pmatrix},$

such that $\bar{L},\bar{K}\in\mathbb{R}^{m\times m}$ and
$\bar{M}\in\mathbb{S}_{+}^{m}$. By the results obtained in Case 2 again, we
have

$\displaystyle 0$
$\displaystyle\leqslant\bar{L}^{\top}\bar{M}\bar{L}-\bar{L}^{\top}\bar{M}\bar{K}(\bar{K}^{\top}\bar{M}\bar{K}+\delta
I_{m})^{-1}\bar{K}^{\top}\bar{M}\bar{L}$
$\displaystyle=\begin{pmatrix}L^{\top}ML&\textbf{0}\\\
\textbf{0}&\textbf{0}\end{pmatrix}-\begin{pmatrix}L^{\top}MK\\\
\textbf{0}\end{pmatrix}(K^{\top}MK+\delta
I_{m})^{-1}\begin{pmatrix}K^{\top}ML&\textbf{0}\end{pmatrix}$
$\displaystyle=\begin{pmatrix}L^{\top}ML-L^{\top}MK(K^{\top}MK+\delta
I_{m})^{-1}K^{\top}ML&\textbf{0}\\\ \textbf{0}&\textbf{0}\end{pmatrix},$ (7.5)

which implies that

$L^{\top}ML-L^{\top}MK(K^{\top}MK+\delta I_{m})^{-1}K^{\top}ML\geqslant 0.$

## Acknowledgements

The authors would like to thank the associate editor and the anonymous
referees for their suggestive comments, which lead to this improved version of
the paper.

## References

  * [1] N. U. Ahmed, Nonlinear diffusion governed by McKean–Vlasov equation on Hilbert space and optimal control, SIAM J. Control Optim., 46 (2007), pp. 356–378.
  * [2] D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Appl. Math. Optim., 63 (2011), pp. 341–356.
  * [3] J. Barreiro-Gomez, T. E. Duncan, and H. Tembine, Linear-Quadratic Mean-Field-Type Games: Jump-Diffusion Process With Regime Switching, IEEE Trans. Automat. Contr., 64 (2019), pp. 4329–4336.
  * [4] A. Bensoussan, J. Frehse, and P. Yam, Mean field games and mean field type control theory, New York, Springer, 2013.
  * [5] A. Bensoussan, K. C. J. Sung, S. C. P. Yam, and S. P. Yung, Linear-quadratic mean field games, J. Optim. Theory Appl., 169 (2016), pp. 496–529.
  * [6] P. Bernhard, Linear-quadratic, two-person, zero-sum differential games: Necessary and sufficient conditions, J. Optim. Theory Appl., 27 (1979), pp. 51–69.
  * [7] R. Buckdahn, B. Djehiche, and J. Li, A general stochastic maximum principle for SDEs of mean-field type, Appl. Math. Optim., 64 (2011) 197–216.
  * [8] R. Carmona and F. Delarue, Probabilistic Theory of Mean Field Games with Applications, I–II, Springer Nature, 2018.
  * [9] M. C. Delfour, Linear quadratic differential games: Saddle point and Riccati differential equations, SIAM J. Control Optim., 46 (2007), pp. 750–774.
  * [10] M. C. Delfour and O. D. Sbarba, Linear quadratic differential games: Closed loop saddle points, SIAM J. Control Optim., 47 (2009), pp. 3138–3166.
  * [11] P. J. Graber, Linear quadratic mean field type control and mean field games with common noise, with application to production of an exhaustible resource, Appl. Math. Optim., 74 (2016), pp. 459–486.
  * [12] Y. Hu and S. Peng. Solution of forward-backward stochastic differential equations, Probab. Theory Related Fields, 103 (1995), pp. 273–283.
  * [13] J. Huang, X. Li, and J. Yong, A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon, Math. Control Relat. Fields, 5 (2015), pp. 97–139.
  * [14] M. Huang, R. P. Malhamé, and P. E. Caines, Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle, Communications in Information and Systems, 6 (2006), pp. 221–252.
  * [15] J. M. Lasry and P. L. Lions, Mean field games, Jpn. J. Math., 2 (2007), pp. 229–260.
  * [16] X. Li, J. Shi, and J. Yong, Mean-Field Linear-Quadratic Stochastic Differential Games in an Infinite Horizon, arXiv:2007.06130, 2020.
  * [17] X. Li, J. Sun, and J. Xiong, Linear quadratic optimal control problems for mean-field backward stochastic differential equations, Appl. Math. Optim., 80 (2019), pp. 223–250.
  * [18] A. E. B. Lim and X. Y. Zhou, Linear-quadratic control of backward stochastic differential equations, SIAM J. Control Optim., 40 (2001), pp. 450–474.
  * [19] T. Meyer-Brandis, B. Øksendal, and X. Y. Zhou, A mean-field stochastic maximum principle via Malliavin calculus, Stochastics, 84 (2012), pp. 643–666.
  * [20] J. Moon, Linear-quadratic mean field stochastic zero-sum differential games, Automatica, 120 (2020), 109067.
  * [21] L. Mou and J. Yong, Two-person zero-sum linear quadratic stochastic differential games by a Hilbert space method, J. Industrial Management Optim., 2 (2006), pp. 95–117.
  * [22] H. Pham and X. Wei, Bellman equation and viscosity solutions for mean-field stochastic control problem, ESAIM Control Optim. Calc. Var., 24 (2018), pp. 437–461.
  * [23] J. Sun, Mean-field stochastic linear quadratic optimal control problems: Open-loop solvabilities, ESAIM Control Optim. Calc. Var., 23 (2017), pp. 1099–1127.
  * [24] J. Sun, Two-Person Zero-Sum Stochastic Linear-Quadratic Differential Games, SIAM J. Control Optim., 59 (2021), pp. 1804–1829.
  * [25] J. Sun, X. Li, and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM J. Control Optim., 54 (2016), pp. 2274–2308.
  * [26] J. Sun and H. Wang, Mean-field stochastic linear-quadratic optimal control problems: weak closed-loop solvability, Math. Control Relat. Fields, 11 (2021), pp. 47–71.
  * [27] J. Sun and H. Wang, Linear-Quadratic Optimal Control for Backward Stochastic Differential Equations with Random Coefficients, ESAIM Control Optim. Calc. Var., published online, https://doi.org/10.1051/cocv/2021049.
  * [28] J. Sun and J. Yong, Linear quadratic stocahastic differential games: open-loop and closed-loop saddle points, SIAM J. Control Optim., 52 (2014), pp. 4082–4121.
  * [29] J. Sun and J. Yong, Stochastic Linear-Quadratic Optimal Control Theory: Differential Games and Mean-Field Problems, Springer Briefs in Mathematics, 2020.
  * [30] R. Tian, Z. Yu, and R. Zhang, A closed-loop saddle point for zero-sum linear-quadratic stochastic differential games with mean-field type, Syst. Control Lett., 136 (2020), 104624.
  * [31] H. Wang, J. Sun, and J. Yong, Weak closed-loop solvability of stochastic linear-quadratic optimal control problems, Disc. Conti. Dyn. Sys., 39 (2019), pp. 2785–2805.
  * [32] H. Wang, T. Zariphopoulou, and X. Y. Zhou, Reinforcement learning in continuous time and space: A stochastic control approach, J. Mach. Learn. Res., 21 (2020), pp. 1–34.
  * [33] J. Yong, Linear-Quadratic Optimal Control Problems for Mean-Field Stochastic Differential Equations, SIAM J. Control Optim., 51 (2013), pp. 2809–2838.
  * [34] J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations – time-consistent solutions, Trans. Amer. Math. Soc., 369 (2017), pp. 5467–5523.
  * [35] J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer-Verlag, New York, 1999.
  * [36] K. Yosida, Functional Analysis, 6th ed., Springer-Verlag, Berlin, 1980.
  * [37] Z. Yu, An optimal feedback control-strategy pair for zero-sum linear-quadratic stochastic differential game: the Riccati equation approach, SIAM J. Control Optim., 53 (2015), pp. 2141–2167.
  * [38] P. Zhang, Some results on two-person zero-sum linear quadratic differential games, SIAM J. Control Optim., 43 (2005), pp. 2157–2165.
  * [39] X. Y. Zhou and D. Li, Continuous-time mean-variance portfolio selection: A stochastic LQ framework, Appl. Math. Optim., 42 (2000), pp. 19–33.