# ††thanks: This work was supported by the National Natural Science
Foundations of China under Grants 11571205, 11201264.

Wenyu Cong , Jingtao Shi School of Mathematics, Shandong University, Jinan
250100, P.R. China, E-mail<EMAIL_ADDRESS>author.
School of Mathematics, Shandong University, Jinan 250100, P.R. China, E-mail:
<EMAIL_ADDRESS>
# Direct approach of linear-quadratic Stackelberg mean field games of
backward-forward stochastic systems††thanks: This work is supported by
National Key R&D Program of China (2022YFA1006104), National Natural Science
Foundations of China (11971266, 12271304, 11831010), and Shandong Provincial
Natural Science Foundations (ZR2022JQ01, ZR2020ZD24, ZR2019ZD42).

Wenyu Cong , Jingtao Shi School of Mathematics, Shandong University, Jinan
250100, P.R. China, E-mail<EMAIL_ADDRESS>author.
School of Mathematics, Shandong University, Jinan 250100, P.R. China, E-mail:
<EMAIL_ADDRESS>
Abstract: This paper is concerned with a linear-quadratic (LQ) Stackelberg
mean field games of backward-forward stochastic systems, involving a backward
leader and a substantial number of forward followers. The leader initiates by
providing its strategy, and subsequently, each follower optimizes its
individual cost. A direct approach is applied to solve this game. Initially,
we address a mean field game problem, determining the optimal response of
followers to the leader’s strategy. Following the implementation of followers’
strategies, the leader faces an optimal control problem driven by high-
dimensional forward-backward stochastic differential equations (FBSDEs).
Through the decoupling of the high-dimensional Hamiltonian system using mean
field approximations, we formulate a set of decentralized strategies for all
players, demonstrated to be an $(\epsilon_{1},\epsilon_{2})$-Stackelberg
equilibrium.

Keywords: Stackelberg mean field game, direct approach, linear-quadratic
stochastic optimal control, forward-backward stochastic differential equation,
Stackelberg equilibrium

Mathematics Subject Classification: 93E20, 60H10, 49K45, 49N70, 91A23

##  1 Introduction

Mean field games (MFGs) have garnered increasing scholarly attention, finding
applications in diverse fields such as system control, applied mathematics,
and economics ([6], [13], [8], [9]). MFG theory serves as a framework for
describing the behavior of models characterized by large populations, where
the influence of the overall population is significant, despite the negligible
impact on individual entities. The methodological foundations of MFG,
initially proposed by Lasry and Lions [21] and independently by Huang et al.
[19], have proven effective and tractable for analyzing weakly coupled
stochastic controlled systems with mean field interactions, establishing
approximate Nash equilibria. In particular, within the linear-quadratic (LQ)
framework, MFGs offer a versatile modeling tool applicable to a myriad of
practical problems. The solutions derived from LQ-MFGs exhibit noteworthy and
elegant properties. Current scholarly discourse has extensively explored MFGs,
particularly within the LQ framework ([22], [7], [26], [20]). Huang et al.
[18] conducted a study on $\epsilon$-Nash equilibrium strategies in the
context of LQ-MFGs with discounted costs. This investigation was rooted in the
Nash certainty equivalence (NCE) approach. Subsequently, the NCE approach was
applied to scenarios involving long run average costs, in Li and Zhang [22].
In the domain of MFGs featuring major players, Huang [17] delved into
continuous-time LQ games, providing insights into $\epsilon$-Nash equilibrium
strategies. Huang et al. [15] introduced a backward-major and forward-minor
setup for an LQ-MFG, and decentralized $\epsilon$-Nash equilibrium strategies
for major and minor agents were obtained. Huang et al. [16] examined the
backward LQ-MFG of weakly coupled stochastic large population systems under
both full and partial information scenarios. Huang and Li [14] delved into an
LQ-MFG concerning a class of stochastic delayed systems. Xu and Zhang [39]
explored a general LQ-MFG for stochastic large population systems, where the
individual diffusion coefficient is contingent on the state and control of the
agent. Bensoussan et al. [5] considered an LQ-MFG with partial observation and
common noise.

The Stackelberg differential game problem, also known as the leader-follower
differential game problem, arises in markets where certain companies possess
greater authority to dominate others or individual entities. In response to
this characteristic, Stackelberg [34] introduced the concept of hierarchical
solution. Within the context of the Stackelberg differential game, there are
two players with asymmetric roles, one designated as the leader and the other
as the follower. To attain a pair of Stackelberg equilibrium solutions, the
differential game problem is typically bifurcated into two segments. In the
first segment, the problem of the follower is addressed. Initially, the leader
publicly announces their strategy, transforming the two-player differential
game into the single-player optimal control problem for the follower. In other
words, the follower promptly reacts, selecting an optimal strategy in response
to the strategy disclosed by the leader, aiming to minimize (or maximize)
their own cost functional. The second segment involves the leader choosing an
optimal strategy, given the assumption that the follower will adopt such an
optimal strategy, to minimize their own cost functional. This constitutes
another optimal control problem for the leader. In summary, decision-making
must be jointly accomplished by both players. Due to the asymmetry in roles,
one player must be subordinate to the other, necessitating that one player
makes their decision after the other has concluded their decision-making
process. The Stackelberg differential game problem holds significant relevance
in financial and economic practices, prompting an increasing focus in applied
research. Bagchi and Başar [1] explored an LQ Stackelberg stochastic
differential game, where the diffusion coefficient in the state equation does
not involve state and control variables. Yong [41] delved into a more
generalized framework of LQ leader-follower differential game problems. In
this study, coefficients of the state system and cost functional are
stochastic, the diffusion coefficient in the state equation includes control
variables, and the weight matrix in front of the control variables in the cost
functional is not necessarily positive definite. Bensoussan et al. [2]
introduced several solution concepts based on players’ information sets and
investigated LQ Stackelberg differential games under adaptive open-loop and
closed-loop memoryless information structures, where control variables do not
enter the diffusion coefficient in the state equation. Zheng and Shi [42]
investigated a Stackelberg game involving backward stochastic differential
equations (BSDEs) (Pardoux and Peng [31], Ma and Yong [25]). Feng et al. [12]
examined a Stackelberg game associated with BSDE featuring constraints. Sun et
al. [33] conducted research on a zero-sum LQ Stackelberg stochastic
differential game.

Stackelberg mean field games (Stackelberg MFGs), distinct from Stackelberg
stochastic differential games of mean field type (incorporating the expected
values of state and control variables, as seen in references [11], [23], [38],
[28], [24], etc.), have been increasingly capturing the attention of
researchers. Nourian et al. [30] studied a large population LQ leader-follower
stochastic multi-agent systems and established their
$(\epsilon_{1},\epsilon_{2})$-Stackelberg-Nash equilibrium. Bensoussan et al.
[4] and Bensoussan et al. [3] investigated Stackelberg MFGs featuring delayed
responses. Wang and Zhang [37] examined hierarchical games for multi-agent
systems involving a leader and a large number of followers with infinite
horizon tracking-type costs. Moon and Başar [27] considered the LQ Stackelberg
MFG with the adapted open-loop information structure, and derived
$(\epsilon_{1},\epsilon_{2})$-Stackelberg-Nash equilibrium. Yang and Huang
[40] conducted a study on LQ Stackelberg MFGs involving a major player
(leader) and $N$ minor players (followers). Si and Wu [32] explored a
backward-forward LQ Stackelberg MFG, where the leader’s state equation is
backward, and the followers’ state equation is forward. Wang [35] employed a
direct method to solve LQ Stackelberg MFGs with a leader and a substantial
number of followers. A static output feedback strategy for robust incentive
Stackelberg games with a large population for mean field stochastic systems
was investigated in Mukaidani et al. [29]. Dayanikli and Laurière [10]
proposed a numerical approach of machine learning techniques to solve
Stackelberg problems between a principal and a mean field of agents.

Conventionally, two approaches are employed in the resolution of mean field
games. One is termed the fixed-point approach (or top-down approach, NCE
approach, see [18], [19], [22], [6], [9]), which initiates the process by
employing mean field approximation and formulating a fixed-point equation. By
tackling the fixed-point equation and scrutinizing the optimal response of a
representative player, decentralized strategies can be formulated. The
alternative approach is known as the direct approach (or bottom-up approach,
refer to [21], [36], [20], [35]). This method commences by formally solving an
$N$-player game problem within a vast and finite population setting.
Subsequently, by decoupling or reducing high-dimensional systems, centralized
control can be explicitly derived, contingent on the state of a specific
player and the average state of the population. As the population size $N$
approaches infinity, the construction of decentralized strategies becomes
feasible. In [20], the authors addressed the connection and difference of
these two routes in an LQ setting.

In this paper, we explore a backward-forward LQ Stackelberg MFG with a leader
and multiple followers. The leader initiates the process by disclosing their
strategy, following which each follower optimizes its individual cost.
Employing the direct approach, we formulate
$(\epsilon_{1},\epsilon_{2})$-Stackelberg equilibrium strategies. With the
leader’s strategy given, we first address a MFG by the stochastic maximum
principle (SMP), resulting in a system of forward-backward stochastic
differential equations (FBSDEs). Subsequent to the followers implementing
their strategies, the leader is encounters an optimal control problem driven
by a forward SDE and two BSDEs. By variational analysis, we obtain the
centralized strategy for the leader. By decoupling a high-dimensional FBSDE
with mean field approximations, we construct a set of decentralized
$(\epsilon_{1},\epsilon_{2})$-Stackelberg equilibrium strategies, in terms of
a 6-dimensional FBSDE.

The main contributions of the paper are outlined as follows.

$\bullet$ We formulate a new type of LQ Stackelberg MFGs, where the state of
leader is a backward equation, and followers are forward equations. Note that
in our model, both the state and control variables of the leader enter into
the drift coefficient of the forward equation, which is different from those
in [15], [32]. Our model is also different with that in [35], where the
leader’s state is a forward equation. Moreover, the cross terms of the
leader’s and followers’ control variables enter into the followers’ cost
functionals. These new structures bring mathematical difficulties, and, to
some degree, they have some potential applications in reality.

$\bullet$ In lieu of the traditional fixed-point methodology, we embrace a
direct approach to tackle the complexities of our game problem, which was used
in [20], [35]. This strategic shift involves the meticulous decoupling of the
high-dimensional Hamiltonian system using mean field approximations. By
disentangling the high-dimensional Hamiltonian system through mean field
approximations, we formulate a set of decentralized strategies for all
players. This set is subsequently demonstrated to constitute an
$(\epsilon_{1},\epsilon_{2})$-Stackelberg equilibrium.

The paper is structured as follows. In Section 2, we formulate the problem of
backward-forward LQ Stackelberg MFG. In Section 3, we explore and determine
centralized strategies for $N$ followers and the leader. In Section 4, we
construct decentralized strategies and the proof of the asymptotic optimality
is rigorously presented. Section 5 engages in a numerical simulation to
demonstrate the effectiveness of our obtained results. Finally, some
conclusions and future research directions are given in Section 6.

The following notations will be used throughout this paper. We use $||\cdot||$
to denote the norm of a Euclidean space, or the Frobenius norm for matrices.
$\top$ denotes the transpose of a vector or matrix. For a symmetric matrix $Q$
and a vector $z$, $||z||^{2}_{Q}\equiv z^{\top}Qz$. For any real-valued scalar
functions $f(\cdot)$ and $g(\cdot)$ defined on $\mathbb{R}$, $f(x)=O(g(x))$
means that there exists a constant $C>0$ such that
$\lim_{x\to\infty}|\frac{f(x)}{g(x)}|=C$, where $|\cdot|$ is an absolute
value, which is also equivalent to saying that there exist $C>0$ and $x$ such
that $|f(x)|\leq C|g(x)|$ for any $x\geq x^{\prime}$.

Let $T>0$ be a finite time duration and
$(\Omega,\mathcal{F},\\{\mathcal{H}_{t}\\}_{0\leq t\leq T},\mathbb{P})$ be a
complete filtered probability space with the filtration
$\\{\mathcal{H}_{t}\\}_{0\leq t\leq T}$ augmented by all the $\mathbb{P}$-null
sets in $\mathcal{F}$. $\mathbb{E}$ denoted the expectation with respect to
$\mathbb{P}$. Let $L_{\mathcal{H}}^{2}(0,T;\cdot)$ be the set of all vector-
valued (or matrix-valued) $\mathcal{H}_{t}$-adapted processes $f(\cdot)$ such
that $\mathbb{E}\big{[}\int^{T}_{0}||f(t)||^{2}dt\big{]}<\infty$ and
$L^{2}_{\mathcal{H}_{t}}(\Omega;\cdot)$ be the set of
$\mathcal{H}_{t}$-measurable random variables, for $t\in[0,T]$.

##  2 Problem formulation

We consider a large-population system with one leader and $N$ followers, where
$N$ can be arbitrarily large. The states equation of the leader and the $i$th
follower, $1\leq i\leq N$, are given by the following controlled linear BSDE
and SDE, respectively:

$\left\\{\begin{aligned}
dx_{0}(t)&=\big{[}A_{0}x_{0}(t)+B_{0}u_{0}(t)+C_{0}z_{0}(t)+f_{0}\big{]}dt+z_{0}(t)dW_{0}(t),\\\
dx_{i}(t)&=\big{[}Ax_{i}(t)+Bu_{i}(t)+Fx_{0}(t)+Gu_{0}(t)+f\big{]}dt+DdW_{i}(t),\quad
t\in[0,T],\\\ x_{0}(T)&=\xi_{0},\quad x_{i}(0)=\xi_{i},\quad
i=1,\cdots,N,\end{aligned}\right.$ (2.1)

where $x_{0}(\cdot)$, $u_{0}(\cdot)$ are the state process and the control
process, $\xi_{0}\in L^{2}_{\mathcal{F}_{T}^{0}}(\Omega;\mathbb{R})$ is the
given terminal value of the leader; similarly, $x_{i}(\cdot)$, $u_{i}(\cdot)$
and $\xi_{i}$ are the state process, control process and initial value (random
variable) of the $i$th follower. In this paper, for simplicity, we assume the
dimensions of state process and control process are both one-dimensional.
Here, $A_{0},B_{0},C_{0},f_{0},A,B,F,G,f,D$ are scalar constants.
$W_{i}(\cdot),i=0,\cdots,N$ are a sequence of one-dimensional Brownian motions
defined on $(\Omega,\mathcal{F},\\{\mathcal{F}_{t}\\}_{0\leq t\leq
T},\mathbb{P})$. Let $\mathcal{F}_{t}$ be the $\sigma$-algebra generated by
$\\{\xi_{i},W_{0}(s),W_{i}(s),s\leq t,1\leq i\leq N\\}$. Denote
$\mathcal{F}_{t}^{0}$ be the $\sigma$-algebra generated by $\\{W_{0}(s),s\leq
t\\}$ and $\mathcal{F}_{t}^{i}$ be the $\sigma$-algebra generated by
$\\{\xi_{i},W_{0}(s),W_{i}(s),s\leq t\\},1\leq i\leq N$. Define the
decentralized control set for the leader as

$\mathscr{U}_{0}[0,T]:=\big{\\{}u_{0}(\cdot)|u_{0}(\cdot)\in
L^{2}_{\mathcal{F}^{0}}(0,T;\mathbb{R})\big{\\}}.$

Define the decentralized control set for the followers and $i$th follower as

$\mathscr{U}_{d}[0,T]:=\big{\\{}(u_{1}(\cdot),\cdots,u_{N}(\cdot))|u_{i}(\cdot)\in
L^{2}_{\mathcal{F}^{i}}(0,T;\mathbb{R})\big{\\}},$
$\mathscr{U}_{i}[0,T]:=\big{\\{}u_{i}(\cdot)|u_{i}(\cdot)\in
L^{2}_{\mathcal{F}^{i}}(0,T;\mathbb{R})\big{\\}},$

and the centralized control set for the followers as

$\mathscr{U}_{c}[0,T]:=\big{\\{}(u_{1}(\cdot),\cdots,u_{N}(\cdot))|u_{i}(\cdot)\in
L^{2}_{\mathcal{F}}(0,T;\mathbb{R}),1\leq i\leq N\big{\\}}.$

The cost functional of the leader is given by

$J^{N}_{0}(u_{0}(\cdot),u^{N}(\cdot))=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||x_{0}(t)-\Gamma_{0}x^{(N)}(t)-\eta_{0}||^{2}_{Q_{0}}+||u_{0}(t)||^{2}_{R_{0}}\right]dt+||x_{0}(0)||^{2}_{H_{0}}\right\\},$
(2.2)

where $u^{N}(\cdot):=(u_{1}(\cdot),\cdots,u_{N}(\cdot))$,
$x^{(N)}(\cdot):=\frac{1}{N}\sum_{i=1}^{N}x_{i}(\cdot)$ is called the state
average or mean field term of all followers, $Q_{0}$, $R_{0}$, $H_{0}$,
$\Gamma_{0}$ and $\eta_{0}$ are scalar constants. Notice that $Q_{0}$,
$\Gamma_{0}$ and $\eta_{0}$ determine the coupling between the leader and the
mean field of the $N$ followers, $R_{0}$ is the control performance weighting
parameter of the leader, and $H_{0}$ is the initial state cost parameter of
the leader.

For the $i$th follower, the cost functional is defined by

$\displaystyle J^{N}_{i}(u_{i}(\cdot),u_{-i}(\cdot),u_{0}(\cdot))$ (2.3)
$\displaystyle=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||x_{i}(t)-\Gamma
x^{(N)}(t)-\Gamma_{1}x_{0}(t)-\eta||^{2}_{Q}+||u_{i}(t)||^{2}_{R}+2u_{i}(t)Lu_{0}(t)\right]dt+||x_{i}(T)||^{2}_{H}\right\\},$

where
$u_{-i}(\cdot):=(u_{1}(\cdot),\cdots,u_{i-1}(\cdot),u_{i+1}(\cdot),\cdots,u_{N}(\cdot))$,
$Q$, $R$, $L$, $H$, $\Gamma$, $\Gamma_{1}$ and $\eta$ are scalar constants.
$Q$, $\Gamma$, $\Gamma_{1}$ and $\eta$ determine the coupling between the
$i$th follower, leader and followers’ mean field, whereas $L$ influences the
coupling between the leader and the $i$th follower through control term. Also,
$R$ serves as the control performance weighting parameter of the $i$th
follower, and $H$ represents the terminal state cost parameter. It is
noteworthy that the followers are (weakly) coupled with each other through the
mean field term $x^{(N)}$, and are (strongly) coupled with the leader’s state
$x_{0}$ and control $u_{0}$ included in their cost functionals.

We posit the following assumptions:

(A1) $\\{\xi_{i}\\},i=1,2,\cdots,N$ are a sequence of independent and
identically distributed (i.i.d., for short) random variables with
$\mathbb{E}[\xi_{i}]=\bar{\xi},i=1,2,\cdots,N$, and there exists a constant
$c$ such that $\sup_{1\leq i\leq N}\mathbb{E}[||\xi_{i}||^{2}]\leq c.$

(A2) $\\{W_{i}(t),0\leq i\leq N\\}$ are independent of each other, which are
also independent of $\\{\xi_{i},1\leq i\leq N\\}$.

(A3) $Q_{0}\geq 0,R_{0}>0,H_{0}\geq 0$ and $Q\geq 0,R>0,H\geq 0$.

Now, we present the precise definition of an
$(\epsilon_{1},\epsilon_{2})$-Stackelberg equilibrium.

######  Definition 2.1.

A set of strategies
$(u_{0}^{*}(\cdot),u_{1}^{*}(\cdot),\cdots,u_{N}^{*}(\cdot))$ is an
$(\epsilon_{1},\epsilon_{2})$-Stackelberg equilibrium with respect to
$\\{J^{N}_{i},0\leq i\leq N\\}$ if the following hold:

(i) For a given strategy of the leader $u_{0}(\cdot)\in\mathscr{U}_{0}[0,T]$,
$u^{N*}(\cdot)=(u_{1}^{*}(\cdot),\cdots,u_{N}^{*}(\cdot))$ constitutes an
$\epsilon_{1}$-Nash equilibrium, if there exists a constant $\epsilon_{1}\geq
0$ such that for all $i,1\leq i\leq N$,

$J^{N}_{i}(u_{i}^{*}(\cdot),u_{-i}^{*}(\cdot),u_{0}(\cdot))\leq\inf_{u_{i}(\cdot)\in\mathscr{U}_{i}[0,T]}J^{N}_{i}(u_{i}(\cdot),u_{-i}^{*}(\cdot),u_{0}(\cdot))+\epsilon_{1};$

(ii) There exists a constant $\epsilon_{2}\geq 0$ such that

$J^{N}_{0}(u_{0}^{*}(\cdot),u^{N*}[\cdot;u_{0}^{*}(\cdot)])\leq\inf_{u_{0}(\cdot)\in\mathscr{U}_{0}[0,T]}J^{N}_{0}(u_{0}(\cdot),u^{N*}[\cdot;u_{0}(\cdot)])+\epsilon_{2}.$

In this paper, we investigate the following problem.

(P0): Find an $(\epsilon_{1},\epsilon_{2})$-Stackelberg equilibrium solution
to (2.2), (2.3), subject to (2.1).

##  3 Centralized stratigies

###  3.1 Mean field Nash games for the $N$ followers

In this subsection, we consider the mean field Nash game for the $N$ followers
under an arbitrary strategy of the leader
$u_{0}(\cdot)\in\mathscr{U}_{0}[0,T]$. We suppose $u_{0}(\cdot)$ is fixed.
Then, due to first equation of (2.1), $x_{0}(\cdot)$ is also fixed. We now
consider the following game problem for $N$ followers.

(P1): Minimize $J^{N}_{i},i=1,\cdots,N$ of (2.3) over
$u(\cdot)\in\mathscr{U}_{c}[0,T]$.

For the sake of simplicity, the time variable $t$ will be omitted without
ambiguity.

We have the following result.

######  Proposition 3.1.

Under Assumption (A1)-(A3), let $u_{0}(\cdot)\in\mathscr{U}_{0}[0,T]$ be
given, for the initial value $\xi_{i},i=1,\cdots,N$, (P1) admits an optimal
control $\check{u}_{i}(\cdot)\in
L^{2}_{\mathcal{F}}(0,T;\mathbb{R}),i=1,\cdots,N$, if and only if the
following two conditions hold:

(i) The adapted solution
$(\check{x}_{i}(\cdot),\check{p}_{i}(\cdot),\check{q}^{j}_{i}(\cdot),i=1,\cdots,N,j=0,1,\cdots,N)$
to the FBSDE

$\left\\{\begin{aligned}
d\check{x}_{i}&=\big{[}A\check{x}_{i}+B\check{u}_{i}+Fx_{0}+Gu_{0}+f\big{]}dt+DdW_{i},\\\
d\check{p}_{i}&=-\left[A\check{p}_{i}+\left(1-\frac{\Gamma}{N}\right)Q\left(\check{x}_{i}-\Gamma\check{x}^{(N)}-\Gamma_{1}x_{0}-\eta\right)\right]dt+\sum_{j=0}^{N}\check{q}_{i}^{j}dW_{j},\quad
t\in[0,T],\\\
\check{x}_{i}(0)&=\xi_{i},\quad\check{p}_{i}(T)=H\check{x}_{i}(T),\quad
i=1,\cdots,N,\end{aligned}\right.$ (3.1)

satisfies the following stationarity condition:

$B\check{p}_{i}+R\check{u}_{i}+Lu_{0}=0,\quad a.e.,\ a.s.,\quad i=1,\cdots,N.$
(3.2)

(ii) For $i=1,\cdots,N$, the following convexity condition holds:

$\mathbb{E}\left\\{\int_{0}^{T}\left[Q\left(1-\frac{\Gamma}{N}\right)^{2}\tilde{x}_{i}^{2}+Ru_{i}^{2}\right]dt+H\tilde{x}_{i}^{2}(T)\right\\}\geq
0,\ i=1,\cdots,N,\quad\forall u_{i}(\cdot)\in
L^{2}_{\mathcal{F}}(0,T;\mathbb{R}),$ (3.3)

where $\tilde{x}_{i}(\cdot)$ is the solution to the following random
differential equation (RDE):

$\left\\{\begin{aligned} &d\tilde{x}_{i}=[A\tilde{x}_{i}+Bu_{i}]dt,\\\
&\tilde{x}_{i}(0)=0.\end{aligned}\right.$ (3.4)

###### Proof.

We consider the $i$th follower. For given $\xi_{i}\in
L^{2}_{\mathcal{F}_{0}^{i}}(\Omega;\mathbb{R})$,
$u_{0}(\cdot)\in\mathscr{U}_{0}[0,T]$, $\check{u}_{i}(\cdot)\in
L^{2}_{\mathcal{F}}(0,T;\mathbb{R})$, suppose that
$(\check{x}_{i}(\cdot),\check{p}_{i}(\cdot),\check{q}^{j}_{i}(\cdot),j=0,1,\cdots,N)$
is an adapted solution to FBSDE (3.1). For any $u_{i}(\cdot)\in
L^{2}_{\mathcal{F}}(0,T;\mathbb{R})$ and $\varepsilon\in\mathbb{R}$, let
$x_{i}^{\varepsilon}(\cdot)$ be the solution to the following perturbed state
equation:

$\left\\{\begin{aligned}
dx_{i}^{\varepsilon}&=\big{[}Ax_{i}^{\varepsilon}+B(\check{u}_{i}+\varepsilon
u_{i})+Fx_{0}+Gu_{0}+f\big{]}dt+DdW_{i},\\\
x_{i}^{\varepsilon}(0)&=\xi_{i}.\end{aligned}\right.$ (3.5)

Then,
$\tilde{x}_{i}^{\varepsilon}(\cdot):=\frac{x_{i}^{\varepsilon}(\cdot)-\check{x}_{i}(\cdot)}{\varepsilon}$
is independent of $\varepsilon$ and satisfies (3.4).

Applying Itô’s formula to $\check{p}_{i}(\cdot)\tilde{x}_{i}(\cdot)$,
integrating from 0 to $T$, and taking the expectation, we have

$\displaystyle
H\check{x}_{i}(T)\tilde{x}_{i}(T)=\mathbb{E}\big{[}\check{p}_{i}(T)\tilde{x}_{i}(T)-\check{p}_{i}(0)\tilde{x}_{i}(0)\big{]}$
$\displaystyle=\mathbb{E}\int_{0}^{T}\left\\{-\left[A\check{p}_{i}+\left(1-\frac{\Gamma}{N}\right)Q\left(\check{x}_{i}-\Gamma\check{x}^{(N)}-\Gamma_{1}x_{0}-\eta\right)\right]\tilde{x}_{i}+\big{[}A\tilde{x}_{i}+Bu_{i}\big{]}\check{p}_{i}\right\\}dt$
$\displaystyle=\mathbb{E}\int_{0}^{T}\left\\{-\left(1-\frac{\Gamma}{N}\right)Q\left(\check{x}_{i}-\Gamma\check{x}^{(N)}-\Gamma_{1}x_{0}-\eta\right)\tilde{x}_{i}+Bu_{i}\check{p}_{i}\right\\}dt.$

Hence,

$\displaystyle J^{N}_{i}(\check{u}_{i}(\cdot)+\varepsilon
u_{i}(\cdot),u_{-i}(\cdot),u_{0}(\cdot))-J^{N}_{i}(\check{u}_{i}(\cdot),u_{-i}(\cdot),u_{0}(\cdot))$
$\displaystyle=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||x_{i}^{\varepsilon}-\Gamma
x^{\varepsilon(N)}-\Gamma_{1}x_{0}-\eta||^{2}_{Q}+||\check{u}_{i}+\varepsilon
u_{i}||^{2}_{R}+2(\check{u}_{i}+\varepsilon
u_{i})Lu_{0}\right]dt+||x_{i}^{\varepsilon}(T)||^{2}_{H}\right\\}$
$\displaystyle\quad-\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||\check{x}_{i}-\Gamma\check{x}^{(N)}-\Gamma_{1}x_{0}-\eta||^{2}_{Q}+||\check{u}_{i}||^{2}_{R}+2\check{u}_{i}Lu_{0}\right]dt+||\check{x}_{i}(T)||^{2}_{H}\right\\}$
$\displaystyle=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||\left(1-\frac{\Gamma}{N}\right)(\check{x}_{i}+\varepsilon\tilde{x}_{i})-\Gamma
x^{(N-1)}_{-i}-\Gamma_{1}x_{0}-\eta||^{2}_{Q}+||\check{u}_{i}+\varepsilon
u_{i}||^{2}_{R}\right.\right.$
$\displaystyle\qquad\qquad+2(\check{u}_{i}+\varepsilon
u_{i})Lu_{0}\bigg{]}dt+||(\check{x}_{i}(T)+\varepsilon\tilde{x}_{i}(T))||^{2}_{H}\bigg{\\}}$
$\displaystyle\quad-\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||\left(1-\frac{\Gamma}{N}\right)\check{x}_{i}-\Gamma\check{x}^{(N-1)}_{-i}-\Gamma_{1}x_{0}-\eta||^{2}_{Q}+||\check{u}_{i}||^{2}_{R}+2\check{u}_{i}Lu_{0}\right]dt+||\check{x}_{i}(T)||^{2}_{H}\right\\}$
$\displaystyle=\frac{1}{2}\varepsilon^{2}\mathbb{E}\left\\{\int_{0}^{T}\bigg{[}Q\left(1-\frac{\Gamma}{N}\right)^{2}\tilde{x}_{i}^{2}+Ru_{i}^{2}\bigg{]}dt+H\tilde{x}_{i}^{2}(T)\right\\}$
$\displaystyle\quad+\varepsilon\mathbb{E}\left\\{\int_{0}^{T}\left[Q\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\left(\check{x}_{i}-\Gamma\check{x}^{(N)}-\Gamma_{1}x_{0}-\eta\right)+u_{i}R\check{u}_{i}+u_{i}Lu_{0}\right]dt+H\check{x}_{i}(T)\tilde{x}_{i}(T)\right\\}$
$\displaystyle=\frac{1}{2}\varepsilon^{2}\mathbb{E}\left\\{\int_{0}^{T}\bigg{[}Q\left(1-\frac{\Gamma}{N}\right)^{2}\tilde{x}_{i}^{2}+Ru_{i}^{2}\bigg{]}dt+H\tilde{x}_{i}^{2}(T)\right\\}+\varepsilon\mathbb{E}\left\\{\int_{0}^{T}u_{i}\left[B\check{p}_{i}+R\check{u}_{i}+Lu_{0}\right]dt\right\\},$

where only here
$x^{\varepsilon(N)}(\cdot):=\frac{x_{i}^{\varepsilon}(\cdot)}{N}+\frac{1}{N}\sum_{j\neq
i}x_{j}(\cdot)$,
$\check{x}^{(N)}(\cdot):=\frac{\check{x}_{i}(\cdot)}{N}+\frac{1}{N}\sum_{j\neq
i}x_{j}(\cdot)$, and $x^{(N-1)}_{-i}(\cdot)=\frac{1}{N}\sum_{j\neq
i}x_{j}(\cdot)$. Therefore,

$J^{N}_{i}(\check{u}_{i}(\cdot),u_{-i}(\cdot),u_{0}(\cdot))\leq
J^{N}_{i}(\check{u}_{i}(\cdot)+\varepsilon
u_{i}(\cdot),u_{-i}(\cdot),u_{0}(\cdot))$

if and only if (3.2) and (3.3) hold. The proof is complete. ∎

Based on Assumption (A3), we can figure out that the open-loop optimal
strategies for followers are

$\check{u}_{i}=-R^{-1}[B\check{p}_{i}+Lu_{0}],\quad a.e.,\ a.s.,\quad
i=1,\cdots,N,$ (3.6)

so the related Hamiltonian system can be represented by

$\left\\{\begin{aligned}
d\check{x}_{i}&=\big{[}A\check{x}_{i}-BR^{-1}B\check{p}_{i}+Fx_{0}+(G-BR^{-1}L)u_{0}+f\big{]}dt+DdW_{i},\\\
d\check{p}_{i}&=-\left[A\check{p}_{i}+\left(1-\frac{\Gamma}{N}\right)Q\left(\check{x}_{i}-\Gamma\check{x}^{(N)}-\Gamma_{1}x_{0}-\eta\right)\right]dt+\sum_{j=0}^{N}\check{q}_{i}^{j}dW_{j},\\\
\check{x}_{i}(0)&=\xi_{i},\quad\check{p}_{i}(T)=H\check{x}_{i}(T),\quad
i=1,\cdots,N.\end{aligned}\right.$ (3.7)

We consider the transformation
$\check{p}_{i}(\cdot)=P_{N}(\cdot)\check{x}_{i}(\cdot)+K_{N}(\cdot)\check{x}^{(N)}(\cdot)+\check{\phi}_{N}(\cdot),i=1,\cdots,N$,
where $P_{N}(\cdot)$, $K_{N}(\cdot)$ are differential functions with
$P_{N}(T)=H$, $K_{N}(T)=0$, process pair $(\phi_{N}(\cdot),V_{N}(\cdot))$
satisfies a BSDE as follows:

$d\phi_{N}(t)=\Lambda_{N}(t)dt+V_{N}(t)dW_{0}(t),\quad\phi_{N}(T)=0,$

where process $\Lambda_{N}(\cdot)$ will be determined later. By Itô’s formula,
we get

$\displaystyle d\check{p}_{i}$
$\displaystyle=\dot{P}_{N}\check{x}_{i}dt+P_{N}\Big{\\{}\big{[}A\check{x}_{i}-BR^{-1}B\big{(}P_{N}\check{x}_{i}+K_{N}\check{x}^{(N)}+\check{\phi}_{N}\big{)}+Fx_{0}+(G-BR^{-1}L)u_{0}+f\big{]}dt$
$\displaystyle\quad+DdW_{i}\Big{\\}}+\dot{K}_{N}\check{x}^{(N)}dt+K_{N}\Big{\\{}\big{[}A\check{x}^{(N)}-BR^{-1}B\big{(}(P_{N}+K_{N})\check{x}^{(N)}+\check{\phi}_{N}\big{)}+Fx_{0}$
$\displaystyle\quad+(G-BR^{-1}L)u_{0}+f\big{]}dt+\frac{1}{N}\sum_{j=1}^{N}DdW_{j}\Big{\\}}+\check{\Lambda}_{N}dt+\check{V}_{N}dW_{0}$
$\displaystyle=-\left[A\left(P_{N}\check{x}_{i}+K_{N}\check{x}^{(N)}+\check{\phi}_{N}\right)+\left(1-\frac{\Gamma}{N}\right)Q\left(\check{x}_{i}-\Gamma\check{x}^{(N)}-\Gamma_{1}x_{0}-\eta\right)\right]dt+\sum_{j=0}^{N}\check{q}_{i}^{j}dW_{j},$

where the last equality is due to the second equation of (3.7). Comparing the
coefficients of the corresponding terms, we have

$\check{q}_{i}^{i}=P_{N}D+\frac{K_{N}D}{N},\quad\check{q}_{i}^{j}=\frac{K_{N}D}{N},\quad
1\leq j\neq i\leq N,\quad\check{q}_{i}^{0}=\check{V}_{N},$
$\dot{P}_{N}+AP_{N}+P_{N}A-P_{N}BR^{-1}BP_{N}+\left(1-\frac{\Gamma}{N}\right)Q=0,\quad
P_{N}(T)=H,$ (3.8)
$\dot{K}_{N}+AK_{N}+K_{N}A-P_{N}BR^{-1}BK_{N}-K_{N}BR^{-1}B(P_{N}+K_{N})-\left(1-\frac{\Gamma}{N}\right)Q\Gamma=0,\quad
K_{N}(T)=0,$ (3.9) $\displaystyle d\check{\phi}_{N}$
$\displaystyle=-\bigg{\\{}\left[A-(P_{N}+K_{N})BR^{-1}B\right]\check{\phi}_{N}+(P_{N}+K_{N})(G-BR^{-1}L)u_{0}+(P_{N}+K_{N})Fx_{0}$
(3.10)
$\displaystyle\quad+(P_{N}+K_{N})f-\left(1-\frac{\Gamma}{N}\right)Q(\Gamma_{1}x_{0}+\eta)\bigg{\\}}dt+\check{V}_{N}dW_{0},\quad\phi_{N}(T)=0.$

Let $\Pi_{N}(\cdot):=P_{N}(\cdot)+K_{N}(\cdot)$, then $\Pi_{N}(\cdot)$
satisfies

$\dot{\Pi}_{N}+A\Pi_{N}+\Pi_{N}A-\Pi_{N}BR^{-1}B\Pi_{N}+\left(1-\frac{\Gamma}{N}\right)Q-\left(1-\frac{\Gamma}{N}\right)Q\Gamma=0,\quad\Pi_{N}(T)=H,$
(3.11)

thus (3.10) can be rewritten as

$\displaystyle d\check{\phi}_{N}$
$\displaystyle=-\bigg{\\{}\left[A-\Pi_{N}BR^{-1}B\right]\check{\phi}_{N}+\Pi_{N}(G-BR^{-1}L)u_{0}+\Pi_{N}Fx_{0}$
(3.12)
$\displaystyle\quad+\Pi_{N}f-\left(1-\frac{\Gamma}{N}\right)Q(\Gamma_{1}x_{0}+\eta)\bigg{\\}}dt+\check{V}_{N}dW_{0},\quad\phi_{N}(T)=0.$

Note that (3.8) is a symmetric Riccati differential equation, if it satisfies
Assumption (A3) and $1-\frac{\Gamma}{N}\geq 0$, then it admits a unique
solution. Similarly, (3.11) is a symmetric Riccati differential equation, if
it satisfies Assumption (A3) and $(1-\frac{\Gamma}{N})(1-\Gamma)\geq 0$, then
it admits a unique solution. Thus (3.9) admits a unique solution. And then
(3.12) admits a unique solution.

From the above discussion, Proposition 3.4 and Theorem 4.1 on page 47 of Ma
and Yong [25], we have the following result.

######  Theorem 3.1.

Under Assumptions (A1)-(A3), for given $u_{0}(\cdot)\in\mathscr{U}_{0}[0,T]$,
if $1-\frac{\Gamma}{N}\geq 0$ and $(1-\frac{\Gamma}{N})(1-\Gamma)\geq 0$, then
Problem (P1) admits a unique solution

$\check{u}_{i}=-R^{-1}\left[B\big{(}P_{N}\check{x}_{i}+K_{N}\check{x}^{(N)}+\check{\phi}_{N}\big{)}+Lu_{0}\right],\quad
i=1,\cdots,N.$ (3.13)

###  3.2 Optimal strategy of the leader

Upon implementing the strategies of followers
$\check{u}_{i}(\cdot),i=1,\cdots,N$ according to (3.13), we delve into an
optimal control problem for the leader.

(P2): Minimize $\check{J}^{N}_{0}(u_{0}(\cdot))$ over
$u_{0}(\cdot)\in\mathscr{U}_{0}[0,T]$, where

$\check{J}^{N}_{0}(u_{0}(\cdot)):=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||x_{0}(t)-\Gamma_{0}\check{x}^{(N)}(t)-\eta_{0}||^{2}_{Q_{0}}+||u_{0}(t)||^{2}_{R_{0}}\right]dt+||x_{0}(0)||^{2}_{H_{0}}\right\\},$
(3.14)

subject to

$\left\\{\begin{aligned}
dx_{0}&=\big{[}A_{0}x_{0}+B_{0}u_{0}+C_{0}z_{0}+f_{0}\big{]}dt+z_{0}dW_{0},\quad
x_{0}(T)=\xi_{0},\\\
d\check{x}_{i}&=\big{[}(A-BR^{-1}BP_{N})\check{x}_{i}-BR^{-1}BK_{N}\check{x}^{(N)}-BR^{-1}B\check{\phi}_{N}+Fx_{0}\\\
&\quad+(G-BR^{-1}L)u_{0}+f\big{]}dt+DdW_{i},\quad\check{x}_{i}(0)=\xi_{i},\quad
i=1,\cdots,N,\\\
d\check{\phi}_{N}&=-\bigg{[}(A-\Pi_{N}BR^{-1}B)\check{\phi}_{N}+\Pi_{N}(G-BR^{-1}L)u_{0}+\Pi_{N}Fx_{0}\\\
&\quad+\Pi_{N}f-\left(1-\frac{\Gamma}{N}\right)Q(\Gamma_{1}x_{0}+\eta)\bigg{]}dt+\check{V}_{N}dW_{0},\quad\check{\phi}_{N}(T)=0.\end{aligned}\right.$
(3.15)

Note that here $\check{x}^{(N)}(\cdot)$ means that all
$x_{i}(\cdot),i=1,\cdots,N$ take the optimal $\check{x}_{i}(\cdot)$, i.e.,
$\check{x}^{(N)}(\cdot)=\frac{1}{N}\sum_{i=1}^{N}\check{x}_{i}(\cdot)$. Denote
$W^{(N)}(\cdot):=\frac{1}{N}\sum_{i=1}^{N}W_{i}(\cdot)$, and
$\xi^{(N)}:=\frac{1}{N}\sum_{i=1}^{N}\xi_{i}$.

The result of this subsection is as follows.

######  Theorem 3.2.

Under Assumptions (A1)-(A3), let the followers adopt the optimal strategy
(3.13). Then Problem (P2) admits an optimal control
$\check{u}_{0}(\cdot)\in\mathscr{U}_{0}[0,T]$, if and only if the following
two conditions hold:

(i) The adapted solution
$(\check{x}_{0}(\cdot),\check{z}_{0}(\cdot),\check{x}^{(N)}(\cdot),\check{\phi}_{N}(\cdot),\check{V}_{N}(\cdot),\check{y}_{0}(\cdot),\check{y}^{(N)}(\cdot),\check{\beta}^{(N)}(\cdot),\check{\psi}_{N}(\cdot))$
to the FBSDE

$\left\\{\begin{aligned} d\check{x}_{0}=&\
\big{[}A_{0}\check{x}_{0}+B_{0}\check{u}_{0}+C_{0}\check{z}_{0}+f_{0}\big{]}dt+\check{z}_{0}dW_{0},\quad\check{x}_{0}(T)=\xi_{0},\\\
d\check{x}^{(N)}=&\
\big{[}(A-BR^{-1}B\Pi_{N})\check{x}^{(N)}-BR^{-1}B\check{\phi}_{N}+F\check{x}_{0}+(G-BR^{-1}L)\check{u}_{0}+f\big{]}dt\\\
&+DdW^{(N)},\quad\check{x}^{(N)}(0)=\xi^{(N)},\\\ d\check{\phi}_{N}=&\
-\bigg{[}(A-\Pi_{N}BR^{-1}B)\check{\phi}_{N}+\Pi_{N}(G-BR^{-1}L)\check{u}_{0}+\Pi_{N}F\check{x}_{0}\\\
&+\Pi_{N}f-\left(1-\frac{\Gamma}{N}\right)Q(\Gamma_{1}\check{x}_{0}+\eta)\bigg{]}dt+\check{V}_{N}dW_{0},\quad\check{\phi}_{N}(T)=0,\\\
d\check{y}_{0}=&\
-\left[Q_{0}(\check{x}_{0}-\Gamma_{0}\check{x}^{(N)}-\eta_{0})+A_{0}\check{y}_{0}+F\check{y}^{(N)}-\left(\Pi_{N}F-\left(1-\frac{\Gamma}{N}\right)Q\Gamma_{1}\right)\check{\psi}_{N}\right]dt\\\
&\ -C_{0}\check{y}_{0}dW_{0},\quad\check{y}_{0}(0)=H_{0}\check{x}_{0}(0),\\\
d\check{y}^{(N)}=&\
-\big{[}(A-\Pi_{N}BR^{-1}B)\check{y}^{(N)}-Q_{0}\Gamma_{0}(\check{x}_{0}-\Gamma_{0}\check{x}^{(N)}-\eta_{0})\big{]}dt\\\
&\ +\check{\beta}^{(N)}dW^{(N)},\quad\check{y}^{(N)}(T)=0,\\\
d\check{\psi}_{N}=&\
\big{[}BR^{-1}B\check{y}^{(N)}+(A-\Pi_{N}BR^{-1}B)\check{\psi}_{N}\big{]}dt,\quad\check{\psi}_{N}(0)=0,\end{aligned}\right.$
(3.16)

satisfies the following stationarity condition:

$B_{0}\check{y}_{0}+R_{0}\check{u}_{0}+(G-BR^{-1}L)\check{y}^{(N)}-\Pi_{N}(G-BR^{-1}L)\check{\psi}_{N}=0,\quad
a.e.,\ a.s..$ (3.17)

(ii) The following convexity condition holds:

$\mathbb{E}\left\\{\int_{0}^{T}\left[Q_{0}(\tilde{x}_{0}-\Gamma_{0}\tilde{x}^{(N)})^{2}+R_{0}u_{0}^{2}\right]dt+H_{0}\tilde{x}_{0}^{2}(0)\right\\}\geq
0,\quad\forall u_{0}(\cdot)\in\mathscr{U}_{0}[0,T],$ (3.18)

where
$(\tilde{x}_{0}(\cdot),\tilde{z}_{0}(\cdot),\tilde{x}^{(N)}(\cdot),\tilde{\phi}_{N}(\cdot),\tilde{V}_{N}(\cdot))$
is the solution to the following FBSDE

$\left\\{\begin{aligned} d\tilde{x}_{0}=&\
\big{[}A_{0}\tilde{x}_{0}+B_{0}u_{0}+C_{0}\tilde{z}_{0}\big{]}dt+\tilde{z}_{0}dW_{0},\quad\tilde{x}_{0}(T)=0,\\\
d\tilde{x}^{(N)}=&\
\big{[}(A-BR^{-1}B\Pi_{N})\tilde{x}^{(N)}-BR^{-1}B\tilde{\phi}_{N}+F\tilde{x}_{0}+(G-BR^{-1}L)u_{0}\big{]}dt,\quad\tilde{x}^{(N)}(0)=0,\\\
d\tilde{\phi}_{N}=&-\left[(A-\Pi_{N}BR^{-1}B)\tilde{\phi}_{N}+\left(\Pi_{N}F-\left(1-\frac{\Gamma}{N}\right)Q\Gamma_{1}\right)\tilde{x}_{0}\right.\\\
&\quad+\Pi_{N}(G-BR^{-1}L)u_{0}\bigg{]}dt+\tilde{V}_{N}dW_{0},\quad\tilde{\phi}_{N}(T)=0.\end{aligned}\right.$
(3.19)

###### Proof.

For given $\xi_{0}\in L^{2}_{\mathcal{F}_{T}^{0}}(\Omega;\mathbb{R})$ and
$\check{u}_{0}(\cdot)\in\mathscr{U}_{0}[0,T]$, let ($\check{x}_{0}(\cdot)$,
$\check{z}_{0}(\cdot)$, $\check{x}^{(N)}(\cdot)$, $\check{\phi}_{N}(\cdot)$,
$\check{V}_{N}(\cdot)$, $\check{y}_{0}(\cdot)$, $\check{y}^{(N)}(\cdot)$,
$\check{\beta}^{(N)}(\cdot)$, $\check{\psi}_{N}(\cdot)$) be an adapted
solution to FBSDE (3.16). For any $u_{0}(\cdot)\in\mathscr{U}_{0}[0,T]$ and
$\varepsilon\in\mathbb{R}$, let ($x_{0}^{\varepsilon}(\cdot)$,
$z_{0}^{\varepsilon}(\cdot)$, $x^{\varepsilon(N)}(\cdot)$,
$\phi_{N}^{\varepsilon}(\cdot)$, $V_{N}^{\varepsilon}(\cdot)$) be the solution
to the following perturbed state equation of the leader:

$\left\\{\begin{aligned} dx_{0}^{\varepsilon}=&\
\big{[}A_{0}x_{0}^{\varepsilon}+B_{0}(\check{u}_{0}+\varepsilon
u_{0})+C_{0}x_{0}^{\varepsilon}+f_{0}\big{]}dt+z_{0}^{\varepsilon}dW_{0},\quad
x_{0}^{\varepsilon}(T)=\xi_{0},\\\ dx^{\varepsilon(N)}=&\
\big{[}(A-BR^{-1}B\Pi_{N})x^{\varepsilon(N)}-BR^{-1}B\phi^{\varepsilon}+Fx_{0}^{\varepsilon}+(G-BR^{-1}L)(\check{u}_{0}+\varepsilon
u_{0})\\\ &\quad+f\big{]}dt+DdW^{(N)},\quad
x^{\varepsilon(N)}(0)=\xi^{(N)},\\\
d\phi_{N}^{\varepsilon}=&-\bigg{[}(A-\Pi_{N}BR^{-1}B)\phi_{N}^{\varepsilon}+\Pi_{N}(G-BR^{-1}L)(\check{u}_{0}+\varepsilon
u_{0})+\Pi_{N}Fx_{0}^{\varepsilon}\\\
&+\Pi_{N}f-\left(1-\frac{\Gamma}{N}\right)Q(\Gamma_{1}x_{0}^{\varepsilon}+\eta)\bigg{]}dt+V_{N}^{\varepsilon}dW_{0},\quad\phi_{N}^{\varepsilon}(T)=0.\\\
\end{aligned}\right.$ (3.20)

Then, denoting by
$(\tilde{x}_{0}(\cdot),\tilde{z}_{0}(\cdot),\tilde{x}^{(N)}(\cdot),\tilde{\phi}_{N}(\cdot),\tilde{V}_{N}(\cdot))$
the solution to (3.19), we have
$x_{0}^{\varepsilon}(\cdot)=\check{x}_{0}(\cdot)+\varepsilon\tilde{x}_{0}(\cdot)$,
$z_{0}^{\varepsilon}(\cdot)=\check{z}_{0}(\cdot)+\varepsilon\tilde{z}_{0}(\cdot)$,
$x^{\varepsilon(N)}(\cdot)=\check{x}^{(N)}(\cdot)+\varepsilon\tilde{x}^{(N)}(\cdot)$,
$\phi_{N}^{\varepsilon}(\cdot)=\check{\phi}_{N}(\cdot)+\varepsilon\tilde{\phi}_{N}(\cdot)$,
$V_{N}^{\varepsilon}(\cdot)=\check{V}_{N}(\cdot)+\varepsilon\tilde{V}_{N}(\cdot)$,
and

$\begin{aligned} &\check{J}^{N}_{0}(\check{u}_{0}(\cdot)+\varepsilon
u_{0}(\cdot))-\check{J}^{N}_{0}(\check{u}_{0}(\cdot))\\\
&=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||x_{0}^{\varepsilon}-\Gamma_{0}\check{x}^{\varepsilon(N)}-\eta_{0}||^{2}_{Q_{0}}+||\check{u}_{0}+\varepsilon
u_{0}||^{2}_{R_{0}}\right]dt+||x_{0}^{\varepsilon}(0)||^{2}_{H_{0}}\right\\}\\\
&\quad-\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||\check{x}_{0}-\Gamma_{0}\check{x}^{(N)}-\eta_{0}||^{2}_{Q_{0}}+||\check{u}_{0}||^{2}_{R_{0}}\right]dt+||\check{x}_{0}(0)||^{2}_{H_{0}}\right\\}\\\
&=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||(\check{x}_{0}+\varepsilon\tilde{x}_{0})-\Gamma_{0}(\check{x}^{(N)}+\varepsilon\tilde{x}^{(N)})-\eta_{0}||^{2}_{Q_{0}}+||\check{u}_{0}+\varepsilon
u_{0}||^{2}_{R_{0}}\right]dt\right.\\\
&\qquad+||\check{x}_{0}(0)+\varepsilon\tilde{x}_{0}(0)||^{2}_{H_{0}}\bigg{\\}}-\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||\check{x}_{0}-\Gamma_{0}\check{x}^{(N)}-\eta_{0}||^{2}_{Q_{0}}+||\check{u}_{0}||^{2}_{R_{0}}\right]dt+||\check{x}_{0}(0)||^{2}_{H_{0}}\right\\}\\\
&=\frac{1}{2}\varepsilon^{2}\mathbb{E}\left\\{\int_{0}^{T}\left[Q_{0}(\tilde{x}_{0}-\Gamma_{0}\tilde{x}^{(N)})^{2}+R_{0}u_{0}^{2}\right]dt+H_{0}\tilde{x}_{0}^{2}(0)\right\\}\\\
&\quad+\varepsilon\mathbb{E}\left\\{\int_{0}^{T}\left[Q_{0}(\tilde{x}_{0}-\Gamma_{0}\tilde{x}^{(N)})(\check{x}_{0}-\Gamma_{0}\check{x}^{(N)}-\eta_{0})+R_{0}\check{u}_{0}u_{0}\right]dt+H_{0}\check{x}_{0}(0)\tilde{x}_{0}(0)\right\\}.\end{aligned}\
$

On the other hand, applying Itô’s formula to
$\tilde{x}_{0}(\cdot)\check{y}_{0}(\cdot)+\tilde{x}^{(N)}(\cdot)\check{y}^{(N)}(\cdot)+\tilde{\phi}_{N}(\cdot)\check{\psi}_{N}(\cdot)$,
integrating from 0 to $T$ and taking expectation, we obtain

$\displaystyle\
-\mathbb{E}\big{[}H_{0}\check{x}_{0}(0)\tilde{x}_{0}(0)\big{]}$
$\displaystyle=\mathbb{E}\int_{0}^{T}\bigg{[}Q_{0}(\tilde{x}_{0}-\Gamma_{0}\tilde{x}^{(N)})\left(\check{x}_{0}-\Gamma_{0}\check{x}^{(N)}-\eta_{0}\right)-B_{0}\check{y}_{0}u_{0}-(G-BR^{-1}L)\check{y}^{(N)}u_{0}$
$\displaystyle\qquad+\Pi_{N}(G-BR^{-1}L)\check{\psi}_{N}u_{0}\bigg{]}dt.$

Hence,

$\check{J}^{N}_{0}(\check{u}_{0}(\cdot))\leq\check{J}^{N}_{0}(\check{u}_{0}(\cdot)+\varepsilon
u_{0}(\cdot))$

if and only if (3.17) and (3.18) hold. The proof is complete. ∎

Based on Assumption (A3), we can compute out the optimal control of the leader
is

$\check{u}_{0}=-R_{0}^{-1}\big{[}B_{0}\check{y}_{0}+(G-BR^{-1}L)\check{y}^{(N)}-\Pi_{N}(G-BR^{-1}L)\check{\psi}_{N}\big{]},\quad
a.e.,\ a.s..$ (3.21)

So the related Hamiltonian system can be represented by

$\left\\{\begin{aligned}
d\check{x}_{0}&=\Big{\\{}A_{0}\check{x}_{0}-B_{0}R_{0}^{-1}\big{[}B_{0}\check{y}_{0}+(G-BR^{-1}L)\check{y}^{(N)}-\Pi_{N}(G-BR^{-1}L)\check{\psi}_{N}\big{]}\\\
&\quad+C_{0}\check{z}_{0}+f_{0}\Big{\\}}dt+\check{z}_{0}dW_{0},\quad\check{x}_{0}(T)=\xi_{0},\\\
d\check{x}^{(N)}&=\Big{\\{}(A-BR^{-1}B\Pi_{N})\check{x}^{(N)}-BR^{-1}B\check{\phi}_{N}+F\check{x}_{0}-(G-BR^{-1}L)R_{0}^{-1}\big{[}B_{0}\check{y}_{0}\\\
&\quad+(G-BR^{-1}L)\check{y}^{(N)}-\Pi_{N}(G-BR^{-1}L)\check{\psi}_{N}\big{]}+f\Big{\\}}dt+DdW^{(N)},\quad\check{x}^{(N)}(0)=\xi^{(N)},\\\
d\check{\phi}_{N}&=-\Big{\\{}(A-\Pi_{N}BR^{-1}B)\check{\phi}_{N}+\Pi_{N}(G-BR^{-1}L)R_{0}^{-1}\big{[}B_{0}\check{y}_{0}+(G-BR^{-1}L)\check{y}^{(N)}\\\
&\quad-\Pi_{N}(G-BR^{-1}L)\check{\psi}_{N}\big{]}+\Pi_{N}F\check{x}_{0}+\Pi_{N}f-(1-\frac{\Gamma}{N})Q(\Gamma_{1}\check{x}_{0}+\eta)\Big{\\}}dt\\\
&\quad+\check{V}_{N}dW_{0},\quad\check{\phi}_{N}(T)=0,\\\
d\check{y}_{0}&=-\bigg{[}Q_{0}\big{(}\check{x}_{0}-\Gamma_{0}\check{x}^{(N)}-\eta_{0}\big{)}+A_{0}\check{y}_{0}+F\check{y}^{(N)}-\left(\Pi_{N}F-\left(1-\frac{\Gamma}{N}\right)Q\Gamma_{1}\right)\check{\psi}_{N}\bigg{]}dt\\\
&\quad-
C_{0}\check{y}_{0}dW_{0},\quad\check{y}_{0}(0)=H_{0}\check{x}_{0}(0),\\\
d\check{y}^{(N)}&=-\big{[}(A-\Pi_{N}BR^{-1}B)\check{y}^{(N)}-Q_{0}\Gamma_{0}\big{(}\check{x}_{0}-\Gamma_{0}\check{x}^{(N)}-\eta_{0}\big{)}\big{]}dt\\\
&\quad+\check{\beta}^{(N)}dW^{(N)},\quad\check{y}^{(N)}(T)=0,\\\
d\check{\psi}_{N}&=[BR^{-1}B\check{y}^{(N)}+(A-\Pi_{N}BR^{-1}B)\check{\psi}_{N}]dt,\quad\check{\psi}_{N}(0)=0.\end{aligned}\right.$
(3.22)

##  4 Decentralized strategies and $(\epsilon_{1},\epsilon_{2})$-Stackelberg
equilibria

Let $N\to\infty$ in (3.8) and (3.9), then $P_{N}(\cdot)\to\bar{P}(\cdot)$,
$K_{N}(\cdot)\to\bar{K}(\cdot)$, where $\bar{P}(\cdot)$, $\bar{K}(\cdot)$
satisfy

$\dot{\bar{P}}+A\bar{P}+\bar{P}A-\bar{P}BR^{-1}B\bar{P}+Q=0,\quad\bar{P}(T)=H,$
(4.1)
$\dot{\bar{K}}+A\bar{K}+\bar{K}A-\bar{P}BR^{-1}B\bar{K}-\bar{K}BR^{-1}B(\bar{P}+\bar{K})-Q\Gamma=0,\quad\bar{K}(T)=0.$
(4.2)

Let $\bar{\Pi}(\cdot):=\bar{P}(\cdot)+\bar{K}(\cdot)$, then $\bar{\Pi}(\cdot)$
satisfies

$\dot{\bar{\Pi}}+A\bar{\Pi}+\bar{\Pi}A-\bar{\Pi}BR^{-1}B\bar{\Pi}+Q-Q\Gamma=0,\quad\bar{\Pi}(T)=H,$
(4.3)

From Assumption (A3), it follows that (4.1) and (4.2) admits a unique
solution, respectively.

Inspired by (3.22), we consider

$\left\\{\begin{aligned}
d\bar{x}_{0}&=\Big{\\{}A_{0}\bar{x}_{0}-B_{0}R_{0}^{-1}\big{[}B_{0}\bar{y}_{0}+(G-BR^{-1}L)\bar{y}-\bar{\Pi}(G-BR^{-1}L)\bar{\psi}\big{]}\\\
&\quad+C_{0}\bar{z}_{0}+f_{0}\Big{\\}}dt+\bar{z}_{0}dW_{0},\quad
x_{0}(T)=\xi_{0},\\\
d\bar{x}&=\Big{\\{}(A-BR^{-1}B\bar{\Pi})\bar{x}-BR^{-1}B\bar{\phi}+F\bar{x}_{0}-(G-BR^{-1}L)R_{0}^{-1}\big{[}B_{0}\bar{y}_{0}\\\
&\quad+(G-BR^{-1}L)\bar{y}-\bar{\Pi}(G-BR^{-1}L)\bar{\psi}\big{]}+f\Big{\\}}dt,\quad\bar{x}(0)=\bar{\xi},\\\
d\bar{\phi}&=-\Big{\\{}(A-\bar{\Pi}BR^{-1}B)\bar{\phi}+\bar{\Pi}(G-BR^{-1}L)R_{0}^{-1}\big{[}B_{0}\bar{y}_{0}+(G-BR^{-1}L)\bar{y}\\\
&\quad-\bar{\Pi}(G-BR^{-1}L)\bar{\psi}\big{]}+\bar{\Pi}F\bar{x}_{0}+\bar{\Pi}f-Q(\Gamma_{1}\bar{x}_{0}+\eta)\Big{\\}}dt+\bar{V}dW_{0},\quad\bar{\phi}(T)=0,\\\
d\bar{y}_{0}&=-\big{[}Q_{0}(\bar{x}_{0}-\Gamma_{0}\bar{x}-\eta_{0})+A_{0}\bar{y}_{0}+F\bar{y}-(\bar{\Pi}F-Q\Gamma_{1})\bar{\psi}\big{]}dt\\\
&\quad-C_{0}\bar{y}_{0}dW_{0},\quad\bar{y}_{0}(0)=H_{0}\bar{x}_{0}(0),\\\
d\bar{y}&=-\big{[}(A-\bar{\Pi}BR^{-1}B)\bar{y}-Q_{0}\Gamma_{0}(\bar{x}_{0}-\Gamma_{0}\bar{x}-\eta_{0})\big{]}dt,\quad\bar{y}(T)=0,\\\
d\bar{\psi}&=\big{[}BR^{-1}B\bar{y}+(A-\bar{\Pi}BR^{-1}B)\bar{\psi}\big{]}dt,\quad\bar{\psi}(0)=0.\end{aligned}\right.$
(4.4)

Denote $X:=[\bar{x}_{0},\bar{y},\bar{\phi}]^{\top}$,
$Y:=[\bar{y}_{0},\bar{x},\bar{\psi}]^{\top}$,
$Z:=[\bar{z}_{0},0,\bar{V}]^{\top}$,
$\mathcal{D}:=[-C_{0}\bar{y}_{0},0,0]^{\top}$ and

$\displaystyle\mathcal{A}_{1}$
$\displaystyle:=\begin{bmatrix}A_{0}&-B_{0}R_{0}^{-1}(G-BR^{-1}L)&0\\\
Q_{0}\Gamma_{0}&-A+BR^{-1}B\bar{\Pi}&0\\\
-\bar{\Pi}F+Q\Gamma_{1}&\bar{\Pi}R_{0}^{-1}(G-BR^{-1}L)^{2}&-A+BR^{-1}B\bar{\Pi}\\\
\end{bmatrix},$ $\displaystyle\mathcal{B}_{1}$
$\displaystyle:=\begin{bmatrix}B_{0}R_{0}^{-1}B_{0}&0&-B_{0}R_{0}^{-1}\bar{\Pi}(G-BR^{-1}L)\\\
0&Q_{0}\Gamma_{0}^{2}&0\\\
-B_{0}R_{0}^{-1}\bar{\Pi}(G-BR^{-1}L)&0&\bar{\Pi}^{2}R_{0}^{-1}(G-BR^{-1}L)^{2}\\\
\end{bmatrix},$ $\displaystyle\mathcal{A}_{2}$
$\displaystyle:=\begin{bmatrix}-Q_{0}&-F&0\\\
F&-R_{0}^{-1}(G-BR^{-1}L)^{2}&-BR^{-1}B\\\ 0&BR^{-1}B&0\\\ \end{bmatrix},$
$\displaystyle\mathcal{B}_{2}$
$\displaystyle:=\begin{bmatrix}A_{0}&-Q_{0}\Gamma_{0}&-\bar{\Pi}F+Q\Gamma_{1}\\\
B_{0}R_{0}^{-1}(G-BR^{-1}L)&-A+BR^{-1}B\bar{\Pi}&-\bar{\Pi}R_{0}^{-1}(G-BR^{-1}L)^{2}\\\
0&0&-A+BR^{-1}B\bar{\Pi}\\\ \end{bmatrix},$ $\displaystyle\mathcal{C}$
$\displaystyle:=\begin{bmatrix}C_{0}&0&0\\\ 0&0&0\\\ 0&0&0\\\
\end{bmatrix},\quad\mathfrak{f}_{0}:=\begin{bmatrix}f_{0}\\\
-Q_{0}\Gamma_{0}\eta_{0}\\\ Q\eta-\bar{\Pi}f\\\
\end{bmatrix},\quad\mathfrak{f}:=\begin{bmatrix}Q_{0}\eta_{0}\\\ f\\\ 0\\\
\end{bmatrix}.$

With the above notions, we can rewrite (4.4) as

$\left\\{\begin{aligned}
dX&=\big{[}\mathcal{A}_{1}X-\mathcal{B}_{1}Y+\mathcal{C}Z+\mathfrak{f}_{0}\big{]}dt+ZdW_{0},\quad
X(T)=[\xi_{0},0,0]^{\top},\\\
dY&=\big{[}\mathcal{A}_{2}X-\mathcal{B}_{2}Y+\mathfrak{f}\big{]}dt+\mathcal{D}dW_{0},\quad
Y(0)=[H_{0}\bar{x}_{0}(0),\bar{\xi},0]^{\top}.\end{aligned}\right.$ (4.5)

Suppose $(X(\cdot),Y(\cdot),Z(\cdot))$ is an adapted solution to (4.5). We
assume that $X(\cdot)$ and $Y(\cdot)$ are related by the following affine
transformation

$Y(\cdot)=\Phi(\cdot)X(\cdot)+\Psi(\cdot),$ (4.6)

where $\Phi(\cdot)$ is a differentiable function, with
$\Phi(0)=\begin{bmatrix}H_{0}&0&0\\\ 0&0&0\\\ 0&0&0\\\ \end{bmatrix},$ and
process pair $(\Psi(\cdot),\Sigma(\cdot))$ satisfies a BSDE as follows:

$d\Psi=\Delta dt+\Sigma dW_{0},\quad\Psi(0)=[0,\bar{\xi},0]^{\top},$

where $\Delta(\cdot)$ will be determined later. Next, by Itô’s formula, we
have

$dY=\dot{\Phi}Xdt+\Phi\big{[}\mathcal{A}_{1}X-\mathcal{B}_{1}(\Phi
X+\Psi)+\mathcal{C}Z+\mathfrak{f}_{0}\big{]}dt+\Phi ZdW_{0}+\Delta dt+\Sigma
dW_{0}.$ (4.7)

Now, compare (4.7) with the second equation in (4.5), we obtain that

$\Phi Z=\mathcal{D}-\Sigma,$ (4.8)
$\dot{\Phi}+\Phi\mathcal{A}_{1}-\Phi\mathcal{B}_{1}\Phi+\mathcal{B}_{2}\Phi-\mathcal{A}_{2}=0,\quad\Phi(0)=\begin{bmatrix}H_{0}&0&0\\\
0&0&0\\\ 0&0&0\\\ \end{bmatrix},$ (4.9)
$d\Psi=\big{[}\Phi\mathcal{B}_{1}\Psi-\Phi\mathcal{C}Z-\Phi\mathfrak{f}_{0}-\mathcal{B}_{2}\Psi+\mathfrak{f}\big{]}dt+\Sigma
dW_{0},\quad\Psi(0)=[0,\bar{\xi},0]^{\top}.$ (4.10)

If $\Phi$ is invertible, then $Z=\Phi^{-1}(\mathcal{D}-\Sigma)$, thus (4.10)
can be rewritten as

$d\Psi=\big{[}(\Phi\mathcal{B}_{1}-\mathcal{B}_{2})\Psi-\Phi\mathcal{C}\Phi^{-1}(\mathcal{D}-\Sigma)-\Phi\mathfrak{f}_{0}+\mathfrak{f}\big{]}dt+\Sigma
dW_{0},\quad\Psi(0)=[0,\bar{\xi},0]^{\top}.$ (4.11)

Note that the Riccati equation (4.9) is nonsymmetric. By Theorem 4.1 on page
47 of [25] again, if (4.9) admits a solution $\Phi(\cdot)$, then FBSDE (4.5)
admits a unique adapted solution $(X(\cdot),Y(\cdot),Z(\cdot))$.

We are going to identify conditions for the existence of a unique solution to
(4.9).

######  Proposition 4.1.

Let the pair $(\alpha(\cdot),\beta(\cdot))$ be the solution to the following
differential equation:

$\frac{d}{dt}\begin{pmatrix}\alpha(t)\\\ \beta(t)\\\
\end{pmatrix}=\begin{pmatrix}\mathcal{A}_{1}&-\mathcal{B}_{1}\\\
\mathcal{A}_{2}&-\mathcal{B}_{2}\\\ \end{pmatrix}\begin{pmatrix}\alpha(t)\\\
\beta(t)\\\ \end{pmatrix},\quad\begin{pmatrix}\alpha(0)\\\ \beta(0)\\\
\end{pmatrix}=\begin{pmatrix}\begin{bmatrix}H_{0}&0&0\\\ 0&0&0\\\ 0&0&0\\\
\end{bmatrix}\\\ I_{3}\\\ \end{pmatrix}.$

If $\beta(t)$ is invertible for all $t\in[0,T]$, then the Riccati equation
(4.9) has a unique solution $\Phi(t)=\alpha(t)\beta^{-1}(t)$ for all
$t\in[0,T]$.

###### Proof.

Note that

$\frac{d\beta^{-1}(t)}{dt}=-\beta^{-1}(t)\frac{d\beta(t)}{dt}\beta^{-1}(t)=-\beta^{-1}(t)\mathcal{A}_{2}\alpha(t)\beta^{-1}(t)+\beta^{-1}(t)\mathcal{B}_{2},$

which implies

$\displaystyle\frac{d\Phi(t)}{dt}$
$\displaystyle=\frac{d\alpha(t)}{dt}\beta^{-1}(t)+\alpha(t)\frac{d\beta^{-1}(t)}{dt}$
$\displaystyle=\mathcal{A}_{1}\alpha(t)\beta^{-1}(t)-\mathcal{B}_{1}-\alpha(t)\beta^{-1}(t)\mathcal{A}_{2}\alpha(t)\beta^{-1}(t)+\alpha(t)\beta^{-1}(t)\mathcal{B}_{2}$
$\displaystyle=\mathcal{A}_{1}\Phi(t)-\mathcal{B}_{1}-\Phi(t)\mathcal{A}_{2}\Phi(t)+\Phi(t)\mathcal{B}_{2},$

and the proof is achieved. ∎

Motivated by (3.13) and (3.21), we design the decentralized strategies below:

$\left\\{\begin{aligned}
\hat{u}_{0}&=-R_{0}^{-1}\big{[}B_{0}\bar{y}_{0}+(G-BR^{-1}L)\bar{y}-\bar{\Pi}(G-BR^{-1}L)\bar{\psi}\big{]},\\\
\hat{u}_{i}&=-R^{-1}\big{[}B(\bar{P}\hat{x}_{i}+\bar{K}\bar{x}+\bar{\phi})+Lu_{0}\big{]},\quad
i=1,\cdots,N,\end{aligned}\right.$ (4.12)

where
$\bar{y}_{0}(\cdot),\bar{y}(\cdot),\bar{\psi}(\cdot),\bar{x}(\cdot),\bar{\phi}(\cdot)$
are given by (4.4), and $\hat{x}_{i}(\cdot)$ satisfies

$\displaystyle d\hat{x}_{i}$
$\displaystyle=\big{[}(A-BR^{-1}B\bar{P})\hat{x}_{i}-BR^{-1}B(\bar{K}\bar{x}+\bar{\phi})+Fx_{0}+(G-BR^{-1}L)u_{0}+f\big{]}dt$
$\displaystyle\quad+DdW_{i},\quad\hat{x}_{i}(0)=\xi_{i},\quad i=1,\cdots,N.$
(4.13)

Next, we will show that the decentralized strategies (4.12) of the leader and
the followers, constitute an approximated
$(\epsilon_{1},\epsilon_{2})$-Stackelberg equilibrium.

For some technical reason, in the following we have to introduce the following
assumption:

(A4) $\Gamma=0.$

######  Theorem 4.1.

Assume that Assumptions (A1)-(A4) hold. Then
$(\hat{u}_{0}(\cdot),\hat{u}_{1}(\cdot),\cdots,\hat{u}_{N}(\cdot))$ given in
(4.12) constitutes an $(\epsilon_{1},\epsilon_{2})$-Stackelberg equilibrium,
where $\epsilon_{1}=\epsilon_{2}=O(\frac{1}{\sqrt{N}})$.

###### Proof.

The followers’ problem.

By (4) and the second equation of (4.4), it can be verified that

$\mathbb{E}\int_{0}^{T}||\hat{x}^{(N)}-\bar{x}||^{2}dt=O(\frac{1}{N}).$ (4.14)

For $i=1,\cdots,N$, denote
$\tilde{u}_{i}(\cdot):=u_{i}(\cdot)-\hat{u}_{i}(\cdot)$ and
$\tilde{x}_{i}(\cdot):=x_{i}(\cdot)-\hat{x}_{i}(\cdot)$. Then
$\tilde{x}_{i}(\cdot)$ satisfies

$d\tilde{x}_{i}=(A\tilde{x}_{i}+B\tilde{u}_{i})dt,\quad\tilde{x}_{i}(0)=0,\quad
i=1,\cdots,N.$

Thus,

$\sum_{i=1}^{N}\mathbb{E}\int_{0}^{T}\left(||\tilde{x}_{i}(t)||^{2}+||\tilde{u}_{i}(t)||^{2}\right)dt<\infty.$

From (2.3), we have

$J_{i}^{N}(u_{i}(\cdot),\hat{u}_{-i}(\cdot),u_{0}(\cdot))=J_{i}^{N}(\hat{u}_{i}(\cdot),\hat{u}_{-i}(\cdot),u_{0}(\cdot))+\tilde{J}_{i}^{N}(\tilde{u}_{i}(\cdot),\hat{u}_{-i}(\cdot),u_{0}(\cdot))+\mathcal{I}_{i}^{N},$
(4.15)

where

$\tilde{J}_{i}^{N}(\tilde{u}_{i}(\cdot),\hat{u}_{-i}(\cdot),u_{0}(\cdot)):=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}||^{2}_{Q}+||\tilde{u}_{i}||^{2}_{R}\right]dt+||\tilde{x}_{i}(T)||^{2}_{H}\right\\},$

and

$\mathcal{I}_{i}^{N}:=\mathbb{E}\left\\{\int_{0}^{T}\bigg{[}Q\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\left(\hat{x}_{i}-\Gamma\hat{x}^{(N)}-\Gamma_{1}x_{0}-\eta\right)+R\tilde{u}_{i}\hat{u}_{i}+\tilde{u}_{i}Lu_{0}\bigg{]}dt+H\tilde{x}_{i}(T)\hat{x}_{i}(T)\right\\}.$

Applying Itô’s formula to
$\tilde{x}_{i}(\cdot)\big{(}\bar{P}(\cdot)\hat{x}_{i}(\cdot)+\bar{K}(\cdot)\bar{x}(\cdot)+\bar{\phi}(\cdot)\big{)}$,
integrating from 0 to $T$, and taking expectation, we have

$\displaystyle\quad\mathbb{E}\big{[}H\tilde{x}_{i}(T)\hat{x}_{i}(T)\big{]}$
$\displaystyle=\mathbb{E}\big{[}\tilde{x}_{i}(T)\big{(}\bar{P}(T)\hat{x}_{i}(T)+\bar{K}(T)\bar{x}(T)+\bar{\phi}(T)\big{)}-\tilde{x}_{i}(0)\big{(}\bar{P}(0)\hat{x}_{i}(0)+\bar{K}(0)\bar{x}(0)+\bar{\phi}(0)\big{)}\big{]}$
$\displaystyle=\mathbb{E}\int_{0}^{T}\bigg{\\{}(A\tilde{x}_{i}+B\tilde{u}_{i})(\bar{P}\hat{x}_{i}+\bar{K}\bar{x}+\bar{\phi})+\tilde{x}_{i}\Big{[}\dot{\bar{P}}\hat{x}_{i}+\bar{P}[(A-BR^{-1}B\bar{P})\hat{x}_{i}$
$\displaystyle\qquad-
BR^{-1}B(\bar{K}\bar{x}+\bar{\phi})+Fx_{0}+(G-BR^{-1}L)u_{0}+f\Big{]}+\dot{\bar{K}}\hat{x}$
$\displaystyle\qquad+\bar{K}\Big{[}(A-BR^{-1}B\bar{\Pi})\bar{x}-BR^{-1}B\bar{\phi}+F\bar{x}_{0}+(G-BR^{-1}L)u_{0}+f\Big{]}$
$\displaystyle\qquad-\Big{[}(A-\bar{\Pi}BR^{-1}B)\bar{\phi}-(G-BR^{-1}L)u_{0}+\bar{\Pi}F\bar{x}_{0}+\bar{\Pi}f-Q(\Gamma_{1}\bar{x}_{0}+\eta)\Big{]}\bigg{\\}}dt$
$\displaystyle=\mathbb{E}\int_{0}^{T}\bigg{\\{}\tilde{x}_{i}\hat{x}_{i}\big{[}A\bar{P}+\dot{\bar{P}}+\bar{P}(A-BR^{-1}B\bar{P})\big{]}+\tilde{x}_{i}\bar{x}\big{[}A\bar{K}-\bar{P}BR^{-1}B\bar{K}$
$\displaystyle\qquad+\dot{\bar{K}}+\bar{K}(A-BR^{-1}B\bar{\Pi})\big{]}-\tilde{x}_{i}Q(\Gamma_{1}\bar{x}_{0}+\eta)+B\tilde{u}_{i}(\bar{P}\hat{x}_{i}+\bar{K}\bar{x}+\bar{\phi})\bigg{\\}}dt$
$\displaystyle=\mathbb{E}\int_{0}^{T}\Big{[}-\tilde{x}_{i}Q\big{(}\hat{x}_{i}-\Gamma\bar{x}-\Gamma_{1}\bar{x}_{0}-\eta\big{)}-\tilde{u}_{i}R\hat{u}_{i}-\tilde{u}_{i}Lu_{0}\Big{]}dt.$

Then,

$\displaystyle\mathcal{I}_{i}^{N}=\mathbb{E}\left\\{\int_{0}^{T}\left[Q\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\left(\hat{x}_{i}-\Gamma\hat{x}^{(N)}-\Gamma_{1}x_{0}-\eta\right)+R\tilde{u}_{i}\hat{u}_{i}+\tilde{u}_{i}Lu_{0}\right]dt+H\tilde{x}_{i}(T)\hat{x}_{i}(T)\right\\}$
$\displaystyle=\mathbb{E}\bigg{\\{}\int_{0}^{T}\bigg{[}Q\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\big{(}\hat{x}_{i}-\Gamma\bar{x}-\Gamma_{1}x_{0}-\eta\big{)}+Q\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\Gamma(\bar{x}-\hat{x}^{(N)})$
$\displaystyle\qquad+R\tilde{u}_{i}\hat{u}_{i}+\tilde{u}_{i}Lu_{0}\bigg{]}dt+H\tilde{x}_{i}(T)\hat{x}_{i}(T)\bigg{\\}}$
$\displaystyle=-\left(1-\frac{\Gamma}{N}\right)\mathbb{E}\left\\{\int_{0}^{T}\left[\tilde{u}_{i}R\hat{u}_{i}+\tilde{u}_{i}Lu_{0}\right]dt+H\tilde{x}_{i}(T)\hat{x}_{i}(T)\right\\}$
$\displaystyle\quad+\mathbb{E}\left\\{\int_{0}^{T}\left[Q\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\Gamma(\bar{x}-\hat{x}^{(N)})+R\tilde{u}_{i}\hat{u}_{i}+\tilde{u}_{i}Lu_{0}\right]dt+H\tilde{x}_{i}(T)\hat{x}_{i}(T)\right\\}$
$\displaystyle=\mathbb{E}\left\\{\int_{0}^{T}\left[\frac{\Gamma}{N}\tilde{u}_{i}R\hat{u}_{i}+\frac{\Gamma}{N}\tilde{u}_{i}Lu_{0}+Q\left(1-\frac{\Gamma}{N}\right)\tilde{x}_{i}\Gamma(\bar{x}-\hat{x}^{(N)})\right]dt+\frac{\Gamma}{N}H\tilde{x}_{i}(T)\hat{x}_{i}(T)\right\\}=O(\frac{1}{\sqrt{N}}).$

Thereby,

$J_{i}^{N}(\hat{u}_{i}(\cdot),\hat{u}_{-i}(\cdot),u_{0}(\cdot))\leq
J_{i}^{N}(u_{i}(\cdot),\hat{u}_{-i}(\cdot),u_{0}(\cdot))+\epsilon_{1}.$

Thus, $(\hat{u}_{1}(\cdot),\cdots,\hat{u}_{N}(\cdot))$ is an
$\epsilon_{1}$-Nash equilibrium, where $\epsilon_{1}=O(\frac{1}{\sqrt{N}})$.

Problem of the leader.

Let $\check{X}:=[\check{x}_{0},\check{y}^{(N)},\check{\phi}]^{\top}$,
$\check{Y}:=[\check{y}_{0},\check{x}^{(N)},\check{\psi}]^{\top}$,
$\check{Z}:=[\check{z}_{0},0,\check{V}_{N}]^{\top}$,
$\check{\mathcal{D}}:=[-C_{0}\check{y}_{0},0,0]^{\top}$,
$\check{Z}^{(N)}:=[0,\check{\beta}^{(N)},0]^{\top}$,
$\check{D}^{(N)}:=[0,D,0]^{\top}$ and

$\displaystyle\check{\mathcal{A}}_{1}$
$\displaystyle:=\begin{bmatrix}A_{0}&-B_{0}R_{0}^{-1}(G-BR^{-1}L)&0\\\
Q_{0}\Gamma_{0}&-A+BR^{-1}B\Pi_{N}&0\\\
-\Pi_{N}F+(1-\frac{\Gamma}{N})Q\Gamma_{1}&\Pi_{N}R_{0}^{-1}(G-BR^{-1}L)^{2}&-A+BR^{-1}B\Pi_{N}\\\
\end{bmatrix},$ $\displaystyle\check{\mathcal{B}}_{1}$
$\displaystyle:=\begin{bmatrix}B_{0}R_{0}^{-1}B_{0}&0&-B_{0}R_{0}^{-1}\Pi_{N}(G-BR^{-1}L)\\\
0&Q_{0}\Gamma_{0}^{2}&0\\\
-B_{0}R_{0}^{-1}\Pi_{N}(G-BR^{-1}L)&0&\Pi_{N}^{2}R_{0}^{-1}(G-BR^{-1}L)^{2}\\\
\end{bmatrix},$ $\displaystyle\check{\mathcal{A}}_{2}$
$\displaystyle:=\begin{bmatrix}-Q_{0}&-F&0\\\
F&-R_{0}^{-1}(G-BR^{-1}L)^{2}&-BR^{-1}B\\\ 0&BR^{-1}B&0\\\ \end{bmatrix},$
$\displaystyle\check{\mathcal{B}}_{2}$
$\displaystyle:=\begin{bmatrix}A_{0}&-Q_{0}\Gamma_{0}&-\Pi_{N}F+(1-\frac{\Gamma}{N})Q\Gamma_{1}\\\
B_{0}R_{0}^{-1}(G-BR^{-1}L)&-A+BR^{-1}B\Pi_{N}&-\Pi_{N}R_{0}^{-1}(G-BR^{-1}L)^{2}\\\
0&0&-A+BR^{-1}B\Pi_{N}\\\ \end{bmatrix},$ $\displaystyle\check{\mathcal{C}}$
$\displaystyle:=\begin{bmatrix}C_{0}&0&0\\\ 0&0&0\\\ 0&0&0\\\
\end{bmatrix},\quad\check{\mathfrak{f}}_{0}:=\begin{bmatrix}f_{0}\\\
-Q_{0}\Gamma_{0}\eta_{0}\\\ (1-\frac{\Gamma}{N})Q\eta-\Pi_{N}f\\\
\end{bmatrix},\quad\check{\mathfrak{f}}:=\begin{bmatrix}Q_{0}\eta_{0}\\\ f\\\
0\\\ \end{bmatrix}.$

With the above notions, we can rewrite (3.22) as

$\left\\{\begin{aligned}
d\check{X}&=\big{[}\check{\mathcal{A}}_{1}\check{X}-\check{\mathcal{B}}_{1}\check{Y}+\check{\mathcal{C}}\check{Z}+\check{\mathfrak{f}}_{0}\big{]}dt+\check{Z}dW_{0}+\check{Z}^{(N)}dW^{(N)},\quad\check{X}(T)=[\xi_{0},0,0]^{\top},\\\
d\check{Y}&=\big{[}\check{\mathcal{A}}_{2}\check{X}-\check{\mathcal{B}}_{2}\check{Y}+\check{\mathfrak{f}}\big{]}dt+\check{\mathcal{D}}dW_{0}+\check{\mathcal{D}}^{(N)}dW^{(N)},\quad\check{Y}(0)=[H_{0}\check{x}_{0}(0),\xi^{(N)},0]^{\top}.\end{aligned}\right.$
(4.16)

When Assumption (A4) holds, we have $\Pi_{N}=\bar{\Pi}$. Thus
$\check{\mathcal{A}}_{1}=\mathcal{A}_{1}$,
$\check{\mathcal{B}}_{1}=\mathcal{B}_{1}$, $\check{\mathcal{C}}=\mathcal{C}$,
$\check{\mathfrak{f}}_{0}=\mathfrak{f}_{0}$,
$\check{\mathcal{A}}_{2}=\mathcal{A}_{2}$,
$\check{\mathcal{B}}_{2}=\mathcal{B}_{2}$,
$\check{\mathfrak{f}}=\mathfrak{f}$.

Denote $\tilde{X}:=\check{X}-X$, $\tilde{Y}:=\check{Y}-Y$,
$\tilde{Z}:=\check{Z}-Z$,
$\tilde{\mathcal{D}}:=\check{\mathcal{D}}-\mathcal{D}$, we have

$\left\\{\begin{aligned}
d\tilde{X}&=\big{[}\mathcal{A}_{1}\tilde{X}-\mathcal{B}_{1}\tilde{Y}+\mathcal{C}\tilde{Z}\big{]}dt+\tilde{Z}dW_{0}+\check{Z}^{(N)}dW^{(N)},\quad\tilde{X}(T)=[\xi_{0},0,0]^{\top},\\\
d\tilde{Y}&=\big{[}\mathcal{A}_{2}\tilde{X}-\mathcal{B}_{2}\tilde{Y}\big{]}dt+\tilde{\mathcal{D}}dW_{0}+\check{\mathcal{D}}^{(N)}dW^{(N)},\quad\tilde{Y}(0)=[H_{0}(\check{x}_{0}(0)-\bar{x}_{0}(0)),\xi^{(N)}-\bar{\xi},0]^{\top}.\end{aligned}\right.$
(4.17)

We assume that $\tilde{X}(\cdot)=\Theta(\cdot)\tilde{Y}(\cdot)$, where
$\Theta(\cdot)$ is a differential function with $\Theta(T)=0$. By Itô’s
formula, we have

$\displaystyle d\tilde{X}$
$\displaystyle=\dot{\Theta}\tilde{Y}dt+\Theta\big{[}(\mathcal{A}_{2}\Theta\tilde{Y}-\mathcal{B}_{2}\tilde{Y})dt+\tilde{\mathcal{D}}dW_{0}+\check{\mathcal{D}}^{(N)}dW^{(N)}\big{]}$
$\displaystyle=\big{[}\mathcal{A}_{1}\Theta\tilde{Y}-\mathcal{B}_{1}\tilde{Y}+\mathcal{C}\tilde{Z}\big{]}dt+\tilde{Z}dW_{0}+\check{Z}^{(N)}dW^{(N)}.$

Comparing the corresponding coefficients, we have

$\dot{\Theta}+\Theta\mathcal{A}_{2}\Theta-\Theta\mathcal{B}_{2}-\mathcal{A}_{1}\Theta+\mathcal{B}_{1}=0,\quad\Theta(T)=0.$
(4.18)

Then we achieve

$d\tilde{Y}=(\mathcal{A}_{2}\Theta-\mathcal{B}_{2})\tilde{Y}dt+\tilde{\mathcal{D}}dW_{0}+\check{\mathcal{D}}^{(N)}dW^{(N)},$

with
$\tilde{Y}(0)=[H_{0}(\check{x}_{0}(0)-\bar{x}_{0}(0)),\xi^{(N)}-\bar{\xi},0]^{\top}$.
This implies

$\tilde{Y}(t)=e^{(\mathcal{A}_{2}\Theta-\mathcal{B}_{2})t}\tilde{Y}(0)+\int_{0}^{t}e^{(\mathcal{A}_{2}\Theta-\mathcal{B}_{2})(t-s)}\big{[}\tilde{\mathcal{D}}dW_{0}(s)+\check{\mathcal{D}}^{(N)}dW^{(N)}(s)\big{]},$

which gives

$\sup_{0\leq t\leq T}\mathbb{E}|\tilde{Y}(t)|^{2}=O(\frac{1}{N}),$ (4.19)

and since $\tilde{X}(\cdot)=\Theta(\cdot)\tilde{Y}(\cdot)$, then

$\sup_{0\leq t\leq T}\mathbb{E}|\tilde{X}(t)|^{2}=O(\frac{1}{N}),$ (4.20)

which means

$\mathbb{E}\int_{0}^{T}||\check{x}_{0}-\bar{x}_{0}||^{2}dt=O(\frac{1}{N}),\quad\mathbb{E}\int_{0}^{T}||\check{x}^{(N)}-\bar{x}||^{2}dt=O(\frac{1}{N}),$
(4.21)

and

$\displaystyle\mathbb{E}\int_{0}^{T}||\check{y}_{0}-\bar{y}_{0}||^{2}dt=O(\frac{1}{N}),\quad\mathbb{E}\int_{0}^{T}||\check{y}^{(N)}-\bar{y}||^{2}dt=O(\frac{1}{N}),\quad\mathbb{E}\int_{0}^{T}||\hat{\psi}-\bar{\psi}||^{2}dt=O(\frac{1}{N}).$
(4.22)

Combined with (4.14), we have

$\mathbb{E}\int_{0}^{T}||\hat{x}^{(N)}-\check{x}^{(N)}||^{2}dt=O(\frac{1}{N}),$
(4.23)

and $\bar{x}_{0}(t)=\hat{x}_{0}(t)$, for all $t\in[0,T]$. Then

$\mathbb{E}\int_{0}^{T}||\hat{x}_{0}-\check{x}_{0}||^{2}dt=O(\frac{1}{N}).$
(4.24)

We need two steps. First, by (4.23), we get

$\displaystyle
J_{0}^{N}(\check{u}_{0}(\cdot),\check{u}^{N}[\cdot;\check{u}_{0}(\cdot)])\leq
J_{0}^{N}(u_{0}(\cdot),\check{u}^{N}[\cdot;u_{0}(\cdot)])$ (4.25)
$\displaystyle=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||x_{0}-\Gamma_{0}\check{x}^{(N)}-\eta_{0}||^{2}_{Q_{0}}+||u_{0}||^{2}_{R_{0}}\right]dt+||x_{0}(0)||^{2}_{H_{0}}\right\\}$
$\displaystyle=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||x_{0}-\Gamma_{0}\hat{x}^{(N)}-\eta_{0}+\Gamma\left(\hat{x}^{(N)}-\check{x}^{(N)}\right)||^{2}_{Q_{0}}+||u_{0}||^{2}_{R_{0}}\right]dt+||x_{0}(0)||^{2}_{H_{0}}\right\\}$
$\displaystyle\leq
J_{0}^{N}(u_{0}(\cdot),\hat{u}^{N}[\cdot;u_{0}(\cdot)])+O(\frac{1}{N})+O(\frac{1}{\sqrt{N}}).$

Next, from (4.22)-(4.24), we have

$\displaystyle
J_{0}^{N}(\hat{u}_{0}(\cdot),\hat{u}^{N}[\cdot;\hat{u}_{0}(\cdot)])$ (4.26)
$\displaystyle=\frac{1}{2}\mathbb{E}\left\\{\int_{0}^{T}\left[||\hat{x}_{0}-\Gamma_{0}\hat{x}^{(N)}-\eta_{0}||^{2}_{Q_{0}}+||\hat{u}_{0}||^{2}_{R_{0}}\right]dt+||\hat{x}_{0}(0)||^{2}_{H_{0}}\right\\}$
$\displaystyle=\frac{1}{2}\mathbb{E}\bigg{\\{}\int_{0}^{T}\Big{[}||\hat{x}_{0}-\Gamma_{0}\hat{x}^{(N)}-\eta_{0}||^{2}_{Q_{0}}+||R_{0}^{-1}\big{[}B_{0}\bar{y}_{0}+(G-BR^{-1}L)\bar{y}$
$\displaystyle\qquad\quad-\bar{\Pi}(G-BR^{-1}L)\bar{\psi}\big{]}||^{2}_{R_{0}}\Big{]}dt+||\hat{x}_{0}(0)||^{2}_{H_{0}}\bigg{\\}}$
$\displaystyle=\frac{1}{2}\mathbb{E}\bigg{\\{}\int_{0}^{T}\Big{[}||\check{x}_{0}-\Gamma_{0}\check{x}^{(N)}-\eta_{0}+(\hat{x}_{0}-\check{x}_{0})+\Gamma_{0}\left(\check{x}^{(N)}-\hat{x}^{(N)}\right)||^{2}_{Q_{0}}$
$\displaystyle\qquad\quad+||R_{0}^{-1}\big{[}B_{0}(\check{y}_{0}+(\bar{y}_{0}-\check{y}_{0}))+(G-BR^{-1}L)\left(\check{y}^{(N)}+(\bar{y}-\check{y}^{(N)})\right)$
$\displaystyle\qquad\quad-\bar{\Pi}(G-BR^{-1}L)(\check{\psi}+(\bar{\psi}-\check{\psi}))\big{]}||^{2}_{R_{0}}\Big{]}dt+||\check{x}_{0}(0)+(\hat{x}_{0}(0)-\check{x}_{0}(0))||^{2}_{H_{0}}\bigg{\\}}$
$\displaystyle\leq
J_{0}^{N}(\check{u}_{0}(\cdot),\check{u}^{N}[\cdot;\check{u}_{0}(\cdot)])+O(\frac{1}{N})+O(\frac{1}{\sqrt{N}}).$

Hence, from (4.25) and (4.26), we have

$J_{0}^{N}(\hat{u}_{0}(\cdot),\hat{u}^{N}[\cdot;\hat{u}_{0}(\cdot)])\leq
J_{0}^{N}(u_{0}(\cdot),\hat{u}^{N}[\cdot;u_{0}(\cdot)])+\epsilon_{2}.$

Thus, by Definition 2.1,
$(\hat{u}_{0}(\cdot),\hat{u}_{1}(\cdot),\cdots,\hat{u}_{N}(\cdot))$ is an
$(\epsilon_{1},\epsilon_{2})$-Stackelberg equilibrium, where
$\epsilon_{1}=\epsilon_{2}=O(\frac{1}{\sqrt{N}})$. The proof is complete. ∎

######  Remark 4.1.

We introduced Assumption (A4) in the part of the proof of the leader as a last
resort, because without it, the proof would have a case where, for example,
$\Pi_{N}\hat{\psi}_{N}-\bar{\Pi}\bar{\psi}$, which would entail great
complexity and difficulty, a situation that we cannot deal with for the time
being. We leave it as an open question for discussion in the future.

##  5 Numerical simulation

This section provides numerical examples for Problem (P0) to verify the
result. We set
$A=A_{0}=B=B_{0}=F=G=\Gamma_{0}=\Gamma_{1}=Q=Q_{0}=R=R_{0}=H=H_{0}=L=1$,
$\Gamma=2$, $T=10$, it is easy to check that $\bar{\Pi}(\cdot)\equiv 1$. The
solution to the Riccati equation (4.9), $\Phi(\cdot)\in\mathbb{R}^{3\times
3}$, is depicted in Figure 1. Note that $\Phi(\cdot)$ is neither symmetric nor
positive semi-definite.

Figure 1: The curve of $\Phi(\cdot)$

##  6 Conclusions

In this paper, we have explored a backward-forward mean field LQ Stackelberg
stochastic differential game featuring a backward leader and numerous forward
followers. Initially, we address a mean field game problem concerning $N$
followers. Through the decoupling of the high-dimensional Hamiltonian system
using mean field approximations, we formulate a set of open-loop decentralized
strategies for all players. Subsequently, these strategies are demonstrated to
constitute an $(\epsilon_{1},\epsilon_{2})$-Stackelberg equilibrium. In
proving $(\epsilon_{1},\epsilon_{2})$-Stackelberg equilibrium, we introduced a
hypothesis (Assumption (A4)) to mitigate potential challenges. While this
expedited problem resolution, the assumption appeared somewhat abrupt. We aim
to address this issue in subsequent research endeavors.

Other topics including the case when the diffusion coefficients are dependent
of state and control variables ([39], [32]), and with common noise ([5]), are
our future research interests.

## References

  * [1] A. Bagchi, T. Başar, Stackelberg strategies in linear-quadratic stochastic differential games. J. Optim. Theory Appl., 1981, 35(3): 443-464.
  * [2] A. Bensoussan, S.K. Chen, and S.P. Sethi, The maximum principle for global solutions of stochastic Stackelberg differential games. SIAM J. Control Optim., 2015, 53(4): 1956-1981.
  * [3] A. Bensoussan, M.H.M. Chau, Y. Lai, and S.C.P. Yam, Linear-quadratic mean field Stackelberg games with state and control delays. SIAM J. Control Optim., 2017, 55(4): 2748-2781.
  * [4] A. Bensoussan, M.H.M. Chau, and S.C.P. Yam, Mean field Stackelberg games: aggregation of delayed instructions. SIAM J. Control Optim., 2015, 53(4): 2237-2266.
  * [5] A. Bensoussan, X.W. Feng, and J.H. Huang, Linear-quadratic-Gaussian mean-field-game with partial observation and common noise. Math. Control Relat. Fields, 2021, 11(1): 23-46.
  * [6] A. Bensoussan, J. Frehse, and P. Yam, Mean Field Games and Mean Field Type Control Theory. Springer, New York, 2013.
  * [7] A. Bensoussan, K.C.J. Sung, S.C.P. Yam, and S.P. Yung, Linear-quadratic mean field games. J. Optim. Theory Appl., 2016, 169(2): 496-529.
  * [8] P.E. Caines, M.Y. Huang, and R.P. Malhamé, Mean field games. In T. Başar, G. Zaccour, Handbook of Dynamic Game Theory, Springer, Berlin, 2017.
  * [9] R. Carmona, F. Delarue, Probabilistic Theory of Mean Field Games with Applications I, II. Springer, Cham, Switzerland, 2018.
  * [10] G. Dayanıklı, M. Laurière, A machine learning method for Stackelberg mean field games. arXiv:2302.10440
  * [11] K. Du, Z. Wu, Linear-quadratic Stackelberg game for mean-field backward stochastic differential system and application. Math. Prob. Engin., 2019, 1798585, 17 pages.
  * [12] X.W. Feng, Y. Hu, and J.H. Huang, Backward Stackelberg differential game with constraints: a mixed terminal-perturbation and linear-quadratic approach. SIAM J. Control Optim., 2022, 66(3): 1488-1518.
  * [13] D.A. Gomes, J. Saúde, Mean field games models-a brief survey. Dyn. Games Appl., 2014, 4(2): 110-154.
  * [14] J.H. Huang, N. Li, Linear-quadratic mean-field game for stochastic delayed systems. IEEE Trans. Automat. Control, 2018, 63(8): 2711-2729.
  * [15] J.H. Huang, S.J. Wang, and Z. Wu, Backward-forward linear-quadratic mean-field games with major and minor agents. Probab. Uncer. Quant. Risk, 2016, 1: 8.
  * [16] J.H. Huang, S.J. Wang, and Z. Wu, Backward mean-field linear-quadratic-Gaussian (LQG) games: full and partial information. IEEE Trans. Automat. Control, 2016, 61(12): 3784-3796.
  * [17] M.Y. Huang, Large-population LQG games involving a major player: the Nash certainty equivalence principle, SIAM J. Control Optim., 2010, 48(5): 3318-3353.
  * [18] M.Y. Huang, P.E. Caines, and R.P. Malhamé, Large-population cost-coupled LQG problems with nonuniform agents: individual-mass behavior and decentralized $\epsilon$-Nash equilibria. IEEE Trans. Automat. Control, 2007, 52(9): 1560-1571.
  * [19] M.Y. Huang, R.P. Malhamé, and P.E. Caines, Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle. Commun. Inf. Syst., 2006, 6(3): 221-251.
  * [20] M.Y. Huang, M.J. Zhou, Linear quadratic mean field games: asymptotic solvability and relation to the fixed point approach. IEEE Trans. Automat. Control, 2020, 65(4): 1397-1412.
  * [21] J.M. Lasry, P.L. Lions, Mean field games. Jpn. J. Math., 2007, 2(1): 229-260.
  * [22] T. Li, J.F. Zhang, Asymptotically optimal decentralized control for large population stochastic multiagent systems. IEEE Trans. Automat. Control, 2008, 53(7): 1643-1660.
  * [23] Y.N. Lin, X.S. Jiang, and W.H. Zhang, An open-loop Stackelberg strategy for the linear quadratic mean-field stochastic differential game. IEEE Trans. Automat. Control, 2019, 64(1): 97-110.
  * [24] Y.N. Lin, W.H. Zhang, Feedback Stackelberg solution for mean-field type stochastic systems with multiple followers. J. Syst. Sci. Complex., 2023, 36(4): 1519-1539.
  * [25] J. Ma, J.M. Yong, Forward-backward Stochastic Differential Equations and Their Applications. Lecture Notes Math., Springer, New York, 1999.
  * [26] J. Moon, T. Başar, Linear quadratic risk-sensitive and robust mean field games. IEEE Trans. Automat. Control, 2017, 62(3): 1062-1077.
  * [27] J. Moon, T. Başar, Linear quadratic mean field Stackelberg differential games. Automatica, 2018, 97: 200-213.
  * [28] J. Moon, H.J. Yang, Linear-quadratic time-inconsistent mean-field type Stackelberg differential games: time-consistent open-loop solutions. IEEE Trans. Automat. Control, 2021, 66(1): 375-382.
  * [29] H. Mukaidani, S. Irie, H, Xu, and W.H. Zhuang, Robust incentive Stackelberg games with a large population for stochastic mean-field systems. IEEE Control Syst. Lett., 2022, 6: 1934-1939.
  * [30] M. Nourian, P.E. Caines, R.P. Malhamé, and M.Y. Huang, Mean field LQG control in leader-follower stochastic multi-agent systems: likelihood ratio based adaptation. IEEE Trans. Automat. Control, 2012, 57(11): 2801-2816.
  * [31] E. Pardoux, S.G. Peng, Adapted solution of a backward stochastic differential equation. Syst. $\&$ Control Lett., 1990, 14(1): 55-61.
  * [32] K.H. Si, Z. Wu, Backward-forward linear-quadratic mean-field Stackelberg games. Adv. Difference Equ., 2021, 73: 23.
  * [33] J.R. Sun, H.X. Wang, and J.Q. Wen, Zero-sum Stackelberg stochastic linear-quadratic differential games. SIAM J. Control Optim., 2023, 61(1): 250-282.
  * [34] H. von Stackelberg, The Theory of the Market Economy. Oxford University Press, London, 1952.
  * [35] B.C. Wang, Leader-follower mean field LQ games: a direct method. Asian J Control, DOI:10.1002/asjc.3007
  * [36] B.C. Wang, H.S. Zhang, J.F. Zhang, Mean field linear-quadratic control: uniform stabilization and social optimality. Automatica, 2020, 121: 109088.
  * [37] B.C. Wang, J.F. Zhang, Hierarchical mean field games for multiagent systems with tracking-type costs: distributed $\epsilon$-Stackelberg equilibria. IEEE Trans. Automat. Control, 2014, 59(8): 2241-2247.
  * [38] G.C. Wang, S.S. Zhang, A mean-field linear-quadratic stochastic Stackelberg differential game with one leader and two followers. J. Syst. Sci. Complex., 2020, 33(5): 1383-1401.
  * [39] R.M. Xu, F. Zhang, $\epsilon$-Nash mean-field games for general linear-quadratic systems with applications. Automatica, 2020, 114: 108835.
  * [40] X.W. Yang, M.Y. Huang, Linear quadratic mean field Stackelberg games: master equations and time consistent feedback strategies. Proc. 60th IEEE Conf. Decision Control, 171-176, December 13-15, Austin, Texas, 2021.
  * [41] J.M. Yong, A leader-follower stochastic linear quadratic differential games. SIAM J. Control Optim., 2002, 41(4): 1015-1041.
  * [42] Y.Y. Zheng, J.T. Shi, A Stackelberg game of backward stochastic differential equations with applications. Dyna. Games Appl., 2020, 10(4): 968-992.