#  Global maximum principle for mean-field forward-backward stochastic systems
with delay and application to finance

Tao HAO School of Statistics, Shandong University of Finance and Economics,
Jinan 250014, P. R. China<EMAIL_ADDRESS>Research supported by
National Natural Science Foundation of China (Grant Nos.
71671104,11871309,11801315,71803097), the Ministry of Education of Humanities
and Social Science Project (Grant No. 16YJA910003), Natural Science Foundation
of Shandong Province (No. ZR2018QA001), A Project of Shandong Province Higher
Educational Science and Technology Program (Grant Nos. J17KA162, J17KA163),
and Incubation Group Project of Financial Statistics and Risk Management of
SDUFE. Qingxin MENG Qingxin Meng is the corresponding author. Department of
Mathematics, Huzhou University, Zhejiang 313000, P. R. China<EMAIL_ADDRESS>Research supported by Natural Science Foundation of Zhejiang Province for
Distinguished Young Scholar (Grant No. LR15A010001), and the National Natural
Science Foundation of China (Grant No. 11871121,11471079).

###### Abstract

The purpose of this paper is to explore the necessary conditions for
optimality of mean-field forward-backward delay control systems. A new
estimate is proved, which is a powerful tool to deal with the optimal control
problems of mean-field type with delay. Different from the classical
situation, in our case the first-order adjoint system is an anticipated mean-
field backward stochastic differential equation, and the second-order adjoint
system is a system of matrix-valued process, not mean-field type. With the
help of two adjoint systems, the second-order expansion of the variation of
the state $Y$ is proved, and therewith the Peng’s stochastic maximum
principle. As an illustrative example, we apply our result to the mean-field
game in Finance. Although we just investigate the case of one pointwise delay
for convenience, but our method is adequate for analysing the case of
pointwise delay.

Key words: Stochastic control; Maximum principle; Mean-field forward-backward
stochastic differential equation with delay, adjoint equation

MSC-classification: 93E20; 60H10

##  1 Introduction

In this paper, we are interested in the following mean-field stochastic
control problem in $\mathbb{R}$ with one pointwise delay in the state:

$\left\\{\begin{aligned}
dX^{v}(t)&=b(t,X^{v}(t),X^{v}(t-l),P_{X^{v}(t)},v(t))dt+\sigma(t,X^{v}(t),X^{v}(t-l),P_{X^{v}(t)},v(t))dW_{t},\
t\in[0,T],\\\ X^{v}(0)&=x(\theta),\ \theta\in[-l,0],\\\
-dY^{v}(t)&=f(t,X^{v}(t),X^{v}(t-l),Y^{v}(t),Z^{v}(t),P_{(X^{v}(t),Y^{v}(t))},v(t))dt-Z^{v}(t)dW_{t},\
t\in[0,T],\\\ Y^{v}(T)&=\Phi(X^{v}(T),P_{X^{v}(T)}),\end{aligned}\right.$
(1.1)

where $W$ is a one dimensional Brownian motion, $P_{\xi}=P\circ\xi^{-1}$ is
the law of $\xi$, $l>0$ is a time delay parameter, $T$ is a given time horizon
satisfying $l<T\leq 2l$, $x(\cdot)$ is a continuous function defined on
$[-l,0]$, $U$ is a subset of $\mathbb{R}$, not necessarily convex,
$\mathcal{P}_{2}(\mathbb{R}^{d})$ is the space of square integrable
probability measures on $\mathbb{R}^{d}$ with 2-Wasserstein metric,
$b,\sigma:[0,T]\times\mathbb{R}\times\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R})\times
U\mapsto\mathbb{R},$
$f:[0,T]\times\mathbb{R}\times\mathbb{R}\times\mathbb{R}\times\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R}^{2})\times
U\mapsto\mathbb{R}$,
$\Phi:\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R})\mapsto\mathbb{R}$ satisfy
some suitable conditions, and the associated cost functional is defined by
$J(v):=Y^{v}(0)$.

The purpose of the paper is to find the necessary condition of the optimal
control $u^{*}$ satisfying

$J(u^{*})=\min_{v\in\mathcal{U}_{ad}}J(v).$

Apart from the potential enormous applications in Finance of mean-field
control systems with delay, see [1], [11], [12], [25], or see the example in
Section 6, the solvability of Peng’s open problem, refer to [18], and the
progress in studying the necessary conditions of optimality for control
systems with pointwise delay in the state and with control dependent noise,
refer to [15], and the rapid development of the theories of mean-field
forward-backward stochastic differential equations (FBSDEs) and mean-field
FBSDEs with delay in recent years, refer to [2], [3], [4], [7], [8], [10],
[16], give a great impulse for the research of present work.

First, by considering the second-order term in the Taylor expansion of the
variation of the process $X$, Peng [24] in 1990 obtained the necessary
condition of the optimality for control systems where the diffusion
coefficient $\sigma$ depends on control and the control domain $U$ need not be
convex. Later, he proposed an open problem in 1998: How to extend the
classical optimal control problem to the recursive case in the above
situation, i.e., the coefficient $f$ of the BSDE depends on $(y,z)$. There are
two main obstacles met when $f$ depending on $z$ nonlinearly (see Yong [26]):
What is the second-order variational equation and how to obtain the
corresponding second-order adjoint equation? Hence, this open problem was not
solved completely until 2017 by Hu [18]. Hu overcome the above two
difficulties by building a new second-order Taylor expansion of the variation
of $Y$. His idea has been borrowed to investigate the control problems of
fully coupled forward-backward stochastic control systems, see [19].

Second, in the real world, many stochastic differential equations evolve
depending not only on the current state but also on the state at past some
time, which is called stochastic delay differential equations. This kind of
equations can be found frequently in the fields of both natural and social
sciences, for example, Finance, Economics, and Physics. Hence, the optimal
control problems with delay attracted many people’s attention, such as
Elsanousi, $\phi$ksandal, Sulem [13], Shen, Meng, Shi [25]. As we know, it is
difficult to deal with the stochastic delay control problems. The difficulties
come on the one hand from the infinite-dimensional problem, on the other hand
from the absence of Itô’s formula to handle the delay part of state.
Especially, when the control domain $U$ is not convex, there have not been
corresponding works to give the necessary condition of the optimal driven by a
delay control system until Guatteri and Masiero [15] in 2018. The authors in
[15] noticed that in the delay case the square of variation can not be written
a closed form by one equation because of the mixed term of present and past.
Hence an auxiliary matrix-value equation is brought in to deal with this
problem.

Third, mean-field stochastic differential equations (SDEs), also named McKean-
Vlasov equations, were investigated by Kac in the early 1950s [20] and McKean
in the 1960s [23]. But, only by the year of 2009 was the theory of mean-field
backward stochastic differential equations (BSDEs) built by Buckdahn,
Djehiche, Li and Peng [2], Buckdahn, Li and Peng [3]. It should be pointed
that for these types of mean-field FBSDEs the coefficients depend on the
expectation of the solution, not the law of the solution. Recently, inspired
by the work of Lions [22] in which the first-order derivative of a function
with respect to a measure was given, many scholars showed great enthusiasm for
first-order mean-field games, see for example [7]. In 2017, Buckdahn, Li,
Peng, Rainer [4] defined the second-order derivative of a function with
respect to a measure, and studied the relationship between general mean-field
SDEs and associated nonlocal PDEs. Afterwards, there were many works on the
research of general mean-field FBSDEs and their applications, such as [4],
[5], [10], [17], [21]. Note that since the expectation of a random variable
(or a stochastic process) can be expressed by the law of this random variable
(or stochastic process), we call these mean-field FBSDEs where the
coefficients depends on the law of the solution, the general mean-field
FBSDEs.

As for the optimal control problems of mean-field delay control systems, there
are also fruitful works. For example, Guo, Xiong, Zheng [16] investigated the
first-order necessary and sufficient conditions for optimality of mean-field
delay control problems where the coefficients depend on the expectation.
Buckdahn, Li, Ma [6] considered a stochastic control problem with partial
observations for general mean-field systems in the case where the coefficients
depend on the paths of the state and the conditional law of the state.

Based on the previous works [18], [15], [16], [4], [5], a natural question is:
Is it possible to develop the necessary condition for optimality of the
general mean-field forward-backward control systems with delay? We confirmed
the question.

Although our method to some extent follows the schemes in [15], there are
still some potential obstacles:

(i) Since we consider the recursive case, i.e., the coefficient $f$ depending
on $(y,z)$, the power of the term
$p(t)\delta\sigma(t)\mathbbm{1}_{E_{\varepsilon}}$ in the variation of $z$ is
$O(\varepsilon)$, but not $o(\varepsilon)$. As a consequent beyond that the
expansion of $X^{\varepsilon}$ are considered, which consists in the classical
case, we also need the second-order expansion of the variation of the state
$Y$, i.e.,

$Y^{\varepsilon}(t)=Y^{*}(t)+p(t)(X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t))+\frac{1}{2}P(t)(X^{1,\varepsilon}(t))^{2}+P_{1}(t)X^{1,\varepsilon}(t)X^{1,\varepsilon}(t-l)+\breve{Y}(t)+o(\varepsilon),$
(1.2)

which is different to the classical case given by Hu [18]:

$Y^{\varepsilon}(t)=Y^{*}(t)+p(t)(X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t))+\frac{1}{2}P(t)(X^{1,\varepsilon}(t))^{2}+\breve{Y}(t)+o(\varepsilon).$
(1.3)

(ii) Due to considering the general mean-field FBSDEs in our case, the first-
and second-order derivatives of the coefficients with respect to the measure,
and some corresponding estimates need to be handled carefully. We argue that
the power of the second-order derivative with respect to the measure
($\partial^{2}_{\nu\nu}$, see Definition 2.3) are all the higher order of
$\varepsilon$. So only the mixed second-order derivative $\partial^{2}_{\nu
a}$ is in force. For this phenomenon, the reader can refer to [4] Lemma 2.1
and note in section 6 [5] for more details. Besides, we also note that the
estimates given by Buckdahn, Li, Ma in Proposition 4.3 [5] originally for
estimating the derivatives of the coefficients with respect to the measure is
not enough for the recursive case with delay. Instead, we establish a more
general estimate, see Lemma 3.1.

The result of this paper can be summarized as follows: Suppose $u^{*}(t)$ is
the optimal control and $(X^{*}(t),Y^{*}(t),Z^{*}(t))$ is the optimal
trajectory. Consider the Hamiltonian

$\displaystyle H(t,x,x^{\prime},y,z,\nu,\mu,v;p,q,P)$ (1.4)
$\displaystyle=pb(t,x,x^{\prime},\nu,v)+q\sigma(t,x,x^{\prime},\nu,v)+\frac{1}{2}P\Big{(}\sigma(t,x,x^{\prime},\nu,v)-\sigma(t,X^{*}(t),X^{*}(t-l),P_{X^{*}(t)},u^{*}(t))\Big{)}^{2}$
$\displaystyle\quad+f(t,x,x^{\prime},y,z+p(\sigma(t,x,x^{\prime},\nu,v)-\sigma(t,X^{*}(t),X^{*}(t-l),P_{X^{*}(t)},u^{*}(t)),\mu,v),$

where
$(t,x,x^{\prime},y,z,\nu,\mu,v,p,q,P)\in[0,T]\times\mathbb{R}^{4}\times\mathcal{P}_{2}(\mathbb{R})\times\mathcal{P}_{2}(\mathbb{R}^{2})\times
U\times\mathbb{R}^{3}$.

If

$\widehat{f}^{*}_{\mu_{2}}(t):=(\frac{\partial
f}{\partial\mu})_{2}(t,\widehat{X}^{*}(t),\widehat{X}^{*}(t-l),\widehat{Y}^{*}(t),\widehat{Z}^{*}(t),P_{(X^{*}(t),Y^{*}(t))},\widehat{u}^{*}(t);X^{*}(t),Y^{*}(t))>0,\
t\in[0,T],$

$\widehat{P}\otimes P$-a.s., then there exist two pairs of stochastic
processes $(p,q),((P,Q),$ $(P_{1},Q_{1}))$, which are the solutions of the
first- and second-order adjoint equations, separately, such that for given
$l,T$ with $l<T\leq 2l$, the following the following inequality holds true:

$\displaystyle
H(t,X^{*}(t),X^{*}(t-l),Y^{*}(t),Z^{*}(t),P_{X^{*}(t)},P_{(X^{*}(t),Y^{*}(t))},v,p(t),q(t),P(t))$
(1.5) $\displaystyle\geq
H(t,X^{*}(t),X^{*}(t-l),Y^{*}(t),Z^{*}(t),P_{X^{*}(t)},P_{(X^{*}(t),Y^{*}(t))},u^{*}(t),p(t),q(t),P(t)),$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad
t\in[0,T],\text{a.s.}\ \text{a.e.}$

There are two points needed to be stressed:

a) We assume that the coefficients just depend on the law of the state
$(X_{t},Y_{t})$, $P_{(X_{t},Y_{t})}$, however, there is no essential change
when the coefficients depend on the laws of the state and the delay,
$P_{(X_{t},X_{t-l},Y_{t},Y_{t-l})}$, meanwhile.

b) If $T\leq l$, our control problem reduces to the non delay case. Hence, in
this paper we present the case of one pointwise delay, i.e., $l<T\leq 2l.$ But
our method can also be used to handle the pointwise delay case, i.e.,
$Nl<T\leq(N+1)l,\ N>1,\ N\in\mathbb{N}$, see Remark 3.7.

The paper is arranged as follows: Section 2 introduces some element theory of
derivative in the Wasserstein Space and some usual spaces. In Section 3, we
formulate the control problem, and establish the variational and adjoint
equations. The expansion of $Y^{\varepsilon}$ is stated in Section 4. Section
5 is devoted to introducing the main result—Stochastic Maximum Principle. An
illustration is given in Section 6. In the last Section we list some notations
and show the expansion of $I_{22}$ accurately.

##  2 Preliminaries

###  2.1 Derivative in the Wasserstein Space

Let $\mathcal{P}_{2}(\mathbb{R}^{d})$ denote the space of all probability
measure $\nu$ on $(\mathbb{R}^{d},\mathcal{B}(\mathbb{R}^{d}))$ with finite
second moment, which is endowed with the 2-Wassertein metric:

$\displaystyle W_{2}(\nu_{1},\nu_{2})$
$\displaystyle=\inf\Big{\\{}(\int_{\mathbb{R}^{d}\times\mathbb{R}^{d}}|y_{1}-y_{2}|^{2}\rho(dy_{1},dy_{2}))^{\frac{1}{2}},\
\rho\in\mathcal{P}_{2}(\mathbb{R}^{2d})\ \ \text{satisfying}\ $
$\displaystyle\qquad\qquad\qquad\qquad\rho(A\times\mathbb{R}^{d})=\nu_{1}(A),\
\rho(\mathbb{R}^{d}\times B)=\nu_{2}(B),\
A,B\in\mathcal{B}(\mathbb{R})\Big{\\}}.$

Let us recall the differentiable of a function
$h:\mathcal{P}_{2}(\mathbb{R}^{d})\rightarrow\mathbb{R}$ with respect to a
measure. The idea is to identify a distribution
$\mu\in\mathcal{P}_{2}(\mathbb{R}^{d})$ with a random variable $\xi\in
L^{2}(\mathcal{F};\mathbb{R}^{d})$ so that $\mu=P_{\xi}$.

###### Definition 2.1

A function $h:\mathcal{P}_{2}(\mathbb{R}^{d})\rightarrow\mathbb{R}$ is called
differentiable in $\mu_{0}\in\mathcal{P}_{2}(\mathbb{R}^{d})$, if the
functional $\tilde{h}:L^{2}(\mathcal{F};\mathbb{R}^{d})\rightarrow\mathbb{R}$
defined $\tilde{h}(\xi):=h(P_{\xi})$ is differentiable (in Fréchet sense) in
$\xi_{0}$ with $P_{\xi_{0}}=\mu_{0}$.

That means, for any $\zeta\in L^{2}(\mathcal{F};\mathbb{R}^{d})$,

$\tilde{h}(\xi_{0}+\zeta)-\tilde{h}(\xi_{0})=D\tilde{h}(\xi_{0})(\zeta)+o(||\zeta||_{L_{2}}).$

But $D\tilde{h}(\xi_{0})$ is a continuous linear operator in
$L^{2}(\mathcal{F};\mathbb{R}^{d})$. From Riesz’s Representation Theorem,
there exists a unique random variable $\eta_{0}\in
L^{2}(\mathcal{F};\mathbb{R}^{d})$ such that
$D\tilde{h}(\xi_{0})(\zeta)=<\eta_{0},\zeta>=E[\eta_{0}\zeta],\ \zeta\in
L^{2}(\mathcal{F};\mathbb{R}^{d})$, where $<\cdot,\cdot>$ denotes the“dual
product” on $L^{2}(\mathcal{F};\mathbb{R}^{d})$. Later, Cardaliaguet proved
that $\eta_{0}$ can be written $\eta_{0}=\phi(\xi_{0})$, where $\phi$ is a
Borel measure function and depends only on the law of $\xi_{0}$, but not
$\xi_{0}$ itself. We define $\partial_{\nu}h(P_{\xi_{0}};a):=\phi(a),\
a\in\mathbb{R}^{d}.$ Note that $\partial_{\nu}h(P_{\xi_{0}};a)$ is only
$P_{\xi_{0}}(da)$-a.e. uniquely determined. Moreover, we can identify
$D\tilde{h}(\xi_{0})$ by

$D\tilde{h}(\xi_{0})=\phi(\xi_{0})=\partial_{\nu}h(P_{\xi_{0}};\xi_{0}).$

Since we shall consider the functions $h$, which are differentiable on the
whole space $\mathcal{P}_{2}(\mathbb{R}^{d})$, we assume that
$\widetilde{h}:L^{2}(\mathcal{F};\mathbb{R}^{d})\rightarrow\mathbb{R}$ is
Fréchet differentiable at all elements of $L^{2}(\mathcal{F};\mathbb{R}^{d})$.
Thereby, we have, for all $\xi\in L^{2}(\mathcal{F};\mathbb{R}^{d})$ the
derivative $\partial_{\nu}h(P_{\xi};a),\ a\in\mathbb{R}^{d}$ defined
$P_{\xi}$-a.e. Lemma 3.2 in [9] allows to show that if the Fréchet derivative
$\xi\mapsto D\tilde{h}(\xi)=\partial_{\nu}h(P_{\xi};\xi)$ is Lipschitz
continuous with a positive Lipschitz constant $L$, then there exists for all
$\xi\in L^{2}(\mathcal{F};\mathbb{R}^{d})$ a $P_{\xi}$-version of
$\partial_{\nu}h(P_{\xi};\cdot):\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ such
that $\partial_{\nu}h(P_{\xi};\cdot)$ is Lipschitz continuous with the same
constant $L$.

###### Definition 2.2

By $C_{b}^{1,1}(\mathcal{P}_{2}(\mathbb{R}^{d}))$ we denote the space of
continuously differentiable function $h$ on $\mathcal{P}_{2}(\mathbb{R}^{d})$
with the properties:

$\mathrm{i)}$ (boundness) for all
$\nu\in\mathcal{P}_{2}(\mathbb{R}^{d}),a\in\mathbb{R}^{d},$
$\partial_{\nu}h(\nu;a)$ is bounded by a positive constant $L$;

$\mathrm{ii)}$(Lipschitz continuous) $\partial_{\nu}h(\cdot;\cdot)$ is
Lipschitz continuous in $(\nu,a).$

###### Definition 2.3

By $C_{b}^{2,1}(\mathcal{P}_{2}(\mathbb{R}^{d}))$ we denote the space of
function $h\in C_{b}^{1,1}(\mathcal{P}_{2}(\mathbb{R}^{d}))$ with the
properties:

$\mathrm{i)}$ for each
$\nu\in\mathcal{P}_{2}(\mathbb{R}^{d}),(\partial_{\nu}h)_{j}(\nu;\cdot)\in
C_{b}^{1,1}(\mathbb{R}^{d}),\ 1\leq j\leq d,$

$\mathrm{ii)}$ for each $a\in\mathbb{R}^{d}$,
$(\partial_{\nu}h)_{j}(\cdot;a)\in
C_{b}^{1,1}(\mathcal{P}_{2}(\mathbb{R}^{d})),\ 1\leq j\leq d,$

$\mathrm{iii)}$ all the derivatives of $h$ up to order 2,
$(\partial_{\nu}h,\partial^{2}_{\nu\nu}h,\partial^{2}_{\nu a}h)$, are bounded
and Lipschitz continuous with respect to all variables.

Let us recall the second-order Taylor expansion of
$h:\mathcal{P}_{2}(R^{d})\rightarrow\mathbb{R}$, which plays an important role
in our analysis, see [4] for more details. Let
$(\widehat{\Omega},\widehat{\mathcal{F}},\widehat{P})$ be an independent copy
of the probability space $(\Omega,\mathcal{F},P)$. By
$(\widehat{\xi}_{0},\widehat{\zeta})$ defined on
$(\widehat{\Omega},\widehat{\mathcal{F}},\widehat{P})$ we denote the copy of
the pair $(\xi_{0},\zeta)$ defined on $(\Omega,\mathcal{F},P)$, i.e.,
$\widehat{P}_{(\widehat{\xi}_{0},\widehat{\zeta})}=P_{(\xi_{0},\zeta)}$. The
expectation $\widehat{E}[\cdot]$ only works on the variables with “hat”. By
$(\cdot,\cdot)$ we denote the inner product of $\mathbb{R}^{d}$. We now
consider the product space

$(\Omega\times\widehat{\Omega},\mathcal{F}\otimes\widehat{\mathcal{F}},P\otimes\widehat{P})=(\Omega\times\Omega,\mathcal{F}\otimes\mathcal{F},P\otimes
P),$

and identify
$(\widehat{\xi}_{0},\widehat{\zeta})(\widehat{\omega},\omega):=(\widehat{\xi}_{0},\widehat{\zeta})(\widehat{\omega}),\
(\widehat{\omega},\omega)\in\widehat{\Omega}\times\Omega.$

Let $h\in C^{2,1}_{b}(\mathcal{P}_{2}(\mathbb{R}^{d}))$, one has

$\displaystyle
h(P_{\xi_{0}+\zeta})-h(P_{\xi_{0}})=\widetilde{h}(\xi_{0}+\zeta)-\widetilde{h}(\xi_{0})$
(2.1)
$\displaystyle=\int_{0}^{1}<D\widetilde{h}(\xi_{0}+\rho\zeta),\zeta>d\rho=\int_{0}^{1}E[\partial_{\nu}h(P_{\xi_{0}+\rho\zeta};\xi_{0}+\rho\zeta)\cdot\zeta]d\rho$
$\displaystyle=E[\partial_{\nu}h(P_{\xi_{0}};\xi_{0})\cdot\zeta]+\int_{0}^{1}E[(\partial_{\nu}h(P_{\xi_{0}+\rho\zeta};\xi_{0}+\rho\zeta)-\partial_{\nu}h(P_{\xi_{0}};\xi_{0}))\cdot\zeta]d\rho.$

However, it is clear that

$\displaystyle\partial_{\nu}h(P_{\xi_{0}+\rho\zeta};\xi_{0}+\rho\zeta)-\partial_{\nu}h(P_{\xi_{0}};\xi_{0})=D\widetilde{\partial_{\nu}h}(\xi_{0}+\rho\xi)-D\widetilde{\partial_{\nu}h}(\xi_{0})$
(2.2)
$\displaystyle=\rho\int_{0}^{1}<D\widetilde{\partial_{\nu}h}(\xi_{0}+\lambda\rho\zeta;a),\zeta>d\lambda\Big{|}_{a=\xi_{0}+\rho\xi}+\rho\int_{0}^{1}(\partial_{a}\partial_{\nu}h(P_{\xi_{0}};\xi_{0}+\lambda\rho\zeta),\zeta)d\lambda$
$\displaystyle=\rho<D\widetilde{\partial_{\nu}h}(\xi_{0};a),\zeta>\Big{|}_{a=\xi_{0}}+\rho(\partial_{a}\partial_{\nu}h(P_{\xi_{0}};\xi_{0}),\zeta)+R_{1}(\rho,\zeta),$

where

$\displaystyle R_{1}(\rho,\zeta):$
$\displaystyle=\rho\int_{0}^{1}<D\widetilde{\partial_{\nu}h}(\xi_{0}+\lambda\rho\zeta;a),\zeta>d\lambda\Big{|}_{a=\xi_{0}+\rho\xi}-\rho<D\widetilde{\partial_{\nu}h}(\xi_{0};a),\zeta>\Big{|}_{a=\xi_{0}}$
$\displaystyle+\rho\int_{0}^{1}(\partial_{a}\partial_{\nu}h(P_{\xi_{0}};\xi_{0}+\lambda\rho\zeta),\zeta)d\lambda-\rho(\partial_{a}\partial_{\nu}h(P_{\xi_{0}};\xi_{0}),\zeta).$

Due to

$D\widetilde{\partial_{\nu}h}(\xi_{0};a)=\partial_{\nu}(\partial_{\nu}h(\cdot;a))(P_{\xi_{0}};\xi_{0}):=\partial^{2}_{\nu}h(P_{\xi_{0}};a,b)\Big{|}_{b=\xi_{0}},$

then we have

$\displaystyle
D\widetilde{\partial_{\nu}h}(\xi_{0}+\rho\xi)-D\widetilde{\partial_{\nu}h}(\xi_{0})=\rho\widehat{E}[\partial_{\nu}^{2}h(P_{\xi_{0}};a,b)|_{b=\widehat{\xi}_{0}}\cdot\widehat{\zeta}]|_{a=\xi_{0}}+\rho
E[\partial_{a}\partial_{\nu}h(P_{\xi};a)|_{a=\xi_{0}}\cdot\zeta]+R_{1}(\rho,\zeta).$

Accordingly, we obtain

$\displaystyle h(P_{\xi_{0}+\zeta})-h(P_{\xi_{0}})$
$\displaystyle=E[\partial_{\nu}h(P_{\xi_{0}};\xi_{0})\cdot\zeta]+\frac{1}{2}E\widehat{E}\Big{\\{}tr[\partial_{\nu}^{2}h(P_{\xi_{0}};\xi_{0},\widehat{\xi}_{0})\zeta\otimes\widehat{\zeta}]\Big{\\}}$
(2.3)
$\displaystyle\quad+\frac{1}{2}E\Big{\\{}tr[\partial_{a}\partial_{\nu}h(P_{\xi_{0}};\xi_{0})\zeta\otimes\zeta]\Big{\\}}+R(\zeta),$

where $R(\zeta)=\int_{0}^{1}E[R_{1}(\rho,\zeta)\cdot\zeta]d\rho$. Clearly,
$|R(\zeta)|\leq CE[|\zeta|^{3}\wedge|\zeta|^{2}]$.

For arbitrary $\nu_{0}\in\mathcal{P}_{2}(\mathbb{R}^{d})$ with
$\nu_{0}=P_{\xi_{0}}$, and $\zeta\in L^{2}(\mathcal{F};\mathbb{R}^{d})$, we
call $D^{2}_{\zeta}h(\nu_{0})$ the second order derivative of $h$ at
$\nu_{0}\in\mathcal{P}_{2}(\mathbb{R}^{d})$ in the direction $\zeta$, where
$D^{2}_{\zeta}h(\nu_{0})$ is given by

$\displaystyle
D^{2}_{\zeta}h(\nu_{0})=D^{2}_{\zeta}h(P_{\xi_{0}})=E\widehat{E}\Big{\\{}tr[\partial_{\nu}^{2}h(P_{\xi_{0}};\xi_{0},\widehat{\xi}_{0})\zeta\otimes\widehat{\zeta}]\Big{\\}}+E\Big{\\{}tr[\partial_{y}\partial_{\nu}h(P_{\xi_{0}};\xi_{0})\zeta\otimes\zeta]\Big{\\}}.$

Consequently, (2.3) can be written as

$h(P_{\xi_{0}+\zeta})-h(P_{\xi_{0}})=D_{\zeta}h(P_{\xi_{0}})+\frac{1}{2}D^{2}_{\zeta}h(P_{\xi_{0}})+R(\zeta),$

where $|R(\zeta)|\leq CE[|\zeta|^{3}\wedge|\zeta|^{2}]=O(||\zeta||_{2}^{2})$.

###  2.2 Function spaces

Let $(\Omega,\mathcal{F},P;\mathbb{F}=\\{\mathcal{F}_{t}\\}_{t\geq 0})$ be a
complete, filtered probability space, on which a 1-dimensional Brownian motion
$W$ is defined. $T>0$ is a fixed time horizon, $l>0$ is a time delay parameter
and $\mathcal{F}^{*}$ is a sub-$\sigma$-field of $\mathcal{F}$ satisfying the
following two properties:

i) $\mathcal{F}^{*}$ is independent of the Brownian motion $W$;

ii) $\mathcal{P}_{2}(\mathbb{R}^{d})=\\{P_{\xi},\xi\in
L^{2}(\mathcal{F}^{*};\mathbb{R^{d}})\\}$.
Let $\mathcal{F}_{t}=\sigma\\{B_{s},0\leq s\leq
t\\}\vee\mathcal{F}^{*}\vee\mathcal{N}$, where $\mathcal{N}$ is the set of all
$P$-null subset. By $U^{\intercal}$ we denote the transpose of any vector or
matrix $U$.

Let $\ell\geq 1$, we introduce the following spaces, which are used frequently
later:

$\bullet$ $L^{\ell}(\mathcal{F}_{t};\mathbb{R}^{d})$ denotes the set of
$\mathbb{R}^{d}$-valued, $\mathcal{F}_{t}$-measurable random variable $\xi$
with

$||\xi||_{L^{\ell}}:=(E[|\xi|^{\ell}])^{\frac{1}{\ell}}<+\infty;$

$\bullet$ $\mathcal{S}_{\mathbb{F}}^{\ell}(t,T;\mathbb{R}^{d})$ denotes the
set of $\mathbb{R}^{d}$-valued, $\mathcal{F}_{s}$-predictable processes $\psi$
on $[t,T]$, such that

$||\psi||_{\mathcal{S}^{\ell}_{\mathbb{F}}}=\Big{\\{}E[\sup_{s\in[t,T]}|\psi(s)|^{\ell}]\Big{\\}}^{\frac{1}{\ell}}<+\infty;$

$\bullet$ $\mathcal{H}_{\mathbb{F}}^{\ell}(t,T;\mathbb{R}^{d})$ denotes the
set of $\mathbb{R}^{d}$-valued, $\mathcal{F}_{s}$-progressively measurable
processes $\psi$ on $[t,T]$, such that

$||\psi||_{\mathcal{H}^{\ell}_{\mathbb{F}}}=\Big{\\{}E[(\int_{t}^{T}|\psi(s)|^{2}ds)^{\frac{\ell}{2}}]\Big{\\}}^{\frac{1}{\ell}}<+\infty.$

##  3 Formulation

Suppose $U$ is a subset of $\mathbb{R}$, not necessarily convex. Let
$\mathcal{U}_{ad}$ be the set of all admissible controls, that is, the set of
all $\mathcal{F}_{t}$-adapted stochastic processes
$v:[0,T]\times\Omega\rightarrow U$ with for all $\beta\geq 1$,

$E[\sup_{t\in[0,T]}|v(t)|^{\beta}dt]<+\infty.$

Let the mappings

$\displaystyle(b,\sigma)(t,x,x^{\prime},\nu,v):[0,T]\times\mathbb{R}\times\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R})\times
U\rightarrow\mathbb{R},\
\Phi(x,\nu):\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R})\rightarrow\mathbb{R},$
$\displaystyle
f(t,x,x^{\prime},y,z,\mu,v):[0,T]\times\mathbb{R}\times\mathbb{R}\times\mathbb{R}\times\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R}^{2})\times
U\rightarrow\mathbb{R},$

satisfy the following assumptions:

(H3.1) $b,\sigma,\Phi$ are bounded by
$C(1+|x|+|x^{\prime}|+(\int_{\mathbb{R}}x^{2}\nu(dx))^{\frac{1}{2}}+|v|)$, $f$
is bounded by
$C(1+|x|+|x^{\prime}|+|y|+|z|+(\int_{\mathbb{R}}x^{2}\nu(dx))^{\frac{1}{2}}+|v|)$,
$b,\sigma,f$ and all the derivatives of $b,\sigma,f$ up to order 2 are
Lipschitz continuous in $v$.

(H3.2) For each $(t,v)\in[0,T]\times U,$ $b,\sigma\in
C_{b}^{1,1}(\mathbb{R}^{2}\times\mathcal{P}_{2}(\mathbb{R})),\ f\in
C_{b}^{1,1}(\mathbb{R}^{4}\times\mathcal{P}_{2}(\mathbb{R}^{2})),\ \Phi\in
C_{b}^{1,1}(\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R}))$, i.e.,

i) For each $(t,x,x^{\prime},y,z,v)\in[0,T]\times\mathbb{R}^{4}\times U$,
$(b,\sigma)(t,x,x^{\prime},\cdot,v)$ belong to
$C_{b}^{1,1}(\mathcal{P}_{2}(\mathbb{R})),$ $\Phi(x,\cdot)$ belongs to
$C_{b}^{1,1}(\mathcal{P}_{2}(\mathbb{R})),$ $f(t,x,x^{\prime},y,z,\cdot,v)$
belongs to $C_{b}^{1,1}(\mathcal{P}_{2}(\mathbb{R}^{2}))$;

ii) For each
$(t,\nu,\mu,v)\in[0,T]\times\mathcal{P}_{2}(\mathbb{R})\times\mathcal{P}_{2}(\mathbb{R}^{2})\times
U$, $(b,\sigma)(t,\cdot,\cdot,\nu,v)$ belong to $C_{b}^{1}(\mathbb{R}^{2})$,
$\Phi(\cdot,\nu)$ belongs to $C_{b}^{1}(\mathbb{R}^{2})$, and
$f(t,\cdot,\cdot,\cdot,\cdot,\mu,v)$ belongs to $C_{b}^{1}(\mathbb{R}^{4})$;

iii) The derivatives of $b,\sigma,\Phi,f$ up to order $1$ are Lipschitz
continuous and bounded.

(H3.3) For each $(t,v)\in[0,T]\times U,$ $(b,\sigma)\in
C_{b}^{2,1}(\mathbb{R}^{2}\times\mathcal{P}_{2}(\mathbb{R})),$ $f\in
C_{b}^{2,1}(\mathbb{R}^{4}\times\mathcal{P}_{2}(\mathbb{R}^{2}))$, $\Phi\in
C_{b}^{2,1}(\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R}))$, i.e., $b,\sigma\in
C_{b}^{1,1}(\mathbb{R}^{2}\times\mathcal{P}_{2}(\mathbb{R})),\ f\in
C_{b}^{1,1}(\mathbb{R}^{4}\times\mathcal{P}_{2}(\mathbb{R}^{2})),\ \Phi\in
C_{b}^{1,1}(\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R}))$ satisfy:
i) For each $(t,v)\in[0,T]\times U$, $\partial_{k}\phi(t,\cdot,\cdot,\cdot,v)$
belongs to $C_{b}^{1,1}(\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R})),$
$\phi=b,\sigma,\ k=x,x^{\prime},$ $\partial_{x}\Phi(\cdot,\cdot)$ belongs to
$C_{b}^{1,1}(\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R})),$
$\partial_{k}f(t,\cdot,\cdot,\cdot,\cdot,\cdot,v)$ belongs to
$C_{b}^{1,1}(\mathbb{R}^{4}\times\mathcal{P}_{2}(\mathbb{R}^{2})),\
k=x,x^{\prime},y,z$;

ii) For each $(t,v)\in[0,T]\times U,$
$\partial_{\nu}\phi(t,\cdot,\cdot,v;\cdot)$ belongs to
$C_{b}^{1,1}(\mathbb{R}^{2}\times\mathcal{P}_{2}(\mathbb{R})\times\mathbb{R}),\phi=b,\sigma,$
$\partial_{\nu}\Phi(\cdot,\cdot;\cdot)$ belongs to
$C_{b}^{1,1}(\mathbb{R}^{2}\times\mathcal{P}_{2}(\mathbb{R})\times\mathbb{R})$,
$(\partial_{\mu}f)_{j}(t,\cdot,\cdot,\cdot,\cdot,\cdot,v;\cdot,\cdot)$ belongs
to
$C_{b}^{1,1}(\mathbb{R}^{4}\times\mathcal{P}_{2}(\mathbb{R}^{2})\times\mathbb{R}^{2}),\
j=1,2$;

iii) All the derivatives of $b,\sigma,\Phi,f$ up to order $2$ are Lipschitz
continuous and bounded.

We consider the system (1.1) and the cost functional

$\displaystyle J(v)=Y^{v}(0).$ (3.1)

The purpose of this paper is to investigate the necessary condition of the
optimal control $u^{*}$.

In subsequent sections, if no confusion, by $u^{*}$ and $(X^{*},Y^{*},Z^{*})$
we always denote the optimal control and the corresponding trajectory, by
$\rho:(0,+\infty)\rightarrow(0,+\infty)$ denote a measurable function with
property: $\rho(\varepsilon)\rightarrow 0$ as $\varepsilon\downarrow 0$, as
well as by $C$ denote a positive constant. $\rho$ and $C$ may be different
from line to line.

###  3.1 Variational equations

In this subsection we investigate the differentiation of the state process
$X^{\varepsilon}$ by the spike variation method, which we now introduce.

Let $\varepsilon>0$ and $E_{\varepsilon}$ be a Borel subset of $[0,T]$ with
the Lebesgue measure $\lambda(E_{\varepsilon})=\varepsilon.$ Suppose
$v\in\mathcal{U}_{ad}$, the spike variation of $u^{*}$ is described as
follows: for $t\in[0,T]$,

$v^{\varepsilon}(t)=v(t)\mathbbm{1}_{E_{\varepsilon}}(t)+u^{*}(t)\mathbbm{1}_{(E_{\varepsilon})^{C}}(t).$
(3.2)

Accordingly, by
$(X^{\varepsilon},Y^{\varepsilon},Z^{\varepsilon}):=(X^{v^{\varepsilon}},Y^{v^{\varepsilon}},Z^{v^{\varepsilon}})$
we denote the solution of the system (1.1) with $v^{\varepsilon}$.

Let
$(\overline{\Omega},\overline{\mathcal{F}},\overline{P}),(\widehat{\Omega},\widehat{\mathcal{F}},\widehat{P}),\
(\widetilde{\Omega},\widetilde{\mathcal{F}},\widetilde{P})$ be three complete
probability spaces. By $W,\ E,\ \xi,\ X(\cdot)$ with “bar”, “hat”, “tilde” we
denote the Brown motion, expectation, random variable, stochastic process
defined on the spaces
$(\overline{\Omega},\overline{\mathcal{F}},\overline{P}),(\widehat{\Omega},\widehat{\mathcal{F}},\widehat{P}),\
(\widetilde{\Omega},\widetilde{\mathcal{F}},\widetilde{P})$, respectively.
Note that the spaces
$(\Omega,\mathcal{F},P),(\overline{\Omega},\overline{\mathcal{F}},\overline{P}),(\widehat{\Omega},\widehat{\mathcal{F}},\widehat{P})$
and $(\widetilde{\Omega},\widetilde{\mathcal{F}},\widetilde{P})$ are
independent.
For
$\phi=b,\sigma,b_{x},\sigma_{x},b_{x^{\prime}},\sigma_{x^{\prime}},\psi=b,\sigma$,
define

$\displaystyle\Delta\phi(t):=\phi(t,X^{*}(t),X^{*}(t-l),P_{X^{*}(t)},v(t))-\phi(t,X^{*}(t),X^{*}(t-l),P_{X^{*}(t)},u^{*}(t)),$
(3.3) $\displaystyle(\psi_{x},\psi_{xx})(t):=(\frac{\partial\psi}{\partial
x},\frac{\partial^{2}\psi}{\partial
x^{2}})(t,X^{*}(t),X^{*}(t-l),P_{X^{*}(t)},u^{*}(t)),$
$\displaystyle(\widehat{\psi}_{\nu},\widehat{\psi}_{\nu
a})(t):=(\frac{\partial\psi}{\partial\nu},\frac{\partial^{2}\psi}{\partial\nu\partial
a})(t,X^{*}(t),X^{*}(t-l),P_{X^{*}(t)},u^{*}(t);\widehat{X}^{*}(t)),$
$\displaystyle(\widehat{\psi}^{*}_{\nu},\widehat{\psi}^{*}_{\nu
a})(t):=(\frac{\partial\phi}{\partial\nu},\frac{\partial^{2}\phi}{\partial\nu\partial
a})(t,\widehat{X}^{*}(t),\widehat{X}^{*}(t-l),P_{X^{*}(t)},\widehat{u}^{*}(t);X^{*}(t)),$
$\displaystyle\widetilde{\widehat{\psi}}_{\nu\nu}(t):=\frac{\partial^{2}\psi}{\partial\nu^{2}}(t,X^{*}(t),X^{*}(t-l),P_{X^{*}(t)},u^{*}(t);\widehat{X}^{*}(t),\widetilde{X}^{*}(t)).$

Inspired by the work of Peng [24], when $\sigma$ depending on control and the
control set $U$ being not necessarily convex, for each $\varepsilon>0,$ we can
find two stochastic processes $X^{1,\varepsilon},X^{2,\varepsilon}$ such that

$\displaystyle\mathrm{i)}\
X^{\varepsilon}(t)-X^{*}(t)-X^{1,\varepsilon}(t)=O(\varepsilon),$ (3.4)
$\displaystyle\mathrm{ii)}\
X^{\varepsilon}(t)-X^{*}(t)-X^{1,\varepsilon}(t)-X^{2,\varepsilon}(t)=o(\varepsilon).$

In our case, it is easy to check that $X^{1,\varepsilon},X^{2,\varepsilon}$
satisfy the following two variational equations. The first-order variational
equation is

$\left\\{\begin{aligned}
dX^{1,\varepsilon}(t)&=\Big{\\{}b_{x}(t)X^{1,\varepsilon}(t)+b_{x^{\prime}}(t)X^{1,\varepsilon}(t-l)+\widehat{E}[\widehat{b}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]+\Delta
b(t)\mathbbm{1}_{E_{\varepsilon}}(t)\Big{\\}}dt\\\
&\quad+\Big{\\{}\sigma_{x}(t)X^{1,\varepsilon}(t)+\sigma_{x^{\prime}}(t)X^{1,\varepsilon}(t-l)+\widehat{E}[\widehat{\sigma}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]\\\
&\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\
+\Delta\sigma(t)\mathbbm{1}_{E_{\varepsilon}}(t)\Big{\\}}dW(t),\ t\in[0,T],\\\
X^{1,\varepsilon}(t)&=0,\ t\in[-l,0],\end{aligned}\right.$ (3.5)

and the second-order variational equation, which is a linear mean-field
forward SDE with delay, has the following form:

$\displaystyle dX^{2,\varepsilon}(t)$
$\displaystyle=\Big{\\{}b_{x}(t)X^{2,\varepsilon}(t)+b_{x^{\prime}}(t)X^{2,\varepsilon}(t-l)+\widehat{E}[\widehat{b}_{\nu}(t)\widehat{X}^{2,\varepsilon}(t)]+b_{xx^{\prime}}(t)X^{1,\varepsilon}(t)X^{1,\varepsilon}(t-l)$
(3.6)
$\displaystyle\quad+\frac{1}{2}\Big{(}b_{xx}(t)(X^{1,\varepsilon}(t))^{2}+\widehat{E}[\widehat{b}_{\nu
a}(t)(\widehat{X}^{1,\varepsilon}(t))^{2}]+b_{x^{\prime}x^{\prime}}(t)(X^{1,\varepsilon}(t-l))^{2}\Big{)}$
$\displaystyle\quad+\Big{(}\Delta b_{x}(t)X^{1,\varepsilon}(t)+\Delta
b_{x^{\prime}}(t)X^{1,\varepsilon}(t-l)+\widehat{E}[\Delta\widehat{b}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]\Big{)}\mathbbm{1}_{E_{\varepsilon}}(t)\Big{\\}}dt$
$\displaystyle\quad+\Big{\\{}\sigma_{x}(t)X^{2,\varepsilon}(t)+\sigma_{x^{\prime}}(t)X^{2,\varepsilon}(t-l)+\widehat{E}[\widehat{\sigma}_{\nu}(t)\widehat{X}^{2,\varepsilon}(t)]+\sigma_{xx^{\prime}}(t)X^{1,\varepsilon}(t)X^{1,\varepsilon}(t-l)$
$\displaystyle\quad+\frac{1}{2}\Big{(}\sigma_{xx}(t)(X^{1,\varepsilon}(t))^{2}+\widehat{E}[\widehat{\sigma}_{\nu
a}(t)(\widehat{X}^{1,\varepsilon}(t))^{2}]+\sigma_{x^{\prime}x^{\prime}}(t)(X^{1,\varepsilon}(t-l))^{2}\Big{)}$
$\displaystyle\quad+\Big{(}\Delta\sigma_{x}(t)X^{1,\varepsilon}(t)+\Delta\sigma_{x^{\prime}}(t)X^{1,\varepsilon}(t-l)+\widehat{E}[\Delta\widehat{\sigma}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]\Big{)}\mathbbm{1}_{E_{\varepsilon}}(t)\Big{\\}}dW_{t},\
t\in[0,T],$ $\displaystyle X^{2,\varepsilon}(t)$ $\displaystyle=0,\
t\in[-l,0].$

Under the assumptions (H3.1)-(H3.3), the equation (3.5) and equation (3.6)
exist unique solutions
$X^{1,\varepsilon}=(X^{1,\varepsilon}(t))_{t\in[-l,T]}\in\mathcal{S}^{\beta}_{\mathbb{F}}(-l,T;\mathbb{R})$
($\mathcal{S}^{\beta}_{\mathbb{F}}(-l,T)$, for short) and
$X^{2,\varepsilon}=(X^{2,\varepsilon}(t))_{t\in[-l,T]}\in\mathcal{S}^{\beta}_{\mathbb{F}}(-l,T)$
with $\beta\geq 2,$ respectively, see Shen, Meng and Shi [25].

Let us first introduce a useful estimate in studying stochastic maximum
principle especially driven by general mean-field forward-backward control
systems with delay.

###### Lemma 3.1

Under the assumptions (H3.1), (H3.2), let
$(\overline{\Omega},\overline{\mathcal{F}},\overline{P})$ be an intermediate
probability space and independent of space $(\Omega,\mathcal{F},P)$, and let
$(\Psi_{1}(\omega,\overline{\omega},t))_{t\in[-l,T]},(\Psi_{2}(\overline{\omega},t))_{t\in[-l,T]}$
be two stochastic processes defined on the product space
$(\Omega\times\overline{\Omega},\mathcal{F}\times\overline{\mathcal{F}},P\otimes\overline{P})$
and the space $(\overline{\Omega},\overline{\mathcal{F}},\overline{P})$,
respectively. Moreover, assume $\Psi_{i},\ i=1,2$ satisfies the following
properties:
$\mathrm{i)}$ There exists a constant $C>0$, such that for $t\in[-l,T],$
$|\Psi_{1}(\omega,\overline{\omega},t)|\leq C,P\otimes\overline{P}$-a.s.
$\mathrm{ii)}$ For $\beta\geq 1$,
$\overline{E}[\sup_{t\in[-l,T]}|\Psi_{2}(\overline{\omega},t)|^{2\beta}]\leq
C_{\beta}$.
Then

$E\Big{[}\int_{-l}^{T}|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{X}^{1,\varepsilon}(t)]|^{4}dt\Big{]}\leq\varepsilon^{2}\rho(\varepsilon).$
(3.7)

Proof. From the definitions of $\hat{b}_{\nu}(t),\hat{\sigma}_{\nu}(t)$ (see
(3.3)), for the convenience of edit we denote, for $h=b,\sigma$,

$\overline{\widehat{h}}_{\nu}(t)=\frac{\partial
h}{\partial\nu}(t,\overline{X}^{*}(t),\overline{X}^{*}(t-l),P_{X^{*}(t)},\overline{u}^{*}(t);\widehat{X}^{*}(t)).$
(3.8)

Recall in this paper we consider the case of one pointwise delay, i.e.,
$l<T\leq 2l$. From the property of integral interval additivity, it is easy to
see

$\displaystyle
E\Big{[}\int_{-l}^{T}|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{X}^{1,\varepsilon}(t)]|^{4}dt\Big{]}=E\Big{[}\int_{-l}^{0}|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{X}^{1,\varepsilon}(t)]|^{4}dt\Big{]}$
$\displaystyle+E\Big{[}\int_{0}^{l}|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{X}^{1,\varepsilon}(t)]|^{4}dt\Big{]}+E\Big{[}\int_{l}^{T}|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{X}^{1,\varepsilon}(t)]|^{4}dt\Big{]}.$

Hence, in what follows we will prove (3.7) in three cases.

Case 1 If $-l\leq t<0,$ $X^{1,\varepsilon}(t)\equiv 0$. Obviously,

$E\Big{[}\int_{-l}^{0}|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{X}^{1,\varepsilon}(t)]|^{4}dt\Big{]}\leq\varepsilon^{2}\rho(\varepsilon).$
(3.9)

Case 2 If $0\leq t<l$, $X^{1,\varepsilon}(t-l)\equiv 0$ which means that the
delay terms will vanish. Hence, the equation (1.1) becomes a mean-field FBSDE
without delay, and the corresponding first-order variational equation (3.5)
can be rewritten as

$\left\\{\begin{aligned}
dX^{1,\varepsilon}(t)&=\Big{\\{}b_{x}(t)X^{1,\varepsilon}(t)+\widehat{E}[\widehat{b}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]+\Delta
b(t)\mathbbm{1}_{E_{\varepsilon}}(t)\Big{\\}}dt\\\
&\quad+\Big{\\{}\sigma_{x}(t)X^{1,\varepsilon}(t)+\widehat{E}[\widehat{\sigma}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]+\Delta\sigma(t)\mathbbm{1}_{E_{\varepsilon}}(t)\Big{\\}}dW(t),\
t\in[0,T],\\\ X^{1,\varepsilon}(t)&=0,\ t\in[-l,0].\end{aligned}\right.$
(3.10)

Let us consider the following SDE:

$\left\\{\begin{aligned}
d\eta^{1}(t)&=-(b_{x}(t)-|\sigma_{x}(t)|^{2})\eta^{1}(t)dt-\sigma_{x}(t)\eta^{1}(t)dW(t),\
t\in[0,l],\\\ \eta^{1}(0)&=1,\ \eta^{1}(t)=0,\
t\in[-l,0).\end{aligned}\right.$ (3.11)

Obviously,

$\left\\{\begin{aligned}
\eta^{1}(t)&=\exp\Big{\\{}-\int_{0}^{t}\sigma_{x}(s)dW(s)-\int_{0}^{t}(b_{x}(s)-\frac{1}{2}|\sigma_{x}(s)|^{2})ds\Big{\\}},\
t\in[0,l],\\\ \eta^{1}(t)&=0,\ t\in[-l,0).\end{aligned}\right.$ (3.12)

We define
$\rho^{1}(t)=(\eta^{1}(t))^{-1}=\exp\Big{\\{}\int_{0}^{t}\sigma_{x}(s)dW(s)+\int_{0}^{t}(b_{x}(s)-\frac{1}{2}|\sigma_{x}(s)|^{2})ds\Big{\\}},\
t\in[0,l]$ and $\rho^{1}(t)=0,\ t\in[-l,0).$

Thanks to the boundness of $b_{x},\ \sigma_{x}$, one can check that for
$\beta\geq 1$, there exists a positive constant $C_{\beta}$ such that

$E\Big{[}\sup_{t\in[-l,l]}(|\rho^{1}(t)|^{\beta}+|\eta^{1}(t)|^{\beta})\Big{]}\leq
C_{\beta}.$ (3.13)

Applying Itô’s formula to $\eta^{1}(t)X^{1,\varepsilon}(t)$ over $[0,l]$, we
have, for $t\in[0,l]$,

$\displaystyle X^{1,\varepsilon}(t)=\rho^{1}(t)\Delta_{1}(t)+\Delta_{2}(t),$
(3.14)

where

$\displaystyle\Delta_{1}(t)$
$\displaystyle:=\int_{0}^{t}\eta^{1}(s)(\widehat{E}[\widehat{\sigma}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]+\Delta\sigma(s)\mathbbm{1}_{E_{\varepsilon}}(s))dW(s),$
$\displaystyle\Delta_{2}(t)$
$\displaystyle:=\rho^{1}(t)\int_{0}^{t}\eta^{1}(s)(\widehat{E}[\widehat{b}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]+\Delta
b(s)\mathbbm{1}_{E_{\varepsilon}}(s))ds$
$\displaystyle\quad-\rho^{1}(t)\int_{0}^{t}\eta^{1}(s)\sigma_{x}(s)(\widehat{E}[\widehat{\sigma}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]+\Delta\sigma(s)\mathbbm{1}_{E_{\varepsilon}}(s))ds\Big{\\}}.$

Therefore,

$\displaystyle\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{X}^{1,\varepsilon}(t)]$
$\displaystyle=\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)\overline{\Delta}_{1}(t)]+\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\Delta}_{2}(t)]:=\Theta_{1}(t,\varepsilon)+\Theta_{2}(t,\varepsilon).$

Next we shall analyse $\Theta_{1}(t,\varepsilon)$ and
$\Theta_{2}(t,\varepsilon)$ one by one.

First, for each $\omega\in\Omega$, we consider the process
$\Psi_{1}^{\omega}(\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t):=\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)$.
For each given $t\in[0,l]$, notice

$\displaystyle\overline{E}[|\Psi_{1}^{\omega}(\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)|^{2}]\leq
C\overline{E}[\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)|^{2}]$
$\displaystyle\leq
C\Big{\\{}\overline{E}[|\Psi_{2}(\overline{\omega},t)|^{4}]\Big{\\}}^{\frac{1}{2}}\Big{\\{}\overline{E}[|\overline{\rho}^{1}(t)|^{4}]\Big{\\}}^{\frac{1}{2}}\leq
C,$

and recall $\mathbb{F}=\mathcal{F}^{*}\vee\mathbb{F}^{B}$, then from Itô’s
Martingale Representation Theorem we know that there exists a unique
$\overline{\gamma}^{1}_{t}(\cdot)\in\mathcal{H}^{2}_{\overline{\mathbb{F}}}(0,t)$
such that

$\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)=\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)]+\int_{0}^{t}\overline{\gamma}^{1}_{t}(s)d\overline{W}_{s},\
t\in[0,l].$ (3.15)

Moreover, $(\overline{\gamma}^{1}_{t})_{t\in[0,l]}$ enjoys the following
property: For $\beta\geq 1$, there exists a $C_{\beta}>0$, such that

$\overline{E}[(\int_{0}^{t}|\overline{\gamma}^{1}_{t}(s)|^{2}ds)^{\frac{\beta}{2}}]\leq
C_{\beta}.$ (3.16)

Indeed, for $\beta>1$, with the help of the Burkholder-Davis-Gundy, Doob’s
maximal inequality and Hölder inequalities, it follows

$\displaystyle\overline{E}[(\int_{0}^{t}|\overline{\gamma}^{1}_{t}(s)|^{2}ds)^{\frac{\beta}{2}}]$
$\displaystyle\leq
C_{\beta}\overline{E}\Big{[}\sup_{s\in[0,t]}|\int_{0}^{s}\overline{\gamma}^{1}_{t}(r)d\overline{W}(r)|^{\beta}\Big{]}$
(3.17) $\displaystyle\leq
C_{\beta}(\frac{\beta}{\beta-1})^{\beta}\overline{E}[|\int_{0}^{t}\overline{\gamma}^{1}_{t}(r)d\overline{W}(r)|^{\beta}]$
$\displaystyle\leq
C_{\beta}\overline{E}[|\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)-\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)]|^{\beta}]$
$\displaystyle\leq
C_{\beta}\overline{E}[|\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)|^{\beta}]\leq
C_{\beta}.$

For $\beta=1$, (3.16) can be obtained by Hölder inequality.

Let us now calculate $\Theta_{1}(t,\varepsilon)$. Clearly, according to (3.15)
one has

$\displaystyle\Theta_{1}(t,\varepsilon)$
$\displaystyle=\overline{E}[\int_{0}^{t}\overline{\gamma}^{1}_{t}(s)\overline{\eta}^{1}(s)\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]ds]+\overline{E}[\int_{0}^{t}\overline{\gamma}^{1}_{t}(s)\overline{\eta}^{1}(s)\Delta\overline{\sigma}(s)\mathbbm{1}_{E_{\varepsilon}}(s)ds]$
(3.18)
$\displaystyle:=\Theta_{1,1}(t,\varepsilon)+\Theta_{1,2}(t,\varepsilon).$

On the one hand, Hölder inequality allows to show

$\displaystyle|\Theta_{1,1}(t,\varepsilon)|^{2}$
$\displaystyle\leq\overline{E}\Big{[}\sup_{s\in[0,l]}|\overline{\eta}^{1}(s)|^{2}\int_{0}^{t}|\overline{\gamma}^{1}_{t}(s)|^{2}ds\int_{0}^{t}|\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{2}ds\Big{]}$
(3.19) $\displaystyle\leq
C\Big{\\{}\overline{E}\Big{[}\int_{0}^{t}|\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}ds\Big{]}\Big{\\}}^{\frac{1}{2}}\cdot\Big{\\{}E[\sup_{s\in[0,l]}|\overline{\eta}^{1}(s)|^{8}]\Big{\\}}^{\frac{1}{4}}\cdot$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\Big{\\{}\overline{E}[(\int_{0}^{t}|\overline{\gamma}^{1}_{t}(s)|^{2}ds)^{4}]\Big{\\}}^{\frac{1}{4}}$
$\displaystyle\leq
C\Big{\\{}\overline{E}\Big{[}\int_{0}^{t}|\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}ds\Big{]}\Big{\\}}^{\frac{1}{2}}.$

On the other hand, from Hölder inequality, again, it follows

$\displaystyle|\Theta_{1,2}(t,\varepsilon)|^{2}$
$\displaystyle=|\overline{E}[\int_{0}^{t}\overline{\gamma}^{1}_{t}(s)\overline{\eta}^{1}(s)\Delta\overline{\sigma}(s)\mathbbm{1}_{E_{\varepsilon}}(s)ds]|^{2}$
(3.20) $\displaystyle\leq
C\overline{E}\Big{[}\sup_{s\in[0,l]}|\overline{\eta}^{1}(s)|^{2}\Big{(}\int_{0}^{t}|\overline{\gamma}^{1}_{t}(s)|\cdot|\overline{v}(s)-\overline{u}^{*}(s)|\mathbbm{1}_{E_{\varepsilon}}(s)\Big{)}^{2}\Big{]}$
$\displaystyle\leq
C\varepsilon\overline{E}\Big{[}\sup_{s\in[0,l]}|\overline{\eta}^{1}(s)|^{2}\cdot\sup_{s\in[0,l]}|\overline{v}(s)-\overline{u}^{*}(s)|^{2}\cdot\int_{0}^{t}|\overline{\gamma}^{1}_{t}(s)|^{2}\mathbbm{1}_{E_{\varepsilon}}(s)ds\Big{]}$
$\displaystyle\leq
C\varepsilon\Big{\\{}\overline{E}[(\int_{0}^{t}|\overline{\gamma}^{1}_{t}(s)|^{2}\mathbbm{1}_{E_{\varepsilon}}(s)ds)^{2}]\Big{\\}}^{\frac{1}{2}}\Big{\\{}\overline{E}[\sup_{s\in[0,l]}|\overline{\eta}^{1}(s)|^{8}]\Big{\\}}^{\frac{1}{4}}\Big{\\{}\overline{E}[\sup_{s\in[0,l]}|\overline{v}(s)-\overline{u}^{*}(s)|^{8}]\Big{\\}}^{\frac{1}{4}}$
$\displaystyle\leq
C\varepsilon\Big{\\{}\overline{E}[(\int_{0}^{t}|\overline{\gamma}^{1}_{t}(s)|^{2}\mathbbm{1}_{E_{\varepsilon}}(s)ds)^{2}]\Big{\\}}^{\frac{1}{2}}.$

Combining (3.18), (3.19), (3.20), we obtain

$\displaystyle|\Theta_{1}(t,\varepsilon)|^{4}\leq
C\overline{E}[\int_{0}^{t}|\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}ds]+C\varepsilon^{2}\overline{E}[(\int_{0}^{t}|\overline{\gamma}^{1}_{t}(s)|^{2}\mathbbm{1}_{E_{\varepsilon}}(s)ds)^{2}].$
(3.21)

Hence, for $0\leq r\leq l$,

$\displaystyle\int_{0}^{r}E[|\Theta_{1}(t,\varepsilon)|^{4}]dt$
$\displaystyle\leq
C\int_{0}^{r}\overline{E}[\int_{0}^{t}|\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}ds]dt+C\varepsilon^{2}E\overline{E}[\int_{0}^{T}(\int_{0}^{t}|\overline{\gamma}^{1}_{t}(s)|^{2}\mathbbm{1}_{E_{\varepsilon}}(s)ds)^{2}dt]$
(3.22) $\displaystyle\leq
C\int_{0}^{r}\int_{0}^{t}\overline{E}[|\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}]dsdt+\varepsilon^{2}\varrho_{1}(\varepsilon),$

where
$\varrho_{1}(\varepsilon):=CE\overline{E}[\int_{0}^{T}(\int_{0}^{t}|\gamma^{1}_{t}(s)|^{2}\mathbbm{1}_{E_{\varepsilon}}(s)ds)^{2}dt].$
Dominated Convergence Theorem allows to show
$\varrho_{1}(\varepsilon)\rightarrow 0$, as $\varepsilon\rightarrow 0$.

Let us now focus on $\Theta_{2}(t,\varepsilon)$. From the assumptions on
$\Psi_{1}$, $\Psi_{2}$ and (3.13) , it follows

$\displaystyle|\Theta_{2}(t,\varepsilon)|$ $\displaystyle\leq
C\overline{E}\Big{[}|\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)\int_{0}^{t}\overline{\eta}^{1}(s)\widehat{E}[\overline{\widehat{b}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]ds|$
$\displaystyle\quad+|\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)\int_{0}^{t}\overline{\eta}^{1}(s)\overline{\sigma}_{x}(s)\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]ds|$
$\displaystyle\quad+|\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)\int_{0}^{t}\overline{\eta}^{1}(s)\Delta\overline{b}(s)\mathbbm{1}_{E_{\varepsilon}}(s)ds|$
$\displaystyle\quad+|\Psi_{2}(\overline{\omega},t)\overline{\rho}^{1}(t)\int_{0}^{t}\overline{\eta}^{1}(s)\sigma_{x}(s)\Delta\overline{\sigma}(s)\mathbbm{1}_{E_{\varepsilon}}(s)ds|\Big{]}$
$\displaystyle\leq
C\Big{\\{}\overline{E}[\int_{0}^{t}|\widehat{E}[\overline{\widehat{b}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{2}ds]\Big{\\}}^{\frac{1}{2}}+C\Big{\\{}\overline{E}[\int_{0}^{t}|\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{2}ds]\Big{\\}}^{\frac{1}{2}}+C\varepsilon,$

which implies

$|\Theta_{2}(t,\varepsilon)|^{4}\leq
C\varepsilon^{4}+C\int_{0}^{t}\overline{E}[|\widehat{E}[\overline{\widehat{b}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}+|\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}]ds.$

Hence,

$\displaystyle\int_{0}^{r}E[|\Theta_{2}(t,\varepsilon)|^{4}]dt\leq
C\varepsilon^{4}+C\int_{0}^{r}\int_{0}^{t}\overline{E}[|\widehat{E}[\overline{\widehat{b}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}+|\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}]dsdt.$
(3.23)

According to (3.22), (3.23), we have, for $r\in[0,l]$,

$\displaystyle\int_{0}^{r}E|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{X}^{1,\varepsilon}(t)]|^{4}dt$
(3.24)
$\displaystyle\leq\varepsilon^{2}\rho(\varepsilon)+C\int_{0}^{r}\int_{0}^{t}E\Big{[}|\widehat{E}[\widehat{b}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}+|\widehat{E}[\widehat{\sigma}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}\Big{]}dsdt.$

Taking
$(\overline{\Omega},\overline{\mathcal{F}},\overline{P})=(\widehat{\Omega},\widehat{\mathcal{F}},\widehat{P})$,
and set
$\Psi_{1}(\omega,\widehat{\omega},t)=\widehat{b}_{\nu}(t),\widehat{\sigma}_{\nu}(t),\
\Psi_{2}(\widehat{\omega},t)=1,$ respectively, the Gronwall inequality allows
to show

$\displaystyle\int_{0}^{l}E\Big{[}|\widehat{E}[\widehat{b}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}+|\widehat{E}[\widehat{\sigma}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}\Big{]}ds\leq\varepsilon^{2}\rho(\varepsilon).$
(3.25)

Case 3 If $l\leq t\leq T$, we have known from (3.14) the specific expression
of $X^{1,\varepsilon}(t-l)$, i.e., for $t\in[l,T]$,

$\displaystyle X^{1,\varepsilon}(t-l)$
$\displaystyle=\rho^{1}(t-l)\int_{l}^{t}\eta^{1}(s-l)(\widehat{E}[\widehat{\sigma}_{\nu}(s-l)\widehat{X}^{1,\varepsilon}(s-l)]+\Delta\sigma(s-l)\mathbbm{1}_{E_{\varepsilon}}(s-l))dW(s)$
(3.26)
$\displaystyle\quad+\rho^{1}(t-l)\int_{l}^{t}\eta^{1}(s-l)(\widehat{E}[\widehat{b}_{\nu}(s-l)\widehat{X}^{1,\varepsilon}(s-l)]+\Delta
b(s-l)\mathbbm{1}_{E_{\varepsilon}}(s-l))ds$
$\displaystyle\quad-\rho^{1}(t-l)\int_{l}^{t}\eta^{1}(s-l)\sigma_{x}(s-l)(\widehat{E}[\widehat{\sigma}_{\nu}(s-l)\widehat{X}^{1,\varepsilon}(s-l)]+\Delta\sigma(s-l)\mathbbm{1}_{E_{\varepsilon}}(s-l))ds.$

In this case, $X^{1,\varepsilon}$ is of the form:

$\displaystyle dX^{1,\varepsilon}(t)$
$\displaystyle=\Big{\\{}b_{x}(t)X^{1,\varepsilon}(t)+b_{x^{\prime}}(t)X^{1,\varepsilon}(t-l)+\widehat{E}[\widehat{b}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]+\Delta
b(t)\mathbbm{1}_{E_{\varepsilon}}(t)\Big{\\}}dt$ (3.27)
$\displaystyle\quad+\Big{\\{}\sigma_{x}(t)X^{1,\varepsilon}(t)+\sigma_{x^{\prime}}(t)X^{1,\varepsilon}(t-l)+\widehat{E}[\widehat{\sigma}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\
+\Delta\sigma(t)\mathbbm{1}_{E_{\varepsilon}}(t)\Big{\\}}dW(t),\ t\in(l,T],$

where $X^{1,\varepsilon}(t-l)$ is given in (3.26).

We define

$\left\\{\begin{aligned}
d\eta^{2}(t)&=-(b_{x}(t)-|\sigma_{x}(t)|^{2})dt-\sigma_{x}(t)dW(t),\
t\in[l,T],\\\ \eta^{2}(l)&=1,\end{aligned}\right.$ (3.28)

and $\rho^{2}(t)=(\eta^{2}(t))^{-1},\ t\in[l,T]$.
Obviously, for $\beta\geq 1$,

$E\Big{[}\sup_{t\in[l,T]}(|\rho^{2}(t)|^{\beta}+|\eta^{2}(t)|^{\beta})\Big{]}\leq
C_{\beta}.$ (3.29)

Applying Itô’s formula to $\eta^{2}(s)X^{1,\varepsilon}(s)$ over $[l,t]$, we
have

$\displaystyle
X^{1,\varepsilon}(t)=J_{1}(t,\varepsilon)+J_{2}(t,\varepsilon)+J_{3}(t,\varepsilon)+J_{4}(t,\varepsilon),$
(3.30)

where

$\displaystyle J_{1}(t,\varepsilon)$
$\displaystyle=\rho^{2}(t)\int_{l}^{t}\eta^{2}(s)\sigma_{x^{\prime}}(s)X^{1,\varepsilon}(s-l)ds,$
$\displaystyle J_{2}(t,\varepsilon)$
$\displaystyle=\rho^{2}(t)\int_{l}^{t}\eta^{2}(s)b_{x^{\prime}}(s)X^{1,\varepsilon}(s-l)dW(s),\quad
J_{3}(t,\varepsilon)=\rho^{2}(t)X^{1,\varepsilon}(l),$ $\displaystyle
J_{4}(t,\varepsilon)$
$\displaystyle=\rho^{2}(t)\int_{l}^{t}(\eta^{2}(s)E[\widehat{\sigma}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]+\eta^{2}(s)\Delta\sigma(s)\mathbbm{1}_{E_{\varepsilon}}(s))dW(s)$
$\displaystyle\quad+\rho^{2}(t)\int_{l}^{t}(\eta^{2}(s)E[\widehat{b}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]+\eta^{2}(s)\Delta
b(s)\mathbbm{1}_{E_{\varepsilon}}(s))ds$
$\displaystyle\quad-\rho^{2}(t)\int_{l}^{t}(\eta^{2}(s)\sigma_{x}(s)E[\widehat{\sigma}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]+\eta^{2}(s)\sigma_{x}(s)\Delta\sigma(s)\mathbbm{1}_{E_{\varepsilon}}(s))ds.$

In what follows, we mainly analyse $J_{1}(t,\varepsilon)$, since
$J_{2}(t,\varepsilon),J_{3}(t,\varepsilon)$ can be estimated with the similar
argument. As for $J_{4}(t,\varepsilon)$, it can be calculated following the
method used in Case 2.

To begin with, insert (3.26) into $J_{1}(t,\varepsilon)$ we can get

$\displaystyle
J_{1}(t,\varepsilon)=J_{1,1}(t,\varepsilon)+J_{1,2}(t,\varepsilon)+J_{1,3}(t,\varepsilon),$
(3.31)

where

$\displaystyle J_{1,1}(t,\varepsilon)$
$\displaystyle=\rho^{2}(t)\int_{l}^{t}\int_{0}^{s-l}\eta^{2}(s)\sigma_{x^{\prime}}(s)\rho^{1}(s-l)\eta^{1}(r)(\widehat{E}[\widehat{\sigma}_{\nu}(r)\widehat{X}^{1,\varepsilon}(r)]+\Delta\sigma(r)\mathbbm{1}_{E_{\varepsilon}}(r))dW(r)ds,$
$\displaystyle J_{1,2}(t,\varepsilon)$
$\displaystyle=\rho^{2}(t)\int_{l}^{t}\int_{0}^{s-l}\eta^{2}(s)\sigma_{x^{\prime}}(s)\rho^{1}(s-l)\eta^{1}(r)(\widehat{E}[\widehat{b}_{\nu}(r)\widehat{X}^{1,\varepsilon}(r)]+\Delta
b(r)\mathbbm{1}_{E_{\varepsilon}}(r))drds,$ $\displaystyle
J_{1,3}(t,\varepsilon)$
$\displaystyle=-\rho^{2}(t)\int_{l}^{t}\int_{0}^{s-l}\eta^{2}(s)\sigma_{x^{\prime}}(s)\rho^{1}(s-l)\eta^{1}(r)\sigma_{x}(r)(\widehat{E}[\widehat{\sigma}_{\nu}(r)\widehat{X}^{1,\varepsilon}(r)]+\Delta\sigma(r)\mathbbm{1}_{E_{\varepsilon}}(r))drds.$

Observe $J_{1,1}(t,\varepsilon)$, exchanging the order of integration, it
yields

$\displaystyle J_{1,1}(t,\varepsilon)$
$\displaystyle=\rho^{2}(t)\int_{l}^{t}\eta^{1}(r-l)(\widehat{E}[\widehat{\sigma}_{\nu}(r-l)\widehat{X}^{1,\varepsilon}(r-l)]+\Delta\sigma(r-l)\mathbbm{1}_{E_{\varepsilon}}(r-l))\cdot$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\int_{r}^{t}\eta^{2}(s)\sigma_{x^{\prime}}(s)\rho^{1}(s-l)dsdW(r).$

For $l\leq u\leq T$, set
$\varphi(u)=\int^{u}_{l}\eta^{2}(s)\sigma_{x^{\prime}}(s)\rho^{1}(s-l)ds$.
Clearly, for $\beta\geq 1$,

$E[\sup_{l\leq u\leq T}|\varphi(u)|^{\beta}]\leq C_{\beta},$ (3.32)

where $C_{\beta}$ is a constant depending on $\beta$.

In addition, utilizing the similar argument in Case 2, we know that for each
$t\in[l,T]$, there exists a unique
$\overline{\gamma}^{2}_{t}(\cdot)\in\mathcal{H}^{2}_{\overline{\mathbb{F}}}(l,T)$
with the property: for any $\beta\geq 1$,

$\overline{E}[(\int_{l}^{T}|\overline{\gamma}^{2}_{t}(s)|^{2}ds)^{\frac{\beta}{2}}]\leq
C_{\beta},$ (3.33)

such that

$\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\rho}^{2}(t)=\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\rho}^{2}(t)]+\int_{l}^{t}\overline{\gamma}^{2}_{t}(s)d\overline{W}_{s},\
t\in[l,T].$ (3.34)

Consequence, from (3.13), (3.32), (3.33), (3.34), one can check that for
$l\leq u\leq T$,

$\displaystyle\int_{l}^{u}E|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{J}_{1,1}(t,\varepsilon)]|^{4}dt$
(3.35)
$\displaystyle=\int_{l}^{u}E\big{|}\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{\rho}^{2}(t)\int_{l}^{t}\overline{\eta}^{1}(r-l)(\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(r-l)\widehat{X}^{1,\varepsilon}(r-l)]+\Delta\overline{\sigma}(r-l)\mathbbm{1}_{E_{\varepsilon}}(r-l))\cdot$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad(\overline{\varphi}(t)-\overline{\varphi}(r))d\overline{W}(r)]\big{|}^{4}dt$
$\displaystyle=\int_{l}^{u}E|\overline{E}[\int_{l}^{t}\overline{\gamma}^{2}_{t}(r)\overline{\eta}^{1}(r-l)(\widehat{E}[\overline{\widehat{\sigma}}_{\nu}(r-l)\widehat{X}^{1,\varepsilon}(r-l)]+\Delta\overline{\sigma}(r-l)\mathbbm{1}_{E_{\varepsilon}}(r-l))\cdot(\overline{\varphi}(t)-\overline{\varphi}(r))dr]|^{4}dt$
$\displaystyle\leq
C\int_{l}^{u}\int_{l}^{t}E[|\widehat{E}[\widehat{\sigma}_{\nu}(r-l)\widehat{X}^{1,\varepsilon}(r-l)]|^{4}]drdt+C\varepsilon^{2}E\overline{E}[\int_{l}^{T}(\int_{l}^{t}|\overline{\gamma}^{2}_{t}(r)|^{2}\mathbbm{1}_{E_{\varepsilon}}(r-l)dr)^{2}dt]$
$\displaystyle\leq\varepsilon^{2}\rho(\varepsilon)+\varepsilon^{2}\varrho_{2}(\varepsilon),$

where
$\varrho_{2}(\varepsilon):=E\overline{E}[\int_{l}^{T}(\int_{l}^{t}|\overline{\gamma}^{2}_{t}(r)|^{2}\mathbbm{1}_{E_{\varepsilon}}(r-l)dr)^{2}dt]\rightarrow
0,$ as $\varepsilon\rightarrow 0$.
$J_{1,2}$ and $J_{1,3}$ can be estimated with the similar argument as above.
Consequence,

$\displaystyle\int_{l}^{u}E|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{J}_{1,2}(t,\varepsilon)]|^{4}dt+\int_{l}^{u}E|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{J}_{1,3}(t,\varepsilon)]|^{4}dt\leq\varepsilon^{2}\varrho_{3}(\varepsilon),$
(3.36)

where $\varrho_{3}(\varepsilon)\rightarrow 0$ as $\varepsilon\rightarrow 0$.
Thanks to (3.35) and (3.36), we obtain, for $l\leq u\leq T$,

$\displaystyle\int_{l}^{u}E|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{J}_{1}(t,\varepsilon)]|^{4}dt\leq\varepsilon^{2}\rho(\varepsilon).$
(3.37)

Similarly, one also has

$\displaystyle\int_{l}^{u}E|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{J}_{2}(t,\varepsilon)]|^{4}dt+\int_{l}^{u}E|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{J}_{3}(t,\varepsilon)]|^{4}dt\leq\varepsilon^{2}\rho(\varepsilon).$
(3.38)

Last but not least, combining (3.37) with (3.38), following the analyses in
Case 2 we get, for $l\leq u\leq T$,

$\displaystyle\int_{l}^{u}E|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{X}^{1,\varepsilon}(t,\varepsilon)]|^{4}dt$
(3.39)
$\displaystyle\leq\varepsilon^{2}\rho(\varepsilon)+\int_{l}^{u}E|\overline{E}[\Psi_{1}(\omega,\overline{\omega},t)\Psi_{2}(\overline{\omega},t)\overline{J}_{4}(t,\varepsilon)]|^{4}dt$
$\displaystyle\leq\varepsilon^{2}\rho(\varepsilon)+\int_{l}^{u}\int_{l}^{t}E\Big{[}|\widehat{E}[\widehat{b}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}+|\widehat{E}[\widehat{\sigma}_{\nu}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}\Big{]}dsdt.$

Let
$(\overline{\Omega},\overline{\mathcal{F}},\overline{P})=(\widehat{\Omega},\widehat{\mathcal{F}},\widehat{P})$,
and $\Psi_{1}(\omega,\widehat{\omega},t)=\widehat{b}_{\nu}(t)$ (resp.,
$\Psi_{1}(\omega,\widehat{\omega},t)=\widehat{\sigma}_{\nu}(t)$), and
$\Psi_{2}(\widehat{\omega},t)=1$ (resp., $\Psi_{2}(\widehat{\omega},t)=1$),
the the Gronwall inequality implies the desired result. The proof is
completed. $\Box$

###### Remark 3.2

In particular, in Lemma 3.1 we set
$\Psi_{1}(\omega,\overline{\omega},t)=\overline{b}_{\nu}(t),\
\Psi_{2}(\overline{\omega},t)=1$ and
$\Psi_{1}(\omega,\overline{\omega},t)=\overline{\sigma}_{\nu}(t),\
\Psi_{2}(\overline{\omega},t)=1$, separately, then

$E\Big{[}\int_{0}^{T}(|\overline{E}[\overline{b}_{\nu}(t)\overline{X}^{1,\varepsilon}(t)]|^{4}+|\overline{E}[\overline{\sigma}_{\nu}(t)\overline{X}^{1,\varepsilon}(t)]|^{4})dt\Big{]}\leq\varepsilon^{2}\rho(\varepsilon),$
(3.40)

which is just the estimate (4.14) given by Buckdahn, Li and Ma [5].

With the help of Remark 3.2, it is easy to check

###### Proposition 3.3

Let the assumptions (H3.1)-(H3.3) be in force, then for $\beta\geq 1$,

$\displaystyle\mathrm{i)}\
E[\sup_{t\in[-l,T]}|X^{\varepsilon}(t)-X^{*}(t)|^{2\beta}]\leq
C\varepsilon^{\beta},\quad\mathrm{ii)}\
E[\sup_{t\in[-l,T]}|X^{1,\varepsilon}(t)|^{2\beta}]\leq C\varepsilon^{\beta},$
(3.41) $\displaystyle\mathrm{iii)}\
E[\sup_{t\in[-l,T]}|X^{2,\varepsilon}(t)|^{2\beta}]\leq
C\varepsilon^{2\beta},\quad\mathrm{iv)}\
E[\sup_{t\in[-l,T]}|X^{\varepsilon}(t)-X^{*}(t)-X^{1,\varepsilon}(t)|^{2\beta}]\leq
C\varepsilon^{2\beta},$ $\displaystyle\mathrm{v)}\
E[\sup_{t\in[-l,T]}|X^{\varepsilon}(t)-X^{*}(t)-X^{1,\varepsilon}(t)-X^{2,\varepsilon}(t)|^{2}]\leq\varepsilon^{2}\rho(\varepsilon).$

###### Remark 3.4

We can rewrite Proposition 3.3-v) as

$X^{\varepsilon}(t)=X^{*}(t)+X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t)+o(\varepsilon),$
(3.42)

where the convergence is in $L^{2}(\mathcal{F},C[-l,T])$ sense. In fact, we
have a slightly strong result, i.e., we can find a $\alpha_{0}>2$, such that
(3.42) holds true under $L^{\alpha}(\mathcal{F};C[-l,T])$,
$\alpha\in[2,\alpha_{0})$. The reader can refer to [5] for more details.

###  3.2 Adjoint equations

In this subsection two adjoint equations are introduced which are the basic-
materials to apply duality method to study our stochastic maximum principle.
Compared with the classical case, see Hu [18], there are two notable
differences. The first one is the first-order adjoint equation is an
anticipated mean-field BSDE. The second one comes from the second-order
adjoint equation. Due to the delay term, the equation
$(X^{1,\varepsilon})^{2}$ is not sufficient to solve our control problem. It
means that we have to bring in an additional auxiliary equation
$X^{1,\varepsilon}(t)X^{2,\varepsilon}(t)$. Hence, the second-order adjoint
system is a system of equations for matrix valued processes. Let us describe
them in detail.

Denote for $\ell=x,x^{\prime},y,z,$ $\theta=x,x^{\prime},y,z,a_{i}$,
$i,j=1,2$,

$\displaystyle\Xi^{*}(t)=(X^{*}(t),X^{*}(t-l),Y^{*}(t),Z^{*}(t)),\
\Lambda^{*}(t)=(X^{*}(t),Y^{*}(t)),\ f_{\ell}(s)=\frac{\partial
f}{\partial\ell}(s,\Xi^{*}(s),P_{\Lambda^{*}(s)},u^{*}(s)),$ (3.43)
$\displaystyle(\widehat{f}_{\mu_{i}},\widehat{f}_{\mu_{i}\theta})(s)=((\frac{\partial
f}{\partial\mu})_{i},\frac{\partial}{\partial\theta}((\frac{\partial
f}{\partial\mu})_{i}))(s,\Xi^{*}(s),P_{\Lambda^{*}(s)},u^{*}(s);\widehat{\Lambda}^{*}(s)),$
$\displaystyle(\widehat{f}^{*}_{\mu_{i}},\widehat{f}^{*}_{\mu_{i}\theta})(s)=((\frac{\partial
f}{\partial\mu})_{i},\frac{\partial}{\partial\theta}((\frac{\partial
f}{\partial\mu})_{i}))(s,\widehat{\Xi}^{*}(s),P_{\Lambda^{*}(s)},\widehat{u}^{*}(s);\Lambda^{*}(s)),$
$\displaystyle\widetilde{\widehat{f}}_{\mu_{i}\mu_{j}}(s)=(\frac{\partial}{\partial\mu}((\frac{\partial
f}{\partial\mu})_{i}))_{j}(s,\Xi^{*}(s),P_{\Lambda^{*}(s)},u^{*}(s);\widehat{\Lambda}^{*}(s),\widetilde{\Lambda}^{*}(s)).$

The first order adjoint equation in our case is the following anticipated
mean-field BSDE:

$\left\\{\begin{aligned} -dp(s)&=F(s)ds-q(s)dW(s),s\in[0,T],\\\
p(T)&=\Phi_{x}(T)+\widehat{E}[\widehat{\Phi}^{*}_{\nu}(T)],\ p(t)=0,\
t\in(T,T+l],\ q(t)=0,\ t\in[T,T+l],\end{aligned}\right.$ (3.44)

where
$\widehat{\Phi}^{*}_{\nu}(T)=\frac{\partial\Phi}{\partial\nu}(\widehat{X}^{*}_{T},P_{X^{*}_{T}};X^{*}_{T})$
and

$\displaystyle F(s)$
$\displaystyle=p(s)\Big{(}f_{y}(s)+\widehat{E}[\widehat{f}^{*}_{\mu_{2}}(s)]+f_{z}(s)\sigma_{x}(s)+b_{x}(s)\Big{)}+\widehat{E}\Big{[}\widehat{p}(s)\Big{(}\widehat{f}_{z}(s)\widehat{\sigma}^{*}_{\nu}(s)+\widehat{b}^{*}_{\nu}(s)\Big{)}\Big{]}$
$\displaystyle\quad+E^{\mathcal{F}_{s}}[p(s+l)(f_{z}(s+l)\sigma_{x^{\prime}}(s+l)+b_{x^{\prime}}(s+l))]$
$\displaystyle\quad+q(s)\Big{(}f_{z}(s)+\sigma_{x}(s)\Big{)}+\widehat{E}[\widehat{q}(s)\widehat{\sigma}^{*}_{\nu}(s)]+E^{\mathcal{F}_{s}}[q(s+l)\sigma_{x^{\prime}}(s+l)]$
$\displaystyle\quad+f_{x}(s)+\widehat{E}[\widehat{f}^{*}_{\mu_{1}}(s)]+E^{\mathcal{F}_{s}}[f_{x^{\prime}}(s+l)].$

According to Theorem 3.1 in Guo, Xiong and Zheng [16] under the assumptions
(H3.1)-(H3.2) the anticipated mean-field BSDE (3.44) admits a unique solution
$(p,q)\in\mathcal{S}_{\mathbb{F}}^{2}(0,T+l)\times\mathcal{H}_{\mathbb{F}}^{2}(0,T+l)$,
and moreover, from the boundness of the derivatives of the coefficients it
follows that, for $\beta\geq 2$, there exists a constant $C_{\beta}$ depending
on $\beta$ such that

$E\Big{[}\sup_{t\in[0,T+l]}|p(t)|^{\beta}+(\int_{0}^{T+l}|q(t)|^{2}dt)^{\frac{\beta}{2}}\Big{]}\leq
C_{\beta}.$ (3.45)

From Lemma 3.1, we have the following estimates.

###### Corollary 3.5

Let the assumptions (H3.1)-(H3.3) be in force, and set for
$\ell=x,x^{\prime},y,z,$

$\displaystyle\widehat{M}_{1}(s)$
$\displaystyle:=(\widehat{f}_{\mu_{1}},\widehat{f}_{\mu_{2}})(s),\
\widehat{M}_{2}(s):=(\widehat{f}_{\ell\mu_{1}},\widehat{f}_{\ell\mu_{2}})(s),\
\widetilde{\widehat{M}}_{3}(s):=(\widetilde{\widehat{f}}_{\mu_{1}\mu_{1}},\widetilde{\widehat{f}}_{\mu_{1}\mu_{2}},\widetilde{\widehat{f}}_{\mu_{2}\mu_{2}})(s).$

Let $X^{1,\varepsilon}$ and $p$ be the solutions of (3.5) and (3.44),
separately, then

$\displaystyle\mathrm{i)}\
E\Big{[}\int_{0}^{T}\Big{(}|\widehat{E}[\widehat{M}_{1}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}+|\widehat{E}[\widehat{M}_{2}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}\Big{)}ds\Big{]}\leq\varepsilon^{2}\rho(\varepsilon);$
(3.46) $\displaystyle\mathrm{ii)}\
E\Big{[}\int_{0}^{T}\Big{(}|\widehat{E}[\widehat{M}_{1}(s)\widehat{p}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}+|\widehat{E}[\widehat{M}_{2}(s)\widehat{p}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}\Big{)}ds\Big{]}\leq\varepsilon^{2}\rho(\varepsilon);$
$\displaystyle\mathrm{iii)}\
E\widetilde{E}\Big{[}\int_{0}^{T}|\widehat{E}[\widetilde{\widehat{M}}_{3}(s)\widehat{p}(s)\widehat{X}^{1,\varepsilon}(s)]|^{4}ds\Big{]}\leq\varepsilon^{2}\rho(\varepsilon).$

###### Remark 3.6

Estimates (3.40) and (3.46) are powerful tools to calculate the mean-field
terms, especially those involving the derivatives of the coefficients with
respect to the measure and the first-order variation $X^{1,\varepsilon}$,
appearing in the expansion of $X^{\varepsilon}$ and $Y^{\varepsilon}$. As an
example, the reader can refer to the proof of (4.18) and the expansion of
$I_{22}(s)$ in Appendix.

Let us now analyse the second-order adjoint matrix-valued system. We set
$K^{\varepsilon}(t):=(X^{1,\varepsilon}(t))^{2},$
$K_{1}^{\varepsilon}(t):=X^{1,\varepsilon}(t)X^{1,\varepsilon}(t-l)$. First,
applying Itô’s formula to $K^{\varepsilon}(t)$ we have

$\left\\{\begin{aligned}
dK^{\varepsilon}(t)&=\Big{\\{}\Big{(}2b_{x}(t)+(\sigma_{x}(t))^{2}\Big{)}K^{\varepsilon}(t)+\Big{(}2b_{x^{\prime}}(t)+2\sigma_{x}(t)\sigma_{x^{\prime}}(t)\Big{)}K_{1}^{\varepsilon}(t)+\Big{(}\sigma_{x^{\prime}}(t)\Big{)}^{2}K^{\varepsilon}(t-l)\\\
&\quad+(\Delta\sigma(t))^{2}\mathbbm{1}_{E_{\varepsilon}}(t)\Big{\\}}dt+\Big{\\{}2\sigma_{x}(t)K^{\varepsilon}(t)+2\sigma_{x^{\prime}}(t)K^{\varepsilon}_{1}(t)\Big{\\}}dW_{t}+L_{1}(t)dt+L_{2}(t)dW_{t},\\\
K^{\varepsilon}(t)&=0,\ t\in[-l,0],\end{aligned}\right.$ (3.47)

where

$\displaystyle L_{1}(t)$
$\displaystyle=\mathbbm{1}_{E_{\varepsilon}}(t)\Big{\\{}2\Delta
b(t)X^{1,\varepsilon}(t)+2\sigma_{x}(t)\Delta\sigma(t)X^{1,\varepsilon}(t)+2\sigma_{x^{\prime}}(t)\Delta\sigma(t)X^{1,\varepsilon}(t-l)\Big{\\}}$
(3.48)
$\displaystyle\quad+\Big{\\{}2\widehat{E}[\widehat{b}^{*}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]X^{1,\varepsilon}(t)+(\widehat{E}[\widehat{\sigma}^{*}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)])^{2}+2\sigma_{x}(t)X^{1,\varepsilon}(t)\widehat{E}[\widehat{\sigma}^{*}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]$
$\displaystyle\quad+2\sigma_{x^{\prime}}(t)X^{1,\varepsilon}(t-l)\widehat{E}[\widehat{\sigma}^{*}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]+2\Delta\sigma(t)\widehat{E}[\widehat{\sigma}^{*}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]\mathbbm{1}_{E_{\varepsilon}}(t)\Big{\\}},$
$\displaystyle L_{2}(t)$
$\displaystyle=2\Delta\sigma(t)X^{1,\varepsilon}(t)\mathbbm{1}_{E_{\varepsilon}}(t)+\widehat{E}[\widehat{\sigma}^{*}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]X^{1,\varepsilon}(t).$

From Remark 3.2, the continuous property of $f$ with respect to $v$, the
boundness of $b_{\nu},\ \sigma_{\nu}$ it yields

$E\Big{[}\Big{(}\int_{0}^{T}|L_{1}(t)|+|L_{2}(t)|dt\Big{)}^{2}\Big{]}\leq\varepsilon^{2}\rho(\varepsilon)$
(3.49)

(see Appendix for more details).

From the structure of the equation of $K^{\varepsilon}(t)$, we know that
$K^{\varepsilon}(t)$ depends not only on the quadratic terms
$K^{\varepsilon}(t),K^{\varepsilon}(t-l)$, but also on the mixed term
$K_{1}^{\varepsilon}(t)$. That means that it is impossible that
$K^{\varepsilon}(t)$ solves an equation in a closed form of
$K^{\varepsilon}(t),K^{\varepsilon}(t-l)$. So we are ready to consider an
auxiliary equation $K_{1}^{\varepsilon}(t)$. Let us first introduce the
equation of $X^{1,\varepsilon}(t-l)$. Notice $X^{1,\varepsilon}$ is not null
only after positive times, then

$\left\\{\begin{aligned}
dX^{1,\varepsilon}(t-l)&=\Big{\\{}b_{x}(t-l)X^{1,\varepsilon}(t-l)+b_{x^{\prime}}(t-l)X^{1,\varepsilon}(t-2l)+\widehat{E}[\widehat{b}^{*}_{\nu}(t-l)\widehat{X}^{1,\varepsilon}(t-l)]\Big{\\}}dt\\\
&\quad+\Big{\\{}\sigma_{x}(t-l)X^{1,\varepsilon}(t-l)+\sigma_{x^{\prime}}(t-l)X^{1,\varepsilon}(t-2l)+\widehat{E}[\widehat{\sigma}^{*}_{\nu}(t-l)\widehat{X}^{1,\varepsilon}(t-l)]\Big{\\}}dW_{t}\\\
&\quad+\Big{\\{}\Delta
b(t-l)\mathbbm{1}_{E_{\varepsilon}}(t-l)\Big{\\}}dt+\Big{\\{}\Delta\sigma(t-l)\mathbbm{1}_{E_{\varepsilon}}(t-l)\Big{\\}}dW_{t},\
t\in[l,T],\\\ X^{1,\varepsilon}(t-l)&=0,\ t\in[-l,l].\end{aligned}\right.$
(3.50)

In our case, due to $t<T\leq 2l$, hence $X^{1,\varepsilon}(t-2l)\equiv 0,\
t\in[0,T].$ Consequently,

$\left\\{\begin{aligned}
dK^{\varepsilon}_{1}(t)&=\Big{\\{}\Big{(}b_{x}(t)+b_{x}(t-l)+\sigma_{x}(t)\sigma_{x}(t-l)\Big{)}K^{\varepsilon}_{1}(t)+\Big{(}b_{x^{\prime}}(t)+\sigma_{x^{\prime}}(t)\sigma_{x}(t-l)\Big{)}K^{\varepsilon}(t-l)\Big{\\}}dt\\\
&\quad+\Big{\\{}\Big{(}\sigma_{x}(t)+\sigma_{x}(t-l)\Big{)}K^{\varepsilon}_{1}(t)+\sigma_{x^{\prime}}(t)K^{\varepsilon}(t-l)\Big{\\}}dW_{t}+L_{3}(t)dt+L_{4}(t)dW_{t},\\\
&\quad+\\{\Delta\sigma(t)\Delta\sigma(t-l)\mathbbm{1}_{E_{\varepsilon}}(t-l)\mathbbm{1}_{E_{\varepsilon}}(t)\\}dt,\
t\in[l,T],\\\ K^{\varepsilon}_{1}(t)&=0,\ t\in[-l,l],\end{aligned}\right.$
(3.51)

where

$\displaystyle L_{3}(t)$ $\displaystyle=X^{1,\varepsilon}(t)\Delta
b(t-l)\mathbbm{1}_{E_{\varepsilon}}(t-l)+\Delta
b(t)X^{1,\varepsilon}(t-l)\mathbbm{1}_{E_{\varepsilon}}(t)+\sigma_{x}(t)\Delta\sigma(t-l)X^{1,\varepsilon}(t)\mathbbm{1}_{E_{\varepsilon}}(t-l)$
$\displaystyle\quad+\sigma_{x^{\prime}}(t)\Delta\sigma(t-l)X^{1,\varepsilon}(t-l)\mathbbm{1}_{E_{\varepsilon}}(t-l)+\sigma_{x}(t-l)\Delta\sigma(t)X^{1,\varepsilon}(t-l)\mathbbm{1}_{E_{\varepsilon}}(t)$
$\displaystyle\quad+\Delta\sigma(t-l)\mathbbm{1}_{E_{\varepsilon}}(t-l)\widehat{E}[\widehat{\sigma}^{*}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]+\Delta\sigma(t)\mathbbm{1}_{E_{\varepsilon}}(t)\widehat{E}[\widehat{\sigma}^{*}_{\nu}(t-l)\widehat{X}^{1,\varepsilon}(t-l)]$
$\displaystyle\quad+X^{1,\varepsilon}(t-l)\widehat{E}[\widehat{b}^{*}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]+\widehat{E}[\widehat{\sigma}^{*}_{\nu}(t-l)\widehat{X}^{1,\varepsilon}(t-l)](\sigma_{x}(t)X^{1,\varepsilon}(t)+\sigma_{x^{\prime}}(t)X^{1,\varepsilon}(t-l))$
$\displaystyle\quad+\sigma_{x}(t-l)X^{1,\varepsilon}(t-l)\widehat{E}[\widehat{\sigma}^{*}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)]+X^{1,\varepsilon}(t)\widehat{E}[\widehat{b}^{*}_{\nu}(t-l)\widehat{X}^{1,\varepsilon}(t-l)],$
$\displaystyle L_{4}(t)$
$\displaystyle=\Delta\sigma(t-l)X^{1,\varepsilon}(t)\mathbbm{1}_{E_{\varepsilon}}(t-l)+\Delta\sigma(t)X^{1,\varepsilon}(t-l))\mathbbm{1}_{E_{\varepsilon}}(t)$
$\displaystyle\quad+X^{1,\varepsilon}(t)\widehat{E}[\widehat{\sigma}^{*}_{\nu}(t-l)\widehat{X}^{1,\varepsilon}(t-l)]+X^{1,\varepsilon}(t-l)\widehat{E}[\widehat{\sigma}^{*}_{\nu}(t)\widehat{X}^{1,\varepsilon}(t)].$

Note that
$\mathbbm{1}_{E_{\varepsilon}}(t-l)\mathbbm{1}_{E_{\varepsilon}}(t)\equiv 0$
for enough small $\varepsilon$, for example $\varepsilon<\frac{l}{3}$.
Moreover, following the proof of (3.49) (see Appendix), from Remark 3.2 again,
it yields

$E\Big{[}\Big{(}\int_{0}^{T}|L_{3}(t)|+|L_{4}(t)|dt\Big{)}^{2}\Big{]}\leq\varepsilon^{2}\rho(\varepsilon).$
(3.52)

The coupled characteristic of the equations (3.47) and (3.51) inspires us to
consider the following coupled second-order adjoint equation, which is
different to the classical case:

$\left\\{\begin{aligned} -dP(s)&=G(s)ds-Q(s)dW(s)\ s\in[0,T],\\\
P(T)&=\Phi_{xx}(T)+\widehat{E}[\widehat{\Phi}^{*}_{\nu a}(T)],P(t)=0,\
t\in(T,T+l],\ Q(t)=0,\ t\in[T,T+l],\\\ -dP_{1}(s)&=G_{1}(s)ds-Q_{1}(s)dW(s)\
s\in[0,T],\\\ P_{1}(T)&=0,\ t\in[T,T+l],\ Q_{1}(t)=0,\
t\in[T,T+l],\end{aligned}\right.$ (3.53)

where

$\displaystyle G(t)$
$\displaystyle=P(t)\Big{(}f_{y}(t)+\widehat{E}[\widehat{f}^{*}_{\nu_{2}}(t)]+2f_{z}(t)\sigma_{x}(t)+2b_{x}(t)+(\sigma_{x}(t))^{2}\Big{)}+E^{\mathcal{F}_{t}}[P(t+l)(\sigma_{x^{\prime}}(t+l))^{2}]$
(3.54)
$\displaystyle\quad+Q(t)(f_{z}(t)+2\sigma_{x}(t))+E^{\mathcal{F}_{t}}[2P_{1}(t+l)(\sigma_{x^{\prime}}(t+l)+b_{x^{\prime}}(t+l)+\sigma_{x^{\prime}}(t+l)\sigma_{x}(t))]$
$\displaystyle\quad+E^{\mathcal{F}_{t}}[2Q_{1}(t+l)\sigma_{x^{\prime}}(t+l)]+p(t)(b_{xx}(t)+f_{z}(t)\sigma_{xx}(t))+\widehat{E}[\widehat{p}(t)(\widehat{b}^{*}_{\nu
a}(t)+\widehat{f}_{z}(t)\widehat{\sigma}^{*}_{\nu a}(t))]$
$\displaystyle\quad+E^{\mathcal{F}_{t}}[p(t+l)(b_{x^{\prime}x^{\prime}}(t+l)+f_{z}(t+l)\sigma_{x^{\prime}x^{\prime}}(t+l))]+q(t)\sigma_{xx}(t)+\widehat{E}[\widehat{q}(t)\widehat{\sigma}^{*}_{\nu
a}(t)]$
$\displaystyle\quad+E^{\mathcal{F}_{t}}[q(t+l)\sigma_{x^{\prime}x^{\prime}}(t+l)]+UD^{2}fU^{\intercal}+E^{\mathcal{F}_{t}}[f_{x^{\prime}x^{\prime}}(t+l)]+\widehat{E}[\widehat{f}^{*}_{\mu_{1}a_{1}}(t)]+(p(t))^{2}\widehat{E}[\widehat{f}^{*}_{\mu_{2}a_{2}}(t)],$
$\displaystyle G_{1}(t)$
$\displaystyle=P_{1}(t)\Big{(}b_{x}(t)+b_{x}(t-l)+\sigma_{x}(t)\sigma_{x}(t-l)+f_{y}(t)+f_{z}(t)(\sigma_{x^{\prime}}(t)+\sigma_{x}(t-l))+\widehat{E}[\widehat{f}^{*}_{\mu_{2}}(t)]\Big{)}$
$\displaystyle\quad+Q_{1}(t)\Big{(}\sigma_{x}(t)+\sigma_{x}(t-l)+f_{z}(t)\Big{)}+P(t)\Big{(}b_{x^{\prime}}(t)+\sigma_{x}(t)\sigma_{x^{\prime}}(t)+f_{z}(t)\sigma_{x^{\prime}}(t)\Big{)}$
$\displaystyle\quad+Q(t)\sigma_{x^{\prime}}(t)+p(t)\Big{(}b_{xx^{\prime}}(t)+f_{z}(t)\sigma_{xx^{\prime}}(t)+f_{yx^{\prime}}(t)+\sigma_{x}(t)f_{zx^{\prime}}(t)\Big{)}+q(t)(\sigma_{xx^{\prime}}(t)+f_{zx^{\prime}}(t))$
$\displaystyle\quad+f_{xx^{\prime}}(t),$

and $U=(1,p(t),p(t)\sigma_{x}(t)+q(t)),\
D^{2}f=\begin{pmatrix}f_{xx}(t)&f_{xy}(t)&f_{xz}(t)\\\
f_{yx}(t)&f_{yy}(t)&f_{yz}(t)\\\ f_{zx}(t)&f_{zy}(t)&f_{zz}(t)\end{pmatrix}.$

Obviously, according to Guo, Xiong, Zheng [16], under the assumptions
(H3.1)-(H3.3) the anticipated linear BSDE (3.53)-(3.54) possesses a unique
solution $\Big{(}(P,Q),(P_{1},Q_{1})\Big{)}$ satisfying, for $\beta\geq 2$,

$E\Big{[}\sup_{t\in[0,T+l]}(|P(t)|^{\beta}+|P_{1}(t)|^{\beta})+(\int_{0}^{T+l}|Q(t)|^{2}+|Q_{1}(t)|^{2}dt)^{\frac{\beta}{2}}\Big{]}\leq
C_{\beta}.$ (3.55)

###### Remark 3.7

If $T\leq l$, the control problem becomes the non delay case. In fact, the
term $X^{1,\varepsilon}(t-l)$ in the first-order variational equation will
vanish. In this setting, our stochastic maximum principle reduces to the
global stochastic maximum principle but driven by a mean-field forward-
backward control system.

###### Remark 3.8

Although in above we present the case of one pointwise delay, our approach is
also suitable for the pointwise delay case, i.e., $Nl<T\leq(N+1)l,\
N\in\mathbb{N},N>1$. Indeed, in this situation the first-order variational
equation (3.5) can be extended to
$(X^{1,\varepsilon}(t),X^{1,\varepsilon}(t-l),\cdot\cdot\cdot,X^{1,\varepsilon}(t-Nl))$
naturally, and accordingly, the equation $K^{\varepsilon}(t)$ will depend on
$(X^{1,\varepsilon}(t))^{2},(X^{1,\varepsilon}(t-l))^{2},\cdot\cdot\cdot,(X^{1,\varepsilon}(t-Nl))^{2}$.
Hence, some corresponding auxiliary equations, similar to $K^{1}(t)$,
$K^{2}(t):=X^{1,\varepsilon}(t)X^{1,\varepsilon}(t-2l),\cdot\cdot\cdot,K^{N}(t):=X^{1,\varepsilon}(t)X^{1,\varepsilon}(t-Nl),$
will be considered, and subsequently, the second-order adjoint system changes
to a system of equations for high-dimensional matrix-valued processes, just
like the statement in Remark 4.2.

##  4 Expansion of cost functional $Y^{\varepsilon}$

In order to obtain the general stochastic maximum principle, Peng [24] put
forward to consider the second-order Taylor expansion of the variation
$Y^{\varepsilon}$, i.e.,

$Y^{\varepsilon}(t)=Y^{*}(t)+p(t)(X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t))+\frac{1}{2}P(t)(X^{1,\varepsilon}(t))^{2}+o(\varepsilon),$
(4.1)

where $p,P$ are the solutions of the first- and second-order adjoint equation,
respectively.
Generally speaking, when the coefficient $f$ depends on $(y,z)$, (4.1) does
not hold true any more since the term
$p(t)\delta\sigma(t)\mathbbm{1}_{E_{\varepsilon}}(t)$ in the variation of $z$
is $O(\varepsilon)$, but not $o(\varepsilon)$. Hence, Hu [18] constructed an
auxiliary BSDE and considered the following expansion:

$Y^{\varepsilon}(t)=Y^{*}(t)+p(t)(X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t))+\frac{1}{2}P(t)(X^{1,\varepsilon}(t))^{2}+\breve{Y}(t)+o(\varepsilon),\
t\in[0,T],$ (4.2)

where $\breve{Y}$ is the solution of an auxiliary BSDE.

Our argument in some degree follows their scheme, but there are still
potential difficulties due to the appearance of delay term in the general
mean-field case. But it turns out that these difficulties can be solved by
considering the auxiliary equation of mixed term
$X^{1,\varepsilon}(s)X^{1,\varepsilon}(s-l)$, by constructing a new auxiliary
mean-field BSDE and by applying the new generic estimate, see Lemma 3.1. Let
us now state it in detail.

We first prove that there exists a process $(\breve{Y}(t))_{t\in[0,T]}$ with
$\breve{Y}(T)=0$, such that for $t\in[0,T]$, $P$-a.s.,

$Y^{\varepsilon}(t)=Y^{*}(t)+p(t)(X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t))+\frac{1}{2}P(t)(X^{1,\varepsilon}(t))^{2}+P_{1}(t)X^{1,\varepsilon}(t)X^{1,\varepsilon}(t-l)+\breve{Y}(t)+o(\varepsilon),$
(4.3)

where the convergence is in $L^{2}(\Omega,C[0,T])$ sense.

For this, consider the following mean-field BSDE:

$\left\\{\begin{aligned}
-d\breve{Y}(s)&=f_{y}(t)\breve{Y}(t)+f_{z}(t)\breve{Z}(t)+\widehat{E}[\widehat{f}_{\mu_{2}}(t)\widehat{\breve{Y}}(t)]+\Big{(}A_{1}(t)+\Delta
f(t)\mathbbm{1}_{E_{\varepsilon}}(t)\Big{)}dt\\\ &\quad-\breve{Z}(t)dW(t),\
t\in[0,T),\\\ \breve{Y}(t)&=0,\ \breve{Z}(t)=0,\
t\in[T,T+l],\end{aligned}\right.$ (4.4)

where

$\displaystyle A_{1}(t)$ $\displaystyle:=p(t)\Delta
b(t)+q(t)\Delta\sigma(t)+\frac{1}{2}P(t)(\Delta\sigma(t))^{2},$ (4.5)
$\displaystyle\Delta f(t)$
$\displaystyle:=f(t,X^{*}(t),X^{*}(t-l),Y^{*}(t),Z^{*}(t)+p(t)\Delta\sigma(t),P_{(X^{*}(t),Y^{*}(t))},v(t))$
$\displaystyle\quad-f(t,X^{*}(t),X^{*}(t-l),Y^{*}(t),Z^{*}(t),P_{(X^{*}(t),Y^{*}(t))},u^{*}(t)).$

It is clear that under the assumptions (H3.1), (H3.2) the equation (4.4) has a
unique solution
$(\breve{Y},\breve{Z})\in\mathcal{S}^{2}_{\mathbb{F}}(0,T+l)\times\mathcal{H}^{2}_{\mathbb{F}}(0,T+l)$.
Furthermore, from standard argument of classical BSDE, we have, for $\beta\geq
2$,

$\displaystyle
E[\sup_{t\in[0,T+l]}|\breve{Y}(t)|^{\beta}+(\int_{0}^{T+l}|\breve{Z}(t)|^{2}dt)^{\frac{\beta}{2}}]\leq\varepsilon^{\frac{\beta}{2}}\rho(\varepsilon).$
(4.6)

###### Theorem 4.1

Suppose the assumptions (H3.1)-(H3.3) hold true, then we have the following
second-order expansion of $Y^{\varepsilon}$

$\displaystyle
E\Big{[}\sup_{t\in[0,T+l]}|Y^{\varepsilon}(t)-Y^{*}(t)-p(t)(X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t))-\frac{1}{2}P(t)(X^{1,\varepsilon}(t))^{2}$
(4.7) $\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad-
P_{1}(t)X^{1,\varepsilon}(t)X^{1,\varepsilon}(t-l)-\breve{Y}(t)|^{2}\Big{]}\leq\varepsilon^{2}\rho(\varepsilon).$

Proof. The proof is split into two steps.

Step 1. Define

$M(t):=p(t)(X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t))+\frac{1}{2}P(t)K^{\varepsilon}(t)+P_{1}(t)K^{\varepsilon}_{1}(t).$
(4.8)

Following Itô’s formula, one can check

$\displaystyle dM(t)$
$\displaystyle=\Big{\\{}A_{1}(t)\mathbbm{1}_{E_{\varepsilon}}(t)+A_{2}(t)+A_{3}(t)+\overline{A}_{3}(t)+A_{4}(t)\Big{\\}}dt$
(4.9)
$\displaystyle\quad+\Big{\\{}p(t)\Delta\sigma(t)\mathbbm{1}_{E_{\varepsilon}}(t)+B_{2}(t)+B_{3}(t)+\overline{B}_{3}(t)+B_{4}(t)\Big{\\}}dW_{t}.$

Here $A_{1},A_{2},\cdot\cdot\cdot,B_{4}$ see appendix.
From Remark 3.2, we have

$E\Big{[}\Big{(}\int_{0}^{T}|A_{4}(s)+B_{4}(s)|ds\Big{)}^{2}\Big{]}\leq\varepsilon^{2}\rho(\varepsilon).$
(4.10)

Hence, (4.9) can be written as

$\displaystyle dM(t)$
$\displaystyle=\Big{\\{}A_{1}(t)\mathbbm{1}_{E_{\varepsilon}}(t)+A_{2}(t)+A_{3}(t)+\overline{A}_{3}(t)\Big{\\}}dt$
(4.11)
$\displaystyle\quad+\Big{\\{}p(t)\Delta\sigma(t)\mathbbm{1}_{E_{\varepsilon}}(t)+B_{2}(t)+B_{3}(t)+\overline{B}_{3}(t)\Big{\\}}dW_{t}+o(\varepsilon).$

Step 2. Define

$\displaystyle\Delta Y(t)$
$\displaystyle=Y^{\varepsilon}(t)-Y^{*}(t)-\breve{Y}(t)-M(t),$
$\displaystyle\Delta Z(t)$
$\displaystyle=Z^{\varepsilon}(t)-Z^{*}(t)-\breve{Z}(t)-(p(t)\Delta\sigma(t)\mathbbm{1}_{E_{\varepsilon}}(t)+B_{2}(t)+B_{3}(t)+\overline{B}_{3}(t)).$

Then

$\displaystyle\Delta Y(t)=\Delta Y(T)$
$\displaystyle+\int_{t}^{T}\Big{(}f(s,X^{\varepsilon}(s),X^{\varepsilon}(t-l),Y^{\varepsilon}(s),Z^{\varepsilon}(s),P_{(X^{\varepsilon}(s),Y^{\varepsilon}(s))},u^{\varepsilon}(s))$
(4.12)
$\displaystyle\quad-f(s,X^{*}(s),X^{*}(t-l),Y^{*}(s),Z^{*}(s),P_{(X^{*}(s),Y^{*}(s))},u^{*}(s))$
$\displaystyle\quad-(f_{y}(s)\breve{Y}(s)+f_{z}(s)\breve{Z}(s)+\widehat{E}[\widehat{f}_{\mu_{2}}(s)\widehat{\breve{Y}}(s)]+\Delta
f(s)\mathbbm{1}_{E_{\varepsilon}}(s))$
$\displaystyle\quad-(A_{2}(t)+A_{3}(t)+\overline{A}_{3}(t))\Big{)}ds-\int_{t}^{T}\Delta
Z(s)dW_{s}+o(\varepsilon).$

Let us now analyse

$\displaystyle I(s):$
$\displaystyle=f(s,X^{\varepsilon}(s),X^{\varepsilon}(s-l),Y^{\varepsilon}(s),Z^{\varepsilon}(s),P_{(X^{\varepsilon}(s),Y^{\varepsilon}(s))},u^{\varepsilon}(s))$
(4.13)
$\displaystyle-f(s,X^{*}(s),X^{*}(s-l),Y^{*}(s),Z^{*}(s),P_{(X^{*}(s),Y^{*}(s))},u^{*}(s)).$

First, recall the definitions of $\Delta Y,\Delta Z$, (4.13) can read as

$\displaystyle
I(s):=I_{1}(s)+I_{2}(s)+f(s,X^{*}(s),X^{*}(s-l),Y^{*}(s),Z^{*}(s)+p(s)\Delta(s)\mathbbm{1}_{E_{\varepsilon}}(s),P_{(X^{*}(s),Y^{*}(s))},u^{\varepsilon}(s))$
(4.14)
$\displaystyle\qquad-f(s,X^{*}(s),X^{*}(s-l),Y^{*}(s),Z^{*}(s),P_{(X^{*}(s),Y^{*}(s))},u^{*}(s))$
$\displaystyle\qquad=I_{1}(s)+I_{2}(s)+\mathbbm{1}_{E_{\varepsilon}}(s)\triangle
f(s),$

where

$\displaystyle I_{1}(s):$
$\displaystyle=f(s,X^{\varepsilon}(s),X^{\varepsilon}(s-l),Y^{\varepsilon}(s),Z^{\varepsilon}(s),P_{X^{\varepsilon}(s),Y^{\varepsilon}(s)},u^{\varepsilon}(s))$
(4.15)
$\displaystyle-f(s,X^{*}(s)+X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s),X^{*}(s-l)+X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l),$
$\displaystyle\quad
Y^{*}(s)+\breve{Y}(s)+M(s),Z^{*}(s)+\breve{Z}(s)+(p(s)\Delta\sigma(s)\mathbbm{1}_{E_{\varepsilon}}(s)+B_{2}(t)+B_{3}(s)+\overline{B}_{3}(s)),$
$\displaystyle\quad
P_{(X^{*}(s)+X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s),Y^{*}(s)+\breve{Y}(s)+M(s))},u^{\varepsilon}(s)),$
$\displaystyle I_{2}(s):$
$\displaystyle=f(s,X^{*}(s)+X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s),X^{*}(s-l)+X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l),$
$\displaystyle\quad
Y^{*}(s)+\breve{Y}(s)+M(s),Z^{*}(s)+\breve{Z}(s)+(p(s)\Delta\sigma(s)\mathbbm{1}_{E_{\varepsilon}}(s)+B_{2}(s)+B_{3}(s)+\overline{B}_{3}(s)),$
$\displaystyle\quad
P_{(X^{*}(s)+X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s),Y^{*}(s)+\breve{Y}(s)+M(s))},u^{\varepsilon}(s))$
$\displaystyle-f(s,X^{*}(s),X^{*}(s-l),Y^{*}(s),Z^{*}(s)+p(s)\Delta\sigma(s)\mathbbm{1}_{E_{\varepsilon}}(s),P_{(X^{*}(s),Y^{*}(s))},u^{\varepsilon}(s)).$

From the Lipschitz property of $f$ and the fact
$W(P_{\xi},P_{\eta})\leq\\{E|\xi-\eta|^{2}\\}^{\frac{1}{2}},\ \xi,\eta\in
L^{2}(\Omega,\mathcal{F}_{T},P)$, we know

$\displaystyle|I_{1}(s)|$
$\displaystyle\leq|X^{\varepsilon}(s)-X^{*}(s)-X^{1,\varepsilon}(s)-X^{2,\varepsilon}(s)|+|X^{\varepsilon}(s-l)-X^{*}(s-l)-X^{1,\varepsilon}(s-l)-X^{2,\varepsilon}(s-l)|$
(4.16) $\displaystyle+|\Delta Y(s)|+|\Delta
Z(s)|+\\{E|X^{\varepsilon}(s)-X^{*}(s)-X^{1,\varepsilon}(s)-X^{2,\varepsilon}(s)|^{2}\\}^{\frac{1}{2}}+\\{E|\Delta
Y(s)|^{2}\\}^{\frac{1}{2}}.$

As for $I_{2}(s)$, obviously,

$I_{2}(s)=\mathbbm{1}_{E_{\varepsilon}}(s)I_{2,1}(s)+\mathbbm{1}_{(E_{\varepsilon})^{c}}(s)I_{2,2}(s)=I_{2,2}(s)+\mathbbm{1}_{E_{\varepsilon}}(s)(I_{2,1}(s)-I_{2,2}(s)),$

here

$\displaystyle I_{2,1}(s)$
$\displaystyle=f(s,X^{*}(s)+X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s),X^{*}(s-l)+X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l),$
(4.17) $\displaystyle\quad\
Y^{*}(s)+\breve{Y}(s)+M(s),Z^{*}(s)+\breve{Z}(s)+p(s)\Delta\sigma(s)+B_{2}(s)+B_{3}(s)+\overline{B}_{3}(s),$
$\displaystyle\quad\
P_{(X^{*}(s)+X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s),Y^{*}(s)+\breve{Y}(s)+M(s))},v(s))$
$\displaystyle-f(s,X^{*}(s),X^{*}(s-l),Y^{*}(s),Z^{*}(s)+p(s)\Delta\sigma(s),P_{(X^{*}(s),Y^{*}(s))},v(s)),$
$\displaystyle I_{2,2}(s)$
$\displaystyle=f(s,X^{*}(s)+X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s),X^{*}(s-l)+X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l),$
$\displaystyle\quad\
Y^{*}(s)+\breve{Y}(s)+M(s),Z^{*}(s)+\breve{Z}(s)+B_{2}(s)+B_{3}(s)+\overline{B}_{3}(s),$
$\displaystyle\quad\
P_{(X^{*}(s)+X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s),Y^{*}(s)+\breve{Y}(s)+M(s))},u^{*}(s))$
$\displaystyle-f(s,X^{*}(s),X^{*}(s-l),Y^{*}(s),Z^{*}(s),P_{(X^{*}(s),Y^{*}(s))},u^{*}(s)).$

Since $f$ is Lipschitz continuous, we can obtain

$\displaystyle|I_{2,1}(s)-I_{2,2}(s)|$
$\displaystyle\leq|X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s)|+|X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l)|+|\breve{Y}(s)+M(s)|$
$\displaystyle+|\breve{Z}(s)+B_{2}(s)+B_{3}(s)+\overline{B}_{3}(s)|+\\{E|X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s)|^{2}\\}^{\frac{1}{2}}$
$\displaystyle+\\{E|\breve{Y}(s)+M(s)|^{2}\\}^{\frac{1}{2}}.$

From (3.40), (3.41), (4.6) we have

$\displaystyle
E[(\int_{0}^{T}|I_{2,1}(s)-I_{2,2}(s)|\mathbbm{1}_{E_{\varepsilon}}(s)ds)^{2}]\leq\varepsilon^{2}\rho(\varepsilon)$
(4.18)

(The proof of (4.18) refers to Appendix).

In order to complete the proof, it remains to calculate $I_{2,2}(s)$. Applying
the Taylor expansion, see Appendix for details, it follows

$\displaystyle I_{2,2}(s)$
$\displaystyle=f_{y}(s)\breve{Y}(s)+f_{z}(s)\breve{Z}(s)+\widehat{E}[\widehat{f}_{\mu_{2}}(s)\widehat{\breve{Y}}(s)]+(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))\Big{(}f_{x}(s)+f_{y}(s)p(s)$
(4.19)
$\displaystyle+f_{z}(s)(p(s)\sigma_{x}(s)+q(s))\Big{)}+(X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l))\Big{(}f_{x^{\prime}}(s)+f_{z}(s)p(s)\sigma_{x^{\prime}}(s)\Big{)}$
$\displaystyle+\widehat{E}\Big{[}(\widehat{X}^{1,\varepsilon}(s)+\widehat{X}^{2,\varepsilon}(s))\Big{(}f_{z}(s)p(s)\widehat{\sigma}_{\nu}(s)+\widehat{f}_{\mu_{1}}(s)+\widehat{f}_{\mu_{2}}(s)\widehat{p}(s)\Big{)}\Big{]}$
$\displaystyle+\frac{1}{2}K^{\varepsilon}(s)\Big{(}f_{y}(s)P(s)+f_{z}(s)\big{(}p(s)\sigma_{xx}(s)+2P(s)\sigma_{x}(s)+Q(s)\big{)}\Big{)}$
$\displaystyle+\frac{1}{2}\widehat{E}\Big{[}\widehat{K}^{\varepsilon}(s)\Big{(}f_{z}(s)p(s)\widehat{\sigma}_{\nu
a}(s)+\widehat{f}_{\mu_{2}}(s)\widehat{P}(s)\Big{)}\Big{]}+\frac{1}{2}K^{\varepsilon}(s-l)\Big{(}f_{z}(s)(p(s)\sigma_{x^{\prime}x^{\prime}}(s)+2P_{1}(s)\sigma_{x^{\prime}}(s))\Big{)}$
$\displaystyle+K^{\varepsilon}_{1}(s)\Big{(}f_{y}(s)P_{1}(s)+f_{z}(s)\Big{(}p(s)\sigma_{xx^{\prime}}(s)+P(s)\sigma_{x^{\prime}}(s)+P_{1}(s)\sigma_{x}(s)+P_{1}(s)\sigma_{x}(s-l)+Q_{1}(s)\Big{)}\Big{)}$
$\displaystyle+\widehat{E}[\widehat{f}_{\mu_{2}}(s)\widehat{P}_{1}(s)\widehat{K}^{\varepsilon}_{1}(s)]+\frac{1}{2}(X^{1,\varepsilon}(s))^{2}UD^{2}fU^{\intercal}+\frac{1}{2}f_{x^{\prime}x^{\prime}}(s)K^{\varepsilon}(s-l)$
$\displaystyle+K^{\varepsilon}_{1}(s)(f_{xx^{\prime}}(s),f_{yx^{\prime}}(s),f_{zx^{\prime}}(s))U^{\intercal}+\frac{1}{2}\widehat{E}[\widehat{K}^{\varepsilon}(s)(\widehat{f}_{\mu_{1}a_{1}}(s)+\widehat{f}_{\mu_{2}a_{2}}(s)(\widehat{p}(s))^{2})]+o(\varepsilon),$

where $U=(1,p(s),p(s)\sigma_{x}(s)+q(s))$, $D^{2}f$ is the Hessian matrix of
$f$ with respect to $(x,y,z)$.
Notice the equality

$E[\int_{0}^{T}E^{\mathcal{F}_{s}}[b_{x^{\prime}}(s+l)p(s+l)X^{1,\varepsilon}(s)]ds]=E[\int_{0}^{T}b_{x^{\prime}}(s)p(s)X^{1,\varepsilon}(s-l)ds],$

which comes from a change of variable combining with the final condition for
$p(s)$ and the initial condition for $X^{1,\varepsilon}(s)$, and notice the
fact

$E\widehat{E}[\widehat{f}_{\mu_{2}}(s)\widehat{P}_{1}(s)\widehat{X}^{1,\varepsilon}(s)\widehat{X}^{1,\varepsilon}(s-l)]=E\widehat{E}[\widehat{f}^{*}_{\mu_{2}}(s)P_{1}(s)X^{1,\varepsilon}(s)X^{1,\varepsilon}(s-l)],$

which comes from the definitions of $\widehat{f}_{\mu_{2}}(s)$ and
$\widehat{f}^{*}_{\mu_{2}}(s)$ (see (3.43)) and the independent copy
assumption, i.e.,
$\widehat{P}_{(\widehat{X}^{*}(s),\widehat{Y}^{*}(s))}=P_{(X^{*}(s),Y^{*}(s))}$,
as well as recall the definitions of $A_{2}(s),A_{3}(s),\overline{A_{3}}(s),$
it yields

$\displaystyle\Delta
Y(t)=\int_{t}^{T}\Big{(}I_{1}(s)+(I_{2,2}(s)-I_{2,1}(s))\mathbbm{1}_{E_{\varepsilon}}(s)\Big{)}ds-\int_{t}^{T}\Delta
Z(s)dW_{s}+o(\varepsilon),\ t\in[0,T].$ (4.20)

Thanks to (4.16), (4.18) and Gronwall lemma, one gets

$\displaystyle E[\sup_{t\in[0,T]}|\Delta Y(s)|^{2}+\int_{0}^{T}|\Delta
Z(s)|^{2}ds]\leq\varepsilon^{2}\rho(\varepsilon).$ (4.21)

We finish the proof. $\Box$

###### Remark 4.2

Notice $K^{\varepsilon}(t)=(X^{1,\varepsilon}(t))^{2},\
K^{\varepsilon}_{1}(t)=X^{1,\varepsilon}(t)X^{1,\varepsilon}(t-l),$ then (4.3)
can be written as

$\displaystyle Y^{\varepsilon}(t)$
$\displaystyle=Y^{*}(t)+p(t)(X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t))+\breve{Y}(t)+o(\varepsilon)$
(4.22)
$\displaystyle\quad+\frac{1}{2}(X^{1,\varepsilon}(t),X^{1,\varepsilon}(t-l))\begin{pmatrix}P(t)&P_{1}(t)\\\
P_{1}(t)&0\end{pmatrix}\begin{pmatrix}X^{1,\varepsilon}(t)&\\\
X^{1,\varepsilon}(t-l)&\end{pmatrix},$

where $(P(t),P_{1}(t))_{t\in[0,T+l]}$ is the solution of (3.53).
It implies that for the pointwise delay case $Nl<T\leq(N+1)l,\
N>1,N\in\mathbb{N}$ we naturally have the following second-order expansion of
the variation process

$\displaystyle
Y^{\varepsilon}(t)=Y^{*}(t)+p(t)(X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t))+\breve{Y}(t)+o(\varepsilon)$
(4.23) $\displaystyle+\frac{1}{2}\begin{pmatrix}X^{1,\varepsilon}(t)&\\\
X^{1,\varepsilon}(t-l)&\\\ \cdot&\\\ \cdot&\\\ \cdot&\\\
X^{1,\varepsilon}(t-Nl)\end{pmatrix}^{\intercal}\begin{pmatrix}P_{00}(t)&\cdot\cdot\cdot&P_{0(N-1)}(t)&P_{0N}(t)\\\
P_{10}(t)&\cdot\cdot\cdot&P_{1(N-1)}(t)&P_{1N}(t)\\\
\cdot\cdot\cdot&\cdot\cdot\cdot&\cdot\cdot\cdot&\cdot\cdot\cdot\\\
P_{(N-1)0}(t)&\cdot\cdot\cdot&P_{(N-1)(N-1)}(t)&P_{(N-1)N}(t)\\\
P_{N0}(t)&\cdot\cdot\cdot&P_{N(N-1)}(t)&0\\\
\end{pmatrix}\begin{pmatrix}X^{1,\varepsilon}(t)&\\\
X^{1,\varepsilon}(t-l)&\\\ \cdot&\\\ \cdot&\\\ \cdot&\\\
X^{1,\varepsilon}(t-Nl)\end{pmatrix}.$

Here $P_{ij},\ i,j=0,1,\cdot\cdot\cdot,N$ denotes the adjoint process with
$P_{ij}=P_{ji}$ and $P_{NN}\equiv 0$. Clearly, here $(P_{00},P_{01})$ is just
$(P,P_{1})$ in (4.22).

##  5 Stochastic maximum principle

_Hamiltonian function_ We define

$\displaystyle H(t,x,x^{\prime},y,z,\nu,\mu,v;p,q,P)$ (5.1)
$\displaystyle=pb(t,x,x^{\prime},\nu,v)+q\sigma(t,x,x^{\prime},\nu,v)+\frac{1}{2}P\Big{(}\sigma(t,x,x^{\prime},\nu,v)-\sigma(t,X^{*}(t),X^{*}(t-l),P_{X^{*}(t)},u^{*}(t))\Big{)}^{2}$
$\displaystyle\quad+f(t,x,x^{\prime},y,z+p(\sigma(t,x,x^{\prime},\nu,v)-\sigma(t,X^{*}(t),X^{*}(t-l),P_{X^{*}(t)},u^{*}(t)),\mu,v),$

where
$(t,x,x^{\prime},y,z,\nu,\mu,v,p,q,P)\in[0,T]\times\mathbb{R}^{4}\times\mathcal{P}_{2}(\mathbb{R})\times\mathcal{P}_{2}(\mathbb{R}^{2})\times
U\times\mathbb{R}^{3}$.

###### Theorem 5.1

Let the assumptions (H3.1)-(H3.3) be in force, and
$\widehat{f}^{*}_{\mu_{2}}(t)>0,\ t\in[0,T],\widehat{P}\otimes P$-a.s., and
let $u^{*}(t)$ be the optimal control and $(X^{*}(t),Y^{*}(t),Z^{*}(t))$ be
the optimal trajectory. Then there exist two pairs of stochastic processes
$(p,q),((P,Q),(P_{1},Q_{1}))$, which are the solutions of the first- and
second-order adjoint equations, separately, satisfying (3.45) and (3.55), such
that for given $l,T$ with $l<T\leq 2l$, the following inequality holds true

$\displaystyle
H(t,X^{*}(t),X^{*}(t-l),Y^{*}(t),Z^{*}(t),P_{X^{*}(t)},P_{(X^{*}(t),Y^{*}(t))},v,p(t),q(t),P(t))$
(5.2) $\displaystyle\geq
H(t,X^{*}(t),X^{*}(t-l),Y^{*}(t),Z^{*}(t),P_{X^{*}(t)},P_{(X^{*}(t),Y^{*}(t))},u^{*}(t),p(t),q(t),P(t)),$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad
t\in[0,T],\text{a.s.}\ \text{a.e.}$

Proof. From the definition of $J(v(\cdot))$, (4.2) and
$X^{1,\varepsilon}(0)=X^{2,\varepsilon}(0)=0$, we derive

$J(u^{\varepsilon}(\cdot))-J(u^{*}(\cdot))=Y^{\varepsilon}(0)-Y^{*}(0)=\breve{Y}(0)+o(\varepsilon)\geq
0,$

where $\breve{Y}$ is the solution of the equation (4.4).

In order to complete the proof, let us introduce the following mean-field SDE

$\left\\{\begin{aligned}
d\Gamma(t)&=(f_{y}(t)\Gamma(t)+E[\widehat{f}^{*}_{\mu_{2}}(t)\widehat{\Gamma}(t)])dt+f_{z}(t)\Gamma(t)dW_{t},\
t\in[0,T],\\\ \Gamma(0)&=1.\end{aligned}\right.$ (5.3)

We argue that

$\Gamma(t)>0,\ t\in[0,T],\ P\text{-a.s.}$ (5.4)

Indeed, consider a forward stochastic differential equation:

$\left\\{\begin{aligned}
d\Gamma^{o}(t)&=f_{y}(t)\Gamma^{o}(t)dt+f_{z}(t)\Gamma^{o}(t)dW_{t},\
t\in[0,T],\\\ \Gamma^{0}(0)&=1.\end{aligned}\right.$ (5.5)

Set $\Delta\Gamma(t):=\Gamma^{o}(t)-\Gamma(t)$. Applying Itô’s formula to
$(\Delta\Gamma(t)^{+})^{2}$ and notice $\widehat{f}^{*}_{\mu_{2}}>0$, we have

$\displaystyle E((\Delta\Gamma(t))^{+})^{2}$ (5.6) $\displaystyle=$
$\displaystyle
E\int_{0}^{t}2\Delta\Gamma(s)^{+}f_{y}(s)\Delta\Gamma(s)+1_{\\{\Delta\Gamma(s)>0\\}}f_{z}^{2}(s)(\Delta\Gamma(s))^{2}-2\Delta\Gamma(s)^{+}\widehat{E}[\widehat{f}^{*}_{\mu_{2}}(s)\widehat{\Gamma}(s)]ds$
$\displaystyle=$ $\displaystyle
E\int_{0}^{t}[2f_{y}(s)+f_{z}^{2}(s)](\Delta\Gamma(s)^{+})^{2}ds+2E\int_{0}^{t}\Delta\Gamma(s)^{+}\widehat{E}[\widehat{f}^{*}_{\mu_{2}}(s)(\Delta\widehat{\Gamma}(s)-\widehat{\Gamma}^{o}(s))]ds$
$\displaystyle\leq$ $\displaystyle
E\int_{0}^{t}[2f_{y}(s)+f_{z}^{2}(s)](\Delta\Gamma(s)^{+})^{2}ds+2E\int_{0}^{t}\Delta\Gamma(s)^{+}\widehat{E}[\widehat{f}^{*}_{\mu_{2}}(s)\Delta\widehat{\Gamma}(s)]ds$
$\displaystyle\leq$ $\displaystyle CE\int_{0}^{t}(\Delta\Gamma(s)^{+})^{2}ds.$

The Gronwall lemma allows to show $\Gamma(t)\geq\Gamma^{o}(t)>0,\ t\in[0,T],\
P$-a.s.
Now applying Itô’s formula to $\breve{Y}(t)\Gamma(t)$, one has

$\breve{Y}(0)=\widehat{E}[\int_{0}^{T}\Gamma(s)(A_{1}(s)+\Delta
f(s))\mathbbm{1}_{E_{\varepsilon}}(s)ds].$ (5.7)

Combining $\Gamma(t)\geq\Gamma^{o}(t)>0,\ t\in[0,T],\ P$-a.s., and applying
the Lebesgue differentiation theorem, we obtain (5.2). $\Box$

###### Remark 5.2

From the above proof, we know that nothing would change for the pointwise
delay case: $Nl<T\leq(N+1)l,\ N\in\mathbb{N},\ N>1$ once obtaining the
expansion (4.23).

Hence, as for the pointwise delay case, we give the following result without
proof.

###### Corollary 5.3

Let the assumptions (H3.1)-(H3.3) be in force, and
$\widehat{f}^{*}_{\mu_{2}}(t)>0,\ t\in[0,T],\widehat{P}\otimes P$-a.s., and
let $u^{*}(t)$ be the optimal control and $(X^{*}(t),Y^{*}(t),Z^{*}(t))$ be
the optimal trajectory. Then there exist two pair of stochastic processes
$(p,q),((P_{00},Q_{00}),(P_{01},Q_{01}),\cdot\cdot\cdot,(P_{(N-1)N},Q_{(N-1)N}))$
which are the solutions of the first- and second-order adjoint equations,
separately, such that for given $l,T$ with $(N-1)l<T\leq Nl,\
N\in\mathbb{N},N\geq 1$, the following inequality holds true

$\displaystyle
H(t,X^{*}(t),X^{*}(t-l),Y^{*}(t),Z^{*}(t),P_{X^{*}(t)},P_{(X^{*}(t),Y^{*}(t))},v,p(t),q(t),P_{00}(t))$
(5.8) $\displaystyle\geq
H(t,X^{*}(t),X^{*}(t-l),Y^{*}(t),Z^{*}(t),P_{X^{*}(t)},P_{(X^{*}(t),Y^{*}(t))},u^{*}(t),p(t),q(t),P_{00}(t)),$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad
t\in[0,T],\text{a.s.}\ \text{a.e.}$

##  6 Application to Finance

Let us consider a continuous-time financial market with a risk-free bond and a
risky share being traded. The bond price process denoted by
$\\{S_{0}(t)\\}_{t\in[0,T]}$ is described by

$\left\\{\begin{aligned} dS_{0}(t)&=\alpha(t)S_{0}(t)dt,\ t\in[0,T],\\\
S_{0}(0)&=1,\end{aligned}\right.$ (6.1)

where $\alpha(t)$ is the risk-free interest rate. It is a uniformly bounded,
deterministic function.
The price process of the risky share $\\{S_{1}(t)\\}_{t\in[0,T]}$ evolves
following the equation

$\left\\{\begin{aligned} dS_{1}(t)&=b(t)S_{1}(t)dt+\sigma(t)S_{1}(t)dW(t),\
t\in[0,T],\\\ S_{1}(0)&=s_{1}>0,\end{aligned}\right.$ (6.2)

where $b:[0,T]\rightarrow\mathbb{R},\ \sigma:[0,T]\rightarrow(0,+\infty)$ are
uniformly bounded deterministic function of $t$, which denote the appreciation
rate and the volatility, respectively.

By $v(t)$ we denote the amount of an investor’s wealth invested in the share
at time $t$. Then the value $U(t)$ of the investor of a self-financing
portfolio consisting of the risk-free and the risky assets is

$\displaystyle
dU(t)=(\alpha(t)U(t)+v(t)(b(t)-\alpha(t)))dt+v(t)\sigma(t)dW(t).$ (6.3)

But in practice there must be some unavoidable delays arising in various
situations. Hence, Shen, Meng and Shi [25] considered the following wealth
process with delays and jumps

$\left\\{\begin{aligned}
dU(t)&=(\alpha(t)U(t)+\beta(t)U(t-l)+v(t)(b(t)-\alpha(t)))dt+v(t)\sigma(t)dW(t)\\\
&\quad+\int_{\mathbb{R}_{0}}v(t)r(t,z)\widetilde{N}(dz,dt),\ t\in[0,T],\\\
U(t)&=x>0,\ t\in[-l,0].\end{aligned}\right.$ (6.4)

In our case, we want to consider the corresponding mean-field game without the
jump term. More precisely, suppose $T<l\leq 2T$ and let
$g^{i}:\mathbb{R}\rightarrow\mathbb{R}$ be a twice continuously differentiable
functions. Consider $N$ individual investors and suppose that the $i$-th
investor’s wealth process is of the form

$\left\\{\begin{aligned}
dU^{i,v^{i}}(t)&=(\alpha^{i}(t)U^{i,v^{i}}(t)+\beta^{i}(t)U^{i,v^{i}}(t-l)+g^{i}(\overline{\nu}^{N}(t))+v^{i}(t)(b(t)-\alpha(t))dt\\\
&\quad+v^{i}(t)\sigma(t)dW^{i}(t),\ t\in[0,T],\\\ U^{i,v^{i}}(0)&=x^{i}>0,\
t\in[-l,0],\end{aligned}\right.$ (6.5)

where $\overline{\nu}^{N}(s)=\frac{1}{N}\sum_{i=1}^{N}\delta_{U^{i,v^{i}}(s)}$
and $\delta_{x}$ is the Dirac measure at $x$.

Assume that the $i$-th investor wants to minimize the utility $V^{i,v^{i}}(t)$
resulting from $v^{i}$. Without loss of generality, we define

$V^{i,v^{i}}(t)=E^{\mathcal{F}_{t}}[U^{i,v^{i}}(T)+\int_{t}^{T}f^{i}(s,V^{i,v^{i}}(s),\overline{\overline{\nu}}^{N}(s),v^{i}(s))ds].$
(6.6)

Here $f^{i}$ satisfies the assumptions (H3.1)-(H3.3) and
$\overline{\overline{\nu}}^{N}(s)=\frac{1}{N}\sum_{i=1}^{N}\delta_{(U^{i,v^{i}}(s),V^{i,v^{i}}(s))}.$
Following El Karoui, Peng and Quenez [14] the recursive utility
$V^{i,v^{i}}(t)$ satisfies the equation

$\left\\{\begin{aligned}
-dV^{i,v^{i}}(t)&=f^{i}(t,V^{i,v^{i}}(t),\overline{\overline{\nu}}^{N}(t),v^{i}(t))dt-R^{i,v^{i}}(t)dW^{i}(t),\
t\in[0,T],\\\ V^{i,v^{i}}(T)&=U^{i,v^{i}}(T).\end{aligned}\right.$ (6.7)

Consequently, the control problem can be formulated as

$\left\\{\begin{aligned}
dU^{i,v^{i}}(t)&=(\alpha^{i}(t)U^{i,v^{i}}(t)+\beta^{i}(t)U^{i,v^{i}}(t-l)+g^{i}(\overline{\nu}^{N}(t))+v^{i}(t)(b(t)-\alpha(t))dt\\\
&\quad+v^{i}(t)\sigma(t)dW^{i}(t),\ t\in[0,T],\\\ U^{i,v^{i}}(0)&=x^{i}>0,\
t\in[-l,0],\\\
-dV^{i,v^{i}}(t)&=f^{i}(t,V^{i,v^{i}}(t),\overline{\overline{\nu}}^{N}(t),v^{i}(t))dt-R^{i,v^{i}}(t)dW^{i}(t),\
t\in[0,T],\\\ V^{i,v^{i}}(T)&=U^{i,v^{i}}(T),\end{aligned}\right.$ (6.8)

and the target is to minimum $J^{i}(v^{i}(\cdot))=V^{i,v^{i}}(0),\ 1\leq i\leq
N.$

Suppose that the game is symmetric, in other words,
$\alpha^{i}(t)=\alpha(t),\beta^{i}(t)=\beta(t),x^{i}=x,f^{i}=f,g^{i}=g$ are
all independent of $i$, and let $N$ converge to $+\infty$, then following the
scheme of Lasry and Lions we can find approximate Nash equilibriums by a
limiting dynamics and assign representative investor the unified strategy $v$,
which is determined by the following mean-field FBSDE

$\left\\{\begin{aligned}
dX^{v}(t)&=(\alpha(t)X^{v}(t)+\beta(t)X^{v}(t-l)+g(P_{X^{v}(t)})+v(t)(b(t)-\alpha(t))dt\\\
&\quad+v(t)\sigma(t)dW(t),\ t\in[0,T],\\\ X^{v}(0)&=x>0,\ t\in[-l,0],\\\
-dY^{v}(t)&=f(t,Y^{v}(t),P_{(X^{v}(t),Y^{v}(t))},v(t))dt-Z^{v}(t)dW(t),\
t\in[0,T],\\\ Y^{v}(T)&=X^{v}(T).\end{aligned}\right.$ (6.9)

Accordingly, the cost functional changes to

$J(v(\cdot))=Y^{v}(0).$ (6.10)

Let $u^{*}(t)$ be the optimal control, $(X^{*},Y^{*},Z^{*})$ be the optimal
trajectory. We define _Hamiltonian function_

$\displaystyle H(t,x,x^{\prime},y,\nu,\mu,v;p,q,P)$
$\displaystyle=p(\alpha(t)x+\beta(t)x^{\prime}+\nu+v(b(t)-\alpha(t)))$ (6.11)
$\displaystyle\quad+qv\sigma(t)+\frac{1}{2}P(v-u^{*}(t))^{2}(\sigma(t))^{2}+f(t,y,\mu,v).$

Then from Theorem 5.1 we know that $u^{*}$ satisfies

$\displaystyle
H(t,X^{*}(t),X^{*}(t-l),Y^{*}(t),P_{X^{*}(t)},P_{(X^{*}(t),Y^{*}(t))},v;p(t),q(t),P(t))$
(6.12) $\displaystyle\geq
H(t,X^{*}(t),X^{*}(t-l),Y^{*}(t),P_{X^{*}(t)},P_{(X^{*}(t),Y^{*}(t))},u^{*}(t);p(t),q(t),P(t))$

i.e.,

$\displaystyle\Big{(}p(t)(b(t)-\alpha(t))-q(t)\sigma(t)\Big{)}(v-u^{*}(t))+\frac{1}{2}P(t)(\sigma(t))^{2}(v-u^{*}(t))^{2}$
(6.13)
$\displaystyle+f(t,Y^{*}(t),P_{(X^{*}(t),Y^{*}(t))},v)-f(t,Y^{*}(t),P_{(X^{*}(t),Y^{*}(t))},u^{*}(t))\geq
0,$

where $(p,q),((P,Q),(P_{1},Q_{1}))$ are the solutions of the first- and
second-order adjoint equations, respectively,

$\left\\{\begin{aligned}
-dp(t)&=\Big{(}p(t)(f_{y}(t)+\widehat{E}[\widehat{f}_{\mu_{2}}^{*}(t)]+\alpha(t))+\widehat{E}[\widehat{p}(t)\widehat{g}^{*}_{\nu}(t)]+E^{\mathcal{F}_{t}}[p(t+l)\beta(t+l)]+\widehat{E}[\hat{f}^{*}_{\mu_{1}}(t)]\Big{)}dt\\\
&\quad-q(t)dW(t),\ t\in[0,T],\\\ p(T)&=1,\ p(t)=0,\ t\in(T,T+l],\ q(t)=0,\
t\in[T,T+l].\end{aligned}\right.$ (6.14)

$\left\\{\begin{aligned}
-dP(t)&=\Big{(}P(t)(f_{y}(t)+\widehat{E}[\widehat{f}^{*}_{\mu_{2}}(t)]+2\alpha(t))+2E^{\mathcal{F}_{t}}[P_{1}(t+l)\beta(t+l)]+\widehat{E}[\widehat{p}(t)\widehat{g}^{*}_{\nu
a}(t)]\\\
&\quad+(p(t))^{2}f_{yy}(t)+\widehat{E}[\widehat{f}^{*}_{\mu_{1}a_{1}}(t)]+(p(t))^{2}\widehat{E}[\widehat{f}^{*}_{\mu_{2}a_{2}}(t)]\Big{)}dt-Q(t)dW(t),\
t\in[0,T),\\\ P(t)&=0,\ Q(t)=0,\ t\in[T,T+l],\\\
-dP_{1}(t)&=\Big{(}P_{1}(t)(\alpha(t)+\alpha(t-l)+f_{y}(t)+\widehat{E}[\widehat{f}^{*}_{\mu_{2}}(t)])+P(t)\beta(t)\Big{)}dt-
Q_{1}(t)dW(t),\ t\in[0,T),\\\ P_{1}(t)&=0,\ Q_{1}(t)=0,\ t\in[T,T+l],\\\
\end{aligned}\right.$ (6.15)

where

$\displaystyle(\widehat{g}^{*}_{\nu},\widehat{g}^{*}_{\nu a})(t)$
$\displaystyle=(\frac{\partial
g}{\partial\nu},\frac{\partial^{2}g}{\partial\nu\partial
a})(P_{X^{*}(t)};\widehat{X}^{*}(t)),$
$\displaystyle(\widehat{f}^{*}_{\mu_{i}},\widehat{f}^{*}_{\mu_{i}a_{i}})(t)$
$\displaystyle=((\frac{\partial
f}{\partial\mu})_{i},\frac{\partial}{\partial{a_{i}}}((\frac{\partial
f}{\partial\mu})_{i}))(t,\widehat{Y}^{*}(t),P_{(X^{*}(t),Y^{*}(t))},\widehat{u}^{*}(t);X^{*}(t),Y^{*}(t)),\
i=1,2.$

###### Remark 6.1

In (6.14), (6.15), we don’t give the specific expression of the derivatives of
the coefficients $g,f$ with respect to the measure. But we argue that it is
possible to show the accurate derivatives for some special $f,g$ with respect
to the measure. For example, let
$h_{0},h_{1}:\mathbb{R}\rightarrow\mathbb{R},\
h_{2}:\mathbb{R}^{2}\rightarrow\mathbb{R}$ be twice continuously
differentiable functions with bounded derivatives of all order, and
$f_{1}:[0,T]\times\mathbb{R}\times U\rightarrow\mathbb{R}$ satisfy assumptions
(H3.1)-(H3.3). For $\xi,\eta\in L^{2}(\Omega,\mathcal{F},P)$, define

$\displaystyle g(P_{\xi})$ $\displaystyle:=h_{0}(E[h_{1}(\xi)]),\quad
f(t,y,P_{(\xi,\eta)},v):=f_{1}(t,y,v)+h_{0}(E[h_{2}(\xi,\eta)]).$

Then

$\displaystyle\frac{\partial
g}{\partial\nu}(P_{\xi};a)=h_{0}^{\prime}(E[h_{1}(\xi)])h_{1}^{\prime}(a),\quad\frac{\partial^{2}g}{\partial\nu\partial
a}(P_{\xi};a)=h_{0}^{\prime}(E[h_{1}(\xi)])h_{1}^{\prime\prime}(a).$

For fixed $\eta\in L^{2}(\Omega,\mathcal{F},P)$,

$\displaystyle(\frac{\partial
f}{\partial\mu})_{1}(t,y,P_{(\xi,\eta)},v;a_{1},\eta)=h^{\prime}_{0}(E[h_{2}(\xi,\eta)])\frac{\partial
h_{2}}{\partial a_{1}}(a_{1},\eta),$ $\displaystyle\frac{\partial}{\partial
a_{1}}((\frac{\partial
f}{\partial\mu})_{1})(t,y,P_{(\xi,\eta)},v;a_{1},\eta)=h^{\prime}_{0}(E[h_{2}(\xi,\eta)])\frac{\partial^{2}h_{2}}{\partial
a_{1}^{2}}(a_{1},\eta).$

$(\frac{\partial f}{\partial\mu})_{2}$ and $\frac{\partial}{\partial
a_{2}}((\frac{\partial f}{\partial\mu})_{2})$ can be understood in the same
meaning.

##  7 Appendix

###  7.1 Notations

$\displaystyle M(t)$
$\displaystyle=p(t)(X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t))+\frac{1}{2}P(t)K^{\varepsilon}(t)+P_{1}(t)K^{\varepsilon}_{1}(t),$
$\displaystyle A_{1}(t)$ $\displaystyle=p(t)\Delta
b(t)+q(t)\Delta\sigma(t)+\frac{1}{2}P(t)(\Delta\sigma(t))^{2},$ $\displaystyle
A_{2}(t)$
$\displaystyle=(X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t))\Big{(}p(t)b_{x}(t)+q(t)\sigma_{x}(t)-F(t)\Big{)}+(X^{1,\varepsilon}(t-l)+X^{2,\varepsilon}(t-l))\cdot$
$\displaystyle\quad\
\Big{(}p(t)b_{x^{\prime}}(t)+q(t)\sigma_{x^{\prime}}(t)\Big{)}+\widehat{E}\Big{[}(\widehat{X}^{1,\varepsilon}(t)+\widehat{X}^{2,\varepsilon}(t))\Big{(}p(t)\widehat{b}_{\nu}(t)+q(t)\widehat{\sigma}_{\nu}(t)\Big{)}\Big{]},$
$\displaystyle A_{3}(t)$
$\displaystyle=\frac{1}{2}(X^{1,\varepsilon}(t))^{2}\Big{(}b_{xx}(t)p(t)+\sigma_{xx}(t)q(t)+2b_{x}(t)P(t)+(\sigma_{x}(t))^{2}P(t)+2\sigma_{x}(t)Q(t)-G(t)\Big{)}$
$\displaystyle\quad+\frac{1}{2}(X^{1,\varepsilon}(t-l))^{2}\Big{(}b_{x^{\prime}x^{\prime}}(t)p(t)+\sigma_{x^{\prime}x^{\prime}}(t)q(t)+(\sigma_{x^{\prime}}(t))^{2}P(t)+2b_{x^{\prime}}(t)P_{1}(t)$
$\displaystyle\quad+2\sigma_{x^{\prime}}(t)\sigma_{x}(t-l)P_{1}(t)+2\sigma_{x^{\prime}}(t)Q_{1}(t)\Big{)}+\frac{1}{2}\widehat{E}[(X^{1,\varepsilon}(t))^{2}(p(t)\widehat{b}_{\nu
a}(t)+q(t)\widehat{\sigma}_{\nu a}(t))],$ $\displaystyle\overline{A}_{3}(t)$
$\displaystyle=(X^{1,\varepsilon}(t)X^{1,\varepsilon}(t-l))\Big{\\{}b_{xx^{\prime}}(t)p(t)+\sigma_{xx^{\prime}}(t)q(t)+b_{x^{\prime}}(t)P(t)+\sigma_{x}(t)\sigma_{x^{\prime}}(t)P(t)+\sigma_{x^{\prime}}(t)Q(t)$
$\displaystyle\quad+b_{x}(t)P_{1}(t)+b_{x}(t-l)P_{1}(t)+\sigma_{x}(t)\sigma_{x}(t-l)P_{1}(t)+\sigma_{x}(t)Q_{1}(t)+\sigma_{x}(t-l)Q_{1}(t)-G_{1}(t)\Big{\\}},$
$\displaystyle A_{4}(t)$
$\displaystyle=\mathbbm{1}_{E_{\varepsilon}}(t)X^{1,\varepsilon}(t)\Big{\\{}p(t)\Delta
b_{x}(t)+q(t)\Delta\sigma_{x}(t)\Big{\\}}+\mathbbm{1}_{E_{\varepsilon}}(t)X^{1,\varepsilon}(t-l)\Big{\\{}p(t)\Delta
b_{x^{\prime}}(t-l)+q(t)\Delta\sigma_{x^{\prime}}(t-l)\Big{\\}}$
$\displaystyle\quad+\widehat{E}[\widehat{X}^{1,\varepsilon}(t)(p(t)\Delta\widehat{b}_{\nu}(t)+q(t)\Delta\widehat{\sigma}_{\nu}(t))]+\frac{1}{2}P(t)L_{1}(t)+\frac{1}{2}P_{1}(t)L_{3}(t),$
$\displaystyle B_{2}(t)$
$\displaystyle=(X^{1,\varepsilon}(t)+X^{2,\varepsilon}(t))\Big{(}p(t)\sigma_{x}(t)+q(t)\Big{)}+(X^{1,\varepsilon}(t-l)+X^{2,\varepsilon}(t-l))p(t)\sigma_{x^{\prime}}(t)$
$\displaystyle\quad\
\widehat{E}[(\widehat{X}^{1,\varepsilon}(t)+\widehat{X}^{2,\varepsilon}(t))p(t)\widehat{\sigma}_{\nu}(t)],$
$\displaystyle B_{3}(t)$
$\displaystyle=\frac{1}{2}(X^{1,\varepsilon}(t))^{2}\Big{(}p(t)\sigma_{xx}(t)+2P(t)\sigma_{x}(t)+Q(t)\Big{)}+\frac{1}{2}(X^{1,\varepsilon}(t-l))^{2}\Big{(}p(t)\sigma_{x^{\prime}x^{\prime}}(t)+2P_{1}(t)\sigma_{x^{\prime}}(t)\Big{)}$
$\displaystyle\quad+(X^{1,\varepsilon}(t)X^{1,\varepsilon}(t-l))\Big{(}p(t)\sigma_{xx^{\prime}}(t)+P(t)\sigma_{x^{\prime}}(t)+P_{1}(t)\sigma_{x}(t)+P_{1}(t)\sigma_{x}(t-l)+Q_{1}(t)\Big{)}$
$\displaystyle\quad+\frac{1}{2}\widehat{E}[p(t)\widehat{\sigma}_{\nu
a}(t)(\widehat{X}^{1,\varepsilon})^{2}],$ $\displaystyle\overline{B}_{3}(t)$
$\displaystyle=(X^{1,\varepsilon}(t)X^{1,\varepsilon}(t-l))\Big{(}p(t)\sigma_{x^{\prime}}(t)+P_{1}(t)\sigma_{x}(t)+P_{1}(t)\sigma_{x}(t-l)+Q_{1}(t)\Big{)},$
$\displaystyle B_{4}(t)$
$\displaystyle=\mathbbm{1}_{E_{\varepsilon}}(t)X^{1,\varepsilon}(t)p(t)\Delta\sigma_{x}(t)+\mathbbm{1}_{E_{\varepsilon}}(t)X^{1,\varepsilon}(t-l)p(t)\Delta\sigma_{x^{\prime}}(t)+\mathbbm{1}_{E_{\varepsilon}}(t)\widehat{E}[\Delta\widehat{\sigma}_{\nu}(t)\widehat{X}^{1,\varepsilon}]$
$\displaystyle\quad+\frac{1}{2}P(t)L_{2}(t)+\frac{1}{2}P_{1}(t)L_{4}(t).$

Here $F(t),G(t),G_{1}(t)$ are given in (3.44), (3.54); $L_{i}(t),i=1,2,3,4$
are introduced in (3.48) and (3.51).

###  7.2 The proof of (3.49)

In order to prove (3.49), we just need to prove the following three estimates
since the similar argument can be applied to calculate the rest terms:

$\displaystyle\mathrm{a)}\ E[(\int_{0}^{T}|X^{1,\varepsilon}(t)2\Delta
b(t)\mathbbm{1}_{E_{\varepsilon}}(t)|dt)^{2}]\leq\varepsilon^{2}\rho(\varepsilon);$
(7.1) $\displaystyle\mathrm{b)}\
E[(\int_{0}^{T}|2\sigma_{x^{\prime}}(t)X^{1,\varepsilon}(t-l)\widehat{E}[\widehat{\sigma}_{\nu}^{*}(t)\widehat{X}^{1,\varepsilon}(t)]|dt)^{2}]\leq\varepsilon^{2}\rho(\varepsilon);$
$\displaystyle\mathrm{c)}\
E[(\int_{0}^{T}|2\Delta\sigma(t)\widehat{E}[\widehat{\sigma}_{\nu}^{*}(t)\widehat{X}^{1,\varepsilon}(t)]\mathbbm{1}_{E_{\varepsilon}}(t)|dt)^{2}]\leq\varepsilon^{2}\rho(\varepsilon).$

For $\mathrm{a)}$, due to the coefficient $b$ being continuous with respect to
$v$, (3.41) and Hölder inequality allow to show

$\displaystyle E[(\int_{0}^{T}|X^{1,\varepsilon}(t)2\Delta
b(t)\mathbbm{1}_{E_{\varepsilon}}(t)|dt)^{2}]$ (7.2) $\displaystyle\leq
C\varepsilon^{2}\Big{\\{}E[\sup_{t\in[0,T]}|X^{1,\varepsilon}(t)|^{4}]\Big{\\}}^{\frac{1}{2}}\Big{\\{}E[\sup_{t\in[0,T]}|v(t)-u^{*}(t)|^{4}]\Big{\\}}^{\frac{1}{2}}\leq
C\varepsilon^{3}.$

As for $\mathrm{b)}$, from the boundness of $\sigma_{x^{\prime}}$, (3.40) and
(3.41), one can check

$\displaystyle
E[(\int_{0}^{T}|2\sigma_{x^{\prime}}(t)X^{1,\varepsilon}(t-l)\widehat{E}[\widehat{\sigma}_{\nu}^{*}(t)\widehat{X}^{1,\varepsilon}(t)]|dt)^{2}]$
(7.3) $\displaystyle\leq
CT^{\frac{3}{2}}\Big{\\{}E\Big{[}\sup_{t\in[0,T]}|X^{1,\varepsilon}(t-l)|^{4}\Big{]}\Big{\\}}^{\frac{1}{2}}\Big{\\{}E\Big{[}\int_{0}^{T}|\widehat{E}[\widehat{\sigma}_{\nu}^{*}(t)\widehat{X}^{1,\varepsilon}(t)]|^{4}dt\Big{]}\Big{\\}}^{\frac{1}{2}}\leq\varepsilon^{2}\rho(\varepsilon).$

We now analyse $\mathrm{c)}$. Notice that $\sigma$ is continuous in $v$, and
$\sigma_{\nu}$ is bounded, then (3.40) implies

$\displaystyle
E[(\int_{0}^{T}|2\Delta\sigma(t)\widehat{E}[\widehat{\sigma}_{\nu}^{*}(t)\widehat{X}^{1,\varepsilon}(t)]\mathbbm{1}_{E_{\varepsilon}}(t)|dt)^{2}]$
(7.4) $\displaystyle\leq
C\varepsilon^{2}E\Big{[}\sup_{t\in[0,T]}|X^{1,\varepsilon}(t)|^{2}\Big{]}E\Big{[}\sup_{t\in[0,T]}|v(t)-u^{*}(t)|^{2}\Big{]}\leq
C\varepsilon^{3}.$

###  7.3 The proof of (4.18)

_Proof of ( 4.18)._ Notice that

$\displaystyle
E[(\int_{0}^{T}|I_{2,1}(s)-I_{2,2}(s)|\mathbbm{1}_{E_{\varepsilon}}(s)ds)^{2}]$
(7.5)
$\displaystyle=E\Big{[}\Big{(}\int_{0}^{T}\mathbbm{1}_{E_{\varepsilon}}(s)\Big{(}|X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s)|+|X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l)|+|\breve{Y}(s)+M(s)|+|\breve{Z}(s)$
$\displaystyle\quad+B_{2}(s)+B_{3}(s)+\overline{B}_{3}(s)|+\\{E|\breve{Y}(s)+M(s)|^{2}\\}^{\frac{1}{2}}+\\{E|X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s)|^{2}\\}^{\frac{1}{2}}\Big{)}ds\Big{)}^{2}\Big{]}.$

We mainly prove

$\displaystyle
E\Big{[}\Big{(}\int_{0}^{T}\mathbbm{1}_{E_{\varepsilon}}(s)|\breve{Z}(s)+B_{2}(s)+B_{3}(s)+\overline{B}_{3}(s)|ds\Big{)}^{2}\Big{]}\leq\varepsilon^{2}\rho(\varepsilon),$
(7.6)

since the other terms can be argued similarly.
Observe that

$\displaystyle
E\Big{[}\Big{(}\int_{0}^{T}\mathbbm{1}_{E_{\varepsilon}}(s)|\breve{Z}(s)+B_{2}(s)+B_{3}(s)+\overline{B}_{3}(s)|ds\Big{)}^{2}\Big{]}$
(7.7) $\displaystyle\leq
CE[(\int_{0}^{T}\mathbbm{1}_{E_{\varepsilon}}(s)|\breve{Z}(s)|ds)^{2}]+CE[(\int_{0}^{T}\mathbbm{1}_{E_{\varepsilon}}(s)|B_{2}(s)|ds)^{2}]$
$\displaystyle\quad+CE[(\int_{0}^{T}\mathbbm{1}_{E_{\varepsilon}}(s)|B_{3}(s)|ds)^{2}]+CE[(\int_{0}^{T}\mathbbm{1}_{E_{\varepsilon}}(s)|\overline{B}_{3}(s)|ds)^{2}]\Big{\\}}$
$\displaystyle=:\triangle_{1}(\varepsilon)+\triangle_{2}(\varepsilon)+\triangle_{3}(\varepsilon)+\triangle_{4}(\varepsilon).$

From the definitions of $B_{3}$ and $\overline{B}_{3}$, it is not difficult to
check

$\bigtriangleup_{1}(\varepsilon)+\bigtriangleup_{3}(\varepsilon)+\bigtriangleup_{4}(\varepsilon)\leq\varepsilon^{2}\rho(\varepsilon).$
(7.8)

As for $\triangle_{2}(\varepsilon)$, it can be written as

$\displaystyle\bigtriangleup_{2}(\varepsilon)$ $\displaystyle\leq
CE[(\int_{0}^{T}\mathbbm{1}_{E_{\varepsilon}}(s)|(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))(p(s)\sigma_{x}(s)+q(s))|ds)^{2}]$
(7.9)
$\displaystyle+CE[(\int_{0}^{T}\mathbbm{1}_{E_{\varepsilon}}(s)|(X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l))p(s)\sigma_{x^{\prime}}(s)|ds)^{2}]$
$\displaystyle+CE[(\int_{0}^{T}\mathbbm{1}_{E_{\varepsilon}}(s)|\widehat{E}[(\widehat{X}^{1,\varepsilon}(s)+\widehat{X}^{2,\varepsilon}(s))p(s)\widehat{\sigma}_{\nu}(s)]|ds)^{2}]$
$\displaystyle=:\triangle_{2,1}(\varepsilon)+\triangle_{2,2}(\varepsilon)+\triangle_{2,3}(\varepsilon).$

However, from the boundness of $\sigma_{x}$, (3.41) and (3.45), it follows

$\displaystyle\triangle_{2,1}(\varepsilon)$ $\displaystyle\leq
CE\Big{[}(\sup_{t\in[0,T]}|X^{1,\varepsilon}(s)|^{2}+\sup_{t\in[0,T]}|X^{2,\varepsilon}(s)|^{2})\int_{0}^{T}|p(s)\sigma_{x}(s)+q(s)|^{2}\mathbbm{1}_{E_{\varepsilon}}(s)ds\cdot\varepsilon\Big{]}$
(7.10) $\displaystyle\leq
C\varepsilon\Big{(}\\{E[\sup_{t\in[0,T]}|X^{1,\varepsilon}(s)|^{4}]\\}^{\frac{1}{2}}+\\{E[\sup_{t\in[0,T]}|X^{2,\varepsilon}(s)|^{4}]\\}^{\frac{1}{2}}\Big{)}\cdot$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\quad\\{E[(\int_{0}^{T}|p(s)\sigma_{x}(s)+q(s)|^{2}\mathbbm{1}_{E_{\varepsilon}}(s)ds)^{2}]\\}^{\frac{1}{2}}$
$\displaystyle\leq
C\varepsilon(\varepsilon+\varepsilon^{2})\Big{(}\varepsilon+(E[(\int_{0}^{T}|q(s)|^{2}\mathbbm{1}_{E_{\varepsilon}}(s)ds)^{2}])^{\frac{1}{2}}\Big{)}.$

The Dominated Convergence Theorem implies
$(E[(\int_{0}^{T}|q(s)|^{2}\mathbbm{1}_{E_{\varepsilon}}(s)ds)^{2}])^{\frac{1}{2}}\rightarrow
0$ as $\varepsilon\downarrow 0$. Consequently,
$\rho_{1}(\varepsilon):=C(1+\varepsilon)\Big{(}\varepsilon+(E[(\int_{0}^{T}|q(s)|^{2}\mathbbm{1}_{E_{\varepsilon}}(s)ds)^{2}])^{\frac{1}{2}}\Big{)}\rightarrow
0$ as $\varepsilon\downarrow 0$. $\triangle_{2,2}(\varepsilon)$ can be
estimated with the similar argument.
We now turn to analyse $\triangle_{2,3}(\varepsilon)$. Notice

$\displaystyle\triangle_{2,3}(\varepsilon)$ $\displaystyle\leq
CE[(\int_{0}^{T}\mathbbm{1}_{E_{\varepsilon}}(s)|p(s)||\widehat{E}[\widehat{X}^{1,\varepsilon}(s)\widehat{\sigma}_{\nu}(s)]|ds)^{2}]+CE[(\int_{0}^{T}\mathbbm{1}_{E_{\varepsilon}}(s)|p(s)||\widehat{E}[\widehat{X}^{2,\varepsilon}(s)\widehat{\sigma}_{\nu}(s)]|ds)^{2}]$
(7.11)
$\displaystyle:=\triangle_{2,3,1}(\varepsilon)+\triangle_{2,3,2}(\varepsilon).$

Thanks to (3.40), we have

$\displaystyle\triangle_{2,3,1}(\varepsilon)$ $\displaystyle\leq
CE\Big{[}\sup_{s\in[0,T]}|p(s)|^{2}\cdot\varepsilon\cdot\int_{0}^{T}|\widehat{E}[\widehat{X}^{1,\varepsilon}(s)\widehat{\sigma}_{\nu}(s)|^{2}ds\Big{]}$
(7.12) $\displaystyle\leq
CT^{\frac{1}{2}}\varepsilon\Big{\\{}E[\int_{0}^{T}|\widehat{E}[\widehat{X}^{1,\varepsilon}(s)\widehat{\sigma}_{\nu}(s)|^{4}ds]\Big{\\}}^{\frac{1}{2}}\leq\varepsilon^{2}\rho_{2}(\varepsilon),$

where $\rho_{2}(\varepsilon)\rightarrow 0$ as $\varepsilon\rightarrow 0$.
Besides, obviously, $\triangle_{2,3,2}(\varepsilon)\leq C\varepsilon^{4}.$
Hence, we obtain
$\bigtriangleup_{2}(\varepsilon)\leq\varepsilon^{2}\rho(\varepsilon).$

###  7.4 The expansion of $I_{2,2}(s)$

The first-order Taylor expansion of $I_{2,2}(s)$ allows to show

$\displaystyle I_{2,2}$
$\displaystyle=f_{x}(s)(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))+f_{x^{\prime}}(s)(X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l))+f_{y}(s)(\breve{Y}(s)+M(s))$
(7.13)
$\displaystyle\quad+f_{z}(s)(\breve{Z}(s)+B_{2}(s)+B_{3}(s)+\overline{B}_{3}(s))+\widehat{E}[\widehat{f}_{\mu_{1}}(s)(\widehat{X}^{1,\varepsilon}(s)+\widehat{X}^{2,\varepsilon}(s))]$
$\displaystyle\quad+\widehat{E}[\widehat{f}_{\mu_{2}}(s)(\widehat{\breve{Y}}(s)+\widehat{M}(s))]+R(s),$

where

$\displaystyle R(s)$
$\displaystyle=\int_{0}^{1}d\rho\Big{\\{}(f^{\rho}_{x}(s)-f_{x}(s))(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))+(f^{\rho}_{x^{\prime}}(s)-f_{x^{\prime}}(s))(X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l))$
$\displaystyle\quad+(f^{\rho}_{y}(s)-f_{y}(s))(\breve{Y}(s)+M(s))+(f^{\rho}_{z}(s)-f_{z}(s))(\breve{Z}(s)+B_{2}(s)+B_{3}(s)+\overline{B}_{3}(s))$
$\displaystyle\quad+\widehat{E}[(\widehat{f}^{\rho}_{\mu_{1}}(s)-\widehat{f}_{\mu_{1}}(s))(\widehat{X}^{1,\varepsilon}(s)+\widehat{X}^{2,\varepsilon}(s))]+\widehat{E}[(\widehat{f}^{\rho}_{\mu_{2}}(s)-\widehat{f}_{\mu_{2}}(s))(\widehat{\breve{Y}}(s)+M(s))]\Big{\\}},$

and we define, for $0\leq\lambda\leq 1$,

$\displaystyle f^{\lambda}_{k}(s)$
$\displaystyle=f(s,X^{*}(s)+\lambda(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s)),X^{*}(s-l)+\lambda(X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l)),$
$\displaystyle\quad\
Y^{*}(s)+\lambda(\breve{Y}(s)+M(s)),Z^{*}(s)+\lambda(\breve{Z}(s)+B_{2}(s)+B_{3}(s)+\overline{B}_{3}(s)),$
$\displaystyle\quad\
P_{(X^{*}(s)+\lambda(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s)),Y^{*}(s)+\lambda(\breve{Y}(s)+M(s)))},u^{*}(s)).$

Recall the definitions of $M(s),B_{2}(s),B_{3}(s),\overline{B}_{3}(s)$,
$I_{2,2}$ changes to

$\displaystyle I_{2,2}(s)$
$\displaystyle=f_{y}(s)\breve{Y}(s)+f_{z}(s)\breve{Z}(s)+\widehat{E}[\widehat{f}_{\mu_{2}}(s)\widehat{\breve{Y}}(s)]+(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))\Big{\\{}f_{x}(s)+f_{y}(s)p(s)$
(7.14)
$\displaystyle\quad+f_{z}(s)(p(s)\sigma_{x}(s)+q(s))\Big{\\}}+(X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l))\Big{\\{}f_{x^{\prime}}(s)+f_{z}(s)p(s)\sigma_{x^{\prime}}(s)\Big{\\}}$
$\displaystyle\quad+\widehat{E}\Big{[}(\widehat{X}^{1,\varepsilon}(s)+\widehat{X}^{2,\varepsilon}(s))\Big{(}\widehat{\sigma}_{\nu}(s)f_{z}(s)p(s)+\widehat{f}_{\mu_{1}}(s)+\widehat{f}_{\mu_{2}}(s)\widehat{p}(s)\Big{)}\Big{]}$
$\displaystyle\quad+\frac{1}{2}(X^{1,\varepsilon}(s))^{2}\Big{\\{}f_{y}(s)P(s)+f_{z}(s)(p(s)\sigma_{xx}(s)+2P(s)\sigma_{x}(s)+Q(s))\Big{\\}}$
$\displaystyle\quad+\frac{1}{2}f_{z}(s)p(s)E\Big{[}(\widehat{X}^{1,\varepsilon})^{2}\Big{(}\widehat{\sigma}_{\nu
y}(s)+\widehat{f}_{\mu_{2}}(s)\widehat{P}(s)\Big{)}\Big{]}+\frac{1}{2}(X^{1,\varepsilon}(s-l))^{2}\Big{\\{}f_{z}(s)(p(s)\sigma_{x^{\prime}x^{\prime}}(s)$
$\displaystyle\quad+2P_{1}(s)\sigma_{x^{\prime}}(s))\Big{\\}}+X^{1,\varepsilon}(s)X^{1,\varepsilon}(s-l)\Big{\\{}f_{y}(s)P_{1}(s)+f_{z}(s)\Big{(}p(s)\sigma_{xx^{\prime}}(s)+P(s)\sigma_{x^{\prime}}(s)$
$\displaystyle\quad+P_{1}(s)\sigma_{x}(s)+P_{1}(s)\sigma_{x}(s-l)+Q_{1}(s)\Big{)}\Big{\\}}+E[\widehat{f}_{\mu_{2}}(s)\widehat{P}_{1}(s)(\widehat{X}^{1,\varepsilon}(s)\widehat{X}^{1,\varepsilon}(s-l))]+R(s).$

We want to prove
$E[(\int_{0}^{T}|R(s)ds|)^{2}]\leq\varepsilon^{2}\rho(\varepsilon).$ Firstly,
notice that

$\displaystyle\bullet$
$\displaystyle\int_{0}^{1}(f_{x}^{\rho}(s)-f_{x}(s))(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))d\rho$
(7.15)
$\displaystyle=\frac{1}{2}(X^{1,\varepsilon}(s))^{2}\Big{\\{}f_{xx}(s)+f_{xy}(s)p(s)+f_{xz}(s)(p(s)\sigma_{x}(s)+q(s))\Big{\\}}+\frac{1}{2}f_{xx^{\prime}}(s)X^{1,\varepsilon}(s-l)X^{1,\varepsilon}(s)$
$\displaystyle\quad+R_{1}(s,\varepsilon),$

where

$\displaystyle R_{1}(s,\varepsilon)$ $\displaystyle=\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\Big{\\{}f_{xx}^{\rho\theta}(s)(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))^{2}-f_{xx}(s)(X^{1,\varepsilon}(s))^{2}\Big{\\}}$
(7.16) $\displaystyle+\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\Big{\\{}f_{xx^{\prime}}^{\rho\theta}(s)(X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l))(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\quad-
f_{xx^{\prime}}(s)X^{1,\varepsilon}(s)X^{1,\varepsilon}(s-l)\Big{\\}}$
$\displaystyle+\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\Big{\\{}f_{xy}^{\rho\theta}(s)(\breve{Y}(s)+M(s))(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))-f_{xy}(s)p(s)(X^{1,\varepsilon}(s))^{2}\Big{\\}}$
$\displaystyle+\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\Big{\\{}f_{xz}^{\rho\theta}(s)(\breve{Z}(s)+B_{2}(s)+B_{3}(s)+\overline{B}_{3}(s))(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad-
f_{xz}(s)(p(s)\sigma_{x}(s)+q(s))(X^{1,\varepsilon}(s))^{2}\Big{\\}}$
$\displaystyle+\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\Big{\\{}\widehat{E}[\widehat{f}_{x\mu_{1}}^{\rho\theta}(\widehat{X}^{1,\varepsilon}(s)+\widehat{X}^{2,\varepsilon}(s))](X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))\Big{\\}}$
$\displaystyle+\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\Big{\\{}\widehat{E}[\widehat{f}_{x\mu_{2}}^{\rho\theta}(\widehat{\breve{Y}}(s)+\widehat{M}(s))](X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))\Big{\\}}.$

It is clear that

$E[(\int_{0}^{T}|R_{1}(s,\varepsilon)|ds)^{2}]\leq\varepsilon^{2}\rho(\varepsilon).$
(7.17)

In fact, the kernel to deal with (7.17) is to prove the following inequality
involving the second-order derivative $f_{x\mu_{2}}$ and
$X^{1,\varepsilon}(s)$:

$E\bigg{[}\int_{0}^{T}\Big{|}\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\Big{\\{}\widehat{E}[\widehat{f}_{x\mu_{2}}^{\rho\theta}(s)\widehat{p}(s)\widehat{X}^{1,\varepsilon}(s)]X^{1,\varepsilon}(s)\Big{\\}}\Big{|}^{2}ds\bigg{]}\leq\varepsilon^{2}\rho(\varepsilon).$
(7.18)

From Hölder inequality, Fubini theorem, (3.41) and (3.46), one can check

$\displaystyle E\Big{[}\int_{0}^{T}\Big{|}\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\Big{\\{}\widehat{E}[\widehat{f}_{x\mu_{2}}^{\rho\theta}(s)\widehat{p}(s)\widehat{X}^{1,\varepsilon}(s)]X^{1,\varepsilon}(s)\Big{\\}}\Big{|}^{2}ds\Big{]}$
(7.19) $\displaystyle\leq
CE\Big{[}\sup_{s\in[0,T]}|X^{1,\varepsilon}(s)|^{2}\cdot\int_{0}^{T}\int_{0}^{1}\rho\int_{0}^{1}\Big{|}\widehat{E}[\widehat{f}_{x\mu_{2}}^{\rho\theta}(s)\widehat{p}(s)\widehat{X}^{1,\varepsilon}(s)]\Big{|}^{2}d\theta
d\rho ds\Big{]}$ $\displaystyle\leq
C\Big{\\{}E\Big{[}\sup_{s\in[0,T]}|X^{1,\varepsilon}(s)|^{4}\Big{\\}}^{\frac{1}{2}}\cdot\Big{\\{}E\Big{[}\Big{(}\int_{0}^{T}\int_{0}^{1}\rho\int_{0}^{1}\Big{|}\widehat{E}[\widehat{f}_{x\mu_{2}}^{\rho\theta}(s)\widehat{p}(s)\widehat{X}^{1,\varepsilon}(s)]\Big{|}^{2}d\theta
d\rho ds\Big{)}^{2}\Big{]}\Big{\\}}^{\frac{1}{2}}$ $\displaystyle\leq
C\varepsilon\Big{\\{}\int_{0}^{1}\rho\int_{0}^{1}E\Big{[}\int_{0}^{T}\Big{|}\widehat{E}[\widehat{f}_{x\mu_{2}}^{\rho\theta}(s)\widehat{p}(s)\widehat{X}^{1,\varepsilon}(s)]\Big{|}^{4}ds\Big{]}d\theta
d\rho\Big{\\}}^{\frac{1}{2}}\leq\varepsilon^{2}\rho(\varepsilon).$

Similar to (7.15), it is easy to check

$\displaystyle\bullet\quad\int_{0}^{1}(f_{y}^{\rho}(s)-f_{y}(s))(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))d\rho$
(7.20)
$\displaystyle\qquad=\frac{1}{2}p(s)(X^{1,\varepsilon}(s))^{2}\Big{\\{}f_{yx}(s)+f_{yy}(s)p(s)+f_{yz}(s)(p(s)\sigma_{x}(s)+q(s))\Big{\\}}$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad+\frac{1}{2}p(s)f_{yx^{\prime}}(s)X^{1,\varepsilon}(s-l)X^{1,\varepsilon}(s)+R_{2}(s,\varepsilon);$
$\displaystyle\bullet\quad\int_{0}^{1}(f_{z}^{\rho}(s)-f_{z}(s))(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))d\rho$
$\displaystyle\qquad=\frac{1}{2}(p(s)\sigma_{x}(s)+q(s))(X^{1,\varepsilon}(s))^{2}\Big{\\{}f_{zx}(s)+f_{zy}(s)p(s)+f_{zz}(s)(p(s)\sigma_{x}(s)+q(s))\Big{\\}}$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad+\frac{1}{2}(p(s)\sigma_{x}(s)+q(s))f_{zx^{\prime}}(s)X^{1,\varepsilon}(s-l)X^{1,\varepsilon}(s)+R_{3}(s,\varepsilon);$
$\displaystyle\bullet\quad\int_{0}^{1}d\rho(f_{x^{\prime}}^{\rho}(s)-f_{x^{\prime}}(s))(X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l))$
$\displaystyle\qquad=\frac{1}{2}\Big{\\{}f_{x^{\prime}x}(s)(X^{1,\varepsilon}(s-l)X^{1,\varepsilon}(s))+f_{x^{\prime}x^{\prime}}(s)(X^{1,\varepsilon}(s-l))^{2}+f_{x^{\prime}y}(s)p(s)(X^{1,\varepsilon}(s-l)X^{1,\varepsilon}(s))$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad+f_{x^{\prime}z}(s)(p(s)\sigma_{x}(s)+q(s))(X^{1,\varepsilon}(s-l)X^{1,\varepsilon}(s))\Big{\\}}+R_{4}(s,\varepsilon),$

where
$E[(\int_{0}^{T}|R_{i}(s,\varepsilon)|ds)^{2}]\leq\varepsilon^{2}\rho(\varepsilon),\
i=2,3,4.$

In order to complete our proof, we still need to prove the following estimates

$\displaystyle\bullet\quad\int_{0}^{1}\widehat{E}[(\widehat{f}_{\mu_{1}}^{\rho}(s)-\widehat{f}_{\mu_{1}}(s))(\widehat{X}^{1,\varepsilon}(s)+\widehat{X}^{2,\varepsilon}(s))]d\rho=\frac{1}{2}\widehat{E}\Big{[}\widehat{f}_{\mu_{1}a_{1}}(s)(\widehat{X}^{1,\varepsilon}(s))^{2}\Big{]}+R_{5}(s,\varepsilon),$
(7.21)
$\displaystyle\bullet\quad\int_{0}^{1}\widehat{E}[(\widehat{f}_{\mu_{2}}^{\rho}(s)-\widehat{f}_{\mu_{2}}(s))(\widehat{\breve{Y}}(s)+\widehat{M}(s))]d\rho=\frac{1}{2}\widehat{E}\Big{[}\widehat{f}_{\mu_{2}a_{2}}(s)(\widehat{p}(s))^{2}(\widehat{X}^{1,\varepsilon}(s))^{2}\Big{]}+R_{6}(s,\varepsilon),$

where
$E[(\int_{0}^{T}|R_{5}(s,e)|+|R_{6}(s,e)|ds)^{2}]\leq\varepsilon^{2}\rho(\varepsilon)$.

We mainly deal with the first one in (7.21). The second one can be dealt with
the similar argument.
Notice that

$\displaystyle\int_{0}^{1}\widehat{E}[(\widehat{f}_{\mu_{1}}^{\rho}(s)-\widehat{f}_{\mu_{1}}(s))(\widehat{X}^{1,\varepsilon}(s)+\widehat{X}^{2,\varepsilon}(s))]d\rho-\frac{1}{2}\widehat{E}\Big{[}\widehat{f}_{\mu_{1}a_{1}}(s)(\widehat{X}^{1,\varepsilon}(s))^{2}\Big{]}$
(7.22)
$\displaystyle=\Gamma_{1}(s,\varepsilon)+\Gamma_{2}(s,\varepsilon)+\Gamma_{3}(s,\varepsilon)+\Gamma_{4}(s,\varepsilon),$

where

$\displaystyle\Gamma_{1}(s,\varepsilon):=\int_{0}^{1}d\rho\int_{0}^{1}d\theta\widehat{E}[\widehat{f}_{\mu_{1}a_{1}}^{\rho\theta}(s)(\widehat{X}^{1,\varepsilon}(s)+\widehat{X}^{2,\varepsilon}(s))^{2}]-\frac{1}{2}\widehat{E}\Big{[}\widehat{f}_{\mu_{1}a_{1}}(s)(\widehat{X}^{1,\varepsilon}(s))^{2}\Big{]},$
$\displaystyle\Gamma_{2}(s,\varepsilon):=\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\widehat{E}\Big{[}(\widehat{X}^{1,\varepsilon}(s)+\widehat{X}^{2,\varepsilon}(s))\widetilde{E}[\widetilde{\widehat{f}}_{\mu_{1}\mu_{1}}^{\rho\theta}(s)(\widetilde{X}^{1,\varepsilon}(s)+\widetilde{X}^{2,\varepsilon}(s))]\Big{]},$
$\displaystyle\Gamma_{3}(s,\varepsilon):=\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\widehat{E}\Big{[}(\widehat{X}^{1,\varepsilon}(s)+\widehat{X}^{2,\varepsilon}(s))\widetilde{E}[\widetilde{\widehat{f}}_{\mu_{1}\mu_{2}}^{\rho\theta}(s)(\widetilde{\breve{Y}}(s)+\widetilde{M}(s))]\Big{]},$
$\displaystyle\Gamma_{4}(s,\varepsilon):=\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\widehat{E}\Big{[}(\widehat{X}^{1,\varepsilon}(s)+\widehat{X}^{2,\varepsilon}(s))\Big{\\{}\widehat{f}_{\mu_{1}x}^{\rho\theta}(s)(X^{1,\varepsilon}(s)+X^{2,\varepsilon}(s))$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\quad+\widehat{f}_{\mu_{1}x^{\prime}}^{\rho\theta}(s)(X^{1,\varepsilon}(s-l)+X^{2,\varepsilon}(s-l))+\widehat{f}_{\mu_{1}y}^{\rho\theta}(s)(\breve{Y}(s)+M(s))$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\quad+\widehat{f}_{\mu_{1}z}^{\rho\theta}(s)(\breve{Z}(s)+B_{2}(s)+B_{3}(s)+\overline{B}_{3}(s))\Big{\\}}\Big{]}.$

Let us rewrite $\Gamma_{1}(s,\varepsilon)$ as follows

$\displaystyle\Gamma_{1}(s,\varepsilon)$
$\displaystyle=\int_{0}^{1}d\rho\int_{0}^{1}d\theta\widehat{E}[(\widehat{f}_{\mu_{1}a_{1}}^{\rho\theta}(s)-\widehat{f}_{\mu_{1}a_{1}}(s))(\widehat{X}^{1,\varepsilon}(s))^{2}]$
(7.23)
$\displaystyle\quad+\int_{0}^{1}d\rho\int_{0}^{1}d\theta\widehat{E}[\widehat{f}_{\mu_{1}a_{1}}^{\rho\theta}(s)(2\widehat{X}^{1,\varepsilon}(s)\widehat{X}^{2,\varepsilon}(s)+(\widehat{X}^{2,\varepsilon}(s))^{2}).$

From the Lipschitz property of $\widehat{f}_{\mu_{1}a_{1}}(s)$ and (3.41), one
can show
$E[(\int_{0}^{T}|\Gamma_{1}(s,\varepsilon)|ds)^{2}]\leq\varepsilon^{2}\rho(\varepsilon)$
easily.

As for $\Gamma_{3}(s,\varepsilon)$, according to (3.41) and (3.46), one has

$\displaystyle E\Big{[}\int_{0}^{T}\Big{|}\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\widehat{E}[\widehat{X}^{1,\varepsilon}(s)\widetilde{E}[\widetilde{\widehat{f}}_{\mu_{1}\mu_{2}}^{\rho\theta}(s)\widetilde{p}(s)\widetilde{X}^{1,\varepsilon}(s)]]\Big{|}^{2}ds\Big{]}$
(7.24) $\displaystyle\leq\frac{1}{2}E\Big{[}\int_{0}^{T}\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta
E[\sup_{s\in[0,T]}|X^{1,\varepsilon}(s)|^{2}]\cdot\widehat{E}\Big{|}\widetilde{E}[\widetilde{\widehat{f}}_{\mu_{1}\mu_{2}}^{\rho\theta}(s)\widetilde{p}(s)\widetilde{X}^{1,\varepsilon}(s)]\Big{|}^{2}ds\Big{]}$
$\displaystyle\leq\frac{1}{2}\varepsilon\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\Big{\\{}E\widehat{E}[\int_{0}^{T}|\widetilde{E}[\widetilde{\widehat{f}}_{\mu_{1}\mu_{2}}^{\rho\theta}(s)\widetilde{p}(s)\widetilde{X}^{1,\varepsilon}(s)]|^{2}ds]\Big{\\}}$
$\displaystyle\leq\frac{\sqrt{T}}{4}\varepsilon\int_{0}^{1}\rho
d\rho\int_{0}^{1}d\theta\Big{\\{}E\widehat{E}[\int_{0}^{T}|\widetilde{E}[\widetilde{\widehat{f}}_{\mu_{1}\mu_{2}}^{\rho\theta}(s)\widetilde{p}(s)\widetilde{X}^{1,\varepsilon}(s)]|^{4}ds]\Big{\\}}^{\frac{1}{2}}$
$\displaystyle\leq\varepsilon^{2}\rho(\varepsilon).$

$\Gamma_{2}(s,\varepsilon)$ can be calculated with the similar method. We now
turn our attention to $\Gamma_{4}(s,\varepsilon)$. The critical component of
$\Gamma_{4}(s,\varepsilon)$ is