# Necessary Conditions In Infinite-Horizon Control Problem
That Need No Asymptotic Assumptions

D. V. Khlopin
Email<EMAIL_ADDRESS>

Krasovskii Institute of Mathematics and Mechanics,
16 S.Kovalevskaja St.,Yekaterinburg, Russia

###### Abstract

We consider an infinite-horizon optimal control problem with asymptotic
terminal constraint. We propose a modification of Halkin’s general
construction of necessary conditions in which the terminal transversality
condition at infinity is obtained through the theorems on the stability of
subdifferentials. For the weakly overtaking criterion, a necessary boundary
condition on co-state arcs is deduced, without any assumptions about the
asymptotic behavior of the co-state arc and cost functional. In the absence of
an asymptotic terminal constraint (without any asymptotic assumptions), the
Pontryagin Maximum Principle with this boundary condition allows us to raise
some convex hull of the co-state arcs, corresponding to a convex
subdifferential of the payoff function (fixing the optimal control) at
infinity. In the case of the smooth cost functional at infinity (and under the
absence of an asymptotic terminal constraint), this condition raises the co-
state arc uniquely determined formula proposed by S. M. Aseev and A. V.
Kryazhimskii. These results are illustrated by some examples, including one
similar to the Ramsey–Cass–Koopmans model of an aggregate economy with an
arbitrary rate of discount.

Keywords: Infinite-horizon control problem, transversality condition on
infinity; overtaking optimal control; convergence of subdifferentials; optimal
growth

MSC2010 49K15, 49J53, 93C15, 91B62

##  1 Introduction

We will consider an infinite-horizon optimal control problem,

minimize $\displaystyle
l(y(0))+\int_{0}^{\infty}f_{0}(\tau,y(\tau),u(\tau))\,d\tau$ subject to
$\displaystyle\frac{d{y}(t)}{dt}=f(t,y(t),u(t))\quad\textrm{a.e.},$
$\displaystyle y(t)\in\mathbb{R}^{m},\ u(t)\in U,\ y(0)\in\mathcal{C}_{0},\
\operatorname*{Limsup}_{\theta\uparrow\infty}\\{y(\theta)\\}\subset\mathcal{C}_{\infty}$

with the relations of the Pontryagin Maximum Principle corresponding to this
problem:

$\displaystyle\frac{d{y}(t)}{dt}$ $\displaystyle=$ $\displaystyle
f\big{(}t,y(t),{u}(t)\big{)},$ (1a) $\displaystyle-\frac{d{\psi}(t)}{dt}$
$\displaystyle=$ $\displaystyle\frac{\partial{H}}{\partial
x}\big{(}y(t),\psi(t),{u}(t),\lambda,t\big{)},$ (1b)
$\displaystyle\sup_{\upsilon\in
U}H\big{(}y(t),\psi(t),\upsilon,\lambda,t\big{)}$ $\displaystyle=$
$\displaystyle H\big{(}y(t),\psi(t),{u}(t),\lambda,t\big{)}.$ (1c)

Here, ${\mathbb{R}}_{+}\stackrel{{\scriptstyle\triangle}}{{=}}[0;\infty)$ is
the time interval of the control system, the sets $\mathcal{C}_{0}$ and
$\mathcal{C}_{\infty}$ are nonempty subsets of
$\mathbb{X}\stackrel{{\scriptstyle\triangle}}{{=}}\mathbb{R}^{m}$, $l$ is a
scalar function on $\mathbb{X}$, the set $U$ of control parameters is a closed
subset of a certain finite-dimensional space, and the Hamilton–Pontryagin
function
$H:{\mathbb{X}}\times{\mathbb{X}}^{*}\times{U}\times{\mathbb{R}_{+}}\times{\mathbb{R}_{+}}\to{\mathbb{R}}$
is given by

$H(x,\psi,u,\lambda,t)\stackrel{{\scriptstyle\triangle}}{{=}}\psi
f\big{(}t,x,u\big{)}-\lambda
f_{0}\big{(}t,x,u\big{)}\quad\forall(x,\psi,u,\lambda,t)\in{\mathbb{X}}\times{\mathbb{X}}^{*}\times{U}\times{\mathbb{R}_{+}}\times{\mathbb{R}_{+}}.$

It is well known that the system (1a)–(1c) is the necessary condition for
finite-horizon control problems [1]. For infinite-horizon control problems,
the Pontryagin Maximum Principle was proved in the pioneering paper [2] in the
case of the finite optimality criterion (the optimality on each finite
interval of the corresponding problem with fixed ends). In this sense, each
optimal process as an extension of optimal processes for the finite horizon
problems, admits the extension of necessary conditions from each finite
interval to $\mathbb{R}_{+}$. By passing to the limit, one obtains the
necessary conditions on the whole $\mathbb{R}_{+}$, system (1a)–(1c) and the
usual transversality condition at zero.

Naturally, on the one hand, this proof does not require any supplementary
information on system (1a)–(1c) at infinity; on the other hand, nor is any
such knowledge gained. In particular, this system of necessary relations lacks
one more boundary condition on the co-state arc, which corresponds to the
transversality condition at the right end. In fact, without a such condition,
relations (1a)–(1c) only serve to point towards the variety of their solutions
without offering a tool to choose any one among them.

To limit the search, various supplementary conditions are used; otherwise, in
each specific problem, one could exhaustively search through all solutions [3,
4]. The limit value at infinity for the trajectory itself can be specified [1,
Subsect. 4.24]. The solution and/or control to remain within a certain class
of functions can be required [33, 32, 36]. It is also possible to connect some
condition with the value function [5, 6, 7]. One can try to apply some
relations due to the economic reasons [8] and one of convexity [9] and
stability [10, 11]. Nevertheless, some supplementary information (in the form
of boundary conditions) can be also tried to reclaim from a certain optimality
criterion. In this paper, we obtain such conditions for rather mild optimality
criteria such as the weakly overtaking criterion and overtaking criterion (the
optimality for the upper and lower pointwise limits of the cost functional,
respectively).

In [12] the necessary conditions, including a complete set of transversality
conditions for the co-state arc at infinity, were considered. In the case of a
final state dependent cost functional as well as of an asymptotic terminal
constraint of a linear structure, these conditions were proved under very
strong asymptopic assumptions on the trajectories and cost. In [13] a similar
result was shown under some asymptotic assumptions, guaranteeing the existence
of the limit of trajectories and their controllability at infinity. In [14]
the necessary transversality condition was obtained as a consequence of the
stability of the limiting subdifferentials with respect to the uniform
convergence. In this paper, this approach is applied to the derivation of
necessary conditions based on the stability of subdifferentials. It allows us,
while continuing to follow Halkin’s method, to reformulate the overtaking
criterion in these terms, and pass to the limit within the necessary
conditions for specially selected problems for the increasing sequence of time
intervals.

Adhering to this approach, in this paper we show two ways to derive some
necessary conditions. With this aim in mind, first, we take the classical
results [15] with respect to the Fréchet subdifferential for the epigraphical
convergence and apply them directly to the usual Pontryagin Maximum Principle
[16, 17] for some Bolza problem. Thus, we establish the necessary conditions
for the overtaking criteria (see Theorems 4.5 and 4.6), which notably require
no asymptotic assumptions on the adjoint system and/or gradients of the cost
functional.

The second way consists of proving the Pontryagin Maximum Principle for the
infinite-horizon control problem through an upper-estimate of subdifferentials
that remain stable with respect to the pointwise convergence. To prove this,
we refine the upper-estimate [18, Theorem 6.2] of the Fréchet subdifferential
of the pointwise upper limit in the case of smooth functions, which are
defined on a convex finite-dimensional subset (see Lemma 6.7). Subsequently,
similarly [19], we reduce the original control problem to the sequence of
constrained optimization problems in finite-dimensional spaces, whereby
admissible points for the finite-dimensional problems map into admissible
processes for the original problem. Finally, the Lagrange multiplier rule [20,
21] entails all desired necessary conditions of the original problem,
including all transversality conditions (see Theorems 4.1 and 4.2).

A special focus of this paper is the question of the accuracy of the
considered transversality conditions at infinity for the problem without
asymptotic constraints. If the cost functional gradients have a limit for
large time, the proposed transversality condition (7) is explicit, i.e. it
points to a unique solution of the adjoint system of (1b); moreover, this
solution coincides with the co-state arc representation [22] proposed by A. V.
Kryazhimskii and S. M. Aseev. On the other hand, in the case of the periodic
functional, even for an infinite-horizon control problem linear in $x$ (see
Example 5.3), there is no hope to construct any explicit transversality
condition on $\psi$ necessary for the weakly overtaking optimal criterion;
each such condition is going to contain a ball. However, in this example, the
proposed transversality condition (7) is the strongest of all consistent with
(1c) conditions, depending only on the co-state arc.

The rest of the paper is organised as follows. First, we introduce the
statement of the infinite-horizon control problem, the dynamics and cost
functional, and formulate the basic hypotheses on them; we also define all
needed optimality criteria. In Section 3, we recall the concepts and notions
of variational analysis (cones, subdifferentials, and epigraphical limits).
Section 4 exhibits formulations of all theorems and corollaries and the
discussion of their assumptions and conditions. The subsequent section is
devoted to examples. Subdifferentials of lower and upper limits of Lipschitz
continuous functions defined on convex finite-dimensional sets are considered
in Section 6. The last part of the paper (Sections 7,8) contains the proofs of
all theorems.

##  2 The statement of the infinite-horizon control problem

Let ${\mathbb{R}}_{+}=[0;\infty)$ be the time interval of an investigated
control system, and let ${\mathbb{X}}={\mathbb{R}}^{m}$ be its state space.
Let some boundary sets $\mathcal{C}_{0}\subset{\mathbb{X}}$ and
$\mathcal{C}_{\infty}\subset{\mathbb{X}}$ be given. Let a closed subset $U$ of
a finite-dimensional real vector space also be given; denote by $\mathcal{U}$
the set of all admissible controls, all Borel measurable functions
$u:\mathbb{R}_{+}\to U$.

Consider the following infinite-horizon optimal control problem:

minimize $\displaystyle
l(y(0))+\int_{0}^{\infty}f_{0}(\tau,y(\tau),u(\tau))\,d\tau$ (2a) subject to
$\displaystyle\frac{d{y}(t)}{dt}=f(t,y(t),u(t))\ \quad\textrm{a.e.},$ (2b)
$\displaystyle y(t)\in\mathbb{X}=\mathbb{R}^{m},\quad
y(0)\in\mathcal{C}_{0},\quad\operatorname*{Limsup}_{\theta\uparrow\infty}\\{y(\theta)\\}\subset\mathcal{C}_{\infty},\quad
u\in\mathcal{U}.$ (2c)

Hereinafter, we assume the following conditions to hold:

  * (H0)

$l:\mathbb{X}\to\mathbb{R}$ is a locally Lipschitz continuous scalar function
and ${\mathcal{C}}_{0}$ is a closed subset of ${\mathbb{X}}$;

  * (H1)

$f:\mathbb{R}_{+}\times\mathbb{X}\times U\to\mathbb{X}$ and
$f_{0}:\mathbb{R}_{+}\times\mathbb{X}\times U\to\mathbb{R}$ are continuous in
$(x,u)$ and Borel measurable in $t$;

  * (H2)

for all $(t,x)\in\mathbb{R}\times\mathbb{X}$ and control $u\in\mathcal{U}$,
there exists a unique generated by ${u}$ solution
$y\in(AC)(\mathbb{R}_{+},\mathbb{X})$ of (2b) satisfying $y(t)=x$; further,
the map $t\mapsto f_{0}(t,y(t),u(t))$ is locally summable (summable on each
compact interval).

Under these hypotheses, for every admissible control $u\in{\mathcal{U}}$, time
$t\in\mathbb{R}_{+}$, and initial state $x\in{\mathbb{X}}$, there exists a
defined on ${\mathbb{R}_{+}}$ unique solution
$y(\cdot)=\mathrm{y}(x,t,u;\cdot)$ of (2b) with the initial condition
$y(t)=x$. Let us now introduce a scalar function $J$ as follows:

$J(x,t,u;\theta)\stackrel{{\scriptstyle\triangle}}{{=}}\int_{t}^{\theta}f_{0}\big{(}\tau,\mathrm{y}(x,t,u;\tau),u(\tau)\big{)}\,d\tau\quad\forall
x\in\mathbb{X},u\in\mathcal{U},t\in\mathbb{R}_{+},\theta\geq t.$

A pair $(y,u)\in C(\mathbb{R}_{+},\mathbb{X})\times{\mathcal{U}}$ is called an
admissible control process if $y(\cdot)=\mathrm{y}(y(0),0,u;\cdot)$,
$y(0)\in{\mathcal{C}}_{0}$, and
$\operatorname*{Limsup}_{\theta\uparrow\infty}\\{y(\theta)\\}\subset\mathcal{C}_{\infty}$
hold (the symbol $\operatorname*{Limsup}$ is defined in Section 3, see (4)).

Later we will use the following optimality criteria:

######  Definition 2.1.

Call an admissible process $({\hat{y}},{\hat{u}})$ overtaking optimal [10, 23]
for problem (2a)–(2c) if for every admissible process $(y,u)$ it holds that

$\displaystyle
l(y(0))-l({\hat{y}}(0))+\liminf_{\theta\uparrow\infty}\int_{0}^{\theta}\big{[}f_{0}(\tau,y(\tau),u(\tau))-f_{0}(\tau,{\hat{y}}(\tau),{\hat{u}}(\tau))\big{]}\,d\tau\geq
0.$ (3a)

######  Definition 2.2.

Call an admissible process $({\hat{y}},{\hat{u}})$ weakly overtaking optimal
[10, 23] for problem (2a)–(2c) if for every admissible process $(y,u)$ it
holds that

$\displaystyle
l(y(0))-l({\hat{y}}(0))+\limsup_{\theta\uparrow\infty}\int_{0}^{\theta}\big{[}f_{0}(\tau,y(\tau),u(\tau))-f_{0}(\tau,{\hat{y}}(\tau),{\hat{u}}(\tau))\big{]}\,d\tau\geq
0.$ (3b)

Clearly, an overtaking optimal process is weakly overtaking optimal, but we
will relax the last criterion, considering merely its local variant [24].

######  Definition 2.3.

Call an admissible process $({\hat{y}},{\hat{u}})$ locally weakly overtaking
optimal for problem (2a)–(2c) if for every positive $T$ there exists a
positive $\beta(T)$ such that, for each admissible process $(y,u)$, from

$\displaystyle\max_{t\in[0;T]}||y(t)-{\hat{y}}(t)||+\operatorname*{meas}\\{t\in[0;T]\mid
u(t)\neq{\hat{u}}(t)\\}<\beta(T),\ u|_{[T;\infty)}={\hat{u}}|_{[T;\infty)}$
(3c)

there follows inequality (3b).

Hereinafter, we assume that a certain admissible control process
$({\hat{y}},{\hat{u}})$ is locally weakly overtaking optimal for problem
(2a)–(2c). Some conditions for the existence of weakly overtaking optimal and
overtaking optimal processes are given in [25, 26, 27]. In this paper, the
existence theory is not directly concerned. We also do not concern ourselves
with sufficient optimality conditions (see [9, 10, 5, 28]).

For brevity, let us also introduce

${\hat{J}}(x;\theta)\stackrel{{\scriptstyle\triangle}}{{=}}J(x,0,{\hat{u}};\theta)\qquad\forall\theta\geq
0,x\in\mathbb{X}.$

Later we will also consider the following hypotheses:

  * (H3)

there exists a neighbourhood ${\hat{\mathbb{G}}}_{\exists}\subset\mathbb{X}$
of the point ${\hat{y}}(0)$ such that the map
${\hat{\mathbb{G}}}_{\exists}\times\mathbb{R}_{+}\ni(x,\theta)\mapsto\frac{\partial{\hat{J}}(x;\theta)}{\partial
x}$ is well defined and continuous;

  * (H4)

there exist a neighbourhood
${\hat{\mathbb{G}}}_{\partial}\subset\mathbb{R}_{+}\times\mathbb{X}$ of the
graph of ${\hat{y}}$ and a locally summable function
${\hat{L}}:\mathbb{R}_{+}\to\mathbb{R}_{+}$ such that maps
${\hat{\mathbb{G}}}_{\partial}\ni(t,x)\mapsto\frac{\partial f}{\partial
x}(t,x,{\hat{u}}(t))$ and
${\hat{\mathbb{G}}}_{\partial}\ni(t,x)\mapsto\frac{\partial f_{0}}{\partial
x}(t,x,{\hat{u}}(t))$ are well defined and majorized by the function
${\hat{L}}$;

  * (H5)

for each $u\in U$, there exists a neighbourhood
$\mathbb{G}_{u}\subset\mathbb{R}_{+}\times\mathbb{X}$ of the graph of
${\hat{y}}$ such that the maps $(t,x)\mapsto f(t,x,u)$ and $(t,x)\mapsto
f_{0}(t,x,u)$ are continuously differentiable in $x$ on this neighbourhood;

  * (H5’)

for each $u\in U$, there exist a neighbourhood
$\mathbb{G}^{\prime}_{u}\subset\mathbb{R}_{+}\times\mathbb{X}$ of the graph of
${\hat{y}}$ and a locally summable function
$L_{u}:\mathbb{R}_{+}\to\mathbb{R}_{+}$ such that for all $(t,x^{\prime})$,
$(t,x^{\prime\prime})\in\mathbb{G}^{\prime}_{u}$ one has
$\big{\|}f(t,x^{\prime},u)-{f}(t,x^{\prime\prime},u)\big{\|}+\big{\|}{f_{0}}(t,x^{\prime},u)-{f_{0}}(t,x^{\prime\prime},u)\big{\|}\leq
L_{u}(t)\|x^{\prime}-x^{\prime\prime}\|.$

##  3 Some definitions from set-valued and variational analyses

We will use elementary notions from the set-valued and variational analyses
[29, 30, 31].

Consider an nonempty set ${\mathcal{X}}$ of real Euclidean space $\mathbb{X}$.
Let $\operatorname*{co}{{\mathcal{X}}}$, $\operatorname*{cl}{{\mathcal{X}}}$,
and $\operatorname*{int}{{\mathcal{X}}}$ denote the convex hull, closure, and
interior of ${{\mathcal{X}}}$. The affine hull of ${{\mathcal{X}}}$, the
smallest affine set that includes ${{\mathcal{X}}}$, is denoted by
$\operatorname*{aff}{{\mathcal{X}}}$ . The interior of ${{\mathcal{X}}}$
relative to its affine hull is the relative interior of ${{\mathcal{X}}}$; it
is denoted by $\operatorname*{ri}{{\mathcal{X}}}$ [29, Ch. 2.H]. The symbol
$\delta_{\mathcal{X}}$ denotes the indicator function of the set
${\mathcal{X}}$; this function has value $0$ on ${\mathcal{X}}$, but $+\infty$
elsewhere. Recall also that the sequential Painlevé–Kuratowski upper limit of
a set-valued map $F:\mathbb{X}\rightrightarrows\mathbb{X}$ at a point
$x\in\mathbb{X}$ is

$\operatorname*{Limsup}_{z\to
x}F(z)\stackrel{{\scriptstyle\triangle}}{{=}}\bigcap_{\varkappa>0}\operatorname*{cl}\bigcup_{||z-x||<\varkappa}F(z)\\\
=\\{\zeta\in\mathbb{X}\mid\exists\ \textrm{sequences of}\ z_{k}\to x,\
\zeta_{k}\to\zeta\ \textrm{with}\ \zeta_{k}\in F(z_{k})\ \textrm{for all}\
k\in\mathbb{N}\\}.$ (4)

By $[1\\!:\\!n]$ denote the set $\\{1,2,\dots,n\\}$ for each $n\in\mathbb{N}$.

For a point $x\in{{\mathcal{X}}}$, the contingent (Bouligand tangent) cone to
${{\mathcal{X}}}$ at $x$ is the set $T(x;{{\mathcal{X}}})$ of all
$v\in\mathbb{X}$ such that there exist a decreasing to $0$ sequence of
positive $t_{n}$ and a converging to $v$ sequence of $v_{n}\in\mathbb{X}$
satisfying $x+t_{n}v_{n}\in{{\mathcal{X}}}$ for all natural $n$. For a
nonnegative $\varepsilon\geq 0$ and a point $x\in\mathbb{X}$, we say that
$\zeta\in\mathbb{X}^{*}$ is $\varepsilon$-normal to ${{\mathcal{X}}}$ at $x$
if one has $x\in{{\mathcal{X}}}$ and

$\limsup_{z_{n}\to x}\frac{\zeta(z_{n}-x)}{||z_{n}-x||}\leq\varepsilon$

for all sequences of $z_{n}\in{{\mathcal{X}}}$ converging to $x$. Denote by
${\hat{N}}_{\varepsilon}(x;{{\mathcal{X}}})$ the set of all
$\varepsilon$-normals to ${{\mathcal{X}}}$ at $x$; put
${\hat{N}}_{\varepsilon}(x;{{\mathcal{X}}})\stackrel{{\scriptstyle\triangle}}{{=}}\varnothing$
if $x\in\mathbb{X}\setminus{{\mathcal{X}}}$. The sequential
Painlevé–Kuratowski upper limit of
${\hat{N}}_{\varepsilon}(z;{{\mathcal{X}}})$ as $z\to x,\varepsilon\downarrow
0$ is the set

${N}(x;{{\mathcal{X}}})\stackrel{{\scriptstyle\triangle}}{{=}}\operatorname*{Limsup}_{z\to
x,\ \varepsilon\downarrow 0}{\hat{N}}_{\varepsilon}(z;{{\mathcal{X}}}),$

which is called the limiting (basic, Mordukhovich) normal cone to
${{\mathcal{X}}}$ at $x$. For any cone $K\subset\mathbb{R}^{m}$, the polar
cone of $K$ is defined to be the cone $\\{\zeta\in\mathbb{R}^{m}\mid\forall
v\in K\ \zeta v\leq 0\\}.$ Due to [30, Theorem 1.10] and [29, Corollary 6.21],
since $\mathbb{X}$ is finite-dimensional, the cones
${\hat{N}}_{0}(x;{{\mathcal{X}}})$ and
$\operatorname*{co}\operatorname*{cl}T(x;{{\mathcal{X}}})$ are the polar cones
of $T(x;{{\mathcal{X}}})$ and ${\hat{N}}_{0}(x;{{\mathcal{X}}})$,
respectively.

Consider an extended-real-valued function
$g:\mathbb{X}\to\mathbb{R}\cup\\{-\infty,+\infty\\}.$ Define its epigraph
$\operatorname*{epi}g\stackrel{{\scriptstyle\triangle}}{{=}}\\{(x,a)\in\mathbb{X}\times\mathbb{R}\mid
a\geq g(x)\\}$. Also, recall that the lower semicontinuous envelope of the
function $g$ is defined as follows:

$\operatorname*{lsc}g(x)\stackrel{{\scriptstyle\triangle}}{{=}}\liminf_{\varkappa\downarrow
0}\inf_{||x-z||<\varkappa}g(z)\qquad\forall x\in\mathbb{X}.$

Note that this function is lower semicontinuous.

For a point $x\in\mathbb{X}$ with $|g(x)|<+\infty$, define the limiting
(basic, Mordukhovich) subdifferential [30, Definition 1.77(i)] of $g$ at $x$
as $\displaystyle\partial
g(x)\stackrel{{\scriptstyle\triangle}}{{=}}\\{\zeta\in\mathbb{X}^{*}\mid(\zeta,-1)\in
N(x,g(x);\operatorname*{epi}g)\\},$ (5a) the singular limiting subdifferential
[30, Definition 1.77(ii)] of $g$ at $x$ as
$\displaystyle\partial^{\infty}g(x)\stackrel{{\scriptstyle\triangle}}{{=}}\\{\zeta\in\mathbb{X}^{*}\mid(\zeta,0)\in
N(x,g(x);\operatorname*{epi}g)\\},$ (5b) and the Fréchet (regular)
subdifferential [31, (1.36)] of $g$ at $x$ as
$\displaystyle{\hat{\partial}}g(x)\stackrel{{\scriptstyle\triangle}}{{=}}\\{\zeta\in\mathbb{X}^{*}\mid(\zeta,-1)\in{\hat{N}}_{0}(x,g(x);\operatorname*{epi}g)\\}.$
(5c) Put $\partial
g(x)=\partial^{\infty}g(x)={\hat{\partial}}g(x)=\varnothing$ if
$|g(x)|=\infty$.

Note that, since $\mathbb{X}$ is finite-dimensional, for a lower
semicontinuous around $x$ function $g$, according to [30, Theorem 1.6], one
has

$N(x,g(x);\operatorname*{epi}g)=\operatorname*{Limsup}_{z\to x,\ g(z)\to
g(x)}{\hat{N}}_{0}(z,g(z);\operatorname*{epi}g);$ (5d)

in particular, according to [31, (1.37) and (1.38)], a point $\zeta$ in
$\mathbb{X}^{*}$ lies in $\partial g(x)$ iff

$\textrm{one finds sequences of
}x_{n}\in\mathbb{X},\zeta_{n}\in{\hat{\partial}}g(x_{n})\textrm{ satisfying
}x_{n}\to x,\zeta_{n}\to\zeta,g(x_{n})\to g(x),$

and a point $\zeta$ in $\mathbb{X}^{*}$ lies in $\partial^{\infty}g(x)$ iff

$\textrm{one finds sequences of
}\lambda_{n}>0,x_{n}\in\mathbb{X},\zeta_{n}\in{\hat{\partial}}g(x_{n})\\\
\textrm{ satisfying }\lambda_{n}\downarrow 0,x_{n}\to
x,\lambda_{n}\zeta_{n}\to\zeta,g(x_{n})\to g(x).$

Also, in the case of Lipschitz continuous function $g$, one has
$\partial^{\infty}g(x)=\\{0\\}$, although $\partial g(x)$ is not empty and is
bounded [30, Corollary 1.81]; in addition,
${\hat{N}}_{0}(x,g(x);\operatorname*{epi}g)=\cup_{r>0}r({\hat{\partial}}W_{{\theta}}(x)\times\\{-1\\})$
and $\operatorname*{co}\partial g(x)=\operatorname*{co}(-\partial(-g))(x)$
hold by [29, Theorem 8.9] and by [31, (1.75) and (1.83)]), respectively.

Following [29, Sect. 7.3], for every sequence of
$g_{n}:{\mathbb{X}}\to\mathbb{R}$, denote by
$\operatorname*{e-liminf}_{n\uparrow\infty}g_{n}$ and
$\operatorname*{e-limsup}_{n\uparrow\infty}g_{n}$ the epigraphical lower and
upper limits of $g_{n}$:

$\operatorname*{e-liminf}_{n\uparrow\infty}g_{n}(x)\stackrel{{\scriptstyle\triangle}}{{=}}\lim_{\varkappa\downarrow
0}\liminf_{n\uparrow\infty}\inf_{||x-z||\leq\varkappa}g_{n}(z)\in\mathbb{R}\cup\\{-\infty,+\infty\\},\\\
\operatorname*{e-limsup}_{n\uparrow\infty}g_{n}(x)\stackrel{{\scriptstyle\triangle}}{{=}}\lim_{\varkappa\downarrow
0}\limsup_{n\uparrow\infty}\inf_{||x-z||\leq\varkappa}g_{n}(z)\in\mathbb{R}\cup\\{-\infty,+\infty\\}\qquad\forall
x\in\mathbb{X}.$

When these two functions coincide, we will say that the functions $g_{n}$ epi-
converge to
$\operatorname*{e-lim}_{n\uparrow\infty}g_{n}\stackrel{{\scriptstyle\triangle}}{{=}}\operatorname*{e-liminf}_{n\uparrow\infty}g_{n}=\operatorname*{e-limsup}_{n\uparrow\infty}g_{n}$.
Note that the epigraphical lower and upper limits of functions have to be
lower semicontinuous [29, Proposition 7.4(a)].

##  4 The main results

In this section we will formulate and discuss all main results of the article.

First, consider the homeward set for all generated by ${\hat{u}}$ motions that
passed the asymptotic constraint $\mathcal{C}_{\infty}$, which is the set

$\mathcal{C}_{\operatorname*{\textrm{home}}}\stackrel{{\scriptstyle\triangle}}{{=}}\big{\\{}x\in\mathbb{X}\,\big{|}\,\operatorname*{Limsup}_{\theta\uparrow\infty}\\{\mathrm{y}(x,0,{\hat{u}};\theta)\\}\subset\mathcal{C}_{\infty}\big{\\}}.$

This set will be used below instead of $\mathcal{C}_{\infty}$, because the
transversality conditions at infinity will also transfer at zero.

######  Theorem 4.1.

Under conditions $(H0)$–$(H5)$ let an admissible control process
$({\hat{y}},{\hat{u}})$ be locally weakly overtaking optimal for problem
(2a)–(2c). Let also $\mathcal{C}_{\operatorname*{\textrm{home}}}$ be convex.

Then, there exists a nonzero solution $({\hat{\psi}},{\hat{\lambda}})\in
C(\mathbb{R}_{+};\mathbb{X}^{*})\times\\{0,1\\}$ of the corresponding to
$({\hat{y}},{\hat{u}})$ system (1b)–(1c) with transversality conditions (6)
and (7):

$\displaystyle{\hat{\psi}}(0)\in{\hat{\lambda}}\partial
l({\hat{y}}(0))+N({\hat{y}}(0);{\mathcal{C}_{0}});$ (6)
$\displaystyle-{\hat{\psi}}(0)\in
N({\hat{y}}(0);\mathcal{C}_{\operatorname*{\textrm{home}}})+\operatorname*{co}\operatorname*{Limsup}_{\begin{subarray}{c}\theta_{n}\uparrow\infty,\
x_{n}\to{\hat{y}}(0),\ 0<\lambda_{n}\to{\hat{\lambda}},\\\
{\hat{J}}(x_{n};\theta_{n})-{\hat{J}}({\hat{y}}(0);\theta_{n})\to
0\end{subarray}}\Big{\\{}\lambda_{n}\frac{\partial{\hat{J}}}{\partial
x}(x_{n};\theta_{n})\Big{\\}}.$ (7)

######  Theorem 4.2.

Under conditions $(H0)$–$(H5)$ let an admissible control process
$({\hat{y}},{\hat{u}})$ be locally weakly overtaking optimal for problem
(2a)–(2c). Assume that the map

$\displaystyle\mathcal{C}_{\operatorname*{\textrm{home}}}\cap G\ni
x\in\limsup_{\theta\uparrow\infty}\Big{[}{\hat{J}}(x;\theta)-{\hat{J}}({\hat{y}}(0);\theta)\Big{]}$
(8)

is lower semicontinuous for some open relative to
$\operatorname*{aff}\mathcal{C}_{\operatorname*{\textrm{home}}}$ set $G$
containing ${\hat{y}}(0)$. Assume also the following constraint qualification
condition:

$\displaystyle
N({\hat{y}}(0);\mathcal{C}_{\operatorname*{\textrm{home}}})\cap\operatorname*{co}\operatorname*{Limsup}_{\begin{subarray}{c}\lambda_{n}\downarrow
0,\ \theta_{n}\uparrow\infty,\ G\ni x_{n}\to{\hat{y}}(0),\\\
{\hat{J}}(x_{n};\theta_{n})-{\hat{J}}({\hat{y}}(0);\theta_{n})\to
0\end{subarray}}\Big{\\{}-\lambda_{n}\frac{\partial{\hat{J}}}{\partial
x}(x_{n};\theta_{n})\Big{\\}}\subset\\{0\\}.$ (9)

Then the conclusion of Theorem 4.1 holds.

The proofs of these theorems are presented in Section 7.

######  Remark 1.

The condition of the convexity of
$\mathcal{C}_{\operatorname*{\textrm{home}}}$ is a strong requirement and its
verification is difficult in the general case. However, this verification can
be simple in the case of linear dynamics and in the one-dimensional case (see
Example 5.1). The verification in the absence of the asymptotic constraint
($\mathcal{C}_{\operatorname*{\textrm{home}}}=\mathbb{X}$) is also understood.

######  Remark 2.

Condition (7) can possess the continuum of solutions of (1b), but it is
inescapable. In Example 5.3, the cost functional oscillates at infinity. It
was enough that in the corresponding infinite-horizon control problem (with a
fixed initial state) the dimension of the family of optimal processes reaches
$\dim\mathbb{X}$. Since the dynamics and integrand in this example are also
linear in $x$, the same dimension is inevitable for an arbitrary
transversality condition on co-state arcs. This example demonstrates that for
the weakly overtaking optimal criterion, without additional asymptotic
assumptions, there is no hope to construct an explicit transversality
condition on co-state arcs consistent with (1c). However, in Example 5.3, each
optimal process system (1b), (1c), (7) has a unique solution; furthermore,
each co-state arc satisfies (7) iff this arc corresponds to some weakly
overtaking optimal process. Therefore, inclusion (7) is the tightest of all
conditions on co-state arcs for the weakly overtaking criterion.

######  Remark 3.

The lower semicontinuity of map (8) precludes some ill-posedness pathology in
the original problem. In particular, it guarantees the well-posedness of the
functional value in regard to small perturbation of the initial condition. The
lower semicontinuity of map (8) holds if this map (8) coincides with the
corresponding epigraphical (in particular, uniform) limit; see Remark 5 below.
One could also require the motion, control, and/or functional to remain within
a certain class of functions [32, 33, 12, 34, 35, 36].

However, the condition of the lower semicontinuity of map (8) can be verified
in terms of the problem data. Indeed, assume that the family of maps

$\displaystyle\mathbb{R}_{+}\ni
t\mapsto\Delta^{-}f_{0}(x;t)\stackrel{{\scriptstyle\triangle}}{{=}}\min(0,f_{0}(t,\mathrm{y}(x,0,{\hat{u}};t),{\hat{u}}(t))-f_{0}(t,{\hat{y}}(t),{\hat{u}}(t))),\
x\in G,$ (10)

is strongly uniformly integrable [25, Definition 2.3], i.e., for all positive
$\varepsilon$, there exists a summable function
$m_{\varepsilon}:\mathbb{R}_{+}\to\mathbb{R}_{+}$ satisfying

$\displaystyle\inf_{x\in G}\int_{\\{t\geq 0\,|\,\Delta^{-}f_{0}(x;t)\leq-
m_{\varepsilon}(t)\\}}\Delta^{-}f_{0}(x;t)\,dt>-\varepsilon.$ (11)

In the case of the finite total variation of ${\hat{J}}({\hat{y}}(0),\cdot)$
on $\mathbb{R}_{+}$ (i.e., the map $\mathbb{R}_{+}\ni t\mapsto
f_{0}(t,{\hat{y}}(t),{\hat{u}}(t))$ is summable), the family of maps (10) is
strongly uniformly integrable iff for some summable function
$\bar{m}_{\varepsilon}:\mathbb{R}_{+}\to\mathbb{R}_{+}$ one has

$\inf_{y(\cdot)=\mathrm{y}(x,0,{\hat{u}};\cdot),x\in G\ \ }\int_{\\{t\geq
0\,|\,f_{0}(t,y(t),{\hat{u}}(t))\leq-\bar{m}_{\varepsilon}(t)\\}}\min\big{(}0,f_{0}(t,y(t),{\hat{u}}(t))\big{)}\,dt>-\varepsilon.$

Due to [25, Theorem 2.6], assumption (11) implies the lower semicontinuity of
map (8). The strong uniform integrability of $f_{0}$ is similar to a lot of
assumptions [25, (3.24)], [26, ($\Pi 8$)], [27, (xii))], [35, $(I_{\int}7)$],
which are inherent in the existence results.

In the absence of asymptotic constraints, under the uniform bounded gradients
$\frac{\partial{\hat{J}}}{\partial x}(x;\theta)$, necessary condition (7) is
deduced for an overtaking criterion in [14] and [37]. However, Theorem 4.1 and
Theorem 4.2 entail, respectively, the stronger corollaries.

######  Corollary 4.3.

Under conditions $(H0)$–$(H5)$ let a process $({\hat{y}},{\hat{u}})$ be
locally weakly overtaking optimal for problem (2a)–(2c). Let also
$\mathcal{C}_{\operatorname*{\textrm{home}}}=\mathbb{X}$.

Then the conclusion of Theorem 4.1 holds.

Furthermore, if the maps $x\mapsto\frac{\partial{\hat{J}}}{\partial
x}(x;\theta)$, $\theta>0$, are uniformly bounded on some neighbourhood of
${\hat{y}}(0)$, one can assume ${\hat{\lambda}}=1$; in addition, for all
$\varkappa>0$, for all natural $i\in[1\\!:\\!m+1]$, there exist a time instant
$\theta_{i}>1/\varkappa$, a point $x_{i}\in\mathbb{X}$, and a number
$\alpha_{i}\geq 0$ such that one has $\sum_{k=1}^{m+1}\alpha_{k}=1$,
$\big{\|}{\hat{\psi}}(0)+\sum_{k=1}^{m+1}\alpha_{k}\frac{\partial{\hat{J}}}{\partial
x}({x}_{i};\theta_{i})\big{\|}<\varkappa$, $\|x_{i}-{\hat{y}}(0)\|<\varkappa$,
and
$\|{\hat{J}}({x}_{i};\theta_{i})-{\hat{J}}({\hat{y}}(0);\theta_{i})\|<\varkappa$.

######  Corollary 4.4.

Under conditions $(H0)$–$(H5)$ let a process $({\hat{y}},{\hat{u}})$ be
locally weakly overtaking optimal for problem (2a)–(2c). Let also the maps
$x\mapsto\frac{\partial{\hat{J}}}{\partial x}(x;\theta)$, $\theta>0$, be
uniformly bounded on some open relative to
$\operatorname*{aff}\mathcal{C}_{\operatorname*{\textrm{home}}}$ set
containing ${\hat{y}}(0)$.

Then the conclusion of Theorem 4.1 holds.

Similarly the last corollary, both the following theorems require softer
conditions on $\mathcal{C}_{\operatorname*{\textrm{home}}}$ and stricter
conditions on ${\hat{J}}$.

######  Theorem 4.5.

Under conditions $(H0)$–$(H4),(H5^{\prime})$ let the process
$({\hat{y}},{\hat{u}})$ be weakly overtaking optimal for problem (2a)–(2c).
Let $\mathcal{C}_{\operatorname*{\textrm{home}}}$ be closed. Assume also that
there is a neighbourhood $G$ of ${\hat{y}}(0)$ such that, for all
$x\in\mathcal{C}_{\operatorname*{\textrm{home}}}\cap G$, one has

$\limsup_{\theta\uparrow\infty}\Big{[}{\hat{J}}(x;\theta)-{\hat{J}}({\hat{y}}(0);\theta)\Big{]}=\operatorname*{e-limsup}_{\theta\uparrow\infty}\Big{[}{\hat{J}}(x;\theta)-{\hat{J}}({\hat{y}}(0);\theta)+\delta_{\mathcal{C}_{\operatorname*{\textrm{home}}}}(x)\Big{]}.$
(12)

Then, there exists a nonzero solution $({\hat{\psi}},{\hat{\lambda}})\in
C(\mathbb{R}_{+};\mathbb{X}^{*})\times\\{0,1\\}$ of the corresponding to
$({\hat{y}},{\hat{u}})$ system (1b)–(1c) with transversality conditions (6)
and (13):

$\displaystyle-{\hat{\psi}}(0)\in
N({\hat{y}}(0);\mathcal{C}_{\operatorname*{\textrm{home}}})+\operatorname*{co}\operatorname*{Limsup}_{\theta_{n}\uparrow\infty,\
x_{n}\to{\hat{y}}(0),\
0<\lambda_{n}\to{\hat{\lambda}}}\Big{\\{}\lambda_{n}\frac{\partial{\hat{J}}}{\partial
x}(x_{n};\theta_{n})\Big{\\}}.$ (13)

######  Theorem 4.6.

Under conditions $(H0)$–$(H4),(H5^{\prime})$ let the process
$({\hat{y}},{\hat{u}})$ be overtaking optimal for problem (2a)–(2c). Let
$\mathcal{C}_{\operatorname*{\textrm{home}}}$ be closed. Assume also that
there is a neighbourhood $G$ of ${\hat{y}}(0)$ such that, for all
$x\in\mathcal{C}_{\operatorname*{\textrm{home}}}\cap G$, one has

$\liminf_{\theta\uparrow\infty}\Big{[}{\hat{J}}(x;\theta)-{\hat{J}}({\hat{y}}(0);\theta)\Big{]}=\operatorname*{e-liminf}_{\theta\uparrow\infty}\Big{[}{\hat{J}}(x;\theta)-{\hat{J}}({\hat{y}}(0);\theta)+\delta_{\mathcal{C}_{\operatorname*{\textrm{home}}}}(x)\Big{]}.$
(14)

Then, there exists a nonzero solution $({\hat{\psi}},{\hat{\lambda}})\in
C(\mathbb{R}_{+};\mathbb{X}^{*})\times\\{0,1\\}$ of the corresponding to
$({\hat{y}},{\hat{u}})$ system (1b)–(1c) with transversality conditions (6)
and (15):

$\displaystyle-{\hat{\psi}}(0)\in
N({\hat{y}}(0);\mathcal{C}_{\operatorname*{\textrm{home}}})+\bigcap_{\begin{subarray}{c}(\theta_{n})_{n\in\mathbb{N}}\in\mathbb{R}_{+}^{\mathbb{N}},\\\
\theta_{n}\uparrow\infty\end{subarray}}\operatorname*{Limsup}_{\begin{subarray}{c}\mathcal{C}_{\operatorname*{\textrm{home}}}\ni
x_{n}\to{\hat{y}}(0),\ 0<\lambda_{n}\to{\hat{\lambda}},\\\
{\hat{J}}(x_{n};\theta_{n})-{\hat{J}}({\hat{y}}(0);\theta_{n})\to
0\end{subarray}}\Big{\\{}\lambda_{n}\frac{\partial{\hat{J}}}{\partial
x}(x_{n};\theta_{n})\Big{\\}}.$ (15)

The proofs of these theorems are also presented in Section 8.

######  Remark 4.

Theorems 4.5 and 4.6 assume the closedness of
$\mathcal{C}_{\operatorname*{\textrm{home}}}.$ This requirement holds if the
set $\mathbb{X}\setminus\mathcal{C}_{\infty}$ is open and strong invariant
with respect to the dynamics $\frac{dy(t)}{dt}=f(t,y(t),{\hat{u}}(t))$, i.e.,
for all $x\in\mathbb{X}$, $t\in\mathbb{R}_{+}$ satisfying
$\mathrm{y}(x,0,{\hat{u}};t)\notin\mathcal{C}_{\infty}$, one has
$\mathrm{y}(x,0,{\hat{u}};\theta)\notin\mathcal{C}_{\infty}$ for all
$\theta>t$.

######  Remark 5.

Equalities (12) and (14) can be verified in terms of the problem data. Assume
that for all points
$\check{x}\in\mathcal{C}_{\operatorname*{\textrm{home}}}\cap G$ the family of
maps $\mathcal{C}_{\operatorname*{\textrm{home}}}\ni x\to{\hat{J}}(x;\theta)$
($\theta>0$) is equi-lsc [29, Section 7.C] at a point $\check{x}$ relative to
$\mathcal{C}_{\operatorname*{\textrm{home}}}$, i.e., for all positive
$\varepsilon$ there exists a positive $\theta_{0}$ and a neighbourhood
$G^{\prime}\subset G$ of the point $\check{x}$ satisfying
${\hat{J}}(\check{x};\theta)\geq{\hat{J}}({x};\theta)-\varepsilon$ for all
$\theta>\theta_{0}$ and $x\in
G^{\prime}\cap\mathcal{C}_{\operatorname*{\textrm{home}}}$. Due to [29,
Theorem 7.10], this property for all maps
$\mathcal{C}_{\operatorname*{\textrm{home}}}\ni x\to{\hat{J}}(x;\theta)$
($\theta>0$) at all points
$\check{x}\in\mathcal{C}_{\operatorname*{\textrm{home}}}\cap G$ leads to (12)
and (14).

In particular, these equalities hold if the maps ${\hat{J}}(\cdot;\theta)$
uniformly converge as $\theta\uparrow\infty$.

######  Remark 6.

In the definitions of the overtaking criterion and weakly overtaking
criterion, the time parameter $\theta$ tends to infinity arbitrarily. We could
fix some unbounded set $\mathbb{T}$ and consider these definitions with the
additional restriction $\theta_{n}\in\mathbb{T}$. We could apply Theorem 4.5
and Theorem 4.6 to such definitions, but this restriction should have been
added in conditions (7), (13), and (15), maps (8), (12), and (14). In
particular, this idea could be very useful in the case of equicontinuity of
the family of ${\hat{J}}(\cdot;\theta)-{\hat{J}}({\hat{y}}(0);\theta)$ because
it ensures the uniform (and epigraphical) convergence of
${\hat{J}}(\cdot;\theta_{n})-{\hat{J}}({\hat{y}}(0);\theta_{n})$ for some
elaborated sequence of $\theta_{n}$.

######  Remark 7.

The conclusion of Theorem 4.6 also holds if the process
$({\hat{y}},{\hat{u}})$ is uniformly overtaking optimal [38, Proposition 2.1].
The corresponding modification for some unbounded sequence of $\theta_{n}$
allows us to apply this conclusion to weakly uniformly overtaking criterion
[38]. In both cases, the condition (14), or (12) can be excluded from
consideration.

Note that neither theorem implies the other. Theorem 4.1 allows one to avoid
the verification of maps (8), (12), or (14); this theorem is the best in the
absence of any asymptotic assumption. Theorem 4.2 requires the minimal
condition on (8) and the hard to its verification condition (9); however this
theorem implies Corollary 4.3, the best result in the case of globally
Lipschitz continuous data. Theorem 4.5 demands the soft condition on
$\mathcal{C}_{\operatorname*{\textrm{home}}}$ and the strict condition (12) on
${\hat{J}}$; whilst it also requires no asymptotic assumptions on the adjoint
system and/or gradients of the cost functional; note that hypothesis
$(H5^{\prime})$ is also weaker than $(H5)$. However, its transversality
condition (13) is also weaker than (7) because it drops the requirement
${\hat{J}}(x;\theta)-{\hat{J}}({\hat{y}}(0);\theta)\to 0$. Theorem 4.6
essentially refines its transversality condition, but its optimality criterion
is also stronger.

At last, if an asymptotic terminal constraint is active, assuming the
convexity of $\mathcal{C}_{\operatorname*{\textrm{home}}}$, we formally
require the asymptotic assumption on the dynamics. However, without this
assumption, some weak transversality condition also can be proved (see Remark
8 below).

Consider now in more detail the infinite-horizon control problem with free
right endpoint, i.e., the case of the absence of asymptotic constraints
($\mathcal{C}_{\infty}=\mathcal{C}_{\operatorname*{\textrm{home}}}=\mathbb{X}$).
Corollary 4.3 allows us to apply (7) without any asymptotic assumptions, but
this condition may be satisfied by a continuum of the co-state arcs. Consider
another approach: let us start by searching for an explicit transversality
condition, i.e., an asymptotic condition that would select exactly one co-
state arc for each optimal process.

For this purpose, [12, Theorem 8.1] proposed to find ${\hat{\psi}}$ such that
it is a pointwise limit of a sequence of the co-state arcs that equal zero on
certain unbounded sequence of time instants $\theta_{n}$. The corresponding
necessary condition was proved for the infinite-horizon control problem under
some strong assumptions on asymptotics of $\mathrm{y}$, $J$ and their
gradients. Under these assumptions the proposed condition is equivalent to
(7). Later, for the same purpose, in [22] and then in [34, 39, 40, 38, 24, 36,
41, 42], many assumptions on the asymptotic behavior of $f,f_{0},J$, and their
gradients were considered. Under these assumptions, some solution
$({\hat{\psi}},{\hat{\lambda}})$ of (1b)–(1c) is determined by the following
formula:

$-{\hat{\psi}}(0)=\lim_{\theta\uparrow\infty}\int_{0}^{\theta}\frac{\partial
f_{0}}{\partial
x}\big{(}\tau,{\hat{y}}(\tau),{\hat{u}}(\tau)\big{)}\,{\hat{A}}(\tau)\,d\tau,\
{\hat{\lambda}}=1.$ (16a) Here, ${\hat{A}}\in
C({\mathbb{R}_{+}},{\mathbb{R}^{\dim\mathbb{X}\times\dim\mathbb{X}}})$ is the
solution of the Cauchy problem $\frac{d{{\hat{A}}}(t)}{dt}=\frac{\partial
f}{\partial
x}\big{(}t,{\hat{y}}(t),{\hat{u}}(t)\big{)}{\hat{A}}(t),\quad{\hat{A}}(0)=E.$
Let us also note two equivalent representations of this formula. The first
one, obtained in [40], is expressed as
$\lim_{\theta\uparrow\infty}{\hat{\psi}}(\theta){\hat{A}}(\theta)=0,\
{\hat{\lambda}}=1$ (16b) closely echoes the the famous Shell’s condition [8,
5] and the Arrow-like condition [5, 9, 28]. The second equivalent to (16a)
expression
$-{\hat{\psi}}(0)=\lim_{\theta\uparrow\infty}\frac{\partial{\hat{J}}}{\partial
x}({\hat{y}}(0);\theta),\ {\hat{\lambda}}=1$ (16c) is useful in light of
conditions (7) and [12, (38b)].

As shown below in Example 5.2, condition (16a) may be inconsistent with system
(1b), (1c) corresponding to an overtaking optimal process when the gradient at
the initial state of the limit of ${\hat{J}}$ does not coincide with the limit
of gradients of ${\hat{J}}$ at this state. This commutativity as a basic
hypothesis for deducing some transversality condition was considered, in
particular, in [43, (3.4)]. Under similar assumptions, the corresponding
results in [34, 39, 40, 38, 41, 14, 42] do not imply the following corollary.

######  Corollary 4.7.

Let a process $({\hat{y}},{\hat{u}})$ be locally weakly uniformly overtaking
optimal for problem (2a)–(2c) with $\mathcal{C}_{\infty}=\mathbb{X}$. Let
there also exist a finite limit

$\lim_{\begin{subarray}{c}\theta\uparrow\infty,\ x\to{\hat{y}}(0),\\\
{\hat{J}}(x;\theta)-{\hat{J}}({\hat{y}}(0);\theta)\to
0\end{subarray}}\frac{\partial{\hat{J}}}{\partial x}(x;\theta).$ (17)

Then, the system of relations (1b)–(1c), (16a) has exactly one solution
$({\hat{\psi}},{\hat{\lambda}})$. Moreover, this solution also satisfies
conditions (6), (16b), (16c), and

$\inf_{u\in\mathcal{U}}\liminf_{\theta\uparrow\infty}\Big{[}{H}\Big{(}{\hat{y}}(t),-\frac{\partial
J}{\partial x}({\hat{y}}(t),t,{\hat{u}};\theta),{\hat{u}}(t),1,t\Big{)}\\\
-{H}\Big{(}{\hat{y}}(t),-\frac{\partial J}{\partial
x}({\hat{y}}(t),t,{\hat{u}};\theta),u(t),1,t\Big{)}\Big{]}\geq 0\quad\ {a.a.\
}t\geq 0.$ (18)

###### Proof.

Note that the existence and the finiteness of the limit in (16a) is an
immediate consequence of (17) and equalities

$\frac{\partial\mathrm{y}}{\partial
x}(x,0,{\hat{u}};\theta)={\hat{A}}(\theta),\ \frac{\partial{\hat{J}}}{\partial
x}(x;\theta)=\int_{0}^{\theta}\frac{\partial f_{0}}{\partial
x}\big{(}\tau,{\hat{y}}(\tau),{\hat{u}}(\tau)\big{)}\,{\hat{A}}(\tau)\,d\tau\quad\forall\theta\in{\mathbb{R}_{+}}.$
(19)

By Theorem 4.1, one can find a solution $({\hat{\psi}},1)$ of the
corresponding to $({\hat{y}},{\hat{u}})$ system (1b)–(1c) such that
$-{\hat{\psi}}(0)$ is a convex combination of partial limits of
$\frac{\partial{\hat{J}}}{\partial x}(x_{n};\theta_{n})$ for certain sequences
$x_{n}\to{\hat{y}}(0),\theta_{n}\uparrow\infty$ satisfying
$J(x_{n},0,{\hat{u}};\theta_{n})-J({\hat{y}}(0),0,{\hat{u}};\theta_{n})\to 0$.
Then, by (17), it is also the limit of $\frac{\partial{\hat{J}}}{\partial
x}({\hat{y}}(0);\theta)$ as $\theta\uparrow\infty$. So, we have proved (16c).
Now, from (19), we see that (16a) holds for ${\hat{\psi}}$. Note that
condition (16a) allows us to reconstruct ${\hat{\psi}}$ uniquely. At the same
time, (1c) holds for all $t\geq 0$ except a possibly empty subset
$\mathcal{N}\subset\mathbb{R}_{+}$ of zero measure. Fix this set.

Note that, since ${\hat{\psi}}$ as a solution of (1b) satisfies the Cauchy
formula

${\hat{\psi}}(\theta){\hat{A}}(\theta)-{\hat{\psi}}(t){\hat{A}}(t)=\int_{t}^{\theta}\frac{\partial
f_{0}}{\partial
x}\big{(}\tau,{\hat{y}}(\tau),{\hat{u}}(\tau)\big{)}\,{\hat{A}}(\tau)\,d\tau\qquad\forall\theta>0,$
(20)

the passage to the limit as $\theta\uparrow\infty$ with $t=0$ leads to (16b).

Before we proceed further, we claim that

$\lim_{\theta\uparrow\infty}\frac{\partial J}{\partial
x}(x,t,{\hat{u}};\theta)\Big{|}_{x={\hat{y}}(t)}=-{\hat{\psi}}(t)\qquad\forall
t>0.$ (21)

Fix a nonnegative $t$. One has the equality
${\hat{J}}(x;\theta)-{\hat{J}}(x;t)=J\big{(}\mathrm{y}(x,0,{\hat{u}};t),t,{\hat{u}};\theta\big{)}$
for each positive $\theta>t$. Differentiating it in $x$ at ${\hat{y}}(0)$, we
have

$\displaystyle\int_{t}^{\theta}\frac{\partial f_{0}}{\partial
x}\big{(}\tau,{\hat{y}}(\tau),{\hat{u}}(\tau)\big{)}\,{\hat{A}}(\tau)\,d\tau$
$\displaystyle\stackrel{{\scriptstyle(\ref{4C})}}{{=}}\frac{\partial{\hat{J}}}{\partial
x}(x;\theta)\Big{|}_{x={\hat{y}}(0)}-\frac{\partial{\hat{J}}}{\partial
x}(x;t)\Big{|}_{x={\hat{y}}(0)}$ $\displaystyle\;=\;\frac{\partial
J(\mathrm{y}(z,0,{\hat{u}};t),t,{\hat{u}};\theta)}{\partial
z}\Big{|}_{z={\hat{y}}(0)}\stackrel{{\scriptstyle(\ref{4C})}}{{=}}\frac{\partial
J}{\partial x}(x,t,{\hat{u}};\theta)\Big{|}_{x={\hat{y}}(t)}{\hat{A}}(t).$

Combining it with (20) leads to $\frac{\partial J}{\partial
x}({\hat{y}}(t),t,{\hat{u}};\theta)=\big{(}{\hat{\psi}}(\theta){\hat{A}}(\theta)-{\hat{\psi}}(t){\hat{A}}(t)\big{)}{\hat{A}}^{-1}(t).$
Passing to the limit as $\theta\uparrow\infty$, by (16b), we obtain (21) for
all nonnegative $t$.

Let us prove condition (18). Suppose it is false. Then, there exists a
$\tau\in\mathbb{R}_{+}\setminus\mathcal{N}$, a $u\in\mathcal{U}$, an
$\varepsilon>0$, and an unboundedly increasing sequence of $\theta_{n}>0$
satisfying

$\ {H}\Big{(}{\hat{y}}(\tau),-\frac{\partial J}{\partial
x}({\hat{y}}(\tau),\tau,{\hat{u}};\theta_{n}),{\hat{u}}(\tau),1,\tau\Big{)}\leq{H}\Big{(}{\hat{y}}(\tau),-\frac{\partial
J}{\partial
x}({\hat{y}}(\tau),\tau,{\hat{u}};\theta_{n}),u(\tau),1,\tau\Big{)}-\varepsilon.$

By (21), ${\hat{\psi}}(\tau)$ is the pointwise limit of $-\frac{\partial
J}{\partial x}({\hat{y}}(\tau),\tau,{\hat{u}};\theta_{n})$ as
$n\uparrow\infty$; therefore, one could have
${H}\big{(}{\hat{y}}(\tau),{\hat{\psi}}(\tau),{\hat{u}}(\tau),1,\tau\big{)}\leq{H}\big{(}{\hat{y}}(\tau),{\hat{\psi}}(\tau),u(\tau),1,\tau\big{)}-\varepsilon.$
This would contradict condition (1c) for
$\tau\in\mathbb{R}_{+}\setminus\mathcal{N}$. Thus, condition (18) has been
proved. ∎

##  5 Examples

The first example was inspired by the classical Ramsey–Cass–Koopmans model of
an aggregate economy [3]. The one-dimensional case of this model for an
arbitrary discount rate $\varrho$ could be reduced to Example 5.1. In
addition, on the one hand, this example allows us entirely and directly to
apply Theorem 4.1. Along the road, we will consider some outfit of tools and
methods, different from the direct calculation of any motions, costs and their
gradients. On the other hand, we will see that these tools and methods are not
simple and user-friendly, and so should be improved.

######  Example 5.1.

Let functions $f_{0}:\mathbb{R}\to\mathbb{R}$ and $f:\mathbb{R}\to\mathbb{R}$
be given. Fix an arbitrary number $\varrho\in\mathbb{R}$ and nonnegative
number $x_{*}$.

Let $({\hat{y}},{\hat{u}})$ be a locally weakly overtaking optimal solution of
the following problem:

minimize $\displaystyle\int_{0}^{\infty}e^{-\varrho\tau}f_{0}(u(\tau))\,d\tau$
subject to $\displaystyle\frac{d{y}(t)}{dt}=f(y(t))-u(t)\ \textrm{a.e.},$
$\displaystyle y(0)=x_{*}\geq 0,\
y(t)\in\mathbb{X}\stackrel{{\scriptstyle\triangle}}{{=}}\mathbb{R},\ u(t)\in
U\stackrel{{\scriptstyle\triangle}}{{=}}\mathbb{R}_{+},\
\operatorname*{Limsup}_{\theta\uparrow\infty}\\{y(\theta)\\}\subset\mathbb{R}_{+}.$

First, assume that $f$ and $f_{0}$ are $C^{2}$-smooth on $\mathbb{R}_{+}$, the
function $f_{0}$ is strongly decreasing, and $f(0)$ is nonpositive. Denote by
$x_{**}$ the minimal nonnegative root of the equation $f(x)=0$. Then, we can
guarantee that $x_{*}\geq x_{**}$ because $({\hat{y}},{\hat{u}})$ is
admissible. In the case $x_{*}=x_{**}$, the process $(x_{*},{\hat{u}}=0)$ is
the unique admissible control process. Then, one can consider $x_{*}>x_{**}.$

Since $\mathbb{X}=\mathbb{R}$ is one-dimensional, the set
$\mathcal{C}_{\operatorname*{\textrm{home}}}$ has to be an interval unbounded
above. So, $\mathcal{C}_{\operatorname*{\textrm{home}}}$ is convex. Hence, by
Theorem 4.1, there exists a nonzero solution $({\hat{\psi}},{\hat{\lambda}})$
of the relations (1b), (1c), and (7).

We claim that ${\hat{\psi}}(t)$ is nonzero for each nonnegative $t$. Suppose
that ${\hat{\psi}}(t)=0$ at some $t$. Then, by (1b), the co-state arc
${\hat{\psi}}$ as solution $\frac{d\psi(t)}{dt}=-\psi(t)\frac{\partial
f({\hat{y}}(t))}{\partial x}$ should be equalled to zero, and, by (1c),
control $\upsilon={\hat{u}}(t)$ should minimize strongly decreasing function
$\upsilon\mapsto f_{0}(\upsilon)$; therefore, this control should be unbounded
for almost all $t\geq 0$. Hence $u\notin\mathcal{U}$, a contradiction. Thus,
${\hat{\psi}}(t)$ has to be nonzero for all nonnegative $t$.

Since $f_{0}$ is independent of $x$,
$\frac{\partial{\hat{J}}(x;\theta)}{\partial x}$ is zero whenever $x,\theta$.
It follows from $\eqref{WAKK}$ that $-{\hat{\psi}}(0)$ lies in
$N(x_{*};\mathcal{C}_{\operatorname*{\textrm{home}}}).$ Since
${\hat{\psi}}\not\equiv 0$, the point $x_{*}={\hat{y}}(0)$ lies on the
boundary of $\mathcal{C}_{\operatorname*{\textrm{home}}};$ further, one has
$\mathcal{C}_{\operatorname*{\textrm{home}}}=[x_{*};\infty)$ and
$N(x_{*};\mathcal{C}_{\operatorname*{\textrm{home}}})=-\mathbb{R}_{+}$. Thus,
${\hat{\psi}}$ is positive.

In the case of ${\hat{\lambda}}=0$, the value of ${\hat{\psi}}(t)\upsilon$
should attain a minimum for each $t$. Then, ${\hat{u}}(t)\equiv 0$ holds.
Thus, ${\hat{y}}\equiv x_{*}$. Since, for each control process $(x,u)$
different from $(x_{*},0)$, we have $J(x_{*},0,u;T)>J(x_{*},0,0;T)$ for $T$
large enough, the process $(x_{*},0)$ is unique admissible control process.
Thus, $x_{*}=x_{**}.$ This case was considered above.

Then, one has to consider the case ${\hat{\lambda}}=1$. In this case,
${\hat{\psi}}>0$ holds and the point $x_{*}={\hat{y}}(0)$ lies on the boundary
of $\mathcal{C}_{\operatorname*{\textrm{home}}}.$ Since $f(0)$ is nonpositive,
for each $\varepsilon>0$ small enough, the motion
$\mathrm{y}(x_{*}-\varepsilon,0,{\hat{u}};\cdot)$ must be caught in the
negative semi-axis for a sufficiently long passage of time. So, motion
${\hat{y}}$ is positive; furthermore, it is the minimal of all positive
motions generated by ${\hat{u}}$. However, since ${\hat{u}}$ and
${\hat{\psi}}$ are unknown, we have to seek out all positive ${\psi}(0)$,
calculate some solution $(y,\psi,u,1)$ of (1a)–(1c), and verify the
unstability of the sign of ${y}$. It is very difficult.

To further research assume only that

$\displaystyle f(0)\leq 0,\quad\frac{\partial^{2}f}{\partial x^{2}}(x)\leq
0,\quad\frac{\partial
f_{0}}{\partial\upsilon}(\upsilon)<0,\quad\textrm{and}\quad\frac{\partial^{2}f_{0}}{\partial\upsilon^{2}}(\upsilon)>0\qquad\forall
x>0,\upsilon>0.$

These conditions on the dynamics and integrand are typical in optimal growth
models; compare with [44, (1.5a)-(1.5c)] and [45, (3.19a)-(3.19d)].

Now, for all $p>0$, the value of $-p\upsilon-f_{0}(\upsilon)$ over all
$\upsilon>0$ attains its maximum at a unique point $\eta(p)>0$, which is the
unique solution of the equation $-p=\frac{\partial
f_{0}}{\partial\upsilon}(\upsilon)$; the corresponding map $p\mapsto\eta(p)$
is continuous, positive and decreasing.

Fix a small $\varepsilon>0$. Together with the motion
$y_{\varepsilon}(\cdot)\stackrel{{\scriptstyle\triangle}}{{=}}\mathrm{y}(x_{*}-\varepsilon,0,{\hat{u}};\cdot)$
consider the solution
$({\tilde{z}_{\varepsilon}}(\cdot),\tilde{q}(\cdot),\lambda=1,u(\cdot)=\eta(\tilde{q}(\cdot)e^{\varrho\cdot}))$
of (1a)–(1c) satisfying the initial conditions
${\tilde{z}_{\varepsilon}}(0)=x_{*}-2\varepsilon$,
$\tilde{q}(0)={\hat{\psi}}(0)-\varepsilon.$ Fix a natural $n>1/\varepsilon$.
Define $T_{n}\stackrel{{\scriptstyle\triangle}}{{=}}\sup\\{t\geq
0\,|\,\tilde{q}(t)<{\hat{\psi}}(t),\
1/n<{\tilde{z}_{\varepsilon}}(t)<y_{\varepsilon}(t)\\};$ note that $T_{n}$ is
positive. Define
$M_{n}(t)\stackrel{{\scriptstyle\triangle}}{{=}}\max\big{\\{}|\frac{\partial
f}{\partial x}(x)|\,\big{|}\,x\in[1/n;{\hat{y}}(t)]\big{\\}}$ for all
$t\in[0;T_{n}].$ First, for all positive $t<T_{n}$, according to
${\hat{\psi}}(t)>\tilde{q}(t)$,
$\varepsilon<{\tilde{z}_{\varepsilon}}(t)<y_{\varepsilon}(t)<{\hat{y}}(t)$,
and $\frac{\partial f}{\partial x}({\hat{y}}(t))\leq\frac{\partial f}{\partial
x}({\tilde{z}_{\varepsilon}}(t))$, we obtain

$\displaystyle\frac{d(\tilde{q}(t)-{\psi}(t))}{dt}={\hat{\psi}}(t)\frac{\partial
f}{\partial x}({\hat{y}}(t))-\tilde{q}(t)\frac{\partial f}{\partial
x}({\tilde{z}_{\varepsilon}}(t))\leq({\hat{\psi}}(t)-\tilde{q}(t))\frac{\partial
f}{\partial x}({\tilde{z}_{\varepsilon}}(t)).$

Hence one has ${\hat{\psi}}(t)-\tilde{q}(t)\geq\varepsilon
e^{-\int_{0}^{t}M_{n}(\tau)\,d\tau}$ and $\eta({\hat{\psi}}(t)e^{\varrho
t})<\eta(\tilde{q}(t)e^{\varrho t})$ for all $t\in[0;T_{n})$; in particular,
${\hat{\psi}}(T_{n})>\tilde{q}(T_{n})$. Second, for all $t<T_{n}$, we have

$\frac{d(y_{\varepsilon}(t)-{\tilde{z}_{\varepsilon}}(t))}{dt}=f({\tilde{z}_{\varepsilon}}(t))-f(y_{\varepsilon}(t))+\eta(\tilde{q}(t)e^{\varrho
t})-\eta({\hat{\psi}}(t)e^{\varrho
t})<M_{n}(t)(y_{\varepsilon}(t)-{\tilde{z}_{\varepsilon}}(t)).$

Then, we obtain
$y_{\varepsilon}(t)-{\tilde{z}_{\varepsilon}}(t)\geq\varepsilon
e^{-\int_{0}^{t}M_{n}(\tau)\,d\tau}$ for all $t<T_{n}$; furthermore,
$y_{\varepsilon}(T_{n})<{\tilde{z}_{\varepsilon}}(T_{n})$. According to
$\tilde{q}(T_{n})<{\hat{\psi}}(T_{n})$ and
${\tilde{z}_{\varepsilon}}(T_{n})<{y}_{\varepsilon}(T_{n})$, we obtain
${\tilde{z}_{\varepsilon}}(T_{n})=1/n$. We have also proved that
$y_{\varepsilon}(T_{n})>1/n$. Since each $T_{n}$ is less than the minimal
positive root of the equation $y_{\varepsilon}(t)=0$, all $T_{n}$ are bounded
and the equation ${\tilde{z}_{\varepsilon}}(t)=0$ also has a positive root.
Thus, ${\tilde{z}_{\varepsilon}}$ changes its sign for all positive
$\varepsilon$ small enough.

Accordingly, since
$||{\hat{y}}(0)-{\tilde{z}_{\varepsilon}}(0)||+||{\hat{\psi}}(0)-\tilde{q}(0)||$
can be chosen arbitrary small, for the solution
$(y(\cdot),p(\cdot))=({\hat{y}}(\cdot),{\hat{\psi}}(\cdot)e^{\varrho\cdot})$
of the following system

$\displaystyle\frac{dy(t)}{dt}=f(y(t))-\eta(p(t)),\
\frac{d{p}(t)}{dt}=\big{(}\varrho-\frac{\partial f}{\partial
x}(y(t))\big{)}p(t),$

the sign of ${\hat{y}}$ is positive, but unstable. Now, the control
${\hat{u}}$ and the co-state arc ${\hat{\psi}}$ are still unknown, but
solutions of just one dynamical system must be verified. In addition, one can
again use the strict concavity of $f$.

As was shown above, each $\eta(p)$ is the unique solution of the equation
$-p=\frac{\partial f_{0}}{\partial\upsilon}(\upsilon)$; hence we also have
$-1=\frac{\partial^{2}f_{0}}{\partial\upsilon^{2}}(\eta(p))\frac{\partial\eta}{\partial
p}(p).$ Each solution $(y,u,\psi,1)$ of (1a)–(1c) (with ${y}(0)>x_{**}$) due
to substitutions $\eta(\psi(t)e^{\varrho t})=u(t)$ and $\frac{\partial
f_{0}}{\partial\upsilon}(u(t))=-\psi(t)e^{\varrho t}$ also satisfies the
following system:

$\displaystyle\frac{dy(t)}{dt}=f(y(t))-u(t),\quad\frac{d{u}(t)}{dt}=\Big{(}\varrho-\frac{\partial
f}{\partial x}(y(t))\Big{)}\frac{\frac{\partial
f_{0}}{\partial\upsilon}(u(t))}{\frac{\partial^{2}f_{0}}{\partial\upsilon^{2}}(u(t))}.$
(22)

Hence the sign of ${\hat{y}}$ in the solution $(y,u)=({\hat{y}},{\hat{u}})$ of
system (22) must also be positive but unstable. Since $-f_{0}$ and $f$ are
strictly convex, this dynamical system has the unique stationary point
$(x_{0},f(x_{0}))$, where $x_{0}>x_{**}$ is the unique root of $\frac{\partial
f}{\partial x}(x_{0})=\varrho$. Further, this point is a saddle point because
of $\frac{\partial^{2}f}{\partial x^{2}}(x_{0})\neq 0$. Therefore, the graph
of $({\hat{y}},{\hat{u}})$ is contained in the stable manifold [46] of the
dynamical system (22). Hence $({\hat{y}},{\hat{u}})$ has to become either this
saddle point $(x_{0},f(x_{0}))$, or one of two stable paths of system (22)
converging to $(x_{0},f(x_{0}))$. In particular, $f(x_{0})$ must be
nonnegative and $x_{0}\geq x_{**}$ holds. Thus, in this example, whatever
discount rate $\varrho$ and initial state $x_{*}$, the Pontryagin Maximum
Principle with the transversality condition (7) elicits at most one motion.

The remaining examples illustrate the borderline of consistent with (1c)
conditions on co-state arcs in the case of the lack of asymptotic constraints.
Before we proceed further, we investigate system (1a)–(1c), corresponding to
both examples.

Fix a finite-dimensional real vector space $\mathbb{X}$ and a $C^{2}$-smooth
function $S:\mathbb{R}\times\mathbb{X}\to\mathbb{R}$. For each $\theta>0$,
$x_{*},\ x_{\theta}\in\mathbb{X}$, consider the following auxiliary problem:

minimize
$\displaystyle\int_{0}^{\theta}\Big{[}\frac{1}{4}\|u(\tau)\|^{2}+\frac{\partial
S}{\partial x}\big{(}{\tau},y(\tau)\big{)}e^{-\tau}u(\tau)+\frac{\partial
S}{\partial t}\big{(}{\tau},y(\tau)\big{)}\Big{]}\,d\tau$ subject to
$\displaystyle\frac{d{y}(t)}{dt}=e^{-t}u(t),\ t\in[0;\theta]\ \textrm{a.e.},$
$\displaystyle y(0)=x_{*},\quad y(\theta)=x_{\theta},\quad
y(t)\in\mathbb{X},\quad u(t)\in
U\stackrel{{\scriptstyle\triangle}}{{=}}\mathbb{X}.$

Consider a solution $({\psi},{\lambda})$ (on $[0;\theta]$) of relations
(1b)–(1c) corresponding to a given admissible process $({y},{u})$ of this
auxiliary problem. It follows from (1b) that
$\frac{d\psi(t)}{dt}={\lambda}\frac{\partial^{2}S}{\partial
x^{2}}\big{(}{t},{y}(t)\big{)}e^{-t}u(t)+{\lambda}\frac{\partial S}{\partial
t\partial
x}\big{(}{t},{y}(t)\big{)}={\lambda}\frac{d}{dt}\Big{(}\frac{\partial
S}{\partial x}\big{(}{t},{y}(t)\big{)}\Big{)}$ for almost all
$t\in[0;\theta]$. Then, one can find a $C\in\mathbb{X}$ satisfying
${\psi}(t)=\lambda\frac{\partial S}{\partial x}(t,{y}(t))+{C}$ for all
$t\in[0;\theta]$. Because $u(t)$ minimizes
$\frac{\lambda}{4}\|\upsilon\|^{2}+e^{-t}\big{(}\lambda\frac{\partial
S}{\partial
x}({t},y(t))-\psi(t)\big{)}\upsilon=\frac{\lambda}{4}\|\upsilon\|^{2}-e^{-t}C\upsilon$
over all $\upsilon\in\mathbb{X}$, we have $\lambda>0$. Then, putting
$\lambda=1$, one has $\displaystyle{\psi}(t)=\frac{\partial S}{\partial
x}(t,{y}(t))+{C},\quad{u}(t)=2e^{-t}{C},\quad{y}(t)=x_{*}+(1-e^{-2t}){C},$
(23a) $\displaystyle
J(x_{*},0,{u};t)-S(t,y(t))+S(0,x_{*})=\frac{1-e^{-2t}}{2}||{C}||^{2}=\frac{||{y}(t)-x_{*}||^{2}}{2(1-e^{-2t})}$
(23b) for all $t\in[0;\theta]$. In particular, since
$J(x,0,{u};t)-S(t,\mathrm{y}(x,0,{u},t))+S(0,x)$ is independent of
$x\in\mathbb{X}$, we also have $\frac{\partial J}{\partial
x}(y(0),0,{u};t)=\frac{\partial S}{\partial x}(t,y(t))-\frac{\partial
S}{\partial x}(0,x_{*})=\psi(t)-\psi(0)\qquad\forall t\in[0,\theta].$ (23c)
Note also that no co-state arc satisfies relations (1b)–(1c) with
${\lambda}=0$.

Having obtained all necessary formulas, let us finish considering this
auxiliary problem and return to examples of infinite-horizon control problems.

The consequent example will show that condition (16a) might fail if the
gradient of the limit of ${\hat{J}}$ does not coincide with the limit of
gradients of ${\hat{J}}$ at $x_{*}$ (cf. (17)).

######  Example 5.2.

Assume that the function $\mathbb{X}\ni
x\mapsto\limsup_{\theta\uparrow\infty}S(\theta,x)$ attains its minimum at the
point $x_{*}=0$.

Then, the control process $({\hat{y}},{\hat{u}})=(0,0)$ is weakly overtaking
optimal in the problem

minimize
$\displaystyle\int_{0}^{\infty}\Big{[}\frac{1}{4}\|u(\tau)\|^{2}+\frac{\partial
S}{\partial x}\big{(}{\tau},y(\tau)\big{)}e^{-\tau}u(\tau)+\frac{\partial
S}{\partial t}\big{(}{\tau},y(\tau)\big{)}\Big{]}\,d\tau$ subject to
$\displaystyle\frac{d{y}(t)}{dt}=e^{-t}u(t)\ \textrm{a.e.},$ $\displaystyle
y(0)=0,\ y(t)\in\mathbb{X},\ u(t)\in U=\mathbb{X}.$

Hence there exists a unique co-state arc ${\hat{\psi}}\in
C(\mathbb{R}_{+};\mathbb{X}^{*})$ satisfying the corresponding to
$({\hat{y}},{\hat{u}})$ relations (1b)-(1c) with ${\hat{\lambda}}=1$;
moreover, (23a) holds for some ${\hat{C}}$. It follows from ${\hat{u}}\equiv
0$ that ${\hat{C}}=0$ and ${\hat{\psi}}(t)=\frac{\partial S}{\partial x}(t,0)$
for all positive $t\geq 0$.

Consider the transversality condition (16a). By
$\frac{\partial{\hat{J}}}{\partial x}(0;\theta)=\psi(\theta)-\psi(0)$, the co-
state arc ${\hat{\psi}}$ satisfies this condition iff the co-state arc
${\hat{\psi}}(\theta)=\frac{\partial S}{\partial x}(\theta,0)$ converges to
zero as $\theta\uparrow\infty$. Note that, in this example, condition (16a) is
equivalent to Shell’s condition [8],[5]: $\psi(\theta)\to 0$; by comparison,
$\frac{\partial S}{\partial x}(\theta,0)=0$ holds for all $\theta$ large
enough iff the Arrow-like condition [9],[5] holds:
$\displaystyle\limsup\nolimits_{\theta\uparrow\infty}{\hat{\psi}}(\theta)({\hat{y}}(\theta)-{y}(\theta))\geq
0$ for all admissible processes $(y,u)$.

So, in Example 5.2, the corresponding to a weakly overtaking optimal process
$({\hat{y}},{\hat{u}})=(0,0)$ relations (1b)–(1c) have a solution satisfying
the transversality condition (16a) iff the function $S$ satisfies the
following additional asymptotic assumption: the gradients $\frac{\partial
S}{\partial x}(\theta,0)$ converge to zero as $\theta\uparrow\infty.$ For
instance, this is false if we take

$\mathbb{X}\stackrel{{\scriptstyle\triangle}}{{=}}\mathbb{R},\qquad\mathbb{R}_{+}\times\mathbb{R}\ni(t,x)\mapsto
S(t,x)\stackrel{{\scriptstyle\triangle}}{{=}}e^{-t}\sin(e^{t}x)-e^{-x^{2}}.$

In this case the cost functional $J$ and all its derivatives are bounded for
all control processes. In particular, there could not be $\lambda=0$, i.e.,
all solutions of the Pontryagin Maximum Principle are not abnormal whenever
$x_{*}$. Furthermore, the corresponding infinite-horizon problem has the
unique overtaking optimal process (moreover, a unique strongly optimal [23], a
unique classical optimal [35], and a unique (O)-optimal [47] process), this
process has a unique (up to a positive factor) solution of (1a)–(1c), and the
corresponding transversality condition (16a) is well defined; after all, this
condition does not be consistent with (1c). Thus, the necessity of (16a)
depends primarily on the asymptotics of $\frac{\partial{\hat{J}}}{\partial x}$
rather than the qualitative properties of system (1a)–(1c).

The dynamics and the integrand in the next example are linear in $x$, whilst
the functions $S$ and $\frac{\partial S}{\partial x}$ are periodic. It was
enough that for a given $x_{*}$ in the corresponding infinite-horizon control
problem the dimension of the set of optimal processes coincides with
$\dim\mathbb{X}$. By this reason, their co-state arcs at $t=0$ also form a
ball in $\mathbb{X}^{*}$. Inclusion (7) as a unified necessary transversality
condition has to take this into account.

######  Example 5.3.

Put $\mathbb{X}\stackrel{{\scriptstyle\triangle}}{{=}}\mathbb{R}^{2}$. Define
map $S$ as follows:

$\mathbb{R}_{+}\times\mathbb{R}^{2}\ni(t,x_{1},x_{2})\mapsto
S(t,x_{1},x_{2})\stackrel{{\scriptstyle\triangle}}{{=}}{x}_{1}\sin(t)-{x}_{2}\cos(t).$

Then, the corresponding infinite-horizon control problem is

minimize
$\displaystyle\int_{0}^{\infty}\Big{[}\frac{\|u(\tau)\|^{2}}{4}+(e^{-\tau}{u}_{1}(\tau)+{y}_{2}(\tau))\sin(\tau)+({y}_{1}(\tau)-e^{-\tau}{u}_{2}(\tau))\cos(\tau)\Big{]}\,d\tau$
subject to
$\displaystyle\frac{d{y_{1}}(t)}{dt}=e^{-t}u_{1}(t),\frac{d{y_{2}}(t)}{dt}=e^{-t}u_{2}(t)\
\textrm{a.e.\ }t\geq 0,$ $\displaystyle(y_{1},y_{2})(0)=x_{*},\
(y_{1},y_{2})(t)\in\mathbb{R}^{2},\ (u_{1},u_{2})(t)\in U=\mathbb{R}^{2}.$

Consider a weakly overtaking optimal process $({\hat{y}};{\hat{u}})$ in this
problem. By Theorem 4.1, there exists a nonzero solution
$({\hat{\psi}},{\hat{\lambda}})$ of system (1b)–(1c) and (7), corresponding to
this process. Note that for each positive $\theta$ this pair is also the
solution of this system for the auxiliary control problem with
$x_{\theta}={\hat{y}}(\theta).$ Then, first, ${\hat{\lambda}}$ is positive,
and one can take ${\hat{\lambda}}=1$; second, by (23a), for some
${\hat{C}}\in\mathbb{X}^{*}$, we obtain ${\hat{\psi}}(t)=\frac{\partial
S}{\partial x}(t,{\hat{y}}(t))+{\hat{C}}$ for all nonnegative $t$. Hence, by
(23c), condition (7) is equivalent to

$-\frac{\partial S}{\partial
x}(0,x_{*})-{\hat{C}}=-{\hat{\psi}}(0)\in\operatorname*{co}\big{\\{}(\sin\theta,-\cos\theta)\,\big{|}\,\theta\in[0;2\pi]\big{\\}}-\frac{\partial
S}{\partial x}(0,x_{*}),$

i.e., $||{\hat{C}}||\leq 1$. Thus, according to (7), every weakly overtaking
optimal process satisfies (23a) for some ${\hat{C}}$, $||{\hat{C}}||\leq 1$.

We claim that each process with this property is a weakly overtaking optimal.
Fix an admissible process $(\bar{y},\bar{u})$ satisfying (23a) for some
$\bar{C}$, $||\bar{C}||\leq 1$. Consider an admissible process $(y,u)$. By
(23b), the function
$J(x_{*},0,\bar{u};\theta)-S(\theta,\bar{y}(\theta))+S(0,x_{*})$ converges to
$||\bar{C}||^{2}/2$ as $\theta\uparrow\infty$. Then, it is necessary to find
some $\phi\in[0;2\pi]$ satisfying

$\limsup_{n\to\infty}\big{[}J(x_{*},0,u;\phi+2\pi
n)-S(\phi,x_{*}+\bar{C})+S(0,x_{*})-{||\bar{C}||^{2}}/{2}\big{]}\geq 0.$

Note that $J(x_{*},0,u;\theta)$ tends to infinity as $\theta\uparrow\infty$ if
the total variation of $y$ is unbounded. Therefore, one can assume that the
motion $y$ converges to some $x_{*}+C_{\infty}\in\mathbb{X}$ at infinity. For
every positive $\theta$, there exists an admissible process
$(y_{\theta},u_{\theta})$ optimal in the auxiliary problem with
$x_{\theta}=y_{\theta}(\theta)$; in particular, $J(x_{*},0,u;\theta)\geq
J(x_{*},0,u_{\theta};\theta)$. Hence, by (23b), it follows from
$y_{\theta}(\theta)\to C_{\infty}$ that one has

$\limsup_{n\to\infty}J(x_{*},0,u;\phi+2\pi n)\geq
S(\phi,x_{*}+C_{\infty})-S(0,x_{*})+{||C_{\infty}||^{2}}/{2}.$

Then, we must find a $\phi\in[0;2\pi]$ such that the number

$S(\phi,x_{*}+C_{\infty})-S(\phi,x_{*}+\bar{C})+\frac{||C_{\infty}||^{2}-||\bar{C}||^{2}}{2}=S(\phi,C_{\infty}-\bar{C})+\frac{||C_{\infty}||^{2}-||\bar{C}||^{2}}{2}$

has to be nonnegative. Note that for each vector $z\in\mathbb{X}$ one can find
a $\phi\in[0;2\pi]$ such that $S(\phi,z)=||z||$. Therefore, one should prove
that one has
$2||x-\bar{C}||\geq||\bar{C}||^{2}-||x||^{2}=\big{(}\|\bar{C}\|-\|x\|\big{)}\big{(}\|\bar{C}\|+\|x\|\big{)}$
for all vectors $x=C_{\infty}\in\mathbb{R}^{2}$. The latter is evident at all
$x$ where $||x||+||\bar{C}||\leq 2$, while at vectors $x$ where
$||x||>2-||\bar{C}||$ it follows from $||\bar{C}||^{2}\leq
1\leq(2-||\bar{C}||)^{2}<||x||^{2}$ because of $||\bar{C}||\leq 1$.

Thus, it is shown that every process satisfying (23a) for some $\bar{C}$,
$||\bar{C}||\leq 1$, is weakly overtaking optimal. Then, each co-state arc
satisfying (7) generates the unique weakly overtaking optimal process and each
weakly overtaking optimal process possesses its own co-state arc satisfying
(7).

In summing up, for every weakly overtaking optimal process in this example,
there exists a unique co-state arc satisfying (1c). Therefore, each necessary
condition on co-state arc must either be satisfied each of them or distinguish
them. Due to the linearity of the dynamics and integrand in $x$, any necessary
condition depending only on a co-state arc is satisfied for all such co-state
arc. In this example, all such co-state arcs $\psi$ satisfy
$||\psi(0)+(0,1)||\leq 1$. Since inclusion (7) also points to the same set,
this inclusion becomes the tightest of all conditions on the co-state arcs
consistent with (1c).

##  6 Subdiffentials of lower and upper limits of Lipschitz continuous
functions

Throughout this section, let ${{\mathcal{X}}}$ be a nonempty subset of a
finite-dimensional space $\mathbb{X}$.

Let a family of maps ${W}_{\theta}:\mathbb{X}\to\mathbb{R}\cup\\{+\infty\\}$,
$\theta\geq 0$, be given. Define the maps $W_{\inf}$ and $W_{\sup}$ from
$\mathbb{X}$ to $\mathbb{R}\cup\\{-\infty,+\infty\\}$ by the rules

$W_{\inf}(x)\stackrel{{\scriptstyle\triangle}}{{=}}\liminf_{\theta\uparrow\infty}{W}_{\theta}(x),\qquad
W_{\sup}(x)\stackrel{{\scriptstyle\triangle}}{{=}}\limsup_{\theta\uparrow\infty}{W}_{\theta}(x)\qquad\forall
x\in\mathbb{X}.$ (24)

Also, define the maps $\operatorname*{lsc}_{{\mathcal{X}}}W_{\inf}$ and
$\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup}$ from $\mathbb{X}$ to
$\mathbb{R}\cup\\{-\infty,+\infty\\}$ as follows:

$\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\inf}(x)\stackrel{{\scriptstyle\triangle}}{{=}}\liminf_{\theta_{n}\uparrow\infty,\
{{\mathcal{X}}}\ni x_{n}\to
x}{W}_{\theta_{n}}(x_{n}),\quad\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\sup}(x)\stackrel{{\scriptstyle\triangle}}{{=}}\limsup_{\theta_{n}\uparrow\infty,\
{{\mathcal{X}}}\ni x_{n}\to x}{W}_{\theta_{n}}(x_{n})$ (25)

if $x\in\operatorname*{cl}{{\mathcal{X}}}$, but $+\infty$ otherwise. Note that
both maps are lower semicontinuous.

First, consider subdifferentials of the lower epigraphical limit of
$W_{\theta}$. The first lemma is the simple consequence of the corresponding
results of [15].

######  Lemma 6.1.

Let $\mathbb{T}$ be an unbounded set of positive numbers. Let maps
${W}_{t}:\mathbb{X}\to\mathbb{R}$ be lower semicontinuous on some open set
${G}\subset\mathbb{X}$ for all $t\in\mathbb{T}$. Assume that

$\displaystyle\operatorname*{e-liminf}_{\theta\in\mathbb{T},\
\theta\uparrow\infty}W_{\theta}(z)=W_{\inf}(z)\qquad\forall z\in{G}.$ (26)

Then, for all $\check{z}\in{G}$ satisfying $|W_{\inf}(\check{z})|<+\infty$ one
has

${\hat{\partial}}W_{\inf}(\check{z})\subset\bigcap_{\begin{subarray}{c}(\theta_{n})_{n\in\mathbb{N}}\in\mathbb{R}_{+}^{\mathbb{N}},\
\theta_{n}\uparrow\infty,\\\ W_{\theta_{n}}(\check{z})\to
W_{\inf}(\check{z})\end{subarray}}\operatorname*{Limsup}_{\begin{subarray}{c}z_{n}\to\check{z},\\\
|W_{\theta_{n}}(z_{n})-W_{\theta_{n}}(\check{z})|\to
0\end{subarray}}{\hat{\partial}}W_{\theta_{n}}(z_{n}),$ (27)

i.e., for every subgradient $\xi\in{\hat{\partial}}W_{\inf}(\check{z})$ and
for every unbounded increasing sequence of positive numbers
$\theta_{n}\in\mathbb{T}$ satisfying $W_{\theta_{n}}(\check{z})\to
W_{\inf}(\check{z})$, there exist converging to $\check{z}$ and $\xi$,
respectively, sequences of points $z_{n}\in\mathbb{X}$ and gradients
$\zeta_{n}\in{\hat{\partial}}W_{\theta_{n}}(z_{n})$ satisfying
$|W_{\theta_{n}}(z_{n})-W_{\theta_{n}}(\check{z})|\to 0$ as $n\uparrow\infty$.

###### Proof.

Consider a point $\check{z}\in{G}$ and a subgradient
$\xi\in{\hat{\partial}}W_{\inf}(\check{z})$. By [31, Theorem 1.27], since
$\mathbb{X}$ is finite-dimensional, there exist a $C^{1}$-smooth function
$g:\mathbb{X}\to\mathbb{R}$ and a neighbourhood $G^{\prime}\subset G$ of the
point $\check{z}$ such that one has $\xi=\frac{\partial g}{\partial
z}(\check{z})$, ${W}_{\inf}(\check{z})=g(\check{z})$, and
$\operatorname*{lsc}{W}_{\inf}(z)-\operatorname*{lsc}{W}_{\inf}(\check{z})\geq
g(z)-g(\check{z})$ for all $z\in G^{\prime}$.

Let $\mathcal{T}$ be the set of all unbounded increasing sequences of positive
$\theta_{n}\in\mathbb{T}$ such that numbers $W_{\theta_{n}}(\check{z})$
converge to $W_{\inf}(\check{z})$ and the sequence of maps

$\mathbb{X}\ni z\mapsto\ W_{\theta_{n}}(z)$ (28)

epi-converge. Fix such a sequence of $\theta_{n}$, consider the corresponding
epi-limit

$\mathbb{X}\ni
z\mapsto\widetilde{W}_{\\{\theta_{n}\\}}(z)\stackrel{{\scriptstyle\triangle}}{{=}}\operatorname*{e-lim}_{n\uparrow\infty}W_{\theta_{n}}(z);$

this function is lower semicontinuous by [29, Proposition 7.4(a)]. Now, on the
one hand, (26) entails $\widetilde{W}_{\\{\theta_{n}\\}}({z})\geq
W_{\inf}({z})$ whenever $z$; on the other hand, $W_{\theta_{n}}(\check{z})\to
W_{\inf}(\check{z})$ leads to $\widetilde{W}_{\\{\theta_{n}\\}}(\check{z})\leq
W_{\inf}(\check{z})$. Then one has
${W}_{\\{\theta_{n}\\}}(\check{z})=W_{\inf}(\check{z})=g(\check{z})$ and
$\widetilde{W}_{\\{\theta_{n}\\}}(z)-{W}_{\\{\theta_{n}\\}}(\check{z})\geq
g(z)-g(\check{z})$ for all $z\in G^{\prime}$. By [31, Theorem 1.27], $\xi$
lies in ${\hat{\partial}}\widetilde{W}_{\\{\theta_{n}\\}}(\check{z})$.

Recall that the sequence of functions $W_{\theta_{n}}$ is called
asymptotically locally equicoercive [48, Definition 4.4] if for every bounded
sequence of $x_{k}\in\mathbb{X}$, from
$\sup_{k\in\mathbb{N}}W_{\theta_{n(k)}}(x_{k})<+\infty$ for some unbounded
sequence of $n(k)$, it follows that there exists a converging subsequence of
the sequence of $x_{k}.$ Note that $\mathbb{X}$ is finite-dimensional;
therefore, the sequence of $W_{\theta_{n}}$ has this property. Thanks to [15,
Proposition 2.2(v) and Theorem 5.3(ii)], it follows that

${\hat{\partial}}\widetilde{W}_{\\{\theta_{n}\\}}(\bar{z})\subset\operatorname*{Limsup}_{{z\to\bar{z},\
n\uparrow\infty,\
W_{\theta_{n}}(z)\to\widetilde{W}_{\\{\theta_{n}\\}}(\check{z})}}{\hat{\partial}}{W}_{\theta_{n}}(z)\qquad\forall\bar{z}\in{G}$

holds because of the epigraphical convergence of (28) to
$\widetilde{W}_{\\{\theta_{n}\\}}$. Hence, for all sequences
$(\theta_{n})_{n\in\mathbb{N}}\in\mathcal{T}$, combining
$W_{\theta_{n}}(\check{z})\to\widetilde{W}_{\\{\theta_{n}\\}}(\check{z})={W}_{\inf}(\check{z})$
with $\xi\in{\hat{\partial}}\widetilde{W}_{\\{\theta_{n}\\}}(\check{z})$ gives

$\xi\in\operatorname*{Limsup}_{{z\to\check{z},\ n\uparrow\infty,\
|{W}_{\theta_{n}}(z)-{W}_{\inf}(\check{z})|\to
0}}{\hat{\partial}}{W}_{\theta_{n}}(z).$ (29)

To prove the desired inclusion, suppose to the contrary that one can find a
positive $\varepsilon$ and an unbounded increasing sequence of positive
numbers $\theta_{n}\subset\mathbb{T}$ satisfying $W_{\theta_{n}}(\check{z})\to
W_{\inf}(\check{z})$ such that one has
$|W_{\theta_{n}}({z}_{n})-W_{\theta_{n}}(\check{z})|+||\zeta_{n}-\xi||+||z_{n}-\check{z}||>\varepsilon$
for all sequences of points $z_{n}\in{G}$ and gradients
$\zeta_{n}\in{\hat{\partial}}{W}_{\theta_{n}}(z_{n})$. Then, the sequence of
$\theta_{n}$ (and every its subsequence) would not satisfies (29). Hence the
sequence of maps (28) would not have any epi-converging subsequence. However,
since $W_{\inf}(\check{z})$ is finite, the sequence of ${W}_{\theta_{n}}$ does
not escape epigraphically to the horizon, and, by [29, Theorem 7.6], possesses
an epi-converging subsequence. We get a contradiction.

The lemma has been proved. ∎

For Theorem 4.6 we need to prove the similar result for
$\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\inf}$.

######  Lemma 6.2.

Let $\mathbb{T}$ be an unbounded set of positive numbers. Let maps
${W}_{t}:\mathbb{X}\to\mathbb{R}$ be locally Lipschitz continuous on a
neighbourhood of ${{\mathcal{X}}}$ for all positive $t$, and let the set
${{\mathcal{X}}}$ be closed. Assume that there exists an open set
${G}\subset\mathbb{X}$ such that one has

$\operatorname*{e-liminf}_{\theta\in\mathbb{T},\
\theta\uparrow\infty}(W_{\theta}+\delta_{{{\mathcal{X}}}})(z)=W_{\inf}(z)\qquad\forall
z\in{G}\cap{{\mathcal{X}}}.$

Then, for all $\check{z}\in{G}\cap{{\mathcal{X}}}$ satisfying
$|W_{\inf}(\check{z})|<+\infty$, one has

$\displaystyle
N(\check{z},\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\inf}(\check{z});\operatorname*{epi}\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\inf})\subset
N(\check{z};{{\mathcal{X}}})\times\\{0\\}+$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\quad\
\bigcap_{\begin{subarray}{c}(\theta_{n})_{n\in\mathbb{N}}\in\mathbb{R}_{+}^{\mathbb{N}},\
\theta_{n}\uparrow\infty,\\\ W_{\theta_{n}}(\check{z})\to
W_{\inf}(\check{z})\end{subarray}}\operatorname*{Limsup}_{\begin{subarray}{c}{{\mathcal{X}}}\ni
z_{n}\to\check{z},\ 0<\lambda_{n}\to{\hat{\lambda}},\\\
|W_{\theta_{n}}(z_{n})-W_{\theta_{n}}(\check{z})|\to
0\end{subarray}}\lambda_{n}\big{(}{\hat{\partial}}W_{\theta_{n}}(z_{n})\times\\{-1\\}\big{)}.$

###### Proof.

On the one hand, the map $\delta_{{{\mathcal{X}}}}$ is lower semicontinuous;
hence, since all the maps $W_{\theta}$ are locally Lipschitz continuous, the
sum rule [30, Theorem 3.36] leads to

${\partial}(W_{\theta}+\delta_{{{\mathcal{X}}}})({z})\subset
N({z};{{\mathcal{X}}})+{\partial}W_{\theta}({z})\qquad\forall{z}\in{{\mathcal{X}}}.$

On the other hand, since the map
$W_{\inf}|_{{\mathcal{X}}}=(W_{\inf}+\delta_{{{\mathcal{X}}}})|_{{\mathcal{X}}}$
as a lower epigraphical limit of $W_{\theta}+\delta_{{{\mathcal{X}}}}$ is
lower semicontinuous, this map coincides with
$\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\inf}$ on ${{\mathcal{X}}}$.
Then, applying (27) to the lower semicontinuous maps
$W_{\theta}+\delta_{{{\mathcal{X}}}}$ instead of $W_{\theta}$, we obtain

${\hat{\partial}}\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\inf}(\check{z})\subset
N(\check{z};{{\mathcal{X}}})+\bigcap_{\begin{subarray}{c}(\theta_{n})_{n\in\mathbb{N}}\in\mathbb{R}_{+}^{\mathbb{N}},\
\theta_{n}\uparrow\infty,\\\ W_{\theta_{n}}(\check{z})\to
W_{\inf}(\check{z})\end{subarray}}\operatorname*{Limsup}_{\begin{subarray}{c}{{\mathcal{X}}}\ni
z_{n}\to\check{z},\\\ |W_{\theta_{n}}(z_{n})-W_{\theta_{n}}(\check{z})|\to
0\end{subarray}}{\hat{\partial}}W_{\theta_{n}}(z_{n}).$

It follows from this inclusion for Fréchet subdifferentials that the similar
inclusions hold for limiting and singular limiting subdifferentials. Now, the
desired inclusion is justified through the fact that
$N(x,g(x);\operatorname*{epi}g)=\cup_{r>0}r\big{(}\partial
g(x)\times\\{-1\\}\big{)}\cup\partial^{\infty}g(x)\times\\{0\\}$ by (5a) and
(5b). ∎

Now we will consider subdifferentials of the upper epigraphical limit of
$W_{\theta}$.

In [18, Theorem 6.1(i)] it was shown the following upper-estimate of Fréchet
subdifferentials of the upper epigraphical limit of lower semicontinuous
functions $W_{\theta}$:

$\displaystyle{\hat{\partial}}\Big{(}\operatorname*{e-limsup}_{\theta\uparrow\infty}W_{\theta}\Big{)}(\check{z})\subset\bigcap_{\varkappa>0}\operatorname*{co}\operatorname*{cl}\bigcup_{\theta>1/\varkappa,\
{x\in\mathbb{X},\
||x-\check{z}||<\varkappa}}{\hat{\partial}}{W}_{\theta}(x)\qquad\forall\check{z}\in\mathbb{X}.$
(30)

Repeating the proof of Lemma 6.2 word-for-word and using (30) instead to (27),
we obtain the following, needed for the proof of Theorem 4.5, result.

######  Lemma 6.3.

Let the set ${{\mathcal{X}}}$ be closed, maps
${W}_{t}:\mathbb{X}\to\mathbb{R}$ be locally Lipschitz continuous on a
neighbourhood of ${{\mathcal{X}}}$ for all positive $t$. Assume that there
exists an open set ${G}\subset\mathbb{X}$ such that one has

$\displaystyle\operatorname*{e-limsup}_{\theta\uparrow\infty}\big{[}W_{\theta}(z)+\delta_{{{\mathcal{X}}}}(z)\big{]}=\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\sup}(z)\qquad\forall
z\in{G}\cap{{\mathcal{X}}}.$

Then, for all $\check{z}\in{G}\cap{{\mathcal{X}}}$ satisfying
$|\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\sup}(\check{z})|<+\infty$,
one has

$\displaystyle
N(\check{z},\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\sup}(\check{z});\operatorname*{epi}\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\inf})\subset
N(\check{z};{{\mathcal{X}}})\times\\{0\\}+\qquad\qquad\qquad\qquad\qquad\qquad$
$\displaystyle\bigcap_{\varkappa>0}\operatorname*{co}\operatorname*{cl}\bigcup_{\begin{subarray}{c}\theta>1/\varkappa,\
x\in{{\mathcal{X}}},\ \lambda>0,\\\ {||x-\check{z}||<\varkappa,\
|\lambda-{\hat{\lambda}}|<\varkappa}\end{subarray}}\lambda_{n}\big{(}{\hat{\partial}}W_{\theta_{n}}(z_{n})\times\\{-1\\}\big{)}.$

Now our purpose is to prove Lemma 6.7: an analogue of Lemmas 6.2 and 6.3 for
pointwise upper limit $\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\sup}$.
For proving the corresponding upper-estimate, we will consider some auxiliary
assertions.

First, we write the following multidimensional mean value inequalities [18,
Theorems 2.1 and 2.2]:

######  Proposition 6.4.

Let a scalar function $W$ be continuous on the open convex set
${{\mathcal{X}}}$.

Then, for arbitrary points $\check{x}_{-},\check{x}_{+}\in{{\mathcal{X}}}$,
numbers $\check{s}<W(\check{x}_{+})-W(\check{x}_{-})$, and
$\check{\varkappa}>0$, there exist
$z^{\prime}_{-},z^{\prime}_{+}\in[\check{x}_{-};\check{x}_{+}]$,
$z_{-},z_{+}\in{{\mathcal{X}}}$, $\zeta_{-}\in{\hat{\partial}}W(z_{-})$, and
$\zeta_{+}\in{\hat{\partial}}W(z_{+})$ such that

$\displaystyle\check{s}<\zeta_{-}(\check{x}_{+}-\check{x}_{-}),\quad||z_{-}-z^{\prime}_{-}||<\check{\varkappa},\quad
W(z_{-})<W(\check{x}_{-})+\max(0,\check{s})+\check{\varkappa},$
$\displaystyle\check{s}<\zeta_{+}(\check{x}_{+}-\check{x}_{-}),\quad||z_{+}-z^{\prime}_{+}||<\check{\varkappa},\quad
W(z_{+})>W(\check{x}_{+})-\max(0,\check{s})-\check{\varkappa}.\,$

We will also need the following variant of mean value inequalities:

######  Lemma 6.5.

Let a scalar function $W$ be defined on $\mathbb{X}$ and continuous on the
convex set ${{\mathcal{X}}}$. Assume that some different points
$x_{-},x_{+}\in{{\mathcal{X}}}$ satisfy
$(x_{-};x_{+})\subset\operatorname*{int}{{\mathcal{X}}}$.

Then, for every real number $s<W(x_{+})-W(x_{-})$, for every positive
$\varepsilon$, there exist some point
${\hat{z}}\in\operatorname*{int}{{\mathcal{X}}}$, number $r^{\prime}\in[0;1]$,
and subgradient ${\hat{\zeta}}\in{\hat{\partial}}W({\hat{z}})$ such that one
has

$s<{\hat{\zeta}}(x_{+}-x_{-}),\
||x_{-}+r^{\prime}(x_{+}-x_{-})-{\hat{z}}||<\varepsilon,\
|W(x_{-})-W({\hat{z}})|<|s|+\varepsilon.$

###### Proof.

First, we may assume that $x_{+},x_{-}\in\operatorname*{int}{{\mathcal{X}}}$.
Indeed, for otherwise we could choose a positive $\varepsilon^{\prime}$ small
enough that
$x^{\prime}_{-}\stackrel{{\scriptstyle\triangle}}{{=}}x_{-}+\varepsilon^{\prime}(x_{+}-x_{-})$
and
$x^{\prime}_{+}\stackrel{{\scriptstyle\triangle}}{{=}}x_{+}-\varepsilon^{\prime}(x_{+}-x_{-})$
satisfy $s<W(x^{\prime}_{+})-W(x^{\prime}_{-})$. It can be done since $W$ is
continuous on $[x_{-};x_{+}]$.

Second, we may as well take $x_{-}=0$, $W(x_{-})=0$. So, we have $s<W(x_{+}).$

Case $s<0$. Let $s$ be negative. We claim that for all sufficient small
positive $\varepsilon$ there exist some ${\hat{z}}^{\prime}\in[0;x_{+}]$,
${\hat{z}}\in\operatorname*{int}{{\mathcal{X}}}$, and
${\hat{\zeta}}\in{\hat{\partial}}W({\hat{z}})$ such that one has

$\displaystyle-|s|<{\hat{\zeta}}x_{+},\quad||{\hat{z}}-{\hat{z}}^{\prime}||<\varepsilon,\quad|W({\hat{z}}^{\prime})-W({\hat{z}})|<\varepsilon/3,\quad|W({\hat{z}})|<|s|+2\varepsilon/3.$
(31)

Since $W$ is continuous on $\mathcal{X}$, by $W(x_{+})>s$, one can find a
positive ${\hat{t}}<1$ such that

$\displaystyle|s|>W({\hat{t}}x_{+})\geq-{\hat{t}}|s|>-|s|=s.$ (32)

In addition, one can find a positive $\varkappa<\varepsilon/3$ such that one
has

$\displaystyle x+z^{\prime}\in\operatorname*{int}\mathcal{X},\
|W(x+z^{\prime})-W(x)|<\varepsilon/3\qquad\forall
x\in[0;x_{+}],z^{\prime}\in{{\mathbb{X}}},||z^{\prime}||<{\varkappa}.$ (33)

We will find ${\hat{z}}^{\prime}\in[0;{\hat{t}}x_{+}]$,
${\hat{z}}\in\operatorname*{int}\mathcal{X}$, and
${\hat{\zeta}}\in{\hat{\partial}}W({\hat{z}})$ satisfying
$|W({\hat{z}})|<|s|+2\varepsilon/3$ and

$-|s|<{\hat{\zeta}}x_{+},\ ||{\hat{z}}-{\hat{z}}^{\prime}||<\varkappa,\
|W({\hat{z}})-W({\hat{z}}^{\prime})|<\varepsilon/3.$ (34)

Consider the continuous map

$[0;{\hat{t}}\;\\!]\ni\tau\mapsto h(\tau)=W(\tau
x_{+})-{\tau}W({\hat{t}}x_{+})/{{\hat{t}}}.$

Since $h({\hat{t}})=h(0)=0$ holds, there exists a positive
${\hat{\tau}}\leq{\hat{t}}$ satisfying the equality $h({\hat{\tau}})=0$ and at
least one of the following conditions:

$\textrm{(i)}\quad{\hat{\tau}}<\varkappa;\qquad\textrm{(ii)}\quad
h|_{(0,{\hat{\tau}}]}\textrm{\ is nonpositive};\qquad\textrm{(iii)}\quad
h|_{(0,{\hat{\tau}}]}\textrm{\ is nonnegative}.$

Now, the relations $0<{\hat{\tau}}\leq{\hat{t}}<1$, $h({\hat{\tau}})=0$, and
(32) yield the inequality

$\displaystyle-|s|\leq-|s|{\hat{\tau}}\stackrel{{\scriptstyle(\ref{RRR})}}{{\leq}}{{\hat{\tau}}}W({\hat{t}}x_{+})/{{\hat{t}}}=W({\hat{\tau}}x_{+}).$
(35)

Applying Proposition 6.4 with
$\check{x}_{-}\stackrel{{\scriptstyle\triangle}}{{=}}0$,
$\check{x}_{+}\stackrel{{\scriptstyle\triangle}}{{=}}{\hat{\tau}}x_{+}$,
$\check{s}\stackrel{{\scriptstyle\triangle}}{{=}}-|s|{\hat{\tau}}$, and
$\check{\varkappa}\stackrel{{\scriptstyle\triangle}}{{=}}{\varkappa}$, there
exist some $r_{-},r_{+}\in[0;{\hat{\tau}}],$
$z_{-},z_{+}\in\operatorname*{int}{{\mathcal{X}}}$,
$\zeta_{-}\in{\hat{\partial}}W(z_{-})$, and
$\zeta_{+}\in{\hat{\partial}}W(z_{+})$ satisfying

$\displaystyle-|s|{\hat{\tau}}<{\hat{\tau}}\zeta_{-}x_{+},\quad||r_{-}x_{+}-z_{-}||<\varkappa,\quad
W(z_{-})$ $\displaystyle-W(0)\;<\,\varkappa,\quad$
$\displaystyle-|s|{\hat{\tau}}<{\hat{\tau}}\zeta_{+}x_{+},\quad||r_{+}x_{+}-z_{+}||<\varkappa,\quad
W(z_{+})$ $\displaystyle-W({\hat{\tau}}x_{+})>-\varkappa.$

Next, taking into account the inequalities ${\hat{\tau}}>0$,
$\varkappa<\varepsilon/3$, ${\hat{t}}<1$, and (33), we have

$\displaystyle-|s|<{\hat{\zeta}}_{\pm}x_{+},\quad||z_{\pm}-r_{\pm}x_{+}||<\varkappa<\varepsilon/3,\quad|W({\hat{z}}_{\pm})-W(r_{\pm}x_{+})|\stackrel{{\scriptstyle(\ref{NR})}}{{\leq}}\varepsilon/3,$
$\displaystyle W(z_{-})<W(0)+\varkappa=\varkappa<\varepsilon/3,\
W(z_{+})>-\varkappa+W({\hat{\tau}}x_{+})\stackrel{{\scriptstyle(\ref{RR})}}{{\geq}}-\varkappa-|s|\geq-\varepsilon/3-|s|.$

Now, in the case of ${\hat{\tau}}<\varkappa$ (item (i)) and in the case of
nonpositive $h|_{[0;{\hat{\tau}}]}$ (item (ii)), let us set
${\hat{z}}\stackrel{{\scriptstyle\triangle}}{{=}}z_{+}$,
${\hat{z}}^{\prime}\stackrel{{\scriptstyle\triangle}}{{=}}r_{+}x_{+}$, and
${\hat{\zeta}}\stackrel{{\scriptstyle\triangle}}{{=}}{\zeta}_{+}$; and in the
case of nonnegative $h|_{[0;{\hat{\tau}}]}$ (item (iii)), set
${\hat{z}}\stackrel{{\scriptstyle\triangle}}{{=}}z_{-}$,
${\hat{z}}^{\prime}\stackrel{{\scriptstyle\triangle}}{{=}}r_{-}x_{+}$, and
${\hat{\zeta}}\stackrel{{\scriptstyle\triangle}}{{=}}{\zeta}_{-}$. Then, in
all these cases, we have proved (34). So, we need to check only
$|W({\hat{z}})|<|s|+2\varepsilon/3.$

Indeed, in the case of nonnegative $h|_{[0;{\hat{\tau}}]}$ (item (iii)), by
the choice of $r_{-}$ and $z_{-}$, we obtain $0\leq
h(r_{-})=W(r_{-}x_{+})-{r_{-}}W({\hat{t}}x_{+})/{{\hat{t}}}\leq
W(r_{-}x_{+})+|W({\hat{t}}x_{+})|$ and

$\varepsilon/3>W({\hat{z}})=W(z_{-})\stackrel{{\scriptstyle(\ref{NR})}}{{\geq}}W(r_{-}x_{+})-\varepsilon/3\geq-|W({\hat{t}}x_{+})|-\varepsilon/3\stackrel{{\scriptstyle(\ref{RRR})}}{{\geq}}-|s|-\varepsilon/3.$

In the case of $h|_{[0;{\hat{\tau}}]}\leq 0$ (item (ii)), one has $0\geq
h(r_{+})\geq W(r_{+}x_{+})-|W({\hat{t}}x_{+})|$ and

$-\varepsilon/3-|s|<W({\hat{z}})=W(z_{+})\stackrel{{\scriptstyle(\ref{NR})}}{{\leq}}W(r_{+}x_{+})+\varepsilon/3\leq|W({\hat{t}}x_{+})|+\varepsilon/3\stackrel{{\scriptstyle(\ref{RRR})}}{{\leq}}|s|+\varepsilon/3.$

Finally, in the case ${\hat{\tau}}<\varkappa$ (item (i)), (33) yields
$|W({z}_{-})|<|W(r_{-}x_{+})|+\varepsilon/3<2\varepsilon/3$.

The inequalities (31) have been proved.

Case $s\geq 0$. Assume that $s$ is nonnegative. Decreasing $\varepsilon$ if
necessary, one can suppose that $s+\varepsilon/5<W(x_{+})$. Put
$\bar{s}\stackrel{{\scriptstyle\triangle}}{{=}}s+\varepsilon/5$ and
$\bar{\varepsilon}\stackrel{{\scriptstyle\triangle}}{{=}}\min(\varepsilon,\varepsilon/\bar{s},1)/5$.
Now, we guarantee $0<\bar{s}<W(x_{+})$. Consider the map
$\bar{W}:\mathbb{X}\times\mathbb{R}\to\mathbb{R}$ defined by

$\bar{W}(x,r)\stackrel{{\scriptstyle\triangle}}{{=}}W(x)-(1+\bar{\varepsilon})r\bar{s}\qquad\forall
x\in\mathbb{X},r\in\mathbb{R}.$

Then, we have
${\hat{\partial}}\bar{W}(x,r)={\hat{\partial}}{W}(x)\times\\{-(1+\bar{\varepsilon})\bar{s}\\}$,
$\bar{W}(0,0)=0$, and

$\bar{W}(x_{+},1)=\bar{W}(x_{+},1)-\bar{W}(0,0)=W(x_{+})-(1+\bar{\varepsilon})\bar{s}>\bar{s}-(1+\bar{\varepsilon})\bar{s}=-\bar{\varepsilon}\bar{s}.$

Since $-\bar{\varepsilon}\bar{s}<0$, we may apply the first case of our lemma
to the inequality $-\bar{\varepsilon}\bar{s}<\bar{W}(x_{+},1)-\bar{W}(0,0)$.
Then, there exist some number $r\in[0;1]$, points
$\bar{z}^{\prime}=(rx_{+},{r})$,
$\bar{z}=({\hat{z}},{\hat{r}})\in\operatorname*{int}{{\mathcal{X}}}\times\mathbb{R}$,
and the subgradient
$\bar{\zeta}=({\hat{\zeta}},-(1+\bar{\varepsilon})\bar{s})\in{\hat{\partial}}\bar{W}({\hat{z}},{\hat{r}})$
satisfying (31), i.e.,

$-\bar{\varepsilon}|\bar{s}|<{\hat{\zeta}}x_{+}-(1+\bar{\varepsilon})\bar{s},\
\max\big{(}||{\hat{z}}-rx_{+}||,|{\hat{r}}-r|\big{)}<\bar{\varepsilon},\
\textrm{and}\
|\bar{W}({\hat{z}},{\hat{r}})|<\bar{\varepsilon}|\bar{s}|+\bar{\varepsilon}.$

Hence, according to the inequalities $s<\bar{s}$,
$|{\hat{r}}-r|<\bar{\varepsilon}=\min(\varepsilon,\varepsilon/\bar{s},1)/5\leq\varepsilon/5$
and putting ${\hat{z}}^{\prime}\stackrel{{\scriptstyle\triangle}}{{=}}rx_{+}$,
we obtain $s<\bar{s}<{\hat{\zeta}}x_{+}$,
$||{\hat{z}}-{\hat{z}}^{\prime}||<\bar{\varepsilon}<\varepsilon$,
${\hat{r}}<r+\bar{\varepsilon}\leq 1+\bar{\varepsilon},$ and

$\displaystyle|{W}({\hat{z}})|$
$\displaystyle=|\bar{W}({\hat{z}},{\hat{r}})+(1+\bar{\varepsilon}){\hat{r}}\bar{s}|<\bar{\varepsilon}\bar{s}+\bar{\varepsilon}+(1+\bar{\varepsilon}){\hat{r}}\bar{s}$
$\displaystyle<\bar{\varepsilon}\bar{s}+\bar{\varepsilon}+(1+\bar{\varepsilon})(\bar{s}+\varepsilon/5)<2{\varepsilon/5}+\bar{s}+2\varepsilon/5+\bar{\varepsilon}\varepsilon/5<\bar{s}+\varepsilon.$

The lemma has been proved. ∎

Hereinafter, in this section, we assume that every function $W_{\theta}$ is
locally Lipschitz continuous. For each
$(\check{x},\check{y})\in\operatorname*{epi}\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup}$
and $\varkappa>0$, denote by $\mathcal{Z}_{\varkappa}(\check{x},\check{y})$
the closed convex hull of the union of
${\hat{N}}_{0}(x,W_{{\theta}}(x);\operatorname*{epi}W_{{\theta}})$ for all
pairs $(\theta,x)\in\mathbb{R}_{+}\times{{\mathcal{X}}}$ satisfying

$\displaystyle\theta>1/\varkappa,\quad||x-\check{x}||<\varkappa,\quad\textrm{and}\quad|W_{\theta}(x)-\check{y}|<\varkappa.$
(36)

Note that, for such a $(\theta,x)$ we have
${\hat{N}}_{0}(x,W_{{\theta}}(x);\operatorname*{epi}W_{{\theta}})=\cup_{r\geq
0}r({\hat{\partial}}W_{{\theta}}(x)\times\\{-1\\})$ by [29, Theorem 8.9] and
$(-{\hat{\partial}}(-W_{{\theta}}))(x)\times\\{-1\\}\subset\mathcal{Z}_{\varkappa}(\check{x},\check{y})$
by [31, (1.75) and (1.83)].

######  Lemma 6.6.

Assume that the convex subset ${{\mathcal{X}}}\subset\mathbb{X}$ satisfies
${{\mathcal{X}}}=\operatorname*{cl}\operatorname*{int}{{\mathcal{X}}}$. Let,
for each positive $\theta$, the function $W_{\theta}:\mathbb{X}\to\mathbb{R}$
be locally Lipschitz continuous on ${{\mathcal{X}}}$. Let a point
$(\check{x},\check{y})$ lie on the boundary of
$\operatorname*{epi}\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup}$.

Then, for all sufficiently small $\varkappa>0$, the cone

$\mathcal{P}_{\varkappa}(\check{x},\check{y})\stackrel{{\scriptstyle\triangle}}{{=}}\\{(v,w)\in
T(\check{x};\operatorname*{cl}{{\mathcal{X}}})\times\mathbb{R}\,|\,\forall(\xi,-\lambda)\in\mathcal{Z}_{\varkappa}(\check{x},\check{y})\
\xi v-\lambda w\leq 0\\}$

is contained in the tangent cone
$T(\check{x},\check{y};\operatorname*{epi}\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup})$.

###### Proof.

Case 1. Let $(\check{x},\check{y})$ lie in the closure of the graph of
$\operatorname*{epi}\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup}.$ Hence there
exists a converging to $\check{x}$ sequence of $x_{n}\in\mathcal{X}$ such that
the sequence of $W_{\sup}(x_{n})$ converges to $\check{y}$; in particular,
$\check{x}$ lies in the closure of ${\mathcal{X}}$. Fix a positive
$\varkappa$.

Assume that a nonzero vector $v\in
T(\check{x};\operatorname*{cl}{{\mathcal{X}}})$, a number $w$, and a positive
number $t$ satisfy $t+t||v||+t|w|<\varkappa/3$,
$\check{x}+tv\in\operatorname*{int}{{\mathcal{X}}}$, and
$(\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup})(\check{x}+tv)>\check{y}+tw$. We
claim that $(v,w)\notin\mathcal{P}_{\varkappa}(\check{x},\check{y}).$

Choose a positive $\varepsilon<1$ satisfying
$t+t||v||+t|w|+\varepsilon<\varkappa/3$ and
$(\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup})(\check{x}+tv)>\check{y}+tw+\varepsilon$.
By the lower semicontinuity of $\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup}$
at $\check{x}+tv\in\operatorname*{int}{{\mathcal{X}}}$, we have
${x}_{n}+tv\in\operatorname*{int}{{\mathcal{X}}}$ and
$(\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup})({x}_{n}+tv)>\check{y}+tw+\varepsilon$
for all sufficient large $n$; further, it follows
$(x_{n};x_{n}+tv)\subset\operatorname*{int}\mathcal{X}$ by the convexity of
$\mathcal{X}$. We can also ensure that $||\check{x}-x_{n}||<\varepsilon$. For
a such $n$ one can find an unbounded sequence of $\theta_{k}(n)>1/\varkappa$
satisfying $W_{\theta_{k}(n)}(x_{n}+tv)>\check{y}+tw+\varepsilon$ for all
natural $k$. Now, since the sequence of $W_{\sup}(x_{n})$ converges to
$\check{y}$, we can fix natural $n$ and $k$ large enough that

$\displaystyle\theta_{k}(n)>1/\varkappa,\quad
W_{\theta_{k}(n)}(x_{n})<\check{y}+\varepsilon,\quad\check{y}+tw+\varepsilon<W_{\theta_{k}(n)}(x_{n}+tv).$
(37)

In the case $w\geq 0$, consider the following continuous function:

$[0;t]\ni\tau\mapsto
h(\tau)\stackrel{{\scriptstyle\triangle}}{{=}}W_{\theta_{k}(n)}(x_{n}+\tau
v)-\tau|w|-\check{y}-\varepsilon.$

Since (37) gives $h(0)<0<h(t)$, one can find a positive $\tau<t$ satisfying
$h(\tau)=0$. In this case one has
$W_{\theta_{k}(n)}(x_{n}+tv)-W_{\theta_{k}(n)}(x_{n}+\tau v)>tw-\tau w$. Put
$\bar{W}\stackrel{{\scriptstyle\triangle}}{{=}}W_{\theta_{k}(n)}$,
$x_{+}\stackrel{{\scriptstyle\triangle}}{{=}}x_{n}+tv$,
$x_{-}\stackrel{{\scriptstyle\triangle}}{{=}}x_{n}+\tau v\in[x_{n};x_{+}]$,
and $t^{\prime}\stackrel{{\scriptstyle\triangle}}{{=}}t-\tau>0$; hence we
obtain

$\displaystyle(x_{-};x_{+})\subset\operatorname*{int}\mathcal{X},\
x_{+}-x_{-}=t^{\prime}v,\ |t^{\prime}|\bar{W}=t^{\prime}W_{\theta_{k}(n)},\
\bar{W}(x_{+})-\bar{W}(x_{-})>t^{\prime}w,$ (38)
$\displaystyle|W_{\theta_{k}(n)}(x_{-})-\check{y}-\varepsilon|\leq 2t|w|.$
(39)

Assume that $w<0$ and
$W_{\theta_{k}(n)}(x_{n})\geq-2t|w|+\check{y}+\varepsilon$. By (37), the
inequality $W_{\theta_{k}(n)}(x_{n}+tv)-W_{\theta_{k}(n)}(x_{n})>tw$ holds. In
this case, putting
$\bar{W}\stackrel{{\scriptstyle\triangle}}{{=}}W_{\theta_{k}(n)}$,
$x_{+}\stackrel{{\scriptstyle\triangle}}{{=}}x_{n}+tv$,
$x_{-}\stackrel{{\scriptstyle\triangle}}{{=}}x_{n}$, and
$t^{\prime}\stackrel{{\scriptstyle\triangle}}{{=}}t>0$, we obtain (38);
further, we also have (39) by (37) and our assumption on
$W_{\theta_{k}(n)}(x_{n})$.

Assume that $w<0$, $W_{\theta_{k}(n)}(x_{n})<-2t|w|+\check{y}+\varepsilon$,
and $W_{\theta_{k}(n)}(x_{n}+tv)\leq t|w|+\check{y}+\varepsilon$. By (37), one
has
$W_{\theta_{k}(n)}(x_{n})-W_{\theta_{k}(n)}(x_{n}+tv)<-2t|w|+\check{y}+\varepsilon-(\check{y}+\varepsilon+tw)=-t|w|$.
Putting $\bar{W}\stackrel{{\scriptstyle\triangle}}{{=}}-W_{\theta_{k}(n)}$,
$x_{+}\stackrel{{\scriptstyle\triangle}}{{=}}x_{n}$,
$x_{-}\stackrel{{\scriptstyle\triangle}}{{=}}x_{n}+tv$, and
$t^{\prime}\stackrel{{\scriptstyle\triangle}}{{=}}-t<0$, we have (38) again.
In this case, we again obtain (39) by (37) and our assumption on
$W_{\theta_{k}(n)}(x_{n}+tv)$.

Finally, in the case $w<0$ and
$W_{\theta_{k}(n)}(x_{n}+tv)>t|w|+\check{y}+\varepsilon$, again consider the
continuous function

$[0;t]\ni\tau\mapsto
h(\tau)\stackrel{{\scriptstyle\triangle}}{{=}}W_{\theta_{k}(n)}(x_{n}+\tau
v)-\tau|w|-\check{y}-\varepsilon$

and choose a positive $\tau<t$ satisfying $h(\tau)=0$. Hence the inequality

$-W_{\theta_{k}(n)}(x_{n})+W_{\theta_{k}(n)}(x_{n}+\tau
v)-h(\tau)>-\check{y}-\varepsilon+(\tau|w|+\check{y}+\varepsilon)=\tau|w|$

holds. Putting
$\bar{W}\stackrel{{\scriptstyle\triangle}}{{=}}-W_{\theta_{k}(n)}$,
$x_{+}\stackrel{{\scriptstyle\triangle}}{{=}}x_{n}$,
$x_{-}\stackrel{{\scriptstyle\triangle}}{{=}}x_{n}+\tau v$, and
$t^{\prime}\stackrel{{\scriptstyle\triangle}}{{=}}-\tau<0$, we have (38)
again. In this case, we also obtain (39) by the choice of $\tau$ and $x_{-}$.

In all these cases, by (38) one has
$(x_{-};x_{+})\subset\operatorname*{int}\mathcal{X}$ and
$\bar{W}(x_{+})-\bar{W}(x_{-})>t^{\prime}w$; now, Lemma 6.5 with $W=\bar{W}$,
$s=t^{\prime}w$ gives a point ${\hat{z}}\in\operatorname*{int}{{\mathcal{X}}}$
and a subgradient
$t^{\prime}\zeta^{\prime}\in{\hat{\partial}}(t^{\prime}\bar{W})({\hat{z}})$
satisfying

$t^{\prime}w<\zeta^{\prime}t^{\prime}v,\qquad||{\hat{z}}-x_{-}||<\varepsilon+||t^{\prime}v||,\qquad|\bar{W}({\hat{z}})-\bar{W}(x_{-})|<\varepsilon+|t^{\prime}w|.$

In the case $w\geq 0$, it implies $w<\zeta^{\prime}v=\zeta v$ for some
$\zeta\in{\hat{\partial}}W_{\theta_{k}(n)}({\hat{z}})$. In the case $w<0$, it
leads to $w<\zeta v$ for some
$\zeta\in{\hat{\partial}}W_{\theta_{k}(n)}({\hat{z}})\cup(-{\hat{\partial}}(-W_{\theta_{k}(n)})({\hat{z}}))$.
In all these cases, in addition to $\zeta v-w>0$, $x_{-}\in[x_{n};x_{n}+tv]$,
$|t^{\prime}|\leq t$, and (39), by the choice of $n$ and $\varepsilon$, the
following inequalities hold:

$\displaystyle||{\hat{z}}-\check{x}||$
$\displaystyle\leq||\check{x}-x_{n}||+||x_{n}-x_{-}||+||x_{-}-{\hat{z}}||<2(\varepsilon+t||v||)<\varkappa,$
$\displaystyle|W_{\theta_{k}(n)}({\hat{z}})-\check{y}|$
$\displaystyle\leq\varepsilon+t|w|+|W_{\theta_{k}(n)}(x_{-})-\check{y}|\leq
2\varepsilon+3t|w|<\varkappa.$

Now, since $\theta_{k}(n)>1/\varkappa$, taking into account (36), the vector
$(\zeta,-1)$ lies in $\mathcal{Z}_{\varkappa}(\check{x},\check{y})$. So, we
obtain that $\zeta v-\lambda w>0$ for some
$(\zeta,-\lambda)\in\mathcal{Z}_{\varkappa}(\check{x},\check{y})$, i.e.,
$(v,w)\notin\mathcal{P}_{\varkappa}(\check{x},\check{y})$. We have proved that
$(v,w)\notin\mathcal{P}_{\varkappa}(\check{x},\check{y})$.

Thus, for each $(v,w)\in\mathcal{P}_{\varkappa}(\check{x},\check{y})$ we have
shown

$\displaystyle(\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\sup})(\check{x}+tv)\leq\check{y}+tw\qquad\forall
t\in\big{(}0;\frac{\varkappa}{3(1+||v||+|w|}\big{)}$ (40)

if $tv\in-\check{x}+\operatorname*{int}{{\mathcal{X}}}$; in particular, due to
the convexity of
$\mathcal{X}=\operatorname*{cl}\operatorname*{int}\mathcal{X}$, (40) holds if
$v\in\operatorname*{int}T(\check{x};\operatorname*{cl}{{\mathcal{X}}})$. Since
the function $\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup}$ is lower
semicontinuous, (40) holds even if $v\in
T(\check{x};\operatorname*{cl}{{\mathcal{X}}})$. However,
$(v,w)\in\mathcal{P}_{\varkappa}(\check{x},\check{y})$ implies $v\in
T(\check{x};\operatorname*{cl}{{\mathcal{X}}})$ by the definition of
$\mathcal{P}_{\varkappa}(\check{x},\check{y})$. Then, one has (40) for each
$(v,w)\in\mathcal{P}_{\varkappa}(\check{x},\check{y})$. The passage to the
limit as $t\downarrow 0$ ensures $(v,w)\in
T(\check{x},\check{y};\operatorname*{epi}\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup})$
for each $(v,w)\in\mathcal{P}_{\varkappa}(\check{x},\check{y})$ by definition
of the tangent cone. Then, we have proved
$\mathcal{P}_{\varkappa}(\check{x},\check{y})\subset
T(\check{x},\check{y};\operatorname*{epi}\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup})$
in this case.

Case 2. Assume that a point $(\check{x},\check{y})$ lies on the closure of the
boundary of $\operatorname*{epi}\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup}.$
Excluding Case 1, one can also suppose that this point does not lie in the
closure of the graph of $\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup}$. Hence
there exists a sufficient small positive $\varkappa$ such that
$|\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup}(x)-\check{y}|>\varkappa$ holds
for all $x\in\mathbb{X},||x-\check{x}||<\varkappa$.

Consider a vector $(v,w)\in\mathbb{X}\times\mathbb{R}$ and a positive $t$
satisfying
$(\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup})(\check{x}+tv)>\check{y}$,
$\check{x}+tv\in\operatorname*{int}{{\mathcal{X}}}$, and
$t+t||v||+t|w|<\varkappa/3$. We claim that
$(v,w)\notin\mathcal{P}_{\varkappa}(x,y).$

By choice of $\varkappa$, it follows from $||tv||<\varkappa$ and
$(\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup})(\check{x}+tv)>\check{y}$ that
$(\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup})(\check{x}+tv)>\check{y}+\varkappa$.
On the other hand,
$(\check{x},\check{y})\in\operatorname*{epi}\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup}$
implies
$(\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup})(\check{x})\leq\check{y}$. Then,
one can choose a point $\bar{x}\in{{\mathcal{X}}}$ near enough to $\check{x}$
such that $||\bar{x}-\check{x}||<\varkappa/6$,
$\bar{x}+tv\in\operatorname*{int}{{\mathcal{X}}}$,
$W_{\sup}(\bar{x})<\check{y}+\varkappa/4$, and
$W_{\sup}(\bar{x}+tv)>\check{y}+\varkappa/2$ hold. Hence there is a positive
$\theta>1/\varkappa$ satisfying $W_{\theta}(\bar{x})<\check{y}+\varkappa/3$
and $W_{\theta}(\bar{x}+tv)>\check{y}+\varkappa/2$.

Then, since $W_{\theta}$ is continuous, one can find positive
$t^{\prime},t^{\prime\prime}$ ($0<t^{\prime}<t^{\prime\prime}<t$) satisfying

$W_{\theta}(\bar{x}+t^{\prime}v)=\check{y}+\varkappa/3<\check{y}+\varkappa/2=W_{\theta}(\bar{x}+t^{\prime\prime}v).$

Due to the convexity of $\mathcal{X}$ it follows from
$\bar{x}\in{{\mathcal{X}}}$ and
$\bar{x}+tv\in\operatorname*{int}{{\mathcal{X}}}$ that the points
$\bar{x}+t^{\prime}v$ and $\bar{x}+t^{\prime\prime}v$ lie in
$\operatorname*{int}{{\mathcal{X}}}$; further, one has
$||\bar{x}+t^{\prime}v-\check{x}||<\varkappa/3+||\bar{x}-\check{x}||<\varkappa/2.$

Therefore, we obtain
$W_{\theta}(\bar{x}+t^{\prime\prime}v)-W_{\theta}(\bar{x}+t^{\prime}v)>\varkappa/6.$
By Lemma 6.5 with $W=W_{\theta}$, $x_{+}=\bar{x}+t^{\prime\prime}v$,
$x_{-}=\bar{x}+t^{\prime}v$, $s=\varkappa/6$, and $\varepsilon=\varkappa/3$,
one can find a point $z\in\operatorname*{int}{{\mathcal{X}}}$ and a
subgradient $\zeta\in{\hat{\partial}}W_{\theta}(z)$ such that

$\varkappa/6<(t^{\prime\prime}-t^{\prime})\zeta v,\
||z-\bar{x}+t^{\prime}v||<\varkappa/2,\
|W_{\theta}(z)-W_{\theta}(\bar{x}+t^{\prime}v)|<\varkappa/2.$

Now,
$|W_{\theta}(z)-\check{y}|\leq|W_{\theta}(z)-W_{\theta}(\bar{x}+t^{\prime}v)|+\varkappa/3<\varkappa$
and
$||z-\check{x}||\leq||z-\bar{x}-t^{\prime}v||+||\bar{x}+t^{\prime}v-\check{x}||<\varkappa/2+\varkappa/2=\varkappa$
hold. Since $\theta>1/\varkappa$, the vector $(\zeta,-1)$ lies in
$\mathcal{Z}_{\varkappa}(\check{x},\check{y})$. Further, by the choice of $t$,
we have

$0\leq|w|-w<\frac{\varkappa}{6t}-w\leq\frac{\varkappa}{6(t^{\prime\prime}-t^{\prime})}-w<\zeta
v-w.$

We obtain $\zeta v-\lambda w>0$ for some $(\zeta,\lambda=-1)$ lying in
$\mathcal{Z}_{\varkappa}(\check{x},\check{y})$. By the definition of
$\mathcal{P}_{\varkappa}(\check{x},\check{y})$, we have proved
$(v,w)\notin\mathcal{P}_{\varkappa}(\check{x},\check{y})$.

Thus, for each $(v,w)\in\mathcal{P}_{\varkappa}(\check{x},\check{y})$ we have
proved (40) again if $tv\in-\check{x}+\operatorname*{int}{{\mathcal{X}}}$;
repeating the last paragraph of Case 1 literally, we obtain the inclusion
$\mathcal{P}_{\varkappa}(\check{x},\check{y})\subset
T(\check{x},\check{y};\operatorname*{epi}\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup})$
in the latter case. ∎

The following lemma does not follow from the similar results in [18, 49, 50].

######  Lemma 6.7.

Assume that the nonempty convex subset ${{\mathcal{X}}}$ of $\mathbb{X}$ and
the family of functions $W_{\theta}:\mathbb{X}\to\mathbb{R}$, $\theta>0$
satisfy one of the following conditions:

  * (i)

each function $W_{\theta}$ is locally Lipschitz continuous on
${{\mathcal{X}}}$ and
${{\mathcal{X}}}=\operatorname*{cl}\operatorname*{int}{{\mathcal{X}}}$;

  * (ii)

each function $W_{\theta}$ is Fréchet differentiable on ${{\mathcal{X}}}$.

Then, for every point $(\check{x},\check{y})$ of the graph of
$\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup}$, one has

$N(\check{x},\check{y};\operatorname*{epi}\operatorname*{lsc}\nolimits_{{\mathcal{X}}}{W}_{\sup})\subset
N(\check{x};{{\mathcal{X}}})\times\\{0\\}+\bigcap_{\varkappa>0}\operatorname*{co}\operatorname*{cl}\\!\\!\\!\\!\\!\\!\bigcup_{\begin{subarray}{c}\theta>1/\varkappa,\
x\in{{\mathcal{X}}},\ \\\
||x-\check{x}||<\varkappa,|{W}_{\theta}(x)-\check{y}|<\varkappa\end{subarray}}\\!\\!\\!\\!\\!\\!{\hat{N}}_{0}(x,W_{\theta}(x);\operatorname*{epi}{W}_{\theta}).$
(41)

###### Proof.

By [30, Theorem 1.10], since $\mathbb{X}$ is finite-dimensional, the cone
${\hat{N}}_{0}(x;\operatorname*{cl}{{\mathcal{X}}})$ is the polar cone of
$T(x;\operatorname*{cl}{{\mathcal{X}}})$. Since the set ${{\mathcal{X}}}$ is
convex, the convex cones
$T(\check{x};\operatorname*{cl}{{\mathcal{X}}})\times\mathbb{R}$ and
$\check{N}_{0}(\check{x};\operatorname*{cl}{{\mathcal{X}}})\times\\{0\\}=N(\check{x};{{\mathcal{X}}})\times\\{0\\}$
are polar to each other; therefore, one has

$T(\check{x};\operatorname*{cl}{{\mathcal{X}}})\times\mathbb{R}=\big{\\{}(v,-\lambda)\in\mathbb{X}\times\mathbb{R}\,\big{|}\,\forall(\zeta,0)\in
N(\check{x};\operatorname*{cl}{{\mathcal{X}}})\times\\{0\\}\ \zeta
v-0\cdot\lambda\leq 0\big{\\}}.$

Then, $(v,w)$ lies in $\mathcal{P}_{\varkappa}(\check{x},\check{y})$ iff
$(\xi+\zeta)v-\lambda w\leq 0$ holds for all
$(\xi,-\lambda)+(\zeta,0)\in\mathcal{Z}_{\varkappa}(\check{x},\check{y})+N(\check{x};{{\mathcal{X}}})\times\\{0\\}$.
So, $\mathcal{P}_{\varkappa}(\check{x},\check{y})$ is the polar cone of
$\mathcal{Z}_{\varkappa}(\check{x},\check{y})+N(\check{x};{{\mathcal{X}}})\times\\{0\\}$.

Let the assumption $(i)$ hold. Since the closed cone
$\mathcal{Z}_{\varkappa}(\check{x},\check{y})+N(\check{x};{{\mathcal{X}}})\times\\{0\\}$
is also convex, it coincides with its bipolar cone by [29, Corollary 6.21].
Then, the inclusion
$\mathcal{P}_{\varkappa}(\check{x},\check{y})\subset\operatorname*{co}T(\check{x},\check{y};\operatorname*{epi}\operatorname*{lsc}_{{\mathcal{X}}}W_{\sup})$
for all $\varkappa>0$ small enough implies the inclusion

$\hat{N}_{0}(\check{x},\check{y};\operatorname*{epi}\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\sup})\subset\bigcap_{\varkappa>0}\mathcal{Z}_{\varkappa}(\check{x},\check{y})+N(\check{x};{{\mathcal{X}}})\times\\{0\\}.$

Note that, for any set $A$, one has $\hat{N}_{0}(z;A)\subset\\{0\\}$ if the
point $z$ does not lies on the boundary of $A$; therefore,

$\operatorname*{Limsup}_{(x,y)\to(\check{x},\check{y})}\hat{N}_{0}(x,y;\operatorname*{epi}\operatorname*{lsc}\nolimits_{{\mathcal{X}}}W_{\sup})\subset\bigcap_{\varkappa>0}\mathcal{Z}_{\varkappa}(\check{x},\check{y})+N(\check{x};{{\mathcal{X}}})\times\\{0\\}.$

Now, since $\mathbb{X}$ is finite-dimensional, the desired inclusion is
obtained by (5d).

Under assumption $(ii)$ the set ${{\mathcal{X}}}$ and functions
$W_{\theta}|_{\operatorname*{aff}{{\mathcal{X}}}}$ satisfy $(i)$ with
$\operatorname*{aff}{{\mathcal{X}}}$ instead of $\mathbb{X}$ because
$A=\operatorname*{cl}\operatorname*{ri}{A}$ holds for all convex set $A$.
Since
${\hat{\partial}}(W_{\theta}|_{\operatorname*{aff}{{\mathcal{X}}}})(\check{x})$
and ${\hat{\partial}}W_{\theta}(\check{x})$ are singletons, by
${\hat{\partial}}(W_{\theta}|_{\operatorname*{aff}{{\mathcal{X}}}})(\check{x})-{\hat{\partial}}W_{\theta}(\check{x})\subset
N(\check{x};\operatorname*{aff}{{\mathcal{X}}})$, the first item of our lemma
entails the needed inclusion.

The final lemma has been proved. ∎

##  7 Proofs of Theorems 4.1 and 4.2

We will prove both theorems simultaneously.

###### Proof.

We need some notations. For all $r\in\mathbb{R}$, $x\in\mathbb{X}$,
$\upsilon\in U$, $u\in\mathcal{U}$, $t\in\mathbb{R}_{+}$, $\theta\geq t$, set

$\displaystyle\bar{\mathbb{X}}\stackrel{{\scriptstyle\triangle}}{{=}}\mathbb{R}\times\mathbb{X},$
$\displaystyle{\hat{\bar{x}}}\stackrel{{\scriptstyle\triangle}}{{=}}(0,{\hat{y}}(0))\in\bar{\mathbb{X}},$
$\displaystyle\quad\bar{e}_{0}^{*}\stackrel{{\scriptstyle\triangle}}{{=}}(1,0,\dots,0)\in\bar{\mathbb{X}}^{*},$
$\displaystyle\bar{f}(t,(r,x),\upsilon)$
$\displaystyle\stackrel{{\scriptstyle\triangle}}{{=}}(f_{0}(t,x,\upsilon),f(t,x,\upsilon)),$
$\displaystyle\mathrm{z}((r,x),t,u;\theta)$
$\displaystyle\stackrel{{\scriptstyle\triangle}}{{=}}(r+J(x,t,u;\theta),\mathrm{y}(x,t,u;\theta)),$
$\displaystyle\Delta\bar{f}(t,(r,x),\upsilon)$
$\displaystyle\stackrel{{\scriptstyle\triangle}}{{=}}\bar{f}(t,(r,x),\upsilon)-\bar{f}(t,(r,x),{\hat{u}}(t)).$

The reduction of (1b) to $(\lambda,-\psi)\in\mathfrak{Q}_{\lim}$

Take the sets ${\hat{\mathbb{G}}}_{\exists}$ and
${\hat{\mathbb{G}}}_{\partial}$ from $(H3)$ and $(H4)$. Consider a point
$x\in{\hat{\mathbb{G}}}_{\exists}$ satisfying
$(0,x)\in{\hat{\mathbb{G}}}_{\partial}$. For such $x$ define transition
matrixes

$\displaystyle{A}_{x,\theta}(t)$
$\displaystyle\stackrel{{\scriptstyle\triangle}}{{=}}\frac{\partial\mathrm{z}(\bar{x},t,{\hat{u}};\theta)}{\partial\bar{x}}\Big{|}_{\bar{x}=(r,\mathrm{y}(x,0,{\hat{u}};t))}\qquad\forall\theta\in\mathbb{R}_{+},t\in[0;\theta].$

This definition is valid because
$\frac{\partial\mathrm{z}((r,x),t,{\hat{u}};\theta)}{\partial
r}=\frac{\partial(r+J(x,t,{\hat{u}};\theta),\mathrm{y}(x,t,{\hat{u}};\theta))}{\partial
r}$ is independent of $r$ whenever $t,\theta$. One verifies easily the
following properties of transition matrix:

$\displaystyle{A}_{x,T}(t){A}_{x,t}(0)={A}_{x,T}(0),\
{A}^{-1}_{x,t}(0)={A}^{-1}_{x,T}(0){A}_{x,T}(t)\ \qquad\forall
x\in\mathbb{X},T\geq t\geq 0.$ (42)

Further, ${A}_{x,\theta}|_{[0;\theta]}$ is the solution of the Cauchy problem

$\displaystyle\frac{d{A}_{x,\theta}(t)}{dt}=-{A}_{x,\theta}(t)\frac{\partial\bar{f}}{\partial\bar{x}}(t,\bar{x},{\hat{u}}(t))\Big{|}_{\bar{x}=\mathrm{z}((0,x),0,{\hat{u}};t)},\
{A}_{x,\theta}(\theta)=E\qquad\forall t\in[0;\theta].$ (43)

Note that the graph of ${\hat{y}}$ is contained in
${\hat{\mathbb{G}}}_{\partial}$ and
${\hat{y}}(0)\in{\hat{\mathbb{G}}}_{\exists}$. Hence we may define

$\mathfrak{Q}_{\lim}\stackrel{{\scriptstyle\triangle}}{{=}}\big{\\{}(\lambda,-\psi)\in\mathbb{R}_{+}\times
C(\mathbb{R}_{+};\mathbb{X})\,\big{|}\,\,\forall t\geq 0\
(\lambda,-\psi(0)){A}^{-1}_{{\hat{y}}(0),t}(0)=(\lambda,-\psi(t))\big{\\}}.$
(44)

Consider a pair $({\hat{\lambda}},-{\hat{\psi}})\in\mathfrak{Q}_{\lim}$. Then,
for almost all positive $t$, one has the equality
$({\hat{\lambda}},-{\hat{\psi}}(t))\bar{f}\big{(}t,\mathrm{z}({\hat{\bar{x}}},0,{\hat{u}};t),\upsilon\big{)}=-H\big{(}{\hat{y}}(t),{\hat{\psi}}(t),\upsilon,{\hat{\lambda}},t\big{)}.$
First, it follows from this equality that to prove (1c) we must only verify

$\displaystyle
0\leq({\hat{\lambda}},-{\hat{\psi}}(0)){A}^{-1}_{{\hat{y}}(0),t}(0)\Big{(}\bar{f}\big{(}t,\mathrm{z}\big{(}{\hat{\bar{x}}},0,{\hat{u}};t\big{)},\upsilon\big{)}-\bar{f}\big{(}t,\mathrm{z}\big{(}{\hat{\bar{x}}},0,{\hat{u}};t\big{)},{\hat{u}}(t)\big{)}\Big{)}$
(45)

for all $\upsilon\in U$ and for almost all positive $t$. Second, by (42), the
mentioned equality for $\upsilon={\hat{u}}(t)$ also leads to

$\displaystyle\frac{d({\hat{\lambda}},-{\hat{\psi}}(t))}{dt}$
$\displaystyle\stackrel{{\scriptstyle(\ref{1983})}}{{=}}({\hat{\lambda}},-{\hat{\psi}}(0)){A}^{-1}_{{\hat{y}}(0),\theta}(0)\frac{d{{A}_{{\hat{y}}(0),\theta}(t)}}{dt}$
$\displaystyle\stackrel{{\scriptstyle(\ref{1956})}}{{=}}(-{\hat{\lambda}},{\hat{\psi}}(0)){A}^{-1}_{{\hat{y}}(0),\theta}(0){A}_{{\hat{y}}(0),\theta}(t)\frac{\partial\bar{f}}{\partial\bar{x}}(t,\bar{x},{\hat{u}}(t))\Big{|}_{\bar{x}=\mathrm{z}({\hat{\bar{x}}},0,{\hat{u}};t)}$
$\displaystyle\stackrel{{\scriptstyle(\ref{1983})}}{{=}}\Big{(}0,\frac{\partial
H}{\partial{x}}\big{(}t,{\hat{y}}(t),{\hat{\psi}}(t),{\hat{u}}(t),{\hat{\lambda}},t\big{)}\Big{)}.$

Thus, each pair $({\hat{\lambda}},-{\hat{\psi}})\in\mathfrak{Q}_{\lim}$
satisfies (1b).

So, we must find a nontrivial
$({\hat{\lambda}},-{\hat{\psi}})\in\mathfrak{Q}_{\lim}$ satisfying all the
desired transversality conditions and the condition (45) for all $\upsilon\in
U$ and almost all positive $t$.

The construction of an open set
$\mathfrak{T}^{\textrm{full}}\stackrel{{\scriptstyle\triangle}}{{=}}\cup_{i\in\mathbb{N}}\operatorname*{int}\cap_{j\geq
i}\mathfrak{F}_{j}$ of full measure

Let $U^{\textrm{dense}}=\\{u^{1},u^{2},\dots\\}$ be a dense countable subset
of $U$; fix its numeration. Take from $(H4)$ and $(H5)$ open sets
${\hat{\mathbb{G}}}_{\partial},{\mathbb{G}}_{u^{1}},{\mathbb{G}}_{u^{2}},\dots$
For each natural $n$ the intersection of
${\hat{\mathbb{G}}}_{\partial},{\mathbb{G}}_{u^{1}},\dots,{\mathbb{G}}_{u^{n}}$
contains the graph of ${\hat{y}}$; accordingly, for all $i\in[1\\!:\\!n]$, the
maps

$(t,\bar{x})\mapsto\big{(}\Delta\bar{f}(t,\bar{x},u^{i}),\bar{f}(t,\bar{x},{\hat{u}}(t))\big{)},\qquad(t,\bar{x})\mapsto\Big{(}\frac{\partial\Delta\bar{f}}{\partial\bar{x}}(t,\bar{x},u^{i}),\frac{\partial\bar{f}}{\partial\bar{x}}(t,\bar{x},{\hat{u}}(t))\Big{)}$
(46)

are Borel measurable in $t$ and continuous in $\bar{x}$ on the open set
$\widetilde{\mathfrak{G}}_{n}\stackrel{{\scriptstyle\triangle}}{{=}}\\{(t,r,x)\in\mathbb{R}_{+}\times\bar{\mathbb{X}}\,|\,(t,x)\in{\hat{\mathbb{G}}}_{\partial}\cap{\mathbb{G}}_{u^{1}}\cap\cdots\cap{\mathbb{G}}_{u^{n}}\\}$.

Then, for each natural $n$, due to Scorza-Dragoni’s theorem [51, 52], one can
find a closed subset $\mathfrak{F}_{n}\subset[0;n]$ with
$\operatorname*{meas}(\mathfrak{F}_{n})>n-2^{-n}$ such that maps (46) are
continuous in $(t,\bar{x})$ on
$(\mathfrak{F}_{n}\times\bar{{\mathbb{X}}})\times\widetilde{\mathfrak{G}}_{n}$
for all $i\in[1\\!:\\!n]$. Since, for all natural $n$ large enough,
$\operatorname*{meas}([0;n]\setminus\operatorname*{int}\cap_{j\geq
n}\mathfrak{F}_{j})=\operatorname*{meas}([0;n]\setminus\cap_{j\geq
n}^{\infty}\mathfrak{F}_{j})<2^{-n+1}$ holds, for each natural $n$, the
intersection of the interval $[0;n]$ and the set

$\mathfrak{T}^{\textrm{full}}\stackrel{{\scriptstyle\triangle}}{{=}}\cup_{i\in\mathbb{N}}\operatorname*{int}\cap_{j\geq
i}\mathfrak{F}_{j}$

has full measure. Now, each $t\in\mathfrak{T}^{\textrm{full}}$ lies in
$\operatorname*{int}\mathfrak{F}_{j}$ and has a neighbourhood within
$\mathfrak{F}_{j}$ for all sufficiently large natural $j$; in particular, the
set $\mathfrak{T}^{\textrm{full}}$ also has no isolated points. Further, for
all natural $i$, maps (46) are continuous on $\mathfrak{T}^{\textrm{full}}$.
In particular, we need only prove that there exists a nonzero
$({\hat{\lambda}},-{\hat{\psi}})\in\mathfrak{Q}_{\lim}$ satisfying (6) and
(44) such that (45) holds for all $\upsilon=u^{i}\in U^{\textrm{dense}}$,
$t\in\mathfrak{T}^{\textrm{full}}$.

The construction of some countable family of $(t_{i},u_{i})$ dense in
$\mathfrak{T}^{\textrm{full}}\times U$

Similarly to the proof of [53, Lemma 4.1], we will construct countable sets
$\mathfrak{D}^{k}\subset\mathfrak{T}^{\textrm{full}}$, dense in
$\operatorname*{int}\mathfrak{F}_{k}$. Specifically, for each natural $k$ and
time $t\in\operatorname*{int}\mathfrak{F}_{k}$, one can find a sequence of
elements $\hat{\tau}_{n}(t,k)\in\mathfrak{D}^{n}$ converging to $t$. In
addition, due to $\mathfrak{D}^{k}\subset\mathfrak{F}_{k}$, putting
$\tau_{n}(t)\stackrel{{\scriptstyle\triangle}}{{=}}\hat{\tau}_{n}(t,k)$, we
obtain the following convergences

$\displaystyle\bar{f}(\tau_{n}(t),\bar{x},{\hat{u}}(\tau_{n}(t)))\to\bar{f}(t,\bar{x},{\hat{u}}(t))\textrm{\
and\
}\Delta\bar{f}(\tau_{n}(t),\bar{x},u^{i})\to\Delta\bar{f}(t,\bar{x},u^{i})$
(47)

as $n\uparrow\infty$ for all $i\in[1\\!:\\!k]$ and
$(t,\bar{x})\in\widetilde{\mathfrak{G}}_{k}$. Since
$\mathfrak{T}^{\textrm{full}}\cap\operatorname*{int}\mathfrak{F}_{k}$ has no
isolated points, one can conclude that all sets $\mathfrak{D}^{j}$ are
disjoint, because we may choose the elements of $\mathfrak{D}^{k}$ from
$(\mathfrak{T}^{\textrm{full}}\cap\operatorname*{int}\mathfrak{F}_{k})\setminus\cup_{j<k}\mathfrak{D}^{j}$
instead of
$\mathfrak{T}^{\textrm{full}}\cap\operatorname*{int}\mathfrak{F}_{k}$.

Fix a prime number $q$. Now, put
$\mathfrak{T}_{q}^{\textrm{dense}}\stackrel{{\scriptstyle\triangle}}{{=}}\cup_{j\in\mathbb{N}}\mathfrak{D}^{q^{j}}\subset\mathfrak{T}^{\textrm{full}}$;
this set is a countable dense subset of $\mathfrak{T}^{\textrm{full}}$. Hence,
since every $t\in\mathfrak{T}^{\textrm{full}}$ lies in the interior of
$\cap_{j>N}\mathfrak{F}_{q^{j}}$ for a certain natural $N$, we will pick a
converging to $t$ sequence of elements $\tau_{n}$ of
$\mathfrak{T}_{q}^{\textrm{dense}}$. Indeed, first, put $k(1)=1$ and
$\tau_{1}\stackrel{{\scriptstyle\triangle}}{{=}}\hat{\tau}_{k(1)}(t,q^{N})$.
After the choice of
$\tau_{n}\in\mathfrak{D}^{q^{N+n-1}}\subset\mathfrak{T}_{q}^{\textrm{dense}}$,
one can choose the next natural $k(n+1)>k(n)$ such that
$\tau_{n+1}\stackrel{{\scriptstyle\triangle}}{{=}}\hat{\tau}_{k(n+1)}(t,q^{N+n})\in\mathfrak{D}^{q^{N+n}}\subset\mathfrak{T}_{q}^{\textrm{dense}}$
satisfies $|\tau_{n+1}-t|\leq|\tau_{n}-t|/2$. Repeating this procedure, we
obtain a converging to $t$ sequence of points
$\tau_{n}\in\mathfrak{T}_{q}^{\textrm{dense}}$ such that, for all natural $i$,
one has the convergences (47) with $\tau_{n}(t)=\tau_{n}$; further, maps (46)
are continuous in $(t,\bar{x})$ at every point $(\tau_{n},\bar{x})$ lying in
$\widetilde{\mathfrak{G}}_{n}$.

Set
$\mathfrak{T}_{*}^{\textrm{dense}}\stackrel{{\scriptstyle\triangle}}{{=}}\cup_{q\in\mathbb{P}}{\mathfrak{T}}^{\textrm{dense}}_{q}$;
here $\mathbb{P}$ is the set of all prime numbers. Let
$(t_{i})_{i\in\mathbb{N}}$ and $(q_{i})_{i\in\mathbb{N}}$ be some orderings of
the elements of $\mathfrak{T}_{*}^{\textrm{dense}}$ and $\mathbb{P}$,
respectively. Since, for each natural $i$, the time instant
$t_{i}\in\mathfrak{T}_{*}^{\textrm{dense}}$ lies in a unique set
$\mathfrak{T}_{q_{i}}^{\textrm{dense}}$ for a unique prime $q_{i}$, we take
the correspondence

$\mathbb{N}\ni
i\mapsto(t_{i},u_{i}\stackrel{{\scriptstyle\triangle}}{{=}}u^{q_{i}})\in\mathfrak{T}_{*}^{\textrm{dense}}\times
U^{\textrm{dense}}.$

So, by the construction of $\mathfrak{T}_{*}^{\textrm{dense}}$ and
$U^{\textrm{dense}}$, due to the continuity of maps (46) in $x$, taking into
account (47), we may verify (45) only for
$(t,\upsilon)\stackrel{{\scriptstyle\triangle}}{{=}}(t_{i},u_{i})$
($i\in\mathbb{N}$). Then, we must merely to assign to every natural $K$ some
number $T>K$ and a certain non-trivial pair
$({\hat{\lambda}}_{K},-{\hat{\psi}}_{K})\in\mathfrak{Q}_{\lim}$ satisfying
(6), (7) on $[0;T]$ such that each
$(t,\upsilon)\stackrel{{\scriptstyle\triangle}}{{=}}(t_{i},u_{i})$ satisfies
(45) with $({\hat{\lambda}}_{K},-{\hat{\psi}}_{K})\in\mathfrak{Q}_{\lim}$ for
all $K$ large enough.

The choice of $T$, $\varepsilon$, and $\varkappa$. The construction of maps
$u_{\omega}$, $z_{\omega}$, and $b_{\omega}$

Fix a natural $K$. Put
$T\stackrel{{\scriptstyle\triangle}}{{=}}\max\big{(}t_{1},\dots,t_{K},K\big{)}+1$
and

$I_{K}\stackrel{{\scriptstyle\triangle}}{{=}}\Big{\\{}i\in[1\\!:\\!K]\,\Big{|}\,||\Delta\bar{f}(t_{i},(0,{\hat{y}}(t_{i})),u_{i})||+\Big{\|}\frac{\partial\Delta\bar{f}(t,\bar{x},u_{i})}{\partial\bar{x}}\Big{|}_{\bar{x}=(0,{\hat{y}}(t_{i}))}\Big{\|}<K\Big{\\}}.$

Since each $t_{i}$ lies in
$\mathfrak{T}_{q_{i}}^{\textrm{dense}}\subset\mathfrak{T}^{\textrm{full}}=\cup_{n\in\mathbb{N}}\operatorname*{int}\cap_{j>n}\mathfrak{F}_{j}$,
one can choose a natural $N>T$ such that
$\\{u_{1},\dots,u_{K}\\}\subset\\{u^{1},\dots,u^{N}\\}$ holds and $t_{i}$ lie
in $\operatorname*{int}\cap_{j\geq N}\mathfrak{F}_{j}$ for all
$i\in[1\\!:\\!K]$. Then, there exists a positive $\varepsilon<1/K$ such that
the $2\varepsilon$-neighbourhood of each $t_{i}$ ($i\in[1\\!:\\!K]$) is
contained in $\operatorname*{int}\mathfrak{F}_{N}$. Further, decreasing
$\varepsilon$ if necessary, one can guarantee that these neighbourhoods of
$t_{i}$ are disjoint. Note that all maps (46) with $i\in[1\\!:\\!K]$ are
continuous in $(t,x)$ on the open set
$(\operatorname*{int}\mathfrak{F}_{N}\times\bar{\mathbb{X}})\cap\widetilde{\mathfrak{G}}_{N}$.
Therefore, again decreasing $\varepsilon$ if necessary, one can ensure
$(\tau,\mathrm{z}(\bar{x},0,{\hat{u}};\tau))\in\widetilde{\mathfrak{G}}_{N}$
for all $\tau\in[0;T]$ and

$||\Delta\bar{f}(t,\mathrm{z}(\bar{x},0,{\hat{u}};t),u_{i})||+\Big{\|}\frac{\partial\Delta\bar{f}(t,\bar{x}^{\prime},u_{i})}{\partial\bar{x}}\Big{|}_{\bar{x}^{\prime}=\mathrm{z}(\bar{x},0,{\hat{u}};t)}\Big{\|}<K\qquad\forall
t\in[t_{i};t_{i}+\varepsilon]$ (48)

if $i\in I_{K}$ and $\bar{x}=(r,x)\in\bar{\mathbb{X}}$ satisfying
$||x-{\hat{y}}(0)||\leq\varepsilon$. For all positive
$\varkappa\leq\varepsilon$ put

$\Omega_{\varkappa}\stackrel{{\scriptstyle\triangle}}{{=}}\big{\\{}(r,\Delta
x,\gamma)\in\mathbb{R}\times{\mathbb{X}}\times[0;\varkappa]^{K}\,\big{|}\,||\Delta
x||\leq\varkappa\big{\\}}.$

For each $\omega=(r,\Delta
x,\gamma_{1},\dots,\gamma_{K})\in\Omega_{\varepsilon}$, define the admissible
control ${u}_{\omega}\in\mathcal{U}$ as follows: ${u}_{\omega}(t)={u}_{i}$ if
$t$ lies in $(t_{i};t_{i}+\gamma_{i}]$ for some $i\in I_{K}$, and
${u}_{\omega}(t)={\hat{u}}(t)$ otherwise; note that this definition is valid
by the choice of $\varepsilon$. Now, consider the Cauchy problem

$\frac{d{{z}}_{\omega}(t)}{dt}=\bar{f}(t,{z}_{\omega},{u}_{\omega}),\
{z}_{\omega}(0)=(r,{\hat{y}}(0)+\Delta x)$

and its solution on $[0;\infty)$: the motion ${z}_{\omega}\in
C(\mathbb{R}_{+},\bar{\mathbb{X}})$. Finally, define

$\displaystyle
y_{\omega}(t)\stackrel{{\scriptstyle\triangle}}{{=}}\mathrm{y}({\hat{y}}(0)+\Delta
x,0,{u}_{\omega};t),\quad$ $\displaystyle
b_{\omega}\stackrel{{\scriptstyle\triangle}}{{=}}\mathrm{y}(y_{\omega}(T),T,{\hat{u}};0),\
\textrm{ and}$ $\displaystyle
w_{\omega}\stackrel{{\scriptstyle\triangle}}{{=}}r+J({\hat{y}}(0)+\Delta
x,0,u_{\omega};T)-J(b_{\omega},0,{\hat{u}};T)$

for all $\omega=(r,\Delta
x,\gamma_{1},\dots,\gamma_{K})\in\Omega_{\varepsilon}$, $t\geq 0$. Now, we
have the equalities

${z}_{\omega}(\cdot)=(\bar{e}_{0}^{*}z_{\omega}(\cdot),y_{\omega}(\cdot)),\
\mathrm{y}(b_{\omega},0,{\hat{u}};T)=y_{\omega}(T),\
(w_{\omega},b_{\omega})=\mathrm{z}(z_{\omega}(T),T,{\hat{u}};0).$

Since the maps $\omega\mapsto b_{\omega}$ and $\omega\mapsto
z_{\omega}(\cdot)$ are continuous and
$||\frac{dz_{\omega}(t)}{dt}-\bar{f}(t,z_{\omega}(t),{\hat{u}}(t))||$ is
bounded by (48), one finds a positive $\varkappa<\varepsilon/4$ satisfying

$\max_{t\in[0;T]}||y_{\omega}(t)-{\hat{y}}(t)||+\operatorname*{meas}\\{t\geq
0\mid u_{\omega}(t)\neq{\hat{u}}(t)\\}<\beta(T),\\\
u_{\omega}|_{[T;\infty)}={\hat{u}}|_{[T;\infty)},\ \
(\tau,y_{\omega}(\tau))\in{\hat{\mathbb{G}}}_{\partial}\cap{\mathbb{G}}_{u^{1}}\cap\cdots\cap{\mathbb{G}}_{u^{N}},\textrm{\
\ and\ \ }b_{\omega}\in{\hat{\mathbb{G}}}_{\exists}$ (49)

for all $\tau\in[0;T]$, $\omega\in\Omega_{\varkappa}$. Then, by the choice of
$\varkappa$ and $T$, for each such $\omega$, the process
$(u_{\omega},y_{\omega})$ satisfies (3c); and further, $A_{b_{\omega},\theta}$
is well defined for all $\theta$.

Accordingly, it follows from
$[t_{i};t_{i}+\varepsilon)\subset\operatorname*{int}\mathfrak{F}_{N}$ that the
maps

$\displaystyle[t_{i};t_{i}+\gamma_{i}]\ni
t\mapsto\Delta\bar{f}(t,z_{\omega}(t),u_{\omega}(t)),\qquad\quad[t_{i};t_{i}+\gamma_{i}]\ni
t\mapsto\bar{f}(t,z_{\omega}(t),{\hat{u}}(t)),\qquad$
$\displaystyle[t_{i};t_{i}+\gamma_{i}]\ni
t\mapsto\frac{\partial\Delta\bar{f}}{\partial\bar{x}}(t,\bar{x},u_{\omega}(t))\Big{|}_{\bar{x}=z_{\omega}(t)},\quad[t_{i};t_{i}+\gamma_{i}]\ni
t\mapsto\frac{\partial\bar{f}}{\partial\bar{x}}(t,\bar{x},{\hat{u}}(t))\Big{|}_{\bar{x}=z_{\omega}(t)}$

are continuous for all $i\in[1\\!:\\!K]$ and
$\omega=(\bar{\Delta}x,\gamma)\in\Omega_{\varkappa}$. Further, by (48) and
$(H4)$ we also obtain $||\frac{d{{z}}_{\omega}(t)}{dt}||\leq K+{\hat{L}}(t)$
for almost all $t\in[0;T]$. By [53, Lemma 3.1], since all
$t_{k}+\gamma_{k}\in\mathfrak{T}^{\textrm{full}}$ are the Lebesgue points of
(46) if $\omega=(\bar{\Delta}x,\gamma)\in\Omega_{\varkappa}$, the map
$[0;T]\ni
t\mapsto\bar{A}_{\omega}(t)\stackrel{{\scriptstyle\triangle}}{{=}}\frac{\partial\mathrm{z}(\bar{x},t,{u}_{\omega};T)}{\partial\bar{x}}\big{|}_{\bar{x}={z}_{\omega}(t)}$
is well defined and continuous at such an $\omega$.

The calculation of gradients and subdifferentials

Define vectors $\bar{\Delta}\bar{f}_{k}\in\bar{\mathbb{X}}$ as follows:
$\bar{\Delta}\bar{f}_{k}=0$ for all $k\in[1\\!:\\!K]\setminus I_{K}$ and

$\displaystyle\bar{\Delta}\bar{f}_{k}=\bar{f}\big{(}t_{k},\mathrm{z}\big{(}{\hat{\bar{x}}},0,{\hat{u}};t_{k}\big{)},u_{k}\big{)}-\bar{f}\big{(}t_{k},\mathrm{z}\big{(}{\hat{\bar{x}}},0,{\hat{u}};t_{k}\big{)},{\hat{u}}(t_{k})\big{)}\qquad\forall
k\in I_{K}.$

Fix an ${\omega}=(r,\Delta x,\gamma)\in\Omega_{\varkappa}$; define
$\Xi(\omega)\stackrel{{\scriptstyle\triangle}}{{=}}(\Xi_{0}(\omega),\dots,\Xi_{K}(\omega))\in\bar{\mathbb{X}}^{K+1}$
by the rule

$\displaystyle\Xi_{0}(\omega)$
$\displaystyle\stackrel{{\scriptstyle\triangle}}{{=}}{A}^{-1}_{b_{\omega},T}(0)\bar{A}_{\omega}(0),$
$\displaystyle\Xi_{k}(\omega)$
$\displaystyle\stackrel{{\scriptstyle\triangle}}{{=}}{A}^{-1}_{b_{\omega},T}(0)\bar{A}_{\omega}(t_{k}+\gamma_{k})\Delta\bar{f}(t_{k}+\gamma_{k},{z}_{\omega}(t_{k}+\gamma_{k}),u_{k})\qquad\forall
k\in[1\\!:\\!K].$

Recall that $t\mapsto\Delta\bar{f}(t,{z}_{\omega}(t),u_{k})$ are continuous on
$[t_{k};t_{k}+\varepsilon]$. Then, due to the continuity of maps $b_{\omega}$,
${A}^{-1}_{x,t}(0)$, and $\bar{A}_{\omega}(t)$ in $\omega$, it follows that
the map $\Xi$ is also continuous on $\Omega_{\varkappa}$. Further, taking into
account the equalities $(0,b_{0})={\hat{\bar{x}}}={z}_{0}(0)$ and

${A}_{b_{0},T}(t)=\frac{\partial\mathrm{z}(\bar{x},t,{\hat{u}};T)}{\partial\bar{x}}\Big{|}_{\bar{x}=\mathrm{z}(z_{0},0,{\hat{u}};t)}=\frac{\partial\mathrm{z}(\bar{x},t,{u}_{0};T)}{\partial\bar{x}}\Big{|}_{\bar{x}={z}_{0}(t)}=\bar{A}_{0}(t),$

we have
${A}^{-1}_{b_{0},T}(0)\bar{A}_{0}(0)={A}^{-1}_{b_{0},T}(t_{k})\bar{A}_{0}(t_{k})=E$;
now, using the continuity of $\omega\mapsto b_{\omega}$, we obtain

$\displaystyle\Xi(0)$
$\displaystyle\;=\;({A}^{-1}_{b_{0},T}(0){A}_{b_{0},T}(0),{A}^{-1}_{b_{0},T}(0){A}_{b_{0},T}(t_{1})\bar{\Delta}\bar{f}_{1},\dots,{A}^{-1}_{b_{0},T}(0){A}_{b_{0},T}(t_{K})\bar{\Delta}\bar{f}_{K})$
$\displaystyle\stackrel{{\scriptstyle(\ref{tto0})}}{{=}}(E,{A}^{-1}_{b_{0},t_{1}}(0)\bar{\Delta}\bar{f}_{1},\dots,{A}^{-1}_{b_{0},t_{K}}(0)\bar{\Delta}\bar{f}_{K}).$
(50)

Fix a positive $\theta\geq T$ and point
${\omega}=(\bar{\Delta}x,\gamma_{1},\dots,\gamma_{K})\in\Omega_{\varkappa}$.
Recall that all $t_{k}+\gamma_{k}\in\mathfrak{T}^{\textrm{full}}$ are the
Lebesgue points of (46). Therefore, the map
$\Omega_{\varkappa}\ni\omega\mapsto z_{\omega}(T)$ is Fréchet differentiable
by [53, Lemma 3.1]; further, one has

$\displaystyle\frac{\partial z_{\omega}(T)}{\partial\bar{\Delta}x}$
$\displaystyle=\bar{A}_{\omega}(0)={A}_{b_{\omega},T}(0)\Xi_{0}(\omega),$
$\displaystyle\frac{\partial z_{\omega}(T)}{\partial\gamma_{k}}$
$\displaystyle=\bar{A}_{\omega}(t_{k}+{\gamma}_{k})\bar{\Delta}{f}(t_{k}+{\gamma}_{k},{z}_{\omega}(t_{k}+{\gamma}_{k}),u_{k})={A}_{b_{\omega},T}(0)\Xi_{k}(\omega)\qquad\forall
k\in[1\\!:\\!K].$

According to $(w_{\omega},b_{\omega})=\mathrm{z}(z_{\omega}(T),T,{\hat{u}};0)$
and
$\frac{\partial\mathrm{z}(\bar{x},0,{\hat{u}};\theta)}{\partial\bar{x}}\Big{|}_{\bar{x}=(w_{\omega},b_{\omega})}=A_{b_{\omega},\theta}(0)$,
for all $\omega=(\bar{\Delta}x,\gamma)\in\Omega_{\varkappa}$, we have

$\displaystyle\frac{\partial(w_{\omega},b_{\omega})}{\partial\bar{\Delta}x}$
$\displaystyle=\frac{\partial\mathrm{z}(z_{\omega}(T),T,{\hat{u}};0)}{\partial\bar{\Delta}x}=A^{-1}_{b_{\omega},T}(0)\frac{\partial
z_{\omega}(T)}{\partial\bar{\Delta}x}=\Xi_{0}(\omega),$ (51)
$\displaystyle\frac{\partial(w_{\omega},b_{\omega})}{\partial\gamma_{k}}$
$\displaystyle=\frac{\partial\mathrm{z}(z_{\omega}(T),T,{\hat{u}};0)}{\partial\gamma_{k}}=A^{-1}_{b_{\omega},T}(0)\frac{\partial
z_{\omega}(T)}{\partial\gamma_{k}}=\Xi_{k}(\omega)\quad\forall
k\in[1\\!:\\!K].$ (52)

Thus, since all maps $\Xi_{j}$ ($j\in[0\\!:\\!K]$) are continuous on
$\Omega_{\varkappa}$, the maps $b_{\omega}$ and $w_{\omega}$ are
$C^{1}$-smooth in $\omega$. Hence, due to the Whitney extension theorem (see
also [53, Lemma 2.11]), one can suppose that these functions are defined and
$C^{1}$-smooth on $\bar{\mathbb{X}}\times\mathbb{R}^{K}$. Define a
$C^{1}$-smooth function
$S:\bar{\mathbb{X}}\times\bar{\mathbb{X}}\times\mathbb{R}^{K}\to\mathbb{X}$ by
the rule

$\displaystyle
S(\bar{x},\bar{\Delta}x,\gamma)\stackrel{{\scriptstyle\triangle}}{{=}}-\bar{x}+(w_{\bar{\Delta}x,\gamma},b_{\bar{\Delta}x,\gamma})\qquad\qquad\forall(\bar{x},\bar{\Delta}x,\gamma)\in\bar{\mathbb{X}}\times\bar{\mathbb{X}}\times\mathbb{R}^{K}.$

Put
${\hat{\chi}}\stackrel{{\scriptstyle\triangle}}{{=}}({\hat{\bar{x}}},0,0)\in\bar{\mathbb{X}}\times\bar{\mathbb{X}}\times\mathbb{R}^{K}$.
Now, (50), (51), and (52) yield

$\displaystyle\frac{\partial
S(\bar{x},\bar{\Delta}x,\gamma)}{\partial\bar{\Delta}x}\Big{|}_{(\bar{x},\bar{\Delta}x,\gamma)={\hat{\chi}}}$
$\displaystyle=E,\ \frac{\partial
S(\bar{x},\bar{\Delta}x,\gamma)}{\partial\bar{x}}\Big{|}_{(\bar{x},\bar{\Delta}x,\gamma)={\hat{\chi}}}=-E,$
$\displaystyle\frac{\partial