# A converse Lyapunov-type theorem
for control systems with regulated cost

Anna Chiara Lai  Anna Chiara Lai. Dipartimento di Scienze di Base e Applicate
per l’Ingegneria, Sapienza Università di Roma, via Antonio Scarpa,10 - 00161,
Rome, Italy<EMAIL_ADDRESS>and  Monica Motta  Monica Motta.
Dipartimento di Matematica “Tullio Levi-Civita”, Università di Padova, via
Trieste, 63 - 35121, Padua, Italy<EMAIL_ADDRESS>
###### Abstract.

Given a nonlinear control system, a target set, a nonnegative integral cost,
and a continuous function $W$, we say that the system is globally
asymptotically controllable to the target with $W$-regulated cost, whenever,
starting from any point $z$, among the strategies that achieve classical
asymptotic controllability we can select one that also keeps the cost less
than $W(z)$. In this paper, assuming mild regularity hypotheses on the data,
we prove that a necessary and sufficient condition for global asymptotic
controllability with regulated cost is the existence of a special, continuous
Control Lyapunov function, called a Minimum Restraint function. The main
novelty is the necessity implication, obtained here for the first time.
Nevertheless, the sufficiency condition extends previous results based on
semiconcavity of the Minimum Restraint function, while we require mere
continuity.

###### Key words and phrases:

Converse Lyapunov-type theorem , Asymptotic controllability with regulated
cost, Optimal control , Nonlinear theory, Viscosity solutions

###### 2020 Mathematics Subject Classification:

93D30, 93B05, 49J15, 93C10, 49L25

*Corresponding author

This research is partially supported by the INdAM-GNAMPA Project 2023 CUP
E53C22001930001.

##  1\. Introduction

In this paper we extend the classical equivalence result between global
asymptotic controllability to a closed set
$\mathcal{C}\subseteq{\mathbb{R}}^{n}$ and the existence of a Control Lyapunov
function, to the case where the control system is associated with a cost which
has to be regulated (i.e., loosely speaking, kept bounded). In particular, we
consider a nonlinear control system of the form

(1) $\dot{x}(t)=f(x(t),u(t)),\qquad
x(0)=z\in{\mathbb{R}}^{n}\setminus\mathcal{C},\qquad u(t)\in
U\subseteq{\mathbb{R}}^{m},$

and an integral cost

(2) $\int_{0}^{{T}_{z}(u)}l(x(t),u(t))\,dt,$

where the running cost $l$ is nonnegative and the time $T_{z}(u)\leq+\infty$
satisfies111We will consider assumptions under which, given $z$ and $u$, the
corresponding solution to (1) is uniquely determined.

(3) $x(t)\in{\mathbb{R}}^{n}\setminus\mathcal{C}\ \ \text{for all
$t\in[0,T_{z}(u))$},\quad\lim_{t\to{T}^{-}_{z}(u)}\mathbf{d}(x(t))=0$

(for any $y\in{\mathbb{R}}^{n}$, $\mathbf{d}(y)$ denotes the distance of $y$
from $\mathcal{C}$). As customary, the control system (1) is called globally
asymptotically controllable to $\mathcal{C}$ if there exists a
$\mathcal{K}\mathcal{L}$ function $\beta$ such that, for any initial point
$z$, there is a measurable control $u$ whose corresponding solution to (1)
satisfies $\mathbf{d}(x(t))\leq\beta(\mathbf{d}(z),t)$ for all $t\geq 0$. If,
in addition, there exists a continuous, proper, and positive definite function
$W:\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\to[0,+\infty)$, such that

$\int_{0}^{{T}_{z}(u)}l(x(t),u(t))\,dt\leq W(z),$

we say that (1)-(2) is globally asymptotically controllable to $\mathcal{C}$
with $W$-regulated (or simply, regulated) cost. Slightly extending the
original definition in [19], we define as Minimum Restraint function any
function $V:\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\to{\mathbb{R}}$,
which is continuous, positive definite, and proper, and satisfies the decrease
condition

$\min_{u\in U}\Big{\\{}\langle
p\,,\,f(z,u)\rangle+p_{0}(V(z))\,l(z,u)\Big{\\}}\leq-\gamma(V(z))\ \ \text{for
all}\ z\in{\mathbb{R}}^{n}\setminus\mathcal{C},\ p\in\partial_{P}V(z),$

where $\partial_{P}V(z)$ is the proximal subdifferential, for some continuous
increasing functions $p_{0}:(0,+\infty)\to[0,1]$ and
$\gamma:(0,+\infty)\to(0,+\infty)$. Since $p_{0}$ and $l$ are nonnegative,
this decrease condition implies the classical relation which characterizes
Control Lyapunov functions, so that a Minimum Restraint function is actually a
special Control Lyapunov function. The main result of the paper is a ‘Converse
Lyapunov-type theorem’, which consists in proving that the existence of a
continuous Minimum Restraint function for some $p_{0}$ such that $1/p_{0}$ is
integrable at $0^{+}$, is a necessary and sufficient condition for global
asymptotic controllability with regulated cost. In the case without cost
(i.e., for $l\equiv 0$), the equivalence between asymptotic controllability to
a point or to a set of a nonlinear control system and the existence of a
(possibly, non differentiable) Control Lyapunov function, has been a central
topic in control theory since the 1980s and nowadays it is established under
very general assumptions (see e.g. the survey papers [3, 11], the references
therein, and a recent extension to impulsive control systems [15]). The key
idea of many converse Lyapunov theorems is to convert the control system into
a differential inclusion (see, for instance, [5, 22], where
$\mathcal{C}=\\{0\\}$, and [12], for a general target) and then use the result
on the existence of a Lyapunov function for the differential inclusion to get
the promised Control Lyapunov function. However, it seems difficult to adapt
this approach to the case with a cost, which we need to estimate from above.
Also note that the cost cannot be treated as an additional state variable,
because it is increasing and cannot be associated with any target. In this
paper we are inspired instead by early work [24], in which a suitable cost is
associated with the original control system, whose value function turns out to
be a continuous Control Lyapunov function. However, even following this
approach, the generalization of the nonsmooth Lyapunov literature to allow for
the presence of the cost (2) is far from trivial. In particular, we cannot, as
one might think, simply add to the cost considered in [24] the current cost
$l$. In fact, the presence of the cost (2) substantially changes the whole
construction of both the function $\beta$, characterizing asymptotic
controllability, and the cost used to obtain a Control Lyapunov function.

The proof technique is also novel, in that it is based on a combination of the
classical Lyapunov procedures adopted in [24] and a viscosity solutions
approach. Thanks to this, differently from [24], in the proof of necessity we
do not need to use relaxed controls and exhibit an explicit construction of
the $\mathcal{K}\mathcal{L}$ function $\beta$ and of the bound $W$ on the
cost. Specifically, starting from the observation that a continuous, positive
definite and proper function
$V:\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\to{\mathbb{R}}$ solves the
decrease condition if and only if it is a viscosity supersolution of the
Hamilton-Jacobi-Bellman equation

$\max_{u\in
U}\Big{\\{}-Dv(z)\,,\,f(z,u)\rangle-[p_{0}(v(z))\,l(z,u)+\gamma(V(z))]\Big{\\}}=0\quad\
\text{for all}\ z\in{\mathbb{R}}^{n}\setminus\mathcal{C},$

we derive both the facts that the value function built in the proof of the
necessity implication satisfies the decrease condition and the sufficiency
implication, from a viscosity super-optimality principle (Proposition 4.1
below). However, this principle is not included in the known theory, since
$p_{0}$ is merely continuous and we do not assume the usual linear growth
hypothesis on the dynamics function $f$, but $x$-local Lipschitz continuity
only (see [1, Thm. 2.40] or [10, Thm. 3.3]). We prove it by introducing a
slight generalization of a classical comparison principle for infinite horizon
problems (Lemma 4.1 below), interesting in itself. We leave for future
investigation the issue of the existence of a semiconcave Minimum Restraint
function, which, as is well known, plays a key role in the feedback
stabilizability of nonlinear systems, both with cost (see [16, 14, 13], and
[9, 8] for a notion of degree-$k$ Minimum Restraint function) and without (see
e.g. [22, 23, 12, 15], the survey paper [3] and references therein, and [20,
7] for a notion of degree-$k$ Control Lyapunov function). The paper is
organized as follows. In the remaining part of this section we give some
notations. In Section 2 we introduce precisely assumptions and definitions and
state our Converse Lyapunov-type theorem. Section 3 is devoted to prove that
global asymptotic controllability with $W$-regulated cost for some $W$,
implies the existence of a continuous Minimum Restraint function, while the
converse implication is obtained in Section 4.

###  1.1. Notation

For $a,b\in{\mathbb{R}}$, we set $a\vee b:=\max\\{a,b\\}$, $a\land
b:=\min\\{a,b\\}$. Let $\Omega\subseteq{\mathbb{R}}^{N}$ for some integer
$N\geq 1$ be a nonempty set. For every $r\geq 0$, we set
$B_{r}(\Omega):=\\{x\in{\mathbb{R}}^{n}\mid\ d(x,\Omega)\leq r\\}$, where $d$
is the usual Euclidean distance. We use $\overline{\Omega}$, $\partial\Omega$,
and $\mathring{\Omega}$ to denote the closure, the boundary, and the interior
of $\Omega$, respectively. For any interval $I\subseteq{\mathbb{R}}$,
$\mathcal{M}(I,\Omega)$, $AC(I,\Omega)$ are the sets of functions
$x:I\to\Omega$, which are Lebesgue measurable or absolutely continuous,
respectively, on $I$. When no confusion may arise, we simply write
$\mathcal{M}(I)$, $AC(I)$. As customary, we use ${\mathcal{KL}}$ to denote the
set of all continuous functions
$\beta:[0,+\infty)\times[0,+\infty)\to[0,+\infty)$ such that: (1)
$\beta(0,t)=0$ and $\beta(\cdot,t)$ is strictly increasing and unbounded for
each $t\geq 0$; (2) $\beta(r,\cdot)$ is strictly decreasing for each $r\geq
0$; (3) $\beta(r,t)\to 0$ as $t\to+\infty$ for each $r\geq 0$. Given an open
set $\Omega\subseteq{\mathbb{R}}^{N}$, a continuous function
$W:\overline{\Omega}\to[0,+\infty)$ is said positive definite if $W(x)>0$
$\forall x\in\Omega$ and $W(x)=0$ $\forall x\in\partial\Omega$. It is called
proper  if the pre-image $W^{-1}(K)$ of any compact set $K\subset[0,+\infty)$
is compact. Let $x\in\Omega$. The set

$D^{-}W(x):=\left\\{p\in{\mathbb{R}}^{n}\mid\ \liminf_{y\to
x}\frac{W(y)-W(x)-p(y-x)}{|y-x|}\geq 0\right\\},$

is the (possibly empty) viscosity subdifferential of $W$ at $x$. We recall
that $p\in D^{-}W(x)$ if and only if there exists $\varphi\in C^{1}(\Omega)$
such that $D\varphi(x)=p$ and $W-\varphi$ has a local minimum at $x$ (see e.g.
[1]). We use $\partial_{P}W(x)$ to denote the proximal subdifferential of $W$
at $x$ (which may very well be empty). As it is known, $p$ belongs to
$\partial_{P}W(x)$ if and only if there exist $\sigma$ and $\eta>0$ such that

$W(y)-W(x)+\sigma|y-x|^{2}\geq\langle p\,,\,y-x\rangle\qquad\ \text{for all}\
y\in B_{\eta}(\\{x\\}).$

The limiting subdifferential $\partial_{L}W(x)$ of $W$ at $x\in\Omega$, is
defined as

$\displaystyle\partial_{L}W(x):=\Big{\\{}\lim_{i\to+\infty}\,p_{i}\mid\
p_{i}\in\partial_{P}W(x_{i}),\ \lim_{i\to+\infty}x_{i}=x\Big{\\}}.$

The set $\partial_{L}W(x)$ is always closed. If $W$ is locally Lipschitz
continuous on $\Omega$, $\partial_{L}W(x)$ is compact, nonempty at every
point, the set-valued map $x\rightsquigarrow\partial_{L}W(x)$ is upper
semicontinuous, and the Clarke generalized gradient at $x$ coincides with co
$\partial_{L}W(x)$. As sources for nonsmooth analysis we refer e.g. to [2, 6,
27].

Let $W:\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\to[0,+\infty)$ be a
continuous, proper, and positive definite function. We can relate the level
sets of $W$ with the ones of the distance function $\mathbf{d}$, by
introducing the functions $d_{W^{+}}$, $d_{W^{-}}:(0,+\infty)\to(0,+\infty)$,
given by

(4) $\displaystyle d_{W^{-}}(r):=\sup\left\\{\alpha>0\mid\ \ \\{\tilde{z}\mid\
W(\tilde{z})\leq\alpha\\}\subseteq\\{\tilde{z}\mid\ \mathbf{d}(\tilde{z})\leq
r\\}\right\\},$ (5) $\displaystyle d_{W^{+}}(r):=\inf\left\\{\alpha>0\mid\ \
\\{\tilde{z}\mid\ W(\tilde{z})\leq\alpha\\}\supseteq\\{\tilde{z}\mid\
\mathbf{d}(\tilde{z})\leq r\\}\right\\}.$

By [13, Lemma 3.6], these functions are well-defined, increasing, and

(6) $\lim_{r\to 0^{+}}d_{W^{+}}(r)=\lim_{r\to
0^{+}}d_{W^{-}}(r)=0,\quad\lim_{r\to+\infty}d_{W^{+}}(r)=\lim_{r\to+\infty}d_{W^{-}}(r)=+\infty.$

Moreover, one has

(7) $d_{W^{-}}(\mathbf{d}(x))\leq W(x)\leq d_{W^{+}}(\mathbf{d}(x))\quad\
\text{for all}\ x\in{\mathbb{R}}^{n}\setminus\mathcal{C}.$

Approximating $d_{W^{-}}$ from below and $d_{W^{+}}$ from above if necessary,
we can thus assume the existence of continuous, strictly increasing functions,
still denoted $d_{W^{-}}$ and $d_{W^{+}}$, satisfying (6) and (7).

##  2\. A Converse Theorem for Minimum Restraint functions

Throughout the whole paper we assume that:

  * (i)

$U\subset{\mathbb{R}}^{m}$ is a nonempty compact set,
$\mathcal{C}\subset{\mathbb{R}}^{n}$ is a nonempty, closed subset with compact
boundary;

  * (ii)

the functions $f:{\mathbb{R}}^{n}\times U\to{\mathbb{R}}^{n}$,
$l:{\mathbb{R}}^{n}\times U\to[0,+\infty)$ are continuous on
${\mathbb{R}}^{n}\times U$, $x\mapsto f(x,u)$ and $x\mapsto l(x,u)$ are
locally Lipschitz continuous, uniformly with respect to $u\in U$.

Under these assumptions, given $z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$ and
$u\in\mathcal{M}([0,+\infty),U)$, there exist a maximal time
$T^{max}\leq+\infty$ and a unique solution $x\in
AC([0,T^{\max}),{\mathbb{R}}^{n})$ such that $x(0)=z$ and

(8) $\dot{x}(t)=f(x(t),u(t)),\qquad\text{a.e. $t\in[0,T^{\max})$.}$

This solution (or trajectory) will be denoted $x(\cdot\,,u,z)$. If, in
addition, some $c\geq 0$ is given, we define the corresponding cost
$x^{0}(\cdot\,,u,c,z)$ as

(9) $x^{0}(t,u,c,z):=c+\int_{0}^{t}l(x(t,u,z),u(t))\,dt\qquad\ \text{for all}\
t\in[0,T^{\max}).$

Let us preliminarily introduce some definitions.

######  Definition 2.1 (Admissible controls, trajectories, and costs).

Given an initial condition $z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$, a
control $u\in\mathcal{M}([0,+\infty),U)$ is called admissible from $z$ if
there exists $0<T_{z}(u)\leq T^{\max}\leq+\infty$ such that

$x(t):=x(t,u,z)\notin\mathcal{C}\quad\ \text{for all}\
t\in[0,T_{z}(u));\quad\lim_{t\to T^{-}_{z}(u)}\mathbf{d}(x(t))=0\quad\text{if
$T_{z}(u)<+\infty$.}$

The set of admissible controls from $z$ will be denoted $\mathcal{U}(z)$.
Given $z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$, $c\geq 0$, and a control
$u\in\mathcal{U}(z)$, we will extend the corresponding trajectory
$x=x(\cdot\,,u,z)$ and cost $x^{0}:=x^{0}(\cdot\,,u,c,z)$ to $[0+\infty)$, by
setting 222The limit always exists, as $\partial\mathcal{C}$ is compact and
$(x^{0},x)$ is Lipschitz continuous in any compact set
$\overline{B_{R}(\mathcal{C})\setminus\mathcal{C}}$, $R>0$.

$(x^{0},x)(t):=\lim_{t\to T^{-}_{z}(u)}(x^{0},x)(t)\qquad\text{for any $t\geq
T_{z}(u)$.}$

We will call $(x,u)$ and $(x^{0},x,u)$ (both defined on $[0,+\infty)$) an
admissible pair from $z$ and an admissible triple from $(c,z)$, respectively.

We recall the notion of global asymptotic controllability with regulated cost,
first introduced in [19]. In the following, we will refer to any function
$\beta\in{\mathcal{KL}}$ as a descent rate.

######  Definition 2.2 (GAC with regulated cost).

The control system (8) is globally asymptotically controllable – in short, GAC
– to $\mathcal{C}$ if there exists a descent rate $\beta$ such that, for any
initial point $z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$, there is an
admissible pair $(x,u)$ from $z$, satisfying

(10) $\mathbf{d}(x(t))\leq\beta(\mathbf{d}(z),t)\qquad\ \text{for all}\ t\geq
0.$

If, in addition, there exists a continuous, proper, and positive definite
function $W:\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\to[0,+\infty)$,
such that there is an admissible triple $(x^{0},x,u)$ from $(0,z)$ associated
with a pair $(x,u)$ for which (10) is valid, also satisfies

(11) $x^{0}(t)=\int_{0}^{t}l(x(s),u(s))\,ds\leq W(x)\qquad\ \text{for all}\
t\geq 0,$

we say that (8) with the cost (9) is globally asymptotically controllable to
$\mathcal{C}$ with $W$-regulated cost. In this case, we will often simply say
that (8)-(9) is GAC (to $\mathcal{C}$) with regulated cost.

Given a descent rate $\beta$ and a continuous, proper, and positive definite
function $W:\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\to[0,+\infty)$,
for any $z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$ such that
$\mathcal{U}(z)\neq\emptyset$, we set

$\mathcal{U}_{\beta,W}(z):=\left\\{\begin{array}[]{l}u\in\mathcal{U}(z)\mid\text{the
admissible triple $(x^{0},x,u)$ from $(0,z)$ satisfies}\\\\[6.45831pt] \
\qquad\qquad\text{$\mathbf{d}(x(t))<\beta(\mathbf{d}(z),t)$ and $x^{0}(t)\leq
W(z)$ for all $t\geq 0$}\end{array}\right\\}.$

Since in the definition of GAC it is clearly equivalent to replace the
“$\leq$” in (10) with “$<$”, when (8)-(9) meet the properties in Def. 2.2 for
some $\beta$, $W$, we can assume without loss of generality that
$\mathcal{U}_{\beta,W}(z)\neq\emptyset$ for all
$z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$.

Let us now give a slightly extended version of the notion of Minimum Restraint
function for (8)-(9), firstly introduced in [19]. To this end, we consider the
Hamiltonian

(12) $\displaystyle H(x,p_{0},p):=\min_{u\in U}\big{\\{}\langle
p\,,\,f(x,u)\rangle+p_{0}\,l(x,u)\big{\\}}.$

######  Definition 2.3 (MRF).

Let $W:\overline{{\mathbb{R}}^{n}\setminus{\mathcal{C}}}\to[0,+\infty)$ be a
continuous function, which is positive definite and proper. We say that $W$ is
a _Minimum Restraint function – in short, MRF –_ for (8)-(9) if it satisfies
the _decrease condition_ : 333This means that
$H(x,p_{0}(W(x)),p)\leq-\gamma(W(x))$ for every $p\in\partial_{P}W(x)$, where
$\partial_{P}W(x)$ is the proximal differential of $W$ at $x$ (see Subsection
1.1).

(13) $H(x,p_{0}(W(x)),\partial_{P}W(x))\leq-\gamma(W(x))\quad\ \text{for all}\
x\in{{{\mathbb{R}}^{n}}\setminus\mathcal{C}},$

for some continuous, increasing function $p_{0}:(0,+\infty)\to[0,1]$ and some
continuous, strictly increasing function $\gamma:(0,+\infty)\to(0,+\infty)$.

As noted in the introduction, a MRF is a particular Control Lyapunov function,
in which the classical decrease condition

$\min_{u\in U}\langle\partial_{P}W(x)\,,\,f(x,u)\rangle\leq-\gamma(W(x))\quad\
\text{for all}\ x\in{{{\mathbb{R}}^{n}}\setminus\mathcal{C}},$

is replaced by the stronger condition (13), also involving the current cost
$l$.

Finally, we consider the following integrability condition.

######  Definition 2.4.

Let $p_{0}:(0,+\infty)\to[0,1]$ be an increasing, continuous function. We say
that $p_{0}$ satisfies the integrability condition (IC), when $1/p_{0}$ is
integrable at $0^{+}$, namely, we can define the $C^{1}$, strictly increasing
function $P:[0,+\infty)\to[0,+\infty)$, given by

(14) $P(v):=\int_{0}^{v}\frac{dv}{p_{0}(v)}\qquad\ \text{for all}\ v\geq 0,$

and, moreover, $P$ satisfies $\displaystyle\lim_{v\to+\infty}P(v)=+\infty$.

Clearly, if $p_{0}$ is a positive constant, as in the original definition of
MRF, it trivially satisfies condition (IC). We are now ready to state our main
result.

######  Theorem 2.5 (Converse MRF Thm.).

The following properties are equivalent:

  * (i)

system (8) with cost (9) is GAC to $\mathcal{C}$ with regulated cost;

  * (ii)

there exists a continuous MRF for (8)-(9), for some $p_{0}$ and $\gamma$ such
that $p_{0}$ satisfies the integrability condition (IC).

The rest of the paper is devoted to the proof of Theorem 2.5.

######  Remark 2.6.

Starting with the assumption that the system is GAC with regulated cost, in
the proof below we will explicitly build a MRF with $p_{0}\equiv 1$, as unique
continuous viscosity solution of the Hamilton-Jacobi-Bellman equation (in
short, HJB) associated with an exit-time problem with vanishing lagrangian. As
is well known, these HJB equations are highly degenerate and have in general
multiple solutions, for which the continuity on the target does not propagate
to the whole domain (see [21] and [26, 17, 18]). The proof technique thus
consists in showing the continuity of the solution and establishing an ad hoc
comparison principle. In addition to allowing us to extend the main result of
[24] to the case with cost, this technique also provides an alternative
approach to obtaining the classical result.

######  Remark 2.7.

In this paper we continue the study, begun with [19], aimed at constructing a
unified theory, which has as extreme situations asymptotic controllability
(with cost $l\equiv 0$) on the one hand, and the minimum time problem (with
cost $l\equiv 1$) on the other, and for which the $l\geq 0$ case represents,
in a sense, the intermediate stage. In the original notion of MRF in [19],
$p_{0}$ was a positive constant. Extending the definition by considering
$p_{0}$ an increasing function, possibly vanishing at the origin but
satisfying the integrability condition (IC), generalizes the cost bound
obtained in [19], as it implies GAC with $\bar{W}$-regulated cost, where
$\bar{W}(x)=4\,P(W(x)/2)$ for $P$ as in (14) (see estimate (4)
below).444Actually, refining some estimates, we could likely get
$\bar{W}(x)=P(W(x))$, as a consequence of the fact that
$\bar{W}(x)\leq(1+2\varepsilon)(P(W(x)/(1+\varepsilon)))$ for every
$\varepsilon>0$. In the special case of the minimum time problem, this
extension finally provides a result which is entirely consistent with the
existing literature. Indeed, let $l\equiv 1$ and assume that the distance
function $\mathbf{d}$ is a MRF for some functions $p_{0}$ and $\gamma$ such
that (IC) holds true. Then, from the decrease condition (13), it follows that
$\mathbf{d}$ satisfies

(15) $\min_{u\in
U}\langle\partial_{P}\mathbf{d}(x)\,,\,f(x,u)\rangle\leq-\tilde{\gamma}(\mathbf{d}(z))\qquad\
\text{for all}\ z\in{\mathbb{R}}^{n}\setminus\mathcal{C},$

where $\tilde{\gamma}(r):=p_{0}(r)+\gamma(r)$. As it is well-known, this
condition combined with the integrability property (IC) for $\tilde{\gamma}$
–sometimes called weak Petrov condition– guarantees the small time local
controllability of system (8) to $\mathcal{C}$, and implies for the minimum
time function $T$ the estimate
$T(z)\leq\int_{0}^{\mathbf{d}(z)}(1/\tilde{\gamma}(r))\,dr$, in line with our
result (4) (see e.g. [2]). By the expression “weak”, we mean that
$\tilde{\gamma}$ can be 0 at 0, to distinguish it from the classical Petrov
condition, in which $\tilde{\gamma}$ is replaced by a positive constant.
Notice that, considering only $p_{0}\equiv\bar{p}_{0}>0$ constant, we would
have $\tilde{\gamma}\geq\bar{p}_{0}>0$, so our conditions would include just
the ordinary, i.e. non-weak, Petrov condition, which implies local Lipschitz
continuity of $T$ (see e.g. [1]).

Furthermore, we think that considering $p_{0}$ not constant could play a role
in regularizing a MRF, in the fashion of [22].

##  3\. Proof of implication (i) $\Longrightarrow$ (ii)

Suppose that (8)-(9) is GAC to $\mathcal{C}$ with $W$-regulated cost, for a
cost bound $W$ and a descent rate $\beta$. We split the proof into several
lemmas. In particular, in Lemma 3.1, we build new functions $\bar{\beta}$ and
$\bar{W}$ and an admissible triple $(\hat{x}^{0},\hat{x},\hat{u})$ from
$(0,z)$ such that $\hat{u}\in\mathcal{U}_{\bar{\beta},\bar{W}}(z)$, for every
$z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$. These objects play a key role in
the construction of a (larger) cost functional $J$, in Lemma 3.2. Furthermore,
in Lemmas 3.3, 3.4 we show that the value function $V$ associated with the
control system (8) and the new cost $J$, is a continuous MRF. Lemmas 3.1, 3.2
are inspired by [24, Lemmas 3.8, 3.17], where, however, no cost is considered
and relaxed controls are used. As already observed, differently from [24], the
present results are formulated in terms of (explicitly built) new descent rate
and cost bound, and are obtained by mixing nonsmooth analysis and viscosity
methods, which incidentally allow us to disregard relaxed controls.

We begin by introducing some definitions. We consider a bilateral sequence
$(r_{i})_{i\in{\mathbb{Z}}}$, given by 555 As $\beta^{-1}(r,0)$, we mean the
inverse of the strictly increasing function $r^{\prime}\mapsto
r=\beta(r^{\prime},0)$.

(16) $r_{0}:=1,\qquad
r_{i}:=\min\left\\{\beta^{-1}(r_{i-1},0),d^{-1}_{W^{+}}\left(\frac{1}{4}d_{W^{-}}(r_{i-1})\right)\right\\}\quad\
\text{for all}\ i\in{\mathbb{Z}}.$

Clearly, $(r_{i})_{i\in{\mathbb{Z}}}$ is positive, strictly decreasing, so
that $r_{1}<1$, and satisfies

$\displaystyle\lim_{i\to-\infty}r_{i}=+\infty,\qquad\displaystyle\lim_{i\to+\infty}r_{i}=0.$

Hence, we have

$\beta(r_{i},0)\leq r_{i-1},\quad
d_{W^{+}}(r_{i})\leq\frac{1}{4}d_{W^{-}}(r_{i-1})\quad\ \text{for all}\
i\in{\mathbb{Z}}$

and, consequently, recalling that $d_{W^{-}}\leq d_{W^{+}}$ (see (4) and (5)),

(17)
$d_{W^{+}}(r_{i+N})\leq\frac{1}{4^{N}}d_{W^{-}}(r_{i})\leq\frac{1}{4^{N}}d_{W^{+}}(r_{i})\quad\
\text{for all}\ i\in{\mathbb{Z}},\ \ \text{for all}\ N\in{\mathbb{N}},\ N\geq
1.$

For any $i\in{\mathbb{Z}}$, let

(18) $\mathcal{B}_{i}:=\\{z\in{\mathbb{R}}^{n}\setminus\mathcal{C}\mid\
\mathbf{d}(z)\in[r_{i},r_{i-1}]\\},$

so that
$\displaystyle{\mathbb{R}}^{n}\setminus\mathcal{C}=\cup_{i\in{\mathbb{Z}}}\mathcal{B}_{i}$.
Finally, we define the _$i$ -th $(\beta,W)$-strip_ $\mathcal{A}_{i}$, as

$\mathcal{A}_{i}:=\left\\{\begin{array}[]{l}(x^{0},x,u,z)\mid(x^{0},x,u)\
\text{admissible triple from $(0,z)$,}\\\
\qquad\qquad\qquad\qquad\qquad\qquad\qquad u\in\mathcal{U}_{\beta,W}(z),\
z\in\mathcal{B}_{i}\end{array}\right\\}.$

######  Lemma 3.1.

There exist a $\mathcal{K}\mathcal{L}$ function $\bar{\beta}\geq\beta$, a
continuous, unbounded, strictly increasing map
$\Phi:[0,+\infty)\to[0,+\infty)$ with $\Phi(0)=0$, and a function
$T:(0,+\infty)\to[0,+\infty)$ with $T(R)=0$ for all $R\leq r_{1}$,666The value
$r_{1}$ is defined as in (16). such that for any
$z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$ there exists an admissible triple
$(\hat{x}^{0},\hat{x},\hat{u})$ from $(0,z)$ enjoying the following
properties:

  1. (i)

$\mathbf{d}(\hat{x}(t))\leq\bar{\beta}(\mathbf{d}(z),t)$ for all $t\geq 0$;

  2. (ii)

$\hat{x}^{0}(t)\leq\bar{W}(z):=\Phi(\mathbf{d}(z))$ for all $t\geq 0$;

  3. (iii)

$\mathbf{d}(\hat{x}(t))\leq\bar{\beta}(1,t-T(\mathbf{d}(z)))$ for all $t\geq
T(\mathbf{d}(z))$.

###### Proof.

Step 1 (Properties of the $(\beta,W)$-strips). Fix $i\in{\mathbb{Z}}$ and let
$(u^{0},x,u,z)\in\mathcal{A}_{i}$. From the definitions of
$(r_{i})_{i\in{\mathbb{Z}}}$, $d_{W^{-}}$, and $d_{W^{+}}$, it follows that

(19)
$\mathbf{d}(x(t))<\beta(\mathbf{d}(z),t)\leq\beta(r_{i-1},0)<r_{i-2}\qquad\text{for
all }t\geq 0,$

and

(20) $W(z)\leq d_{W^{+}}(r_{i-1})\leq\frac{1}{4}d_{W^{-}}(r_{i-2}).$

Define

$T_{i,z}:=\inf\left\\{t\geq
0\mid\mathbf{d}(x(t))=\frac{r_{i}+r_{i+1}}{2}\right\\}.$

Clearly, $0<T_{i,z}<T_{z}(u)$, where $T_{z}(u)$ is as in Def. 2.1. Set

$\tilde{\varepsilon}_{i,z}:=\inf\left\\{\frac{1}{2}\big{(}\beta(\mathbf{d}(z),t)-\mathbf{d}(x(t))\big{)}\mid
t\in[0,T_{i,z}]\right\\}$

Note that by the continuity of $\beta$ and of $x$, $\tilde{\varepsilon}_{i,z}$
is actually a minimum and $\tilde{\varepsilon}_{i,z}>0$. Also define
$\hat{\varepsilon}_{i}:=\frac{1}{4}d_{W^{+}}(r_{i-1})$,
$\bar{\varepsilon}_{i}:=\frac{r_{i}-r_{i+1}}{4}$ and, finally, set

$\varepsilon_{i,z}:=\min\\{\tilde{\varepsilon}_{i,z},\hat{\varepsilon}_{i},\bar{\varepsilon}_{i}\\}.$

Since by assumption $f(\cdot,u)$ and $l(\cdot,u)$ are locally Lipschitz
continuous, uniformly with respect to $u\in U$, then there exists
$\delta_{i,z}>0$ such that, for all
$\bar{z}\in{\mathbb{R}}^{n}\setminus\mathcal{C}$ satisfying
$|\bar{z}-z|<\delta_{i,z}$, the cost $x^{0}(\cdot\,,u,0,\bar{z})$ and the
trajectory $x(\cdot\,,u,\bar{z})$ are defined on $[0,T_{i,z}]$ and satisfy

$|x(t)-x(t,u,\bar{z})|\leq\varepsilon_{i,z},\quad|x^{0}(t)-x^{0}(t,u,0,\bar{z})|\leq\varepsilon_{i,z}\qquad\text{for
all }t\in[0,T_{i,z}].$

Hence, for all $t\in[0,T_{i,z}]$, using (20), we get

$\displaystyle x^{0}(t,u,0,\bar{z})$ $\displaystyle\leq
x^{0}(t)+\varepsilon_{i,z}\leq W(z)+\hat{\varepsilon}_{i}\leq
W(z)+\frac{1}{4}d_{W^{+}}(r_{i-1})$
$\displaystyle\leq\frac{1}{4}d_{W^{-}}(r_{i-2})+\frac{1}{4}d_{W^{+}}(r_{i-1})\leq\frac{1}{2}d_{W^{+}}(r_{i-2}).$

Moreover, in view of the definition of $\varepsilon_{i,z}$, we also have

$\displaystyle\mathbf{d}(x(t,u,\bar{z}))$
$\displaystyle\leq\mathbf{d}(x(t))+\frac{1}{2}\big{(}\beta(\mathbf{d}(z),t)-\mathbf{d}(x(t))\big{)}=\frac{1}{2}\beta(\mathbf{d}(z),t)+\frac{1}{2}\mathbf{d}(x(t))$
$\displaystyle<\beta(\mathbf{d}(z),t)\leq\beta(r_{i-1},0)\leq r_{i-2},$

whereas the definition of $T_{i,z}$ implies

$\displaystyle\mathbf{d}(x(t,u,\bar{z}))$
$\displaystyle\geq\mathbf{d}(x(t))-\bar{\varepsilon}_{i}\geq\mathbf{d}(x(T_{i,z}))-\bar{\varepsilon}_{i}$
$\displaystyle\geq\frac{r_{i}+r_{i+1}}{2}-\frac{r_{i}-r_{i+1}}{4}=\frac{r_{i}}{4}+\frac{3}{4}r_{i+1}>r_{i+1},$

and

$\displaystyle\mathbf{d}(x(T_{i,z},u,\bar{z}))$
$\displaystyle\leq\mathbf{d}(x(T_{i,z}))+\bar{\varepsilon}_{i}=\frac{r_{i}+r_{i+1}}{2}+\frac{r_{i}-r_{i+1}}{4}$
$\displaystyle=\frac{3}{4}r_{i}+\frac{1}{4}r_{i+1}<r_{i}.$

Summarizing the above results, we can conclude that, for every
$i\in{\mathbb{Z}}$ and $z\in\mathcal{B}_{i}$ ($\mathcal{B}_{i}$ as in (18)),
with which we can consider an element $(x^{0},x,u,z)\in\mathcal{A}_{i}$ to be
associated, by the axiom of choice, there exists $\delta_{i,z}>0$ such that,
for all $\bar{z}\in{\mathbb{R}}^{n}\setminus\mathcal{C}$ with
$|z-\bar{z}|<\delta_{i,z}$, one has

  1. (a)

$\mathbf{d}(x(t,u,\bar{z}))\in(r_{i+1},r_{i-2})$ for all $t\in[0,T_{i,z}]$;

  2. (b)

$\mathbf{d}(x(T_{i,z},u,\bar{z}))\in(r_{i+1},r_{i})$, i.e.
$x(T_{i,z},u,\bar{z})\in\mathring{\mathcal{B}}_{i+1}$;

  3. (c)

$x^{0}(T_{i,z},u,\bar{z})\leq\frac{1}{2}d_{W^{+}}(r_{i-2})$.

Step 2 (Construction of a suitable admissible triple) Preliminarily, observe
that, since $\partial\mathcal{C}$ is compact, for every $i\in{\mathbb{Z}}$ the
set $\mathcal{B}_{i}$ is compact. Therefore, the cover of $\mathcal{B}_{i}$
given by the open balls $\mathring{B}_{\delta_{i,z}}(\\{z\\})$,
$z\in\mathcal{B}_{i}$, admits a finite subcover corresponding to the points
$z\in Z_{i}$, for some finite subset $Z_{i}$ of $\mathcal{B}_{i}$. Fix a
positive bilateral sequence $(T_{i})_{i\in{\mathbb{Z}}}$ such that

(21) $T_{i}\geq\max\\{T_{i,z}\mid z\in
Z_{i}\\},\quad\qquad\sum_{j=0}^{\infty}T_{i+j}=+\infty,\quad\text{for every
$i\in{\mathbb{Z}}$.}$

Furthermore, thanks to the properties of the bilateral sequence
$(r_{i})_{i\in{\mathbb{Z}}}$, we can define the map
$i:(0,+\infty)\to{\mathbb{Z}}$, given by

(22) $i(r)=i\qquad\text{if $r\in(r_{i},r_{i-1}]$.}$

Fix now $\bar{z}\in{\mathbb{R}}^{n}\setminus\mathcal{C}$ and let
$i:=i(\mathbf{d}(\bar{z}))$. Then,
$\bar{z}\in\mathring{B}_{\delta_{i,z_{{}_{0}}}}(\\{z_{{}_{0}}\\})$ for some
$z_{{}_{0}}\in Z_{i}\subset\mathcal{B}_{i}$. Let
$(x^{0}_{{}_{0}},x_{{}_{0}},u_{{}_{0}},z_{{}_{0}})\in\mathcal{A}_{i}$ be the
associated process from $(0,z_{{}_{0}})$. Since
$|\bar{z}-z_{{}_{0}}|<\delta_{i,z_{{}_{0}}}$, from Step 1 it follows that
$\hat{x}^{0}:=(\cdot\,,u_{{}_{0}},0,\bar{z})$ and
$\hat{x}:=x(\cdot\,,u_{{}_{0}},\bar{z})$ are defined on the interval
$[0,\hat{t}_{{}_{0}}]$, $\hat{t}_{{}_{0}}:=T_{i,z_{{}_{0}}}\leq T_{i}$, and
satisfy

  1. (a.0)

$\mathbf{d}(\hat{x}(t))\in(r_{i+1},r_{i-2})$ for all
$t\in[0,\hat{t}_{{}_{0}}]$;

  2. (b.0)

$\hat{x}(\hat{t}_{{}_{0}})\in\mathring{\mathcal{B}}_{i+1}$;

  3. (c.0)

$\hat{x}^{0}(\hat{t}_{{}_{0}})\leq\frac{1}{2}d_{W^{+}}(r_{i-2})$.

Repeating the above procedure with the initial conditions
$\bar{z}_{1}:=\hat{x}(\hat{t}_{0})\in\mathcal{B}_{i+1}$ and
$\bar{c}_{1}:=\hat{x}_{0}(\hat{t}_{0})$, we get the existence of an admissible
control $u_{1}\in\mathcal{U}_{\beta,W}(\bar{z}_{1})$ and of a time
$\hat{t}_{1}\leq T_{i+1}$, such that extending $\hat{x}^{0}$ and $\hat{x}$ by
setting
$\hat{x}^{0}(t)=x^{0}(t-\hat{t}_{{}_{0}},u_{1},\bar{c}_{1},\bar{z}_{1})$ and
$\hat{x}(t)=x(t-\hat{t}_{{}_{0}},u_{1},\bar{z}_{1})$ for
$t\in[\hat{t}_{{}_{0}},\hat{t}_{{}_{0}}+\hat{t}_{1}]$, respectively, one has

  1. (a.1)

$\mathbf{d}(\hat{x}(t))\in(r_{i+2},r_{i-1})$ for all
$t\in[\hat{t}_{{}_{0}},\hat{t}_{{}_{0}}+\hat{t}_{1}]$;

  2. (b.1)

$\hat{x}(\hat{t}_{{}_{0}}+\hat{t}_{1})\in\mathring{\mathcal{B}}_{i+2}$;

  3. (c.1)

$\hat{x}^{0}(\hat{t}_{{}_{0}}+\hat{t}_{1})\leq\frac{1}{2}d_{W^{+}}(r_{i-2})+\frac{1}{2}d_{W^{+}}(r_{i-1})\leq\frac{1}{2}d_{W^{+}}(r_{i-2})+\frac{1}{8}d_{W^{+}}(r_{i-2}).$

Of course, setting
$\hat{u}(t):=u_{{}_{0}}(t)\chi_{{}_{[0,\hat{t}_{{}_{0}}]}}(t)+u_{1}(t-\hat{t}_{{}_{0}})\chi_{{}_{(\hat{t}_{{}_{0}},\hat{t}_{{}_{0}}+\hat{t}_{1}]}}(t)$,
we have $\hat{x}^{0}=x(\cdot\,,\hat{u},0,\bar{z})$ and
$\hat{x}=x(\cdot\,,\hat{u},\bar{z})$. In this way, we can recursively
construct sequences of controls $(u_{N})_{N\in{\mathbb{N}}}$ and times
$(\hat{t}_{N})_{N\in{\mathbb{N}}}$ with $\hat{t}_{N}\leq T_{i+N}$ for all $N$,
such that, setting

$\hat{T}_{-1}:=0,\quad\hat{T}_{N}:=\sum_{j=0}^{N}\hat{t}_{j},\quad\hat{T}_{\infty}:=\sum_{j=0}^{+\infty}\hat{t}_{j}$

and

$\hat{u}(t):=u_{N}(t-\hat{T}_{N-1})\qquad\text{for all
}t\in(\hat{T}_{N-1},\hat{T}_{N}]$

we have $u\in\mathcal{M}([0,\hat{T}_{\infty}),U)$ and the corresponding
solution $(\hat{x}^{0},\hat{x})$ from $(0,\bar{z})$ is defined on the whole
interval $[0,\hat{T}_{\infty})$ and satisfies

  1. (a.N)

$\mathbf{d}(\hat{x}(t))\in(r_{i+N+1},r_{i+N-2})$ for all
$t\in[\hat{T}_{N-1},\hat{T}_{N}]$;

  2. (b.N)

$\hat{x}(\hat{T}_{N})\in\mathring{\mathcal{B}}_{i+N+1}$;

  3. (c.N)

$\hat{x}^{0}(\hat{T}_{N})\leq\hat{x}_{0}(\hat{T}_{N-1})+\frac{1}{2}d_{W^{+}}(r_{i+N-2})\leq
Cd_{W^{+}}(r_{i-2}),$ for $C:=\frac{1}{2}\sum_{j=0}^{\infty}\frac{1}{4^{j}}$.

(The latter inequality can be easily proved by induction.) From these
relations, it follows immediately that $\hat{T}_{\infty}=T_{z}(\hat{u})$,
$\lim_{t\to\hat{T}_{\infty}^{-}}\mathbf{d}(\hat{x}(t))=0$, and
$\hat{x}^{0}(\hat{T}_{\infty})\leq Cd_{W^{+}}(r_{i-2})$ (actually, even if
$\hat{T}_{\infty}=+\infty$). Hence, if in case $\hat{T}_{\infty}<+\infty$ we
extend the process $(\hat{x}^{0},\hat{x},\hat{u})$ to $[0,+\infty)$ by setting
$\hat{u}(t)=w$ ($w\in U$ arbitrary), and
$(\hat{x}^{0},\hat{x})(t)=\lim_{t\to\hat{T}_{\infty}^{-}}(\hat{x}^{0},\hat{x})(t)$
for any $t\geq\hat{T}_{\infty}$, we can conclude that
$(\hat{x}^{0},\hat{x},\hat{u})$ is an admissible triple from $(0,\bar{z})$.

Step 3 (Construction of $\bar{\beta}$, $\Phi$, and $T$). For all
$i\in{\mathbb{Z}}$ and $N\in{\mathbb{N}}$, set

$\bar{T}_{i,-1}:=0,\qquad\bar{T}_{i,N}:=\sum_{j=0}^{N}T_{i+j},$

where the times $T_{i}$ are as in (21), so that for any $i\in{\mathbb{Z}}$,
$\bar{T}_{i,N}\to+\infty$ as $N\to+\infty$. Then, we define the function
$T:(0,+\infty)\to[0+\infty)$, as

$T(R):=\bar{T}_{i(R),-1\vee(-i(R)+1)}\qquad\ \text{for all}\ R>0,$

where $i(\cdot)$ is as in (22). Note that if $R\leq r_{1}<1$ then $i(R)\geq 2$
and this implies $T(R)=\bar{T}_{i(R),-1}=0$. Since $d_{W}^{+}$ as well as
$R\mapsto r_{i(R)}$ are increasing functions, the composition
$R\mapsto\,d_{W}^{+}(r_{i(R)-2})$ is a positive, piecewise constant,
increasing function, such that $d_{W}^{+}(r_{i(R)-2})\to 0$ as $R\to 0^{+}$
and $d_{W}^{+}(r_{i(R)-2})\to+\infty$ as $R\to+\infty$. There is thus a
continuous, strictly increasing approximation from above
$\Phi:[0,+\infty)\to[0,+\infty)$ of this composition times $C$ ($C$ as in
(c.N) above), vanishing at zero and unbounded, namely

$C\,d_{W}^{+}(r_{i(R)-2})\leq\Phi(R)\qquad\ \text{for all}\ R>0.$

Finally, we introduce the function
$b:[0,+\infty)\times[0,+\infty)\to[0,+\infty)$, given by

$\begin{cases}b(R,t):=r_{i+N-2}&\text{if
}(R,t)\in{(r_{i},r_{i-1}]}\times[\bar{T}_{i,N-1},\bar{T}_{i,N})\quad\
\text{for all}\ i\in{\mathbb{Z}},\ N\in{\mathbb{N}}\\\ b(0,t):=0&\ \text{for
all}\ t\geq 0.\end{cases}$

Note that $b(\cdot,t)$ is increasing and $b(R,t)\to+\infty$ as $R\to+\infty$,
for all $t\geq 0$. Similarly, $b(R,\cdot)$ is positive, decreasing and
$b(R,t)\to 0$ as $t\to+\infty$ for all $R>0$. Using e.g. a linear
interpolation procedure, it is not difficult to show that the discontinuous
function $b$ can be approximated from above by some $\mathcal{KL}$ function
$\bar{\beta}$, which is $\geq\beta$ by construction.

Let $\bar{z}\in{\mathbb{R}}^{n}\setminus\mathcal{C}$ and set
$i:=i(\mathbf{d}(\bar{z}))$. Then, since $\hat{T}_{N}\leq\bar{T}_{i,N}$ for
all $N\geq-1$, the admissible triple $(\hat{x}^{0},\hat{x},\hat{u})$ from
$(0,\bar{z})$ built in Step 2, satisfies

(23)
$\mathbf{d}(\hat{x}(t))<r_{i+N-2}\cdot\chi_{{}_{[\bar{T}_{i,N-1},\bar{T}_{i,N})}}(t)\leq
b(\mathbf{d}(\bar{z}),t)\leq\bar{\beta}(\mathbf{d}(\bar{z}),t)\qquad\
\text{for all}\ t\geq 0,$

and

$\hat{x}^{0}(t)\leq\bar{W}(\bar{z}):=\Phi(\mathbf{d}(\bar{z}))\qquad\
\text{for all}\ t\geq 0,$

so that $\hat{u}\in\mathcal{U}_{\bar{\beta},\bar{W}}(\bar{z})$. Notice that
$\bar{W}:\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\to[0,+\infty)$ is a
continuous, proper and positive definite function, in view of the properties
of $\Phi$ and $\mathbf{d}$. This concludes the proof of statements (i) and
(ii).

In order to prove (iii), let us first suppose $\mathbf{d}(\bar{z})\leq
r_{1}<r_{0}=1$. In this case, $i(\mathbf{d}(\bar{z}))\geq 2$ and
$T(\mathbf{d}(\bar{z}))=\bar{T}_{i(\mathbf{d}(z)),-1}=0$. Hence (iii) follows
from (23), because $\bar{\beta}(\cdot,t)$ is strictly increasing for every
$t\geq 0$, so that

$\mathbf{d}(\hat{x}(t))<\bar{\beta}(\mathbf{d}(\bar{z}),t)\leq\bar{\beta}(1,t)=\bar{\beta}(1,t-T(\mathbf{d}(\bar{z})))\quad\text{
for all $t\geq T(\mathbf{d}(\bar{z})=0$.}$

If instead $\mathbf{d}(\bar{z})>r_{1}$, we have $i:=i(\mathbf{d}(\bar{z}))\leq
1$. Set $N:=-i(\mathbf{d}(z))+2$ ($\geq 1$) and
$\bar{z}_{N}:=\hat{x}(\hat{T}_{N-1})$, so that, by property (b.N) of Step 2,
$r_{2}<\mathbf{d}(\bar{z}_{N})<r_{1}<r_{0}=1$. Note that, applying the above
construction from the initial condition $(0,\bar{z}_{N})$, the obtained
admissible triple, say $(\hat{x}^{0}_{N},\hat{x}_{N},\hat{u}_{N})$ from
$(0,\bar{z}_{N})$, satisfying

$\mathbf{d}(\hat{x}_{N}(s))<\bar{\beta}(\mathbf{d}(\bar{z}_{N}),s),\quad\hat{x}^{0}_{N}(s)\leq\Phi(\mathbf{d}(\bar{z}_{N}))\quad\
\text{for all}\ s\geq 0,$

is simply given by

$\hat{u}_{N}(\cdot)=\hat{u}(\cdot\,+\hat{T}_{N-1}),\quad\hat{x}^{0}_{N}=x^{0}(\cdot\,,\hat{u}_{N},0,\bar{z}_{N}),\quad\hat{x}_{N}=x(\cdot\,,\hat{u}_{N},\bar{z}_{N}).$

This fact is crucial for property (iii) to hold. Indeed, using the
monotonicities of $\bar{\beta}$ and the inequality
$\hat{T}_{N-1}\leq\bar{T}_{i,N-1}=T(\mathbf{d}(\bar{z}))$, it implies that

$\displaystyle\mathbf{d}(\hat{x}(t))$
$\displaystyle=\mathbf{d}(\hat{x}_{N}(t-\hat{T}_{N-1}))<\beta(\mathbf{d}(\bar{z}_{N}),t-\hat{T}_{N-1})$
$\displaystyle<\bar{\beta}(1,t-\hat{T}_{N-1})\leq\bar{\beta}(1,t-T(\mathbf{d}(\bar{z})))\qquad\text{
for all $t\geq T(\mathbf{d}(\bar{z})).$}$

∎

######  Lemma 3.2.

There exist two continuous, strictly increasing functions $\ell$,
$\Psi:[0,+\infty)\to[0,+\infty)$, with $\ell(0)=0$,
$\displaystyle\lim_{R\to+\infty}\ell(R)=+\infty$, such that the functional
$J:({\mathbb{R}}^{n}\setminus\mathcal{C})\times\mathcal{M}([0,+\infty),U)\to[0,+\infty)\cup\\{+\infty\\}$,
defined as

$J(z,u):=\left\\{\begin{array}[]{l}\displaystyle\int_{0}^{T_{z}(u)}\left[\ell(\mathbf{d}(x(t,z,u)))+l(x(t,z,u),u(t))\right]dt\qquad\qquad\text{if
$u\in\mathcal{U}(z)$,}\\\
+\infty\qquad\qquad\qquad\qquad\qquad\qquad\qquad\text{if
$u\in\mathcal{M}([0,+\infty),U)\setminus\mathcal{U}(z)$,}\end{array}\right.$

enjoys the properties (i)–(v) below.

For every $z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$, let
$(\hat{x}^{0},\hat{x},\hat{u})$ be the admissible triple from $(0,z)$ built in
Lemma 3.1. Using the notations of Lemma 3.1, we have

  1. (i)

$J(z,\hat{u})<+\infty$;

  2. (ii)

if $\mathbf{d}(z)<r_{1}$, then
$J(z,\hat{u})\leq\bar{\beta}(\mathbf{d}(z),0)+\Phi(\mathbf{d}(z))$;

  3. (iii)

for any $u\in\mathcal{M}([0,+\infty),U)$ such that $J(z,u)\leq J(z,\hat{u})$,
($u\in\mathcal{U}(z)$ and) $\mathbf{d}(x(t,u,z))\leq\Psi(\mathbf{d}(z))$ for
all $t\geq 0$;

  4. (iv)

for all $\alpha>0$ there exists $\Theta>0$ such that for any
$u\in\mathcal{M}([0,+\infty),U)$ with $J(z,u)<\alpha$, then
$\mathbf{d}(z)\leq\Theta$;

  5. (v)

for all $\alpha>0$ there exists $\delta>0$ such that, if
$\mathbf{d}(z)>\alpha$, then $J(z,u)>\delta$ for all
$u\in\mathcal{M}([0,+\infty),U)$.

###### Proof.

Let $\bar{\beta}$ be a $\mathcal{KL}$ function as in Lemma 3.1. Extend the
continuous strictly decreasing function $[0,+\infty)\ni
t\mapsto\bar{\beta}(1,t)$ to a continuous strictly decreasing function defined
on ${\mathbb{R}}$ and tending to $+\infty$ as $t\to-\infty$. Let
$\tau:(0,+\infty)\to{\mathbb{R}}$ be the inverse of this map, so that, in
particular, $\tau$ is continuous, strictly decreasing, $\tau(R)\to+\infty$ as
$R\to 0^{+}$ and $\tau(R)\to-\infty$ as $R\to+\infty$.

_Step 1. (Construction of a function $\ell_{1}$)_ Define the continuous,
strictly increasing, and unbounded function
$\ell_{1}:[0,+\infty)\to[0,+\infty)$, given by

$\ell_{1}(R):=Re^{-\tau(R)}\quad\text{for all $R>0$,}\qquad\ell_{1}(0):=0.$

For any $0<b<c$, we claim that there exists a function $\varkappa(b,c)$ such
that, if $z_{1},z_{2}\in{\mathbb{R}}^{n}\setminus\mathcal{C}$,
$u\in\mathcal{M}([0,+\infty),U)$, and $T>0$ satisfy $z_{2}=x(T,u,z_{1})$ and
$b\leq\mathbf{d}(x(t,u,z_{1}))\leq c$ for all $t\in[0,T]$, then

(24)
$\int_{0}^{T}\ell_{1}(\mathbf{d}(x(t,u,z_{1})))dt\geq\varkappa(b,c)|z_{1}-z_{2}|\geq{\varkappa(b,c)|\mathbf{d}(z_{1})-\mathbf{d}(z_{2})|.}$

Indeeed, if $z_{1}=z_{2}$, (24) is trivially satisfied. If instead $z_{1}\neq
z_{2}$, then $\bar{M}(b,c):=\max\\{|f(x,u)|\mid b\leq\mathbf{d}(x)\leq
c,\,u\in U\\}>0$ and $|z_{1}-z_{2}|{\leq}T\bar{M}(b,c)$. Setting
$\varkappa(b,c):=\ell_{1}(b)\bar{M}(b,c)^{-1}$, since $\ell_{1}$ is
increasing, we have $\ell_{1}(\mathbf{d}(x(t,u,z_{1})))\geq\ell_{1}(b)$ for
all $t\in[0,T]$, and consequently

$\int_{0}^{T}\ell_{1}(\mathbf{d}(x(t,u,z_{1})))dt\geq
T\ell_{1}(b)\geq\varkappa(b,c){|z_{1}-z_{2}|}.$

_Step 2 (Recursive definition of a sequence $\ell_{j}$)_ Starting from
$\ell_{1}$ introduced in Step 1, for every $j\geq 1$ we will recursively
define an increasing sequence of functions
$\ell_{j}:[0,+\infty)\to{\mathbb{R}}$ with the following property:

(25) $0\leq
R\leq\bar{\beta}(j,0)\quad\Longrightarrow\quad\ell_{j+1}(R)=\ell_{j}(R).$

Then, we will prove that the pointwise limit
$\displaystyle\ell:=\lim_{j\to+\infty}\ell_{j}$ (finite, because of (25))
together with the map $\Psi$ defined below, satisfies (i)–(v). Further, all
the functions $\ell_{j}$, (hence $\ell$ itself) will be continuous. We begin
by assuming that, for some fixed $j\geq 2$, we already defined the functions
$\ell_{i}$ for all $i\leq j$, satisfying (25) for all $i=1,\dots,j-1$. Given
$z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$ and any control
$u\in\mathcal{U}(z)$, let us set

$J_{j}(z,u):=\int_{0}^{T_{z}(u)}\ell_{j}(\mathbf{d}(x(t,u,z)))+l(x(t,u,z),u(t))dt.$

Observe that, if we consider the admissible triple $(x^{0},x,u)$ from $(0,z)$
defined on $[0,+\infty)$ (even in case $T_{z}(u)<+\infty$, as in Def. 2.1), we
can equivalently write

$J_{j}(z,u)=\int_{0}^{+\infty}\ell_{j}(\mathbf{d}(x(t)))+\lim_{t\to
T^{-}_{z}(u)}x^{0}(t),$

as $\mathbf{d}(x(t))=0$ for all $t\geq T_{z}(u)$, whenever $T_{z}(u)<+\infty$.
Assume $\mathbf{d}(z)\leq j$. Let $T(\cdot)$ and
$(\hat{x}^{0},\hat{x},\hat{u})$ be the uniform time and the admissible triple
from $(0,z)$ built in Lemma 3.1, respectively. Then, we have

$\mathbf{d}(\hat{x}(t))\leq\bar{\beta}(1,t-T(j))\leq\bar{\beta}(1,0)\qquad\text{for
all $t\geq T(j)$.}$

In particular, (25) (together with the fact that $\bar{\beta}(\cdot,0)$ is
increasing) implies

$\ell_{j}(\mathbf{d}(\hat{x}(t)))=\ell_{1}(\mathbf{d}(\hat{x}(t)))\qquad\
\text{for all}\ t\geq T(j).$

If otherwise $t\leq T(j)$, we get

$\mathbf{d}(\hat{x}(t))<\bar{\beta}(\mathbf{d}(z),0)\leq\bar{\beta}(j,0).$

Hence, we have

$\displaystyle\int_{0}^{+\infty}\ell_{j}$
$\displaystyle(\mathbf{d}(\hat{x}(t)))dt=\int_{0}^{T(j)}\ell_{j}(\mathbf{d}((\hat{x}(t))))+\int_{T(j)}^{+\infty}\ell_{j}(\mathbf{d}(\hat{x}(t)))dt$
$\displaystyle\leq\ell_{j}(\bar{\beta}(j,0))T(j)+\int_{T(j)}^{+\infty}\ell_{1}(\mathbf{d}(\hat{x}(t)))dt$
$\displaystyle=\ell_{j}(\bar{\beta}(j,0))T(j)+\int_{T(j)}^{+\infty}\mathbf{d}(\hat{x}(t))e^{-t(\mathbf{d}(\hat{x}(t)))}dt$
$\displaystyle\leq\ell_{j}(\bar{\beta}(j,0))T(j)+\int_{T(j)}^{+\infty}\bar{\beta}(1,(t-T(j)))\,e^{-\tau(\bar{\beta}(1,(t-T(j)))}dt$
$\displaystyle\leq\ell_{j}(\bar{\beta}(j,0))T(j)+\bar{\beta}(1,0)\int_{T(j)}^{+\infty}e^{-t+T(j)}dt$
$\displaystyle=\ell_{j}(\bar{\beta}(j,0))T(j)+\bar{\beta}(1,0):=L_{j}.$

Therefore, from Lemma 3.1, (ii) it follows that

(26) $J_{j}(z,\hat{u})\leq L_{j}+\Phi(j)\quad\text{for all
}z\in{\mathbb{R}}^{n}\setminus\mathcal{C}\text{ such that }\mathbf{d}(z)\leq
j.$

Now, set $\varkappa_{j}:=\varkappa(\bar{\beta}(j,0)+1,\bar{\beta}(j,0)+2)$ and
consider a continuous function $\rho_{j}:[0,+\infty)\to[0,+\infty)$ such that

$\rho_{j}(R)=\begin{cases}0&\text{ if $R\leq\bar{\beta}(j,0)$ or
$R\geq\bar{\beta}(j,0)+3$},\\\ \dfrac{L_{j}+\Phi(j)}{\varkappa_{j}}&\text{ if
$\bar{\beta}(j,0)+1\leq R\leq\bar{\beta}(j,0)+2$.}\end{cases}$

Finally, define

$\ell_{j+1}(R):=(1+\rho_{j}(R))\ell_{j}(R)\qquad\text{for all $R\geq 0$.}$

Clearly, $\ell_{j+1}$ satisfies (25).

_Step 3. (Definition of $\ell$ and $\Psi$)_ As anticipated above, we define
$\ell$ as the pointwise limit of the increasing sequence $(\ell_{j})$.
Incidentally notice that, by construction, $\ell$ is continuous and
$\ell(R)\geq\ell_{j}(R)$ for all $R\in[0,+\infty)$ and all $j\geq 1$.
Furthermore, let $\Psi:[0,+\infty)\to[0,+\infty)$ be any continuous strictly
increasing function, satisfying

${R\in[j-1,j]}\quad\Rightarrow\quad\Psi(R)\geq\bar{\beta}(j,0)+2\quad\
\text{for all}\ j\geq 1.$

We show that $\ell$ and $\Psi$ have the required properties. Let
$z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$ be given.

In order to prove (i), it suffices to observe that there exists an integer
$j\geq 1$ such that $j-1<\mathbf{d}(z)\leq j$. Then, (26) implies that

(27) $J(z,\hat{u})=J_{j}(z,\hat{u})\leq L_{j}+\Phi(j)<+\infty.$

Let us now prove (ii). Assume $\mathbf{d}(z)\leq r_{1}<1$. Then,
$T(\mathbf{d}(z))=0$,
$\ell(\mathbf{d}(\hat{x}(t)))=\ell_{1}(\mathbf{d}(\hat{x}(t)))$ for all $t\geq
0$, and, arguing as in Step 2, we obtain

$\displaystyle J(z,\hat{u})$
$\displaystyle=\int_{0}^{+\infty}\left[\ell_{1}(\mathbf{d}(\hat{x}(t))+l(\hat{x}(t),\hat{u}(t))\right]dt\leq\int_{0}^{+\infty}\bar{\beta}(\mathbf{d}(z),t)\,e^{-t}\,dt+\Phi(\mathbf{d}(z))$
$\displaystyle\leq\bar{\beta}(\mathbf{d}(z),0)+\Phi(\mathbf{d}(z))\qquad\text{for
all $t\geq 0$.}$

To prove (iii), consider $u\in\mathcal{M}([0,+\infty),U)$ satisfying
$J(z,u)\leq J(z,\hat{u})$. Actually, $u\in\mathcal{U}(z)$, so let
$(x^{0},x,u)$ be the corresponding admissible triple from $(0,z)$. As above,
let $j$ be the integer $\geq 1$ such that $j-1<\mathbf{d}(z)\leq j$. By the
properties of $\bar{\beta}$, we have $\bar{\beta}(j,0)>j$, so that
$\mathbf{d}(z)<\bar{\beta}(j,0)$. In contradiction to claim (iii), suppose
that there exists $t>0$ such that

$\mathbf{d}(x(t))>\Psi(\mathbf{d}(z))>\bar{\beta}(j,0)+2.$

Then, there exist $0<t_{1}<t_{2}<T_{z}(u)$ such that

(28)
$\begin{array}[]{l}\mathbf{d}(x(t_{1}))=\bar{\beta}(j,0)+1,\qquad\mathbf{d}(x(t_{2}))=\bar{\beta}(j,0)+2,\\\\[6.45831pt]
\bar{\beta}(j,0)+1\leq\mathbf{d}(x(t))\leq\bar{\beta}(j,0)+2\qquad\ \text{for
all}\ t\in[t_{1},t_{2}].\end{array}$

Therefore, in view of (27), (24), and the definition of $(\ell_{j})_{j}$, we
have

$\displaystyle J(z,u)$
$\displaystyle=\int_{0}^{T_{z}(u)}\left[\ell(\mathbf{d}(x(t)))+l(x(t),u(t))\right]dt\geq\int_{0}^{T_{z}(u)}\ell(\mathbf{d}(x(t)))dt$
$\displaystyle>\int_{t_{1}}^{t_{2}}\ell_{j+1}(\mathbf{d}(x(t)))dt\geq(1+\rho_{j}(\bar{\beta}(j,0)+1))\int_{t_{1}}^{t_{2}}\ell_{1}(\mathbf{d}(x(t)))dt$
$\displaystyle\geq(1+\rho_{j}(\bar{\beta}(j,0)+1))\varkappa_{j}=L_{j}+\Phi(j)\geq
J_{j}(z,\hat{u})=J(z,\hat{u}).$

This provides the required contradiction and the proof of (iii) is complete.

As in claim (iv), let us now suppose that $u\in\mathcal{U}(z)$ is a control
satisfying $J(z,u)<\alpha$, for some $\alpha>0$. Let $(x^{0},x,u)$ be the
corresponding admissible triple from $(0,z)$, and let $j\geq 1$ be the
smallest integer such that $L_{j}+\Phi(j)>\alpha$. We want to show that

$\mathbf{d}(z)<\Theta:=\bar{\beta}(j,0)+2.$

Indeed, assume instead that $\mathbf{d}(z)\geq\bar{\beta}(j,0)+2$. If there
exists a time $t\geq 0$ such that $\mathbf{d}(x(t))<c:=\bar{\beta}(j,0)+1$,
then there are $0<t_{1}<t_{2}<T_{z}(u)$ as in (28). Thus, arguing as in the
proof of claim (iii) we can deduce the inequality
$J(z,u)>L_{j}+\Phi(j)>\alpha$, in contradiction with the hypothesis
$J(z,u)<\alpha$. If instead $\mathbf{d}(x(t,z,u))\geq c$ for all $t\leq
T_{z}(u)$, then $T_{z}(u)=+\infty$ and we get the contradiction

$J(z,u)\geq\int_{0}^{+\infty}\ell_{1}(\mathbf{d}(x(t)))dt\geq\int_{0}^{+\infty}\ell_{1}(c)dt=+\infty.$

Let finally prove (v). Let $\alpha>0$ and assume $\mathbf{d}(z)>\alpha$. For
any $u\in\mathcal{M}([0,+\infty),U)\setminus\mathcal{U}(z)$, the cost
$J(z,u)=+\infty$, so (v) is trivially true. If $u\in\mathcal{U}(z)$, by the
last part of the proof of (iv) (for $c:=\alpha/2$) there exists a positive,
finite time
$t_{2}:=\inf\\{t\in[0,T_{z}(u))\mid\mathbf{d}(x(t,u,z))\leq\alpha/2\\}$. Let
$0<t_{1}<t_{2}$ be a time such that
$\alpha/2\leq\mathbf{d}(x(t,u,z)\leq\alpha$ for all $t\in[t_{1},t_{2}]$. Then,
in view of (24), we have

$J(z,u)\geq\int_{t_{1}}^{t_{2}}\ell_{1}(\mathbf{d}(x(t,u,z)))dt\geq\frac{\alpha}{2}\varkappa(\alpha/2,\alpha)=:\delta>0.$

∎

Let $J$ be as in Lemma 3.2. For any
$z\in\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}$, we define the value
function

$V(z):=\inf_{u\in\mathcal{M}([0,+\infty),U)}J(z,u)\quad\text{for all}\
z\in{\mathbb{R}}^{n}\setminus\mathcal{C},\qquad V(z):=0\quad\text{for all
$z\in\partial\mathcal{C}$.}$

In the following two lemmas we show that $V$ is a MRF for (8)-(9).

######  Lemma 3.3.

The function $V:\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\to[0,+\infty)$
has the following properties:

  1. (i)

$\text{dom }V:=\\{z\in\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\mid
V(z)<+\infty\\}=\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}$;

  2. (ii)

$V$ is positive definite;

  3. (iii)

$V$ is proper;

  4. (iv)

$V$ is continuous.

###### Proof.

Property (i) holds, because, if $z\in\partial\mathcal{C}$, then
$V(z)=0<+\infty$, while, if $\mathbf{d}(z)>0$, by Lemma 3.2, (i) we have
$V(z)\leq J(z,\hat{u})<+\infty$.

In order to prove (ii), observe that given
$z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$, then $\mathbf{d}(z)>\alpha$ for
some $\alpha>0$. Hence, by Lemma 3.2, (v) there exists $\delta>0$, depending
only on $\alpha$, such that $J(z,u)>\delta$ for all
$u\in\mathcal{M}([0,+\infty),U)$. As a consequence, $V(z)\geq\delta>0$, that
is $V$ is positive outside the target.

The function $V$ satisfies (iii) whenever, for all $\alpha>0$, the sublevel
set $E_{\alpha}:=\\{z\in\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\mid
V(z)<\alpha\\}$ is bounded. If $V(z)<\alpha$, then by definition there exists
$u\in\mathcal{U}(z)$ such that $J(z,u)<\alpha$, as well. Hence, by Lemma 3.2,
(iv) we deduce that $\mathbf{d}(z)<\Theta$ for some $\Theta>0$ (depending only
on $\alpha$) and, consequently, the set $E_{\alpha}$ is bounded, as
$E_{\alpha}\subset\overline{B_{\Theta}(\mathcal{C})\setminus\mathcal{C}}$.

Let us finally prove (iv), i.e. the continuity of $V$. Fix $\varepsilon>0$ and
let us first consider $\bar{z}\in\partial\mathcal{C}$. By virtue of Lemma 3.2,
(ii), for any $z$ such that $\mathbf{d}(z)<r_{1}$, we have $V(z)\leq
J(z,\hat{u})\leq\hat{\Phi}(\mathbf{d}(z)):=\bar{\beta}(\mathbf{d}(z),0)+\Phi(\mathbf{d}(z))$,
where, in particular, $\hat{\Phi}$ is continuous, strictly increasing and
equal to 0 at 0. Hence, choosing

(29)
$0<\delta_{\varepsilon}<\frac{1}{2}\left(r_{1}\land\hat{\Phi}^{-1}(\varepsilon)\right),$

we get the continuity of $V$ at $\bar{z}$, as

(30) $|V(z)-V(\bar{z})|=V(z)\leq
J(z,\hat{u})\leq\hat{\Phi}(\mathbf{d}(z))<\varepsilon\quad\text{for all $z\in
B_{2\delta_{\varepsilon}}(\bar{z})$. }$

Assume now $\bar{z}\in{\mathbb{R}}^{n}\setminus\mathcal{C}$. Setting
$\tilde{\delta}_{\bar{z},\varepsilon}:=\delta_{\varepsilon}\land\frac{\mathbf{d}(\bar{z})}{4}$,
we have
$B_{2\tilde{\delta}_{\bar{z},\varepsilon}}(\bar{z})\subset{\mathbb{R}}^{n}\setminus\mathcal{C}$.
We claim that for any $z\in
B_{\tilde{\delta}_{\bar{z},\varepsilon}/2}(\\{\bar{z}\\})$ there exists an
admissible triple $(x^{0},x,u)$ from $(0,z)$ such that

(31) $\begin{array}[]{l}\displaystyle\mathbf{d}(x(t))\leq
M_{\bar{z}}:=\max\left\\{\Psi(\mathbf{d}(z))\mid
z\in\overline{B_{r_{1}/4}(\\{\bar{z}\\})}\right\\}\quad\text{for all $t\geq
0$,}\\\\[6.45831pt] J(z,u)\leq V(z)+\varepsilon.\end{array}$

Indeed, an $\varepsilon$-optimal triple $(x^{0},x,u)$ satisfying $J(z,u)\leq
V(z)+\varepsilon$ always exists, as $V(z)<+\infty$ by (i) above. Furthermore,
we can clearly assume $J(z,u)\leq J(z,\hat{u})$, where
$(\hat{x}^{0},\hat{x},\hat{u})$ is the admissible triple from $(0,z)$ built in
Lemma 3.2. But then the first of the inequalities above follows from Lemma
3.2, (iii).

Set

(32) $\tilde{T}_{z}:=\inf\\{t\geq
0\mid\mathbf{d}(x(t))<\tilde{\delta}_{\bar{z},\varepsilon}\\}.$

Note that $\tilde{T}_{z}>0$, since
$\mathbf{d}(z)>\mathbf{d}(\bar{z})-\frac{\tilde{\delta}_{\bar{z},\varepsilon}}{2}>\frac{3}{2}\tilde{\delta}_{\bar{z},\varepsilon}$.
Let $j=j_{\bar{z},\varepsilon}$ be an integer $\geq 1$ such that
$j\geq\mathbf{d}(\bar{z})+\frac{\tilde{\delta}_{\bar{z},\varepsilon}}{2}>\mathbf{d}(z)$.
Then, using (27) (which is valid for every $j\geq\mathbf{d}(z)$, in view of
(26)) and setting $\displaystyle
m_{\bar{z},\varepsilon}:=\min_{R\in[\tilde{\delta}_{\bar{z},\varepsilon},M_{\bar{z}}]}\ell_{1}(R)>0$,
we get

$\displaystyle L_{j}+\Phi(j)$ $\displaystyle\geq J(z,\hat{u})\geq
J(z,u)\geq\int_{0}^{\tilde{T}_{z}}\ell_{1}(\mathbf{d}(x(t)))dt\geq\tilde{T}_{z}m_{\bar{z},\varepsilon}.$

Thus, there exists an uniform upper bound for the times $\tilde{T}_{z}$.
Precisely, we have

$\tilde{T}_{z}\leq
T_{\bar{z},\varepsilon}:=\frac{L_{j_{\bar{z},\varepsilon}}+\Phi(j_{\bar{z},\varepsilon})}{m_{\bar{z},\varepsilon}}\qquad\text{for
all $z\in B_{\tilde{\delta}_{\bar{z},\varepsilon}/2}(\\{\bar{z}\\})$.}$

To prove the continuity of $V$ at $\bar{z}$, consider arbitrary points
$z_{1},z_{2}\in B_{\tilde{\delta}_{\bar{z},\varepsilon}/2}(\\{\bar{z}\\})$ and
suppose, for instance, $V(z_{1})\leq V(z_{2})$. Let $(x_{1}^{0},x_{1},u_{1})$
be an admissible triple from $(0,z_{1})$ satisfying (31) (for $z=z_{1}$). In
particular, this implies that $x_{1}(t)$ lies in a compact set $\mathcal{K}$
depending only on $\bar{z}$ for all $t\geq 0$. Hence, if $L_{\bar{z}}$ denotes
the Lipschitz constant in $x$ of the dynamics function $f$ on the compact set
$\overline{B_{1}(\mathcal{K})}$, by a standard cut-off technique we can derive
that the trajectory $x(\cdot,u_{1},z_{2})$ is defined for all
$t\in[0,T_{\bar{z},\varepsilon}]$ and satisfies

$\displaystyle\sup_{t\in[0,T_{\bar{z},\varepsilon}]}|x_{1}(t)-x(t,u_{1},z_{2})|\leq|z_{1}-z_{2}|\,e^{L_{\bar{z}}\,T_{\bar{z},\varepsilon}},$

as soon as $|z_{1}-z_{2}|<e^{-L_{\bar{z}}\,T_{\bar{z},\varepsilon}}$.
Actually, from this inequality it also follows that, setting

$\bar{\delta}_{\bar{z},\varepsilon}:=\tilde{\delta}_{\bar{z},\varepsilon}\land\left(\frac{\delta_{\varepsilon}}{2}\land
1\right)e^{-L_{\bar{z}}\,T_{\bar{z},\varepsilon}}$

($\delta_{\varepsilon}$ as in (29)), and assuming $z_{1}$, $z_{2}\in
B_{\bar{\delta}_{\bar{z},\varepsilon}/2}(\\{\bar{z}\\})$, we have that
($|z_{1}-z_{2}|<\bar{\delta}_{\bar{z},\varepsilon}$ and)
$|x(\tilde{T}_{z_{1}},u_{1},z_{2})-x_{1}(\tilde{T}_{z_{1}})|\leq\frac{\delta_{\varepsilon}}{2}$
($\tilde{T}_{z_{1}}$ is as in (32), for $x=x_{1}$). Since
$\mathbf{d}(x_{1}(\tilde{T}_{z_{1}}))\leq\delta_{\varepsilon}$, this implies
that $\bar{z}_{2}:=x(\tilde{T}_{z_{1}},u_{1},z_{2})\in
B_{2\delta_{\varepsilon}}(\bar{z})$. At this point, if
$\hat{u}_{2}\in\mathcal{U}(z_{2})$ denotes an admissible control from
$(0,\bar{z}_{2})$ as in Lemma 3.2, the last part of (30) implies that
$J(\bar{z}_{2},\hat{u}_{2})<\varepsilon$. Therefore, the control $u_{2}$ given
by

$u_{2}(t):=\begin{cases}u_{1}(t)\quad&t\in[0,\tilde{T}_{z_{1}}]\\\
\hat{u}_{2}(t-\tilde{T}_{z_{1}})&t\in(\tilde{T}_{z_{1}},+\infty)\end{cases}$

belongs to $\mathcal{U}(z_{2})$, $x_{2}(t):=x(t,u_{2},z_{2})$ belongs to
$\overline{B_{1}(\mathcal{K})}$ for all $t\in[0,\tilde{T}_{z_{1}}]$, and
denoting with $\omega_{\bar{z}}$ the modulus of continuity of
$\overline{B_{1}(\mathcal{K})}\ni x\mapsto\ell(\mathbf{d}(x))+l(x,u)$ (uniform
w.r.t. the control, because of the assumptions on $l$), we finally obtain

$\displaystyle 0$ $\displaystyle\leq V(z_{2})-V(z_{1})\leq
J(z_{2},u_{2})-J(z_{1},u_{1})+\varepsilon$
$\displaystyle\leq\int_{0}^{\tilde{T}_{z_{1}}}\left[|\ell(\mathbf{d}(x_{2}(t)))-\ell(\mathbf{d}(x_{1}(t)))|\,dt+|l(x_{2}(t),u_{1}(t))-l(x_{1}(t),u_{1}(t))|\right]dt$
$\displaystyle\qquad+J(\bar{z}_{2},\hat{u}_{2})+\varepsilon\leq
T_{\bar{z},\varepsilon}\,\omega_{\bar{z}}(\bar{\delta}_{\bar{z},\varepsilon})+2\varepsilon\leq
3\varepsilon,$

where the last inequality holds by replacing
$\bar{\delta}_{\bar{z},\varepsilon}$ with
$\bar{\delta}_{\bar{z},\varepsilon}\land\omega_{\bar{z}}^{-1}\left(\frac{\varepsilon}{T_{\bar{z},\varepsilon}}\right)$.
The continuity of $V$ at $\bar{z}$ hence follows by the arbitrariness of
$\varepsilon$.

∎

######  Lemma 3.4.

The value function $V$ satisfies the decrease condition (13).

###### Proof.

We divide the proof in three steps. Step 1. Let us first show that, if $V$ is
a viscosity supersolution of the following Hamilton-Jacobi-Bellman equation

(33) $\max_{u\in U}\\{-\langle
DV(z),f(z,u)\rangle-l(z,u)\\}=\ell(\mathbf{d}(z))\qquad\text{for all}\
z\in{\mathbb{R}}^{n}\setminus\mathcal{C},$

then it satisfies the decrease condition (13), characterizing MRFs. Indeed,
(33) implies that (see e.g. the survey paper [3])

$\max_{u\in U}\\{-\langle
p,f(z,u)\rangle-l(z,u)\\}\geq\ell(\mathbf{d}(z))\qquad\text{for all}\
z\in{\mathbb{R}}^{n}\setminus\mathcal{C},\quad\text{for all}\
p\in\partial_{P}V(z),$

so that, for $H$ defined as in (12) and $p_{0}\equiv 1$, one has

$H(z,1,\partial_{P}V(z))\leq-\ell(\mathbf{d}(z))\qquad\text{for all}\
z\in{\mathbb{R}}^{n}\setminus\mathcal{C},\quad\text{for all}\
p\in\partial_{P}V(z),$

Setting $\gamma:=\ell\circ d_{V^{+}}^{-1}$ and recalling that $L$ is
increasing as well as $d_{V^{+}}$, by (7) we finally obtain that $V$ satisfies
condition (13) for $p_{0}\equiv 1$ and such a $\gamma$, namely

$H(z,1,\partial_{P}V(x))\leq-\gamma(V(z))\quad\ \text{for all}\
z\in{\mathbb{R}}^{n}\setminus\mathcal{C}.$

Thus, the next two steps will be devoted to prove that $V$ is a viscosity
supersolution of (33). This proof is not completely standard, because we do
not have the usual growth hypotheses on $f$, $\ell$ and $l$ (see e.g. [1, Ch.
III]). Actually, these assumptions can be avoided here thanks to the results
in Lemma 3.2. Step 2. Let us show that, for every $T>0$ and every
$z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$, one has

(34)
$V(z)\geq\displaystyle\inf_{u\in\hat{\mathcal{U}}(z)}\left\\{\int_{0}^{T_{z}(u)\land
T}[\ell(\mathbf{d}(x(t)))+l(x(t),u(t))]dt+V(x(T_{z}(u)\land T))\right\\},$

where $x:=x(\cdot\,,u,z)$,
$\hat{\mathcal{U}}(z):=\\{u\in\hat{\mathcal{U}}(z)\mid\ J(z,u)\leq
J(z,\hat{u})\\}$, and $\hat{u}$ is as in Lemma 3.2. In view of Lemma 3.2,(i),
the set $\hat{\mathcal{U}}(z)\neq\emptyset$ and

$V(z)=\inf_{u\in\hat{\mathcal{U}}(z)}\int_{0}^{T_{z}(u)}[\ell(\mathbf{d}(x(t)))+l(x(t),u(t))]dt<+\infty.$

Let us refer to the right-hand side of (34) as $v_{T}(z)$. Given
$u\in\hat{\mathcal{U}}(z)$, if $T_{z}(u)\leq T$ we have $V(x(T_{z}(u)\land
T))=0$ and $J(x,u)\geq v_{T}(z)$. If instead $T_{z}(u)>T$, in view of the
definition of $v_{T}$, we get

$\begin{array}[]{l}\displaystyle
J(z,u)=\int_{0}^{T}[\ell(\mathbf{d}(x(t)))+l(x(t),u(t))]dt+\int_{T}^{T_{z}(u)}[\ell(\mathbf{d}(x(t)))+l(x(t),u(t))]dt\\\
\displaystyle=\int_{0}^{T}[\ell(\mathbf{d}(x(t)))+l(x(t),u(t))]dt+\int_{0}^{T_{x_{T}(0)}(u_{T})}[\ell(\mathbf{d}(x_{T}(t)))+l(x_{T}(t),u_{T}(t))]dt\\\
\displaystyle\geq\int_{0}^{T}[\ell(\mathbf{d}(x(t)))+l(x(t),u(t))]dt+V(x(T))\geq
v_{T}(z),\end{array}$

where $x_{T}(t):=x(t+T)$, $u_{T}(t):=u(t+T)$, and
$T_{x_{T}(0)}(u_{T})=T_{z}(u)-T$. Therefore,
$V(z)=\inf_{u\in\hat{\mathcal{U}}(z)}J(z,u)\geq v_{T}(z)$, that is, the
relation (34) is proven.

Step 3. Let us now deduce from (34) that $V$ is a viscosity supersolution of
(33). To this aim, fixed $z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$, we
preliminarily observe that, in view of Lemma 3.2,(iii), every admissible
trajectory-control pair $(x,u)$ with $u\in\hat{\mathcal{U}}(z)$, satisfies

$\mathbf{d}(x(t))\leq\Psi(\mathbf{d}(z))=:R_{z}\qquad\text{for all}\ t\geq 0.$

Furthermore, on the compact set
$\overline{B(\mathcal{C},R_{z})\setminus\mathcal{C}}$, depending only on $z$,
the functions $f$ and $l$ are Lipschitz continuous in $x$, uniformly w.r.t.
the control, and we can fix a modulus of continuity for $\ell$ and a bound
$M_{z}>0$ for $|f|$, $\ell$, and $l$. Hence, choosing e.g.
$\bar{T}_{z}=\frac{\mathbf{d}(z)}{2M_{z}}$, for any $u\in\hat{\mathcal{U}}(z)$
the trajectory $x(\cdot\,,u,z)$ is defined on $[0,\bar{T}_{z}]$ and satisfies
$0<\mathbf{d}(x(t,u,z))\leq R_{z}$ for all $t\in[0,\bar{T}_{z}]$. For
arbitrary $\varepsilon>0$ and $0<T<\bar{T}_{z}$, (34) implies that there
exists some $\bar{u}=\bar{u}_{\varepsilon,T}\in\hat{\mathcal{U}}(z)$, such
that

$\int_{0}^{T}[\ell(\mathbf{d}(\bar{x}(t)))+l(\bar{x}(t),\bar{u}(t))]dt+V(\bar{x}(T))\leq
V(z)+\varepsilon T,$

where $\bar{x}:=x(\cdot,\,\bar{u},z)$. In view of the above considerations,
from now on, taken a test function $\varphi\in C^{1}({\mathbb{R}}^{n})$ such
that $\varphi(z)=V(z)$ and $\varphi(\tilde{z})\leq V(\tilde{z})$ for all
$\tilde{z}\in B(z,r)$, for some $r>0$, the proof that $V$ is a viscosity
supersolution of (33) proceeds as usual, hence we omit it (see e.g. [1, Prop.
III, 2.8]).

∎

Surveying the results on $V$ in Lemmas 3.3 and 3.4, we see that the proof of
implication (i) $\Longrightarrow$ (ii) is concluded. ∎

##  4\. Proof of implication (ii) $\Longrightarrow$ (i)

The proof that the existence of a MRF implies GAC to $\mathcal{C}$ with
regulated cost relies on the following result, establishing a super-optimality
principle satisfied by any MRF.

######  Proposition 4.1.

Let $W:\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\to[0,+\infty)$ be a
continuous MRF for (8)-(9) for some continuous and increasing function
$p_{0}:(0,+\infty)\to[0,1]$ and some continuous and strictly increasing
function $\gamma:(0,+\infty)\to(0,+\infty)$. Then, for any
$z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$, we have

(35) $W(z)\geq\inf_{u\in\mathcal{U}(z)}\sup_{0\leq
T<T_{z}(u)}\left\\{\begin{array}[]{r}\displaystyle\int_{0}^{T}\left[p_{0}(W(x(t)))l(x(t),u(t))+\gamma(W(x(t)))\right]dt\\\
+W(x(T))\end{array}\right\\},$

where $x(\cdot):=x(\cdot\,,u,z)$.

The proof of this proposition relies on the definite positiveness and
properness of $W$ coupled with an extension of results which are already known
only under more restrictive assumptions than ours (see e.g. [1, Thm. 2.40] and
[10, Thm. 3.3]), and it will be given at the end of the section.

Let $W:\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\to[0,+\infty)$ be a
continuous MRF for (8)-(9) for some functions $p_{0}:(0,+\infty)\to[0,1]$ and
$\gamma:(0,+\infty)\to(0,+\infty)$ as in Def. 2.3. Moreover, assume that
$p_{0}$ satisfies the integrability condition (IC). Fix
$z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$. From (35) it follows that, for
$\varepsilon_{1}:=\frac{W(z)}{2}>0$ there exists some admissible control
$u_{1}\in\mathcal{U}(z)$ such that, for any $0\leq T<T_{z}(u_{1})$, we have

$\displaystyle\displaystyle\int_{0}^{T}[p_{0}(W(x_{1}(t)))l(x_{1}(t),u_{1}(t))+\gamma(W(x_{1}(t)))]dt$
$\displaystyle+W(x_{1}(T))$ (36) $\displaystyle\leq W(z)+\frac{W(z)}{2},$

where $x_{1}(\cdot):=x(\cdot\,,u_{1},z)$. We set

$t_{1}:=\inf\left\\{t\in[0,T_{z}(u_{1}))\mid\
W(x_{1}(t))\leq\frac{W(z)}{2}\right\\},\qquad z_{1}:=x_{1}(t_{1}).$

Clearly, $t_{1}$ is $>0$ and is actually a minimum. Hence, (4) implies that

(37) $\left\\{\begin{array}[]{l}\displaystyle\frac{1}{2}\,W(z)\leq
W(x_{1}(t))\leq\frac{3}{2}\,W(z)\qquad\text{for all
$t\in[0,t_{1}]$,}\\\\[6.45831pt] \displaystyle
W(z_{1})=W(x_{1}(t_{1}))=\frac{1}{2}\,W(z),\end{array}\right.$

so that

$\displaystyle\displaystyle p_{0}(W(z_{1}))\,\int_{0}^{t_{1}}$ $\displaystyle
l(x_{1}(t),u_{1}(t))\,dt\leq\int_{0}^{t_{1}}p_{0}(W(x_{1}(t)))\,l(x_{1}(t),u_{1}(t))\,dt$
$\displaystyle\leq W(z)+\frac{W(z)}{2}-W(z_{1})=W(z)=2[W(z)-W(z_{1})],$

which yields the cost bound

(38) $\int_{0}^{t_{1}}l(x_{1}(t),u_{1}(t))\,dt\leq
2\frac{W(z)-W(z_{1})}{p_{0}(W(z_{1}))}.$

From (4), using the functions $d_{W^{-}}$, $d_{W^{+}}$ introduced in (7), we
also obtain

$\gamma\left(\frac{1}{2}\,d_{W^{-}}(\mathbf{d}(z))\right)\,t_{1}\leq\gamma(W(z_{1}))\,t_{1}\leq\int_{0}^{t_{1}}\gamma(W(x_{1}(t)))\,dt\leq\frac{3}{2}\,W(z)\leq\frac{3}{2}\,d_{W^{+}}(\mathbf{d}(z)).$

Define now the continuous function $\bar{T}_{1}:(0,+\infty)\to(0,+\infty)$,
given by

$\bar{T}_{1}(R):=\frac{3d_{W^{+}}(R)}{2\gamma\left(\frac{1}{2}\,d_{W^{-}}(R)\right)}\qquad\text{for
all $R>0$.}$

Hence, the latter inequality yields the following uniform time bound

(39) $t_{1}\leq\bar{T}_{1}(\mathbf{d}(z)).$

Starting from $z_{1}$ and choosing
$\varepsilon_{2}:=\frac{W(z_{1})}{2}=\frac{W(z)}{4}$, arguing as above we can
deduce from (4) the existence of a control $u_{2}\in\mathcal{U}(z_{1})$ and a
time $t_{2}>0$, such that, denoting by $x_{2}$ the trajectory
$x(\cdot\,,u_{2},z_{1})$ and setting $z_{2}:=x_{2}(t_{2})$, we get relations
(37)-(39) with $z$, $z_{1}$, $u_{1}$, $x_{1}$, and $t_{1}$ replaced by
$z_{1}$, $z_{2}$, $u_{2}$, $x_{2}$, and $t_{2}$, respectively. Set $z_{0}:=z$.
In a recursive way, for any integer $N\geq 1$, we can thus choose
$\varepsilon_{N}:=\frac{W(z)}{2^{N}}$ and construct $z_{N}$, $u_{N}$, $x_{N}$,
and $t_{N}>0$, such that $u_{N}\in\mathcal{U}(z_{N-1})$, and
$x_{N}(\cdot):=x(\cdot\,,u_{N},z_{N-1})$, $z_{N}:=x_{N}(t_{N})$, satisfy

(40) $\left\\{\begin{array}[]{l}\displaystyle\frac{1}{2}\,W(z_{N-1})\leq
W(x_{N}(t))\leq\frac{3}{2}\,W(z_{N-1})\qquad\text{for all
$t\in[0,t_{N}]$,}\\\\[6.45831pt] \displaystyle
W(z_{N})=\frac{1}{2}\,W(z_{N-1})=\frac{1}{2^{N}}\,W(z),\end{array}\right.$
(41) $\int_{0}^{t_{N}}l(x_{N}(t),u_{N}(t))\,dt\leq
2\,\frac{W(z_{N-1})-W(z_{N})}{p_{0}(W(z_{N}))}=4\,\frac{W(z_{N})-W(z_{N+1})}{p_{0}(W(z_{N}))},$

and

(42) $t_{N}\leq\bar{T}_{N}(\mathbf{d}(z)),$

where

$\bar{T}_{N}(R):=\frac{3d_{W^{+}}(R)}{2^{N}\gamma\left(\frac{1}{2^{N}}\,d_{W^{-}}(R)\right)}\qquad\text{for
all $R>0$.}$

Set now $T_{0}:=0$, $T_{N}:=\sum_{j=1}^{N}t_{j}$, and
$T_{\infty}:=\sum_{j=1}^{+\infty}t_{j}$ and, for every integer $N\geq 1$,
define the control $u\in\mathcal{M}([0,+\infty),U)$ given by

$\displaystyle u(t)$ $\displaystyle:=u_{N}(t-T_{N-1})\qquad\text{for all
}t\in[T_{N-1},\ T_{N}),$ $\displaystyle u(t)$ $\displaystyle:=w\qquad\text{for
all }t\geq T_{\infty},\quad\text{if $T_{\infty}<+\infty$,}$

(for $w\in U$ arbitrary). Recalling that $W$ is proper and positive definite,
from (40) it is easy to deduce that $\lim_{t\to
T_{\infty}^{-}}\mathbf{d}(x(t,u,z))=0$, so $u\in\mathcal{U}(z)$. Let
$(x^{0},x,u)$ be the corresponding admissible triple from $(0,z)$ (defined on
$[0,+\infty)$). In view of (41) and recalling that $1/p_{0}$ is decreasing,
the cost $x^{0}$ satisfies

(43) $\displaystyle x^{0}(t)\leq\int_{0}^{T_{\infty}}l(x(t),u(t))\,dt$
$\displaystyle\leq\sum_{N=1}^{+\infty}4\,\frac{W(z_{N})-W(z_{N+1})}{p_{0}(W(z_{N}))}$
$\displaystyle\leq
4\,\int_{0}^{W(z)/2}\frac{dv}{p_{0}(v)}=\bar{W}(z)\qquad\text{for every $t\geq
0$,}$

as soon as we set $\bar{W}(z):=4P(W(z)/2)$ for all
$z\in\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}$ ($P$ as in (14)). Notice
that this function $\bar{W}$ is continuous, proper and positive definite, by
the integrability assumption (IC). So, the triple $(x^{0},x,u)$ satisfies the
cost bound condition (11) with regulation function $\bar{W}$. To conclude the
proof that control system (8) with cost (9) is GAC to $\mathcal{C}$ with
regulated cost, it remains only to prove the existence of a descent rate
$\beta$, such that

(44) $\mathbf{d}(x(t))\leq\beta(\mathbf{d}(z),t)\qquad\text{for every $t\geq
0$.}$

First of all, we claim that there exist a strictly increasing, unbounded,
continuous function $\Gamma:{\mathbb{R}}_{\geq 0}\to{\mathbb{R}}_{\geq 0}$
with $\Gamma(0)=0$, and a function
$\mathbf{T}:{\mathbb{R}}_{>0}^{2}\to{\mathbb{R}}_{>0}$, such that, for any
$0<r<R$, for every $z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$ with
$\mathbf{d}(z)\leq R$, the trajectory $x$ from $z$ considered above satisfies
the following conditions:

$\begin{array}[]{lllll}&{\rm(a)}&\mathbf{d}(x(t))\leq{\bf\Gamma}(R)&\
\text{for all}\ t\geq 0,\\\\[6.45831pt] &{\rm(b)}&\mathbf{d}(x(t))\leq r&\
\text{for all}\ t\geq\mathbf{T}(R,r).\end{array}$

Condition (a) follows from (40), because by (7) we have

$\mathbf{d}(x(t))\leq d^{-1}_{W^{-}}(W(x(t)))\leq
d^{-1}_{W^{-}}\left(\frac{3}{2}\,d_{W^{+}}(\mathbf{d}(z))\right)\leq\Gamma(R)\quad\text{for
all $t\geq 0$,}$

as soon as we choose
$\Gamma(R):=d^{-1}_{W^{-}}\left(\frac{3}{2}\,d_{W^{+}}(R)\right)$, $R\geq 0$.
This $\Gamma$ has all the required properties in view of the properties of
$d_{W^{-}}$ and $d_{W^{+}}$. In order to derive (b), we observe that (40)
implies

$\mathbf{d}(x(t))\leq d^{-1}_{W^{-}}(W(x(t)))\leq
d^{-1}_{W^{-}}\left(\frac{3}{2^{N}}\,d_{W^{+}}(R)\right)\quad\text{ for all
$t\geq T_{N}$,}$

so, if $N(R,r)$ is the smallest integer
$\geq\log_{2}\left(3\frac{d_{W^{+}}(R)}{d_{W^{-}}(r)}\right)$, we get

$\mathbf{d}(x(t))\leq r\qquad\text{ for all $t\geq T_{N(R,r)}$.}$

The time $T_{N(R,r)}$ depends on $z$, but, by (42), the value

$\mathbf{T}(R,r):=\sum_{j=1}^{N(R,r)}\bar{T}_{j}(R)$

is a uniform upper bound for $T_{N(R,r)}$. Hence, also condition (b) is valid.
Now, arguing as in [25, Sec. 5], for any $R>0$ let us introduce a strictly
increasing, diverging sequence of positive times $(t_{j})_{j\geq 1}$ depending
on $R$, such that

$t_{j}\geq\mathbf{T}\left(R,\frac{R}{j+1}\right)$

and define the function $b:[0,+\infty)\times[0,+\infty)\to[0,+\infty)$, given
by

$b(R,t):=\begin{cases}\Gamma(R)\qquad\text{for all $t\in[0,t_{1})$,}\\\
\dfrac{R}{j+1}\qquad\text{for all $t\in[t_{j},t_{j+1})$, \, $j\geq
1$.}\end{cases}$

From (a) and (b) it follows that

$\mathbf{d}(x(t))\leq b(\mathbf{d}(z),t)\qquad\text{for all $t\geq 0$.}$

As already noticed in the proof of Lemma 3.1, Step 3, it is actually a routine
exercise to find a $\mathcal{K}\mathcal{L}$ function $\beta\geq b$. Therefore,
the proof of implication (ii) $\Longrightarrow$ (i) is thus complete. ∎

###  4.1. A Comparison Principle and the proof of Proposition 4.1

In the proof of Proposition 4.1, we will use the slightly modified version
below of the classical Comparison Principle for the infinite horizon problem.
In particular, in the known results it is basically necessary to assume the
unilateral Lipschitz continuity hypothesis (i) below777An alternative
hypothesis to (i) is local Lipschitz continuity and at most linear growth in
the state, uniformly w.r.t. the control. (see e.g. the comments after [1, III.
Thm. 2.12]). In the following lemma we show that, when the state has two
components and the dynamics $F$ depends on only one of them, $x$-Lipschitz
continuity of the $F$-component that corresponds to the missing variable is
not necessary.

######  Lemma 4.1 (Comparison Principle).

Let $U\subset{\mathbb{R}}^{m}$, $A\subset{\mathbb{R}}^{m^{\prime}}$ be compact
control sets (not both empty), let
$L:{\mathbb{R}}^{n}\times{\mathbb{R}}^{n^{\prime}}\times U\times
A\to{\mathbb{R}}$ be a continuous running cost, and let
$F_{1}:{\mathbb{R}}^{n}\times U\times A\to{\mathbb{R}}^{n}$,
$F_{2}:{\mathbb{R}}^{n}\times U\times A\to{\mathbb{R}}^{n^{\prime}}$ be
continuous dynamics components, such that

  1. (i)

for some $C>0$, $\langle
F_{1}(x,u,a)-F_{1}(x^{\prime},u,a),x-x^{\prime}\rangle\leq
C|x-x^{\prime}|^{2}$ for all $x$, $x^{\prime}\in{\mathbb{R}}^{n}$ and $u\in
U$, $a\in A$;

  2. (ii)

for some $\bar{K}>0$ , $|F_{2}(x,u,a)|\leq\bar{K}(1+|x|)$ for all
$(x,u,a)\in{\mathbb{R}}^{n}\times U\times A$ and $x\mapsto F_{2}(x,u,a)$ is
uniformly continuous, uniformly w.r.t. the controls;

  3. (iii)

$L$ is bounded and $(x,z)\mapsto L(x,z,u,a)$ is uniformly continuous,
uniformly w.r.t. the controls.

Let
$H:{\mathbb{R}}^{n}\times{\mathbb{R}}^{n^{\prime}}\times{\mathbb{R}}^{n}\times{\mathbb{R}}^{n^{\prime}}\to{\mathbb{R}}$
be the Hamiltonian defined as

$H(x,z,p_{1},p_{2}):=\min_{a\in A}\max_{u\in
U}\Big{\\{}-\langle(p_{1},p_{2}),(F_{1}(x,u,a),F_{2}(x,u,a))\rangle-L(x,z,u,a)\Big{\\}}.$

Given $\sigma>0$, if $v_{1}$,
$v_{2}:{\mathbb{R}}^{n}\times{\mathbb{R}}^{n^{\prime}}\to{\mathbb{R}}$,
bounded and continuous, are, respectively, a viscosity sub- and supersolution
of

$\sigma\,v(x,z)+H(x,z,Dv(x,z))=0\qquad\text{for all
}(x,z)\in{\mathbb{R}}^{n}\times{\mathbb{R}}^{n^{\prime}},$

then $v_{1}(x,z)\leq v_{2}(x,z)$ for all
$(x,z)\in{\mathbb{R}}^{n}\times{\mathbb{R}}^{n^{\prime}}$.

###### Proof.

The proof is a careful adaptation of the proof of [1, III. Thm. 2.12], which
we give in detail for the sakes of clarity and selfconsistency. If we divide
the equation by $\sigma>0$, the functions $F_{1}/\sigma$, $F_{2}/\sigma$ and
$L/\sigma$ satisfy the same structural hypotheses as above. Hence, we can
assume $\sigma=1$ without loss of generality. In view of [1, III. Rem. 2.13]
we can also assume $v_{1}$ and $v_{2}$ uniformly continuous.

Set $\langle x\rangle:=(1+|x|^{2})^{\frac{1}{2}}$. Take the map
$\Phi:{\mathbb{R}}^{n}\times{\mathbb{R}}^{n^{\prime}}\times{\mathbb{R}}^{n}\times{\mathbb{R}}^{n^{\prime}}\to{\mathbb{R}}$,
given by

$\begin{array}[]{l}\displaystyle\Phi(x,z,y,w):=v_{1}(x,z)-v_{2}(y,w)-\frac{|x-y|^{2}}{2\varepsilon}-\frac{|z-w|^{2}}{2\rho}\\\\[6.45831pt]
\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad-\beta(\langle
x\rangle^{N}+\langle z\rangle^{N}+\langle y\rangle^{N}+\langle
w\rangle^{N})\end{array}$

where $\varepsilon$, $\rho$, $\beta$, $N$ are positive parameters to be chosen
conveniently.

Suppose by contradiction that there exist $\delta>0$ and
$(\tilde{x},\tilde{z})\in{\mathbb{R}}^{n}\times{\mathbb{R}}^{n^{\prime}}$ such
that $v_{1}(\tilde{x},\tilde{z})-v_{2}(\tilde{x},\tilde{z})=\delta$. Choose
$\beta>0$ such that $\beta\langle\tilde{x}\rangle\leq\delta/8$ and
$\beta\langle\tilde{z}\rangle\leq\delta/8$, so that for all $0<N\leq 1$ we
have $2\beta(\langle\tilde{x}\rangle^{N}+\langle\tilde{z}\rangle^{N})\leq
2\left(\frac{\delta}{8}+\frac{\delta}{8}\right)=\frac{\delta}{2}$, and

$\frac{\delta}{2}\leq\delta-\frac{\delta}{2}\leq
v_{1}(\tilde{x},\tilde{z})-v_{2}(\tilde{x},\tilde{z})-2\beta(\langle\tilde{x}\rangle^{N}+\langle\tilde{z}\rangle^{N})=\Phi(\tilde{x},\tilde{z},\tilde{x},\tilde{z})\leq\sup\Phi.$

Since $\Phi$ is continuous and tends to $-\infty$ as
$|x|+|z|+|y|+|w|\to+\infty$, there exists a maximum point
$(\bar{x},\bar{z},\bar{y},\bar{w})$, for which, in particular, we get

(45) $0<\frac{\delta}{2}\leq\Phi(\bar{x},\bar{z},\bar{y},\bar{w})=\sup\Phi.$

The obvious inequality
$\Phi(\bar{x},\bar{z},\bar{x},\bar{z})+\Phi(\bar{y},\bar{w},\bar{y},\bar{w})\leq
2\Phi(\bar{x},\bar{z},\bar{y},\bar{w})$, yields

$\frac{|\bar{x}-\bar{y}|^{2}}{\varepsilon}+\frac{|\bar{z}-\bar{w}|^{2}}{\rho}\leq
v_{1}(\bar{x},\bar{z})-v_{1}(\bar{y},\bar{w})+v_{2}(\bar{x},\bar{z})-v_{2}(\bar{y},\bar{w}),$

so the boundedness of $v_{1}$ and $v_{2}$ implies that

(46) $|\bar{x}-\bar{y}|\leq c\sqrt{\varepsilon},\qquad|\bar{z}-\bar{w}|\leq
c\sqrt{\rho},$

for some $c>0$. From the inequality
$\Phi(\bar{x},\bar{z},\bar{x},\bar{w})+\Phi(\bar{y},\bar{z},\bar{y},\bar{w})\leq
2\Phi(\bar{x},\bar{z},\bar{y},\bar{w})$ and thanks to the uniform continuity
of $v_{1}$, $v_{2}$, we can thus derive that

(47) $\frac{|\bar{x}-\bar{y}|^{2}}{\varepsilon}\leq
v_{1}(\bar{x},\bar{z})-v_{1}(\bar{y},\bar{z})+v_{2}(\bar{x},\bar{w})-v_{2}(\bar{y},\bar{w})\leq\omega(|\bar{x}-\bar{y}|)\leq\omega(c\sqrt{\varepsilon}),$

for some modulus of continuity $\omega$.

Consider now the $C^{1}$ test functions

$\begin{array}[]{l}\displaystyle\varphi(x,z):=v_{2}(\bar{y},\bar{w})+\frac{|x-\bar{y}|^{2}}{2\varepsilon}+\frac{|z-\bar{w}|^{2}}{2\rho}+\beta(\langle
x\rangle^{N}+\langle
z\rangle^{N}+\langle\bar{y}\rangle^{N}+\langle\bar{w}\rangle^{N}),\\\\[6.45831pt]
\displaystyle\psi(y,w):=v_{1}(\bar{x},\bar{z})-\frac{|\bar{x}-y|^{2}}{2\varepsilon}-\frac{|\bar{z}-w|^{2}}{2\rho}-\beta(\langle\bar{x}\rangle^{N}+\langle\bar{z}\rangle^{N}+\langle
y\rangle^{N}+\langle w\rangle^{N}).\end{array}$

By definition of $(\bar{x},\bar{z},\bar{y},\bar{w})$, the function
$v_{1}(x,z)-\varphi(x,z)=\Phi(x,z,\bar{y},\bar{w})$ obtains its maximum at
$(\bar{x},\bar{z})$, while $v_{2}(y,w)-\psi(y,w)=-\Phi(\bar{x},\bar{z},y,w)$
obtains its minimum at $(\bar{y},\bar{w})$. As it is easy to see, we have

$\begin{array}[]{l}\displaystyle
D\varphi(\bar{x},\bar{z})=(D_{x}\varphi,D_{z}\varphi)(\bar{x},\bar{z})=\left(\frac{\bar{x}-\bar{y}}{\varepsilon}+\gamma_{1}\bar{x},\frac{\bar{z}-\bar{w}}{\rho}+\gamma_{2}\bar{z}\right),\\\\[6.45831pt]
\displaystyle
D\psi(\bar{y},\bar{w})=(D_{x}\psi,D_{z}\psi)(\bar{y},\bar{w})=\left(\frac{\bar{x}-\bar{y}}{\varepsilon}+\tau_{1}\bar{y},\frac{\bar{z}-\bar{w}}{\rho}+\tau_{2}\bar{w}\right),\end{array}$

if $\gamma_{1}:=\beta N\langle\bar{x}\rangle^{N-2}$, $\gamma_{2}:=\beta
N\langle\bar{z}\rangle^{N-2}$, $\tau_{1}:=\beta N\langle\bar{y}\rangle^{N-2}$,
$\tau_{2}:=\beta N\langle\bar{w}\rangle^{N-2}$. The definition of viscosity
sub- and supersolution yields

$v_{1}(\bar{x},\bar{z})+H\left(\bar{x},\bar{z},D\varphi(\bar{x},\bar{z})\right)\leq
0\leq
v_{2}(\bar{y},\bar{w})+H\left(\bar{y},\bar{w},D\psi(\bar{y},\bar{w})\right),$

so that, for some $u\in U$ and $a\in A$, we have

$\displaystyle v_{1}$ $\displaystyle(\bar{x},\bar{z})-v_{2}(\bar{y},\bar{w})$
$\displaystyle\leq
H\left(\bar{y},\bar{w},\frac{\bar{x}-\bar{y}}{\varepsilon}+\tau_{1}\bar{y},\frac{\bar{z}-\bar{w}}{\rho}+\tau_{2}\bar{w}\right)-H\left(\bar{x},\bar{z},\frac{\bar{x}-\bar{y}}{\varepsilon}+\gamma_{1}\bar{x},\frac{\bar{z}-\bar{w}}{\rho}+\gamma_{2}\bar{z}\right)$
$\displaystyle\leq\left\langle\frac{\bar{x}-\bar{y}}{\varepsilon}+\gamma_{1}\bar{x},F_{1}(\bar{x},u,a)\right\rangle+\left\langle\frac{\bar{z}-\bar{w}}{\rho}+\gamma_{2}\bar{z},F_{2}(\bar{x},u,a)\right\rangle+L(\bar{x},\bar{z},u,a)$
$\displaystyle\qquad-\left\langle\frac{\bar{x}-\bar{y}}{\varepsilon}+\tau_{1}\bar{y},F_{1}(\bar{y},u,a)\right\rangle+\left\langle\frac{\bar{z}-\bar{w}}{\rho}+\tau_{2}\bar{w},F_{2}(\bar{y},u,a)\right\rangle-L(\bar{y},\bar{w},u,a).$

Hence, by standard calculations (see also [1, Lemma 2.11]), using the
definitions of $\gamma_{1}$, $\gamma_{2}$, $\tau_{1}$, and $\tau_{2}$, (46)
and (47), we get

$\displaystyle v_{1}(\bar{x},\,\bar{z})-v_{2}(\bar{y},\bar{w})\leq
C\frac{|\bar{x}-\bar{y}|^{2}}{\varepsilon}+\frac{|\bar{z}-\bar{w}|}{\rho}\,\omega_{2}(|\bar{x}-\bar{y}|)+\omega_{L}(|\bar{x}-\bar{y}|+|\bar{z}-\bar{w}|)$
$\displaystyle\qquad\qquad\qquad+\gamma_{1}K(1+|\bar{x}|^{2})+\tau_{1}K(1+|\bar{y}|^{2})+\gamma_{2}K(1+|\bar{z}|^{2})+\tau_{2}K(1+|\bar{w}|^{2})$
$\displaystyle\leq
C\omega(c\sqrt{\varepsilon})+\frac{c\,\omega_{2}(c\sqrt{\varepsilon})}{\sqrt{\rho}}+\omega_{L}(c\sqrt{\varepsilon}+c\sqrt{\rho})+K\beta
N(\langle\bar{x}\rangle^{N}+\langle\bar{z}\rangle^{N}+\langle\bar{y}\rangle^{N}+\langle\bar{w}\rangle^{N}),$

where $C$ is as in (i) above, $K$ is a suitable positive constant (depending
on $C$ and $\bar{K}$, $\bar{K}$ as in (ii)), and $\omega_{2}$, $\omega_{L}$
are the moduli of continuity of $F_{2}$ and $L$, respectively. At this point,
by choosing $N:=1\land\frac{1}{K}$, $\rho:=\omega_{2}(c\sqrt{\varepsilon})$,
and using (45), we obtain

$\displaystyle\frac{\delta}{2}$
$\displaystyle\leq\Phi(\bar{x},\bar{z},\bar{y},\bar{w})\leq
v_{1}(\bar{x},\bar{z})-v_{2}(\bar{y},\bar{w})-\beta(\langle\bar{x}\rangle^{N}+\langle\bar{z}\rangle^{N}+\langle\bar{y}\rangle^{N}+\langle\bar{w}\rangle^{N})$
$\displaystyle\leq
C\omega(c\sqrt{\varepsilon})+c\,\omega^{1/2}_{2}(c\sqrt{\varepsilon})+\omega_{L}(c\sqrt{\varepsilon}+c\,\omega_{2}(c\sqrt{\varepsilon})),$

which leads to a contradiction as soon as we make $\varepsilon>0$ small
enough. ∎

This comparison principle is interesting in itself. For instance, it implies
that the lipschitzianity conditions on the current cost under which the
optimality principles in [26, 10, 18, 21] were obtained, can be replaced by
mere continuity plus growth assumptions.

Proof of Proposition 4.1 From [4, Thm. 8.1] it follows immediately that, given
a MRF $W$ for some $p_{0}$ and $\gamma$, the decrease condition (13) is
equivalent to the viscosity supersolution condition

(48) $\max_{u\in U}\left\\{-\langle D^{-}W(z),f(z,u)\rangle-
p_{0}(W(z))l(z,u)-\gamma(W(z))\right\\}\geq 0$

for all $z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$. For any $0<b<c$, we define
the sets

$\mathcal{S}_{b}:=\\{z\in\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\mid\
\
W(z)<b\\},\quad\mathcal{S}_{(b,c)}:=\\{z\in\overline{{\mathbb{R}}^{n}\setminus\mathcal{C}}\mid\
\ b<W(z)<c\\}.$

Since $W$ is continuous, proper, and positive definite and
$\partial\mathcal{C}$ is compact, these sets are open, bounded, and nonempty.
Step 1. Fix $M>0$. Then, the set $\mathcal{S}:=\overline{\mathcal{S}}_{M+1}$
is contained in the interior of
$\mathcal{S}^{\prime}:=\overline{\mathcal{S}}_{M+2}$ and we can define a
$C^{1}$ function $\tilde{\eta}:{\mathbb{R}}^{n}\to[0,1]$ such that

$\tilde{\eta}(z):=\begin{cases}1\qquad\text{if}\ z\in\mathcal{S},\\\
0\qquad\text{if}\
z\in{\mathbb{R}}^{n}\setminus\mathcal{S}^{\prime}.\end{cases}$

For all $(z,u)\in{\mathbb{R}}^{n}\times U$, we set

$\tilde{f}(z,u):=\tilde{\eta}(z)f(z,u),\quad\tilde{\ell}(z,u):=\tilde{\eta}(z)\left[p_{0}(W(z))l(z,u)+\gamma(W(z))\right]\
(\geq 0).$

The functions $\tilde{f}$, $\tilde{\ell}$ are continuous, bounded,
$x\mapsto\tilde{f}(x,u)$ is (globally) Lipschitz continuous and
$x\mapsto\tilde{\ell}(x,u)$ is uniformly continuous, uniformly w.r.t. the
control. Since $\tilde{\eta}\geq 0$, from (48) it follows that $W$ also
satisfies

(49) $\max_{u\in U}\left\\{-\langle
D^{-}W(z),\tilde{f}(z,u)\rangle-\tilde{\ell}(z,u)\right\\}\geq 0\quad\text{for
all }z\in{\mathbb{R}}^{n}\setminus\mathcal{C}.$

Step 2. Fix $\varepsilon\in(0,1)$ and let
$\eta_{\varepsilon}:{\mathbb{R}}^{n}\to[0,1]$ be a $C^{1}$ function such that

$\eta_{\varepsilon}(z):=\begin{cases}1\qquad\text{if}\
z\in\mathcal{S}_{\left(\varepsilon,\frac{1}{\varepsilon}\right)},\\\
0\qquad\text{if}\
z\in{\mathbb{R}}^{n}\setminus\mathcal{S}_{\left(\frac{\varepsilon}{2},\frac{2}{\varepsilon}\right)}.\end{cases}$

Setting, for all $(z,u)\in{\mathbb{R}}^{n}\times U$,

$\tilde{f}_{\varepsilon}(z,u):=\eta_{\varepsilon}(z)\tilde{f}(z,u),\quad\tilde{\ell}_{\varepsilon}(z,u):=\eta_{\varepsilon}(z)\tilde{\ell}(z,u),$

we finally obtain that $W$ satisfies

(50) $\max_{u\in U}\left\\{-\langle
D^{-}W(z),\tilde{f}_{\varepsilon}(z,u)\rangle-\tilde{\ell}_{\varepsilon}(z,u)\right\\}\geq
0\quad\text{for all }z\in{\mathbb{R}}^{n}.$

Let $\lambda:{\mathbb{R}}\to(0,1)$ be a $C^{1}$ function such that
$0<\dot{\lambda}\leq\bar{C}$ for some $\bar{C}>0$, $\lambda(s)\to 0$ as
$s\to-\infty$ and $\lambda(s)\to 1$ as $s\to+\infty$, as, for instance,
$\lambda(s)=\frac{1}{\pi}\left(\arctan(s)+\frac{\pi}{2}\right)$. Hence, by [1,
Prop. 2.5], the function

$V(z,r):=\lambda(W(z)+r)$

turns out to be a bounded, continuous, and nonnegative viscosity supersolution
of the Hamilton-Jacobi-Bellman equation

$\max_{u\in U}\left\\{-\langle
Dv(z,r)\,,\,\tilde{F}_{\varepsilon}(z,u)\rangle\right\\}=0\quad\text{for all
}(z,r)\in{\mathbb{R}}^{n+1},$

where
$\tilde{F}_{\varepsilon}(z,u):=(\tilde{f}_{\varepsilon}(z,u),\tilde{\ell}_{\varepsilon}(z,u))$.
Set $\Psi(z,r):=V(z,r)$. Since $V\geq 0$, given $\sigma>0$, $V$ is also a
viscosity supersolution of the obstacle equation

(51) $\min\left\\{\sigma\,v(z,r)+\max_{u\in U}\left\\{-\langle
Dv(z,r)\,,\,\tilde{F}_{\varepsilon}(z,u)\rangle\right\\},v(z,r)-\Psi(z,r)\right\\}=0\
\text{on}\ {\mathbb{R}}^{n+1}.$

By introducing a new control $a\in\\{0,1\\}$ and defining
$\hat{F}_{\varepsilon}(z,u,a):=a\,\tilde{F}_{\varepsilon}(z,u)$,
$\hat{L}(z,r,u,a):=(1-a)\sigma\Psi(z,r)$, we can reformulate (51) as

(52) $\sigma\,v(z,r)+\min_{a\in\\{0,1\\}}\max_{u\in U}\left\\{-\langle
Dv(z,r)\,,\,\hat{F}_{\varepsilon}(z,u,a)\rangle-\hat{L}(z,r,u,a)\right\\}=0\
\text{on}\ {\mathbb{R}}^{n+1},$

where all the assumptions of Lemma 4.1 are met. Thus, (52) (equivalently,
(51)) satisfies the comparison principle for bounded viscosity sub- and
supersolutions. In particular, by a standard dynamic programming procedure,
the unique bounded, continuous solution of (51) is the value function

$V^{\varepsilon}_{\sigma}(z,r):=\inf_{u\in\mathcal{M}([0,+\infty),U)}\sup_{T\geq
0}e^{-\sigma
T}\Psi(\tilde{x}_{\varepsilon}(T),\tilde{x}^{0}_{\varepsilon}(T))\qquad\text{for
all}\ (z,r)\in{\mathbb{R}}^{n+1},$

where $(\tilde{x}_{\varepsilon},\tilde{x}^{0}_{\varepsilon})$ is the unique
solution of the control system
$(\dot{x},\dot{x}^{0})=\tilde{F}_{\varepsilon}(x,u)$ with initial condition
$(z,r)$. From the comparison principle,

(53) $V(z,r)\geq V^{\varepsilon}_{\sigma}(z,r)\qquad\text{for all}\
(z,r)\in{\mathbb{R}}^{n+1}.$

Given $z\in\mathcal{S}_{\left(\varepsilon,\frac{1}{\varepsilon}\right)}$ and
for every $u\in\mathcal{M}([0,+\infty),U)$, set

$\tilde{T}_{z}^{\varepsilon}(u):=\inf\left\\{t\geq 0\mid\ \
W(\tilde{x}_{\varepsilon}(t))\geq\frac{1}{\varepsilon}\ \text{or}\
W(\tilde{x}_{\varepsilon}(t))\leq\varepsilon\right\\}\leq+\infty.$

Clearly, $\tilde{T}_{z}^{\varepsilon}(u)>0$, as
$\varepsilon<W(z)<\frac{1}{\varepsilon}$. Furthermore, for all
$t\in[0,\tilde{T}_{z}^{\varepsilon}(u))$, the solution
$(\tilde{x}_{\varepsilon},\tilde{x}^{0}_{\varepsilon})$ corresponding to $u$
coincides with the solution, say $(\tilde{x},\tilde{x}^{0})$, of the control
system $(\dot{x},\dot{x}^{0})=(\tilde{f},\tilde{\ell})(x,u)$ with
$(\tilde{x},\tilde{x}^{0})(0)=(z,r)$. Hence, letting $\sigma$ tend to zero in
(53), for every
$(z,r)\in\mathcal{S}_{\left(\varepsilon,\frac{1}{\varepsilon}\right)}\times{\mathbb{R}}$,
we get the inequality

$V(z,r)\geq\inf_{u\in\mathcal{M}([0,+\infty),U)}\sup_{T\in\left[0,\tilde{T}_{z}^{\varepsilon}(u)\right)}V(\tilde{x}(T),\tilde{x}^{0}(T)),$

that, by a recursive procedure as in [26], finally implies

(54)
$V(z,r)\geq\inf_{u\in\mathcal{M}([0,+\infty),U)}\sup_{T\in\left[0,\tilde{T}_{z}(u)\right)}V(\tilde{x}(T),\tilde{x}^{0}(T))\quad\text{for
all}\ (z,r)\in{\mathbb{R}}^{n+1},$

if $\tilde{T}_{z}(u)$ is the first time at which $\tilde{x}$ reaches the
target $\mathcal{C}$ (possibly equal to $+\infty$). Step 3. Let
$z\in\overline{\mathcal{S}}_{M}\setminus\mathcal{C}$, namely $0<W(z)\leq M$,
and fix $0<\rho\leq-\lambda(M)+\lambda(M+1)$. From (54) with initial point
$(z,0)$, it follows that there exists a control
$u\in\mathcal{M}([0,+\infty),U)$ such that, for any
$T\in[0,\tilde{T}_{z}(u))$, we get

$\displaystyle
V(\tilde{x}(T),\tilde{x}^{0}(T))=\lambda\left(W(\tilde{x}(T))+\int_{0}^{T}\tilde{\ell}(\tilde{x}(t),u(t))\,dt\right)\leq
V(z,0)+\rho\leq\lambda(M+1),$

namely,

$\displaystyle
W(\tilde{x}(T))+\int_{0}^{T}\tilde{\ell}(\tilde{x}(t),u(t))\,dt\leq M+1.$

Therefore, $\tilde{x}(t)$ belongs to $\mathcal{S}$ for all
$t\in[0,\tilde{T}_{z}(u))$, so that $\tilde{\eta}$ as in Step 1 is identically
1. As a consequence, we have $\tilde{f}(x,u)\equiv f(x,u)$,
$\tilde{\ell}(x,u)\equiv p_{0}(W(x))l(x,u)+\gamma(W(x))$,
$\tilde{x}(\cdot)\equiv x(\cdot\,,u,z)$, and $\tilde{T}_{z}(u)\equiv T_{z}(u)$
as in Def. 2.1, so that, in particular $u\in\mathcal{U}(z)$. By the
arbitrariness of $M>0$, the above considerations imply that $W$ satisfies (35)
for all $z\in{\mathbb{R}}^{n}\setminus\mathcal{C}$. The proof of Proposition
4.1 is thus complete. ∎

## References

  * [1] M. Bardi and I. Capuzzo Dolcetta.  Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations, volume 12.  Springer, 1997.
  * [2] P. Cannarsa and C. Sinestrari.  Semiconcave functions, Hamilton-Jacobi equations, and optimal control, volume 58.  Springer Science & Business Media, 2004.
  * [3] F. H. Clarke.  Discontinuous feedbacks in nonlinear control.  In Proc. IFAC Conf. Nonlinear Control (NOLCOS), Bologna, pages 1–29. IFAC, 2010.
  * [4] F. H. Clarke and Y.S. Ledyaev.  Mean value inequalities in Hilbert space.  Transactions of the american mathematical society, 344(1):307–324, 1994.
  * [5] F. H. Clarke, Y.S. Ledyaev, L. Rifford, and R.J. Stern.  Feedback stabilization and Lyapunov functions.  SIAM Journal on Control and Optimization, 39(1):25–48, 2000.
  * [6] F. H. Clarke, Y.S. Ledyaev, R.J. Stern, and P.R. Wolenski.  Nonsmooth analysis and control theory, volume 178.  Springer Science & Business Media, 2008.
  * [7] G. Fusco.  Robust feedback stabilization by means of Lyapunov-like functions determined by Lie brackets.  J. Differential Equations, 287:88–112, 2021.
  * [8] G. Fusco, M. Motta, and F. Rampazzo.  HJ inequalities involving Lie brackets and feedback stabilizability with cost regulation.  Submitted. http://arxiv.org/abs/2302.09078.
  * [9] G. Fusco, M. Motta, and F. Rampazzo.  A Lie-bracket-based notion of stabilizing feedback in optimal control.  Submitted, http://arxiv.org/abs/2302.08915.
  * [10] M. Garavello and P. Soravia.  Optimality principles and uniqueness for Bellman equations of unbounded control problems with discontinuous running cost.  NoDEA, Nonlinear differ. equ. appl., 11:271–298, 2004.
  * [11] C.M. Kellett.  Classical converse theorems in Lyapunov’s second method.  Discrete and Continuous Dynamical Systems - Series B, 20(8):2333–2360, 2015.
  * [12] C.M. Kellett and A.R. Teel.  Weak converse Lyapunov theorems and control-Lyapunov functions.  SIAM Journal on Control and Optimization, 42(6):1934–1959, 2004\.
  * [13] A. C. Lai and M. Motta.  Stabilizability in optimal control.  Nonlinear Differential Equations and Applications, 27(4):41, 2020\.
  * [14] A. C. Lai and M. Motta.  Stabilizability in optimization problems with unbounded data.  Discrete and Continuous Dynamical Systems - Series A, 41(5):2447–2474, 2021.
  * [15] A. C. Lai and M. Motta.  Converse Lyapunov theorems for control systems with unbounded controls.  Journal of Differential Equations, 312:347–373, 2022.
  * [16] Anna C. Lai and M. Motta.  Stabilizability in impulsive optimization problems.  IFAC-PapersOnLine, 52(16):352–357, 2019.
  * [17] M. Malisoff.  Bounded-from-below solutions of the Hamilton-Jacobi equation for optimal control problems with exit times: vanishing Lagrangians, eikonal equations, and shape-from-shading.  NoDEA Nonlinear Differential Equations Appl., 11(1):95–122, 2004\.
  * [18] M. Motta.  Viscosity solutions of HJB equations with unbounded data and characteristic points.  Appl. Math. Optim., 49(1):1–26, 2004.
  * [19] M. Motta and F. Rampazzo.  Asymptotic controllability and optimal control.  Journal of Differential Equations, 254(7):2744–2763, 2013.
  * [20] M. Motta and F. Rampazzo.  Asymptotic controllability and Lyapunov-like functions determined by Lie brackets.  SIAM J. Control Optim., 56(2):1508–1534, 2018.
  * [21] M. Motta and C. Sartori.  The value function of an asymptotic exit-time optimal control problem.  Nonlinear Differential Equations and Applications NoDEA, 22(1):21–44, 2015.
  * [22] L. Rifford.  Existence of Lipschitz and semiconcave control-Lyapunov functions.  SIAM Journal on Control and Optimization, 39(4):1043–1064, 2000\.
  * [23] L. Rifford.  Semiconcave control-Lyapunov functions and stabilizing feedbacks.  SIAM Journal on Control and Optimization, 41(3):659–681, 2003.
  * [24] E. D. Sontag.  Lyapunov-like characterization of asymptotic controllability.  SIAM Journal on Control and Optimization, 21(3):462–471, 1983.
  * [25] E. D. Sontag.  Stability and stabilization: discontinuities and the effect of disturbances.  In Dordrecht Kluwer Acad. Publ., editor, Nonlinear analysis, differential equations and control (Montreal, QC, 1998), volume 528 of  NATO Sci. Ser. C Math. Phys. Sci., 528, pages 551–598, 1999.
  * [26] P. Soravia.  Optimality principles and representation formulas for viscosity solutions of Hamilton-Jacobi equations. I. Equations of unbounded and degenerate control problems without uniqueness.  Advances Diff. Equations, 4:275–296, 1999.
  * [27] R. Vinter.  Optimal Control.  Springer Science & Business Media, 2000.