# Uniform non-convex optimisation via Extremum Seeking

Nicola Mimmo, Lorenzo Marconi and Giuseppe Notarstefano This research was
partially supported by the European Project ”AerIal RoBotic technologies for
professiOnal seaRch aNd rescuE” (AirBorne), Call: H2020, ICT-25-2016/17, Grant
Agreement no: 780960.The authors are with the Department of Electrical and
Information Engineering, ”Guglielmo Marconi”, University of Bologna, 40126
Bologna, Italy, e-mail: {nicola.mimmo2, lorenzo.marconi,
<EMAIL_ADDRESS>
###### Abstract

The paper deals with a well-known extremum seeking scheme by proving
uniformity properties with respect to the amplitudes of the dither signal and
of the cost function. Those properties are then used to show that the scheme
guarantees the global minimiser to be semi-global practically stable despite
the presence of local minima. Under the assumption of a globally Lipschitz
cost function, it is shown that the scheme, improved through a high-pass
filter, makes the global minimiser practically stable with a global domain of
attraction.

Keywords—Extremum Seeking, Fourier Series, Non-Convex Optimisation.

##  I Introduction

The early research on the Extremum Seeking (ES) dates back to the 1920s [1]
and since then this strategy has been extensively exploited to solve several
optimisation problems in electronics [2], mechatronics [3], mechanics [4],
aerodynamics [5], thermohydraulics [6], and thermoacoustic [7]. Some of the
most popular ES schemes are those proposed in [8, 9], which represent the
subject of the proposed analysis, although a remarkable variety of schemes
were proposed, such as the adoption of integral action in [10], the use of a
cost function’s parameter estimator [11, 12], the introduction of an observer
[13], the extension to fractional derivatives in [3], the use of a predictor
to compensate output delays in [14], the implementation of a Newton-based
algorithm avoiding the Hessian matrix inversion [15, 16], and the concurrent
use of a simplex-method to find the global minimiser [17].

All the methods, in a way or another, share the common philosophy of
perturbing the system subject to optimisation using the so-called dither
signal, which is periodic in most of the proposed solutions, to unveil in
which direction the associated cost function decreases (in the case of a
minimisation problem). The analysis tool that is typically adopted to prove
practical convergence to the minimiser exploits the averaging theory [18]. It
is shown that the average system asymptotically converges to the minimiser and
that the trajectories of the original system remain closed to the average ones
if a design parameter, namely $\gamma$, is kept sufficiently small [19, 20, 9,
21, 22].

A further fundamental parameter of most of the ES schemes is represented by
the dither amplitude, named $\delta$ from now on. Roughly, it represents the
local range in which the cost function is evaluated to estimate the local cost
variation and so the decreasing direction. Differently from $\gamma$, $\delta$
is subject to a design compromise [23]. From one hand, the smaller $\delta$
the more the estimation of the local cost variation approximates the local
cost gradient [20, 24]. Moreover, assuming the current optimisation variable
sufficiently close to the actual minimiser, the smaller $\delta$ the better is
the cost optimisation. On the other hand, as far as local minima are
conceived, small $\delta$ could trap the scheme at a local minimiser due to a
local inversion of the cost gradient. So, to overcome this issue, the dither
amplitude should be sufficiently large to make the estimation of the cost
variation robust in respect of local cost fluctuations [25]. For this reason,
some authors suggested an adaptation policy for the dither amplitude which
shrinks $\delta$ over time so that at the beginning a large dither lets the ES
overcome local minima while asymptotically a small dither guarantees good
optimisation performances [26, 27].

For sake of completeness, it is worth mentioning the results in [28, 29] in
which the value of the cost function is directly used to perturb the phase of
the dither signal rather than to estimate the local gradient. Lie-derivative
arguments, instead of averaging techniques, are then adopted in the analysis.

Within the previous research context, this paper proposes three contributions.

First, We propose to study the average systems presented in [8] via Fourier
series arguments. This alternative approach allows handling a class of non-
convex cost functions with a unique global minimiser. We show that the average
system trajectories converge to a neighbourhood of the minimiser. The size of
this neighbourhood is proportional to $\delta$, which is not required to be
small. As for the second contribution, it is shown that the addition of a
high-pass filter makes $\gamma$ independent (uniform) of the value of the cost
function. We prove that semi-global and practical convergence to the minimiser
is achieved with the parameter $\gamma$ only depending on the Lipschitz
constant of the cost function in the domain of interest and not on its value.
This allows for tuning of the algorithm that preserves good convergence speed
even in presence of a large domain of attractions. Third and final, we show
that if the cost function is globally Lipschitz and under certain regularity
conditions on the average system, the investigated ES scheme makes the global
minimiser practically stable with a global domain of attraction.

The rest of this paper is organised as follows. Section II provides the
formulation of the problem and reviews the basic ES scheme proposed by [8]. In
this section, we show that the ES scheme applicability can be extended to non
strictly convex cost functions via averaging analyses, which are not based on
Taylor’s arguments. Section III describes the ES scheme improved with the
high-pass filter and states the two main results: the first, about the
uniformity of $\gamma$ with respect to the cost amplitude; the second, on the
global domain of attraction of the minimiser. The performance of the
investigated ES schemes is tested through the simulations detailed in Section
IV. Finally, Section V closes this paper with some concluding comments.

##  II Problem Formulation

The ES problem consists of the optimisation of an unknown cost function
$h\,:\,\mathbb{R}\to\mathbb{R}$ satisfying the following two assumptions.

######  Assumption 1.

The function $h$ is smooth and there exists a $x^{\star}\in\mathbb{R}$ such
that

$h(x)-h(x^{\star})>0\quad\forall\,x\in\mathbb{R}\,:\,x\neq x^{\star}.$

######  Assumption 2.

There exist a locally Lipschitz and strictly quasi-convex function
$m\,:\,\mathbb{R}\to\mathbb{R}$, a class-${\cal K}_{\infty}$ function
$\alpha(\cdot)$, and a $A\geq 0$ such that

  1. 1.

$m(x)-A\leq h(x)\leq m(x)+A$ for all $x\in\mathbb{R}$

  2. 2.

for all $x_{1},x_{2}\in\mathbb{R}\,:\,(x_{1}-x^{\star})(x_{2}-x^{\star})\geq
0$

$|m(x_{2})-m(x_{1})|\geq\alpha(|x_{2}-x_{1}|)$

######  Remark 1.

As for Assumption 2, we observe that the function $h(\cdot)$ might have local
minima whose ”depth” is bounded by the number $A$. When $A=0$ (which implies
$m(\cdot)\equiv h(\cdot)$) Assumption 2 asks for a (not necessarily strict)
monotone behaviour of $h$, where the latter could have isolated saddle points.
In other words, $m(\cdot)$ belongs to the class of the so-called strictly
quasi-convex functions [30]. Assumption 2 is weaker than the common assumption
$(\partial h(x)/\partial x)x>0$ for any $x\neq x^{\star}$ typically present in
literature, (see, among the others, [[8], Assumptions 3 and 4]) ruling out the
existence of local minima or even saddle points.

$x$$h(x),\,m(x)$$m(x)$$h(x)$$A$$\bullet$$h(x^{\star})$$\bullet$$x^{\star}$
Figure 1: Graphical representation of $h(\cdot)$, $m(\cdot)$, $x^{\star}$, and
$A$ declared in Assumption 2.

The problem of semi-global extremum seeking can be formulated in the following
way. For any $\epsilon>0$ and $r_{0}>0$, design a system of the form

$\dot{x}=\varphi_{\epsilon,r_{0}}(x,h(x),t)\quad x(0)=x_{0},$

so that for all $x_{0}$ satisfying $|x_{0}-x^{\star}|\leq r_{0}$ the resulting
trajectories $x(t)$ are bounded and satisfy
$\lim_{t\to\infty}\sup|x(t)-x^{\star}|\leq\epsilon$.

If the convergence property is sought $\forall x_{0}\in\mathbb{R}$ with
$\varphi$ independent of $\epsilon$ and $x_{0}$, then the problem is referred
to as global extremum seeking.

Among the different ES schemes proposed in the literature to solve the
previous problem, a common one is given by (see [8])

$\dot{x}=-\gamma\,y_{\delta}(x,t)\,u(t)\qquad x(0)=x_{0}$ (1)

in which $y_{\delta}(x,t):=h(x+\delta u(t))$, $u(t):=\sin(2\pi t)$ is the
dither signal and $\gamma,\delta>0$ are tunable parameters111 The general case
of a dither takes the form $\tau\mapsto\sin(\omega\tau)$ with $\omega>0$,
$\tau\in\mathbb{R}$ (as considered in [8]) can be always obtained by rescaling
the time as $t=\tau\,2\pi/\omega$.. The block diagram of this algorithm is
represented in Figure 3. In the next part, we briefly comment on the main
properties, as available in the literature, of this algorithm and strengthen
them.

Since the right-hand side of (1) is 1-periodic, the average system linked to
(1) is given by ([31], §10.4)

$\dot{{x}}_{a}=-\gamma\int_{0}^{1}y_{\delta}({x}_{a},t)u(t)dt\,.$ (2)

In the remaining part of the section, we present a different route for the
analysis of (2) based on a Fourier series expansion rather than on the Taylor
one used in the available literature. This analysis allows one to claim
stronger results on (1).

$x$$h(x)$$h(s)$$t$$y_{\delta}(s,t)$$s+\delta u(t)$$t$$s$

Figure 2: Graphical representation of the periodic behaviour of
$y_{\delta}(s,t)$. Conceiving $s$ as a parameter, we see that the oscillation
induced by $u(t)$ is elaborated by the non-linear map $h(\cdot)$ and results
in a periodic function of time (top-right). This latter can be described
through its Fourier coefficients $a_{k}(s)$ and $b_{k}(s)$.

The smoothness of $h(\cdot)$ in Assumption 1 is essentially asked to guarantee
the existence of the Fourier series of the function $y_{\delta}$222Milder
regularity properties guaranteeing the existence of the series could be
assumed.(plus other regularity properties used in the proof of the forthcoming
Lemma 1 and Proposition 1). Then, since $y_{\delta}(x_{a},t)$ and its time
derivatives are continuous and periodic, $y_{\delta}(x_{a},t)$ can be
expressed in terms of its Fourier series as

$\displaystyle y_{\delta}(x_{a},t)=\dfrac{a_{0,\delta}(x_{a})}{2}+$ (3)
$\displaystyle\sum_{k=1}^{\infty}a_{k,\delta}(x_{a})\cos\left(k2\pi
t\right)+b_{k,\delta}(x_{a})\sin\left(k2\pi t\right)$

where

$\displaystyle a_{k,\delta}(x_{a})$
$\displaystyle:=2\int_{0}^{1}y_{\delta}(x_{a},t)\cos(k2\pi t)\,dt$ (4)
$\displaystyle b_{k,\delta}(x_{a})$
$\displaystyle:=2\int_{0}^{1}y_{\delta}(x_{a},t)\sin(k2\pi t)\,dt\,.$

Embedding (3) in (2) it is immediately seen that the average system linked to
(1) reads as

$\dot{x}_{a}=-{\gamma\over 2}b_{1,\delta}(x_{a})\,.$ (5)

For this system the following result holds. In the result, we refer to the
class-${\cal K}_{\infty}$ function $\underline{\delta}^{\star}(\cdot)$ defined
as

$\underline{\delta}^{\star}(s):=2\int_{0}^{1/2}\alpha(s\sin(2\pi t))\,dt$ (6)

######  Lemma 1.

Let $h(\cdot)$ be such that Assumptions 1-2 are satisfied. Then:

  * a)

for all positive $\delta$ and $A$ such that
$\underline{\delta}^{\star}(\delta)\geq A+\bar{b}$ for some $\bar{b}>0$, there
exists a compact set ${{\cal
A}_{\delta}}\subseteq[x^{\star}-\delta,x^{\star}+\delta]$ that is globally
asymptotically and locally exponentially stable for (5).

  * b)

There exists a $\bar{\delta}^{\star}>0$ such that, for all positive $\delta$
and $A$ such that $\delta\leq\bar{\delta}^{\star}$ and
$\underline{\delta}^{\star}(\delta)\geq A+\bar{b}$ for some $\bar{b}>0$, there
exists an equilibrium point $x_{a\delta}^{\star}\in\mbox{\em int }{{\cal
A}_{\delta}}$ that is locally exponentially stable for system (5). If, in
addition, the function $\bar{h}(s):=h(x^{\star}+s)-h(x^{\star})$ is even, then
$x_{a\delta}^{\star}=x^{\star}$.

The Lemma is proved in Appendix A.

Item a) of the previous lemma states that the trajectories of the average
system reach a compact set ${\cal A}_{\delta}$ that is contained in a $\delta$
neighbourhood of $x^{\star}$ for all possible $\delta$ that are sufficiently
large with respect to the ”depth” $A$ of the local minima in a global way.
This, in particular, implies that there exists a class $\cal KL$ function
$\beta(\cdot,\cdot):\mathbb{R}\times\mathbb{R}_{+}\to\mathbb{R}_{+}$ such that

$|x_{a}(t)|_{{\cal A}_{\delta}}\leq\beta(|x_{a}(0)|_{{\cal A}_{\delta}},t)\,$

where $|\cdot|_{\mathcal{A}_{\delta}}$ denotes the distance to the set
$\mathcal{A}_{\delta}$. In case $A=0$, namely only saddle points can be
present, then $\delta$ is only required to be positive. Otherwise, $\delta$
must be taken sufficiently large to not get stuck in local minima. Item $b)$
claims that the set ${\cal A}_{\delta}$ collapses to an equilibrium point if
$\delta$ is also taken sufficiently small, besides being, as before,
sufficiently large according to $A$. which could require Assumption 2 to hold
with a sufficiently small $A$. Moreover, the last point of item b) shows that
$x^{\star}$ represents the equilibrium point only for cost functions that are
locally symmetric around the optimum.

Standard averaging results can be then used to show that the same property is
preserved also for the trajectories of the original system (1) for
sufficiently small $\gamma$ but in a semi-global and practical way. This is
detailed in the next Proposition 1 where we refer to the class ${\cal
K}_{\infty}$ function $\chi(s)$ defined as

$\chi(s):=\beta^{-1}(s,0).$

In the following analysis we denote by $L_{r}>0$ and $M_{r}>0$, respectively
the local Lipschitz constant and the upper bound of the function $h(\cdot)$ on
a closed interval of length $r$. In particular, regularity of $h$ implies that

  * •

for all $r>0$ there exists $L_{r}>0$ such that for all
$x_{1},x_{2}\in[x^{\star}-r,\,x^{\star}+r]$

$|h(x_{1})-h(x_{2})|\leq L_{r}|x_{1}-x_{2}|\,.$ (7)

  * •

for all $r>0$, there exists $M_{r}>0$ such that for all
$x\in[x^{\star}-r,\,x^{\star}+r]$

$|h(x)|\leq M_{r}\,.$ (8)

######  Proposition 1.

Let $h(\cdot)$ be such that Assumptions 1-2 hold and let $r,\delta,d$ be
arbitrary positive numbers such that $r-d-2\delta>0$ and
$\underline{\delta}^{\star}(\delta)\geq A+\bar{b}$ for some $\bar{b}>0$. Let
$r_{0}:=\chi(r-d-2\delta)$. There exist $\bar{t}(r_{0},d)$ and
$\gamma^{\star}(M_{r},L_{r},\delta,d)>0$ such that for any
$\gamma\in(0,\,\gamma^{\star})$, any
$x_{0}\in\mathbb{R}\,:\,|x_{0}-x^{\star}|\leq r_{0}$, the trajectories of (1)
are bounded and

$|x(t)|_{{\cal A}_{\delta}}\leq d\qquad\forall\,t\geq{\bar{t}\over\gamma}\,.$

Appendix B details the proof of this Proposition.

An immediate consequence of the previous result is the next corollary showing
that under Assumptions 1-2 system (1) solves the problem of semi-global
extremum seeking formulated before provided that Assumption 2 is fulfilled
with a sufficiently small $A$.

######  Corollary 1.

Let $h(\cdot)$ be such that Assumptions 1-2 are fulfilled and let $r_{0}$ and
$\epsilon$ be arbitrary positive numbers. Then, there exist
$\bar{t}(r_{0},\epsilon)>0$, $\bar{\delta}^{\star}(\epsilon)>0$ and
$\gamma^{\star}(r_{0},\bar{\delta}^{\star},\epsilon)>0$ such that for any
$\delta\in(0,\bar{\delta}^{\star})$ and
$\underline{\delta}^{\star}(\delta)\geq A+\bar{b}$ for some $\bar{b}>0$, any
$\gamma\in(0,\,\gamma^{\star})$ and any
$x_{0}\in\mathbb{R}\,:\,|x_{0}-x^{\star}|\leq r_{0}$, the trajectories of (1)
are bounded and

$|x(t)-x^{\star}|\leq\epsilon\qquad\forall\,t\geq{\bar{t}\over\gamma}\,.$

By going through the proof of Proposition 1, it is immediately seen that
$\gamma^{\star}$ is inversely proportional to $M_{r}$. As a consequence, the
higher the cost function is within the set where $x(t)$ ranges, the lower the
value of $\gamma$ and, in turn, the slower the convergence rate of $x$ to the
neighbourhood of the optimum. Section III presents an improvement of (1)
overtaking this limitation and, in turn, paving the way for a global result.

###  II-A Comments on Taylor expansion-based averaging analyses

The analysis of (2) is typically approached, see [8], by using a Taylor
expansion of $h(\cdot)$ to obtain a system of the form

$\dot{{x}}_{a}=\,-\gamma c_{1}\delta\left.\dfrac{\partial h}{\partial
x}\right|_{{x}_{a}}-\gamma\sum_{k=2}^{\infty}c_{k}\,{\delta^{2k-1}}\,\left.\dfrac{\partial^{2k-1}h}{\partial
x^{2k-1}}\right|_{{x}_{a}}$ (9)

where $c_{k}$ are suitably defined positive coefficients. A key role in the
study of this system is played by the first-order approximation

$\dot{{x}}_{a}=\,-\gamma c_{1}\delta\left.\dfrac{\partial h}{\partial
x}\right|_{{x}_{a}}\,.$ (10)

In fact, if our Assumption 2 is strengthen by asking that, for any $x\neq
x^{\star}$, it holds $(\partial h(x)/\partial x)(x-x^{\star})>0$ (respectively
$>\alpha(|x-x^{\star}|)$ with $\alpha(\cdot)$ a class-${\cal K}$ function, see
[8], Assumptions 3 and 4), then the Lyapunov arguments of [[8], eq. (45)]
demonstrate $x^{\star}$ to be a stable (respectively globally asymptotically
stable) equilibrium point for (10). These stability properties are transferred
to (2) for sufficiently small $\delta$. Then, averaging techniques [31] can be
used to prove that, for sufficiently small $\gamma$, the trajectories of (1)
and (2) remain arbitrarily close. Namely, the semi-global extremum seeking
problem is solved.

The fact that the asymptotic properties of the average system (2) are just
ensured by the first-order term (10), and thus by the gradient of $h$, implies
that isolated local minima, or even saddle points, of $h$, cannot be handled
by that proof technique. This justifies why [[8], Assumption 3], which is
stronger than our Assumption 2, is needed. Furthermore, we observe that the
previous analysis requires that the dither amplitude is kept sufficiently
small for the higher-order terms of the average dynamics to be negligible.

##  III The High-Pass Filter (HPF)-ES Algorithm

$h(s)$$+$Eq. (11b)$-\bar{y}$$\dot{{x}}=-\gamma
v$$\times$$+$$v$${x}$$s$$u$$\delta$ Figure 3: The HPF-ES scheme consists of an
extension of the classic ES algorithm with the addition of the dashed block.

In [8] the authors proposed the next modification of (1)

$\displaystyle\dot{x}=$
$\displaystyle\,-\gamma\left(y_{\delta}({x},t)-\bar{y}\right)\,u(t)$
$\displaystyle{x}(0)={x}_{0}$ (11a) $\displaystyle\dot{\bar{y}}=$
$\displaystyle\,\gamma\left(y_{\delta}({x},t)-\bar{y}\right)$
$\displaystyle\bar{y}(0)=\bar{y}_{0}$ (11b)

where $u(t)$ is the dither signal defined before, and
$(x,\bar{y})\in\mathbb{R}\times\mathbb{R}$. A block representation of (11) is
depicted in Figure 3.

The intuition behind the previous scheme is to interpret
$y_{\delta}({x},t)-\bar{y}$ as the output of a high pass filter of
$y_{\delta}$. Moreover, the difference $y_{\delta}({x},t)-\bar{y}$ represents
a proxy of the local mean variation of $h(x)$, directly proportional to the
mean local Lipschitz constant. In the following, we show how this feature
guarantees that the upper bound for the value of $\gamma$ is not dependent on
$M_{r}$ (eq. (8)) but rather only on $L_{r}$ ((7)). As for $\gamma$, similarly
to (1), it must be small to let $x$ and $\bar{y}$ be sufficiently slow to
preserve the correlation between the oscillations of
$y_{\delta}({x},t)-\bar{y}$ and those of $u(t)$. The average system of (11) is
defined as

$\displaystyle\dot{{x}}_{a}$
$\displaystyle=-\gamma\int_{0}^{1}\left(y_{\delta}({x_{a}},\tau)-\bar{y}_{a}\right)\,u(\tau)\,d\tau$
(12a) $\displaystyle\dot{\bar{y}}_{a}$
$\displaystyle=-\gamma\,\bar{y}_{a}+\gamma\int_{0}^{1}y_{\delta}({x}_{a},\tau)\,d\tau.$
(12b)

By expanding $y_{\delta}(x_{a},t)$ in terms of the Fourier series as in the
previous section, it turns out that

$\displaystyle\displaystyle\int_{0}^{1}y_{\delta}(x_{a},\tau)\,d\tau$
$\displaystyle=\dfrac{a_{0,\delta}(x_{a})}{2}$ (13a)
$\displaystyle\displaystyle\int_{0}^{1}\left(y_{\delta}({x_{a}},\tau)-\bar{y}_{a}\right)u(\tau)\,d\tau$
$\displaystyle=\dfrac{b_{1,\delta}(x_{a})}{2}$ (13b) where in the latter we
exploited $\int_{0}^{1}\bar{y}_{a}u(\tau)d\tau=0$ and (4).

Hence, the average system reads as

$\displaystyle\dot{{x}}_{a}$
$\displaystyle=-\gamma\dfrac{b_{1,\delta}(x_{a})}{2}$ (14a)
$\displaystyle\dot{\bar{y}}_{a}$
$\displaystyle=-\gamma\,\bar{y}_{a}+\gamma\,\dfrac{a_{0,\delta}({x}_{a})}{2}$
(14b)

which is a cascade where the first subsystem coincides with (5) and the second
subsystem is linear and asymptotically stable. From this, the next result
follows from Lemma 1.

######  Lemma 2.

Let $h(\cdot)$ be such that Assumptions 1-2 are satisfied. Then:

  * a)

for any positive $\delta>0$ and $A>0$ such that
$\underline{\delta}^{\star}(\delta)\geq A+\bar{b}$ for some $\bar{b}>0$, there
exist a compact set ${\cal
A}_{\delta}\subseteq[x^{\star}-\delta,x^{\star}+\delta]$ and a continuous
function $\tau:\mathbb{R}\to\mathbb{R}$ such that the set

$\mbox{\rm graph}\left.\tau\right|_{{\cal
A}_{\delta}}=\\{(x_{a},y_{a})\in{\cal
A}_{\delta}\times\mathbb{R}\;:\;y_{a}=\tau(x_{a})\\}$

is globally asymptotically and locally exponentially stable for (14).

  * b)

There exists $\bar{\delta}^{\star}>0$ such that, for any
$\delta\in(0,\bar{\delta}^{\star})$ and any $A>0$ such that
$\underline{\delta}^{\star}(\delta)\geq A+\bar{b}$ for some $\bar{b}>0$, there
exists an equilibrium point
$(x_{a\delta}^{\star},\bar{y}_{a\delta}^{\star})\in\mbox{\rm
graph}\left.\tau\right|_{{\cal A}_{\delta}}$ that is globally asymptotically
and locally exponentially stable for system (5). If, in addition, the function
$\bar{h}(s):=h(x^{\star}+s)-h(x^{\star})$ is even, then
$x_{a\delta}^{\star}=x^{\star}$.

A sketch of the proof of this Lemma, with special regard to the definition of
the function $\tau(\cdot)$, is presented in Appendix C.

From this, the following result mimics the one of Proposition 1 with the
remarkable difference that the upper bound $\gamma^{\star}$ of $\gamma$ is
uniform with respect to $M_{r}$.

######  Theorem 1.

Let $h(\cdot)$ be such that Assumptions 1-2 are fulfilled and let $r,\delta,d$
be arbitrary positive numbers such that $r-d-2\delta>0$ and
$\underline{\delta}^{\star}(\delta)\geq A+\bar{b}$ for some $\bar{b}>0$. Let
$r_{0}:=\chi(r-d-2\delta)$. There exist $\bar{t}(r_{0},d)$ and
$\gamma^{\star}(L_{r},\delta,d)>0$ such that for any
$\gamma\in(0,\,\gamma^{\star})$ and any $(x_{0},\,\bar{y}_{0})$ fulfilling
$|x_{0}-x^{\star}|\leq r_{0}$ and
$|\bar{y}_{0}-a_{0,\delta}(x_{0})/2|\leq\gamma^{\star}$, then the trajectories
of (11) are bounded and

$\|(x(t),\bar{y}(t))\|_{\mbox{\rm graph}\left.\tau\right|_{{\cal
A}_{\delta}}}\leq d\qquad\forall\,t\geq{\bar{t}\over\gamma}\,.$

This Theorem is proved in Appendix LABEL:sec:DetailedProofHPFE. The fact that
$\gamma^{\star}$ does not depend anymore on $M_{r}$ but only on $L_{r}$
suggests that in presence of a globally Lipschitz cost function the HPF
extremum seeking scheme in (11) is global. This intuition is confirmed in the
following theorem.

######  Theorem 2.

Let $h(\cdot)$ be such that Assumptions 1-2 are fulfilled. Moreover, assume
$h(\cdot)$ be globally Lipschitz. Then, for any $\epsilon>0$ there exist
$\gamma^{\star}(\epsilon)>0$ and $\overline{\delta}(\epsilon)$ such that for
any $\gamma\in(0,\gamma^{\star})$, for any $\delta$ and $A>0$ fulfilling
$\delta\leq\overline{\delta}(\epsilon)$ and
$\underline{\delta}^{\star}(\delta)\geq A+\overline{b}$ for some $\bar{b}>0$,
and for any $x_{0}\in\mathbb{R}$ and
$\bar{y}_{0}\in\mathbb{R}\,:\,|\bar{y}_{0}-a_{0,\delta}(x_{0})/2|\leq\gamma$,
then the trajectories of (11) are bounded and

$\limsup_{t\to\infty}|x(t)-x^{\star}|\leq\epsilon.$

This theorem is proved in Appendix LABEL:sec:ProofCorollary.

##  IV Numerical Results

This section presents numerical results obtained adopting the following cost
function $h(\cdot)\,:\,\mathbb{R}\to\mathbb{R}$

$\displaystyle h(x)=$ $\displaystyle\,h_{0}+A\sin(10x)$ (15)
$\displaystyle\,+\left\\{\begin{array}[]{cc}(x-\pi)^{2}-1&x<\pi\\\
\cos(x-\pi)-2&x\in[\pi,\,2\pi)\\\ (x-2\pi)^{2}-3&x\geq 2\pi\end{array}\right.$

where $h_{0}\in\mathbb{R}$. This function, which verifies Assumptions 1 and 2,
is depicted in Figure 4(a) for $A=0$ and $h_{0}=10$, and in Figure 5(a) for
$A=1/4$ and $h_{0}=10$, on $[-\pi,\,3\pi]$.

The tests are grouped into two categories, the first of which highlights the
performance of (1) whereas the second deals with the behaviour of (11).

As for the performance of (1), the results are presented in agreement with an
incremental complexity policy. First, for the case of a local strongly convex
cost, we show that the Fourier-based averaging (5) predicts the asymptotic
equilibrium of (1) more accurately than the first-order Taylor expansion-based
averaging (10). Then, assuming a quasi-strongly convex cost, we test that the
trajectory of (10) does not track the trajectory of (1) as good as done by
(5). Moreover, we introduce a non-convex cost function and we show that
increasing $\delta$ lets (1) to pass over local minima.

(a)

(b)

(c)

(d)

Figure 4: (a) Cost function (15) for $A=0,\,h_{0}=10$. It presents multiple
points with null first derivative and it is also non-symmetric around the
minimiser. The saddle point is located at $x=\pi$ and the minimiser is at
$x^{\star}=2\pi$. (b)-(c) The average of the classic ES converges to an
equilibrium point within the set $[x^{\star}-\delta,\,x^{\star}+\delta]$
accordingly to what foreseen in Lemma 1. (d) The presence of saddle points
stacks the Taylor-based averaging (yellow) of the classic ES (blue). Vice
versa, the Fourier-based averaging (red) better represents the actual
behaviour of the classic ES (blue). In general, the average based on the
Fourier series tracks the classic ES more accurately than the average based on
the Taylor expansion. These results are obtained for $\gamma=0.1$.

To simulate the case of a strictly convex function, $x_{0}=6$ is picked
sufficiently close to the minimiser of the cost function in Figure 4(a). Then,
Figures 4(b)-4(c) show the convergence performance of the basic ES together
with the average computed by (10) (yellow) and by (5) (red). These simulations
confirm that, in agreement with Lemma 1 and Proposition 1, the optimisation
variable $x$ converges to
$\mathcal{A}_{\delta}\subset[x^{\star}-\delta,\,x^{\star}+\delta]$. In more
detail, the asymmetry of $h(\cdot)$ in the neighbourhood of $x^{\star}$
implies that the equilibrium point of (2) does not correspond to $x^{\star}$,
see yellow lines in Figures 4(c) and 4(b).

To simulate a quasi-strictly convex function, $x_{0}=-\pi$ is picked
sufficiently large to make (1) passing through the saddle point of the cost
function depicted in Figure 4(a). In this scenario, Figure 4(d) shows that the
trajectory of (10) (yellow) gets stacked in presence of local extrema whereas
the trajectories of the proposed Fourier-based (5) (red) remain close to those
of (1), as foreseen by Proposition 1.

(a)

(b)

(c)

(d)

Figure 5: Behaviour of $x(t)$ and $x_{a}(t)$ computed through (10) (yellow)
and (5) (red) with $\gamma=0.1$. A dither amplitude of $\delta=0.5$ is not
sufficient to let (1) (blue) to overcome local extrema whereas, for
$\delta\in\\{0.75,1\\}$ the plot demonstrate that the set
$\Delta=[x^{\star}-\delta,\,x^{\star}+\delta]$ is reached in finite time. In
all these simulations, the averaging trajectories of (10) (yellow) get stuck
at $x\approx 2.42$ where $\partial h(x)/\partial x=0$.

The behaviour of (1) in the case of local minima is shown in Figure 5, for the
cost function depicted in Figure 5(a). A selection of a too-small $\delta$
could trap the ES (1) (blue) in the neighbourhood of local minima, see Figure
5(b). Then, following Lemma 1, for sufficiently large $\delta$, the scheme (1)
(blue) can pass through local minima, see Figures 5(c) and 5(d). It is worth
noting that, also in this case, the averaging based on the first-order Taylor
expansion (10) (yellow) is less accurate than (5) (red) in describing the
trajectories of (1) (blue). A comparison of Figures 5(c) and 5(d) confirms the
correctness of the bounds provided in Lemma 1.

Before investigating the performance of (11), the following test is performed
to show that the classic ES suffers of large values of $|h(\cdot)|$. Indeed,
as depicted in Figure 6 (blue lines in subplots (a) and (c)), while keeping
$\gamma$ fixed, larger $M_{r}$ lead to more oscillatory behaviours. To
mathematically support this result we observe that

$h(x+\delta u)=h(x)+R(x,\delta u)$ (16)

where $R(\cdot,\cdot)$ represents the remainder of the Taylor expansion around
$x$ of $h(\cdot)$. Exploit the definition of the Lipschitz constant of
$h(\cdot)$ and $|u(t)|_{\infty}\leq 1$ to bound the remainder from above as

$|h(x+\delta u)-h(x)|=|R(x,\delta u)|\leq L_{r}\delta.$ (17)

Substitute (16) into (1)

$\dot{x}=-\gamma h(x+\delta u)u=-\gamma(h(x)+R(x,\delta u))u$ (18)

and investigate the following support system

$\dot{x}_{1}=-\gamma h(x_{1})u\qquad x(0)=x_{10}$ (19)

conceivable as approximation of (18) for $|h(x)|\gg L_{r}\delta$. Let
$H(x):=\int h(x)^{-1}dx$ and solve (19) by parts as

$x_{1}(t)=H^{-1}(H(x_{10})-\gamma(\cos(t)-1))$ (20)

which is a pure oscillation whose amplitude is proportional to $|H(x_{10})|$,
evident in subplot (c) of Figure 6.

(a)

(b)

(c)

(d)

Figure 6: Comparison of the behaviours of the classic ES (left) and that
improved through the high-pass filter (right). In these simulations, we set
$A=0$, $x_{0}=4\pi$, and we increased the value of $h_{0}$ from $10$ to
$10^{3}$. Subplots (a) and (c) demonstrate that, keeping fixed $\gamma$, the
classic ES (blue) becomes oscillatory for large values of the cost function.
On the opposite, subplots (b) and (d) show that the ES improved with the high-
pass filter demonstrates a converge rate, uniform in $M_{r}$. These
simulations are performed with $\gamma=\delta=0.1$.

The performance of the ES improved with the high-pass filter (11) is tested
increasing the cost function. Figure 6 shows that the convergence rate of (11)
is uniform with the amplitude of the cost. To conclude, the stability
properties of the dynamics of the high-pass filter are tested in the
simulations of Figure 7 for the same test conditions of Figure 6. These tests
show that the average (14b) (red) accurately tracks the exact dynamics (11b)
(blue).

Figure 7: The behaviour of the HPF-ES does not change for increasing $M_{r}$.
In these simulations the value of $h_{0}$ is increased from $10$ to $10^{3}$
demonstrating that, keeping fixed $\gamma$, the average (14b) (red) accurately
tracks the exact dynamics (11b) (blue). These simulations are performed with
$\gamma=\delta=0.1$.

Through the proposed simulations we confirm three results: a) the classic ES
can deal with cost functions with local minima where, on the opposite, the
average of the classic ES obtained through the Taylor expansion gets stacked
(Figure 4(d) and Figure 5); b) the classic ES and its average via Taylor
expansion do not converge to the same equilibrium point (Figure 4(c)); c) in
case of large cost functions the classic ES behaves as an oscillator whose
amplitude is proportional to the value of the cost function (Figure 6c). Vice
versa, the adoption of the high-pass filter makes the basic ES convergence
rate uniform with respect to the amplitude of the cost function (Figures 6b
and 6d).

###  IV-A Solar Panel Optimisation

As demonstrated in [32], the ES algorithm can be efficiently applied to
optimise the performance of solar panels. In this section, we compare the
performance of (1) and (11) when applied to the photovoltaic array model
presented in [32], and briefly recalled hereafter.

Let $\zeta\in\mathbb{R}^{2}$ be the state of the photovoltaic array modelling
the dynamics of the output voltage $x\in\mathbb{R}$, forced by the input
voltage $z\in\mathbb{R}$. Then, the plant is modelled as a linear time-
invariant system

$\displaystyle\dot{\zeta}=$ $\displaystyle\,A\zeta+Bz$ (21) $\displaystyle x=$
$\displaystyle\,C\zeta$

with $A$ Hurwitz. Since $A$ is Hurwitz, there exists a linear map
$L\,:\mathbb{R}\to\mathbb{R}$ such that the set
$\Omega:=\\{x=Lz,\,z\in\mathbb{R}\\}$ is globally exponentially stable. On the
other hand, let $w\in\mathbb{R}$ be the solar panel output current, then there
exists a non-linear smooth map $q\,:\,\mathbb{R}\to\mathbb{R}$ such that
$w=q(x)$. The cost function (to be maximised) is represented by the output
power $y:=x\,w$ which, constrained on $\Omega$, reads as
$y|_{\Omega}=h(z):=Lz\,q(Lz)$. Figure 8 (top-left) graphically represents
$h(z)$. Use (1) and (21), and let $(x_{b}(t),y_{b}(t))$ be provided by the
solutions of

$\displaystyle\dot{z}_{b}=$ $\displaystyle\,-\gamma y_{b}u(t)$ $\displaystyle
z(0)=z_{0}$ $\displaystyle\dot{\zeta}_{b}=$ $\displaystyle\,A\zeta_{b}+Bz_{b}$
$\displaystyle\zeta_{b}(0)=\zeta_{0}$ $\displaystyle x_{b}=$
$\displaystyle\,C\zeta_{b}$ $\displaystyle y_{b}=$
$\displaystyle\,x_{b}\,q(x_{b})+\nu(t)$

where $\nu(t)\in\mathbb{R}$ represents the watt-meter noise. Moreover, use
(11) and (21), and let $(x_{h}(t),y_{h}(t))$ be provided by the solutions of

$\displaystyle\dot{z}_{h}=$ $\displaystyle\,-\gamma(y_{h}-\bar{y})u(t)$
$\displaystyle z(0)=z_{0}$ $\displaystyle\dot{\bar{y}}=$
$\displaystyle\,\gamma(y_{h}-\bar{y})$ $\displaystyle\zeta_{h}(0)=\zeta_{0}$
$\displaystyle\dot{\zeta}_{h}=$ $\displaystyle\,A\zeta_{h}+Bz_{h}$
$\displaystyle\bar{y}(0)=\bar{y}_{0}$ $\displaystyle x_{h}=$
$\displaystyle\,C\zeta_{h}$ $\displaystyle y_{h}=$
$\displaystyle\,x_{h}\,q(x_{h})+\nu(t).$

Then, the performance indices of these two schemes are compared in Figure 8 in
which the following parameters have been adopted: $\omega=115.2$ rad/s,
$\gamma=0.5$ J-1, $\delta=0.05$ V, and $\|\nu(t)\|_{\infty}=1$ W. As
previously described, the convergence speed of the basic ES (1) is deeply
influenced by the magnitude of the cost function whereas (11) is not. Indeed,
as can be seen in the bottom-right plot of Figure 8, the average speed of the
ES with the high-pass filter is nearly constant and motivated by the quasi-
linearity of the map $h(x)$ for $x\in[0,\,20]$ V. At the opposite, and
accordingly to the study of the toy-example of Figure 6, as soon as the cost
increases the zero-mean oscillations of (1) dominate the slope-related
contents thus stopping the seeking machinery. The final result is that (11)
performs much better than (1).

(a)

(b)

(c)

(d)

Figure 8: (a) Representation of the static map $y|_{\Omega}=h(z)$. (b)
Comparison of the output powers provided by the adoption of the base ES (1)
and the advanced (11). (c) Comparison of the maximiser voltage determined by
the application of (1) and (11). (d) Crude (numerical) averaging of
$y_{b}(t)u(t)$ and $(y_{h}(t)-\bar{y}(t))u(t)$.

##  V Conclusions

This paper deals with two well-known extremum seeking schemes to show that
they work under less restrictive assumptions. Relying on averaging and
Fourier-series arguments, it is demonstrated that these schemes can deal with
non-convex cost functions making the global minimiser (assumed to be unique)
semi-global practically stable. Moreover, it is shown that the presence of a
high-pass filter elaborating the cost function makes the tuning of the
parameters independent of the cost magnitude. This feature is then exploited
to demonstrate that the global minimiser has a global domain of attraction if
the cost function is globally Lipschitz and certain regularity conditions are
verified by the average system.

## References

  * [1] D. Dochain, M. Perrier, and M. Guay, “Extremum seeking control and its application to process and reaction systems: A survey,” Mathematics and Computers in Simulation, vol. 82, no. 3, pp. 369 – 380, 2011.  6th Vienna International Conference on Mathematical Modelling.
  * [2] S. F. Toloue and M. Moallem, “Multivariable sliding-mode extremum seeking control with application to mppt of an alternator-based energy conversion system,” IEEE Transactions on Industrial Electronics, vol. 64, no. 8, pp. 6383–6391, 2017.
  * [3] H. Malek and Y. Chen, “Fractional order extremum seeking control: Performance and stability analysis,” IEEE/ASME Transactions on Mechatronics, vol. 21, no. 3, pp. 1620–1628, 2016.
  * [4] C. Zhang and R. Ordonez, “Numerical optimization-based extremum seeking control with application to abs design,” IEEE Transactions on Automatic Control, vol. 52, no. 3, pp. 454–467, 2007.
  * [5] Hsin-Hsiung Wang, S. Yeung, and M. Krstic, “Experimental application of extremum seeking on an axial-flow compressor,” IEEE Transactions on Control Systems Technology, vol. 8, no. 2, pp. 300–309, 2000.
  * [6] D. J. Burns, C. R. Laughman, and M. Guay, “Proportional–integral extremum seeking for vapor compression systems,” IEEE Transactions on Control Systems Technology, vol. 28, no. 2, pp. 403–412, 2020.
  * [7] W. H. Moase, C. Manzie, and M. J. Brear, “Newton-like extremum-seeking for the control of thermoacoustic instability,” IEEE Transactions on Automatic Control, vol. 55, no. 9, pp. 2094–2105, 2010.
  * [8] Y. Tan, D. Nešić, and I. Mareels, “On non-local stability properties of extremum seeking control,” Automatica, vol. 42, no. 6, pp. 889–903, 2006.
  * [9] M. Krstić and H.-H. Wang, “Stability of extremum seeking feedback for general nonlinear dynamic systems,” Automatica, vol. 36, no. 4, pp. 595 – 601, 2000.
  * [10] M. Guay, “A perturbation-based proportional integral extremum-seeking control approach,” IEEE Transactions on Automatic Control, vol. 61, no. 11, pp. 3370–3381, 2016.
  * [11] M. Guay and D. Dochain, “A minmax extremum-seeking controller design technique,” IEEE Transactions on Automatic Control, vol. 59, no. 7, pp. 1874–1886, 2014.
  * [12] D. Nesic, A. Mohammadi, and C. Manzie, “A framework for extremum seeking control of systems with parameter uncertainties,” IEEE Transactions on Automatic Control, vol. 58, no. 2, pp. 435–448, 2013.
  * [13] L. Hazeleger, M. Haring, and N. van de Wouw, “Extremum-seeking control for optimization of time-varying steady-state responses of nonlinear systems,” Automatica, vol. 119, p. 109068, 2020.
  * [14] T. R. Oliveira, M. Krstić, and D. Tsubakino, “Extremum seeking for static maps with delays,” IEEE Transactions on Automatic Control, vol. 62, no. 4, pp. 1911–1926, 2017.
  * [15] C. Labar, E. Garone, M. Kinnaert, and C. Ebenbauer, “Newton-based extremum seeking: A second-order lie bracket approximation approach,” Automatica, vol. 105, pp. 356–367, 2019.
  * [16] T. R. Oliveira, J. Feiling, S. Koga, and M. Krstić, “Multivariable extremum seeking for pde dynamic systems,” IEEE Transactions on Automatic Control, vol. 65, no. 11, pp. 4949–4956, 2020.
  * [17] Y. Zhang, M. Rotea, and N. Gans, “Simplex guided extremum seeking control with convergence detection to improve global performance,” IEEE Transactions on Control Systems Technology, vol. 24, no. 4, pp. 1266–1278, 2016.
  * [18] F. V. J.A. Sanders, Averaging Methods in Nonlinear Dynamical Systems.  Applied Mathematical Sciences, Springer-Verlag, 1985.
  * [19] A. Teel, L. Moreau, and D. Nesic, “A unified framework for input-to-state stability in systems with two time scales,” IEEE Transactions on Automatic Control, vol. 48, no. 9, pp. 1526–1544, 2003.
  * [20] Y. Tan, D. Nešić, and I. Mareels, “On non-local stability properties of extremum seeking control,” IFAC Proceedings Volumes, vol. 38, no. 1, pp. 550 – 555, 2005.  16th IFAC World Congress.
  * [21] A. Teel and D. Nešić, “Averaging with disturbances and closeness of solutions,” Systems & Control Letters, vol. 40, no. 5, pp. 317 – 323, 2000.
  * [22] A. R. Teel, J. Peuteman, and D. Aeyels, “Semi-global practical asymptotic stability and averaging,” Systems & Control Letters, vol. 37, no. 5, pp. 329 – 334, 1999.
  * [23] D. Nesic, Y. Tan, and I. Mareels, “On the choice of dither in extremum seeking systems: a case study,” in Proceedings of the 45th IEEE Conference on Decision and Control, pp. 2789–2794, 2006.
  * [24] H.-B. Dürr, M. S. Stanković, C. Ebenbauer, and K. H. Johansson, “Lie bracket approximation of extremum seeking systems,” Automatica, vol. 49, no. 6, pp. 1538–1552, 2013.
  * [25] R. Suttner and S. Dashkovskiy, “Robustness and averaging properties of a large-amplitude, high-frequency extremum seeking control scheme,” Automatica, vol. 136, p. 110020, 2022.
  * [26] Y. Tan, D. Nešić, I. Mareels, and A. Astolfi, “On global extremum seeking in the presence of local extrema,” Automatica, vol. 45, no. 1, pp. 245 – 251, 2009.
  * [27] D. Bhattacharjee and K. Subbarao, “Extremum seeking control with attenuated steady-state oscillations,” Automatica, vol. 125, p. 109432, 2021.
  * [28] A. Scheinker and M. Krstić, “Extremum seeking with bounded update rates,” Systems & Control Letters, vol. 63, pp. 25–31, 2014.
  * [29] Y. Zhu and E. Fridman, “Extremum seeking via a time-delay approach to averaging,” Automatica, vol. 135, p. 109965, 2022.
  * [30] S. Karamardian, “Strictly quasi-convex (concave) functions and duality in mathematical programming,” Journal of Mathematical Analysis and Applications, vol. 20, no. 2, pp. 344–358, 1967.
  * [31] H. K. Khalil and J. W. Grizzle, Nonlinear systems, vol. 3.  Prentice hall Upper Saddle River, NJ, 2002.
  * [32] X. Li, Y. Li, J. E. Seem, and P. Lei, “Detection of internal resistance change for photovoltaic arrays using extremum-seeking control mppt signals,” IEEE Transactions on Control Systems Technology, vol. 24, no. 1, pp. 325–333, 2016.
  * [33] J. K. Hale, L. T. Magalhães, and W. Oliva, Dynamics in infinite dimensions, vol. 47.  Springer Science & Business Media, 2006.
  * [34] L. Marconi, L. Praly, and A. Isidori, “Robust asymptotic stabilization of nonlinear systems with non-hyperbolic zero dynamics,” IEEE Transactions on Automatic Control, vol. 55, no. 4, pp. 907–921, 2010.
  * [35] L. Marconi and L. Praly, “Essential and redundant internal models in nonlinear output regulation,” in Analysis and Design of Nonlinear Control Systems, pp. 263–283, Springer, 2008.
  * [36] L. Marconi, L. Praly, and A. Isidori, “Output stabilization via nonlinear luenberger observers,” SIAM Journal on Control and Optimization, vol. 45, no. 6, pp. 2277–2298, 2007.

##  Appendix A Proof of Lemma 1

By letting $\tilde{h}(x):=h(x)-m(x)$ and using the fact that

$\int_{0}^{1}m(s)\sin(2\pi t)\,dt=0$

we observe, from (4), that

$\displaystyle\dfrac{b_{1,\delta}(x)}{2}=\,\int_{0}^{1}h(x+\delta\sin(2\pi
t))\sin(2\pi t)\,dt$ $\displaystyle=$
$\displaystyle\,\int_{0}^{1}[\tilde{h}(x+\delta\sin(2\pi
t))+m(x+\delta\sin(2\pi t))]\sin(2\pi t)\,dt$ $\displaystyle=$
$\displaystyle\,\int_{0}^{1}\tilde{h}(x+\delta\sin(2\pi t))\sin(2\pi t)\,dt$
$\displaystyle\,+\int_{0}^{1}[m(x+\delta\sin(2\pi t))-m(x)]\sin(2\pi t)\,dt.$

As for the first term, note that $\forall\,x\,\in\mathbb{R}$

$\left|\int_{0}^{1}\tilde{h}(x+\delta\sin(2\pi t))\sin(2\pi t)\,dt\right|\leq
A\,.$

As for the second term, we observe that it can be split as the sum of the
following two terms

$\displaystyle\int_{0}^{1/2}\left[m(x+\delta\sin(2\pi t))-m(x)\right]\sin(2\pi
t)\,dt$ $\displaystyle\int_{0}^{1/2}\left[m(x)-m(x-\delta\sin(2\pi
t))\right]\sin(2\pi t)\,dt.$

By using point 2) of Assumption 2, with $x_{2}=x+\delta\sin(2\pi t)$ and
$x_{1}=x$ in the first term, and with $x_{2}=x$ and $x_{1}=x-\delta\sin(2\pi
t)$ in the second one, it holds that

$\int_{0}^{1}[m(x+\delta\sin(2\pi t))-m(x)]\sin(2\pi
t)\,dt\geq\underline{\delta}^{\star}(\delta),$

with $\underline{\delta}^{\star}(\delta)$ is defined in (6), for all $x$ such
that $x\geq x^{\star}+\delta$. Similarly, using the second part of Assumption
2 with the same choices of $x_{1}$ and $x_{2}$, we observe that

$\int_{0}^{1}[m(x+\delta\sin(2\pi t))-m(x)]\sin(2\pi
t)\,dt\leq-\underline{\delta}^{\star}(\delta)$

for all $x$ such that $x\leq x^{\star}-\delta$. Overall, we thus obtain that

$\displaystyle\dfrac{b_{1,\delta}(x)}{2}$
$\displaystyle\geq-A+\underline{\delta}^{\star}(\delta)\qquad$
$\displaystyle\forall\,x\geq x^{\star}+\delta$
$\displaystyle\dfrac{b_{1,\delta}(x)}{2}$ $\displaystyle\leq
A-\underline{\delta}^{\star}(\delta)\qquad$ $\displaystyle\forall\,x\leq
x^{\star}-\delta\,,$

namely, using the constraint on $A$ and $\delta$ specified in the claim,

$\displaystyle\dfrac{b_{1,\delta}(x)}{2}$ $\displaystyle\geq\bar{b}\qquad$
$\displaystyle\forall\,x\geq x^{\star}+\delta$ (22)
$\displaystyle\dfrac{b_{1,\delta}(x)}{2}$ $\displaystyle\leq-\bar{b}\qquad$
$\displaystyle\forall\,x\leq x^{\star}-\delta\,.$

Standard arguments can be then used to claim that the trajectories of (5) are
ultimately bounded and enter in finite time (dependent on $\gamma$) the set
$\bar{\Delta}=[x^{\star}-\delta,x^{\star}+\delta]$. This implies that the
omega limit set of the set $\bar{\Delta}$ of the system (5), denoted by
$\omega(\bar{\Delta})$ (see [33]), is non-empty, it is locally asymptotically
stable for the trajectories of (5) and
$\omega(\bar{\Delta})\subseteq\bar{\Delta}$. Furthermore, the arguments in
Lemma 1 in [34] (see also [35]) show that there exists a set ${\cal
A}_{\delta}\subseteq\bar{\Delta}$, $\omega(\bar{\Delta})\subseteq{\cal
A}_{\delta}$, which is locally exponentially stable for (5). This proves item
a).

To prove item b), we exploit the fact that $b_{1,\delta}(s)$ is a continuous
function fulfilling (22) and we prove that $b_{1,\delta}(s)$ is a strictly
monotonic function for $s\in(x^{\star}-\delta,x^{\star}+\delta)$.

To this end, compute

$\dfrac{\partial b_{1,\delta}(x)}{\partial
x}=\int_{0}^{1}\left.\dfrac{\partial h(s)}{\partial
s}\right|_{x+\delta\sin(2\pi t)}\sin(2\pi t)dt$

By Taylor expansion of ${\partial h(s)}/{\partial s}$ around $x$ we obtain

$\displaystyle\dfrac{\partial h(s)}{\partial s}$
$\displaystyle=\left.\dfrac{\partial h(s)}{\partial
s}\right|_{x}+\left.\dfrac{\partial^{2}h(s)}{\partial
s^{2}}\right|_{x}(s-x)+O((s-x)^{2}).$

By evaluating the previous expression with $s=x+\delta\sin(2\pi t)$, it turns
out that

$\displaystyle\left.\dfrac{\partial h(s)}{\partial
s}\right|_{s=x+\delta\sin(2\pi t)}=$ $\displaystyle\,\left.\dfrac{\partial
h(s)}{\partial s}\right|_{x}$
$\displaystyle+\left.\dfrac{\partial^{2}h(s)}{\partial
s^{2}}\right|_{x}\delta\sin(2\pi t)+O(\delta^{2})$

Hence

$\begin{array}[]{rcl}\dfrac{\partial b_{1,\delta}(x)}{\partial
x}&=&\displaystyle\left.\dfrac{\partial h(s)}{\partial
s}\right|_{x}\int_{0}^{1}\sin(2\pi t)+\\\
&&\displaystyle\delta\left.\dfrac{\partial^{2}h(s)}{\partial
s^{2}}\right|_{x}\int_{0}^{1}\sin^{2}(2\pi t)dt+O(\delta^{2})\\\\[8.53581pt]
&=&\delta\pi\left.\dfrac{\partial^{2}h(s)}{\partial
s^{2}}\right|_{x}+O(\delta^{2})\,.\end{array}$

By Assumption 2, there exists a constant $\bar{c}>0$ such that
$\left.{\partial^{2}h(s)}/{\partial s^{2}}\right|_{x^{\star}}=\bar{c}$.
Furthermore, continuity of ${\partial^{2}h(s)}/{\partial s^{2}}$ implies that
there exists an interval $\Delta$, independent of $\delta$ and with
$x^{\star}\in\Delta$, such that $\dfrac{\partial^{2}h(s)}{\partial
s^{2}}\geq\bar{c}/2$ for all $s\in\Delta$. From this, it follows that

$\dfrac{\partial b_{1,\delta}(x)}{\partial
x}\geq\delta\dfrac{\bar{c}}{4}+O(\delta^{2})$

for all $x\in\Delta$. The result claimed in item b then follows by taking
$\delta^{\star}$ sufficiently small so that
$[x^{\star}-\delta^{\star},x^{\star}+\delta^{\star}]\subseteq\Delta$ and
$|O(\delta^{2})|<\delta{\bar{c}}/{4}$. To prove the last point of item b) we
evaluate

$\displaystyle b_{1,\delta}(x^{\star})=$
$\displaystyle\,2\int_{0}^{1}h(x^{\star}+\delta\sin(2\pi t))\sin(2\pi t)\,dt$
$\displaystyle=$ $\displaystyle\,2\int_{0}^{1/2}h(x^{\star}+\delta\sin(2\pi
t))\sin(2\pi t)\,dt$ $\displaystyle-2\int_{0}^{1/2}h(x^{\star}-\delta\sin(2\pi
t))\sin(2\pi t)\,dt$

in which we have exploited $\sin(2\pi t)=-\sin(2\pi t+\pi)$ for all
$t\in\mathbb{R}$. Then, assuming $\delta\leq\delta^{\star}$, and
$h(x^{\star}-s)=h(x^{\star}+s)$ for any $s\in[0,\,\delta]$ it follows that

$\displaystyle b_{1,\delta}(x^{\star})=$
$\displaystyle\,2\int_{0}^{1/2}h(x^{\star}+\delta\sin(2\pi t))\sin(2\pi
t)\,dt$ $\displaystyle-2\int_{0}^{1/2}h(x^{\star}+\delta\sin(2\pi t))\sin(2\pi
t)\,dt=0.$

##  Appendix B Proof of Proposition 1

The proof of Proposition 1 follows by standard averaging results as C1) and
C2) of Lemma 3 (or, for instance, Theorem 2.6.1 and Theorem 4.2.1 in [18]) and
it is thus just sketched with a particular eye in showing the dependence of
certain key quantities from $M_{r}$.

In the first part of the proof we show that trajectories of (1) and (5)
originating from $x(0)=x_{a}(0)$ are arbitrary closed for an arbitrary large
finite timespan if $\gamma$ is taken sufficiently small. By using the
definition of $r_{0}$, the fact that ${\cal
A}_{\delta}\subseteq[x^{\star}-\delta,x^{\star}+\delta]$ and
$|x_{a}(t)|_{{\cal A}_{\delta}}\leq\beta(|x_{a}(0)|_{{\cal A}_{\delta}},0)$
for all $t\geq 0$, it turns out that

$\displaystyle|x_{a}(t)-x^{\star}|$ $\displaystyle\leq\beta(|x_{a}(0)|_{{\cal
A}_{\delta}},0)+\delta\leq\beta(|x_{a}(0)-x^{\star}|,0)+\delta$
$\displaystyle\leq\beta(r_{0},0)+\delta=r-d-\delta$

for all $t\geq 0$. This implies that $|x_{a}(t)+\delta u(t)-x^{\star}|\leq
r-d<r$ for all $t\geq 0$. Furthermore, for all $x(t)$ such that
$|x(t)-x_{a}(t)|\leq d$ we have $|x(t)+\delta u(t)-x^{\star}|\leq r$. These
facts will used later in conjunction with Assumption 2.

Let

$\epsilon(x_{a},t):=\int_{0}^{t}h(x_{a}+\delta
u(\tau))u(\tau)-\dfrac{b_{1,\delta}(x_{a})}{2}\,d\tau$ (23)

and note that $\epsilon(x_{a},\cdot)$ is $1$-periodic with
$\epsilon(x_{a},n)=0$ for all $n\in\mathbb{N}$. Pick $N\in\mathbb{N}$ as the
largest integer lower than $t$ and decompose the domain $[0,\,t]$ as the union
of $[n-1,n)$ and $[N,\,t]$, with $n=1,\cdots,N$. Then

$\epsilon(x_{a},t)=\int_{N}^{t}h(x_{a}+\delta
u(\tau))u(\tau)-\dfrac{b_{1,\delta}(x_{a})}{2}\,d\tau.$ (24)

Now, add and subtract $h(x_{a})\,u(\tau)$ into the integral (24), and note
that

$\displaystyle\left|(h(x_{a}+\delta u(\tau))\pm
h(x_{a}))u(\tau)-\dfrac{b_{1,\delta}(x_{a})}{2}\right|\leq$ (25)
$\displaystyle\left|(h(x_{a}+\delta
u(\tau))-h(x_{a}))u(\tau)\right|+\left|\dfrac{b_{1,\delta}(x_{a})}{2}\right|$
$\displaystyle+|h(x_{a})||u(\tau)|.$

By Assumption 1, the first term in the previous relation can be bounded as

$\left|(h(x_{a}+\delta u(\tau))-h(x_{a}))u(\tau)\right|\leq L_{r}\delta.$ (26)

Similarly, by definition (4) and Assumption 1, the second term can be bounded
as

$\displaystyle\dfrac{|b_{1,\delta}(x_{a})|}{2}=$
$\displaystyle\,\left|\int_{0}^{1}h(x_{a}+\delta u(t))u(t)\,dt\right|$ (27)
$\displaystyle=$ $\displaystyle\,\left|\int_{0}^{1}(h(x_{a}+\delta
u(t))-h(x_{a}))u(t)\,dt\right|$ $\displaystyle\leq$
$\displaystyle\,\int_{0}^{1}\left|h(x_{a}+\delta u(t))-h(x_{a})\right|\,dt\leq
L_{r}\delta.$

As for the third term, we have $|h(x_{a})||u(\cdot)|<M_{r}$. Overall,
$|\epsilon(x_{a},t)|$ cen be bounded as

$\displaystyle|\epsilon(x_{a},t)|\leq\bar{\epsilon}(L_{r},M_{r},\delta):=2L_{r}\delta+M_{r}\,.$
(28)

Define $z(t):=x_{a}(t)-\gamma\epsilon(x_{a}(t),t)$, and note that, using (28),

$\displaystyle|x(t)-x_{a}(t)|$ $\displaystyle=|x(t)-z(t)-x_{a}(t)+z(t)|$ (29)
$\displaystyle\leq|x(t)-z(t)|+\gamma\bar{\epsilon}(L_{r},M_{r},\delta).$

We consider now the relation

$x(t)-z(t)=x(0)-z(0)+\int_{0}^{t}\dot{x}(\tau)-\dot{z}(\tau)\,d\tau$

and we observe that

$\displaystyle\dot{x}(t)$ $\displaystyle-\dot{z}(t)=$ (30)
$\displaystyle-\gamma\left(h(x(t)+\delta
u(t))u(t)-\dfrac{b_{1,\delta}(x_{a}(t))}{2}\right)$
$\displaystyle-\left.\gamma^{2}\dfrac{\partial\epsilon(x,t)}{\partial
x}\right|_{x=x_{a}(t)}\dfrac{b_{1,\delta}(x_{a}(t))}{2}$
$\displaystyle+\gamma\left.\dfrac{\partial\epsilon(x,t)}{\partial
t}\right|_{x=x_{a}(t)}.$

By adding and subtracting the term

$\gamma\,(\,h(z(t)+\delta u(t))+h(x_{a}(t)+\delta u(t))\,)\,u(t)$ (31)

to (30) and exploiting (23), we obtain

$\displaystyle\dot{x}(t)$ $\displaystyle-\dot{z}(t)=$ (32)
$\displaystyle-\gamma(h(x(t)+\delta u(t))-h(z(t)+\delta u(t)))u(t)$
$\displaystyle-\gamma(h(z(t)+\delta u(t))-h(x_{a}(t)+\delta u(t)))u(t)$
$\displaystyle-\left.\gamma^{2}\dfrac{\partial\epsilon(x,t)}{\partial
x}\right|_{x=x_{a}(t)}\dfrac{b_{1,\delta}(x_{a}(t))}{2}.$

By using Assumption 1 we observe that for all $x(t)$ satisfying
$|x(t)-x_{a}(t)|\leq d$ and for all $z(t)$ satisfying

$|z(t)+\delta u(t)-x^{\star}|\leq r$ (33)

we have

$\displaystyle|h(x(t)+\delta u(t))-h(z(t)+\delta u(t))|\leq$
$\displaystyle\,L_{r}|x(t)-z(t)|$ (34) $\displaystyle|h(z(t)+\delta
u(t))-h(x_{a}(t)+\delta u(t))|\leq$ $\displaystyle\,\gamma
L_{r}|\epsilon(x_{a}(t),t)|.$

Relation (33) is fulfilled if $|z(t)-x_{a}(t)|\leq d$, which, by recalling the
definition of $z$ and the bound (28), is true if
$\gamma\leq\gamma^{\star}_{1}$ with

$\gamma_{1}^{\star}:=\dfrac{\bar{c}}{\bar{\epsilon}(L_{r},M_{r},\delta)}$ (35)

in which $\bar{c}$ is any positive number with $\bar{c}<d$. Now use (4) into
(24) and use $|u(\cdot)|\leq 1$ to obtain

$\displaystyle\left|\left.\dfrac{\partial\epsilon(x,t)}{\partial
x}\right|_{x=x_{a}}\right|\leq$
$\displaystyle\int_{N}^{t}\left|\left.\dfrac{\partial h(x)}{\partial
x}\right|_{x=x_{a}+\delta u(\tau)}u(\tau)\right|\,d\tau$
$\displaystyle+\dfrac{1}{2}\int_{N}^{t}\left|\left.\dfrac{\partial
b_{1,\delta}(x)}{\partial x}\right|_{x=x_{a}}\right|\,d\tau$
$\displaystyle\leq$ $\displaystyle\int_{N}^{t}\left|\left.\dfrac{\partial
h(x)}{\partial x}\right|_{x=x_{a}+\delta u(\tau)}\right|\,d\tau$
$\displaystyle+\int_{N}^{t}\int_{0}^{1}\left|\left.\dfrac{\partial
h(x)}{\partial x}\right|_{x=x_{a}+\delta u(s)}\right|\,ds\,d\tau$
$\displaystyle\leq$ $\displaystyle\,2L_{r}.$ (36)

Overall, by using (27), (28), (34) and (36), it turns out that

$\displaystyle|x(t)-z$
$\displaystyle(t)|\leq|x(0)-z(0)|+\int_{0}^{t}\left|\dot{x}(\tau)-\dot{z}(\tau)\right|\,d\tau$
$\displaystyle\leq$ $\displaystyle\,|x(0)-z(0)|+\gamma
L_{r}\int_{0}^{t}|x(\tau)-z(\tau)|\,d\tau$
$\displaystyle+\gamma^{2}\bar{k}(L_{r},M_{r},\delta)t$

where
$\bar{k}(L_{r},M_{r},\delta):=L_{r}\bar{\epsilon}(L_{r},M_{r},\delta)+2L_{r}^{2}\delta$,
namely, by the Gronwall Lemma [[18], §1.3] to obtain

$\displaystyle|x(t)-z(t)|\leq$ $\displaystyle|x(0)-z(0)|e^{\gamma L_{r}t}$
(37) $\displaystyle+\left(e^{\gamma
L_{r}t}-1\right)\dfrac{\gamma\bar{k}(L_{r},M_{r},\delta)}{L_{r}}.$

Now let $\bar{t}>0$ and $\bar{c}\in(0,d)$ be arbitrary numbers, and let
$\gamma^{\star}(\bar{t},\bar{c},L_{r},M_{r},\delta)$ be defined as
$\gamma^{\star}:=\min\\{\gamma_{2}^{\star},\,\gamma_{3}^{\star},\gamma_{4}^{\star}\\}$
in which

$\displaystyle\gamma_{2}^{\star}:=\dfrac{\bar{c}}{3e^{L_{r}\bar{t}}}\quad\gamma_{3}^{\star}:=\dfrac{\bar{c}L_{r}}{3\bar{k}(L_{r},M_{r},\delta)(e^{L_{r}\bar{t}}-1)}$

and $\gamma_{4}^{\star}:=\gamma_{1}^{\star}/3$. Using (29) and (37), simple
computations show that for any $\gamma\in(0,\,\gamma^{\star}]$

$|x(t)-x_{a}(t)|\leq\bar{c}\quad\mbox{for all }t\in[0,\bar{t}/\gamma]\,.$

We use now the local exponential stability of the attractor ${\cal
A}_{\delta}$ to extend the previous bound for any $t\geq\bar{t}/\gamma$. The
assumption in question implies the existence of $\bar{c}_{0}>0$, $\kappa>0$,
$\lambda>0$ such that the trajectories of (5) starting at
$|x_{a}(0)|_{\mathcal{A}_{\delta}}\leq\bar{c}_{0}$ are bounded by

$|x_{a}(t)|_{\mathcal{A}_{\delta}}\leq\kappa|x_{a}(0)|_{\mathcal{A}_{\delta}}e^{-\gamma\lambda
t}\,.$

Without loss of generality let $\bar{c}_{0}<{d/2\kappa}$ and fix
$\bar{c}\in(0,\bar{c}_{0})$. Furthermore. let $\bar{t}_{1},\bar{t}_{2}>0$ be
such that

$\beta(r,\bar{t}_{1})<\bar{c}_{0}-\bar{c}\qquad\bar{t}_{2}>\dfrac{1}{\lambda}\ln\left(\dfrac{\kappa\,\bar{c}_{0}}{\bar{c}_{0}-\bar{c}}\right)\,,$

fix $\bar{t}=\max\\{\bar{t}_{1},\bar{t}_{2}\\}$ and fix once for all
$\gamma\in(0,\min\\{\gamma^{\star},1\\})$, with
$\gamma^{\star}(\bar{t},\bar{c},L_{r},M_{r},\delta)$ defined before. Now
divide the time axis into sub-intervals of the form

$I_{n}:=\left[n\dfrac{\bar{t}}{\gamma},\,(n+1)\dfrac{\bar{t}}{\gamma}\right)\quad
n\in{\mathbb{N}}$

and, with $x_{a}(t,x_{a0})$ the trajectory of the average system at time $t$
with initial condition $x_{a0}$ at time $t=0$, let

$x_{n}(t):=x_{a}(t-n{\bar{t}}/{\gamma},x(n{\bar{t}}/{\gamma}))\qquad\forall\,t\in
I_{n}.$

Because of the first part of the proof and by the definition of $\bar{c}$, it
turns out that $|x(\bar{t}/\gamma)|_{\mathcal{A}_{\delta}}<\bar{c}_{0}$ and
thus
$|x_{1}(t)|_{\mathcal{A}_{\delta}}<\kappa\bar{c}_{0}e^{-\gamma\lambda(t-\bar{t}/\gamma)}$
for all $t\in I_{1}$. The same arguments used in the first part of the proof
show that

$|x(t)-x_{1}(t)|\leq\bar{c}\qquad\forall\,t\in I_{1}$

Moreover,

$\displaystyle|x(t)|_{\mathcal{A}_{\delta}}$
$\displaystyle\leq|x_{1}(t)|_{\mathcal{A}_{\delta}}+|x(t)-x_{1}(t)|$
$\displaystyle\leq\kappa\bar{c}_{0}e^{-\gamma\lambda(t-\bar{t}/\gamma)}+\bar{c}$
$\displaystyle\leq\kappa\bar{c}_{0}+\bar{c}\leq d$ $\displaystyle\forall\,t\in
I_{1}$

and

$\displaystyle|x(2\bar{t}/\gamma)|_{\mathcal{A}_{\delta}}$
$\displaystyle\leq\kappa\bar{c}_{0}e^{-\gamma\lambda\bar{t}/\gamma}+\bar{c}$
$\displaystyle\leq\kappa\bar{c}_{0}e^{-\lambda\bar{t}_{2}}+\bar{c}<\bar{c}_{0}.$

The last steps can be then iterated for all $n=2,3,...$ and this proves the
Proposition.

##  Appendix C Proof of Lemma 2

This proof follows from standard cascade arguments in view of the result in
Lemma 1 and from the fact that $\gamma$ is a positive parameter. We only
present some detail behind the construction of the (non-unique) function
$\tau(\cdot)$. By (22) in Lemma 1 we have that the set
$\bar{\Delta}=[x^{\star}-\delta,x^{\star}+\delta]$ is forward invariant and
reached in a finite time $t^{\star}(\gamma,x_{a0})$ for the trajectories
$x_{a}(t,x_{a0})$ of (14a) with initial condition $x_{a0}$. Let $\hat{\Delta}$
a superset of $\bar{\Delta}$ and let
$\hat{b}_{1,\delta}\,:\,\mathbb{R}\to\mathbb{R}$ a smooth function defined as

$\hat{b}_{1,\delta}(x)=\left\\{\begin{array}[]{cc}b_{1,\delta}(x)&x\in\bar{\Delta}\\\
0&x\not\in\hat{\Delta}\,.\end{array}\right.$

Denote with $\hat{x}_{a}(t,x_{a0})$ the flow of
$\dot{\hat{x}}_{a}=-\gamma\hat{b}_{1,\delta}(\hat{x}_{a})/2$ at time $t$ with
initial condition $x_{a0}$ and note that for all $x_{a0}\in\bar{\Delta}$ we
have $\hat{x}_{a}(t,x_{a0})={x}_{a}(t,x_{a0})$. Define the map
$\tau\,:\,\mathbb{R}\to\mathbb{R}$ with

$\tau(x)=\dfrac{\gamma}{2}\int_{-\infty}^{0}e^{\gamma\tau}a_{0,\delta}(\hat{x}_{a}(\tau,x))\,d\tau.$

Let $(x_{a}(t,x_{a0}),\bar{y}_{a}(t,\bar{y}_{a0},x_{a0}))$ be the solution of
(14) with initial conditions $(x_{a0},\bar{y}_{a0})$ and note that for any
$x_{a0}\in\mathcal{A}_{\delta}$ the following holds

$\displaystyle\bar{y}_{a}(t,\tau(x_{a0}),x_{a0})=$
$\displaystyle\,\tau(x_{a0})e^{-\gamma t}$
$\displaystyle\,+\dfrac{\gamma}{2}\int_{0}^{t}e^{-\gamma(t-\tau)}a_{0,\delta}(x_{a}(\tau,x_{a0}))\,d\tau$
$\displaystyle=$
$\displaystyle\,\dfrac{\gamma}{2}\int_{-\infty}^{t}e^{-\gamma(t-\tau)}a_{0,\delta}(\hat{x}_{a}(\tau,x_{a0}))\,d\tau$
$\displaystyle=$
$\displaystyle\,\dfrac{\gamma}{2}\int_{-\infty}^{0}e^{\gamma\tau}a_{0,\delta}(\hat{x}_{a}(\tau+t,x_{a0}))\,d\tau$
$\displaystyle=$
$\displaystyle\,\dfrac{\gamma}{2}\int_{-\infty}^{0}e^{\gamma\tau}a_{0,\delta}(\hat{x}_{a}(\tau,\hat{x}_{a}(t,x_{a0})))\,d\tau$
$\displaystyle=$ $\displaystyle\,\tau(\hat{x}_{a}(t,x_{a0}))$ $\displaystyle=$
$\displaystyle\,\tau(x_{a}(t,x_{a0}))\,.$

Since $\mathcal{A}_{\delta}$ is forward invariant for (14a), this implies that
$\mathtt{graph}(\tau|_{\mathcal{A}_{\delta}})$ is forward invariant for (14).
Point a) then follows by using, for instance, the arguments in [36]. As for
point b), adopt the same arguments to prove point b) of Lemma 1, use the
cascade connection, and the forward invariance of
$\mathtt{graph}(\tau|_{\mathcal{A}_{\delta}})$.

##  Appendix D Proof of Theorem 1

First, the trajectories of (11) are demonstrated to remain close to those of
(14) for a finite timespan. Second, exploiting the local exponential stability
of $\mbox{\rm graph}\left.\tau\right|_{{\cal A}_{\delta}}$, the trajectories
of (11) are demonstrated to remain bounded and close to this set for an
infinite time horizon.

Define $\eta=(x,\bar{y})$ and $\eta_{a}=(x_{a},\bar{y}_{a})$, and let

$\displaystyle F(\eta,t)$
$\displaystyle:=\left(\begin{array}[]{c}-\varepsilon_{\delta}({x},\bar{y},t)u(t)\\\
\varepsilon_{\delta}({x},\bar{y},t)\end{array}\right)$

and

$\displaystyle F_{a}(\eta_{a})$
$\displaystyle:=\left(\begin{array}[]{c}-\dfrac{b_{1,\delta}(x_{a})}{2}\\\
-\bar{y}_{a}+\dfrac{a_{0,\delta}(x_{a})}{2}\end{array}\right).$

Moreover, let

$\epsilon(\eta_{a},t):=\int_{0}^{t}F(\eta_{a},\tau)-F_{a}(\eta_{a})\,d\tau$

and note that $\epsilon(\eta_{a},t)$ is $1$-periodic and
$\epsilon(\eta_{a},n)=0$ for each $n\in\mathbb{N}$ and for any
$\eta_{a}\in\mathbb{R}^{2}$. Let $N\in\mathbb{N}$ be the largest integer such
that $N\leq t$, then

$\displaystyle\epsilon(\eta_{a},t)=$
$\displaystyle\sum_{n=1}^{N}\int_{n-1}^{n}F(\eta_{a},\tau)-F_{a}(\eta_{a})\,d\tau$
$\displaystyle+\int_{N}^{t}F(\eta_{a},\tau)-F_{a}(\eta_{a})\,d\tau$
$\displaystyle=$
$\displaystyle\int_{N}^{t}F(\eta_{a},\tau)-F_{a}(\eta_{a})\,d\tau.$

Let $\epsilon_{1}(\eta_{a},t)$ and $\epsilon_{2}(\eta_{a},t)$ be the first and
second entries of $\epsilon(\eta_{a},t)$. We now find a bound for these two
quantities.

As a starting point, let $e_{y_{a}}:=a_{0,\delta}(x_{a})/2-\bar{y}_{a}$ and
compute

$\dot{e}_{y_{a}}=-\gamma e_{y_{a}}-\dfrac{\gamma}{4}\dfrac{\partial
a_{0,\delta}(x_{a})}{\partial x_{a}}b_{1,\delta}(x_{a})\quad e_{y_{a}}(0)=0$
(38)

in which, without loss of generality, we set
$\bar{y}_{a0}=a_{0,\delta}(x_{a}(0))/2$. Now, assume
$|x_{a}|_{\mathcal{A}_{\delta}}<r$ and use the definition of the Lipschitz
constant of $h(\cdot)$ to bound

$\left|\dfrac{\partial a_{0,\delta}(x)}{\partial x}\right|_{x=x_{a}}\leq
2\int_{0}^{1}\left|\left.\dfrac{\partial h(x)}{\partial
x}\right|_{x=x_{a}+\delta u(t)}\right|\,dt\leq 2L_{r}.$

Solve (38) and use (27) to obtain

$|{e}_{y_{a}}(t)|\leq\delta L_{r}^{2}\qquad\forall\,t\geq 0.$ (39)

As for

$\epsilon_{1}(\eta_{a},t)=\int_{N}^{t}\varepsilon_{\delta}({x}_{a},\bar{y}_{a},\tau)u(\tau)-b_{1,\delta}(x_{a})/2\,d\tau$
(40)

we add and subtract $h(x_{a})u$ and $a_{0,\delta}(x_{a})u/2$ inside the
integral and we use the triangle inequality to bound

$\displaystyle|\epsilon_{1}$ $\displaystyle(\eta_{a},t)|\leq$ (41)
$\displaystyle\int_{N}^{t}\left|\varepsilon_{\delta}({x}_{a},\bar{y}_{a},\tau)u(\tau)-b_{1,\delta}(x_{a})/2\right|\,d\tau\leq$
$\displaystyle\int_{N}^{t}|(h(x_{a}+\delta u)-h(x_{a}))u|\,d\tau$
$\displaystyle+\int_{N}^{t}|b_{1,\delta}(x_{a})/2|\,d\tau$
$\displaystyle+\int_{N}^{t}|h(x_{a})-a_{0,\delta}(x_{a})/2|\,d\tau$
$\displaystyle+\int_{N}^{t}|a_{0,\delta}(x_{a})/2-\bar{y}_{a}|\,d\tau.$

Then, in this order, for the terms appearing in (41) we use the definition of
the Lipschitz constant of $h(\cdot)$, (27) , the Mean Value Theorem, and (39)
to obtain

$\displaystyle|\epsilon_{1}(\eta_{a},t)|\leq\delta L_{r}(3+L_{r}).$ (42)

The benefit introduced by the presence of $\bar{y}$ is evident when (42) is
compared to (28). Indeed, in (42) only the Lipschitz constant of $h(\cdot)$
appears whereas (28) is bounded by the supremum of $|h(\cdot)|$ which is
radially unbounded. As for

$\epsilon_{2}(\eta_{a},t)=\int_{N}^{t}h(x_{a}+\delta
u(\tau))-\dfrac{a_{0,\delta}(x_{a})}{2}\,d\tau$ (43)

thanks to the Mean Value Theorem we note that there exists
$\bar{x}_{1}\in[x_{a}-\delta,\,x_{a}+\delta]$ such that
$h(\bar{x}_{1})=\dfrac{a_{0,\delta}(x_{a})}{2}$. Then

$\displaystyle|\epsilon_{2}(\eta_{a},t)|\leq\int_{N}^{t}\left|h(x_{a}+\delta
u(\tau))-h(\bar{x}_{1})\right|\,d\tau\leq L_{r}2\delta.$ (44)

Define $z(t)=\eta_{a}(t)-\gamma\epsilon(\eta_{a}(t),t)$, exploit the triangle
inequality, and use (42) and (44) to write

$\displaystyle\|\eta(t)$ $\displaystyle-\eta_{a}(t)\|=\|\eta(t)-\eta_{a}(t)\pm
z(t)\|$ (45) $\displaystyle\leq$
$\displaystyle\|\eta(t)-z(t)\|+\|z(t)-\eta_{a}(t)\|$ $\displaystyle\leq$
$\displaystyle\|\eta(t)-z(t)\|+\gamma\|\epsilon(\eta_{a}(t),t)\|$
$\displaystyle\leq$
$\displaystyle\|\eta(t)-z(t)\|+\gamma\bar{k}_{3}(L_{r},\delta).$

with $\bar{k}_{3}(L_{r},\delta):=\delta L_{r}\sqrt{4+(3+L_{r})^{2}}$. Denote
with $z_{1}$ and $z_{2}$ the first and second entry of $z$. In the next steps
we find a bound for $x-z_{1}$ and $\bar{y}-z_{2}$ which represent the first
and second entry of $\eta-z$. Let us compute

$\displaystyle x(t)-z_{1}(t)=$
$\displaystyle\,x(0)-z_{1}(0)+\int_{0}^{t}\dot{x}(\tau)-\dot{x}_{a}(\tau)d\tau$
(46)
$\displaystyle-\gamma\int_{0}^{t}\dfrac{\partial\epsilon_{1}(\eta,t)}{\partial\eta}\Big{|}_{\eta=\eta_{a}(\tau)}\dot{\eta}_{a}(\tau)d\tau$
$\displaystyle-\gamma\int_{0}^{t}\dfrac{\partial}{\partial\tau}\epsilon_{1}(\eta_{a}(\tau),\tau)d\tau.$

Then, we use (11a), (14a), and (40) to rewrite (46) as

$\displaystyle x(t)-z_{1}(t)=\,x(0)-z_{1}(0)$ (47)
$\displaystyle+\gamma\int_{0}^{t}\varepsilon_{\delta}(x(\tau),\bar{y}(\tau),\tau)u(\tau)-b_{1,\delta}(x_{a}(\tau))/2d\tau$
$\displaystyle-\gamma\int_{0}^{t}\dfrac{\partial\epsilon_{1}(\eta,t)}{\partial\eta}\Big{|}_{\eta=\eta_{a}(\tau)}\dot{\eta}_{a}(\tau)d\tau$
$\displaystyle-\gamma\int_{0}^{t}\varepsilon_{\delta}(x_{a}(\tau),\bar{y}_{a}(\tau),\tau)u(\tau)-b_{1,\delta}(x_{a}(\tau))/2d\tau.$

Add and subtract the term

$\varepsilon_{\delta}(z_{1}(\tau),\bar{y}(\tau),\tau)u(\tau)$

inside the first integral of the right member of (47) and rearrange this
latter as follows

$\displaystyle x(t)-$ $\displaystyle z_{1}(t)=\,x(0)-z_{1}(0)$
$\displaystyle+\gamma\int_{0}^{t}$
$\displaystyle\varepsilon_{\delta}(x(\tau),\bar{y}(\tau),\tau)u(\tau)-\varepsilon_{\delta}(z_{1}(\tau),\bar{y}(\tau),\tau)u(\tau)d\tau$
$\displaystyle+\gamma\int_{0}^{t}$
$\displaystyle\varepsilon_{\delta}(z_{1}(\tau),\bar{y}(\tau),\tau)u(\tau)-\varepsilon_{\delta}(x_{a}(\tau),\bar{y}_{a}(\tau),\tau)u(\tau)d\tau$
$\displaystyle-\gamma\int_{0}^{t}\left.\dfrac{\partial\epsilon_{1}(\eta,\tau)}{\partial\eta}\right|_{\eta=\eta_{a}(\tau)}\dot{\eta}_{a}(\tau)d\tau.$
(48)

In analogy with (33)-(35), let $\bar{c}\in(0,d)$ and define

$\gamma_{0}^{\star}=\dfrac{1}{2L_{r}}\left(\sqrt{9+4\bar{c}/\delta}-3\right).$
(49)

Then, in agreement with the definition of $z_{1}$, $|z_{1}(t)-x_{a}(t)|\leq d$
for all $\gamma\in(0,\gamma_{0}^{\star})$.

As consequence,

$|z_{1}(t)+\delta u(t)-x^{\star}|\leq r$ (50)

and so for all $x(t)$ satisfying $|x(t)-x_{a}(t)|\leq d$ we can exploit the
Lipschitz constant of $h(\cdot)$ to derive

$|\varepsilon_{\delta}(x,\bar{y},t)-\varepsilon_{\delta}(z_{1},\bar{y},t)|\leq
L_{r}|x-z_{1}|.$ (51)

On the other hand, the use of the triangle inequality, the definition of
$z_{2}$, and (44) lead to

$\displaystyle|\bar{y}-\bar{y}_{a}|=$ $\displaystyle\,|\bar{y}-\bar{y}_{a}\pm
z_{2}|\leq|\bar{y}-z_{2}|+|z_{2}-\bar{y}_{a}|$ $\displaystyle\leq$
$\displaystyle\,|\bar{y}-z_{2}|+\gamma|\epsilon_{2}(\eta_{a},t)|$
$\displaystyle\leq$ $\displaystyle\,|\bar{y}-z_{2}|+\gamma L_{r}2\delta$

which, with the use of the definitions of $\varepsilon_{\delta}$ and $z_{1}$,
jointly with (42), implies

$\displaystyle|\varepsilon_{\delta}(z_{1},\bar{y}$
$\displaystyle,t)-\varepsilon_{\delta}(x_{a},\bar{y}_{a},t)|$ (52)
$\displaystyle\leq$ $\displaystyle
L_{r}\gamma|\epsilon_{1}(\eta_{a},t)|+|\bar{y}(t)-\bar{y}_{a}(t)|$
$\displaystyle\leq$ $\displaystyle\gamma\delta L_{r}\left(2+L_{r}\
(3+L_{r})\right)+|\bar{y}-z_{2}|.$

Moreover, through the triangle inequality we get

$\displaystyle\Bigg{|}$
$\displaystyle\dfrac{\partial\epsilon_{1}(\eta,t)}{\partial\eta}\Bigg{|}_{\eta=\eta_{a}}\dot{\eta}_{a}\Bigg{|}\leq\,\gamma|a_{0,\delta}(x_{a})/2-\bar{y}_{a}|$
(53)
$\displaystyle+\gamma\left|\left.\dfrac{\partial\varepsilon_{\delta}(x,\bar{y}_{a},t)-b_{1,\delta}(x)/2}{\partial
x}\right|_{x=x_{a}}\right|\dfrac{|b_{1,\delta}(x_{a})|}{2}.$

Now, since $\partial\varepsilon_{\delta}(x,\bar{y},t)/\partial x=\partial
y_{\delta}(x)/\partial x$ and (5) coincides with (14a), we can exploit (36) to
bound

$\left|\left.\dfrac{\partial\varepsilon_{\delta}(x,\bar{y}_{a},t)-b_{1,\delta}(x)/2}{\partial
x}\right|_{x=x_{a}}\right|\leq 2L_{r}.$ (54)

Then, substitute (54) into (53) and use (27) and (39) to bound

$\Bigg{|}\dfrac{\partial\epsilon_{1}(\eta,t)}{\partial\eta}\Bigg{|}_{\eta=\eta_{a}}\dot{\eta}_{a}\Bigg{|}\leq\gamma\delta
L_{r}^{2}3$ (55)

and use (42), (51), (52) and (55) to bound (48) as

$\displaystyle|x$
$\displaystyle(t)-z_{1}(t)|\leq\,|x(0)-z_{1}(0)|+t\gamma^{2}\bar{k}_{3}(L_{r},\delta)$
(56)
$\displaystyle+\gamma\int_{0}^{t}L_{r}|x(\tau)-z_{1}(\tau)|+|\bar{y}(\tau)-z_{2}(\tau)|d\tau$

where $\bar{k}_{1}(L_{r},\delta):=\delta
L_{r}\left(2+L_{r}(3+L_{r})+3L_{r}\right)$. We now use the same conceptual
steps (46)-(56) to find a bound for $|\bar{y}(\tau)-z_{2}(\tau)|$. Compute

$\displaystyle\bar{y}(t)$ $\displaystyle-
z_{2}(t)=\bar{y}(0)-z_{2}(0)+\int_{0}^{t}\dot{y}(\tau)-\dot{z}_{2}(\tau)\,d\tau$
(57) $\displaystyle=$ $\displaystyle\,\bar{y}(0)-z_{2}(0)$
$\displaystyle+\gamma\int_{0}^{t}h(x(\tau)+\delta
u(\tau))-\dfrac{a_{0,\delta}(x_{a}(\tau))}{2}\,d\tau$
$\displaystyle-\gamma\int_{0}^{t}\bar{y}(\tau)-\bar{y}_{a}(\tau)\,d\tau$
$\displaystyle+\gamma\int_{0}^{t}\left.\dfrac{\partial\epsilon_{2}(\eta,\tau)}{\partial\eta}\right|_{\eta=\eta_{a}(\tau)}\dot{\eta}_{a}(\tau)\,d\tau$
$\displaystyle+\gamma\int_{0}^{t}\dfrac{\partial\epsilon_{2}(\eta_{a}(\tau),\tau)}{\partial\tau}\,d\tau.$

Add and subtract to the first and the second integral $h(x_{a}(\tau)+\delta
u(\tau))$ and $z_{2}(\tau)$ respectively, use (43) and rearrange (57) as
follows

$\displaystyle\bar{y}(t)$ $\displaystyle-z_{2}(t)=\bar{y}(0)-z_{2}(0)$ (58)
$\displaystyle+\gamma\int_{0}^{t}h(x(\tau)+\delta
u(\tau))-h(x_{a}(\tau)+\delta u(\tau))\,d\tau$
$\displaystyle-\gamma\int_{0}^{t}\bar{y}(\tau)-z_{2}(\tau)\,d\tau-\gamma^{2}\int_{0}^{t}\epsilon_{2}(\eta_{2}(\tau),\tau)\,d\tau$
$\displaystyle+\gamma\int_{0}^{t}\left.\dfrac{\partial\epsilon_{2}(\eta,\tau)}{\partial\eta}\right|_{\eta=\eta_{a}(\tau)}\dot{\eta}_{a}(\tau)\,d\tau.$

Use the defiition of the Lipschitz constant of $h$, the definition of $z_{1}$,
the triangle inequality, (39), and (42) to bound

$\displaystyle|h(x+$ $\displaystyle\delta u(\tau))-h(x_{a}+\delta
u(\tau))|\leq L_{r}|x-x_{a}|$ (59) $\displaystyle\leq
L_{r}(|x-z_{1}|+\gamma|\epsilon_{1}(\eta_{a},\tau)|)$ $\displaystyle\leq
L_{r}(|x-z_{1}|+\gamma\delta L_{r}(3+L_{r})).$

On the other hand, we use (4) into (43), the Lipschitz constant of $h(\cdot)$,
and (27) to write

$\displaystyle\Bigg{|}$
$\displaystyle\left.\dfrac{\partial\epsilon_{2}(\eta,\tau)}{\partial\eta}\right|_{\eta=\eta_{a}(\tau)}\dot{\eta}_{a}(\tau)\Bigg{|}\leq$
(60)
$\displaystyle\gamma\Bigg{|}\left.\dfrac{\partial\epsilon_{2}(\eta,\tau)}{\partial
x_{a}}\right|_{\eta=\eta_{a}(\tau)}\Bigg{|}\dfrac{|b_{1,\delta}(x_{a})|}{2}\leq\gamma
2L_{r}^{2}\delta^{2}.$

Use (44), (59), (60), to bound (58) as

$\displaystyle|\bar{y}(t)$ $\displaystyle-
z_{2}(t)|\leq|\bar{y}(0)-z_{2}(0)|+t\gamma^{2}\bar{k}_{2}(L_{r},\delta)$ (61)
$\displaystyle+\gamma\int_{0}^{t}L_{r}|x-z_{1}|+|\bar{y}(\tau)-z_{2}(\tau)|\,d\tau$

where $\bar{k}_{2}(L_{r},\delta):=\delta
L_{r}\left(2+L_{r}(2\delta+3+L_{r})\right)$.

Now define $f(t)=|x(t)-z_{1}(t)|+|\bar{y}(t)-z_{2}(t)|$ and
$A=\max\\{1,L_{r}\\}$, sum (56) and (61) to obtain

$\displaystyle f(t)\leq$
$\displaystyle\,f(0)+t\gamma^{2}\left(\bar{k}_{3}(L_{r},\delta)+\bar{k}_{2}(L_{r},\delta)\right)$
$\displaystyle+\gamma A\int_{0}^{t}f(\tau)\,d\tau,$

and use the specific Gronwall Lemma [[18], §1.3] to determine

$f(t)\leq\gamma\dfrac{\bar{k}_{3}(L_{r},\delta)+\bar{k}_{2}(L_{r},\delta)}{A}\left(e^{\gamma
At}-1\right)+f(0)e^{\gamma At}$

where, since $\epsilon(\eta_{a}(0),0)=0$, we have

$\displaystyle f(0)=$ $\displaystyle\,|x(0)-z_{1}(0)|+|\bar{y}(0)-z_{2}(0)|$
$\displaystyle=$
$\displaystyle\,|x(0)-x_{a}(0)+\gamma\epsilon_{1}(\eta_{a}(0),0)|$
$\displaystyle\,+|\bar{y}(0)-\bar{y}_{a}(0)+\gamma\epsilon_{2}(\eta_{a}(0),0)|$
$\displaystyle=$ $\displaystyle\,|x(0)-x_{a}(0)|+|\bar{y}(0)-\bar{y}_{a}(0)|.$

Use (45) and $\|\cdot\|_{2}\leq\|\cdot\|_{1}$ to bound

$\displaystyle\|\eta(t)-\eta_{a}(t)\|_{2}\leq$
$\displaystyle\,\|\eta(t)-z(t)\|_{1}+\gamma\bar{k}_{3}(L_{r},\delta)$
$\displaystyle=$ $\displaystyle\,f(t)+\gamma\bar{k}_{3}(L_{r},\delta).$

Finally, let $\bar{t}>0$ and define

$\displaystyle\gamma_{1}^{\star}$
$\displaystyle:=\dfrac{\bar{c}}{3}\dfrac{A}{\bar{k}_{1}(L_{r},\delta,\bar{c}/3)+\bar{k}_{2}(L_{r},\delta,\bar{c}/3)}\dfrac{1}{e^{A\bar{t}}-1}$
$\displaystyle\gamma_{2}^{\star}$
$\displaystyle:=\dfrac{\bar{c}}{3}\dfrac{1}{e^{A\bar{t}}}$
$\displaystyle\gamma_{3}^{\star}$
$\displaystyle:=\dfrac{\bar{c}}{3}\dfrac{1}{\bar{k}_{3}(L_{r},\delta)}$
$\displaystyle\gamma^{\star}$
$\displaystyle:=\min\\{\gamma_{0}^{\star},\gamma_{1}^{\star},\,\gamma_{2}^{\star},\,\gamma_{3}^{\star}\\}$

then for any $t\in[0,\bar{t}/\gamma]$, for any
$\|\eta(0)-\eta_{a}(0)\|_{1}\leq\gamma^{\star}$, and for any
$\gamma\in(0,\gamma^{\star})$

$\displaystyle\|\eta(t)-\eta_{a}(t)\|_{2}\leq$
$\displaystyle\,f(t)+\gamma\bar{k}_{3}(L_{r},\delta)$ $\displaystyle\leq$
$\displaystyle\,\gamma\dfrac{\bar{k}_{3}(L_{r},\delta)+\bar{k}_{2}(L_{r},\delta)}{A}\left(e^{\gamma
At}-1\right)$ $\displaystyle\,+f(0)e^{\gamma
At}+\gamma\bar{k}_{3}(L_{r},\delta)$ $\displaystyle\leq$
$\displaystyle\,\gamma_{1}^{\star}\dfrac{\bar{k}_{3}(L_{r},\delta)+\bar{k}_{2}(L_{r},\delta)}{A}\left(e^{\gamma
At}-1\right)$ $\displaystyle\,+\gamma_{2}^{\star}e^{\gamma
At}+\gamma_{3}^{\star}\bar{k}_{3}(L_{r},\delta)$ $\displaystyle\leq$
$\displaystyle\,\dfrac{\bar{c}}{3}\dfrac{\bar{k}_{3}(L_{r},\delta)+\bar{k}_{2}(L_{r},\delta)}{\bar{k}_{1}(L_{r},\delta,\bar{c}/3)+\bar{k}_{2}(L_{r},\delta,\bar{c}/3)}\dfrac{e^{\gamma
At}-1}{e^{A\bar{t}}-1}$ $\displaystyle\,+\dfrac{\bar{c}}{3}\dfrac{e^{\gamma
At}}{e^{A\bar{t}}}+\dfrac{\bar{c}}{3}\dfrac{\bar{k}_{3}(L_{r},\delta)}{\bar{k}_{3}(L_{r},\delta)}\leq\bar{c}.$

It is worth noting that $A$, $\bar{k}_{1}$, $\bar{k}_{2}$, and $\bar{k}_{3}$
are not dependent on the cost function magnitude $M_{r}$. To stress this point
let
$\gamma^{\star}(\bar{t},\bar{c},L_{r},\delta):=\min\\{\gamma_{0}^{\star},\gamma_{1}^{\star},\,\gamma_{2}^{\star},\,\gamma_{3}^{\star}\\}$.

Finally, we use the same arguments adopted at the end of the proof of
Proposition 1 to exploit the local exponential stability of the attractor
$\mbox{\rm graph}\left.\tau\right|_{{\cal A}_{\delta}}$ to extend the previous
bound for any $t\geq\bar{t}/\gamma$.

##  Appendix E Results on Averaging

Let $f(\cdot,\cdot)\,:\,\mathbb{R}^{n}\times[0,\infty)\to\mathbb{R}$, let
$\gamma>0$ and consider the system

$\dot{x}=\gamma f(x,t)\qquad x(0)=x_{0}.$ (62a) Let $T>0$ and assume $f$ to be
$T$-periodic in $t$, then the associated average system is defined as
$\dot{x}_{a}=\gamma f_{a}(x_{a})\qquad x_{a}(0)=x_{0}$ (62b) with
$f_{a}(x):=\dfrac{1}{T}\int_{0}^{T}f(x,t)\,dt.$ (62c)

The results provided in Lemma 3 are grounded on the following assumptions:

  * A1)

$f(x,t)$ is Lipschitz continuous and bounded on any compact set
$\mathcal{D}\subset\mathbb{R}^{n}$, uniformly in $t$, i.e. there exist
(finite) $L_{\mathcal{D}},\,M_{\mathcal{D}}>0$ such that

    * –

$\|f(x_{1},t)-f(x_{2},t)\|\leq L_{\mathcal{D}}\|x_{1}-x_{2}\|$ for all
$x_{1},x_{2}\in\mathcal{D}$ and for all $t\geq 0$

    * –

$\|f(x,t)\|\leq M_{\mathcal{D}}$ for all $x\in\mathcal{D}$ and for all $t\geq
0$

  * A2)

There exists $\mathcal{A}\subset\mathbb{R}^{n}$, and
$\beta(\cdot,\cdot)\in\mathcal{KL}$ such that

$\|x_{a}(t)\|_{\mathcal{A}}\leq\beta(\|x_{a}(0)\|_{\mathcal{A}},\gamma
t)\qquad\forall\,x_{a}(0)\in\mathbb{R}^{n}.$

  * A3)

Let A1) be verified for $\mathcal{D}=\mathbb{R}^{n}$. Let A2) and assume that
for any $\rho,d>0$ with $\rho>d$ there exists (finite) $\bar{t}^{\star}>0$
such that

$\beta(\rho,\bar{t})<\rho-d\qquad\forall\,\bar{t}>\bar{t}^{\star}.$

######  Lemma 3.

Let (62), then the following claims are true:

  1. C1)

Closeness. Let A1) be verified and pick $d>0$ and
$\underline{\mathcal{D}}\subset\mathbb{R}^{n}$. Define $\mathcal{D}$ as the
smallest set such that if $\|x\|_{\underline{\mathcal{D}}}\leq d$ then
$x\in\mathcal{D}$. Then for any $\bar{t}>0$ there exists
$\gamma^{\star}(d,\bar{t},M_{\mathcal{D}},L_{\mathcal{D}})>0$ such that for
any $\gamma\in(0,\gamma^{\star})$

$\|x(t)-x_{a}(t)\|\leq d$

for all $t\in[0,\,\bar{t}/\gamma]$ such that
$x(t),x_{a}(t)\in\underline{\mathcal{D}}$;

  2. C2)

Semi-Global Practical Stability. Let A1) and A2) be verified. Then, for any
$d,\rho>0$ there exists $\bar{t}>0$ and
$\gamma^{\star}(d,\bar{t},M_{\mathcal{D}},L_{\mathcal{D}})$ such that for any
$x_{0}\in\mathbb{R}^{n}\,:\,\|x_{0}\|_{\mathcal{A}}\leq\rho$ and for any
$\gamma\in(0,\gamma^{\star})$ the trajectories $t\mapsto x(t)$ are bounded and

$\|x(t)\|_{\mathcal{A}}\leq d\qquad\forall\,t\geq\dfrac{\bar{t}}{\gamma}.$

  3. C3)

Global Practical Stability. Let A3) be verified. Then, there exists
$\gamma^{\star}>0$ such that for any $\gamma\in(0,\gamma^{\star})$, and for
any $x_{0}\in\mathbb{R}^{n}$ the trajectories $t\mapsto x(t)$ are bounded and

$\limsup_{t\to\infty}\|x(t)\|_{\mathcal{A}}\leq d.$

###### Proof.

Let A1) be verified. As a starting point, we note that $\|f_{a}(x)\|\leq
M_{\mathcal{D}}$ for all $x\in\mathcal{D}$ thanks to the $t$-uniform
boundedness of $f(x,t)$ on $\mathcal{D}$ and thanks to the definition of
$f_{a}(x)$. Let

$\epsilon(x,t):=\int_{0}^{t}f(x,\tau)-f_{a}(x)\,d\tau,$

and define $N\in\mathbb{N}$ as the largest integer such that $NT\leq t$. Then,
thanks to the $T$-periodicity of $f(x,t)$ we have

$\epsilon(x,t)=\int_{0}^{t}f(x,\tau)-f_{a}(x)\,d\tau=\int_{NT}^{t}f(x,\tau)-f_{a}(x)\,d\tau,$

from which, using the triangle inequality,

$\epsilon(x,t)\leq\int_{NT}^{t}\|f(x,\tau)\|+\|f_{a}(x)\|\,d\tau\leq
2M_{\mathcal{D}}T.$

Exploit $z(t):=x_{a}(t)+\gamma\epsilon(x_{a}(t),t)$ to bound

$\displaystyle\|x(t)-x_{a}(t)\|$
$\displaystyle\,\leq\|x(t)-z(t)\|+\|z(t)-x_{a}(t)\|$
$\displaystyle\,\leq\|x(t)-z(t)\|+\gamma\|\epsilon(x_{a}(t),t)\|$
$\displaystyle\,\leq\|x(t)-z(t)\|+\gamma 2M_{\mathcal{D}}T.$

On the other hand

$\displaystyle x(t)-z(t)=$
$\displaystyle\,x(0)-z(0)+\int_{0}^{t}\dot{x}(\tau)-\dot{z}(\tau)\,d\tau$
$\displaystyle=$ $\displaystyle\,x(0)-z(0)$
$\displaystyle\,+\gamma\int_{0}^{t}f(x(\tau),\tau)\,d\tau$
$\displaystyle\,-\int_{0}^{t}\dot{x}_{a}(\tau)\,d\tau$
$\displaystyle\,-\gamma\int_{0}^{t}\left.\dfrac{\partial\epsilon(s,\tau)}{\partial
s}\right|_{s=x_{a}(\tau)}\dot{x}_{a}(\tau)\,d\tau$
$\displaystyle\,-\gamma\int_{0}^{t}\left.\dfrac{\partial\epsilon(x_{a}(\tau),s)}{\partial
s}\right|_{s=\tau}\,d\tau$ $\displaystyle=$ $\displaystyle\,x(0)-z(0)$
$\displaystyle\,+\gamma\int_{0}^{t}f(x(\tau),\tau)-f(x_{a}(\tau),\tau)\,d\tau$
$\displaystyle\,-\gamma^{2}\int_{0}^{t}\left.\dfrac{\partial\epsilon(s,\tau)}{\partial
s}\right|_{s=x_{a}(\tau)}f_{a}(x_{a}(\tau))\,d\tau$

Add and subtract within the first integral $f(z(\tau),\tau)$, let
$\bar{c}\in(0,\,d)$, and $\gamma_{1}^{\star}:=\bar{c}/(2M_{\mathcal{D}}T)$.
Define $L_{\mathcal{D}}$ as the Lipschitz constant of $f(x,t)$ for
$x\in\mathcal{D}$. Then, for any $\gamma\in(0,\gamma_{1}^{\star})$ and for any
$x_{a}(t)\in\underline{\mathcal{D}}$ we have $z(t)\in\mathcal{D}$ and

$\displaystyle\|f(z(\tau),\tau)-f(x_{a}(\tau),\tau)\|\leq$
$\displaystyle\,L_{\mathcal{D}}\|z(\tau)-x_{a}(\tau)\|$ $\displaystyle\leq$
$\displaystyle\,\gamma 2L_{\mathcal{D}}M_{\mathcal{D}}T$
$\displaystyle\|f(x(\tau),\tau)-f(z(\tau),\tau)\|\leq$
$\displaystyle\,L_{\mathcal{D}}\|x(\tau)-z(\tau)\|.$

Moreover, note that

$\displaystyle\left\|\dfrac{\partial\epsilon(s,t)}{\partial
s}\right\|\leq\int_{NT}^{t}\left\|\dfrac{\partial f(s,\tau)}{\partial
s}\right\|+\left\|\dfrac{\partial f_{a}(s)}{\partial s}\right\|\,d\tau\leq
2L_{\mathcal{D}}T,$

then

$\displaystyle\|x(t)-z(t)\|\leq$ $\displaystyle\,\|x(0)-z(0)\|$
$\displaystyle\,+\gamma\int_{0}^{t}\|f(x(\tau),\tau)-f(z(\tau),\tau)\|\,d\tau$
$\displaystyle\,+\gamma\int_{0}^{t}\|f(z(\tau),\tau)-f(x_{a}(\tau),\tau)\|\,d\tau$
$\displaystyle\,+\gamma^{2}\int_{0}^{t}\left\|\left.\dfrac{\partial\epsilon(s,\tau)}{\partial
s}\right|_{s=x_{a}(\tau)}\right\|\|f_{a}(x_{a}(\tau))\|\,d\tau$
$\displaystyle\leq$ $\displaystyle\,\|x(0)-z(0)\|$ $\displaystyle\,+\gamma
L_{\mathcal{D}}\int_{0}^{t}\|x(\tau)-z(\tau)\|\,d\tau$
$\displaystyle\,+\gamma^{2}3L_{\mathcal{D}}M_{\mathcal{D}}Tt.$

Apply the Gronwall Lemma to compute

$\displaystyle\|x(t)-z(t)\|\leq\|x(0)-z(0)\|+\gamma^{2}3L_{\mathcal{D}}M_{\mathcal{D}}Tt$
$\displaystyle+\gamma
L_{\mathcal{D}}\int_{0}^{t}(\|x(0)-z(0)\|+\gamma^{2}3L_{\mathcal{D}}M_{\mathcal{D}}T\tau)$
$\displaystyle\qquad\cdot e^{\gamma L_{\mathcal{D}}(t-\tau)}\,d\tau$
$\displaystyle=\|x(0)-z(0)\|e^{\gamma L_{\mathcal{D}}t}+\gamma
3M_{\mathcal{D}}T\left(e^{\gamma L_{\mathcal{D}}t}-1\right)$

from which

$\displaystyle\|x(t)-x_{a}(t)\|\leq$ $\displaystyle\,\|x(0)-z(0)\|e^{\gamma
L_{\mathcal{D}}t}$ $\displaystyle\,+\gamma 3M_{\mathcal{D}}T\left(e^{\gamma
L_{\mathcal{D}}t}-1\right)$ $\displaystyle\,+\gamma 2M_{\mathcal{D}}T.$

Then, let $\bar{t}>0$,

$\displaystyle\gamma_{2}^{\star}:=$
$\displaystyle\,\dfrac{d}{6M_{\mathcal{D}}Te^{L_{\mathcal{D}}\bar{t}}}$
$\displaystyle\gamma_{3}^{\star}:=$
$\displaystyle\,\dfrac{d}{9M_{\mathcal{D}}T\left(e^{L_{\mathcal{D}}\bar{t}}-1\right)}$
$\displaystyle\gamma_{4}^{\star}:=$
$\displaystyle\,\dfrac{d}{6M_{\mathcal{D}}T},$

and define
$\gamma^{\star}(d,\bar{t},M_{\mathcal{D}},L_{\mathcal{D}})=\min\\{1,\,\gamma_{1}^{\star},\gamma_{2}^{\star},\gamma_{3}^{\star},\gamma_{4}^{\star}\\}$.
Without loss of generality pick $x_{a}(0)=x_{0}$, then for any
$\gamma\in(0,\,\gamma^{\star})$

$\displaystyle\|x(t)-x_{a}(t)\|\leq$ $\displaystyle\,\gamma
2M_{\mathcal{D}}Te^{\gamma L_{\mathcal{D}}t}$ $\displaystyle\,+\gamma
3M_{\mathcal{D}}T\left(e^{\gamma L_{\mathcal{D}}t}-1\right)$
$\displaystyle\,+\gamma 2M_{\mathcal{D}}T$ $\displaystyle\leq$
$\displaystyle\,\gamma_{1}^{\star}2M_{\mathcal{D}}Te^{\gamma
L_{\mathcal{D}}t}$
$\displaystyle\,+\gamma_{2}^{\star}3M_{\mathcal{D}}T\left(e^{\gamma
L_{\mathcal{D}}t}-1\right)$
$\displaystyle\,+\gamma_{3}^{\star}2M_{\mathcal{D}}T$ $\displaystyle\leq$
$\displaystyle\,\dfrac{d}{3}\left(\dfrac{e^{\gamma
L_{\mathcal{D}}t}}{e^{L_{\mathcal{D}}\bar{t}}}+\dfrac{e^{\gamma
L_{\mathcal{D}}t}-1}{e^{L_{\mathcal{D}}\bar{t}}-1}+1\right)\leq d$

for all $t\in[0,\bar{t}/\gamma]$ as long as
$x(t),x_{a}(t)\in\underline{\mathcal{D}}$. This proves C1).

To prove C2) let

$\mathcal{D}:=\\{x\in\mathbb{R}^{n}\,:\,\|x\|_{\mathcal{A}}\leq\beta(\rho,0)+d\\}.$

Let $\underline{\rho}$ be the largest real such that

$\beta(\underline{\rho},0)<d/2.$

Define $\tilde{d}\in(0,\underline{\rho})$ and find $\bar{t}_{1}^{\star}>0$
such that

$\beta(\rho,\tau)+\tilde{d}\leq\underline{\rho}\qquad\forall\,\tau\geq\bar{t}_{1}^{\star}.$

Let $\bar{t}>\bar{t}_{1}^{\star}$, then for any
$\gamma\in(0,\,\gamma^{\star}(\tilde{d},\bar{t},M_{\mathcal{D}},L_{\mathcal{D}}))$
and any $x_{0}\in\mathbb{R}^{n}$ such that $\|x_{0}\|_{\mathcal{A}}\leq\rho$
we have

$\displaystyle\|x(t)\|_{\mathcal{A}}$
$\displaystyle\leq\|x(t)-x_{a}(t)\|+\|x_{a}(t)\|_{\mathcal{A}}$
$\displaystyle\leq\tilde{d}+\beta(\|x_{0}\|_{\mathcal{A}},\gamma t)$
$\displaystyle\forall\,t\in[0,\bar{t}/\gamma]$

and

$\|x(\bar{t}/\gamma)\|_{\mathcal{A}}\leq\underline{\rho}.$

Now divide the time axis into sub-intervals of the form

$I_{n}:=\left[n\dfrac{\bar{t}}{\gamma},\,(n+1)\dfrac{\bar{t}}{\gamma}\right)\quad
n\in{\mathbb{N}}$

and, with $x_{a}(t,x_{a0})$ the trajectory of the average system at time $t$
with initial condition $x_{a0}$ at time $t=0$, let

$x_{n}(t):=x_{a}(t-n{\bar{t}}/{\gamma},x(n{\bar{t}}/{\gamma}))\qquad\forall\,t\in
I_{n}.$

The same arguments used in the first part of the proof show that

$\|x(t)-x_{n}(t)\|\leq\tilde{d}\qquad\forall\,t\in
I_{n},\,\forall\,n\in\mathbb{N}$

Moreover,

$\displaystyle\|x(t)\|_{\mathcal{A}}$
$\displaystyle\leq\|x_{n}(t)\|_{\mathcal{A}}+\|x(t)-x_{n}(t)\|$
$\displaystyle\leq\beta(\|x(n\bar{t}/\gamma)\|_{\mathcal{A}},\gamma
t-n\bar{t})+\tilde{d}$
$\displaystyle\leq\beta(\|x(n\bar{t}/\gamma)\|_{\mathcal{A}},0)+\tilde{d}$
$\displaystyle\forall\,t\in I_{n},\,\forall\,n\in\mathbb{N}$

and

$\displaystyle\|x((n+1)\bar{t}/\gamma)\|_{\mathcal{A}}$
$\displaystyle\leq\beta(\|x(n\bar{t}/\gamma)\|_{\mathcal{A}},\bar{t})+\tilde{d}$
$\displaystyle\forall\,n\in\mathbb{N}.$

Finally, since for $n=1$

$\displaystyle\|x(t)\|_{\mathcal{A}}$
$\displaystyle\leq\beta(\|x(\bar{t}/\gamma)\|_{\mathcal{A}},0)+\tilde{d}$
$\displaystyle\leq\beta(\underline{\rho},0)+\tilde{d}$ $\displaystyle\leq d$
$\displaystyle\forall\,t\in I_{1}$

and

$\displaystyle\|x(2\bar{t}/\gamma)\|_{\mathcal{A}}$
$\displaystyle\leq\beta(\|x(\bar{t}/\gamma)\|_{\mathcal{A}},\bar{t})+\tilde{d}$
$\displaystyle\leq\beta(\underline{\rho},\bar{t})+\tilde{d}$
$\displaystyle\leq\beta(\rho,\bar{t})+\tilde{d}$
$\displaystyle\leq\underline{\rho}$

the arguments can be iterated for any $n>1$. This proves C2).

To prove C3), define $\underline{\rho}$, $\tilde{d}$ and $\bar{t}_{1}^{\star}$
by following the same steps used before to prove C2). Using A3) define
$\bar{t}_{2}^{\star}>0$ such that

$\beta(r,\bar{t})<r-\underline{\rho}\qquad\forall\,r>\underline{\rho},\,\forall\,\bar{t}>\bar{t}_{2}^{\star}.$

Define

$\bar{t}>\max\\{\bar{t}_{1}^{\star},\,\bar{t}_{2}^{\star}\\}$

and pick
$\gamma\in(0,\,\gamma^{\star}(\tilde{d},\bar{t},M_{\mathbb{R}^{n}},L_{\mathbb{R}^{n}}))$.
As done to prove C2) divide the time axis into sub-intervals $I_{n}$ and on
each interval define $x_{n}(t)$. As consequence, for any $n\in\mathbb{N}$ and
any $x_{0}\in\mathbb{R}^{n}$, we have that $x_{a0}=x_{0}$ implies

$\|x(t)-x_{n}(t)\|\leq\tilde{d}\qquad\forall\,t\in I_{n}.$

Then, as far as $\|x((n\bar{t})/\gamma)\|_{\mathcal{A}}>\underline{\rho}$

$\displaystyle\|x((n+1)\bar{t}/\gamma)\|_{\mathcal{A}}\leq$
$\displaystyle\,\|x_{n}((n+1)\bar{t}/\gamma)\|_{\mathcal{A}}$
$\displaystyle\,+\|x((n+1)\bar{t}/\gamma)-x_{n}((n+1)\bar{t}/\gamma)\|$
$\displaystyle\leq$
$\displaystyle\,\|x_{a}(\bar{t}/\gamma,x(n\bar{t}/\gamma)\|_{\mathcal{A}}+\tilde{d}$
$\displaystyle\leq$
$\displaystyle\,\beta(\|x(n\bar{t}/\gamma)\|_{\mathcal{A}},\bar{t}/\gamma)+\tilde{d}$
$\displaystyle\leq$
$\displaystyle\,\beta(\|x(n\bar{t}/\gamma)\|_{\mathcal{A}},\bar{t}_{2}^{\star}/\gamma)+\tilde{d}$
$\displaystyle<$ $\displaystyle\,\|x(n\bar{t}/\gamma)\|_{\mathcal{A}}.$

Then we can conclude that as far as
$\|x(n\bar{t}/\gamma)\|_{\mathcal{A}}>\underline{\rho}$ the sequence

$\\{\|x(n\bar{t}/\gamma)\|_{\mathcal{A}}\\}_{n\in\mathbb{N}}$

is strictly decreasing and

$\limsup_{n\to\infty}\|x(n\bar{t}/\gamma)\|_{\mathcal{A}}\leq\underline{\rho}.$

The proof of C3) can be then obtained by following the same final steps used
to prove C2).