# A fast continuous time approach with time scaling for nonsmooth convex
optimization

Radu Ioan Boţ 111Faculty of Mathematics, University of Vienna, Oskar-
Morgenstern-Platz 1, 1090 Vienna, Austria, email<EMAIL_ADDRESS>Research partially supported by FWF (Austrian Science Fund), project P
34922-N. Mikhail A. Karapetyants 222Faculty of Mathematics, University of
Vienna, Oskar-Morgenstern-Platz 1, 1090 Vienna, Austria, email:
<EMAIL_ADDRESS>Research supported by the Doctoral
Programme _Vienna Graduate School on Computational Optimization (VGSCO)_ which
is funded by FWF (Austrian Science Fund), project W 1260.

###### Abstract

In a Hilbert setting we study the convergence properties of a second order in
time dynamical system combining viscous and Hessian-driven damping with time
scaling in relation with the minimization of a nonsmooth and convex function.
The system is formulated in terms of the gradient of the Moreau envelope of
the objective function with time-dependent parameter. We show fast convergence
rates for the Moreau envelope and its gradient along the trajectory, and also
for the velocity of the system. From here we derive fast convergence rates for
the objective function along a path which is the image of the trajectory of
the system through the proximal operator of the first. Moreover, we prove the
weak convergence of the trajectory of the system to a global minimizer of the
objective function. Finally, we provide multiple numerical examples which
illustrate the theoretical results.

Key words: Nonsmooth convex optimization; Damped inertial dynamics; Hessian-
driven damping; Time scaling; Moreau envelope; Proximal operator

AMS subject classification: 37N40, 46N10, 49M99, 65K05, 65K10, 90C25

##  1 Introduction

Let $H$ be a real Hilbert space endowed with the scalar product
$\langle\cdot,\cdot\rangle$ and norm $\|x\|=\sqrt{\langle x,x\rangle}$ for
$x\in H$. In connection with the minimization problem

$\min_{x\in H}\Phi(x),$

we will study the asymptotic behaviour of the second order in time evolution
equation

$\ddot{x}(t)+\frac{\alpha}{t}\dot{x}(t)+\beta(t)\frac{d}{dt}\nabla\Phi_{\lambda(t)}(x(t))+b(t)\nabla\Phi_{\lambda(t)}(x(t))=0,$
(1)

with initial conditions $x(t_{0})=x_{0}\in H$, $\dot{x}(t_{0})=u_{0}\in H$,
where $\alpha\geq 1$, $t_{0}>0$, and
$\beta:[t_{0},+\infty)\longrightarrow[0,+\infty)$ and
$b,\lambda:[t_{0},+\infty)\longrightarrow(0,+\infty)$ are differentiable
functions.

We assume that
$\Phi:H\longrightarrow\overline{\mathbb{R}}=\mathbb{R}\cup\\{\pm\infty\\}$ is
a proper, convex and lower semicontinuous function and denote by
$\Phi_{\lambda}:H\longrightarrow\mathbb{R}$ its Moreau envelope of parameter
$\lambda>0$. In addition, we assume that $\operatorname*{argmin}\Phi$, the set
of global minimizers of $\Phi$, is not empty and denote by $\Phi^{*}$ the
optimal objective value of $\Phi$.

Our aim is to derive rates of convergence for the Moreau envelope of the
objective function and the objective function itself to $\Phi^{*}$, as well as
for the gradient of the Moreau envelope of the objective function and the
velocity of the trajectory to zero in terms of the Moreau parameter function
$\lambda$ and the time scaling function $b$. In addition, we will provide a
setting which also guarantees the weak convergence of the trajectory of the
dynamical system to a minimizer of $\Phi$. The theoretical results will be
illustrated by multiple numerical experiments.

###  1.1 Historical remarks

Inertial dynamics were introduced by Polyak in [22] in form of the so-called
heavy ball with friction method

$\ddot{x}(t)+\alpha\dot{x}(t)+\nabla\Phi(x(t))=0,$

with fixed viscous coefficient $\alpha>0$, in order to accelerate the gradient
method for the minimization of a continuous differentiable function
$\Phi:H\to\mathbb{R}$. This system was later studied by Alvarez-Attouch [1, 2]
and by Attouch-Goudou-Redont [10]. For a convex function $\Phi$ an asymptotic
convergence rate of $\Phi(x(t))$ to $\Phi^{*}$ of order
$O\left(\frac{1}{t}\right)$ as $t\to+\infty$, as well as an improvement for a
strongly convex function $\Phi$ to an exponential rate of convergence were
proved. The weak convergence of the trajectories to a minimizer of $\Phi$ was
also established.

A major step to obtain faster asymptotic convergence in the convex regime was
done by Su-Boyd-Candes [23], by considering in the second order dynamical
system an asymptotic vanishing damping coefficient

$\ddot{x}(t)+\frac{\alpha}{t}\dot{x}(t)+\nabla\Phi(x(t))=0,$ (2)

for $t\geq t_{0}$ and $\alpha\geq 3$. Second order dynamical systems with
variable and vanishing damping coefficients for optimization were studied, for
instance, in [16, 17, 18]. The system (2) corresponds to a continuous version
of Nesterov’s accelerated gradient method [20]. For the function values, rates
of convergence of

$\Phi(x(t))-\Phi^{*}=O\left(\frac{1}{t^{2}}\right)\ \text{as}\ t\to+\infty$

were obtained. For $\alpha>3$, in [9] it was shown that the trajectory of (2)
converges weakly to an element of $\operatorname*{argmin}\Phi$, and in [13,
19] the asymptotic convergence rate of the function values was improved to
$o\left(\frac{1}{t^{2}}\right)$ as $t\to+\infty$.

The following system which combines asymptotic vanishing damping with Hessian-
driven damping was proposed by Attouch-Peypouquet-Redont in [14]

$\ddot{x}(t)+\frac{\alpha}{t}\dot{x}(t)+\beta\nabla^{2}\Phi(x(t))\dot{x}(t)+\nabla\Phi(x(t))=0$
(3)

for $t\geq t_{0}$, where $\Phi:H\longrightarrow\mathbb{R}$ twice continuously
differentiable and convex, $\alpha\geq 3$ and $\beta\geq 0$. Hessian-driven
damping has a natural link with Newton’s method and gives rise to dynamical
inertial Newton systems [3]. The system (3) preserves the convergence
properties of (2), while having for $\beta>0$ other important features,
namely,

$\lim_{t\rightarrow+\infty}\|\nabla\Phi(x(t))\|=0\ \mbox{and}\
\int_{t_{0}}^{+\infty}t^{2}\|\nabla\Phi(x(t))\|^{2}dt<+\infty.$

In addition, possible oscillations exhibited by the solutions of (2) are
neutralized by (3).

###  1.2 Time scaling

Time scaling of the dynamical system (2) was used in order to accelerate the
rate of convergence of the values of the function $\Phi$ along the trajectory.
The system (2) becomes through time scaling a dynamical system of the form

$\ddot{x}(t)+\frac{\alpha}{t}\dot{x}(t)+b(t)\nabla\Phi(x(t))=0,$ (4)

where $\alpha\geq 3$ and $b:[t_{0},+\infty)\longrightarrow(0,+\infty)$ is a
continuous scalar function, as it was introduced and studied by Attouch-
Chbani-Riahi in [8]. For (4) it was shown that

$\Phi(x(t))-\Phi^{*}=O\left(\frac{1}{t^{2}b(t)}\right)\text{ as }t\to+\infty,$

a convergence rate which can be improved to
$o\left(\frac{1}{t^{2}b(t)}\right)$ as $t\to+\infty$, if $\alpha>3$.

In [7] (see also [5]) the dynamical system

$\ddot{x}(t)+\frac{\alpha}{t}\dot{x}(t)+\beta(t)\nabla^{2}\Phi(x(t))\dot{x}(t)+b(t)\nabla\Phi(x(t))=0,$
(5)

which combines viscous and Hessian-driven damping with time scaling, where
$\alpha\geq 1$ and $\beta,b:[t_{0},+\infty)\longrightarrow(0,+\infty)$ are
functions with appropriate differentiability properties, was investigated. A
quite general setting formulated in terms of the dynamical system parameter
functions was identified in which the properties of (5) concerning the
convergence of the function values are preserved, while the gradient of $\Phi$
strongly converges along the trajectory to zero and the trajectory converges
weakly to a minimizer of the objective function. In [7] a numerical algorithm
obtained via time discretization of (5) was also introduced, exhibiting
analogous convergence properties to the dynamical system.

###  1.3 Nonsmooth optimization

The Moreau envelope of a proper, convex and lower semicontinuous function
$\Phi:H\to\overline{\mathbb{R}}$ has played a significant role in the
literature when designing continuous-time approaches and numerical algorithms
for the minimization of $\Phi$. This is defined as

$\Phi_{\lambda}:H\to\mathbb{R},\quad\Phi_{\lambda}(x)\ =\ \inf_{y\in
H}\left\\{\Phi(y)+\frac{1}{2\lambda}\|x-y\|^{2}\right\\},$

where $\lambda>0$ is called the parameter of the Moreau envelope (see, for
instance, [15]). For every $\lambda>0$, the functions $\Phi$ and
$\Phi_{\lambda}$ share the same optimal objective value and the same set of
minimizers. In addition, $\Phi_{\lambda}$ is convex and continuously
differentiable with

$\displaystyle\nabla\Phi_{\lambda}(x)=\frac{1}{\lambda}(x-\operatorname*{prox}\nolimits_{\lambda\Phi}(x))\quad\forall
x\in H,$ (6)

and $\nabla\Phi_{\lambda}$ is $\frac{1}{\lambda}$-Lipschitz continuous. Here,

$\operatorname*{prox}\nolimits_{\lambda\Phi}:H\to
H,\quad\operatorname*{prox}\nolimits_{\lambda\Phi}(x)=\operatorname*{argmin}_{y\in
H}\left\\{\Phi(y)+\frac{1}{2\lambda}\|x-y\|^{2}\right\\},$

denotes the proximal operator of $\Phi$ of parameter $\lambda$. For every
$x\in H$ and $\lambda,\mu>0$ we have

$\|\operatorname*{prox}\nolimits_{\lambda\Phi}(x)-\operatorname*{prox}\nolimits_{\mu\Phi}(x)\|\leq|\lambda-\mu|\|\nabla\Phi_{\lambda}(x)\|.$
(7)

On the other hand, for every $x\in H$, the function
$\lambda\in(0,+\infty)\to\Phi_{\lambda}(x)$ is nonincreasing and
differentiable, namely,

$\frac{d}{d\lambda}\Phi_{\lambda}(x)=-\frac{1}{2}\|\nabla\Phi_{\lambda}(x)\|^{2}\quad\forall\lambda>0.$

Attouch-Cabot considered in [6] (see also [12] for a more general approach for
monotone inclusions) in connection with the minimization of the proper, convex
and lower semicontinuous function $\Phi:H\to\overline{\mathbb{R}}$ the
following second order differential equation

$\ddot{x}(t)+\frac{\alpha}{t}\dot{x}(t)+\nabla\Phi_{\lambda(t)}(x(t))=0$ (8)

for $t\geq t_{0}$, where $\alpha\geq 1$ and
$\lambda:[t_{0},+\infty)\longrightarrow(0,+\infty)$ is continuously
differentiable and non-decreasing. Convergence rates for the values of the
Moreau envelope as well as for the velocity of the system were obtained

$\Phi_{\lambda(t)}(x(t))-\Phi^{*}=o\left(\frac{1}{t^{2}}\right)\text{ and
}\|\dot{x}(t)\|=o\left(\frac{1}{t}\right)\text{ as }t\to+\infty,$

from where convergence rates for the $\Phi$ along $x(t)$ were deduced

$\Phi\big{(}\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t))\big{)}-\Phi^{*}=o\left(\frac{1}{t^{2}}\right)\text{
and
}\|\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t))-x(t)\|=o\left(\frac{\sqrt{\lambda(t)}}{t}\right)\text{
as }t\to+\infty.$

In addition, the weak convergence of the trajectories $x(t)$ to a minimizer of
$\Phi$ as $t\to+\infty$ was established.

Attouch-László considered in [11] in the same context the dynamical system

$\ddot{x}(t)+\frac{\alpha}{t}\dot{x}(t)+\beta\frac{d}{dt}\nabla\Phi_{\lambda(t)}(x(t))+\nabla\Phi_{\lambda(t)}(x(t))=0$
(9)

where $\alpha>1$ and $\beta>0$, and the term
$\frac{d}{dt}\nabla\Phi_{\lambda(t)}(x(t))$ is inspired by the Hessian driven
damping. It was shown that for $\lambda(t)=\lambda t^{2}$, where $\lambda>0$,
the system (9) inherits all major convergence properties of (8) and, in
addition, the following convergence rates for the gradient of the Moreau
envelope of parameter $\lambda(t)$ and its time derivative along $x(t)$ were
established

$\|\nabla\Phi_{\lambda(t)}(x(t))\|=o\left(\frac{1}{t^{2}}\right)\text{ and
}\left\|\frac{d}{dt}\nabla\Phi_{\lambda(t)}(x(t))\right\|=o\left(\frac{1}{t^{2}}\right)\text{
as }t\to+\infty.$

###  1.4 Our contribution

In this paper, we derive a setting formulated in terms of $\alpha\geq 1$ and
the parameter functions $\beta$, $b$ and $\lambda$ of the dynamical system (1)
associated with the minimization of the proper, convex and lower
semicontinuous function $\Phi:H\to\overline{\mathbb{R}}$, which allow us to
prove

  * •

convergence rates for the Moreau envelope, its gradient and the velocity of
the trajectory

$\Phi_{\lambda(t)}(x(t))-\Phi^{*}=o\left(\frac{1}{t^{2}b(t)}\right),\
\|\nabla\Phi_{\lambda(t)}(x(t))\|\ =\
o\left(\frac{1}{t\sqrt{b(t)\lambda(t)}}\right)\text{ and
}\|\dot{x}(t)\|=o\left(\frac{1}{t}\right)$

as $t\to+\infty$, respectively;

  * •

convergence rates for the objective function

$\Phi\big{(}\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t))\big{)}-\Phi^{*}=o\left(\frac{1}{t^{2}b(t)}\right)\text{
and }\|\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t))-x(t)\|\ =\
o\left(\frac{\sqrt{\lambda(t)}}{t\sqrt{b(t)}}\right)$

as $t\to+\infty$;

  * •

the weak convergence of the trajectory $x(t)$ to a minimizer of $\Phi$ as
$t\to+\infty$.

In addition, we provide a particular formulation of the derived general
setting for the case when the parameter functions are chosen to be polynomials
and illustrate the influence of the latter on the convergence behaviour of the
dynamical system by multiple numerical experiments.

###  1.5 Existence and uniqueness of strong global solution

This section is devoted to the topic of existence and uniqueness of a strong
global solution of the system of our interest. To this aim we will rewrite (1)
as a system of the first order in time equations in the product space $H\times
H$.

We assume first that $\beta:[t_{0},+\infty)\longrightarrow[0,+\infty)$ is
twice continuously differentiable with $\beta(t)>0$ for every $t\geq t_{0}$.
We integrate (1) from $t_{0}$ to $t$ to obtain

$\displaystyle\dot{x}(t)+\beta(t)\nabla\Phi_{\lambda(t)}(x(t))+\int_{t_{0}}^{t}\left(\frac{\alpha}{s}\dot{x}(s)+b(s)\nabla\Phi_{\lambda(s)}(x(s))\right)ds-\
\int_{t_{0}}^{t}\nabla\Phi_{\lambda(s)}(x(s))\dot{\beta}(s)ds$
$\displaystyle-\left(\dot{x}(t_{0})+\beta(t_{0})\nabla\Phi_{\lambda(t_{0})}(x(t_{0}))\right)$
$\displaystyle=0.$

We denote
$z(t):=\int_{t_{0}}^{t}\left(\frac{\alpha}{s}\dot{x}(s)+\left(b(s)-\dot{\beta}(s)\right)\nabla\Phi_{\lambda(s)}(x(s))\right)ds-\big{(}u_{0}+\beta(t_{0})\nabla\Phi_{\lambda(t_{0})}(x_{0}))\big{)}$
for every $t\geq t_{0}$. Since
$\dot{z}(t)=\frac{\alpha}{t}\dot{x}(t)+\left(b(t)-\dot{\beta}(t)\right)\nabla\Phi_{\lambda(t)}(x(t))$
we notice, that (1) is equivalent to

$\begin{cases}&\dot{x}(t)+\beta(t)\nabla\Phi_{\lambda(t)}(x(t))+z(t)=0,\\\
&\dot{z}(t)-\frac{\alpha}{t}\dot{x}(t)-\left(b(t)-\dot{\beta}(t)\right)\nabla\Phi_{\lambda(t)}(x(t))=0,\\\
&x(t_{0})=x_{0},\
z(t_{0})=-\left(u_{0}+\beta(t_{0})\nabla\Phi_{\lambda(t_{0})}(x_{0})\right).\end{cases}$

After multiplying the first line by $b(t)-\dot{\beta}(t)$ and the second one
by $\beta(t)$, by summing them we get rid of the gradient of the Moreau
envelope in the second equation

$\begin{cases}&\dot{x}(t)+\beta(t)\nabla\Phi_{\lambda(t)}(x(t))+z(t)=0,\\\
&\beta(t)\dot{z}(t)+\left(b(t)-\dot{\beta}(t)-\frac{\alpha\beta(t)}{t}\right)\dot{x}(t)+\left(b(t)-\dot{\beta}(t)\right)z(t)=0,\\\
&x(t_{0})=x_{0},\
z(t_{0})=-\left(u_{0}+\beta(t_{0})\nabla\Phi_{\lambda(t_{0})}(x_{0})\right).\end{cases}$

We denote
$y(t)=\beta(t)z(t)+\left(b(t)-\dot{\beta}(t)-\frac{\alpha\beta(t)}{t}\right)x(t)$,
and, after simplification, we obtain for the dynamical system the following
equivalent formulation

$\begin{cases}&\dot{x}(t)+\beta(t)\nabla\Phi_{\lambda(t)}(x(t))+\left(\frac{\dot{\beta}(t)-b(t)}{\beta(t)}+\frac{\alpha}{t}\right)x(t)+\frac{1}{\beta(t)}y(t)=0,\\\
&\dot{y}(t)+\left(\ddot{\beta}(t)+\frac{3b(t)\dot{\beta}(t)-2\dot{\beta}^{2}(t)-b^{2}(t)}{\beta(t)}+\frac{\alpha}{t}\left(b(t)-\dot{\beta}(t)-\frac{\beta(t)}{t}\right)-\dot{b}(t)\right)x(t)+\frac{b(t)-2\dot{\beta}(t)}{\beta(t)}y(t)=0,\\\
&x(t_{0})=x_{0},\
y(t_{0})=-\beta(t_{0})\left(u_{0}+\beta(t_{0})\nabla\Phi_{\lambda(t_{0})}(x_{0})\right)+\left(b(t_{0})-\dot{\beta}(t_{0})-\frac{\alpha\beta(t_{0})}{t_{0}}\right)x_{0}.\end{cases}$

In case $\beta(t)=0$ for every $t\geq t_{0}$, (1) can be equivalently written
as

$\begin{cases}&\dot{x}(t)-y(t)=0,\\\
&\dot{y}(t)+\frac{\alpha}{t}y(t)+b(t)\nabla\Phi_{\lambda(t)}(x(t))=0,\\\
&x(t_{0})=x_{0},\ y(t_{0})=u_{0}.\end{cases}$

Based on the two reformulation of the dynamical system (1) we can formulate
the following existence and uniqueness result, which is a consequence of the
Cauchy-Lipschitz theorem for strong global solutions. The result can be proved
in the lines of the proofs of Theorem 1 in [11] or of Theorem 1.1 in [14] with
some small adjustments.

######  Theorem 1.

Suppose that $\beta:[t_{0},+\infty)\longrightarrow[0,+\infty)$ is twice
continuously differentiable such that either $\beta(t)>0$ for every $t\geq
t_{0}$ or $\beta(t)=0$ for every $t\geq t_{0}$, and that there exists
$\lambda_{0}>0$ such that $\lambda(t)\geq\lambda_{0}$ for all $t\geq t_{0}$.
Then for every $(x_{0},u_{0})\in H\times H$ there exists a unique strong
global solution $x:[t_{0},+\infty)\mapsto H$ of the continuous dynamics (1)
which satisfies the Cauchy initial conditions $x(t_{0})=x_{0}$ and
$\dot{x}(t_{0})=u_{0}$.

##  2 Energy function and rates of convergence for function values

In this section we will define for the dynamical system (1) an energy function
and investigate its dissipativity properties. These will play a crucial role
in the derivation of rates of convergence for the Moreau envelope of $\Phi$
and the objective function itself.

To shorten the calculations, we introduce the auxiliary function (see also
[7])

$w:[t_{0},+\infty)\to\mathbb{R},\quad w(t)\ =\
b(t)-\dot{\beta}(t)-\frac{\beta(t)}{t}.$

For $z\in\operatorname*{argmin}\Phi$ and

$0\leq c\leq\alpha-1,$ (10)

consider the energy function $E_{c}:[t_{0},+\infty)\rightarrow[0,+\infty)$,

$\displaystyle E_{c}(t)\ =\ $
$\displaystyle\big{(}t^{2}w(t)+(\alpha-1-c)t\beta(t)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})+\frac{1}{2}\left\|c(x(t)-z)+t\dot{x}(t)+t\beta(t)\nabla\Phi_{\lambda(t)}(x(t))\right\|^{2}$
$\displaystyle+\ \frac{c(\alpha-1-c)}{2}\|x(t)-z\|^{2}.$

In the following theorem we formulate sufficient conditions that guarantee the
decay of the energy of the the dynamical system (1) and discuss some of its
consequences.

######  Theorem 2.

Suppose that $\alpha\geq 1$, $\lambda$ is nondecreasing on $[t_{0},+\infty)$
and the following conditions

$b(t)>\dot{\beta}(t)+\frac{\beta(t)}{t}\ \mbox{for every}\ t\geq t_{0}$ (11)

and

$(\alpha-3)w(t)-t\dot{w}(t)\geq 0\ \mbox{for every}\ t\geq t_{0}$ (12)

are satisfied. Then, for a solution $x:[t_{0},+\infty)\rightarrow H$ to (1),
the following statements are true:

  1. (i)

$\dot{E}_{c}(t)\leq 0$ for every $t\geq t_{0}$;

  2. (ii)

$\Phi_{\lambda(t)}(x(t))-\Phi^{*}\ \leq\
\frac{E_{\alpha-1}(t_{0})}{t^{2}w(t)}$ for every $t\geq t_{0}$;

  3. (iii)

$\int_{t_{0}}^{+\infty}\Big{(}t^{2}w(t)\frac{\dot{\lambda}(t)}{2}+t^{2}\beta(t)w(t)\Big{)}\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}dt\
<\ +\infty$;

  4. (iv)

$\int_{t_{0}}^{+\infty}\big{(}(\alpha-3)tw(t)-t^{2}\dot{w}(t)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})dt\
<\ +\infty$.

Assuming moreover that $\alpha>1$ and that

$\mbox{there exists}\ \varepsilon\in(0,\alpha-1)\ \mbox{such that}\
(\alpha-3)w(t)-t\dot{w}(t)\ \geq\ \varepsilon b(t)\quad\forall t\geq t_{0},$
(13)

it holds

  5. (v)

$\int_{t_{0}}^{+\infty}t\|\dot{x}(t)\|^{2}dt\ <\ +\infty$;

  6. (vi)

the trajectory $x$ is bounded and

  7. (vii)

$\int_{t_{0}}^{+\infty}tb(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})dt\ <\ +\infty$.

###### Proof.

For every $t\geq t_{0}$ we obtain

$\displaystyle\dot{E}_{c}(t)\ =\ $
$\displaystyle\big{(}2tw(t)+t^{2}\dot{w}(t)+\beta(t)(\alpha-1-c)+(\alpha-1-c)t\dot{\beta}(t)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})$
$\displaystyle+\
\big{(}t^{2}w(t)+\beta(t)t(\alpha-1-c)\big{)}\Big{(}\langle\nabla\Phi_{\lambda(t)}(x(t)),\dot{x}(t)\rangle-\frac{\dot{\lambda}(t)}{2}\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}\Big{)}$
$\displaystyle+\
\Big{\langle}c(x(t)-z)+t\dot{x}(t)+t\beta(t)\nabla\Phi_{\lambda(t)}(x(t)),(c+1)\dot{x}(t)+t\ddot{x}(t)+t\beta(t)\frac{d}{dt}\left(\nabla\Phi_{\lambda(t)}(x(t))\right)$
$\displaystyle+\
(\beta(t)+t\dot{\beta}(t))\nabla\Phi_{\lambda(t)}(x(t))\Big{\rangle}\ +\
c(\alpha-1-c)\langle x(t)-z,\dot{x}(t)\rangle,$

where we used that

$\frac{d}{dt}\left(\Phi_{\lambda(t)}(x(t))-\Phi^{*}\right)=\big{\langle}\nabla\Phi_{\lambda(t)}(x(t)),\dot{x}(t)\big{\rangle}-\frac{\dot{\lambda}(t)}{2}\|\nabla\Phi_{\lambda(t)}x(t)\|^{2}.$
(14)

Using (1) to replace $\ddot{x}(t)$, we may write the third summand in the
formulation of $\dot{E}_{c}(t)$ for every $t\geq t_{0}$ as

$\displaystyle\Big{\langle}c(x(t)-z)+t\dot{x}(t)+t\beta(t)\nabla\Phi_{\lambda(t)}(x(t)),(c+1-\alpha)\dot{x}(t)+(\beta(t)+t\dot{\beta}(t)-tb(t))\nabla\Phi_{\lambda(t)}(x(t))\Big{\rangle}$
$\displaystyle=\ $ $\displaystyle c(c+1-\alpha)\langle
x(t)-z,\dot{x}(t)\rangle+c\big{(}\beta(t)+t\dot{\beta}(t)-tb(t)\big{)}\big{\langle}x(t)-z,\nabla\Phi_{\lambda(t)}(x(t))\big{\rangle}+(c+1-\alpha)t\|\dot{x}(t)\|^{2}$
$\displaystyle+\
\big{(}\beta(t)+t\dot{\beta}(t)-tb(t)\big{)}t\big{\langle}\dot{x}(t),\nabla\Phi_{\lambda(t)}(x(t))\big{\rangle}+t\beta(t)(c+1-\alpha)\big{\langle}\dot{x}(t),\nabla\Phi_{\lambda(t)}(x(t))\big{\rangle}$
$\displaystyle+\
t\beta(t)(\beta(t)+t\dot{\beta}(t)-tb(t))\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}.$

Overall, since $\beta(t)+t\dot{\beta}(t)-tb(t)=-tw(t)$, we obtain for every
$t\geq t_{0}$

$\displaystyle\dot{E}_{c}(t)\ =\ $
$\displaystyle\big{(}2tw(t)+t^{2}\dot{w}(t)-(\beta(t)+t\dot{\beta}(t))(c+1-\alpha)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})$
$\displaystyle+\
\big{(}t^{2}w(t)-t\beta(t)(c+1-\alpha)\big{)}\Big{(}\langle\nabla\Phi_{\lambda(t)}(x(t)),\dot{x}(t)\rangle-\frac{\dot{\lambda}(t)}{2}\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}\Big{)}$
$\displaystyle-\
ctw(t)\big{\langle}x(t)-z,\nabla\Phi_{\lambda(t)}(x(t))\big{\rangle}+(c+1-\alpha)t\|\dot{x}(t)\|^{2}-t^{2}w(t)\langle\dot{x}(t),\nabla\Phi_{\lambda(t)}(x(t))\rangle$
$\displaystyle+\
t\beta(t)(c+1-\alpha)\big{\langle}\dot{x}(t),\nabla\Phi_{\lambda(t)}(x(t))\big{\rangle}-t^{2}\beta(t)w(t)\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}.$

Notice that the terms with
$\langle\nabla\Phi_{\lambda(t)}(x(t)),\dot{x}(t)\rangle$ cancel each other,
thus, after simplification we obtain for every $t\geq t_{0}$

$\begin{split}\dot{E}_{c}(t)\ =\
&\big{(}2tw(t)+t^{2}\dot{w}(t)+(\beta(t)+t\dot{\beta}(t))(\alpha-1-c)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})\\\
&-\
\big{(}t^{2}w(t)+t\beta(t)(\alpha-1-c)\big{)}\frac{\dot{\lambda}(t)}{2}\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}-ctw(t)\big{\langle}x(t)-z,\nabla\Phi_{\lambda(t)}(x(t))\big{\rangle}\\\
&-\
(\alpha-1-c)t\|\dot{x}(t)\|^{2}-t^{2}\beta(t)w(t)\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}.\end{split}$
(15)

Thanks to (11), $w(t)$ is positive for every $t\geq t_{0}$, thus

$-ctw(t)\big{\langle}x(t)-z,\nabla\Phi_{\lambda(t)}(x(t))\big{\rangle}\ \leq\
-ctw(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*}),$

which leads to

$\begin{split}\dot{E}_{c}(t)\ \leq\
&\big{(}(2-c)tw(t)+t^{2}\dot{w}(t)+(\beta(t)+t\dot{\beta}(t))(\alpha-1-c)\big{)}\big{(}\Phi_{\lambda(t)}(x(t))-\Phi^{*}\big{)}\\\
&-\
\Bigg{(}\big{(}t^{2}w(t)+t\beta(t)(\alpha-1-c)\big{)}\frac{\dot{\lambda}(t)}{2}+t^{2}\beta(t)w(t)\Bigg{)}\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}-\
(\alpha-c-1)t\|\dot{x}(t)\|^{2}.\end{split}$ (16)

By (10) and the fact that $\lambda$ is nondecreasing, we deduce that

$\big{(}t^{2}w(t)+t\beta(t)(\alpha-1-c)\big{)}\frac{\dot{\lambda}(t)}{2}+t^{2}\beta(t)w(t)\
\geq\ 0,$

so, we obtain for every $t\geq t_{0}$

$\begin{split}\dot{E}_{c}(t)\ \leq\
&\big{(}(2-c)tw(t)+t^{2}\dot{w}(t)+(\beta(t)+t\dot{\beta}(t))(\alpha-1-c)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})\\\
&-\
\Bigg{(}\big{(}t^{2}w(t)+t\beta(t)(\alpha-c-1)\big{)}\frac{\dot{\lambda}(t)}{2}+t^{2}\beta(t)w(t)\Bigg{)}\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}.\end{split}$
(17)

Let us choose $c:=\alpha-1$. According to (12) we obtain for the coefficient
of $\Phi_{\lambda(t)}(x(t))-\Phi^{*}$ in (17)

$(2-c)tw(t)+t^{2}\dot{w}(t)+(\beta(t)+t\dot{\beta}(t))(\alpha-1-c)\
=-t\big{(}(\alpha-3)w(t)-t\dot{w}(t)\big{)}\ \leq\ 0.$

Therefore, (17) allows us to deduce

$\displaystyle\dot{E}_{\alpha-1}(t)\ $ $\displaystyle=\
-\big{(}(\alpha-3)tw(t)-t^{2}\dot{w}(t)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})-\Big{(}t^{2}w(t)\frac{\dot{\lambda}(t)}{2}+t^{2}\beta(t)w(t)\Big{)}\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}$
$\displaystyle\leq\ 0.$

We have just established that $E_{\alpha-1}$ is nonincreasing, which leads for
every $t\geq t_{0}$ to

$\displaystyle E_{\alpha-1}(t)\ $ $\displaystyle=\
t^{2}w(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})+\frac{1}{2}\left\|(\alpha-1)(x(t)-z)+t\dot{x}(t)+t\beta(t)\nabla\Phi_{\lambda(t)}(x(t))\right\|^{2}$
$\displaystyle\leq\ E_{\alpha-1}(t_{0}).$

From here we obtain for every $t\geq t_{0}$

$\Phi_{\lambda(t)}(x(t))-\Phi^{*}\ \leq\
\frac{E_{\alpha-1}(t_{0})}{t^{2}w(t)},$ (18)

which proves (ii). Moreover, by integration, we obtain

$\int_{t_{0}}^{+\infty}\left(t^{2}w(t)\frac{\dot{\lambda}(t)}{2}+t^{2}\beta(t)w(t)\right)\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}dt\
\leq\ E_{\alpha-1}(t_{0})\ <\ +\infty$ (19)

and

$\int_{t_{0}}^{+\infty}\big{(}(\alpha-3)tw(t)-t^{2}\dot{w}(t)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})dt\
\leq\ E_{\alpha-1}(t_{0})\ <\ +\infty,$ (20)

which are the claims (iii) and (iv).

From now on we assume that $\alpha>1$ and choose $c:=\alpha-1-\varepsilon$,
where $\varepsilon$ is given by (13). In this setting, (16) reads for every
$t\geq t_{0}$,

$\displaystyle\dot{E}_{\alpha-1-\varepsilon}(t)\ \leq\ $
$\displaystyle\big{(}(3-\alpha+\varepsilon)tw(t)+t^{2}\dot{w}(t)+\varepsilon(\beta(t)+t\dot{\beta}(t))\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})$
$\displaystyle-\ \Bigg{(}\big{(}t^{2}w(t)+\varepsilon
t\beta(t)\big{)}\frac{\dot{\lambda}(t)}{2}+t^{2}\beta(t)w(t)\Bigg{)}\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}-\varepsilon
t\|\dot{x}(t)\|^{2}$ $\displaystyle\ =\ $
$\displaystyle-t\big{(}(\alpha-3)w(t)-t\dot{w}(t)-\varepsilon
b(t)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})$ $\displaystyle-\
\Bigg{(}\big{(}t^{2}w(t)+\varepsilon
t\beta(t)\big{)}\frac{\dot{\lambda}(t)}{2}+t^{2}\beta(t)w(t)\Bigg{)}\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}-\varepsilon
t\|\dot{x}(t)\|^{2}.$ (21)

So, under the condition (13), $\dot{E}_{\alpha-1-\varepsilon}(t)\ \leq\ 0$ for
every $t\geq t_{0}$. Integrating (2) we obtain

$\int_{t_{0}}^{+\infty}t\|\dot{x}(t)\|^{2}dt\ <\ +\infty,$ (22)

which gives the claim (v). From the fact that the energy function

$\displaystyle E_{\alpha-1-\varepsilon}(t)\ =\ $
$\displaystyle\big{(}t^{2}w(t)+\varepsilon
t\beta(t)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})$ $\displaystyle+\
\frac{1}{2}\|(\alpha-1-\varepsilon)(x(t)-z)+t\dot{x}(t)+t\beta(t)\nabla\Phi_{\lambda(t)}(x(t))\|^{2}+\frac{(\alpha-1-\varepsilon)\varepsilon}{2}\|x(t)-z\|^{2}$

is bounded from above and it is nonnegative on $[t_{0},+\infty)$, it follows
that the trajectory $x$ is bounded, which is item (vi). Finally, from (13) and
(20) we deduce the claim (vii)

$\int_{t_{0}}^{+\infty}\varepsilon tb(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})dt\
\leq\
\int_{t_{0}}^{+\infty}\big{(}(\alpha-3)tw(t)-t^{2}\dot{w}(t)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})dt\
<\ +\infty,$ (23)

which finishes the proof. ∎

The following auxiliary result will be needed later.

######  Lemma 3.

Suppose that $\alpha>1$ and (13) holds, that $\lambda$ and $\beta$ are
nondecreasing on $[t_{0},+\infty)$, and that (11) holds. Then, for a solution
$x:[t_{0},+\infty)\rightarrow H$ to (1), it holds

$\int_{t_{0}}^{+\infty}tw(t)\langle\nabla\Phi_{\lambda(t)}(x(t)),x(t)-z\rangle
dt\ <\ +\infty.$ (24)

###### Proof.

Recall that according to (15) we have for every $t\geq t_{0}$

$\displaystyle\dot{E}_{c}(t)\ =\ $
$\displaystyle\big{(}2tw(t)+t^{2}\dot{w}(t)+(\beta(t)+t\dot{\beta}(t))(\alpha-1-c)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})$
$\displaystyle-\
\big{(}t^{2}w(t)+t\beta(t)(\alpha-1-c)\big{)}\frac{\dot{\lambda}(t)}{2}\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}$
$\displaystyle-\
ctw(t)\big{\langle}x(t)-z,\nabla\Phi_{\lambda(t)}(x(t))\big{\rangle}-(\alpha-1-ct)\|\dot{x}(t)\|^{2}-t^{2}\beta(t)w(t)\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}.$

We choose again $c:=\alpha-1$ and split the term $(\alpha-1)tw(t)\langle
x(t)-z,\nabla\Phi_{\lambda(t)}(x(t))\rangle$ into the sum of two expressed in
terms of $\varepsilon$ given by (13). For every $t\geq t_{0}$, we have

$\displaystyle\dot{E}_{\alpha-1}(t)\ \leq\ $
$\displaystyle\big{(}2tw(t)+t^{2}\dot{w}(t)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})$
$\displaystyle-\
(\alpha-1-\varepsilon)tw(t)\big{\langle}x(t)-z,\nabla\Phi_{\lambda(t)}(x(t))\rangle-\varepsilon
tw(t)\big{\langle}x(t)-z,\nabla\Phi_{\lambda(t)}(x(t))\big{\rangle}.$

By applying the convex subdifferential inequality we obtain for every $t\geq
t_{0}$

$\displaystyle\dot{E}_{\alpha-1}(t)\ \leq\ $
$\displaystyle\big{(}2tw(t)+t^{2}\dot{w}(t)-(\alpha-1-\varepsilon)tw(t)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})-\
\varepsilon
tw(t)\big{\langle}x(t)-z,\nabla\Phi_{\lambda(t)}(x(t))\big{\rangle}$
$\displaystyle\ =\ $
$\displaystyle\big{(}t^{2}\dot{w}(t)-\left(\alpha-3-\varepsilon\right)tw(t)\big{)}(\Phi_{\lambda(t)}(x(t))-\Phi^{*})-\
\varepsilon
tw(t)\big{\langle}x(t)-z,\nabla\Phi_{\lambda(t)}(x(t))\big{\rangle}.$ (25)

Since $\beta$ is nondecreasing, for every $t\geq t_{0}$ it holds

$b(t)=w(t)+\dot{\beta}(t)+\frac{\beta(t)}{t}\geq w(t),$

thus, (13) leads to $t^{2}\dot{w}(t)-(\alpha-3-\varepsilon)tw(t)\leq 0$.
Consequently, we obtain from (2) by integration

$\int_{t_{0}}^{+\infty}tw(t)\langle\nabla\Phi_{\lambda(t)}(x(t)),x(t)-z\rangle
dt\ \leq\ E_{\alpha-1}(t_{0})\ <\ +\infty.$

∎

Now we are in position to improve the convergence rates which we obtained
previously in (18) and to derive from here convergence rates for $\Phi$.

######  Theorem 4.

Suppose that $\alpha>1$ and (13) holds, that $\lambda$ and $\beta$ are
nondecreasing on $[t_{0},+\infty)$, and that (11) holds. Assume in addition

$\int_{t_{0}}^{+\infty}\left[\frac{\big{(}\dot{\lambda}(t)\big{)}^{2}t^{3}\beta^{2}(t)}{\lambda^{4}(t)}-\frac{\dot{\lambda}(t)t^{2}b(t)}{2\lambda^{2}(t)}\right]_{+}dt<+\infty,$
(26)

where $[\cdot]_{+}$ denotes the positive part of the expression inside the
brackets, and that there exists $C>0$ such that

$\frac{d}{dt}\left(t^{2}b(t)\right)\ \leq\ Ctb(t)\ \mbox{for every}\ t\geq
t_{0}.$ (27)

Then, for a solution $x:[t_{0},+\infty)\rightarrow H$ to (1), it holds

$\Phi_{\lambda(t)}(x(t))-\Phi^{*}=o\left(\frac{1}{t^{2}b(t)}\right)\text{ and
}\|\dot{x}(t)\|=o\left(\frac{1}{t}\right)\text{ as }t\to+\infty.$ (28)

Moreover,

$\|\nabla\Phi_{\lambda(t)}(x(t))\|\ =\
o\left(\frac{1}{t\sqrt{b(t)\lambda(t)}}\right)\text{ as }t\to+\infty,$ (29)

and

$\Phi(\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t)))-\Phi^{*}=o\left(\frac{1}{t^{2}b(t)}\right)\text{
and }\|\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t))-x(t)\|\ =\
o\left(\frac{\sqrt{\lambda(t)}}{t\sqrt{b(t)}}\right)\text{ as }t\to+\infty.$
(30)

###### Proof.

First we notice that for every $t\geq t_{0}$ it holds

$\displaystyle\left\langle\frac{d}{dt}\left(\nabla\Phi_{\lambda(t)}(x(t))\right),\dot{x}(t)\right\rangle\
$ $\displaystyle=\ \left\langle\lim_{h\to
0}\frac{\nabla\Phi_{\lambda(t+h)}(x(t+h))-\nabla\Phi_{\lambda(t)}(x(t))}{h},\dot{x}(t)\right\rangle$
$\displaystyle=\ \left\langle\lim_{h\to
0}\frac{\nabla\Phi_{\lambda(t+h)}(x(t+h))-\nabla\Phi_{\lambda(t+h)}(x(t))}{h},\dot{x}(t)\right\rangle$
$\displaystyle\ \ \ \ +\ \left\langle\lim_{h\to
0}\frac{\nabla\Phi_{\lambda(t+h)}(x(t))-\nabla\Phi_{\lambda(t)}(x(t))}{h},\dot{x}(t)\right\rangle.$

For every $h>0$, by the monotonicity of the gradient of a convex function, we
have

$\left\langle\frac{\nabla\Phi_{\lambda(t+h)}(x(t+h))-\nabla\Phi_{\lambda(t+h)}(x(t))}{h},\frac{x(t+h)-x(t)}{h}\right\rangle\
\geq\ 0,$

so letting $h$ tend to zero we obtain

$\left\langle\lim_{h\to
0}\frac{\nabla\Phi_{\lambda(t+h)}(x(t+h))-\nabla\Phi_{\lambda(t+h)}(x(t))}{h},\dot{x}(t)\right\rangle\
\geq\ 0.$

Consequently, for every $t\geq t_{0}$ it holds

$\displaystyle\left\langle\frac{d}{dt}\left(\nabla\Phi_{\lambda(t)}(x(t))\right),\dot{x}(t)\right\rangle$
$\displaystyle\geq$ $\displaystyle\ \left\langle\lim_{h\to
0}\frac{\nabla\Phi_{\lambda(t+h)}(x(t))-\nabla\Phi_{\lambda(t)}(x(t))}{h},\dot{x}(t)\right\rangle$
$\displaystyle=\ $ $\displaystyle\lim_{h\to
0}\left\langle\frac{(\lambda(t+h))\operatorname*{prox}_{\lambda(t)\Phi}(x(t))-\lambda(t)\operatorname*{prox}_{(\lambda(t+h))\Phi}(x(t))-\big{(}\lambda(t+h)-\lambda(t)\big{)}x(t)}{\lambda(t)\lambda(t+h)h},\dot{x}(t)\right\rangle$
$\displaystyle=\ $ $\displaystyle\lim_{h\to
0}\left\langle\frac{\big{(}\lambda(t+h)-\lambda(t)\big{)}\big{(}\operatorname*{prox}_{\lambda(t)\Phi}(x(t))-x(t)\big{)}}{\lambda(t)\lambda(t+h)h},\dot{x}(t)\right\rangle$
$\displaystyle-\ \lim_{h\to
0}\left\langle\frac{\operatorname*{prox}_{(\lambda(t+h))\Phi}(x(t))-\operatorname*{prox}_{\lambda(t)\Phi}(x(t))}{\lambda(t+h)h},\dot{x}(t)\right\rangle$
$\displaystyle\geq\ $
$\displaystyle\frac{\dot{\lambda}(t)}{\lambda^{2}(t)}\left\langle\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t))-x(t),\dot{x}(t)\right\rangle\
-\ \lim_{h\to
0}\frac{(\lambda(t+h)-\lambda(t))\|\nabla\Phi_{\lambda(t)}(x(t))\|\|\dot{x}(t)\|}{\lambda(t+h)h}$
$\displaystyle=\ $
$\displaystyle\frac{\dot{\lambda}(t)}{\lambda^{2}(t)}\left\langle\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t))-x(t),\dot{x}(t)\right\rangle\
-\
\frac{\dot{\lambda}(t)\|\nabla\Phi_{\lambda(t)}(x(t))\|\|\dot{x}(t)\|}{\lambda(t)}$
$\displaystyle=\ $
$\displaystyle-\frac{\dot{\lambda}(t)}{\lambda(t)}\left\langle\nabla\Phi_{\lambda(t)}(x(t)),\dot{x}(t)\right\rangle-\
\frac{\dot{\lambda}(t)\|\nabla\Phi_{\lambda(t)}(x(t))\|\|\dot{x}(t)\|}{\lambda(t)}\
\geq\
-\frac{2\dot{\lambda}(t)\|\nabla\Phi_{\lambda(t)}(x(t))\|\|\dot{x}(t)\|}{\lambda(t)},$

where we used (6), (7) and the Cauchy-Schwarz inequality. Now we multiply (1)
by $t^{2}\dot{x}(t)$ to deduce, by using the inequality above and (14), for
every $t\geq t_{0}$

$\displaystyle 0\ =$ $\displaystyle\
t^{2}\langle\ddot{x}(t),\dot{x}(t)\rangle+\alpha
t\|\dot{x}(t)\|^{2}+t^{2}\beta(t)\left\langle\frac{d}{dt}\left(\nabla\Phi_{\lambda(t)}(x(t))\right),\dot{x}(t)\right\rangle+t^{2}b(t)\big{\langle}\nabla\Phi_{\lambda(t)}(x(t)),\dot{x}(t)\big{\rangle}$
$\displaystyle\geq$ $\displaystyle\
t^{2}\frac{d}{dt}\left(\frac{1}{2}\|\dot{x}(t)\|^{2}\right)+\alpha
t\|\dot{x}(t)\|^{2}+t^{2}b(t)\frac{d}{dt}\left(\Phi_{\lambda(t)}(x(t))-\Phi^{*}\right)+\frac{\dot{\lambda}(t)t^{2}b(t)}{2}\|\nabla\Phi_{\lambda(t)}x(t)\|^{2}$
$\displaystyle-\
\frac{2t^{2}\beta(t)\dot{\lambda}(t)}{\lambda(t)}\|\nabla\Phi_{\lambda(t)}(x(t))\|\|\dot{x}(t)\|$
$\displaystyle\geq$ $\displaystyle\
\frac{d}{dt}\left(\frac{t^{2}}{2}\|\dot{x}(t)\|^{2}+t^{2}b(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})\right)+(\alpha-1)t\|\dot{x}(t)\|^{2}-\big{(}\Phi_{\lambda(t)}(x(t))-\Phi^{*}\big{)}\frac{d}{dt}\left(t^{2}b(t)\right)$
$\displaystyle-\
\left\\{\left[\left(\frac{\dot{\lambda}(t)}{\lambda(t)}\right)^{2}t^{3}\beta^{2}(t)-\frac{\dot{\lambda}(t)t^{2}b(t)}{2}\right]\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}+t\|\dot{x}(t)\|^{2}\right\\}.$

Using (27) we obtain for every $t\geq t_{0}$

$\displaystyle\frac{d}{dt}\left(\frac{t^{2}}{2}\|\dot{x}(t)\|^{2}+t^{2}b(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})\right)\leq\
$
$\displaystyle[2-\alpha]_{+}t\|\dot{x}(t)\|^{2}+\big{(}\Phi_{\lambda(t)}(x(t))-\Phi^{*}\big{)}Ctb(t)$
$\displaystyle+\
\left[\left(\frac{\dot{\lambda}(t)}{\lambda(t)}\right)^{2}t^{3}\beta^{2}(t)-\frac{\dot{\lambda}(t)t^{2}b(t)}{2}\right]_{+}\|\nabla\Phi_{\lambda(t)}(x(t))\|^{2}.$

Next we show the integrability of the right-hand side of the expression above.
The first term is integrable according to Theorem 2 (v) and the second one is
integrable according to Theorem 2 (vii). Further, since

$\|\nabla\Phi_{\lambda(t)}(x(t))-\nabla\Phi_{\lambda(t)}(z)\|\ \leq\
\frac{1}{\lambda(t)}\|x(t)-z\|\ \forall t\geq t_{0},$

and taking into the account the boundedness of the trajectory $x$ established
in Theorem 2 (vi) and that $z\in\operatorname*{argmin}\Phi$, we deduce

$\|\nabla\Phi_{\lambda(t)}(x(t))\|\ =\
O\left(\frac{1}{\lambda(t)}\right)\text{ as }t\to+\infty.$

So, under the assumption (26), we obtain that there exists $\widetilde{C}>0$
such that for every $t\geq t_{0}$

$\displaystyle\int_{t_{0}}^{t}\left[\left(\frac{\dot{\lambda}(s)}{\lambda(s)}\right)^{2}s^{3}\beta^{2}(s)-\frac{\dot{\lambda}(s)s^{2}b(s)}{2}\right]_{+}\|\nabla\Phi_{\lambda(s)}(x(s))\|^{2}ds$
$\displaystyle\leq\widetilde{C}\int_{t_{0}}^{t}\left[\frac{\big{(}\dot{\lambda}(s)\big{)}^{2}s^{3}\beta^{2}(s)}{\lambda^{4}(s)}-\frac{\dot{\lambda}(s)s^{2}b(s)}{2\lambda^{2}(s)}\right]_{+}ds$
$\displaystyle<\ +\infty.$

Applying Lemma 6 in the Appendix, we conclude that the following limit

$L:=\lim_{t\to+\infty}\left(\frac{t^{2}}{2}\|\dot{x}(t)\|^{2}+t^{2}b(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})\right)\geq
0$

exists. We will show that $L=0$. Supposing that $L>0$, we deduce that there
exists $t^{*}\geq t_{0}$ such that for every $t\geq t^{*}$

$\frac{t}{2}\|\dot{x}(t)\|^{2}+tb(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})\geq\frac{L}{2t}.$

Integrating the last inequality on $[t^{*},+\infty)$, we arrive at the
contradiction with the integrability of the left-hand side as proved in
Theorem 2 (v) and (vii). Therefore, $L=0$ and we obtain

$\Phi_{\lambda(t)}(x(t))-\Phi^{*}=o\left(\frac{1}{t^{2}b(t)}\right)\text{ and
}\|\dot{x}(t)\|=o\left(\frac{1}{t}\right)\text{ as }t\to+\infty.$

Using the definition of the proximal mapping, we derive

$\Phi_{\lambda(t)}(x(t))-\Phi^{*}=\
\Phi(\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t)))-\Phi^{*}+\frac{1}{2\lambda(t)}\|\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t))-x(t)\|^{2}\quad\forall
t\geq t_{0},$ (31)

which yields

$\Phi(\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t)))-\Phi^{*}=o\left(\frac{1}{t^{2}b(t)}\right)\text{
and }\|\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t))-x(t)\|\ =\
o\left(\frac{\sqrt{\lambda(t)}}{t\sqrt{b(t)}}\right)\text{ as }t\to+\infty.$

According to (6) we obtain from here

$\|\nabla\Phi_{\lambda(t)}(x(t))\|\ =\
o\left(\frac{1}{t\sqrt{b(t)\lambda(t)}}\right)\text{ as }t\to+\infty.$

∎

##  3 Convergence of the trajectories

In this section we will investigate the weak convergence of the trajectory $x$
to a minimizer of $\Phi$.

######  Theorem 5.

Suppose that $\alpha>1$, (11) and (13) hold and that $\lambda$ and $\beta$ are
nondecreasing on $[t_{0},+\infty)$. Assume in addition that

$\lim_{t\to+\infty}\frac{\beta(t)}{tw(t)}\ =\ 0$ (32)

and

$\sup_{t\geq t_{0}}\frac{\lambda(t)}{t}\ <\ +\infty.$ (33)

If $x:[t_{0},+\infty)\rightarrow H$ is a solution to (1), then $x(t)$
converges weakly to a minimizer of $\Phi$ as $t\to+\infty$.

###### Proof.

Let $z\in\operatorname*{argmin}\Phi$. Previously, in Theorem 2, we established
the existence of the limit of $E_{c}(t)$ as $t\to+\infty$ for $c=\alpha-1$ and
$c=\alpha-1-\varepsilon$, where $\varepsilon\in(0,\alpha-1)$ is given by (13).
Thus, computing the difference

$\displaystyle E_{\alpha-1-\varepsilon}(t)-E_{\alpha-1}(t)\ =\ $
$\displaystyle\varepsilon
t\beta(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})+\frac{\varepsilon(\alpha-1)}{2}\|x(t)-z\|^{2}$
$\displaystyle-\varepsilon\langle(\alpha-1)(x(t)-z)+t(\dot{x}(t)+\beta(t)\nabla\Phi_{\lambda(t)}(x(t)),x(t)-z\rangle$
$\displaystyle=\ $ $\displaystyle\varepsilon
t\beta(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})-\frac{\varepsilon(\alpha-1)}{2}\|x(t)-z\|^{2}$
$\displaystyle-\ \varepsilon
t\langle\dot{x}(t)+\beta(t)\nabla\Phi_{\lambda(t)}(x(t)),x(t)-z\rangle,$

we deduce that the limit of the right-hand side exists. Thanks to (18), we
derive for every $t\geq t_{0}$

$t\beta(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})\ \leq\
t\beta(t)\frac{E_{\alpha-1}(t_{0})}{t^{2}w(t)}\ =\
E_{\alpha-1}(t_{0})\frac{\beta(t)}{tw(t)}$

and from here, based on the assumption (32), we obtain

$\displaystyle\lim_{t\to+\infty}t\beta(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})=0.$
(34)

Hereby, we derived that the limit of the quantity

$p(t):=\frac{\alpha-1}{2}\|x(t)-z\|^{2}+t\langle\dot{x}(t),x(t)-z\rangle+t\beta(t)\langle\nabla\Phi_{\lambda(t)}(x(t)),x(t)-z\rangle$

exists as $t\to+\infty$. Now we are ready to prove the existence of the limit
of $\|x(t)-z\|$ as $t\to+\infty$. Denote

$q(t):=\frac{\alpha-1}{2}\|x(t)-z\|^{2}+(\alpha-1)\int_{t_{0}}^{t}\beta(s)\langle\nabla\Phi_{\lambda(s)}(x(s)),x(s)-z\rangle
ds\ \forall t\geq t_{0}.$

For every $t\geq t_{0}$, it holds that

$p(t)=q(t)+\frac{t}{\alpha-1}\dot{q}(t)-(\alpha-1)\int_{t_{0}}^{t}\beta(s)\langle\nabla\Phi_{\lambda(s)}(x(s)),x(s)-z\rangle
ds,$

since

$\dot{q}(t)=(\alpha-1)\langle
x(t)-z,\dot{x}(t)\rangle+\big{(}\alpha-1\big{)}\big{(}\beta(s)\langle\nabla\Phi_{\lambda(s)}(x(s)),x(s)-z\rangle\big{)}$

and

$\displaystyle q(t)+\frac{t}{\alpha-1}\dot{q}(t)\ =\ $
$\displaystyle\frac{\alpha-1}{2}\|x(t)-z\|^{2}+(\alpha-1)\int_{t_{0}}^{t}\beta(s)\langle\nabla\Phi_{\lambda(s)}(x(s)),x(s)-z\rangle
ds$ $\displaystyle+\ t\langle
x(t)-z,\dot{x}(t)\rangle+t\big{(}\beta(s)\langle\nabla\Phi_{\lambda(s)}(x(s)),x(s)-z\rangle\big{)}.$

By Lemma 3 we established that
$\int_{t_{0}}^{+\infty}sw(s)\langle\nabla\Phi_{\lambda(s)}(x(s)),x(s)-z\rangle
ds<+\infty$. In turn, (32) yields that

$\lim_{t\to+\infty}\int_{t_{0}}^{t}\beta(s)\langle\nabla\Phi_{\lambda(s)}(x(s)),x(s)-z\rangle
ds\text{ exists }.$ (35)

Finally,

$\lim_{t\to+\infty}\left(q(t)+\frac{t}{\alpha-1}\dot{q}(t)\right)\text{ also
exists. }$

Applying now Lemma 7 in the Appendix, we immediately get the existence of the
limit of $q(t)$ as $t\to+\infty$. By the definition of $q$ and (35) we
establish the first statement of the Opial’s Lemma (see Lemma 8 in the
Appendix), namely, that, for any $z\in\operatorname*{argmin}\Phi$

$\lim_{t\to+\infty}\|x(t)-z\|\;\mbox{ exists}.$

To establish the second term of the Opial’s Lemma, first note that from (31)
and (34) we have, by denoting
$\xi(t):=\operatorname*{prox}_{\lambda(t)\Phi}(x(t))$,
$\lim_{t\rightarrow+\infty}t\beta(t)(\Phi(\xi(t))-\Phi^{*})=0$ and
$\lim_{t\rightarrow+\infty}\frac{t\beta(t)}{\lambda(t)}\|\xi(t)-x(t)\|^{2}=0$.
Using that $\beta$ is nondecreasing and assumption (33), we deduce

$\lim_{t\to+\infty}\Phi(\xi(t))\ =\
\Phi^{*}\quad\mbox{and}\quad\lim_{t\to+\infty}\|\xi(t)-x(t)\|=0.$

Considering a sequence $\\{t_{k}\\}_{k\in\mathbb{N}}$ such that
$\\{x(t_{k})\\}_{k\in\mathbb{N}}$ converges weakly to an element $z\in H$ as
$k\to+\infty$, we notice that $\\{\xi(t_{k})\\}_{k\in\mathbb{N}}$ converges
weakly to $z$ as $k\to+\infty$. Now, the function $\Phi$ being convex and
lower semicontinuous in the weak topology, allows us to write

$\Phi(z)\ \leq\ \liminf_{k\to+\infty}\Phi(\xi(t_{k}))\ =\
\lim_{t\to+\infty}\Phi(\xi(t))\ =\ \Phi^{*}.$

Hence, $z\in\operatorname*{argmin}\Phi$, and the second statement of the
Opial’s Lemma is shown. This gives the weak convergence of the trajectory
$x(t)$ to a minimizer of $\Phi$ as $t\rightarrow+\infty$. ∎

######  Remark 1.

In the hypotheses of Theorem 4, in order to obtain the convergence of the
trajectories, besides (33) it is enough to assume that

$\sup_{t\geq t_{0}}\frac{\beta(t)}{tw(t)}<+\infty$

in order to guarantee (35). Indeed, in this case (34) follows from the
conclusion of Theorem 4

$\lim_{t\rightarrow+\infty}t\beta(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})\leq\lim_{t\rightarrow+\infty}t^{2}b(t)(\Phi_{\lambda(t)}(x(t))-\Phi^{*})\frac{\beta(t)}{tw(t)}=0.$

##  4 Polynomial choices for the system parameter functions

According to the previous two sections, in order to guarantee both the fast
convergence rates in Theorem 4 and the convergence of the trajectory to a
minimizer of $\Phi$ in Theorem 5, by taking also into account Remark 1, it is
enough to make the following assumptions on the system parameter functions

  1. (I)

$\alpha>1$ and there exists $\varepsilon\in(0,\alpha-1)$ such that
$(\alpha-3)w(t)-t\dot{w}(t)\geq\varepsilon b(t)$ for every $t\geq t_{0}$;

  2. (II)

$\beta$ and $\lambda$ are nondecreasing on $[t_{0},+\infty)$;

  3. (III)

$b(t)>\dot{\beta}(t)+\frac{\beta(t)}{t}$ for every $t\geq t_{0}$;

  4. (IV)

$\int_{t_{0}}^{+\infty}\left[\frac{\beta^{2}(t)(\dot{\lambda}(t))^{2}t^{3}}{\lambda^{4}(t)}-\frac{\dot{\lambda}(t)t^{2}b(t)}{2\lambda^{2}(t)}\right]_{+}dt<+\infty$;

  5. (V)

there exists $C>0$ such that $\frac{d}{dt}\left(t^{2}b(t)\right)\ \leq\
Ctb(t)$ for every $t\geq t_{0}$;

  6. (VI)

$\sup_{t\geq t_{0}}\frac{\beta(t)}{tw(t)}<+\infty$;

  7. (VII)

$\sup_{t\geq t_{0}}\frac{\lambda(t)}{t}\ <\ +\infty$.

In this section we will investigate the fulfillment of these conditions for

$b(t)=bt^{n},\quad\beta(t)=\beta t^{m}\quad\mbox{and}\quad\lambda(t)=\lambda
t^{l},$

where $n,m,l\in\mathbb{R}$, $b,\lambda>0$ and $\beta\geq 0$.

For this choice of $b$, condition (V) is fulfilled.

We assume first that $\beta=0$. Then the conditions (III), (IV) and (VI) are
fulfilled, while the conditions (II) and (VI) are nothing else than $0\leq
l\leq 1$. Condition (I) asks for $\alpha>1$ and for the existence of
$\varepsilon\in(0,\alpha-1)$ such that for every $t\geq t_{0}$

$(\alpha-3-n-\varepsilon)bt^{n}\geq 0$

or, equivalently, $\alpha-3-n\geq\varepsilon$. To this end it is enough to
have that $\alpha-3>n$.

In case $\beta>0$, conditions (II) and (VII) are nothing else than $m\geq 0$
and $0\leq l\leq 1$. Condition (III) reads for every $t\geq t_{0}$

$bt^{n}>m\beta t^{m-1}+\beta t^{m-1}=(m+1)\beta t^{m-1},$

or, equivalently,

$t^{n-m+1}>\frac{(m+1)\beta}{b}.$

From here we get

$0\leq m\leq n+1.$

and $b>(m+1)\beta t_{0}^{m-1-n}$.

Condition (VI) requires that

$\sup_{t\geq t_{0}}\frac{\beta t^{m}}{t(bt^{n}-\beta mt^{m-1}-\beta
t^{m-1})}=\sup_{t\geq t_{0}}\frac{\beta t^{m}}{bt^{n+1}-\beta
t^{m}(m+1)}<+\infty$

and it is obviously fulfilled.

Condition (I) asks for $\alpha>1$ and for the existence of
$\varepsilon\in(0,\alpha-1)$ such that for every $t\geq t_{0}$

$(\alpha-3)(bt^{n}-\beta mt^{m-1}-\beta t^{m-1})-t(bnt^{n-1}-\beta
m(m-1)t^{m-2}-\beta(m-1)t^{m-2})\geq\varepsilon bt^{n}.$

After simplification we obtain that for every $t\geq t_{0}$

$(\alpha-3-n-\varepsilon)bt^{n}+\beta(m+1)(m+2-\alpha)t^{m-1}\geq 0$

or, equivalently,

$(\alpha-3-n-\varepsilon)bt^{n-m+1}\geq\beta(m+1)(\alpha-m-2).$

On the one hand we have $m=n+1$ and
$(\alpha-3-n)\left(1-\frac{\beta(n+2)}{b}\right)>\varepsilon$, which requires
that $\alpha-3-n>0$. On the other hand, we have $m<n+1$, which also requires
that $\alpha-3-n>0$.

Consequently, we have to assume that

$\alpha-3>n\quad\mbox{and}\quad
b>\frac{\beta(m+1)(\alpha-m-2)}{(\alpha-3-n)t_{0}^{n-m+1}}.$

In this case, there will be always an $\varepsilon\in(0,\alpha-1)$ such that
$\alpha-3-n-\varepsilon>0$ and

$b>\frac{\beta(m+1)(\alpha-m-2)}{(\alpha-3-n-\varepsilon)t_{0}^{n-m+1}}>\frac{\beta(m+1)(\alpha-m-2)}{(\alpha-3-n)t_{0}^{n-m+1}},$

in other words, which satisfies condition (I).

Finally, let us have a closer look at condition (V). This reads as

$\int_{t_{0}}^{+\infty}\left[\frac{\beta^{2}t^{2m}(l\lambda)^{2}t^{2l-2}t^{3}}{\lambda^{4}t^{4l}}-\frac{l\lambda
t^{l-1}t^{2}bt^{n}}{2\lambda^{2}t^{2l}}\right]_{+}dt<+\infty$

or, equivalently,

$\int_{t_{0}}^{+\infty}\left[\left(\frac{\beta
l}{\lambda}\right)^{2}t^{2m-2l+1}-\frac{lb}{2\lambda}t^{n-l+1}\right]_{+}dt=\int_{t_{0}}^{+\infty}\left[\left(\frac{\beta
l}{\lambda}\right)^{2}-\frac{lb}{2\lambda}t^{n+l-2m}\right]_{+}t^{2m-2l+1}dt<+\infty.$

1\. In case

$n+l>2m,$

there exists $t_{1}\geq t_{0}$ such that for every $t\geq t_{1}$

$\left(\frac{\beta l}{\lambda}\right)^{2}-\frac{lb}{2\lambda}t^{n+l-2m}\leq
0.$

Therefore, we obtain

$\displaystyle\int_{t_{0}}^{+\infty}\left[\left(\frac{\beta
l}{\lambda}\right)^{2}-\frac{lb}{2\lambda}t^{n+l-2m}\right]_{+}t^{2m-2l+1}dt$
$\displaystyle=$ $\displaystyle\int_{t_{0}}^{t_{1}}\left[\left(\frac{\beta
l}{\lambda}\right)^{2}-\frac{lb}{2\lambda}t^{n+l-2m}\right]_{+}t^{2m-2l+1}dt+\
\int_{t_{1}}^{+\infty}\left[\left(\frac{\beta
l}{\lambda}\right)^{2}-\frac{lb}{2\lambda}t^{n+l-2m}\right]_{+}t^{2m-2l+1}dt$
$\displaystyle=$ $\displaystyle\int_{t_{0}}^{t_{1}}\left[\left(\frac{\beta
l}{\lambda}\right)^{2}-\frac{lb}{2\lambda}t^{n+l-2m}\right]_{+}t^{2m-2l+1}dt<+\infty,$

thus (V) is fulfilled.

2\. In case

$n+l<2m,$

there exist $\delta>0$ and $t_{2}\geq t_{0}$ such that for all $t\geq t_{2}$

$\left(\frac{\beta
l}{\lambda}\right)^{2}-\frac{lb}{2\lambda}t^{n+l-2m}>\delta.$

Taking into account that $2m-2l+1>n+1-l\geq-1$, we have

$\displaystyle\int_{t_{0}}^{+\infty}\left[\left(\frac{\beta
l}{\lambda}\right)^{2}-\frac{lb}{2\lambda}t^{n+l-2m}\right]_{+}t^{2m-2l+1}dt$
$\displaystyle\geq$ $\displaystyle\
\int_{t_{0}}^{t_{2}}\left[\left(\frac{\beta
l}{\lambda}\right)^{2}-\frac{lb}{2\lambda}t^{n+l-2m}\right]_{+}t^{2m-2l+1}dt+\delta\int_{t_{2}}^{+\infty}t^{2m-2l+1}dt\
=+\infty,$

thus (V) is not fulfilled.

3\. It is only left to consider the case

$n+l=2m.$

Condition (V) becomes

$\int_{t_{0}}^{+\infty}\left[\left(\frac{\beta
l}{\lambda}\right)^{2}-\frac{lb}{2\lambda}\right]_{+}t^{2m-2l+1}dt<+\infty.$

If $b\geq\frac{2\beta^{2}l}{\lambda}$, then it is fulfilled. Otherwise, since
$2m-2l+1=n-l+1\geq-1$, it is not fulfilled.

Summarising, all convergence statements in Theorem 4 and Theorem 5 hold in the
two settings

  1. 1.

$\alpha>1$, $\beta=0$, $\alpha-3>n$, $0\leq l\leq 1$, and $b,\lambda>0$;

  2. 2.

$\alpha>1$, $\beta>0$, $\alpha-3>n$, $0\leq l\leq 1$, $0\leq m\leq n+1$,
$b>\frac{(m+1)(\alpha-m-2)\beta}{(\alpha-3-n)t_{0}^{n-m+1}}$, $\lambda>0$, and
either $2m<n+l$, or $2m=n+l$ and $b\geq\frac{2l\beta^{2}}{\lambda}$.

######  Remark 2.

Theorem 4 is providing for the choices $b(t)=bt^{n}$ and $\lambda(t)=\lambda
t^{l}$ the following convergence rates

$\Phi(\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t)))-\Phi^{*}=o\left(\frac{1}{t^{n+2}}\right),\quad\|\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t))-x(t)\|\
=\ o\left(\frac{1}{t^{\frac{n}{2}+1-\frac{l}{2}}}\right)$

and

$\|\nabla\Phi_{\lambda(t)}(x(t))\|\ =\
o\left(\frac{1}{t^{\frac{n}{2}+1+\frac{l}{2}}}\right),$

as $t\to+\infty$. Clearly, the bigger the $n$ is the faster the convergence
is. On the other hand, concerning the exponent $l$ things are a bit more
complicated: we may gain in one case, but inevitably lose in the other.
Interesting case is when $l=0$, which corresponds to $\lambda$ being a
constant function. In this case, one can notice a balance between accelerating
the convergence of $\|\nabla\Phi_{\lambda(t)}(x(t))\|$ and slowing the latter
for $\|\operatorname*{prox}\nolimits_{\lambda(t)\Phi}(x(t))-x(t)\|$, since
none of them are affected by $l$ anymore.

##  5 Numerical examples

In this section we will conduct series of experiments to investigate the
influence of the system parameters $\lambda$, $\beta$ and $b$ on the
convergence behaviour of dynamical system. We will successively fix two of
them and vary the last one in order to do so. For the numerical experiments we
will restrict ourselves to the polynomial choices addressed in the previous
section $\lambda(t)=t^{l}$, $\beta(t)=t^{m}$, $b(t)=bt^{n}$ with
$b=\frac{(m+1)(\alpha-m-2)\beta}{(\alpha-3-n)t_{0}^{n-m+1}}+1$, as well as
$x(t_{0})=x_{0}=10$, $\dot{x}(t_{0})=0$, and $t_{0}=1$.

###  5.1 The influence of $b$ on the dynamical behaviour

First let us choose as objective function
$\Phi:\mathbb{R}\rightarrow\mathbb{R}_{+},\Phi(x)=|x|$, fix $m=0$, $\alpha=9$
and $l=1$, and vary $n$.

(a) Trajectories

(b) Moreau envelope values

(c) Moreau envelope gradient

Figure 1: $m=0$, $\alpha=9$ and $l=1$

In Figure 1 we clearly see that the faster the exponent of the function $b$
grows the faster the convergence of the function values of the Moreau envelope
and its gradient are, starting with the slowest pace for $n=0$ and
accelerating until $n=4.99$, confirming the theoretical convergence rates. In
addition, the increase in the exponent of $b$ seems to improve the convergence
behaviour of the trajectory, too. Fast growing exponents for $b$ will improve
the convergence greatly, however, as seen in the previous section, they are
limited by the upper bound value $\alpha-3$.

###  5.2 The influence of $\lambda$ on the dynamical behaviour

For the same objective function as in the previous subsection, we study the
behaviour of the dynamics when varying the exponent $l$ to investigate the
influence of the function $\lambda$. To this end we fix $m=0$, $\alpha=9$ and
$n=5<\alpha-3$, and take for $l$ three different values from $0$ to $1$.

(a) Trajectories

(b) Moreau envelope values

(c) Moreau envelope gradient

Figure 2: $m=0$, $\alpha=9$ and $n=5$

One can notice in Figure 2 that the convergence behaviour of the functions
values of the Moreau envelope and its gradient is better the higher $l$ is,
whereas, interestingly enough, for the convergence of the trajectories an
opposite phenomenon takes place.

###  5.3 The influence of $\beta$ on the dynamical behaviour

Let $\Phi:\mathbb{R}\rightarrow\mathbb{R}_{+},\Phi(x)=|x|+\frac{x^{2}}{2}$,
$\alpha=13$, $n=9<\alpha-3$ and $l=1$. We vary the exponent $m$ such that
$2m<n+l$ to study the influence of the function $\beta$ on the convergence
behaviour of the system.

(a) Trajectories

(b) Moreau envelope values

(c) Moreau envelope gradient

Figure 3: $n=9$, $\alpha=13$ and $l=1$

In Figure 3 we see that, even though $m$ does not explicitly appear in the
theoretical convergence rates for the gradient of the Moreau envelope and the
trajectory of the system, it influences the convergence behaviour of both of
them as well as of the function values of the Moreau envelope, in the sense
that these are faster the higher the values of $m$ are.

(a) $n=9$, $\alpha=13$, $l=1$ and $m=12$

(b) $n=4$, $\alpha=2$, $l=4$ and $m=6$

Figure 4: Divergence of the trajectories

Finally, we consider two parameter choices which lie outside the convergence
setting derived in the previous section and notice that these fundamentally
affects the convergence of the trajectory. In Figure 4 (a) we choose $m$ such
that that condition $2m<n+l$ is violated, and in Figure (b) we choose $\alpha$
and $n$ such that the condition $\alpha-3>n$ is also violated. One can see
that in both settings the trajectories diverge.

## Appendix

In this appendix we collect some lemmas which play an important role in the
proof of the main results of the paper. For the proof of the following lemma
we refer to [4].

######  Lemma 6.

Suppose that $f:[t_{0},+\infty)\to\mathbb{R}$ is locally absolutely continuous
and bounded from below and there exists $g\in
L^{1}([t_{0},+\infty),\mathbb{R})$ such that for almost all $t\geq t_{0}$

$\frac{d}{dt}f(t)\ \leq\ g(t).$

Then there exists $\lim_{t\to+\infty}f(t)\in\mathbb{R}$.

For the proof of the following lemma we refer to [14].

######  Lemma 7.

Let $H$ be a real Hilbert space and
$x:[t_{0},+\infty)\longrightarrow\mathbb{H}$ a continuously differentiable
function satisfying $x(t)+\frac{t}{\alpha}\dot{x}(t)\to\ L$ as $t\to+\infty$,
with $\alpha>0$ and $L\in\mathbb{H}$. Then $x(t)\to L$ as $t\to+\infty$.

Finally, we state a continuous version of Opial’s Lemma (see [21]), which is
used in the proof of the convergence of the trajectory.

######  Lemma 8.

Let $S$ be a non-empty subset of a real Hilbert space $H$ and
$x:[0,+\infty)\mapsto H$ a given map. Assume that

  * •

for every $z\in S$, $\lim_{t\to+\infty}\|x(t)-z\|$ exists;

  * •

every weak sequential cluster point of the map $x$ belongs to $S$.

Then $x(t)$ converges weakly to some element of $S$ as $t\to+\infty$.

## References

  * [1] H. Attouch, F. Alvarez,  The heavy ball with friction dynamical system for convex constrained minimization problems,  Lecture Notes in Economics and Mathematical Systems 481, 25–35, 2000.
  * [2] H. Attouch, F. Alvarez,  An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping,  Set-Valued Analysis 9, 3–11, 2001.
  * [3] F. Alvarez, H. Attouch, J. Bolte, P. Redont,  A second-order gradient-like dissipative dynamical system with Hessian-driven damping,  Application to optimization and mechanics, Journal de Mathématiques Pures et Appliquées 81(8), 747–779, 2002.
  * [4] H. Attouch, B. Abbas, B. F. Svaiter,  Newton-like dynamics and forward-backward methods for structured monotone inclusions in Hilbert spaces,  Journal of Optimization Theory and Applications 161(2), 331-360, 2014.
  * [5] H. Attouch, A. Balhag, Z. Chbani, H. Riahi,  Fast convex optimization via inertial dynamics combining viscous and Hessian-driven damping with time rescaling,  Evolution Equations & Control Theory 11(2), 487-514, 2022.
  * [6] H. Attouch, A. Cabot,  Convergence of damped inertial dynamics governed by regularized maximally monotone operators,  Journal of Differential Equations 264, 7138–7182, 2018.
  * [7] H. Attouch, Z. Chbani, J. Fadili, H. Riahi,  Convergence of iterates for first-order optimization algorithms with inertia and Hessian driven damping,  Optimization, DOI: 10.1080/02331934.2021.2009828, 2022.
  * [8] H. Attouch, Z. Chbani, H. Riahi,  Fast convex optimization via time scaling of damped inertial gradient dynamics,  SIAM Journal on Optimization 29(3), 2227-2256, 2019.
  * [9] H. Attouch, Z. Chbani, J. Peypouquet, P. Redont,  Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity,  Mathematical Programming 168, 123-175, 2018.
  * [10] H. Attouch, X. Goudou, P. Redont,  The heavy ball with friction method. The continuous dynamical system, global exploration of the local minima of a real-valued function by asymptotical analysis of a dissipative dynamical system.,  Communications in Contemporary Mathematics 2(1), 1-34, 2000.
  * [11] H. Attouch, S. László,  Continuous Newton-like inertial dynamics for monotone inclusions,  Set-Valued and Variational Analysis 29, 555–581, 2021.
  * [12] H. Attouch, J. Peypouquet,  Convergence of inertial dynamics and proximal algorithms governed by maximal monotone operators,  Mathematical Programming 174(1-2), 391–432, 2019.
  * [13] H. Attouch, J. Peypouquet,  The rate of convergence of Nesterov’s accelerated forward-backward method is actually faster than $\frac{1}{k^{2}}$,  SIAM Journal on Optimization 26(3), 1824-1834, 2016.
  * [14] H. Attouch, J. Peypouquet, P. Redont,  Fast convex optimization via inertial dynamics with Hessian driven damping damping,  Journal of Differential Equations 261(10), 5734-5783, 2016.
  * [15] H. H. Bauschke, P. L. Combettes,  Convex Analysis and Monotone Operator Theory in Hilbert Spaces,  CMS Books in Mathematics, Springer, 2016.
  * [16] R. I. Boţ, E.R. Csetnek,  Second order forward-backward dynamical systems for monotone inclusion problems,  SIAM Journal on Control and Optimization 54(3), 1423-1443, 2016.
  * [17] A. Cabot, H. Engler, S. Gadat,  On the long time behavior of second order differential equations with asymptotically small dissipation and insights,  Transactions of the American Mathematical Society 361, 5983–6017, 2009.
  * [18] A. Cabot, H. Engler, S. Gadat,  Second order differential equations with asymptotically small dissipation and piecewise flat potentials,  Electronic Journal of Differential Equations, 17, 33–38, 2009.
  * [19] R. May,  Asymptotic for a second-order evolution equation with convex potential and vanishing damping term,  Turkish Journal of Mathematics 41(3), 681-685, 2017.
  * [20] Y. Nesterov,  A method for solving the convex programming problem with convergence rate $O\left(\frac{1}{k^{2}}\right)$,  Doklady Akademii Nauk SSSR 269(3), 543-547, 1983.
  * [21] Z. Opial,  Weak convergence of the sequence of successive approximations for nonexpansive mappings,  Bulletin of the American Mathematical Society 73(4), 591-597, 1967.
  * [22] B. T. Polyak,  Some methods of speeding up the convergence of iterative methods,  USSR Computational Mathematics and Mathematical Physics 4(5), 1-17, 1964.
  * [23] W. Su, S. Boyd, E.J. Candès,  A differential equation for modeling Nesterov’s accelerated gradient method: theory and insights,  Journal of Machine Learning Research 17, 1-43, 2016.