∎

11institutetext: Xiaoxue Jiang 22institutetext: College of Mathematics,
Sichuan University
Chengdu 610065, China
<EMAIL_ADDRESS>
# A nonmonotone proximal quasi-Newton method for multiobjective optimization

Xiaoxue Jiang

(Received: date / Accepted: date)

###### Abstract

This paper proposes a nonmonotone proximal quasi-Newton algorithm for
unconstrained convex multiobjective composite optimization problems. To design
the search direction, we minimize the max-scalarization of the variations of
the Hessian approximations and nonsmooth terms. Subsequently, a nonmonotone
line search is used to determine the step size, we allow for the decrease of a
convex combination of recent function values. Under the assumption of strong
convexity of the objective function, we prove that the sequence generated by
this method converges to a Pareto optimal. Furthermore, based on the strong
convexity, Hessian continuity and Dennis-Moré criterion, we use a basic
inequality to derive the local superlinear convergence rate of the proposed
algorithm. Numerical experiments results demonstrate the feasibility and
effectiveness of the proposed algorithm on a set of test problems.

###### Keywords:

Multiobjective optimization Composite optimization Proximal gradient Quasi-
Newton Superlinear convergence

###### MSC:

90C29 90C30

††journal:

##  1 Introduction

In this paper, we consider the following multiobjective composite optimization
problems (MCOPs):

$\displaystyle\min_{x\in\mathbb{R}^{n}}\
F(x)=(F_{1}(x),F_{2}(x),...,F_{m}(x))^{T},$ (MCOP)

where $F_{j}:\mathbb{R}^{n}\rightarrow\mathbb{R},j=1,2,...,m,$ have the
following special separable structure:

$F_{j}(x)=f_{j}(x)+g_{j}(x),$

$f_{j}(x)$ is convex and continuously differentiable, the gradient $\nabla
f_{j}$ is Lipschitz continuous and $g_{j}(x)$ is convex and continuous but not
necessarily differentiable. Some of the applications of MCOPs include problems
in the areas of image processing GZY16 , robust optimization FW14  and machine
learning BMTB20 , etc.

In the past two decades, descent methods have attracted extensive attention in
multiobjective optimization community (see, e.g., AS20 ; BCNLSS18 ; FS00 ;
FDS09 ; GP21 ; LPP18 ; P14 ; WHWLY19  and references therein). Initially,
Fliege et al. FS00  proposed the multiobjective steepest descent method, which
obtains the descent direction by solving an auxiliary and non-parametric
scalar subproblem at each iteration, eliminating the need for predefined
parameters. Since then, the steepest descent method has led to a series of
multiobjective optimization algorithms.

Recently, Tanabe et al. TFY19  proposed a multiobjective proximal gradient
algorithm (MPGA) for solving MCOPs. Subsequently, in TFY22 , they used a merit
function to analyze the convergence rate of MPGA under nonconvex, convex, and
strongly convex conditions. These results are consistent with their
counterparts in scalar optimization. The convergence rate analysis in TFY22
revealed that the MPGA demonstrates slow convergence when encountering ill-
conditioned problems. In order to deal with this problem, Ansary A22  uses the
idea of multiobjective Newton method introduced by Filege et al. FDS09  and
proposes a Newton-type proximal gradient algorithm (NPGA) for MCOPs, which
extends the corresponding method in scalar optimization LSS14 . Specifically,
in each iteration of NPGA, a subproblem is constructed using the second-order
approximation of the smooth term, followed by a Armijo line search strategy to
determine the step size. Under reasonable assumptions, Ansary proved that
every accumulation point of the sequence generated by the NGPA converges to
the critical point. However, the convergence rate of the NGPA wasn’t obtained
in A22 . Subsequently, Chen et al. CJTY23  analyzed the convergence rate of
NPGA for MCOPs by using a fundamental inequality, and obtained the quadratic
convergence rate of NPGA under the strongly convex assumption. Likewise, a
challenge associated with this method is that NPGA uses precise Hessian
information, which is usually not easy to obtain. In the conclusion of A22 ,
Ansary pointed out that NPGA is restricted to convex multiobjective
optimization problems and Hessian of each smooth function is used in each
iteration which is computationally expensive. And also pointed out that it is
a research path to overcome this limitation by using the idea of quasi-Newton
method. In such cases, approximate information can be used to approximate the
second order Hessian.

Lately, Peng et al. PR22  proposed a proximal quasi-Newton algorithm (PQNA)
for MCOPs. The subproblem of PQNA is constructed by using the BFGS quadratic
approximation the Hessian of the smooth term and an additional quadratic
regularization term. Then the monotone line search is used to determine the
step size. Under mild assumptions, they obtained that each accumulation point
of the sequence generated by PQNA, if exists, is a Pareto stationary point.
However, it’s worth mentioning that the superlinear convergence rate of the
PQNA wasn’t obtained in PR22 . It is well known that the single-objective
proximal quasi-Newton algorithm leverages the non-expansiveness of the
proximal operator to establish superlinear convergence LSS14 . But, in the
case of multiobjective, when the subproblem is formulated as a proximal
operator by involving dual variables, the dual variables are different and
variable in each iteration. Consequently, as in scalar optimization, it is not
feasible to prove the superlinear convergence of the multiobjective proximal
quasi-Newton algorithm according to the characteristics of the proximal
operator. Meanwhile, in the realm of multiobjective optimization, nonmonotone
line search techniques are extensively employed and have been demonstrated to
be more effective than monotone algorithms AS20 .

In this paper, we revisit PQNA for MCOPs. Taking inspiration from the quasi-
Newton method, which approximates second derivative matrices instead of
explicitly evaluating them. By incorporating the nonmonotone line search
technique, we propose a nonmonotone proximal quasi-Newton algorithm (NPQNA)
for MCOPs. It must be pointed out that the NPQNA of MCOPs differs from the
PQNA of MCOPs proposed by Peng et al. in PR22  in several ways. Firstly, we
use the nonmonotone linear search strategy to find the step size, whereas the
latter uses the monotone linear search strategy. Secondly, if Hessian
approximation matrix is positive definite and the subproblem exhibits strong
convexity, we do not require the additional quadratic regularization term
considered by Peng et al. PR22  to achieve strong convexity. Consequently, we
omit the regularization term in the construction of the subproblem. Under
standard assumptions, we establish that the sequence generated by NPQNA
converges to a Pareto optimal. Furthermore, under the assumptions of Hessian
continuity, the Dennis-Moré criterion and strong convexity, we use a basic
inequality to obtain that the local superlinear convergence rate of NPQNA,
which was not obtained in Peng et al. PR22 . Numerical experiments show that
the NPQNA outperform the NPGA A22  and the PQNA PR22  in terms of the number
of iterations, the number of function evaluations and CPU time for some test
problems.

The remainder of this paper is organized as follows. Section 2 gives some
preliminary remarks. In Section 3, we recall the nonmonotone line search.
Section 4 presents the nonmonotone proximal quasi-Newton method. In Section 5,
we establish the global convergence, and the local superlinear convergence
rate of NPQNA. Section 6 presents the results of our numerical experiments.

##  2 Preliminaries

Throughout this paper, the $n$-dimensional Euclidean space $\mathbb{R}^{n}$ is
equipped with the inner product $\langle\cdot,\cdot\rangle$ and the induced
norm $\|\cdot\|$. For a positive definite matrix $H$, the notation
$\|x\|_{H}=\sqrt{\langle x,Hx\rangle}$ is used to represent the norm induced
by $H$ on vector $x$. For simplicity, we utilize the notation
$\Lambda_{m}=\\{1,2,...,m\\}$ for any $m\in\mathbb{N}$, and define

$\Delta_{m}:=\left\\{\lambda:\sum_{j\in\Lambda_{m}}\lambda_{j}=1,\lambda_{j}\geq
0\right\\}.$

Let $\mathbb{R}^{m}$ be the $m$-dimensional Euclidean space with the partial
order “$\preceq$” in $\mathbb{R}^{m}$ induced by the Paretian cone
$\mathbb{R}_{+}^{m}$, given by $y\preceq z$ (or $z\succeq y$) if and only if
$z-y\in\mathbb{R}_{+}^{m}$ with its associate relation “$\prec$”, given by
$y\prec z$ (or $z\succ y$) if and only if $z-y\in\mathbb{R}_{++}^{m}$, where

$\mathbb{R}_{+}^{m}:=\\{x\in\mathbb{R}^{m}|x_{i}\geq 0,\ \forall
i\in\Lambda_{m}\\},$

and

$\mathbb{R}_{++}^{m}:=\\{x\in\mathbb{R}^{m}|x_{i}>0,\ \forall
i\in\Lambda_{m}\\}.$

Since no unique solution which minimizes all objective functions
simultaneously exists, we must decide which objective to improve. Hence, the
concept of optimality has to be replaced by the concept of Pareto optimality
or efficiency. Given a vector-valued function
$F:\mathbb{R}^{n}\rightarrow\mathbb{R}^{m}$, we analyze the NPQNA for finding
a Pareto optimum of (MCOP). A point $x^{*}\in\mathbb{R}^{n}$ is said to be an
Pareto optimum of the $F$ if there does not exist $x\in\mathbb{R}^{n}$ such
that $F(x)\leq F(x^{*})$ and $F(x)\neq F(x^{*})$. A feasible point
$x^{*}\in\mathbb{R}^{n}$ is said to be a weakly Pareto optimum of the $F$ if
there does not exist $x\in\mathbb{R}^{n}$ such that $F(x)<F(x^{*})$. It is
clear that every efficient solution of the $F$ is a weakly Pareto solution,
but the converse is not true.

###### Definition 1

(QLGLJ14 ) $x^{*}\in\mathbb{R}^{n}$ is a critical point of the (MCOP), if

$R(\partial{F_{j}(x^{*};d)})\cap(-\mathbb{R}_{++}^{m})=\emptyset,$

where $R(\partial{F_{j}(x^{*};d)})$ denotes the range or image space of the
generalized Jacobian of the function $F_{j}$ at $x^{*}$.

The classic directional derivative of $F_{j}$ at $x^{*}$ in direction $d$ is
defined as

$F_{j}^{{}^{\prime}}(x^{*};d):=\lim\limits_{\alpha\downarrow
0}\frac{F_{j}(x^{*}+\alpha d)-F_{j}(x^{*})}{\alpha}.$

Definition 1 shows that

$(F_{1}^{{}^{\prime}}(x^{*};d),F_{2}^{{}^{\prime}}(x^{*};d),...,F_{m}^{{}^{\prime}}(x^{*};d))\notin-\mathbb{R}_{++}^{m},\
{\rm for\ all}\ d\in\mathbb{R}^{n}.$

That is

$\displaystyle\max_{j\in\Lambda_{m}}F_{j}^{{}^{\prime}}(x^{*};d)\geq 0,\ {\rm
for\ all}\ d\in\mathbb{R}^{n}.$ (1)

###### Definition 2

(A22 ) A function $f:\mathbb{R}^{n}\rightarrow\mathbb{R}$ is said to be a
$m$-strongly convex if for every $x,y\in\mathbb{R}^{n}$ and $\alpha\in[0,1]$,

$f(\alpha x+(1-\alpha)y)\leq\alpha
f(x)+(1-\alpha)f(y)-\frac{1}{2}m\alpha(1-\alpha)\|x-y\|^{2}.$

######  Definition 3 (DD11 )

Suppose $h:\mathbb{R}^{n}\rightarrow(-\infty,+\infty]$ be a proper function
and $x\in{\rm dom}(h)$. Then subdifferential of $h$ at $x$ is denoted by
$\partial{h(x)}$ and defined as

$\displaystyle\partial{h(x)}:=\\{\xi\in\mathbb{R}^{n}|h(y)\geq
h(x)+\xi^{\top}(y-x),\ {\rm for\ all}\ y\in\mathbb{R}^{n}\\}.$

If $x\notin{\rm dom}(h)$ then we define $\partial{h(x)}=\emptyset$.

Next we review some properties of subdifferentials.

######  Theorem 2.1 (Proposition 2.82, DD11 )

Let $h:\mathbb{R}^{n}\rightarrow(-\infty,\infty]$ be a proper convex function,
and assume that $x\in{\rm int}({\rm dom}h)$. Then $\partial{h(x)}$ is nonempty
and bounded. Moreover, if $h$ is continuous at $x\in{\rm dom}(h)$, then
$\partial{h(x)}$ is compact.

######  Theorem 2.2 (Theorem 2.91, DD11 )

Consider two proper convex functions
$h_{i}:\mathbb{R}^{n}\rightarrow\mathbb{R},i=1,2$. Suppose that ${\rm ri}\
{\rm dom}(h_{1})\cap{\rm ri}\ {\rm dom}(h_{2})\neq\emptyset$. Then

$\displaystyle\begin{aligned}
\partial({h_{1}(x)+h_{2}(x)})=\partial{h_{1}(x)}+\partial{h_{2}(x)},\end{aligned}$

for every $x\in{\rm dom}(h_{1}+h_{2})$.

######  Theorem 2.3 (Theorem 2.96, DD11 )

Consider convex functions
$h_{j}:\mathbb{R}^{n}\rightarrow\mathbb{R},j\in\Lambda_{m}$ and let
$h(x)=\max\\{h_{1}(x),h_{2}(x),...,h_{m}(x)\\}$. Then

$\displaystyle\begin{aligned} \partial{h(x)}=Co\mathop{\cup}\limits_{j\in
I(x)}\partial{h_{j}(x)},\end{aligned}$

where $I(x)=\\{j\in\Lambda_{m}|h(x)=h_{j}(x)\\}$ is the active index set and
$Co$ is the convex hull.

According to the properties of the subdifferential, we recall the necessary
conditions for the critical point of (MCOP).

######  Lemma 1 (A22  _Lemma 1_)

Suppose $0\in Co\mathop{\cup}\limits_{j\in\Lambda_{m}}\partial{F_{j}(x^{*})}$
for some $x^{*}\in\mathbb{R}^{n}$, then $x^{*}$ is a critical point of the
(MCOP).

###### Proposition 1

FDS09  The following statements hold:

  1. 1.

If $x^{*}$ is locally weakly Pareto optimal, then $x^{*}$ is a critical point
for $F$.

  2. 2.

If $F$ is $\mathbb{R}_{+}^{m}$-convex and $x^{*}$ is critical for $F$, then
$x^{*}$ is weakly Pareto optimal.

  3. 3.

If $F$ is $\mathbb{R}_{+}^{m}$-strictly convex and $x^{*}$ is critical for
$F$, then $x^{*}$ is Pareto optimal.

Define

$\displaystyle\begin{aligned} Q_{j}(x,d)&:=\langle\nabla
f_{j}(x),d\rangle+\frac{1}{2}d^{T}B_{j}(x)d+g_{j}(x+d)-g_{j}(x),j\in\Lambda_{m}.\end{aligned}$

The matrix $B_{j}(x)$ aims to approximate the Hessian matrix of the function
$f_{j}$ at iteration $x$. Define

$Q(x,d):=\mathop{\max}\limits_{j\in\Lambda_{m}}Q_{j}(x,d).$

For any fixed $x\in\mathbb{R}^{n}$, $Q_{j}$ is continuous for $d$, and hence
$Q$ is continuous for $d$. Denote

$I(x,d):=\\{j\in\Lambda_{m}\ |\ Q(x,d)=Q_{j}(x,d)\\}.$

For $x\in\mathbb{R}^{n}$, we solve the following subproblem to find a suitable
descent direction of the (MCOP)

$\displaystyle\begin{aligned}
\mathop{\min}_{d\in\mathbb{R}^{n}}Q(x,d)\Leftrightarrow\mathop{\min}_{d\in\mathbb{R}^{n}}\mathop{\max}_{j\in\Lambda_{m}}Q_{j}(x,d).\end{aligned}$
(2)

Denote

$\displaystyle\begin{aligned}
d(x)&=\mathop{\arg\min}\limits_{d\in\mathbb{R}^{n}}Q(x,d)\\\
&=\mathop{\arg\min}\limits_{d\in\mathbb{R}^{n}}\max_{j\in\Lambda_{m}}\left\\{\nabla
f_{j}(x)^{T}d+\frac{1}{2}d^{T}B_{j}(x)d+g_{j}(x+d)-g_{j}(x)\right\\},\end{aligned}$
(3)

and

$\displaystyle\begin{aligned} \theta(x)&=Q(x,d(x))\\\
&=\max_{j\in\Lambda_{m}}\left\\{\nabla
f_{j}(x)^{T}d(x)+\frac{1}{2}d(x)^{T}B_{j}(x)d(x)+g_{j}(x+d(x))-g_{j}(x)\right\\}.\end{aligned}$
(4)

###### Remark 1

If $g_{j}=0$ for all $j\in\Lambda_{m}$, then (2) coincides with the subproblem
in AS20 , if $m=1$ then (2) coincides with the subproblem used in Step 3 of
Algorithm 1 in LSS14 , and if $B_{j}(x)=\nabla^{2}f_{j}(x)$ then (2) coincides
with the subproblem used in Step 2 of Algorithm 1 in A22 .

We can see that if the matrix $B_{j}(x)\succeq mI$ for all $j$, then
$Q_{j}(x,d)$ is $m$-strongly convex function for $d$. Hence $Q(x,d)$ is
$m$-strongly convex function for $d$. This implies that (2) has a unique
finite minimizer.

Clearly for every $x\in\mathbb{R}^{n}$,

$\displaystyle\begin{aligned} \theta(x)=Q(x,d(x))\leq Q(x,0)=0.\end{aligned}$
(5)

Since $d(x)$ is the solution of (2), according to the Lemma 1, we have

$0\in\partial_{d}Q_{j}(x,d(x)).$

The original problem (2) can be transformed into:

$\displaystyle\begin{aligned} &\min\ t\\\ &\ {\rm s.t.}\ \nabla
f_{j}(x)^{T}d+\frac{1}{2}d^{T}B_{j}(x)d+g_{j}(x+d)-g_{j}(x)-t\leq 0,\
j\in\Lambda_{m},\\\ &\ \ \ \ \ \
(t,d)\in\mathbb{R}\times\mathbb{R}^{n},\end{aligned}$ (6)

which is a convex quadratic problem, we assume that the
$B_{j}(x),j\in\Lambda_{m}$ are positive definite.

The Lagrangian function of the problem (6) is:

$\displaystyle\begin{aligned}
L(t,d,\lambda)=t+\sum\limits_{j\in\Lambda_{m}}\lambda_{j}\left(\nabla
f_{j}(x)^{T}d+\frac{1}{2}d^{T}B_{j}(x)d+g_{j}(x+d)-g_{j}(x)-t\right).\end{aligned}$

As reported in GP21 , in the smooth case, there are the well-known Karush-
Kuhn-Tucker (KKT) conditions, which are based on the gradients of the
objective functions. In case the objective functions are nonsmooth, the KKT
conditions can be generalized using the concept of subdifferentials. Thus,
there exist $\lambda\in\Delta_{m}$ and $\xi_{j}\in\partial_{d}{g_{j}(x+d)}$,
such that

$\displaystyle\begin{aligned}
\sum\limits_{j\in\Lambda_{m}}\lambda_{j}\left(\nabla
f_{j}(x)+B_{j}(x)d+\xi_{j}\right)=0,\end{aligned}$ (7)

$\displaystyle\begin{aligned} \nabla
f_{j}(x)^{T}d+\frac{1}{2}d^{T}B_{j}(x)d+g_{j}(x+d)-g_{j}(x)\leq t,\
j\in\Lambda_{m},\end{aligned}$ (8) $\displaystyle\begin{aligned}
\lambda_{j}\left(\nabla
f_{j}(x)^{T}d+\frac{1}{2}d^{T}B_{j}(x)d+g_{j}(x+d)-g_{j}(x)-t\right)=0,\
j\in\Lambda_{m}.\end{aligned}$ (9)

Thus, if $d(x)$ is the solution of (6) and $t=\theta(x)$, then there exists
$\lambda\in\triangle_{m}$ such that $(d(x),\theta(x))$ satisfy the condition
(7)-(9).

###### Theorem 2.4

Suppose $B_{j}(x)$ is a positive definite matrix for all $x\in\mathbb{R}^{n}$
and consider $\theta(x)$ as defined by equality (4). Then,

(a)For all $x\in\mathbb{R}^{n}$, $\theta(x)\leq 0$.

(b)The following conditions are equivalent:

(1) The point $x$ is not critical.

(2) $\theta(x)<0$.

(3) $d(x)\neq 0$.

(c)The function $\theta$ is continuous.

Proof Note that, by equality (4),

$\displaystyle\begin{aligned}
\theta(x)&=\min\limits_{d\in\mathbb{R}^{n}}\max_{j\in\Lambda_{m}}\nabla
f_{j}(x)^{T}d+\frac{1}{2}d^{T}B_{j}(x)d+g_{j}(x+d))-g_{j}(x)\\\
&\leq\max_{j\in\Lambda_{m}}\nabla
f_{j}(x)^{T}0+\frac{1}{2}0^{T}B_{j}(x)0+g_{j}(x)-g_{j}(x)\\\
&=0.\end{aligned}$

So, item (a) holds.

Let us now prove the equivalences of item (b).

First, assume that (1) holds, i.e. $R(\partial
F_{j}(x))\bigcap(-\mathbb{R}_{++}^{m})\neq\emptyset,$ which in turn means that
there exists $\widetilde{d}\in\mathbb{R}^{n}$ such that $\partial
F_{j}(x)^{T}\widetilde{d}<0,$ Thus, using equality (4), for all $t>0$, we have

$\displaystyle\begin{aligned} \theta(x)&\leq\max_{j\in\Lambda_{m}}\nabla
f_{j}(x)^{T}t\widetilde{d}+\frac{1}{2}t^{2}\widetilde{d}^{T}B_{j}(x)\widetilde{d}+g_{j}(x+t\widetilde{d})-g_{j}(x)\\\
&=t\max_{j\in\Lambda_{m}}\nabla
f_{j}(x)^{T}\widetilde{d}+\frac{t}{2}\widetilde{d}^{T}B_{j}(x)\widetilde{d}+\frac{g_{j}(x+t\widetilde{d})-g_{j}(x)}{t}\\\
&\leq t\max_{j\in\Lambda_{m}}\nabla
f_{j}(x)^{T}\widetilde{d}+\frac{t}{2}\widetilde{d}^{T}B_{j}(x)\widetilde{d}+\xi_{j}^{T}\widetilde{d},\
(\xi_{j}\in\partial g_{j}(x+t\widetilde{d}))\\\
&=t\max_{j\in\Lambda_{m}}\nabla f_{j}(x)^{T}\widetilde{d}-\nabla
f_{j}(x+t\widetilde{d})^{T}\widetilde{d}+\nabla
f_{j}(x+t\widetilde{d})^{T}\widetilde{d}+\frac{t}{2}\widetilde{d}^{T}B_{j}(x)\widetilde{d}+\xi_{j}^{T}\widetilde{d},\\\
&\leq t\max_{j\in\Lambda_{m}}L(-t\widetilde{d})\widetilde{d}+\nabla
f_{j}(x+t\widetilde{d})^{T}\widetilde{d}+\frac{t}{2}\widetilde{d}^{T}B_{j}(x)\widetilde{d}+\xi_{j}^{T}\widetilde{d},\\\
&=t\max_{j\in\Lambda_{m}}-tL\widetilde{d}^{2}+(\nabla
f_{j}(x+t\widetilde{d})+\xi_{j})^{T}\widetilde{d}+\frac{t}{2}\widetilde{d}^{T}B_{j}(x)\widetilde{d}.\end{aligned}$

Therefore, for $t>0$ sufficiently small, the third term on the right hand side
of the above inequality can be viewed as an infinitesimal of $t$, and $\nabla
f_{j}$ is Lipschitz continuous, the right hand side of the above inequality is
negative and (2) holds.

To prove that (2) implies (3), recall that $\theta(x)$ is the optimal valve of
problem (2) and so, being negative, the solution to this problem cannot be
$d(x)=0.$

Finally, let us see that (3) implies (1).

For the purpose of contradiction that, let $x$ be a critical point. Since
$B_{j}(x)$ is a positive definite matrix for all $x\in\mathbb{R}^{n}$, from
$\theta(x)\leq 0,$

$\displaystyle\begin{aligned} \nabla
f_{j}(x)^{T}d(x)+g_{j}(x+d(x))-g_{j}(x)\leq-\frac{1}{2}d(x)^{T}B_{j}(x)d(x)<0.\end{aligned}$
(10)

Since $g_{j}$ is convex, for any $\alpha\in(0,1)$

$\displaystyle\begin{aligned} g_{j}(x+\alpha
d(x))-g_{j}(x)&=g_{j}\left(\alpha(x+d(x))+(1-\alpha)x\right)-g_{j}(x)\\\
&\leq\alpha g_{j}(x+d(x))+(1-\alpha)g_{j}(x)-g_{j}(x)\\\
&=\alpha(g_{j}(x+d(x))-g_{j}(x)).\\\ \end{aligned}$ (11)

From inequality(10) and (11),

$\displaystyle\begin{aligned} \alpha\nabla f_{j}(x)^{T}d(x)+g_{j}(x+\alpha
d(x))-g_{j}(x)&\leq\alpha\nabla
f_{j}(x)^{T}d(x)+\alpha\left(g_{j}(x+d(x))-g_{j}(x)\right)\\\
&=\alpha\left(\nabla f_{j}(x)^{T}d(x)+g_{j}(x+d(x))-g_{j}(x)\right)\\\
&<0.\end{aligned}$

This implies

$\displaystyle\begin{aligned} \frac{1}{\alpha}\left(\alpha\nabla
f_{j}(x)^{T}d(x)+g_{j}(x+\alpha d(x))-g_{j}(x)\right)<0.\end{aligned}$

Taking the limit $\alpha\rightarrow 0^{+}$ in the above inequality we have
$F^{{}^{\prime}}_{j}(x,d(x))<0$ for all $j\in\Lambda_{m}.$ This contradicts
with $x$ is a critical point. Therefore, $x$ is not a critical point.

We now prove item (c).

It is easy to see that the function $\max\limits_{j\in\Lambda_{m}}\nabla
f_{j}(x)^{T}d(x)+\frac{1}{2}d(x)^{T}B_{j}(x)d(x)+g_{j}(x+d(x))-g_{j}(x)$ is
continuous with respect to $x$ and $d(x)$. Therefore, the optimal value
function $\theta(x)$ is also continuous from [B63 , Maximum Theorem].
Moreover, since the optimal set mapping $d(x)$ is unique, $d(x)$ is continuous
from [H73 , Corollary 8.1] ∎

###### Lemma 2

Suppose $B_{j}(x)\succeq mI$, a point $x\in\mathbb{R}^{n}$ is critical for $f$
if and only if $d(x)=0$, or equivalently, if and only if $\theta(x)=0.$

Proof The proof of sufficiency is the previous Theorem 2.4.

Conversely suppose $d(x)=0$. Then from condition (7) - (9), there exists
$\lambda\in\Delta_{m}$ such that $\sum\limits_{j\in\Lambda_{m}}(\nabla
f_{j}(x)+\xi_{j})=0.$ Where $\xi_{j}\in\partial g_{j}(x)$ for
$j\in\Lambda_{m}.$ This implies

$\displaystyle\begin{aligned} 0\in
Co\mathop{\cup}\limits_{j\in\Lambda_{m}}\partial{F_{j}(x)}.\end{aligned}$

Hence, $x$ is a critical point of the (MCOP). ∎

We develop a nonmonotone line search technique to find a suitable step length
that allows the objective function values to increase in some iterations. We
introduce a classical average-type nonmonotone line search.

##  3 Nonmonotone Line Search

In nonmonotone line search, some growth in the function value is permitted.
The well-known nonmonotone line search is proposed by Zhang and Hager ZH04 ,
which takes the average value of the continuous function. We introduce an
average-type nonmonotone line search extended to the multiobjective case in
the MFY22 .

$\displaystyle\begin{aligned} F_{j}(x^{k}+\alpha_{k}d^{k})\leq
C_{j}^{k}+\tau\alpha_{k}\theta(x^{k}),\ {\rm for\ all}\ j,\end{aligned}$ (12)

with $\tau\in(0,1)$ and $C_{j}^{k}\geq F_{j}(x^{k})$, where

$\displaystyle\begin{aligned} q_{k+1}=\eta q_{k}+1,\end{aligned}$ (13)
$\displaystyle\begin{aligned} C_{j}^{k+1}=\frac{\eta
q_{k}}{q_{k+1}}C_{j}^{k}+\frac{1}{q_{k+1}}F_{j}(x^{k+1}).\end{aligned}$ (14)

We prove that the average-type nonmonotone line search for proximal quasi-
Newton method of MCOPs is well defined.

###### Theorem 3.1

Suppose $B_{j}(x)\succeq mI$ for every $j$ and $d(x)$ is the solution of (2).
Then

$\theta(x)\leq-\frac{m}{2}\|d(x)\|^{2}.$

Further, if $x$ is noncritical, the inequality (12) holds for every $\alpha>0$
sufficiently small.

Proof Suppose $d(x)$ is the solution of (2) and $\theta(x)=Q(x,d(x))$.

Then there exists $\lambda\in\Delta_{m}$ such that $(d(x),\theta(x),\lambda)$
satisfies conditions (7) - (9). Since $g_{j}$ is convex and
$\xi_{j}\in\partial g_{j}(x+d(x)),$ we have

$\displaystyle\begin{aligned}
g_{j}(x+d(x))-g_{j}(x)\leq\xi_{j}^{T}d(x).\end{aligned}$ (15)

Then, multiplying both sides of (7) by $d(x)$, we obtain

$\sum\limits_{j\in\Lambda_{m}}\lambda_{j}\left(\nabla
f_{j}(x)^{T}d(x)+d(x)^{T}B_{j}(x)d(x)+\xi_{j}^{T}d(x)\right)=0.$

Hence from inequality (15), we get

$\displaystyle\begin{aligned}
&\sum\limits_{j\in\Lambda_{m}}\lambda_{j}\left(\nabla
f_{j}(x)^{T}d(x)+d(x)^{T}B_{j}(x)d(x)+g_{j}(x+d(x))-g_{j}(x)\right)\\\
&\leq\sum\limits_{j\in\Lambda_{m}}\lambda_{j}\left(\nabla
f_{j}(x)^{T}d(x)+d(x)^{T}B_{j}(x)d(x)+\xi_{j}^{T}d(x)\right)\\\ &=0.\\\
\end{aligned}$

Take sum over $j\in\Lambda_{m}$ in (9) and using
$\sum\limits_{j\in\Lambda_{m}}{\lambda_{j}}=1$, we have

$\displaystyle\begin{aligned}
&\sum\limits_{j\in\Lambda_{m}}\lambda_{j}\left(\nabla
f_{j}(x)^{T}d(x)+d(x)^{T}B_{j}(x)d(x)+g_{j}(x+d(x))-g_{j}(x)\right)\\\
&=\frac{1}{2}\sum\limits_{j\in\Lambda_{m}}{\lambda_{j}}d(x)^{T}B_{j}(x)d(x)+\theta(x)\\\
&\leq 0.\end{aligned}$

Using the above two formulas can be obtained

$\displaystyle\begin{aligned}
\theta(x)\leq-\frac{1}{2}\sum\limits_{j\in\Lambda_{m}}{\lambda_{j}}d(x)^{T}B_{j}(x)d(x).\end{aligned}$
(16)

Since $B_{j}(x)\succeq mI$ for every $j$, $d(x)^{T}B_{j}(x)d(x)\geq
m\|d(x)\|^{2}$ holds for every $j$. Then from inequality (16) and
$\sum\limits_{j\in\Lambda_{m}}{\lambda_{j}}=1$, we have

$\displaystyle\begin{aligned}
\theta(x)\leq-\frac{m}{2}\left\|d(x)\right\|^{2}.\end{aligned}$

Suppose $x$ is noncritical, then from Theorem 2.4, we get $d(x)\neq 0$. So,

$\theta(x)\leq-\frac{m}{2}\|d(x)\|^{2}<0.$

Since $g_{j}$ is convex, for any $\alpha\in[0,1]$, we have

$\displaystyle\begin{aligned} F_{j}(x+\alpha d(x))-F_{j}(x)&=f_{j}(x+\alpha
d(x))-f_{j}(x)+g_{j}(x+\alpha d(x))-g_{j}(x)\\\ &\leq\alpha\nabla
f_{j}(x)^{T}d(x)+(1-\alpha)g_{j}(x)+\alpha g_{j}(x+d(x))\\\ &\ \ \
-g_{j}(x)+o(\alpha^{2})\\\ &=\alpha\left(\nabla
f_{j}(x)^{T}d(x)+g_{j}(x+d(x))-g_{j}(x)\right)+o(\alpha^{2})\\\
&\leq\alpha\theta(x)+o(\alpha^{2})\ \ \ \
(\frac{1}{2}d(x)^{T}B_{j}(x)d(x)>0).\end{aligned}$

Then for each $j\in\Lambda_{m}$, subtract $\tau\alpha\theta(x)$ on both sides
of the about inequality, we obtain

$\displaystyle\begin{aligned} F_{j}(x+\alpha
d(x))-C_{j}^{k}-\tau\alpha\theta(x)&<F_{j}(x+\alpha
d(x))-F_{j}(x)-\tau\alpha\theta(x)\\\
&<\alpha(1-\tau)\theta(x)+o(\alpha^{2})\\\ &<0.\end{aligned}$ (17)

Since $\tau\in(0,1)$ and $\theta(x)<0$, the right hand side term in inequality
(17) becomes nonpositive for every $\alpha>0$ sufficiently small. This implies
that the average-type Armijo condition holds for every $\alpha>0$ sufficiently
small, and the proof is complete. ∎

##  4 Nonmonotone proximal quasi-Newton method for MCOPs

The framework for a complete NPQNA for MCOPs is described as follows:

Algorithm 1 NPQNA for MCOPs
---
Step 1 Choose initinal approximation $x^{0}\in\mathbb{R}^{n}$, scalars,
$\rho,\eta,\tau\in(0,1),$
and $\epsilon>0,$ $\mu>0$, $q_{0}=1$, $C_{j}^{0}=F_{j}(x^{0}),$
$B_{j}(x^{0})=I,j\in\Lambda_{m}$,
$0<\eta_{min}\leq\eta_{max}<1,$ set $k:=0.$
Step 2 Solve the subproblem (2) to find $d^{k}$ and $\theta(x^{k}).$
Where, for $k\geq 1$,
$B_{j}(x^{k})=\left\\{\begin{array}[]{lc}B_{j}(x^{k-1})-\frac{B_{j}(x^{k-1})S_{k-1}S_{k-1}^{T}B_{j}(x^{k-1})}{S_{k-1}^{T}B_{j}(x^{k-1})S_{k-1}}+\frac{y_{j}^{k-1}(y_{j}^{k-1})^{T}}{S_{k-1}^{T}y_{j}^{k-1}},&S_{k-1}^{T}y_{j}^{k-1}>0,\\\
B_{j}(x^{k-1}),&otherwise.\\\ \end{array}\right.$
With $S_{k-1}=x^{k}-x^{k-1}$ and $y_{j}^{k-1}=\nabla f_{j}(x^{k})-\nabla
f_{j}(x^{k-1}).$
Step 3 If $|\theta(x^{k})|<\epsilon$, then stop. Else, go to step 4.
Step 4 Take $\alpha_{k}=\mu\rho^{h_{k}}$, where $h_{k}$ is the smallest
nonnegative integer such
that the average-type condition (12) holds.
Where choose $\eta_{min}\leq\eta\leq\eta_{max}$, the update of $q_{k}$ and
$C_{j}^{k}$ is in (13) and (14).
Step 5 Update $x^{k+1}=x^{k}+\alpha_{k}d^{k}$.
Step 6 Set $k=k+1$ and return to Step 2.

##  5 Convergence Analysis

Before proving the convergence of NPQNA for MCOPs, we give the following
related technical lemma.

###### Lemma 3

MFY22  Let $\\{x^{k}\\}$ be the sequence generated by Algorithm 1, and $F_{j}$
is bounded from below. Then, $\\{C_{j}^{k}\\}$ is nonincreasing and admits a
limit when $k\rightarrow\infty.$

###### Lemma 4

MFY22  For a sequence of iterates $\\{x^{k}\\}$ and search directions
$\\{d^{k}\\}$. Then there exist positive constants $\Gamma_{1}\in(0,1)$ such
that

$\displaystyle\begin{aligned} \max_{j\in\Lambda_{m}}\\{\nabla
f_{j}(x^{k})^{T}d^{k}+g_{j}(x^{k}+d^{k})-g_{j}(x^{k})\\}\leq-\Gamma_{1}|\theta(x^{k})|.\end{aligned}$
(18)

###### Lemma 5

Let $\\{x^{k}\\}$ be the sequence generated by Algorithm 1. Assume that
$\eta<1$, Then, we have

${\lim_{k\to\infty}}\alpha_{k}|\theta(x^{k})|=0.$

proof Recall that from Lemma 3. $\\{C_{j}^{k}\\}_{k}$ admits a limit for
$k\to\infty$. From the definition of $C_{j}^{k+1}$ in (14), we get

$\displaystyle\begin{aligned} C_{j}^{k+1}&=\frac{\eta
q_{k}}{q_{k+1}}C_{j}^{k}+\frac{1}{q_{k+1}}F_{j}(x^{k+1})\\\ &\leq\frac{\eta
q_{k}}{q_{k+1}}C_{j}^{k}+\frac{1}{q_{k+1}}(C_{j}^{k}+\tau\alpha_{k}\theta(x^{k}))\\\
&=(\frac{\eta
q_{k}}{q_{k+1}}+\frac{1}{{q_{k+1}}})C_{j}^{k}+\frac{1}{{q_{k+1}}}\tau\alpha_{k}\theta(x^{k})\\\
&=C_{j}^{k}+\frac{1}{{q_{k+1}}}\tau\alpha_{k}\theta(x^{k}),\end{aligned}$

where the above inequality holds from inequality (12) and the last equality
follows from inequality (13). Since $\frac{\tau\alpha_{k}}{{q_{k+1}}}\geq 0$
and $\theta(x^{k})\leq 0$, since $\alpha_{k}=\rho^{h_{k}}$, as $k\to\infty$,
$\alpha_{k}\to 0$, while $q_{k+1}\in(0,1)$. So we have

$\displaystyle\begin{aligned}
\lim_{k\to\infty}\frac{\alpha_{k}}{q_{k+1}}\theta(x^{k})=0.\end{aligned}$ (19)

Furthermore, from inequality (18) in Lemma 4, we obtain

$\displaystyle\begin{aligned} &\frac{\alpha_{k}}{q_{k+1}}\left(\nabla
f_{j}(x^{k})^{T}d^{k}+g_{j}(x^{k}+d^{k})-g_{j}(x^{k})\right)\\\
&\leq-\frac{\alpha_{k}}{q_{k+1}}\Gamma_{1}|\theta(x^{k})|\\\ &\leq
0.\end{aligned}$ (20)

Now, observe that inequality (13) yields

$q_{k+1}=1+\sum\limits_{j=0}^{k}\eta^{j+1}\leq\sum\limits_{j=0}^{+\infty}\eta^{j}=\frac{1}{1-\eta}.$

Since $0<\eta<1$, we have that $\\{\frac{1}{q_{k+1}}\\}$ is bounded from
below. So, we can take the limit in inequality (20) and use inequality (19) to
show that

$\displaystyle\begin{aligned}
\lim_{k\to\infty}\alpha_{k}|\theta(x^{k})|=0,\end{aligned}$

and the conclusion follows. ∎

We will establish the convergence of the NPQNA for MCOPs.

###### Theorem 5.1

Suppose that $\\{x^{k}\\}$ is a sequence generated by Algorithm 1,
$B_{j}(x)\succeq mI$, $f_{j}(x)$ is strongly convex with module $m>0$ and
bounded from below, $\nabla f_{j}$ is Lipschitz continuous. Then every
accumulation point of $\\{x^{k}\\}$ is a Pareto optimal of the (MCOP).

proof Since $\nabla f_{j}$ is Lipschitz continuous for every $j$ with
Lipschitz constant $L_{j}$, where $L=\max\limits_{j\in\Lambda_{m}}L_{j}$, from
the so-called descent Lemma (B99, , Proposition A.24) for any $\alpha$ we have

$\displaystyle\begin{aligned} f_{j}(x^{k}+\alpha d^{k})&\leq
f_{j}(x^{k})+\alpha\nabla
f_{j}(x^{k})^{T}d^{k}+\frac{L}{2}\alpha^{2}\|d^{k}\|^{2},\ \forall
j\in\Lambda_{m},\end{aligned}$ (21)

also since $g_{j}$ is convex, for any $\alpha\in(0,1)$, we have

$\displaystyle\begin{aligned} g_{j}(x^{k}+\alpha
d^{k})-g_{j}(x^{k})&=g_{j}\left(\alpha(x^{k}+d^{k})+(1-\alpha)x^{k}\right)-g_{j}(x^{k})\\\
&\leq\alpha g_{j}(x^{k}+d^{k})+(1-\alpha)g_{j}(x^{k})-g_{j}(x^{k})\\\
&=\alpha\left(g_{j}(x^{k}+d^{k})-g_{j}(x^{k})\right).\\\ \end{aligned}$ (22)

From inequality (21) and inequality (22) for any $\alpha\in(0,1)$, we get

$\displaystyle\begin{aligned} F_{j}(x^{k}+\alpha
d^{k})-F_{j}(x^{k})&=f_{j}(x^{k}+\alpha d^{k})+g_{j}(x^{k}+\alpha
d^{k})-f_{j}(x^{k})-g_{j}(x^{k})\\\ &\leq\alpha\nabla
f_{j}(x^{k})^{T}d^{k}+\frac{L}{2}\alpha^{2}\|d^{k}\|^{2}+\alpha\left(g_{j}(x^{k}+d^{k})-g_{j}(x^{k})\right)\\\
&\leq\alpha\left(\nabla
f_{j}(x^{k})^{T}d^{k}+g_{j}(x^{k}+d^{k})-g_{j}(x^{k})\right)+\frac{L}{2}\alpha^{2}\|d^{k}\|^{2}\\\
&\leq\alpha\left(\nabla
f_{j}(x^{k})^{T}d^{k}+\frac{1}{2}(d^{k})^{T}B_{k}d^{k}+g_{j}(x^{k}+d^{k})-g_{j}(x^{k})\right)\\\
&\ \ \ +\frac{L}{2}\alpha^{2}\|d^{k}\|^{2}\\\
&\leq\alpha\theta(x^{k})+\frac{L}{2}\alpha^{2}\|d^{k}\|^{2},\
j\in\Lambda_{m},\\\ \end{aligned}$

where the third inequality holds since $B_{j}(x)\succeq mI$. From Step 4 of
Algorithm 1, either $\alpha_{k}\geq\rho$ or there exists $h_{k}$ such that
$\alpha_{k}=\rho^{h_{k}}<\rho$ satisfies inequality (12). Since
$F_{j}(x^{k})\leq C_{j}^{k}$, if $\alpha=\frac{\alpha_{k}}{\rho}$ doesn’t
satisfy inequality (12), we can get

$\displaystyle\begin{aligned}
F_{j}(x^{k}+\frac{\alpha_{k}}{\rho}d^{k})&>C_{j}^{k}+\tau\frac{\alpha_{k}}{\rho}\theta(x^{k})\\\
&\geq F_{j}(x^{k})+\tau\frac{\alpha_{k}}{\rho}\theta(x^{k}),\
j\in\Lambda_{m}.\\\ \end{aligned}$

From the above two inequalities, we have

$\displaystyle\begin{aligned}
\tau\frac{\alpha_{k}}{\rho}\theta(x^{k})\leq\frac{\alpha_{k}}{\rho}\theta(x^{k})+\frac{L}{2}(\frac{\alpha_{k}}{\rho})^{2}\|d^{k}\|^{2}.\end{aligned}$

Simplify the above inequality

$\displaystyle\begin{aligned}
(\tau-1)\theta(x^{k})\leq\frac{L}{2}\frac{\alpha_{k}}{\rho}\|d^{k}\|^{2}.\end{aligned}$

Therefore

$\displaystyle\begin{aligned}
\alpha_{k}\geq\frac{2\rho(1-\tau)}{L}\frac{|\theta(x^{k})|}{\|d^{k}\|^{2}}.\end{aligned}$

Furthermore, by inequality (12), we have

$\displaystyle\begin{aligned} F_{j}(x^{k+1})\leq
C_{j}^{k}+\tau\frac{2\rho(1-\tau)}{L}\frac{|\theta(x^{k})|}{\|d^{k}\|^{2}}\theta(x^{k}),\
j\in\Lambda_{m}.\end{aligned}$

So,

$\displaystyle\begin{aligned} F_{j}(x^{k+1})\leq
C_{j}^{k}-\frac{2\tau\rho(1-\tau)}{L}\frac{|\theta(x^{k})|^{2}}{\|d^{k}\|^{2}},\
j\in\Lambda_{m}.\end{aligned}$

By Theorem 3.1, we have

$\displaystyle\begin{aligned}
|\theta(x^{k})|\geq\frac{m}{2}\|d(x^{k})\|^{2}.\end{aligned}$

Finally, we have

$\displaystyle\begin{aligned} F_{j}(x^{k+1})&\leq
C_{j}^{k}-\frac{m}{2}\frac{2\tau\rho(1-\tau)}{L}|\theta(x^{k})|\\\
&=C_{j}^{k}-\frac{m\tau\rho(1-\tau)}{L}|\theta(x^{k})|,\ j\in\Lambda_{m}.\\\
\end{aligned}$ (23)

Denote $\frac{m\tau\rho(1-\tau)}{L}$ as $\beta$, combining the relations (13)
and (14) with inequality (23), we obtain:

$\displaystyle\begin{aligned} C_{j}^{k+1}&=\frac{\eta
q_{k}C_{j}^{k}+F_{j}(x^{k+1})}{q_{k+1}}\\\ &\leq\frac{\eta
q_{k}C_{j}^{k}+C_{j}^{k}-\beta|\theta(x^{k})|}{q_{k+1}}\\\
&=C_{j}^{k}-\frac{\beta|\theta(x^{k})|}{q_{k+1}},\ \forall
j\in\Lambda_{m}.\end{aligned}$ (24)

Since $F_{j},j\in\Lambda_{m}$ are bounded from below and $F_{j}(x^{k})\leq
C_{j}^{k}$, for all $k$, we conclude that $C_{j}^{k},j\in\Lambda_{m}$ are
bounded from below. It follows from (24) and Lemma (3) that

$\displaystyle\begin{aligned}
\sum\limits_{k=0}^{+\infty}\frac{|\theta(x^{k})|}{q_{k+1}}\leq
C_{j}^{0}-C^{\infty}_{j}<\infty.\end{aligned}$ (25)

Since $f_{j}$ is strongly convex and continuously differentiable, $g_{j}$ is
convex and continuous, the level set
$\mathcal{L}_{F}(x^{0})\subset\\{x:F_{j}(x)\leq C_{j}^{0}\\}$ is compact. We
can get $\\{x^{k}\\}\subset\mathcal{L}_{F}(x^{0})$ by the fact that
$F_{j}(x^{k})\leq C_{j}^{k}\leq C_{j}^{0}$. From the compactness of
$\mathcal{L}_{F}(x^{0})$, it follows that $\\{x^{k}\\}$ has an accumulation
point. Now, suppose that $x^{*}$ is an accumulation point of the sequence
$\\{x^{k}\\}$. So, there exists a subsequence $\\{x^{k}\\}_{k\in K}$ which
converges to $x^{*}$. Then we can prove $\theta(x^{*})=0$, by contradiction.
Assume $\theta(x^{*})<0$, which implies that there are $\epsilon>0$ and
$\delta_{0}>0$ so that for all $0<\delta<\delta_{0}$ and for all $k\in K$ such
that $|x^{k}-x^{*}|\leq\delta$, we have

$\displaystyle\begin{aligned} |\theta(x^{k})|\geq\epsilon>0.\end{aligned}$

This means that

$\displaystyle\begin{aligned}
\sum\limits_{k=0}^{+\infty}\frac{|\theta(x^{k})|}{q_{k+1}}\geq\sum\limits_{k\in\\{k\in
K|\|x^{k}-x^{*}\|\leq\delta\\}}\frac{\epsilon}{q_{k+1}}.\end{aligned}$ (26)

According to inequality (13) and $\eta_{max}<1$, we have

$\displaystyle\begin{aligned}
q_{k+1}&=1+\sum\limits_{i=0}^{k}\prod\limits_{l=0}^{i}\eta^{k-l}\leq
1+\sum\limits_{i=0}^{k}\eta^{i}_{max}\\\
&\leq\sum\limits_{i=0}^{+\infty}\eta^{i}_{max}=\frac{1}{1-\eta_{max}}.\end{aligned}$

Consequently, following inequality (26), we have

$\displaystyle\begin{aligned}
\sum\limits_{k=0}^{+\infty}\frac{|\theta(x^{k})|}{q_{k+1}}&\geq\sum\limits_{k\in\\{k\in
K|\|x^{k}-x^{*}\|\leq\delta\\}}\frac{\epsilon}{q_{k+1}}\\\
&\geq\sum\limits_{k\in\\{k\in
K|\|x^{k}-x^{*}\|\leq\delta\\}}(1-\eta_{max})\epsilon\\\
&=+\infty.\end{aligned}$

This contradicts inequality (25). Hence, we have $\theta(x^{*})=0$ and it
follows from Theorem 2.4 and Lemma 2 that $x^{*}$ is critical for $F$, since
$F$ is $\mathbb{R}^{m}_{+}$-strongly convex, on the basis of Proposition 1, so
$x^{*}$ is Pareto optimal for $F$. ∎

We can get the strong convergence of NPQNA for MCOPs.

###### Theorem 5.2

Suppose $f_{j}$ is strongly convex with module $m>0$ for all
$j\in\Lambda_{m}$. Let $\\{x^{k}\\}$ be the sequence generated by Algorithm 1.
Then $\\{x^{k}\\}$ converges to some Pareto optimal $x^{*}$.

proof

According to Theorem 5.1, we have that $x^{*}$ is a Pareto optimal of $F$. By
Lemma 3, we have $\\{C^{k}\\}$ is nonincreasing and admits a limit when
$k\rightarrow\infty.$ Let $\lim\limits_{k\rightarrow\infty}C^{k}=F^{*}$.
According to the definition of $C^{k}$ in (14) and the definition of $q_{k}$
in (13) , we have

$F(x^{k})=q_{k}C^{k}-\eta q_{k-1}C^{k-1},$

Since the limits of $\\{C^{k}\\}$ exist, we take the limits on both sides of
the above equation and obtain

$\displaystyle\begin{aligned}
\lim\limits_{k\rightarrow\infty}F(x^{k})&=\lim\limits_{k\rightarrow\infty}q_{k}C^{k}-\eta
q_{k-1}C^{k-1}\\\ &=\frac{1}{1-\eta}F^{*}-\frac{\eta}{1-\eta}F^{*}\\\
&=F^{*}\end{aligned}$

Next, we prove the uniqueness of $x^{*}$ by proof by contradiction. Suppose
instead that there is another accumulation point $x_{1}^{*}$. Due to the
strong convexity of $F$, the following inequality holds:

$F(\lambda x^{*}+(1-\lambda)x_{1}^{*})\prec\lambda
F(x^{*})+(1-\lambda)F(x_{1}^{*})=F^{*},$

where the equality follows from the convergence of $\\{F(x^{k})\\}$. However,
this contradicts the fact that $x^{*}$ is a Pareto optimal. The uniqueness of
accumulation point of $\\{x^{k}\\}$ implies that $\\{x^{k}\\}$ converges to
$x^{*}$. ∎

Referring to the proof of the convergence rate of the Newton-type proximal
gradient method proposed by chen et al. in CJTY23 , we establish the local
superlinear convergence of the NPQNA for MCOPs.

In the same way, we denote by

$h_{\lambda}(x):=\sum\limits_{j\in\Lambda_{m}}\lambda_{j}h_{j}(x),$ $\nabla
h_{\lambda}(x):=\sum\limits_{j\in\Lambda_{m}}\lambda_{j}\nabla h_{j}(x),$
$\nabla^{2}h_{\lambda}(x):=\sum\limits_{j\in\Lambda_{m}}\lambda_{j}\nabla^{2}h_{j}(x).$

According to Sion’s minimax theorem S58 , there exists
$\lambda^{k}\in\Delta_{m}$ such that

$d^{k}=\mathop{\arg\min}\limits_{d\in\mathbb{R}^{n}}\left\\{\nabla
f_{\lambda^{k}}(x^{k})^{T}d+\frac{1}{2}d^{T}B_{\lambda^{k}}(x^{k})d+g_{\lambda^{k}}(x^{k}+d)-g_{\lambda^{k}}(x^{k})\right\\},$

by means of the first-order optimality condition, we have

$\displaystyle\begin{aligned} -\nabla
f_{\lambda^{k}}(x^{k})-B_{\lambda^{k}}(x^{k})d^{k}\in\partial
g_{\lambda^{k}}(x^{k}+d^{k}).\end{aligned}$ (27)

Similar to the modified fundamental inequality in CJTY23 , we propose the
following inequality.

###### Proposition 2

Suppose $f_{j}$ is strictly convex for all $j\in\Lambda_{m}$. Let
$\\{x^{k}\\}$ be the sequence generated by Algorithm 1. If $\alpha_{k}=1$,
then there exists $\lambda^{k}\in\Delta_{m}$ such that

$\displaystyle\begin{aligned} &F_{\lambda^{k}}(x^{k+1})-F_{\lambda^{k}}(x)\\\
&\leq\frac{1}{2}\|x^{k}-x\|^{2}_{B_{\lambda^{k}}}-\frac{1}{2}\|x^{k+1}-x^{k}\|^{2}_{B_{\lambda^{k}}}-\frac{1}{2}\|x^{k+1}-x\|^{2}_{B_{\lambda^{k}}}\\\
&\ \ \ +\langle
x^{k+1}-x^{k},\int_{0}^{1}\int_{0}^{1}\nabla^{2}f_{\lambda^{k}}(x^{k}+st(x^{k+1}-x^{k}))ds(t(x^{k+1}-x^{k}))dt\rangle\\\
&\ \ \ -\langle
x-x^{k},\int_{0}^{1}\int_{0}^{1}\nabla^{2}f_{\lambda^{k}}(x^{k}+st(x-x^{k}))ds(t(x-x^{k}))dt\rangle.\end{aligned}$
(28)

proof In the light of the twice continuity of $f_{j}$, we can deduce that

$\displaystyle\begin{aligned} &F_{j}(x^{k+1})-F_{j}(x)\\\
&=f_{j}(x^{k+1})-f_{j}(x^{k})-(f_{j}(x)-f_{j}(x^{k}))+g_{j}(x^{k+1})-g_{j}(x)\\\
&=\langle\nabla f_{j}(x^{k}),x^{k+1}-x^{k}\rangle+\langle\nabla
f_{j}(x^{k}),x^{k}-x\rangle+g_{j}(x^{k+1})-g_{j}(x)\\\ &\ \ \ +\langle
x^{k+1}-x^{k},\int_{0}^{1}\int_{0}^{1}\nabla^{2}f_{j}(x^{k}+st(x^{k+1}-x^{k}))ds(t(x^{k+1}-x^{k}))dt\rangle\\\
&\ \ \ -\langle
x-x^{k},\int_{0}^{1}\int_{0}^{1}\nabla^{2}f_{j}(x^{k}+st(x-x^{k}))ds(t(x-x^{k}))dt\rangle\\\
&=\langle\nabla f_{j}(x^{k}),x^{k+1}-x\rangle+g_{j}(x^{k+1})-g_{j}(x)\\\ &\ \
\ +\langle
x^{k+1}-x^{k},\int_{0}^{1}\int_{0}^{1}\nabla^{2}f_{j}(x^{k}+st(x^{k+1}-x^{k}))ds(t(x^{k+1}-x^{k}))dt\rangle\\\
&\ \ \ -\langle
x-x^{k},\int_{0}^{1}\int_{0}^{1}\nabla^{2}f_{j}(x^{k}+st(x-x^{k}))ds(t(x-x^{k}))dt\rangle.\end{aligned}$

On the other hand, from (27), we have

$-\nabla f_{\lambda^{k}}(x^{k})-B_{\lambda^{k}}(x^{k})d^{k}\in\partial
g_{\lambda^{k}}(x^{k}+d^{k}),\lambda^{k}\in\Delta_{m}.$

This, together with the fact that $\alpha_{k}=1$, implies

$\displaystyle\begin{aligned}
g_{\lambda^{k}}(x^{k+1})-g_{\lambda^{k}}(x)&\leq\langle-\nabla
f_{\lambda^{k}}(x^{k})-B_{\lambda^{k}}(x^{k})d^{k},x^{k+1}-x\rangle\\\
&=\langle-\nabla
f_{\lambda^{k}}(x^{k})-B_{\lambda^{k}}(x^{k})(x^{k+1}-x^{k}),x^{k+1}-x\rangle.\end{aligned}$

with the help of the last two relations, we have

$\displaystyle\begin{aligned} &F_{\lambda^{k}}(x^{k+1})-F_{\lambda^{k}}(x)\\\
&\leq\langle B_{\lambda^{k}}(x^{k})(x^{k}-x^{k+1}),x^{k+1}-x\rangle\\\ &\ \ \
+\langle
x^{k+1}-x^{k},\int_{0}^{1}\int_{0}^{1}\nabla^{2}f_{\lambda^{k}}(x^{k}+st(x^{k+1}-x^{k}))ds(t(x^{k+1}-x^{k}))dt\rangle\\\
&\ \ \ -\langle
x-x^{k},\int_{0}^{1}\int_{0}^{1}\nabla^{2}f_{\lambda^{k}}(x^{k}+st(x-x^{k}))ds(t(x-x^{k}))dt\rangle\\\
&=\frac{1}{2}\|x^{k}-x\|^{2}_{B_{\lambda^{k}}}-\frac{1}{2}\|x^{k+1}-x^{k}\|^{2}_{B_{\lambda^{k}}}-\frac{1}{2}\|x^{k+1}-x\|^{2}_{B_{\lambda^{k}}}\\\
&\ \ \ +\langle
x^{k+1}-x^{k},\int_{0}^{1}\int_{0}^{1}\nabla^{2}f_{\lambda^{k}}(x^{k}+st(x^{k+1}-x^{k}))ds(t(x^{k+1}-x^{k}))dt\rangle\\\
&\ \ \ -\langle
x-x^{k},\int_{0}^{1}\int_{0}^{1}\nabla^{2}f_{\lambda^{k}}(x^{k}+st(x-x^{k}))ds(t(x-x^{k}))dt\rangle.\\\
\end{aligned}$

∎

Again, assuming that $f_{j}$ is twice continuously differentiable and strongly
convex with constant $m$, and $\nabla^{2}f_{j}$ are continuous, and the
following assumption holds.

###### Assumption 5.1

AS20  For all $\epsilon>0$ there exist $k^{0}\in\mathbb{N}$ such that for all
$k\geq k^{0}$, we have

$\displaystyle\begin{aligned}
\frac{\left\|(\nabla^{2}f_{j}(x^{*})-B_{j}(x^{k}))d^{k}\right\|}{\|d^{k}\|}\leq\epsilon,j\in\Lambda_{m},\end{aligned}$
(29)

where $\\{x^{k}\\}$ is a sequence generated by Algorithm 1 and
$\\{B_{j}(x^{k})\\},j\in\Lambda_{m},$ are the sequences obtained by the BFGS
updates.

$\displaystyle\begin{aligned}
B_{j}(x^{k+1})=B_{j}(x^{k})-\frac{B_{j}(x^{k})s^{k}(s^{k})^{T}B_{j}(x^{k})}{(s^{k})^{T}B_{j}(x^{k})s^{k}}+\frac{y_{j}^{k}(y_{j}^{k})^{T}}{(y_{j}^{k})^{T}s^{k}},\end{aligned}$

where

$\displaystyle\begin{aligned} s^{k}=x^{k+1}-x^{k},\ y_{j}^{k}=\nabla
f_{j}(x^{k+1})-\nabla f_{j}(x^{k}).\end{aligned}$

The Assumption 5.1 is also called Dennis-Moré criterion.

###### Lemma 6

Suppose $f_{j}$ is twice continuously differentiable and strongly convex with
constant $m$, $\nabla^{2}f_{j}(x^{k})$ is continuous, the sequence
$\\{B_{j}(x^{k})\\}$ satisfies the Assumption (5.1). Let $\\{x^{k}\\}$ be the
sequence generated by Algorithm 1. Then, for any
$0<\epsilon\leq\frac{m(1-\tau)}{3}$, the unit step length satisfies the
nonmonotone line search conditions (12) after sufficiently many iterations.

proof

Since $f_{j}$ is twice continuously differentiable, we have

$\displaystyle\begin{aligned} f_{j}(x^{k}+d^{k})\leq f_{j}(x^{k})+\nabla
f_{j}(x^{k})^{T}d^{k}+\frac{1}{2}(d^{k})^{T}\nabla^{2}f_{j}(x^{k})d^{k}+\frac{\epsilon}{2}\left\|d^{k}\right\|^{2},\end{aligned}$

we add $g_{j}(x^{k}+d^{k})$ to both sides to obtain

$\displaystyle\begin{aligned} F_{j}(x^{k}+d^{k})\leq f_{j}(x^{k})+\nabla
f_{j}(x^{k})^{T}d^{k}+\frac{1}{2}(d^{k})^{T}\nabla^{2}f_{j}(x^{k})d^{k}+\frac{\epsilon}{2}\left\|d^{k}\right\|^{2}+g_{j}(x^{k}+d^{k}),\end{aligned}$

we then add and subtract $g_{j}(x^{k})$ from the right-hand side to obtain

$\displaystyle\begin{aligned} F_{j}(x^{k}+d^{k})&\leq
f_{j}(x^{k})+g_{j}(x^{k})+\nabla
f_{j}(x^{k})^{T}d^{k}+g_{j}(x^{k}+d^{k})-g_{j}(x^{k})\\\ &\ \ \
+\frac{1}{2}(d^{k})^{T}\nabla^{2}f_{j}(x^{k})d^{k}+\frac{\epsilon}{2}\left\|d^{k}\right\|^{2}+\frac{1}{2}(d^{k})^{T}B_{j}(x^{k})d^{k}\\\
&\ \ \ -\frac{1}{2}(d^{k})^{T}B_{j}(x^{k})d^{k}\\\ &\leq
F_{j}(x^{k})+\theta(x^{k})+\frac{1}{2}(d^{k})^{T}\left(\nabla^{2}f_{j}(x^{k})-\nabla^{2}f_{j}(x^{*}))\right)d^{k}\\\
&\ \ \
+\frac{1}{2}(d^{k})^{T}\left(\nabla^{2}f_{j}(x^{*})-B_{j}(x^{k})\right)d^{k}+\frac{\epsilon}{2}\left\|d^{k}\right\|^{2}\\\
&\leq
F_{j}(x^{k})+\theta(x^{k})+\frac{3\epsilon}{2}\left\|d^{k}\right\|^{2}\\\
&\leq
F_{j}(x^{k})+\tau\theta(x^{k})+(1-\tau)\theta(x^{k})+\frac{3\epsilon}{2}\left\|d^{k}\right\|^{2}\\\
&\leq
F_{j}(x^{k})+\tau\theta(x^{k})+(\frac{3\epsilon}{2}-\frac{m(1-\tau)}{2})\left\|d^{k}\right\|^{2}\\\
&\leq C_{j}^{k}+\tau\theta(x^{k}),\end{aligned}$ (30)

where the third inequality comes from the Assumption 5.1 and $\nabla^{2}f_{j}$
is continuous. The second to last inequality is derived from the Theorem 3.1,
and $0<\epsilon\leq\frac{(1-\tau)m}{3}$, we have

$\displaystyle\begin{aligned} \frac{3\epsilon}{2}-\frac{m(1-\tau)}{2}\leq
0,\end{aligned}$

so, the last inequality holds. therefore, the nonmonotone line search
conditions hold for $\alpha_{k}=1.$ ∎

###### Theorem 5.3

Suppose $f_{j}$ is strongly convex with module $m>0$, and its Hessian is
continuous for $j\in\Lambda_{m}$, and $\\{B_{j}(x^{k})\\}$ satisfies the
Assumption 5.1 and $B_{j}(x^{k})\succeq mI$ for some $m>0$. Let $\\{x^{k}\\}$
denote a bounded sequence generated by Algorithm 1. Then, for any
$0<\epsilon\leq\frac{m(1-\tau)}{3}$, there exists $K_{\epsilon}>0$ such that

$\displaystyle\begin{aligned}
\|x^{k+1}-x^{*}\|\leq\sqrt{\frac{3\epsilon(1+\tau_{k}^{2})}{m}}\|x^{k}-x^{*}\|.\end{aligned}$

holds for all $k\geq K_{\epsilon}$, where
$\tau_{k}:=\frac{\|x^{k+1}-x^{k}\|}{\|x^{k}-x^{*}\|}\in[\frac{m-\sqrt{6m\epsilon-9\epsilon^{2}}}{m-3\epsilon},\frac{m+\sqrt{6m\epsilon-9\epsilon^{2}}}{m-3\epsilon}]$.
Furthermore, the sequence $\\{x^{k}\\}$ converges superlinearly to $x^{*}$.

proof Since the assumptions of Lemma 6 are satisfied, unit step lengths
satisfy the nonmonotone line search conditions after sufficiently many
iterations:

$\alpha_{k}=1,$ $x^{k+1}=x^{k}+d^{k}.$

Substituting $x=x^{*}$ into inequality (28), we obtain

$\displaystyle\begin{aligned} 0&\leq
F_{\lambda^{k}}(x^{k+1})-F_{\lambda^{k}}(x^{*})\\\
&\leq\frac{1}{2}\|x^{k}-x^{*}\|^{2}_{B_{\lambda^{k}}(x^{k})}-\frac{1}{2}\|x^{k+1}-x^{k}\|^{2}_{B_{\lambda^{k}}(x^{k})}-\frac{1}{2}\|x^{k+1}-x^{*}\|^{2}_{B_{\lambda^{k}}(x^{k})}\\\
&\ \ \ +\langle
x^{k+1}-x^{k},\int_{0}^{1}\int_{0}^{1}\nabla^{2}f_{\lambda^{k}}(x^{k}+st(x^{k+1}-x^{k}))ds(t(x^{k+1}-x^{k}))dt\rangle\\\
&\ \ \ -\langle
x^{*}-x^{k},\int_{0}^{1}\int_{0}^{1}\nabla^{2}f_{\lambda^{k}}(x^{k}+st(x^{*}-x^{k}))ds(t(x^{*}-x^{k}))dt\rangle.\end{aligned}$
(31)

where the first inequality comes from the fact $F(x^{*})\preceq F(x^{k})$ for
all $k$. On the other hand,

$\frac{1}{2}\|x-x^{k}\|^{2}_{B_{\lambda^{k}}(x^{k})}=\langle
x-x^{k},\int_{0}^{1}\int_{0}^{1}B_{\lambda^{k}}(x^{k})ds(t(x-x^{k}))dt\rangle,$

then, inserting $x=x^{*}$ and $x=x^{k+1}$ respectively into the above formula,
we have

$\frac{1}{2}\|x^{*}-x^{k}\|^{2}_{B_{\lambda^{k}}(x^{k})}=\langle
x^{*}-x^{k},\int_{0}^{1}\int_{0}^{1}B_{\lambda^{k}}(x^{k})ds(t(x^{*}-x^{k}))dt\rangle,$
$\frac{1}{2}\|x^{k+1}-x^{k}\|^{2}_{B_{\lambda^{k}}(x^{k})}=\langle
x^{k+1}-x^{k},\int_{0}^{1}\int_{0}^{1}B_{\lambda^{k}}(x^{k})ds(t(x^{k+1}-x^{k}))dt\rangle.$

Since the accumulation point of $\\{x^{k}\\}$ is Pareto optimal, there exists
$K_{\epsilon}\geq K_{\epsilon}^{1}$ such that, for all $k\geq K_{\epsilon}$,

$\|\nabla^{2}f_{\lambda^{k}}(x^{k}+st(x^{k+1}-x^{k})-\nabla^{2}f_{\lambda^{k}}(x^{k})\|\leq\epsilon,\forall
s,t\in[0,1],$
$\|\nabla^{2}f_{\lambda^{k}}(x^{k}+st(x^{*}-x^{k})-\nabla^{2}f_{\lambda^{k}}(x^{k})\|\leq\epsilon,\forall
s,t\in[0,1].$

Substituting the above relation into (31), we obtain

$\displaystyle\begin{aligned}
&\frac{1}{2}\|x^{k+1}-x^{*}\|^{2}_{B_{\lambda^{k}}(x^{k})}\\\ &\leq\langle
x^{k+1}-x^{k},\int_{0}^{1}\int_{0}^{1}(\nabla^{2}f_{\lambda^{k}}(x^{k}+st(x^{k+1}-x^{k}))-B_{\lambda^{k}}(x^{k}))ds(t(x^{k+1}-x^{k}))dt\rangle\\\
&\ \ \ -\langle
x^{*}-x^{k},\int_{0}^{1}\int_{0}^{1}(\nabla^{2}f_{\lambda^{k}}(x^{k}+st(x^{*}-x^{k}))-B_{\lambda^{k}}(x^{k}))ds(t(x^{*}-x^{k}))dt\rangle\\\
&\leq\frac{\epsilon}{2}\|x^{k+1}-x^{k}\|^{2}+\frac{\epsilon}{2}\|x^{*}-x^{k}\|^{2}\\\
&\ \ \ +\langle
x^{k+1}-x^{k},\int_{0}^{1}\int_{0}^{1}(\nabla^{2}f_{\lambda^{k}}(x^{k})-B_{\lambda^{k}}(x^{k}))ds(t(x^{k+1}-x^{k}))dt\rangle\\\
&\ \ \ -\langle
x^{*}-x^{k},\int_{0}^{1}\int_{0}^{1}(\nabla^{2}f_{\lambda^{k}}(x^{k})-B_{\lambda^{k}}(x^{k}))ds(t(x^{*}-x^{k}))dt\rangle\\\
&\leq\epsilon\|x^{k+1}-x^{k}\|^{2}+\epsilon\|x^{*}-x^{k}\|^{2}\\\ &\ \ \
+\langle
x^{k+1}-x^{k},\int_{0}^{1}\int_{0}^{1}(\nabla^{2}f_{\lambda^{k}}(x^{*})-B_{\lambda^{k}}(x^{k}))ds(t(x^{k+1}-x^{k}))dt\rangle\\\
&\ \ \ -\langle
x^{*}-x^{k},\int_{0}^{1}\int_{0}^{1}(\nabla^{2}f_{\lambda^{k}}(x^{*})-B_{\lambda^{k}}(x^{k}))ds(t(x^{*}-x^{k}))dt\rangle\\\
&\leq\frac{3\epsilon}{2}\|x^{k+1}-x^{k}\|^{2}+\frac{3\epsilon}{2}\|x^{*}-x^{k}\|^{2}.\end{aligned}$

where the last inequality holds because $\nabla^{2}f_{j}$ is continuous and
the Assumption (5.1) holds. This together with $B_{j}\succeq mI$, implies

$\displaystyle\begin{aligned} m\|x^{k+1}-x^{*}\|^{2}\leq
3\epsilon\|x^{k+1}-x^{k}\|^{2}+3\epsilon\|x^{*}-x^{k}\|^{2}.\end{aligned}$
(32)

By direct calculation, we have

$\displaystyle\begin{aligned}
&3\epsilon\|x^{k+1}-x^{k}\|^{2}+3\epsilon\|x^{*}-x^{k}\|^{2}\\\ &\geq
m\|x^{k+1}-x^{*}\|^{2}\\\ &=m\|x^{k+1}-x^{k}+x^{k}-x^{*}\|^{2}\\\ &\geq
m\|x^{k+1}-x^{k}\|^{2}+m\|x^{k}-x^{*}\|^{2}-2m\|x^{k+1}-x^{k}\|\|x^{k}-x^{*}\|.\end{aligned}$

Rearranging, we have

$\displaystyle\begin{aligned}
2m\|x^{k+1}-x^{k}\|\|x^{k}-x^{*}\|\geq(m-3\epsilon)\|x^{k+1}-x^{k}\|^{2}+(m-3\epsilon)\|x^{k}-x^{*}\|^{2},\end{aligned}$

dividing by $\|x^{k}-x^{*}\|^{2}$, we get

$\displaystyle\begin{aligned}
2m\frac{\|x^{k+1}-x^{k}\|}{\|x^{k}-x^{*}\|}\geq(m-3\epsilon)\frac{\|x^{k+1}-x^{k}\|^{2}}{\|x^{k}-x^{*}\|^{2}}+(m-3\epsilon),\end{aligned}$

let $\tau_{k}=\frac{\|x^{k+1}-x^{k}\|}{\|x^{k}-x^{*}\|}$, so we get

$\displaystyle\begin{aligned}
(m-3\epsilon)\tau_{k}^{2}-2m\tau_{k}+m-3\epsilon\leq 0.\end{aligned}$

Since $\epsilon\leq\frac{m(1-\tau)}{3}$, we deduce that
$\tau_{k}\in[\frac{m-\sqrt{6m\epsilon-9\epsilon^{2}}}{m-3\epsilon},\frac{m+\sqrt{6m\epsilon-9\epsilon^{2}}}{m-3\epsilon}]$.
Substituting $\tau_{k}$ into relation (32), we derive that

$\displaystyle\begin{aligned}
\|x^{k+1}-x^{*}\|\leq\sqrt{\frac{3\epsilon(1+\tau_{k}^{2})}{m}}\|x^{k}-x^{*}\|.\end{aligned}$

Furthermore, since $\epsilon$ tends to 0 as $k$ tends to infinity, it follows
that

$\displaystyle\begin{aligned}
\lim_{k\rightarrow\infty}\tau_{k}\in\lim_{\epsilon\rightarrow
0}[\frac{m-\sqrt{6m\epsilon-9\epsilon^{2}}}{m-3\epsilon},\frac{m+\sqrt{6m\epsilon-9\epsilon^{2}}}{m-3\epsilon}]=\\{1\\}.\end{aligned}$

We use the relation to get

$\displaystyle\begin{aligned}
\lim_{k\rightarrow\infty}\frac{\|x^{k+1}-x^{*}\|}{\|x^{k}-x^{*}\|}=0.\end{aligned}$

This concludes the proof. ∎

##  6 Numerical Experiment

In this section, we presents some numerical experiments in order to illustrate
the applicability of our Algorithm NPQNA for MCOPs. Based on it, we compare:

  * •

A nonmonotone proximal quasi-Newton algorithm (NPQNA) for MCOPs.

  * •

A proximal quasi-Newton algorithm (PQNA) for MCOPs in PR22 .

  * •

A Newton-type proximal gradient algorithm (NPGA) for MCOPs in A22 .

We use python 3.9 to program the Algorithm 1, and run on a computer with CPU
Intel Core i7 2.90GHz and 32GB of memory, and the subproblems is solved by the
solver ’SLSQP’ in python. $\|d^{k}\|\leq 10^{-6}$ or maximum 300 iterations is
considered as stopping criteria. Solution of a multiobjective optimization
problem is not isolated optimum points, but a set of efficient solutions. To
generate an approximate set of efficient solutions we have considered multi-
start technique. Following steps are executed in this technique.
(1) A set of 100 uniformly distributed random initial points between lower
bounds $lb$ and upper bounds $ub$ are considered, where $lb$,
$ub\in\mathbb{R}^{n}$ and $lb<ub$.
(2) Algorithm 1 is executed individually.
(3) We implemented the methods using the nonmonotone line search step size
strategy with parameters $\eta=10^{-4}$.

Test Problems: We have constructed a set of nonlinear 2 or 3 objective
optimization problems to illustrate the effectiveness of our proposed NPQNA
for MCOPs and found a good pareto front. Some differentiable multiobjective
test problems $f_{j}(x)$ are paired with the nonsmooth multiobjective problems
listed below. Details are provided in Table 1. The dimension of variables and
objective functions are presented in the second columns.

Since the subproblem is a convex nonsmooth quadratic constrained optimization
problem, where the objective function is a linear function and the constraint
is a quadratic constraint function with a nonsmooth term.

$\displaystyle\begin{aligned} &\min\ t\\\ &s.t.\ \nabla
f_{j}(x)^{T}d+\frac{1}{2}d^{T}B_{j}(x)d+g_{j}(x+d)-g_{j}(x)-t\leq 0,\
j\in\Lambda_{m},\\\ &\ \ \ \ \ \
(t,d)\in\mathbb{R}\times\mathbb{R}^{n}.\end{aligned}$ (33)

We refer to the way of dealing with nonsmooth term $g_{j}$ in AFP23 , when we
similar consider the linear programming with box constraints in the nonsmooth
part of the original multiobjective optimization problem. For each $j\in J$,
we assume that ${\rm dom}(g_{j})=\\{x\in\mathbb{R}^{n}|lb\preceq x\preceq
ub\\}=:C$, where define $g_{j}:C\rightarrow\mathbb{R}$ by

$\displaystyle\begin{aligned} &g_{j}(x)=\max_{z\in Z_{j}}\langle
x,z\rangle,\end{aligned}$ (34)

where $Z_{j}\in\mathbb{R}^{n}$ is the uncertainty set. Let
$B_{j}\in\mathbb{R}^{n}\times\mathbb{R}$ a nonsingular matrix and $\delta>0$
be given. We set

$\displaystyle\begin{aligned} &Z_{j}:=\\{z\in\mathbb{R}^{n}|-\delta e\preceq
B_{j}z\preceq\delta e\\},\end{aligned}$ (35)

where $e=(1,...,1)^{T}\in\mathbb{R}^{n}$. Since $Z_{j}$ is a nonempty and
compact, $g_{j}(x)$ is well-defined. In our tests, the elements of the matrix
$B_{j}$ were randomly chosen between 0 and 1. In turn, given an arbitrary
point $\bar{x}\in C$, parameter $\delta$ was taken as

$\displaystyle\begin{aligned} &\delta:=\bar{\delta}\|\bar{x}\|,\end{aligned}$
(36)

where $0.02\leq\bar{\delta}\leq 0.10$ was also chosen at random.

Next, we refer to the method of dealing with the nonsmooth term $g_{j}$ in the
subproblem of the generalized conditional gradient for MCOPs proposed by
Assunção in AFP23  to deal with the nonsmooth term in the subproblem of our
algorithm NPQNA for MCOPs. We define
$A_{j}=[B_{j};-B_{j}]\in\mathbb{R}^{2n\times n}$ and $b_{j}:=\delta
e\in\mathbb{R}^{2n}$, then (34) and (35) can be rewritten as

$\displaystyle\begin{aligned} &\ \ \max_{z}\langle x,z\rangle\\\ &s.t.\ \
A_{j}z\preceq b_{j}.\end{aligned}$ (37)

Since the variables in (33) are $t$ and $d$, we convert the nonsmooth term
$g_{j}(x+d)$ into a dual form.

$\displaystyle\begin{aligned} &\ \ \min_{w}\langle b_{j},w\rangle\\\ &s.t.\ \
A_{j}^{T}w=x+d,\\\ &\ \ \ \ \ w\succeq 0.\end{aligned}$ (38)

By using duality theory, substituted the expression of (38) into (6), we can
get

$\displaystyle\begin{aligned} &\min\ t\\\ &s.t.\ \nabla
f_{j}(x)^{T}d+\frac{1}{2}d^{T}B_{j}(x)d+b_{j}^{T}w-g_{j}(x)-t\leq 0,\\\ &\ \ \
\ \ \ A_{j}^{T}w_{j}=x+d,\\\ &\ \ \ \ \ \ w_{j}\succeq 0,\ j\in\Lambda_{m},\\\
&\ \ \ \ \ \ (t,d)\in\mathbb{R}\times\mathbb{R}^{n}.\end{aligned}$ (39)

We briefly review the subproblem of the proximal Newton-type algorithm NPGA
for MCOPs.

$\min_{d\in\mathbb{R}^{n}}\max_{j\in\Lambda_{m}}\nabla
f_{j}(x)^{T}d+\frac{1}{2}d^{T}\nabla^{2}f_{j}(x)d+g_{j}(x+d)-g_{j}(x).$

Likewise, the subproblem of the NPGA for MCOPs can be reformulated as the
following quadratic programming problem

$\displaystyle\begin{aligned} &\min\ t\\\ &s.t.\ \nabla
f_{j}(x)^{T}d+\frac{1}{2}d^{T}\nabla^{2}f_{j}(x)d+b_{j}^{T}w-g_{j}(x)-t\leq
0,\\\ &\ \ \ \ \ \ A_{j}^{j}w_{j}=x+d,\\\ &\ \ \ \ \ \ w_{j}\succeq 0,\
j\in\Lambda_{m},\\\ &\ \ \ \ \ \
(t,d)\in\mathbb{R}\times\mathbb{R}^{n}.\end{aligned}$ (40)

And the subproblem of the proximal quasi-Newton algorithm PQNA for MCOPs.

$\min_{d\in\mathbb{R}^{n}}\max_{j\in\Lambda_{m}}\nabla
f_{j}(x)^{T}d+\frac{1}{2}d^{T}B_{j}(x)d+g_{j}(x+d)-g_{j}(x)+\frac{w}{2}\|d\|^{2}.$

Likewise, the subproblem of the PQNA for MCOPs can be reformulated as the
following quadratic programming problem

$\displaystyle\begin{aligned} &\min\ t\\\ &s.t.\ \nabla
f_{j}(x)^{T}d+\frac{1}{2}d^{T}B_{j}(x)d+\frac{w}{2}\|d\|^{2}+b_{j}^{T}w-g_{j}(x)-t\leq
0,\\\ &\ \ \ \ \ \ A_{j}^{j}w_{j}=x+d,\\\ &\ \ \ \ \ \ w_{j}\succeq 0,\
j\in\Lambda_{m},\\\ &\ \ \ \ \ \
(t,d)\in\mathbb{R}\times\mathbb{R}^{n}.\end{aligned}$ (41)

In our codes, we use the solver ’SLSQP’ for solving convex quadratic
programming problem in python 3.9 to solve problem (39), (40) and (41).

Pareto fronts using NPQNA, PQNA and NPGA of a two objective and a three
objective test problem (Problem 1, 3, 7, 9 and 14 in Table 1) are provided in
Figures 1, 2, 3, 4 and 5 respectively. Similarly we have generated approximate
Pareto fronts of all test problems mentioned in Table 1.

Table 1: Details of test problems.

S1.No |  |  | (m,n) |  |  | f |  |  | lbT |  |  | ubT |  |  | Ref
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
1 |  |  | $(2,2)$ |  |  | AN1 |  |  | (-3,-3) |  |  | (7,7) |  |  | AP15
2 |  |  | $(2,2)$ |  |  | AP2 |  |  | (-5,-5) |  |  | (5,5) |  |  | DAPM00
3 |  |  | $(2,2)$ |  |  | BK1 |  |  | (-3,-3) |  |  | (5,5) |  |  | CLV07
4 |  |  | $(3,3)$ |  |  | FDS |  |  | (-2,-2,-2) |  |  | (4,4,4) |  |  | CF98
5 |  |  | $(3,5)$ |  |  | FDS |  |  | (-2,…,-2) |  |  | (2,…,2) |  |  | CF98
6 |  |  | $(3,2)$ |  |  | IKK1 |  |  | (-2,-2) |  |  | (3,3) |  |  | CLV07
7 |  |  | $(3,3)$ |  |  | IKK1 |  |  | (-2,-2,-2) |  |  | (2,2,2) |  |  | CLV07
8 |  |  | $(2,2)$ |  |  | JOS1 |  |  | (-5,-5) |  |  | (5,5) |  |  | D17
9 |  |  | $(2,2)$ |  |  | LOVISON1 |  |  | (-3,-3) |  |  | (5,5) |  |  | DJ13
10 |  |  | $(2,2)$ |  |  | LRS1 |  |  | (-50,-50) |  |  | (50,50) |  |  | CLV07
11 |  |  | $(3,3)$ |  |  | MHHM2 |  |  | (-4,-4,-4) |  |  | (4,4,4) |  |  | CLV07
12 |  |  | $(3,2)$ |  |  | MHHM2 |  |  | (-4,-4) |  |  | (4,4) |  |  | CLV07
13 |  |  | $(2,1)$ |  |  | MOP1 |  |  | (-100) |  |  | (100) |  |  | CLV07
14 |  |  | $(3,3)$ |  |  | MOP7 |  |  | (-4,-4,-4) |  |  | (4,4,4) |  |  | CLV07
15 |  |  | $(3,2)$ |  |  | MOP7 |  |  | (-4,-4) |  |  | (4,4) |  |  | CLV07
16 |  |  | $(2,2)$ |  |  | MS1 |  |  | (-2,-2) |  |  | (2,2) |  |  | CCMP95
17 |  |  | $(2,4)$ |  |  | MS2 |  |  | (-2,…,-2) |  |  | (2,…,2) |  |  | CCMP95
18 |  |  | $(3,3)$ |  |  | SDD1 |  |  | (-2,-2,-2) |  |  | (2,2,2) |  |  | DGW92
19 |  |  | $(2,2)$ |  |  | SP1 |  |  | (-1,-1) |  |  | (5,5) |  |  | CLV07
20 |  |  | $(3,2)$ |  |  | VFM1 |  |  | (-2,-2) |  |  | (4,4) |  |  | CLV07
21 |  |  | $(3,3)$ |  |  | VFM1 |  |  | (-2,-2,-2) |  |  | (4,4,4) |  |  | CLV07
22 |  |  | $(2,2)$ |  |  | VU1 |  |  | (-3,-3) |  |  | (3,3) |  |  | CLV07
23 |  |  | $(3,3)$ |  |  | ZLT1 |  |  | (-3,-3,-3) |  |  | (3,3,3) |  |  | CLV07

(a) Approximal Pareto fronts by NPQNA

(b) Approximal Pareto fronts by PQNA

(c) Approximal Pareto fronts by NPGA

Figure 1: Approximate Pareto fronts of problem 1. (a)Approximate Pareto fronts
by NPQNA; (b)Approximate Pareto fronts by PQNA; (c)Approximate Pareto fronts
by NPGA.

(a) Approximal Pareto fronts by NPQNA

(b) Approximal Pareto fronts by PQNA

(c) Approximal Pareto fronts by NPGA

Figure 2: Approximate Pareto fronts of problem 3. (a)Approximate Pareto fronts
by NPQNA; (b)Approximate Pareto fronts by PQNA; (c)Approximate Pareto fronts
by NPGA.

(a) Approximal Pareto fronts by NPQNA

(b) Approximal Pareto fronts by PQNA

(c) Approximal Pareto fronts by NPGA

Figure 3: Approximate Pareto fronts of problem 9. (a)Approximate Pareto fronts
by NPQNA; (b)Approximate Pareto fronts by PQNA; (c)Approximate Pareto fronts
by NPGA.

(a) Approximal Pareto fronts by NPQNA

(b) Approximal Pareto fronts by PQNA

(c) Approximal Pareto fronts by NPGA

Figure 4: Approximate Pareto fronts of problem 7. (a)Approximate Pareto fronts
by NPQNA; (b)Approximate Pareto fronts by PQNA; (c)Approximate Pareto fronts
by NPGA.

(a) Approximal Pareto fronts by NPQNA

(b) Approximal Pareto fronts by PQNA

(c) Approximal Pareto fronts by NPGA

Figure 5: Approximate Pareto fronts of problem 14. (a)Approximate Pareto
fronts by NPQNA; (b)Approximate Pareto fronts by PQNA; (c)Approximate Pareto
fronts by NPGA. Table 2: Computational Details.

Problem |  | NPQNA |  | PQNA |  | NPGA
---|---|---|---|---|---|---
| it | f | CPU |  | it | f | CPU |  | it | f | CPU
1 |  | 276.23 | 277.23 | 7.08 |  | 300 | 301 | 13.28 |  | 291.18 | 292.18 | 9.49
2 |  | 161.88 | 162.88 | 7.95 |  | 300 | 301 | 15.45 |  | 300 | 301 | 11.32
3 |  | 297.01 | 298.01 | 1.91 |  | 226.34 | 227.34 | 3.23 |  | 204.32 | 205.32 | 1.86
4 |  | 177.53 | 178.53 | 1.34 |  | 297.05 | 298.05 | 3.68 |  | 162.52 | 163.52 | 1.35
5 |  | 233.43 | 234.43 | 2.41 |  | 297.07 | 298.07 | 4.5 |  | 156.57 | 157.57 | 1.78
6 |  | 249.3 | 250.3 | 1.96 |  | 264.93 | 265.93 | 4.9 |  | 225.25 | 226.25 | 2.71
7 |  | 198.64 | 199.64 | 5.25 |  | 270.78 | 271.78 | 12.73 |  | 180.4 | 181.4 | 5.55
8 |  | 198.34 | 199.34 | 0.79 |  | 244.17 | 245.17 | 1.75 |  | 198.34 | 199.34 | 0.96
9 |  | 279.08 | 280.08 | 9.27 |  | 235.02 | 236.02 | 16.99 |  | 192.36 | 193.36 | 12.44
10 |  | 255.24 | 256.24 | 2.78 |  | 97.09 | 98.09 | 3.84 |  | 96.68 | 97.68 | 2.33
11 |  | 39.36 | 40.36 | 5.66 |  | 252.39 | 253.39 | 15.74 |  | 180.4 | 181.4 | 7.41
12 |  | 50.47 | 51.47 | 2.39 |  | 266.37 | 267.37 | 7.26 |  | 162.97 | 163.97 | 3.92
13 |  | 5.18 | 6.18 | 0.13 |  | 165.25 | 166.25 | 3.32 |  | 66.78 | 67.78 | 1.07
14 |  | 201.59 | 202.59 | 4.05 |  | 262.05 | 263.05 | 9.75 |  | 180.4 | 181.4 | 4.50
15 |  | 122.43 | 123.43 | 0.77 |  | 294.18 | 295.18 | 2.82 |  | 219.27 | 220.27 | 1.48
16 |  | 146.28 | 147.28 | 5.6 |  | 300 | 301 | 11.32 |  | 228.24 | 229.24 | 7.8
17 |  | 12.79 | 13.79 | 0.41 |  | 300 | 301 | 10.43 |  | 124.69 | 125.69 | 3
18 |  | 239.84 | 240.84 | 9.38 |  | 300 | 301 | 22.9 |  | 297.08 | 298.08 | 13.04
19 |  | 212.98 | 213.98 | 3.89 |  | 297.79 | 298.79 | 6.03 |  | 300 | 301 | 4.22
20 |  | 214.78 | 215.78 | 3.48 |  | 300 | 301 | 9.51 |  | 102.66 | 103.66 | 4.6
21 |  | 186.3 | 187.3 | 6.87 |  | 300 | 301 | 18.98 |  | 300 | 301 | 9.86
22 |  | 300 | 301 | 4.75 |  | 300 | 301 | 8.55 |  | 300 | 301 | 5.39
23 |  | 182.57 | 183.57 | 2.46 |  | 300 | 301 | 6.34 |  | 225.25 | 226.25 | 3.06

Numerical experiments show that our algorithm NPQNA of MCOPs has less
iteration times, function evaluation times and CPU running time than NPGA of
MCOPs in A22  and PQNA of MCOPs in PR22  on most test problems. Although our
algorithm NPQNA has more iterations than NPGA and PQNA on some test problems,
our algorithm has less CPU running time, which also shows the superiority of
our algorithm. By analysing the related problems that the number of iterations
of NPQNA is more than that of NPGA, we find that if the Hessian matrix of the
objective function has an explicit expression, the number of iterations
updated by Newton is smaller than that updated by quasi-Newton. However, if
the Hessian matrix of the objective function is not easy to obtain, our NPQNA
algorithm has obvious advantages.

##  7 Conclusions

In this paper, we have developed a nonmonotone proximal quasi-Newton algorithm
for unconstrained convex MCOPs. Under standard assumptions, we proved that the
sequence generated by this method converges to a Pareto optimal. Furthermore,
under the assumptions that the Hessian continuity, the Dennis-Moré criterion
and strong convexity, the local superlinear convergence rate of the algorithm
is obtained. We consider convex MCOPs, but in real life, many MCOPs are
nonconvex, so it is of practical value to study the algorithm for solving
nonconvex MCOPs. And this method only starts from the quadratic approximation
of the smooth function, and does not take full advantage of the
characteristics of the nonsmooth term. Lin et al.LMH19  proposed a proximal
quasi-Newton algorithm in scalar optimization, which makes full use of the
properties of nonsmooth function, performs Moreau envelope processing on
nonsmooth term, and then uses the properties of Moreau envelope to perform
quasi-Newton acceleration on the whole smooth problem. This is a very nice
approach, but there is no research on MCOPs yet. In the future, we try to
study smoothing techniques for MCOPs and nonconvex MCOPs.

## References

  * (1) Amiri, N. M., Sadaghiani, F. S.: A superlinearly convergent nonmonotone quasi-Newton method for unconstrained multiobjective optimization. Optim. Methods Softw. https://doi.org/10.1080/10556788.2020.1737691 (2020).
  * (2) Ansary, M. A. T.: A Newton-type proximal gradient method for nonlinear multiobjective optimization problems. Optim. Methods Softw. https://doi.org/10.1080/10556788.2022.2157000 (2023).
  * (3) Ansary, M. A. T., Panda, G.: A modified quasi-Newton method for vector optimization problem. Optimization. 64, 2289-2306 (2015).
  * (4) Assunção, P. B., Ferreira, O. P., Prudente, L. F.: A generalized conditional gradient method for multiobjective composite optimization problems. arXiv:2302.12912v1 (2023).
  * (5) Berge, C.: Topological spaces. Dover publications, Edinburgh, (1963).
  * (6) Bento, G. d. C., Cruz Neto, J. X., Lopez, G., Soubeyran, A., Souza, J. C. O.: The proximal point method for locally lipschitz functions in multiobjective optimization with application to the compromise problem. SIAM J. Optim. 28, 1104-1120 (2018).
  * (7) Bertsekas, D. P.: Nonlinear programming. Athena Scientific, Belmont, Mass, Second edition, (1999).
  * (8) Binder, M., Moosbauer, J., Thomas, J., Bischl, B.: Multi-objective hyperparameter tuning and feature selection using filter ensembles, in Proceedings of the 2020 Genetic and Evolutionary Computation Conference, Association for Computing Machinery, 471-479 (2020).
  * (9) Carrizosa, E., Frenk, J. B. G.: Dominating sets for convex functions with some applications. J. Optim. Theory Appl. 96, 281-295 (1998).
  * (10) Carrizosa, E., Conde, E., Munoz, M., Puerto, J.: Planar point objective location problems with nonconvex constraints: A geometrical construction. J. Global Optim. 6, 77-86 (1995).
  * (11) Chen, J., Jiang, X. X., Tang, L. P., Yang, X. M.: On the Convergence of Newton-type Proximal Gradient Method for Multiobjective Optimization Problems. arXiv:2308.10140 (2023).
  * (12) Coello, C. A. C., Lamont, G. B., Van Veldhuizen, D. A.: Evolutionary Algorithms for Solving MultiObjective Problems. 5, Springer (2007).
  * (13) De, P., Gosh, J. B., Wells, C. E.: On the minimization of completion time variance with a bicriteria extension. Oper. Res. 40, 1148-1155 (1992).
  * (14) Deb, K.: Recent advances in evolutionary multi-criterion optimization (EMO). in Proceedings of the Genetic and Evolutionary Computation Conference Companion. 702-735 (2017).
  * (15) Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast elitist non-dominated sorting genetic algorithm for multiobjective optimization: NSGA-II. in International Conference on Parallel Problem Solving from Nature, Springer, 849-858 (2000).
  * (16) Deb, K., Jain, H.: An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: Solving problems with box constraints. IEEE T. Evolut. Comput. 18, 577-601 (2013).
  * (17) Dhara, A., Dutta, J.: Optimality conditions in convex optimization: a finite-dimensional view. CRC Press, London, (2011).
  * (18) Fliege, J., Svaiter, B. F.: Steepest descent methods for multicriteria optimization. Math. Methods Oper. Res. 51, 479-494 (2000).
  * (19) Fliege, J., Drummond, L. M. G., Svaiter, B. F.: Newton’s method for multiobjective optimization. SIAM J. Optim. 20, 602-626 (2009).
  * (20) Fliege, J., Werner, R.: Robust multiobjective optimization and applications in portfolio optimization, Eur. J. Oper. Res. 234, 422-433 (2014).
  * (21) Gebken, B., Peitz, S.: An Efficient Descent Method for Locally Lipschitz Multiobjective Optimization Problems. J. Optim. Theory Appl. 188, 696-723 (2021).
  * (22) Gong, M., Zhang, M., Yuan, Y.: Unsupervised band selection based on evolutionary multiobjective optimization for hyperspectral images, IEEE Transactions on Geoscience and Remote Sensing, 54, 544-557 (2016).
  * (23) Hogan, W. W.: Point-to-set maps in mathematical programming. SIAM Review. 15, 591-603 (1973).
  * (24) Lee, J. D., Sun, Y., Saunders, M. A.: Proximal Newton-type methods for minimizing composite functions. SIAM J. Optim. 24, 1420-1443 (2014).
  * (25) Lin, H. Z., Mairal, J., Harchaoui, Z.: An inexact variable metric proximal point algorithm for generic quasi-Newton acceleration. SIAM J. Optim. 29, 1408-1443 (2019).
  * (26) Lucambio Pérez, L. R., Prudente, L. F.: Nonlinear conjugate gradient methods for vector optimization. SIAM J. Optim. 28, 2690-2720 (2018).
  * (27) Mita, K., Fukuda, E. H., Yamashita, N.: Nonmonotone line searches for unconstrained multiobjective optimization problems. J. Global Optim. 75, 63-90 (2019).
  * (28) Peng, J. W., Ren, J.: Proximal quasi-Newton methods for multiobjective optimization problems. arXiv:2108.00125v2 (2022).
  * (29) Povalej, Z.: Quasi-Newton’s method for multiobjective optimization. J. Comput. Appl. Math. 255, 765-777 (2014).
  * (30) Qu, S., Liu, C., Goh, M., Li, Y., Ji,Y.: Nonsmooth multiobjective programming with quasi-Newton methods. Eur. J. Oper. Res. 235, 503-510 (2014).
  * (31) Sion, M.: On general minimax theorems. Pacific Journal of Mathematics. 8(1), 171-176 (1958).
  * (32) Tanabe, H., Fukuda, E. H., Yamashita, N.: Proximal gradient methods for multiobjective optimization and their application. Comput. Optim. Appl. 72, 339-361 (2019).
  * (33) Tanabe, H., Fukuda, E. H., Yamashita, N.: Convergence rates analysis of a multiobjective proximal gradient method. Optim Lett. 17, 333-350 (2023).
  * (34) J. Wang, Y. Hu, C. K. Wai Yu, C. Li, X. Yang. Extended Newton methods for multiobjective optimization: majorizing function technique and convergence analysis. SIAMJ. Optim. 29, 2388-2421 (2019).
  * (35) Zhang, H., Hager, W. W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 14, 1043-1056 (2004).