# Tikhonov Regularized Iterative Methods for Nonlinear Problems

Avinash Dixit<EMAIL_ADDRESS>D. R. Sahu<EMAIL_ADDRESS>Pankaj
Gautam<EMAIL_ADDRESS>T. Som<EMAIL_ADDRESS>Department
of Mathematical Sciences, Indian Institute of Technology (BHU) Varanasi
–221005, India  Department of Mathematics, Banaras Hindu University,
Varanasi-221005, India

###### Abstract

We consider the monotone inclusion problems in real Hilbert spaces. Proximal
splitting algorithms are very popular technique to solve it and generally
achieve weak convergence under mild assumptions. Researchers assume the strong
conditions like strong convexity or strong monotonicity on the considered
operators to prove strong convergence of the algorithms. Mann iteration method
and normal S-iteration method are popular methods to solve fixed point
problems. We propose a new common fixed point algorithm based on normal
S-iteration method using Tikhonov regularization to find common fixed point of
nonexpansive operators and prove strong convergence of the generated sequence
to the set of common fixed points without assuming strong convexity and strong
monotonicity. Based on proposed fixed point algorithm, we propose a forward-
backward-type algorithm and a Douglas-Rachford algorithm in connection with
Tikhonov regularization to find the solution of monotone inclusion problems.
Further, we consider the complexly structured monotone inclusion problems
which are very popular these days. We also propose a strongly convergent
forward-backward-type primal-dual algorithm and a Douglas-Rachford-type
primal-dual algorithm to solve the monotone inclusion problems. Finally, we
conduct a numerical experiment to solve image deblurring problems.

###### keywords:

Fixed points of nonexpansive mappings , Tikhonov regularization , Splitting
methods , Forward–backward algorithm , Douglas–Rachford algorithm ,
Primal–dual algorithm.
AMS Mathematics Subject Classification (2010): 47J25 $\cdot$ 47H09 $\cdot$
47H05 $\cdot$ 47A52.

##  1 Introduction

Throughout the paper, $\mathcal{H}$ denotes a real Hilbert space with inner
product $\langle\cdot,\cdot\rangle$ and norm $\|\cdot\|$, respectively.
Consider $\operatorname{T}:\mathcal{H}\to 2^{\mathcal{H}}$ is a set-valued
monotone operator. The monotone inclusion problem is to find $x\in\mathcal{H}$
such that

$0\in T(x).$ (1.1)

The monotone inclusion problem (1.1) plays important role in nonlinear
analysis. Many problems arising in engineering, economics and physics can be
framed as monotone inclusion problem (see [4, 7, 13, 21, 22, 42]). Martinet
[38] has proposed proximal point algorithm, which is very popular to solve
monotone inclusion problem. The proximal point algorithm is given by

$x_{n+1}=J_{\lambda_{n}{T}}(x_{n})\quad\forall n\in\mathbb{N},$ (1.2)

where $J_{\lambda_{n}{T}}=(Id+\lambda_{n}T)^{-1}$, $\lambda_{n}>0$ is a
regularization parameter and $x_{1}\in\mathcal{H}.$ Rockafellar [45, 46] has
proved that proximal point algorithm converges weakly to solution set of
inclusion problems in real Hilbert space framework. Further, he has introduced
the inexact proximal point algorithm as follows:

$x_{n+1}=J_{\lambda_{n}T}(x_{n}+{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\epsilon_{n}}),\quad\forall
n\in\mathbb{N},$ (1.3)

where $\\{\epsilon_{n}\\}$ is an error sequence in $\mathcal{H}$. The sequence
$\\{x_{n}\\}$ also converges weakly to solution set of inclusion problem
provided $\sum_{n=1}^{\infty}\|\epsilon_{n}\|<\infty.$ Guler [29] has shown by
an example that sequence generated by proximal point algorithm (1.2) converges
weakly, but not strongly. It becomes a matter of interest for the research
community to modify the proximal point algorithm to obtain the strong
convergence. In such consequences, Tikhonov method has been proposed which
generates as follows:

$x_{n+1}=J_{\lambda_{n}T}(x),$ (1.4)

where $x\in\mathcal{H}$ and $\lambda_{n}>0$ such that $\lambda_{n}\to\infty.$
Detailed study of Tikhonov regularization method can be found in [14, 51, 52,
53, 55]. Lehdili and Moudafi [33] have combined the idea of proximal algorithm
and Tikhonov regularization to find an algorithm converges strongly to
solution of inclusion problem (1.1). They have solved the inclusion problem
(1.1) by solving inclusion problem of fixed approximation of $T$, which is
$T_{n}=T+\mu_{n}Id$, i.e.,

$\text{find }x\in\mathcal{H}\text{ such that }0\in T_{n}(x),$

where $\mu_{n}$ is regularization parameter of $T$. The proximal-Tikhonov
algorithm is given by

$x_{n+1}=J_{\lambda_{n}T_{n}}(x_{n}).$

The Tikhonov regularization term $\mu_{n}Id$ has impelled the strong
convergence to the algorithm. In absence of Tikhonov regularization term,
proximal-Tikhonov algorithm becomes proximal algorithm which shows only weak
convergence in most of the cases. The strong convergence of the algorithm can
be obtained by using some other techniques also, some of them can be found in
[5, 31].

Evaluation of resolvent is sometimes as hard as the original problem. This
problem has been tried to resolve by splitting the operator in two operators,
i.e., $T=A+B,$ whose resolvents are easy to compute. For $T=A+B,$ the monotone
inclusion problem (1.1) becomes

$\text{find }x\in\mathcal{H}\text{ such that }0\in(A+B)x,$ (1.5)

where $A:\mathcal{H}\to 2^{\mathcal{H}}$ is maximally monotone operator and
$B$ is an operator. Problem (1.1) is also a generalization of the variational
inequality problem:

$\text{find }x_{1}\in\mathcal{H}\text{ such that }(\exists x^{*}\in
g(x_{1}))(\forall x_{2}\in\mathcal{H})\langle x_{1}-x_{2},x^{*}\rangle\leq
f(x_{2})-f(x_{1}),$

where $f:\mathcal{H}\to\mathcal{H}$ is a proper, convex, lower semicontinuous
and $g:\mathcal{H}\to 2^{\mathcal{H}}$ be a maximally monotone operator. The
problem (1.1) serves as a blanket for various nonlinear problems viz. image
denoising problem; clustering problem; wireless sensor network localization
problem; matrix factorization problem; generalized Nash equilibrium problem
and many more (see [8, 10, 11, 28, 30]).

Forward-backward splitting algorithm and Douglas-Rachford algorithm have been
proposed to solve Problem (1.5). Forward-backward splitting method has been
proposed by Lions and Mercier [34], Passty [43], which is given by

$x_{n+1}=(Id+\lambda_{n}A)^{-1}(Id-\lambda_{n}B)x_{n},$ (1.6)

where $\lambda_{n}>0$ and $B:\mathcal{H}\to\mathcal{H}$ is a cocoercive
operator. Mercier [39] and Gabay [26] have studied the convergence behavior of
forward-backward method when $A^{-1}$ is $\gamma$-strongly monotone with
$\gamma>0.$ They have proved that forward-backward algorithm converges weakly
to the point in the solution set provided $\lambda_{n}<2\gamma$, is constant.
In addition, if $A$ is strongly monotone, then $\\{x_{n}\\}$ shows strong
convergence to the unique solution of Problem (1.5). Chen and Rockafellar [18]
have also assumed the strong monotonicity of $A$ to prove the strong
convergence of forward-backward method which depends on Lipschitz constant and
modulus of strong monotonicity. Further, forward-backward method has been
extensively studied, few of them can be found in ([17, 18, 40, 41]) and
references therein.

Douglas-Rachford method has been proposed to solve problem (1.5) when both $A$
and $B$ are set-valued. It has been originally proposed by Douglas and
Rachford [25] to solve linear equations arising in heat-conduction problems.
Lions and Mercier [34] have extended the Douglas-Rachford algorithm to
monotone operators. Douglas-Rachford algorithm is given as follows$\colon$

$x_{n+1}=R_{B}R_{A}x_{n},\quad\forall n\in\mathbb{N},$ (1.7)

where $R_{B}$ and $R_{A}$ are reflected resolvent of operators $B$ and $A$,
respectively. Lions and Mercier [34] have proved that Douglas-Rachford
algorithm converges weakly to a fixed point of operator $T$ which helps to
obtain the solution of the Problem (1.5). Svaiter [50] has supported the
results of Lions and Mercier by proving the weak convergence of the shadow
sequence to a solution. Further the analysis of the Douglas-Rachford algorithm
can be found in ([2, 23, 35, 44]).

Let $\mathcal{C}$ be a nonempty closed convex subset of $\mathcal{H}$ and
$\mathcal{S}:\mathcal{C}\to\mathcal{C}$ be a nonexpansive operator. There are
a number of iterative methods for finding fixed points of nonexpansive
operators. We recall some well known fixed point methods, which are given
below$\colon$

  1. $\bullet$

Mann iteration method [37]:

$x_{n+1}=(1-\beta_{n})x_{n}+\beta_{n}\mathcal{S}(x_{n}),\quad\forall
n\in\mathbb{N};$

  2. $\bullet$

S-iteration method [1]:

$x_{n+1}=(1-\alpha_{n})\mathcal{S}(x_{n})+\alpha_{n}\mathcal{S}[(1-\beta_{n})x_{n}+\beta_{n}\mathcal{S}x_{n}],\quad\forall
n\in\mathbb{N};$

  3. $\bullet$

Normal S-iteration method [47]:

$x_{n+1}=\mathcal{S}[(1-\beta_{n})x_{n}+\beta_{n}\mathcal{S}x_{n}],\quad\forall
n\in\mathbb{N};$

where $\alpha_{n},\beta_{n}\in(0,1).$ The importance of these algorithms are
not limited to solve fixed point problems, but these algorithms are also
useful for solving inclusion problems of sum of a set-valued maximally
monotone operator and a single-valued cocoercive operator, and inclusion
problems of sum of two set-valued maximally monotone operators. The
S-iteration methodology has been applied for solving various nonlinear
problems, inclusion problems, optimization problems and image recovery
problems. Recently, it has been demonstrated by Avinash et al. [24] that the
inertial normal S-iteration method has better performance compared to the
inertial Mann iteration method. The S-iteration method and normal S-iteration
method are also useful for finding common fixed points of nonexpansive
operators. Since last few years, these properties of normal S-iteration make
it popular among research community to find fixed point. Several research
articles related to S-iteration and normal S-iteration can be found in [15,
16, 19, 48, 49]. The weak convergence of the fixed point algorithms have
reduced its applicability in infinite dimensional spaces. To achieve the
strong convergence of algorithms one assumes stronger assumptions like strong
monotonicity and strong convexity, which is difficult to achieve in many
applications. This situation leaves a question to research community: can we
find the strongly convergent algorithms without assuming these strong
assumptions? The answer to this question is replied positively by Boţ et al.
[9]. They have modified the Mann algorithm as follows:

$x_{n+1}=e_{n}x_{n}+\theta_{n}(\mathcal{S}(e_{n}x_{n})-e_{n}x_{n}),$ (1.8)

where $e_{n},\theta_{n}$ are positive real numbers. The strong convergence of
algorithm (1.8) for nonexpansive operator $\mathcal{S}$ has been studied by
Boţ et al. [9] when set of fixed points of $\mathcal{S}$ is nonempty and
parameters $\theta_{n}$ and $e_{n}$ satisfy the following:

  1. (i)

$0<e_{n}<1$ for all $n\in\mathbb{N},$ $\lim\limits_{n\to\infty}e_{n}=1$,
$\sum_{n=1}^{\infty}(1-e_{n})=\infty$ and
$\sum_{n=1}^{\infty}|e_{n}-e_{n-1}|<\infty;$

  2. (ii)

$0<\theta_{n}\leq 1$ for all $n\in\mathbb{N},$
$0<\liminf_{n\to\infty}\theta_{n},$
$\sum_{n=1}^{\infty}|\theta_{n}-\theta_{n-1}|<\infty.$

We consider the following more general problem:

######  Problem 1.1.

Consider $\mathcal{T},\mathcal{S}:\mathcal{H}\to\mathcal{H}$ are nonexpansive
operators. Find an element $x\in\mathcal{H}$ such that
$x\in\operatorname{Fix}(\mathcal{T})\cap\operatorname{Fix}(\mathcal{S})$.

The study on the common solutions of system of problems can be found in [27,
32, 36]

######  Remark 1.1.

The algorithm (1.8) proposed by Boţ et al. [9] can not directly apply to solve
inclusion problem (1.1).

In this paper, we introduce the normal S-iteration method based fixed point
algorithm to find common fixed point of nonexpansive operators
$\mathcal{T},\mathcal{S}:\mathcal{H}\to\mathcal{H}$, which converges strongly
to common solutions of fixed point problem of operators $\mathcal{S}$ and
$\mathcal{T}$. Based on the proposed fixed point algorithm, we develop a
forward-backward algorithm and a Douglas-Rachford algorithm containing
Tikhonov regularization term to solve the monotone inclusion problems. In many
cases, monotone inclusion problems are very complex, they contain mixture of
linear and parallel sum monotone operators. Recently, many researchers have
proposed primal-dual algorithms to precisely solve the considered complex
monotone inclusion system [3, 8, 12, 20, 54]. We have proposed a forward-
backward type primal-dual algorithm and a Doughlas-Rachford type primal-dual
algorithm having Tikhonov regularization term to find the common solution of
the complexly structured monotone inclusion problems. The proposed algorithms
have a special property that resolvents of all the operators are evaluated
separately.

The paper is organized as follows: Next section recalls some important
definitions and results in nonlinear analysis. In Section 3, we propose a
normal S-iteration based Tikhonov regularized fixed point algorithm and study
its convergence behavior. In Section 4, we propose a forward-backward-type
algorithm and a forward-backward-type primal-dual algorithm to solve inclusion
problem and complexly structured monotone inclusion problem, respectively. In
Section 5, we propose Douglas-Rachford-type algorithms to solve monotone
inclusion problems and complexly structured monotone inclusion problems of
set-valued operators. In the last, we perform a numerical experiment to show
the importance of proposed algorithms in solving image deblurring problems.

##  2 Preliminaries

This section devotes some important definitions and results from nonlinear
analysis and operator theory. Let $\mathbb{N}$ and $\mathbb{R}$ denote set of
natural numbers and set of real numbers, respectively and ‘$Id$’ denotes
identity operator. Consider the operator $T:\mathcal{H}\to 2^{\mathcal{H}}.$
Let $Gr(T)$ denote the graph of $T$, $\operatorname{Zer}(T)$ denote set of
zeros of operator $T$ and $\operatorname{Fix}(T)$ denote set of fixed points
of $T$. The symbol $m$ is used to denote a strictly positive integer
throughout the paper. The set of proper convex lower semicontinuous functions
from $\mathcal{H}$ to $[-\infty,+\infty]$ is denoted by $\Gamma(\mathcal{H}).$
Let $f\in\Gamma(\mathcal{H}),$ then
$argmin_{x\in\mathcal{H}}f(x)=\\{x^{*}\in\mathcal{H}:f(x^{*})\leq f(y),\forall
y\in\mathcal{H}\\}$ and
$argmax_{x\in\mathcal{H}}f(x)=\\{x^{*}\in\mathcal{H}:f(x^{*})\geq f(y),\forall
y\in\mathcal{H}\\}.$ Let $A:\mathcal{H}\to 2^{\mathcal{H}}$ be an operator.
Domain of $A$ is $\operatorname{dom}$
$(A)=\\{x\in\mathcal{H}:Ax\neq\emptyset\\}$. Range of $A$ is denoted by ran
$(A)$ = $\cup_{x\in\mathcal{H}}$ $Ax$. $A$ is said to be monotone if

$\langle x-y,u-v\rangle\geq 0,\ \forall(x,u),(y,v)\in Gr(A).$

$A$ is said to be maximally monotone if there exists no monotone operator
$B:\mathcal{H}\to 2^{\mathcal{H}}$ such that $Gr(B)$ properly contains
$Gr(A).$ $A$ is strongly monotone with constant $\beta\in(0,\infty)$ if

$\langle x-y,u-v\rangle\leq\beta\|x-y\|^{2}\ \forall(x,u),(y,v)\in Gr(A).$

The resolvent of $A$ is defined by $J_{A}=(Id+A)^{-1}$ and the reflected
resolvent of $A$ is $R_{A}=2J_{A}-Id.$ Consider
$f:\mathcal{H}\to[-\infty,\infty]$. The conjugate of $f$ is defined by
$f^{*}:\mathcal{H}\to[-\infty,\infty],$

$u\mapsto\sup_{x\in\mathcal{H}}\left(\langle x,u\rangle-f(x)\right).$

Let $f:\mathcal{H}\to[-\infty,\infty]$ be a proper function. The
subdifferential of $f$ is $\partial f:\mathcal{H}\to 2^{\mathcal{H}}$ is
defined by

$x\mapsto\\{u\in\mathcal{H}|f(y)\geq f(x)+\langle y-x,u\rangle\ \forall
y\in\mathcal{H}\\}.$

If $f\in\Gamma(\mathcal{H}),$ then $\partial f$ is maximally monotone. The
resolvent of subdifferential of $f$ is $prox_{f}$, where
$prox_{f}:\mathcal{H}\to\mathcal{H}$ defined by

$prox_{f}(x)=argmin_{y\in\mathcal{H}}\left\\{f(y)+\frac{1}{2}\|y-x\|^{2}\right\\}.$

######  Definition 2.1.

Let $\mathcal{C}$ be a nonempty subset of $\mathcal{H}$. Then:

interior of $\mathcal{C}$ is

$\text{ int
}\mathcal{C}=\\{x\in\mathcal{C}:(\exists\rho>0)B(0;\rho)\subset\mathcal{C}-x\\};$

strong relative interior of $\mathcal{C}$ is

$\text{sri
}\mathcal{C}=\\{x\in\mathcal{C}:cone(\mathcal{C}-x)=\overline{span}(\mathcal{C}-x)\\};$

strong quasi-relative interior of $\mathcal{C}$ is

$\text{sqri
}\mathcal{C}=\\{x\in\mathcal{C}:\bigcup_{\rho>0}\rho(\mathcal{C}-x)\text{is a
closed linear subspace of space }\mathcal{H}\\}.$

In case $\mathcal{H}$ is finite dimensional, sqri and sri are equivalent.

######  Definition 2.2.

Let $C$ be a nonempty subset of $\mathcal{H},$ and $T:C\to\mathcal{H}$ be a
nonexpansive operator. $T$ is said to be

  1. (i)

nonexpansive if

$\|Tx-Ty\|\leq\|x-y\|\ \forall x,y\in C;$

  2. (ii)

firmly nonexpansive if

$\|Tx-Ty\|^{2}+\|(Id-T)x-(Id-T)y\|^{2}\leq\|x-y\|^{2}\ \forall x,y\in C;$

  3. (iii)

$\beta$-cocoercive ($\beta>0$) if

$\langle x-y,Tx-Ty\rangle\leq\beta\|Tx-Ty\|^{2}\ \forall x,y\in C$

  4. (iv)

$\alpha$-averaged for $\alpha\in(0,1)$ if there exists a nonexpansive operator
$R:C\to\mathcal{H}$ such that $T=(1-\alpha)Id+\alpha R$.

An operator $T:\mathcal{H}\to 2^{\mathcal{H}}$ is strongly monotone with
$\beta\in(0,\infty)$ implies $T^{-1}:\mathcal{H}\to\mathcal{H}$ is
$\beta$-cocoerceive.

######  Definition 2.3.

[6] Let $\mathcal{C}$ be a nonempty subset of $\mathcal{H}.$ Then:

The indicator function $i_{\mathcal{C}}:\mathcal{H}\to[-\infty,+\infty]$ is
defined by

$\displaystyle{i_{\mathcal{C}}(x)=\left\\{\begin{array}[]{lc@{}c@{}r}0,\ \ if\
x\in\mathcal{C}\\\ \infty\ \ otherwise\end{array}\right.}.$ (2.3)

The projection of a point $x\in\mathcal{H}$ on $\mathcal{C}$ is defined by
$\operatorname{proj}_{C}(x)=\left\\{u\in\mathcal{C}:u=argmin_{z\in\mathcal{C}}\|x-z\|\right\\}.$

Suppose $\mathcal{C}$ is convex, then normal cone to $\mathcal{C}$ at $x$ is
defined by

$\displaystyle{N_{\mathcal{C}}(x)=\left\\{\begin{array}[]{lc@{}c@{}r}u\in\mathcal{H}:\sup\langle
y-x,u\rangle\leq 0~{}\forall y\in\mathcal{C},\ if\ x\in\mathcal{C}\\\
\emptyset,\ \ \ \ otherwise.\end{array}\right.}$ (2.6)

######  Definition 2.4.

[6, Proposition 4.32] The parallel sum of two operators
$T_{1},T_{2}:\mathcal{H}\to 2^{\mathcal{H}}$ is $T_{1}\Box
T_{2}:\mathcal{H}\to 2^{\mathcal{H}}$ defined by $T_{1}\Box
T_{2}=(T_{1}^{-1}+T_{2}^{-1})^{-1}$.

The subdifferential of parallel sum of operators $T_{1}$ and $T_{2}$ is
$\partial(T_{1}\square T_{2})=\partial T_{1}\square\partial T_{2}$.

######  Remark 2.1.

If $T_{1}$ and $T_{2}$ are monotone then the set of zeros of their sum
$\operatorname{Zer}(T_{1}+T_{2})=J_{\gamma T_{2}}(\operatorname{Fix}(R_{\gamma
T_{1}}R_{\gamma T_{2}}))$ $\forall\gamma>0$ and $R_{\gamma T_{i}}=2J_{\gamma
T_{i}}-Id,i=1,2$.

######  Proposition 2.1.

[6] Consider $T_{1},T_{2}:\mathcal{H}\to\mathcal{H}$ be
$\alpha_{1},\alpha_{2}$-averaged operates, respectively. Then the averaged
operator $T_{1}\circ T_{2}$ is
$\alpha=\frac{\alpha_{1}+\alpha_{2}-2\alpha_{1}\alpha_{2}}{1-\alpha_{1}\alpha_{2}}$-averaged.

######  Lemma 2.1.

_[6, Corollary 4.18]_ Let $T:\mathcal{H}\to\mathcal{H}$ be a nonexpansive
mapping. Let $\\{u_{n}\\}$ be a sequence in $\mathcal{H}$ and
$u\in\mathcal{H}$ such that $u_{n}\rightharpoonup u$ and $u_{n}-Tu_{n}\to 0$
as $n\to\infty$. Then $u\in\operatorname{Fix}(T)$.

######  Lemma 2.2.

[55, Lemma 2.5] Let $\\{a_{n}\\}$ be a sequence of nonnegative real numbers
satisfying the inequality:

$a_{n+1}\leq(1-\theta_{n})a_{n}+\theta_{n}b_{n}+\epsilon_{n}\ \ \forall n\geq
0,$

where

  1. (i)

$0\leq\theta_{n}\leq 1$ for all $n\geq 0$ and $\sum_{n\geq
0}\theta_{n}=\infty;$

  2. (ii)

$\limsup_{n\to\infty}b_{n}\leq 0$;

  3. (iii)

$\epsilon_{n}\geq 0\text{ for all }n\geq 0\text{ and }\sum_{n\geq
0}\epsilon_{n}<\infty.$

Then the sequence $\\{a_{n}\\}$ converges to $0$.

##  3 Tikhonov Regularized Strongly Convergent Fixed Point Algorithm

This section devotes to investigate a computational theory for finding common
fixed points of nonexpansive operators. We introduce a common fixed point
algorithm such that sequence generated by the algorithm strongly converges to
the set of common fixed points of mappings.

######  Algorithm 3.1.

Let $\mathcal{S},\mathcal{T}:\mathcal{H}\to\mathcal{H}$ be nonexpansive
mappings. Select $\\{e_{n}\\}$, $\\{\theta_{n}\\}\subset(0,1)$ and compute the
$(n+1)^{th}$ iteration as follows:

$y_{n+1}=\mathcal{S}[(1-\theta_{n})e_{n}y_{n}+\theta_{n}\mathcal{T}(e_{n}y_{n})]\
\ \ \ \textrm{for all}\ n\in\mathbb{N}.$ (3.1)

We now study the convergence behavior of Algorithm 3.1 for finding the common
fixed point of $\mathcal{S}$ and $\mathcal{T}$.

######  Theorem 3.1.

Let $\mathcal{S},\mathcal{T}:\mathcal{H}\to\mathcal{H}$ be nonexpansive
mappings such that
$\Omega:=\operatorname{Fix}(\mathcal{T})\cap\operatorname{Fix}(\mathcal{S})\neq\emptyset$.
Let $\\{y_{n}\\}$ be a sequence in $\mathcal{H}$ defined by Algorithm 3.1,
where $\\{\theta_{n}\\}$ and $\\{e_{n}\\}$ are real sequences satisfy the
following conditions:

  1. (i)

$0<e_{n}<1$ for all $n\in\mathbb{N},$ $\lim\limits_{n\to\infty}e_{n}=1$,
$\sum_{n=1}^{\infty}(1-e_{n})=\infty$ and
$\sum_{n=1}^{\infty}|e_{n}-e_{n-1}|<\infty;$

  2. (ii)

$0<\underline{\theta}\leq\theta_{n}\leq\overline{\theta}<1$ for all
$n\in\mathbb{N},$ and $\sum_{n=1}^{\infty}|\theta_{n}-\theta_{n-1}|<\infty.$

Then the sequence $\\{y_{n}\\}$ converges strongly to
$\operatorname{proj}_{\Omega}(0).$

###### Proof.

In order to prove the convergence of the sequence $\\{y_{n}\\}$, we proceed
with following steps:

  1. Step 1.

Sequence $\\{y_{n}\\}$ is bounded.

Let $y\in\Omega$. Since $\mathcal{S}$ and $\mathcal{T}$ are nonexpansive, we
have following

$\displaystyle\|y_{n+1}-y\|$ $\displaystyle=$
$\displaystyle\|\mathcal{S}[(1-\theta_{n})e_{n}y_{n}+\theta_{n}\mathcal{T}(e_{n}y_{n})]-y\|$
$\displaystyle\leq$
$\displaystyle\|(1-\theta_{n})e_{n}y_{n}+\theta_{n}\mathcal{T}(e_{n}y_{n})-y\|$
$\displaystyle\leq$
$\displaystyle(1-\theta_{n})\|e_{n}y_{n}-y\|+\theta_{n}\|\mathcal{T}(e_{n}y_{n})-y\|$
$\displaystyle\leq$ $\displaystyle\|e_{n}y_{n}-y\|$ $\displaystyle=$
$\displaystyle\|e_{n}(y_{n}-y)-(1-e_{n})y\|$ $\displaystyle\leq$
$\displaystyle e_{n}\|(y_{n}-y)\|+(1-e_{n})\|y\|$ $\displaystyle\leq$
$\displaystyle\text{max}\\{\|y_{0}-y\|,\|y\|\\}.$

Thus, $\\{y_{n}\\}$ is bounded.

  2. Step 2.

$\|y_{n+1}-y_{n}\|\to 0$ as $n\to\infty.$

Using nonexpansivity of $\mathcal{S}\text{ and }\mathcal{T}$, we have

$\displaystyle\|y_{n+1}-y_{n}\|$ $\displaystyle=$
$\displaystyle\|\mathcal{S}[(1-\theta_{n})e_{n}y_{n}+\theta_{n}\mathcal{T}(e_{n}y_{n})]-\mathcal{S}[(1-\theta_{n-1})e_{n-1}y_{n-1}+\theta_{n-1}\mathcal{T}(e_{n-1}y_{n-1})]\|$
$\displaystyle\leq$
$\displaystyle\|(1-\theta_{n})e_{n}y_{n}+\theta_{n}\mathcal{T}(e_{n}y_{n})-(1-\theta_{n-1})e_{n-1}y_{n-1}-\theta_{n-1}\mathcal{T}(e_{n-1}y_{n-1})\|$
$\displaystyle=$
$\displaystyle\|(1-\theta_{n})e_{n}y_{n}-(1-\theta_{n-1})e_{n-1}y_{n-1}+\theta_{n}\mathcal{T}(e_{n}y_{n})-\theta_{n-1}\mathcal{T}(e_{n-1}y_{n-1})\|$
$\displaystyle\leq$
$\displaystyle\|(1-\theta_{n})(e_{n}y_{n}-e_{n-1}y_{n-1})+(\theta_{n-1}-\theta_{n})e_{n-1}y_{n-1})\|$
$\displaystyle+$
$\displaystyle\|\theta_{n}(\mathcal{T}(e_{n}y_{n})-\mathcal{T}(e_{n-1}y_{n-1}))+(\theta_{n}-\theta_{n-1})\mathcal{T}(e_{n-1}y_{n-1})\|$
$\displaystyle\leq$
$\displaystyle\|e_{n}y_{n}-e_{n-1}y_{n-1}\|+|\theta_{n}-\theta_{n-1}|\mathcal{C}_{1}$
$\displaystyle=$
$\displaystyle\|e_{n}(y_{n}-y_{n-1})+(e_{n}-e_{n-1})y_{n-1}\|+|\theta_{n}-\theta_{n-1}|\mathcal{C}_{1}$
$\displaystyle\leq$ $\displaystyle
e_{n}\|y_{n}-y_{n-1}\|+|e_{n}-e_{n-1}|\mathcal{C}_{2}+|\theta_{n}-\theta_{n-1}|\mathcal{C}_{1},$

for some $\mathcal{C}_{1},$ $\mathcal{C}_{2}>0$. By applying Lemma 2.2 with
$a_{n}=\|y_{n}-y_{n-1}\|,b_{n}=0$,
$\epsilon_{n}=|e_{n}-e_{n-1}|\mathcal{C}_{2}+|\theta_{n}-\theta_{n-1}|\mathcal{C}_{1}\text{
and }\theta_{n}=1-e_{n},\forall n\in\mathbb{N}$, we obtain that
$\|y_{n+1}-y_{n}\|\to 0$.

  3. Step 3.

$\|y_{n}-\mathcal{T}y_{n}\|$ and $\|y_{n}-\mathcal{S}y_{n}\|\to 0$ as
$n\to\infty$.

Let $y\in\Omega$. Note

$\displaystyle\|y_{n+1}-y\|^{2}$ $\displaystyle=$
$\displaystyle\|\mathcal{S}[(1-\theta_{n})e_{n}y_{n}+\theta_{n}\mathcal{T}(e_{n}y_{n})]-y\|^{2}$
(3.3) $\displaystyle\leq$
$\displaystyle\|(1-\theta_{n})e_{n}y_{n}+\theta_{n}\mathcal{T}(e_{n}y_{n})-y\|^{2}$
$\displaystyle=$
$\displaystyle(1-\theta_{n})\|e_{n}y_{n}-y\|^{2}+\theta_{n}\|\mathcal{T}(e_{n}y_{n})-y\|^{2}-\theta_{n}(1-\theta_{n})\|e_{n}y_{n}-\mathcal{T}(e_{n}y_{n})\|^{2}$
$\displaystyle\leq$
$\displaystyle(1-\theta_{n})\|e_{n}y_{n}-y\|^{2}+\theta_{n}\|e_{n}y_{n}-y\|^{2}-\theta_{n}(1-\theta_{n})\|e_{n}y_{n}-\mathcal{T}(e_{n}y_{n})\|^{2}$
$\displaystyle=$
$\displaystyle\|e_{n}y_{n}-y\|^{2}-\theta_{n}(1-\theta_{n})\|e_{n}y_{n}-\mathcal{T}(e_{n}y_{n})\|^{2},$

which implies that

$\displaystyle\theta_{n}(1-\theta_{n})$
$\displaystyle\|e_{n}y_{n}-\mathcal{T}(e_{n}y_{n})\|^{2}$
$\displaystyle\leq\|e_{n}y_{n}-y\|^{2}-\|y_{n+1}-y\|^{2}$
$\displaystyle=\|e_{n}y_{n}-e_{n}y+e_{n}y-y\|^{2}-\|y_{n+1}-y\|^{2}$
$\displaystyle=(1-e_{n})\|y\|^{2}+e_{n}\|y_{n}-y\|^{2}-e_{n}(1-e_{n})\|y_{n}\|^{2}-\|y_{n+1}-y\|^{2}$
$\displaystyle\leq(1-e_{n})\|y\|^{2}+e_{n}\|y_{n}-y\|^{2}-e_{n}(1-e_{n})\|y_{n}\|^{2}-e_{n}\|y_{n+1}-y\|^{2}$
$\displaystyle=e_{n}\\{\|y_{n}-y\|^{2}-\|y_{n+1}-y\|^{2}\\}+(1-e_{n})\|y\|^{2}$
$\displaystyle-e_{n}(1-e_{n})\|y_{n}\|^{2}.$ (3.4)

Since,

$\|y_{n}-y\|^{2}\leq\|y_{n}-y_{n+1}\|^{2}+\|y_{n+1}-y\|^{2}+2\|y_{n}-y_{n+1}\|\|y_{n+1}-y\|$

which can be rewritten as

$\|y_{n}-y\|^{2}-\|y_{n+1}-y\|^{2}\leq\|y_{n}-y_{n+1}\|^{2}+2\|y_{n}-y_{n+1}\|\|y_{n+1}-y\|.$

Thus (Step 3.) becomes

$\displaystyle\theta_{n}(1-\theta_{n})\|e_{n}y_{n}-\mathcal{T}(e_{n}y_{n})\|^{2}$
$\displaystyle\leq$ $\displaystyle
e_{n}\\{\|y_{n}-y_{n+1}\|^{2}+2\|y_{n}-y_{n+1}\|\|y_{n+1}-y\|\\}$ (3.5)
$\displaystyle+$
$\displaystyle(1-e_{n})\|y\|^{2}-e_{n}(1-e_{n})\|y_{n}\|^{2}.$

Using Step 1, Step 2 and condition (ii) in Theorem 3.1, we obtain
$\theta_{n}(1-\theta_{n})\|e_{n}y_{n}-\mathcal{T}(e_{n}y_{n})\|^{2}\to 0.$

Now

$\displaystyle\|y_{n}-\mathcal{T}y_{n}\|$ $\displaystyle=$
$\displaystyle\|y_{n}-e_{n}y_{n}+e_{n}y_{n}-\mathcal{T}(e_{n}y_{n})+\mathcal{T}(e_{n}y_{n})-\mathcal{T}y_{n}\|$
$\displaystyle\leq$
$\displaystyle\|y_{n}-e_{n}y_{n}\|+\|e_{n}y_{n}-\mathcal{T}(e_{n}y_{n})\|+\|\mathcal{T}(e_{n}y_{n})-\mathcal{T}y_{n}\|$
$\displaystyle\leq$ $\displaystyle
2(1-e_{n})\|y_{n}\|+\|e_{n}y_{n}-\mathcal{T}(e_{n}y_{n})\|\to 0\text{ as
}n\to\infty.$

Observe that

$\displaystyle\|y_{n}-\mathcal{S}y_{n}\|$ $\displaystyle\leq$
$\displaystyle\|y_{n}-y_{n+1}\|+\|y_{n+1}-\mathcal{S}y_{n}\|$ $\displaystyle=$
$\displaystyle\|y_{n}-y_{n+1}\|+\|\mathcal{S}[(1-\theta_{n})e_{n}y_{n}+\theta_{n}\mathcal{T}(e_{n}y_{n})]-\mathcal{S}y_{n}\|$
$\displaystyle\leq$
$\displaystyle\|y_{n}-y_{n+1}\|+\|(1-\theta_{n})e_{n}y_{n}+\theta_{n}\mathcal{T}(e_{n}y_{n})-y_{n}\|$
$\displaystyle\leq$
$\displaystyle\|y_{n}-y_{n+1}\|+(1-\theta_{n})\|e_{n}y_{n}-y_{n}\|+\theta_{n}\|\mathcal{T}(e_{n}y_{n})-y_{n}\|$
$\displaystyle\leq$
$\displaystyle\|y_{n}-y_{n+1}\|+(1-\theta_{n})(1-e_{n})\|y_{n}\|+\theta_{n}\|\mathcal{T}(e_{n}y_{n})-\mathcal{T}y_{n}+\mathcal{T}y_{n}-y_{n}\|$
$\displaystyle\leq$
$\displaystyle\|y_{n}-y_{n+1}\|+(1-\theta_{n})(1-e_{n})\|y_{n}\|+\theta_{n}\|e_{n}y_{n}-y_{n}\|+\theta_{n}\|\mathcal{T}y_{n}-y_{n}\|$
$\displaystyle=$
$\displaystyle\|y_{n}-y_{n+1}\|+(1-e_{n})\|y_{n}\|+\theta_{n}\|\mathcal{T}y_{n}-y_{n}\|\to
0\text{ as }n\to\infty.$

Since $e_{n}\to 1$ and $\|y_{n}-y_{n-1}\|$ and $\|y_{n}-\mathcal{T}y_{n}\|\to
0\text{ as }n\to\infty,$ we have $\|y_{n}-\mathcal{S}y_{n}\|\to 0.$

  4. Step 4.

$\\{y_{n}\\}$ converges strongly to $\bar{y}=\operatorname{proj}_{\Omega}(0).$

From (Step 1.), we set

$\displaystyle\|y_{n+1}-\bar{y}\|$ $\displaystyle\leq$
$\displaystyle\|\mathcal{S}[(1-\theta_{n})e_{n}y_{n}+\theta_{n}\mathcal{T}(e_{n}y_{n})]-\bar{y}\|$
(3.6) $\displaystyle=$
$\displaystyle\|(1-\theta_{n})e_{n}y_{n}+\theta_{n}\mathcal{T}(e_{n}y_{n})-\bar{y}\|$
$\displaystyle=$
$\displaystyle\|(1-\theta_{n})(e_{n}y_{n}-\bar{y})+\theta_{n}\mathcal{T}((e_{n}y_{n})-\bar{y})\|$
$\displaystyle\leq$ $\displaystyle\|e_{n}y_{n}-\bar{y}\|.$

Hence

$\displaystyle\|y_{n+1}-\bar{y}\|^{2}$ $\displaystyle\leq$
$\displaystyle\|e_{n}y_{n}-\bar{y}\|^{2}$ (3.7) $\displaystyle\leq$
$\displaystyle\|e_{n}(y_{n}-\bar{y})-(1-e_{n})\bar{y}\|^{2}$
$\displaystyle\leq$ $\displaystyle
e_{n}^{2}\|y_{n}-\bar{y}\|^{2}+2e_{n}(1-e_{n})\langle-\bar{y},y_{n}-\bar{y}\rangle+(1-e_{n})^{2}\|\bar{y}\|^{2}$
$\displaystyle\leq$ $\displaystyle
e_{n}\|y_{n}-\bar{y}\|^{2}+2e_{n}(1-e_{n})\langle-\bar{y},y_{n}-\bar{y}\rangle+(1-e_{n})\|\bar{y}\|^{2}.$

Next we show that

$\limsup_{n\to\infty}\langle-\bar{y},y_{n}-\bar{y}\rangle\leq 0.$ (3.8)

Contrarily assume a real number $l$ and a subsequence $\\{y_{n_{j}}\\}$ of
$\\{y_{n}\\}$ satisfying

$\langle-\bar{y},y_{n_{j}}-\bar{y}\rangle\geq l>0~{}\forall j\in\mathbb{N}.$
(3.9)

Since $\\{y_{n}\\}$ is bounded, there exists a subsequence $\\{y_{n_{j}}\\}$
which converges weakly to an element $y\in\mathcal{H}.$ Lemma 2.1 alongwith
Step Step 2. implies that $y\in\Omega.$ By using variational characterazation
of projection, we can easily derive

$\lim_{j\to\infty}\langle-\bar{y},y_{n_{j}}-\bar{y}\rangle=\langle-\bar{y},y-\bar{y}\rangle\leq
0,$ (3.10)

which is a contradiction. Thus, (3.8) holds and

$\displaystyle\limsup_{n\to\infty}\left(2e_{n}\langle-\bar{y},y_{n}-\bar{y}\rangle+(1-e_{n})\|\bar{y}\|^{2}\right)\leq
0.$ (3.11)

Consider
$a_{n}=\|y_{n}-\bar{y}\|^{2},b_{n}=2e_{n}\langle-\bar{y},y_{n}-\bar{y}\rangle+(1-e_{n})\|\bar{y}\|^{2},\epsilon_{n}=0$
and $\theta_{n}=1-e_{n}$ in (3.7) and apply Lemma 2.2, we get the desired
conclusion.

∎

######  Corollary 3.1.

Let $R_{1},R_{2}:\mathcal{H}\to\mathcal{H}$ be
$\alpha_{1},\alpha_{2}$-averaged operators respectively, such that
$\operatorname{Fix}{(R_{1})}\cap\operatorname{Fix}(R_{2})\neq\emptyset$. For
$y_{1}\in\mathcal{H}$, let $\\{y_{n}\\}$ be sequence in $\mathcal{H}$ defined
by

$y_{n+1}=R_{2}\\{e_{n}y_{n}+\theta_{n}(R_{1}(e_{n}y_{n})-e_{n}y_{n})\\}\ \
\forall n\in\mathbb{N},$ (3.12)

where $\\{\theta_{n}\\}$ and $\\{e_{n}\\}$ are real sequences satisfy the
condition (i) given in Theorem 3.1 and the conditions:

$0<\underline{\Theta}\leq\alpha_{1}\theta_{n}\leq\overline{\Theta}<1\textrm{
for all }n\in\mathbb{N}\textrm{ and
}\sum_{n=1}^{\infty}|\theta_{n}-\theta_{n-1}|<\infty.$

Then the sequence $\\{y_{n}\\}$ converges strongly to
$\operatorname{proj}_{\operatorname{Fix}{(R_{1})}\cap\operatorname{Fix}(R_{2})}(0).$

##  4 Tikhonov Regularized Forward-Backward-type Algorithms

In this section, we propose a forward-backward algorithm based on Algorithm
3.1 to simultaneously solve the monotone inclusion problems of the sum of two
maximally monotone operators in which one is single-valued. Further, we also
propose a forward-backward-type primal-dual algorithm based on Algorithm 3.1
to solve a complexly structured monotone inclusion problem containing
composition with linear operators and parallel-sum operators.

###  4.1 Tikhonov Regularized Forward-Backward Algorithm

Let $A_{1},A_{2}:\mathcal{H}\to 2^{\mathcal{H}}$ be maximally monotone
operators and $B_{1},B_{2}:\mathcal{H}\to\mathcal{H}$ be
$\alpha_{1},\alpha_{2}$-cocoercive operators. We consider the monotone
inclusion problem

$\text{ find }x\in\mathcal{H}\text{ such that
}0\in(A_{1}+B_{1})x\cap(A_{2}+B_{2})x.$ (4.1)

We propose a forward-backward algorithm to solve the monotone inclusion
problem (4.1) such that generated sequence converges strongly to the solution
set of the Problem (4.1).

######  Theorem 4.1.

Suppose
$\operatorname{Zer}(A_{1}+B_{1})\cap\operatorname{Zer}(A_{2}+B_{2})\neq\emptyset$
and $\gamma_{1}\in(0,2\alpha_{1})\text{ and }\gamma_{2}\in(0,2\alpha_{2}).$
For $y_{1}\in\mathcal{H}$, consider the forward-backward algorithm defined as
follows:

$y_{n+1}=J_{\gamma_{2}A_{2}}(Id-\gamma_{2}B_{2})\left\\{(1-\theta_{n})e_{n}y_{n}+\theta_{n}J_{\gamma_{1}A_{1}}(e_{n}y_{n}-\gamma_{1}B_{1}(e_{n}y_{n}))\right\\}\forall
n\in\mathbb{N}.$ (4.2)

where $\\{\theta_{n}\\}$ and $\\{e_{n}\\}$ are real sequences satisfy the
condition (i) given in Theorem 3.1 and the conditions:

$0<\underline{\Theta}\leq\frac{2\alpha_{1}}{4\alpha_{1}-\gamma_{1}}\theta_{n}\leq\overline{\Theta}<1\textrm{
for all }n\in\mathbb{N}\textrm{ and
}\sum_{n=1}^{\infty}|\theta_{n}-\theta_{n-1}|<\infty.$

Then $\\{y_{n}\\}$ converges strongly to
$\operatorname{proj}_{\operatorname{Zer}(A_{1}+B_{1})\cap\operatorname{Zer}(A_{2}+B_{2})}(0).$

###### Proof.

Set $T_{1}=J_{\gamma_{1}A_{1}}(Id-\gamma_{1}B_{1})$ and
$T_{2}=J_{\gamma_{2}A_{2}}(Id-\gamma_{2}B_{2}),$ then algorithm (4.2) can be
rewritten as:

$y_{n+1}=T_{2}\\{(1-\theta_{n})e_{n}y_{n}+\theta_{n}T_{1}(e_{n}y_{n})\\}\
\forall\ n\in\mathbb{N}.$ (4.3)

Since $J_{\gamma_{1}A_{1}}$ is $\frac{1}{2}$-cocoercive [6, Corollary 23.8]
and $Id-\gamma_{1}B_{1}$ is $\frac{\gamma_{1}}{2\alpha_{1}}$-averaged [6,
proposition 4.33], $T_{1}$ is
$\frac{2\alpha_{1}}{4\alpha_{1}-\gamma_{1}}$-averaged. Using the fact that
$\operatorname{Zer}(A_{i}+B_{i})=\operatorname{Fix}(T_{i}),i=1,2$ and
assumption
$\operatorname{Zer}(A_{1}+B_{1})\cap\operatorname{Zer}(A_{2}+B_{2})\neq\emptyset$
[6, Proposition 25.1],
$\operatorname{Fix}(T_{1})\cap\operatorname{Fix}(T_{2})\neq\emptyset.$
Therefore, Theorem 4.1 follows from Corollary 3.1. ∎

Further, we consider the following minimization problem and propose a
proximal-point-type algorithm to solve it.

######  Problem 4.1.

Consider strictly positive real numbers $\beta_{1},\beta_{2}.$ Let
$f_{1},f_{2}:\mathcal{H}\to\mathbb{R}\cup\\{\infty\\}$ be proper convex lower
semicontinuous functions and $g_{1},g_{2}:\mathcal{H}\to\mathbb{R}$ be convex
and Frechet-differentiable functions with
$\frac{1}{\beta_{1}},\frac{1}{\beta_{2}}$-Lipschitz continuous gradients,
respectively. The problem is to find a point $y\in\mathcal{H}$ satisfying

$y\in argmin_{x\in\mathcal{H}}\left\\{(f_{1}+g_{1})(x)\right\\}\bigcap
argmin_{x\in\mathcal{H}}\left\\{(f_{2}+g_{2})(x)\right\\}.$ (4.4)

######  Theorem 4.2.

Consider the functions $f_{1},f_{2},g_{1}\text{ and }g_{2}$ are as in Problem
4.1. Let $argmin(f_{1}+g_{1})\cap argmin(f_{2}+g_{2})\neq\emptyset.$ For
$\gamma_{1}\in(0,2\beta_{1}]\text{ and }\gamma_{2}\in(0,2\beta_{2}],$ consider
an algorithm with initial point $y_{1}\in\mathcal{H},$

$y_{n+1}=prox_{\gamma_{2}f_{2}}(Id-\gamma_{2}\nabla
g_{2})\\{(1-\theta_{n})e_{n}y_{n}+\theta_{n}prox_{\gamma_{1}f_{1}}(e_{n}y_{n}-\gamma_{1}\nabla
g_{1}(e_{n}y_{n}))\\}\ \ \forall n\in\mathbb{N},$ (4.5)

where $\theta_{n}\in(0,1]\text{ and
}e_{n}\in(0,\frac{4\beta_{1}-\gamma_{1}}{2\beta_{1}})$ satisfy the condition
(i) in Theorem 3.1 and the conditions:

$0<\underline{\Theta}\leq\frac{2\beta_{1}}{4\beta_{1}-\gamma_{1}}\theta_{n}\leq\overline{\Theta}<1\textrm{
and }\sum_{n=1}^{\infty}|\theta_{n}-\theta_{n-1}|<\infty.$

Then $\\{y_{n}\\}$ converges strongly to minimal norm solution $y$ of Problem
4.1.

###### Proof.

Consider $A_{1}=\partial f_{1},A_{2}=\partial f_{2},B_{1}=\nabla
g_{1},B_{1}=\nabla g_{2}$. Since

$\operatorname{Zer}(\partial f_{i}+\nabla g_{i})=argmin(f_{i}+g_{i}),\ i=1,2$

and $\nabla g_{1},\nabla g_{2}$ are $\beta_{1},\beta_{2}$-cocoercive,
respectively (Ballion-Hadded Theorem [6, Corollary 16.18]). Thus, by Theorem
4.1, $\\{y_{n}\\}$ converges strongly to a point $y$ in
$argmin(f_{1}+g_{1})\cap argmin(f_{2}+g_{2})$. ∎

###  4.2 Tikhonov Regularized Forward-Backward-type Primal-Dual Algorithm

######  Problem 4.2.

Suppose $\Omega_{1},\dots,\Omega_{m}$ are real Hilbert spaces. Consider the
following operators:

  1. $\bullet A,B:\mathcal{H}\to 2^{\mathcal{H}}$ are maximally monotone operators,
$\bullet C,D:\mathcal{H}\to\mathcal{H}$ are $\mu_{1},\mu_{2}$-cocoercive
operators, respectively, $\mu_{1},\mu_{2}>0,$
$\bullet P_{i},Q_{i},R_{i},S_{i}:\Omega_{i}\to 2^{\Omega_{i}}$ are maximally
monotone operators such that $Q_{i}$ is $\nu_{i}$-strongly monotone and
$S_{i}$ is $\delta_{i}$-strongly monotone, $\nu_{i},\delta_{i}>0,$
$i=1,\dots,m$,
$\bullet$ nonzero continuous linear operators
$L_{i}:\mathcal{H}\to\Omega_{i},i=1,\dots,m.$

The primal inclusion problem is to find $\bar{y}\in\mathcal{H}$ satisfying

$\displaystyle 0\in A\bar{y}+\sum_{i=1}^{m}L_{i}^{*}(P_{i}\square
Q_{i})(L_{i}\bar{y})+C\bar{y}$ and $\displaystyle 0\in
B\bar{y}+\sum_{i=1}^{k}L_{i}^{*}(R_{i}\square S_{i})(L_{i}\bar{y})+D\bar{y}$

together with dual inclusion problem

$\displaystyle{find\ \
\bar{v}_{1}\in\varOmega_{1},\dots,\bar{v}_{m}\in\varOmega_{m}\text{ such that
}\exists y\in\mathcal{H}\text{ and
}\left\\{\begin{array}[]{lc@{}c@{}r}-\sum_{i=1}^{m}L_{i}^{*}\bar{v}_{i}\in
Ay+Cy\\\ \bar{v}_{i}\in(P_{i}\square Q_{i})(L_{i}y)\\\ \text{ and }\\\
-\sum_{i=1}^{m}L_{i}^{*}\bar{v}_{i}\in By+Dy\\\ \bar{v}_{i}\in(R_{i}\square
S_{i})(L_{i}y),\ \ \ i=1,\dots m.\end{array}\right.}$ (4.11)

A point
$(\bar{y},\bar{v}_{1},\dots,\bar{v}_{m})\in\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m}$
be a primal-dual solution of Problem 4.2 if it satisfies the following:

$\displaystyle{\left\\{\begin{array}[]{lc@{}c@{}r}-\sum_{i=1}^{m}L_{i}^{*}\bar{v}_{i}\in
A\bar{y}+C\bar{y},\\\ -\sum_{i=1}^{m}L_{i}^{*}\bar{v}_{i}\in
B\bar{y}+D\bar{y},\\\ \bar{v}_{i}\in(P_{i}\square Q_{i})(L_{i}\bar{y}),\\\
\bar{v}_{i}\in(R_{i}\square S_{i})(L_{i}\bar{y})\ \ \
i=1,\dots,m.\end{array}\right.}$ (4.16)

######  Theorem 4.3.

Consider the operators as in Problem 4.2. Assume

$0\in ran\left(A+\sum_{i=1}^{m}L_{i}^{*}\circ(P_{i}\square Q_{i})\circ
L_{i}+C\right)\bigcap ran\left(B+\sum_{i=1}^{m}L_{i}^{*}\circ(R_{i}\square
S_{i})\circ L_{i}+D\right).$ (4.17)

Let $\tau,\sigma_{1},\dots,\sigma_{m}>0$ such that

$2\rho\min\\{\beta_{1},\beta_{2}\\}\geq 1,$

where

$\displaystyle\rho=\min\left\\{\frac{1}{\tau},\frac{1}{\sigma_{1}},\dots,\frac{1}{\sigma_{m}}\right\\}\left(1-\sqrt{\tau\sum_{i=1}^{m}\sigma_{i}\|L_{i}\|^{2}}\right)$
$\displaystyle\beta_{1}=\min\\{\mu_{1},\nu_{1},\dots,\nu_{m}\\}\textrm{ and
}\beta_{2}=\min\\{\mu_{2},\delta_{1},\dots,\delta_{m}\\}.$

Consider the algorithm with initial point
$(y_{1},v_{1,1},\dots,v_{m,1})\in\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m}$
and defined by

Input:

  1. 1.

initial points
$(y_{1},v_{1,1},\dots,v_{m,1})\in\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m},$

  2. 2.

real numbers $\tau,\sigma_{i}>0$, $i=1,2,...,m$ be such that
$\tau\sum_{i=1}^{m}\sigma_{i}\|L_{i}\|^{2}<4,$

  3. 3.

$\theta_{n}\in(0,\frac{4\beta_{1}\rho-1}{2\beta_{1}\rho}],e_{n}\in(0,1)$ .

For $n\in\mathbb{N}$;

$p_{n}=J_{\tau
A}\left[e_{n}y_{n}-\tau\left(e_{n}\sum_{i=1}^{m}L_{i}^{*}v_{i,n}+C(e_{n}y_{n})\right)\right]$

$u_{n}=e_{n}y_{n}+\theta_{n}(p_{n}-e_{n}y_{n})$

For $i=1,\dots,m$;

$q_{i,n}=J_{\sigma_{i}P_{i}^{-1}}\left[e_{n}v_{i,n}+\sigma_{i}(L_{i}(2p_{n}-e_{n}y_{n})-Q_{i}^{-1}(e_{n}v_{i,n}))\right]$

$u_{i,n}=e_{n}v_{i,n}+\theta_{n}(q_{i,n}-e_{n}v_{i,n})$

$y_{n+1}=J_{\tau
B}\left[u_{n}-\tau\left(\sum_{i=1}^{m}L_{i}^{*}u_{i,n}+D(u_{n})\right)\right]$

$v_{i,n+1}=J_{\sigma_{i}R_{i}^{-1}}\left[u_{i,n}+\sigma_{i}(L_{i}(2y_{n+1}-u_{n})-S_{i}^{-1}(u_{i,n}))\right]$

Output: $(y_{n+1},v_{1,n+1},\dots,v_{m,n+1})$

Algorithm 1 To optimize the complexly structured Problem 4.2

where sequences $\\{\theta_{n}\\}$ and $\\{e_{n}\\}$ satisfy the condition (i)
and the conditions:

$0<\underline{\Theta}\leq\frac{2\beta_{1}\rho}{4\beta_{1}\rho-1}\theta_{n}\leq\overline{\Theta}<1\textrm{
and }\sum_{n=1}^{\infty}|\theta_{n}-\theta_{n-1}|<\infty.$

Then there exists
$(\bar{y},\bar{v}_{1},\dots,\bar{v}_{m})\in\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m}$
such that sequence $\\{(y_{n},v_{1,n},\dots,v_{m,n})\\}$ converges strongly to
$(\bar{y},\bar{v}_{1},\dots,\bar{v}_{m})$ and satisfies the Problem 4.2.

###### Proof.

Consider the real Hilbert space $\mathcal{K}\equiv$
$\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m}$ endowed with
inner product

$\langle(x,u_{1},\dots,u_{m}),(y,v_{1},\dots,v_{m})\rangle_{\mathcal{K}}=\langle
x,y\rangle_{\mathcal{H}}+\sum_{i=1}^{m}\langle
u_{i},v_{i}\rangle_{\varOmega_{i}}$

and corresponding norm

$\|(x,u_{1},\dots,u_{m})\|_{\mathcal{K}}=\sqrt{\|x\|_{\mathcal{H}}^{2}+\sum_{i=1}^{m}\|u_{i}\|^{2}_{\varOmega_{i}}},\
\ \ \forall(x,u_{1},\dots u_{m}),(y,v_{1},\dots,v_{m})\in\mathcal{K}.$

Further we consider following operators on real Hilbert space $\mathcal{K}$:

  1. 1.

$\phi_{1}:\mathcal{K}\to 2^{\mathcal{K}}$, defined by
$(x,u_{1},\dots,u_{m})\mapsto(Ax,P_{1}^{-1}u_{1},\dots,P_{m}^{-1}u_{m}),$

  2. 2.

$\phi_{2}:\mathcal{K}\to 2^{\mathcal{K}}$, defined by
$(x,u_{1},\dots,u_{m})\mapsto(Bx,R_{1}^{-1}u_{1},\dots,R_{m}^{-1}u_{m}),$

  3. 3.

$\xi:\mathcal{K}\to\mathcal{K}$, defined by
$(x,u_{1},\dots,u_{m})\mapsto\left(\sum_{i=1}^{m}L_{i}^{*}u_{i},-L_{1}x,\dots,-L_{m}x\right),$

  4. 4.

$\psi_{1}:\mathcal{K}\to\mathcal{K}$, defined by
$(x,u_{1},\dots,u_{m})\mapsto\left(Cx,Q_{1}^{-1}u_{1},\dots,Q_{m}^{-1}u_{m}\right),$

  5. 5.

$\psi_{2}:\mathcal{K}\to\mathcal{K}$, defined by
$(x,u_{1},\dots,u_{m})\mapsto\left(Dx,S_{1}^{-1}u_{1},\dots,S_{m}^{-1}u_{m}\right).$

These operators are maximally monotone as $A,B,P_{i},R_{i},Q_{i},S_{i},\
i=1,\dots,m$ are maximally monotone, $C,D\text{ are
}\mu_{1},\mu_{2}$-cocoercive, respectively and $\xi$ is skew-symmetric, i.e.,
$\xi^{*}=-\xi$ ([6, Proposition 20.22, 20.23 and Example 20.30]). Now, define
the continuous linear operator $\textbf{V}:\mathcal{K}\to\mathcal{K}$ by,

$(x,u_{1},\dots,u_{m})\to\left(\frac{x}{\tau}-\sum_{i=1}^{m}L_{i}^{*}u_{i},\frac{u_{1}}{\sigma_{1}}-L_{1}x,\dots,\frac{u_{m}}{\sigma_{m}}-L_{m}x\right),$

which is self-adjoint and $\rho$-strongly positive, i.e.,
$\langle\textbf{x},\textbf{Vx}\rangle_{\mathcal{K}}\geq\rho\|\textbf{x}\|_{\mathcal{K}}^{2}\
\ \forall\textbf{x}\in\mathcal{K}$ [54]. Therefore inverse of operator V
exists and satisfy $\|\textbf{V}^{-1}\|\leq\frac{1}{\rho}.$
Now, consider the sequences

$\displaystyle{\forall
n\in\mathbb{N}\left\\{\begin{array}[]{lc@{}c@{}r}\textbf{y}_{n}$=$(y_{n},v_{1,n},\dots,v_{m,n}),\\\
\textbf{u}_{n}$=$(u_{n},u_{1,n},\dots,u_{m,n}),\\\
\textbf{x}_{n}=(p_{n},q_{1,n},\dots,q_{m,n}).\end{array}\right.}$ (4.21)

By taking into account the sequences
$\\{\textbf{y}_{n}\\}$,$\\{\textbf{x}_{n}\\}$ and $\\{\textbf{u}_{n}\\}$ and
operator V, Algorithm 1 can be rewritten as

$\displaystyle{\forall
n\in\mathbb{N}\left\\{\begin{array}[]{lc@{}c@{}r}e_{n}\text{{V}}(\text{{y}}_{n})-\text{{V}}(\text{{x}}_{n})-\text{{$\psi_{1}$}}(e_{n}\text{{y}}_{n})\in(\phi_{1}+\xi)(\text{{x}}_{n})\\\
\text{{u}}_{n}=e_{n}\text{{y}}_{n}+\theta_{n}(\text{{x}}_{n}-e_{n}\text{{y}}_{n})\\\
\text{{V}{$\textbf{u}_{n}$}-{V}{y}}_{n+1}-\psi_{2}\text{{$\textbf{u}_{n}$}}\in(\phi_{2}+\xi)\textbf{y}_{n+1}.\end{array}\right.}$
(4.25)

On further analysing Algorithm 1, we get

$\displaystyle{\forall
n\in\mathbb{N}\left\\{\begin{array}[]{lc@{}c@{}r}\textbf{x}_{n}=J_{\textbf{A}_{1}}(e_{n}\textbf{y}_{n}-\textbf{B}_{1}(e_{n}\textbf{y}_{n}))\\\
\textbf{u}_{n}=e_{n}\textbf{y}_{n}+\theta_{n}(\textbf{x}_{n}-e_{n}\textbf{y}_{n})\\\
\textbf{y}_{n+1}=J_{\textbf{A}_{2}}(\textbf{u}_{n}-\textbf{B}_{2}\textbf{u}_{n}),\end{array}\right.}$
(4.29)

where $\textbf{A}_{1}=\textbf{V}^{-1}(\phi_{1}+\xi)$ ,
$\textbf{B}_{1}=\textbf{V}^{-1}\psi_{1}$,
$\textbf{A}_{2}=\textbf{V}^{-1}(\phi_{2}+\xi)$ and
$\textbf{B}_{2}=\textbf{V}^{-1}\psi_{2}.$ The Algorithm (4.29) is in the form
of Algorithm (4.1) for $\gamma=1$ and $A_{i}=\textbf{A}_{i}$ and
$B_{i}=\textbf{B}_{i},\ i=1,2$. Now, we define the real Hilbert space
$\mathcal{K}_{\textbf{V}}\equiv\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m}$
endowed with inner product
$\langle\textbf{x},\textbf{y}\rangle_{\mathcal{K}_{\textbf{V}}}=\langle\textbf{x},{\textbf{V}}\textbf{y}\rangle_{\mathcal{K}}$
and corresponding norm is given by,
$\|\textbf{x}\|_{\mathcal{K}_{\textbf{V}}}=\sqrt{\langle\textbf{x},{\textbf{V}}\textbf{x}\rangle_{\mathcal{K}}}\
\ \forall\textbf{x},\textbf{y}\in\mathcal{K}_{\textbf{V}}.$
In view of real Hilbert space $\mathcal{K}_{\textbf{V}}$ and Algorithm 1, we
observe the following:

  1. (i)

since dom$(\xi)=\mathcal{K}$, $\phi_{i}+\xi$ are maximally monotone on
$\mathcal{K}$ and thus maximal monotonocity of $\textbf{A}_{i}$ and
$\textbf{B}_{i}$ on $\mathcal{K}_{\textbf{V}}$ are followed, for $i=1,2$ [54].

  2. (ii)

$\textbf{B}_{i}$ are $\beta_{i}\rho$-cocoercive on $\mathcal{K}_{\textbf{V}}$
as $\psi_{i}$ are $\beta_{i}$-cocoercive in $\mathcal{K}$ [54, Page 672], for
$i=1,2$.

  3. (iii)

$\operatorname{Zer}(\textbf{A}_{i}+\textbf{B}_{i}$)=$\operatorname{Zer}({\textbf{V}^{-1}}(\phi_{i}+\xi+\psi_{i}))$=$\operatorname{Zer}(\phi_{i}+\xi+\psi_{i}),\
i=1,2$ and from condition $(\ref{e6.4})$, we can easily obtain that
$\operatorname{Zer}(\textbf{A}_{1}+\textbf{B}_{1})\cap\operatorname{Zer}(\textbf{A}_{2}+\textbf{B}_{2})\neq\emptyset.$

Since V is self-adjoint and $\rho$-strongly positive, weak convergence and
strong convergence of sequences are same in both Hilbert spaces $\mathcal{K}$
and $\mathcal{K}_{\textbf{V}}$. Operators $\textbf{A}_{i},\textbf{B}_{i},\
i=1,2$ and sequences $\\{\theta_{n}\\},\\{e_{n}\\}$ satisfy the assumptions in
Theorem 4.1, therefore, according to Theorem 4.1, $\\{\textbf{y}_{n}\\}$
converges strongly to $(\bar{y},\bar{v}_{1},\dots,\bar{v}_{m})\in$
$\operatorname{proj}_{\operatorname{Zer}(\textbf{A}_{1}+\textbf{B}_{1})\cap\operatorname{Zer}(\textbf{A}_{2}+\textbf{B}_{2})}(0,\dots,0)$
in the space ${\mathcal{K}}_{\textbf{V}}$ as $n\to\infty.$ Thus, we obtain the
conclusion as
$(\bar{y},\bar{v}_{1},\dots,\bar{v}_{m})\in\operatorname{Zer}(\phi_{1}+\xi+\psi_{1})\cap\operatorname{Zer}(\phi_{2}+\xi+\psi_{2}),$
also satisfy primal-dual Problem 4.2.

∎

Next, we define a complexly structured convex optimization problem and their
Fenchel duals. Further, we propose an algorithm to solve the considered
problem and study the convergence property of algorithm to find simultaneously
the common solutions of optimization problems and common solutions of their
Fenchel duals. Let $m$ is a positive integer. We have considered the following
problem:

######  Problem 4.3.

Let $f_{1},f_{2}\in\varGamma(\mathcal{H})$ and $h_{1},h_{2}$ be convex
differentiable function with $\mu_{1}^{-1},\mu_{2}^{-1}$\- Lipschitz
continuous gradients, for some $\mu_{1},\mu_{2}>0.$ Let $\varOmega_{i}$ be
real Hilbert spaces and $g_{i},l_{i},s_{i},t_{i}\in\varGamma(\varOmega_{i})$
such that $l_{i},t_{i}$ are $\nu_{i},\delta_{i}$-strongly convex,
respectively, and $L_{i}:\mathcal{H}\to\varOmega_{i}$ be non-zero linear
continuous operator $\forall i=1,2,\dots,m$. The optimization problem under
consideration is to find $y\in\mathcal{H}$ such that

$\displaystyle y\in
argmin_{x\in\mathcal{H}}\left\\{f_{1}(x)+\sum_{i=1}^{m}(g_{i}\square
l_{i})(L_{i}x)+h_{1}(x)\right\\}$ $\displaystyle\bigcap
argmin_{x\in\mathcal{H}}\left\\{f_{2}(x)+\sum_{i=1}^{m}(s_{i}\square
t_{i})(L_{i}x)+h_{2}(x)\right\\}$ (4.30)

with its Fenchel-dual problem is to find
$(v^{*}_{1},\dots,v^{*}_{m})\in\varOmega_{1}\times\cdots\times\varOmega_{m}$
such that

$\displaystyle(v^{*}_{1},\dots,v^{*}_{m})\in
argmax_{(v_{1},\dots,v_{m})\in\varOmega_{1}\times\cdots\times\varOmega_{m}}\left\\{-(f_{1}^{*}\square
h_{1}^{*})\left(-\sum_{i=1}^{m}L_{i}^{*}v_{i}\right)-\sum_{i=1}^{m}\left(g_{i}^{*}(v_{i})+l_{i}^{*}(v_{i})\right)\right\\}$
$\displaystyle\bigcap
argmax_{(v_{1},\dots,v_{m})\in\varOmega_{1}\times\cdots\times\varOmega_{m}}\left\\{-(f_{2}^{*}\square
h_{2}^{*})\left(-\sum_{i=1}^{m}L_{i}^{*}v_{i}\right)-\sum_{i=1}^{m}\left(s_{i}^{*}(v_{i})+t_{i}^{*}(v_{i})\right)\right\\}.$
(4.31)

In following corollary, we propose an algorithm and study its convergence
behaviour. The point of convergence will be the solution of Problem 4.3.

######  Corollary 4.1.

Assume in Problem 4.3

$0\in ran\left(\partial f_{1}+\sum_{i=1}^{m}L_{i}^{*}\circ(\partial
g_{i}\square\partial l_{i})\circ L_{i}+\nabla h_{1}\right)\bigcap
ran\left(\partial f_{2}+\sum_{i=1}^{m}L_{i}^{*}\circ(\partial
s_{i}\square\partial t_{i})\circ L_{i}+\nabla h_{2}\right).$ (4.32)

Consider $\tau,\sigma_{i}>0,\ i=1,2,\dots,m$ such that

$2\rho\min\\{\beta_{1},\beta_{2}\\}\geq 1.$

where

$\displaystyle\rho=\min\\{\tau^{-1},\sigma_{1}^{-1},\dots,\sigma_{m}^{-1}\\}\left(1-\sqrt{\tau\sum_{i=1}^{m}\sigma_{i}\|L_{i}\|^{2}}\right),$
$\displaystyle\beta_{1}=\min\\{\mu_{1},\nu_{1},\dots,\nu_{m}\\}\textrm{ and
}\beta_{2}=\min\\{\mu_{2},\delta_{1},\dots,\delta_{m}\\}.$

Consider the iterative scheme with intial point
$(x_{1},v_{1,1},\dots,v_{m,1})\in\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m}$
and defined by

$\displaystyle{\forall
n\in\mathbb{N}\left\\{\begin{array}[]{lc@{}c@{}r}p_{n}=prox_{\tau
f_{1}}\left[e_{n}x_{n}-\tau\left(e_{n}\sum_{i=1}^{m}L_{i}^{*}v_{i,n}+\nabla
h_{1}(e_{n}x_{n})\right)\right]\\\
u_{n}=e_{n}x_{n}+\theta_{n}(p_{n}-e_{n}x_{n})\\\ For\ i=1,2,\dots,m\\\
q_{i,n}=prox_{\sigma_{i}g_{i}^{*}}\left[e_{n}v_{i,n}+\sigma_{i}(L_{i}(2p_{n}-e_{n}x_{n})-\nabla
l_{i}^{*}(e_{n}v_{i,n}))\right]\\\
u_{i,n}=e_{n}v_{i,n}+\theta_{n}(q_{i,n}-e_{n}v_{i,n})\\\ x_{n+1}=prox_{\tau
f_{2}}\left[u_{n}-\tau\left(\sum_{i=1}^{m}L_{i}^{*}u_{i,n}+\nabla
h_{2}(u_{n})\right)\right]\\\
v_{i,n+1}=prox_{\sigma_{i}s_{i}^{*}}\left[u_{i,n}+\sigma_{i}(L_{i}(2x_{n+1}-u_{n})-\nabla
t_{i}^{*}(u_{i,n}))\right],\end{array}\right.}$ (4.40)

where sequences $\\{\theta_{n}\\}$ and $\\{e_{n}\\}$ satisfy the condition (i)
in Theorem 3.1 and the conditions:

$0<\underline{\Theta}\leq\frac{2\beta_{1}\rho}{4\beta_{1}\rho-1}\theta_{n}\leq\overline{\Theta}<1\textrm{
and }\sum_{n=1}^{\infty}|\theta_{n}-\theta_{n-1}|<\infty.$

Then, there exists
$(\bar{x},\bar{v}_{1},\dots,\bar{v}_{m})\in\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m}$
such that sequence $\\{(x_{n},v_{1,n},\dots,v_{m,n})\\}$ converges strongly to
$(\bar{x},\bar{v}_{1},\dots,\bar{v}_{m})$ as $n\to\infty$ and
$(\bar{x},\bar{v}_{1},\dots,\bar{v}_{m})$ satisfies Problem 4.3.

##  5 Tikhonov Regularized Douglas-Rachford-type Algorithms

In this section, using Algorithm 3.1 we propose a new Douglas-Rachford
algorithm to solve monotone inclusion problem of sum of two maximally monotone
operators. Further using proposed Douglas-Rachford algorithm, we propose a
Douglas-Rachford-type primal-dual algorithm to solve complexly structured
monotone inclusion problem containing linearly composite and parallel-sum
operators.

###  5.1 Tikhonov Regularized Douglas-Rachford Algorithm

Let $A,B:\mathcal{H}\to 2^{\mathcal{H}}$ be maximally monotone operators. In
this section, we consider the following monotone inclusion problem:

$\text{ find }x\in\mathcal{H}\text{ such that }0\in(A+B)x.$ (5.1)

We propose a new Douglas-Rachford algorithm based on Algorithm 3.1 such that
generated sequence converges strongly to a point in the solution set.

######  Theorem 5.1.

Consider $x_{1}\in\mathcal{H}$ and $\gamma>0,$ then algorithm is given by:

$\displaystyle{\forall
n\in\mathbb{N}\left\\{\begin{array}[]{lc@{}c@{}r}y_{n}=J_{\gamma
B}(e_{n}x_{n})\\\ z_{n}=J_{\gamma A}(2y_{n}-e_{n}x_{n})\\\
u_{n}=e_{n}x_{n}+\theta_{n}(z_{n}-y_{n})\\\ x_{n+1}=(2J_{\gamma
A}-Id)(2J_{\gamma B}-Id)u_{n}.\end{array}\right.}$ (5.6)

Let $\operatorname{Zer}(A+B)\neq\emptyset$ and sequences $e_{n}\in(0,1)$ and
$\theta_{n}\in(0,2]$ satisfy the condition (i) in Theorem 3.1 and the
conditions:

$0<\underline{\Theta}\leq\frac{\theta_{n}}{2}\leq\overline{\Theta}<1\textrm{
and }\sum_{n=1}^{\infty}|\theta_{n}-\theta_{n-1}|<\infty.$

Then the following statements are true:

  1. (i)

$\\{x_{n}\\}$ converges strongly to
$\bar{x}=\operatorname{proj}_{\operatorname{Fix}R_{\gamma A}R_{\gamma B}}(0)$.

  2. (ii)

$\\{y_{n}\\}$ and $\\{z_{n}\\}$ converges strongly to $J_{\gamma
B}(\bar{x})\in\operatorname{Zer}(A+B)$.

###### Proof.

Consider the operator $T\equiv R_{\gamma A}R_{\gamma B}:\mathcal{H}\to
2^{\mathcal{H}}$. From the definition of reflected resolvent, and definition
of operator $T$, algorithm (5.6) can be rewritten as

$\displaystyle x_{n+1}$ $\displaystyle=$ $\displaystyle R_{\gamma A}R_{\gamma
B}\\{e_{n}x_{n}+\frac{\theta_{n}}{2}(R_{\gamma A}R_{\gamma
B})(e_{n}x_{n})-e_{n}x_{n}\\}$ (5.7) $\displaystyle=$ $\displaystyle
T\\{e_{n}x_{n}+\frac{\theta_{n}}{2}(T(e_{n}x_{n})-e_{n}x_{n})\\}.$

Since resolvent operator is nonexpansive [6, Corollary 23.10(ii)], $T$ is
nonexpansive. Suppose $x^{*}\in\operatorname{Zer}(A+B)$ and results from [6,
Proposition 25.1(ii)], we have $\operatorname{Zer}(A+B)=J_{\gamma
B}(\operatorname{Fix}(T)),$ which collectively implies that
$\operatorname{Fix}{(T)}\neq\emptyset.$ Applying Theorem 3.1 with
$A_{1}=A_{2}=A,B_{1}=B_{2}=B$, we conclued that $\\{x_{n}\\}$ converges
strongly to $\bar{x}=\operatorname{proj}_{\operatorname{Fix}(T)}(0)$ as
$n\to\infty.$
The continuity of resolvent operator forces the sequence $\\{y_{n}\\}$ to
converge strongly to $J_{\gamma B}\bar{x}\in\operatorname{Zer}(A+B).$ Finally,
since $z_{n}-y_{n}=\frac{1}{2}(T(e_{n}x_{n})-e_{n}x_{n}),$ which converges
strongly to 0, concludes (ii).
∎

######  Problem 5.1.

Let $f,g:\mathcal{H}\to\mathbb{R}\cup\\{\infty\\}$ are convex proper and lower
semicontinuous functions. Consider the minimization problem

$\min_{x\in\mathcal{H}}f(x)+g(x).$ (5.8)

Using Karush-Kuhn-Tucker condition, (5.8) is equivalent to solve the inclusion
problem

$\text{find}{\ x\in\mathcal{H}}\ 0\in\partial f(x)+\partial g(x).$ (5.9)

In order to solve such type of problem, we propose an iterative scheme and
study its convergence behavior which can be summarized in following corollary.

######  Corollary 5.1.

Let $f,g$ be as in Problem 5.1 with
$argmin_{x\in\mathcal{H}}\\{f(x)+g(x)\\}\neq\emptyset$ and $0\in sqri(dom\
f-dom\ g).$ Consider the following iterative scheme with
$x_{1}\in\mathcal{H}$:

$\displaystyle{\forall
n\in\mathbb{N}\left\\{\begin{array}[]{lc@{}c@{}r}y_{n}=prox_{\gamma
g}(e_{n}x_{n})\\\ z_{n}=prox_{\gamma f}(2y_{n}-e_{n}x_{n})\\\
u_{n}=e_{n}x_{n}+\theta_{n}(z_{n}-y_{n})\\\ x_{n+1}=(2prox_{\gamma
f}-Id)(2prox_{\gamma g}-Id)u_{n},\par\end{array}\right.}$ (5.14)

where $\gamma>0$ and sequences $\\{\theta_{n}\\}\text{ and }\\{e_{n}\\}$
satisfy the condition (i) in Theorem 3.1 and the conditions:

$0<\underline{\Theta}\leq\frac{\theta_{n}}{2}\leq\overline{\Theta}<1\textrm{
and }\sum_{n=1}^{\infty}|\theta_{n}-\theta_{n-1}|<\infty.$

Then we have the following:

  1. (i)

$\\{x_{n}\\}$ converges strongly to
$\bar{x}=\operatorname{proj}_{\operatorname{Fix}(T)}(0)$ where
$T=(2prox_{\gamma f}-Id)(2prox_{\gamma g}-Id)$.

  2. (ii)

$\\{y_{n}\\}$ and $\\{z_{n}\\}$ converge strongly to $prox_{\gamma
g}(\bar{x})\in argmin_{x\in\mathcal{H}}\\{f(x)+g(x)\\}$.

###### Proof.

Since $argmin_{x\in\mathcal{H}}\\{f(x)+g(x)\\}\neq\emptyset$ and $0\in
sqri(dom\ f-dom\ g)$ ([6, Proposition 7.2]) ensures that
$\operatorname{Zer}(A+B)=argmin_{x\in\mathcal{H}}\\{f(x)+g(x)\\}.$ The results
can be obtained by choosing $A=\partial f,B=\partial g$ in Theorem 5.1. ∎

###  5.2 Tikhonov Regularized Douglas-Rachford-type Primal-Dual Algorithm

In this section, we propose Douglas-Rachford-type primal-dual algorithms to
solve the complex-structured monotone inclusion problem having mixtures of
composition of linear operators and parallel-sum operators. We consider the
monotone inclusion problem is as follows:

######  Problem 5.2.

Let $A:\mathcal{H}\to 2^{\mathcal{H}}$ be a maximally monotone operator.
Consider for each $i=1,\dots,m$, $\varOmega_{i}$ is a real Hilbert space,
$P_{i},Q_{i}:\varOmega_{i}\to 2^{\varOmega_{i}}$ are maximally monotone
operators and $L_{i}:\mathcal{H}\to\varOmega_{i}$ are nonzero linear
continuous operator. The primal inclusion problem is to find
$\bar{x}\in\mathcal{H}$ satisfying

$\displaystyle 0\in A\bar{x}+\sum_{i=1}^{m}L_{i}^{*}(P_{i}\square
Q_{i})(L_{i}\bar{x})$

together with dual inclusion problem

$\displaystyle{find\ \
\bar{v}_{1}\in\varOmega_{1},\dots,\bar{v}_{m}\in\varOmega_{m}\text{ such that
}(\exists
x\in\mathcal{H})\left\\{\begin{array}[]{lc@{}c@{}r}-\sum_{i=1}^{m}L_{i}^{*}\bar{v}_{i}\in
Ax\\\ \bar{v}_{i}\in(P_{i}\square Q_{i})(L_{i}x),\
i=1,\dots,m.\end{array}\right.}$ (5.17)

Here, operators $P_{i},Q_{i},i=1,\dots,m$ are not cocoercive, thus to solve
the Problem 5.2, we have to evaluate the resolvent of each operator, which
makes the Douglas-Rachford algorithm based primal-dual algorithm is more
appropriate to solve the problem.

######  Theorem 5.2.

In addition to assumption in Problem 5.2, we assume that

$0\in ran\left(A+\sum_{i=1}^{m}L_{i}^{*}\circ(P_{i}\square Q_{i})\circ
L_{i}\right).$ (5.18)

Consider the strictly positive integers $\tau,\sigma_{i},i=1,\dots,m$
satisfying

$\tau\sum_{i=1}^{m}\sigma_{i}\|L_{i}\|^{2}<4.$ (5.19)

Consider the initial point
$(x_{1},v_{1,1},\dots,v_{m,1})\in\mathcal{H}\times\varOmega_{i}\times\cdots\times\varOmega_{m}.$
The primal- dual algorithm to solve Problem 5.2 is given by

Input:

  1. 1.

initial points
$(x_{1},v_{1,1},\dots,v_{m,1})\in\mathcal{H}\times\varOmega_{i}\times\cdots\times\varOmega_{m}.$

  2. 2.

strictly positive real numbers $\tau,\sigma_{i}$, $i=1,2,...,m$ be such that
$\tau\sum_{i=1}^{m}\sigma_{i}\|L_{i}\|^{2}<4$.

  3. 3.

sequences $e_{n}\in(0,1),\theta_{n}\in(0,2]$

For $n\in\mathbb{N}$;

$p_{1,n}=J_{\tau
A}(e_{n}x_{n}-\frac{\tau}{2}e_{n}\sum_{i=1}^{m}L_{i}^{*}v_{i,n})$

$w_{1,n}=2p_{1,n}-e_{n}x_{n}$

For $i=1,\dots,m$;

$p_{2,i,n}=J_{\sigma_{i}P_{i}^{-1}}(e_{n}v_{i,n}+\frac{\sigma_{i}}{2}L_{i}w_{1,n})$

$w_{2,i,n}=2p_{2,i,n}-e_{n}v_{i,n}$

$z_{1,n}=w_{1,n}-\frac{\tau}{2}\sum_{i=1}^{m}L_{i}^{*}w_{2,i,n}$

$u_{1,n}=e_{n}x_{n}+\theta_{n}(z_{1,n}-p_{1,n})$

For $i=1,\dots,m$;

$z_{2,i,n}=J_{\sigma_{i}Q_{i}^{-1}}(w_{2,i,n}+\frac{\sigma_{i}}{2}L_{i}(2z_{1,n}-w_{1,n}))$

$u_{2,i,n}=e_{n}v_{i,n}+\theta_{n}(z_{2,i,n}-p_{2,i,n})$

$q_{1,n}=J_{\tau A}(u_{1,n}-\frac{\tau}{2}\sum_{i=1}^{m}L_{i}^{*}(u_{2,i,n}))$

$s_{1,n}=2q_{1,n}-u_{1,n}$

For $i=1,\dots,m$;

$q_{2,i,n}=J_{\sigma_{i}P_{i}^{-1}}(u_{2,i,n}+\frac{\sigma_{i}}{2}L_{i}s_{1,n})$

$s_{2,i,n}=2q_{2,i,n}-u_{2,i,n}$

$d_{1,n}=s_{1,n}-\frac{\tau}{2}\sum_{i=1}^{m}L_{i}^{*}(s_{2,i,n})$

$x_{n+1}=2d_{1,n}-s_{1,n}$

For $i=1,\dots,m$;

$d_{2,i,n}=J_{\sigma_{i}Q_{i}^{-1}}(s_{2,i,n}+\frac{\sigma_{i}}{2}L_{i}(x_{n+1}))$

$v_{2,i,n}=2d_{2,i,n}-s_{2,i,n}$

Output: $(x_{n+1},v_{1,n+1},\dots,v_{m,n+1})$

Algorithm 1 To optimize the complexly structured monotone inclusion Problem
5.2

where sequences $\\{\theta_{n}\\}$ and $\\{e_{n}\\}$ satisfy the condition (i)
in Theorem 3.1 and the conditions:

$0<\underline{\Theta}\leq\frac{\theta_{n}}{2}\leq\overline{\Theta}<1\textrm{
and }\sum_{n=1}^{\infty}|\theta_{n}-\theta_{n-1}|<\infty.$

Then there exist an element
$(\bar{x},\bar{v}_{1},\dots,\bar{v}_{m})\in\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m}$
such that following statements are true:

  1. (a)

Denote
$\bar{p}_{1}=J_{\tau
A}\left(\bar{x}-\frac{\tau}{2}\sum_{i=1}^{m}L_{i}^{*}\bar{v}_{i}\right)$
$\bar{p}_{2,i}=J_{\sigma_{i}P_{i}^{-1}}\left(\bar{v}_{i}+\frac{\sigma_{i}}{2}L_{i}(2\bar{p}_{1}-\bar{x})\right),i=1,\dots,m.$
Then the point
$(\bar{p}_{1},\bar{p}_{2,1},\dots,\bar{p}_{2,m})\in\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m}$
is a primal-dual solution of Problem 5.2.

  2. (b)

$\\{(x_{n},v_{1,n},\dots,v_{m,n})\\}$ converges strongly to
$(\bar{x},\bar{v}_{1},\dots,\bar{v}_{m})$.

  3. (c)

$\\{(p_{1,n},p_{2,1,n},\dots,p_{2,m,n})\\}$ and
$\\{(z_{1,n},z_{2,1,n},\dots,z_{2,m,n})\\}$ converge strongly to
$(\bar{p}_{1},\bar{p}_{2,1},\dots,\bar{p}_{2,m})$.

###### Proof.

Consider the real Hilbert space $\mathcal{K}$ and operators

  1. (i)

$\phi:\mathcal{K}\to 2^{\mathcal{K}}$, defined by
$(x,u_{1},\dots,u_{m})\mapsto(Ax,P_{1}^{-1}u_{1},\dots,P_{m}^{-1}u_{m}),$

  2. (ii)

$\xi:\mathcal{K}\to\mathcal{K}$, defined by
$(x,u_{1},\dots,u_{m})\mapsto\left(\sum_{i=1}^{m}L_{i}^{*}u_{i},-L_{1}x,\dots,-L_{m}x\right),$

  3. (iii)

$\psi:\mathcal{K}\to 2^{\mathcal{K}}$, defined by
$\psi(x,u_{1},\dots,u_{m})=\left(0,Q_{1}^{-1}u_{1},\dots,Q_{m}^{-1}u_{m}\right).$

We can observe the following:

  1. (i)

operator $\frac{1}{2}\xi+\psi\text{ and }\frac{1}{2}\xi+\phi$ are maximally
monotone as dom $\xi=\mathcal{K}$ ([6, Corollary 24.4(i)]),

  2. (ii)

condition (5.18) implies $\operatorname{Zer}(\phi+\xi+\psi)\neq\emptyset$,

  3. (iii)

every point in $\operatorname{Zer}(\phi+\xi+\psi)$ solves Problem 5.2.

Define the linear continuous operator $\textbf{W}:\mathcal{K}\to\mathcal{K}$,
defined by

$\textbf{W}(x,u_{1},\dots,u_{m})=\left(\frac{x}{\tau}-\frac{1}{2}\sum_{i=1}^{m}L_{i}^{*}u_{i},\frac{u_{1}}{\sigma_{1}}-\frac{1}{2}L_{1}x,\dots,\frac{u_{m}}{\sigma_{m}}-\frac{1}{2}L_{m}x\right)$

which is self-adjoint. Consider

$\rho=\left(1-\frac{1}{2}\sqrt{\tau\sum_{i=1}^{m}\sigma_{i}\|L_{i}\|^{2}}\right)\min\left\\{\frac{1}{\tau},\frac{1}{\sigma_{1}},\dots,\frac{1}{\sigma_{m}}\right\\}>0.$

The operator W is $\rho$\- strongly positive in $\mathcal{K}_{\textbf{W}}$
([54]) and satisfies the following inequality

$\langle x,\textbf{W}x\rangle_{\mathcal{K}}\geq\rho\|x\|_{\mathcal{K}}^{2}\ \
\forall x\in\mathcal{K}.$

Thus the inverse of W exists and satisfies
$\|\textbf{W}^{-1}\|\leq\frac{1}{\rho}.$ Consider the sequences

$\displaystyle{\forall
n\in\mathbb{N}\left\\{\begin{array}[]{lc@{}c@{}r}\text{{x}}_{n}=(x_{n},v_{1,n},\dots,v_{m,n})\\\
\text{{y}}_{n}=(p_{1,n},p_{2,1,n},\dots,p_{2,m,n})\\\
\text{{z}}_{n}=(z_{1,n},z_{2,1,n},\dots,z_{2,m,n})\\\
\text{{u}}_{n}=(u_{1,n},u_{2,1,n},\dots,u_{2,m,n})\\\
\text{{q}}_{n}=(q_{1,n},q_{2,1,n},\dots,q_{2,m,n})\\\
\text{{s}}_{n}=(s_{1,n},s_{2,1,n},\dots,s_{2,m,n})\\\
\text{{d}}_{n}=(d_{1,n},d_{2,1,n},\dots,d_{2,m,n}).\end{array}\right.}$ (5.27)

Using the definition of operators $\phi,\xi,\psi$ and W, Algorithm 1 can be
written equivalently as

$\displaystyle{\forall
n\in\mathbb{N}\left\\{\begin{array}[]{lc@{}c@{}r}\textbf{W}(e_{n}\textbf{x}_{n}-\textbf{y}_{n})\in(\frac{1}{2}\xi+\phi)\textbf{y}_{n}\\\
\textbf{W}(2\textbf{y}_{n}-e_{n}\textbf{x}_{n}-\textbf{z}_{n})\in(\frac{1}{2}\xi+\psi)\textbf{z}_{n}\\\
\textbf{u}_{n}=e_{n}\textbf{x}_{n}+\theta_{n}(\textbf{z}_{n}-\textbf{y}_{n})\\\
\textbf{W}(\textbf{u}_{n}-\textbf{q}_{n})\in(\frac{1}{2}\xi+\phi)\textbf{q}_{n}\\\
\textbf{W}(\textbf{s}_{n}-\textbf{d}_{n})\in(\frac{1}{2}\xi+\psi)(\textbf{d}_{n})\\\
\textbf{x}_{n+1}=2\textbf{d}_{n}-\textbf{s}_{n},\end{array}\right.}$ (5.34)

which is further equivalent to

$\displaystyle{\forall
n\in\mathbb{N}\left\\{\begin{array}[]{lc@{}c@{}r}\textbf{y}_{n}=(Id+\textbf{W}^{-1}(\frac{1}{2}\xi+\phi))^{-1}(e_{n}\textbf{x}_{n})\\\
{\textbf{z}_{n}=(Id+\textbf{W}^{-1}(\frac{1}{2}\xi+\psi))^{-1}(2\textbf{y}_{n}-e_{n}\textbf{x}_{n})}\\\
{\textbf{u}_{n}=e_{n}\textbf{x}_{n}+\theta_{n}(\textbf{z}_{n}-\textbf{y}_{n})}\\\
\textbf{q}_{n}=(Id+\textbf{W}^{-1}(\frac{1}{2}\xi+\phi))^{-1}(\textbf{u}_{n})\\\
\textbf{s}_{n}=2\textbf{q}_{n}-\textbf{u}_{n}\\\
\textbf{d}_{n}=(Id+\textbf{W}^{-1}(\frac{1}{2}\xi+\psi))^{-1}(\textbf{s}_{n})\\\
\textbf{x}_{n+1}=2\textbf{d}_{n}-\textbf{s}_{n}.\end{array}\right.}$ (5.42)

Now, consider the real Hilbert space
$\mathcal{K}_{\text{{W}}}=\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m}$
with inner product and norm defined as
$\langle\textbf{x},\textbf{y}\rangle_{\mathcal{K}_{\textbf{W}}}=\langle\textbf{x},\textbf{W}\textbf{y}\rangle$
and
$\|\textbf{x}\|_{\mathcal{K}_{\textbf{W}}}=\sqrt{\langle\textbf{x},\textbf{W}\textbf{x}\rangle_{\mathcal{K}}}$
respectively.
Now, define the operators
$\textbf{A}\equiv\textbf{W}^{-1}(\frac{1}{2}\xi+\psi)$ and
$\textbf{B}\equiv\textbf{W}^{-1}(\frac{1}{2}\xi+\phi),$ which are maximally
monotone on $\mathcal{K}_{\textbf{W}}$ as $\frac{1}{2}\xi+\phi$ and
$\frac{1}{2}\xi+\psi$ are maximally monotone on $\mathcal{K}.$ The Algorithm 1
can be written in the form of Douglas-Rachford algorithm as

$\displaystyle{\forall
n\in\mathbb{N}\left\\{\begin{array}[]{lc@{}c@{}r}\textbf{y}_{n}=\textbf{J}_{\textbf{B}}(e_{n}\textbf{x}_{n})\\\
\textbf{u}_{n}=e_{n}\textbf{x}_{n}+\theta_{n}\left(\textbf{J}_{\textbf{A}}(2\textbf{y}_{n}-e_{n}\textbf{x}_{n})-\textbf{y}_{n}\right)\\\
\textbf{x}_{n+1}=(2\textbf{J}_{\textbf{A}}-Id)(2\textbf{J}_{\textbf{B}}-Id)\textbf{z}_{n},\end{array}\right.}$
(5.46)

which is of the form Algorithm (5.6) for $\gamma=1.$ From assumption (5.18),
we have

$\operatorname{Zer}(\textbf{A}+\textbf{B})=\operatorname{Zer}(\textbf{W}^{-1}(\textbf{M}+\textbf{S}+\textbf{Q}))=\operatorname{Zer}(\textbf{M}+\textbf{S}+\textbf{Q}).$

Applying Theorem 5.1, we can find
$\bar{\textbf{x}}\in\operatorname{Fix}(R_{\textbf{A}}R_{\textbf{B}})$ such
that
$\textbf{J}_{\text{{B}}}{\bar{\textbf{x}}}\in\operatorname{Zer}(\textbf{A}+\textbf{B}).$
∎

At the end of this section, we study iterative technique to solve following
convex optimization problem:

######  Problem 5.3.

Let $f\in\Gamma(\mathcal{H})$. Consider $\varOmega_{i}$ are real Hilbert
spaces, $g_{i},l_{i}\in\Gamma(\varOmega_{i})$ and
$L_{i}:\mathcal{H}\to\varOmega_{i}$ are linear continuous operators,
$i=1,\dots,m$. The optimization problem is given by

$\inf_{x\in\mathcal{H}}\left[f(x)+\sum_{i=1}^{m}(g_{i}\square
l_{i})(L_{i}x)\right]$ (5.47)

with conjugate-dual problem is given by

$\sup_{v_{i}\in\varOmega,i=1,2,\dots,m}\left\\{-f^{*}\left(-\sum_{i=1}^{m}L_{i}^{*}v_{i}\right)-\sum_{i=1}^{m}(g_{i}^{*}(v_{i})+l_{i}^{*}(v_{i}))\right\\}.$
(5.48)

Consider stricly positive integers $\tau,\sigma_{i},i=1,\dots,m$ and initial
point
$(x_{1},v_{1,1},\dots,v_{m,1})\in\mathcal{H}\times\varOmega_{i}\times\cdots\times\varOmega_{m}.$
The primal-dual algorithm to solve Problem 5.3 is given by

Input:

  1. 1.

initial points
$(x_{1},v_{1,1},\dots,v_{m,1})\in\mathcal{H}\times\varOmega_{i}\times\cdots\times\varOmega_{m}.$

  2. 2.

Positive real numbers $\tau,\sigma_{i}$, $i=1,2,...,m$ be such that
$\tau\sum_{i=1}^{m}\sigma_{i}\|L_{i}\|^{2}<4$.

  3. 3.

The sequences $\\{\theta_{n}\\},\\{e_{n}\\}$ satisfying the assumptions in
Theorem 5.2.

For $n\in\mathbb{N}$;

$p_{1,n}=prox_{\tau
f}(e_{n}x_{n}-\frac{\tau}{2}e_{n}\sum_{i=1}^{m}L_{i}^{*}v_{i,n})$

$w_{1,n}=2p_{1,n}-e_{n}x_{n}$

For $i=1,\dots,m$;

$p_{2,i,n}=prox_{\sigma_{i}g_{i}^{*}}(e_{n}v_{i,n}+\frac{\sigma_{i}}{2}L_{i}w_{1,n})$

$w_{2,i,n}=2p_{2,i,n}-e_{n}v_{i,n}$

$z_{1,n}=w_{1,n}-\frac{\tau}{2}\sum_{i=1}^{m}L_{i}^{*}w_{2,i,n}$

$u_{1,n}=e_{n}x_{n}+\theta_{n}(z_{1,n}-p_{1,n})$

For $i=1,\dots,m$;

$z_{2,i,n}=prox_{\sigma_{i}l_{i}^{*}}(w_{2,i,n}+\frac{\sigma_{i}}{2}L_{i}(2z_{1,n}-w_{1,n}))$

$u_{2,i,n}=e_{n}v_{i,n}+\theta_{n}(z_{2,i,n}-p_{2,i,n})$

$q_{1,n}=prox_{\tau
f}(u_{1,n}-\frac{\tau}{2}\sum_{i=1}^{m}L_{i}^{*}(u_{2,i,n}))$

$s_{1,n}=2q_{1,n}-u_{1,n}$

For $i=1,\dots,m$;

$q_{2,i,n}=prox_{\sigma_{i}g_{i}^{*}}(u_{2,i,n}+\frac{\sigma_{i}}{2}L_{i}s_{1,n})$

$s_{2,i,n}=2q_{2,i,n}-u_{2,i,n}$

$t_{1,n}=s_{1,n}-\frac{\tau}{2}\sum_{i=1}^{m}L_{i}^{*}(s_{2,i,n})$

$x_{n+1}=2t_{1,n}-s_{1,n}$

For $i=1,\dots,m$;

$t_{2,i,n}=prox_{\sigma_{i}l_{i}^{*}}(s_{2,i,n}+\frac{\sigma_{i}}{2}L_{i}(x_{n+1}))$

$v_{2,i,n}=2t_{2,i,n}-s_{2,i,n}$

Output: $(x_{n+1},v_{1,n+1},\dots,v_{m,n+1})$

Algorithm 2 To optimize the complexly structured monotone inclusion Problem
5.3

where $\\{\theta_{n}\\}\text{ and }\\{e_{n}\\}$ are real sequences.

######  Corollary 5.2.

In addition to assumptions in Problem 5.3, consider

$0\in ran\left(\partial f+\sum_{i=1}^{m}L_{i}^{*}\circ(\partial
g_{i}\square\partial l_{i})\circ L_{i}\right).$ (5.49)

Then, there exists an element
$(\bar{x},\bar{v}_{1},\dots,\bar{v}_{m})\in\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m}$
such that following statements are true:

  1. (a)

Denote
$\bar{p}_{1}=prox_{\tau
f}\left(\bar{x}-\frac{\tau}{2}\sum_{i=1}^{m}L_{i}^{*}\bar{v}_{i}\right)$
$\bar{p}_{2,i}=prox_{\sigma_{i}g^{*}_{i}}\left(\bar{v}_{i}+\frac{\sigma_{i}}{2}L_{i}(2\bar{p}_{i}-\bar{x})\right),i=1,\dots,m.$
Then the point
$(\bar{p}_{1},\bar{p}_{2,1},\dots,\bar{p}_{2,m})\in\mathcal{H}\times\varOmega_{1}\times\cdots\times\varOmega_{m}$
is a primal-dual solution of Problem 5.3.

  2. (b)

$\\{(x_{n},v_{1,n},\dots,v_{m,n})\\}$ converges strongly to
$(\bar{x},\bar{v}_{1},\dots,\bar{v}_{m})$.

  3. (c)

$\\{(p_{1,n},p_{2,1,n},\dots,p_{2,m,n})\\}$ and
$\\{(z_{1,n},z_{2,1,n},\dots,z_{2,m,n})\\}$ converge strongly to
$(\bar{p}_{1},\bar{p}_{2,1},\dots,\bar{p}_{2,m})$.

##  6 Numerical Experiment

In this section, we make an experimental setup to solve the wavelet based
image deblurring problem. In image deblurring, we develop mathematical methods
to recover the original, sharp image from blurred image. The mathematical
formulation of the blurring process can be written as linear inverse problem,

$\text{ find }x\in\mathbb{R}^{d}\text{ such that }Ax=b+w$ (6.1)

where $A\in\mathbb{R}^{m\times d}$ is a blurring operator,
$b\in\mathbb{R}^{m}$ is blurred image and $w$ is an unknown noise. A classical
approach to solve Problem (6.1) is to minimize the least-square term
$\|Ax-b\|^{2}$. In the deblurring case, the problem is ill-conditioned as the
norm solution has a huge norm. To remove the difficulty, the ill-conditioned
problem is replaced by a nearly well-conditioned problem. In wavelet domain,
most images are sparse in nature, that’s why we choose $l_{1}$ regularization.
For $l_{1}$ regularization, the image processing problem becomes

$\min_{x\in\mathbb{R}^{2}}F(x)=\|Ax-b\|^{2}+\lambda\|x\|_{1},$ (6.2)

where $\lambda$ is a sparsity controlling parameter and provides a tradeoff
between fidelity to the measurements and noise sensitivity. The $l_{1}$
regularization produces sparse images having sharp edges since it is less
sensitive to outliers. Using subdifferential characterization of the minimum
of a convex function, a point $x^{*}$ minimizes $F(x)$ if and only if

$0\in A^{T}(Ax^{*}-b)+\partial\lambda\|x^{*}\|_{1}.$

Thus we can apply forward-backward Algorithm (4.2) with
$A_{1}=A_{2}=A^{T}(Ax^{*}-b)$ and $B_{1}=B_{2}=\partial\lambda\|x^{*}\|_{1}$
to solve the deblurring problem (6.2).

For numerical experiment purposes, we have chosen images from publicly
available domain and assumed reflexive (Neumann) boundary conditions. We
blurred the images using gaussian blur of size $9\times 9$ and standard
deviation 4. We have compared the algorithm (4.2) with [9, Algorithm 8]. The
operator $A=RW$, where $W$ is the three stage haar wavelet transform and $R$
is the blur operator. The original and corresponding blurred images were shown
in Figure 1.

(a) Original.

(b) Blurred

(c) Original

(d) Blurred

Figure 1: The original and blurred images of Lenna and crowd. Peak signal to
noise ratio (PSNR) of image (b) is 28.3111 and of image (d) is 21.3171

The regularization parameter was chosen to be $\lambda=2\times 10^{-5}$, and
the initial image was the blurred image. The objective function value is
denoted by $F(x^{*})$ and function value at $n^{\text{th}}$ iteration is
denoted by $F(x_{n})$. Sequences $\\{e_{n}\\}$ and $\\{\theta_{n}\\}$ are
chosen as $\\{1-\frac{1}{n+1}\\}$ and $\\{0.9\\}$ respectively. The images
recovered by the algorithms for 1000 iterations are shown in the figure. The
graphical representation of convergence of $F(x_{n})-F(x^{*})$ is depicted in
Figure 2.

(a) Lenna.

(b) Crowd

Figure 2: The variation of $F(x_{n})-F(x^{*})$ with respect to number of
iteration for different images.

For deblurring methods, lower the value of $F(x_{n})-F(x^{*})$ higher the
quality of recovered images.

(a) Algorithm (4.2). PSNR = 34.8448.

(b) [9, Algorithm 8]. PSNR = 34.5123

(c) Algorithm (4.2). PSNR = 29.3581

(d) [9, Algorithm 8]. PSNR = 28.5764

Figure 3: The recovered images using different algorithms for 1000 iterations.

(a) Lenna.

(b) Crowd

Figure 4: The variation of PSNR value of recovered image at each iteration
with original image as a reference image.

It can be observed from Figures 2 and 3 that proposed Algorithm (4.2)
outperforms over [9, Algorithm 8]. This can also be confirmed from the peak
signal to noise ratio (PSNR) value of the recovered images and the fact that
higher the PSNR value, better the image quality. The variation of PSNR value
of recovered image at each iteration with original image as a reference is
also plotted in Figure 4.

## Acknowledgements

The first author acknowledges the Indian Institute of Technology (BHU),
Varanasi for Senior Research Fellowship in terms of teaching assistantship.
The third author is thankful to University Grant Commission for the Senior
Research Fellowship.

## References

  * [1] Agarwal, R. P., O Regan, D., and Sahu, D. R. (2007). Iterative construction of fixed points of nearly asymptotically nonexpansive mappings. Journal of Nonlinear and convex Analysis, 8(1), 61.
  * [2] Artacho, F. J. A., and Borwein, J. M. (2013). Global convergence of a non-convex Douglas–Rachford iteration. Journal of Global Optimization, 57(3), 753-769.
  * [3] Attouch, H., and Théra, M. (1996). A general duality principle for the sum of two operators. Journal of Convex Analysis, 3, 1-24.
  * [4] Barty, K., Roy, J. S., and Strugarek, C. (2007). Hilbert-valued perturbed subgradient algorithms. Mathematics of Operations Research, 32(3), 551-562.
  * [5] Bauschke, H. H., and Combettes, P. L. (2001). A weak-to-strong convergence principle for Fejér-monotone methods in Hilbert spaces. Mathematics of operations research, 26(2), 248-264..
  * [6] Bauschke, H. H., and Combettes, P. L. (2011). Convex analysis and monotone operator theory in Hilbert spaces (Vol. 408). New York: Springer.
  * [7] Bennar, A., and Monnez, J. M. (2007). Almost sure convergence of a stochastic approximation process in a convex set. International Journal of Apllied Mathematics, 20(5), 713-722.
  * [8] Boţ, R. I., Csetnek, E. R., and Heinrich, A. (2013). A primal-dual splitting algorithm for finding zeros of sums of maximal monotone operators. SIAM Journal on Optimization, 23(4), 2011-2036.
  * [9] Boţ, R. I., Csetnek, E. R., and Meier, D. (2019). Inducing strong convergence into the asymptotic behaviour of proximal splitting algorithms in Hilbert spaces. Optimization Methods and Software, 34(3), 489-514.
  * [10] Boţ, R. I., and Hendrich, C. (2014). Convergence analysis for a primal-dual monotone+ skew splitting algorithm with applications to total variation minimization. Journal of mathematical imaging and vision, 49(3), 551-568.
  * [11] Boţ, R. I., and Nguyen, D. K. (2021). Factorization of completely positive matrices using iterative projected gradient steps. Numerical Linear Algebra with Applications, e2391.
  * [12] Briceno-Arias, L. M., and Combettes, P. L. (2011). A monotone+ skew splitting model for composite monotone inclusions in duality. SIAM Journal on Optimization, 21(4), 1230-1250.
  * [13] Briceno-Arias, L. M., and Combettes, P. L. (2013). Monotone operator methods for Nash equilibria in non-potential games. In Computational and analytical mathematics (pp. 143-159). Springer, New York, NY.
  * [14] Butnariu, D., and Kassay, G. (2008). A proximal-projection method for finding zeros of set-valued operators. SIAM Journal on Control and Optimization, 47(4), 2096-2136.
  * [15] Chang, S. S., Wang, L., Lee, H. W. J., and Chan, C. K. (2013). Strong and $\bigtriangleup$-convergence for mixed type total asymptotically nonexpansive mappings in CAT (0) spaces. Fixed Point Theory and Applications, 1, 1-16.
  * [16] Chang, S. S., Wang, G., Wang, L., Tang, Y. K., and Ma, Z. L. (2014). $\bigtriangleup$-convergence theorems for multi-valued nonexpansive mappings in hyperbolic spaces. Applied Mathematics and Computation, 249, 535-540.
  * [17] Chen, G. H. G. (1994). Forward-backward splitting techniques: theory and applications (Doctoral dissertation, University of Washington).
  * [18] Chen, G. H., and Rockafellar, R. T. (1997). Convergence rates in forward–backward splitting. SIAM Journal on Optimization, 7(2), 421-444.
  * [19] Cholamjiak, P., Abdou, A. A., and Cho, Y. J. (2015). Proximal point algorithms involving fixed points of nonexpansive mappings in $\operatorname{CAT}(0)$ spaces. Fixed Point Theory and Applications, 1, 1-13.
  * [20] Combettes, P. L., and Pesquet, J. C. (2012). Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set-Valued and variational analysis, 20(2), 307-330.
  * [21] Combettes, P. L., and Wajs, V. R. (2005). Signal recovery by proximal forward-backward splitting. Multiscale Modeling and Simulation, 4(4), 1168-1200.
  * [22] Daubechies, I., Defrise, M., and De Mol, C. (2004). An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences, 57(11), 1413-1457.
  * [23] Davis, D. (2015). Convergence rate analysis of the forward-Douglas-Rachford splitting scheme. SIAM Journal on Optimization, 25(3), 1760-1786.
  * [24] Dixit, A., Sahu, D. R., Singh, A. K., and Som, T. (2020). Application of a new accelerated algorithm to regression problems. Soft Computing, 24(2), 1539-1552.
  * [25] Douglas, J., and Rachford, H. H. (1956). On the numerical solution of heat conduction problems in two and three space variables. Transactions of the American mathematical Society, 82(2), 421-439.
  * [26] Fortin, M., and Glowinski, R. (2000). Augmented Lagrangian methods: applications to the numerical solution of boundary-value problems. Elsevier.
  * [27] Gautam, P., Dixit, A., Sahu, D. R., and Som, T. (2020). Application of new strongly convergent iterative methods to split equality problems. Computational and Applied Mathematics, 39, 1-28.
  * [28] Gautam, P., Sahu, D. R., Dixit, A., and Som, T. (2021). Forward–Backward–Half Forward Dynamical Systems for Monotone Inclusion Problems with Application to v-GNE. Journal of Optimization Theory and Applications, 190(2), 491-523.
  * [29] Güler, O. (1991). On the convergence of the proximal point algorithm for convex minimization. SIAM Journal on Control and Optimization, 29(2), 403-419.
  * [30] Gur, E., Sabach, S., and Shtern, S. (2020). Alternating minimization based first-order method for the wireless sensor network localization problem. IEEE Transactions on Signal Processing, 68, 6418-6431.
  * [31] Haugazeau, Y. (1968). Sur les inéquations variationnelles et la minimisation de fonctionnelles convexes. These, Universite de Paris.
  * [32] Iemoto, S., and Takahashi, W. (2009). Approximating common fixed points of nonexpansive mappings and nonspreading mappings in a Hilbert space. Nonlinear analysis: theory, methods and applications, 71(12), e2082-e2089.
  * [33] Lehdili, N., and Moudafi, A. (1996). Combining the proximal algorithm and Tikhonov regularization. Optimization, 37(3), 239-252.
  * [34] Lions, P. L., and Mercier, B. (1979). Splitting algorithms for the sum of two nonlinear operators. SIAM Journal on Numerical Analysis, 16(6), 964-979.
  * [35] Luke, D. R., and Martins, A. L. (2020). Convergence Analysis of the Relaxed Douglas–Rachford Algorithm. SIAM Journal on Optimization, 30(1), 542-584.
  * [36] Maingé, P. E. (2007). Approximation methods for common fixed points of nonexpansive mappings in Hilbert spaces. Journal of Mathematical Analysis and Applications, 325(1), 469-479.
  * [37] Mann, W. R. (1953). Mean value methods in iteration. Proceedings of the American Mathematical Society, 4(3), 506-510.
  * [38] Martinet, B. (1972). Détermination approchée d’un point fixe d’une application pseudo-contractante. CR Acad. Sci. Paris, 274(2), 163-165.
  * [39] Mercier, B. (1980). Inéquations variationnelles de la mécanique. Université de Paris-Sud, Département de mathématique.
  * [40] Mouallif, K., Nguyen, V. H., and Strodiot, J. J. (1991). A perturbed parallel decomposition method for a class of nonsmooth convex minimization problems. SIAM journal on control and optimization, 29(4), 829-847.
  * [41] Moudafi, A., and Théra, M. (1997). Finding a zero of the sum of two maximal monotone operators. Journal of Optimization Theory and Applications, 94(2), 425-448.
  * [42] Nemirovski, A., Juditsky, A., Lan, G., and Shapiro, A. (2009). Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization, 19(4), 1574-1609.
  * [43] Passty, G. B. (1979). Ergodic convergence to a zero of the sum of monotone operators in Hilbert space. Journal of Mathematical Analysis and Applications, 72(2), 383-390.
  * [44] Phan, H. M. (2016). Linear convergence of the Douglas–Rachford method for two closed sets. Optimization, 65(2), 369-385.
  * [45] Rockafellar, R. T. (1976). Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Mathematics of operations research, 1(2), 97-116.
  * [46] Rockafellar, R. T. (1976). Monotone operators and the proximal point algorithm. SIAM journal on control and optimization, 14(5), 877-898.
  * [47] Sahu, D. R. (2011). Applications of the S-iteration process to constrained minimization problems and split feasibility problems. Fixed Point Theory, 12(1), 187-204.
  * [48] Sahu, D. R., Pitea, A., and Verma, M. (2020). A new iteration technique for nonlinear operators as concerns convex programming and feasibility problems. Numerical Algorithms, 83(2), 421-449.
  * [49] Sahu, D. R., Kumar, A., and Kang, S. M. (2021). Proximal point algorithms based on S-iterative technique for nearly asymptotically quasi-nonexpansive mappings and applications. Numerical Algorithms, 86(4), 1561-1590.
  * [50] Svaiter, B. F. (2011). On weak convergence of the Douglas–Rachford method. SIAM Journal on Control and Optimization, 49(1), 280-287.
  * [51] Tikhonov, A. N. (1965). Improper problems of optimal planning and stable methods of their solution. In Soviet Math. Doklady (Vol. 6, pp. 1264-1267).
  * [52] Tikhonov, A.N., Arsenin, V.J., (1977). Methods for Solving Ill-Posed Problems. Wiley, New York.
  * [53] Tihonov, A. N. (1963). Solution of incorrectly formulated problems and the regularization method. Soviet Math., 4, 1035-1038.
  * [54] Vũ, B. C. (2013). A splitting algorithm for dual monotone inclusions involving cocoercive operators. Advances in Computational Mathematics, 38(3), 667-681.
  * [55] Xu, H. K. (2002). Iterative algorithms for nonlinear operators. Journal of the London Mathematical Society, 66(1), 240-256.