# On Componental Operators
in Hilbert Space

Andrzej Cegielski1 and Yair Censor${}^{2}\vskip 12.0pt plus 4.0pt minus 4.0pt$
1Institute of Mathematics
University of Zielona Góra
Zielona Góra, Poland
<EMAIL_ADDRESS>

2Department of Mathematics
University of Haifa
Mt. Carmel, Haifa 3498838, Israel
<EMAIL_ADDRESS>
(July 15th, 2021.
Revised: November 2nd, 2021 and November 20, 2021.)

###### Abstract

We consider a Hilbert space that is a product of a finite number of Hilbert
spaces and operators that are represented by “componental operators” acting on
the Hilbert spaces that form the product space. We attribute operatorial
properties to the componental operators rather than to the full operators. The
operatorial properties that we discuss include nonexpansivity, firm
nonexpansivity, relaxed firm nonexpansivity, averagedness, being a cutter,
quasi-nonexpansivity, strong quasi-nonexpansivity, strict quasi-nonexpansivity
and contraction.

Some relationships between operators whose componental operators have such
properties and operators that have these properties on the product space are
studied. This enables also to define componental fixed point sets and to study
their properties. For componental contractions we offer a variant of the
Banach fixed point theorem.

Our motivation comes from the desire to extend a fully-simultaneous method
that takes into account sparsity of the linear system in order to accelerate
convergence [Censor et al., On diagonally relaxed orthogonal projection
methods, SIAM J. Sci. Comput. 30 (2008), 473–504]. This was originally
applicable to the linear case only and gives rise to an iterative process that
uses different componental operators during iterations.

##  1 Introduction

Let $\mathcal{H}_{j}$, $j\in J:=\\{1,2,...,n\\},$ be real Hilbert spaces and
$\mathcal{H}:\mathcal{=H}_{1}\times\mathcal{H}_{2}\times\cdots\times\mathcal{H}_{n}$
for $n\geqslant 2$. Consider an operator $T:\mathcal{H}\rightarrow\mathcal{H}$
that maps the Hilbert space $\mathcal{H}$ into itself and is represented by

$T(x)=((T(x))_{1},(T(x))_{2},...,(T(x))_{n})\text{,}$ (1)

where, for all $j\in J,$ $(T(\cdot))_{j}:\mathcal{H}\rightarrow$
$\mathcal{H}_{j}$ denotes “the $j$-th componental operator of $T(\cdot)$”.

In this paper we attribute operatorial properties to the componental operators
of an operator $T$. To explain what we mean, let us take, by way of example,
the well-known nonexpansivness property. We define that the operator $T$ is
“$j$-nonexpansive ($j$-NE)” if for all $x,y\in\mathcal{H}$

$\|(T(x))_{j}-(T(y))_{j}\|_{j}\leq\|x_{j}-y_{j}\|_{j}\text{,}$ (2)

for some $j\in J,$ where $\|\cdot\|_{j}$ is the norm in $\mathcal{H}_{j}.$ If
$T$ is $j$-NE for all $j\in J$ then we say that $T$ is “component-wise NE (CW-
NE)”.

Obviously, such definitions imply that if $T$ is CW-NE then it is NE but not
vice versa, making the set of CW-NE operators a proper subset of the NE
operators. In this sense, a CW-NE operator is “stronger” than an NE operator
which is not necessarily CW-NE.

We look at various operatorial properties of the componental operators of an
operator $T$ and investigate their relationships with such properties of the
operator $T$ itself. To do so, we define the notions of $j$-nonexpansive
($j$-NE), $j$-firmly nonexpansive ($j$-FNE), $j$-relaxed firmly nonexpansive
($j$-RFNE), $j$-averaged ($j$-AV), $j$-cutter, $j$-quasi-nonexpansive
($j$-QNE), $(\rho_{j},j)$-strongly quasi-nonexpansive ($(\rho_{j},j)$-SQNE),
$j$-strictly quasi-nonexpansive ($j$-sQNE) and $(\alpha_{j},j)$-contraction
($(\alpha_{j},j)$-CONT) and their associated “CW-X” properties, where “X”
stands for any of the, above mentioned, properties: NE, FNE, RFNE, AV, cutter,
QNE, SQNE, sQNE or CONT. To say that $T$ is CW-X means that it is “$j$-X” for
all $j\in J.$

For $j\in J$ we also define the “$j$-th fixed point set of $T$” by

$\operatorname*{Fix}{}^{j}T:=\\{z\in\mathcal{H}\mid(T(z))_{j}=z_{j}\\}$ (3)

and study its properties.

Our motivation comes from a desire to study an iterative process that uses, as
the iterations proceed, different componental operators of the operator $T.$
We devote below a special section to describe this motivating topic.

Working with “blocks” of variables is a common and wide-ranging subject in the
study of iterative processes. An early approach to analyzing iterative
processes over Cartesian product sets and “component solution methods” can be
found in the book of Bertsekas and Tsitsiklis [6, Subsection 3.1.2]. A product
space formulation, which became classical by now, for feasibility-seeking and
optimization problems, is the work of Pierra [14]. A kind of stochastic
component solution methods were studied in [12].

Here we set forth a general framework that fits a large variety of operatorial
properties. It remains to be discovered whether the subsets of operators of
the form “CW-X”, for any of the above mentioned properties “X”, can generate
stronger results in fixed point theory due to the fact that a CW-X operator is
“stronger” than its associated operator with property X which is not CW-X.

The paper is structured as follows. In Subsections 2.1 and 2.2 we develop our
framework of componental properties of operators. In Subsection 2.3 we define
$\alpha_{j}$-contractions and formulate the Banach fixed point theorem for
them. In Section 3 we define and study the componental regularity property,
and in Section 4 we present and analyze our motivating case of extending the
fully-simultaneous Diagonally-Relaxed Orthogonal Projections (DROP) method of
[10], originally applicable to the linear case only.

##  2 Componental operators

Let $\mathcal{H}_{j}$ be a real Hilbert space with inner product
$\langle\cdot,\cdot\rangle_{j}$ and induced norm $\|\cdot\|_{j}$, $j\in
J:=\\{1,2,...,n\\}$. Define the product Hilbert space
$\mathcal{H}:=\mathcal{H}_{1}\times\cdots\times\mathcal{H}_{n}$ with inner
product $\langle\cdot,\cdot\rangle:\mathcal{H\times H}\rightarrow\mathbb{R}$
defined by $\langle x,y\rangle$ $:=\sum_{j=1}^{n}\langle
x_{j},y_{j}\rangle_{j}$ and the induced norm
$\|\cdot\|:\mathcal{H}\rightarrow\mathbb{R}$ defined by
$\|x\|:=\sqrt{\sum_{j=1}^{n}\|x_{j}\|_{j}^{2}}$, where
$x=(x_{1},x_{2},...,x_{n})\in\mathcal{H}$,
$y=(y_{1},y_{2},...,y_{n})\in\mathcal{H}$ with
$x_{j},y_{j}\in\mathcal{H}_{j}$, $j\in J$. Let
$T:\mathcal{H}\rightarrow\mathcal{H}$, i.e.,

$T(x)=((T(x))_{1},(T(x))_{2},...,(T(x))_{n})\text{,}$ (4)

where $(T(x))_{j}\in\mathcal{H}_{j}$ denotes “the $j$-th componental operator
of $T(x)$”, $j\in J$. Denote by
$T_{\lambda}:=\operatorname*{Id}+\lambda(T-\operatorname*{Id})$ the
$\lambda$-relaxation of $T$, where $\lambda\geq 0$ and $\operatorname*{Id}$
denotes the identity.

###  2.1 Componental nonexpansive, firmly nonexpansive, relaxed firmly
nonexpansive and averaged operators

Bearing in mind the well-known definitions of operators that are nonexpansive
(NE), firmly nonexpansive (FNE), relaxed firmly nonexpansive (RFNE), or
averaged (AV), see, e.g., [8] or [3], we introduce the following new
definitions. We refer to any of these types of operators by the general name
of “componental operators”.

###### Definition 1

Let $j\in J$. We say that an operator $T:\mathcal{H}\rightarrow\mathcal{H}$
is:

  1. (i)

$j$-nonexpansive ($j$-NE) if for all $x,y\in\mathcal{H}$

$\|(T(x))_{j}-(T(y))_{j}\|_{j}\leq\|x_{j}-y_{j}\|_{j}\text{;}$ (5)

  2. (ii)

$j$-firmly nonexpansive ($j$-FNE) if for all $x,y\in\mathcal{H}$

$\langle(T(x))_{j}-(T(y))_{j},x_{j}-y_{j}\rangle_{j}\geq\|(T(x))_{j}-(T(y))_{j}\|_{j}^{2}\text{;}$
(6)

  3. (iii)

$j$-relaxed firmly nonexpansive ($j$-RFNE), if there exist a constant
$\lambda\in[0,2]$ and a $j$-FNE operator $U:\mathcal{H}\rightarrow\mathcal{H}$
such that $(T(\cdot))_{j}$ is a $\lambda$-relaxation of $(U(\cdot))_{j}$;

  4. (iv)

$j$-averaged ($j$-AV), if there exist a constant $\alpha\in(0,1)$ and a $j$-NE
operator $U$ such that

$(T(x))_{j}=(1-\alpha)x_{j}+\alpha(U(x))_{j};$ (7)

  5. (v)

component-wise NE (CW-FNE, CW-RFNE, CW-AV) if $T$ is $j$-NE ($j$-FNE,
$j$-RFNE, $j$-AV) for all $j\in J$.

Alternatively, if we want to emphasize the constants $\lambda$ in (iii) and
$\alpha$ in (iv) explicitly, then we say that $T$ is $(\lambda,j)$-relaxed
firmly nonexpansive ($(\lambda,j)$-RFNE) in (iii) and $(\alpha,j)$-averaged
($(\alpha,j)$-AV) in (iv).

Clearly, a CW-NE (CW-FNE) operator is NE (FNE). Note, however, that the
converse is not true.

###### Example 2

The operator $T:\mathbb{R}^{2}\rightarrow\mathbb{R}^{2}$ defined by
$T(x_{1},x_{2})=(x_{2},x_{1})$ is NE but not CW-NE.

###### Fact 3

Let $j\in J$ and let $T$ be $j$-NE. Let $x,y\in\mathcal{H}$ be such that
$x_{j}=y_{j}$. Then

$(T(x))_{j}=(T(y))_{j}\text{.}$ (8)

Thus, one can define an operator
$T^{j}:\mathcal{H}_{j}\rightarrow\mathcal{H}_{j}$ by

$T^{j}(x_{j}):=(T(x))_{j}.$ (9)

Proof. Equality (8) follows directly from Definition 1.

Fact 3 has a consequence that the componental operators of a CW-NE operator
$T$ are NE operators.

###### Fact 4

If an operator $T:\mathcal{H}\rightarrow\mathcal{H}$ is CW-NE, then for any
$x\in\mathcal{H}$ it holds

$T(x)=(T^{1}(x_{1}),T^{2}(x_{2}),...,T^{n}(x_{n}))$ (10)

and $T^{j},j\in J$, are NE. Conversely, if the operators
$T^{j}:\mathcal{H}_{j}\rightarrow\mathcal{H}_{j}$, $j\in J$, are NE then the
operator $T$ defined by (10) is CW-NE.

Proof. In view of (4), Equality (10) follows from Fact 3.

The following corollary shows that we can set a common constant
$\alpha\in(0,1)$ and a common CW-NE operator $U$ in the definition of a CW-AV
operator $T$.

###### Corollary 5

An operator $T:\mathcal{H}\rightarrow\mathcal{H}$ is CW-AV if and only if
there is a constant $\alpha\in(0,1)$ and a CW-NE operator $U$ such that
$T=V_{\alpha}:=(1-\alpha)\operatorname*{Id}+\alpha U$.

Proof. The “if” part is obvious. Suppose that $T$ is CW-AV. Then for any $j\in
J$ there are $\alpha_{j}\in(0,1)$ and CW-NE operators
$U_{j}:\mathcal{H}\rightarrow\mathcal{H}$ such that for all $x\in\mathcal{H}$
it holds

$(T(x))_{j}=(1-\alpha_{j})x_{j}+\alpha_{j}(U_{j}(x))_{j}.$ (11)

By Fact 4,
$U_{j}(x)=(U_{j}^{1}(x_{1}),U_{j}^{2}(x_{2}),...,U_{j}^{n}(x_{n}))$, $j\in J$,
$x\in\mathcal{H}$, and all operators
$U_{j}^{i}:\mathcal{H}_{i}\rightarrow\mathcal{H}_{i}$, $i,j\in J$, defined by
$U_{j}^{i}(x_{i}):=(U_{j}(x))_{i}$ are NE. Let $\alpha:=\max_{j\in
J}\alpha_{j}$ and $\beta_{j}:=\alpha_{j}/\alpha\in(0,1]$. Define
$V_{j}:=(U_{j}^{j})_{\beta_{j}}=(1-\beta_{j})\operatorname*{Id}+\beta_{j}U_{j}^{j}$,
$j\in J$. Clearly, $U_{j}^{j}=(V_{j})_{\beta_{j}^{-1}}$, $j\in J$. Define
$V:\mathcal{H}\rightarrow\mathcal{H}$ by
$V(x)=(V_{1}(x_{1}),V_{2}(x_{2}),...,V_{n}(x_{n}))$, $x\in\mathcal{H}$. The
operators $V_{j}$, $j\in J$, are NE as convex combinations of NE operators
$U_{j}^{j}$ and $\operatorname*{Id}$. Again, by Fact 4, $V$ is CW-NE. We have

$(T(x))_{j}=(U_{j}^{j})_{\alpha_{j}}(x_{j})=((V_{j})_{\beta_{j}^{-1}})_{\alpha_{j}}(x_{j})=(V_{j})_{\alpha}(x_{j})=(V_{\alpha}(x))_{j}\text{.}$
(12)

Thus, $T=V_{\alpha}$, where $V$ is CW-NE and $\alpha\in(0,1)$.

In a similar way one can prove that we can set a common constant
$\lambda\in[0,2]$ and a common CW-FNE operator $U$ in the definition of a CW-
RFNE operator $T$. This yields that a CW-RFNE operator is RFNE.

###### Fact 6

Let $j\in J$ and let $T:\mathcal{H}\rightarrow\mathcal{H}$. The following
conditions are equivalent:

  1. $\mathrm{(i)}$

$T$ is $j$-FNE;

  2. $\mathrm{(ii)}$

$T_{\lambda}$ is $j$-NE for all $\lambda\in[0,2]$;

  3. $\mathrm{(iii)}$

$T$ has the form $T=\frac{1}{2}(\operatorname*{Id}+S)$ for some $j$-NE
operator $S$, i.e., $T$ is $(\frac{1}{2},j)$-AV;

  4. $\mathrm{(iv)}$

$\operatorname*{Id}-T$ is $j$-FNE;

  5. $\mathrm{(v)}$

The following inequality holds for all $x,y\in\mathcal{H}$

$\|(T(x))_{j}-(T(y))_{j}\|_{j}^{2}\leq\|x_{j}-y_{j}\|_{j}^{2}-\|(x_{j}-(T(x))_{j})-(y_{j}-(T(y))_{j})\|_{j}^{2}.$
(13)

Proof. Similar to [8, Theorem 2.2.10].

Denote $I:=\\{1,2,...,m\\}$, where $m\in\mathbb{N}$.

###### Corollary 7

Let $j\in J$, $T_{i}:\mathcal{H}\rightarrow\mathcal{H}$ be $j$-FNE, for all
$i\in I$ and
$w=(w_{1},w_{2},...,w_{m})\in\Delta_{m}:=\\{w\in\mathbb{R}^{m}\mid w\geq 0,$
$\sum_{i=1}^{m}w_{i}=1\\}.$ Then the operator
$T:\mathcal{H}\rightarrow\mathcal{H},$ defined by
$T(x):=\sum_{i=1}^{m}w_{i}T_{i}(x),$ is $j$-FNE.

Proof. Similar to [8, Corollary 2.2.20].

For a $j$-NE operator $T_{i}:\mathcal{H}\rightarrow\mathcal{H},$ the operators
$T_{i}^{j}:\mathcal{H}_{j}\rightarrow\mathcal{H}_{j}$ are defined in a similar
way as in (9), i.e.,

$T_{i}^{j}(x_{j}):=(T_{i}(x))_{j}\text{, }i\in I,\text{ }j\in J.$ (14)

###### Fact 8

Let $j\in J$, $w_{ij}\geq 0$, $i\in I$, $w_{\cdot j}:=\sum_{i=1}^{m}w_{ij}>0$.
If $T_{i}:\mathcal{H}\rightarrow\mathcal{H}$, $i\in I$, are $j$-FNE, then the
operator $T:\mathcal{H}\rightarrow\mathcal{H}$ defined by its components
$T^{j}:\mathcal{H}_{j}\rightarrow\mathcal{H}_{j}$,

$T^{j}(x_{j}):=x_{j}+\lambda\sum_{i=1}^{m}w_{ij}(T_{i}^{j}(x_{j})-x_{j})\text{,}$
(15)

where $\lambda\in[0,2/w_{\cdot j}]$, is $(\lambda w_{\cdot j},j)$-RFNE.

Proof. Clearly, $T_{i}^{j}$ are FNEs, $i\in I$. Define an operator
$S_{j}:\mathcal{H}_{j}\rightarrow\mathcal{H}_{j}$ by

$S_{j}(x_{j}):=\sum_{i=1}^{m}v_{i}T_{i}^{j}(x_{j})\text{,}$ (16)

where $v_{i}:=\frac{{\textstyle w_{ij}}}{{\textstyle w_{\cdot j}}}$. Since
$\sum_{i=1}^{m}v_{i}=1$, the operator $S_{j}$ is FNE, see [8, Corollary
2.2.20], and we have that $T^{j}=(S_{j})_{\lambda w_{\cdot j}}$, the $(\lambda
w_{\cdot j})$-relaxation of $S_{j},$ i.e., $T^{j}$ is $(\lambda w_{\cdot
j},j)$-RFNE.

###### Corollary 9

Let $w_{ij}\geq 0$, $w_{\cdot j}:=\sum_{i=1}^{m}w_{ij}>0,$ $i\in I,$ $j\in J$,
and $w:=\max_{j\in J}w_{\cdot j}$. If
$T_{i}:\mathcal{H}\rightarrow\mathcal{H}$, $i\in I$, are CW-FNE, then the
operator $T$ defined by

$(T(x))_{j}:=x_{j}+\lambda\sum_{i=1}^{m}w_{ij}((T_{i}(x_{j}))_{j}-x_{j})\text{,}$
(17)

$j\in J$, where $\lambda\in[0,2/w]$, is $(\lambda w)$-CW-RFNE.

Proof. By Facts 4 and 8, $T^{j}$ is $\lambda w_{\cdot j}$-RFNE, $j\in J$.
Since $0\leq\lambda w_{\cdot j}\leq\lambda w\leq 2$, $T^{j}$ is $\lambda
w$-RFNE, $j\in J$. Noting that
$T(x)=(T^{1}(x_{1}),T^{2}(x_{2}),...,T^{n}(x_{n}))$, tells us that $T$ is
$\lambda w$-RFNE.

###### Example 10

Consider a consistent system of linear equations $Ax=b$, where $A$ is an
$m\times n$ matrix with rows $a_{i}\in\mathbb{R}^{n}$ and
$b\in\mathbb{R}^{m}$. Consider the hyperplanes
$H_{i}:=\\{x\in\mathbb{R}^{n}\mid\left\langle a_{i},x\right\rangle=b_{i}\\}$,
$i\in I$. A special case of an operator $T$ defined by (17), occurs when
$T_{i}:=P_{H_{i}}$, the metric projection onto $H_{i},$ $w_{ij}:=w_{i}/s_{j}$,
$i\in I$, $i\in J$, with $w=(w_{1},w_{2},...,w_{m})\in\Delta_{m}$ and $s_{j}$
being the number of nonzero elements in the $j$-th column of $A.$ This special
case, introduced and investigated in [10, Equation (1.11)], is our motivating
example, discussed in Section 4 below.

###  2.2 Componental cutter, quasi-nonexpansive, strongly quasi-nonexpansive
and strictly quasi-nonexpansive operators

Let $T:\mathcal{H}\rightarrow\mathcal{H}$ be defined by (4). For $j\in J$ we
define the “$j$-th fixed point set of $T$” by

$\operatorname*{Fix}{}^{j}T:=\\{z\in\mathcal{H}\mid(T(z))_{j}=z_{j}\\}\text{.}$
(18)

Clearly, $\operatorname*{Fix}T=\bigcap_{j\in J}\operatorname*{Fix}{}^{j}T$.

Bearing in mind the well-known definitions of operators that are cutter,
quasi-nonexpansive (QNE), strongly quasi-nonexpansive (SQNE), or strictly
quasi-nonexpansive (sQNE), see, e.g., [8], we introduce the following
additional new definitions of componental operators.

###### Definition 11

Let $j\in J$ and let $T:\mathcal{H}\rightarrow\mathcal{H}$ be an operator with
$\operatorname*{Fix}{}^{j}T\neq\emptyset$. We say that $T$ is:

  1. (i)

a $j$-cutter, if for all $x\in H$ and $z\in\operatorname*{Fix}{}^{j}T$ it
holds that

$\langle x_{j}-(T(x))_{j},z_{j}-(T(x))_{j}\rangle_{j}\leq 0\text{;}$ (19)

  2. (ii)

$j$-quasi-nonexpansive ($j$-QNE), if for all $x\in H$ and
$z\in\operatorname*{Fix}{}^{j}T$ it holds that

$\|(T(x))_{j}-z_{j}\|_{j}\leq\|x_{j}-z_{j}\|_{j}\text{;}$ (20)

  3. (iii)

$j$-strongly quasi-nonexpansive ($j$-SQNE), if there is a constant
$\rho_{j}>0$ such that for all $x\in H$ and $z\in\operatorname*{Fix}{}^{j}T$
it holds that

$\|(T(x))_{j}-z_{j}\|_{j}^{2}\leq\|x_{j}-z_{j}\|_{j}^{2}-\rho_{j}\|(T(x))_{j}-x_{j}\|_{j}^{2}\text{;}$
(21)

  4. (iv)

$j$-strictly quasi-nonexpansive ($j$-sQNE), if for all
$x\notin\operatorname*{Fix}{}^{j}T$ and $z\in\operatorname*{Fix}{}^{j}T$ it
holds that

$\|(T(x))_{j}-z_{j}\|_{j}<\|x_{j}-z_{j}\|_{j}\text{;}$ (22)

  5. (v)

a component-wise cutter (CW-QNE, CW-SQNE, CW-sQNE ) if $T$ is $j$-cutter
($j$-QNE, $j$-SQNE, $j$-sQNE) for all $j\in J$.

Alternatively, if we want to emphasize the constant $\rho_{j}$ in (iii) or the
vector $r:=(\rho_{1},\rho_{2},...,\rho_{n})$ in (v) explicitly, then we say
that $T$ is $(\rho_{j},j)$-SQNE or $T$ is $r$-CW-SQNE, respectively.

Note that if $T$ is $(\rho_{j},j)$-SQNE and $\rho_{j}\geq\rho>0$ then $T$ is
$(\rho,j)$-SQNE. Thus, one can set the constant $\rho$ in (v), which does not
depend on $j$, e.g., $\rho:=\min_{j\in J}\rho_{j}$.

###### Example 12

Let $U^{j}:\mathcal{H}_{j}\rightarrow\mathcal{H}_{j}$, $j\in J$, and define
$U:\mathcal{H}\rightarrow\mathcal{H}$ by

$U(x):=(U^{1}(x_{1}),U^{2}(x_{2}),...,U^{n}(x_{n}))\text{.}$ (23)

Clearly, $\operatorname*{Fix}U=\prod_{j=1}^{n}\operatorname*{Fix}U^{j}$. By
definition, $(U(x))_{j}=U^{j}(x_{j})$. Thus, it follows from Definitions 1 and
11 that $U^{j}$ is NE (a cutter, QNE, SQNE), if and only if $U$ is $j$-NE
($j$-cutter, $j$-QNE, $j$-SQNE), $j\in J$. Consequently, $U^{j}$ are NE (a
cutter, QNE, SQNE) for all $j\in J$, if and only if $U$ is CW-NE (CW-cutter,
CW-QNE, CW-SQNE).

###### Example 13

Let $S_{j}:\mathcal{H}\rightarrow\mathcal{H}$, $j\in J$, and define
$S:\mathcal{H}\rightarrow\mathcal{H}$ by

$S(x):=((S_{1}(x))_{1},(S_{2}(x))_{2},...,(S_{n}(x))_{n})\text{.}$ (24)

Then $(S(x))_{j}=(S_{j}(x))_{j}$, $j\in J$. Consequently, $S$ is a CW-cutter
($r$-CW-SQNE, where $r:=(\rho_{1},\rho_{2},...,\rho_{n})$ with $\rho_{j}>0$,
$j\in J)$ if and only if $S_{j}$ is a $j$-cutter ($(\rho_{j},j)$-SQNE) for all
$j\in J$.

For $j\in J$ and $T:\mathcal{H}\rightarrow\mathcal{H}$ define

$F^{j}(T):=\\{z\in\mathcal{H}\mid\|(T(x))_{j}-z_{j}\|_{j}\leq\|x_{j}-z_{j}\|_{j}\text{
for all }x\in\mathcal{H}\\}$ (25)

and

$F_{j}(T):=\\{z_{j}\in\mathcal{H}_{j}\mid\|(T(x))_{j}-z_{j}\|_{j}\leq\|x_{j}-z_{j}\|_{j}\text{
for all }x\in\mathcal{H}\\}\text{.}$ (26)

Clearly,

$F_{j}(T)=\bigcap_{x\in\mathcal{H}}\\{z_{j}\in\mathcal{H}_{j}\mid\|(T(x))_{j}-z_{j}\|_{j}\leq\|x_{j}-z_{j}\|_{j}\\}\text{,}$
(27)

thus, $F_{j}$ is a closed convex subset as intersection of closed half-spaces.

For a $j$-QNE operator $T$, where $j\in J$, the property expressed in Fact 3
is not true in general. Thus, contrary to an NE operator, a CW-QNE operator
cannot be decomposed. Nevertheless, the fixed point set of a CW-QNE operator
is a Cartesian product of some sets.

###### Fact 14

Let $j\in J$.

  1. $\mathrm{(i)}$

The following inclusion holds

$F^{j}(T)\subseteq\operatorname*{Fix}{}^{j}T\text{.}$ (28)

  2. $\mathrm{(ii)}$

If $T$ is $j$-QNE then the converse inclusion is also true. Consequently, for
a CW-QNE operator $T$ we have

$\operatorname*{Fix}T=\bigcap_{j=1}^{n}\operatorname*{Fix}{}^{j}T=\bigcap_{j=1}^{n}F^{j}(T)=\prod_{j=1}^{n}F_{j}(T)\text{.}$
(29)

  3. $\mathrm{(iii)}$

If $T:\mathcal{H}\rightarrow\mathcal{H}$ is a CW-cutter (CW-QNE, CW-SQNE, CW-
sQNE), then $T$ is a cutter (QNE, SQNE, sQNE).

Proof. (i) If $F^{j}(T)=\emptyset$ then the inclusion in (28) is clear. Let
now $F^{j}(T)\neq\emptyset$ and $z\in F^{j}(T)$. If we take $x=z$ in (25) then
we obtain

$\|(T(z))_{j}-z_{j}\|_{j}\leq\|z_{j}-z_{j}\|_{j}=0\text{,}$ (30)

i.e., $z\in\operatorname*{Fix}{}^{j}T$.

(ii) Suppose that $T$ is $j$-QNE and let $z\in\operatorname*{Fix}{}^{j}T$.
Then, by definition, $z\in F^{j}(T)$. This together with (i) implies the
second equality in (29) if $T$ is CW-QNE. The first and the last equalities in
(29) are obvious.

(iii) Follows from the definition of the inner product in $\mathcal{H}$ and
from the definition of a CW-cutter (CW-QNE, CW-SQNE operator).

If we consider a relaxation of an operator
$T:\mathcal{H}\rightarrow\mathcal{H}$ we can associate different relaxation
parameters $\lambda_{j}$ with various components $(T(\cdot))_{j}$. Let
$y:=(\lambda_{1},\lambda_{2},...,\lambda_{n})\in\mathbb{R}_{+}^{n}$. We say
that the operator $T_{y},$ defined by,

$(T_{y}(x))_{j}:=x_{j}+\lambda_{j}((T(x))_{j}-x_{j})\text{, }j\in J\text{,}$
(31)

is the $y$-CW-relaxation of $T$ or, in short, a CW-relaxation of $T$. Clearly,
if $\lambda_{j}=\lambda$ for all $j\in J$ then the notion of the $y$-CW-
relaxation of $T$ coincides with the notion of the $\lambda$-relaxation of
$T$.

###### Fact 15

Let $T:\mathcal{H}\rightarrow\mathcal{H}$ with
$\operatorname*{Fix}T\neq\emptyset$, let
$y=(\lambda_{1},\lambda_{2},...,\lambda_{n})\in(0,2]^{n}$. If $T$ is a CW-
cutter then its $y$-relaxation $T_{y}$ is $r$-CW-SQNE, where
$r:=(\rho_{1},\rho_{2},...,\rho_{n})$ with
$\rho_{j}=(2-\lambda_{j})/\lambda_{j}$, $j\in J$.

Proof. Suppose that $T$ is a CW-cutter and let $z\in\operatorname*{Fix}T$. For
any $j\in J$ we have

$\displaystyle\|(T_{y}(x))_{j}-z_{j}\|_{j}^{2}$
$\displaystyle=\|x_{j}+\lambda_{j}((T(x))_{j}-x_{j})-z_{j}\|_{j}^{2}$
$\displaystyle=\|x_{j}-z_{j}\|_{j}^{2}+\lambda_{j}^{2}\|(T(x))_{j}-x_{j}\|_{j}^{2}$
$\displaystyle+2\lambda_{j}\langle
x_{j}-(T(x))_{j},z_{j}-(T(x))_{j}\rangle_{j}-2\lambda_{j}\|(T(x))_{j}-x_{j}\|_{j}^{2}$
$\displaystyle\leq\|x_{j}-z_{j}\|_{j}^{2}-\lambda_{j}(2-\lambda_{j})\|(T(x))_{j}-x_{j}\|_{j}^{2}$
$\displaystyle=\|x_{j}-z_{j}\|_{j}^{2}-\frac{2-\lambda_{j}}{\lambda_{j}}\|(T_{\lambda}(x))_{j}-x_{j}\|_{j}^{2}\text{,}$
(32)

which means that $T_{y}$ is $r$-CW-SQNE.

###### Fact 16

Let $j\in J$. A $j$-NE operator having a fixed point is $j$-QNE. Consequently,
a CW-NE operator having a fixed point is CW-QNE.

Proof. Let $T$ be $j$-NE and let $z\in\operatorname*{Fix}{}^{j}T$. Then
$(T(z))_{j}=z_{j}$ and, for all $x\in\mathcal{H},$ we have

$\|(T(x))_{j}-z_{j}\|_{j}=\|(T(x))_{j}-(T(z))_{j}\|_{j}\leq\|x_{j}-z_{j}\|_{j}\text{.}$
(33)

This means that $T$ is $j$-QNE.

###### Fact 17

Let $j\in J$, let $T_{i}:\mathcal{H}\rightarrow\mathcal{H}$ be $j$-sQNE, $i\in
I$, with $\bigcap_{i\in I}F^{j}(T_{i})\neq\emptyset$, and
$T:=\sum_{i=1}^{m}w_{i}T_{i}$, where $w\in\operatorname*{ri}\Delta_{m}$
($\operatorname*{ri}$ is the relative interior). Then

$F^{j}(T)=\bigcap_{i\in I}F^{j}(T_{i})$ (34)

and $T$ is $j$-sQNE.

Proof. To prove the inclusion $\supseteq$ in (34) , let $z\in\bigcap_{i\in
I}F^{j}(T_{i})$ and $x\in\mathcal{H}$ be arbitrary. The convexity of the norm
and the assumption that $T_{i}$ are $j$-QNE, $i\in I$, yield

$\|(T(x))_{j}-z_{j}\|_{j}\leq\sum_{i=1}^{m}w_{i}\|(T_{i}(x))_{j}-z_{j}\|_{j}\leq\sum_{i=1}^{m}w_{i}\|x_{j}-z_{j}\|_{j}=\|x_{j}-z_{j}\|_{j}\text{,}$
(35)

which shows that $z\in F^{j}(T)$.

To prove the inclusion $\subseteq$ in (34) we observe that the inclusion is
clear if $\bigcap_{i\in I}F^{j}(T_{i})=\mathcal{H}$. Suppose the opposite and
let $x\notin\bigcap_{i\in I}F^{j}(T_{i})$ and $z\in\bigcap_{i\in
I}F^{j}(T_{i})$. The convexity of the norm and the assumption that $T_{i}$ are
$j$-sQNE, $i\in I$, yield

$\|(T(x))_{j}-z_{j}\|_{j}\leq\sum_{i=1}^{m}w_{i}\|(T_{i}(x))_{j}-z_{j}\|_{j}<\sum_{i=1}^{m}w_{i}\|x_{j}-z_{j}\|_{j}=\|x_{j}-z_{j}\|_{j}\text{,}$
(36)

because $\|(T_{i_{0}}(x))_{j}-z_{j}\|_{j}<\|x_{j}-z_{j}\|_{j}$ for some
$i_{0}$, and $w_{i_{0}}>0$. Now it is clear that
$x\notin\operatorname*{Fix}{}^{j}T=F^{j}(T)$, because otherwise,

$\|x_{j}-z_{j}\|_{j}=\|(T(x))_{j}-z_{j}\|_{j}<\|x_{j}-z_{j}\|_{j}$ (37)

which would lead to a contradiction.

###### Fact 18

Let $j\in J$, $T_{i}:\mathcal{H}\rightarrow\mathcal{H}$ be $j$-cutters having
a common fixed point, $i\in I$ and $w\in\operatorname*{ri}\Delta_{m}$. Then
the operator $T:\mathcal{H}\rightarrow\mathcal{H}$ defined by
$T(x):=\sum_{i=1}^{m}w_{i}T_{i}(x)$ is a $j$-cutter.

Proof. Similar to [8, Corollary 2.1.49].

###### Corollary 19

Let $T_{i}:\mathcal{H}\rightarrow\mathcal{H}$ be CW-cutters having a common
fixed point, $i\in I$, and $w\in\Delta_{m}$. Then the operator
$T:\mathcal{H}\rightarrow\mathcal{H}$ defined by
$T(x)=\sum_{i=1}^{m}w_{i}T_{i}(x)$ is a CW-cutter. If, moreover, $w_{i}>0$,
$i\in I$, then $\operatorname*{Fix}T=\bigcap_{i\in
I}\operatorname*{Fix}T_{i}$.

###### Fact 20

Let $w_{ij}\geq 0$, $i\in I$, $w_{\cdot j}:=\sum_{i=1}^{m}w_{ij}>0$, $j\in J$.
If $T_{i}:\mathcal{H}\rightarrow\mathcal{H}$, $i\in I$, are CW-cutters having
a common fixed point then the operator $T:\mathcal{H}\rightarrow\mathcal{H},$
defined by

$(T(x))_{j}:=x_{j}+\lambda_{j}\sum_{i=1}^{m}w_{ij}((T_{i}(x))_{j}-x_{j})\text{,}$
(38)

$j\in J$, where $\lambda_{j}\in(0,2/w_{\cdot j})$, is $r$-CW-SQNE, where
$r:=(\rho_{1},\rho_{2},...,\rho_{n})$ with
$\rho_{j}=(2-\lambda_{j})/\lambda_{j},j\in J$.

Proof. Define an operator $S_{j}:\mathcal{H}\rightarrow\mathcal{H}$ by

$S_{j}(x):=\sum_{i=1}^{m}\frac{w_{ij}}{w_{\cdot j}}T_{i}(x)\text{, }j\in J.$
(39)

By Fact 18, $S_{j}$ is a $j$-cutter, $j\in J$. Define, for $x\in\mathcal{H}$,

$S(x)=((S_{1}(x))_{1},(S_{2}(x))_{2},...,(S_{n}(x))_{n})\text{.}$ (40)

The operator $S:\mathcal{H}\rightarrow\mathcal{H}$ is a CW-cutter (see Example
13). We have

$(S(x))_{j}=(S_{j}(x))_{j}=\sum_{i=1}^{m}\frac{w_{ij}}{w_{\cdot
j}}(T_{i}(x))_{j}\text{,}$ (41)

and

$(T(x))_{j}=x_{j}+\lambda w_{\cdot j}((S(x))_{j}-x_{j})\text{.}$ (42)

Denote $y=(\lambda w_{\cdot 1},\lambda w_{\cdot 2},...,\lambda w_{\cdot n})$.
Equality (42) means that $T=S_{y}$, i.e., $T$ is a $y$-CW-relaxation of a CW-
cutter $S$. Thus, Fact 15 yields that the operator $T$ is $r$-CW-SQNE, where
$r:=(\rho_{1},\rho_{2},...,\rho_{n})$ with
$\rho_{j}=(2-\lambda_{j})/\lambda_{j},j\in J$.

###  2.3 Componental contractions

###### Definition 21

An operator $T:\mathcal{H}\rightarrow\mathcal{H}$ is a $j$-contraction
($j$-CONT) if, for some $j\in J,$

$\|(T(x))_{j}-(T(y))_{j}\|_{j}\leq\alpha_{j}\|x_{j}-y_{j}\|_{j},\text{ for all
}x,y\in\mathcal{H},$ (43)

with an $\alpha_{j}\in[0,1)$. Alternatively, if we want to emphasize the
constant $\alpha_{j}$ explicitly, then we say that $T$ is an
$(\alpha_{j},j)$-contraction.

If $T$ is an $(\alpha_{j},j)$-contraction for all $j\in J$ then it is an
$\alpha$-contraction with $\alpha:=\max_{1\leq j\leq n}\alpha_{j}.$ That the
opposite is not true follows from the counter example
$T(x_{1},x_{2}):=\left({\textstyle\frac{1}{2}x_{2},{\textstyle\frac{1}{2}x_{1}}}\right)$
which is an $\alpha$-contraction, with, e.g., $\alpha=0.5$, since

$\left\|T(x)-T(y)\right\|=\frac{1}{2}\left\|x-y\right\|.$ (44)

However, for the points $(0,0)$ and $(1,5)$ there does not exist a real number
$\alpha_{1}\in[0,1)$ with which $T$ is $(\alpha_{1},1)$-contractive.

Following the Banach fixed point theorem, as reformulated in Berinde’s book
[5, Theorem 2.1], we formulate a componental contraction mapping principle as
follows.

###### Theorem 22

Let $j\in J$ and let $T:\mathcal{H}\rightarrow\mathcal{H}$ be an
$(\alpha_{j},j)$-contraction. Then

  1. $\mathrm{(i)}$

The $j$-th componental fixed point set of $T$ is nonempty, i.e.,
$\operatorname*{Fix}{}^{j}T:=\\{x\in\mathcal{H}\mid(T(x))_{j}=x_{j}\\}\neq\emptyset$
and the variable $x_{j}$ is unique for all $x\in\operatorname*{Fix}{}^{j}T,$
henceforth denoted as $x_{j}=x_{j\ast}$.

  2. $\mathrm{(ii)}$

The sequence $\\{x_{j}^{k}\\}_{k=0}^{\infty}$ of the $j$-th components of any
sequence $\\{x^{k}\\}_{k=0}^{\infty},$ generated by the Picard iteration
$x^{k+1}=T(x^{k})$ associated with $T,$ converges for any initial point
$x^{0}\in\mathcal{H},$ and

$\lim_{k\rightarrow\infty}x_{j}^{k}=x_{j\ast}.$ (45)

  3. $\mathrm{(iii)}$

The following a priori and a posteriori error estimates hold:

$\left\|x_{j}^{k}-x_{j\ast}\right\|_{j}\leq\frac{(\alpha_{j})^{k}}{1-\alpha_{j}}\left\|x_{j}^{0}-x_{j}^{1}\right\|_{j},\text{
for all }k=1,2,\ldots,$ (46)
$\left\|x_{j}^{k}-x_{j\ast}\right\|_{j}\leq\frac{\alpha_{j}}{1-\alpha_{j}}\left\|x_{j}^{k-1}-x_{j}^{k}\right\|_{j},\text{
for all }k=1,2,\ldots.$ (47)

  4. $\mathrm{(iv)}$

The rate of convergence of the sequence $\\{x_{j}^{k}\\}_{k=0}^{\infty},$ in
$\mathrm{(ii)}$ above, is given by

$\left\|x_{j}^{k}-x_{j\ast}\right\|_{j}\leq\alpha_{j}\left\|x_{j}^{k-1}-x_{j\ast}\right\|_{j}\leq(\alpha_{j})^{k}\left\|x_{j}^{0}-x_{j\ast}\right\|_{j},\text{
for all }k=1,2,\ldots.$ (48)

Proof. This can be proved exactly along the lines of the proof in [5, Theorem
2.1]. Alternatively, one can introduce an operator $U$
$:\mathcal{H}\rightarrow\mathcal{H}$ by $U(x):=(0,...,0,T_{j}(x),0,...,0)$ and
apply [5, Theorem 2.1] to it.

To justify Theorem 22 we build an example of an operator which is not an
$\alpha$-contraction but is an $\alpha_{j}$-contractions for some indices
$j\in J,$ but not all. The original Banach fixed point theorem would not apply
to them but our theorem would.

###### Example 23

Let $T:\mathbb{R}^{2}\rightarrow\mathbb{R}^{2}$ be defined by

$T(x_{1},x_{2})=\left(\begin{array}[c]{c}T_{1}(x_{1},x_{2})\\\
T_{2}(x_{1},x_{2})\end{array}\right):=\left(\begin{array}[c]{c}\frac{x_{1}}{2}+3\\\
8x_{2}\end{array}\right).$ (49)

This $T$ is an $\alpha_{1}$-contraction with, e.g., $\alpha_{1}=1/2$ because

$\left|T_{1}(x_{1},x_{2})-T_{1}(y_{1},y_{2})\right|=\frac{1}{2}\left|x_{1}-y_{1}\right|.$
(50)

But $T$ is not an $\alpha$-contraction because

$\left\|T(x)-T(y)\right\|^{2}=\frac{1}{4}(x_{1}-y_{1})^{2}+64(x_{2}-y_{2})^{2}$
(51)

and there is no $\alpha\in[0,1)$ for which

$\frac{1}{4}(x_{1}-y_{1})^{2}+64(x_{2}-y_{2})^{2}\leq\alpha^{2}\left((x_{1}-y_{1})^{2}+(x_{2}-y_{2})^{2}\right)$
(52)

for all $x,y\in\mathbb{R}^{2}.$

##  3 Regularity of component-wise quasi-nonexpansive operators

The notions of asymptotic regularity of sequences and operators play a central
role in fixed point theory, see, e.g., [3] or [8]. We define next a notion of
componental regularity.

###### Definition 24

Let $T:\mathcal{H}\rightarrow\mathcal{H}$ be a QNE operator and let $j\in J$.

  1. (i)

We say that $T$ is $j$-weakly regular ($j$-WR) if, for any sequence
$\\{x^{k}\\}_{k=0}^{\infty}\subseteq\mathcal{H}$ and some $y\in\mathcal{H}$,

$x_{j}^{k}\rightharpoonup y_{j}\text{ and
}\lim_{k\rightarrow\infty}\|(T(x^{k}))_{j}-x_{j}^{k}\|_{j}=0\Longrightarrow
y\in\operatorname*{Fix}{}^{j}T.$ (53)

  2. (ii)

If $T$ is $j$-weakly regular for all $j\in J$, then we say that $T$ is CW-
weakly regular (CW-WR).

  3. (iii)

If (53) holds after all $j$ indices are removed from it then we say that $T$
is weakly regular (WR) (cf. [9, Definition 3.1]).

The weak regularity of an operator $T$ means that $T-\operatorname*{Id}$ is
demi-closed at $0$ (cf. [13]).

###### Fact 25

A QNE operator $T:\mathcal{H}\rightarrow\mathcal{H}$ is CW-WR if and only if
$T$ is WR.

Proof. Suppose that $T$ is CW-WR. Let $x^{k}\rightharpoonup y$ and
$\lim_{k\rightarrow\infty}\|T(x^{k})-x^{k}\|=0$. Then
$x_{j}^{k}\rightharpoonup y_{j}$ and
$\lim_{k\rightarrow\infty}\|(T(x^{k}))_{j}-x_{j}^{k}\|_{j}=0$ for all $j\in
J$. Since $T$ is $j$-WR, $y\in\operatorname*{Fix}{}^{j}T$, for all $j\in J$,
thus, $y\in\bigcap_{j\in J}\operatorname*{Fix}{}^{j}T=\operatorname*{Fix}T$
which means that $T$ is WR. The converse can be proved similarly.

###### Theorem 26

Let $T:\mathcal{H}\rightarrow\mathcal{H}$ be an $r$-CW-SQNE and CW-WR
operator, where $r=(\rho_{1},\rho_{2},...,\rho_{n})$ with $\rho_{j}>0,j\in J$,
and let the sequence $\\{x^{k}\\}_{k=0}^{\infty}$ be generated by the
iteration

$x_{j}^{k+1}=(T(x^{k}))_{j},\text{\ }k\geq 0,$ (54)

$j\in J$ , where $x^{0}\in\mathcal{H}$ is arbitrary. Then
$\\{x^{k}\\}_{k=0}^{\infty}$ converges weakly to some
$x^{\ast}\in\operatorname*{Fix}T$.

Proof. By assumption, $T$ is $\rho_{j}$-SQNE, $j\in J$, i.e., for any
$z=(z_{1},z_{2},...,z_{n})\in\operatorname*{Fix}T=\bigcap_{j\in
J}\operatorname*{Fix}{}^{j}T$ it holds

$\|x_{j}^{k+1}-z_{j}\|_{j}^{2}=\|(T(x^{k}))_{j}-z_{j}\|_{j}^{2}\leq\|x_{j}^{k}-z_{j}\|_{j}^{2}-\rho_{j}\|(T(x^{k}))_{j}-x_{j}^{k}\|_{j}^{2}\text{.}$
(55)

That ${\operatorname*{Fix}}T$ is nonempty follows from Definition 11. Using
standard arguments this leads to the boundedness of $x_{j}^{k}$, $j\in J$ and
to $\|(T(x^{k}))_{j}-x_{j}^{k}\|_{j}\rightarrow 0$ as $k\rightarrow\infty$.
Thus, there exists a subsequence $\\{x_{j}^{n_{k}}\\}_{k=0}^{\infty}$ which
converges weakly to some $x_{j}^{\ast}$, $j\in J$. Let
$x^{\ast}=(x_{1}^{\ast},x_{2}^{\ast},...,x_{n}^{\ast})$. Because $T$ is
$j$-weakly regular, we have $x^{\ast}\in\operatorname*{Fix}{}^{j}T$, $j\in J$.
This and Fact 14(ii) yields $x^{\ast}\in\operatorname*{Fix}T$. By standard
arguments, like in [2, Theorem 2.16(ii)], the whole sequence
$\\{x^{k}\\}_{k=0}^{\infty}$ converges weakly to $x^{\ast}$.

##  4 Nonlinear DROP as a simultaneous projection method with component-wise
weights

Many problems in mathematics, in physical sciences and in real-world
applications can be modeled as a convex feasibility problem (CFP); i.e., a
problem of finding a point $x^{\ast}\in Q:=\cap_{i=1}^{m}Q_{i}$ in the
intersection of finitely many closed convex sets
$Q_{i}\subseteq\mathbb{R}^{n}$ in the finite-dimensional Euclidean space. The
literature on this subject is enormously large, see, e.g., [2] or [3] and [8]
and references therein.

Fully-simultaneous (parallel) algorithmic schemes for the CFP employ iterative
steps of the form

$x^{k+1}=x^{k}+\lambda_{k}\left(\sum_{i=1}^{m}w_{i}\left(P_{Q_{i}}(x^{k})-x^{k}\right)\right),$
(56)

where $P_{\Omega}(x)$ stands for the orthogonal (nearest Euclidean distance)
projection of a point $x$ onto the closed convex set $\Omega,$ the parameters
$\\{w_{i}\\}_{i=1}^{m}$ are a system of weights such that $w_{i}>0$ for all
$i=1,2,\ldots,m$ and $\sum_{i=1}^{m}w_{i}=1$, and the relaxation parameters
$\\{\lambda_{k}\\}_{k=0}^{\infty}$ are user-chosen and, in most convergence
analyses, must remain in a certain fixed interval, in order to guarantee
convergence.

For linear equations, represented by hyperplanes, i.e.,

$Q_{i}=H_{i}:=\left\\{x\in\mathbb{R}^{n}\mid\langle
a^{i},x\rangle=b_{i}\right\\},$ (57)

for $i=1,2,\ldots,m$, the orthogonal projection $P_{i}(z)$ of a point
$z\in\mathbb{R}^{n}$ onto $H_{i}$ is

$P_{i}(z)~{}=~{}z+\frac{b_{i}-\langle
a^{i},z\rangle}{\left\|a^{i}\right\|_{2}^{2}}a^{i}\,,$ (58)

where $a^{i}=(a_{j}^{i})_{j=1}^{n}\in\mathbb{R}^{n}$, $a^{i}\neq 0$, and
$b_{i}\in\mathbb{R}$ are the given data of the linear equations and
$\|\cdot\|_{2}$ is the Euclidean norm. The iterative steps of (56) then take
the form

$x^{k+1}~{}=~{}x^{k}+\lambda_{k}\sum_{i=1}^{m}w_{i}\frac{b_{i}-\langle
a^{i},x^{k}\rangle}{\left\|a^{i}\right\|_{2}^{2}}a^{i}.$ (59)

This algorithm was first proposed by Cimmino [11] (read about the profound
impact of this paper on applied scientific computing in Benzi’s paper [4]) and
generalized to convex sets by Auslender [1].

For the case of equal weights $w_{i}=1/m$ (59) becomes

$x^{k+1}~{}=~{}x^{k}+\frac{\lambda_{k}}{m}\sum_{i=1}^{m}\frac{b_{i}-\langle
a^{i},x^{k}\rangle}{\left\|a^{i}\right\|_{2}^{2}}a^{i}.$ (60)

When the $m\times n$ system matrix $A=(a_{j}^{i})$ is sparse, as often happens
in some important real-world applications, only a relatively small number of
the elements $\\{a_{j}^{1},a_{j}^{2},\ldots,a_{j}^{m}\\}$ in the $j$-th column
of $A$ are nonzero, but in (60) the sum of their contributions is divided by
the relatively large $m$ – slowing down the progress of the algorithm. This
observation led us, in [10], to consider replacement of the factor $1/m$ in
(60) by a factor that depends only on the number of nonzero elements in the
set $\\{a_{j}^{1},a_{j}^{2},\ldots,a_{j}^{m}\\}$. Specifically, for each
$j=1,2,\ldots,n$, we denoted by $s_{j}$ the number of nonzero elements in
column $j$ of the matrix $A$, assuming that all columns of $A$ are nonzero,
thus $s_{j}\neq 0,$ for all $j.$ Then we replaced (60) by

$x_{j}^{k+1}~{}=~{}x_{j}^{k}+\frac{\lambda_{k}}{s_{j}}\sum_{i=1}^{m}\frac{b_{i}-\langle
a^{i},x^{k}\rangle}{\left\|a^{i}\right\|_{2}^{2}}a_{j}^{i}\,,\text{ for
}j=1,2,\ldots,n.$ (61)

This iterative formula is the backbone of the proposed fully-simultaneous
Diagonally-Relaxed Orthogonal Projections (DROP) method for linear equations
in [10]. Certainly, if the matrix $A$ is sparse, then the $s_{j}$ values will
be much smaller than $m$ and using them instead of the large $m$ will enlarge
the additive updates in the iterative process and lead to faster initial
convergence.

This leads naturally to the question whether the weights $w_{i}$ in (56) be
allowed to depend on the component index $j$ as iterations proceed, without
loosing the guaranteed convergence of the algorithm? Or, phrased
mathematically, may the iterations proceed according to

$x_{j}^{k+1}=x_{j}^{k}+\lambda_{k}\left(\sum_{i=1}^{m}w_{ij}\left(\left(P_{Q_{i}}(x^{k})\right)_{j}-x_{j}^{k}\right)\right),\text{
for }j=1,2,\ldots,n,$ (62)

where the parameters $\\{w_{ij}\\}_{i=1}^{m}$ form $n$ systems of weights such
that, for $j=1,2,\ldots,n,$ $w_{ij}\geq 0$ for all $i=1,2,\ldots,m,$ and
$\sum_{i=1}^{m}w_{ij}=1$? If such component-wise relaxation is possible then
we could use it to exploit sparsity of the underlying problem and to control
asynchronous (block) iterations.

The convergence of such a scheme, like (61), for the linear case is studied in
[10] but the general case of (62) remained unsettled.

With $T_{i}:=P_{Q_{i}},$ for all $i=1,2,\ldots,m,$ and with a fixed
$\lambda\in(0,2],$ such that $\lambda_{k}=\lambda,$ for all $k\geq 0,$ Eq.
(62) describes the iterative process in (54) with the operator in (38). The
following theorem yields the weak convergence of sequences generated by the
iterative process (62).

###### Theorem 27

Let $w_{ij}\geq 0$ with $\sum_{i=1}^{m}w_{ij}=1$, $i\in I$, $j\in J$, and
$\lambda\in(0,2)$. If $T_{i}:\mathcal{H}\rightarrow\mathcal{H}$ are CW-cutters
and CW-WR operators having a common fixed point then any sequence generated by
the iterative process

$x_{j}^{k+1}=x_{j}^{k}+\lambda\left(\sum_{i=1}^{m}w_{ij}\left(\left(T_{i}(x^{k})\right)_{j}-x_{j}^{k}\right)\right),\text{
for }j=1,2,\ldots,n\text{,}$ (63)

where $x^{0}\in\mathcal{H}$ is arbitrary, converges weakly to some point
$x^{\ast}\in\operatorname*{Fix}T=\bigcap_{j\in J}\operatorname*{Fix}{}^{j}T$.

Proof. Iteration (63) can be written as $x_{j}^{k+1}=(T(x^{k}))_{j}$, where
$T$ is defined by (38) with $\lambda_{j}=\lambda$. By Fact 20, $T$ is $r$-CW-
SQNE, where $r=\frac{{\textstyle 2-\lambda}}{{\textstyle\lambda}}{\textstyle
e}$ with $e:=(1,1,...,1)\in\mathbb{R}^{n}$. By Fact 25 and [9, Corollary
5.3(i)], $T$ is CW-WR. Thus, the theorem follows from Theorem 26.

It is, of course, possible to replace the constant relaxation parameter
$\lambda$ in (63) with $\lambda_{k}\in[\varepsilon,2-\varepsilon]$ for some
positive $\varepsilon$ and the proof will remain true. Therefore, since the
operator $T$ in the above proof is a CW-WR operator, and so are also
orthogonal projections, Theorem 27 is a generalization of the convergence
theorem of the DROP method, [10, Theorem 2.3].

Acknowledgement. We thank Eliahu Levy and Daniel Reem for some fruitful
discussions on this topic. The work of Yair Censor is supported by the ISF-
NSFC joint research program Grant No. 2874/19.

## References

  * [1] A. Auslender, Optimisation: Méthodes Numériques, Masson, Paris, France, 1976.
  * [2] H.H. Bauschke and J.M. Borwein, On projection algorithms for solving convex feasibility problems, SIAM Review 38 (1996), 367–426.
  * [3] H.H. Bauschke and P.L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Space, Springer International Publishing AG, 2nd edition, 2017.
  * [4] M. Benzi, Gianfranco Cimmino’s contributions to numerical mathematics, Seminario di Analisi Matematica, Dipartimento di Matematica dell’Università di Bologna, Ciclo di Conferenze in Ricordo di Gianfranco Cimmino, Marzo-Maggio 2004. Tecnoprint, Bologna, Italy (2005), pp. 87–109.
  * [5] V. Berinde, Iterative Approximation of Fixed Points, Springer-Verlag, Berlin, Heidelberg, Germany, 2nd revised and enlarged edition, 2007.
  * [6] D.P. Bertsekas and J.N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, Prentice-Hall, Inc., Englewood Cliffs, NJ, USA (1989).
  * [7] C. Byrne, Iterative oblique projection onto convex sets and the split feasibility problem, Inverse Problems 18 (2002), 441–453.
  * [8] A. Cegielski, Iterative Methods for Fixed Point Problems in Hilbert Spaces, Springer-Verlag, Berlin, Heidelberg, Germany (2012).
  * [9] A. Cegielski, S. Reich and R. Zalas, Regular sequences of quasi-nonexpansive operators and their applications, SIAM Journal on Optimization 28 (2018), 1508–1532.
  * [10] Y. Censor, T. Elfving, G. T. Herman and T. Nikazad, On diagonally relaxed orthogonal projection methods, SIAM Journal Scientific Computing 30 (2008), 473–504.
  * [11] G. Cimmino, Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari, La Ricerca Scientifica, XVI, Series II, Anno IX, 1 (1938), 326–333.
  * [12] P.L. Combettes and J.-C. Pesquet, Linear convergence of stochastic block-coordinate fixed point algorithms, 2018 26th European Signal Processing Conference (EUSIPCO), 2018, pp. 742–746, doi: 10.23919/EUSIPCO.2018.8552941.
  * [13] Z. Opial, Weak convergence of the sequence of successive approximations for nonexpansive mappings, Bulletin of The American Mathematical Society 73 (1967), 591–597.
  * [14] Pierra, G. Decomposition through formalization in a product space, Mathematical Programming 28, 96–115 (1984).