# Bounded linear operators in PN-spaces

Delavar Varasteh Tafti1, Mahdi Azhini2

( Department of Mathematics, Science and Research Branch, Islamic Azad
University, Tehran, Iran.)

###### Abstract

The manner in which the strong completeness of probabilistic Banach spaces is
frequently exploited depends on the Baire theorem about strong complete
probabilistic metric spaces. By using the Baire theorem, in this paper, we
present the open mapping, closed Graph, Principle of Uniform Boundedness and
Banach-Steinhaus theorems in PN-spaces.

Key words: $\check{S}$erstnev probabilistic normed spaces, bounded linear
operators, the open mapping theorem.
Mathematics Subject Classification (2010): 46S50, 54E70. ††1 Email:
<EMAIL_ADDRESS>†† 2 Corresponding author: Mahdi Azhini.
<EMAIL_ADDRESS>
## 1\. Introduction

Probabilistic Normed (briefly PN) spaces were first introduced by
$\check{S}$erstnev in a series of papers [1-4]. Then a new definition was
proposed by Alsina, Schweizer and Sklar [2]. Linear operators in probabilistic
normed spaces were first studied by Lafuerza-Guill$\acute{e}$n, Rodrignez-
Lallena, and Sempi in [5]. In section 2, we recall some notations and
definitions of probabilistic normed spaces according to those of [2] and [8].
In section 3, we present another way of formulating a PN-space. In section 4,
by using this formulating, we prove all the known classical theorems of
functional analysis, such as Open Mapping and Closed Graph and Principle of
Uniform Boundedness and Banach-Steinhaus theorems in PN-spaces.

## 2\. Preliminaries

### Definition 2.1.

A distribution function (briefly a d.f.) is a function
$F:\bar{\mbox{$\mathbb{R}$}}\longrightarrow[0,1]$ that is nondecreasing and
left-continuous on $\mathbb{R}$, Moreover, $F(-\infty)=0$ and $F(+\infty)=1$.
Here $\bar{\mbox{$\mathbb{R}$}}=\mbox{$\mathbb{R}$}\cup\\{-\infty,+\infty\\}$.
The set of all the d.f.’s will be denoted by $\Delta$ and the subset of those
d.f.’s called distance d.f.’s, such that $F(0)=0$, by $\Delta^{+}$. We shall
also consider $\mathcal{D}$ and $\mathcal{D}^{+}$, the subsets of $\Delta$ and
$\Delta^{+}$, respectively, formed by the proper d.f.’s, i.e., by those d.f.’s
$F\in\Delta$ that satisfy the conditions

$\lim_{x\longrightarrow-\infty}F(x)=0\quad\text{and}\quad\lim_{x\longrightarrow+\infty}F(x)=1.$

The first of these is obviously satisfied in all of $\Delta^{+}$, since, in
it, $F(0)=0$. By setting $F\leq G$ whenever $F(x)\leq G(x)$ for every
$x\in\mbox{$\mathbb{R}$}$, one introduces in natural ordering in $\Delta$ and
in $\Delta^{+}$. The maximal element for $\Delta^{+}$ in this order in the
d.f., given by

$H_{a}(x)=\begin{cases}0&\text{if }\quad x\leq a,\\\ 1&\text{if}\quad
x>a\end{cases}$

where $a\in\mbox{$\mathbb{R}$}$.

The space $\Delta$ can be metrized in several way [10], [12], [14], but we
shall here adopt the modified Levy metric $d_{L}$. If $F$ and $G$ are d.f.’s
and $h$ is in $(0,1]$, let $(F,G;h)$ denote the condition

$F(x-h)-h\leq G(x)\leq F(x+h)+h\quad\text{for
all}~{}x\in(-\frac{1}{h},\frac{1}{h}).$

Then the modified Levy metric $d_{L}$ is defined by

$d_{L}(F,G)=\inf\\{h\in(0,1]:\text{both}(F,G;h)\text{and}(G,F;h)\text{hold}\\},$

under which a sequence of distribution functions
$\\{F_{n}\\}_{n\in\mbox{$\mathbb{N}$}}$ converges to $F\in\Delta^{+}$ if and
only if at each point $t\in\mbox{$\mathbb{R}$}$, where $F$ is continuous,
$F_{n}(t)\longrightarrow F(t)$ [7].

### Definition 2.2.

A triangle function is a binary operation on $\Delta^{+}$, namely a function
$\tau:\Delta^{+}\times\Delta^{+}\longrightarrow\Delta^{+}$ that is
associative, commutative, nondecreasing and which has $H_{0}$ as unit, that
is, for all $F,G,H\in\Delta^{+}$, we have
$\tau(\tau(F,G),H)=\tau(F,\tau(G,H)),$
$\tau(F,G)=\tau(G,F),$
$\tau(F,H)\leq\tau(G,H),\quad\text{if}\;F\leq G,$
$\tau(F,H_{0})=F.$
Typical continuous triangle function are convolution and the operators
$\tau_{T}$ and $\tau_{T^{*}}$, which are, respectively, given by
$\tau_{T}(F,G)(x)=\sup\\{T(F(s),G(t))|s+t=x\\}\quad\text{and}$
$\tau_{T^{*}}(F,G)(x)=\inf\\{T^{*}(F(s),G(t))|s+t=x\\}$
for all $F,G$ in $\Delta^{+}$ and all $x\in\mbox{$\mathbb{R}$}$ [7], here $T$
is a continuous $t$-norm, i.e, a continuous binary operator on $[0,1]$ that is
associative, commutative, nondecreasing and has 1 as identity; $T^{*}$ is a
continuous $t$-conorm, namely a continuous binary operation on $[0,1]$ that is
related to continuous $t$-norm through

$T^{*}(x,y)=1-T(1-x,1-y).$

The most important $t$-norms are the functions $W,\Pi$ and $M$ which are
defined, respectively by

$\begin{array}[]{l}W(a,b)=\max\\{a+b-1,0\\},\\\ \Pi(a,b)=a.b,\\\
M(a,b)=\min\\{a,b\\}.\end{array}$

Their corresponding $t$-conorms are given, respectively by

$\begin{array}[]{l}W^{*}(a,b)=\min\\{a+b,1\\},\\\ \Pi^{*}(a,b)=a+b-ab,\\\
M^{*}(a,b)=\max\\{a,b\\}.\end{array}$

It follows without difficulty that

$\tau_{T}(H_{a},H_{b})=H_{a+b}=\tau_{T^{*}}(H_{a},H_{b}),$

for any continuous $t$-norm $T$, any continuous $t$-conorm $T^{*}$ and any
$a,b\geq 0$.

### Definition 2.3.

[5] A $\check{S}$erstnev PN-space or a $\check{S}$erstnev space is a triple
$(V,\nu,\tau)$, where $V$ is a (real or complex) vector space, $\nu$ is a
mapping from $V$ into $\Delta^{+}$ and $\tau$ is a continuous triangle
function and the following conditions are satisfied for all $p$ and $q$ in
$V$:
$(N1)\ \nu_{p}=H_{0},\;\text{if and only if,}~{}p=\theta(\theta~{}\text{is the
null vector in}V);$
$(N3)\ \nu_{p+q}\geq\tau(\nu_{p},\nu_{q});$
$(\check{S})$ $\ \forall\alpha\in\mbox{$\mathbb{R}$}\backslash\\{0\\},\forall
x\in\bar{\mbox{$\mathbb{R}$}}_{+},\nu_{\alpha
p}(x)=\nu_{p}(\frac{x}{|\alpha|}).$
$\ \text{Notice that condition}(\check{S})\text{implies}$
$(N2)\ \forall p\in V,\nu_{-p}=\nu_{p}.$

### Definition 2.4.

[5] A PN-space is a quadruple $(V,\nu,\tau,\tau^{*})$, where $V$ is a real
vector space, $\tau$ and $\tau^{*}$ are continuous triangle functions such
that $\tau\leq\tau^{*}$, and the mapping $\nu:V\longrightarrow\Delta^{+}$
satisfies, for all $p$ and $q$ in $V$, the conditions:
$(N1)\ \nu_{p}=H_{0},\;\text{if and only if,}p=\theta(\theta\text{ is the null
vector in V)};$
$(N2)\ \forall p\in V,\;\nu_{p}=\nu_{p};$
$(N3)\ \nu_{p+q}\geq\tau(\nu_{p},\nu_{q});$
$(N4)\ \forall\alpha\in[0,1],\nu_{p}\leq\tau^{*}(\nu_{\alpha
p},\nu_{(1-\alpha)p}).$
The function $\nu$ is called the probabilistic norm.

If $\nu$ satisfies the condition, weaker than $(N1),\nu_{\theta}=H_{0}$, then
$(V,\nu,\tau,\tau^{*})$ is called a Probabilistic Pseudo-Normed space
(briefly, a PPN space).

If $\nu$ satisfies the condition $(N1)$ and $(N2)$, then
$(V,\nu,\tau,\tau^{*})$ is said to be a Probabilistic seminormed space
(briefly, PSN space).

If $\tau=\tau_{T}$ and $\tau^{*}=\tau_{T^{*}}$ for some continuous $t$-norm
$T$ and its $t$-conorm $T^{*}$, then $(V,\nu,\tau_{T},\tau_{T^{*}})$ is
denoted by $(V,\nu,T)$ and is called a Menger space.

### Definition 2.5.

[5] There is a natural topology on a PN-space $(V,\nu,\tau,\tau^{*})$, called
the strong topology, which is defined by the system of neighborhoods

$N_{p}(t)=\\{q\in V|~{}\nu_{p-q}(t)>1-t\\}=\\{q\in
V|d_{L}(\nu_{p-q},H_{0})<t\\},\text{where}~{}p\in V\text{and}~{}t>0.$

Let $(V,\nu,\tau,\tau^{*})$ be a PN-space, then

  * i)

A sequence $\\{p_{n}\\}$ in $V$ is said to be strongly convergent to a point
$p$ in $V$, and we write $p_{n}\longrightarrow p$, if for each $t>0$, there
exist a positive integer $m$ such that $p_{n}\in N_{p}(t)$, for $n\geq m$.

  * ii)

A sequence $\\{p_{n}\\}$ in $V$ is called a strong cauchy sequence if for
every $t>0$, there is a positive integer $N$ such that
$\nu_{p_{n}-p_{m}}(t)>1-t$, whenever $m,n>N$.

  * iii)

The PN-space $(V,\nu,\tau,\tau^{*})$ is said to be distributionally compact
($D$-compact) if every sequence $\\{p_{n}\\}$ in $V$ has a convergent
subsequence $\\{p_{n_{k}}\\}$. A subset $A$ of a PN-space
$(V,\nu,\tau,\tau^{*})$ is said to be $D$-compact if every sequence
$\\{p_{m}\\}$ in $A$ has a subsequence $\\{p_{m_{k}}\\}$ that converges to a
point $p\in A$.

  * iv)

In the strong topology, the closure $\overline{N_{p}(t)}$ of $N_{p}(t)$ is
defined by $\overline{N_{p}(t)}=N_{p}(t)\cup N^{\prime}_{p}(t)$, where
$N^{\prime}_{p}(t)$ is the set of limit points of all convergent sequence in
$N_{p}(t)$.

In this paper, we will consider those PN-space for which
$\tau=\tau_{M}=\tau^{*}$, which are clearly $\check{S}$erstnev as well as
Menger PN-space also, and will denote such spaces simply by the pair
$(V,\nu)$.

## 3\. Bounded linear operators in SPN-spaces

### Definition 3.1.

Let $\nabla$ be the set of all nondecreasing and right continuous functions
$f:[0,1]\longrightarrow\bar{\mbox{$\mathbb{R}$}}$, and $\nabla^{+}$ the subset
consisting of all non-negative $f\in\nabla$. Suppose $\mathcal{R}^{+}$in the
set of all $f\in\nabla^{+}$ with $f(w)<+\infty$, for all $w\in(0,1)$. For
$F\in\Delta$, define $\hat{F}:[0,1]\longrightarrow\mbox{$\mathbb{R}$}$ by

$\hat{F}(w)=\sup\\{t\in\mbox{$\mathbb{R}$}|F(t)<w\\}.$

### Theorem 3.2.

[7] If $F\in\mathcal{D}^{+}$, then $\hat{F}\in\mathcal{R}^{+}$. Moreover, for
$F$ and $G$ in $\Delta^{+}$,

  * i)

if $F\leq G$, then $\hat{F}\geq\hat{G}$;

  * ii)

if $\hat{F}\geq\hat{G}$, then $F(t)\leq G(t+h)$ for all
$t\in\mbox{$\mathbb{R}$}$ and $h>0$;

  * iii)

$\widehat{(\tau_{M}(F,G))}=\hat{F}+\hat{G}$;

  * iv)

$\widehat{F(\frac{t}{h}))}=h\hat{F}(t),\text{for all
}t\in\mbox{$\mathbb{R}$}~{}\text{and}~{}h>0.$

### Theorem 3.3.

[7] The map $\wedge:\Delta^{+}\longrightarrow\nabla^{+}$ is one-to-one.

### Definition 3.4.

Let $(V,\nu)$ be a PN-space. The map $V\longrightarrow\mathcal{R}^{+}$ given
through the composition of maps
$V\overset{\nu}{\longrightarrow}\Delta^{+}\overset{\widehat{}}{\longrightarrow}\mathcal{R}^{+}$
will be denoted by $\|\cdot\|$, and for $x\in V$, the value of $\|x\|$ at
$w\in(0,1)$, simply by $\|x\|_{w}$, i.e, $\|x\|_{w}=(\widehat{\nu_{x}})(w)$.
The proof of the following Theorem is obtained, using the definition of PN-
space, Theorem 3.2.

### Theorem 3.5.

Suppose $(V,\nu)$ is a PN-space. The map
$\|\cdot\|:V\longrightarrow\nabla^{+}$ has the following properties:

  * i)

For $x\in V$, $\|x\|\geq 0$, and $\|x\|=0$ if and only if $x=0$.

  * ii)

For $x\in V$ and $\alpha\in\mbox{$\mathbb{R}$}$, $\|\alpha x\|=|\alpha|\|x\|.$

  * iii)

For $x$ and $y$ in $V$, $\|x+y\|\leq\|x\|+\|y\|.$

### Remark 3.6.

Note that all assertion of theorem 3.5 are to be understood point-wise. Thus
for every $w\in(0,1)$, the map
$\|\cdot\|_{w}:V\longrightarrow\mbox{$\mathbb{R}$}$ is a norm on $V$.

### Theorem 3.7.

[12] Let $(V,\nu)$ be a PN-space. For $p\in V,r>0$, and $w\in(0,1)$ let
$B_{w}(p;r)$ be defined by $B_{w}(p;r)=\\{x\in V|\|x-p\|_{w}<r\\}$. Then the
family $\\{B_{w}(p;r)|p\in V,r>0,w\in(0,1)\\}$ forms a basis for the strong
topology on $V$.

### Theorem 3.8.

[6] A PN-space $(V,\nu)$ is a topological vector space if and only if
$\|x\|\in\mathcal{R}^{+}$ for every $x\in V$. Moreover it is seen this
topology is locally convex.
Now, we denote the strong topology of PN-space $(V,\nu)$ by $\tau$, and the
corresponding dual space by $V^{*}$. Hence $f\in V^{*}$ is a continuous
(equivalently bounded because of PN-space $(V,\nu)$ (see [15])) linear
functional on $V$.

### Theorem 3.9.

[12] Let $f:V\longrightarrow\mbox{$\mathbb{R}$}$ be linear. Then $f\in V^{*}$
if and only if there is some $w\in(0,1)$, with

$\sup_{x\in B_{V,1-w}}|f(x)|<+\infty,\text{where }B_{V,1-w}=\\{x\in
V|\|x\|_{w}\leq 1\\}.$

Moreover if $\|f\|:[0,1)\longrightarrow[0,+\infty)$ is defined by
$\|f\|_{w}=\inf_{w<w^{\prime}}(\sup_{x\in B}|f(x)|)$, then $\|f\|$ is a
nondecreasing and right-continuous function, and

$|f(x)|\leq\|f\|_{w}\|x\|_{w^{\prime}},\text{for all}~{}x\in
V\text{and}~{}w\in(0,1).$

By theorem 3.9, a linear functional $f:V\longrightarrow\mbox{$\mathbb{R}$}$
belongs to $V^{*}$ if and only if $\|f\|_{w}<+\infty$ for some $w\in(0,1)$.
The following theorem introduces a simple way of constructing PN-space. We
recall that for a nondecreasing function
$f:(0,1)\longrightarrow\mbox{$\mathbb{R}$}$, the function
$l^{-}f:(0,1)\longrightarrow\mbox{$\mathbb{R}$}$ is defined by

$l^{-}f(w_{0})=\lim_{w\longrightarrow w_{0}^{-}}f(w)=\sup_{w<w_{0}}f(w).$

Clearly $l^{-}f$ is nondecreasing and left-continuous. We also denote the
Lebesgue measure on $(0,1)$ by $m$.

### Theorem 3.10 [13]

Let $V$ be a real vector space. Suppose
$p:V\times(0,1)\longrightarrow[0,+\infty)$, satisfies the following
conditions:

  * i)

for every $x\in V$, the function $P(x,.):(0,1)\longrightarrow[0,+\infty)$ is
nondecreasing,

  * ii)

for every $w\in(0,1)$, the map $p(.,w):V\longrightarrow[0,+\infty)$ is a semi-
norm on $V$,

  * iii)

the family of semi-norms $\\{p(.,w)|w\in(0,1)\\}$ is separating on $V$.

then, there exists a unique map $\nu:V\longrightarrow\Delta^{+}$, defined by

$\nu_{x}(t)=m(\\{w\in(0,1)|p(x,w)<t\\}),$

such that $(V,\nu)$ is a PN-space. Moreover, $\|x\|_{w}=l^{-}p(x,w)$ for all
$x\in V$ and $w\in(0,1)$.

### Theorem 3.11. [13]

Let $V$ be a real vector space. Suppose
$p:V\times(0,1)\longrightarrow[0,+\infty)$ is a map which satisfies conditions
of theorem 3.10, with $\nu:V\longrightarrow\Delta^{+}$ the corresponding
probabilistic norm on $V$. Then the strong topology on $V$ is induced by the
separating family of semi-norm $\\{p(.,w)|w\in(0,1)\cap Q\\}$. hence $V$ under
the strong topology is a locally topological vector space.

### Remark 3.12.

Let $(V,\nu)$ be a PN-space, and $p:V\times(0,1)\longrightarrow[0,+\infty)$ be
given by $p(x,w)=\|x\|_{w}=\hat{\nu}_{x}(t)$. Then by Remak 3.6 and Theorem
3.11, the family

$\\{B_{V,w}(0;r)|w\in(0,1),r>0\\},$

where $B_{V,w}(0;r)$ is given by $\\{x\in V,\|x-x_{0}\|_{w}<r\\}$, forms a
local basis for the strong topology. Moreover, a subset $E$ of $V$ is bounded
if and only if the map $p(.,w):V\longrightarrow[0,+\infty)$ is bounded on $E$,
for every $w\in(0,1)$ (see [16] ).

Using theorem 3.10, we may construct many examples of PN-spaces.

### Example 3.13.

Let $(V,\|\cdot\|)$ be a real normed vector space. If we define
$p:V\times(0,1)\longrightarrow[0,+\infty)$ by

$p(x,w)=\|x\|\quad\forall x\in V,\quad\forall w\in(0,1),$

then $p$ satisfies all condition of theorem 3.10. Using
$\nu_{x}(t)=m(\\{w\in(0,1)|p(x,w)<t\\})$, we obtain $\nu_{x}=H_{\|x\|}$.

### Example 3.14.

Let $(V,\nu)$ and $(W,\mu)$ be two PN-spaces. Then, the map $\theta:V\times
W\longrightarrow\Delta^{+}$ defined by

$\theta(x,y)=\tau_{M}(\nu_{x},\nu_{y}),$

is a probabilistic norm on $V\times W$, i.e., $(V\times W,\theta)$ is a PN-
space. Because define $p:V\times W\times(0,1)\longrightarrow[0,+\infty)$ by

$p(x,y,w)=\|x\|_{V,w}+\|y\|_{W,w}$

where $\|\cdot\|_{V}:V\longrightarrow\mathcal{R}^{+}$ and
$\|\cdot\|_{W}:W\longrightarrow\mathcal{R}^{+}$ are the composite maps given
by Definition 3.4. Then the desired is obtained by Theorem 3.10 and Theorem
3.2 (iii).

### Definition 3.15.

Let $V$ and $W$ be two PN-spaces. We remind that, for topological vector space
$V$ and $W$, a linear operator $T:V\longrightarrow W$ is called bounded if
$T(E)$ is bounded (in $W$) whenever $E$ is bounded subset of $V$.
Metrisability of the PN-spaces implies that $T$ is bounded if and only if $T$
is continuous (see [16] ). As usual, the set of all bounded linear operators
from $V$ to $W$ is denoted by $B(V,W)$.

### Definition 3.16.

Let $V$ and $W$ be two PN-spaces. For linear operator $T:V\longrightarrow W$
and a pair $(w,w^{\prime})\in
J=\\{(w,w^{\prime})|w,w^{\prime}\in(0,1)\\}\subseteq\mbox{$\mathbb{R}$}^{2}$,
the norm $\|T\|_{(w,w^{\prime})}$ is defined by

$\|T\|_{(w,w^{\prime})}=\sup\\{\|Tx\|_{w^{\prime}}|~{}x\in
B_{V,w}\\},\text{where}~{}B_{V,w}=\\{x\in V|\|x\|_{w}\leq 1\\}.$

### Theorem 3.17.

[13] For a pair $(w,w^{\prime})\in J$, if $\|T\|_{(w,w^{\prime})}<+\infty$,
then

$\|Tx\|_{w^{\prime}}\leq\|T\|_{(w,w^{\prime})}\|x\|_{w},\text{for all}~{}x\in
V.$

### Theorem 3.18.

[13] Let $V$ and $W$ be two PN-spaces, and $T:V\longrightarrow W$ be a linear
operator. Then $T\in B(V,W)$ if and only if, for every $w^{\prime}\in(0,1)$
there exists $w\in(0,1)$ such that $\|T\|_{(w,w^{\prime})}<+\infty$.

## 4\. Main Results

Baire theorem is one of the two keystones of Linear Functional Analysis, the
other one being the Hahn-Banach theorem. Bair’s theorem for reaching
consequences include such basic theorems as the Banach open mapping theorem,
the Banach closed graph theorem and the Banach-Steinhaus theorem. This chapter
concludes consequences of Bair’s theorem in PN-spaces.

It is impossible to speak about PN-spaces without making reference to concept
of a Probabilistic Metric space (briefly a PM space). PM spaces are studied in
depth in the fundamental book [7] by Schweizer and Sklar.

### Definition 4.1.

A triple $(V,\mbox{$\mathcal{F}$},\tau)$, where $V$ is a nonempty set and
$\mathcal{F}$ is a function from $V\times V$ into $\Delta^{+}$ and $\tau$ is a
triangle function, is a Probabilistic Metric space if the following conditions
are satisfied for all points $p,q$ and $r$ in $V$:
$(PM1)\ {\mbox{$\mathcal{F}$}}(p,p)=H_{0};$
$(PM2)\ {\mbox{$\mathcal{F}$}}(p,q)\neq H_{0}~{}\text{if}~{}p\neq q;$
$(PM3)\ {\mbox{$\mathcal{F}$}}(p,q)={\mbox{$\mathcal{F}$}}(q,p);$
$(PM4)\
{\mbox{$\mathcal{F}$}}(p,r)\geq\tau({\mbox{$\mathcal{F}$}}(p,q),{\mbox{$\mathcal{F}$}}(q,r)).$
Notice that, given PN space $(V,\nu)$, a probabilistic metric
${\mbox{$\mathcal{F}$}}:V\times V\longrightarrow\Delta^{+}$ is defined via
${\mbox{$\mathcal{F}$}}(p,q)=\nu_{p-q}$, so that the triple
$(V,{\mbox{$\mathcal{F}$}},\tau)$ becomes a PM space.

### Theorem 4.2 (Baire Theorem in PM spaces).[11]

Let $(V,{\mbox{$\mathcal{F}$}},\tau)$ be a strong complete PM space under a
continuous triangle function $\tau$. If $\\{G_{n}\\}$ is a sequence of dense
and strongly open subset of $V$, then $\cap_{n=1}^{\infty}G_{n}$ is not empty.
(In fact it is dense in $V$).

### Corollary 4.3.

Let $(V,{\mbox{$\mathcal{F}$}},\tau)$ be a strong complete PM space under a
continuous triangle function $\tau$. Then the following two equivalent
properties hold:

  * a)

Let $\\{f_{n}\\}_{n=0}^{\infty}$ be a sequence of strongly closed subsets of
$V$ such that $\text{int}F_{n}=\phi$ for all $n\geq 0$. Then
$\text{int}(\cup_{n=0}^{\infty}F_{n})=\phi$.

  * b)

Let $\\{G_{n}\\}_{n=0}^{\infty}$ be a sequence of strongly open subsets of $V$
such that $G_{n}=V$ for all $n\geq 0$. Then
$\overline{\cap_{n=0}^{\infty}G_{n}}=V$.

### Corollary 4.4.

Let $(V,{\mbox{$\mathcal{F}$}},\tau)$ be a PM space under a continuous
triangle function $\tau$, and let $F_{n}$, $n\geq 0$, be strongly closed
subsets of $V$ such that $V=\cup_{n=0}^{\infty}F_{n}$.

  * a)

If $\text{int}F_{n}=\phi$ for all $n\geq 0$, then $V$ is not strong complete.

  * b)

If $V$ is strong complete, there exists $n_{0}\geq 0$ such that
$\text{int}F_{n_{0}}\neq\phi$.

### Theorem 4.5 (Open Mapping theorem in PN-spaces).

If $T$ is a probabilistic bounded linear operator from a strong complete PN-
space $(V,\nu)$ onto a strong complete $(W,\mu)$, then $T$ is an open mapping.

### Proof.

The Theorem will be proved by the following steps:
Step 1: The set $T(B_{V,w}(0;1))$, where $B_{V,w}(0;r)=\\{x\in
V|\|x\|_{w}<r\\}$ for all $r>0$ and $w\in(0,1)$, contains an strong open ball.
Now fix $w\in(0,1)$. Given any $y\in W$, there exists $x\in V$ such that
$y=Tx$, since $T$ is sujective. Since $x\in B_{V,w}(0;n)$ for some integer
$n\geq 1$, this shows that $W=\bigcup_{n=1}^{\infty}T(B_{V,w}(0;n))$, hence
$W=\bigcup_{n=1}^{\infty}\overline{T(B_{V,w}(0;n))}$. The space $W$ being
strong complete, hence Corollary 4.4 (b) shows that there exists an integer
$n_{0}\geq 1$ such that

$\text{int}\overline{T(B_{V,w}(0;n_{0}))}\neq\phi.$

Therefore $\text{int}\overline{T(B_{V,w}(0;1))}\neq\phi$, since
$\overline{T(B_{V,w}(0;1))}=\frac{1}{n_{0}}\overline{T(B_{V,w}(0;n_{0}))}$ by
the linearity of $T$. Hence the set $\overline{T(B_{V,w}(0;1))}$ contains on
strong open ball.
Step 2: The set $\overline{T(B_{V,w}(0;1))}$ contains an strong open ball
centered at the origin of $W$. By Step 1, there exist $y\in W$ and $s>0$ such
that $B_{W,w}(y;2s)\subset\overline{T(B_{V,w}(0;1))},$ and hence

$B_{W,w}(0;2s)=\\{-y\\}+B_{W,w}(y;2s)\subset\\{-y\\}+\overline{T(B_{V,w}(0;1)))}.$

Since $-y\in\overline{T(B_{V,w}(0;1))}$(because
$y\in\overline{T(B_{V,w}(0;1))}$ and $T$ is linear), it follows that

$\\{-y\\}+\overline{T(B_{V,w}(0;1))}\subset 2T(B_{V,w}(0;1).$

Step 3: The set $T(B_{V,w}(0;1))$ contains an strong open ball centered at the
origin of $W$. To prove this assertion, we will show that
$B_{W,w}(0;\frac{s}{2})\subset T(B_{V,w}(0;1))$, where $s>0$ is the radius of
the ball $B_{W,w}(0;s)$ found in Step 2. This means that, given any $y\in
B_{V,w}(0;\frac{s}{2})$, we need to find $x\in B_{V,w}(0;1)$ such that $y=Tx$.
So, let $y\in B_{W,w}(0;\frac{s}{2})$ be given. Since $y\in
B_{W,w}(0;\frac{s}{2})\subset\overline{T(B_{V,w}(0;\frac{1}{2}))}$ by Step 2,
there exists $x_{1}\in B_{V,w}(0;\frac{1}{2})$ such that
$\|y-Tx_{1}\|_{w}<\frac{s}{2^{2}}$, since $y-Tx_{1}\in
B_{W,w}(0;\frac{s}{2^{2}})\subset\overline{T(B_{V,w}(0;\frac{1}{2^{2}}))},$
again by Step 2, thee exists $x_{2}\in B_{V,w}(0;\frac{1}{2^{2}})$ such that
$\|y-Tx_{1}-Tx_{2}\|_{w}<\frac{s}{2^{3}}$, and so on. In this manner we
construct a sequence $\\{x_{n}\\}_{n=1}^{\infty}$ of points $x_{n}\in V$ with
the following properties: $x_{n}\in B_{V,w}(0;\frac{1}{2^{n}})$ and
$\|y-T(\sum_{k=1}^{n}x_{k})\|_{w}<\frac{s}{2^{n+1}}$, for all $n\geq 1$ .
Since the series $\sum_{n=1}^{\infty}x_{n}$ is absolutely convergent (because
$\|x_{n}\|_{w}<\frac{1}{2^{n}}$, for all $n\geq 1$) and the space $V$ is
strong complete, the series $\sum_{n=1}^{\infty}x_{n}$ converges to a point
$x\in V$ and

$\|x\|_{w}\leq\sum_{n=1}^{\infty}\|x_{n}\|_{w}<\frac{1}{2}+\frac{1}{2^{2}}+\cdots+\frac{1}{2^{n+1}}+\cdots=1,$

by Theorem 3.5, so that $x\in B_{V,w}(0;1)$. Furthermore,
$y=\lim_{n\longrightarrow\infty}T(\sum_{k=1}^{n}x_{k})=Tx,$ since $T$ is
continuous. Hence the assertion is proved.
Step 4: The mapping $T$ is open strongly. Given any strong open subset $U$ of
$V$ and give any $y\in T(U)$, we must find $\delta>0$ such that
$B_{W,w}(y;\delta)\subseteq T(U)$. So let $x\in U$ be such that $y=Tx$. Since
$U$ is strong open, thee exists $r>0$ such that
$B_{V,w}(x;r)=\\{x\\}+B_{V,w}(0;r)\subseteq U$, and by Step 3, there exists
$\delta>0$ such that $B_{W,w}(0;\delta)\subseteq T(B_{V,w}(0;r))$. Hence

$\displaystyle B_{W,w}(y;\delta)=\\{y\\}+B_{W,w}(0;\delta)$
$\displaystyle\subseteq\\{y\\}+T(B_{V,w}(0;r))$
$\displaystyle=T(\\{x\\}+B_{V,w}(0;r))$ $\displaystyle\subseteq
T(U).\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\Box$

### Corollary 4.6.

Let $(V,u)$ and $(W,\mu)$ be two strong complete PN-spaces and let $T\in
B(V,W)$ be bijective. Then $T^{-1}\in B(W,V)$.

### Proof.

First fix $w\in(0;1)$. It is clear that $T^{-1}:W\longrightarrow V$ is linear,
since $T\in B(V,W)$, hence by Definition 3.15, suffices to show that
$T^{-1}:W\longrightarrow V$ is continuous at the origin, i.e, that given any
strong open ball $B_{V,w}(0;\epsilon)\subseteq V$, there exists an strong open
ball $B_{V,w}(0;\delta)\subseteq W$, such that
$T^{-1}(B_{W,w}(0;\delta))\subseteq B_{V,w}(0;\delta)$, by definition of
continuouty at a point. But the inclusion $T^{-1}(B_{W,w}(0;\delta))\subseteq
B_{V,w}(0;\epsilon)$, is equivalent to the inclusion
$B_{W,w}(0;\delta)\subseteq T(B_{V,w}(0;\epsilon))$, since the mapping
$T:V\longrightarrow W$ is bijective, and open Mapping Theorem precisely show
that this last inclusion hold for some $\delta>0$. $\Box$

### Theorem 4.7.

Let $\|\cdot\|_{w}$ and $\|\cdot\|^{\prime}_{w}$, for all $w\in(0,1)$, be two
norms on the same vector space $V$, with the following properties: both
$(V,\|\cdot\|_{w})$ and $(V,\|\cdot\|^{\prime}_{w})$, are strong complete PN-
spaces, and there exists a constant $C$ such that $\|x\|^{\prime}_{w}\leq
C\|x\|_{w}$ for all $x\in V$. Then there exists $w^{\prime}\in(0,1)$ and
$c_{w^{\prime}}\in\mbox{$\mathbb{R}$}$ such that $\|x\|_{w^{\prime}}\leq
c_{w^{\prime}}\|x\|^{\prime}_{w}$, for all $x\in V$.

### Proof.

Fix $w\in(0,1)$. The bijective and linear identity mapping
$i:(V,\|\cdot\|_{w})\longrightarrow(V,\|\cdot\|^{\prime}_{w})$ belongs to
$B(V,V)$ by assumption. Corollary 4.6, therefore shows that the inverse
mapping $i^{-1}:(V,\|\cdot\|^{\prime}_{w})\longrightarrow(V,\|\cdot\|_{w})$,
belongs to $B(V,V)$, hence by Theorems 3.18 and 3.17, there exists
$w^{\prime}\in(0,1)$ and $c_{w^{\prime}}$ such that $\|x\|_{w^{\prime}}\leq
c_{w^{\prime}}\|x\|^{\prime}_{w}$, for all $x\in V$. $\Box$

### Definition 4.8.

Let $(V,\nu)$ and $(W,\mu)$ be two PN-spaces. The graph $GrT$ of a mapping
$T:V\longrightarrow W$ is the subset of the product $V\times W$ defined by

$GrT=\\{(x,Tx)\in V\times W;x\in V\\}.$

Let $(V,\nu)$ and $(W,\mu)$ be two PN-spaces. By Example 3.14 $(V\times
W,\theta)$ is a PN-space, where $\theta(x,y)=\tau_{M}(\nu_{x},\mu_{y})$, for
all $x\in V$ and $y\in W$, is a probabilistic norm on $V\times W$.

### Definition 4.9.

If $(V,\nu)$ and $(W,\mu)$ are PN-spaces, a mapping $T:V\longrightarrow W$ is
said to be closed graph, if its graph $GrT$ is strong closed in the PN-space
$V\times W$ (equipped with strong product topology, by Example 3.14 and
Theorem 3.11). Therefore a mapping $T:V\longrightarrow W$ is strongly closed
if and only if $x_{n}\longrightarrow x$ in $V$ and $Tx_{n}\longrightarrow y$
in $W$ implies $y=Tx$.

### Theorem 4.10 (Closed Graph Theorem in PN-spaces).

Let $(V,\nu)$ and $(W,\mu)$ be strongly complete PN-spaces, and
$T:V\longrightarrow W$ be a closed linear operator. Then $T\in B(V,W)$.

### Proof.

First fix $w\in(0,1)$ and $w^{\prime}\in(0,1)$. By using Theorem 3.5, we
define another norm on $V$ by

$\|x\|^{\prime}_{V,w}=\|x\|_{V,w}+\|Tx\|_{W,w^{\prime}},$

and $x\in V$. On the other hand by the definition of norm
$\|\cdot\|^{\prime}_{V,w}$, if $\\{x_{n}\\}_{n=1}^{\infty}$ is a Cauchy
sequence with respect to $\|\cdot\|^{\prime}_{V,w}$, then
$\\{x_{n}\\}_{n=1}^{\infty}$ and $\\{Tx_{n}\\}_{n=1}^{\infty}$ are Cauchy
sequences in the PN-spaces $V$ and $W$, respectively. Since both spaces are
strongly complete, there exist $x\in V$ and $y\in W$ such that
$x_{n}\longrightarrow x$ in $V$ and $Tx_{n}\longrightarrow y$ in $W$, and thus
$y=Tx$, since $T$ is strongly closed by assumption. Therefore,

$\|x_{n}-x\|^{\prime}_{V,w}=\|x_{n}-x\|_{V,w}+\|Tx_{n}-Tx\|_{W,w^{\prime}}=\|x_{n}-x\|_{V,w}+\|Tx_{n}-y\|_{W,w^{\prime}}\longrightarrow
0$

as $n\longrightarrow\infty$, which shows that $(V,\|\cdot\|^{\prime}_{w})$ is
also strongly complete. Since $\|x\|_{V,w}\leq\|x\|^{\prime}_{V,w}$, for all
$x\in V$, Theorem 4.7 shows that there exists $w^{\prime\prime}\in(0,1)$ and
$C_{w^{\prime\prime}}$ such that
$\|Tx\|_{W,w^{\prime}}\leq\|x\|_{V,w}+\|Tx\|_{W,w^{\prime}}=\|x\|^{\prime}_{V,w^{\prime\prime}}\leq
C_{w^{\prime\prime}}\|x\|_{V,w}$ for all $x\in V$. Therefore by Definition
3.16 we have $\|T\|_{(w^{\prime\prime},w^{\prime})}<+\infty$, where
$\|T\|_{(w^{\prime\prime},w^{\prime})}=\sup\\{\|Tx\|_{w^{\prime}}|\|x\|_{w^{\prime\prime}}\leq
1\\}$. Hence by Theorem 3.18, $T\in B(V,W)$. $\Box$

### Theorem 4.11 (Uniform boundedness principle theorem in PN-spaces).

Let $(V,\nu)$ be a strongly complete PN-space and let $(W,\mu)$ be a PN-space,
and let $\\{T_{n}\\}_{n\in\mbox{$\mathbb{N}$}}$ be a family of operators
$T_{n}\in B(V,W)$, that satisfy

$\sup\\{\|T_{n}x\|_{w^{\prime}}:T_{n}\in B(V,W)\\}<\infty,$

for all $x\in V$ and $w^{\prime}\in(0,1)$, then there exists $w\in(0,1)$ such
that

$\sup_{n\in\mbox{$\mathbb{N}$}}\|T_{n}\|_{(w,w^{\prime})}<\infty$

### Proof.

First fix $n\in\mbox{$\mathbb{N}$}$ and $w^{\prime}\in(0,1)$, and define the
set

$F_{n}=\\{x\in V|\sup\|T_{n}x\|_{w^{\prime}}\leq n,\;\forall T_{n}\in
B(V,W)\\}.$

Given any $x\in V$,
$\sup_{n\in\mbox{$\mathbb{N}$}}\|T_{n}x\|_{w^{\prime}}<\infty$, by assumption,
hence there exists an integer $n_{x}\geq 0$ such that
$\sup_{n\in\mbox{$\mathbb{N}$}\cup\\{0\\}}\|T_{n}x\|_{w^{\prime}}\leq n_{x}$,
which means that $x\in F_{n_{x}}$, hence $V=\cup_{n=0}^{\infty}F_{n}$. On the
other hand $F_{n}$ is strongly closed subset of $V$. Since $V$ is strongly
complete by Corollary 4.4 (b), there exists an integer $n_{0}\geq 0$ such that
$\text{int}F_{n_{0}}\neq\phi$. Hence there exists $x_{0}\in F_{n_{0}}$ and
$r>0$ such that $\overline{B_{V,w}(x_{0};r)}\subset F_{n_{0}}$, where

$B_{V,w}(x_{0};r)=\\{x\in V|\|x-x_{0}\|_{w}<r\\},\text{for some}~{}w\in(0,1).$

By definition of $F_{n_{0}}$, and Theorem 3.18 for all
$z\in\overline{B_{V,w}(x_{0};r)}$ and for all
$n\in\mbox{$\mathbb{N}$}\cup\\{0\\}$ $\|T_{n}z\|_{w^{\prime}}\leq n_{0}$ Since
any nonzero $x\in V$ can be written as $x=\frac{\|x\|_{w}}{r}(z-x_{0})$ with
$z=(x_{0}+\frac{r}{\|x\|_{w}}x)\in\overline{B_{V,w}(x_{0};r)}.$ By Theorem 3.5
we hae

$\displaystyle\|T_{n}x\|_{w^{\prime}}$
$\displaystyle\leq\frac{\|x\|_{w}}{r}(\|T_{n}z\|_{w^{\prime}}+\|T_{n}x_{0}\|_{w^{\prime}})$
$\displaystyle\leq\frac{1}{r}(n_{0}+\|T_{n}x_{0}\|_{w^{\prime}})\|x\|_{w}$
$\displaystyle\leq\frac{1}{r}(n_{0}+\sup_{n\in\mbox{$\mathbb{N}$}\cup\\{0\\}}\|T_{n}x_{0}\|_{w^{\prime}})\|x\|_{w},$

for all $x\in V$.
Therefore, by definition of $\|T\|_{(w,w^{\prime})}$;
$\sup_{n\in\mbox{$\mathbb{N}$}\cup\\{0\\}}\|T_{n}\|_{(w,w^{\prime})}\leq\frac{1}{r}(n_{0}+\sup_{n\in\mbox{$\mathbb{N}$}\cup\\{0\\}}\|T_{n}x_{0}\|_{w^{\prime}})<\infty$,
since
$\sup_{n\in\mbox{$\mathbb{N}$}\cup\\{0\\}}\|T_{n}x_{0}\|_{w^{\prime}}<\infty$,
by assumption. $\Box$

### Theorem 4.12 (Banach-Steinhaus Theorem in PN-spaces).

Let $(V,\nu)$ be a strongly complete PN-space, let $(W,\mu)$ be a PN-space,
let $\\{T_{n}\\}_{n=1}^{\infty}$ be a family of operators $T_{n}\in B(V,W)$
such that, for all $x\in V$, the sequence $\\{T_{n}x\\}_{n=1}^{\infty}$
strongly converges in $W$, that is $T_{n}x\longrightarrow Tx$, as
$n\longrightarrow\infty$, then $T\in B(V,W).$

### Proof.

The linearity of each operators $T_{n}$, combined with the continuity of the
addition and scalar multiplication shows that the operator $T:V\longrightarrow
W$ defined by $Tx=\lim_{n\longrightarrow\infty}T_{n}x$ for each $x\in V$ in
linear. The convergence of each sequence $\\{T_{n}x\\}_{n=1}^{\infty}$ and
Theorem 3.17 implies that :
$\sup_{n\in\mbox{$\mathbb{N}$}}\|T_{n}x\|_{w^{\prime}}<\infty$, for all $x\in
V$ and $w^{\prime}\in(0,1).$ By Theorem 4.11 there exist $M>0$ and $w\in(0,1)$
such that $\|T_{n}\|_{(w,w^{\prime})}\leq M$, for all
$n\in\mbox{$\mathbb{N}$}$. Now by Theorem 3.17 and Theorem 3.5, for all
$x\in\overline{B_{V,w}(0;1)}$ and $n\in\mbox{$\mathbb{N}$}$ we have

$\displaystyle\|Tx\|_{w^{\prime}}$ $\displaystyle\leq\|Tx-
T_{n}x\|_{w^{\prime}}+\|T_{n}x\|_{w^{\prime}}$ $\displaystyle\leq\|Tx-
T_{n}x\|_{w^{\prime}}+M.$

Hence $\|Tx\|_{w}^{\prime}\leq M$, when $n\longrightarrow\infty$, for all
$x\in\overline{B_{V,w}(0;1)}$ therefore by definition of
$\|T\|_{(w,w^{\prime})}$, we have $T\in B(V,W)$. $\Box$

## References

  * [1] A. N. $\check{S}$erstnev, _Random normed spaces, Questions of completeness_ , Kazan Gos. Univ. Uc$\acute{e}$n. Zap. 122(1962), 3-20.
  * [2] A. N. $\check{S}$erstnev, _On the notion of a random normed space_ , Dokl. Akad. Nauk. SSSR 149(no. 2), 280-283 (Soviet Math. Doklady 4(1963), 388-390).
  * [3] A. N. $\check{S}$erstnev, _Best approximation problems in random normed spaces_ , Dokl. Akad. Nauk. SSSR 149 (1963), 539-542.
  * [4] A. N. $\check{S}$erstnev, _On a probabilistic generalization of metric spaces_ , Kazan Gos. Univ. U$\check{c}$en. Zap. 124 (1964), 3-11.
  * [5] B. Lafuerza Guill$\acute{e}$n, P. Harikrishnan, _Probabilistic Normed Spaces_ , Imperial College Press, London, (2014).
  * [6] B. Lafuerza Guill$\acute{e}$n, J. A. Rod$\acute{r}$iguez Lallena and C. Sempi, _Normability of_ _probabilistic normed spaces_ , Note di Mathematica, 29(2008) 99-111. MR 2779883. Zbl 1213. 54045.
  * [7] B. Schweizer and A. Sklar, _Probabilistic Metric Spaces_ , Spring, North Holland, (1983).
  * [8] B. Schweizer, _Multiplication on the space of probability distribation_ _functions_ : Aequationes Math. 12(1975) 156-183.
  * [9] C. Alsina, B. Schweizer and A. Sklar,_On the definition of probabilistic_ _normed space_ , Aequationes Math. 46(no. 1-2) (1993), 91-98.
  * [10] D. R. Sibley,_A metric for weak convergence of distribation function_ , Rocky Mountain J. Math. 1 (1971) 427-430.
  * [11] D. Varasteh, M. Azhini, _Completeness in Probabilitic Metric Spaces_ ,
  * [12] F. Bahrami, _Some topologies on a $\check{S}$erstnev probabilistic normed_ _space_. J. Math. Anal. Appl. 344 (2006) 857-868.
  * [13] F. Bahrami, M. Nikfar, _The topological structure of a certain Menger space_ , J. Math. Anal. 334 (2007) 172-182.
  * [14] M. D. Taylor, _New metrics for weak convergence of distribuation function_ , Stochastical (1985) 5-17.
  * [15] Philippe, G. Ciarlet, _Linear and Nonlinear Functional Analysis with_ _Applications_ , the Society for Industrial and Applies Mathematic, USA, (2013).
  * [16] W. Rudin, _Functional Analysis_ , Mc Graw-Hill Book Company, New York. 1973.


# Annotation-Free Group Robustness via Loss-Based Resampling

Mahdi Ghaznavi
<EMAIL_ADDRESS>Hesam Asadollahzadeh
<EMAIL_ADDRESS>HamidReza Yaghoubi Araghi
<EMAIL_ADDRESS>Fahimeh HosseiniNoohdani
<EMAIL_ADDRESS>Mohammad Hossein Rohban
<EMAIL_ADDRESS>Mahdieh Soleymani Baghshah
<EMAIL_ADDRESS>Sharif University of Technology
Tehran, Iran

###### Abstract

It is well-known that training neural networks for image classification with
empirical risk minimization (ERM) makes them vulnerable to relying on spurious
attributes instead of causal ones for prediction. Previously, deep feature re-
weighting (DFR) has proposed retraining the last layer of a pre-trained
network on balanced data concerning spurious attributes, making it robust to
spurious correlation. However, spurious attribute annotations are not always
available. In order to provide group robustness without such annotations, we
propose a new method, called loss-based feature re-weighting (LFR), in which
we infer a grouping of the data by evaluating an ERM-pre-trained model on a
small left-out split of the training data. Then, a balanced number of samples
is chosen by selecting high-loss samples from misclassified data points and
low-loss samples from correctly-classified ones. Finally, we retrain the last
layer on the selected balanced groups to make the model robust to spurious
correlation. For a complete assessment, we evaluate LFR on various versions of
Waterbirds and CelebA datasets with different spurious correlations, which is
a novel technique for observing the model’s performance in a wide range of
spuriosity rates. While LFR is extremely fast and straightforward, it
outperforms the previous methods that do not assume group label availability,
as well as the DFR with group annotations provided, in cases of high spurious
correlation in the training data.

##  1 Introduction

Neural networks have the tendency to rely on patterns in input data that are
highly correlated with the target class in image classification. If the
correlations that a model relies on are spurious, i.e. non-causal, the model’s
performance will drop under spurious correlation shift in the test time [21,
10, 18]. In such situations, the model performs well on majority groups of
training data which have the spurious pattern, while the performance is poor
on minority groups [1, 2]. As an example, in Waterbirds [16] dataset, there is
a spurious correlation between the waterbird/landbird label and the water/land
background, which implies waterbirds on water and landbirds on land as the
majority groups and the other two combinations as minority groups. The final
goal is to achieve the best worst group accuracy (WGA) among all groups for
the sake of robustness to correlation shift.

Some earlier methods consider accessing group annotations for the whole data
[16, 7] or a small portion of it [20, 9]. [16] proposes group distributionally
robust optimization (GDRO), which softly optimizes the worst-case loss among
groups during training. [7] suggests that retraining the last layer of a
classification model on a balanced dataset concerning group annotation and
label is sufficient for making the model robust to spurious correlation shift.
Their deep feature re-weighting (DFR) method has achieved good performance
while being simple and fast. Nonetheless, group annotations are not always
available. Some works do not assume such annotation requirement [11, 14, 19,
3, 12], but they need to train or fine-tune the whole model to achieve
robustness.

As stated in [7], models trained with empirical risk minimization (ERM) learn
core (causal) features besides the spurious ones, regardless of how strong the
spurious correlation is. Thus, it seems that all we need is to find the proper
linear combination of features to achieve the desired WGA. Recently, [15]
proposed automatic feature re-weighting (AFR), which re-weights each sample
while training the model’s last layer, based on the model’s probability of
predicting the correct label on a given sample. However, as their weighting
does not consider the size of minority groups, it does not work in situations
where the size of the minority group is too small. Our work is focused on
addressing this issue.

In this work, we propose a new method for robustness to spurious correlation
shifts. First, we infer groups based on the model’s correctness of prediction
on a set of data. We then select a few number of high-loss samples from the
misclassified ones and low-loss samples from the correctly-classified images
in each group. Finally, we apply DFR for feature re-weighting based on the
selected groups. Our method, called loss-based feature re-weighting (LFR) is a
simple yet effective method that achieves state-of-the-art (SOTA) on various
famous datasets with high spurious correlation. The overview of LFR is shown
in Figure 1.

Figure 1: Overview of LFR method. After training a model on unbalanced data,
we evaluate it on a small portion of the dataset (about 20% of the total).
Then, we select images with low loss values in correctly-classified images and
high loss in misclassified images and select an equal number of them in each
class. At last, retraining the last layer of the model on the balanced dataset
improves the model’s robustness.

Using LFR at the end of any training pipeline can improve WGA free of charge
since it is fast, straightforward, and doesn’t assume any inductive bias. LFR
works well on a wide range of high to low spurious correlations by using only
a small subset of training data for last layer retraining. By visualizing the
saliency map of the model before and after last-layer retraining, we observe
that the model’s attention regions shift from spurious to core attributes in
both majority and minority samples.

##  2 Related Work

We can divide the related work in the field by different means.

Availability of group annotations: Some works in the literature have an
assumption of access to group annotations [16, 7]. On the other hand, some
works have tried to make a model robust to group shift without the
availability of group annotations [11, 14, 22, 15].

There are also some works that only need a small amount of group-annotated
data. [20] used these group annotations to train a classifier for predicting
group labels of the remaining data points, which are then fed into GDRO. [9]
utilizes a set of diverse classifiers on a dataset without using group
annotations, and then annotates a few samples for selecting the favorable
model among this set.

Scope of Training: Different methods have different scopes of training, which
leads to different amounts of required computation and effectiveness. Some
works make a model robust during initial training. GDRO [16] solves a two-
level optimization to minimize the maximum loss among all groups. [14]
proposes learning from failure (LfF), which trains two models: a super-biased
model to majority groups, and another model that upweights images with high
loss value in a biased model. Thus, the second model goes in a way to be
robust.
On the other hand, some works fine-tune a biased pre-trained model that had
been trained by ERM. [11] proposes just training twice (JTT), which upweights
misclassified data points during the retraining. [22] uses a contrastive
method to make samples of a class that are correctly or misclassified by a
pre-trained model close together. Both aforementioned methods need validation
data for tuning weighting hyperparameters.
However, some methods just retrain only the last layer of pre-trained models.
These methods are based on DFR [7], which suggest retraining the last layer of
the model on group-balanced data to make the model robust to spurious
correlation. Other methods were proposed to solve the requirement of group
annotation by different techniques for grouping or weighting. [8] suggests
putting samples of each class whose prediction changed after dropout in one
group and the rest in another for using DFR. AFR [15] is a recently proposed
method in this line, which upweights samples with lower confidence on the
ground truth label during retraining the last layer.
Our LFR method does not need access to group annotations and retrains only the
last layer of a model which is pre-trained with ERM. Our setting is similar to
AFR, and we compare it to LFR.

##  3 Method

###  3.1 Setting

Consider an image classification task among two classes on a dataset
$\mathcal{D}\subset{\mathbb{R}^{3\times H\times W}}$ where $H$ and $W$ are the
height and width of input images. Suppose images of each class $i\in\\{1,2\\}$
in the dataset are split into unknown majority $Maj_{i}$ and $Min_{i}$ groups
such that $|Maj_{i}|\gg|Min_{i}|$ and images in $Maj_{i}$ have predictive
attributes which are simpler than the core attribute of class $i$. We first
train a deep neural network $f:\mathcal{\mathbb{R}}^{3\times H\times
W}\rightarrow{}[0,1]^{2}$ with ERM and loss function
$\mathcal{L}:[0,1]^{2}\rightarrow{}\mathbb{R}$ on a randomly selected 80% of
$\mathcal{D}$ with class balancing. Class balancing is a conventional
necessary technique for training a good model in any classification task, even
without strong spurious correlation. The function $f=softmax(h\circ g)$ is
composed of a feature extractor $g:\mathbb{R}^{3\times H\times
W}\rightarrow{}\mathbb{R}^{d}$ and a linear last layer function
$h:\mathbb{R}^{d}\rightarrow{}\mathbb{R}^{2}$ with a softmax function on top
of them. Then we are going to retrain the last layer $h$ of the model on the
remaining 20% of $\mathcal{D}$, which refers to it by $\mathcal{D}_{ll}$. We
hope to have a higher WGA in groups
$\\{Maj_{1},Maj_{2}\\}\cup\\{Min_{1},Min_{2}\\}$ than before retraining.

###  3.2 Loss-Based Feature Reweighting

As it is illustrated in Fig. 1, after initial training, we evaluate the model
$f$ on $\mathcal{D}_{ll}$. We refer to the correctly-classified and
misclassified samples of class $i$ by $C_{i}$ and $M_{i}$. As we explain later
in this section, these sets are good representatives of $Maj_{i}$ and
$Min_{i}$ groups respectively.

After that, as has been suggested by [7], we want to select an equal number
$s$ of data from each group in the set
$\mathcal{G}=\bigcup_{i=1}^{2}\\{M_{i},C_{i}\\}$. $s$, the number of groups’
samples, is a hyperparameter and can be any number, but we usually select it
approximately as $s=min(|M_{1}|,|M_{2}|)$. The optimal amount $s$ is found by
hyperparameter selection based on worst group accuracy on the validation set.
In each majority group like $C_{i}$, we select $s$ samples with the highest
loss value $\mathcal{L}(f(x))$, whereas, for a minority group $M_{i}$, we
select $s$ images with the lowest value of loss function. Then, as has been
suggested by [7], we reinitialize the last layer $h$ of the model and then
train it on a balanced subset of $\mathcal{D}_{ll}$ which consists $s$ samples
from each group in $\mathcal{G}$.

Figure 2: An abstraction of our LFR method in 2D space. The majority groups of
data make the initial classifier biased towards themselves (black line).
Selecting an equal number of samples from high-loss value misclassified and
low-loss classified samples mitigates the bias towards a more robust
classifier (green line)

It is important to select samples based on loss value, not randomly. If a
model is pre-trained with ERM on a biased dataset, it probably relies on
spurious correlation for pre- diction. Thus, the loss value of the model is a
good sign of belonging to majority or minority groups. So, in classified
samples, if a sample has low loss, there is a great chance that the sample has
a spurious attribute which has made the model biased. On the other hand, in a
misclassified group, if a sample has a very high loss, it probably belongs to
the minority group without the spurious attribute. The success of loss-based
selection in comparison to random selection in the ablation study also
suggests that among correctly-classified samples, the ones that rely on
spurious attributes have lower loss value than the ones that attend to core
attributes. It is true as well for misclassified samples, as those with higher
loss values are more likely to have not the spurious attribute. Figure 2 shows
an abstraction of LFR in 2D space, which is compatible with the above
explanations. Practical experiments also confirm our descriptions.

##  4 Experiments

Table 1: Evaluation of worst and mean (in parenthesis) group accuracy on the Waterbirds dataset with various amounts of spurious correlation. Method | 70% | 80% | 90% | 95% | 99% | 100%
---|---|---|---|---|---|---
ERM | 84.42 (93.13) | 83.42 (90.97) | 76.36 (88.06) | 71.31 (85.57) | 48.74 (75.73) | 30.33 (66.83)
DFR | 91.59 (94.29) | 87.32 (91.73) | 88.43 (91.16) | 88.79 (92.48) | 83.80 (91.30) | 83.33 (91.20)
AFR | 86.71 | 83.43 | 76.42 | 77.40 | 78.82 | 84.73
CFR (Ours) | 75.45 (87.15) | 78.62 (87.57) | 75.09 (84.93) | 67.45 (80.12) | 78.76 (87.08) | 82.09 (89.48)
LFR (Ours) | 79.67 (85.25) |  83.81 (87.88) |  86.45 (91.02) |  86.85 (91.25) |  85.31 (89.24) |  86.29 (90.54)
Table 2: Evaluation of worst and mean (in parenthesis) group accuracy on the CelebA dataset with various amounts of spurious correlation. Method | 70% | 80% | 90% | 95% | 99% | 100%
---|---|---|---|---|---|---
ERM | 62.22 (93.56) | 62.22 (92.81) | 61.11 (92.65) | 57.22 (93.28) | 45.56 (93.33) | 40.00 (93.16)
DFR | 74.45 (92.71) | 77.44 (92.53) | 77.22 (91.71) | 76.11 (91.76) | 80.56 (91.42) | 74.44 (91.29)
AFR | 84.06% | 85.34% | 78.89% | 66.30% | 82.41% | 77.96%
CFR (Ours) |  86.67 (88.26) | 85.30 (85.17) | 80.00 (84.40) | 63.89 (86.97) | 83.59 (85.17) | 82.56 (84.83)
LFR (Ours) | 83.89 (87.20) |  86.73 (88.06) |  85.35 (84.26) |  79.36 (81.09) |  84.63 (86.06) |  82.78 (84.93)

###  4.1 Dataset

We evaluate LFR on two popular datasets.

Waterbirds: This dataset has been introduced by [16]. It is a classification
between waterbirds and landbirds, where most of the waterbirds are in the sea
and most of the landbirds are seen in the land background. Here, the
background scene is considered a spurious feature in both classes.

CelebA: Using the dataset which has been proposed by [13], CelebA is a hair
color classification of celebrities dataset. In this dataset, most celebrities
with blonde hair are women. So, only the blond class has a spurious
correlation between sex and the color of hair. CelebA has also some label
noise, i.e., some labels are not correct as we have seen.

Unlike previous methods which only have considered standard versions of these
datasets, we have created six versions of these datasets with various spurious
correlations. All datasets have four partitions for initial training (Tr),
last layer retraining (LLR), validation (V), and test (Te). In each dataset,
the size of the LLR partition is approximately $1/4$ of the size of Tr data.
Also, the spurious correlation is equal in LLR and Tr data, except for
datasets with 99%, and 100% spurious correlation, in which the LLR has 95%
spurious correlation. This is because we need a few samples of minority groups
in LLR data, which can not be satisfied when less than 1% of data belongs to
them unless we have much more data than we already have.

###  4.2 Setup

Similar to previous works [16] [7] [11], we have used ResNet-50 [6]
architecture which had been pre-trained on ImageNet [4]. We have changed the
last linear layer of the model to match our output dimension. Then, we trained
the model on the Tr partition of each dataset until convergence had been
achieved. After reinitializing the last linear layer of the model, we have
used the setting in [7] for retraining the last layer. We have used validation
data V for hyperparameter tuning. Finally, we have reported the resulting WGA
on test partition Te. More details about experiments’ settings are proposed in
B.

For AFR, we ran their source code and selected the best hyperparameter
$\gamma$, which specifies how much to upweights examples with poor
predictions. It gives an extra and maybe unfair advantage to AFR, as larger
$\gamma$ is suitable for high spurious correlation, and lower $\gamma$ is
better for low one. So, selecting the best $\gamma$ enables extra information
about the amount of spurious correlation. Nevertheless, we outperform AFR with
this setting except for low spurious correlation.

###  4.3 Results

Table 3: Effect of loss-based selection and random selection in each spurious correlation of Waterbirds dataset on worst and mean (in parentheses) group accuracy. Results are in percentage. CR-ML and CL-MR stand for Correctly classified with Random selection- Misclassified with Loss-based selection and Misclassified with Loss-based selection-Correctly classified with Random selection respectively. While when the spurious correlation is not lower than 90% LFR has the best performance, when the correlation is 70% or 60% random selection of misclassified samples leads to a higher WGA. Method | 70% | 80% | 90% | 95% | 99% | 100%
---|---|---|---|---|---|---
LFR | 79.67 (85.25) | 83.81 (87.88) |  86.45 (91.02) |  86.85 (91.25) |  85.31 (89.24) |  86.29 (90.54)
CFR | 75.45 (87.15) | 78.62 (87.57) | 75.09 (84.93) | 67.45 (80.12) | 78.76 (87.08) | 82.09 (89.48)
LFR-CR-ML | 81.09 (89.61) | 83.36 (88.65) | 82.68 (88.44) | 83.80 (90.39) | 80.84 (87.93) | 83.80 (90.07)
LFR-CL-MR |  84.65 (90.27) |  84.27 (91.21) | 80.37 (88.89) | 86.45 (92.03) | 83.96 (88.87) | 83.96 (89.35)

The results of Waterbirds and CelebA datasets are shown in tables 1 and 2
respectively. As can be observed, when the spurious correlation is high
enough, LFR outperforms the previous AFR method and even the DFR baseline, in
which we have access to the ground truth for the groups.

We also put the method to the test by randomly selecting an equivalent number
of samples from both the correctly classified and misclassified data in each
class. This simplified version of our technique is termed Classification-based
Feature Re-weighting (CFR). As illustrated in table 3, the loss-based
selection is important in both correctly classified and misclassified groups.
However, when the spurious correlation is low, the impact of loss-based
selection diminishes, particularly for misclassified samples. It’s as if the
loss value no longer serves as a reliable indicator of whether a sample
belongs to the majority or minority groups. In other words, when the spurious
correlation is low, a sample’s misclassification could be due to factors other
than the model’s reliance on the spurious features of a minority sample.

In low spurious correlations like 70%, AFR has the best WGA. As we have stated
before, it is because of the way we evaluate AFR, and give it an advantage AFR
probably underperforms in high spurious correlation settings because they do
not have any consideration for the number of examples from the majority group.
So, even if the underweight samples with lower loss are probably from the
majority group, the overall effect of the majority samples still remains high.
On the other hand, as LFR selects an equal number of samples from both the
majority and minority, it does not have the aforementioned problem.

Figure 3: Saliency map of models before (ERM) and after last layer retraining
with AFR and LFR. LFR has a better quality in terms of pushing the model to
attend to core attributes.

We also visualize our method’s attention map with Grad-Cam [5] to compare it
with other models. As it can be seen in figure 3, using LFR, the model attends
more to core attributes, in contrast to the pre-trained ERM model and AFR.
More results are available in C.

##  5 Conclusion

We proposed loss-based feature re-weighting, a method for selecting a few
images from a small proportion of training data to retrain the last linear of
the model which improves robustness to spurious correlation. While LFR is fast
and straightforward to use, it achieved SOTA in many various datasets in
comparison to previously proposed methods and baselines, even works with the
assumption of group annotations availability, in addition to better
performance in attending core attributes in input images.

## References

  * [1] Joy Buolamwini and Timnit Gebru.  Gender shades: Intersectional accuracy disparities in commercial gender classification.  In Sorelle A. Friedler and Christo Wilson, editors, Proceedings of the 1st Conference on Fairness, Accountability and Transparency, volume 81 of Proceedings of Machine Learning Research, pages 77–91. PMLR, 23–24 Feb 2018.
  * [2] Alexandra Chouldechova.  Fair prediction with disparate impact: A study of bias in recidivism prediction instruments.  Big Data, 5(2):153–163, 2017.  PMID: 28632438.
  * [3] Elliot Creager, Jörn-Henrik Jacobsen, and Richard S. Zemel.  Environment inference for invariant learning.  In International Conference on Machine Learning, 2020.
  * [4] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 248–255. IEEE, 2009.
  * [5] Jacob Gildenblat and contributors.  Pytorch library for cam methods.  https://github.com/jacobgil/pytorch-grad-cam, 2021.
  * [6] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016.
  * [7] P. Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson.  Last layer re-training is sufficient for robustness to spurious correlations.  ArXiv, abs/2204.02937, 2022.
  * [8] Tyler LaBonte, Vidya Muthukumar, and Abhishek Kumar.  Dropout disagreement: A recipe for group robustness with fewer annotations.  In NeurIPS 2022 Workshop on Distribution Shifts: Connecting Methods and Applications, 2022.
  * [9] Yoonho Lee, Huaxiu Yao, and Chelsea Finn.  Diversify and disambiguate: Out-of-distribution robustness via disagreement.  In The Eleventh International Conference on Learning Representations, 2023.
  * [10] Zhiheng Li, Ivan Evtimov, Albert Gordo, Caner Hazirbas, Tal Hassner, Cristian Canton Ferrer, Chenliang Xu, and Mark Ibrahim.  A whac-a-mole dilemma: Shortcuts come in multiples where mitigating one amplifies others.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20071–20082, June 2023.
  * [11] Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn.  Just train twice: Improving group robustness without training group information.  In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 6781–6792. PMLR, 18–24 Jul 2021.
  * [12] Jiashuo Liu, Zheyuan Hu, Peng Cui, B. Li, and Zheyan Shen.  Heterogeneous risk minimization.  In International Conference on Machine Learning, 2021.
  * [13] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Deep learning face attributes in the wild.  In 2015 IEEE International Conference on Computer Vision (ICCV), pages 3730–3738, 2015.
  * [14] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin.  Learning from failure: Training debiased classifier from biased classifier.  In Advances in Neural Information Processing Systems, 2020.
  * [15] Shikai Qiu, Andres Potapczynski, Pavel Izmailov, and Andrew Gordon Wilson.  Simple and fast group robustness by automatic feature reweighting.  ICML 2023.
  * [16] Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang.  Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization.  ArXiv, abs/1911.08731, 2019.
  * [17] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-CAM: Visual explanations from deep networks via gradient-based localization.  International Journal of Computer Vision, 128(2):336–359, oct 2019.
  * [18] Sahil Singla and Soheil Feizi.  Salient imagenet: How to discover spurious features in deep learning?  In International Conference on Learning Representations, 2022.
  * [19] Nimit Sharad Sohoni, Jared A. Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré.  No subclass left behind: Fine-grained robustness in coarse-grained classification problems.  ArXiv, abs/2011.12945, 2020.
  * [20] Nimit Sharad Sohoni, Maziar Sanjabi, Nicolas Ballas, Aditya Grover, Shaoliang Nie, Hamed Firooz, and Christopher Re.  BARACK: Partially supervised group robustness with guarantees.  In ICML 2022: Workshop on Spurious Correlations, Invariance and Stability, 2022.
  * [21] Kai Yuanqing Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry.  Noise or signal: The role of image backgrounds in object recognition.  In International Conference on Learning Representations, 2021.
  * [22] Michael Zhang, Nimit S Sohoni, Hongyang R Zhang, Chelsea Finn, and Christopher Re.  Correct-n-contrast: a contrastive approach for improving robustness to spurious correlations.  In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 26484–26516. PMLR, 17–23 Jul 2022.

##  Appendix A Datasets

The table 4 demonstrates the number of samples in each group for Tr and LL
partitions.

Table 4: Number of samples in each dataset in the experiments. Dataset | Split | Label | Spurious | 70% | 80% | 90% | 95% | 99% | 100%
---|---|---|---|---|---|---|---|---|---
Waterbirds | Train | Landbird | Land | 872 | 881 | 857 | 865 | 864 | 865
Water | 363 | 200 | 104 | 45 | 9 | 0
Waterbird | Land | 370 | 216 | 96 | 45 | 8 | 0
Water | 865 | 865 | 865 | 865 | 865 | 865
Last Layer | Landbird | Land | 202 | 192 | 194 | 723 | 723 | 723
Water | 72 | 48 | 19 | 33 | 33 | 33
Waterbird | Land | 82 | 48 | 21 | 11 | 11 | 11
Water | 192 | 192 | 192 | 192 | 192 | 192
CelebA | Train | Black Hair | Female | 13524 | 11762 | 10567 | 10018 | 9636 | 9346
Male | 12561 | 11063 | 9721 | 9203 | 8808 | 8914
Blond Hair | Female | 18260 | 18260 | 18260 | 18260 | 18260 | 18260
Male | 7825 | 4565 | 2028 | 961 | 184 | 0
Last Layer | Black Hair | Female | 3386 | 3003 | 2632 | 2477 | 2498 | 2520
Male | 3214 | 2772 | 2501 | 2386 | 2365 | 2343
Blond Hair | Female | 4620 | 4620 | 4620 | 4620 | 4620 | 4620
Male | 1980 | 1155 | 513 | 243 | 243 | 243

##  Appendix B Setup

To fine-tune our method’s hyperparameters, we carefully considered several
important factors. Specifically, when adjusting the sample size (the number of
samples chosen from misclassified and correctly-classified groups for each
label), our aim was to ensure that the final dataset exhibited a rough balance
among different group labels. For example, when working with the Waterbirds
dataset, we found that using a sample size of about 30 was sufficient to
achieve state-of-the-art performance after re-training. This meant we selected
120 data points using our loss-based method. Additionally, we incorporated a
weight decay of 1e-3 or 1e-4 and experimented with learning rates ranging from
1e-4 to 1e-3 to further enhance our model’s performance. On the other hand,
for the CelebA dataset, which had more data points, we needed a sample size of
nearly 700 to get good results.

Figure 4: Additional GradCAM visualizations for CelebA dataset. Both ERM and
AFR models output higher salience for non-causal attributes. LFR corrects this
both for majority and minority examples. Figure 5: Additional GradCAM
visualizations for Waterbirds dataset. Both ERM and AFR models output higher
salience for spurious attributes (here is the background). LFR corrects this
both for majority and minority examples.

##  Appendix C Saliency Maps of Model’s attention regions

We have incorporated additional visualizations to support the effectiveness of
our method. By employing Grad-CAM [17], we illustrate that models trained with
ERM tend to classify inputs based on spurious features. Additionally, AFR
appears to rely on non-causal features for prediction, possibly due to the
regularization term in its loss function, which encourages the weights to stay
close to those of ERM. LFR rectifies this phenomenon and relies on core
features for both majority and minority group examples. Figure 4 and 5 provide
visual representations of these trends on the CelebA and Waterbirds datasets,
respectively, vividly demonstrating how our approach effectively addresses
these issues.