# The Macías topology on integral domains

Jhixon Macías  Jhixon Macías
University of Puerto Rico at Mayaguez, Mayaguez, PR, USA
United States of America<EMAIL_ADDRESS>
###### Abstract.

In this manuscript a recent topology on the positive integers generated by the
collection of $\\{\sigma_{n}:n\in\mathbb{N}\\}$ where
$\sigma_{n}:=\\{m:\gcd(n,m)=1\\}$ is generalized over integral domains. Some
of its topological properties are studied. Properties of this topology on
infinite principal ideal domains that are not fields are also explored, and a
new topological proof of the infinitude of prime elements is obtained
(assuming the set of units is finite or not open), different from those
presented in the style of H. Furstenberg. Finally, some problems are proposed.

###### Key words and phrases:

Primes, Topology, Integral Domains, Golomb’s topology

###### 2020 Mathematics Subject Classification:

54A05; 54G05; 54H11

##  1\. Introduction

In 1955, H. Furstenberg introduced a topology $\tau_{F}$ on the integers
$\mathbb{Z}$ generated by the collection of arithmetic progressions of the
form $a+b\mathbb{Z}$ ($a,b\in\mathbb{Z},b\geq 1$) and with it presented the
first topological proof of the infinitude of prime numbers (now very famous),
see [6]. A couple of years earlier (1953), M. Brown had introduced a topology
$\tau_{G}$ on the natural numbers $\mathbb{N}$ that turns out to be coarser
than Furstenberg’s topology induced on $\mathbb{N}$. This latter topology is
generated by the collection of arithmetic progressions $a+b\mathbb{N}$ with
$a\in\mathbb{N}$ and $b\in\mathbb{N}\cup\\{0\\}$ such that $\gcd(a,b)=1$. It
was not until 1959 that the topology introduced by Brown was popularized by S.
Golomb (now known as Golomb’s topology) who in [7] proves that Dirichlet’s
theorem on arithmetic progressions is equivalent to the set of prime numbers
$\mathbb{P}$ being dense in the topological space $(\mathbb{N},\tau_{G})$.
Golomb also presents a topological proof of the infinitude of prime numbers,
verifying that Furstenberg’s argument also works on $(\mathbb{N},\tau_{G})$.
In 1969, A.M. Kirch [9] defines a topology $\tau_{K}$ coarser than Golomb’s
topology generated by the collection of arithmetic progressions
$a+b\mathbb{N}$ with $a\in\mathbb{N},b\in\mathbb{N}\cup\\{0\\}$ and $a$
square-free such that $\gcd(a,b)=1$. Over the years these topologies have been
widely studied, see for example the following works [8, 1, 19, 9, 21, 2, 18,
11, 3, 20, 22, 23, 12]. In 1997, J. Knopemacher and S. Porubsky [10]
generalize both Furstenberg’s and Golomb’s topologies in integral domains, and
study the arithmetic properties obtained in these equipped with these
topologies (especially with the generalization of Golomb’s topology). In
subsequent years, Golomb spaces have been studied in various algebraic
structures, see [4, 17, 15, 16, 14].

Recently, in [5], a new topology $\tau_{M}$ is introduced, called the Macías
topology, which is generated by the collection of sets
$\sigma_{n}:=\\{m\in\mathbb{N}:\gcd(n,m)=1\\}$. Additionally, some of its
properties are studied. For example, the topological space
$(\mathbb{N},\tau_{M})$ does not satisfy the $\mathrm{T}_{0}$ separation
axiom, satisfies (vacuously) the $\mathrm{T}_{4}$ separation axiom, is
ultraconnected, hyperconnected (therefore connected, locally connected, path-
connected), not countably compact (therefore not compact), but it is limit
point compact and pseudocompact. On the other hand, for $n,m\in\mathbb{N}$, it
holds that
$\textbf{Cl}_{(\mathbb{N},\tau_{M})}(\\{nm\\})=\textbf{Cl}_{(\mathbb{N},\tau_{M})}(\\{n\\})\cap\textbf{Cl}_{(\mathbb{N},\tau_{M})}(\\{m\\})$
where $\textbf{Cl}_{(\mathbb{N},\tau_{M})}(\\{n\\})$ denotes the closure of
the singleton set $\\{n\\}$ in the topological space $(\mathbb{N},\tau_{M})$.
Furthermore, in [13], a new topological proof of the infinitude of prime
numbers is presented (different from the proofs of Furstenberg and Golomb). In
the same work, the infinitude of any non-empty subset of prime numbers is
characterized in the following sense: if $A\subset\mathbb{P}$ (non-empty),
then $A$ is infinite if and only if $A$ is dense in $(\mathbb{N},\tau_{M})$,
see [13, Theorem 4].

In the present manuscript, the Macías topology is generalized over integral
domains that are not fields and some of its basic properties are studied
(Section 2), and it is compared with Golomb’s topology (Subsection 2.7). We
also study some properties of the Macías topology in principal ideal domains
(PIDs) that are not fields (Section 3). Furthermore, the infinitude of prime
elements is characterized and the proof of the infinitude of primes presented
in [13] is generalized over infinite PIDs with a finite number of units
(Subsection 3.16).

##  2\. The Macías topology on integral domains

Throughout this section, we denote by $R$ a commutative ring with identity
$1=1_{R}\neq 0_{R}$, and without zero divisors (an integral domain) that is
not a field. For each $r\in R$, define

$\sigma_{r}:=\\{s\in R:\langle r\rangle+\langle s\rangle=R\\}=\\{s\in R\mid\ \
\exists u,v\in R:ur+vs=1\\}$

where for all $r\in R$, $\langle r\rangle$ is the ideal generated by $r$. Let
$\beta_{R}:=\\{\sigma_{r}:r\in R\\}$.

######  Theorem 2.1.

The set $\beta_{R}$ is a basis for some topology on $R$.

###### Proof.

It is clear that for all $r\in R$, we have $\langle 1\rangle+\langle
r\rangle=R$, so $\sigma_{1}=R$. Now, observe that
$\sigma_{r_{1}r_{2}}=\sigma_{r_{1}}\cap\sigma_{r_{2}}$ for all $r_{1},r_{2}\in
R$. Indeed, let $s\in\sigma_{r_{1}r_{2}}$. Then there exist $u,v\in R$ such
that

$1=u(r_{1}r_{2})+vs=(ur_{1})r_{2}+vs=(ur_{2})r_{1}+vs.$

Thus, $s\in\sigma_{r_{1}}\cap\sigma_{r_{2}}$. On the other hand, let
$s\in\sigma_{r_{1}}\cap\sigma_{r_{2}}$. Then, there exist
$u_{1},v_{1},u_{2},v_{2}\in R$ such that $u_{1}r_{1}+v_{1}s=1$ and
$u_{2}r_{2}+v_{2}s=1$. Thus:

$\begin{split}1=(u_{1}r_{1}+v_{1}s)(u_{2}r_{2}+v_{2}s)&=u_{1}r_{1}u_{2}r_{2}+u_{1}r_{1}v_{2}s+v_{1}su_{2}r_{2}+v_{1}sv_{2}s\\\
&=(u_{1}u_{2})r_{1}r_{2}+(u_{1}r_{1}v_{2}+v_{1}u_{2}r_{2}+v_{1}sv_{2})s,\end{split}$

which implies that $s\in\sigma_{r_{1}r_{2}}$. Therefore, $\beta_{R}$ is a
basis for some topology on $R$. ∎

We denote by $\widetilde{M(R)}$ the topological space $(R,\tau_{R})$, where
$\tau_{R}$ is the topology generated by $\beta_{R}$.

The definition of $\sigma_{r}$ (as defined) is inspired by the Bézout’s
identity. As much as possible, we want to absorb the properties of relatively
prime elements over integral domains (even when a $\gcd$ does not necessarily
exist for every pair of elements in $R$). Therefore, the study in this
manuscript does not focus on $R$, but on $R^{0}:=R\setminus\\{0\\}$. In
general, for $X\subset R$, $X^{0}=X\setminus\\{0\\}$. Thus, we define the
Macías topological space $M(R)$ as the topological space
$(R^{0},\tau_{R^{0}})$ where $\tau_{R^{0}}$ is the topology generated by the
basis $\beta_{R^{0}}:=\\{\sigma_{k}^{0}:k\in R^{0}\\}$. The reason why we do
not want to consider $\sigma_{0}$ is that we do not want the units of $R$,
denoted by $U(R)$, to be open sets (note that $\sigma_{0}=U(R)$). This will
become evident in Section 3.16. For now, we have the following theorem:

######  Theorem 2.2.

The following propositions hold over $R$:

  1. (1)

$u\in U(R)$ if and only if $\sigma_{u}=R$.

  2. (2)

$0\in\sigma_{k}$ if and only if $k\in U(R)$.

  3. (3)

$\textbf{cl}_{\widetilde{M(R)}}(\\{0\\})=U(R)$.

  4. (4)

If $u\in U(R)$, then $u\in\sigma_{r}$ for all $r\in R$.

  5. (5)

If $u\in U(R)$, then $\textbf{cl}_{M(R)}(\\{u\\})=R^{0}$ and
$\textbf{cl}_{\widetilde{M(R)}}(\\{u\\})=R$.

###### Proof.

$(1)$, $(2)$, and $(4)$ are immediate. $(3)$ follows from $(2)$ and $(5)$ from
$(4)$. ∎

On the other hand, note that the fundamental properties of $R$ for $\beta_{R}$
to be a basis are: commutativity and the existence of $1_{R}$. The condition
of having no zero divisors or that $R$ is not a field is initially required
due to the general interest in algebraic-arithmetical domains, which is
generated by previous studies of Golomb’s topology on integral domains. We
also want to avoid trivial topologies. For example, if
$R=\mathbb{Z}/2\mathbb{Z}=\\{\overline{0},\overline{1}\\}$, we have that
$M(R)$ is the indiscrete space (see Theorem 2.11).

Now, some basic properties of $\widetilde{M(R)}$ and $M(R)$.

######  Theorem 2.3.

$\widetilde{M(R)}$ is a topological semigroup.

###### Proof.

Let $\cdot_{R}:R\times R\to R$ be the product of $R$. Note that for each $r\in
R$ we have:

$\begin{split}\cdot_{R}^{-1}(\sigma_{r})&=\\{(x,y)\in R:xy\in\sigma_{r}\\}\\\
&=\\{(x,y)\in R:\langle r\rangle+\langle xy\rangle=R\\}\\\ &=\\{(x,y)\in
R:\langle r\rangle+\langle x\rangle=R\ \ \text{and}\ \ \langle
r\rangle+\langle y\rangle=R\\}\\\ &=\\{(x,y)\in R:x,y\in\sigma_{r}\\}\\\
&=\sigma_{r}\times\sigma_{r},\\\ \end{split}$

so $\cdot_{R}$ is continuous. Therefore, $\widetilde{M(R)}$ is a topological
semigroup. ∎

######  Corollary 2.4.

$M(R)$ is a topological semigroup.

A topological space $X$ is hyperconnected if there are not two disjoint open
sets. Given that $1\in\sigma_{r}$ for all $r\in R$, we have the following
theorem.

######  Theorem 2.5.

$\widetilde{M(R)}$ and $M(R)$ are hyperconnected.

Since a hyperconnected space cannot be Hausdorff unless it contains only a
single point and every hyperconnected space is both connected and locally
connected and moreover extremally disconnected, we have the following
corollary.

######  Corollary 2.6.

$\widetilde{M(R)}$ and $M(R)$ are not Hausdorff, but are both connected and
locally connected and extremally disconnected.

###  2.7. The Macías topology vs. the Golomb topology

In [4], the topological space $\widetilde{G(R)}$ is defined, which is obtained
by equipping $R$ with the topology generated by the collection of coprime
cosets ${x+I}$ where $x\in R$ and $I$ is a nonzero ideal of $R$. The Golomb
topological space $G(R)$ is obtained by equipping $R^{0}$ with the subspace
topology of $\widetilde{G(R)}$.

In the case of $\mathbb{N}$, the Macías topology is coarser than the Golomb
topology. Therefore, it is interesting to ask if this occurs in general over
$R$. The answer to this latter question is affirmative.

######  Theorem 2.8.

Every open set in $\widetilde{M(R)}$ (except for $\sigma_{0}=U(R)$) is open in
$\widetilde{G(R)}$.

###### Proof.

Let $\sigma_{k}\in\beta_{R}$. If $x\in\sigma_{k}$, then there exist $u,v\in R$
such that $ux+vr=1$. Let $r\in R$, then $1=u(x+kr)+(v-uk)r$. Thus,
$x+rk\in\sigma_{k}$ for all $r\in R$. On the other hand, consider the element
$r+nk$ where $\langle r\rangle+\langle k\rangle=R$ and $n\in R$. Since
$r\in\sigma_{k}$, there exist $u,v\in R$ such that $ur+vk=1$. Then
$u(r+nk)+(v-un)k=1$. Thus, $r+nk\in\sigma_{k}$. Therefore, for all
$\sigma_{k}\in\beta_{R}$, we have:

$\sigma_{k}=\bigcup_{\begin{subarray}{c}r\in R\\\ \langle r\rangle+\langle
k\rangle=R\end{subarray}}r+\langle k\rangle.$

From this last equality, the result follows. ∎

Note from this last result that every open set in $M(R)$ is open in $G(R)$. Of
course, in $M(R)$ and $G(R)$ the units do not form an open set. Thus, we have
the following corollary.

######  Corollary 2.9.

The Macías topology is strictly coarser than the Golomb topology.

Now, in [4] the following theorem is proven:

######  Theorem 2.10 (Lemma 1).

For an integral domain $R$, the following are equivalent

  1. (1)

The space $\widetilde{G(R)}$ is indiscrete.

  2. (2)

The space $G(R)$ is indiscrete.

  3. (3)

The ring $R$ is a field.

Therefore, by virtue of Corollary 2.9, we have the following theorem:

######  Theorem 2.11.

For an integral domain $R$, the following are equivalent

  1. (1)

The space $M(R)$ is indiscrete.

  2. (2)

The ring $R$ is a field.

Note that $M(R)$ can be indiscrete, but $\widetilde{M(R)}$ (for $R$ an
integral domain) cannot. Consider the following example.

######  Example 2.12.

Consider the Boolean Field
$R=\mathbb{Z}/2\mathbb{Z}=\\{\overline{0},\overline{1}\\}$, then
$\widetilde{M(R)}$ is not the indiscrete space. Indeed,
$\beta_{R}=\\{\\{1\\},R\\}$.

Finally, recall that if $R$ is a domain that is not a field, then $R$ is
infinite and every nonzero ideal $I$ of $R$ is infinite. Now, since $R$ is an
integral domain, the map $R\to I$ defined by $r\mapsto rx$ ($x\in I^{0}$) is
an injection. Therefore, the non-empty open sets in $G(R)$ are infinite, and
hence, the non-empty open sets in $M(R)$ are infinite. Another way to see that
the non-empty open sets in $M(R)$ are infinite is the following: since
$1\in\sigma_{k}$ for all $k\in R$, then $1+rk\in\sigma_{k}$ for all $r,k\in
R$. Since $R$ is an integral domain, if $r_{1}\neq r_{2}$, then $1+r_{1}k\neq
1+r_{2}k$. Therefore, $\sigma_{k}$ is infinite for all $k\in R$. This last
property is crucial for obtaining a topological proof of the infinitude of
primes over PIDs.

##  3\. The Macías topology over principal ideal domains

From this point on, unless otherwise indicated or an additional condition is
assumed, we will suppose that $R$ is an infinite PID (not a field).

######  Theorem 3.1.

Let $p\in R$ be irreducible (prime). Then
$\textbf{cl}_{\widetilde{M(R)}}(\\{p\\})=\langle p\rangle$.

###### Proof.

Let $x\in\textbf{cl}_{\widetilde{M(R)}}(\\{p\\})$. Suppose $x\notin\langle
p\rangle$. Since $R$ is a PID, $\langle p\rangle$ is maximal in $R$. Thus,
$R/\langle p\rangle$ is a field, and hence the coset $x+\langle p\rangle$ is
invertible. Thus, there exist $u,v\in R$ such that $ux+vp=1$, implying
$x\in\sigma_{p}$. This implies $p\in\sigma_{p}$, which is absurd since $p$
should be a unit in this case. On the other hand, let $x\in\langle p\rangle$.
Then $x=rp$ for some $r\in R$. Let $\sigma_{k}\in\beta_{R}$ such that
$x\in\sigma_{k}$. Thus, there exist $u,v\in R$ such that
$1=uk+v(rp)=uk+(vr)p$, implying $p\in\sigma_{k}$. Therefore,
$x\in\textbf{cl}_{M(R)}(\\{p\\})$. ∎

######  Corollary 3.2.

Let $p\in R$ be irreducible (prime). Then $\textbf{cl}_{M(R)}(\\{p\\})=\langle
p\rangle^{0}$.

######  Corollary 3.3.

Let $r\in R^{0}-U(R)$. We can write
$r=p_{1}{}^{m_{1}}p_{2}{}^{m_{2}}...p_{n}{}^{m_{n}}$ where $p_{i}$ is
irreducible (prime) such that $\langle p_{i}\rangle+\langle p_{j}\rangle=R$
for $i\neq j$. Then

$\textbf{cl}_{\widetilde{M(R)}}(\\{r\\})=\bigcap_{i}\langle p_{i}\rangle$

###### Proof.

Let $x\in\bigcap_{i}\langle p_{i}\rangle$. Then, by the previous theorem,
$x\in\textbf{cl}_{\widetilde{M(R)}}(\\{p_{i}\\})$ for every $i$. Thus, for any
basic element $\sigma_{k}\in\beta_{R}$ such that $x\in\sigma_{k}$,
$p_{i}\in\sigma_{k}$ for every $i$. Hence,
$r=p_{1}{}^{m_{1}}p_{2}{}^{m_{2}}...p_{n}{}^{m_{n}}\in\sigma_{k}$. Therefore,
$x\in\textbf{cl}_{\widetilde{M(R)}}(\\{r\\})$. On the other hand, let
$x\in\textbf{cl}_{\widetilde{M(R)}}(\\{r\\})$. Then, for every basic element
$\sigma_{k}\in\beta_{R}$ such that $x\in\sigma_{k}$,
$r=p_{1}{}^{m_{1}}p_{2}{}^{m_{2}}...p_{n}{}^{m_{n}}\in\sigma_{k}$, from which
it follows that $p_{i}\in\sigma_{k}$ for every $i$. Thus,
$x\in\textbf{cl}_{\widetilde{M(R)}}(\\{p_{i}\\})$ for every $i$, and by the
previous theorem, $x\in\langle p_{i}\rangle$ for every $i$. Therefore,
$\textbf{cl}_{\widetilde{M(R)}}(\\{r\\})=\bigcap_{i}\langle p_{i}\rangle$. ∎

######  Corollary 3.4.

Let $r\in R^{0}-U(R)$. We can write
$r=p_{1}{}^{m_{1}}p_{2}{}^{m_{2}}...p_{n}{}^{m_{n}}$ where $p_{i}$ is
irreducible (prime) such that $\langle p_{i}\rangle+\langle p_{j}\rangle=R$
for $i\neq j$. Then

$\textbf{cl}_{M(R)}(\\{r\\})=\bigcap_{i}\langle p_{i}\rangle^{0}$

######  Remark 3.5.

Note that if $x,y\in\sigma_{k}$, then $xy\in\sigma_{k}$. Of course, if
$x,y\in\sigma_{k}$, there exist $u_{x},v_{x},u_{y},v_{y}\in R$ such that
$1=u_{x}k+v_{x}x=u_{y}k+v_{y}y$. Thus,
$(u_{x}ku_{y}+u_{x}v_{y}y+v_{x}xu_{y})k+(v_{x}v_{y})xy=1$, implying
$xy\in\sigma_{k}$.

######  Corollary 3.6.

Let $x,y$ be any elements in $R$. Then

$\textbf{cl}_{\widetilde{M(R)}}(\\{xy\\})=\textbf{cl}_{\widetilde{M(R)}}(\\{x\\})\cap\textbf{cl}_{\widetilde{M(R)}}(\\{y\\}).$

###### Proof.

If $x$ or $y$ is a unit or $x=0$ or $y=0$, the result is clear, see Theorem
2.2. Thus, suppose neither $x$ nor $y$ are units and neither are 0. Then we
can have $x=p_{1}{}^{a_{1}}\cdots p_{r}{}^{a_{r}}$ and
$y=q_{1}{}^{b_{1}}\cdots q_{s}{}^{b_{s}}$ written in their prime
decompositions. Then, by Corollary 3.3,

$\textbf{cl}_{\widetilde{M(R)}}(\\{x\\})\cap\textbf{cl}_{\widetilde{M(R)}}(\\{x\\})=\bigcap_{i}\langle
p_{i}\rangle\cap\bigcap_{j}\langle
q_{j}\rangle=\textbf{cl}_{\widetilde{M(R)}}(\\{xy\\}).$

The proof is complete. ∎

Similarly, we prove the following corollary.

######  Corollary 3.7.

Let $x,y$ be any elements in $R^{0}$. Then

$\textbf{cl}_{M(R)}(\\{xy\\})=\textbf{cl}_{M(R)}(\\{x\\})\cap\textbf{cl}_{M(R)}(\\{y\\}).$

A topological space $X$ is ultraconnected if every two non-empty closed
subsets are not disjoint. Equivalently, $X$ is ultraconnected if and only if
the closure of two distinct points always have non-trivial intersection.

######  Corollary 3.8.

$\widetilde{M(R)}$ is ultraconnected.

###### Proof.

Let $F$ and $G$ be two non-trivial closed sets in $\widetilde{M(R)}$. Let
$x\in F$ and $y\in G$. Then, by Corollary 3.6, we have that

$\\{x\cdot
y\\}\subset\textbf{cl}_{\widetilde{M(R)}}(\\{xy\\})=\textbf{cl}_{\widetilde{M(R)}}(\\{x\\})\cap\textbf{cl}_{\widetilde{M(R)}}(\\{y\\})\subset
F\cap G.$

Therefore, $F\cap G\neq\emptyset$. ∎

Similarly, we prove the following corollary.

######  Corollary 3.9.

$M(R)$ is ultraconnected.

Since every ultraconnected space is normal, limit point compact, pseudocompact
and path-connected, we have the following corollary.

######  Corollary 3.10.

$\widetilde{M(R)}$ and $M(R)$ are normal, limit point compact, pseudocompact
and path-connected.

A $\mathrm{T_{0}}$ space is a topological space in which every pair of
distinct points is topologically distinguishable. That is, for any two
different points $x$ and $y$ there is an open set that contains one of these
points and not the other.

For each $x\in R^{0}-U(R)$, define

$\mathcal{F}(x):=\\{p\in R^{0}:p\ \ \text{is an irreducible element and }\ \
p\mid x\\}.$

######  Theorem 3.11.

Let $x,y\in R^{0}-U(R)$. Then, $x$ and $y$ are topologically indistinguishable
if and only if $\mathcal{F}(x)=\mathcal{F}(y)$.

###### Proof.

The proof is straightforward. ∎

######  Corollary 3.12.

$M(R)$ does not satisfy the $\mathrm{T}_{0}$ separation axiom.

Up to this point it’s worth asking the following questions:

######  Question 3.13.

Let $R$ be an integral domain that is not a PID. Is $\widetilde{M(R)}$ or
$M(R)$ ultraconnected?

######  Question 3.14.

Let $R$ be an integral domain that is not a PID. Is $\widetilde{M(R)}$ or
$M(R)$ not satisfying the $\mathrm{T}_{0}$ axiom?

######  Example 3.15.

Note that if $R$ is not an integral domain, it is possible for $M(R)$ to not
be ultraconnected. For example, consider $R=\mathbb{Z}/6\mathbb{Z}$. Then,
$\sigma^{0}_{\overline{2}}=\\{\overline{1},\overline{3},\overline{5}\\}$ and
$\sigma^{0}_{\overline{3}}=\\{\overline{1},\overline{2},\overline{4},\overline{5}\\}$.
Thus, $\\{\overline{3}\\}$ and $\\{\overline{2},\overline{5}\\}$ are closed
and disjoint in $M(R)$, hence it is not ultraconnected. However,
$\widetilde{M(R)}$ is ultraconnected (since $\\{\overline{0},\overline{3}\\}$
and $\\{\overline{0},\overline{2},\overline{5}\\}$ are closed in
$\widetilde{M(R)}$).

###  3.16. On the infinitude of primes

In this subsection, $R$ is an infinite PID (not a field) with a finite number
of units. We aim to prove, topologically, the following proposition.

######  Proposition 3.17.

$R$ has infinitely many maximal ideals.

The idea is to generalize the topological proof of the infinitude of prime
numbers, presented in [13]. In this regard, we present the following result.

######  Theorem 3.18.

Let $\mathcal{P}$ be the collection of prime elements of $R$. Then,
$\mathcal{P}$ is infinite if and only if $\mathcal{P}$ is dense in $M(R)$.

###### Proof.

Suppose $\mathcal{P}$ is infinite. Let $\sigma_{k}^{0}\in\beta_{R^{0}}$. Then,
we can take $p\in\mathcal{P}$ such that $\langle k\rangle+\langle p\rangle=R$.
Thus $\sigma_{k}^{0}\cap\mathcal{P}\neq\emptyset$. Therefore, $\mathcal{P}$ is
dense in $M(R)$. Conversely, suppose $\mathcal{P}$ is dense in $M(R)$ and
consider a finite collection of elements from $\mathcal{P}$, say
$\\{p_{1},p_{2},\dots,p_{n}\\}$. Let $x=p_{1}\cdot p_{2}\cdots p_{n}$. Then,
$x\in R^{0}$. Now, observe that $\sigma_{x}^{0}\neq U(R)$, since
$\sigma_{x}^{0}$ is infinite. Therefore, since $\mathcal{P}$ is dense in
$M(R)$, there exists a $q\in\mathcal{P}\setminus\\{p_{1},p_{2},\dots p_{n}\\}$
such that $q\in\sigma_{x}^{0}\cap\mathcal{P}$. Hence, $\mathcal{P}$ is dense
in $M(R)$. ∎

######  Theorem 3.19.

If $\mathcal{P}$ is dense in $R^{0}\setminus U(R)$ with the induced subspace
topology from $M(R)$, denoted by $M(R^{0}\setminus U(R))$, then $\mathcal{P}$
is dense in $M(R)$.

###### Proof.

Note that $R^{0}\setminus U(R)$ is dense in $M(R)$. The result follows from
the transitive property of density. ∎

######  Theorem 3.20.

$\mathcal{P}$ is dense in $R^{0}\setminus U(R)$ with the induced subspace
topology from $M(R)$.

###### Proof.

In any topological space, the union of closures of subsets of that space is
contained in the closure of the union of those sets. Therefore,

$\displaystyle\bigcup_{p\in\mathcal{P}}\mathbf{cl}_{M(R^{0}\setminus
U(R))}(\\{p\\})\subset\mathbf{cl}_{M(R^{0}\setminus U(R))}(\mathcal{P})\subset
R^{0}\setminus U(R).$

Moreover, by Corollary 3.4, we have

$\bigcup_{p\in\mathcal{P}}\mathbf{cl}_{M(R^{0}\setminus
U(R))}(\\{p\\})=\bigcup_{p\in\mathcal{P}}(\mathbf{cl}_{M(R^{0})}(\\{p\\})\cap(R^{0}\setminus
U(R)))=\bigcup_{p\in\mathcal{P}}\langle p\rangle^{0}.$

Now, since $R$ is a unique factorization domain, we have $R^{0}\setminus
U(R)=\bigcup_{p\in\mathcal{P}}\langle p\rangle^{0}$. Therefore,
$\mathbf{cl}_{M(R^{0}\setminus U(R))}(\mathcal{P})=R^{0}\setminus U(R)$. ∎

The last three theorems establish Proposition 3.17. Thus, we obtain a new
topological proof of the infinitude of primes for rings such as the integers
$\mathbb{Z}$, the Gaussian integers $\mathbb{Z}[i]$, the Eisenstein integers
$\mathbb{Z}[\omega]$ (where $\omega$ is a primitive cube root of 1), etc.

Note that the argument to prove Theorem 3.18 works because under the given
hypothesis for $U(R)$, it is guaranteed that it is not open. Therefore, we
have the following corollary.

######  Corollary 3.21.

If $U(R)$ is not open in $M(R)$, then $R$ has infinitely many maximal ideals.

Finally, we want to present the following theorem.

######  Theorem 3.22.

$M(R)$ is not compact.

###### Proof.

The proof is straightforward. Consider the cover
$\\{\sigma_{p}^{0}:p\in\mathcal{P}\\}$. ∎

##  4\. Final Comment

This manuscript is an introductory work that generalizes the topological space
$X:=(\mathbb{N},\tau_{M})$ in the following sense: If $R=\mathbb{Z}$, then $X$
is the topological space obtained by equipping $\mathbb{N}$ with the subspace
topology induced by $M(R)$. Furthermore, it can be widely extended (the
studies conducted on the Golomb topology can be replicated as much as
possible). For example, one can study the properties of the quotient topology
induced by the Macías topology on the semigroup of all associate-classes of
non-zero elements in an integral domain $R$, similar to what was done with the
Golomb topology in [10]. A similar study to that conducted for the Golomb
topology in [4] can also be carried out. Moreover, one can study the
homeomorphism problem in $M(R)$, just as has been done and continues to be
done with the Golomb topology, see [4, Section 3] . In this regard, the
following problem is posed.

######  Problem 4.1.

Let $R$ and $S$ be countably infinite semiprimitive integral domains. Decide
whether $M(R)$ and $M(S)$ are homeomorphic.

## References

  * [1] Taras Banakh, Jerzy Mioduszewski and Slawomir Turek  “On continuous self-maps and homeomorphisms of the Golomb space”  In _arXiv preprint arXiv:1711.06749_ , 2017
  * [2] Taras Banakh, Dario Spirito and Sławomir Turek  “The Golomb space is topologically rigid”  In _arXiv preprint arXiv:1912.01994_ , 2019
  * [3] José Carmen Alberto-Domínguez, Gerardo Acosta and Gerardo Delgadillo-Piñón  “Totally Brown subsets of the Golomb space and the Kirch space”  In _Commentationes Mathematicae Universitatis Carolinae_ 62.2 Charles University in Prague, Faculty of MathematicsPhysics, 2022, pp. 189–219
  * [4] Pete L Clark, Noah Lebowitz-Lockard and Paul Pollack  “A note on Golomb topologies”  In _Quaestiones Mathematicae_ 42.1 Taylor & Francis, 2019, pp. 73–86
  * [5] L. Mejías E. Aponte and J. Vielma  “A strongly connected topology on the positive integers” Submitted, 2023
  * [6] Harry Furstenberg  “On the infinitude of primes”  In _Amer. Math. Monthly_ 62.5, 1955, pp. 353
  * [7] Solomon W Golomb  “A connected topology for the integers”  In _The American Mathematical Monthly_ 66.8 Taylor & Francis, 1959, pp. 663–665
  * [8] Solomon W Golomb  “Arithmetica topologica”  In _General Topology and its Relations to Modern Analysis and Algebra_ Academia Publishing House of the Czechoslovak Academy of Sciences, 1962, pp. 179–186
  * [9] AM Kirch  “A countable, connected, locally connected Hausdorff space”  In _The American Mathematical Monthly_ 76.2 Taylor & Francis, 1969, pp. 169–171
  * [10] J Knopemacher and Stefan Porubsky  “Topologies related to arithmetical properties of integral domains”  In _Expositiones Mathematicae_ 15 SPEKTRUM ACADEMISCHER VERLAG, 1997, pp. 131–148
  * [11] Ignazio Longhi, Yunzhu Mu and Francesco Maria Saettone  “Coset topologies on Z and arithmetic applications”  In _Expositiones Mathematicae_ 41.1 Elsevier, 2023, pp. 71–114
  * [12] Rezsö L Lovas and István Mezö  “On an exotic topology of the integers”  In _arXiv preprint arXiv:1008.0713_ , 2010
  * [13] J. Macías  “Another Topological Proof of the Infinitude of Prime Numbers”  In _Integers_ 24.47 Walter de Gruyter GmbH, 2024
  * [14] Chris Orum  “Golomb’s arithmetical semigroup topology and a semiprime sufficiency condition for Dirichlet’s theorem”  In _The Rocky Mountain Journal of Mathematics_ 46.3 JSTOR, 2016, pp. 939–949
  * [15] Dario Spirito  “The Golomb topology of polynomial rings”  In _Quaestiones Mathematicae_ 44.4 Taylor & Francis, 2021, pp. 447–468
  * [16] Dario Spirito  “The Golomb topology of polynomial rings, II”  In _Quaestiones Mathematicae_ 47.1 Taylor & Francis, 2024, pp. 113–122
  * [17] Dario Spirito  “The Golomb topology on a Dedekind domain and the group of units of its quotients”  In _Topology and its Applications_ 273 Elsevier, 2020, pp. 107101
  * [18] Paulina Szczuka  “Connections between connected topological spaces on the set of positive integers”  In _Central European Journal of Mathematics_ 11 Springer, 2013, pp. 876–881
  * [19] Paulina Szczuka  “The connectedness of arithmetic progressions in Furstenberg’s, Golomb’s, and Kirch’s topologies”  In _Demonstratio Mathematica_ 43.4 De Gruyter Open, 2010, pp. 899–910
  * [20] Paulina Szczuka  “The Darboux property for polynomials in Golomb’s and Kirch’s topologies”  In _Demonstratio Mathematica_ 46.2 De Gruyter Open, 2013, pp. 429–435
  * [21] Paulina Szyszkowska  “Increasing sequence of topologies on the set of positive integers”  In _Topology and its Applications_ 314 Elsevier, 2022, pp. 108096
  * [22] Paulina Szyszkowska  “Properties of increasing sequence of Kirch-type topologies on the set of positive integers”  In _Topology and its Applications_ 317 Elsevier, 2022, pp. 108188
  * [23] Fahed Zulfeqarr  “Some interesting consequences of Furstenberg topology”  In _Resonance_ 24.7 Springer, 2019, pp. 755–765


#  Sempart: Self-supervised Multi-resolution Partitioning of Image Semantics

Sriram Ravindran
Adobe
<EMAIL_ADDRESS>Debraj Basu
Adobe
<EMAIL_ADDRESS>
###### Abstract

Accurately determining salient regions of an image is challenging when labeled
data is scarce. DINO-based self-supervised approaches have recently leveraged
meaningful image semantics captured by patch-wise features for locating
foreground objects. Recent methods have also incorporated intuitive priors and
demonstrated value in unsupervised methods for object partitioning. In this
paper, we propose Sempart, which jointly infers coarse and fine bi-partitions
over an image’s DINO-based semantic graph. Furthermore, Sempart preserves fine
boundary details using graph-driven regularization and successfully distills
the coarse mask semantics into the fine mask. Our salient object detection and
single object localization findings suggest that Sempart produces high-quality
masks rapidly without additional post-processing and benefits from co-
optimizing the coarse and fine branches.

Figure 1: Each image is the original image with an overlayed saliency mask.
The first row of every column utilizes the ground truth saliency mask, the
second and third rows overlay with the self-supervised Sempart-coarse and
-fine masks on the same image respectively.

##  1 Introduction

Figure 2: Overview of Sempart: We refine the SSL features into co-optimized
low resolution coarse and high resolution fine masks, based on graph cut and
guided super-resolution respectively.

Identifying salient regions of an image prone to holding visual attention
remains a long-standing fuzzy problem [59] relying significantly on carefully
annotated data [51, 5, 54]. Recently self-supervised (SSL) mechanisms based on
large-scale pre-trained backbones [9, 6, 22], such as DINO [7], have
demonstrated increased capability in segmenting images [21, 30] and extracting
objects in the foreground [41, 39, 54, 4, 42].

The unavailability of labels is limiting to inferring high-quality object
masks. However, many recent methods have demonstrated that incorporating well-
informed priors into the partitioning process is significantly beneficial to
finding saliency regions and foreground objects in an unsupervised setting
[36, 41, 46, 47, 31, 54, 4, 39].

Different forms of statistical independence of the foreground have driven
recent approaches, with the most recent state-of-the-art focusing on
movability [4] of the salient object. Distinguishability and predictability of
the foreground from the background have also been successful indicators. For
example, statistical variations such as in color and texture of the foreground
minimally alter the overall distribution of the population [8]. Furthermore,
in-painting models such as MAE [22] have been particularly effective at
measuring predictability [36] and defining movability [4].

Inferring graph signals [32] for partitioning a semantic graph over an image
has gained popularity [39, 54, 41, 30, 1], with recent methods establishing
surprisingly strong baselines using traditional techniques. In particular, the
solution to the relaxation of the NP-complete discrete normalized cut problem
[37] first demonstrated promise in unsupervised image segmentation, which has
further translated to recent findings in [39, 54, 40].

[20, 19] discuss the benefit of learning to predict spectral decomposition for
a graph and employ graph neural networks in a reinforcement learning setup for
predictively performing the normalized cut. More recently, [43] leveraged
normalized cut for regularizing a convolutional network driven by partial
cross entropy loss in a weakly supervised setting and demonstrated significant
performance improvement. More broadly, spectral partitioning of semantic
graphs [39, 54, 30, 1] has become an emerging underlying theme for detecting
salient regions.

Contributions. In this paper, we propose Sempart, which builds on ideas from
[54, 12, 11] for producing high-quality foreground masks in an SSL setting.
Sempart learns a transformer-based encoder that refines the patchwise DINO
features for inferring a relaxation of graph cut that minimizes the expected
normalized cut loss [20] over a semantic graph informed by DINO feature
correspondences.

As seen in [54, 41, 7], the foreground masks obtained abandon the fine
boundary details from processing features at a low resolution. Unlike [39, 54,
41], which perform successive refinement of the coarse masks post-inference,
Sempart implements a convolutional fine branch that processes and supplements
the transformed DINO features with RGB features at progressively increasing
resolutions for producing original resolution fine masks. Motivated by [12,
11], Sempart treats the coarse mask as the source and the image as a guide for
inferring high-quality fine masks (see Figure 1, Table 1) regularized by
weighted neighborhood-based graph total variation [48].

In summary, our contributions are as follows:

  * •

We propose a novel strategy for co-optimizing coarse and fine masks, that
decouples image partitioning into semantic separation of rich self-supervised
features and high-frequency detailing, respectively.

  * •

Sempart outperforms recent state-of-the-art methods in saliency detection by
3.7% in $\max F_{\beta}$ and 2.7% in IoU on average and emits high-quality
bounding boxes for locating objects.

  * •

Sempart produces high-quality fine masks rapidly by eliminating time-consuming
post-inference iterative refinement and saving 200ms on average.

##  2 Related work

Vision systems have historically benefited from segmenting a scene into
objects constituting salient regions [51]. Supervised mechanisms [33, 58] have
dominated the landscape despite the prohibitive costs of obtaining labeled
data. Traditional unsupervised approaches [5, 29] have encoded beliefs about
the foreground region, such as differences in color and contrast and
objectness and depth perception, into partitioning techniques.

Spectral methods. Graph-based techniques have received interest wherein
spectral partitioning is undertaken over a graphical representation of an
image deduced from the priors. [37] proposed normalized cut as an improvement
over the min cut criterion [56], for producing clusters that are well
balanced. The relaxation of the discrete problem involved a spectral analysis
of the symmetrically normalized graph Laplacian

$\displaystyle L=I-D^{-\nicefrac{{1}}{{2}}}WD^{-\nicefrac{{1}}{{2}}}.$ (1)

The unavailability of effective semantic similarity measures between regions
of an image for populating the adjacency matrix $W$ inhibited the quality of
resulting partitions.

Self-supervised representations. With the emergence of deep techniques for
learning contextually aware representations [7, 22, 9, 6],many of these
traditional prior-based techniques have demonstrated increased effectiveness
and therefore received renewed interest. The semantically aware DINO [7]
features were used for implementing seed expansion into salient regions,
initialized with patches that are least similar to other patches as seed in
LOST [41]. On the contrary, FOUND [42] locates a background seed first and
then expands it. In [53], a SOLO [52] model is trained on coarse masks
extracted using SSL features, for instance segmentation.

The memory bottleneck of attention mechanism [14] prevents low-resolution deep
SSL features from capturing high-frequency details of an image which often are
only helpful in predicting coarse masks [41, 39, 54]. Therefore, despite
significant performance gains, these methods require computationally heavy
post-processing [3, 25, 26] to generate high-quality fine masks.

Inpainting as a helpful object detection tool was first proposed in [36],
which hypothesized that it is difficult to predict the foreground given a
background and vice versa. SSL features from masked autoencoder (MAE [22])
were also leveraged by recent state-of-the-art MOVE [4] for adversarially
training a convolutional mask generator for distinguishing between real- and
fake-inpainted images based on movability of salient objects. MOVE established
superiority in detecting both salient regions as well as single objects. The
movability criterion allows MOVE to directly predict saliency masks at a high
resolution which is also why it outperformed its counterparts without post-
processing.

SelfMask [39] uses multi-model SSL features [7, 9, 6] for populating $W$ and
constructs pseudo ground truth saliency masks for a subsequent MaskFormer [10]
training by clustering eigenvectors of the unnormalized graph Laplacian. Along
similar lines, [30] employs clustering based on normalized Laplacian for
semantic segmentation and object localization.

Our work is most closely related to [54, 20, 12, 11]. TokenCut [54] makes the
bi-partitioning mathematically precise by using the eigenvector with the
second smallest eigenvalue, which corresponds to a relaxation of the
normalized cut [37] problem and demonstrates value in pursuing graph-based
techniques for detecting salient regions.

Iterative computations during inference with expensive post-processing [54,
30], or otherwise training in two stages leveraging multiple SSL models [39]
for improving performance, can be limiting. To alleviate this, we follow
MOVE’s approach of training a single bi-partitioning model as a transformation
of the DINO backbone (see Figure 2) and encode our novel strategies into the
loss functions (see Section 3.2, Section 3.3). The Sempart architecture
involves a fine branch inspired by graph-driven iterative techniques for
super-resolution [12, 11] for predicting accurate high-resolution masks.

Minimizing expected graph cut losses over a population was previously
evaluated in [20, 19, 1], which proposed to optimize expected normalized cut
using graph neural networks. We show that Sempart exhibits similar benefits
(see Table 1, Table 3) from jointly inferred graph-driven bi-partitioning and
graph regularized guided super-resolution for generating high-fidelity
saliency masks rapidly without any post-processing or multi-stage training.

##  3 Approach

In this work, we detect salient regions and localize single objects within an
image by learning to partition the image into two regions that are
semantically less related [54, 21, 39, 1]. We leverage DINO [7], which
provides effective pre-trained SSL feature correspondences [54, 21, 30] for
learning a coarse binary mask that partitions a semantic graph constructed
between image patches as nodes. Motivated by image-guided super-resolution
[12] and graph regularization [11, 48], we co-optimize and infer masks at the
original resolution in parallel, thereby correcting a coarse mask’s
inaccuracies, preserving fine boundary details.

###  3.1 Background

Normalized Cut. The normalized cut [37] of a weighted undirected complete
graph $G=(V,E,w)$ where $w_{ij}>0$ denotes the weight of $(i,j)\in E$, is
given by a binary graph signal $s:v\in V\rightarrow s(v)\in\\{0,1\\}$ that
minimizes

$\displaystyle\text{Ncut}(A,B)=\frac{w(A,B)}{w(A,V)}+\frac{w(B,A)}{w(B,V)}$
(2)

where $A\coloneqq\\{v|v\in V,s(v)=0\\}$, $B\coloneqq\\{v|v\in V,s(v)=1\\}$ and
$w(A,B)\coloneqq\sum_{s(i)=0,s(j)=1}w_{i,j}$.

Being NP-complete, Shi et al. [37] first proposed to solve a relaxation which
amounts to solving a generalized eigensystem followed by discretization. More
recently, the relaxation of (6) has been effective at semantically segmenting
images in a self-supervised manner [54]. Motivated by [20, 19], non-linear
parameterizations of the graph signal have enabled deep partitioning [1] and
regularization [43] based on normalized cut.

Deep self-supervised feature correspondences. Large-scale pre-trained self-
supervised image embedders such as DINO [7], MAE [22], MoCo [9], SwAV [6]
possess beneficial emergent properties for downstream tasks [41, 54, 4, 39,
21]. These models are based on vision transformers [15], which generate an
embedding for each patch. Specifically, given an image of dimensions $C\times
H\times W$, and an SSL embedder operating with patch size $p$, we obtain a
tensor of size $\mathit{D\times(H/p\times W/p+1)}$, including the embedding
for the [CLS] token that represents the entire image. In this paper, we
leverage DINO as it emits semantically relevant embeddings [7, 54, 41, 42,
21].

In particular, [54] computed an affinity matrix using the feature
correspondences from DINO. A graph view of the output is considered where the
graph $G=(V,E)$ contains patches $V$, and connections between any two patches
are encoded in the edge list $E$. Each patch $v\in V$ has an associated
normalized DINO embedding $F_{v}$. The affinity matrix is given by the feature
correspondences,

$\displaystyle W_{ij}=\left\\{\begin{array}[]{l}1\mid\langle
F_{v_{i}},F_{v_{j}}\rangle>\tau\\\ \epsilon\mid otherwise.\end{array}\right.$
(5)

###  3.2 Self-supervised multi-resolution partitioning (Sempart)

We propose Sempart, which converts an image into a semantic graph $G$ over
non-overlapping patches, which form the set of nodes $V$. Sempart’s
architecture (see Figure 2) has two main branches that infer a coarse and fine
mask jointly, which are informed by normalized cut and image-guided super-
resolution, respectively. We posit that guided super-resolution not only
refines the coarse mask into a fine mask by preserving high-resolution
details. It also helps regularize the overall learning and justifies our co-
optimization strategy.

Normalized cut for coarse mask.

|  Model | DUT-OMRON [57] | DUTS-TE [49] | ECSSD [38]
---|---|---|---|---
| Acc | IoU | $\mathbf{maxF_{\beta}}$ | Acc | IoU | $\mathbf{maxF_{\beta}}$ | Acc | IoU | $\mathbf{maxF_{\beta}}$
Method | LOST [41] | .797 | .410 | .473 | .871 | .518 | .611 | .895 | .654 | .758
TokenCut [54] | .880 | .533 | .600 | .903 | .576 | .672 | .918 | .712 | .803
FreeSOLO [53] | .909 | .560 | .684 | .924 | .613 | .750 | .917 | .703 | .858
MOVE [4] | .923 | .615 | .712 | .950 | .713 | .815 | .954 | .830 | .916
Sempart-Coarse | .932 | .640 | .755 | .956 | .727 | .864 | .961 | .837 | .943
Sempart-Fine | .932 | .668 | .764 | .959 | .749 | .867 | .964 | .855 | .947
\+ BF | LOST+BF | .818 | .489 | .578 | .887 | .572 | .697 | .916 | .723 | .837
TokenCut+BF | .897 | .618 | .697 | .914 | .624 | .755 | .934 | .772 | .874
MOVE+BF | .931 | .636 | .734 | .951 | .687 | .821 | .953 | .801 | .916
Sempart-Coarse+BF | .934 | .661 | .764 | .957 | .697 | .858 | .960 | .820 | .932
Sempart-Fine+BF | .933 | .653 | .760 | .955 | .685 | .853 | .959 | .816 | .931
\+ SelfMask |  SelfMask on pseudo + BF [39] | .919 | .655 | (.774)* | .933 | .660 | (.819)* | .955 | .818 | (.911)*
SelfMask on MOVE | .933 | .666 | .756 | .954 | .728 | .829 | .956 | .835 | .921
SelfMask on MOVE + BF | .937 | .665 | .766 | .952 | .687 | .827 | .952 | .800 | .917
SelfMask on Sempart-Coarse | .936 | .675 | .773 | .958 | .743 | .872 | .962 | .843 | .938
SelfMask on Sempart-Fine | .942 | .698 | .799 | .958 | .749 | .879 | .963 | .850 | .944
2 -Net (supervised) | .928 | .693 | .771 | .943 | .733 | .822 | .967 | .878 | .947 |

* The $\max F_{\beta}$ for SelfMask on pseudo + BF is reported within brackets (), from the reevaluation in [4] upon confirming that it was originally calculated incorrectly [4, 42].

Table 1: Quantitative comparison of Sempart with state-of-the-art MOVE and
other related works for saliency detection. Sempart-Coarse and -Fine
outperform MOVE significantly in all three evaluation categories (Method, +BF,
+SelfMask) across all datasets. The best-performing method in a category and
across categories is in bold and underlined, respectively.

A frozen DINO backbone transforms the input image $X\in\mathbb{R}^{3\times
320\times 320}$ into low-resolution SSL features $F\in\mathbb{R}^{64\times
40\times 40}$. We apply a single layer transformer encoder with two attention
heads, followed by a coarse branch (see Figure 2) comprised of a linear
classification head, for transforming the low resolution features into a
coarse saliency mask in the form of a soft partitioning indicator vector
$S_{\text{coarse}}\in[0,1]^{|V|}$ where $|V|=40\times 40$. For partitions A
and B with their indicator vectors $S_{A}=S_{\text{coarse}}$ and
$S_{B}=1-S_{A}$, (2) is rewritten as

$\displaystyle\mathcal{L}_{\text{Ncut}}(X)\coloneqq\text{Ncut}(A,B)=\sum_{i\in\\{A,B\\}}\frac{S_{i}^{T}W(1-S_{i})}{{S_{i}}^{T}W\mathbf{1}}.$
(6)

This results in a coarse mask at $40\times 40$, which amplifies the semantic
distinguishability between the two partitions where the affinity between image
patches $i$ and $j$ is computed using the DINO embeddings in (5) and denoted
by $W_{ij}$. Upon minimizing this heuristic over the entire population, we see
a significant improvement in performance over solving the generalized
eigensystem in [54] (see Table 1).

Guided super-resolution for fine mask. The generated coarse mask often fails
to capture finer high-frequency details [54, 12] at the original image
resolution, which is detrimental to the performance in detecting salient
regions. Previously, such methods have employed expensive iterative post-
processing such as Bilateral Filtering [3, 39, 41, 54] or CRF [25, 21] for
every inferenced image. These methods utilize pixels’ color and positional
information to readjust the generated coarse masks. The possibility of erosion
of the mask has been discussed as a limitation in [4].

By delegating the generation of linearly separable semantic features to the
coarse branch, our architecture enables a refinement network to exclusively
focus on detailing and denoising at higher frequencies and around the edges.
We jointly optimize a fine branch (see Figure 2) comprised of a convolutional
mask refinement network inspired by a recent guided super-resolution technique
[12] which trains a multi-layer perceptron for enhancing the mask with
guidance from the image. While [12] performs iterative refinement per image,
we co-optimize our refinement network for predicting a fine mask which aligns
with the coarse mask (see Figure 1).

The output from the transformer encoder layer is gradually scaled up from
$40\times 40$ to $320\times 320$ in 3 steps. In each step, the image is first
scaled up $2\times$ using bilinear interpolation and processed through a
convolutional block described in Suppl. Note that we also concatenate the
appropriately resized input image to the input of each convolutional block.
This information is pertinent for conditioning the fine branch to satisfy the
regularization in Section 3.3.

The features $\widehat{F}\in\mathbb{R}^{131\times 320\times 320}$ from the
last convolutional block are linearly classified into
$S_{\text{fine}}\in[0,1]^{320\times 320}$ which is subsequently average pooled
to $\widehat{S}_{\text{fine}}\in[0,1]^{40\times 40}$ for aligning with the
$S_{\text{coarse}}\in[0,1]^{40\times 40}$. The corresponding loss function is
given as

$\displaystyle\mathcal{L}_{\text{SR}}(X)\coloneqq\|\widehat{S}_{\text{fine}}-S_{\text{coarse}}\|_{2}^{2}.$
(7)

###  3.3 Graph total variation regularization (GTV)

Graph-based regularization has yielded benefits in capturing high-frequency
details of an image in [11, 12]. A similarity metric between pixels of an
image $X$ is used to populate the affinity matrix $A>0$, which is then used to
compute the degree matrix $D$. The graph Laplacian $L=D-A$ is used to compute
the graph regularizer as the quadratic form for a graph signal [32] $s$, given
by

$\displaystyle\mathcal{L}_{reg}=\frac{1}{2}\sum_{(i,j)\in
E}A_{ij}(s(i)-s(j))^{2}.$ (8)

Considering significant computational complexity from the total number of
pairs of pixels, we enforce $A_{ij}=0$ when pixels $X_{i}$ and $X_{j}$ are not
vertically or horizontally adjacent, also known as the pixel neighborhood
$\mathcal{N}$. This is equivalent to a weighted version of the total variation
(TV) loss [28, 16], which has been previously used for denoising images and
other signals [2, 23, 16, 34]. A natural extension to graphs is discussed in
[48].

GTV fine. The guided super-resolution can result in more than one fine mask
for a given coarse mask, which is where our graph total variation (GTV) loss
not only works as a denoiser but plays a more important role as a regularizer.
More specifically, $A_{ij}=\exp{\left(-\|X_{i}-X_{j}\|_{2}^{2}/\sigma\right)}$
is given by the euclidean similarity between the pairwise pixels. As a result,
the $\mathcal{L}_{\text{GTV-fine}}$ loss encourages the upsampler along the
fine branch in Figure 2 to leverage the color information.

GTV coarse. We also implement a similar graph TV regularizer denoted by
$\mathcal{L}_{\text{GTV-coarse}}$ for the coarse mask based on
$A_{ij}=W_{ij}\mathbf{1}\\{i\in\mathcal{N}(j)\\}$ where $W_{ij}$ is as defined
in (5). This is responsible for denoising and predicting a smooth coarse mask.

Figure 3: Qualitative comparison of Sempart-coarse and -fine with TokenCut
[54] and MOVE [4] for samples from DUT-OMRON [57].

###  3.4 Loss formulation

The Sempart losses in Section 3.2 together with the GTV losses in Section 3.3
drive the joint learning of coarse and fine masks. While the Sempart losses
are driven by DINO feature correspondences for inferring accurate image
partitions, the GTV losses are significantly involved in denoising the
predicted masks and regularizing the overall learning process. The loss
functions for the coarse and fine branches, respectively, are,

$\displaystyle\mathcal{L}_{\text{coarse}}(x)$
$\displaystyle=\mathcal{L}_{\text{Ncut}}(x)+\lambda_{\text{GTV-
coarse}}\mathcal{L}_{\text{GTV-coarse}}(x)$
$\displaystyle\mathcal{L}_{\text{fine}}(x)$ $\displaystyle=\lambda_{\text{GTV-
fine}}\mathcal{L}_{\text{GTV-fine}}(x)$
$\displaystyle\mathcal{L}_{\text{joint}}(x)$
$\displaystyle=\lambda_{\text{SR}}\mathcal{L}_{\text{SR}}(x).$ (9)

This gives us our final expected self-supervised loss function
$\mathcal{L}_{\textsc{Sempart}}=\underset{x\sim\mathbb{P}(X)}{\mathbb{E}}[\mathcal{L}_{\text{coarse}}(x)+\mathcal{L}_{\text{fine}}(x)+\mathcal{L}_{\text{joint}}(x)]$.

##  4 Experiments

As done in [54, 4], we evaluate Sempart on unsupervised saliency segmentation
and single object detection.

###  4.1 Implementation

In our work, we use the self-supervised [7] ViT-s/8 transformer from the
official implementation of DINO [7]. DINO uses an $8\times 8$ non-overlapping
patch on a $3\times 320\times 320$ input and emits $384\times 40\times 40$
output which is provided to our simple transformer encoder layer and then
routed through both the coarse and fine branches in Figure 2. We employ Adam
optimizer [24] with a learning rate of $0.0001$ and $\beta=(0.9,0.999)$. We
implemented Sempart in PyTorch and trained our models for 20 epochs with a
batch size of 8 on a single NVIDIA Tesla P40 GPU. Following careful
consideration, hyperparameters $\lambda_{\text{GTV-coarse}}=0.0006$,
$\lambda_{\text{SR}}=20$, $\lambda_{\text{GTV-fine}}=0.0002$ have been applied
for all results of Sempart.

Graph affinity. (a) Normalized cut. Our implementation follows [54] in
computing the affinity matrix $W$ based on (5) with a minor deviation. We set
$W_{ii}=0$ to discard self-loops that do not belong to a graph cut. We show
empirically that this improves model performance. Additionally we set
$\tau=0.2$ and $\epsilon=$1e-6 in (5) for the $\mathcal{L}_{\text{Ncut}}$
loss. (b) GTV Coarse. In addition to details provided in Section 3.3, we set
$\tau=0$ and $\epsilon=$1e-6 for numerical stability. (c) GTV Fine.
$\mathcal{L}_{\text{GTV-fine}}$ regularizes the fine mask by limiting the
possible solutions. The convolutional blocks learn to generate features that
leverage both the contextual features from the transformer encoder and the RGB
image features for predicting fine masks that mimic the coarse mask but also
preserve the high-frequency image details.

In addition to details provided in Section 3.3, we also set $\sigma=1$.

Foreground selection. We first binarize the indicator vector with threshold =
$0.5$. In order to pick the foreground, we consider four strategies. (a)
Select the patch with a lower average distance to the center as the
foreground. (b) Discarding partitions with full spatial width or height as
background, selecting the smaller partition to break a tie. (c) Select the
partition with greatest attention from the last layer of DINO. (d) Select the
partition occupying the least number of corners. If there is a tie, select the
smaller partition.

###  4.2 Unsupervised saliency segmentation

Method |  Avg. Time |  Model |  RES |  GPU |  CPU
---|---|---|---|---|---
TokenCut | 130ms | No | Low | Yes | Yes
TokenCut+BF | 337ms | No | High | Yes | Yes
MOVE | 13ms | Yes | High | Yes | No
Sempart | 14ms | Yes | High | Yes | No

Table 2: Both Sempart and MOVE train a model, generate high-resolution masks,
and have comparable average inference times per image.

Datasets. As done in [4, 1, 39], we trained Sempart on the train split of DUTS
[49], known as DUTS-TR and evaluate the performance of our model on the
corresponding test split DUTS-TE [49], as well as DUT-OMRON [57] and ECSSD
[38]. DUTS-TR contains 10,553 images, DUTS-TE contains $5,019$ images, DUT-
OMRON contains 5,168 images, and ECSSD contains 1000 images.

Evaluation. As done in [4, 54], we compute the per-pixel mask accuracy (Acc),
intersection over union (IoU), and $\max F_{\beta}$ [54] for evaluating the
performance of Sempart. Accuracy is the fraction of pixels correctly predicted
into the foreground or background. The overlap between the binary saliency
mask and the ground truth gives IoU. We set $\beta=0.3$ as per [4, 54] where
$\max F_{\beta}$ is given for the threshold used for binarizing the mask that
maximizes $F_{\beta}$.

Results. We compared the performance of Sempart with recent state-of-the-art
MOVE [4] and several other standard baselines referenced therein. Table 1
contains three horizontal sections corresponding to the baseline method,
followed by applying a bilateral filtering [3] step. The final section
involves generating pseudo ground truth based on the baseline method and
training MaskFormer [10] in a class agnostic manner, as in [39].

We observe that applying the bilateral filter after inferencing Sempart on a
per-image basis is detrimental to the overall performance, as is also seen in
[4], with the performance of Sempart-Fine deteriorating significantly.

Sempart significantly outperformed all other baselines in all three sections
across all datasets. Although Sempart is primarily motivated by the normalized
cut minimization in [54], the expected normalized cut loss in Section 3.4 co-
optimized with the image-guided graph-based super-resolution loss results in
significant improvement in performance. As seen in Figure 3, the per-image
optimization in TokenCut selects regions that are not salient or present in
the foreground.

Figure 4: Sempart for single object detection. Green boxes are ground truth
bounding boxes, and the red box is our predicted bounding box. Intersection
area is highlighted.

Sempart significantly outperforms the movability [4] heuristic in all three
sections for all datasets. From Figure 3, we find that MOVE may include
multiple semantically unrelated patches into the movable object mask.
Additionally, we note that MOVE greatly relies on retraining according to
SelfMask [39] for outperforming previous state-of-the-art. While Sempart-
Coarse predicts noisy masks (see Figure 3-A,C,D, and E) with slight errors as
seen in the last example, Sempart-Fine results in refinement with improved
ground truth alignment.

###  4.3 Single object detection

Datasets. We evaluate our model on three datasets - the train split of COCO20K
[27] and the training and validation splits of VOC07 [17] and VOC12 [18]. Each
image in these datasets has one or more bounding boxes corresponding to each
object. The objective is to localize any single object.

Evaluation. We detect connected components for separating multiple objects for
an image’s Sempart mask111If multiple objects lie in a component this
evaluation is less reliable.. The component with the largest bounding box is
used as the object prediction. Suppose the highest IoU between our predicted
bounding box and all ground truth bounding boxes exceeds $0.5$. In that case,
we treat it as a successful prediction and use this to compute Correct
Localization (CorLoc) metric which is simply the accuracy of prediction.

Results. Sempart results in superior bounding-boxes which perform comparably
with state-of-the-art MOVE, outperforming it on COCO20k dataset (see Table 3).
Our findings suggest that increasing $\tau$ to $0.25$ helps us prevent co-
located disparate objects from lying in the same connected component and
results in a slight improvement.

Method |  VOC07 |  VOC12 |  COCO20K
---|---|---|---
DDT+ [55] | 50.2 | 53.1 | 38.2
rOSD [44] | 54.5 | 55.3 | 48.5
LOD [45] | 53.6 | 55.1 | 48.5
FreeSOLO [53] | 56.1 | 56.7 | 52.8
LOST [41] | 61.9 | 64.0 | 50.7
Deep Spectral [30] | 62.7 | 66.4 | 52.2
TokenCut [54] | 68.8 | 72.1 | 58.8
MOVE [4] | 76.0 | 78.8 | 66.6
Sempart-Coarse | 74.7 | 77.4 | 66.9
Sempart-Fine | 75.1 | 76.8 | 66.4

Table 3: Sempart bounding boxes exhibits a high CorLoc comparable to state-of-
the-art MOVE [4], for single object discovery on VOC2007 [17], VOC2012[18] and
outperforms it on COCO20K [27] dataset.

Method | OMRON* | D-TE* | ECSSD
---|---|---|---
fs: framing prior | 0.663 | 0.730 | 0.825
fs: centrality | 0.652 | 0.736 | 0.854
fs: total attention | 0.668 | 0.745 | 0.853
w/ self-loops in $W$ | 0.667 | 0.743 | 0.846
w/o GTV coarse | 0.646 | 0.749 | 0.848
w/o GTV fine | 0.637 | 0.717 | 0.818
train fine mask directly | 0.645 | 0.738 | 0.845
w/o joint training | 0.662 | 0.743 | 0.849
Sempart-Fine | 0.668 | 0.749 | 0.855

Table 4: Ablations of Sempart for saliency, using mIoU. *Shorthand has been
used due to space constraints; OMRON refers to DUT-OMRON [57] and D-TE refers
to DUTS-TE [49]; fs denotes foreground selection.

Figure 5: Limitations of Sempart. Human bias towards humans and moving objects
are shown in A and B. Sempart cannot capture the intricate details and smooths
over narrow regions in B and C. An immovable background object is included,
which is not as visually salient as the rooster in D. The crib is the same
color as the wall in E; therefore, the toys are prominent. However, DINO
highlights the semantic differences for partitioning the entire crib from the
background.

###  4.4 Ablations

We ablated Sempart for saliency segmentation as follows,

Foreground selection. Unlike [4], where the foreground is given by the movable
object, Sempart selects partitions based on occupying least corners given by
Sempart-Fine in Table 4. Motivated by [39, 41], we compare with selection
based on closeness to the image center (centrality), as well as the framing
prior [39], which labels the segment occupying full spatial width or height as
background while breaking ties based on selecting the smaller partition as
foreground. Another heuristic that is a close contender to least corners is
total attention, in which the partition having the highest total overlap with
the DINO [CLS] token attention map as foreground.

Self-loops. We populate $W_{ii}$ with (5) instead of 0 for
$\mathcal{L}_{\text{Ncut}}$ and demonstrate that the performance deteriorates.

Graph TV regularization. Sempart without either $\mathcal{L}_{\text{GTV-
coarse}}$ or $\mathcal{L}_{\text{GTV-fine}}$ is detrimental to performance.
The absence of the GTV-fine loss has a greater negative impact.

Training fine mask directly. We evaluate a setting where we only have a fine
branch (see Figure 2), and the $\mathcal{L}_{\text{Ncut}}$ and
$\mathcal{L}_{\text{GTV-fine}}$ losses. Table 4 demonstrates that this is
inferior to Sempart despite being almost equivalent in the number of
parameters. We attribute this to the absence of the coarse branch and the
corresponding $\mathcal{L}_{\text{Ncut}}$ loss which in turn regularized the
transformer encoder for subsequent consumption by the convolutional blocks.

Joint training. We evaluate a variant of Sempart, where the coarse and fine
branch are trained independently. While the $\mathcal{L}_{\text{coarse}}$ only
optimizes the coarse branch and the transformer encoder (see deviations from
Figure 2 in Suppl.), the gradients from $\mathcal{L}_{\text{fine}}$ and
$\mathcal{L}_{\text{joint}}$ are prohibited from optimizing these modules. As
seen in Table 4, this is detrimental to performance on all datasets, verifying
our hypothesis that co-optimizing coarse and fine mask is mutually beneficial.

###  4.5 Limitations

Visual saliency is not agnostic to various human biases in favor of humans and
animals, as well as objects which are likely to move in a subsequent frame or
have high contrast with the background. Figure 5 discusses examples where the
ground truth favors a human, a train crossing a bridge, and a rooster over all
other objects. Sempart results in over-selection here as it does not
explicitly incorporate these priors or even control the object size.
Furthermore, our graph TV loss can sometimes merge narrow co-located regions
into the mask, as seen in the Figure 5-B and C, which can also be detrimental
to localizing objects.

##  5 Conclusion

Sempart demonstrates the efficacy of graph-driven objectives towards self-
supervised image partitioning and establishes state-of-the-art performance for
detecting salient regions and a competitive advantage in localizing objects.
We address the limitations of expensive post-processing, limited resolution,
and noise artifacts in saliency masks. We demonstrate the value of a joint
learning paradigm for inferring high-quality masks at multiple resolutions
using Sempart, which will hopefully be a vital enabler of the subsequent
investigation into class-aware object detection for diverse vision systems.

Acknowledgements The authors gratefully thank Ambareesh Revanur and Deepak Pai
for their valuable feedback, and the anonymous reviewers for their comments.

## References

  * [1] Amit Aflalo, Shai Bagon, Tamar Kashti, and Yonina C. Eldar.  Deepcut: Unsupervised segmentation using graph neural networks clustering.  CoRR, abs/2212.05853, 2022.
  * [2] William K. Allard.  Total variation regularization for image denoising, i. geometric theory.  SIAM Journal on Mathematical Analysis, 39(4):1150–1190, 2008.
  * [3] Jonathan T. Barron and Ben Poole.  The fast bilateral solver.  In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III, volume 9907 of Lecture Notes in Computer Science, pages 617–632. Springer, 2016.
  * [4] Adam Bielski and Paolo Favaro.  MOVE: unsupervised movable object segmentation and detection.  CoRR, abs/2210.07920, 2022.
  * [5] Ali Borji, Ming-Ming Cheng, Huaizu Jiang, and Jia Li.  Salient object detection: A survey.  CoRR, abs/1411.5878, 2014.
  * [6] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin.  Unsupervised learning of visual features by contrasting cluster assignments.  In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
  * [7] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.  Emerging properties in self-supervised vision transformers.  In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 9630–9640. IEEE, 2021.
  * [8] Mickaël Chen, Thierry Artières, and Ludovic Denoyer.  Unsupervised object segmentation by redrawing.  In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 12705–12716, 2019.
  * [9] Xinlei Chen, Haoqi Fan, Ross B. Girshick, and Kaiming He.  Improved baselines with momentum contrastive learning.  CoRR, abs/2003.04297, 2020.
  * [10] Bowen Cheng, Alexander G. Schwing, and Alexander Kirillov.  Per-pixel classification is not all you need for semantic segmentation.  In Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 17864–17875, 2021.
  * [11] Riccardo de Lutio, Alexander Becker, Stefano D’Aronco, Stefania Russo, Jan D. Wegner, and Konrad Schindler.  Learning graph regularisation for guided super-resolution.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 1969–1978. IEEE, 2022.
  * [12] Riccardo de Lutio, Stefano D’Aronco, Jan Dirk Wegner, and Konrad Schindler.  Guided super-resolution as pixel-to-pixel transformation.  In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pages 8828–8836. IEEE, 2019.
  * [13] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  * [14] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: pre-training of deep bidirectional transformers for language understanding.  CoRR, abs/1810.04805, 2018.
  * [15] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale.  In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
  * [16] Vania Vieira Estrela, Hermes Aguiar Magalhaes, and Osamu Saotome.  Total variation applications in computer vision.  CoRR, abs/1603.09599, 2016.
  * [17] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman.  The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results.  http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html.
  * [18] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman.  The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results.  http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html.
  * [19] Alice Gatti, Zhixiong Hu, Tess Smidt, Esmond G. Ng, and Pieter Ghysels.  Graph partitioning and sparse matrix ordering using reinforcement learning and graph neural networks.  Journal of Machine Learning Research, 23(303):1–28, 2022.
  * [20] Alice Gatti, Zhixiong Hu, Tess E. Smidt, Esmond G. Ng, and Pieter Ghysels.  Deep learning and spectral embedding for graph partitioning.  In Xiaoye S. Li and Keita Teranishi, editors, Proceedings of the 2022 SIAM Conference on Parallel Processing for Scientific Computing, PPSC 2022, Seattle, WA, USA, February 23-26, 2022, pages 25–36. SIAM, 2022.
  * [21] Mark Hamilton, Zhoutong Zhang, Bharath Hariharan, Noah Snavely, and William T. Freeman.  Unsupervised semantic segmentation by distilling feature correspondences.  In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
  * [22] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick.  Masked autoencoders are scalable vision learners.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 15979–15988. IEEE, 2022.
  * [23] Xianxu Hou, Linlin Shen, Or Patashnik, Daniel Cohen-Or, and Hui Huang.  Feat: Face editing with attention, 2022.
  * [24] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
  * [25] Philipp Krähenbühl and Vladlen Koltun.  Efficient inference in fully connected crfs with gaussian edge potentials.  In John Shawe-Taylor, Richard S. Zemel, Peter L. Bartlett, Fernando C. N. Pereira, and Kilian Q. Weinberger, editors, Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011. Proceedings of a meeting held 12-14 December 2011, Granada, Spain, pages 109–117, 2011.
  * [26] John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira.  Conditional random fields: Probabilistic models for segmenting and labeling sequence data.  In Carla E. Brodley and Andrea Pohoreckyj Danyluk, editors, Proceedings of the Eighteenth International Conference on Machine Learning (ICML 2001), Williams College, Williamstown, MA, USA, June 28 - July 1, 2001, pages 282–289. Morgan Kaufmann, 2001.
  * [27] Tsung-Yi Lin, Michael Maire, Serge J. Belongie, Lubomir D. Bourdev, Ross B. Girshick, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll’a r, and C. Lawrence Zitnick.  Microsoft COCO: common objects in context.  CoRR, abs/1405.0312, 2014.
  * [28] Jiaming Liu, Yu Sun, Xiaojian Xu, and Ulugbek S. Kamilov.  Image restoration using total variation regularized deep image prior.  CoRR, abs/1810.12864, 2018.
  * [29] Tie Liu, Jian Sun, Nanning Zheng, Xiaoou Tang, and Heung-Yeung Shum.  Learning to detect A salient object.  In 2007 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2007), 18-23 June 2007, Minneapolis, Minnesota, USA. IEEE Computer Society, 2007.
  * [30] Luke Melas-Kyriazi, Christian Rupprecht, Iro Laina, and Andrea Vedaldi.  Deep spectral methods: A surprisingly strong baseline for unsupervised semantic segmentation and localization.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 8354–8365. IEEE, 2022.
  * [31] Luke Melas-Kyriazi, Christian Rupprecht, Iro Laina, and Andrea Vedaldi.  Finding an unsupervised image segmenter in each of your deep generative models.  In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
  * [32] Antonio Ortega, Pascal Frossard, Jelena Kovačević, José M. F. Moura, and Pierre Vandergheynst.  Graph signal processing: Overview, challenges, and applications.  Proceedings of the IEEE, 106(5):808–828, 2018.
  * [33] Xuebin Qin, Zichen Zhang, Chenyang Huang, Masood Dehghan, Osmar R. Zaïane, and Martin Jägersand.  U${}^{\mbox{2}}$-net: Going deeper with nested u-structure for salient object detection.  Pattern Recognit., 106:107404, 2020.
  * [34] Ambareesh Revanur, Debraj Basu, Shradha Agrawal, Dhwanit Agarwal, and Deepak Pai.  Coralstyleclip: Co-optimized region and layer selection for image editing, 2023.
  * [35] Denise Rey and Markus Neuhäuser.  Wilcoxon-signed-rank test.  In International Encyclopedia of Statistical Science, 2011.
  * [36] Pedro Savarese, Sunnie S. Y. Kim, Michael Maire, Greg Shakhnarovich, and David McAllester.  Information-theoretic segmentation by inpainting error maximization.  In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 4029–4039. Computer Vision Foundation / IEEE, 2021.
  * [37] Jianbo Shi and Jitendra Malik.  Normalized cuts and image segmentation.  IEEE Trans. Pattern Anal. Mach. Intell., 22(8):888–905, 2000.
  * [38] Jianping Shi, Qiong Yan, Li Xu, and Jiaya Jia.  Hierarchical image saliency detection on extended CSSD.  IEEE Trans. Pattern Anal. Mach. Intell., 38(4):717–729, 2016.
  * [39] Gyungin Shin, Samuel Albanie, and Weidi Xie.  Unsupervised salient object detection with spectral cluster voting.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2022, New Orleans, LA, USA, June 19-20, 2022, pages 3970–3979. IEEE, 2022.
  * [40] Gyungin Shin, Weidi Xie, and Samuel Albanie.  Namedmask: Distilling segmenters from complementary foundation models.  CoRR, abs/2209.11228, 2022.
  * [41] Oriane Siméoni, Gilles Puy, Huy V. Vo, Simon Roburin, Spyros Gidaris, Andrei Bursuc, Patrick Pérez, Renaud Marlet, and Jean Ponce.  Localizing objects with self-supervised transformers and no labels.  In 32nd British Machine Vision Conference 2021, BMVC 2021, Online, November 22-25, 2021, page 310. BMVA Press, 2021.
  * [42] Oriane Siméoni, Chloé Sekkat, Gilles Puy, Antonín Vobecký, Éloi Zablocki, and Patrick Pérez.  Unsupervised object localization: Observing the background to discover objects.  CoRR, abs/2212.07834, 2022.
  * [43] Meng Tang, Abdelaziz Djelouah, Federico Perazzi, Yuri Boykov, and Christopher Schroers.  Normalized cut loss for weakly-supervised CNN segmentation.  In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 1818–1827. Computer Vision Foundation / IEEE Computer Society, 2018.
  * [44] Huy V. Vo, Patrick Pérez, and Jean Ponce.  Toward unsupervised, multi-object discovery in large-scale image collections.  CoRR, abs/2007.02662, 2020.
  * [45] Van Huy Vo, Elena Sizikova, Cordelia Schmid, Patrick Pérez, and Jean Ponce.  Large-scale unsupervised object discovery.  In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 16764–16778. Curran Associates, Inc., 2021.
  * [46] Andrey Voynov and Artem Babenko.  Unsupervised discovery of interpretable directions in the GAN latent space.  In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 9786–9796. PMLR, 2020.
  * [47] Andrey Voynov, Stanislav Morozov, and Artem Babenko.  Object segmentation without labels with large-scale generative models.  In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 10596–10606. PMLR, 2021.
  * [48] Huy Vu, Gene Cheung, and Yonina C. Eldar.  Unrolling of deep graph total variation for image denoising.  In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2021, Toronto, ON, Canada, June 6-11, 2021, pages 2050–2054. IEEE, 2021.
  * [49] Lijun Wang, Huchuan Lu, Yifan Wang, Mengyang Feng, Dong Wang, Baocai Yin, and Xiang Ruan.  Learning to detect salient objects with image-level supervision.  In CVPR, 2017.
  * [50] Peng Wang, Jingdong Wang, Gang Zeng, Jie Feng, Hongbin Zha, and Shipeng Li.  Salient object detection for searched web images via global saliency.  In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 3194–3201, 2012.
  * [51] Wenguan Wang, Qiuxia Lai, Huazhu Fu, Jianbing Shen, and Haibin Ling.  Salient object detection in the deep learning era: An in-depth survey.  CoRR, abs/1904.09146, 2019.
  * [52] Xinlong Wang, Tao Kong, Chunhua Shen, Yuning Jiang, and Lei Li.  SOLO: segmenting objects by locations.  In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVIII, volume 12363 of Lecture Notes in Computer Science, pages 649–665. Springer, 2020.
  * [53] Xinlong Wang, Zhiding Yu, Shalini De Mello, Jan Kautz, Anima Anandkumar, Chunhua Shen, and Jose M. Alvarez.  Freesolo: Learning to segment objects without annotations.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 14156–14166. IEEE, 2022.
  * [54] Yangtao Wang, Xi Shen, Yuan Yuan, Yuming Du, Maomao Li, Shell Xu Hu, James L Crowley, and Dominique Vaufreydaz.  Tokencut: Segmenting objects in images and videos with self-supervised transformer and normalized cut.  arXiv preprint arXiv:2209.00383, 2022.
  * [55] Xiu-Shen Wei, Chen-Lin Zhang, Jianxin Wu, Chunhua Shen, and Zhi-Hua Zhou.  Unsupervised object discovery and co-localization by deep descriptor transforming, 2017.
  * [56] Zhenyu Wu and Richard M. Leahy.  An optimal graph theoretic approach to data clustering: Theory and its application to image segmentation.  IEEE Trans. Pattern Anal. Mach. Intell., 15(11):1101–1113, 1993.
  * [57] Chuan Yang, Lihe Zhang, Ruan Xiang Lu, Huchuan, and Ming-Hsuan Yang.  Saliency detection via graph-based manifold ranking.  In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pages 3166–3173. IEEE, 2013.
  * [58] Yi Ke Yun and Weisi Lin.  Selfreformer: Self-refined network with transformer for salient object detection.  CoRR, abs/2205.11283, 2022.
  * [59] Yuan Zhou, Ailing Mao, Shuwei Huo, Jianjun Lei, and Sun-Yuan Kung.  Salient object detection via fuzzy theory and object-level enhancement.  IEEE Transactions on Multimedia, 21(1):74–85, 2019.

##  Appendix A Notation

Figure 6: Expanded overview of Sempart: In addition to the details presented
in Figure 2, we zoom in to the transformer encoder in Figure 6 (a) and the
convolutional mask refinement network in Figure 6 (b). Block is as defined in
(12).

For an image $X\in\mathbb{R}^{3\times 320\times 320}$, we represent the self-
supervised features of DINO [7] obtained for all $8\times 8$ non overlapping
patches as $F\in\mathbb{R}^{384\times 40\times 40}$. We use $\mathcal{L}$ to
denote loss functions. $\mathbb{P}(M)$ and $\mathbb{E}[M]$ denote the
distribution and the expected value of the random variable $M$.
$\mathbf{1}\\{\cdot\\}$ is used to denote the indicator function.

For a graph, $G=(V,E)$, $V$, and $E$ denote the vertex and edge set,
respectively. $W$ and $A$ represent the adjacency or affinity matrix for the
$\mathcal{L}_{\text{Ncut}}$ in Section 3.2 and the GTV losses in Section 3.3
respectively.

$I$ denotes the identity matrix. $D$ and $L$ correspond to the degree matrix
and the Laplacian matrix for the graph $G$, respectively. $s:v\in V\rightarrow
s(v)\in R$ has been used to denote a scalar signal as a function defined over
the graph’s nodes $v\in V$ as the domain. The definition of $S$ naturally
follows as $S\coloneqq[s(1),s(2),\ldots,s(|V|)]^{T}$.

##  Appendix B Architecture for Section 3

Section 3 describes the essential details of the Sempart architecture in which
we emphasize the importance of two vital learnable components: (a) the
transformer encoder as a shared parametrized module between both the coarse
and fine branch, (b) the convolutional mask refinement network for generating
high resolution fine masks. Figure 6 (a) and (b) presents the transformer
encoder as well as the convolutional mask refinement network, respectively, in
greater detail. Furthermore, we also elaborate upon these individual modules
in Appendix C.

##  Appendix C Pseudocode for Section 3

Sempart is a self-supervised multi-resolution image bi-partitioning heuristic
that successfully distills the encoded information from DINO [7] towards high-
quality unsupervised semantically meaningful partitions that significantly
resonate with the notion of visual saliency for an image. In this section, we
elaborate upon the forward pass described in Section 3.2 to Section 3.4
culminating in Algorithm 1.

DINO backbone [7]: DINO [7] is a widely adopted self-supervised vision model
which emits features that are contextually aware and captures the semantic
richness of an image (see [7, Figure 1]). Sempart leverages the self-
supervised [7] ViT-s/8 transformer based on [15] from the official
implementation of DINO [7], which processes a $320\times 320$ image $X$ as a
$40\times 40$ positionally aware flattened sequence of $8\times 8$ non
overlapping patches. We denote the transformation by

$\displaystyle\textsc{DINO}(X):X\in\mathbb{R}^{3\times 320\times
320}\rightarrow F\in\mathbb{R}^{384\times 40\times 40}.$ (10)

Note that in fact DINO emits $\mathbb{R}^{384\times(1+40\times 40)}$, however
we discard the [CLS] token feature for subsequent modules. In our
implementation, the DINO backbone remains frozen.

Transformer encoder [15]: We apply a single layer transformer
encoder222Implementation is borrowed from [15]. with two attention heads that
transform $F\in\mathbb{R}^{384\times 40\times 40}$ to
$\widetilde{F}\in\mathbb{R}^{64\times 40\times 40}$.

$\displaystyle\widetilde{F}\leftarrow\textsc{TransformerEncoder}(F).$ (11)

Emitted features $\widetilde{F}$ are shared between both the Sempart-Coarse
and Sempart-Fine branches (see Figure 2).

Convolutional mask refinement network (Section 3.2): As also done in [4], we
define ${\textsc{Block}_{\text{out\\_ch}}^{\text{in\\_ch}}}$ as

$\displaystyle 3\times 3{\textsc{
Conv}_{\text{out\\_ch}}^{\text{in\\_ch}}}\rightarrow\textsc{BatchNorm}\rightarrow\textsc{LeakyReLU}$
(12)

where $K\times K{\textsc{ Conv}_{\text{out\\_ch}}^{\text{in\\_ch}}}$ is a
padded $K\times K$ convolution with stride = 1, $\text{in}\\_\text{ch}$ and
$\text{out}\\_\text{ch}$ correspond to the number of input and output channels
respectively. Before each block, we also concatenate - denoted by the $||_{c}$
operator - an appropriately resized image along the channel dimension.

Consequently, our convolutional mask refinement network is given by
alternating bilinear Upsample and Block as follows

$\displaystyle\widetilde{F}^{\prime}$
$\displaystyle\leftarrow{\textsc{Block}_{192}^{67}}\left[{\textsc{Upsample}_{\text{bilinear}}^{2\times
2}}\left(\widetilde{F}\right)||_{c}X^{3\times 80\times 80}\right]$
$\displaystyle\widetilde{F}^{\prime\prime}$
$\displaystyle\leftarrow{\textsc{Block}_{128}^{195}}\left[{\textsc{Upsample}_{\text{bilinear}}^{2\times
2}}\left(\widetilde{F}^{\prime}\right)||_{c}X^{3\times 160\times 160}\right]$
$\displaystyle\widetilde{F}^{\prime\prime\prime}$
$\displaystyle\leftarrow{\textsc{Block}_{128}^{131}}\left[{\textsc{Upsample}_{\text{bilinear}}^{2\times
2}}\left(\widetilde{F}^{\prime\prime}\right)||_{c}X^{3\times 320\times
320}\right]$ $\displaystyle\widehat{F}$
$\displaystyle\leftarrow{\textsc{Block}_{128}^{128}}\left(\widetilde{F}^{\prime\prime\prime}\right)||_{c}X^{3\times
320\times 320}.$ (13)

The image $X$ is provided as side information and is essential for
conditioning the convolutional mask refinement network towards generating fine
masks driven by the $\mathcal{L}_{\text{GTV-fine}}$ loss. We modularize the
complete convolutional mask refinement transformation given in (13) as
follows,

$\displaystyle\widehat{F}\leftarrow\textsc{ConvMaskRefine}(\widetilde{F},X).$
(14)

Coarse branch (Section 3.2): The coarse branch applies a binary linear
classification head (LCH) as a composition of a linear layer followed by
sigmoid to $\widetilde{F}$, resulting in $S_{\text{coarse}}\in[0,1]^{40\times
40}$.

$\displaystyle
S_{\text{coarse}}\leftarrow\textsc{LCH}_{1}^{64}\left(\widetilde{F}\right).$
(15)

Here $\textsc{LCH}_{1}^{\text{in}\\_\text{ch}}$ corresponds to

$\displaystyle 1\times 1\textsc{
Conv}_{1}^{\text{in}\\_\text{ch}}\rightarrow\textsc{sigmoid}.$ (16)

We denote this operation as follows

$\displaystyle
S_{\text{coarse}}\leftarrow\textsc{CoarseBranch}(\widetilde{F}).$ (17)

Fine branch (Section 3.2): The fine branch involves the composition of the
TransformerEncoder features $\widetilde{F}$ with convolutional mask refinement
network in (14), which produces $\widehat{F}$. Along the lines of (15), a
binary classification head is subsequently applied as follows

$\displaystyle
S_{\text{fine}}\leftarrow\textsc{LCH}_{1}^{131}\left(\widehat{F}\right)$ (18)

Here $S_{\text{fine}}\in[0,1]^{320\times 320}$ is the high resolution fine
mask. Therefore we denote the fine branch as

$\displaystyle S_{\text{fine}}\leftarrow\textsc{FineBranch}(X,\widetilde{F}).$
(19)

where FineBranch is given by

$\displaystyle\textsc{ConvMaskRefine}\rightarrow\textsc{LCH}_{1}^{131}$ (20)

Sempart (Section 3.4): The loss functions described in Section 3.2 and Section
3.3 are motivated by graph-based bi-partitioning of images based on deep
semantic correspondences between regions as well as driven by graph total
variation of the generated masks over the entire image. This results in high-
quality self-supervised masks based on principles of normalized cut and guided
super-resolution. We compute the corresponding loss functions in Section 3.4
to give us the eventual Sempart loss in Algorithm 1.

Algorithm 1 Sempart

Input $X\in\mathbb{R}^{3\times 320\times 320}$ in RGB space
Output Loss $\mathcal{L}_{\textsc{Sempart}}$

1:function Loss($X$)

2: $F=\textsc{DIN O}(X)$

3: $\widetilde{F}=\textsc{TransformerEncoder}(F)$

4: $S_{\text{coarse}}=\textsc{CoarseBranch}(\widetilde{F})$

5: $S_{\text{fine}}=\textsc{FineBranch}(X,\widetilde{F})$

6:
$\mathcal{L}_{\text{Ncut}}=\mathcal{L}_{\text{Ncut}}(F,S_{\text{coarse}})$See
(6)

7: $\mathcal{L}_{\text{GTV-coarse}}=\mathcal{L}_{\text{GTV-
coarse}}(F,S_{\text{coarse}})$See Section 3.3

8:
$\mathcal{L}_{\text{SR}}=\mathcal{L}_{\text{SR}}(S_{\text{coarse}},S_{\text{fine}})$333Note
that this involves an average pooling step for aligning the spatial
dimensions. See section on guided super-resolution in Section 3.2.See (7)

9: $\mathcal{L}_{\text{GTV-fine}}=\mathcal{L}_{\text{GTV-
fine}}(X,S_{\text{fine}})$See Section 3.3

10: $\mathcal{L}_{\text{coarse}}=\mathcal{L}_{\text{Ncut}}+\lambda_{\text{GTV-
coarse}}\mathcal{L}_{\text{GTV-coarse}}$

11: $\mathcal{L}_{\text{fine}}=\lambda_{\text{GTV-
fine}}\mathcal{L}_{\text{GTV-fine}}$

12: $\mathcal{L}_{\text{joint}}=\lambda_{\text{SR}}\mathcal{L}_{\text{SR}}$

13:
$\mathcal{L}_{\textsc{Sempart}}=\mathcal{L}_{\text{coarse}}+\mathcal{L}_{\text{fine}}+\mathcal{L}_{\text{joint}}$See
Section 3.4

14: return $\mathcal{L}_{\textsc{Sempart}}$

15:end function

The parameters of the transformer encoder, the convolutional mask refinement
network, and the two binary classification heads are refined iteratively as
per the loss $\mathcal{L}_{\textsc{Sempart}}$. Note that this is an entirely
unsupervised scheme where the DINO feature correspondences serve as the key
source of self-supervision.

##  Appendix D Supplementary material for Section 4

Figure 7: Comparison of Sempart with ablations of its architecture in
decreasing order of performance from (a) to (c) (see Table 4).

Architecture ablation comparison. Figure 7 demonstrates the architectural
differences between Sempart, and the ablations we compare with. In particular,
as discussed in Section 4.4, we demonstrate the value of co-optimizing our
coarse and fine branches (see Figure 7 (a)) as compared to only having the
fine branch (see Figure 7 (c)) or having both branches trained independently
(see Figure 7 (b)). Results of the paired Wilcoxon signed-rank test [35] on
the IoU metric, shown in Table 5, confirm the value of architectural choices,
using significance level of $0.05$.

Method | DUT-OMRON | DUTS-TE | ECSSD
---|---|---|---
w/o GTV coarse | $0.646\,(<{0.001})$ | $\mathbf{0.749\,(-)}$ | $0.848\,(<{0.001})$
w/o GTV fine | $0.637\,(<{0.001})$ | $0.717\,(<{0.001})$ | $0.818\,(<{0.001})$
train fine mask directly | $0.645\,(<{0.001})$ | $0.738\,(<{0.001})$ | $0.845\,(<{0.001})$
w/o joint training | $0.662\,(<{0.001})$ | $0.743\,(<{0.001})$ | $0.849\,({0.007})$
Sempart-Fine | $\mathbf{0.668}$ | $\mathbf{0.749}$ | $\mathbf{0.855}$

Table 5: Ablations of Sempart for saliency, using mIoU ($p$-value).

As described in Section 4.4, Figure 7 (b) demonstrates that normalized cut
loss only affects the transformer encoder and the coarse branch. In contrast,
the gradients from the guided reconstruction only affect the fine branch. The
gradients from the corresponding GTV losses also only affect the respective
branches. In Figure 7 (c), however, the coarse branch is completely discarded,
and the fine branch is utilized both for optimizing the expected normalized
cut loss as well as the corresponding $\mathcal{L}_{\text{GTV-fine}}$ loss.

In our experiments (see Table 4), we observe that the performance in terms of
the mean IoU of unsupervised saliency detection deteriorates consistently
across all our evaluation datasets as we go from Figure 7 (a) to (b) to (c).
This aligns with our intuition by demonstrating that not only is there value
in separately inferring a coarse mask using the coarse branch, which
effectively has the impact of a regularizer of the TransformerEncoder, but it
is also beneficial to co-optimize the fine branch with the coarse branch.

Method | OMRON* | D-TE* | ECSSD
---|---|---|---
Sempart-Fine | 0.668 | 0.749 | 0.855
Sempart-Fine† | 0.673 | 0.755 | 0.857
Selfmask on Sempart-Fine | 0.698 | 0.749 | 0.850
$\textsc{U}^{2}\textsc{-Net}$[33] | 0.693 | 0.733 | 0.878
SelfReformer[58] | 0.744 | 0.830 | 0.900

†indicates that validation images were included during unsupervised training.

Table 6: We compare Sempart variants with $\textsc{U}^{2}\textsc{-Net}$ and
SelfReformer both of which are supervised.

Comparison with supervised methods. Table 6 compares the performance of
Sempart with recent state-of-the-art supervised methods [33, 58]. We show that
using Sempart masks for SelfMask training results in high quality masks
outperforming the supervised $\textsc{U}^{2}\textsc{-Net}$ on DUT-OMRON and
DUTS-TE. However, a more recent supervised method [58] still outperforms
SEMPART by a significant margin.

We also observe that scaling the training set to also include the validation
images improves the performance of Sempart, indicated by Sempart-Fine†.

Comparison with alternate backbones. Our experiments with alternate backbones
in Table 7, indicates that the degree of pixelation (DoP), defined as the
ratio of patch to image areas affects the performance. A larger ViT patch size
is detrimental, and SSL features with lower DoP result in superior Sempart
saliency masks (Table 7 A, B vs. C, D). Nevertheless, the fine mask always
outperforms its accompanying coarse mask by preserving high-frequency details.

| Backbone | Arch | Type | Input | DoP | OMRON | D-TE | ECSSD
---|---|---|---|---|---|---|---|---
A. |  DINOv2 (2023) | ViT-S/14 | Coarse | $224^{2}$ | $3.9\textrm{e-}3$ | 0.460 | 0.539 | 0.659
B. | Fine | $224^{2}$ | $3.9\textrm{e-}3$ | 0.523 | 0.598 | 0.717
C. | Coarse | $560^{2}$ | $6.25\textrm{e-}4$ | $0.554$ | $0.671$ | $0.773$
D. | Fine | $560^{2}$ | $6.25\textrm{e-}4$ | $0.57$ | $0.686$ | $0.796$
E. | DINO | ViT-S/16 | Coarse | $320^{2}$ | $2.5\textrm{e-}3$ | 0.573 | 0.640 | 0.766
F. | Fine | $320^{2}$ | $2.5\textrm{e-}3$ | 0.596 | 0.656 | 0.793
G. | ViT-S/8 | Coarse | $320^{2}$ | $6.25\textrm{e-}4$ | 0.640 | 0.727 | 0.837
H. | Fine | $320^{2}$ | $6.25\textrm{e-}4$ | 0.668 | 0.749 | 0.855

Table 7: Sempart IoU (last three columns) for DINOv2 and DINO.

Figure 8: Hyperparameter sensitivity analysis of SEMPART-Fine.

Hyperparameter sensitivity analysis. Figure 8 (A.1, A.2) show that the
performance is typically robust to changes in $\lambda_{\text{SR}}$ and
$\lambda_{\text{GTV-coarse}}$ respectively. Figure 8 (A.3, B) show that the
performance suffers with low and high $\lambda_{\text{GTV-fine}}$ values due
to jaggedness and over-smoothing respectively.

Additional results. Figure 9, Figure 10 and Figure 11 present additional
results for both Sempart-coarse and -fine as well as also training
SelfMask+Sempart-coarse and -fine as compared to TokenCut, MOVE, and the
ground truth. The performance metrics in Table 1 indicate that the average
performance of additionally training SelfMask on Sempart as pseudo masks
results in an improvement of 3% and 3.5% in IoU and $\max\text{F}_{\beta}$
respectively for the DUT-OMRON dataset. At the same time, the gains are
debatable for DUTS-TE and, in particular, for ECSSD, for which the performance
deteriorates for the SelfMask variant.

Across Figure 9, Figure 10, and Figure 11, the superiority of Sempart over
MOVE and TokenCut is a prevalent trend. As also seen previously in Figure 3,
TokenCut, which is optimized on a per image basis, not only results in coarse
masks that do not capture several high-frequency details but can also select
the incorrect object more often than its counterparts (see Figure 9 (I)) as
well as under select the salient region (see Figure 9 (D, H), Figure 11 (C)).

On the other hand, MOVE outperforms TokenCut by generating more accurate and
high-resolution masks based on the perception of movability of foreground
objects. This heuristic outperforms previous state-of-the-art significantly,
as demonstrated in [4]. However, we find that in addition to being noisy
around the edges in most examples, it exhibits noisy artifacts both inside
(see Figure 9 (G), Figure 10 (A, F), Figure 11 (B)) and outside (see Figure 9
(I), Figure 10 (E), Figure 11 (B, F))the visually salient regions. For the
most part, MOVE can identify at least one of the salient objects. However, it
seems likely that this heuristic also results in the over-selection of
artifacts distinctly separated from the key salient object(s).

Compared to TokenCut and recent state-of-the-art MOVE, our method Sempart and
its SelfMask variants signify a superior heuristic for unsupervised image bi-
partitioning and a significantly better overlap with the ground truth saliency
masks across all datasets. We also observe that the fine mask captures high-
frequency details more accurately, especially at image boundaries than the
corresponding jointly inferred coarse mask. The joint optimization involved in
the Sempart architecture is valuable towards image bi-partitioning without
involving any post-inference processing. Therefore the inference times are a
fraction of its counterparts and comparable with other methods that also learn
a segmentation model, such as MOVE.

Figure 9: Additional examples on the DUT-OMRON [57] dataset.

Figure 10: Additional examples on the ECSSD [38] dataset.

Figure 11: Additional examples on the DUTS-TE [49] dataset.

Figure 12: Attention map of the transformer encoder [CLS] token. The Sempart
attention map aligns with the background.

Attention map. The TransformerEncoder in (11) is further elaborated in Figure
6 (a). To get a better understanding of the reasoning process of Sempart, we
have looked at the average attention map across both heads for the [CLS] token
of the TransformerEncoder in Figure 12. Interestingly we find that although
the output of the TransformerEncoder for this particular token is discarded
(see Figure 6 (a)), the corresponding attention map is insightful. This is
because the [CLS] token is attended to by the remaining $40\times 40$ patch
tokens for generating $\widetilde{F}$ in (11). Therefore, the underlying [CLS]
embeddings get leveraged for the $\widetilde{F}$ output. In particular, the
attention map resonates with the background444[42] adopted a heuristic that
expands the mask from background seeds located first.. It reflects the clear
distinction between an image’s salient and non-salient regions. On the other
hand, the DINO [CLS] token attention maps appear to attend to the foreground
regions.

##  Appendix E Ethical aspects

We benchmark our approach using publicly available datasets [49, 57, 38, 17,
18, 27]. Although our approach infers unsupervised partitions of images,
Sempart still inherits biases present in DINO [7], which was trained on
ImageNet [13] without labels and in a self-supervised manner.

##  Appendix F Future applications

The merits of Sempart in generating high-quality masks at multiple resolutions
can be particularly effective when applied to class-aware object detection,
such as in [40]. More generally, Sempart can also help improve search and
recommendation systems [50] in applications where users seek to retrieve
images of specific objects with the underlying assumption that the object
under consideration will likely be prominent and in the foreground.