# GCoNet+: A Stronger Group Collaborative Co-Salient Object Detector

Peng Zheng, Huazhu Fu, Deng-Ping Fan†, Qi Fan, Jie Qin, Yu-Wing Tai,
Chi-Keung Tang, and Luc Van Gool  Peng Zheng is with the College of Computer
Science and Technology, Nanjing University of Aeronautics and Astronautics,
Nanjing, China, with Mohamed bin Zayed University of Artificial Intelligence
(MBZUAI), Abu Dhabi, United Arab Emirates, and also with Aalto University.
Huazhu Fu is with the Institute of High Performance Computing (IHPC), Agency
for Science, Technology and Research (A*STAR), Singapore. Deng-Ping Fan is
with the Computer Vision Lab (CVL), ETH Zurich, Zurich, Switzerland. Qi Fan is
with the Hong Kong University of Science and Technology. Jie Qin is with the
College of Computer Science and Technology, Nanjing University of Aeronautics
and Astronautics, Nanjing, China. Yu-Wing Tai is with Kuaishou Technology and
the Hong Kong University of Science and Technology. Chi-Keung Tang is with the
Hong Kong University of Science and Technology. Luc Van Gool is with the
Computer Vision Lab, ETH Zurich, Zurich, Switzerland, and with KU Leuven,
Leuven, Belgium. A preliminary version of this work has appeared in CVPR 2021
[1] Peng Zheng and Huazhu Fu share equal contributions. $\dagger$ The major
part of this work was done while Peng Zheng was an intern at IIAI mentored by
Deng-Ping Fan (Corresponding author: dengpfan@gmail.com).

###### Abstract

In this paper, we present a novel end-to-end group collaborative learning
network, termed GCoNet+, which can effectively and efficiently (250 fps)
identify co-salient objects in natural scenes. The proposed GCoNet+ achieves
the new state-of-the-art performance for co-salient object detection (CoSOD)
through mining consensus representations based on the following two essential
criteria: 1) intra-group compactness to better formulate the consistency among
co-salient objects by capturing their inherent shared attributes using our
novel group affinity module (GAM); 2) inter-group separability to effectively
suppress the influence of noisy objects on the output by introducing our new
group collaborating module (GCM) conditioning on the inconsistent consensus.
To further improve the accuracy, we design a series of simple yet effective
components as follows: i) a recurrent auxiliary classification module (RACM)
promoting model learning at the semantic level; ii) a confidence enhancement
module (CEM) assisting the model in improving the quality of the final
predictions; and iii) a group-based symmetric triplet (GST) loss guiding the
model to learn more discriminative features. Extensive experiments on three
challenging benchmarks, _i.e._ , CoCA, CoSOD3k, and CoSal2015, demonstrate
that our GCoNet+ outperforms the existing 12 cutting-edge models. Code has
been released at https://github.com/ZhengPeng7/GCoNet_plus.

###### Index Terms:

Co-saliency, CoSOD, Group Collaborative Learning, Deep Learning.

##  1 Introduction

Co-Salient object detection (CoSOD) aims at detecting the most common salient
objects among a group of given relevant images. Compared with the standard
salient object detection (SOD) task, CoSOD is more challenging and requires
distinguishing co-occurring objects across different images where others act
as distractors. To this end, intra-class compactness and inter-class
separability are two important cues and should be learned simultaneously. With
the increasing accuracy and efficiency achieved by the latest CoSOD methods,
CoSOD is not only used as a pre-processing component for other vision tasks
[2, 3, 4, 5, 6] but also employed in many practical applications [1, 7, 8].

Figure 1: Comparisons of seven representative CoSOD approaches and ours on the
CoSOD3k dataset [9]. We conduct the comparison of existing representative
deep-learning-based CoSOD approaches in terms of both speed (the horizontal
axis) and accuracy (the vertical axis). Smaller bubbles mean lighter models.
Our GCoNet+ outperforms these models in terms of both efficiency and
effectiveness. The “Train-1, 2, and 3” represents the DUTS_class, COCO-9k, and
COCO-SEG datasets, respectively (see Tab. III for more related details). All
the models are tested with batch size 2 on an A100-80G. Our benchmark for
inference speed can be found at
https://github.com/ZhengPeng7/CoSOD_fps_collection.
\begin{overpic}[width=411.93767pt]{FPN_features.pdf}
\put(15.2,17.0){\rotatebox{90.0}{GCoNet}}
\put(15.2,4.0){\rotatebox{90.0}{{GCoNet+}{}}} \put(6.2,-2.0){(a)}
\put(22.8,-2.0){(b)} \put(36.8,-2.0){(c)} \put(50.5,-2.0){(d)}
\put(64.5,-2.0){(e)} \put(78.5,-2.0){(f)} \put(92.0,-2.0){(g)} \end{overpic}
Figure 2: Visualizations of feature maps. (a) Source image and ground truth.
(b-f) Feature maps on different levels of the decoder from our GCoNet [1] and
GCoNet+ (Train-1 in Tab. III), captured from high to low levels. The feature
maps shown in (b) have the lowest resolution. As (b) shows, our GCoNet+ gives
a more global response and does not make a specific prediction in the very
early stages, where the quality of feature maps is inadequate for producing
precise results. (g) Prediction of co-saliency maps. Compared with GCoNet,
GCoNet+ obtained a more global response on the objects and its surrounding.

Existing works attempt to facilitate the consistency among given images to
solve the CoSOD task within an image group by leveraging semantic connections
[10, 11, 12] or varied shared cues [13, 14, 15]. In [9, 16], the proposed
models jointly optimize a unified network for generating saliency maps and co-
saliency information. Despite the improvement brought by these methods, most
existing models only depend on the consistent feature representations in an
individual group [16, 17, 18, 19, 20, 21], which may introduce the following
limitations. First, images from the same group can only provide positive
relations instead of both positive and negative relations between different
objects. Training models with only positive samples from a single group may
lead to overfitting and ambiguous results for outlier images. Besides, there
are typically a limited number of images in one group (20 to 40 images for
most groups on existing CoSOD datasets). Therefore, information learned from a
single group is usually insufficient for a discriminative representation.
Finally, individual image groups may not be easy to mine semantic cues, which
are vital in distinguishing noisy objects during testing in complex real-world
scenes. Due to the complexity of image context in real scenarios, a module
designed for common information mining is in high demand. Apart from these,
when supervised with the Binary Cross Entropy (BCE) loss, pixel values of
generated saliency maps tend to get closer to 0.5 instead of 0 or 1. Suffering
from the uncertainty, these maps are difficult to directly apply in realistic
applications.

To overcome the above restrictions, we propose a new group collaborative
learning network (GCoNet), which establishes semantic consensus within the
same group and distinction among different image groups. Our GCoNet includes
three basic modules: Group Affinity Module (GAM), Group Collaborating Module
(GCM), and Auxiliary Classification Module (ACM), which simultaneously guide
our GCoNet to learn the inter-group separability and intra-group compactness
in a better way. Specifically, the GAM enables the model to learn the
consensus feature in the same image group, while the GCM discriminates target
attributes among different groups, thus making the network trainable on
existing rich SOD datasets111There are about 60k SOD images publicly
available, which is about 10 times larger than existing CoSOD datasets. This
means the insufficient training data issue in CoSOD may be partially
alleviated in the proposed framework.. To learn a better embedding space, we
utilize the ACM on each image to improve the feature representation at a
global semantic level.

We have improved our original GCoNet by providing a more precise explanation
of the existing contributions, _i.e._ , a concise network for CoSOD, three
additional components that can improve the ability to learn consensus and
difference, discussions on the shortcomings of existing training sets and the
corresponding solution.

In summary, we have extended our GCoNet significantly to GCoNet+ with major
differences as follows:

  * •

Novel Approaches. We propose three new components which improve the
performance and robustness of our GCoNet+, _i.e._ , Confidence Enhancement
Module (CEM), Group-based Symmetric Triplet (GST) Loss, and Recurrent
Auxiliary Classification Module (RACM) to deal with the existing weaknesses of
our GCoNet. 1) Confidence Enhancement Module (CEM): To make output maps less
uncertain, we employ the differentiable binarization and a hybrid saliency
loss in our confidence enhancement module, which brings maps of higher quality
and further improves the overall performance. 2) Group-based Symmetric Triplet
(GST) Loss: We are one of the first ones to apply metric learning to deep-
based CoSOD models, which makes the learned features of different groups more
discriminative in a metric learning way. 3) Recurrent Auxiliary Classification
Module (RACM): To better represent the auxiliary classification feature, we
extend the original auxiliary classification module to its recurrent version,
which focuses more precisely on the pixels of target objects. Besides, we
improve the GCoNet [1] into a more lightweight and powerful network as our
baseline. These three components and the new baseline network are organically
combined and achieved great performance on all existing datasets and realistic
applications in our experiments.

  * •

Experiments. Although the development of Co-Salient Object Detection is rapid,
there are three usually used datasets for training, _i.e._ , DUTS_class,
COCO-9k, and COCO-SEG, while there is no standard for choosing the training
sets for this task. Different from existing works where the used training sets
are not the same, we conduct more comprehensive experiments with all different
combinations of these three training sets for fair experimental comparisons.
With the combinations of the newly proposed components in this paper, as
mentioned above in novel approaches, we obtain $\sim$3.2% relative improvement
on the $E_{\xi}^{\text{max}}$ [22] and $S_{\alpha}$ [23] compared with our
GCoNet [1] with the same training set, achieving the state-of-the-art
performance to this day among all publicly available CoSOD models [9]222Public
leaderboard of CoSOD models: https://paperswithcode.com/task/co-saliency-
detection.

  * •

New Insights. Based on the obtained experimental results, we found the
potential problems of the existing CoSOD training sets and provided the
corresponding analyses on how to improve them in the future.

##  2 Related Work

###  2.1 Salient Object Detection

Among traditional salient object detection (SOD) methods, hand-crafted
features play the most important role in detection [24, 25, 26, 27]. In the
early years of deep learning, features are extracted from image patches,
object proposals [28, 29, 30], or super-pixels [31, 32, 33, 34]. Although
these methods have made some progress, they are time-consuming for extracting
the target regions and their features. With the success of fully convolutional
networks [35] in segmentation tasks, recent SOD researches mainly focus on the
models which make pixel-wise predictions. More details and a summary can be
found in recent review works [8, 36, 37], where the latest work [8] provides
the most comprehensive benchmark and analysis on the performance, robustness,
and generalization of existing SOD models on a variety of challenging SOD
datasets, which also presents a constructive discussion for SOD open issues
and future research directions. In [38], the network architectures in SOD
methods are divided into five categories, _i.e._ , single-steam, multi-stream,
side-fusion, U-shape, and multi-branch. Among these architectures, the U-shape
is the most widely used one, especially the base structure of FPN [39] and
U-Net [40]. Multi-stage supervision is employed at the early stages by
aggregating features from different stages of these U-shape networks to make
the output features more robust and stable [41, 1, 16]. In [42, 43, 44, 45],
the attention mechanism and the related modules are designed for improvement.
Besides, external information is introduced as extra guidance on its training
processes, such as edge [41] and boundary [46].

In the binary segmentation tasks (_e.g._ , Salient Object Detection [1, 46,
42], Optical Character Recognition [47, 48, 49]), ground truths are the binary
maps of target objects. However, the predicted maps are not fully binary ones
due to pixel-level loss (_i.e._ , mean-square error loss, binary cross-entropy
loss). In many practical applications, maps with much uncertainty are
unsuitable for programs to make decisions [50]. In that case, some up-to-date
methods are proposed for improving the quality of binary maps. In [51],
specific components are designed to enhance the integrity of objects. In [46],
hybrid losses are also employed to make models focus on more attributes beyond
pixel-level errors.

###  2.2 Image Co-Segmentation

Image co-segmentation, a fundamental and active computer vision task, segments
the common objects from a group of images. This has been widely adopted in
many related areas, such as co-salient object detection [17, 1, 9, 52], few-
shot learning [53, 54], semantic segmentation [55, 56],  _etc_. Many existing
methods in co-segmentation employ the Siamese network to find the common
feature of the input image pair [57, 58]. Based on the comparison within the
image pair, Chang et al.  [59] and Rother et al.  [60] used saliency and color
histograms, respectively to guide a more precise comparison of the visual
features. With the development of deep learning methods, co-segmentation
models tend to use implicit semantic features to find common objects. From the
aspect of the model, Wei et al.  [17] and Fan et al.  [9] embedded the co-
attention in their network to generate the group consensus, Chen et al.  [61]
leveraged channel attentions for better object co-segmentation, and LSTM [62]
is employed by Zhang et al.  [63] and Li et al.  [64] to exchange the
information between two images and enhance the group representations. From the
aspect of the training strategy, Wang et al.  [65] explored the saliency-
guided iterative refinement on result maps with weakly supervised strategy,
and Hsu et al.  [66] made use of the intra-image object discrepancy and inter-
image figure-ground separation to achieve image co-segmentation in an
unsupervised manner.

###  2.3 Co-Salient Object Detection

The SOD task [67, 46, 68, 69] aims at segmenting salient objects separately in
a single image, while CoSOD targets finding common salient objects across a
group of semantic-related images. Previous CoSOD methods mainly aimed at
mining intra-group cues to segment the co-salient objects. For example, early
CoSOD approaches used to explore the correspondence among a group of relevant
images based on handcrafted cues. With computational fractions (_e.g._ ,
superpixels [70]) segmented from each image, these methods establish the
correspondence model and discover the common regions by employing a ranking
scheme, clustering guidance, or translation alignment [71]. Metric learning
[11, 72], statistics of histograms and contrasts [24], and pairwise similarity
ranking are also applied to formulate better semantic attributes for further
computation.

In the deep learning era, many end-to-end deep CoSOD models have been
proposed. The authors of [17, 11] attempt to discover the common objects by
learning the consensus in a single group. With the development of the upstream
deep learning methods, existing methods [73, 1, 18, 16, 74] build their models
with powerful CNN models (_e.g._ , ResNet [75], VGGNet [76], and Inception
[77]) or even Transformer models (_e.g._ , ViT [78] and PVT [79, 80]), which
help achieve SOTA performances. Besides most existing works designing their
models by full supervision, weakly-supervised strategies (_e.g._ , GWSCoSal
[81], FASS [82], SP-MIL [83], CODW [7], and GONet [84]) achieve acceptable
results as well.

###  2.4 Intra- and Inter-image Consistency Learning

With the rapid development of deep learning, deep models achieved great
performance in exploring intra- and inter-image consistency, such as graph
convolutional networks (GCN) [85, 86, 87], co-attention [9], co-clustering
[88], recurrent units [89], correlation techniques [20], self-learning methods
[10], and quality measuring [90].

Among the implementations of intra-image consistency learning, co-attention is
one of the most widely used components for exploring the consensus for
segmentation on similar images since it was first explored in [91].
Furthermore, many follow-up works [92, 93, 94] then dive deeper via more
information and better methods, including pixel contrast, relational data, and
graph network. These works show great effectiveness and have brought much
improvement to the research in related areas.

Besides, intra- and inter-image consistency also show its effectiveness in
other research areas, such as object detection [95, 96], semantic segmentation
[97], and salient object detection [98], especially for establishing the
relations between objects to obtain better semantic features of different
categories on weakly supervised learning.

In previous CoSOD methods, intra-group consistency has been studied in detail
[1, 16, 17, 7].In contrast, less attention has been paid to the inter-group,
which, however, contributes significantly to guiding the model to learn more
discriminative and general features for each class. In [16], a jigsaw training
strategy is used to introduce images from other groups to implicitly
facilitate group training. In [7], multiple groups of images are fed into
their model to learn the intra-image contrast. Without a more advanced and
explicit design for learning inter-group information, their models still
mainly target intra-group information. Our approach varies a lot from existing
models in exploring inter-group relations. We try to learn discriminative
features semantically, explicitly, and precisely at a group level.

##  3 Methodology

We introduce our GCoNet+ for the CoSOD task. The overview of the architecture
is presented in Sec. 3.1. Then, we sequentially introduce the proposed basic
modules: group affinity module (GAM), group collaborating module (GCM),
confidence enhancement module (CEM), group-based symmetric triplet (GST) loss,
and the recurrent auxiliary classification module (RACM).

###  3.1 Overview

The basic framework of the proposed GCoNet+ is based on our GCoNet [1], which
is one of the latest state-of-the-art methods. Unlike existing CoSOD models
[9, 18, 16, 20] that only exploit the common information inside a single class
group, GCoNet+ exploits both the internal and external relationship between
different groups in a siamese style.

The flowchart of GCoNet+ is illustrated in Fig. 3. First, our model
simultaneously takes two groups of raw images ${G_{1},G_{2}}$ as input. With
the concatenated image groups ($\copyright$), our encoder extracts the feature
maps $\mathcal{F}$, which is then fed to the auxiliary classification module
(ACM) for classification and to our group collaborative module (GCoM) for
further processing. In GCoM, $\mathcal{F}$ is split into two parts by their
classes, _i.e._ ,
$\mathcal{F}_{1}=\\{F_{1,n}\\}_{n=1}^{N},\mathcal{F}_{2}=\\{F_{2,n}\\}_{n=1}^{N}\in\mathbb{R}^{N\times
C\times H\times W}$, where C denotes the channel number, $H\times W$ is the
spatial size, and $N$ denotes the group size. These two features are
separately given to the group affinity module (GAM), where all the single-
image features are combined to distill the consensus features
${\textit{E}_{1}^{\alpha}}\in\mathbb{R}^{1\times C\times 1\times 1}$.
Meanwhile, a group collaborating module (GCM) is applied to obtain a more
discriminative representation of target attributes between different image
groups. The output features ${\mathcal{F}_{1}^{out},\mathcal{F}_{2}^{out}}$ of
GCoM are concatenated to be fed to our decoder. Simultaneously, the decoder is
connected with the encoder by 1x1 convolution layers. Then, the confidence
enhancement module (CEM) takes the prediction of decoder $\mathcal{F}_{d}$ as
input to refine and provide the final co-saliency maps
${\mathcal{M}_{1},\mathcal{M}_{2}}$. At last, the network’s output is
multiplied with original images $G$ to eliminate the irrelevant regions. Our
group-based symmetric triplet (GST) loss is applied on the masked images
$G_{\mathcal{M}}$ to supervise GCoNet+ in a metric learning way. Besides, the
masked images are then fed to the encoder again to obtain the masked encoded
feature $\mathcal{F}_{r}$. Different from $\mathcal{F}$, $\mathcal{F}_{r}$
contains only the features of predicted regions and has a more precise
semantic representation to be applied in the recurrent auxiliary
classification module (RACM) to obtain the classification loss.

\begin{overpic}[width=411.93767pt]{arch.pdf}
\put(1.0,9.5){\footnotesize{${L}_{\text{BCE}}+{L}_{\text{IoU}}$}}
\put(0.0,7.5){\footnotesize{(Eq.~{}\ref{eqn:loss_BCE}),
(Eq.~{}\ref{eqn:loss_IoU})}} \put(60.5,24.8){\footnotesize{${L}_{\text{GCM}}$
(Eq.~{}\ref{eqn:GCM})}} \put(44.0,41.5){\footnotesize{${L}_{\text{GST}}$
(Eq.~{}\ref{eqn:loss_GST})}} \put(90.0,36.2){\footnotesize{${L}_{\text{CLS}}$
(Eq.~{}\ref{eqn:loss_cls})}} \end{overpic} Figure 3: Pipeline of the proposed
Group Collaborative Learning Network plus (GCoNet+). Input images are obtained
from two groups and fed into an encoder. Then we employ the GCoM (Group
Collaborative Module), where intra-group collaborative learning is conducted
for each group by the group affinity module (GAM), and the inter-group
collaborative learning is conducted via the group collaborating module (GCM).
The original images and RoIs masked by the output are given to the encoder to
do an auxiliary classification to make the features of different classes more
discriminative to each other. The decoder output is put through the confidence
enhancement module (CEM) to make the final result more binarized and easy to
use. Furthermore, the RoIs of two groups obtained by the multiplication of
original images and predicted saliency maps are measured with a triplet loss
to enlarge the distance between inter-group features and reduce the distance
between intra-group features.

###  3.2 Group Affinity Module (GAM)

In most cases of real life, objects of the same class share similarity in
their appearance and features, which have been widely used in many computer
vision tasks. For example, self-supervised video tracking methods [99, 100,
101, 102] often propagate the segmentation maps of target objects based on the
pixel-wise correspondences between two adjacent frames. Therefore, we
introduce this motivation to the CoSOD task by computing the global affinity
among all images in the same group.

For features $\\{F_{1,n},F_{1,m}\\}\in\mathcal{F}_{1}$ of any two images333All
analyses in Sec. 3.2 on $\mathcal{F}_{1}$ can be applied to $\mathcal{F}_{2}$.
We omit the group subscript for notation simplicity, _i.e._ , we use $F_{n}$
to represent $F_{1,n}$., we compute their pixel-wise correlations in the
format of an inner product:

$S_{(n,m)}=\theta(F_{n})^{T}\phi(F_{m}),$ (1)

where $\theta,\phi$ denote linear embedding functions ($3\times 3\times 512$
convolutional layer). The affinity map $S_{(n,m)}\in\mathbb{R}^{HW\times HW}$
efficiently captures the common features of co-salient objects in the given
image pair $(n,m)$. Then we can generate $F_{n}$’s affinity map
$A_{n\leftarrow m}\in\mathbb{R}^{HW\times 1}$ by finding the maxima for each
of $F_{n}$’s pixels conditioned on $F_{m}$, which alleviates the influence of
noisy correlation values in the map.

Similarly, we can extend the use of the local affinity of an image pair to the
global affinity of an entire image group. Specifically, we compute the
affinity map $S_{\mathcal{F}}\in\mathbb{R}^{NHW\times NHW}$ of all image
features $\mathcal{F}$ using Eq. 1. Then, we find the maxima for each image
$A_{\mathcal{F}}^{\prime}\in\mathbb{R}^{NHW\times N}$ from $S_{\mathcal{F}}$,
and average all the maxima of $N$ images to generate the global affinity
attention map $A_{\mathcal{F}}\in\mathbb{R}^{NHW\times 1}$. In this way, the
affinity attention map is globally optimized on all images, and the influence
of occasional co-occurring bias is thus alleviated. Then, we use a softmax
operation to normalize $A_{\mathcal{F}}$ and reshape it to produce the
attention map $A_{S}\in\mathbb{R}^{N\times(1\times H\times W)}$. With the
attention map $A_{S}$, we multiply it with the original feature $\mathcal{F}$
to generate the attention feature maps $\mathcal{F}^{a}\in\mathbb{R}^{N\times
C\times H\times W}$. Finally, the attention feature maps $\mathcal{F}^{a}$ for
the whole group are used to generate the attention consensus $\textit{E}^{a}$
by the average pooling along both the batch dimension and spatial dimension,
as illustrated in Fig. 4.

The GAM focuses on capturing the commonality of co-occurring salient objects
within the same group, thus improving the intra-group compactness of the
consensus representation. Such intra-group compactness alleviates the
distraction made by co-occurring noise and encourages the model to concentrate
on the co-salient regions. This allows the shared attributes of co-salient
objects to be better captured, resulting in better consensus representation.
The obtained attention consensus $\textit{E}^{a}$ is combined with the
original feature maps $\mathcal{F}$ through depth-wise correlation [103, 104]
to achieve efficient information association. The generated feature maps
$\mathcal{F}^{out}$ of different groups are then concatenated together and fed
to the decoder. After the confidence enhancement module (CEM), final co-
saliency maps $\mathcal{M}$ are produced for all images.

Figure 4: Group Affinity Module. We first utilize affinity attention to obtain
the attention maps of the input features by collaborating all images in a
group. Subsequently, the maps are multiplied with the input features to
generate the consensus for the group. Then the obtained consensus is used to
coordinate the original feature maps and is also fed to the GCM for inter-
group collaborative learning. Figure 5: Group Collaborating Module. Both
groups’ original feature maps and consensus are fed to the GCM. The predicted
output conditioned on the consistent feature and consensus (from the same
group) is supervised with the available ground truth labels. Otherwise, it is
supervised by the all-zero maps.

###  3.3 Group Collaborating Module (GCM)

Currently, most existing CoSOD approaches tend to focus on the intra-group
compactness of the consensus. Still, the inter-group separability is crucial
for distinguishing distracting objects, especially when processing complex
images with more than one salient object. To this end, we propose a simple but
effective module, _i.e._ , the GCM, by learning to encode the inter-group
separability.

With the GAM, we can obtain the attention consensus
$\\{\textit{E}_{1}^{a},\textit{E}_{2}^{a}\\}$ of images in two groups. Then,
we apply an intra- and inter-group cross-multiplication ($\cdot$) between the
corresponding features $\\{\mathcal{F}_{1},\mathcal{F}_{2}\\}$ and the
attention consensus to get the intra-group collaboration:
$\mathcal{F}_{1}^{1}=\mathcal{F}_{1}\cdot\textit{E}_{1}^{a}$ and
$\mathcal{F}_{2}^{2}=\mathcal{F}_{2}\cdot\textit{E}_{2}^{a}$. In contrast, the
inter-group multiplication deals with the features and consensus of different
groups, _i.e._ , $\mathcal{F}_{1}^{2}=\mathcal{F}_{1}\cdot\textit{E}_{2}^{a}$
and $\mathcal{F}_{2}^{1}=\mathcal{F}_{2}\cdot\textit{E}_{1}^{a}$, to represent
the inter-group interaction. The intra-group representation
$\mathcal{F}^{+}=\\{\mathcal{F}_{1}^{1},\mathcal{F}_{2}^{2}\\}$ is computed to
predict co-saliency maps, and the inter-group representation
$\mathcal{F}^{-}=\\{\mathcal{F}_{1}^{2},\mathcal{F}_{2}^{1}\\}$ is employed
for a consensus with group separability. Specifically, we feed the inter-group
and intra-group features $\\{\mathcal{F}^{+},\mathcal{F}^{-}\\}$ to a small
convolutional network with an upsampling layer and obtain the saliency maps
$\\{\mathcal{M}^{+},\mathcal{M}^{-}\\}$444$\mathcal{M}^{+}=\\{\mathcal{M}_{1}^{+},\mathcal{M}_{2}^{+}\\}$
and $\mathcal{M}^{-}=\\{\mathcal{M}_{1}^{-},\mathcal{M}_{2}^{-}\\}$. with
different supervision signals. As shown in Fig. 5, we use the ground truth
maps to make supervision on $\mathcal{F}^{+}$, while all-zero maps on
$\mathcal{F}^{-}$. The loss function is:

${L}_{\text{GCM}}=\frac{1}{N}\sum_{n}^{N}{L}_{\text{FL}}(<\mathcal{M}_{n}^{+},\mathcal{M}_{n}^{-}>,<\mathcal{G}_{n},\mathcal{G}_{n}^{0}>),$
(2)

where ${L}_{\text{FL}}$ denotes the focal loss [39], $\mathcal{G}_{n}$ denotes
the ground truth, $\mathcal{G}_{n}^{0}$ denotes the all-zero map, and
$<\cdot>$ denotes the concatenation operation.

Consequently, GCM lets the consensus have a high inter-group separability
between different groups and makes it easier to identify distractors in a
complex environment. Specifically, this module doesn’t introduce additional
computation during inference and can be fully discarded.

\begin{overpic}[width=424.94574pt]{BCE_vs_IoU.pdf}
\put(0.0,47.5){\rotatebox{90.0}{Butterfly}}
\put(0.0,33.0){\rotatebox{90.0}{Guitar}} \put(0.0,20.0){\rotatebox{90.0}{Axe}}
\put(0.0,-1.0){\rotatebox{90.0}{Armadillo}} \put(11.5,-3.0){{(a)}}
\put(31.0,-3.0){{(b)}} \put(50.0,-3.0){{(c)}} \put(69.0,-3.0){{(d)}}
\put(88.5,-3.0){{(e)}} \end{overpic} Figure 6: Prediction results produced by
our GCoNet+ trained with different losses. (a) Source image. (b) Ground truth.
(c) Results of GCoNet+ trained with only BCE loss. (d) Results of GCoNet+
trained with only IoU loss. (e) Results of GCoNet+ trained with balanced BCE
and IoU loss. All the results here are generated from models trained with the
DUTS_class dataset.

###  3.4 Confidence Enhancement Module (CEM)

In SOD tasks, the pixel values of predicted saliency maps range between 0 and
1 since the network is usually appended with a Sigmoid function. Although the
pixel values of ground truth maps are either 0 or 1, those of the predicted
saliency maps may approach 0.5, which indicates more uncertainty and noise in
the prediction. On the very hard cases, results with more uncertainty and
noise tend to get higher scores by some classic metrics [22],  _e.g._ , Fbw
[105], IoU [106], MAE,  _etc_ , while terrible in practice, which is opposite
to the final goal.

To deal with the uncertain values of the prediction, we conduct research from
both the perspective of the loss function and network architecture. From the
view of the loss function, we set up comparative experiments to verify that
different loss functions can introduce different optimization directions to
the same network. To be more specific, IoU loss guides the outputs to be
almost 0 or 1, but with the low accuracy of existing metrics, _i.e._ ,
S-measure [23], E-measure [22]. In contrast, BCE loss directs the network to
predict more uncertain values but achieve better scores in the above metrics.
As the expected maps are shown in Fig. 6, although IoU loss brings high
confidence to predicted maps, the optimization is too rough. It acts up in
terms of the integrity of the saliency maps. Therefore, BCE loss is still a
necessity for training. To improve the quality of saliency maps in terms of
binarization for practical application, we try to balance the BCE and IoU loss
as a mixed pixel loss for supervision.

From the view of network architecture, we employ the confidence enhancement
module (CEM) at the end of Fig. 3. In previous SOD approaches, the Sigmoid
function is usually applied to squeeze the output values from 0 to 1. However,
as is described in [47], the Sigmoid activation function is not steep enough,
and the values produced by it are not binarized enough. To address this issue,
as shown in Fig. 7, the output feature $\mathcal{F}_{d}$ of the decoder is fed
into the CEM. Firstly, the feature $\mathcal{F}_{d}$ goes through two parallel
branches with two 3x3 convolution layers, which are both followed by batch
normalization, a ReLU activation function, and a 1x1 convolution layer
followed by a Sigmoid activation function. After that, the probability map $P$
and the threshold map $T$ are generated and put into the differentiable
binarization function to obtain the final prediction. According to [47], the
final co-saliency maps $\mathcal{M}$ can be represented as:

$\mathcal{M}_{i,j}=\frac{1}{1+e^{-k(P_{i,j}-T_{i,j})}},$ (3)

where $k$ is the factor that controls the steepness of the step function. In
our implementation, $k$ is set to 300 as the default value. When loss meets
NaN during training, it will be replaced with 50 for current propagation.

\begin{overpic}[width=429.28616pt]{CEM-2.pdf}
\put(1.0,29.0){\footnotesize{${L}_{\text{BCE}}+{L}_{\text{IoU}}$}}
\put(96.0,15.0){{$\mathcal{F}_{d}$}} \end{overpic} Figure 7: Confidence
Enhancement Module. After the decoder, we adapt the CEM to bring higher
quality and binarization to the predicted saliency maps. CBR means a
convolution layer followed by a batch normalization layer and a ReLU
activation function.

###  3.5 Group-based Symmetric Triplet (GST) Loss

In the past few years, some approaches have been designed to solve the CoSOD
task from the perspective of metric learning [11, 72]. However, most existing
CoSOD approaches based on metric learning use super-pixel [107] to extract
fractions as the unit of measurement. Most of these methods are usually not
end-to-end and in low efficiency. Besides, existing works typically introduce
class labels to help the model learn more representative features with high
semantics. Specifically, in [16], Zhang et al. divide the DUTS dataset [108]
into different groups by the class of main salient objects to build the
training set. However, absolute class labels may not be given in realistic
scenarios. In contrast, there is only the relative label of two groups
(whether they belong to the same group). In 2015, Schroff et al. proposed the
triplet loss [109] to help face recognition, which is a good way to learn the
discriminative feature of different identities by pulling the positive samples
and pushing negative samples. Because of the success of triplet loss in face
recognition [109], visual tracking [110], person re-id [111],  _etc_ , we
modify the original triplet loss to GST loss to learn more discriminative
features from different groups, which could improve the uniqueness and
discrimination of the consensus features of objects with different class
labels.

\begin{overpic}[width=424.94574pt]{GST-2.pdf}
\put(6.0,47.0){\footnotesize{$G_{1}$}}
\put(32.0,47.0){\footnotesize{$\mathcal{M}_{1}$}}
\put(62.0,47.0){\footnotesize{$\mathcal{M}_{2}$}}
\put(88.0,47.0){\footnotesize{$G_{2}$}}
\put(1.0,26.0){\footnotesize{$\mathcal{F}_{r}^{1A}$}}
\put(25.0,25.0){\footnotesize{$\mathcal{F}_{r}^{1B}$}}
\put(66.0,26.0){\footnotesize{$\mathcal{F}_{r}^{2A}$}}
\put(83.0,25.0){\footnotesize{$\mathcal{F}_{r}^{2B}$}}
\put(18.5,30.5){\footnotesize{$\mathcal{F}_{r}^{1}$}}
\put(75.0,30.5){\footnotesize{$\mathcal{F}_{r}^{2}$}}
\put(48.0,2.0){\footnotesize{Euclidean~{}Distance}} \par\end{overpic} Figure
8: Group-based Symmetric Triplet Loss. In GST loss, each group is divided
averagely into two sub-groups. The sub-groups from the same group pull to each
other and push to features of other groups. $\Phi_{\theta}$ represents the
backbone (see Fig. 3).

Note that our GST loss is only activated in the training process.
Specifically, it is applied on $\mathcal{F}_{r}$, which is the output feature
extracted by the encoder from $G_{\mathcal{M}}$, _i.e._ , the multiplication
result between predicted saliency maps $\mathcal{M}$ and the original images
$G$ (see Fig. 3). In this way, only the pixels of targeted objects are used
for the measurement. Taking $G_{1}$ in Fig. 8 as an example, the backbone
$\Phi_{\theta}$ extracts the semantic representation $\mathcal{F}_{r}^{1}$
from the raw images masked with $\mathcal{M}_{1}$. Then, the
$\mathcal{F}_{r}^{1}$ is split into two parts by class, _i.e._ ,
$\mathcal{F}_{r}^{1A}$, $\mathcal{F}_{r}^{2A}$. Features from the same group
are seen as positive samples of each other, while those from the other group
are negative. As shown in Fig. 8, our GST loss is calculated in a symmetric
structure. Finally, the triplet loss is computed on both the
$(\mathcal{F}_{r}^{1A},~{}\mathcal{F}_{r}^{1B},~{}\mathcal{F}_{r}^{2A})$ and
$(\mathcal{F}_{r}^{1B},~{}\mathcal{F}_{r}^{2B},~{}\mathcal{F}_{r}^{2A})$,
where the distances between features are measured with Euclidean distance.
Specifically,
${L}_{\text{Tri}}(\mathcal{F}_{r}^{1A},~{}\mathcal{F}_{r}^{1B},~{}\mathcal{F}_{r}^{2A})$
can be denoted as follows:

$||\mathcal{F}_{r}^{1A}-\mathcal{F}_{r}^{1B}||_{2}-||\mathcal{F}_{r}^{1B}-\mathcal{F}_{r}^{2A}||_{2}+\alpha,$
(4)

where $\alpha$ denotes the margin that is a hyper-parameter enforced between
positive and negative pairs [109]. $||\cdot||_{2}$ denotes the two-norm of
input. Because of the symmetry of GST loss,
${L}_{\text{Tri}}(\mathcal{F}_{r}^{1B},~{}\mathcal{F}_{r}^{2B},~{}\mathcal{F}_{r}^{2A})$
is also measured with the Euclidean distance in the same way.

The final GST loss is a combination of double $L_{\text{Tri}}$ when $G_{1}$
and $G_{2}$ act as the positive samples alternately from the images masked
with predicted maps:

${L}_{\text{GST}}={L}_{\text{Tri}}(\mathcal{F}_{r}^{1A},~{}\mathcal{F}_{r}^{1B},~{}\mathcal{F}_{r}^{2A})~{}+~{}{L}_{\text{Tri}}(\mathcal{F}_{r}^{1B},~{}\mathcal{F}_{r}^{2B},~{}\mathcal{F}_{r}^{2A}),$
(5)

###  3.6 Recurrent Auxiliary Classification Module (RACM)

Existing works typically train the model with images within the same group to
extract common information. Specifically, images in a certain batch only have
ground truth maps on the objects belonging to the same class, where only the
common intra-group features can be learned. However, since there are no
constraints on the features learned, common features of different classes may
get close to each other and be hard to distinguish.

In [1], the auxiliary classification module (ACM) facilitates the high-level
semantic representation to obtain more discriminative features for consensus
learning. Specifically, a class predictor consisting of a global average
pooling layer and one fully connected layer is applied after the backbone. The
features of objects with the same class are clustered together through class-
level supervision. Although ACM works well in GCoNet [1], it has some defects:
the features from the backbone are unstable and can be something else other
than the right objects. As a consequence, the ACM may give a wrong optimizing
direction. Meanwhile, it runs implicitly and is difficult to monitor.

We propose to use the Recurrent ACM (RACM) to overcome the problems mentioned
above. The pipeline of RACM is kept almost the same as that of the original
ACM. In contrast, RACM takes the model’s output as the mask to obtain the
pixels of target objects only rather than the whole image used in ACM. Then
the masked images will be sent again to the encoder and class predictor. After
eliminating other distracting regions, our RACM focuses only on the
interesting areas. When the prediction of our GCoNet+ is far from the ground
truth map, RACM can give an enhanced penalty to help accelerate the
convergence of training. Combining the raw images and ground truth maps to
formulate the loss, RACM enables the model to learn more discriminative
features for inter-group separability and intra-group compactness,
respectively. The loss functions of classification are as follows:

$\hat{Y}_{\text{ACM}}=\varphi(\Phi_{\theta}(G)),$ (6)
$\hat{Y}_{\text{RACM}}=\varphi(\Phi_{\theta}(G\otimes\mathcal{M})),$ (7)
${L}_{\text{CLS}}={L}_{\text{CE}}(\hat{Y}_{\text{RACM}},~{}Y_{\text{CLS}})+{L}_{\text{CE}}(\hat{Y}_{\text{ACM}},~{}Y_{\text{CLS}}),$
(8)

where $\varphi$ and $\Phi_{\theta}$ denote the class predictor (GAP and one
linear layer) and encoder, respectively. $L_{\text{CE}}$ is the cross-entropy
loss, $Y_{\text{CLS}}$ are the ground truth class labels, and
$\hat{Y}_{\text{ACM}}$ and $\hat{Y}_{\text{RACM}}$ are the class labels
predicted by ACM and RACM, respectively.

###  3.7 Objective Function

The objective function is a weighted combination of saliency map loss
(combination of BCE loss and IoU loss), GCM loss, our GST loss, and
classification loss. The BCE loss and IoU loss are illustrated as:

${L}_{\text{BCE}}=-\sum{[Ylog(\hat{Y}),(1-Y)log(1-\hat{Y})]},$ (9)
${L}_{\text{IoU}}=1-\frac{1}{N}\sum{\frac{Y\cap\hat{Y}}{Y\cup\hat{Y}}},$ (10)

where $Y$ is the ground truth and $\hat{Y}$ is the prediction. With the GCM
loss (Eq. 2), GST loss (Eq. 5), and classification loss (Eq. 8), our final
objective function is:

${L}=\lambda_{1}{L}_{\text{BCE}}+\lambda_{2}{L}_{\text{IoU}}+\lambda_{3}{L}_{\text{GCM}}+\lambda_{4}{L}_{\text{GST}}+\lambda_{5}{L}_{\text{CLS}},$
(11)

where
$\lambda_{1},~{}\lambda_{2},~{}\lambda_{3},~{}\lambda_{4},~{}and~{}\lambda_{5}$
are respectively set to 30, 0.5, 250, 3, and 3 to keep all the losses on the
same quantitative level at the beginning of training.

##  4 Experiments

This section provides the guidelines and details in our base and extensive
experiments, _i.e._ , datasets, settings, evaluation protocol, and analysis in
training and testing, respectively.

###  4.1 Datasets

Training Sets. We follow the GICD [16] to use DUTS_class as our training set
to design the experiments. After removing the noisy samples by Zhang et al.
[16], the whole DUTS_class is divided into 291 groups, which contain 8,250
images in total. The DUTS_class dataset is the only training set used for
evaluation in our ablation study. Nowadays, there is still a lack of a fully
recognized training dataset. To make a fair comparison with up-to-date works
[17, 112, 20, 113, 18], we employed the widely-adopted COCO-9k [17], a subset
of the COCO [114] with 9,213 images of 65 groups, and the COCO-SEG [112] which
is also a subset of the COCO [114] and contains 200k images, to train our
GCoNet+ as supplementary experiments.

Test Sets. To obtain a comprehensive evaluation of our GCoNet+, we test it on
three widely used CoSOD datasets, _i.e._ , CoCA [16], CoSOD3k [9], and
CoSal2015 [7]. Among these three datasets, CoCA is the most challenging
dataset. It is of much higher diversity and complexity in terms of background,
occlusion, illumination, surrounding objects,  _etc_. Following the latest
benchmark [9], we do not evaluate on iCoseg [115] and MSRC [116], since only
one salient object is given in most images there. It is more convincing to
evaluate CoSOD methods on images with more salient objects, which is closer to
real-life applications.

###  4.2 Evaluation Protocol

Following the GCoNet [1], we employ the S-measure [23], maximum F-measure
[117], maximum E-measure [22], and mean absolute error (MAE) to evaluate the
performance in our experiments. Evaluation toolbox can be referred to
https://github.com/zzhanghub/eval-co-sod.

S-measure [23] is a structural similarity measurement between a saliency map
and its corresponding ground truth map. The evaluation with $S_{\alpha}$ can
be obtained at high speed without binarization. S-measure is computed as:

$S_{\alpha}=\alpha\times S_{o}+(1-\alpha)\times S_{r},$ (12)

where $S_{o}$ and $S_{r}$ denote object-aware and region-aware structural
similarity, and $\alpha$ is set to 0.5 by default, as suggested by Fan et al.
in [1].

F-measure [117] is designed to evaluate the weighted harmonic mean value of
precision and recall. The output of the saliency map is binarized with
different thresholds to obtain a set of binary saliency predictions. The
predicted saliency maps and ground truth maps are compared for precision and
recall values. The best F-measure score obtained with the best threshold for
the whole dataset is defined as $F_{\beta}^{max}$. F-measure can be computed
as:

$F_{\beta}=\frac{(1+\beta^{2})Precision\times
Recall}{\beta^{2}(Precision+Recall)},$ (13)

where $\beta^{2}$ is set to 0.3 to emphasize the precision over recall,
following [36].

E-measure [22] is designed as a perceptual metric to evaluate the similarity
between the predicted maps and ground truth maps from both local and global
views. E-measure is defined as:

$E_{\xi}=\frac{1}{W\times H}\sum_{x=1}^{W}\sum_{y=1}^{H}\phi_{\xi}(x,y),$ (14)

where $\phi_{\xi}$ indicates the enhanced alignment matrix. Similar to the
F-measure, we also adopt the max E-measure ($E_{\xi}^{max}$) as our evaluation
metrics.

TABLE I: Quantitative ablation studies of the overall modification on the framework of our GCoNet+. We conduct the ablation studies of our GCoNet+ on the effectiveness of overall modification on the framework, including network simplification (Net-Sim), batch normalization (BN), and hybrid loss (HL).  | Modules |  CoCA [16] |  CoSOD3k [9] |  CoSal2015 [7]
---|---|---|---|---
ID |  Net-Sim | BN | HL | $E_{\xi}^{\text{ max}}\uparrow$ | $S_{\alpha}\uparrow$ | $F_{\beta}^{\text{ max}}\uparrow$ | $\epsilon\downarrow$ | $E_{\xi}^{\text{ max}}\uparrow$ | $S_{\alpha}\uparrow$ | $F_{\beta}^{\text{ max}}\uparrow$ | $\epsilon\downarrow$ | $E_{\xi}^{\text{ max}}\uparrow$ | $S_{\alpha}\uparrow$ | $F_{\beta}^{\text{ max}}\uparrow$ | $\epsilon\downarrow$
1 |  |  |  | 0.760 | 0.673 | 0.544 | 0.105 | 0.860 | 0.802 | 0.777 | 0.071 | 0.888 | 0.845 | 0.847 | 0.068
2 | ✓ |  |  | 0.752 | 0.676 | 0.538 | 0.100 | 0.872 | 0.815 | 0.796 | 0.063 | 0.895 | 0.853 | 0.858 | 0.063
3 | ✓ | ✓ |  | 0.747 | 0.683 | 0.556 | 0.110 | 0.884 | 0.824 | 0.806 | 0.062 | 0.912 | 0.868 | 0.874 | 0.051
4 | ✓ |  | ✓ | 0.774 | 0.691 | 0.562 | 0.106 | 0.879 | 0.831 | 0.806 | 0.065 | 0.901 | 0.867 | 0.865 | 0.062
5 | ✓ | ✓ | ✓ | 0.779 | 0.681 | 0.558 | 0.119 | 0.882 | 0.828 | 0.807 | 0.068 | 0.913 | 0.875 | 0.877 | 0.055

MAE $\epsilon$ is a simple pixel-level evaluation metric that measures the
absolute difference between the predicted maps and the ground truth maps
without binarization. It is defined as:

$\epsilon=\frac{1}{W\times
H}\sum_{x=1}^{W}\sum_{y=1}^{H}|\hat{Y}(x,y)-\text{GT}(x,y)|.$ (15)

###  4.3 Implementation Details

Based on GCoNet [1], we employ VGG-16 with batch normalization [118] as the
backbone. We randomly pick $N$ samples from two different groups in each
training batch.

$N=\min(\\#groupA,\\#groupB,32),$ (16)

where $N$ denotes the batch size for training, and $\\#$ means the number of
images in the corresponding group. Due to the small number of images in some
groups, we chose the minimum between 32 and the smaller number of images in
the two groups which we randomly selected. Note that $N$ for training and
testing can be different. During testing, we follow previous works [1, 16,
113, 18, 119, 120] to set the exact number of images in the given group as the
batch size $N$.

To clarify our proposed network, we provide the hyper-parameters in the newly
proposed modules. The steep step function produces some NaN values after the
backpropagation in the confidence enhancement module (CEM). So, we set the $k$
in differentiable binarization (DB) to a radical value of 300 and a
conservative value of 50. When NaN is produced in a certain step, 50 will be
used for replacement, which never produces NaN in our experiments. In group-
based symmetric triplet (GST) loss, the margin value is set to 1.0.

The images are resized to 256x256 for training and testing. The output maps
are resized to the original size for evaluation. Three data augmentation
strategies are applied in our training process, _i.e._ , horizontal flip,
color enhancement, and rotation. Our GCoNet+ is trained over 320 epochs with
the Adam optimizer. The initial learning rate is set to 3e-4, $\beta_{1}=0.9$,
and $\beta_{2}=0.99$. The whole training process takes around 20 hours. All
the experiments are implemented based on PyTorch [121] with a single Tesla
V100 GPU.

Figure 9: Learning curve comparison. We record the overall losses obtained
during the training of our baseline (see Sec. 4.4) with additional RACM and
with only original ACM, where DUTS_class is used as the training set.

###  4.4 Ablation Study

We study the effectiveness of each extension component (_i.e._ , RACM, CEM,
and GST) employed in our GCoNet+ and investigate why they can help learn both
good consensus features and discriminative features in our framework. The
qualitative results regarding each module are shown in Fig. 11. For more
ablation studies and experimental settings, can be referred to our conference
version [1].

TABLE II: Quantitative ablation studies of the proposed components in our GCoNet+. We conduct the ablation studies of our GCoNet+ on the effectiveness of the proposed components, including RACM (Recurrent Auxiliary Classification Module), CEM (Confidence Enhancement Module), GST (Group-based Symmetric Triplet Loss), and their combinations. | Modules |  CoCA [16] |  CoSOD3k [9] |  CoSal2015 [7]
---|---|---|---|---
ID |  RACM | CEM | GST | $E_{\xi}^{\text{ max}}\uparrow$ | $S_{\alpha}\uparrow$ | $F_{\beta}^{\text{ max}}\uparrow$ | $\epsilon\downarrow$ | $E_{\xi}^{\text{ max}}\uparrow$ | $S_{\alpha}\uparrow$ | $F_{\beta}^{\text{ max}}\uparrow$ | $\epsilon\downarrow$ | $E_{\xi}^{\text{ max}}\uparrow$ | $S_{\alpha}\uparrow$ | $F_{\beta}^{\text{ max}}\uparrow$ | $\epsilon\downarrow$
1 |  |  |  | 0.779 | 0.681 | 0.558 | 0.119 | 0.882 | 0.828 | 0.807 | 0.068 | 0.913 | 0.875 | 0.877 | 0.055
2 | ✓ |  |  | 0.780 | 0.684 | 0.570 | 0.120 | 0.884 | 0.829 | 0.809 | 0.067 | 0.912 | 0.873 | 0.875 | 0.056
3 | ✓ | ✓ |  | 0.779 | 0.686 | 0.565 | 0.117 | 0.881 | 0.829 | 0.805 | 0.068 | 0.913 | 0.872 | 0.873 | 0.057
4 | ✓ |  | ✓ | 0.780 | 0.683 | 0.559 | 0.118 | 0.882 | 0.831 | 0.810 | 0.068 | 0.914 | 0.876 | 0.876 | 0.055
5 | ✓ | ✓ | ✓ | 0.786 | 0.691 | 0.574 | 0.113 | 0.881 | 0.828 | 0.807 | 0.068 | 0.917 | 0.875 | 0.876 | 0.054
\begin{overpic}[width=420.61192pt]{CAM-3.pdf}
\put(-1.5,29.5){\rotatebox{90.0}{Beer Bottle}}
\put(-1.5,16.0){\rotatebox{90.0}{Tennis Ball}}
\put(-1.5,4.0){\rotatebox{90.0}{Tablet}} \put(16.0,-2.0){{(a)}}
\put(49.0,-2.0){{(b)}} \put(82.5,-2.0){{(c)}} \end{overpic} Figure 10:  The
class activation maps [122] are obtained on the classification branch. For
each comparison cell, the left half results are the activation maps obtained
from the original GCoNet [1] using ACM; the right half results are the
activation maps generated by our GCoNet+ with extra RACM. As column (a) shows,
our GCoNet+ has superiority in precisely putting its attention on the target
object next to other objects around. In (b), our GCoNet+ shows better
performance in focusing on objects of the correct class though it has some
disturbing surroundings. In the last column (c), some complex samples make
both models mistaken. Although some wrong attention is put on objects of the
wrong classes, our GCoNet+ can still put most of the attention on the right
objects and see them as the main parts of the images. The classification
activation maps provided here are produced by our GCoNet+ trained on
DUTS_class only.

Baseline. We follow GCoNet [1] to design our GCoNet+ in a siamese way. Note
that GCoNet follows the architecture of GICD [16] without extensive
experiments on the validity of each component in GICD, including the multi-
head supervision, loss function, feature normalization,  _etc_. Although these
components bring additional parameters and complexity to the network itself,
experimental proof still does not support the effectiveness. Instead of taking
these components as granted, we conduct extensive experiments on each
component. Firstly, we try to substitute the blocks of multiple convolutions
in lateral connections with only one 1x1 convolution layer as the original FPN
[39] does. Secondly, we try to remove the multi-stage supervision of the
saliency maps on the decoder. Thirdly, we try to add batch normalization
behind every convolution layer except 1x1 convolution layers. Finally, as our
experiments show, BCE loss brings higher accuracy to our experiments, while
IoU loss brings more binarized final saliency maps and faster convergence. To
better combine the two losses, we control the initial BCE and IoU loss on the
same quantity level with different weights and sum them up.

These modifications can be summarized into three parts, _i.e._ , network
architecture simplification, normalization layers, and the hybrid loss.
Following Occam’s Razor555https://en.wikipedia.org/wiki/Occams_razor, we try
to remove all uncertain modules used in many existing works without enough
experimental proof. These modifications improve our GCoNet+ in terms of both
simplicity and accuracy with a large margin compared with the baseline model
GCoNet (ID:1 in Tab. I). As shown in Tab. I, combining all of them yields 2.6%
and 2.8% relative improvement on CoSOD3k and CoSal2015 in terms of E-measure,
respectively. It also achieves 2.5% E-measure relative improvement on CoCA,
the most challenging CoSOD test set.

Effectiveness of RACM. The RACM guides the model to learn more discriminative
features to distinguish objects of different classes. Compared with the
original ACM, it works more accurately and accelerates (see Fig. 9) the
convergence of our GCoNet+. As seen in Tab. II, the RACM slightly improves the
baseline performance on CoCA and CoSOD3k in terms of most metrics. The
activation maps in Fig. 10 show that our GCoNet+ gives a higher accuracy in
various cases and guides the model to focus on the targets more precisely. The
feature maps on each stage of the decoder in GCoNet+ are shown in Fig. 2. As
the results show, GCoNet+ has a better performance than GCoNet [1] in
discriminating objects of different classes.

TABLE III: Quantitative comparisons between our GCoNet+ and other methods. “$\uparrow$” (“$\downarrow$”) means that the higher (lower) is better. Methods are attached with links to open-source codes or paper sources. Since there are several datasets used in the CoSOD task for training, we list all the training sets used in corresponding methods, _i.e._ , Train-1, 2, and 3 represent the DUTS_class [16], COCO-9k [17], and COCO-SEG [112], respectively. |  |  |  CoCA [16] |  CoSOD3k [9] |  CoSal2015 [7]
---|---|---|---|---|---
Method | Pub. & Year | Training Set | $E_{\xi}^{\text{max}}\uparrow$ | $S_{\alpha}\uparrow$ | $F_{\beta}^{\text{ max}}\uparrow$ | $\epsilon\downarrow$ | $E_{\xi}^{\text{ max}}\uparrow$ | $S_{\alpha}\uparrow$ | $F_{\beta}^{\text{ max}}\uparrow$ | $\epsilon\downarrow$ | $E_{\xi}^{\text{ max}}\uparrow$ | $S_{\alpha}\uparrow$ | $F_{\beta}^{\text{ max}}\uparrow$ | $\epsilon\downarrow$
CBCS [14] | TIP 2013 | - | 0.641 | 0.523 | 0.313 | 0.180 | 0.637 | 0.528 | 0.466 | 0.228 | 0.656 | 0.544 | 0.532 | 0.233
GWD [17] | IJCAI 2017 | Train-2 | 0.701 | 0.602 | 0.408 | 0.166 | 0.777 | 0.716 | 0.649 | 0.147 | 0.802 | 0.744 | 0.706 | 0.148
RCAN [89] | IJCAI 2019 | Train-2 | 0.702 | 0.616 | 0.422 | 0.160 | 0.808 | 0.744 | 0.688 | 0.130 | 0.842 | 0.779 | 0.764 | 0.126
GCAGC [87] | CVPR 2020 | Train-3 | 0.754 | 0.669 | 0.523 | 0.111 | 0.816 | 0.785 | 0.740 | 0.100 | 0.866 | 0.817 | 0.813 | 0.085
GICD [16] | ECCV 2020 | Train-1 | 0.715 | 0.658 | 0.513 | 0.126 | 0.848 | 0.797 | 0.770 | 0.079 | 0.887 | 0.844 | 0.844 | 0.071
ICNet [20] | NeurIPS 2020 | Train-2 | 0.698 | 0.651 | 0.506 | 0.148 | 0.832 | 0.780 | 0.743 | 0.097 | 0.900 | 0.856 | 0.855 | 0.058
CoADNet [18] | NeurIPS 2020 | Train-1, 3 | - | - | - | - | 0.878 | 0.824 | 0.791 | 0.076 | 0.914 | 0.861 | 0.858 | 0.064
CoEGNet [9] | TPAMI 2021 | Train-1 | 0.717 | 0.612 | 0.493 | 0.106 | 0.837 | 0.778 | 0.758 | 0.084 | 0.884 | 0.838 | 0.836 | 0.078
DeepACG [120] | CVPR 2021 | Train-3 | 0.771 | 0.688 | 0.552 | 0.102 | 0.838 | 0.792 | 0.756 | 0.089 | 0.892 | 0.854 | 0.842 | 0.064
CADC [113] | ICCV 2021 | Train-1, 2 | 0.744 | 0.681 | 0.548 | 0.132 | 0.840 | 0.801 | 0.859 | 0.096 | 0.906 | 0.866 | 0.862 | 0.064
DCFM [119] | CVPR 2022 | Train-2 | 0.783 | 0.710 | 0.598 | 0.085 | 0.874 | 0.810 | 0.805 | 0.067 | 0.892 | 0.838 | 0.856 | 0.067
UFO [52] | ArXiv 2022 | Train-3 | 0.782 | 0.697 | 0.571 | 0.095 | 0.874 | 0.819 | 0.797 | 0.073 | 0.906 | 0.860 | 0.865 | 0.064
GCoNet (Ours) | CVPR 2021 | Train-1 | 0.760 | 0.673 | 0.544 | 0.105 | 0.860 | 0.802 | 0.777 | 0.071 | 0.887 | 0.845 | 0.847 | 0.068
GCoNet+ (Ours) | Submission | Train-1 | 0.786 | 0.691 | 0.574 | 0.113 | 0.881 | 0.828 | 0.807 | 0.068 | 0.917 | 0.875 | 0.876 | 0.054
GCoNet+ (Ours) | Submission | Train-2 | 0.798 | 0.717 | 0.605 | 0.098 | 0.877 | 0.819 | 0.796 | 0.075 | 0.902 | 0.853 | 0.857 | 0.073
GCoNet+ (Ours) | Submission | Train-3 | 0.787 | 0.712 | 0.602 | 0.100 | 0.875 | 0.820 | 0.793 | 0.075 | 0.899 | 0.853 | 0.852 | 0.071
GCoNet+ (Ours) | Submission | Train-1, 2 | 0.808 | 0.734 | 0.626 | 0.088 | 0.894 | 0.839 | 0.822 | 0.065 | 0.919 | 0.876 | 0.880 | 0.058
GCoNet+ (Ours) | Submission | Train-1, 3 | 0.814 | 0.738 | 0.637 | 0.081 | 0.901 | 0.843 | 0.834 | 0.062 | 0.924 | 0.881 | 0.891 | 0.056

Effectiveness of CEM. Among the previous CoSOD methods, IoU and BCE loss tend
to be employed as the training loss. However, in most of these methods, only
one single loss is used for the supervision during training. BCE guides the
supervision from the pixel perspective, and IoU guides the supervision from
the view of regions. Despite the outstanding performance achieved by many
existing methods [87, 18, 120, 1, 119], using the BCE and IoU separately
suffers from some issues. Specifically, with IoU loss supervising the model on
the region level, the predicted saliency maps are usually rough and cannot
handle the small details very well. BCE can guide the model to focus on the
details. At the same time, saliency maps supervised with it tend to contain
much uncertainty, which makes the predictions challenging to use in the
application directly. In this case, we apply the CEM to simultaneously predict
more accurate and binarized maps closer to the demand of real-world
applications. As shown in Fig. 11 and Tab. II, the CEM can make the predicted
maps better in terms of both accuracy and visualization.

\begin{overpic}[width=420.61192pt]{ablation.pdf}
\put(-1.5,39.0){\rotatebox{90.0}{UAV}}
\put(-1.5,25.0){\rotatebox{90.0}{Helicopter}}
\put(-1.5,14.0){\rotatebox{90.0}{Handbag}}
\put(-1.5,3.5){\rotatebox{90.0}{Poker}} \put(7.0,-2.0){{(a)}}
\put(21.0,-2.0){{(b)}} \put(35.0,-2.0){{(c)}} \put(49.0,-2.0){{(d)}}
\put(63.0,-2.0){{(e)}} \put(77.5,-2.0){{(f)}} \put(91.5,-2.0){{(g)}}
\end{overpic} Figure 11: Qualitative ablation studies of our GCoNet+ on
different modules and their combinations. (a) Source image; (b) Ground truth;
(c) GCoNet [1]; (d) Our new baseline; (e) Baseline+RACM; (f)
Baseline+RACM+CEM; (g) Baseline+RACM+CEM+GST, the final version of our
GCoNet+. To keep consistency with GCoNet, the predicted maps provided here are
generated by our GCoNet+ trained on DUTS_class only.

Effectiveness of GST Loss. Consensus features play an important role in the
CoSOD task for detecting common objects. However, the consensus features of
some categories may get too close to each other. To this end, we need to keep
consensus features more distinguishing and the distance far away from other
features. We introduce the GST loss to make features of different classes
learned more discriminative to each other. As experiments show in Tab. II and
Fig. 11, GST loss successfully differentiates the features on a global and RoI
level and further improves the model’s competitiveness.

###  4.5 Competing Methods

Since not all CoSOD models are publicly available, we only compare our GCoNet
and GCoNet+ with one representative traditional algorithm CBCS [14] and 11
deep-learning based CoSOD models, including all update-to-date models, _i.e._
, GWD [17], RCAN [89], CSMG [123], GCAGC [87], GICD [16], ICNet [20], CoADNet
[18], CoEGNet [9], DeepACG [120], CADC [113], UFO [52], and DCFM [119].
Because of the much more excellent performance of the latest CoSOD methods
compared with single-SOD methods, we do not list the single-SOD ones. A
complete leaderboard of previous methods can be found in [9].

\begin{overpic}[width=416.27809pt]{qual.pdf}
\put(-1.5,52.0){\rotatebox{90.0}{Input}} \put(-1.5,45.0){\rotatebox{90.0}{GT}}
\put(-1.5,34.0){\rotatebox{90.0}{{{GCoNet+}{}}}}
\put(-1.5,26.0){\rotatebox{90.0}{GCoNet}}
\put(-1.5,18.5){\rotatebox{90.0}{GICD}}
\put(-1.5,10.0){\rotatebox{90.0}{CoEG}} \put(-1.5,1.0){\rotatebox{90.0}{CADC}}
\put(7.0,-2.0){{Beer Bottle}} \put(34.5,-2.0){{Crutch}}
\put(60.0,-2.0){{Tennis}} \put(85.0,-2.0){{Tomato}} \end{overpic} Figure 12:
Qualitative comparisons of our GCoNet+ and other methods. “GT” denotes the
ground truth. The predictions in the row of Ours are produced by our GCoNet+ ,
which is trained with the DUTS_class dataset.

Quantitative Results. Tab. III shows the quantitative results of our GCoNet+
and previous state-of-the-art methods. Our GCoNet+ outperforms all of them in
all metrics, especially on the CoCA and CoSOD3k datasets. CoCA is the most
difficult dataset to specify the common objects compared with two other
datasets because of the larger number of objects in a single image and more
diverse backgrounds. Our GCoNet+ shows a stronger ability in segmentation,
which benefits from the improved features in terms of saliency detection and
consensus learning, respectively. CoSOD3k has similar attributes, and our
GCoNet+ keeps its best performance over all other methods on this dataset.
CoSal2015 is the easiest dataset since most of its images contain only one
salient object, making it easy to handle with single-SOD methods. Despite the
low difficulty and absence of co-saliency, our GCoNet+ still outperforms other
methods with a relatively smaller margin. Besides, our GCoNet+ has fewer
parameters and makes the faster inference compared with most of the existing
methods, as shown in Tab. IV.

TABLE IV: Runtime comparisons of different methods. Batch size is set as 2 for all methods during their inference on an A100 GPU. Methods | Inference Time (ms) | Parameters (MB)
---|---|---
GICD [16] | 7.1 | 1060.7
ICNet [20] | 6.6 | 70.3
CoADNet [18] | 13.1 | 113.2
GCAGC [87] | 58.1 | 280.7
CADC [113] | 58.0 | 1498.7
DCFM [119] | 4.6 | 542.9
GCoNet [1] (Ours) | 2.1 | 541.7
GCoNet+ (Ours) | 3.5 | 70.3

Qualitative Results. Fig. 12 shows the saliency maps generated by different
methods for qualitative comparison. Images of beer_bottle group contain
multiple salient objects of numerous classes, where our GCoNet+ can precisely
detect the co-salient objects while others cannot. In the crutch group,
targets are slim sketches, but our GCoNet+ can still segment the sketches with
high accuracy while others even fail to make the correct segmentation. We put
the tennis group here to compare the ability of models to detect small
objects, where our GCoNet+ performs better than others in terms of both
classification and precision. In contrast, others may miss the small objects
or focus on objects with other classes. Many tomatoes appear as co-salient
objects in the tomato group and should be detected simultaneously. Our method
can find all the tomatoes with a good saliency map, but others may omit some
of these salient tomatoes or segment other very close objects. Among the
examples above, our GCoNet+ better finds the intra-group common information
and discriminates the inter-group information.

\begin{overpic}[width=433.62pt]{duts_coco-2.pdf}
\put(16.0,-3.0){{DUTS\\_class}} \put(53.0,-3.0){{COCO-9k and COCO-SEG}}
\end{overpic} Figure 13: Ground truth with problem occurs in DUTS_class
dataset, COCO-9k, and COCO-SEG dataset. As the examples given show, salient
objects of different classes occur together in one image of DUTS_class
dataset, whose regions are wrong ground truth in DUTS_class. In COCO-9k and
COCO-SEG, many objects are not salient, which also plays a bad role in
training a CoSOD model.

###  4.6 Discussion of Existing CoSOD Training Sets

Even though many great works have been proposed in CoSOD, there remains a lack
of a standard training set. DUTS_class, COCO-9k, and COCO-SEG are three
commonly-used training sets but have their limitations,  _e.g._ , inaccurate
GT maps, and a small number of target objects.

\begin{overpic}[width=416.27809pt]{qual_dataset.pdf}
\put(-1.5,52.5){\rotatebox{90.0}{Input}} \put(-1.5,43.5){\rotatebox{90.0}{GT}}
\put(-1.5,31.5){\rotatebox{90.0}{Mix-1, 3}}
\put(-1.5,21.5){\rotatebox{90.0}{Mix-1, 2}}
\put(-1.5,11.0){\rotatebox{90.0}{COCO-9k}}
\put(-1.5,0.1){\rotatebox{90.0}{DUTS\\_class}} \put(22.0,-2.0){{CoCA}}
\put(71.0,-2.0){{CoSal2015}} \end{overpic} Figure 14:  Qualitative results
produced by GCoNet+ trained on different training sets. We adopt different
datasets to set up experiments to validate the different optimization
directions of the DUTS_class dataset [108] and COCO-9k/COCO-SEG datasets [17,
112]. “Mix-1, 2” denotes both the DUTS_class and COCO-9k are used in training,
“Mix-1, 3” denotes both the DUTS_class and COCO-SEG are used in training. CoCA
is the most complex CoSOD test set and needs more attention to be paid to
finding objects of the common class. At the same time, CoSal2015 is a
relatively simple one that almost only measures the ability of salient object
detection in most cases.

DUTS_class. Due to DUTS_class aiming only at detecting salient objects, there
are salient objects of different classes in a single image. As Fig. 13 shows,
objects with wrong categories still exist in the ground truth, which gives the
models a wrong optimization direction. Whatsmore, there are only a very few
target objects in a single image, which makes the training lack the
segmentation ability of common objects.

COCO-9k/COCO-SEG. As mentioned in [17, 112], COCO-9k and COCO-SEG are both
collected from COCO dataset [114]. However, neither of them takes the salient
objects into account. Therefore, objects with ground truth may not be salient.
Thus, models trained on only COCO-9k or only COCO-SEG may perform well on
segmenting common objects while badly on segmenting salient objects.

Experiments. Among the three public test sets and realistic scenarios, cases
can be difficult or easy, with various objects and complex contexts, or simply
a dominant object on a white paper. To deal with all these cases with a
satisfying result, the models need to behave well on both the common object
segmentation and salient object detection, which are the main optimization
goals that can be learned from COCO-9k [17]/COCO-SEG [112] and DUTS_class
[16], respectively. As mentioned in Sec. 4.1, CoCA [16] focuses more on
segmenting the common objects in complex contexts, while CoSal2015 [7] plays a
more critical role in testing the ability of models to detect salient objects.
We take these two datasets to check the different aspects of the model’s
performance.

We train GCoNet+ on the DUTS_class and COCO-9k/COCO-SEG both separately and
jointly. Taking the results of CoSal2015 [7] shown in Fig. 14, models trained
on DUTS_class [16] show a better performance on SOD tasks but weakness on
detecting objects of the common class. However, getting trained on COCO-9k or
COCO-SEG enables the model to learn a good ability to segment objects with a
common class while having a relatively worse performance on detecting simple
salient objects. Compared with models training on DUTS_class, models trained
only on COCO-9k/COCO-9k often fail to detect the salient objects.

To deal with the two sub-tasks in CoSOD, _i.e._ , segmenting common objects
and detecting salient objects, we need to optimize our GCoNet+ in two
directions. Therefore, we set a joint training of our GCoNet+ on DUTS_class
[16] and COCO-9k/COCO-SEG [17, 112]. Under the setting of combined training,
the same model shows more robust performance in both the two directions
mentioned above. As the performance shown in Tab. III, the jointly trained
(_i.e._ , Train-1, 3) model achieves much better results on all these three
test sets. Specifically, compared with the model trained only on DUTS_class,
our GCoNet+ shows comparable performance on CoSal2015 while much better
performance on CoCA. Meanwhile, compared with the model trained only COCO-9k
or only COCO-SEG, the jointly trained model shows similar performance on CoCA
while much better results on CoSal2015. The same phenomenon also occurs on the
predicted maps, as shown in Fig. 14.

###  4.7 Failure Cases

\begin{overpic}[width=420.61192pt]{failures.pdf} \put(1.8,-2.0){{Strawberry}}
\put(17.5,-2.0){{GT}} \put(27.0,-2.0){{{GCoNet+}{}}} \put(39.0,-2.0){{GCoNet}}
\put(51.5,-2.0){{Chopsticks}} \put(67.5,-2.0){{GT}}
\put(76.5,-2.0){{{GCoNet+}{}}} \put(89.5,-2.0){{GCoNet}} \end{overpic} Figure
15:  Failure cases in the results produced by our GCoNet+. We provide the
typical failure cases made by our GCoNet+ and GCoNet [1]. On the left side,
the inaccurate classification of the surrounding objects makes the bad CoSOD
results. On the right side, there lacks the ability of fine segmentation
making inaccurate predictions of saliency maps.

There are two main sub-goals of the co-salient object detection networks,
which means we can delineate the ability of the network from two perspectives,
_i.e._ , finding the common objects and segmenting the salient ones of them.
Thus, we select the typical failures of these two types for analysis. As shown
in Fig. 15, when too many similar objects of different classes appear in a
single image or target objects are hard to separate from neighboring objects,
our models may misidentify them and give inaccurate predictions.

Specifically, for the strawberries shown on the left of Fig. 15, our well-
trained GCoNet+ tends to focus on the texture and color of objects. The
pockmark textures can be misidentified as those of strawberries. Thus, the
cranberries and cherries are misidentified as strawberries while the
blueberries can be identified. For the chopsticks on the right side in the
figure, our GCoNet+ is more capable of finding the target objects while still
cannot handle the difficult segmentation problem. Although our GCoNet+ still
faces some problems in these very difficult cases, it still demonstrates high
potential while outperforming our previous GCoNet.

To further improve the models in these difficult cases, a larger training set
consisting of more classes will be needed. A larger number of classes will
bring forth a stronger ability to discriminate similar objects of different
classes, while more segmentation examples will enhance the general
segmentation ability to accurately segment objects in complex scenarios. As
mentioned in Sec. 4.6, this may be a major potential contribution to the CoSOD
task in the future.

##  5 Potential Applications

We show the potential to utilize the extracted co-saliency maps to produce
segmentation masks of high quality for related downstream image processing
tasks.

Figure 16: Application #1. Content aware object co-segmentation visual results
(“Helicopter”) obtained by our GCoNet+.

Application #1: Content-Aware Co-Segmentation. Co-saliency maps have been
widely used in image pre-processing tasks. Taking the unsupervised object
segmentation in our implementation as an example, we first find a group of
images by keywords on the Internet. Then, our GCoNet+ is applied to generate
co-saliency maps. Finally, the salient objects of the specific group can be
extracted with the co-saliency maps. Following [24], we can use GrabCut [124]
to obtain the final segmentation results. Adaptive threshold [125] is chosen
here to initialize GrabCut for the binary version of the saliency maps. As
shown in Fig. 16, our method works well in the content-aware object co-
segmentation task, which should benefit existing E-commerce applications in
the background replacement.

Application #2: Automatic Thumbnails. The idea of paired-image thumbnails is
derived from [71]. With the same goal666Jacobs et al. ’s work [71] is limited
to the case of image pairs, we introduce a CNN-based application of
photographic triage, which is valuable for sharing images on the website. As
Fig. 17 shows, the orange box is generated by the saliency maps obtained from
GCoNet+. We can also scale them up with the orange box and get the larger red
one. Finally, the collection-aware crops technique [71] can be adapted to
produce the results shown in the second row.

Figure 17: Application #2. Co-localization based automatic thumbnails
(“Butterfly”) generated by our GCoNet+.

##  6 Conclusion

This work proposes a novel group collaborative model (GCoNet+) to deal with
the CoSOD task. With the experiments conducted, we find that group-level
consensus can introduce effective semantic information, auxiliary
classification, and metric learning to improve the feature representation in
terms of intra-group compactness and inter-group separability. Qualitative and
quantitative experiments demonstrate the superiority and state-of-the-art
performance of our GCoNet+. We show that the techniques of GCoNet+ can also be
transferred and easily used in many relevant applications, such as co-
detection and co-segmentation.

## Acknowledgments

The authors would like to thank the anonymous reviewers and editor for their
helpful comments on this manuscript. We thank Prof. Ling Shao for his
insightful feedback. This work is partially supported by Huazhu Fu’s A*STAR
Central Research Fund, and Career Development Fund (C222812010). This work is
also supported by the National Natural Science Foundation of China (No.
62276129).

## References

  * [1] Q. Fan, D.-P. Fan, H. Fu, C.-K. Tang, L. Shao, and Y.-W. Tai, “Group collaborative learning for co-salient object detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2021, pp. 12 283–12 293.
  * [2] W. Wang and J. Shen, “Higher-order image co-segmentation,” _IEEE Trans. Multimedia_ , vol. 18, no. 6, pp. 1011–1021, 2016.
  * [3] K.-J. Hsu, Y.-Y. Lin, and Y.-Y. Chuang, “Deepco3: Deep instance co-segmentation by co-peak search and co-saliency detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2019, pp. 8846–8855.
  * [4] Y. Zeng, Y. Zhuge, H. Lu, and L. Zhang, “Joint learning of saliency detection and weakly supervised semantic segmentation,” in _Int. Conf. Comput. Vis._ , 2019, pp. 7223–7233.
  * [5] K. R. Jerripothula, J. Cai, and J. Yuan, “Efficient video object co-localization with co-saliency activated tracklets,” _IEEE Trans. Circ. Syst. Video Technol._ , vol. 29, no. 3, pp. 744–755, 2018.
  * [6] X. Wang, X. Liang, B. Yang, and F. W. Li, “No-reference synthetic image quality assessment with convolutional neural network and local image saliency,” _Comput. Vis. Media_ , vol. 5, no. 2, pp. 193–208, 2019.
  * [7] D. Zhang, J. Han, C. Li, J. Wang, and X. Li, “Detection of co-salient objects by looking deep and wide,” _Int. J. Comput. Vis._ , vol. 120, no. 2, pp. 215–232, 2016.
  * [8] W. Wang, Q. Lai, H. Fu, J. Shen, H. Ling, and R. Yang, “Salient object detection in the deep learning era: An in-depth survey,” _IEEE Trans. Pattern Anal. Mach. Intell._ , pp. 1–1, 2021.
  * [9] D.-P. Fan, T. Li, Z. Lin, G.-P. Ji, D. Zhang, M.-M. Cheng, H. Fu, and J. Shen, “Re-thinking co-salient object detection,” _IEEE Trans. Pattern Anal. Mach. Intell._ , pp. 1–1, 2021.
  * [10] D. Zhang, D. Meng, and J. Han, “Co-saliency detection via a self-paced multiple-instance learning framework,” _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 39, no. 5, pp. 865–878, 2016.
  * [11] J. Han, G. Cheng, Z. Li, and D. Zhang, “A unified metric learning-based framework for co-saliency detection,” _IEEE Trans. Circ. Syst. Video Technol._ , vol. 28, no. 10, pp. 2473–2483, 2018.
  * [12] K.-J. Hsu, C.-C. Tsai, Y.-Y. Lin, X. Qian, and Y.-Y. Chuang, “Unsupervised cnn-based co-saliency detection with graphical optimization,” in _Eur. Conf. Comput. Vis._ , 2018, pp. 485–501.
  * [13] H. Li and K. N. Ngan, “A co-saliency model of image pairs,” _IEEE Trans. Image Process._ , vol. 20, no. 12, pp. 3365–3375, 2011.
  * [14] H. Fu, X. Cao, and Z. Tu, “Cluster-based co-saliency detection,” _IEEE Trans. Image Process._ , vol. 22, no. 10, pp. 3766–3778, 2013.
  * [15] X. Cao, Z. Tao, B. Zhang, H. Fu, and W. Feng, “Self-adaptively weighted co-saliency detection via rank constraint,” _IEEE Trans. Image Process._ , vol. 23, no. 9, pp. 4175–4186, 2014.
  * [16] Z. Zhang, W. Jin, J. Xu, and M.-M. Cheng, “Gradient-induced co-saliency detection,” in _Eur. Conf. Comput. Vis._ , 2020, pp. 455–472.
  * [17] L. Wei, S. Zhao, O. E. F. Bourahla, X. Li, and F. Wu, “Group-wise deep co-saliency detection,” in _Int. Joint Conf. Artif. Intell._ , 2017, pp. 3041–3047.
  * [18] Q. Zhang, R. Cong, J. Hou, C. Li, and Y. Zhao, “Coadnet: Collaborative aggregation-and-distribution networks for co-salient object detection,” in _Adv. Neural Inform. Process. Syst._ , 2020, pp. 6959–6970.
  * [19] R. Cong, N. Yang, C. Li, H. Fu, Y. Zhao, Q. Huang, and S. Kwong, “Global-and-local collaborative learning for co-salient object detection,” _IEEE Trans. Cybern._ , pp. 1–1, 2022.
  * [20] W.-D. Jin, J. Xu, M.-M. Cheng, Y. Zhang, and W. Guo, “Icnet: Intra-saliency correlation network for co-saliency detection,” in _Adv. Neural Inform. Process. Syst._ , 2020, pp. 18 749–18 759.
  * [21] L. Tang, B. Li, S. Kuang, M. Song, and S. Ding, “Re-thinking the relations in co-saliency detection,” _IEEE Trans. Circ. Syst. Video Technol._ , pp. 1–1, 2022.
  * [22] D.-P. Fan, C. Gong, Y. Cao, B. Ren, M.-M. Cheng, and A. Borji, “Enhanced-alignment measure for binary foreground map evaluation,” in _Int. Joint Conf. Artif. Intell._ , 2018, pp. 698–704.
  * [23] D.-P. Fan, M.-M. Cheng, Y. Liu, T. Li, and A. Borji, “Structure-measure: A new way to evaluate foreground maps,” in _Int. Conf. Comput. Vis._ , 2017, pp. 4558–4567.
  * [24] M.-M. Cheng, G.-X. Zhang, N. J. Mitra, X. Huang, and S. Hu, “Global contrast based salient region detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2011, pp. 409–416.
  * [25] H. Jiang, Z. Yuan, M.-M. Cheng, Y. Gong, N. Zheng, and J. Wang, “Salient object detection: A discriminative regional feature integration approach,” _Int. J. Comput. Vis._ , vol. 123, pp. 251–268, 2013.
  * [26] X. Li, H. Lu, L. Zhang, X. Ruan, and M.-H. Yang, “Saliency detection via dense and sparse reconstruction,” in _Int. Conf. Comput. Vis._ , 2013, pp. 2976–2983.
  * [27] F. Perazzi, P. Krähenbühl, Y. Pritch, and A. Sorkine-Hornung, “Saliency filters: Contrast based filtering for salient region detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2012, pp. 733–740.
  * [28] L. Wang, H. Lu, X. Ruan, and M.-H. Yang, “Deep networks for saliency detection via local estimation and global search,” in _Conf. Comput. Vis. Pattern Recog._ , 2015, pp. 3183–3192.
  * [29] J. Zhang, S. Sclaroff, Z. L. Lin, X. Shen, B. L. Price, and R. Mech, “Unconstrained salient object detection via proposal subset optimization,” in _Conf. Comput. Vis. Pattern Recog._ , 2016, pp. 5733–5742.
  * [30] J. Kim and V. Pavlovic, “A shape-based approach for salient object detection using deep learning,” in _Eur. Conf. Comput. Vis._ , 2016, pp. 455–470.
  * [31] G. Li and Y. Yu, “Visual saliency based on multiscale deep features,” in _Conf. Comput. Vis. Pattern Recog._ , 2015, pp. 5455–5463.
  * [32] R. Zhao, W. Ouyang, H. Li, and X. Wang, “Saliency detection by multi-context deep learning,” in _Conf. Comput. Vis. Pattern Recog._ , 2015, pp. 1265–1274.
  * [33] G. Lee, Y.-W. Tai, and J. Kim, “Deep saliency with encoded low level distance map and high level features,” in _Conf. Comput. Vis. Pattern Recog._ , 2016, pp. 660–668.
  * [34] S. He, R. W. H. Lau, W. Liu, Z. Huang, and Q. Yang, “Supercnn: A superpixelwise convolutional neural network for salient object detection,” _Int. J. Comput. Vis._ , vol. 115, pp. 330–344, 2015.
  * [35] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in _Conf. Comput. Vis. Pattern Recog._ , 2015, pp. 3431–3440.
  * [36] A. Borji, M.-M. Cheng, H. Jiang, and J. Li, “Salient object detection: A benchmark,” _IEEE Trans. Image Process._ , vol. 24, pp. 5706–5722, 2015\.
  * [37] Ali Borji, M.-M. Cheng, H. Jiang, and J. Li, “Salient object detection: A survey,” _Comput. Vis. Media_ , vol. 5, pp. 117–150, 2019.
  * [38] J.-J. Liu, Q. Hou, Z.-A. Liu, and M.-M. Cheng, “Poolnet+: Exploring the potential of pooling for salient object detection,” _IEEE Trans. Pattern Anal. Mach. Intell._ , pp. 1–1, 2022.
  * [39] T.-Y. Lin, P. Dollár, R. B. Girshick, K. He, B. Hariharan, and S. J. Belongie, “Feature pyramid networks for object detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2017, pp. 936–944.
  * [40] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in _Med. Image. Comput. Comput. Assist. Interv._ , 2015, pp. 234–241.
  * [41] J.-X. Zhao, J. Liu, D.-P. Fan, Y. Cao, J. Yang, and M.-M. Cheng, “Egnet: Edge guidance network for salient object detection,” in _Int. Conf. Comput. Vis._ , 2019, pp. 8778–8787.
  * [42] X. Zhang, T. Wang, J. Qi, H. Lu, and G. Wang, “Progressive attention guided recurrent network for salient object detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2018, pp. 714–722.
  * [43] N. Liu, J. Han, and M.-H. Yang, “Picanet: Learning pixel-wise contextual attention for saliency detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2018, pp. 3089–3098.
  * [44] T. Zhao and X. Wu, “Pyramid feature attention network for saliency detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2019, pp. 3080–3089.
  * [45] S. Chen, X. Tan, B. Wang, and X. Hu, “Reverse attention for salient object detection,” in _Eur. Conf. Comput. Vis._ , 2018, pp. 236–252.
  * [46] X. Qin, Z. Zhang, C. Huang, C. Gao, M. Dehghan, and M. Jagersand, “Basnet: Boundary-aware salient object detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2019, pp. 7479–7489.
  * [47] M. Liao, Z. Zou, Z. Wan, C. Yao, and X. Bai, “Real-time scene text detection with differentiable binarization and adaptive scale fusion,” _IEEE Trans. Pattern Anal. Mach. Intell._ , pp. 1–1, 2022.
  * [48] Y. Liu, H. Chen, C. Shen, T. He, L. Jin, and L. Wang, “Abcnet: Real-time scene text spotting with adaptive bezier-curve network,” in _Conf. Comput. Vis. Pattern Recog._ , 2020, pp. 9806–9815.
  * [49] Y. Zhu and J. Du, “Textmountain: Accurate scene text detection via instance segmentation,” _Pattern Recognit._ , vol. 110, p. 107336, 2021.
  * [50] X. Qin, D.-P. Fan, C. Huang, C. Diagne, Z. Zhang, A. C. Sant’Anna, A. Suàrez, M. Jagersand, and L. Shao, “Boundary-aware segmentation network for mobile and web applications,” _arXiv preprint arXiv:2101.04704_ , 2021.
  * [51] M. Zhuge, D.-P. Fan, N. Liu, D. Zhang, D. Xu, and L. Shao, “Salient object detection via integrity learning,” _IEEE Trans. Pattern Anal. Mach. Intell._ , 2022.
  * [52] Y. Su, J. Deng, R. Sun, G. Lin, and Q. Wu, “A unified transformer framework for group-based segmentation: Co-segmentation, co-saliency detection and video salient object detection,” _arXiv preprint arXiv:2203.04708_ , 2022\.
  * [53] W. Liu, C. Zhang, G. Lin, and F. Liu, “Crnet: Cross-reference networks for few-shot segmentation,” in _Conf. Comput. Vis. Pattern Recog._ , 2020, pp. 4164–4172.
  * [54] M. Siam, N. Doraiswamy, B. N. Oreshkin, H. Yao, and M. Jägersand, “Weakly supervised few-shot object segmentation using co-attention with visual and semantic embeddings,” in _Int. Joint Conf. Artif. Intell._ , 2020.
  * [55] T.-W. Ke, J.-J. Hwang, Y. Guo, X. Wang, and S. X. Yu, “Unsupervised hierarchical semantic segmentation with multiview cosegmentation and clustering transformers,” in _Conf. Comput. Vis. Pattern Recog._ , 2022, pp. 2571–2581.
  * [56] H. Zhang, H. Zhang, C. Wang, and J. Xie, “Co-occurrent features in semantic segmentation,” in _Conf. Comput. Vis. Pattern Recog._ , 2019, pp. 548–557.
  * [57] W. Liu, C. Zhang, G. Lin, and F. Liu, “Crnet: Cross-reference networks for few-shot segmentation,” in _Conf. Comput. Vis. Pattern Recog._ , 2020, pp. 4165–4173.
  * [58] W. Li, O. Hosseini Jafari, and C. Rother, “Deep object co-segmentation,” in _Asian Conf. Comput. Vis._ , 2018, pp. 638–653.
  * [59] K.-Y. Chang, T.-L. Liu, and S.-H. Lai, “From co-saliency to co-segmentation: An efficient and fully unsupervised energy minimization model,” in _Conf. Comput. Vis. Pattern Recog._ , 2011, pp. 2129–2136.
  * [60] C. Rother, T. Minka, A. Blake, and V. Kolmogorov, “Cosegmentation of image pairs by histogram matching-incorporating a global constraint into mrfs,” in _Conf. Comput. Vis. Pattern Recog._ , 2006, pp. 993–1000.
  * [61] H. Chen, Y. Huang, and H. Nakayama, “Semantic aware attention based deep object co-segmentation,” in _Asian Conf. Comput. Vis._ , 2018, pp. 435–450.
  * [62] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” _Neural computation_ , vol. 9, no. 8, pp. 1735–1780, 1997.
  * [63] C. Zhang, G. Li, G. Lin, Q. Wu, and R. Yao, “Cyclesegnet: Object co-segmentation with cycle refinement and region correspondence,” _IEEE Trans. Image Process._ , vol. 30, pp. 5652–5664, 2021.
  * [64] B. Li, Z. Sun, Q. Li, Y. Wu, and A. Hu, “Group-wise deep object co-segmentation with co-attention recurrent neural network,” in _Int. Conf. Comput. Vis._ , 2019, pp. 8519–8528.
  * [65] X. Wang, S. You, X. Li, and H. Ma, “Weakly-supervised semantic segmentation by iteratively mining common object features,” in _Conf. Comput. Vis. Pattern Recog._ , 2018, pp. 1354–1362.
  * [66] K.-J. Hsu, Y.-Y. Lin, and Y.-Y. Chuang, “Co-attention cnns for unsupervised object co-segmentation,” in _Int. Joint Conf. Artif. Intell._ , 2018, pp. 748–756.
  * [67] N. Liu and J. Han, “Dhsnet: Deep hierarchical saliency network for salient object detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2016, pp. 678–686.
  * [68] G. Li and Y. Yu, “Deep contrast learning for salient object detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2016, pp. 478–487.
  * [69] X. Li, L. Zhao, L. Wei, M.-H. Yang, F. Wu, Y. Zhuang, H. Ling, and J. Wang, “Deepsaliency: Multi-task deep neural network model for salient object detection,” _IEEE Trans. Image Process._ , vol. 25, no. 8, pp. 3919–3930, 2016.
  * [70] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, “Slic superpixels compared to state-of-the-art superpixel methods,” _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 34, no. 11, pp. 2274–2282, 2012.
  * [71] D. E. Jacobs, D. B. Goldman, and E. Shechtman, “Cosaliency: Where people look when comparing images,” in _ACM symposium on User interface software and technology_ , 2010, pp. 219–228.
  * [72] D. Zhang, D. Meng, and J. Han, “Co-saliency detection via a self-paced multiple-instance learning framework,” _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 39, no. 5, pp. 865–878, 2017.
  * [73] L. Tang, “Cosformer: Detecting co-salient object with transformers,” _arXiv preprint arXiv:2104.14729_ , 2021.
  * [74] G. Ren, T. Dai, and T. Stathaki, “Adaptive intra-group aggregation for co-saliency detection,” in _IEEE Int. Conf. Acoust. Speech SP_ , 2022, pp. 2520–2524.
  * [75] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Conf. Comput. Vis. Pattern Recog._ , 2016, pp. 770–778.
  * [76] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in _Int. Conf. Learn. Represent._ , 2015\.
  * [77] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in _Conf. Comput. Vis. Pattern Recog._ , 2016, pp. 2818–2826.
  * [78] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in _Int. Conf. Learn. Represent._ , 2021.
  * [79] W. Wang, E. Xie, X. Li, D.-P. Fan, K. Song, D. Liang, T. Lu, P. Luo, and L. Shao, “Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,” in _Int. Conf. Comput. Vis._ , 2021, pp. 548–558.
  * [80] W. Wang, E. Xie, X. Li, D.-P. Fan, K. Song, D. Liang, T. Lu, P. Luo, and L. Shao, “Pvtv2: Improved baselines with pyramid vision transformer,” _Comput. Vis. Media_ , vol. 8, no. 3, pp. 1–10, 2022.
  * [81] X. Qian, Y. Zeng, W. Wang, and Q. Zhang, “Co-saliency detection guided by group weakly supervised learning,” _IEEE Trans. Multimedia_ , pp. 1–1, 2022.
  * [82] X. Zheng, Z. Zha, and L. Zhuang, “A feature-adaptive semi-supervised framework for co-saliency detection,” in _ACM Int. Conf. Multimedia_ , 2018, pp. 959–966.
  * [83] D. Zhang, D. Meng, and J. Han, “Co-saliency detection via a self-paced multiple-instance learning framework,” _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 39, pp. 865–878, 2017.
  * [84] K.-J. Hsu, C.-C. Tsai, Y.-Y. Lin, X. Qian, and Y.-Y. Chuang, “Unsupervised cnn-based co-saliency detection with graphical optimization,” in _Eur. Conf. Comput. Vis._ , 2018, pp. 502–518.
  * [85] B. Jiang, X. Jiang, A. Zhou, J. Tang, and B. Luo, “A unified multiple graph learning and convolutional network model for co-saliency estimation,” in _ACM Int. Conf. Multimedia_ , 2019, pp. 1375–1382.
  * [86] B. Jiang, X. Jiang, J. Tang, B. Luo, and S. Huang, “Multiple graph convolutional networks for co-saliency detection,” in _Int. Conf. Multimedia and Expo_ , 2019, pp. 332–337.
  * [87] K. Zhang, T. Li, S. Shen, B. Liu, J. Chen, and Q. Liu, “Adaptive graph convolutional network with attention graph clustering for co-saliency detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2020, pp. 9047–9056.
  * [88] X. Yao, J. Han, D. Zhang, and F. Nie, “Revisiting co-saliency detection: A novel approach based on two-stage multi-view spectral rotation co-clustering,” _IEEE Trans. Image Process._ , vol. 26, no. 7, pp. 3196–3209, 2017.
  * [89] B. Li, Z. Sun, L. Tang, Y. Sun, and J. Shi, “Detecting robust co-saliency with recurrent co-attention neural network,” in _Int. Joint Conf. Artif. Intell._ , 2019, pp. 818–825.
  * [90] K. R. Jerripothula, J. Cai, and J. Yuan, “Quality-guided fusion-based co-saliency estimation for image co-segmentation and colocalization,” _IEEE Trans. Multimedia_ , vol. 20, no. 9, pp. 2466–2477, 2018.
  * [91] X. Lu, W. Wang, C. Ma, J. Shen, L. Shao, and F. Porikli, “See more, know more: Unsupervised video object segmentation with co-attention siamese networks,” in _Conf. Comput. Vis. Pattern Recog._ , 2019, pp. 3623–3632.
  * [92] W. Wang, T. Zhou, F. Yu, J. Dai, E. Konukoglu, and L. Van Gool, “Exploring cross-image pixel contrast for semantic segmentation,” in _Int. Conf. Comput. Vis._ , 2021, pp. 7303–7313.
  * [93] X. Lu, W. Wang, J. Shen, D. J. Crandall, and L. Van Gool, “Segmenting objects from relational visual data,” _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 44, no. 11, pp. 7885–7897, 2021.
  * [94] W. Wang, X. Lu, J. Shen, D. J. Crandall, and L. Shao, “Zero-shot video object segmentation via attentive graph neural networks,” in _Int. Conf. Comput. Vis._ , 2019, pp. 9236–9245.
  * [95] X. Zhang, Y. Wei, and Y. Yang, “Inter-image communication for weakly supervised localization,” in _Eur. Conf. Comput. Vis._ , 2020, pp. 271–287.
  * [96] J. Jeong, S. Lee, J. Kim, and N. Kwak, “Consistency-based semi-supervised learning for object detection,” _Adv. Neural Inform. Process. Syst._ , vol. 32, 2019.
  * [97] T. Zhou, L. Li, X. Li, C.-M. Feng, J. Li, and L. Shao, “Group-wise learning for weakly supervised semantic segmentation,” _IEEE Trans. Image Process._ , vol. 31, pp. 799–811, 2021.
  * [98] G. Ma, C. Chen, S. Li, C. Peng, A. Hao, and H. Qin, “Salient object detection via multiple instance joint re-learning,” _IEEE Trans. Multimedia_ , vol. 22, no. 2, pp. 324–336, 2019.
  * [99] C. Vondrick, A. Shrivastava, A. Fathi, S. Guadarrama, and K. Murphy, “Tracking emerges by colorizing videos,” in _Eur. Conf. Comput. Vis._ , 2018, pp. 391–408.
  * [100] Z. Lai and W. Xie, “Self-supervised video representation learning for correspondence flow,” in _Brit. Mach. Vis. Conf._ , 2019, p. 299.
  * [101] X. Wang, A. Jabri, and A. A. Efros, “Learning correspondence from the cycle-consistency of time,” in _Conf. Comput. Vis. Pattern Recog._ , 2019, pp. 2566–2576.
  * [102] Z. Lai, E. Lu, and W. Xie, “Mast: A memory-augmented self-supervised tracker,” in _Conf. Comput. Vis. Pattern Recog._ , 2020, pp. 6479–6488.
  * [103] B. Li, W. Wu, Q. Wang, F. Zhang, J. Xing, and J. Yan, “Siamrpn++: Evolution of siamese visual tracking with very deep networks,” in _Conf. Comput. Vis. Pattern Recog._ , 2019, pp. 4282–4291.
  * [104] Q. Fan, W. Zhuo, C.-K. Tang, and Y.-W. Tai, “Few-shot object detection with attention-rpn and multi-relation detector,” in _Conf. Comput. Vis. Pattern Recog._ , 2020, pp. 4013–4022.
  * [105] R. Margolin, L. Zelnik-Manor, and A. Tal, “How to evaluate foreground maps?” in _Conf. Comput. Vis. Pattern Recog._ , 2014, pp. 248–255.
  * [106] M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” _Int. J. Comput. Vis._ , vol. 88, pp. 303–308, 2009.
  * [107] W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, “Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,” in _Conf. Comput. Vis. Pattern Recog._ , 2016, pp. 1874–1883.
  * [108] L. Wang, H. Lu, Y. Wang, M. Feng, D. Wang, B. Yin, and X. Ruan, “Learning to detect salient objects with image-level supervision,” in _Conf. Comput. Vis. Pattern Recog._ , 2017, pp. 3796–3805.
  * [109] F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in _Conf. Comput. Vis. Pattern Recog._ , 2015, pp. 815–823.
  * [110] X. Dong and J. Shen, “Triplet loss in siamese network for object tracking,” in _Eur. Conf. Comput. Vis._ , 2018, pp. 459–474.
  * [111] A. Hermans, L. Beyer, and B. Leibe, “In defense of the triplet loss for person re-identification,” _arXiv preprint arXiv:1703.07737_ , 2017.
  * [112] C. Wang, Z. Zha, D. Liu, and H. Xie, “Robust deep co-saliency detection with group semantic,” in _AAAI Conf. Art. Intell._ , 2019, pp. 8917–8924.
  * [113] N. Zhang, J. Han, N. Liu, and L. Shao, “Summarize and search: Learning consensus-aware dynamic convolution for co-saliency detection,” in _Int. Conf. Comput. Vis._ , 2021, pp. 4167–4176.
  * [114] T.-Y. Lin, M. Maire, S. J. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in _Eur. Conf. Comput. Vis._ , 2014, pp. 740–755.
  * [115] D. Batra, A. Kowdle, D. Parikh, J. Luo, and T. Chen, “icoseg: Interactive co-segmentation with intelligent scribble guidance,” in _Conf. Comput. Vis. Pattern Recog._ , 2010, pp. 3169–3176.
  * [116] J. Winn, A. Criminisi, and T. Minka, “Object categorization by learned universal visual dictionary,” in _Int. Conf. Comput. Vis._ , vol. 2, 2005, pp. 1800–1807.
  * [117] R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, “Frequency-tuned salient region detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2009, pp. 1597–1604.
  * [118] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in _Int. Conf. Mach. Learn._ , 2015, pp. 448–456.
  * [119] S. Yu, J. Xiao, B. Zhang, and E. G. Lim, “Democracy does matter: Comprehensive feature mining for co-salient object detection,” in _Conf. Comput. Vis. Pattern Recog._ , 2022.
  * [120] K. Zhang, M. Dong, B. Liu, X.-T. Yuan, and Q. Liu, “Deepacg: Co-saliency detection via semantic-aware contrast gromov-wasserstein distance,” in _Conf. Comput. Vis. Pattern Recog._ , 2021, pp. 13 698–13 707.
  * [121] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga _et al._ , “Pytorch: An imperative style, high-performance deep learning library,” in _Adv. Neural Inform. Process. Syst._ , vol. 32, 2019.
  * [122] B. Zhou, A. Khosla, À. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” in _Conf. Comput. Vis. Pattern Recog._ , 2016, pp. 2921–2929.
  * [123] K. Zhang, T. Li, B. Liu, and Q. Liu, “Co-saliency detection via mask-guided fully convolutional networks with multi-scale label smoothing,” in _Conf. Comput. Vis. Pattern Recog._ , 2019, pp. 3090–3099.
  * [124] C. Rother, V. Kolmogorov, and A. Blake, “”grabcut”: interactive foreground extraction using iterated graph cuts,” _ACM Trans. Graph._ , vol. 23, no. 3, pp. 309–314, 2004.
  * [125] H. Peng, B. Li, W. Xiong, W. Hu, and R. Ji, “Rgbd salient object detection: A benchmark and algorithms,” in _Eur. Conf. Comput. Vis._ , 2014, pp. 92–109.