# Image Clustering with Contrastive Learning and Multi-scale Graph
Convolutional Networks

Yuanku Xu, Dong Huang, Chang-Dong Wang,
and Jian-Huang Lai  This project was supported by the NSFC (61976097, 61876193
& 62076258) and the Natural Science Foundation of Guangdong Province
(2021A1515012203). (Corresponding author: Dong Huang) Y. Xu and D. Huang are
with the College of Mathematics and Informatics, South China Agricultural
University, Guangzhou, China.
E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>C.-D. Wang and J.-H.
Lai are with the School of Computer Science and Engineering, Sun Yat-sen
University, Guangzhou, China, and also with Guangdong Key Laboratory of
Information Security Technology, Guangzhou, China, and also with Key
Laboratory of Machine Intelligence and Advanced Computing, Ministry of
Education, China.
E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Deep clustering has recently attracted significant attention with its
promising ability of joint representation learning and clustering via deep
neural networks. Despite the remarkable progress, most of the previous deep
clustering works still suffer from two limitations. First, many of them focus
on some distribution-based clustering loss, lacking the ability to exploit
sample-wise (or augmentation-wise) relationships via contrastive learning.
Second, they often neglect the indirect sample-wise structure information,
overlooking the rich possibilities of multi-scale neighborhood structure
learning. In view of this, this paper presents a new deep clustering approach
termed Image clustering with contrastive learning and multi-scale Graph
Convolutional Networks (IcicleGCN), which bridges the gap between
convolutional neural network (CNN) and graph convolutional network (GCN) as
well as the gap between contrastive learning and multi-scale neighborhood
structure learning for the image clustering task. The proposed IcicleGCN
framework consists of four main modules, namely, the CNN-based backbone, the
Instance Similarity Module (ISM), the Joint Cluster Structure Learning and
Instance reconstruction Module (JC-SLIM), and the Multi-scale GCN module
(M-GCN). Specifically, with two random augmentations performed on each image,
the backbone network with two weight-sharing views is utilized to learn the
representations for the augmented samples, which are then fed to ISM and JC-
SLIM for instance-level and cluster-level contrastive learning, respectively,
during which an auto-encoder in JC-SLIM is also pretrained to serve as a
bridge to the M-GCN module. Further, to enforce multi-scale neighborhood
structure learning, two streams of GCNs and the auto-encoder (from JC-SLIM)
are simultaneously trained via (i) the layer-wise interaction with
representation fusion and (ii) the joint self-adaptive learning that ensures
their last-layer output distributions to be consistent. Experiments on
multiple image datasets demonstrate the superior clustering performance of
IcicleGCN over the state-of-the-art.

###### Index Terms:

Deep clustering, Image clustering, Graph convolutional network, Contrastive
learning, Multi-scale neighborhood structure learning.

##  I Introduction

Deep learning has achieved remarkable success in many supervised learning
applications [1], which typically requires a considerable amount of training
samples with true labels. To alleviate the probably labor-intensive task of
data annotation, in recent years, the unsupervised learning techniques have
gained increasing attention, among which the clustering analysis plays a
fundamental role [2, 3, 4, 5, 6, 7, 8, 9].

The traditional clustering methods [10, 11, 12] generally rely on hand-crafted
features, which lack the ability of feature representation learning and may
result in sub-optimal clustering performance for high-dimensional complex data
like images and videos. Recently some efforts have been made to incorporate
the deep learning technique into the unsupervised clustering task. For
example, Yang et al.[2] proposed the Deep Clustering Networks (DCN) method,
where a reconstruction loss of the auto-encoder and a $K$-means based
clustering loss are jointly optimized. Chang et al.[3] presented the Deep
Adaptive image Clustering (DAC) method by formulating the clustering problem
as a binary pairwise classification problem and enforcing the learned labels
features to be one-hot vectors which can be used for image clustering. Xie et
al.[4] developed the Deep Embedded Cluster (DEC) method, which aims to map the
learned features in the data space to a low-dimensional feature space with a
Kullback-Leibler (KL) divergence based clustering loss. Yang et al. [5]
devised the Joint Unsupervised LEarning (JULE) method, which iteratively
updates a convolutional neural network (CNN) by performing the agglomerative
clustering in the forward propagation and learning the feature representations
in the backward propagation.

Although these deep clustering works [2, 3, 4, 5, 6, 7, 8, 9] have made
significant progress, yet there are still some critical questions that remain
to be addressed. Especially, in this paper, our focus lies in the following
three key questions.

Figure 1: The network architecture of IcicleGCN consists of four modules,
i.e., the CNN-based backbone (which learns the representations for two
augmentation views with shared weights), the ISM module (which enforces the
instance-level contrastive learning), the JC-SLIM module (which enforces
cluster-level contrastive learning and meanwhile pretrains an auto-encoder as
a bridge to the neighborhood structure learning in the next module), and the
M-GCN module (which jointly leverages multiple GCNs for multi-scale
neighborhood structure learning).

  1. Q1:

Many of the previous works tend to utilize some clustering loss (usually
related to the label distributions) to guide the unsupervised training, which
often overlook the sample-wise (or augmentation-wise) relationships in their
learning process. With the contrastive learning recently showing its promising
ability of self-supervised learning via positive and negative sample pairs
[13], the first question arises as to how to incorporate the contrastive
learning into the deep clustering framework for better representation learning
and clustering.

  2. Q2:

As the contrastive learning usually considers the direct sample-wise
relationships, e.g., the relationships between the positive pairs and the
negative pairs, the second question arises as to how to go beyond the direct
sample-wise relationship to explore the rich information in neighborhood
structures, or even enforce neighborhood structure learning for deep
clustering.

  3. Q3:

Starting from the first two questions, the third question emerges as to how to
extend the neighborhood structure learning (in Q2) from single-scale to multi-
scale, and jointly leverage multi-scale neighborhood structure learning and
contrastive learning in a unified framework.

Recently, several efforts have been carried out to address some of the above
three questions. Li et al. [14] proposed the Contrastive Clustering (CC)
method to incorporate contrastive learning into deep clustering, which takes
into account the sample-wise relationships (between positive sample-pairs and
negative sample pairs) but still overlooks the sample-wise neighborhood
structure. To investigate the neighborhood structure, van Gansbeke et al. [15]
presented a two-stage deep clustering method termed Semantic Clustering by
Adopting Nearest neighbors (SCAN), where the first stage employs the
contrastive learning to learn the feature representation for constructing a
$k$-nearest neighbor ($k$-NN) graph and the second stage aims to maximize the
similarity between each sample and its $k$-NNs. Zhong et al. [16] designed the
Graph Contrastive Clustering (GCC) method, which also utilizes a $k$-NN graph
to provide more structure information for contrastive representation learning
and clustering. Although SCAN [15] and GCC [16] have gone one step further to
exploit the neighborhood structure information, yet they are still restricted
to the static neighborhood connections and lack the ability to dynamically
explore the higher-order connections via neighborhood structure learning.

In terms of neighborhood structure learning, the Graph Convolutional Network
(GCN) provides an alternative and powerful tool [17]. As an early attempt, Bo
et al. [18] devised the Structural Deep Clustering Network (SDCN) method,
which first pretrains an auto-encoder and then simultaneously trains the
encoder and the GCN with a KL divergence loss. With the incorporation of GCN
[18], the neighborhood structure learning can be enforced in SDCN. However, on
the one hand, SDCN takes vectorized feature representations as input, which
lacks convolutional layers to extract spatial information from complex image
data. On the other hand, SDCN typically uses a single $k$-NN graph (which
represents a specific scale of the neighborhood structure) as the initial
graph for GCN, but ignores the possibilities of extending the neighborhood
structure learning from a single scale to multiple scales, so as to explore
more comprehensive structure information. It remains a challenging problem how
to bridge the gap between CNN and GCN as well as the gap between contrastive
learning and neighborhood structure learning, and further, how to go from
single-scale to multi-scale neighborhood structure learning in a unified deep
image clustering framework.

To address the above problem, in this paper, we present a novel deep
clustering approach termed Image clustering with contrastive learning and
multi-scale Graph Convolutional Networks (IcicleGCN). The overall architecture
of IcicleGCN consists of four modules, namely, the backbone network, the
Instance Similarity Module (ISM), the Joint Cluster Structure Learning and
Instance reconstruction Module (JC-SLIM), and the Multi-scale GCN module
(M-GCN) (as shown in Fig. 1). Specifically, with two types of data
augmentations randomly performed on each sample image, we utilize a weight-
sharing CNN to learn the representations of the sample pairs, which are then
fed to the later modules for contrastive learning and multi-scale neighborhood
structure learning (via GCNs). In ISM, we utilize a two-layer multilayer
perceptron (MLP) with an instance-level contrastive loss is to maximize the
similarity between positive pairs and minimize the similarity between negative
pairs. In JC-SLIM, an auto-encoder is incorporated, which simultaneously
exploits a cluster-level contrastive loss to learn the cluster structures and
an instance reconstruction loss to pretrain the encoder that also serves as a
bridge to the M-GCN module. In M-GCN, two $k$-NN graphs are first built to
represent two different scales of neighborhood structures, upon which two
streams of GCNs and the auto-encoder (which is shared with the JC-SLIM module)
are iteratively trained for multi-scale neighborhood structure learning and
clustering. Extensive experiments are conducted on four image datasets, which
confirm the superiority of the proposed IcicleGCN approach over the state-of-
the-art deep clustering approaches.

For clarity, the main contributions of this paper are summarized as follows:

  1. 1.

This paper for the first time (to our knowledge) enables the multi-scale
neighborhood structure learning for the image clustering task by taking
advantage of multi-scale GCNs with joint self-adaptive learning.

  2. 2.

This paper presents a novel deep image clustering approach termed IcicleGCN,
which simultaneously models the instance-level contrastive learning, the
global clustering structure learning, and the multi-scale neighborhood
structure learning in a unified framework.

  3. 3.

Extensive experiments demonstrate the superior clustering performance of our
IcicleGCN approach on several challenging image datasets in comparison with
the state-of-the-art deep clustering approaches.

The rest of the paper is organized as follows. Section II reviews the related
works on deep clustering and GCN. Section III describes the overall framework
of IcicleGCN. Section IV reports the experimental results. Finally, Section V
concludes the paper.

##  II Related Work

In this section, we review the related works on deep clustering and GCN in
Sections II-A and II-B, respectively.

###  II-A Deep Clustering

Deep clustering aims to learn high-quality feature representations to improve
the clustering result, where the clustering result may in turn guide the deep
neural network to learn better feature representations [19]. A considerable
number of deep clustering methods have been designed in recent years. For
example, Yang et al.[2] combined the auto-encoder network with the clustering
task, where an auto-encoder is trained to reconstruct original features and a
$K$-means clustering loss is incorporated to learn the cluster structure.
Chang et al.[3] mapped the clustering problem into a binary pairwise
classification framework to judge whether each image pair belongs to the same
cluster, and clustered the images by the local maximum response of the label
features in the deep neural network. Xie et al.[4] presented the DEC method,
which aims to map the learned features in the data space to a low-dimensional
feature space, where the KL-divergence loss is used to iteratively optimize
the cluster structure. Huang et al.[20] first utilized an auto-encoder to
learn simplified representation from raw data, and then exploited a local
preservation constraint to preserve the local structural properties of the
data. Yang et al. [5] performed deep clustering by iteratively updating the
CNN with the agglomerative clustering conducted in the forward propagation and
the feature representations learned in the backward propagation.

Besides these deep clustering methods that utilize some clustering loss in the
deep neural network for cluster structure learning, another popular direction
in recent years is to incorporate the contrastive learning paradigm [14, 21,
22, 23] for better representation learning and clustering. Specifically, Li et
al. [14] presented an end-to-end deep clustering method which performs the
instance-level contrastive learning and the cluster-level contrastive learning
at the same time. Van Gansbeke et al. [15] developed a two-stage method termed
SCAN. In the first stage, it performs contrastive learning to learn feature
representation for finding the nearest neighbors of each image. In the second
stage, it obtains the clustering result via a semantic clustering loss with
the nearest neighbors of each image considered [15].

###  II-B Graph Convolutional Network

The concept of graph neural network (GNN) was first proposed by Gori et
al.[24] and further elaborated in GNN*[25]. Early works on GNN tend to use
neural networks to transmit neighborhood information in an iterative way until
reaching a stable fixed point.

The significant success of CNN in the field of computer vision has inspired
some researchers to focus on convolutional operators for learning neighborhood
information, which give rise to the GCN. Bruna et al.[26] designed a variant
of graph convolution based on spectral graph theory, which uses the spectral
domain network to generalize convolutional networks through graph Fourier
transform. Henaff et al. [27] trained a graph convolutional layer that can
perform forward and back propagation given a Fourier matrix, an interpolation
kernel, and weights. Defferrard et al.[28] developed ChebNet to generalize
CNNs to graph data thoroughly using spectral theoretical formulations, where
the graph signal filtering and the graph coarsening are performed in the
feature extraction stage. These spectral-based methods generally learn the
entire graph structure at the same time. Since the processing of the entire
graph structure is computationally expensive, these spectral-based methods are
often difficult to generalize to large datasets.

Besides the spectral-based methods, there is an increasing number of spatial-
based graph convolution methods in recent years. Monti et al. [29] developed
the MoNet framework to generalize traditional CNNs to non-Euclidean spaces.
Niepert et al. [30] proposed a general and efficient framework termed PATCHY-
SAN for representation learning on arbitrary graph networks, which constructs
locally connected neighborhoods in the graph and performs operations such as
convolution on these neighborhoods. Gao et al. [31] presented the Large-scale
learnable Graph Convolutional Networks (LGCN) method, which creates the
learnable graph convolutional layers and converts general graph data into
grid-structure data with regular convolution operations. These methods
directly perform convolution operations on the graph domain by aggregating the
information of the neighboring nodes, while ignoring the representation
information of the data itself. Shi et al. [32] exploited graph convolution
operations on multi-scale prototype graphs for face recognition with image
sets, which relies on the prior knowledge of image sets for generating
prototypes and cannot perform sample-wise contrastive learning and
unsupervised clustering. Recently, to exploit the GCN for the unsupervised
clustering task, Bo et al. [18] presented the SDCN method, which combines the
features obtained from each layer of an encoder with the features learned by
the GCN, and iteratively optimizes the network via a KL-divergence loss.
Although SDCN considers the representation information of the data itself and
the sample-wise structural information, yet it only takes the single-scale
neighborhood information (with a single $k$-NN graph) as input, which
overlooks the potential opportunities of jointly utilizing multi-scale
neighborhood structures. Furthermore, SDCN relies on vectorized features,
which restricts it application for complex images due to its lack of image-
wise convolutional layers.

##  III Proposed Framework

In this section, we describe the proposed IcicleGCN approach, which bridges
the gap between CNN and GCN and also the gap between contrastive learning and
multi-scale neighborhood structure learning. As illustrated in the Fig. 1,
IcicleGCN consists of four modules, the CNN-based backbone, the ISM module,
the JC-SLIM module, and the M-GCN module, which will be described in Sections
III-A, III-B, III-C, and III-D, respectively.

###  III-A Backbone

Our IcicleGCN approach utilizes a CNN as the backbone, which incorporates two
augmentations views with shared weights, so as to produce two augmented
samples for each input image. Given a mini-batch of $N$ images, denoted as
$\\{y_{1},y_{2},\cdots,y_{N}\\}$, two types of augmentations are randomly
selected for each image, leading to a total of $2\cdot N$ augmented samples
for each mini-batch, which will then be fed to the following three modules
(i.e., ISM, JC-SLIM, and M-GCN) for the instance-level contrastive learning,
the joint cluster-level contrastive learning and auto-encoder training, and
the multi-scale neighborhood structure learning, respectively.

###  III-B ISM

The purpose of contrastive learning is to train the neural network by
maximizing the similarity between the positive pairs while minimizing the
similarity between the negative pairs [13]. In this section, we describe the
ISM module, which enforces the instance-level contrastive learning.

For an input image $y_{i}$, two augmented samples are generated, denoted as
$y_{i}^{a}$ and $y_{i}^{b}$, respectively. Then the two augmented samples from
the same input image are regarded as a positive sample pair, while the other
$2\cdot(N-1)$ sample pairs are regarded as the negative pairs. For an
augmented sample $y_{i}^{u}$, its feature representation learned by the
backbone is denoted as $z_{i}^{u}$ (for $u\in\\{a,b\\}$).

In the ISM module, a neural network with two fully-connected layers is
leveraged to map the representations $z_{i}^{a}$ and $z_{i}^{b}$ to a low-
dimensional space, denoted as $m_{i}^{a}=S\left(z_{i}^{a}\right)$ and
$m_{i}^{b}=S\left(z_{i}^{b}\right)$. Thereby, the similarity between a pair of
samples can be computed by the cosine similarity, that is

$\displaystyle s\left(m_{i}^{u_{1}},m_{j}^{u_{2}}\right)=$
$\displaystyle\frac{\left(m_{i}^{u_{1}}\right)\left(m_{j}^{u_{2}}\right)^{\top}}{\left\|m_{i}^{u_{1}}\right\|\left\|m_{j}^{u_{2}}\right\|},$
(1) $u_{1},u_{2}\in\\{a,b\\}$ $\displaystyle,\text{$i,j\in[1,N]$}.$

To maximize the similarity between positive pairs and minimize the similarity
between negative pairs, the contrastive instance similarity loss for a sample
$y_{i}^{a}$ is defined as [13]

$\displaystyle{\ell}_{i}^{a}=-\log\left(\frac{e^{s(m_{i}^{a},m_{i}^{b})/\tau_{I}}}{\sum_{j=1}^{N}(e^{s(m_{i}^{a},m_{j}^{a})/\tau_{I}}+e^{s(m_{i}^{a},m_{j}^{b})/\tau_{I}})}\right),$
(2)

where $\tau_{I}$ is the temperature parameter. With the two augmented samples
of each image considered, we have the contrastive instance similarity loss
(for the $N$ images in a mini-batch) as

$\displaystyle\mathcal{L}_{CIS-
Loss}=\frac{1}{2N}\sum_{i=1}^{N}\left(\ell_{i}^{a}+\ell_{i}^{b}\right).$ (3)

Thereby, with the instance-level contrastive learning enforced, we proceed to
incorporate the cluster-level contrastive learning and the multi-scale
neighborhood structure learning in the following.

###  III-C JC-SLIM

In this section, we describe the JC-SLIM module, whose technical role is two-
fold. On the one hand, it enforces the cluster-level contrastive learning. On
the other hand, it also serves as a bridge between the contrastive learning
via CNN and the neighborhood structure learning via GCN by sharing the
pretrained auto-encoder with the M-GCN module.

Specifically, an auto-encoder is utilized in the JC-SLIM module, where the
encoder maps the representations learned by the backbone to a low-dimensional
space for the cluster structure learning (i.e., the cluster-level contrastive
learning) and meanwhile collaborates with the decoder for the instance
reconstruction. In the following, the contrastive cluster structure loss and
the instance reconstruction loss will be presented in Sections III-C1 and
III-C2, respectively.

####  III-C1 Contrastive Cluster Structure Loss

Besides the contrastive learning at the instance-level, it is also expected to
learn the cluster structures by considering the consistency of the two
augmentation views. The intuition is to maximize the similarity between the
cluster distributions of the two augmentation views, which gives rise to the
cluster-level contrastive learning [14].

Given the feature representations (of an augmented pair $y_{i}^{a}$ and
$y_{i}^{b}$) learned by the backbone, i.e., $z_{i}^{a}$ and $z_{i}^{b}$, we
respectively map them to the $K$-dimensional representations $w^{a}_{i}$ and
$w^{b}_{i}$ via the encoder (with softmax operation) of the auto-encoder.
Here, the $K$-dimensional vector $w^{a}_{i}$ can be viewed as the probability
of the sample $y_{i}^{a}$ belonging to each of the $K$ clusters.

By mapping the $N$ pairs of augmented samples to the $K$-dimensional space via
the encoder, we can obtain the representation matrices for the two
augmentation views, denoted as $W^{a}\in\mathbb{R}^{N\times K}$ and
$W^{b}\in\mathbb{R}^{N\times K}$, respectively. Here, $w^{u}_{i}$ is the
$i$-th row in $W^{u}$ (for $u\in\\{a,b\\}$). Let $\hat{w}_{i}^{u}$ denote the
$i$-th column in $W^{u}$, which indicates the probability of the $N$ samples
belonging to the $i$-th cluster. Specifically, $\hat{w}_{i}^{a}$ and
$\hat{w}_{i}^{b}$ are regarded as a positive cluster pair, while the other
$2\cdot(K-1)$ cluster pairs are regarded as the negative pairs. Then the
contrastive cluster structure loss for $\hat{w}_{i}^{a}$ is defined as

$\displaystyle\hat{\ell}_{i}^{a}=-\log\left(\frac{e^{s(\hat{w}_{i}^{a},\hat{w}_{i}^{b})/\tau_{C}}}{\sum_{j=1}^{K}(e^{s(\hat{w}_{i}^{a},\hat{w}_{j}^{a})/\tau_{C}}+e^{s(\hat{w}_{i}^{a},\hat{w}_{j}^{b})/\tau_{C}})}\right),$
(4)

where $\tau_{C}$ is the temperature parameter. With two augmentation views
$\hat{w}_{j}^{a}$ and $\hat{w}_{j}^{b}$ considered, the contrastive cluster
structure loss for the $K$ clusters is defined as

$\displaystyle\mathcal{L}_{CCS-
Loss}=\frac{1}{2K}\sum_{i=1}^{K}\left(\hat{\ell}_{i}^{a}+\hat{\ell}_{i}^{b}\right)-H(W^{a})-H(W^{b}),$
(5)

where $H(W^{u})=-\sum_{i=1}^{K}\left(P\left(\hat{w}_{i}^{u}\right)\log
P\left(\hat{w}_{i}^{u}\right)\right)$ (for $u\in\\{a,b\\}$) is an entropy term
that is incorporated to avoid the trivial solution of assigning all samples to
a single cluster, and
$P\left(\hat{w}_{i}^{u}\right)=\frac{1}{N}\sum_{j=1}^{N}w_{ji}^{u}$, where
$w_{ji}^{u}$ is the $(j,i)$-th entry in $W^{u}$ (for $u\in\\{a,b\\}$).

####  III-C2 Instance Reconstruction Loss

In JC-SLIM, the encoder of the auto-encoder is exploited for both the
contrastive cluster structure learning and the neighborhood structure learning
(for the next module of M-GCN). Thus, besides the contrastive cluster
structure loss, we also incorporate the instance reconstruction loss to aid
the training of the encoder, that is

$\displaystyle\mathcal{L}_{re-Loss}$
$\displaystyle=\frac{1}{2N}\sum_{i=1}^{N}\sum_{u=\\{a,b\\}}\left\|\mathbf{z}_{i}^{u}-\hat{\mathbf{z}}_{i}^{u}\right\|_{2}^{2}$
$\displaystyle=\frac{1}{2N}\sum_{u=\\{a,b\\}}\|\mathbf{Z}^{u}-\hat{\mathbf{Z}^{u}}\|_{F}^{2},$
(6)

where $\hat{\mathbf{z}}_{i}^{u}$ is the representation reconstructed by the
decoder. Before the neighborhood structure learning in the M-GCN module, we
first jointly train the backbone, the ISM module, and the JC-SLIM module via
the loss function $\mathcal{L}_{1}$, that is

$\displaystyle\mathcal{L}_{1}=\mathcal{L}_{CIS-Loss}+\mathcal{L}_{CCS-
Loss}+\mathcal{L}_{re-Loss}.$ (7)

It is noteworthy that, empirically, a linear combination of the three losses
without additional hyper-parameters to adjust their influences can already
lead to quite robust performance. By optimizing the loss function
$\mathcal{L}_{1}$, the instance-level and cluster-level contrastive learning
and the pretraining of the auto-encoder (for the next module) are
simultaneously enforced. In the next section, the auto-encoder and the
multiple GCNs will collaboratively perform the multi-scale neighborhood
structure learning for exploring more indepth and versatile sample-wise
relationships for deep clustering.

###  III-D M-GCN

In this section, we describe the M-GCN module in detail, which exploits the
auto-encoder pretrained in the JC-SLIM module, takes advantage of the multiple
GCNs for multi-scale neighborhood structure learning, and unifies the these
components via joint self-adaptive learning.

####  III-D1 Multi-Scale Neighborhood Structures

The graph structure is widely used to capture the sample-wise relationship in
data, where a $k$-NN graph (with a fixed $k$) can reflect the sample-wise
neighborhood structure of a specific scale. [15, 16, 18]. Due to variations of
different datasets or even the variations of local structures within a given
dataset, a proper scale (with an optimal $k$) is not easy to be determined for
the $k$-NN graph construction in practice. Though some previous deep
clustering methods attempt to explore the neighborhood structure [15, 16, 18],
yet they generally rely on a single $k$-NN graph and cannot go beyond the
single-scale neighborhood structure to explore the diversity in multiple
scales. In light of this, in the M-GCN module of our IcicleGCN framework,
multiple scales of neighborhood structures are jointly investigated via
multiple GCNs.

Specifically, in the M-GCN module, two $k$-NN graphs are initialized to
capture the information of different scales of neighborhood structures, which
are then jointly propagated and refined by two streams of GCNs. Note that
M-GCN can also be extended to three or more streams of GCNs. Yet we
empirically find that two scales of neighborhood structure have brought in
sufficient improvement and thus we formulate the M-GCN module with two-stream
GCNs (coupled with the auto-encoder from the JC-SLIM module).

With the backbone and the auto-encoder trained via the loss function
$\mathcal{L}_{1}$ with the instance-level and cluster-level contrastive
learning enforced, this section focuses on the neighborhood structure learning
and does not involve the contrastive learning, which means that the
augmentation pairs are no longer required here. Therefore, for each input
image $y_{i}$, it is augmented once, and then the backbone obtains its
representation $z^{i}$. The representations of $N$ samples can be stacked as a
feature matrix $Z^{b}\in\mathbb{R}^{N\times d}$, where $d$ is the output
dimension of the backbone. To initialize the multi-scale neighborhood
structures, we need to construct $k$-NN graphs of different scales from the
feature matrix, and then aggregate the multi-scale neighborhood information
from different $k$-NN graphs via the propagation of GCNs. Here, to define the
similarity between two samples, say, $z_{i}$ and $z_{j}$, the heat kernel [18]
is utilized, that is

$\displaystyle\mathrm{S}_{ij}=e^{-\frac{\left\|\mathrm{z}_{i}-\mathrm{z}_{j}\right\|^{2}}{t}},$
(8)

where $t$ is the time parameter in the heat kernel. In IcicleGCN, we use $t=1$
for all experiments. Further, we construct two different $k$-NN graphs to
reflect two different scales of neighborhood structures, whose numbers of
nearest neighhors are set to $k_{a}$ and $k_{b}$ (with $k_{a}\neq k_{b}$),
respectively. Let $A_{a}$ and $A_{b}$ denote the adjacency matrices of the two
constructed $k$-NN graphs. In the next section, we will utilize these two
$k$-NN graphs and the feature embedding learned by the backbone as the input
to the trident network with GCNs.

####  III-D2 Trident Network Architecture with GCNs

In the M-GCN module, three sources of input are taken, namely, the feature
embedding which is learned by the backbone and fed to the auto-encoder, and
the two scales of $k$-NN graphs which are fed to the two streams of GCNs.
Thereby, three streams of networks (including an auto-encoder and two GCNs)
are exploited in M-GCN, which is called a trident network architecture for our
multi-scale neighborhood structure learning in the proposed IcicleGCN
framework (as shown in Fig. 1).

Formally, in terms of the auto-encoder, let $L$ be the number of layers and
$\mathbf{H}^{(\ell)}$ be the learned feature embedding at the $l$-th layer of
its encoder. Thus the feedforward propagation of the encoder can be
represented as

$\displaystyle\mathbf{H}^{(\ell)}=\Phi\left(\mathbb{W}_{e}^{(\ell)}\mathbf{H}^{(\ell-1)}+\beta_{e}^{(\ell)}\right),$
(9)

where $\Phi$ is the activation function in the encoder, such as ReLU [33], and
$\mathbb{W}_{e}^{(\ell)}$ and $\beta_{e}^{(\ell)}$ are the weight matrix and
the bias of the encoder at the $\ell$-th layer, respectively. In particular,
we use the feature matrix $\mathbf{Z}^{b}$ obtained from the backbone as the
input $\mathbf{H}^{(0)}$.

Similarly, with the decoding layers corresponding to the encoding layers, each
decoding layer reconstructs the input data as follows:

$\displaystyle\mathbf{H}_{d}^{(\ell)}=\Phi\left(\mathbb{W}_{d}^{(\ell)}\mathbf{H}_{d}^{(\ell-1)}+\beta_{d}^{(\ell)}\right),$
(10)

where $\mathbb{W}_{d}^{(\ell)}$ and $\beta_{d}^{(\ell)}$ represent the weight
matrix and the bias of the decoder at the $\ell$-th layer, respectively.

For convenience, we denote the output of the last decoder layer (i.e.,
$\mathbf{H}_{d}^{(L)}$) as $\hat{\mathbf{Z}^{b}}$. Then the reconstruction
loss of the auto-encoder is represented as

$\displaystyle\mathcal{L}_{mgcn\\_re-Loss}=$
$\displaystyle\frac{1}{N}\|\mathbf{H}^{(0)}-\mathbf{H}_{d}^{(L)}\|_{F}^{2}$
$\displaystyle=$
$\displaystyle\frac{1}{N}\|\mathbf{Z}^{b}-\hat{\mathbf{Z}^{b}}\|_{F}^{2}.$
(11)

Note that the loss (11) is similar to but in fact different from the loss
(III-C2). The loss (III-C2) is further used to collaborate with the
contrastive loss in the unified loss (7) of the ISM and JC-SLIM modules, where
both augmentation views should be incorporated for contrastive learning and
thus the reconstruction loss of both augmented samples (of each input sample)
should also be considered. Since the contrastive learning is not involved in
the M-GCN module, the necessity to consider both augmentation views no longer
holds, which gives rise to the reconstruction loss (11).

Further, two streams of GCNs are incorporated in the M-GCN module to
_interact_ with the encoder of the auto-encoder. Before introducing the
interaction (in the layer-wise update and the joint self-adaptive learning),
we first describe the GCN and its learning process.

Each sample is regarded as a vertex in the graph. Let $V$ denote the set of
all the sample vertices and $A$ denote the adjacency matrix. Then the $i$-th
sample $V_{i}$ aggregates the neighborhood information as

$\displaystyle\operatorname{Aggregate}(V)$
$\displaystyle=\widetilde{D}^{-\frac{1}{2}}\widetilde{A}\widetilde{D}^{-\frac{1}{2}}V$
(12)
$\displaystyle\left(\widetilde{D}^{-\frac{1}{2}}\widetilde{A}\widetilde{D}^{-\frac{1}{2}}V\right)_{i}$
$\displaystyle=\left(\widetilde{D}^{-\frac{1}{2}}\widetilde{A}\right)_{i}\widetilde{D}^{-\frac{1}{2}}V$
$\displaystyle=\left(\sum_{k}\widetilde{D}_{ik}^{-\frac{1}{2}}\widetilde{A}_{i}\right)\widetilde{D}^{-\frac{1}{2}}V$
$\displaystyle=\widetilde{D}_{ii}^{-\frac{1}{2}}\sum_{j}\widetilde{A}_{ij}\sum_{k}\widetilde{D}_{jk}^{-\frac{1}{2}}V_{j}$
$\displaystyle=\widetilde{D}_{ii}^{-\frac{1}{2}}\sum_{j}\widetilde{A}_{ij}\widetilde{D}_{jj}^{-\frac{1}{2}}V_{j}$
$\displaystyle=\sum_{j}\frac{1}{\sqrt{\widetilde{D}_{ii}\widetilde{D}_{jj}}}\widetilde{A}_{ij}V_{j},$
(13)

where $\widetilde{A}=A+I$, $I$ is the identity diagonal matrix, and
$\widetilde{{D}}$ is the degree matrix with
$\widetilde{{D}}_{ii}=\sum_{j}\widetilde{{A}}_{{ij}}$. The purpose of
$\widetilde{A}$ is to obtain the information of the node itself in the
adjacency matrix $A$.

Then we proceed to describe the learning process of the two-stream GCNs. Let
$\mathbf{G}^{(\ell)}_{a}$ and $\mathbf{G}^{(\ell)}_{b}$ denote the
representations respectively learned by the first and the second GCNs at the
$\ell$-th layer, where the graph convolution operation can be conducted as

$\displaystyle\mathbf{G}^{(\ell)}_{u}=\Phi\left(\widetilde{{D}}^{-\frac{1}{2}}\widetilde{{A}}_{u}\widetilde{{D}}^{-\frac{1}{2}}\mathbf{G}^{(\ell-1)}_{u}\mathbb{W}^{(\ell-1)}_{u}\right),\text{~{}for~{}}u\in\\{a,b\\},$
(14)

where $\mathbb{W}^{\ell-1}_{u}$ (for $u\in\\{a,b\\})$ is the weight matrix of
the ${(\ell-1)}$-th layer of the corresponding GCN. Under the guidance of the
symmetric normalized adjacency matrix
$\widetilde{{D}}^{-\frac{1}{2}}\widetilde{{A}}_{u}\widetilde{{D}}^{-\frac{1}{2}}$,
we use the representation $\mathbf{G}^{(\ell-1)}_{u}$ of the ${(\ell-1)}$-th
layer to generate the representation $\mathbf{G}^{(\ell)}_{u}$ of the
$\ell$-th layer through the graph convolution operation (14).

Besides the two streams of GCNs, the representation learned by the $l$-th
layer of the encoder (of the auto-encoder) is denoted as
$\mathbf{H}^{(\ell)}$. Then we update each of the two-stream GCNs as follows:

$\displaystyle\widetilde{\mathbf{G}}_{a}^{(\ell-1)}=\sigma\mathbf{G}_{a}^{(\ell-1)}+\gamma\mathbf{G}_{b}^{(\ell-1)}+(1-\sigma-\gamma)\mathbf{H}^{(\ell-1)},$
(15)
$\displaystyle\widetilde{\mathbf{G}}_{b}^{(\ell-1)}=\sigma\mathbf{G}_{b}^{(\ell-1)}+\gamma\mathbf{G}_{a}^{(\ell-1)}+(1-\sigma-\gamma)\mathbf{H}^{(\ell-1)},$
(16)

where $\sigma$ and $\gamma$ are two balance coefficients. In this work, we set
$\sigma$ to 0.4 and $\gamma$ to 0.2 on all experiments. Through this layer-by-
layer connection, the representation learning processes of the encoder and the
two-stream GCNs are jointly leveraged.

Thereafter, we feed $\widetilde{\mathbf{G}}_{u}^{(\ell-1)}(u\in\\{a,b\\})$
into the $\ell$-th layer to generate the representation
$\widetilde{\mathbf{G}}_{u}^{(\ell)}$ as

$\displaystyle\widetilde{\mathbf{G}}_{u}^{(\ell)}=\Phi\left(\widetilde{{D}}^{-\frac{1}{2}}\widetilde{{A}}_{u}\widetilde{{D}}^{-\frac{1}{2}}\widetilde{\mathbf{G}}_{u}^{(\ell-1)}\mathbb{W}^{(\ell-1)}_{u}\right).$
(17)

The GCN can aggregate the neighborhood information of each node in its first
layer, which is called the first-order neighborhood information. And the
neighbors are also aggregating their own neighborhood information. Therefore,
In the second layer, when the same node aggregates its neighborhood
information again, the neighborhood information of its neighbors can be
aggregated, which is called the second-order neighborhood information.
Theoretically, with the number of layers increased to sufficiently large, a
node can aggregate the information of all nodes in the $k$-NN graph. However,
in practical applications, the number of GCN layers generally will not exceed
4 or 5. In fact, as the number of layers increases to a large number, the
aggregation of such a large amount of information will make the nodes in the
graph undistinguishable, which is called the over-smoothing and may degrade
the model performance.

In our M-GCN module, we use five layers of GCN, in which the last layer is
associated with a softmax operation. Note that the feature representation
$\mathbf{Z}^{b}$ obtained from the backbone is used as the original input of
the node features (to the first GCN layer), that is

$\displaystyle\mathbf{G}^{(1)}_{u}=\Phi\left(\widetilde{{D}}^{-\frac{1}{2}}\widetilde{{A}}_{u}\widetilde{{D}}^{-\frac{1}{2}}\mathbf{Z}^{b}\mathbb{W}^{(0)}_{u}\right),$
(18)

The last layer of the GCN is associated with a softmax operation, that is

$\displaystyle\mathbf{G}^{s}_{u}=\operatorname{softmax}\left(\mathbf{G}_{u}^{(L)}\right).$
(19)

Thus, with the dimension of the softmax layer set to the number of clusters
$K$, we obtain two probability matrices $\mathbf{G}^{s}_{u}$ (with
$u\in\\{a,b\\}$) for the cluster assignments, where the $(i,j)$-th entry of
$\mathbf{G}^{s}_{u}$ represents the probability that the sample $i$ should be
assigned to cluster $j$. Therefore, $\mathbf{G}^{s}_{a}$ and
$\mathbf{G}^{s}_{b}$ can be regarded as two probability distributions, which
will be exploited in the joint self-adaptive learning process.

####  III-D3 Joint Self-Adaptive Learning

With the auto-encoder and the two streams of GCNs connected via the layer-wise
updating, the consistency of their final-layer representations will be our
next focus. In this section, we design a joint self-adaptive learning
mechanism to simultaneously guide the learning of the three streams of
networks in M-GCN.

First, we use the Student’s t-distribution [34] to convert the representations
obtained by the auto-encoder into a probability distribution. That is, the
probability that the $i$-th sample is assigned to the $j$-th cluster can be
obtained as follows:

$\displaystyle
q_{ij}=\frac{\left(1+\left\|\mathbf{h}_{i}-\boldsymbol{\mu}_{j}\right\|^{2}/t\right)^{-\frac{t+1}{2}}}{\sum_{j^{\prime}}\left(1+\left\|\mathbf{h}_{i}-\boldsymbol{\mu}_{j^{\prime}}\right\|^{2}/t\right)^{-\frac{t+1}{2}}},$
(20)

where $t$ is the degree of freedom of the Student’s t-distribution in the
probability distribution function, and $\boldsymbol{\mu}_{j}$ is the $j$-th
cluster center initialized by $K$-means through the pre-trained
representations of the auto-encoder. In the encoder part, we obtain the data
representation matrix $\mathbf{H}^{L}$, where the $i$-th row $\mathbf{h}_{i}$
is the representation of the $i$-th sample. Here, the probability of assigning
sample $i$ to cluster $j$ (i.e., $q_{ij}$) is obtained by computing the
similarity between $\mathbf{h}_{i}$ and $\boldsymbol{\mu}_{j}$. By considering
the probability distribution that the samples assigned to different clusters,
we can denote the probability distribution matrix as $Q$, where $q_{ij}$ is
its $(i,j)$-th entry.

To make the representations obtained from the auto-encoder closer to the
center of the corresponding cluster, we proceed to compute a target
distribution $P$ from $Q$, whose $(i,j)$-th entry $p_{ij}$ is defined as

$\displaystyle p_{ij}=$
$\displaystyle\frac{q_{ij}^{2}/f_{j}}{\sum_{j^{\prime}}q_{ij^{\prime}}^{2}/f_{j^{\prime}}},$
(21)

with

$\displaystyle f_{j}=$ $\displaystyle\sum_{i}q_{ij}.$ (22)

Since $q_{ij}$ is a soft assignment probability, $f_{j}$ can thus be regarded
as the soft cluster assignment frequency. In turn, we can use the target
distribution $P$ to supervise the learning of the distribution $Q$:

$\displaystyle\mathcal{L}_{cluster-
Loss}=KL(P\|Q)=\sum_{i}\sum_{j}p_{ij}\log\frac{p_{ij}}{q_{ij}}.$ (23)

By this formulation, we then perform self-adaptive learning of the $Q$
distribution by minimizing the KL-divergence between the $P$ and $Q$
distributions.

Further, by regarding the representations $\mathbf{G}^{s}_{a}$ and
$\mathbf{G}^{s}_{b}$ respectively learned by the two GCNs as probability
distributions, we can also use the probability distribution $P$ to supervise
the update and learning of the two-stream GCNs by minimizing their KL-
divergence losses as follows:

$\displaystyle\mathcal{L}_{mgcn_{-}a}=KL(P\|\mathbf{G}^{s}_{a})=\sum_{i}\sum_{j}p_{ij}\log\frac{p_{ij}}{g^{s}_{a,ij}},$
(24)

$\displaystyle\mathcal{L}_{mgcn_{-}b}=KL(P\|\mathbf{G}^{s}_{b})=\sum_{i}\sum_{j}p_{ij}\log\frac{p_{ij}}{g^{s}_{b,ij}},$
(25)

where $g^{s}_{a,ij}$ and $g^{s}_{b,ij}$ denote the $(i,j)$-th entries of
$\mathbf{G}^{s}_{a}$ and $\mathbf{G}^{s}_{b}$, respectively. For the joint
self-adaptive learning of the auto-encoder and the two-stream GCNs, we define
the overall loss function $\mathcal{L}_{2}$ of the M-GCN module as

$\displaystyle\mathcal{L}_{2}=$ $\displaystyle\mathcal{L}_{mgcn\\_re-
Loss}+\alpha\mathcal{L}_{cluster-Loss}$
$\displaystyle+\beta\mathcal{L}_{mgcn_{-}a}+\eta\mathcal{L}_{mgcn_{-}b},$ (26)

where $\alpha$, $\beta$, and $\eta$ are hyper-parameters to balance the
influences of different terms. Through this formulation, the information of
the data itself and the multi-scale neighborhood structure information can be
adaptively aggregated, and the final clustering can be obtained from the
probability distribution learned by the GCNs.

Input: Image dataset $\mathbf{Y}$, the training epochs ${E}$, the batch size
${N}$, the number of iterations in M-GCN ${N}_{it}$, the temperature
parameters $\tau_{I}$ and $\tau_{C}$, and the number of clusters $K$.

Output: The clustering result with $K$ clusters.

1 for  _each epoch_ do

2 $\mathbf{Step1:}$ Randomly select a mini-batch of ${N}$ images from the
image dataset;

3 $\mathbf{Step2:}$ Perform two random data augmentations on each image in the
mini-batch;

4 $\mathbf{Step3:}$ Calculate the contrastive instance similarity loss by Eq.
(3);

5 $\mathbf{Step4:}$ Calculate the contrastive cluster structure loss by Eq.
(5);

6 $\mathbf{Step5:}$ Calculate the instance reconstruction loss by Eq.
(III-C2);

7 $\mathbf{Step6:}$ Jointly update the backbone, the ISM, and the JC-SLIM by
minimizing $\mathcal{L}_{1}$ in Eq. (7);

8

9 end for

10for  _each iteration in ${N}_{it}$_ do

11 $\mathbf{Step7:}$ Calculate the probability distribution of the two-stream
GCN networks by Eq. (19);

12 $\mathbf{Step8:}$ Calculate the $Q$ distribution by Eq. (20);

13 $\mathbf{Step9:}$ Calculate the $P$ distribution by Eq. (21);

14 $\mathbf{Step10:}$ Update the GCNs and the auto-encoder by minimizing
$\mathcal{L}_{2}$ in Eq. (III-D3) ;

15

16 end for

17Obtain the final clustering from the probability distribution learned by the
M-GCN module;

Algorithm 1 Training algorithm for IcicleGCN.

###  III-E Implementation Details

Our IcicleGCN approach consists of four modules, including the backbone, the
ISM module, the JC-SLIM module, and the M-GCN module. Specifically, we adopt
the ResNet-34 as the backbone network, and resize the images in the dataset to
the size of $224\times 224\times 3$. Note that there is an overlapping (or
sharing) component between the JC-SLIM module and the M-GCN module, that is,
the auto-encoder, which is pretrained in JC-SLIM and further participates in
the updating process of the multi-scale GCNs. As for the network training, we
first use the loss function $\mathcal{L}_{1}$ to train the backbone, the ISM,
and the JC-SLIM, and then use the loss function $\mathcal{L}_{2}$ to train the
M-GCN. The dimension of the auto-encoder is set to input-500-500-2000-$K$,
where $K$ is the number of clusters. The batch size is set to 128. We use the
Adam optimizer with a learning rate of 1e-4. The parameters $\sigma$ and
$\gamma$ are set to 0.4 and 0.2, respectively. The parameters $\alpha$,
$\beta$ and $\eta$ in the loss function $\mathcal{L}_{2}$ are tuned in the
range of $\\{$0.01, 0.05, 0.1$\\}$. To construct the two different $k$-NN
graphs, the numbers of nearest neighbors are set of 1 and 10, respectively.
That is, a $1$-NN graph and a $10$-NN graph are initialized for M-GCN. All
experiments are carried out on a machine with an Nvidia GeForce RTX 3090 GPU
and a CPU with 12 cores and 2.6GHz.

(a) CIFAR-10 (b) CIFAR-100 (c) ImageNet-10 (d) ImageNet-Dogs

Figure 2: Some examples of the four image datasets.

##  IV Experiments

In this section, we experimentally compare the proposed IcicleGCN algorithm
against several deep and non-deep clustering algorithms on multiple image
datasets.

###  IV-A Datasets and Evaluation Metrics

We conduct experiments on four image datasets. Some sample images of these
datasets are shown in Fig. 2. The details of the four datasets are given
below.

  * •

CIFAR-10 [35]: The CIFAR-10 dataset consists of 60,000 color images of size
32$\times$32$\times$3\. These images are divided into 10 classes, including
airplanes, cars, birds, cats, trucks, etc.

  * •

CIFAR-100 [35]: The CIFAR-100 datasets consists of 60,000 color images of size
32$\times$32$\times$3\. Unlike CIFAR-10, each image in CIFAR-100 has an
“elaborate” class label (corresponding to a total of 100 classes) and a
“coarse” class label (corresponding to a total of 20 super-classes). In this
paper, the 20 super-classes are taken as the ground-truth.

  * •

ImageNet-10 [3]: The ImageNet-10 dataset is a subset of the ImageNet dataset,
containing a total of 13,000 color images with 10 classes. The size of each
image is 96$\times$96$\times$3.

  * •

ImageNet-Dogs [3]: The ImageNet-Dogs dataset is also a subset of the ImageNet
dataset, containing 19,500 images with 15 dog categories. The size of each
image is 96$\times$96$\times$3.

To quantitatively compare the clustering results by different clustering
methods, we adopt three widely-used evaluation metrics, namely, Accuracy (ACC)
[36], Normalized Mutual Information (NMI) [36], and Adjusted Rand Index (ARI)
[37]. Notice that larger values of the three metrics indicate better
clustering performance.

###  IV-B Comparisons with Other Clustering Methods

TABLE I: The clustering performance w.r.t. ACC(%) by different clustering methods on the benchmark datasets Dataset | CIFAR-10 | CIFAR-100 | ImageNet-10 | ImageNet-Dogs
---|---|---|---|---
K-means | 22.9 | 13.0 | 24.1 | 10.5
SC | 24.7 | 13.6 | 27.4 | 11.1
AC | 22.8 | 13.8 | 24.2 | 13.9
NMF | 19.0 | 11.8 | 23.0 | 11.8
AE | 31.4 | 16.5 | 31.7 | 18.5
DAE | 29.7 | 15.1 | 30.4 | 19.0
DCGAN | 31.5 | 15.3 | 34.6 | 17.4
DeCNN | 28.2 | 13.3 | 31.3 | 17.5
VAE | 29.1 | 15.2 | 33.4 | 17.9
JULE | 27.2 | 13.7 | 30.0 | 13.8
DEC | 30.1 | 18.5 | 38.1 | 19.5
DAC | 52.2 | 23.8 | 52.7 | 27.5
DDC | 52.4 | - | 57.7 | -
DCCM | 62.3 | 32.7 | 71.0 | 38.3
IIC | 61.7 | 25.7 | - | -
GATCluster | 62.3 | 32.7 | 73.9 | 32.2
PICA | 69.6 | 33.7 | 87.0 | 35.2
DRC | 72.7 | 36.7 | 88.4 | 38.9
CC | 79.0 | 42.9 | 89.5 | 34.2
IcicleGCN | 80.7 | 46.1 | 95.5 | 41.5

TABLE II: The clustering performance w.r.t. NMI(%) by different clustering methods on the benchmark datasets Dataset | CIFAR-10 | CIFAR-100 | ImageNet-10 | ImageNet-Dogs
---|---|---|---|---
K-means | 8.7 | 8.4 | 11.9 | 5.5
SC | 10.3 | 9.0 | 15.1 | 3.8
AC | 10.5 | 9.8 | 13.8 | 3.7
NMF | 8.1 | 7.9 | 13.2 | 4.4
AE | 23.9 | 10.0 | 21.0 | 10.4
DAE | 25.1 | 11.1 | 20.6 | 10.4
DCGAN | 26.5 | 12.0 | 22.5 | 12.1
DeCNN | 24.0 | 9.2 | 18.6 | 9.8
VAE | 24.5 | 10.8 | 19.3 | 10.7
JULE | 19.2 | 10.3 | 17.5 | 5.4
DEC | 25.7 | 13.6 | 28.2 | 12.2
DAC | 39.6 | 18.5 | 39.4 | 21.9
DDC | 42.4 | - | 43.3 | -
DCCM | 49.6 | 28.5 | 60.8 | 32.1
IIC | 51.1 | 22.5 | - | -
GATCluster | 49.6 | 28.5 | 59.4 | 28.1
PICA | 59.1 | 31.0 | 80.2 | 35.2
DRC | 62.1 | 35.6 | 83.0 | 38.4
CC | 70.5 | 43.1 | 86.2 | 40.1
IcicleGCN | 72.9 | 45.9 | 90.4 | 45.8

In this section, we compare our IcicleGCN method with nineteen deep and non-
deep clustering methods, including $K$-means[38], Spectral Clustering (SC)
[39], Agglomerative Clustering (AC) [40], Nonnegative Matrix Factorization
(NMF) [12], Auto-Encoder (AE) [41], Denoising Auto-Encoder (DAE) [42], Deep
Convolution Generative Adversarial Network (DCGAN) [43], Deconvolutional
Network (DeCNN) [44], Variational Auto-Encoder (VAE) [45], Jointly
Unsupervised Learning (JULE) [5], Deep Embedding Clustering (DEC) [4], Deep
Adaptive image Clustering (DAC) [3], Deep Discriminative Clustering (DDC)
[46], Deep Comprehensive Correlation Mining (DCCM) [6], Invariant Information
Clustering (IIC) [47], self-supervised Gaussian-ATtention network for image
Clustering (GATCluster) [48], PartItion Confidence mAximisation (PICA) [8],
Deep Robust Clustering (DRC) [49], and Contrastive Clustering (CC) [14]. The
clustering performances of different clustering methods w.r.t. ACC, NMI, and
ARI are reported in Tables I, II, and III, respectively.

TABLE III: The clustering performance w.r.t. ARI(%) by different clustering methods on the benchmark datasets Dataset | CIFAR-10 | CIFAR-100 | ImageNet-10 | ImageNet-Dogs
---|---|---|---|---
K-means | 4.9 | 2.8 | 5.7 | 2.0
SC | 8.5 | 2.2 | 7.6 | 1.3
AC | 6.5 | 3.4 | 6.7 | 2.1
NMF | 3.4 | 2.6 | 6.5 | 1.6
AE | 16.9 | 4.8 | 15.2 | 7.3
DAE | 16.3 | 4.6 | 13.8 | 7.8
DCGAN | 17.6 | 4.5 | 15.7 | 7.8
DeCNN | 17.4 | 3.8 | 14.2 | 7.3
VAE | 16.7 | 4.0 | 16.8 | 7.9
JULE | 13.8 | 3.3 | 13.8 | 2.8
DEC | 16.1 | 5.0 | 20.3 | 7.9
DAC | 30.6 | 8.8 | 30.2 | 11.1
DDC | 32.9 | - | 34.5 | -
DCCM | 40.8 | 17.3 | 55.5 | 18.2
IIC | 41.1 | 11.7 | - | -
GATCluster | 40.8 | 17.3 | 55.2 | 16.3
PICA | 51.2 | 17.1 | 76.1 | 20.1
DRC | 54.7 | 20.8 | 79.8 | 23.3
CC | 63.7 | 26.6 | 82.5 | 22.5
IcicleGCN | 66.0 | 31.1 | 90.5 | 27.9

In terms of ACC, as shown in Table II, our IcicleGCN method outperforms or
significantly outperforms the baseline clustering methods on all the four
benchmark datasets. Especially, on the CIFAR-100, ImageNet-10, and ImageNet-
Dogs datasets, our IcicleGCN method achieves ACC(%) scores of 46.1, 95.5, and
41.5, respectively, while the best baseline method (i.e., CC) only obtains
scores of 42.9, 89.5, and 34.2, respectively. In terms of NMI and ARI, similar
advantages of IcicleGCN can also be observed . As shown in Table II, on the
four benchmark datasets, our IcicleGCN method achieves NMI(%) scores of 72.9,
45.9, 90.4, and 45.8, respectively, while the best baseline method only
obtains scores of 70.5, 43.1, 86.2, and 40.1. As shown in Table III, IcicleGCN
achieves ARI(%) scores of 66.0, 31.1, 90.5, and 27.9 on the four datasets,
while the best baseline method only obtains scores of 63.7, 26.6, 82.5, and
22.5, respectively. The experimental results in Tables I, II, and III confirm
the superior clustering performance of the proposed IcicleGCN method over the
baseline deep and non-deep clustering methods on the benchmark datasets.

TABLE IV: Influence of three losses in the ISM module and the JC-SLIM module Loss Function | ImageNet-Dogs
---|---
ACC | NMI | ARI
$\mathcal{L}_{re-Loss}$ \+ $\mathcal{L}_{CIS-Loss}$ \+ $\mathcal{L}_{CCS-Loss}$ | 41.5 | 45.8 | 27.9
$\mathcal{L}_{re-Loss}$ \+ $\mathcal{L}_{CIS-Loss}$ | 25.8 | 23.2 | 12.5
$\mathcal{L}_{re-Loss}$ \+ $\mathcal{L}_{CCS-Loss}$ | 9.2 | 1.0 | 0.3
$\mathcal{L}_{re-Loss}$ | 6.9 | 0.4 | 0.0
TABLE V: Influence of the GCNs and the auto-encoder (AE) in the M-GCN module Dataset | Ablation of Components | ACC | NMI | ARI
---|---|---|---|---
|  with GCNs and AE | 41.5 | 45.8 | 27.9
ImageNet-Dogs |  without GCNs | 39.3 | 41.4 | 24.5
|  without GCNs and AE | 34.2 | 40.1 | 22.5

###  IV-C Ablation Study

In this section, we conduct ablation analysis to test the influences of
different modules (and the components inside each module). We first test the
clustering performance of IcicleGCN with different components in ISM and JC-
SLIM or even the whole module removed in Section IV-C1. Then we test the
clustering performance with the components in M-GCN removed in Section IV-C2.
Further, we test the influence of the multi-scale GCNs and the different
neighborhood combinations in Sections IV-C3 and IV-C4, respectively.

####  IV-C1 Influences of the Components in ISM and JC-SLIM

This section conducts ablation analysis on the components in ISM and JC-SLIM.
Specifically, three losses are incorporated in the ISM and JC-SLIM modules,
namely, the contrastive instance similarity loss $\mathcal{L}_{CIS-Loss}$, the
contrastive cluster structure loss $\mathcal{L}_{CCS-Loss}$, and the instance
reconstruction loss (of the auto-encoder) $\mathcal{L}_{re-Loss}$, which
jointly contribute to the self-supervised training of the CNN in our IcicleGCN
framework. Notice that the auto-encoder serves as a bridge between the JC-SLIM
module and the M-GCN module. Therefore, the reconstruction loss
$\mathcal{L}_{re-Loss}$ will be preserved to keep this connection to M-GCN,
while the other two losses will be removed and tested in this section. In
fact, when the two contrastive losses are removed, the effects of the ISM
module and the JC-SLIM module almost disappear, except that the auto-encoder
still remains for the sake of M-GCN. As shown in Table IV, both the
contrastive instance similarity loss and the contrastive cluster structure
loss play a substantial role in IcicleGCN. Especially, the joint incorporation
of the three losses leads to the best clustering performance (w.r.t. ACC, NMI,
and ARI) when compared to the variants with one or two components removed.

####  IV-C2 Influence of the Components in M-GCN

There are three streams of networks in the M-GCN module, including an auto-
encoder and two streams of GCNs. In this section, we test the influence of the
GCNs and the auto-encoder. Note that when the auto-encoder is removed, we will
put back a nonlinear multi-layer MLP instead, so as to make the JC-SLIM module
still functionable. As shown in Table V, removing the GCNs degrades the
ACC(%), NMI(%), and ARI(%) scores from 41.5, 45.8, and 27.9 to 39.3, 41.4, and
24.5, respectively, while removing both the GCNs and the auto-encoder further
degrades the ACC(%), NMI(%), and ARI(%) scores to 34.2, 40.1, and 22.5,
respectively, which demonstrate the contributions of the components in the
M-GCN module. In the following, we will further test the influence of using
different settings of the GCNs.

####  IV-C3 Multi-scale GCNs VS Single GCN

TABLE VI: The clustering performance of IcicleGCN using a single GCN and using multi-scale GCNs (with different neighborhood combinations) Neighborhood Combinations in M-GCN | ImageNet-Dogs
---|---
ACC | NMI | ARI
_1_ -NN | 40.1 | 43.5 | 25.5
_3_ -NN | 39.8 | 43.4 | 25.6
_5_ -NN | 39.7 | 43.4 | 24.8
_10_ -NN | 39.6 | 43.5 | 25.3
_1_ -NN+_10_ -NN | 41.5 | 45.8 | 27.9
_3_ -NN+_10_ -NN | 41.1 | 45.3 | 26.0
_3_ -NN+_5_ -NN | 41.4 | 45.3 | 27.0

(a) CIFAR-10

(b) CIFAR-100

(c) ImageNet-10

(d) ImageNet-Dogs

Figure 3: Confusion matrices for the clustering results of IcicleGCN on the
four image datasets.
Figure 4: Case study on ImageNet-Dogs. The left sub-figure shows the
successful cases, the middle sub-figure shows the false negative cases, and
the right sub-figure shows the false positive cases.

In this paper, our IcicleGCN method captures the multi-scale neighborhood
information via multiple streams of GCNs, for which multiple $k$-NN graphs
with different neighborhood sizes are constructed for the initial input. In
this work, two streams of GCN are incorporated. This section tests the
performance of IcicleGCN using the two-stream GCNs against using a single GCN.
As shown in Table VI, the incorporation of two scales of $k$-NN graphs for
M-GCN yields consistently better clustering performance w.r.t. ACC, NMI, and
ARI than using the GCN with a single-scale $k$-NN graph.

####  IV-C4 Influence of Different Neighborhood Combinations

This section tests the influence of using different combinations of $k$-NN
graphs in the M-GCN module. In M-GCN, two streams of GCNs are utilized, each
of which requires a $k$-NN graph as the input graph. As shown in Table VI,
using two different scales of $k$-NN graphs is often beneficial for the
clustering performance. Typically, using the combination of a $1$-NN graph and
a $10$-NN graph leads to the optimal clustering performance, probably due to
the complementariness between the nearest neighborhood and a relative larger
neighborhood. Throughout this paper, we use the combination of a $1$-NN graph
and a $10$-NN graph for our multi-scale neighborhood structure learning via
GCNs.

###  IV-D Visualization of Confusion Matrices

In Fig. 3, we visualize the confusion matrices of the clustering results by
IcicleGCN on the four datasets. The confusion matrices for CIFAR-10 and
ImageNet-10 have a clear block diagonal structure that implies our IcicleGCN
method successfully partitions most of the images into semantic clusters. For
CIFAR-100 and ImageNet-Dogs, the block diagonal structure can still be
observed, though it is not as clear as the other two confusion matrices,
possibly due to the fact that (i) the images in CIFAR-100 are mostly small and
blurry and that (ii) the categories of dogs in ImageNet-Dogs are sometimes
difficult to be distinguished even for humans (as further illustrated in Fig.
4). However, even on the challenging datasets of CIFAR-100 and ImageNet-Dogs,
the proposed IcicleGCN method still exhibits a substantial advantage over the
state-of-the-art deep clustering methods (as shown in Tables I, II, and III).

###  IV-E Case Study

To gain a deeper understanding of how our IcicleGCN method performs, we
investigate the success and failure cases on the ImageNet-Dogs dataset.
Specifically, we study three situations of four dog categories on ImageNet-
Dogs, namely, (i) the successful cases that correspond to the correctly
clustered samples, (ii) the false negative failure cases that correspond to
the samples which belong to this cluster but are incorrectly assigned to other
clusters, and (iii) the false positive cases that correspond to the samples
which do not belong to this cluster but are incorrectly assigned to this
cluster. As shown in Fig. 4, IcicleGCN successfully assigns many images of the
same category to the same cluster. However, when there are multiple objects of
different categories in the images, some of them may be assigned to the
incorrect clusters (as shown in the middle sub-figure of Fig. 4). Furthermore,
some categories of dogs look very similar to each other, which may also
contribute to the incorrect cluster assignments (as shown in the right sub-
figure of Fig. 4). How to distinguish multiple objects in the same image and
how to distinguish different fine-grained classes with similar appearance is
still a very challenging problem for unsupervised learning and clustering.

##  V Conclusion

In this paper, we propose a novel deep clustering approach termed IcicleGCN,
which bridges the gap between CNN and GCN as well as the gap between
contrastive learning and multi-scale neighborhood structure learning for the
deep image clustering task. The proposed IcicleGCN approach consists of four
main modules, i.e., the backbone, the ISM, the JC-SLIM, and the M-GCN. With
two augmented samples generated for each input image, the backbone with two
weight-sharing views is utilized to extract their feature representations,
which are then fed to the following three modules. In ISM and JC-SLIM, three
types of losses are designed and jointly optimized, namely, the contrastive
instance similarity loss, the contrastive cluster similarity loss, and the
instance reconstruction loss (via an auto-encoder). Notice that the auto-
encoder serves as a bridge between the JC-SLIM module and the M-GCN module. In
M-GCN, two streams of GCNs and the auto-encoder are iteratively and
simultaneously updated with layer-wise representation fusion, upon which a
joint self-adaptive learning mechanism is further incorporated to ensure the
consistency of their last-layer clustering distributions. Experiments are
conducted on four challenging image datasets, which have shown the
advantageous clustering performance of IcicleGCN over the state-of-the-art
deep clustering approaches.

## References

  * [1] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , 2016, pp. 770–778.
  * [2] B. Yang, X. Fu, N. D. Sidiropoulos, and M. Hong, “Towards k-means-friendly spaces: Simultaneous deep learning and clustering,” in _International Conference on Machine Learning_. PMLR, 2017, pp. 3861–3870.
  * [3] J. Chang, L. Wang, G. Meng, S. Xiang, and C. Pan, “Deep adaptive image clustering,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2017, pp. 5879–5887.
  * [4] J. Xie, R. Girshick, and A. Farhadi, “Unsupervised deep embedding for clustering analysis,” in _International conference on Machine Learning_. PMLR, 2016, pp. 478–487.
  * [5] J. Yang, D. Parikh, and D. Batra, “Joint unsupervised learning of deep representations and image clusters,” in _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , 2016, pp. 5147–5156.
  * [6] J. Wu, K. Long, F. Wang, C. Qian, C. Li, Z. Lin, and H. Zha, “Deep comprehensive correlation mining for image clustering,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2019, pp. 8150–8159.
  * [7] Y. Yan, H. Hao, B. Xu, J. Zhao, and F. Shen, “Image clustering via deep embedded dimensionality reduction and probability-based triplet loss,” _IEEE Transactions on Image Processing_ , vol. 29, pp. 5652–5661, 2020.
  * [8] J. Huang, S. Gong, and X. Zhu, “Deep semantic clustering by partition confidence maximisation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 8849–8858.
  * [9] J. Lv, Z. Kang, X. Lu, and Z. Xu, “Pseudo-supervised deep subspace clustering,” _IEEE Transactions on Image Processing_ , vol. 30, pp. 5252–5263, 2021.
  * [10] L. Zelnik-Manor and P. Perona, “Self-tuning spectral clustering,” in _Advances in Neural Information Processing Systems (NIPS)_ , 2004.
  * [11] K. C. Gowda and G. Krishna, “Agglomerative clustering using the concept of mutual nearest neighbourhood,” _Systems Man & Cybernetics IEEE Transactions on_, vol. 10, no. 2, pp. 105–112, 1978.
  * [12] D. Cai, X. He, X. Wang, H. Bao, and J. Han, “Locality preserving nonnegative matrix factorization,” in _International Jont Conference on Artifical Intelligence_ , 2009.
  * [13] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in _Proc. of International Conference on Machine Learning (ICML)_. PMLR, 2020, pp. 1597–1607.
  * [14] Y. Li, P. Hu, Z. Liu, D. Peng, J. T. Zhou, and X. Peng, “Contrastive clustering,” in _2021 AAAI Conference on Artificial Intelligence (AAAI)_ , 2021.
  * [15] W. Van Gansbeke, S. Vandenhende, S. Georgoulis, M. Proesmans, and L. Van Gool, “Scan: Learning to classify images without labels,” in _European Conference on Computer Vision_. Springer, 2020, pp. 268–285.
  * [16] H. Zhong, J. Wu, C. Chen, J. Huang, M. Deng, L. Nie, Z. Lin, and X.-S. Hua, “Graph contrastive clustering,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 9224–9233.
  * [17] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” _arXiv preprint arXiv:1609.02907_ , 2016.
  * [18] D. Bo, X. Wang, C. Shi, M. Zhu, E. Lu, and P. Cui, “Structural deep clustering network,” in _Proceedings of The Web Conference 2020_ , 2020, pp. 1400–1410.
  * [19] X. Peng, Z. Yi, and H. Tang, “Robust subspace clustering via thresholding ridge regression,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 29, no. 1, 2015.
  * [20] P. Huang, Y. Huang, W. Wang, and L. Wang, “Deep embedding network for clustering,” in _2014 22nd International Conference on Pattern Recognition_. IEEE, 2014, pp. 1532–1537.
  * [21] Z. Wu, Y. Xiong, S. X. Yu, and D. Lin, “Unsupervised feature learning via non-parametric instance discrimination,” in _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , 2018, pp. 3733–3742.
  * [22] C. Zhuang, A. L. Zhai, and D. Yamins, “Local aggregation for unsupervised learning of visual embeddings,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 6002–6012.
  * [23] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in _Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition_ , 2020, pp. 9729–9738.
  * [24] M. Gori, G. Monfardini, and F. Scarselli, “A new model for learning in graph domains,” in _Proceedings. 2005 IEEE International Joint Conference on Neural Networks_ , vol. 2, no. 2005, 2005, pp. 729–734.
  * [25] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” _IEEE Transactions on Neural Networks_ , vol. 20, no. 1, pp. 61–80, 2008.
  * [26] J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun, “Spectral networks and locally connected networks on graphs,” _arXiv preprint arXiv:1312.6203_ , 2013.
  * [27] M. Henaff, J. Bruna, and Y. LeCun, “Deep convolutional networks on graph-structured data,” _arXiv preprint arXiv:1506.05163_ , 2015.
  * [28] M. Defferrard, X. Bresson, and P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” _Advances in Neural Information Processing Systems_ , vol. 29, 2016.
  * [29] F. Monti, D. Boscaini, J. Masci, E. Rodola, J. Svoboda, and M. M. Bronstein, “Geometric deep learning on graphs and manifolds using mixture model cnns,” in _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , 2017, pp. 5115–5124.
  * [30] M. Niepert, M. Ahmed, and K. Kutzkov, “Learning convolutional neural networks for graphs,” in _International Conference on Machine Learning_. PMLR, 2016, pp. 2014–2023.
  * [31] H. Gao, Z. Wang, and S. Ji, “Large-scale learnable graph convolutional networks,” in _Proceedings of the 24th ACM SIGKDD international conference on Knowledge Discovery & Data mining_, 2018, pp. 1416–1424.
  * [32] X. Shi, X. Chai, J. Xie, and T. Sun, “Mc-gcn: A multi-scale contrastive graph convolutional network for unconstrained face recognition with image sets,” _IEEE Transactions on Image Processing_ , vol. 31, pp. 3046–3055, 2022.
  * [33] V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines vinod nair,” in _International Conference on International Conference on Machine Learning_ , 2010.
  * [34] L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.” _Journal of Machine Learning Research_ , vol. 9, no. 11, 2008.
  * [35] A. Krizhevsky, G. Hinton _et al._ , “Learning multiple layers of features from tiny images,” 2009.
  * [36] D. Huang, C.-D. Wang, J.-S. Wu, J.-H. Lai, and C.-K. Kwoh, “Ultra-scalable spectral clustering and ensemble clustering,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 32, no. 6, pp. 1212–1226, 2020.
  * [37] D. Huang, C.-D. Wang, J.-H. Lai, and C.-K. Kwoh, “Toward multidiversified ensemble clustering of high-dimensional data: From subspaces to metrics and beyond,” _IEEE Transactions on Cybernetics, in press_ , 2021.
  * [38] J. MacQueen _et al._ , “Some methods for classification and analysis of multivariate observations,” in _Proceedings of the fifth Berkeley Symposium on Mathematical Statistics and Probability_ , vol. 1, no. 14. Oakland, CA, USA, 1967, pp. 281–297.
  * [39] L. Zelnik-Manor and P. Perona, “Self-tuning spectral clustering,” _Advances in Neural Information Processing Systems_ , vol. 17, 2004.
  * [40] K. C. Gowda and G. Krishna, “Agglomerative clustering using the concept of mutual nearest neighbourhood,” _Pattern Recognition_ , vol. 10, no. 2, pp. 105–112, 1978.
  * [41] Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, “Greedy layer-wise training of deep networks,” _Advances in Neural Information Processing Systems_ , vol. 19, 2006.
  * [42] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, and L. Bottou, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion.” _Journal of Machine Learning Research_ , vol. 11, no. 12, 2010.
  * [43] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” _arXiv preprint arXiv:1511.06434_ , 2015.
  * [44] M. D. Zeiler, D. Krishnan, G. W. Taylor, and R. Fergus, “Deconvolutional networks,” in _2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition_. IEEE, 2010, pp. 2528–2535.
  * [45] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” _arXiv preprint arXiv:1312.6114_ , 2013.
  * [46] J. Chang, Y. Guo, L. Wang, G. Meng, S. Xiang, and C. Pan, “Deep discriminative clustering analysis,” _arXiv preprint arXiv:1905.01681_ , 2019.
  * [47] X. Ji, J. F. Henriques, and A. Vedaldi, “Invariant information clustering for unsupervised image classification and segmentation,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2019, pp. 9865–9874.
  * [48] C. Niu, J. Zhang, G. Wang, and J. Liang, “Gatcluster: Self-supervised gaussian-attention network for image clustering,” in _European Conference on Computer Vision_. Springer, 2020, pp. 735–751.
  * [49] H. Zhong, C. Chen, Z. Jin, and X.-S. Hua, “Deep robust clustering by contrastive learning,” _arXiv preprint arXiv:2008.03030_ , 2020.

|  Yuankun Xu received the B.Eng. degree in software engineering from Jilin
University, ChangChun, China, in 2020. He is currently pursuing the master
degree in computer science with the College of Mathematics and Informatics,
South China Agricultural University, Guangzhou, China. His research interests
include deep clustering and graph neural network.
---|---
|  Dong Huang received the B.S. degree in computer science in 2009 from South
China University of Technology, Guangzhou, China. He received the M.Sc. degree
in computer science in 2011 and the Ph.D. degree in computer science in 2015,
both from Sun Yat-sen University, Guangzhou, China. He joined South China
Agricultural University in 2015, where he is currently an Associate Professor
with the College of Mathematics and Informatics. From July 2017 to July 2018,
he was a visiting fellow with the School of Computer Science and Engineering,
Nanyang Technological University, Singapore. His research interests include
data mining and machine learning. He has published more than 50 papers in
international journals and conferences, such as IEEE TKDE, IEEE TCYB, IEEE
TSMC-S, ACM TKDD, SIGKDD, AAAI, and ICDM. He was the recipient of the 2020 ACM
Guangzhou Rising Star Award.
---|---
|  Chang-Dong Wang received the B.S. degree in applied mathematics in 2008,
the M.Sc. degree in computer science in 2010, and the Ph.D. degree in computer
science in 2013, all from Sun Yat-sen University, Guangzhou, China. He was a
visiting student at the University of Illinois at Chicago from January 2012 to
November 2012. He is currently an Associate Professor with the School of Data
and Computer Science, Sun Yat-sen University, Guangzhou, China. His current
research interests include machine learning and data mining. He has published
more than 100 scientific papers in international journals and conferences such
as IEEE TPAMI, IEEE TKDE, IEEE TNNLS, IEEE TSMC-C, ACM TKDD, Pattern
Recognition, SIGKDD, ICDM and SDM. His ICDM 2010 paper won the Honorable
Mention for Best Research Paper Award. He was awarded 2015 Chinese Association
for Artificial Intelligence (CAAI) Outstanding Dissertation.
---|---
|  Jian-Huang Lai received the M.Sc. degree in applied mathematics in 1989 and
the Ph.D. degree in mathematics in 1999 from Sun Yat-sen University, China. He
joined Sun Yat-sen University in 1989 as an Assistant Professor, where he is
currently a Professor with the School of Data and Computer Science. His
current research interests include the areas of digital image processing,
pattern recognition, multimedia communication, wavelet and its applications.
He has published more than 200 scientific papers in the international journals
and conferences on image processing and pattern recognition, such as IEEE
TPAMI, IEEE TKDE, IEEE TNN, IEEE TIP, IEEE TSMC-B, Pattern Recognition, ICCV,
CVPR, IJCAI, ICDM and SDM. Prof. Lai serves as a Standing Member of the Image
and Graphics Association of China, and also serves as a Standing Director of
the Image and Graphics Association of Guangdong.
---|---