11institutetext: Nepal Applied Mathematics and Informatics Institute for
research(NAAMII), Lalitpur, Nepal 22institutetext: Fogsphere(Redev AI Ltd.),
64 Southwark Bridge Rd, SE1 0AS, London, UK 33institutetext: West Virginia
University, USA44institutetext: School of Natural and Computing Sciences,
University of Aberdeen, Aberdeen, UK
44email<EMAIL_ADDRESS>
# TTA-OOD: Test-time Augmentation for Improving Out-of-Distribution Detection
in Gastrointestinal Vision

Sandesh Pokhrel Equal Contribution11 Sanjay Bhandari* 11 Eduard Vazquez 22
Tryphon Lambrou 44 Prashnna Gyawali 33 Binod Bhattarai 1144

###### Abstract

Deep learning has significantly advanced the field of gastrointestinal vision,
enhancing disease diagnosis capabilities. One major challenge in automating
diagnosis within gastrointestinal settings is the detection of abnormal cases
in endoscopic images. Due to the sparsity of data, this process of
distinguishing normal from abnormal cases has faced significant challenges,
particularly with rare and unseen conditions. To address this issue, we frame
abnormality detection as an out-of-distribution (OOD) detection problem. In
this setup, a model trained on In-Distribution (ID) data, which represents a
healthy GI tract, can accurately identify healthy cases, while abnormalities
are detected as OOD, regardless of their class. We introduce a test-time
augmentation segment into the OOD detection pipeline, which enhances the
distinction between ID and OOD examples, thereby improving the effectiveness
of existing OOD methods with the same model. This augmentation shifts the
pixel space, which translates into a more distinct semantic representation for
OOD examples compared to ID examples. We evaluated our method against existing
state-of-the-art OOD scores, showing improvements with test-time augmentation
over the baseline approach.

###### Keywords:

OOD Detection Gastrointestinal Disease Test Time Augmentation Abnormality

##  1 Introduction

Out of all the reported diseases worldwide, diseases with a digestive etiology
have burdened the field of medicine with over seven billion incidents reported
in 2019 alone [20]. Owing to technological advancements, the fatality rates
for most diseases have decreased; however, the number of deaths from
gastrointestinal diseases increased from 2000 to 2019 [20]. Invasive
endoscopic procedures are vital for diagnosing various diseases, aiding in the
detection of abnormalities and guiding necessary treatments. Recent
advancements in deep learning, particularly Convolutional Neural Networks
(CNNs), show great promise in automating these procedures. The rise of
portable endoscopy methods, such as capsule endoscopy, has amplified the
demand for automated diagnostic tools. CNNs have been effective in detecting
diseases like gastric cancer [5], polyps, ulcerative colitis, and esophagitis
[10, 15, 14]. They have also been used to classify anatomical landmarks in
endoscopic images. While these models perform well in recognizing normal
landmarks, their accuracy decreases with abnormalities and rare cases, leading
to trust issues [21]. OOD Methods help models detect anomalies, improving
their deployability in real-world scenarios without much modifications.
Researchers utilize logits, probability, gradient, and feature space
information for generalized out-of-distribution detection. Scoring methods in
OODs condense this information to identify in-distribution (ID) or out-of-
distribution (OOD) samples. OOD detection methods range from approaches like
MSP [4] and Entropy [1], which use softmax probabilities, to techniques like
maximum logit [3] and energy score [8]. Advanced methods such as ODIN [7], ViM
[19] and Mahanalobis [6] incorporate additional information like gradients and
feature space distances. KNN-OOD [16] takes a non-parametric approach,
leveraging nearest neighbour distances in feature space to identify out-of-
distribution samples. Test-time augmentations (TTAs) have been demonstrated to
effecteively enhance the performance of various deep learning models across
different domains. Moshkov et al. (2020) [11] showed that simple TTAs, like
rotation and flipping, improve cell segmentation accuracy in microscopy
images. Wang et al. (2019) [18] found TTAs boost brain tumor segmentation
performance. He et al. (2022) [2] demonstrated TTAs’ ability to distinguish
between in-distribution and out-distribution samples for OOD detection.

OOD detection in the medical domain is crucial as it only requires healthy
examples to identify abnormalities. MOOD [22] highlights the importance of
flagging OODs in medical imaging to catch erroneous predictions early. Arnau
et al. [13] proposed a self-supervised approach using ODIN as an OOD scoring
metric to separate anomalies from healthy abdomen sections. Another work [17]
uses ViM for OOD detection with mixup and long-tail ID data calibration
techniques. Mehta et al. [9] employed mixup in long-tail data for skin lesion
detection, classifying new and emerging skin abnormalities. Motivated by these
works, we study the effect of perturbations in image space and how it
translates to semantic representation and whether any tell-tale signs of OOD
data can be extracted from the shift in the representation. A new direction of
research is emerging in gastrovision as well where models are trained solely
on normal anatomical findings for landmark classification, and any deviation
in endoscopic images is treated as an abnormality [17, 13]. However, there is
a lack of studies exploring this approach in diverse settings with different
Out-of-Distribution (OOD) scoring methods and how these scores perform under
augmentation. We address this problem by formulating diseases as abnormalities
in gastrointestinal images. We propose a supervised approach to the
classification of anatomical structures in gastrovision, while these scoring
metrics assist us in discerning healthy regions from unhealthy ones. Our
contributions can be summarised as:

  * $\cdot$

We conceptualize abnormalities in gastrovision as Out-of-Distribution (OOD)
instances, eliminating the necessity for models to be trained on abnormal
images to detect them.

  * $\cdot$

We enhance existing OOD detection methods by integrating test time
augmentations and assess their performance in perturbed scenarios.

  * $\cdot$

Our approach is simple, model-agnostic, and OOD score-independent, making it
versatile and easy to implement across various model architectures and
compatible with any OOD scoring method.

We evaluated our method using different classifier backbone architectures (ViT
and ResNet) on the KvasirV2 dataset [12], showcasing its effectiveness.

##  2 Method

We study how test time augmentations can improve OOD detection methods in
identifying out-of-distribution instances, particularly in gastrointestinal
settings. While existing methods effectively distinguish in-distribution from
OOD examples, they struggle with near-OOD cases that have close semantic
similarity. We explore how test time augmentations on robust models create
drift for test samples, enhancing OOD detection performance. Our method
integrates easily with existing architectures and OOD detection approaches
without changing their processes.

Figure 1: a) OOD based classification pipeline which separates ID (normal
anatomy) and OOD (Disease) data based on feature, logit or gradient
information. b) TTA based OOD detection pipeline which works on images with
individual or composite augmentations.

Problem Formulation: We focus on supervised multi-class classification where,
$X$ is input space, $Y=\\{1,2,\ldots,C\\}$ is label space and
$\text{D}_{in}={(x_{i},y_{i})}^{n}_{i=1}$ is training set (i.i.d. from
$P_{X,Y}$), composed of healthy gastrointestinal images (in-distribution
data). A neural network $f:{X}\rightarrow\mathbb{R}^{|Y|}$ is trained on ID
data using cross-entropy loss to classify healthy classes. The penultimate
layer $\varphi:X\rightarrow\mathbb{R}^{m}$ produces feature vector
$\varphi(x)$ for feature-based OOD detection. Logits-based methods use the
final linear layer output for OOD detection.

Data Augmentations: A set of pre-defined transformations is employed to
generate augmented versions of each test-time sample. These transformations
can be broadly categorized into:

Individual Augmentations (IA): These base operations modify a single aspect of
the data. Examples include horizontal flip (H), vertical flip (V), and color
jittering (CJ). For ID samples, these pixel-level transformations often result
in only minor changes to the semantic representations learned by the model.
However, for OOD samples, the same augmentations can lead to more significant
shifts in the feature space representation, as the model has not encountered
these types of images during training. This discrepancy in the model’s
response to augmented ID and OOD samples can be leveraged to enhance the
performance of OOD detection methods.

Composite Augmentations (CA): These combine multiple individual augmentations
to create more diverse variations. A composite augmentation can be represented
as the sequential application of individual transformations (e.g. H
$\rightarrow$ V $\rightarrow$CJ, H $\rightarrow$ V). In contrast to individual
augmentation, composite augmentations can induce more complex and diverse
perturbations to the input data which leads to more significant shift in the
feature space. This desired perturbation cause more pronounced drifts for OOD
samples than ID samples, which results in larger separations between ID and
OOD data representations and ultimately enhancing the OOD detection
performance. This idea is also empirically verified by Table 2, Table 3 and
Figure 2(a), where composite augmentation seems to be more effective in
enhancing the OOD detection algorithm’s ability to differentiate between in-
distribution (ID) and out-of-distribution (OOD) samples, highlighting the
utility of composite augmentations in improving OOD detection methods. The
equation below formalizes the generation of an augmented sample (x’):

$x^{\prime}=T(x)$ (1)

where, $x$ is original input sample and $T$ is transformation, which can be
either an individual augmentation ($IA_{i}$) or a composite augmentation
($C(IA_{j},...,IA_{k})$).

OOD Score Calculation: Augmented samples are processed through a neural
network trained on in-distribution (ID) data. The model’s output is used to
compute an OOD score for test samples, with the specific calculation method
varying based on model architecture and desired properties. This approach
exploits the model’s response to augmented data to identify deviations from
the ID data manifold, potentially indicating OOD samples. Random
transformations introduce controlled variations in test-time data, simulating
drift and making samples appear more OOD to the model. We hypothesize that a
robust model will show minimal feature space distinction for pixel-space
perturbations in ID inputs, while OOD samples, being unfamiliar, will exhibit
greater differences in semantic understanding.

Following this, the decision for whether a test image is OOD or not can be
formulated as

$\begin{split}g(x)=&\left\\{\begin{array}[]{ll}\textit{OOD},&\text{if
}\text{score}\geq\lambda\\\ \textit{ID},&\text{otherwise}\end{array}\right.\\\
\end{split}$ (2)

where $\lambda$ is the threshold that is decided ensuring that the validation
set retains at least a given true-positive rate.

##  3 Experiments

###  3.1 Datasets and Implementation Details

We utilized the Kvasirv2 [12] multi-class endoscopy dataset annotated which is
verified by experienced endoscopist and is designed for a range of tasks
including Gastrointestinal Disease Detection.

Dataset: The Kvasirv2[12] dataset consists 8 categories of upper and lower GI
tract anatomical landmarks, most common pathological findings, or endoscopic
procedures within the gastrointestinal (GI) tract. The three anatomical
landmarks provided in this dataset are the Z-line, Pylorus, and Cecum.
Pathological finding in the dataset include Esophagitis(ESO), Polyps(POL), and
Ulcerative colitis(UC). In addition to normal and anatomical findings this
dataset also consist of images related to polyp removal, the “dyed and lifted
polyp”(DLP) class and the “dyed resection margins”(DRM). For our experimental
settings, the anatomical landmarks are our classification task classes(ID-
data) that represent healthy GI regions. The remaining five classes are marked
as OOD as they constitute of some form of abnormality that you would see in
the abdomen. For model training, we selected 2400 in-distribution images(80%)
and used 600 in-distribution images(20%), combined with 5000 out-of-
distribution images, to assess the model’s final performance on OOD
capabilities.
Implementation Details: The Resnet-18 model was trained for 20 epochs. The
ViT-Small model was also trained for 20 epochs with a patch size of 16. The
batch size in training was 32 and we used the Adam optimizer with an initial
learning rate of 1×10-4 for both models. The models were initialized with
Imagenet pre-trained weights for better accuracy. The input image size for
both models is resized to 224x224 pixels and the standard cross-entropy loss
function was used to train the models. The models were trained and tested in
PyTorch v2.1.0 with an NVIDIA A100 GPU.

Table 1: Metrics of the optimal model on downstream task (classification of normal landmarks on Kvasirv2 dataset) which is used for OOD detection Model | Parameters | Feat.Dim | Accuracy
---|---|---|---
Resnet-18 | 11M | 512 | 98.57
ViT-small | 21M | 384 | 99.10

###  3.2 Results

Our study presents enhancing out-of-distribution (OOD) detection methods by
employing test time augmentations in gastrointestinal contexts. We assessed
the capabilities of established OOD detection techniques,logit-based methods
like MSP [4], ODIN [7], Energy [8], Entropy [1], MaxLogit [3], feature-based
methods like Mahalanobis [6], and method like Vim [19] which combine both
feature as well as logit information and investigated how test time
augmentations induce subtle drift in test samples and exploited this
phenomenon to improve OOD detection performance of respective methods. The
model deemed optimal for downstream classification task was selected as the
feature extractor for OOD detection task shown in Table 1.

Quantitative Results: AUC and FPR95 are the most commonly used metrics to
evaluate the OOD detection performance. AUC measures the area under the
Receiver Operator Characteristic curve, with higher values indicating better
performance. FPR represents the false positive rate when the true positive
rate is 95%, with smaller values indicating better performance.

Table 2: Quantitative Comparison of enhancement of different OOD methods with
TTA in terms of AUC$\uparrow$ and FPR$\downarrow$ for Resnet-18 on Kvasirv2
dataset. Both values are presented in percentage. Bold characters denote the
best improvement for a particular OOD method under respective augmentation and
underline represents the improvement over baseline with respective TTA.

Augmentations MSP Odin Energy MaxLogit Mahalanobis ViM AUC FPR AUC FPR AUC FPR
AUC FPR AUC FPR AUC FPR None 87.57 33.06 86.95 40.48 86.00 43.28 86.04 43.04
88.58 33.06 89.65 29.86 Hflip 87.97 34.80 87.20 37.88 86.38 40.18 86.43 39.88
88.76 33.34 89.84 27.7 Vflip 87.72 34.42 86.86 40.42 86.10 41.54 86.15 41.22
88.88 31.98 89.5 26.42 Color Jitter 87.66 34.1 86.83 39.56 85.43 46.76 85.93
41.24 88.31 35.7 89.78 28.48 Hflip + Vflip 87.98 34.9 87.08 41.12 86.42 42.00
86.46 41.76 89.63 31.28 90.37 26.7 Hflip + Color jit 87.99 33.28 87.23 37.66
86.39 38.86 86.32 38.44 88.25 32.62 89.59 28.7 Vflip + Color jit 87.57 32.46
86.46 40.34 85.72 40.22 85.89 40.86 88.7 33.02 89.42 27.92 H + V + Color jit
87.66 35.34 86.78 41.32 86.22 43.2 86.12 44.8 89.2 32.78 90.18 27.00

Table 3: Quantitative Comparison of enhancement of different OOD methods with
TTA in terms of AUC$\uparrow$ and FPR$\downarrow$ for ViT-small on Kvasirv2
dataset. Bold characters denote the best improvement for a particular method
under respective augmentation and underline represents improvement over
baseline with respective TTA.

Augmentations MSP Odin Energy MaxLogit Mahalanobis ViM AUC FPR AUC FPR AUC FPR
AUC FPR AUC FPR AUC FPR None 85.93 39.74 83.84 42.1 83.52 43.12 83.53 43.12
93.06 25.66 93.70 26.34 Hflip 85.16 43.32 82.79 43.78 82.41 44.34 82.41 44.32
92.36 28.06 93.59 28.3 Vflip 87.04 36.5 85.11 40.1 84.11 41.76 84.11 41.74
91.99 22.42 93.54 22.08 Color Jitter 86 39.32 83.92 41.86 83.51 43.48 83.51
42.5 93.00 25.92 93.74 25.48 Hflip + Vflip 86.35 36.58 84.24 41.3 83.37 43.82
83.37 43.82 91.38 23.52 92.83 20.92 Hflip + Color Jit 85.22 43.2 82.8 43.52
82.4 44.82 82.47 43.84 92.35 27.12 93.55 29 Vflip + Color Jit 87.14 36.16 85.2
39.92 84.25 42.34 84.25 42.74 91.97 21.06 93.86 20.98 H + V + Color Jit 86.38
36.28 84.43 41.3 83.46 43.32 83.42 44.02 91.34 24.26 92.88 20.3

Tables 2 and 3 show that test-time augmentation significantly improves out-of-
distribution (OOD) detection performance. For Resnet-18, ViM achieves the
lowest False Positive Rate (FPR) of 26.7% with Vertical flipping, a 3.16%
improvement over the baseline. For ViT-small, ViM achieves a 20.3% FPR with
Horizontal flipping, Vertical flipping, and Color Jitter, a 6.04% enhancement.
It’s evident that the application of test-time augmentations consistently
enhances the performance of all OOD detection baseline methods. Achieving a
low FPR95 is particularly crucial as it directly contributes to reducing false
positives and thereby increasing the trustworthiness of the model’s
predictions. By incorporating test-time augmentations, we not only strengthen
the robustness of the model but also equip it with better generalization
capabilities in terms of OOD detection, which are paramount for real-world
applications where unseen data instances are common.

Qualitative Results: The qualitative results in Fig 2 shows the improvement in
performance of MaxLogit method on different OOD classes when subject to
augmentation. The effect of different augmentations and their predictions
either OOD or misclassified as healthy organs are presented in Fig 2.a. It is
evident from these results that Test Time Augmentation introduces drift in
representation space allowing existing OOD metrics to distinguish ID and OOD
better. Moreover, the drift introduced in representation space is different
for different augmentations owing to difference in pixel space perturbation.
In general, composite augmentations are stronger and create a larger drift as
seen in the predictions.

Figure 2: a) Qualitative comparison of Maxlogit method for Kvasirv2 on
Resnet18: OOD examples over-confidently predicted by the corresponding method
as healthy ID data (red) and correctly identified as abnormality (green) under
respective test time augmentation technique. b) Performance improvement on
Maxlogit method on FPR95$\downarrow$ on separate OOD classes for Kvasirv2 on
Resnet18 model.

Ablation Study: Ablation experiments were performed to assess the impact of
different augmentation methods on out-of-distribution (OOD) scoring
performance. Table 4 shows that Horizontal Flip (Hflip), Vertical Flip
(Vflip), and Color Jitter improve OOD detection for both logit-based and
feature-based methods. Conversely, Equalize and Invert exhibit a detrimental
effect, reducing the performance for both methods. In addition, Table 5
demonstrates that augmenting OOD images causes a drift in representation
space, aiding in distinguishing OOD data. The mean OOD score, higher for OOD
data, increases with augmentation. ViM shows an average drift of 2.71, raising
the mean OOD score from 6.70 to 9.41 with Horizontal and Vertical Flipping.
Similar results are observed in logits-based methods like MaxLogit.

Table 4: Ablation study for impact of different augmentation method in terms
of AUC$\uparrow$ and FPR$\downarrow$ for Resnet-18 on Kvasirv2 dataset.

Augmentations MaxLogit ViM AUC FPR AUC FPR None 86.04 43.04 89.65 29.86 Hflip
86.43 39.88 89.84 27.7 Vflip 86.15 41.22 89.5 26.42 Color Jitter 85.93 41.24
89.78 28.48 Equalize 86.43 46.88 90.36 30.00 Invert 82.27 58.88 89.16 33.94

Table 5: Ablation study on drift introduced by TTA that enhances the
robustness of OOD detection methods. The average OOD score on OOD examples
increases indicating that we get larger values for each OOD example in general
which resonates with the fact that OOD examples get higher score values than
ID data.

Augmentations Mean OOD Score MaxLogit ViM None 5.98 6.70 Hflip + Vflip 6.03
9.41

##  4 Conclusion

We reformulate abnormality detection in gastrointestinal images as an out-of-
distribution (OOD) problem, enabling the recognition of abnormalities in the
GI tract using a model trained solely on normal anatomical findings. To
achieve this, we introduce test-time augmentation (TTA), which significantly
improves OOD detection performance by augmenting the input data during
inference. This approach, when combined with existing OOD detection methods
from the OOD literature, enhances their generalization and robustness
regarding OOD detection. Overall, our findings highlight the considerable
potential of supervised models in OOD detection, serving as a versatile tool
for identifying abnormalities in endoscopy images from gastrointestinal
settings—an area largely unexplored in medical imaging research.

## References

  * [1] Chan, R., Rottmann, M., Gottschalk, H.: Entropy maximization and meta classification for out-of-distribution detection in semantic segmentation (2021)
  * [2] He, H., Teng, J., Yuan, Y.: Anomaly detection with test time augmentation and consistency evaluation (2022)
  * [3] Hendrycks, D., Basart, S., Mazeika, M., Zou, A., Kwon, J., Mostajabi, M., Steinhardt, J., Song, D.: Scaling out-of-distribution detection for real-world settings (2022)
  * [4] Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. In: International Conference on Learning Representations (2017)
  * [5] Hirasawa, T., Aoyama, K., Tanimoto, T., Ishihara, S., Shichijo, S., Ozawa, T., Ohnishi, T., Fujishiro, M., Matsuo, K., Fujisaki, J., et al.: Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer 21, 653–660 (2018)
  * [6] Lee, K., Lee, K., Lee, H., Shin, J.: A simple unified framework for detecting out-of-distribution samples and adversarial attacks (2018)
  * [7] Liang, S., Li, Y., Srikant, R.: Enhancing the reliability of out-of-distribution image detection in neural networks (2020)
  * [8] Liu, W., Wang, X., Owens, J., Li, Y.: Energy-based out-of-distribution detection. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems. vol. 33, pp. 21464–21475. Curran Associates, Inc. (2020)
  * [9] Mehta, D., Gal, Y., Bowling, A., Bonnington, P., Ge, Z.: Out-of-distribution detection for long-tailed and fine-grained skin lesion images. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. pp. 732–742. Springer Nature Switzerland, Cham (2022)
  * [10] Melaku Bitew Haile, Ayodeji Olalekan Salau, B.E., Belay, A.J.: Detection and classification of gastrointestinal disease using convolutional neural network and svm. Cogent Engineering 9(1), 2084878 (2022)
  * [11] Moshkov, N., Mathe, B., Kertesz-Farkas, A., Hollandi, R., Horvath, P.: Test-time augmentation for deep learning-based cell segmentation on microscopy images. Scientific reports 10(1), 5068 (2020)
  * [12] Pogorelov, K., Randel, K.R., Griwodz, C., Eskeland, S.L., de Lange, T., Johansen, D., Spampinato, C., Dang-Nguyen, D.T., Lux, M., Schmidt, P.T., Riegler, M., Halvorsen, P.: Kvasir: A multi-class image dataset for computer aided gastrointestinal disease detection (2017)
  * [13] Quindós, A., Laiz, P., Vitrià, J., Seguí, S.: Self-supervised out-of-distribution detection in wireless capsule endoscopy images. Artificial Intelligence in Medicine 143, 102606 (2023)
  * [14] Sharma, A., Kumar, R., Garg, P.: Deep learning-based prediction model for diagnosing gastrointestinal diseases using endoscopy images. Int J Med Inform 177, 105142 (Jul 2023)
  * [15] Sivari, E., Bostanci, E., Guzel, M.S., Acici, K., Asuroglu, T., Ercelebi Ayyildiz, T.: A new approach for gastrointestinal tract findings detection and classification: Deep learning-based hybrid stacking ensemble models. Diagnostics 13(4) (2023)
  * [16] Sun, Y., Ming, Y., Zhu, X., Li, Y.: Out-of-distribution detection with deep nearest neighbors. In: Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., Sabato, S. (eds.) Proceedings of the 39th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 162, pp. 20827–20840. PMLR (17–23 Jul 2022)
  * [17] Tan, Q., Bai, L., Wang, G., Islam, M., Ren, H.: Endoood: Uncertainty-aware out-of-distribution detection in capsule endoscopy diagnosis (2024)
  * [18] Wang, G., Li, W., Ourselin, S., Vercauteren, T.: Automatic brain tumor segmentation using convolutional neural networks with test-time augmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part II 4. pp. 61–72. Springer (2019)
  * [19] Wang, H., Li, Z., Feng, L., Zhang, W.: Vim: Out-of-distribution with virtual-logit matching (2022)
  * [20] Wang, Y., Huang, Y., Chase, R.C., Li, T., Ramai, D., Li, S., Huang, X., Antwi, S.O., Keaveny, A.P., Pang, M.: Global burden of digestive diseases: A systematic analysis of the global burden of diseases study, 1990 to 2019. Gastroenterology 165(3), 773–783.e15 (Jun 2023)
  * [21] Zadorozhny, K., Thoral, P., Elbers, P., Cinà, G.: Out-of-Distribution Detection for Medical Applications: Guidelines for Practical Evaluation, pp. 137–153. Springer International Publishing, Cham (2023)
  * [22] Zimmerer, D., Full, P.M., Isensee, F., Jäger, P., Adler, T., Petersen, J., Köhler, G., Ross, T., Reinke, A., Kascenas, A., Jensen, B.S., O’Neil, A.Q., Tan, J., Hou, B., Batten, J., Qiu, H., Kainz, B., Shvetsova, N., Fedulova, I., Dylov, D.V., Yu, B., Zhai, J., Hu, J., Si, R., Zhou, S., Wang, S., Li, X., Chen, X., Zhao, Y., Marimont, S.N., Tarroni, G., Saase, V., Maier-Hein, L., Maier-Hein, K.: Mood 2020: A public benchmark for out-of-distribution detection and localization on medical images. IEEE Transactions on Medical Imaging 41(10), 2728–2738 (Oct 2022)


# Strongly Augmented Contrastive Clustering

Xiaozhi Deng, Dong Huang, Ding-Hua Chen
Chang-Dong Wang, , Jian-Huang Lai  This project was supported by the NSFC
(61976097, 61876193 & 62076258) and the Natural Science Foundation of
Guangdong Province (2021A1515012203). (Corresponding author: Dong Huang) X.
Deng, D. Huang, and D.-H. Chen are with the College of Mathematics and
Informatics, South China Agricultural University, Guangzhou, China.
E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>C.-D. Wang and J.-H. Lai are with the School of
Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China,
and also with Guangdong Key Laboratory of Information Security Technology,
Guangzhou, China, and also with Key Laboratory of Machine Intelligence and
Advanced Computing, Ministry of Education, China.
E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Deep clustering has attracted increasing attention in recent years due to its
capability of joint representation learning and clustering via deep neural
networks. In its latest developments, the contrastive learning has emerged as
an effective technique to substantially enhance the deep clustering
performance. However, the existing contrastive learning based deep clustering
algorithms mostly focus on some carefully-designed augmentations (often with
limited transformations to preserve the structure), referred to as weak
augmentations, but cannot go beyond the weak augmentations to explore the more
opportunities in stronger augmentations (with more aggressive transformations
or even severe distortions). In this paper, we present an end-to-end deep
clustering approach termed Strongly Augmented Contrastive Clustering (SACC),
which extends the conventional two-augmentation-view paradigm to multiple
views and jointly leverages strong and weak augmentations for strengthened
deep clustering. Particularly, we utilize a backbone network with triply-
shared weights, where a strongly augmented view and two weakly augmented views
are incorporated. Based on the representations produced by the backbone, the
weak-weak view pair and the strong-weak view pairs are simultaneously
exploited for the instance-level contrastive learning (via an instance
projector) and the cluster-level contrastive learning (via a cluster
projector), which, together with the backbone, can be jointly optimized in a
purely unsupervised manner. Experimental results on five challenging image
datasets have shown the superiority of our SACC approach over the state-of-
the-art. The code is available at https://github.com/dengxiaozhi/SACC.

###### Index Terms:

Data clustering, Deep clustering, Image clustering, Contrastive learning, Deep
neural network

##  I Introduction

Data clustering is a fundamental task in unsupervised learning, which aims to
group a set of data samples into different unlabeled clusters. The traditional
clustering algorithms, such as $K$-means [1], agglomerative clustering (AC)
[2], and spectral clustering (SC) [3], typically rely on the hand-crafted data
features, which lack the representation learning ability and may lead to poor
clustering performance when dealing with some complex high-dimensional data,
such as images and videos, where the proper features are not easy to be
manually extracted.

With the rapid development of deep learning, the deep neural network has
recently been adopted to learn proper representations for the clustering task
on complex high-dimensional data. In the past few years, many clustering
algorithms based on deep neural networks (referred to as deep clustering
algorithms) have been devised [4, 5, 6, 7, 8]. As one of the earliest deep
clustering works, Xie et al. [9] proposed a deep embedded clustering (DEC)
method to simultaneously perform feature representations learning and
clustering in a deep neural network, where the distribution of soft labels and
an auxiliary target distribution are constrained via a Kullback-Leibler (KL)
divergence based loss. Guo et al. [5] developed an improved deep embedded
clustering (IDEC) method by learning the feature representation and the
cluster assignment with local structure preservation. Caron et al. [6]
iteratively clustered the learned features via $K$-means and update the
weights of the deep neural network by using the cluster assignment as soft
labels. Ji et al. [7] designed a deep clustering method termed invariant
information clustering (IIC), which seeks to maximize the mutual information
between the original image and the augmented one for more robust
representation learning and clustering.

Figure 1: The visualization of weak and strong augmentations. (a) Original
image. (b) Weak augmentations. (c) Strong augmentations.  Figure 2:
Illustration of the proposed SACC framework. By simultaneously incorporating
strong and weak augmentations, a backbone network with triply-shared weights
is used to learn the representations of three augmentation views, which are
then fed to two types of projectors for instance-level and cluster-level
contrastive learning, respectively.

Despite the considerable progress, these methods [4, 5, 6, 7] often perform
the feature representation learning and clustering by considering the overall
distributions (such as the distribution of soft labels or some other target
distributions), which overlook the the sample-wise relationships and their
contrastiveness. Recently, the contrastive learning has emerged as an
effective technique for enhancing the deep clustering performance, which
typically generates positive sample pairs and negative sample pairs via data
augmentations, and aims to maximize the agreement between positive pairs and
minimize the agreement between negative pairs. For example, van Gansbeke et
al. [10] proposed a two-stage deep clustering method termed semantic
clustering by adaptive nearest neighbors (SCAN), which first utilizes the
contrastive learning to learn the discriminant features for finding the $K$
nearest neighbors, and then trains the network via a loss function that aims
to pull each sample and its $K$ nearest neighbors closer. Dang et al. [11]
extended the SCAN method by matching both local and global nearest neighbors.
Li et al. [12] devised a one-stage method termed contrastive clustering (CC),
which jointly exploits instance-level and cluster-level contrastive learning
in an end-to-end manner.

Though these contrastive learning based deep clustering methods [10, 11, 12]
have shown substantial improvements on some complex image datasets, yet there
are still two limitations to most of them. First, the previous deep clustering
methods with contrastive learning tend to utilize weak augmentations (with
limited transformations to preserve the structure) for original images, but
mostly overlook the opportunities in stronger augmentations which may even be
coupled with more aggressive transformations or distortions (as illustrated in
Fig. 1). Second, they mostly design the network with two augmentation views
(typically drawn from the same family of augmentations), but cannot go beyond
two augmentation views to explore multiple views of augmentations (especially
with varying degrees of transformations or distortions). More recently, Wang
and Qi [13] shown that the incorporation of stronger augmentations can enhance
the feature representation learned by contrastive learning, which, however, is
designed for the general-purpose contrastive learning but lacks the ability to
achieve the representation learning and clustering simultaneously. It remains
an open problem how to simultaneously leverage the strong and weak
augmentations while extending the conventional two-view network architecture
to explore opportunities in multiple augmentation views in a unified deep
clustering framework.

In light of this, this paper presents a novel end-to-end deep clustering
approach termed strongly augmented contrastive clustering (SACC), which is
able to jointly learn the feature representation and the cluster assignments
with the strong and weak augmentations simultaneously leveraged in a network
of multiple augmentation views (as illustrated in Fig. 2). In particular, our
SACC approach utilizes a backbone network with triply-shared weights to
produce the feature embeddings of a strongly augmented view and two weakly
augmented views, upon which the weak-weak view pair and the strong-weak view
pairs can be constructed for the instance-level and cluster-level contrastive
learning. With the strong and weak augmentations as well as the instance-level
and cluster-level contrastive learning jointly leveraged, the network training
can thus be performed in a purely unsupervised manner and the deep clustering
result is therefore obtained. Extensive experiments are conducted on five
challenging image datasets, which demonstrate the superiority of our SACC
approach over the state-of-the-art deep clustering approaches.

For clarity, the key contributions of this work are summarized as follows.

  * •

This paper for the first time, to the best of our knowledge, jointly leverages
strong and weak augmentations for the task of unsupervised image clustering.

  * •

A novel end-to-end deep clustering approach termed SACC is proposed, which
utilizes three augmentations views for simultaneous instance-level and
cluster-level contrastive learning..

  * •

Extensive experimental results have confirmed that our SACC approach
outperforms the state-of-the-art deep clustering approaches on several
challenging image datasets.

The remainder of this paper is organized as follows. Section II reviews the
related works on deep clustering. Section III describes the proposed SACC
framework. Section IV reports the experimental results. Finally, we conclude
this paper in Section V.

##  II Related Work

Deep learning has proved to be an advantageous technique for unsupervised
clustering of very complex data. Many deep clustering methods have been
designed[4, 5, 6, 7, 8, 14, 15, 16, 17, 18, 19, 20], whose difference can
often be reflected by their network losses, such as the reconstruction loss of
autoencoder (AE), the variational loss of variational autoencoder (VAE) [21],
the loss of generative adversarial network (GAN) [22], and some specific
clustering losses [23, 24].

The AE-based deep clustering methods generally optimize the networks by both
the reconstruction loss and some clustering loss. The reconstruction loss
measures the disagreement between the original input and the reconstruction.
Yang et al. [14] presented the deep clustering network (DCN) method with the
dimensionality reduction with the $K$-means clustering jointly modeled. Ji et
al. [15] proposed an AE-based deep clustering method with a self-expressive
layer for deep subspace clustering. Dizaji et al. [16] developed the deep
embedded regularized clustering (DEPICT) method based on AE embedding and
relative entropy minimization.

The VAE-based deep clustering methods utilize the VAE to regularize the
network training to avoid over-fitting by enforcing the latent space to follow
some predefined distribution. Jiang et al. [17] presented a variational deep
embedding (VaDE) method that optimizes the VAE by maximizing the evidence
lower bound. Dilokthanakul et al. [18] proposed a Gaussian mixture variational
autoencoder (GMVAE) method by incorporating a variational Bayes in its
optimization objective.

The GAN-based deep clustering methods seek to train the network with a min-max
adversarial game. Springenberg [19] proposed a categorical generative
adversarial network (CatGAN) method that jointly exploits GAN and regularized
information maximization (RIM) to train the network. Chen et al. [25]
developed an information maximizing generative adversarial network (InfoGAN)
method that aims to extract interpretable and disentangled features for deep
clustering.

Different from the above three categories that usually combine the clustering
loss with some network losses (such as the losses of AE, VAE, and GAN),
another category of deep clustering methods aim to train the network with only
the clustering loss. For example, Yang et al. [23] leveraged a convolutional
neural network to learn representation feature and image clusters by a
weighted triplet loss. Xie et al. [9] devised a deep embedding clustering
(DEC) method that jointly optimizes deep embedding and clustering with a KL
divergence based loss between the distribution of soft labels and an auxiliary
target distribution. Guo et al. [24] developed an adaptive self-paced deep
clustering with data augmentation (ASPC-DA) method that incorporates data
augmentation and self-paced learning into deep clustering.

Recently, the contrastive learning has become a popular topic [26], and
several attempts have been made to utilize the contrastive loss to improve the
deep clustering performance [10, 11, 12]. Typically, van Gansbeke et al. [10]
proposed a two-stage deep clustering method which adopts the contrastive
learning as a pretext task to learn discriminant features and then exploits
the $K$ nearest neighbors (via the learned features) in the second-stage
network training. Dang et al. [11] presented a nearest neighbor matching (NNM)
method by considering not only the global nearest neighbors but also the local
nearest neighbors. Li et al. [12] performed contrastive learning at both
instance-level and cluster-level and obtained the clustering result via a
cluster projector.

##  III Proposed Framework

In this section, we describe the proposed SACC framework. Specifically, an
overview the of the framework is given in Section III-A. The weak and strong
augmentations are introduced in Section III-B. The design of the network
architecture is provided in Section III-C. Finally, the implementation details
are presented in Section III-D.

###  III-A Framework Overview

The overall framework of SACC is illustrated in Fig. 2. In SACC, we utilize a
backbone network with triply-shared weights, where the representations of
three augmentation views (including one strong augmentation view and two weak
augmentation views) are learned. Specifically, given a mini-batch of $N$ image
samples, we performs one type of strong augmentation and two types of weak
augmentations on each input image, denoted as $x_{i}$, which lead to $3\cdot
N$ augmented samples denoted as
$\\{v_{1}^{1},\ldots,v_{N}^{1},v_{1}^{2},\ldots,v_{N}^{2},v_{1}^{3},\ldots,v_{N}^{3}\\}$,
with $N$ strongly augmented samples and $2\cdot N$ weakly augmented samples.
The backbone network $f_{\theta}$ transforms each augmented sample $v_{i}^{j}$
to $z_{i}^{j}$, with $i\in[1,N]$ and $j\in\\{1,2,3\\}$, which will then be fed
to the instance projector and the cluster projector. With the instance
projector $g_{\theta}$ transforming $z_{i}^{j}$ to $y_{i}^{j}$ and the cluster
predictor $h_{\theta}$ transforming $z_{i}^{j}$ to $c_{i}^{j}$, two types of
feature matrices are built (for this mini-batch of samples) via the two
projectors, respectively. Thereafter, unsupervised network training can be
performed by simultaneous optimizing the instance-level contrastive loss (in
the row space of the feature matrix in the instance projector) and the
cluster-level contrastive loss (in the column space of the feature matrix in
the cluster projector).

###  III-B Augmentations: From Weak to Strong

The contrastive learning has shown its promising ability in unsupervised
representation learning [26], and has been utilized in some recent deep
clustering methods [10, 11, 12]. In previous deep clustering methods with
contrastive learning, some weak augmentations (with limited transformations to
preserve the image structure) are generally exploited to form the positive
pairs. However, few of them have gone beyond the weak augmentations to utilize
some stronger augmentations (with more aggressive transformations or even
severe distortions).

In this paper, we have shown that the joint use of strong and weak
augmentations can substantially strengthen the contrastive learning ability
for representation learning and clustering in the deep clustering framework.
Specifically, in terms of the weak augmentations, we adopt a family of four
often-used augmentations, namely, ResizedCrop, HorizontalFlip, ColorJitter,
and Grayscale, to generate weakly augmented samples. For the two weak
augmentation views in SACC, two augmentations are randomly chosen from the
family of weak augmentations for each input image. Besides the weak
augmentations, we adopt a family of fourteen stronger transformations [13],
including AutoContrast, Brightness, Color, Contrast, Equalize, Identity,
Posterize, Rotate, Sharpness, ShearX/Y, Solarize and TranslateX/Y. Since the
strong augmentations transform the original image more aggressively, it can
provide some additional clues that do not exist in the weak augmentations for
learning distinctive representations. By jointly modeling strong and weak
augmentations, the proposed SACC framework is able to obtain more
representative patterns and semantic information of images for learning
clustering-friendly representations.

###  III-C Network Architecture

The network architecture of SACC consists of three modules, namely, the
backbone network, the instance projector, and the cluster projector. An
instance-level contrastive loss and an cluster-level contrastive loss are
utilized in the instance projector and the cluster projector, respectively,
which are jointly trained with both the weak and strong augmentation views. In
the following, we will describe the three modules as well as the overall
objective function in detail.

####  III-C1 Backbone Network with Triply-Shared Weights

In the proposed framework, we utilize a backbone network with triply-shared
weights (as shown in Fig. 2). Specifically, three augmentation views,
including a strong augmentation view and two weak augmentation views share the
backbone network $f_{\theta}$, through which three views of representations
can be learned, denoted as $z_{i}^{1}=f(v_{i}^{1})$, $z_{i}^{2}=f(v_{i}^{2})$,
and $z_{i}^{3}=f(v_{i}^{3})$. Then three feature representations are fed to
each of the two projectors for the later instance-level and cluster-level
contrastive learning. Note that we can adopt different network structures as
the backbone. In this work, we adopt the widely-used ResNet-34 [27] as our
backbone network.

####  III-C2 Instance Projector with Weak-Strong Augmentations

In the instance projector, a two-layer nonlinear multilayer perceptron (MLP),
denoted as $g(\cdot)$, is used to transform $z_{i}^{j}$ to a lower-dimensional
space, that is, $y_{i}^{j}=g(z_{i}^{j})$, where $y_{i}^{j}$ is interpreted as
the instance representation, with $i\in[1,N]$ and $j\in\\{1,2,3\\}$.

As there are three augmentation views, including two weak augmentation views
and a strong augmentation view, we use a weak-weak pair and a strong-strong
pair to form the positive pairs and the negative pairs. Specifically, for each
input image, its two weakly augmented samples form a positive pair, and its
strongly augmented sample and its first weakly augmented sample form another
positive pair. In the meantime, negative pairs are formed between the
augmented samples from different input images.

With the positive pairs and negative pairs defined, the instance-level
contrastive loss is utilized to maximize the agreement of positive pairs while
increasing the distance of negative pairs. To measure the similarity of
instance pairs, the cosine similarity can be used, that is

$s(u,v)=\frac{u^{\top}v}{\|u\|\|v\|},$ (1)

where $u$ and $v$ denote two feature vectors. To optimize the agreement of
contrastive pairs constructed from two augmentations, say, augmentation $a$
and augmentation $b$, the contrastive loss for an augmented sample $v_{i}^{a}$
is defined as

$\ell_{i}^{a}=-\log\frac{\exp(s(y_{i}^{a},y_{i}^{b})/\tau_{g})}{\sum_{j=1}^{N}[\exp(s(y_{i}^{a},y_{j}^{a})/\tau_{g})+\exp(s(y_{i}^{a},y_{j}^{b})/\tau_{g})]},$
(2)

with $i,j\in[1,N]$ and $a,b\in\\{1,2,3\\}$. The parameter $\tau_{g}$ is the
temperature parameter. In order to identify all positive pairs from two
augmentations (say, $a$ and $b$), the instance-level contrastive loss is
calculated over every augmented examples, that is

$\mathcal{L}_{instance(a,b)}=\frac{1}{2N}\sum_{i=1}^{N}(\ell_{i}^{a}+\ell_{i}^{b})$
(3)

In the instance projector, we construct two augmented view pairs for each
original input. One pair consists of a strongly augmented view and a weakly
augmented view, while the other one consists of two weakly augmented views.
Thus the contrastive loss for the instance projector is defined as

$\mathcal{L}_{instance}=\mathcal{L}_{instance(1,2)}+\mathcal{L}_{instance(2,3)}$
(4)

####  III-C3 Cluster Projector with Weak-Strong Augmentations

The cluster projector is a two-layer nonlinear MLP with a softmax layer,
denoted as $h(\cdot)$. The dimension of the output layer of the cluster
projector, denoted as $M$, is equal to the number of classes (or the desired
number of clusters). The output representation (for each sample) in the
cluster projector, computed by $\tilde{c}_{i}^{j}=h(z_{i}^{j})$, can be
treated as the probabilities of this sample belonging to different classes.
Thus, $\tilde{c}_{i}^{j}$ can serve as a soft label for the augmented sample.

For each of the three augmentation views, a feature matrix with $N$ rows and
$M$ columns can be obtained for a mini-batch of $N$ samples, where $c_{m}^{j}$
denotes the $m$-th column of the feature matrix, with $m\in[1,M]$ and
$j\in\\{1,2,3\\}$. That is, $c_{m}^{j}$ can be regarded as the distribution of
the $N$ samples in the $m$-th cluster of the augmentation $j$. We treat the
same cluster from two different augmentation views as a positive cluster pair,
and the other cluster pairs as the negative cluster pairs. Then, for a cluster
$c_{m}^{j}$, the cluster-level contrastive loss can be defined as

$\hat{\ell}_{m}^{a}=-\log\frac{\exp(s(c_{m}^{a},c_{m}^{b})/\tau_{h})}{\sum_{n=1}^{M}[\exp(s(c_{m}^{a},c_{n}^{a})/\tau_{h})+\exp(s(c_{m}^{a},c_{n}^{b})/\tau_{h})]}$
(5)

with $m,n\in[1,N]$ and $a,b\in\\{1,2,3\\}$. The parameter $\tau_{h}$ is the
temperature parameter. After traversing all clusters, the cluster-level
contrastive loss can further be represented as

$\displaystyle\mathcal{L}_{cluster(a,b)}=\frac{1}{2M}\sum_{m=1}^{M}(\hat{\ell}_{m}^{a}+\hat{\ell}_{m}^{b})-H(Y),$
(6) $\displaystyle H(Y)=-\sum_{m=1}^{M}[P(c_{m}^{a})\log
P(c_{m}^{a})+P(c_{m}^{b})\log P(c_{m}^{b})],$ (7)

where $H(Y)$ is the entropy of the cluster assignment probabilities with
$P(c_{m}^{k})=\sum_{n=1}^{N}Y_{nm}^{k}/\|Y\|_{1}$, for $k\in\\{a,b\\}$ within
a mini-batch under each data augmentation. This term is incorporated to avoid
the trivial solution that most samples are assigned to the same cluster.

For every original image, we utilize three augmentation view pairs in the
cluster projector, i.e., every two augmentation views form a view pair. Then
the contrastive loss in the cluster projector is defined as

$\mathcal{L}_{cluster}=\mathcal{L}_{cluster(1,2)}+\mathcal{L}_{cluster(1,3)}+\mathcal{L}_{cluster(2,3)}$
(8)

####  III-C4 Overall Objective

The optimization of the backbone network, the instance projector, and the
cluster projector is jointly performed in an end-to-end manner. The overall
objective function is composed of the instance-level contrastive loss and the
cluster-level contrastive loss, that is

$\mathcal{L}=\mathcal{L}_{cluster}+\mathcal{L}_{instance}.$ (9)

Thereby, the unsupervised network training of our SACC approach can be
conducted with both instance-level and cluster-level contrastive learning upon
three weak/strong augmentation views.

###  III-D Implementation Details

In the proposed framework, all original images of different sizes are resized
to the size of 224$\times$224\. We use the ResNet-34 as the backbone network
which is designed for input images with 224$\times$224 pixels. As for the
instance projector, its output dimensionality set to 128 in order to hold
sufficient information after the transformation. As for the cluster projector,
the output dimensionality is set to the desired number of clusters, where the
output feature vector can be treated as the soft label. The temperature
parameters of the instance projector and the cluster projector are fixed to
0.5 and 1, respectively. In the training process, we adopt Adam optimizer with
a learning rate of 0.0003 to simultaneously optimize the backbone network and
the two projectors. The batch size for training is set to 200, and the number
of training epochs is set to 1000.

TABLE I: Description of the benchmark datasets. Dataset |  #Images |  #Classes
---|---|---
CIFAR-10 |  60,000 |  10
CIFAR-100 |  60,000 |  20
STL-10 |  13,000 |  10
ImageNet-10 |  13,000 |  10
ImageNet-Dogs |  19,500 |  15

(a) CIFAR-10

(b) CIFAR-100

(c) STL-10

(d) ImageNet-10

(e) ImageNet-Dogs

Figure 3: Visualization of some image samples from the five benchmark datasets. TABLE II: The clustering performance (w.r.t. NMI(%)) by different clustering algorithms on the five image datasets. The best score in each column is in bold. Dataset |  CIFAR-10 |  CIFAR-100 |  STL-10 |  ImageNet-10 |  ImageNet-Dogs
---|---|---|---|---|---
K-means [1] |  8.7 |  8.4 |  12.5 |  11.9 |  5.5
SC [3] |  10.3 |  9.0 |  9.8 |  15.1 |  3.8
AC [2] |  10.5 |  9.8 |  23.9 |  13.8 |  3.7
NMF [28] |  8.1 |  7.9 |  9.6 |  13.2 |  4.4
AE [29] |  23.9 |  10.0 |  25.0 |  21.0 |  10.4
DAE [30] |  25.1 |  11.1 |  22.4 |  20.6 |  10.4
DCGAN [31] |  26.5 |  12.0 |  21.0 |  22.5 |  12.1
DeCNN [32] |  24.0 |  9.2 |  22.7 |  18.6 |  9.8
VAE [21] |  24.5 |  10.8 |  20.0 |  19.3 |  10.7
JULE [23] |  19.2 |  10.3 |  18.2 |  17.5 |  5.4
DEC [9] |  25.7 |  13.6 |  27.6 |  28.2 |  12.2
DAC [33] |  39.6 |  18.5 |  36.6 |  39.4 |  21.9
DDC [34] |  42.4 |  - |  37.1 |  43.3 |  -
DCCM [35] |  49.6 |  28.5 |  37.6 |  60.8 |  32.1
IIC [7] |  51.1 |  22.5 |  49.6 |  - |  -
GATCluster [36] |  49.6 |  28.5 |  44.6 |  59.4 |  28.1
PICA [37] |  59.1 |  31.0 |  61.1 |  80.2 |  35.2
DRC [38] |  62.1 |  35.6 |  64.4 |  83.0 |  38.4
CC [12] |  68.1 |  42.4 |  67.4 |  86.2 |  40.1
SACC(our) |  76.5 |  44.8 |  69.1 |  87.7 |  45.5

##  IV Experiments

In this section, we conduct extensive experiments on five image datasets to
evaluate the clustering performance of our SACC algorithm against eighteen
traditional and deep clustering algorithms.

###  IV-A Datasets and Evaluation Metrics

In our experiments, five challenging image datasets are used, namely, CIFAR-10
[39], CIFAR-100 [39], STL-10 [40], ImageNet-10 [33], and ImageNet-Dogs [33].
Similar to the previous deep clustering works [35, 37, 12], we jointly utilize
the training and testing samples of each dataset, due to the unsupervised
nature of the clustering task. Specifically, the five benchmark datasets are
introduced below.

  * •

CIFAR-10 is a natural image dataset which consists of 60,000 images from 10
object classes.

  * •

CIFAR-100 has the same size and the same number of samples as the CIFAR-10
dataset, but it contains 20 super-classes, which can be further divided into
100 classes. Following the previous works, we use the 20 super-classes as the
ground-truth when evaluating the clustering performance.

  * •

STL-10 is an ImageNet-sourced dataset which collects 13,000 color images with
the size of $96\times 96$ from 10 classes.

  * •

ImageNet-10 is a subset of ImageNet with 10 classes, each of which consists of
1,300 samples with varying image sizes.

  * •

ImageNet-Dogs is constructed in a similar way to ImageNet-10, but it selects a
total of 19,500 dog images of 15 breeds from the ImageNet dataset.

For clarity, we visualize some samples from the five image datasets in Fig. 3,
and provide the statistics of them in Table I.

To evaluate the clustering results of different clustering algorithms, we
adopt three widely-used evaluation metrics [35, 41, 12], namely, clustering
accuracy (ACC), normalized mutual information (NMI) and adjusted rand index
(ARI). Note that greater values of the three metrics indicate better
clustering results.

TABLE III: The clustering performance (w.r.t. ACC(%)) by different clustering algorithms on the five image datasets. The best score in each column is in bold. Dataset |  CIFAR-10 |  CIFAR-100 |  STL-10 |  ImageNet-10 |  ImageNet-Dogs
---|---|---|---|---|---
K-means [1] |  22.9 |  13.0 |  19.2 |  24.1 |  10.5
SC [3] |  24.7 |  13.6 |  15.9 |  27.4 |  11.1
AC [2] |  22.8 |  13.8 |  33.2 |  24.2 |  13.9
NMF [28] |  19.0 |  11.8 |  18.0 |  23.0 |  11.8
AE [29] |  31.4 |  16.5 |  30.3 |  31.7 |  18.5
DAE [30] |  29.7 |  15.1 |  30.2 |  30.4 |  19.0
DCGAN [31] |  31.5 |  15.3 |  29.8 |  34.6 |  17.4
DeCNN [32] |  28.2 |  13.3 |  29.9 |  31.3 |  17.5
VAE [21] |  29.1 |  15.2 |  28.2 |  33.4 |  17.9
JULE [23] |  27.2 |  13.7 |  27.7 |  30.0 |  13.8
DEC [9] |  30.1 |  18.5 |  35.9 |  38.1 |  19.5
DAC [33] |  52.2 |  23.8 |  47.0 |  52.7 |  27.5
DDC [34] |  52.4 |  - |  48.9 |  57.7 |  -
DCCM [35] |  62.3 |  32.7 |  48.2 |  71.0 |  38.3
IIC [7] |  61.7 |  25.7 |  59.6 |  - |  -
GATCluster [36] |  62.3 |  32.7 |  58.3 |  73.9 |  32.2
PICA [37] |  69.6 |  33.7 |  71.3 |  87.0 |  35.2
DRC [38] |  72.7 |  36.7 |  74.7 |  88.4 |  38.9
CC [12] |  76.6 |  42.6 |  74.7 |  89.5 |  34.2
SACC(our) |  85.1 |  44.3 |  75.9 |  90.5 |  43.7
TABLE IV: The clustering performance (w.r.t. ARI(%)) by different clustering algorithms on the five image datasets. The best score in each column is in bold. Dataset |  CIFAR-10 |  CIFAR-100 |  STL-10 |  ImageNet-10 |  ImageNet-Dogs
---|---|---|---|---|---
K-means [1] |  4.9 |  2.8 |  6.1 |  5.7 |  2.0
SC [3] |  8.5 |  2.2 |  4.8 |  7.6 |  1.3
AC [2] |  6.5 |  3.4 |  14.0 |  6.7 |  2.1
NMF [28] |  3.4 |  2.6 |  4.6 |  6.5 |  1.6
AE [29] |  16.9 |  4.8 |  16.1 |  15.2 |  7.3
DAE [30] |  16.3 |  4.6 |  15.2 |  13.8 |  7.8
DCGAN [31] |  17.6 |  4.5 |  13.9 |  15.7 |  7.8
DeCNN [32] |  17.4 |  3.8 |  16.2 |  14.2 |  7.3
VAE [21] |  16.7 |  4.0 |  14.6 |  16.8 |  7.9
JULE [23] |  13.8 |  3.3 |  16.4 |  13.8 |  2.8
DEC [9] |  16.1 |  5.0 |  18.6 |  20.3 |  7.9
DAC [33] |  30.6 |  8.8 |  25.7 |  30.2 |  11.1
DDC [34] |  32.9 |  - |  26.7 |  34.5 |  -
DCCM [35] |  40.8 |  17.3 |  26.2 |  55.5 |  18.2
IIC [7] |  41.1 |  11.7 |  39.7 |  - |  -
GATCluster [36] |  40.8 |  17.3 |  36.3 |  55.2 |  16.3
PICA [37] |  51.2 |  17.1 |  53.1 |  76.1 |  20.1
DRC [38] |  54.7 |  20.8 |  56.9 |  79.8 |  23.3
CC [12] |  60.6 |  26.7 |  60.6 |  82.5 |  22.5
SACC(our) |  72.4 |  28.2 |  62.6 |  84.3 |  28.5

###  IV-B Baseline Methods

In the experiments, the proposed SACC method is compared with both traditional
clustering methods and deep clustering methods. Specifically, eighteen
baseline clustering methods are compared, including four traditional
clustering methods, namely, $K$-means [1], agglomerative clustering (AC) [2],
spectral clustering (SC) [3], and nonnegative matrix factorization (NMF) [28],
and fourteen deep clustering methods, including autoencoders (AE) [29],
denoising autoencoder (DAE) [30], deep convolutional generative adversarial
networks (DCGAN) [31], deconvolutional networks (DeCNN) [32], variational
auto-encoder (VAE) [21], joint unsupervised learning (JULE) [23], deep
embedded clustering (DEC) [9], deep adaptive clustering (DAC) [33], deep
discriminative clustering (DDC) [34], deep comprehensive correlation mining
(DCCM) [35], invariant information clustering (IIC) [7], Gaussian attention
network for image clustering (GATCluster) [36], partition confidence
maximization (PICA) [37], deep robust clustering (DRC) [38] and contrastive
clustering (CC) [12]. For the CC method, the NMI, ACC, and ARI scores are
reproduced by running the authors’ code [12], while the scores of the other
baseline methods are taken from the corresponding papers.

###  IV-C Results and Analysis

(a) CIFAR-10 (b) CIFAR-100 (c) STL-10 (d) ImageNet-10

(e) ImageNet-Dogs

Figure 4: The confusion matrices on the five image datasets, where the rows
are the ground-truth labels and the columns are the predicted labels by SACC.

(a) CIFAR-10 (b) CIFAR-100 (c) STL-10 (d) ImageNet-10

(e) ImageNet-dogs

Figure 5: The NMI(%) performance of SACC as the number of epochs increases.

(a) CIFAR-10 (b) CIFAR-100 (c) STL-10 (d) ImageNet-10

(e) ImageNet-dogs

Figure 6: The ACC(%) performance of SACC as the number of epochs increases.

(a) CIFAR-10 (b) CIFAR-100 (c) STL-10 (d) ImageNet-10

(e) ImageNet-dogs

Figure 7: The ARI(%) performance of SACC as the number of epochs increases.

(a) 0 epoch (b) 100 epoch (c) 200 epoch (d) 300 epoch

(e) 500 epoch (f) 1000 epoch

Figure 8: The t-SNE visualization of SACC on the CIFAR-10 dataset.

In this section, we compare the proposed SACC method against both traditional
and deep clustering methods on the five benchmark datasets.

The NMI, ACC, and ARI scores of different clustering methods are reported in
Tables II, III, and IV, respectively. From Table II, we can observe that the
deep clustering methods can significantly outperform the traditional (non-
deep) clustering methods, due to the representation learning ability of the
deep neural networks. In terms of the proposed method, SACC achieves the best
NMI score on the five benchmark datasets. Especially, on the CIFAR-10 and
ImageNet-Dogs datasets, SACC achieves NMI(%) scores of 76.5 and 45.5,
respective, where the second best NMI(%) scores are only 68.1 and 40.1,
respectively. On the other three datasets, SACC also obtains better NMI scores
than the other deep clustering methods. Similar advantages can be observed in
Tables III and IV, where our SACC method also achieves the best ACC and ARI
scores on all the five benchmark datasets.

Besides the quantitative evaluation, we further provide visual analysis on the
clustering results of our SACC method. Specifically, Fig. 4 illustrates the
confusion matrices between the true and predicted labels on the five image
datasets. As shown in Fig. 4, clear block-diagonal structures can be observed
in the confusion matrices for the CIFAR-10, STL-10, and ImageNet-10 datasets.
Even for the more challenging datasets of CIFAR-100 and ImageNet-Dogs, we can
still observe the block-diagonal structures, though they are not as clear as
the other three datasets. Notably, even for the challenging datasets like
CIFAR-100 and ImageNet-Dogs, our SACC method can still yield better clustering
performance than the state-of-the-art deep clustering methods (as shown in
Tables II, III, and IV).

###  IV-D Ablation Study

In this section, we conduct the ablation study on the CIFAR-10 dataset to test
the influence of the strong and strong augmentations and that of the two
contrastive projectors in our SACC method.

TABLE V: The NMI(%), ACC(%), and ARI(%) by SACC using different combinations of weak and strong augmentations. Augmentations |  NMI |  ACC |  ARI
---|---|---|---
Weak+Weak |  68.1 |  76.6 |  60.6
Weak+Strong |  72.6 |  82.6 |  67.9
Weak+Weak+Strong |  76.5 |  85.1 |  72.4
TABLE VI: The NMI(%), ACC(%), and ARI(%) by SACC using one or two of its contrastive projectors. Used Projectors |  NMI |  ACC |  ARI
---|---|---|---
With Only Cluster Projector |  67.5 |  73.4 |  58.8
With Only Instance Projector |  69.2 |  75.5 |  61.5
With Both Projectors |  76.5 |  85.1 |  72.4

####  IV-D1 Influence of Strong and Weak Augmentations

In our SACC framework, three augmentation views are utilized, including a
strongly augmented views and two weakly augmented views (as shown in Fig. 2).
In the section, we test the influence of the three augmentation views. Note
that at least two augmentation views should be preserved to make contrastive
learning feasible. As shown in Table V, using a weak augmentation and a strong
augmentation leads to better clustering performance than using two weak
augmentations. Further, the proposed SACC method using all three augmentations
significantly outperforms that variant of using two augmentations (weak+weak
or weak+strong), which demonstrate the substantial benefits brought in by our
network architecture with three augmentation views.

####  IV-D2 Influence of Two Contrastive Projectors

In SACC, we utilize two projectors, namely, the instance projector and the
cluster projector, for the instance-level and cluster-level contrastive
learning, respectively. In the section, we test the influence of these two
projectors by evaluating the performance of SACC with one of the two
projectors removed. Note that in SACC, the clustering result is obtained in
the cluster projector. For the variant with the cluster projector removed, we
obtain the clustering result by performing $K$-means on the feature
representation learned by the instance projector. As shown in Table VI, using
both projectors can lead to significantly better clustering performance
(w.r.t. NMI, ACC, and ARI) than using only one of the two projectors, which
confirm the advantage of the joint instance-level and cluster-level
contrastive learning in our SACC method.

###  IV-E Convergence Analysis

In this section, we evaluate the convergence of the proposed SACC method.
Specifically, the NMI, ACC, and ARI scores of the proposed method are recorded
for each 100 epochs, which are then plotted in Fig. 5, 6 and 7. As shown Fig.
5, 6 and 7, the NMI, ACC, and ARI scores of SACC consistently increase as the
number of epochs grows. On most of the datasets, the proposed SACC method can
reach high-quality clustering results when the number of epochs is greater
than 500. In this paper, the number of epochs is set to 1,000 for all
datasets.

Further, we visualize convergence of SACC by performing t-SNE [42] on the
learned feature representations the learned feature representation on the
CIFAR-10 dataset, which is shown in Fig. 8, where different colors denote
different ground-truth labels. As can be observed in Fig. 8, at the beginning,
the data samples are mostly mixed. After training 100 epochs, many samples in
the same class have been grouped closer. As the learning process proceeds, the
distribution of the data samples reach stability after about 500 epochs, where
the separability of the samples in most classes becomes relatively clear.

##  V Conclusion

In this paper, we propose a novel deep clustering approach termed SACC.
Different from the previous contrastive learning based deep clustering
approaches which typically use some weak augmentations with limited
transformations, our SACC approach is able to jointly leverage strong and weak
augmentations for enhancing the simultaneous contrastive representation
learning and clustering. In particular, we utilize a backbone network with
triply-shared weights for three augmentation views, including a strongly
augmented view and two weakly augmented views. Three views of representations
(for weakly and strongly augmented samples) can be obtained from the backbone
network, which are then fed to two types of projectors, namely, the instance
projector and the cluster projector, so as to enable the instance-level
contrastive learning and the cluster-level contrastive learning, respectively.
Further, the unsupervised training is performed to simultaneously optimize the
backbone and the two projectors in an end-to-end manner, where the final
clustering result can therefore be achieved in the cluster projector.
Extensive experiments are conducted on five benchmark image datasets, which
have confirmed the superior clustering performance of the proposed SACC
approach over the state-of-the-art deep clustering approaches.

## References

  * [1] J. MacQueen _et al._ , “Some methods for classification and analysis of multivariate observations,” in _Berkeley Symposium on Mathematical Statistics and Probability_ , vol. 1, no. 14, 1967, pp. 281–297.
  * [2] K. C. Gowda and G. Krishna, “Agglomerative clustering using the concept of mutual nearest neighbourhood,” _Pattern Recognition_ , vol. 10, no. 2, pp. 105–112, 1978.
  * [3] L. Zelnik-Manor and P. Perona, “Self-tuning spectral clustering,” _Advances in Neural Information Processing Systems_ , vol. 17, 2004.
  * [4] J. R. Hershey, Z. Chen, J. Le Roux, and S. Watanabe, “Deep clustering: Discriminative embeddings for segmentation and separation,” in _IEEE International Conference on Acoustics, Speech and Signal Processing_ , 2016, pp. 31–35.
  * [5] X. Guo, L. Gao, X. Liu, and J. Yin, “Improved deep embedded clustering with local structure preservation.” in _International Joint Conference on Artificial Intelligence_ , 2017, pp. 1753–1759.
  * [6] M. Caron, P. Bojanowski, A. Joulin, and M. Douze, “Deep clustering for unsupervised learning of visual features,” in _European Conference on Computer Vision_ , 2018, pp. 132–149.
  * [7] X. Ji, J. F. Henriques, and A. Vedaldi, “Invariant information clustering for unsupervised image classification and segmentation,” in _IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 9865–9874.
  * [8] T. Wang, W. W. Y. Ng, J. Li, Q. Wu, S. Zhang, C. Nugent, and C. Shewell, “A deep clustering via automatic feature embedded learning for human activity recognition,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 32, no. 1, pp. 210–223, 2022.
  * [9] J. Xie, R. Girshick, and A. Farhadi, “Unsupervised deep embedding for clustering analysis,” in _International Conference on Machine Learning_ , 2016, pp. 478–487.
  * [10] W. van Gansbeke, S. Vandenhende, S. Georgoulis, M. Proesmans, and L. Van Gool, “SCAN: Learning to classify images without labels,” in _European Conference on Computer Vision_ , 2020, pp. 268–285.
  * [11] Z. Dang, C. Deng, X. Yang, K. Wei, and H. Huang, “Nearest neighbor matching for deep clustering,” in _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 13 693–13 702.
  * [12] Y. Li, P. Hu, Z. Liu, D. Peng, J. T. Zhou, and X. Peng, “Contrastive clustering,” in _AAAI Conference on Artificial Intelligence_ , 2021.
  * [13] X. Wang and G.-J. Qi, “Contrastive learning with stronger augmentations,” _arXiv preprint arXiv:2104.07713_ , 2021.
  * [14] B. Yang, X. Fu, N. D. Sidiropoulos, and M. Hong, “Towards k-means-friendly spaces: Simultaneous deep learning and clustering,” in _International Conference on Machine Learning_ , 2017, pp. 3861–3870.
  * [15] P. Ji, T. Zhang, H. Li, M. Salzmann, and I. Reid, “Deep subspace clustering networks,” _Advances in Neural Information Processing Systems_ , vol. 30, 2017.
  * [16] K. G. Dizaji, A. Herandi, C. Deng, W. Cai, and H. Huang, “Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization,” in _IEEE International Conference on Computer Vision_ , 2017, pp. 5736–5745.
  * [17] Z. Jiang, Y. Zheng, H. Tan, B. Tang, and H. Zhou, “Variational deep embedding: An unsupervised and generative approach to clustering,” _arXiv preprint arXiv:1611.05148_ , 2016.
  * [18] N. Dilokthanakul, P. A. Mediano, M. Garnelo, M. C. Lee, H. Salimbeni, K. Arulkumaran, and M. Shanahan, “Deep unsupervised clustering with gaussian mixture variational autoencoders,” _arXiv preprint arXiv:1611.02648_ , 2016\.
  * [19] J. T. Springenberg, “Unsupervised and semi-supervised learning with categorical generative adversarial networks,” _arXiv preprint arXiv:1511.06390_ , 2015.
  * [20] X. Dong, L. Liu, L. Zhu, Z. Cheng, and H. Zhang, “Unsupervised deep k-means hashing for efficient image retrieval and clustering,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 31, no. 8, pp. 3266–3277, 2021.
  * [21] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” _arXiv preprint arXiv:1312.6114_ , 2013.
  * [22] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” _Advances in Neural Information Processing Systems_ , vol. 27, 2014.
  * [23] J. Yang, D. Parikh, and D. Batra, “Joint unsupervised learning of deep representations and image clusters,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 5147–5156.
  * [24] X. Guo, X. Liu, E. Zhu, X. Zhu, M. Li, X. Xu, and J. Yin, “Adaptive self-paced deep clustering with data augmentation,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 32, no. 9, pp. 1680–1693, 2019.
  * [25] X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, “InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets,” _Advances in Neural Information Processing Systems_ , vol. 29, 2016.
  * [26] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in _International Conference on Machine Learning_ , 2020, pp. 1597–1607.
  * [27] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 770–778.
  * [28] D. Cai, X. He, X. Wang, H. Bao, and J. Han, “Locality preserving nonnegative matrix factorization,” in _International Joint Conference on Artificial Intelligence_ , 2009.
  * [29] Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, “Greedy layer-wise training of deep networks,” _Advances in Neural Information Processing Systems_ , vol. 19, 2006.
  * [30] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, and L. Bottou, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion.” _Journal of Machine Learning Research_ , vol. 11, no. 12, 2010.
  * [31] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” _arXiv preprint arXiv:1511.06434_ , 2015.
  * [32] M. D. Zeiler, D. Krishnan, G. W. Taylor, and R. Fergus, “Deconvolutional networks,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2010, pp. 2528–2535.
  * [33] J. Chang, L. Wang, G. Meng, S. Xiang, and C. Pan, “Deep adaptive image clustering,” in _IEEE International Conference on Computer Vision_ , 2017, pp. 5879–5887.
  * [34] J. Chang, Y. Guo, L. Wang, G. Meng, S. Xiang, and C. Pan, “Deep discriminative clustering analysis,” _arXiv preprint arXiv:1905.01681_ , 2019.
  * [35] J. Wu, K. Long, F. Wang, C. Qian, C. Li, Z. Lin, and H. Zha, “Deep comprehensive correlation mining for image clustering,” in _IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 8150–8159.
  * [36] C. Niu, J. Zhang, G. Wang, and J. Liang, “GATCluster: Self-supervised gaussian-attention network for image clustering,” in _European Conference on Computer Vision_ , 2020, pp. 735–751.
  * [37] J. Huang, S. Gong, and X. Zhu, “Deep semantic clustering by partition confidence maximisation,” in _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 8849–8858.
  * [38] H. Zhong, C. Chen, Z. Jin, and X.-S. Hua, “Deep robust clustering by contrastive learning,” _arXiv preprint arXiv:2008.03030_ , 2020.
  * [39] A. Krizhevsky, G. Hinton _et al._ , “Learning multiple layers of features from tiny images,” 2009.
  * [40] A. Coates, A. Ng, and H. Lee, “An analysis of single-layer networks in unsupervised feature learning,” in _International Conference on Artificial Intelligence and Statistics_ , 2011, pp. 215–223.
  * [41] D. Huang, C.-D. Wang, J.-S. Wu, J.-H. Lai, and C.-K. Kwoh, “Ultra-scalable spectral clustering and ensemble clustering,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 32, no. 6, pp. 1212–1226, 2020.
  * [42] L. Van der Maaten and G. Hinton, “Visualizing data using t-SNE,” _Journal of Machine Learning Research_ , vol. 9, no. 11, 2008.

|  Xiaozhi Deng received the B.S. degree in 2020 from South China Agricultural
University, Guangzhou, China, where he is currently pursuing the master degree
in computer science with the College of Mathematics and Informatics. His
research interests include clustering analysis and deep learning.
---|---
|  Dong Huang received the B.S. degree in computer science in 2009 from South
China University of Technology, Guangzhou, China. He received the M.Sc. degree
in computer science in 2011 and the Ph.D. degree in computer science in 2015,
both from Sun Yat-sen University, Guangzhou, China. He joined South China
Agricultural University in 2015, where he is currently an Associate Professor
with the College of Mathematics and Informatics. From July 2017 to July 2018,
he was a visiting fellow with the School of Computer Science and Engineering,
Nanyang Technological University, Singapore. His research interests include
data mining and machine learning. He has published more than 50 papers in
international journals and conferences, such as IEEE TKDE, IEEE TCYB, IEEE
TSMC-S, ACM TKDD, SIGKDD, AAAI, and ICDM. He was the recipient of the 2020 ACM
Guangzhou Rising Star Award.
---|---
|  Ding-Hua Chen received the B.S. degree in computer science and technology
in 2019 from South China Agricultural University, Guangzhou, China, where he
is currently pursuing the master degree in computer science with the College
of Mathematics and Informatics. His current research interests include deep
learning and clustering analysis.
---|---
|  Chang-Dong Wang received the B.S. degree in applied mathematics in 2008,
the M.Sc. degree in computer science in 2010, and the Ph.D. degree in computer
science in 2013, all from Sun Yat-sen University, Guangzhou, China. He was a
visiting student at the University of Illinois at Chicago from January 2012 to
November 2012. He is currently an Associate Professor with the School of Data
and Computer Science, Sun Yat-sen University, Guangzhou, China. His current
research interests include machine learning and data mining. He has published
more than 100 scientific papers in international journals and conferences such
as IEEE TPAMI, IEEE TKDE, IEEE TNNLS, IEEE TSMC-C, ACM TKDD, Pattern
Recognition, SIGKDD, ICDM and SDM. His ICDM 2010 paper won the Honorable
Mention for Best Research Paper Award. He was awarded 2015 Chinese Association
for Artificial Intelligence (CAAI) Outstanding Dissertation.
---|---
|  Jian-Huang Lai received the M.Sc. degree in applied mathematics in 1989 and
the Ph.D. degree in mathematics in 1999 from Sun Yat-sen University, China. He
joined Sun Yat-sen University in 1989 as an Assistant Professor, where he is
currently a Professor with the School of Data and Computer Science. His
current research interests include the areas of digital image processing,
pattern recognition, multimedia communication, wavelet and its applications.
He has published more than 200 scientific papers in the international journals
and conferences on image processing and pattern recognition, such as IEEE
TPAMI, IEEE TKDE, IEEE TNN, IEEE TIP, IEEE TSMC-B, Pattern Recognition, ICCV,
CVPR, IJCAI, ICDM and SDM. Prof. Lai serves as a Standing Member of the Image
and Graphics Association of China, and also serves as a Standing Director of
the Image and Graphics Association of Guangdong.
---|---