#  Fermat’s Last Theorem, Schur’s Theorem (in Ramsey Theory), and the
Infinitude of the Primes

William Gasarch<EMAIL_ADDRESS>Univ. of MD at College Park, Dept of Comp.
Sci.

###### Abstract

Alpoge and Granville (separately) gave novel proofs that the primes are
infinite that use Ramsey Theory. In particular, they use Van der Waerden’s
Theorem and some number theory. We prove the primes are infinite using an
easier theorem from Ramsey Theory, namely Schur’s Theorem, and some number
theory (Elsholtz independently obtained the same proof that the primes were
infinite). In particular, we use the $n=3$ case of Fermat’s last theorem. We
also apply our method to show other domains have an infinite number of
irreducibles.

###### keywords:

Primes; Ramsey Theory; Schur’s Theorem; Fermat’s Last Theorem; Irreducibles;
Colorings

###### MSC:

[2020] 11A41, 05D10

††journal: Discrete Mathematics

##  1 Introduction

###### Notation 1.1

We take $\mathbb{N}$ to be $\\{0,1,2,3,\ldots\\}$.

###### Def 1.2

Let $a\in\mathbb{N}$ and ${\rm D}$ be a domain.

  1. 1.

${\rm FLT}_{a}$ holds in ${\rm D}$ means that the equation

$x^{a}+y^{a}=z^{a}$

has no solution in ${\rm D}-\\{0\\}$.

  2. 2.

${\rm FLT}_{a}$ means ${\rm FLT}_{a}$ holds in $\mathbb{Z}$.

In 1770 Euler proved ${\rm FLT}_{3}$ (see the texts of Ireland & Rosen [1] or
Hardy & Wright [2] for a modern treatment of Euler’s proof). In 1916 Schur
proved a theorem in Ramsey Theory (which we will state later) that is referred
to as Schur’s Theorem (in Ramsey Theory) (see the texts of Graham-Rothschild-
Spencer [3] or Landman & Robertson [4] for a modern treatment of Schur’s
proof). In this paper we use these two theorems to prove the primes are
infinite. (Elsholtz [5] independently obtained the same proof that the primes
were infinite.) While there are of course easier proofs, we think it is of
interest that it can be derived from Schur’s Theorem and ${\rm FLT}_{3}$.

Alpoge [6] proved the primes were infinite using elementary number theory and
Van der Warden’s theorem. Granville [7] proved that the primes were infinite
from the fact that that there can never be four squares in arithmetic
progression (attributed to Fermat) and Van der Warden’s theorem. Our proof
compares to their proofs as follows:

  * 1.

Our proof uses easier Ramsey Theory then Alpoge’s or Granville’s proof.

  * 2.

Our proof uses harder number theory than Alpoge’s proof.

  * 3.

Our proof uses about the same level of number theory as Granville’s proof.

  * 4.

We prove a general theorem that allows us to show other domains have an
infinite number of irreducibles.

In Section 2 we present Schur’s Theorem and definitions from number theory. In
Section 3 we present a condition on an integral domains ${\rm D}$ that implies
${\rm D}$ has an infinite number of irreducibles. That condition easily
applies to $\mathbb{Z}$. Hence we obtain that $\mathbb{Z}$ has an infinite
number of irreducibles. Since in $\mathbb{Z}$, every irreducible is a prime,
we also get that there are an infinite number of primes. In Section 5 we use
our results to show that, for all $d\in\mathbb{N}$, $\mathbb{Z}[\sqrt{-d}]$
has an infinite number of irreducibles. In Section 6 we use our results,
together with a widely believed conjecture, to show that many domains have an
infinite number of irreducibles. In Section 7 we present an open problem.

##  2 Preliminaries

The following is Schur’s Theorem (from Ramsey theory). It can be proven from
Ramsey’s Theorem.

###### Lemma 2.1

For all $c$, for all $c$-colorings $COL:\mathbb{N}-\\{0\\}\rightarrow[c]$,
there exist $x,y,z\in\mathbb{N}-\\{0\\}$ with $x+y=z$ such that

$COL(x)=COL(y)=COL(z).$

The following definitions are standard.

###### Def 2.2

Let ${\rm D}$ be an integral domain.

  1. 1.

A unit is a $u\in{\rm D}$ such that there exists $v\in{\rm D}$ with $uv=1$. We
let ${\rm U}$ be the set of units if the domain is understood.

  2. 2.

An irreducible is a $p\in{\rm D}-{\rm U}$ such that if $p=ab$ then either
$a\in{\rm U}$ or $b\in{\rm U}$. We let ${\rm I}$ be the set of irreducibles if
the domain is understood.

  3. 3.

A prime is a $p\in{\rm D}$ such that if $p$ divides $ab$ then either $p$
divides $a$ or $p$ divides $b$. In any integral domain all primes are
irreducible. There are integral domains with irreducibles that are not primes.
The set

$\\{a+b\sqrt{-5}{\ \colon\ }a,b\in\mathbb{Z}\\}$ is one such example: (a) The
element 2 is irreducible, yet (b) 2 is not prime since 2 divides
$(1+\sqrt{-5})(1-\sqrt{-5})=6$ but 2 does not divide either $1+\sqrt{-5}$ or
$1+\sqrt{-5}$.

  4. 4.

We impose an equivalence relation on ${\rm I}$: $p$ and $q$ are equivalent if
there exists $u\in{\rm U}$ such that $p=uq$. We say ${\rm I}$ is infinite up
to units if the number of equivalence classes is infinite. In this paper
infinite will mean infinite up to units.

  5. 5.

An Atomic Integral Domain is an integral domain such that every element of
${\rm D}-({\rm U}\cup\\{0\\})$ can be written (not necessarily uniquely) as
$p_{1}^{x_{1}}\cdots p_{m}^{x_{m}}$ where the $p_{i}$’s are irreducible. The
domains $\mathbb{Z}$ and $\mathbb{Z}[\sqrt{d}]$ are known to be atomic by
using norms. The set of algebraic integers (complex numbers that satisfy monic
polynomials over $\mathbb{Z}[x]$) is an integral domain that is not atomic for
a funny reason: there are no irreducibles. If $a$ is a nonzero nonunit
algebraic integer then $\sqrt{a}$ is a nonzero nonunit algebraic integer, and
$a=\sqrt{a}\times\sqrt{a}$, so $a$ is not irreducible.

##  3 A Condition for a Domain to Have an Infinite Number of Irreducibles

Theorem 3.1 says that if an integral domain ${\rm D}$ has a finite number of
irreducibles then an equation similar to that in FLT has a solution. We will
use Theorem 3.1 to derive conditions on ${\rm D}$ that imply it has an
infinite number of irreducibles.

The coloring in the proof of Theorem 3.1 is similar to the one used by Alpoge
[6], Granville [7], and Elsholtz [5].

###### Theorem 3.1

Let ${\rm D}$ be an atomic integral domain that contains $\mathbb{N}$. Assume
there exists an $n\geq 2$ such that the following equation has no solution:

$u_{x}X^{n}+u_{y}Y^{n}=u_{z}Z^{n}$

where $u_{x},u_{y},u_{z}\in{\rm U}$ and $X,Y,Z\in{\rm D}-\\{0\\}$. Then ${\rm
D}$ has an infinite number of irreducibles.

Proof:  Assume the premise is true. Assume, by way of contradiction, that
${\rm I}$ is finite. Let ${\rm I}=\\{p_{1},\ldots,p_{m}\\}$ be formed by
taking an irreducible from each equivalence class.

Since ${\rm D}$ is atomic, every $x\in{\rm D}-\\{0\\}$ can be written as
$up_{1}^{x_{1}}\cdots p_{m}^{x_{m}}$ where $u\in{\rm U}$ and
$x_{1},\ldots,x_{m}\in\mathbb{N}$ (an $x_{i}$ can be 0). This need not be
unique; however, for the sake of definiteness, we will take
$(x_{1},\ldots,x_{m})$ to be the lexicographically least tuple.

Recall that $\mathbb{N}\subseteq{\rm D}$. Let $n$ be as in the premise. We
define a coloring ${\rm COL}$ of $\mathbb{N}-\\{0\\}$ as follows: Color
$x=up_{1}^{x_{1}}\cdots p_{m}^{x_{m}}$ by the vector

$(x_{1}\bmod n,\ldots,x_{m}\bmod n).$

There are $n^{m}$ colors, which is finite. By Lemma 2.1 there exists
$(x,y,z)$, and a color $(e_{1},\ldots,e_{m})$, such that

${\rm COL}(x)={\rm COL}(y)={\rm COL}(z)=(e_{1},\ldots,e_{m}).$

and

$x+y=z.$

We now reason about $x$ but the same logic applies to $y,z$. Note that there
exist $u\in{\rm U}$ and $k_{1},\ldots,k_{m}\in\mathbb{N}$ such that

$x=up_{1}^{k_{1}n+e_{1}}\cdots p_{m}^{k_{m}n+e_{m}}$

hence

$xp_{1}^{n-e_{1}}\cdots p_{m}^{n-e_{m}}=up_{1}^{(k_{1}+1)n}\cdots
p_{m}^{(k_{m}+1)n}=uX^{n}$

where $X=p_{1}^{(k_{1}+1)}\cdots p_{m}^{(k_{m}+1)}\in{\rm D}$.

Since the same logic applies to $y,z$ we have that there exist $X,Y,Z\in{\rm
D}$ and $u_{x},u_{y},u_{z}\in{\rm U}$ such that

$xp_{1}^{n-e_{1}}\cdots p_{m}^{n-e_{m}}=u_{x}X^{n}$

$yp_{1}^{n-e_{1}}\cdots p_{m}^{n-e_{m}}=u_{y}Y^{n}$

$zp_{1}^{n-e_{1}}\cdots p_{m}^{n-e_{m}}=u_{z}Z^{n}$.

Note that the following hold:

  * 1.

$u_{x}X^{n}+u_{y}Y^{n}=u_{z}Z^{n}$.

  * 2.

$u_{x},u_{y},u_{z}\in{\rm U}$.

  * 3.

$X,Y,Z\in{\rm D}-\\{0\\}$.

This contradicts the premise of the theorem.

###### Theorem 3.2

Let ${\rm D}$ be an atomic integral domain.

  1. 1.

Assume that there is an $n_{0}\in\mathbb{N}$, $n_{0}\geq 2$, such that the
following hold:

     * (a)

For all $u\in{\rm U}$, there is $v\in{\rm D}$ such that $v^{n_{0}}=u$.

     * (b)

${\rm FLT}_{n_{0}}$ holds for ${\rm D}$.

Then ${\rm D}$ has an infinite number of irreducibles.

  2. 2.

Assume that there is an $n_{0}\in\mathbb{N}$, $n_{0}\geq 2$, such that the
following hold:

     * (a)

For all $u\in{\rm U}$, $u^{n_{0}}=u$.

     * (b)

${\rm FLT}_{n_{0}}$ holds for ${\rm D}$.

Then ${\rm D}$ has an infinite number of irreducibles. (This follows from Part
1.)

Proof:

Assume, by way of contradiction, that ${\rm D}$ has a finite number of
irreducibles. By Theorem 3.1, for all $n\in\mathbb{N}$ there exist
$u_{x},u_{y},u_{z}\in{\rm U}$ and $X,Y,Z\in{\rm D}-\\{0\\}$ such that the
following holds:

$u_{x}X^{n}+u_{y}Y^{n}=u_{z}Z^{n}.$

Take $n=n_{0}$. By the first premise, there exists $v_{x},v_{y},v_{z}$ such
that $v_{x}^{n_{0}}=u_{x}$, $v_{y}^{n_{0}}=u_{y}$, $v_{z}^{n_{0}}=u_{z}$.
Hence

$(v_{x}X)^{n_{0}}+(v_{y}Y)^{n_{0}}=(v_{z}Z)^{n_{0}}.$

By the second premise, that ${\rm FLT}_{n_{0}}$ holds for ${\rm D}$, this is a
contradiction.

###### Corollary 3.3

  1. 1.

$\mathbb{Z}$ has an infinite number of irreducibles.

  2. 2.

$\mathbb{Z}$ has an infinite number of primes.

Proof:

1) Let $n=3$. The only units in $\mathbb{Z}$ are $\\{-1,1\\}$. Note that (a)
all $u\in\\{-1,1\\}$ satisfy $u^{3}=u$, and (b) ${\rm FLT}_{3}$ holds for
$\mathbb{Z}$. Hence, by Theorem 3.2.2, $\mathbb{Z}$ has an infinite number of
irreducibles.

2) In $\mathbb{Z}$ all irreducibles are primes. Hence $\mathbb{Z}$ has an
infinite number of primes.

##  4 A Sanity Check

As a sanity check on Theorem 3.1 we look at two integral domains that have a
finite number of irreducibles.

  1. 1.

Consider $\mathbb{Q}$. Note that ${\rm U}=\mathbb{Q}-\\{0\\}$, so there are no
irreducibles. Fix $n\geq 3$. The premise of Theorem 3.1 does not hold. For all
$n$ there is a solution to

$u_{x}X^{n}+u_{y}Y^{n}=u_{z}Z^{n}$

with $u_{x},u_{y},u_{z}\in{\rm U}$, namely $u_{x}=u_{y}=\frac{1}{2}$,
$u_{z}=1$, $X=Y=Z=1$.

  2. 2.

In this example the variables $a,b,c,d$ are always in $\mathbb{Z}$. Let ${\rm
D}$ be the domain with set

$\biggl{\\{}\frac{a}{b}{\ \colon\ }b\equiv 1\pmod{2}\biggr{\\}}.$

Clearly

${\rm U}=\biggl{\\{}\frac{a}{b}{\ \colon\ }a,b\equiv 1\pmod{2}\biggl{\\}}.$

We show that ${\rm I}=\\{2\\}$. Recall that what we really mean is that all
irreducibles are of the form $2u$ where $u\in{\rm U}$.

The nonzero elements that are not in ${\rm U}$ are in one of the following
sets.

    1. (a)

$\\{\frac{2c}{b}{\ \colon\ }c\equiv 1\pmod{2},\ \ b\equiv 1\pmod{2}\\}.$ Since
$\frac{c}{b}\in{\rm U}$, these elements are irreducibles in the same
equivalence class as 2.

    2. (b)

$\\{\frac{2^{d}c}{b}{\ \colon\ }d\geq 2,c\equiv 1\pmod{2},\ \ b\equiv
1\pmod{2}\\}.$ These elements are reducible since
$\frac{2^{d}c}{b}=2\times\frac{2^{d-1}c}{b}$ and, since $d\geq 2$,
$\frac{2^{d-1}c}{b}$ is not a unit.

We must now see how ${\rm D}$ violates the premise of Theorem 3.1. We need to
show that, for all $n\in\mathbb{N}$, there is a solution to

$u_{x}X^{n}+u_{y}Y^{n}=u_{z}Z^{n}$

with $u_{x},u_{y},u_{z}\in{\rm U}$.

For $n=1$ we can take $u_{x}=u_{y}=u_{z}=X=Y=1$ and $Z=2$. For $n\geq 2$ we
can take $u_{x}=2^{n-1}-1$, $u_{y}=2^{n-1}+1$, $X=Y=1$, $Z=2$.

##  5 The Domain $\mathbb{Z}[\sqrt{-d}]$ Has an Infinite Number of
Irreducibles

###### Lemma 5.1

Let $d\in\mathbb{N}$.

  1. 1.

If $d=1$ then the only units in $\mathbb{Z}[\sqrt{-d}]$ are $\\{-1,1,-i,i\\}$

  2. 2.

If $d\geq 2$ then the only units in $\mathbb{Z}[\sqrt{-d}]$ are $\\{-1,1\\}$

  3. 3.

If $d\in\mathbb{N}$ and $u$ is a unit of $\mathbb{Z}[\sqrt{-d}]$ then
$u^{9}=u$ (This follows from Part 1 and 2. It is also the case that $u^{5}=u$;
however, 9 is useful to us and, alas, 5 is not)

Proof:

Let $N$ be the standard norm

$N(a+b\sqrt{-d})=(a+b\sqrt{-d})(a-b\sqrt{-d})=a^{2}+b^{2}d.$

It is well known and easy to verify that $N(xy)=N(x)N(y)$.

If $a_{1}+b_{1}\sqrt{-d}$ is a unit then there exist $a_{2},b_{2}$ such that

$(a_{1}+b_{1}\sqrt{-d})(a_{2}+b_{2}\sqrt{-d})=1$

Take the norm of both sides to get

$(a_{1}^{2}+b_{1}^{2}d)(a_{2}^{2}+b_{2}^{2}d)=1$

Since squares are positive we have that $a_{1}^{2}+b_{1}^{2}d=1$.

If $d=1$ then we have $a_{1}^{2}+b_{1}^{2}=1$, so $(a_{1},b_{1})$ is either
$(1,0)$, $(-1,0)$, $(0,1)$, or $(0,-1)$. This yields units $\\{-1,1,-i,i\\}$

If $d\geq 2$ then $b_{1}=0$ so the only units are $-1,1$.

Aigner [8] proved the following (see also Ribenbiom [9]).

###### Lemma 5.2

For all $d\in\mathbb{Z}$, ${\rm FLT}_{6}$ and ${\rm FLT}_{9}$ hold in
$\mathbb{Q}(\sqrt{-d})$ and hence in $\mathbb{Z}[\sqrt{-d}]$. (We will only
use ${\rm FLT}_{9}$.)

Note The following counterexamples show why Lemma 5.2 does not work for ${\rm
FLT}_{3}$, ${\rm FLT}_{4}$, or ${\rm FLT}_{6k\pm 1}$. As far as we know it is
an open problem as to whether Lemma 5.2 is true for 8.

  * 1.

In $\mathbb{Q}(\sqrt{2})$: $(18+17\sqrt{2})^{3}+(18-17\sqrt{2})^{3}=42^{3}.$

  * 2.

In $\mathbb{Q}(\sqrt{-7})$: $(1+\sqrt{-7})^{4}+(1-\sqrt{-7})^{4}=2^{4}.$

  * 3.

In $\mathbb{Q}(\sqrt{-3})$: $(1+\sqrt{-3})^{6k\pm 1}+(1-\sqrt{-3})^{6k\pm
1}=2^{6k\pm 1}.$

###### Theorem 5.3

Let $d\geq 1$. Then there are an infinite number of irreducibles in
$\mathbb{Z}[\sqrt{-d}]$.

Proof:  Let ${\rm D}=\mathbb{Z}[\sqrt{-d}]$. One can show that ${\rm D}$ is
atomic using norms.

Let $n_{0}=9$. By Lemma 5.1, for all $u\in{\rm U}$, $u^{n_{0}}=u$. By Lemma
5.2 ${\rm FLT}_{n_{0}}$ holds for ${\rm D}$. By Theorem 3.2.2 with $n_{0}=9$,
${\rm D}$ has an infinite number of irreducibles.

##  6 Conjecturally, Some ${\rm D}$ Have an Infinite Number of Irreducibles

Debarre-Klassen [10] stated the following conjecture:

###### Conjecture 6.1

Let ${\rm K}$ be a number field of degree $d$ over $\mathbb{Q}$. Let $n\geq
d+2$. Then ${\rm FLT}_{n}$ holds for ${\rm K}$.

###### Theorem 6.2

Assume Conjecture 6.1 is true. Let ${\rm K}$ be a number field of finite
degree over $\mathbb{Q}$. Let ${\rm D}$ be an atomic subdomain of ${\rm K}$
with a finite number of units. Then ${\rm D}$ has an infinite number of
irreducibles.

Proof:  Let ${\rm K}$ and ${\rm D}$ be as in the premise.

Since ${\rm D}$ has a finite number of units, for each unit $u$, there exists
$n_{u}$ such that $u^{n_{u}}=1$. Let $n_{U}$ be the lcm of all the $n_{u}$.
Note that, for all units $u$, $u^{n_{U}}=1$. Hence, for all $n\equiv
1\pmod{n_{U}}$, $u^{n}=u$.

Let $n_{0}$ be such that $n_{0}\equiv 1\pmod{n_{U}}$ and $n_{0}\geq d+2$. Then
(1) ${\rm FLT}_{n_{0}}$ holds in ${\rm D}$, and (2) for all $u\in{\rm U}$,
$u^{n_{0}}=u$. By Theorem 3.2.2, ${\rm D}$ has an infinite number of
irreducibles.

##  7 Open Problem

Find other domains to apply Theorem 3.1 to. This might involve proving, for
fixed $n$, variants of ${\rm FLT}_{n}$ that allow units as coefficients.

##  8 Acknowledgments

I thank Nathan Cho, Emily Kaplitz, Issac Mammel, David Marcus, Adam Melrod,
Yuang Shen, Larry Washington, and Zan Xu for proofreading and commentary. We
thank the referees for insightful comments and references that improved both
the readability and correctness of this paper.

## References

  * [1] K. Ireland, M. Rosen, A classical introduction to modern number theory, Springer-Verlag, New York, Heidelberg, Berlin, 1982.
  * [2] G. Hardy, E. Wright, An introduction to the theory of numbers, Clarendon Press, Oxford, 1979, fifth Edition. The first edition was in 1938.
  * [3] R. Graham, B. Rothschild, J. Spencer, Ramsey Theory, Wiley, New York, 1990.
  * [4] B. Landman, A. Robertson, Ramsey Theory on the integers, AMS, Providence, 2004\.
  * [5] C. Elsholtz, Fermat’s last theorem implies Euclid’s infinitude of primes, American Mathematics Monthly 128 (3) (2021) 250–257,
https://arxiv.org/abs/2009.06722.

  * [6] L. Alpoge, Van der waerden and the primes, American Mathematical Monthly 122 (2015) 784–785, https://www.jstor.org/stable/10.4169/amer.math.monthly.122.8.784.
  * [7] A. Granville, Squares in arithmetic progression and infinitly many primes, American Mathematical Monthly 124 (2017) 951–954.
  * [8] A. Aigner, Die unmoglichetin von $x^{6}+y^{6}=z^{6}$ and $x^{9}+y^{9}=z^{9}$ in quadratischen Korpern, Monatsh. f Math. (1957) 147–150.
  * [9] P. Ribenbiom, 13 lectures on Fermat’s last theorem, Springer-Verlag, New York, 1979, http://staff.math.su.se/shapiro/ProblemSolving/13%20Lectures%20on%20Fermat’s%20Last%20Theorem.pdf.
  * [10] Debarre, Klasen, Points of low degree on smooth plane curves, J. Reine Angew. Math. 446 (1994) 81–87.


11institutetext: Department of Information Systems, University of Maryland,
Baltimore County, Baltimore, MD, United States
11email: {omarfaruque, fnji1, mcham2<EMAIL_ADDRESS>
https://bdal.umbc.edu 22institutetext: Climate Science Section, Lawrence
Livermore National Laboratory, Livermore, CA, United States
22email<EMAIL_ADDRESS>
# Deep Spatiotemporal Clustering: A Temporal Clustering Approach for Multi-
dimensional Climate Data

Omar Faruque 11 Francis Ndikum Nji 11 Mostafa Cham 11 Rohan Mandar Salvi 11
Xue Zheng 22 Jianwu Wang 11 0000-0002-9933-1170

###### Abstract

Clustering high-dimensional spatiotemporal data using an unsupervised approach
is a challenging problem for many data-driven applications. Existing state-of-
the-art methods for unsupervised clustering use different similarity and
distance functions but focus on either spatial or temporal features of the
data. Concentrating on joint deep representation learning of spatial and
temporal features, we propose Deep Spatiotemporal Clustering (DSC), a novel
algorithm for the temporal clustering of high-dimensional spatiotemporal data
using an unsupervised deep learning method. Inspired by the U-net
architecture, DSC utilizes an autoencoder integrating CNN-RNN layers to learn
latent representations of the spatiotemporal data. DSC also includes a unique
layer for cluster assignment on latent representations that uses the Student’s
t-distribution. By optimizing the clustering loss and data reconstruction loss
simultaneously, the algorithm gradually improves clustering assignments and
the nonlinear mapping between low-dimensional latent feature space and high-
dimensional original data space. A multivariate spatiotemporal climate dataset
is used to evaluate the efficacy of the proposed method. Our extensive
experiments show our approach outperforms both conventional and deep learning-
based unsupervised clustering algorithms. Additionally, we compared the
proposed model with its various variants (CNN encoder, CNN autoencoder, CNN-
RNN encoder, CNN-RNN autoencoder, etc.) to get insight into using both the CNN
and RNN layers in the autoencoder, and our proposed technique outperforms
these variants in terms of clustering results.

###### Keywords:

Temporal clustering Multivariate spatiotemporal climate data Deep neural
network U-net.

##  1 Introduction

Spatiotemporal data is commonly available in many disciplines such as Earth
sciences, atmospheric science, and environmental science. Such data is often
generated by monitoring a certain area over a period of time, which results in
datasets in four dimensions (4D): time, longitude, latitude, and measured
variables such as temperature and humidity. One important way to study such
spatiotemporal data is to categorize the records into smaller groups by
conducting unsupervised data clustering along the temporal dimension.

There are several challenges in clustering such 4D spatiotemporal data. First,
most traditional clustering algorithms like $k$-means [b1] only work on 2D
tabular data and face challenges to work with 4D spatiotemporal data.
Converting 4D spatiotemporal data directly into 2D tabular data will not only
end up with very high dimensional data but also lose spatial and temporal
patterns in the original data. Second, common dimension reduction approaches
such as PCA [b7] can reduce data dimensionality before applying clustering
algorithms, but such efforts [b7, b8] fail to preserve the dataset’s nonlinear
relationship, which leads to subpar clustering accuracy. Third, recently
developed deep learning-based clustering algorithms [b10, b11, b12, b13] are
able to learn nonlinear characteristics of the dataset. But these techniques
only focused on the spatial or temporal features of the dataset, not
spatiotemporal features jointly.

To address the above challenges of high-dimensional spatiotemporal clustering,
we propose a novel spatiotemporal autoencoder model drawing inspiration from
the recent success of the U-net architecture [b19] in representation learning.
Our model is applied to a popular climate data called ECMWF ERA-5 global
reanalysis product [b21] to evaluate its performance. The proposed model
nonlinearly maps the input dataset with the lower dimensional hidden feature
space. To make this latent feature generation more robust, we evaluated both
the spatial and temporal properties of the dataset by combining CNN and LSTM
layers in the encoder module. A custom clustering layer is applied to the
latent features to generate clustering results. The clustering layer uses the
inherent logic of the Student’s t-distribution and iteratively improves the
result. At the same time, the decoder module adjusts its weights to reduce the
disparity between the input and reconstructed data while learning to
reconstruct the high-dimensional input data from lower-dimensional latent
features. From our experimental analysis, it is evident that the proposed
model achieved significant improvement in cluster accuracy and holds the
ability to capture the generic properties of different time series. In
summary, our contributions are i) end-to-end learning of both spatial and
temporal features in the same model, ii) significant improvement in clustering
accuracy, and iii) iterative joint optimization of latent features and
clustering assignment.

The remainder of the paper is structured as follows. Section 2 explains the
background and definition of our clustering task. State-of-the-art related
works are summarized in Section 3. Section 4 gives a detailed description of
the proposed procedure. The experimental details of the proposed model are
provided in Section 5 along with the results and an ablation study of the
proposed model. Finally, we conclude and summarize in Section
LABEL:conclusion.

##  2 Background and Problem Definition

###  2.1 Clustering Multi-dimensional Climate Data

Earth’s climate system is a highly complicated and interconnected global
system formed with a large number of dynamic components such as global
temperature, ocean temperature, arctic sea ice, precipitation, wind pattern,
pressure, aerosol, cloud, etc. Climate change is one of the most threatening
issues because it has diverse effects on the global ecosystem and will make
the weather more hazardous [b1]. Due to its high importance, many researchers
focused on studying the interactions of constituent components of the climate
system and the changes caused by each other. In Earth’s climate system, one of
the significant components is the air-sea-cloud and their inherent
interactions. The dynamic, thermodynamic, and anthropogenic processes that
connect the atmosphere and oceans through clouds are referred to as air-sea-
cloud interaction. Specifically, marine boundary layer clouds play an
important role in air-sea-cloud interactions, as it has a strong influence to
lower the sea surface and earth’s temperature, and also the microphysical and
dynamical characteristics of marine boundary layer clouds are sensitive to the
sea surface situation [b2].

Observations of air-sea-cloud interactions show the tendency of high
variability for a wider range of temporal and spatial scales. This is caused
by different interacting atmospheric components that fuse various
uncertainties in the Earth system. To untangle the effect of different
components on atmospheric interactions as a function of synoptic-scale
(approximately 1000 km) changes requires quantification of their interaction
patterns [b3]. To achieve this goal, it is required to study different
atmospheric properties of synoptic-scale regions covering a wide range of
longitude and latitude over a longer period. Studying this large volume of
spatial and temporal data is very complex and time-consuming for domain
experts. The inherent complexity of atmospheric system study can be reduced by
grouping environmental contexts based on spatial and temporal similarity, as
each sub-group will demonstrate a higher small-scale perturbation and will
minimize the boundary of the global atmospheric effects.

To study the atmospheric properties over a synaptic regime we need to consider
the measurements of different variables over a range of longitude and
latitude. To untangle the interaction and relative effect of these components
we need to quantify these atmospheric properties for a longer period. Hence,
the problem size can be represented by a dataset of four dimensions: time,
longitude, latitude, and variables. We can consider the task as an
unsupervised spatiotemporal clustering problem of the multidimensional data,
as the observations contain both the location and time-varying features. Also,
the absence of the labeled dataset makes the problem an unsupervised learning
task.

###  2.2 Problem Definition

The goal of the proposed model is to assign observation records of the dataset
into different clusters based on the latent spatial and temporal features
learned by the deep autoencoder model. Let us assume, $n$ atmospheric
variables ($v_{i}$) are measured over a grid region covering $L$ longitudes
and $W$ latitudes and stored in a vector
$V=\\{v_{1},v_{2},v_{3},...,v_{n}\\}$. So for each time step, every grid
location has $n$ values for all variables. Also, these variables are measured
for $T$ different time steps, $V_{i}=\\{v_{1},v_{2},v_{3},...,v_{n}\\}$,
$i\in\\{1,...,T\\}$.

Input

$Dataset=\\{V_{1},V_{2},V_{3},...,V_{T}\\}$

$\begin{split}V_{i}&=\begin{Bmatrix}\begin{bmatrix}v_{1}(1,1)&v_{1}(1,2)&\cdots&v_{1}(1,W)\\\
v_{1}(2,1)&v_{1}(2,2)&\cdots&v_{1}(2,W)\\\ \vdots&\vdots&\ddots&\vdots\\\
v_{1}(L,1)&v_{1}(L,2)&\cdots&v_{1}(L,W)\end{bmatrix}\vspace{0.2cm}\\\
\begin{bmatrix}v_{2}(1,1)&v_{2}(1,2)&\cdots&v_{2}(1,W)\\\
v_{2}(2,1)&v_{2}(2,2)&\cdots&v_{2}(2,W)\\\ \vdots&\vdots&\ddots&\vdots\\\
v_{2}(L,1)&v_{2}(L,2)&\cdots&v_{2}(L,W)\end{bmatrix}\\\ \vdots\\\
\begin{bmatrix}v_{n}(1,1)&v_{n}(1,2)&\cdots&v_{n}(1,W)\\\
v_{n}(2,1)&v_{n}(2,2)&\cdots&v_{n}(2,W)\\\ \vdots&\vdots&\ddots&\vdots\\\
v_{n}(L,1)&v_{n}(L,2)&\cdots&v_{n}(L,W)\end{bmatrix}\end{Bmatrix}\end{split}$

Here, $V_{i}$ represents one observation, $v$ means one variable of an
observation, $i\in\\{1,...,T\\}$, $T$ is the number of time steps, $n$ is the
number of atmospheric variables, $L$ is the longitude and $W$ is the latitude.

Output

In particular, the proposed model will categorize the dataset
$\\{V_{1},V_{2},V_{3},...,\\\ V_{T}\\}$ into $k$ clusters: $C_{1}$, $C_{2}$,
$C_{3}$, …, $C_{k}$, where $k<T$, so that the members of a cluster are more
similar to each other and dissimilar from the members of all other clusters.
Formally:

$C_{1}=\\{V_{C_{1}}^{1},V_{C_{1}}^{2},...,V_{C_{1}}^{n_{1}}\\},C_{2}=\\{V_{C_{2}}^{1},V_{C_{2}}^{2},...,V_{C_{2}}^{n_{2}}\\},...,C_{k}=\\{V_{C_{k}}^{1},V_{C_{k}}^{2},...,V_{C_{k}}^{n_{k}}\\}$
$V_{C_{j}}^{i}\in V,i\in\\{1,...,n_{j}\\},j\in\\{1,...,k\\}$

$n_{j}$= number of observations of cluster $j$.

$\bigcup_{j=1}^{k}C_{j}=V\textrm{ and }C_{j}\cap C_{l}=\varnothing$

Here, $j\neq l$ and $(j,l)\in\\{1,...,k\\}$.

##  3 Related Works

###  3.1 Traditional Clustering Algorithms

Unsupervised clustering is an extensively explored branch of machine learning.
A large number of clustering algorithms have been developed by focusing on
feature selection, similarity measure, grouping process, and cluster
validation strategies. The $k$-means algorithm [b14] is one of the most
popular automated clustering methods. This method is very effective for a
large number of unsupervised clustering problems and computationally very
efficient. However, the $k$-means algorithm works in the low dimensional data
space and tends to be ineffective in high dimensional data space. Density-
based spatial clustering of applications with noise (DBSCAN) [b15] is another
popular unsupervised clustering technique that works mainly focusing on the
number of neighboring data points in the input data space. This method can
work with any shape of the cluster and it automatically identifies the number
of clusters required to separate the dataset. The drawback of the BDSCAN is
that it fails to generate effective clustering if the density of the dataset
varies by a large margin. The agglomerative hierarchical clustering technique
[b16] is a bottom-up approach, which initially considers each data point as a
separate cluster and iteratively reduces the number of clusters by aggregating
two similar clusters into a new cluster until the desired number of clusters
is found. The time and space complexity of this method is high and also does
not perform well for high-dimensional datasets. To apply the $k$-means,
BDSCAN, and hierarchical clustering algorithms on a high dimensional dataset,
several variations were proposed in [b17, b18] using dimensionality reduction.
But these dimensionality reduction methods create a linear mapping between the
input data space to the low-dimensional embeddings and fail to address
nonlinear complex data.

###  3.2 Deep Learning based Clustering

In recent years, deep learning models have been applied dominantly to learn
the nonlinear and complex patterns of the input data. Deep Embedded Clustering
(DEC) [b10] is an unsupervised clustering method that maps the input data into
a low-dimensional embedding space using a deep neural network. Starting with
an initial representation and cluster assignment DEC iteratively optimizes
both by using Kullback-Leibler (KL) divergence loss. Although DEC generates
better classification results but loses the spatial details of the image. To
solve this problem the Clustering-Augmented Segmentation (CAS) [b12] model
proposed an autoencoder-based representation learning method using U-net
architecture. CAS model generates the land segmentation using clustering loss
and data reconstruction loss. Deep Temporal Clustering Representation (DTCR)
[b13] is a novel unsupervised clustering model for time series data that used
the bidirectional recurrent neural network to learn temporal representation.
In this model data reconstruction and $k$-means loss are combined into the
seq2seq model to generate cluster-specific representations. To cluster the
time series data, the Deep Temporal Clustering (DTC) [b11] model utilizes an
autoencoder to reduce the input data in a lower dimensional space and then
optimizes the clustering objective. The DTC model used temporal reconstruction
loss and clustering loss to optimize the model parameters and clustering
results. One limitation of these papers is that they either concentrate on
temporal features or pixel-level similarities. So these approaches lack a
general methodology for learning effective latent representation and
unsupervised clustering of spatiotemporal data.

##  4 Proposed Methodology

In this paper, we propose a novel autoencoder model for clustering unlabeled
spatiotemporal datasets. To capture the hidden spatial and sequential features
from the dataset, we integrate the Convolution Neural Network (CNN) and Long
Short Term Memory (LSTM) in the model. The proposed model is illustrated in
Figure 1. The encoder module transforms the input data into a latent
representation ignoring any cluster-specific features. Then, using that latent
representation, the decoder reconstructs the input data. The clustering
objective is also incorporated into the model to train it for better outcomes
in representation learning and clustering.

Figure 1: Illustration of the proposed deep spatiotemporal clustering model
architecture (best viewed in color). The number at the bottom of each block
means the number of feature layers.

###  4.1 Overview of our Deep Spatiotemporal Clustering (DSC) Approach

Given the spatiotemporal dataset $V$ such that $V_{t}\in\mathbb{R}^{lon\times
lat\times n}$, where $n$ is the number of the time series variables and
$t\in\\{1,...,T\\}$. To sub-group $T$ timesteps into $k<T$ clusters the
encoder module transforms the input data $V$ into a high-level latent
representation $E$ with reduced data dimension. The mapping from the input
data space to the latent feature space is a nonlinear encoder function
$f_{en}:=V\rightarrow E$. Here $E\in\mathbb{R}^{m}$ is an $m$-dimensional
high-level representation of all the variables at each timestep.

$E_{t}=f_{en}(V_{t}),t=\\{1,...,T\\}$ (1)

For unsupervised clustering of the high dimensional data, it is very crucial
to learn a latent representation that well represents the input data. To
achieve this requirement in the nonlinear encoder, we used 2-dimensional
convolution, which extracts the hidden spatial features from the dataset. To
extract latent features at different scales and reduce the dimensionality, we
used several stacks of 2-dimensional convolution and max pooling layer. For
further processing, dimensionality reduction is essential to prevent extremely
long sequences, which might result in weak performance, and to overcome the
curse of dimensionality. The activations of the convolution layers are then
applied to the LSTM layers to learn temporal changes. LSTM layers cast the
latent representation into a more compact space in the temporal direction.
Finally, using the latent features of the encoder model, the clustering layer
assigns the set of data points $V_{i}$, $i=\\{1,...,T\\}$, into $k$ clusters
with distinct spatiotemporal features. The decoding process of the proposed
model performs the inverse operation of the encoder in a nonlinear fashion,
$f_{de}:=E\rightarrow\widehat{V}$. It maps the compact latent features of the
encoder function into a new feature space that is identical to the input
dataset, $\widehat{V}\approx V$. The final reconstruction of the decoder
module $\widehat{V}\in\mathbb{R}^{lon\times lat\times n}$ is defined as

$\widehat{V}_{t}=f_{dc}(E_{t}),t=\\{1,...,T\\}$ (2)

By reconstructing the input data from latent representations the decoder
verifies that the behaviors of the spatiotemporal sequences are well preserved
after several dimensionality reductions. In the proposed model the decoder
process is constructed using stacked upsampling and convolution methods. This
helps the decoder network to learn effective reconstruction parameters and to
reduce the difference between the reconstructed and input data.

The proposed model clusters data points by simultaneously learning a set of
clusters in the latent feature space by minimizing two objective functions
jointly (more at Section 4.3). The first objective function emphasizes
generating well-separated groups in the latent space by reducing the
clustering loss. The second objective function tries to reduce the mean square
error of the reconstructed data from the input dataset, called reconstruction
loss. The collective optimization of both objective functions guides the
autoencoder network to extract efficient spatiotemporal features that are best
suited to distribute the input sequences into $k$ categories. The property of
learning spatial and temporal features through end-to-end optimization
establishes our autoencoder model prominent from the state-of-the-art
unsupervised clustering algorithms. The traditional models either emphasize
the clustering loss or the reconstruction. Some state-of-the-art models
optimize both the clustering and reconstruction loss but loosely concentrate
on the temporal and spatial features together. The experiments in Section 5
will show the improvement of our proposed model over these related
unsupervised clustering algorithms.

###  4.2 Clustering Assignment

At the beginning of the model training process, the input data $V_{i}$ is
applied to the freshly initialized autoencoder to obtain the first generation
latent representation $E_{i}$ which is a vector of 256 dimensions. To estimate
the initial cluster centroids $C_{j}$, $j=\\{1,...,k\\}$ the $k$-means
algorithm applied on these latent features $E_{i}$. Then the proposed
unsupervised clustering layer iteratively improves the clustering by a two-
step procedure inspired by the previous work [b10]. In the first step, a soft
assignment of each data point $V_{i}$ to all clusters is computed based on the
similarity between the cluster centroid $C_{j}$ and latent representation
$E_{i}$. In the second step, the nonlinear mapping $E_{i}$ and the cluster
centroids $C_{j}$ are refined by using the clustering loss function, which
learns from high-confidence cluster assignment of the current target
distribution. Each data point’s soft assignment is computed using the
Student’s t-distribution [b20], which measures the similarity between the
embedded representation $E_{i}$ and the cluster centroids $C_{j}$. The soft
assignment of data point $i$ to cluster $j$ is given by:

$q_{ij}=\frac{(1+||E_{i}-C_{j}||^{2}/\alpha)^{-\frac{\alpha+1}{2}}}{\sum_{l=1}^{k}(1+||E_{i}-C_{l}||^{2}/\alpha)^{-\frac{\alpha+1}{2}}}$
(3)

Here $q_{ij}$ represents the probability of assigning the $i$’th data point to
the $j$’th cluster and $\alpha$ is the degree of the freedom of the Student’s
t-distribution which is set as $1$ in our proposed model. At each iteration,
the value of $q_{ij}$ is computed based on the previous iteration’s cluster
centroids. As $q_{ij}$ are soft assignments, the predictions are strengthened
using a target distribution of data points. The target distribution puts more
attention on data points with the high-confidence cluster assignment and
normalizes the loss contribution of each centroid to prevent the distortion of
the latent feature space. The target assignment $p_{ij}$ is given by:

$p_{ij}=\frac{{q_{ij}^{2}}/{\sum_{i=1}^{n}q_{ij}}}{\sum_{l=1}^{k}({q_{il}^{2}}/{\sum_{i=1}^{n}q_{il}})}$
(4)

Once the values of $q_{ij}$ and $p_{ij}$ are computed for all data points, the
cluster centroids $C_{j}$ are refined using gradient descent. Each data
point’s cluster assignment gets recalculated following the change of the
cluster centroids, and it is then compared to the previous cluster assignment.
If there is no change in the cluster assignment of any data point for several
consecutive iterations, the model terminates the optimization process and
returns the cluster assignment from that iteration.

###  4.3 Joint Optimization

In the proposed unsupervised clustering process, we used the autoencoder model
to reconstruct the input data $V$ and the clustering layer for cluster
assignment from latent features $E_{i}$ of the input data $V_{i}$. During the
learning process, we jointly optimized the autoencoder weights and the
clustering by minimizing the mean squared error and the Kullback-Leibler (KL)
divergence loss respectively. This optimization task is implemented using the
Stochastic Gradient Descent (SGD) method with momentum. SGD guides the
autoencoder model to learn efficient latent embeddings to capture distinctive
and representative features of the input dataset. The reconstruction loss of
the model is computed by the mean squared error, which is given by

$L_{rec}=min(\frac{1}{T}\sum_{i=1}^{T}\left\|V_{i}-\widehat{V_{i}}\right\|_{2}^{2})$
(5)

Here $V_{i}\in\mathbb{R}^{lon\times lat\times n}$ is the input data and
$\widehat{V_{i}}\in\mathbb{R}^{lon\times lat\times n}$ is the output of the
autoencoder model. The cluster centroids and eventually the cluster assignment
is iteratively refined in the model by minimizing the clustering loss. The KL
divergence is used to compute the clustering loss between the soft assignment
$q_{ij}$ and the target distribution $p_{ij}$ of each data point to clusters.
The formula of the KL divergence loss is

$\begin{split}L_{clus}=min(KL(P\parallel
Q))=min(\frac{1}{T}\sum_{i=1}^{T}\sum_{j=1}^{k}p_{ij}log\frac{p_{ij}}{q_{ij}})\end{split}$
(6)

Here $T$ is the number of timestamp/data points and $k$ is the number of
clusters, $k<T$. Finally, the combined loss for training the proposed
unsupervised clustering models is defined by:

$\begin{split}L&=L_{clus}+L_{rec}\\\
&=\frac{1}{T}\sum_{i=1}^{T}\left(\left\|V_{i}-\widehat{V_{i}}\right\|_{2}^{2}+\sum_{j=1}^{k}p_{ij}log\frac{p_{ij}}{q_{ij}}\right)\end{split}$
(7)

##  5 Experiments

Our proposed model was implemented using the Python deep-learning libraries
Keras 2.11 and TensorFlow 2. All the baseline models and proposed models were
tested in the Google Colab notebook with 12 GB GPU memory support. Our model
and data are open-sourced at this link https://drive.google.com/drive/fo
lders/1-0PN3E9Gt7Equ2-PHQAAq5Ls9TF1Fved.

###  5.1 Dataset

For this study, we use the atmospheric reanalysis data available with open
access which can be obtained from European Centre for Medium-Range Weather
Forecasts (ECMWF) ERA-5 global reanalysis product [b21]. The ERA5 atmospheric
dataset contains 31 km high-resolution reanalysis and a reduced resolution
ten-member ensemble [b22]. The ensemble is required for the data assimilation
procedure, but as a by-product also provides an estimate of the relative,
random uncertainty. Seven atmospheric reanalysis variables from the ERA5
dataset were selected for this study. These are Sea Surface Temperature (unit:
$k$, and range: 285 to 300), Sea Level Pressure (unit: $Pa$, and range: 98260
to 103788), Surface Sensible Heat Flux (unit: $J/m^{2}$, and range: -674528 to
200024), Surface Latent Heat Flux (unit: $J/m^{2}$, and range: -1840906 to
90131), 2-meter Air Temperature (unit: $k$, and range: 281 to 299), 10-meter U
Wind (unit: $m/s$, and range: -16 to 19), and 10-meter V Wind (unit: $m/s$,
and range: -15 to 16). These variables are included in the dataset based on
their impact on the air-sea-cloud interaction system. These variables have the
same spatial and temporal coverage and resolution but different units and
value ranges. These variables were measured in a latitude-longitude grid of
(41x41). Temporally, the dataset covers one year period and one observation
per day. If we directly convert the data into 2D tabular data frame, the total
feature count for each record would be 11,767 (41x41x7), which is clearly a
high-dimensional dataset.

###  5.2 Data Preprocessing

After exploring the dataset we found the presence of null values. These null
values may occur due to the sensor malfunction or any physical conditions. The
null values are replaced by the overall mean of the dataset because the
replacement with any neighboring values may create a different pattern in the
data, which obviously will change the actual behavior of the variable. From
the description of the dataset, it is clear that the value ranges of the
variables are different. It is also necessary to make the features of the
dataset into the same scale for better feature learning through a deep neural
network [b23]. Normalization is a technique often applied as part of data
preprocessing for machine learning to change the values of numeric columns in
the dataset to a common scale, without distorting differences in the ranges of
values. For this dataset, we utilized the standard Min–Max Normalization (MMN)
normalization to rescale all features within the range of 0 to 1. Each
variable $V_{i}$ rescaled using the following formula:

$V_{i}=\frac{V_{i}-min(V_{i})}{max(V_{i})-min(V_{i})}$ (8)

###  5.3 Baseline Methods

We compared the result of the proposed model with the $k$-means algorithm and
the hierarchical clustering algorithm. These methods are the most popular
methods for unsupervised clustering. The original 4D input dataset is
transformed into a 2D matrix with one row per timestep to apply these
algorithms. To contrast with our suggested approach, we additionally used the
DEC [b10], DTC [b11], and DTCR [b13] algorithms from the deep-learning family
and these algorithms employed a deep neural network for dimensionality
reduction after transforming each observation’s data from a multi-dimension
matrix to a one-dimensional row. The low-dimensional latent features produced
by the deep neural network were then subjected to the clustering algorithm.
The comparison between baseline models and the proposed model is depicted in
Table 1.

Table 1: Comparison among baseline methods.

Method Input Shape Technology Pretraining k-means 2D Matrix Traditional
distance/similarity-based model Not required Hierarchical Clustering 2D Matrix
Traditional distance/similarity-based model Not required DEC 2D Matrix Deep
learning model with dense layers Required DTC 2D Matrix Deep learning model
with LSTM layers Required DTCR 2D Matrix Deep learning model with GRU layers
Required DSC (ours) 4D Matrix Deep learning model with CNN and LSTM layers Not
required

###  5.4 Evaluation Metrics

In this section, we explain metrics used in our experiments for performance
comparison, which are Average Intercluster Distance, Average Variance,
${RMSE}_{mean}$, Silhouette coefficient, and Davies-Bouldin score.

As the dataset does not have any ground truth cluster values, intercluster
distance will give a reasonable estimation of the quality of the clustering.
Intercluster distance is measured as the minimum distance between any two data
points belonging to two different clusters.

$\Delta_{inter}(C_{a},C_{b})={min}_{C_{a}\neq C_{b},X\in C_{a},Y\in
C_{b}}d(X,Y)$ (9)

Here $C_{a}$ and $C_{b}$ are two different clusters, $X\in C_{a}$ means every
observation of cluster $C_{a}$, and similarly $Y\in C_{b}$ means every
observation of cluster $C_{b}$.

The above distance metric measures give an approximation of how the clusters
are separated in the actual high-dimensional data space. By measuring the
variance of the clusters it is possible to evaluate the homogeneity and
compactness of the clusters. Also, the distribution of time series over
clusters should be carried out to minimize intracluster variance. The variance
of each cluster will be computed as an evaluation criterion of the proposed
method using the following equation

${Var}_{C_{j}}=\frac{1}{n_{C_{j}}}\sum_{X\in
C_{j}}\left\|X-{\mu}_{C_{j}}\right\|^{2}$ (10)

Here ${n_{C_{j}}}$ is the total number of observations in the cluster $C_{i}$
and ${\mu}_{C_{j}}$ is mean of the cluster.

The root-mean-squared error (RMSE) is another common measure of clustering
quality evaluation. For each observation, the error is the distance between
every observation and its associated cluster centroid. The total clustering
error is therefore the sum of the errors associated with each data point. Then
we take root over the average of the total error.

${RMSE}_{mean}=\sqrt{\frac{1}{n}\sum_{j=1}^{k}\sum_{X\in
C_{j}}\left\|X-{\mu}_{C_{j}}\right\|^{2}}$ (11)

Here $n$ is the length of the dataset, $k$ is the number of generated
clusters, $X$ is the member of cluster $C_{j}$, and ${\mu}_{C_{j}}$ is the
mean of the cluster $C_{j}$. The lower ${RMSE}_{mean}$ score means the members
of each cluster are very similar and close to the cluster centroid. On the
other hand, the higher ${RMSE}_{mean}$ means the opposite and is not desired
from a clustering algorithm.

Besides the above metrics, we also used silhouette coefficient and Davies-
Bouldin score as additional metrics. The silhouette coefficient [b24] is
applied to measure the cohesion and separation of the generated clusters. This
is defined in the range $[-1,1]$, where positive numbers denote a significant
degree of cluster separation. Moreover, negative values show that the clusters
are jumbled together (i.e., an indication of overlapping clusters). The data
are said to be evenly dispersed when the silhouette coefficient is zero.
Further, the Davies-Bouldin score [b25] is also applied to compare the results
of implemented models. The Davies-Bouldin score measures the similarity of
various formed clusters and a lower value indicates better clustering, with
zero being the lowest attainable value.

###  5.5 Experiment Results

The comparison of the proposed method with baseline algorithms is presented in
Table 5.5. The best results for each metric are in bold. We initialized the
cluster number hyperparameter as 7 for clustering the dataset. To compute
these evaluation results we trained each model several times and picked the
best result from them. The results of DEC and DTC methods are generated by
running their published codes. The hyperparameters of these models like batch
size, learning rate, maximum iteration, etc. are finetuned to get better
results on the applied dataset. The $k$-means and hierarchical clustering
method are taken from sklearn python library and applied to preprocessed data
without any dimension reduction. As shown in Table 5.5, the proposed DSC
algorithm outperforms all the baseline methods except that the DEC algorithm
[b10] has a lower ${RMSE}_{mean}$ score. Note that the DEC method does not
consider spatial features and uses a greedy pretraining process. In contrast,
our proposed method achieved ${RMSE}_{mean}$ close to DEC without any
preprocessing step and significant improvement for the other four evaluation
metrics.

Table 2: Evaluation result comparison of the proposed DSC method with baseline
methods.