# S2Mamba: A Spatial-spectral State Space Model for Hyperspectral Image
Classification

Guanchun Wang1 Xiangrong Zhang1 Zelin Peng2
Tianyang Zhang1 Xiuping Jia3 Licheng Jiao1
1 School of Artificial Intelligence, Xidian University
2 MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University
3 School of Engineering and Information Technology, University of New South
Wales Correspondence to<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Land cover analysis using hyperspectral images (HSI) remains an open problem
due to their low spatial resolution and complex spectral information. Recent
studies are primarily dedicated to designing Transformer-based architectures
for spatial-spectral long-range dependencies modeling, which is
computationally expensive with quadratic complexity. Selective structured
state space model (Mamba), which is efficient for modeling long-range
dependencies with linear complexity, has recently shown promising progress.
However, its potential in hyperspectral image processing that requires
handling numerous spectral bands has not yet been explored. In this paper, we
innovatively propose S2Mamba, a spatial-spectral state space model for
hyperspectral image classification, to excavate spatial-spectral contextual
features, resulting in more efficient and accurate land cover analysis. In
S2Mamba, two selective structured state space models through different
dimensions are designed for feature extraction, one for spatial, and the other
for spectral, along with a spatial-spectral mixture gate for optimal fusion.
More specifically, S2Mamba first captures spatial contextual relations by
interacting each pixel with its adjacent through a Patch Cross Scanning module
and then explores semantic information from continuous spectral bands through
a Bi-directional Spectral Scanning module. Considering the distinct expertise
of the two attributes in homogenous and complicated texture scenes, we realize
the Spatial-spectral Mixture Gate by a group of learnable matrices, allowing
for the adaptive incorporation of representations learned across different
dimensions. Extensive experiments conducted on HSI classification benchmarks
demonstrate the superiority and prospect of S2Mamba. The code will be
available at: https://github.com/PURE-melo/S2Mamba.

##  1 Introduction

Hyperspectral images (HSI), consisting of numerous spectral bands, are capable
of land cover analysis due to their rich material information [1, 2, 3, 4],
with extensive application in domains such as precision agriculture, mineral
exploration, and environmental monitoring [5, 6]. Therefore, there is a strong
incentive to design a more effective and efficient model for hyperspectral
image classification. Convolutional neural networks (CNNs) [7, 8, 9, 10, 11]
as a widespread paradigm have been widely studied in hyperspectral image
classification. However, this paradigm is limited by its local receptive
fields and is unable to comprehensively capture continuous spectral
properties. Transformer architectures [12, 13, 14, 15, 16, 17] have been
recently explored in hyperspectral image classification, which exhibits
remarkable performance due to their ability to extract global contextual
information on both spatial and spectral dimensions.

Despite its powerful representation ability, dealing HSI data with
transformer-based models is computationally expensive, primarily due to the
self-attention mechanism with quadratic computational complexity
$\mathcal{O}\left(N^{2}\right)$ [18, 19]. As an efficient alternative to self-
attention mechanisms, the selective structured state space model (Mamba) [20]
has recently emerged as a powerful tool with linear complexity for modeling
long-range dependency in sequence processing. Derived from this, a series of
Mamba-based models [21, 22, 23, 24, 25, 26, 27, 28, 29] have been explored for
various computer vision tasks, such as image classification, semantic
segmentation, and others. However, most of these models are only applied in
natural image processing, leaving a blank in hyperspectral image
classification due to the difficulty in handling complicated spatial-spectral
information.

Drawing inspiration from the success of state space models, this paper seeks
to explore their potential for HSI processing. To this end, we propose a
spatial-spectral state space model (S2Mamba) to jointly excavate long-range
spatial relations and continuous spectral features for hyperspectral image
classification. Our S2Mamba comprises Patch Cross Scanning and Bi-directional
Spectral Scanning modules for capturing spatial and spectral information,
respectively, and merging them with a Spatial-spectral Mixture Gate.
Specifically, we build a patch cross scanning mechanism to capture the
contextual relations between adjacent pixels, wherein the patch data is first
flattened into pixel sequences through different route generation ways and a
selective structured state space model is then applied to these sequences for
capturing contextual features. Considering the rich knowledge within
continuous spectral bands, we design an extra scanning module on the spectral
dimension for retrieving semantic properties in HSI data by a bi-directional
interaction between each band.

\begin{overpic}[width=420.61192pt]{params.pdf} \end{overpic} Figure 1: Overall
Accuracy (%) and Parameters (M) comparison on Indian Pines, Pavia University,
Houston 2013 dataset. Our proposed S2Mamba achieves optimal results in terms
of overall accuracy and parameters compared to existing methods [12, 13, 14,
30, 31].

The remaining dilemma lies in optimally merging spatial and spectral
attributes of HSI data. We observe that spectral information exhibits a more
significant role for uniform land cover regions than those regions with
complex textures. This is due to the paucity of spatial cues within homogenous
regions, necessitating the reliance on spectral information for determining
land cover categories. In contrast, spatial information can offer a
considerable prior in dealing with regions containing complex textures,
thereby enhancing classification performance. To address this, we here propose
a spatial-spectral mixture gate to dynamically merge the above features pixel
by pixel, wherein each spatial location is assigned a group of learnable
weights to determine the ratio of various features. This gating mechanism,
promoting the competition of the two features, facilitates the integration of
spatial-spectral attributes by truncating those redundant ones, greatly
boosting the accuracy.

Through experimental assessment, we verify the effectiveness of S2Mamba on the
three public hyperspectral image classification datasets. Fig. 1 illustrates
that our S2Mamba improves the previous SOTA method by 0.86%, 6.74%, and 2.56%
in terms of overall accuracy on Indian Pines, Pavia University, and Houston
2013 datasets, respectively. Meanwhile, it outperforms Transformer-based
models with the fewest parameters (about 0.12M) and a linear complexity
$\mathcal{O}\left(kN\right)$, where $k\ll N$.

##  2 Related Work

###  2.1 Hyperspectral Image Classification

Existing methods typically solve the hyperspectral image classification
problem by leveraging well-crafted deep neural networks, which can be
primarily divided into several categories: convolutional neural network-based
[8, 9, 32, 33, 34, 35, 36, 7, 37, 38, 10, 11, 39, 40, 41, 42], recurrent
neural network-based [43, 44, 30], and Transformer-based [45, 13, 15, 16, 12,
14, 17, 46, 47, 31, 48].

##### Traditional Models.

CNNs are widely applied in hyperspectral classification tasks, an excellent
feature extraction architecture that captures spatial and local semantic
information. Recent works have explored CNNs to individually extract spatial
[36, 7] and spectral features [37, 38] from hyperspectral remote sensing
images, or to learn spatial-spectral joint representation [8, 9, 32, 33, 34,
35], achieving remarkable progress. Considering that CNN structures are
incapable of modeling irregular data, some research introduced graph
convolutional networks (GCNs) [10, 11, 39, 40] to mine the potential spatial
semantic information of HSI data. Another group of studies serves the spectral
information across different bands of HSI data as continuous sequences and
employs recurrent neural networks (RNNs) [43, 44, 30] to extract spectral
features for classification. However, these methods often struggle to extract
global spectral information due to their limited capability in long-range
dependency.

##### Transformer-based Models.

Transformer is a powerful architecture consisting of multiple self-attention
mechanisms to extract global contextual information, and currently, they have
been explored for hyperspectral image classification. Most of these methods
[45, 13, 15, 16, 12, 14, 17, 46, 47, 31, 48] attempt to learn the global
sequential information on both spatial and spectral dimensions. Spectralformer
[12] is the first one in its kind to introduce the Transformer architecture
into hyperspectral image classification, which jointly captures the local and
global information by grouping the adjacent bands. Beyond single Transformer
structures, some methods adopt hybrid networks to acquire spatial-spectral
features. SSFTT [13] utilizes convolutional layers to describe the low-level
features and integrate them through Transformer layers. morphFormer [14] uses
morphological convolution layers to learn both spatial and spectral
representations and merges them by applying Transformer layers. Besides, other
works [48] incorporate RNNs with Transformers to collaboratively extract
continuous spectral features and spatial context features, significantly
enhancing the classification performance.

Unlike previous methods that capture long-range dependencies by compute-
intensive Transformer structures, we innovatively explore a fully sequential
architecture based on selective structured state space models for efficient
global spatial-spectral feature extraction.

\begin{overpic}[width=433.62pt]{Overall.pdf} \end{overpic} Figure 2:
Illustration of our proposed S2Mamba, consisting of a patch cross scanning
(PCS) mechanism, a bi-directional spectral scanning (BSS) mechanism, and a
spatial-spectral mixture gate (SMG).

###  2.2 State Space Models

Most recently, state space models (SSMs) [49, 50, 51, 52, 53], especially the
structured state space models (S4) [50] have shown promising progress in
sequence analysis, which are capable of long-range sequence modeling with
linear computational complexity. By introducing a selective mechanism into
[50], Mamba [20] further optimizes its context compression ability and
exhibits superior performance to Transformers. Considering its outstanding
performance in sequence data processing, many works have explored the
potential of Mamba [20] in computer vision and achieved promising
advancements. In particular, visual state space model (Vmamba) [22] and vision
Mamba (Vim) [21] have recently emerged as powerful tools for various computer
vision tasks due to their efficiency in modeling long-range dependencies. On
the basis of them, a series of visual state space models have been proposed,
such as medical image analysis [25, 27, 54, 55], video understanding [28, 56],
and others [26, 24, 57, 29, 58, 23]. However, most of these methods are only
applied in RGB images, leaving a blank in hyperspectral image classification
that requires handling complex spectral information. Thus, we propose a Mamba-
based architecture for hyperspectral image classification, fully exploiting
spatial-spectral features by state space models.

##  3 Proposed Method

###  3.1 Preliminaries

##### State Space Models.

State space models are foundational statistical models that describe the
dynamic behavior of systems and are widely applied in domains such as time
series analysis and control systems. In SSMs, the continuous evolution of
systems is calculated through a set of ordinary differential equations (ODEs),
which map an input signal into a latent space and decode it into an output
sequence. This operation can be defined as:

$\displaystyle h^{\prime}(t)=\mathbf{A}h(t)+\mathbf{B}x(t)$ (1) $\displaystyle
y(t)=\mathbf{C}h(t)+\mathbf{D}x(t)$

where $h(t)\in\mathbb{R}^{N}$, $x(t)\in\mathbb{R}$, and $y(t)\in\mathbb{R}$
indicate latent state, input signal, and output signal, respectively.
$h^{\prime}(t)\in\mathbb{R}^{N}$ refers to the time derivative of $h(t)$. $N$
and $L$ represent the dimensions of the latent space and sequences,
respectively. Additionally, $\mathbf{A}\in\mathbb{R}^{N\times N}$ is the state
transition matrix. $\mathbf{B}\in\mathbb{R}^{N}$ and
$\mathbf{C}\in\mathbb{R}^{N}$ are projection matrices. $\mathbf{D}$ is usually
served as a residual connected operation and thus discarded in the equation.
To facilitate the integration of continuous state space models with discrete
sequences, Mamba [20] employs a zero-order hold technique to discretize the
ordinary differential equations as follows:

$\displaystyle\overline{\mathbf{A}}=\exp({\Delta\mathbf{A}})$ (2)
$\displaystyle\overline{\mathbf{B}}=({\Delta}\mathbf{A})^{-1}(\exp(\Delta\mathbf{A})-\mathbf{I})\cdot\Delta\mathbf{B}$

where $\overline{\mathbf{A}}$ and $\overline{\mathbf{B}}$ represent the
discretized forms of the parameters $\mathbf{A}$ and $\mathbf{B}$,
respectively, through a discretization step size $\Delta$. As stated in [20],
the projection matrix $\overline{\mathbf{B}}$ can be approximated using the
first-order Taylor series:

$\overline{\mathbf{B}}=(\exp(\mathbf{A})-I)\mathbf{A}^{-1}\mathbf{B}\approx({\Delta}\mathbf{A})({\Delta}\mathbf{A})^{-1}{\Delta}\mathbf{B}=\mathrm{\Delta}\mathbf{B}$
(3)

After discretization, the ODEs of SSMs can be represented as follows:

$\displaystyle h_{t}$
$\displaystyle=\overline{\mathbf{A}}h_{t-1}+\overline{\mathbf{B}}x_{t}$ (4)
$\displaystyle y_{t}$ $\displaystyle=\mathbf{C}h_{t}$

##### Selective Scan Mechanism.

Traditional SSMs are linear time-invariant, i.e., the projection matrices do
not vary with input signals, resulting in non-selective attention on each
sequence unit. To alleviate this, Mamba [20] modifies the parameter matrices
to be input-dependent, i.e., $\mathbf{B}\in\mathbb{R}^{B\times L\times N}$,
$\mathbf{C}\in\mathbb{R}^{B\times L\times N}$ and
$\Delta\in\mathbb{R}^{B\times L\times D}$ are calculated based on
$\mathbf{X}\in\mathbb{R}^{B\times L\times D}$, further improving the ability
for handling complex sequences by transforming the SSMs into linear time-
varying systems.

###  3.2 S2Mamba

Fig. 2 shows the overall architecture of our proposed S2Mamba framework for
hyperspectral image classification. The input data is first fed into a
convolutional layer for embedding and then fed into patch cross scanning and
bi-directional scanning modules to capture spatial and spectral features.
Finally, the above terms are fused by a spatial-spectral mixture gate for
category prediction. Unlike recent advanced approaches that utilize
Transformer-based networks with quadratic complexity to capture global
spatial-spectral features, our S2Mamba is a powerful spatial-spectral
information extraction network with linear computational complexity.

####  3.2.1 Patch Cross Scanning Mechanism

To facilitate the selective scanning mechanism with HSI inputs, we first
extend the vanilla selective scan mechanism [20, 22] to the patch-level HSI
data and design a patch cross scanning mechanism, which captures spatial
contextual relations by interacting each pixel with its adjacent through a
state space model. Given the input of HSI patch
$\mathbf{X}\in\mathbb{R}^{P\times P\times K}$, where $P$ and $K$ denote the
patch size and spectral band number of the data cube, respectively, we perform
a pixel-by-pixel scanning strategy on four different routes. As illustrated in
Fig. 2, each route is generated from different directions, such as top to
bottom, left to right, and vice versa.

More specifically, we first flatten the patch data into one-dimensional
sequences with the preset routes, and then recurrently calculate each item of
sequences
$\mathcal{X}_{\text{seq}}=\\{[\mathbf{x}^{i}_{0},\mathbf{x}^{i}_{1},\ldots,\mathbf{x}^{i}_{P\times
P}]\ |\mathbf{x}^{i}_{j}\in\mathbb{R}^{1\times K},i\in\\{0,1,2,3\\}\\}$ by
using rewritten Eq. (4):

$\displaystyle\mathbf{h}^{i}_{j}$
$\displaystyle=\overline{\mathbf{A}}_{\text{spa}}\mathbf{h}^{i}_{j-1}+\overline{\mathbf{B}}_{\text{spa}}\mathbf{x}^{i}_{j}$
(5) $\displaystyle\mathbf{y}^{i}_{j}$
$\displaystyle=\mathbf{C}_{\text{spa}}\mathbf{h}^{i}_{j}+\mathbf{x}^{i}_{j}$

where $\overline{\mathbf{A}}_{\text{spa}}$,
$\overline{\mathbf{B}}_{\text{spa}}$ and $\overline{\mathbf{C}}_{\text{ spa}}$
represent the trainable parameters in PCS. After scanning, we can obtain a set
of output sequences
$\mathcal{Y}_{\text{seq}}=\\{[\mathbf{y}^{i}_{0},\mathbf{y}^{i}_{1},\ldots,\mathbf{y}^{i}_{P\times
P}]\ |\mathbf{y}^{i}_{j}\in\mathbb{R}^{1\times K},i\in\\{0,1,2,3\\}\\}$. Next,
the output sequences obtained from different scanning routes are fused
according to the operations in Fig. 2, such as flipping or transposing
sequences. As a result, each element in the output sequence
$\mathbf{Y}\in\mathbb{R}^{P\times P\times K}$ can integrate influences with
its adjacent regions from different directions.

####  3.2.2 Bi-directional Spectral Scanning Mechanism

Although the above scanning mechanism involves the spatial contextual
information within the data cube, it lacks consideration for continuous
spectral band information inherent in HSI data. One naive approach to address
this issue is to scan the data cube band by band, capturing the semantic cues
from rich spectral bands. However, due to the unidirectional information
induction attribute of state space models, the spectral scanning mechanism
conducted in a single direction may fail to adequately capture the contextual
information between spectral bands, leading to limited spectral utilization.

To this end, we further design a bi-directional spectral scanning mechanism,
analyzing the varying tendency of the continuous spectrum from multiple
directions by scanning spectral dimension band by band. We first flatten the
HSI patch along the spatial dimensions to acquire the data matrix
$\mathbf{S}\in\mathbb{R}^{K\times P^{2}}$, and then recurrently calculate each
item of sequences
$\mathcal{S}_{seq}=\\{[\mathbf{s}^{i}_{0},\mathbf{s}^{i}_{1},\ldots,\mathbf{s}^{i}_{K}]\
|\mathbf{s}^{i}_{j}\in\mathbb{R}^{1\times P^{2}},i\in\\{0,1\\}\\}$ by
following operation:

$\displaystyle\tilde{\mathbf{h}}^{i}_{j}$
$\displaystyle=\overline{\mathbf{A}}_{\text{spe}}\tilde{\mathbf{h}}^{i}_{j-1}+\overline{\mathbf{B}}_{\text{spe}}\mathbf{s}^{i}_{j}$
(6) $\displaystyle\mathbf{p}^{i}_{j}$
$\displaystyle=\mathbf{C}_{\text{spe}}\tilde{\mathbf{h}}^{i}_{j}+\mathbf{s}^{i}_{j}$

where $\mathbf{p}^{i}_{j}$ denotes the $j$th element in the $i$th order of
output sequences. $\overline{\mathbf{A}}_{\text{spe}}$,
$\overline{\mathbf{B}}_{\text{spe}}$ and $\overline{\mathbf{C}}_{\text{ spe}}$
represent the trainable parameters in BSS. After scanning, we can obtain a set
of output sequences
$\mathcal{P}_{\text{seq}}=\\{[\mathbf{p}^{i}_{0},\mathbf{p}^{i}_{1},\ldots,\mathbf{p}^{i}_{K}]\
|\mathbf{p}^{i}_{j}\in\mathbb{R}^{1\times P^{2}},i\in\\{0,1\\}\\}$. Next, the
output sequences $\mathbf{P}\in\mathbb{R}^{P\times P\times K}$ are fused from
different routes, which integrate influences from adjacent bands into each
spectral band, further boosting its discriminative ability.

####  3.2.3 Spatial-spectral Mixture Gate

After acquiring the spatial and spectral information of HSI through the two
scanning modules, calculating the optimal mixture representation becomes a key
challenge. As discussed in Sec. 1, the effectiveness of spatial and spectral
features in HSI classification differs across different scenarios, thereby
their direct merging without prior may lead to contradictions.

Specifically, we note that spectral information acts more prominence in the
classification of uniform regions as opposed to those characterized by
complicated textures. This is attributable to the scarcity of spatial cues
within homogenous regions, which in turn emphasizes the importance of spectral
information for differentiating these land cover. In this case, PCS may
contribute to misleading directions as a consequence of the redundant
features. Inversely, it offers considerable prior in dealing with regions
characterized by complex textures, thereby enhancing the richness of
discriminative representation.

To address this, we here propose a spatial-spectral mixture gate to
dynamically merge the above features for each position, wherein each location
is assigned a group of learnable weights
$\tilde{\mathbf{M}}\in\mathbb{R}^{P\times P\times 2}$ to determine the ratio
of various features as follows:

$\displaystyle\tilde{\mathbf{M}}_{0}=\frac{\exp(\mathcal{H}(\mathbf{Y};{\Theta}_{g}))}{\exp(\mathcal{H}(\mathbf{Y};{\Theta}_{g}))+\exp(\mathcal{H}(\mathbf{P};{\Theta}_{g}))}$
(7)
$\displaystyle\tilde{\mathbf{M}}_{1}=\frac{\exp(\mathcal{H}(\mathbf{P};{\Theta}_{g}))}{\exp(\mathcal{H}(\mathbf{Y};{\Theta}_{g}))+\exp(\mathcal{H}(\mathbf{P};{\Theta}_{g}))}$

where $\mathcal{H}(*;{\Theta}_{g})$ indicates the feature encoder in SMG. It
consists of two fully connected layers with a gaussian error linear unit
activation function. Next, a softmax activation function is applied to
transfer them into probability maps, whose values are between 0 and 1.
Subsequently, we merge the above features as follows:

$\mathbf{F}=(\tilde{\mathbf{M}}_{0}\cdot\mathbbm{1}(\tilde{\mathbf{M}}_{0}>\tau))\odot\mathbf{Y}+(\tilde{\mathbf{M}}_{1}\cdot\mathbbm{1}(\tilde{\mathbf{M}}_{1}>\tau))\odot\mathbf{P}$
(8)

where $\tau$ is the threshold for pruning those low contribution features. By
employing this gating mechanism, redundant features that do not contribute to
HSI classification can be effectively truncated, facilitating the integration
of spatial-spectral attributes. The intuition behind SMG is to encourage
competition between the two types of features to select the most
discriminative ones under different scenarios, thereby achieving a more
satisfactory fusion.

Table 1: Comparison results (OA%/AA%/$\kappa$) on the Indian Pines test set.
The best is in bold.

Category | CNNs | RNN | Transformers | Mamba
---|---|---|---|---
1-D CNN | 2-D CNN | miniGCN | RNN | CasRNN | ViT | SpectralFormer | morphFormer | SSFTT | GraphGST | S2Mamba
1 | 47.83 | 65.90 | 72.54 | 69.00 | 61.78 | 53.25 | 70.52 | 93.14 | 91.18 | 95.81 | 94.44
2 | 42.35 | 76.66 | 55.99 | 58.93 | 57.78 | 66.20 | 81.89 | 97.70 | 98.72 | 98.85 | 100.00
3 | 60.87 | 92.39 | 92.93 | 77.17 | 75.00 | 86.41 | 91.30 | 100.00 | 100.00 | 100.00 | 100.00
4 | 89.49 | 93.96 | 92.62 | 82.33 | 90.16 | 89.71 | 95.53 | 96.87 | 96.19 | 97.09 | 98.43
5 | 92.40 | 87.23 | 94.98 | 67.72 | 81.35 | 87.66 | 85.51 | 99.86 | 100.00 | 98.57 | 100.00
6 | 97.04 | 97.27 | 98.63 | 89.07 | 87.70 | 89.98 | 99.32 | 99.77 | 100.00 | 99.54 | 100.00
7 | 59.69 | 77.23 | 64.71 | 69.06 | 79.08 | 72.22 | 81.81 | 86.71 | 95.10 | 97.93 | 98.47
8 | 65.38 | 57.03 | 68.78 | 63.56 | 56.29 | 66.00 | 75.48 | 97.93 | 94.44 | 94.50 | 98.10
9 | 93.44 | 72.87 | 69.33 | 65.07 | 60.11 | 57.09 | 73.76 | 94.33 | 90.96 | 95.04 | 95.04
10 | 99.38 | 100.00 | 98.77 | 95.06 | 95.06 | 97.53 | 98.77 | 100.00 | 100.00 | 98.76 | 100.00
11 | 84.00 | 92.85 | 87.78 | 88.67 | 82.23 | 87.62 | 93.17 | 99.12 | 99.35 | 99.59 | 97.67
12 | 86.06 | 88.18 | 50.00 | 50.00 | 46.97 | 63.94 | 78.48 | 99.09 | 100.00 | 98.48 | 100.00
13 | 91.11 | 100.00 | 100.00 | 97.78 | 97.78 | 95.56 | 100.00 | 100.00 | 100.00 | 100.00 | 100.00
14 | 84.62 | 84.62 | 48.72 | 66.67 | 56.41 | 79.49 | 79.49 | 92.31 | 100.00 | 100.00 | 100.00
15 | 100.00 | 100.00 | 72.73 | 81.82 | 81.82 | 90.91 | 100.00 | 100.00 | 100.00 | 100.00 | 100.00
16 | 80.00 | 100.00 | 80.00 | 100.00 | 100.00 | 80.00 | 100.00 | 100.00 | 100.00 | 100.00 | 100.00
OA (%) | 70.43 | 75.89 | 75.11 | 70.66 | 68.65 | 71.86 | 81.76 | 96.38 | 96.11 | 97.06 | 97.92
AA (%) | 79.60 | 86.64 | 78.03 | 76.37 | 75.59 | 78.97 | 87.81 | 97.30 | 97.92 | 98.39 | 98.88
$\kappa$ | 0.6642 | 0.7281 | 0.7164 | 0.6673 | 0.6464 | 0.6804 | 0.7919 | 0.9584 | 0.9555 | 0.9664 | 0.9761

Table 2: Comparison results (OA%/AA%/$\kappa$) on the Pavia University test
set. The best is in bold.

Category | CNNs | RNN | Transformers | Mamba
---|---|---|---|---
1-D CNN | 2-D CNN | miniGCN | RNN | CasRNN | ViT | SpectralFormer | morphFormer | SSFTT | GraphGST | S2Mamba
1 | 88.90 | 80.98 | 96.35 | 84.01 | 77.62 | 71.51 | 82.73 | 89.90 | 87.64 | 84.99 | 96.24
2 | 58.81 | 81.70 | 89.43 | 66.95 | 63.41 | 76.82 | 94.03 | 75.26 | 76.60 | 82.43 | 98.75
3 | 73.11 | 67.99 | 87.01 | 58.46 | 57.30 | 46.39 | 73.66 | 85.90 | 85.56 | 79.94 | 85.95
4 | 82.07 | 97.36 | 94.26 | 97.70 | 98.42 | 96.39 | 93.75 | 86.33 | 95.54 | 90.80 | 97.73
5 | 99.46 | 99.64 | 99.82 | 99.10 | 99.37 | 99.19 | 99.28 | 95.87 | 100.00 | 100.00 | 99.10
6 | 97.92 | 97.59 | 43.12 | 83.18 | 75.17 | 83.18 | 90.75 | 95.54 | 98.08 | 90.66 | 99.30
7 | 88.07 | 82.47 | 90.96 | 83.08 | 88.48 | 83.08 | 87.56 | 98.27 | 99.18 | 98.78 | 99.90
8 | 88.14 | 97.62 | 77.42 | 89.63 | 87.25 | 89.63 | 95.81 | 98.01 | 94.41 | 93.10 | 98.93
9 | 99.87 | 95.60 | 87.27 | 96.48 | 99.11 | 96.48 | 94.21 | 96.86 | 98.62 | 99.25 | 98.36
OA (%) | 75.50 | 86.05 | 79.79 | 77.13 | 73.60 | 76.99 | 91.07 | 84.65 | 85.72 | 86.39 | 97.81
AA (%) | 86.26 | 88.99 | 85.07 | 84.29 | 82.91 | 80.22 | 90.20 | 91.32 | 92.85 | 91.11 | 97.14
$\kappa$ | 0.6948 | 0.8187 | 0.7367 | 0.7101 | 0.6677 | 0.7010 | 0.8805 | 0.8162 | 0.7941 | 0.8220 | 0.9705

##  4 Experiment

###  4.1 Experimental Setup

Datasets. We conduct evaluations of our S2Mamba on three publicly available
datasets, focusing on hyperspectral image classification: Indian Pines, Pavia
University, and Houston 2013 datasets. The Indian Pines dataset consists of
145$\times$145 pixels at a ground sampling distance of 20 m and 220 spectral
bands spanning the wavelength range of 400–2500 nm, in which 200 spectral
bands are preserved after removing 20 noisy and water absorption bands. The
dataset is annotated using 16 land cover categories, including crops, trees,
and other vegetation. The Pavia University dataset consists of 610$\times$340
pixels at a ground sampling distance of 1.3 m and 103 spectral bands spanning
the wavelength range of 430 to 860 nm, which is annotated using 9 land cover
categories, including asphalt, meadows, gravel, trees, metal sheets, bare
soil, bitumen, bricks, and shadows. The Houston 2013 dataset consists of
349$\times$1905 pixels at a ground sampling distance of 2.5 m and 144 spectral
bands spanning the wavelength range of 380 to 1050 nm. The dataset is
annotated using 15 land cover categories, including healthy grass, stressed
grass, synthetic grass, trees, soil, water, residential, and others. Notably,
all experiments use the same training and validate samples as [12] for a fair
comparison, which is also detailed in the supplementary materials.

Evaluation. To evaluate the classification performance of our S2Mamba, we
utilized three commonly used evaluation metrics: overall accuracy (OA),
average accuracy (AA), and kappa coefficient ($\kappa$).

Implementation Details. All the experiments are conducted within the PyTorch
framework, utilizing a single NVIDIA GeForce RTX 4090 with 24-GB GPU memory.
For the initialization of our S2Mamba, its parameters are randomly initialized
by a zero-mean normal distribution with a standard deviation of 0.01. AdamW
[59] is adopted as the optimizer for S2Mamba, where the exponential learning
rate strategy is applied with an initial value of 0.0001. The model is trained
with 400 epochs, where the batch size is set to 64. The patch size of inputs
is set to 7, 11, and 9 for Indian Pines, Pavia University and Houston 2013,
respectively. The number of blocks $N$ is set to 1.

Table 3: Comparison results (OA%/AA%/$\kappa$) on the Houston 2013 test set.
The best is in bold.

Category | CNNs | RNN | Transformers | Mamba
---|---|---|---|---
1-D CNN | 2-D CNN | miniGCN | RNN | CasRNN | ViT | SpectralFormer | morphFormer | SSFTT | GraphGST | S2Mamba
1 | 87.27 | 85.09 | 98.39 | 82.34 | 82.62 | 82.81 | 81.86 | 93.63 | 85.37 | 85.66 | 83.10
2 | 98.21 | 99.91 | 92.11 | 94.27 | 96.90 | 96.62 | 100.00 | 98.02 | 99.81 | 99.91 | 100.00
3 | 100.00 | 77.23 | 99.60 | 99.60 | 99.60 | 99.80 | 95.25 | 99.20 | 95.05 | 95.64 | 99.60
4 | 92.99 | 97.73 | 96.78 | 97.54 | 96.97 | 99.24 | 96.12 | 95.64 | 90.24 | 91.29 | 98.20
5 | 97.35 | 99.53 | 97.73 | 93.28 | 97.35 | 97.73 | 99.53 | 98.95 | 100.00 | 100.00 | 100.00
6 | 95.10 | 92.31 | 95.10 | 95.10 | 95.10 | 95.10 | 94.41 | 100.00 | 100.00 | 100.00 | 95.80
7 | 77.33 | 92.16 | 57.28 | 83.77 | 76.21 | 76.77 | 83.12 | 86.66 | 75.00 | 76.96 | 89.37
8 | 51.38 | 79.39 | 68.09 | 56.03 | 44.63 | 55.65 | 76.73 | 83.85 | 89.74 | 89.84 | 88.60
9 | 27.95 | 86.31 | 53.92 | 72.14 | 64.97 | 67.42 | 79.32 | 75.92 | 76.86 | 83.57 | 92.45
10 | 90.83 | 43.73 | 77.41 | 84.17 | 78.28 | 68.05 | 78.86 | 66.50 | 90.92 | 91.79 | 92.57
11 | 79.32 | 87.00 | 84.91 | 82.83 | 88.43 | 82.35 | 88.71 | 82.06 | 87.95 | 91.46 | 91.56
12 | 76.56 | 66.28 | 77.23 | 70.61 | 66.38 | 58.50 | 87.32 | 87.22 | 92.02 | 90.49 | 90.97
13 | 69.47 | 90.18 | 50.88 | 69.12 | 70.53 | 60.00 | 72.63 | 84.91 | 80.00 | 81.40 | 89.12
14 | 99.19 | 90.69 | 98.38 | 98.79 | 100.00 | 98.79 | 100.00 | 100.00 | 100.00 | 100.00 | 100.00
15 | 98.10 | 77.80 | 98.52 | 95.98 | 96.62 | 98.73 | 99.79 | 99.57 | 100.00 | 100.00 | 100.00
OA (%) | 80.04 | 83.72 | 81.71 | 83.23 | 81.22 | 80.41 | 88.01 | 88.27 | 89.62 | 90.80 | 93.36
AA (%) | 82.74 | 84.35 | 83.09 | 85.04 | 83.64 | 82.50 | 88.91 | 90.15 | 90.87 | 91.93 | 94.09
$\kappa$ | 0.7835 | 0.8231 | 0.8018 | 0.8183 | 0.7967 | 0.7876 | 0.8699 | 0.8727 | 0.8873 | 0.8853 | 0.9279

\begin{overpic}[width=390.25534pt]{Indian_result.pdf}
\put(7.0,32.8){{\bf{False-color Map}}} \put(7.7,-2.5){{\bf{Ground Truth}}}
\put(38.0,32.8){{\bf{SpectralFormer\cite[cite]{[\@@bibref{Number}{Spectralformer}{}{}]}}}}
\put(76.0,32.8){{\bf{SSFTT\cite[cite]{[\@@bibref{Number}{SSFTT}{}{}]}}}}
\put(41.0,-2.5){{\bf{GraphGST\cite[cite]{[\@@bibref{Number}{GraphGST}{}{}]}}}}
\put(75.0,-2.5){{\bf{S${}^{2}$Mamba}}} \end{overpic} Figure 3: Classification
results on the Indian Pines dataset.

\begin{overpic}[width=433.62pt]{patchsize.pdf} \end{overpic} Figure 4:
Parameter analysis of the patch size in S2Mamba.

###  4.2 Comparison with State-of-the-Arts

We conduct a comprehensive evaluation of our S2Mamba and compare it with
several comparative approaches, including CNN-based (1-D CNN, 2-D CNN, miniGCN
[11]), RNN-based (RNN, CasRNN [30]), and Transformer-based (ViT [19],
Spectralformer [12], morphformer [14], SSFTT [13], GraphGST [31]) methods,
where 1-D CNN, 2-D CNN, RNN, and ViT are implemented following [12]. All
methods are tested using the optimal experimental settings reported in their
papers or re-implemented by their official code.

\begin{overpic}[width=433.62pt]{PU_result.pdf} \put(2.0,-1.0){{\bf{False-color
Map}}} \put(25.0,-1.0){{\bf{Ground Truth}}}
\put(43.0,-1.0){{\bf{SpectralFormer\cite[cite]{[\@@bibref{Number}{Spectralformer}{}{}]}}}}
\put(66.0,-1.0){{\bf{SSFTT\cite[cite]{[\@@bibref{Number}{SSFTT}{}{}]}}}}
\put(86.0,-1.0){{\bf{S${}^{2}$Mamba}}} \end{overpic} Figure 5: Classification
results on the Pavia University dataset. Table 4: Ablation study on the impact
of each component in S2Mamba. "PCS": Patch Cross Scanning. "BSS": Bi-
directional Spectral Scanning. "SMG": Spatial-spectral Mixture Gate.

PCS | BSS | SMG | Indian Pines | Pavia University | Houston 2013
---|---|---|---|---|---
OA (%) | AA (%) | $\kappa$ | OA (%) | AA (%) | $\kappa$ | OA (%) | AA (%) | $\kappa$
$\checkmark$ |  |  | 96.45 | 97.99 | 0.9593 | 96.42 | 96.53 | 0.9521 | 90.78 | 92.05 | 0.9002
| $\checkmark$ |  | 96.51 | 98.27 | 0.9599 | 96.24 | 96.21 | 0.9493 | 92.19 | 93.29 | 0.9154
$\checkmark$ | $\checkmark$ |  | 97.19 | 98.56 | 0.9678 | 97.17 | 97.12 | 0.9620 | 92.74 | 93.36 | 0.9211
$\checkmark$ | $\checkmark$ | $\checkmark$ | 97.92 | 98.88 | 0.9761 | 97.81 | 97.14 | 0.9705 | 93.36 | 94.09 | 0.9279

Indian Pines. In Table 1, we benchmark performances on the Indian Pines
dataset. The results, presented in Table 1, demonstrate that our approach
significantly outperforms state-of-the-art hyperspectral image classification
methods on 14 categories, achieving the best comprehensive performance in
terms of OA (97.92% vs. 97.06%), AA (98.88% vs. 98.39%), and $\kappa$ (0.9761
vs. 0.9664). In particular, we surpass the performance of the typical
Transformer-based method, i.e., SpectralFormer [12], which utilizes
transformer architecture to extract long-range dependencies from continuous
spectral bands. In contrast, our S2Mamba, involving more efficient basic
structures and elaborate designs, achieves superior results in terms of OA,
AA, and $\kappa$ (e.g., improving the OA from 81.76% to 97.92%). Furthermore,
Fig. 4 shows examples of prediction maps. It demonstrates that our S2Mamba is
capable of producing accurate predictions of each category.

Pavia University. We further evaluate our S2Mamba on the Pavia University
dataset, whose scenario includes numerous complex spatial textures. As
illustrated in Table 2, our method significantly outperforms the comparative
techniques with at least 6.74%, 4.29%, and 9.00% in terms of OA, AA, and
$\kappa$, respectively. This confirms that our S2Mamba can offer a more
satisfactory solution under complex scenarios. Fig. 5 shows that our approach
achieves superior performance than others with more completed prediction maps,
such as asphalt (masked in light green color) and meadow (masked in dark green
color).

Houston 2013. Furthermore, we compare our proposed method with the state-of-
the-art methods on the more challenging dataset, i.e., Houston 2013. The
experimental results, presented in Table 3, demonstrate that our S2Mamba
significantly outperforms state-of-the-art hyperspectral image classification
methods on 8 categories, achieving the best comprehensive performance in terms
of OA, AA, and Kappa coefficient. Specifically, it exhibits 2.56%, 2.16%, and
4.26% gains in comparison with the advanced Transformer-based method [31] on
OA, AA, and $\kappa$, respectively. Fig. 6 shows classification predictions,
which demonstrates that our S2Mamba realizes the most precise results. For
instance, our method is capable of accurately identifying highways (masked in
blue color) under shadows, demonstrating its effectiveness.

\begin{overpic}[width=433.62pt]{Houston_result.pdf}
\put(100.2,76.0){\rotatebox{270.0}{\bf{False-color Map}}}
\put(100.2,60.0){\rotatebox{270.0}{\bf{Ground Truth}}}
\put(100.2,46.0){\rotatebox{270.0}{\bf{SpectralFormer\cite[cite]{[\@@bibref{Number}{Spectralformer}{}{}]}}}}
\put(100.2,27.3){\rotatebox{270.0}{\bf{SSFTT\cite[cite]{[\@@bibref{Number}{SSFTT}{}{}]}}}}
\put(100.2,12.5){\rotatebox{270.0}{\bf{S${}^{2}$Mamba}}} \end{overpic} Figure
6: Classification results on the Houston 2013 dataset.

\begin{overpic}[width=433.62pt]{Gating_Threshold.pdf} \end{overpic} Figure 7:
Parameter analysis of different gating thresholds.

\begin{overpic}[width=433.62pt]{Latent_Dimension.pdf} \end{overpic} Figure 8:
Parameter analysis of different latent dimensions.

\begin{overpic}[width=433.62pt]{Layer_Number.pdf} \end{overpic} Figure 9:
Parameter analysis of different layer numbers.

###  4.3 Ablation Study

The effectiveness of each component in S2Mamba. We conduct ablation studies in
Table 4 to show the impact of each component in S2Mamba. The first row shows
the performance using the PCS mechanism, which efficiently considers the
spatial relations of pixels by a Mamba-based module. As can be seen, it
outperforms most comparative methods with 96.45%, 96.42%, and 90.78% OA on
Indian Pines, Pavia University, and Houston 2013 datasets, respectively, due
to its effective ability in modeling spatial contexts. Then, by incorporating
a bi-directional spectral scanning mechanism, we improve performance to
96.72%, 97.17%, and 92.74% OA on the three datasets. This suggests that the
BSS can offer more discriminative cues by scanning continuous spectral bands.
Finally, by combining both the PCS, BSS, and SMG, we further boost the
performance to 97.92%/98.80%/0.9746, 97.81%/97.14%/0.9705, and
93.36%/94.09%/0.9279 in terms of OA/AA/k on the three datasets, which confirms
the SMG module can preferably merge spatial and spectral features for each
location by a feature competition. The comprehensive results demonstrate that
our S2Mamba can effectively learn discriminative representations of HSI data.

Parameter analysis of each component in S2Mamba. We study the effectiveness of
hyperparameters in S2Mamba, including patch size, gating threshold, latent
dimension, and layer numbers. As shown in Fig. 4, it can be observed that the
optimal patch sizes of Indian Pines, Pavia University, and Houston 2013 are 7,
11, and 9, separately, which is consistent with the expectation that the last
two datasets comprise more complex spatial boundaries, thereby larger patch
inputs are required. Fig. 9 demonstrates that firming the gating threshold to
0.1 can achieve satisfactory results, which can filter those redundant
features. Figs. 9 and 9 denote that 64 and 1 are the optimal values of hidden
dimension and layer number, respectively. This confirms our S2Mamba can
achieve state-of-the-art performance with a single-block network, whereas
deeper layers or larger hidden dimensions can not yield additional
improvements.

##  5 Conclusion

In this paper, we propose S2Mamba, a novel architecture for hyperspectral
image classification. S2Mamba comprises a Patch Cross Scanning mechanism and a
Bi-directional Spectral Scanning mechanism for learning the contextual
information from spatial and spectral aspects, respectively, which utilize the
selective structured state space models as alternatives to self-attention
mechanisms for capturing long-range dependency with linear complexity, thereby
efficiently improving the results. Furthermore, to optimally merge the above
features, a Spatial-spectral Mixture Gate is proposed to adjust the fusion
ratio for each position with learnable matrices, further enhancing the
classification performance. Experimental results on three datasets verify the
superiority of our S2Mamba.

## References

  * [1] Maryam Imani and Hassan Ghassemian.  An overview on spectral and spatial information fusion for hyperspectral image classification: Current trends and challenges.  Information fusion, 59:59–83, 2020.
  * [2] Behnood Rasti, Danfeng Hong, Renlong Hang, Pedram Ghamisi, Xudong Kang, Jocelyn Chanussot, and Jon Atli Benediktsson.  Feature extraction for hyperspectral imagery: The evolution from shallow to deep: Overview and toolbox.  IEEE Geoscience and Remote Sensing Magazine, 8(4):60–88, 2020.
  * [3] Shutao Li, Weiwei Song, Leyuan Fang, Yushi Chen, Pedram Ghamisi, and Jon Atli Benediktsson.  Deep learning for hyperspectral image classification: An overview.  IEEE Transactions on Geoscience and Remote Sensing, 57(9):6690–6709, 2019.
  * [4] Danfeng Hong, Bing Zhang, Xuyang Li, Yuxuan Li, Chenyu Li, Jing Yao, Naoto Yokoya, Hao Li, Pedram Ghamisi, Xiuping Jia, et al.  Spectralgpt: Spectral remote sensing foundation model.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
  * [5] Antonio Plaza, Jon Atli Benediktsson, Joseph W Boardman, Jason Brazile, Lorenzo Bruzzone, Gustavo Camps-Valls, Jocelyn Chanussot, Mathieu Fauvel, Paolo Gamba, Anthony Gualtieri, et al.  Recent advances in techniques for hyperspectral image processing.  Remote sensing of environment, 113:S110–S122, 2009.
  * [6] Xiangrong Zhang, Shunli Tian, Guanchun Wang, Xu Tang, Jie Feng, and Licheng Jiao.  Cast: A cascade spectral aware transformer for hyperspectral image change detection.  IEEE Transactions on Geoscience and Remote Sensing, 2023.
  * [7] Yonghao Xu, Bo Du, Fan Zhang, and Liangpei Zhang.  Hyperspectral image classification via a random patches network.  ISPRS journal of photogrammetry and remote sensing, 142:344–357, 2018.
  * [8] Zilong Zhong, Jonathan Li, Zhiming Luo, and Michael Chapman.  Spectral–spatial residual network for hyperspectral image classification: A 3-d deep learning framework.  IEEE Transactions on Geoscience and Remote Sensing, 56(2):847–858, 2017.
  * [9] Mercedes E Paoletti, Juan Mario Haut, Javier Plaza, and Antonio Plaza.  A new deep convolutional neural network for fast hyperspectral image classification.  ISPRS journal of photogrammetry and remote sensing, 145:120–147, 2018.
  * [10] Yanni Dong, Quanwei Liu, Bo Du, and Liangpei Zhang.  Weighted feature fusion of convolutional neural network and graph attention network for hyperspectral image classification.  IEEE Transactions on Image Processing, 31:1559–1572, 2022.
  * [11] Danfeng Hong, Lianru Gao, Jing Yao, Bing Zhang, Antonio Plaza, and Jocelyn Chanussot.  Graph convolutional networks for hyperspectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 59(7):5966–5978, 2020.
  * [12] Danfeng Hong, Zhu Han, Jing Yao, Lianru Gao, Bing Zhang, Antonio Plaza, and Jocelyn Chanussot.  Spectralformer: Rethinking hyperspectral image classification with transformers.  IEEE Transactions on Geoscience and Remote Sensing, 60:1–15, 2021.
  * [13] Le Sun, Guangrui Zhao, Yuhui Zheng, and Zebin Wu.  Spectral–spatial feature tokenization transformer for hyperspectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2022.
  * [14] Swalpa Kumar Roy, Ankur Deria, Chiranjibi Shah, Juan M Haut, Qian Du, and Antonio Plaza.  Spectral–spatial morphological attention transformer for hyperspectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 61:1–15, 2023.
  * [15] Haoyang Yu, Zhen Xu, Ke Zheng, Danfeng Hong, Hao Yang, and Meiping Song.  Mstnet: A multilevel spectral–spatial transformer network for hyperspectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 60:1–13, 2022.
  * [16] Lianhui Liang, Ying Zhang, Shaoquan Zhang, Jun Li, Antonio Plaza, and Xudong Kang.  Fast hyperspectral image classification combining transformers and simam-based cnns.  IEEE Transactions on Geoscience and Remote Sensing, 2023.
  * [17] Xiaofei Yang, Weijia Cao, Yao Lu, and Yicong Zhou.  Hyperspectral image transformer classification networks.  IEEE Transactions on Geoscience and Remote Sensing, 60:1–15, 2022.
  * [18] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [19] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  In International Conference on Learning Representations, 2020.
  * [20] Albert Gu and Tri Dao.  Mamba: Linear-time sequence modeling with selective state spaces.  arXiv preprint arXiv:2312.00752, 2023.
  * [21] Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang.  Vision mamba: Efficient visual representation learning with bidirectional state space model.  arXiv preprint arXiv:2401.09417, 2024.
  * [22] Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, and Yunfan Liu.  Vmamba: Visual state space model.  arXiv preprint arXiv:2401.10166, 2024.
  * [23] Xiaohuan Pei, Tao Huang, and Chang Xu.  Efficientvmamba: Atrous selective scan for light weight visual mamba.  arXiv preprint arXiv:2403.09977, 2024.
  * [24] Keyan Chen, Bowen Chen, Chenyang Liu, Wenyuan Li, Zhengxia Zou, and Zhenwei Shi.  Rsmamba: Remote sensing image classification with state space model.  arXiv preprint arXiv:2403.19654, 2024.
  * [25] Jun Ma, Feifei Li, and Bo Wang.  U-mamba: Enhancing long-range dependency for biomedical image segmentation.  arXiv preprint arXiv:2401.04722, 2024.
  * [26] Sijie Zhao, Hao Chen, Xueliang Zhang, Pengfeng Xiao, Lei Bai, and Wanli Ouyang.  Rs-mamba for large remote sensing image dense prediction.  arXiv preprint arXiv:2404.02668, 2024.
  * [27] Zhaohu Xing, Tian Ye, Yijun Yang, Guang Liu, and Lei Zhu.  Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation.  arXiv preprint arXiv:2401.13560, 2024.
  * [28] Kunchang Li, Xinhao Li, Yi Wang, Yinan He, Yali Wang, Limin Wang, and Yu Qiao.  Videomamba: State space model for efficient video understanding.  arXiv preprint arXiv:2403.06977, 2024.
  * [29] Dingkang Liang, Xin Zhou, Xinyu Wang, Xingkui Zhu, Wei Xu, Zhikang Zou, Xiaoqing Ye, and Xiang Bai.  Pointmamba: A simple state space model for point cloud analysis.  arXiv preprint arXiv:2402.10739, 2024.
  * [30] Renlong Hang, Qingshan Liu, Danfeng Hong, and Pedram Ghamisi.  Cascaded recurrent neural networks for hyperspectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 57(8):5384–5394, 2019.
  * [31] Mengying Jiang, Yuanchao Su, Lianru Gao, Antonio Plaza, Xi-Le Zhao, Xu Sun, and Guizhong Liu.  Graphgst: Graph generative structure-aware transformer for hyperspectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 2024.
  * [32] Haokui Zhang, Ying Li, Yenan Jiang, Peng Wang, Qiang Shen, and Chunhua Shen.  Hyperspectral classification based on lightweight 3-d-cnn with transfer learning.  IEEE Transactions on Geoscience and Remote Sensing, 57(8):5813–5828, 2019.
  * [33] Gong Cheng, Zhenpeng Li, Junwei Han, Xiwen Yao, and Lei Guo.  Exploring hierarchical convolutional features for hyperspectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 56(11):6712–6722, 2018.
  * [34] Weiwei Song, Shutao Li, Leyuan Fang, and Ting Lu.  Hyperspectral image classification with deep feature fusion network.  IEEE Transactions on Geoscience and Remote Sensing, 56(6):3173–3184, 2018.
  * [35] Swalpa Kumar Roy, Gopal Krishna, Shiv Ram Dubey, and Bidyut B Chaudhuri.  Hybridsn: Exploring 3-d–2-d cnn feature hierarchy for hyperspectral image classification.  IEEE Geoscience and Remote Sensing Letters, 17(2):277–281, 2019.
  * [36] Juan Mario Haut, Mercedes E Paoletti, Javier Plaza, Antonio Plaza, and Jun Li.  Hyperspectral image classification using random occlusion data augmentation.  IEEE Geoscience and Remote Sensing Letters, 16(11):1751–1755, 2019.
  * [37] Wei Zhao, Licheng Jiao, Wenping Ma, Jiaqi Zhao, Jin Zhao, Hongying Liu, Xianghai Cao, and Shuyuan Yang.  Superpixel-based multiple local cnn for panchromatic and multispectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 55(7):4141–4156, 2017.
  * [38] Shiqi Yu, Sen Jia, and Chunyan Xu.  Convolutional neural networks for hyperspectral image classification.  Neurocomputing, 219:88–98, 2017.
  * [39] Lichao Mou, Xiaoqiang Lu, Xuelong Li, and Xiao Xiang Zhu.  Nonlocal graph convolutional networks for hyperspectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 58(12):8246–8257, 2020.
  * [40] Sheng Wan, Chen Gong, Ping Zhong, Shirui Pan, Guangyu Li, and Jian Yang.  Hyperspectral image classification with context-aware dynamic graph convolutional network.  IEEE Transactions on Geoscience and Remote Sensing, 59(1):597–612, 2020.
  * [41] Zhuo Zheng, Yanfei Zhong, Ailong Ma, and Liangpei Zhang.  Fpga: Fast patch-free global learning framework for fully end-to-end hyperspectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 58(8):5612–5626, 2020.
  * [42] Xiangrong Zhang, Shouwang Shang, Xu Tang, Jie Feng, and Licheng Jiao.  Spectral partitioning residual network with spatial attention mechanism for hyperspectral image classification.  IEEE transactions on geoscience and remote sensing, 60:1–14, 2021.
  * [43] Lichao Mou, Pedram Ghamisi, and Xiao Xiang Zhu.  Deep recurrent neural networks for hyperspectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 55(7):3639–3655, 2017.
  * [44] Atharva Sharma, Xiuwen Liu, and Xiaojun Yang.  Land cover classification from multi-temporal, multi-spectral remotely sensed imagery using patch-based recurrent neural networks.  Neural Networks, 105:346–355, 2018.
  * [45] Xin He, Yushi Chen, and Zhouhan Lin.  Spatial-spectral transformer for hyperspectral image classification.  Remote Sensing, 13(3):498, 2021.
  * [46] Chunhui Zhao, Boao Qin, Shou Feng, Wenxiang Zhu, Weiwei Sun, Wei Li, and Xiuping Jia.  Hyperspectral image classification with multi-attention transformer and adaptive superpixel segmentation-based active learning.  IEEE Transactions on Image Processing, 2023.
  * [47] Zhao Qiu, Jie Xu, Jiangtao Peng, and Weiwei Sun.  Cross-channel dynamic spatial-spectral fusion transformer for hyperspectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 2023.
  * [48] Weilian Zhou, Sei-Ichiro Kamata, Haipeng Wang, and Xi Xue.  Multiscanning-based rnn-transformer for hyperspectral image classification.  IEEE Transactions on Geoscience and Remote Sensing, 2023.
  * [49] Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré.  Hippo: Recurrent memory with optimal polynomial projections.  Advances in neural information processing systems, 33:1474–1487, 2020.
  * [50] Albert Gu, Karan Goel, and Christopher Re.  Efficiently modeling long sequences with structured state spaces.  In International Conference on Learning Representations, 2021.
  * [51] Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré.  Combining recurrent, convolutional, and continuous-time models with linear state space layers.  Advances in neural information processing systems, 34:572–585, 2021.
  * [52] Jimmy TH Smith, Andrew Warrington, and Scott Linderman.  Simplified state space layers for sequence modeling.  In The Eleventh International Conference on Learning Representations, 2022.
  * [53] Ankit Gupta, Albert Gu, and Jonathan Berant.  Diagonal state spaces are as effective as structured state spaces.  Advances in Neural Information Processing Systems, 35:22982–22994, 2022.
  * [54] Jiacheng Ruan and Suncheng Xiang.  Vm-unet: Vision mamba unet for medical image segmentation.  arXiv preprint arXiv:2402.02491, 2024.
  * [55] Jiarun Liu, Hao Yang, Hong-Yu Zhou, Yan Xi, Lequan Yu, Yizhou Yu, Yong Liang, Guangming Shi, Shaoting Zhang, Hairong Zheng, et al.  Swin-umamba: Mamba-based unet with imagenet-based pretraining.  arXiv preprint arXiv:2402.03302, 2024.
  * [56] Guo Chen, Yifei Huang, Jilan Xu, Baoqi Pei, Zhe Chen, Zhiqi Li, Jiahao Wang, Kunchang Li, Tong Lu, and Limin Wang.  Video mamba suite: State space model as a versatile alternative for video understanding.  arXiv preprint arXiv:2403.09626, 2024.
  * [57] Xuanhua He, Ke Cao, Keyu Yan, Rui Li, Chengjun Xie, Jie Zhang, and Man Zhou.  Pan-mamba: Effective pan-sharpening with state space model.  arXiv preprint arXiv:2402.12192, 2024.
  * [58] Hongruixuan Chen, Jian Song, Chengxi Han, Junshi Xia, and Naoto Yokoya.  Changemamba: Remote sensing change detection with spatio-temporal state space model.  arXiv preprint arXiv:2404.03425, 2024.
  * [59] Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  In International Conference on Learning Representations, 2018.