# Transformer based Endmember Fusion with Spatial Context for Hyperspectral
Unmixing

H. M. K. D. Wickramathilaka, R. M. K. L. Ratnayake, D. M. U. P. Sumanasekara,
G. M. R. I. Godaliyadda, M. P. B. Ekanayake, H. M. V. R. Herath All the
authors of this paper are with the Department of Electrical and Electronic
Engineering, University of Peradeniya, Sri Lanka. (Email addresses are listed
according to the authors’ order. e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

In recent years, transformer-based deep learning networks have gained
popularity in Hyperspectral (HS) unmixing applications due to their superior
performance. The attention mechanism within transformers facilitates input-
dependent weighting and enhances contextual awareness during training. Drawing
inspiration from this, we propose a novel attention-based Hyperspectral
Unmixing algorithm called Transformer based Endmember Fusion with Spatial
Context for Hyperspectral Unmixing (FusionNet). This algorithm can effectively
fuse endmember signatures obtained from different endmember extraction
algorithms, as opposed to the classical HS Unmixing approaches that rely on a
single Endmember Extraction Algorithm (EEA). The FusionNet incorporates an
Approximation Network (AN), introducing contextual awareness into abundance
prediction by considering neighborhood pixels. Unlike Convolutional Neural
Networks (CNNs), which are constrained by specific kernel shapes, the Fusion
network offers flexibility in choosing any arbitrary configuration of the
neighborhood. We conducted a comparative analysis between the FusionNet
algorithm and state-of-the-art algorithms using two popular datasets.
Remarkably, FusionNet outperformed other algorithms, achieving the lowest Root
Mean Square Error (RMSE)for abundance predictions and competitive Spectral
Angle Distance (SAD)for signatures associated with each endmember for both
datasets.

###### Index Terms:

Hyperspectral Unmixing, Fusion, Endmember Extraction Algorithms, Multihead
Attention

##  I Introduction

In the remote sensing field, Hyperspectral (HS)imaging plays a vital role. As
HS images contain numerous bands, they contain a lot of information compared
to conventional RGB images and Multispectral images [1]. Due to this, HS
images have become useful in a vast range of applications such as Agriculture
[2, 3], Mineral Mapping [4, 5], Military applications [6, 7], Environmental
Monitoring [8, 9] etc. Each pixel in the HS image can be considered as a
mixture of pure materials. These pure materials are called endmembers and
their spectral signatures are known as endmember signatures. The percentage of
each pure material that appears in a pixel is called endmember abundance. The
process of extracting the spectral signature and the abundance of each
endmember is known as HS unmixing.

Although the spectral resolution of HS images is high, they lack spatial
resolution [10, 11]. In addition to that, HS images contain noise [12], [13].
These are the key challenges that must be addressed in HS unmixing. Due to the
physical meaning of the endmember abundances and signatures, some constraints
should be considered in the HS Unmixing process. The first constraint is that
the values of the endmember abundances and the signatures cannot be negative.
The second constraint is that the sum of the abundances of all the endmembers
in each pixel must be one because every single pixel is a combination of these
endmembers. These two conditions are called non-negativity constraint (ANC)and
abundance sum-to-one constraint (ASC)respectively [14].

Every pixel’s signature is a mixture of these endmember signatures. This
mixing process can happen macroscopically and microscopically [15, 16].
According to that, the Linear Mixiture Model (LMM)and several non-linear
mixture models have been proposed [17].

Many algorithms have been proposed to solve the unmixing problem and they can
be divided into four main categories. They are Geometrical methods [18, 19,
20], Statistical methods [21, 22, 23, 24, 25, 26], Sparse Regression methods
[27, 28, 29], and Deep Learning methods [30, 31, 32, 33]. Geometrical methods
use either pure pixels [19, 18], or minimum volume simplex to unmix the HS
image[34, 35], Statistical approach is based on the statistical properties of
the HS image. It tends to solve the matrix using blind source separation [36].
When the data is highly mixed, the statistical method gives better results
compared to geometrical methods as there are not enough spectral vectors in
the simplex facets. The sparse-regression methods use pre-obtained laboratory
signatures to construct the pixel signatures [37]. In this method, the optimal
subset of material signatures, that can span the pixel spectrum space is
determined. Deep learning approaches use neural networks to unmix the HS
images. This is done either semi-supervised [38] manner or unsupervised [39]
manner.

Due to increasing computational power and the ability to find complex
relationships, the deep learning approaches show promising results on HS
unmixing. A common architecture that has been used for unmixing is the Auto-
encoder [40], which consists of one encoder to compress the data to a lower
dimension space (In HS Unmixing, this lower dimension space can be used as
abundances by setting constraints) and a decoder to reconstruct the data from
the lower dimensional space. Dense layers [40, 32], convolutional layers [41,
30], and recurrent layers [42] are being used to perform the unmixing process
effectively. Many architectures that are inspired by auto-encoder, have been
proposed with competitive results [40, 43, 30, 31, 32].

In addition to that, the recent advancements in transformer architecture [44]
have encouraged the application of transformer-based algorithms for HS
unmixing. [33] uses a transformer encoder to unmixing HS image by capturing
long-range dependencies. [45] uses a swin-transformer to extract the spatial
information and a simplified attention mechanism to obtain the spectral
information. Using window multi-head self-attention and shifted window multi-
head self-attention mechanisms, [45] can extract global spatial priors.
Addressing the sparse unmixing problem, [46] has used a special attention
mechanism named window-based pixel level multi-head self-attention. [47]
exploits two properties of HS images which are, region homogeneity and
spectral correlation. Two modules namely, the Score-based homogeneous aware
module and the spectral group aware module have been used to address these
properties. [48] uses a bilateral global attention network to fuse the
spectral and spatial information optimally. This architecture has two separate
branches to extract the spatial and spectral features.

Deep learning methods often use Endmember Extraction Algorithms (EEA)[19, 18]
to initialize the decoder part of the auto-encoder. These existing methods
have used a single EEA for decoder initialization. From the literature review,
it was found that the performance of the EEA s depends on the dataset [49].
Each of the EEAs makes different assumptions about the environment. These
assumptions have varying validity in each environment. Therefore, the best-
performing EEA is different in each environment. Because of that, if a single
EEA is used, the optimum initialization might not be achieved. This will have
an adverse impact on results and may affect the rate of convergence. Using a
fusion of different endmember signatures which have been extracted by
different EEA s to guide the network could resolve the above-mentioned issue.
Despite this, there is a lack of research on the fusion of endmember
signatures extracted by different EEA s.

Due to the low spatial resolution of the HS images, there is a high
correlation between the pixels and their neighborhoods. Therefore, in the HS
unmixing process, the spatial context is a strong property. This can be
utilized to solve the unmixing problem effectively. When considering noisy
pixels, the context of the neighboring pixels would be useful to increase the
accuracy [50]. Also, the contextual data are important to predict the
abundance maps and enhance spatial smoothness [50].

The spatial context is often incorporated using CNN s or transformers. The
CNN-based methods incorporate spatial context in the form of spatial
correlation [30, 41]. Therefore, they only consider local pixels that are
limited by the size of the kernel of the convolution layer. On the other hand,
transformer-based algorithms often strive to feed global spatial context to
the unmixing process, which causes locally irrelevant spatial information to
feed through in the abundance map generation process.

Therefore, a methodology that can incorporate spatial context without being
bound by a local window or fixed globally would be desirable. The strategy
adopted is motivated to some degree by [50] and extends upon exploiting the
attention mechanism used in transformers.

We bridge the research gap with the contribution of this paper, which can be
summarized as follows,

  1. 1.

_We introduce a novel attention-based mechanism to fuse endmember signatures
taken by different Endmember Extraction Algorithms (EEA)s. We leverage the
attention mechanism to learn and predict better endmember signatures. Each
endmember has the flexibility to learn separately in the fusion mechanism.
Furthermore, the attention mechanism allows to learn the optimum fusion
weights for each spectral band of endmember signatures separately._

  2. 2.

_Introducing the Approximation Network (AN)that uses an attention mechanism to
exploit the contextual data to create context-aware spectral signatures. Here,
we provide the center pixel as the query and neighboring pixels as the key and
value for the attention mechanism. The proposed attention mechanism is input-
dependent and considers the correlation between the center pixel and
neighboring pixels when creating context-aware spectral signatures. Therefore,
the proposed mechanism allows for the local context of a given environment to
be learned as opposed to imposing a rigid global structure on the entire
image_

The remaining sections of the paper are as follows. Section II expounds upon
the proposed architecture and Section III describes the experimental setup and
results for two different datasets and different network architectures.
Finally, the concluding remarks and the summary of the findings is
encapsulated in the Section IV

##  II Problem Formulation and Methodology

###  II-A Brief Review of attention networks and AE-Based unmixing networks

####  II-A1 Attention networks

The attention mechanism introduced in transformers has demonstrated remarkable
performance compared to the layers utilized in other architectures. Since the
attention mechanism introduces dynamic weighting, wherein the model adjusts
the importance it assigns to different parts of the input based on its
content, rather than using fixed weights as in linear layers. This dynamism
ensures the model’s ability to focus on what’s relevant for specific tasks or
contexts.

The attention layer in the encoder of a transformer should be provided with 3
parameters as inputs namely, query $Q$, value $V$, and key $K$. The attention
layer can be mathematically represented as follows,

$Attention(Q,K,V)=softmax\left(\frac{QK^{T}}{\sqrt{d_{k}}}\right)V$ (1)

In most traditional models without attention, each input is processed in a
somewhat isolated manner, often with a fixed transformation based on the
model’s learned parameters. The model might, for instance, apply the same
transformation to an input irrespective of other inputs present.

In contrast, attention mechanisms introduce a degree of relational reasoning.
When processing an input, attention doesn’t just consider that input in
isolation. Instead, it determines the importance or relevance of inputs with
the context. This is achieved by computing scores (or weights) that signify
how much focus should be placed on each input element. Considering the above
formula, this is accomplished by ${QK^{T}}$. Crucially, these scores are not
fixed and they are computed dynamically based on the interplay of inputs.

####  II-A2 Linear Mixiture Model (LMM)

In HS imaging, using the Linear Mixiture Model (LMM)the spectral signature can
be represented as,

$X=SA+N$ (2)

where, $X\in\mathbb{R}^{L\times N}$ is the observed HS image containing $L$
bands and $N$ pixels. $S\in\mathbb{R}^{L\times M}$ is the endmember spectral
signature matrix containing $M$ endmembers. $A\in\mathbb{R}^{M\times N}$ is
the abundances matrix. Here, $N\in\mathbb{R}^{L\times N}$ is the noise
matrtix.

In addition to that, three physical constraints need to be satisfied in the
unmixing problem. More precisely, the endmember matrix should be non-negative.
In addition to that the abundance vectors $\mathbf{a_{j}}$ should satisfy the
ANC and the ASC by the following equations:

$\mathbf{a_{j}}\geq 0\;\;\;\;\;\;\forall\;1\leq j\leq M$ (3)

$\sum\limits_{j=1}^{M}\boldmath{(a_{j})}=1$ (4)

####  II-A3 Autoencoder

Autoencoder (AE)is a type of artificial neural network designed for
unsupervised learning that learns low-dimensional representations of data by
encoding and reconstructing input patterns. Comprising an encoder and a
decoder, it aims to capture essential features, reducing dimensionality while
maintaining the crucial information needed for accurate reconstruction.

Encoder: The encoder transforms the ${\mathbf{i}^{th}}$ input vector
$X_{i}\in\mathbb{R}^{L}$ into a hidden representation $V_{i}\in\mathbb{R}^{M}$
by utilizing some trainable network parameters. In general, this can be
expressed as,

$V_{i}=f_{E}(X_{i})$ (5)

where ${f_{E}}$ denotes the encoder transformation.

Decoder: In an AE architecture, the decoder is responsible for reconstructing
the original data from the low-dimensional representation produced by the
encoder network. Usually, in the context of HS Unmixing, the decoder is
responsible for reconstructing the original pixel from the abundance values
produced by the encoder network.

In our proposed method, a modified Autoencoder architecture is used.

###  II-B Proposed Method

TABLE I: The Proposed Architecture’s Layer Description Network | Layers | Dimensions
---|---|---
Number of heads | Q | K | V | Input | Output
Approximation Network | Multihead Attention | $H_{\lx@glossaries@gls@link{acronym}{an}{\leavevmode AN}}$ |  $L\times$B | $L\times 1$ |  $L\times$B | N/A | $L\times 1$
Abundance Predictor | Linear Layer | N/A | N/A | N/A | N/A | $L\times 1$ | $L\times 1$
Multihead Attention | $H_{\lx@glossaries@gls@link{acronym}{ap}{\leavevmode AP}}$ | $H_{\lx@glossaries@gls@link{acronym}{ap}{\leavevmode AP}}\times L$ | $H_{\lx@glossaries@gls@link{acronym}{ap}{\leavevmode AP}}\times L$ | $H_{\lx@glossaries@gls@link{acronym}{ap}{\leavevmode AP}}\times L$ | N/A | $H_{\lx@glossaries@gls@link{acronym}{ap}{\leavevmode AP}}\times L$
Linear Layer | N/A | N/A | N/A | N/A | $L\times 1$ | $M\times 1$
Signature Predictor |  Single Head Attention $\times$ $M$ | N/A | $L\times R$ | $L\times 1$ | $L\times R$ | N/A | $L\times 1$

The proposed architecture consists of three main components, namely
Approximation Network (AN), Abundance Predictor (AP), and Signature Predictor
(SP). The architecture is shown in Figure 1. AN creates a context-aware
spectral signature by taking a pixel and its neighboring pixels as inputs. AP
takes the context-aware spectral signature, that is produced by AN and
predicts the abundances of the considered pixel. SP’s task is to predict
endmember signatures by fusing the endmember signatures that are generated
through different EEA s. A layer-wise description of the architecture is
summarized in table I. In the following sections, these components are
discussed in detail.

####  II-B1 Approximation Network (AN)

The Approximation Network (AN)is for predicting spectral signatures that
consist of spatial context. The multi-head attention layer with
$H_{\lx@glossaries@gls@link{acronym}{an}{\leavevmode AN}}$ heads is used for
this purpose.

Here, a pixel (Which henceforth, will be referred to as the center pixel) is
provided as the query ${(\mathbf{q_{AN}})}$ in the attention layer. The
neighborhood pixels of the center pixel are provided as the keys
${(\mathbf{k_{AN}})}$ as well as values ${(\mathbf{v_{AN}})}$. The output of
the AN is a context-aware representation of the center pixel
(${\widehat{P}_{C}}$).

Initially, provided $\mathbf{q_{AN}}$, $\mathbf{k_{AN}}$ and $\mathbf{v_{AN}}$
are transformed to a new space through learnable linear transformations. it
can be mathematically represented as follows,

$Q_{AN}=W_{AN}^{Q}\mathbf{q}_{AN}$ (6) $K_{AN}=W_{AN}^{K}\mathbf{k}_{AN}$ (7)
$V_{AN}=W_{AN}^{V}\mathbf{v}_{AN}$ (8)

Where, ${W_{AN}^{Q}}$, ${W_{AN}^{K}}$, ${W_{AN}^{V}}$ are the matrices of the
linear transformation. This gives greater adaptability for the attention layer
since it helps to map $\mathbf{q}_{AN}$, $\mathbf{k}_{AN}$ and
$\mathbf{v}_{AN}$ to more desirable space for the next stage.

Afterward, the correlation between the transformed center pixel and the
transformed neighboring pixels is calculated using ${QK^{T}}$ which will
effectively take the dot product between the transformed center pixel and each
transformed neighbor pixel. Then it is scaled by the square root of the number
of bands ${\sqrt{d_{k}}}$. After that, this is passed through a softmax
function to obtain a weights vector known as the attention weights
(${W_{AN}^{A}}$). Finally, it is multiplied by the transformed neighbor pixels
${V_{AN}}$ to get a context-aware representation of the center pixel
${\widehat{P}_{C}}$.

${\widehat{P}_{C}}=W_{AN}^{A}\mathbf{V_{AN}}$ (9)

Figure 1: Graphical representation of the proposed Transformer based Endmember
Fusion with Spatial Context for Hyperspectral Unmixing (FusionNet)model

Strategies Selecting Neighbors: In the proposed FusionNet algorithm, there is
the flexibility to select neighbors in any configuration in contrast to
kernels used in CNNs. For this paper, 3 configurations have been explored.
These are Doughnut configuration, Circle configuration, and Random Normal
configuration (Randomly sampled from a normal distribution). Their shapes are
shown in Figure 2(a).

####  II-B2 Abundance Predictor (AP)

Abundance Predictor (AP)is for predicting endmember abundances from a given
context-aware spectral signature produced by the AN. To achieve that, self-
attention is used since giving attention to the context-aware spectral
signature itself enables one to grasp the properties of the spectrum to
predict the end member abundances. A detailed layer-wise explanation is as
follows.

The AP consists of three main blocks, a linear layer, a multi-head attention
layer, and another linear layer. The input to the AP (and also the output from
AN) is a vector of the form $x_{1}\in\mathbb{R}^{L}$. Then, this is fed
through the first linear layer which is of the form
$W_{1}\in\mathbb{R}^{L\times L}$. This results in a vector
$x_{2}\in\mathbb{R}^{L}$, which is then fed to the multi-head attention layer
and self-attention is computed. The number of heads is given by
$H_{\lx@glossaries@gls@link{acronym}{ap}{\leavevmode AP}}$. This means that
$x_{2}$ is passed as the $Q,K,V$ values in the attention layer. The output
from the attention layer is another vector $x_{3}\in\mathbb{R}^{L}$. Then
there is a residual connection from the output of $W_{1}$ and this is
implemented as $x_{2}+x_{3}$. This final value is sent through the final
linear layer $W_{2}\in\mathbb{R}^{L\times M}$ where $M$ is the number of
endmembers. Finally, The softmax function is applied to the resulting vector
to obtain the predicted abundance values $\widehat{A}\in\mathbb{R}^{M}$. The
softmax ensures that the abundance values satisfy the non-negativity and the
sum-to-one constraints.

As a summary, the function done by AP is given in equation 10

$\widehat{A}=f_{AP}(\widehat{P}_{C})$ (10)

Donut

(a) *

Circle

(b) *

Random Normal

(c) * Figure 2: Different Neighborhood Selecting Criteria

[.3] [.3] [.3]

####  II-B3 Signature Predictor (SP)

The Signature Predictor (SP)is responsible for fusing the endmember
signatures. The inputs of the SP are the spectral signatures extracted by
different EEA s and the outputs of SP are better endmember spectral signatures
for each endmember. Using the attention mechanism, a weighted linear
combination of the different signatures obtained by the EEA s is taken to
predict the final signatures. However, just linearly combining the initial
signatures obtained by the EEA s does not provide enough flexibility to the
network. Therefore, the signatures are allowed to be trained as well while
enabling _spectral band-selective learning_. In the following section, a more
detailed mathematical description of the model is presented.

Let $R$$\in\mathbb{Z^{+}}$, be the number of EEA s that are used for fusion
and $M$ is the number of endmembers as mentioned before. Initially, $M$ number
of matrices $\\{W_{i}\\}_{i=1}^{M}$ where $W_{i}\in\mathbb{R}^{L\times R}$ are
created, one for each endmember. Consider the matrix $W_{i}$ corresponding to
the $i^{th}$ endmember. This matrix consists of $R$ columns where the $r^{th}$
column is the signature computed by the $r^{th}$ EEA for the $i^{th}$
endmember. Therefore this matrix contains $R$ columns and $L$ rows where $L$
is the number of bands as mentioned before.

Here, each single-head attention layer is used for predicting each endmember
signature. In other words, $i^{th}$ single head attention layer predicts the
$i^{th}$ endmeber. The $i^{th}$ single head attention layer takes $W_{i}$ as
the key (${k_{SP_{i}}}$) as well as the value (${v_{SP_{i}}}$). For the query
(${q_{SP_{i}}}$) of the $i^{th}$ single head attention layer learnable
parameter $P_{i}\in\mathbb{R}^{L}$ is used. There is M number of learnable
parameters ($\\{P_{i}\\}_{i=1}^{M}$), one for each single head attention
layer.

For further study, consider the $i^{th}$ single-head attention layer
predicting the $i^{th}$ endmember (The process is identical for all the
endmembers)

Before passing ${q_{SP_{i}}}$,${k_{SP_{i}}}$, and ${v_{SP_{i}}}$ to the
attention layer, these will be transformed into another space through linear
transformations. It can be mathematically represented as follows,

$Q_{SP_{i}}=W_{SP_{i}}^{Q}\mathbf{q}_{SP_{i}}$ (11)
$K_{SP_{i}}=W_{SP_{i}}^{K}\mathbf{k}_{SP_{i}}$ (12)
$V_{SP_{i}}=W_{SP_{i}}^{V}\mathbf{v}_{SP_{i}}$ (13)

where, ${W_{SP_{i}}^{Q}}$, ${W_{SP_{i}}^{K}}$, ${W_{SP_{i}}^{V}}$ are the
matrices of the linear transformation. This gives greater adaptability for the
attention layer. This will be further discussed in Section II-D.

Here, the dot product is taken between ${Q_{SP_{i}}}$ and ${K_{SP_{i}}}$.
After that, it is scaled by ${\sqrt{d_{k}}}$, and the softmax of the result is
computed. This results in a vector namely ${W_{SP_{i}}^{A}}$ which contains K
elements. These elements give attention weights for each EEA for the endmember
under consideration. Then, the final $i^{th}$ endmember (${\widehat{S}_{i}}$)
is predicted by taking the weighted sum of ${V_{SP_{i}}}$, weighted by the
attention weights. The exact mechanism of this architecture will be further
clarified under Section II-D because the workings of this mechanism are
greatly dependent on the particulars of the training process.

${\widehat{S_{i}}}=W_{SP_{i}}^{A}\mathbf{V_{SP_{i}}}$ (14)

####  II-B4 Pixel Reconstruction

Since LMM is used for this unmixing process, the matrix product between the
predicted abundances ${\widehat{A}}$ by the Abundances Predictor and the
predicted endmember signatures ${\widehat{S}}$ (whose $i^{\text{th}}$ column
is the $i^{\text{th}}$ predicted endmember $\widehat{S}_{i}$) by the SP gives
the reconstructed pixel ${\widehat{P}_{RC}}$.

${{\widehat{P}_{RC}}=\widehat{S}\;\widehat{A}}$ (15)

###  II-C CONSTRAINS

Before discussing how each of the constraints was used in the different
stages, a brief explanation of the mathematical formulation and the conceptual
interpretation of the constraints are presented.

####  II-C1 Mean Square Error (MSE)

Mean Square Error (MSE) is the square of the Euclidean distance of two vectors
as given in the equation 16

$L_{MSE}(\widehat{x},x)=\frac{1}{N}\sum\limits_{i=1}^{N}(\widehat{x_{i}}-x_{i})^{2}$
(16)

####  II-C2 Spectral Angle Distance (SAD)

Spectral Angle Distance is the angular distance between two vectors and it is
calculated as given in the equation 17

$\displaystyle
L_{SAD}(\mathbf{\widehat{x}},\mathbf{x})=\arccos\left({\frac{\mathbf{\widehat{x}^{T}}\mathbf{x}}{||\mathbf{\widehat{x}}||\;||\mathbf{x}||}}\right)$
(17)

where $||\cdot||$ reprensts the 2-norm of the vector.

####  II-C3 Endmember Simplex Volume Constrain (ESVC)

Guidance by ASC and ANC constraints is not enough to guarantee proper
convergence of the SP to the accurate end member signatures. This is because
there is a large solution space that satisfies the above constraints. The
desired solution in the solution space is the endmember signatures that form
the minimum volume simplex [51], [34], [35]

To address this problem, we constrain the volume of the simplex formed by the
endmembers, to be less than a certain threshold (How this threshold is
determined will be explained later). To calculate the simplex volume, the
method proposed by the N-FINDR paper [18] was used. The mathematical
formulation of the method is presented below.

$\displaystyle\mathbf{E}$ $\displaystyle=\begin{bmatrix}1&1&\dots&1\\\
e_{1}&e_{2}&\dots&e_{M}\end{bmatrix}$ (18)
$\displaystyle\text{Vol}(\mathbf{E})$
$\displaystyle=\frac{1}{(M-1)!}\text{abs}(|\mathbf{E}|)$ (19)

$|\cdot|$ represents the determinant of the matrix. $E\in\mathbf{R}^{M\times
M}$ is formed by $\\{e_{i}\\}_{i=1}^{M}$ which are the endmembers porjected to
a $(M-1)$ dimensional space. PCA fitted to the pixel space was used to
calculate the projection matrix used for this dimensionality reduction.

The loss is calculated by taking the ReLU of the difference between the
threshold value and the calculated volume of the simplex.

$\displaystyle L_{ESVC}(\mathbf{E},V_{T})$
$\displaystyle=ReLU\left(\text{Vol}(\mathbf{E})-V_{T}\right)$ (20)

###  II-D TRAINING PROCESS

The architecture is trained in three stages. These are, Training the
Approximation Network, the Frozen Training, and the Unfrozen Training.

####  II-D1 Training the Approximation Network

Here, the Approximation Network is trained with the objective of minimizing
the Mean Square Error (MSE) between the center pixel and the predicted
context-aware spectral signature. The number of epochs was optimized to
prevent the network from over-fitting to the center pixel since it reduces the
amount of context data included in the context-aware spectral signature.

$L_{AP}(\widehat{P}_{C},P_{C})=L_{MSE}(\widehat{P}_{C},P_{C})$ (21)

####  II-D2 Frozen Training

In the training process, it should be ensured that the abundance predictor
predicts the proper abundance. But simultaneously training the abundance
predictor and SP does not ensure this. This is because the initial output from
the untrained SP which is still untrained is most likely inaccurate.
Therefore, the necessary guidance for the abundance predictor is not achieved.
This may lead the network astray and cause it to converge to a local minimum
while giving less accurate abundances and signatures.

To prevent this and to give the abundance predictor the necessary guidance, a
technique which henceforth will be referred to as freezing the SP is used.
This technique allows us to ensure that the SP outputs a Linear Combination of
the initial signatures obtained by different EEA s for each endmember
throughout this training process. In addition to this, the weights used for
taking the linear combination are updated in the training process. How this is
achieved is explained in detail in the following paragraphs.

Ensuring the SP outputs a Linear Combination of the initial signatures is
achieved by keeping the ${W_{SP_{i}}^{V}}$ fixed to the identity matrix
throughout the training process. This ensures that the matrix $V_{SP_{i}}$ is
the initial signatures obtained by the different EEA s. This means the
predicted signature ($\widehat{S_{i}}$) is a linear combination of the initial
signatures predicted by the EEA s according to equation (14)

To reduce the negative effect due to the faulty signature computed by certain
EEA s for certain endmembers, the weights used to take the linear combination,
need to be updated throughout this training stage as mentioned previously.
This is achieved by making the $P_{i}$ learnable parameters trainable.

Here, we use the weighted sum of the MSE and SAD loss between the actual pixel
(${{P}_{C}}$). and the reconstructed pixel (${\widehat{P}_{RC}}$) and the
equation is given below.

$L_{AP}(\widehat{P}_{C},P_{C})=\alpha L_{MSE}(\widehat{P}_{C},P_{C})+\beta
L_{SAD}(\widehat{P}_{C},P_{C})$ (22)

####  II-D3 Unfrozen Training

By this stage, the abundance predictor has been guided to a reasonable level.
Since AP has also been trained to a certain level, the SP is capable of fusing
signatures obtained by EEA s to produce more accurate signatures. However, by
this stage, signatures themselves have not been trained.

In this stage, we allow all the SP weights to update freely (This is in
addition to the weights that were trained in the frozen stage both in the
abundance predictor and the SP). By doing so, ${W_{SP_{i}}^{V}}$ updates which
means $V_{SP_{i}}$ is the transformed version of the original signatures,
provided by EEA s. This means that the signatures used in the fusion get
updated during the training process.

For this stage, MSE, SAD, and ESVC are used. The total loss is calculated as
follows

$\displaystyle L_{AP}(\widehat{P}_{C},P_{C},\mathbf{E},V_{T})$
$\displaystyle=\alpha L_{MSE}(\widehat{P}_{C},P_{C})+\beta
L_{SAD}(\widehat{P}_{C},P_{C})$ $\displaystyle\;\;\;\;+\gamma
L_{ESVC}(\mathbf{E},V_{T})$ (23)

The threshold value required for ESVC loss in the above equation is calculated
by getting the volume of the simplex created by the endmembers predicted at
the end of the Frozen training process. This volume is calculated according to
the equation 19

In summary, at the frozen stage, the signatures obtained by the different EEA
s for a certain endmember, are fused. in the unfrozen stage, In addition to
the signature fusion, signatures are allowed to update themselves.

TABLE II: Optimum hyper-parameters of FusionNet for each dataset Dataset | Epochs (AN) | Epochs (Frozen) | Epochs (Unfrozen) | SAD | MSE | ESVC | Non-neg
---|---|---|---|---|---|---|---
Samson | 100 | 1000 | 500 | 1.125 | 1 | 0.0025 | 1e-8
Jasper | 200 | 1000 | 350 | 1e-5 | 1 | 5e-5 | 1e-6

##  III EXPERIMENTS

For the performance evaluation, Samson and Jasper-Ridge datasets have been
used. The RGB images of the datasets are shown in Figure 3(a). For the results
comparison the State-Of-The-Art algorithms such as DeepTrans [33], MiSiCNet
[52], CyCU [41], CNNAEU [53], KbSNMF [21], and DAEU [40] has been used.

Samson

(a) *

Jasper-Ridge

(b) * Figure 3:  The RGB images of the datasets

[.45] [.45]

###  III-A Datasets

####  III-A1 Samson

Samson dataset is a widely used dataset in HS unmixing. It consists of three
endmembers namely, tree, sand, and water. The number of bands present in the
dataset is 156 covering from 401nm to 889nm with a resolution up to 3.13nm.
The original image has 952x952 pixels. For simplicity, a patch of 95x95 has
been used.

####  III-A2 Jasper Ridge

The The Jasper Ridge dataset is another commonly used dataset. The original
image has 512x614 pixels. As it is too complex to get the ground truth, a
subimage of 100x100 is considered. In the original image, there are 224
channels from 380nm to 2500nm with a resolution of up to 9.46nm. After
removing 1-3, 108-112, 154-166, and 220-224 bands due to dense water vapor and
atmospheric effects there are 198 channels in the HS image. The endmembers are
road, soil, water, and trees that are present in this HS image.

###  III-B EXPERIMENT SETUP

####  III-B1 Evaluation Metrics

For the performance comparison, the Root Mean Square Error (RMSE)and Spectral
Angle Distance (SAD)values have been compared for each endmember and the
average value. RMSE reflects the abundance maps accuracy. It calculates the
root mean squared error between the generated abundance maps and the ground
truth abundance maps. The equation is shown in 24. SAD evaluates the accuracy
of the endmember signatures. The comparing parameter in SAD is the angle
distance between the ground truth signature and the generated signature. The
equation is shown in 25.

$RMSE(\widehat{\mathbf{a}},\mathbf{a})=\sqrt{\frac{1}{N}\sum\limits_{i=1}^{N}(\widehat{\mathbf{a_{i}}}-\mathbf{a_{i}})^{2}}$
(24)

$\displaystyle
SAD(\mathbf{\widehat{x}},\mathbf{x})=\arccos\left({\frac{\mathbf{\widehat{x}^{T}}\mathbf{x}}{||\mathbf{\widehat{x}}||\;||\mathbf{x}||}}\right)$
(25)

Figure 4: Comparison of the abundance maps obtained by the different unmixing algorithms for Samson dataset Figure 5: Comparison of the endmember spectral signatures obtained by the different unmixing algorithms for Samson dataset TABLE III: Unmixing performance comparison for different Hyperspectral (HS)Unmixing algorithms for Samson Dataset. Error | Endmember | Fusion | Transformer | MiSiCNet | CyCU | CNNAEU | KbSNMF | DAEU
---|---|---|---|---|---|---|---|---
RMSE | Soil | 0.04421 |  0.17173 | 0.1818 | 0.2372 | 0.1924 |  0.15742 | 0.3042
Tree | 0.02661 |  0.17203 | 0.1791 | 0.2635 | 0.1846 |  0.09112 | 0.5845
Water | 0.02581 |  0.17153 | 0.3131 | 0.1918 | 0.1744 |  0.09272 | 0.2462
Average | 0.03331 |  0.17173 | 0.2332 | 0.2327 | 0.1839 |  0.11372 | 0.4061
SAD | Soil |  0.01173 |  0.01082 | 0.01031 | 0.0649 | 0.0512 | 0.2078 | 0.9611
Tree |  0.03092 | 0.0425 |  0.03543 | 0.02751 | 0.0408 | 0.0647 | 0.7004
Water | 0.03261 |  0.05682 | 0.4011 |  0.10713 | 0.1544 | 0.2014 | 1.4270
Average | 0.02501 |  0.03672 | 0.1489 |  0.06653 | 0.0821 | 0.1580 | 1.0295

####  III-B2 Endmember Extraction Algorithm for Fusion

As discussed in the methodology section, SP uses endmember signatures
extracted by different EEAs to fuse and predict better spectral signatures.
Technically, SP can fuse endmember signatures from any number of EEAs. In this
paper, 3 key EEAs are used for demonstration which are VCA [19], N-FINDR [18],
and ATGP.

Initially, every combination of these 3 EEAs has been used to find the best
combination that gives the best RMSE and SAD results. For the Samson dataset,
the obtained results are tabulated in Table V.

It can be clearly seen that fusing all EEAs gives the best RMSE and SAD.
Therefore that combination is used for following sections.

####  III-B3 Hyperparameter Settings

The used hyperparameters in the proposed FusionNet algorithm are listed below:
The layers for each section in taken to align with the conceptual idea that
has been discussed in the methodology. Adam optimizer is used for each
training stage and the learning rate is taken as $10^{-4}$. The input batch
size is 400. Apart from these, the optimum weights of the SAD, MSE, Non-
negative and Min-vol losses in the loss function were different from dataset
to dataset. Also, the number of epochs for each training stage differs
according to the dataset. The optimum values are given in the Table II for
each dataset.

RMSE VS SNR

(a) * (c) *

[.25][.25] [.1]

SAD VS SNR

(b) *

Figure 6: Comparison of the results by the different unmixing algorithms for
Samson dataset under 20dB, 10dB and 5dB SNR

###  III-C Experiment with Samson dataset

TABLE IV: Performance of FusionNet under different levels of noise for Samson dataset (Tested 10 times) | Original Image | 20dB SNR | 10dB SNR | 5dB SNR
---|---|---|---|---
Error | Endmember | Mean | Std | Mean | Std | Mean | Std | Mean | Std
RMSE | Soil | 0.0632 | 0.0093 | 0.0714 | 0.0103 | 0.8790 | 0.0100 | 0.1181 | 0.0057
Tree | 0.0361 | 0.0058 | 0.0575 | 0.0089 | 0.0854 | 0.0076 | 0.1282 | 0.0039
Water | 0.0351 | 0.0064 | 0.0497 | 0.0059 | 0.1034 | 0.0077 | 0.1397 | 0.0049
Average | 0.0467 | 0.0069 | 0.0604 | 0.0064 | 0.0927 | 0.0072 | 0.1290 | 0.0036
SAD | Soil | 0.0123 | 0.0017 | 0.0332 | 0.0050 | 0.0830 | 0.0057 | 0.1218 | 0.0068
Tree | 0.0328 | 0.0021 | 0.0526 | 0.0047 | 0.0982 | 0.0190 | 0.1570 | 0.0112
Water | 0.0330 | 0.0019 | 0.1653 | 0.0070 | 0.2721 | 0.0144 | 0.2861 | 0.0051
Average | 0.0260 | 0.0009 | 0.0837 | 0.0030 | 0.1511 | 0.0055 | 0.1883 | 0.0028
TABLE V: Unmixing performance comparison for the three EEAs combinations for Samson dataset. Error | Endmember | VCA | NFINDR | ATGP | VCA + ATGP | VCA + NFINDR | ARGP + NFINDR | All EEAs
---|---|---|---|---|---|---|---|---
RMSE | Soil | 0.0645 | 0.1030 | 0.1605 | 0.0552 | 0.0888 | 0.0712 | 0.0442
Tree | 0.0334 | 0.0908 | 0.1192 | 0.0317 | 0.0701 | 0.0446 | 0.0266
Water | 0.0406 | 0.0374 | 0.1199 | 0.0317 | 0.0443 | 0.0414 | 0.0258
Average | 0.0481 | 0.0822 | 0.1346 | 0.0410 | 0.0701 | 0.0541 | 0.0333
SAD | Soil | 0.0203 | 0.0764 | 0.1568 | 0.0132 | 0.0560 | 0.0300 | 0.0117
Tree | 0.0328 | 0.0334 | 0.0387 | 0.0325 | 0.0335 | 0.0352 | 0.0309
Water | 0.0391 | 0.0359 | 0.1440 | 0.0311 | 0.0416 | 0.0326 | 0.0326
Average | 0.0307 | 0.0485 | 0.1132 | 0.0256 | 0.0437 | 0.0326 | 0.0250
TABLE VI: Perfomance of FusionNet Network without the Approximation Network for Samson dataset Error Metric | Soil | Tree | Water | Average
---|---|---|---|---
RMSE | 0.0656 | 0.0448 | 0.0293 | 0.0489
SAD | 0.0136 | 0.029 | 0.0328 | 0.0251

####  III-C1 Perfomance Comparison with the State-Of-The-Art Algorithms

For the Samson dataset, the abundance maps and the endmember signatures are
shown in Figure 4 and Figure 5 respectively. Numerical value comparison is
shown in Table II and time taken for training and the prediction are tabulated
in VIII.

In terms of RMSE, the proposed algorithm stands out with the best results for
all endmembers. Except for the SAD values associated with soil and tree
endmembers, where MiSiCNet and CyCU algorithms exhibit slightly superior
performance, the proposed method demonstrates improved results for the water
endmember and, on average, outperforms the other algorithms.

####  III-C2 Noise Robustness

In this section, the robustness of each algorithm under different noise
conditions is analyzed. To perform this, the original Samson HS image with
added noise is used. The noise is added to generate images with Signal-to-
Noise-Ratios (SNRs) of 20 dB, 10 dB, and 5 dB. The results obtained from each
algorithm for these images are shown in figure 6.

The best results for RMSE was given by the proposed algorithm for each noise
level with a noticeable margin. Considering SAD, for 5dB SNR and 10 dB SNR,
the proposed algorithm gives the best results and it is slightly better than
DeepTrans. For 20 dB SNR, DeepTrans surpasses the proposed algorithm by a
slight margin. In summary, the robustness to noises of the proposed method is
higher compared to other algorithms

Furthermore, a detailed analysis of the performance of the proposed algorithm
under different noise conditions was conducted by running the algorithm 10
times for each noise level. The mean and standard deviation values for each
error metric are tabulated in Table IV.

###  III-D Experiment with Jasper-Ridge Dataset

####  III-D1 Perfomance Comparison with the State-Of-The-Art Algorithms

For the Jasper-Ridge dataset, the ground truth abundance maps and the
predicted abundance maps for each algorithm are shown in Figure 8. The
comparison of the predicted signatures with the ground truth for each
algorithm is given in Figure 7. A comparison of the different error metrics
for each algorithm is given in the table IX and the times taken for training
and the prediction are tabulated in VIII.

When considering the RMSE of the abundance, the proposed method has given the
best results for each endmember as well as for the average value. For most
cases the margin is noticeable. On the other hand, for SAD, the proposed
algorithm gives the second-best average results, while achieving the best SAD
value for soil and road endmembers among all the algorithms. These two
endmembers are poorly extracted by EEAs and further discussion has been done
in Section III-D2

TABLE VII: Ablation study for the performance of FusionNet under different levels of noise for Samson dataset | Doughnut | Circle | Normal
---|---|---|---
Error | Endmember | 3 | 4 | 5 | 3 | 4 | 5 | 3 | 4 | 5
RMSE | Soil | 0.0516 | 0.0703 | 0.0754 | 0.0740 | 0.0443 | 0.0553 | 0.1217 | 0.0932 | 0.0975
Tree | 0.0306 | 0.0409 | 0.0503 | 0.0406 | 0.0267 | 0.0328 | 0.1267 | 0.0844 | 0.0799
Water | 0.0267 | 0.0415 | 0.0376 | 0.0438 | 0.0258 | 0.0308 | 0.0367 | 0.0268 | 0.0389
Average | 0.0379 | 0.0528 | 0.0566 | 0.0549 | 0.0334 | 0.0412 | 0.1036 | 0.0742 | 0.0762
SAD | Soil | 0.0122 | 0.0103 | 0.0119 | 0.0170 | 0.0117 | 0.0142 | 0.0157 | 0.0141 | 0.0335
Tree | 0.0326 | 0.0339 | 0.0326 | 0.0368 | 0.0309 | 0.0333 | 0.0311 | 0.0294 | 0.0298
Water | 0.0331 | 0.0363 | 0.0310 | 0.0301 | 0.0326 | 0.0302 | 0.0312 | 0.0305 | 0.0387
Average | 0.0260 | 0.0268 | 0.0252 | 0.0280 | 0.0251 | 0.0259 | 0.0260 | 0.0246 | 0.0340
TABLE VIII: Times taken by each dataset for the training and the prediction Dataset | AN Training (s) | AP Training (s) | SP Training (s) | AP Prediction (s) | SP Prediction (s)
---|---|---|---|---|---
Samson | 117.07 | 440.53 | 245.71 | 0.0285 | 0.0031
Jasper-Ridge | 254.04 | 662.01 | 243.18 | 0.0495 | 0.0035

####  III-D2 Performance of the Signature Predictor

Figure 7: Comparison of the abundance maps obtained by the different unmixing algorithms for Jasper-Ridge dataset Figure 8: Comparison of the endmember spectral signatures obtained by the different unmixing algorithms for Jasper-Ridge dataset TABLE IX: Unmixing performance comparison for different Hyperspectral (HS)Unmixing algorithms for Jasper Dataset. Error | Endmember | Fusion | Transformer | MiSiCNet | CyCU | CNNAEU | KbSNMF | DAEU
---|---|---|---|---|---|---|---|---
RMSE | Tree | 0.08221 | 0.1388 |  0.09542 |  0.10823 | 0.2172 | 0.1091 | 0.1633
Water | 0.08071 |  0.09013 | 0.1141 | 0.1044 | 0.1454 |  0.08242 | 0.1272
Soil | 0.07761 |  0.20683 | 0.2216 |  0.16902 | 0.3256 | 0.2207 | 0.2099
Road | 0.10481 |  0.15453 | 0.2487 |  0.13622 | 0.2383 | 0.1747 | 0.2165
| Average | 0.08541 |  0.15333 | 0.1824 |  0.13202 | 0.2404 | 0.1565 | 0.1829
SAD | Tree | 0.1222 |  0.05312 | 0.04341 |  0.06623 | 0.0818 | 0.1986 | 0.1807
Water | 0.1355 |  0.07433 | 0.2894 | 0.1739 | 0.03341 | 0.1295 |  0.06392
Soil | 0.04271 | 0.0915 | 0.0662 |  0.04772 |  0.05653 | 0.8619 | 0.3453
Road | 0.04211 |  0.06982 | 0.3295 |  0.20713 | 0.5969 | 1.0160 | 0.4180
| Average |  0.08702 | 0.07221 | 0.1821 |  0.12373 | 0.1921 | 0.5515 | 0.2520

Signature Predictor is used to fuse the different EEAs to obtain a better
spectral signature. For this fusion process, the endmember signatures
extracted by N-FINDR, VCA, and ATGP have been used. When comparing the
individual endmember signatures obtained from each EEA with the endmember
signatures that the Signature Predictor predicted, the effectiveness of the
fusion process can be seen and a comparison for the Jasper-Ridge dataset is
given in Figure 9.

In the Jasper-Ridge dataset, all EEAs were able to capture the tree endmember
while only VCA and N-FNDR were able to capture the water endmember. Soil
endmember was captured by both VCA and ATGP EEAs. However, none of these EEAs
were able to capture the road endmember correctly. This is illustrated in
Figure 9.

When considering the proposed method, it was able to extract all the
endmembers correctly by fusing the signatures obtained from the EEAs
regardless of some poor signatures of EEAs. This is a good example to showcase
the synergetic learning of the SP while enabling _spectral band-selective
learning_ for better performance. This implies that even if the EEAs perform
poorly, the SP can predict the endmember signatures correctly.

Figure 9: Comparison of the endmember signatures obtained by FusionNet and
different EEA s for Jasper-Ridge dataset

###  III-E Performance of different network configuration

####  III-E1 Different Neighborhood Settings

The experiment was done with three different neighborhood shapes. They are
doughnuts, circles, and random shapes with a normal distribution. It was found
that the circle shape with a padding level of 4 gives the best performance for
the Samson dataset. A comparison between the shapes and different padding
levels for the Samson dataset is given in Table VII. It can be seen that the
other two shapes also give competitive results.

####  III-E2 The Performance without AN

To study the effect on the performance without the AN, the architecture was
trained by omitting AN. In this variant of the architecture, the center pixel
is directly fed to the AP instead of the context-aware pixel constructed using
the AN. This also means that the Approximation Network (AN)training stage is
not present in this run. The obtained results for the Samson dataset are given
in table VI. It is apparent that the RMSE for abundances is much worse when
compared with the results from the original architecture when observing the
results in table III. The results for SAD are not greatly affected although
slightly worse overall (this deviation is within natural variations). These
results confirm the importance of the construction of the context-aware pixel
using the AN for better endmember abundances.

##  IV Conclusion

In this paper, we propose a novel transformer-based deep learning algorithm
for HS unmixing that is capable of fusing endmember signatures extracted by
different endmember extraction algorithms to predict better endmember
signatures. In addition to that, the proposed method can take spatial context
into account in the training process which leads to higher accuracy in
abundance prediction. This algorithm has been tested for two popular HS
datasets and the results have been compared with the state-of-the-art
algorithms. Using the obtained results, we demonstrated the ability to
leverage the attention mechanism of the transformer to not only inject the
spatial context for better abundance map generation but also synergetically
fuse endmembers obtained from endmember extraction algorithms resulting in
better performance. Therefore, we would assume that this work would promote
more research along with this direction of utilizing transformer-based
endmember fusion for remote sensing.

## References

  * [1] N. Keshava and J.F. Mustard  “Spectral unmixing”  In _IEEE Signal Processing Magazine_ 19.1, 2002, pp. 44–57  DOI: 10.1109/79.974727
  * [2] Loganathan Agilandeeswari et al.  “Crop Classification for Agricultural Applications in Hyperspectral Remote Sensing Images”  In _Applied Sciences_ 12.3, 2022  DOI: 10.3390/app12031670
  * [3] Atiya Khan, Amol D. Vibhute, Shankar Mali and C.H. Patil  “A systematic review on hyperspectral imaging technology with a machine and deep learning methodology for agricultural applications”  In _Ecological Informatics_ 69, 2022, pp. 101678  DOI: https://doi.org/10.1016/j.ecoinf.2022.101678
  * [4] John B. Adams, Milton O. Smith and Paul E. Johnson  “Spectral mixture modeling: A new analysis of rock and soil types at the Viking Lander 1 Site”  In _Journal of Geophysical Research_ 91, 1986, pp. 8098–8112
  * [5] DYL Ranasinghe et al.  “Single-target mineral detection with site-specific endmember extraction for survey points identification: A case study of Jaffna, Sri Lanka”  In _arXiv preprint arXiv:2109.08143_ , 2021
  * [6] Michal Shimoni, Rob Haelterman and Christiaan Perneel  “Hypersectral Imaging for Military and Security Applications: Combining Myriad Processing and Sensing Techniques”  In _IEEE Geoscience and Remote Sensing Magazine_ 7.2, 2019, pp. 101–117  DOI: 10.1109/MGRS.2019.2902525
  * [7] David Stein, Jon Schoonmaker and Eric Coolbaugh  “Hyperspectral imaging for intelligence, surveillance, and reconnaissance”  In _Space and Naval Systems Warfare Center (SSC) San Diego Biennial Review_ 108116, 2001
  * [8] M.E. Schaepman et al.  “Earth system science related imaging spectroscopy - An assessment”  In _Remote Sensing of Environment_ 113.Suppl.1 Elsevier, 2009, pp. S123–S137  DOI: 10.1016/j.rse.2009.03.001
  * [9] Muhammad Jaleed Khan et al.  “Modern Trends in Hyperspectral Image Analysis: A Review”  In _IEEE Access_ 6, 2018, pp. 14118–14129  DOI: 10.1109/ACCESS.2018.2812999
  * [10] Jin-Fan Hu et al.  “Hyperspectral Image Super-Resolution via Deep Spatiospectral Attention Convolutional Neural Networks”  In _IEEE Transactions on Neural Networks and Learning Systems_ 33.12, 2022, pp. 7251–7265  DOI: 10.1109/TNNLS.2021.3084682
  * [11] Jianwen Hu, Yuan Tang, Yaoting Liu and Shaosheng Fan  “Hyperspectral Image Super-Resolution Based on Multiscale Mixed Attention Network Fusion”  In _IEEE Geoscience and Remote Sensing Letters_ 19, 2022, pp. 1–5  DOI: 10.1109/LGRS.2021.3124974
  * [12] Azam Karami, Rob Heylen and Paul Scheunders  “Hyperspectral image noise reduction and its effect on spectral unmixing”  In _2014 6th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS)_ , 2014, pp. 1–4  DOI: 10.1109/WHISPERS.2014.8077632
  * [13] Behnood Rasti et al.  “Noise Reduction in Hyperspectral Imagery: Overview and Application”  In _Remote Sensing_ 10.3, 2018  DOI: 10.3390/rs10030482
  * [14] D.C. Heinz and Chein-I-Chang  “Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imagery”  In _IEEE Transactions on Geoscience and Remote Sensing_ 39.3, 2001, pp. 529–545  DOI: 10.1109/36.911111
  * [15] José M. Bioucas-Dias et al.  “Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches”  In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 5.2, 2012, pp. 354–379  DOI: 10.1109/JSTARS.2012.2194696
  * [16] S Liangrocapart and Maria Petrou  “Mixed pixels classification”  In _Image and Signal Processing for Remote Sensing IV_ 3500, 1998, pp. 72–83  SPIE
  * [17] Rob Heylen, Mario Parente and Paul Gader  “A Review of Nonlinear Hyperspectral Unmixing Methods”  In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 7.6, 2014, pp. 1844–1868  DOI: 10.1109/JSTARS.2014.2320576
  * [18] Michael E. Winter  “N-FINDR: an algorithm for fast autonomous spectral end-member determination in hyperspectral data”  In _Optics & Photonics_, 1999  URL: https://api.semanticscholar.org/CorpusID:64222754
  * [19] J.M.P. Nascimento and J.M.B. Dias  “Vertex component analysis: a fast algorithm to unmix hyperspectral data”  In _IEEE Transactions on Geoscience and Remote Sensing_ 43.4, 2005, pp. 898–910  DOI: 10.1109/TGRS.2005.844293
  * [20] Joseph W. Boardman  “Automating spectral unmixing of AVIRIS data using convex geometry concepts”, 1993  URL: https://api.semanticscholar.org/CorpusID:140591692
  * [21] E…. Ekanayake et al.  “Constrained Nonnegative Matrix Factorization for Blind Hyperspectral Unmixing Incorporating Endmember Independence”  In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 14, 2021, pp. 11853–11869  DOI: 10.1109/JSTARS.2021.3126664
  * [22] Jessica D Bayliss, J Anthony Gualtieri and Robert F Cromp  “Analyzing hyperspectral data with independent component analysis”  In _26th AIPR workshop: Exploiting new image sources and sensors_ 3240, 1998, pp. 133–143  SPIE
  * [23] Bhathiya Rathnayake et al.  “Graph-Based Blind Hyperspectral Unmixing via Nonnegative Matrix Factorization”  In _IEEE Transactions on Geoscience and Remote Sensing_ 58.9, 2020, pp. 6391–6409  DOI: 10.1109/TGRS.2020.2976799
  * [24] Xin-Ru Feng et al.  “Hyperspectral Unmixing Using Sparsity-Constrained Deep Nonnegative Matrix Factorization With Total Variation”  In _IEEE Transactions on Geoscience and Remote Sensing_ 56.10, 2018, pp. 6245–6257  DOI: 10.1109/TGRS.2018.2834567
  * [25] Shaoquan Zhang et al.  “Spectral-Spatial Hyperspectral Unmixing Using Nonnegative Matrix Factorization”  In _IEEE Transactions on Geoscience and Remote Sensing_ 60, 2022, pp. 1–13  DOI: 10.1109/TGRS.2021.3074364
  * [26] Le Dong, Yuan Yuan and Xiaoqiang Luxs  “Spectral–Spatial Joint Sparse NMF for Hyperspectral Unmixing”  In _IEEE Transactions on Geoscience and Remote Sensing_ 59.3, 2021, pp. 2391–2402  DOI: 10.1109/TGRS.2020.3006109
  * [27] Tao Chen et al.  “Superpixel-Based Collaborative and Low-Rank Regularization for Sparse Hyperspectral Unmixing”  In _IEEE Transactions on Geoscience and Remote Sensing_ 60, 2022, pp. 1–16  DOI: 10.1109/TGRS.2022.3177636
  * [28] Longfei Ren, Zheng Ma, Francesca Bovolo and Lorenzo Bruzzone  “A nonconvex framework for sparse unmixing incorporating the group structure of the spectral library”  In _IEEE Transactions on Geoscience and Remote Sensing_ 60 IEEE, 2021, pp. 1–19
  * [29] Xinxin Zhang, Yuan Yuan and Xuelong Li  “Sparse unmixing based on adaptive loss minimization”  In _IEEE Transactions on Geoscience and Remote Sensing_ 60 IEEE, 2022, pp. 1–14
  * [30] Min Zhao, Mou Wang, Jie Chen and Susanto Rahardja  “Hyperspectral Unmixing for Additive Nonlinear Models With a 3-D-CNN Autoencoder Network”  In _IEEE Transactions on Geoscience and Remote Sensing_ 60, 2022, pp. 1–15  DOI: 10.1109/TGRS.2021.3098745
  * [31] Rui Guo, Wei Wang and Hairong Qi  “Hyperspectral image unmixing using autoencoder cascade”  In _2015 7th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS)_ , 2015, pp. 1–4  DOI: 10.1109/WHISPERS.2015.8075378
  * [32] Xia Xu et al.  “Deep Autoencoder for Hyperspectral Unmixing via Global-Local Smoothing”  In _IEEE Transactions on Geoscience and Remote Sensing_ 60, 2022, pp. 1–16  DOI: 10.1109/TGRS.2022.3152782
  * [33] Preetam Ghosh et al.  “Hyperspectral Unmixing Using Transformer Network”  In _IEEE Transactions on Geoscience and Remote Sensing_ 60, 2022, pp. 1–16  DOI: 10.1109/TGRS.2022.3196057
  * [34] M.D. Craig  “Minimum-volume transforms for remotely sensed data”  In _IEEE Transactions on Geoscience and Remote Sensing_ 32.3, 1994, pp. 542–552  DOI: 10.1109/36.297973
  * [35] Lidan Miao and Hairong Qi  “Endmember Extraction From Highly Mixed Data Using Minimum Volume Constrained Nonnegative Matrix Factorization”  In _IEEE Transactions on Geoscience and Remote Sensing_ 45.3, 2007, pp. 765–777  DOI: 10.1109/TGRS.2006.888466
  * [36] Pierre Comon  “Independent component analysis, a new concept?”  In _Signal processing_ 36.3 Elsevier, 1994, pp. 287–314
  * [37] Xia Xu et al.  “Simultaneously Multiobjective Sparse Unmixing and Library Pruning for Hyperspectral Imagery”  In _IEEE Transactions on Geoscience and Remote Sensing_ 59.4, 2021, pp. 3383–3395  DOI: 10.1109/TGRS.2020.3016941
  * [38] Heng-Chao Li et al.  “Self-Supervised Robust Deep Matrix Factorization for Hyperspectral Unmixing”  In _IEEE Transactions on Geoscience and Remote Sensing_ 60, 2022, pp. 1–14  DOI: 10.1109/TGRS.2021.3107151
  * [39] Qiwen Jin, Yong Ma, Xiaoguang Mei and Jiayi Ma  “TANet: An Unsupervised Two-Stream Autoencoder Network for Hyperspectral Unmixing”  In _IEEE Transactions on Geoscience and Remote Sensing_ 60, 2022, pp. 1–15  DOI: 10.1109/TGRS.2021.3094884
  * [40] Burkni Palsson, Jakob Sigurdsson, Johannes R. Sveinsson and Magnus O. Ulfarsson  “Hyperspectral Unmixing Using a Neural Network Autoencoder”  In _IEEE Access_ 6, 2018, pp. 25646–25656  DOI: 10.1109/ACCESS.2018.2818280
  * [41] Lianru Gao et al.  “CyCU-Net: Cycle-Consistency Unmixing Network by Learning Cascaded Autoencoders”  In _IEEE Transactions on Geoscience and Remote Sensing_ 60, 2022, pp. 1–14  DOI: 10.1109/TGRS.2021.3064958
  * [42] Ricardo A. Borsoi, Tales Imbiriba and Pau Closas  “Dynamical Hyperspectral Unmixing With Variational Recurrent Neural Networks”  In _IEEE Transactions on Image Processing_ 32, 2023, pp. 2279–2294  DOI: 10.1109/TIP.2023.3266660
  * [43] Burkni Palsson, Johannes R. Sveinsson and Magnus O. Ulfarsson  “Blind Hyperspectral Unmixing Using Autoencoders: A Critical Comparison”  In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 15, 2022, pp. 1340–1372  DOI: 10.1109/JSTARS.2021.3140154
  * [44] Ashish Vaswani et al.  “Attention is All you Need”  In _Advances in Neural Information Processing Systems_ 30 Curran Associates, Inc., 2017  URL: https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
  * [45] Yun Wang, Shuaikai Shi and Jie Chen  “Efficient Blind Hyperspectral Unmixing with Non-Local Spatial Information Based on Swin Transformer”  In _IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium_ , 2023, pp. 5898–5901  DOI: 10.1109/IGARSS52108.2023.10281443
  * [46] Fanqiang Kong et al.  “Window Transformer Convolutional Autoencoder for Hyperspectral Sparse Unmixing”  In _IEEE Geoscience and Remote Sensing Letters_ 20, 2023, pp. 1–5  DOI: 10.1109/LGRS.2023.3308206
  * [47] Yuexin Duan et al.  “UnDAT: Double-Aware Transformer for Hyperspectral Unmixing”  In _IEEE Transactions on Geoscience and Remote Sensing_ 61, 2023, pp. 1–12  DOI: 10.1109/TGRS.2023.3310155
  * [48] Zhiru Yang et al.  “Spatial-Spectral Attention Bilateral Network for Hyperspectral Unmixing”  In _IEEE Geoscience and Remote Sensing Letters_ 20, 2023, pp. 1–5  DOI: 10.1109/LGRS.2023.3295437
  * [49] H…. Wickramathilaka et al.  “Endmember Abundance Prediction in Hyperspectral Unmixing: The Impact of Endmember Extraction Algorithms and Self-Attention in Autoencoders”  In _2023 IEEE 17th International Conference on Industrial and Information Systems (ICIIS)_ , 2023, pp. 524–529  DOI: 10.1109/ICIIS58898.2023.10253564
  * [50] H.M.K.D. Wickramathilaka et al.  “GAUSS: Guided encoder - decoder Architecture for hyperspectral Unmixing with Spatial Smoothness”  In _European Journal of Remote Sensing_ 56.1, 2023, pp. 227–213  DOI: 10.1080/22797254.2023.2277213
  * [51] Andersen Man Shun Ang and Nicolas Gillis  “Algorithms and Comparisons of Nonnegative Matrix Factorizations With Volume Regularization for Hyperspectral Unmixing”  In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 12.12, 2019, pp. 4843–4853  DOI: 10.1109/JSTARS.2019.2925098
  * [52] Behnood Rasti, Bikram Koirala, Paul Scheunders and Jocelyn Chanussot  “MiSiCNet: Minimum Simplex Convolutional Network for Deep Hyperspectral Unmixing”  In _IEEE Transactions on Geoscience and Remote Sensing_ 60, 2022, pp. 1–15  DOI: 10.1109/TGRS.2022.3146904
  * [53] Burkni Palsson, Magnus O. Ulfarsson and Johannes R. Sveinsson  “Convolutional Autoencoder for Spectral–Spatial Hyperspectral Unmixing”  In _IEEE Transactions on Geoscience and Remote Sensing_ 59.1, 2021, pp. 535–549  DOI: 10.1109/TGRS.2020.2992743