# Endmember-Guided Unmixing Network (EGU-Net): A General Deep Learning
Framework for Self-Supervised Hyperspectral Unmixing

Danfeng Hong, Lianru Gao, Jing Yao, Naoto Yokoya, Jocelyn Chanussot, Uta
Heiden, and Bing Zhang This work was supported in part by the National Natural
Science Foundation of China under Grant 42030111 and Grant 41722108, by the
MIAI@Grenoble Alpes (ANR-19-P3IA-0003) and the AXA Research Fund, and by the
Japan Society for the Promotion of Science (KAKENHI 18K18067). (_Corresponding
author: Lianru Gao_)D. Hong is with the Remote Sensing Technology Institute
(IMF), German Aerospace Center (DLR), 82234 Wessling, Germany, and also with
the Univ. Grenoble Alpes, CNRS, Grenoble INP, GIPSA-lab, 38000 Grenoble,
France. (e-mail: danfeng.hong@dlr.de)L. Gao and J. Yao are with the Key
Laboratory of Digital Earth Science, Aerospace Information Research Institute,
Chinese Academy of Sciences, Beijing 100094, China. (e-mail:
<EMAIL_ADDRESS>gaolr@aircas.ac.cn)N. Yokoya is with Department of
Complexity of Science and Engineering, The University of Tokyo and with
Geoinformatics Unit, RIKEN Center for Advanced Intelligence Project (AIP),
Japan. (email: yokoya@k.u-tokyo.ac.jp)J. Chanussot is with the Univ. Grenoble
Alpes, INRIA, CNRS, Grenoble INP, LJK, F-38000 Grenoble, France, and also with
the Aerospace Information Research Institute, Chinese Academy of Sciences,
Beijing 100094, China. (e-mail: jocelyn@hi.is)U. Heiden is with the Remote
Sensing Technology Institute (IMF), German Aerospace Center (DLR), 82234
Wessling, Germany. (e-mail: uta.heiden@dlr.de)B. Zhang is with the Key
Laboratory of Digital Earth Science, Aerospace Information Research Institute,
Chinese Academy of Sciences, Beijing 100094, China, and also with the College
of Resources and Environment, University of Chinese Academy of Sciences,
Beijing 100049, China. (e-mail<EMAIL_ADDRESS>
###### Abstract

This is the pre-acceptance version, to read the final version please go to
IEEE Transactions on Neural Networks and Learning Systems on IEEE Xplore. Over
the past decades, enormous efforts have been made to improve the performance
of linear or nonlinear mixing models for hyperspectral unmixing, yet their
ability to simultaneously generalize various spectral variabilities and
extract physically meaningful endmembers still remains limited due to the poor
ability in data fitting and reconstruction and the sensitivity to various
spectral variabilities. Inspired by the powerful learning ability of deep
learning, we attempt to develop a general deep learning approach for
hyperspectral unmixing, by fully considering the properties of endmembers
extracted from the hyperspectral imagery, called endmember-guided unmixing
network (EGU-Net). Beyond the alone autoencoder-like architecture, EGU-Net is
a two-stream Siamese deep network, which learns an additional network from the
pure or nearly-pure endmembers to correct the weights of another unmixing
network by sharing network parameters and adding spectrally meaningful
constraints (e.g., non-negativity and sum-to-one) towards a more accurate and
interpretable unmixing solution. Furthermore, the resulting general framework
is not only limited to pixel-wise spectral unmixing but also applicable to
spatial information modeling with convolutional operators for spatial-spectral
unmixing. Experimental results conducted on three different datasets with the
ground-truth of abundance maps corresponding to each material demonstrate the
effectiveness and superiority of the EGU-Net over state-of-the-art unmixing
algorithms. The codes will be available from the website:
https://github.com/danfenghong/IEEE_TNNLS_EGU-Net.

###### Index Terms:

Convolutional neural network, deep learning, hyperspectral imagery, self-
supervised, spatial-spectral, spectral unmixing, spectral variability, two-
stream network.

##  I Introduction

Airborne or spaceborne spectroscopy (or hyperspectral) imagery is a remotely
sensed three-dimensional imaging product of collecting hundreds or thousands
of images finely sampled from the continuous electromagnetic spectrum,
yielding very rich spectral information [1]. The resulting narrower swath
width in hyperspectral imagery (HSI) enables the discrimination and detection
of materials of interest at a more accurate level, particularly for some
categories that are extremely similar each other in the range of visual light.
This, therefore, has led to an increasing interest in hyperspectral data
processing and analysis, such as dimensionality reduction [2, 3], image fusion
and enhancement [4, 5, 6], image segmentation and classification [7, 8, 9,
10], spectral unmixing [11, 12, 13, 14, 15, 16], and object / anomaly
detection [17, 18, 19]. Due to the low spatial resolution of HSI, there are a
large number of mixed pixels in HSI, inevitably degrading spectral
discrimination.

(a) linear mixing (b) nonlinear: intimate mixing (c) nonlinear: multilayered
case

Figure 1: Linear and nonlinear mixing scenarios: (a) linear mixing. (b)
nonlinear mixing of intimate mixture. (c) nonlinear mixing of multilayered
scattering: a two-layered case.

Spectral unmixing is usually viewed as an important pre-processing procedure
prior to the high-level data analysis, aiming at simultaneously estimating a
collection of individual components (termed as endmembers) and a series of
corresponding fractional percentages (termed as abundances). In ideal
conditions, that is, there are no additional errors in spectral imaging either
from internal uncertainties of observed spectra or external nonlinear mixing
between different materials, the unmixing process can be perfectly modeled by
a linear mixing model (LMM) [11], as shown in Fig. 1(a). However, the LMM
hardly makes an accurate unmixing in reality, due to the ubiquitous error
inevitably existed in spectral signatures that passively transfers the
unpredicted errors into LMM. In this paper, we uniformly call these general
errors the spectral variability (SV). These SVs might consist of scaling
factors caused by illumination and topography change, complex noise from
environmental condition or instrumental sensors, physically and chemically
atmospheric effects, and nonlinear mixing of materials, which, as often as
not, leads to the deformation of spectral signatures and further makes it very
difficult to accurately unmix these mixed pixels. Up to present, hyperspectral
unmixing (HU), as a variant of blind source separation, is still a hot-spot
and challenging problem in hyperspectral remote sensing.

###  I-A Motivation and Objectives

Due to the existence of the complex SV, powerful nonlinear models are
preferred. Unlike those traditional nonlinear HU approaches that might only
work well in several special cases, highly powerful learning ability of deep
learning (DL) techniques makes it possible to nonlinearly unmix the HSI in a
more generalized way. Most previously-developed DL-based unmixing approaches
basically follow autoencoder (AE)-like architecture. Despite being able to
finely reconstruct the data, these methods tend to generate physically
meaningless endmembers in practice, since there is a lack of effective
guidance for real endmembers in the blind HU. This could further lead to poor
estimation on abundances and high model’s uncertainty resulted from the
sensitivity to SVs.

We have to admit, however, that the DL-based strategy is a feasible and
potential solution for the HU task to make a good trade-off between unmixing
accuracy and generalization ability (against SVs) of the model. Therefore,
this motivates us to develop a general DL-based framework accounting for the
endmember information for high-performance HU.

###  I-B Method Overview and Contributions

Towards the aforementioned goal, an endmember-guided unmixing network (EGU-
Net) is proposed by fully considering the physically meaningful information of
endmembers extracted from HSI. EGU-Net follows a two-stream Siamese deep
network architecture for blind HU in a self-supervised fashion. One stream is
to abstract the properties of endmembers obtained from the hyperspectral scene
in a layer-wise manner and transfer them into another unmixing network stream
via parameter sharing strategy. In addition, non-negativity and sum-to-one
constraints are wisely embedded into network learning in the form of ReLU and
softmax layers, respectively. More generally, the proposed EGU-Net is not only
applicable to pixel-wise HU with deep neural network (DNN) but also to
spatial-spectral HU with convolutional neural network (CNN). More
specifically, main contributions in this paper can be summarized as follows:

  * •

Owing to the powerful learning ability of DL techniques in data representation
and fitting, we heuristically develop a general deep learning framework,
called EGU-Net, to address the issue of nonlinear blind HU in a more effective
and generalized way.

  * •

The proposed general framework consists of two different versions: one is only
considering the pixel-level HU dominated by DNN, another is jointly
considering spatial-spectral information to unmix the HSI with CNN-dominated
architecture.

  * •

Beyond the classic AE-like unmixing architecture, an end-to-end two-stream
deep unmixing network is proposed to model the physically meaningful
properties of real endmembers by the means of a novel self-supervised
strategy. To the best of our knowledge, this is the first time to consider
such endmember information in the DL-based spectral unmixing.

  * •

Due to the lack of ground truth, these unmixing algorithms are hardly able to
be assessed quantitatively. To this end, we provide a new hyperspectral scene
over Munich with functional ground truth for HU. The ground truth can be
generated by means of a simple but feasible processing chain.

The paper is organized as follows. Related work in unmixing is introduced in
Section II. Section III presents the proposed network architecture. Section IV
gives the experimental results and discussion on three datasets. Finally, a
conclusion after presenting a brief summary is drawn in Section V.

Figure 2: An overview of the proposed DL-based self-supervised unmixing
framework (EGU-Net): a pixel-wise version. It consists of two subnetworks:
E-Net and UR-Net, aiming at learning endmember properties and unmixing the
HSI, respectively, and they can be effectively associated with shared weights
in a layer-wise way. The abundances in E-Net are initialized by some existing
unmixing models, i.e. LMM.

##  II Related Work

###  II-A Linear Mixing Model-based Unmixing

In the HU topics, many advanced unmixing methods on basis of LMM have been
developed by an attempt to address the issue of spectral variability. Fu et
al. [20] developed the dictionary-adjusted unmixing approach by regrading the
SV as the mismatch between endmember dictionary from the spectral library and
observed spectral signatures. Similarly, authors of [13] proposed the
perturbed LMM (PLMM) by modeling the SV as an additive perturbation
information on each endmember. One same assumption that the spectral
variability should statistically follow the Gaussian distribution is made in
the aforementioned two methods. In reality, there is hardly certain explicit
distribution that can represent various SVs. For example, scaling factors, as
a principal SV, have shown a highly coherent property with endmembers [21].
Obviously, such attributed SV can not be explained by only a perturbed error
term well. For this reason, an extended LMM (ELMM) was proposed in [14] to
model the scaling factors associated with endmembers, providing an effective
solution for scaled unmixing issue from the geometrical perspective. To
address other SVs that ELMM fails to consider, Hong et al. [16] augmented the
ELMM model by learning an additional SV-based dictionary that is reasonably
assumed to be incoherent or low-coherent with endmembers. The resulting
augmented LMM (ALMM) shows its superiority in unmixing performance and ability
to generalize various SVs. Recently, Qi et al. [22] proposed a region-based
multi-view sparse regression model by incorporating a spectral library to
correct the mismatches caused by SVs in HU. The same investigators in [23]
further extended their work by means of a joint dictionary framework for a
more effective HU. More advanced LMM-based models [24, 25, 26] have sought to
eliminate the effects of SVs in the process of unmixing. Nevertheless, these
methods tend to meet the bottleneck particularly when unmixing the complex
hyperspectral scene, due to the limitations of linearized models in data
decomposition and reconstruction.

###  II-B Bilinear Mixing Model-based Unmixing

To overcome the shortcomings of linearized unmixing techniques, some nonlinear
models have been developed to enhance unmixing performance by accounting for
certain particular nonlinear effects, thereby improving HU’s performance to
some extent. As shown in Figs 1(b) and (c), intimate and multi-layered
mixtures of materials are two kinds of common mixing behavior. One typical
nonlinear model – bilinear model (BM) in [27] – has provided a solution with
an emphasis on the two-layered mixing scene. Fan et al. [28] improved the BM,
making it applicable to more cases of material mixing. It is clear, however,
that the BM-based approaches do not have the ability to generalize the HU
well, as the quantity of nonlinear interactions in each pixel is only
restricted in two materials. To effectively relax this constraint, a
generalized bilinear model (GBM) was proposed in [29] for nonlinear unmixing.
Subsequently, numerous GBM-based variants have been appeared, such as sparse
and low-rank BM [30], GBM with semi-nonnegative matrix factorization [31],
multi-linear mixing model [32], geometrically characterized BM [33], and
adjacency effect-based BM [34]. Although these methods have gradually shown
their advancement in handling the SV, particularly for the issue of multi-
layered material mixture, yet they work well only in several special mixing
patterns of materials due to lacking of high capability in data generalization
and representation. The limitations make it difficult to cover a wider range
of type of SVs.

Figure 3: An overview of the proposed DL-based self-supervised unmixing
framework (EGU-Net): a spatial-spectral version. The main improvement lies in
the use of convolutional operations instead of fully connected encoders and
the ‘global’ unmixing with the whole HSI as the model’s input.

###  II-C Deep Learning-based Unmixing

The recent success of DL-based techniques has shown great power and potential
in the issue of blind HU. Some exploratory efforts have been made to challenge
this task with deep networks [35, 36, 37, 38, 39, 40, 41, 42]. An early work
appeared in [35], where artificial neural networks (ANNs) are used to learn
the abundance fractions on a small training set. The same investigators
generalized their model by intelligently selecting training samples for ANNs-
based spectral mixture analysis [36]. Another similar work presented in [37]
is to unmix the spectral signatures by means of pixel-wise ANNs. These
learning-based methods, as expected, perform better in spectral unmixing
compared to classic linear and nonlinear models. With the recent booming
development of machine learning, particularly DL, AE and its variants, as a
common group of DL tool, have been widely applied to HU, achieving more
competitive unmixing performance. For example, Pallson et al. [38] performed
blind HU in the form of a neural network AE, which consists of two parts: one
nonlinear unmixing and one linear reconstruction. In [39], a two-stage AE
approach that starts with a stacked AE for outlier removal and then performs
HU with another AE is designed with non-negative sparse constraints for robust
HU. They [40] further extended their work by replacing the second-step AE with
a variational AE that is capable of better inferring components and fractions
from mixed materials. Similarly, authors of [43] proposed to decompose the
hyperspectral data with an untied denoising sparse AE, providing a potential
solution for HU. Very recently, several CNN-induced researches [44, 45]
attempted to unmix the HSI in a spatial-spectral fashion, demonstrating the
effectiveness of spatial information modeling in the tasks of HU. Inspired by
convolutional operations in CNNs, our networks will also consider modeling
spatial relationship between the target pixel and its neighbours in the
process of unmixing.

##  III Endmember-Guided Unmixing Network

In this section, we will provide a HU-targeted solution by developing a
general DL framework (EGU-Net) to enhance the unmixing performance without
greatly sacrificing the model’s generalization ability in the process of
network learning.

The proposed EGU-Net is a self-supervised two-stream end-to-end network,
including endmember network (E-Net) and unmixing-reconstruction network (UR-
Net). The former aims at learning hierarchical representations of the
endmembers by mapping the pseudo-pure endmembers extracted from the real HS
scene to their one-hot-like abundances obtained by simple HU approaches (e.g.,
LMM). The latter is a common encoder-decoder architecture, which is usually
taken as a backbone in unmixing networks. Significantly, parameter sharing
strategy enables compact relation between the two streams, which is capable of
better interacting specific information into the whole network system.
Moreover, E-Net is able to transfer the potentially intrinsic properties of
the endmembers into UR-Net, and in turn the feedback from UR-Net that takes
global pixels into account can also guide the E-Net towards more stable and
efficient network learning. Fig. 2 illustrates a basic overview of the two-
stream network architecture (EGU-Net).

###  III-A Endmember Network (E-Net)

To stabilize the process of blind HU and characterize the physical
significance in the estimations of endmembers and abundances, we propose to
learn an additional network from the input of some relatively pure endmembers
obtained from the HSI via certain endmember extraction methods, such as vertex
component analysis (VCA) [46], and the corresponding output of approximate
one-hot encoding abundances given by existing unmixing models (e.g., LMM).
This is a typical self-supervised learning strategy without any manual
labeling involved. More specifically, to provide sufficient endmember samples
for training E-Net, we extract spectral bundles111Spectral bundle is a classic
and widely-used endmember extraction method to alleviate the effects of SVs
effectively in the unmixing process. [47, 14] from the HSI. Inspired by [47],
the spectral bundles of endmembers used in E-Net can be obtained by following
three steps.

  * 1)

The HSI is divided into partially overlapped blocks, the number of blocks is
determined by default given in [47];

  * 2)

VCA is used to extract endmembers from each block, the number of endmembers is
automatically estimated by HySime [48];

  * 3)

A clustering algorithm, e.g., K-Means, is applied to remove repeated
endmembers and aggregate all extracted endmembers to $K$ clusters. The $K$
value is experimentally and empirically set to around 20% of all hyperspectral
pixels in our cases by referring to the setting in [47].

It should be noted that such spectral bundles are of benefit against SVs,
since multiple endmembers corresponding to the same material with different
scaling factors, perturbed information, and noises can be obtained in spectral
bundles. This enables us to fully consider and model the SV in the training
phase of E-Net, which is conducive to more accurate abundance estimation.

Given the extracted endmembers, denoted as
$\\{\mathbf{x}_{i}\\}_{i=1}^{N_{e}}\in\mathbb{R}^{B}$ with $B$ bands by
$N_{e}$ pixels, and their nearly-pure abundances
$\\{\mathbf{y}_{i}\\}_{i=1}^{N_{e}}\in\mathbb{R}^{C}$ with $C$ categories, the
$i$-th endmember representation in the $l$-th encoder layer of E-Net, defined
by $\mathbf{z}_{i}^{(l)}$, can be then formulated as follows

$\displaystyle\mathbf{z}^{(l)}_{i}=\begin{cases}g(\mathbf{W}_{e}^{(l)},\mathbf{b}_{e}^{(l)},\mathbf{x}_{i}),&l=1,\\\
g(\mathbf{W}_{e}^{(l)},\mathbf{b}_{e}^{(l)},\mathbf{z}_{i}^{(l-1)}),&l=2,...,m,\end{cases}$
(1)

where $g(\cdot)$ is defined as the nonlinear activation function. The
variables $\mathbf{W}_{e}$ and $\mathbf{b}_{e}$ represent the collections
containing the weights and biases of all layers ($l=1,2,...,m$) in the E-Net.

Each output of encoder layer would follow up a batch normalization (BN) layer
to speed up the parameter learning and alleviate the problems of exploding or
vanishing gradients by reducing the internal covariance shift. Following [49],
the output in the BN layer is derived by

$\displaystyle\mathbf{z_{BN}}^{(l)}_{i}=\gamma\mathbf{\hat{z}}_{i}^{(l)}+\beta,$
(2)

where $\mathbf{\hat{z}}_{i}^{(l)}$ is the $z$-score result of
$\mathbf{z}_{i}^{(l)}$, $\gamma$ and $\beta$ denote the learnable network
parameters.

Before feeding the $\mathbf{z_{BN}}^{(l)}_{i}$ into the activation function, a
dropout layer can be employed to remove possible outliers and SVs to some
extents. It should be noted, however, that in our case, the dropout layer is
only activated in the first block222We define a set of layers that consists of
encoder layer, BN layer, activation function layer, or dropout layer, as a
block in our network architecture.. Herein, we define
$\mathbf{z_{Drop}}^{(l)}_{i}$ as the output of the dropout layer, and then the
final representation ($\mathbf{a}_{i}^{(l)}$) behind an nonlinear activation
in one block can be written as

$\displaystyle\mathbf{a}_{i}^{(l)}=g(\mathbf{z_{Drop}}^{(l)}_{i}).$ (3)

As illustrated in Fig. 2, we enforce the ASC and ANC in the last two blocks by
means of ReLU layer and softmax layer, respectively, which are

$\displaystyle\mathbf{a_{ReLU}}_{i}^{(l)}=g_{ReLU}(\mathbf{z_{BN}}^{(l)}_{i})={\rm
max}(\mathbf{0},\;\mathbf{z_{BN}}^{(l)}_{i}),$ (4)

and

$\displaystyle\mathbf{a_{Soft}}_{i}^{(l)}=g_{Soft}(\mathbf{z}^{(l)}_{i})=\frac{e^{\mathbf{z}_{i}^{(l)}}}{\sum_{j=1}^{C}e^{\mathbf{z}_{j}^{(l)}}}.$
(5)

With the output of Eq. (5), the cross-entropy is used to measure the loss
($L_{E}$) between $\mathbf{a_{Soft}}^{(l)}$ and
$\\{\mathbf{y}_{i=1}^{N_{e}}\\}$ in the E-Net

$\displaystyle L_{E}=\frac{-1}{N_{e}}\sum_{i=1}^{N_{e}}[\mathbf{y}_{i}{\rm
log}\mathbf{a_{Soft}}_{i}+(1-\mathbf{y}_{i}){\rm
log}(1-\mathbf{a_{Soft}}_{i})].$ (6)

Blind SU methods usually tend to generate physically meaningless spectral
signatures, e.g., corresponding to unknown or nonexistent materials, perturbed
information, noises, further limiting the performance of the abundance
estimation. For this reason, endmember-guided SU models have proven to be
effective by means of some reference endmembers obtained by endmember
extraction algorithms, e.g., VCA. In our proposed EGU-Net, E-Net plays a
similar role, i.e., to guide the unmixing process in UR-Net by providing
endmember-related properties or information (e.g., curve shape, spectral
absorption in particular wavelengths reflecting material characteristics),
thereby yielding better abundance estimation results.

What is more noteworthy is that unlike existing AE-based unmixing networks
[45, 41, 50] that directly use VCA results to initialize or regularize the
decoder part to embed the endmember information, our E-Net prefers to play the
role of guidance or correction in the AE-like UR-Net by hierarchically
abstracting endmember properties and progressively transferring them into UR-
Net in a layer-by-layer manner. Plus, the nonlinear reconstruction in the
decoder part in UR-Net is more apt to yield more accurate estimation of
abundance maps.

###  III-B Unmixing-Reconstruction Network (UR-Net)

Inspired by the recent success of AE-based unmixing framework, we design a
similar UR-Net, as shown in the bottom of Fig. 2, involving two same basic
modules: unmixing and reconstruction. In our UR-Net, two distinctive settings
might lead to effective performance improvement in unmixing applications, as
listed in the following.

  * •

We still adopt nonlinear activation functions in reconstruction part rather
than widely-used linear ones. Due to the complexity of various SVs, the
linearized operation fails to finely reconstruct the original spectral
signatures. Consequently, the resulting reconstruction gap may further have a
negative feedback on the to-be-estimated abundances. In other words, we hardly
seek out only several linearly-reconstructed endmembers that are able to
address various SVs well.

  * •

More importantly, the network parameters in our UR-Net stream are partially
shared with those of the E-Net stream, making it effective to hierarchically
transfer different endmember information into UR-Net. Since the proposed EGU-
Net is an end-to-end network, its subnetworks: E-Net and UR-Net are then
jointly trained in a partial parameter sharing fashion (see Figs. 2 and 3).
Intuitively, if we denote $f_{\Phi}$ as the to-be-estimated unmixing function
with respect to a given HSI, the $f_{\Phi}$ is then applicable to both the
endmembers extracted from the HSI and the rest mixed pixels. Accordingly, it
is natural to perform the parameter sharing strategy between E-Net and UR-Net.

By doing so, our network can to a great extent ensure stable and desirable
unmixing results. Accordingly, the optimization problem for the UR-Net stream
is given by minimizing the following reconstruction loss ($L_{UR}$).

$\displaystyle\mathop{\min}_{\mathbf{W}_{ur},\mathbf{b}_{ur}}$
$\displaystyle\lVert\mathbf{X}-f_{r}(g_{u}(\mathbf{W}_{ur},\mathbf{b}_{ur},\mathbf{X}))\rVert_{\operatorname{F}}^{2},$
(7) $\displaystyle{\rm
s.t.}\;\;\mathbf{W}_{ur}=\mathbf{W}_{e},\;\mathbf{b}_{ur}=\mathbf{b}_{e},$

where $f_{r}$ and $g_{u}$ correspond to the mapping functions of unmixing and
reconstruction modules with respect to the weights $\mathbf{W}_{ur}$ and
biases $\mathbf{b}_{ur}$ of the UR-Net, respectively. With the trained
network, the final abundances can be directly inferred via learned parameters,
that is, $\mathbf{a_{Soft}}$ (see the output in Fig. 2). Accordingly, the
overall loss of EGU-Net is

$\displaystyle L_{O}=L_{E}+L_{UR}.$ (8)

###  III-C Spatial-Spectral Unmixing with CNNs: An Extension

The proposed EGU-Net, which takes the Siamese network as a backborne, is a
general but effective DL framework for HU. Intuitively, it can be extended to
a spatial-spectral unmixing version by means of convolutional neurons. For
this reason, we attempt to model the spatial relation of the target pixel and
its neighbors with CNNs in the process of unmixing. More specifically, the
resulting spatial-spectral EGU-Net, EGU-Net-ss for short, holds the basically
identical network architecture with pixel-wise EGU-Net (see Fig. 2), EGU-Net-
pw for short. The main differences between the two lie in two aspects. In one,
despite same input in the E-Net, EGU-Net-ss takes advantage of $1\times 1$
convolution kernel instead of the fully connected encoder and only shares such
convolutional kernels in the last two blocks with UR-Net. In the other, beyond
the pixel-by-pixel reconstruction of spectral signatures, the UR-Net of EGU-
Net-ss is able to feed the whole HSI into the network, yielding a ‘global’
unmixing, by paying equivalent attentions on spatial information with
different-size receptive fields. Fig. 3 gives more details regarding EGU-Net-
ss in the network architecture setting.

TABLE I: Network configuration in each layer of our EGU-Nets: pixel-wise EGU-
Net (EGU-Net-pw) and spatial-spectral EGU-Net (EGU-Net-ss). FC, Conv, and
AvgPool are abbreviations of fully connected, convolution, and average
pooling, respectively. The symbols of ‘$\leftrightarrow$’ and ‘–’ represent
the parameter sharing and no operations, respectively.

| EGU-Net-pw | EGU-Net-ss
---|---|---
Pathway | E-Net |  | UR-Net | E-Net |  | UR-Net
Block1 | FC Encoder | $\leftrightarrow$ | FC Encoder |  $1\times 1$ Conv | – |  $5\times 5$ Conv
BN | BN | BN | BN
Dropout | Dropout | Dropout | Dropout
– | – | – |  $2\times 2$ AvgPool
Tanh | Tanh | Tanh | Tanh
Block2 | FC Encoder | $\leftrightarrow$ | FC Encoder |  $1\times 1$ Conv | – |  $3\times 3$ Conv
BN | BN | BN | BN
– | – | – |  $2\times 2$ AvgPool
Tanh | Tanh | Tanh | Tanh
Block3 | FC Encoder | $\leftrightarrow$ | FC Encoder |  $1\times 1$ Conv | $\leftrightarrow$ |  $1\times 1$ Conv
BN | BN | BN | BN
– | – | – |  $2\times 2$ AvgPool
ReLU | ReLU | ReLU | ReLU
Block4 | FC Encoder | $\leftrightarrow$ | FC Encoder |  $1\times 1$ Conv | $\leftrightarrow$ |  $1\times 1$ DeConv
Softmax | Softmax | Softmax | Softmax
Block5 | – | – | FC Decoder | – | – |  $1\times 1$ DeConv
– | BN | – | BN
– | Sigmoid | – | Sigmoid
Block6 | – | – | FC Decoder | – | – |  $1\times 1$ DeConv
– | BN | – | BN
– | Sigmoid | – | Sigmoid
Block7 | – | – | FC Decoder | – | – |  $3\times 3$ DeConv
– | BN | – | BN
– | Sigmoid | – | Sigmoid
Block8 | – | – | FC Decoder | – | – |  $5\times 5$ DeConv
– | BN | – | BN
– | Sigmoid | – | Sigmoid

###  III-D Clarifying Details on Our Network Architecture

The requires to unmixing applications and effective network training drive us
to particularize the network architecture in a layer-by-layer way. EGU-Net
successively starts with one encoder layer, one BN layer, one dropout layer,
and one activation function layer in the first block, where the dropout layer
added in the beginning is to remove outliers against SVs to some extent. A
ReLU layer is then specified to meet an ANC demand in the penultimate layer of
the whole network, while the ASC on the abundances is added in the network by
means of softmax layer. More specific network configuration in each layer is
detailed in Table I.

(a) Synthetic Data

(b) AVIRIS Data

(c) EnMAP Data

Figure 4: Three hyperspectral datasets used for HU: (a) synthetic data, (b)
AVIRIS data over Jasper Ridge, and (c) EnMAP data over Munich.

##  IV Experiments

###  IV-A Data Description

####  IV-A1 Synthetic Dataset

This dataset was simulated by selecting five reference endmembers from the
United States Geological Survey (USGS) spectral library. The full image
consists of 200$\times$200 pixels with 224 spectral bands covering the
wavelength range from 0.4 $\mu m$ to 2.5 $\mu m$. There are various SVs in the
synthetic dataset, including the principal scaling factors caused by
illumination and topology change, and non-Gaussian noises as other unknown and
complex SVs. More details about the simulating process (e.g., using physical-
inspired Hapke model) can be found in [14, 51]. Fig. 4(a) shows the false
color cube of the HSI.

####  IV-A2 Jasper Ridge Dataset

The real hyperspectral scene was acquired by the airborne visible / infrared
imaging spectrometer (AVIRIS) sensor over a rural area at Jasper Ridge,
California, USA. The original image contains $512\times 614$ pixels with $224$
wavelength bands ranging from 0.38 $\mu m$ to 2.5 $\mu m$ at a ground sampling
distance (GSD) of 20 $m$. We selected a widely-used region of interest (ROI)
with the size of $100\times 100$ pixels and retained $198$ bands after
removing noisy and water absorption bands. An example for the hyperspectral
region is given in Fig. 4(b). In this studied scene, four main endmembers are
investigated, i.e., #1 Tree, #2 Water, #3 Soil, and #4 Road, with referential
abundance maps given from the website333https://rslab.ut.ac.ir/data.

####  IV-A3 EnMAP Munich Dataset

The second EnMAP dataset is generated from corresponding HyMap data over an
urban area, Munich, Germany, using an end-to-end EnMAP simulation tool: EeteS
[52]. We selected a ROI that comprises $93\times 171$ pixels at 30 $m$ GSD
with 221 bands in the spectral range from 0.38 $\mu m$ to 2.5 $\mu m$ (see
Fig. 4(c)). Five dominated materials in this scene are investigated, such as
#1 Roof, #2 Asphalt, #3 Soil, #4 Water, and #5 Vegetation.

Besides, we will herein introduce a simple but feasible processing chain for
the ground truth generation of abundances and endmmebers of HSIs by relying on
a high-quality and high-resolution (e.g., HyMap’s GSD) ground truth used for
land cover and land use mapping. More specifically, a comprehensive processing
flow is given as follows.

  * 1)

Given high-resolution HSIs (e.g., HyMap), low-resolution mixed HSIs (e.g.,
EnMAP) can be obtained by using EeteS simulator or simple Gaussian
downsampling.

  * 2)

High-accuracy classification maps are generated by powerful classifiers or
given by manually labeling on high-resolution HSIs.

  * 3)

Ground truth for abundances can be computed by the classification maps and the
known sampling rate (e.g., GSD) between high-resolution and low-resolution
HSIs.

  * 4)

With the generated abundances, potential endmembers can be simply selected
from those pixels that only belong to one material and then the final $C$
reference endmembers can be further obtained by averaging these pure
endmembers belonging to the same category.

Fig. 5 illustrates the above workflow. The design of the processing chain just
aims to provide a feasible idea and procedure to generate a convincing ground
truth (e.g., abundance maps and endmembers) for the HU task. As the imaging
techniques mature or new hyperspectral satellites launch successfully, the
low-resolution and high-resolution images corresponding to the same scene are
available easier. At that moment, we believe that the proposed processing
chain would be of great help and benefit to the quantitative performance
assessment of real hyperspectral scenes in the unmixing task.

Figure 5: An illustration of the processing chain for generating abundance
maps and endmembers.

###  IV-B Experimental Setup

####  IV-B1 Implementation details

Our approach is implemented on the Tensorflow framework. The model is trained
on the training set, and the hyper-parameters are determined using a grid
search on the validation set. In the training phase, we adopt the Adam
optimizer with the “poly” learning rate policy. The current learning rate can
be updated by multiplying the base one with
$(1-\frac{iter}{maxIter})^{power}$, where the base learning rate and power are
set to 0.1 and 0.99, respectively. Moreover, the momentum is parameterized by
0.9 and the rate of dropout is 0.9 in our case. The model training ends up
with 200 epochs with a minibatch size same with the number of extracted
endmembers used for training. The setting with regard to such a minibatch
training can be specifically explained as follows. We firstly have to clarify
that the unmixing problem is usually seen as regression rather than
classification. That is, unlike the classification that only needs to find the
position corresponding to the maximum response or score (e.g., in the softmax
layer), there are higher requirements for the regression accuracy in the
unmixing task. Therefore, a larger batch or full batch in E-Net tends to step
towards optimal solutions of networks accurately and stably. Furthermore, as
E-Net and UR-Net are jointly trained in our end-to-end EGU-Net and the number
of samples in UR-Net is larger than that in E-Net, we adopt the number of
extracted endmembers as the minibatch size in the training phase of EGU-Net.
Note that the number of these pseudo-endmembers extracted by VCA is determined
by the strategy of spectral bundles presented in [47].

####  IV-B2 Evaluation metrics

To quantitatively evaluate the unmixing performance, the abundance overall
root mean square error (aRMSE) is calculated by collecting all pixel-wise
abundances, as defined by

$\displaystyle
aRMSE=\frac{1}{N}\sum_{i=1}^{N}\sqrt{\frac{1}{C}\sum_{j=1}^{C}(\mathbf{y}_{ji}-\hat{\mathbf{y}}_{ji})^{2}},$
(9)

and the spectral angle distance (SAD) is used to measure the similarity
between the estimated endmember and the real one of each material by the
following definition

$\displaystyle
SAD_{j}=arccos(\frac{\mathbf{e}_{j}^{\operatorname{T}}\mathbf{\hat{e}}_{j}}{\lVert\mathbf{e}_{j}\rVert\lVert\mathbf{\hat{e}}_{j}\rVert}).$
(10)

With the results of Eq. (10), an average SAD (aSAD) index is also given by

$\displaystyle aSAD=\frac{1}{C}\sum_{j=1}^{C}SAD_{j}.$ (11)

The parameter $C$ denotes the number of endmembers, which is $5$, $4$, and $5$
for three used hyperspectral data, respectively.

Notably, different from linear-mixture/nonlinear-fluctuation models presented
in [53, 50] that can represent explicit mappings or weights (i.e.,
endmembers), the reconstruction (i.e., decoder part) in UR-Net is fully
nonlinear, leading to indirect endmember extraction. Alternatively, the
endmembers of our model ($\mathbf{E}=\\{\mathbf{e}_{j}\\}_{j=1}^{C}$) can be
estimated by solving a simple linear system when the abundances are given, we
then have

$\displaystyle\mathop{\min}_{\mathbf{E}}\lVert\mathbf{X}-\mathbf{E}\mathbf{Y}\rVert_{\operatorname{F}}^{2},\;\;{\rm
s.t.}\;\;\mathbf{E}\succeq\mathbf{0}.$ (12)

We have to clarify, however, that the use of a linear model in Eq. (12) only
aims to visualize the endmember spectrum and further to evaluate the quality
of estimated abundances by comparing the spectral profile and absorption in
certain wavelengths of endmembers with reference endmembers.

###  IV-C Ablation Study on the Synthetic Data

Linearly-reconstructed AE [38] (DAEU) and CNN [45] (CNNAEU) can be taken as
the baselines for pixel-wise and spatial-spectral unmixing, respectively.
There are two different modules in our network, namely E-Net and nonlinear
reconstruction part. As a result, we perform the ablation analysis on the
synthetic data to investigate the effectiveness of these components for HU
applications.

Table II lists the performance gain in terms of aRMSE by integrating different
modules, where the averaged aRMSEs are reported out of 10 runs due to the
randomized network initialization and slightly different endmembers extracted
by VCA (if needed). In detail, the unmixing performance of two baselines (DAEU
and CNNAEU) with the linearized reconstruction is relatively inferior to that
of their corresponding nonlinear versions (UR-Net-pw and UR-Net-ss). Unlike
these fully unsupervised methods that fail to model the endmember information
effectively, the self-supervised E-Net that only considers the limited number
of pseudo-endmembers has shown a desirable result at an increase of
approximate 0.0040 over UR-Net-pw in terms of aRMSE. More significantly, our
proposed EGU-Nets – EGU-Net-pw and EGU-Net-ss – outperform the others
dramatically, bringing at least 0.01 performance improvement in aRMSE compared
to the baselines. This greatly demonstrates the superiority of the proposed
general DL framework (EGU-Net) for the HU task.

TABLE II: Ablation analysis on the synthetic dataset, where ‘linear’ and
‘nonlinear’ stand for the reconstruction types in the AE-based networks, and
E-Net represents the endmember-based additional network, while ‘FC’ and ‘Conv’
denote the fully connected encoder and convolutional operations, respectively.
The best results are shown in bold.

Model | Linear | Nonlinear | E-Net | FC | Conv | aRMSE
---|---|---|---|---|---|---
DAEU | $\checkmark$ | $\times$ | $\times$ | $\checkmark$ | $\times$ | 0.0327
CNNAEU | $\checkmark$ | $\times$ | $\times$ | $\times$ | $\checkmark$ | 0.0249
UR-Net-pw | $\times$ | $\checkmark$ | $\times$ | $\checkmark$ | $\times$ | 0.0257
UR-Net-ss | $\times$ | $\checkmark$ | $\times$ | $\times$ | $\checkmark$ | 0.0232
E-Net | $\times$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\times$ | 0.0216
EGU-Net-pw | $\times$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\times$ | 0.0200
EGU-Net-ss | $\times$ | $\checkmark$ | $\checkmark$ | $\times$ | $\checkmark$ | 0.0183

###  IV-D Unmixing Performance Comparison with State-of-the-Art

Several state-of-the-art competitors related to the blind HU task are selected
for performance comparison, including

Non-DL-based unmixing approaches: fully constrained least squares unmixing
(FCLSU) [54], partial constrained least squares unmixing (PCLSU) [55], sparse
unmixing by variable splitting and augmented Lagrangian (SUnSAL) [56],
subspace unmixing with low-rank attribute embedding
(SULoRA)444https://github.com/danfenghong/IEEE_JSTSP_SULoRA [25], and
ALMM555https://github.com/danfenghong/ALMM_TIP [16]. Note that we simply
extended some of the above methods to the blind HU versions by simultaneously
updating the endmembers for a fair comparison;

DL-based unmixing approaches: DAEU [38], deep autoencoder networks (DAEN)
[40], CNNAEU [45], and our two versions: EGU-Net-pw and EGU-Net-ss.

Regarding the parameter setting of these existing models, default parameters
given in original references are applied in our experiments for simplicity.

Table III lists the quantitative performance comparison in terms of aRMSE,
aSAD, and SAD for each endmember on the synthetic data. Overall, FCLSU yields
poor unmixing results for both abundance estimation and endmember extraction,
due to the effects of various SVs. Unlike FCLSU, PCLSU assumes that the
estimated abundances should be in a cone rather than in a simplex by dropping
the ASC, bringing around 2.5% performance improvement in terms of aRMSE over
FCLSU. By making use of sparsity prior on the abundances effectively, SUnSAL
can further enhance the unmixing performance in spite of being improved by
only 0.3%. SULoRA attempts to alleviate the effects of high-dimensional SVs by
searching a low-rank subspace where the unmixing is proceeded more robustly,
yielding an obvious improvement on aRMSE (about 1.4%) and aSAD (0.06%) as
compared to the SUnSAL. Similarly, ALMM models SVs more finely, which allows a
better result in aRMSE and aSAD as well as SAD for almost all endmembers.

In general, DL-based models might pose a greater potential to blindly unmix
the hyperspectral data. Despite being slight lower than the two powerful
linear models (SULoRA and ALMM) in terms of aRMSE, yet they, such as DAEU,
DAEN, and CNNAEU, hold advantage in extracting endmembers. Not unexpectedly,
the proposed simple but effective DL framework either EGU-Net-pw or EGU-Net-ss
outperforms other unmixing algorithms dramatically, which indicates that our
networks are capable of improving the unmixing accuracies for both abundances
and endmembers in the presence of complex SVs. In all indices, the two EGU-
Nets basically achieve the best estimations, in which the CNN-induced spatial-
spectral unmixing more tends to generate the optimal results.

TABLE III: Quantitative results on the synthetic dataset, where aRMSE and aSAD
as well as SAD for each material are reported with their standard deviations
from 10 Monte Carlo runs. The best one is shown in bold.

Methods | aRMSE | SAD_$\mathbf{e}_{1}$ | SAD_$\mathbf{e}_{2}$ | SAD_$\mathbf{e}_{3}$ | SAD_$\mathbf{e}_{4}$ | SAD_$\mathbf{e}_{5}$ | aSAD
---|---|---|---|---|---|---|---
FCLSU | 6.78 $\pm$ 0.18% | 2.75 $\pm$ 0.44% | 2.69 $\pm$ 0.54% | 0.65 $\pm$ 0.04% | 1.26 $\pm$ 0.21% | 0.70 $\pm$ 0.03% | 1.61 $\pm$ 0.20%
PCLSU | 4.33 $\pm$ 0.42% | 0.64 $\pm$ 0.63% | 0.82 $\pm$ 0.35% | 0.23 $\pm$ 0.03% | 0.57 $\pm$ 0.40% | 0.44 $\pm$ 0.01% | 0.54 $\pm$ 0.16%
SUnSAL | 4.05 $\pm$ 0.19% | 0.77 $\pm$ 0.55% | 1.12 $\pm$ 0.27% | 0.25 $\pm$ 0.05% | 0.57 $\pm$ 0.39% | 0.44 $\pm$ 0.01% | 0.63 $\pm$ 0.12%
SULoRA | 2.61 $\pm$ 0.21% | 0.69 $\pm$ 0.59% | 0.93 $\pm$ 0.31% | 0.23 $\pm$ 0.03% | 0.57 $\pm$ 0.40% | 0.43 $\pm$ 0.01% | 0.57 $\pm$ 0.14%
ALMM | 2.47 $\pm$ 0.19% | 0.88 $\pm$ 0.61% | 0.70 $\pm$ 0.22% | 0.24 $\pm$ 0.02% | 0.37 $\pm$ 0.29% | 0.12 $\pm$ 0.02% | 0.46 $\pm$ 0.12%
DAEU | 3.27 $\pm$ 0.24% | 0.49 $\pm$ 0.67% | 0.67 $\pm$ 0.21% | 0.25 $\pm$ 0.08% | 0.30 $\pm$ 0.11% | 0.40 $\pm$ 0.03% | 0.42 $\pm$ 0.22%
DAEN | 2.57 $\pm$ 0.11% | 0.48 $\pm$ 0.24% | 0.46 $\pm$ 0.27% | 0.18 $\pm$ 0.05% | 0.18 $\pm$ 0.34% | 0.32 $\pm$ 0.01% | 0.33 $\pm$ 0.09%
CNNAEU | 2.49 $\pm$ 0.16% | 0.47 $\pm$ 0.51% | 0.73 $\pm$ 0.26% | 0.23 $\pm$ 0.07% | 0.15 $\pm$ 0.13% | 0.29 $\pm$ 0.01% | 0.38 $\pm$ 0.27%
EGU-Net-pw | 2.00 $\pm$ 0.10% | 0.51 $\pm$ 0.32% | 0.38 $\pm$ 0.19% | 0.17 $\pm$ 0.01% | 0.13 $\pm$ 0.14% | 0.40 $\pm$ 0.02% | 0.32 $\pm$ 0.15%
EGU-Net-ss | 1.83 $\pm$ 0.07% | 0.52 $\pm$ 0.27% | 0.33 $\pm$ 0.13% | 0.13 $\pm$ 0.04% | 0.19 $\pm$ 0.18% | 0.20 $\pm$ 0.01% | 0.27 $\pm$ 0.12%

TABLE IV: Quantitative results on the Jasper Ridge dataset, where aRMSE and
aSAD as well as SAD for each material are reported with their standard
deviations from 10 Monte Carlo runs. The best one is shown in bold.

Methods | aRMSE | Tree | Water | Soil | Road | aSAD
---|---|---|---|---|---|---
FCLSU | 0.1783 $\pm$ 0.0098 | 0.1580 $\pm$ 0.0336 | 0.1781 $\pm$ 0.0505 | 0.1179 $\pm$ 0.0171 | 0.0997 $\pm$ 0.0826 | 0.1384 $\pm$ 0.0177
PCLSU | 0.1673 $\pm$ 0.0083 | 0.1494 $\pm$ 0.0326 | 0.2236 $\pm$ 0.0695 | 0.1137 $\pm$ 0.0204 | 0.1327 $\pm$ 0.0418 | 0.1548 $\pm$ 0.0201
SUnSAL | 0.1649 $\pm$ 0.0069 | 0.1471 $\pm$ 0.0343 | 0.2374 $\pm$ 0.0717 | 0.1076 $\pm$ 0.0185 | 0.1199 $\pm$ 0.0348 | 0.1530 $\pm$ 0.0212
SULoRA | 0.1088 $\pm$ 0.0151 | 0.1242 $\pm$ 0.0360 | 0.2740 $\pm$ 0.1533 | 0.0786 $\pm$ 0.0345 | 0.1674 $\pm$ 0.0648 | 0.1611 $\pm$ 0.0532
ALMM | 0.1015 $\pm$ 0.0172 | 0.1091 $\pm$ 0.0401 | 0.2226 $\pm$ 0.0460 | 0.0837 $\pm$ 0.0148 | 0.1182 $\pm$ 0.0551 | 0.1334 $\pm$ 0.0174
DAEU | 0.1091 $\pm$ 0.0150 | 0.1532 $\pm$ 0.0316 | 0.1245 $\pm$ 0.0810 | 0.1170 $\pm$ 0.0177 | 0.1077 $\pm$ 0.0232 | 0.1256 $\pm$ 0.0240
DAEN | 0.0957 $\pm$ 0.0234 | 0.1433 $\pm$ 0.0439 | 0.1305 $\pm$ 0.0317 | 0.0748 $\pm$ 0.0144 | 0.1005 $\pm$ 0.0483 | 0.1123 $\pm$ 0.0245
CNNAEU | 0.0934 $\pm$ 0.0211 | 0.1112 $\pm$ 0.0385 | 0.1544 $\pm$ 0.0913 | 0.0787 $\pm$ 0.0160 | 0.1001 $\pm$ 0.0552 | 0.1111 $\pm$ 0.0367
EGU-Net-pw | 0.0896 $\pm$ 0.0226 | 0.1072 $\pm$ 0.0343 | 0.1733 $\pm$ 0.0748 | 0.0706 $\pm$ 0.0118 | 0.1052 $\pm$ 0.0540 | 0.1141 $\pm$ 0.0388
EGU-Net-ss | 0.0861 $\pm$ 0.0211 | 0.1002 $\pm$ 0.0338 | 0.1684 $\pm$ 0.0725 | 0.0622 $\pm$ 0.0155 | 0.1032 $\pm$ 0.0540 | 0.1085 $\pm$ 0.0362

TABLE V: Quantitative results on the EnMAP Munich dataset, where aRMSE and
aSAD as well as SAD for each material are reported with their standard
deviations from 10 Monte Carlo runs. The best one is shown in bold.

Methods | aRMSE | Roof | Asphalt | Soil | Water | Vegetation | aSAD
---|---|---|---|---|---|---|---
FCLSU | 33.15 $\pm$ 1.85% | 50.73 $\pm$ 16.99% | 43.17 $\pm$ 11.99% | 17.25 $\pm$ 4.58% | 12.97 $\pm$ 3.08% | 17.63 $\pm$ 5.29% | 28.35 $\pm$ 3.04%
PCLSU | 24.73 $\pm$ 4.72% | 34.63 $\pm$ 20.06% | 28.27 $\pm$ 13.79% | 15.23 $\pm$ 4.67% | 32.69 $\pm$ 14.06% | 12.81 $\pm$ 6.72% | 24.73 $\pm$ 4.64%
SUnSAL | 23.98 $\pm$ 4.06% | 31.32 $\pm$ 20.67% | 25.86 $\pm$ 13.56% | 13.93 $\pm$ 3.62% | 35.89 $\pm$ 14.04% | 11.73 $\pm$ 6.94% | 23.74 $\pm$ 5.18%
SULoRA | 22.48 $\pm$ 4.74% | 19.32 $\pm$ 10.21% | 24.31 $\pm$ 23.88% | 12.57 $\pm$ 3.80% | 21.93 $\pm$ 13.02% | 9.48 $\pm$ 10.12% | 17.52 $\pm$ 3.23%
ALMM | 22.97 $\pm$ 3.15% | 18.95 $\pm$ 10.98% | 14.32 $\pm$ 6.83% | 15.32 $\pm$ 0.42% | 13.73 $\pm$ 0.79% | 13.13 $\pm$ 7.11% | 15.09 $\pm$ 3.36%
DAEU | 23.55 $\pm$ 4.33% | 29.23 $\pm$ 21.81% | 28.38 $\pm$ 19.88% | 9.56 $\pm$ 2.70% | 15.68 $\pm$ 7.39% | 13.89 $\pm$ 9.63% | 19.35 $\pm$ 6.39%
DAEN | 21.18 $\pm$ 4.36% | 19.37 $\pm$ 8.45% | 16.54 $\pm$ 9.27% | 15.33 $\pm$ 7.03% | 20.75 $\pm$ 6.01% | 10.68 $\pm$ 15.72% | 16.53 $\pm$ 4.37%
CNNAEU | 18.34 $\pm$ 1.34% | 6.22 $\pm$ 1.93% | 4.53 $\pm$ 2.00% | 6.36 $\pm$ 0.66% | 10.64 $\pm$ 1.51% | 4.01 $\pm$ 2.32% | 6.35 $\pm$ 1.25%
EGU-Net-pw | 17.49 $\pm$ 1.40% | 5.60 $\pm$ 1.32% | 4.13 $\pm$ 2.40% | 6.28 $\pm$ 0.40% | 10.00 $\pm$ 0.56% | 3.26 $\pm$ 1.38% | 5.85 $\pm$ 1.52%
EGU-Net-ss | 16.80 $\pm$ 1.07% | 6.02 $\pm$ 0.90% | 3.88 $\pm$ 1.38% | 6.08 $\pm$ 0.42% | 9.08 $\pm$ 0.31% | 5.42 $\pm$ 1.43% | 6.13 $\pm$ 1.12%

Figure 6: Visualization of abundance maps using different unmixing methods on
the AVIRIS Jasper Ridge data Figure 7: Visualization of abundance maps using
different unmixing methods on the EnMAP Munich data

(a) endmembers of AVIRIS Jasper Ridge data

(b) endmembers of EnMAP Munich data

Figure 8: Extracted endmember comparison between the proposed EGU-Net-ss and
corresponding GT. (a) AVIRIS Jasper Ridge Data. (b) EnMAP Munich data.

###  IV-E Results and Analysis on the Real Data

To further verify the effectiveness and generalization ability, we evaluate
our network architecture on two more challenging real datasets: Jasper Ridge
and Munich in comparison with the same compared methods. Similarly, the three
indices: aRMSE, SAD for each endmember and their mean value aSAD are
calculated for the quantitative assessment of unmixing results, as detailed in
Tables IV and V.

By strictly following the ASC, FCLSU performs extremely poor estimation for
the abundances and endmembers as well. One possible reason is that more
complex SVs lie in the real HSIs. Although PCLSU outperforms FCLSU with an
improvement of more than 0.01 and 9% aRMSEs on the two datasets, respectively,
owing to the effective relaxation of the ASC, it still fails to address the
SVs. The sparsity-prompting constraint (SUnSAL) can not also play a big role
in the blind HU of real scene. By designedly modeling the SVs, SULoRA and ALMM
bring increments of at least 0.05 and 0.06 aRMSEs on the first real data as
well as 1.5% and 1% aRMSEs on the second real data over SUnSAL.

For the DL-based unmixing networks, they are, by and large, superior to those
traditional LMM-based methods either in aRMSE or SADs and aSAD on both
datasets. In particular, this trend becomes more obvious in the real data.
Remarkably, our EGU-Net overcomes other competitors from the perspective of
either abundance estimation or endmember extraction at a further improvement
of nearly 0.005 and 2.5% aRMSEs on the basis of CNNAEU for the two real HSIs,
respectively. In addition, there is a similar trend between-in EGU-Net-pw and
EGU-Net-ss, that is, the latter that emphasizes simultaneous spatial-spectral
modeling observably exceeds the former with a focus on pixel-wise strategy.
This not only demonstrates the effectiveness and superiority of the proposed
EGU-Net framework but also the use of convolutional operations (e.g., CNNs) in
the blind HU tasks.

###  IV-F Visual Evaluation

We also perform a visual comparison of compared methods on the estimations of
abundance maps and endmembers.

####  IV-F1 Abundance Visualization

As shown in Fig. 6, FCLSU fails to unmix the materials well, especially Water
leading to the large-area errors. By relaxing the strong ASC, the PCLSU’s
abundance map of Water looks much better than FCLSU’s, but the fractional
values between PCLSU and ground truth (GT) still exist the large gap. The
similar results also occurs in SUnSAL. By attempts to eliminate the effects of
SVs, the abundance maps of SULoRA more approach to the GTs visually, while
ALMM shows relatively smooth maps, particularly the material Water. For those
network-based unmixing models, they all behave more realistic on the abundance
maps, although DAEU and DAEN wrongly recognize the Road as Water to some
extent. As expected, our EGU-Net makes the basically identical visual effects
with GTs, and meanwhile the EGU-Net-ss shows more robust and smooth maps.
Furthermore, a similar trend can be observed in Fig. 7 for the EnMAP Munich
data. This more complex urban scene lies in greater challenges. This,
therefore, makes more noisy and easily-confused abundance maps. Despite so,
our proposed networks are still able to visualize the nearly-identical shapes
for each material compared to the GT.

####  IV-F2 Endmember Comparison

With desirable abundance maps, we are also expected to have good endmember
estimations, e.g., in shapes or band absorption. Towards this goal, we make a
visual comparison for endmembers between our EGU-Net-ss and GTs, as shown in
Fig. 8. The AVIRIS Jasper Ridge image is a easy-understanding scene, which
only contains four common materials: Tree, Water, Soil, and Road. As a result,
EGU-Net-ss performs relatively accurate endmember estimations (see Fig. 8(a)).
For another more challenging EnMAP Munich data, there exists a gap between the
extracted endmembers and GTs. This indicates, to some extent, that the studied
urban scene is complex, possibly involving many unknown SVs. We have to admit,
however, that in such complex urban scene, our network is still capable of
learning high-quality endmembers that can be matched well in shapes, e.g.,
Asphalt and Vegetation, as shown in Fig. 8(b).

##  V Conclusion

In this paper, we propose a general self-supervised unmixing network, called
EGU-Net, which consists of EGU-Net-pw and EGU-Net-ss. The two resulting
networks are applicable to pixel-wise spectral unmixing and spatial-spectral
unmixing with fully connected encoder and convolutional operations,
respectively. In addition, unlike the widely-used autoencoder-like models,
EGU-Net additionally learns an E-Net from pure or nearly-pure endmembers and
transfers its parameters into the another autoencoder-based unmixing network:
UR-Net. In the meanwhile, the unmixing results of UR-Net can be also fed back
to the E-Net to guide a better endmember learning, thereby yielding more
reasonable and superior unmixing. In our future work, we would like to
optimize the network architecture by using more advanced reconstruction loss
(e.g., SAD or sparsity-promoting $\ell_{1}$-norm) and investigate the optimal
combination of endmember extraction and unmixing methods used in E-Net as well
as develop a more general and powerful network-based framework with the aid of
the multi-modal data (e.g., multispectral data, Lidar) to address this
unmixing issue more effectively.

## References

  * [1] D. Hong, W. He, N. Yokoya, J. Yao, L. Gao, L. Zhang, J. Chanussot, and X. X. Zhu, “Interpretable hyperspectral artificial intelligence: When non-convex modeling meets hyperspectral remote sensing,” IEEE Geosci. Remote Sens. Mag., 2021.  DOI: 10.1109/MGRS.2021.3064051.
  * [2] X. Wang, Y. Kong, Y. Gao, and Y. Cheng, “Dimensionality reduction for hyperspectral data based on pairwise constraint discriminative analysis and nonnegative sparse divergence,” IEEE Trans. Neural Netw. Learn. Syst., vol. 10, no. 4, pp. 1552–1562, 2017.
  * [3] D. Hong, N. Yokoya, J. Chanussot, J. Xu, and X. X. Zhu, “Learning to propagate labels on graphs: An iterative multitask regression framework for semi-supervised hyperspectral dimensionality reduction,” ISPRS J. Photogramm. Remote Sens., vol. 158, pp. 35–49, 2019.
  * [4] D. Hong, N. Yokoya, J. Chanussot, and X. Zhu, “CoSpace: Common subspace learning from hyperspectral-multispectral correspondences,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 7, pp. 4349–4359, 2019.
  * [5] R. Dian, S. Li, and L. Fang, “Learning a low tensor-train rank representation for hyperspectral image super-resolution,” IEEE Trans. Neural Netw. Learn. Syst., vol. 30, no. 9, pp. 2672–2683, 2019.
  * [6] D. Hong, L. Gao, N. Yokoya, J. Yao, J. Chanussot, D. Qian, and B. Zhang, “More diverse means better: Multimodal deep learning meets remote-sensing imagery classification,” IEEE Trans. Geosci. Remote Sens., vol. 59, no. 5, pp. 4340–4354, 2021.
  * [7] S. Yang, Z. Feng, M. Wang, and K. Zhang, “Self-paced learning-based probability subspace projection for hyperspectral image classification,” IEEE Trans. Neural Netw. Learn. Syst., vol. 30, no. 2, pp. 630–635, 2018\.
  * [8] J. Peng, L. Li, and Y. Tang, “Maximum likelihood estimation-based joint sparse representation for the classification of hyperspectral remote sensing images,” IEEE Trans. Neural Netw. Learn. Syst., vol. 30, no. 6, pp. 1790–1802, 2018.
  * [9] R. Hang, Q. Liu, D. Hong, and P. Ghamisi, “Cascaded recurrent neural networks for hyperspectral image classification,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 8, pp. 5384–5394, 2019.
  * [10] D. Hong, L. Gao, J. Yao, B. Zhang, P. Antonio, and J. Chanussot, “Graph convolutional networks for hyperspectral image classification,” IEEE Trans. Geosci. Remote Sens., 2020.  DOI: 10.1109/TGRS.2020.3015157.
  * [11] J. Bioucas-Dias, A. Plaza, N. Dobigeon, M. Parente, Q. Du, P. Gader, and J. Chanussot, “Hyperspectral unmixing overview: Geometrical, statistical, and sparse regression-based approaches,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 5, no. 2, pp. 354–379, 2012.
  * [12] T. Uezato, R. Murphy, A. Melkumyan, and A. Chlingaryan, “Incorporating spatial information and endmember variability into unmixing analyses to improve abundance estimates,” IEEE Trans. Image Process., vol. 25, no. 12, pp. 5563–5575, 2016.
  * [13] P. Thouvenin, N. Dobigeon, and J. Tourneret, “Hyperspectral unmixing with spectral variability using a perturbed linear mixing model,” IEEE Trans. Signal Process., vol. 64, no. 2, pp. 525–538, 2016.
  * [14] L. Drumetz, M. Veganzones, S. Henrot, R. Phlypo, J. Chanussot, and C. Jutten, “Blind hyperspectral unmixing using an extended linear mixing model to address spectral variability,” IEEE Trans. Image Process., vol. 25, pp. 3890–3905, Aug. 2016.
  * [15] T. Uezato, M. Fauvel, and N. Dobigeon, “Hyperspectral image unmixing with lidar data-aided spatial regularization,” IEEE Trans. Geosci. Remote Sens., vol. 56, no. 7, pp. 4098–4108, 2018.
  * [16] D. Hong, N. Yokoya, J. Chanussot, and X. Zhu, “An augmented linear mixing model to address spectral variability for hyperspectral unmixing,” IEEE Trans. Image Process., vol. 28, no. 4, pp. 1923–1938, 2019.
  * [17] X. Kang, X. Zhang, S. Li, K. Li, J. Li, and J. Benediktsson, “Hyperspectral anomaly detection with attribute and edge-preserving filters,” IEEE Trans. Geosci. Remote Sens., vol. 55, no. 10, pp. 5600–5611, 2017.
  * [18] Y. Xu, Z. Wu, J. Chanussot, and Z. Wei, “Joint reconstruction and anomaly detection from compressive hyperspectral images using mahalanobis distance-regularized tensor rpca,” IEEE Trans. Geosci. Remote Sens., vol. 56, no. 5, pp. 2919–2930, 2018.
  * [19] X. Wu, D. Hong, J. Tian, J. Chanussot, W. Li, and R. Tao, “ORSIm Detector: A novel object detection framework in optical remote sensing imagery using spatial-frequency channel features,” IEEE Trans. Geosci. Remote Sens., vol. 57, pp. 5146–5158, July 2019.
  * [20] X. Fu, W. Ma, J. Bioucas-Dias, and T. Chan, “Semiblind hyperspectral unmixing in the presence of spectral library mismatches,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 9, pp. 5171–5184, 2016.
  * [21] D. Hong, N. Yokoya, J. Chanussot, and X. Zhu, “Learning a low-coherence dictionary to address spectral variability for hyperspectral unmixing,” in Proc. ICIP, pp. 235–239, IEEE, 2017.
  * [22] L. Qi, J. Li, Y. Wang, and X. Gao, “Region-based multiview sparse hyperspectral unmixing incorporating spectral library,” IEEE Geosci. Remote Sens. Lett., vol. 16, no. 7, pp. 1140–1144, 2019.
  * [23] L. Qi, J. Li, X. Gao, Y. Wang, C. Zhao, and Y. Zheng, “A novel joint dictionary framework for sparse hyperspectral unmixing incorporating spectral library,” Neurocomputing, vol. 356, pp. 97–106, 2019.
  * [24] R. Heylen, A. Zare, P. Gader, and P. Scheunders, “Hyperspectral unmixing with endmember variability via alternating angle minimization,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 8, pp. 4983–4993, 2016.
  * [25] D. Hong and X. Zhu, “SULoRA: Subspace unmixing with low-rank attribute embedding for hyperspectral data analysis,” IEEE J. Sel. Topics Signal Process., vol. 12, no. 6, pp. 1351–1363, 2018.
  * [26] T. Uezato, M. Fauvel, and N. Dobigeon, “Hyperspectral unmixing with spectral variability using adaptive bundles and double sparsity,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 6, pp. 3980–3992, 2019.
  * [27] B. Somers, K. Cools, S. Delalieux, J. Stuckens, D. V. der Zande, W. Verstraeten, and P. Coppin, “Nonlinear hyperspectral mixture analysis for tree cover estimates in orchards,” Remote Sens. Environ., vol. 113, no. 6, pp. 1183–1193, 2009.
  * [28] W. Fan, B. Hu, J. Miller, and M. Li, “Comparative study between a new nonlinear model and common linear model for analysing laboratory simulated-forest hyperspectral data,” Int. J. Remote Sens., vol. 30, no. 11, pp. 2951–2962, 2009.
  * [29] A. Halimi, Y. Altmann, N. Dobigeon, and J. Tourneret, “Nonlinear unmixing of hyperspectral images using a generalized bilinear model,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 11, pp. 4153–4162, 2011.
  * [30] Q. Qu, N. Nasrabadi, and T. Tran, “Abundance estimation for bilinear mixture models via joint sparse and low-rank representation,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 7, pp. 4404–4423, 2013.
  * [31] N. Yokoya, J. Chanussot, and A. Iwasaki, “Nonlinear unmixing of hyperspectral data using semi-nonnegative matrix factorization,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 2, pp. 1430–1437, 2013.
  * [32] R. Heylen and P. Scheunders, “A multilinear mixing model for nonlinear spectral unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 1, pp. 240–251, 2015.
  * [33] B. Yang, B. B. Wang, and Z. Wu, “Nonlinear hyperspectral unmixing based on geometric characteristics of bilinear mixture models,” IEEE Trans. Geosci. Remote Sens., vol. 56, no. 2, pp. 694–714, 2017.
  * [34] X. Wang, Y. Zhong, L. Zhang, and Y. Xu, “Blind hyperspectral unmixing considering the adjacency effect,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 9, pp. 6633–6649, 2019.
  * [35] J. Plaza, A. Plaza, R. Perez, and P. Martinez, “On the use of small training sets for neural network-based characterization of mixed pixels in remotely sensed hyperspectral images,” Pattern Recognit., vol. 42, no. 11, pp. 3032–3045, 2009.
  * [36] J. Plaza and A. Plaza, “Spectral mixture analysis of hyperspectral scenes using intelligently selected training samples,” IEEE Geosci. Remote Sens. Lett., vol. 7, no. 2, pp. 371–375, 2009.
  * [37] G. Licciardi and F. D. Frate, “Pixel unmixing in hyperspectral data by means of neural networks,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 11, pp. 4163–4172, 2011.
  * [38] B. Palsson, J. Sigurdsson, J. R. Sveinsson, and M. O. Ulfarsson, “Hyperspectral unmixing using a neural network autoencoder,” IEEE Access, vol. 6, pp. 25646–25656, 2018.
  * [39] Y. Su, A. Marinoni, J. Li, J. Plaza, and P. Gamba, “Stacked nonnegative sparse autoencoders for robust hyperspectral unmixing,” IEEE Geosci. Remote Sens. Lett., vol. 15, no. 9, pp. 1427–1431, 2018.
  * [40] Y. Su, J. Li, A. Plaza, A. Marinoni, P. Gamba, and S. Chakravortty, “DAEN: Deep autoencoder networks for hyperspectral unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 7, pp. 4309–4321, 2019.
  * [41] M. Wang, M. Zhao, J. Chen, and S. Rahardja, “Nonlinear unmixing of hyperspectral data via deep autoencoder networks,” IEEE Geosci. Remote Sens. Lett., vol. 16, no. 9, pp. 1467–1471, 2019.
  * [42] D. Hong, J. Chanussot, N. Yokoya, U. Heiden, W. Heldens, and X. Zhu, “WU-Net: A weakly-supervised unmixing network for remotely sensed hyperspectral imagery,” in IGARSS, pp. 373–376, IEEE, 2019.
  * [43] Y. Qu and H. Qi, “udas: An untied denoising autoencoder with sparsity for spectral unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 3, pp. 1698–1712, 2018.
  * [44] X. Zhang, Y. Sun, J. Zhang, P. Wu, and L. Jiao, “Hyperspectral unmixing via deep convolutional neural networks,” IEEE Geosci. Remote Sens. Lett., vol. 15, no. 11, pp. 1755–1759, 2018.
  * [45] B. Palsson, M. O. Ulfarsson, and J. R. Sveinsson, “Convolutional autoencoder for spatial-spectral hyperspectral unmixing,” in IGARSS, pp. 357–360, IEEE, 2019.
  * [46] J. Nascimento and J. Dias, “Vertex component analysis: A fast algorithm to unmix hyperspectral data,” IEEE Trans. Geosci. Remote Sens., vol. 43, no. 4, pp. 898–910, 2005.
  * [47] B. Somers, M. Zortea, A. Plaza, and G. Asner, “Automated extraction of image-based endmember bundles for improved spectral unmixing,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 5, no. 2, pp. 396–408, 2012\.
  * [48] J. M. Bioucas-Dias and J. M. Nascimento, “Hyperspectral subspace identification,” IEEE Trans. Geosci. Remote Sens., vol. 46, no. 8, pp. 2435–2445, 2008.
  * [49] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167, 2015.
  * [50] M. Zhao, M. Wang, J. Chen, and S. Rahardja, “Hyperspectral unmixing via deep autoencoder networks for a generalized linear-mixture/nonlinear-fluctuation model,” arXiv preprint arXiv:1904.13017, 2019.
  * [51] J. Yao, D. Hong, L. Xu, D. Meng, J. Chanussot, and Z. Xu, “Sparsity-enhanced convolutional decomposition: A novel tensor-based paradigm for blind hyperspectral unmixing,” IEEE Trans. Geosci. Remote Sens., 2021.  DOI:10.1109/TGRS.2021.3069845.
  * [52] K. Segl, L. Guanter, C. Rogass, T. Kuester, S. Roessner, H. Kaufmann, B. Sang, V. Mogulsky, and S. Hofer, “EeteS-—The EnMAP end-to-end simulation tool,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 5, no. 2, pp. 522–530, 2012.
  * [53] J. Chen, C. Richard, and P. Honeine, “Nonlinear unmixing of hyperspectral data based on a linear-mixture/nonlinear-fluctuation model,” IEEE Trans. Signal Process., vol. 61, no. 2, pp. 480–492, 2012.
  * [54] D. C. Heinz and C. I. Chang, “Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imagery,” IEEE Trans. Geosci. Remote Sens., vol. 39, no. 3, pp. 529–545, 2001.
  * [55] R. Heylen, D. Burazerovic, and P. Scheunders, “Fully constrained least squares spectral unmixing by simplex projection,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 11, pp. 4112–4122, 2011.
  * [56] J. Bioucas-Dias and M. Figueiredo, “Alternating direction algorithms for constrained sparse regression: Application to hyperspectral unmixing,” in WHISPERS, pp. 1–4, IEEE, 2010.

|  Danfeng Hong (S’16–M’19–SM’21) received the M.Sc. degree (summa cum laude)
in computer vision, College of Information Engineering, Qingdao University,
Qingdao, China, in 2015, the Dr. -Ing degree (summa cum laude) in Signal
Processing in Earth Observation (SiPEO), Technical University of Munich (TUM),
Munich, Germany, in 2019. Since 2015, he has worked as a Research Associate at
the Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR),
Oberpfaffenhofen, Germany. Currently, he is a research scientist and leads a
Spectral Vision working group at IMF, DLR, and also an adjunct scientist in
GIPSA-lab, Grenoble INP, CNRS, Univ. Grenoble Alpes, Grenoble, France. His
research interests include signal / image processing and analysis,
hyperspectral remote sensing, machine / deep learning, artificial intelligence
and their applications in Earth Vision. Dr. Hong was a recipient of the Best
Reviewer award of the IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING in
2020, and the Jose Bioucas Dias award for recognizing the outstanding paper at
the Workshop on Hyperspectral Imaging and Signal Processing: Evolution in
Remote Sensing (WHISPERS) in 2021. He is a Guest Editor for the IEEE JOURNAL
OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING and REMOTE
SENSING. Currently, he is a Editorial aboard member of the REMOTE SENSING and
a Topical Associate Editor of the IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE
SENSING.
---|---
|  Lianru Gao (M’12–SM’18) received the B.S. degree in civil engineering from
Tsinghua University, Beijing, China, in 2002, the Ph.D. degree in cartography
and geographic information system from Institute of Remote Sensing
Applications, Chinese Academy of Sciences (CAS), Beijing, China, in 2007. He
is currently a Professor with the Key Laboratory of Digital Earth Science,
Aerospace Information Research Institute, CAS. He also has been a visiting
scholar at the University of Extremadura, Cáceres, Spain, in 2014, and at the
Mississippi State University (MSU), Starkville, USA, in 2016. His research
focuses on hyperspectral image processing and information extraction. In last
ten years, he was the PI of 10 scientific research projects at national and
ministerial levels, including projects by the National Natural Science
Foundation of China (2010-2012, 2016-2019, 2018-2020), and by the Key Research
Program of the CAS (2013-2015). He has published more than 160 peer-reviewed
papers, and there are more than 80 journal papers included by SCI. He was
coauthor of an academic book entitled “Hyperspectral Image Classification And
Target Detection”. He obtained 28 National Invention Patents in China. He was
awarded the Outstanding Science and Technology Achievement Prize of the CAS in
2016, and was supported by the China National Science Fund for Excellent Young
Scholars in 2017, and won the Second Prize of The State Scientific and
Technological Progress Award in 2018. He received the recognition of the Best
Reviewers of the IEEE Journal of Selected Topics in Applied Earth Observations
and Remote Sensing in 2015, and the Best Reviewers of the IEEE Transactions on
Geoscience and Remote Sensing in 2017.
---|---
|  Jing Yao received the B.Sc. degree from Northwest University, Xi’an, China,
in 2014, and the Ph.D. degree in the School of Mathematics and Statistics,
Xi’an Jiaotong University, Xi’an, China, in 2021. He is currently an Assistant
Professor with the Key Laboratory of Digital Earth Science, Aerospace
Information Research Institute, Chinese Academy of Sciences, Beijing, China.
From 2019 to 2020, he was a visiting student at Signal Processing in Earth
Observation (SiPEO), Technical University of Munich (TUM), Munich, Germany,
and at the Remote Sensing Technology Institute (IMF), German Aerospace Center
(DLR), Oberpfaffenhofen, Germany. His research interests include low-rank
modeling, hyperspectral image analysis and deep learning-based image
processing methods.
---|---
|  Naoto Yokoya (S’10–M’13) received the M.Eng. and Ph.D. degrees from the
Department of Aeronautics and Astronautics, the University of Tokyo, Tokyo,
Japan, in 2010 and 2013, respectively. He is currently a Lecturer at the
University of Tokyo and a Unit Leader at the RIKEN Center for Advanced
Intelligence Project, Tokyo, Japan, where he leads the Geoinformatics Unit. He
was an Assistant Professor at the University of Tokyo from 2013 to 2017. In
2015-2017, he was an Alexander von Humboldt Fellow, working at the German
Aerospace Center (DLR), Oberpfaffenhofen, and Technical University of Munich
(TUM), Munich, Germany. His research is focused on the development of image
processing, data fusion, and machine learning algorithms for understanding
remote sensing images, with applications to disaster management. Dr. Yokoya
won the first place in the 2017 IEEE Geoscience and Remote Sensing Society
(GRSS) Data Fusion Contest organized by the Image Analysis and Data Fusion
Technical Committee (IADF TC). He is the Chair (2019-2021) and was a Co-Chair
(2017-2019) of IEEE GRSS IADF TC and also the secretary of the IEEE GRSS All
Japan Joint Chapter since 2018. He is an Associate Editor for the IEEE Journal
of Selected Topics in Applied Earth Observations and Remote Sensing (JSTARS)
since 2018. He is/was a Guest Editor for the IEEE JSTARS in 2015-2016, for
Remote Sensing in 2016-2020, and for the IEEE Geoscience and Remote Sensing
Letters (GRSL) in 2018-2019.
---|---
|  Jocelyn Chanussot (M’04–SM’04–F’12) received the M.Sc. degree in electrical
engineering from the Grenoble Institute of Technology (Grenoble INP),
Grenoble, France, in 1995, and the Ph.D. degree from the Université de Savoie,
Annecy, France, in 1998. Since 1999, he has been with Grenoble INP, where he
is currently a Professor of signal and image processing. His research
interests include image analysis, hyperspectral remote sensing, data fusion,
machine learning and artificial intelligence. He has been a visiting scholar
at Stanford University (USA), KTH (Sweden) and NUS (Singapore). Since 2013, he
is an Adjunct Professor of the University of Iceland. In 2015-2017, he was a
visiting professor at the University of California, Los Angeles (UCLA). He
holds the AXA chair in remote sensing and is an Adjunct professor at the
Chinese Academy of Sciences, Aerospace Information research Institute,
Beijing. Dr. Chanussot is the founding President of IEEE Geoscience and Remote
Sensing French chapter (2007-2010) which received the 2010 IEEE GRS-S Chapter
Excellence Award. He has received multiple outstanding paper awards. He was
the Vice-President of the IEEE Geoscience and Remote Sensing Society, in
charge of meetings and symposia (2017-2019). He was the General Chair of the
first IEEE GRSS Workshop on Hyperspectral Image and Signal Processing,
Evolution in Remote sensing (WHISPERS). He was the Chair (2009-2011) and
Cochair of the GRS Data Fusion Technical Committee (2005-2008). He was a
member of the Machine Learning for Signal Processing Technical Committee of
the IEEE Signal Processing Society (2006-2008) and the Program Chair of the
IEEE International Workshop on Machine Learning for Signal Processing (2009).
He is an Associate Editor for the IEEE Transactions on Geoscience and Remote
Sensing, the IEEE Transactions on Image Processing and the Proceedings of the
IEEE. He was the Editor-in-Chief of the IEEE Journal of Selected Topics in
Applied Earth Observations and Remote Sensing (2011-2015). In 2014 he served
as a Guest Editor for the IEEE Signal Processing Magazine. He is a Fellow of
the IEEE, a member of the Institut Universitaire de France (2012-2017) and a
Highly Cited Researcher (Clarivate Analytics/Thomson Reuters).
---|---
|  Uta Heiden (IEEE Member) received the Ph.D. degree in urban spectroscopy
from the Technical University of Berlin, Berlin, Germany, and the GFZ German
Research Centre for Geosciences, Potsdam, Germany, in 2003. She is also a
Researcher with the Remote Sensing Technology Institute (IMF), German
Aerospace Center (DLR), Weßling, Germany. She is also interested in
application development using airborne and spaceborne imaging spectroscopy
data with a focus on the use of material-based reflectance properties of urban
surfaces. She has widened her interests toward a better understanding of the
function of soils in degrading ecosystems. Dr. Heiden is also a member of the
Science Advisory Group of the German spaceborne imaging spectrometer mission
EnMAP. She is also a Science Coordinator for the spaceborne imaging
spectroscopy mission DESIS onboard the ISS and the Chair of the “Geoscience
Spaceborne Imaging Spectroscopy” Technical Committee (GSIS-TC) of the IEEE
Geoscience and Remote Sensing Society.
---|---
|  Bing Zhang (M’11–SM’12–F’19) received the B.S. degree in geography from
Peking University, Beijing, China, in 1991, and the M.S. and Ph.D. degrees in
remote sensing from the Institute of Remote Sensing Applications, Chinese
Academy of Sciences (CAS), Beijing, China, in 1994 and 2003, respectively.
Currently, he is a Full Professor and the Deputy Director of the Aerospace
Information Research Institute, CAS, where he has been leading lots of key
scientific projects in the area of hyperspectral remote sensing for more than
25 years. His research interests include the development of Mathematical and
Physical models and image processing software for the analysis of
hyperspectral remote sensing data in many different areas. He has developed 5
software systems in the image processing and applications. His creative
achievements were rewarded 10 important prizes from Chinese government, and
special government allowances of the Chinese State Council. He was awarded the
National Science Foundation for Distinguished Young Scholars of China in 2013,
and was awarded the 2016 Outstanding Science and Technology Achievement Prize
of the Chinese Academy of Sciences, the highest level of Awards for the CAS
scholars. Dr. Zhang has authored more than 300 publications, including more
than 170 journal papers. He has edited 6 books/contributed book chapters on
hyperspectral image processing and subsequent applications. He is the IEEE
fellow and currently serving as the Associate Editor for IEEE Journal of
Selected Topics in Applied Earth Observations and Remote Sensing. He has been
serving as Technical Committee Member of IEEE Workshop on Hyperspectral Image
and Signal Processing since 2011, and as the president of hyperspectral remote
sensing committee of China National Committee of International Society for
Digital Earth since 2012, and as the Standing Director of Chinese Society of
Space Research (CSSR) since 2016. He is the Student Paper Competition
Committee member in IGARSS from 2015-2019.
---|---