# Multisource Semisupervised Adversarial Domain Generalization Network for
Cross-Scene Sea–Land Clutter Classification

Xiaoxuan Zhang, , Quan Pan, , Salvador García This work was supported in part
by the China Scholarship Council (CSC) under Grant 202306290161.
(Corresponding author: Xiaoxuan Zhang.)Xiaoxuan Zhang and Quan Pan are with
the School of Automation and the Key Laboratory of Information Fusion
Technology, Ministry of Education, Northwestern Polytechnical University,
Xi’an 710072, China (e-mail<EMAIL_ADDRESS>quanpan@nwpu.edu.cn).Salvador García is with the Department of Computer
Science and Artificial Intelligence, University of Granada, Granada 18071,
Spain (e-mail: salvagl@decsai.ugr.es).

###### Abstract

Deep learning (DL)-based sea–land clutter classification for sky-wave over-
the-horizon-radar (OTHR) has become a novel research topic. In engineering
applications, real-time predictions of sea–land clutter with existing
distribution discrepancies are crucial. To solve this problem, this article
proposes a novel Multisource Semisupervised Adversarial Domain Generalization
Network (MSADGN) for cross-scene sea–land clutter classification. MSADGN can
extract domain-invariant and domain-specific features from one labeled source
domain and multiple unlabeled source domains, and then generalize these
features to an arbitrary unseen target domain for real-time prediction of
sea–land clutter. Specifically, MSADGN consists of three modules: domain-
related pseudolabeling module, domain-invariant module, and domain-specific
module. The first module introduces an improved pseudolabel method called
domain-related pseudolabel, which is designed to generate reliable
pseudolabels to fully exploit unlabeled source domains. The second module
utilizes a generative adversarial network (GAN) with a multidiscriminator to
extract domain-invariant features, to enhance the model’s transferability in
the target domain. The third module employs a parallel multiclassifier branch
to extract domain-specific features, to enhance the model’s discriminability
in the target domain. The effectiveness of our method is validated in twelve
domain generalizations (DG) scenarios. Meanwhile, we selected 10 state-of-the-
art DG methods for comparison. The experimental results demonstrate the
superiority of our method.

###### Index Terms:

Clutter classification, domain generalization (DG), generative adversarial
network (GAN), over-the-horizon radar, remote sensing, semisupervised
learning.

##  I Introduction

The rapid advancement of deep learning (DL) has brought significant
opportunities for the development of remote sensing methods [1]. DL-based
sea–land clutter classification for sky-wave over-the-horizon-radar (OTHR), as
a branch of remote sensing, has recently emerged as a novel research topic.
The sea–land clutter classification involves identifying whether the
background clutter in each range-azimuth cell originates from the sea or land.
Matching these classification results with existing geographic information
provides coordinate registration parameters essential for target localization,
which holds considerable potential for cost-effectively improving target
localization accuracy [2].

###  I-A Related Works on DL-based Sea–Land Clutter Classification

Over the past five years, remarkable achievements have been made in DL-based
sea–land clutter classification. A large amount of work [3], [4], [5], [6],
[7] has delved into the various challenges encountered in sea–land clutter
classification, such as few-shot learning [5], class imbalance learning [5],
cross-domain classification [6], and cross-scale classification [4]. DL-based
methods include deep convolutional neural networks (DCNNs) [8], generative
adversarial networks (GANs) [9], and variational autoencoders (VAEs) [10]. Li
et al. [3], [4] proposed DCNN-based fully supervised learning methods, capable
of achieving end-to-end sea–land clutter classification. The DCNN-based method
in [3] constructed a network with three convolutional layers followed by three
fully connected (FC) layers, outperforming traditional sea–land clutter
classification methods, including support vector machine (SVM)-based [11] and
least mean square (LMS)-based [12] methods. The DCNN-based method in [4]
implemented cross-scale sea–land clutter classification based on algebraic
multigrid within the ResNet-18 framework. Zhang et al. [5], [6], [7]
introduced GAN-based and VAE-based semisupervised and unsupervised learning
methods, capable of achieving more challenging sea–land clutter
classification. The GAN-based and VAE-based methods in [5] used an auxiliary
classifier VAE GAN (AC-VAEGAN) for data augmentation of sea–land clutter,
enabling effective few-shot learning and class imbalance learning. The GAN-
based method in [6] used a triple loss adversarial domain adaptation network
(TLADAN) to learn domain-invariant features between the source and target
domains, successfully achieving cross-domain sea–land clutter classification.
The GAN-based method in [7] employed a weighted loss semisupervised GAN (WL-
SSGAN) to extract hierarchical features from several unlabeled samples,
facilitating the semisupervised classification of sea–land clutter.

In summary, the existing DL-based sea–land clutter classification methods can
be divided into two categories: 1) empirical risk minimization (ERM)-based
methods [3], [4], [5], [7] and 2) domain adaptation (DA)-based methods [6].
ERM is a common situation in which training and test data follow the
independent and identically distributed (i.i.d.) assumption. However, due to
various operating conditions or noise interference, a distribution discrepancy
between training and test data inevitably emerges. In this situation, the
performance of the ERM-based method is significantly decreased. As shown in
Fig. 1(a), the ERM-based method excels at accurately classifying the training
set but performs poorly on three different test sets. To overcome this
dilemma, DA-based methods have emerged, that mitigate the distribution
discrepancy between the source domain (training set) and the target domain
(test set) by learning domain-invariant features. As shown in Fig. 1(b), the
DA-based method excels at accurately classifying the source domain and
performs well on the target domain.

Figure 1: Taxonomy of DL-based sea–land clutter classification methods. (a)
ERM-based. (b) DA-based. (c) DG-based. (Best viewed in color.)

###  I-B Related Works on DA and DG

DA-based methods have made significant progress in fields related to sea–land
clutter or OTHR, such as computer vision (CV), fault diagnosis, and remote
sensing. In general, according to different strategies for aligning feature
distributions, DA-based methods can be divided into three categories [13],
[14]: 1) discrepancy-based DA; 2) reconstruction-based DA; and 3) adversarial-
based DA. The discrepancy-based DA aims to reduce the distance or divergence
between different distributions by selecting certain metrics, such as maximum
mean discrepancy (MMD) [15], multiple kernel MMD (MK-MMD) [16], deep
correlation alignment (Deep CORAL) [17], and central moment discrepancy (CMD)
[18]. For instance, Wan et al. [19] proposed an MK-MMD-driven deep convolution
multiadversarial DA(DCMADA) network for rolling bearing fault diagnosis. Yan
et al. [20] presented a cross-domain distance metric learning (CDDML)
framework for scene classification of aerial images. The reconstruction-based
DA strives to learn domain-invariant features by accurately classifying the
labeled source domain and reconstructing the source and target domains [21].
For instance, Zhu et al. [22] introduced a cycle-consistent adversarial DA
network (CCADAN) to reconstruct the source and target domains to achieve
intelligent bearing fault diagnosis. The adversarial-based DA attempts to
confuse the features between the source and target domains by employing GAN’s
adversarial training strategy [23]. For instance, Wang et al. [24] used a deep
adversarial DA network (DADAN) to transfer fault diagnosis knowledge. Huang et
al. [25] presented a spatial-spectral weighted adversarial DA (SSWADA) network
for cross-domain wetland mapping. Zhang et al. [6] proposed a TLADAN for
cross-domain sea–land clutter classification and high-resolution remote
sensing image classification.

However, DA-based methods are only suitable for offline scenarios, designed
for a specific target domain without good generalization capability. As shown
in Fig. 1(b), DA-based methods exhibit poor performance on two unseen domains.
Domain generalization (DG), a more practical yet challenging topic, involves
extracting domain-invariant features from multisource domains and generalizing
them to an unseen target domain [see Fig. 1(c)]. Therefore, DG-based methods
can make real-time predictions for unseen target domains. In the context of
sea–land clutter classification, to emphasize the specificity of methods,
problems addressed using DA-based methods are termed cross-domain
classification, while those addressed using DG-based methods are termed cross-
scene classification. Commonly, DG-based methods can be divided into three
categories [26]: 1) data-based DG; 2) feature-based DG; and 3) learning
strategy-based DG. The data-based DG includes data augmentation and data
generation. For example, Zhou et al. [27] proposed a mixup neural network for
cross-scene image classification. This method can generate new domains in the
feature space, thus achieving data augmentation. Fan et al. [28] presented a
deep mixed DG network (DMDGN) for intelligent fault diagnosis, in which data
augmentation is employed for both class and domain spaces. The feature-based
DG includes domain-invariant representation learning and feature
disentanglement. For example, Zhu et al. [29] proposed a style and content
separation network (SCSN) for cross-scene remote sensing image classification.
By effectively disentangling and recognizing the style and content
information, SCSN improves the model’s generalization capability. The learning
strategy-based DG includes meta-learning, self-supervised learning and
ensemble learning. For example, Ren et al. [30] introduced a model-agnostic
meta-learning-based training framework called Meta-GENE for fault diagnosis
DG.

###  I-C Motivations and Contributions

The most relevant to this article is our latest proposed TLADAN [6], which
considers a distribution discrepancy between training and test data. As
mentioned earlier, TLADAN is only applicable for the offline scenario.
Unfortunately, for OTHR, various working conditions or noise interference may
give rise to many unseen target domains. Thus, a more generalized DG model, as
shown in Fig. 1(c), is desired to address cross-scene sea–land clutter
classification. Furthermore, manual labeling of collected sea–land clutter
data is extremely expensive and time-consuming. A more intricate setting
called semisupervised DG (SSDG) is considered. In this setting, one labeled
source domain and multiple unlabeled source domains are available. Therefore,
our motivation is to explore how to utilize these multisource domains to
design an SSDG model that works well on an arbitrary unseen target domain.

To this end, this article proposes a feature-based DG method called
Multisource Semisupervised Adversarial DG Network (MSADGN), which is capable
of extracting domain-invariant and domain-specific features for generalization
to the unseen target domain. Specifically, MSADGN consists of three
submodules: domain-related pseudolabeling module, domain-invariant module, and
domain-specific module. In the first submodule, we propose an improved
pseudolabel method named domain-related pseudolabel to generate pseudolabels
for unlabeled source domains. The deep semisupervised DG Network (DSDGN) [31]
utilizes the traditional pseudolabel method to assign pseudolabels for
unlabeled source domains. However, it does not consider distribution
differences between these domains; thus, the quality of the generated
pseudolabels may be poor. To solve this problem, we introduce a domain-related
similarity score into the traditional pseudolabel method, incorporating a
global iteration scheme and a dynamic threshold to generate reliable
pseudolabels. In the second submodule, we integrate an improved domain
adversarial neural network (DANN) [32] into MSADGN. As an adversarial-based DA
method, DANN achieves DA by confusing the features between the source and
target domains. Inspired by this, a multidiscriminator is designed to learn
domain-invariant features between multisource domains. This is achieved by
learning domain-invariant features between any two domains. In the third
submodule, we integrate a multiclassifier branch to learn domain-specific
features, to further enhance the model’s generalization capability. The
motivation is that the sea–land clutter from the unseen target domain may
share similarities with the domain-specific features of one or more domains in
the multisource domains. We hope to explore more comprehensive features that
are both domain-invariant and domain-specific, enabling MSADGN to generalize
well to the unseen target domain.

The main contributions of this article can be summarized as follows.

  * 1)

For the first time, a novel DG architecture, MSADGN, is proposed for cross-
scene sea–land clutter classification. The proposed MSADGN can extract
comprehensive domain-invariant and domain-specific features from one labeled
source domain and multiple unlabeled source domains, thereby exhibiting strong
transferability and discriminability in the unseen target domain.

  * 2)

Most DG-based methods assume that labels from multisource domains are
available. This article explores a more complex SSDG scenario. Considering the
distribution differences of sea–land clutter between different domains, an
improved pseudolabel method called domain-related pseudolabel is proposed to
generate reliable pseudolabels, thus indirectly obtaining a fully labeled
multisource domain for DG.

  * 3)

Most DG-based methods focus on learning domain-invariant features from
multisource domains. MSADGN is capable of learning domain-invariant features
through a multidiscriminator branch and domain-specific features through a
multiclassifier branch. Consequently, the real-time prediction results of
sea–land clutter can benefit from the learned domain-specific features.

  * 4)

To the best of our knowledge, the cross-scene classification problem of
sea–land clutter has not been considered in the literature. The effectiveness
of MSADGN is verified in twelve DG scenarios using both the 1-D cross-scene
sea–land clutter signal (CS-SLCS) dataset and the publicly available 2-D
cross-scene high-resolution remote sensing image (CS-HRRSI) dataset.
Meanwhile, the superiority of MSADGN is validated through the comparison with
ten state-of-the-art methods, including five DA-based DG methods and five pure
SSDG methods.

The rest of this article is organized as follows. The preliminaries, including
problem formulation, GAN, and semisupervised pseudolabel, are presented in
Section II. In Section III, we elaborate on the proposed MSADGN, including the
submodule details, algorithmic procedure, and theoretical insight. In Section
IV, the effectiveness and superiority of MSADGN are validated using CS-SLCS
and CS-HRRSI. Section V summarizes this article.

##  II Preliminaries

###  II-A Problem Formulation

DG is more challenging than DA in the cross-scene sea–land clutter
classification task, because there is no prior knowledge of the target domain.
DG extracts domain-invariant (and domain-specific) features from multisource
domains ${\mathcal{D}}_{s}=\\{{\mathcal{D}}_{s}^{k}\\}_{k=1}^{K}$ and
generalize them to an arbitrary unseen target domain ${\mathcal{D}}_{t}$,
where $K$ is the number of multisource domains. This article focuses on an
SSDG problem, where only the labels of ${\mathcal{D}}_{s}^{1}$ are accessible,
whereas the labels of the other $\\{{\mathcal{D}}_{s}^{k}\\}_{k=2}^{K}$ are
inaccessible. In other words,
${\mathcal{D}}_{s}^{1}=\\{({\mathbf{x}}_{i}^{l},y_{i}^{l},d_{i}^{l}=1)\\}_{i=1}^{n_{l}}$
and ${\mathcal{D}}_{s}^{k\neq
1}=\\{({\mathbf{x}}_{i}^{u},d_{i}^{u}=k)\\}_{i=1}^{n_{k}}$, where
${\mathbf{x}}_{i}^{*}$, $y_{i}^{l}$, $d_{i}^{*}$, $n_{l}$ and $n_{k}$ denote
an input sample, a category label, a domain label, the number of labeled
samples in ${\mathcal{D}}_{s}^{1}$, and number of unlabeled samples in
${\mathcal{D}}_{s}^{k\neq 1}$, respectively. For different domains, the label
space remains the same, whereas the feature space or data distribution tends
to be different, i.e. $P_{s}^{i}({{\mathbf{X}}})\neq
P_{s}^{j}({{\mathbf{X}}})\neq P_{t}({{\mathbf{X}}}),1\leq i\neq j\leq K$,
where $P_{s}^{*}({\mathbf{X}})$ denotes the source distribution and
$P_{t}({\mathbf{X}})$ denotes the target distribution. The goal of this
article is to develop a generalizable MSADGN to minimize the risk of unseen
target classification,
$\mathbb{E}_{({\mathbf{x}},y)\in{\mathcal{D}}_{t}}[\mathcal{F}({\mathbf{x}})\neq
y]$, where $\mathbb{E}$ represents the expectation operator. Table I lists the
most commonly used notations in this article along with their explanations.

TABLE I: The Most Commonly Used Notations and Their Explanations Notations | Explanations
---|---
${\mathcal{D}}_{s}$ | Multisource domains
${\mathcal{D}}_{s}^{1}$ | A labeled source domain in ${\mathcal{D}}_{s}$
${\mathcal{D}}_{s}^{k\neq 1}$ | Unlabeled source domains in ${\mathcal{D}}_{s}$
${\mathcal{D}}_{s^{\prime}}$ | Multisource domains output by the domain-related pseudolabeling module
${\mathcal{D}}_{s^{\prime}}^{1}$ | A labeled source domain in ${\mathcal{D}}_{s^{\prime}}$
${\mathcal{D}}_{s^{\prime}}^{k\neq 1}$ | Pseudolabeled source domains in ${\mathcal{D}}_{s^{\prime}}$
${\mathcal{D}}_{t}$ | An unseen target domain
$({\mathbf{x}}_{i}^{l},y_{i}^{l},d_{i}^{l}=1)$ | A labeled sample in ${\mathcal{D}}_{s}^{1}$, where ${\mathbf{x}}_{i}^{l}$, $y_{i}^{l}$ and $d_{i}^{l}$ denote labeled sample, category label and domain label
$({\mathbf{x}}_{i}^{u},d_{i}^{u}=k)$ | An unlabeled sample in ${\mathcal{D}}_{s}^{k\neq 1}$, where ${\mathbf{x}}_{i}^{u}$ and $d_{i}^{u}$ denote unlabeled sample and domain label
$({\mathbf{x}}_{i}^{u},\widehat{y_{i}^{u}},d_{i}^{u}=k)$ | A pseudolabeled sample in ${\mathcal{D}}_{s^{\prime}}^{k\neq 1}$, where ${\mathbf{x}}_{i}^{u}$, $\widehat{y_{i}^{u}}$ and $d_{i}^{u}$ denote pseudolabeled sample, pseudolabel and domain label
$n_{l}$ | The number of labeled samples in ${\mathcal{D}}_{s}^{1}$
$n_{k}$ | The number of unlabeled samples in ${\mathcal{D}}_{s}^{k\neq 1}$
$n^{\prime}_{k}$ | The number of pseudolabeled samples in ${\mathcal{D}}_{s^{\prime}}^{k\neq 1}$
${\mathbf{f}}_{k}$ | A domain-related feature embedding
${\mathbf{M}}_{1}$ | A category-related feature embedding
${\mathbf{z}}_{i,k}$ | A domain-specific feature for ${\mathbf{x}}_{i}\in{\mathcal{D}}_{s^{\prime}}$ in domain $k$
${\mathbf{z}}_{i}$ | A weighted domain-specific feature for ${\mathbf{x}}_{i}\in{\mathcal{D}}_{s^{\prime}}$
${\mathbf{w}}_{i}$ | A domain-related similarity weight for ${\mathbf{x}}_{i}\in{\mathcal{D}}_{s^{\prime}}$
$K$ | The number of multisource domains
$C$ | The number of classes in each domain
$L$ | The dimension of feature embedding in ${\mathcal{D}}_{s}^{1}$
$I$ | The number of training iterations
$\phi({\mathbf{x}}_{i}^{u})$ | A probability score in the domain-related pseudolabeling module
$\psi({\mathbf{x}}_{i}^{u})$ | A similarity score in the domain-related pseudolabeling module
$\Phi({\mathbf{x}}_{i}^{u})$ | A pseudolabel score in the domain-related pseudolabeling module
$\tau_{0}$ | A fixed threshold in the traditional pseudolabel method
$\tau$ | A dynamic threshold in the domain-related pseudolabeling module
${\mathcal{l}}_{\text{CE}}$ | The crossentropy loss
${\mathcal{L}}_{\text{inv}}$ | The domain-invariant loss of MSADGN
${\mathcal{L}}_{\text{spe}}$ | The domain-specific loss of MSADGN
$\mathcal{L}$ | The overall loss of MSADGN
$\alpha$ | The tradeoff parameter between $\phi({\mathbf{x}}_{i}^{u})$ and $\psi({\mathbf{x}}_{i}^{u})$
$\lambda$ | The tradeoff parameter between ${\mathcal{L}}_{\text{inv}}$ (or ${\mathcal{L}}_{\text{adv}}$) and ${\mathcal{L}}_{\text{spe}}$
$\rho_{I}$ | The tradeoff parameter between ${\mathbf{M}}_{1}^{(I)}$ and ${\mathbf{M}}_{1}^{(I-1)}$
$F_{\text{shared}}$, $F_{\text{weighted}}$ | A shared feature extractor and a weighted feature extractor
$D_{1}$, $D_{2}$, $\dots$, $D_{K(K-1)/2}$ | Domain classifiers
$C_{1}$, $C_{2}$, $\dots$, $C_{K}$ | Task-specific classifiers
$C_{\text{weighted}}$ | A weighted classifier
$\theta_{f_{1}}$, $\theta_{f_{2}}$ | The parameters of $F_{\text{shared}}$ and $F_{\text{weighted}}$
$\theta_{d_{1}}$, $\theta_{d_{2}}$, $\dots$, $\theta_{d_{K(K-1)/2}}$ | The parameters of $D_{1}$, $D_{2}$, $\dots$, and $D_{K(K-1)/2}$
$\theta_{c_{1}}$, $\theta_{c_{2}}$, …, $\theta_{c_{K}}$ | The parameters of $C_{1}$, $C_{2}$, …, and $C_{K}$
$\theta_{c_{K+1}}$ | The parameter of $C_{\text{weighted}}$

###  II-B GAN

The original GAN [9] consists of two adversarial neural networks: a generator
($G$) and a discriminator ($D$). $G$ generates realistic samples, whereas $D$
strives to differentiate between samples generated by $G$ and real samples. In
this way, adversarial training of the two networks can be described as a two-
player minimax game with the following value function:

$\displaystyle\min\limits_{G}\max\limits_{D}V(D,G)=$
$\displaystyle{\mathbb{E}_{{\mathbf{x}}\sim p_{{}_{\rm data}}}[\log
D({\mathbf{x}})]}$ (1) $\displaystyle+{\mathbb{E}_{{\mathbf{z}}\sim
p_{{}_{\mathbf{z}}}}[1-\log D(G({\mathbf{z}}))]}$

where $p_{{}_{\rm data}}$ is the distribution of real data ${\mathbf{x}}$,
$p_{{}_{\mathbf{z}}}$ is the distribution of prior noise ${\mathbf{z}}$,
$D({\mathbf{x}})$ denotes the probability that ${\mathbf{x}}$ comes from the
real data, $G({\mathbf{z}})$ denotes the data generated by $G$, and
$D(G({\mathbf{z}}))$ denotes the probability that $G({\mathbf{z}})$ comes from
the generated data.

In the MSADGN submodule, GAN is employed to extract domain-invariant features.
Specifically, $G$ is used to extract features from the multisource domains. In
contrast, $D$ distinguishes from which source domain these features originate.

###  II-C Semisupervised Pseudolabel

Pseudolabel [33] is a prevalent semisupervised learning method that utilizes
unlabeled samples to enhance the performance of deep neural networks (DNNs).
The core idea behind this method is to use existing models to predict
unlabeled samples, treat these predictions as pseudolabels, and then merge
them with labeled samples for further model training. Specifically, the
traditional pseudolabel selects the token with the maximum prediction
probability as the pseudolabel for each unlabeled sample. The one-hot coding
pseudolabel
$\widehat{{\mathbf{y}}^{u}}=[\widehat{y_{1}^{u}},\widehat{y_{2}^{u}},\dots,\widehat{y_{C}^{u}}]$
can be expressed as follows:

$\widehat{y_{j}^{u}}=\begin{cases}1&\text{if}~{}j=\text{argmax}_{j^{\prime}}\phi_{j^{\prime}}({\mathbf{x}}^{u})~{}\text{and}~{}\phi_{j^{\prime}}({\mathbf{x}}^{u})>\tau_{0}\\\
0&\text{otherwise}~{}~{}j=1,2,\dots,C\end{cases}$ (2)

where ${\mathbf{x}}^{u}$ is an input unlabeled sample,
$\phi_{j^{\prime}}({\mathbf{x}}^{u})$ denotes the $j^{\prime}$-th element of
the output vector $\phi({\mathbf{x}}^{u})$, $\tau_{0}$ is a fixed threshold,
and $C$ is the number of classes.

##  III Proposed MSADGN for Cross-Scene Sea–Land Clutter Classification

In this section, we first introduce the overall architecture of MSADGN,
followed by a detailed description of its submodules, including the domain-
related pseudolabeling module, domain-invariant module, and domain-specific
module. Next, the training procedure of MSADGN is presented. Finally, we
provide a theoretical insight into MSADGN.

###  III-A Overall Architecture of MSADGN

The overall architecture of MSADGN for cross-scene sea–land clutter
classification is illustrated in Fig. 2, which consists of three modules:
domain-related pseudolabeling module, domain-invariant, and domain-specific
modules. In more detail, MSADGN comprises two feature extractors, $K+1$
classifiers, and $K(K-1)/2$ discriminators. Feature extractor 1 is the shared
feature extractor $F_{\text{shared}}$, feature extractor 2 is the weighted
feature extractor $F_{\text{weighted}}$, the first $K$ classifiers are task-
specific classifiers $C_{1}$, $C_{2}$, $\dots$, and $C_{K}$, the ($K+1$)-th
classifier is the weighted classifier $C_{\text{weighted}}$, and the
$K(K-1)/2$ discriminators represent the domain classifiers $D_{1}$, $D_{2}$,
$\dots$, and $D_{K(K-1)/2}$.

The input sea–land clutter samples consist of two parts: labeled samples from
${\mathcal{D}}_{s}^{1}$ and unlabeled samples from ${\mathcal{D}}_{s}^{k\neq
1}$. On the one hand, the input samples enter the domain-related
pseudolabeling module, where unlabeled samples with pseudolabel scores
exceeding a certain threshold are selected and assigned pseudolabels. As a
result, labeled samples from all source domains are indirectly acquired, and
we denote the current multisource domains as
${\mathcal{D}}_{s^{\prime}}=\\{{\mathcal{D}}_{s^{\prime}}^{k}\\}_{k=1}^{K}$.
Conversely, since the domain-invariant module only requires samples with
domain labels, we take all samples from ${\mathcal{D}}_{s}$ as input into this
module to extract domain-invariant features. Furthermore, to enhance the
model’s generalization capability, the domain-specific module is integrated,
and all labeled samples from ${\mathcal{D}}_{s^{\prime}}$ are input into this
module to extract domain-specific features. Because of the more comprehensive
features extracted by the domain-invariant and domain-specific modules, our
model exhibits better transferability and discriminability in the unseen
target domain.

Figure 2: Illustration of cross-scene sea–land clutter classification via the
proposed MSADGN. (Best viewed in color.)

###  III-B Domain-Related Pseudolabeling Module

The domain-related pseudolabeling module comprises $F_{\text{shared}}$ and
$C_{1}$. The traditional pseudolabel method is applicable when the training
samples follow the i.i.d. assumption. However, in the DG problem, due to
various working conditions or noise interference, training samples from
multisource domains may have different probability distributions. If
pseudolabels are assigned fairly to unlabeled samples in
${\mathcal{D}}_{s}^{k\neq 1}$ using the aforementioned method without
considering domain gaps, poor pseudolabels may be obtained. To solve this
problem, we propose an improved pseudolabel method called domain-related
pseudolabel. In the proposed method, if the pseudolabel score
$\Phi({\mathbf{x}}_{i}^{u})$ exceeds a certain threshold, pseudolabels are
assigned unfairly to unlabeled samples in ${\mathcal{D}}_{s}^{k\neq 1}$ as we
expect, where ${\mathbf{x}}_{i}^{u}\in{\mathcal{D}}_{s}^{k\neq 1}$. To this
end, we need to gradually adjust the values of the following three components
during the training process: 1) probability score
$\phi({\mathbf{x}}_{i}^{u})$; 2) similarity score
$\psi({\mathbf{x}}_{i}^{u})$; and 3) dynamic threshold $\tau$.

####  III-B1 Probability Score

We still obtain $\phi({\mathbf{x}}_{i}^{u})$ using the traditional pseudolabel
method. In the domain-related pseudolabeling module, the initial step involves
updating both $F_{\text{shared}}$ and $C_{1}$ using labeled samples from
$\mathcal{D}_{s}^{1}$. Subsequently, $\phi({\mathbf{x}}_{i}^{u})$ can be
obtained from
$\phi({\mathbf{x}}_{i}^{u})=softmax(C_{1}(F_{\text{shared}}({\mathbf{x}}_{i}^{u})))$,
where $softmax(\cdot)$ represents the softmax activation function.

####  III-B2 Similarity Score

Further, $\psi({\mathbf{x}}_{i}^{u})$ is introduced as a correction term for
$\phi({\mathbf{x}}_{i}^{u})$ to consider domain-related information. Our
motivation is that the domain-related feature embedding
${\mathbf{f}}_{k}=F_{\text{shared}}({\mathbf{x}}_{i}^{u})\in{\mathbb{R}}^{1\times
L}$ in ${\mathcal{D}}_{s}^{k\neq 1}$ is supposed to gather around the
category-related feature embeddings ${\mathbf{M}}_{1}\in{\mathbb{R}}^{L\times
C}$ in ${\mathcal{D}}_{s}^{1}$, where $L$ represents the feature embedding
dimensions. And ${\mathbf{M}}_{1}$ is defined as follows:

$\displaystyle{\mathbf{M}}_{1}=[{\mathbf{f}}_{1}(c=1),{\mathbf{f}}_{1}(c=2),\dots,{\mathbf{f}}_{1}(c=C)]^{\text{T}}$
(3)
$\displaystyle{\mathbf{f}}_{1}(c=j)=\frac{1}{n_{l}^{j}}\sum\nolimits_{({\mathbf{x}}_{i}^{l},y_{i}^{l}=j)\in{\mathcal{D}}_{s}^{1}}F_{\text{shared}}({\mathbf{x}}_{i}^{l})$

where $n_{l}^{j}$ denotes the number of labeled samples in the $j$-th class.
Then, $\psi({\mathbf{x}}_{i}^{u})$ can then be obtained by measuring the
similarity between ${\mathbf{f}}_{k}$ and ${\mathbf{M}}_{1}$:

$\psi({\mathbf{x}}_{i}^{u})=CS({\mathbf{f}}_{k},{\mathbf{M}}_{1}),k=2,3,\dots,K$
(4)

where $CS(\cdot,\cdot)$ denotes the cosine similarity function.

A straightforward way to compute $\psi({\mathbf{x}}_{i}^{u})$ is by measuring
the similarity between ${\mathbf{f}}_{k}$ and ${\mathbf{M}}_{1}$ obtained
locally from the current mini-batch of labeled samples from
${\mathcal{D}}_{s}^{1}$. However, the limited number of samples in the mini-
batch may result in an inadequate ${\mathbf{M}}_{1}$. For example, the current
mini-batch might not include feature embeddings for certain categories. In
addition, even one labeled sample with high-confidence misclassification may
cause high similarity bias.

To address this problem, we propose a global iteration scheme to obtain
${\mathbf{M}}_{1}$. First, we reformulate ${\mathbf{M}}_{1}$ into a form
associated with the number of training iterations $I$:

$\displaystyle{\mathbf{M}}_{1}^{(I)}=[{\mathbf{f}}_{1}^{(I)}(c=1),{\mathbf{f}}_{1}^{(I)}(c=2),\dots,{\mathbf{f}}_{1}^{(I)}(c=C)]^{\text{T}}$
(5)
$\displaystyle{\mathbf{f}}_{1}^{(I)}(c=j)=\frac{1}{n_{l}^{j}}\sum\nolimits_{({\mathbf{x}}_{i}^{l},y_{i}^{l}=j)\in{\mathcal{D}}_{s}^{1}(I)}F_{\text{shared}}({\mathbf{x}}_{i}^{l})$

where ${\mathcal{D}}_{s}^{1}(I)$ represents the mini-batch of
${\mathcal{D}}_{s}^{1}$ in the $I$-th iteration. Then, ${\mathbf{M}}_{1}$ is
initialized with ${\mathbf{M}}_{1}^{(0)}$ and subsequently updated based on
the last iteration as follows:

$\displaystyle\rho_{I}$
$\displaystyle=CS({\mathbf{M}}_{1}^{(I)},{\mathbf{M}}_{1}^{(I-1)})$ (6)
$\displaystyle{\mathbf{M}}_{1}^{(I)}$
$\displaystyle\leftarrow\rho_{I}^{2}{\mathbf{M}}_{1}^{(I)}+(1-\rho_{I}^{2}){\mathbf{M}}_{1}^{(I-1)}$

where $\rho_{I}$ is the tradeoff parameter. The advantage of the global
iteration scheme is that if ${\mathbf{M}}_{1}^{(I)}$ is inadequate,
intuitively, its similarity to ${\mathbf{M}}_{1}^{(I-1)}$ should be low,
indicating that $\rho_{I}$ tends to 0. At this time, according to Eq. (6), one
can take ${\mathbf{M}}_{1}^{(I-1)}$ as the ${\mathbf{M}}_{1}^{(I)}$ in the
$I$-th iteration.

####  III-B3 Dynamic Threshold

A suitable threshold is crucial for selecting a reliable pseudolabel. If the
fixed threshold $\tau_{0}$ mentioned earlier is directly applied, more
unlabeled samples with high-confidence misclassification may be selected as
the training progresses. However, a dynamic threshold $\tau$ gradually
increasing with training epochs may alleviate the aforementioned potential
errors:

$\tau=\frac{1}{1+e^{-10*p}}-0.1$ (7)

where $p$ is the training progress that changes from 0 to 1. Here, we set
$p=(n_{e-1}*L_{e}+s_{e})/(N_{e}*L_{e})$, where $n_{e-1}$ denotes the last
training epoch, $L_{e}$ denotes the number of data batches in each epoch,
$s_{e}$ denotes the current training step within the current epoch, and
$N_{e}$ is the maximum number of training epochs.

Finally, with the definitions of $\phi({\mathbf{x}}_{i}^{u})$,
$\psi({\mathbf{x}}_{i}^{u})$ and $\tau$, the domain-related pseudolabel can be
selected. First, the new pseudolabel score $\Phi({\mathbf{x}}_{i}^{u})$ can be
given by the following formula:

$\Phi({\mathbf{x}}_{i}^{u})=\alpha\phi({\mathbf{x}}_{i}^{u})+(1-\alpha)\psi({\mathbf{x}}_{i}^{u})$
(8)

where $\alpha$ is the tradeoff parameter.

Next, the domain-related pseudolabel can be selected in a manner similar to
Eq. (2). One-hot coding domain-related pseudolabel
$\widehat{{\mathbf{y}}_{i}^{u}}=[\widehat{y_{i,1}^{u}},\widehat{y_{i,2}^{u}},\dots,\widehat{y_{i,C}^{u}}]$
of $\widehat{y_{i}^{u}}$ can be expressed as follows:

$\widehat{y_{i,j}^{u}}=\begin{cases}1&\text{if}~{}j=\text{argmax}_{j^{\prime}}\Phi_{j^{\prime}}({\mathbf{x}}_{i}^{u})~{}\text{and}~{}\Phi_{j^{\prime}}({\mathbf{x}}_{i}^{u})>\tau\\\
0&\text{otherwise}~{}~{}j=1,2,\dots,C\end{cases}$ (9)

where $\Phi_{j^{\prime}}({\mathbf{x}}_{i}^{u})$ denotes the $j^{\prime}$-th
element of $\Phi({\mathbf{x}}_{i}^{u})$.

After the domain-related pseudolabeling module, we can obtain the fully
labeled multisource domains
${\mathcal{D}}_{s^{\prime}}=\\{{\mathcal{D}}_{s^{\prime}}^{k}\\}_{k=1}^{K}$,
where ${\mathcal{D}}_{s^{\prime}}^{1}={\mathcal{D}}_{s}^{1}$ and
${\mathcal{D}}_{s^{\prime}}^{k\neq
1}=\\{({\mathbf{x}}_{i}^{u},\widehat{y_{i}^{u}},d_{i}^{u}=k)\\}_{i=1}^{{n^{\prime}_{k}}}$,
with $n^{\prime}_{k}$ denoting the number of pseudolabeled samples in
${\mathcal{D}}_{s^{\prime}}^{k\neq 1}$, which provides sufficient labeled
training samples for the subsequent domain-specific module.

###  III-C Domain-Invariant Module

The domain-invariant module comprises $F_{\text{shared}}$, $D_{1}$, $D_{2}$,
$\dots$, and $D_{K(K-1)/2}$. The design of this module is inspired by DANN. It
is worth noting that DANN is primarily employed for DA, achieving the
alignment of feature distributions between the source and target domains
through adversarial training. Since ${\mathcal{D}}_{t}$ is unseen during the
training phase of MSADGN, one can equivalently treat
${\mathcal{D}}_{s}^{k_{1}}$ as the source domain of DANN, and
${\mathcal{D}}_{s}^{k_{2}}$ as the target domain of DANN, where $1\leq
k_{1}\neq k_{2}\leq K$. In other words, $K(K-1)/2$ discriminators are designed
to distinguish the features of any two source domains.

In this module, the domain-invariant feature is extracted through adversarial
training similar to Eq. (1) as follows:

$\displaystyle{\mathcal{L}}_{\text{adv}}$
$\displaystyle=\frac{1}{K(K-1)/2}\sum_{k=1}^{K(K-1)/2}{\mathcal{L}}_{\text{adv}}^{k}$
(10) $\displaystyle{\mathcal{L}}_{\text{adv}}^{k}$
$\displaystyle=\min_{F_{\text{shared}}}\max_{D_{k}}-\frac{\lambda}{n}\sum_{{\mathbf{x}}_{i}\in{\mathcal{D}}}{\mathcal{l}}_{\text{CE}}(D_{k}(F_{\text{shared}}({\mathbf{x}}_{i})),s_{i})$

where $\lambda$ denotes the tradeoff between ${\mathcal{L}}_{\text{adv}}$ and
the domain-specific loss ${\mathcal{L}}_{\text{spe}}$ defined in Section
III-D, $n=n_{k_{1}}+n_{k_{2}}$ represents the total number of samples in
${\mathcal{D}}={\mathcal{D}}_{s}^{k_{1}}\cup{\mathcal{D}}_{s}^{k_{2}}$,
${\mathcal{l}}_{\text{CE}}(\cdot,\cdot)$ denotes the crossentropy loss, and
$s_{i}$ represents a binary label of the ${\mathcal{D}}_{s}^{k_{1}}$ and
${\mathcal{D}}_{s}^{k_{2}}$ such that $s_{i}=1$ if
${\mathbf{x}}_{i}\in{\mathcal{D}}_{s}^{k_{1}}$ or $s_{i}=0$ if
${\mathbf{x}}_{i}\in{\mathcal{D}}_{s}^{k_{2}}$. Here, we set
$\lambda=2/(1+\exp(-10*p))-1$, as there is a significant distribution gap
among multisource domains in the early stages of the training process. The
gradually increasing tradeoff $\lambda$ helps prevent the model from
overfitting to the labeled source data.

Furthermore, a gradient reversal layer is incorporated into the
backpropagation process to facilitate a one-step update, which can be defined
using the following pseudofunction:

$\displaystyle F_{\text{shared}}^{\lambda}({\mathbf{x}})$
$\displaystyle={\mathbf{x}}$ (11)
$\displaystyle\frac{{dF_{\text{shared}}^{\lambda}({\mathbf{x}})}}{{d{\mathbf{x}}}}$
$\displaystyle=-\lambda{\bf{I}}$

where $\bf{I}$ is an identity matrix. Then $F_{\text{shared}}$, $D_{1}$,
$D_{2}$, $\dots$, and $D_{K(K-1)/2}$ can be simultaneously optimized by
minimizing the following domain-invariant loss ${\mathcal{L}}_{\text{inv}}$:

${\mathcal{L}}_{\text{inv}}=\frac{1}{K(K-1)/2}\sum_{k=1}^{K(K-1)/2}\frac{1}{n}\sum_{{\mathbf{x}}_{i}\in{\mathcal{D}}}{\mathcal{l}}_{\text{CE}}(D_{k}(F_{\text{shared}}^{\lambda}({\mathbf{x}}_{i})),s_{i})$
(12)

###  III-D Domain-Specific Module

Figure 3: Schematic diagram of domain-invariant feature and domain-specific
features. (Best viewed in color.)

The domain-specific module comprises $F_{\text{shared}}$, $C_{1}$, $C_{2}$,
$\dots$, $C_{K}$, $F_{\text{weighted}}$ and $C_{\text{weighted}}$. Although we
have extracted domain-invariant features through the domain-invariant module,
in practical applications, the unseen target domain may share similarities
with the domain-specific features of one or more domains in the multisource
domains (see Fig. 3). Thus, we designed a domain-specific module aimed at
extracting domain-specific features to further enhance the model’s
generalization capability.

Specifically, $F_{\text{shared}}$ and $C_{k}$ are introduced to extract the
domain-specific feature
${\mathbf{z}}_{i,k}=C_{k}(F_{\text{shared}}({\mathbf{x}}_{i}))$, where
${\mathbf{x}}_{i}\in{\mathcal{D}}_{s^{\prime}}$ and $k=1,2,\dots,K$. The
features are combined through a domain-related similarity weight
${\mathbf{w}}_{i}=\left[w_{i,1},w_{i,2},\dots,w_{i,K}\right]^{\text{T}}$:

${\mathbf{z}}_{i}=\sum_{k=1}^{K}w_{i,k}\cdot{\mathbf{z}}_{i,k},~{}0\leq
w_{i,k}\leq 1~{}\text{and}~{}\sum_{k=1}^{K}w_{i,k}=1$ (13)

where $w_{i,k}$ denotes the similarity between the input ${\mathbf{x}}_{i}$
and the $k$-th domain. Next, we design an appropriate ${\mathbf{w}}_{i}$ for
${\mathcal{D}}_{s^{\prime}}$ and ${\mathcal{D}}_{t}$, respectively.

On the one hand, if ${\mathbf{x}}_{i}\in{\mathcal{D}}_{s^{\prime}}^{k}$, then
its domain label $d_{i}=k$. In other words, we can consider that
${\mathbf{x}}_{i}$ is exclusively similar to domain $k$ and unrelated to other
source domains. Thus, $w_{i,k}$ can be defined as follows:

$w_{i,k}=\begin{cases}1&\text{if}~{}d_{i}=k\\\
0&\text{otherwise}~{}~{}d_{i}=1,2,\dots,K\end{cases}$ (14)

As a result, the parameters of $F_{\text{shared}}$ and $C_{k}$ can be updated
solely by averaging the losses computed from the labeled samples in
${\mathcal{D}}_{s^{\prime}}^{k}$:

${\mathcal{L}}_{\text{cls}}=\frac{1}{K}\sum_{k=1}^{K}\frac{1}{n^{\prime}_{k}}\sum_{{\mathbf{x}}_{i}\in{\mathcal{D}}_{s^{\prime}}^{k}}{\mathcal{l}}_{\text{CE}}({\mathbf{z}}_{i},y_{i})$
(15)

where $n^{\prime}_{1}=n_{l}$, and if
$({\mathbf{x}}_{i},y_{i})\in{\mathcal{D}}_{s^{\prime}}^{1}$ then
$({\mathbf{x}}_{i},y_{i})=({\mathbf{x}}_{i}^{l},y_{i}^{l})$; if
$({\mathbf{x}}_{i},y_{i})\in{\mathcal{D}}_{s^{\prime}}^{k\neq 1}$ then
$({\mathbf{x}}_{i},y_{i})=({\mathbf{x}}_{i}^{u},\widehat{y_{i}^{u}})$.

On the other hand, if ${\mathbf{x}}_{i}\in{\mathcal{D}}_{t}$ and its domain
label $d_{i}=k$, then one can obtain the corresponding category label through
the trained $F_{\text{shared}}$ and $C_{k}$. Unfortunately, the domain label
is unknown at test time. To solve this problem, a parallel domain classifier,
which is composed of $F_{\text{weighted}}$ and $C_{\text{weighted}}$, is
incorporated into our deep architecture to learn ${\mathbf{w}}_{i}$ between
the target domain and the multisource domains as follows:

${\mathbf{w}}_{i}=C_{\text{weighted}}(F_{\text{weighted}}({\mathbf{x}}_{i}))$
(16)

where $w_{i,k}$ denotes the probability that the input ${\mathbf{x}}_{i}$
belongs to the $k$-th domain, which implies the similarity between
${\mathbf{x}}_{i}$ and the $k$-th domain.

Since the domain label $d_{i}$ for each
${\mathbf{x}}_{i}\in{\mathcal{D}}_{s^{\prime}}$ is known at the train time,
the parameters of $F_{\text{weighted}}$ and $C_{\text{weighted}}$ can be
updated by the following loss:

${\mathcal{L}}_{\text{w}}=\frac{1}{\sum\nolimits_{k=1}^{K}n^{\prime}_{k}}\sum_{{\mathbf{x}}_{i}\in{\mathcal{D}}_{s^{\prime}}}{\mathcal{l}}_{\text{CE}}({\mathbf{w}}_{i},d_{i})$
(17)

where if $({\mathbf{x}}_{i},d_{i})\in{\mathcal{D}}_{s^{\prime}}^{1}$ then
$({\mathbf{x}}_{i},d_{i})=({\mathbf{x}}_{i}^{l},d_{i}^{l}=1)$; if
$({\mathbf{x}}_{i},d_{i})\in{\mathcal{D}}_{s^{\prime}}^{k\neq 1}$ then
$({\mathbf{x}}_{i},d_{i})=({\mathbf{x}}_{i}^{u},d_{i}^{u}=k)$.

Finally, the total domain-specific loss ${\mathcal{L}}_{\text{spe}}$ can be
expressed as follows:

${\mathcal{L}}_{\text{spe}}={\mathcal{L}}_{\text{cls}}+{\mathcal{L}}_{\text{w}}$
(18)

###  III-E Training Procedure of MSADGN

With the definition of ${\mathcal{L}}_{\text{inv}}$ and
${\mathcal{L}}_{\text{spe}}$, the overall loss of MSADGN can be described as
follows:

${\mathcal{L}}={\mathcal{L}}_{\text{inv}}+{\mathcal{L}}_{\text{spe}}$ (19)

The optimal parameters of MSADGN can be obtained using the following
optimization:

$({\hat{\theta}_{f}},{\hat{\theta}_{c}},{\hat{\theta}_{d}})=\mathop{\arg\min}\limits_{{\theta_{f}},{\theta_{c}},{\theta_{d}}}{\mathcal{L}}({\theta_{f}},{\theta_{c}},{\theta_{d}})$
(20)

where $\theta_{f}=(\theta_{f_{1}},\theta_{f_{2}})$ represents the parameters
of $F_{\text{shared}}$ and $F_{\text{weighted}}$, respectively;
$\theta_{c}=(\theta_{c_{1}},\theta_{c_{2}},\dots,\theta_{c_{K}},\theta_{c_{K+1}})$
denotes the parameters of $C_{1}$, $C_{2}$, …, $C_{K}$, and
$C_{\text{weighted}}$, respectively; and $\theta_{d}=(\theta_{d_{1}}$,
$\theta_{d_{2}}$, $\dots$, $\theta_{d_{K(K-1)/2}})$ denotes the parameters of
$D_{1}$, $D_{2}$, $\dots$, and $D_{K(K-1)/2}$, respectively.

Finally, according to Eqs. (13) and (16), our model is capable of making real-
time predictions for the unseen target domain. More importantly, the
prediction results may benefit from the domain-specific features learned from
the multisource domains. The detailed training procedure of MSADGN is outlined
in Algorithm 1. We believe that the proposed MSADGN is expected to improve the
cross-scene sea–land clutter classification performance.

1

Input: Labeled source data
${\mathcal{D}}_{s}^{1}=\\{({\mathbf{x}}_{i}^{l},y_{i}^{l},d_{i}^{l}=1)\\}_{i=1}^{n_{l}}$;

Unlabeled source data ${\mathcal{D}}_{s}^{k\neq
1}=\\{({\mathbf{x}}_{i}^{u},d_{i}^{u}=k)\\}_{i=1}^{n_{k}}$;

Unlabeled target data
${\mathcal{D}}_{t}=\\{{\mathbf{x}}_{j}^{t}\\}_{j=1}^{n_{t}}$

2

Output: Target category labels
${\mathcal{D}}_{t}=\\{y_{j}^{t}\\}_{j=1}^{{n_{t}}}$

3

4// TRAINING

5Initialize Weight parameters: $\theta_{f}$, $\theta_{c}$, and $\theta_{d}$;

6 Maximum number of training epochs: $N_{e}$;

7 Mini-batch size: $m$;

8 Maximum number of mini-batch: $N_{b}$;

9for  _$n_{e}\leftarrow 1$ to $N_{e}$_ do

10 for  _$n_{b}\leftarrow 1$ to $N_{b}$_ do

11 Randomly sample
${\mathcal{M}}_{s}^{1}=\\{({\mathbf{x}}_{i}^{l},y_{i}^{l},d_{i}^{l}=1)\\}_{i=1}^{m}$
from

12 ${\mathcal{D}}_{s}^{1}$;

13 Randomly sample ${\mathcal{M}}_{s}^{k\neq
1}=\\{({\mathbf{x}}_{i}^{u},d_{i}^{u}=k)\\}_{i=1}^{m}$ from

14 ${\mathcal{D}}_{s}^{k\neq 1}$;

15 // Perform domain-related pseudolabeling.

16 Select ${\mathcal{M}}_{s^{\prime}}^{k\neq
1}=\\{({\mathbf{x}}_{i}^{u},\widehat{y_{i}^{u}},d_{i}^{u}=k)\\}_{i=1}^{m^{\prime}_{k}}$
by Eq. (9),

17 where $m^{\prime}_{k}$ represents the number of pseudolabeled

18 samples in ${\mathcal{M}}_{s^{\prime}}^{k\neq 1}$;

19 // Perform domain-invariant and domain-specific

20 DGs.

21 Forward: calculate ${\mathcal{L}}$ by Eq. (19);

22 Backward: update $\theta_{f}$, $\theta_{c}$, and $\theta_{d}$ by:

23
$\theta_{f}\leftarrow\theta_{f}-\frac{\partial{\mathcal{L}}}{\partial\theta_{f}}$

24
$\theta_{c}\leftarrow\theta_{c}-\frac{\partial\mathcal{L}}{\partial\theta_{c}}$

25
$\theta_{d}\leftarrow\theta_{d}-\frac{\partial\mathcal{L}}{\partial\theta_{d}}$.

26 end for

27

28 end for

29// PREDICTION

30Classify the target data $\\{{\mathbf{x}}_{j}^{t}\\}_{j=1}^{{n_{t}}}$ by
Eqs. (13) and (16).

return $\\{y_{j}^{t}\\}_{j=1}^{{n_{t}}}$

Algorithm 1 Training procedure of MSADGN.

###  III-F Theoretical Insight

In this section, we theoretically demonstrate the rationality of our method
using the DG theory proposed in [26].

Theorem 1  (Domain generalization error upper bound [26]) Let
$\xi=d_{\mathcal{H}}({\mathcal{D}}_{t},\bar{\mathcal{D}}_{t})$ denotes the
$\mathcal{H}$-divergence between the unseen target domain ${\mathcal{D}}_{t}$
and its nearest neighbor in the convex hull of multisource domains
${\mathcal{D}}_{s^{\prime}}$. Let $\eta$ denotes the largest distribution
divergence between ${\mathcal{D}}_{s}^{i}$ and ${\mathcal{D}}_{s}^{j}$, $i\neq
j$. Then, the error on ${\mathcal{D}}_{t}$ of hypothesis $h$ is upper-bound by
the weighted error on ${\mathcal{D}}_{s^{\prime}}$:

$\displaystyle\epsilon_{t}(h)\leq$
$\displaystyle\sum_{i=1}^{N_{s^{\prime}}}\pi_{i}^{*}\epsilon_{s^{\prime}}^{i}(h)+\frac{\xi+\eta}{2}$
(21) $\displaystyle+\min\\{{\mathbb{E}_{\bar{\mathcal{D}}_{t}}}[\lvert
f_{t}-f_{s^{\prime}_{\pi}}\rvert],{\mathbb{E}_{{\mathcal{D}}_{t}}}[\lvert
f_{t}-f_{s^{\prime}_{\pi}}\rvert]\\}$

where $\min\\{{\mathbb{E}_{\bar{\mathcal{D}}_{t}}}[\lvert
f_{t}-f_{s^{\prime}_{\pi}}\rvert],{\mathbb{E}_{{\mathcal{D}}_{t}}}[\lvert
f_{t}-f_{s^{\prime}_{\pi}}\rvert]\\}$ denotes the different labeling functions
between $f_{t}$ and $f_{s^{\prime}_{\pi}}$. $\square$

In the context of the cross-scene sea–land clutter classification problem, we
consider that the label space of ${\mathcal{D}}_{s}$ (or
${\mathcal{D}}_{s^{\prime}}$) and ${\mathcal{D}}_{t}$ is the same, and the
distribution difference between the two is not particularly large. This aligns
approximately with the covariate shift assumption, suggesting that variations
in the input multisource domain distribution do not significantly impact the
target domain label distribution. Thus, it can be inferred that $\xi$ and the
labeling function error ($\min\\{\cdot,\cdot\\}$) are relatively small.

On the other hand, the risk $\eta$ is bounded by Eq. (12) of the domain-
invariant module. In addition, the weighted error
$\sum_{i=1}^{N_{s^{\prime}}}\pi_{i}^{*}\epsilon_{s^{\prime}}^{i}(h)$ on
${\mathcal{D}}_{s^{\prime}}$ is bounded by Eqs. (13) and (16) of the domain-
specific module. Therefore, our MSADGN can be guaranteed by Theorem III-F.

##  IV Experiments and Analysis

In this section, extensive experiments are conducted to validate the
effectiveness and superiority of our proposed MSADGN. First, we introduce two
benchmark datasets: CS-SLCS and CS-HRRSI, along with experimental
implementation details. The experiments on CS-HRRSI aim to demonstrate the
generality of MSADGN. Second, we compare MSADGN with state-of-the-art methods
to verify its superiority. Third, an ablation study is conducted to
demonstrate the effectiveness of MSADGN. Finally, further empirical analysis
is performed for our method.

###  IV-A Experimental Settings

####  IV-A1 Dataset Preparation

See Fig. 4, CS-SLCS is the clutter spectrum obtained from OTHR, which contains
three categories of signals (1000 sea clutter, 1000 land clutter, and 1000
sea–land boundary clutter) from four domains defined by different coherent
integration numbers (CINs): 128 CIN, 256 CIN, 512 CIN, and 1024 CIN. Please
refer to our previous work [6] for a detailed description of CS-SLCS, where
CS-SLCS with 256 CIN, 512 CIN, and 1024 CIN is utilized. Here, to
comprehensively validate the performance of MSADGN, we add the domain 128 CIN.
For convenience, 128 CIN, 256 CIN, 512 CIN, and 1024 CIN are abbreviated as R,
S, M, and L, respectively. Besides, the dimension of signals from all domains
is resized to $1\times 512$ to match the model’s input requirements.

(a) R

(b) S

(c) M

(d) L

Figure 4: Samples of CS-SLCS. Order of samples from left to right: 1) sea
clutter; 2) land clutter; and 3) sea–land boundary clutter.

See Fig. 5, CS-HRRSI contains nine categories of images (beach, commercial,
residential, farmland, forest, parking, port, river, and overpass) from four
domains: NWPU-RESISC45 [34], AID [35], UC Merced [36], and WHU-RS19 [37].
Please refer to our previous work [6] for a detailed description of CS-HRRSI,
where CS-HRRSI with NWPU-RESISC45, AID, and UC Merced is utilized. Here, to
comprehensively validate the performance of MSADGN, we add the domain WHU-
RS19. For convenience, NWPU-RESISC45, AID, UC Merced and WHU-RS19 are
abbreviated as N, A, U and W, respectively. Besides, the dimension of images
from all domains is resized to $224\times 224$ pixels to match the model’s
input requirements.

(a) N

(b) A

(c) U

(d) W

Figure 5: Samples of CS-HRRSI. Order of samples from top to bottom: 1) beach;
2) commercial; 3) residential; 4) farmland; 5) forest; 6) parking; 7) port; 8)
river; and 9) overpass.

####  IV-A2 Implementation Details

In our experiment, we adopt the DCNN architecture used in [6] as the backbone
of MSADGN and all comparison methods. In other words, in the CS-SLCS task, the
feature extractor is composed of four 1-D convolutional (Conv1d) layers, the
classifier or discriminator is composed of three FC layers, and Gaussian
weight initialization is applied to the feature extractor, classifier, and
discriminator to stabilize training; in the CS-HRRSI task, the feature
extractor is composed of ResNet-50 pretrained on the ImageNet dataset, the
classifier or discriminator consists of three FC layers, and Gaussian weight
initialization is applied to the classifier and discriminator to stabilize
training. The only difference is that the output dimension of
$C_{\text{weighted}}$ for these two tasks is changed from $C$ to $K$. Besides,
we set the batch size to 32 for each domain and employ the Adam optimizer [38]
with an initial learning rate of 0.0001 and a decay rate of 0.5 every 10
epochs. The tradeoff parameter $\alpha$ in Eq. (8) is set to 0.2. To ensure
reliable results, we conduct five experiments and report the mean
classification accuracy after 50 epochs. All experiments are conducted using
the PyTorch DL library [39] on an NVIDIA GeForce RTX 3090 GPU.

###  IV-B Comparisons with State-of-the-Art methods

Abundant state-of-the-art methods are selected to verify the superiority of
MSADGN. These methods are categorized into three groups: one baseline (BL)
method, five DA-based DG methods, and five pure SSDG methods. For fairness in
comparative results, all methods adopt consistent datasets, hyperparameters,
and submodules, as detailed in Section IV-A2. The comparison methods are
listed as follows.

  * 1)

BL. DCNN without DG [40] (also called ERM) is used as the BL method, meaning
only the labeled source domain is used to train DCNN.

  * 2)

DA. The selected DA-based DG methods include deep domain confusion (DDC) [15],
deep adaptation network (DAN) [16], DANN [32], error-correcting boundaries
mechanism with feature adaptation metric (ECB-FAM) [41], and TLADAN [6]. Among
these, DDC and DAN can be considered as discrepancy-based methods; DANN can be
regarded as an adversarial-based method; and ECB-FAM and TLADAN can be viewed
as discrepancy- and adversarial-based methods. In this group, the model is
trained using both the labeled source domain and the unlabeled source domain
(by combining all unlabeled source domains into one domain).

  * 3)

DSDGN. It is an SSDG method in fault diagnosis. Due to the commonality of 1-D
signals in both sea–land clutter and fault diagnosis data, it is also
applicable to CS-SLCS. DSDGN integrates the traditional pseudolabel method
into DANN, enabling the realization of SSDG with an accessible labeled source
domain and an accessible unlabeled source domain. In the context of
multisource SSDG, the discriminator of DSDGN is expanded to $K(K-1)/2$ to
distinguish the features of any two source domains. Since our MSADGN is an
improved version of DSDGN, it can be viewed as a BL method at the DG level.

  * 4)

DFGN. It is also an SSDG method in fault diagnosis and employs the pseudolabel
technique. Different from DSDGN, domain fuzzy generalization network (DFGN)
[42] utilizes pseudolabel to compute the center loss and expands the output
dimension of the discriminator into $K$ to achieve multisource SSDG.

  * 5)

BPL. It is an improved version of DSDGN in CV. On the one hand, the
traditional pseudolabel is improved to a domainaware pseudolabeling module in
better pseudolabel (BPL) [43]. Note that the difference between this module
and our domain-related pseudolabeling module lies in the definition of
$\psi({\mathbf{x}}_{i}^{u})$. The former computes ${\mathbf{M}}_{1}$ using
${\mathcal{D}}_{s}^{k\neq 1}$, while the latter computes ${\mathbf{M}}_{1}$
using ${\mathcal{D}}_{s}^{1}$. On the other hand, the mixup data augmentation
technique is applied to generate additional domains and a dual classifier is
integrated into the network to enhance the model’s generalization capability.

  * 6)

ADGN. It is also an SSDG method in fault diagnosis. Note that the adversarial
DG network (ADGN) [44] can be viewed as an improved version of the DA via a
task-specific classifier (DATSNET) [45], which is a DA method in remote
sensing. Specifically, the total discrepancy loss of the dual classifier is
the sum of discrepancy losses computed from each unlabeled source domain.

  * 7)

ECB-FAM-DG. To the best of our knowledge, there is no SSDG method in remote
sensing, particularly in OTHR. Similar to the way ADGN improves DATSNET, we
improve ECB-FAM for DG (ECB-FAM-DG). On the one hand, the total discrepancy
loss of the three classifiers is also the sum of discrepancy losses computed
from each unlabeled source domain. On the other hand, in the FAM structure,
the shallow common features are extracted from multisource domains.

Table. II lists the experimental results on CS-SLCS. From the average results,
it can be seen that all DA-based DG methods are better than the BL method (by
about 5%-8%) and worse than the pure DG methods (by about 2%-7%). The reason
for the former is that they attempt to align features between the labeled
source domain and the unlabeled source domain, enabling the learned features
to generalize well to the unseen target domain. The reason for the latter is
that, although feature alignment is performed, the unlabeled source domains
are considered together, thus failing to adequately extract domain-invariant
features between multisource domains. Further, MSADGN performs the best (about
12% higher than BL). We suppose the reason is that MSADGN can extract
excellent domain-specific features. Besides, MSADGN achieves the best or
second-best results in each DG task. This validates the superiority of our
method.

Table. III lists the experimental results on CS-HRRSI. From the average
results, it can be seen that all DA-based DG methods are better than the BL
method (by about 1%-4%) and worse than the pure DG methods (by about 1%-5%).
Besides, MSADGN performs the best (about 6% higher than BL). Due to the
increased complexity of DG tasks on CS-HRRSI compared to CS-SLCS, the accuracy
improvement of all DA-based DG methods and pure DG methods on CS-HRRSI is not
as obvious as the improvement observed on CS-SLCS. However, a consistent
conclusion can be drawn that pure DG methods outperform DA-based DG methods,
with MSADGN exhibiting optimal performance. Besides, MSADGN achieves the best
or second-best results in most DG tasks. This further validates the
superiority of our method.

TABLE II: Classification Accuracy (%) with State-of-the-Art Methods on CS-SLCS
(The Title in The First Row Denotes ${\mathcal{D}}_{t}$ and in The Second Row
Denotes ${\mathcal{D}}_{s}^{1}$, and The Best Results Are in Bold and Second-
Best Results Are Underlined)

Method | R | S | M | L | Avg
---|---|---|---|---|---
|  | S | M | L | R | M | L | R | S | L | R | S | M |
BL | ERM [40] | 89.01 | 82.37 | 73.40 | 81.33 | 83.83 | 73.90 | 76.71 | 77.39 | 87.48 | 68.11 | 78.23 | 92.56 | 80.36
DA | DDC [15] | 92.43 | 85.58 | 80.00 | 87.39 | 89.50 | 81.32 | 82.11 | 85.46 | 89.02 | 80.25 | 82.72 | 94.44 | 85.85
DAN [16] | 92.67 | 86.31 | 81.62 | 87.21 | 88.58 | 81.57 | 83.98 | 86.82 | 88.53 | 81.40 | 81.18 | 93.97 | 86.15
DANN [32] | 92.99 | 88.05 | 83.50 | 88.98 | 90.34 | 82.73 | 84.03 | 87.29 | 89.31 | 83.17 | 82.80 | 94.22 | 87.28
ECB-FAM [41] | 92.91 | 87.44 | 84.02 | 89.70 | 90.59 | 83.10 | 85.20 | 88.18 | 89.02 | 83.32 | 83.11 | 93.59 | 87.52
TLADAN [6] | 93.32 | 88.06 | 83.88 | 89.99 | 90.82 | 84.19 | 86.30 | 89.52 | 89.35 | 84.01 | 83.38 | 94.27 | 88.09
DG | DSDGN [31] | 94.13 | 91.52 | 85.87 | 92.01 | 93.30 | 86.00 | 88.56 | 92.21 | 90.77 | 86.87 | 85.49 | 95.93 | 90.22
DFGN [42] | 95.20 | 91.03 | 86.35 | 92.53 | 92.99 | 85.87 | 88.53 | 92.48 | 91.53 | 87.72 | 86.21 | 96.08 | 90.54
BPL [43] | 94.63 | 90.88 | 87.25 | 92.42 | 93.71 | 86.66 | 88.00 | 92.73 | 92.03 | 88.48 | 86.81 | 96.13 | 90.81
ADGN [44] | 95.03 | 90.67 | 87.39 | 91.87 | 93.52 | 86.34 | 87.98 | 93.50 | 91.54 | 88.02 | 87.55 | 95.28 | 90.72
ECB-FAM-DG [41] | 94.39 | 90.94 | 87.40 | 93.04 | 93.00 | 85.86 | 88.36 | 93.99 | 91.84 | 89.30 | 87.48 | 96.07 | 90.97
MSADGN (Ours) | 95.17 | 91.44 | 88.48 | 94.80 | 95.38 | 87.42 | 90.00 | 94.09 | 93.52 | 89.27 | 88.01 | 97.73 | 92.11

TABLE III: Classification Accuracy (%) with State-of-the-Art Methods on CS-
HRRSI (The Title in The First Row Denotes ${\mathcal{D}}_{t}$ and in The
Second Row Denotes ${\mathcal{D}}_{s}^{1}$, and The Best Results Are in Bold
and Second-Best Results Are Underlined)

Method | N | A | U | W | Avg
---|---|---|---|---|---
|  | A | U | W | N | U | W | N | A | W | N | A | U |
BL | ERM [40] | 85.14 | 64.17 | 73.12 | 94.46 | 67.01 | 86.93 | 84.82 | 75.64 | 65.55 | 95.21 | 98.75 | 71.88 | 80.22
DA | DDC [15] | 88.48 | 64.89 | 73.42 | 95.14 | 69.12 | 87.33 | 86.09 | 76.64 | 67.90 | 97.08 | 99.13 | 72.71 | 81.49
DAN [16] | 89.21 | 66.67 | 73.85 | 94.89 | 70.40 | 87.71 | 86.45 | 76.27 | 65.64 | 96.63 | 99.33 | 73.12 | 81.68
DANN [32] | 89.04 | 76.21 | 73.57 | 95.67 | 80.53 | 87.51 | 85.36 | 80.09 | 67.91 | 96.46 | 99.36 | 79.58 | 84.27
ECB-FAM [41] | 89.21 | 73.23 | 74.56 | 94.91 | 78.52 | 87.68 | 87.00 | 79.55 | 66.00 | 96.67 | 99.42 | 81.87 | 84.05
TLADAN [6] | 89.15 | 75.45 | 75.02 | 96.20 | 81.22 | 87.08 | 86.73 | 78.00 | 66.00 | 97.29 | 99.38 | 81.67 | 84.43
DG | DSDGN [31] | 89.87 | 79.93 | 79.00 | 95.84 | 80.05 | 90.13 | 87.18 | 79.55 | 67.55 | 96.04 | 98.75 | 83.75 | 85.64
DFGN [42] | 91.61 | 76.07 | 77.24 | 95.67 | 77.99 | 89.47 | 86.73 | 78.73 | 66.55 | 97.71 | 99.58 | 77.92 | 84.61
BPL [43] | 90.27 | 77.74 | 78.43 | 96.65 | 79.92 | 92.37 | 86.73 | 78.55 | 71.36 | 96.46 | 99.79 | 78.33 | 85.55
ADGN [44] | 90.33 | 77.55 | 77.02 | 96.25 | 79.04 | 89.82 | 87.36 | 80.09 | 70.00 | 97.29 | 99.88 | 86.04 | 85.89
ECB-FAM-DG [41] | 90.75 | 75.80 | 77.76 | 95.97 | 83.07 | 92.17 | 88.73 | 79.35 | 70.09 | 97.08 | 99.38 | 84.00 | 86.18
MSADGN (Ours) | 90.93 | 80.68 | 78.49 | 96.69 | 84.87 | 92.19 | 87.01 | 80.64 | 70.82 | 98.01 | 98.92 | 84.58 | 86.99

###  IV-C Ablation Study

An ablation study is conducted to demonstrate the effectiveness of different
components of MSADGN. We randomly select two cross-scene scenarios on CS-SLCS
and CS-HRRSI, respectively, and obtain the results as shown in Table IV using
the following methods.

  * 1)

M1. It refers to ERM, which does not include a domain-related pseudolabeling
module, domain-invariant module, or domain-specific module, serving as a BL
method.

  * 2)

M2. It does not take into account domain-related information in the
pseudolabeling module, that is, the traditional pseudolabel method is employed
in this module.

  * 3)

M3. It replaces the dynamic threshold of MSADGN with a traditional fixed
threshold. Specifically, setting $p=0$ in Eq. (7) yields $\tau_{0}=0.4$.

  * 4)

M4. It refers to using the local iteration scheme instead of the global
iteration scheme to update ${\mathbf{M}}_{1}$ in Eq. (3).

  * 5)

M5. It removes the domain-invariant module to assess the importance of the
domain-invariant feature for cross-scene classification.

  * 6)

M6. It removes the domain-specific module to assess the importance of the
domain-specific features for cross-scene classification.

  * 7)

M7. It refers to the full version of MSADGN.

From the experimental results, it can be observed that the performance of M2
to M7 is superior to that of M1. This is because the domain-invariant or
domain-specific features learned by these methods help reduce the distribution
difference between the multisource domains and the target domain. Among them,
our proposed M7 achieves the best performance. This is due to the effective
utilization of unlabeled source domains by the domain-related pseudolabeling
module, as well as the more comprehensive features learned by the domain-
invariant module and domain-specific module. Therefore, the effectiveness of
different components of MSADGN is verified.

TABLE IV: Ablation Study Results (%), Where the First Domain Denotes
${\mathcal{D}}_{s}^{1}$

Method | Domain-related pseudolabeling module | Domain-invariant module | Domain-specific module | CS-SLCS | CS-HRRSI
---|---|---|---|---|---
| Probability score | Similarity score | Dynamic threshold | Global iteration scheme |  |  | LRS$\rightarrow$M | SRM$\rightarrow$L | AUW$\rightarrow$N | WNA$\rightarrow$U
M1 |  |  |  |  |  |  | 87.48 | 78.23 | 85.14 | 65.55
M2 | ✓ |  |  |  | ✓ | ✓ | 91.38 | 87.06 | 87.32 | 68.64
M3 | ✓ | ✓ |  | ✓ | ✓ | ✓ | 93.21 | 87.30 | 87.58 | 67.64
M4 | ✓ | ✓ | ✓ |  | ✓ | ✓ | 91.72 | 86.44 | 89.17 | 68.73
M5 | ✓ | ✓ | ✓ | ✓ |  | ✓ | 91.60 | 85.86 | 88.69 | 65.82
M6 | ✓ | ✓ | ✓ | ✓ | ✓ |  | 92.43 | 86.05 | 89.77 | 66.09
M7 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 93.52 | 88.01 | 90.93 | 70.82

###  IV-D Further Empirical Analysis

In this section, we conduct further empirical analysis on the hyperparameters,
network structure, and convergence of MSADGN to validate the superiority of
the method. The benchmarks utilized include CS-SLCS and CS-HRRSI. Unless
specified otherwise, CS-SLCS is illustrated with the scenario of
LRS$\rightarrow$M, and CS-HRRSI is demonstrated with the scenario of
AUW$\rightarrow$N, where the first domain denotes the labeled source domain.

####  IV-D1 Hyperparameters of The Domain-Related pseudolabeling Module

Here, different values of $\alpha$ in Eq. (8) are set to explore the influence
of the proportion between $\phi({\mathbf{x}}_{i}^{u})$ and
$\psi({\mathbf{x}}_{i}^{u})$ on the performance of MSADGN. Specifically, we
set $\alpha=[0.0,1.0]$ with a step size of 0.1 and conduct experiments on the
benchmarks. The experimental results are shown in Fig.
6LABEL:sub@subfig:_parameter, from which the following conclusions can be
drawn. When $\alpha$ is larger, MSADGN performs poorly on the benchmarks. This
could be attributed to the dominance of the traditional
$\phi({\mathbf{x}}_{i}^{u})$, especially when $\alpha=1.0$, resulting in the
worst performance due to the exclusive presence of
$\phi({\mathbf{x}}_{i}^{u})$. Conversely, when $\alpha$ is smaller, MSADGN
demonstrates better performance on the benchmarks, with the optimal
performance observed at $\alpha=0.2$. This is owing to the incorporation of
$\psi({\mathbf{x}}_{i}^{u})$. Therefore, as mentioned earlier, we set
$\alpha=0.2$ in all experiments.

####  IV-D2 Discriminators of The Domain-Invariant Module

In the domain-invariant module, the $K(K-1)/2$ binary discriminators are
replaced by a single $K$-class discriminator to investigate the influence of
the number of discriminators on MSADGN. It can be seen from Fig.
6LABEL:sub@subfig:_discriminator that the performance of the former surpasses
that of the latter on the benchmarks. One possible reason for this is that the
latter simultaneously extracts the domain-invariant feature from $K$
multisource domains, which may increase confusion between categories and make
it difficult for the model to accurately generalize to the unseen target
domain.

####  IV-D3 Weights of The Domain-Specific Module

The domain-related similarity weights of a sample randomly selected from the
unseen target domain of the benchmarks are shown in Fig.
6LABEL:sub@subfig:_weight. As mentioned earlier, the weights imply the
similarity between the target domain and the multisource domains. Therefore,
MSADGN effectively learns the similarity between the target domain and each
source domain, so that the prediction results can benefit from the domain-
specific features learned from the multisource domains.

####  IV-D4 Convergence of The Proposed MSADGN

we further empirically analyze the convergence of the proposed MSADGN. Fig.
6LABEL:sub@subfig:_loss plots the loss curve of MSADGN on CS-HRRSI. Similar
observations are noted for CS-SLCS. The experimental results show that the
training of MSADGN is stable and converges rapidly.

(a)

(b)

(c)

(d)

Figure 6: Further empirical analysis of MSADGN. (a) Sensitivity analysis of
$\alpha$ in the domain-related pseudolabeling module on benchmarks. (b)
Analysis of the number of discriminators in the domain-invariant module. (c)
Weights of a sample randomly selected from the unseen target domain of
benchmarks. (d) Loss curve of MSADGN on CS-HRRSI.

####  IV-D5 Categorywise Accuracy

It should be emphasized that MSADGN is an improved version of DSDGN. To
demonstrate the superiority of our method over DSDGN, we utilize the confusion
matrix for categorywise accuracy analysis on the benchmarks. For both CS-SLCS
and CS-HRRSI, consistent conclusions can be drawn from Fig. 7, indicating that
the performance of both DSDGN and MSADGN significantly surpasses that of ERM.
This is attributed to their capability for DG. Furthermore, the performance of
MSADGN surpasses that of DSDGN, owing to its superior domain-related
pseudolabeling module and the exclusive domain-specific module. For example,
ERM exhibits poor classification accuracy (75%) for category 0 on CS-SLCS,
whereas both DSDGN and MSADGN show significant improvements in classification
accuracies of 78% and 82% for category 0, respectively. Similarly, ERM
exhibits poor classification accuracy (74%) for category 7 and (54%) for
category 8 on CS-HRRSI. In contrast, both DSDGN and MSADGN show significant
improvements in classification accuracies of 84% and 86% for category 7, and
71% and 77% for category 8, respectively. Therefore, the superiority of MSADGN
in categorywise accuracy is validated.

(a)

(b)

(c)

Figure 7: Confusion matrices on (top) CS-SLCS and (bottom) CS-HRRSI. (a) ERM.
(b) DSDGN. (c) MSADGN.

####  IV-D6 Feature Visualization

Finally, the $\bm{t}$-distributed stochastic neighbor embedding ($\bm{t}$-SNE)
[46] is utilized to visualize the features learned by $F_{\text{shared}}$ with
various methods. To more intuitively demonstrate the superiority of DG over DA
and MSADGN over other methods, we conduct experiments on a randomly selected
UNA$\rightarrow$W task with large differences in classification accuracy,
where $\bf U$ denotes the labeled source domain. Specifically, six methods are
selected, including one ERM as BL, three DA-based DG methods (DAN without
adversarial training, DANN with adversarial training, and TLADAN with
excellent DA performance proposed in our previous work [6]), and two pure DG
methods (DSDGN and the proposed MSADGN). The experimental results are shown in
Fig. 8, from which the following conclusions can be drawn.

  * 1)

See Fig. 8LABEL:sub@subfig:_tsne_ERM, the features learned by ERM exhibit
significant confusion among categories, particularly between categories
$\text{T}_{1}$ and $\text{T}_{2}$. This is attributed to the domain bias
between the source and target domains, and ERM does not incorporate DG
techniques.

  * 2)

See Figs. 8LABEL:sub@subfig:_tsne_DAN, 8LABEL:sub@subfig:_tsne_DANN and
8LABEL:sub@subfig:_tsne_TLADAN, after applying DA-based DG methods, the
feature alignment effect is improved compared to ERM. Notably, the improvement
effect of DAN is not obvious, while DANN and TLADAN exhibit more significant
improvements, with almost all features of the same categories gathered
together. This is due to the adversarial training used by the latter to learn
more effective features.

  * 3)

See Figs. 8LABEL:sub@subfig:_tsne_DSDGN and 8LABEL:sub@subfig:_tsne_MSADGN,
after using pure DG methods DSDGN and MSADGN, the intraclass aggregation and
interclass separation between features are more obvious than those of the DA-
based DG methods. Although the intraclass aggregation ability of MSADGN is
slightly inferior to that of DSDGN (e.g., in categories $\text{T}_{1}$,
$\text{T}_{6}$, and $\text{T}_{8}$), MSADGN demonstrates better interclass
separation ability (e.g., between categories $\text{T}_{1}$ and
$\text{T}_{3}$, and between categories $\text{T}_{4}$ and $\text{T}_{7}$).
This makes the nine feature clusters from the target domain learned by MSADGN
more distinguishable. Therefore, MSADGN performs the best among all methods,
benefiting from the more comprehensive features learned by the domain-specific
module.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 8: $\bm{t}$-SNE feature visualization on UNA$\rightarrow$W, where “S”
and “T” in the legend denote ${\mathcal{D}}_{s}^{1}$ and ${\mathcal{D}}_{t}$.
(a) ERM. (b) DAN. (c) DANN. (d) TLADAN. (e) DSDGN. (f) MSADGN.

##  V Conclusion

This study proposed a novel MSADGN for cross-scene sea–land clutter
classification. We considered a more practical yet complex SSDG scenario in
which only one labeled source domain and multiple unlabeled source domains
were available. Specifically, an improved pseudolabel method called domain-
related pseudolabel was proposed to generate reliable pseudolabels.
Furthermore, the domain-invariant features learned by a multidiscriminator and
the domain-specific features learned by a multiclassifier enabled MSADGN to
exhibit excellent transferability and discriminability in the unseen target
domain. Our method outperformed state-of-the-art DG methods in numerous cross-
scene classification tasks.

Future research involves more complex sea–land clutter classification. For
example, if only one labeled source domain is available, our model will be
greatly limited. Therefore, it is necessary to expand the investigation into
single-source DG.

## References

  * [1] X. Zhu _et al._ , “Deep learning in remote sensing: A comprehensive review and list of resources,” _IEEE geoscience and remote sensing magazine_ , vol. 5, no. 4, pp. 8–36, 2017.
  * [2] Z. Guo, Z. Wang, Y. Hao, H. Lan, and Q. Pan, “An improved coordinate registration for over-the-horizon radar using reference sources,” _Electronics_ , vol. 10, no. 24, p. 3086, 2021.
  * [3] C. Li, Z. Wang, Z. Zhang, H. Lan, and K. Lu, “Sea/land clutter recognition for over-the-horizon radar via deep CNN,” in _Proceedings of 2019 International Conference on Control, Automation and Information Sciences_. IEEE, 2019, pp. 1–5.
  * [4] C. Li, Y. Zhang, Z. Wang, K. Lu, and Q. Pan, “Cross-scale land/sea clutter classification method for over-the-horizon radar based on algebraic multigrid,” _Acta Electronica Sinica_ , vol. 50, no. 12, pp. 3021–3029, 2022.
  * [5] X. Zhang, Z. Wang, K. Lu, Q. Pan, and Y. Li, “Data augmentation and classification of sea–land clutter for over-the-horizon radar using AC-VAEGAN,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 61, pp. 1–16, 2023.
  * [6] X. Zhang, Y. Li, Q. Pan, and C. Yu, “Triple loss adversarial domain adaptation network for cross-domain sea–land clutter classification,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 61, pp. 1–18, 2023.
  * [7] X. Zhang, Z. Wang, M. Ji, Y. Li, Q. Pan, and K. Lu, “A sea–land clutter classification framework for over-the-horizon radar based on weighted loss semi-supervised generative adversarial network (In press),” _Engineering Applications of Artificial Intelligence_ , 2024.
  * [8] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” _Advances in neural information processing systems_ , vol. 25, pp. 1097–1105, 2012.
  * [9] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” _Advances in neural information processing systems_ , vol. 27, pp. 1–9, 2014.
  * [10] D. P. Kingma and M. Welling, “Auto-encoding variational Bayes,” in _Proceedings of the International Conference on Learning Representation_ , 2014, pp. 1–14.
  * [11] Z. Jin, Q. Pan, C. Zhao, and W. Zhou, “SVM based land/sea clutter classification algorithm,” _Applied Mechanics and Materials_ , vol. 236-237, pp. 1156–1162, 2012.
  * [12] M. Turley, R. Gardiner-Garden, and D. Holdsworth, “High-resolution wide area remote sensing for HF radar track registration,” in _Proceedings of 2013 International Conference on Radar_. IEEE, 2013, pp. 128–133.
  * [13] M. HassanPour Zonoozi and V. Seydi, “A survey on adversarial domain adaptation,” _Neural Processing Letters_ , vol. 55, no. 3, pp. 2429–2469, 2023.
  * [14] G. Wilson and D. J. Cook, “A survey of unsupervised deep domain adaptation,” _ACM Transactions on Intelligent Systems and Technology_ , vol. 11, no. 5, pp. 1–46, 2020.
  * [15] E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, and T. Darrell, “Deep domain confusion: Maximizing for domain invariance,” _arXiv preprint arXiv:1412.3474_ , 2014.
  * [16] M. Long, Y. Cao, J. Wang, and M. Jordan, “Learning transferable features with deep adaptation networks,” in _International conference on machine learning_. PMLR, 2015, pp. 97–105.
  * [17] B. Sun and K. Saenko, “Deep coral: Correlation alignment for deep domain adaptation,” in _Computer Vision–ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part III 14_. Springer, 2016, pp. 443–450.
  * [18] W. Zellinger, T. Grubinger, E. Lughofer, T. Natschläger, and S. Saminger-Platz, “Central moment discrepancy (cmd) for domain-invariant representation learning,” _arXiv preprint arXiv:1702.08811_ , 2017.
  * [19] L. Wan, Y. Li, K. Chen, K. Gong, and C. Li, “A novel deep convolution multi-adversarial domain adaptation model for rolling bearing fault diagnosis,” _Measurement_ , vol. 191, p. 110752, 2022.
  * [20] L. Yan, R. Zhu, N. Mo, and Y. Liu, “Cross-domain distance metric learning framework with limited target samples for scene classification of aerial images,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 57, no. 6, pp. 3840–3857, 2019.
  * [21] M. Ghifary, W. B. Kleijn, M. Zhang, D. Balduzzi, and W. Li, “Deep reconstruction-classification networks for unsupervised domain adaptation,” in _Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14_. Springer, 2016, pp. 597–613.
  * [22] W. Zhu, B. Shi, Z. Feng, and J. Tang, “An unsupervised domain adaptation method for intelligent bearing fault diagnosis based on signal reconstruction by cycle-consistent adversarial learning,” _IEEE Sensors Journal_ , vol. 23, no. 16, pp. 18 477–18 485, 2023.
  * [23] Y. Ganin _et al._ , “Domain-adversarial training of neural networks,” _The journal of machine learning research_ , vol. 17, no. 1, pp. 2096–2030, 2016.
  * [24] Y. Wang, X. Sun, J. Li, and Y. Yang, “Intelligent fault diagnosis with deep adversarial domain adaptation,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 70, pp. 1–9, 2020.
  * [25] Y. Huang _et al._ , “Cross-scene wetland mapping on hyperspectral remote sensing images using adversarial domain adaptation network,” _ISPRS Journal of Photogrammetry and Remote Sensing_ , vol. 203, pp. 37–54, 2023.
  * [26] J. Wang _et al._ , “Generalizing to unseen domains: A survey on domain generalization,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 35, no. 8, pp. 8052–8072, 2023.
  * [27] K. Zhou, Y. Yang, Y. Qiao, and T. Xiang, “Domain generalization with mixstyle,” in _International Conference on Learning Representations_ , 2021, pp. 1–15.
  * [28] Z. Fan, Q. Xu, C. Jiang, and S. X. Ding, “Deep mixed domain generalization network for intelligent fault diagnosis under unseen conditions,” _IEEE Transactions on Industrial Electronics_ , vol. 71, no. 1, pp. 965–974, 2024.
  * [29] S. Zhu, C. Wu, B. Du, and L. Zhang, “Style and content separation network for remote sensing image cross-scene generalization,” _ISPRS Journal of Photogrammetry and Remote Sensing_ , vol. 201, pp. 1–11, 2023.
  * [30] L. Ren, T. Mo, and X. Cheng, “Meta-learning based domain generalization framework for fault diagnosis with gradient aligning and semantic matching,” _IEEE Transactions on Industrial Informatics_ , vol. 20, no. 1, pp. 754–764, 2024.
  * [31] Y. Liao, R. Huang, J. Li, Z. Chen, and W. Li, “Deep semisupervised domain generalization network for rotary machinery fault diagnosis under variable speed,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 69, no. 10, pp. 8064–8075, 2020.
  * [32] Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” in _International conference on machine learning_. PMLR, 2015, pp. 1180–1189.
  * [33] D.-H. Lee, “Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,” in _Workshop on challenges in representation learning, ICML_ , vol. 3, no. 2. Atlanta, 2013, p. 896.
  * [34] G. Cheng, J. Han, and X. Lu, “Remote sensing image scene classification: Benchmark and state of the art,” _Proceedings of the IEEE_ , vol. 105, no. 10, pp. 1865–1883, 2017.
  * [35] G.-S. Xia _et al._ , “Aid: A benchmark data set for performance evaluation of aerial scene classification,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 55, no. 7, pp. 3965–3981, 2017.
  * [36] Y. Yang and S. Newsam, “Bag-of-visual-words and spatial extensions for land-use classification,” in _Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems_. ACM, 2010, pp. 270–279.
  * [37] G.-S. Xia, W. Yang, J. Delon, Y. Gousseau, H. Sun, and H. Maître, “Structural high-resolution satellite image indexing,” in _ISPRS TC VII Symposium-100 Years ISPRS_ , 2010, pp. 298–303.
  * [38] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [39] A. Paszke, _et al._ , “Pytorch: An imperative style, high-performance deep learning library,” _Advances in neural information processing systems_ , vol. 32, pp. 8026–8037, 2019.
  * [40] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_. IEEE, 2016, pp. 770–778.
  * [41] C. Ma, D. Sha, and X. Mu, “Unsupervised adversarial domain adaptation with error-correcting boundaries and feature adaption metric for remote-sensing scene classification,” _Remote Sensing_ , vol. 13, no. 7, p. 1270, 2021.
  * [42] H. Ren, J. Wang, Z. Zhu, J. Shi, and W. Huang, “Domain fuzzy generalization networks for semi-supervised intelligent fault diagnosis under unseen working conditions,” _Mechanical Systems and Signal Processing_ , vol. 200, p. 110579, 2023.
  * [43] R. Wang, L. Qi, Y. Shi, and Y. Gao, “Better pseudo-label: Joint domain-aware label and dual-classifier for semi-supervised domain generalization,” _Pattern Recognition_ , vol. 133, p. 108987, 2023.
  * [44] J. Li, C. Shen, L. Kong, D. Wang, M. Xia, and Z. Zhu, “A new adversarial domain generalization network based on class boundary feature detection for bearing fault diagnosis,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 71, pp. 1–9, 2022.
  * [45] Z. Zheng, Y. Zhong, Y. Su, and A. Ma, “Domain adaptation via a task-specific classifier framework for remote sensing cross-scene classification,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–13, 2022.
  * [46] L. Van der Maaten and G. Hinton, “Visualizing data using t-SNE,” _Journal of machine learning research_ , vol. 9, no. 11, pp. 2579–2605, 2008.