# Cross-Domain Label Propagation for Domain Adaptation with Discriminative
Graph Self-Learning

Lei Tian*, Yongqiang Tang*, Liangchen Hu and Wensheng Zhang * indicates equal
contributions.L. Tian, and W. Zhang are with the Research Center of Precision
Sensing and Control, Institute of Automation, Chinese Academy of Sciences,
Beijing, 100190, China, and the School of Artificial Intelligence, University
of Chinese Academy of Sciences, Beijing, 101408, China.
<EMAIL_ADDRESS><EMAIL_ADDRESS>Tang is with the
Research Center of Precision Sensing and Control, Institute of Automation,
Chinese Academy of Sciences, Beijing, 100190, China.
<EMAIL_ADDRESS>Hu is with the School of Computer Science and
Engineering, Nanjing University of Science and Technology, Nanjing, 210094,
China<EMAIL_ADDRESS>
###### Abstract

Domain adaptation manages to transfer the knowledge of well-labeled source
data to unlabeled target data. Many recent efforts focus on improving the
prediction accuracy of target pseudo-labels to reduce conditional distribution
shift. In this paper, we propose a novel domain adaptation method, which
infers target pseudo-labels through cross-domain label propagation, such that
the underlying manifold structure of two domain data can be explored. Unlike
existing cross-domain label propagation methods that separate domain-invariant
feature learning, affinity matrix constructing and target labels inferring
into three independent stages, we propose to integrate them into a unified
optimization framework. In such way, these three parts can boost each other
from an iterative optimization perspective and thus more effective knowledge
transfer can be achieved. Furthermore, to construct a high-quality affinity
matrix, we propose a discriminative graph self-learning strategy, which can
not only adaptively capture the inherent similarity of the data from two
domains but also effectively exploit the discriminative information contained
in well-labeled source data and pseudo-labeled target data. An efficient
iterative optimization algorithm is designed to solve the objective function
of our proposal. Notably, the proposed method can be extended to semi-
supervised domain adaptation in a simple but effective way and the
corresponding optimization problem can be solved with the identical algorithm.
Extensive experiments on six standard datasets verify the significant
superiority of our proposal in both unsupervised and semi-supervised domain
adaptation settings.

###### Index Terms:

domain adaptation, transfer learning, label propagation, discriminative graph
learning, domain-invariant feature learning.

##  I Introduction

One common assumption of statistical learning theory is that the training data
and test data are drawn from an identical feature distribution, which may be
violated in many situations. Moreover, in practical applications, collecting
labeled training data is often expensive and time-consuming. Thus, there is a
strong demand to leverage the knowledge from a source domain with sufficient
labels to help design effective model for the unlabeled target domain data,
which follows a different feature distribution. To this end, considerable
efforts have been devoted to domain adaptation [1], and impressive progress
has been made in various tasks, e.g., object recognition [2, 3, 4], semantic
segmentation [5, 6], and sentiment analysis [7, 8].

Figure 1: Flowchart of our proposed CDGS. We integrate domain-invariant
feature learning, adaptive graph learning and cross-domain label propagation
into a unified optimization framework. Besides, in order to construct a high-
quality affinity matrix in our CDGS, we further propose a discriminative graph
self-learning strategy. To be specific, instead of predefining the similarity
metric, our proposal could adaptively assign neighbors for each sample
according to the local distance in the projected feature space. To fully
explore the discriminative information contained in well-labeled source data
and pseudo-labeled target data, we further impose block diagonal structure
constraint on source data and label smoothness constraint on two domain data.

The goal of domain adaptation is to mitigate the distribution discrepancy
between the source and target domains, such that the classifier could be
applicable across two domains. To accomplish this, numerous works [10, 11, 12,
13, 14] have devoted to learning a domain-invariant space where distribution
discrepancy can be significantly reduced via minimizing a distance metric,
e.g., the widely used maximum mean discrepancy (MMD) [15]. Along this line,
JDA [10] is a pioneering method, which aims to reduce the joint distribution
shift between two domains by simultaneously aligning the marginal distribution
and conditional distribution. Inheriting the core idea of minimizing joint
distribution discrepancy, tremendous subsequent studies following JDA [11, 12,
14], focus on further reducing the conditional distribution discrepancy by
improving the prediction accuracy of target pseudo-labels. Despite the
brilliant achievements in the literature, most of them generally overlook the
underlying data manifold structure in the process of inferring data labels on
target domain, thus making the performance of domain adaptation far from
satisfactory.

More recently, to explore the data distribution structure, several studies
[12, 16, 17, 18] innovatively propose to infer target pseudo-labels by cross-
domain label propagation [19]. Generally, these methods follow a multi-stage
paradigm in each iteration: 1) projecting the source and target data into a
domain-invariant common feature space; 2) constructing a affinity matrix by
calculating the sample similarity in the projected space with a predefined
metric, e.g., the gaussian kernel similarity [16, 17]; 3) assigning pseudo-
labels for target data via propagating the labels of source data with the
constructed affinity matrix. Although improved performance has been achieved
by these methods, they still suffer from three crucial issues:

  * •

Issue 1 The domain-invariant feature learning, affinity matrix constructing
and target labels inferring are separated into three independent stages. Thus,
the correlation among these three parts could not be fully exploited.

  * •

Issue 2 Constructing the affinity matrix with predefined metric may not
capture the inherent similarity of samples in both domains, which might
seriously affect the effectiveness of cross-domain label propagation.

  * •

Issue 3 During the construction of affinity matrix, the discriminative
information contained in the ground-truth labels of source data, as well as in
the pseudo-labels of target data is less explored.

In this study, we propose a novel domain adaptation method called Cross-domain
label propagation with Discriminative Graph Self-learning (CDGS) to remedy the
above three issues. As illustrated in Fig. 1, to tackle the first issue, we
propose to formulate the three parts of cross-domain label propagation into a
unified optimization framework, which learns domain-invariant features,
constructs affinity matrix and infers target labels simultaneously. In the
unified framework, these three parts can assist each other from an iterative
optimization perspective. For the second issue, inspired by [20, 21], we
resort to a graph self-learning strategy, which assigns adaptive neighbors for
each sample according to the local distance in the projected feature space. In
such way, the underlying data manifold structure of two domains could be
captured more effectively. To handle the third issue, for well-annotated
source data, we enforce the learned connected subgraph to have a block
diagonal structure, which means that only source samples within the same
category are allowed to be connected, while the connection weight of source
samples between different categories is forcibly set to 0. In this manner, the
discriminative information of source data can be exploited to the maximum
extent. Beyond that, inspired by [21, 22], we further impose the label
smoothness constraint during the graph self-learning, such that the weakly
supervised information contained in target pseudo-labels can be well inserted
into the adaptive graph.

It is noteworthy that, except for unsupervised domain adaptation (UDA), our
CDGS could be readily extended to the semi-supervised domain adaptation (SDA)
scenario where some labeled target samples are available. Interestingly, the
extended SDA model could be solved with the same algorithm as UDA. To sum up,
we list our contributions in fourfolds:

  1. 1.

We propose a novel cross-domain label propagation method for domain adaptation
named CDGS, which integrates domain-invariant feature learning, affinity
matrix constructing and target labels inferring into a unified optimization
framework. Through the joint optimization, the three parts could boost each
other and thus more effective knowledge transfer can be achieved.

  2. 2.

To construct a high-quality affinity matrix in CDGS, we propose a
discriminative graph self-learning strategy, which can not only adaptively
capture the local connectivity structure of data from two domains but also
effectively explore the discriminative information.

  3. 3.

An efficient optimization algorithm is designed to solve the objective
function of our CDGS. In addition to UDA, we further extend CDGS to the semi-
supervised scenario in a direct but effective way and solve the extended model
with the identical optimization algorithm.

  4. 4.

Extensive experiments on six standard datasets verify that the proposed CDGS
can consistently outperform the state-of-the-art methods in both UDA and SDA
settings.

The rest of this paper is organized as follows. Section II provides a brief
review on related domain adaptation and label propagation methods. Section III
introduces the proposed CDGS approach, the optimization algorithm, the
computational complexity and the extension to SDA. Extensive experimental
analysis is presented in Section IV. Finally, this paper is summarized in
Section V.

##  II Related Work

In this section, we review the related works in terms of domain adaptation and
label propagation, and highlight the difference between the previous works and
our proposal.

###  II-A Domain Adaptation

Domain adaptation aims to leverage the knowledge from a well-labeled source
domain to an unlabeled but related target domain. In general, domain
adaptation can be grouped as UDA and SDA. In UDA, no labeled target samples
are available. While in SDA, the target domain contains few labeled samples.

Generally, existing UDA methods can be roughly divided into three categories:
instance reweighting [23, 24], classifier adaptation [25, 26] and feature
adaptation [9, 10, 11, 12] methods. Instance reweighting methods assign source
samples with different weights to reduce the distribution shift between two
domains. Classifier adaptation methods adapt the classifier trained on source
data to target data. Feature adaptation methods seek a common feature space
[10] or latent intermediate subspaces [27] to make the two domains have
similar distributions. The proposed CDGS falls into the former line of feature
adaptation methods, thus we focus on reviewing the works related to it. Among
existing works, TCA [9] proposes to align marginal distribution between two
domains with MMD metric for the first time. Following this idea, JDA [10]
further considers the conditional distribution, such that the joint
distribution alignment can be achieved. To boost the classification
performance, several subsequent works propose to employ the discriminative
information by encouraging intra-class compactness and inter-class dispersion
[11] simultaneously or promoting domain-irrelevant class clustering [12]. To
refine the target pseudo-labels to further mitigate the conditional
distribution discrepancy, several recent works attempt to exploit the
geometric structure underlying data manifold by assigning target pseudo-labels
via cross-domain label propagation [12, 16, 17, 18] or performing label
propagation just on target domain [13, 28], and promising performance have
been achieved by them.

Our CDGS also employs cross-domain label propagation strategy to assign target
pseudo-labels. However, CDGS is significantly different from these methods.
First, CDGS integrates domain-invariant feature learning, affinity matrix
constructing and target labels inferring into a unified optimization
formulation while [12, 16, 17] separate the three parts into independent
stages, and [18] only combines the domain-invariant feature learning and
target labels inferring. Through the joint optimization in our CDGS, the three
parts could benefit from each other to yield a superior performance. Second,
CDGS presents a novel self-learning strategy to construct a discriminative
graph. Specifically, the neighbors of each sample are adaptively assigned
according to the local distance, which is calculated based on the projected
features and label information of source and target data. Besides, only source
samples within the same class are enforced to be connected to exploit the
source discriminative information. Thus, the discriminative graph can not only
faithfully capture the inherent local connectivity structure of samples but
also effectively explore the discriminative information contained in source
ground-truth labels and target pseudo-labels, which is beneficial to effective
target pseudo-labels assignment.

In the past few years, deep domain adaptation methods have attracted
considerable interest and different strategies have been proposed to align
deep features. For example, DAN [29] exploits the multikernel MMD to reduce
the marginal distribution discrepancy in the reproducing kernel Hilbert space
(RKHS). Based on this framework, JAN [30] proposes to align the joint
distribution between two domains. To capture the fine-grained information,
DSAN [31] further aligns the relevant subdomain distributions within the same
category in two domains based on a local MMD. Different from them, DANN [32]
tries to learn domain agnostic feature representations with adversarial
learning. Later, MADA [33] trains a class-wise domain discriminator for each
class. To enhance positive transfer and relieve negative transfer, Wang _et
al_. [34] introduced a self-adaptive re-weighted adversarial approach to
promote domain alignment in terms of conditional distribution. However, these
deep methods may confront the challenges of long training time and massive
resource consumption while CDGS is faster and can achieve excellent
performance by just using off-the-shelf deep features.

Many methods have also been developed for SDA [35, 36, 37]. For instance, MMDT
[35] learns the transformation matrix and classifier parameters jointly by
making samples within the same class have high similarity. CDLS [36] aligns
the conditional distribution by selecting representative landmarks. OBTL [37]
is a Bayesian transfer learning framework, which relates the two domains by
joint prior density. The proposed CDGS can be readily extended to SDA.
Specifically, we take the labeled and unlabeled target data as a whole. In
such case, we can estimate target class means more accurately, which can
result in more accurate conditional distribution alignment. Besides, as a
common strategy in semi-supervised learning, reliable connections between
labeled and unlabeled data are built by discriminative graph self-learning,
thus the knowledge from labeled samples can be propagated to the unlabeled
ones. Moreover, the resulting optimization problem has the same formula as
that of the unsupervised setting, thus they can be solved with the same
optimization algorithm.

###  II-B Label Propagation

The goal of label propagation is to propagate the label information of limited
labeled samples to amounts of unlabeled samples through graph. In the graph, a
vertex represents a sample and the weight of the edge between two vertexes
measures the similarity of the corresponding samples.

GFHF [39] and LGC [40] are two classical methods. Both of them first use the
gaussian kernel similarity to build the affinity matrix and then utilize label
propagation to predict the unknown labels via gaussian fields and harmonic
function, or the local and global consistency. However, they can not exploit
the relationship of the affinity matrix and label information of samples due
to the two separated stages. To overcome this limitation, STSSL [22]
integrates the affinity matrix constructing and the unknown labels inferring
into one unified optimization framework to exploit the correlation between
them. Following this idea, AWSSL [21] futher proposes to adaptively assign the
neighbors of each sample and effectively extract robust features by auto-
weighting feature selection.

There are several classifier adaptation methods, which borrow the advantages
of cross-domain label propagation to assign target pseudo-labels, e.g., ARTL
[25] and MEDA [26]. ARTL is also a unified framework, which learns an adaptive
classifier by jointly optimizing the source structural risk, joint
distribution alignment and manifold regulation, which is relevant to our CDGS.
However, CDGS differs from ARTL in three aspects. First, ARTL learns the
classifier with the original features while CDGS conducts subspace learning,
which is more flexible and effective. Second, CDGS learns domain-invariant
features, constructs affinity matrix and infers target labels jointly to fully
exploit the relationship among them. Third, CDGS and ARTL use different
strategies to construct the affinity matrix. Specifically, CDGS introduces a
self-learning strategy to capture the intrinsic similarity of samples as well
as effectively explore the label information of source and target data. By
contrast, ARTL just utilizes the predefined metric to calculate the similarity
for all samples.

##  III Proposed Method

In this section, the key notations throughout this paper are first introduced.
Then, we describe the details of the proposed CDGS. Next, we design an
iterative algorithm to solve the optimization problem and provide the
computational complexity analysis. Finally, we extend our method to SDA.

###  III-A Notations

In UDA, the labeled source data
$\mathcal{D}_{s}=\\{\mathbf{X}_{s},\mathbf{Y}_{s}\\}=\\{(\mathbf{x}_{si},y_{si})\\}_{i=1}^{n_{s}}$
and unlabeled target data
$\mathcal{D}_{t}=\\{\mathbf{X}_{t}\\}=\\{\mathbf{x}_{tj}\\}_{j=1}^{n_{t}}$ are
given, where $\mathbf{x}_{si}\in\mathbb{R}^{m}$ is a source sample
($y_{si}\in\mathbb{R}$ is its label), $\mathbf{x}_{tj}\in\mathbb{R}^{m}$ is a
target sample, $n_{s}$ and $n_{t}$ represent the number of source and target
samples. The entire data matrix is denoted as
$\mathbf{X}=[\mathbf{X}_{s},\mathbf{X}_{t}]=\\{\mathbf{x}_{i}\\}_{i=1}^{n}$,
where $n=n_{s}+n_{t}$. For clarity, the key notations throughout this paper
and their descriptions are summarized in Table I.

TABLE I: Frequently Used Notations and Their Descriptions Notation |  Description
---|---
$\mathbf{X}_{s}/\mathbf{X}_{t}/\mathbf{X}$ |  source/target/all data
$n_{s}/n_{t}/n$ |  number of source/target/all data matrix
$\mathbf{S}/\mathbf{L}$ |  affinity matrix/Laplacian matrix for all data
$\mathbf{P}$ |  projection matrix
$\mathbf{F}$ |  label matrix for all data
$\mathbf{H}$ |  centering matrix
$\mathbf{I}_{d}$ |  identity matrix with dimension $d$
$m/d$ |  dimension of original/projected features
$C$ |  number of shared classes
$\mathbf{1}_{p\times q}$ |  a matrix of size $p\times q$ with all elements as $1$
$\mathbf{1}_{p}$ |  a column vector of size $p$ with all elements as $1$

###  III-B Problem Formulation

In this paper, we propose the CDGS framework to address domain adaptation
problem, which integrates domain-invariant feature learning, affinity matrix
constructing and target labels inferring into a unified optimization
objective. The overall framework of our CDGS can be formulated as:

$\min_{\mathbf{P},\mathbf{S},\mathbf{F}}\Omega(\mathbf{P},\mathbf{X})+\alpha\Theta(\mathbf{P},\mathbf{S},\mathbf{X})+\beta\Psi(\mathbf{F},\mathbf{S})+\gamma\Phi(\mathbf{P})$
(1)

where $\mathbf{P}\in\mathbb{R}^{m\times d}$ denotes the projection matrix,
$\mathbf{F}\in\mathbb{R}^{n\times C}$ is the label matrix for all data and
$\mathbf{S}\in\mathbb{R}^{n\times n}$ represents the affinity matrix.
$\Omega(\mathbf{P},\mathbf{X})$ is employed to learn domain-invariant
features. $\Theta(\mathbf{P},\mathbf{S},\mathbf{X})$ is utilized to adaptively
construct the affinity matrix with the projected features.
$\Psi(\mathbf{F},\mathbf{S})$ is used to infer the target labels by cross-
domain label propagation. $\Phi(\mathbf{P})$ is the regularization term for
the projection matrix to avoid overfitting. $\alpha$, $\beta$ and $\gamma$ are
hyperparameters to balance the importance of different parts. As we can see,
by integrating the three parts into the joint optimization objective, they
could well communicate with each other to achieve more effective knowledge
transfer. Next, more details about the three parts are presented.

####  III-B1 Domain-invariant Feature Learning

When $\mathbf{X}_{s}$ and $\mathbf{X}_{t}$ are drawn from different feature
distributions, it is crucial to reduce the distribution discrepancy between
two domains, such that the classifier trained on source data can be directly
applied to target domain. To measure the distribution discrepancy, numerous
metrics have been proposed. Among them, MMD [15] is probably the most widely
used one. In the projected space, the MMD distance between two domains can be
calculated as the distance between the sample means of the source and target
data [11]. Considering the large distribution discrepancy across domains, we
minimize the marginal distribution distance and the conditional distribution
distance simultaneously, and denote them by $\mathcal{L}_{mmd}^{m}$ and
$\mathcal{L}_{mmd}^{c}$, respectively. With the MMD metric, marginal
distribution distance can be stated as:

$\displaystyle\mathcal{L}_{mmd}^{m}$
$\displaystyle=\|\frac{1}{n_{s}}\sum\limits_{i=1}^{n_{s}}\mathbf{P}^{\mathrm{T}}\mathbf{x}_{si}-\frac{1}{n_{t}}\sum\limits_{j=1}^{n_{t}}\mathbf{P}^{\mathrm{T}}\mathbf{x}_{tj}\|_{2}^{2}$
(2)
$\displaystyle=\mathrm{tr}(\mathbf{P}^{\mathrm{T}}\mathbf{X}\mathbf{M}_{0}\mathbf{X}^{\mathrm{T}}\mathbf{P})$

where $\mathrm{tr}(\cdot)$ is the trace operator, and
$\mathbf{M}_{0}\in\mathbb{R}^{n\times n}$ represents the marginal MMD matrix
calculated as:

$\displaystyle\mathbf{M}_{0}=\begin{bmatrix}\frac{1}{n_{s}^{2}}\mathbf{1}_{n_{s}\times
n_{s}}&-\frac{1}{n_{s}n_{t}}\mathbf{1}_{n_{s}\times n_{t}}\\\
-\frac{1}{n_{s}n_{t}}\mathbf{1}_{n_{t}\times
n_{s}}&\frac{1}{n_{t}^{2}}\mathbf{1}_{n_{t}\times n_{t}}\end{bmatrix}$ (3)

The calculation of conditional MMD distance requires to get the labels of
target samples, which is generally infeasible in domain adaptation task. To
remedy this issue, we employ the target pseudo-labels instead of the
unavailable true labels to compute the conditional distribution distance as
follows:

$\displaystyle\mathcal{L}_{mmd}^{c}$
$\displaystyle=\sum\limits_{c=1}^{C}\|\frac{1}{n_{s}^{c}}\sum\limits_{\mathbf{x}_{si}\in\mathbf{X}_{s}^{c}}\mathbf{P}^{\mathrm{T}}\mathbf{x}_{si}-\frac{1}{n_{t}^{c}}\sum\limits_{\mathbf{x}_{tj}\in\mathbf{X}_{t}^{c}}\mathbf{P}^{\mathrm{T}}\mathbf{x}_{tj}\|_{2}^{2}$
(4)
$\displaystyle=\mathrm{tr}(\mathbf{P}^{\mathrm{T}}\mathbf{X}(\sum\nolimits_{c=1}^{C}\mathbf{M}_{c})\mathbf{X}^{\mathrm{T}}\mathbf{P})$

where $C$ is the number of classes, $\mathbf{M}_{c}\in\mathbb{R}^{n\times n}$
is conditional MMD matrix defined as:

$(\mathbf{M}_{c})_{ij}=\begin{cases}\frac{1}{n_{s}^{c}n_{s}^{c}},&\mathrm{if}\
\mathbf{x}_{i},\mathbf{x}_{j}\in\mathbf{X}_{s}^{c};\\\
\frac{1}{n_{t}^{c}n_{t}^{c}},&\mathrm{if}\
\mathbf{x}_{i},\mathbf{x}_{j}\in\mathbf{X}_{t}^{c};\\\
\frac{-1}{n_{s}^{c}n_{t}^{c}},&\mathrm{if}\
\mathbf{x}_{i}\in\mathbf{X}_{s}^{c}\wedge\mathbf{x}_{j}\in\mathbf{X}_{t}^{c};\\\
\frac{-1}{n_{s}^{c}n_{t}^{c}},&\mathrm{if}\
\mathbf{x}_{j}\in\mathbf{X}_{s}^{c}\wedge\mathbf{x}_{i}\in\mathbf{X}_{t}^{c};\\\
0,&\text{otherwise}\end{cases}$ (5)

$\mathbf{X}_{s}^{c}$ represents all source samples in class $c$, and
$n_{s}^{c}$ is the corresponding number of samples. Similar definitions can be
applied for target samples according to the pseudo-labels. Denote
$\mathbf{M}=\sum\nolimits_{c=0}^{C}\mathbf{M}_{c}$, then we have the following
formula:

$\displaystyle\Omega(\mathbf{P},\mathbf{X})=\mathrm{tr}(\mathbf{P}^{\mathrm{T}}\mathbf{XMX}^{\mathrm{T}}\mathbf{P})$
(6)

####  III-B2 Graph Self-Learning with Source Domain Discriminative Structure
Preserving

Obviously, the quality of the affinity matrix is crucial to the performance of
cross-domain label propagation. Most of previous works [12, 16, 28] use the
same strategy to construct it, which calculates the similarity for all samples
with predefined similarity metric, e.g., the heatkenel similarity [12, 16].
This strategy may not capture the inherent similarity of samples, thus hinders
the correctness of cross-domain label propagation and results in serious
misclassification for target data. The wrong pseudo-labels will further
mislead the conditional distribution alignment in the next iteration, which
ultimately results in significant performance degradation. To tackle this
issue, inspired by several recent works [20, 21], we adopt a self-learning
strategy, which constructs the affinity matrix by assigning the adaptive
neighbors for each sample according to the local distance in the projected
space. In light of this, the optimization objective of graph self-learning can
be stated as follows:

$\displaystyle\min_{\mathbf{S}}\sum\nolimits_{i=1}^{n}((\sum\nolimits_{j=1}^{n}\|\mathbf{z}_{i}-\mathbf{z}_{j}\|_{2}^{2}S_{ij}+\lambda_{i}\|\mathbf{S}_{i,:}\|_{2}^{2})$
(7)
$\displaystyle=\min_{\mathbf{S}}\mathrm{tr}(\mathbf{P}^{\mathrm{T}}\mathbf{XL}\mathbf{X}^{\mathrm{T}}\mathbf{P})+\|\mathbf{\Lambda}\mathbf{S}\|_{F}^{2}$
$\displaystyle s.t.\quad\mathbf{S}\mathbf{1}_{n}=\mathbf{1}_{n},\ 0\leq
S_{ij}\leq 1$

where $\mathbf{z}_{i}=\mathbf{P}^{\mathrm{T}}\mathbf{x}_{i}$ is the projection
of sample $\mathbf{x}_{i}$, $\mathbf{S}_{i,:}$ represents the $i$-th row of
$\mathbf{S}$, and
$\mathbf{\Lambda}=\mathrm{diag}(\sqrt{\lambda_{1}},\sqrt{\lambda_{2}},...,\sqrt{\lambda_{n}})$.
$\mathbf{L}$ is the graph Laplacian matrix calculated as
$\mathbf{L}=\mathbf{D}-\mathbf{S}$, and $\mathbf{D}$ is a diagonal matrix with
the $i$-th element $D_{ii}=\sum\nolimits_{j=1}^{n}S_{ij}$. An $F$-norm
regulation term is imposed on the $i$-th ($i=1,2,\ldots,n$) row of
$\mathbf{S}$ and the corresponding regularization term is $\lambda_{i}$, which
can be determined automatically and will be elaborated in Section III-C. Then,
we can obtain the following formula for
$\Theta(\mathbf{P},\mathbf{S},\mathbf{X})$:

$\Theta(\mathbf{P},\mathbf{S},\mathbf{X})=\mathrm{tr}(\mathbf{P}^{\mathrm{T}}\mathbf{XL}\mathbf{X}^{\mathrm{T}}\mathbf{P})+\|\mathbf{\Lambda}\mathbf{S}\|_{F}^{2}$
(8)

In addition, several previous works [11, 12, 16] have shown that the
performance of domain adaptation can be significantly enhanced if the
discriminative information of source data is exploited. To this end, we adopt
an intuitive strategy for labeled source data that only the samples belonging
to the same category are allowed to be connected. In such case, each source
sample could be connected with two parts, one of which is the source samples
within the identical class and the other is all target samples. For
simplicity, we fix the probability as $\delta$ and $1-\delta$ for these two
parts, respectively. That is, when $i\leq n_{s}$, we have
$\sum\nolimits_{j=1}^{n_{s}}S_{ij}=\delta$ and
$\sum\nolimits_{j=n_{s}+1}^{n_{s}+n_{t}}S_{ij}=1-\delta$, where
$\delta\in[0,1]$ is a hyperparameter to control the partition of probability.
In this way, the learned adaptive discriminative graph owns the following
structure:

$\mathbf{S}=\left[\begin{array}[]{cc}\begin{array}[]{c|c}\overbrace{\begin{array}[]{cccc}\mathbf{S}_{ss}^{(1)}&\mathbf{0}&\cdots&\mathbf{0}\\\
\mathbf{0}&\mathbf{S}_{ss}^{(2)}&\cdots&\mathbf{0}\\\
\vdots&\vdots&\ddots&\vdots\\\
\mathbf{0}&\mathbf{0}&\cdots&\mathbf{S}_{ss}^{(C)}\end{array}}^{\delta}&\overbrace{\begin{array}[]{ccc}&&\\\
&\\\ &\mathbf{S}_{st}&\\\ &&\\\ &&\end{array}}^{1-\delta}\end{array}\\\
\hline\cr\begin{array}[]{ccccccc}&&&&&&\\\ &&&&\mathbf{S}_{ta}&&\\\ &&&&&&\\\
\end{array}\end{array}\right]$ (9)

where $\mathbf{S}_{ss}^{(c)}$ represents the affinity matrix of all source
samples in class $c$, $\mathbf{S}_{ss}$ is the connected subgraph with block
diagonal structure for all source samples, $\mathbf{S}_{st}$ is the connected
subgraph between source and target samples, and $\mathbf{S}_{ta}$ denotes the
connected subgraph between target samples and all samples. Then, considering
the above constraints, the objective of graph self-learning with source domain
discriminative structure preserving term
$\Theta(\mathbf{P},\mathbf{S},\mathbf{X})$ can be formulated as:

$\displaystyle\min_{\mathbf{S}}\sum\nolimits_{i=1}^{n}((\sum\nolimits_{j=1}^{n}\|\mathbf{z}_{i}-\mathbf{z}_{j}\|_{2}^{2}S_{ij}+\lambda_{i}\|\mathbf{S}_{i,:}\|_{2}^{2})$
(10)
$\displaystyle=\min_{\mathbf{S}}\mathrm{tr}(\mathbf{P}^{\mathrm{T}}\mathbf{XL}\mathbf{X}^{\mathrm{T}}\mathbf{P})+\|\mathbf{\Lambda}\mathbf{S}\|_{F}^{2}$
$\displaystyle s.t.\ \mathbf{S}\mathbf{1}_{n}=\mathbf{1}_{n},\ 0\leq
S_{ij}\leq 1,\ \sum\limits_{j=1}^{n_{s}}S_{ij}=\delta,\ i\leq n_{s},$
$\displaystyle\qquad\ S_{ij}=0,\ i,j\leq n_{s}\wedge y_{si}\neq y_{sj}$

####  III-B3 Cross-Domain Label Propagation with Self-Learned Graph

The main assumption of label propagation [19] is that the adjacent points tend
to similar labels. Thus, assigning the target pseudo-labels by cross-domain
label propagation could effectively exploit the geometric structure underlying
the source and target data manifolds, which can improve the accuracy of target
pseudo-label prediction and further benefit the subsequent domain-invariant
feature learning. To achieve this goal, numerous current label propagation
algorithms can be our candidates. For the sake of simplicity, we employ the
classical GFHF algorithm [39]. Suppose the source label matrix is
$\mathbf{F}_{s}\in\mathbb{R}^{n_{s}\times C}$, where the ($i$, $j$)-th element
is 1 if $y_{si}=j$, and 0 otherwise. Denote
$\mathbf{F}=[\mathbf{F}_{l};\mathbf{F}_{t}]$, where
$\mathbf{F}_{l}=\mathbf{F}_{s}$, and $\mathbf{F}_{t}\in\mathbb{R}^{n_{t}\times
C}$ is the inferred target label matrix. Then, given the self-learned graph
$\mathbf{S}$, the optimization problem of the GFHF algorithm can be formulated
as:

$\min_{\mathbf{F}}\sum\limits_{i,j=1}^{n}\|\mathbf{F}_{i}-\mathbf{F}_{j}\|_{2}^{2}S_{ij}=\min_{\mathbf{F}}\mathrm{tr}(\mathbf{F}^{\mathrm{T}}\mathbf{L}\mathbf{F})\
s.t.\ \mathbf{F}_{l}=\mathbf{F}_{s}$ (11)

Thus, we can obtain the following formula for $\Psi(\mathbf{F},\mathbf{S})$:

$\Psi(\mathbf{F},\mathbf{S})=\mathrm{tr}(\mathbf{F}^{\mathrm{T}}\mathbf{L}\mathbf{F})$
(12)

####  III-B4 The Final Form for Discriminative Graph Self-Learning

Actually, the pseudo-labels of target data has been proven to be able to
provide useful “weakly” supervised information to yield a better connectivity
graph [21, 22]. In our CDGS, we further impose label smoothness constraint
(i.e., Eq.(12)) during graph self-learning to effectively explore the target
pseudo-label information. Thus, our final optimization objective of
discriminative graph self-learning can be formulated as:

$\displaystyle\min_{\mathbf{S}}\sum\nolimits_{i=1}^{n}((\sum\nolimits_{j=1}^{n}\|\mathbf{z}_{i}-\mathbf{z}_{j}\|_{2}^{2}S_{ij}+\beta\|\mathbf{F}_{i}-\mathbf{F}_{j}\|_{2}^{2}S_{ij})$
(13) $\displaystyle\qquad\qquad+\lambda_{i}\|\mathbf{S}_{i,:}\|_{2}^{2})$
$\displaystyle\quad=\min_{\mathbf{S}}\mathrm{tr}(\mathbf{P}^{\mathrm{T}}\mathbf{XL}\mathbf{X}^{\mathrm{T}}\mathbf{P})+\|\mathbf{\Lambda}\mathbf{S}\|_{F}^{2}+\beta\mathrm{tr}(\mathbf{F}^{\mathrm{T}}\mathbf{L}\mathbf{F})$
$\displaystyle s.t.\ \mathbf{S}\mathbf{1}_{n}=\mathbf{1}_{n},\ 0\leq
S_{ij}\leq 1,\ \sum\limits_{j=1}^{n_{s}}S_{ij}=\delta,\ i\leq n_{s},$
$\displaystyle\qquad\ S_{ij}=0,\ i,j\leq n_{s}\wedge y_{si}\neq y_{sj}$

To avoid overfitting and improve the generalization capacity, we further
impose an $F$-norm regularization term on $\mathbf{P}$, then we have:

$\Phi(\mathbf{P})=\|\mathbf{P}\|_{F}^{2}$ (14)

Finally, by combining Eq. (6), Eq. (11), Eq. (13) and Eq. (14), we obtain the
final formulation of our CDGS:

$\displaystyle\min_{\mathbf{P},\mathbf{S},\mathbf{F}}\mathrm{tr}(\mathbf{P}^{\mathrm{T}}\mathbf{XMX}^{\mathrm{T}}\mathbf{P})+\alpha(\mathrm{tr}(\mathbf{P}^{\mathrm{T}}\mathbf{X}\mathbf{L}\mathbf{X}^{\mathrm{T}}\mathbf{P})$
(15) $\displaystyle\qquad\ +\|\mathbf{\Lambda
S}\|_{F}^{2})+\beta\mathrm{tr}(\mathbf{F}^{\mathrm{T}}\mathbf{L}\mathbf{F})+\gamma\|\mathbf{P}\|_{F}^{2}$
$\displaystyle\ \ s.t.\
\mathbf{P}^{\mathrm{T}}\mathbf{XHX}^{\mathrm{T}}\mathbf{P}=\mathbf{I}_{d},\
\mathbf{S}\mathbf{1}_{n}=\mathbf{1}_{n},0\leq S_{ij}\leq 1,$
$\displaystyle\qquad\ \mathbf{F}_{l}=\mathbf{F}_{s},\
\sum\nolimits_{j=1}^{n_{s}}S_{ij}=\delta,\ i\leq n_{s},$ $\displaystyle\qquad\
S_{ij}=0,\ i,j\leq n_{s}\wedge y_{si}\neq y_{sj}$

where $\mathbf{H}$ is the centering matrix defined as
$\mathbf{H}=\mathbf{I}_{n}-\frac{1}{n}\mathbf{1}_{n\times n}$. The first
constraint is to maximize the variance of all data [10] in the projected
space, which is inspired by the principal component analysis. Similar to [11],
$\mathbf{M}$ and $\mathbf{L}$ can be normalized into the same scale. Thus, we
set $\alpha=1.0$ for all cases.

###  III-C Optimization Procedure

In problem (15), we need to optimize three variables $\mathbf{P}$,
$\mathbf{S}$, and $\mathbf{F}$. As it is not jointly convex with the three
variables, we update each variable alternatively with the others fixed. To be
specific, we solve each subproblem as follows.

1\. $\mathbf{P}$-Subproblem: When we fix $\mathbf{S}$ and $\mathbf{F}$, the
optimization problem (15) becomes:

$\displaystyle\min_{\mathbf{P}}\mathrm{tr}(\mathbf{P}^{\mathrm{T}}(\mathbf{X}\mathbf{M}\mathbf{X}^{\mathrm{T}}+\alpha\mathbf{X}\mathbf{L}\mathbf{X}^{\mathrm{T}}+\gamma\mathbf{I}_{m})\mathbf{P})$
(16) $\displaystyle\qquad s.t.\
\mathbf{P}^{\mathrm{T}}\mathbf{XHX}^{\mathrm{T}}=\mathbf{I}_{d}$

We employ the Lagrange techniques to solve it. The corresponding Lagrangian
function can be formulated as:

$\displaystyle L(\mathbf{P},\mathbf{\Theta})=$ $\displaystyle\
\mathrm{tr}(\mathbf{P}^{\mathrm{T}}(\mathbf{X}\mathbf{M}\mathbf{X}^{\mathrm{T}}+\alpha\mathbf{X}\mathbf{L}\mathbf{X}^{\mathrm{T}}+\gamma\mathbf{I}_{m})\mathbf{P})$
(17)
$\displaystyle+\mathrm{tr}((\mathbf{I}_{d}-\mathbf{P}^{\mathrm{T}}\mathbf{XHX}^{\mathrm{T}}\mathbf{P})\mathbf{\Pi})$

where
$\mathbf{\Pi}=\mathrm{diag}(\pi_{1},\pi_{2},...,\pi_{d})\in\mathbb{R}^{d\times
d}$ is a diagonal matrix and each element is a Lagrange Multiplier. By setting
the gradient of (17) with respect to $\mathbf{P}$ to zero, we obtain:

$(\mathbf{X}\mathbf{M}\mathbf{X}^{\mathrm{T}}+\alpha\mathbf{X}\mathbf{L}\mathbf{X}^{\mathrm{T}}+\gamma\mathbf{I}_{m})\mathbf{P}=\mathbf{XHX}^{\mathrm{T}}\mathbf{P}\mathbf{\Theta}$
(18)

Then the optimal solution can be obtained by computing the eigenvectors of
(18) regarding to the $d$-smallest eigenvalues.

2\. $\mathbf{S}$-Subproblem: When $\mathbf{P}$ and $\mathbf{F}$ are fixed, the
optimization problem (15) with regard to $\mathbf{S}$ is equal to problem
(13). Actually, problem (13) can be divided into $n$ subproblems and each of
them is formulated as:

$\displaystyle\min_{\mathbf{S}_{i,:}}\sum\nolimits_{j=1}^{n}(\|\mathbf{z}_{i}-\mathbf{z}_{j}\|_{2}^{2}S_{ij}+\beta\|\mathbf{F}_{i}-\mathbf{F}_{j}\|_{2}^{2}S_{ij})$
(19) $\displaystyle\qquad+\lambda_{i}\|\mathbf{S}_{i,:}\|_{2}^{2}$
$\displaystyle s.t.\ \mathbf{S}_{i,:}\mathbf{1}_{n}=1,\ 0\leq S_{ij}\leq 1,\
\sum\nolimits_{j=1}^{n_{s}}S_{ij}=\delta,\ i\leq n_{s}$ $\displaystyle\quad\
S_{ij}=0,\ i,j\leq n_{s}\wedge y_{si}\neq y_{sj}$

Case 1: First of all, we show how to obtain the optimal solution when $i\
\textgreater\ n_{s}$. We define
$A_{ij}=\|\mathbf{z}_{i}-\mathbf{z}_{j}\|_{2}^{2}+\beta\|\mathbf{F}_{i}-\mathbf{F}_{j}\|_{2}^{2}$,
then the above problem can be reformulated as:

$\min_{\mathbf{S}_{i,:}\mathbf{1}_{n}=1,0\leq S_{ij}\leq
1}\|\mathbf{S}_{i,:}+\frac{\mathbf{A}_{i,:}}{2\lambda_{i}}\|_{2}^{2}$ (20)

Input: Source data $\\{\mathbf{X}_{s},\mathbf{Y}_{s}\\}$; Target data
$\\{\mathbf{X}_{t}\\}$; Initial affinity matrix $\mathbf{S}$; Hyper-parameters
$\alpha=1.0$, $k=20$, $\delta=0.8$, $\beta$, $\gamma$, $d$; Maximum iteration
$T$ = 10.

Output: Target pseudo-labels $\mathbf{\widehat{Y}}$.

1 $t$ = 1;

2 while  _ not converge and $t$ $\leq T$_ do

3 // Projection matrix $\mathbf{P}$

4 Update $\mathbf{P}$ by solving (18);

5 // Affinity matrix $\mathbf{S}$

6 Update each row of $\mathbf{S}$ by (24), (26) and (28);

7 // Target label matrix $\mathbf{F}_{t}$

8 Update $\mathbf{F}_{t}$ by (29);

9 //Target pseudo-labels $\mathbf{\widehat{Y}}$

10 Update the target pseudo-labels $\mathbf{\widehat{Y}}$ by (30);

11 $t$ = $t$ \+ 1;

12 end while

Return Target pseudo-labels $\mathbf{\widehat{Y}}$.

Algorithm 1 CDGS Algorithm

The corresponding Lagrangian function is

$\min_{\mathbf{S}_{i,:}}\|\mathbf{S}_{i,:}+\frac{\mathbf{A}_{i,:}}{2\lambda_{i}}\|_{2}^{2}-\mu(\mathbf{S}_{i,:}\mathbf{1}_{n}-1)-\mathbf{S}_{i,:}\bm{\eta}^{\mathrm{T}}$
(21)

where $\mu$ and $\bm{\eta}$ are the Lagrangian multipliers. To utilize the
local structure of data and relieve computation burden, we learn a sparse
$\mathbf{S}_{i,:}$, i.e., each sample is only locally connected with its
$k$-nearest neighbors. Based on the KKT condition, problem (21) has a closed-
form solution as follows:

$S_{ij}=\mathrm{max}(z-\frac{A_{ij}}{2\lambda_{i}},0)$ (22)

where
$z=\frac{1}{k}+\frac{1}{2k\lambda_{i}}\sum\nolimits_{j=1}^{k}\tilde{A}_{ij}$
and $\tilde{A}_{ij}$ is the entry of matrix $\tilde{\mathbf{A}}$, which is
obtained by sorting the elements of each row of $\mathbf{A}$ from small to
large. To ensure that each $\mathbf{S}_{i,:}$ has exactly $k$ nonzero
elements, we could set $z-\tilde{A}_{i,k+1}/(2\lambda_{i})=0$, then we have:

$\lambda_{i}=\frac{1}{2}(k\tilde{A}_{i,k+1}-\sum\nolimits_{j=1}^{k}\tilde{A}_{ij})$
(23)

Submitting Eq. (23) into Eq. (22), we can obtain:

$S_{ij}=\mathrm{max}(\frac{\tilde{A}_{i,k+1}-A_{ij}}{k\tilde{A}_{i,k+1}-\sum\nolimits_{j=1}^{k}\tilde{A}_{ij}},0)$
(24)

Case 2: When $i,j\leq n_{s}$, Eq.(19) can be reformulated as:

$\displaystyle\min_{\mathbf{S}_{i,:}}\sum\nolimits_{j=1}^{n_{s}}\|\mathbf{z}_{i}-\mathbf{z}_{j}\|_{2}^{2}S_{ij}+\beta\|\mathbf{F}_{i}-\mathbf{F}_{j}\|_{2}^{2}S_{ij}+\lambda_{i}S_{ij}^{2}$
(25) $\displaystyle s.t.\sum\nolimits_{j=1}^{n_{s}}S_{ij}=\delta,\ 0\leq
S_{ij}\leq 1,\ S_{ij}=0,\ y_{si}\neq y_{sj}$

To satisfy the last constraint, we could set $A_{ij}=+\infty$ if $y_{si}\neq
y_{sj}$. Similar to problem (20), we can obtain the closed-form solution of
problem (25):

$S_{ij}=\delta\mathrm{max}(\frac{\tilde{A}_{i,k_{1}+1}-A_{ij}}{k_{1}\tilde{A}_{i,k_{1}+1}-\sum\nolimits_{j=1}^{k_{1}}\tilde{A}_{ij}},0)$
(26)

where $k_{1}=\mathrm{min}(k,n_{s}^{y_{si}})$ as in practice, some classes may
have very small-size samples.

Case 3: When $i\leq n_{s},j>n_{s}$, problem (19) can be rewritten as:

$\displaystyle\min_{\mathbf{S}_{i,:}}\sum\nolimits_{j=n_{s}+1}^{n}\|\mathbf{z}_{i}-\mathbf{z}_{j}\|_{2}^{2}S_{ij}+\beta\|\mathbf{F}_{i}-\mathbf{F}_{j}\|_{2}^{2}S_{ij}$
(27) $\displaystyle\qquad+\lambda_{i}S_{ij}^{2}$ $\displaystyle
s.t.\sum\nolimits_{j=n_{s}+1}^{n}S_{ij}=1-\delta,\ 0\leq S_{ij}\leq 1$

Similarly, the closed-form solution of problem (27) is:

$S_{ij}=(1-\delta)\mathrm{max}(\frac{\tilde{A}_{i,k+1}-A_{ij}}{k\tilde{A}_{i,k+1}-\sum\nolimits_{j=1}^{k}\tilde{A}_{ij}},0)$
(28)

3\. $\mathbf{F}$-Subproblem: With fixed $\mathbf{P}$ and $\mathbf{S}$, the
optimization problem (15) with respect to $\mathbf{F}$ is equal to solve
problem (11). According to [39], we only need to update $\mathbf{F}_{t}$.
Split $\mathbf{L}$ into four blocks:
$\mathbf{L}=\begin{bmatrix}\mathbf{L}_{ss}&\mathbf{L}_{st}\\\
\mathbf{L}_{ts}&\mathbf{L}_{tt}\end{bmatrix}$, where
$\mathbf{L}_{ss}\in\mathbb{R}^{n_{s}\times n_{s}}$,
$\mathbf{L}_{st}\in\mathbb{R}^{n_{s}\times n_{t}}$,
$\mathbf{L}_{ts}\in\mathbb{R}^{n_{t}\times n_{s}}$ and
$\mathbf{L}_{tt}\in\mathbb{R}^{n_{t}\times n_{t}}$, and then, the optimal
solution of problem (11) is:

$\mathbf{F}_{t}=-\mathbf{L}_{tt}^{-1}\mathbf{L}_{ts}\mathbf{F}_{s}$ (29)

Eventually, the target pseudo-labels can be obtained based on the following
decision function:

$\widehat{y}_{ti}=\mathrm{argmax}_{j}\ (\mathbf{F}_{t})_{ij}$ (30)

The affinity matrix $\mathbf{S}$ is initialized according to (26) in the
original feature space. We summarize the detailed optimization steps of the
proposed CDGS in Algorithm 1.

###  III-D Computational Complexity Analysis

To find the optimal solutions for the optimization Algorithm 1, we need to
solve three subproblems. The complexity of each subproblem in each iteration
is induced as follows: First, constructing and solving the eigen-decomposition
problem (18) for $\mathbf{P}$-subproblem costs $\mathcal{O}(n^{2}m+dm^{2})$;
Then, updating the affinity matrix $\mathbf{S}$ needs a time cost of
$\mathcal{O}(n^{2}\mathrm{log}(n))$; Finally, the complexity of obtaining the
target estimated label matrix $\mathbf{F}_{t}$ and the pseudo-labels
$\mathbf{\widehat{Y}}$ is $\mathcal{O}(n_{t}^{3})$. Thus, the overall
computational complexity of our proposal is
$\mathcal{O}(Tn^{2}m+Tdm^{2}+Tn^{2}\mathrm{log}(n)+Tn_{t}^{3})$, where $T$ is
the number of iterations.

###  III-E Extension to Semi-supervised Domain Adaptation

We denote the target data as $\mathbf{X}_{t}=[\mathbf{X}_{l},\mathbf{X}_{u}]$,
where $\mathbf{X}_{l}=\\{\mathbf{x}_{li}\\}_{i=1}^{n_{l}}$ is the labeled data
and $\mathbf{X}_{u}=\\{\mathbf{x}_{uj}\\}_{j=1}^{n_{u}}$ is the unlabeled
data. Then, by submitting $\mathbf{X}_{s}$ and $\mathbf{X}_{t}$ into Eq. (15),
the semi-supervised extension for our CDGS can be stated as:

$\displaystyle\min_{\mathbf{P},\mathbf{S},\mathbf{F}}\mathrm{tr}(\mathbf{P}^{\mathrm{T}}\mathbf{XMX}^{\mathrm{T}}\mathbf{P})+\alpha(\mathrm{tr}(\mathbf{P}^{\mathrm{T}}\mathbf{X}\mathbf{L}\mathbf{X}^{\mathrm{T}}\mathbf{P})$
(31) $\displaystyle\qquad\ +\|\mathbf{\Lambda
S}\|_{F}^{2})+\beta\mathrm{tr}(\mathbf{F}^{\mathrm{T}}\mathbf{L}\mathbf{F})+\gamma\|\mathbf{P}\|_{F}^{2}$
$\displaystyle\ \ s.t.\
\mathbf{P}^{\mathrm{T}}\mathbf{XHX}^{\mathrm{T}}\mathbf{P}=\mathbf{I}_{d},\
\mathbf{S}\mathbf{1}_{n}=\mathbf{1}_{n},0\leq S_{ij}\leq 1,$
$\displaystyle\qquad\ \mathbf{F}_{l}=\mathbf{F}_{s},\
\sum\nolimits_{j=1}^{n_{s}}S_{ij}=\delta,\ i\leq n_{s},$ $\displaystyle\qquad\
S_{ij}=0,\ i,j\leq n_{s}\wedge y_{si}\neq y_{sj}$

where $n=n_{s}+n_{l}+n_{u}$. Obviously, Eq. (31) owns the same formula with
Eq. (15), thus they can be solved with the identical algorithm.

Actually, our semi-supervised extension can be effective for the following two
reasons: 1) The estimation of target class means is more accurate when some
labeled target samples are available, which can promote to perform conditional
distribution alignment more accurately; 2) Through Eq. (7), reliable
connections between the labeled and unlabeled data are built, which can
transfer the knowledge of labeled samples to the unlabeled ones via cross-
domain label propagation.

##  IV Experiments

In this section, we first describe the six benchmark datasets. Then, the
details of experimental setup are shown. Next, we present the evaluation
results of UDA, ablation study, parameter sensitivity and convergence
analysis. Finally, the results for SDA are reported. The source code of this
paper is available at https://drive.google.com/drive/folders/19Fqxxuf9MTcd-1em
XstZE01G60JUyAst?usp=sharing.

###  IV-A Datasets and Descriptions

We adopt six benchmark datasets in our experiments, including Office31,
Office-Caltech10, ImageNet-VOC2007, Office-Home, MNIST-USPS and PIE, which are
widely used cross-domain object, digit and face datasets. Overall descriptions
about these datasets are summarized in Table II. We will introduce more
details for each dataset as follows.

_Office31_ [42] contains 4,110 images with 31 categories collected from three
domains: Amazon (A), DSLR (D) and Webcam (W). Amazon images are downloaded
from the online merchants. DSLR images are captured by a digital SLR camera
while Webcam images are recorded by a web camera. Following [43], we utilize
the AlexNet-FC7 features111
https://github.com/VisionLearningGroup/CORAL/tree/master/dataset fine-tuned on
the source domain.

_Office-Caltech10_ [27] includes 2,533 images in 10 shared categories from the
Office31 dataset and the Caltech256 (C) dataset, which is a widely used
dataset for object recognition. Following [27], we exploit the SURF
features222http://boqinggong.info/assets/GFK.zip. Besides, the VGG-FC6,7
features333<EMAIL_ADDRESS>provided by [44] are
used.

TABLE II: Statistics of the Six Benchmark Datasets

Dataset | Subsets (Abbr.) | Samples | Feature (Size) | Classes
---|---|---|---|---
Office31 | Amazon (A) | 2,817 | Alexnet-FC7 (4,096) | 31
DSLR (D) | 498
Webcam (W) | 795
Office-Caltech10 | Amazon (A) | 958 |  SURF (800) VGG-FC6 (4,096) VGG-FC7 (4,096) | 10
Caltech (C) | 1,123
DSLR (D) | 157
Webcam (W) | 295
PIE | C05 | 3,332 | Pixel (1024) | 65
C07 | 1,629
C09 | 1,632
C27 | 3,329
C29 | 1,632
Office-Home | Art (Ar) | 2,421 | Resnet50 (2,048) | 68
Clipart (Cl) | 4,379
Product (Pr) | 4,428
Real-World (Re) | 4,357
MNIST-USPS | MNIST (M) | 2,000 | Pixel (256) | 10
USPS (U) | 1,800
ImageNet-VOC2007 | ImageNet (I) | 7,341 | DeCAF6 (4,096) | 5
VOC2007 (V) | 3,376

_PIE_ [45] involves 41,638 facial images of 68 people with different poses,
illuminations, and expression changes. Following [10], we focus on five poses:
C05 (left), C07 (upward), C09 (downward), C27 (frontal) and C29 (right). All
images were converted to grayscale and cropped to the size 32 $\times$ 32\. We
adopt the pixel features444
https://github.com/jindongwang/transferlearning/tree/master/data.

TABLE III: Recognition Accuracies (%) on Office31 Dataset

Task | 1-NN | SVM | JDA | DICD | PACET | MCS | DTLC | ARTL | MEDA | DGA-DA | DICElp | CDGS
---|---|---|---|---|---|---|---|---|---|---|---|---
A$\rightarrow$D | 59.4 | 59.2 | 65.7 | 66.7 | 69.1 | 71.9 | 66.1 | 64.7 | 69.5 | 64.5 | 67.7 | 73.5
A$\rightarrow$W | 57.5 | 57.9 | 69.1 | 70.7 | 71.7 | 75.1 | 67.5 | 71.7 | 69.9 | 65.0 | 70.7 | 79.5
D$\rightarrow$A | 47.2 | 48.8 | 57.4 | 57.7 | 62.3 | 58.8 | 58.9 | 59.5 | 58.0 | 55.0 | 56.5 | 61.8
D$\rightarrow$W | 96.1 | 95.2 | 98.0 | 97.0 | 97.4 | 96.7 | 98.0 | 96.0 | 94.0 | 97.2 | 97.2 | 97.2
W$\rightarrow$A | 44.8 | 46.5 | 54.1 | 58.1 | 59.2 | 57.2 | 55.1 | 58.5 | 56.0 | 53.8 | 57.7 | 61.2
W$\rightarrow$D | 99.0 | 98.8 | 99.6 | 99.8 | 100.0 | 99.4 | 99.6 | 99.4 | 96.8 | 99.8 | 100.0 | 100.0
Average | 67.3 | 67.7 | 74.0 | 75.0 | 76.6 | 76.5 | 74.2 | 74.9 | 74.0 | 72.5 | 75.0 | 78.9

_MNIST-USPS_ is made up of two handwritten digit image datasets: MNIST (M) and
USPS (U). Following [10], we randomly choose 2,000 images in MNIST and 1,800
images in USPS and utilize the pixel features4.

_ImageNet-VOC2007_ consists of two large image recognition datasets, ImageNet
(I) and VOC2007 (V). Following [26], we extract all images from five common
classes of the two datasets, i.e., bird, cat, chair, dog and person. The
DeCAF6 feature4 is employed.

_Office-Home_ [46] includes 15,585 object images in 65 categories from four
domains: Art (artistic depictions of objects, Ar), Clipart (clipart images,
Cl), Product (object images without background, Pr) and Real-World (images
captured by a regular camera, Re). We employ the Resnet50 features extracted
by a Resnet50 model [47] pretrained on ImageNet.

For simplicity, in our experiments, each cross-domain task is denoted by
S$\rightarrow$ T, where S represents the source domain and T is the target
domain.

###  IV-B Experimental Setup

####  IV-B1 Comparison Methods

For UDA, we compare the performance of our CDGS with massive methods, which
can be classified into two categories: _shallow methods_ : 1-NN,
SVM555https://www.csie.ntu.edu.tw/~cjlin/liblinear/, JDA [10], DICD [11],
PACET [14], MCS [48], DTLC [13], ARTL [25], MEDA [26], DGA-DA [16] and DICElp
[12], _deep methods_ : the method of [34], DRCN [49], DSAN [31], the method of
[50], and GSP [51]. For SDA, the competitors include MMDT [35], CDLS [36], ILS
[44], TFMKL-S [52] and OBTL [37].

####  IV-B2 Training Protocol

We exploit all source data for training, known as full protocol, on all
datasets in Table II. Besides, regarding the Office-Caltech10 dataset, two
kinds of sampling protocols are also adopted, where only few labeled source
samples per category are employed for training. For the first sampling
protocol, similar to [12], we use the SURF features and 20 instances per class
are randomly selected for domain A while 8 instances per class for other
domains as sources. For the second sampling protocol, following [48], VGG-FC6
features are utilized and 8 samples per category are selected for domain D
while 20 samples per category for the others.

####  IV-B3 Parameter Setting

In UDA and SDA, sufficient labeled target samples are unavailable, thus we
cannot perform a standard cross-validation procedure to decide the optimal
parameters. Following [11], we report the best results by grid-searching the
hyper-parameter space. For all competitors, we run the public codes provided
by the authors using the default parameters or following the given procedure
to tune parameters. For all approaches requiring a subspace dimension, the
optimal value is searched in $d\in\\{1C,2C,3C,4C,5C,6C\\}$, where $C$ is the
number of classes for the corresponding dataset. The regulation parameter for
projection matrix is searched in
$\gamma\in\\{0.005,0.01,0.05,0.1,0.5,1.0,5.0,10.0\\}$. For the other
parameters in our CDGS, we fix $\alpha=1.0$, $k=20$, $\delta=0.8$, $T=10$ and
set $\beta=0.5$ for Office-Home and Office-Caltech10 datasets, $\beta=0.01$
for PIE dataset and $\beta=0.1$ for other datasets. We also provide the
optimal parameters for UDA setting: Office31 ($d=124$, $\gamma=0.01$), Office-
Caltech10 ($d=30$, $\gamma=0.5$ for SURF, $d=30$, $\gamma=0.1$ for SURF split,
$d=40$, $\gamma=0.1$ for VGG-FC6,7 split), MNIST-USPS ($d=40$, $\gamma=0.5$),
ImageNet-VOC2007 ($d=30$, $\gamma=0.01$), PIE ($d=340$, $\gamma=0.005$) and
Office-Home ($d=130$, $\gamma=0.005$).

TABLE IV: Recognition Accuracies (%) on Office-Caltech10 Dataset with SURF
Features

Task | 1-NN | SVM | JDA | DICD | PACET | MCS | DTLC | ARTL | MEDA | DGA-DA | DICElp | CDGS
---|---|---|---|---|---|---|---|---|---|---|---|---
A$\rightarrow$C | 26.0 | 35.6 | 39.4 | 42.4 | 42.7 | 40.8 | 46.6 | 41.3 | 43.9 | 41.3 | 44.1 | 42.7
A$\rightarrow$D | 25.5 | 36.3 | 39.5 | 38.9 | 50.3 | 45.2 | 45.4 | 38.9 | 45.9 | 38.2 | 49.0 | 51.0
A$\rightarrow$W | 29.8 | 31.9 | 38.0 | 45.1 | 53.2 | 50.8 | 48.1 | 39.0 | 53.2 | 38.3 | 52.9 | 52.5
C$\rightarrow$A | 23.7 | 42.9 | 44.8 | 47.3 | 52.2 | 58.8 | 50.3 | 54.9 | 56.5 | 52.1 | 53.7 | 56.8
C$\rightarrow$D | 25.5 | 33.8 | 45.2 | 49.7 | 52.2 | 45.2 | 52.4 | 44.6 | 50.3 | 45.9 | 51.6 | 59.2
C$\rightarrow$W | 25.8 | 34.6 | 41.7 | 46.4 | 51.5 | 51.9 | 54.4 | 50.5 | 53.9 | 47.1 | 53.9 | 55.9
D$\rightarrow$A | 28.5 | 34.3 | 33.1 | 34.5 | 40.8 | 37.1 | 36.2 | 38.1 | 41.2 | 33.6 | 41.2 | 45.2
D$\rightarrow$C | 26.3 | 32.1 | 31.5 | 34.6 | 34.5 | 31.3 | 32.1 | 31.0 | 34.9 | 33.7 | 34.5 | 39.4
D$\rightarrow$W | 63.4 | 78.0 | 89.5 | 91.2 | 91.5 | 86.1 | 92.9 | 83.4 | 87.5 | 93.2 | 84.1 | 92.5
W$\rightarrow$A | 23.0 | 37.5 | 32.8 | 34.1 | 40.8 | 37.8 | 33.5 | 40.1 | 42.7 | 41.8 | 33.1 | 47.4
W$\rightarrow$C | 19.9 | 33.9 | 31.2 | 33.6 | 39.0 | 29.8 | 33.8 | 34.8 | 34.0 | 33.3 | 37.8 | 38.2
W$\rightarrow$D | 59.2 | 80.9 | 89.2 | 89.8 | 92.4 | 83.4 | 87.3 | 78.3 | 88.5 | 89.8 | 87.3 | 94.3
Average | 31.4 | 42.6 | 46.3 | 49.0 | 53.6 | 50.1 | 51.1 | 47.9 | 52.7 | 49.0 | 51.9 | 56.3

TABLE V: Recognition Accuracies (%) on Office-Caltech10 Dataset with SURF and
VGG-FC6,7 Features under Different Splitting Protocols

Feature | SURF | VGG-FC6 | VGG-FC7
---|---|---|---
Task | MCS | ARTL | MEDA | DICElp | CDGS | MCS | ARTL | MEDA | DICElp | CDGS | MCS | ARTL | MEDA | DICElp | CDGS
A$\rightarrow$C | 40.4 | 36.4 | 38.1 | 39.6 | 40.8 | 87.1 | 84.6 | 85.2 | 83.9 | 85.1 | 86.3 | 84.1 | 84.4 | 83.6 | 84.8
A$\rightarrow$D | 43.7 | 38.2 | 39.0 | 39.7 | 44.3 | 74.8 | 75.0 | 71.7 | 66.4 | 82.6 | 72.8 | 74.8 | 70.8 | 64.9 | 81.0
A$\rightarrow$W | 48.3 | 38.3 | 45.8 | 42.9 | 49.8 | 84.8 | 90.1 | 88.5 | 77.2 | 97.9 | 86.6 | 87.8 | 88.2 | 79.4 | 94.5
C$\rightarrow$A | 43.2 | 41.5 | 44.3 | 44.8 | 44.5 | 92.3 | 89.0 | 90.8 | 91.6 | 88.8 | 92.8 | 91.1 | 91.6 | 91.6 | 88.9
C$\rightarrow$D | 45.3 | 39.0 | 39.9 | 39.6 | 46.7 | 77.3 | 79.9 | 78.1 | 68.2 | 84.5 | 73.0 | 79.0 | 75.7 | 65.4 | 83.5
C$\rightarrow$W | 43.8 | 35.8 | 40.3 | 40.5 | 44.7 | 87.1 | 89.9 | 90.3 | 83.8 | 93.9 | 89.3 | 90.6 | 90.2 | 84.2 | 93.0
D$\rightarrow$A | 37.7 | 37.0 | 40.3 | 40.9 | 45.0 | 84.7 | 90.4 | 86.3 | 85.0 | 84.8 | 84.6 | 88.9 | 83.7 | 83.3 | 85.7
D$\rightarrow$C | 30.8 | 32.1 | 33.5 | 33.8 | 35.6 | 76.0 | 75.5 | 81.2 | 77.9 | 71.5 | 76.5 | 74.9 | 73.8 | 76.0 | 70.8
D$\rightarrow$W | 78.1 | 79.7 | 82.5 | 81.7 | 86.2 | 95.9 | 95.7 | 96.1 | 95.7 | 96.9 | 95.5 | 94.3 | 95.8 | 94.8 | 96.9
W$\rightarrow$A | 36.3 | 37.6 | 40.9 | 38.7 | 44.0 | 88.9 | 92.0 | 90.6 | 89.8 | 92.2 | 90.4 | 92.8 | 90.2 | 89.4 | 92.5
W$\rightarrow$C | 32.6 | 32.3 | 33.1 | 34.5 | 35.4 | 87.4 | 85.6 | 85.2 | 81.0 | 85.6 | 85.6 | 85.8 | 84.6 | 82.3 | 85.0
W$\rightarrow$D | 73.3 | 68.5 | 74.6 | 76.1 | 78.8 | 92.9 | 92.8 | 97.2 | 93.9 | 96.7 | 88.9 | 91.1 | 93.4 | 90.8 | 93.5
Averaged | 46.1 | 42.9 | 46.0 | 46.1 | 49.7 | 85.8 | 86.7 | 86.8 | 82.9 | 88.2 | 85.2 | 86.3 | 85.2 | 82.1 | 87.5

####  IV-B4 Evaluation Metric

Following many previous works [10, 11, 12], we adopt the classification
accuracy of target data as the evaluation metric, which is computed as:

$\mathrm{Accuracy}=\frac{|\mathbf{x}:\mathbf{x}\in\mathbf{X}_{t}\cap\tilde{y}=y|}{|\mathbf{x}:\mathbf{x}\in\mathbf{X}_{t}|}$
(32)

where $\mathbf{x}$ is a target sample, $y$ is the truth label of $\mathbf{x}$,
and $\tilde{y}$ is the corresponding pseudo-label.

###  IV-C Unsupervised Domain Adaptation

####  IV-C1 The Experimental Results on Unsupervised Domain Adaptation

a) Results on Office31 Dataset. The classification accuracies of all methods
on this dataset are listed in Table III, where the highest accuracy for each
task is boldfaced. The results of DGA-DA are copied from [12]. It is observed
that CDGS performs much better than all competitors. Specifically, CDGS
achieves 78.9$\%$ average accuracy, which leads the second best method PACET
by 2.3$\%$. DICElp and DGA-DA both explore the geometric structure underlying
data manifold to assign target pseudo-labels by cross-domain label
propagation. However, CDGS further integrates domain-invariant feature
learning, affinity matrix constructing and target labels inferring into one
framework. Therefore, CDGS could make the three parts interact with each other
to yield a superior performance. Besides, CDGS employs a self-learning
strategy to construct a discriminative graph to capture the inherent
similarity of samples as well as explore the label information of source and
target data. In such case, the discriminative graph can transfer source
knowledge to target domain more effectively.

b) Results on Office-Caltech10 Dataset. The results on Office-Caltech10
dataset with SURF features under the full protocol are shown in table IV. In
terms of the average accuracy, CDGS owns a large advantage, which improves
2.7$\%$ over the best competitor PACET. CDGS works the best for 7 out of 12
tasks while PACET only wins two tasks, which verifies the significant
effectiveness of CDGS. Compared with these methods which employ cross-domain
label propagation to infer target labels, i.e., ARTL, MEDA, DGA-DA and DICElp,
the improvement of CDGS is 3.6$\%$, which illustrates the superiority of our
CDGS over the counterparts.

Then, we also compare our CDGS with several competitors under different
splitting protocols with different features. The results over 20 random splits
are illustrated in table V. For SURF features, CDGS performs much better than
other methods in terms of the average accuracy. CDGS achieves 49.7$\%$ average
performance, which owns 3.6$\%$ improvement compared with the best
competitors, MCS and DICElp. Notably, CDGS performs the best on all tasks
except for C$\rightarrow$A. For VGG-FC6,7 features, CDGS outperforms all
comparison methods again. Carefully comparing the results of SURF and VGG-
FC6,7 features, we can find that CDGS can consistently achieve good
performance regardless of the features, which illustrates that CDGS holds
better generalization capacity.

c) Results on MNIST-USPS and ImageNet-VOC2007 Datasets. To verify the
effectiveness of CDGS on digit images, we further conduct experiments on
MNIST-USPS dataset. The comparison results are listed in Table VI. CDGS
achieves the highest average accuracy compared with all competitors. We can
observe that CDGS is much superior to feature adaptation approaches, e.g.,
DGA-DA and DICElp, and owns 5.7$\%$ improvement in terms of the average
accuracy, which demonstrates the superiority of our proposal. The
classification results of all methods on ImageNet-VOC2007 dataset are also
provided in Table VI. CDGS performs much better than other methods. Moreover,
compared with the related methods, i.e., ARTL, MEDA and DICElp, CDGS shows
large improvement up to 8.3$\%$, which confirms the advancement of our CDGS.

TABLE VI: Recognition Accuracies (%) on MNIST-USPS and ImageNet-VOC2007
Datasets. “$-$” Indicates That the Results Are Unavailable

Task | 1-NN | SVM | JDA | DICD | PACET | MCS | DTLC | ARTL | MEDA | DGA-DA | DICElp | CDGS
---|---|---|---|---|---|---|---|---|---|---|---|---
M$\rightarrow$U | 65.9 | 50.0 | 67.3 | 77.8 | 77.0 | 53.4 | 70.7 | 88.5 | 89.5 | 82.3 | 78.3 | 87.1
U$\rightarrow$M | 44.7 | 29.9 | 59.7 | 65.2 | 64.5 | 51.3 | 56.9 | 61.8 | 72.1 | 70.8 | 65.2 | 77.3
Average | 55.3 | 40.0 | 63.5 | 71.5 | 70.7 | 52.4 | 63.8 | 75.1 | 80.8 | 76.5 | 71.8 | 82.2
I$\rightarrow$V | 65.4 | 69.9 | 63.9 | 64.5 | 63.9 | 60.6 | 64.8 | 65.4 | 67.3 | $-$ | 65.8 | 72.3
V$\rightarrow$I | 73.7 | 78.7 | 72.4 | 78.2 | 72.8 | 84.2 | 85.8 | 77.8 | 74.7 | $-$ | 76.0 | 87.5
Average | 69.5 | 74.3 | 68.1 | 71.3 | 68.4 | 72.4 | 75.3 | 71.6 | 71.0 | $-$ | 70.9 | 79.9

TABLE VII: Recognition Accuracies (%) on PIE Dataset

Task | 1-NN | SVM | JDA | DICD | PACET | MCS | DTLC | ARTL | MEDA | DGA-DA | DICElp | CDGS
---|---|---|---|---|---|---|---|---|---|---|---|---
C05$\rightarrow$C07 | 26.1 | 30.9 | 58.8 | 73.0 | 82.2 | 68.9 | 85.1 | 59.7 | 64.2 | 65.3 | 83.9 | 84.8
C05$\rightarrow$C09 | 26.6 | 33.9 | 54.2 | 72.0 | 80.8 | 62.9 | 82.7 | 57.8 | 59.4 | 62.8 | 77.5 | 81.4
C05$\rightarrow$C27 | 30.7 | 41.4 | 84.5 | 92.2 | 94.9 | 87.9 | 97.1 | 86.3 | 84.2 | 83.5 | 95.9 | 98.5
C05$\rightarrow$C29 | 16.7 | 23.8 | 49.8 | 66.9 | 64.5 | 53.6 | 77.2 | 47.5 | 46.5 | 56.1 | 66.0 | 72.9
C07$\rightarrow$C05 | 24.5 | 31.8 | 57.6 | 69.9 | 82.9 | 62.3 | 82.8 | 78.3 | 77.5 | 63.7 | 81.4 | 80.0
C07$\rightarrow$C09 | 46.6 | 41.0 | 62.9 | 65.9 | 73.5 | 52.1 | 83.9 | 69.0 | 71.4 | 61.3 | 74.1 | 86.5
C07$\rightarrow$C27 | 54.1 | 62.2 | 75.8 | 85.3 | 90.1 | 80.8 | 92.1 | 90.0 | 89.2 | 82.4 | 88.4 | 93.6
C07$\rightarrow$C29 | 26.5 | 28.8 | 39.9 | 48.7 | 72.4 | 58.8 | 79.7 | 54.2 | 58.6 | 46.6 | 68.0 | 81.4
C09$\rightarrow$C05 | 21.4 | 32.3 | 51.0 | 69.4 | 79.7 | 58.4 | 80.0 | 72.3 | 73.2 | 56.7 | 78.0 | 85.3
C09$\rightarrow$C07 | 41.0 | 39.7 | 58.0 | 65.4 | 79.3 | 52.5 | 84.4 | 66.5 | 68.8 | 61.3 | 75.9 | 82.6
C09$\rightarrow$C27 | 46.5 | 61.9 | 68.5 | 83.4 | 84.6 | 82.8 | 94.3 | 85.5 | 86.9 | 77.8 | 85.2 | 95.6
C09$\rightarrow$C29 | 26.2 | 37.7 | 40.0 | 61.4 | 70.2 | 59.1 | 79.9 | 60.0 | 65.6 | 44.2 | 71.3 | 81.0
C27$\rightarrow$C05 | 33.0 | 57.7 | 80.6 | 93.1 | 94.0 | 87.7 | 96.7 | 88.7 | 89.6 | 81.8 | 93.3 | 98.8
C27$\rightarrow$C07 | 62.7 | 69.2 | 82.6 | 90.1 | 93.5 | 87.2 | 94.8 | 86.7 | 88.6 | 85.3 | 95.0 | 95.2
C27$\rightarrow$C09 | 73.2 | 69.7 | 87.3 | 89.0 | 91.3 | 83.6 | 95.4 | 87.6 | 88.8 | 91.0 | 92.3 | 93.9
C27$\rightarrow$C29 | 37.2 | 48.7 | 54.7 | 75.6 | 77.0 | 79.2 | 84.4 | 71.0 | 78.3 | 53.8 | 80.5 | 88.6
C29$\rightarrow$C05 | 18.5 | 29.4 | 46.5 | 62.9 | 76.2 | 48.7 | 75.4 | 66.1 | 65.2 | 57.4 | 74.2 | 84.4
C29$\rightarrow$C07 | 24.2 | 33.1 | 42.1 | 57.0 | 69.2 | 58.4 | 77.8 | 57.3 | 58.1 | 53.8 | 69.2 | 75.7
C29$\rightarrow$C09 | 28.3 | 40.6 | 53.3 | 65.9 | 79.2 | 63.4 | 82.4 | 62.9 | 68.1 | 55.3 | 74.6 | 83.5
C29$\rightarrow$C27 | 31.2 | 51.5 | 57.0 | 74.8 | 85.3 | 76.2 | 89.7 | 76.2 | 78.0 | 61.8 | 83.5 | 89.5
Average | 34.8 | 43.3 | 60.3 | 73.1 | 81.0 | 68.2 | 85.8 | 71.2 | 73.0 | 65.1 | 80.4 | 86.7

TABLE VIII: Recognition Accuracies (%) on Office-Home Dataset. Deep Learning
Methods Are Below CDGS

Method | Ar$\rightarrow$Cl | Ar$\rightarrow$Pr | Ar$\rightarrow$Re | Cl$\rightarrow$Ar | Cl$\rightarrow$Pr | Cl$\rightarrow$Re | Pr$\rightarrow$Ar | Pr$\rightarrow$Cl | Pr$\rightarrow$Re | Re$\rightarrow$Ar | Re$\rightarrow$Cl | Re$\rightarrow$Pr | Average
---|---|---|---|---|---|---|---|---|---|---|---|---|---
1-NN | 38.0 | 54.4 | 61.5 | 40.7 | 52.6 | 52.6 | 47.1 | 41.0 | 66.7 | 57.1 | 45.1 | 73.2 | 52.5
SVM | 47.1 | 66.2 | 73.3 | 50.8 | 62.1 | 63.9 | 54.0 | 44.4 | 73.6 | 62.5 | 47.9 | 77.3 | 60.3
JDA | 45.3 | 62.5 | 65.7 | 51.9 | 62.1 | 62.2 | 55.0 | 47.2 | 71.8 | 60.6 | 50.3 | 73.8 | 59.0
DICD | 46.4 | 63.3 | 68.4 | 53.7 | 62.7 | 64.4 | 56.0 | 45.6 | 72.0 | 63.2 | 50.2 | 76.7 | 60.2
PACET | 52.1 | 71.5 | 76.3 | 62.2 | 75.4 | 75.5 | 60.8 | 50.5 | 79.3 | 67.8 | 56.6 | 81.5 | 67.4
MCS | 54.2 | 76.4 | 78.9 | 63.7 | 74.5 | 78.3 | 55.9 | 53.2 | 79.9 | 68.1 | 55.9 | 80.2 | 69.1
DTLC | 51.9 | 74.1 | 75.1 | 61.4 | 70.7 | 73.7 | 63.0 | 51.1 | 76.1 | 66.7 | 54.9 | 79.0 | 66.5
ARTL | 52.9 | 73.8 | 76.7 | 63.0 | 78.5 | 77.1 | 63.6 | 52.4 | 78.7 | 70.1 | 55.2 | 82.6 | 68.7
MEDA | 52.9 | 75.7 | 77.4 | 60.3 | 77.6 | 77.8 | 62.5 | 52.8 | 79.3 | 68.4 | 54.7 | 82.4 | 68.5
DICElp | 48.4 | 70.8 | 72.7 | 52.9 | 65.2 | 65.6 | 59.3 | 49.0 | 76.4 | 65.1 | 52.6 | 79.0 | 63.1
CDGS | 55.6 | 77.0 | 80.1 | 67.1 | 77.2 | 78.4 | 66.5 | 54.4 | 81.0 | 70.5 | 57.9 | 83.3 | 70.8
Wang _et al_. [34] | 55.5 | 73.5 | 78.7 | 60.7 | 74.1 | 73.1 | 59.5 | 55.0 | 80.4 | 72.4 | 60.3 | 84.3 | 68.9
DRCN | 50.6 | 72.4 | 76.8 | 61.9 | 69.5 | 71.3 | 60.4 | 48.6 | 76.8 | 72.9 | 56.1 | 81.4 | 66.6
DSAN | 54.4 | 70.8 | 75.4 | 60.4 | 67.8 | 68.0 | 62.6 | 55.9 | 78.5 | 73.8 | 60.6 | 83.1 | 67.6
Liang _et al._ [50] | 54.1 | 74.2 | 77.7 | 62.9 | 73.6 | 74.6 | 63.4 | 54.9 | 80.4 | 73.1 | 58.2 | 83.6 | 69.2
GSP | 56.8 | 75.5 | 78.9 | 61.3 | 69.4 | 74.9 | 61.3 | 52.6 | 79.9 | 73.3 | 54.2 | 83.2 | 68.4

d) Results on PIE Dataset. Table VII summarizes the classification performance
of CDGS and other methods on PIE dataset. We can observe that CDGS performs
better than all competitors in terms of the average performance. Specifically,
CDGS achieves the highest average classification accuracy, which owns 0.9$\%$
improvement against the best competitor DTLC. Besides, CDGS wins 12 out of 20
tasks while DTLC only performs the best on 7 tasks. It is worthy to note that
compared with ARTL, MEDA, DGA-DA and DICElp, CDGS achieves 6.3$\%$
improvement, which indicates that our CDGS is more conductive for cross-domain
face recognition tasks.

e) Results on Office-Home Dataset. For this large-scale dataset, we use the
Resnet50 model pretrained on ImageNet to extract features. The classification
results are shown in Table VIII. Here, we also report the results of five
recent deep domain adaptation methods, which take the Resnet50 model as the
backbone. It is clearly observed that our CDGS outperforms all traditional and
deep comparison methods in average accuracy. Specifically, CDGS leads the best
traditional competitor MCS by 1.7$\%$. In addition, CDGS is the best method on
5 out of 12 tasks while MCS only wins one task, which verifies the significant
effectiveness of our proposal against the traditional competitors. Compared
with the best deep competitor, CDGS achieves 1.6$\%$ improvement, which
validates the superiority of our proposal when equipped with off-the-self deep
features.

For a complete understanding, we summarize the average accuracy of several
competitors and our CDGS on all benchmark datasets under the full protocol in
Table IX. We discover that CDGS obtains the highest average accuracy, leading
the best competitor MEDA by 5.8$\%$, which validates that our CDGS is capable
of addressing various DA tasks effectively.

TABLE IX: Average Accuracies of Our CDGS and Several Competitors on All Six
Datasets under the Full Protocol

Method | MCS | DTLC | ARTL | DICElp | MEDA | CDGS
---|---|---|---|---|---|---
Average accuracy | 64.7 | 69.5 | 68.2 | 68.9 | 70.0 | 75.8

####  IV-C2 Ablation Study

To understand our method more deeply, we propose three variants of CDGS: _a_)
CDGSsp, Separates domain-invariant feature learning, affinity matrix
constructing and target labels inferring into three independent stages and
constructs the affinity matrix with Predefined similarity metric, i.e., the
gaussian kernel similarity with kernel width 1.0; _b_) CDGSdg, integrates
Domain-invariant feature learning and Graph self-learning into one framework,
i.e., jointing Eq. (6), Eq. (7) and Eq. (14); _c_) CDGSds, jointly performs
Domain-invariant feature learning and graph self-learning with Source domain
discriminative structure preserving, i.e., unifying Eq. (6), Eq. (10) and Eq.
(14). It is worthy noting that compared with CDGSds, our CDGS further
considers the label smoothness constraint during the discriminative graph
self-learning. In Table X, we list the average classification accuracy of CDGS
and the three variants on all datasets under the full protocol. Based on this
table, more detailed analysis about our CDGS is presented as follows.

a) Effectiveness of Graph Self-learning. As we can see, CDGSdg is superior to
CDGSsp on all datasets except for PIE, which verifies the effectiveness of
graph self-learning. Particularly, compared with CDGSsp, CDGSdg achieves
5.9$\%$ improvement on ImageNet-VOC2007 dataset and 3.3$\%$ improvement on
Office-Home dataset respectively, which confirms the superiority of graph
self-learning. By integrating the domain-invariant feature learning and graph
self-learning into one framework, we can capture the inherent similarity
connections among source and target samples more effectively, and thus improve
the classification performance of cross-domain label propagation.

b) Effectiveness of Graph Self-learning with Source Discriminative Structure
Preserving. We can see that CDGSds performs much better than CDGSdg in terms
of average accuracy, which achieves a large improvement of 5.1$\%$. Notably,
on datasets MNIST-USPS and PIE, CDGSds even achieves more than 12.9$\%$
advancement. The above results demonstrate that preserving the source
discriminative structure in graph self-learning process is of vital importance
to improve the quality of affinity matrix, such that the knowledge from source
domain can be transferred to target domain more effectively.

c) Effectiveness of Label Smoothness Constraint for Discriminative Graph Self-
learning. It is observed that our CDGS outperforms CDGSds on 5 out of all 6
datasets and achieves superior performance in terms of average accuracy. This
phenomenon indicates that the introduction of weakly supervised information
contained in target pseudo-labels can promote to yield a discriminative graph
with higher quality, and thus the source knowledge can be propagated to target
domain more adequately.

Figure 2: Parameter Sensitivity Analysis of CDGS with respect to $d$,
$\gamma$, $\beta$ and $T$. TABLE X: The Average Classification Accuracies of
CDGS and the Three Variants on All Datasets under the Full Protocol

Dataset |  CDGSsp |  CDGSdg |  CDGSds |  CDGS
---|---|---|---|---
Office31 |  74.5 |  75.6 |  78.5 |  78.9
Office-Caltech10 |  53.3 |  54.5 |  55.2 |  56.3
MNIST-USPS |  66.0 |  66.7 |  80.4 |  82.2
ImageNet-VOC2007 |  71.7 |  77.6 |  76.6 |  79.9
PIE |  75.2 |  73.9 |  86.8 |  86.7
Office-Home |  65.3 |  68.6 |  69.8 |  70.8
Average |  67.7 |  69.5 |  74.6 |  75.8

TABLE XI: Recognition Accuracies (%) on Office-Caltech10 Dataset with SURF
Features for SDA.

Method | A$\rightarrow$C | A$\rightarrow$D | A$\rightarrow$W | C$\rightarrow$A | C$\rightarrow$D | C$\rightarrow$W | D$\rightarrow$A | D$\rightarrow$C | D$\rightarrow$W | W$\rightarrow$A | W$\rightarrow$C | W$\rightarrow$D | Averaged
---|---|---|---|---|---|---|---|---|---|---|---|---|---
MMDT | 36.4 | 56.7 | 64.6 | 49.4 | 56.5 | 63.8 | 46.9 | 34.1 | 74.1 | 47.7 | 32.2 | 67.0 | 52.5
CDLS | 35.3 | 60.4 | 68.7 | 50.9 | 59.8 | 66.3 | 50.7 | 34.9 | 68.5 | 51.8 | 33.5 | 60.7 | 53.5
ILS | 43.6 | 49.8 | 68.7 | 55.1 | 56.2 | 62.9 | 55.0 | 41.0 | 80.1 | 54.3 | 38.6 | 70.8 | 55.6
TFMKL-S | 43.8 | 62.0 | 70.9 | 54.2 | 60.1 | 68.1 | 53.1 | 38.9 | 79.1 | 54.4 | 36.2 | 69.1 | 57.5
OBTL | 41.5 | 60.2 | 72.4 | 54.8 | 56.2 | 71.1 | 54.4 | 40.3 | 83.2 | 55.0 | 37.4 | 75.0 | 58.9
CDGS | 44.7 | 60.9 | 73.5 | 57.9 | 63.9 | 75.0 | 57.3 | 40.8 | 87.1 | 54.1 | 39.4 | 79.3 | 61.1

TABLE XII: Recognition Accuracies (%) on MNIST-USPS Dataset for SDA.

Task | MMDT | CDLS | ILS | TFMKL-S | CDGS
---|---|---|---|---|---
M$\rightarrow$U | 61.7 | 79.0 | 45.2 | 69.5 | 88.0
U$\rightarrow$M | 47.8 | 68.6 | 50.4 | 53.5 | 79.1
Averaged | 54.8 | 73.8 | 47.8 | 61.5 | 83.5

####  IV-C3 Parameter Sensitivity and Convergence Analysis

Three tunable parameters are involved in our CDGS: $d$, $\gamma$, $\beta$. We
have conducted extensive parameter sensitivity analysis on object, digit and
face datasets by varying one parameter once in a wide range and fixing the
other parameters to the optimal values. We display the results of task
C$\rightarrow$ D (SURF), U$\rightarrow$M, C29$\rightarrow$C05 and
Cl$\rightarrow$Pr in Fig. 2 (a) $\sim$ (c). To verify the effectiveness of our
CDGS, the results of the best competitor for each task are also provided as
the dash lines.

First, we run CDGS as $d$ varies in $d\in[1C,2C,...,10C]$, where $C$ is the
number of classes for the corresponding task. From Fig. 2 (a), we can observe
that our CDGS is robust to different values of $d$. We empirically find that
$d\in[2C,7C]$ is an optimal choice. Then, we investigate the sensitivity of
$\gamma$ by varying it from 0.001 to 10.0. Theoretically, when
$\gamma\rightarrow 0$, the optimization problem is ill-defined, while when
$\gamma\rightarrow\infty$, the domain-invariant feature learning and
discriminative graph self-learning are not performed, thus our CDGS can not
learn robust features for cross-domain label propagation. As we can see from
Fig. 2 (b), determining the optimal value of $\gamma$ is infeasible and a
reasonable one will make CDGS outperform the best competitor generally.
Finally, we vary the value of $\beta$ from 0.001 to 10.0 to evaluate its
influence. Theoretically, too small (large) values of $\beta$ make the label
smoothness constraint (graph self-learning with the projected features)
ineffective, which hinders us to construct a high-quality affinity matrix. A
proper value of $\beta$ helps to capture the intrinsic similarity of samples,
thereby improving the performance of cross-domain label propagation. From Fig.
2 (c), we can discover that $\beta\in[0.01,5.0]$ is an optimal choice.
Moreover, we display the convergence analysis in Fig. 2 (d), where the maximum
iteration is 15. We can observe that our CDGS can quickly converge within
several iterations.

###  IV-D Semi-supervised Domain Adaptation

####  IV-D1 Results on Office-Caltech10 dataset

We follow the standard experimental setup of [35], where 20 samples per class
are randomly selected for amazon domain while 8 for the others as the sources.
Besides, three labeled target samples per category are selected for training
with the rest for testing. For fair comparison, we use the train/test splits
released by [35]. The average accuracies for each task over 20 random splits
are shown in Table XI. We also report the performance of OBTL [37], which to
our knowledge, is the best method on this dataset. We can observe that in
terms of the average accuracy, CDGS obtains 2.2$\%$ improvement over OBTL.
Besides, CDGS works the best for 9 out of all 12 tasks while OBTL just wins
one task, which verifies the significant effectiveness of our semi-supervised
extension. Carefully comparing the results of Table XI and Table V, we find
that when few labeled target samples are available, CDGS obtains 11.4$\%$ gain
in the average classification performance, which highlights the value of our
extension.

####  IV-D2 Results on MNIST-USPS dataset

We follow the protocol of [52]. Specifically, all source samples are utilized
for training, and 2 labeled target samples per category are also selected for
training with the remaining to be recognized. The average classification
accuracies over 5 random splits are reported in Table XII, where some results
are copied from [52]. We can observe that our CDGS is the best method for all
tasks and achieves 83.5 $\%$ averaged accuracy, leading the second best method
CDLS by 9.7$\%$, which confirms the superiority of our semi-supervised
extension.

##  V Conclusion and Future Work

In this paper, a novel domain adaptation approach called CDGS is proposed,
which infers target pseudo-labels by cross-domain label propagation. Different
from existing cross-domain label propagation methods that separate domain-
invariant learning, affinity matrix constructing and target labels inferring
into three independent stages, our CDGS integrates these three parts into one
unified optimization framework, such that they can assist each other to
achieve more effective knowledge transfer. Furthermore, to construct a high-
quality affinity matrix in CDGS, we propose a discriminative graph self-
learning strategy, which can capture the inherent data manifold structure by
adaptively calculating sample similarity in the projected space and exploring
the discriminative information contained in well-labeled source data and
pseudo-labeled target data. An iterative optimization algorithm is designed to
solve the CDGS optimization problem. We further extend our CDGS to the SDA
scenario in a direct but effective way and the corresponding optimization
problem can be solved with the identical optimization algorithm. Extensive
experimental results on six benchmark datasets have verified the significant
superiority of our CDGS against the competitors in both UDA and SDA settings.

## References

  * [1] S. J. Pan and Q. Yang, “A survey on transfer learning,” _IEEE Trans. Knowl. Data Eng._ , vol. 22, no. 10, pp. 1345-1359, Oct. 2010.
  * [2] Y. Chen, S. Song, S. Li, L. Yang, and C. Wu, “Domain space transfer extreme learning machine for domain adaptation,” _IEEE Trans. Cybern._ , vol. 49, no. 5, pp. 1909-1922, May 2019.
  * [3] Y. Liu, W. Tu, B. Du, L. Zhang, and D. Tao, “Homologous component analysis for domain adaptation,” _IEEE Trans. Image Process._ , vol. 29, pp. 1074-1089, Jul. 2019.
  * [4] J. Li, K. Lu, Z. H, L. Zhu, and H. Shen, “Transfer independently together: A generalized framework for domain adaptation,” _IEEE Trans. Cybern._ , vol. 49, no. 6, pp. 2144-2155, Jun. 2019.
  * [5] Q. Zhang, J. Zhang, W. Liu, and D Tao, “Category anchor-guided unsupervised domain adaptation for semantic segmentation,” in _Proc. Adv. Neural Inf. Process. Syst._ , 2019, pp. 433-443.
  * [6] Q. Wang, J. Gao, and X. Li, “Weakly supervised adversarial domain adaptation for semantic segmentation in urban scenes,” _IEEE Trans. Image Process._ , vol. 28, no. 9, pp. 4376-4386, Sept. 2019.
  * [7] S. Yang, K. Yu, F. Cao, H. Wang, and X. Wu, “Dual-representation-based autoencoder for domain adaptation,” _IEEE Trans. Cybern._ , early acess, doi:10.1109/TCYB.2020.3040763.
  * [8] D. Wang, C. Lu, J. Wu, H. Liu, W. Zhang, F. Zhuang and H. Zhang, “Softly associative transfer learning for cross-domain classification,” _IEEE Trans. Cybern._ , vol. 50, no. 11, pp. 4709-4721, Nov. 2020.
  * [9] S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang, “Domain adaptation via transfer component analysis,” _IEEE Trans. Neural Netw._ , vol. 22, no. 2, pp. 199-210, Feb. 2011.
  * [10] M. Long, J. Wang, G. Ding, J. Sun, and P. S. Yu, “Transfer feature learning with joint distribution adaptation,” in _Proc. IEEE Conf. Comput. Vis. Pattern Recognit._ , 2013, pp. 2200-2207.
  * [11] S. Li, S. Song, G. Huang, and Z. Ding, “Domain invariant and class discriminative feature learning for visual domain adaptation,” _IEEE Trans. Image Process._ , vol. 27, no. 9, pp. 4260-4273, Sept. 2018.
  * [12] J. Liang, R. He, and T. Tan, “Aggregating randomized clustering-promoting invariant projections for domain adaptation,” _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 41, no. 5, pp. 1027-1042, May 2019.
  * [13] S. Li , C. .Liu, L. Su, B. Xie, Z. Ding, C. L. P. Chen, and D. Wu, “Discriminative transfer feature and label consistency for cross-Domain image classification,” _IEEE Trans. Neural Netw. Learn. Syst._ , Jan. 07, 2020, doi: 10.1109/TNNLS.2019.2958152.
  * [14] J. Liang, R. He, Z. Sun, and T. Tan, “Exploring uncertainty in pseudo-label guided unsupervised domain adaptation,” _Pattern Recognit._ , vol. 96, Dec. 2019, Art. no. 106996.
  * [15] A. Gretton, K. M. Borgwardt, M. Rasch, B. Scholkopf, and A. J. Smola, “A kernel method for the two-sample-problem,” in _Proc. Adv. in Neural Inf. Process. Syst._ , 2007, pp. 513-520.
  * [16] L. Luo, L. Chen, S. Hu, Y. Lu, and X. Wang, “Discriminative and geometry-aware unsupervised domain adaptation,” _IEEE Trans. Cybern._ , vol. 50, no. 9, pp. 3914-3927, Sept. 2020.
  * [17] J. Li, M. Jing, K. Lu, L. Zhu and H. Shen, “Locality preserving joint transfer for domain adaptation,” _IEEE Trans. Image Process._ , vol. 28, no. 12, pp. 6103-6115, Dec. 2019.
  * [18] Z. Ding, S. Li, M. S, and Y. Fu, “Graph adaptive knowledge transfer for unsupervised domain adaptation,” in _Proc. Eur. Conf. Comput. Vis._ , 2018, pp. 37-52.
  * [19] X. Zhu and Z. Ghahramani, “Learning from labeled and unlabeled data with label propagation,” Carnegie Mellon Univ., Pittsburgh, PA, USA, Tech. Rep. CMU-CALD-02-107, 2002.
  * [20] F. Nie, X. Wang, and H. Huang, “Clustering and projected clustering with adaptive neighbors,” in _Proc. 20th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining_ , 2014, pp. 977-986.
  * [21] F. Nie, S. Shi and X. Li, “Semi-supervised learning with auto-weighting feature and adaptive graph,” _IEEE Trans. Knowl. Data Eng._ , vol. 32, no. 6, pp. 1167-1178, Jun. 2020.
  * [22] C.-G. Li, Z. Lin, H. Zhang, and J. Guo, “Learning semi-supervised representation towards a unified optimization framework for semi-supervised learning,” in _Proc. IEEE Int. Conf. Comput. Vis._ , 2015, pp. 2767-2775.
  * [23] M. Sugiyama, M. Krauledat, and K.-R. Muller, “Covariate shift adaptation by importance weighted cross validation,” _J. Mach. Learn. Res._ , vol. 8, pp. 985-1005, May 2007.
  * [24] M. Long, J. Wang, G. Ding, J. Sun, and P. S. Yu, “Transfer joint matching for unsupervised domain adaptation,” in _Proc. IEEE Conf. Comput. Vis. Pattern Recognit._ , 2014, pp. 1410-1417.
  * [25] M. Long, J. Wang, G. Ding, S. J. Pan, and P. S. Yu, “Adaptation regularization: a general framework for transfer learning,” _IEEE Trans. Knowl. Data Eng._ , vol. 26, no. 5, pp. 1076-1089, May 2014.
  * [26] J. Wang, W. Feng, Y. Chen, H. Yu, M. Huang, and P. S. Yu, “Visual domain adaptation with manifold embedded distribution alignment,” in _Proc. ACM Multimedia Conf. Multimedia Conf._ , 2018, pp. 402-410.
  * [27] B. Gong, Y. Shi, F. Sha, and K. Grauman, “Geodesic flow kernel for unsupervised domain adaptation,” in _Proc. IEEE Conf. Comput. Vis. Pattern Recognit._ , 2012, pp. 2066-2073.
  * [28] C.-A. Hou, Y.-H. H. Tsai, Y.-R. Yeh, and Y.-C. F. Wang, “Unsupervised domain adaptation with label and structural consistency,” _IEEE Trans. Image Process._ , vol. 25, no. 12, pp. 5552-5562, Dec. 2016.
  * [29] M. Long, Y. Cao, Z. Cao, J. Wang, and M. I. Jordan, “Transferable representation learning with deep adaptation networks,” _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 41, no. 12, pp. 3071-3085, Dec. 2019.
  * [30] M. Long, J. Wang, and M. I. Jordan, “Deep transfer learning with joint adaptation networks,” in _Proc. Int. Conf. Mach. Learn._ , 2017, pp. 2208-2217.
  * [31] Y. Zhu, F. Zhuang, J. Wang, G. Ke, J.Chen, J. Bian, H. Xiong and Q. He, “Deep subdomain adaptation network for image classification,” _IEEE Trans. Neural Netw. Learn. Syst._ , May 13, 2020, doi: 10.1109/TNNLS.2020.2988928.
  * [32] Y. Ganin _et al_., “Domain-adversarial training of neural networks,” _J. Mach. Learn. Res._ , vol. 17, no. 1, pp. 2096-3030, 2016.
  * [33] Z. Pei, Z. Cao, M. Long, and J. Wang, “Multi-adversarial domain adaptation,” in _Proc. Amer. Assoc. Artif. Intell. Conf._ , 2018, pp. 3934-3941.
  * [34] S. Wang and L. Zhang. (2020). “Self-adaptive re-weighted adversarial domain adaptation.” [Online]. Available: https://arxiv.org/abs/2006.00223
  * [35] J. Hoffman, E. Rodner, J. Donahue, B. Kulis, and K. Saenko, “Asymmetric and category invariant feature transformations for domain adaptation,” _Int. J. Comput. Vis._ , vol. 41, nos. 1-2, pp. 28-41, 2014.
  * [36] Y.-H. H. Tsai, Y.-R. Yeh, and Y.-C. F. Wang, “Learning cross-domain landmarks for heterogeneous domain adaptation,” in _Proc. IEEE Conf. Comput. Vis. Pattern Recognit._ , 2016, pp. 5081-5090.
  * [37] A. Karbalayghareh, X. Qian, and E. R. Dougherty, “Optimal Bayesian transfer learning,” _IEEE Trans. Signal Process._ , vol. 66, no. 14, pp. 3724-3739, Jul. 2018.
  * [38] Z. Ding, S. M. Nasrabadi, and Y. Fu, “Semi-supervised deep domain adaptation via coupled neural networks,” _IEEE Trans. Image Process._ , vol. 27, no. 11, pp. 5214-5224, Nov. 2018.
  * [39] X. Zhu, Z. Ghahramani, and J. D. Lafferty, “Semi-supervised learning using Gaussian fields and harmonic functions,” in _Proc. Int. Conf. Mach. Learn._ , 2003, pp. 912-919.
  * [40] D. Zhou, O. Bousquet, T. N. Lal, J. Weston, and B. Scholkopf, “Learning with local and global consistency,” in _Proc. Adv. Neural Inf. Process. Syst._ , 2004, pp. 321-328.
  * [41] B. Scolkopf, R. Herbrich, and A. J. Smola, “A generalized representer theorem,” in _Proc. 14th Annu. Conf. COLT_ , Amsterdam, The Netherlands, 2001.
  * [42] K. Saenko, B. Kulis, M. Fritz, and T. Darrell, “Adapting visual category models to new domains,” in _Proc. Eur. Conf. Comput. Vis._ , 2010, pp. 213-226.
  * [43] B. Sun, J. Feng, and K. Saenko, “Return of frustratingly easy domain adaptation,” in _Proc. Amer. Assoc. Artif. Intell. Conf._ , 2016, pp. 2058-2065.
  * [44] S. Herath, M. Harandi, and F. Porikli, “Learning an invariant hilbert space for domain adaptation,” in _Proc. IEEE Conf. Comput. Vis. Pattern Recognit._ , 2017, pp. 3845-3854.
  * [45] T. Sim, S. Baker, and M. Bsat, “The CMU pose, illumination, and expression database,” _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
  * [46] H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan, “Deep hashing network for unsupervised domain adaptation,” in _Proc. IEEE Conf. Comput. Vis. Pattern Recognit._ , Jun. 2017, pp. 5018-5027.
  * [47] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proc. IEEE Conf. Comput. Vis. Pattern Recognit._ , Jun. 2016, pp. 770-778.
  * [48] J. Liang, R. He, Z. Sun and T. Tan, “Distant Supervised Centroid Shift: A Simple and Efficient Approach to Visual Domain Adaptation,” in _Proc. IEEE Conf. Comput. Vis. Pattern Recognit._ , 2019, pp. 2975-2984.
  * [49] S. Li, C. Liu, Q. Lin, Q. Wen, L. Su, G. Huang, and Z. Ding, “Deep residual correction network for partial domain adaptation,” _IEEE Trans. Pattern Anal. Mach. Intell._ , 2020.
  * [50] J. Liang, Y. Wang, D.g Hu, R. He, and J. Feng. (2020). “A balanced and uncertainty-aware approach for partial domain adaptation.” [Online]. Available: https://arxiv.org/abs/2003.02541
  * [51] H. Xia and Z. Ding, “Structure preserving generative cross-Domain learning,” in _Proc. IEEE Conf. Comput. Vis. Pattern Recognit._ , 2020, pp. 4364-4373.
  * [52] W. Wang, H. Wang, Z. Zhang, C. Zhang, and Y. Gao, “Semi-supervised domain adaptation via Fredholm integral based kernel methods,” _Pattern Recognit._ , vol. 85, pp. 185-197, Jan. 2019.